backtesting vs live trading

[smallworld]

I have a number of systems that I trade. The transition between backtesting and live trading is always a bit haphazard. Most of the time is spent of backtesting and development, but not a lot is spending in migrating from backtesting to live trading, and extracting data to feedback the development process.

by and large, after the system testing is completed, I would write up explorations and scans with alert function to give me signals to tell me which stock, how many and at what price and score the position with the view that position with high score will be entered first.

The problem comes when there are a number of competing signals, and some of the signals would be a little late, ie., slippage. at this point I would usually use my discretion to select which stock to enter, balancing between choosing stock with high position score versus stock with low slippage. I have use stop orders with mixed success, sometime the system makes it impractical to have stop orders. this is particularly bad on days with a lot of signals, and on those days, from my experience, are the best days to get it right.

The end result is that my portfolio is no longer the same as in backtesting. Since I am not a purist, that doesnt worry me that much really. Except that I would like a way to feedback the real life trading results into tweaking/refining the system.

I would be interested in hearing others sharing their experience in these kind of situation, and how you have structured to migrate from backtesting to life trading, and how you feedback your results in refining your systems

Cheers

 

[stephencrowley]

What timeframe are you working on? I backtest with 1-second intervals with full depth of book data. My system gives immediate recommendations and I only trade high volume issues and I do realtime trading decisions (e.g. i dont optimize parameters ahead of time) and I always simulate limit orders by walking the full depth of book, and I can tweak the latency parameter to say "simulate a limit order that walks the book and executes within 300 ms" so that gives me a pretty accurate price and I can play with varying latency simulations, but I've found that I'm not very sensitive to latency (fast broker api) so not a big issue for me.

 

[giggollo]

sounds like ur using amibroker...either way, slippage is a function of volume and other things, have u factored average volume into your postionscore or yesterday's volume, u can make your positionscore variable as simple or as complex as you want, and i think the only way to see whether your new way of selecting which stocks to enter is better than the one you were using before is by testing one way for some period of time, and then testing another way and checking if there's an improvement in performance.

Backtesting results can differ (sometimes significantly) from realtime results and so u need to discard or at least be more suspicious of systems that may look good "on paper" (ie backtesting) but have a low average profit. Errors in data used for backtesting can also complicate things, giving "false signals" that wouldnt be triggered live or missing trades that would be triggered live. Although it depends on your data source, the higher your data frequency, the more prone that data is to errors i find

 

[maxpi]

Fills are not realistic on even a simulaton account. Trading some real $ but small amounts is probably the best way to get in touch with reality and then scale up slowly to find the next level of "gotcha's". I had one system, looked good on paper, very good in fact, looked good on the simulations, was a big loser due to liquidity issues. The big gains on paper were coming from illiquid issues and the big losses were coming at times of great liquidity for some other issues. I could catch that problem by scaling up slowly, gains were great with almost no size and they leveled off as I scaled up.

You have to use caution in fielding a system, it is very much like an engineering project, small incremental steps and enough of them to achieve final rollout.