Backtesting vs Simulation vs Actual Trading

[Avalanche]

Quote from StocksSniper:

Here is the results of the actual trading:

Those are great results. Would you mind sharing some additional stats such as:

Share sizes/Capital per position, do you scale in or enter at only one price for each trade, max capital in use, max consecutive winners/losers, and what the avg hold time is.

Also can you share with the forum atleast whether the strategy is trend, counter-trend, mean regression, etc. Nothing specific but I think alot of people have tried automated trading and found it much more difficult than one would think.

again, good stuff, thanks for sharing. Always interesting to see how real results compare to simulation or backtesting.

 

[Lawrence Chan]

1- Backtesting can be far from reality (even when we didn't optimize much).

Yep. I receive emails everyday asking questions whether their systems performance make any sense - my standard reply is that have they tried testing that using worst case scenario, ruling out fill at extreme price (high and low of a price bar), etc.

If worst case anlysis is done and the system is still profitable, it is then very likely the system will still perform fine in real-time.

2- Even real time simulation can be far from reality (partial fill, no fill, running out of capital a the worst moment, second guessing the system, busted trades, IB being down...). For the record I have nothing against IB. I love them actually.

Not all real-time simulation are built equal - aside from Ninja Trader and NeoTicker, I have not seen any other platform that can limit your fill if and only if your order price and size is absolutely fillable. i.e. sell price is showing at the bid with enough bid size to cover your order size.

3- Nothing gives more confidence, learning than real trading. And actually it grows your account and allow you to buy stuff.

When real trading is involved and your system is not ready, sometimes it is too late and disaster could have hit. So taking care of a system by properly testing it against historical data is very important. It could be very tedious but well worth your time.

 

[vital-analitix]

Backtesting and mechanical trading is like trying to drive a formula 1 car by remote control.

Actually let me rephrase that, it is like driving on a dark night, without any lights, on a hilly & winding road, with big canyons on the side at full speed looking in the rearview mirror.

Good luck with that fallacy.

On the other hand: if you know the limitations of simulation and know how to set it up properly and do not use limit orders and are using a liquid market in larger timeframes then that may be useful.

vital analitics

 

[man]

Quote from nononsense:

...

Don't expect to find in any place:
(1) Actual Trading strategies (that work);
(2) Backtesting methods useful for (1);
(3) Simulation methods useful for the above.

...

dearest nono,

which category of your categories does your category of categories violate in the end? and does it so categorically?

i sense some epidemes' oil ...

 

[StocksSniper]

Quote from Avalanche:



Would you mind sharing some additional stats such as:

Share sizes/Capital per position, do you scale in or enter at only one price for each trade, max capital in use, max consecutive winners/losers, and what the avg hold time is.

Also can you share with the forum atleast whether the strategy is trend, counter-trend, mean regression, etc.

I tried not to include too many stats becasue my goal was just to compare Backtesting, Simulation, RealTrading.

But anyway, since you're asking here it is:

Capital per trade:
Average $9,857
Median $9,953
Max $39,529
Min $1,200 (partial fill)

Capital in the market:
Daily Average $213,413
Median $172,047
Max $755,835 -that is with leverage from IB. I don't have that much money )
Min $0

Holdtime:
Average 6.78 Days
Median 5 Days
Max 46 Days
Min 0 Days

Max consecutive winners/losers: N/A. I have to go back thru all my trades.

Scaling in:
I do and I don't scale in. Let me explain:
For every signal I get in at once (for that price). But if I have a new signal to buy for a stock that I already own, I buy it again. At once. Basically, I get in at once for every signal. But if I get a second signal, I get in again, a third I get in again...

Strategy:
Strategy is both trend following/counter-trend. Without going too much into details I have two criteria that I look for in my watchlist and whenever a stock hit one of these criteria I get a signa (Some criteria are for trend-following, others are for counter-trend). I had to mix both in the same system in order to have many trades every day.
I just can't sit and not trades. If I don't have a signal, I might get tempted to trade on emotions (baaaad). My way of dealing with it is haveing 2 approaches combined in one system.

Good trading to you,

 

[maxpi]

Sniper, thanks for the thread.

One real problem in backtesting is making the simulation as real as possible as attended to in this thread by Lawrence Chan. I found a bug in my backtesting recently wherein I was using the low of the day of the entry for some system filtering. That was a case of using data that could not be known at the time of the trade. It got past me somehow, something always does. I wonder how much of the disparity of backtested results and reatime operation is due to badly done back tests?