Careful with Historical Analysis v. Real Performance

[schizo]

I created an automated strategy on Ninjatrader and ran a 6 month historical analysis:

You call 6 months "historical"?

BTW you need to take volatility into account. Six months ago, we hardly had any volatility. It was so dull, I cried myself to sleep every night. Now, the volatility is nearly 10x. You can't trade today's market with the strategy from 6 months ago and expect the same outcome.

 

[El OchoCinco]

6 months covered a nice range of quiet days and volatile days for intraday futures trading. I could go back 3 years but will not improve the results. I been tweaking it on and off for past year and a half in chunks when I have time so it covers a long period when all the testing taking into account.

I think of it as more of a hobby right now so I am ok if I keep having to tweak and then leave it alone for a while.

Also the point of the post was to point out how people got suckered in with historical backtests that look amazing...

 

[El OchoCinco]

I decided to trade it unautomated with me LIVE on the SIM to test something and I believe my problem is my approach cannot be automated. Still discretion in reading the price action and S/R areas when I made the trades.

Me trading it live produced this (1 contract on NQ on almost all trades so the profits look small to some of you but I always test on 1 contract and then test scale):

Trades #16 and #17 were good signals so I would take them every time...even in hindsight. But trusting the approach allowed me to take Trade #18 and #19 (which was one trade actually so total $600 gain).

Trade #20 was a good entry but whipped me out on my stop and then I hit REVERSE by accident so leading to loss in Trade #21 until I got back in short at #22. This is the human error BS I was hoping to avoid but we all make mistakes.

Trade #22 - #24 were solid discretionary entries.

My point? I tried but somethings the eye sees cannot be automated. I am going to go back live with this method that I have been trying to automate off and on for a long time but do it live (not SIM, I just did it today as I thought I would be interrupted a lot) and let the profit accumulate the old fashioned way...manually.

Good luck to those who can fully program and automate, for some things I guess human eyes is the best machine.

 

[Gasparov]

Ninja is one of the most robust platforms out there. You have full power of C# at your disposal. And NT8 is multi threaded, so performance is amazing as well.

Backtest vs real have nothing to do with the platform.

they do when NT uses different historical data than other platforms may have available.

 

[RedDuke]

they do when NT uses different historical data than other platforms may have available.

Ninja does not care where you get historical data. I did extensive comparison between kinetick, iqfeed, esignal and IB. Kinetick and IQfeed data was identical. Esignal very close. IB different but not too far, but it is expected with IB, since they aggregate ticks and not reporting them all. IB should be used to execution and not historical data.

You can purchase your own data from CME, format it for Ninja and import there as well.

 

[Gasparov]

Ninja does not care where you get historical data. I did extensive comparison between kinetick, iqfeed, esignal and IB. Kinetick and IQfeed data was identical. Esignal very close. IB different but not too far, but it is expected with IB, since they aggregate ticks and not reporting them all. IB should be used to execution and not historical data.

You can purchase your own data from CME, format it for Ninja and import there as well.

Oh that's different then. I didn't purchase any data, just used what NT provided and knew there were gaps in the data sometimes in older years

 

[RedDuke]

Oh that's different then. I didn't purchase any data, just used what NT provided and knew there were gaps in the data sometimes in older years

NT gives you sample data just to learn platform. It is not reliable for backtest. You need to purchase your own. Kinetick is really cheap for non professional and accurate to a tick.

 

[El OchoCinco]

Have you tried running it on any other platforms?

Just to come back to this, it took me numerous iterations to put true slippage into the system to show accurate fills and when I finally finished that research it finally showed up red.

Again I am not posting this to say I was naive or stupid, but I knew it was wrong and many newbies get enamoured with sims but the devil is always in the details...

 

[Laissez Faire]

Where are you getting slippage? Are you using market orders both for entries and exits?