Confidence Shattered in NT 8

[MrMuppet]

That's a painful experience. I sympathize.

The only way to help avoid it in the future is to cross-check raw data from your data source, with the results from your back testing.

In regard to any platform results, trust but verify before going forward.

It has nothing to do with the data. You can input the exact same datasets and you'd still get two different results. The issue is known and not even NT knows why that is.

All retail platforms have the same problem. They are set up slick and clean until the peons chime in with feature requests: "can I have a purple Stochastic?" "Can I have trendlines on the indicator XYZ" "Can we have OCO orders and trailing stops?" You get the deal.

After years of adding one module after another, the entire code is just fucked up and finding a bug becomes a nightmare.

 

[shatteredx]

Hi Guys,

I know there are real guru traders here. And I think I can take advantage of your wisdom and experience by listening to your advice here.

The issue I encountered here is that, I spent almost a year to familiarize myself with the NinjaTrader 8 platform. Writing indicators, strategies, optimization fitness, and in the last even a genetic optimizer. During this process, I find several bugs of their program, but I am not a professional programmer. Sometime I have to spend like half a month to explain to them where is going wrong. And some bugs, they have not yet correct it until now. Suspicion is growing, certainly.

Until yesterday, I did a test informed by another elite trader here: install NT8 on two computers, use the same strategy, same data, same parameter, same time interval, in a word, set everything the same except that it is running on two different computers. And guest what? The results are totally different !

My confidence of the reliability of the platform is shattered!

And my fellow elite trader, do me a favor, install NT 8 on two computers and run a test with every thing set up the same, will you get the same results?

Kind of low these days, I think I will convert to Python. Any suggestions ?

Have you tried deleting db\cache on both installs? Both NT 7 and 8 have problems with the cache corrupting itself on a regular basis.

Might also want to check that you're using the same session (trading hours) on both backtests. I know that NT8's holiday calendar was messed up last year, not sure if they ever fixed it on their server.

With that said, have you examined both strategy results in detail and looked at the actual trades? It should only take a few minutes to compare the trades of both side-by-side to see where they diverge.

I still use NT7 for generating intraday strategies, but I think learning Python and moving to longer-term strategies is where I'm headed...

 

[Simples]

NT8 is "fine", if you make sure to recalculate previous bar (as it is then finally complete), use your own datalayer, replace exception and logging with your own, skip recalculating same bar more than once per ie. 100 ms, plus a bunch of other workarounds as you find the flaws. But, at least the platform allow you to do your own, which is a plus. I find the drawing system in NT8 to be somewhat improved over NT7 performance-wise. Though, the broken threading usage may require you to invoke new threads or use sleep() to keep somewhat in sync, as NT devs certainly use threads without securing them whatsoever. Of course, band aid cannot guarantee to remove all race conditions, though mostly only affect Log(), draw objects and z-order.

Recalculating previous bar paint the same as when doing full recalculation.

Agree, if you know your needs and are capable, best to create your own platform, though one need something to prototype on and creating your own for that may become a timesink and waste.

I've yet to encounter something that cannot be worked around creatively.

 

[Warren]

What was different? Was the difference in real trading or sim?

In the backtest, same data, same strategy, same parameter values, different results (max net profit). Try it your self, if interested.

 

[Warren]

Totally different how? Care to elaborate?

I was very happy with NT7, but migrating to NT8 which was supposed to be a far superior platform have been a disappointment. I don't run any automated strategies except for order placement, but have coded quite a lot of indicators/tools which I use.

If not for that investment, I would probably have gone back to NT7 or some other platform.

But NT8 is working fine for me now for what I'm doing. And having had the platform for 10 years now - it's familiar and not something I would want to replace unless I experience major flaws/issues with it.

see my reply to Morse, you can try it yourself.

 

[Warren]

Hi dude, I'd suggest leaving those pseudo languages all together. Easy language, Pine Script, learning all this BS takes as much time and effort as it takes to learn a real language.

Take the Anaconda distribution of Python, use Jupyter Notebook as your environment and Github to get sample codes. There are a lot of financial libraries written for Python. It won't take long till you can perform the simple tasks.


Really...NT 8 is spaghetti code. Leave that trash behind.

thanks man, hard to believe it they had such a big flaw in the first place.

 

[Overnight]

thanks man, hard to believe it they had such a big flaw in the first place.

Ninja's sim engine leaves much to be desired. Doesn't surprise me you are having issues. NT7 is no better.

You can run the same strat test on the same PC three times in a row, with NO changes to anything, and get different results. It is like an above poster mentioned, it is spaghetti code.

 

[Warren]

NT8 is "fine", if you make sure to recalculate previous bar (as it is then finally complete), use your own datalayer, replace exception and logging with your own, skip recalculating same bar more than once per ie. 100 ms, plus a bunch of other workarounds as you find the flaws. But, at least the platform allow you to do your own, which is a plus. I find the drawing system in NT8 to be somewhat improved over NT7 performance-wise. Though, the broken threading usage may require you to invoke new threads or use sleep() to keep somewhat in sync, as NT devs certainly use threads without securing them whatsoever. Of course, band aid cannot guarantee to remove all race conditions, though mostly only affect Log(), draw objects and z-order.

Recalculating previous bar paint the same as when doing full recalculation.

Agree, if you know your needs and are capable, best to create your own platform, though one need something to prototype on and creating your own for that may become a timesink and waste.

I've yet to encounter something that cannot be worked around creatively.

Then, I guess you are a very experienced C# user. What you have mentioned here is beyond my knowledge. Not sure whether I am dipping that deep into C#. With the same amount of time and energy, I guess Python will be a better choice? After all, I can use Python for other purpose(I am in social science). Plus, with enough eye balls, program bugs are easier to spot. Kind of frustrated when you spot a bug, while the program is wrapped up in the dlls.

For example, everytime you run a back test or a optimization on NT8, there is a number of iterations shown on the strategy analyzer panel. This number is pre calculated and must be set up before a optimizer initiated.

But, guess what? This is a fake number! The only purpose of this is to show the user how many iterations you gonna run. You can run any number of iterations you like beyond this limit. And for that, their default genetic optimizer have to use a termination of a fixed number of iterations which greatly undermine the power of a genetic algorithm!

You never know how they coded this. And this give me the impression that they only wanna impress the amature users with seemingly powerful functions. While they really do not care whether it is really powerful.

 

[Simples]

Then, I guess you are a very experienced C# user. What you have mentioned here is beyond my knowledge. Not sure whether I am dipping that deep into C#. With the same amount of time and energy, I guess Python will be a better choice? After all, I can use Python for other purpose(I am in social science). Plus, with enough eye balls, program bugs are easier to spot. Kind of frustrated when you spot a bug, while the program is wrapped up in the dlls.

For example, everytime you run a back test or a optimization on NT8, there is a number of iterations shown on the strategy analyzer panel. This number is pre calculated and must be set up before a optimizer initiated.

But, guess what? This is a fake number! The only purpose of this is to show the user how many iterations you gonna run. You can run any number of iterations you like beyond this limit. And for that, their default genetic optimizer have to use a termination of a fixed number of iterations which greatly undermine the power of a genetic algorithm!

You never know how they coded this. And this give me the impression that they only wanna impress the amature users with seemingly powerful functions. While they really do not care whether it is really powerful.

Many years of coding practice. Not so much experience with C#, but the libraries are very standard and offer standard features, the rest is online search and intuitions. NT is not so closed down as other trading platforms, so you often can guess about the underlying design and weaknesses. Haven't met any platform or language that didn't require some workaround or was cripped (iAnything). Actually, golang comes very far and near being ideal, though fills more of a niche, is simple and straightforward, but not necessarily easy for beginners.

True, for genetic, ML, etc., you could find something Free software or make your own, however, comes with a whole bunch of other issues. A valid approach is to try prototyping stuff quickly, and search for better suitable solutions meanwhile. The hard part is not really the tech, but actually trade consistently and profitably. If one can do that, the tech choice becomes that much more obvious!

Making a trading UI yourself, or using someone else's, would be really alot of tricky work, though could be rewarding as a project of its own.

 

[Laissez Faire]

I remember when I first migrated to NT8 they had an issue with a lagging cross-hair. Who the hell has a lagging cross hair on their new state-of-the art platform?

Not only that, but the problem had allegedly persisted for an entire year.

I did eventually get fixed, but come on...

Another annoyance is that you can't chart daily/weekly/monthly bar using the RTH session template.

Both of these features worked just fine with NT7.