What is the best platform for backtesting?

[Shay]

...I find very obvious errors that the backtests made

like lindq said above
plus
if you have errors in your code, the problem will be in every tool

 

[userque]

Everyone keeps saying "data." But, he said that there were obvious errors. Sounds to me that the backtesting system is not back-trading as he expects it to with the data given; regardless as to whether that data is bad or good.

Given the same data, good or bad, I would expect a backtesting system to come up with the same results that I do by manually testing some of the data. I expect different backtesting platforms to have the same results if using the same data--good or bad.

And, don't these platforms provide their own data? If so, then it doesn't matter if it's the platform's data, or the software that's the problem. He's looking for something more reliable regardless as to which is bad.

 

[d08]

You can circumvent bad data with programming if the errors are consistent and easily detectable. Now if you have wrong values that aren't wrong enough to be detected, you're out of luck.

 

[M.ST.]

Does anybody know what the most accurate form of backtesting is?

Ninjatrader. End of story.

Pretty good joke.

 

[userque]

You can circumvent bad data with programming if the errors are consistent and easily detectable. Now if you have wrong values that aren't wrong enough to be detected, you're out of luck.

Yes, programming is going to be involved, or course. But you also need data from other sources. For example, how are you going to fix this with just programming:

Open=20, High=23, Low=21, Close=22

It's an obvious error--as you required. But how would you fix it? Without additional data, you can only GUESS at what it should be, maybe it'll be "fixed;" maybe not.

Is the Open wrong, or the Low wrong? How do you know?

 

[Surgo]

Yes, programming is going to be involved, or course. But you also need data from other sources. For example, how are you going to fix this with just programming:

Open=20, High=23, Low=21, Close=22

Throw it out. Don't run any trades that would go through the bar.

 

[userque]

Throw it out. Don't run any trades that would go through the bar.

That works. (But in some systems, one bad bar will 'corrupt' several surrounding bars). You could also guess by analyzing the surrounding data.

I guess I just wanted it to be clear that none of these work-arounds actually 'fix' the problem of initially obtaining bad data.

 

[d08]

Yes, programming is going to be involved, or course. But you also need data from other sources. For example, how are you going to fix this with just programming:

Open=20, High=23, Low=21, Close=22

It's an obvious error--as you required. But how would you fix it? Without additional data, you can only GUESS at what it should be, maybe it'll be "fixed;" maybe not.

Is the Open wrong, or the Low wrong? How do you know?

That's why I said "wrong enough" and "easily detectable". I mean open at $20, low at $1.

 

[syswizard]

I have backtested strategies on both Tradestation and Multicharts. While the backtests produce results, if I comb through the results I find very obvious errors that the backtests made.

Did you submit a thread about your findings of error on the MC forums ?
They have superb tech support. I would be very surprised that no one else discovered the same errors. Exactly what "errors" did you find ?

 

[M.ST.]

Did you submit a thread about your findings of error on the MC forums ?
They have superb tech support. I would be very surprised that no one else discovered the same errors. Exactly what "errors" did you find ?

You mean the MC support guys who don't understand basic math?