backtest for 3 years, blow up in 3 days,

[CyborgTrading]

Quote from iuykcif:

Thanks I will.

How is your product organized? Do you make available a few strategies among which the user can choose?

[I am currently with IB, and while rewriting the robot to connect to another broker too would not be impossible, I think I will undertake such effort only if the broker is among the top ones and has nice .NET API available. ]

____________________
Tom
My <a href="http://www.datatime.eu/public/gbot/2009Oct19/default.htm" target="_blank">auto trading</a> journal

Not exactly. While we do offer canned algos (http://www.youtube.com/CyborgTrading#p/u/0/ngwpcJy7CTA). Our product is designed to allows users to create their own strategies using our GUI. User can also do things like spread trading. Automating Trade-Ideas signals. News Trading. Or Simulation Trading for testing.
Cyborg is a bridge with allows you to do these things on sterling and soon laser through the api, like you did with IB.

 

[Alex55]

3 full years I tested the most advanced strategies (special kind of Neural-nets included), and only thing I learned was that:

a) noice in real-time historical data makes it hard to filter "general working strategies".

b) the found "working strategies" gave nice statistics (more than 80% winners) but in real world trading results were nearly zero (because of Slippage and commissions, and technical issues like an internet-stop).

c) the trained history, doesn't repeat in future

After I learned this I went to discretionary trading, and doing quite well.

 

[d08]

Quote from Alex55:

3 full years I tested the most advanced strategies (special kind of Neural-nets included), and only thing I learned was that:

a) noice in real-time historical data makes it hard to filter "general working strategies".

b) the found "working strategies" gave nice statistics (more than 80% winners) but in real world trading results were nearly zero (because of Slippage and commissions, and technical issues like an internet-stop).

c) the trained history, doesn't repeat in future

After I learned this I went to discretionary trading, and doing quite well.

Of course, most strategies have problems working real-time and will be discarded eventually - only the cream of the crop will be used. In my experience anyway. Also, there are successful traders around who use neural nets but it's definitely an approach prone to curve fitting (it essentially is a curve fitting method I would even say).

 

[iuykcif]

Quote from Alex55:

3 full years I tested the most advanced strategies (special kind of Neural-nets included), and only thing I learned was that:

...
b) the found "working strategies" gave nice statistics (more than 80% winners) but in real world trading results were nearly zero (because of Slippage and commissions, and technical issues like an internet-stop).

...

That's an example of the unsuitable stats I was referring to, to measure a strategy performance.

That would only lead to overfitting, especially with signal-based strategies.

That's what the pc does better: you provide the data it overfits them perfectly.

That's what i call the "prevision of the past"


Tom

 

[increasenow]

Quote from jacksmith:

One common story I had heard about trading system is that,

Successfully backtest for 3 year's data, yet the system blows up in 3 days,

what causes this ?

Thanks.

did this actually and for real happen to you?

 

[Alex55]

Quote from iuykcif:

That's an example of the unsuitable stats I was referring to, to measure a strategy performance.

That would only lead to overfitting, especially with signal-based strategies.

That's what the pc does better: you provide the data it overfits them perfectly.

That's what i call the "prevision of the past"


Tom

I know very well what overfitting means. My test were all done this way:
a) split the 5 years data-set into 3 parts.
b) use ONLY 1/3 for learning process
c) when ready use the second 2/3 for selecting the best strategies from set (b)
d) keep 1/3 for a last test before going life.

 

[iuykcif]

Quote from Alex55:

I know very well what overfitting means. My test were all done this way:
a) split the 5 years data-set into 3 parts.
b) use ONLY 1/3 for learning process
c) when ready use the second 2/3 for selecting the best strategies from set (b)
d) keep 1/3 for a last test before going life.

A "supervised" approach does not work, because there is actually nothing to "learn".

It's similar to computing correlation between unrelated phenomenons.
Whatever coefficient you obtain, is meaningless (spurious correlation).


Tom

____________________
Tom
My <a href="http://www.datatime.eu/public/gbot/2009Oct19/default.htm" target="_blank">auto trading</a> journal

 

[Alex55]

Quote from iuykcif:

A "supervised" approach does not work, because there is actually nothing to "learn".

What you want to learn is a filtering out the noise and keeping a general-rule. Which I managed quite well. But this general-rule wasn't usable to be profitable. Probably because it's the noise (odd, not repeating one time events) which makes trading profitable. ...just think about that!

 

[iuykcif]

Quote from Alex55:

What you want to learn is a filtering out the noise and keeping a general-rule. Which I managed quite well. But this general-rule wasn't usable to be profitable. Probably because it's the noise (odd, not repeating one time events) which makes trading profitable. ...just think about that!

No. Filtering is another kind of issue.

Actually the point is that there is no such a thing like a "general-rule".

I am sure you (and many others) may disagree, and you are fully entitled to you opinion. That's just mine.


____________________
Tom
My <a href="http://www.datatime.eu/public/gbot/2009Oct22/default.htm" target="_blank">autotrading</a> journal

 

[Alex55]

Quote from iuykcif:

Actually the point is that there is no such a thing like a "general-rule".

Then we both agree that automated trading doesn't work.
As all automated-trading engines are geared to exploit one rule (call it an edge) which works generally in the past and future.