Interesting Article.

Quote from ssrrkk:

]

okay two things that are believable: profitability didn't last (of course, it's curve-fit), and he couldn't sell the algo. other than that, the rest of the story is still fishy.
More...

Hi, I'm the author of the article in question. Would be happy to clear up any fishy-ness. A couple of corrections. I only sold one business. When I got back into trading I had no money. Borrowed 10k from my dad to trade Russell futures manually. Built that up to 30k before I turned on the automated program.

Could you describe what you mean by curve-fit? I want to make sure we are talking about the same thing.
 
Quote from jspauld:

Hi, I'm the author of the article in question. Would be happy to clear up any fishy-ness. A couple of corrections. I only sold one business. When I got back into trading I had no money. Borrowed 10k from my dad to trade Russell futures manually. Built that up to 30k before I turned on the automated program.

Could you describe what you mean by curve-fit? I want to make sure we are talking about the same thing.
More...

do you really think machine learning had anything to do with profits or was it all getting into a position and getting out when profitable? in the years you were profitable the markets were swinging.
 
Quote from 2rosy:

do you really think machine learning had anything to do with profits or was it all getting into a position and getting out when profitable? in the years you were profitable the markets were swinging.
More...

Yes I think ML had everything to do with it. The algo had hundreds of variables and they were all optimized using ML.

I certainly didn't have a strategy of "getting out when profitable". My algo did not make any of it's decisions based on the price that a trade was entered.
 
Quote from jspauld:

Hi, I'm the author of the article in question. Would be happy to clear up any fishy-ness. A couple of corrections. I only sold one business. When I got back into trading I had no money. Borrowed 10k from my dad to trade Russell futures manually. Built that up to 30k before I turned on the automated program.

Could you describe what you mean by curve-fit? I want to make sure we are talking about the same thing.
More...

Curve-fit means you have too many parameters in your model such that it fits perfectly the data you trained on (past data) but has very little ability to generalize to future data. So you will have a perfect back test but it will perform absolutely horribly during live trading. This is not a theoretical concern: it happens to countless novice algorithm traders...

So what happened to the money from the sold business? Also why give up so easily after making 500K? I would think anyone who made money with their algo will try desperately to find ways to get the algorithm to work again... often people will spend years, may be even decades to recover such a thing.

Also six months development time for the simulator, the model development, algorithm development, all of the fitting exercises, then writing the actual execution code, debugging all of that, then testing on a paper account, well that all seems to be a bit of a stretch. May be you code like 20 hours a day with no break but even then it seems a bit unfeasible.
 
Quote from 2rosy:

do you really think machine learning had anything to do with profits or was it all getting into a position and getting out when profitable? in the years you were profitable the markets were swinging.
More...

His pnl decay is very typical of results from survivorship bias. Note to the guy who did this: survivorship bias in this setting isn't about how many trades you did, but all the others who tried the same thing, failed, and never wrote a blog post about it.

Anyway, I'm inclined to believe he made 500k. But I also believe if he kept it running, he would be in the negative.
 
Quote from ssrrkk:

Curve-fit means you have too many parameters in your model such that it fits perfectly the data you trained on (past data) but has very little ability to generalize to future data. So you will have a perfect back test but it will perform absolutely horribly during live trading. This is not a theoretical concern: it happens to countless novice algorithm traders...

So what happened to the money from the sold business? Also why give up so easily after making 500K? I would think anyone who made money with their algo will try desperately to find ways to get the algorithm to work again... often people will spend years, may be even decades to recover such a thing.

Also six months development time for the simulator, the model development, algorithm development, all of the fitting exercises, then writing the actual execution code, debugging all of that, then testing on a paper account, well that all seems to be a bit of a stretch. May be you code like 20 hours a day with no break but even then it seems a bit unfeasible.
More...

Okay, you are talking about overfitting. I was very aware of this as I setup my system. It really wasn't much of an issue. (ie my system performed almost as well on new data as it did on training data.) It's pretty easy to check for this by assigning a portion of your training data as validation data. Also, keep in mind all of my indicators were for very short term market movements and so each day gave me thousands of data points for each indicator.

The money from selling my first business got used up traveling and starting the second. With regard to giving up easily. I do not feel I did this. As mentioned I spent four months trying everything I could to improve profitability. I even paid $20,000 to get a new data stream to see if it helped - it didn't. On the other hand, my passion is startups and not really finance so maybe I did give up to easily.

I can't really respond to the 6 months thing other than to say that seems like a decent chunk of time to me. Especially considering I never knew for sure it was going to work.
 
Quote from CT10Gov:

His pnl decay is very typical of results from survivorship bias. Note to the guy who did this: survivorship bias in this setting isn't about how many trades you did, but all the others who tried the same thing, failed, and never wrote a blog post about it.

Anyway, I'm inclined to believe he made 500k. But I also believe if he kept it running, he would be in the negative.
More...

What do you mean my P&L decay is typical of the results from survivorship bias? How so exactly? btw, I agree with you that if I kept my program running my results would be negative. In fact I KNEW that the results would be negative and that's why I turned it off.

When saying survivorship bias are you just making the point that others have tried similar things and lost money? If so, yes, and that's clearly good to keep in mind. But, it sounds like you are saying I'm basically the guy that went to the casino and won a million dollars and wrote a blog post about it. (ie. I just got the most lucky out of everyone that tried)
 
Are you currently tapped out of ideas in terms of implementing new systems?. You should have stuck with it, but I understand if its not your passion. A boutique algo trading firm would have been a nice startup, probably could have sold it for millions.
 
If you google survivorship bias, you will get the gyst of the meaning, but in trading it can mean curve fitting via backtesting to a flawed method which is backtesting only on stocks which are now currently trading, not the true measure which is backtesting to those currently trading AND those which no longer are in existence. It can also mean backtesting the winners and not the losers.

jspauld, I think you are a noob, no disrespect, your system is curve fitting similar to neural networks I think, which sounds good in theory buy not practice.

It is possible to get your system to work if you implement it only during certain events, ie don't trade it anytime of the day, implement it only when for example when all markets are rising or falling or whatever and maybe just after the market opens etc.

Anyhow, thanks for sharing, wishing you luck.
 
Quote from jspauld:

What do you mean my P&L decay is typical of the results from survivorship bias? How so exactly? btw, I agree with you that if I kept my program running my results would be negative. In fact I W that the results would be negative and that's why I turned it off.

When saying survivorship bias are you just making the point that others have tried similar things and lost money? If so, yes, and that's clearly good to keep in mind. But, it sounds like you are saying I'm basically the guy that went to the casino and won a million dollars and wrote a blog post about it. (ie. I just got the most lucky out of everyone that tried)
More...

Your PNL peaked early and then decayed pretty quick over the next few months.

Many many others have tried basically the same idea. Some failed immediately. Others experienced what you did. The luck part is that its not possible to pick which of the two states you end up in. I believe personally thatthe way you approach this is a dead end.

So, I do think you got lucky. But that isn't to take away the work you've done. In any case, its better to be lucky than to be right. And you are half a mil richer.
 
You must log in or register to reply here.
Back
Top