how much trades for validate a strategy

[kut2k2]

Quote from nonlinear5:

Give me any instrument, and I'll find a system that generates 20 winning trades in a row for that instrument. That's the magic of "curve-fitting". The more trades, the more difficult it is to curve-fit. What I'd suggest is calculating a system quality measure which includes the number of trades. For example:

System Quality = sqrt(number of trades) * AveTrade / StandardDeviation(all trades)

That makes the system quality unconditionally dependent on the number of trades, which doesn't make sense. I devised the System Achievement Score to get around that flaw.

SAS == 4*k*max[ 0, E ]*PF*min[ 1, N/mant ] ,
where
SAS is the System Achievement Score,
k is the solution to the Kelly equation (see below),
E is the expectation (see below),
PF is the profit factor (see below),
N is the number of trades in the SAS evaluation,
mant is the minimum acceptable number of trades.

E == sum[ Ri ]_i=1toN / N ,
where
Ri is the return (%) of the i'th trade.

The Kelly equation is
0 == sum[ Ri/(1+k*Ri) ]_i=1toN

PF == sum[ max[ 0, Ri ] ]_i=1toN / sum[ max[ 0, -Ri ] ]_i=1toN

 

[Dhalsim]

Quote from bashatrader:

The absolute minimum for EOD is 20-30 trades. This is what some statistics books say. For ID there should be way more. Mike Harris has proposed a formula for the number of trades N = 20 x Np, Np is the number of bars in timeframe of operation. For 1-min stock data this comes to about 7800 trades. For 15-min data it goes down to 1950 trades. I think these numbers are reasonable.

I think this is nonsense.

Unless you are doing some form of HFT or arbitrage there is no way there are that many opportunities for a good trade in a given day.

If i am trading ES for example, at most in a day there is probably 3 good trades if you are lucky (on a low volatility period maybe you will get lucky to have one good trade).

You guys act like the be all to end all is the number of trades, to me this makes no sense because if the market is mostly random with very few opportunities to exploit some sort of edge. Hence, trading 100s of trades in a mostly random environment is silly.

Now one way to get around the back-testing data fact is especially if strategy is based on stocks - simply test the strategy against 100 stocks. So for a strategy that averages around 100 trades a year then traded against basket of 100 stocks then that's 10000 trades for a given year.

Don't understand who these people are that can trade more than 10 times on ES on a given day without HFT or extreme scalping and then make money. It is mostly random and therefore no chance to make money in long run.

Also, time-frame has not much to do with anything, i have strategies which are based on 1min time-frame but often stay in trades for hours or the whole day. Time-frame is simply used as method to pinpoint entries.

 

[Handle123]

Quote from bashatrader:

The absolute minimum for EOD is 20-30 trades. This is what some statistics books say. For ID there should be way more. Mike Harris has proposed a formula for the number of trades N = 20 x Np, Np is the number of bars in timeframe of operation. For 1-min stock data this comes to about 7800 trades. For 15-min data it goes down to 1950 trades. I think these numbers are reasonable.

The ES system I backtested three years ago had sample size of over 35,000 trades, it ran back for ten years on tick data. I know everything this method has done in past 13 years, I forward tested it one year before trading it real time. Because of extensive backtesting, and what history has shown as far as weekly/daily drawdowns, I can average down, double and even triple contract size. This is not to say the future will never get worse than the past as it will one day, but day trading is a numbers game just like the casinos. And if you think 20-30 sample size is enough, you will be running through accounts like Obama with our billions of dollars.

 

[Handle123]

Quote from Slope Trader:

Hi Handle,

I'm glad to see that you're still sharing your thoughts here at ET and I wish you well.

I backtest methods manually, primarily on daily charts, looking at one bar at a time over a period of years and recording entries and exits as I scroll the chart forward. I pretty much always start with January 1st of a year and end on December 31st of a future year.

I test enough years to get 50-100 trades and then I calculate several standard performance metrics, such as net profit, profit factor, maxDD, and several others. I do this in one futures market that I trade and then I do it in 2 or 3 other unrelated futures markets that I also trade.

A couple of my best recent strategies have had profit factors above 1.5 and win rates of 54-62% in each market backtested. So far, they're also doing well in real-time forward testing in those same markets, though it's only been a few months.

I understand that you normally want 1000's of trades in your backtests and I only have a few hundred for these methods. In your experience, do you think it's possible to have confidence in data derived from less than 1000's of trades if they exhibit a certain high level of performance and it is observed in multiple uncorrelated markets?

If the subsequent walk forward tests were also strong, though limited to only ~ 50 trades so far, would that be more convincing... or would you still need a lot more data to feel confident that the system is tradable?

I am sorry but economies change, supply/demand changes. Just like when a President is voted in, he is running under last Presidents budgets, it be like two years before one can only start to see what a new Pres might be able to do. Let's put it this way, if you own a home outright, are you willing to risk 5% of the home by using your sample size? I know my long term method will generate between 70-85% losing futures trades, some backtests went back to 1920's, using options to hedge changes entire overall trade, plus when the futures positions become profitable, after first targets are meet, there is 12 to 1 reward to risk. The longer the backtest, the more questions you can ask of IT and more confidense you can increase contract size. Learn to program would be good. Go to a Community college and chat with dean of computer science, see if he willing to have a couple classes make what you want to do a class assignment for cost of you giving pizza parties, that's what I use to do before I learned to program.

 

[Dhalsim]

Quote from Handle123:

The ES system I backtested three years ago had sample size of over 35,000 trades, it ran back for ten years on tick data. I know everything this method has done in past 13 years, I forward tested it one year before trading it real time. Because of extensive backtesting, and what history has shown as far as weekly/daily drawdowns, I can average down, double and even triple contract size. This is not to say the future will never get worse than the past as it will one day, but day trading is a numbers game just like the casinos. And if you think 20-30 sample size is enough, you will be running through accounts like Obama with our billions of dollars.

3500 trades a year on ES?? HFT??

 

[Sergio77]

Quote from Dhalsim:

Don't understand who these people are that can trade more than 10 times on ES on a given day without HFT or extreme scalping and then make money. It is mostly random and therefore no chance to make money in long run.

I basically agree but if I don't understand something it don't mean it can't be done.

Quote from Dhalsim:

Also, time-frame has not much to do with anything, i have strategies which are based on 1min time-frame but often stay in trades for hours or the whole day. Time-frame is simply used as method to pinpoint entries.

If you hold a trade longer than a few minutes it means your entries are not even relevant and you may as well play only with your exits. Prices have little memory, maybe a few bars, then you are probably following a trend and your entry is irrelevant.

 

[Dhalsim]

Quote from Sergio77:

I basically agree but if I don't understand something it don't mean it can't be done.




If you hold a trade longer than a few minutes it means your entries are not even relevant and you may as well play only with your exits. Prices have little memory, maybe a few bars, then you are probably following a trend and your entry is irrelevant.

Pure nonsense again. So how do you predict start of a trend? How do you enter an intraday position where chance of trend is higher than usual. I guess you don't test your entry against random entries and hence deduce whether ur entry has an edge on itself.

There are entries which put probability in ur favour of getting a reversal or chance of a trend or volatility expansion.

Just like other ET guys your just trying to make it more complicated than it is. Most people who don't scalp and trade intraday with holding periods of more than 30 mins play mean reversion. Mean reversion is played best with an entry which puts you in good position of meaningful reversal or small stop out.

Prices does care about the past and hence why trends exist on all time frames.

 

[Slope Trader]

Quote from Handle123:

...I know my long term method will generate between 70-85% losing futures trades, some backtests went back to 1920's, using options to hedge changes entire overall trade, plus when the futures positions become profitable, after first targets are meet, there is 12 to 1 reward to risk. The longer the backtest, the more questions you can ask of IT and more confidense you can increase contract size. Learn to program would be good. Go to a Community college and chat with dean of computer science, see if he willing to have a couple classes make what you want to do a class assignment for cost of you giving pizza parties, that's what I use to do before I learned to program.

I definitely get your point about longer backtests giving one more confidence in the results, and working with a computer science professor was an ingenious idea.

Even though it's a bit time-consuming, I'm going to keep performing manual backtesting because it's been working pretty well. I've been profitable every month except one (last December) during the past 2 years and I'm up 52% for this year, which is my best year ever.

I used to play a lot of competitive chess and analyzing price bar patterns and indicator patterns on charts feels similar to analyzing positions on a chessboard. Instead of “What’s the best move?”, though, the question is “What entry and exit rules give a reproducible, strong net profit, profit factor greater than 1.5, win % greater than or equal to 50%, and reasonable max drawdown?”

There may be a psychological advantage to manual backtesting too. Following a price chart and moving it forward one bar at a time to identify entries and exits is, in a sense, like “practicing” trading. It has helped make it easier for me to take all entry signals, even when volatility becomes very high, and this has improved my performance.

Other than net profit, are there one or two particular performance metrics that you focus on when evaluating your systems?

 

[ronblack]

Quote from Dhalsim:

Prices does care about the past and hence why trends exist on all time frames.

You can get very nice and smooth trends with a random number generator. "If it has four legs, it is not always a dog." You better read Aronson's book (pp. 115-121). Although you have genuine intentions your thinking is seriously flawed.

 

[Dhalsim]

Quote from ronblack:

You can get very nice and smooth trends with a random number generator. "If it has four legs, it is not always a dog." You better read Aronson's book (pp. 115-121). Although you have genuine intentions your thinking is seriously flawed.

If you believe in random walk then why not explain to us what kind of strategies you use?