TA forward testing sample size?

I

illstep

What kind of statistical significance would one need to think that he MIGHT have some sort of an edge'?

I'm someone whose been trying to trade for about 4 years and has been consistently losing. Almost everyday. Can't buy a winning day.

Sometimes i'll find what I think might be an 'edge' through lots of backtesting, but when live trading, it tends to break even at best. (on sim and real money)

In fact, in all my time trying to do this, i've never had a 'spell' of LIVE trading (either SIM or real money) that's made me think ''Hmmmm....I wonder if i'm onto something here! This is my 2nd week of making money, adn i've made almost every day!''.

Until very recently that is.

I seem to make money most days at the moment. long and short. Although i'm super pessimistic (realistic?) by nature about this sort of thing so i'm not getting carried away. At all!! Could always just be luck, I guess

If you are daytrading the futures, and maybe you typically take around 6 trades per day (stops of maybe 2 points and targets maybe slightly bigger, although all dependent on various technicals) what sort of results should give you confidence? How many trades/day trading would you need for a good sample size?
 
Statistical significance of a backtest can not be evaluated in a vacuum. It also depends on the process that was used to create/discover the trading rules. What is your process for creating your systems, do you understand the concept of curve fitting and are you taking appropriate steps to avoid it? Are you using walk forward optimization?

Relevant factors for assessing significance:
Complexity of trading rules (simpler is better)
Logical sense of trading rules (is there a plausible economic explanation for why they might be profitable)
Number of trades,
Number of years of backtest (10 years is far better than 1, anything less than 1 is completely worthless unless you are doing >100 trades per day)
Number of instruments traded (if it only works on one instrument, its probably curve fit)
Performance in various market regimes, bull,bear etc.

Personally the main thing I look for is a daily sharpe around 4 or higher before I consider something tradable, but also depending on all the other factors mentioned above.

The other thing is, once you get to real trading, are you actually executing your system with 100% accuracy or are there discrepancies - if you extend your backtest does it 100% match what happened in real life? If not then you need to figure out why and either fix it or make sure that it doesn't matter. Faithful execution is key when trading based on backtests.
 
I like a chart historic of your account values, dump your closing trades into excel, make a line chart, censor the value and post that, that'll give us a good indication of performance.

Add a bar chart profit or loss individual trades aswell, censored is fine, values aren't important.
 
illstep said:
What kind of statistical significance would one need to think that he MIGHT have some sort of an edge'?
More...

First, you need to understand that the 'statistical significance' you're seeking stems from a paradigm that assumes each 'trial' [moment in the market from which to snatch a trade] is independent from the prior trials, and of course unrelated to the future trials, right?

Except, a problem: the best predictor in any market of today's price, is yesterday's price, right? Serial correlation. :mad:
https://en.wikipedia.org/wiki/Independent_and_identically_distributed_random_variables

ALL IS NOT LOST! But, you definitely want to pause and re-think things......
 
Take this from somebody who has been down this rabbit hole...it doesn't matter how many trades you have in your back-test or what fancy stats tests you use, if you're just mashing together indicators, all you are doing is fitting noise to noise and it will break down live. Your trades need to work for a *reason*, or to put it another way, you need to be exploiting some fact about the market which is correct but not widely known.
 
at the moment i'm just trading live in SIM.

I know it's probably nothing, but it's just that this is the first time I've even had a HINT of positive, somewhat consistent results.

Maybe i'll just try to do like 10,000 trades, and if i'm still looking good, i'll go live money!

zG9iC1M.png


HmoFKAd.png



au0PyQY.png







 
Only take 6months at 67per week to get to 10,000, get on it!!

Win rate 53%, any way to improve a few %%'s ??

Other than that, profitable Demo for another week, then go Live and set yourself a Min Account level before you stop and go Demo again, the pressures of REAL money play with your mind, you need to experience this, then back to demo to work through it.
 
over 100 trades now. (that's the number someone on trade2win told me to take to see whether I was still profitable). New account hights. Plan to maybe do 500 trades and then if i'm still in profit, maybe fund a micro account


3hp3QTb.png
 
You must log in or register to reply here.
Back
Top