how much live trading is relevant?

[NihabaAshi]

Quote from flatron:

Alot of people say that backtesting/backtested results are pointless and that trading the sytem in realtime with real money is the only way?

...How long would you want to see reults like this until you were confident enough to start trading 'real' stakes (E.G futures contractw rather than spreadbet trades which can be as low as the equivilant of a 10th of a contract)

thanks

If you're a system designer, coder or academia (researcher)...backtesting is extremely important and you may not care to much about real money results until it's your money on the line.

In contrast, if you're trading for a living or trying to do such...real money trading is extremely important and you may not care to much about backtesting results mainly because it's now old news. Further, you can't put a time duration on how long it takes to become "confident" in your trade method because some guys become confident after a few trades (extreme optimistic types) while others with something that works for several years never become confident (the paranoid types).

Mark

 

[flatron]

Quote from NihabaAshi:

If you're a system designer, coder or academia (researcher)...backtesting is extremely important and you may not care to much about real money results until it's your money on the line.

In contrast, if you're trading for a living or trying to do such...real money trading is extremely important and you may not care to much about backtesting results mainly because it's now old news. Further, you can't put a time duration on how long it takes to become "confident" in your trade method because some guys become confident after a few trades (extreme optimistic types) while others with something that works for several years never become confident (the paranoid types).

Mark

Thanks. Im definitely in the latter category. Im pescimistic by nature, and the posts about how my method likely stop working soon doesn't help that matter!! lol.

Im not a coder/programmer. Dont know anything about that sort of stuff. I just know that ive been trading my method live for a month or so, taken about 300 trades, and it feels like ive 'got something'.

Take you for example. Are you profitable? Are you waiting for the enevitable moment when your current method stops working?

Do you know of any traders who just trade the chart as they see it and have traded that same 'way' for a long time and have no intention of having to change anything regarding their trading style?

cheers

 

[duhmentor]

Quote from flatron:


How long would you want to see reults like this until you were confident enough to start trading 'real' stakes (E.G futures contractw rather than spreadbet trades which can be as low as the equivilant of a 10th of a contract)

thanks

The focus should be more on risk than on time.

Am I ready to risk “real stakes”

Do I have enough confidence in my system to stick with it when committing “real stakes” ?

 

[TheGoonior]

I would want to see how the system performs in as many market conditions as possible and the more trades you can make, the better. If you're generating 15 trades a day, that's going to give you a pretty good sample space in a month or two.
Since you are trading intra-day, compare how it handles the various scenarios (ranging day up/down, trending day up/down, news day, holiday week, options day, etc) and see how consistent it is. If you are making money now, why not just gradually scale up your sizing slowly rather than jumping in with a bigger position suddenly? You're basically getting paid to test your system at the moment.

Regarding backtesting, I just prefer out of sample forward testing because it's so much easier and it parallels how you would need to make adjustments when actually trading.

 

[flatron]

But by the sounds of it, trading methods only work for a few months, and then you have to start over again, and since this one has been profitable for a little while now, maybe id be pushing my luck anyway by increasing size, since it cant be profitable for much longer...?....

Any profitable traders want to help put my mind at rest, and dispute the above theory?

thx

 

[flatron]

Quote from TheGoonior:

I would want to see how the system performs in as many market conditions as possible and the more trades you can make, the better. If you're generating 15 trades a day, that's going to give you a pretty good sample space in a month or two.
Since you are trading intra-day, compare how it handles the various scenarios (ranging day up/down, trending day up/down, news day, holiday week, options day, etc) and see how consistent it is. If you are making money now, why not just gradually scale up your sizing slowly rather than jumping in with a bigger position suddenly? You're basically getting paid to test your system at the moment.

Regarding backtesting, I just prefer out of sample forward testing because it's so much easier and it parallels how you would need to make adjustments when actually trading.

cheers. For now im gonna just keep on doing what im doing, and generate lots of live/real results, increasing the sample size, and wait until the system enevitably stops working (apparently )

 

[Swan Noir]

You really can't mix and match the terms "system" and "method" -- they are simply different things. I work with a method that because it involves a fair amount of discretion can adapt in many ways without me changing the shell -- the basic methodology -- I use.

You are talking about a system and by definition something more rigid. Even though I am not a system trader I can see that systems need to be modified, changed dramatically and maybe even turned upside down at times. That said systems lend themselves to rigorous back testing. Years ago I owned and ran a number of direct market businesses here and in Western Europe. While not an expert on systems I can not believe that 60 trades is a large enough sample. Our rule of thumb on direct mail was that we needed a bare minimum of 35 responses and preferred 50 or 60 before we could make educated guesses as to whether we had a potential winning package.

Once we guessed we MIGHT have a winner we would at least triple our next mailing as a second test and if our returns were similar we would step on the gas and increase the budget by 10, 20 or even 40 times. In order to be sure to obtain the initial 35 responses we always mailed at least 5,000 pieces. I think 60 trades leaves you at the vagaries of far to many regularly occurring patterns that exist within the law of large numbers to draw any conclusions that you can even pretend are scientific. I say back test at least 200 trades (a modest number) do a bit of guesswork to decide if you MIGHT be on to something and then back test another 800 or so trades and see if both batches are in the same ballpark.

It's manageable in terms of size and effort and probably 100X (or more) valid than a straight 60.

Quote from flatron:

,but because mt 'method' is very short term it naturally makes quite a few trades and I believe that only a few months consistency is likely statistically significant.



However, when you talk about how 'most systems will work for a period and then stop working so i should get cracking', I find that kind of bizarre.



thanks again!

 

[flatron]

Quote from Swan Noir:

You really can't mix and match the terms "system" and "method" -- they are simply different things. I work with a method that because it involves a fair amount of discretion can adapt in many ways without me changing the shell -- the basic methodology -- I use.

You are talking about a system and by definition something more rigid. Even though I am not a system trader I can see that systems need to be modified, changed dramatically and maybe even turned upside down at times. That said systems lend themselves to rigorous back testing. Years ago I owned and ran a number of direct market businesses here and in Western Europe. While not an expert on systems I can not believe that 60 trades is a large enough sample. Our rule of thumb on direct mail was that we needed a bare minimum of 35 responses and preferred 50 or 60 before we could make educated guesses as to whether we had a potential winning package.

Once we guessed we MIGHT have a winner we would at least triple our next mailing as a second test and if our returns were similar we would step on the gas and increase the budget by 10, 20 or even 40 times. In order to be sure to obtain the initial 35 responses we always mailed at least 5,000 pieces. I think 60 trades leaves you at the vagaries of far to many regularly occurring patterns that exist within the law of large numbers to draw any conclusions that you can even pretend are scientific. I say back test at least 200 trades (a modest number) do a bit of guesswork to decide if you MIGHT be on to something and then back test another 800 or so trades and see if both batches are in the same ballpark.

It's manageable in terms of size and effort and probably 100X (or more) valid than a straight 60.

Thanks fro the reply Swan.
Where did you get '60 trades' from? Its actual a little over 300 (but I guess thats besides the point as its still a small number)

I like what you say about 'systems' and 'methods' and how the 2 are different. Maybe you given a whole new angle on the ongoing debate, as im sure lots of people, like me, dont seperate the 2 and consider them one in the same

Its like my earlier post where I said that the only way I can imagine something suddenly stop working is if it was a 'SYSTEM' like: "when CCI crosses 0 and the RSI is sloping up at x degree angle wait for an macd cross and go long" etc etc.

I guess what I am working with is a 'method' (not a 'system' ..(?)). Its pretty much a naked chart and experience has shown me cetain instances where i can take a trade with a small stop and a slightly bigger target, and it often wins.

I couldn't backtest it as its slightly discressionary and i cant program. I could (and have done) do some manual backtesting though.

 

[market_mapper]

lots of traders seem to think that 100+ sample sizes should yield some sort of quality statistical results. the more data you have the more solid the results. however, in order to test an edge in the markets you only need a 25 trade sample size. you clearly have more data than that. does this mean you can just throw it out there and let it do its thing? absolutely not, you will need to keep an eye on the outcome of your edge for the rest of your trading life. markets change, variables change and an edge now might not be an edge in the future.

 

[Swan Noir]

FLATIRON ...

I was commenting on the quote below. That's where I got 60 trades from and I was expressing my opinion that it was far from sufficient to even make guesstimates.

Quote from abattia:

My view?

A history of at least 60 trades starts to give some "statistical comfort" that the behaviour you are seeing is a measure of the system's performance, rather than just a statistically insignificant sequence of random outcomes. I am sure that many traders will say 60 trades is not enough, and I would be interested to see what others think...