Edge - Shock Proofing your Strategy

R

Rapunzel

Hi folks, hope you are well. Do you guys know how one can shock test a trading signal model against unseen market conditions.

Most models will be developed using history data for an asset and will rely on the replication of the market conditions on which the models were created to an extent, for a replication of the model results.

I am looking for some strategy testing tools that will allow you to generate random price data for an asset. Random data within certain pre-defined parameters, like volatility, trend direction, trend strength etc...so that I can then go on to use the randomly generated data to 'back-test' my strategies and understand the conditions in which to switch off my signal generator.
 
My opinion: Use data from the past. Trying to predict something that has never happened before rarely turns out well. You can make up all sorts of data to test your strategy with but how are you ensuring your data simulates the real world?
 
Hi, thanks for the post. I think that even the best strategy is fit round data from the past. If the past market conditions were different then the filters values would be different to the current model values. So there re conditions in which your model will work optimally and those that it will not since every strategy fails in some conditions.

So the aim is to understand the conditions on which the strategy will begin to fail and the appropriate conditions where it works best.

it's not really looking for grey or black swan events just the parameters in which your model works best and those where it begins to fail.

I believe in most cases where a model fails to replicate on new data is when the market conditions in the new data are starkly different from the data on which the model was initially trained.
 
Rapunzel said:
So the aim is to understand the conditions on which the strategy will begin to fail and the appropriate conditions where it works best
More...

You could use historical data to test during a major event and see how your account will handle it but its something you cannot rely on because such a test would give you an idea of what COULD happen and not what WILL happen under real trading conditions.

I say this because when extreme events take place, strange things happen.
For example, your trading software gets disconnected from the server and you cannot close your trades, or your broker's liquidity provider halts trading and your SL's get filled when connection resumes 500 pips later, or the spread goes through the roof and your orders get filled at the worst possible price.
 
Rapunzel said:
Hi, thanks for the post. I think that even the best strategy is fit round data from the past. If the past market conditions were different then the filters values would be different to the current model values. So there re conditions in which your model will work optimally and those that it will not since every strategy fails in some conditions.

So the aim is to understand the conditions on which the strategy will begin to fail and the appropriate conditions where it works best.

it's not really looking for grey or black swan events just the parameters in which your model works best and those where it begins to fail.

I believe in most cases where a model fails to replicate on new data is when the market conditions in the new data are starkly different from the data on which the model was initially trained.
More...


I am just starting to get into automation myself and looking to have some bots / strategies built, plus not the smartest person ever so I could be over simplifying it. But wouldn't it just be easier to create a benchmark based on larger charts or other data to determine if market is bullish (to what degree) or neutral, or bearish(and to what degree). Once you have that, than you can choose to turn on or off your long or short strategy based upon that.

Not saying it's easy to come up with that, but point is if the market is bullish almost any decent buy signal or trigger is going to have high probability of achieving a net profit, just like if market is bearish almost any decent short signal is going to have high probability of achieving a net profit.

This is how I intend on doing my automation. Will determine based on set factors if market is probability bullish, probability bearish or neutral and than turn on the strategies best suited for the current perimeters or day.


EDIT: This won't really help for your "shock" scenario, but like others mentioned there's not really much you can do about that. You can look at flash crashes and Covid events, to look at what your max damage would be.... but even than you really have no clue about fills and etc.
 
Concinnity said:
I am just starting to get into automation myself and looking to have some bots / strategies built, plus not the smartest person ever so I could be over simplifying it. But wouldn't it just be easier to create a benchmark based on larger charts or other data to determine if market is bullish (to what degree) or neutral, or bearish(and to what degree). Once you have that, than you can choose to turn on or off your long or short strategy based upon that.

Not saying it's easy to come up with that, but point is if the market is bullish almost any decent buy signal or trigger is going to have high probability of achieving a net profit, just like if market is bearish almost any decent short signal is going to have high probability of achieving a net profit.

This is how I intend on doing my automation. Will determine based on set factors if market is probability bullish, probability bearish or neutral and than turn on the strategies best suited for the current perimeters or day.


EDIT: This won't really help for your "shock" scenario, but like others mentioned there's not really much you can do about that. You can look at flash crashes and Covid events, to look at what your max damage would be.... but even than you really have no clue about fills and etc.
More...

Thanks for the post. That's a great idea, and something that incorporates ^ would probably be a good way to go.
 
You must log in or register to reply here.
Back
Top