A/B Split Testing New Trading Ideas

[KCalhoun]

I've been testing trading ideas for over 20 years, one of my favorite techniques is to compare two different ideas to see which one works best. Has anybody tried a similar approach?

A simple example would be to test profitability for using two different stop-loss values, whether for day or swing trading.

I got the idea from advertising, where they would do things like test two different headlines or subject matter email topics to see which one converts best.

After you find which of the two initial trading ideas works best, you then test to variations of it to drill down and get more specific. I was doing this all last week with day trades and it's working extremely well.

An example is testing entries either in the middle of consolidation ranges, vs waiting for support pivots or above the prior high for a breakout. I almost always got stopped out when trading inside the middle of consolidation ranges, so I'm going to remember to stop doing that next week.

Another example is using a wide trailing stop versus hard stop.

When split testing trailing stop strategies, I was testing using a .2 trailing stop and just letting it ride, versus manually tightening it from $0.20 to $0.05 once the price action has run up in my favor at least .6, and I found that manually tightening produced the most profit per trade.

 

[SPX Options Trader]

I've been testing trading ideas for over 20 years

Has anybody tried a similar approach?

Well, we all tried various good, great, weird, strange, bad and odd approaches to our trading.

That's similar to someone, or generally a young person, discovering what they want to do with their life...or picking, choosing, dating, marrying another person.
It can be a complex, lengthy, sloppy, procedure of many trials and errors.

There is no right or wrong way, per se. Different people may see different lights from various vantage points -- but we're all focused on the same, central, dot.

 

[R1234]

I've been testing trading ideas for over 20 years, one of my favorite techniques is to compare two different ideas to see which one works best. Has anybody tried a similar approach?

A simple example would be to test profitability for using two different stop-loss values, whether for day or swing trading.

I got the idea from advertising, where they would do things like test two different headlines or subject matter email topics to see which one converts best.

After you find which of the two initial trading ideas works best, you then test to variations of it to drill down and get more specific. I was doing this all last week with day trades and it's working extremely well.

An example is testing entries either in the middle of consolidation ranges, vs waiting for support pivots or above the prior high for a breakout. I almost always got stopped out when trading inside the middle of consolidation ranges, so I'm going to remember to stop doing that next week.

Another example is using a wide trailing stop versus hard stop.

When split testing trailing stop strategies, I was testing using a .2 trailing stop and just letting it ride, versus manually tightening it from $0.20 to $0.05 once the price action has run up in my favor at least .6, and I found that manually tightening produced the most profit per trade.

I've been forward testing an intraday technique with real money. This strategy is impossible to backtest because there is a fair amount of discretion involved in entry points as well as the stocks to choose.

I have been trying varying entry points and exit methods (eg. time stop, trailing stop). I have been putting detailed notes in my tradelog beside each completed trade. As I accumulate a big sample size of trades the goal is to start doing some statistical analysis to see which entry and exit methods seem to have more edge.

 

[padutrader]

I almost always got stopped out when trading inside the middle of consolidation ranges, so I'm going to remember to stop doing that next week.

i realised the same thing........after much heart burn

trading consolidation is like searching a dustbin, enduring lot of rotten things and stink, expecting to find a diamond.

what is worse after going through 3-4 emotionally draining losses, you miss the real move.

which is why the market consolidates in the first place: to frustrate the hell out of you. and it usually succeeds.

if you think about it, a consolidation is where no one is in control, neither a bull or bear, so it is idiotic to think you can be in control.

looking out and waiting for a move, even if it means sitting on hands for some time, is quite the opposite emotionally.

it makes you feel smart and relaxed and in control.

in short it makes you feel like a master trader, one that realises that taking only the best trades is less frightening in a business that scares the hell out of everyone.

the prospect, nay the certainty, of losing money is none too pleasant and hangs ,like Damocles sword, over every trader's head.

but the good news is that YOU ARE IN CONTROL ......OF YOUR DESTINY. you choose your trades.

very few human activities offer this and it is priceless

the bad news is: there is no on else to blame the losses on , other than yourself and that is why people leave trading.......

trading destroys the ego and the human being is nothing but all ego

 

[Bad_Badness]

If you are going to "test" you need to understand what that means. Testing requires measurements. So figure out what you want to measure, not just P/L. The first problem in testing is finding what to measure. Second problem is discerning which measurements are relevant to your goal. If you can get started correctly with these points, you have a reasonable chance of progress.

So before one goes of "testing", know what you are doing. This akin to a person going to "trade" and not know what that means. Testing, without knowing what you are doing, is worse than going with your gut. Probably the same inconclusive results, but at least you spend less time lol.

Below are four charts that summarize the measurements I take over a month on any particular strategy. Each input is the summary of a single day of trading. In this example only day 2 meets the criterion.



PS: My background is professional software testing.

 

[KCalhoun]

If you are going to "test" you need to understand what that means. Testing requires measurements. So figure out what you want to measure, not just P/L. The first problem in testing is finding what to measure. Second problem is discerning which measurements are relevant to your goal. If you can get started correctly with these points, you have a reasonable chance of progress.

So before one goes of "testing", know what you are doing. This akin to a person going to "trade" and not know what that means. Testing, without knowing what you are doing, is worse than going with your gut. Probably the same inconclusive results, but at least you spend less time lol.

Below are four charts that summarize the measurements I take over a month on any particular strategy. Each input is the summary of a single day of trading. In this example only day 2 meets the criterion.



PS: My background is professional software testing.

Excellent points. I used to be a statistician at Ford. Extensive experience with DOE, multivariate analysis etc