Converting a probability to actionable data

[nooby_mcnoob]

I've been doing some some relatively complex conditional probability hypothesis testing and one of the permutations turned out to be too ridiculous to ignore (i.e., I can predict foo given x,y,z to 90% accuracy - obviously bullshit)

What I want to do is quickly test whether this is potentially profitable without worrying too much about specific strategy.

The approach I am considering is to enter at bar X, and exit at the "best" high/low simply to see the potential max profit - I know this is forward bias.

Is there another method you guys and gals use to test probabilities in backtests?

 

[fordewind]

Exit out of fear?You have to sit out an egg!

 

[MarkBrown]

interesting thread, there are so many things to test it will occupy your life forever.

we tend to think of trading data as going to the shopping mall, then upon arrival discover there are actually 1000 stores inside that mall.

we tend to test on "mall" data and then we are shocked our favorite "store" has closed and something else taken it's place.

the data is complex and more influential that any method or edge you might happen upon. so what i am saying is that your work is wasted if you don't first have a mall map of where all the stores are located. you will just be walking in a big circle on multiple levels with no idea where the store you want to shop at is located.

think of data as a map and drill down on the specific area your interested in going rather than getting lost spinning a globe of planet earth. by specifying areas of data i mean like deciding if you are trading momentum moves, day trading, scalping etc. all the while considering your cost of doing business. cost including commissions, data feeds, platforms, overhead and time commitment to the trading itself.

m

 

[nooby_mcnoob]

interesting thread, there are so many things to test it will occupy your life forever.

we tend to think of trading data as going to the shopping mall, then upon arrival discover there are actually 1000 stores inside that mall.

we tend to test on "mall" data and then we are shocked our favorite "store" has closed and something else taken it's place.

the data is complex and more influential that any method or edge you might happen upon. so what i am saying is that your work is wasted if you don't first have a mall map of where all the stores are located. you will just be walking in a big circle on multiple levels with no idea where the store you want to shop at is located.

think of data as a map and drill down on the specific area your interested in going rather than getting lost spinning a globe of planet earth. by specifying areas of data i mean like deciding if you are trading momentum moves, day trading, scalping etc. all the while considering your cost of doing business. cost including commissions, data feeds, platforms, overhead and time commitment to the trading itself.

m

Can you dumb it down for me a bit

 

[Overnight]

...
Is there another method you guys and gals use to test probabilities in backtests?

I can tell you this...In index futures, all bets are off.

Backtests would have proven for decades that index future A would not fall a certain number of points in a given timeframe. That shit has all been shattered this year. This means that since the beginning of time, when early in Feb 2018 the Dow fell for the most number of points in history in a single day, your backtests would not have prepared you for it, your probabilities would have been eradicated, and your system would fail.

This was exacerbated when it happened in the same fashion, but albeit in more slo-mo. From Oct to now. There are some folks out there who will decry that this happened in 1987, but I counter with, "Yes, the market fell by huge percentages, nothing like what is happening today". But in my trading world, it is the point count that matters, not the percentages.

Now if you design your backtests to provide for the probability that the market will fall another 1000 points from where it is now in a month, and how that correlates to previous market downdrafts, you may have something.

But here we are again, with percentages vs. points. Back during the major crashes, the market moved, say, 50% from previous peak. Your system may have found that manageable with the money you had in the account.

If the markets were to fall 50% today? You'd probably need 20-30 times the amount of cash in your account to survive.

Just do the math on point values at this time, work it into your backtesting, and come up with a way to hedge against these nasty swans that keep shitting in the pond of finance.

 

[SteveH]

You don't want high probability events. They ALWAYS result in 1:1 or worse reward to risk ratios in the long term. It's not optimal to be right too often. The absolute limit is 70% which WILL converge to 1:1 RR.

You want 40-60% area, then you make the gains primarily off of the reward to risk ratio average. You are allowed to F up more often and still win over time which, my guess is, a human being is apt to do from time to time.

This is the simple fact:

If you can guess at least 40% of the time that you are in a trend in real-time IN THE PRESENCE OF GOOD VOLATILITY then the entry is trivial and you will be a "winning trader" in the long term. How? Prove it to yourself. Look at the reward to risk ratios of taking opposing bar closes as entries in trends on the price-favored side of 9 wmas and/or 20 emas. [No, really, look over 1000's of charts. It's there...the obviousness is overwhelming.]

But, hey, if you want to throw a bunch of number spaghetti on the wall and have deep discussions on Bayesian probability models, etc, then go knock yourself out.

[Now, I could also tell you that cancer is a metabolic disease, a mitochondrial damage problem, NOT a nuclear genetic disease, which basically means that virtually all of the billions of $$$ flooding the cancer treatment market are WRONG. But you would simply laugh, saying, what does a nobody like you know that they don't? Well, you don't have to be the smartest person in the room to know this just like you don't for trading...you simply LISTEN TO PEOPLE WHO FOLLOW THE CORRECT EVIDENCE, NOT THOSE BOUNDED BY THE DOGMA.]

Seems like a good place to stop posting on ET. I've said my piece.

 

[fan27]

I've been doing some some relatively complex conditional probability hypothesis testing and one of the permutations turned out to be too ridiculous to ignore (i.e., I can predict foo given x,y,z to 90% accuracy - obviously bullshit)

What I want to do is quickly test whether this is potentially profitable without worrying too much about specific strategy.

The approach I am considering is to enter at bar X, and exit at the "best" high/low simply to see the potential max profit - I know this is forward bias.

Is there another method you guys and gals use to test probabilities in backtests?

Basically, there are two components to a strategy....entry and exit. When constructing a strategy, I first focus on the entry. Here is what I do:

1. Test the strategy entry by using a stop and limit order that are equal distance from the entry price. The stop and limit are a multiplier of the ATR (i.e. stop = entry price - (ATR * 4)), limit = entry price + (ATR * 4)).

2. Let's say the result of the back test is 250 trades with a win rate of 70%. Next, I will run a simulation over the same data set where I will randomly select entry points with each run consisting of 250 trades. This will be performed 1000 times and I will see how many of the runs beat a 70% win rate. I will usually select strategies that beat random entry 95% of the time.

3. Next, optimize the exit criteria.

 

[userque]

I've been doing some some relatively complex conditional probability hypothesis testing and one of the permutations turned out to be too ridiculous to ignore (i.e., I can predict foo given x,y,z to 90% accuracy - obviously bullshit)

What I want to do is quickly test whether this is potentially profitable without worrying too much about specific strategy.

The approach I am considering is to enter at bar X, and exit at the "best" high/low simply to see the potential max profit - I know this is forward bias.

Is there another method you guys and gals use to test probabilities in backtests?

Not sure I understand you fully.

But I've tested with a simple 'exit long when LAST < prior bar LOW' ... and vice versa, of course.

 

[MarkBrown]

If you can guess at least 40% of the time that you are in a trend in real-time IN THE PRESENCE OF GOOD VOLATILITY then the entry is trivial and you will be a "winning trader" in the long term. How? Prove it to yourself. Look at the reward to risk ratios of taking opposing bar closes as entries in trends on the price-favored side of 9 wmas and/or 20 emas. [No, really, look over 1000's of charts. It's there...the obviousness is overwhelming.]

it's identifying the trend or rather the extraordinary momentum as i like to call it. by the time most do identify the trend it's over and about to chop you to death. do you have a suggestion for a quick way to get on the trend before it ends?

btw for those not paying attention @SteveH made a brilliant post about how to survive trading. it deserves study.

 

[Sig]

I can tell you this...In index futures, all bets are off.

Backtests would have proven for decades that index future A would not fall a certain number of points in a given timeframe. That shit has all been shattered this year. This means that since the beginning of time, when early in Feb 2018 the Dow fell for the most number of points in history in a single day, your backtests would not have prepared you for it, your probabilities would have been eradicated, and your system would fail.

This was exacerbated when it happened in the same fashion, but albeit in more slo-mo. From Oct to now. There are some folks out there who will decry that this happened in 1987, but I counter with, "Yes, the market fell by huge percentages, nothing like what is happening today". But in my trading world, it is the point count that matters, not the percentages.

Now if you design your backtests to provide for the probability that the market will fall another 1000 points from where it is now in a month, and how that correlates to previous market downdrafts, you may have something.

But here we are again, with percentages vs. points. Back during the major crashes, the market moved, say, 50% from previous peak. Your system may have found that manageable with the money you had in the account.

If the markets were to fall 50% today? You'd probably need 20-30 times the amount of cash in your account to survive.

Just do the math on point values at this time, work it into your backtesting, and come up with a way to hedge against these nasty swans that keep shitting in the pond of finance.

To summarize (and agree), the max loss and gain are as relevant or more relevant then the percentage of accurate predictions. If you have a bet where you stand to gain 1 or lose 100, being able to predict with 95% accuracy is obviously worthless. Witness anyone who sells naked options, does a "carry trade" or plays the vix forward curve.