20% more profit with weighted price data

[jcl]

Normally a strategy uses historic price data for parameter optimizing - even if it's a parameter-free strategy that calculates them itself. We found that applying a weight slope to the price data in the optimization process can remarkably improve the annual return and the Sharpe ratio.

We're using the following weight formula:

w = 2(1+t(s-1))/(s+1)

where t is the position into the price data in the 0..1 range, and s is the weight slope. At s = 2, the end of the price data set has twice the weight than the begin.

The weight determines the contribution of this part of the price data to the objective function of the optimizer. The theory behind this is that the markets change over time, and thus the last part of the price data affects your trading more than the first part. Giving it a higher weight improves the parameter quality.

We found that an s value between 1.5 and 2 can noticeably increase the Sharpe ratio on unseen data. At too large weight differences we're getting effects that amount to data reduction and overfitting, and the Sharpe ratio goes down again.

This method works with almost all strategies that use bars in the hour to days range. It does not work with very short bars in the minute range, because long term market changes have no effect on those strategies. The Zorro platform has this weight slope function built in, but the method can also be sort of simulated with other platforms. You just have to multiply all trade results with the above formula when optimizing. Note that it would not do to multiply the trade size - the trade result must be multiplied.

Has someone else also experimented with price data weights?

 

[trade4ever2day]

Could you be more specific with a simple example? Let us for the shake of an example say that I have a long only startegy that gives a signal at the close when the high of today is higher than the previous 4 highs. How do you use the weighted price data to get more profit? Wouldn't that cause a violation of entry signals?

 

[bone]

Just curious as to why someone would "weight" raw price data ? I could understand "cleaning" data for legit/relevant price tics and stray artifacts, but in my mind I get paid on price, so I would naturally conclude that I would want to model the real unvarnished truth as it were.

 

[ssrrkk]

Quote from bone:

Just curious as to why someone would "weight" raw price data ? I could understand "cleaning" data for legit/relevant price tics and stray artifacts, but in my mind I get paid on price, so I would naturally conclude that I would want to model the real unvarnished truth as it were.

I think he means weight the importance of the data when using it to train (aka fit) his parameters, i.e., give it more weight in the objective function that he is minimizing to get the best fit of his model on the data.

 

[jcl]

Quote from ssrrkk:

I think he means weight the importance of the data when using it to train (aka fit) his parameters, i.e., give it more weight in the objective function that he is minimizing to get the best fit of his model on the data.

Yes, weight is equivalent to the importance of the data. However, to avoid misunderstandings, it does not get a better fit. In fact weighted data generates in most cases a worse fit than unweighted data, so the result on in-sample data is worse. Only the result on out-of-sample data is better, and only when the out-of-sample data is taken directly after the optimization period. And of course, the result in real trading is better.

 

[jcl]

Quote from alexandermerwe:

Could you be more specific with a simple example? Let us for the shake of an example say that I have a long only startegy that gives a signal at the close when the high of today is higher than the previous 4 highs. How do you use the weighted price data to get more profit? Wouldn't that cause a violation of entry signals?

In your example, your optimizer would probably attempt to optimize the number of previous highs, and the stop loss. I assume that your trade platform allows you to define the optimizier's objective function. In the simplest case when you optimize for the maximum of raw profit, the objective function would look like this:

var objective()
{
   var sum = 0;
   for(TRADE trade = Trades; trade; trade = trade.next)
      sum += trade.profit;
   return sum;
}

For simulating weighted price data with your platform, you would change this function, like this:

var objective()
{
   var sum = 0;
   var slope = 2;
   for(TRADE trade = Trades; trade; trade = trade.next) {
      var t = trade.bar / NumBars;
      var weight = 2*(1+t*(slope-1))/(slope+1);
      sum += weight * trade.profit;
  }
   return sum;
}

I hope this makes the concept more clear? The modified objective function will in almost all cases improve the profit of your optimized strategy in a walk forward test and in real trading. Depending on your trade volume, this can make a difference of several 1000 EUR per month. It can even make a slightly unprofitable strategy profitable.

 

[ssrrkk]

Quote from jcl:

Yes, weight is equivalent to the importance of the data. However, to avoid misunderstandings, it does not get a better fit. In fact weighted data generates in most cases a worse fit than unweighted data, so the result on in-sample data is worse. Only the result on out-of-sample data is better, and only when the out-of-sample data is taken directly after the optimization period. And of course, the result in real trading is better.

Yes but "worse fit" "best fit" depends on your metric, aka objective function. If you are saying that in the RMSE sense or L2 norm, then yes weighting will make it worse. However, min RMSE doesn't necessarily imply optimal fit for all purposes such as when you are trying to optimize profit.

 

[jcl]

Quote from ssrrkk:

Yes but "worse fit" "best fit" depends on your metric, aka objective function. If you are saying that in the RMSE sense or L2 norm, then yes weighting will make it worse. However, min RMSE doesn't necessarily imply optimal fit for all purposes such as when you are trying to optimize profit.

I admit that I had to google RMSE and L2, but I think I know what you mean. When optimizing a trade function, you maximize a derived parameter, usually the profit. At the same time you have to minimize the distance to the same parameter derived from another curve, the out-of-sample data. The weight slope method does not help with maximizing the objective, but it causes the curves to behave "more similar" and thus reduces the distance of the objectives derived from the two curves.