Fully automated futures trading

[AvantGarde]

I'm not sure what you mean by "mean square error" so I'm going to guess.

If returns are uncorrelated and normally distributed then the standard deviation will always scale with the square root of time. So the daily standard deviation will be roughly 1/16 of the annual standard deviation.

The sampling error of the mean (or with certain assumptions the Sharpe Ratio) of a return series (like trading system performance) is equal to the standard deviation divided by the square root of a number of observations. A t-test for positive returns at some frequency is equal to the mean divided by the sampling error of the mean.

For example: Daily returns; average 0.01%. Standard deviation 0.4%
Analogous annual returns: 0.01% * 256 = 2.56% Standard deviation * 16 = 6.4%.

With 16 years of history the annual sampling error on the mean is 6.4 / sqrt(16) = 1.6%
A t-test for positive returns comes out at 2.56% / 1.6% = 1.6 (not significant, significance is normally 2)

With 16*256 = 4096 days of history the daily sampling error on the mean is 0.4% / sqrt(4096) = 0.00625%
A t-test for positive returns comes out at 0.01 / 0.00625 = 1.6 (same value as before)

(Both the numerator and denominator of the t-test have square root of time in them. So changing time has no effect)

The implication of this is that unless the system has a relatively high Sharpe Ratio, and trades relatively quickly, you can't get a narrower confidence interval*. To put it another way only getting a LONGER HISTORY of data will improve your confidence in your backtest, getting MORE FREQUENT returns won't help at all.

(Non Gaussian returns changes this slightly - in particular daily returns give you more chance of seeing the ugly drops that characterise negative skew. Correlated returns also changes this slightly - if they are negatively correlated using annual returns will flatter performance)

GAT

* With a high enough Sharpe Ratio just one year or a few months of data is enough to get a significant t-test using daily returns.

This is precisely what I meant. Thanks for the explanation

 

[globalarbtrader]

This is precisely what I meant. Thanks for the explanation

My mind reading skills are pretty good, except with my wife.

GAT

 

[JMW]

Here's some things I learnt for anyone else trying to implement this with Python:

• Working with data on one machine? Try HDF5 in tables mode.
• Working with data across many machines? Try Arctic (AHL's ticker plant).
• Try concurrent.futures.ProcessPoolExecutor().map() for backtests. My first implementation took 10 minutes to backtest 45 instruments. Now it takes 20 seconds.
• Vectorize everything.
• Mongo is much better than SQL in every way when combined with Pandas. I no longer use SQL at all.
• Docker is a good idea, especially for deploying.
• Digital Ocean is much easier than AWS etc.
• pytest is good
• Want to buy a computer for backtesting? I bought an 8 core Xeon w/ 32GB RAM on ebay for £200. It must have cost thousands just a couple of years ago.

Hi Isotope1 - I am intrigued by your improvement of backtest from 10 mins to 20 seconds. How did you use the ProcessPollExecutor to achieve this (what were you putting in concurrent threads?) Also what do you mean when you say "vectorize everything", and did that also contribute to your performance gains.

 

[Elder]

Hi GAT, on page 18 of this thread, you have helpfully written down some formulas showing how to estimate turnover and I was wondering if there might be a small typo. Should there be an ‘N’ term in the denominator in the last couple of expressions? Otherwise the result seems to have unbounded growth with sample size.

 

[globalarbtrader]

Hi GAT, on page 18 of this thread, you have helpfully written down some formulas showing how to estimate turnover and I was wondering if there might be a small typo. Should there be an ‘N’ term in the denominator in the last couple of expressions? Otherwise the result seems to have unbounded growth with sample size.

er probably... could you copy and paste the formulas to save me looking

GAT

 

[Elder]

Hi

Turnover is basically trades / 2 x absolute average position

From page 186 of the print edition (chapter 12 'Estimating the number of round trips'):

Turnover=
Average number of blocks traded per year / (2 x average absolute number of blocks held)

So suppose your block is one futures contract and you trade daily. If your series of positions is p0, p1, ... pt
and you have N years of trading history then turnover=

(1/N)*(abs(p1-p0)+ans(p2-p1)+ ... ans(pt - pt-1)) / (2 x mean(abs(p0), abs(p1), abs(p2), ....)

I can also write this more conveniently as 250 (# of business days in a year) times the average daily turnover:

(250)*(mean(abs(p1-p0)+ans(p2-p1)+ ... ans(pt - pt-1)) / (2 x mean(abs(p0), abs(p1), abs(p2), ....)

Now for long backtests this might not be too accurate, especially if volatility has changed a lot over time. I can also rewrite the above as

(250)* ([abs(p1-p0)/ (2 x P0)] + [abs(p2-p1)/ (2 x P1)] + ....[abs(pt-pt-1)/ (2 x Pt-1)]

Where P is a smooth moving average of absolute position

(technically this is not exactly equal to the previous expression, but takes a moving average rather than one over the whole backtest)

Or I could also write:

(250)* ([abs(p1-p0)/ 2 x A0] + [abs(p2-p1)/ 2 x A1] + ....[abs(pt-pt-1)/ 2 x At-1]

Where A0... is the position I would get with a forecast of +10 given the vol etc at that time (the 'no rule' rule position).

[Recall that any position = instr. weight * IDM * forecast * daily cash vol target / ( 10 * price vol per day * block value * fx rate )

And for A0.... At-1 the forecast is +10]

This is probably the best estimator, . Notice that if we're measuring the turnover of a FORECAST, rather than a final position, this becomes: (for forecast f1.... ft)

(250)* ([abs(f1-f0)/ 2 x 10] + [abs(f2-f1)/ 2 x 10] + ....[abs(ft-ft-1)/ 2 x 10]

Hope that makes sense
GAT

Sorry of course - here you go

 

[globalarbtrader]

Sorry of course - here you go

You're right all these are missing a 1/n term:

(250)* ([abs(p1-p0)/ (2 x P0)] + [abs(p2-p1)/ (2 x P1)] + ....[abs(pt-pt-1)/ (2 x Pt-1)]

(250)* ([abs(p1-p0)/ 2 x A0] + [abs(p2-p1)/ 2 x A1] + ....[abs(pt-pt-1)/ 2 x At-1]

(250)* ([abs(f1-f0)/ 2 x 10] + [abs(f2-f1)/ 2 x 10] + ....[abs(ft-ft-1)/ 2 x 10]

GAT

 

[Elder]

You're right all these are missing a 1/n term:

(250)* ([abs(p1-p0)/ (2 x P0)] + [abs(p2-p1)/ (2 x P1)] + ....[abs(pt-pt-1)/ (2 x Pt-1)]

(250)* ([abs(p1-p0)/ 2 x A0] + [abs(p2-p1)/ 2 x A1] + ....[abs(pt-pt-1)/ 2 x At-1]

(250)* ([abs(f1-f0)/ 2 x 10] + [abs(f2-f1)/ 2 x 10] + ....[abs(ft-ft-1)/ 2 x 10]

GAT

Thanks, also had this minor conundrum I can't resolve. If 90% of your system is profitable and your risk capital is going up (assuming you are below your fixed capital amount so kelly scaling applies), the rising risk capital will cause you to add to positions for the profitable part. However, for 10% that is losing money you will have the opposing effects of increasing capital and falling abs forecasts so in principle it is possible to be increasing positions to poorly performing subsystems. Do you see any value in tinkering with this behaviour and having some sort of override?

 

[globalarbtrader]

Thanks, also had this minor conundrum I can't resolve. If 90% of your system is profitable and your risk capital is going up (assuming you are below your fixed capital amount so kelly scaling applies), the rising risk capital will cause you to add to positions for the profitable part. However, for 10% that is losing money you will have the opposing effects of increasing capital and falling abs forecasts so in principle it is possible to be increasing positions to poorly performing subsystems. Do you see any value in tinkering with this behaviour and having some sort of override?

Absolutely not. You're confusing the various reasons why you should change risk capital. We're not reducing / increasing risk capital because the system is doing well / performing poorly.

There are a few different reasons why you'd reduce positions:

A- higher vol
B- because a particular trade is going against you
C- because you don't believe in a particular trading subsystem anymore
D- because you have less money to play with (kelly criteria)

These are all independent. And C&D are exogenous to the system.

So for example in a trend following system covering multiple assets you'd be scaling positions by vol (A), with the trading rule automatically reducing positions when the trend turns against you (B). Of course in a non trend following system I'd strongly advocate including some kind of stop loss to perform the function of B.

To justify C you need strong statistical evidence that the subsystem is no longer working. Evidence you're unlikely to get given the sort of time frames and Sharpe ratios that this thread is dealing with (this isn't HFT, double digit SR, land). Changes because of C should be implemented by changing instrument weights (or forecast weights if it's a particular trading rule that bothers you) - ideally in a robust out of sample way.

D affects the entire system. It may also be affected by other factors - for example I could decide to add more money to my account. Or change my risk target. I might reduce the money I had in my account if I was worried about the performance of the entire system collectively; but again ideally that would only be based on significant evidence I'm unlikely to have in practice.

GAT

 

[Elder]

Thanks GAT. Let's say you don't meddle with the risk capital your system is telling you to use. Let's say you start the trading year with a loss, and have therefore reduced your risk capital in proportion. But then you start to make money, and the +ve P&L increases the available risk capital again (until you reach your max). This would cause you scale up your positions across the board. But there is still a section of your portfolio (10%) where the trend is going against your positions yet the rise in capital would force an increase in all your positions. Rare event but if it occurs it seemed counter-intuitive to add to a section of the portfolio where forecasts might be falling. It seems what you are saying is that you should go with whatever your system tells you because this is still the best diversified portfolio?