Actually I just realized, normally to calculate portfolio volatility we also need volatilities of each instrument, but in your optimization you're not using them:
solution_gap = weights - self.weights_optimal_as_np
track_error = (solution_gap.dot(self.covariance_matrix_as_np).dot(solution_gap))**.5
- this code has only weights and correlations, I guess it's because we only need the "magnitude of the result" for comparing portfolios., and individual-instrument volatilities will be the same in all portfolios, so they'll cancel out and can be omitted?
Some technical thoughts:
- I will probably limit the number of the outer-loop iterations to some constant and if the loop didn't exit naturally by finding a local minima, I'll compare gap-volatility of the best-found portfolio so far with the current system's positions and return the smallest + generate a warning. This constant can be pretty-small I think, because on every outer-iteration the search will add at least one contract, and the total reasonable number of contracts I hold at any time wouldn't be large, so something like '300' should probably be plenty. But I'll also log the number of iterations it normally takes in the paper system to get a better sense..
- Also, the inner loop, where it tries to add one more contract of each instrument, can be done in parallel (evaluating every portfolio is independent) to utilize multiple CPU cores..
On a more philosophical note, I'm not sure I internally understand why "minimizing the substracted volatility" makes the most similar portfolio.. I mean obviously it works empirically, and also volatility of things is the most predictable measure in finance so it makes sense to use it, but we're really not looking at anything else to determine which portfolios are the most similar to the target (well, we look at correlations too, but as a part of identifying volatility of the whole portfolio)..
No! Covariance is both correlation AND standard deviation! We absolutely need to use the standard
You're unlikely to have problems with the outer loop size, unless you have 100 instruments and 100 million bucks (in which case, why bother with dynamic optimisation?), having less capital usually reduces the solution set to manageable proportions. Also doing this means you are guaranteed to have less risk than you want if you hit the constraint.
The parallelism on the inner loop is a good idea in principle.
In terms of intuition correlation is actually key here. Standard deviations just scale everything.
Think about a simple 3 body problem with a zero optimal stock position (say SP500 with 16% vol), and two bonds, one with a zero optimal position but small standard deviation (say US10 with 8% vol), one with a positive optimal position but much less than 1 contract (say US30 with optimal position 0.2 and vol 16%) and similar standard deviation to stocks.
Assume zero stock/bond correlation and positive intrabond correlation of say 0.9
To get the risk it wants (long US30) we'll try and go long the US2 and not long SP500. Why? Consider the two candidate portfolios:
A: Long 1 contract SP500
B: Long 2 contracts US10
These produce the following gap portfolios:
A: Long 1 sp500, short 0.8 US30
B: Long2 US10, short 0.8 US30
Because US2 and US30 are positively correlated the variance of portfolio B will be much lower than that of portfolio A. We can show that all other candidate portfolios are inferior to B. Try plugging the numbers into an online portfolio optimisation tool and confirm this for yourself.
In fact if the vol ratio between US10 and US30 was exactly 2/0.8 and the correlation was 1.0 portfolio B would have zero risk!
I hope that helps.
GAT
