Depends on both the Sharpe and the number of trades, no? An HFT guy with a Sharpe of 12 and 2 million trades over a month would be pretty reliable
Fully automated futures trading
- Thread starter globalarbtrader
- Start date
He knows the system I am trading (he laid the foundations for it), so he knows that his statement is correct.Depends on both the Sharpe and the number of trades, no? An HFT guy with a Sharpe of 12 and 2 million trades over a month would be pretty reliable
Edit: I see now these graphs are really large, I'll post smaller going forward, or use thumbnails.
Price stitching is hard
I had some mismatches in the 2 processes I use (moving to a new process now that I got some price data going back in history) so I wanted to see what Rob's prices look like from pysystemtrade repo, to break the tie. First graph above is VSTOXX.
All is good in the land of bonds tho, not many ways to mess it up
But then I stumbled upon Lean Hogs:
Doing some more digging, Rob, I think there might be something wonky happening with your price data in pysystemtrade with lean hogs. Prior to 2014, there are some weird rolling schedules, for example, here's an entry from multiple prices CSV:
Seems like you're using July 2013 for carry, but that would have been long expired. I checked your roll calendar config for this, and looks like for a bunch of years prior to 2014, you were rolling only twice a year, in April and May.
Not sure if that's intentional or not, just wanted to give you a heads up.
Besides this, I checked 11 other instruments over a period from 1/1/2001 until today, and our adjusted prices match closely. Some deviation is expected of course, as we don't roll on exact same days.
Price stitching is hard
I had some mismatches in the 2 processes I use (moving to a new process now that I got some price data going back in history) so I wanted to see what Rob's prices look like from pysystemtrade repo, to break the tie. First graph above is VSTOXX.
All is good in the land of bonds tho, not many ways to mess it up
But then I stumbled upon Lean Hogs:
Doing some more digging, Rob, I think there might be something wonky happening with your price data in pysystemtrade with lean hogs. Prior to 2014, there are some weird rolling schedules, for example, here's an entry from multiple prices CSV:
Code:
2014-02-24 15:51:57,,20140600,,20130700,107.7375,20140700Seems like you're using July 2013 for carry, but that would have been long expired. I checked your roll calendar config for this, and looks like for a bunch of years prior to 2014, you were rolling only twice a year, in April and May.
Not sure if that's intentional or not, just wanted to give you a heads up.
Besides this, I checked 11 other instruments over a period from 1/1/2001 until today, and our adjusted prices match closely. Some deviation is expected of course, as we don't roll on exact same days.
Edit: I see now these graphs are really large, I'll post smaller going forward, or use thumbnails.
Price stitching is hard
Let me look into this and get back to you. It is hard.... especially fully automated!
GAT
I had some mismatches in the 2 processes I use (moving to a new process now that I got some price data going back in history) so I wanted to see what Rob's prices look like from pysystemtrade repo, to break the tie. First graph above is VSTOXX.
All is good in the land of bonds tho, not many ways to mess it up
View attachment 236127
But then I stumbled upon Lean Hogs:
View attachment 236128
Doing some more digging, Rob, I think there might be something wonky happening with your price data in pysystemtrade with lean hogs. Prior to 2014, there are some weird rolling schedules, for example, here's an entry from multiple prices CSV:
Code:2014-02-24 15:51:57,,20140600,,20130700,107.7375,20140700
Seems like you're using July 2013 for carry, but that would have been long expired. I checked your roll calendar config for this, and looks like for a bunch of years prior to 2014, you were rolling only twice a year, in April and May.
Not sure if that's intentional or not, just wanted to give you a heads up.
Besides this, I checked 11 other instruments over a period from 1/1/2001 until today, and our adjusted prices match closely. Some deviation is expected of course, as we don't roll on exact same days.
Edit: I see now these graphs are really large, I'll post smaller going forward, or use thumbnails.
Price stitching is hard
I had some mismatches in the 2 processes I use (moving to a new process now that I got some price data going back in history) so I wanted to see what Rob's prices look like from pysystemtrade repo, to break the tie. First graph above is VSTOXX.
All is good in the land of bonds tho, not many ways to mess it up
View attachment 236127
But then I stumbled upon Lean Hogs:
View attachment 236128
Doing some more digging, Rob, I think there might be something wonky happening with your price data in pysystemtrade with lean hogs. Prior to 2014, there are some weird rolling schedules, for example, here's an entry from multiple prices CSV:
Code:2014-02-24 15:51:57,,20140600,,20130700,107.7375,20140700
Seems like you're using July 2013 for carry, but that would have been long expired. I checked your roll calendar config for this, and looks like for a bunch of years prior to 2014, you were rolling only twice a year, in April and May.
Not sure if that's intentional or not, just wanted to give you a heads up.
Besides this, I checked 11 other instruments over a period from 1/1/2001 until today, and our adjusted prices match closely. Some deviation is expected of course, as we don't roll on exact same days.
I've added this to github as an issue since I'm not sure when I will get round to it, and this way I won't forget about it
https://github.com/robcarver17/pysystemtrade/issues/213
GAT
I have 2 somewhat unrelated questions.
1) What is the authoritative source for contract specs like expiry dates, for long expired contracts? I'm currently interested in some Platinum contracts from 2003-2004. I'm aware of the Contract Specs page on CME (https://www.cmegroup.com/trading/metals/precious/platinum_contract_specifications.html) but that only has stuff for current contract. Expiration calendar (https://www.cmegroup.com/tools-information/calendars/expiration-calendar/) only goes back to 2013. Does anybody know of a better source, ideally free?
2) Which prices do you use for what when backtesting? For example, I've been using adjusted prices for everything, except computing the carry rule - that also means I used it to compute risk too, which I'm now thinking is not really the way to go, as the back-adjusted prices can drift a lot the further back in history we go, while the returns stay the same, so the risk would be inflated or understated. I also use adjusted prices for computing trade profits which made sense as we're interested in magnitude of the move, and not the absolute price, but am now concerned that I'm not representing the rolls correctly. Do you account for rolls in your backtests, like trading costs and the spread between contracts?
Thanks!
1) What is the authoritative source for contract specs like expiry dates, for long expired contracts? I'm currently interested in some Platinum contracts from 2003-2004. I'm aware of the Contract Specs page on CME (https://www.cmegroup.com/trading/metals/precious/platinum_contract_specifications.html) but that only has stuff for current contract. Expiration calendar (https://www.cmegroup.com/tools-information/calendars/expiration-calendar/) only goes back to 2013. Does anybody know of a better source, ideally free?
2) Which prices do you use for what when backtesting? For example, I've been using adjusted prices for everything, except computing the carry rule - that also means I used it to compute risk too, which I'm now thinking is not really the way to go, as the back-adjusted prices can drift a lot the further back in history we go, while the returns stay the same, so the risk would be inflated or understated. I also use adjusted prices for computing trade profits which made sense as we're interested in magnitude of the move, and not the absolute price, but am now concerned that I'm not representing the rolls correctly. Do you account for rolls in your backtests, like trading costs and the spread between contracts?
Thanks!
I have 2 somewhat unrelated questions.
1) What is the authoritative source for contract specs like expiry dates, for long expired contracts? I'm currently interested in some Platinum contracts from 2003-2004. I'm aware of the Contract Specs page on CME (https://www.cmegroup.com/trading/metals/precious/platinum_contract_specifications.html) but that only has stuff for current contract. Expiration calendar (https://www.cmegroup.com/tools-information/calendars/expiration-calendar/) only goes back to 2013. Does anybody know of a better source, ideally free?
2) Which prices do you use for what when backtesting? For example, I've been using adjusted prices for everything, except computing the carry rule - that also means I used it to compute risk too, which I'm now thinking is not really the way to go, as the back-adjusted prices can drift a lot the further back in history we go, while the returns stay the same, so the risk would be inflated or understated. I also use adjusted prices for computing trade profits which made sense as we're interested in magnitude of the move, and not the absolute price, but am now concerned that I'm not representing the rolls correctly. Do you account for rolls in your backtests, like trading costs and the spread between contracts?
Thanks!
You can get several years back of older contracts from IB itself (you can actually add them to the watch-list in TWS and see details), but that doesn't go very far back.
In my system I pre-generated roll dates (historical and future ~ 100 years total) simply using the rules described in the contract specs on CME web-site or other corresponding exchanges, like "every third Friday before the last Monday prior to the next Tuesday"
, I used an open-source library that knows all historical public holidays to adjust for them. I then generated SQL insert-scripts for all the contracts I need using these rules. The result isn't perfect of course, but I only need to know these dates approximately as I usually have at least several days to roll, so +- couple of days should be a good enough precision..I guess Quandl should have that data as well, but never tried that..
As for which prices to use, when I run backtest through my system, I'm using it exactly the same way as it would've been used in reality, so I'm pumping historical EOD prices as if they were real-time, and the system stitches them internally as in normally would (I don't even store stitched prices anywhere, they're always caclulated on the fly and cached in memory of the running system).
When I use PySystemTrade to backtest, that's a different story, I guess it's using stitched prices internally? frankly I didn't dig into it
Last edited:
Hi @globalarbtrader , been listening to the Global Macro podcast series (https://www.toptradersunplugged.com/category/global-macro/) which you're co-hosting.. I liked it, it's a good listen. Although, it's very much different from the whole systematic approach we're mostly discussing here.. But I guess in trading in general anything goes, and it might also be useful to look at things from the analytical "human-story-telling" perspective.. Not sure, but so far the actionable conclusions I made from the last episodes is that I should probably buy more gold(silver?) in my long-only investment portfolio, nothing crazy, but perhaps move the allocation from ~4% to like 7% of the portfolio.. And maybe sell some US stocks and buy EU\UK stocks instead.. again, nothing drastic, as really nobody knows.. Bond ETFs still somehow pay some dividends (VGLT: 1.98%, VCLT: 3.63%), even though the interest-rates are almost zero almost everywhere.. So probably nothing changed there, one should still have at least 10-20% bond etfs in his investment portfolio..
I mean they keep printing money, inflating the bubble, so stocks become riskier and riskier, but where else can you go (apart from the trend-following system and some real-estate).. Cash will be devalued with a much higher certainty than anything else (2% is just the target-rate, and in reality probably even more with all the newly-printed money).
Anyway, thanks for all the material, and waiting for the new stuff (was intrigued by the previous post about the possibility of dynamic instrument selection based on the signal-strength from a wider group, and possibly some shorter-term mean-reversion trading you mentioned a while ago..)
I mean they keep printing money, inflating the bubble, so stocks become riskier and riskier, but where else can you go (apart from the trend-following system and some real-estate).. Cash will be devalued with a much higher certainty than anything else (2% is just the target-rate
, and in reality probably even more with all the newly-printed money).Anyway, thanks for all the material, and waiting for the new stuff (was intrigued by the previous post about the possibility of dynamic instrument selection based on the signal-strength from a wider group, and possibly some shorter-term mean-reversion trading you mentioned a while ago..)
Yes, which is why if you are using percentage returns for risk you should use the *current priced contract* as the denominator, and return difference of adjusted price as the numerator (this is what happens inside pysystemtrade). If you use return difference of the current contract you will get jumps when the contract rolls, so don't do that. Or you could not use percentage returns at all: the denominator of the percentage return (whatever it is) actually cancels out in the inverse volatility position calculation. I only include percentage returns in my books as that what people understand, especially from an equities background.
I don't actually, but in my experience this isn't a significant part of costs unless you are trading *very* slowly.
GAT
Hi @globalarbtrader , been listening to the Global Macro podcast series (https://www.toptradersunplugged.com/category/global-macro/) which you're co-hosting.. I liked it, it's a good listen. Although, it's very much different from the whole systematic approach we're mostly discussing here.. But I guess in trading in general anything goes, and it might also be useful to look at things from the analytical "human-story-telling" perspective.. Not sure, but so far the actionable conclusions I made from the last episodes is that I should probably buy more gold(silver?) in my long-only investment portfolio, nothing crazy, but perhaps move the allocation from ~4% to like 7% of the portfolio.. And maybe sell some US stocks and buy EU\UK stocks instead.. again, nothing drastic, as really nobody knows.. Bond ETFs still somehow pay some dividends (VGLT: 1.98%, VCLT: 3.63%), even though the interest-rates are almost zero almost everywhere.. So probably nothing changed there, one should still have at least 10-20% bond etfs in his investment portfolio..
I mean they keep printing money, inflating the bubble, so stocks become riskier and riskier, but where else can you go (apart from the trend-following system and some real-estate).. Cash will be devalued with a much higher certainty than anything else (2% is just the target-rate
Anyway, thanks for all the material, and waiting for the new stuff (was intrigued by the previous post about the possibility of dynamic instrument selection based on the signal-strength from a wider group, and possibly some shorter-term mean-reversion trading you mentioned a while ago..)
FWIW I don't plan on using any of the material from those podcasts to make investing decisions, where they might be useful is in thinking about new trading rules, especially cross asset rules.
GAT