Oh no, not another python backtester...

wind_ · Jan 7, 2018

globalarbtrader said:
Very late in the day but this seems to be fixed.

GAT

It doesn't seem to be fixed on the latest version.

Code:

C:\pysystemtrade>git log
commit 95ce276859bfe6cf6c5220e052af708b2667976b
Author: rob@systematicmoney.org <rob@qoppac.com>
Date:   Thu Jan 4 11:01:40 2018 +0000

    version 16.1

The P&L for the next day after exchange holiday is still zero.

Code:

>>> system.data.daily_prices("GBP").tail(10)
2017-12-25       NaN   <--- exchange holiday
2017-12-26    1.3431
2017-12-27    1.3440
2017-12-28    1.3479
2017-12-29    1.3557
2018-01-01       NaN   <--- exchange holiday
2018-01-02    1.3628
2018-01-03    1.3547
2018-01-04    1.3586
2018-01-05    1.3597
Freq: B, Name: price, dtype: float64
>>> system.accounts.get_notional_position("GBP").tail(10)
2017-12-25    44.949005
2017-12-26    44.469445
2017-12-27    46.485309
2017-12-28    48.276678
2017-12-29    48.615382
2018-01-01    48.615389
2018-01-02    51.620324
2018-01-03    49.690158
2018-01-04    52.426681
2018-01-05    56.820880
Freq: B, dtype: float64
>>> system.accounts.pandl_for_instrument("GBP").tail(10)
Calculating pandl for instrument for GBP
Calculating buffered positions
2017-12-25       0.000000   <--- exchange holiday
2017-12-26       0.000000   <--- still zero
2017-12-27     308.806873
2017-12-28    1328.743155
2017-12-29    2992.167840
2018-01-01       0.000000   <--- exchange holiday
2018-01-02       0.000000   <--- still zero
2018-01-03   -3095.374570
2018-01-04    1488.347216
2018-01-05     410.335081

traider · Jan 9, 2018

Does anyone know if PRICE column of data in CRUDE_W_carrydata.csv refers to settlement price or last price? I'm trying to extend the files already provided.
Thanks!

DATETIME,PRICE,CARRY,CARRY_CONTRACT,PRICE_CONTRACT
1987-11-25,18.12,,198811,198812

globalarbtrader · Jan 10, 2018

wind_ said:

It doesn't seem to be fixed on the latest version.

Code:

C:\pysystemtrade>git log
commit 95ce276859bfe6cf6c5220e052af708b2667976b
Author: rob@systematicmoney.org <rob@qoppac.com>
Date:   Thu Jan 4 11:01:40 2018 +0000

    version 16.1

The P&L for the next day after exchange holiday is still zero.

Code:

>>> system.data.daily_prices("GBP").tail(10)
2017-12-25       NaN   <--- exchange holiday
2017-12-26    1.3431
2017-12-27    1.3440
2017-12-28    1.3479
2017-12-29    1.3557
2018-01-01       NaN   <--- exchange holiday
2018-01-02    1.3628
2018-01-03    1.3547
2018-01-04    1.3586
2018-01-05    1.3597
Freq: B, Name: price, dtype: float64
>>> system.accounts.get_notional_position("GBP").tail(10)
2017-12-25    44.949005
2017-12-26    44.469445
2017-12-27    46.485309
2017-12-28    48.276678
2017-12-29    48.615382
2018-01-01    48.615389
2018-01-02    51.620324
2018-01-03    49.690158
2018-01-04    52.426681
2018-01-05    56.820880
Freq: B, dtype: float64
>>> system.accounts.pandl_for_instrument("GBP").tail(10)
Calculating pandl for instrument for GBP
Calculating buffered positions
2017-12-25       0.000000   <--- exchange holiday
2017-12-26       0.000000   <--- still zero
2017-12-27     308.806873
2017-12-28    1328.743155
2017-12-29    2992.167840
2018-01-01       0.000000   <--- exchange holiday
2018-01-02       0.000000   <--- still zero
2018-01-03   -3095.374570
2018-01-04    1488.347216
2018-01-05     410.335081

Try updating now

GAT

traider · Jan 10, 2018

globalarbtrader said:
Try updating now

GAT

Hey GAT,

trying to extend your csv data files, may I know if PRICE column of data in CRUDE_W_carrydata.csv (also all the futures data file) refers to settlement price or last price? I'm trying to extend the files already provided.
Thanks!

DATETIME,PRICE,CARRY,CARRY_CONTRACT,PRICE_CONTRACT
1987-11-25,18.12,,198811,198812

globalarbtrader · Jan 10, 2018

traider said:
Hey GAT,

trying to extend your csv data files, may I know if PRICE column of data in CRUDE_W_carrydata.csv (also all the futures data file) refers to settlement price or last price? I'm trying to extend the files already provided.
Thanks!

DATETIME,PRICE,CARRY,CARRY_CONTRACT,PRICE_CONTRACT
1987-11-25,18.12,,198811,198812

It's the price IB provide with the historical data function set to get daily prices. I think it's settlement, but not sure.

GAT

HobbyTrading · Jan 10, 2018

globalarbtrader said:
It's the price IB provide with the historical data function set to get daily prices. I think it's settlement, but not sure.

GAT

I think that IB only provides OHLC bars, so it would be the close price if that is what you filter out. I'm not sure whether IB even provides settlement prices.

traider · Jan 10, 2018

HobbyTrading said:
I think that IB only provides OHLC bars, so it would be the close price if that is what you filter out. I'm not sure whether IB even provides settlement prices.

Do you know if the futures all close at the same time?
Does it depend on exchange ?
Does it depend on instrument?

HobbyTrading · Jan 10, 2018

I numbered your questions for simplicity:

traider said:
1. Do you know if the futures all close at the same time?
2. Does it depend on exchange ?
3. Does it depend on instrument?

1. Not all futures close at the same time.
2. Yes, it does depend on the exchange.
3. Yes, it does depend on instrument.

Also, if IB is your source of data and you use daily OLHC bars, you should be careful what you request: either the full day (which might be almost 24 hours, depending on instrument). Or a time period which IB calls "liquid trading hours" or "regular trading hours". These time periods are defined for certain instruments, and depend per instrument.

djames · Jan 11, 2018

On a daily price level my research indicated that it is irrelevant whether we use close or settlement price.

I recreated the provided csv files using Quandl sourced data (sometimes stitching GATs as he has more history for some instruments than that which Quandl provide) - the backtests were comparable. But don't take my word for it!

HobbyTrading · Jan 11, 2018

djames said:
On a daily price level my research indicated that it is irrelevant whether we use close or settlement price.

I recreated the provided csv files using Quandl sourced data (sometimes stitching GATs as he has more history for some instruments than that which Quandl provide) - the backtests were comparable. But don't take my word for it!

My experience is that you have to be careful with data obtained from Quandl. In some cases do they provide open-high-low-close data, whereas in other cases they provide open-high-low-settle data.
The difference between using settlement prices versus daily close prices might be not large for a system which reviews once per day. But if your system works in a different way (e.g. once per hour) it might have an influence.