different history data for futures

[stasbz]

i look at different sources
http://pitrading.com/free_market_data.htm
http://www.futurespros.com/softs/us-coffee-c-historical-data
http://www.tradingblox.com/tradingblox/free-historical-data.htm
in all of that sources data is different. i check some days of coffee. Why?
what data is reliable?
where to get data?

 

[makemoney007]

If your looking for useful information on different furtes I highly suggest going to this link.

http://single-stockfutures.com/?page_id=11

Good managers will reap even greater rewards from their efforts, resulting in increased productivity, lower costs and new innovations.

 

[Paul Rose]

I don't know the definitive answer to your question, but long term historical futures can be fun.

For TA, people want to see a "continuous" contract even though such a thing doesn't exist.

To generate the illusion, the data is pieced together from whatever contract was in front on a particular date. However, this leaves discontinuities (gaps) on the maturity dates.

There are lots of ways to make the data look more continuous (and I've seen three):
1) do nothing, leave the gaps
2) translation -- leave the newest contract alone, add(subtract) a value to the previous contract to get rid of the gap
3) scaling -- leave the newest contract alone, but multiply(divide) the previous contract by a constant to get rid of the gap

Also, the rollover in the history may be based exclusively on maturity date or by paying attention to when the next contract's volume or open-interest exceeds that of the current contract.

Those three data sources you listed may have rolled up the data using different version of the above (or something else altogether), or maybe it is something else altogether.

I used to write the continuous contract history simulation for a (now obsolete) market data terminal many years ago. I don't have first hand knowledge of the sources you listed, but I have seen how the sausage comes out of the grinder (so to speak).

 

[Murray Ruggiero]

One issue is how you roll over. How you decided when. Many software vendors use volume or volume and open interest. This can create issues, both rolls forward and back again , which can occur or sometimes this volume change happens after first day of notice. This can happen in markets like crude.

I like rolling on fixed dates relative to expiration. Believe it or not how you pick the rolls can effect profits by even 25%. Using a set of dates, in my mind is the best way to do this. This is how Pinnacle Data does it.

 

[mizhael]

Quote from Murray Ruggiero:

One issue is how you roll over. How you decided when. Many software vendors use volume or volume and open interest. This can create issues, both rolls forward and back again , which can occur or sometimes this volume change happens after first day of notice. This can happen in markets like crude.

I like rolling on fixed dates relative to expiration. Believe it or not how you pick the rolls can effect profits by even 25%. Using a set of dates, in my mind is the best way to do this. This is how Pinnacle Data does it.

Hi Murray,

Could you elaborate on how the rolls can affect profits by even 25%?

Thanks a lot!

 

[Murray Ruggiero]

Quote from mizhael:

Hi Murray,

Could you elaborate on how the rolls can affect profits by even 25%?

Thanks a lot!

Yes, because when you blend contracts there are really two gaps. The first in price due to carry. We all know that one. The second is in range, so when you roll will effect trades. In real trading using system assist, this becomes an issue. I had two different broker use different rolls and a handful of trades were different. Often times these where big trades.

Try the following

Use simple channel breakout on 30 year bond with a 20 period lenght. Test with Volume and open interest roll over. Then test using the 26th of the month before expiration. You will see a large differerence in your results even using the same data vendor.

 

[Occam]

Quote from Paul Rose:

I don't know the definitive answer to your question, but long term historical futures can be fun.

For TA, people want to see a "continuous" contract even though such a thing doesn't exist.

To generate the illusion, the data is pieced together from whatever contract was in front on a particular date. However, this leaves discontinuities (gaps) on the maturity dates.

There are lots of ways to make the data look more continuous (and I've seen three):
1) do nothing, leave the gaps
2) translation -- leave the newest contract alone, add(subtract) a value to the previous contract to get rid of the gap
3) scaling -- leave the newest contract alone, but multiply(divide) the previous contract by a constant to get rid of the gap

Also, the rollover in the history may be based exclusively on maturity date or by paying attention to when the next contract's volume or open-interest exceeds that of the current contract.

Those three data sources you listed may have rolled up the data using different version of the above (or something else altogether), or maybe it is something else altogether.

I used to write the continuous contract history simulation for a (now obsolete) market data terminal many years ago. I don't have first hand knowledge of the sources you listed, but I have seen how the sausage comes out of the grinder (so to speak).

Interesting post; thanks, Paul -- makes me appreciate the difficulty of this problem.

 

[intradaybill]

Results shoud not vary with ways contracts are rolled. If results vary, you are doing something wrong. Think about it. I agree it is tricky but you should be able to figure it out. These are the basic things one must know before system trading. Most people will jump to learning C# and fancy platforms before understanding the basics. This is what happened to a clueless newbie I new, a really disgusting individual like some you meet here in ET, thank God they are few, who think they know everything. The rolling of futures data cost him about 50K. Yes, he learned the platform but he knew nothing about the basics of trading.

 

[Murray Ruggiero]

Quote from intradaybill:

Results shoud not vary with ways contracts are rolled. If results vary, you are doing something wrong. Think about it. I agree it is tricky but you should be able to figure it out. These are the basic things one must know before system trading. Most people will jump to learning C# and fancy platforms before understanding the basics. This is what happened to a clueless newbie I new, a really disgusting individual like some you meet here in ET, thank God they are few, who think they know everything. The rolling of futures data cost him about 50K. Yes, he learned the platform but he knew nothing about the basics of trading.

I disagree with you and I have been in this business a long time and have had many systems activity traded. Try the experiment I said with bonds and you will see. The key is to understand how you should roll. I use fixed dates and have had good luck and none of the problems volume / open interest based rolling can cause.

 

[intradaybill]

Quote from Murray Ruggiero:

I disagree with you and I have been in this business a long time and have had many systems activity traded. Try the experiment I said with bonds and you will see. The key is to understand how you should roll. I use fixed dates and have had good luck and none of the problems volume / open interest based rolling can cause.

Dear Dr. Ruggiero, I respect you for all the work you have done, but the time someone has been in business means nothing to me. I know people who believe in proven false paradigms all their life. People believed in empirically falsified concepts for thousands of years. A notable example is the idea that the speed of a free-falling object depends on its position, the famous speed-distance law Galileo disproved. Yet, that was not enough to convince some people, they continue affirming their belifs without justification, the well-know problem of epistemology.

Now, given the above, I repeat: if your system trades futures and the results depend on the roll-date or method, you are doing something wrong.

Instead of asking me, what is it that I think people do wrong, you responded with an informal fallacy of an appeal to "have been in business for a long time". As a result, the conversation stops here and there is no reason for me to reveal for free to you or to anyone lese what has cost me time and money to learn, while you appear to know everything.

Thank you for your understanding.