Dilemma in modeling idiosyncratic high-frequency noise

To Tums:
Detrending method:
I'm using a redundant haar wavelet and applying it to tick bar data. The best way to think of this would be to apply a 3rd to 5th order polynomial to 4 minute bar data. After subtracting this trend, I'm left with a residual or "noise".

To Hook N Sinker and trader3cnd:
Fair enough. I don't know that the residual is "noise". For sake of clarity, I'm referring to a residual that is not necessarily noise.

However, if I'm recalling this correctly, I believe that when using a properly specified ARMA model the the expectation is that the residual is a white noise N(0, 1) process. That's what I had in mind as a benchmark.

The residual that I'm observing is definitely not white noise and it's persistent over the 3 year samples that I've used on 4 different instruments.

To jdezeero05,
I'm not using DSP. Though some of the techniques are DSP-like. I've never chkd out nuclear phynance, thx for the suggestion.

Thanks everyone for your ideas,
bluelou
 
BlueLou,

First the signal to noise ratio on ET is very low so you'll need to do your own filtering....

Second, I can only attempt to understand your question at this point; you are saying you are finding that your De-Trended data does not replicate a white-noise series and that you have found a discernible pattern in this series? Are you asking what structurally in the market place causes this?

Also would I be right to assume that your model is a counter-trend/mean-reversion model?
 
Quote from bluelou:

Why does idiosyncratic noise exist in the first place? What causes the idiosyncracies? Is there any academic research on idiosyncratic noise signatures in high frequency data (of any type) that you would recommend? I'm open to your suggestions.

Background:
I use a model based on ideas derived from statistical physics to trade FX and other futures on an intraday basis. The only inputs to the model are price and bar count.

I'm refering to idiosyncratic noise, not signal. That is, I've detrended the data series so that all that is remaining is a noise residual. I believe that I'm finding that FX pairs (and other futures instruments) can have unique and persistent high-frequency noise signatures.
More...

Until all of the interbank feeds are combined to show all the combined trading of the pairs and we are able to apply something other than time bars to the charts . . . the noise will exist.
 
BLB,
I wish! I'm not getting a constant periodicity but by using a few simple threshold value-type rules I've developed a very good trading model. I've completed backtests going back 3 years on 4 instruments so far.

It all looks great so far but that wasn't really the point of my posting. My point was, provided I'm correct in describing what I've observed, what are the potential causes of such a "noise" signature? Is there any existing research out there on the subject? Any ideas?
 
knocks420,
You're right about the ET signal/noise ratio. I guess it just requires some patience.

On your second point... Yes, you're pretty close. But, I wouldn't call it a discernable pattern. Rather, by setting simple thresholds for the residual values, I'm able to make a trading decision.

Yes, I'd like to know what causes this structurally in the marketpalce. What are your thoughts?

No, it's not a counter-trend model. I let the model decide whether to trade trend or ctr-trend. It can trade in either direction inside of the same day. The performance is better trading in the direction of the daily trend, though ctr-trend the P&L is also positive generally.



======

ProfLogic,
I've been using the model to trade FX futures not FX spot and I'm using tick data in all but one case.

Why do you say that the noise will exist since the interbank feeds are not combined? Why would non-white noise exist in this case? Is this something that you've worked with or are you familiar w/any research on the subject?

Are you very familiar with FX data feeds? I've been getting spot FX "tick" data from IQfeed (Tullet and Barclays feeds). I suspect that this is some sort of bid-ask interpolation. Do you know for certain?

Also, I've spoken with HotSpot's Institutional group. They've assured me that since the transactions are done across their platform that they are providing true time & sales tick data. Are you familiar with the Hotspot Institutional tick data feed?
 
Quote from bluelou:

knocks420,
You're right about the ET signal/noise ratio. I guess it just requires some patience.

On your second point... Yes, you're pretty close. But, I wouldn't call it a discernable pattern. Rather, by setting simple thresholds for the residual values, I'm able to make a trading decision.

No, it's not a counter-trend model. I let the model decide whether to trade trend or ctr-trend. It can trade in either direction inside of the same day. The performance is better trading in the direction of the daily trend, though ctr-trend the P&L is also positive generally.

======

ProfLogic,
I've been using the model to trade FX futures not FX spot and I'm using tick data in all but one case.

Why do you say that the noise will exist since the interbank feeds are not combined? Why would non-white noise exist in this case? Is this something that you've worked with or are you familiar w/any research on the subject?

Are you very familiar with FX data feeds? I've been getting spot FX "tick" data from IQfeed (Tullet and Barclays feeds). I suspect that this is some sort of bid-ask interpolation. Do you know for certain?

Also, I've spoken with HotSpot's Institutional group. They've assured me that since the transactions are done across their platform that they are providing true time & sales tick data. Are you familiar with the Hotspot Institutional tick data feed?
More...

There are numerous interbank feeds and no where are all of the spot markets combined. Some of the larger banks combine 3 or four feeds when trading their currency pairs but again not all of them. They each trade market segments not the market as a whole. Every group offering feeds to trade the pairs are no different. You are trading a segment of the market not the market as a whole.

Globex and Reuters were in negotiations at the beginning of 2007 to join all of the feeds so total tick and total volume data could be extracted from the mix but for whatever reason the talks fell through. Hopefully at some point they will pool all of the data into one feed.

The FX feeds are different in that they are traded independently and uniquely.

If Hotspot is supplying you with spot data, they are probably supplying you with true time and sales data but only from the specific feed or feeds they use. It can not be from all of them.

The noise you speak of is created by variable charting. Those are time based, tick based or other non consistent bar charting. It's too long of an explanation to go over here but do a search of Constant Volume Bar Charts here on ET and you will find quite a few explanations. I've been researching this and watching it for over 5 years now.
 
ProfLogic,
Please correct me if I'm wrong in my interpretation of what you just posted:

1) The noise that I'm observing is caused by traders using different data series and different frequencies for the same instrument.

2) This noise is exacerbated in the spot FX market due to the lack of a consolidated data feed.

3) You didn't say this, but my guess is that you'd agree with the following. Activity-based bars such as tick- or volume-based bars are closer to the true data generating process found in high frequency financial time series. The reason for this is that the true process is not i.i.d..

Are 1, 2, and 3 consistent with what you implied? Do you have any research that validates 1) regarding noise observations.

Thx,
bluelou
 
Quote from bluelou:

ProfLogic,
Please correct me if I'm wrong in my interpretation of what you just posted:

1) The noise that I'm observing is caused by traders using different data series and different frequencies for the same instrument.

2) This noise is exacerbated in the spot FX market due to the lack of a consolidated data feed.

3) You didn't say this, but my guess is that you'd agree with the following. Activity-based bars such as tick- or volume-based bars are closer to the true data generating process found in high frequency financial time series. The reason for this is that the true process is not i.i.d..

Are 1, 2, and 3 consistent with what you implied? Do you have any research that validates 1) regarding noise observations.

Thx,
bluelou
More...

1. The noise exists in the chart YOU use to view a market. The noise is compounded by you only viewing a segment of all of the data that exists for a particular market (spot markets). How someone else views or trades the market has no effect on the noise you see in a chart. The FX markets inherently have the ability to allow you filter out as much noise possible using Volume Bar Charts. That option isn't available to you in the Spot markets . . . yet.

2. Correct

3. Correct. Constant Volume Bar Charts are perfect and Tick Charts are not quite perfect. Whenever available I personally choose perfection . . . :D

I'm putting together all of my research now that I will be openly publishing this Spring in numerious publications. It is the sum of about 14 plus years of work. Hope you find it useful.
 
Quote from bluelou:

...It all looks great so far but that wasn't really the point of my posting. My point was, provided I'm correct in describing what I've observed, what are the potential causes of such a "noise" signature? Is there any existing research out there on the subject? Any ideas?
More...

I'm not sure there's much value in trying to decipher the "cause" of the noise. Why not simply create a model of it, and use that on a moving forward basis?
 
You must log in or register to reply here.
Back
Top