Quantifying randomness: variance ratio

[mysticman]

Stephan,
You seem to be running a very complex and sophisticated operation, taking quotes out of the market depth book into a computational engine, which most of us would not be capable of doing without a great deal of programming assistance. I'm wondering whether something more simple could also produce worthwhile results.
You don't say what your period is, but since your stocks must be trading many times a minute, I assume your "period" is less than a minute. Could not worthwhile results also be produced from 1 or 5 minute bars using the actual trades? It would seem that since VR has been shown to be significant for daily data, surely there must be a time period between daily data and where you are in extreme high-frequency that would also be worthwhile to work with?

 

[stephencrowley]

I have been growing this software for several years now so it does have quite a bit of capabiltiies.. I don't trade several times a minute.. I can, but I've found it more optimal for my trades to last 10 to 20 minutes. I built my system to operate at any intraday frequency, I like to have the ability to trade quickly if I need to.

The variance ratio test can be applied to any series.. however you want to define it. If you want to take the average trade within 1 minute or 5 minute bars I'm sure it would work just as well.

If you have matlab then you might give this a try, it could stand some optimization.

% generate two random walks
x=cumsum(randn(10000,2));
plot(x);

% generate vr profile between 1 and 50 steps
v=vrm(x,1:50);
plot(v);

The VR profiles should be pretty close to 1

Now generate some random walk with drift

>> x=cumsum(randn(10000,2)+0.1);
>> plot(x)
>> plot(mvr(x,1:50))

The VR profiles for this one should be sloping strongly above 1.

Quote from mysticman:

Stephan,
You seem to be running a very complex and sophisticated operation, taking quotes out of the market depth book into a computational engine, which most of us would not be capable of doing without a great deal of programming assistance. I'm wondering whether something more simple could also produce worthwhile results.
You don't say what your period is, but since your stocks must be trading many times a minute, I assume your "period" is less than a minute. Could not worthwhile results also be produced from 1 or 5 minute bars using the actual trades? It would seem that since VR has been shown to be significant for daily data, surely there must be a time period between daily data and where you are in extreme high-frequency that would also be worthwhile to work with?

 

[nicholaf]

Stephen,

Thanks for the great work and matlab files

however, it looks like we need the LeSage econometrics toolbox for the files to run - i had errors for the cols, trimr functions, etc.

however, i later found it at:

http://www.spatial-econometrics.com/

do you use matlab to trade?

Quote from stephencrowley:

I have been growing this software for several years now so it does have quite a bit of capabiltiies.. I don't trade several times a minute.. I can, but I've found it more optimal for my trades to last 10 to 20 minutes. I built my system to operate at any intraday frequency, I like to have the ability to trade quickly if I need to.

The variance ratio test can be applied to any series.. however you want to define it. If you want to take the average trade within 1 minute or 5 minute bars I'm sure it would work just as well.

If you have matlab then you might give this a try, it could stand some optimization.

% generate two random walks
x=cumsum(randn(10000,2));
plot(x);

% generate vr profile between 1 and 50 steps
v=vrm(x,1:50);
plot(v);

The VR profiles should be pretty close to 1

Now generate some random walk with drift

>> x=cumsum(randn(10000,2)+0.1);
>> plot(x)
>> plot(mvr(x,1:50))

The VR profiles for this one should be sloping strongly above 1.

 

[stephencrowley]

No problem. I'm not 100% sure the calculations are correct but they are at least pretty close cause my stuff generates close to the same results in the paper..

I don't trade from matlab but I use it for modelling and playing around with stuff. ANything that I find useful in matlab and I need to do realtime then I rewrite it in java.

For non-realtime stuff I estimate the models in matlab and then read the data into my app to trade.. I can read/write .mat files.

Quote from nicholaf:

Stephen,

Thanks for the great work and matlab files

however, it looks like we need the LeSage econometrics toolbox for the files to run - i had errors for the cols, trimr functions, etc.

however, i later found it at:

http://www.spatial-econometrics.com/

do you use matlab to trade?

 

[mysticman]

Would it be possible for those of us with Excel to do this test? I believe the VR is also called an F-test. Would you suggest a stock so that we could post the results and compare with yours?

 

[stephencrowley]

Right, the VR is also the F-test. The code I posted is for the modified VR, which can detect trends where the standard VR would only detect randomness.

Just checkout QQQQ or some indices.. I only do stuff at very high frequencies and dont look at anything beyond a day so our results would probably vary.

I don't know if excel can do it... the length of time required to calcuate the profile increases exponentially with the size. Probably with daily data you'll be fine.

Quote from mysticman:

Would it be possible for those of us with Excel to do this test? I believe the VR is also called an F-test. Would you suggest a stock so that we could post the results and compare with yours?

 

[mysticman]

Stephen, so how many days of daily data do you recommend?
If we have 2 years of data, can we do a VR on a 1 month base period, compare that to 6 months, then roll this forward to get a graph of the VR through the whole 2 years. Or how would you go about it?

 

[stephencrowley]

There are formulas that will tell you exactly how much you need for it to be significant but I don't have them for the modified version.

A good rule of thumb would be the max length of the VR you should calculate is 3.3% of the sample length.

Here is a demonstration, i generated 10 random walks of length 1000. Their VR profiles should be close to 1.

I tried to calculate VR profiles for up to 100:1.. note that at 100:1 ratio we only have 10 samples to work.. so as the length of the VR profile approachs the length of the series itself its accuracy decreases and it gets pretty much useless.

Quote from mysticman:

so how many days of daily data do you recommend?

 

[mysticman]

A good rule of thumb would be the max length of the VR you should calculate is 3.3% of the sample length.

So let's say we have 4 years of data (1000 days). The max length of the VR period would be 33 days? That's a long way from the length of the series. And the minimum length?

I tried to calculate VR profiles for up to 100:1.. note that at 100:1 ratio we only have 10 samples to work.. so as the length of the VR profile approachs the length of the series itself its accuracy decreases and it gets pretty much useless.

Why only 10 samples? Is that 1000 random walk length / 100?
I'm not sure what the 100:1 ratio means. Does it mean 100 sample length and 1 VR period?
So if you have 100 days you use only a 1 day period? How can that be?

 

[squeeze]

Quote from stephencrowley:

There are formulas that will tell you exactly how much you need for it to be significant but I don't have them for the modified version.

A good rule of thumb would be the max length of the VR you should calculate is 3.3% of the sample length.

Here is a demonstration, i generated 10 random walks of length 1000. Their VR profiles should be close to 1.

I tried to calculate VR profiles for up to 100:1.. note that at 100:1 ratio we only have 10 samples to work.. so as the length of the VR profile approachs the length of the series itself its accuracy decreases and it gets pretty much useless.

I was wondering about sample size bias when using the modified VR over the daily session. I have been coming to the conclusion that if you average the profiles over multiple days the bias should also average out. Is this right?

Another related measurement to the modified VR would be to look at the proportion of values >1 for each interval over multiple days. This would produce a probability of trend for each interval and would be an improvement on the modified VR as it would also factor in dispersion about the mean.