Realtime Correlation Analysis

B

botpro

Anybody doing correlation analysis?

I think doing realtime correlation analysis among say 100 stocks would give enormous insight
into the market, and of course give good trading opportunities for daytrading.

Such an analysis is time consuming, so doing it every 5 or 10 minutes should be ok I guess.
In this case each stock would be compared to each of the other 99 stocks, ie. 100 x 100 = 10000 comparisons,
and in each comparison all the intraday data would be scanned for calculating the correlation factors and collecting them in a matrix.

I think to optimize it one should have a correlation routine that can update its values when new data come in,
instead of each time recalculating everything anew.

And if one collects the matrices then one can even make a 3D-analysis... ;-)

This could be the next hot thing after HFT...
.
 
Last edited:
botpro said:
Anybody doing correlation analysis?

I think doing realtime correlation analysis among say 100 stocks would give enormous insight
into the market, and of course give good trading opportunities for daytrading.

Such an analysis is time consuming, so doing it every 5 or 10 minutes should be ok I guess.
In this case each stock would be compared to each of the other 99 stocks, ie. 100 x 100 = 10000 comparisons,
and in each comparison all the intraday data would be scanned for calculating the correlation factors and collecting them in a matrix.

I think to optimize it one should have a correlation routine that can update its values when new data come in,
instead of each time recalculating everything anew.

And if one collects the matrices then one can even make a 3D-analysis... ;-)

This could be the next hot thing after HFT...
.
More...

Are you developing some kind of pairs trading strategy?
I presume you are developing using short time frames... have a look at journals on pairs trading strategy and you might be able to find some helpful methodologies
 
victorycountry said:
Are you developing some kind of pairs trading strategy?
More...
I don't know where this will end finally, I'm just experimenting and evaluating the usefulnes of that method for trading strategies.
I just have a strong feeling that it could be a very useful tool for system trading.

I presume you are developing using short time frames... have a look at journals on pairs trading strategy and you might be able to find some helpful methodologies
More...
Thx, I'll research.
 
Last edited:
botpro said:
Anybody doing correlation analysis?

I think doing realtime correlation analysis among say 100 stocks would give enormous insight
into the market, and of course give good trading opportunities for daytrading.

Such an analysis is time consuming, so doing it every 5 or 10 minutes should be ok I guess.
In this case each stock would be compared to each of the other 99 stocks, ie. 100 x 100 = 10000 comparisons,
and in each comparison all the intraday data would be scanned for calculating the correlation factors and collecting them in a matrix.

I think to optimize it one should have a correlation routine that can update its values when new data come in,
instead of each time recalculating everything anew.

And if one collects the matrices then one can even make a 3D-analysis... ;-)

This could be the next hot thing after HFT...
.
More...

You're thinking about calculating correlations every 5 minutes?!?!

To do correlation analysis you need synchronised prices. So what granularity is your data that you can treat prices as effectively synchronised? This is normally a lower frequency than your data starts with (I calculate correlations on weekly prices, downsampled from daily data). Otherwise your correlations will be biased downwards, or include some stale prices.

How many data points do you need to get a decent estimate of correlation for a 100 square matrix with 100^2 - 100 / 2 degrees of freedom? Quite a few.

In summary I don't think you'd get meaningful information using 20 minutes of tick data to calculate correlations. So there is no point updating every 5 minutes....

GAT
 
There's a sponsor here that's about "pairs trading" and there's a website (I can not remember its name) that analyzes in "real-time" the tops stock correlations for pairs trading...the top 100 stocks for such.

It gave the top 100 for daily pairs, weekly pairs and monthly pairs...all in real-time. Along with showing how the correlation for each had changed historically over the past year. The site only dealt with stocks and industry groups.

I saw that website accidentally when an ET member (now dead) by the name of Don Bright (a sponsor) discussed high stock correlations. It may be worth your time to research Don Bright message posts here and search Google for pairs trading correlations and correlation analysis and intermarket correlation analysis.

Also, you may want to contact the website MRCI @ http://www.mrci.com/special/correl.php that deals with futures correlations. They have 5 traders strongly into that stuff that runs the website. Maybe they will remember that stock website that I can not remember the name of such.

In addition, research twitter and stocktwits about stock correlation analysis, I've seen a few guys on there doing real-time stock correlation analysis...I don't know if they're using a program on 100s or 1000s of stocks or just a few stocks.
 
Last edited:
botpro said:
How was your experience, did it help?
CEP = Complex event processing ?
More...
yes, complex event processing. used rolling correlation for spreading about 8-9 years ago but not much benefit relative to simpler things
 
A correlation analysis of _uncorrelated_ GBM data
Params: stocks=5, vola=30%, bars_per_day=78 (ie. 5-minute bars), 21 days. Showing only the last 9 observed correlations of the 21st day:
Code: 
d=21 b=70:
  1.00000 -0.02427  0.02960  0.00452  0.01382
-0.02427  1.00000  0.00035  0.02210 -0.01759
  0.02960  0.00035  1.00000 -0.02583  0.04739
  0.00452  0.02210 -0.02583  1.00000 -0.01700
  0.01382 -0.01759  0.04739 -0.01700  1.00000

d=21 b=71:
  1.00000 -0.02404  0.03033  0.00390  0.01455
-0.02404  1.00000  0.00106  0.02151 -0.01689
  0.03033  0.00106  1.00000 -0.02769  0.04959
  0.00390  0.02151 -0.02769  1.00000 -0.01881
  0.01455 -0.01689  0.04959 -0.01881  1.00000

d=21 b=72:
  1.00000 -0.02408  0.03036  0.00389  0.01464
-0.02408  1.00000  0.00103  0.02153 -0.01697
  0.03036  0.00103  1.00000 -0.02769  0.04965
  0.00389  0.02153 -0.02769  1.00000 -0.01883
  0.01464 -0.01697  0.04965 -0.01883  1.00000

d=21 b=73:
  1.00000 -0.02461  0.03042  0.00259  0.01459
-0.02461  1.00000  0.00091  0.02409 -0.01686
  0.03042  0.00091  1.00000 -0.02788  0.04964
  0.00259  0.02409 -0.02788  1.00000 -0.01854
  0.01459 -0.01686  0.04964 -0.01854  1.00000

d=21 b=74:
  1.00000 -0.02437  0.03016  0.00265  0.01415
-0.02437  1.00000  0.00062  0.02416 -0.01736
  0.03016  0.00062  1.00000 -0.02795  0.05014
  0.00265  0.02416 -0.02795  1.00000 -0.01866
  0.01415 -0.01736  0.05014 -0.01866  1.00000

d=21 b=75:
  1.00000 -0.02525  0.03038  0.00115  0.01448
-0.02525  1.00000  0.00034  0.02591 -0.01775
  0.03038  0.00034  1.00000 -0.02838  0.05024
  0.00115  0.02591 -0.02838  1.00000 -0.01930
  0.01448 -0.01775  0.05024 -0.01930  1.00000

d=21 b=76:
  1.00000 -0.02516  0.03019  0.00099  0.01474
-0.02516  1.00000 -0.00018  0.02540 -0.01687
  0.03019 -0.00018  1.00000 -0.02732  0.04839
  0.00099  0.02540 -0.02732  1.00000 -0.02096
  0.01474 -0.01687  0.04839 -0.02096  1.00000

d=21 b=77:
  1.00000 -0.02545  0.02982  0.00299  0.01470
-0.02545  1.00000 -0.00010  0.02494 -0.01687
  0.02982 -0.00010  1.00000 -0.02777  0.04839
  0.00299  0.02494 -0.02777  1.00000 -0.02097
  0.01470 -0.01687  0.04839 -0.02097  1.00000

d=21 b=78:
  1.00000 -0.02542  0.02978  0.00304  0.01470
-0.02542  1.00000 -0.00022  0.02508 -0.01685
  0.02978 -0.00022  1.00000 -0.02796  0.04837
  0.00304  0.02508 -0.02796  1.00000 -0.02095
  0.01470 -0.01685  0.04837 -0.02095  1.00000
 
Last edited:
You must log in or register to reply here.
Back
Top