software for detecting changes in the order book across thousands of symbols

M

mokwit

Maybe this is better under 'programming'

Want to know what is the best software for detecting changes in the order book across thousands of non US symbols. Probably not giving enough info but would likely need some kind of storage array of order book levels and the ability to compare snapshots historically with a few stored historical snapshots. Ability to count data field changes also. I know what I am looking for when I see it but can't watch thousands of order books for it.

So far I think it could coded in Excel but maybe the thousands of messages could swamp it* but can split Excel DDE feeds one per market per data crunching computer if limits reached. Perhaps more of a factor is the arrays I would use do it by using complex circular references) - 64bit still has a low addressable limit for spreadsheet of ~4GB?

Multicharts X64 with its order book coding capability might be the way to go if I can use an array for comparison and can save historical snapshots at certain times.

Reason for thinking of these two first is I know the language and know how I can get non US data into them from where.

Needs a Radarscreen/Excel sorted columns waterfall type of display of alerts triggered

Data source has to be eSignal Zenith or Excel DDE/RTD from broker

Any thoughts? remember non us markets so that rules out things like Worden product and HFT products as feeds or front end - that said may be applicable to US markets so still interested in retail price point US market solutions such as QuantQuote TickView or maybe Smartquant type programs.

*(I have previously been told "not recommend DDE more than 200 counters " and "support a huge of stock list.. It's not different between querying 5 Bid-offer including last price and querying only last price. I used to test loading 100 stocks with market index and status and didn't found any problems. iExcel will create one thread for 10 stocks so to handle 100 stock need 10 threads which is still ok. But the number at 500 is to much."
 
Would be interested to receive the benefit of your experiences with DDE - what were the issues you came up against.?
 
Quote from mokwit:

Would be interested to receive the benefit of your experiences with DDE - what were the issues you came up against.?
More...

well... there is some..

http://en.wikipedia.org/wiki/Dynamic_Data_Exchange

Dynamic Data Exchange was first introduced in 1987 with the release of Windows 2.0
More...

last time i talked to IQ feed about it-they said-we can provide the L2 for whatever number of tickers. the question is-can YOU and your internet provider handle it?
i can pass it to you-can you handle it? keep in mind that the nanex boys posted examples with 40-60K quotes PER SECOND! on single ticker. yes. prices changed 60000 times in a blink of your eye. and that's only on L f** 1

oh..i'm sorry..you said-non US tickers..what are they? you have to be more specific. cause i suspect HFT is now everywhere
 
Not trading BAC = will be Asian tickers with low message number levels, but appreciate you bringing up that point - HFT is certainly around in Asia* but to a lesser degree and more expensive to do so would likely be focused on large liquid stocks - that said I have seen some strange price action since HFT came here and suspect that there is a strategy employed of tricking algos into buying via reverse engineering their triggers so can sell to them - or the algo just plain got it wrong.

DDE replacement RTD (RTS?) is seemingly a lot more capable of handing multiple tickers than DDE But I note: 'Real Time Data : RTD although in many ways technically superior to DDE, has been slow to gain acceptance, since it requires non-trivial programming skills, and when first released was neither adequately documented nor supported by the major data vendors.[22][23]'

* I strongly suspect Tokyo had its own flash crash recently i.e. sell of was exacerbated by HFT algos (mixed with panicked local retail) but it was hushed up maybe
 
The problem with things like DDE is that they ar not ever planned for that - which leads to ease of use design decisions which hurt performance.

Any decent program written for that has no problem handling 100.000 or 500.000 updates per second from a feed. My own nanex feed handlers - in replay, mind you - are up to about 1.5 million ticks per second handled (which is why replay - run as fast as possible). Thsi is during data export routines - and includes a lot of level 2 checking for sanity (crossed quotes) to get a consolidated sensible feed.

You just must go custom programming for that. Not cross process boundaries per quote (which is what DDE does).
 
You must log in or register to reply here.
Back
Top