Record high-frequency order book depth, hardware

[winter]

Echoing what others have said, I would stream it to a binary file and then (off-hours) bulk-import into SQL which tends to be a much more efficient operation then many single row insert operations.

 

[alanm]

Like winter said, a bulk insert can be orders of magnitude faster than individual row inserts. The latency/overhead of those millions of ODBC calls is huge.

 

[stephencrowley]

I just switched to a Berkeley DB backend for my high-volume tables and still use SQL for everything

nice and fast.

http://www.sleepycat.com/