Thanks kt and rosy for mentioning hdf5 and pytable because it is just awesome.
On a laptop and slow hard drive, using python, I'm writing 1 million tick messages (date,time,price,size,total,bid,ask) in 4 seconds, with a compressed size of 3.6 Megs (<4bytes per row!). Retrieval time is a...