Database for minute price feeds

[blueraincap]

Hello,
Could someone pls suggest a database to use for storing 30sec-60second live price feeds for 100+ securities, which a java system will retrieve for evaluation and order routing

 

[MarcG]

Many databases can do without too much tweaking. Personally for this rather easy requirement I would go for mongodb and use documents as buckets (i.e. 1h or 1d data per instrument in one bucket). works nicely in most situations.
Further option: influxdb.

 

[blueraincap]

Many databases can do without too much tweaking. Personally for this rather easy requirement I would go for mongodb and use documents as buckets (i.e. 1h or 1d data per instrument in one bucket). works nicely in most situations.
Further option: influxdb.

Thanks. Mongo is actually on top of my list.

 

[MarcG]

Thanks. Mongo is actually on top of my list.

ok. just as a hint: if you go with mongo and documents as buckets keep the document size limit in mind. personally I store data in binary format or as arrays (bad update performance) in buckets.

 

[Kevin Schmit]

Hello,
Could someone pls suggest a database to use for storing 30sec-60second live price feeds for 100+ securities, which a java system will retrieve for evaluation and order routing

For 30/60 sec bars on only 100 names, almost any database or even flat files will do. Use whatever database you are most familiar with.

 

[blueraincap]

For 30/60 sec bars on only 100 names, almost any database or even flat files will do. Use whatever database you are most familiar with.

Thanks. Will scale up the data requirements substantially going forward. In terms of capacity, speed, and functionality, how do you compare mongo with sqlite and mysql?

 

[Tim Smith]

Thanks. Will scale up the data requirements substantially going forward. In terms of capacity, speed, and functionality, how do you compare mongo with sqlite and mysql?

How about looking at PostgreSQL too ?

Also don't forget, databases are not black boxes. If you don't make use of a database's specific features you're wasting your time.

As for the "how do you compare" question. Do your own testing based up on your own use-cases and see which best fits your own application.

 

[stevegee58]

It sounds like you don't need a full-blown client/server DB solution. I've been successfully using sqlite for years for this kind of data in C++ applications. A Java developer coworker uses H2 (the Java equivalent of sqlite) for the same purposes.
Both sqlite and H2 are file-based relational DBMS solutions.

 

[2rosy]

agree any database will do. Whatever is easier. Take a look at https://www.pipelinedb.com/

 

[blueraincap]

Thanks. Just pick up a book on NoSQL vs SQL and Seven Databases in Seven Weeks (PM me if want a copy).

Btw, which spot FX broker has better API, price feed, and automation functionality than IB?