C++ concurrent data structure for fast retrieval

a structure with say name, price, hash and lock (other items to suit your need). You will need a hash to search the structure otherwise it will become a linear search. A lock maintained with a compare and swap so only one updater writes to it. If you maintain it in memory you will also need a mechanism to rebuild in the event of a crash.
 
So multiple threads will access/modify same order? Why? What if you run out of threads, how do you scale?
Depends on the data. If each data record is long, then it can indeed be necessary to update parts of it in parallel by multiple threads. But normally one should avoid that.
And on a standard SMT2-core 2 HW-threads can run in parallel, but one can run much more (even hundreds) of SW-threads on the same core.
One would better work with job queues, ie. producer/consumer stuff, with many worker threads...
 
Last edited:
Hey OP, here's your solution: use a 64 core / 128 thread AMD CPU (Threadripper or even Epyc), add at least 64 GB RAM to the machine, plus SSD, and do normal standard multithreading. Problem solved. Next problem, please! :)
What data structure to hold the orders?
 
Depends on the data. If each data record is long, then it can indeed be necessary to update parts of it in parallel by multiple threads. But normally one should avoid that.
And on a standard SMT2-core 2 HW-threads can run in parallel, but one can run much more (even hundreds) of SW-threads on the same core.
One would better work with job queues, ie. producer/consumer stuff, with many worker threads...
Really, h/w thread and s/w thread?.
 
What data structure to hold the orders?
Traversable container(s), ie. vector and/or set of own classes, plus a hash-like key-mechanism for fast direct access, or an index for similar purpose (though with set and correctly defined key part this is not necessary as it's implicitly given in set as it uses internally a btree-like mechanism, called red-black-tree or something that, it's implementation dependent, but usually with B*tree functionality for fast finding...)...
And: one can always improve with an own cache mechanism, should it be necessary.
But you need to create an internal API first, for all the running threads (and for processes, in case that is really a requirement)...
 
Last edited:
Really, h/w thread and s/w thread?.
What exactly don't you understand?
User should just use SW-threads, the language internally will manage how to do this with and on the available HW-threads on the CPU-cores.
Remember: this (ie. OPs requirement) is normal application programming, not system programming.
 
Last edited:
KDB

GAT

GAT, have you been following Whitney's latest project, Shakti?

https://shakti.com/

There's also a Shakti Google Group:

https://groups.google.com/g/shaktidb?pli=1

Really interesting stuff. Allows use of CUDA too.

For a higher level language though, Clojure seems like an easier fit than C++ for the OP's task. All that concurrent worry nonsense goes away. But I get it. He's married to C++ so has to come up with a C++ solution. Good luck!
 
Last edited:
Back
Top