C++ concurrent data structure for fast retrieval

[gaussian]

@gaussian, the spec was then insufficient. He should clearly say that he wants to see an implementation, ie. the design, of such a function, not it's mere use.

I'm beginning to doubt your knowledge of the performance characteristics of software.

 

[MoneyMatthew]

mov ecx,stringSize
mov esi,0

Loop1:
movzx eax,aString[esi]
push eax              
inc  esi
loop Loop1


mov  ecx,stringSize
mov  esi,0

Loop2:
pop  eax              
mov  aString[esi],al    
inc  esi
loop Loop2

 

[thecoder]

I'm beginning to doubt your knowledge of the performance characteristics of software.

How come? You know nothing of me and of my work.
I can tell you this, from what I have seen of you: you might be a good theoretician and you can articulate very good, but it belongs much more for such a task.

 

[thecoder]

Oh, ASM needed in C++ !
What a BS!

 

[Snuskpelle]

For a problem like this I would start with an existing concurrent map implementation as a baseline, and then when I have as a realistic test as possible optimize it further. Just IMO not worth worrying about beyond algorithmic complexity until you're in the position to test it. Apologies if you are at that stage, though.

Also lol @ derailing with x86 string reversal.

 

[Turveyd]

Oh, ASM needed in C++ !
What a BS!

Either I'm way out of date coding wise or yep this is BS!!

The Data is too much, need to simplify for 1 thing, scanning millions of records still going to take too long, loaded into ram and to what end, there is no advantage to doing that as the past doesn't represent the future that often.

 

[thecoder]

Hey OP, here's your solution: use a 64 core / 128 thread AMD CPU (Threadripper or even Epyc), add at least 64 GB RAM to the machine, plus SSD, and do normal standard multithreading. Problem solved. Next problem, please!

 

[qlai]

Multiple threads and processes will be reading and writing concurrently

Why is this a requirement? Can you have multiple ledgers - one per thread/process? For example, if you have a matching engine, it would be one per symbol.

 

[thecoder]

Why is this a requirement? Can you have multiple ledgers - one per thread/process? For example, if you have a matching engine, it would be one per symbol.

He wants to create, update & modify thousands of orders per second. Ie. looks like HFT.
For process level access I would use an API via a shared lib (aka DLL), though shared mem is possible too, but IMO not necessary.

 

[qlai]

He wants to create, update & modify thousands of orders per second.

So multiple threads will access/modify same order? Why? What if you run out of threads, how do you scale?