the algorithm is complex, involves lots of computation.
is ECC memory necessary for quantitative trading?
- Thread starter trend2009
- Start date
the algorithm is complex, involves lots of computation.
If it's Entire stock market M1 data and data back 3 years, can't see why, I find history kinda useless doesn't repeat enough.
Making a profit ?? or still working on ??
In the past, I have experimented with running systems that watch about 1000 of the most liquid stocks. Looking at every tick and building any bars that the systems need (1 min, 5 min, volume bars, etc). On an old i7-6700 Acer with 16 GB, I use around 3% CPU within 30 seconds of the open. I do way more calculations than most. I use to run on an i7 1.7GHz laptop with no problem until it died. Unless you are training a NN at each 1min interval you will require very little CPU power.
On the other hand, of the 3% CPU usage, 99% of that is parsing the incoming data. I would guess every tick for all U.S. stocks would only double the amount of data of the 1000 most liquid stocks. Seriously, look at the number of floating-point calculations that today's i7s and Ryzens can do. That's probably not a problem.
What is slow:
Parsing incoming data
IO (reading and writing to a hard drive)
Graphics
Consumer trading platforms
Languages like Python (more than fine for most people)
What is not slow:
Your network connection. Unless you have really, really slow download data speeds, the amount of data for all U.S. stocks, every tick, is trivial. If you are HFT latency is another matter.
Floating-point calculations
Of course, I don't really know what you are doing and you probably know these things already and have tested enough to know that you need more power. And besides, who doesn't want a new ThreadRipper?
I'm retired and just screw around with this stuff as a hobby so...
On the other hand, of the 3% CPU usage, 99% of that is parsing the incoming data. I would guess every tick for all U.S. stocks would only double the amount of data of the 1000 most liquid stocks. Seriously, look at the number of floating-point calculations that today's i7s and Ryzens can do. That's probably not a problem.
What is slow:
Parsing incoming data
IO (reading and writing to a hard drive)
Graphics
Consumer trading platforms
Languages like Python (more than fine for most people)
What is not slow:
Your network connection. Unless you have really, really slow download data speeds, the amount of data for all U.S. stocks, every tick, is trivial. If you are HFT latency is another matter.
Floating-point calculations
Of course, I don't really know what you are doing and you probably know these things already and have tested enough to know that you need more power. And besides, who doesn't want a new ThreadRipper?
I'm retired and just screw around with this stuff as a hobby so...
I looked at the AMD 3950X specs. It only states it supports 3200 MHz DDR4 RAM, dual channel. As an example I then looked at the specs for a better motherboard, the ROG Strix X570-E Gaming. The memory specs state it does support ECC.
Looking at the specs for the AMD 3950X at the WikiChip site, it states the 3950X does support ECC. On the Asus Memory/Device Support page for the ROG Strix X570-E Gaming, there is a file called AMD_Matisse_for_ROG_STRIX_X570-E_GAMING_DRAM_QVL_20190912.pdf. It lists all the RAM sticks which Asus has tested for the ROG Strix X570-E Gaming. The problem with this list is it does not, for some reason, list whether the stick is ECC or non-ECC. Looking for an ECC stick in this PDF would be difficult.
I then looked at another RAM QVL list for the Asus ROG Zenith II Extreme Alpha motherboard. This page is better organized. You can do a quick search for any specification you want. Doing a search for ECC RAM produced a workable list. I chose one, Kingston KSM26ED8/16ME 2666 MHz. Doing a quick text search in the PDF, I found the Kingston also in the PDF list. The problem is, NONE of the ECC RAM in the webpage list are 3200 MHz. I searched on the Crucial site for 16 GB 3200 MHz ECC DDR4 UDIMM but found nothing. Doing 2 web searches, I could not find any 3200 MHz ECC RAM anywhere.
The ROG Strix X570-E Gaming motherboard specs state it can support 2666 MHz ECC RAM. I could not find any specs for the AMD 3950X which state it supports lower speed RAM. I assume the 3950X may support 2666 MHz.
You would need to call AMD and find out if the 3950X fully works with and supports 2666 MHz ECC RAM. You will also need to call the company tech support for whichever motherboard you choose to see if the motherboard works with the AMD 3950X and 2666 MHz ECC RAM combination. The specs on most motherboard pages are usually not 100% clear.
wow, great information. thank you so much. are you also planning to build 3950x trading machne?
Don't conflate bandwidth and latency as they are two orthogonal concepts, i.e. how much can be transmitted per unit time and how much time it takes for a signal to travel. A lot of the time your low latency setup will have pretty low bandwidth, in some cases extremely so (e.g. shortwave setups for NYLon arbitrage).
No, I was just curious whether the 3950X was ECC capable. It appears it is although you would need to call AMD tech support to confirm.
Are you sure you need ECC? I believe ECC is mostly required for scientific, file server, and multi-user applications. Also, computer RAM is less affected at sea level than at higher altitudes. You may want to consider getting non-ECC RAM and then testing it on a periodic basis using free MemTest86. However, I understand if you still want to use ECC RAM.
BTW, you mentioned you are writing your own algorithm. The 3950X has 16 cores. How do you plan to distribute the program load across all cores? What language are you writing in? You might want to look into CUDA for use on an nVidia video card such as the GeForce RTX 2060 6GB with 1920 CUDA Cores. Here are some CUDACast example YouTube videos. Although, you would be forced to learn how to program a GPU.
If I were to upgrade my computer, I would probably choose the Ryzen 9 3900XT, 32 GB non-ECC and the Samsung 970 EVO Plus 2 TB NVMe M.2. The 3900XT has 12 cores and an 8.5% faster base speed than the 3950X. It is also easier to air cool rather than liquid cool. Liquid cooling can leak and cause downtime, not to mention destroy your computer: Noctua NH-U14S TR4-SP3 heatsink, Noctua NF-A15 HS-4-Pin PWM 140mm Premium Fan. I would also choose a B550 motherboard rather than an X570. The B550 does not use a fan and is therefore more dependable. I would also probably get a Corsair 850-1000 watt PSU again.
The i7, 16 GB, 5 monitor computer I built some time ago works fine. The desktop can still do everything I need. It does need some basic maintenance though, software upgrades, installs, dust cleaning, etc... The hardware still works OK. Building a new computer takes a lot of time and tweaking, especially with COVID-19. Amazon now takes 12 days for shipping instead of 3.
thank you so much for information. You are right maybe 3900x is better given high speed on single thread computing. I do not think I need gpu, that requires to spend extra time on the project. I do not know if ecc is a must in a trading machine, maybe not so critical
If you really have money to spend. Why not design a duplex system where 2 separate computers do same task and compare results before taking any action.
Alot more than just RAM can fail in modern PC......
"Never go to sea with two chronometers; take one or three."
Or even using triplex system, so if 1 fails could use vote of 2 to proceed.
This method probably used by military nunclear , nasa etc.
"Some ECC memory uses triple modular redundancy hardware (rather than the more common Hamming code), because triple modular redundancy hardware is faster than Hamming error correction software."
TBH i would not care , but lets say you do 2 week long calculation , maybe few bits were off.. Or travel alot and would like 2nd computer to take over if 1 fails.
Alot more than just RAM can fail in modern PC......
"Never go to sea with two chronometers; take one or three."
Or even using triplex system, so if 1 fails could use vote of 2 to proceed.
This method probably used by military nunclear , nasa etc.
"Some ECC memory uses triple modular redundancy hardware (rather than the more common Hamming code), because triple modular redundancy hardware is faster than Hamming error correction software."
TBH i would not care , but lets say you do 2 week long calculation , maybe few bits were off.. Or travel alot and would like 2nd computer to take over if 1 fails.
Last edited:
From their FAQ:
"What will be the differences with my own desktop ?
Nothing.
You will have a Windows 2008 R2 operating system installed, with its official license. Windows 2008 R2 is the professional version of Windows 7. The look will feel will be a bit different, but that’s all."
Year 2020, VPS provided on unsupported Windows OS?....