weighting equipment importance for backtesting
- Thread starter cdcaveman
- Start date
Your older posts suggest that you are trying to recuperate costs by selling strategy signals online. They say that the smallest dogs bark the loudest. I'm not interested in exchanging the mudslinging and have been educated with greater sense of propriety than to bother proving my credibility over yours.
@Threadstarter:
I just want you to think for a moment if what he recommended is any different from what I said:
- Get a SSD (I said that.)
- He uses a i7-3930k (I said any regular Ivy Bridge processor up to $300 would give the best value.)
- He uses about 50 GB of memory for optimization. (I said that most people would not be using more than 16 GB of memory, so make incremental upgrades if it is necessary. I think a sample size of 1 goes nowhere near disproving this.)
So what remains is for me to (1) prove for a fact that NT is notoriously single-threaded, and (2) debate the merits of using >16 GB of memory.
Here are some facts. I said that NT is notoriously single-threaded.
1. All windows besides the strategy analyzer only use a single core.
2. Backtesting runs on a single core. (http://www.ninjatrader.com/support/forum/showpost.php?p=315408&postcount=9)
3. Only optimization benefits from multiple cores, but with several huge caveats:
a. Using enums and bools will prevent the optimizer from using multiple threads. (http://www.ninjatrader.com/support/forum/showpost.php?p=315403&postcount=8)
b. This is what your processor does in their fancy "using all cores" during optimization. (Max utilization should be 400% on this test machine.) If you take the RMS area under the GC curve and deduct it from the actual processing time, it is effectively single-threaded. That's why NinjaTrader has publicly admitted that they reworked the GC for their strategy analyzer in NT 8 altogether.
Overall, for most use cases, you *will* benefit more from having higher clock frequency than more cores. I said that there's workarounds about NT's limitations - I've bypassed them altogether by interoping OpenMP's Fortran API calls for critical calculations (see OMP thread).
This is just one of the reasons why I didn't like NT. I played with it with hopes to have some fun in .NET but ended up doing almost everything in Fortran.
Lastly, I reiterate that there's a very good chance you won't use >16 GB of memory. Incremental upgrades are a very wise choice, this especially if you've done HPC because there's a good chance for you to overbuild and not utilize all of your resources. There's no point in getting a dual socket board until late '2013 when the Ivy Bridge-E processors are released. Even if you do by that time, for reasons stated above, you will be very disappointed.
Misaki, you're too funny ... you gave plain bad advice on CPU & RAM, and won't admit to it.
Anyone who says #core is less important than CPU frequency for strategy development & backtesting/optimization in Ninja has severe issues.
The attached picture shows Ninja running 94% on 12 cores.
WRT RAM, the only reason why most people won't need it is because they work off 1-year of intraday data - or less - and then wonder why their strat doesn't perform in forward testing.
Incremental RAM upgrades is difficult & silly - difficult because you have to buy matching RAM for the one you already have, and silly because you need to plan for it anyways => buy a mobo supporting 64GB.
Anyone who says #core is less important than CPU frequency for strategy development & backtesting/optimization in Ninja has severe issues.
The attached picture shows Ninja running 94% on 12 cores.
WRT RAM, the only reason why most people won't need it is because they work off 1-year of intraday data - or less - and then wonder why their strat doesn't perform in forward testing.
Incremental RAM upgrades is difficult & silly - difficult because you have to buy matching RAM for the one you already have, and silly because you need to plan for it anyways => buy a mobo supporting 64GB.
1. RAM is very fungible. If matching RAM with the same model and specs is a concern, sticks that I've bought 2 years ago are still widely available, widely used, and only cheaper today. The threadstarter is using Corsair Vengeance, which is still much in service. But even if disaster strikes and Micron/Samsung's factories get wiped out in a nuclear disaster, you do NOT have to be running the exact same models of RAM for your system to work. This is a huge myth. Even if the specs are different, there are workarounds by tweaking your BIOS settings. Seeing that you are painfully clueless and out of touch with reality about any of this, I highly doubt that your experience in assembling PCs. I've catered my answer to the threadstarter's needs. Very clearly, budget is a constraint and he doesn't want to max out all of his DIMMs unless he has to. So much for your hardware advice when the threadstarter's hardware implementation will eventually end up close to what I had suggested, simply because I bothered to listen.
2. I'd give you one thing though, I'll be honest that I haven't used NT for more than 3 years of backtesting in tick data, although I've already pointed out at the start of this thread that my usage of NT is limited. However, I can definitely say:
i - I see you believe that more data makes a model more statistically robust. This is an elementary myth.
ii. - You still do not need 16 GB of memory to backtest >1 year of tick data on NT. The problem lies with your strategy's memory access patterns, not NT, nor the amount of data you are testing on.
3. You evidently did not read any of the caveats I pointed out about optimization and how exactly those cores are utilized. And it's not only your word against mine, but it's your word against NinjaTrader staff and their posts that I've linked to. I think I have done enough to help other readers see the worth of your advice, which is all that matters. I apologize that I do not find it worth anyone's time to reply to you any further, as I have greater priorities.
All I have to say is: nice Windows resource monitor. I find it hilarious that you can't even afford basic software to test your claims. My testing software costs more than your entire machine and NT license combined.
Now it makes sense to me why you have limited experience in assembling PCs, seeing your budget constraints. Good luck selling your strategy signals.
2. I'd give you one thing though, I'll be honest that I haven't used NT for more than 3 years of backtesting in tick data, although I've already pointed out at the start of this thread that my usage of NT is limited. However, I can definitely say:
i - I see you believe that more data makes a model more statistically robust. This is an elementary myth.
ii. - You still do not need 16 GB of memory to backtest >1 year of tick data on NT. The problem lies with your strategy's memory access patterns, not NT, nor the amount of data you are testing on.
3. You evidently did not read any of the caveats I pointed out about optimization and how exactly those cores are utilized. And it's not only your word against mine, but it's your word against NinjaTrader staff and their posts that I've linked to. I think I have done enough to help other readers see the worth of your advice, which is all that matters. I apologize that I do not find it worth anyone's time to reply to you any further, as I have greater priorities.
All I have to say is: nice Windows resource monitor. I find it hilarious that you can't even afford basic software to test your claims. My testing software costs more than your entire machine and NT license combined.
Now it makes sense to me why you have limited experience in assembling PCs, seeing your budget constraints. Good luck selling your strategy signals.It really depends on the software. Generally speaking, in my experience, with general purpose -- or god forbid, poorly written -- software, RAM and number of cores, or SSDs vs HDDs (which are the same speed for sequential access), are more important because the software doesn't utilize resources well. With software which is architected for a very particular purpose, it gets much more subtle, and you start having to look into things like how large the cache is and how deep/wide the instruction pipeline is. But among other things good software won't hog RAM and it won't hammer the hard drive with requests, so those are not as important.
So the answer to your question is: the best machine for doing backtesting on is a machine running good software on it. Just get 16G's and some i7's and be done with it.
hft_boy,
i'm speculating with talking dbs... figuring that having access to a resource locally would be more efficient... but if thats not the case.. then thats not the case... i was just reading the amibroker pdf. . and it seems to make sense to subscribe to a feed and let them maintain the data... maintaining a db can be a job itself.
Quote from misaki:
1. RAM is very fungible. If matching RAM with the same model and specs is a concern, sticks that I've bought 2 years ago are still widely available, widely used, and only cheaper today. The threadstarter is using Corsair Vengeance, which is still much in service. But even if disaster strikes and Micron/Samsung's factories get wiped out in a nuclear disaster, you do NOT have to be running the exact same models of RAM for your system to work. This is a huge myth. Even if the specs are different, there are workarounds by tweaking your BIOS settings. Seeing that you are painfully clueless and out of touch with reality about any of this, I highly doubt that your experience in assembling PCs. I've catered my answer to the threadstarter's needs. Very clearly, budget is a constraint and he doesn't want to max out all of his DIMMs unless he has to. So much for your hardware advice when the threadstarter's hardware implementation will eventually end up close to what I had suggested, simply because I bothered to listen.
2. I'd give you one thing though, I'll be honest that I haven't used NT for more than 3 years of backtesting in tick data, although I've already pointed out at the start of this thread that my usage of NT is limited. However, I can definitely say:
i - I see you believe that more data makes a model more statistically robust. This is an elementary myth.
ii. - You still do not need 16 GB of memory to backtest >1 year of tick data on NT. The problem lies with your strategy's memory access patterns, not NT, nor the amount of data you are testing on.
3. You evidently did not read any of the caveats I pointed out about optimization and how exactly those cores are utilized. And it's not only your word against mine, but it's your word against NinjaTrader staff and their posts that I've linked to. I think I have done enough to help other readers see the worth of your advice, which is all that matters. I apologize that I do not find it worth anyone's time to reply to you any further, as I have greater priorities.
All I have to say is: nice Windows resource monitor. I find it hilarious that you can't even afford basic software to test your claims. My testing software costs more than your entire machine and NT license combined.
i can tell you are legit... i appreciate your help... dual chipsets and 64 gigs isn't out of my price range... but my thoughts are that this isn't a resource i could make sense of as i am just getting started... i've only used ninja trader and i'm limited on my capacity to program.. that being said.. but i am currently in the process of getting an education in math and programming.. i have a a very limited and conceptual understanding of programming.. i've read alot and done very little.. i am not afraid of it at all.. it more facinates me to be honest.. the amount of control programming gives to its employer is to me priceless... i figure after a decent education, and a good understanding of trading/dervatives.. i will become very useful. ..
i tonight RMAed my 8 gigs of ram... and have to wait over 2 weeks for it to come back.. i'm not going to of course.. i'm going to buy 16 gigs in two sticks of the same kind and frequency.. i do realize that ram is compatble.. i don't think EC ram is though... i will end up with 24 gigs when i get my 8 gig back from rma... i have had the most frustrating time trying to figure out why i'm getting bluescreens. .. and trust me memtester etc.. tried it all.. it all checks out.. memory_management to page_file bluescreen erros..
i'm going to work with what i have for a while.... dual xeons with more cache, more ram, better SSD's to me would be the most optimal.. but i'm sure i have more then i can use asof right now considering my programming capacity..