My next motherboard

I have been running Sandra benchmarks and I will let you know what I find. The first thing I noticed is with one memory stick in per CPU, I got half the memory bandwidth as when I add in the other two sticks that are reserved for the other CPUs, so having the right number of sticks per CPU in is critical.

As regarding the fans, see above post.

I am now looking into NUMA. Sandra is reporting that NUMA is not turned on. I know it is turned on in the BIOS, so it has to be a Windows thing. I believe I have to turn on PAE (Physical Address Extension) in Windows Server Enterprise to get NUMA, but I am not sure. Report later when I research it.

FWIW, the memory bandwidth I am getting as of now without NUMA and with 128 memory interleaving as reported by Sandra is 7.6GBs. Without the 128 bit interleaving, it was at about 3.5GBs. I expect to get to 10GBs with NUMA.

Still waiting on the other two Opterons.

Compatibility with my software has forced me to use the 32 bit version of W2003ServerEnterprise :( The problem is that the developer version of .Net Framework 1.1 will not install in x64 Windows. So much for 100% compatibility.

One final note - I have gotten wildly different results with Sandra depending on run. It is hard to get a real handle on it.

nitro
Quote from prophet:

Surprisingly this issue is rarely mentioned. That’s why I ordered 4 256MB modules when I would have otherwise used 2 512MB modules.

Answer #5 in the http://www.tyan.com/support/html/f_s2885.html FAQ describes what you are talking about. The “node interleave” mode orders all of CPU 0’s memory first, then CPU 1 and so on. The second mode is called “bank interleave” or “page interleave” which interleaves memory among CPUs per page, apparently to normalize latencies across all memory addresses, regardless of thread affinity and memory location. This also disallows NUMA. I have benchmarked both configurations with my backtesting software and don’t see much of a difference so far. Let me know if you find a performance difference.

I had the same problem. In the BIOS there is an option to throttle CPU fan speed depending on temperature. It makes a huge difference. Otherwise the fans run at 100% RPM all the time and are quite noisy. Are you using the AMD retail heatsink/fans?

In other news… my dual system has been rock solid. I’ve had it number crunching nearly non stop for about 40 days, mostly with both CPUs fully utilized (two processes). There is no evidence of memory contention between processes. I once tried to overclock the CPU-CPU hypertransport from 800MHz to 1GHz resulting in a lock up after an hour of number crunching. The only other concern is that SiSoftware Sandra reports 6.2GB memory bandwidth when I should be seeing the 10GB reported by various reviews of this motherboard.
More...
 
Quote from LivermoreRisen:

I remember I was only able to catch bloomberg after I got a cable box; this will probably be the case with the tv tuner card as well. Also, what do you mean by tv tuner PLUS DISPLAY? Thanks.
More...

1. If Bloomberg is part of your "cable" package, you will be able to get it as any other channel. If Bloomberg in your area requires a decrypting device, that's a different issue.

2. "Tuner + Display" = The card is both a tuner and a normal video card all in one, whereas a "Tuner Card only" does not act as a monitor driver itself, but only plays through a separate, primary monitor... With a TV Tuner, you will need an additional video card, which you already have... but 2 slots are taken up.
 
Fixed NUMA issue, turning on PAE (Physical Address Extension) in Windows Server Enterprise was the ticket.

Just ported over all my trading systems to it and I am now in final phase of testing. Looks good. Now I just need to wait for the other two Opterons before I send her off to the colocation facility.

nitro :cool:
 
Quote from nitro:

Fixed NUMA issue, turning on PAE (Physical Address Extension) in Windows Server Enterprise was the ticket.

Just ported over all my trading systems to it and I am now in final phase of testing. Looks good. Now I just need to wait for the other two Opterons before I send her off to the colocation facility.

nitro :cool:
More...

You're gonna look kinda funny trading from a wire cage at an SBC colo... good news, my wife can hook you up to an OC-192.:p
 
Quote from SethArb:

hey nitro

do you think you have the necc
skills , time , equipment to fix a toshiba laptop running Windows me
that refuses to boot up
90 -95-100 % of the time ?

this is driving me crazy ... I am now using a apple computer to trade while I wait
for a good sale on a pc
More...

Hi Seth,

My2ctsworth:

Why don't you concentrate on making a good trade? Buy a good laptop from the profits and dump the old one. :D

Be good,

nononsense
 
Quote from riskarb:

You're gonna look kinda funny trading from a wire cage at an SBC colo... good news, my wife can hook you up to an OC-192.:p
More...
Why would I have to trade from the cage at the colo? I use Remote Desktop from anywhere in the world that has a connection to the Internet and it is as if I am sitting in front of the machine at the colocation facility.

nitro
 
The other two Opterons arrive on Monday. The fact that I only had two for a week has been interesting because I have been able to do some Sandra tests against my dual Hyperthreaded Xeon machine.

I will post later some results, some intersting, some don't make sense.

nitro
 
Quote from nitro:

Why would I have to trade from the cage at the colo? I use Remote Desktop from anywhere in the world that has a connection to the Internet and it is as if I am sitting in front of the machine at the colocation facility.

nitro
More...

I was having a little fun; it was obviously meant in jest. Let us know how the Opteron's benchmark.
 
I ran two Sandra tests on two machines that I own, the Memory Bandwidth test, and the Memory/Cache Bandwidth test.

The Xeon machine came back with 2600/2600 for the Memory Bandwidth test. In the Memory/Cache test Xeons came in at a Combined Index of 9444 MB/s and a Speed Factor of 62.8.

The Opteron machine came back with 9600/9600 for the Memory Bandwidth test, but the results varied wildly and got as low as 3600/3600. On the Mem/Cache test the Opterons Combined Index of 9600 MB/s and a Speed Factor of 11.8.

Clearly the Opterons had a much higher memory bandwidth test, but it got creamed on the mem/cache test against the Xeons. Why is the Opteron mem/cache test so much worse than the Xeons, especially seeing that the Opterons clearly have a much higher results on the Mem Bandwidth test?

This led me to understand memory ratings better on memory modules. The Xeons have two 512 K sticks of DDR PC2100 ECC Registered buffered SDRAM. The rating on the sticks is 2.0-2-2-5CL 1CMD. The Opterons have four 512 K sticks of DDR PC3100 ECC Registered buffered SDRAM. The rating on the sticks is 3.0-4-4-4CL 2CMD.

From Tom's Hardware:
How Memory Access Works

Information is stored by first separating the memory area into rows and columns. The capacity of the individual chips determines the number of rows and columns per module. When several arrays are combined, they create memory banks.

The chips are actually accessed by means of control signals such as row address strobe (RAS), column address strobe (CAS), write enable (WE), chip select (CS) and several additional commands (DQ). You also need to know something about which row is active in the memory matrix at any given moment.

In today's computers, a command rate is defined in BIOS - generally 1-2 cycles. This describes the amount of time it takes for the RAS to be executed after the memory chip has been selected.

"The memory controller selects the active row. But before the row will actually become active so that the columns can be accessed, the controller has to wait for 2-3 cycles - tRCD (RAS-to-CAS delay). Then it sends the actual read command, which is also followed by a delay - the CAS latency. For DDR RAM, CAS latency is 2, 2.5 or 3 cycles. Once this time has lapsed, the data will be sent to the DQ pins. After the data has been retrieved, the controller has to deactivate the row again, which is done within tRP (RAS precharge time).

There is one more technical restriction - tRAS (active-to-precharge delay). This is the fewest number of cycles that a row has to be active before it can be deactivated again. 5-8 cycles are about average for tRAS."

Memory timings are generally cited in order of importance:"
More...
[An explanation of how to decode the code is given below in the link.]

Now, I am not sure about all of this, but it seems to me that I got slow RAM in the Tyan (Opteron) machine. However, I then went to the Crucial site and looked for RAM with faster ratings. To my surprise, all these memory sticks that are ECC registered buffered and are 512K don't come any faster!

I don't know what to do at this point. I will be contacting Tyan on Monday to see if there is anything I can do. The Sandra software suggests that I might be able to alter the BIOS settings to get faster response times from the memory...

It is very possible that the chipset on the AMD machines are way underpowered and are starving the CPU(s).
 

Attachments

You must log in or register to reply here.
Back
Top