Re: [coreboot] K8 machine with (a lot of) ram from 2 vendors keeps resetting
Hi Ward, On Wed, Sep 2, 2009 at 8:03 PM, Ward Vandewegew...@gnu.org wrote: Hi Marc, On Wed, Sep 02, 2009 at 04:03:45PM -0600, Marc Jones wrote: With the proprietary BIOS, this setup works perfectly. That said, the manual for the board does say it's not recommended to mix memory types. Our system integrator mentions that when 16 banks are used, the memory runs at maximum 533MHz. As we talked about, it looks like the memory is sized correctly and the next thing to try forcing the memory speed slower. 8 dual rank dimms (16 banks) s[eed limitation might be spec'd or errata. A check might need to go into the main k8 mem init code. I've forced the speed down to 533MHz with this patch: --- src/northbridge/amd/amdk8/raminit_f.code(revision 4625) +++ src/northbridge/amd/amdk8/raminit_f.code(working copy) @@ -1811,6 +1811,10 @@ } min_latency = 3; + // Force minimum cycle time to 3.75ns (i.e. 266MHz) + min_cycle_time = 0x375; + + printk_raminit(1 bios_cycle_time: %08x\n, bios_cycle_time); printk_raminit(1 min_cycle_time: %08x\n, min_cycle_time); /* Compute the least latency with the fastest clock supported Which appears to do the right thing, but the behavior is unchanged. Here's a boot log prior to the patch: http://ward.vandewege.net/coreboot/h8dme/minicom-20090902-with-64G-samsung-on-cpu1-kingston-on-cpu2-before-533MHz-limit.cap and here's after the patch: http://ward.vandewege.net/coreboot/h8dme/minicom-20090902-with-64G-samsung-on-cpu1-kingston-on-cpu2-after-533MHz-limit.cap This is with all samsung ram on CPU1 and all kingston ram on CPU2, as you suggested on irc. I don't think that this is a RAM matching problem. Each CPU/MC is completely has completely matched RAM in this setup. The memory is being sized correctly and I had hoped that slowing down would help. The next step is to do a memory test or instrument the memory clear to see how it fails. If it is on a boundary it would indicate an addressing problem. Something more random would indicate timing. But it is a little bit of guess work from here. Comparing the MC PCI registers (function1 and 2) against the legacy bios might reveal something as well. The actual reset is probably a triple fault which probably started with a op-code exception. We can instrument the exception handler if you get really stuck. Marc -- http://marcjonesconsulting.com -- coreboot mailing list: coreboot@coreboot.org http://www.coreboot.org/mailman/listinfo/coreboot
Re: [coreboot] K8 machine with (a lot of) ram from 2 vendors keeps resetting
Hi Ward, On Wed, Sep 2, 2009 at 3:14 PM, Ward Vandewegew...@gnu.org wrote: Hi all, I'm having some problems with a supermicro h8dme board with 2 K8 processors and 64 GB of ram. Sadly, the ram is of two types (that was not my decision :/). We have 8x 4GB Samsung M393T5160QZA-CE6 and 8x 4GB Kingston KVR667D2D4P5/4G. Both of these types of ram are dual rank DDR2, 667 MHz, CL5, 1.8 V, registered, ECC. The memory is installed like this: BANK CPU1 CPU2 1B KING KING 1A KING KING 2B KING KING 2A KING KING 3B SAMS SAMS 3A SAMS SAMS 4B SAMS SAMS 4A SAMS SAMS With the proprietary BIOS, this setup works perfectly. That said, the manual for the board does say it's not recommended to mix memory types. Our system integrator mentions that when 16 banks are used, the memory runs at maximum 533MHz. As we talked about, it looks like the memory is sized correctly and the next thing to try forcing the memory speed slower. 8 dual rank dimms (16 banks) s[eed limitation might be spec'd or errata. A check might need to go into the main k8 mem init code. Marc -- http://marcjonesconsulting.com -- coreboot mailing list: coreboot@coreboot.org http://www.coreboot.org/mailman/listinfo/coreboot
Re: [coreboot] K8 machine with (a lot of) ram from 2 vendors keeps resetting
Hi Marc, On Wed, Sep 02, 2009 at 04:03:45PM -0600, Marc Jones wrote: With the proprietary BIOS, this setup works perfectly. That said, the manual for the board does say it's not recommended to mix memory types. Our system integrator mentions that when 16 banks are used, the memory runs at maximum 533MHz. As we talked about, it looks like the memory is sized correctly and the next thing to try forcing the memory speed slower. 8 dual rank dimms (16 banks) s[eed limitation might be spec'd or errata. A check might need to go into the main k8 mem init code. I've forced the speed down to 533MHz with this patch: --- src/northbridge/amd/amdk8/raminit_f.code(revision 4625) +++ src/northbridge/amd/amdk8/raminit_f.code(working copy) @@ -1811,6 +1811,10 @@ } min_latency = 3; + // Force minimum cycle time to 3.75ns (i.e. 266MHz) + min_cycle_time = 0x375; + + printk_raminit(1 bios_cycle_time: %08x\n, bios_cycle_time); printk_raminit(1 min_cycle_time: %08x\n, min_cycle_time); /* Compute the least latency with the fastest clock supported Which appears to do the right thing, but the behavior is unchanged. Here's a boot log prior to the patch: http://ward.vandewege.net/coreboot/h8dme/minicom-20090902-with-64G-samsung-on-cpu1-kingston-on-cpu2-before-533MHz-limit.cap and here's after the patch: http://ward.vandewege.net/coreboot/h8dme/minicom-20090902-with-64G-samsung-on-cpu1-kingston-on-cpu2-after-533MHz-limit.cap This is with all samsung ram on CPU1 and all kingston ram on CPU2, as you suggested on irc. Anything else I should try? Thanks, Ward. -- Ward Vandewege w...@fsf.org Free Software Foundation - Senior Systems Administrator -- coreboot mailing list: coreboot@coreboot.org http://www.coreboot.org/mailman/listinfo/coreboot
Re: [coreboot] K8 machine with (a lot of) ram from 2 vendors keeps resetting
On 02.09.2009 23:14, Ward Vandewege wrote: I'm having some problems with a supermicro h8dme board with 2 K8 processors and 64 GB of ram. Sadly, the ram is of two types (that was not my decision :/). We have 8x 4GB Samsung M393T5160QZA-CE6 and 8x 4GB Kingston KVR667D2D4P5/4G. Both of these types of ram are dual rank DDR2, 667 MHz, CL5, 1.8 V, registered, ECC. The K8 memory code has some oddities which cause it to break down even if the SPD parameters you use are identical. Not only must all SPD timing parameters be identical, they also have to be stored at the same byte addresses in each SPD. I sent a patch for this to the list sometime last year, but it was never completely finished. I can try to dig it up if you want to take a look. Regards, Carl-Daniel -- http://www.hailfinger.org/ -- coreboot mailing list: coreboot@coreboot.org http://www.coreboot.org/mailman/listinfo/coreboot
