This problem is solved by setting the CPU internal clock as NON-SSC(spread spectrum clock).
PCIe spec says seperate ref clock should be Non-SSC. We only set device clock as Non-SSC before, ignoring the clock at CPU side. Now the system is stable. The"slot clock configuration" and "common clock configuration" are both still 1. It doesn't seem to matter. Thanks to you all. Zheng ________________________________ From: Kyösti Mälkki <[email protected]> Sent: Friday, January 13, 2017 4:58 PM To: Zheng Bao Cc: Predrag Vidic; [email protected]; Zoran Stojsavljevic Subject: Re: [coreboot] Question about PCIe separate reference clock On Fri, Jan 13, 2017 at 6:01 PM, Zheng Bao <[email protected]<mailto:[email protected]>> wrote: About "Asynchronous clock mode on mainboard side", I guess. Both the device and bridge have the fields "slot clock configuration" and "common clock configuration". Again assuming the mainboard side is CPU (some AMD SoC) and you are running coreboot and can bodify binaryPI / AGESA firmware. You can spoof this in the firmware and leave "Common Clock Configuration" bit unset on both sides, even if they report "Slot Clock Configuration = 1". This affects the ASPM L0s and L1 latencies devices report in the Link Capabilities register, but may also change hardware behaviour since specs require link retraining after bit is set. On our board, both the"slot clock configuration" of device (E8860) and bridge are 1. Does it mean the "on mainboard" side it does not support "Asynchronous clock mode"? So, any specs from the mainboard or CPU side of the link? Ultimately your 100MHz clock stability and jitter is affected by the choice of your 25MHz crystal part and PCB layout, right? Kyösti ________________________________ From: Predrag Vidic <[email protected]<mailto:[email protected]>> Sent: Friday, January 13, 2017 10:29 AM To: Zheng Bao Cc: Zoran Stojsavljevic; [email protected]<mailto:[email protected]> Subject: Re: [coreboot] Question about PCIe separate reference clock Hi Zheng, Schematic is simple and OK. I assume you are trying to run Wolf VPXxx-E8860-xxxx or similar GPU borad. You can't do much there if there is a hardware problem but it is most possibly a software setup problem. VPX does not have incomming clock so it is by default in async_clock mode. The question is did you allowed that mode on the main board side? Asynchronous clock mode is not default mode for a motherboard. Check if the mode is allowed. Second, check if the other PCIe device can work on the motherboard PCIe slot. Regards, Predrag On Fri, Jan 13, 2017 at 6:44 AM, Zheng Bao <[email protected]<mailto:[email protected]>> wrote: Here is the ref clk part. Please review. No refclk routes to VPX connector. Thanks. Zheng ________________________________ From: Predrag Vidic <[email protected]<mailto:[email protected]>> Sent: Thursday, January 12, 2017 6:53 PM To: Zheng Bao Cc: Zoran Stojsavljevic; [email protected]<mailto:[email protected]> Subject: Re: [coreboot] Question about PCIe separate reference clock Hi Zheng, Without knowing the particular solution for the PCIe transmitter you have on your board, I'd check refclk pins on your design for the proper termination. The problem can be in irregular readings from the pins. Reragds, Predrag On Thu, Jan 12, 2017 at 3:26 PM, Zheng Bao <[email protected]<mailto:[email protected]>> wrote: Q: do you use local xtal attached to Si52111-B5 to generate local PCIe 25MHz clock? Zheng: yes Q: If you dothis, my next question is how you synchronize these two clocks: Local PCIe 25 MHz and common reference clock from CPU? Zheng: We do NOT sync these two. Q: Since these two clocks, as I understand above scenario, are asynchronous to each other?! Zheng: yes. Asynchronous. The VPX connector does not have a PCIe ref clock signal, so we can not send CPU PCIe ref clock to device. The PCIe spec says if the separate refclk on devices should be 100MHz ± 300PPM, with SSC off. We believe our board meet this requirement. So we doubt the problem lies in PCI configration space. Zheng ________________________________ From: Zoran Stojsavljevic <[email protected]<mailto:[email protected]>> Sent: Thursday, January 12, 2017 9:22 AM To: Zheng Bao; Predrag Vidic Cc: [email protected]<mailto:[email protected]> Subject: Re: [coreboot] Question about PCIe separate reference clock Hello Zheng, For decades, I've been FW/SW engineer, but I do understand a little bit of a HW. I have looked into the Si52111-B5 data sheet for clarification. My problem here is to understand, your use case: do you use local xtal attached to Si52111-B5 to generate local PCIe 25MHz clock? If you do this, my next question is how you synchronize these two clocks: Local PCIe 25 MHz and common reference clock from CPU? Since these two clocks, as I understand above scenario, are asynchronous to each other?! Please, clarify for us your use case. Thank you, Zoran _______ On 1/12/17, Zheng Bao <[email protected]<mailto:[email protected]>> wrote: > Our VPX design uses separate reference clock source, which is Si52111-B5 (No > spread), instead of common ref clock from CPU. > Now The system is unstable. Reading PCIE configuration space is unstable > too. (If we add some fly wire to make it work with common ref clock, the > system becomes stable.) > > (abstracted from PCIe spec: 12 Slot Clock Configuration - This bit indicates > that the > component uses the same physical reference clock that the > platform provides on the connector. If the device uses an > independent clock irrespective of the presence of a reference > clock on the connector, this bit must be clear. > For a multi-Function device, each Function must report the > same value for this bit.) > > Based on my understanding, the BIOS need to read bit "Slot Clock > Configurationclear" to see if > separate ref clock is used. BIOS then write bit "Common Clock > Configuration". > > On our board, the bit "Slot Clock Configuration" is always 1, which I assume > should be 0. > > My question is, how the hardware affect the bit "Slot Clock Configuration"? > How do we need to design our board to make the bit "Slot Clock > Configuration" be 0? > > Thanks. > > Zheng > > -- coreboot mailing list: [email protected]<mailto:[email protected]> https://www.coreboot.org/mailman/listinfo/coreboot
-- coreboot mailing list: [email protected] https://www.coreboot.org/mailman/listinfo/coreboot
