[casper] ROACH2 one_gbe block losing data on reception
Hi, I'm experimenting with the one_gbe block on ROACH2. So far data transmission looks flawless, I can capture all the bytes I send. However, receiving data from a computer seems to result in missing data. My design is very simple, I have rx_ack tied high, and then counters on rx_val and the rising edge of rx_eof and also on rx_overrun and rx_badframe. If I just send a single packet, all looks good, the rx_val counter shows the number of bytes I sent, and rx_eof shows one packet received. But when I try to send a sequence of packets, I end up with the rx_val counter showing fewer bytes than I sent, by about ~ 0.5-5% depending on the exact combination of parameters I use in sending packets. Sometimes all the ~1024 packets arrive, as indicated by the rx_eof counter, but other times it seems 1-3 are missing. I see consistent results whether the packet payload is 1024 bytes, 4096 bytes, or 8192 bytes. I never see any counts on the rx_overrun or rx_badframe line. I am using sendall to send the packets from the host computer, and it reports that all bytes are being sent, so I assume it's not dropping them on the way outbound (plus it seems like it would be weird for the host computer to send partial packets). I've tried this test with two different network adapters (both directly connected to the ROACH2 fabric ethernet port, and with various lengths of CAT 6 ethernet cables. Has anyone used the one_gbe block in this way? Am I missing something? Thanks, Glenn
Re: [casper] ROACH2 one_gbe block losing data on reception
Hi Glen, I assume you are running your fabric > 125MHz? What is the data rate you are trying to sustain? HK On Mon, Jul 27, 2015 at 10:06 PM, G Jones wrote: > Hi, > I'm experimenting with the one_gbe block on ROACH2. So far data > transmission looks flawless, I can capture all the bytes I send. However, > receiving data from a computer seems to result in missing data. > My design is very simple, I have rx_ack tied high, and then counters on > rx_val and the rising edge of rx_eof and also on rx_overrun and rx_badframe. > > If I just send a single packet, all looks good, the rx_val counter shows > the number of bytes I sent, and rx_eof shows one packet received. But when > I try to send a sequence of packets, I end up with the rx_val counter > showing fewer bytes than I sent, by about ~ 0.5-5% depending on the exact > combination of parameters I use in sending packets. Sometimes all the ~1024 > packets arrive, as indicated by the rx_eof counter, but other times it > seems 1-3 are missing. > > I see consistent results whether the packet payload is 1024 bytes, 4096 > bytes, or 8192 bytes. > > I never see any counts on the rx_overrun or rx_badframe line. > > I am using sendall to send the packets from the host computer, and it > reports that all bytes are being sent, so I assume it's not dropping them > on the way outbound (plus it seems like it would be weird for the host > computer to send partial packets). > > I've tried this test with two different network adapters (both directly > connected to the ROACH2 fabric ethernet port, and with various lengths of > CAT 6 ethernet cables. > > Has anyone used the one_gbe block in this way? Am I missing something? > > Thanks, > Glenn > -- Kind regards, Henno Kriel Manager: Hardware Engineering SKA South Africa (p) +27 (0)21 506 7374 (direct) (m) +27 (0)84 504 5050 web: www.ska.ac.za
Re: [casper] ROACH2 one_gbe block losing data on reception
Hi Henno, Thanks for replying. I meant to mention the simulink clock is 250 MHz and the design meets timing. The ultimate goal is loading waveforms into the DRAM since there isn't a CPU DRAM interface for ROACH2 yet, so any speed better than loading via KATCP would be great. But I've tried delays of up to 10 ms between each packet (i.e. down to about 100 KB/s), and am still seeing this behavior. The data loss fraction doesn't seem to improve substantially with reduced data rate. This leads me to suspect some sort of signal integrity issue, but I'm not sure where. Glenn On Jul 28, 2015 12:50 AM, "Henno Kriel" wrote: > Hi Glen, > > I assume you are running your fabric > 125MHz? What is the data rate you > are trying to sustain? > > HK > > On Mon, Jul 27, 2015 at 10:06 PM, G Jones wrote: > >> Hi, >> I'm experimenting with the one_gbe block on ROACH2. So far data >> transmission looks flawless, I can capture all the bytes I send. However, >> receiving data from a computer seems to result in missing data. >> My design is very simple, I have rx_ack tied high, and then counters on >> rx_val and the rising edge of rx_eof and also on rx_overrun and rx_badframe. >> >> If I just send a single packet, all looks good, the rx_val counter shows >> the number of bytes I sent, and rx_eof shows one packet received. But when >> I try to send a sequence of packets, I end up with the rx_val counter >> showing fewer bytes than I sent, by about ~ 0.5-5% depending on the exact >> combination of parameters I use in sending packets. Sometimes all the ~1024 >> packets arrive, as indicated by the rx_eof counter, but other times it >> seems 1-3 are missing. >> >> I see consistent results whether the packet payload is 1024 bytes, 4096 >> bytes, or 8192 bytes. >> >> I never see any counts on the rx_overrun or rx_badframe line. >> >> I am using sendall to send the packets from the host computer, and it >> reports that all bytes are being sent, so I assume it's not dropping them >> on the way outbound (plus it seems like it would be weird for the host >> computer to send partial packets). >> >> I've tried this test with two different network adapters (both directly >> connected to the ROACH2 fabric ethernet port, and with various lengths of >> CAT 6 ethernet cables. >> >> Has anyone used the one_gbe block in this way? Am I missing something? >> >> Thanks, >> Glenn >> > > > > -- > Kind regards, > Henno Kriel > > Manager: Hardware Engineering > SKA South Africa > (p) +27 (0)21 506 7374 (direct) > (m) +27 (0)84 504 5050 > web: www.ska.ac.za >
Re: [casper] ROACH2 one_gbe block losing data on reception
Hi Glenn, Everything seems to be in order, however if there was signal integrity issues the MAC (temac) would report bad frames due to CRC failure (the CRC is calculated over the entire Ethernet frame). Just another sanity check would be to send this data to another PC and just confirm that all the packets are received. HK On Tue, Jul 28, 2015 at 9:10 AM, G Jones wrote: > Hi Henno, > Thanks for replying. I meant to mention the simulink clock is 250 MHz and > the design meets timing. The ultimate goal is loading waveforms into the > DRAM since there isn't a CPU DRAM interface for ROACH2 yet, so any speed > better than loading via KATCP would be great. But I've tried delays of up > to 10 ms between each packet (i.e. down to about 100 KB/s), and am still > seeing this behavior. The data loss fraction doesn't seem to improve > substantially with reduced data rate. This leads me to suspect some sort of > signal integrity issue, but I'm not sure where. > > Glenn > On Jul 28, 2015 12:50 AM, "Henno Kriel" wrote: > >> Hi Glen, >> >> I assume you are running your fabric > 125MHz? What is the data rate you >> are trying to sustain? >> >> HK >> >> On Mon, Jul 27, 2015 at 10:06 PM, G Jones >> wrote: >> >>> Hi, >>> I'm experimenting with the one_gbe block on ROACH2. So far data >>> transmission looks flawless, I can capture all the bytes I send. However, >>> receiving data from a computer seems to result in missing data. >>> My design is very simple, I have rx_ack tied high, and then counters on >>> rx_val and the rising edge of rx_eof and also on rx_overrun and rx_badframe. >>> >>> If I just send a single packet, all looks good, the rx_val counter shows >>> the number of bytes I sent, and rx_eof shows one packet received. But when >>> I try to send a sequence of packets, I end up with the rx_val counter >>> showing fewer bytes than I sent, by about ~ 0.5-5% depending on the exact >>> combination of parameters I use in sending packets. Sometimes all the ~1024 >>> packets arrive, as indicated by the rx_eof counter, but other times it >>> seems 1-3 are missing. >>> >>> I see consistent results whether the packet payload is 1024 bytes, 4096 >>> bytes, or 8192 bytes. >>> >>> I never see any counts on the rx_overrun or rx_badframe line. >>> >>> I am using sendall to send the packets from the host computer, and it >>> reports that all bytes are being sent, so I assume it's not dropping them >>> on the way outbound (plus it seems like it would be weird for the host >>> computer to send partial packets). >>> >>> I've tried this test with two different network adapters (both directly >>> connected to the ROACH2 fabric ethernet port, and with various lengths of >>> CAT 6 ethernet cables. >>> >>> Has anyone used the one_gbe block in this way? Am I missing something? >>> >>> Thanks, >>> Glenn >>> >> >> >> >> -- >> Kind regards, >> Henno Kriel >> >> Manager: Hardware Engineering >> SKA South Africa >> (p) +27 (0)21 506 7374 (direct) >> (m) +27 (0)84 504 5050 >> web: www.ska.ac.za >> > -- Kind regards, Henno Kriel Manager: Hardware Engineering SKA South Africa (p) +27 (0)21 506 7374 (direct) (m) +27 (0)84 504 5050 web: www.ska.ac.za
Re: [casper] ROACH2 one_gbe block losing data on reception
> Hi Glenn, > > Everything seems to be in order, however if there was signal integrity > issues the MAC (temac) would report bad frames due to CRC failure (the CRC > is calculated over the entire Ethernet frame). > > Just another sanity check would be to send this data to another PC and > just > confirm that all the packets are received. Hi Glenn. Are you actually not receiving all the data? I wonder if there's some kind of error in the statistics gathering, i.e. maybe it strips off one of the headers before counting bytes, etc... Just a thought... Another thing is that the PC may be sending them in a bursty fashion, which makes it imperative that you handle the packet and stick it in a buffer right away, even though the average data rate is low. I'm not sure how to measure that. John > > HK > > On Tue, Jul 28, 2015 at 9:10 AM, G Jones wrote: > >> Hi Henno, >> Thanks for replying. I meant to mention the simulink clock is 250 MHz >> and >> the design meets timing. The ultimate goal is loading waveforms into the >> DRAM since there isn't a CPU DRAM interface for ROACH2 yet, so any speed >> better than loading via KATCP would be great. But I've tried delays of >> up >> to 10 ms between each packet (i.e. down to about 100 KB/s), and am still >> seeing this behavior. The data loss fraction doesn't seem to improve >> substantially with reduced data rate. This leads me to suspect some sort >> of >> signal integrity issue, but I'm not sure where. >> >> Glenn >> On Jul 28, 2015 12:50 AM, "Henno Kriel" wrote: >> >>> Hi Glen, >>> >>> I assume you are running your fabric > 125MHz? What is the data rate >>> you >>> are trying to sustain? >>> >>> HK >>> >>> On Mon, Jul 27, 2015 at 10:06 PM, G Jones >>> wrote: >>> Hi, I'm experimenting with the one_gbe block on ROACH2. So far data transmission looks flawless, I can capture all the bytes I send. However, receiving data from a computer seems to result in missing data. My design is very simple, I have rx_ack tied high, and then counters on rx_val and the rising edge of rx_eof and also on rx_overrun and rx_badframe. If I just send a single packet, all looks good, the rx_val counter shows the number of bytes I sent, and rx_eof shows one packet received. But when I try to send a sequence of packets, I end up with the rx_val counter showing fewer bytes than I sent, by about ~ 0.5-5% depending on the exact combination of parameters I use in sending packets. Sometimes all the ~1024 packets arrive, as indicated by the rx_eof counter, but other times it seems 1-3 are missing. I see consistent results whether the packet payload is 1024 bytes, 4096 bytes, or 8192 bytes. I never see any counts on the rx_overrun or rx_badframe line. I am using sendall to send the packets from the host computer, and it reports that all bytes are being sent, so I assume it's not dropping them on the way outbound (plus it seems like it would be weird for the host computer to send partial packets). I've tried this test with two different network adapters (both directly connected to the ROACH2 fabric ethernet port, and with various lengths of CAT 6 ethernet cables. Has anyone used the one_gbe block in this way? Am I missing something? Thanks, Glenn >>> >>> >>> >>> -- >>> Kind regards, >>> Henno Kriel >>> >>> Manager: Hardware Engineering >>> SKA South Africa >>> (p) +27 (0)21 506 7374 (direct) >>> (m) +27 (0)84 504 5050 >>> web: www.ska.ac.za >>> >> > > > -- > Kind regards, > Henno Kriel > > Manager: Hardware Engineering > SKA South Africa > (p) +27 (0)21 506 7374 (direct) > (m) +27 (0)84 504 5050 > web: www.ska.ac.za >
