Hi everyone Thank you all for your suggestions. I would like to update you about the progress.
As a quick background and as I had mentioned in last email, despite of getting correct simulation results, I wasn't getting the link established between VCU118 and destination machines, neither a switch nor a Linux Machine. I'm using Si5328 as my clock and I am converting the differential clock to single-ended clock using a IBUFDS. This a an updated procedure: 1- I adjusted the clock values. 2- Implemented near-end PMA loopback and the result is : PASS 3- Used a loopback module and the result is: PASS 4- Using the loopback module, I learned that I am using wrong GTY transmission lanes, so I corrected them. 5- The connection to the 100G Switch: PASS 6- The direct connection to the linux machine, equipped Mellanox ConnetX-4 : FAIL I think the reason that the Linux machine is rejecting the packets is that the data I'm sending is not wrapped in an standard way like UDP frames. The traffic is flowing to the switch though. I'm writing some Verilog to wrap the data in UDP frames. I'll update you once I get the data flow from switch to the linux machine. Thank you all for your help and support. I would appreciate it if you could share your opinion and feedback. Best Regards Arash Roshanineshat On Tue, Feb 27, 2018 at 3:38 PM, Adam Isaacson <aisaac...@ska.ac.za> wrote: > Hi Arash, > > I just spoke with Jack Hickish and it seems Hong Chen, Benjamin Hlope > (SARAO have contracted him to develop the 100GbE yellow block, as well as > other functionality) and you are all going to or are developing the 100GbE > yellow block. I would suggest we try work together and not all develop > separate versions of the core. What do you think? Benjamin may also be able > to help you with your query below. > > Food for thought. The time lines may not be possible, but we should try > and converge. > > Kind regards, > > Adam Isaacson > SARAO > DBE Hardware Manager > Cell: (+27) 825639602 <+27%2082%20563%209602> > Tel: (+27) 215067300 <+27%2021%20506%207300> > email: aisaac...@ska.ac.za > > > On Tue, Feb 27, 2018 at 1:34 PM, Adam Isaacson <aisaac...@ska.ac.za> > wrote: > >> Hi Arash, >> >> I am afraid I don't have the expertise on this yet or a solution to this >> problem. We also have a VCU118 board and will be implementing a 100GbE >> yellow block in the months to come, so it would be good to stay in touch >> about this - maybe not duplicate work, but help one another if time >> pressures allow. >> >> Kind regards, >> >> Adam Isaacson >> SARAO >> DBE Hardware Manager >> Cell: (+27) 825639602 <+27%2082%20563%209602> >> Tel: (+27) 215067300 <+27%2021%20506%207300> >> email: aisaac...@ska.ac.za >> >> >> On Mon, Feb 26, 2018 at 10:25 PM, Jack Hickish <jackhick...@gmail.com> >> wrote: >> >>> Hi Arash, >>> >>> It's awesome that you're doing this. >>> >>> I only have experience bringing up the 10GbE core on SNAP, but a few >>> things you might try are: >>> - Make sure all the (many!) clocks are running at the correct speed -- >>> easy to misconfigure. >>> - Check and double check resets >>> - Setting the transceivers into loopback mode, and checking for sanity. >>> - Using a QSFP loopback module to similar effect. I believe one is >>> supplied with the VCU kit. >>> >>> I'm pretty sure I have less expertise in this than you, but they're my >>> first thoughts. >>> >>> I assume that the infiniband switch is dual-personality, and capable of >>> establishing a 100GbE link? >>> >>> Anyway, it's great that you're working on this, >>> >>> Jack >>> >>> >>> >>> On Mon, 26 Feb 2018 at 12:13 Roshanineshat, Arash < >>> arash.roshanines...@cfa.harvard.edu> wrote: >>> >>>> Hi >>>> >>>> I'm trying to implement a design using UltraScale+ 100G Ethernet on a >>>> VCU118 board. I use Vivado 2017.4 and UltraScale+ 100G Ethernet IP >>>> core subsystem version 2.4. >>>> >>>> I'll give you a quick background about what I have done so far, please >>>> let me know if I took a wrong step at any point. >>>> >>>> I want to implement OSI model which consists of following layers: >>>> >>>> 7-Application layer >>>> 6-Presentation layer >>>> 5-Session Layer >>>> 4-Transport >>>> 3-Network Layer >>>> 2-Data Link Layer >>>> 1-Physical layer >>>> >>>> According to what I learned from Xilinx's forum community, 100G IP >>>> core will take care of Ethernet protocol (Layers 1 and 2) and I have >>>> to implement Layers 3 and 4. However, Linux operating system will take >>>> care of Layers 3 and 4 for me. Therefore, I only need to generate the >>>> data and use GTY transceivers to transfer them to the network. Right? >>>> >>>> To customize GTY Transceivers to use QSFP1, and to test it I did the >>>> following: >>>> >>>> 1-As GTY documentation says, QSFP1 is located in left side quads and >>>> in quad 231. >>>> 2-I selected X1Y48~51 channels of CMACE4 X0Y7. I chose this value >>>> because I saw it will put the transceivers on quad bank 231. >>>> 3-I chose CAUI4 mode (4 lanes x 25.7812G) Simple TX. >>>> 4-I generated an IP core using the above settings. >>>> 5-I generated an Open IP Example design using the IP generated above >>>> to experiment with it. >>>> 6-I connected gt_refclks to QSFP_SI570_CLOCK_C_P/N for QSFP1 >>>> 7-I initiated a IBUFS to convert a LVDS clock to single-ended clock >>>> and connected it to init_clk >>>> 8-Create a xdc file according to my VCU118 pinouts. >>>> 9-generated the bitcode >>>> >>>> I loaded the bitcode to the FPGA board and I noticed TX_gt_locked, >>>> TX_done, TX_busy LEDs are blinking correctly. So I *guess* the data is >>>> being generated too. the problem I have now is that I cannot bring up >>>> the QSFP interface and our 100G infiniband switch, reports that the >>>> QSFP port connected to the FPGA is "down". >>>> >>>> What I'm suspicious about is that I have selected a wrong GTY >>>> transceiver configuration, however, It seems to me, everything is >>>> according to the documentation and the other reason I can think of is >>>> the clock values. >>>> >>>> I would be grateful if you could advise me on possible solutions or >>>> debugging methods that I can try. >>>> >>>> Best Regards >>>> Arash Roshanineshat >>>> >>>> -- >>>> You received this message because you are subscribed to the Google >>>> Groups "casper@lists.berkeley.edu" group. >>>> To unsubscribe from this group and stop receiving emails from it, send >>>> an email to casper+unsubscr...@lists.berkeley.edu. >>>> To post to this group, send email to casper@lists.berkeley.edu. >>>> >>> -- >>> You received this message because you are subscribed to the Google >>> Groups "casper@lists.berkeley.edu" group. >>> To unsubscribe from this group and stop receiving emails from it, send >>> an email to casper+unsubscr...@lists.berkeley.edu. >>> To post to this group, send email to casper@lists.berkeley.edu. >>> >> >> > -- You received this message because you are subscribed to the Google Groups "casper@lists.berkeley.edu" group. To unsubscribe from this group and stop receiving emails from it, send an email to casper+unsubscr...@lists.berkeley.edu. To post to this group, send email to casper@lists.berkeley.edu.
