Hi Jeff, I am sorry to hear that you are struggling to program the SKARAB. The Python version needs to be 2.7, which your screen capture seems to show. I have only ever used the non-virtual Python 2.7 version for Red Pitaya and SKARAB when it comes to casperfpga, but I use the virtual Python 3.6 when it comes to the toolflow for both. That is because we ported to Python 3 for the toolflow and baselined SKARAB with the following versions, as show below:
Toolflow: Ubuntu 18.04LTS, Python 3.6 (Virtual environment), Matlab R2018a, Vivado 2019.1.1. casperfpga: Ubuntu 18.04LTS, Python 2.7 (non-Virtual environment) There was some effort made to get casperfpga to work with Python 3, but some functions work and others don't e.g. the programming. You are using the correct Python version for casperfpga, so this should not be an issue. Red Pitaya uses tcpborphserver, like ROACH2 did as it has a Zynq processor onboard, but SKARAB does not have a host processor (it is possible to add one, if needed) on board to eliminate any RFI. It uses the microblaze and does not use the KATCP protocol like ROACH2 did. Therefore, you can't compare the two. SKARAB also utilises a C program called "Progska" within casperfpga, which is meant to improve the network offloading from the 40GbE NIC in order to meet the throughput requirements for the SKARAB. This also includes the programming requirements. If progska is not properly installed then you could very well have issues. This often occurs when you do a pip install for SKARAB - see point 5) below. Therefore, let's try the following: 1) What version of casperfpga are you using? Type: casperfpga.__version__ in Ipython and let me know what it returns? 2) What casperfpga repo are you using and what is the githash? What toolflow repo are you using and what is the githash? 3) Are you running the 40GbE via a switch as well? If so, are the switch ports set to MTU 9000? I don't think this is the issue though based on the screen capture. 4) You can ping the copper and the fibre? Are you working with both the 1GbE and the 40GbE? Can you try to program via the 1GbE interface and then the 40GbE interface - both must have the host MTU 9000 and the Switch ports must have an MTU 9000. Can you do a ping flood to full MTU size: "ping -f -i 2ms -s 8192"? Are any packets dropped via ping? This will also tell whether the MTU size = 9000 is correct? 5) Did you have any issues or errors with progska when you installed casperfpga? Can you provide me with some screen shots of the casperfpga install process? Did you install via pip - see <goog_631249138> https://casper-toolflow.readthedocs.io/en/latest/src/How-to-install-casperfpga.html or via this method, which I use: https://github.com/ska-sa/casperfpga ("sudo python setup.py install"). I have never been able to successfully use pip install for SKARAB as it always left out some dependencies that were missing. Anyway, I hope this helps you debug. You are definitely using the right casperfpga Python version for SKARAB. I suspect that you may have installed casperfpga via pip and not "sudo python setup.py install" and this created some install issues. The majority of CASPER boards work with Python 3.8 now and that is why the CASPER documentation uses this version and pip install. SKARAB differs mostly when it comes to the casperfpga install, Python version and progska for programming. Kind regards, Adam On Wed, Aug 30, 2023 at 11:03 PM 'Kobesky, Jeffrey CIV USN NRL (5533) Washington DC (USA)' via [email protected] < [email protected]> wrote: > Hi, I think my questions are for Adam and Co., I can program a .fpg file > into Red Pitaya but I cannot program one into SKARAB. Please see > attachments, note errors in SKARAB programming screenshot and also note > fpga.is_connected()=True. I’m using: Toolflow: Ubuntu 18.04LTS, > Python > > > > > > > > > > > > > > > > > > > > > > > > > > > <https://za.report.cybergraph.mimecast.com/alert-details/?dep=yl2lXqDYoP%2FiKjCsaJIVMA%3D%3DfwMNStY6Uuq%2BYvlJ4lVaGrQcX26DRKUPwZc%2BvyCbpXUQu6FzyJvDV%2FMpaVF3oCSkfIRJVePac2mMYk2W2658eyA5Z0IvCw2Is%2BGou97032a1MIqtqD1qSbCtq%2FWWAeww8eqUHmdRrNtGgyrzNLEtiw6Zj2tPgRriMlyepKjsBt45Sjl7cm38ObvYRtnaNZObdxC%2BKeCXwlHn75G00Lks0c8sdHAFzIA0C9vwzHCGwOd%2BAx9cy8fKEByhGCSwCaR3RlSZiP9Du%2FC8PMw5NqCdvwAAXTgKnuVM%2FiW6K%2FqaOZhcFZ64mZUrbhfKYXXaNEhwgZWcb2ojZHzdEodPzCoqq5mUZOfGINtMHs%2BJm3pSak%2FK2hRTWs8t%2BbVP%2FqYp6v4ZMLCiMs5ysORc%2BdY0aZ5yM0oyRWrwZvA8%2BnAWPP6ws%2Fv88pL5uGUUsVEQ5mzmiwyntBXYI%2FOusjgt3OADiYhG8%2FQldl0va%2BrQ67ghk3M2oTnn%2BzriGIppWPhCTvLnBekNnLe1H47fz6JCfRVrdH0vMoT6k7%2BeizmYvPfEVVU12fUmZ84Bt89l3C2WfM6fudzvLNrEsoZ31P5AgO0telOM6h5PawMcary4cOqL2e7pUWUBoz4e6ykb5zs0nxM1jM7jn%2FxplQfivPiuhqkSh405Bb56%2FYSzVko5c8O87OjXdGI%3D> > > Hi, I think my questions are for Adam and Co., > > > > I can program a .fpg file into Red Pitaya but I cannot program one into > SKARAB. Please see attachments, note errors in SKARAB programming > screenshot and also note fpga.is_connected()=True. > > > > I’m using: > > Toolflow: Ubuntu 18.04LTS, Python 3.6 (Virtual environment), Matlab > R2018a, Vivado 2019.1.1. > > casperfpga: Ubuntu 18.04LTS, Python 2.7 (non-Virtual environment) > > > > Note I’m using non-virtual environment for casperfpga. I’ve looked at > older emails from Adam where he used non-virtual for casperfpga, but below > specifies virtual. Has something changed/been discovered with programming > SKARAB? Not sure why my non-virtual environment would work for Pitaya but > not SKARAB. > > > > Things I’ve considered or tried include: > > > > - Able to ping SKARAB copper port and fiber ports. > - Set mtu=9000 on my host, per Peralex documentation. > - Thought of programming SKARAB by hostname instead of IP address, but > don’t know if it even has a hostname or what it would be. > > > > Note this is the first time I’m programming the SKARAB, does something > need to be set up for first time? > > > > Thank you so much in advance! -- Jeff > > > > > > *From:* [email protected] <[email protected]> *On Behalf > Of *Adam Isaacson > *Sent:* Monday, May 29, 2023 4:53 AM > *To:* [email protected] > *Subject:* Re: [casper] Question on Casperfpga python 3.8 version > > > > Dear June, > > > > I am sorry you are encountering this problem. SKARAB has only been > baselined and fully tested using Python 2.7 for casperfpga. Yes, some > attempts have been made to port it to Python 3.8, but as far as I am aware > there have been issues that were never fully sorted out. It looks like you > may be encountering those. > > > > I would try to run it using Python 2.7 first. NB: SKARAB baseline testing > suggested versions: > > > > *Toolflow*: Ubuntu 18.04LTS, Python 3.6 (Virtual environment), Matlab > R2018a, Vivado 2019.1.1. > > *casperfpga*: Ubuntu 18.04LTS, Python 2.7 (Virtual environment) > > > > It should be pointed out that work has been done on the toolflow to get it > to work with Ubuntu 18.04LTS, Python 3.8, Matlab R2021a and Vivado 2021.1. > NB: Please note that we had issues running some of the tutorials and that > the base driver IP for Ethernet and most drivers remain at the Vivado > 2019.1.1 version for SKARAB, so it is not fully ported at all levels. > > > > It should be pointed out that at this point our DSP team are currently > using SKARABs for new development with these new versions, so they may end > up sorting all these issues out in the long run, but if you want a fully > baselined and tested SKARAB version now then I would stick with the > recommended baseline versions. > > > > Good luck! > > > > Kind regards, > > > > Adam > > > > On Mon, May 29, 2023 at 9:56 AM June Tantiparimongkol <[email protected]> > wrote: > > Dear all, > > > > Due to the upgrade of '*casperfpga*' library from python 2.7 to 3.8, I > have tried to upgrade my software to a newer version. But still got stuck > on problems that I still can't properly install and use it as I have > followed the information on > https://casper-toolflow.readthedocs.io/en/latest/src/How-to-install-casperfpga.html > <https://casper-toolflow.readthedocs.io/en/latest/src/How-to-install-casperfpga.html>. > I have pulled '*casperfpga*' repository from > https://github.com/casper-astro/casperfpga.git > <https://github.com/casper-astro/casperfpga.git> on > *py38* branch > > > > During installation, I have found error, "*katcp==0.9.1 required tornado > <5 >=4.3*". So, I have re-install it by define "*tornado<5*" in > requirements.txt and do pip install. Later, it shows another error, > "*circus==0.18.0 > required tornado>5*", which circus is defined in *setup.py * > > > > Later, when finish the installation procedure, I am able to import ' > *casperfpga*' library, connect with my SKARAB hardware port. But I am > unable to upload and program firmware file (.fpg) to hardware with error > occured as *UnicodeDecodeError: 'utf-8' codec can't decode byte 0x8b in > position 33033: invalid start byte *as detailed belows > > > > Python 3.8.10 (default, Mar 13 2023, 10:26:41) > Type 'copyright', 'credits' or 'license' for more information > IPython 8.12.2 -- An enhanced Interactive Python. Type '?' for help. > > > > > > > *In [1]: import casperfpga In [2]: fpga = > casperfpga.CasperFpga('192.168.91.121') In [3]: fpga.is_connected() Out[3]: > True* > > *In [4]: > fpga.upload_to_ram_and_program('Data_streaming_2ps_40gbe_2022-10-26_2143.fpg')* > 2023-05-29 14:26:02.83 ERROR 192.168.91.121 transport_skarab.py:715 - > Failed to program. > --------------------------------------------------------------------------- > UnicodeDecodeError Traceback (most recent call last) > Cell In[4], line 1 > ----> 1 > fpga.upload_to_ram_and_program('Data_streaming_2ps_40gbe_2022-10-26_2143.fpg') > > File > ~/work/skarab/casperfpga-py3/cfpga_venv/lib/python3.8/site-packages/casperfpga/casperfpga.py:348, > in CasperFpga.upload_to_ram_and_program(self, filename, wait_complete, > initialise_objects, **kwargs) > 345 else: > 346 filename = self.bitstream > --> 348 rv = self.transport.upload_to_ram_and_program( > 349 filename=filename, wait_complete=wait_complete, **kwargs) > 351 if not wait_complete: > 352 return True > > File > ~/work/skarab/casperfpga-py3/cfpga_venv/lib/python3.8/site-packages/casperfpga/transport_skarab.py:712, > in SkarabTransport.upload_to_ram_and_program(self, filename, port, timeout, > wait_complete, skip_verification, **kwargs) > 709 chunk_size = 1988 > 711 try: > --> 712 upload_time = self.upload_to_ram(filename, not > skip_verification, chunk_size) > 714 except: > 715 self.logger.error('Failed to program.') > > File > ~/work/skarab/casperfpga-py3/cfpga_venv/lib/python3.8/site-packages/casperfpga/transport_skarab.py:641, > in SkarabTransport.upload_to_ram(self, filename, verify, chunk_size) > 638 raise ValueError(errmsg) > 639 # else: Continue! > --> 641 upload_time = skfops.upload_to_ram_progska(filename, > [self.parent], chunk_size) > 642 self.logger.debug('Uploaded bitstream in %.1f seconds.' % > upload_time) > 643 return upload_time > > File > ~/work/skarab/casperfpga-py3/cfpga_venv/lib/python3.8/site-packages/casperfpga/skarab_fileops.py:252, > in upload_to_ram_progska(filename, fpga_list, chunk_size) > 250 processor = choose_processor(filename) > 251 processor = processor(filename, binname) > --> 252 binname = processor.make_bin()[1] > 253 fpga_hosts = [fpga.host for fpga in fpga_list] > 255 # clear sdram of all fpgas before uploading > > File > ~/work/skarab/casperfpga-py3/cfpga_venv/lib/python3.8/site-packages/casperfpga/skarab_fileops.py:75, > in FpgProcessor.make_bin(self) > 71 """ > 72 :return: the name of a produced .bin file > 73 """ > 74 fpg_file = open(self.image_file, 'r') > ---> 75 fpg_contents = fpg_file.read() > 76 fpg_file.close() > 78 # scan for the end of the fpg header > > File /usr/lib/python3.8/codecs.py:322, in > BufferedIncrementalDecoder.decode(self, input, final) > 319 def decode(self, input, final=False): > 320 # decode input (taking the buffer into account) > 321 data = self.buffer + input > --> 322 (result, consumed) = self._buffer_decode(data, self.errors, > final) > 323 # keep undecoded input until the next call > 324 self.buffer = data[consumed:] > > *UnicodeDecodeError: 'utf-8' codec can't decode byte 0x8b in position > 33033: invalid start byte* > > > > From my opinion, I think the error might be occurred by the python library > installation, e.g. tornado version conflict with katcp and circus in > different direction. But I have already tried on install "tornado<5" and > "circus==0.16.0" (which the error of tornado version would not shown.) But > still got *'utf-8' codec can't decode byte 0x8b in position 33033: > invalid start byte* error with the casperfpga version shown as > "casperfpga-3.4..... .*dirty*." > > > > As an unexpert, I really don't know what is the root cause for this > problem. So, anyone please do suggest me about it. > > > > Thanks and regards, > > June > > > > > ------------------------------------------------------------------------------------------------------------------------------ > > National Astronomical Research Institute of Thailand (NARIT) > > 260, Moo 4, Donkaew, Maerim, Chiang mai, 50180 > > > > > > -- > You received this message because you are subscribed to the Google Groups " > [email protected]" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected]. > To view this discussion on the web visit > https://groups.google.com/a/lists.berkeley.edu/d/msgid/casper/CABKTCqET74QWjnNhKMAHMC%2BLq6-h_vgUaisw0i5ngro5Pb%2BzpA%40mail.gmail.com > <https://groups.google.com/a/lists.berkeley.edu/d/msgid/casper/CABKTCqET74QWjnNhKMAHMC%2BLq6-h_vgUaisw0i5ngro5Pb%2BzpA%40mail.gmail.com?utm_medium=email&utm_source=footer> > . > > > > *Disclaimer* > > The information contained in this communication from the sender is > confidential. It is intended solely for use by the recipient and others > authorized to receive it. If you are not the recipient, you are hereby > notified that any disclosure, copying, distribution or taking action in > relation of the contents of this information is strictly prohibited and may > be unlawful. > > This email has been scanned for viruses and malware, and may have been > automatically archived by Mimecast, a leader in email security and cyber > resilience. Mimecast integrates email defenses with brand protection, > security awareness training, web security, compliance and other essential > capabilities. Mimecast helps protect large and small organizations from > malicious activity, human error and technology failure; and to lead the > movement toward building a more resilient world. To find out more, visit > our website. > > -- > You received this message because you are subscribed to the Google Groups " > [email protected]" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected]. > To view this discussion on the web visit > https://groups.google.com/a/lists.berkeley.edu/d/msgid/casper/CADTJ%3DnEH3SiXerVAZppTAzLX8WKiR_DHKV0qohaf-6mOrzjrWw%40mail.gmail.com > <https://groups.google.com/a/lists.berkeley.edu/d/msgid/casper/CADTJ%3DnEH3SiXerVAZppTAzLX8WKiR_DHKV0qohaf-6mOrzjrWw%40mail.gmail.com?utm_medium=email&utm_source=footer> > . > > -- > You received this message because you are subscribed to the Google Groups " > [email protected]" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected]. > To view this discussion on the web visit > https://groups.google.com/a/lists.berkeley.edu/d/msgid/casper/315bf9426d334533b50e2f81b897e732%40nrl.navy.mil > <https://groups.google.com/a/lists.berkeley.edu/d/msgid/casper/315bf9426d334533b50e2f81b897e732%40nrl.navy.mil?utm_medium=email&utm_source=footer> > . > -- Disclaimer The information contained in this communication from the sender is confidential. It is intended solely for use by the recipient and others authorized to receive it. If you are not the recipient, you are hereby notified that any disclosure, copying, distribution or taking action in relation of the contents of this information is strictly prohibited and may be unlawful. This email has been scanned for viruses and malware, and may have been automatically archived by Mimecast, a leader in email security and cyber resilience. Mimecast integrates email defenses with brand protection, security awareness training, web security, compliance and other essential capabilities. Mimecast helps protect large and small organizations from malicious activity, human error and technology failure; and to lead the movement toward building a more resilient world. To find out more, visit our website. -- You received this message because you are subscribed to the Google Groups "[email protected]" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/a/lists.berkeley.edu/d/msgid/casper/CADTJ%3DnH6rnrNs5bnRCm7m-DZTXz2mbsDPdF%3DcXogbx5jf1H48g%40mail.gmail.com.

