Hi Casperites, Okay, so it turns out that creating the virtual environment for python using "virtualen -p python3 <name_of_env>" does not work. This is what you need to do in order to create a successful virtual environment and get your designs to build. I tested it on a brand new virtual environment and it works. Thanks to Clifford van Wyk (Peralex), Morag Brown and Jack Hickish for their assistance. The docs will definitely need to be updated - we will add this to the agenda for the CASPER meeting.
The following needs to be done if you want to generate a proper virtual environment that will build in the toolflow - Kaj, I recommend the below way: 1) Create the virtual environment: "python3 -m venv <some-venv-name>". The old way of using virtualenv -p python 3 <some-venv-name> just doesn't work. Thanks to Jack and Morag for pointing this out to me. 2) Activate the environment: "source <some-venv-name>/bin/activate" 3) Go to the mlib_devel directory and edit the requirements.txt file. It should have "numpy<1.9" (thanks for your sleuth work, Jack!). Now save the file. 3) Go to mlib_devel directory and type exactly: "pip install -r requirements.txt". This will install without error or issues. if you check the site-packages in the virtual environment you will see what I mean - all the python packages will be installed properly in your virtual environment. 4) You will need to edit line 32 of castro.py (located in the same folder as mlib_devel/jasper_library), so that it is "c = yaml.load(fh, Loader=yaml.Loader) . Refer to https://github.com/yaml/pyyaml/issues/266 <https://github.com/yaml/pyyaml/issues/266>for more info on this version change issue. I tested the build with my original working virtualenv and the newly generated virtualenv and it works for both. Don't worry, I will be committing this change once I have done a bit more investigation - maybe adding versions in requirements.txt is not a bad idea for future support. You can now start your matlab session and build your designs :). Hooray! Kind regards, Adam Isaacson South African Radio Astronomy Observatory (SARAO) Hardware Manager Cell: (+27) 825639602 Tel: (+27) 215067300 email: aisaac...@ska.ac.za On Thu, Jan 28, 2021 at 9:55 AM Adam Isaacson <aisaac...@ska.ac.za> wrote: > Hi David, > > Yes, it was a bit of a rabbit hole exercise, but we (between Kaj and > Peralex) worked our way through it and now debugging other issues :). > > You raise a valid point. There is no reason we shouldn't port to Ubuntu > 18.04LTS or higher. It makes sense with what Kaj and Peralex are > experiencing. > > Please note, my team (SARAO) is baselining the SKARAB work at Ubuntu > 16.04LTS. We are coming to the end of the MeerKAT development for the F and > X engines. I do believe there are those that have got the toolflow to work > on Ubuntu 18.04LTS. I know Wesley New has struggled to get the SKARAB > toolflow to work using Ubuntu 18.04LTS. I haven't tried it on my side yet. > > My team's focus this year is to update the toolflow to work with Vitis and > target those Alveo boards. This will definitely involve moving to a newer > version of Ubuntu. For now that is likely to be Ubuntu 18.04LTS, Vitis > 2020.2 and Vivado 2020.2. > > I think this is something we should discuss at the next CASPER meeting on > the 11th Feb. We should allocate a resource for this, but SARAO is focusing > on other areas for now. > > Thanks for your email :). > > Kind regards, > > Adam Isaacson > South African Radio Astronomy Observatory (SARAO) > Hardware Manager > Cell: (+27) 825639602 > Tel: (+27) 215067300 > email: aisaac...@ska.ac.za > > > > On Thu, Jan 28, 2021 at 12:09 AM David MacMahon <dav...@berkeley.edu> > wrote: > >> Wow, thanks for these truly awesome forensics, Adam! It sounds like you >> went down a rabbit hole and lived to tell the tale. I'm sure many of us >> will benefit from these details. >> >> Sorry if this is a FAQ, but what are the prospects for moving the tool >> flow beyond Ubuntu 16.04? That release is really starting to show its age, >> especially since its stock Python3 is only 3.5. (Though to be fair, Python >> is at least partly to blame due to its poor track record on version >> compatibility. :P) >> >> Cheers, >> Dave >> >> On Jan 27, 2021, at 13:00, Adam Isaacson <aisaac...@ska.ac.za> wrote: >> >> Hi Kaj, >> >> I am including the CASPER community in this email thread as it applies to >> everyone. >> >> Interesting, so I have run into another person with the same virtualenv >> install issue that you encountered as shown in red below. I have been >> helping him debug too on a new machine and I am pleased to say that his >> virtualenv is now working. It seems like the new Ubuntu 16.04LTS installs >> come with a newer version of virtualenv which is not compatible with python >> 3.5 - fun, fun, fun. This definitely applies to all Casperites who are >> installing the toolflow on their new machines. >> >> kjwiik@casperx:~/work$ virtualenv -p python3 casper_venv >> Traceback (most recent call last): >> File "/usr/local/bin/virtualenv", line 7, in <module> >> from virtualenv.__main__ import run_with_catch >> File >> "/home/kjwiik/.local/lib/python3.5/site-packages/virtualenv/__init__.py", >> line 3, in <module> >> from .run import cli_run, session_via_cli >> File >> "/home/kjwiik/.local/lib/python3.5/site-packages/virtualenv/run/__init__.py", >> line 13, in <module> >> from .plugin.activators import ActivationSelector >> File >> "/home/kjwiik/.local/lib/python3.5/site-packages/virtualenv/run/plugin/activators.py", >> line 6, in <module> >> from .base import ComponentBuilder >> File >> "/home/kjwiik/.local/lib/python3.5/site-packages/virtualenv/run/plugin/base.py", >> line 9, in <module> >> from importlib_metadata import entry_points >> File >> "/home/kjwiik/.local/lib/python3.5/site-packages/importlib_metadata/__init__.py", >> line 88 >> dist: Optional['Distribution'] = None >> ^ >> SyntaxError: invalid syntax >> kjwiik@casperx:~/work$ >> >> I want you to check the following versions for me: >> >> 1) virtualenv -> type "virtualenv --version" at the prompt. You should >> get 16.7.5. If not then you will need to uninstall the virtualenv by >> typing: "pip3 uninstall virtualenv" and then we will need to install the >> 16.7.5 version. To install an exact version then type: "pip3 install >> virtualenv==16.7.5". The use of "sudo" with "-H" arguments may or may >> not be needed. The installs will guide you. >> >> 2) What is your Ubuntu 16.04LTS version? -> type "lsb_release -a". I am >> using Ubuntu 16.04.7 LTS, Xenial. >> >> 3) What is your pip3 version? -> type "pip3 --version". I am using pip >> 8.1.1. >> >> 4) What is your python version? -> type: "python --version". I am using >> python version 2.7.12 >> >> 5) What is your python3 version? -> type: "python3 --version". I am using >> python3 version 3.5.2. >> >> Please make sure all these versions are the same before continuing. I >> suspect that the latest Ubuntu 16.04LTS installs have made some upgrades >> that are incompatible. >> >> Once these versions are as above then try the following step to create >> the virtual env: >> >> 1) Type: "virtualenv -p python3 <name_of_virtual_env>". This should >> create a folder with the same name as "name_of_virtual_env". There >> should be no issues with the install. It should complete without issues. >> 2) To activate the virtual environment -> type: "source >> <path_to_folder>/<name_of_virtual_env>/bin/activate >> 3) Once activated then you can deactivate it by typing "deactivate". >> 4) If the virtual environment is activated then if you type "python" you >> should get version 3.5.2 >> 5) If the virtual environment is deactivated then if you type "python" >> you should get 2.7.12. >> >> I think if you can achieve this then you are ready to install the >> toolflow by following these steps: >> >> 1) >> https://casper-toolflow.readthedocs.io/en/latest/src/Installing-the-Toolflow.html# >> 2) >> https://casper-toolflow.readthedocs.io/en/latest/src/How-to-install-Matlab.html >> 3) >> https://casper-toolflow.readthedocs.io/en/latest/src/How-to-install-Xilinx-Vivado.html >> 4) >> https://casper-toolflow.readthedocs.io/en/latest/src/Configuring-the-Toolflow.html >> 5) >> https://casper-toolflow.readthedocs.io/en/latest/src/Running-the-Toolflow.html >> 6) https://casper-toolflow.readthedocs.io/projects/casperfpga/en/latest/ >> (This >> is how to install casperfpga) >> >> @Jonathon Kocz <jxk...@gmail.com> : we will need to update the docs to >> reflect this new information. I will wait until both parties have working >> systems before updating the ReadtheDocs. One of the parties is using a >> proxy and has some additional information on how to install with a proxy >> that will be useful for other casperites. >> >> I hope this helps! >> >> Kind regards, >> >> Adam Isaacson >> South African Radio Astronomy Observatory (SARAO) >> Hardware Manager >> Cell: (+27) 825639602 >> Tel: (+27) 215067300 >> email: aisaac...@ska.ac.za >> >> >> >> On Sun, Jan 24, 2021 at 8:09 PM Adam Isaacson <aisaac...@ska.ac.za> >> wrote: >> >>> Hi Kaj, >>> >>> Are you running a python virtual environment using python 3.5? Once you >>> start the virtual environment and type python you should see version 3.5.2 >>> or something like that. It doesn't look like it. Here is some documentation >>> that should be helpful - make sure you follow these steps: >>> >>> 1) How to configure the toollflow: >>> https://casper-toolflow.readthedocs.io/en/latest/src/Configuring-the-Toolflow.html >>> 2) Running the toolflow: >>> https://casper-toolflow.readthedocs.io/en/latest/src/Running-the-Toolflow.html >>> >>> Does your startsg.local file look like this: >>> >>> export XILINX_PATH=/opt/Xilinx/Vivado/2019.1export >>> MATLAB_PATH=/usr/local/MATLAB/R2018aexport PLATFORM=lin64export >>> JASPER_BACKEND=vivado >>> # over-ride the MATLAB libexpat version with the OS's one.# Using >>> LD_PRELOAD=${LD_PRELOAD}:"..." rather than just LD_PRELOAD="..."# ensures >>> that we preserve any other settings already configuredexport >>> LD_PRELOAD=${LD_PRELOAD}:"/usr/lib/x86_64-linux-gnu/libexpat.so" >>> # Activate a custom python environment on loadexport >>> CASPER_PYTHON_VENV_ON_START=/home/user/work/casper_venv >>> >>> >>> You must run everything from the virtual environment in order to work. >>> The last line in the startsg.local will automatically start your virtual >>> environment session, but you can also do it manually e.g. source >>> ~/venv/bin/activate will give you the following "(venv) >>> aisaacson@adam-cm:~/work/git_work/ska-sa/2019_1/mlib_devel$". The red >>> (venv) tells you that you are in the virtual python environment. >>> Remember you also need to install requirements.txt while the virtual >>> environment is running. >>> >>> Then all you really need to do is run "./startsg" at the prompt and it >>> should open up Matlab where you can build your slx file by typing in >>> "jasper" or "jasper_frontend. if you use "jasper_frontend" then it will >>> just compile sysgen and the simulink frontend. You can then open another >>> terminal session and under "mlib_devel" type "source startsg >>> startsg.local". This will make sure everything is in your path. You can >>> then copy and paste the exec_flow.py command from the Matlab command window >>> and then it should build the middle end and backend. Remember to start the >>> virtual environment for this terminal window as well. This should all be in >>> the above docs, but if you are still struggling then please send us a step >>> by step terminal screen capture, so we can see if you are doing everything >>> correctly, thanks. >>> >>> I hope this helps! >>> >>> Kind regards, >>> >>> Adam Isaacson >>> South African Radio Astronomy Observatory (SARAO) >>> Hardware Manager >>> Cell: (+27) 825639602 >>> Tel: (+27) 215067300 >>> email: aisaac...@ska.ac.za >>> >>> >>> >>> On Sun, Jan 24, 2021 at 7:15 PM Kaj Wiik <kjw...@utu.fi> wrote: >>> >>>> Hi! >>>> >>>> A followup... >>>> >>>> I was able to start (but not run, see below...) the toolflow with the >>>> following modifications: >>>> >>>> - to run jasper_frontend, I had to install python-yaml. I tried first to >>>> >>>> add python3-yaml but it seems that the toolflow is mixing python2 and >>>> python3? >>>> >>>> - I added 'export LM_LICENSE_FILE=/opt/Xilinx/Xilinx.lic' and >>>> 'export MLIB_DEVEL_PATH=~/mlib_devel' to startsg.local >>>> >>>> When matlab gave a command line and I run it, I got the following error >>>> >>>> message that I do not (yet!) understand: >>>> >>>> kjwiik@casperx:~/tmp$ source ../mlib_devel/startsg.local >>>> kjwiik@casperx:~/tmp$ /usr/bin/python >>>> /home/kjwiik/mlib_devel/jasper_library/exec_flow.py -m >>>> /home/kjwiik/mlib_devel/jasper_library/test_models/test_zcu111.slx >>>> --middleware --backend --software >>>> Starting compile >>>> Starting Toolflow! >>>> Frontend is simulink >>>> Setting compile directory: >>>> /home/kjwiik/mlib_devel/jasper_library/test_models/test_zcu111 >>>> /home/kjwiik/mlib_devel/jasper_library/platforms/zcu111.yaml >>>> {'backend_target': 'vivado', 'name': 'zcu111', 'fpga': >>>> 'xczu28dr-ffvg1517-2-e', 'mmbus_architecture': 'AXI4-Lite', 'sources': >>>> [], 'cfgbvs': 'GND', 'provides': ['sys_clk'], 'config_voltage': 1.8, >>>> 'pins': {'clk_100_p': {'loc': 'AM15', 'iostd': 'LVDS'}, 'clk_100_n': >>>> {'loc': 'AN15', 'iostd': 'LVDS'}, 'led': {'loc': ['AR13', 'AP13', >>>> 'AR16', 'AP16', 'AP15', 'AN16', 'AN17', 'AV15'], 'iostd': 'LVCMOS18'}}, >>>> >>>> 'mmbus_base_address': 2684354560, 'mmbus_address_alignment': 4, >>>> 'manufacturer': 'Xilinx', 'constraints': []} >>>> sw_reg >>>> Traceback (most recent call last): >>>> File "/home/kjwiik/mlib_devel/jasper_library/exec_flow.py", line 197, >>>> >>>> in <module> >>>> tf.gen_periph_objs() >>>> File "/home/kjwiik/mlib_devel/jasper_library/toolflow.py", line 363, >>>> >>>> in gen_periph_objs >>>> self.peripherals[pk], self.plat)) >>>> File >>>> "/home/kjwiik/mlib_devel/jasper_library/yellow_blocks/yellow_block.py", >>>> >>>> line 65, in make_block >>>> return cls(blk,platform,hdl_root=hdl_root) >>>> File >>>> "/home/kjwiik/mlib_devel/jasper_library/yellow_blocks/yellow_block.py", >>>> >>>> line 92, in __init__ >>>> self.hdl_root = os.getenv('HDL_ROOT').rstrip('/') #: The base >>>> directory from which source file's locations are specified >>>> AttributeError: 'NoneType' object has no attribute 'rstrip' >>>> ----------------------------------------------------------------------- >>>> >>>> Could this be again a python version problem? >>>> >>>> Thanks, >>>> Kaj >>>> >>>> On 1/22/21 6:50 PM, Kaj Wiik wrote: >>>> > Hi! >>>> > >>>> > I tried a bit different route: I installed python3-numpy and >>>> > python3-setuptools-git Ubuntu packages first and commented out numpy >>>> > from requirements.txt. At least the installation went fine using these >>>> >>>> > workarounds. >>>> > >>>> > About ZCU111 Slack channel, I am very interested (and my colleague >>>> > derek.mc...@utu.fi also), could you please send us an invitation? >>>> > >>>> > Many thanks for your kind help! >>>> > >>>> > Kaj >>>> > >>>> > On 1/20/21 10:22 PM, Dan Werthimer wrote: >>>> >> >>>> >> hi kaj, >>>> >> >>>> >> regarding your interest in ZCU111: >>>> >> there's a casper slack channel on RFSOC that you might find useful. >>>> >> wei liu recently developed a casper ADC yellow block for the ZCU111. >>>> >> >>>> >> best wishes, >>>> >> >>>> >> dan >>>> >> >>>> >> >>>> >> >>>> >> Dan Werthimer >>>> >> Marilyn and Watson Alberts Chair >>>> >> Astronomy Dept and Space Sciences Lab >>>> >> University of California, Berkeley >>>> >> >>>> >> >>>> >> On Wed, Jan 20, 2021 at 12:17 PM Kaj Wiik <kaj.w...@utu.fi >>>> >> <mailto:kaj.w...@utu.fi>> wrote: >>>> >> >>>> >> >>>> >> >>>> >> On 20/01/2021 18:49, Jack Hickish wrote: >>>> >> >>>> >> > I've been using Ubuntu 18.04 LTS without issues, at least with >>>> >> the boards which use Vivado 2019. I'm not sure what versions of >>>> >> python the OS came with, but I'm currently running the toolflow >>>> in a >>>> >> python 3.6.9 virtual env. >>>> >> >>>> >> I first tried with Ubuntu 18.04 LTS but I got stuck with a >>>> problem >>>> >> with Matlab and some system library versions. >>>> >> >>>> >> I think I'll change the requirements.txt file to read >>>> "numpy<1.19" >>>> >> and try with that first. >>>> >> >>>> >> I am very interested of all information and experiences with >>>> Casper >>>> >> on ZCU111. Thanks to Adam for pointing out of a mlib_devel ZCU111 >>>> >> branch, I haven't noticed that! >>>> >> >>>> >> Thanks! >>>> >> >>>> >> Kaj >>>> >> >>>> >> -- You received this message because you are subscribed to >>>> the >>>> >> Google >>>> >> Groups "casper@lists.berkeley.edu >>>> >> <mailto:casper@lists.berkeley.edu>" group. >>>> >> To unsubscribe from this group and stop receiving emails from it, >>>> >> send an email to casper+unsubscr...@lists.berkeley.edu >>>> >> <mailto:casper%2bunsubscr...@lists.berkeley.edu>. >>>> >> To view this discussion on the web visit >>>> >> >>>> >> >>>> https://groups.google.com/a/lists.berkeley.edu/d/msgid/casper/020a6f84-ded9-49cc-2d4a-00a766e743be%40utu.fi >>>> . >>>> >> >>>> >> >>>> >> -- >>>> >> You received this message because you are subscribed to the Google >>>> >> Groups "casper@lists.berkeley.edu" group. >>>> >> To unsubscribe from this group and stop receiving emails from it, >>>> send >>>> >> an email to casper+unsubscr...@lists.berkeley.edu >>>> >> <mailto:casper+unsubscr...@lists.berkeley.edu>. >>>> >> To view this discussion on the web visit >>>> >> >>>> https://groups.google.com/a/lists.berkeley.edu/d/msgid/casper/CAGHS_vHQ1bTha%3DFjCe0sGddd3Gt6GrYkh8dgz_eKDWXebf87iQ%40mail.gmail.com >>>> >>>> >> < >>>> https://groups.google.com/a/lists.berkeley.edu/d/msgid/casper/CAGHS_vHQ1bTha%3DFjCe0sGddd3Gt6GrYkh8dgz_eKDWXebf87iQ%40mail.gmail.com?utm_medium=email&utm_source=footer >>>> >. >>>> >> >>>> > >>>> >>>> -- >>>> You received this message because you are subscribed to the Google >>>> Groups "casper@lists.berkeley.edu" group. >>>> To unsubscribe from this group and stop receiving emails from it, send >>>> an email to casper+unsubscr...@lists.berkeley.edu. >>>> To view this discussion on the web visit >>>> https://groups.google.com/a/lists.berkeley.edu/d/msgid/casper/335be9c2-39ca-d9f9-fb8f-960076bfffa1%40utu.fi >>>> . >>>> >>> >> -- >> You received this message because you are subscribed to the Google Groups >> "casper@lists.berkeley.edu" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to casper+unsubscr...@lists.berkeley.edu. >> To view this discussion on the web visit >> https://groups.google.com/a/lists.berkeley.edu/d/msgid/casper/CADTJ%3DnGgmuxdVex74bZPBOn0kpwxMO4rPrUyGcu_PPHvgvU8XA%40mail.gmail.com >> <https://groups.google.com/a/lists.berkeley.edu/d/msgid/casper/CADTJ%3DnGgmuxdVex74bZPBOn0kpwxMO4rPrUyGcu_PPHvgvU8XA%40mail.gmail.com?utm_medium=email&utm_source=footer> >> . >> >> >> -- >> You received this message because you are subscribed to the Google Groups >> "casper@lists.berkeley.edu" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to casper+unsubscr...@lists.berkeley.edu. >> To view this discussion on the web visit >> https://groups.google.com/a/lists.berkeley.edu/d/msgid/casper/34A86FAB-52C2-4923-83A3-B14BD83A8F2A%40berkeley.edu >> <https://groups.google.com/a/lists.berkeley.edu/d/msgid/casper/34A86FAB-52C2-4923-83A3-B14BD83A8F2A%40berkeley.edu?utm_medium=email&utm_source=footer> >> . >> > -- You received this message because you are subscribed to the Google Groups "casper@lists.berkeley.edu" group. To unsubscribe from this group and stop receiving emails from it, send an email to casper+unsubscr...@lists.berkeley.edu. To view this discussion on the web visit https://groups.google.com/a/lists.berkeley.edu/d/msgid/casper/CADTJ%3DnFZ3q-zK3La9b_vtUs5dLi8Sjw-EUGpjdzei1SxsPdw6Q%40mail.gmail.com.
