Hi Laszlo, Thank you for the feedback. I agree there are some documentation updates required.
The spell check requirements are documented here: https://github.com/tianocore/edk2/blob/master/.pytool/Readme.md#spell-checking---cspell You do need to install nodejs and cspell. * Install nodejs from https://nodejs.org/en/ * Install cspell 1. Open cmd prompt with access to node and npm 2. Run `npm install -g cspell` I agree that the verbosity of the output is very brief. There is a more complete log that is always produced in the build output directory in Build/CI_BUILDLOG.txt. Can you look at that file and see if it provide a more detailed reason for the failure? Thanks, Mike > -----Original Message----- > From: devel@edk2.groups.io <devel@edk2.groups.io> On Behalf Of Laszlo Ersek > Sent: Wednesday, August 19, 2020 5:27 AM > To: edk2-devel-groups-io <devel@edk2.groups.io>; Sean Brogan > <sean.bro...@microsoft.com>; Bret Barkelew > <bret.barke...@microsoft.com> > Cc: Vladimir Olovyannikov <vladimir.olovyanni...@broadcom.com>; Rebecca Cran > <rebe...@bsdio.com>; Tom Lendacky > <thomas.lenda...@amd.com> > Subject: [edk2-devel] running CI locally > > Hi! > > I'd like to test CI locally. I'm going through ".pytool/Readme.md" with > the tree checked out at 7e6f150b6902 (= current HEAD). I'm doing this in > a RHEL8 VM, with a python3 virtual environment set up / entered. > > > * My first note is that the command > > pip install --upgrade pip-requirements.txt > > under "Prerequisets", has a small typo; it should be > > pip install --upgrade -r pip-requirements.txt > > (the "-r" option is missing). > > ( > > After adding "-r", the following components are now installed in my > virtual env: > > - edk2-pytool-library: 0.10.12 > - edk2-pytool-extensions: 0.13.9 > - antlr4-python3-runtime: 4.7.1 > - pyyaml: 5.3.1 > > Stating this because it might matter for the rest of my email. > > ) > > > * Second, when I run the following command: > > stuart_update -c .pytool/CISettings.py TOOL_CHAIN_TAG=GCC5 > > I get the following warnings: > > > WARNING - [SDE] Failed to fetch NugetDependecy: mu_nasm@2.14.02: [Nuget] We > > failed to install this version 2.14.02 of mu_nasm > > WARNING - [SDE] Failed to fetch NugetDependecy: iasl@20190215.0.0: [Nuget] > > We failed to install this version 20190215.0.0 of iasl > > (repeated one more time:) > > > WARNING - [SDE] Failed to fetch NugetDependecy: mu_nasm@2.14.02: [Nuget] We > > failed to install this version 2.14.02 of mu_nasm > > WARNING - [SDE] Failed to fetch NugetDependecy: iasl@20190215.0.0: [Nuget] > > We failed to install this version 20190215.0.0 of iasl > > and then finally: > > > ERROR - We were unable to successfully update 2 dependencies in environment > > ERROR - Error > > The virtual machine has NASM installed (2.13.03-2.el8) and IASL too > (acpica-tools-20180629-3.el8). > > Where do the NASM and IASL version requirements (2.14.02 and > 20190215.0.0, respectively) come from? > > Hm... After a git-grep for those version numbers, I find: > > - BaseTools/Bin/nasm_ext_dep.yaml > - BaseTools/Bin/iasl_ext_dep.yaml > > I was about to say that these version requirements are too strict: for > example, "BaseTools/Conf/tools_def.template" requires "NASM 2.10 or > later for use with the GCC toolchain family". What I have installed > satisfies that, and so CI shouldn't require anything more recent. > *However*, both of the above YAML files have very helpful comments, so I > understand these high versions are downloaded afresh, and only for the > CI run. > > And so my question becomes: why do the "nuget" downloads fail for me > (because, presumably, they work fine in the central CI env on github / > Azure); and how can I fix the issue if it pops up again? > > I've checked "nuget.org" in my browser, and it has: > > - https://www.nuget.org/packages/mu_nasm/ --> 2.14.2 > - https://www.nuget.org/packages/iasl/ --> 20190215.0.0 > > ... On a hunch, I've attempted adding the "--verbose" option to the > "stuart_update" command; this is the output (excerpt): > > > DEBUG - Verify 'mu_nasm' returning 'False'. > > DEBUG - Verify 'iasl' returning 'False'. > > DEBUG - Creating 4 threads for the SDE update > > UpdatingDEBUG - Verify 'mu_nasm' returning 'False'. > > DEBUG - Verify 'gcc_aarch64_linux' returning 'True'. > > DEBUG - Cleaning dependency directory for 'mu_nasm'... > > DEBUG - Verify 'gcc_arm_linux' returning 'True'. > > INFO - Cmd to run is: mono > > /root/py3venv/lib/python3.6/site-packages/edk2toolext/bin/NuGet.exe locals > > global-packages -list > > DEBUG - Verify 'iasl' returning 'False'. > > INFO - ------------------------------------------------ > > DEBUG - Cleaning dependency directory for 'iasl'... > > DEBUG - Verify 'gcc_riscv64_unknown' returning 'True'. > > INFO - --------------Cmd Output Starting--------------- > > INFO - Cmd to run is: mono > > /root/py3venv/lib/python3.6/site-packages/edk2toolext/bin/NuGet.exe locals > > global-packages -list > > INFO - ------------------------------------------------ > > INFO - ------------------------------------------------ > > INFO - --------------Cmd Output Starting--------------- > > INFO - ------------------------------------------------ > > INFO - /bin/sh: mono: command not found > > INFO - /bin/sh: mono: command not found > > INFO - ------------------------------------------------ > > INFO - --------------Cmd Output Finished--------------- > > INFO - --------- Running Time (mm:ss): 00:00 ---------- > > INFO - ----------- Return Code: 0x0000007f ------------ > > INFO - ------------------------------------------------ > > INFO - Nuget was unable to provide global packages cache location. > > INFO - Cmd to run is: mono > > /root/py3venv/lib/python3.6/site-packages/edk2toolext/bin/NuGet.exe install > > mu_nasm -Source > https://api.nuget.org/v3/index.json -ExcludeVersion -NonInteractive -Version > 2.14.02 -Verbosity detailed -OutputDirectory > "/root/src/rhel8/edk2/BaseTools/Bin/mu_nasm_extdep_temp" > > INFO - ------------------------------------------------ > > INFO - --------------Cmd Output Starting--------------- > > INFO - ------------------------------------------------ > > INFO - ------------------------------------------------ > > INFO - --------------Cmd Output Finished--------------- > > INFO - --------- Running Time (mm:ss): 00:00 ---------- > > INFO - ----------- Return Code: 0x0000007f ------------ > > INFO - ------------------------------------------------ > > INFO - Nuget was unable to provide global packages cache location. > > INFO - Cmd to run is: mono > > /root/py3venv/lib/python3.6/site-packages/edk2toolext/bin/NuGet.exe install > > iasl -Source > https://api.nuget.org/v3/index.json -ExcludeVersion -NonInteractive -Version > 20190215.0.0 -Verbosity detailed -OutputDirectory > "/root/src/rhel8/edk2/BaseTools/Bin/iasl_extdep_temp" > > INFO - ------------------------------------------------ > > INFO - --------------Cmd Output Starting--------------- > > INFO - /bin/sh: mono: command not found > > INFO - ------------------------------------------------ > > INFO - ------------------------------------------------ > > INFO - --------------Cmd Output Finished--------------- > > INFO - --------- Running Time (mm:ss): 00:00 ---------- > > INFO - ----------- Return Code: 0x0000007f ------------ > > INFO - ------------------------------------------------ > > WARNING - [SDE] Failed to fetch NugetDependecy: mu_nasm@2.14.02: [Nuget] We > > failed to install this version 2.14.02 of mu_nasm > > Aha! So "mono" is another requirement. > > After installing the "mono-complete" package from EPEL8 (together with > its dependencies), the "stuart_update" command completes fine for me. > > Can we spell out the "mono" dependency in ".pytool/Readme.md" somewhere, > or can we perhaps check for mono automatically? > > > * Third, attempting > > stuart_ci_build -c .pytool/CISettings.py -p OvmfPkg -a IA32,X64 -t DEBUG > TOOL_CHAIN_TAG=GCC5 > > doesn't seem to do anything: > > > SECTION - Init SDE > > SECTION - Loading Plugins > > SECTION - Start Invocable Tool > > SECTION - Getting Environment > > SECTION - Loading plugins > > SECTION - Building OvmfPkg Package > > PROGRESS - --Running OvmfPkg: Compiler Plugin DEBUG -- > > WARNING - --->Test Skipped: in plugin! Compiler Plugin DEBUG > > PROGRESS - Overall Build Status: Success > > SECTION - Summary > > PROGRESS - Success > > If I remove the "-t DEBUG" option, a bit more happens (but it also > fails): > > > SECTION - Init SDE > > SECTION - Loading Plugins > > SECTION - Start Invocable Tool > > SECTION - Getting Environment > > SECTION - Loading plugins > > SECTION - Building OvmfPkg Package > > PROGRESS - --Running OvmfPkg: Char Encoding Check Test NO-TARGET -- > > PROGRESS - --->Test Success: Char Encoding Check Test NO-TARGET > > PROGRESS - --Running OvmfPkg: Compiler Plugin DEBUG -- > > WARNING - --->Test Skipped: in plugin! Compiler Plugin DEBUG > > PROGRESS - --Running OvmfPkg: Compiler Plugin RELEASE -- > > WARNING - --->Test Skipped: in plugin! Compiler Plugin RELEASE > > PROGRESS - --Running OvmfPkg: Dependency Check Test NO-TARGET -- > > PROGRESS - --->Test Success: Dependency Check Test NO-TARGET > > PROGRESS - --Running OvmfPkg: Dsc Complete Check Test NO-TARGET -- > > PROGRESS - --->Test Success: Dsc Complete Check Test NO-TARGET > > PROGRESS - --Running OvmfPkg: Guid Check Test NO-TARGET -- > > PROGRESS - --->Test Success: Guid Check Test NO-TARGET > > PROGRESS - --Running OvmfPkg: Library Class Check Test NO-TARGET -- > > PROGRESS - --->Test Success: Library Class Check Test NO-TARGET > > PROGRESS - --Running OvmfPkg: Spell Check Test NO-TARGET -- > > WARNING - NodeJs not installed. Test can't run > > WARNING - --->Test Skipped: in plugin! Spell Check Test NO-TARGET > > PROGRESS - --Running OvmfPkg: EccCheck Test NO-TARGET -- > > PROGRESS - --->Test Success: EccCheck Test NO-TARGET > > PROGRESS - --Running OvmfPkg: License Check Test NO-TARGET -- > > PROGRESS - --->Test Success: License Check Test NO-TARGET > > PROGRESS - --Running OvmfPkg: Host Unit Test Compiler Plugin NOOPT -- > > WARNING - --->Test Skipped: in plugin! Host Unit Test Compiler Plugin NOOPT > > PROGRESS - --Running OvmfPkg: Host Unit Test Dsc Complete Check Test > > NO-TARGET -- > > PROGRESS - --->Test Success: Host Unit Test Dsc Complete Check Test > > NO-TARGET > > PROGRESS - Overall Build Status: Success > > SECTION - Summary > > PROGRESS - Success > > I'm not sure about the "NodeJs not installed. Test can't run" message -- > I guess it only relates to the "Spell Check Test". > > But what's missing for the actual build? I've noticed the following > warning: > > > Test Skipped: in plugin! Compiler Plugin DEBUG > > But even if I add "--verbose", the only lines around that warning are: > > > INFO - Running on Package: OvmfPkg > > PROGRESS - --Running OvmfPkg: Compiler Plugin DEBUG -- > > WARNING - --->Test Skipped: in plugin! Compiler Plugin DEBUG > > PROGRESS - Overall Build Status: Success > > Why is the build skipped? > > --*-- > > Summary: > > - the typo in "pip install --upgrade -r pip-requirements.txt" should be fixed > > - "mono" should be documented, or auto-installed, as a dependency > > - I don't know why "stuart_ci_build" skips the actual build. > > Thanks! > Laszlo > > > -=-=-=-=-=-=-=-=-=-=-=- Groups.io Links: You receive all messages sent to this group. View/Reply Online (#64447): https://edk2.groups.io/g/devel/message/64447 Mute This Topic: https://groups.io/mt/76285051/21656 Group Owner: devel+ow...@edk2.groups.io Unsubscribe: https://edk2.groups.io/g/devel/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-