[edk2-devel] Buildtools error
Hey all, I got this error when doing a build: build.py... : error C0DE: Unknown fatal error when processing [/edk2/MdePkg/Library/BaseMemoryLibOptPei/BaseMemoryLibOptPei.inf] (Please send email to edk2-de...@lists.01.org for help, attaching following call stack trace!) (Python 2.7.16 on darwin) Traceback (most recent call last): File "/BaseTools/BinWrappers/PosixLike/../../Source/Python/build/build.py", line 2257, in Main MyBuild.Launch() File "/BaseTools/BinWrappers/PosixLike/../../Source/Python/build/build.py", line 2009, in Launch self._MultiThreadBuildPlatform() File "/BaseTools/BinWrappers/PosixLike/../../Source/Python/build/build.py", line 1804, in _MultiThreadBuildPlatform self.Progress File "/BaseTools/Source/Python/AutoGen/AutoGen.py", line 175, in __new__ if not AutoGenObject._Init(Workspace, MetaFile, Target, Toolchain, Arch, *args, **kwargs): File "/BaseTools/Source/Python/AutoGen/AutoGen.py", line 547, in _Init Pkgs = list(PkgSet) + list(PGen.PackageList) File "/BaseTools/Source/Python/AutoGen/AutoGen.py", line 1785, in _GetPackageList for La in self.LibraryAutoGenList: File "/BaseTools/Source/Python/AutoGen/AutoGen.py", line 1895, in _GetLibraryAutoGenList self._GetAutoGenObjectList() File "/BaseTools/Source/Python/AutoGen/AutoGen.py", line 1880, in _GetAutoGenObjectList for La in Ma.LibraryAutoGenList: File "/BaseTools/Source/Python/AutoGen/AutoGen.py", line 3989, in _GetLibraryAutoGenList for Lib in La.CodaTargetList: File "/BaseTools/Source/Python/AutoGen/AutoGen.py", line 3222, in _GetFinalTargetList self._GetTargets() File "/BaseTools/Source/Python/AutoGen/AutoGen.py", line 3208, in _GetTargets if self.SourceFileList: File "/BaseTools/Source/Python/AutoGen/AutoGen.py", line 3042, in _GetSourceFileList self._MatchBuildRuleOrder(self._SourceFileList) File "/BaseTools/Source/Python/AutoGen/AutoGen.py", line 3067, in _MatchBuildRuleOrder FileList.remove(item) ValueError: list.remove(x): x not in list Happy to provide more information but in the meantime, I’ll be trying to figure out what’s going on. Thanks! - Matthew Carlson -=-=-=-=-=-=-=-=-=-=-=- Groups.io Links: You receive all messages sent to this group. View/Reply Online (#77992): https://edk2.groups.io/g/devel/message/77992 Mute This Topic: https://groups.io/mt/84344469/21656 Group Owner: devel+ow...@edk2.groups.io Unsubscribe: https://edk2.groups.io/g/devel/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
Re: [edk2-devel] [PATCH] EmulatorPkg: Add RngLib to satisfy dependency of OpensslLib
Thanks for catching this! -- - Matthew Carlson -=-=-=-=-=-=-=-=-=-=-=- Groups.io Links: You receive all messages sent to this group. View/Reply Online (#65411): https://edk2.groups.io/g/devel/message/65411 Mute This Topic: https://groups.io/mt/76949410/21656 Group Owner: devel+ow...@edk2.groups.io Unsubscribe: https://edk2.groups.io/g/devel/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
Re: [edk2-devel] 回复: [edk2-devel] [PATCH v10 0/5] Use RngLib instead of TimerLib for OpensslLib
Hey Liming! I added the two reviewed by and pushed to the PR here: https://github.com/tianocore/edk2/pull/933 Feel free to use that PR to merge in -- - Matthew Carlson -=-=-=-=-=-=-=-=-=-=-=- Groups.io Links: You receive all messages sent to this group. View/Reply Online (#65342): https://edk2.groups.io/g/devel/message/65342 Mute This Topic: https://groups.io/mt/76879301/21656 Group Owner: devel+ow...@edk2.groups.io Unsubscribe: https://edk2.groups.io/g/devel/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
Re: [edk2-devel] [PATCH v10 0/5] Use RngLib instead of TimerLib for OpensslLib
Just pinging this thread to see what needs to get done next. Thank you Liming for the reviewed by on the MdeModulePkg changes. -- - Matthew Carlson -=-=-=-=-=-=-=-=-=-=-=- Groups.io Links: You receive all messages sent to this group. View/Reply Online (#65285): https://edk2.groups.io/g/devel/message/65285 Mute This Topic: https://groups.io/mt/76563982/21656 Group Owner: devel+ow...@edk2.groups.io Unsubscribe: https://edk2.groups.io/g/devel/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
Re: [edk2-devel] [PATCH v5 0/5] Use RngLib instead of TimerLib for OpensslLib
Will do. -- - Matthew Carlson -=-=-=-=-=-=-=-=-=-=-=- Groups.io Links: You receive all messages sent to this group. View/Reply Online (#64123): https://edk2.groups.io/g/devel/message/64123 Mute This Topic: https://groups.io/mt/76140302/21656 Group Owner: devel+ow...@edk2.groups.io Unsubscribe: https://edk2.groups.io/g/devel/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
Re: [edk2-devel] [PATCH v5 4/5] ArmVirtPkg: Add RngLib based on TimerLib for CryptoPkg
You are correct, I'll fix it in v6 -- - Matthew Carlson -=-=-=-=-=-=-=-=-=-=-=- Groups.io Links: You receive all messages sent to this group. View/Reply Online (#64122): https://edk2.groups.io/g/devel/message/64122 Mute This Topic: https://groups.io/mt/76140306/21656 Group Owner: devel+ow...@edk2.groups.io Unsubscribe: https://edk2.groups.io/g/devel/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
Re: [edk2-devel] [PATCH v5 3/5] OvmfPkg: Add RngLib based on TimerLib for Crypto
I'll fix this for v6. Good spot. -- - Matthew Carlson -=-=-=-=-=-=-=-=-=-=-=- Groups.io Links: You receive all messages sent to this group. View/Reply Online (#64121): https://edk2.groups.io/g/devel/message/64121 Mute This Topic: https://groups.io/mt/76140305/21656 Group Owner: devel+ow...@edk2.groups.io Unsubscribe: https://edk2.groups.io/g/devel/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
Re: [edk2-devel] [PATCH v4 3/5] OvmfPkg: Add RngLib based on TimerLib for Crypto
Thank you for the helpful comments Lazlo! I sent out an updated series (v5) that fixes the things you mentioned. I added more description, so hopefully that helps. Sorry I didn't notice BhyvePkg, I thought you couldn't have packages under other packages, so I didn't think to check for other DSC's. It should be fixed up like the other Ovmf DSC's. I've been following your excellent guide for sending mailing list patches (Lazlo's Guide). ( https://github.com/tianocore/tianocore.github.io/wiki/Laszlo's-unkempt-git-guide-for-edk2-contributors-and-maintainers ) Is there a better way to get all the CC's from all the patches other than just copy and pasting them all? Perhaps the GetMaintainers.py where you specify multiple commits? Specifying a range didn't produce the desired behavior. -- - Matthew Carlson -=-=-=-=-=-=-=-=-=-=-=- Groups.io Links: You receive all messages sent to this group. View/Reply Online (#64041): https://edk2.groups.io/g/devel/message/64041 Mute This Topic: https://groups.io/mt/76119014/21656 Group Owner: devel+ow...@edk2.groups.io Unsubscribe: https://edk2.groups.io/g/devel/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
Re: [edk2-devel] [PATCH v4 0/5] Use RngLib instead of TimerLib for OpensslLib
Thanks Guomin! I sent out a version 5 of this series. -- - Matthew Carlson -=-=-=-=-=-=-=-=-=-=-=- Groups.io Links: You receive all messages sent to this group. View/Reply Online (#64040): https://edk2.groups.io/g/devel/message/64040 Mute This Topic: https://groups.io/mt/76119011/21656 Group Owner: devel+ow...@edk2.groups.io Unsubscribe: https://edk2.groups.io/g/devel/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
Re: [edk2-devel] [Patch v2 0/2] Use RngLib instead of TimerLib for OpensslLib
I included a null lib and I was including the wrong version of Rng anyway. I think that would make sense to split them into two different DSC files. Typo fixed, good spot! -- - Matthew Carlson -=-=-=-=-=-=-=-=-=-=-=- Groups.io Links: You receive all messages sent to this group. View/Reply Online (#63601): https://edk2.groups.io/g/devel/message/63601 Mute This Topic: https://groups.io/mt/75890825/21656 Group Owner: devel+ow...@edk2.groups.io Unsubscribe: https://edk2.groups.io/g/devel/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
Re: [edk2-devel] [Patch v2 0/2] Use RngLib instead of TimerLib for OpensslLib
Hey Mike! Thanks for the comments. I've addressed the feedback below. 1. All good comments. I think I've addressed them all. New patch series should be coming soon. 2. This is a problem for BaseCryptLibOnProtocol. We can't use a null lib for RngLib since it explodes when it tries to seed OpenSSL with entropy. I think for now, let's stick with established dependencies and figure this out at a later date. I added an RngLib that uses the DXE RNG Protocol. 3. I don't see a typo? Calls RandomNumber64 to fill a buffer of arbitrary size with random bytes. @param [in] LengthSize of the buffer, in bytes, to fill with. @param [out] RandBufferPointer to the buffer to store the random result. @retval TrueRandom bytes generation succeeded. @retval False Failed to request random bytes. -- - Matthew Carlson -=-=-=-=-=-=-=-=-=-=-=- Groups.io Links: You receive all messages sent to this group. View/Reply Online (#63591): https://edk2.groups.io/g/devel/message/63591 Mute This Topic: https://groups.io/mt/75890825/21656 Group Owner: devel+ow...@edk2.groups.io Unsubscribe: https://edk2.groups.io/g/devel/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
Re: [edk2-devel] [PATCH v1 1/3] UnitTestFrameworkPkg : BaseTimerLib: Adds a host-based timer Lib
I sent out a new patch series. Not sure, it might be waiting in the -- - Matthew Carlson -=-=-=-=-=-=-=-=-=-=-=- Groups.io Links: You receive all messages sent to this group. View/Reply Online (#62316): https://edk2.groups.io/g/devel/message/62316 Mute This Topic: https://groups.io/mt/75379678/21656 Group Owner: devel+ow...@edk2.groups.io Unsubscribe: https://edk2.groups.io/g/devel/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
Re: [edk2-devel] [PATCH v1 1/3] UnitTestFrameworkPkg : BaseTimerLib: Adds a host-based timer Lib
Great point. I'll fix this. -- - Matthew Carlson -=-=-=-=-=-=-=-=-=-=-=- Groups.io Links: You receive all messages sent to this group. View/Reply Online (#62257): https://edk2.groups.io/g/devel/message/62257 Mute This Topic: https://groups.io/mt/75379678/21656 Group Owner: devel+ow...@edk2.groups.io Unsubscribe: https://edk2.groups.io/g/devel/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
Re: [edk2-devel] Creating a edk2-pytool-base repo under tianocore
Thanks Mike! -- - Matthew Carlson -=-=-=-=-=-=-=-=-=-=-=- Groups.io Links: You receive all messages sent to this group. View/Reply Online (#61756): https://edk2.groups.io/g/devel/message/61756 Mute This Topic: https://groups.io/mt/75109251/21656 Group Owner: devel+ow...@edk2.groups.io Unsubscribe: https://edk2.groups.io/g/devel/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
[edk2-devel] Creating a edk2-pytool-base repo under tianocore
Based on this RFC: https://edk2.groups.io/g/rfc/topic/rfc_basetools_source_python/74009714?p=,,,20,0,0,0::recentpostdate%2Fsticky,,,20,2,0,74009714 We'd like to request a repository be made under tianocore with similar permissions to edk2-pytool-library or edk2-pytool-extensions We'd prefer it to be called edk2-pytool-base or edk2-basetools This will help us start the next stage. -- - Matthew Carlson -=-=-=-=-=-=-=-=-=-=-=- Groups.io Links: You receive all messages sent to this group. View/Reply Online (#61731): https://edk2.groups.io/g/devel/message/61731 Mute This Topic: https://groups.io/mt/75109251/21656 Group Owner: devel+ow...@edk2.groups.io Unsubscribe: https://edk2.groups.io/g/devel/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
Re: [edk2-devel] FW: Discussion: Basetools a separate repo
I think you've got it. The version of basetools will be carried via a pip-requirements file. Where would "pip install -r pip-requirements.txt" *fetch* the required basetools version from? I believe that it has a cache internal to pip. But if you want to fetch a basetools that hasn't been fetched before, yes, it would require a network download. Alternatively, if you're without internet access, you can check it out locally and use the pip install -e. Can you please explain the effects of the "pip install -e in more detail? So I'm a little murky on how it works but I believe that it creates symlinks or some other mechanism to repoint the global python module (global meaning your pip install or virtualenv). This is something that works automatically. The setup.py in the root of the repo takes care of this since it's used also to package the pip module. You can do pip install -e., and it just works (tm). Any basetools commit you check out (once it is in it's own repo, going back a given basetools commit two years ago is unfortunately not as easy) will work with pip install -e with not setup or configuration on your part as a developer. I use this whenever I work on the pytools and it works quite well. It will also show up in the BuildToolsReport and pip freeze as being locally installed and it will tell you the git commit it is currently at. I agree- I think we're trending towards very very frequent releases for basetools to provide high granularity. But that is pending community feedback and the decision of the tools maintainer. It is certainly trivial to automate the release pipeline in such a way that a release with every commit that passes CI is easy and mostly automatic for the maintainers. Thank you for the excellent questions! -- - Matthew Carlson -=-=-=-=-=-=-=-=-=-=-=- Groups.io Links: You receive all messages sent to this group. View/Reply Online (#57874): https://edk2.groups.io/g/devel/message/57874 Mute This Topic: https://groups.io/mt/73069134/21656 Group Owner: devel+ow...@edk2.groups.io Unsubscribe: https://edk2.groups.io/g/devel/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
Re: [edk2-devel] FW: Discussion: Basetools a separate repo
Hey Lazlo,
This is a great point of discussion. Just to make sure we're on the same page,
let me paraphrase your scenario. Two features at the same time or a improperly
tested new feature were committed in BaseTools and a new release was spun out.
EDK2 was updated to have it's pip-requirements.txt to point to the new version
committed. In an ideal world, CI should catch the build break for MdeModulePkg
but let's say it didn't catch it. Your scenario is, how do you go back to the
previous version of basetools that EDK2 used or perhaps go the the version of
BaseTools that last worked.
There are several great ways to do this. The release cadence of a pip based
BaseTools hasn't really been discussed but one option was every feature in
BaseTools would be a new release (this would be easy to do via pipelines). The
exact versioning would need to be discussed but there would likely be some
notion of breaking changes with major and minor version. This means that you
would be able to easily pip install the basetools 1:1 with the feature commit
history. In the scenario you mentioned, it would be easy to roll back to C0,
just "pip install -r the previous requirements file". If a virtual environment
is setup, this pip install would be entirely localized to the workspace the
user is operating in. The git show command you mentioned would show the
pip-requirements file. We would likely tag the pip_requirements file to a
specific release of BaseTools and there might be another file like
pip_suggestions that would allow for a looser version such as 0.3.x were any
minor version in 0.3 is fine. That said, determining what exact basetools was
used in a given build would likely be best accomplished through build
artifacts. We have a BuildToolsReport generator that specifically does this (
https://github.com/tianocore/edk2/tree/master/BaseTools/Plugin/BuildToolsReport
) and it would generate a report that would detail the exact pip version used
in that build. Without build artifacts, it grows a little bit more fuzzy, but
you can have reasonable confidence since it would need to at least conform to
pip suggestions and likely was on pip_requirements. To summarize, moving back
in time is simple and even trivial with minimal delay to a developer (pip
installs are usually sub-second procedures). There might even be a part of
edksetup that might do the pip install for you if that's what the community
wants.
Going to the next step, if you wanted to debug the BaseTools at C1, you might
want to generate a diff of the basetools to try and see where the error and
compare the diff. This I think better addresses your question of bisectability
(not sure if that's a word). You could clone the basetools repo and do a pip
install -e {path to basetools}. You would checkout the version in C1
pip_requirements and generate the diff between it and C0. Then you could apply
changes or revert changes and test them locally as the basetools used in your
edk2 repo will be the local repo you cloned. Once you're done, simply do a pip
install of your pip_requirements file and the symlinks will be overwritten and
things will go back to the way they were before.
Hopefully that addresses your concerns.
Thanks,
- Matthew Carlson
-=-=-=-=-=-=-=-=-=-=-=-
Groups.io Links: You receive all messages sent to this group.
View/Reply Online (#57770): https://edk2.groups.io/g/devel/message/57770
Mute This Topic: https://groups.io/mt/73069134/21656
Group Owner: devel+ow...@edk2.groups.io
Unsubscribe: https://edk2.groups.io/g/devel/unsub [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-
Re: [edk2-devel] [PATCH] CryptoPkg/FltUsedLib: Add FltUsedLib for float.
So it's not required by OpenSSL, it's required by the compiler whenever floating point is used, which can be in multiple places. For example, this is used in mu_plus (the Microsoft UEFI value add to EDK2) by the OnScreenKeyboard driver as well as the UiToolKit driver. If you happen to use BaseCryptLib or the intrinsic in a driver that happens to need crypto as well, it can break due to multiple. I do agree, this only applies to MSVC and requiring every platform to add a line in their DSC would be a situation I would prefer to avoid if possible. Is there a way to specify a library dependency only if a toolchain is being used? -- - Matthew Carlson -=-=-=-=-=-=-=-=-=-=-=- Groups.io Links: You receive all messages sent to this group. View/Reply Online (#56689): https://edk2.groups.io/g/devel/message/56689 Mute This Topic: https://groups.io/mt/72648022/21656 Group Owner: devel+ow...@edk2.groups.io Unsubscribe: https://edk2.groups.io/g/devel/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
