On Thu, Jan 25, 2018 at 01:37:39PM +0000, Zhu, Yonghong wrote: > Hi Gary, > > Thanks for your patches. I am still in evaluating these changes and do some > verification for it. > I still need some more days to give you comment. Thanks. I forgot to mention my branch in the cover letter.
https://github.com/lcp/edk2/tree/python3-futurize It would be easier to review/apply patches from a git branch. Thanks, Gary Lin > > Best Regards, > Zhu Yonghong > > > -----Original Message----- > From: Gary Lin [mailto:g...@suse.com] > Sent: Friday, January 19, 2018 12:43 PM > To: edk2-devel@lists.01.org > Cc: Zhu, Yonghong <yonghong....@intel.com>; Gao, Liming <liming....@intel.com> > Subject: [PATCH 00/15] BaseTools: One step toward python3 > > Since python2 will be EOF in 2020, we start to evaluate the impact of the > python2 removal. As expected, OMVF building failed the test. It's actually a > task noted in the wiki page: > > https://github.com/tianocore/tianocore.github.io/wiki/Tasks-BaseTools-Python3-Support > > Maybe it's time to convert the python scripts gradully. > > This patchset doesn't make the python scripts in BaseTools compatible with > python3 immediately. It aims to do the trivial and safe conversion and > replacement to make some statements compatible with both python2 and python3, > so we can deal with the difficult cases later. > > With the help of "futurize" from python-future, it's easier to refactor the > statements. This patchset is basically equivalent to "futurize -1" > plus "StringIO.StringIO => io.BytesIO" and minus "fix_absolute_import". > The reason to skip "fix_absolute_import" is that python2 failed to find some > modules after converting to absolute import, and it might take time to figure > out a proper fix. > > For the "io.BytesIO" change, it MIGHT introduce slow down to the build time > since io.BytesIO is slower than StringIO.StringIO in python2(*). > For a quick test, I built OVMF with the following command based on > 8ab0bd2397c9d3922e0c7dbb1aa6f7e08799079f: > > $ rm -rf Build && make -C BaseTools/ clean $ time ./OvmfPkg/build.sh -D > SECURE_BOOT_ENABLE \ > -D NETWORK_IP6_ENABLE \ > -D HTTP_BOOT_ENABLE \ > -D TLS_ENABLE > > Before io.BytesIO: > > Build total time: 00:03:56 > real 4m22.991s > user 3m55.874s > sys 0m27.250s > > After io.BytesIO: > > Build total time: 00:03:57 > real 4m23.953s > user 3m57.526s > sys 0m27.192s > > The difference is only 1 second, and I would say the impact is subtle. > > The next step will be fixing relative import and maybe applying more futurize > fixes. We won't get there soon but at least we are moving... > > (*) > https://stackoverflow.com/questions/37462075/confusing-about-stringio-cstringio-and-byteio > > Contributed-under: TianoCore Contribution Agreement 1.1 > Cc: Yonghong Zhu <yonghong....@intel.com> > Cc: Liming Gao <liming....@intel.com> > Signed-off-by: Gary Lin <g...@suse.com> > > Gary Lin (15): > BaseTools: Refactor python except statements > BaseTools: Refactor python print statements > BaseTools: Remove the old python "not-equal" > BaseTools: Use the python3-range functions > BaseTools: Remove tuple parameter in python scripts > BaseTools: Remove the deprecated hash_key() > BaseTools: Import reduce() from functools > BaseTools: Replace StandardError with Expression > BaseTools: Remove types.TypeType > BaseTools: Refactor python raise statement > BaseTools: Adjust the spaces around commas and colons > BaseTools: Migrate to the new octal literal > BaseTools: Unify long int and int in python scripts > BaseTools: Adjust old python2 idioms > BaseTools: Replace StringIO.StringIO with io.BytesIO > > BaseTools/Bin/CYGWIN_NT-5.1-i686/armcc_wrapper.py | 5 > +- > BaseTools/Scripts/BinToPcd.py | 46 > +++--- > BaseTools/Scripts/ConvertMasmToNasm.py | 1 > + > BaseTools/Scripts/ConvertUni.py | 5 > - > BaseTools/Scripts/MemoryProfileSymbolGen.py | 22 > +-- > BaseTools/Scripts/PatchCheck.py | 7 > +- > BaseTools/Scripts/RunMakefile.py | 2 > +- > BaseTools/Scripts/SmiHandlerProfileSymbolGen.py | 20 > +-- > BaseTools/Scripts/UpdateBuildVersions.py | 18 > +- > BaseTools/Source/Python/AutoGen/AutoGen.py | 91 > +++++----- > BaseTools/Source/Python/AutoGen/BuildEngine.py | 38 > +++-- > BaseTools/Source/Python/AutoGen/GenC.py | 5 > +- > BaseTools/Source/Python/AutoGen/GenDepex.py | 8 > +- > BaseTools/Source/Python/AutoGen/GenMake.py | 8 > +- > BaseTools/Source/Python/AutoGen/GenPcdDb.py | 142 > ++++++++-------- > BaseTools/Source/Python/AutoGen/GenVar.py | 165 > +++++++++---------- > BaseTools/Source/Python/AutoGen/IdfClassObject.py | 1 > - > BaseTools/Source/Python/AutoGen/InfSectionParser.py | 1 > + > BaseTools/Source/Python/AutoGen/StrGather.py | 5 > +- > BaseTools/Source/Python/AutoGen/UniClassObject.py | 18 > +- > BaseTools/Source/Python/AutoGen/ValidCheckingInfoObject.py | 10 > +- > BaseTools/Source/Python/BPDG/BPDG.py | 3 > +- > BaseTools/Source/Python/BPDG/GenVpd.py | 25 > +-- > BaseTools/Source/Python/Common/DataType.py | 4 > +- > BaseTools/Source/Python/Common/DecClassObject.py | 39 > ++--- > BaseTools/Source/Python/Common/Dictionary.py | 9 > +- > BaseTools/Source/Python/Common/DscClassObject.py | 70 > ++++---- > BaseTools/Source/Python/Common/EdkIIWorkspace.py | 25 > +-- > BaseTools/Source/Python/Common/EdkIIWorkspaceBuild.py | 133 > +++++++-------- > BaseTools/Source/Python/Common/Expression.py | 81 > ++++----- > BaseTools/Source/Python/Common/FdfClassObject.py | 1 > + > BaseTools/Source/Python/Common/FdfParserLite.py | 47 > +++--- > BaseTools/Source/Python/Common/InfClassObject.py | 113 > ++++++------- > BaseTools/Source/Python/Common/LongFilePathOs.py | 2 > +- > BaseTools/Source/Python/Common/MigrationUtilities.py | 1 > + > BaseTools/Source/Python/Common/Misc.py | 70 > ++++---- > BaseTools/Source/Python/Common/Parsing.py | 1 > + > BaseTools/Source/Python/Common/RangeExpression.py | 32 > ++-- > BaseTools/Source/Python/Common/String.py | 7 > +- > BaseTools/Source/Python/Common/TargetTxtClassObject.py | 15 > +- > BaseTools/Source/Python/Common/ToolDefClassObject.py | 3 > +- > BaseTools/Source/Python/Common/VpdInfoFile.py | 23 > +-- > BaseTools/Source/Python/Ecc/CParser.py | 173 > ++++++++++---------- > BaseTools/Source/Python/Ecc/Check.py | 1 > + > BaseTools/Source/Python/Ecc/CodeFragmentCollector.py | 69 > ++++---- > BaseTools/Source/Python/Ecc/Configuration.py | 5 > +- > BaseTools/Source/Python/Ecc/Exception.py | 3 > +- > BaseTools/Source/Python/Ecc/MetaDataParser.py | 3 > +- > BaseTools/Source/Python/Ecc/MetaFileWorkspace/MetaDataTable.py | 5 > +- > BaseTools/Source/Python/Ecc/MetaFileWorkspace/MetaFileParser.py | 41 > ++--- > BaseTools/Source/Python/Ecc/Xml/XmlRoutines.py | 9 > +- > BaseTools/Source/Python/Ecc/c.py | 15 > +- > BaseTools/Source/Python/Eot/CParser.py | 173 > ++++++++++---------- > BaseTools/Source/Python/Eot/CodeFragmentCollector.py | 61 > +++---- > BaseTools/Source/Python/Eot/FvImage.py | 17 > +- > BaseTools/Source/Python/Eot/InfParserLite.py | 8 > +- > BaseTools/Source/Python/Eot/Parser.py | 2 > +- > BaseTools/Source/Python/Eot/c.py | 23 > +-- > BaseTools/Source/Python/GenFds/AprioriSection.py | 7 > +- > BaseTools/Source/Python/GenFds/Capsule.py | 10 > +- > BaseTools/Source/Python/GenFds/CapsuleData.py | 6 > +- > BaseTools/Source/Python/GenFds/EfiSection.py | 6 > +- > BaseTools/Source/Python/GenFds/Fd.py | 12 > +- > BaseTools/Source/Python/GenFds/FdfParser.py | 43 > ++--- > BaseTools/Source/Python/GenFds/FfsFileStatement.py | 5 > +- > BaseTools/Source/Python/GenFds/FfsInfStatement.py | 16 > +- > BaseTools/Source/Python/GenFds/Fv.py | 13 > +- > BaseTools/Source/Python/GenFds/FvImageSection.py | 8 > +- > BaseTools/Source/Python/GenFds/GenFds.py | 20 > ++- > BaseTools/Source/Python/GenFds/GenFdsGlobalVariable.py | 10 > +- > BaseTools/Source/Python/GenFds/OptionRom.py | 3 > - > BaseTools/Source/Python/GenFds/Region.py | 14 > +- > BaseTools/Source/Python/GenPatchPcdTable/GenPatchPcdTable.py | 9 > +- > BaseTools/Source/Python/PatchPcdValue/PatchPcdValue.py | 1 > + > BaseTools/Source/Python/Pkcs7Sign/Pkcs7Sign.py | 32 > ++-- > BaseTools/Source/Python/Rsa2048Sha256Sign/Rsa2048Sha256GenerateKeys.py | 30 > ++-- > BaseTools/Source/Python/Rsa2048Sha256Sign/Rsa2048Sha256Sign.py | 36 > ++-- > BaseTools/Source/Python/TargetTool/TargetTool.py | 39 > ++--- > BaseTools/Source/Python/Trim/Trim.py | 25 > +-- > BaseTools/Source/Python/UPT/Core/DependencyRules.py | 12 > +- > BaseTools/Source/Python/UPT/Core/DistributionPackageClass.py | 4 > +- > BaseTools/Source/Python/UPT/Core/FileHook.py | 2 > +- > BaseTools/Source/Python/UPT/Core/IpiDb.py | 6 > +- > BaseTools/Source/Python/UPT/Core/PackageFile.py | 12 > +- > BaseTools/Source/Python/UPT/GenMetaFile/GenDecFile.py | 15 > +- > BaseTools/Source/Python/UPT/GenMetaFile/GenInfFile.py | 42 > ++--- > BaseTools/Source/Python/UPT/InstallPkg.py | 2 > +- > BaseTools/Source/Python/UPT/InventoryWs.py | 2 > +- > BaseTools/Source/Python/UPT/Library/CommentParsing.py | 5 > +- > BaseTools/Source/Python/UPT/Library/ExpressionValidate.py | 11 > +- > BaseTools/Source/Python/UPT/Library/Misc.py | 11 > +- > BaseTools/Source/Python/UPT/Library/ParserValidate.py | 2 > +- > BaseTools/Source/Python/UPT/Library/Parsing.py | 3 > +- > BaseTools/Source/Python/UPT/Library/String.py | 5 > +- > BaseTools/Source/Python/UPT/Library/UniClassObject.py | 20 > ++- > BaseTools/Source/Python/UPT/Library/Xml/XmlRoutines.py | 4 > +- > BaseTools/Source/Python/UPT/MkPkg.py | 2 > +- > BaseTools/Source/Python/UPT/Object/Parser/InfBinaryObject.py | 6 > +- > BaseTools/Source/Python/UPT/Object/Parser/InfDefineObject.py | 2 > +- > BaseTools/Source/Python/UPT/Object/Parser/InfGuidObject.py | 4 > +- > BaseTools/Source/Python/UPT/Object/Parser/InfLibraryClassesObject.py | 2 > +- > BaseTools/Source/Python/UPT/Object/Parser/InfMisc.py | 4 > +- > BaseTools/Source/Python/UPT/Object/Parser/InfPackagesObject.py | 4 > +- > BaseTools/Source/Python/UPT/Object/Parser/InfPcdObject.py | 4 > +- > BaseTools/Source/Python/UPT/Object/Parser/InfPpiObject.py | 4 > +- > BaseTools/Source/Python/UPT/Object/Parser/InfProtocolObject.py | 2 > +- > BaseTools/Source/Python/UPT/Object/Parser/InfSoucesObject.py | 3 > +- > BaseTools/Source/Python/UPT/Object/Parser/InfUserExtensionObject.py | 4 > +- > BaseTools/Source/Python/UPT/Parser/DecParserMisc.py | 1 > + > BaseTools/Source/Python/UPT/Parser/InfSectionParser.py | 3 > +- > BaseTools/Source/Python/UPT/PomAdapter/DecPomAlignment.py | 57 > +++---- > BaseTools/Source/Python/UPT/PomAdapter/InfPomAlignment.py | 3 > +- > BaseTools/Source/Python/UPT/PomAdapter/InfPomAlignmentMisc.py | 3 > +- > BaseTools/Source/Python/UPT/ReplacePkg.py | 2 > +- > BaseTools/Source/Python/UPT/RmPkg.py | 2 > +- > BaseTools/Source/Python/UPT/TestInstall.py | 4 > +- > BaseTools/Source/Python/UPT/UPT.py | 9 > +- > BaseTools/Source/Python/UPT/UnitTest/DecParserTest.py | 5 > +- > BaseTools/Source/Python/UPT/UnitTest/InfBinarySectionTest.py | 10 > +- > BaseTools/Source/Python/UPT/Xml/CommonXml.py | 2 > +- > BaseTools/Source/Python/UPT/Xml/IniToXml.py | 1 > + > BaseTools/Source/Python/UPT/Xml/XmlParser.py | 25 > +-- > BaseTools/Source/Python/UPT/Xml/XmlParserMisc.py | 3 > +- > BaseTools/Source/Python/Workspace/BuildClassObject.py | 2 > +- > BaseTools/Source/Python/Workspace/DecBuildData.py | 14 > +- > BaseTools/Source/Python/Workspace/DscBuildData.py | 142 > ++++++++-------- > BaseTools/Source/Python/Workspace/InfBuildData.py | 3 > +- > BaseTools/Source/Python/Workspace/MetaFileParser.py | 74 > +++++---- > BaseTools/Source/Python/Workspace/MetaFileTable.py | 10 > +- > BaseTools/Source/Python/Workspace/WorkspaceCommon.py | 2 > +- > BaseTools/Source/Python/build/BuildReport.py | 17 > +- > BaseTools/Source/Python/build/build.py | 35 > ++-- > BaseTools/Tests/CheckPythonSyntax.py | 2 > +- > BaseTools/Tests/TestTools.py | 13 > +- > BaseTools/Tests/TianoCompress.py | 6 > +- > BaseTools/gcc/mingw-gcc-build.py | 112 > ++++++------- > 136 files changed, 1559 insertions(+), 1477 deletions(-) > > -- > 2.15.1 > > _______________________________________________ > edk2-devel mailing list > edk2-devel@lists.01.org > https://lists.01.org/mailman/listinfo/edk2-devel > _______________________________________________ edk2-devel mailing list edk2-devel@lists.01.org https://lists.01.org/mailman/listinfo/edk2-devel
