Re: [edk2] [PATCH] BaseTools/EOT: Change to call a program instead of calling Python API.
Reviewed-by: Yonghong Zhu Best Regards, Zhu Yonghong -Original Message- From: Carsey, Jaben Sent: Tuesday, October 16, 2018 5:42 AM To: Chen, Hesheng ; Zhu, Yonghong ; edk2-devel@lists.01.org Subject: RE: [edk2] [PATCH] BaseTools/EOT: Change to call a program instead of calling Python API. Hess, Thanks for the clarification. Makes sense! I glanced at, but didn't read in detail the code... so. Acked-by: Jaben Carsey > -Original Message- > From: Chen, Hesheng > Sent: Monday, October 15, 2018 2:34 PM > To: Carsey, Jaben ; Zhu, Yonghong > ; edk2-devel@lists.01.org > Subject: RE: [edk2] [PATCH] BaseTools/EOT: Change to call a program > instead of calling Python API. > Importance: High > > Hello Jaben, > The API is provided by C code and we want Python tool to use it. The > tool used to call Python API from DLL files and now we need run Python > tools from source so we can't build a specific version of DLL binary > for it. The DLL file may be different for different version of Python. > So now we just call the C program directly for the API. > > Best Regards, > Chen, Hess > Intel China Software Center > Tel: +86-21-6116-6740 > Email: hesheng.c...@intel.com > > -Original Message- > From: Carsey, Jaben > Sent: Tuesday, October 16, 2018 1:45 AM > To: Zhu, Yonghong ; edk2-devel@lists.01.org > Cc: Chen, Hesheng > Subject: RE: [edk2] [PATCH] BaseTools/EOT: Change to call a program > instead of calling Python API. > > > > -Original Message- > > From: edk2-devel [mailto:edk2-devel-boun...@lists.01.org] On Behalf > > Of Yonghong Zhu > > Sent: Monday, October 15, 2018 3:24 AM > > To: edk2-devel@lists.01.org > > Cc: Chen, Hesheng > > Subject: [edk2] [PATCH] BaseTools/EOT: Change to call a program > > instead of calling Python API. > > > > From: hchen30 > > > > Update the EOT tool to call the program itself instead of calling > > the Python API when parsing FV images. > > Why do we prefer to launch the separate python program instead of > calling the APIs? > > > > > > Contributed-under: TianoCore Contribution Agreement 1.0 > > Signed-off-by: Hess Chen > > --- > > BaseTools/Source/Python/Eot/{Eot.py => EotMain.py} | 465 > > +++-- > > > > BaseTools/Source/Python/Eot/InfParserLite.py | 26 +- > > BaseTools/Source/Python/Eot/Parser.py | 28 +- > > BaseTools/Source/Python/Eot/Report.py | 6 +- > > BaseTools/Source/Python/build/BuildReport.py | 2 +- > > 5 files changed, 84 insertions(+), 443 deletions(-) rename > > BaseTools/Source/Python/Eot/{Eot.py => EotMain.py} (75%) > > > > diff --git a/BaseTools/Source/Python/Eot/Eot.py > > b/BaseTools/Source/Python/Eot/EotMain.py > > similarity index 75% > > rename from BaseTools/Source/Python/Eot/Eot.py > > rename to BaseTools/Source/Python/Eot/EotMain.py > > index ce83da1495..49a1494126 100644 > > --- a/BaseTools/Source/Python/Eot/Eot.py > > +++ b/BaseTools/Source/Python/Eot/EotMain.py > > @@ -17,18 +17,20 @@ > > from __future__ import absolute_import import > Common.LongFilePathOs > > as os, time, glob import Common.EdkLogger as EdkLogger -from . > > import EotGlobalData > > +import Eot.EotGlobalData as EotGlobalData > > from optparse import OptionParser > > from Common.StringUtils import NormPath from Common import > > BuildToolError from Common.Misc import > > GuidStructureStringToGuidString, sdict -from .InfParserLite import * > > -from . import c -from . import Database > > +from Eot.Parser import * > > +from Eot.InfParserLite import EdkInfParser from Common.StringUtils > > +import GetSplitValueList from Eot import c from Eot import Database > > from array import array > > -from .Report import Report > > +from Eot.Report import Report > > from Common.BuildVersion import gBUILD_VERSION -from .Parser import > > ConvertGuid > > +from Eot.Parser import ConvertGuid > > from Common.LongFilePathSupport import OpenLongFilePath as open > > import struct import uuid @@ -58,14 +60,14 @@ class Image(array): > > > > self._SubImages = sdict() # {offset: Image()} > > > > -array.__init__(self, 'B') > > +array.__init__(self) > > > > def __repr__(self): > > return self._ID_ > > > > def __len__(self): > > Len = array.__len__(self) > > -for Offset in self._SubImages: > > +for Offset in self._SubImages.keys(): > >
Re: [edk2] [PATCH] BaseTools/EOT: Change to call a program instead of calling Python API.
Hess, Thanks for the clarification. Makes sense! I glanced at, but didn't read in detail the code... so. Acked-by: Jaben Carsey > -Original Message- > From: Chen, Hesheng > Sent: Monday, October 15, 2018 2:34 PM > To: Carsey, Jaben ; Zhu, Yonghong > ; edk2-devel@lists.01.org > Subject: RE: [edk2] [PATCH] BaseTools/EOT: Change to call a program instead > of calling Python API. > Importance: High > > Hello Jaben, > The API is provided by C code and we want Python tool to use it. The tool > used to call Python API from DLL files and now we need run Python tools > from source so we can't build a specific version of DLL binary for it. The DLL > file may be different for different version of Python. So now we just call the > C program directly for the API. > > Best Regards, > Chen, Hess > Intel China Software Center > Tel: +86-21-6116-6740 > Email: hesheng.c...@intel.com > > -Original Message- > From: Carsey, Jaben > Sent: Tuesday, October 16, 2018 1:45 AM > To: Zhu, Yonghong ; edk2-devel@lists.01.org > Cc: Chen, Hesheng > Subject: RE: [edk2] [PATCH] BaseTools/EOT: Change to call a program instead > of calling Python API. > > > > -Original Message- > > From: edk2-devel [mailto:edk2-devel-boun...@lists.01.org] On Behalf Of > > Yonghong Zhu > > Sent: Monday, October 15, 2018 3:24 AM > > To: edk2-devel@lists.01.org > > Cc: Chen, Hesheng > > Subject: [edk2] [PATCH] BaseTools/EOT: Change to call a program > > instead of calling Python API. > > > > From: hchen30 > > > > Update the EOT tool to call the program itself instead of calling the > > Python API when parsing FV images. > > Why do we prefer to launch the separate python program instead of calling > the APIs? > > > > > > Contributed-under: TianoCore Contribution Agreement 1.0 > > Signed-off-by: Hess Chen > > --- > > BaseTools/Source/Python/Eot/{Eot.py => EotMain.py} | 465 > > +++-- > > > > BaseTools/Source/Python/Eot/InfParserLite.py | 26 +- > > BaseTools/Source/Python/Eot/Parser.py | 28 +- > > BaseTools/Source/Python/Eot/Report.py | 6 +- > > BaseTools/Source/Python/build/BuildReport.py | 2 +- > > 5 files changed, 84 insertions(+), 443 deletions(-) rename > > BaseTools/Source/Python/Eot/{Eot.py => EotMain.py} (75%) > > > > diff --git a/BaseTools/Source/Python/Eot/Eot.py > > b/BaseTools/Source/Python/Eot/EotMain.py > > similarity index 75% > > rename from BaseTools/Source/Python/Eot/Eot.py > > rename to BaseTools/Source/Python/Eot/EotMain.py > > index ce83da1495..49a1494126 100644 > > --- a/BaseTools/Source/Python/Eot/Eot.py > > +++ b/BaseTools/Source/Python/Eot/EotMain.py > > @@ -17,18 +17,20 @@ > > from __future__ import absolute_import import > Common.LongFilePathOs > > as os, time, glob import Common.EdkLogger as EdkLogger -from . import > > EotGlobalData > > +import Eot.EotGlobalData as EotGlobalData > > from optparse import OptionParser > > from Common.StringUtils import NormPath from Common import > > BuildToolError from Common.Misc import > > GuidStructureStringToGuidString, sdict -from .InfParserLite import * > > -from . import c -from . import Database > > +from Eot.Parser import * > > +from Eot.InfParserLite import EdkInfParser from Common.StringUtils > > +import GetSplitValueList from Eot import c from Eot import Database > > from array import array > > -from .Report import Report > > +from Eot.Report import Report > > from Common.BuildVersion import gBUILD_VERSION -from .Parser import > > ConvertGuid > > +from Eot.Parser import ConvertGuid > > from Common.LongFilePathSupport import OpenLongFilePath as open > > import struct import uuid @@ -58,14 +60,14 @@ class Image(array): > > > > self._SubImages = sdict() # {offset: Image()} > > > > -array.__init__(self, 'B') > > +array.__init__(self) > > > > def __repr__(self): > > return self._ID_ > > > > def __len__(self): > > Len = array.__len__(self) > > -for Offset in self._SubImages: > > +for Offset in self._SubImages.keys(): > > Len += len(self._SubImages[Offset]) > > return Len > > > > @@ -154,19 +156,11 @@ class CompressedImage(Image): > > > > def _GetSections(self): > > try: > > -from . import EfiCompressor > > -TmpData = EfiCompressor.FrameworkDecompress( &
Re: [edk2] [PATCH] BaseTools/EOT: Change to call a program instead of calling Python API.
Hello Jaben, The API is provided by C code and we want Python tool to use it. The tool used to call Python API from DLL files and now we need run Python tools from source so we can't build a specific version of DLL binary for it. The DLL file may be different for different version of Python. So now we just call the C program directly for the API. Best Regards, Chen, Hess Intel China Software Center Tel: +86-21-6116-6740 Email: hesheng.c...@intel.com -Original Message- From: Carsey, Jaben Sent: Tuesday, October 16, 2018 1:45 AM To: Zhu, Yonghong ; edk2-devel@lists.01.org Cc: Chen, Hesheng Subject: RE: [edk2] [PATCH] BaseTools/EOT: Change to call a program instead of calling Python API. > -Original Message- > From: edk2-devel [mailto:edk2-devel-boun...@lists.01.org] On Behalf Of > Yonghong Zhu > Sent: Monday, October 15, 2018 3:24 AM > To: edk2-devel@lists.01.org > Cc: Chen, Hesheng > Subject: [edk2] [PATCH] BaseTools/EOT: Change to call a program > instead of calling Python API. > > From: hchen30 > > Update the EOT tool to call the program itself instead of calling the > Python API when parsing FV images. Why do we prefer to launch the separate python program instead of calling the APIs? > > Contributed-under: TianoCore Contribution Agreement 1.0 > Signed-off-by: Hess Chen > --- > BaseTools/Source/Python/Eot/{Eot.py => EotMain.py} | 465 > +++-- > > BaseTools/Source/Python/Eot/InfParserLite.py | 26 +- > BaseTools/Source/Python/Eot/Parser.py | 28 +- > BaseTools/Source/Python/Eot/Report.py | 6 +- > BaseTools/Source/Python/build/BuildReport.py | 2 +- > 5 files changed, 84 insertions(+), 443 deletions(-) rename > BaseTools/Source/Python/Eot/{Eot.py => EotMain.py} (75%) > > diff --git a/BaseTools/Source/Python/Eot/Eot.py > b/BaseTools/Source/Python/Eot/EotMain.py > similarity index 75% > rename from BaseTools/Source/Python/Eot/Eot.py > rename to BaseTools/Source/Python/Eot/EotMain.py > index ce83da1495..49a1494126 100644 > --- a/BaseTools/Source/Python/Eot/Eot.py > +++ b/BaseTools/Source/Python/Eot/EotMain.py > @@ -17,18 +17,20 @@ > from __future__ import absolute_import import Common.LongFilePathOs > as os, time, glob import Common.EdkLogger as EdkLogger -from . import > EotGlobalData > +import Eot.EotGlobalData as EotGlobalData > from optparse import OptionParser > from Common.StringUtils import NormPath from Common import > BuildToolError from Common.Misc import > GuidStructureStringToGuidString, sdict -from .InfParserLite import * > -from . import c -from . import Database > +from Eot.Parser import * > +from Eot.InfParserLite import EdkInfParser from Common.StringUtils > +import GetSplitValueList from Eot import c from Eot import Database > from array import array > -from .Report import Report > +from Eot.Report import Report > from Common.BuildVersion import gBUILD_VERSION -from .Parser import > ConvertGuid > +from Eot.Parser import ConvertGuid > from Common.LongFilePathSupport import OpenLongFilePath as open > import struct import uuid @@ -58,14 +60,14 @@ class Image(array): > > self._SubImages = sdict() # {offset: Image()} > > -array.__init__(self, 'B') > +array.__init__(self) > > def __repr__(self): > return self._ID_ > > def __len__(self): > Len = array.__len__(self) > -for Offset in self._SubImages: > +for Offset in self._SubImages.keys(): > Len += len(self._SubImages[Offset]) > return Len > > @@ -154,19 +156,11 @@ class CompressedImage(Image): > > def _GetSections(self): > try: > -from . import EfiCompressor > -TmpData = EfiCompressor.FrameworkDecompress( > -self[self._HEADER_SIZE_:], > -len(self) - self._HEADER_SIZE_ > -) > +TmpData = DeCompress('Efi', self[self._HEADER_SIZE_:]) > DecData = array('B') > DecData.fromstring(TmpData) > except: > -from . import EfiCompressor > -TmpData = EfiCompressor.UefiDecompress( > -self[self._HEADER_SIZE_:], > -len(self) - self._HEADER_SIZE_ > -) > +TmpData = DeCompress('Framework', > + self[self._HEADER_SIZE_:]) > DecData = array('B') > DecData.fromstring(TmpData) > > @@ -297,7 +291,7 @@ class Depex(Image): > >
Re: [edk2] [PATCH] BaseTools/EOT: Change to call a program instead of calling Python API.
> -Original Message- > From: edk2-devel [mailto:edk2-devel-boun...@lists.01.org] On Behalf Of > Yonghong Zhu > Sent: Monday, October 15, 2018 3:24 AM > To: edk2-devel@lists.01.org > Cc: Chen, Hesheng > Subject: [edk2] [PATCH] BaseTools/EOT: Change to call a program instead of > calling Python API. > > From: hchen30 > > Update the EOT tool to call the program itself instead of calling the Python > API when parsing FV images. Why do we prefer to launch the separate python program instead of calling the APIs? > > Contributed-under: TianoCore Contribution Agreement 1.0 > Signed-off-by: Hess Chen > --- > BaseTools/Source/Python/Eot/{Eot.py => EotMain.py} | 465 +++-- > > BaseTools/Source/Python/Eot/InfParserLite.py | 26 +- > BaseTools/Source/Python/Eot/Parser.py | 28 +- > BaseTools/Source/Python/Eot/Report.py | 6 +- > BaseTools/Source/Python/build/BuildReport.py | 2 +- > 5 files changed, 84 insertions(+), 443 deletions(-) > rename BaseTools/Source/Python/Eot/{Eot.py => EotMain.py} (75%) > > diff --git a/BaseTools/Source/Python/Eot/Eot.py > b/BaseTools/Source/Python/Eot/EotMain.py > similarity index 75% > rename from BaseTools/Source/Python/Eot/Eot.py > rename to BaseTools/Source/Python/Eot/EotMain.py > index ce83da1495..49a1494126 100644 > --- a/BaseTools/Source/Python/Eot/Eot.py > +++ b/BaseTools/Source/Python/Eot/EotMain.py > @@ -17,18 +17,20 @@ > from __future__ import absolute_import > import Common.LongFilePathOs as os, time, glob > import Common.EdkLogger as EdkLogger > -from . import EotGlobalData > +import Eot.EotGlobalData as EotGlobalData > from optparse import OptionParser > from Common.StringUtils import NormPath > from Common import BuildToolError > from Common.Misc import GuidStructureStringToGuidString, sdict > -from .InfParserLite import * > -from . import c > -from . import Database > +from Eot.Parser import * > +from Eot.InfParserLite import EdkInfParser > +from Common.StringUtils import GetSplitValueList > +from Eot import c > +from Eot import Database > from array import array > -from .Report import Report > +from Eot.Report import Report > from Common.BuildVersion import gBUILD_VERSION > -from .Parser import ConvertGuid > +from Eot.Parser import ConvertGuid > from Common.LongFilePathSupport import OpenLongFilePath as open > import struct > import uuid > @@ -58,14 +60,14 @@ class Image(array): > > self._SubImages = sdict() # {offset: Image()} > > -array.__init__(self, 'B') > +array.__init__(self) > > def __repr__(self): > return self._ID_ > > def __len__(self): > Len = array.__len__(self) > -for Offset in self._SubImages: > +for Offset in self._SubImages.keys(): > Len += len(self._SubImages[Offset]) > return Len > > @@ -154,19 +156,11 @@ class CompressedImage(Image): > > def _GetSections(self): > try: > -from . import EfiCompressor > -TmpData = EfiCompressor.FrameworkDecompress( > -self[self._HEADER_SIZE_:], > -len(self) - self._HEADER_SIZE_ > -) > +TmpData = DeCompress('Efi', self[self._HEADER_SIZE_:]) > DecData = array('B') > DecData.fromstring(TmpData) > except: > -from . import EfiCompressor > -TmpData = EfiCompressor.UefiDecompress( > -self[self._HEADER_SIZE_:], > -len(self) - self._HEADER_SIZE_ > -) > +TmpData = DeCompress('Framework', self[self._HEADER_SIZE_:]) > DecData = array('B') > DecData.fromstring(TmpData) > > @@ -297,7 +291,7 @@ class Depex(Image): > > Expression = property(_GetExpression) > > -## FirmwareVolume() class > +# # FirmwareVolume() class > # > # A class for Firmware Volume > # > @@ -308,12 +302,12 @@ class FirmwareVolume(Image): > > _FfsGuid = "8C8CE578-8A3D-4F1C-9935-896185C32DD3" > > -_GUID_ = struct.Struct("16x 1I2H8B") > -_LENGTH_= struct.Struct("16x 16x 1Q") > -_SIG_ = struct.Struct("16x 16x 8x 1I") > -_ATTR_ = struct.Struct("16x 16x 8x 4x 1I") > -_HLEN_ = struct.Struct("16x 16x 8x 4x 4x 1H") > -_CHECKSUM_ = struct.Struct("16x 16x 8x 4x 4
[edk2] [PATCH] BaseTools/EOT: Change to call a program instead of calling Python API.
From: hchen30 Update the EOT tool to call the program itself instead of calling the Python API when parsing FV images. Contributed-under: TianoCore Contribution Agreement 1.0 Signed-off-by: Hess Chen --- BaseTools/Source/Python/Eot/{Eot.py => EotMain.py} | 465 +++-- BaseTools/Source/Python/Eot/InfParserLite.py | 26 +- BaseTools/Source/Python/Eot/Parser.py | 28 +- BaseTools/Source/Python/Eot/Report.py | 6 +- BaseTools/Source/Python/build/BuildReport.py | 2 +- 5 files changed, 84 insertions(+), 443 deletions(-) rename BaseTools/Source/Python/Eot/{Eot.py => EotMain.py} (75%) diff --git a/BaseTools/Source/Python/Eot/Eot.py b/BaseTools/Source/Python/Eot/EotMain.py similarity index 75% rename from BaseTools/Source/Python/Eot/Eot.py rename to BaseTools/Source/Python/Eot/EotMain.py index ce83da1495..49a1494126 100644 --- a/BaseTools/Source/Python/Eot/Eot.py +++ b/BaseTools/Source/Python/Eot/EotMain.py @@ -17,18 +17,20 @@ from __future__ import absolute_import import Common.LongFilePathOs as os, time, glob import Common.EdkLogger as EdkLogger -from . import EotGlobalData +import Eot.EotGlobalData as EotGlobalData from optparse import OptionParser from Common.StringUtils import NormPath from Common import BuildToolError from Common.Misc import GuidStructureStringToGuidString, sdict -from .InfParserLite import * -from . import c -from . import Database +from Eot.Parser import * +from Eot.InfParserLite import EdkInfParser +from Common.StringUtils import GetSplitValueList +from Eot import c +from Eot import Database from array import array -from .Report import Report +from Eot.Report import Report from Common.BuildVersion import gBUILD_VERSION -from .Parser import ConvertGuid +from Eot.Parser import ConvertGuid from Common.LongFilePathSupport import OpenLongFilePath as open import struct import uuid @@ -58,14 +60,14 @@ class Image(array): self._SubImages = sdict() # {offset: Image()} -array.__init__(self, 'B') +array.__init__(self) def __repr__(self): return self._ID_ def __len__(self): Len = array.__len__(self) -for Offset in self._SubImages: +for Offset in self._SubImages.keys(): Len += len(self._SubImages[Offset]) return Len @@ -154,19 +156,11 @@ class CompressedImage(Image): def _GetSections(self): try: -from . import EfiCompressor -TmpData = EfiCompressor.FrameworkDecompress( -self[self._HEADER_SIZE_:], -len(self) - self._HEADER_SIZE_ -) +TmpData = DeCompress('Efi', self[self._HEADER_SIZE_:]) DecData = array('B') DecData.fromstring(TmpData) except: -from . import EfiCompressor -TmpData = EfiCompressor.UefiDecompress( -self[self._HEADER_SIZE_:], -len(self) - self._HEADER_SIZE_ -) +TmpData = DeCompress('Framework', self[self._HEADER_SIZE_:]) DecData = array('B') DecData.fromstring(TmpData) @@ -297,7 +291,7 @@ class Depex(Image): Expression = property(_GetExpression) -## FirmwareVolume() class +# # FirmwareVolume() class # # A class for Firmware Volume # @@ -308,12 +302,12 @@ class FirmwareVolume(Image): _FfsGuid = "8C8CE578-8A3D-4F1C-9935-896185C32DD3" -_GUID_ = struct.Struct("16x 1I2H8B") -_LENGTH_= struct.Struct("16x 16x 1Q") -_SIG_ = struct.Struct("16x 16x 8x 1I") -_ATTR_ = struct.Struct("16x 16x 8x 4x 1I") -_HLEN_ = struct.Struct("16x 16x 8x 4x 4x 1H") -_CHECKSUM_ = struct.Struct("16x 16x 8x 4x 4x 2x 1H") +_GUID_ = struct.Struct("16x 1I2H8B") +_LENGTH_ = struct.Struct("16x 16x 1Q") +_SIG_ = struct.Struct("16x 16x 8x 1I") +_ATTR_ = struct.Struct("16x 16x 8x 4x 1I") +_HLEN_ = struct.Struct("16x 16x 8x 4x 4x 1H") +_CHECKSUM_ = struct.Struct("16x 16x 8x 4x 4x 2x 1H") def __init__(self, Name=''): Image.__init__(self) @@ -387,7 +381,7 @@ class FirmwareVolume(Image): DepexString = DepexList[0].strip() return (CouldBeLoaded, DepexString, FileDepex) -def Dispatch(self, Db = None): +def Dispatch(self, Db=None): if Db is None: return False self.UnDispatchedFfsDict = copy.copy(self.FfsDict) @@ -397,7 +391,7 @@ class FirmwareVolume(Image): FfsDxeCoreGuid = None FfsPeiPrioriGuid = None FfsDxePrioriGuid = None -for FfsID in self.UnDispatchedFfsDict: +for FfsID in self.UnDispatchedFfsDict.keys(): Ffs = self.UnDispatchedFfsDict[FfsID] if Ffs.Type == 0x03: FfsSecCoreGuid = FfsID @@ -439,6 +433,7 @@ class Fi