On Aug 26, 2014, at 7:29 AM, Gao, Liming <liming....@intel.com> wrote:
> Andrew: > If the package has been pass UPT clean up, their contents are same to UPT > output. You can drop these code base by UPT tool, and create the same layout > to your code base. Then, you can compare them and get the real difference. > Liming, this 2017+ world you talk about is interesting, but it is not the real world. UDK2014 is a zip file. The 300 INF files from the Core/MdePkg folks come in zip files. So this is the real world. 1) Core/MdePkg folks run UPT on the code base to “clean it up”. Then they make a zip file of this tree and send it to customers. 2) We have a tree, maybe we run UPT on it to clean it up. 3) Our tree is not a branch for every vendor drop. So we have N versions of vender code in our tree at any one time. So we can have up to N different trees that have been run through UPT. For each N some may have run UPT, some may have not. Not every one agrees on the same tree structure. Maybe one team is Core/MdePkg, and another is UDK2014/MdePkg, and yet another is edk2/MdePkg. Depending of the phase of the project we can get updates from the vendors weekly. At a weekly rate the updates are probably minor, but not always. If running UPT causes massive diff changes. You quickly end up in a world where you get a 100% diff hit on every vendor/IHV/3rd party software INF in your tree. Every time you do a git/svn merge with the vendor code all the INF files are out of sync. This makes syncing 3rd party code very painful. That is why customizing the .dec file path in the .inf is such a bad idea. If the tree mapping was done in the .dsc file things would be much simpler. In the Core/MdePkg example we would only have to change a couple of lines in the .dsc file to map the vendors tree structure to ours. In the current scheme we get changes to hundreds of files. At this point I think we are going to require our vendors to not run UPT, or if they do run UPT they must have the tree structure we want, not the tree structure the vendor wants. Thanks, Andrew Fish > Seemly, you request is to reorganize Package layout without UPT tool. I > don't think it is conflict with UPT clean up. > > Your concern may be the impact on the daily development work. Now, there is > no requirement to do daily UPT clean up. So, there is no impact. > > Thanks > Liming > -----Original Message----- > From: Andrew Fish [mailto:af...@apple.com] > Sent: Tuesday, August 26, 2014 1:31 PM > To: edk2-devel@lists.sourceforge.net > Subject: Re: [edk2] [Patch 2/2] [MdePkg] INF/DEC file updates to EDK II > packages > > > On Aug 25, 2014, at 10:05 PM, Gao, Liming <liming....@intel.com> wrote: > >> Andrew: >> Yes. 100% change in INF file should be the path difference if they are from >> the same package.dist file. But, if original one is not UPT clean, the >> difference will be hard to be seen. >> > > So when the Core/MdePkg vendor drops me a code base with 300+ INF files that > have the edk2 location shifted if I shift to my code base location I get 300 > files that show up in a git diff + what ever changed between the versions. > This is a lot of overhead to find the changes. If I get a lot of incremental > drops I get bogged down by all these changes. > >> The main UPT change is [Section] order and some alignment. Why do you think >> it will bring hard to the real work? >> > > Adding new content that remains consistent between code bases is not an > issue. > > Thanks, > > Andrew Fish > >> Thanks >> Liming >> -----Original Message----- >> From: Andrew Fish [mailto:af...@apple.com] >> Sent: Tuesday, August 26, 2014 11:38 AM >> To: edk2-devel@lists.sourceforge.net >> Subject: Re: [edk2] [Patch 2/2] [MdePkg] INF/DEC file updates to EDK >> II packages >> >> >> On Aug 25, 2014, at 7:01 PM, Gao, Liming <liming....@intel.com> wrote: >> >>> Jordan: >>> The real requirement is that some users use UPT to install core package >>> into the different directories, such as Core\MdePkg. After the >>> installation, they want to easily compare the original package and >>> installed package. >>> >> >> How is this compare done? How is it going to be easy if I get the weakly >> development version from some one who has a different location for the >> MdePkg than in my tree? So I will get a diff hit on 100% of the INF files, >> when I really just looking for the incremental changes. Not the relative >> changes caused by the path differences. >> >> How does the build deal with MdePkg and MdeModulePkg containing >> MdePkg/MdePkg.dec and the "Other code" containing Core/MdePkg/MdePkg.dec. >> How do I combine edk2 + vendor a + vendor b code together in a working tree >> and still grep against their original drops? As the edk2, vendor a, and >> vendor b should all not get the final say in the tree structure we end up >> with. Do I need to put all the edk2 packages in / (edk2) Core/ (vendor a) >> and edk2/ (vendor b) so each version matches? >> >> Currently we work around this by adding search paths to $WORKSPACE as a >> local hack to the BaseTools. >> >> Also when is this stuff going to be real? It looks like UDK 2014 is svn tags >> and a Zip file of code. Does that mean the UDK 2017 (2014 + (2014 - 2011)) >> we will use UPT to install the packages as the only option? The folks doing >> Core\MdePkg also like to release zip files with code. So all this UPT >> cleanup is making getting real work done harder... >> >> Seems like you guys should add some features to the build system so that >> daily development is not made worse by these UPT changes. >> 1) Allow search paths in the $WORKSPACE. So if some one moves the edk2 >> packages to Core/ you can add $(WORKSPACE)/Core as a search path and not >> need to change every INF in the system. >> 2) Given the current UPT cleanup we should add $WORKSPACE aliases so you >> don't need to have MdePkg in /, Core/, and edk2/. >> >> Thanks, >> >> Andrew Fish >> >> >>> This change is required by EDKII project major release, but not required >>> for daily development. >>> >>> The section order will not be changed unless new section are introduced in >>> INF/DEC. >>> >>> Yes. Those changes are directly output from UPT tool. And, we have test to >>> cover this tool. So, I have confidence. >>> >>> So far, I have no branch for those change. If you request, I could send zip >>> the source INF/DEC (before UPT and after UPT) to you. >>> >>> Thanks >>> Liming >>> -----Original Message----- >>> From: Jordan Justen [mailto:jljus...@gmail.com] >>> Sent: Tuesday, August 26, 2014 5:03 AM >>> To: edk2-devel@lists.sourceforge.net >>> Subject: Re: [edk2] [Patch 2/2] [MdePkg] INF/DEC file updates to EDK >>> II packages >>> >>> On Mon, Aug 25, 2014 at 3:17 AM, Gao, Liming <liming....@intel.com> wrote: >>>> Hi, all >>>> >>>> This patch is for below ITEM 7. INF/DEC are generated from UPT tool. >>>> DEC_SPECIFICATION and INF_VERSION will be updated to 0x00010017. >>>> Please help review them. >>>> >>>> MdePkg: Make sure order of DEC/INF content matches the order that >>>> UPT generates in the XML -> INF conversion >>> >>> This patch subject line is much longer than the recommended 70 character >>> limit: >>> https://github.com/tianocore/tianocore.github.io/wiki/Commit-Message- >>> F >>> ormat >>> >>> How about: >>> MdePkg DEC/INF: Match format required by UPT >>> >>> These changes seem to arbitrarily match the 'order of some tool', but why >>> is that required? >>> >>> What happens when someone edits these files, and doesn't get the order >>> 'just right'? >>> >>> Is the order that UPT uses strict, or will it potentially change in the >>> future, if for example, the version of some library being used by the tool >>> decides to change the way it orders things. >>> >>> There seems to be other formatting happening, such as spaces removed from >>> the copyright notice. >>> >>> All this rolled up into a single change seems to have produced something >>> that is not reviewable in a reasonable amount of time. >>> >>> Another question I have is, were these changes the result of the output of >>> the tool? I guess it would be easier to have some confidence in the output >>> of a tool rather than all of these changes having been manually applied. >>> >>> Do you have the changes available in a branch to make it easy to test? >>> (No, I would not want an svn branch to be added for this.) >>> >>> -Jordan >>> >>>> 1) Section Order in INF/DEC to match the ones generated from UPT >>>> >>>> 2) Guid value in section will be align. >>>> >>>> 3) Usage comments in section will be align. >>>> >>>> 4) One PCD section includes one PCD type. If one PCD supports more PCD >>>> types, it will be listed in each PCD type section in DEC file. >>>> >>>> >>>> >>>> Contributed-under: TianoCore Contribution Agreement 1.0 >>>> >>>> Signed-off-by: Gao, Liming <liming....@intel.com> >>>> >>>> >>>> >>>> Thanks >>>> >>>> Liming >>>> >>>> From: Gao, Liming [mailto:liming....@intel.com] >>>> Sent: Thursday, August 14, 2014 10:13 AM >>>> To: edk2-devel@lists.sourceforge.net >>>> Subject: [edk2] [Patch 1/2] [MdePkg] INF/DEC file updates to EDK II >>>> packages >>>> >>>> >>>> >>>> Hi, all >>>> >>>> Could you help review this patch? It includes the following changes >>>> 1-6 for MdePkg. The patch is a little big. For new added UNI file, I >>>> zip them together. >>>> >>>> >>>> >>>> The second patch for below item 7 will be sent later >>>> >>>> >>>> >>>> Thanks >>>> >>>> Liming >>>> >>>> From: Kinney, Michael D [mailto:michael.d.kin...@intel.com] >>>> Sent: Thursday, August 07, 2014 5:32 AM >>>> To: edk2-devel@lists.sourceforge.net >>>> Subject: [edk2] INF/DEC file updates to EDK II packages >>>> >>>> >>>> >>>> Hello, >>>> >>>> >>>> >>>> I wanted to let everyone know about a number of patch reviews for >>>> EDK II packages that will be sent out over the next couple of weeks. >>>> These patches impact the order of content in INF/DEC files and >>>> comment blocks in INF/DEC files, and should not have any build or >>>> functionality impacts. These patches will address the following issues: >>>> >>>> >>>> >>>> 1) Usage information in INF file comment blocks are either incomplete >>>> or incorrect. This includes usage information for >>>> Protocols/PPIs/GUIDs/PCDs/HOBs/Events/BootModes. The syntax for >>>> usage information in comment blocks is defined in the EDK II Module >>>> Information >>>> (INF) Specification >>>> >>>> 2) Add MODULE_UNI_FILE to INF [Defines] section along with UNI file >>>> that contains the localized Abstract and Description of a module. >>>> >>>> a. Addresses an information gap between INF files and the UEFI >>>> Distribution Packaging Specification XML schema >>>> >>>> b. There will be an associated update to UPT in BaseTools to consume >>>> MODULE_UNI_FILE and associated UNI file during UDP creation that >>>> performs the INF -> XML conversion. >>>> >>>> c. There will be an associated update to UPT in BaseTools to produce >>>> MODULE_UNI_FILE and associated UNI file during UDP installation that >>>> performs the XML -> INF conversion. >>>> >>>> 3) Add [UserExtensions.TianoCore."ExtraFiles"] section to INF files >>>> along with associated UNI file that provides the localized Name of a >>>> module. >>>> >>>> a. [UserExtensions.TianoCore."ExtraFiles"] provides an easy method >>>> for >>>> a module to specify extra files not listed in [Sources] or >>>> [Binaries] sections to be added to a UDP without having to list the >>>> files in the UPT package information data file. >>>> >>>> b. There will be an associated update to UPT in BaseTools to package >>>> up >>>> files listed in [UserExtensions.TianoCore."ExtraFiles"] during UDP >>>> creation. >>>> >>>> c. UNI file contains localized name of a module to go along with the >>>> localized Abstract and Description from the MODULE_UNI_FILE. >>>> >>>> 4) PCD information in DEC file comment blocks are either incomplete or >>>> incorrect. This includes detailed description, @Prompt, >>>> @ValidRange, @ValidList, @Expression, and [Error.<TokenSpaceGuid>] >>>> validation error messages >>>> >>>> 5) Add PACKAGE_UNI_FILE to DEC [Defines] section along with UNI file >>>> that contains the localized Abstract and Description of a package >>>> and localized strings associated with PCDs. >>>> >>>> a. Addresses an information gap between DEC files and the UEFI >>>> Distribution Packaging Specification XML schema >>>> >>>> b. There will be an associated update to UPT in BaseTools to consume >>>> PACKAGE_UNI_FILE and associated UNI file during UDP creation that >>>> performs the DEC -> XML conversion. >>>> >>>> c. There will be an associated update to UPT in BaseTools to produce >>>> PACKAGE_UNI_FILE and associated UNI file during UDP installation >>>> that performs the XML -> DEC conversion. >>>> >>>> 6) Add [UserExtensions.TianoCore."ExtraFiles"] section to DEC files >>>> along with associated UNI file that provides the localized Name of a >>>> package. >>>> >>>> a. [UserExtensions.TianoCore."ExtraFiles"] provides an easy method >>>> for >>>> a package to specify extra files to be added to a UDP without having >>>> to list the files in the UPT package information data file. >>>> >>>> b. There will be an associated update to UPT in BaseTools to package >>>> up >>>> files listed in [UserExtensions.TianoCore."ExtraFiles"] during UDP >>>> creation. >>>> >>>> c. UNI file contains localized name of a package to go along with the >>>> localized Abstract and Description from the PACKAGE_UNI_FILE. >>>> >>>> 7) Make sure order of DEC/INF content matches the order that UPT >>>> generates in the XML -> INF conversion >>>> >>>> a. This allows UDP packages installed by UPT to be compared against >>>> EDK II trunk/branches using standard diff utilities. >>>> >>>> >>>> >>>> Patches for the following EDK II packages are being prepared >>>> >>>> 1) MdePkg >>>> >>>> 2) MdeModulePkg >>>> >>>> 3) IntelFrameworkPkg >>>> >>>> 4) IntelFrameworkModulePkg >>>> >>>> 5) FatPkg >>>> >>>> 6) ShellPkg >>>> >>>> 7) PcAtChipsetPkg >>>> >>>> 8) UefiCpuPkg >>>> >>>> 9) SourceLevelDebugPkg >>>> >>>> 10) CryptoPkg >>>> >>>> 11) SecurityPkg >>>> >>>> 12) NetworkPkg >>>> >>>> >>>> >>>> Best regards, >>>> >>>> >>>> >>>> Mike >>>> >>>> >>>> >>>> >>>> -------------------------------------------------------------------- >>>> - >>>> - >>>> -------- >>>> Slashdot TV. >>>> Video for Nerds. Stuff that matters. >>>> http://tv.slashdot.org/ >>>> _______________________________________________ >>>> edk2-devel mailing list >>>> edk2-devel@lists.sourceforge.net >>>> https://lists.sourceforge.net/lists/listinfo/edk2-devel >>>> >>> >>> --------------------------------------------------------------------- >>> - >>> -------- >>> Slashdot TV. >>> Video for Nerds. Stuff that matters. >>> http://tv.slashdot.org/ >>> _______________________________________________ >>> edk2-devel mailing list >>> edk2-devel@lists.sourceforge.net >>> https://lists.sourceforge.net/lists/listinfo/edk2-devel >>> --------------------------------------------------------------------- >>> - >>> -------- >>> Slashdot TV. >>> Video for Nerds. Stuff that matters. >>> http://tv.slashdot.org/ >>> _______________________________________________ >>> edk2-devel mailing list >>> edk2-devel@lists.sourceforge.net >>> https://lists.sourceforge.net/lists/listinfo/edk2-devel >> >> >> ---------------------------------------------------------------------- >> -------- >> Slashdot TV. >> Video for Nerds. Stuff that matters. >> http://tv.slashdot.org/ >> _______________________________________________ >> edk2-devel mailing list >> edk2-devel@lists.sourceforge.net >> https://lists.sourceforge.net/lists/listinfo/edk2-devel >> >> ---------------------------------------------------------------------- >> -------- >> Slashdot TV. >> Video for Nerds. Stuff that matters. >> http://tv.slashdot.org/ >> _______________________________________________ >> edk2-devel mailing list >> edk2-devel@lists.sourceforge.net >> https://lists.sourceforge.net/lists/listinfo/edk2-devel > > > ------------------------------------------------------------------------------ > Slashdot TV. > Video for Nerds. Stuff that matters. > http://tv.slashdot.org/ > _______________________________________________ > edk2-devel mailing list > edk2-devel@lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/edk2-devel > > ------------------------------------------------------------------------------ > Slashdot TV. > Video for Nerds. Stuff that matters. > http://tv.slashdot.org/ > _______________________________________________ > edk2-devel mailing list > edk2-devel@lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/edk2-devel ------------------------------------------------------------------------------ Slashdot TV. Video for Nerds. Stuff that matters. http://tv.slashdot.org/ _______________________________________________ edk2-devel mailing list edk2-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/edk2-devel
