> On Jun 29, 2016, at 3:06 PM, Kinney, Michael D <michael.d.kin...@intel.com>
> wrote:
>
> Why can't we simply use the target.txt TOOL_CHAIN_CONF define
> in target.txt as it works today? You could split tools_def.txt
> into several versions, with the default one being the maintained
> one and developers can opt into community or deprecated ones.
>
> I do want to make sure we are focused on the right problem here
>
> 1) # lines in tools_def.txt
> 2) maintenance of tools_def.txt
> 3) build performance
Mike,
If parsing the file 47 times was enough to make a function stand out in the
Python performance log I would guess the parsing it 1 time is not that
efficient.
In general the built tools seems architected to "go slow", well to make it
easier to debug the build system and performance was not important.
1) I saw regex was getting called 1,000,000 times in a Python performance log.
I think most of it is scanning all the C code to build the dependencies for the
makefile
a) Likely make + compiler is better performance optimized for this, and is
continuously getting improved.
b) A build option to do this the normal way would be nice.
c) I'm guessing a lot of the early ... time in the build is this step.
2) Parallel builds are done from Python threads vs make -j
a) See #1
b) It would be nice to have a build phase that just generated the makefiles
including a top level makefile was constructed to enable parallel build. Then
the top level makefile could call the generated makefile. After that build can
be called to construct the FDs.
c) When we build DEBUG and RELEASE in parallel, or multiple platforms it
seems like the Python is fighting the make -j. If it was all make -j then make
could load balance.
3) The FV construction seems slow and it generates multiple text files per
section per file and there is lots of calling out between Python and C tools.
a) Maybe combining all the FD construction into a single C program would
allow all the data to be in memory vs. on disk.
4) It does not seem that the Python code has really been performance tuned. A
large file was getting parsed 47 time, regex getting called a 1,000,000 times
etc.
5) Most of the build targets use scripts.
a) A top level makefile, and moving some of the work to makefiles could help
with parallelism.
b) I've got this working on our proprietary tree.
1) Had to make Conf/ part of the Buid/ output. I made a small mod to the
tool to copy the Conf over if the alternate Conf location passed to build did
not exit.
2) I had to add .NOTPARALLEL: to some of the BaseTools GNUmakefiles since
they break for a parallel build.
3) Then you port the build.sh (or build.bat) to be a makefile.
Our old parallelized EDK builds took about 2m10s to do a DEBUG build for a
target, and with the edk2 it is 3m52s.
Thanks,
Andrew Fish
PS As we know from performance tuning the ROM the most important thing to do is
profile, and not listen to the old grumpy guy :).
_______________________________________________
edk2-devel mailing list
edk2-devel@lists.01.org
https://lists.01.org/mailman/listinfo/edk2-devel
