Andrew got this conversation started on the meeting minutes email. I
wanted to be sure it didn't get lost in the daily noise of the list.
I'll be pulling out some other discussions as well to give folks a
chance to comment before our next Community Meeting (happening next
week, invite coming soon).
Some ideas to start the discussion:
-How should we track which toolchains are being used, which are
validated, and which are known to create reproducible builds?
-When toolchains are known to be orphaned, should they be archived or
simply removed?
-Could we add some kconfig-like tool that allows introspection into what
type of builds are available?
-How can we better track the code quality of BaseTools and the current
build system on the whole?
-Should we add a "requires documentation change" flag to BZ so that it
will be easier to compile a list of required doc changes?
I've included Andrew's email for reference.
Cheers,
Stephano
> I'll help start the conversation on the build.
>
> 1) Making the process easier for newcomers.
>
> a) Getting it to build the 1st time
> I think we could aspire to a more Unix like flow:
> git clone
> ./configure
> make ovmf
> make ovmf.run
>
> The ./config step would automate resolving dependencies. Things like
figuring out what compiler is installed, what extra tools need to be
installed etc. It could be interactive?
>
> make <target> would build all the build tools, and do what ever local
config is required to get the build working. Then build the requested
target.
>
> make <target>.run is more of an optional step. It could do things
like launch Ovmf with your new ROM you just constructed.
>
> b) Debugging failures.
> Better quality error messages.
> Making the build.log from the build command a default setting(or have
an abstracted defined database that can be queried), and add
documentation on how to use it. Maybe make the output easier to consume?
>
> c) Making changes.
> That can be a phase 2.
>
> 2) Build performance, complexity, and maintainability.
> I'm about to complain about the build system, but I was around when
it was first invented so I'm to blame as much as anyone....
>
> We have way too much custom Python that is very complex, hard to
maintain, and slow. One thing we did better on the EDK was break up the
build tools into smaller chunks (maybe too small in some cases) and I
always found I could figure things out, but then again all the code was
C. Another interesting data point is we added parallel build, via
makefiles, to the EDK. When we moved from the EDK to the edk2 we noticed
the builds slowed down a lot. I did some profiling on the edk2 build and
I noticed 1,000,000 calls to regex (part of that long ..... sequence on
a build of a complex platform). It turns out we have custom Python code
that is generating the makefile dependencies so it was scanning all the
C code. The other issue I see is build parallelism is controlled from
the Python code. It also seems the edk2 build system is not a complete
solution and a lot of full platform implementations have shell scripts
or batch files, that wrap the calls to build.
>
> So I'd like to see a build system:
> 1) Start with a top level makefile in the root of the tree.
> 2) Replaces custom batch or shell scripts with makefiles to handle
pre and post build tasks.
> 3) Use a tool to construct a parser for the ekd2 build files (INF,
DSC, DEC). Use this data to construct the makefiles and build a database
for other build components to use. Chris Roberts has messed around and
built an Earley parser using the EBNF in the build documents. There are
some problematic issues in the grammar, but some grammar streamlining
might make it possible to use tooling to construct a parser vs. writing
one from scratch in Python.
> 4) Move to using the compiler to generate dependencies. gcc/clang
support this via -M*. I think it may be a little more complex for NMAKE
but there is a /showIncludes compiler flag.
> 5) Have the build tool terminate after parallel makefile generation
> 6) Top level makefile can invoke parallel build on generated makefiles.
> 7) Invoke build tool to generate FVs and FDs from makefile.
> 8) Invoke custom post processing steps in the makefile.
>
> Thanks,
> Andrew Fish
_______________________________________________
edk2-devel mailing list
edk2-devel@lists.01.org
https://lists.01.org/mailman/listinfo/edk2-devel
