linguini1 commented on code in PR #17162: URL: https://github.com/apache/nuttx/pull/17162#discussion_r2525148570
########## Documentation/implementation/make_build_system.rst: ########## @@ -0,0 +1,603 @@ +================= +Make Build System +================= + +Currently, NuttX supports both CMake and Make build systems. +This guide explains the NuttX `make`-based build system. + +Due to *requirements, constraints, and the complexity of the build process*, NuttX divides +this work into multiple files, each handling specific parts of the build process. + +As stated in :doc:`/introduction/inviolables`, multiple platforms should be supported: + +- :ref:`win_mk`: handles windows platform support. +- :ref:`unix_mk`: handles unix-like platforms support. + +NuttX supports multiple build modes. See :doc:`/guides/protected_build`: + +- :ref:`flatlibs_mk`: Kernel and user-space built into a single `blob`. +- :ref:`protectedlibs_mk`: Kernel and user-space built as two separate `blobs`. +- :ref:`kernelibs_mk`: Kernel built into single `blob`. User apps must be loaded + into memory for execution. + +NuttX targets multiple libs, or `silos`, each handling its own compilation: + +.. note:: + + Gregory Nutt has a nice presentation about + `NuttX architecture <https://cwiki.apache.org/confluence/pages/viewpage.action? + pageId=139629399&preview=/139629402/140774623/nuttx-3-archoverview.pdf>`_ + + There the `silo` concept is explained. Only the `silos` there are listed below as libs. Review Comment: When use you the back ticks, are intending for italics? Because you need double back ticks to achieve truetype/code formatting (`silo`). I forget this every time I write RST until I render it. ########## Documentation/implementation/index.rst: ########## @@ -6,6 +6,7 @@ Implementation Details :maxdepth: 2 :caption: Contents: + make_build_system.rst Review Comment: If all of the files in this directory are to be listed, I'd just use a glob path to avoid having to update the listing each time a page is added. ########## Documentation/implementation/make_build_system.rst: ########## @@ -0,0 +1,603 @@ +================= +Make Build System +================= + +Currently, NuttX supports both CMake and Make build systems. +This guide explains the NuttX `make`-based build system. + +Due to *requirements, constraints, and the complexity of the build process*, NuttX divides +this work into multiple files, each handling specific parts of the build process. + +As stated in :doc:`/introduction/inviolables`, multiple platforms should be supported: + +- :ref:`win_mk`: handles windows platform support. +- :ref:`unix_mk`: handles unix-like platforms support. + +NuttX supports multiple build modes. See :doc:`/guides/protected_build`: + +- :ref:`flatlibs_mk`: Kernel and user-space built into a single `blob`. +- :ref:`protectedlibs_mk`: Kernel and user-space built as two separate `blobs`. +- :ref:`kernelibs_mk`: Kernel built into single `blob`. User apps must be loaded + into memory for execution. + +NuttX targets multiple libs, or `silos`, each handling its own compilation: + +.. note:: + + Gregory Nutt has a nice presentation about + `NuttX architecture <https://cwiki.apache.org/confluence/pages/viewpage.action? + pageId=139629399&preview=/139629402/140774623/nuttx-3-archoverview.pdf>`_ + + There the `silo` concept is explained. Only the `silos` there are listed below as libs. + The build mode influences the needed libs. + +.. code-block:: bash + + $ ls -l staging/ + drwxr-xr-x 2 xxx xxx 4096 Oct 6 16:02 . + drwxr-xr-x 27 xxx xxx 4096 Oct 6 16:02 .. + -rw-r--r-- 1 xxx xxx 323640 Oct 6 16:02 libapps.a + -rw-r--r-- 1 xxx xxx 384352 Oct 6 16:02 libarch.a + -rw-r--r-- 1 xxx xxx 62182 Oct 6 16:02 libbinfmt.a + -rw-r--r-- 1 xxx xxx 6468 Oct 6 16:01 libboards.a + -rw-r--r-- 1 xxx xxx 2820054 Oct 6 16:02 libc.a + -rw-r--r-- 1 xxx xxx 161486 Oct 6 16:01 libdrivers.a + -rw-r--r-- 1 xxx xxx 981638 Oct 6 16:02 libfs.a + -rw-r--r-- 1 xxx xxx 224446 Oct 6 16:02 libmm.a + -rw-r--r-- 1 xxx xxx 2435746 Oct 6 16:01 libsched.a + -rw-r--r-- 1 xxx xxx 51768 Oct 6 16:02 libxx.a + +.. _verbosity: + +Verbosity +--------- + +The ``V`` variable can be passed to ``make`` to control the build verbosity. + +- **Quiet (Default):** The build output is minimal. +- **Verbose (`V=1 V=2`):** Shows the full compiler commands *(enables command echo)*. +- **Verbose Tools (`V=2`):** Enables verbose output for tools and scripts. + +.. code-block:: console + + # V=1,2: Enable echo of commands + $ make V=1 + + # V=2: Enable bug/verbose options in tools and scripts + $ make V=2 + +Build Process +------------- + +.. note:: + + Keep the configuration step and board folder layout short. + Separate docs should be created. + +The ``Make`` based build process starts with the NuttX tree configuration. +This is done by running ``tools/configure.sh`` script. +The configuration step, prepares the NuttX kernel tree, setting the board specific +arch, chip and board files. + +The ``Make`` build system refers the needed subsystems using *generic* naming: + +- The *current* architecture is refered as ``arch`` +- The *current* chip is refered as ``chip`` +- The *current* board is refered as ``board`` + +These *generic* names are mapped to the *actual* names by symlinks: + +- The *current* chip directory gets symlinked to ``nutt/include/arch/chip``. +- The *current* board directory gets symlinked to ``nutt/include/arch/board``. +- The *current* arch directory gets symlinked to ``nutt/include/arch``. + +The board config is stored as ``defconfig`` file, which is a minimal config, +storing only the configs that differs from default config values. +Due to NuttX's particularity of strict dependance to the ``app`` +directory, the ``.config`` file is not generated by either ``kconfiglib`` or +``kconfig-frontends``, but rather by the an in-tree ``tools/process_config.sh`` +script. This script takes a "base" input file (the boards ``defconfig`` file), +additional include paths (the most relevant being the ``apps`` top directory), +and generate an output file (the ``$(TOPDIR)/.config`` file). + +.. code-block:: bash + + # part of configure.sh shell script, starting at line 240 + # + # src_config=${configpath}/defconfig + # dest_config="${TOPDIR}/.config" + # original_config="${TOPDIR}/.config.orig" + # backup_config="${TOPDIR}/defconfig" + + $ ln -sf ${src_makedefs} ${dest_makedefs} || \ + { echo "Failed to symlink ${src_makedefs}" ; exit 8 ; } + $ ${TOPDIR}/tools/process_config.sh -I ${configpath}/../../common/configs \ + -I ${configpath}/../common \ + -I ${configpath} -I ${TOPDIR}/../apps \ + -I ${TOPDIR}/../nuttx-apps \ + -o ${dest_config} ${src_config} + +Starting the Build +^^^^^^^^^^^^^^^^^^ + +The root **Makefile** is the build process entrypoint. Its main job is +to check for a ``.config`` file and include the appropriate **host-specific Makefile**. +The "actual" first steps of the build process are handled by the host-specific +**Makefile** (either ``Win.mk`` or ``Unix.mk``). + +Early on, during parsing, both host-specific **Makefile** will also include + +- board's ``Make.defs`` file mentioned above. this will include also + - the main ``.config`` file. + - the tools ``config.mk`` file. + - the arch ``toolchain.defs`` file. +- based on the build mode, one of the following files: + - :ref:`flatlibs_mk` + - :ref:`protectedlibs_mk` + - :ref:`kernelibs_mk` +- :ref:`directories_mk` + +Built-in dependency mechanism +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +.. note:: + + The documentation for :ref:`mkdeps` should be extended. Lots of details + are missing. Part of the documentation here should be moved. + +NuttX implements a built-in dependency mechanism. See :ref:`mkdeps`. +This mechanism uses ``gcc`` to generate a ``*.d`` like dependency files that can be +included by the **Makefile** to track file dependencies. +This mechanism is kick-started by the ``depend`` target, and targets specific +directories in the NuttX tree based on the build mode. It is *required* by both +``pass1dep`` and ``pass2dep``. + +.. code-block:: makefile + + pass1dep: context tools/mkdeps$(HOSTEXEEXT) tools/cnvwindeps$(HOSTEXEEXT) + $(Q) for dir in $(USERDEPDIRS) ; do \ + $(MAKE) -C $$dir depend || exit; \ + done + pass2dep: context tools/mkdeps$(HOSTEXEEXT) tools/cnvwindeps$(HOSTEXEEXT) + $(Q) for dir in $(KERNDEPDIRS) ; do \ + $(MAKE) -C $$dir EXTRAFLAGS="$(KDEFINE) $(EXTRAFLAGS)" depend || exit; \ + done + +Both ``pass1dep`` and ``pass2dep`` sets different directories and orders +in which the ``depend`` target is ran. See :ref:`directories_mk`. + +Building NuttX libs +^^^^^^^^^^^^^^^^^^^ + +The host-specific **Makefile** will not build the required NuttX libs. +It will defer work by "recursively" calling make in each of the directories +listed in the ``$(USERLIBS)`` and ``$(NUTTXLIBS)`` variables. + +pass1 & pass2 +^^^^^^^^^^^^^ + +The NuttX binary is always generated by running both ``pass1`` and ``pass2`` targets. +The actual dependencies of the mentioned targets may vary depending on the build mode. + +Different NuttX build modes will not influence the "execution" of the ``pass1`` and ``pass2`` targets, +but rather will influence the dependencies pulled by those targets. + +- ``pass1`` target depends on the ``$(USERLIBS)``. +- ``pass2`` target depends on the ``$(NUTTXLIBS)``. + +.. code-block:: makefile + + all: pass1 pass2 + + pass1: $(USERLIBS) + pass2: $(NUTTXLIBS) + +The content of the ``$(USERLIBS)`` and ``$(NUTTXLIBS)`` variables is defined in each build mode makefile. +See :ref:`build_modes` above. + +Staging the libs +^^^^^^^^^^^^^^^^ + +After compiling libraries defined by the ``$(USERLIBS)`` and ``$(NUTTXLIBS)`` at the previous step, +the ``make`` build system will ``install`` them at a special ``staging/`` directory, residing at the +root of the NuttX tree. + +These libraries are passed to the the final target rule (``$(BIN)``). + +.. code-block:: makefile + + # Unix.mk : line 536 + $(BIN): pass1 pass2 + # ... + $(Q) $(MAKE) -C $(ARCH_SRC) EXTRA_OBJS="$(EXTRA_OBJS)" LINKLIBS="$(LINKLIBS)" APPDIR="$(APPDIR)" EXTRAFLAGS="$(KDEFINE) $(EXTRAFLAGS)" $(BIN) + # ... + +.. _win_mk: + +Win.mk +------ + +Although targeting different platforms, both **Win.mk** and **Unix.mk** aim to produce +the same output. The need for independent files is due to the differences in the +platform's approaches. + +forward vs back slashes Review Comment: Capitalize to be consistent ########## Documentation/implementation/make_build_system.rst: ########## @@ -0,0 +1,603 @@ +================= +Make Build System +================= + +Currently, NuttX supports both CMake and Make build systems. +This guide explains the NuttX `make`-based build system. + +Due to *requirements, constraints, and the complexity of the build process*, NuttX divides +this work into multiple files, each handling specific parts of the build process. + +As stated in :doc:`/introduction/inviolables`, multiple platforms should be supported: + +- :ref:`win_mk`: handles windows platform support. +- :ref:`unix_mk`: handles unix-like platforms support. + +NuttX supports multiple build modes. See :doc:`/guides/protected_build`: + +- :ref:`flatlibs_mk`: Kernel and user-space built into a single `blob`. +- :ref:`protectedlibs_mk`: Kernel and user-space built as two separate `blobs`. +- :ref:`kernelibs_mk`: Kernel built into single `blob`. User apps must be loaded + into memory for execution. + +NuttX targets multiple libs, or `silos`, each handling its own compilation: + +.. note:: + + Gregory Nutt has a nice presentation about + `NuttX architecture <https://cwiki.apache.org/confluence/pages/viewpage.action? + pageId=139629399&preview=/139629402/140774623/nuttx-3-archoverview.pdf>`_ + + There the `silo` concept is explained. Only the `silos` there are listed below as libs. + The build mode influences the needed libs. + +.. code-block:: bash + + $ ls -l staging/ + drwxr-xr-x 2 xxx xxx 4096 Oct 6 16:02 . + drwxr-xr-x 27 xxx xxx 4096 Oct 6 16:02 .. + -rw-r--r-- 1 xxx xxx 323640 Oct 6 16:02 libapps.a + -rw-r--r-- 1 xxx xxx 384352 Oct 6 16:02 libarch.a + -rw-r--r-- 1 xxx xxx 62182 Oct 6 16:02 libbinfmt.a + -rw-r--r-- 1 xxx xxx 6468 Oct 6 16:01 libboards.a + -rw-r--r-- 1 xxx xxx 2820054 Oct 6 16:02 libc.a + -rw-r--r-- 1 xxx xxx 161486 Oct 6 16:01 libdrivers.a + -rw-r--r-- 1 xxx xxx 981638 Oct 6 16:02 libfs.a + -rw-r--r-- 1 xxx xxx 224446 Oct 6 16:02 libmm.a + -rw-r--r-- 1 xxx xxx 2435746 Oct 6 16:01 libsched.a + -rw-r--r-- 1 xxx xxx 51768 Oct 6 16:02 libxx.a + +.. _verbosity: + +Verbosity +--------- + +The ``V`` variable can be passed to ``make`` to control the build verbosity. + +- **Quiet (Default):** The build output is minimal. +- **Verbose (`V=1 V=2`):** Shows the full compiler commands *(enables command echo)*. +- **Verbose Tools (`V=2`):** Enables verbose output for tools and scripts. + +.. code-block:: console + + # V=1,2: Enable echo of commands + $ make V=1 + + # V=2: Enable bug/verbose options in tools and scripts + $ make V=2 + +Build Process +------------- + +.. note:: + + Keep the configuration step and board folder layout short. + Separate docs should be created. + +The ``Make`` based build process starts with the NuttX tree configuration. +This is done by running ``tools/configure.sh`` script. +The configuration step, prepares the NuttX kernel tree, setting the board specific +arch, chip and board files. + +The ``Make`` build system refers the needed subsystems using *generic* naming: + +- The *current* architecture is refered as ``arch`` +- The *current* chip is refered as ``chip`` +- The *current* board is refered as ``board`` + +These *generic* names are mapped to the *actual* names by symlinks: + +- The *current* chip directory gets symlinked to ``nutt/include/arch/chip``. +- The *current* board directory gets symlinked to ``nutt/include/arch/board``. +- The *current* arch directory gets symlinked to ``nutt/include/arch``. + +The board config is stored as ``defconfig`` file, which is a minimal config, +storing only the configs that differs from default config values. +Due to NuttX's particularity of strict dependance to the ``app`` +directory, the ``.config`` file is not generated by either ``kconfiglib`` or +``kconfig-frontends``, but rather by the an in-tree ``tools/process_config.sh`` +script. This script takes a "base" input file (the boards ``defconfig`` file), +additional include paths (the most relevant being the ``apps`` top directory), +and generate an output file (the ``$(TOPDIR)/.config`` file). + +.. code-block:: bash + + # part of configure.sh shell script, starting at line 240 + # + # src_config=${configpath}/defconfig + # dest_config="${TOPDIR}/.config" + # original_config="${TOPDIR}/.config.orig" + # backup_config="${TOPDIR}/defconfig" + + $ ln -sf ${src_makedefs} ${dest_makedefs} || \ + { echo "Failed to symlink ${src_makedefs}" ; exit 8 ; } + $ ${TOPDIR}/tools/process_config.sh -I ${configpath}/../../common/configs \ + -I ${configpath}/../common \ + -I ${configpath} -I ${TOPDIR}/../apps \ + -I ${TOPDIR}/../nuttx-apps \ + -o ${dest_config} ${src_config} + +Starting the Build +^^^^^^^^^^^^^^^^^^ + +The root **Makefile** is the build process entrypoint. Its main job is +to check for a ``.config`` file and include the appropriate **host-specific Makefile**. +The "actual" first steps of the build process are handled by the host-specific +**Makefile** (either ``Win.mk`` or ``Unix.mk``). + +Early on, during parsing, both host-specific **Makefile** will also include + +- board's ``Make.defs`` file mentioned above. this will include also + - the main ``.config`` file. + - the tools ``config.mk`` file. + - the arch ``toolchain.defs`` file. +- based on the build mode, one of the following files: + - :ref:`flatlibs_mk` + - :ref:`protectedlibs_mk` + - :ref:`kernelibs_mk` +- :ref:`directories_mk` + +Built-in dependency mechanism +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +.. note:: + + The documentation for :ref:`mkdeps` should be extended. Lots of details + are missing. Part of the documentation here should be moved. + +NuttX implements a built-in dependency mechanism. See :ref:`mkdeps`. +This mechanism uses ``gcc`` to generate a ``*.d`` like dependency files that can be +included by the **Makefile** to track file dependencies. +This mechanism is kick-started by the ``depend`` target, and targets specific +directories in the NuttX tree based on the build mode. It is *required* by both +``pass1dep`` and ``pass2dep``. + +.. code-block:: makefile + + pass1dep: context tools/mkdeps$(HOSTEXEEXT) tools/cnvwindeps$(HOSTEXEEXT) + $(Q) for dir in $(USERDEPDIRS) ; do \ + $(MAKE) -C $$dir depend || exit; \ + done + pass2dep: context tools/mkdeps$(HOSTEXEEXT) tools/cnvwindeps$(HOSTEXEEXT) + $(Q) for dir in $(KERNDEPDIRS) ; do \ + $(MAKE) -C $$dir EXTRAFLAGS="$(KDEFINE) $(EXTRAFLAGS)" depend || exit; \ + done + +Both ``pass1dep`` and ``pass2dep`` sets different directories and orders +in which the ``depend`` target is ran. See :ref:`directories_mk`. + +Building NuttX libs +^^^^^^^^^^^^^^^^^^^ + +The host-specific **Makefile** will not build the required NuttX libs. +It will defer work by "recursively" calling make in each of the directories +listed in the ``$(USERLIBS)`` and ``$(NUTTXLIBS)`` variables. + +pass1 & pass2 +^^^^^^^^^^^^^ + +The NuttX binary is always generated by running both ``pass1`` and ``pass2`` targets. +The actual dependencies of the mentioned targets may vary depending on the build mode. + +Different NuttX build modes will not influence the "execution" of the ``pass1`` and ``pass2`` targets, +but rather will influence the dependencies pulled by those targets. + +- ``pass1`` target depends on the ``$(USERLIBS)``. +- ``pass2`` target depends on the ``$(NUTTXLIBS)``. + +.. code-block:: makefile + + all: pass1 pass2 + + pass1: $(USERLIBS) + pass2: $(NUTTXLIBS) + +The content of the ``$(USERLIBS)`` and ``$(NUTTXLIBS)`` variables is defined in each build mode makefile. +See :ref:`build_modes` above. + +Staging the libs +^^^^^^^^^^^^^^^^ + +After compiling libraries defined by the ``$(USERLIBS)`` and ``$(NUTTXLIBS)`` at the previous step, +the ``make`` build system will ``install`` them at a special ``staging/`` directory, residing at the +root of the NuttX tree. + +These libraries are passed to the the final target rule (``$(BIN)``). + +.. code-block:: makefile + + # Unix.mk : line 536 + $(BIN): pass1 pass2 + # ... + $(Q) $(MAKE) -C $(ARCH_SRC) EXTRA_OBJS="$(EXTRA_OBJS)" LINKLIBS="$(LINKLIBS)" APPDIR="$(APPDIR)" EXTRAFLAGS="$(KDEFINE) $(EXTRAFLAGS)" $(BIN) + # ... + +.. _win_mk: + +Win.mk +------ + +Although targeting different platforms, both **Win.mk** and **Unix.mk** aim to produce +the same output. The need for independent files is due to the differences in the +platform's approaches. + +forward vs back slashes +^^^^^^^^^^^^^^^^^^^^^^^ + +One of the main differences is the use of forward slashes +(``/``) on unix-like platforms versus backslashes (``\``) on windows + +${HOSTEXEEXT} ${HOSTDYNEXT} +^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +These variables are used by the build system to configure the executable suffix required +by the used platform. They are defined in :ref:`config_mk`. + +For windows platform: + +- ``${HOSTEXEEXT}`` is set to ``.exe``. +- ``${HOSTDYNEXT}`` is set to ``.dll``. + +Symbolic Linking +^^^^^^^^^^^^^^^^ + +For the windows platform, the build system handles symbolic links differently. + +.. _unix_mk: + +Unix.mk +------- + +Versioning +^^^^^^^^^^ + +The build system will impact versioning if NuttX is cloned as a repo. See :ref:`versioning`. + +config.h .config mkconfig Review Comment: Surround with double back ticks each, and put commas between each item -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
