Re: Packages buildbot is erratic, both master and 23.05 packages fail often
The sender domain has a DMARC Reject/Quarantine policy which disallows sending mailing list messages using the original "From" header. To mitigate this problem, the original message has been wrapped automatically by the mailing list software.--- Begin Message --- Hi all, > > Looking at that observation about gettext and recursive "confdir3/", it is > > plausible that gettext has problem that manifests in some builds, or > > trouble with parallelism on some occasions. > > > > Gettext was heavily reorganised in May, near the same time as the buildbot > > code was revamped. So, this might quite well be related to the gettext > > package and not the new buildbot code. There is no such subdirectory in gettext source like "confdir3". I don't really know how buildbot builds differ from normal builds on a local machine, so my opinion is not worth much right now... If I had to guess something wrong with gettext right now, it would be that this has something to do with the fact that HOST_BUILD_DIR and PKG_BUILD_DIR currently have a custom definition for gettext, so that in build_dir it is a path named "gettext" instead of "gettext-full". However, that is expected to be completely harmless. > > No gettext completion before the final timeout error. Hunderds of other > > packages were compiled in the time when gettext was was being recursively > > compiled? > > > I wouldn’t pay too much attention to this build failure until the space > problems are resolved. > Running out of space can wreck havoc in many different ways and we may simply > be looking at side effects (possibly across containers) of that. > I can't help but agree. It's very strange to me to see a subdirectory called "confdir3", let alone that it is infinitely recursing. One of the changes I made was to select subdirectories to build using a variable value of "SUBDIRS" with the Makefile in order to remove a patch, but of course, none of the subdirectories have a name like that... Hope you can fix it, -- MCP --- End Message --- ___ openwrt-devel mailing list openwrt-devel@lists.openwrt.org https://lists.openwrt.org/mailman/listinfo/openwrt-devel
[PATCH v2] image: fix device profile specific COMPILE targets
The sender domain has a DMARC Reject/Quarantine policy which disallows
sending mailing list messages using the original "From" header.
To mitigate this problem, the original message has been wrapped
automatically by the mailing list software.--- Begin Message ---
Commit a01d23e75 ("image: always rebuild kernel loaders")
is a step in the right direction, but exposed some issues
and regressions in the makefile.
1. Some of the files made by device specific COMPILE targets
start with an "append" command (i.e. >> instead of > redirection)
and if the file already exists, the target file is an
input to itself before the first recipe input to the file.
2. Some of the device specific COMPILE targets,
like the lzma-loader have multiple steps
and the uImage target requires the bin or ELF target first,
so the two COMPILE targets cannot be run in parallel.
3. A side-effect of FORCE, is that all COMPILE targets
will be built, even for devices in a build that are not selected,
causing for example, the lzma-loader for some devices
to be built when the image for that device is not built.
4. By making the COMPILE targets a prerequisite of
the "target/compile" phase of the build, it is being built
again in the "target/install" phase of the build, since
the compile phase is a prerequisite of the install phase
and the FORCE target is used on each file in each submake,
but COMPILE targets are not needed before "target/install".
This commit resolves these issues by calling COMPILE targets
only in "target/install", using pattern substitution to list
prerequisites between COMPILE targets in order, deleting
the target before building it, and using device profile
selection to determine if the COMPILE targets are FORCE built.
Because of the nature of defining prerequisites in Make,
(reverse dependency before dependency)
the entire list of COMPILE targets is reversed
in order to list prerequisites between them
without having to work with individual words.
Fixes: a01d23e75 ("image: always rebuild kernel loaders")
Fixes: a7fb589e8 ("image: always rebuild kernel loaders")
Signed-off-by: Michael Pratt
---
include/image.mk | 8 ++--
rules.mk | 2 ++
2 files changed, 8 insertions(+), 2 deletions(-)
diff --git a/include/image.mk b/include/image.mk
index e9dc53f82e..c09dba3f2c 100644
--- a/include/image.mk
+++ b/include/image.mk
@@ -517,9 +517,13 @@ endef
endif
define Device/Build/compile
- $$(_COMPILE_TARGET): $(KDIR)/$(1)
+ $$(_TARGET): $(KDIR)/$(1)
$(eval $(call Device/Export,$(KDIR)/$(1)))
- $(KDIR)/$(1): FORCE
+
+ $(patsubst %$(1),%$(1):,$(call reverse,$(foreach
compile,$(COMPILE),$(KDIR)/$(compile
+
+ $(KDIR)/$(1): $(if $(_PROFILE_SET),FORCE)
+ rm -f $(KDIR)/$(1)
$$(call concat_cmd,$(COMPILE/$(1)))
endef
diff --git a/rules.mk b/rules.mk
index 2de43d490d..782f84d625 100644
--- a/rules.mk
+++ b/rules.mk
@@ -53,6 +53,8 @@ __tr_template = $(__tr_head)$$(1)$(__tr_tail)
$(eval toupper = $(call __tr_template,$(chars_lower),$(chars_upper)))
$(eval tolower = $(call __tr_template,$(chars_upper),$(chars_lower)))
+reverse = $(if $(word 2,$(1)),$(call reverse,$(wordlist 2,$(words $(1)),$(1)))
$(firstword $(1)),$(1))
+
version_abbrev = $(if $(if $(CHECK),,$(DUMP)),$(1),$(shell printf '%.8s' $(1)))
_SINGLE=export MAKEFLAGS=$(space);
--
2.30.2
--- End Message ---
___
openwrt-devel mailing list
openwrt-devel@lists.openwrt.org
https://lists.openwrt.org/mailman/listinfo/openwrt-devel
[PATCH] image: fix device profile specific COMPILE targets
The sender domain has a DMARC Reject/Quarantine policy which disallows
sending mailing list messages using the original "From" header.
To mitigate this problem, the original message has been wrapped
automatically by the mailing list software.--- Begin Message ---
Commit a01d23e75 ("image: always rebuild kernel loaders")
is a step in the right direction, but exposed some issues
and regressions in the makefile.
1. Some of the files made by device specific COMPILE targets
start with an "append" command (i.e. >> instead of > redirection)
and if the file already exists, the target file is the
input to itself before the first recipe-specified input.
2. Some of the device specific COMPILE targets, like the lzma-loader
have multiple steps and the uImage target requires the bin or ELF first,
so the two COMPILE operations cannot be run in parallel.
Fixes: a01d23e75 ("image: always rebuild kernel loaders")
Fixes: a7fb589e8 ("image: always rebuild kernel loaders")
Signed-off-by: Michael Pratt
---
include/image.mk | 3 +++
1 file changed, 3 insertions(+)
diff --git a/include/image.mk b/include/image.mk
index e9dc53f82e..575df15899 100644
--- a/include/image.mk
+++ b/include/image.mk
@@ -520,8 +520,11 @@ define Device/Build/compile
$$(_COMPILE_TARGET): $(KDIR)/$(1)
$(eval $(call Device/Export,$(KDIR)/$(1)))
$(KDIR)/$(1): FORCE
+ rm -f $(KDIR)/$(1)
$$(call concat_cmd,$(COMPILE/$(1)))
+ .NOTPARALLEL: $$(_COMPILE_TARGET)
+
endef
ifndef IB
--
2.30.2
--- End Message ---
___
openwrt-devel mailing list
openwrt-devel@lists.openwrt.org
https://lists.openwrt.org/mailman/listinfo/openwrt-devel
[PATCH] build: touch stampfile after subtarget run
The sender domain has a DMARC Reject/Quarantine policy which disallows sending mailing list messages using the original "From" header. To mitigate this problem, the original message has been wrapped automatically by the mailing list software.--- Begin Message --- Each individual build directory has stampfiles that are touched as part of their build process when each stage of the build process completes. However, the subtargets themselves do not touch the stampfiles that are defined for them. They are only touched when a target that has that stampfile as a prerequisite is ran. For example, "make tools/compile" will not touch .tools_compile_... but "make toolchain/compile" will, after each build directory in tools/ is checked again, because the stampfile is a prerequisite, not the tools/compile target itself. This makes each subtarget touch a stampfile, if defined, when the subtarget has completed successfully. A small amount of build time is expected to be saved when rebuilding after 'make clean' or an interruption. Signed-off-by: Michael Pratt --- include/subdir.mk | 1 + 1 file changed, 1 insertion(+) diff --git a/include/subdir.mk b/include/subdir.mk index 95009f814e..6d3bd55994 100644 --- a/include/subdir.mk +++ b/include/subdir.mk @@ -16,6 +16,7 @@ subtarget-default = $(filter-out ., \ define subtarget $(call warn_eval,$(1),t,T,$(1)/$(2): $($(1)/) $(foreach bd,$(call subtarget-default,$(1),$(2)),$(1)/$(bd)/$(2))) + -touch $($(1)/stamp-$(2)) endef -- 2.30.2 --- End Message --- ___ openwrt-devel mailing list openwrt-devel@lists.openwrt.org https://lists.openwrt.org/mailman/listinfo/openwrt-devel
