On 9/27/19 1:03 PM, Alex Kiernan wrote:
On Thu, Sep 26, 2019 at 6:58 PM Joshua Watt <jpewhac...@gmail.com> wrote:
The deployment of the source date epoch file had a race condition where
any task attempting to read from the file would race with creation of
the sstate archive for the do_deploy_source_date_epoch task. The
creation of the sstate archive requires moving the directory to a
temporary location, then moving it back. This means that the file
disappears for a short period of time, which will cause a failure if any
other task is running and trying to open the file to get the current
source date epoch.
The solution is to copy the source date epoch file to a separate
directory when deploying so the file never disappears. When the file is
restored from sstate, it is moved to the correct location after being
extracted.
[YOCTO #13501]
Signed-off-by: Joshua Watt <jpewhac...@gmail.com>
---
meta/classes/reproducible_build.bbclass | 12 ++++++++++--
1 file changed, 10 insertions(+), 2 deletions(-)
diff --git a/meta/classes/reproducible_build.bbclass
b/meta/classes/reproducible_build.bbclass
index 8788ad7145c..99b749a9ee2 100644
--- a/meta/classes/reproducible_build.bbclass
+++ b/meta/classes/reproducible_build.bbclass
@@ -39,19 +39,27 @@ inherit
${@oe.utils.ifelse(d.getVar('BUILD_REPRODUCIBLE_BINARIES') == '1', 'repr
SDE_DIR ="${WORKDIR}/source-date-epoch"
SDE_FILE = "${SDE_DIR}/__source_date_epoch.txt"
+SDE_DEPLOYDIR = "${WORKDIR}/deploy-source-date-epoch"
SSTATETASKS += "do_deploy_source_date_epoch"
do_deploy_source_date_epoch () {
echo "Deploying SDE to ${SDE_DIR}."
+ mkdir -p ${SDE_DEPLOYDIR}
+ if [ -e ${SDE_FILE} ]; then
+ cp -p ${SDE_FILE} ${SDE_DEPLOYDIR}/__source_date_epoch.txt
+ fi
}
python do_deploy_source_date_epoch_setscene () {
sstate_setscene(d)
+ sde_file = os.path.join(d.getVar('SDE_DEPLOYDIR'),
'__source_date_epoch.txt')
+ if os.path.exists(sde_file):
+ os.rename(sde_file, d.getVar('SDE_FILE'))
}
-do_deploy_source_date_epoch[dirs] = "${SDE_DIR}"
-do_deploy_source_date_epoch[sstate-plaindirs] = "${SDE_DIR}"
+do_deploy_source_date_epoch[dirs] = "${SDE_DEPLOYDIR}"
+do_deploy_source_date_epoch[sstate-plaindirs] = "${SDE_DEPLOYDIR}"
addtask do_deploy_source_date_epoch_setscene
addtask do_deploy_source_date_epoch before do_configure after do_patch
This seems to have broken builds with reproducible builds enabled and
sstate downloaded from a mirror - I just get a blizzard of:
ERROR: Logfile of failure stored in:
/home/akiernan/nanohub/build/tmp/work/x86_64-linux/automake-native/1.16.1-r0/temp/log.do_deploy_source_date_epoch_setscene.113823
WARNING: Setscene task
(virtual:native:/home/akiernan/nanohub/build/../poky/meta/recipes-devtools/automake/automake_1.16.1.bb:do_deploy_source_date_epoch_setscene)
failed with exit code '1' - real task will be run instead
ERROR: autoconf-native-2.69-r11 do_deploy_source_date_epoch_setscene:
Error executing a python function in exec_python_func() autogenerated:
The stack trace of python calls that resulted in this exception/failure was:
File: 'exec_python_func() autogenerated', lineno: 2, function: <module>
0001:
*** 0002:do_deploy_source_date_epoch_setscene(d)
0003:
File:
'/home/akiernan/nanohub/build/../poky/meta/classes/reproducible_build.bbclass',
lineno: 58, function: do_deploy_source_date_epoch_setscene
0054:python do_deploy_source_date_epoch_setscene () {
0055: sstate_setscene(d)
0056: sde_file = os.path.join(d.getVar('SDE_DEPLOYDIR'),
'__source_date_epoch.txt')
0057: if os.path.exists(sde_file):
*** 0058: os.rename(sde_file, d.getVar('SDE_FILE'))
0059:}
0060:
0061:do_deploy_source_date_epoch[dirs] = "${SDE_DEPLOYDIR}"
0062:do_deploy_source_date_epoch[sstate-plaindirs] = "${SDE_DEPLOYDIR}"
Exception: FileNotFoundError: [Errno 2] No such file or directory:
'/home/akiernan/nanohub/build/tmp/work/x86_64-linux/autoconf-native/2.69-r11/deploy-source-date-epoch/__source_date_epoch.txt'
->
'/home/akiernan/nanohub/build/tmp/work/x86_64-linux/autoconf-native/2.69-r11/source-date-epoch/__source_date_epoch.txt'
Sorry about that. It looks like destination directory isn't getting
created. You can try giving
http://git.yoctoproject.org/cgit.cgi/poky-contrib/commit/?h=jpew/hash-equivalence&id=7177bb4c4c174f0852b984b9b2d2d23de92e70cd
a try to see if it resolves the issue.
ERROR: Logfile of failure stored in:
/home/akiernan/nanohub/build/tmp/work/x86_64-linux/autoconf-native/2.69-r11/temp/log.do_deploy_source_date_epoch_setscene.113869
WARNING: Setscene task
(virtual:native:/home/akiernan/nanohub/build/../poky/meta/recipes-devtools/autoconf/autoconf_2.69.bb:do_deploy_source_date_epoch_setscene)
failed with exit code '1' - real task will be run instead
ERROR: m4-native-1.4.18-r0 do_deploy_source_date_epoch_setscene: Error
executing a python function in exec_python_func() autogenerated:
The stack trace of python calls that resulted in this exception/failure was:
File: 'exec_python_func() autogenerated', lineno: 2, function: <module>
0001:
*** 0002:do_deploy_source_date_epoch_setscene(d)
0003:
File:
'/home/akiernan/nanohub/build/../poky/meta/classes/reproducible_build.bbclass',
lineno: 58, function: do_deploy_source_date_epoch_setscene
0054:python do_deploy_source_date_epoch_setscene () {
0055: sstate_setscene(d)
0056: sde_file = os.path.join(d.getVar('SDE_DEPLOYDIR'),
'__source_date_epoch.txt')
0057: if os.path.exists(sde_file):
*** 0058: os.rename(sde_file, d.getVar('SDE_FILE'))
0059:}
0060:
0061:do_deploy_source_date_epoch[dirs] = "${SDE_DEPLOYDIR}"
0062:do_deploy_source_date_epoch[sstate-plaindirs] = "${SDE_DEPLOYDIR}"
Exception: FileNotFoundError: [Errno 2] No such file or directory:
'/home/akiernan/nanohub/build/tmp/work/x86_64-linux/m4-native/1.4.18-r0/deploy-source-date-epoch/__source_date_epoch.txt'
->
'/home/akiernan/nanohub/build/tmp/work/x86_64-linux/m4-native/1.4.18-r0/source-date-epoch/__source_date_epoch.txt'
ERROR: Logfile of failure stored in:
/home/akiernan/nanohub/build/tmp/work/x86_64-linux/m4-native/1.4.18-r0/temp/log.do_deploy_source_date_epoch_setscene.113897
--
_______________________________________________
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core
