On Thu, Sep 26, 2019 at 6:58 PM Joshua Watt <jpewhac...@gmail.com> wrote: > > The deployment of the source date epoch file had a race condition where > any task attempting to read from the file would race with creation of > the sstate archive for the do_deploy_source_date_epoch task. The > creation of the sstate archive requires moving the directory to a > temporary location, then moving it back. This means that the file > disappears for a short period of time, which will cause a failure if any > other task is running and trying to open the file to get the current > source date epoch. > > The solution is to copy the source date epoch file to a separate > directory when deploying so the file never disappears. When the file is > restored from sstate, it is moved to the correct location after being > extracted. > > [YOCTO #13501] > > Signed-off-by: Joshua Watt <jpewhac...@gmail.com> > --- > meta/classes/reproducible_build.bbclass | 12 ++++++++++-- > 1 file changed, 10 insertions(+), 2 deletions(-) > > diff --git a/meta/classes/reproducible_build.bbclass > b/meta/classes/reproducible_build.bbclass > index 8788ad7145c..99b749a9ee2 100644 > --- a/meta/classes/reproducible_build.bbclass > +++ b/meta/classes/reproducible_build.bbclass > @@ -39,19 +39,27 @@ inherit > ${@oe.utils.ifelse(d.getVar('BUILD_REPRODUCIBLE_BINARIES') == '1', 'repr > > SDE_DIR ="${WORKDIR}/source-date-epoch" > SDE_FILE = "${SDE_DIR}/__source_date_epoch.txt" > +SDE_DEPLOYDIR = "${WORKDIR}/deploy-source-date-epoch" > > SSTATETASKS += "do_deploy_source_date_epoch" > > do_deploy_source_date_epoch () { > echo "Deploying SDE to ${SDE_DIR}." > + mkdir -p ${SDE_DEPLOYDIR} > + if [ -e ${SDE_FILE} ]; then > + cp -p ${SDE_FILE} ${SDE_DEPLOYDIR}/__source_date_epoch.txt > + fi > } > > python do_deploy_source_date_epoch_setscene () { > sstate_setscene(d) > + sde_file = os.path.join(d.getVar('SDE_DEPLOYDIR'), > '__source_date_epoch.txt') > + if os.path.exists(sde_file): > + os.rename(sde_file, d.getVar('SDE_FILE')) > } > > -do_deploy_source_date_epoch[dirs] = "${SDE_DIR}" > -do_deploy_source_date_epoch[sstate-plaindirs] = "${SDE_DIR}" > +do_deploy_source_date_epoch[dirs] = "${SDE_DEPLOYDIR}" > +do_deploy_source_date_epoch[sstate-plaindirs] = "${SDE_DEPLOYDIR}" > addtask do_deploy_source_date_epoch_setscene > addtask do_deploy_source_date_epoch before do_configure after do_patch >
This seems to have broken builds with reproducible builds enabled and sstate downloaded from a mirror - I just get a blizzard of: ERROR: Logfile of failure stored in: /home/akiernan/nanohub/build/tmp/work/x86_64-linux/automake-native/1.16.1-r0/temp/log.do_deploy_source_date_epoch_setscene.113823 WARNING: Setscene task (virtual:native:/home/akiernan/nanohub/build/../poky/meta/recipes-devtools/automake/automake_1.16.1.bb:do_deploy_source_date_epoch_setscene) failed with exit code '1' - real task will be run instead ERROR: autoconf-native-2.69-r11 do_deploy_source_date_epoch_setscene: Error executing a python function in exec_python_func() autogenerated: The stack trace of python calls that resulted in this exception/failure was: File: 'exec_python_func() autogenerated', lineno: 2, function: <module> 0001: *** 0002:do_deploy_source_date_epoch_setscene(d) 0003: File: '/home/akiernan/nanohub/build/../poky/meta/classes/reproducible_build.bbclass', lineno: 58, function: do_deploy_source_date_epoch_setscene 0054:python do_deploy_source_date_epoch_setscene () { 0055: sstate_setscene(d) 0056: sde_file = os.path.join(d.getVar('SDE_DEPLOYDIR'), '__source_date_epoch.txt') 0057: if os.path.exists(sde_file): *** 0058: os.rename(sde_file, d.getVar('SDE_FILE')) 0059:} 0060: 0061:do_deploy_source_date_epoch[dirs] = "${SDE_DEPLOYDIR}" 0062:do_deploy_source_date_epoch[sstate-plaindirs] = "${SDE_DEPLOYDIR}" Exception: FileNotFoundError: [Errno 2] No such file or directory: '/home/akiernan/nanohub/build/tmp/work/x86_64-linux/autoconf-native/2.69-r11/deploy-source-date-epoch/__source_date_epoch.txt' -> '/home/akiernan/nanohub/build/tmp/work/x86_64-linux/autoconf-native/2.69-r11/source-date-epoch/__source_date_epoch.txt' ERROR: Logfile of failure stored in: /home/akiernan/nanohub/build/tmp/work/x86_64-linux/autoconf-native/2.69-r11/temp/log.do_deploy_source_date_epoch_setscene.113869 WARNING: Setscene task (virtual:native:/home/akiernan/nanohub/build/../poky/meta/recipes-devtools/autoconf/autoconf_2.69.bb:do_deploy_source_date_epoch_setscene) failed with exit code '1' - real task will be run instead ERROR: m4-native-1.4.18-r0 do_deploy_source_date_epoch_setscene: Error executing a python function in exec_python_func() autogenerated: The stack trace of python calls that resulted in this exception/failure was: File: 'exec_python_func() autogenerated', lineno: 2, function: <module> 0001: *** 0002:do_deploy_source_date_epoch_setscene(d) 0003: File: '/home/akiernan/nanohub/build/../poky/meta/classes/reproducible_build.bbclass', lineno: 58, function: do_deploy_source_date_epoch_setscene 0054:python do_deploy_source_date_epoch_setscene () { 0055: sstate_setscene(d) 0056: sde_file = os.path.join(d.getVar('SDE_DEPLOYDIR'), '__source_date_epoch.txt') 0057: if os.path.exists(sde_file): *** 0058: os.rename(sde_file, d.getVar('SDE_FILE')) 0059:} 0060: 0061:do_deploy_source_date_epoch[dirs] = "${SDE_DEPLOYDIR}" 0062:do_deploy_source_date_epoch[sstate-plaindirs] = "${SDE_DEPLOYDIR}" Exception: FileNotFoundError: [Errno 2] No such file or directory: '/home/akiernan/nanohub/build/tmp/work/x86_64-linux/m4-native/1.4.18-r0/deploy-source-date-epoch/__source_date_epoch.txt' -> '/home/akiernan/nanohub/build/tmp/work/x86_64-linux/m4-native/1.4.18-r0/source-date-epoch/__source_date_epoch.txt' ERROR: Logfile of failure stored in: /home/akiernan/nanohub/build/tmp/work/x86_64-linux/m4-native/1.4.18-r0/temp/log.do_deploy_source_date_epoch_setscene.113897 -- Alex Kiernan -- _______________________________________________ Openembedded-core mailing list Openembedded-core@lists.openembedded.org http://lists.openembedded.org/mailman/listinfo/openembedded-core