Re: [OE-core] [PATCH 1/2] image_types: Ensure tar archives are reproducible

2021-02-05 Thread Richard Purdie 
On Fri, 2021-02-05 at 13:14 +0100, Martin Jansa wrote:
> On Sat, Jan 30, 2021 at 7:02 PM Richard Purdie <
> richard.pur...@linuxfoundation.org> wrote:
> > The tar output seems to vary depending on the version of tar used
> > and distro
> > configuration. Be explict about the output format to avoid this and
> > be
> > determinstic.
> > 
> 
> Is it the tar format selection which isn't deterministic or something
> else when the format isn't set to gnu?

Different versions of tar seemed to be picking different output
formats, I therefore picked one. I looked at which output format
reproducible builds were recommending.

> I'm asking because gnu format isn't compatible with --xattrs used
> e.g. here:
>  
> https://github.com/advancedtelematic/meta-updater/blob/d3a832f66e8802cb45536ff278d5c77f946d341d/classes/image_types_ostree.bbclass#L16
> causing do_image_tar failing with:
> 
> | tar: --xattrs can be used only on POSIX archives
> | Try 'tar --help' or 'tar --usage' for more information.
> 
> I've already tested that --format=posix here would fix meta-updater,
> but not sure if it's still as deterministic as --format=gnu. Do you
> remember which distros were producing different output?

I think it was centos7 differing to ubuntu 18.04

Cheers,

Richard


-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#147693): 
https://lists.openembedded.org/g/openembedded-core/message/147693
Mute This Topic: https://lists.openembedded.org/mt/80240566/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [OE-core] [PATCH 1/2] image_types: Ensure tar archives are reproducible

2021-02-05 Thread Martin Jansa 
On Sat, Jan 30, 2021 at 7:02 PM Richard Purdie <
richard.pur...@linuxfoundation.org> wrote:

> The tar output seems to vary depending on the version of tar used and
> distro
> configuration. Be explict about the output format to avoid this and be
> determinstic.
>

Is it the tar format selection which isn't deterministic or something else
when the format isn't set to gnu?

I'm asking because gnu format isn't compatible with --xattrs used e.g. here:
https://github.com/advancedtelematic/meta-updater/blob/d3a832f66e8802cb45536ff278d5c77f946d341d/classes/image_types_ostree.bbclass#L16
causing do_image_tar failing with:

| tar: --xattrs can be used only on POSIX archives
| Try 'tar --help' or 'tar --usage' for more information.

I've already tested that --format=posix here would fix meta-updater, but
not sure if it's still as deterministic as --format=gnu. Do you remember
which distros were producing different output?

Thanks


> Signed-off-by: Richard Purdie 
> ---
>  meta/classes/image_types.bbclass | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
>
> diff --git a/meta/classes/image_types.bbclass
> b/meta/classes/image_types.bbclass
> index c937ae45384..76949d1051d 100644
> --- a/meta/classes/image_types.bbclass
> +++ b/meta/classes/image_types.bbclass
> @@ -110,7 +110,7 @@ IMAGE_CMD_squashfs-lz4 = "mksquashfs ${IMAGE_ROOTFS}
> ${IMGDEPLOYDIR}/${IMAGE_NAM
>
>  IMAGE_CMD_TAR ?= "tar"
>  # ignore return code 1 "file changed as we read it" as other tasks(e.g.
> do_image_wic) may be hardlinking rootfs
> -IMAGE_CMD_tar = "${IMAGE_CMD_TAR} --sort=name --numeric-owner -cf
> ${IMGDEPLOYDIR}/${IMAGE_NAME}${IMAGE_NAME_SUFFIX}.tar -C ${IMAGE_ROOTFS} .
> || [ $? -eq 1 ]"
> +IMAGE_CMD_tar = "${IMAGE_CMD_TAR} --sort=name --format=gnu
> --numeric-owner -cf ${IMGDEPLOYDIR}/${IMAGE_NAME}${IMAGE_NAME_SUFFIX}.tar
> -C ${IMAGE_ROOTFS} . || [ $? -eq 1 ]"
>
>  do_image_cpio[cleandirs] += "${WORKDIR}/cpio_append"
>  IMAGE_CMD_cpio () {
> --
> 2.27.0
>
>
> 
>
>

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#147690): 
https://lists.openembedded.org/g/openembedded-core/message/147690
Mute This Topic: https://lists.openembedded.org/mt/80240566/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

[OE-core] [PATCH 1/2] image_types: Ensure tar archives are reproducible

2021-01-30 Thread Richard Purdie 
The tar output seems to vary depending on the version of tar used and distro
configuration. Be explict about the output format to avoid this and be
determinstic.

Signed-off-by: Richard Purdie 
---
 meta/classes/image_types.bbclass | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/meta/classes/image_types.bbclass b/meta/classes/image_types.bbclass
index c937ae45384..76949d1051d 100644
--- a/meta/classes/image_types.bbclass
+++ b/meta/classes/image_types.bbclass
@@ -110,7 +110,7 @@ IMAGE_CMD_squashfs-lz4 = "mksquashfs ${IMAGE_ROOTFS} 
${IMGDEPLOYDIR}/${IMAGE_NAM
 
 IMAGE_CMD_TAR ?= "tar"
 # ignore return code 1 "file changed as we read it" as other tasks(e.g. 
do_image_wic) may be hardlinking rootfs
-IMAGE_CMD_tar = "${IMAGE_CMD_TAR} --sort=name --numeric-owner -cf 
${IMGDEPLOYDIR}/${IMAGE_NAME}${IMAGE_NAME_SUFFIX}.tar -C ${IMAGE_ROOTFS} . || [ 
$? -eq 1 ]"
+IMAGE_CMD_tar = "${IMAGE_CMD_TAR} --sort=name --format=gnu --numeric-owner -cf 
${IMGDEPLOYDIR}/${IMAGE_NAME}${IMAGE_NAME_SUFFIX}.tar -C ${IMAGE_ROOTFS} . || [ 
$? -eq 1 ]"
 
 do_image_cpio[cleandirs] += "${WORKDIR}/cpio_append"
 IMAGE_CMD_cpio () {
-- 
2.27.0


-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#147480): 
https://lists.openembedded.org/g/openembedded-core/message/147480
Mute This Topic: https://lists.openembedded.org/mt/80240566/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-