Bug#824808: gdal: please make the build reproducible (fileordering)

[Sebastiaan Couwenberg]

Hi Alexis,

On 05/20/2016 11:29 AM, Alexis Bienvenüe wrote:
> Le 20/05/2016 11:18, Sebastiaan Couwenberg a écrit :
>> The second patches seems better because it doesn't rely on external ls
>> which is not a given for the Windows builds.
> 
> OK I see.
> 
>> I don't understand the need for the buildit target, isn't it sufficient
>> to do the wildcard sorting in the $(LIBGDAL) target?
> 
> In some situations some of the target dependencies do not exist before
> the "make target" call. When this arises, as $(wildcard) is expanded
> before building the dependencies, it can miss some files… I was not sure
> of the situation here, so I added a transitional target to be safe.
> 
>> I've modified your patch to do that, and that seems to work too.
>>
>> https://anonscm.debian.org/cgit/pkg-grass/gdal.git/tree/debian/patches/sort-files-in-static-library.patch?id=caf37a5adc5ea187af41b0faca799f45d2364bec
> 
> Thanks!

Thanks for the clarification. I'll include that when forwarding the
patch upstream, we may use the separate target after all if upstream
likes that better too.

We're currently waiting for the vtk6 build on mipsel for the gdal
transition (#823335) to complete. When GDAL 2.1.0 is in testing I'll
upload a new revision to unstable which will include the patch for this
issue.

Kind Regards,

Bas

-- 
 GPG Key ID: 4096R/6750F10AE88D4AF1
Fingerprint: 8182 DE41 7056 408D 6146  50D1 6750 F10A E88D 4AF1

Bug#824808: gdal: please make the build reproducible (fileordering)

[Alexis Bienvenüe]

Le 20/05/2016 11:18, Sebastiaan Couwenberg a écrit :
> The second patches seems better because it doesn't rely on external ls
> which is not a given for the Windows builds.

OK I see.

> I don't understand the need for the buildit target, isn't it sufficient
> to do the wildcard sorting in the $(LIBGDAL) target?

In some situations some of the target dependencies do not exist before
the "make target" call. When this arises, as $(wildcard) is expanded
before building the dependencies, it can miss some files… I was not sure
of the situation here, so I added a transitional target to be safe.

> I've modified your patch to do that, and that seems to work too.
> 
> https://anonscm.debian.org/cgit/pkg-grass/gdal.git/tree/debian/patches/sort-files-in-static-library.patch?id=caf37a5adc5ea187af41b0faca799f45d2364bec

Thanks!

Regards,
Alexis.

Bug#824808: gdal: please make the build reproducible (fileordering)

[Sebastiaan Couwenberg]

Control: tags -1 pending

Hi Alexis,

Thanks for the patches.

On 05/20/2016 01:28 AM, Alexis Bienvenüe wrote:
> Either one of the two attached patches fixes the order files are passed
> to libtool — but I don't know if one of them could be an acceptable
> solution.

The second patches seems better because it doesn't rely on external ls
which is not a given for the Windows builds.

I don't understand the need for the buildit target, isn't it sufficient
to do the wildcard sorting in the $(LIBGDAL) target?

I've modified your patch to do that, and that seems to work too.

https://anonscm.debian.org/cgit/pkg-grass/gdal.git/tree/debian/patches/sort-files-in-static-library.patch?id=caf37a5adc5ea187af41b0faca799f45d2364bec

Kind Regards,

Bas

-- 
 GPG Key ID: 4096R/6750F10AE88D4AF1
Fingerprint: 8182 DE41 7056 408D 6146  50D1 6750 F10A E88D 4AF1

Bug#824808: gdal: please make the build reproducible (fileordering)

[Alexis Bienvenüe]

Source: gdal
Version: 2.1.0+dfsg-2
Severity: wishlist
Tags: patch upstream
User: reproducible-bui...@lists.alioth.debian.org
Usertags: fileordering
X-Debbugs-Cc: reproducible-bui...@lists.alioth.debian.org

Dear Maintainer,

While working on the “reproducible builds” effort [1], we have noticed
that 'gdal' could not be built reproducibly.

Either one of the two attached patches fixes the order files are passed
to libtool — but I don't know if one of them could be an acceptable
solution.
One applied, gdal can be built reproducibly in our current experimental
framework.

Regards,
Alexis Bienvenüe.

[1]: https://wiki.debian.org/ReproducibleBuilds
Description: Sort files
 Sort files passed as arguments to make the build reproducible.
Author: Alexis Bienvenüe 

Index: gdal-2.1.0+dfsg/GNUmakefile
===
--- gdal-2.1.0+dfsg.orig/GNUmakefile
+++ gdal-2.1.0+dfsg/GNUmakefile
@@ -53,7 +53,7 @@ $(GDAL_SLIB): $(GDAL_OBJ) $(GDAL_LIB)
-o $(GDAL_SLIB)
 
 $(LIBGDAL):$(GDAL_OBJ:.o=.lo)
-   $(LD) $(LDFLAGS) $(LIBS) -o $@ $(GDAL_OBJ:.o=.lo) \
+   $(LD) $(LDFLAGS) $(LIBS) -o $@ `LC_ALL=C ls $(GDAL_OBJ:.o=.lo) 
2>/dev/null` \
-rpath $(INST_LIB) \
-no-undefined \
-version-info $(LIBGDAL_CURRENT):$(LIBGDAL_REVISION):$(LIBGDAL_AGE)
Description: Sort files
 Sort files passed as arguments to make the build reproducible.
Author: Alexis Bienvenüe 

Index: gdal-2.1.0+dfsg/GNUmakefile
===
--- gdal-2.1.0+dfsg.orig/GNUmakefile
+++ gdal-2.1.0+dfsg/GNUmakefile
@@ -53,7 +53,11 @@ $(GDAL_SLIB):$(GDAL_OBJ) $(GDAL_LIB)
-o $(GDAL_SLIB)
 
 $(LIBGDAL):$(GDAL_OBJ:.o=.lo)
-   $(LD) $(LDFLAGS) $(LIBS) -o $@ $(GDAL_OBJ:.o=.lo) \
+   $(MAKE) $(LIBGDAL).buildit
+
+.PHONY: $(LIBGDAL).buildit
+$(LIBGDAL).buildit:
+   $(LD) $(LDFLAGS) $(LIBS) -o $(LIBGDAL) $(sort $(wildcard 
$(GDAL_OBJ:.o=.lo))) \
-rpath $(INST_LIB) \
-no-undefined \
-version-info $(LIBGDAL_CURRENT):$(LIBGDAL_REVISION):$(LIBGDAL_AGE)