Bug#759794: [Debian-med-packaging] Bug#759794: insighttoolkit4: FTBFS on amd64 with ENOSPC

[Gert Wollny]

Disk space used for building of  ITK 4.8 with only 2 and 3 dimensions
to wrap: 

chroot base system with all deps installed: 3.0GB 

Source code unpacked:   ~1.0GB 
Build dir 36GB 
Debian dir   5.1GB Sum: 42GB 

Total:  45GB 

The now successful build in the log report 50.72 GB [1], which means we
look at 5-6 GB difference. 

Also of interest: 

The difference between the stripped and unstripped python bindings *.so
files is about 3.4GB (for 2;3 dims wrapped), I guess that a copy of
this stripped info would add to the build process if debug packages are
generated automatically. 


Best, 
Gert




 

Bug#759794: [Debian-med-packaging] Bug#759794: insighttoolkit4: FTBFS on amd64 with ENOSPC

[Gert Wollny]

Hello Steven, 

On Sun, 2015-10-04 at 11:09 -0500, Steve M. Robbins wrote:
> There are two things that could probably help:
> > 
> >  - change the python-binding dimensions to use only 2 an 3. 
> 
> I can definitely make this change for the next upload.  
> Did you ever quantify whether this saves on disk space?

At that time I didn't :-/ I'll see what this does now. I'll get back to
you when it's done. 

> 
> I think in the coming months there will be an archive-wide automatic
> debug package generation [1] .  We can then drop the -dbg package.  I
> haven't checked, but I suspect it will still require building with 
> -g.

Which means that during the package build at least the compressed size
of the python bindings debug info might be added to the build size. 

Best, 
Gert 

Bug#759794: [Debian-med-packaging] Bug#759794: insighttoolkit4: FTBFS on amd64 with ENOSPC

[Steve M. Robbins]

Hello Gert,

I ran into this issue again today which prompted me to re-read this bug log.

On Sat, 30 Aug 2014 16:15:19 +0200 Gert Wollny  wrote:
> > insighttoolkit4 repeatedly FTBFS on amd64 [1] because of ENOSPC. A
> > manual build on porterbox barriere.debian.org reported a need of ~44GB
> > while it failed on buildd barber at approx 37GB of disk space.
> 
> There are two things that could probably help:
> 
>  - change the python-binding dimensions to use only 2 an 3. I enabled
>also 4 because in our lab we have to deal with 3D+t data, but in the
>end using 4D data with python is not such a good idea memory-wise. 
>(On Monday I will check how much this actually saves)

I can definitely make this change for the next upload.  
Did you ever quantify whether this saves on disk space?

>  - build with -g1 or even -g0 (have to look up if this is not against
>Debian Policy).  The python bindings creates a lots of debugging info
>that gets thrown away anyway. We would then also have to disable the
>lib*-dbg package.

I think in the coming months there will be an archive-wide automatic debug 
package generation [1] .  We can then drop the -dbg package.  I haven't 
checked, but I suspect it will still require building with -g.

[1]  https://lists.debian.org/debian-devel/2015/08/msg00443.html

Thanks,
-Steve


signature.asc
Description: This is a digitally signed message part.

Bug#759794: insighttoolkit4: FTBFS on amd64 with ENOSPC

[Gilles Filippini]

Gilles Filippini a écrit , Le 31/08/2014 10:47:
 Kurt Roeckx a écrit , Le 30/08/2014 22:32:
 On Sat, Aug 30, 2014 at 08:10:41PM +0200, Philipp Kern wrote:
 On Sat, Aug 30, 2014 at 03:34:32PM +0200, Gilles Filippini wrote:
 insighttoolkit4 repeatedly FTBFS on amd64 [1] because of ENOSPC. A
 manual build on porterbox barriere.debian.org reported a need of ~44GB
 while it failed on buildd barber at approx 37GB of disk space.

 [1] 
 https://buildd.debian.org/status/logs.php?pkg=insighttoolkit4arch=amd64

 I really don't know how the build space could be optimized. The only
 solutions I can think of right now are:
 * force the build on a buildd with at least 44GB of free space
 * do a source + amd64 binary upload instead of source only upload.

 Note: this is blocking the ongoing hdf5 transition.

 I wonder if we should standardize on 50 GB everywhere. But then at some 
 point
 there needs to be a cut-off. And if the packaging could be optimized to need
 less (i.e. avoid unnecessary disk use), that'd be splendid.

 I actually don't have an amd64 buildd that has both enough RAM and
 disk space.  Brahms is the only one with enough disk space, but it
 only has 2 GB of RAM and gcc gets OOM killed there.

 So if DSA can arange 50 GB of disk space on barber, it would could
 build it there.

 Since it was already build on the porterbox, do you plan to upload
 that?
 
 That's what I intend if there is no solution on the maintainer or buildd
 sides.

Uploaded.
The build actually required ~48GB of disk space (before dh_compress and
dh_strip).

Thanks,

_g.




signature.asc
Description: OpenPGP digital signature

Bug#759794: insighttoolkit4: FTBFS on amd64 with ENOSPC

[Gilles Filippini]

Kurt Roeckx a écrit , Le 30/08/2014 22:32:
 On Sat, Aug 30, 2014 at 08:10:41PM +0200, Philipp Kern wrote:
 On Sat, Aug 30, 2014 at 03:34:32PM +0200, Gilles Filippini wrote:
 insighttoolkit4 repeatedly FTBFS on amd64 [1] because of ENOSPC. A
 manual build on porterbox barriere.debian.org reported a need of ~44GB
 while it failed on buildd barber at approx 37GB of disk space.

 [1] https://buildd.debian.org/status/logs.php?pkg=insighttoolkit4arch=amd64

 I really don't know how the build space could be optimized. The only
 solutions I can think of right now are:
 * force the build on a buildd with at least 44GB of free space
 * do a source + amd64 binary upload instead of source only upload.

 Note: this is blocking the ongoing hdf5 transition.

 I wonder if we should standardize on 50 GB everywhere. But then at some point
 there needs to be a cut-off. And if the packaging could be optimized to need
 less (i.e. avoid unnecessary disk use), that'd be splendid.
 
 I actually don't have an amd64 buildd that has both enough RAM and
 disk space.  Brahms is the only one with enough disk space, but it
 only has 2 GB of RAM and gcc gets OOM killed there.
 
 So if DSA can arange 50 GB of disk space on barber, it would could
 build it there.
 
 Since it was already build on the porterbox, do you plan to upload
 that?

That's what I intend if there is no solution on the maintainer or buildd
sides.

Thanks,

_g.




signature.asc
Description: OpenPGP digital signature

Bug#759794: insighttoolkit4: FTBFS on amd64 with ENOSPC

[Gilles Filippini]

Source: insighttoolkit4
Version: 4.6.0-3
Severity: serious
Justification: FTBFS on amd64

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA256

Hi,

insighttoolkit4 repeatedly FTBFS on amd64 [1] because of ENOSPC. A
manual build on porterbox barriere.debian.org reported a need of ~44GB
while it failed on buildd barber at approx 37GB of disk space.

[1] https://buildd.debian.org/status/logs.php?pkg=insighttoolkit4arch=amd64

I really don't know how the build space could be optimized. The only
solutions I can think of right now are:
* force the build on a buildd with at least 44GB of free space
* do a source + amd64 binary upload instead of source only upload.

Note: this is blocking the ongoing hdf5 transition.

Thanks,

_g.

- -- System Information:
Debian Release: jessie/sid
  APT prefers testing
  APT policy: (500, 'testing'), (1, 'experimental')
Architecture: i386 (i686)

Kernel: Linux 3.14-1-486
Locale: LANG=fr_FR.UTF-8, LC_CTYPE=fr_FR.UTF-8 (charmap=UTF-8)
Shell: /bin/sh linked to /bin/dash

-BEGIN PGP SIGNATURE-
Version: GnuPG v1

iQEcBAEBCAAGBQJUAdLeAAoJEO/obGx//s+DCfQH/jxEPWbWLA9y3zwYneWcqwD3
MG48P5ZB4qhHLQyH6xkmozI9DLcJXiuG385I5E+NAKJvc/pwNd+p0IV0DAuaE27d
CJbS83U2k7T7J0jI8xUfJeT+SU2hLb9O6MSQF7z1UJU1R427hyE7z+qIBmVZ3vPC
qXfyNZZ554O8j87f5J5GL4b88/xU3fvFwPW3c7vm3dBxAB2Pm4LY2FfIXU0s4HBy
gSeEhXPDi0W1SABv44BlIyKDtD5WjdM+422KcNDfmZTqQNuyyVBCbP7ETC3KgfpI
jimDmqMXyFjbJU0Moy1bn0KWhGjLKn4OqKeNlKHtdeu1M61YMyULYf58S97ea0g=
=E5S6
-END PGP SIGNATURE-


-- 
To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org
with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org

Bug#759794: [Debian-med-packaging] Bug#759794: insighttoolkit4: FTBFS on amd64 with ENOSPC

[Gert Wollny]

 insighttoolkit4 repeatedly FTBFS on amd64 [1] because of ENOSPC. A
 manual build on porterbox barriere.debian.org reported a need of ~44GB
 while it failed on buildd barber at approx 37GB of disk space.

There are two things that could probably help:

 - change the python-binding dimensions to use only 2 an 3. I enabled
   also 4 because in our lab we have to deal with 3D+t data, but in the
   end using 4D data with python is not such a good idea memory-wise. 
   (On Monday I will check how much this actually saves)
 - build with -g1 or even -g0 (have to look up if this is not against
   Debian Policy).  The python bindings creates a lots of debugging info
   that gets thrown away anyway. We would then also have to disable the
   lib*-dbg package.


   


-- 
To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org
with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org

Bug#759794: insighttoolkit4: FTBFS on amd64 with ENOSPC

[Philipp Kern]

On Sat, Aug 30, 2014 at 03:34:32PM +0200, Gilles Filippini wrote:
 insighttoolkit4 repeatedly FTBFS on amd64 [1] because of ENOSPC. A
 manual build on porterbox barriere.debian.org reported a need of ~44GB
 while it failed on buildd barber at approx 37GB of disk space.
 
 [1] https://buildd.debian.org/status/logs.php?pkg=insighttoolkit4arch=amd64
 
 I really don't know how the build space could be optimized. The only
 solutions I can think of right now are:
 * force the build on a buildd with at least 44GB of free space
 * do a source + amd64 binary upload instead of source only upload.
 
 Note: this is blocking the ongoing hdf5 transition.

I wonder if we should standardize on 50 GB everywhere. But then at some point
there needs to be a cut-off. And if the packaging could be optimized to need
less (i.e. avoid unnecessary disk use), that'd be splendid.

Kind regards
Philipp Kern


signature.asc
Description: Digital signature

Bug#759794: insighttoolkit4: FTBFS on amd64 with ENOSPC

[Kurt Roeckx]

On Sat, Aug 30, 2014 at 08:10:41PM +0200, Philipp Kern wrote:
 On Sat, Aug 30, 2014 at 03:34:32PM +0200, Gilles Filippini wrote:
  insighttoolkit4 repeatedly FTBFS on amd64 [1] because of ENOSPC. A
  manual build on porterbox barriere.debian.org reported a need of ~44GB
  while it failed on buildd barber at approx 37GB of disk space.
  
  [1] https://buildd.debian.org/status/logs.php?pkg=insighttoolkit4arch=amd64
  
  I really don't know how the build space could be optimized. The only
  solutions I can think of right now are:
  * force the build on a buildd with at least 44GB of free space
  * do a source + amd64 binary upload instead of source only upload.
  
  Note: this is blocking the ongoing hdf5 transition.
 
 I wonder if we should standardize on 50 GB everywhere. But then at some point
 there needs to be a cut-off. And if the packaging could be optimized to need
 less (i.e. avoid unnecessary disk use), that'd be splendid.

I actually don't have an amd64 buildd that has both enough RAM and
disk space.  Brahms is the only one with enough disk space, but it
only has 2 GB of RAM and gcc gets OOM killed there.

So if DSA can arange 50 GB of disk space on barber, it would could
build it there.

Since it was already build on the porterbox, do you plan to upload
that?


Kurt


-- 
To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org
with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org

Bug#759794: insighttoolkit4: FTBFS on amd64 with ENOSPC

[Philipp Kern]

On Sat, Aug 30, 2014 at 10:32:50PM +0200, Kurt Roeckx wrote:
 On Sat, Aug 30, 2014 at 08:10:41PM +0200, Philipp Kern wrote:
  On Sat, Aug 30, 2014 at 03:34:32PM +0200, Gilles Filippini wrote:
   insighttoolkit4 repeatedly FTBFS on amd64 [1] because of ENOSPC. A
   manual build on porterbox barriere.debian.org reported a need of ~44GB
   while it failed on buildd barber at approx 37GB of disk space.
   
   [1] 
   https://buildd.debian.org/status/logs.php?pkg=insighttoolkit4arch=amd64
   
   I really don't know how the build space could be optimized. The only
   solutions I can think of right now are:
   * force the build on a buildd with at least 44GB of free space
   * do a source + amd64 binary upload instead of source only upload.
   
   Note: this is blocking the ongoing hdf5 transition.
  
  I wonder if we should standardize on 50 GB everywhere. But then at some 
  point
  there needs to be a cut-off. And if the packaging could be optimized to need
  less (i.e. avoid unnecessary disk use), that'd be splendid.
 
 I actually don't have an amd64 buildd that has both enough RAM and
 disk space.  Brahms is the only one with enough disk space, but it
 only has 2 GB of RAM and gcc gets OOM killed there.

I'd argue that 2 GB of RAM for a builder is a tad silly these times.

Kind regards
Philipp Kern


signature.asc
Description: Digital signature