Bug#864260: Missing kernel in debian-installer sd-card images for armhf (stretch)

[Cyril Brulebois]

Karsten Merker  (2017-06-07):
> I have done test builds with both 50MB and 100MB raw image size. 
> The difference in the resulting compressed binary size is less
> than 60kBytes, so that shouldn't pose a problem even for people
> on very-low-bandwidth internet connections.  I suppose the
> additional write time to the target medium also isn't much of a
> problem either.  On modern SD cards, linear write rates are nearly
> always larger than 5MB per second (even cheap SD cards nowadays
> commonly reach nearly 10MB/s on linear writes), so setting the
> image size to 100MB costs at most 10 seconds of extra write time. 
> That shouldn't be a problem even for CI systems that do lots of
> image writes.
> 
> If the size increase eases debugging, I think it is worth that
> cost, so I'll commit a change to ~100MB to d-i git.

Oh, I failed to notice your mail when I wrote the one I've just sent.
Since you're fine with 100, let's go for it; I just needed a fix to be
committed before the upload (which should happen in a few hours), that's
why I was going for a temporary decision. All the better if you're fine
with the suggested new value. :)

Thanks!


KiBi.


signature.asc
Description: Digital signature

Bug#864260: Missing kernel in debian-installer sd-card images for armhf (stretch)

[Cyril Brulebois]

Cyril Brulebois  (2017-06-07):
> I'm fine with whatever value is determined by consensus among arm
> people.
> 
> Currently testing a patch to detect full disk (it's been in our build
> logs for months…).

As mentioned a couple hours ago, patches were pushed for that, and I've
also verified that the 40k to 50k bump is sufficient. We can experiment
with a higher value (like the proposed 100k) in unstable, and maybe
backport the bump in a stretch point release, but I'll go for a minimal
bump that is sufficient for all builds for the time being.

Thanks everyone for your collective input. Feel free to discuss the
merits of a bump to a given specific value in a bug report against
src:debian-installer, for further action.


KiBi.


signature.asc
Description: Digital signature

Bug#864260: Missing kernel in debian-installer sd-card images for armhf (stretch)

[Cyril Brulebois]

Karsten Merker  (2017-06-06):
> I guess I have found the problem: the images have a fixed size
> (~40MB uncompressed) and the netboot build has grown a few kBytes
> larger than that while the hd-media build still fits (although
> tightly). Increasing the image size should solve the problem.

I've added a patch to detect such issues and bail out when they happen,
tested in asachi's armhf sid chroot:
  https://anonscm.debian.org/cgit/d-i/debian-installer.git/commit/?id=3313d94

[ It's a little sad to have missed the “Disk full” message in the logs
for months. ;) ]

As far as I can tell, only this needed an update:
  https://anonscm.debian.org/cgit/d-i/debian-installer.git/commit/?id=199fe0b


And I've also checked that bumping from 40k to 50k is sufficient to have
a successful build of src:debian-installer (in release build mode: with
unstable in debian/changelog, so building against udebs from testing).

I'll leave you or vagrant push an updated value once you agree on
something.


KiBi.


signature.asc
Description: Digital signature

Bug#864260: Missing kernel in debian-installer sd-card images for armhf (stretch)

[Cyril Brulebois]

Vagrant Cascadian  (2017-06-06):
> I'd like to gently push for making it even larger, say 100MB. when
> debugging different kernel versions, it's often helpful to be able to
> append all modules to the initrd.gz, which can result in ~30MB+ initrd
> images, which usually means an extra step to copy all the files onto
> another partition, rather than copying them in-place...
> 
> There are very few (micro)SD cards or USB sticks that can't handle that
> size, though I guess it would needlessly increase write times for the
> images to write the extra empty space...

I'm fine with whatever value is determined by consensus among arm people.

Currently testing a patch to detect full disk (it's been in our build
logs for months…).


KiBi.


signature.asc
Description: Digital signature

Bug#864260: Missing kernel in debian-installer sd-card images for armhf (stretch)

[Vagrant Cascadian]

On 2017-06-06, Karsten Merker wrote:
> On Tue, Jun 06, 2017 at 10:47:55PM +0200, Karsten Merker wrote:

> I guess I have found the problem: the images have a fixed size
> (~40MB uncompressed) and the netboot build has grown a few kBytes
> larger than that while the hd-media build still fits (although
> tightly). Increasing the image size should solve the problem.

I'd like to gently push for making it even larger, say 100MB. when
debugging different kernel versions, it's often helpful to be able to
append all modules to the initrd.gz, which can result in ~30MB+ initrd
images, which usually means an extra step to copy all the files onto
another partition, rather than copying them in-place...

There are very few (micro)SD cards or USB sticks that can't handle that
size, though I guess it would needlessly increase write times for the
images to write the extra empty space...


live well,
  vagrant


signature.asc
Description: PGP signature

Bug#864260: Missing kernel in debian-installer sd-card images for armhf (stretch)

[Cyril Brulebois]

Hi,

Karsten Merker  (2017-06-06):
> I guess I have found the problem: the images have a fixed size (~40MB
> uncompressed) and the netboot build has grown a few kBytes larger than
> that while the hd-media build still fits (although tightly).
> Increasing the image size should solve the problem.

Were there any reasons for the 40MB size or was it deemed sufficient at
some point and never increased?

> Attached is a corresponding patch. A test build is running but will
> need some time.  I'll check the results when it's ready and commit the
> change to the d-i git if everything looks good.

Thanks!

Even if that fixes the issue this time, it would be great to detect when
the media has an insufficient size, so as to break the build properly
instead of pretending everything is OK.

The immediate fix, once tested, is more than welcome for r0. The ENOSPC
detection can wait for a point release, I don't think there's any hurry
there (unless it's feared we might have other images overflowing with
the new value).

I might try to spend some time on this, but can't promise anything at
this point.


KiBi.


signature.asc
Description: Digital signature

Bug#864260: Missing kernel in debian-installer sd-card images for armhf (stretch)

[Diego Roversi]

On Tue, 6 Jun 2017 02:24:41 +0200
Cyril Brulebois  wrote:

> 
> I'm not familiar with those images, but they should contain u-boot
> configuration and needed bits. From the log above, it looks like a
> vmlinuz was found, but initrd.gz couldn't be read. Cc'ing debian-arm@
> since I don't know much more about this.
> 

My mistake, sorry. I copy a log from a daily image instead of current 
image. The right output is:

U-Boot SPL 2016.11+dfsg1-4 (Mar 27 2017 - 18:39:51)  
DRAM: 512 MiB
CPU: 91200Hz, AXI/AHB/APB: 3/2/2 
Trying to boot from MMC1 
 
 
U-Boot 2016.11+dfsg1-4 (Mar 27 2017 - 18:39:51 +) Allwinner Technology
 
CPU:   Allwinner A10 (SUN4I) 
Model: Olimex A10-OLinuXino-LIME 
I2C:   ready 
DRAM:  512 MiB   
MMC:   SUNXI SD/MMC: 0   
*** Warning - bad CRC, using default environment
 
In:serial
Out:   serial
Err:   serial
SCSI:  SATA link 0 timeout.  
AHCI 0001.0100 32 slots 1 ports 3 Gbps 0x1 impl SATA mode
flags: ncq stag pm led clo only pmp pio slum part ccc apst   
Net:   eth0: ethernet@01c0b000   
starting USB...  
USB0:   USB EHCI 1.00
USB1:   USB OHCI 1.0 
USB2:   USB EHCI 1.00
USB3:   USB OHCI 1.0 
scanning bus 0 for devices... 1 USB Device(s) found  
scanning bus 2 for devices... 1 USB Device(s) found  
Hit any key to stop autoboot:  0 
switch to partitions #0, OK
mmc0 is current device
Scanning mmc 0:1...
Found U-Boot script /boot.scr
reading /boot.scr
1575 bytes read in 26 ms (58.6 KiB/s)
## Executing script at 4310
Mainline u-boot / new-style environment detected.
reading vmlinuz
** Unable to read file vmlinuz **
SCRIPT FAILED: continuing...

PS: The daily image, indeed had the vmlinuz, but not the initrd.img.

Thanks in advance.

-- 
Diego Roversi 

Bug#864260: Missing kernel in debian-installer sd-card images for armhf (stretch)

[Cyril Brulebois]

Hi,

Diego Roversi  (2017-06-05):
> Package: debian-installer
> Version: 20170525
> 
> The sd-card images of debian-installer for arm-hf don't have a kernel
> image inside, and they can't boot from sd.
> 
> To reproduce the problem:
> 
> #wget 
> http://ftp.uk.debian.org/debian/dists/stretch/main/installer-armhf/current/images/netboot/SD-card-images/firmware.A10-OLinuXino-Lime.img.gz
> 
> #wget 
> http://ftp.uk.debian.org/debian/dists/stretch/main/installer-armhf/current/images/netboot/SD-card-images/partition.img.gz
> 
> #zcat firmware.A10-OLinuXino-Lime.img.gz partition.img.gz > image_a10.img
> 
> # ls -lrt
> total 63628
> -rw-r--r-- 1 root root 23891122 May 25 12:21 partition.img.gz
> -rw-r--r-- 1 root root   224112 May 25 12:21 
> firmware.A10-OLinuXino-Lime.img.gz
> -rw-r--r-- 1 root root 4096 Jun  4 21:57 image_a10.img
> 
> # losetup /dev/loop0 image_a10.img 
> # fdisk -lu /dev/loop0
> Disk /dev/loop0: 39.1 MiB, 4096 bytes, 8 sectors
> Units: sectors of 1 * 512 = 512 bytes
> Sector size (logical/physical): 512 bytes / 512 bytes
> I/O size (minimum/optimal): 512 bytes / 512 bytes
> Disklabel type: dos
> Disk identifier: 0x0eded707
> 
> Device   Boot Start   End Sectors  Size Id Type
> /dev/loop0p1 * 2048 7   77952 38.1M  c W95 FAT32 (LBA)
> 
> # partprobe /dev/loop0
> # mount /dev/loop0p1 /mnt/card
> # ls -l /mnt/card/
> total 20732
> -rwxr-xr-x 1 root root 1575 May 25 10:50 boot.scr
> drwxr-xr-x 2 root root41984 May 25 10:50 dtbs
> -rwxr-xr-x 1 root root 21185369 May 25 10:50 initrd.gz
> 
> The boot from this sd image give this error:
> 
> Scanning mmc 0:1...   
> Found U-Boot script /boot.scr 
> reading /boot.scr 
> 1575 bytes read in 25 ms (61.5 KiB/s) 
> ## Executing script at 4310   
> Mainline u-boot / new-style environment detected. 
> reading vmlinuz   
> 3701008 bytes read in 391 ms (9 MiB/s)
> reading dtbs/sun4i-a10-olinuxino-lime.dtb 
> 27941 bytes read in 621 ms (43 KiB/s) 
> reading initrd.gz 
> ** Unable to read file initrd.gz **   
> SCRIPT FAILED: continuing...

I'm not familiar with those images, but they should contain u-boot
configuration and needed bits. From the log above, it looks like a
vmlinuz was found, but initrd.gz couldn't be read. Cc'ing debian-arm@
since I don't know much more about this.



KiBi.


signature.asc
Description: Digital signature

Bug#864260: Missing kernel in debian-installer sd-card images for armhf (stretch)

[Diego Roversi]

Package: debian-installer
Version: 20170525

The sd-card images of debian-installer for arm-hf don't have a kernel image 
inside, and they can't boot from sd.

To reproduce the problem:

#wget 
http://ftp.uk.debian.org/debian/dists/stretch/main/installer-armhf/current/images/netboot/SD-card-images/firmware.A10-OLinuXino-Lime.img.gz

#wget 
http://ftp.uk.debian.org/debian/dists/stretch/main/installer-armhf/current/images/netboot/SD-card-images/partition.img.gz

#zcat firmware.A10-OLinuXino-Lime.img.gz partition.img.gz > image_a10.img

# ls -lrt
total 63628
-rw-r--r-- 1 root root 23891122 May 25 12:21 partition.img.gz
-rw-r--r-- 1 root root   224112 May 25 12:21 firmware.A10-OLinuXino-Lime.img.gz
-rw-r--r-- 1 root root 4096 Jun  4 21:57 image_a10.img

# losetup /dev/loop0 image_a10.img 
# fdisk -lu /dev/loop0
Disk /dev/loop0: 39.1 MiB, 4096 bytes, 8 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: dos
Disk identifier: 0x0eded707

Device   Boot Start   End Sectors  Size Id Type
/dev/loop0p1 * 2048 7   77952 38.1M  c W95 FAT32 (LBA)

# partprobe /dev/loop0
# mount /dev/loop0p1 /mnt/card
# ls -l /mnt/card/
total 20732
-rwxr-xr-x 1 root root 1575 May 25 10:50 boot.scr
drwxr-xr-x 2 root root41984 May 25 10:50 dtbs
-rwxr-xr-x 1 root root 21185369 May 25 10:50 initrd.gz

The boot from this sd image give this error:

Scanning mmc 0:1...   
Found U-Boot script /boot.scr 
reading /boot.scr 
1575 bytes read in 25 ms (61.5 KiB/s) 
## Executing script at 4310   
Mainline u-boot / new-style environment detected. 
reading vmlinuz   
3701008 bytes read in 391 ms (9 MiB/s)
reading dtbs/sun4i-a10-olinuxino-lime.dtb 
27941 bytes read in 621 ms (43 KiB/s) 
reading initrd.gz 
** Unable to read file initrd.gz **   
SCRIPT FAILED: continuing...

I've also tried to download the image from 
http://d-i.debian.org/daily-images/armhf/daily/, but initrd.img is missing 
instead of vmlinuz.

-- 
Diego Roversi