Re: SD card flashing time
On 9/15/2010 4:51 PM, Martin Langhoff wrote: On Wed, Sep 15, 2010 at 10:14 PM, James Cameronqu...@laptop.org wrote: For interest, I've compared the time it takes to fs-update using OpenFirmware versus a Linux kernel and shell mediated installation method. Why not compared to (linux-based) dd? Because I did not want to write blocks that did not need to be written. Both dd and fs-update write everything. What is interesting about the filesystem result is that it takes longer despite writing far less data; 1.6 GB out of 4 GB. And there was no ext3 journal involved during the write. That'd be more fair I wasn't trying to be fair, I was trying to win. and gives the linux kernel an opportunity to show us whether better IO scheduling helps, without gains being clobbered by FS. Test completed, but fs-update still wins. fs-update takes 00:19:42, and dd takes 00:20:46 without fdatasync, and 00:20:51 with fdatasync. At least these results are close. Here are the results in more detail, again with a USB HDD booted using the OLPC kernel, and a 4 GB microSD: 1. baseline timings 04:48 dd if=q of=/dev/null bs=65536 09:35 dd if=/dev/mmcblk0 of=/dev/null bs=65536 2. reading from internal microSD 09:46 dd if=/dev/mmcblk0 of=q bs=65536 09:58 dd if=/dev/mmcblk0 of=q bs=65536 conv=fdatasync 3. writing to internal microSD 19:42 fs-update 20:46 dd if=q of=/dev/mmcblk0 bs=65536 20:51 dd if=q of=/dev/mmcblk0 bs=65536 conv=fdatasync Experiments were also conducted with iflag=direct, oflag=direct, and oflag=nonblock. No useful results. They all acted to slow it down. -- James Cameron http://quozl.linux.org.au/ ___ Devel mailing list Devel@lists.laptop.org http://lists.laptop.org/listinfo/devel
Re: SD card flashing time
On Thu, Sep 16, 2010 at 2:15 AM, James Cameron qu...@laptop.org wrote: What is interesting about the filesystem result is that it takes longer despite writing far less data; 1.6 GB out of 4 GB. And there was no ext3 journal involved during the write. But probably a lot of fs updates. That'd be more fair I wasn't trying to be fair, I was trying to win. I like that. I am still very curious about resize behaviour. Does resize2fs zero-out the blocks? cheers, m -- martin.langh...@gmail.com mar...@laptop.org -- School Server Architect - ask interesting questions - don't get distracted with shiny stuff - working code first - http://wiki.laptop.org/go/User:Martinlanghoff ___ Devel mailing list Devel@lists.laptop.org http://lists.laptop.org/listinfo/devel
Re: SD card flashing time
On Thu, Sep 16, 2010 at 05:42:03PM -0400, Martin Langhoff wrote: On Thu, Sep 16, 2010 at 2:15 AM, James Cameron qu...@laptop.org wrote: What is interesting about the filesystem result is that it takes longer despite writing far less data; 1.6 GB out of 4 GB. ?And there was no ext3 journal involved during the write. But probably a lot of fs updates. Redundant writes of filesystem metadata, yes. That'd be more fair I wasn't trying to be fair, I was trying to win. I like that. I am still very curious about resize behaviour. Does resize2fs zero-out the blocks? No, certainly not. There's no justification for either mke2fs or resize2fs to write to blocks that do not form part of the filesystem metadata. Why would you want to zero-out blocks anyway? -- James Cameron http://quozl.linux.org.au/ ___ Devel mailing list Devel@lists.laptop.org http://lists.laptop.org/listinfo/devel
Re: SD card flashing time
On Thu, Sep 16, 2010 at 7:01 PM, James Cameron qu...@laptop.org wrote: No, certainly not. There's no justification for either mke2fs or resize2fs to write to blocks that do not form part of the filesystem metadata. Why would you want to zero-out blocks anyway? Thinking of a deployment trying minimize reflash times. What if they have a signed Forth script to run on a USB stick that... ? - runs fs-update u:\myimage.zd -- a small-sized img - boots a signed kernel and signed initramfs from the USB stick -- where the initramfs just executes resize2fs, prints success/failure and waits to be switched off cheers, m -- martin.langh...@gmail.com mar...@laptop.org -- School Server Architect - ask interesting questions - don't get distracted with shiny stuff - working code first - http://wiki.laptop.org/go/User:Martinlanghoff ___ Devel mailing list Devel@lists.laptop.org http://lists.laptop.org/listinfo/devel
Re: SD card flashing time
On 9/16/2010 1:28 PM, Martin Langhoff wrote: On Thu, Sep 16, 2010 at 7:01 PM, James Cameronqu...@laptop.org wrote: No, certainly not. There's no justification for either mke2fs or resize2fs to write to blocks that do not form part of the filesystem metadata. Why would you want to zero-out blocks anyway? Thinking of a deployment trying minimize reflash times. What if they have a signed Forth script to run on a USB stick that... ? - runs fs-update u:\myimage.zd -- a small-sized img - boots a signed kernel and signed initramfs from the USB stick -- where the initramfs just executes resize2fs, prints success/failure and waits to be switched off That would probably work. cheers, m ___ Devel mailing list Devel@lists.laptop.org http://lists.laptop.org/listinfo/devel
Re: SD card flashing time
On Thu, Sep 16, 2010 at 07:28:39PM -0400, Martin Langhoff wrote: On Thu, Sep 16, 2010 at 7:01 PM, James Cameron qu...@laptop.org wrote: No, certainly not. ?There's no justification for either mke2fs or resize2fs to write to blocks that do not form part of the filesystem metadata. ?Why would you want to zero-out blocks anyway? Thinking of a deployment trying minimize reflash times. Make sure to mention NANDblaster. What if they have a signed Forth script to run on a USB stick that... ? - runs fs-update u:\myimage.zd -- a small-sized img - boots a signed kernel and signed initramfs from the USB stick -- where the initramfs just executes resize2fs, prints success/failure and waits to be switched off I agree, this should work. It seems about the same amount of work as building all images small and resizing them up on first boot. -- James Cameron http://quozl.linux.org.au/ ___ Devel mailing list Devel@lists.laptop.org http://lists.laptop.org/listinfo/devel
Re: SD card flashing time
On Thu, Sep 16, 2010 at 7:53 PM, James Cameron qu...@laptop.org wrote: I agree, this should work. It seems about the same amount of work as building all images small and resizing them up on first boot. Indeed. The core idea that is interesting is having the firstboot be a special boot that execs while still in the warehouse, instead of in kids' hands. Lowers risk significantly. Maybe from a separate initrams, or from the standard one (resize2fs and its libs aren't that large). m -- martin.langh...@gmail.com mar...@laptop.org -- School Server Architect - ask interesting questions - don't get distracted with shiny stuff - working code first - http://wiki.laptop.org/go/User:Martinlanghoff ___ Devel mailing list Devel@lists.laptop.org http://lists.laptop.org/listinfo/devel
SD card flashing time
[thread misdirected to an internal list, mea culpa. listed here in chronological order] On Sep 15, 2010, at 8:17 AM, Esteban Arias wrote: Hi, it is possible reduce the time of flashing xo-1.5? Dextrose version, delays 20 mins. thanks, -- Esteban Arias Investigación y Desarrollo - Plan Ceibal On Sep 15, 2010, at 3:41 PM, John Watlington wrote: Possibly. I don't know what SD card you have installed, but changing from a class 2 to a class 6 will greatly increase the speed of reflasing. Otherwise, not really. You are trying to program a 4GB storage device which has an average write speed of 2MB/s. The reason it doesn't actually take 2000 seconds is that the write speed is actually higher than rated. More information on this is available at: http://wiki.laptop.org/go/SD_and_USB_FLASH_Drive_Performance The best approach to speeding up the existing cards is probably to write as small an image as possible, then resize it to fill the entire card. This is not currently done for technical reasons described at: http://dev.laptop.org/ticket/10040 We do believe that this is possible, but never agreed on the most reliable, least intrusive way to implement it. On Sep 15, 2010, at 10:14 PM, James Cameron wrote: For interest, I've compared the time it takes to fs-update using OpenFirmware versus a Linux kernel and shell mediated installation method. The input data was os852.zd, tested on a specific laptop SHC016013D9 SKU99. It took 00:19:42 to fs-update. Using Linux, it took 00:24:17. Conclusion is no improvement over fs-update. Method, for the curious ... - boot Ubuntu from USB HDD, - capture filesystems using tar, mkdir /a mount /dev/mmcblk0p1 /a cd /a tar cf /var/tmp/boot.tar . cd umount /a mount /dev/mmcblk0p2 /a cd /a tar cf /var/tmp/root.tar . cd umount /a rmdir /a - recreate and restore filesystems using tar, time ( mke2fs -q -O dir_index,^resize_inode -L Boot /dev/mmcblk0p1 mount /dev/mmcblk0p1 /mnt tar --extract --file /var/tmp/boot.tar --directory /mnt umount /mnt mke2fs -q -O dir_index,^huge_file -E resize=8G -m1 -L OLPCRoot /dev/mmcblk0p2 mount /dev/mmcblk0p2 /mnt tar --extract --file /var/tmp/root.tar --directory /mnt umount /mnt tune2fs -j -o journal_data_ordered /dev/mmcblk0p2 ) Adding the journal *before* extracting cost another ten minutes total time. On 9/15/2010 4:51 PM, Martin Langhoff wrote: On Wed, Sep 15, 2010 at 10:14 PM, James Cameronqu...@laptop.org wrote: For interest, I've compared the time it takes to fs-update using OpenFirmware versus a Linux kernel and shell mediated installation method. Why not compared to (linux-based) dd? That'd be more fair and gives the linux kernel an opportunity to show us whether better IO scheduling helps, without gains being clobbered by FS. Still, I think OFW is going as fast as possible. Do we have numbers on resizing a small fs? Could we implement this in the initramfs as a special codepath only for firstboot? So with a new OFW: - fs-update runs, completes - if successful, checks for a signed firstboot initramfs - or passes a firstboot init param to the existing initramfs - the work we described before as happening in the initramfs does not change but it is in a more controlled situation (rather than the literal firstboot in the hands of kids) and is not expected to continue the boot process. cheers, m On Sep 15, 2010, at 11:33 PM, Mitch Bradley wrote: The time is heavily dominated by large-block-size sequential writes to the SD card, so there is little opportunity for better I/O scheduling. OFW already overlaps the read from the USB device with SD writing, so there isn't much additional parallelism available for picking. It's possible that writing a small filesystem image, then resizing it, would help, but I'm not expecting a huge gain, because of the way that ext2/3/4 allocates block maps. It's not as simple as just tacking free space onto the end. On Sep 15, 2010, at 11:39 PM, Paul Fox wrote: martin wrote: Why not compared to (linux-based) dd? That'd be more fair and gives the linux kernel an opportunity to show us whether better IO scheduling helps, without gains being clobbered by FS. Still, I think OFW is going as fast as possible. Do we have numbers on resizing a small fs? Could we implement this in the initramfs as a special codepath only for firstboot? So with a new OFW: - fs-update runs, completes - if successful, checks for a signed firstboot initramfs - or passes a firstboot init param to the existing initramfs - the work we described before as happening in the initramfs does not change but it is in a more controlled situation (rather than the literal firstboot in the hands of kids) and is not expected to continue the boot process. all automagically? interesting. we'd have to think through how that changes things for: - deployments
