Re: Changing default root file system to btrfs
On Fri, Aug 5, 2011 at 1:08 PM, James Tunnicliffe wrote: > > I just tested two a nano images using ext4 and they didn't find the > root file system. So, we are at least gated on that bug. I filed https://bugs.launchpad.net/linaro-ubuntu/+bug/822593 to make ext4 our default. Seems that our kernel configs as well as linaro-image-tools have to be updated. For btrfs I remember we have to ensure that btrfs-tools needed are in all rootfs as well. -- - Alexander ___ linaro-dev mailing list linaro-dev@lists.linaro.org http://lists.linaro.org/mailman/listinfo/linaro-dev
Re: Changing default root file system to btrfs
On 4 August 2011 15:37, David Gilbert wrote: > On 4 August 2011 15:28, James Tunnicliffe > wrote: >> On 4 August 2011 14:56, David Gilbert wrote: >>> On 4 August 2011 14:52, James Tunnicliffe >>> wrote: I have seen poor performance when DDing to a card, which I assume is because dd is not writing large aligned chunks. If we can dd the first meg or so of data onto the card, then write in 4MB chunks that are all 4MB aligned that should be quick (at least, if my reading of my flashbench results is correct). >>> >>> What options to dd are you using? dd if=file of=/dev/thewholedevice >>> bs=4096k >>> should write 4MB chunks out >> >> My understanding is they are 4MB chunks, but not aligned to 4MB boundaries. > > What exactly is your dd line? I haven't used dd in a while for this, so I don't have a dd line as such. I did just try the line you gave and it ran at the highest speed that flashbench showed that my SD card could provide, so thanks for that! Not sure what I did differently last time... no matter, this will speed up testing a lot. I am slightly concerned having searched around about using ext* on flash - it sounds like ext2 and ext3 at least may be good at wearing out SD cards. Using dd is good for avoiding the file attribute and journal updates that would happen during the initial copy of the root file system to an SD card, so if I am using ext3 it will be my preferred way of getting a card ready. Unfortunately a lot of what I have read should have [citation needed] attached to it. One post that at least seemed logical was: http://talk.maemo.org/showthread.php?t=16873 pauljohn32 "In the Fedora linux list, we had a long thread about ext3 on Sd cards. Consensus was that it is bad to do this because it will use up the finite number of writes on the card. They recommend VFAT because it does not make as many separate writes when files are accessed. ext2 is expensive because it tracks so many attributes,especially atime.. ext3 is more costly because of journaling." I have done a bit of benchmarking, which involves copying the files that are in the root file system of a Nano image plus a few larger files over to the same SD card formatted with different file systems: btrfs: 0m55.181s ext3: 3m28.145s ext4: 1m20.663s ext4 no journal: 1m17.868s My guess is that not only does a fast copy keep me happy, it also keeps my SD card happy (faster probably indicates fewer read, modify, write cycles = lasts longer). It looks like we should change our default file system to ext4 so we default to a file system that we can all be confident in and those of us who want to can use btrfs. Disabling the journal may help with SD card longevity, but it doesn't seem to make much difference to speed in this case. But... I just tested two a nano images using ext4 and they didn't find the root file system. So, we are at least gated on that bug. -- James Tunnicliffe ___ linaro-dev mailing list linaro-dev@lists.linaro.org http://lists.linaro.org/mailman/listinfo/linaro-dev
Re: Changing default root file system to btrfs
On 4 August 2011 15:28, James Tunnicliffe wrote: > On 4 August 2011 14:56, David Gilbert wrote: >> On 4 August 2011 14:52, James Tunnicliffe >> wrote: >>> I have seen poor performance when DDing to a card, which I assume is >>> because dd is not writing large aligned chunks. If we can dd the first >>> meg or so of data onto the card, then write in 4MB chunks that are all >>> 4MB aligned that should be quick (at least, if my reading of my >>> flashbench results is correct). >> >> What options to dd are you using? dd if=file of=/dev/thewholedevice bs=4096k >> should write 4MB chunks out > > My understanding is they are 4MB chunks, but not aligned to 4MB boundaries. What exactly is your dd line? Dave ___ linaro-dev mailing list linaro-dev@lists.linaro.org http://lists.linaro.org/mailman/listinfo/linaro-dev
Re: Changing default root file system to btrfs
On 4 August 2011 14:56, David Gilbert wrote: > On 4 August 2011 14:52, James Tunnicliffe > wrote: >> I have seen poor performance when DDing to a card, which I assume is >> because dd is not writing large aligned chunks. If we can dd the first >> meg or so of data onto the card, then write in 4MB chunks that are all >> 4MB aligned that should be quick (at least, if my reading of my >> flashbench results is correct). > > What options to dd are you using? dd if=file of=/dev/thewholedevice bs=4096k > should write 4MB chunks out My understanding is they are 4MB chunks, but not aligned to 4MB boundaries. -- James Tunnicliffe ___ linaro-dev mailing list linaro-dev@lists.linaro.org http://lists.linaro.org/mailman/listinfo/linaro-dev
Re: Changing default root file system to btrfs
On 4 August 2011 14:52, James Tunnicliffe wrote: > I have seen poor performance when DDing to a card, which I assume is > because dd is not writing large aligned chunks. If we can dd the first > meg or so of data onto the card, then write in 4MB chunks that are all > 4MB aligned that should be quick (at least, if my reading of my > flashbench results is correct). What options to dd are you using? dd if=file of=/dev/thewholedevice bs=4096k should write 4MB chunks out Dave ___ linaro-dev mailing list linaro-dev@lists.linaro.org http://lists.linaro.org/mailman/listinfo/linaro-dev
Re: Changing default root file system to btrfs
On 4 August 2011 13:07, Daniel Lezcano wrote: > IMHO it is not stable enough and I am not sure it is worth having such > filesystem as it is mainly used for snapshotting. The last time I played > with it, the FS was quickly corrupted but I don't have to complain > because the kernel configuration help was explicit enough :) > > "Btrfs is highly experimental, and THE DISK FORMAT IS NOT YET FINALIZED. > You should say N here unless you are interested in > testing Btrfs with non-critical data." That description is, as far as I can tell, out of date. https://help.ubuntu.com/community/btrfs#Stability I haven't had any problems with my testing, which includes pulling the power on the board I am using, but I haven't torture tested it. Paul: It has been file system option for Ubuntu since 10.10. I have seen poor performance when DDing to a card, which I assume is because dd is not writing large aligned chunks. If we can dd the first meg or so of data onto the card, then write in 4MB chunks that are all 4MB aligned that should be quick (at least, if my reading of my flashbench results is correct). -- James Tunnicliffe ___ linaro-dev mailing list linaro-dev@lists.linaro.org http://lists.linaro.org/mailman/listinfo/linaro-dev
Re: Changing default root file system to btrfs
> As I understand it, btrfs is considered OK for file systems running on > systems that don't suffer from power failure, so for writing an image > and testing it this should be fine. > > So, what do people think about switching? I too would be considered about filesystem integrity given the number of times I'm forced to yank the power cord on my board. What speed class card are you using though? I tried a low-end one at first and quickly decided the extra US$5 for a class 6 card (San Disk Extreme) paid back the next time I used l-m-c. -dl ___ linaro-dev mailing list linaro-dev@lists.linaro.org http://lists.linaro.org/mailman/listinfo/linaro-dev
Re: Changing default root file system to btrfs
Hi James On Thu, Aug 4, 2011 at 6:46 AM, James Tunnicliffe wrote: > Hi, > > Our current default root file system, ext3, is proving to be a > bottleneck for SD card performance. Not only does it take a long time > to format the partitions, but it also takes a long time to write to. > This slows down creating images on SD cards a lot. I just did a very > simple experiment running linaro-media-create, writing an Ubuntu > Desktop image to an SD card: > > As I understand it, btrfs is considered OK for file systems running on > systems that don't suffer from power failure, so for writing an image > and testing it this should be fine. > > So, what do people think about switching? There's no need to switch. btrfs vs ext* is simply a choice to be made at the time you call linaro-media-create. There's nothing stopping you from using btrfs if that's what you want to do with all the caveats etc that have been stated by others. -- Regards, Tom "We want great men who, when fortune frowns will not be discouraged." - Colonel Henry Knox Linaro.org │ Open source software for ARM SoCs w) tom.gall att linaro.org w) tom_gall att vnet.ibm.com h) tom_gall att mac.com ___ linaro-dev mailing list linaro-dev@lists.linaro.org http://lists.linaro.org/mailman/listinfo/linaro-dev
Re: Changing default root file system to btrfs
On Thu, 4 Aug 2011 12:46:58 +0100 James Tunnicliffe wrote: > Hi, > > Our current default root file system, ext3, is proving to be a > bottleneck for SD card performance. Not only does it take a long time > to format the partitions, but it also takes a long time to write to. > This slows down creating images on SD cards a lot. I just did a very > simple experiment running linaro-media-create, writing an Ubuntu > Desktop image to an SD card: > > ext3: > 139.85user 35.27system 44:03.58elapsed 6%CPU (0avgtext+0avgdata > 107360maxresident)k > 2876115inputs+7048200outputs (958major+1677659minor)pagefaults 0swaps > > btrfs: > 146.52user 34.48system 19:57.16elapsed 15%CPU (0avgtext+0avgdata > 107408maxresident)k > 4417521inputs+6542992outputs (138major+1779874minor)pagefaults 0swaps > > As I understand it, btrfs is considered OK for file systems running on > systems that don't suffer from power failure, so for writing an image > and testing it this should be fine. > > So, what do people think about switching? There were few concerns already expressed, so I just add: is btrfs supported out of the box by kernels in last ~3 Ubuntu releases? Because if one can't look at/change contents produced on an SD card, that undermines purpose of evaluation builds much. Those ext3 vs btrfs results are very vivid though, but I wonder if it makes sense to try to tweak ext3 & mount options. And after all, we could prepare partition image(s) on the local HDD and dd them to a card... -- Best Regards, Paul Linaro.org | Open source software for ARM SoCs Follow Linaro: http://www.facebook.com/pages/Linaro http://twitter.com/#!/linaroorg - http://www.linaro.org/linaro-blog ___ linaro-dev mailing list linaro-dev@lists.linaro.org http://lists.linaro.org/mailman/listinfo/linaro-dev
Re: Changing default root file system to btrfs
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 08/04/2011 01:46 PM, James Tunnicliffe wrote: > Hi, > > Our current default root file system, ext3, is proving to be a > bottleneck for SD card performance. Not only does it take a long time > to format the partitions, but it also takes a long time to write to. > This slows down creating images on SD cards a lot. I just did a very > simple experiment running linaro-media-create, writing an Ubuntu > Desktop image to an SD card: > > ext3: > 139.85user 35.27system 44:03.58elapsed 6%CPU (0avgtext+0avgdata > 107360maxresident)k > 2876115inputs+7048200outputs (958major+1677659minor)pagefaults 0swaps > > btrfs: > 146.52user 34.48system 19:57.16elapsed 15%CPU (0avgtext+0avgdata > 107408maxresident)k > 4417521inputs+6542992outputs (138major+1779874minor)pagefaults 0swaps > > As I understand it, btrfs is considered OK for file systems running on > systems that don't suffer from power failure, so for writing an image > and testing it this should be fine. > > So, what do people think about switching? IMHO it is not stable enough and I am not sure it is worth having such filesystem as it is mainly used for snapshotting. The last time I played with it, the FS was quickly corrupted but I don't have to complain because the kernel configuration help was explicit enough :) "Btrfs is highly experimental, and THE DISK FORMAT IS NOT YET FINALIZED. You should say N here unless you are interested in testing Btrfs with non-critical data." -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.11 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iQEcBAEBAgAGBQJOOot+AAoJEAKBbMCpUGYAYB4H/13CMOSsPFoxypWFcV9UfqzE Ni7WTmiiWDRWlcZVUwGIInwJVs/DIb5liVqT+qn7zlQ0kp0++lTFk3VEl28DoKej q1yvWwews9t8RddFYQrgb4LSzEJuQzDeTZyT1tIexSNihJMLa+nUvhsEwwrPlFCn hPF5pBeFze2uXOrvJ0e/tSlBQ7vIdWoXnz4CZJRH13HvaqHjD0c7qq1/hNQv2mWi qel49BuDI4uucODuEKOw3eof1YLAh/l6F29sqpBKzRWEmRMBgpnGpAbzjawMtpRg 4tyGmFOX4A3zSODMr2c2+PFEZiovzJUaKV0vPew+8GoIyE/cOUzx1amJxvMMMDw= =5D2R -END PGP SIGNATURE- ___ linaro-dev mailing list linaro-dev@lists.linaro.org http://lists.linaro.org/mailman/listinfo/linaro-dev
Re: Changing default root file system to btrfs
On 4 August 2011 12:46, James Tunnicliffe wrote: > Hi, > > Our current default root file system, ext3, is proving to be a > bottleneck for SD card performance. Not only does it take a long time > to format the partitions, but it also takes a long time to write to. > This slows down creating images on SD cards a lot. I just did a very > simple experiment running linaro-media-create, writing an Ubuntu > Desktop image to an SD card: > > ext3: > 139.85user 35.27system 44:03.58elapsed 6%CPU (0avgtext+0avgdata > 107360maxresident)k > 2876115inputs+7048200outputs (958major+1677659minor)pagefaults 0swaps > > btrfs: > 146.52user 34.48system 19:57.16elapsed 15%CPU (0avgtext+0avgdata > 107408maxresident)k > 4417521inputs+6542992outputs (138major+1779874minor)pagefaults 0swaps What's dpkg like these days on btrfs - it used to be awfully slow. Some comparisons from the installed running system would really be needed. > As I understand it, btrfs is considered OK for file systems running on > systems that don't suffer from power failure, so for writing an image > and testing it this should be fine. It doesn't sound like the description of development systems though where people will crash them and reset them and generally pull the plug on them. > So, what do people think about switching? To be honest I trust ext* (it's never lost me a byte in many years of use) and btr feels a bit new to me; but that's a personal feeling. Dave ___ linaro-dev mailing list linaro-dev@lists.linaro.org http://lists.linaro.org/mailman/listinfo/linaro-dev
Changing default root file system to btrfs
Hi, Our current default root file system, ext3, is proving to be a bottleneck for SD card performance. Not only does it take a long time to format the partitions, but it also takes a long time to write to. This slows down creating images on SD cards a lot. I just did a very simple experiment running linaro-media-create, writing an Ubuntu Desktop image to an SD card: ext3: 139.85user 35.27system 44:03.58elapsed 6%CPU (0avgtext+0avgdata 107360maxresident)k 2876115inputs+7048200outputs (958major+1677659minor)pagefaults 0swaps btrfs: 146.52user 34.48system 19:57.16elapsed 15%CPU (0avgtext+0avgdata 107408maxresident)k 4417521inputs+6542992outputs (138major+1779874minor)pagefaults 0swaps As I understand it, btrfs is considered OK for file systems running on systems that don't suffer from power failure, so for writing an image and testing it this should be fine. So, what do people think about switching? -- James Tunnicliffe ___ linaro-dev mailing list linaro-dev@lists.linaro.org http://lists.linaro.org/mailman/listinfo/linaro-dev
