Re: A Question about Journalling File Systems and Flash Drives
On Sb, 27 aug 11, 19:39:33, Rob Owens wrote: I recall reading that ext4 is much quicker than ext3 on flash drives, so you may want to consider using that. Although I'm not sure what you have to do to get Lenny to support ext4. Done that: the minimum would be backported kernel + e2fsprogs. If you use tools that deal with filesystems directly you need to check their compatibility as well. Kind regards, Andrei -- Offtopic discussions among Debian users and developers: http://lists.alioth.debian.org/mailman/listinfo/d-community-offtopic signature.asc Description: Digital signature
Re: A Question about Journalling File Systems and Flash Drives
Scott Ferguson wrote at 2011-08-27 08:50 -0500: I don't imagine upgrading will make much difference - the controller distributes the writes evenly, for which reason I reserve 25% of space when installing (instead of the default 10%). NOTE: I could be wrong about that - I'm just guessing. Hmm, that reserves 25% for root use only, something that occurs above the filesystem level. So probably you are giving up 25% of the space and gaining nothing. And the default is 5% according to mke2fs(8). signature.asc Description: Digital signature
Re: A Question about Journalling File Systems and Flash Drives
On 29/08/11 00:03, green wrote: Scott Ferguson wrote at 2011-08-27 08:50 -0500: I don't imagine upgrading will make much difference - the controller distributes the writes evenly, for which reason I reserve 25% of space when installing (instead of the default 10%). NOTE: I could be wrong about that - I'm just guessing. Hmm, that reserves 25% for root use only, something that occurs above the filesystem level. Is that a specific area that's reserved - or just an amount of space kept free for root? So probably you are giving up 25% of the space and gaining nothing. In this instance the install used is less than 300MB, and it's unusual for me to have more than 1GB on the stick - so I'm not giving up anything. The stick is used to move my configs and an OS, not to ferry data - I use the tubes for that. And the default is 5% according to mke2fs(8). That would be a guessing I warned of. ;-p Currently I use ext3 which has (major) fragmentation problems when it runs short on space (90-95% of space) - especially when used for lots of frequent, small writes as it is. My concern was/is that fragmentation would shorten the lifespan - another reason to move to ext4. NOTE: when I first created the sticks I was expecting 6 month lifetimes based on the pessimistic estimates people were making at the time. It was only later I realised those estimates didn't allow for how the controllers distribute writes. Cheers -- You ever noticed how people who believe in Creationism look really unevolved? You ever noticed that? Eyes real close together, eyebrow ridges, big furry hands and feet. I believe God created me in one day Yeah, looks liked He rushed it. — Bill Hicks -- To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/4e5a5c64.6080...@gmail.com
Re: A Question about Journalling File Systems and Flash Drives
Scott Ferguson wrote at 2011-08-28 10:19 -0500: On 29/08/11 00:03, green wrote: Scott Ferguson wrote at 2011-08-27 08:50 -0500: I don't imagine upgrading will make much difference - the controller distributes the writes evenly, for which reason I reserve 25% of space when installing (instead of the default 10%). NOTE: I could be wrong about that - I'm just guessing. Hmm, that reserves 25% for root use only, something that occurs above the filesystem level. Is that a specific area that's reserved - or just an amount of space kept free for root? mke2fs(8): percentage of the filesystem blocks I read that as just an amount of space reserved so that if a regular user fills up the filesystem, he is denied further writes while root processes can still write. Currently I use ext3 which has (major) fragmentation problems when it runs short on space (90-95% of space) - especially when used for lots of frequent, small writes as it is. mke2fs(8) does mention using a reserved-blocks-percentage to reduce fragmentation. But it only limits user processes. signature.asc Description: Digital signature
Re: A Question about Journalling File Systems and Flash Drives
Brad Alexander wrote at 2011-08-26 16:49 -0500: card slot on his brand new win7 laptop...and grabbed her card and Windows partially formatted (corrupted) it. Since that 8GB card was broken, she upgraded to a 16GB card Can Windows really *break* a card (rather than just the filesystem on it)? signature.asc Description: Digital signature
Re: A Question about Journalling File Systems and Flash Drives
On 29/08/11 07:00, green wrote: Scott Ferguson wrote at 2011-08-28 10:19 -0500: On 29/08/11 00:03, green wrote: Scott Ferguson wrote at 2011-08-27 08:50 -0500: I don't imagine upgrading will make much difference - the controller distributes the writes evenly, for which reason I reserve 25% of space when installing (instead of the default 10%). NOTE: I could be wrong about that - I'm just guessing. Hmm, that reserves 25% for root use only, something that occurs above the filesystem level. Is that a specific area that's reserved - or just an amount of space kept free for root? mke2fs(8): percentage of the filesystem blocks I read that as just an amount of space reserved so that if a regular user fills up the filesystem, he is denied further writes while root processes can still write. Which was roughly my interpretation - a certain amount of shuffle space is left which isn't used by user processes, but can at any given time be any areas of the filesystem. If I get a chance I'll dig up the paper written earlier this year about SSD write controller, from foggy memory it was a local research effort - upset the forensics experts at the time, and goes into detail about how the controllers actually present the system.(which is completely different to how conventional physical drives do things). Sorry, brain not working today, very late night battling cPanel limitations and SSL problems. Currently I use ext3 which has (major) fragmentation problems when it runs short on space (90-95% of space) - especially when used for lots of frequent, small writes as it is. mke2fs(8) does mention using a reserved-blocks-percentage to reduce fragmentation. But it only limits user processes. From my original notes - I based that on a post by Theodore Ted Ts'o, (ext3 redhat list?), end of Jan '09 (I've got the original post somewhere... damn Strigi!) - it's possible he didn't put everything into the man. At least I now know how long the stick in question has seen daily use. Cheers -- I've got a bathtub and an imagination, I'm staying indoors this summer. That way I can listen to music that I like. — Bill Hicks -- To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/4e5af173.7070...@gmail.com
Re: A Question about Journalling File Systems and Flash Drives
On 29/08/11 08:49, green wrote: Brad Alexander wrote at 2011-08-26 16:49 -0500: card slot on his brand new win7 laptop...and grabbed her card and Windows partially formatted (corrupted) it. Since that 8GB card was broken, she upgraded to a 16GB card Can Windows really *break* a card (rather than just the filesystem on it)? It 'might' be an urban myth - I've heard it often but most instances I came across were recoverable (parted), some where not. Disclaimer: my experience of that problem is very limited - maybe half a dozen instances. I suspect the problem stems from autoformat, and a problem with Sdbus.sys during ADMA tranfers - MS pushed out some updates in late '09 that seemed to fix it. NOTE: It only seemed to occur with un-updated laptops - so I'm not certain that heat didn't play a part. Cheers -- I've got a bathtub and an imagination, I'm staying indoors this summer. That way I can listen to music that I like. — Bill Hicks -- To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/4e5af871.3090...@gmail.com
Re: A Question about Journalling File Systems and Flash Drives
On Sat 27 Aug 2011 at 11:19:20 +1000, Scott Ferguson wrote: Based on my experiences using ext3 as a file system for running Debian on USB sticks. Single partition (no swap), logging redirected to vt12. 2 identical sticks (major brand) - identical builds - noatime enabled on one, not on the other, both got roughly the same amount of use. The one without noatime died earlier this year after approx 2 years of use - the other has been upgraded to Squeeze and still works fine. I have just put unstable with an ext4 filesystem on a USB stick and find your experience reassuring. Although there will be a backup stick it is useful to not have to anticipate the drive becoming defunct in a month or two. In what way upgrading will affect its lifespan I do not know but it will be interesting to find out! Hardly empirical evidence but... I bought a larger USB stick (cheap and nasty) early this year - installed Debian onto it, forgot to enable noatime - it died last week. I'm not certain of the answer - but would strongly suggest enabling noatime on the flash drive - and moving /tmp and /var to a non-flash drive. Thanks for the reminder about atime. I had completely forgotton to enable it at install time or afterwards. The OP may be interested in http://www.linuxfoundation.org/news-media/blogs/browse/2009/03/ssd’s-journaling-and-noatimerelatime -- To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/20110827123716.GE4474@desktop
Re: A Question about Journalling File Systems and Flash Drives
On 27/08/11 22:37, Brian wrote: On Sat 27 Aug 2011 at 11:19:20 +1000, Scott Ferguson wrote: snipped The one without noatime died earlier this year after approx 2 years of use - the other has been upgraded to Squeeze and still works fine. I have just put unstable with an ext4 filesystem on a USB stick and find your experience reassuring. Although there will be a backup stick it is useful to not have to anticipate the drive becoming defunct in a month or two. In what way upgrading will affect its lifespan I do not know but it will be interesting to find out! I don't imagine upgrading will make much difference - the controller distributes the writes evenly, for which reason I reserve 25% of space when installing (instead of the default 10%). NOTE: I could be wrong about that - I'm just guessing. Full backups:- dd if=/dev/deb_usb | gzip -1 -c ./deb_usb.img.gz Full restores:- zcat ./deb_usb.img.gz | dd of=/dev/deb_usb rsync on a daily basis. To redirect logging to vt12 add:- *.* -/dev/tty12 to the end of /etc/rsyslog.conf dmesg will bitch but still function. snipped I'm not certain of the answer - but would strongly suggest enabling noatime on the flash drive - and moving /tmp and /var to a non-flash drive. Thanks for the reminder about atime. I had completely forgotton to enable it at install time or afterwards. I originally used it to see if it truly improved performance (it does, a little) - increased lifetime was an unintended side effect. The OP may be interested in http://www.linuxfoundation.org/news-media/blogs/browse/2009/03/ssd’s-journaling-and-noatimerelatime Nice article. When my P2V tools support ext4 fully I'm moving all my hdds to it. I've tested it with one server, and VirtualBox supports it with host I/O caching enabled - no problems so far. Cheers -- When two or more people agree on an issue, I form on the other side. — Bill Hicks -- To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/4e58f631.9070...@gmail.com
Re: A Question about Journalling File Systems and Flash Drives
Scott Ferguson prettyfly.producti...@gmail.com writes: […] Full backups:- dd if=/dev/deb_usb | gzip -1 -c ./deb_usb.img.gz Full restores:- zcat ./deb_usb.img.gz | dd of=/dev/deb_usb My e2dis suite, which I hopeful to release soon, will probably be a better fit for such image-level backups. Namely, it'd allow one to identify the free blocks on an Ext2+ FS and only copy the rest. https://gitorious.org/e2dis rsync on a daily basis. I'd second that using rsync(1) instead of cp(1) when copying to flash media is a good idea, as it reduces both wear and the time necessary to make a copy. […] -- FSF associate member #7257 Coming soon: Software Freedom Day http://mail.sf-day.org/lists/listinfo/ planning-ru (ru), sfd-discuss (en) -- To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/86r546izz6@gray.siamics.net
Re: A Question about Journalling File Systems and Flash Drives
On Fri, Aug 26, 2011 at 11:25:31AM -0500, Martin McCormick wrote: How hard is the ext3 file system on present-day flash drives? I have some older Dell systems that run lenny and I put a flash drive on as the boot drive on one of those systems and it works great, but for how long? I've been using Debian Live for quite some time. I used to use ext2 for speed reasons. Now I use ext4. There was a brief period where I used ext3. I've never had a usb stick die on me due to overuse. I recall reading that ext4 is much quicker than ext3 on flash drives, so you may want to consider using that. Although I'm not sure what you have to do to get Lenny to support ext4. -Rob -- To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/20110827233933.ga13...@aurora.owens.net
Re: A Question about Journalling File Systems and Flash Drives
Martin McCormick wrote at 2011-08-26 11:25 -0500: How hard is the ext3 file system on present-day flash drives? Not much worse than without a journal I think, but What got me to thinking was that I have a system using conventional magnetic-based hard drives and ext3 file systems. The second hard drive is not used as often and I noticed that the system shuts it down to rest until one calls for a file off the secondary drive. If the journal for all drives is on the boot drive, then that explains everything. Normally, the journal for a filesystem is stored in the same partition as the filesystem. If not, one would expect the secondary drive to be awake all the time since the journal would write every five seconds or so. If nothing is being written to the drive, nothing needs to be written to the journal. A sync does nothing if there is nothing in the buffers. signature.asc Description: Digital signature
Re: A Question about Journalling File Systems and Flash Drives
green wrote at 2011-08-26 15:06 -0500: Martin McCormick wrote at 2011-08-26 11:25 -0500: How hard is the ext3 file system on present-day flash drives? Not much worse than without a journal I think, but ...that is just the feeling I have gotten from trying in vain to answer that question definitively for myself. signature.asc Description: Digital signature
Re: A Question about Journalling File Systems and Flash Drives
All I can offer is some almost-on-target empirical evidence. My daughter has had an Acer Aspire One for 3 or 4 years. Her /home was an 8GB Sandisk SD card. It never had any problems...until her husband noticed he had an SD card slot on his brand new win7 laptop...and grabbed her card and Windows partially formatted (corrupted) it. Admittedly she was running XFS on it, but it is still a journaling filesystem. Since that 8GB card was broken, she upgraded to a 16GB card, which is running ext4. its been a year and no problems thus far. For the record, I believe drive quality has improved in the past few years. I have heard that the limit on these drives is now a non-issue... --b On Fri, Aug 26, 2011 at 4:06 PM, green greenfreedo...@gmail.com wrote: Martin McCormick wrote at 2011-08-26 11:25 -0500: How hard is the ext3 file system on present-day flash drives? Not much worse than without a journal I think, but What got me to thinking was that I have a system using conventional magnetic-based hard drives and ext3 file systems. The second hard drive is not used as often and I noticed that the system shuts it down to rest until one calls for a file off the secondary drive. If the journal for all drives is on the boot drive, then that explains everything. Normally, the journal for a filesystem is stored in the same partition as the filesystem. If not, one would expect the secondary drive to be awake all the time since the journal would write every five seconds or so. If nothing is being written to the drive, nothing needs to be written to the journal. A sync does nothing if there is nothing in the buffers. -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.10 (GNU/Linux) iEYEARECAAYFAk5X/NsACgkQ682C+dBP+oRVGACeKx2mS+N4OaWizYijs6JLrfib bjAAoI5Sm8PQsUgt2uJ4k/NcJz9ExVwn =BbYI -END PGP SIGNATURE-
Re: A Question about Journalling File Systems and Flash Drives
On 27/08/11 02:25, Martin McCormick wrote: How hard is the ext3 file system on present-day flash drives? I have some older Dell systems that run lenny and I put a flash drive on as the boot drive on one of those systems and it works great, but for how long? What got me to thinking was that I have a system using conventional magnetic-based hard drives and ext3 file systems. The second hard drive is not used as often and I noticed that the system shuts it down to rest until one calls for a file off the secondary drive. If the journal for all drives is on the boot drive, then that explains everything. If not, one would expect the secondary drive to be awake all the time since the journal would write every five seconds or so. Thank you. Martin McCormick WB5AGZ Stillwater, OK Systems Engineer OSU Information Technology Department Telecommunications Services Group Based on my experiences using ext3 as a file system for running Debian on USB sticks. Single partition (no swap), logging redirected to vt12. 2 identical sticks (major brand) - identical builds - noatime enabled on one, not on the other, both got roughly the same amount of use. The one without noatime died earlier this year after approx 2 years of use - the other has been upgraded to Squeeze and still works fine. Hardly empirical evidence but... I bought a larger USB stick (cheap and nasty) early this year - installed Debian onto it, forgot to enable noatime - it died last week. I'm not certain of the answer - but would strongly suggest enabling noatime on the flash drive - and moving /tmp and /var to a non-flash drive. Cheers -- When two or more people agree on an issue, I form on the other side. — Bill Hicks -- To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/4e584618.4080...@gmail.com
