Re: ext2 on flash memory
Hi Jörn :) * Jörn Engel <[EMAIL PROTECTED]> dixit: > On Fri, 15 June 2007 18:22:53 +0200, DervishD wrote: > > A pity that my digital camera won't want to use LogFS :(( > > That's quite ok, actually. FAT is perfect for digital cameras. All the > unix file permissions, different owners/groups, etc. just get in the > way. And wearout is not a problem in a digicam anyway. Yes, but I would like to share my SD and CF cards between Linux and my camera without having to use FAT for that. Not very bad, anyway, but... Raúl Núñez de Arenas Coronado -- Linux Registered User 88736 | http://www.dervishd.net It's my PC and I'll cry if I want to... RAmen! - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: ext2 on flash memory
On Fri, 15 June 2007 18:16:44 +0200, DervishD wrote: > > Yes, I understand. That's worse than I thought. I was right now > thinking about "PortableApps", a set of free software applications that > are a little bit modified to work from a pendrive in Windows. Very > useful, because you can carry your OpenOffice (or AbiWord) and Firefox, > for example, to any Windows you're doomed to use. This applications > write repeteadly to the pendrive (specially Firefox...), and I think > they may wear the drive prematurely if used extensively. Possibly, yes. Estimates based on unknown numbers are hard, though. > So you have a zone which is unusable. Decent filesystems allow you > to mark some blocks as "dontuse" (ext2, for example). Does FAT allow > this? I don't know. If things get bad enough that you experience uncorrectable errors, I'd just trash the stick and get a new one instead. Trouble is - with a controller chip doing the error correction, you have no means of knowing when this happens. > I think I will be writing the pendrive, almost fully, once a week > (enough for an image of my home directory). That means less than 1GiB > written each week. That means 50-100 (in the worst case, making two > backups a week instead of one) writings each year in each block. Writing everything once a week is harmless. Filling the whole device is best-case behaviour. Local hot spots are the problem. Jörn -- Anything that can go wrong, will. -- Finagle's Law - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: ext2 on flash memory
On Fri, 15 June 2007 18:20:01 +0200, DervishD wrote: > > > > Now seriously, I will take a look at LogFS from time to time, and if > > > you want me to, I can do tests on my Kingston DT. > > > > That would be appreciated. I am always happy about bug reports. The > > more you test the faster I can fix things. > > I'm going to upgrade to 2.6.20.x next week, probably, but I'm not > fully sure. Can your patch be applied to 2.6.19.5 or different versions > are needed for each and every latest kernel release? I've seen the code > in your web and looked like it will work in any 2.6.x kernel as long as > it is not very old, am I wrong?. It should, I just haven't tested it. Same for the I_SYNC patch. > If it can be used with 2.6.19.5, I can test ASAP even if I don't > upgrade to 2.6.20.x. I cannot reboot my machine until Monday (except if > a power shortage does the job for me, of course), but then I'll patch my > kernel and test LogFS. I'm impatient! Just try the patches. If both apply without rejects, things should be fine. Otherwise I'll have to take a look. Jörn -- The competent programmer is fully aware of the strictly limited size of his own skull; therefore he approaches the programming task in full humility, and among other things he avoids clever tricks like the plague. -- Edsger W. Dijkstra - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: ext2 on flash memory
On Fri, 15 June 2007 18:22:53 +0200, DervishD wrote: > > A pity that my digital camera won't want to use LogFS :(( That's quite ok, actually. FAT is perfect for digital cameras. All the unix file permissions, different owners/groups, etc. just get in the way. And wearout is not a problem in a digicam anyway. Jörn -- You can take my soul, but not my lack of enthusiasm. -- Wally - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: ext2 on flash memory
Hi Jörn :) * Jörn Engel <[EMAIL PROTECTED]> dixit: > On Fri, 15 June 2007 00:46:46 +0200, DervishD wrote: > > When do you think it will be included mainstream? > > I am horrible at predictions, doubly so when concerning the future. > > Several people believe it is good enough for -mm inclusion now. So it > might make it for 2.6.23 or 2.6.24. I don't really give a heck about that, because I will be using that pendrive only in my system, but I think that pendrives are pretty common nowadays, and using FAT with them is a waste. I don't like losing my metadata just because I have to copy my data on a pendrive. So it's good to see this filesystem mainstream'ed, it will be very good for small distros (specially now that most of mobos allow booting from USB sticks). A pity that my digital camera won't want to use LogFS :(( Raúl Núñez de Arenas Coronado -- Linux Registered User 88736 | http://www.dervishd.net It's my PC and I'll cry if I want to... RAmen! - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: ext2 on flash memory
Hi Jörn :) * Jörn Engel <[EMAIL PROTECTED]> dixit: > On Thu, 14 June 2007 22:20:47 +0200, DervishD wrote: > > I'm with you in that. So stop emailing and go working on it XD > > :) No. Seriously. Go! XDD > > Now seriously, I will take a look at LogFS from time to time, and if > > you want me to, I can do tests on my Kingston DT. > > That would be appreciated. I am always happy about bug reports. The > more you test the faster I can fix things. I'm going to upgrade to 2.6.20.x next week, probably, but I'm not fully sure. Can your patch be applied to 2.6.19.5 or different versions are needed for each and every latest kernel release? I've seen the code in your web and looked like it will work in any 2.6.x kernel as long as it is not very old, am I wrong?. If it can be used with 2.6.19.5, I can test ASAP even if I don't upgrade to 2.6.20.x. I cannot reboot my machine until Monday (except if a power shortage does the job for me, of course), but then I'll patch my kernel and test LogFS. I'm impatient! Raúl Núñez de Arenas Coronado -- Linux Registered User 88736 | http://www.dervishd.net It's my PC and I'll cry if I want to... RAmen! - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: ext2 on flash memory
Hi Jörn :) * Jörn Engel <[EMAIL PROTECTED]> dixit: > On Thu, 14 June 2007 22:17:14 +0200, DervishD wrote: > > * Jörn Engel <[EMAIL PROTECTED]> dixit: > > > 2. Wear leveling > > > > > > Wear leveling happens implicitly by picking a different physical block > > > from the spares on each write. However, some blocks are never used. If > > > a physical block is mapped to a logical block that never gets written, > > > it is out of the rotation. Two seperate 1024-block areas have their > > > internal wear leveling each, but nothing is spreading high wear from one > > > area to another. > > > > I thought that wear leveling wasn't local to a group of blocks. This > > means that you can destroy a flash memory by writing to the same > > positions... which is a very common usage pattern. > > That is correct. Noticing this will take a while, though. Even the > worst chips sold today seem to have a manufacturer guarantee of 10.000 > erases per block. Across a 16MB area, that means you have to write > 160GB of data to that one area before you enter the danger zone. > Significantly less with static data around, but still a lot. Yes, I understand. That's worse than I thought. I was right now thinking about "PortableApps", a set of free software applications that are a little bit modified to work from a pendrive in Windows. Very useful, because you can carry your OpenOffice (or AbiWord) and Firefox, for example, to any Windows you're doomed to use. This applications write repeteadly to the pendrive (specially Firefox...), and I think they may wear the drive prematurely if used extensively. > And even when that happens, the chips doesn't suddenly explode on the > 10.001st erase. Your chances of getting correctable or uncorrectable > errors will increase, though. So you have a zone which is unusable. Decent filesystems allow you to mark some blocks as "dontuse" (ext2, for example). Does FAT allow this? > For the majority of use cases, such a behaviour is good enough. It > beats floppies hands down. You will never wear them out with your > digital camera or mp3 player. Using it instead of a hard disk in your > notebook may be a little risky, though. Yes, I know. I wasn't aware of the local wear leveling. Thanks a lot for explaining, I will be much more gentle with my pendrives O:)) > > > > 4. FAT requirement > > > > > > When I claimed there was nothing more to smartmedia, I was actually > > > lying. Smartmedia has the odd requirement that only FAT is supported as > > > a filesystem. In fact, the specifications describe FAT in great detail. > > > > Right, but I've seen many people using their pendrives with ext2 > > with no problems (e.g. for SLAX). So, what do you mean by "supported"? > > If a filesystem can be used with the memory, do you mean that wear > > leveling and other characteristics of the flash memory are tailored for > > FAT? or do you mean you cannot use reliably (read: you will lose data) > > other filesystems? > > It doesn't matter much what I mean. The question is how the > manufacturer of your device interpreted it. Most likely ext2 will work > just fine within the limitations I've outlined. > > What changes is the failure mode. Floppies died from storing them in a > dark and cool place. Some were sold brand-new and dead. Disks die from > random, mostly mechanical, failures at a rate of 7% per years in a > Google setup. Flash usually dies from writing to it, the more you write > the faster it dies. > > So as long as you rarely write, pick any filesystem you like. The > choice only matters when you write a lot. I think I will be writing the pendrive, almost fully, once a week (enough for an image of my home directory). That means less than 1GiB written each week. That means 50-100 (in the worst case, making two backups a week instead of one) writings each year in each block. > > > a) Do wear leveling! > > > > > > Smartmedia wear leveling is limited to within areas. Any cross-device > > > wear leveling must be done by the filesystem. FAT does that fairly > > > well. The Ext family doesn't. > > > > Cross-device wear leveling? I don't understand, sorry O:) > > Wear leveling across the whole device, not just one area. Oh, sorry, now I understand :)) > > Thanks for your explanation, it has been very educational :) > > No worries! That's the kind of information you never want anywhere when investigating, so I'm very grateful for that, really :) Raúl Núñez de Arenas Coronado -- Linux Registered User 88736 | http://www.dervishd.net It's my PC and I'll cry if I want to... RAmen! - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: ext2 on flash memory
On Fri, 15 June 2007 00:46:46 +0200, DervishD wrote: > > When do you think it will be included mainstream? I am horrible at predictions, doubly so when concerning the future. Several people believe it is good enough for -mm inclusion now. So it might make it for 2.6.23 or 2.6.24. Jörn -- There is no worse hell than that provided by the regrets for wasted opportunities. -- Andre-Louis Moreau in Scarabouche - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: ext2 on flash memory
Hi Jörn :) * Jörn Engel <[EMAIL PROTECTED]> dixit: > On Thu, 14 June 2007 19:19:53 +0200, Tomasz Chmielewski wrote: > > Cool, does it mean we have the first Linux filesystem supporting > > compression, which can be used on USB-sticks (I don't count old > > ext2+compression patches)? :) > > Indeed it does. When do you think it will be included mainstream? Raúl Núñez de Arenas Coronado -- Linux Registered User 88736 | http://www.dervishd.net It's my PC and I'll cry if I want to... RAmen! - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: ext2 on flash memory
Jan Knutar wrote: On Wednesday 13 June 2007 16:48, DervishD wrote: But anyway the memory should last long. Even cheap flash memories with poor wear leveling (if any at all) usually long last. Given that I won't be writing continuously, wear shouldn't be a problem. I'm going to use this as a backup copy of my home. Of course, I can use a tarball too... I did a test on my kingston datatraveler recently, I didn't expect it to survive, but it did. I put reiserfs on it, and copied 394M of data in 200,000 files to it. Reiserfs was slw at writing, the device was probably doing alot of work. ext2 was about 10X faster, but there was hardly any free space left at all at the end :) Considering it surived ReiserFS, I suspect it would last ages with ext2, especially for your backup purposes. I have a couple of years old USB stick, which was used for swap, and for compiling stuff natively on some small mipsel devices, and generally moving files back and forth a lot (ext3 + noatime). Still, it works just fine. -- Tomasz Chmielewski http://wpkg.org - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: ext2 on flash memory
On Thu, 14 June 2007 22:20:47 +0200, DervishD wrote: > > I'm with you in that. So stop emailing and go working on it XD :) > Now seriously, I will take a look at LogFS from time to time, and if > you want me to, I can do tests on my Kingston DT. That would be appreciated. I am always happy about bug reports. The more you test the faster I can fix things. Jörn -- But this is not to say that the main benefit of Linux and other GPL software is lower-cost. Control is the main benefit--cost is secondary. -- Bruce Perens - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: ext2 on flash memory
On Thu, 14 June 2007 22:17:14 +0200, DervishD wrote: > * Jörn Engel <[EMAIL PROTECTED]> dixit: > > So let us look at the problems and how they interact with filesystems. > > > > 1. Write overhead > > > > If a filesystem only writes a small amount of data, typically 512 or > > 4096 bytes, smartmedia has to erase and write a full block. Most > > flashes used in embedded systems has block sizes of 128KiB or so. Most > > flashes used for smartmedia have 16KiB. Writing 16KiB when the > > filesystem only requests writing 4KiB increases the wear 4x and reduces > > performance 4x. > > But this is not easily addressable by ext2... or FAT. Even if you > use 16KiB blocks in ext2, sometimes you will be writing only 512 bytes. > And FAT will be even worse, I imagine, when using 512-byte blocks. Won't > buffer cache (under Linux) alleviate this problem? Sometimes writing 512 bytes is not a big problem. If 99% of the writes are large, everything will be fine. But with a 32x increase from 512 byte writes, those really should be rare. > > 2. Wear leveling > > > > Wear leveling happens implicitly by picking a different physical block > > from the spares on each write. However, some blocks are never used. If > > a physical block is mapped to a logical block that never gets written, > > it is out of the rotation. Two seperate 1024-block areas have their > > internal wear leveling each, but nothing is spreading high wear from one > > area to another. > > I thought that wear leveling wasn't local to a group of blocks. This > means that you can destroy a flash memory by writing to the same > positions... which is a very common usage pattern. That is correct. Noticing this will take a while, though. Even the worst chips sold today seem to have a manufacturer guarantee of 10.000 erases per block. Across a 16MB area, that means you have to write 160GB of data to that one area before you enter the danger zone. Significantly less with static data around, but still a lot. And even when that happens, the chips doesn't suddenly explode on the 10.001st erase. Your chances of getting correctable or uncorrectable errors will increase, though. For the majority of use cases, such a behaviour is good enough. It beats floppies hands down. You will never wear them out with your digital camera or mp3 player. Using it instead of a hard disk in your notebook may be a little risky, though. > > 4. FAT requirement > > > > When I claimed there was nothing more to smartmedia, I was actually > > lying. Smartmedia has the odd requirement that only FAT is supported as > > a filesystem. In fact, the specifications describe FAT in great detail. > > Right, but I've seen many people using their pendrives with ext2 > with no problems (e.g. for SLAX). So, what do you mean by "supported"? > If a filesystem can be used with the memory, do you mean that wear > leveling and other characteristics of the flash memory are tailored for > FAT? or do you mean you cannot use reliably (read: you will lose data) > other filesystems? It doesn't matter much what I mean. The question is how the manufacturer of your device interpreted it. Most likely ext2 will work just fine within the limitations I've outlined. What changes is the failure mode. Floppies died from storing them in a dark and cool place. Some were sold brand-new and dead. Disks die from random, mostly mechanical, failures at a rate of 7% per years in a Google setup. Flash usually dies from writing to it, the more you write the faster it dies. So as long as you rarely write, pick any filesystem you like. The choice only matters when you write a lot. > > a) Do wear leveling! > > > > Smartmedia wear leveling is limited to within areas. Any cross-device > > wear leveling must be done by the filesystem. FAT does that fairly > > well. The Ext family doesn't. > > Cross-device wear leveling? I don't understand, sorry O:) Wear leveling across the whole device, not just one area. > Thanks for your explanation, it has been very educational :) No worries! Jörn -- Homo Sapiens is a goal, not a description. -- unknown - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: ext2 on flash memory
Hi Jörn :) * Jörn Engel <[EMAIL PROTECTED]> dixit: > Any method I can imagine to offer good wear leveling will result in > either a filesystem or at least a simplified one-file-system with the > only file being the "block device" exported outward. So naturally my > answer to the problem is called LogFS. :) I'm with you in that. So stop emailing and go working on it XD Now seriously, I will take a look at LogFS from time to time, and if you want me to, I can do tests on my Kingston DT. Raúl Núñez de Arenas Coronado -- Linux Registered User 88736 | http://www.dervishd.net It's my PC and I'll cry if I want to... RAmen! - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: ext2 on flash memory
Hi Jörn :) * Jörn Engel <[EMAIL PROTECTED]> dixit: > So let us look at the problems and how they interact with filesystems. > > 1. Write overhead > > If a filesystem only writes a small amount of data, typically 512 or > 4096 bytes, smartmedia has to erase and write a full block. Most > flashes used in embedded systems has block sizes of 128KiB or so. Most > flashes used for smartmedia have 16KiB. Writing 16KiB when the > filesystem only requests writing 4KiB increases the wear 4x and reduces > performance 4x. But this is not easily addressable by ext2... or FAT. Even if you use 16KiB blocks in ext2, sometimes you will be writing only 512 bytes. And FAT will be even worse, I imagine, when using 512-byte blocks. Won't buffer cache (under Linux) alleviate this problem? > 2. Wear leveling > > Wear leveling happens implicitly by picking a different physical block > from the spares on each write. However, some blocks are never used. If > a physical block is mapped to a logical block that never gets written, > it is out of the rotation. Two seperate 1024-block areas have their > internal wear leveling each, but nothing is spreading high wear from one > area to another. I thought that wear leveling wasn't local to a group of blocks. This means that you can destroy a flash memory by writing to the same positions... which is a very common usage pattern. > 4. FAT requirement > > When I claimed there was nothing more to smartmedia, I was actually > lying. Smartmedia has the odd requirement that only FAT is supported as > a filesystem. In fact, the specifications describe FAT in great detail. Right, but I've seen many people using their pendrives with ext2 with no problems (e.g. for SLAX). So, what do you mean by "supported"? If a filesystem can be used with the memory, do you mean that wear leveling and other characteristics of the flash memory are tailored for FAT? or do you mean you cannot use reliably (read: you will lose data) other filesystems? > a) Do wear leveling! > > Smartmedia wear leveling is limited to within areas. Any cross-device > wear leveling must be done by the filesystem. FAT does that fairly > well. The Ext family doesn't. Cross-device wear leveling? I don't understand, sorry O:) Thanks for your explanation, it has been very educational :) Raúl Núñez de Arenas Coronado -- Linux Registered User 88736 | http://www.dervishd.net It's my PC and I'll cry if I want to... RAmen! - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: ext2 on flash memory
Hi Jan :) * Jan Knutar <[EMAIL PROTECTED]> dixit: > On Wednesday 13 June 2007 16:48, DervishD wrote: > > But anyway the memory should last long. Even cheap flash memories > > with poor wear leveling (if any at all) usually long last. Given > > that I won't be writing continuously, wear shouldn't be a problem. > > I'm going to use this as a backup copy of my home. Of course, I can > > use a tarball too... > > I did a test on my kingston datatraveler recently, I didn't expect it > to survive, but it did. Agh! Never do such tests again, or at least don't tell me XD > Considering it surived ReiserFS, I suspect it would last ages with > ext2, especially for your backup purposes. I don't really know the number of writes that a flash memory allows nowadays, but probably they're 1 even in cheap ones, and with proper wear leveling that should mean a lot of time. Thanks for the test, really O:)) Raúl Núñez de Arenas Coronado -- Linux Registered User 88736 | http://www.dervishd.net It's my PC and I'll cry if I want to... RAmen! - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: ext2 on flash memory
On Wednesday 13 June 2007 16:48, DervishD wrote: > But anyway the memory should last long. Even cheap flash memories > with poor wear leveling (if any at all) usually long last. Given that > I won't be writing continuously, wear shouldn't be a problem. I'm > going to use this as a backup copy of my home. Of course, I can use a > tarball too... I did a test on my kingston datatraveler recently, I didn't expect it to survive, but it did. I put reiserfs on it, and copied 394M of data in 200,000 files to it. Reiserfs was slw at writing, the device was probably doing alot of work. ext2 was about 10X faster, but there was hardly any free space left at all at the end :) Considering it surived ReiserFS, I suspect it would last ages with ext2, especially for your backup purposes. - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: ext2 on flash memory
On Thu, 14 June 2007 19:45:10 +0200, Jörn Engel wrote: > > Nearly two years ago I have spoken to a person that reverse engineered > the behaviour of several chips used in pendrives. At the time that > reverse engineering apparently covered most of the market. The details > were quite lengthy but can be condensed to two words: Smartmedia format. Maybe I should add that things have allegedly improved. Many devices today support both static and dynamic wear leveling. Dynamic wear leveling is what smartmedia does. It depends on the filesystem writing somewhere to move those blocks. Static wear leveling will also move blocks that are not written. However, the exact nature of wear leveling is not disclosed. And I see no reason to trust an undisclosed "static and dynamic wear leveling" any more than I trust smartmedia. I will even go further and claim that nothing short of a filesystem can do proper wear leveling across the complete device. The reason smartmedia introduced "areas" was to bound the time until it's map is created and the device can get accessed. If the map spanned a large 64GB device, access times would go sky-high. Any method I can imagine to offer good wear leveling will result in either a filesystem or at least a simplified one-file-system with the only file being the "block device" exported outward. So naturally my answer to the problem is called LogFS. :) Jörn -- It's not whether you win or lose, it's how you place the blame. -- unknown - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: ext2 on flash memory
On Thu, 14 June 2007 19:19:53 +0200, Tomasz Chmielewski wrote: > > Cool, does it mean we have the first Linux filesystem supporting > compression, which can be used on USB-sticks (I don't count old > ext2+compression patches)? :) Indeed it does. Jörn -- Anything that can go wrong, will. -- Finagle's Law - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: ext2 on flash memory
On Mon, 11 June 2007 12:13:19 +0200, DervishD wrote: > > I was wondering: is there any reason not to use ext2 on an USB > pendrive? Really my question is not only about USB pendrives, but any > device whose storage is flash based. Let's assume that the device has a > good quality flash memory with wear leveling and the like... Reading through the thread I noticed a lot of guesswork and a total lack of understanding about how the pendrive actually works. Everyone treats it is a black box that has "wear leveling". So let me crack the box open to some degree. Nearly two years ago I have spoken to a person that reverse engineered the behaviour of several chips used in pendrives. At the time that reverse engineering apparently covered most of the market. The details were quite lengthy but can be condensed to two words: Smartmedia format. Smartmedia is a very simple format and can easily be studied. Afaik access to the specs only requires registration. Alternatively one can study drivers/usb/storage/alauda.c, which implements the format. I will summarize the important bits below, assuming the reader has a good understanding about what flash is. All flash memory is split into zones of usually 1024(1) erase blocks. Within each zone, 1000 logical blocks are mapped to the 1024 physical blocks. That leaves 24 spare one. Normally the spare blocks are erased and contain no data. The 1000 mapped physical blocks contain the associated logical number somewhere in the OOB area. Reading from smartmedia first requires reading all logical numbers from all physical blocks. With this information a simple map, essentially a 1024-entry array is created. Once the map is set up, logical number get translated to physical ones with a simple lookup. Writing is slightly more complicated. First one of the spare blocks is selected. New data is written to the spare block. If only a partial block is written, the remainder has to be copied from the old block. After the new block is written, the old block is erased and becomes a spare. If the chip loses power during a write, several physical block may contain the same logical number. In that case the ECC information is used to verify each of the blocks. If one of the block was not written completely, the other is used. If one block has ECC errors, the other is used. If both are correct, a random one is used. That's all there is to smartmedia format. It is _very_ simple. It is also surprisingly efficient, although it does have some shortcomings. So let us look at the problems and how they interact with filesystems. 1. Write overhead If a filesystem only writes a small amount of data, typically 512 or 4096 bytes, smartmedia has to erase and write a full block. Most flashes used in embedded systems has block sizes of 128KiB or so. Most flashes used for smartmedia have 16KiB. Writing 16KiB when the filesystem only requests writing 4KiB increases the wear 4x and reduces performance 4x. 2. Wear leveling Wear leveling happens implicitly by picking a different physical block from the spares on each write. However, some blocks are never used. If a physical block is mapped to a logical block that never gets written, it is out of the rotation. Two seperate 1024-block areas have their internal wear leveling each, but nothing is spreading high wear from one area to another. So if a theoretical filesystem would only ever write to the same logical block, smartmedia can spead the wear over 25 blocks or less. Less, if any physical blocks are bad and cannot get used (2). More realistically, if Ext3 is having a very hot area - the journal - that area is not getting any wear leveling worth mentioning. Journaling filesystems on smartmedia are a bad idea. Most journals are bigger than a smartmedia area, which is usually 1000 * 16KiB. The round-robin access pattern of the journal already provides perfect wear leveling _within that area_. Smartmedia does not add anything. 3. Hidden caching Some chip designers seem to have noticed the 4x write overhead and try to outsmart the filesystem. Their chips start writing the new block, but don't copy data from the old block just yet. Instead they wait for further write requests, hoping to write adjacent data to the same block. If the power fails while the chip is waiting for further data, smarmedia format requires the unfinished block to get erased and its content to get discarded. What happens if the user just typed "sync" and yanks the pendrive as soon as the command returns is anyone's guess. How many people ever considered their pendrives to perform caching and require proper barriers? 4. FAT requirement When I claimed there was nothing more to smartmedia, I was actually lying. Smartmedia has the odd requirement that only FAT is supported as a filesystem. In fact, the specifications describe FAT in great detail. I have already seen FTLs(3) that deliberately look into the FAT and pre-erase blocks if the corre
Re: ext2 on flash memory
Jörn Engel schrieb: On Mon, 11 June 2007 13:53:00 +0200, Tomasz Chmielewski wrote: jffs2 only works on mtd devices, and that excludes pendrives, which are block devices. I know LogFS will work with block devices one day, but currently, it doesn't (and is not in the kernel yet as well). Actually, LogFS does work on block devices now. Performance on hard disks is quite bad. Hopefully that becomes untrue just as fast as your slightly outdated claim. :) Cool, does it mean we have the first Linux filesystem supporting compression, which can be used on USB-sticks (I don't count old ext2+compression patches)? :) -- Tomasz Chmielewski http://wpkg.org - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: ext2 on flash memory
On Mon, 11 June 2007 13:53:00 +0200, Tomasz Chmielewski wrote: > > jffs2 only works on mtd devices, and that excludes pendrives, which are > block devices. I know LogFS will work with block devices one day, but > currently, it doesn't (and is not in the kernel yet as well). Actually, LogFS does work on block devices now. Performance on hard disks is quite bad. Hopefully that becomes untrue just as fast as your slightly outdated claim. :) Jörn -- Measure. Don't tune for speed until you've measured, and even then don't unless one part of the code overwhelms the rest. -- Rob Pike - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: ext2 on flash memory
*This message was transferred with a trial version of CommuniGate(r) Pro* Juergen Beisert wrote: So it makes no sense to find the best filesystem for such a case. There is no best one. It does make sense. Wear leveling isn't the only thing that matters. An important criteria is the total amount of wear that you get when using a filesystem. Some filesystems are simply not suitable: they either write too often to disk (though, as I said, this can be alleviated by COW-ing it or working on a copy and then updating the flash drive), or a single operation requires too many changes to its image/structure. Normal fs-s often change their internal structure, trading for space efficiency or speed. Better storage and accounting of data involve more complicated internal fs structures, that aren't too stable over time (that is, they change often and much). For example, an ISO9660 multisession rewritable CD/DVD trades space efficiency and flexibility for a lower wear and better wear-leveling. This is obvious, as the user triggers flushes to disk (that is, burning a new session) with a lower frequency, when his work is done, and the writes are always done sequentially (=> wear leveling). I'm not saying anything about UDF, since I don't know much about it. - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: ext2 on flash memory
*This message was transferred with a trial version of CommuniGate(r) Pro* Juergen Beisert wrote: So it makes no sense to find the best filesystem for such a case. There is no best one. It does make sense. Wear leveling isn't the only thing that matters. An important criteria is the total amount of wear that you get when using a filesystem. Some filesystems are simply not suitable: they either write too often to disk (though, as I said, this can be alleviated by COW-ing it or working on a copy and then updating the flash drive), or a single operation requires too many changes to its image/structure. Normal fs-s often change their internal structure, trading for space efficiency or speed. Better storage and accounting of data involve more complicated internal fs structures, that aren't too stable over time (that is, they change often and much). For example, an ISO9660 multisession rewritable CD/DVD trades space efficiency and flexibility for a lower wear and better wear-leveling. This is obvious, as the user triggers flushes to disk (that is, burning a new session) with a lower frequency, when his work is done, and the writes are always done sequentially (=> wear leveling). I'm not saying anything about UDF, since I don't know much about it. - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: ext2 on flash memory
On Donnerstag, 14. Juni 2007, DervishD wrote: > Your message is very peculiar... because I already have a similar > thing working on my system ;))) I tried FSVS and I didn't like it fully > (don't ask me why, I don't even remember, that was a time ago), so I > wrote my own system. Well, if you ever remember (or do a new try), please tell me (or the mailing lists) your suggestions/ideas. > Instead of a complex solution, I opted for a simple (but ad-hoc) > solution, writing a pre-commit-hook in Perl and a couple of files to > store the metadata. Very simple but I have all my system configuration > files (and other files that I want to have versioned) under SVN. But you'd still need to manually restore the permissions, don't you? And I think you don't take devices ... BTW, "fsvs status" is much faster than "svn status" ... sometimes faster than find (on cold caches). > Thanks for your suggestion anyway, because I think that the concept > (having versioned system files) is interesting and very useful :)) I think so too. Thank you for your answer! Regards, Phil - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: ext2 on flash memory
Hi Philipp :)) * Ph. Marek <[EMAIL PROTECTED]> dixit: > > I was just wondering if, apart from the excessive wear, there were > > other reasons. One of the reasons I would like to use a good filesystem > > for a pendrive is to be able to store file metadata (UID, GID, mode, > > etc.) properly, for example to store a "live" copy of my home dir in the > > pendrive. > If it would be enough to have a backup (with meta-data), that needs some > software to be read again, you could also try fsvs (http://fsvs.tigris.org). > > That does a full filesystem versioning (with owner, group, mode and mtime) > into a subversion repository. This would have the benefit that you could keep > the "prefered" filesystem, but have a (versioned!) backup of your data. Your message is very peculiar... because I already have a similar thing working on my system ;))) I tried FSVS and I didn't like it fully (don't ask me why, I don't even remember, that was a time ago), so I wrote my own system. Instead of a complex solution, I opted for a simple (but ad-hoc) solution, writing a pre-commit-hook in Perl and a couple of files to store the metadata. Very simple but I have all my system configuration files (and other files that I want to have versioned) under SVN. Thanks for your suggestion anyway, because I think that the concept (having versioned system files) is interesting and very useful :)) Raúl Núñez de Arenas Coronado -- Linux Registered User 88736 | http://www.dervishd.net It's my PC and I'll cry if I want to... RAmen! - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: ext2 on flash memory
(Sorry for this hand-crafted message. I'm not subscribed, lkml.org is down, so I have to manually add the Reply-To header.) > I was just wondering if, apart from the excessive wear, there were > other reasons. One of the reasons I would like to use a good filesystem > for a pendrive is to be able to store file metadata (UID, GID, mode, > etc.) properly, for example to store a "live" copy of my home dir in the > pendrive. If it would be enough to have a backup (with meta-data), that needs some software to be read again, you could also try fsvs (http://fsvs.tigris.org). That does a full filesystem versioning (with owner, group, mode and mtime) into a subversion repository. This would have the benefit that you could keep the "prefered" filesystem, but have a (versioned!) backup of your data. The files can, if fsvs is not available, be restored by subversion binaries (although without meta-data support), so in case of an emergency you could read the data even from windows. (Like when using tar, you'd need to download something or keep this binary on the usb-stick). http://freshmeat.net/projects/fsvs: FSVS stands for "Fast System VerSioning", "File System VerSioning", or "Full System VerSioning". It is used for backup/restore and system configuration management, using a Subversion repository as the backend. For questions "users" or "dev" "at fsvs.tigris.org" are available. Regards, Phil - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: ext2 on flash memory
* Juergen Beisert <[EMAIL PROTECTED]> dixit: > On Monday 11 June 2007 19:42, DervishD wrote: > > I know about cheap pendrives that you cannot format even with FAT32, only > > with FAT16. > > I'm not sure if the price was the reason that they failed with different > filesystems. Some kind of wear leveling tries to guess which blocks of the > filesystem are in use and which are unused (to avoid wear leveling of unused > data). > But it only works if you are using a filesystem that is "known" by the wear > leveling process. If you are using a different one, it fails badly, because > it tries to interpret a FAT that does not exists, and destroys your > filesystem while the wear leveling process is running. That's exactly what I wanted to know! My Kingston Data Traveler is not exactly cheap (while not a very expensive piece of hardware, Kingston is known for the quality of its memory devices), so I hope it likes ext2. I will do a test. > So if you can use a different filesystem than FAT16/32 on your pendrive, it > does not matter what kind of filesystem you are using. The wear leveling > process has no clue about it and always "wear leveling" used *and* unused > data (means: every block of the whole disk) until it ruins the flash memory. But anyway the memory should last long. Even cheap flash memories with poor wear leveling (if any at all) usually long last. Given that I won't be writing continuously, wear shouldn't be a problem. I'm going to use this as a backup copy of my home. Of course, I can use a tarball too... Raúl Núñez de Arenas Coronado -- Linux Registered User 88736 | http://www.dervishd.net It's my PC and I'll cry if I want to... RAmen! - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: ext2 on flash memory
Hi Kevin :) > >I was wondering: is there any reason not to use ext2 on an USB > >pendrive? Really my question is not only about USB pendrives, but any > >device whose storage is flash based. Let's assume that the device > >has a > >good quality flash memory with wear leveling and the like... > My opinion is that, unless the flash is really cheap, or it is being > written to excessively, that it probably doesn't matter too much. > With the growth in size of flash, just how long do you think it will > continue to be used before you go to something larger? The growth in size and the shrink in price ;) I think that if the stick only lasts for a year, it has been a good investment. > One suggestion with ext2 might be to mount it with the noatime > option, so it doesn't update the last access time for directories and > files. Otherwise, you are doing a write even when you only plan to > read a file. I was going to do that. It should minimize the wear when only reading. Anyway, wearing wasn't my main concern, I was more worried by stoopid firmware that assume FAT. Thanks for your answer :) Raúl Núñez de Arenas Coronado -- Linux Registered User 88736 | http://www.dervishd.net It's my PC and I'll cry if I want to... RAmen! - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: ext2 on flash memory
Hi Bernd :) * Bernd Eckenfels <[EMAIL PROTECTED]> dixit: > In article <[EMAIL PROTECTED]> you wrote: > > All of the posts fail to address the question here: what is the > > correct file system, or does one exist yet, for wear leveling flash > > storage. JFFS2 and logfs are nice for MTD, but for better flash > > memories that are likely to be used in the future like solid state > > hard disks, what is the answer? > > FAT - you can stick it into Windows Boxes on the road. But then you lose the UNIX metadata. What if you want to keep the UID/GID and mode (and any other metadata information, for that matter)? Raúl Núñez de Arenas Coronado -- Linux Registered User 88736 | http://www.dervishd.net It's my PC and I'll cry if I want to... RAmen! - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: ext2 on flash memory
On Tuesday 12 June 2007 23:09, Jason Lunz wrote: > In gmane.linux.kernel, you wrote: > > I was wondering: is there any reason not to use ext2 on an USB > > pendrive? Really my question is not only about USB pendrives, but any > > device whose storage is flash based. Let's assume that the device has a > > good quality flash memory with wear leveling and the like... > > Have a look at the UBI layer. It adds wear-levelling to MTD devices. Of > course, to use it on a regular blockdev like this you'll have to do > something like usb-storage (sd) -> block2mtd -> ubi -> jffs2. But it can > be done afaik. Then you will do some kind of wear leveling (jffs2) over wear leveling (pendrive). Does this makes sense? Juergen - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: ext2 on flash memory
In gmane.linux.kernel, you wrote: > I was wondering: is there any reason not to use ext2 on an USB > pendrive? Really my question is not only about USB pendrives, but any > device whose storage is flash based. Let's assume that the device has a > good quality flash memory with wear leveling and the like... Have a look at the UBI layer. It adds wear-levelling to MTD devices. Of course, to use it on a regular blockdev like this you'll have to do something like usb-storage (sd) -> block2mtd -> ubi -> jffs2. But it can be done afaik. Jason - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: ext2 on flash memory
On Tuesday 12 June 2007 02:35, Kevin Bowling wrote: > All of the posts fail to address the question here: what is the > correct file system, or does one exist yet, for wear leveling flash > storage. > JFFS2 and logfs are nice for MTD, but for better flash > memories that are likely to be used in the future like solid state > hard disks, what is the answer? As long the device itself does the wear leveling, there could no specific answer here! On MTD its the job of the filesystem to do the wear leveling because it works on the bare flash. But memory sticks, CFs and so on emulate a block device. I don't know a device where you could switch off its internal wear leveling mechanism. So it makes no sense to find the best filesystem for such a case. There is no best one. Its the same discussion about protecting essential data on CF by using more than one partition. A first partition for read only (system), a second one also for writing (data). And everyone hopes in the case of a crash the read only partition survives. But CFs internal wear leveling does not know partitions and continuous to use *all* blocks on the CF for its job. Juergen - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: ext2 on flash memory
In article <[EMAIL PROTECTED]> you wrote: > All of the posts fail to address the question here: what is the > correct file system, or does one exist yet, for wear leveling flash > storage. JFFS2 and logfs are nice for MTD, but for better flash > memories that are likely to be used in the future like solid state > hard disks, what is the answer? FAT - you can stick it into Windows Boxes on the road. Bernd - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: ext2 on flash memory
On 6/11/07, Kevin K <[EMAIL PROTECTED]> wrote: On Jun 11, 2007, at 5:13 AM, DervishD wrote: > Hi all :) > > I was wondering: is there any reason not to use ext2 on an USB > pendrive? Really my question is not only about USB pendrives, but any > device whose storage is flash based. Let's assume that the device > has a > good quality flash memory with wear leveling and the like... > > Thanks a lot in advance :) > > Raúl Núñez de Arenas Coronado > My opinion is that, unless the flash is really cheap, or it is being written to excessively, that it probably doesn't matter too much. With the growth in size of flash, just how long do you think it will continue to be used before you go to something larger? A 256MB flash of a few years ago has been supplanted in many cases by today's 2-4gb memory. One suggestion with ext2 might be to mount it with the noatime option, so it doesn't update the last access time for directories and files. Otherwise, you are doing a write even when you only plan to read a file. All of the posts fail to address the question here: what is the correct file system, or does one exist yet, for wear leveling flash storage. JFFS2 and logfs are nice for MTD, but for better flash memories that are likely to be used in the future like solid state hard disks, what is the answer? - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: ext2 on flash memory
On Monday 11 June 2007, Tomasz Chmielewski wrote: > Also, ext2 provides a nice feature other filesystems lack: xip. > Especially, if a pendrive is used as a rootfs for a small device. Well, xip cannot work on NAND flash media, including USB pen drives, because the data is not mapped into the addressable memory space, so that is not really an interesting argument. Arnd <>< - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: ext2 on flash memory
On Jun 11, 2007, at 5:13 AM, DervishD wrote: Hi all :) I was wondering: is there any reason not to use ext2 on an USB pendrive? Really my question is not only about USB pendrives, but any device whose storage is flash based. Let's assume that the device has a good quality flash memory with wear leveling and the like... Thanks a lot in advance :) Raúl Núñez de Arenas Coronado My opinion is that, unless the flash is really cheap, or it is being written to excessively, that it probably doesn't matter too much. With the growth in size of flash, just how long do you think it will continue to be used before you go to something larger? A 256MB flash of a few years ago has been supplanted in many cases by today's 2-4gb memory. One suggestion with ext2 might be to mount it with the noatime option, so it doesn't update the last access time for directories and files. Otherwise, you are doing a write even when you only plan to read a file. - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: ext2 on flash memory
On Monday 11 June 2007 19:42, DervishD wrote: > I just was curious about the issue and I was asking to know if > anybody had tried this. Think about compact flash devices. They also using some kind of flash memory and also doing wear leveling. And I think they are not only used with FAT16/32! If they run with different filesystems, then your pendrive stick will also. Only the interface is different. > I know about cheap pendrives that you cannot format even with FAT32, only > with FAT16. I'm not sure if the price was the reason that they failed with different filesystems. Some kind of wear leveling tries to guess which blocks of the filesystem are in use and which are unused (to avoid wear leveling of unused data). But it only works if you are using a filesystem that is "known" by the wear leveling process. If you are using a different one, it fails badly, because it tries to interpret a FAT that does not exists, and destroys your filesystem while the wear leveling process is running. So this cheap pendrive was too intelligent for filesystems other than FAT16 So if you can use a different filesystem than FAT16/32 on your pendrive, it does not matter what kind of filesystem you are using. The wear leveling process has no clue about it and always "wear leveling" used *and* unused data (means: every block of the whole disk) until it ruins the flash memory. Hope it helps Juergen - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: ext2 on flash memory
Hi Alan :) * alan <[EMAIL PROTECTED]> dixit: > On Mon, 11 Jun 2007, DervishD wrote: > > I was wondering: is there any reason not to use ext2 on an USB > >pendrive? Really my question is not only about USB pendrives, but any > >device whose storage is flash based. Let's assume that the device has a > >good quality flash memory with wear leveling and the like... > > Do you need to use OSes other than Linux? FAT16/32 seems to be a pretty > universal filesystem at this point. You can mount ext2 on Windows, but it > is a pain. Not certain what it takes to mount it on OS X. I don't really need to use ext2, I just was curious. I have a couple of pendrives and I was thinking about using one of them to store a "live" copy of my home. So, I wanted ext2 for that. I can access the data from Windows if I need to, but usually I try not to touch a Windows even with a 100 mts pole ;) Portability (so to speak), is not important for me. > I have encountered flash drives that do not format well for anything > other than FAT16, but they were old and small. Hopefully that problem > no longer exists with modern hardware. I would format it then test > the hell out of it before trusting it with important data. That's the answer I was looking for ;))) I mean, will the pendrive have problems with ext2 due to some strange behaviour of the hardware? I know about cheap pendrives that you cannot format even with FAT32, only with FAT16. The pendrive I was considering to have the "live" copy of my home was a Kingston Data Traveler. Of course, I can go with a loopback: this way I'll have a "live" backup of my home, with all UNIX metadata saved but able to reside on a FAT16 filesystem. Not very comfortable, given that I won't use it outside Linux (probably). I just was curious about the issue and I was asking to know if anybody had tried this. Raúl Núñez de Arenas Coronado -- Linux Registered User 88736 | http://www.dervishd.net It's my PC and I'll cry if I want to... RAmen! - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: ext2 on flash memory
Hi Eduard :) * Eduard-Gabriel Munteanu <[EMAIL PROTECTED]> dixit: > >I was wondering: is there any reason not to use ext2 on an USB > >pendrive? Really my question is not only about USB pendrives, but any > >device whose storage is flash based. Let's assume that the device has > >a good quality flash memory with wear leveling and the like... > > Your USB pendrive will wear faster if you use an inappropriate > filesystem. Such filesystems require frequent writes and change their > internal state often. That's what I supposed. > Really, why don't you try a more suitable fs for your pendrive, one that > changes itself less than usual fs's? I was just wondering if, apart from the excessive wear, there were other reasons. One of the reasons I would like to use a good filesystem for a pendrive is to be able to store file metadata (UID, GID, mode, etc.) properly, for example to store a "live" copy of my home dir in the pendrive. Thanks for your answer :) Raúl Núñez de Arenas Coronado -- Linux Registered User 88736 | http://www.dervishd.net It's my PC and I'll cry if I want to... RAmen! - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: ext2 on flash memory
On Mon, 11 Jun 2007, DervishD wrote: Hi all :) I was wondering: is there any reason not to use ext2 on an USB pendrive? Really my question is not only about USB pendrives, but any device whose storage is flash based. Let's assume that the device has a good quality flash memory with wear leveling and the like... Thanks a lot in advance :) It depends... Do you need to use OSes other than Linux? FAT16/32 seems to be a pretty universal filesystem at this point. You can mount ext2 on Windows, but it is a pain. Not certain what it takes to mount it on OS X. I have encountered flash drives that do not format well for anything other than FAT16, but they were old and small. Hopefully that problem no longer exists with modern hardware. I would format it then test the hell out of it before trusting it with important data. -- "ANSI C says access to the padding fields of a struct is undefined. ANSI C also says that struct assignment is a memcpy. Therefore struct assignment in ANSI C is a violation of ANSI C..." - Alan Cox - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: ext2 on flash memory
Eduard-Gabriel Munteanu wrote: (...) Your USB pendrive will wear faster if you use an inappropriate filesystem. Such filesystems require frequent writes and change their internal state often. This could be alleviated by COWing the filesystem somehow and flushing writes when you're finished. But the modifications will be lost if crashes occur. The filesystem structures will still change a lot and require big writes to update it. Really, why don't you try a more suitable fs for your pendrive, one that changes itself less than usual fs's? Hmm, are there any fs (read+write) alternatives for pendrives? jffs2 only works on mtd devices, and that excludes pendrives, which are block devices. I know LogFS will work with block devices one day, but currently, it doesn't (and is not in the kernel yet as well). Also, ext2 provides a nice feature other filesystems lack: xip. Especially, if a pendrive is used as a rootfs for a small device. -- Tomasz Chmielewski http://wpkg.org - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: ext2 on flash memory
*This message was transferred with a trial version of CommuniGate(r) Pro* DervishD wrote: *This message was transferred with a trial version of CommuniGate(r) Pro* *This message was transferred with a trial version of CommuniGate(r) Pro* Hi all :) I was wondering: is there any reason not to use ext2 on an USB pendrive? Really my question is not only about USB pendrives, but any device whose storage is flash based. Let's assume that the device has a good quality flash memory with wear leveling and the like... Thanks a lot in advance :) Raúl Núñez de Arenas Coronado Your USB pendrive will wear faster if you use an inappropriate filesystem. Such filesystems require frequent writes and change their internal state often. This could be alleviated by COWing the filesystem somehow and flushing writes when you're finished. But the modifications will be lost if crashes occur. The filesystem structures will still change a lot and require big writes to update it. Really, why don't you try a more suitable fs for your pendrive, one that changes itself less than usual fs's? - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
ext2 on flash memory
Hi all :) I was wondering: is there any reason not to use ext2 on an USB pendrive? Really my question is not only about USB pendrives, but any device whose storage is flash based. Let's assume that the device has a good quality flash memory with wear leveling and the like... Thanks a lot in advance :) Raúl Núñez de Arenas Coronado -- Linux Registered User 88736 | http://www.dervishd.net It's my PC and I'll cry if I want to... RAmen! - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/