Re: question on booting from mmc
On Mon, Jul 16, 2007 at 04:17:50PM -0700, James Sparenberg wrote: > > jffs runs directly on raw NAND flash so it must do such things. SD cards > > however are normal block devices and my understanding is that they do > > same or similar wear levelling when writing blocks. More to the point, SD cards don't give you access raw NAND flash so you couldn't even use jffs2; it's not technically possible. (Go ahead and try! :-) > Real question I can see at this point is which FS has the smallest > journal and the fastest response on solid state media. BTW the #1 > reason I've lost data on ext3 systems was due to automagic fsck. > Admittedly those systems were primarily Fedora Core2 or RHEL3. Ext3 > may have improved since them but I'm still shy. That hasn't been my experience, nor of most people I've talked to. Note that if you are losing data due to the automagic fsck, that's just probably when you are discovering it, not when the damage actually happened. My guess is that you are running your ext3 systems on flaky hardware, but that's just a guess. - Ted ___ maemo-users mailing list maemo-users@maemo.org https://lists.maemo.org/mailman/listinfo/maemo-users
Re: question on booting from mmc
James Sparenberg wrote: > I could agree with the above if it all pans out as you say. Yes, that is of course the question. I am not sure it really works like described. Just applied common sense which may not be the right thing to do in all situations :-) I am booting from kingston 2GB card since approx. August 2006, have swap on card an also switched to ext3 in beginning of 2007 and my card still works fine. This is of course not enough time to prove anything so only time will tell. Frantisek ___ maemo-users mailing list maemo-users@maemo.org https://lists.maemo.org/mailman/listinfo/maemo-users
Re: question on booting from mmc
On Saturday 14 July 2007 14:16:13 Frantisek Dufka wrote: > James Sparenberg wrote: > > I would think the reference here is to lifetime of cell X of the SD card. > > Meaning here that each "block" (or whatever the correct term is) has a > > set # of wrtie/read events. JFFS from what I'm reading understands this > > and as such move journal information around just enough to even out this > > wear. > > > > My understanding is that jffs also takes steps to minimize the # of times > > and amount of reads/writes it does. ext3 however doesn't and it's > > hitting the HDD a lot more often. One of the reasons I'm not ext3's best > > fan. > > jffs runs directly on raw NAND flash so it must do such things. SD cards > however are normal block devices and my understanding is that they do > same or similar wear levelling when writing blocks. So even if you write > same (logical) block on sd card the write goes somewhere else with each > write. I don't work for flash card manufacturer so I am not 100% sure on > this but I guess there is no other sane way how to do it internally in > the flash controller on the card. You need to handle same problems there > (much bigger physical block in NAND chip than logical, pre-erasing > blocks what data is written, remapping bad blocks, ...) like jffs2 > > I think that flash cards are like harddisks nowadays. You no longer care > about physical geometry and bad blocks on current hard disks, all is > done transparently by disk controller and you simply don't know and > can't even guess where on the disk specific block is really stored. > > > Ext2 and FAT don't do anything special. They also don't do anything over > > and over again in the same spot. > > I think FAT table is the spot which is written over and over againg in > most devices using flash cards. Write caching is not common for > removable media so it may be even updated after doing every write > operation to file. And if manufacturers are giving long/lifetime > warranty despite this common usage I wouldn't bother with ext2 or ext3 > which is typically mounted with write caching. I could agree with the above if it all pans out as you say. Perhaps then the only other anti-ext3 remark would fall along side the journal size. Like I've said though I'm not much of a fan of ext3 at all. (Don't like gnome much either *grin*) I've lost to much data and time to ext3 over the last couple of years and I've only lost data with reiserfs and xfs when the drive itself failed. Real question I can see at this point is which FS has the smallest journal and the fastest response on solid state media. BTW the #1 reason I've lost data on ext3 systems was due to automagic fsck. Admittedly those systems were primarily Fedora Core2 or RHEL3. Ext3 may have improved since them but I'm still shy. James > > Frantisek ___ maemo-users mailing list maemo-users@maemo.org https://lists.maemo.org/mailman/listinfo/maemo-users
Re: question on booting from mmc
James Sparenberg wrote: > I would think the reference here is to lifetime of cell X of the SD card. > Meaning here that each "block" (or whatever the correct term is) has a set # > of wrtie/read events. JFFS from what I'm reading understands this and as > such move journal information around just enough to even out this wear. > > My understanding is that jffs also takes steps to minimize the # of times and > amount of reads/writes it does. ext3 however doesn't and it's hitting the > HDD a lot more often. One of the reasons I'm not ext3's best fan. jffs runs directly on raw NAND flash so it must do such things. SD cards however are normal block devices and my understanding is that they do same or similar wear levelling when writing blocks. So even if you write same (logical) block on sd card the write goes somewhere else with each write. I don't work for flash card manufacturer so I am not 100% sure on this but I guess there is no other sane way how to do it internally in the flash controller on the card. You need to handle same problems there (much bigger physical block in NAND chip than logical, pre-erasing blocks what data is written, remapping bad blocks, ...) like jffs2 I think that flash cards are like harddisks nowadays. You no longer care about physical geometry and bad blocks on current hard disks, all is done transparently by disk controller and you simply don't know and can't even guess where on the disk specific block is really stored. > > Ext2 and FAT don't do anything special. They also don't do anything over and > over again in the same spot. I think FAT table is the spot which is written over and over againg in most devices using flash cards. Write caching is not common for removable media so it may be even updated after doing every write operation to file. And if manufacturers are giving long/lifetime warranty despite this common usage I wouldn't bother with ext2 or ext3 which is typically mounted with write caching. Frantisek ___ maemo-users mailing list maemo-users@maemo.org https://lists.maemo.org/mailman/listinfo/maemo-users
Re: question on booting from mmc
On Friday 13 July 2007 00:14:09 Frantisek Dufka wrote: > Theodore Tso wrote: > > Yeah, but the ext3 journal wears out the flash card much more quickly. > > Since you know better than anyone how ext3 works can you quantify what > means 'much more quickly' with default date=ordered mode? 2x 10x 100x ? > > My only information is from > http://www.redhat.com/support/wpapers/redhat/ext3/ I would think the reference here is to lifetime of cell X of the SD card. Meaning here that each "block" (or whatever the correct term is) has a set # of wrtie/read events. JFFS from what I'm reading understands this and as such move journal information around just enough to even out this wear. My understanding is that jffs also takes steps to minimize the # of times and amount of reads/writes it does. ext3 however doesn't and it's hitting the HDD a lot more often. One of the reasons I'm not ext3's best fan.Given this you have even more of the reason jffs was invented. If all that was needed was compression cramfs would be used more often as it does compress greater and is supposedly more mature. Ext2 and FAT don't do anything special. They also don't do anything over and over again in the same spot. So FAT is a compromise in that "everyone can read FAT" since that FS is long since reverse engineered. EXT2 is the linux equal to FAT Older. Solid and a fallback that always works. My only Question with ext2 is can it do a partition larger than 2GB effectively, as for a long time this was one of it's big limits. James > > Also I suppose the basic IT200x system is somewhat optimized for writes > so typically the amount of data written is not exactly huge. We are also > mounting mmc with 'noatime'. > > > Given how cheap 2GB cards are, maybe we don't care > > Yes, they are cheap and I guess internal wear levelling should be pretty > good to allow 'lifetime' guarantee with FAT filesystem. > Check also (end of) https://bugs.maemo.org/show_bug.cgi?id=598 > > Also cards are getting bigger quickly so most people are not using the > card for long (like >3 years) but upgrade to bigger cards. > > > but given how > > quick e2fsck is on a 2GB flash disk, another approach would be modify > > the boot script so that it detects if the root filesystem is ext2, and > > if it is unclean, force an fsck automatically. > > This is slightly problematic. There is no fsck in initfs partition and > space is really tight there. Also we have no keyboard so realistically > -y is the only option which may sometimes do something wrong. From the > manpage: ... Sometimes an expert may be able to do better driving > the fsck manually. ... AUTHOR Theodore Tso > > Anyway, isn't mount with data=writeback (and possibly also commit=30) > same or better for data integrity (or in kernel fsck chance to do right > thing at boot time) than pure ext2 with e2fsck? > > Frantisek > ___ > maemo-users mailing list > maemo-users@maemo.org > https://lists.maemo.org/mailman/listinfo/maemo-users ___ maemo-users mailing list maemo-users@maemo.org https://lists.maemo.org/mailman/listinfo/maemo-users
Re: question on booting from mmc
Theodore Tso wrote: > It depends on how much metadata updates you do --- I would definitely > recommend using the noatime mount option, but probably somewhere > between 1.2x and 2x. Basically, each time you update some part of the > metadata, it will result in the block being written to the journal, > before it is ultimately written to the primary location on disk. So > the number of writes will go up by 2x for metadata blocks. So the big > question is the ratio between metadata updates and data block updates. Thanks. This doesn't sound so bad. This 1.2x to 2x is for default data=ordered mode and data=journal would be +1, right? What about the data=writeback, is this same or better (as for number of writes)? Frantisek ___ maemo-users mailing list maemo-users@maemo.org https://lists.maemo.org/mailman/listinfo/maemo-users
Re: question on booting from mmc
On Fri, Jul 13, 2007 at 09:14:09AM +0200, Frantisek Dufka wrote: > Theodore Tso wrote: > > >Yeah, but the ext3 journal wears out the flash card much more quickly. > > Since you know better than anyone how ext3 works can you quantify what > means 'much more quickly' with default date=ordered mode? 2x 10x 100x ? It depends on how much metadata updates you do --- I would definitely recommend using the noatime mount option, but probably somewhere between 1.2x and 2x. Basically, each time you update some part of the metadata, it will result in the block being written to the journal, before it is ultimately written to the primary location on disk. So the number of writes will go up by 2x for metadata blocks. So the big question is the ratio between metadata updates and data block updates. > This is slightly problematic. There is no fsck in initfs partition and > space is really tight there. Also we have no keyboard so realistically > -y is the only option which may sometimes do something wrong. From the > manpage: ... Sometimes an expert may be able to do better driving > the fsck manually. ... AUTHOR Theodore Tso Well, the initfs partition could mount the original jffs2 partition and grab the fsck from there. Or you could do what Linux systems did before initfs, which is to mount the root partition read-only, and then run e2fsck, and if the root filesystem was changed, force a reboot; else remount the root partition read/write. As far as e2fsck -y, 99% of the time if the machine just shutsdown uncleanly, the e2fsck -p (preen mode) will do the job just fine. If we do need to run e2fsck interactively, yeah, the only way to do that would be to boot back to the jffs root image, bring up an xterm, and then run it interactively from there. But that hopefully should be a relatively rare case. I'm assuming here that most of the time those people who are using the ext2/3 filesystem on an MMC is keeping a viable boot image in the original jffs2 device, so that can be used as the way to run e2fsck if necessary --- and as the source for the e2fsck binary from the initrd environment, if necessary. Regards, - Ted ___ maemo-users mailing list maemo-users@maemo.org https://lists.maemo.org/mailman/listinfo/maemo-users
Re: question on booting from mmc
Mmmh... I've waited for my birthday to have a gift of a 1GB MMC from my girlfriend ^__^;; I hope the system takes care of it as much as I do :) -- Antonio >> Given how cheap 2GB cards are, maybe we don't care ___ maemo-users mailing list maemo-users@maemo.org https://lists.maemo.org/mailman/listinfo/maemo-users
Re: question on booting from mmc
Theodore Tso wrote: > > Yeah, but the ext3 journal wears out the flash card much more quickly. Since you know better than anyone how ext3 works can you quantify what means 'much more quickly' with default date=ordered mode? 2x 10x 100x ? My only information is from http://www.redhat.com/support/wpapers/redhat/ext3/ Also I suppose the basic IT200x system is somewhat optimized for writes so typically the amount of data written is not exactly huge. We are also mounting mmc with 'noatime'. > Given how cheap 2GB cards are, maybe we don't care Yes, they are cheap and I guess internal wear levelling should be pretty good to allow 'lifetime' guarantee with FAT filesystem. Check also (end of) https://bugs.maemo.org/show_bug.cgi?id=598 Also cards are getting bigger quickly so most people are not using the card for long (like >3 years) but upgrade to bigger cards. > but given how > quick e2fsck is on a 2GB flash disk, another approach would be modify > the boot script so that it detects if the root filesystem is ext2, and > if it is unclean, force an fsck automatically. This is slightly problematic. There is no fsck in initfs partition and space is really tight there. Also we have no keyboard so realistically -y is the only option which may sometimes do something wrong. From the manpage: ... Sometimes an expert may be able to do better driving the fsck manually. ... AUTHOR Theodore Tso Anyway, isn't mount with data=writeback (and possibly also commit=30) same or better for data integrity (or in kernel fsck chance to do right thing at boot time) than pure ext2 with e2fsck? Frantisek ___ maemo-users mailing list maemo-users@maemo.org https://lists.maemo.org/mailman/listinfo/maemo-users
Re: question on booting from mmc
On Thu, Jul 12, 2007 at 09:41:04PM +0200, Frantisek Dufka wrote: > Also as mentioned in that thread (and in README and bootmenu > configuration examples inside initfs flasher), it may be better to use > ext3 not ext2 system as ext3 has similar journaling nature like jffs2 so > you don't need to run fsck when your system does not shutdown properly. Yeah, but the ext3 journal wears out the flash card much more quickly. Given how cheap 2GB cards are, maybe we don't care, but given how quick e2fsck is on a 2GB flash disk, another approach would be modify the boot script so that it detects if the root filesystem is ext2, and if it is unclean, force an fsck automatically. - Ted ___ maemo-users mailing list maemo-users@maemo.org https://lists.maemo.org/mailman/listinfo/maemo-users
Re: question on booting from mmc
Wahlau - wrote: > 1. does the card has to be put in the internet slot instead of the > external slot? i realise that i only can boot from mmc2 when i have > the card in the internet slot... perhaps i missed somethign here. Read the readme that comes inside initfs flasher. You can customize the boot menu and add any partition on any card. The external slot is named mmcblk1. However last time I checked there was some bug (in kernel?) and you needed to have card inserted in internal slot to have the card in external slot be named mmcblk1. With only one card (in external slot) the external slot was named mmcblk0 at boot time but oddly it was named as mmcblk1 after system booted. Maybe this is fixed in newer firmwares? > > 2. is it ok/possible that i make a backup of the ext2 partition (e.g > using dd and just tar it up as an image) so that i can "reflash" my > ext2 partition on the memory card everytime i want to return the > initial state of my dual boot? Yes, either keep such initial state in internal flash and just follow the cloning guide again or boot from flash and mount and backup ext2 partition via tar (perhaps to first fat partition). See the thread on ITT pointed by Antonio. > > 3. i realise now i cannot access the memory card via USB cable anymore > (make sense, since it booted now from the card, so i should not be > able to "detach it". But how about ways to access the FAT partition i > have on the same card? gone? i must reboot back to flash? i will now > resort to scp, which i assume it will work :) Yes, this is new issue in IT2007, solution here http://www.internettablettalk.com/forums/showthread.php?p=40839#post40839 Also as mentioned in that thread (and in README and bootmenu configuration examples inside initfs flasher), it may be better to use ext3 not ext2 system as ext3 has similar journaling nature like jffs2 so you don't need to run fsck when your system does not shutdown properly. Frantisek ___ maemo-users mailing list maemo-users@maemo.org https://lists.maemo.org/mailman/listinfo/maemo-users
Re: question on booting from mmc
Well, this is my very same question I posted here [1] (read the "newbie question" #1 at the end of the post). There were some interesting answers, e.g. this one [2], which I didn't tried for lack of time. Unfortunately no one still has been able to complete the page [3] that the "sebastian.linux" wrote on the wiki with the steps to perform an easy backup/restore under the dual boot setup, hopefully with the same straightforward and simple way of writing. The possibility of an easy backup/restore of the system is a natural consequence of that setup, so it would be appropriate in that wiki page, if someone could test/find a working way to achieve it. -- Antonio [1] http://www.internettablettalk.com/forums/showpost.php?p=35170&postcount=26 [2] http://www.internettablettalk.com/forums/showpost.php?p=35753&postcount=71 [3] http://maemo.org/community/wiki/howto_easily_boot_from_mmc_card/ > 2. is it ok/possible that i make a backup of the ext2 partition (e.g > using dd and just tar it up as an image) so that i can "reflash" my > ext2 partition on the memory card everytime i want to return the > initial state of my dual boot? ___ maemo-users mailing list maemo-users@maemo.org https://lists.maemo.org/mailman/listinfo/maemo-users
question on booting from mmc
Hallo all, i have followed the instruction on the wiki, and manage to now boot from ext2 on my 2GB mmc. However i have some questions on this approach... 1. does the card has to be put in the internet slot instead of the external slot? i realise that i only can boot from mmc2 when i have the card in the internet slot... perhaps i missed somethign here. 2. is it ok/possible that i make a backup of the ext2 partition (e.g using dd and just tar it up as an image) so that i can "reflash" my ext2 partition on the memory card everytime i want to return the initial state of my dual boot? 3. i realise now i cannot access the memory card via USB cable anymore (make sense, since it booted now from the card, so i should not be able to "detach it". But how about ways to access the FAT partition i have on the same card? gone? i must reboot back to flash? i will now resort to scp, which i assume it will work :) thanks. regards, wahlau. -- = : : : --- normal reply to [EMAIL PROTECTED] urgent reply to [EMAIL PROTECTED] --- : : : == ___ maemo-users mailing list maemo-users@maemo.org https://lists.maemo.org/mailman/listinfo/maemo-users
