Re: [CALL FOR TESTING] Make Ext3 fsck way faster [2.6.24-rc6 -mm patch]
No, it didn't. I measured read from a 10GB sequentially laid out file with standard benchmarking practices (cold cache, multiple runs, low std. deviation in results, etc.) and here are the results: File created by vanilla Ext3 being read by vanilla Ext3: Total: 3m16.1s User: 0.0.5s Sys: 13.9 File created by mc Ext3 being read by mc Ext3 (with the buffer boundary logic disabled): Total: 3m15.5s User: 0.05s Sys: 13.6s Thanks, Abhishek On Jan 24, 2008 2:49 AM, Andrew Morton [EMAIL PROTECTED] wrote: On Wed, 23 Jan 2008 04:12:16 -0500 Abhishek Rai [EMAIL PROTECTED] wrote: I'm wondering about the interaction between this code and the buffer_boundary() logic. I guess we should disable the buffer_boundary() handling when this code is in effect. Have you reviewed and tested that aspect? Thanks for pointing this out, I had totally missed this issue in my change. I've now made the call to set_buffer_boundary() in ext3_get_blocks_handle() subject to metacluster option being set. Did it make any performance difference? iirc the buffer_boundary stuff was worth around 10% on a single linear read of a large, well-laid-out file. - To unsubscribe from this list: send the line unsubscribe linux-ext4 in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [CALL FOR TESTING] Make Ext3 fsck way faster [2.6.24-rc6 -mm patch]
On Wed, 23 Jan 2008 04:12:16 -0500 Abhishek Rai [EMAIL PROTECTED] wrote: I'm wondering about the interaction between this code and the buffer_boundary() logic. I guess we should disable the buffer_boundary() handling when this code is in effect. Have you reviewed and tested that aspect? Thanks for pointing this out, I had totally missed this issue in my change. I've now made the call to set_buffer_boundary() in ext3_get_blocks_handle() subject to metacluster option being set. Did it make any performance difference? iirc the buffer_boundary stuff was worth around 10% on a single linear read of a large, well-laid-out file. - To unsubscribe from this list: send the line unsubscribe linux-ext4 in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [CALL FOR TESTING] Make Ext3 fsck way faster [2.6.24-rc6 -mm patch]
On Tuesday 15 January 2008 03:04, Andrew Morton wrote: I'm wondering about the real value of this change, really. In any decent environment, people will fsck their ext3 filesystems during planned downtime, and the benefit of reducing that downtime from 6 hours/machine to 2 hours/machine is probably fairly small, given that there is no service interruption. (The same applies to desktops and laptops). Sure, the benefit is not *zero*, but it's small. Much less than it would be with ext2. I mean, the avoid unplanned fscks feature is the whole reason why ext3 has journalling (and boy is that feature expensive during normal operation). So... it's unobvious that the benefit of this feature is worth its risks and costs? Since I am waiting for an Ext3 fsck to complete right now, I thought I would while away the time by tagging on my me too to the concensus that faster fsck is indeed worth the cost, which is (ahem) free. Regards, Daniel - To unsubscribe from this list: send the line unsubscribe linux-ext4 in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [CALL FOR TESTING] Make Ext3 fsck way faster [2.6.24-rc6 -mm patch]
On Thursday 17 January 2008 04:47, Abhishek Rai wrote: if Abhishek wants to pursue it, would be to pull in all of the indirect blocks when the file is opened, and create an in-memory extent tree that would speed up access to the file. It's rarely worth doing this without metaclustering, since it doesn't help for sequential I/O, only random I/O, but with metaclustering it would also be a win for sequential I/O. (This would also remove the minor performance degradation for sequential I/O imposed by metaclustering, and in fact improve it slightly for really big files.) Also, since the in memory extent tree will now occupy much less space, we can keep them cached for a much longer time which will improve performance of random reads. The new metaclustering patch is more amenable to this trick since it reduces fragmentation thereby reducing the number of extents. I can see value in preemptively loading indirect blocks into the buffer cache, but is building a second-order extent tree really worth the effort? Probing the buffer cache is very fast. Regards, Daniel - To unsubscribe from this list: send the line unsubscribe linux-ext4 in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [CALL FOR TESTING] Make Ext3 fsck way faster [2.6.24-rc6 -mm patch]
On Jan 15, 2008 23:25 -0500, [EMAIL PROTECTED] wrote: I've got multiple boxes across the hall that have 50T of disk on them, in one case as one large filesystem, and the users want *more* *bigger* still (damned researchers - you put a 15 teraflop supercomputer in the room, and then they want someplace to *put* all the numbers that come spewing out of there.. ;) There comes a point where that downtime gets too long to be politically expedient. 6-2 may not be a biggie, because you can likely get a 6 hour window. 24-8 suddenly looks a lot different. (Having said that, I'll admit the one 52T filesystem is an SGI Itanium box running Suse and using XFS rather than ext3). Has anybody done a back-of-envelope of what this would do for fsck times for a max realistically achievable ext3 filesystem (i.e. 100T-200T or ext3 design limit, whichever is smaller)? This is exactly the kind of environment that Lustre is designed for. Not only do you get parallel IO performance, but you can also do parallel e2fsck on the individual filesystems when you need it. Not that we aren't also working on improving e2fsck performance, but 100 * 4TB e2fsck in parallel is much better than 1 * 400TB e2fsck (probably not possible on a system today due to RAM constraints though I haven't really done any calculations either way). I know customers were having RAM problems with 3 * 2TB e2fsck in parallel on a 2GB node. Most customers these days use 2-4 4-8TB filesystems per server (inexpensive SMP node with 2-4GB RAM instead of a single monstrous SGI box). We have many Lustre filesystems in the 100-200TB range today, some over 1PB already and much larger ones being planned. (And one of the research crew had a not-totally-on-crack proposal to get a petabyte of spinning oxide. Figuring out how to back that up would probably have landed firmly in my lap. Ouch. ;) Charge them for 2PB of storage, and use rsync :-). Cheers, Andreas -- Andreas Dilger Sr. Staff Engineer, Lustre Group Sun Microsystems of Canada, Inc. - To unsubscribe from this list: send the line unsubscribe linux-ext4 in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [CALL FOR TESTING] Make Ext3 fsck way faster [2.6.24-rc6 -mm patch]
On Jan 15, 2008 10:28 AM, Theodore Tso [EMAIL PROTECTED] wrote: Also, it's not just reducing fsck times, although that's the main one. The last time this was suggested, the rationale was to speed up the rm dvd.iso case. Also, something which *could* be done, if Abhishek wants to pursue it, would be to pull in all of the indirect blocks when the file is opened, and create an in-memory extent tree that would speed up access to the file. It's rarely worth doing this without metaclustering, since it doesn't help for sequential I/O, only random I/O, but with metaclustering it would also be a win for sequential I/O. (This would also remove the minor performance degradation for sequential I/O imposed by metaclustering, and in fact improve it slightly for really big files.) - Ted Also, since the in memory extent tree will now occupy much less space, we can keep them cached for a much longer time which will improve performance of random reads. The new metaclustering patch is more amenable to this trick since it reduces fragmentation thereby reducing the number of extents. Thanks, Abhishek - To unsubscribe from this list: send the line unsubscribe linux-ext4 in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [CALL FOR TESTING] Make Ext3 fsck way faster [2.6.24-rc6 -mm patch]
I'm wondering about the real value of this change, really. In any decent environment, people will fsck their ext3 filesystems during planned downtime, and the benefit of reducing that downtime from 6 hours/machine to 2 hours/machine is probably fairly small, given that there is no service interruption. (The same applies to desktops and laptops). Sure, the benefit is not *zero*, but it's small. Much less than it would be with ext2. I mean, the avoid unplanned fscks feature is the whole reason why ext3 has journalling (and boy is that feature expensive during normal operation). So... it's unobvious that the benefit of this feature is worth its risks and costs? - To unsubscribe from this list: send the line unsubscribe linux-ext4 in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [CALL FOR TESTING] Make Ext3 fsck way faster [2.6.24-rc6 -mm patch]
On Tue, Jan 15, 2008 at 03:04:41AM -0800, Andrew Morton wrote: I'm wondering about the real value of this change, really. In any decent environment, people will fsck their ext3 filesystems during planned downtime, and the benefit of reducing that downtime from 6 hours/machine to 2 hours/machine is probably fairly small, given that there is no service interruption. (The same applies to desktops and laptops). Sure, the benefit is not *zero*, but it's small. Much less than it would be with ext2. I mean, the avoid unplanned fscks feature is the whole reason why ext3 has journalling (and boy is that feature expensive during normal operation). So... it's unobvious that the benefit of this feature is worth its risks and costs? They won't fsck in planned downtimes. They will have to use fsck when the shit hits the fan and they need to. Not sure about ext3, but big XFS user with a close tie to the US goverment were concerned about this case for really big filesystems and have sponsored speedup including multithreading xfs_repair. I'm pretty sure the same arguments apply to ext3, even if the filesystems are a few magnitudes smaller. - To unsubscribe from this list: send the line unsubscribe linux-ext4 in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html ---end quoted text--- - To unsubscribe from this list: send the line unsubscribe linux-ext4 in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [CALL FOR TESTING] Make Ext3 fsck way faster [2.6.24-rc6 -mm patch]
On Tue, Jan 15, 2008 at 01:15:33PM +, Christoph Hellwig wrote: They won't fsck in planned downtimes. They will have to use fsck when the shit hits the fan and they need to. Not sure about ext3, but big XFS user with a close tie to the US goverment were concerned about this case for really big filesystems and have sponsored speedup including multithreading xfs_repair. I'm pretty sure the same arguments apply to ext3, even if the filesystems are a few magnitudes smaller. And to add to that thanks to the not quite optimal default of peridocially checking that I alwasy forget to turn off on test machines an ext3 fsck speedup would be in my personal interested, and probably that of tons of developers :) - To unsubscribe from this list: send the line unsubscribe linux-ext4 in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [CALL FOR TESTING] Make Ext3 fsck way faster [2.6.24-rc6 -mm patch]
Andrew Morton wrote: I'm wondering about the real value of this change, really. In any decent environment, people will fsck their ext3 filesystems during planned downtime, and the benefit of reducing that downtime from 6 hours/machine to 2 hours/machine is probably fairly small, given that there is no service interruption. (The same applies to desktops and laptops). Sure, the benefit is not *zero*, but it's small. Much less than it would be with ext2. I mean, the avoid unplanned fscks feature is the whole reason why ext3 has journalling (and boy is that feature expensive during normal operation). So... it's unobvious that the benefit of this feature is worth its risks and costs? I actually think that the value of this kind of reduction is huge. We have seen fsck run for days (not just hours) which makes the restore from backup versus fsck decision favor the tapes... ric - To unsubscribe from this list: send the line unsubscribe linux-ext4 in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [CALL FOR TESTING] Make Ext3 fsck way faster [2.6.24-rc6 -mm patch]
On Tue, Jan 15, 2008 at 01:15:33PM +, Christoph Hellwig wrote: They won't fsck in planned downtimes. They will have to use fsck when the shit hits the fan and they need to. Not sure about ext3, but big XFS user with a close tie to the US goverment were concerned about this case for really big filesystems and have sponsored speedup including multithreading xfs_repair. I'm pretty sure the same arguments apply to ext3, even if the filesystems are a few magnitudes smaller. Agreed, 100%. Even if you fsck snapshots during slow periods, it still doesn't help you if the filesystem gets corrupted due to a hardware or software error. That's where this will matter the most. Val Hensen has done a proof of concept patch that multi-threads e2fsck (and she's working on one that would be long-term supportable) that might reduce the value of this patch, but metaclustering should still help. In any decent environment, people will fsck their ext3 filesystems during planned downtime, and the benefit of reducing that downtime from 6 hours/machine to 2 hours/machine is probably fairly small, given that there is no service interruption. (The same applies to desktops and laptops). Sure, the benefit is not *zero*, but it's small. Much less than it would be with ext2. I mean, the avoid unplanned fscks feature is the whole reason why ext3 has journalling (and boy is that feature expensive during normal operation). Also, it's not just reducing fsck times, although that's the main one. The last time this was suggested, the rationale was to speed up the rm dvd.iso case. Also, something which *could* be done, if Abhishek wants to pursue it, would be to pull in all of the indirect blocks when the file is opened, and create an in-memory extent tree that would speed up access to the file. It's rarely worth doing this without metaclustering, since it doesn't help for sequential I/O, only random I/O, but with metaclustering it would also be a win for sequential I/O. (This would also remove the minor performance degradation for sequential I/O imposed by metaclustering, and in fact improve it slightly for really big files.) - Ted - To unsubscribe from this list: send the line unsubscribe linux-ext4 in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [CALL FOR TESTING] Make Ext3 fsck way faster [2.6.24-rc6 -mm patch]
On Tue, 15 Jan 2008 10:09:16 EST, Ric Wheeler said: I actually think that the value of this kind of reduction is huge. We have seen fsck run for days (not just hours) which makes the restore from backup versus fsck decision favor the tapes... Funny thing is that for many of these sorts of cases, restore from backup is *also* a days issue unless you do a *lot* of very clever planning ahead to be able to get multiple tape drives moving at the same time while not causing issues at the receiving end either pgpTTyPj1liLj.pgp Description: PGP signature
