Re: [BackupPC-users] Best FS for BackupPC
Holger Parplies schrieb: Hi, Carl Wilhelm Soderstrom wrote on 2011-05-26 06:05:48 -0500 [Re: [BackupPC-users] Best FS for BackupPC]: On 05/26 12:20 , Adam Goryachev wrote: BTW, specifically related to backuppc, many years ago, reiserfsck was perfect as it doesn't have any concept or limit on 'inodes'... Same for mail and news (nntp) servers. Do XFS/JFS have this feature? I'll look into these things another day, when I have some time :) There are indeed 'inodes' listed in the 'df -i' output of XFS filesystems. However, I've never heard of anyone hitting the inode limit on XFS, unlike ext3. of course XFS *has* inodes, and I wondered about the 'df -i' output, too, when I tried it yesterday. I don't remember reiserfs giving any meaningful information for 'df -i' ... nope, '0 0 0 -'. I sincerely hope that XFS doesn't have *static inode allocation*, meaning I have to choose the number of inodes at file system creation time and waste any space I reserve for them but do not turn out to need. That was one main concern when choosing my pool FS. Actually, mkfs.xfs(8) explains a parameter '-i maxpct=value': This specifies the maximum percentage of space in the filesystem that can be allocated to inodes. The default value is 25% for filesystems under 1TB, 5% for filesystems under 50TB and 1% for filesystems over 50TB. The further explanation says this is achieved by the data block allocator avoiding lower blocks, which are needed for obtaining 32-bit inode numbers. It leaves two questions unanswered (to me, at least): ... have a look at the inode64 mount option. http://xfs.org/index.php/XFS_FAQ#Q:_What_is_the_inode64_mount_option_for.3F Ralf -- vRanger cuts backup time in half-while increasing security. With the market-leading solution for virtual backup and recovery, you get blazing-fast, flexible, and affordable data protection. Download your free trial now. http://p.sf.net/sfu/quest-d2dcopy1 ___ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net List:https://lists.sourceforge.net/lists/listinfo/backuppc-users Wiki:http://backuppc.wiki.sourceforge.net Project: http://backuppc.sourceforge.net/
Re: [BackupPC-users] Best FS for BackupPC
On 05/26 12:20 , Adam Goryachev wrote: BTW, specifically related to backuppc, many years ago, reiserfsck was perfect as it doesn't have any concept or limit on 'inodes'... Same for mail and news (nntp) servers. Do XFS/JFS have this feature? I'll look into these things another day, when I have some time :) There are indeed 'inodes' listed in the 'df -i' output of XFS filesystems. However, I've never heard of anyone hitting the inode limit on XFS, unlike ext3. $ mount /dev/sda1 on / type ext3 (rw,errors=remount-ro) snip /dev/sda3 on /var/lib/backuppc type xfs (rw,noatime) $ df -i FilesystemInodes IUsed IFree IUse% Mounted on /dev/sda11969920 54048 19158723% / snip /dev/sda3383991360 6330684 3776606762% /var/lib/backuppc -- Carl Soderstrom Systems Administrator Real-Time Enterprises www.real-time.com -- vRanger cuts backup time in half-while increasing security. With the market-leading solution for virtual backup and recovery, you get blazing-fast, flexible, and affordable data protection. Download your free trial now. http://p.sf.net/sfu/quest-d2dcopy1 ___ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net List:https://lists.sourceforge.net/lists/listinfo/backuppc-users Wiki:http://backuppc.wiki.sourceforge.net Project: http://backuppc.sourceforge.net/
Re: [BackupPC-users] Best FS for BackupPC
Hi, Carl Wilhelm Soderstrom wrote on 2011-05-26 06:05:48 -0500 [Re: [BackupPC-users] Best FS for BackupPC]: On 05/26 12:20 , Adam Goryachev wrote: BTW, specifically related to backuppc, many years ago, reiserfsck was perfect as it doesn't have any concept or limit on 'inodes'... Same for mail and news (nntp) servers. Do XFS/JFS have this feature? I'll look into these things another day, when I have some time :) There are indeed 'inodes' listed in the 'df -i' output of XFS filesystems. However, I've never heard of anyone hitting the inode limit on XFS, unlike ext3. of course XFS *has* inodes, and I wondered about the 'df -i' output, too, when I tried it yesterday. I don't remember reiserfs giving any meaningful information for 'df -i' ... nope, '0 0 0 -'. I sincerely hope that XFS doesn't have *static inode allocation*, meaning I have to choose the number of inodes at file system creation time and waste any space I reserve for them but do not turn out to need. That was one main concern when choosing my pool FS. Actually, mkfs.xfs(8) explains a parameter '-i maxpct=value': This specifies the maximum percentage of space in the filesystem that can be allocated to inodes. The default value is 25% for filesystems under 1TB, 5% for filesystems under 50TB and 1% for filesystems over 50TB. The further explanation says this is achieved by the data block allocator avoiding lower blocks, which are needed for obtaining 32-bit inode numbers. It leaves two questions unanswered (to me, at least): 1.) Is this a hard limit, or will inodes continue to be allocated in excess of this percentage, (a) if more space happens to be free in the lower blocks, or (b) generating inode numbers exceeding 32 bits, provided the kernel supports them (probably only 64-bit kernels)? 2.) Will the data block allocator use these blocks up once no other blocks are available any more, or is your XFS full, even though you've got another 249GB(!) free on your 1TB FS, that are reserved for inodes? The answer to (2) is most likely the data block allocator will use them, because the man page goes on: Setting the value to 0 means that essentially all of the filesystem can become inode blocks, subject to inode32 restrictions. (however, it could be a special case for the value 0). In fact, the very concept of allocating inodes rather than reserving fixed blocks for them strongly suggests some flexibility in deciding how much space will be used for them and how much for data. In any case, the default percentage seems to allow for far more inodes than with ext[23], which might explain why you hit the boundary later (if at all). Regards, Holger -- vRanger cuts backup time in half-while increasing security. With the market-leading solution for virtual backup and recovery, you get blazing-fast, flexible, and affordable data protection. Download your free trial now. http://p.sf.net/sfu/quest-d2dcopy1 ___ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net List:https://lists.sourceforge.net/lists/listinfo/backuppc-users Wiki:http://backuppc.wiki.sourceforge.net Project: http://backuppc.sourceforge.net/
Re: [BackupPC-users] Best FS for BackupPC
On 24/05/2011 11:25 PM, Michael Stowe wrote: I did a relatively short filesystem comparison when I moved my BackupPC pool to another set of drives. The high level results: jfs, xfs: quick, stable reiserfs: not stable ext4: slow ext3: very slow The not stable designation comes from power-off-during-write tests. Other filesystems generally handled this gracefully, but reiserfs corrupted the entire tree, and the recovery tools didn't get it back intact. Just a couple of my own personal comments on reiserfs: 1) It does usually handle random power-offs on both general servers and backuppc based servers. Usually doesn't really do it for me. The problem seems to be in the structure of the trees and the rebuild tree routines, which just grabs every block that looks like they're reiserfs tree blocks. 2) It does sometimes have problems resurrecting the filesystem when it has been corrupted, I did lose *one* home directory out of 400 once upon a time (about 9 years ago...) Like I said, a filesystem that loses data *sometimes* doesn't really make it to the top of the list, in favor of filesystems that ... don't have this problem. 3) I've used reiserfs on both file servers and backuppc servers for quite a long time (and also desktops until very recently) with no problems that I wouldn't expect from any other FS. One backuppc server I used it with never expired any backup, and did daily backups of about 5 servers with a total of 700G data. This was working fine for over 5 years (turned off recently due to company issues, not technical). There are plenty of things that run perfectly well when unstressed. For example, 32-bit zfs runs *perfectly* well, unless you try to rsync the whole filesystem... in which case it panics the kernel. I would expect that any FS will *sometimes* have a problem fixing it's FS after a power loss unless you use journally on the data as well as the FS info. Perhaps in your testing you either didn't enable the correct journalling options, or found that particular corner case. Perhaps next time it happens jfs/xfs might hit their corner cases. This doesn't ring true nor does it match the results of my testing. I didn't tune any file systems. xfs and jfs were resilient to simple power fail situations, reiserfs was not. You can speculate that xfs and jfs may contain the same flaws but some kind of blind luck kept them working properly, but it seems *really* unlikely. Further, simply running a filesystem is not the same as testing and recovering it. It's certainly possible to have run a FAT filesystem under Windows 3.1 for 20 years. This doesn't make it a robust choice. My understanding of reiserfs development is that it is stable, and being in the linux tree, is maintained. Stable development and being a stable filesystem aren't the same thing, naturally. However, while I liked reiserfs a lot, I've recently found that support for it is declining (can't even select it as a FS option when installing some new OS's), and that other FS's offer a lot of the same performance features, thus making reiserfs somewhat obsolete. It would be nice to see some real performance benchmarks with reiserfs and jfx/xfs but I can't really be bothered, and probably neither is anyone else. For my part, I didn't bother because it didn't pass the stable hurdle. It doesn't matter how fast it is if it can't preserve the data intact. Frankly, it was the only FS that didn't, which included ext3 -- again, I'll stress the fact that I actually tested them, I didn't just trust that everything was intact. I expect reiserfs will eventually go away, and as such I'm migrating away from it as my systems are retired/etc (but it will be in use for a long time as it isn't easy to format and restore or migrate large amounts of data...) I don't mean to disparage xfs/jfs or any testing anybody has done, just wanted to share my personal experiences. Since you don't appear to be arguing that people actually use reiserfs, you're speculating that xfs/jfs contain flaws without any apparent evidence, and your personal experiences don't appear to include testing, I'm not really sure where you're going with this. Due to time constraints, I didn't do a great many trials, but I'll give you an idea of what my testing entailed: On a 7-drive array (5+2 RAID), make a new filesytem, and point the BackupPC pool at it. A script runs that uses X10 to physically power down the box once it senses the presence of a test file. This script is used for both the initial backup and the link phase, both of which are restarted once during the trial. (This script was originally timed to power down one hour into the backup, but since the backups ran at different speeds on the filesystems, it seemed likely that the backups would be at different points. To avoid corner cases, they were powered down while backing up the exact same file.) The pool is then compared (via rsync) to the
Re: [BackupPC-users] Best FS for BackupPC
On 5/25/2011 8:40 AM, Michael Stowe wrote: 2) It does sometimes have problems resurrecting the filesystem when it has been corrupted, I did lose *one* home directory out of 400 once upon a time (about 9 years ago...) Like I said, a filesystem that loses data *sometimes* doesn't really make it to the top of the list, in favor of filesystems that ... don't have this problem. To be fair, those sometimes are crash situations and it is also a good idea to run hardware/operating systems/UPS's, that... don't cause that problem. And to have an offsite copy for the things you can't control. -- Les Mikesell lesmikes...@gmail.com -- vRanger cuts backup time in half-while increasing security. With the market-leading solution for virtual backup and recovery, you get blazing-fast, flexible, and affordable data protection. Download your free trial now. http://p.sf.net/sfu/quest-d2dcopy1 ___ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net List:https://lists.sourceforge.net/lists/listinfo/backuppc-users Wiki:http://backuppc.wiki.sourceforge.net Project: http://backuppc.sourceforge.net/
Re: [BackupPC-users] Best FS for BackupPC
On 5/25/2011 8:40 AM, Michael Stowe wrote: 2) It does sometimes have problems resurrecting the filesystem when it has been corrupted, I did lose *one* home directory out of 400 once upon a time (about 9 years ago...) Like I said, a filesystem that loses data *sometimes* doesn't really make it to the top of the list, in favor of filesystems that ... don't have this problem. To be fair, those sometimes are crash situations and it is also a good idea to run hardware/operating systems/UPS's, that... don't cause that problem. And to have an offsite copy for the things you can't control. I'm not arguing against either one -- though given a choice between a filesystem that will go corrupt if the UPS fails and one that doesn't, I'd rather go with the one that doesn't, whether I expect to experience a power failure or not. I've had UPS batteries blow and power rails burn out in servers before, as well as had electricians accidentally short out an entire data center, so I'm not inclined to believe a design flaw in a filesystem can be covered up by a UPS. (Neither are you, obviously.) In this case, the design flaw appears to be reiserfs's storage of directory entries, et alia, in one massive B+ tree combined with non-synchronous directory operations, combined with BackupPC's propensity to perform a LOT of directory operations. Basically, I'm saying that I believe, based on my testing, that reiserfs is a very poor choice of filesystem for BackupPC. I'm not suggesting that it goes corrupt with normal, continuous use, or that if nothing goes horribly wrong, it won't be fine. I'm suggesting it's a really bad idea to have to count on it. -- vRanger cuts backup time in half-while increasing security. With the market-leading solution for virtual backup and recovery, you get blazing-fast, flexible, and affordable data protection. Download your free trial now. http://p.sf.net/sfu/quest-d2dcopy1 ___ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net List:https://lists.sourceforge.net/lists/listinfo/backuppc-users Wiki:http://backuppc.wiki.sourceforge.net Project: http://backuppc.sourceforge.net/
Re: [BackupPC-users] Best FS for BackupPC
On 5/25/2011 10:12 AM, Michael Stowe wrote: To be fair, those sometimes are crash situations and it is also a good idea to run hardware/operating systems/UPS's, that... don't cause that problem. And to have an offsite copy for the things you can't control. I'm not arguing against either one -- though given a choice between a filesystem that will go corrupt if the UPS fails and one that doesn't, I'd rather go with the one that doesn't, whether I expect to experience a power failure or not. And I'm not really arguing for reiserfs now that there are other reasonable choices. But, I did run it on several systems, including for backuppc for several years back when the other choice was ext2 where any unexpected restart would take a day of fsck operations before you could do anything and in that time I saw several crashes where journal replay too care of everything. I've had UPS batteries blow and power rails burn out in servers before, as well as had electricians accidentally short out an entire data center, so I'm not inclined to believe a design flaw in a filesystem can be covered up by a UPS. (Neither are you, obviously.) Sure - you have to play the odds here and make them better any way you can, but I wouldn't put reiserfs corruption at the top of the list of things that can make my disks unreadable. Basically, I'm saying that I believe, based on my testing, that reiserfs is a very poor choice of filesystem for BackupPC. I'm not suggesting that it goes corrupt with normal, continuous use, or that if nothing goes horribly wrong, it won't be fine. I'm suggesting it's a really bad idea to have to count on it. That's all true, but basically putting data on physical disk drives is a bad idea in the first place because any number of things can go wrong. If you are prepared for your disks to melt, you probably won't have a big problem with the relatively unlikely scenario of filesystem corruption. -- Les Mikesell lesmikes...@gmail.com -- vRanger cuts backup time in half-while increasing security. With the market-leading solution for virtual backup and recovery, you get blazing-fast, flexible, and affordable data protection. Download your free trial now. http://p.sf.net/sfu/quest-d2dcopy1 ___ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net List:https://lists.sourceforge.net/lists/listinfo/backuppc-users Wiki:http://backuppc.wiki.sourceforge.net Project: http://backuppc.sourceforge.net/
Re: [BackupPC-users] Best FS for BackupPC
Hi, first of all, my personal experience with reiserfs is also that it lost a complete pool FS (apparently, the cpool directory disappeared and was re-created by BackupPC *several times* before I noticed the problem). Rebuilding the tree obviously gave me a state that is next to impossible to fix properly (lots of directories in lost+found named by inode - any volunteers for finding out, where and within which pc/ directory to put them? ;-), let alone verify the results. My decision was to move to a different FS. I didn't go the scientific way, I just chose xfs, which apparently was a good choice - at least up to now. So I certainly don't disagree with your results, but I do partly disagree with your reasoning and interpretations. Michael Stowe wrote on 2011-05-25 08:40:10 -0500 [Re: [BackupPC-users] Best FS for BackupPC]: [Adam wrote:] On 24/05/2011 11:25 PM, Michael Stowe wrote: [...] The high level results: jfs, xfs: quick, stable reiserfs: not stable ext4: slow ext3: very slow While that is a nice summary, I, personally, wouldn't base any decisions solely on a summary without having any idea how the results were obtained, because the methods could be flawed or simply not take my main concerns into account (e.g. if I have my BackupPC server on a UPS, power loss is not my primary concern (though it may still be one); long term stability is). For other people, speed may be vital, while the ability to survive a power failure is not. You explain in a followup (see below) how you obtained your results. The not stable designation comes from power-off-during-write tests. [...] Just a couple of my own personal comments on reiserfs: 1) It does usually handle random power-offs on both general servers and backuppc based servers. Usually doesn't really do it for me. I believe that is exactly the point. You simply can't *test* whether a file system handles *every* power-off case correctly. You can prove that it doesn't, or you can find that you didn't manage to trigger any problems. So, while I agree with reiserfs does *not* handle power-offs sufficiently well, I don't see it as *proven* that xfs/jfs/ext4/ext3 are any better. They might be better, they might be worse. They are *probably* better, but that is just speculation. Granted, I'd prefer an FS where I didn't manage to trigger any problems over one where I did, too. Or one, where the majority of the community seems to agree that it performs better. However, both choices are based on experience, not on scientific results. The problem seems to be in the structure of the trees and the rebuild tree routines, which just grabs every block that looks like they're reiserfs tree blocks. If that is the case, it is certainly problematic. What I also dislike is that 'reiserfsck --rebuild-tree' leaves your FS in an unusable state until it has completed - let's hope it does complete. All other 'fsck' programs I can remember having used seem to operate in an incremental way - fixing problems without causing new ones (except maybe trivial wrong count type inconsistencies), so they can [mostly] be interrupted without making the situation worse than it was. 3) I've used reiserfs on both file servers and backuppc servers for quite a long time [...] One backuppc server I used it with [...] did daily backups of about 5 servers with a total of 700G data. [...] There are plenty of things that run perfectly well when unstressed. What is your understanding of unstressed? Perhaps in your testing you either didn't enable the correct journalling options, or found that particular corner case. Perhaps next time it happens jfs/xfs might hit their corner cases. This doesn't ring true nor does it match the results of my testing. I didn't tune any file systems. Perhaps you should have. The default options are not always suitable for obtaining what you need. In what way doesn't next time jfs/xfs might hit their corner cases match the results of your testing? As I said, I don't believe you've proven that jfs/xfs don't *have* corner cases. You just didn't expose any. You can speculate that xfs and jfs may contain the same flaws but some kind of blind luck kept them working properly, but it seems *really* unlikely. The speculation is, that you didn't test the situations that xfs or jfs might have problems with (and reiserfs might handle perfectly). Further, simply running a filesystem is not the same as testing and recovering it. It's certainly possible to have run a FAT filesystem under Windows 3.1 for 20 years. This doesn't make it a robust choice. Certainly true. But all I can see here are different data points from different people's *experience*. You're unlikely to experience running *dozens* of FAT/Win3.1 file systems for 20 years, and if you do, it might well be a robust choice *for your usage pattern*. That doesn't mean it will work equally well with different usage patterns, or that if you suddenly
Re: [BackupPC-users] Best FS for BackupPC
So I certainly don't disagree with your results, but I do partly disagree
with your reasoning and interpretations.
Err, actually, you don't ... or perhaps more accurately, I don't disagree
with any of the points you make, so rather than agree with everything you
said individually, I'll skip ahead.
If that is the case, it is certainly problematic. What I also dislike is
that 'reiserfsck --rebuild-tree' leaves your FS in an unusable state
until it has completed - let's hope it does complete. All other 'fsck'
programs I can remember having used seem to operate in an incremental
way - fixing problems without causing new ones (except maybe trivial
wrong count type inconsistencies), so they can [mostly] be interrupted
without making the situation worse than it was.
While trying to figure out why reiserfs had gone corrupt, I tested out a
scenario where backing up a reiserfs image via BackupPC (without
compression) would be interpreted as part of the fs by --rebuild-tree, and
hopelessly mangled all data on the disk.
Probably not exactly fair to reiserfs, but it does bother me that backing
up certain types of data could make other corruption unrecoverable.
What is your understanding of unstressed?
Without pushing its limits -- depending on the fs, these can be in
different places. None of the file systems melted down when simply
subjected to high amounts of I/O. (Well, zfs did, but that's different.)
The speculation is, that you didn't test the situations that xfs or jfs
might have problems with (and reiserfs might handle perfectly).
Which is reasonable enough, and I'm open to finding out if there are any.
Certainly true. But all I can see here are different data points from
different people's *experience*. You're unlikely to experience running
*dozens* of FAT/Win3.1 file systems for 20 years, and if you do, it might
well be a robust choice *for your usage pattern*. That doesn't mean it
will work equally well with different usage patterns, or that if you
suddenly do encounter corruption, a different FS wouldn't be better
recoverable.
I'm really suggesting that the experience of somebody who has run a file
system for a period of time without (for example) a power failure is
likely to have little to contribute to answer the question on how stable a
file system is during a power failure.
The testing I did has a natural bias toward the scenarios I wanted to
gather data on, and my specific question was stability and speed while
using BackupPC on software RAID, were there distinctions between
filesystems?
In this regard, reiserfs failed miserably, and perhaps unfairly, part of
the reason I tested in the way I did was due to problems I'd experienced
in the past with reiserfs. So unless there's a really compelling reason
TO use reiserfs that somehow overrides the corruption issue, I (for one)
am pretty satisfied in ruling it out.
This is a good example of how hardware may corrupt your FS (or prevent
corruption that would occur with different hardware). If you are truely
interested in testing *the file systems*, you should not introduce the
extra complexity of RAID 6. You were probably more interested in testing
*how the file systems would operate in your hardware environment*.
That
is a difference.
Quite so, and I also made the implicit assumption that what the fs sits on
doesn't really matter, which may or may not be the case.
More or less. You'll have different timestamps in log files, a random
difference in timing (length of the file in progress) ... I'm just
wondering what exactly you are comparing. pool means $TopDir or
$TopDir/{c,}pool or $TopDir/pc?
I actually used FUSE to do a straight compare; since the test box was
quiescent (I eliminated any files that were not) there was a 100% match in
most cases.
What your test doesn't catch is long term stability. In the absense of
power
failures, will your FS operate well over many years? I've heard (rumours,
not
real data points) that reiserfs will operate smoothly up to the point
where
accumulated internal inconsistency (presumably due to bugs) exceeds a
certain
amount, and then it will destroy just about all of your file system. That
might even match my observation - I don't remember whether there was a
power
failure involved or not. I have no long-term first-hand experience with
xfs
(or jfs). Does anyone else?
I've run BackupPC on jfs for a few years now, and it has proven to be
rock-solid. I've run xfs (but not under BackupPC) which has been
similarly trouble-free.
As file systems go, I can recommend jfs, which you can mark down as a
single anecdotal data point.
Regards,
Holger
--
vRanger cuts backup time in half-while increasing security.
With the market-leading solution for virtual backup and recovery,
you get blazing-fast, flexible, and affordable data protection.
Download your free trial now.
Re: [BackupPC-users] Best FS for BackupPC
On 5/25/2011 1:46 PM, Michael Stowe wrote: I'm really suggesting that the experience of somebody who has run a file system for a period of time without (for example) a power failure is likely to have little to contribute to answer the question on how stable a file system is during a power failure. If I quit using every filesystem type where I have seen data lost, I probably wouldn't have a computer any more... Just assume that there is some small risk to every copy of everything. And that risk will change in unpredictable ways with future OS updates too. -- Les Mikesell lesmikes...@gmail.com -- vRanger cuts backup time in half-while increasing security. With the market-leading solution for virtual backup and recovery, you get blazing-fast, flexible, and affordable data protection. Download your free trial now. http://p.sf.net/sfu/quest-d2dcopy1 ___ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net List:https://lists.sourceforge.net/lists/listinfo/backuppc-users Wiki:http://backuppc.wiki.sourceforge.net Project: http://backuppc.sourceforge.net/
Re: [BackupPC-users] Best FS for BackupPC
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 My only other comment I forgot to make in my original response, were the same options enabled for each of the tested filesystems. Some filesystems by default enable data and metadata journalling, while others require an option to enable this (AFAIK). In any case, it sounds like everybody (including me) pretty much agrees that there probably aren't any good reasons to use reiserfs in a new installation today. BTW, specifically related to backuppc, many years ago, reiserfsck was perfect as it doesn't have any concept or limit on 'inodes'... Same for mail and news (nntp) servers. Do XFS/JFS have this feature? I'll look into these things another day, when I have some time :) Regards, Adam - -- Adam Goryachev Website Managers www.websitemanagers.com.au -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.11 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iEYEARECAAYFAk3duPAACgkQGyoxogrTyiUL9wCgwpJ5hUwztGKg/suklk9kNZqA LUkAmwU2ESMgAgJ16TG/FTLgDmYnZ7Pf =yy1r -END PGP SIGNATURE- -- vRanger cuts backup time in half-while increasing security. With the market-leading solution for virtual backup and recovery, you get blazing-fast, flexible, and affordable data protection. Download your free trial now. http://p.sf.net/sfu/quest-d2dcopy1 ___ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net List:https://lists.sourceforge.net/lists/listinfo/backuppc-users Wiki:http://backuppc.wiki.sourceforge.net Project: http://backuppc.sourceforge.net/
Re: [BackupPC-users] Best FS for BackupPC
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 26/05/11 04:59, Les Mikesell wrote: On 5/25/2011 1:46 PM, Michael Stowe wrote: I'm really suggesting that the experience of somebody who has run a file system for a period of time without (for example) a power failure is likely to have little to contribute to answer the question on how stable a file system is during a power failure. If I quit using every filesystem type where I have seen data lost, I probably wouldn't have a computer any more... Just assume that there is some small risk to every copy of everything. And that risk will change in unpredictable ways with future OS updates too. As does the performance of those FS change with each new OS version... Regards, Adam - -- Adam Goryachev Website Managers www.websitemanagers.com.au -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.11 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iEYEARECAAYFAk3duzAACgkQGyoxogrTyiUu2gCdF6Z15eOwFukYT1mvDKBH3HuZ 7YwAnR9SXxYcGdXzjr1IAil2YlsPWjrB =iT5D -END PGP SIGNATURE- -- vRanger cuts backup time in half-while increasing security. With the market-leading solution for virtual backup and recovery, you get blazing-fast, flexible, and affordable data protection. Download your free trial now. http://p.sf.net/sfu/quest-d2dcopy1 ___ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net List:https://lists.sourceforge.net/lists/listinfo/backuppc-users Wiki:http://backuppc.wiki.sourceforge.net Project: http://backuppc.sourceforge.net/
[BackupPC-users] Best FS for BackupPC
Hi everyone, I'm doing some benchmarks with BackupPC and I wanted to ask here about the filesystems you are using and why. Which one do you think is best for BackupPC? I saw on the documentation that some users found Reiser is better than ext3: [docu] Several users have reported significantly better performance using reiserfs compared to ext3 for the BackupPC data file system [/docu] But I doubt to use ReiserFS as it is no longer being developed or updated and I do not know if it's a FS mature enough for production. Thanks in advance. Regards. -- Marcos Lorenzo de Santiago System administrator marcos.lore...@andago.com ÁNDAGO INGENIERÍA Tlf: +34 916 011 373 Álcalde Ángel Arroyo, 10, 1º Mvl: +34 637 741 034 28904, Getafe, Madrid (Spain) Fax: +34 916 011 372 www.andago.com --- Dios no juega a los dados con el universo. -- Albert Einstein - vRanger cuts backup time in half-while increasing security. With the market-leading solution for virtual backup and recovery, you get blazing-fast, flexible, and affordable data protection. Download your free trial now. http://p.sf.net/sfu/quest-d2dcopy1___ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net List:https://lists.sourceforge.net/lists/listinfo/backuppc-users Wiki:http://backuppc.wiki.sourceforge.net Project: http://backuppc.sourceforge.net/
Re: [BackupPC-users] Best FS for BackupPC
-Original Message- From: Marcos Lorenzo de Santiago [mailto:mlore...@andago.com] Sent: Tuesday, May 24, 2011 11:53 AM To: BackupPC-users@lists.sourceforge.net Subject: [BackupPC-users] Best FS for BackupPC I'm doing some benchmarks with BackupPC and I wanted to ask here about the filesystems you are using and why. Which one do you think is best for BackupPC? When I installed a new BackupPC-server I opted to use ext4, as it at that time came out-of-the-box with CentOS 5.6. So far, I believe it's performing much better than ext3 did previously on the older BackupPC-server. Writes and fsck's run way faster is my subjective feeling. -- /Sorin smime.p7s Description: S/MIME cryptographic signature -- vRanger cuts backup time in half-while increasing security. With the market-leading solution for virtual backup and recovery, you get blazing-fast, flexible, and affordable data protection. Download your free trial now. http://p.sf.net/sfu/quest-d2dcopy1___ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net List:https://lists.sourceforge.net/lists/listinfo/backuppc-users Wiki:http://backuppc.wiki.sourceforge.net Project: http://backuppc.sourceforge.net/
Re: [BackupPC-users] Best FS for BackupPC
I did a relatively short filesystem comparison when I moved my BackupPC pool to another set of drives. The high level results: jfs, xfs: quick, stable reiserfs: not stable ext4: slow ext3: very slow The not stable designation comes from power-off-during-write tests. Other filesystems generally handled this gracefully, but reiserfs corrupted the entire tree, and the recovery tools didn't get it back intact. Hi everyone, I'm doing some benchmarks with BackupPC and I wanted to ask here about the filesystems you are using and why. Which one do you think is best for BackupPC? I saw on the documentation that some users found Reiser is better than ext3: [docu] Several users have reported significantly better performance using reiserfs compared to ext3 for the BackupPC data file system [/docu] But I doubt to use ReiserFS as it is no longer being developed or updated and I do not know if it's a FS mature enough for production. Thanks in advance. Regards. -- Marcos Lorenzo de Santiago System administrator marcos.lore...@andago.com ÁNDAGO INGENIERÍA Tlf: +34 916 011 373 Álcalde Ángel Arroyo, 10, 1º Mvl: +34 637 741 034 28904, Getafe, Madrid (Spain) Fax: +34 916 011 372 www.andago.com --- Dios no juega a los dados con el universo. -- Albert Einstein -- vRanger cuts backup time in half-while increasing security. With the market-leading solution for virtual backup and recovery, you get blazing-fast, flexible, and affordable data protection. Download your free trial now. http://p.sf.net/sfu/quest-d2dcopy1 ___ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net List:https://lists.sourceforge.net/lists/listinfo/backuppc-users Wiki:http://backuppc.wiki.sourceforge.net Project: http://backuppc.sourceforge.net/
Re: [BackupPC-users] Best FS for BackupPC
Michael Stowe wrote at about 08:25:35 -0500 on Tuesday, May 24, 2011: I did a relatively short filesystem comparison when I moved my BackupPC pool to another set of drives. The high level results: jfs, xfs: quick, stable reiserfs: not stable ext4: slow ext3: very slow Any idea which particular file operations are causing ext3/4 to be slow relative to jfs/xfs? Is it reading? writing? linking? listing? all of the above equally? -- vRanger cuts backup time in half-while increasing security. With the market-leading solution for virtual backup and recovery, you get blazing-fast, flexible, and affordable data protection. Download your free trial now. http://p.sf.net/sfu/quest-d2dcopy1 ___ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net List:https://lists.sourceforge.net/lists/listinfo/backuppc-users Wiki:http://backuppc.wiki.sourceforge.net Project: http://backuppc.sourceforge.net/
Re: [BackupPC-users] Best FS for BackupPC
On Tue, May 24, 2011 at 5:52 AM, Marcos Lorenzo de Santiago mlore...@andago.com wrote: Which one do you think is best for BackupPC? One of the main reasons I like xfs is the tool xfs_copy. This will allow you to copy a backuppc pool filesystem quickly and efficiently. BackupPC pools have too many hardlinks to make rsync work efficiently. dd works, but copies all of the empty space too. -- Tod -- vRanger cuts backup time in half-while increasing security. With the market-leading solution for virtual backup and recovery, you get blazing-fast, flexible, and affordable data protection. Download your free trial now. http://p.sf.net/sfu/quest-d2dcopy1 ___ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net List:https://lists.sourceforge.net/lists/listinfo/backuppc-users Wiki:http://backuppc.wiki.sourceforge.net Project: http://backuppc.sourceforge.net/
Re: [BackupPC-users] Best FS for BackupPC
-Original Message- From: Jeffrey J. Kosowsky [mailto:backu...@kosowsky.org] Sent: Tuesday, May 24, 2011 3:59 PM To: mst...@chicago.us.mensa.org; General list for user discussion, questions and support Subject: Re: [BackupPC-users] Best FS for BackupPC Any idea which particular file operations are causing ext3/4 to be slow relative to jfs/xfs? Is it reading? writing? linking? listing? all of the above equally? I found this quick and dirty comparison of file systems. Can it be of some use for you? http://maketecheasier.com/choosing-the-best-linux-filesystem/2010/04/13/ -- /Sorin smime.p7s Description: S/MIME cryptographic signature -- vRanger cuts backup time in half-while increasing security. With the market-leading solution for virtual backup and recovery, you get blazing-fast, flexible, and affordable data protection. Download your free trial now. http://p.sf.net/sfu/quest-d2dcopy1___ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net List:https://lists.sourceforge.net/lists/listinfo/backuppc-users Wiki:http://backuppc.wiki.sourceforge.net Project: http://backuppc.sourceforge.net/
Re: [BackupPC-users] Best FS for BackupPC
Tod Detre tod.de...@maine.edu wrote on 05/24/2011 09:48:07 AM: On Tue, May 24, 2011 at 5:52 AM, Marcos Lorenzo de Santiago mlore...@andago.com wrote: Which one do you think is best for BackupPC? One of the main reasons I like xfs is the tool xfs_copy. This will allow you to copy a backuppc pool filesystem quickly and efficiently. BackupPC pools have too many hardlinks to make rsync work efficiently. dd works, but copies all of the empty space too. FWIW, partclone is like xfs_copy for all sorts of filesystems: it understands them well enough to copy them without copying empty space. Timothy J. Massey Out of the Box Solutions, Inc. Creative IT Solutions Made Simple! http://www.OutOfTheBoxSolutions.com tmas...@obscorp.com 22108 Harper Ave. St. Clair Shores, MI 48080 Office: (800)750-4OBS (4627) Cell: (586)945-8796 -- vRanger cuts backup time in half-while increasing security. With the market-leading solution for virtual backup and recovery, you get blazing-fast, flexible, and affordable data protection. Download your free trial now. http://p.sf.net/sfu/quest-d2dcopy1___ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net List:https://lists.sourceforge.net/lists/listinfo/backuppc-users Wiki:http://backuppc.wiki.sourceforge.net Project: http://backuppc.sourceforge.net/
Re: [BackupPC-users] Best FS for BackupPC
yTod Detre tod.de...@maine.edu wrote [9:48am -0400] TD On Tue, May 24, 2011 at 5:52 AM, Marcos Lorenzo de Santiago TD mlore...@andago.com wrote: TD TD Which one do you think is best for BackupPC? TD TD One of the main reasons I like xfs is the tool xfs_copy. This will TD allow you to copy a backuppc pool filesystem quickly and efficiently. TD BackupPC pools have too many hardlinks to make rsync work efficiently. TD dd works, but copies all of the empty space too. This is one great command! I needed this feature, indeed! Thanks everyone! -- Marcos Lorenzo de Santiago System administrator marcos.lore...@andago.com ÁNDAGO INGENIERÍA Tlf: +34 916 011 373 Álcalde Ángel Arroyo, 10, 1º Mvl: +34 637 741 034 28904, Getafe, Madrid (Spain) Fax: +34 916 011 372 www.andago.com --- Si 'debbuging' es el proceso de eliminar errores, entonces la programación debe ser el proceso de ponerlos. -- PC Users. La mejor revista de computación del mundo. - vRanger cuts backup time in half-while increasing security. With the market-leading solution for virtual backup and recovery, you get blazing-fast, flexible, and affordable data protection. Download your free trial now. http://p.sf.net/sfu/quest-d2dcopy1___ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net List:https://lists.sourceforge.net/lists/listinfo/backuppc-users Wiki:http://backuppc.wiki.sourceforge.net Project: http://backuppc.sourceforge.net/
Re: [BackupPC-users] Best FS for BackupPC
Michael Stowe wrote at about 08:25:35 -0500 on Tuesday, May 24, 2011: I did a relatively short filesystem comparison when I moved my BackupPC pool to another set of drives. The high level results: jfs, xfs: quick, stable reiserfs: not stable ext4: slow ext3: very slow Any idea which particular file operations are causing ext3/4 to be slow relative to jfs/xfs? Is it reading? writing? linking? listing? all of the above equally? Linking and listing were where the main differences surfaced -- I speculate it's because xfs and jfs are more efficient at handling directory entries, but I didn't look into it too deeply -- I didn't do any special tuning on any of the filesystems. -- vRanger cuts backup time in half-while increasing security. With the market-leading solution for virtual backup and recovery, you get blazing-fast, flexible, and affordable data protection. Download your free trial now. http://p.sf.net/sfu/quest-d2dcopy1 ___ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net List:https://lists.sourceforge.net/lists/listinfo/backuppc-users Wiki:http://backuppc.wiki.sourceforge.net Project: http://backuppc.sourceforge.net/
