Couldn't BackupPC_nightly and BackupPC_link make use of a semaphore? - Wade
-----Original Message----- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Craig Barratt Sent: Wednesday, November 16, 2005 12:24 AM To: Carl Wilhelm Soderstrom Cc: backuppc-users@lists.sourceforge.net Subject: Re: [BackupPC-users] backuppc do not work parallel jobs (feature request) Carl Wilhelm Soderstrom writes: > how are you planning on dealing with the possible race conditions and the > like? Here's the plan. Can anyone see a problem with doing this? BackupPC_nightly and BackupPC_link are still mutually exclusive. But BackupPC_dump can overlap BackupPC_nightly. Craig ---------- Forwarded message ---------- To: Les Mikesell <[EMAIL PROTECTED]> From: Craig Barratt <[EMAIL PROTECTED]> Cc: Yaroslav Halchenko <[EMAIL PROTECTED]>, Leon Letto <[EMAIL PROTECTED]>, backuppc-users@lists.sourceforge.net Date: Sat, 18 Sep 2004 12:22:17 -0700 Subj: Re: [BackupPC-users] Re: Search for and delete files from the pool Les Mikesell writes: > However, to be safe you must make sure that backuppc is not > running. The link count itself is managed atomically by the > OS, but there is no way to tell when removing the cpool link > that some other running process has not concurrently decided to > make a new link at the same time that you remove it. That's > why all the backup runs must stop before backuppc_nightly does > the cleanup under cpool. This could probably be fixed with a > tiny amount of overhead by adding error handling for the rare > case where the link fails because the target disappeared. At > that point it is an atomic operation in the OS (although I > think I've seen something about versions of ReiserFS not getting > this right) so recreating the cpool entry with the newly copied > file should always maintain integrity. There is also the opposite race condition: the process doing the removing has already checked the link count and will remove the file. Another process makes a new link before the remove. The link succeeds (link count is now 2), and the remove succeeds too. Then there is a file in the backup directory that no longer is in the pool. The minor impact is one of storage: that file will no longer be matched. But the backup integrity is preserved, since the file still exists in the per-PC backup directory. Perhaps there is a strategy using rename: - removing process (BackupPC_nightly): - find each file with just 1 link, then: - rename that file. (This is a new step.) - recheck the number of links. If it is still 1, then remove the file. Otherwise, rename it back. (This is a new step.) - the rename might fail if a new link was made between the two renames. New links into the pool are only made by BackupPC_link, and only a single copy of that ever runs. So if BackupPC_nightly never runs with BackupPC_link (this is already the case) then this should never happen. - linking process (BackupPC_dump etc): - checks for existing files in the pool. - if not matched: add file to NewFileList, as is currently done - if matched: add a link in the per-PC backup directory to the pool, as it currently done. - if that fails it must mean a rename race occurred. So simply treat this as a new file by adding it to the NewFileList. (This is a new step.) - Later, BackupPC_link will run (exclusive to BackupPC_nightly) and it already rechecks each entry in NewFileList. Any failed links will be correctly added at this time. So this allows BackupPC_nightly to run at the same time as BackupPC_dump! BackupPC_nightly will still need to be exclusive with BackupPC_link, as it currently is. This sounds like a big improvement and easy to implement, although testing all the cases is a little tricky. Can anyone find a problem with this approach? Craig ------------------------------------------------------- This SF.Net email is sponsored by the JBoss Inc. Get Certified Today Register for a JBoss Training Course. Free Certification Exam for All Training Attendees Through End of 2005. For more info visit: http://ads.osdn.com/?ad_id=7628&alloc_id=16845&op=click _______________________________________________ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/backuppc-users http://backuppc.sourceforge.net/ ------------------------------------------------------- This SF.Net email is sponsored by the JBoss Inc. Get Certified Today Register for a JBoss Training Course. Free Certification Exam for All Training Attendees Through End of 2005. For more info visit: http://ads.osdn.com/?ad_idv28&alloc_id845&op=click _______________________________________________ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/backuppc-users http://backuppc.sourceforge.net/
