Re: [BackupPC-users] Yet another filesystem thread

2011-07-03 Thread Holger Parplies
Hi,

C. Ronoz wrote on 2011-06-30 12:54:44 +0200 [Re: [BackupPC-users] Yet another 
filesystem thread]:
 [...]
  - How stable is XFS?

unless I missed something, I'd say XFS is perfectly stable - more stable than
reiserfs in any case. The only thing that makes me hesitate with that statement
is Les' remark XFS should also be OK on 64-bit systems - why only on 64 bit
systems? [Of course, for really large pools, a 64 bit system would be
preferable with XFS.]

 [Bowie Bailey asked on 2011-06-29 10:43:28 -0400:]
 How much memory do you have on the backup server?  What backup method
 are you using?
 The server has 1GB memory, but a pretty powerful processor.

A powerful processor doesn't help even marginally with memory problems.
See http://en.wikipedia.org/wiki/Thrashing_(computer_science)

 I found out that BackupPC is ignoring my Excludes though, [...]

This is because your syntax is wrong.

 $Conf{BackupFilesOnly} = {};
 $Conf{BackupFilesExclude} = {'/proc', '/blaat', '/pub', '/tmp'};

While the comments in config.pl state

# This can be set to a string, an array of strings, or, in the case
# of multiple shares, a hash of strings or arrays.

that is actually incorrect. A hash of strings makes no sense. In fact, Perl
would turn your example into a hash with /proc and /pub as keys and
/blaat and /tmp as respective values - certainly not what you want.
Turn your config value into an array (use '[]' instead of '{}'), and you
should be fine. You'll notice that the examples correctly don't include a
hash of strings.

Better yet, use a full hash of arrays. That is easier to read and maintain,
because it's explicit on which shares you want which excludes to apply to:

$Conf {BackupFilesExclude} = { '/' = [ '/proc', '/blaat', '/pub', '/tmp' ] };

The leading '/' on your excludes is just fine, contrary to what has been said.
It anchors them to the transfer root. Without the slashes, you would also be
excluding e.g. /home/user/pub and /home/user/tmp, just as two examples of
things you might *not* want to exclude (well, you might even want to exclude
/home/user/tmp, but really *any* file or directory named tmp? It's your
decision, you can do whatever you want, even things like tmp/ (only
directories) or /home/**/tmp/ (only directories somewhere under /home) or
/home/*/tmp/ (only directories immediately in some user's home directory).
See the rsync man page for details). Just note that if your share name is
*not* /, you'll need to remove that part from the excludes (e.g. for a share
name /var, to exclude /var/tmp you'll need to specify /tmp as the exclude,
not /var/tmp, which would try to exclude /var/var/tmp).

 This could explain why the run takes longer, but it should still finish
 within an hour?

On the first run (or whenever something is added that does not yet exist in
the pool), compression might slow down things considerably, especially if your
exclude of /proc is not working. Just consider how long compressing a large
file (say 1 GB) takes in comparison to how long reading the file takes. The
host status page should tell you more about how much data your backups
contain and how much of that was already in the pool.

 You can just delete the directory and remove the test host from your
 hosts file.
 That will only remove the hardlinks, not the original files in the pool?

What you mean is correct, but you should note that there is nothing more
original about the hardlinks from the pool to the content than those from
the pc directory to the same content. They are all hardlinks and are
indistinguishable from each other. Every normal file on your Linux system is
a hardlink to some content in the file system, just for files with only a
single hardlink we don't usually think much about it (and for files with more
than one hardlink we don't usually *need* to think much about it - it just
works as intended).

 The space should be released when BackuPC_Nightly runs.  If you want to
 start over quickly, I'd make a new filesystem on your archive partition
 (assuming you did mount a separate partition there, which is always a
 good idea...) and re-install the program.

I believe you don't even need to reinstall anything. BackupPC creates most of
the directories it needs, probably excluding $TopDir, which will exist in your
case, because it's the mount point, but which will need to have the correct
permissions (user=backuppc, group=backuppc perms=u=rwx,g=rx,o= - but check
your installation values before unmounting the existing FS). Reinstalling
BackupPC may or may not be the easier option, depending on your preferences.

 I ran backuppc nightly /usr/share/backuppc/bin/BackupPC_nightly 0 255

You shouldn't have. Hopefully, there were no BackupPC_link processes running
during that time. BackupPC_nightly *should* contain a comment something like

# *NEVER* RUN THIS BY HAND WHILE A BackupPC DAEMON IS RUNNING. IF YOU NEED AN
# IMMEDIATE NIGHTLY RUN, TELL THE BackupPC DAEMON TO LAUNCH ONE INSTEAD:
#
# BackupPC_serverMesg 

Re: [BackupPC-users] Recompressing individual files in pool

2011-07-03 Thread Holger Parplies
Hi,

Kelly Sauke wrote on 2011-07-01 09:21:28 -0500 [[BackupPC-users] Recompressing 
individual files in pool]:
 I have a need to modify certain files from backups that I have in 
 BackupPC.  My pool is compressed and I've found I can decompress single 
 files using BackupPC_zcat.  I can then modify those files as needed, 
 however I cannot figure out how to re-compress those modified files to 
 be put back into the pool.  Is there a tool available that can do that?  

no. It's not a common requirement to be able to modify files in backups.
Normally, a backup is intended to reflect the state a file system was in at
the time the backup was taken, not the state the file system *should have*
been in or the state *I'd like it* to have been in. I sure hope you have
legitimate reasons for doing this.

If you are modifying files, you'll need to think about several things.

* Do you want to modify every occurrence of a specific content (i.e. all
  files in all backups linked to one pool file) or only specific files,
  while other files continue to contain the unmodified content?

* If you are modifying every occurrence of a specific content, you'll either
  have to find out which files link to the pool file (hard, with a reasonably
  sized pool) or ensure you're updating the content without changing the inode
  (i.e. open the file for write, not delete and re-create it). If you do that,
  there is not much you can do for failure recovery. Your update had better
  succeed.

* Does your update change the partial file md5sum? If so, you'll need to move
  the pool file to its new name and location. Presuming the new content
  already exists, you should probably create a hash collision. That may be
  less efficient than linking to the target pool file, but it should be legal
  (when the maximum link count is exceeded, a second pool file with identical
  content is created; later on the link count on the first file may drop due
  to expiring backups), and it's certainly simpler than finding all the files
  linked to your modified pool file and re-linking them to the pre-existing
  pool file.

* If you're only changing individual files in a pc/ directory, the matter is
  far more simple. You'll need to take some code from the BackupPC sources
  for compressing anyway, so you might as well take the part that handles
  pooling as well (see BackupPC::PoolWrite and note that you'll be coding in
  Perl ;-).

 Is there a better way to go about modifying certain files within my 
 backups?

Once the contents you want to have in the files are in the pool (probably from
a recent backup), you can just figure out the pool files and link to them. If
you want that to be really easy, ask Jeffrey about his hpool directory ;-).

Regards,
Holger

--
All of the data generated in your IT infrastructure is seriously valuable.
Why? It contains a definitive record of application performance, security 
threats, fraudulent activity, and more. Splunk takes this data and makes 
sense of it. IT sense. And common sense.
http://p.sf.net/sfu/splunk-d2d-c2
___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/


Re: [BackupPC-users] Yet another filesystem thread

2011-07-03 Thread Doug Lytle
Holger Parplies wrote:
 unless I missed something, I'd say XFS is perfectly stable - more stable than
 reiserfs in any case. The only thing that makes me hesitate with that 
 statement
 is Les' remark XFS should also be OK on 64-bit systems - why only on 64 bit

I thought the same thing.  I'm running XFS (LVM)on a 32bit Mandriva 
install with a 1.1TB pool and I have been for years with no issues.

Doug


-- 
Ben Franklin quote:

Those who would give up Essential Liberty to purchase a little Temporary 
Safety, deserve neither Liberty nor Safety.


--
All of the data generated in your IT infrastructure is seriously valuable.
Why? It contains a definitive record of application performance, security 
threats, fraudulent activity, and more. Splunk takes this data and makes 
sense of it. IT sense. And common sense.
http://p.sf.net/sfu/splunk-d2d-c2
___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/


Re: [BackupPC-users] Yet another filesystem thread

2011-07-03 Thread Les Mikesell
On 7/3/11 12:31 PM, Holger Parplies wrote:

 unless I missed something, I'd say XFS is perfectly stable - more stable than
 reiserfs in any case. The only thing that makes me hesitate with that 
 statement
 is Les' remark XFS should also be OK on 64-bit systems - why only on 64 bit
 systems? [Of course, for really large pools, a 64 bit system would be
 preferable with XFS.]

It may not apply to all distributions, but at least Red Hat/CentOS use 4k 
stacks 
in the 32-bit kernel builds and XFS isn't happy with that.


Concerning bare metal recovery, how do you plan to do that? Restoring
to the target host requires an installed and running system, restoring
to a naked new disk mountedsomewhere  requires a plan how to do that
with BackupPC, as well as some preparation (partitioning, file systems)
and some modifications afterwards (boot loader, /etc/fstab, ...).
BackupPC is not designed to handle all of that alone, though it will
obviously handle a large part of the task if that is how you want to
use it.

As long as you know the approximate sizes of the partitions you need, you can 
use a Linux livecd to boot on a new machine, make the partitions and 
filesystems, mount them somewhere, then ssh an appropriate BackupPC_tarCreate 
command to the backuppc server and pipe to a local tar to drop it in place. 
But, it's a lot of grunge work and may take some practice.

This project: http://rear.sourceforge.net/ seems to have all the missing pieces 
to save a description of the disk layout and make a bootable iso that will 
reconstruct it, but it would take some work to integrate the parts with 
backuppc.

-- 
Les Mikesell
 lesmikes...@gmail.com


--
All of the data generated in your IT infrastructure is seriously valuable.
Why? It contains a definitive record of application performance, security 
threats, fraudulent activity, and more. Splunk takes this data and makes 
sense of it. IT sense. And common sense.
http://p.sf.net/sfu/splunk-d2d-c2
___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/


Re: [BackupPC-users] ssh goes defunct but BackupPC is waiting for it

2011-07-03 Thread Holger Parplies
Hi,

Aleksey Tsalolikhin wrote on Fri, 1 Jul 2011 11:56:42 -0700:
 I just noticed one of my servers has not had a successful full backup
 for over 60 days.  Incremental backups still succeed.

have you only got a single full backup for that host, or did full backups work
up to some point in time, when they started failing? And why does BackupPC do
an incremental backup after a failed full backup? Shouldn't the full be
retried until it succeeds? That doesn't sound right.

Les Mikesell wrote on 2011-07-01 21:05:03 -0500 [Re: [BackupPC-users] ssh goes 
defunct but BackupPC is waiting for it]:
 On 7/1/11 6:53 PM, Aleksey Tsalolikhin wrote:
  On Fri, Jul 1, 2011 at 3:16 PM, Les Mikeselllesmikes...@gmail.com  wrote:
  Is there a NAT router, stateful firewall, or similar device between them?
 
  Yes, there is!
 
  Backups with few changes can let the connection time out leaving both ends
  waiting for the other.

Would that lead to a zombie ssh process? See the ssh_config man page for how
to send keepalive messages (TCPKeepAlive or ServerAliveInterval) to keep your
firewall happy.

What is your $Conf{RsyncClientCmd}, or rather, what commands show up in the
XferLOG files for incremental and for full backups? Have you looked at the
XferLOG for a failing full backup? Does it always seem to fail at the same
point?

  Interesting.  I'm doing a full backup... I don't think that would
  qualify as a backup
  with few changes?  I thought full backup means copy everything, hey?
 
 An rsync full means read everything at both ends but only send the
 differences. 
   There can be a lot of dead air in the process.

I'm not sure how that exchange happens. Is it one large list with all the
details at the start (i.e. file list + checksums for all files)? Does the
receiver really need to compute checksums for *all* files (with
--ignore-times), even those the sender side would send anyway due to changed
attributes? Wouldn't that mean reading those files twice? That would certainly
explain why checksum caching makes such a difference (maybe I should switch it
on ;-).

Regards,
Holger

--
All of the data generated in your IT infrastructure is seriously valuable.
Why? It contains a definitive record of application performance, security 
threats, fraudulent activity, and more. Splunk takes this data and makes 
sense of it. IT sense. And common sense.
http://p.sf.net/sfu/splunk-d2d-c2
___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/


Re: [BackupPC-users] ssh goes defunct but BackupPC is waiting for it

2011-07-03 Thread Les Mikesell
On 7/3/11 1:57 PM, Holger Parplies wrote:

 An rsync full means read everything at both ends but only send the
 differences.
There can be a lot of dead air in the process.

 I'm not sure how that exchange happens. Is it one large list with all the
 details at the start (i.e. file list + checksums for all files)? Does the
 receiver really need to compute checksums for *all* files (with
 --ignore-times), even those the sender side would send anyway due to changed
 attributes?

I'm not sure about the details myself, but I've seen the long idle times and it 
has come up as a problem on the list before - and can be fixed either with 
keep-alives on the connection or increasing the idle timer for connections on 
the router/firewall.

 Wouldn't that mean reading those files twice? That would certainly
 explain why checksum caching makes such a difference (maybe I should switch it
 on ;-).

Rsync has some sort of sliding block window to zoom in on changes, not sure 
about inode/attribute changes that don't involve the file contents.  They might 
always be sent anyway.  Checksum caching only affects the server side - the 
sender has to do the reads anyway.

-- 
   Les Mikesell
lesmikes...@gmail.com

--
All of the data generated in your IT infrastructure is seriously valuable.
Why? It contains a definitive record of application performance, security 
threats, fraudulent activity, and more. Splunk takes this data and makes 
sense of it. IT sense. And common sense.
http://p.sf.net/sfu/splunk-d2d-c2
___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/


Re: [BackupPC-users] Yet another filesystem thread

2011-07-03 Thread Jeffrey J. Kosowsky
Holger Parplies wrote at about 19:31:14 +0200 on Sunday, July 3, 2011:
 
  While the comments in config.pl state
  
  # This can be set to a string, an array of strings, or, in the case
  # of multiple shares, a hash of strings or arrays.
  
  that is actually incorrect. A hash of strings makes no sense. In fact, Perl
  would turn your example into a hash with /proc and /pub as keys and
  /blaat and /tmp as respective values - certainly not what you want.
  Turn your config value into an array (use '[]' instead of '{}'), and you
  should be fine. You'll notice that the examples correctly don't include a
  hash of strings.
  

I think by hash of strings, the following is meant:
$Conf {BackupFilesExclude} = { 'share1' = 'exclude-path1',
   'share2' = 'exclude-path2',
...
 }

This is just a simpler case of the hash of arrays that you illustrate
below. While I have not tried that syntax, I imagine that is what the
documentation refers to. Of course, the wording is not terribly clear
except maybe to those who already know what is going on (and
understand perl)...


  Better yet, use a full hash of arrays. That is easier to read and maintain,
  because it's explicit on which shares you want which excludes to apply to:
  
  $Conf {BackupFilesExclude} = { '/' = [ '/proc', '/blaat', '/pub', '/tmp' ] 
  };
  
  The leading '/' on your excludes is just fine, contrary to what has been 
  said.
  It anchors them to the transfer root. Without the slashes, you would also 
  be
  excluding e.g. /home/user/pub and /home/user/tmp, just as two examples of
  things you might *not* want to exclude (well, you might even want to exclude
  /home/user/tmp, but really *any* file or directory named tmp? It's your
  decision, you can do whatever you want, even things like tmp/ (only
  directories) or /home/**/tmp/ (only directories somewhere under /home) or
  /home/*/tmp/ (only directories immediately in some user's home directory).
  See the rsync man page for details). Just note that if your share name is
  *not* /, you'll need to remove that part from the excludes (e.g. for a 
  share
  name /var, to exclude /var/tmp you'll need to specify /tmp as the 
  exclude,
  not /var/tmp, which would try to exclude /var/var/tmp).
  

--
All of the data generated in your IT infrastructure is seriously valuable.
Why? It contains a definitive record of application performance, security 
threats, fraudulent activity, and more. Splunk takes this data and makes 
sense of it. IT sense. And common sense.
http://p.sf.net/sfu/splunk-d2d-c2
___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/


Re: [BackupPC-users] Recompressing individual files in pool

2011-07-03 Thread Jeffrey J. Kosowsky
Holger Parplies wrote at about 20:10:20 +0200 on Sunday, July 3, 2011:
  Hi,
  
  Kelly Sauke wrote on 2011-07-01 09:21:28 -0500 [[BackupPC-users] 
  Recompressing individual files in pool]:
   I have a need to modify certain files from backups that I have in 
   BackupPC.  My pool is compressed and I've found I can decompress single 
   files using BackupPC_zcat.  I can then modify those files as needed, 
   however I cannot figure out how to re-compress those modified files to 
   be put back into the pool.  Is there a tool available that can do that?  
  
  no. It's not a common requirement to be able to modify files in backups.
  Normally, a backup is intended to reflect the state a file system was in at
  the time the backup was taken, not the state the file system *should have*
  been in or the state *I'd like it* to have been in. I sure hope you have
  legitimate reasons for doing this.
  
  If you are modifying files, you'll need to think about several things.
  
  * Do you want to modify every occurrence of a specific content (i.e. all
files in all backups linked to one pool file) or only specific files,
while other files continue to contain the unmodified content?

And this may be subtle. You may have other occurrences that you forgot
about or are not aware of (say another machine with the same file or
an earlier backup you had saved). Destructively editing the pool is
not something to do without thinking...

  * If you are modifying every occurrence of a specific content, you'll either
have to find out which files link to the pool file (hard, with a reasonably
sized pool) or ensure you're updating the content without changing the 
  inode
(i.e. open the file for write, not delete and re-create it). If you do 
  that,
there is not much you can do for failure recovery. Your update had better
succeed.
  
  * Does your update change the partial file md5sum? If so, you'll need to move
the pool file to its new name and location. Presuming the new content
already exists, you should probably create a hash collision. That may be
less efficient than linking to the target pool file, but it should be legal
(when the maximum link count is exceeded, a second pool file with identical
content is created; later on the link count on the first file may drop due
to expiring backups), and it's certainly simpler than finding all the files
linked to your modified pool file and re-linking them to the pre-existing
pool file.
  

Yes - unless you are just changing content between the first and last
chunks (keeping the file size the same), the partial file md5sum will
change.
That being said, while it is technically correct and advisable to
rename the file with the correct partial file md5sum (including
adjusting the suffix for potential collisions), it is not strictly
necessary. Indeed, I have had the pleasure of finding several bugs
within BackupPC or its libraries that result in wrong md5sum names
even under normal conditions. The only real downside of not changing
the name is that new versions of the file will not be pooled and will
be stored under the correct md5sum name (Note: I am not advising not
changing the name, just saying it is not strictly necessary).

Another perhaps more important issue is that you really need to change
the attrib file. While changing the accesss/mod times may not matter,
adjusting the uncompressed filesize (if it changes) is important since
some routines may/do use that file size, rather than decompressing the
entire file to calculate its size. In any case, even if not critical,
having an inconsistency between the actual file size and the size
noted in the attrib file is not a good idea and might suggest to
anybody or any routine not aware of your monkeying with the file that
there has been some serious data corruption.

The bottom line is that editing existing files is possible (and indeed
I do a lot more 'messy' things in my BackupPC_deleteFile routine)
*but* you need to think of all the side-effects and end cases to make
sure you won't be messing anything else up.



  * If you're only changing individual files in a pc/ directory, the matter is
far more simple. You'll need to take some code from the BackupPC sources
for compressing anyway, so you might as well take the part that handles
pooling as well (see BackupPC::PoolWrite and note that you'll be coding in
Perl ;-).
  

--
All of the data generated in your IT infrastructure is seriously valuable.
Why? It contains a definitive record of application performance, security 
threats, fraudulent activity, and more. Splunk takes this data and makes 
sense of it. IT sense. And common sense.
http://p.sf.net/sfu/splunk-d2d-c2
___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List:

[BackupPC-users] BackupFilesExclude syntax (was: Re: Yet another filesystem thread)

2011-07-03 Thread Holger Parplies
Hi,

Jeffrey J. Kosowsky wrote on 2011-07-03 20:18:07 -0400 [Re: [BackupPC-users] 
Yet another filesystem thread]:
 Holger Parplies wrote at about 19:31:14 +0200 on Sunday, July 3, 2011:
   While the comments in config.pl state
   
   # This can be set to a string, an array of strings, or, in the case
   # of multiple shares, a hash of strings or arrays.
   
   that is actually incorrect. A hash of strings makes no sense. In fact, Perl
   would turn your example into a hash with /proc and /pub as keys and
   /blaat and /tmp as respective values - certainly not what you want.
   [...]
 
 I think by hash of strings, the following is meant:
 $Conf {BackupFilesExclude} = { 'share1' = 'exclude-path1',
'share2' = 'exclude-path2',
 ...
  }
 
 This is just a simpler case of the hash of arrays that you illustrate
 below.

in fact, it's the same as what I meant to illustrate above, and in fact, it's
wrong.

Rereading the code for BackupPC::Lib::backupFileConfFix yet again, I'm quite
sure it doesn't do anything meaningful for a hash of strings. For the case
of BackupFiles{Exclude,Only} already being a hash, the values are, in fact,
*not* promoted to arrays if they are scalars. Later on (at least in
Xfer::Rsync - I didn't check the other XferMethods), they are used as array
refs:

foreach my $file ( @{$conf-{BackupFilesExclude}{$t-{shareName}}} )

In absense of 'strict refs', this appears to yield an empty array, which is
preferable to a syntax error, but effectively (and silently) behaves as if
the excludes (or includes) were not specified at all. So, yes, you *can*
specify a hash of strings, but, no, they won't be used as excludes (or
includes).

Before we get into a discussion whether this should be fixed, and because I
had already written this before I realized that it doesn't currently work this
way:

The point is that it's not a hash of strings, it's a hash of key-value
pairs. A hash of strings suggests just what the original poster understood:
several strings of the same meaning, excludes probably. I was first going to
say that it's actually a string of share names (with empty exclude lists,
which makes no sense), when I realized that that's not true. Unless you write
it with = (which, in Perl, is simply a synonym for ,), it's anything but
obvious. Perl won't mind if you write

$Conf {BackupFilesExclude} = { 'share1',
   'exclude-path1' = 'share2',
   'exclude-path2', ...
 };

but it's misleading, just as it is with commas only. The interpretation is
key-value pairs, so it should be stated explicitly that that's what you
[would] need to supply.

I wonder if the case of only a single exclude pattern per share for all shares
is common enough to warrant advocating its usage [or rather implementing it].
Of course, a mixed usage [would then] also work:

$Conf {BackupFilesExclude} = { 'share1' = 'exclude1',
   'share2' = [ 'exclude2a', 'exclude2b' ],
   'share3' = 'exclude3',
 };

But who would write that or encourage others to do so? What's the point in
leaving out the '[]' here? Allowing one single array of excludes to apply to
all shares (array value) makes sense, and extending that to allow one single
exclude pattern to apply to all shares (scalar value) also. Both are
simplifications for people not familiar with Perl syntax. Allowing a single
hash of strings would not be a simplification (in my opinion, at least).
Well, your opinion may vary ;-).

 While I have not tried that syntax, I imagine that is what the
 documentation refers to.

I haven't tried it either, obviously.

 Of course, the wording is not terribly clear except maybe to those who
 already know what is going on (and understand perl)...

Well, it would seem to confuse even those ;-).

Either the wording or the implementation needs to be changed. My suggestion is
to change the documentation to:

# This can be set to a string, an array of strings, or, in the case
# of multiple shares, a hash of (string, array) pairs.

I can't see the need for supporting a hash of strings variant.

Regards,
Holger

--
All of the data generated in your IT infrastructure is seriously valuable.
Why? It contains a definitive record of application performance, security 
threats, fraudulent activity, and more. Splunk takes this data and makes 
sense of it. IT sense. And common sense.
http://p.sf.net/sfu/splunk-d2d-c2
___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:http://backuppc.wiki.sourceforge.net
Project: 

Re: [BackupPC-users] Recompressing individual files in pool

2011-07-03 Thread Holger Parplies
Hi,

Jeffrey J. Kosowsky wrote on 2011-07-03 20:29:48 -0400 [Re: [BackupPC-users] 
Recompressing individual files in pool]:
 Holger Parplies wrote at about 20:10:20 +0200 on Sunday, July 3, 2011:
   Kelly Sauke wrote on 2011-07-01 09:21:28 -0500 [[BackupPC-users] 
 Recompressing individual files in pool]:
   * Does your update change the partial file md5sum? If so, you'll need to
 move the pool file to its new name and location. [...]
 
 Yes - unless you are just changing content between the first and last
 chunks (keeping the file size the same), the partial file md5sum will
 change.

not quite exact ;-), but close enough for this thread.

 That being said, while it is technically correct and advisable to
 rename the file with the correct partial file md5sum (including
 adjusting the suffix for potential collisions), it is not strictly
 necessary.

True, of course. You don't even really need the pool file. Actually, I'd
consider *removing* the pool file rather than leaving it in the pool with
an incorrect name (of course that includes adjusting collision chains).
Incorrectly named pool files just add work for BackupPC when trying to
match new content to existing pool files. Am I missing any side effects
and end cases? Statistics will be inacurate?

 Another perhaps more important issue is that you really need to change
 the attrib file.

And here, again, you need to note that attrib files are also pooled, so you
can't simply change the attrib file in place.

Note that this also rules out simply modifying the pool file - you need all
the pc/ files linking to it, because you need to modify all their attrib files
(presuming you *are* changing the file size). But I somehow have the
impression you weren't going to do that anyway.

 The bottom line is that editing existing files is possible (and indeed
 I do a lot more 'messy' things in my BackupPC_deleteFile routine)
 *but* you need to think of all the side-effects and end cases to make
 sure you won't be messing anything else up.

And as you see, it can get quite complicated. We haven't even touched on
the matter of backup dependencies yet. Supposing you only want to change a
file for a specific (range of) backup(s), you may need to add the file with
its previous contents to the next backup after the one(s) you want to change.

Well, any of that may or may not apply to what you actually want to do. Could
you perhaps be a bit more specific?

Regards,
Holger

--
All of the data generated in your IT infrastructure is seriously valuable.
Why? It contains a definitive record of application performance, security 
threats, fraudulent activity, and more. Splunk takes this data and makes 
sense of it. IT sense. And common sense.
http://p.sf.net/sfu/splunk-d2d-c2
___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/