Re: [BackupPC-users] Per-PC pools
On Thu, 14 Mar 2013, Holger Parplies wrote: The OP can decide for himself, and I wish him the best of luck. I'm confident he won't come here with his problems if he runs into any. I for one can't imagine why he wouldn't come here with his problems. After all, he received such a warm reception this time. ;-D Back to lurking, Abe. -- Everyone hates slow websites. So do we. Make your web apps faster with AppDynamics Download AppDynamics Lite for free today: http://p.sf.net/sfu/appdyn_d2d_mar___ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net List:https://lists.sourceforge.net/lists/listinfo/backuppc-users Wiki:http://backuppc.wiki.sourceforge.net Project: http://backuppc.sourceforge.net/
Re: [BackupPC-users] Per-PC pools
On 2013-03-15 12:08, A Braxis wrote: I for one can't imagine why he wouldn't come here with his problems. After all, he received such a warm reception this time. ;-D I am also not happy with the snarky responses lately. I don't understand why you are doing this, but I'll judge you for it. I didn't read your patch, but here's why it can't possibly work, or will set your head on fire, or will become a time bomb you should vaguely fear. I'm here because I'm a BackupPC user and I want to both learn and help others. I hope the rest of you feel the same. Even if you can't imagine someone's use case, and even if you disagree with it, there is no cause for incivility. Regards, Tyler -- To have a child is to give fate a hostage. -- John F. Kennedy -- Everyone hates slow websites. So do we. Make your web apps faster with AppDynamics Download AppDynamics Lite for free today: http://p.sf.net/sfu/appdyn_d2d_mar ___ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net List:https://lists.sourceforge.net/lists/listinfo/backuppc-users Wiki:http://backuppc.wiki.sourceforge.net Project: http://backuppc.sourceforge.net/
Re: [BackupPC-users] Per-PC pools
On 2013-03-15 15:27, Tyler J. Wagner wrote: I am also not happy with the snarky responses lately. I don't understand why you are doing this, but I'll judge you for it. I didn't read your patch, but here's why it can't possibly work, or will set your head on fire, or will become a time bomb you should vaguely fear. And, my response is snarky. I knew I should have put that in Drafts first. Happy Friday, everybody. Tyler -- To have a child is to give fate a hostage. -- John F. Kennedy -- Everyone hates slow websites. So do we. Make your web apps faster with AppDynamics Download AppDynamics Lite for free today: http://p.sf.net/sfu/appdyn_d2d_mar ___ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net List:https://lists.sourceforge.net/lists/listinfo/backuppc-users Wiki:http://backuppc.wiki.sourceforge.net Project: http://backuppc.sourceforge.net/
Re: [BackupPC-users] Per-PC pools
On Fri, Mar 15, 2013 at 10:27 AM, Tyler J. Wagner ty...@tolaris.com wrote: I for one can't imagine why he wouldn't come here with his problems. After all, he received such a warm reception this time. ;-D I am also not happy with the snarky responses lately. I don't understand why you are doing this, but I'll judge you for it. I didn't read your patch, but here's why it can't possibly work, or will set your head on fire, or will become a time bomb you should vaguely fear. Note that those came _after_ the OP asked for advice, got it, and then complained that he didn't like it... I'm here because I'm a BackupPC user and I want to both learn and help others. I hope the rest of you feel the same. Even if you can't imagine someone's use case, and even if you disagree with it, there is no cause for incivility. The responses might have gone over the top, but there are good reasons to discuss the philosophy behind wanting to do something a system was not designed to do.This change might help some organization avoid working out proper accounting processes and maybe even temporarily save a few dollars in hardware. But the experience here says that changing complex code requires extensive testing before trusting it. It isn't a matter of not imagining this use case - it is going beyond and imagining if something goes wrong, or if you had to take over as a replacement sysadmin where someone had made one-off changes like that and even if they work you won't be able to get any help understanding or maintaining them. But, it doesn't pay to be thin-skinned or to take technical advice personally in this business. I'm sure that if there are more questions on this topic there will be answers - but they will continue to match the experience of the person posting it. -- Les Mikesell lesmikes...@gmail.com -- Everyone hates slow websites. So do we. Make your web apps faster with AppDynamics Download AppDynamics Lite for free today: http://p.sf.net/sfu/appdyn_d2d_mar ___ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net List:https://lists.sourceforge.net/lists/listinfo/backuppc-users Wiki:http://backuppc.wiki.sourceforge.net Project: http://backuppc.sourceforge.net/
Re: [BackupPC-users] Per-PC pools
On Fri, Mar 15, 2013 at 11:44 AM, Tyler J. Wagner ty...@tolaris.com wrote: On 2013-03-15 15:27, Tyler J. Wagner wrote: I am also not happy with the snarky responses lately. I understand both points of view - while this is supposed to be the users list, it has become the list for just about everything backupPC. So coding, development (and people knowledgeable on those) regularly respond alongside the guru users. It's easy to get annoyed with stupid development/coding questions when you just want the person asking the question to use backuppc as it was designed (and visa versa). I forgive everyone, and appreciate all the time you put into backuppc :) -- The fun parts of life are mostly optional. -- Everyone hates slow websites. So do we. Make your web apps faster with AppDynamics Download AppDynamics Lite for free today: http://p.sf.net/sfu/appdyn_d2d_mar ___ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net List:https://lists.sourceforge.net/lists/listinfo/backuppc-users Wiki:http://backuppc.wiki.sourceforge.net Project: http://backuppc.sourceforge.net/
Re: [BackupPC-users] Per-PC pools
On 16/03/13 02:27, Tyler J. Wagner wrote: On 2013-03-15 12:08, A Braxis wrote: I for one can't imagine why he wouldn't come here with his problems. After all, he received such a warm reception this time. ;-D I am also not happy with the snarky responses lately. I don't understand why you are doing this, but I'll judge you for it. I didn't read your patch, but here's why it can't possibly work, or will set your head on fire, or will become a time bomb you should vaguely fear. I'm here because I'm a BackupPC user and I want to both learn and help others. I hope the rest of you feel the same. Even if you can't imagine someone's use case, and even if you disagree with it, there is no cause for incivility. So, how about this question: I need to dig a ditch and think that I can rig some sticks of dynamite, and then plug that into a battery and watch them all go boom, but I'm not sure what voltage battery to use? Has anyone attempted this before? What voltage works best, or did it not work out for you? Don't you think it's a good idea for someone to ask Are you mad? That stuff will kill someone when you least expect it! Just because people haven't actually done it, if they know anything about explosives (I don't, except Wiley Coyote and Acme), then I think it would be really great if they told me that a shovel or a back-hoe would be a much better method and I should definitely not attempt to use dynamite. Just because people don't agree with you, doesn't make them wrong - I have no idea, but I'm sure someone has said that before... Now, can we all please stop getting our knickers in a knot and just calm down. If you don't like a response, hit delete and move on. PS, just for the record, I also think it's not the best path, custom versions of software always lead to pain (in my experience), so unless there is some major commercial advantage, then I steer clear. If it is really a great idea, then get the patch accepted upstream (eg, like the last useful patch for the FTP backup method). Regards, Adam -- Adam Goryachev Website Managers www.websitemanagers.com.au -- Everyone hates slow websites. So do we. Make your web apps faster with AppDynamics Download AppDynamics Lite for free today: http://p.sf.net/sfu/appdyn_d2d_mar ___ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net List:https://lists.sourceforge.net/lists/listinfo/backuppc-users Wiki:http://backuppc.wiki.sourceforge.net Project: http://backuppc.sourceforge.net/
Re: [BackupPC-users] Per-PC pools
On Fri, Mar 15, 2013 at 11:47 AM, Les Mikesell lesmikes...@gmail.comwrote: But the experience here says that changing complex code requires extensive testing before trusting it. It isn't a matter of not imagining this use case - it is going beyond and imagining if something goes wrong, or if you had to take over as a replacement sysadmin where someone had made one-off changes like that and even if they work you won't be able to get any help understanding or maintaining them. No one ever said that the changes wouldn't be tested. I'll probably run the new system for weeks, watching the logs carefully and making any necessary changes, before trusting it. On Fri, 15 Mar 2013, Adam Goryachev wrote: custom versions of software always lead to pain (in my experience), so unless there is some major commercial advantage, then I steer clear. Interesting point of view. I don't agree. I always considered the ability to make changes to the software to make it do what *I* want it to (rather than succumbing to what the original developer thinks I want it to do) as one of the hallmarks of opensource software. If you want to use software as-is, that can be beneficial too: http://news.slashdot.org/story/13/03/13/2052226/why-freeloaders-are-essential-to-foss-project-success V2 of the patch (unified, per request, even though there is no real standard for diff/patch) attached, if anyone cares. BackupPC-3.2.1-per-pc-pools-v2.patch Description: Binary data -- Everyone hates slow websites. So do we. Make your web apps faster with AppDynamics Download AppDynamics Lite for free today: http://p.sf.net/sfu/appdyn_d2d_mar___ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net List:https://lists.sourceforge.net/lists/listinfo/backuppc-users Wiki:http://backuppc.wiki.sourceforge.net Project: http://backuppc.sourceforge.net/
Re: [BackupPC-users] Per-PC pools
Adam Goryachev wrote at about 02:45:51 +1100 on Saturday, March 16, 2013: PS, just for the record, I also think it's not the best path, custom versions of software always lead to pain (in my experience), so unless there is some major commercial advantage, then I steer clear. If it is really a great idea, then get the patch accepted upstream (eg, like the last useful patch for the FTP backup method). Adam makes a very valid point... which I take to heart personally. While I may have contributed literally thousands of lines of code for BackupPC utilities, I have been extremely conservative in modifying the core code for precisely the reasons Adam mentions. Even when I have found bugs in the code, I often hesitate to patch them on my personal version unless the bug is something that is capable of affecting me. Even after 5 years of BackupPC hacking, I carry only 2 patches in my stock 3.2.0 code -- one allows me to automount the BackupPC volume if it is not previously mounted (to prevent it from either failing on startup or from writing to the wrong place) and one that fixes a documented (but always broken) feature to execute perl code in the pre dump command... all this despite having many ideas for cool new features and extensions... -- Everyone hates slow websites. So do we. Make your web apps faster with AppDynamics Download AppDynamics Lite for free today: http://p.sf.net/sfu/appdyn_d2d_mar ___ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net List:https://lists.sourceforge.net/lists/listinfo/backuppc-users Wiki:http://backuppc.wiki.sourceforge.net Project: http://backuppc.sourceforge.net/
Re: [BackupPC-users] Per-PC pools
On Fri, Mar 15, 2013 at 11:34 AM, Stephen Joyce sjb...@gmail.com wrote: On Fri, Mar 15, 2013 at 11:47 AM, Les Mikesell lesmikes...@gmail.com wrote: But the experience here says that changing complex code requires extensive testing before trusting it. It isn't a matter of not imagining this use case - it is going beyond and imagining if something goes wrong, or if you had to take over as a replacement sysadmin where someone had made one-off changes like that and even if they work you won't be able to get any help understanding or maintaining them. No one ever said that the changes wouldn't be tested. I'll probably run the new system for weeks, watching the logs carefully and making any necessary changes, before trusting it. Looking at a few logfiles isn't exactly what I'd call 'extensive' testing. Just off the top of my head, I'd wonder if the code that does the collision-chain fixups still works as intended - or how to verify that. Rsync-based stuff will probably work - or appear to - even if you completely break the pooling because it will link against the previous copy in the pc tree anyway, but other methods may be different. On Fri, 15 Mar 2013, Adam Goryachev wrote: custom versions of software always lead to pain (in my experience), so unless there is some major commercial advantage, then I steer clear. Interesting point of view. I don't agree. I always considered the ability to make changes to the software to make it do what *I* want it to (rather than succumbing to what the original developer thinks I want it to do) as one of the hallmarks of opensource software. If you want to use software as-is, that can be beneficial too: http://news.slashdot.org/story/13/03/13/2052226/why-freeloaders-are-essential-to-foss-project-success Regardless of what the youngsters on slashdot say, what I consider most important about opensource software is how widely tested it is, and the fact that by browsing the changelogs you can see the thousands of subtle bugs that large projects have already fixed so they won't affect you. If you've been using free code for decades you will realize that it is the bug reports and fixes that make it usable as much as the original developer's code. Sure, everyone is free to take their own chances with special-case changes, but when you balance it against the risks, I don't see a big win here unless your change goes into the mainstream code, which doesn't seem likely at this point in backuppc's state. Virtual machines already provide a well-tested and understood solution to the problem of sharing some resources with some level of isolation - with some very usable free implementations. -- Les Mikesell lesmikes...@gmail.com -- Everyone hates slow websites. So do we. Make your web apps faster with AppDynamics Download AppDynamics Lite for free today: http://p.sf.net/sfu/appdyn_d2d_mar ___ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net List:https://lists.sourceforge.net/lists/listinfo/backuppc-users Wiki:http://backuppc.wiki.sourceforge.net Project: http://backuppc.sourceforge.net/
Re: [BackupPC-users] Per-PC pools
Hi, Les Mikesell wrote on 2013-03-15 10:47:34 -0500 [Re: [BackupPC-users] Per-PC pools]: On Fri, Mar 15, 2013 at 10:27 AM, Tyler J. Wagner ty...@tolaris.com wrote: I for one can't imagine why he wouldn't come here with his problems. After all, he received such a warm reception this time. ;-D that's not the point. This is the *BackupPC users* list, not the I-applied-a-wild-hack-against-your-advice-so-please-help-me-now list. There would appear to be somewhat of a consensus that we don't like the idea, so why would we help with something that is clearly no longer a BackupPC matter? I am also not happy with the snarky responses lately. I don't understand why you are doing this, but I'll judge you for it. I didn't read your patch, but here's why it can't possibly work, or will set your head on fire, or will become a time bomb you should vaguely fear. Note that those came _after_ the OP asked for advice, got it, and then complained that he didn't like it... Yes, but sometimes they come earlier, especially when a stupid question is asked via Backup Central, or rather, when a question is asked in a stupid way. That was not the case here, clearly. I must remark, though, that the OP is potentially doing great harm and *ignoring hints to stop it*, so I'm extremely likely to get unfriendly towards the end of this post. I'm here because I'm a BackupPC user and I want to both learn and help others. Right, and that includes people just reading this list or possibly only single posts they find via google. Hey, I always wanted to split up my pool, because it doesn't fit on a single disk. Look, here's a patch that does that. Let's try it out. I hope the rest of you feel the same. Even if you can't imagine someone's use case, and even if you disagree with it, there is no cause for incivility. I can imagine the use case, and I tried suggesting alternatives (as did others). The response was I've already considered everything else and ruled it out. So, he doesn't want advice. Then again, he's asking for it. That is simply irritating. It isn't a matter of not imagining this use case - it is going beyond and imagining if something goes wrong, or if you had to take over as a replacement sysadmin where someone had made one-off changes like that and even if they work you won't be able to get any help understanding or maintaining them. Yes, and there is no indication the OP has thought about that. There's only I know what I'm doing, trust me type of statements. Still, it's not our problem. It's well meant but unwanted advice. But, it doesn't pay to be thin-skinned or to take technical advice personally in this business. Right. You're talking to coders, not diplomats. If you prefer diplomatic advice to technically sound advice, then that's perfectly ok, but this is simply the wrong place. Stephen Joyce wrote on 2013-03-15 12:34:24 -0400 [Re: [BackupPC-users] Per-PC pools]: V2 of the patch (unified, per request, even though there is no real standard for diff/patch) attached, if anyone cares. Let me be very clear. I URGE YOU TO NEVER AGAIN POST POTENTIALLY HARMFUL CODE ON THIS LIST. Ok? Your code is of interest to *noone* except yourself. If someone should really happen to have a similar problem and seek a similar solution, then he can ask. I'm quite sure he will be pointed to you (if he will not be discouraged), and then you can explain to him the prerequisites your code requires, which I somewhat doubt are documented. No, that's not a request for V3 with included documentation. Regards, Holger -- Everyone hates slow websites. So do we. Make your web apps faster with AppDynamics Download AppDynamics Lite for free today: http://p.sf.net/sfu/appdyn_d2d_mar ___ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net List:https://lists.sourceforge.net/lists/listinfo/backuppc-users Wiki:http://backuppc.wiki.sourceforge.net Project: http://backuppc.sourceforge.net/
Re: [BackupPC-users] Per-PC pools
I'm not sure all of this is warranted. I haven't tried the code, but looked at the patch over dinner and his changes in Lib.pm are in a place such that dumps and restores should use them just fine. I think Steven's taken care of the helper utilities that doesn't use Lib.pm too. The premise is sound. And I don't see any see any reason it shouldn't work. I'm not sure if curious enough to test the changes, but I don't think it's fair of Holger to try to silence him. Abe. On Fri, Mar 15, 2013 at 1:42 PM, Holger Parplies wb...@parplies.de wrote: Hi, Les Mikesell wrote on 2013-03-15 10:47:34 -0500 [Re: [BackupPC-users] Per-PC pools]: On Fri, Mar 15, 2013 at 10:27 AM, Tyler J. Wagner ty...@tolaris.com wrote: I for one can't imagine why he wouldn't come here with his problems. After all, he received such a warm reception this time. ;-D that's not the point. This is the *BackupPC users* list, not the I-applied-a-wild-hack-against-your-advice-so-please-help-me-now list. There would appear to be somewhat of a consensus that we don't like the idea, so why would we help with something that is clearly no longer a BackupPC matter? I am also not happy with the snarky responses lately. I don't understand why you are doing this, but I'll judge you for it. I didn't read your patch, but here's why it can't possibly work, or will set your head on fire, or will become a time bomb you should vaguely fear. Note that those came _after_ the OP asked for advice, got it, and then complained that he didn't like it... Yes, but sometimes they come earlier, especially when a stupid question is asked via Backup Central, or rather, when a question is asked in a stupid way. That was not the case here, clearly. I must remark, though, that the OP is potentially doing great harm and *ignoring hints to stop it*, so I'm extremely likely to get unfriendly towards the end of this post. I'm here because I'm a BackupPC user and I want to both learn and help others. Right, and that includes people just reading this list or possibly only single posts they find via google. Hey, I always wanted to split up my pool, because it doesn't fit on a single disk. Look, here's a patch that does that. Let's try it out. I hope the rest of you feel the same. Even if you can't imagine someone's use case, and even if you disagree with it, there is no cause for incivility. I can imagine the use case, and I tried suggesting alternatives (as did others). The response was I've already considered everything else and ruled it out. So, he doesn't want advice. Then again, he's asking for it. That is simply irritating. It isn't a matter of not imagining this use case - it is going beyond and imagining if something goes wrong, or if you had to take over as a replacement sysadmin where someone had made one-off changes like that and even if they work you won't be able to get any help understanding or maintaining them. Yes, and there is no indication the OP has thought about that. There's only I know what I'm doing, trust me type of statements. Still, it's not our problem. It's well meant but unwanted advice. But, it doesn't pay to be thin-skinned or to take technical advice personally in this business. Right. You're talking to coders, not diplomats. If you prefer diplomatic advice to technically sound advice, then that's perfectly ok, but this is simply the wrong place. Stephen Joyce wrote on 2013-03-15 12:34:24 -0400 [Re: [BackupPC-users] Per-PC pools]: V2 of the patch (unified, per request, even though there is no real standard for diff/patch) attached, if anyone cares. Let me be very clear. I URGE YOU TO NEVER AGAIN POST POTENTIALLY HARMFUL CODE ON THIS LIST. Ok? Your code is of interest to *noone* except yourself. If someone should really happen to have a similar problem and seek a similar solution, then he can ask. I'm quite sure he will be pointed to you (if he will not be discouraged), and then you can explain to him the prerequisites your code requires, which I somewhat doubt are documented. No, that's not a request for V3 with included documentation. Regards, Holger -- Everyone hates slow websites. So do we. Make your web apps faster with AppDynamics Download AppDynamics Lite for free today: http://p.sf.net/sfu/appdyn_d2d_mar ___ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net List:https://lists.sourceforge.net/lists/listinfo/backuppc-users Wiki:http://backuppc.wiki.sourceforge.net Project: http://backuppc.sourceforge.net/ -- Everyone hates slow websites. So do we. Make your web apps faster with AppDynamics Download AppDynamics Lite for free today: http://p.sf.net/sfu
Re: [BackupPC-users] Per-PC pools
Les, No need to apologize. I just felt that the arguments made by backu...@kosowsky.org were both specious and unnecessarily abrasive, especially since I didn't solicit them. I've been using BackupPC since 2005 and currently have 6 BPC servers backing up several dozen TBs. While I'm sure there are readers here who pre-date that, I don't consider myself a novice. Anyway, if anyone's interested my patch for per-pc pools for v. 3.2.1 is attached. I'm currently beta-testing it. This patch makes PoolDir and CPoolDir appear as configuration options on the Backup Settings page; they may be over-ridden on a per PC basis as many other configuration settings may be. On Wed, Mar 13, 2013 at 11:32 AM, Les Mikesell lesmikes...@gmail.comwrote: On Wed, Mar 13, 2013 at 10:02 AM, Stephen Joyce sjb...@gmail.com wrote: Thank you for your input, but I've already considered your other suggestions. As a reminder to other gentle readers, and to avoid further philosophical tirades about my foolish idea, my original question posed was Has anyone gone down this path before me? If so, did you succeed or fail? I'd like to compare notes either way. If you haven't, then please don't feel compelled to send an abrasive reply. I'm sorry. I thought you were asking for advice from people with experience. If you have tested VMs and concluded that they are not suitable for your purpose, never mind, then. -- Les Mikesell lesmikes...@gmail.com -- Everyone hates slow websites. So do we. Make your web apps faster with AppDynamics Download AppDynamics Lite for free today: http://p.sf.net/sfu/appdyn_d2d_mar ___ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net List:https://lists.sourceforge.net/lists/listinfo/backuppc-users Wiki:http://backuppc.wiki.sourceforge.net Project: http://backuppc.sourceforge.net/ BackupPC-3.2.1-per-pc-pools.patch Description: Binary data -- Everyone hates slow websites. So do we. Make your web apps faster with AppDynamics Download AppDynamics Lite for free today: http://p.sf.net/sfu/appdyn_d2d_mar___ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net List:https://lists.sourceforge.net/lists/listinfo/backuppc-users Wiki:http://backuppc.wiki.sourceforge.net Project: http://backuppc.sourceforge.net/
Re: [BackupPC-users] Per-PC pools
On Thu, Mar 14, 2013 at 12:32 PM, Stephen Joyce sjb...@gmail.com wrote: I've been using BackupPC since 2005 and currently have 6 BPC servers backing up several dozen TBs. While I'm sure there are readers here who pre-date that, I don't consider myself a novice. I'm still somewhat curious about why you dismissed virtual machines, which seem to me like a more obvious way to divvy up some partly-shared resources - and would offer a cleaner separation of control and better (still not great) methods to migrate to new/different hardware later. -- Les Mikesell lesmikes...@gmail.com -- Everyone hates slow websites. So do we. Make your web apps faster with AppDynamics Download AppDynamics Lite for free today: http://p.sf.net/sfu/appdyn_d2d_mar ___ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net List:https://lists.sourceforge.net/lists/listinfo/backuppc-users Wiki:http://backuppc.wiki.sourceforge.net Project: http://backuppc.sourceforge.net/
Re: [BackupPC-users] Per-PC pools
Stephen Joyce wrote at about 13:32:30 -0400 on Thursday, March 14, 2013: Les, No need to apologize. I just felt that the arguments made by backu...@kosowsky.org were both specious and unnecessarily abrasive, especially since I didn't solicit them. OMG - grow a pair... if you don't like an idea, ignore it, don't cry about it... -- Everyone hates slow websites. So do we. Make your web apps faster with AppDynamics Download AppDynamics Lite for free today: http://p.sf.net/sfu/appdyn_d2d_mar ___ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net List:https://lists.sourceforge.net/lists/listinfo/backuppc-users Wiki:http://backuppc.wiki.sourceforge.net Project: http://backuppc.sourceforge.net/
Re: [BackupPC-users] Per-PC pools
Hi,
Les Mikesell wrote on 2013-03-14 13:18:53 -0500 [Re: [BackupPC-users] Per-PC
pools]:
I'm still somewhat curious about why you dismissed virtual machines,
which seem to me like a more obvious way to divvy up some
partly-shared resources - and would offer a cleaner separation of
control and better (still not great) methods to migrate to
new/different hardware later.
and, in particular, would require no code changes at all. Yes, I'm curious,
too.
Concerning the code, I won't do more than have a quick glance at it, because
I'm not convinced it's a good idea. What the quick glance tells me, is that
the patch is next to unreadable, because it's not in unified format (i.e. no
context).
So, you seem to change PoolDir and CPoolDir in the library (though I don't see
where; let's hope your code is always executed before some part of BackupPC
tries to access the pool). That basically avoids touching any code in PoolWrite
and probably BackupPC_link. And by having a string setting for the pool
location, you enable pool sharing in a simple way. But you (i.e. the
administrator of the BackupPC instance) had better get the configuration right
(i.e. have the relevant pc/ and *pool/ directories on a common file system).
You don't seem to have checks for hard-linking capability. There's not much
help from the software in case of configuration errors.
You had better hope that the code consequently uses the PoolDir and CPoolDir
settings and not $TopDir/{c,}pool (easy enough to check). You probably remember
that it was a long standing bug that you couldn't set $Conf{TopDir} with the
desired effect ...
Furthermore, you lose the ability to use one BackupPC::Lib object for more
than one host (presuming you need the pool location). BackupPC probably
doesn't do that (I'm guessing), but I don't think this is a documented or
intended property.
While you might successfully use the code virtually forever, I would strongly
discourage anyone else from using it. There is just too much you need to
understand and have in mind. It's sort of half-automatic, because only half of
the consistency checks are done by the software. And by exposing the *PoolDir
settings to the web gui, you are suggesting that they are (changeable)
configuration options, while in reality they are descriptions of your disk
layout to BackupPC. I'd probably have preferred a fixed setting of
../{c,}pool relative to the host pc/ directory - i.e. use the pool on the FS
where the pc/ directory is. That is less flexible, but also less error-prone.
Regards,
Holger
--
Everyone hates slow websites. So do we.
Make your web apps faster with AppDynamics
Download AppDynamics Lite for free today:
http://p.sf.net/sfu/appdyn_d2d_mar
___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/
Re: [BackupPC-users] Per-PC pools
Stephen Joyce wrote at about 11:02:59 -0400 on Wednesday, March 13, 2013: I never said anything about philosophy. Those are your words and philosophical arguments. Thank you for your input, but I've already considered your other suggestions. My bad... you are right... you cited political reasons I should have referred you to alt.politics.backuppc if that is what you are seeking. If, however, you are looking for actual technical advice, then you might want to consider what people are trying to tell you. As a reminder to other gentle readers, and to avoid further philosophical tirades about my foolish idea, my original question posed was Has anyone gone down this path before me? If so, did you succeed or fail? I'd like to compare notes either way. If you haven't, then please don't feel compelled to send an abrasive reply. I'm sorry, I thought you actually wanted help from contributors who know a thing or two or three about how BackupPC works and not just hear from people who pursued the same foolish idea (your wording). Perhaps next time you should phrase your request more precisely if you are only interested in hearing from people who succeeded or failed in going down a path that those of us who actually know the working of BackupPC think to be foolish... My guess is you won't receive many answers... -- Everyone hates slow websites. So do we. Make your web apps faster with AppDynamics Download AppDynamics Lite for free today: http://p.sf.net/sfu/appdyn_d2d_mar ___ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net List:https://lists.sourceforge.net/lists/listinfo/backuppc-users Wiki:http://backuppc.wiki.sourceforge.net Project: http://backuppc.sourceforge.net/
Re: [BackupPC-users] Per-PC pools
Holger Parplies wrote at about 03:46:42 +0100 on Friday, March 15, 2013:
Hi,
Les Mikesell wrote on 2013-03-14 13:18:53 -0500 [Re: [BackupPC-users] Per-PC
pools]:
I'm still somewhat curious about why you dismissed virtual machines,
which seem to me like a more obvious way to divvy up some
partly-shared resources - and would offer a cleaner separation of
control and better (still not great) methods to migrate to
new/different hardware later.
and, in particular, would require no code changes at all. Yes, I'm curious,
too.
Concerning the code, I won't do more than have a quick glance at it, because
I'm not convinced it's a good idea. What the quick glance tells me, is that
the patch is next to unreadable, because it's not in unified format (i.e. no
context).
So, you seem to change PoolDir and CPoolDir in the library (though I don't
see
where; let's hope your code is always executed before some part of BackupPC
tries to access the pool). That basically avoids touching any code in
PoolWrite
and probably BackupPC_link. And by having a string setting for the pool
location, you enable pool sharing in a simple way. But you (i.e. the
administrator of the BackupPC instance) had better get the configuration
right
(i.e. have the relevant pc/ and *pool/ directories on a common file system).
You don't seem to have checks for hard-linking capability. There's not much
help from the software in case of configuration errors.
You had better hope that the code consequently uses the PoolDir and CPoolDir
settings and not $TopDir/{c,}pool (easy enough to check). You probably
remember
that it was a long standing bug that you couldn't set $Conf{TopDir} with the
desired effect ...
Furthermore, you lose the ability to use one BackupPC::Lib object for more
than one host (presuming you need the pool location). BackupPC probably
doesn't do that (I'm guessing), but I don't think this is a documented or
intended property.
While you might successfully use the code virtually forever, I would strongly
discourage anyone else from using it. There is just too much you need to
understand and have in mind. It's sort of half-automatic, because only half
of
the consistency checks are done by the software. And by exposing the *PoolDir
settings to the web gui, you are suggesting that they are (changeable)
configuration options, while in reality they are descriptions of your disk
layout to BackupPC. I'd probably have preferred a fixed setting of
../{c,}pool relative to the host pc/ directory - i.e. use the pool on the
FS
where the pc/ directory is. That is less flexible, but also less error-prone.
Beats me how this would work without also changing all the things
referencing the location of the pc tree (remember the super sensitive
OP specifically talked about using separate filesystems). In
particular, I see no reference to changes made to
BackupPC_link. Because as we all know, the pc tree and pool have to be
on the same filesystems... Then again no changes have been made to the
routine that checks for linkability so maybe the OP will never know
about such coding lapses.
Also, based on my playing with the code in Lib.pm and various other
modules, I seem to recall many more hard-coded references to pool
vs. cpool. Of course, it's possible that the OP got lucky and things
just somehow still work, but I sure as heck wouldn't count on it...
The fact that the OP doesn't know how to use standard patch format
doesn't give me a lot of confidence...
--
Everyone hates slow websites. So do we.
Make your web apps faster with AppDynamics
Download AppDynamics Lite for free today:
http://p.sf.net/sfu/appdyn_d2d_mar
___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/
Re: [BackupPC-users] Per-PC pools
Hi,
backu...@kosowsky.org wrote on 2013-03-14 23:05:42 -0400 [Re: [BackupPC-users]
Per-PC pools]:
[...]
Beats me how this would work without also changing all the things
referencing the location of the pc tree (remember the super sensitive
OP specifically talked about using separate filesystems).
my guess is that pc/xyz is a soft link to /somewhere/pc/xyz/ and the
corresponding pool setting is /somewhere/{c,}pool. This means setting up a
new host is manual work. I remember BackupPC_link having problems with soft
links at some point, although pc/ and pool/ were, in fact, on the same FS.
But, honestly, I don't want to waste much more time on this topic. It might
work. The ideas are not bad. And it might not work, but that's not my problem.
The only thing I am worried about is that someone finds the code at the end of
a google search and uses it without further reading or thinking (in
particular, whether it applies to his situation at all, which it doesn't most
of the time the question pops up). For that reason alone I am commenting
(maybe he at least reads the thread). The OP can decide for himself, and I
wish him the best of luck. I'm confident he won't come here with his problems
if he runs into any.
Then again no changes have been made to the
routine that checks for linkability so maybe the OP will never know
about such coding lapses.
They would show up in the logs. Probably.
Also, based on my playing with the code in Lib.pm and various other
modules, I seem to recall many more hard-coded references to pool
vs. cpool.
Possible. Also not my problem :-). I hinted at that, and that's where the
matter ends for me.
Of course, it's possible that the OP got lucky and things
just somehow still work, but I sure as heck wouldn't count on it...
You mean like what happens if the target FS is not mounted? Or in other
corner-cases less obvious? That is probably the real issue. No additional
error cases are handled, but I'm sure numerous ones are introduced.
Regards,
Holger
--
Everyone hates slow websites. So do we.
Make your web apps faster with AppDynamics
Download AppDynamics Lite for free today:
http://p.sf.net/sfu/appdyn_d2d_mar
___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/
Re: [BackupPC-users] Per-PC pools
On Wed, Mar 13, 2013 at 6:52 AM, Stephen Joyce sjb...@gmail.com wrote: I'm in a situation where I find myself desiring per-pc pools.[1] It's been a while since I've dipped my toes into the BackupPC's code, but I've done a bit of preliminary research into this, and think I've identified the places where changes would need to be made to allow the pool or cpool to be overridden for individual PCs -- nominally to be located at $TopDir/pc/$host/cpool, for example. The idea here is would be to allow different PCs' pools to reside on different physical filesystems for political reasons. I don't want to disable pooling entirely since it still has features that would be beneficial even if pooling one (or a few) PCs rather than an entire organization. I haven't read this list regularly in a few years, so I thought I'd ask: has anyone gone down this path before me? If so, did you succeed or fail? I'd like to compare notes either way. [1] My specific situation is that I have multiple linux PCs with large volumes of research data. This data is generally unique to that PC with little duplication between PCs (at least between PCs of different research groups). The storage to backup this data is also usually funded by individual faculty accounts (sometimes grants) and as such should be dedicated to that faculty's PC(s). Separate BackupPC servers (real or virtual) is another option I have considered, but seems unnecessarily wasteful. I don't think I've seen anyone mention trying that. While it is probably feasible, in your situation I would do something a little more drastic and use virtual machines to split things up. If you don't share physical disks (or do use a raid/san with many drives) and don't overcommit memory too much, you don't lose that much performance. I've had good results with the free version of VMware ESXi, but KVM probably works as well these days. There is not much CPU overhead for virtualization on modern processors. -- Les Mikesell lesmikes...@gmaill.com -- Everyone hates slow websites. So do we. Make your web apps faster with AppDynamics Download AppDynamics Lite for free today: http://p.sf.net/sfu/appdyn_d2d_mar ___ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net List:https://lists.sourceforge.net/lists/listinfo/backuppc-users Wiki:http://backuppc.wiki.sourceforge.net Project: http://backuppc.sourceforge.net/
Re: [BackupPC-users] Per-PC pools
Hi, On 3/13/13 12:52 PM, Stephen Joyce wrote: I'm in a situation where I find myself desiring per-pc pools.[1] [...] The storage to backup this data is also usually funded by individual faculty accounts (sometimes grants) and as such should be dedicated to that faculty's PC(s). I would suggest that you pool the money, and keep tabs on how much each faculty's PC is using based on the Full Size sum for each PC, and then split the bill accordingly. -- Best regards, Lars Tobias -- Everyone hates slow websites. So do we. Make your web apps faster with AppDynamics Download AppDynamics Lite for free today: http://p.sf.net/sfu/appdyn_d2d_mar ___ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net List:https://lists.sourceforge.net/lists/listinfo/backuppc-users Wiki:http://backuppc.wiki.sourceforge.net Project: http://backuppc.sourceforge.net/
Re: [BackupPC-users] Per-PC pools
Stephen Joyce wrote at about 07:52:11 -0400 on Wednesday, March 13, 2013: I'm in a situation where I find myself desiring per-pc pools.[1] It's been a while since I've dipped my toes into the BackupPC's code, but I've done a bit of preliminary research into this, and think I've identified the places where changes would need to be made to allow the pool or cpool to be overridden for individual PCs -- nominally to be located at $TopDir/pc/$host/cpool, for example. The idea here is would be to allow different PCs' pools to reside on different physical filesystems for political reasons. I don't want to disable pooling entirely since it still has features that would be beneficial even if pooling one (or a few) PCs rather than an entire organization. I haven't read this list regularly in a few years, so I thought I'd ask: has anyone gone down this path before me? If so, did you succeed or fail? I'd like to compare notes either way. [1] My specific situation is that I have multiple linux PCs with large volumes of research data. This data is generally unique to that PC with little duplication between PCs (at least between PCs of different research groups). The storage to backup this data is also usually funded by individual faculty accounts (sometimes grants) and as such should be dedicated to that faculty's PC(s). Separate BackupPC servers (real or virtual) is another option I have considered, but seems unnecessarily wasteful. I read what you write and come to a different conclusion... 1. First, I don't see why separate budgets requires separate pc pools but not separate BackupPC instances. If it's merely an accounting issue, then come up with a fair metric that roughly aligns with usage. For example, charge each user based upon the relative size of their fulls to the total storage size. That seems a lot simpler and more robust then rewriting the basic code of BackupPC. If the issue were security and encryption or statutory requirements for keeping data isolated, then I could understand the need for pool separation. Otherwise, you are pursuing the wrong solution to the wrong problem. If its not even an accounting issue, but a philosophical one, then perhaps you should be pursuing the question on alt.philosophy.backuppc... because that is beyond the scope of this group... 2. Second, why go to the trouble of rewriting deeply embedded code to separate pools within a single BackupPC daemon process rather than just running separate instances of BackupPC? Once the pools are separate, there is truly de minimus savings to run one vs. multiple BackupPC instances. The storage (pool and pc trees) is completely separate, the BackupPC_dump instances run separately anyway and the BackuPC_nightly processes are separate. The only savings is that you have a single backup daemon running which takes up only trivial processing power and maybe a few small file shared log and config files. On the other hand, you are relying on untested hacked code to perform the critical task of backups with the possibility of introducing subtle errors that may not surface until you need to do a restore and it's too late. Can you be sure you have located all the places in the code where tests are made for pool vs. cpool? Seems a waste of time and foolish to me. 3. Finally, if you want to use separate filesystems for the pools, then you also will need to have separate pc trees. And you will need to edit the backuppc startup code that attempts to test writing hard links and probably also hack how BackupPC moves and deletes 'trash'. So, now you have separate pools, separate pc trees, separate dump (and link) processes (always true), separate (i.e. non-shared) BackupPC_nightly processes, separate (i.e., non-shared) BackupPC_trashClean, etc... Seems to me like you are doing a lot of coding work and taking a huge risk of errors in order to do something that is conceptually not much different from running totally separate BackupPC instances and all just for the sake of philosophy... BTW, running separate instances of BackupPC need not require separate virtual machines. One could probably just make changes to the config.pl to keep separate processes from colliding... -- Everyone hates slow websites. So do we. Make your web apps faster with AppDynamics Download AppDynamics Lite for free today: http://p.sf.net/sfu/appdyn_d2d_mar ___ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net List:https://lists.sourceforge.net/lists/listinfo/backuppc-users Wiki:http://backuppc.wiki.sourceforge.net Project: http://backuppc.sourceforge.net/
Re: [BackupPC-users] Per-PC pools
Hi, backu...@kosowsky.org wrote on 2013-03-13 10:11:17 -0400 [Re: [BackupPC-users] Per-PC pools]: Stephen Joyce wrote at about 07:52:11 -0400 on Wednesday, March 13, 2013: I'm in a situation where I find myself desiring per-pc pools.[1] [...] I read what you write and come to a different conclusion... I'm not saying I don't, but there's one thing you don't mention: [...] 2. Second, why go to the trouble of rewriting deeply embedded code to separate pools within a single BackupPC daemon process rather than just running separate instances of BackupPC? Once the pools are separate, there is truly de minimus savings to run one vs. multiple BackupPC instances. True, but the one thing you *don't* get with independent daemon processes is coordinated scheduling. There are good reasons to limit concurrent backups. With independent storage units that may be less important, but there *can* be other reasons for wanting a global decision process (i.e. which backup should be run first, how many concurrently ...). Depending on your infrastructure, that could either mean using separate servers, modifying the code as you suggested, pooling the money and keeping a single instance, or just running several instances because you don't need coordinated scheduling. In fact, you might even need coordination per group rather than globally, which would be much easier with independent instances. Another point that springs to mind is that all might benefit from a common pool on a faster storage system (more spindles, faster RAID level). While total backup time over all systems might be the same, each individual backup might complete faster. You might just get more out of your money by pooling resources. Regards, Holger -- Everyone hates slow websites. So do we. Make your web apps faster with AppDynamics Download AppDynamics Lite for free today: http://p.sf.net/sfu/appdyn_d2d_mar ___ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net List:https://lists.sourceforge.net/lists/listinfo/backuppc-users Wiki:http://backuppc.wiki.sourceforge.net Project: http://backuppc.sourceforge.net/
Re: [BackupPC-users] Per-PC pools
I never said anything about philosophy. Those are your words and philosophical arguments. Thank you for your input, but I've already considered your other suggestions. As a reminder to other gentle readers, and to avoid further philosophical tirades about my foolish idea, my original question posed was Has anyone gone down this path before me? If so, did you succeed or fail? I'd like to compare notes either way. If you haven't, then please don't feel compelled to send an abrasive reply. On Wed, Mar 13, 2013 at 10:11 AM, backu...@kosowsky.org wrote: Stephen Joyce wrote at about 07:52:11 -0400 on Wednesday, March 13, 2013: I'm in a situation where I find myself desiring per-pc pools.[1] It's been a while since I've dipped my toes into the BackupPC's code, but I've done a bit of preliminary research into this, and think I've identified the places where changes would need to be made to allow the pool or cpool to be overridden for individual PCs -- nominally to be located at $TopDir/pc/$host/cpool, for example. The idea here is would be to allow different PCs' pools to reside on different physical filesystems for political reasons. I don't want to disable pooling entirely since it still has features that would be beneficial even if pooling one (or a few) PCs rather than an entire organization. I haven't read this list regularly in a few years, so I thought I'd ask: has anyone gone down this path before me? If so, did you succeed or fail? I'd like to compare notes either way. [1] My specific situation is that I have multiple linux PCs with large volumes of research data. This data is generally unique to that PC with little duplication between PCs (at least between PCs of different research groups). The storage to backup this data is also usually funded by individual faculty accounts (sometimes grants) and as such should be dedicated to that faculty's PC(s). Separate BackupPC servers (real or virtual) is another option I have considered, but seems unnecessarily wasteful. I read what you write and come to a different conclusion... 1. First, I don't see why separate budgets requires separate pc pools but not separate BackupPC instances. If it's merely an accounting issue, then come up with a fair metric that roughly aligns with usage. For example, charge each user based upon the relative size of their fulls to the total storage size. That seems a lot simpler and more robust then rewriting the basic code of BackupPC. If the issue were security and encryption or statutory requirements for keeping data isolated, then I could understand the need for pool separation. Otherwise, you are pursuing the wrong solution to the wrong problem. If its not even an accounting issue, but a philosophical one, then perhaps you should be pursuing the question on alt.philosophy.backuppc... because that is beyond the scope of this group... 2. Second, why go to the trouble of rewriting deeply embedded code to separate pools within a single BackupPC daemon process rather than just running separate instances of BackupPC? Once the pools are separate, there is truly de minimus savings to run one vs. multiple BackupPC instances. The storage (pool and pc trees) is completely separate, the BackupPC_dump instances run separately anyway and the BackuPC_nightly processes are separate. The only savings is that you have a single backup daemon running which takes up only trivial processing power and maybe a few small file shared log and config files. On the other hand, you are relying on untested hacked code to perform the critical task of backups with the possibility of introducing subtle errors that may not surface until you need to do a restore and it's too late. Can you be sure you have located all the places in the code where tests are made for pool vs. cpool? Seems a waste of time and foolish to me. 3. Finally, if you want to use separate filesystems for the pools, then you also will need to have separate pc trees. And you will need to edit the backuppc startup code that attempts to test writing hard links and probably also hack how BackupPC moves and deletes 'trash'. So, now you have separate pools, separate pc trees, separate dump (and link) processes (always true), separate (i.e. non-shared) BackupPC_nightly processes, separate (i.e., non-shared) BackupPC_trashClean, etc... Seems to me like you are doing a lot of coding work and taking a huge risk of errors in order to do something that is conceptually not much different from running totally separate BackupPC instances and all just for the sake of philosophy... BTW, running separate instances of BackupPC need not require separate virtual machines. One could probably just make changes to the config.pl to keep separate processes from colliding...
Re: [BackupPC-users] Per-PC pools
Hi Holger, Thanks for the reply. The separate pools would be on separate raid arrays, separately funded. Many research grant funding sources have strict regulations about ensuring funds go only to support a given project, hence the separation requirement. Part of this is in an effort to cut down on the number of backup servers I have. In the past I have recommended each group (or sometimes each grant/project) purchase its own backup server and storage. But this just leads to server proliferation. Regarding separate BPC instances on the same hardware, the scheduling issues, need for lots of duplications of code, and likelihood of screwing something up seemed prohibitive. After looking into the code, I think the changes would be trivial. Probably under 50 lines total. It's not necessary to have separate BackupPC_nightly and BackupPC_trashclean processes as one person opined; the existing ones simply need to be made aware of the additional locations on which to operate. ~Stephen On Wed, Mar 13, 2013 at 10:51 AM, Holger Parplies wb...@parplies.de wrote: Hi, backu...@kosowsky.org wrote on 2013-03-13 10:11:17 -0400 [Re: [BackupPC-users] Per-PC pools]: Stephen Joyce wrote at about 07:52:11 -0400 on Wednesday, March 13, 2013: I'm in a situation where I find myself desiring per-pc pools.[1] [...] I read what you write and come to a different conclusion... I'm not saying I don't, but there's one thing you don't mention: [...] 2. Second, why go to the trouble of rewriting deeply embedded code to separate pools within a single BackupPC daemon process rather than just running separate instances of BackupPC? Once the pools are separate, there is truly de minimus savings to run one vs. multiple BackupPC instances. True, but the one thing you *don't* get with independent daemon processes is coordinated scheduling. There are good reasons to limit concurrent backups. With independent storage units that may be less important, but there *can* be other reasons for wanting a global decision process (i.e. which backup should be run first, how many concurrently ...). Depending on your infrastructure, that could either mean using separate servers, modifying the code as you suggested, pooling the money and keeping a single instance, or just running several instances because you don't need coordinated scheduling. In fact, you might even need coordination per group rather than globally, which would be much easier with independent instances. Another point that springs to mind is that all might benefit from a common pool on a faster storage system (more spindles, faster RAID level). While total backup time over all systems might be the same, each individual backup might complete faster. You might just get more out of your money by pooling resources. Regards, Holger -- Everyone hates slow websites. So do we. Make your web apps faster with AppDynamics Download AppDynamics Lite for free today: http://p.sf.net/sfu/appdyn_d2d_mar ___ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net List:https://lists.sourceforge.net/lists/listinfo/backuppc-users Wiki:http://backuppc.wiki.sourceforge.net Project: http://backuppc.sourceforge.net/ -- Everyone hates slow websites. So do we. Make your web apps faster with AppDynamics Download AppDynamics Lite for free today: http://p.sf.net/sfu/appdyn_d2d_mar___ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net List:https://lists.sourceforge.net/lists/listinfo/backuppc-users Wiki:http://backuppc.wiki.sourceforge.net Project: http://backuppc.sourceforge.net/
Re: [BackupPC-users] Per-PC pools
On Wed, Mar 13, 2013 at 10:02 AM, Stephen Joyce sjb...@gmail.com wrote: Thank you for your input, but I've already considered your other suggestions. As a reminder to other gentle readers, and to avoid further philosophical tirades about my foolish idea, my original question posed was Has anyone gone down this path before me? If so, did you succeed or fail? I'd like to compare notes either way. If you haven't, then please don't feel compelled to send an abrasive reply. I'm sorry. I thought you were asking for advice from people with experience. If you have tested VMs and concluded that they are not suitable for your purpose, never mind, then. -- Les Mikesell lesmikes...@gmail.com -- Everyone hates slow websites. So do we. Make your web apps faster with AppDynamics Download AppDynamics Lite for free today: http://p.sf.net/sfu/appdyn_d2d_mar ___ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net List:https://lists.sourceforge.net/lists/listinfo/backuppc-users Wiki:http://backuppc.wiki.sourceforge.net Project: http://backuppc.sourceforge.net/
Re: [BackupPC-users] Per-PC pools
Holger Parplies wrote at about 15:51:19 +0100 on Wednesday, March 13, 2013: Hi, backu...@kosowsky.org wrote on 2013-03-13 10:11:17 -0400 [Re: [BackupPC-users] Per-PC pools]: Stephen Joyce wrote at about 07:52:11 -0400 on Wednesday, March 13, 2013: I'm in a situation where I find myself desiring per-pc pools.[1] [...] I read what you write and come to a different conclusion... I'm not saying I don't, but there's one thing you don't mention: [...] 2. Second, why go to the trouble of rewriting deeply embedded code to separate pools within a single BackupPC daemon process rather than just running separate instances of BackupPC? Once the pools are separate, there is truly de minimus savings to run one vs. multiple BackupPC instances. True, but the one thing you *don't* get with independent daemon processes is coordinated scheduling. There are good reasons to limit concurrent backups. With independent storage units that may be less important, but there *can* be other reasons for wanting a global decision process (i.e. which backup should be run first, how many concurrently ...). Depending on your infrastructure, that could either mean using separate servers, modifying the code as you suggested, pooling the money and keeping a single instance, or just running several instances because you don't need coordinated scheduling. In fact, you might even need coordination per group rather than globally, which would be much easier with independent instances. Another point that springs to mind is that all might benefit from a common pool on a faster storage system (more spindles, faster RAID level). While total backup time over all systems might be the same, each individual backup might complete faster. You might just get more out of your money by pooling resources. I agree with your comment on scheduling... though that would be a manageable issue unless he were supporting large numbers of independent users/pools. Given his use case referencing faculty with separate filesystems, I assumed that the overall number of distinct instances would be relatively small. For a small such number, just reduce the max processes per instance and/or adjust the blackout periods. Since the rate limiting step on most modern systems is file access and not computational power and since the OP was talking about separate filesystems (presumably on separately paid for hardware), the number of concurrent processes is probably not even an issue. This seems a lot less risky than mucking with the code especially since the notion of pool location is scattered throughout the code... -- Everyone hates slow websites. So do we. Make your web apps faster with AppDynamics Download AppDynamics Lite for free today: http://p.sf.net/sfu/appdyn_d2d_mar ___ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net List:https://lists.sourceforge.net/lists/listinfo/backuppc-users Wiki:http://backuppc.wiki.sourceforge.net Project: http://backuppc.sourceforge.net/
