Re: [Bacula-users] Problem with backup failing after 2 hours
On Wed, March 17, 2010 03:32, Jerry Lowry wrote: > Hi, > I have a new installation that I am tweeking the backups on. One of my > backups goes through the firewall from an public ip address to where the > backup server is on an private ip address. The backup works just fine > for the first 2h 11m 15s and then it fails. After the first test failed > I inserted the 'Heartbeat Interval' option and set it for 30 sec. This > slowed down the backup but I wanted to make sure that it continued > through the entire disk. The disk that I am backing up has closed to > 250GB of data on it. I get approx. 50GB backed up before it fails. Is > there any other setting that will help this finish. > The logs are here: That sounds like a firewall is killing the connection, due to a "connection lifetime" time-out rather than a "connection inactive" time-out. I've seen a similar problem trying to get an FTP connection through a firewall that had a 2 minute lifetime time-out, convincing the administrators that it was not a good idea was impossible - they thought that *all* internet traffic was like HTTP, short bursts and transient connections. We could get the time-out increased, but when the shift rotated they would automatically re-set the time-out! Cheers, GaryB-) -- Download Intel® Parallel Studio Eval Try the new software tools for yourself. Speed compiling, find bugs proactively, and fine-tune applications for parallel performance. See why Intel Parallel Studio got high marks during beta. http://p.sf.net/sfu/intel-sw-dev ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
[Bacula-users] Performance issues after upgrade
Hi, On a bacula installation that I originally set up on a machine running Debian "etch", everything was running smoothly. Since etch has been EOL'ed, it was necessary to upgrade it, and we recently did do an upgrade to lenny. This also involved an upgrade of bacula from 1.38 (the version in etch) to 2.4.4 (the version in lenny). The unfortunate result of that has been that a full backup job of a server with slightly more than a tera of data, which used to take just shy of 24 hours blew up to just over three days. Obviously I'm not very happy with that. Investigating turned up that for some reason data spooling was not enabled; enabling that caused a massive gain in performance, with only one thing left. After the backup job is finished (which now takes a bit more than 24 hours, something I consider acceptable), bacula starts spooling attribute data. This, too, takes just over 24 hours, with no output during the whole time this is happening. Since the postgresql 'postmaster' is in 'D' state the whole time when this happens, I suspect this has something to do with postgresql not being properly configured, or similar, but I don't see what exactly is going wrong. Is there a resource somewhere that I could look into to help me figure out what's wrong with my postgresql setup? Or is there something else I should look into? If this is a known issue that is fixed in later bacula versions, please let me know; in that case an upgrade is not out of the question, though I'd prefer not having to do that if it isn't necessary. Thanks, -- The biometric identification system at the gates of the CIA headquarters works because there's a guard with a large gun making sure no one is trying to fool the system. http://www.schneier.com/blog/archives/2009/01/biometrics.html -- Download Intel® Parallel Studio Eval Try the new software tools for yourself. Speed compiling, find bugs proactively, and fine-tune applications for parallel performance. See why Intel Parallel Studio got high marks during beta. http://p.sf.net/sfu/intel-sw-dev ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] accurate backup estimate runs forever
On 3/16/2010 7:02 PM, Brock Palen wrote: > I have had running for about 1 hour > estimate level=incremental job=xen1-host accurate=yes > > I wanted to test the accurate backup, what I see is MySQL using 100% > cpu for a long time and never stops. > I have enabled the slow query log on MySQL and I don't see anything > being added. See if this solution helps: http://wiki.bacula.org/doku.php?id=faq#jobs_with_accurate_filesets_take_forever -- Frank Sweetser fs at wpi.edu | For every problem, there is a solution that WPI Senior Network Engineer | is simple, elegant, and wrong. - HL Mencken GPG fingerprint = 6174 1257 129E 0D21 D8D4 E8A3 8E39 29E3 E2E8 8CEC -- Download Intel® Parallel Studio Eval Try the new software tools for yourself. Speed compiling, find bugs proactively, and fine-tune applications for parallel performance. See why Intel Parallel Studio got high marks during beta. http://p.sf.net/sfu/intel-sw-dev ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
[Bacula-users] accurate backup estimate runs forever
I have had running for about 1 hour estimate level=incremental job=xen1-host accurate=yes I wanted to test the accurate backup, what I see is MySQL using 100% cpu for a long time and never stops. I have enabled the slow query log on MySQL and I don't see anything being added. What kind of information should I be looking at as to why this is taking so long? Thanks! Brock Palen bro...@mlds-networks.com www.mlds-networks.com MLDS Owner Senior Tech. -- Download Intel® Parallel Studio Eval Try the new software tools for yourself. Speed compiling, find bugs proactively, and fine-tune applications for parallel performance. See why Intel Parallel Studio got high marks during beta. http://p.sf.net/sfu/intel-sw-dev ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] verify old volume checksums
Brock Palen schrieb: > Is there a way to verify a volumes internal checksums against its data? > > Even better is there a way to have the catalog compare its checksums > against the files in volumes? I know about the verify job but that > appears to be only for the most recent job run. I would like to do > this as a type of 'verify that a full backup is possible'. No, this is no possible right now. But there is already a feature request. Ralf -- Download Intel® Parallel Studio Eval Try the new software tools for yourself. Speed compiling, find bugs proactively, and fine-tune applications for parallel performance. See why Intel Parallel Studio got high marks during beta. http://p.sf.net/sfu/intel-sw-dev ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Backing up more than 200 servers
> True, but for anyone interested, there is working Gentoo ebuild for Bacula > 5.0.1 here: > > http://bugs.gentoo.org/show_bug.cgi?id=302162 > Thanks for pointing that out. I will have to see what is different from the ebuild in my overlay.. John -- Download Intel® Parallel Studio Eval Try the new software tools for yourself. Speed compiling, find bugs proactively, and fine-tune applications for parallel performance. See why Intel Parallel Studio got high marks during beta. http://p.sf.net/sfu/intel-sw-dev ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Bacula Enhancements Request
Bob Cousins wrote: > - Tape verification option -- verify that the bits read back from the > tape are the same as those originally written. Spooling to disk makes > this easier. Also, if a tape fails verification, it should be easy to > restart the write/verify cycle with new media. I used to do this, wasting time and increasing wear on my tape drive, until I discovered that LTO tape drives (hopefully other modern drives as well), read the data back as it is being written (the read head is positioned behind the write head). If an error is detected, it writes a second copy of the data. These will appear as rewrite errors in the drive error log. Regards, Richard -- Download Intel® Parallel Studio Eval Try the new software tools for yourself. Speed compiling, find bugs proactively, and fine-tune applications for parallel performance. See why Intel Parallel Studio got high marks during beta. http://p.sf.net/sfu/intel-sw-dev ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Backing up more than 200 servers
On 16/03/2010 18:04, Phil Stracchino wrote: > On 03/16/10 13:22, Richard Mortimer wrote: >> On 16/03/2010 15:33, Phil Stracchino wrote: >>> Yes, agreed. Unfortunately, 5.0.1 is not yet readily available on all >>> platforms except by building from source. I was looking at this the >>> other day to see whether it was feasible yet to upgrade all of my >>> machines, and finding that (for example) Debian and Ubuntu *still* have >>> no stable packages newer than 2.4.4-b1, although there are unstable >>> Bacula-5 .debs >> 5.0.1 transitioned into the Debian testing (squeeze) release on the 14th >> March. That is reported as being available for most (all?) supported >> Debian platforms. >> In addition it seems that 5.0.1 is now available in lenny-backports. >> >> http://packages.debian.org/search?keywords=bacula&searchon=names&suite=all§ion=all >> >> I haven't checked Ubuntu recently but now it is available in backports >> I'm going to start testing/moving over to 5.0.x on my setup. > > OK, sounds like I just need to figure out how to add the right > repository on Ubuntu then. > Hi Again, I just checked Ubuntu and it looks like 3.0.2 is the newest that they have packaged. That is the version in both the ppa archive and in the current image for the upcoming lucid release. https://launchpad.net/~ubuntu-bacula/+archive/ppa It should be possible to build locally from the Debian source archive without changing much (if anything). I will likely give this a try in the next week or so but other committments mean that I don't know when. Regards Richard -- Download Intel® Parallel Studio Eval Try the new software tools for yourself. Speed compiling, find bugs proactively, and fine-tune applications for parallel performance. See why Intel Parallel Studio got high marks during beta. http://p.sf.net/sfu/intel-sw-dev ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Bacula Enhancements Request
On 03/16/10 14:15, Gavin McCullagh wrote: > On Tue, 16 Mar 2010, Bob Cousins wrote: >> - Explicit capturing of boot and partitioning information for easier >> bare metal restores. (Bare metal restores should be easy and fast for >> all OSes, but that is easier said than done.) > > Do you mean backing up the MBR? That might be nice alright. Doing this portably is actually a rather difficult problem because there are so many different partitioning schemes and partition table formats out there. It's a mess. Clean bare-metal restore has been a feature that's been worked on since almost the earliest days of Bacula, and there still isn't a clean single solution, precisely *because* it's a complex problem. -- Phil Stracchino, CDK#2 DoD#299792458 ICBM: 43.5607, -71.355 ala...@caerllewys.net ala...@metrocast.net p...@co.ordinate.org Renaissance Man, Unix ronin, Perl hacker, Free Stater It's not the years, it's the mileage. -- Download Intel® Parallel Studio Eval Try the new software tools for yourself. Speed compiling, find bugs proactively, and fine-tune applications for parallel performance. See why Intel Parallel Studio got high marks during beta. http://p.sf.net/sfu/intel-sw-dev ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Bacula Enhancements Request
Hi, On Tue, 16 Mar 2010, Bob Cousins wrote: > - Data staged to disk on the way to tape -- allowing backups to spool > faster than tapes and when tape drives are full. I'm pretty sure this already exists. http://www.bacula.org/en/dev-manual/Data_Spooling.html > - Explicit capturing of boot and partitioning information for easier > bare metal restores. (Bare metal restores should be easy and fast for > all OSes, but that is easier said than done.) Do you mean backing up the MBR? That might be nice alright. > And a couple of really low priority but nice to have ideas: > > - Optional system shutdown after backup -- ideal for desktop machines > which would otherwise be idle for hours. Can you not just use the hook to run a command after backup with whatever shutdown command the computer uses? Admittedly I guess the FD could do it, but it's not massively difficult. > - API for database backups -- backing up MySQL's files while live is > bad. Mysqldump output is big. We need a 'better way.' One way is using a snapshot, where the system in question supports it. For example, if you have your MySQL database on an LVM partition you can stop (or quiesce) mysql, snapshot, start mysql, mount the snapshot and begin backing up the data files off the snapshot. ZFS offers similar: http://blogs.digitar.com/jjww/2008/01/snapback-the-joys-of-backing-up-mysql-with-zfs/ Gavin -- Download Intel® Parallel Studio Eval Try the new software tools for yourself. Speed compiling, find bugs proactively, and fine-tune applications for parallel performance. See why Intel Parallel Studio got high marks during beta. http://p.sf.net/sfu/intel-sw-dev ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Backing up more than 200 servers
> Yes, agreed. Unfortunately, 5.0.1 is not yet readily available on all > platforms except by building from source. I was looking at this the > other day to see whether it was feasible yet to upgrade all of my > machines, and finding that (for example) Debian and Ubuntu still have > no stable packages newer than 2.4.4-b1, although there are unstable > Bacula-5 .debs and Gentoo has nothing newer than 3.0.3 even in unstable. > It seems to be an uphill struggle to keep all the various OS > distributions up to date on Bacula. True, but for anyone interested, there is working Gentoo ebuild for Bacula 5.0.1 here: http://bugs.gentoo.org/show_bug.cgi?id=302162 -- Download Intel® Parallel Studio Eval Try the new software tools for yourself. Speed compiling, find bugs proactively, and fine-tune applications for parallel performance. See why Intel Parallel Studio got high marks during beta. http://p.sf.net/sfu/intel-sw-dev ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Bacula Enhancements Request
On Tue, Mar 16, 2010 at 1:34 PM, Bob Cousins wrote: > The online documentation indicates that enhancement requests should be > posted to this group. I'd like to see a small number of enhancements. > Many of these have been mentioned before, so please just consider this > comment a "Me Too!" to add weight to those who decide what is next. > See here for the proper format: http://bacula.org/en/?page=feature-request John -- Download Intel® Parallel Studio Eval Try the new software tools for yourself. Speed compiling, find bugs proactively, and fine-tune applications for parallel performance. See why Intel Parallel Studio got high marks during beta. http://p.sf.net/sfu/intel-sw-dev ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Backing up more than 200 servers
On 03/16/10 13:22, Richard Mortimer wrote: > On 16/03/2010 15:33, Phil Stracchino wrote: >> Yes, agreed. Unfortunately, 5.0.1 is not yet readily available on all >> platforms except by building from source. I was looking at this the >> other day to see whether it was feasible yet to upgrade all of my >> machines, and finding that (for example) Debian and Ubuntu *still* have >> no stable packages newer than 2.4.4-b1, although there are unstable >> Bacula-5 .debs > 5.0.1 transitioned into the Debian testing (squeeze) release on the 14th > March. That is reported as being available for most (all?) supported > Debian platforms. > In addition it seems that 5.0.1 is now available in lenny-backports. > > http://packages.debian.org/search?keywords=bacula&searchon=names&suite=all§ion=all > > I haven't checked Ubuntu recently but now it is available in backports > I'm going to start testing/moving over to 5.0.x on my setup. OK, sounds like I just need to figure out how to add the right repository on Ubuntu then. -- Phil Stracchino, CDK#2 DoD#299792458 ICBM: 43.5607, -71.355 ala...@caerllewys.net ala...@metrocast.net p...@co.ordinate.org Renaissance Man, Unix ronin, Perl hacker, Free Stater It's not the years, it's the mileage. -- Download Intel® Parallel Studio Eval Try the new software tools for yourself. Speed compiling, find bugs proactively, and fine-tune applications for parallel performance. See why Intel Parallel Studio got high marks during beta. http://p.sf.net/sfu/intel-sw-dev ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
[Bacula-users] verify old volume checksums
Is there a way to verify a volumes internal checksums against its data? Even better is there a way to have the catalog compare its checksums against the files in volumes? I know about the verify job but that appears to be only for the most recent job run. I would like to do this as a type of 'verify that a full backup is possible'. Thanks! Brock Palen bro...@mlds-networks.com www.mlds-networks.com MLDS Owner Senior Tech. -- Download Intel® Parallel Studio Eval Try the new software tools for yourself. Speed compiling, find bugs proactively, and fine-tune applications for parallel performance. See why Intel Parallel Studio got high marks during beta. http://p.sf.net/sfu/intel-sw-dev ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
[Bacula-users] OT: LOPSA-NJ Professional IT Community Conference 2010 May 7-8 New Brunswick NJ.
Please pardon the Interruption. There isn't a way to post this for just folks in the US. LOPSA-NJ Professional IT Community Conference 2010 PICC10 is a gathering of professionals from the diverse IT (computer and network administration) community in New Jersey to learn, share ideas, and network. The conference includes invited speakers and keynotes, training by top-notch experts that is relevant, useful, and recession-friendly; plus an unconference track where attendees propose and host their own topics during the event. We expect attendance of 100 to 150 IT professionals from mid- to large-sized companies and academia from New Jersey/New York/Pennsylvania. We go by many titles but everyone is invited: system administrators, network administrators, network engineers, Windows, Linux, Unix, DBAs, etc. Flyer for the event http://www.lopsanj.org/events/picc10/files/flyer.pdf Link for the event http://lopsanj.org/events/picc10/ John J. Boris, Sr. JEN-A-SyS Administrator Archdiocese of Philadelphia "Remember! That light at the end of the tunnel Just might be the headlight of an oncoming train!" -- Download Intel® Parallel Studio Eval Try the new software tools for yourself. Speed compiling, find bugs proactively, and fine-tune applications for parallel performance. See why Intel Parallel Studio got high marks during beta. http://p.sf.net/sfu/intel-sw-dev ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
[Bacula-users] Bacula Enhancements Request
The online documentation indicates that enhancement requests should be posted to this group. I'd like to see a small number of enhancements. Many of these have been mentioned before, so please just consider this comment a "Me Too!" to add weight to those who decide what is next. - Clean/automatic support for check-pointed file systems (UFS2, ZFS, XFS, BtrFS, etc.) along the same lines as is done in Windows environments. This would make the FD a bit more OS-specific but it would be able to use the VSS (volume shadow copy) logic already developed. - Data staged to disk on the way to tape -- allowing backups to spool faster than tapes and when tape drives are full. - Tape verification option -- verify that the bits read back from the tape are the same as those originally written. Spooling to disk makes this easier. Also, if a tape fails verification, it should be easy to restart the write/verify cycle with new media. - Explicit capturing of boot and partitioning information for easier bare metal restores. (Bare metal restores should be easy and fast for all OSes, but that is easier said than done.) - Movement of per-client information to database from config files for easier management and reporting. This would reduce the hassle of adding new entries as new clients are brought online. And a couple of really low priority but nice to have ideas: - Optional system shutdown after backup -- ideal for desktop machines which would otherwise be idle for hours. - API for database backups -- backing up MySQL's files while live is bad. Mysqldump output is big. We need a 'better way.' - An officially supported bootable CD/DVD recovery disk which contains everything required to recover any x86/ x86_64 machine from bare metal in a semi-automatic fashion. -- Download Intel® Parallel Studio Eval Try the new software tools for yourself. Speed compiling, find bugs proactively, and fine-tune applications for parallel performance. See why Intel Parallel Studio got high marks during beta. http://p.sf.net/sfu/intel-sw-dev ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Backing up more than 200 servers
On 16/03/2010 15:33, Phil Stracchino wrote: > On 03/16/10 11:09, John Drescher wrote: >> On 03/16/10 10:30, Rodrigo Fernandes wrote: >>> My backup server has the bacula version 2.4.4-1 installed via tar.gz in a >>> Red Hat Enterprise Server 5.4 x86_64 >> >> You should really upgrade to 5.0.1 if possible. > > Yes, agreed. Unfortunately, 5.0.1 is not yet readily available on all > platforms except by building from source. I was looking at this the > other day to see whether it was feasible yet to upgrade all of my > machines, and finding that (for example) Debian and Ubuntu *still* have > no stable packages newer than 2.4.4-b1, although there are unstable > Bacula-5 .debs 5.0.1 transitioned into the Debian testing (squeeze) release on the 14th March. That is reported as being available for most (all?) supported Debian platforms. In addition it seems that 5.0.1 is now available in lenny-backports. http://packages.debian.org/search?keywords=bacula&searchon=names&suite=all§ion=all I haven't checked Ubuntu recently but now it is available in backports I'm going to start testing/moving over to 5.0.x on my setup. Regards Richard and Gentoo has nothing newer than 3.0.3 even in unstable. > It seems to be an uphill struggle to keep all the various OS > distributions up to date on Bacula. > > -- Download Intel® Parallel Studio Eval Try the new software tools for yourself. Speed compiling, find bugs proactively, and fine-tune applications for parallel performance. See why Intel Parallel Studio got high marks during beta. http://p.sf.net/sfu/intel-sw-dev ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Backing up more than 200 servers
On Tue, Mar 16, 2010 at 12:54 PM, John Drescher wrote: > On Tue, Mar 16, 2010 at 12:35 PM, Phil Stracchino > wrote: >> On 03/16/10 12:20, Phil Stracchino wrote: >>> On 03/16/10 12:09, John Drescher wrote: On Tue, Mar 16, 2010 at 12:03 PM, Phil Stracchino wrote: > Odd ... I have your overlay loaded, and I see 5.0.0 but not 5.0.1 I just pushed my changes 5 minutes ago. >>> >>> Ah! I'll run an eix-sync then. That'll leave me covered on everything >>> but the two xubuntu clients. >> >> Looks like it's going to take a little while to propagate. I still see >> only 5.0.0. If you do update to bacula-5.0.1 the default query.sql is now empty however there is an example query file in the doc/examples folder. Activating the examples was one of the recent changes I made to the ebuild. That file should be: /usr/share/doc/bacula-5.0.1/examples/sample-query.sql.bz2 John -- Download Intel® Parallel Studio Eval Try the new software tools for yourself. Speed compiling, find bugs proactively, and fine-tune applications for parallel performance. See why Intel Parallel Studio got high marks during beta. http://p.sf.net/sfu/intel-sw-dev ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Backing up more than 200 servers
On Tue, Mar 16, 2010 at 12:35 PM, Phil Stracchino wrote: > On 03/16/10 12:20, Phil Stracchino wrote: >> On 03/16/10 12:09, John Drescher wrote: >>> On Tue, Mar 16, 2010 at 12:03 PM, Phil Stracchino >>> wrote: Odd ... I have your overlay loaded, and I see 5.0.0 but not 5.0.1 >>> >>> I just pushed my changes 5 minutes ago. >> >> Ah! I'll run an eix-sync then. That'll leave me covered on everything >> but the two xubuntu clients. > > Looks like it's going to take a little while to propagate. I still see > only 5.0.0. > > BTW, how's the stability on that samba-3.5.1? > I have not done much testing with that. I do have that on the home network as well so I have only sent a few GB around between samba and windows 7 laptop and a VM. No problems however. At work I am using 3.4.5 on the PDC, BDC and a few member servers since I have just added 4 windows 7 machines. I usually do all my testing at home for a few weeks before I try to roll the changes to the work network.. John John -- Download Intel® Parallel Studio Eval Try the new software tools for yourself. Speed compiling, find bugs proactively, and fine-tune applications for parallel performance. See why Intel Parallel Studio got high marks during beta. http://p.sf.net/sfu/intel-sw-dev ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Backing up more than 200 servers
"Rodrigo Fernandes" kirjoitti viestissä news:ed2736a51003160845t40d1e9edgf2ff6fa4b3278...@mail.gmail.com... > John, > > The Fulls of the weekend are canceled because during the week I have more > than 200 servers (with small volume data) running full backups. When the > incremental backups of these servers starts during the weekend my > bacula-sd stop to responding (I'm using bconsole to check this) and my > full jobs of the weekend stop to work. I believe it is because of the > value "Max Concurrent Jobs" in bacula-dir equals "60". That setting should not cancel anything, it just should make new jobs to wait until there are less than 60 jobs running before they get started at all. Though cancellation might occur if wait times exceed limits set in the conf. Sounds like the bottleneck might be your Bacula server / database engine hw. No idea about your environment, but are there available free resources in machines running dir, sd and catalog at the time when "everything hungs" I don't have that size of system, but it might be helpful to know something more about the system... instead of just "200 servers", how much data to back up in full / incremental jobs? What backup speed can you get when it works? How big is the catalog, on which database engine? How much memory in dir, sd, catalog, what kind of cpus? What kind of media for the backups? > I think adjusting the values of "Max Concurrent Jobs" in bacula-dir and > bacula-fd to 200. In my bacula-sd I set this value to 10. This solves the > problem? Unless you have several SDs, that 10 is the limiting factor then? -- Timo -- Download Intel® Parallel Studio Eval Try the new software tools for yourself. Speed compiling, find bugs proactively, and fine-tune applications for parallel performance. See why Intel Parallel Studio got high marks during beta. http://p.sf.net/sfu/intel-sw-dev ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Backing up more than 200 servers
On 03/16/10 12:20, Phil Stracchino wrote: > On 03/16/10 12:09, John Drescher wrote: >> On Tue, Mar 16, 2010 at 12:03 PM, Phil Stracchino >> wrote: >>> Odd ... I have your overlay loaded, and I see 5.0.0 but not 5.0.1 >> >> I just pushed my changes 5 minutes ago. > > Ah! I'll run an eix-sync then. That'll leave me covered on everything > but the two xubuntu clients. Looks like it's going to take a little while to propagate. I still see only 5.0.0. BTW, how's the stability on that samba-3.5.1? -- Phil Stracchino, CDK#2 DoD#299792458 ICBM: 43.5607, -71.355 ala...@caerllewys.net ala...@metrocast.net p...@co.ordinate.org Renaissance Man, Unix ronin, Perl hacker, Free Stater It's not the years, it's the mileage. -- Download Intel® Parallel Studio Eval Try the new software tools for yourself. Speed compiling, find bugs proactively, and fine-tune applications for parallel performance. See why Intel Parallel Studio got high marks during beta. http://p.sf.net/sfu/intel-sw-dev ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
[Bacula-users] Problem with backup failing after 2 hours
Hi, I have a new installation that I am tweeking the backups on. One of my backups goes through the firewall from an public ip address to where the backup server is on an private ip address. The backup works just fine for the first 2h 11m 15s and then it fails. After the first test failed I inserted the 'Heartbeat Interval' option and set it for 30 sec. This slowed down the backup but I wanted to make sure that it continued through the entire disk. The disk that I am backing up has closed to 250GB of data on it. I get approx. 50GB backed up before it fails. Is there any other setting that will help this finish. The logs are here: First attempt: 15-Mar 10:26 distress-sd JobId 38: Labeled new Volume "hardware-0010" on device "FileStorage1" (/backup1). 15-Mar 10:26 distress-sd JobId 38: Wrote label to prelabeled Volume "hardware-0010" on device "FileStorage1" (/backup1) 15-Mar 12:37 distress-dir JobId 38: Fatal error: Network error with FD during Backup: ERR=Connection timed out 15-Mar 12:37 distress-sd JobId 38: JobId=38 Job="BackupHardware.2010-03-15_10.26.42_06" marked to be canceled. 15-Mar 12:37 distress-sd JobId 38: Job write elapsed time = 02:11:15, Transfer rate = 8.120 M Bytes/second 15-Mar 12:37 distress-sd JobId 38: Error: bsock.c:529 Read expected 65536 got 14596 from client:70.99.222.36:36643 15-Mar 12:37 distress-dir JobId 38: Fatal error: No Job status returned from FD. 15-Mar 12:37 distress-dir JobId 38: Error: Bacula distress-dir 5.0.1 (24Feb10): 15-Mar-2010 12:37:59 Build OS: x86_64-unknown-linux-gnu redhat JobId: 38 Job:BackupHardware.2010-03-15_10.26.42_06 Backup Level: Full (upgraded from Incremental) Client: "swift-fd" 5.0.1 (24Feb10) sparc-sun-solaris2.10,solaris,5.10 FileSet:"Swift Hardware Set" 2010-03-10 11:30:53 Pool: "Pool1" (From Job resource) Catalog:"MyCatalog" (From Client resource) Storage:"File1" (From command line) Scheduled time: 15-Mar-2010 10:26:34 Start time: 15-Mar-2010 10:26:44 End time: 15-Mar-2010 12:37:59 Elapsed time: 2 hours 11 mins 15 secs Priority: 10 FD Files Written: 0 SD Files Written: 707,122 FD Bytes Written: 0 (0 B) SD Bytes Written: 63,952,326,117 (63.95 GB) Rate: 0.0 KB/s Software Compression: None VSS:no Encryption: no Accurate: no Volume name(s): hardware-0010 Volume Session Id: 1 Volume Session Time:1268673669 Last Volume Bytes: 64,023,383,638 (64.02 GB) Non-fatal FD errors:0 SD Errors: 1 FD termination status: Error SD termination status: Canceled Termination:*** Backup Error *** Second attempt: 15-Mar 17:14 distress-dir JobId 40: Fatal error: Network error with FD during Backup: ERR=Connection timed out 15-Mar 17:14 distress-sd JobId 40: JobId=40 Job="BackupHardware.2010-03-15_15.02.58_10" marked to be canceled. 15-Mar 17:14 distress-sd JobId 40: Job write elapsed time = 01:57:35, Transfer rate = 7.432 M Bytes/second 15-Mar 17:14 distress-dir JobId 40: Fatal error: No Job 15-Mar 17:14 distress-dir JobId 40: Error: Bacula distress-dir 5.0.1 (24Feb10): 15-Mar-2010 17:14:15 Build OS: x86_64-unknown-linux-gnu redhat JobId: 40 Job:BackupHardware.2010-03-15_15.02.58_10 Backup Level: Full (upgraded from Incremental) Client: "swift-fd" 5.0.1 (24Feb10) sparc-sun-solaris2.10,solaris,5.10 FileSet:"Swift Hardware Set" 2010-03-10 11:30:53 Pool: "Pool1" (From Job resource) Catalog:"MyCatalog" (From Client resource) Storage:"File1" (From command line) Scheduled time: 15-Mar-2010 15:02:57 Start time: 15-Mar-2010 15:03:00 End time: 15-Mar-2010 17:14:15 Elapsed time: 2 hours 11 mins 15 secs Priority: 10 FD Files Written: 0 SD Files Written: 0 FD Bytes Written: 0 (0 B) SD Bytes Written: 0 (0 B) Rate: 0.0 KB/s Software Compression: None VSS:no Encryption: no Accurate: no Volume name(s): hardware-0012 Volume Session Id: 3 Volume Session Time:1268673669 Last Volume Bytes: 51,996,669,624 (51.99 GB) Non-fatal FD errors:0 SD Errors: 0 FD termination status: Error SD termination status: Running Termination:*** Backup Error *** status returned from FD. thanks, -- --- Jerold Lowry IT Manager / Software Engineer Engineering Design Team (EDT), Inc. a HEICO company 1400
Re: [Bacula-users] Backing up more than 200 servers
On 03/16/10 12:09, John Drescher wrote: > On Tue, Mar 16, 2010 at 12:03 PM, Phil Stracchino > wrote: >> Odd ... I have your overlay loaded, and I see 5.0.0 but not 5.0.1 > > I just pushed my changes 5 minutes ago. Ah! I'll run an eix-sync then. That'll leave me covered on everything but the two xubuntu clients. -- Phil Stracchino, CDK#2 DoD#299792458 ICBM: 43.5607, -71.355 ala...@caerllewys.net ala...@metrocast.net p...@co.ordinate.org Renaissance Man, Unix ronin, Perl hacker, Free Stater It's not the years, it's the mileage. -- Download Intel® Parallel Studio Eval Try the new software tools for yourself. Speed compiling, find bugs proactively, and fine-tune applications for parallel performance. See why Intel Parallel Studio got high marks during beta. http://p.sf.net/sfu/intel-sw-dev ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Backing up more than 200 servers
On Tue, Mar 16, 2010 at 12:03 PM, Phil Stracchino wrote:
> On 03/16/10 11:54, John Drescher wrote:
>>> Gentoo has nothing newer than 3.0.3 even in unstable.
>>> It seems to be an uphill struggle to keep all the various OS
>>> distributions up to date on Bacula.
>>>
>> I have an ebuild for 5.0.1 here:
>>
>> http://github.com/drescherjm/jmdgentoooverlay/tree/master/app-backup/bacula/
>>
>> This is pretty much just a package bump of the 3.0.3 portage ebuild. I
>> use it at home but I have not gone to production at work yet.
>
> Odd ... I have your overlay loaded, and I see 5.0.0 but not 5.0.1
>
I just pushed my changes 5 minutes ago.
Changes ready to be committed ("staged"):
modified: bacula/Manifest
modified: bacula/bacula-5.0.1.ebuild
modified: ../net-fs/samba-client/Manifest
new file: ../net-fs/samba-client/samba-client-3.4.7.ebuild
new file: ../net-fs/samba-client/samba-client-3.5.1.ebuild
modified: ../net-fs/samba-libs/Manifest
new file: ../net-fs/samba-libs/samba-libs-3.4.7.ebuild
new file: ../net-fs/samba-libs/samba-libs-3.5.1.ebuild
modified: ../net-fs/samba-server/Manifest
new file: ../net-fs/samba-server/samba-server-3.4.7.ebuild
new file: ../net-fs/samba-server/samba-server-3.5.1.ebuild
modified: ../net-fs/samba/Manifest
new file: ../net-fs/samba/samba-3.5.1.ebuild
John
--
Download Intel® Parallel Studio Eval
Try the new software tools for yourself. Speed compiling, find bugs
proactively, and fine-tune applications for parallel performance.
See why Intel Parallel Studio got high marks during beta.
http://p.sf.net/sfu/intel-sw-dev
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Backing up more than 200 servers
On 03/16/10 11:54, John Drescher wrote: >> Gentoo has nothing newer than 3.0.3 even in unstable. >> It seems to be an uphill struggle to keep all the various OS >> distributions up to date on Bacula. >> > I have an ebuild for 5.0.1 here: > > http://github.com/drescherjm/jmdgentoooverlay/tree/master/app-backup/bacula/ > > This is pretty much just a package bump of the 3.0.3 portage ebuild. I > use it at home but I have not gone to production at work yet. Odd ... I have your overlay loaded, and I see 5.0.0 but not 5.0.1 -- Phil Stracchino, CDK#2 DoD#299792458 ICBM: 43.5607, -71.355 ala...@caerllewys.net ala...@metrocast.net p...@co.ordinate.org Renaissance Man, Unix ronin, Perl hacker, Free Stater It's not the years, it's the mileage. -- Download Intel® Parallel Studio Eval Try the new software tools for yourself. Speed compiling, find bugs proactively, and fine-tune applications for parallel performance. See why Intel Parallel Studio got high marks during beta. http://p.sf.net/sfu/intel-sw-dev ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Backing up more than 200 servers
> Gentoo has nothing newer than 3.0.3 even in unstable. > It seems to be an uphill struggle to keep all the various OS > distributions up to date on Bacula. > I have an ebuild for 5.0.1 here: http://github.com/drescherjm/jmdgentoooverlay/tree/master/app-backup/bacula/ This is pretty much just a package bump of the 3.0.3 portage ebuild. I use it at home but I have not gone to production at work yet. John -- Download Intel® Parallel Studio Eval Try the new software tools for yourself. Speed compiling, find bugs proactively, and fine-tune applications for parallel performance. See why Intel Parallel Studio got high marks during beta. http://p.sf.net/sfu/intel-sw-dev ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Backing up more than 200 servers
John, The Fulls of the weekend are canceled because during the week I have more than 200 servers (with small volume data) running full backups. When the incremental backups of these servers starts during the weekend my bacula-sd stop to responding (I'm using bconsole to check this) and my full jobs of the weekend stop to work. I believe it is because of the value "Max Concurrent Jobs" in bacula-dir equals "60". My question is: How do full backups during the week (starting on Tuesday, Wednesday and Thursday) and the incremental backups on Saturday and Sunday without generating problems on fulls backups that run during the weekend? I think adjusting the values of "Max Concurrent Jobs" in bacula-dir and bacula-fd to 200. In my bacula-sd I set this value to 10. This solves the problem? I have bacula-client installed on 200 servers Thanks, Rodrigo On Tue, Mar 16, 2010 at 12:09 PM, John Drescher wrote: > > I wonder if someone on the list have knowledge about how to climb the > bacula > > to backup over 200 servers? It´s is possible? > > > > Yes of course. Other bacula users on the list do that. I have heard > reports of 1000 machines being backed up by bacula. > > > Currently, my "Max Concurrent Jobs" on Bacula-dir, bacula-fd is equal to > 60 > > and bacula-sd is equal to 10. > > > > I have 50 servers that run the full backup over the weekend (due to the > > large volume of data) and servers with smaller volume of data run the > full > > backup during the week. The problem is that due to the large number of > > servers (more than 200 during the week) the incremental backup ends up > > creating problems during the weekend, causing backups Full canceled due > to > > problems with bacula-dir (because of the variable max concurrent jobs of > > the file bacula-dir exceeded the 60 jobs). > > > > How does full get canceled? I am confused. > > > > > I was looking at the list of bacula and found an email from Phil > Stracchino > > explaining "how the concurrency jobs works". > > > > It's really necessary to create more than 1 bacula-fd to control the > jobs? > > (How can I do that. I can use any port?). > > I did not look at the Phil Stracchino. Don't you have bacula installed > on all 200 machines? > > >I think to set the "Max Concurrent > > Jobs" in bacula-dir and bacula-fd to 100 and limit the "Max Concurrent > Jobs" > > in bacula-sd to the number of jobs that I really need, considering the > > problem with incremental backup. > > > > Can anybody have some idea about how to resolve this issue? > > > > My backup server has the bacula version 2.4.4-1 installed via tar.gz in a > > Red Hat Enterprise Server 5.4 x86_64 > > > > You should really upgrade to 5.0.1 if possible. > > > > > I appreciate any help right now and apologize for the long text. > > > > Rodrigo Fernandes > > > > > -- > > Download Intel® Parallel Studio Eval > > Try the new software tools for yourself. Speed compiling, find bugs > > proactively, and fine-tune applications for parallel performance. > > See why Intel Parallel Studio got high marks during beta. > > http://p.sf.net/sfu/intel-sw-dev > > ___ > > Bacula-users mailing list > > Bacula-users@lists.sourceforge.net > > https://lists.sourceforge.net/lists/listinfo/bacula-users > > > > > > > > -- > John M. Drescher > -- Download Intel® Parallel Studio Eval Try the new software tools for yourself. Speed compiling, find bugs proactively, and fine-tune applications for parallel performance. See why Intel Parallel Studio got high marks during beta. http://p.sf.net/sfu/intel-sw-dev___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] bacula backuping on wrong network interface
Enable the firewall, so only the internal IP is allowed to accede that
server on those ports
Ciao
---
Carlo Filippetto
2010/3/16 Radosław Korzeniewski :
> Hello,
>
> Check if SD is listening on private network address, using telnet, for
> example - check from client (FD) machine. Next, check if Bacula SD
> configuration uses private address (on private network) instead of public.
> And finally check communication from FD to SD using traceroute and private
> SD address...
>
> regards
>
> Radek
>
> 2010/3/16 Laurent HENRY (EHESS/CRI)
>>
>> Hello all,
>>
>> I have a problem to understandhow to configure bacula to make
>> backups on different network interfaces.
>>
>> My config:
>>
>> - backuped server (server1) with 2 network cards, one for internet traffic
>> (public IP), one for backups (private IP).
>> - bacula server (2.4 on debian) with 2 network cards. One private IP for
>> backups and one public IP for other backups elsewhere.
>>
>> In my bacula-dir config
>> Client {
>> Name = server1-fd
>> Address = 10.2.0.21
>> FDPort = 9102
>> ...
>> }
>>
>> I don't know how but while doing tcpdump during backups, i am noticing
>> bacula network traffic is between my 2 public interfaces and there is no
>> flow at all on my privates cards.
>>
>>
>> Could anyone know where i can take a look ?
>> I've never configured the public IP of the backuped server i don't
>> understand how bacula find it (and that is not what i want him to do)
>>
>> Thank you for any clue.
>>
>>
>>
>>
>> --
>> Download Intel® Parallel Studio Eval
>> Try the new software tools for yourself. Speed compiling, find bugs
>> proactively, and fine-tune applications for parallel performance.
>> See why Intel Parallel Studio got high marks during beta.
>> http://p.sf.net/sfu/intel-sw-dev
>> ___
>> Bacula-users mailing list
>> Bacula-users@lists.sourceforge.net
>> https://lists.sourceforge.net/lists/listinfo/bacula-users
>
>
>
> --
> Radosław Korzeniewski
> rados...@korzeniewski.net
>
> --
> Download Intel® Parallel Studio Eval
> Try the new software tools for yourself. Speed compiling, find bugs
> proactively, and fine-tune applications for parallel performance.
> See why Intel Parallel Studio got high marks during beta.
> http://p.sf.net/sfu/intel-sw-dev
> ___
> Bacula-users mailing list
> Bacula-users@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/bacula-users
>
>
--
Download Intel® Parallel Studio Eval
Try the new software tools for yourself. Speed compiling, find bugs
proactively, and fine-tune applications for parallel performance.
See why Intel Parallel Studio got high marks during beta.
http://p.sf.net/sfu/intel-sw-dev
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Backing up more than 200 servers
On 03/16/10 11:09, John Drescher wrote: > On 03/16/10 10:30, Rodrigo Fernandes wrote: >> My backup server has the bacula version 2.4.4-1 installed via tar.gz in a >> Red Hat Enterprise Server 5.4 x86_64 > > You should really upgrade to 5.0.1 if possible. Yes, agreed. Unfortunately, 5.0.1 is not yet readily available on all platforms except by building from source. I was looking at this the other day to see whether it was feasible yet to upgrade all of my machines, and finding that (for example) Debian and Ubuntu *still* have no stable packages newer than 2.4.4-b1, although there are unstable Bacula-5 .debs and Gentoo has nothing newer than 3.0.3 even in unstable. It seems to be an uphill struggle to keep all the various OS distributions up to date on Bacula. -- Phil Stracchino, CDK#2 DoD#299792458 ICBM: 43.5607, -71.355 ala...@caerllewys.net ala...@metrocast.net p...@co.ordinate.org Renaissance Man, Unix ronin, Perl hacker, Free Stater It's not the years, it's the mileage. -- Download Intel® Parallel Studio Eval Try the new software tools for yourself. Speed compiling, find bugs proactively, and fine-tune applications for parallel performance. See why Intel Parallel Studio got high marks during beta. http://p.sf.net/sfu/intel-sw-dev ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Backing up more than 200 servers
On 03/16/10 10:30, Rodrigo Fernandes wrote: > It's really necessary to create more than 1 bacula-fd to control the > jobs? (How can I do that. I can use any port?). I think to set the "Max > Concurrent Jobs" in bacula-dir and bacula-fd to 100 and limit the "Max > Concurrent Jobs" in bacula-sd to the number of jobs that I really need, > considering the problem with incremental backup. You don't have to have multiple FDs. In particular, there is little point in having multiple FDs on a single host. Doing so generally will not gain you anything. In your position, if you have no other constraints, try simply setting the concurrency to the levels required to support all of your clients. You may find, after a first test run, that you need to do additional tuning to manage the load, possibly by dividing the clients into groups with staggered startup times or staggered priorities. But there is no one-size-fits-all universal answer for large deployments. What will work best for you is heavily dependent on what you have to work with. For instance, in a hypothetical situation where you're backing up to a large SAN with extremely high throughput, you may find that you are CPU or network bound on a single FD while having plenty of I/O capacity left on the SAN, and in such a case it may be beneficial to set up multiple independent FDs on different servers, all using the same SAN as their storage backend. As another example, if it turns out that you are I/O or CPU bound on your Catalog database, you may need to establish multiple Catalogs on different hosts and spread the clients across them. -- Phil Stracchino, CDK#2 DoD#299792458 ICBM: 43.5607, -71.355 ala...@caerllewys.net ala...@metrocast.net p...@co.ordinate.org Renaissance Man, Unix ronin, Perl hacker, Free Stater It's not the years, it's the mileage. -- Download Intel® Parallel Studio Eval Try the new software tools for yourself. Speed compiling, find bugs proactively, and fine-tune applications for parallel performance. See why Intel Parallel Studio got high marks during beta. http://p.sf.net/sfu/intel-sw-dev ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Backing up more than 200 servers
> I wonder if someone on the list have knowledge about how to climb the bacula > to backup over 200 servers? It´s is possible? > Yes of course. Other bacula users on the list do that. I have heard reports of 1000 machines being backed up by bacula. > Currently, my "Max Concurrent Jobs" on Bacula-dir, bacula-fd is equal to 60 > and bacula-sd is equal to 10. > > I have 50 servers that run the full backup over the weekend (due to the > large volume of data) and servers with smaller volume of data run the full > backup during the week. The problem is that due to the large number of > servers (more than 200 during the week) the incremental backup ends up > creating problems during the weekend, causing backups Full canceled due to > problems with bacula-dir (because of the variable max concurrent jobs of > the file bacula-dir exceeded the 60 jobs). > How does full get canceled? I am confused. > > I was looking at the list of bacula and found an email from Phil Stracchino > explaining "how the concurrency jobs works". > > It's really necessary to create more than 1 bacula-fd to control the jobs? > (How can I do that. I can use any port?). I did not look at the Phil Stracchino. Don't you have bacula installed on all 200 machines? >I think to set the "Max Concurrent > Jobs" in bacula-dir and bacula-fd to 100 and limit the "Max Concurrent Jobs" > in bacula-sd to the number of jobs that I really need, considering the > problem with incremental backup. > > Can anybody have some idea about how to resolve this issue? > > My backup server has the bacula version 2.4.4-1 installed via tar.gz in a > Red Hat Enterprise Server 5.4 x86_64 > You should really upgrade to 5.0.1 if possible. > > I appreciate any help right now and apologize for the long text. > > Rodrigo Fernandes > > -- > Download Intel® Parallel Studio Eval > Try the new software tools for yourself. Speed compiling, find bugs > proactively, and fine-tune applications for parallel performance. > See why Intel Parallel Studio got high marks during beta. > http://p.sf.net/sfu/intel-sw-dev > ___ > Bacula-users mailing list > Bacula-users@lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/bacula-users > > -- John M. Drescher -- Download Intel® Parallel Studio Eval Try the new software tools for yourself. Speed compiling, find bugs proactively, and fine-tune applications for parallel performance. See why Intel Parallel Studio got high marks during beta. http://p.sf.net/sfu/intel-sw-dev ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] bacula backuping on wrong network interface
Hello,
Check if SD is listening on private network address, using telnet, for
example - check from client (FD) machine. Next, check if Bacula SD
configuration uses private address (on private network) instead of public.
And finally check communication from FD to SD using traceroute and private
SD address...
regards
Radek
2010/3/16 Laurent HENRY (EHESS/CRI)
> Hello all,
>
> I have a problem to understandhow to configure bacula to make
> backups on different network interfaces.
>
> My config:
>
> - backuped server (server1) with 2 network cards, one for internet traffic
> (public IP), one for backups (private IP).
> - bacula server (2.4 on debian) with 2 network cards. One private IP for
> backups and one public IP for other backups elsewhere.
>
> In my bacula-dir config
> Client {
> Name = server1-fd
> Address = 10.2.0.21
> FDPort = 9102
> ...
> }
>
> I don't know how but while doing tcpdump during backups, i am noticing
> bacula network traffic is between my 2 public interfaces and there is no
> flow at all on my privates cards.
>
>
> Could anyone know where i can take a look ?
> I've never configured the public IP of the backuped server i don't
> understand how bacula find it (and that is not what i want him to do)
>
> Thank you for any clue.
>
>
>
>
> --
> Download Intel® Parallel Studio Eval
> Try the new software tools for yourself. Speed compiling, find bugs
> proactively, and fine-tune applications for parallel performance.
> See why Intel Parallel Studio got high marks during beta.
> http://p.sf.net/sfu/intel-sw-dev
> ___
> Bacula-users mailing list
> Bacula-users@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/bacula-users
>
--
Radosław Korzeniewski
rados...@korzeniewski.net
--
Download Intel® Parallel Studio Eval
Try the new software tools for yourself. Speed compiling, find bugs
proactively, and fine-tune applications for parallel performance.
See why Intel Parallel Studio got high marks during beta.
http://p.sf.net/sfu/intel-sw-dev___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users
[Bacula-users] Bacula 5.0.1 Server installation for Win32 ?
Hello, Can I expect a server installation of bacula in version 5.0.1 for Win32 ? When I start the install only client installation is possible. There is a release date of Win32 server installation in 5.0.1 ? Or have I to compile source with cygwin ? POMMIER Emmanuel Responsable Infrastructure Groupe AYME Mail: pommie...@ayme.fr Tel: 04.90.63.99.53 Fax: 04.90.63.99.01 -- Download Intel® Parallel Studio Eval Try the new software tools for yourself. Speed compiling, find bugs proactively, and fine-tune applications for parallel performance. See why Intel Parallel Studio got high marks during beta. http://p.sf.net/sfu/intel-sw-dev___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
[Bacula-users] Backing up more than 200 servers
I wonder if someone on the list have knowledge about how to climb the bacula to backup over 200 servers? It´s is possible? Currently, my "Max Concurrent Jobs" on Bacula-dir, bacula-fd is equal to 60 and bacula-sd is equal to 10. I have 50 servers that run the full backup over the weekend (due to the large volume of data) and servers with smaller volume of data run the full backup during the week. The problem is that due to the large number of servers (more than 200 during the week) the incremental backup ends up creating problems during the weekend, causing backups Full canceled due to problems with bacula-dir (because of the variable max concurrent jobs of the file bacula-dir exceeded the 60 jobs). I was looking at the list of bacula and found an email from Phil Stracchinoexplaining "how the concurrency jobs works". It's really necessary to create more than 1 bacula-fd to control the jobs? (How can I do that. I can use any port?). I think to set the "Max Concurrent Jobs" in bacula-dir and bacula-fd to 100 and limit the "Max Concurrent Jobs" in bacula-sd to the number of jobs that I really need, considering the problem with incremental backup. Can anybody have some idea about how to resolve this issue? My backup server has the bacula version 2.4.4-1 installed via tar.gz in a Red Hat Enterprise Server 5.4 x86_64 I appreciate any help right now and apologize for the long text. Rodrigo Fernandes -- Download Intel® Parallel Studio Eval Try the new software tools for yourself. Speed compiling, find bugs proactively, and fine-tune applications for parallel performance. See why Intel Parallel Studio got high marks during beta. http://p.sf.net/sfu/intel-sw-dev___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] How to re-use existing volumes in pool?
Am Mon, 15 Mar 2010 14:16:19 -0500 schrieb Sean Carolan: > Due to disk space limitations I would like to re-use the same volumes > each week for a single full backup. Every time I try to run a new job, > I get an error stating that bacula cannot find any append-able volumes. > It has been over six days since my last backup. Anyone know how to make > bacula re-use the same volumes, overwriting or deleting the old backups > each week? maybe change VolumeRetention to 5 days and do an "update volumes from resource" (only changing *.conf won't do the trick). Also change "Use volume once" to "Maximum Volume Jobs = 1 " http://www.bacula.org/5.0.x-manuals/en/main/main/Configuring_Director.html#SECTION001715 - Thomas -- Download Intel® Parallel Studio Eval Try the new software tools for yourself. Speed compiling, find bugs proactively, and fine-tune applications for parallel performance. See why Intel Parallel Studio got high marks during beta. http://p.sf.net/sfu/intel-sw-dev ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
[Bacula-users] Part of Jobs was canceled
Hi, i have new bacula 5.0.1 (new installation) and problem with canceled jobs. From 50 Clients a few don't finish a job (from console command run and Schedule). On bacula 2.4.4 on the same configuration all was ok. My OS is debian and gentoo. Logs from job: rysio-dir Start Backup JobId 54, Job=zubr-configuration.2010-03-16_12.42.50_21 Using Device "FileStorage" rysio-sd Volume "Routers-Configuration-0001" previously written, moving to end of data. Ready to append to end of Volume "Routers-Configuration-0001" size=73459463 rysio-sd JobId=54 Job="zubr-configuration.2010-03-16_12.42.50_21" marked to be canceled. Job write elapsed time = 00:15:41, Transfer rate = 0 Bytes/second rysio-dir Error: Bacula rysio-dir 5.0.1 (24Feb10): 16-Mar-2010 12:58:33 Build OS: i686-pc-linux-gnu debian 4.0 JobId: 54 Job:zubr-configuration.2010-03-16_12.42.50_21 Backup Level: Full Client: "zubr-fd" 5.0.1 (24Feb10) i686-pc-linux-gnu,gentoo, FileSet:"Full Set Configuration For Routers" 2010-03-15 14:55:00 Pool: "Routers-Configuration" (From Job resource) Catalog:"MyCatalog" (From Client resource) Storage:"FileStorage_169_254_166" (From Pool resource) Scheduled time: 16-Mar-2010 12:42:40 Start time: 16-Mar-2010 12:42:52 End time: 16-Mar-2010 12:58:33 Elapsed time: 15 mins 41 secs Priority: 11 FD Files Written: 80 SD Files Written: 1 FD Bytes Written: 88,332 (88.33 KB) SD Bytes Written: 0 (0 B) Rate: 0.1 KB/s Software Compression: 85.2 % VSS:no Encryption: no Accurate: no Volume name(s): Volume Session Id: 54 Volume Session Time:1268644987 Last Volume Bytes: 73,459,463 (73.45 MB) Non-fatal FD errors:0 SD Errors: 0 FD termination status: Error SD termination status: Canceled Termination:*** Backup Error *** zubr-fd Fatal error: backup.c:1019 Network send error to SD. ERR=Connection timed out -- Regards,, Krzysztof Pawlikiewicz -- Download Intel® Parallel Studio Eval Try the new software tools for yourself. Speed compiling, find bugs proactively, and fine-tune applications for parallel performance. See why Intel Parallel Studio got high marks during beta. http://p.sf.net/sfu/intel-sw-dev ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Bacula 3.0.2 and Windows 2003 Server connection problem
Hello
But you can see client status on bconsole??
Please, type on bconsole "status client=client-fd" and show us the result
On Sun, Mar 14, 2010 at 6:55 PM, daveb93 wrote:
>
> Hello All ...
> This is my first post here (but likely not the last :D ) I manage several
> bacula installs all with shall we say issues ...
> Here is my PITA for today.
>
> Bacula Director on Ubuntu Server:
> Version: 3.0.2 (18 July 2009) x86_64-pc-linux-gnu ubuntu 10.04 (<-- not
> sure why 10.04 but I digress)
> It is up and running correctly
>
> I attempt to connect to the client (the windows 2003 server) I get the
> following error from bconsole
>
> JobId 0: Fatal error: Unable to authenticate with File daemon at
> "xxx.xxx.xxx.xxx:9102". Possible causes:
> Passwords or names not the same or
> Maximum Concurrent Jobs exceeded on the FD or
> FD networking messed up (restart daemon).
> Please see http://
> www.bacula.org/en/rel-manual/Bacula_Freque_Asked_Questi.html#SECTION00376for
> help.
>
>
> I can telnet to the client with success on port 9102
>
> turning tracing on from bconsole gives me this nugget
>
> JobId 0: Fatal error: Error sending Hello to File daemon at
> "xxx.xxx.xxx.xxx:9102". ERR=Connection reset by peer
> 14-Mar 12:08 server-dir JobId 0: Error: bsock.c:374 Write
> error sending 45 bytes to Client:
> client-fd:xxx.xxx.xxx.xxx:9102: ERR=Connection reset by peer
>
> The client fd.conf looks like this (pretty standard)
>
> # "Global" File daemon configuration specifications
> #
> FileDaemon {
> # this is me
> Name = client-fd
> FDport = 9102
> # where we listen for the director
> WorkingDirectory = "C:\\Documents and Settings\\All
> Users\\Application Data\\Bacula\\work"
> Pid Directory = "C:\\Documents and Settings\\All
> Users\\Application Data\\Bacula\\work"
> #Plugin Directory = "C:\\Program Files\\Bacula\\bin\\fdplugins"
> Maximum Concurrent Jobs = 20
> }
>
> #
> # List Directors who are permitted to contact this File daemon
> #
> Director {
> Name = server-fd
> Password = "std-password"
> }
>
> #
> # Restricted Director, used by tray-monitor to get the
> # status of the file daemon
> #
> Director {
> Name = 2650server-mon
> Password = "std-password"
> Monitor = yes
> }
>
> # Send all messages except skipped files back to Director
> Messages {
> Name = Standard
> director = server-fd = all, !skipped, !restored
> }
>
>
> And the client section on the server bacula-dir.conf looks like this:
>
> Client {
> Name = client-fd
> Password = "std-password"
> Address = xxx.xxx.xxx.xxx
> FDPort = 9102
> Catalog = MyCatalog
> File Retention = 21 days
> Job Retention = 6 months
> }
>
>
> Everything "looks" right to me ... but I cannot get the darn thing to
> connect ... and no google joy on any similar issues
>
> any ideas ??
> Thanks
>
> +--
> |This was sent by daveb93_2...@yahoo.com via Backup Central.
> |Forward SPAM to ab...@backupcentral.com.
> +--
>
>
>
>
> --
> Download Intel® Parallel Studio Eval
> Try the new software tools for yourself. Speed compiling, find bugs
> proactively, and fine-tune applications for parallel performance.
> See why Intel Parallel Studio got high marks during beta.
> http://p.sf.net/sfu/intel-sw-dev
> ___
> Bacula-users mailing list
> Bacula-users@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/bacula-users
>
--
Download Intel® Parallel Studio Eval
Try the new software tools for yourself. Speed compiling, find bugs
proactively, and fine-tune applications for parallel performance.
See why Intel Parallel Studio got high marks during beta.
http://p.sf.net/sfu/intel-sw-dev___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users
[Bacula-users] bacula backuping on wrong network interface
Hello all,
I have a problem to understandhow to configure bacula to make
backups on different network interfaces.
My config:
- backuped server (server1) with 2 network cards, one for internet traffic
(public IP), one for backups (private IP).
- bacula server (2.4 on debian) with 2 network cards. One private IP for
backups and one public IP for other backups elsewhere.
In my bacula-dir config
Client {
Name = server1-fd
Address = 10.2.0.21
FDPort = 9102
...
}
I don't know how but while doing tcpdump during backups, i am noticing
bacula network traffic is between my 2 public interfaces and there is no
flow at all on my privates cards.
Could anyone know where i can take a look ?
I've never configured the public IP of the backuped server i don't
understand how bacula find it (and that is not what i want him to do)
Thank you for any clue.
--
Download Intel® Parallel Studio Eval
Try the new software tools for yourself. Speed compiling, find bugs
proactively, and fine-tune applications for parallel performance.
See why Intel Parallel Studio got high marks during beta.
http://p.sf.net/sfu/intel-sw-dev
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Marking all appendable volumes "used", comments on script?
Am 15.3.2010 18:02, schrieb Jari Fredriksson:
> On 15.3.2010 13:02, Administrator wrote:
>
>> Hi
>> somebody pointed me to shell scripting bacula in order to mark all
>> volumes used. Long users of bacula will certainly not find this helpfull
>> but well .. hopefully this is not starting a relevance debate ;-)
>>
>> I backup on two external USB-Disks which i rotate every Week (Monday). I
>> need to mark all appendable volumes as used and did not want to use the
>> volchanger script. Instead i now query the mysql db for volumes with
>> sthe status "append" and tell bconsole to mark these as used:
>> ===
>> #!/bin/sh
>> MYSQL=/usr/bin/mysql
>> BACULA_DB=bacula5
>> BACULA_USER=bacula5
>> BACULA_PASS=not24get
>> BCONSOLE=/opt/bacula5/etc/bconsole
>>
>> openVolumes=$(${MYSQL} -u ${BACULA_USER} -p${BACULA_PASS} ${BACULA_DB}
>> -sN -e "select VolumeName from Media where VolStatus='Append'")
>>
>> rm -f updateVolumes.bacula
>> for volumeName in ${openVolumes}
>> do
>> echo "update volume=${volumeName} volstatus=used">>
>> updateVolumes.bacula
>> done
>> ${BCONSOLE}< updateVolumes.bacula
>> ==
>> I could use bconsole to query for open volumes but that would involve
>> some parsing.
>>
>> Criticism is welcome
>>
> If and when you use MySQL anyway, why not just
>
> mysql -u ${BACULA_USER} -p${BACULA_PASS} ${BACULA_DB} \
> -e "update Media set VolStatus='Used' where VolStatus='Append'"
>
Thanks for comment. I am not sure if this suffice. Maybe bacula is
writing some other attributes under certain cirkumstances in any other
table? Reading the DB should have no side effects, but saves me some
parsing. Are you sure that updating this attribute is enough and does
not show side effects?
M. Müller
--
Download Intel® Parallel Studio Eval
Try the new software tools for yourself. Speed compiling, find bugs
proactively, and fine-tune applications for parallel performance.
See why Intel Parallel Studio got high marks during beta.
http://p.sf.net/sfu/intel-sw-dev
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users
[Bacula-users] Bacula Incremental Backups
Hi Guys, I'm experiencing a weird problem with my incremental backups of file server data residing on a SLES 9 Samba server. I'm running the Bacula services on a CentOS 5.4 box, version 3.0.2. My full backup is around 600 GB after compression and this runs fine. The first incremental or differential job runs fine after the Full backup with ~10Gb of data. The incremental jobs that follow then tries to a backup, what looks like, everything all over again. I have to cancel it at ~150 GB though because of disk space constraints. So if a incremental backup job backs-up 10 GB of data, isn't it safe to assume that the next incremental will be less than 10 GB? I've tried both mtimeonly=yes and noatime=yes without success. Any help or tips would be greatly appreciated :) +-- |This was sent by ech...@gmail.com via Backup Central. |Forward SPAM to ab...@backupcentral.com. +-- -- Download Intel® Parallel Studio Eval Try the new software tools for yourself. Speed compiling, find bugs proactively, and fine-tune applications for parallel performance. See why Intel Parallel Studio got high marks during beta. http://p.sf.net/sfu/intel-sw-dev ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Bacula-5.0.1 with self-signed certificates
Take a look here:
http://www.bacula.org/en/dev-manual/Bacula_TLS_Communication.html
(search for "Self-signed Cerftificate").
I´m not sure if "TLS Verify Peer" would work in your situation (since CA
= Your-Server?). However if everything else is right (bacula-sd?) I
guess it should work if you create your own CA somewhere else and sign
your cerftificates yourself. Bacula is working well with SSL (even if
you would not pay for signing certificates) ;)
Greetings,
user100
Am 16.03.2010 02:08, schrieb Lamp Zy:
> I have bacula-5.0.1 compiled with openssl support and installed on
> CentOS5.4(32bit).
>
> All bacula daemons run on the same server - director, storage and file
> daemon. I can successfully backup local files and directories.
>
> The problem is when I try to setup tls encryption (at some point I'll
> backup systems over the network). The error I'm getting is:
> ERR=18:self signed certificate
>
> In bacula-dir.conf:
>
> Client {
> Name = backupserver.domain.com-fd
> Address = backupserver.domain.com
> ...
> # Request encrypted communication with the client
> TLS Enable = yes
> TLS Require = yes
> TLS CA Certificate File = /etc/pki/tls/certs/cert.pem
> TLS Certificate = /etc/pki/tls/certs/backupserver.domain.com.crt
> TLS Key = /etc/pki/tls/certs/backupserver.domain.com.key
> }
>
> In bacula-fd.conf:
>
> Director {
> Name = backupserver.domain.com-dir
> ...
> # Request encrypted communication with the server
> TLS Enable = yes
> TLS Require = yes
> TLS Verify Peer = yes
> TLS Allowed CN = "backupserver.domain.com"
> TLS CA Certificate File = /etc/pki/tls/certs/cert.pem
> TLS Certificate = /etc/pki/tls/certs/backupserver.domain.com.crt
> TLS Key = /etc/pki/tls/certs/backupserver.domain.com.key
> }
>
> Here is the full message on the console:
>
> 15-Mar 16:47 backupserver.domain.com-dir JobId 0: Error: tls.c:92 Error
> with certificate at depth: 0, issuer = /C=US/ST=California/L=San
> Diego/O=MyORG/OU=DEP/CN=backupserver.domain.com/emailaddress=someem...@address,
> subject = /C=US/ST=California/L=San
> Diego/O=UCSD/OU=CSE/CN=backupserver.domain.com/emailaddress=someem...@address,
> ERR=18:self signed certificate
> 15-Mar 16:47 backupserver.domain.com-dir JobId 0: Fatal error: TLS
> negotiation failed with FD at "backupserver.domain.com:9102".
>
> Is it possible to use self-signed certificates with Bacula?
> What am I doing wrong?
>
> Any help is appreciated.
>
> Thanks
>
> --
> Download Intel® Parallel Studio Eval
> Try the new software tools for yourself. Speed compiling, find bugs
> proactively, and fine-tune applications for parallel performance.
> See why Intel Parallel Studio got high marks during beta.
> http://p.sf.net/sfu/intel-sw-dev
> ___
> Bacula-users mailing list
> Bacula-users@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/bacula-users
>
--
Download Intel® Parallel Studio Eval
Try the new software tools for yourself. Speed compiling, find bugs
proactively, and fine-tune applications for parallel performance.
See why Intel Parallel Studio got high marks during beta.
http://p.sf.net/sfu/intel-sw-dev
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users
