Re: [Bacula-users] Broken Pipe messages
Hi, well I don't know wich Job is the culprit. JobId 0 can't be - I have already 48k jobs and this problem appeared just a few days ago. However the director is configured with this: FDConnectTimeout = 3 min SDConnectTimeout = 5 min and every client with: SDConnectTimeout = 2 min I also don't have any max wait time defined (still using Bacula 2.2.8 - don't know if it is valid there). Thanks much Stefan On 30.03.2009, at 13:22, François Mehault wrote: > Could you give me this information about the job: > > Start time > Schedule time > End time > > Max wait time > > I have maybe an idea.. > > François > > -Message d'origine- > De : Stefan Sorin Nicolin [mailto:bac...@nicolinux.org] > Envoyé : lundi 30 mars 2009 13:14 > À : Bacula-users@lists.sourceforge.net > Objet : [Bacula-users] Broken Pipe messages > > Hi, > > I am getting more and more messages like this although the Bacula jobs > do finish fine: > 29-Mär 03:16 db-backup-dir JobId 0: Error: bsock.c:306 Write error > sending 79 bytes to client:192.168.1..236:36131: ERR=Broken pipe > > The IP mentioned in the message ist the one from the vServer where the > director is running. Does anybody knows what this could be? > > Thanks > Stefan Sorin Nicolin > http://nicolinux.org > > --- > Unix guy, Mac head, Rails wannabe, > iPhone Dev-ious, Computer Science > alumnus, usability guesspert and > overall big time visionary > > > -- > ___________ > Bacula-users mailing list > Bacula-users@lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/bacula-users Stefan Sorin Nicolin http://nicolinux.org --- Unix guy, Mac head, Rails wannabe, iPhone Dev-ious, Computer Science alumnus, usability guesspert and overall big time visionary -- ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
[Bacula-users] Broken Pipe messages
Hi, I am getting more and more messages like this although the Bacula jobs do finish fine: 29-Mär 03:16 db-backup-dir JobId 0: Error: bsock.c:306 Write error sending 79 bytes to client:192.168.1..236:36131: ERR=Broken pipe The IP mentioned in the message ist the one from the vServer where the director is running. Does anybody knows what this could be? Thanks Stefan Sorin Nicolin http://nicolinux.org --- Unix guy, Mac head, Rails wannabe, iPhone Dev-ious, Computer Science alumnus, usability guesspert and overall big time visionary -- ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
[Bacula-users] Buffer discarded error with file storage
Hi,
I am encountering the dreaded "Buffer discarded" error upon restoring
a big job from a file based storage. Apart from a few old postings on
this list concerning tapes there seems to be no indicator as why this
happens with file based storage. Is there any explanation excluding
faulty hardware?
18-Mär 13:47 db-backup-smedia-dir JobId 43810: Start Restore Job
RestoreFiles.2009-03-18_13.47.06
18-Mär 13:47 db-backup-smedia-dir JobId 43810: Using Device
"FileStorage"
18-Mär 13:47 db-backup-smedia-sd JobId 43810: Ready to read from
volume "archiv-data6445" on device "FileStorage" (/mnt/backup/store).
18-Mär 13:47 db-backup-smedia-sd JobId 43810: Forward spacing Volume
"archiv-data6445" to file:block 0:225.
18-Mär 13:56 db-backup-smedia-sd JobId 43810: End of Volume at file 2
on device "FileStorage" (/mnt/backup/store), Volume "archiv-data6445"
18-Mär 13:56 db-backup-smedia-sd JobId 43810: Ready to read from
volume "data-inc1709" on device "FileStorage" (/mnt/backup/store).
18-Mär 13:56 db-backup-smedia-sd JobId 43810: Forward spacing Volume
"data-inc1709" to file:block 0:219.
18-Mär 13:56 db-backup-smedia-sd JobId 43810: End of Volume at file 0
on device "FileStorage" (/mnt/backup/store), Volume "data-inc1709"
18-Mär 13:56 db-backup-smedia-sd JobId 43810: Ready to read from
volume "data-inc1756" on device "FileStorage" (/mnt/backup/store).
18-Mär 13:56 db-backup-smedia-sd JobId 43810: Forward spacing Volume
"data-inc1756" to file:block 0:1290450.
18-Mär 13:56 db-backup-smedia-sd JobId 43810: Error: block.c:275
Volume data error at 0:1290450! Wanted ID: "BB02", got "P". Buffer
discarded.
18-Mär 13:56 db-backup-smedia-dir JobId 43810: Error: Bacula db-backup-
smedia-dir 2.2.8 (26Jan08): 18-Mär-2009 13:56:57
Build OS: i686-pc-linux-gnu gentoo 1.6.14
JobId: 43810
Job:RestoreFiles.2009-03-18_13.47.06
Restore Client: d01-box-fd
Start time: 18-Mär-2009 13:47:50
End time: 18-Mär-2009 13:56:57
Files Expected: 924,229
Files Restored: 398,385
Bytes Restored: 12,143,871,860
Rate: 22200.9 KB/s
FD Errors: 0
FD termination status: OK
SD termination status: Error
Termination:*** Restore Error ***
This is a 32bit system running inside a Linux vServer on a Xen domain.
The RAID5 storage array is directly mounted inside the Xen domain and
handed to the vServer. Bacula version is 2.2.8 on Linux Gentoo, kernel
2.6.21
storage daemon:
Device {
Name = FileStorage
Media Type = File
Archive Device = /mnt/backup/store
Spool Directory = /mnt/backup/spool
Maximum Spool Size = 1099511627776
LabelMedia = yes;
Random Access = Yes;
AutomaticMount = yes;
RemovableMedia = no;
AlwaysOpen = no;
}
storage def in director:
Storage {
Name = File
Address = localhost
SDPort = 9103
Password = "secret"
Device = FileStorage
Media Type = File
Maximum Concurrent Jobs = 20
}
I am using spooling to enable concurrent backup jobs (but having the
directive "Maximum Volume Jobs = 1" in the pools config). Could this
somehow produce corrupt volumes?
Thank you.
Stefan Sorin Nicolin
http://nicolinux.org
---
Unix guy, Mac head, Rails wannabe,
iPhone Dev-ious, Computer Science
alumnus, usability guesspert and
overall big time visionary
--
Apps built with the Adobe(R) Flex(R) framework and Flex Builder(TM) are
powering Web 2.0 with engaging, cross-platform capabilities. Quickly and
easily build your RIAs with Flex Builder, the Eclipse(TM)based development
software that enables intelligent coding and step-through debugging.
Download the free 60 day trial. http://p.sf.net/sfu/www-adobe-com
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] wildfile, wilddir and exclude pattern examples(that work with 2.4.4)
On 09.03.2009, at 13:55, Martin Simmons wrote:
>>>>>> On Mon, 9 Mar 2009 12:41:06 +0100, Stefan Sorin Nicolin said:
>>
>> If you include "exclude = yes" in your Exclude {} block then all
>> Files
>> and/or directories selected would be excluded. If you leave out the
>> exclude=yes statement then everything will be _included_ (seems weird
>> to me since the block is clearly called Exclude...)
>
> I think you have this mixed up. All matching items in the Exclude
> {} block
> are excluded and "exclude = yes" is ignored there.
>
> Can you give an example where that is wrong?
Yes, you are right. I mixed it up. Ment to remove this paragraph
before sending but forgot it.
Thanks
>
> __Martin
Stefan Sorin Nicolin
http://nicolinux.org/timr
---
Unix guy, Mac head, Rails wannabe,
iPhone Dev-ious, Computer Science
alumnus, usability guesspert and
overall big time visionary
--
Open Source Business Conference (OSBC), March 24-25, 2009, San Francisco, CA
-OSBC tackles the biggest issue in open source: Open Sourcing the Enterprise
-Strategies to boost innovation and cut costs with open source participation
-Receive a $600 discount off the registration fee with the source code: SFAD
http://p.sf.net/sfu/XcvMzF8H
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] wildfile, wilddir and exclude pattern examples(that work with 2.4.4)
On 09.03.2009, at 11:20, Gerald Leier wrote:
> Hello,
Hi,
>
> Is there some kind of archive containing valid
> bacula FileSet configurations available somewhere?
You might want to check out the wiki:
http://wiki.bacula.org/doku.php?id=sample_configs
>
> Even sniplets like for example exclude patterns
> would be very helpfull in figuring out how to do
> it right.
>
> The following example is something that looks
> ok to me but doesnt work at all and makes bacula
> barf about allmost every line contained within
> the exclude options.(the ones containing wild*)
>
> Also im a little confused about Exclude{} and
> Exclude=yes... comparing rel-bacula.pdf with
> the provided default configurations and the
> few google hits just made that worse.
>
> i use bacula 2.4.4.
>
> ---
> FileSet {
> Name = "WinxpFull"
> Enable VSS = yes
> Include {
>Options {
> signature = SHA1
> compression=GZIP
> ignore case = yes
>}
>File = c:/
>File = d:/
> }
> Exclude {
>File = "c:/temp"
> File = "d:/temp"
>wilddir = "c:/windows/temp*"
>wilddir = "*temporary internet files*"
>wildfile = "*pagefile.sys"
>wildfile = "*.log"
> wildfile = "*system32/perflib*.dat"
> }
> }
> ---
> bacula-dir -t
> Config error: Keyword wilddir not permitted in this resource
>
> any hints would be very appreciated.
If you include "exclude = yes" in your Exclude {} block then all Files
and/or directories selected would be excluded. If you leave out the
exclude=yes statement then everything will be _included_ (seems weird
to me since the block is clearly called Exclude...)
First of all a Exclude {} block accepts only "File" directives. If you
want to exclude files and/or directories based on wildcards or regexps
than you have to put these insind an Options {} block. If you include
"exclude = yes" in your Options {} block then all Files and/or
directories selected would be excluded. If you leave out the
exclude=yes statement then everything will be _included_
Following your example the FileSet should look like this:
FileSet {
Name = "WinxpFull"
Enable VSS = yes
Include {
Options {
wilddir = "c:/windows/temp/*"
wilddir = "*temporary internet files*"
wildfile = "*pagefile.sys"
wildfile = "*.log"
wildfile = "*system32/perflib*.dat"
exclude = yes
}
Options {
signature = SHA1
compression=GZIP
ignore case = yes
}
File = c:/
File = d:/
}
Exclude {
File = "c:/temp"
File = "d:/temp"
}
}
Note that if you put something inside an Exclude {} block it will be
completely ignored (the whole directory). If you have to restore your
system then there will be no temp dir on you disks.
>
>
> sincerely yours
> Gerald
>
> --
>
>
> --
> Open Source Business Conference (OSBC), March 24-25, 2009, San
> Francisco, CA
> -OSBC tackles the biggest issue in open source: Open Sourcing the
> Enterprise
> -Strategies to boost innovation and cut costs with open source
> participation
> -Receive a $600 discount off the registration fee with the source
> code: SFAD
> http://p.sf.net/sfu/XcvMzF8H
> ___
> Bacula-users mailing list
> Bacula-users@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/bacula-users
regards,
Stefan Sorin Nicolin
http://nicolinux.org/timr
---
Unix guy, Mac head, Rails wannabe,
iPhone Dev-ious, Computer Science
alumnus, usability guesspert and
overall big time visionary
--
Open Source Business Conference (OSBC), March 24-25, 2009, San Francisco, CA
-OSBC tackles the biggest issue in open source: Open Sourcing the Enterprise
-Strategies to boost innovation and cut costs with open source participation
-Receive a $600 discount off the registration fee with the source code: SFAD
http://p.sf.net/sfu/XcvMzF8H
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users
[Bacula-users] SD crash with 2.4.4 on Linux
Hi, I've upgraded our Bacula installation from 2.2.8 just to find out that the storage daemon crashes under load :( Sadly I can't do much debuging since this is a production system. I'll try to downgrade. However here are the facts: Gentoo 32bit vServer-Linux, bacula 2.4.4 (installed via Gentoo ebuild with the folowing USE flags: "bacula-console logrotate mysql python readline ssl tcpd"), backup to disk, ~140 backup jobs per night. Bacula clients still a 2.2.8! The storage daemon just hangs after a while (over night) and doesen't accept any more jobs. All other jobs are stuck with "...is waiting on Storage..." or "...is waiting on max Client jobs" (and that although every client is configured with "SDConnectTimeout = 2 min"... but that's another story) Backups fail with: "Storage daemon didn't accept Device "FileStorage" command". If I simply restart the storage daemon (have to kill it since it doesen't respond to my civillized requests) the backups continue. Did anyone encounter such a problem? Could it be because of the old client version? With 40+ clients I am hesitant to update all at once. Thanks much. Stefan Sorin Nicolin http://nicolinux.org/timr --- Unix guy, Mac head, Rails wannabe, iPhone Dev-ious, Computer Science alumnus, usability guesspert and overall big time visionary -- Open Source Business Conference (OSBC), March 24-25, 2009, San Francisco, CA -OSBC tackles the biggest issue in open source: Open Sourcing the Enterprise -Strategies to boost innovation and cut costs with open source participation -Receive a $600 discount off the registration fee with the source code: SFAD http://p.sf.net/sfu/XcvMzF8H ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Exclude file contents
Thanks for the pointer. I see this behavior an 20+ virtualized Gentoo installations (XEN domains). I'll chech it. Stefan On 13.01.2009, at 14:26, John Drescher wrote: >> And also /dev/tty12 on my system is not a large file just a device >> node. >> >> jmd0 john # ls -al /dev/tty12 -al >> crw--- 1 root root 4, 12 Jan 13 08:09 /dev/tty12 >> > > One more comment about this. > > Do you have console output when you hit ctrl-alt-f12 > > if you do not I see the problem. The problem is that at some point you > did not have a /dev/tty12 and that syslog-ng is writing to a file > instead of sending the console messages to the console that you get > when pressing ctrl-alt-f12 > > To fix this. > > stop syslog-ng > > /etc/init.d/syslog-ng stop > > jmd0 john # /etc/init.d/syslog-ng stop > syslog-ng |* Stopping syslog-ng... > > Then kill it to make sure it has quit (it did not on my system): > jmd0 john # pkill syslog-ng > > then delete /dev/tty12 > jmd0 john # rm /dev/tty12 > > Then create /dev/tty12 again > jmd0 john # mknod /dev/tty12 c 4 12 > > > Then restart syslog-ng > jmd0 john # /etc/init.d/syslog-ng restart > syslog-ng |* Starting syslog-ng... > > John Stefan Sorin Nicolin http://nicolinux.org --- Unix guy, Mac head, Rails wannabe, iPhone Dev-ious, Computer Science alumnus, usability guesspert and overall big time visionary -- This SF.net email is sponsored by: SourcForge Community SourceForge wants to tell your story. http://p.sf.net/sfu/sf-spreadtheword ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Exclude file contents
On 13.01.2009, at 13:48, John Drescher wrote: > On Tue, Jan 13, 2009 at 7:22 AM, Stefan Sorin Nicolin > wrote: >> Thank you for your responses. >> Somehow I get the feeling that excluding a single file's contents >> is way >> easier than risking a failed restore. Since I don't mind the few >> MBs in /dev >> (except the hunderts of MBs in /dev/tty12) I'll submit a feature >> request >> regarding this issue. Who knows - it might be useful for other >> scenarios >> too. >> > > You can easily exclude a single file. That is in the documentation. What I ment was excluding the _contents_ of a file. I want to keep everything else (filname, permissions and so on). Stefan > > > John > > -- > This SF.net email is sponsored by: > SourcForge Community > SourceForge wants to tell your story. > http://p.sf.net/sfu/sf-spreadtheword > ___ > Bacula-users mailing list > Bacula-users@lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/bacula-users Stefan Sorin Nicolin http://nicolinux.org --- Unix guy, Mac head, Rails wannabe, iPhone Dev-ious, Computer Science alumnus, usability guesspert and overall big time visionary -- This SF.net email is sponsored by: SourcForge Community SourceForge wants to tell your story. http://p.sf.net/sfu/sf-spreadtheword ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Exclude file contents
Thank you for your responses. Somehow I get the feeling that excluding a single file's contents is way easier than risking a failed restore. Since I don't mind the few MBs in /dev (except the hunderts of MBs in /dev/tty12) I'll submit a feature request regarding this issue. Who knows - it might be useful for other scenarios too. Stefan On 13.01.2009, at 00:39, John Drescher wrote: >>> Have you tested a restore? If you are restoring to 'baremetal', >>> then you >>> might actually need something in /dev before udev starts, which is >>> a bit >>> tricky as it's 'under' udev... >> >> Not necessarily. If you're doing a baremetal restore, you already >> have to >> have to boot some minimum OS before you can run the fd anyway. >> What I have >> done several times is this: >> >> - boot a plain old fedora rescue CD >> - copy the fd binaries, config files, and diskinfo scripts onto it >> - recreate the partition tables and filesystems using the diskinfo >> scripts >> - mount the new partitions on /restore >> - done a full restore with where => /restore >> - reinstall the boot load >> - reboot into the newly restored system >> >> Since the boot scripts are designed to come up with an empty /dev >> and populate >> it at runtime, all you have to do is make sure the /dev/ directory >> exists. >> > > In gentoo, I believe its possible to boot a live cd (or sysrescuecd > http://www.sysresccd.org/Beta-x86) install bacula on tempfs and get > it to restore to a real harddrive without installing the OS first. > > I have not tested that though. > > John > > -- > This SF.net email is sponsored by: > SourcForge Community > SourceForge wants to tell your story. > http://p.sf.net/sfu/sf-spreadtheword > ___ > Bacula-users mailing list > Bacula-users@lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/bacula-users Stefan Sorin Nicolin http://nicolinux.org --- Unix guy, Mac head, Rails wannabe, iPhone Dev-ious, Computer Science alumnus, usability guesspert and overall big time visionary Stefan Sorin Nicolin http://nicolinux.org --- Unix guy, Mac head, Rails wannabe, iPhone Dev-ious, Computer Science alumnus, usability guesspert and overall big time visionary -- This SF.net email is sponsored by: SourcForge Community SourceForge wants to tell your story. http://p.sf.net/sfu/sf-spreadtheword ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
[Bacula-users] Exclude file contents
Hi, I have yet another exclusive configuration need. I'd like to back up a file _without_ it's contents. I noticed that under Gentoo Linux /dev/ tty12 takes a lot of useless space. The terminal output is stored there - somethig that I can very well live without in case I have to restore a system from backup. The only way I can imagine it right now is to exclude the file completely and create a dedicated restore jobdef where /dev/tty12 is created per runscript directive. This doesen't feel right though. Any ideas? Thanks much. Stefan Sorin Nicolin http://nicolinux.org --- Unix guy, Mac head, Rails wannabe, iPhone Dev-ious, Computer Science alumnus, usability guesspert and overall big time visionary -- This SF.net email is sponsored by: SourcForge Community SourceForge wants to tell your story. http://p.sf.net/sfu/sf-spreadtheword ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
[Bacula-users] Rename Storage Daemon
Hi,
I am about to reconfigure a mid sized Bacula intallation. I'd like to
rename the storage daemon meaning the "Name" directive in the Storage
{ } block. Is this asking for trouble? Right now I am a bit nervous
because I just learned (the hard way) that renaming jobs doesn't go
well with restoring old files...
Thanks much.
Stefan Sorin Nicolin
http://nicolinux.org
---
Unix guy, Mac head, Rails wannabe,
Computer Science alumnus, usability
guesspert and overall big time visionary
--
SF.Net email is Sponsored by MIX09, March 18-20, 2009 in Las Vegas, Nevada.
The future of the web can't happen without you. Join us at MIX09 to help
pave the way to the Next Web now. Learn more and register at
http://ad.doubleclick.net/clk;208669438;13503038;i?http://2009.visitmix.com/
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users
[Bacula-users] Jobs blocking Bacula for days!!
Hi, back in june I asked about a problem with the SD failing to move on after encountering connection problems with the clients (no response) http://marc.info/?l=bacula-users&m=121240588225534&w=2 Now I've hit this nasty situation again. Bacula got stuck for 3 whole days (for a production system this a very bad) because there was nobody around to insert a tape when requested. Every other jobs piled up and waited for the first to fail. Is there _ANY_ way to solve this? It seems though that at least one similar scenario is annoying other people too: http://marc.info/?l=bacula-users&m=118772113014249&w=2 Relevant facts: Bacula version 2.4.2 runing on Debian. bacula-dir.conf contains: FDConnectTimeout = 3 min SDConnectTimeout = 5 min All clients configs (bacula-fd.conf) contain: SDConnectTimeout = 2 minute I am aware of the "ClientConnectWait" and "MaximumOpenWait" directives (that I didn't change) and I wonder why the heck the defaults do not kick in when the time exceeds? My bacula setup feels brittle - I don't trust it anymore :( I'd be very grateful for any help. Thanks Stefan Sorin Nicolin http://nicolinux.org --- Unix guy, Mac head, Rails wannabe, computer science alumnus, usability guesspert and overall big time visionary - This SF.Net email is sponsored by the Moblin Your Move Developer's challenge Build the coolest Linux based applications with Moblin SDK & win great prizes Grand prize is a trip for two to an Open Source event anywhere in the world http://moblin-contest.org/redirect.php?banner_id=100&url=/ ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
