Re: [Bacula-users] Fatal error: File daemon at 192.168.0.13:9102rejected Hello command
I dont think it has anything to do with the networkconnection, I've tried to disable the firewall but still had the same issue. Is there someone who know how the hello code work? -Ursprungligt meddelande- Från: Kevin Keane [mailto:[EMAIL PROTECTED] Skickat: den 4 november 2008 20:59 Kopia: bacula-users@lists.sourceforge.net Ämne: Re: [Bacula-users] Fatal error: File daemonat 192.168.0.13:9102rejected Hello command Somebody else mentioned that the same problem occured with 64-bit Vista. As far as I know, Windows 2008 is primarily 64 bit (although there is a 32-bit version available). Maybe it helps if you can use Wireshark or similar to see what is actually being sent/received? Stefan Eriksson wrote: Yes I can telnet 192.168.0.13 on port 9102 with no problem so the port is open, As for vmware networking, it shouldnt be an issue as it works great with windows 2003. And I dont think its a fw issue as I've completely turned of the firewall and still no go. As for connections, the bacula server do connect, I get a standard error if I disable the networking device on the client. Maybe this is a bug, thinking win2008 is rather new Från: Timo Neuvonen [mailto:[EMAIL PROTECTED] Skickat: den 4 november 2008 16:31 Till: bacula-users@lists.sourceforge.net Ämne: Re: [Bacula-users] Fatal error: File daemonat 192.168.0.13:9102rejected Hello command 04-Nov 13:31 backup-dir: test.2008-11-04_13.31.41 Fatal error: File daemon at 192.168.0.13:9102 rejected Hello command 04-Nov 13:31 backup-dir: test.2008-11-04_13.31.41 Error: Bacula backup-dir 2.2.4 (14Sep07): 04-Nov-2008 13:31:44 Hello I'm getting this error with the client using windows 2008. I have the correct password etc as its not the same error message, does anyone know what this error means and how to fix it? I've googled it and there's not much info about it. Sounds like you had a too tight firewall in the client machine? Suspected that, but which type of package is the hello message, I suspect its not on a specific port. More like a echo request. I don't know the Bacula's protocol for sure, but I guess Hello is a token the director uses when trying to establish a connection with a file daemon that is expected to listen at the spesific port. Something similar like HELO / EHLO to port 25 when opening a connection to a mail server. The Bacula Developer Manual could be your friend to find it out more closely. Do you get the same error message if you turn off the file daemon (or disconnect the network cable or...)? If that's the case, it looks like now the director does not get any reply from the file daemon either, and it could be a firewall (or some other networking) issue. If your client is running on a virtual machine on vmware, is vmware set for bridged networking and the ip address belongs to the virtual machine alone? Have you tried to port scan that ip, is there anyone listening at all in the port 9102? - This SF.Net email is sponsored by the Moblin Your Move Developer's challenge Build the coolest Linux based applications with Moblin SDK win great prizes Grand prize is a trip for two to an Open Source event anywhere in the world http://moblin-contest.org/redirect.php?banner_id=100url=/ ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users - This SF.Net email is sponsored by the Moblin Your Move Developer's challenge Build the coolest Linux based applications with Moblin SDK win great prizes Grand prize is a trip for two to an Open Source event anywhere in the world http://moblin-contest.org/redirect.php?banner_id=100url=/ ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
[Bacula-users] RunBeforeJob in background
Hi all, I'm trying to backup a MySQL database and in the Job Resource I define the command RunBeforeJob with a script that dumps the database to a fifo file. I need to run both at the same time, I mean I want to run the script while bacula runs the job. With RunBeforeJob, first dumps the database and then starts the backup of a file. Does anyone know how to run a script in background using RunBeforeJob to perform the dump and the backup job run at the same time? Thank you ISABEL - This SF.Net email is sponsored by the Moblin Your Move Developer's challenge Build the coolest Linux based applications with Moblin SDK win great prizes Grand prize is a trip for two to an Open Source event anywhere in the world http://moblin-contest.org/redirect.php?banner_id=100url=/ ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] file dir computer restarts resulting in mismatched file count error on tape
Hi, 05.11.2008 21:48, Bob Hetzel wrote: I think I may have found a bug or perhaps at least a design limitation in Bacula. I'm running version 2.4.3. I use spooling and I back up to tape. I seem to get periodic errors logged about a file count mismatch resulting in a tape getting marked as status Error. I think I was able to determine the cause of one of these... the backup client running Windows Vista was restarted in the middle of the backup. The next time that tape is used, the file count mismatch is noted and the tape is marked in error. Additionally, bacula apparently filled up the tape before the client was restarted, so I don't know for sure if that had something to do with the problem. This raises some questions... 1) When a client goes away in the middle of a backup, bacula should handle that properly, but it appears to be missing a part of what it would normally do when a backup completes successfully. Bacula does handle that properly, but it can take a *long* time for Bacula to notice the client is gone and, accordingly, cancel the job in question. Once a job is properly canceled, i.e. the SD stopped working on it and notified the DIR, the catalog will be in a state representing the actual tape's contents. Unfortunately, there seem to exist cases where the SD doesn't notice the FD is gone, and thus waits virtually forever for it. I could never reproduce that, IIRC. 2) In theory, if it can't do a whole backup the files that it does get onto the tape should be recoverable too but I've not checked if it handles it such that they are. Should work... You'll have to initiate the restore by giving the job id of the failed job, though. Automatic selection ignores failed jobs (which is reasonable, IMO). 3) When a tape file count mismatch is found, can't it just correct the mismatch, send an e-mail and move on, w/o marking the tape as status Error when the tape is actually fine? No, because there might be many reasons for the file count mismatch. In most cases it's just a off-by-one situation, but imagine the tape in question was used by another Bacula instance or even a different program meanwhile... the tape contents wouldn't be what is noted in the catalog, and that's a situation where you probably don't want to append to or restore from the tape... Arno -- Arno Lehmann IT-Service Lehmann Sandstr. 6, 49080 Osnabrück www.its-lehmann.de - This SF.Net email is sponsored by the Moblin Your Move Developer's challenge Build the coolest Linux based applications with Moblin SDK win great prizes Grand prize is a trip for two to an Open Source event anywhere in the world http://moblin-contest.org/redirect.php?banner_id=100url=/ ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] slow performance copy/migrate disk to tape
On Wed, Nov 05, 2008 at 01:00:19PM +0100, Ulrich Leodolter wrote:
On Wed, 2008-11-05 at 13:38 +0200, Pasi Kärkkäinen wrote:
On Tue, Nov 04, 2008 at 10:21:54PM +0100, Ulrich Leodolter wrote:
Job {
Name = CopyDiskToTape
Type = Copy
Client = dir-fd
Level = Full # must be defined, but is ignored
FileSet = Full Set # must be defined, but is ignored
Pool = DiskBackup
Storage = File
Messages = Standard
Selection Type = PoolUncopiedJobs
Maximum Concurrent Jobs = 10
# SpoolData = yes
}
PoolUncopiedJobs is based on an SQLQuery i posted in the devel list.
i am running 2.5.17 svn at the server side
Btw could you post the subject of that email, or archive link to it..
I could't find it with some searching..
My initial posting
[Bacula-devel] Copy Jobs Selection
Implemented by Marco van Wieringen [EMAIL PROTECTED] as noted in
[Bacula-devel] Implementation of acls and extended attributes
OK, it looks like 'PoolUncopiedJobs' Selection Type is included in 2.5 SVN
version
of Bacula, based on this email:
[Bacula-devel] Implementation of acls and extended attributes:
http://www.mail-archive.com/[EMAIL PROTECTED]/msg03471.html
- support for copy jobs that help for disk-to-disk-to-tape backups
(implemented in bacula as the pooluncopiedjobs copy job
which implements a SQL query first send to the bacula list by
Ulrich Leodolter)
Your initial posting:
[Bacula-devel] Copy Jobs Selection:
http://www.mail-archive.com/[EMAIL PROTECTED]/msg02398.html
But I'm still confused what is the exact SQL query that will be used..
Basicly I have a lot of jobs in the catalog/database, that have been ran,
but the data/files of those jobs doesn't exist anymore on the disk volumes..
because of automatic disk volume recycling.
so I need to copy only the jobs that still are on disk volumes..
I guess I should take a look at the code:)
-- Pasi
-
This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
Build the coolest Linux based applications with Moblin SDK win great prizes
Grand prize is a trip for two to an Open Source event anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100url=/
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] What new feature are you waiting for?
Hi, I'm writing as a Bacula Systems founder here. 06.11.2008 03:45, Dan Langille wrote: On Nov 5, 2008, at 6:23 PM, Florian Heigl wrote: If Kern's Bacula Inc. would release a real, professional guy that doesn't bloat my configs I'd pay for that any day. But bloating the configs just in case a good gui comes around? hell no! sorry to be that harsh, but in this case I rather speak up on time... florian I think you meant GUI, not GUY. Easy typo. But AFAIK, Bacula Inc is there to provide services, not code. We have absolutely no problem creating code. Sure, our main objective is providing service, but creating add-ony, plugins, new features, or whatever you want can also be a valuable service to some customers. Also, with Kern and Eric in the company, we're strongly involved in developing Bacula anyway, so you can expect some code (perhaps even code developed under a customer contract) from Bacula Systems anyway. It's been stately quite clearly that there will not be two code bases. That statement remains true - even if Bacula Systems develops core Bacula code under a customer contract, that code will be published with the regular Bacula sources. That's one of the conditions in all our development offerings. Things *might* be a bit different for add-ons, where restrictively licensed third-party code is included; we don't have such a situation yet, and I don't think we will see something like this soon (after all, even the windows FD is open-source), but I wouldn't say that *every* line of code Bacula Systems ever develops will be open source. Of course, that is no problem for add-ons that don't require linking to the Bacula core code, and with the upcoming plugin interfaces, it won't even be a problem for plugins - but there might be interesting plugins that can not be freely distributed. Again, I don't think we have such a situation in the near future. By the way - among the founders of Bacula Systems, there is no disagreement over these points. We all strongly believe in the open-source idea, and we all will do our best to keep Bacula completely open source, with no closed-source spinoff or dual licensing happening. That said, I wouldn't rule out Bacula Systems creating a commercial, closed source configuration tool - you just have to offer enough money ;-) (But such a configuration tool could also be created by anyone else - the configuration file structure and syntax is not exactly hard to understand, it's extensively documented, and I guess that both parsing an existing configuration and creating a GUI is not exactly rocket science. It just seems to be that, while everybody would like a nice configuration tool, noone wants it bad enough to actually do the work. Which tells us something about the acceptance of text configuration files...) Arno -- Arno Lehmann IT-Service Lehmann Sandstr. 6, 49080 Osnabrück www.its-lehmann.de - This SF.Net email is sponsored by the Moblin Your Move Developer's challenge Build the coolest Linux based applications with Moblin SDK win great prizes Grand prize is a trip for two to an Open Source event anywhere in the world http://moblin-contest.org/redirect.php?banner_id=100url=/ ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
[Bacula-users] Different Catalogs for different Jobs
Dan Langille wrote: - Bacula supports different catalogs for different jobs (I think from the very beginning). My setup uses two different catalogs on the same server, but you can even use them on different servers. Dan, Any chance you could provide some config examples in another thread? I tried to set this up so I could have a separate catalog that for production backups vs. staff machines, but for whatever reason -- even when specifying a catalog for a job -- they seem to get crossed, or it will only use one of the catalogs (generally the wrong one). I'll start from the manual, but I must point out that the above was written by another Bacula developer, not me. :) If I can't get you through this, I'll ask for their config, but it looks straight forward to me. Each Client resource must contain a Catalog directive. What have you been doing? The original idea was different Catalogs for different Jobs, but I'm not sure that's exactly what you want. Can you elaborate. NOTE: you could refer to the same client computer with two different Client resources, each with a different Catalog. -- Dan Langille http://langille.org/ - This SF.Net email is sponsored by the Moblin Your Move Developer's challenge Build the coolest Linux based applications with Moblin SDK win great prizes Grand prize is a trip for two to an Open Source event anywhere in the world http://moblin-contest.org/redirect.php?banner_id=100url=/ ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] slow performance copy/migrate disk to tape
Hi,
Am Di 04.11.2008 21:02 schrieb Ulrich Leodolter
[EMAIL PROTECTED]:
Hi,
Problem: Migrate/Copy jobs from disk pool (DiskBackup)
to tape (DiskCopy) only get overall speed of 10-20MB/s.
full backup job size varies from 10-50GB.
Pool setup is simple, just one pool for full an incremental backups
to disk (automatic recycle works good)
Pool {
Name = DiskBackup
Pool Type = Backup
Recycle = yes
RecyclePool = DiskBackup
AutoPrune = yes
Volume Retention = 15 days
Volume Use Duration = 6 days
Maximum Volume Bytes = 4G
Label Format = Backup-
Next Pool = DiskCopy
}
DiskCopy pool goes to LTO4 tape device.
DiskBackup goes to SATA external raid (6T ext3).
concurrency for DiskBackup jobs is 15, jobs are spread
over DiskBackup Valumes (maybe thats the main problem)
i can read write/continuous on both devices at about 70MB/s
(SATA is not fast)
i tried spooling to local SAS raid, but overall speed is lower
than direct writing to tape.
despooling from SAS raid to Tape runs at Tape maximum spped.
I need some Performance tuning tips, maybe:
we use bacula in the same way, but with version 2.4.3, so we only use
migration instead of copy job.
Maybe you have the same problem as we have. Performance looks ok, if i
use tools other then bacula to test it.
We assume, that the seek process on disk based volumes not really works.
2008-11-06 06:00:03 dss-bacula-dir JobId 39252: The following 1 JobIds
were chosen to be migrated: 38638
2008-11-06 06:00:03 dss-bacula-dir JobId 39252: Migration using
JobId=38638 Job=TCB-PROJECT_DB-WEB.2008-11-03_22.00.14
2008-11-06 06:00:03 dss-bacula-dir JobId 39252: Bootstrap records
written to /var/lib/bacula/dss-bacula-dir.restore.28.bsr
2008-11-06 06:00:03 dss-bacula-dir JobId 39252:
2008-11-06 06:00:03 dss-bacula-dir JobId 39252: The job will require the
following
Volume(s) Storage(s)SD Device(s)
===
2008-11-06 06:00:03 dss-bacula-dir JobId 39252:
2008-11-06 06:00:03 dss-bacula-dir JobId 39252: 00132 File
FileAutoChanger
2008-11-06 06:00:03 dss-bacula-dir JobId 39252:
2008-11-06 13:25:54 dss-bacula-dir JobId 39252: Start Migration JobId
39252, Job=MigrateToTape.2008-11-06_06.00.48
2008-11-06 13:25:54 dss-bacula-dir JobId 39252: Using Device
SL500-1-Drive-1
2008-11-06 13:25:54 dss-bacula-sd JobId 39252: Ready to read from volume
00132 on device FileStorage (/stage0/drive0).
2008-11-06 13:25:54 dss-bacula-sd JobId 39252: Forward spacing Volume
00132 to file:block 21:974164913.
2008-11-06 13:37:49 dss-bacula-sd JobId 39252: End of Volume at file 45
on device FileStorage (/stage0/drive0), Volume 00132
2008-11-06 13:37:49 dss-bacula-sd JobId 39252: End of all volumes.
2008-11-06 13:37:52 dss-bacula-dir JobId 39252: Bacula dss-bacula-dir
2.4.2 (26Jul08): 06-Nov-2008 13:37:52
Build OS: x86_64-pc-linux-gnu debian 4.0
Prev Backup JobId: 38638
New Backup JobId: 39253
Migration JobId:39252
Migration Job: MigrateToTape.2008-11-06_06.00.48
Backup Level: Full
Client: BACULA-DIR
FileSet:LinuxFullSet 2008-04-23 22:00:00
Read Pool: Disk (From Job resource)
Read Storage: File (From Pool resource)
Write Pool: MONTH-TAPE (From Job Pool's NextPool resource)
Write Storage: SL500-1 (From Storage from Pool's NextPool resource)
Start time: 06-Nov-2008 13:25:54
End time: 06-Nov-2008 13:37:52
Elapsed time: 11 mins 58 secs
Priority: 10
SD Files Written: 51
SD Bytes Written: 37,420,967 (37.42 MB)
Rate: 52.1 KB/s
Volume name(s): 000211
Volume Session Id: 3
Volume Session Time:1225972440
Last Volume Bytes: 31,316,447,232 (31.31 GB)
SD Errors: 0
SD termination status: OK
Termination:Migration OK
2008-11-06 13:37:52 dss-bacula-dir JobId 39252: Begin pruning Jobs.
2008-11-06 13:37:52 dss-bacula-dir JobId 39252: Pruned 2 Jobs for client
BACULA-DIR from catalog.
2008-11-06 13:37:52 dss-bacula-dir JobId 39252: Begin pruning Files.
2008-11-06 13:37:52 dss-bacula-dir JobId 39252: No Files found to prune.
2008-11-06 13:37:52 dss-bacula-dir JobId 39252: End auto prune.
As you can see, bacula needs 12 minutes to migrate 37MB. Thats becuase
is does not jump to the correct file position in the disk volume. It
reads the entire volume from the begining to the end if the job was the
last on it. You can see this with stat sd. Bacula reads the volume but
does not write to the tape the most time.
We assume its a configuration problem but do not find any.
Greetings
Sebastian
Limit jobs per Volume in DiskBackup pools?
Split DiskBackup into DiskFull and DiskIncr pools?
Pool {
Name = DiskFull
Pool Type = Backup
Recycle = yes
RecyclePool = DiskFull
AutoPrune = yes
Re: [Bacula-users] copy/migrate uncopied jobs from disk to tape
On Thu, Nov 06, 2008 at 01:14:44PM +0200, Pasi Kärkkäinen wrote:
On Wed, Nov 05, 2008 at 01:00:19PM +0100, Ulrich Leodolter wrote:
On Wed, 2008-11-05 at 13:38 +0200, Pasi Kärkkäinen wrote:
On Tue, Nov 04, 2008 at 10:21:54PM +0100, Ulrich Leodolter wrote:
Job {
Name = CopyDiskToTape
Type = Copy
Client = dir-fd
Level = Full # must be defined, but is ignored
FileSet = Full Set # must be defined, but is ignored
Pool = DiskBackup
Storage = File
Messages = Standard
Selection Type = PoolUncopiedJobs
Maximum Concurrent Jobs = 10
# SpoolData = yes
}
PoolUncopiedJobs is based on an SQLQuery i posted in the devel list.
i am running 2.5.17 svn at the server side
Btw could you post the subject of that email, or archive link to it..
I could't find it with some searching..
My initial posting
[Bacula-devel] Copy Jobs Selection
Implemented by Marco van Wieringen [EMAIL PROTECTED] as noted in
[Bacula-devel] Implementation of acls and extended attributes
OK, it looks like 'PoolUncopiedJobs' Selection Type is included in 2.5 SVN
version
of Bacula, based on this email:
[Bacula-devel] Implementation of acls and extended attributes:
http://www.mail-archive.com/[EMAIL PROTECTED]/msg03471.html
- support for copy jobs that help for disk-to-disk-to-tape backups
(implemented in bacula as the pooluncopiedjobs copy job
which implements a SQL query first send to the bacula list by
Ulrich Leodolter)
Your initial posting:
[Bacula-devel] Copy Jobs Selection:
http://www.mail-archive.com/[EMAIL PROTECTED]/msg02398.html
But I'm still confused what is the exact SQL query that will be used..
Basicly I have a lot of jobs in the catalog/database, that have been ran,
but the data/files of those jobs doesn't exist anymore on the disk volumes..
because of automatic disk volume recycling.
so I need to copy only the jobs that still are on disk volumes..
I guess I should take a look at the code:)
Found it.
src/dird/migrate.c:
const char *sql_jobids_of_pool_uncopied_jobs =
SELECT DISTINCT Job.JobId,Job.StartTime FROM Job,Pool
WHERE Pool.Name = '%s' AND Pool.PoolId = Job.PoolId
AND Job.Type = 'B' AND Job.JobStatus = 'T'
AND Job.JobId NOT IN
(SELECT PriorJobId FROM Job WHERE
Type = 'B' AND Job.JobStatus = 'T'
AND PriorJobId != 0)
ORDER by Job.StartTime;
and find_jobids_of_pool_uncopied_jobs() function.
Running that query manually with mysql client gives correct looking jobids
(and correct looking amount of them).
So I guess I'll try it for real:)
Thanks!
-- Pasi
-
This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
Build the coolest Linux based applications with Moblin SDK win great prizes
Grand prize is a trip for two to an Open Source event anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100url=/
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] What new feature are you waiting for?
Am 05.11.2008 22:03 schrieb Dan Langille: - Bacula has known how to use the current tape that is a drive for a long time now. It is discouraged, but just a matter of configuration. I'm sorry but I'm starting to have serious doubts about that. I have been trying for a long time and in many different ways to get that to work, and each time I ask a question here why my latest attempt doesn't work either, the answer is post config, post log, and then: silence. Surely if it really was just a matter of configuration then somebody should know how to actually configure it? And if it is discouraged, what is the recommended alternative? Thanks, -- Tilman Schmidt Phoenix Software GmbH Bonn, Germany signature.asc Description: OpenPGP digital signature - This SF.Net email is sponsored by the Moblin Your Move Developer's challenge Build the coolest Linux based applications with Moblin SDK win great prizes Grand prize is a trip for two to an Open Source event anywhere in the world http://moblin-contest.org/redirect.php?banner_id=100url=/___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] What new feature are you waiting for?
Doytchin Spiridonov schrieb: By the way I am curious what percentage of users using Bacula are aware of your experience. It is mentioned in the docs but I am sure a lot of people miss that info. We also run a find script - this helps to remove the revived deleted files after restore... The problem in my case was that the affected fs was almost full when it crashed. So there was no space left for the additional 20% deleted/moved/renamed files Ralf - This SF.Net email is sponsored by the Moblin Your Move Developer's challenge Build the coolest Linux based applications with Moblin SDK win great prizes Grand prize is a trip for two to an Open Source event anywhere in the world http://moblin-contest.org/redirect.php?banner_id=100url=/ ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] slow performance copy/migrate disk to tape
Hi,
On Thu, 2008-11-06 at 14:57 +0100, Sebastian Lehmann wrote:
Hi,
Am Di 04.11.2008 21:02 schrieb Ulrich Leodolter
[EMAIL PROTECTED]:
Hi,
Problem: Migrate/Copy jobs from disk pool (DiskBackup)
to tape (DiskCopy) only get overall speed of 10-20MB/s.
full backup job size varies from 10-50GB.
Pool setup is simple, just one pool for full an incremental backups
to disk (automatic recycle works good)
Pool {
Name = DiskBackup
Pool Type = Backup
Recycle = yes
RecyclePool = DiskBackup
AutoPrune = yes
Volume Retention = 15 days
Volume Use Duration = 6 days
Maximum Volume Bytes = 4G
Label Format = Backup-
Next Pool = DiskCopy
}
DiskCopy pool goes to LTO4 tape device.
DiskBackup goes to SATA external raid (6T ext3).
concurrency for DiskBackup jobs is 15, jobs are spread
over DiskBackup Valumes (maybe thats the main problem)
i can read write/continuous on both devices at about 70MB/s
(SATA is not fast)
i tried spooling to local SAS raid, but overall speed is lower
than direct writing to tape.
despooling from SAS raid to Tape runs at Tape maximum spped.
I need some Performance tuning tips, maybe:
we use bacula in the same way, but with version 2.4.3, so we only use
migration instead of copy job.
Maybe you have the same problem as we have. Performance looks ok, if i
use tools other then bacula to test it.
We assume, that the seek process on disk based volumes not really works.
2008-11-06 06:00:03 dss-bacula-dir JobId 39252: The following 1 JobIds
were chosen to be migrated: 38638
2008-11-06 06:00:03 dss-bacula-dir JobId 39252: Migration using
JobId=38638 Job=TCB-PROJECT_DB-WEB.2008-11-03_22.00.14
2008-11-06 06:00:03 dss-bacula-dir JobId 39252: Bootstrap records
written to /var/lib/bacula/dss-bacula-dir.restore.28.bsr
2008-11-06 06:00:03 dss-bacula-dir JobId 39252:
2008-11-06 06:00:03 dss-bacula-dir JobId 39252: The job will require the
following
Volume(s) Storage(s)SD Device(s)
===
2008-11-06 06:00:03 dss-bacula-dir JobId 39252:
2008-11-06 06:00:03 dss-bacula-dir JobId 39252: 00132 File
FileAutoChanger
2008-11-06 06:00:03 dss-bacula-dir JobId 39252:
2008-11-06 13:25:54 dss-bacula-dir JobId 39252: Start Migration JobId
39252, Job=MigrateToTape.2008-11-06_06.00.48
2008-11-06 13:25:54 dss-bacula-dir JobId 39252: Using Device
SL500-1-Drive-1
2008-11-06 13:25:54 dss-bacula-sd JobId 39252: Ready to read from volume
00132 on device FileStorage (/stage0/drive0).
2008-11-06 13:25:54 dss-bacula-sd JobId 39252: Forward spacing Volume
00132 to file:block 21:974164913.
2008-11-06 13:37:49 dss-bacula-sd JobId 39252: End of Volume at file 45
on device FileStorage (/stage0/drive0), Volume 00132
2008-11-06 13:37:49 dss-bacula-sd JobId 39252: End of all volumes.
2008-11-06 13:37:52 dss-bacula-dir JobId 39252: Bacula dss-bacula-dir
2.4.2 (26Jul08): 06-Nov-2008 13:37:52
Build OS: x86_64-pc-linux-gnu debian 4.0
Prev Backup JobId: 38638
New Backup JobId: 39253
Migration JobId:39252
Migration Job: MigrateToTape.2008-11-06_06.00.48
Backup Level: Full
Client: BACULA-DIR
FileSet:LinuxFullSet 2008-04-23 22:00:00
Read Pool: Disk (From Job resource)
Read Storage: File (From Pool resource)
Write Pool: MONTH-TAPE (From Job Pool's NextPool resource)
Write Storage: SL500-1 (From Storage from Pool's NextPool resource)
Start time: 06-Nov-2008 13:25:54
End time: 06-Nov-2008 13:37:52
Elapsed time: 11 mins 58 secs
Priority: 10
SD Files Written: 51
SD Bytes Written: 37,420,967 (37.42 MB)
Rate: 52.1 KB/s
Volume name(s): 000211
Volume Session Id: 3
Volume Session Time:1225972440
Last Volume Bytes: 31,316,447,232 (31.31 GB)
SD Errors: 0
SD termination status: OK
Termination:Migration OK
2008-11-06 13:37:52 dss-bacula-dir JobId 39252: Begin pruning Jobs.
2008-11-06 13:37:52 dss-bacula-dir JobId 39252: Pruned 2 Jobs for client
BACULA-DIR from catalog.
2008-11-06 13:37:52 dss-bacula-dir JobId 39252: Begin pruning Files.
2008-11-06 13:37:52 dss-bacula-dir JobId 39252: No Files found to prune.
2008-11-06 13:37:52 dss-bacula-dir JobId 39252: End auto prune.
As you can see, bacula needs 12 minutes to migrate 37MB. Thats becuase
is does not jump to the correct file position in the disk volume. It
reads the entire volume from the begining to the end if the job was the
last on it. You can see this with stat sd. Bacula reads the volume but
does not write to the tape the most time.
stat sd ???
Forward spacing Volume is done by lseek() on File volumes,
dont think bacula does positioning
Re: [Bacula-users] slow performance copy/migrate disk to tape
Hi,
Am Do 06.11.2008 15:54 schrieb Ulrich Leodolter
[EMAIL PROTECTED]:
Hi,
On Thu, 2008-11-06 at 14:57 +0100, Sebastian Lehmann wrote:
Hi,
Am Di 04.11.2008 21:02 schrieb Ulrich Leodolter
[EMAIL PROTECTED]:
Hi,
Problem: Migrate/Copy jobs from disk pool (DiskBackup)
to tape (DiskCopy) only get overall speed of 10-20MB/s.
full backup job size varies from 10-50GB.
Pool setup is simple, just one pool for full an incremental
backups
to disk (automatic recycle works good)
Pool {
Name = DiskBackup
Pool Type = Backup
Recycle = yes
RecyclePool = DiskBackup
AutoPrune = yes
Volume Retention = 15 days
Volume Use Duration = 6 days
Maximum Volume Bytes = 4G
Label Format = Backup-
Next Pool = DiskCopy
}
DiskCopy pool goes to LTO4 tape device.
DiskBackup goes to SATA external raid (6T ext3).
concurrency for DiskBackup jobs is 15, jobs are spread
over DiskBackup Valumes (maybe thats the main problem)
i can read write/continuous on both devices at about 70MB/s
(SATA is not fast)
i tried spooling to local SAS raid, but overall speed is lower
than direct writing to tape.
despooling from SAS raid to Tape runs at Tape maximum spped.
I need some Performance tuning tips, maybe:
we use bacula in the same way, but with version 2.4.3, so we only
use
migration instead of copy job.
Maybe you have the same problem as we have. Performance looks ok, if
i
use tools other then bacula to test it.
We assume, that the seek process on disk based volumes not really
works.
2008-11-06 06:00:03 dss-bacula-dir JobId 39252: The following 1
JobIds
were chosen to be migrated: 38638
2008-11-06 06:00:03 dss-bacula-dir JobId 39252: Migration using
JobId=38638 Job=TCB-PROJECT_DB-WEB.2008-11-03_22.00.14
2008-11-06 06:00:03 dss-bacula-dir JobId 39252: Bootstrap records
written to /var/lib/bacula/dss-bacula-dir.restore.28.bsr
2008-11-06 06:00:03 dss-bacula-dir JobId 39252:
2008-11-06 06:00:03 dss-bacula-dir JobId 39252: The job will require
the
following
Volume(s) Storage(s)SD Device(s)
===
2008-11-06 06:00:03 dss-bacula-dir JobId 39252:
2008-11-06 06:00:03 dss-bacula-dir JobId 39252: 00132 File
FileAutoChanger
2008-11-06 06:00:03 dss-bacula-dir JobId 39252:
2008-11-06 13:25:54 dss-bacula-dir JobId 39252: Start Migration
JobId
39252, Job=MigrateToTape.2008-11-06_06.00.48
2008-11-06 13:25:54 dss-bacula-dir JobId 39252: Using Device
SL500-1-Drive-1
2008-11-06 13:25:54 dss-bacula-sd JobId 39252: Ready to read from
volume
00132 on device FileStorage (/stage0/drive0).
2008-11-06 13:25:54 dss-bacula-sd JobId 39252: Forward spacing
Volume
00132 to file:block 21:974164913.
2008-11-06 13:37:49 dss-bacula-sd JobId 39252: End of Volume at file
45
on device FileStorage (/stage0/drive0), Volume 00132
2008-11-06 13:37:49 dss-bacula-sd JobId 39252: End of all volumes.
2008-11-06 13:37:52 dss-bacula-dir JobId 39252: Bacula
dss-bacula-dir
2.4.2 (26Jul08): 06-Nov-2008 13:37:52
Build OS: x86_64-pc-linux-gnu debian 4.0
Prev Backup JobId: 38638
New Backup JobId: 39253
Migration JobId:39252
Migration Job: MigrateToTape.2008-11-06_06.00.48
Backup Level: Full
Client: BACULA-DIR
FileSet:LinuxFullSet 2008-04-23 22:00:00
Read Pool: Disk (From Job resource)
Read Storage: File (From Pool resource)
Write Pool: MONTH-TAPE (From Job Pool's NextPool resource)
Write Storage: SL500-1 (From Storage from Pool's NextPool
resource)
Start time: 06-Nov-2008 13:25:54
End time: 06-Nov-2008 13:37:52
Elapsed time: 11 mins 58 secs
Priority: 10
SD Files Written: 51
SD Bytes Written: 37,420,967 (37.42 MB)
Rate: 52.1 KB/s
Volume name(s): 000211
Volume Session Id: 3
Volume Session Time:1225972440
Last Volume Bytes: 31,316,447,232 (31.31 GB)
SD Errors: 0
SD termination status: OK
Termination:Migration OK
2008-11-06 13:37:52 dss-bacula-dir JobId 39252: Begin pruning Jobs.
2008-11-06 13:37:52 dss-bacula-dir JobId 39252: Pruned 2 Jobs for
client
BACULA-DIR from catalog.
2008-11-06 13:37:52 dss-bacula-dir JobId 39252: Begin pruning Files.
2008-11-06 13:37:52 dss-bacula-dir JobId 39252: No Files found to
prune.
2008-11-06 13:37:52 dss-bacula-dir JobId 39252: End auto prune.
As you can see, bacula needs 12 minutes to migrate 37MB. Thats
becuase
is does not jump to the correct file position in the disk volume.
It
reads the entire volume from the begining to the end if the job
Re: [Bacula-users] slow performance copy/migrate disk to tape
On Thu, 2008-11-06 at 16:18 +0100, Sebastian Lehmann wrote:
Hi,
Am Do 06.11.2008 15:54 schrieb Ulrich Leodolter
[EMAIL PROTECTED]:
Hi,
On Thu, 2008-11-06 at 14:57 +0100, Sebastian Lehmann wrote:
Hi,
Am Di 04.11.2008 21:02 schrieb Ulrich Leodolter
[EMAIL PROTECTED]:
Hi,
Problem: Migrate/Copy jobs from disk pool (DiskBackup)
to tape (DiskCopy) only get overall speed of 10-20MB/s.
full backup job size varies from 10-50GB.
Pool setup is simple, just one pool for full an incremental
backups
to disk (automatic recycle works good)
Pool {
Name = DiskBackup
Pool Type = Backup
Recycle = yes
RecyclePool = DiskBackup
AutoPrune = yes
Volume Retention = 15 days
Volume Use Duration = 6 days
Maximum Volume Bytes = 4G
Label Format = Backup-
Next Pool = DiskCopy
}
DiskCopy pool goes to LTO4 tape device.
DiskBackup goes to SATA external raid (6T ext3).
concurrency for DiskBackup jobs is 15, jobs are spread
over DiskBackup Valumes (maybe thats the main problem)
i can read write/continuous on both devices at about 70MB/s
(SATA is not fast)
i tried spooling to local SAS raid, but overall speed is lower
than direct writing to tape.
despooling from SAS raid to Tape runs at Tape maximum spped.
I need some Performance tuning tips, maybe:
we use bacula in the same way, but with version 2.4.3, so we only
use
migration instead of copy job.
Maybe you have the same problem as we have. Performance looks ok, if
i
use tools other then bacula to test it.
We assume, that the seek process on disk based volumes not really
works.
2008-11-06 06:00:03 dss-bacula-dir JobId 39252: The following 1
JobIds
were chosen to be migrated: 38638
2008-11-06 06:00:03 dss-bacula-dir JobId 39252: Migration using
JobId=38638 Job=TCB-PROJECT_DB-WEB.2008-11-03_22.00.14
2008-11-06 06:00:03 dss-bacula-dir JobId 39252: Bootstrap records
written to /var/lib/bacula/dss-bacula-dir.restore.28.bsr
2008-11-06 06:00:03 dss-bacula-dir JobId 39252:
2008-11-06 06:00:03 dss-bacula-dir JobId 39252: The job will require
the
following
Volume(s) Storage(s)SD Device(s)
===
2008-11-06 06:00:03 dss-bacula-dir JobId 39252:
2008-11-06 06:00:03 dss-bacula-dir JobId 39252: 00132 File
FileAutoChanger
2008-11-06 06:00:03 dss-bacula-dir JobId 39252:
2008-11-06 13:25:54 dss-bacula-dir JobId 39252: Start Migration
JobId
39252, Job=MigrateToTape.2008-11-06_06.00.48
2008-11-06 13:25:54 dss-bacula-dir JobId 39252: Using Device
SL500-1-Drive-1
2008-11-06 13:25:54 dss-bacula-sd JobId 39252: Ready to read from
volume
00132 on device FileStorage (/stage0/drive0).
2008-11-06 13:25:54 dss-bacula-sd JobId 39252: Forward spacing
Volume
00132 to file:block 21:974164913.
2008-11-06 13:37:49 dss-bacula-sd JobId 39252: End of Volume at file
45
on device FileStorage (/stage0/drive0), Volume 00132
2008-11-06 13:37:49 dss-bacula-sd JobId 39252: End of all volumes.
2008-11-06 13:37:52 dss-bacula-dir JobId 39252: Bacula
dss-bacula-dir
2.4.2 (26Jul08): 06-Nov-2008 13:37:52
Build OS: x86_64-pc-linux-gnu debian 4.0
Prev Backup JobId: 38638
New Backup JobId: 39253
Migration JobId:39252
Migration Job: MigrateToTape.2008-11-06_06.00.48
Backup Level: Full
Client: BACULA-DIR
FileSet:LinuxFullSet 2008-04-23 22:00:00
Read Pool: Disk (From Job resource)
Read Storage: File (From Pool resource)
Write Pool: MONTH-TAPE (From Job Pool's NextPool resource)
Write Storage: SL500-1 (From Storage from Pool's NextPool
resource)
Start time: 06-Nov-2008 13:25:54
End time: 06-Nov-2008 13:37:52
Elapsed time: 11 mins 58 secs
Priority: 10
SD Files Written: 51
SD Bytes Written: 37,420,967 (37.42 MB)
Rate: 52.1 KB/s
Volume name(s): 000211
Volume Session Id: 3
Volume Session Time:1225972440
Last Volume Bytes: 31,316,447,232 (31.31 GB)
SD Errors: 0
SD termination status: OK
Termination:Migration OK
2008-11-06 13:37:52 dss-bacula-dir JobId 39252: Begin pruning Jobs.
2008-11-06 13:37:52 dss-bacula-dir JobId 39252: Pruned 2 Jobs for
client
BACULA-DIR from catalog.
2008-11-06 13:37:52 dss-bacula-dir JobId 39252: Begin pruning Files.
2008-11-06 13:37:52 dss-bacula-dir JobId 39252: No Files found to
prune.
2008-11-06 13:37:52 dss-bacula-dir JobId 39252: End auto prune.
As you can
Re: [Bacula-users] What new feature are you waiting for?
I would very much like to see the possibility to make only incremental backups, no more full backups required. The main point is to (dramatically) decrease the needed bandwidth, which makes the difference between being able or not to use Bacula for relatively large storage devices on relatively slow internet connections (with relatively few changes/day), like almost all of my backup targets. Ofcourse Bacula can do this if you simply never cycle the original Full Backup and retain *all* incremental backups since the beginning, but this is not what I mean. I mean sort of like rsnapshot does (using the --link-dest option of rsync). That way you need only incremental backups, but you end up with essentially a collection of full backups. Backups that are even extremely disk space efficient. And you can delete as many of them as you like, without harming the other copies. So why not simply use rsnapshot then, if that's so perfect? Well, it isn't that perfect. Bacula has some great advantages with regards to scheduling, reporting, the separation of sd, fd, dir and console, filesets, etc. So implementing a --link-dest kind of feature in Bacula, *that* would be perfect :) I saw some discussion on this topic about one month ago, between Eli Shemer and Dan Langille, but I haven't heard a follow-up since. Nevertheless if something like this could be built, it would make me very happy :) Kind regards, Erik. Dan Langille schreef: With the upcoming release of Bacula, what new feature are you waiting for? Me? I'm waiting for duplicate job control. I had to change a tape today, but I was busy: Running Jobs: JobId Level Name Status == 22833 Fullwocker.2008-11-02_05.55.47 is waiting for a mount request 22834 Fullbast_basic.2008-11-02_05.55.48 is waiting execution 22835 Fullbast_home.2008-11-02_05.55.49 is waiting execution 22836 Fullpolo_basic.2008-11-02_05.55.50 is waiting execution 22837 Fullpolo.2008-11-02_05.55.51 is waiting execution 22838 Fullngaio_basic.2008-11-02_05.55.52 is waiting execution 22839 Fullngaio.2008-11-02_05.55.53 is waiting execution 22840 Fullmacbook.2008-11-02_05.55.54 is waiting execution 22841 Fullnyi.2008-11-02_05.55.55 is waiting execution 22842 Fullnebula.2008-11-02_05.55.56 is waiting execution 22843 Increme nyi_maildir.2008-11-02_06.00.57 is waiting execution 22844 Increme nyi_maildir.2008-11-02_08.00.58 is waiting execution 22845 Fullsupernews.2008-11-02_08.15.59 is waiting execution 22846 Fulllatens_basic.2008-11-02_08.15.00 is waiting execution 22847 Fulllatens_home.2008-11-02_08.15.01 is waiting execution 22848 FullBackupCatalog.2008-11-02_08.15.02 is waiting execution 22849 Increme nyi_maildir.2008-11-02_10.00.03 is waiting execution 22850 Differe nyi_maildir.2008-11-02_12.01.04 is waiting execution 22851 Fullnyi_maildir_tarball.2008-11-02_12.01.05 is waiting execution 22852 Increme nyi_maildir.2008-11-02_16.00.16 is waiting execution 22853 Increme nyi_maildir.2008-11-02_18.00.17 is waiting execution 22854 Increme nyi_maildir.2008-11-02_20.00.18 is waiting execution 22855 Increme nyi_maildir.2008-11-02_22.00.19 is waiting execution - This SF.Net email is sponsored by the Moblin Your Move Developer's challenge Build the coolest Linux based applications with Moblin SDK win great prizes Grand prize is a trip for two to an Open Source event anywhere in the world http://moblin-contest.org/redirect.php?banner_id=100url=/ ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users - This SF.Net email is sponsored by the Moblin Your Move Developer's challenge Build the coolest Linux based applications with Moblin SDK win great prizes Grand prize is a trip for two to an Open Source event anywhere in the world http://moblin-contest.org/redirect.php?banner_id=100url=/ ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
[Bacula-users] Got EOF on tape after reboot
Hello,
I have installed bacula 2.4.3 from source.
Now I am testing the tape-unit with mt and btape.
The first time I do btape -c /usr/local/bacula/bin/bacula-sd.conf /dev/nst0,
the test runs properly.
Then I unload the tape mt -f /dev/nst0 rewoff . I reboot the server. I load
the same tape. At that moment I have the following error when I run the btape
test command:
# btape -c /usr/local/bacula/bin/bacula-sd.conf /dev/nst0
Tape block granularity is 1024 bytes.
btape: butil.c:285 Using device: /dev/nst0 for writing.
btape: btape.c:372 open device lto4drive (/dev/nst0): OK
*test
=== Write, rewind, and re-read test ===
I'm going to write 1000 records and an EOF
then write 1000 records and an EOF, then rewind,
and re-read the data to verify that it is correct.
This is an *essential* feature ...
btape: btape.c:831 Wrote 1000 blocks of 64412 bytes.
btape: btape.c:505 Wrote 1 EOF to lto4drive (/dev/nst0)
btape: btape.c:847 Wrote 1000 blocks of 64412 bytes.
btape: btape.c:505 Wrote 1 EOF to lto4drive (/dev/nst0)
btape: btape.c:856 Rewind OK.
1000 blocks re-read correctly.
Got EOF on tape.
06-Nov 15:30 btape JobId 0: Error: block.c:999 Read error on fd=3 at file:blk
1:0 on device lto4drive (/dev/nst0). ERR=Input/output error.
Got EOF on tape.
Got EOF on tape.
When I load a new tape, the test runs properly, until I reboot the server.
Ik tried the following without success:
mt -f /dev/nst0 defblksize 0
I am running Bacula 2.4.3 on CentOS 5.2.
Harware Server: SunFire X2100
TapeUnit: HP StorageWorks 1760.
This is my bacula-sd.conf:
#
# Default Bacula Storage Daemon Configuration file
#
# For Bacula release 2.4.3 (10 October 2008) -- redhat
#
# You may need to change the name of your tape drive
# on the Archive Device directive in the Device
# resource. If you change the Name and/or the
# Media Type in the Device resource, please ensure
# that dird.conf has corresponding changes.
#
Storage { # definition of myself
Name = itshas-sv11-sd
SDPort = 9103 # Director's port
WorkingDirectory = /usr/local/bacula/
Pid Directory = /usr/local/bacula/bin/working
Maximum Concurrent Jobs = 20
}
#
# List Directors who are permitted to contact Storage daemon
#
Director {
Name = itshas-sv11-dir
Password = xxx
}
#
# Restricted Director, used by tray-monitor to get the
# status of the storage daemon
#
Director {
Name = itshas-sv11-mon
Password = xxx
Monitor = yes
}
#
# Devices supported by this Storage daemon
# To connect, the Director's bacula-dir.conf must have the
# same Name and MediaType.
#
#
# A Linux or Solaris tape drive
#
Device {
Name = lto4drive
Media Type = LTO-4
Archive Device = /dev/nst0
AutomaticMount = yes; # when device opened, read it
AlwaysOpen = no;
RemovableMedia = yes;
RandomAccess = no;
Alert Command = sh -c 'smartctl -H -l error %c'
}
#
# Send all messages to the Director,
# mount messages also are sent to the email address
#
Messages {
Name = Standard
director = itshas-sv11-dir = all
}
There is no logging in the messages-file.
-
This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
Build the coolest Linux based applications with Moblin SDK win great prizes
Grand prize is a trip for two to an Open Source event anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100url=/
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users
[Bacula-users] Restore Errors, terminates early
Below is the console log from my failing restore job. As you can see, the number of files restored is WAY low. I'm trying to figure out how to get what I can out of this restore. I've done small restores before, a file or two, a even a small directory. This is the first time I've had to restore a lot of stuff. Unfortunately, this is not a test :-( Any ideas? TIA, roland Run Restore job JobName:RestoreFiles Bootstrap: /var/spool/bacula/archos-dir.restore.3.bsr Where: /tmp/bacula-restores Replace:always FileSet:System Set Client: aristarchus-fd Storage:File When: 2008-11-06 14:06:05 Catalog:MyCatalog Priority: 10 OK to run? (yes/mod/no): yes Job queued. JobId=289 06-Nov 14:06 archos-dir: Start Restore Job RestoreFiles.2008-11-06_14.06.08 06-Nov 14:06 archos-sd: Ready to read from volume Aristarchus-0004 on device FileStorage (/backup). 06-Nov 14:06 archos-sd: Forward spacing Volume Aristarchus-0004 to file:block 15:2571201622. 06-Nov 14:10 archos-sd: RestoreFiles.2008-11-06_14.06.08 Error: block.c:317 Volume data error at 18:63957951! Block checksum mismatch in block=1199365 len=64512: calc=90c6023e blk=c0deabb4 06-Nov 14:10 aristarchus-fd JobId 289: Error: attribs.c:421 File size of restored file /tmp/bacula-restores/home/roland/tmp/20080429-AstroTrac/img_3470.png not correct. Original 36831200, restored 26476544. 06-Nov 14:10 archos-dir: RestoreFiles.2008-11-06_14.06.08 Error: Bacula 2.0.3 (06Mar07): 06-Nov-2008 14:10:12 JobId: 289 Job:RestoreFiles.2008-11-06_14.06.08 Client: aristarchus-fd Start time: 06-Nov-2008 14:06:10 End time: 06-Nov-2008 14:10:12 Files Expected: 126,091 Files Restored: 14,446 Bytes Restored: 8,439,830,107 Rate: 34875.3 KB/s FD Errors: 1 FD termination status: Error SD termination status: Error Termination:*** Restore Error *** 06-Nov 14:10 archos-dir: Begin pruning Jobs. 06-Nov 14:10 archos-dir: No Jobs found to prune. 06-Nov 14:10 archos-dir: Begin pruning Files. 06-Nov 14:10 archos-dir: No Files found to prune. 06-Nov 14:10 archos-dir: End auto prune. -- PGP Key ID: 66 BC 3B CD Roland B. Roberts, PhD RL Enterprises [EMAIL PROTECTED]6818 Madeline Court [EMAIL PROTECTED] Brooklyn, NY 11220 - This SF.Net email is sponsored by the Moblin Your Move Developer's challenge Build the coolest Linux based applications with Moblin SDK win great prizes Grand prize is a trip for two to an Open Source event anywhere in the world http://moblin-contest.org/redirect.php?banner_id=100url=/ ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] [Bacula-devel] MaximumBlockSize Problem and Question
On Wed, 5 Nov 2008 16:12:51 +0100 Kern Sibbald [EMAIL PROTECTED] wrote: For writing to tape (providing it is LTO-n) I strongly recommend a block size not to exceed 256K. Hi Kern, Why do you say that? Is this thread relevant?: http://www.mail-archive.com/[EMAIL PROTECTED]/msg01246.html Also, I would like to corroborate the OP's experiences; I had an almost identical thread about small block size and slow write speed: http://www.nabble.com/LTO-4-performance--td17407840.html In fact, I was unable to get higher block sizes working at all with btape: http://www.adsm.org/lists/html/Bacula-users/2008-05/msg00504.html So I am still stuck at ~22MB/s writing to LTO-4 with the default block size. Regards, -- Alex Chekholko - This SF.Net email is sponsored by the Moblin Your Move Developer's challenge Build the coolest Linux based applications with Moblin SDK win great prizes Grand prize is a trip for two to an Open Source event anywhere in the world http://moblin-contest.org/redirect.php?banner_id=100url=/ ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] [Bacula-devel] MaximumBlockSize Problem and Question
On Thursday 06 November 2008 22:02:26 Alex Chekholko wrote: On Wed, 5 Nov 2008 16:12:51 +0100 Kern Sibbald [EMAIL PROTECTED] wrote: For writing to tape (providing it is LTO-n) I strongly recommend a block size not to exceed 256K. Hi Kern, Why do you say that? It is my experience with my own testing, and I asked the Quantum technical gurus about using large blocks. They think it is not a good idea at all using values larger than 512K and prefer as I do values around 256K. The bigger it is the more likely the tape writing will get out of phase and IMO the gain is not worth the possible problems. Is this thread relevant?: http://www.mail-archive.com/[EMAIL PROTECTED]/msg01246.htm l Possibly. Also, I would like to corroborate the OP's experiences; I had an almost identical thread about small block size and slow write speed: http://www.nabble.com/LTO-4-performance--td17407840.html In fact, I was unable to get higher block sizes working at all with btape: http://www.adsm.org/lists/html/Bacula-users/2008-05/msg00504.html So I am still stuck at ~22MB/s writing to LTO-4 with the default block size. There are a lot of other parameters to consider, but I cannot give such advice any more -- no time: - Up the block size to 128K or 256K and verify that it is being written that size. - Increase the Maximum File Size to about 5GB. - Use only *very* fast RAID disks running at 15,000 rpm. Maybe soon we can use SSD. - Get only the best disk and tape controllers. - Get good quality, big, fast hardware -- you need hardware experts to get lots of speed. Then you need to do the same kinds of tuning of your network. Regards, Kern - This SF.Net email is sponsored by the Moblin Your Move Developer's challenge Build the coolest Linux based applications with Moblin SDK win great prizes Grand prize is a trip for two to an Open Source event anywhere in the world http://moblin-contest.org/redirect.php?banner_id=100url=/ ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] RunBeforeJob in background
Hi, 06.11.2008 11:32, Isabel Bermejo wrote: Hi all, I'm trying to backup a MySQL database and in the Job Resource I define the command RunBeforeJob with a script that dumps the database to a fifo file. I need to run both at the same time, I mean I want to run the script while bacula runs the job. With RunBeforeJob, first dumps the database and then starts the backup of a file. Does anyone know how to run a script in background using RunBeforeJob to perform the dump and the backup job run at the same time? The solution to that is to start the MySQL dump in the background. In your run before job script, you'd create (or make sure it's useable) the FIFO, then start mysqldump with an at the end of the command line. That sends the mysqldump to the background - you'll probably need to detach from stdin, stdout and stderr first as these file handles, when kept open, can prevent Bacula from going further. I believe I pointed you to a wiki page where a collection of sample scripts is presented which backs up PostgreSQL in that fashion - adapting those to MySQL should be possible without changing the stuff around the actual database interface. Arno Thank you ISABEL - This SF.Net email is sponsored by the Moblin Your Move Developer's challenge Build the coolest Linux based applications with Moblin SDK win great prizes Grand prize is a trip for two to an Open Source event anywhere in the world http://moblin-contest.org/redirect.php?banner_id=100url=/ ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users -- Arno Lehmann IT-Service Lehmann Sandstr. 6, 49080 Osnabrück www.its-lehmann.de - This SF.Net email is sponsored by the Moblin Your Move Developer's challenge Build the coolest Linux based applications with Moblin SDK win great prizes Grand prize is a trip for two to an Open Source event anywhere in the world http://moblin-contest.org/redirect.php?banner_id=100url=/ ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Restore Errors, terminates early
Roland Roberts wrote: Below is the console log from my failing restore job. As you can see, the number of files restored is WAY low. I'm trying to figure out how to get what I can out of this restore. I've done small restores before, a file or two, a even a small directory. This is the first time I've had to restore a lot of stuff. Unfortunately, this is not a test :-( It would appear the problem is in the backend with quoting file names. I have some configuration files that were created via a Java webstart task. Who cares? Well, they are arguably misconfigured 'cause they create their config files as c:\jobwatch.properties which ends up in my home directory as /home/roland/c\:\\jobwatch.properties. That name doesn't get quoted correctly in the SQL query that goes to PostgreSQL, so the query fails (and I get an error in syslog from the postmaster). I've just unmarked those files and will see how far I can get now. It is looking better (since it is still running). I assume I should log this as a bug roland -- PGP Key ID: 66 BC 3B CD Roland B. Roberts, PhD RL Enterprises [EMAIL PROTECTED]6818 Madeline Court [EMAIL PROTECTED] Brooklyn, NY 11220 - This SF.Net email is sponsored by the Moblin Your Move Developer's challenge Build the coolest Linux based applications with Moblin SDK win great prizes Grand prize is a trip for two to an Open Source event anywhere in the world http://moblin-contest.org/redirect.php?banner_id=100url=/ ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] What new feature are you waiting for?
Hi, 06.11.2008 17:34, Erik Logtenberg wrote: I would very much like to see the possibility to make only incremental backups, no more full backups required. The main point is to (dramatically) decrease the needed bandwidth, which makes the difference between being able or not to use Bacula for relatively large storage devices on relatively slow internet connections (with relatively few changes/day), like almost all of my backup targets. Synthetic full backups or consolidation. A feature that is in the development queue and could (in my opinion) be one of the major new things in a 3.1 release, i.e. perhaps next year. To learn more, search the bacula-devel list archive for subjects containing Synthetic or Consolidation - you'll find that, if someone invests money or programmer's time, this project could be started. Ofcourse Bacula can do this if you simply never cycle the original Full Backup and retain *all* incremental backups since the beginning, but this is not what I mean. This is, also, usually not what you want unless you use accurate backups, available in the current development version (because a normal incremental backup doesn't know about moved or renamed files). Arno I mean sort of like rsnapshot does (using the --link-dest option of rsync). That way you need only incremental backups, but you end up with essentially a collection of full backups. Backups that are even extremely disk space efficient. And you can delete as many of them as you like, without harming the other copies. So why not simply use rsnapshot then, if that's so perfect? Well, it isn't that perfect. Bacula has some great advantages with regards to scheduling, reporting, the separation of sd, fd, dir and console, filesets, etc. So implementing a --link-dest kind of feature in Bacula, *that* would be perfect :) I saw some discussion on this topic about one month ago, between Eli Shemer and Dan Langille, but I haven't heard a follow-up since. Nevertheless if something like this could be built, it would make me very happy :) Kind regards, Erik. -- Arno Lehmann IT-Service Lehmann Sandstr. 6, 49080 Osnabrück www.its-lehmann.de - This SF.Net email is sponsored by the Moblin Your Move Developer's challenge Build the coolest Linux based applications with Moblin SDK win great prizes Grand prize is a trip for two to an Open Source event anywhere in the world http://moblin-contest.org/redirect.php?banner_id=100url=/ ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Restore Errors, terminates early
Roland Roberts wrote: It would appear the problem is in the backend with quoting file names. I have some configuration files that were created via a Java webstart task. Who cares? Well, they are arguably misconfigured 'cause they create their config files as c:\jobwatch.properties which ends up in my home directory as /home/roland/c\:\\jobwatch.properties. That name doesn't get quoted correctly in the SQL query that goes to PostgreSQL, so the query fails (and I get an error in syslog from the postmaster). I've just unmarked those files and will see how far I can get now. It is looking better (since it is still running). Well, I spoke too soon. It's clear that this is not the whole story. I'm not getting any logs on the server side to help me with this. It's still quitting early, and syslog does show postgresql errors coincident with the job termination. They look like this: Nov 6 17:14:06 archos postgres[31135]: [30-1] ERROR: table delcandidates does not exist Nov 6 17:14:06 archos postgres[31135]: [30-2] STATEMENT: DROP TABLE DelCandidates Nov 6 17:14:06 archos postgres[31135]: [31-1] ERROR: index delinx1 does not exist Nov 6 17:14:06 archos postgres[31135]: [31-2] STATEMENT: DROP INDEX DelInx1 Nov 6 17:14:06 archos postgres[31135]: [32-1] ERROR: index delinx1 does not exist Nov 6 17:14:06 archos postgres[31135]: [32-2] STATEMENT: DROP INDEX DelInx1 But that may be innocuous as I also seem to get this message when I *issue* the restore command: Nov 6 17:22:09 archos postgres[31135]: [33-1] ERROR: table temp does not exist Nov 6 17:22:09 archos postgres[31135]: [33-2] STATEMENT: DROP TABLE temp Nov 6 17:22:09 archos postgres[31135]: [34-1] ERROR: table temp1 does not exist Nov 6 17:22:09 archos postgres[31135]: [34-2] STATEMENT: DROP TABLE temp1 The restore seems to terminate when it gets any error like a file size not matching. This isn't what I expected from the manual where I expected it to continue on until all files were restored as best as possible. I'm now picking directories, one at a time, and restoring them. Any better ideas on tracking this down? roland -- PGP Key ID: 66 BC 3B CD Roland B. Roberts, PhD RL Enterprises [EMAIL PROTECTED]6818 Madeline Court [EMAIL PROTECTED] Brooklyn, NY 11220 - This SF.Net email is sponsored by the Moblin Your Move Developer's challenge Build the coolest Linux based applications with Moblin SDK win great prizes Grand prize is a trip for two to an Open Source event anywhere in the world http://moblin-contest.org/redirect.php?banner_id=100url=/ ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
[Bacula-users] Running different concurrent client backups to different FileStorages
I have been trying to accomplish the following, and reviewing similar postings as well as RTFM'ing the Bacula online documentation as well. I have one main Bacula system (BK-1) that writes to FileStorage. I have two client systems (Client-1 and Client-2 to keep things simple). What I would like to do is have the backup jobs run concurrent and store in different directories on the main system. What I have tried is creating a FileStore-1 and FileStorage-2 along with Device-1 and Device-2 all of Type = File. The backup jobs (Job-1 and Job-2) will write to their respective places on the same system. What does happen is they run one at a time, and not concurrently (i.e., Job-1 and Job-2 both fire off at 1830h, but Job-2 waits for Job-1 to complete before executing). The Maximum Concurrent Jobs in the bacula-dir.conf and bacula-sd.conf are set at 20, along with both Client-1 and Client-2 bacula-fd.conf set at 20 as well, so that is not the culprit or issue apparently. Reviewing the Bacula manual online some, I see that the Bacula needs to think there are two Device (Device-1 and Device-2 here), which has been done since the backups have been running. Am I missing anything else perhaps? Thoughts and suggestions are extremely welcomed. Also to note, I get this list in Digest Format, so pardon any delays in responding in advance. Thanks in advance too. - This SF.Net email is sponsored by the Moblin Your Move Developer's challenge Build the coolest Linux based applications with Moblin SDK win great prizes Grand prize is a trip for two to an Open Source event anywhere in the world http://moblin-contest.org/redirect.php?banner_id=100url=/ ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Running different concurrent client backups to different FileStorages
On Thu, Nov 6, 2008 at 8:16 PM, Crawford Rainwater [EMAIL PROTECTED] wrote: I have been trying to accomplish the following, and reviewing similar postings as well as RTFM'ing the Bacula online documentation as well. I have one main Bacula system (BK-1) that writes to FileStorage. I have two client systems (Client-1 and Client-2 to keep things simple). What I would like to do is have the backup jobs run concurrent and store in different directories on the main system. What I have tried is creating a FileStore-1 and FileStorage-2 along with Device-1 and Device-2 all of Type = File. The backup jobs (Job-1 and Job-2) will write to their respective places on the same system. What does happen is they run one at a time, and not concurrently (i.e., Job-1 and Job-2 both fire off at 1830h, but Job-2 waits for Job-1 to complete before executing). The Maximum Concurrent Jobs in the bacula-dir.conf and bacula-sd.conf are set at 20, along with both Client-1 and Client-2 bacula-fd.conf set at 20 as well, so that is not the culprit or issue apparently. Reviewing the Bacula manual online some, I see that the Bacula needs to think there are two Device (Device-1 and Device-2 here), which has been done since the backups have been running. Am I missing anything else perhaps? Can you post your bacula-dir.conf? John - This SF.Net email is sponsored by the Moblin Your Move Developer's challenge Build the coolest Linux based applications with Moblin SDK win great prizes Grand prize is a trip for two to an Open Source event anywhere in the world http://moblin-contest.org/redirect.php?banner_id=100url=/ ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Running different concurrent client backups to different FileStorages
On Thu, Nov 6, 2008 at 8:16 PM, Crawford Rainwater [EMAIL PROTECTED] wrote: I have been trying to accomplish the following, and reviewing similar postings as well as RTFM'ing the Bacula online documentation as well. I have one main Bacula system (BK-1) that writes to FileStorage. I have two client systems (Client-1 and Client-2 to keep things simple). What I would like to do is have the backup jobs run concurrent and store in different directories on the main system. What I have tried is creating a FileStore-1 and FileStorage-2 along with Device-1 and Device-2 all of Type = File. The backup jobs (Job-1 and Job-2) will write to their respective places on the same system. What does happen is they run one at a time, and not concurrently (i.e., Job-1 and Job-2 both fire off at 1830h, but Job-2 waits for Job-1 to complete before executing). The Maximum Concurrent Jobs in the bacula-dir.conf and bacula-sd.conf are set at 20, along with both Client-1 and Client-2 bacula-fd.conf set at 20 as well, so that is not the culprit or issue apparently. Reviewing the Bacula manual online some, I see that the Bacula needs to think there are two Device (Device-1 and Device-2 here), which has been done since the backups have been running. Am I missing anything else perhaps? Can you post your bacula-dir.conf? John - This SF.Net email is sponsored by the Moblin Your Move Developer's challenge Build the coolest Linux based applications with Moblin SDK win great prizes Grand prize is a trip for two to an Open Source event anywhere in the world http://moblin-contest.org/redirect.php?banner_id=100url=/ ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Running different concurrent client backups to different FileStorages
- John Drescher [EMAIL PROTECTED] wrote: On Thu, Nov 6, 2008 at 8:16 PM, Crawford Rainwater [EMAIL PROTECTED] wrote: I have been trying to accomplish the following, and reviewing similar postings as well as RTFM'ing the Bacula online documentation as well. I have one main Bacula system (BK-1) that writes to FileStorage. I have two client systems (Client-1 and Client-2 to keep things simple). What I would like to do is have the backup jobs run concurrent and store in different directories on the main system. What I have tried is creating a FileStore-1 and FileStorage-2 along with Device-1 and Device-2 all of Type = File. The backup jobs (Job-1 and Job-2) will write to their respective places on the same system. What does happen is they run one at a time, and not concurrently (i.e., Job-1 and Job-2 both fire off at 1830h, but Job-2 waits for Job-1 to complete before executing). The Maximum Concurrent Jobs in the bacula-dir.conf and bacula-sd.conf are set at 20, along with both Client-1 and Client-2 bacula-fd.conf set at 20 as well, so that is not the culprit or issue apparently. Reviewing the Bacula manual online some, I see that the Bacula needs to think there are two Device (Device-1 and Device-2 here), which has been done since the backups have been running. Am I missing anything else perhaps? Actually I finally found the hidden error. Apparently Client-2's Volume (FileStore-1 as noted above) was not processed through the label via bconsole. I am watching things now, and it seems I have two jobs running at once (finally!). Sorry for the fuss and posting. - This SF.Net email is sponsored by the Moblin Your Move Developer's challenge Build the coolest Linux based applications with Moblin SDK win great prizes Grand prize is a trip for two to an Open Source event anywhere in the world http://moblin-contest.org/redirect.php?banner_id=100url=/ ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] What new feature are you waiting for?
On Nov 6, 2008, at 5:23 AM, Tilman Schmidt wrote: Am 05.11.2008 22:03 schrieb Dan Langille: - Bacula has known how to use the current tape that is a drive for a long time now. It is discouraged, but just a matter of configuration. I'm sorry but I'm starting to have serious doubts about that. I have been trying for a long time and in many different ways to get that to work, and each time I ask a question here why my latest attempt doesn't work either, the answer is post config, post log, and then: silence. What can I say? I'm not responsible for people not answering. Nor is anybody else. That's how it works. Or more precisely, doesn't. Surely if it really was just a matter of configuration then somebody should know how to actually configure it? I don't. I'm repeating what I was told And if it is discouraged, what is the recommended alternative? Work with Bacula, not against it. I don't know what more to say. If the software can't work within the boundaries you are setting, adjust your boundaries. -- Dan Langille http://langille.org/ - This SF.Net email is sponsored by the Moblin Your Move Developer's challenge Build the coolest Linux based applications with Moblin SDK win great prizes Grand prize is a trip for two to an Open Source event anywhere in the world http://moblin-contest.org/redirect.php?banner_id=100url=/ ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] What new feature are you waiting for?
I'm sorry but I'm starting to have serious doubts about that. I have been trying for a long time and in many different ways to get that to work, and each time I ask a question here why my latest attempt doesn't work either, the answer is post config, post log, and then: silence. What can I say? I'm not responsible for people not answering. Nor is anybody else. That's how it works. Or more precisely, doesn't. I can say I try to get involved in as many of these type of questions as I can handle. The biggest reasons after asking for a config or log and then silence from me is I do not see anything that is obvious, I have not experienced that myself (in the 4 to 5 years I have used bacula) and I do not specifically remember a conversation on the list that is similar and had a solution. Sometimes in this case I post the question to the developers but this depends on a lot of factors. One being that I do have a day job that is in no way affiliated with bacula. With that said, I will be pretty silent next week being that I am going to the Rivera Maya on Saturday to soak up a week of sun just before winter hits us here in Pittsburgh. John - This SF.Net email is sponsored by the Moblin Your Move Developer's challenge Build the coolest Linux based applications with Moblin SDK win great prizes Grand prize is a trip for two to an Open Source event anywhere in the world http://moblin-contest.org/redirect.php?banner_id=100url=/ ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] What new feature are you waiting for?
On Nov 6, 2008, at 10:55 PM, John Drescher wrote: I'm sorry but I'm starting to have serious doubts about that. I have been trying for a long time and in many different ways to get that to work, and each time I ask a question here why my latest attempt doesn't work either, the answer is post config, post log, and then: silence. What can I say? I'm not responsible for people not answering. Nor is anybody else. That's how it works. Or more precisely, doesn't. I can say I try to get involved in as many of these type of questions as I can handle. The biggest reasons after asking for a config or log and then silence from me is I do not see anything that is obvious, I have not experienced that myself (in the 4 to 5 years I have used bacula) and I do not specifically remember a conversation on the list that is similar and had a solution. Sometimes in this case I post the question to the developers but this depends on a lot of factors. One being that I do have a day job that is in no way affiliated with bacula. Exactly. We are all volunteers where. Helping out, when, and if, we can. We do this in our *spare* time. Sometimes I know the answer and don't answer because I know it's a pretty simple question that anyone can answer. This encourages new people to get involved with answering. If something is very hard to do, perhaps the plan needs to be rethought. Technology is not always the answer to good manual procedures. -- Dan Langille http://langille.org/ - This SF.Net email is sponsored by the Moblin Your Move Developer's challenge Build the coolest Linux based applications with Moblin SDK win great prizes Grand prize is a trip for two to an Open Source event anywhere in the world http://moblin-contest.org/redirect.php?banner_id=100url=/ ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Enable/disable spooling when starting jobs with run ?
Hi, On Sunday 02 November 2008, Wolfgang Denk wrote: usually I like to disable spooling for full dumps, but enable it for differential and incremental ones. This works fine by selecting one default setting (say: spooling enabled) in the global configuration and overwriting it for the jobs that need the opposite setting (in my example: disable it) in the Run statements in the schedule. However, how can I do the same when running a job manually using the run command on the console? It seems there is no option to enable or disable spooling then? I have had the same problem recently, so starting with SVN revision 7982 (i.e. a snapshot from last Wednesday), there is a spooldata=yes|no flag on the run command line. You will need a SVN version then, though :( Best Bastian -- Collax GmbH . Burkheimer Straße 3 . 79111 Freiburg . Germany p: +49 (0) 761-45684-28 f: +49 (0) 761-45684-10www.collax.com Geschäftsführer: William K. Hite / Boris Nalbach AG München HRB 158898 . Ust.-IdNr: DE 814464942 \ Of all forms of caution, caution in love is the most fatal. - This SF.Net email is sponsored by the Moblin Your Move Developer's challenge Build the coolest Linux based applications with Moblin SDK win great prizes Grand prize is a trip for two to an Open Source event anywhere in the world http://moblin-contest.org/redirect.php?banner_id=100url=/ ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
