Re: [Bacula-users] How many jobs can I schedule for the same time?
Am 3.3.2010 01:55, schrieb Mike Ruskai: On 3/2/2010 6:39 PM, Richard Hartmann wrote: Hi all, how many backups can I start at once without causing errors? Even with only 15 backups starting at once, I get one of the two failures below: Fatal error: sql_create.c:784 Lock Path table Query failed: LOCK TABLES Path write, batch write, Path as p write: ERR=Can't find file: './bacula/Path.frm' (errno: 13) Fatal error: sql_create.c:801 Lock Filename table Query failed: LOCK TABLES Filename write, batch write, Filename as f write: ERR=Can't find file: './bacula/Filename.frm' (errno: 13) That's MySQL complaining about not finding table files. Check its configuration for the value of open_files_limit. Increase the value if it's small. If it looks large enough, check your system configuration for restrictions on the number of open files. Just as a side note: I had good results finding bottlenecks in my MySQL setup using the tuning-primer shell-script. It can be found using Google or maybe here: http://forge.mysql.com/projects/project.php?id=44 If open_files_limit is too low it will most likely tell you. Hope this helps! M. Müller -- Download Intel#174; Parallel Studio Eval Try the new software tools for yourself. Speed compiling, find bugs proactively, and fine-tune applications for parallel performance. See why Intel Parallel Studio got high marks during beta. http://p.sf.net/sfu/intel-sw-dev ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Bacula to the Cloud
Am 02.03.2010 22:56, schrieb Peter Zenge: Hello, 2 year Bacula user but first-time poster. I'm currently dumping about 1.6TB to LTO2 tapes every week and I'm looking to migrate to a new storage medium. The obvious answer, I think, is a direct-attached disk array (which I would be able to put in a remote gigabit-attached datacenter before too long). However, I'm wondering if anyone is currently doing large (or what seem to me to be large) backups to the cloud in some way? Assuming I have a gigabit connection to the Internet from my datacenter, I'm wondering how feasible it would be to either use something like Amazon S3 with s3fs (I'm guessing way too much overhead to be efficient), or a bacula-SD implementation on an EC2 node, using Elastic Block Store (EBS) as local disk, and VPN (Amazon VPC) between my datacenter and the SD. Substitute your favorite cloud provider for Amazon above; I don't use any right now so not tied to any particular provider. It just seems like Amazon has all the necessary pieces today. To do this, and keep customers comfortable with the idea of data in the cloud, we would need to encrypt, so I'm also wondering if it would be possible for the SD to encrypt the backup volume, rather than the FD encrypt the data before sending it to SD (which is what we do now)? Easier to manage if we just handled encryption in one place for all clients. I would love to hear what other people are either doing with Bacula and the cloud, or why you have decided not to. Thanks Peter Zenge Pzenge .at. ilinc .dot. com Sending unencrypted data to the SD for encryption would be OK for doing tape based backups where you do not want to lose the tapes. I would suggest not sending your unencrypted backup data to someone else and trusting the receiving system to encrypt it before someone reads it from RAM. Depending on your needs it might be OK to do that, but AFAIK bacula does not support this mode (yet?). AFAIK you have the options of transport encryption (for the connection and data between the systems) and data encryption (for the data leaving the system, with the receiving SD not having the key to do a restore by itself). I personally use transport and data encryption for saving data to offsite SDs in untrusted, meaning not directly accessible, datacenters. If this takes too much CPU time for the 2x encryption you *MIGHT* want to try data encryption with transport encryption but dropping the transport encryption after authentication.. i am not sure about this though, since metadata can be read from the encrypted data and control structures are sent via this line i would also not suggest doing this. Using data encryption with bacula, imho especially with windows, is a pain because of all the certificate management, but for me it is a requirement. -- Download Intel#174; Parallel Studio Eval Try the new software tools for yourself. Speed compiling, find bugs proactively, and fine-tune applications for parallel performance. See why Intel Parallel Studio got high marks during beta. http://p.sf.net/sfu/intel-sw-dev___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Problems with automatic labeling
do you write the new tape labels on the tape every time you put the new one in it? Actually I don't. I just founded cleaner this labelling cleaner... or how do you read the tape label manually? I don't know, is it possible ? :) you could use Day- as label Format. Bacula will then append a number to it. or label them once manually. Yes I understood that bacula use to append a number to the label volumes, maybe this is my mistake cause I don't use it.. be sure to set Retention times on the pool (e.g. for your schedule maybe 18 days) and and a use duration (maybe 1 day as you change it on a daily basis). I know that my Retention times is wrong right now, but I'm just doing some test so I purge volumes manually. But Thanks for the use duration, I will set it up. Well, again, assuming that the volume is purged from the catalog, the tape is rewind and woef-ed.. I still don't unterstand why bacula doesn't want to relabel the volume. I mean, the label format worked fine for the first backup but apparently there is no way to relabel the tape. Will it really fix the problem if I change the label format ? Gilles -- View this message in context: http://old.nabble.com/Problems-with-automatic-labeling-tp27754420p27766090.html Sent from the Bacula - Users mailing list archive at Nabble.com. -- Download Intel#174; Parallel Studio Eval Try the new software tools for yourself. Speed compiling, find bugs proactively, and fine-tune applications for parallel performance. See why Intel Parallel Studio got high marks during beta. http://p.sf.net/sfu/intel-sw-dev ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] How many jobs can I schedule for the same time?
On Wed, Mar 3, 2010 at 01:55, Mike Ruskai than...@earthlink.net wrote: That's MySQL complaining about not finding table files. Check its configuration for the value of open_files_limit. Increase the value if it's small. If it looks large enough, check your system configuration for restrictions on the number of open files. it's set to 1024 -- I would assume this to be more than enough for one to two dozen running jobs, but I am very interested in values other people work with. To check, just do mysql show variables like open_files_limit; Thanks, Richard -- Download Intel#174; Parallel Studio Eval Try the new software tools for yourself. Speed compiling, find bugs proactively, and fine-tune applications for parallel performance. See why Intel Parallel Studio got high marks during beta. http://p.sf.net/sfu/intel-sw-dev ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Backing up 100 servers
On Sat, 27 Feb 2010 14:46:40 +0100
Stan Meier stan.me...@billigmail.org wrote:
Hello world,
[...]
Furthermore, most of those servers will need a default job performed
(/etc, /root, /opt and so on). While it's easy to reuse a JobDefs
stanza to actually define all those jobs, isn't there any way to
group those servers? Do we really have to define more than 120 jobs,
one for each server?
there's been plenty of very good advice given on bacula... thanks everyone!
so, on a slightly different slant...
I don't know how your servers are used/managed... fwiw, I've been trying to
move towards segregating configuration (/etc , crontabs,etc) from software (rpm
-qa as someone suggested), from data (ie, user generated files, DBs,etc).
If you think of it this way, configuration is managed via Puppet (and you only
need to backup Puppet), software is managed via, say, Kickstart (and you only
backup your kickstart data), and then you worry about your 'non-manageable
data'.
just my $0.02 :)
B
_
{Beto|Norberto|Numard} Meijome
There are no stupid questions, but there are a LOT of inquisitive idiots.
I speak for myself, not my employer. Contents may be hot. Slippery when wet.
Reading disclaimers makes you go blind. Writing them is worse. You have been
Warned.
--
Download Intel#174; Parallel Studio Eval
Try the new software tools for yourself. Speed compiling, find bugs
proactively, and fine-tune applications for parallel performance.
See why Intel Parallel Studio got high marks during beta.
http://p.sf.net/sfu/intel-sw-dev
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users
[Bacula-users] How can I confirm/list all jobs scheduled in the next $x days?
I've searched and experimented with both the bconsole and mysql database and can't get more than 24 hours advance status on jobs. Is there a way to confirm what jobs are scheduled for the next $x days. (We've changed the schedule several times; the last time the syntax looked ok but it seemed to ignore the 'monthly' keyword). Many thanks, -- Katherine Buller Infrastructure GSA Capital +44 (0) 20 7959 7666 +44 (0) 7825 314 478 The information transmitted in this communication is intended only for the person(s) or entity to which this communication is addressed and may contain confidential and/or privileged material. Any review, retransmission, dissemination or other use of, or taking of any action in reliance upon, this information by any persons or entities other than the intended recipient is prohibited. If you received this communication in error, please notify i...@gsacapital.com immediately and then delete this communication and any attachments from any computer. Do not disclose the contents of this document to any other person or take any copies. If this communication was misdirected, no confidentiality or privileges are waived. This communication is for information purposes only. It is not intended as an offer or solicitation for the purchase or sale of any financial instrument or as an official confirmation of any transaction. To the extent this communication constitutes a financial promotion, such promotion is directed exclusively at persons who are both (a) intermediate customers or market counterparties for the purposes of the UK Financial Services Authoritys Conduct of Business Sourcebook (COBS) and (b) of a kind to whom such financial promotion may lawfully be made by a person authorised under the Financial Services and Markets Act 2000 (the Act) by virtue of Section 238(5) of the Act and Annex 5 to Chapter 3 of COBS and consequently investments or services referred to in this email are only available for use by such persons. No other person should act upon or rely on this communication. All market prices, data and other information are not warranted as to completeness or accuracy and are subject to change without notice. GSA Capital Partners LLP, its members and affiliates (together, GSA Capital) do not accept any responsibility to update any opinions or other information contained in this communication and do not make any representations or warranties as to the accuracy and completeness of any information contained in this communication. Internet communications are not secure and therefore GSA Capital does not accept legal responsibility for the content of this communication. Although GSA Capital operates anti-virus programmes, it does not accept responsibility for any damage whatsoever that is caused by viruses being passed. Any comments or statements made herein do not necessarily reflect those of GSA Capital. Replies to this communication may be monitored by GSA Capital for operational or business reasons. GSA Capital Partners LLP is authorised and regulated by the Financial Services Authority in the United Kingdom. GSA Capital Partners LLP is also registered with the Securities Exchange Commission in the United States. Registered Office: First Floor, 11 Berkeley Street, London, W1J 8DS, England. Telephone number: 020 7959 8800. -- Download Intel#174; Parallel Studio Eval Try the new software tools for yourself. Speed compiling, find bugs proactively, and fine-tune applications for parallel performance. See why Intel Parallel Studio got high marks during beta. http://p.sf.net/sfu/intel-sw-dev ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Cannot compile bacula 5.0.1 on arch linux
On Saturday 27 February 2010 08:15:15 Bruno Friedmann wrote: From what I see, it seems you doesn't have openssl-devel package ? I didn't know very well archlinux. Archlinux does not have devel-packages. I have managed to compile the package on an old not recently updated system with openssl version 0.9.8i-2 installed -- Regards Steen -- Download Intel#174; Parallel Studio Eval Try the new software tools for yourself. Speed compiling, find bugs proactively, and fine-tune applications for parallel performance. See why Intel Parallel Studio got high marks during beta. http://p.sf.net/sfu/intel-sw-dev ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Problems with automatic labeling
be sure to set Retention times on the pool (e.g. for your schedule maybe 18 days) and and a use duration (maybe 1 day as you change it on a daily basis). I know that my Retention times is wrong right now, but I'm just doing some test so I purge volumes manually. But Thanks for the use duration, I will set it up. Well, again, assuming that the volume is purged from the catalog, the tape is rewind and woef-ed.. I still don't unterstand why bacula doesn't want to relabel the volume. if you purge a tape , the tape label is still in the database. you would need to delete it. I mean, the label format worked fine for the first backup but apparently there is no way to relabel the tape. Will it really fix the problem if I change the label format ? i've never used your method to write woef with a pre-script. IMHO bacula is not designed to work this way (what doesn't mean that you could not do it this way with scripts and try and error). The tape cycle is the following: label the tape - write to it (append) - full or used (when use duration is over) - prune it (retention time is over, purged) - reuse the tape (recycle) you only label it once. the label remains also with recycling. so you write the bacula label on the tape and you will know which tape to insert if you need to restore something. - Thomas -- Download Intel#174; Parallel Studio Eval Try the new software tools for yourself. Speed compiling, find bugs proactively, and fine-tune applications for parallel performance. See why Intel Parallel Studio got high marks during beta. http://p.sf.net/sfu/intel-sw-dev ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Problems with automatic labeling
if you purge a tape , the tape label is still in the database. you would need to delete it. I didn't get that.. I'm ok now. i've never used your method to write woef with a pre-script. IMHO bacula is not designed to work this way (what doesn't mean that you could not do it this way with scripts and try and error). I did that cause I had errors with labelling.. The tape cycle is the following: label the tape - write to it (append) - full or used (when use duration is over) - prune it (retention time is over, purged) - reuse the tape (recycle) Yeah I saw the schem but it really doesn't explain the labelling process. you only label it once. the label remains also with recycling. so you write the bacula label on the tape and you will know which tape to insert if you need to restore something. Actually the label is rewritten everytime but with the same name. If Recycle is set to yes and all Jobs have been pruned, the volume status will be set to Purged and the volume may then be reused when another volume is needed. If the volume is reused, it is relabeled with the same Volume Name, however all previous data will be lost. I've made a test with the new label, it worked fine. But the volume name hasn't a speaking name, it's just Day-0016, Day-0017.. Thx a lot btw, I couldn't find it by myself ! Gilles -- View this message in context: http://old.nabble.com/Problems-with-automatic-labeling-tp27754420p27768724.html Sent from the Bacula - Users mailing list archive at Nabble.com. -- Download Intel#174; Parallel Studio Eval Try the new software tools for yourself. Speed compiling, find bugs proactively, and fine-tune applications for parallel performance. See why Intel Parallel Studio got high marks during beta. http://p.sf.net/sfu/intel-sw-dev ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Problems with automatic labeling
I've made a test with the new label, it worked fine. But the volume name hasn't a speaking name, it's just Day-0016, Day-0017.. you cold still label it yourself and name it what you like (label command). - Thomas -- Download Intel#174; Parallel Studio Eval Try the new software tools for yourself. Speed compiling, find bugs proactively, and fine-tune applications for parallel performance. See why Intel Parallel Studio got high marks during beta. http://p.sf.net/sfu/intel-sw-dev ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] bacula: parallel jobs
On 03/03/10 09:04, Alexey Wasilyev wrote: Hello! According to http://www.mail-archive.com/bacula-users@lists.sourceforge.net/msg40334.html you configure bacula for runinng multiple jobs using the same device concurrently. Can you show me your bacula config's ? Thank you. It's really pretty simple. The important thing is to have the Maximum Concurrent Jobs set to an adequate value in all the necessary places. The places it needs to go are: - the Director resource in bacula-dir.conf - each Storage resource in bacula-dir.conf - the FileDaemon resource in bacula-sd.conf - each Storage resource in bacula-sd.conf You need to set it adequately in ALL of these places. So, what's adequately? Each Storage resource needs its Maximum Concurrent Jobs set to _at least_ the number of simultaneous jobs you want to be able to use that Storage at any time, _plus at least one_ so that you can make status requests to it. The FileDaemon resource should have its Maximum Concurrent Jobs set to _at least_ the maximum number of jobs you expect to run on ALL of its defined Storage resources, _plus at least one_. And the Director should have be able to handle _at least_ the total number of jobs that will be run on all the FileDaemons it knows about, plus at least one. So, suppose you have a moderately complex installation, with two FileDaemons each controlling three Storage devices, and you want to be able to run ten simultaneous jobs to each Storage. Then you need each Storage set for Maximum Concurrent Jobs = 11 or more. Let's say 12. Each FileDaemon needs to control its three Storages, so each FileDaemon needs Maximum Concurrent Jobs = 37 or more (3x12+1). And the Director needs to control both FileDaemons, so it needs a minimum Maximum Concurrent Jobs = 75 (37x2+1). In this example, I might set 15 per storage, 50 per FileDaemon, and 100 on the Director. On any new installation, the default configuration files are written with Maximum Concurrent Jobs = 20 by default. Realistically, that SHOULD be plenty for most small installations. But if you need more than about 20 jobs to be able to run at once, then try following these guidelines. If you already have Maximum Concurrent Jobs set to 20 or more in all the places it needs to be set, and you still can't run multiple concurrent jobs, something else is wrong. Remember that you can only have a single Volume open at a time on a given Storage device, so if your Pool definitions set a limit of one job per volume or, equivalently, Use Volume Once, then you're effectively limiting yourself to one job per Storage device at any one time. If you're restricting Volume usage, you need to do it in a way that will not prevent you from running as many jobs as you want to on each Storage device. This doesn't mean that you can't limit Volume usage to one job per Volume; but if you do, and you still want to run multiple jobs concurrently, then you need to create enough Storage devices to have one Storage per job available. (If you're using disk storage, then multiple Storage resources can point to the same physical disk device, but for tape storage, you can only have one Storage definition per physical tape drive, because you can only load one tape in a tape drive at once but - in 5.x, at least, not sure about earlier - you can have multiple disk volumes on a physical disk device.) -- Phil Stracchino, CDK#2 DoD#299792458 ICBM: 43.5607, -71.355 ala...@caerllewys.net ala...@metrocast.net p...@co.ordinate.org Renaissance Man, Unix ronin, Perl hacker, Free Stater It's not the years, it's the mileage. -- Download Intel#174; Parallel Studio Eval Try the new software tools for yourself. Speed compiling, find bugs proactively, and fine-tune applications for parallel performance. See why Intel Parallel Studio got high marks during beta. http://p.sf.net/sfu/intel-sw-dev ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
[Bacula-users] Shell script question.
Hi, Looking at Bacula scripts and other scripts, I see a test for an empty string performed by adding an 'x' before the variable, and then comparing this to another string that is just an 'x'. The shell test function has a -z string test that returns true if the string is empty. It appears to me that more people use the 'x' comparison. Is there a technical reason why one would be preferable to the other? Why would you do this: if [ x$var = x ]; then ... fi Instead of this: if [ -z $var ]; then ... fi -- Download Intel#174; Parallel Studio Eval Try the new software tools for yourself. Speed compiling, find bugs proactively, and fine-tune applications for parallel performance. See why Intel Parallel Studio got high marks during beta. http://p.sf.net/sfu/intel-sw-dev ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Shell script question.
On 03/03/10 14:05, Brian Debelius wrote: Hi, Looking at Bacula scripts and other scripts, I see a test for an empty string performed by adding an 'x' before the variable, and then comparing this to another string that is just an 'x'. The shell test function has a -z string test that returns true if the string is empty. It appears to me that more people use the 'x' comparison. Is there a technical reason why one would be preferable to the other? Why would you do this: if [ x$var = x ]; then ... fi Instead of this: if [ -z $var ]; then more properly: if [ -z $var ]; then ... fi I've never understood that one myself. I've always put it down to people who don't know about the -z operator or learned shell scripting in an environment in which it was not available. -- Phil Stracchino, CDK#2 DoD#299792458 ICBM: 43.5607, -71.355 ala...@caerllewys.net ala...@metrocast.net p...@co.ordinate.org Renaissance Man, Unix ronin, Perl hacker, Free Stater It's not the years, it's the mileage. -- Download Intel#174; Parallel Studio Eval Try the new software tools for yourself. Speed compiling, find bugs proactively, and fine-tune applications for parallel performance. See why Intel Parallel Studio got high marks during beta. http://p.sf.net/sfu/intel-sw-dev ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] smartctl permission denied
On Tue, 02 Mar 2010 22:39:46 +0100, Andrea Venturoli said: On 03/02/10 15:32, Martin Simmons wrote: Thanks for your answer. No sure if you solved this yet No, I didn't. but did /etc/group contain that entry when you started bacula-sd? Yes, it did. Does procstat -s 70980 show 5 (i.e. operator) in the groups list for the bacula-sd process? Yes, it does. In that case, I can't understand it. Do you have Access Control Lists or Mandatory Access Control running that could change the traditional model of user/group access to files? __Martin -- Download Intel#174; Parallel Studio Eval Try the new software tools for yourself. Speed compiling, find bugs proactively, and fine-tune applications for parallel performance. See why Intel Parallel Studio got high marks during beta. http://p.sf.net/sfu/intel-sw-dev ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Shell script question.
On Wed, 03 Mar 2010, Brian Debelius might have said: Hi, Looking at Bacula scripts and other scripts, I see a test for an empty string performed by adding an 'x' before the variable, and then comparing this to another string that is just an 'x'. The shell test function has a -z string test that returns true if the string is empty. It appears to me that more people use the 'x' comparison. Is there a technical reason why one would be preferable to the other? Why would you do this: if [ x$var = x ]; then ... fi Instead of this: if [ -z $var ]; then ... fi Historically the -z test is not available on many systems, hence the x$var format. Also, put a space between ] and ; like if [ x$var = x ] ; then ... fi Mike -- Download Intel#174; Parallel Studio Eval Try the new software tools for yourself. Speed compiling, find bugs proactively, and fine-tune applications for parallel performance. See why Intel Parallel Studio got high marks during beta. http://p.sf.net/sfu/intel-sw-dev ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] bscan doesn't restore files entities
On Tue, 2 Mar 2010 17:25:36 +0300, Vladimir Vassiliev said: I used bscan after accidentially purging catalog. But it seems information about files didn't restore. Is this right? Here tail of very long bscan's output. Can someone comment it? Does it look good? Thanks. 02-Mar 16:48 bscan JobId 0: Ready to read from volume www-20081230-2305 on device FileStorage-www-2 (/backup/2/www). bscan: bscan.c:451 Pool record for www-2 found in DB. bscan: bscan.c:465 Pool type Backup is OK. bscan: bscan.c:475 Media record for www-20081230-2305 found in DB. bscan: bscan.c:493 Media type File-www-2 is OK. bscan: bscan.c:503 VOL_LABEL: OK for Volume: www-20081230-2305 bscan: bscan.c:521 SOS_LABEL: Found Job record for JobId: 5360 bscan: bscan.c:1037 Fileset www already exists. bscan: bscan.c:1152 Could not update JobId=5311 record. ERR=sql_update.c:196 Update failed: affected_rows=0 for UPDATE Job SET JobStatus='T',EndTime='2010-02-27 23:06:20',ClientId=1,JobBytes=310015832,ReadBytes=0,JobFiles=988,JobErrors=0,VolSessionId=275,VolSessionTime=1264670552,PoolId=11,FileSetId=8,JobTDate=1267301180,RealEndTime='2010-02-27 23:06:20',PriorJobId=0 WHERE JobId=5311 These errors look wrong to me. Does jobid 5311 exist in the catalog? Can you try running UPDATE Job SET JobStatus='T',EndTime='2010-02-27 23:06:20',ClientId=1,JobBytes=310015832,ReadBytes=0,JobFiles=988,JobErrors=0,VolSessionId=275,VolSessionTime=1264670552,PoolId=11,FileSetId=8,JobTDate=1267301180,RealEndTime='2010-02-27 23:06:20',PriorJobId=0 WHERE JobId=5311; using the bconsole sql command to see if that reports an error? __Martin -- Download Intel#174; Parallel Studio Eval Try the new software tools for yourself. Speed compiling, find bugs proactively, and fine-tune applications for parallel performance. See why Intel Parallel Studio got high marks during beta. http://p.sf.net/sfu/intel-sw-dev ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] bacula and curlftpfs 0.9.2 does not work (anymore)
Hi,
I am still very interested in this topic.
Is anyone using Bacula with FTP as storage backend? How do you handle this
situation?
Thanks for your help!
Cheers,
Dennis
--
two4.IT
http://www.two4.it
--
Am Samstag 13 Februar 2010 21:41:04 schrieb Dennis Petschull:
Hi Karsten and others,
I have encountered the same issue some time ago, too.
Did you ever find a solution/workaround/alternative for it?
I would be very interested!
Cheers,
Dennis
--
two4.IT
http://www.two4.it
--
On Wednesday 6 January 2010 21:31 Karsten Schulze wrote:
I believe that bacula does not work anymore with curlftpfs (version
0.9.2). I have found several reports which describe a similar behavior.
https://bugs.launchpad.net/ubuntu/+source/curlftpfs/+bug/367091
http://sourceforge.net/projects/curlftpfs/forums/forum/542750/topic/32958
31
Finally I found the release notes of curlftpfs 0.9.2:
http://sourceforge.net/project/shownotes.php?release_id=602461
Be aware that some applications might not be able to save files on
curlftpfs from 0.9.2 on, because we don't support open(read+write) or
open(write) and seek anymore.
I have written a small program to test this functionality. You can use
it to verify your environment.
#include stdio.h
#include string.h
int main(void) {
//FILE *fp = fopen(/home/bacula/test,w+b);
//works fine (without curlftpfs)
//FILE *fp = fopen(/home/bacula/archive/Daten/Backup/test,a+b);
//create flags: 0x442 /Daten/Backup/test 0100644 umask=
//ftpfs: operation ftpfs_open failed because Operation not supported
//FILE *fp = fopen(/home/bacula/archive/Daten/Backup/test,a+);
//create flags: 0x442 /Daten/Backup/test 0100644 umask=
//ftpfs: operation ftpfs_open failed because Operation not supported
//FILE *fp = fopen(/home/bacula/archive/Daten/Backup/test,w+);
//create flags: 0x242 /Daten/Backup/test 0100644 umask=
//ftpfs: operation ftpfs_open failed because Operation not supported
FILE *fp = fopen(/home/bacula/archive/Daten/Backup/test,w+b);
//create flags: 0x242 /Daten/Backup/test 0100644 umask=
//ftpfs: operation ftpfs_open failed because Operation not supported
fprintf(stdout, I try to open file\n);
if(fp==NULL)
{
fprintf(stdout,Error: can't open file.\n);
return 1;
}
else {
char str[40];
int i;
strcpy(str,somecharacters);
printf(File opened successfully. Writing\n\n);
for (i=0;i8;i++) {
fputc(str[i],fp);
}
}
fclose(fp);
return 0;
}
I would recommend that the documentation of bacula should mention this
incompatibility.
Using a ftp service you have to find an alternative (which one?)
Br, Karsten
---
--- SOLARIS 10 is the OS for Data Centers - provides features such as
DTrace, Predictive Self Healing and Award Winning ZFS. Get Solaris 10 NOW
http://p.sf.net/sfu/solaris-dev2dev
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users
--
Download Intel#174; Parallel Studio Eval
Try the new software tools for yourself. Speed compiling, find bugs
proactively, and fine-tune applications for parallel performance.
See why Intel Parallel Studio got high marks during beta.
http://p.sf.net/sfu/intel-sw-dev
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] potential concurrency bug in SD?
On Tue, 2 Mar 2010 17:11:59 -0500, Joseph Dickson said: Greetings.. I believe I may be hitting a concurrency bug in the storage director code, but I'm pretty new to Bacula and may just not understand what I'm looking at in the trace output. The quick outline of my scenario is: Bacula version 5.0 SD and DIR running on same box (CentOS 5.4, x86_64) Completely file based backup, using a custom autochanger script (which does not appear to be the problem, but I mention it because it may be relevant) In my setup: - I have a single autochanger defined (changer0) - I have 10 drives of DeviceType=File defined for changer0 - Maximum Concurrent Jobs = 1 is set on each Device entry for each drive - Each drive shares the same media type, changer0_media - I have three pools - OnSiteFull, OnSiteIncremental, and OnSiteScratch I have 5 backup jobs that all start at the same time (1:30am), and which all will want either OnSiteFull or OnSiteIncremental depending on the day. The problem that I am seeing is that volumes are being incorrectly marked as being in Error status each night. My storage director trace output can be found at: http://pastebin.com/raw.php?i=Ne0XJbBh One of the jobs in the trace output that marks a volume as in Error is job 36. If someone more familiar with how volume selection works could take a peek and give me some guidance I'd really appreciate it.. in looking through the trace and trying to follow the code, it looks like there is a concurrency issue when there is a volume which is wanted on a drive other than the one it is currently loaded into. There are several disk-changer bacula-sd configs in the regress/scripts/ directory. Are there any differences in your device config compared to those? __Martin -- Download Intel#174; Parallel Studio Eval Try the new software tools for yourself. Speed compiling, find bugs proactively, and fine-tune applications for parallel performance. See why Intel Parallel Studio got high marks during beta. http://p.sf.net/sfu/intel-sw-dev ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Copying huge number of files - very slow
How does it work? How I would do the restore? Regards, Heitor Faria Sean Carolan wrote: The next thing I'm going to try and to is see if I can simply tar the entire raw partition onto tape. Does bacula support this type of backup? Ok, I'm running a backup of the raw device now and it's nice and zippy now, I've already backed up 6GB in the last couple of minutes. Awesome! Now I suppose I just need a script that can unmount the partition before bacula runs. Has anyone tried this with an LVM snapshot? I'm thinking that if I could snapshot the partition that I might not need to have it unmounted while backing up... -- Download Intel#174; Parallel Studio Eval Try the new software tools for yourself. Speed compiling, find bugs proactively, and fine-tune applications for parallel performance. See why Intel Parallel Studio got high marks during beta. http://p.sf.net/sfu/intel-sw-dev ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users -- Heitor Medrado de Faria www.bacula.com.br Msn: hei...@bacula.com.br Gtalk: heitorfa...@gmail.com Skype: neocodeheitor + 55 71 9132-3349 +55 71 3381-6869 -- Download Intel#174; Parallel Studio Eval Try the new software tools for yourself. Speed compiling, find bugs proactively, and fine-tune applications for parallel performance. See why Intel Parallel Studio got high marks during beta. http://p.sf.net/sfu/intel-sw-dev ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] How many jobs can I schedule for the same time?
On Wed, 3 Mar 2010 10:52:33 +0100 Richard Hartmann richih.mailingl...@gmail.com wrote: On Wed, Mar 3, 2010 at 01:55, Mike Ruskai than...@earthlink.net wrote: That's MySQL complaining about not finding table files. Check its configuration for the value of open_files_limit. Increase the value if it's small. If it looks large enough, check your system configuration for restrictions on the number of open files. it's set to 1024 -- I would assume this to be more than enough for one to two dozen running jobs, but I am very interested in values other people work with. To check, just do mysql show variables like open_files_limit; Mine says: mysql show variables like open_files_limit; +--+---+ | Variable_name| Value | +--+---+ | open_files_limit | 8192 | +--+---+ 1 row in set (0.03 sec) I don't think I've ever had more than 6 jobs running concurrently. Regards, -- Alex Chekholko ch...@pcbi.upenn.edu -- Download Intel#174; Parallel Studio Eval Try the new software tools for yourself. Speed compiling, find bugs proactively, and fine-tune applications for parallel performance. See why Intel Parallel Studio got high marks during beta. http://p.sf.net/sfu/intel-sw-dev ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
[Bacula-users] Job stalled indefinitely when performing a Virtual Full Backup
I just upgraded the backup server that we use for staging to bacula 5.0.1, as I wanted to test the new functionality of being able to perform a virtual full backup to the same pool. However, after adding the necessary changes to have virtual fulls backup to the same pool, I am unable to get the job to run. When I first run the job, the director hangs for a number of seconds. Once it is done hanging, I am able to get the director status, but I notice this message: Running Jobs: Console connected at 03-Mar-10 16:15 JobId Level Name Status == 8463 Virtual fslbackup_server.2010-03-03_15.55.48_06 is waiting on Storage fslbackup-sd The job waits indefinitely on the storage daemon and never runs. What could be going on here? Thanks, Daniel Hansen -- Download Intel#174; Parallel Studio Eval Try the new software tools for yourself. Speed compiling, find bugs proactively, and fine-tune applications for parallel performance. See why Intel Parallel Studio got high marks during beta. http://p.sf.net/sfu/intel-sw-dev___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
[Bacula-users] Change media type of existing media
Hello all, I've got some file storage that I use for backups and I've created a few storage declarations for a few pools which are just different directories on the same file system. I specified Media Type = File for all the declarations and when I went to do a restore that spanned multiple of these storage declarations, I got an error similar to Please mount volume X in storage A, but that volume is in storage B. I was able to just move the file to the storage A directory and it was happy, then I moved it back. I found that what I should have done when I set everything up was to give each storage a different Media Type (like File-A, File-B, File-C, etc). How can I change the media type for existing volumes? In Bat there doesn't seem to be a way to change it on a volume basis. Will 'update' then 'volume parameters' then 'All Volumes from Pool' do the trick if I make the appropriate changes in bacula-dir.conf and bacula-sd.conf first? Or will this require some direct SQL manipulation? Thanks, Robert LeBlanc Life Sciences Undergraduate Education Computer Support Brigham Young University -- Download Intel#174; Parallel Studio Eval Try the new software tools for yourself. Speed compiling, find bugs proactively, and fine-tune applications for parallel performance. See why Intel Parallel Studio got high marks during beta. http://p.sf.net/sfu/intel-sw-dev ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Job stalled indefinitely when performing a Virtual Full Backup
2010/3/3 Daniel Hansen dhan...@byu.net: I just upgraded the backup server that we use for staging to bacula 5.0.1, as I wanted to test the new functionality of being able to perform a virtual full backup to the same pool. However, after adding the necessary changes to have virtual fulls backup to the same pool, I am unable to get the job to run. When I first run the job, the director hangs for a number of seconds. Once it is done hanging, I am able to get the director status, but I notice this message: Running Jobs: Console connected at 03-Mar-10 16:15 JobId Level Name Status == 8463 Virtual fslbackup_server.2010-03-03_15.55.48_06 is waiting on Storage fslbackup-sd The job waits indefinitely on the storage daemon and never runs. What could be going on here? Thanks, How about status storage? John -- Download Intel#174; Parallel Studio Eval Try the new software tools for yourself. Speed compiling, find bugs proactively, and fine-tune applications for parallel performance. See why Intel Parallel Studio got high marks during beta. http://p.sf.net/sfu/intel-sw-dev ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Exabyte Storageloader
I just ran a single tape fill and it failed:
Wrote block=30, file,blk=19,985 VolBytes=18,062,392,320 rate=5.475 MB/s
Wrote block=305000, file,blk=19,5985 VolBytes=18,384,952,320 rate=5.478 MB/s
10:14:27 Flush block, write EOF
01-Jan 10:14 btape JobId 0: Error: block.c:573 Write error at 19:7482 on device
Exabyte_VXA-2 (/dev/nst0). ERR=Input/output error.
01-Jan 10:14 btape JobId 0: Error: Backspace file at EOT failed.
ERR=Input/output error
btape: btape.c:2701 Last block at: 19:7481 this_dev_block_num=7482
btape: btape.c:2736 End of tape 19:0. Volume Bytes=18,481,526,784. Write rate =
5.476 MB/s
btape: btape.c:2311 Wrote 1000 blocks on second tape. Done.
Done writing 0 records ...
btape: btape.c:2380 Wrote state file last_block_num1=7481 last_block_num2=0
btape: btape.c:2395
10:14:44 Done filling tape at 19:0. Now beginning re-read of tape ...
btape: btape.c:2476 Enter do_unfill
01-Jan 10:16 btape JobId 0: Ready to read from volume TestVolume1 on device
Exabyte_VXA-2 (/dev/nst0).
Rewinding.
Reading the first 1 records from 0:0.
1 records read now at 1:5084
Reposition from 1:5084 to 19:7481
Reading block 7481.
The last block on the tape matches. Test succeeded.
btape: btape.c:2403 do_unfill failed.
My bacula-sd Device:
Autochanger {
Name = Storageloader
Device = Exabyte_VXA-2
Changer Device = /dev/sg2
Changer Command = /etc/bacula/scripts/mtx-changer %c %o %S %a %d
}
Device {
Name = Exabyte_VXA-2
# btape cap command:
#EOF BSR BSF FSR FSF !FASTFSF !BSFATEOM !EOM REM !RACCESS !AUTOMOUNT !LABEL
!ANONVOLS ALWAYSOPEN MTIOCGET
Media Type = VXA-2
Archive Device = /dev/nst0
Device Type = Tape
Autochanger = yes # if yes, must specify Autochanger resource
Alert Command = /bin/sh -c '/usr/sbin/tapeinfo -f %c | grep TapeAlert |
cat' # cat returns 0 to prevent error return code from grep
Maximum Changer Wait = 30 minutes
Maximum Rewind Wait = 20 minutes
Maximum Open Wait = 25 minutes
RemovableMedia = yes
AlwaysOpen = yes# default = yes
Random Access = no # no for tapes
Hardware End of Medium = no # default = yes??
BSF at EOM = yes# docs says default is No but errors are
produced: Error: Backspace file at EOT failed
# TWO EOF = yes # default = No
TWO EOF = no# default = No
Forward Space Record = yes # default = yes
Forward Space File = yes# default = yes
Backward Space Record = yes # default = yes
Backward Space File = yes # default = yes
Fast Forward Space File = no# default = yes
Use MTIOCGET = yes # default = yes
Offline On Unmount = no # default = no
#AutomaticMount = yes
Auto Select = yes # default = yes
}
Regards,
David Koski
da...@kosmosisland.com
On Tuesday 02 March 2010, Brian Debelius wrote:
I was just curious if the single tape worked, as it might provide
direction in troubleshooting.
On 3/2/2010 2:15 PM, David Koski wrote:
I think the 172/320 should work the same as VXA-2. I have done some
single tape backups and restores with success but did not try btape with
single tapes. The multitape backup is the big issue. I am using bacula
because BRU would not work and BackupExec is not supported on this unit
as it is firewire.
I am preparing for a multitape real test with 30 1G files.
Thanks,
David Koski
da...@kosmosisland.com
On Tuesday 02 March 2010, Brian Debelius wrote:
I just realized that I have a VXA-172/320 not a VXA-2.
How well does a single tape test work?
On 3/2/2010 1:14 PM, David Koski wrote:
Thank you Brian. That is almost what I have with the following
differences:
AutomaticMount not set, default not defined in doc for version 2.4.
Drive Index not set but default works correctly.
AutoSelect not set but default is same as yours.
Maxmimum Changer Wait is higher on mine (30 minutes)
I think the differences should not matter and btape actually indicates
multiple tape backups work. However, there is an error in the log
generated during the multiple tape backup:
28-Feb 17:33 btape JobId 0: Error: block.c:568 Write error at 26:15457
on device Exabyte_VXA-2 (/dev/nst0). ERR=Input/output error. 28-Feb
17:33 btape JobId 0: Error: Backspace file at EOT failed.
ERR=Input/output error
Still, the backup appears to work:
17:37:30 Done filling tapes at 0:13. Now beginning re-read of first
tape ... 28-Feb 17:37 btape JobId 0: 3307 Issuing autochanger unload
slot 2, drive 0 command. 28-Feb 17:38 btape JobId 0: 3304 Issuing
autochanger load slot 1, drive 0 command. 28-Feb 17:39 btape JobId 0:
3305 Autochanger load slot 1, drive 0, status is OK. 28-Feb 17:40
btape JobId 0: Ready to read from volume TestVolume1 on device
Exabyte_VXA-2 (/dev/nst0). Rewinding.
Reading the first 1 records from 0:0.
1 records read
Re: [Bacula-users] bacula and curlftpfs 0.9.2 does not work (anymore)
* Dennis Petschull schrieb am 03.03.10 um 20:54 Uhr: Hi, I am still very interested in this topic. Is anyone using Bacula with FTP as storage backend? How do you handle this situation? Thanks for your help! Why not use 0.9.1 then instead? I do so and it works. -Marc -- +--+ | -- http://www.links2linux.de -- | | | +---Registered-Linux-User-#136487http://counter.li.org + -- Download Intel#174; Parallel Studio Eval Try the new software tools for yourself. Speed compiling, find bugs proactively, and fine-tune applications for parallel performance. See why Intel Parallel Studio got high marks during beta. http://p.sf.net/sfu/intel-sw-dev ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Shell script question.
* Phil Stracchino schrieb am 03.03.10 um 20:18 Uhr: On 03/03/10 14:05, Brian Debelius wrote: Hi, Looking at Bacula scripts and other scripts, I see a test for an empty string performed by adding an 'x' before the variable, and then comparing this to another string that is just an 'x'. The shell test function has a -z string test that returns true if the string is empty. It appears to me that more people use the 'x' comparison. Is there a technical reason why one would be preferable to the other? Why would you do this: if [ x$var = x ]; then ... fi Instead of this: if [ -z $var ]; then more properly: if [ -z $var ]; then Why not use if [ $var = ]; then if -z may not be available? -Marc -- 8AAC 5F46 83B4 DB70 8317 3723 296C 6CCA 35A6 4134 -- Download Intel#174; Parallel Studio Eval Try the new software tools for yourself. Speed compiling, find bugs proactively, and fine-tune applications for parallel performance. See why Intel Parallel Studio got high marks during beta. http://p.sf.net/sfu/intel-sw-dev ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] bacula and curlftpfs 0.9.2 does not work (anymore)
Hey Marc, Yes, using curlftpfs 0.9.1 would be an option, but when I tried it, I ran into this bug, http://bugs.gentoo.org/223239, which prohibits me to upload large files to the FTP storage. So this makes this version unusable for me, too. Cheers, Dennis -- two4.IT http://www.two4.it -- Am Donnerstag 04 März 2010 03:39:13 schrieb Marc Schiffbauer: * Dennis Petschull schrieb am 03.03.10 um 20:54 Uhr: Hi, I am still very interested in this topic. Is anyone using Bacula with FTP as storage backend? How do you handle this situation? Thanks for your help! Why not use 0.9.1 then instead? I do so and it works. -Marc -- Download Intel#174; Parallel Studio Eval Try the new software tools for yourself. Speed compiling, find bugs proactively, and fine-tune applications for parallel performance. See why Intel Parallel Studio got high marks during beta. http://p.sf.net/sfu/intel-sw-dev ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
