Re: [Bacula-users] bfill error - Attempt to WEOF on non-appendable Volume

[Kern Sibbald]

On Wednesday 22 November 2006 08:12, Ralf Gross wrote:
> Ralf Gross schrieb:
> > I'm testing bacula 1.39.28 with our new NEC-T40A changer (LTO3 Ultrium
> > drive). btape's test and fill commands were successful. Only bfill
> > finished with an error message.
> > 
> > +19-Nov 21:58 btape:
> > End of Volume "" at 425:8650 on device "NEC-T40A" (/dev/nst0). Write
> > of 64512 bytes got -1.
> > 19-Nov 21:58 btape: Re-read of last block succeeded.
> > 
> > Write failed at block 6596150.
> > 19-Nov 21:58 btape: Fatal Error at dev.c:1655 because:
> > dev.c:1654 Attempt to WEOF on non-appendable Volume
> > btape: btape.c:483 Bad status from weof. ERR=dev.c:1654 Attempt to
> > WEOF on non-appendable Volume
> > 
> > 
> > Is this something I have to worry about?
> 
> Any ideas on this? I just want to know if this is something I have to
> take care about before I declare the T40A changer is working fine with
> bacula.

Without knowing the details, this does not look too good.  It appears more 
like a bug than a tape problem, but if I were you I would resolve the problem 
before relying on the changer.  Possibly part of the problem is that you have 
edited the output rather than presenting everything so I cannot see 
everything.

In any case, btape should not be getting this kind of error.  At a first 
glance, it looks like btape attempted to write on the tape after re-reading 
the last block, which is not what it normally does -- I suspect this is 
because you edited the messages.  In any case, with the little snipit of 
output it is impossible to get an idea of what is causing the problem.

> 
> Ralf
> 
> -
> Take Surveys. Earn Cash. Influence the Future of IT
> Join SourceForge.net's Techsay panel and you'll get the chance to share your
> opinions on IT & business topics through brief surveys - and earn cash
> http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV
> ___
> Bacula-users mailing list
> Bacula-users@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/bacula-users
> 

-
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your
opinions on IT & business topics through brief surveys - and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

[Bacula-users] Isn't this great?

[Thorsten Engel]

I did a restore of 2.3 GB in 220.000 files and - thanks to bacula - it only
took 12 minutes! Including 2 tape changes! Our credit card shopserver is up
and running again! Really great is, that restoring 200.000 files from the
first tape in 1.7 GB only took *3 minutes!*.

I only wanted to let you know

thorsten


Betreff: Bacula: Restore Error of batbox-fd Full (Job Id: 7219)

22-Nov 09:55 batbox-dir: Start Restore Job RestoreFiles.2006-11-22_09.55.16
22-Nov 09:55 batbox-sd: 3301 Issuing autochanger "loaded drive 0" command.
22-Nov 09:55 batbox-sd: 3302 Autochanger "loaded drive 0", result is Slot
50.
22-Nov 09:56 batbox-sd: 3301 Issuing autochanger "loaded drive 0" command.
22-Nov 09:56 batbox-sd: 3302 Autochanger "loaded drive 0", result is Slot
50.
22-Nov 09:56 batbox-sd: 3307 Issuing autochanger "unload slot 50, drive 0"
command.
22-Nov 09:57 batbox-sd: 3304 Issuing autochanger "load slot 3, drive 0"
command.
22-Nov 09:57 batbox-sd: 3305 Autochanger "load slot 3, drive 0", status is
OK.
22-Nov 09:57 batbox-sd: 3301 Issuing autochanger "loaded drive 0" command.
22-Nov 09:57 batbox-sd: 3302 Autochanger "loaded drive 0", result is Slot 3.
22-Nov 09:58 batbox-sd: Ready to read from volume "DLT003S" on device
"NEO2000-Drive-1" (/dev/nst0).
22-Nov 09:58 batbox-sd: Forward spacing to file:block 79:0.
22-Nov 10:00 batbox-fd: RestoreFiles.2006-11-22_09.55.16 Error:
attribs.c:420 Unable to set file owner /mnt/batdisk/restore/var/run/cgisock:
ERR=Datei oder Verzeichnis nicht gefunden
22-Nov 10:00 batbox-fd: RestoreFiles.2006-11-22_09.55.16 Error:
attribs.c:426 Unable to set file modes /mnt/batdisk/restore/var/run/cgisock:
ERR=Datei oder Verzeichnis nicht gefunden
22-Nov 10:00 batbox-fd: RestoreFiles.2006-11-22_09.55.16 Error:
attribs.c:436 Unable to set file times /mnt/batdisk/restore/var/run/cgisock:
ERR=Datei oder Verzeichnis nicht gefunden
22-Nov 10:00 batbox-fd: RestoreFiles.2006-11-22_09.55.16 Error:
attribs.c:420 Unable to set file owner
/mnt/batdisk/restore/var/run/pptp/255.255.255.255:213.183.3.252: ERR=Datei
oder Verzeichnis nicht gefunden
22-Nov 10:00 batbox-fd: RestoreFiles.2006-11-22_09.55.16 Error:
attribs.c:426 Unable to set file modes
/mnt/batdisk/restore/var/run/pptp/255.255.255.255:213.183.3.252: ERR=Datei
oder Verzeichnis nicht gefunden
22-Nov 10:00 batbox-fd: RestoreFiles.2006-11-22_09.55.16 Error:
attribs.c:436 Unable to set file times
/mnt/batdisk/restore/var/run/pptp/255.255.255.255:213.183.3.252: ERR=Datei
oder Verzeichnis nicht gefunden
22-Nov 10:01 batbox-fd: RestoreFiles.2006-11-22_09.55.16 Error:
attribs.c:420 Unable to set file owner /mnt/batdisk/restore/dev/log:
ERR=Datei oder Verzeichnis nicht gefunden
22-Nov 10:01 batbox-fd: RestoreFiles.2006-11-22_09.55.16 Error:
attribs.c:426 Unable to set file modes /mnt/batdisk/restore/dev/log:
ERR=Datei oder Verzeichnis nicht gefunden
22-Nov 10:01 batbox-fd: RestoreFiles.2006-11-22_09.55.16 Error:
attribs.c:436 Unable to set file times /mnt/batdisk/restore/dev/log:
ERR=Datei oder Verzeichnis nicht gefunden
22-Nov 10:01 batbox-sd: End of Volume at file 79 on device "NEO2000-Drive-1"
(/dev/nst0), Volume "DLT003S"
22-Nov 10:01 batbox-sd: 3301 Issuing autochanger "loaded drive 0" command.
22-Nov 10:01 batbox-sd: 3302 Autochanger "loaded drive 0", result is Slot 3.
22-Nov 10:03 batbox-sd: RestoreFiles.2006-11-22_09.55.16 Warning:
acquire.c:223 Wrong Volume mounted on device "NEO2000-Drive-1" (/dev/nst0):
Wanted DLT027S have DLT003S
22-Nov 10:03 batbox-sd: 3301 Issuing autochanger "loaded drive 0" command.
22-Nov 10:03 batbox-sd: 3302 Autochanger "loaded drive 0", result is Slot 3.
22-Nov 10:03 batbox-sd: 3307 Issuing autochanger "unload slot 3, drive 0"
command.
22-Nov 10:04 batbox-sd: 3304 Issuing autochanger "load slot 50, drive 0"
command.
22-Nov 10:04 batbox-sd: 3305 Autochanger "load slot 50, drive 0", status is
OK.
22-Nov 10:04 batbox-sd: 3301 Issuing autochanger "loaded drive 0" command.
22-Nov 10:04 batbox-sd: 3302 Autochanger "loaded drive 0", result is Slot
50.
22-Nov 10:04 batbox-sd: Ready to read from volume "DLT027S" on device
"NEO2000-Drive-1" (/dev/nst0).
22-Nov 10:04 batbox-sd: Forward spacing to file:block 149:0.
22-Nov 10:07 batbox-dir: RestoreFiles.2006-11-22_09.55.16 Error: Bacula
1.39.20 (22Aug06): 22-Nov-2006 10:07:45
  JobId:  7219
  Job:RestoreFiles.2006-11-22_09.55.16
  Client: batbox-fd
  Start time: 22-Nov-2006 09:55:18
  End time:   22-Nov-2006 10:07:45
  Files Expected: 223,813
  Files Restored: 223,813
  Bytes Restored: 2,291,493,433
  Rate:   3067.6 KB/s
  FD Errors:  9
  FD termination status:  Error
  SD termination status:  OK
  Termination:*** Restore Error ***

22-Nov 10:07 batbox-dir: Begin pruning Jobs.
22-Nov 10:07 batbox-dir: No Jobs found to prune.
22-Nov 10:07 batbox-dir: Begin pruning Files.
22-Nov 10:07 batbox-dir: Pruned Files from 39 Jobs for client batbox-fd from

[Bacula-users] Problems with autochanger

[Jukka Laaksola]

Hi!

We have Bacula version 1.38.11-5~bpo.1 which is from backports.org to 
Debian Sarge.

The system has a HP autochanger with a magazine with six LTO1 tape 
slots. There are ten clients to backup. Our idea is to do full backups 
on every tuesday and then incremental backups on the rest of the week 
and also on the mondays. And after two weeks to change manually 
magazine. We have now three magazines so same tapes will be overwriten 
after six weeks.
In the configuration there are only one pool to backup. All volumes are 
in this same pool.
The Volume retention of the pool is 30 days. So after six weeks the 
first volumes should be recycleable.

The problem are appendable tapes which are not in changer. We say 
"unmount" and change the magazine. Then we say "update slots" and 
"mount". The inchanger flags gets updated successful. Then we mark all 
removed appendable volumes as Used.
But in the middle of the night at backup time the bacula ask to mount 
some appendable volume which is not in the changer.

Let's take an example of last night.
8<
*list volumes
Pool: Default
++--++---+---+---+---+-+
|MeId| VolumeNa | VolStat| VolRetent |ReC|Slo|InC| LastWritten |
++--++---+---+---+---+-+
|  7 | UF6041L1 | Full   | 2,592,000 | 1 | 1 | 1 | 2006-10-13 01:30:31 |
|  8 | UF6042L1 | Full   | 2,592,000 | 1 | 2 | 1 | 2006-10-15 02:06:52 |
|  9 | UF6043L1 | Full   | 2,592,000 | 1 | 3 | 1 | 2006-10-17 05:30:52 |
| 10 | UF6044L1 | Full   | 2,592,000 | 1 | 4 | 1 | 2006-10-21 02:18:08 |
| 11 | UF6045L1 | Used   | 2,592,000 | 1 | 5 | 1 | 2006-10-23 02:22:07 |
| 12 | UF6046L1 | Full   | 2,592,000 | 1 | 6 | 1 | 2006-11-22 00:59:39 |
| 13 | UF6031L1 | Full   | 2,592,000 | 1 | 1 | 0 | 2006-10-26 01:54:44 |
| 14 | UF6032L1 | Full   | 2,592,000 | 1 | 2 | 0 | 2006-10-29 00:32:00 |
| 15 | UF6033L1 | Full   | 2,592,000 | 1 | 3 | 0 | 2006-10-31 04:57:41 |
| 16 | UF6034L1 | Full   | 2,592,000 | 1 | 4 | 0 | 2006-11-03 02:23:29 |
| 17 | UF6035L1 | Used   | 2,592,000 | 1 | 5 | 0 | 2006-11-06 02:24:25 |
| 18 | UF6036L1 | Full   | 2,592,000 | 1 | 6 | 0 | 2006-10-25 01:05:05 |
| 19 | CLNP35L1 | Cleanin| 2,592,000 | 1 | 7 | 1 | -00-00 00:00:00 |
| 20 | UF6061L1 | Full   | 2,592,000 | 1 | 1 | 0 | 2006-11-08 01:02:17 |
| 21 | UF6062L1 | Full   | 2,592,000 | 1 | 2 | 0 | 2006-11-12 00:39:04 |
| 22 | UF6063L1 | Full   | 2,592,000 | 1 | 3 | 0 | 2006-11-14 05:19:18 |
| 23 | UF6064L1 | Full   | 2,592,000 | 1 | 4 | 0 | 2006-11-17 02:50:41 |
| 24 | UF6065L1 | Used   | 2,592,000 | 1 | 5 | 0 | 2006-11-20 02:31:03 |
| 25 | UF6060L1 | Used   | 2,592,000 | 1 | 6 | 0 | -00-00 00:00:00 |
++--++---+---+---+---+-+
8<

At the 2006-11-20 we changed the magazine with UF606xxx tapes to a 
magazine with UF604xxx tapes. At the 2006-11-21 the full backup was done 
correctly on the tape UF6046L1. It was manually marked Used and the 
LastWritten was "-00-00 00:00:00". That's fine.

But at 2006-11-22 the UF6046L1 fulls and we want the Bacula continue on 
the UF6041L1 tapes. Because it's lastwritten has the oldest time and 
it's volume retention time has gone. And the most important this tape is 
in changer. Here Bacula works againts our thoughts. It want wants 
recycle UF6060L1 propably because of older lastwritten. But that tape is 
not in the changer.

Are there any option to force bacula first to try recycle and fill all 
volumes in changer and only after that to try recycle other volumes. And 
of course we mean to recycle only volumes which volume retention has 
expired.

Here is parts of the Bacula configuration:
8<
bacula-dir.conf:
---
Pool {
   Name = Default
   Pool Type = Backup
   Recycle = yes   # Bacula can automatically 
recycle Volumes
   AutoPrune = yes # Prune expired volumes
   Volume Retention = 30  days # one month
   Accept Any Volume = yes # write on any volume in the pool
   Cleaning Prefix = "CLN"
}


bacula-sd.conf:
---
Autochanger {
   Name = "HPSureStore-Changer"
   Device = LTO-1
   Changer Device = /dev/sg0
   Changer Command = "/etc/bacula/scripts/mtx-changer %c %o %S %a %d"
}

Device {
   Name = "LTO-1"
   Media Type = LTO
   Device Type = Tape
   Archive Device = /dev/nst0
   LabelMedia = no;   # lets Bacula label unlabeled media
   Random Access = No;
   AutomaticMount = yes;   # when device opened, read it
   RemovableMedia = Yes;
   AlwaysOpen = Yes;
   Drive Index = 0
   Autochanger = yes
}

8<




regards
Jukka

-- 
Jukka Laaksola
Netland Oy

-
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your
opinions on IT & business topics through brief surveys - and earn cash
http://www.techsay.c

[Bacula-users] Bacula restore on windows client

[Etienne ETOURNAY]

Hi !

When I want restore a Windows client, "Where:  /tmp/bacula-restore" = 
"C:\tmp\bacula-restore" on Windows.
My drive C:\ is very small : I want to restore to D:\tmp\bacula-restore .
But "Where:  d:/tmp/bacula-restore" don't work !

How I can to do it ?

Regards,
Etienne ETOURNAY

-
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your
opinions on IT & business topics through brief surveys - and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

Re: [Bacula-users] Restore doesn't work

[Jean-Michel Caricand]

I checked. The type of the device is "File". How to solve it ?
I did not find a solution in Bacula's documentation.

Thank.

> Hi alls,
> 
> I try to restore a directory from a file device. Backup works
> fine but restore not. See messages :
> 
> You have messages.
> *messages
> 21-Nov 22:06 lifcsys4-dir: Start Restore Job
> RestoreLuke.2006-11-21_22.06.18
> 21-Nov 22:06 lifcsys4-sd: Ready to read from volume
> "Luke-0001" on device /backup/bacula.
> 21-Nov 22:06 lifcsys4-sd: RestoreLuke.2006-11-21_22.06.18
> Error: block.c:264 Volume data error at 0:0! Wanted ID:
> "BB02", got "". Buffer discarded.
> 21-Nov 22:06 lifcsys4-dir: RestoreLuke.2006-11-21_22.06.18
> Error: Bacula 1.36.2 (28Feb05): 21-Nov-2006 22:06:22
>   JobId:  51
>   Job:RestoreLuke.2006-11-21_22.06.18
>   Client: luke-fd
>   Start time: 21-Nov-2006 22:06:21
>   End time:   21-Nov-2006 22:06:22
>   Files Expected: 5,371
>   Files Restored: 0
>   Bytes Restored: 0
>   Rate:   0.0 KB/s
>   FD Errors:  0
>   FD termination status:  OK
>   SD termination status:  Error
>   Termination:*** Restore Error ***
> 
> *
> 
> Any ideas ?
> 
> Thank.
> 
> 
> Jean-Michel Caricand 
>  
> [EMAIL PROTECTED]
>  
> 
> Accédez au courrier électronique de La Poste 
> sur www.laposte.net ou sur 3615 LAPOSTENET (0,34€ TTC /mn) 
> 1 Giga de stockage gratuit – Antispam et antivirus intégrés 
> 
> 
> 
> 
>
-
> Take Surveys. Earn Cash. Influence the Future of IT
> Join SourceForge.net's Techsay panel and you'll get the
chance to share your
> opinions on IT & business topics through brief surveys - and
earn cash
>
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV
> ___
> Bacula-users mailing list
> Bacula-users@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/bacula-users
> 

Jean-Michel Caricand 
 
[EMAIL PROTECTED]
 

Accédez au courrier électronique de La Poste 
sur www.laposte.net ou sur 3615 LAPOSTENET (0,34€ TTC /mn) 
1 Giga de stockage gratuit – Antispam et antivirus intégrés 




-
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your
opinions on IT & business topics through brief surveys - and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

Re: [Bacula-users] Isn't this great?

[Erich Prinz]

So Thorsten, you're using a Bacula beta build on production server?  
Can this be true?

;-)

Erich

On Nov 22, 2006, at 3:13 AM, Thorsten Engel wrote:

> I did a restore of 2.3 GB in 220.000 files and - thanks to bacula -  
> it only
> took 12 minutes! Including 2 tape changes! Our credit card  
> shopserver is up
> and running again! Really great is, that restoring 200.000 files  
> from the
> first tape in 1.7 GB only took *3 minutes!*.
>
> I only wanted to let you know
>
> thorsten
>
>
> Betreff: Bacula: Restore Error of batbox-fd Full (Job Id: 7219)
>
> 22-Nov 09:55 batbox-dir: Start Restore Job RestoreFiles. 
> 2006-11-22_09.55.16
> 22-Nov 09:55 batbox-sd: 3301 Issuing autochanger "loaded drive 0"  
> command.
> 22-Nov 09:55 batbox-sd: 3302 Autochanger "loaded drive 0", result  
> is Slot
> 50.
> 22-Nov 09:56 batbox-sd: 3301 Issuing autochanger "loaded drive 0"  
> command.
> 22-Nov 09:56 batbox-sd: 3302 Autochanger "loaded drive 0", result  
> is Slot
> 50.
> 22-Nov 09:56 batbox-sd: 3307 Issuing autochanger "unload slot 50,  
> drive 0"
> command.
> 22-Nov 09:57 batbox-sd: 3304 Issuing autochanger "load slot 3,  
> drive 0"
> command.
> 22-Nov 09:57 batbox-sd: 3305 Autochanger "load slot 3, drive 0",  
> status is
> OK.
> 22-Nov 09:57 batbox-sd: 3301 Issuing autochanger "loaded drive 0"  
> command.
> 22-Nov 09:57 batbox-sd: 3302 Autochanger "loaded drive 0", result  
> is Slot 3.
> 22-Nov 09:58 batbox-sd: Ready to read from volume "DLT003S" on device
> "NEO2000-Drive-1" (/dev/nst0).
> 22-Nov 09:58 batbox-sd: Forward spacing to file:block 79:0.
> 22-Nov 10:00 batbox-fd: RestoreFiles.2006-11-22_09.55.16 Error:
> attribs.c:420 Unable to set file owner /mnt/batdisk/restore/var/run/ 
> cgisock:
> ERR=Datei oder Verzeichnis nicht gefunden
> 22-Nov 10:00 batbox-fd: RestoreFiles.2006-11-22_09.55.16 Error:
> attribs.c:426 Unable to set file modes /mnt/batdisk/restore/var/run/ 
> cgisock:
> ERR=Datei oder Verzeichnis nicht gefunden
> 22-Nov 10:00 batbox-fd: RestoreFiles.2006-11-22_09.55.16 Error:
> attribs.c:436 Unable to set file times /mnt/batdisk/restore/var/run/ 
> cgisock:
> ERR=Datei oder Verzeichnis nicht gefunden
> 22-Nov 10:00 batbox-fd: RestoreFiles.2006-11-22_09.55.16 Error:
> attribs.c:420 Unable to set file owner
> /mnt/batdisk/restore/var/run/pptp/255.255.255.255:213.183.3.252:  
> ERR=Datei
> oder Verzeichnis nicht gefunden
> 22-Nov 10:00 batbox-fd: RestoreFiles.2006-11-22_09.55.16 Error:
> attribs.c:426 Unable to set file modes
> /mnt/batdisk/restore/var/run/pptp/255.255.255.255:213.183.3.252:  
> ERR=Datei
> oder Verzeichnis nicht gefunden
> 22-Nov 10:00 batbox-fd: RestoreFiles.2006-11-22_09.55.16 Error:
> attribs.c:436 Unable to set file times
> /mnt/batdisk/restore/var/run/pptp/255.255.255.255:213.183.3.252:  
> ERR=Datei
> oder Verzeichnis nicht gefunden
> 22-Nov 10:01 batbox-fd: RestoreFiles.2006-11-22_09.55.16 Error:
> attribs.c:420 Unable to set file owner /mnt/batdisk/restore/dev/log:
> ERR=Datei oder Verzeichnis nicht gefunden
> 22-Nov 10:01 batbox-fd: RestoreFiles.2006-11-22_09.55.16 Error:
> attribs.c:426 Unable to set file modes /mnt/batdisk/restore/dev/log:
> ERR=Datei oder Verzeichnis nicht gefunden
> 22-Nov 10:01 batbox-fd: RestoreFiles.2006-11-22_09.55.16 Error:
> attribs.c:436 Unable to set file times /mnt/batdisk/restore/dev/log:
> ERR=Datei oder Verzeichnis nicht gefunden
> 22-Nov 10:01 batbox-sd: End of Volume at file 79 on device "NEO2000- 
> Drive-1"
> (/dev/nst0), Volume "DLT003S"
> 22-Nov 10:01 batbox-sd: 3301 Issuing autochanger "loaded drive 0"  
> command.
> 22-Nov 10:01 batbox-sd: 3302 Autochanger "loaded drive 0", result  
> is Slot 3.
> 22-Nov 10:03 batbox-sd: RestoreFiles.2006-11-22_09.55.16 Warning:
> acquire.c:223 Wrong Volume mounted on device "NEO2000-Drive-1" (/ 
> dev/nst0):
> Wanted DLT027S have DLT003S
> 22-Nov 10:03 batbox-sd: 3301 Issuing autochanger "loaded drive 0"  
> command.
> 22-Nov 10:03 batbox-sd: 3302 Autochanger "loaded drive 0", result  
> is Slot 3.
> 22-Nov 10:03 batbox-sd: 3307 Issuing autochanger "unload slot 3,  
> drive 0"
> command.
> 22-Nov 10:04 batbox-sd: 3304 Issuing autochanger "load slot 50,  
> drive 0"
> command.
> 22-Nov 10:04 batbox-sd: 3305 Autochanger "load slot 50, drive 0",  
> status is
> OK.
> 22-Nov 10:04 batbox-sd: 3301 Issuing autochanger "loaded drive 0"  
> command.
> 22-Nov 10:04 batbox-sd: 3302 Autochanger "loaded drive 0", result  
> is Slot
> 50.
> 22-Nov 10:04 batbox-sd: Ready to read from volume "DLT027S" on device
> "NEO2000-Drive-1" (/dev/nst0).
> 22-Nov 10:04 batbox-sd: Forward spacing to file:block 149:0.
> 22-Nov 10:07 batbox-dir: RestoreFiles.2006-11-22_09.55.16 Error:  
> Bacula
> 1.39.20 (22Aug06): 22-Nov-2006 10:07:45
>   JobId:  7219
>   Job:RestoreFiles.2006-11-22_09.55.16
>   Client: batbox-fd
>   Start time: 22-Nov-2006 09:55:18
>   End time:   22-Nov-2006 10:07:45
>   Files Expected: 223,813
>   Files Restor

Re: [Bacula-users] Retaining Job information

[Martin Simmons]

> On Tue, 21 Nov 2006 23:30:00 +0100, Kern Sibbald said:
> 
> Yes, and what people tend to forget and thus claim that pruning does not work 
> is that the Volume Retention interval must also expire before the volume will 
> be recycled.

Maybe there should be an "explain" mode that would print all the decisions
about recycling in a particular job?

__Martin

-
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your
opinions on IT & business topics through brief surveys - and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

Re: [Bacula-users] bfill error - Attempt to WEOF on non-appendable Volume

[Ralf Gross]

Kern Sibbald said:
> On Wednesday 22 November 2006 08:12, Ralf Gross wrote:
>> Ralf Gross schrieb:
>> > I'm testing bacula 1.39.28 with our new NEC-T40A changer (LTO3
Ultrium
>> > drive). btape's test and fill commands were successful. Only bfill
finished with an error message.
>> >
>> > +19-Nov 21:58 btape:
End of Volume "" at 425:8650 on device "NEC-T40A" (/dev/nst0). Write
of 64512 bytes got -1.
>> > 19-Nov 21:58 btape: Re-read of last block succeeded.
>> >
>> > Write failed at block 6596150.
>> > 19-Nov 21:58 btape: Fatal Error at dev.c:1655 because:
>> > dev.c:1654 Attempt to WEOF on non-appendable Volume
>> > btape: btape.c:483 Bad status from weof. ERR=dev.c:1654 Attempt to
WEOF on non-appendable Volume
>> >
>> >
>> > Is this something I have to worry about?
>> Any ideas on this? I just want to know if this is something I have to
take care about before I declare the T40A changer is working fine with
bacula.
>
> Without knowing the details, this does not look too good.  It appears
more
> like a bug than a tape problem, but if I were you I would resolve the
problem
> before relying on the changer.  Possibly part of the problem is that you
have
> edited the output rather than presenting everything so I cannot see
everything.
>
> In any case, btape should not be getting this kind of error.  At a first
glance, it looks like btape attempted to write on the tape after
re-reading
> the last block, which is not what it normally does -- I suspect this is
because you edited the messages.  In any case, with the little snipit of
output it is impossible to get an idea of what is causing the problem.
>

Well, bfill doesn't give me much more output than the above.


--- bfill ---

*bfill
btape: btape.c:2536 Begin writing Bacula blocks of 64512 bytes.
+++
[bacula is filling the tape]
22-Nov 15:05 btape:
End of Volume "" at 427:8518 on device "LTO3" (/dev/nst0). Write of 64512
bytes got -1.
22-Nov 15:05 btape: Re-read of last block succeeded.

Write failed at block 6596018.
22-Nov 15:05 btape: Fatal Error at dev.c:1655 because:
dev.c:1654 Attempt to WEOF on non-appendable Volume
btape: btape.c:483 Bad status from weof. ERR=dev.c:1654 Attempt to WEOF on
non-appendable Volume



Here is btape's output:


--- test ---

./btape /dev/nst0
Tape block granularity is 1024 bytes.
btape: butil.c:272 Using device: "/dev/nst0" for writing.
22-Nov 13:08 btape: 3301 Issuing autochanger "loaded? drive 0" command.
22-Nov 13:08 btape: 3302 Autochanger "loaded? drive 0", result is Slot 1.
22-Nov 13:08 btape: 3301 Issuing autochanger "loaded? drive 0" command.
22-Nov 13:08 btape: 3302 Autochanger "loaded? drive 0", result is Slot 1.
btape: btape.c:356 open device "LTO3" (/dev/nst0): OK
*test

=== Write, rewind, and re-read test ===

I'm going to write 1000 records and an EOF
then write 1000 records and an EOF, then rewind,
and re-read the data to verify that it is correct.

This is an *essential* feature ...

btape: btape.c:813 Wrote 1000 blocks of 64412 bytes.
btape: btape.c:487 Wrote 1 EOF to "LTO3" (/dev/nst0)
btape: btape.c:829 Wrote 1000 blocks of 64412 bytes.
btape: btape.c:487 Wrote 1 EOF to "LTO3" (/dev/nst0)
btape: btape.c:838 Rewind OK.
1000 blocks re-read correctly.
Got EOF on tape.
1000 blocks re-read correctly.
=== Test Succeeded. End Write, rewind, and re-read test ===


=== Write, rewind, and position test ===

I'm going to write 1000 records and an EOF
then write 1000 records and an EOF, then rewind,
and position to a few blocks and verify that it is correct.

This is an *essential* feature ...

btape: btape.c:925 Wrote 1000 blocks of 64412 bytes.
btape: btape.c:487 Wrote 1 EOF to "LTO3" (/dev/nst0)
btape: btape.c:941 Wrote 1000 blocks of 64412 bytes.
btape: btape.c:487 Wrote 1 EOF to "LTO3" (/dev/nst0)
btape: btape.c:950 Rewind OK.
Reposition to file:block 0:4
Block 5 re-read correctly.
Reposition to file:block 0:200
Block 201 re-read correctly.
Reposition to file:block 0:999
Block 1000 re-read correctly.
Reposition to file:block 1:0
Block 1001 re-read correctly.
Reposition to file:block 1:600
Block 1601 re-read correctly.
Reposition to file:block 1:999
Block 2000 re-read correctly.
=== Test Succeeded. End Write, rewind, and re-read test ===



=== Append files test ===

This test is essential to Bacula.

I'm going to write one record  in file 0,
   two records in file 1,
 and three records in file 2

btape: btape.c:457 Rewound "LTO3" (/dev/nst0)
btape: btape.c:1559 Wrote one record of 64412 bytes.
btape: btape.c:1561 Wrote block to device.
btape: btape.c:487 Wrote 1 EOF to "LTO3" (/dev/nst0)
btape: btape.c:1559 Wrote one record of 64412 bytes.
btape: btape.c:1561 Wrote block to device.
btape: btape.c:1559 Wrote one record of 64412 bytes.
btape: btape.c:1561 Wrote block to device.
btape: btape.c:487 Wrote 1

Re: [Bacula-users] attn: Kern Re: Incremental backup, not accurate?

[Alan Brown]

On Tue, 21 Nov 2006, Kern Sibbald wrote:

> The voting determines the priority, but for a feature to be implemented, 
> it requires a developer to implement it.  Item 1 was encryption, and 
> Landon implemented that. Item 2 was migration, I implemented that.  No 
> one signed up for Item 3, which is the project you want, accurate 
> restoration of files.

>> I believe it is not true that other backup systems don't handle this.
>
> I would like to know what you base that belief on.  My understanding is 
> that virtually all backup systems decide to backup or not on the OS time 
> stamps as Bacula does, and hence suffer the same problem.

The only package I'm aware of which doesn't rely on OS timestamps uses a 
database to keep snapshots of the filesystem state (and is quite 
expensive)


Bacula has an extensive database, why not USE it?


==


The fundamental problem with almost all backup software is an assumption 
that file timestamps will only ever increase, never decrease. While this 
is right most of the time, it's not right all the time.



The current problem is that Bacula (in common with almost all other backup 
software) only looks for mtime or ctime higher than the last backup start 
time.

If for whatever reason a file appears with Mtime and ctime PRIOR to the 
last backup start time (eg rsynced in, or last backup was of an unattached 
filesystem stub...), it won't get looked at.

These can be taken care of by comparing the current filesystem
 state with the last known filesystem state in the database and
 backing up files which have appeared regardless of timertamp.

Once you start doing this, it is possible to note if a file has gone 
missing and flag it as deleted in the database - that takes care of 
restores bringing back zombie files, effectively giving "snapshot" 
restoration without having to replicate the entire database.


Another worse-case scenario:

An attacker replaces a system-critical file with another of the same size 
and tweaks ctime+mtime. This won't be backed up, even if the file is on a 
different inode (some versions of file replacement may not change the 
inode being used, some filesystems (reiser) do not store inode 
information. This means there is no way of telling when the file was 
changed and renders all backups suspect.


The only way to deal with this is checksumming the file vs
 stored file checksums and that is computationally expensive.


> Whatever the answer to that question is, knowing it will not change anything
> concerning getting the feature implemented in Bacula.

We are willing to put some money into sponsoring development.

> There are a lot of ways to help get code in Bacula implemented ... it just
> requires a bit of imagination, and some action.  There are a large number of
> *big* users out there, but over the last year there have been very few (zero
> I think) offers of help from them -- no code submissions, no contributions.

*Big* users are the ones who stand to benefit most from these changes, 
however we're academic and cash-strapped. :-(

AB


-
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your
opinions on IT & business topics through brief surveys - and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

Re: [Bacula-users] bfill error - Attempt to WEOF on non-appendable Volume

[Kern Sibbald]

Well, I have no idea what bfill is.  It must be some silly hold over from long 
ago.   I just deleted it from the source code.

If "fill" works, you are in good shape. Please ignore the bfill junk ...

Regards,

Kern


On Wednesday 22 November 2006 17:23, Ralf Gross wrote:
> Kern Sibbald said:
> > On Wednesday 22 November 2006 08:12, Ralf Gross wrote:
> >> Ralf Gross schrieb:
> >> > I'm testing bacula 1.39.28 with our new NEC-T40A changer (LTO3
> Ultrium
> >> > drive). btape's test and fill commands were successful. Only bfill
> finished with an error message.
> >> >
> >> > +19-Nov 21:58 btape:
> End of Volume "" at 425:8650 on device "NEC-T40A" (/dev/nst0). Write
> of 64512 bytes got -1.
> >> > 19-Nov 21:58 btape: Re-read of last block succeeded.
> >> >
> >> > Write failed at block 6596150.
> >> > 19-Nov 21:58 btape: Fatal Error at dev.c:1655 because:
> >> > dev.c:1654 Attempt to WEOF on non-appendable Volume
> >> > btape: btape.c:483 Bad status from weof. ERR=dev.c:1654 Attempt to
> WEOF on non-appendable Volume
> >> >
> >> >
> >> > Is this something I have to worry about?
> >> Any ideas on this? I just want to know if this is something I have to
> take care about before I declare the T40A changer is working fine with
> bacula.
> >
> > Without knowing the details, this does not look too good.  It appears
> more
> > like a bug than a tape problem, but if I were you I would resolve the
> problem
> > before relying on the changer.  Possibly part of the problem is that you
> have
> > edited the output rather than presenting everything so I cannot see
> everything.
> >
> > In any case, btape should not be getting this kind of error.  At a first
> glance, it looks like btape attempted to write on the tape after
> re-reading
> > the last block, which is not what it normally does -- I suspect this is
> because you edited the messages.  In any case, with the little snipit of
> output it is impossible to get an idea of what is causing the problem.
> >
> 
> Well, bfill doesn't give me much more output than the above.
> 
> 
> --- bfill ---
> 
> *bfill
> btape: btape.c:2536 Begin writing Bacula blocks of 64512 bytes.
> +++
> [bacula is filling the tape]
> 22-Nov 15:05 btape:
> End of Volume "" at 427:8518 on device "LTO3" (/dev/nst0). Write of 64512
> bytes got -1.
> 22-Nov 15:05 btape: Re-read of last block succeeded.
> 
> Write failed at block 6596018.
> 22-Nov 15:05 btape: Fatal Error at dev.c:1655 because:
> dev.c:1654 Attempt to WEOF on non-appendable Volume
> btape: btape.c:483 Bad status from weof. ERR=dev.c:1654 Attempt to WEOF on
> non-appendable Volume
> 
> 
> 
> Here is btape's output:
> 
> 
> --- test ---
> 
> ./btape /dev/nst0
> Tape block granularity is 1024 bytes.
> btape: butil.c:272 Using device: "/dev/nst0" for writing.
> 22-Nov 13:08 btape: 3301 Issuing autochanger "loaded? drive 0" command.
> 22-Nov 13:08 btape: 3302 Autochanger "loaded? drive 0", result is Slot 1.
> 22-Nov 13:08 btape: 3301 Issuing autochanger "loaded? drive 0" command.
> 22-Nov 13:08 btape: 3302 Autochanger "loaded? drive 0", result is Slot 1.
> btape: btape.c:356 open device "LTO3" (/dev/nst0): OK
> *test
> 
> === Write, rewind, and re-read test ===
> 
> I'm going to write 1000 records and an EOF
> then write 1000 records and an EOF, then rewind,
> and re-read the data to verify that it is correct.
> 
> This is an *essential* feature ...
> 
> btape: btape.c:813 Wrote 1000 blocks of 64412 bytes.
> btape: btape.c:487 Wrote 1 EOF to "LTO3" (/dev/nst0)
> btape: btape.c:829 Wrote 1000 blocks of 64412 bytes.
> btape: btape.c:487 Wrote 1 EOF to "LTO3" (/dev/nst0)
> btape: btape.c:838 Rewind OK.
> 1000 blocks re-read correctly.
> Got EOF on tape.
> 1000 blocks re-read correctly.
> === Test Succeeded. End Write, rewind, and re-read test ===
> 
> 
> === Write, rewind, and position test ===
> 
> I'm going to write 1000 records and an EOF
> then write 1000 records and an EOF, then rewind,
> and position to a few blocks and verify that it is correct.
> 
> This is an *essential* feature ...
> 
> btape: btape.c:925 Wrote 1000 blocks of 64412 bytes.
> btape: btape.c:487 Wrote 1 EOF to "LTO3" (/dev/nst0)
> btape: btape.c:941 Wrote 1000 blocks of 64412 bytes.
> btape: btape.c:487 Wrote 1 EOF to "LTO3" (/dev/nst0)
> btape: btape.c:950 Rewind OK.
> Reposition to file:block 0:4
> Block 5 re-read correctly.
> Reposition to file:block 0:200
> Block 201 re-read correctly.
> Reposition to file:block 0:999
> Block 1000 re-read correctly.
> Reposition to file:block 1:0
> Block 1001 re-read correctly.
> Reposition to file:block 1:600
> Block 1601 re-read correctly.
> Reposition to file:block 1:999
> Block 2000 re-read correctly.
> === Test Succeeded. End Write, rewind, and re-read test ===
> 
> 
> 
> === Append files test ===
> 
> This test is essential to Bacula.
> 
> I

Re: [Bacula-users] attn: Kern Re: Incremental backup, not accurate?

[Kern Sibbald]

On Wednesday 22 November 2006 17:54, Alan Brown wrote:
> On Tue, 21 Nov 2006, Kern Sibbald wrote:
> 
> > The voting determines the priority, but for a feature to be implemented, 
> > it requires a developer to implement it.  Item 1 was encryption, and 
> > Landon implemented that. Item 2 was migration, I implemented that.  No 
> > one signed up for Item 3, which is the project you want, accurate 
> > restoration of files.
> 
> >> I believe it is not true that other backup systems don't handle this.
> >
> > I would like to know what you base that belief on.  My understanding is 
> > that virtually all backup systems decide to backup or not on the OS time 
> > stamps as Bacula does, and hence suffer the same problem.
> 
> The only package I'm aware of which doesn't rely on OS timestamps uses a 
> database to keep snapshots of the filesystem state (and is quite 
> expensive)
> 
> 
> Bacula has an extensive database, why not USE it?

This is no problem. It has been planned.  This database is already used to 
detect changed files for Verify.  It works very well.

> 
> 
> ==
> 
> 
> The fundamental problem with almost all backup software is an assumption 
> that file timestamps will only ever increase, never decrease. While this 
> is right most of the time, it's not right all the time.
> 
> 
> 
> The current problem is that Bacula (in common with almost all other backup 
> software) only looks for mtime or ctime higher than the last backup start 
> time.
> 
> If for whatever reason a file appears with Mtime and ctime PRIOR to the 
> last backup start time (eg rsynced in, or last backup was of an unattached 
> filesystem stub...), it won't get looked at.
> 
>   These can be taken care of by comparing the current filesystem
>  state with the last known filesystem state in the database and
>  backing up files which have appeared regardless of timertamp.
> 
> Once you start doing this, it is possible to note if a file has gone 
> missing and flag it as deleted in the database - that takes care of 
> restores bringing back zombie files, effectively giving "snapshot" 
> restoration without having to replicate the entire database.
> 
> 
> Another worse-case scenario:
> 
> An attacker replaces a system-critical file with another of the same size 
> and tweaks ctime+mtime. This won't be backed up, even if the file is on a 
> different inode (some versions of file replacement may not change the 
> inode being used, some filesystems (reiser) do not store inode 
> information. This means there is no way of telling when the file was 
> changed and renders all backups suspect.
> 
> 
>   The only way to deal with this is checksumming the file vs
>  stored file checksums and that is computationally expensive.
> 
> 
> > Whatever the answer to that question is, knowing it will not change 
anything
> > concerning getting the feature implemented in Bacula.
> 
> We are willing to put some money into sponsoring development.
> 
> > There are a lot of ways to help get code in Bacula implemented ... it just
> > requires a bit of imagination, and some action.  There are a large number 
of
> > *big* users out there, but over the last year there have been very few 
(zero
> > I think) offers of help from them -- no code submissions, no 
contributions.
> 
> *Big* users are the ones who stand to benefit most from these changes, 
> however we're academic and cash-strapped. :-(

Yes, even though there are some quite large academic sites, I didn't consider 
them as *Big* users in my prior email.  

If there are a few more users like yourself that want this feature and are 
willing to contribute directly to a developer, please get in touch with me 
oof list because a developer contacted me today and I am sure he could be 
encouraged to implement this feature.  If possible please specify what you 
could afford.  

I would really like to see this implemented.  All the mechanisms already 
exist, it is just a matter of implementing them.

Best regards,

Kern

-
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your
opinions on IT & business topics through brief surveys - and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

Re: [Bacula-users] bfill error - Attempt to WEOF on non-appendable Volume

[Ralf Gross]

Kern Sibbald said:
> Well, I have no idea what bfill is.  It must be some silly hold over from
> long ago.   I just deleted it from the source code.
>
> If "fill" works, you are in good shape. Please ignore the bfill junk ...

Ok, very good.

I did a quick test against 1.38.11 where bfill seems to work.

*weof
btape: btape.c:469 Wrote 1 EOF to "LTO3" (/dev/nst0)

Ralf


-
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your
opinions on IT & business topics through brief surveys - and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

[Bacula-users] Storage definitions

[Kern Sibbald]

Hello,

Defining and figuring out what storage device is going to be used has become a 
bit too complicated in version 1.39.x.  This is mainly due to the fact that 
we have two different storage devices for each job and three possible places 
to specify them:

Migration job:

 read storage: job storage resource
 read storage: pool storage resource
 read storage: run override resource
 read storage: command line???

 write storage: pool next pool storage resource
   (yea, try to figure out the above in C it is
 job->pool->next_pool->storage

Backup Job:
 write storage: job storage resource
 write storage: pool storage resource
 write storage: run override resource
 write storage: command line???

Restore Job:
  read storage: who knows, probably the same as
 Migration.  To be checked.

So my question to you all is: do you agree on the precedence for the Migration 
read storage and Backup write storage? 

Best regards,

Kern

-
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your
opinions on IT & business topics through brief surveys - and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

Re: [Bacula-users] Bacula restore on windows client

[Don MacArthur]

Try it with upper case "D", as in "D:/tmp/bacula-restore". 

On Wed, 2006-11-22 at 11:43 +0100, Etienne ETOURNAY wrote:
> Hi !
> 
> When I want restore a Windows client, "Where:  /tmp/bacula-restore" = 
> "C:\tmp\bacula-restore" on Windows.
> My drive C:\ is very small : I want to restore to D:\tmp\bacula-restore .
> But "Where:  d:/tmp/bacula-restore" don't work !
> 
> How I can to do it ?
> 
> Regards,
> Etienne ETOURNAY
> 
> -
> Take Surveys. Earn Cash. Influence the Future of IT
> Join SourceForge.net's Techsay panel and you'll get the chance to share your
> opinions on IT & business topics through brief surveys - and earn cash
> http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV
> ___
> Bacula-users mailing list
> Bacula-users@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/bacula-users

-
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your
opinions on IT & business topics through brief surveys - and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

Re: [Bacula-users] [Bacula-devel] Storage definitions

[David Boyes]

> Defining and figuring out what storage device is going to be used has
> become a
> bit too complicated in version 1.39.x.  This is mainly due to the fact
> that
> we have two different storage devices for each job and three possible
> places
> to specify them:

My 2 cents worth:

This can be simplified dramatically by specifying the Storage resource
only in the Pool definition, and then specifying only the Pool in all
other places. Since (IMHO) pools are highly unlikely to span multiple
SDs, identifying what SD to use is selectable from the devices available
in the Storage resource associated with the pool. Some notes below: 
 
> Migration job:
>  read storage: job storage resource

Shouldn't the migration job always be using the Pool resource being
migrated and feeding to the NextPool, or overriding the NextPool
resource, not the storage resource? For migration, we're looking at
volumes in pools, not individual volumes (other than the case where we
have deliberately limited the selection criteria to a specific volume
within a pool via the selection keyword), so there should be no reason
to specify individual devices for migration jobs. 

>  read storage: pool storage resource

Correct. 

>  read storage: run override resource

See above. Migration is about pools, not devices. If you need to migrate
to volumes that are not in a pool then you set up a temporary pool and
draw it's volumes from the scratch pool. 

>  read storage: command line???

Not sure what you mean by this. 

>  write storage: pool next pool storage resource
>(yea, try to figure out the above in C it is
>  job->pool->next_pool->storage

Correct as I understand the implementation. See comments above. 
 
> Backup Job:
>  write storage: job storage resource

See above for discussion of pools. If pools drive the selection of
volumes, the pool definition will clearly define what SD should be used,
unambiguously and consistently across all actions in Bacula. 

>  write storage: pool storage resource

Ditto. 

>  write storage: run override resource

Ditto.

>  write storage: command line???

Not sure again. 

> Restore Job:
>   read storage: who knows, probably the same as
>  Migration.  To be checked.

In a normal case, the pool definition of the pool containing the volume
should determine the SD to use. In the nasty case of full disaster, you
could temporarily override the pool storage resource from the command
line, but that (in my mind) would be a situation where you are clearly
out of the normal parameters, and are doing it consciously. 

See comments above wrt to Migration. The same cases exist; migration
just makes this problem more visible. 


-
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your
opinions on IT & business topics through brief surveys - and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

Re: [Bacula-users] Retaining Job information

[Mike Reinehr]

Dave,

On Tuesday 21 November 2006 15:43, DAve wrote:
> Martin Simmons wrote:
> >> On Mon, 20 Nov 2006 15:07:27 -0500, DAve  said:
> >>
> >> I've read through the following portions of the manual and I think I
> >> might not be able to do what I need.
> >>
> >> http://bacula.org/rel-manual/Automatic_Volume_Recycling.html
> >> http://bacula.org/rel-manual/Basic_Volume_Management.html
> >> http://bacula.org/rel-manual/Bacula_Freque_Asked_Questi.html#SECTION0003
> >>43
> >> http://bacula.org/rel-manual/Bacula_Freque_Asked_Questi.html#SECTION0003
> >>425000
> >>
> >> I am trying to backup a simple disaster recovery set, kept for a single
> >> day (there is local tape making routine backups), and then reused 24
> >> hours later. The backup set is for off site emergencies only. This all
> >> works, just fine.
> >>
> >> Now I want to generate a report for the past thirty days to show the
> >> backups were made. Currently I can't get my job information to stay in
> >> the DB so I can query it for reporting.
> >>
> >> I have the following set,
> >> Client {
> >>Name = alli-fd
> >>Address = fd.alli.com
> >>FDPort = 49101
> >>Catalog = DataVault
> >>Password = "o/hEQs5oQgO103AfF1cvrmhW21BuodfrzN"
> >>File Retention = 2 days
> >>Job Retention = 30 days
> >>AutoPrune = yes
> >>}
> >>
> >> Pool {
> >>Name = Daily-Alli-Pool
> >>Pool Type = Backup
> >>Recycle = yes
> >>AutoPrune = yes
> >>Volume Retention = 1 day
> >>Accept Any Volume = yes
> >>LabelFormat = "Daily-alli-"
> >>Maximum Volume Jobs = 1
> >>Maximum Volumes = 2
> >>}
> >>
> >> I think I can change my Maximum Volume Jobs to 30 and get the result I
> >> want which is, 30 days of jobs in the catalog, only two Volumes in use.
> >> I am trying it tonight. If I am misunderstanding something, smack me.
> >
> > It will be confusing if you still have Volume Retention = 1 day.
> >
> > Also, it is overkill for the goal of "disaster recovery set kept for a
> > single day ... and then reused 24 hours later."  It implies keeping 30-60
> > days of backups in the 2 volumes.
> >
> > I think the problem you have is that the job information must be removed
> > from the catalog before the volume can be recycled.  Therefore, if you
> > want to keep only 1-2 days of backups then the catalog can only contain
> > 1-2 jobs.
>
> That is how I read the docs as well, and the test showed the same. So I
> understand it correctly, good.
>
> I understand the problem with what they requested, but I don't so sales
> and I am not the IT manager for the client (This is for a client within
> our building). They believe they just need to to get fresh copies of
> certain files into our NOC every night for DR purposes. There is only
> one current copy of the files in each volume, the previous volume is
> purged shortly after.
>
> Doesn't make sense to me, but it pays the same as a strategy that does.
>
> DAve

I imagine you've already thought of this, but it sounds like the solution to 
this dilema is to have a run-after-job which will execute an sql query on the 
database to extract the necessary information and save it in another database 
or text file. Then, once a month you can generate your report from this file, 
purge it and start over.

Cheers!

cmr

-- 
Debian 'Etch': Registered Linux User #241964

"More laws, less justice." -- Marcus Tullius Ciceroca, 42 BC


-
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your
opinions on IT & business topics through brief surveys - and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

Re: [Bacula-users] [Bacula-devel] Storage definitions

[Arno Lehmann]

Hi,

On 11/22/2006 7:35 PM, David Boyes wrote:
 > Kern:
>>Defining and figuring out what storage device is going to be used has
>>become a
>>bit too complicated in version 1.39.x.  This is mainly due to the fact
>>that
>>we have two different storage devices for each job and three possible
>>places
>>to specify them:
> 
> 
> My 2 cents worth:
> 
> This can be simplified dramatically by specifying the Storage resource
> only in the Pool definition, and then specifying only the Pool in all
> other places.

I agree.

I did not always do that, but I learned that the flexibility of pools 
across different storage devices and media types is less valuable than a 
simple setup.

Since the pool definition allows linking to a storage device I'm convinced.

The only exception, in my opinion, would be to define the storage on the 
command line - that should alwys have the highest priority. It would 
allow to create pools across different storage devices, but in case of 
serious problems or during emergency restores this might be helpful. 
Finally, it's not a fault to allow users to shoot themselves in the leg 
- as long as there are enough warnings :-)

Arno
-- 
IT-Service Lehmann[EMAIL PROTECTED]
Arno Lehmann  http://www.its-lehmann.de

-
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your
opinions on IT & business topics through brief surveys - and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

Re: [Bacula-users] Problems with autochanger

[Arno Lehmann]

Hi,

On 11/22/2006 10:56 AM, Jukka Laaksola wrote:
> Hi!
> 
> We have Bacula version 1.38.11-5~bpo.1 which is from backports.org to 
> Debian Sarge.
> 
> The system has a HP autochanger with a magazine with six LTO1 tape 
> slots. There are ten clients to backup. Our idea is to do full backups 
> on every tuesday and then incremental backups on the rest of the week 
> and also on the mondays. And after two weeks to change manually 
> magazine. We have now three magazines so same tapes will be overwriten 
> after six weeks.
> In the configuration there are only one pool to backup. All volumes are 
> in this same pool.
> The Volume retention of the pool is 30 days. So after six weeks the 
> first volumes should be recycleable.
> 
> The problem are appendable tapes which are not in changer. We say 
> "unmount" and change the magazine. Then we say "update slots" and 
> "mount". The inchanger flags gets updated successful. Then we mark all 
> removed appendable volumes as Used.
> But in the middle of the night at backup time the bacula ask to mount 
> some appendable volume which is not in the changer.
> 
> Let's take an example of last night.

Ok, I'll skip the example in my reply. The problem is, Bacula recycles 
volumes not in changer and does not stick to the ones availabale.

This has been discussed a number of times. I don't know how it should 
work, but I know what you can do to fix your problem :-)

- Use 1.39.27 or whatever the current beta version is.
- Make sure you actually want to run beta software in a production 
environment (but I'm quite sure that the features that existed in 1.38 
have not stopped working).
- Use the ned enabled/disabled flag for volumes and disable the volumes 
you remove from the changer.

(David, Kern, I'm still thinking about the volume management stuff, I 
just don't find enough time for real work on it...)

Arno

-- 
IT-Service Lehmann[EMAIL PROTECTED]
Arno Lehmann  http://www.its-lehmann.de

-
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your
opinions on IT & business topics through brief surveys - and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

Re: [Bacula-users] [Bacula-devel] Storage definitions

[David Boyes]

> I did not always do that, but I learned that the flexibility of pools
> across different storage devices and media types is less valuable than
a
> simple setup.

Multi-type pools also make backup and storage management policy almost
impossible to implement. If we define a pool as an administrative group
of volumes with a certain set of behavior characteristics (eg, similar
media type, physical location, capability (size of volume, etc),
handling characteristics, etc) it's a lot clearer what's happening.
Wildly different media types and characteristics in the same entity are
both confusing and almost impossible to schedule. 

There's also the operator characteristics. If this were z/OS or some
other operating system with a common media mount capability, then it
wouldn't be a big deal, but the current approach really makes it hard,
esp for wildly different media types. 

Pre-migration, the mixed pools made sense in that you could copy data
from volume to volume by specifying the right options and you did all
the footwork of keeping track of the devices manually. Now,
post-migration, that's bad behavior. Bacula should be in control of
device selection and usage regardless of what data is involved, and to
do that, the selection process needs to be clear in terms of what
volume, with what capability do we need to select, and where should it
be processed. If pools are clear that "this volume in Pool X is
associated with this SD" then the rest of the selection parts fall into
line pretty quickly. 

The side effect is that if you are sharing drives with other
applications (this means when you change magazines, etc too,
Arno...8-)), then you have to start taking the drive offline to Bacula
before you snitch it for some other purpose. This is a bit more
discipline that some shops have at the moment, but it also results in a
much more predictable behavior. 

> The only exception, in my opinion, would be to define the storage on
the
> command line - that should alwys have the highest priority. It would
> allow to create pools across different storage devices, but in case of
> serious problems or during emergency restores this might be helpful.

I'd even go further and argue that the only place that this makes sense
is in a restore when the entire bacula infrastructure has been destroyed
(eg in brestore). Normal Bacula shouldn't permit specifying devices;
that's a big part of the value of the Bacula engine -- it manages all
that stuff. 

The key item for that would be to provide the ability to disable a
device (in case it's broken, has a stuck tape, etc) and take it out of
the device selection process w/o removing it from the configuration
file. 

> Finally, it's not a fault to allow users to shoot themselves in the
leg
> - as long as there are enough warnings :-)

See above. The only place where this makes sense IMHO is in the case of
total destruction of the Bacula infrastructure. At that point, you don't
have any choice but to permit it (and in fact, you have to require it to
get the necessary info to do the restores). 


-
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your
opinions on IT & business topics through brief surveys - and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

Re: [Bacula-users] Retaining Job information

[DAve]

Mike Reinehr wrote:
> Dave,
> 
> On Tuesday 21 November 2006 15:43, DAve wrote:
>> Martin Simmons wrote:
 On Mon, 20 Nov 2006 15:07:27 -0500, DAve  said:
 I've read through the following portions of the manual and I think I
 might not be able to do what I need.

 http://bacula.org/rel-manual/Automatic_Volume_Recycling.html
 http://bacula.org/rel-manual/Basic_Volume_Management.html
 http://bacula.org/rel-manual/Bacula_Freque_Asked_Questi.html#SECTION0003
 43
 http://bacula.org/rel-manual/Bacula_Freque_Asked_Questi.html#SECTION0003
 425000

 I am trying to backup a simple disaster recovery set, kept for a single
 day (there is local tape making routine backups), and then reused 24
 hours later. The backup set is for off site emergencies only. This all
 works, just fine.

 Now I want to generate a report for the past thirty days to show the
 backups were made. Currently I can't get my job information to stay in
 the DB so I can query it for reporting.

 I have the following set,
 Client {
Name = alli-fd
Address = fd.alli.com
FDPort = 49101
Catalog = DataVault
Password = "o/hEQs5oQgO103AfF1cvrmhW21BuodfrzN"
File Retention = 2 days
Job Retention = 30 days
AutoPrune = yes
}

 Pool {
Name = Daily-Alli-Pool
Pool Type = Backup
Recycle = yes
AutoPrune = yes
Volume Retention = 1 day
Accept Any Volume = yes
LabelFormat = "Daily-alli-"
Maximum Volume Jobs = 1
Maximum Volumes = 2
}

 I think I can change my Maximum Volume Jobs to 30 and get the result I
 want which is, 30 days of jobs in the catalog, only two Volumes in use.
 I am trying it tonight. If I am misunderstanding something, smack me.
>>> It will be confusing if you still have Volume Retention = 1 day.
>>>
>>> Also, it is overkill for the goal of "disaster recovery set kept for a
>>> single day ... and then reused 24 hours later."  It implies keeping 30-60
>>> days of backups in the 2 volumes.
>>>
>>> I think the problem you have is that the job information must be removed
>>> from the catalog before the volume can be recycled.  Therefore, if you
>>> want to keep only 1-2 days of backups then the catalog can only contain
>>> 1-2 jobs.
>> That is how I read the docs as well, and the test showed the same. So I
>> understand it correctly, good.
>>
>> I understand the problem with what they requested, but I don't so sales
>> and I am not the IT manager for the client (This is for a client within
>> our building). They believe they just need to to get fresh copies of
>> certain files into our NOC every night for DR purposes. There is only
>> one current copy of the files in each volume, the previous volume is
>> purged shortly after.
>>
>> Doesn't make sense to me, but it pays the same as a strategy that does.
>>
>> DAve
> 
> I imagine you've already thought of this, but it sounds like the solution to 
> this dilema is to have a run-after-job which will execute an sql query on the 
> database to extract the necessary information and save it in another database 
> or text file. Then, once a month you can generate your report from this file, 
> purge it and start over.
> 
> Cheers!
> 
> cmr
> 

Already working on it, great minds must think alike ;^)

DAve


-- 
Three years now I've asked Google why they don't have a
logo change for Memorial Day. Why do they choose to do logos
for other non-international holidays, but nothing for
Veterans?

Maybe they forgot who made that choice possible.

-
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your
opinions on IT & business topics through brief surveys - and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

Re: [Bacula-users] attn: Kern Re: Incremental backup, not accurate?

[Chris Hoogendyk]

Kern Sibbald wrote:
> On Wednesday 22 November 2006 17:54, Alan Brown wrote:
>   
>> The only package I'm aware of which doesn't rely on OS timestamps uses a 
>> database to keep snapshots of the filesystem state (and is quite 
>> expensive)
>>
>>
>> Bacula has an extensive database, why not USE it?
>> 
>
> This is no problem. It has been planned.  This database is already used to 
> detect changed files for Verify.  It works very well.
>
>   
>> ==
>>
>>
>> The fundamental problem with almost all backup software is an assumption 
>> that file timestamps will only ever increase, never decrease. While this 
>> is right most of the time, it's not right all the time.
>>
>>
>>
>> The current problem is that Bacula (in common with almost all other backup 
>> software) only looks for mtime or ctime higher than the last backup start 
>> time.
>>
>> If for whatever reason a file appears with Mtime and ctime PRIOR to the 
>> last backup start time (eg rsynced in, or last backup was of an unattached 
>> filesystem stub...), it won't get looked at.
>>
>>  These can be taken care of by comparing the current filesystem
>>  state with the last known filesystem state in the database and
>>  backing up files which have appeared regardless of timertamp.
>>
>> Once you start doing this, it is possible to note if a file has gone 
>> missing and flag it as deleted in the database - that takes care of 
>> restores bringing back zombie files, effectively giving "snapshot" 
>> restoration without having to replicate the entire database.
>>
>>
>> Another worse-case scenario:
>>
>> An attacker replaces a system-critical file with another of the same size 
>> and tweaks ctime+mtime. This won't be backed up, even if the file is on a 
>> different inode (some versions of file replacement may not change the 
>> inode being used, some filesystems (reiser) do not store inode 
>> information. This means there is no way of telling when the file was 
>> changed and renders all backups suspect.
>>
>>
>>  The only way to deal with this is checksumming the file vs
>>  stored file checksums and that is computationally expensive.
>>
>> 
> I would really like to see this implemented.  All the mechanisms already 
> exist, it is just a matter of implementing them.
>
> Best regards,
>
> Kern



Kern Sibbald wrote:
> On Wednesday 22 November 2006 17:54, Alan Brown wrote:
>   
>> The only package I'm aware of which doesn't rely on OS timestamps uses a 
>> database to keep snapshots of the filesystem state (and is quite 
>> expensive)
>>
>>
>> Bacula has an extensive database, why not USE it?
>> 
>
> This is no problem. It has been planned.  This database is already used to 
> detect changed files for Verify.  It works very well.
>
>   
>> ==
>>
>>
>> The fundamental problem with almost all backup software is an assumption 
>> that file timestamps will only ever increase, never decrease. While this 
>> is right most of the time, it's not right all the time.
>>
>>
>>
>> The current problem is that Bacula (in common with almost all other backup 
>> software) only looks for mtime or ctime higher than the last backup start 
>> time.
>>
>> If for whatever reason a file appears with Mtime and ctime PRIOR to the 
>> last backup start time (eg rsynced in, or last backup was of an unattached 
>> filesystem stub...), it won't get looked at.
>>
>>  These can be taken care of by comparing the current filesystem
>>  state with the last known filesystem state in the database and
>>  backing up files which have appeared regardless of timertamp.
>>
>> Once you start doing this, it is possible to note if a file has gone 
>> missing and flag it as deleted in the database - that takes care of 
>> restores bringing back zombie files, effectively giving "snapshot" 
>> restoration without having to replicate the entire database.
>>
>>
>> Another worse-case scenario:
>>
>> An attacker replaces a system-critical file with another of the same size 
>> and tweaks ctime+mtime. This won't be backed up, even if the file is on a 
>> different inode (some versions of file replacement may not change the 
>> inode being used, some filesystems (reiser) do not store inode 
>> information. This means there is no way of telling when the file was 
>> changed and renders all backups suspect.
>>
>>
>>  The only way to deal with this is checksumming the file vs
>>  stored file checksums and that is computationally expensive.
>>
>> 
> I would really like to see this implemented.  All the mechanisms already 
> exist, it is just a matter of implementing them.
>
> Best regards,
>
> Kern

I've been following the list for a while, but haven't commented on
anything yet.

Retrospect had an awesome reputation in the Mac world for over a decade.
One of the things that made them really good was their ability to
recognize duplicate files (even on different machines within the same
backup set

[Bacula-users] The number of files mismatch!

[Rob Ostrander]

We are getting "The number of files mismatch!" error when a job tries to 
append to a tape in V1.38.11.

We did not see this until adding a LTO changer with 2 drives(and 
increasing max concurrent jobs).   I set max concurrent to 1, everywhere 
I could find and will see if it fixes it.

I've read through this 
http://bacula.org/rel-manual/Tips_Suggestions.html#SECTION000357000 
and I wonder if anyone else is having this issue or could provide some 
insight to fixing it?

20-Nov 22:44 wcsan1-dir: Recycled volume "Logs-0001"
20-Nov 22:47 wcsan1-dir: Start Backup JobId 456, 
Job=wcsan1.2006-11-16_23.05.00
20-Nov 22:47 wcsan1-sd: 3301 Issuing autochanger "loaded drive 0" command.
20-Nov 22:47 wcsan1-sd: 3302 Autochanger "loaded drive 0", result is Slot 6.
20-Nov 22:47 wcsan1-sd: 3307 Issuing autochanger "unload slot 6, drive 
0" command.
20-Nov 22:49 wcsan1-sd: 3304 Issuing autochanger "load slot 1, drive 0" 
command.
20-Nov 22:50 wcsan1-sd: 3305 Autochanger "load slot 1, drive 0", status 
is OK.
20-Nov 22:50 wcsan1-sd: 3301 Issuing autochanger "loaded drive 0" command.
20-Nov 22:50 wcsan1-sd: 3302 Autochanger "loaded drive 0", result is Slot 1.
20-Nov 22:51 wcsan1-sd: Volume "Inc-0001" previously written, moving to 
end of data.
20-Nov 22:57 wcsan1-sd: wcsan1.2006-11-16_23.05.00 Error: I cannot write 
on Volume "Inc-0001" because:
The number of files mismatch! Volume=14 Catalog=15
20-Nov 22:57 wcsan1-sd: Marking Volume "Inc-0001" in Error in Catalog.
20-Nov 23:00 wcsan1-dir: Recycled volume "Inc-0002"
20-Nov 23:00 wcsan1-sd: 3301 Issuing autochanger "loaded drive 0" command.
20-Nov 23:00 wcsan1-sd: 3302 Autochanger "loaded drive 0", result is Slot 1.
20-Nov 23:00 wcsan1-sd: 3307 Issuing autochanger "unload slot 1, drive 
0" command.
20-Nov 23:02 wcsan1-sd: 3304 Issuing autochanger "load slot 2, drive 0" 
command.
20-Nov 23:02 wcsan1-sd: 3305 Autochanger "load slot 2, drive 0", status 
is OK.
20-Nov 23:02 wcsan1-sd: 3301 Issuing autochanger "loaded drive 0" command.
20-Nov 23:02 wcsan1-sd: 3302 Autochanger "loaded drive 0", result is Slot 2.
20-Nov 23:03 wcsan1-sd: Recycled volume "Inc-0002" on device "Drive-1" 
(/dev/nst0), all previous data lost.
21-Nov 00:38 wcsan1-dir: Bacula 1.38.11 (28Jun06): 21-Nov-2006 00:38:13


-
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your
opinions on IT & business topics through brief surveys - and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

Re: [Bacula-users] restore takes "forever"

[Andras Horvai]

Hi,

Well thanks for your answer. I will change my volumes size useing the
Max Volume Bytes setting. What settings do you reccomend if you used
this feature? If I issue the list volumes command in console I got
this:

+-+---+---++--+--+-+--+---+-+-+
| MediaId | VolumeName| VolStatus | VolBytes   | VolFiles | 
VolRetention | Recycle | Slot | InChanger | MediaType   | LastWritten |
+-+---+---++--+--+-+--+---+-+-+
|   3 | ServersDiff01 | Used  | 51,590,350,000 |   12 |  
432,000 |   1 |0 | 1 | ServersFile | 2006-11-17 00:24:09 |
|   4 | ServersDiff02 | Append| 33,334,244,961 |7 |  
432,000 |   1 |0 | 1 | ServersFile | 2006-11-22 23:01:41 |
+-+---+---++--+--+-+--+---+-+-+

What does VolFiles mean? I didn't find it in the documentation.

Andras

>>
>> Really?  We don't have any volumes quite 50G, but we have several 30G
>> volumes, and have yet to have a problem restoring from them.

> The problem is, as far as I know, that for some reason file volumes are
> not seeked but linearly read from the beginning.

> That means that, if you restore something from the end of a 50GB file 
> the whole 50 GB have to be read but the data is not actually needed. 
> Obviously, a seek would happen virtually instananeous.

> If you have set up your volumes in a way that you usually need the whole
> volume, or only the first part of it, that will not be a problem.

> This was discussed on the mailing lists several times during the last 
> years, but I see nothing mentioning it in the manual.

> Arno



-
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your
opinions on IT & business topics through brief surveys - and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

Re: [Bacula-users] restore takes "forever"

[Bill Moran]

On Wed, 22 Nov 2006 23:11:59 +0100
Andras Horvai <[EMAIL PROTECTED]> wrote:

> Hi,
> 
> Well thanks for your answer. I will change my volumes size useing the
> Max Volume Bytes setting. What settings do you reccomend if you used
> this feature? If I issue the list volumes command in console I got
> this:
> 
> +-+---+---++--+--+-+--+---+-+-+
> | MediaId | VolumeName| VolStatus | VolBytes   | VolFiles | 
> VolRetention | Recycle | Slot | InChanger | MediaType   | LastWritten 
> |
> +-+---+---++--+--+-+--+---+-+-+
> |   3 | ServersDiff01 | Used  | 51,590,350,000 |   12 |  
> 432,000 |   1 |0 | 1 | ServersFile | 2006-11-17 00:24:09 |
> |   4 | ServersDiff02 | Append| 33,334,244,961 |7 |  
> 432,000 |   1 |0 | 1 | ServersFile | 2006-11-22 23:01:41 |
> +-+---+---++--+--+-+--+---+-+-+
> 
> What does VolFiles mean? I didn't find it in the documentation.

I've been wondering about that.  I wish one of the developers would chime in 
here.

I seem to remember that a trick used on tape drives is to write an EOF marker 
every
so often, then when restoring, the drive can quickly seek X EOF markers ahead 
before
it has to slow down to read through the data.

If I'm understanding this correctly, there's no reason Bacula can't do the same
thing with file volumes.  If it writes an EOF marker every 4G (which it seems 
to,
based on your output) it can seek() to the within 4G of the data it needs, then
it only needs to read() through a maximum of 4G to get the data.

Again, I wish a developer would chime in and comment on whether I'm correct or 
not.


-
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your
opinions on IT & business topics through brief surveys - and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

Re: [Bacula-users] restore takes "forever"

[Arno Lehmann]

Hello,

On 11/22/2006 11:11 PM, Andras Horvai wrote:
> Hi,
> 
> Well thanks for your answer. I will change my volumes size useing the
> Max Volume Bytes setting. What settings do you reccomend if you used
> this feature?

I'd recommend something like 2 to 8 GB, but this is not based on any 
testing at all.

I'm having clients with disk-based setups and that size range worked ok, 
but it also depends on your jobs. If you expect to restore mainly full 
jobs, setting your volumes up so that they contain only one job is one 
possibility, but usually I prefer to have a large number of smaller volumes.

Also, using volumes sizes of about 4.5 GB is nice when you want to 
export volumes to DVD.

> If I issue the list volumes command in console I got
> this:

By the way: Keep in mind that changing the configuration only affects 
newly created volumes, not existing ones. The 'update' command is needed 
to modify existing volume entries in the catalog.

Existing volumes will only be truncated when they are recycled.
> +-+---+---++--+--+-+--+---+-+-+
> | MediaId | VolumeName| VolStatus | VolBytes   | VolFiles | 
> VolRetention | Recycle | Slot | InChanger | MediaType   | LastWritten 
> |
> +-+---+---++--+--+-+--+---+-+-+
> |   3 | ServersDiff01 | Used  | 51,590,350,000 |   12 |  
> 432,000 |   1 |0 | 1 | ServersFile | 2006-11-17 00:24:09 |
> |   4 | ServersDiff02 | Append| 33,334,244,961 |7 |  
> 432,000 |   1 |0 | 1 | ServersFile | 2006-11-22 23:01:41 |
> +-+---+---++--+--+-+--+---+-+-+
> 
> What does VolFiles mean? I didn't find it in the documentation.

VolFiles is the number of logical files in a volume. This is tape 
terminology - with tapes, you (usually) position to a certain file and 
read sequentially from then on.

File marks take up tape space, so you've got to find a good balance 
obetween smaller file sizes and a larger number of files for fast 
positioning, and larger files with more usable tape capacity.

You don't have to worry about this with disk based volumes - these have 
other peculiarities. As you found out.

Arno

-- 
IT-Service Lehmann[EMAIL PROTECTED]
Arno Lehmann  http://www.its-lehmann.de

-
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your
opinions on IT & business topics through brief surveys - and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

[Bacula-users] FSFE becomes the legal guardian of the Bacula Project

[Dan Langille]

Read more here:

http://mail.fsfeurope.org/pipermail/press-release/2006q4/000161.html

-- 
Dan Langille : Software Developer looking for work
my resume: http://www.freebsddiary.org/dan_langille.php



-
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your
opinions on IT & business topics through brief surveys - and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

Re: [Bacula-users] attn: Kern Re: Incremental backup, not accurate?

[Kern Sibbald]

Hello,

This feature has been planned from the very first day that a line of code was 
written for Bacula.  It is listed in the "projects" file but still has not 
found a developer.

Regards,

Kern

On Wednesday 22 November 2006 21:33, Chris Hoogendyk wrote:
> 
> Kern Sibbald wrote:
> > On Wednesday 22 November 2006 17:54, Alan Brown wrote:
> >   
> >> The only package I'm aware of which doesn't rely on OS timestamps uses a 
> >> database to keep snapshots of the filesystem state (and is quite 
> >> expensive)
> >>
> >>
> >> Bacula has an extensive database, why not USE it?
> >> 
> >
> > This is no problem. It has been planned.  This database is already used to 
> > detect changed files for Verify.  It works very well.
> >
> >   
> >> ==
> >>
> >>
> >> The fundamental problem with almost all backup software is an assumption 
> >> that file timestamps will only ever increase, never decrease. While this 
> >> is right most of the time, it's not right all the time.
> >>
> >>
> >>
> >> The current problem is that Bacula (in common with almost all other 
backup 
> >> software) only looks for mtime or ctime higher than the last backup start 
> >> time.
> >>
> >> If for whatever reason a file appears with Mtime and ctime PRIOR to the 
> >> last backup start time (eg rsynced in, or last backup was of an 
unattached 
> >> filesystem stub...), it won't get looked at.
> >>
> >>These can be taken care of by comparing the current filesystem
> >>  state with the last known filesystem state in the database and
> >>  backing up files which have appeared regardless of timertamp.
> >>
> >> Once you start doing this, it is possible to note if a file has gone 
> >> missing and flag it as deleted in the database - that takes care of 
> >> restores bringing back zombie files, effectively giving "snapshot" 
> >> restoration without having to replicate the entire database.
> >>
> >>
> >> Another worse-case scenario:
> >>
> >> An attacker replaces a system-critical file with another of the same size 
> >> and tweaks ctime+mtime. This won't be backed up, even if the file is on a 
> >> different inode (some versions of file replacement may not change the 
> >> inode being used, some filesystems (reiser) do not store inode 
> >> information. This means there is no way of telling when the file was 
> >> changed and renders all backups suspect.
> >>
> >>
> >>The only way to deal with this is checksumming the file vs
> >>  stored file checksums and that is computationally expensive.
> >>
> >> 
> > I would really like to see this implemented.  All the mechanisms already 
> > exist, it is just a matter of implementing them.
> >
> > Best regards,
> >
> > Kern
> 
> 
> 
> Kern Sibbald wrote:
> > On Wednesday 22 November 2006 17:54, Alan Brown wrote:
> >   
> >> The only package I'm aware of which doesn't rely on OS timestamps uses a 
> >> database to keep snapshots of the filesystem state (and is quite 
> >> expensive)
> >>
> >>
> >> Bacula has an extensive database, why not USE it?
> >> 
> >
> > This is no problem. It has been planned.  This database is already used to 
> > detect changed files for Verify.  It works very well.
> >
> >   
> >> ==
> >>
> >>
> >> The fundamental problem with almost all backup software is an assumption 
> >> that file timestamps will only ever increase, never decrease. While this 
> >> is right most of the time, it's not right all the time.
> >>
> >>
> >>
> >> The current problem is that Bacula (in common with almost all other 
backup 
> >> software) only looks for mtime or ctime higher than the last backup start 
> >> time.
> >>
> >> If for whatever reason a file appears with Mtime and ctime PRIOR to the 
> >> last backup start time (eg rsynced in, or last backup was of an 
unattached 
> >> filesystem stub...), it won't get looked at.
> >>
> >>These can be taken care of by comparing the current filesystem
> >>  state with the last known filesystem state in the database and
> >>  backing up files which have appeared regardless of timertamp.
> >>
> >> Once you start doing this, it is possible to note if a file has gone 
> >> missing and flag it as deleted in the database - that takes care of 
> >> restores bringing back zombie files, effectively giving "snapshot" 
> >> restoration without having to replicate the entire database.
> >>
> >>
> >> Another worse-case scenario:
> >>
> >> An attacker replaces a system-critical file with another of the same size 
> >> and tweaks ctime+mtime. This won't be backed up, even if the file is on a 
> >> different inode (some versions of file replacement may not change the 
> >> inode being used, some filesystems (reiser) do not store inode 
> >> information. This means there is no way of telling when the file was 
> >> changed and renders all backups suspect.
> >>
> >>
> >>The only way to deal with this is checksumming the file vs
> >>  stored file checksums and that is 

Re: [Bacula-users] The number of files mismatch!

[Kern Sibbald]

Most likely either the Dir or SD has crashed during a backup, or you haven't 
successfully run the btape test and fill commands, in which case, you Device 
resource is not properly configured for your OS/drive.

On Wednesday 22 November 2006 22:17, Rob Ostrander wrote:
> We are getting "The number of files mismatch!" error when a job tries to 
> append to a tape in V1.38.11.
> 
> We did not see this until adding a LTO changer with 2 drives(and 
> increasing max concurrent jobs).   I set max concurrent to 1, everywhere 
> I could find and will see if it fixes it.
> 
> I've read through this 
> 
http://bacula.org/rel-manual/Tips_Suggestions.html#SECTION000357000 
> and I wonder if anyone else is having this issue or could provide some 
> insight to fixing it?
> 
> 20-Nov 22:44 wcsan1-dir: Recycled volume "Logs-0001"
> 20-Nov 22:47 wcsan1-dir: Start Backup JobId 456, 
> Job=wcsan1.2006-11-16_23.05.00
> 20-Nov 22:47 wcsan1-sd: 3301 Issuing autochanger "loaded drive 0" command.
> 20-Nov 22:47 wcsan1-sd: 3302 Autochanger "loaded drive 0", result is Slot 6.
> 20-Nov 22:47 wcsan1-sd: 3307 Issuing autochanger "unload slot 6, drive 
> 0" command.
> 20-Nov 22:49 wcsan1-sd: 3304 Issuing autochanger "load slot 1, drive 0" 
> command.
> 20-Nov 22:50 wcsan1-sd: 3305 Autochanger "load slot 1, drive 0", status 
> is OK.
> 20-Nov 22:50 wcsan1-sd: 3301 Issuing autochanger "loaded drive 0" command.
> 20-Nov 22:50 wcsan1-sd: 3302 Autochanger "loaded drive 0", result is Slot 1.
> 20-Nov 22:51 wcsan1-sd: Volume "Inc-0001" previously written, moving to 
> end of data.
> 20-Nov 22:57 wcsan1-sd: wcsan1.2006-11-16_23.05.00 Error: I cannot write 
> on Volume "Inc-0001" because:
> The number of files mismatch! Volume=14 Catalog=15
> 20-Nov 22:57 wcsan1-sd: Marking Volume "Inc-0001" in Error in Catalog.
> 20-Nov 23:00 wcsan1-dir: Recycled volume "Inc-0002"
> 20-Nov 23:00 wcsan1-sd: 3301 Issuing autochanger "loaded drive 0" command.
> 20-Nov 23:00 wcsan1-sd: 3302 Autochanger "loaded drive 0", result is Slot 1.
> 20-Nov 23:00 wcsan1-sd: 3307 Issuing autochanger "unload slot 1, drive 
> 0" command.
> 20-Nov 23:02 wcsan1-sd: 3304 Issuing autochanger "load slot 2, drive 0" 
> command.
> 20-Nov 23:02 wcsan1-sd: 3305 Autochanger "load slot 2, drive 0", status 
> is OK.
> 20-Nov 23:02 wcsan1-sd: 3301 Issuing autochanger "loaded drive 0" command.
> 20-Nov 23:02 wcsan1-sd: 3302 Autochanger "loaded drive 0", result is Slot 2.
> 20-Nov 23:03 wcsan1-sd: Recycled volume "Inc-0002" on device "Drive-1" 
> (/dev/nst0), all previous data lost.
> 21-Nov 00:38 wcsan1-dir: Bacula 1.38.11 (28Jun06): 21-Nov-2006 00:38:13
> 
> 
> -
> Take Surveys. Earn Cash. Influence the Future of IT
> Join SourceForge.net's Techsay panel and you'll get the chance to share your
> opinions on IT & business topics through brief surveys - and earn cash
> http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV
> ___
> Bacula-users mailing list
> Bacula-users@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/bacula-users
> 

-
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your
opinions on IT & business topics through brief surveys - and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

Re: [Bacula-users] Webmin supports Bacula

[Michael Nelson]

On Tue, November 21, 2006 6:10 pm, Dan Langille said:
> Webmin  , a open source IT system
> administration tool, now has support for Bacula:

Whoa, and it's pretty full-featured and very cool!  I wish I had had
access to that tool when setting up the two backup servers I set up using
"vi".



-- 
This message has been scanned for viruses and
dangerous content by MailScanner, and is
believed to be clean.


-
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your
opinions on IT & business topics through brief surveys - and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

Re: [Bacula-users] Problems with autochanger

[Jukka Laaksola]

Arno Lehmann wrote:
> Hi,
> 

Hi and thanks for the fast answer!

> On 11/22/2006 10:56 AM, Jukka Laaksola wrote:
> 
>>
>>The problem are appendable tapes which are not in changer. We say 
>>"unmount" and change the magazine. Then we say "update slots" and 
>>"mount". The inchanger flags gets updated successful. Then we mark all 
>>removed appendable volumes as Used.
>>But in the middle of the night at backup time the bacula ask to mount 
>>some appendable volume which is not in the changer.
>>
> 
> 
> Ok, I'll skip the example in my reply. The problem is, Bacula recycles 
> volumes not in changer and does not stick to the ones availabale.

Yes. That's right.

> This has been discussed a number of times. I don't know how it should 
> work, but I know what you can do to fix your problem :-)
> 
> - Use 1.39.27 or whatever the current beta version is.
> - Make sure you actually want to run beta software in a production 
> environment (but I'm quite sure that the features that existed in 1.38 
> have not stopped working).
> - Use the ned enabled/disabled flag for volumes and disable the volumes 
> you remove from the changer.


I probably found the solutions with stable 1.38.11 version. When we 
change the magazine we update volumes recycle flags. Put the recycle 
flag enabled for the volumes in changer and disabled for all others. And 
also mark status to Used on all appendable volumes not in changer.

I will do a perl script for changing the magazine. It will first unmount 
the volumes and the ask to change the fysical magazine. After that it 
update slots and automatically update those recycle flags and status of 
appendable volumes not in changer to Used. And finally does mount.

I think so the answer for this problem could be a configuration variable 
like AcceptAnyVolume. Something like AcceptOnlyInChangerVolume Or 
just change values on AcceptAnyVolume from yes|no to yes|no|inchanger...

But for now I will do the script to change recycle flags for the volumes.


regards
Jukka

-- 
Jukka Laaksola
Netland Oy

-
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your
opinions on IT & business topics through brief surveys - and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users