Re: [Bacula-users] Incomplete backup - due to bsock error

2017-09-22 Thread Jerry Lowry 
Yes, kilchis is a bonifide hardware server. Only VM's I have are test
systems running on my desktop.

There are 2 copy jobs on this system. This particular job is the one that
typically runs long enough that it will need a new volume during the
night.  The other one will if it is run late in the day and the current
volume does not have very much space left on it. The other daily backup
jobs will wait until the copy job is finished, but there is nothing else
running on the system that utilizes the network except for VNC traffic.
This problem happened two weeks in a row and this last week it worked just
fine.  The one thing that is different is that I dropped all of the current
backup files and purged them from the DB. I then recreated new files to
backup to.  Just wondering if one of the files was writing on a
questionable sector on disk.  Nothing in the logs and smart does not give
any details on that.

I think I will call it a fluke and keep a watch on it in the future..
Thanks!

On Fri, Sep 22, 2017 at 10:27 AM, Martin Simmons 
wrote:

> That's odd -- the reading side looks normal to me until the error is
> detected.
>
> Also, "Connection reset by peer" doesn't normally occur when connected to
> the
> current machine.
>
> Is kilchis a real computer (not a VM)?
>
> Is this the only copy job that waits overnight for someone to label a new
> volume?
>
> Maybe something happens overnight on the system that causes networking to
> be
> disrupted in some subtle way, causing "Connection reset by peer" when the
> connection is closed cleanly?
>
> __Martin
>
>
> > On Tue, 19 Sep 2017 15:31:46 -0700, Jerry Lowry said:
> >
> > The reading side is the same system.  It is a copy job setup to backup
> > daily backups to the offsite backup disk.
> > The attachment is the bacula jobid 35202.
> >
> > jerry
> >
> > On Tue, Sep 19, 2017 at 10:08 AM, Martin Simmons 
> > wrote:
> >
> > > The email below is from the writing side of the copy job and the
> message:
> > >
> > > 13-Sep 08:43 kilchis JobId 35203: Error: bsock.c:849 Read error from
> > > Storage daemon:kilchis:9103: ERR=Connection reset by peer
> > >
> > > shows that the connection to the reading side of the job was closed
> > > unexpectedly from the reading end.
> > >
> > > Do you have the corresponding email from the reading side?  It will
> have a
> > > different JobId (but should mention JobId 35203) and should start with
> > > something like "Using Device ... to read."
> > >
> > > __Martin
> > >
> > >
> > > > On Mon, 18 Sep 2017 13:42:19 -0700, Jerry Lowry said:
> > > >
> > > > Martin,
> > > > Here is the complete email that was sent just before the "Copy Error"
> > > > message:
> > > >
> > > > 12-Sep 15:09 kilchis-dir JobId 35203: Using Device "MidSwap" to
> write.
> > > > 12-Sep 15:09 kilchis JobId 35203: Volume "homeMS-200" previously
> > > written, moving to end of data.
> > > > 12-Sep 15:27 kilchis JobId 35203: End of medium on Volume
> "homeMS-200"
> > > Bytes=1,932,735,274,146 Blocks=29,959,317 at 12-Sep-2017 15:27.
> > > > 12-Sep 15:28 kilchis JobId 35203: Job BackupUsers.2017-09-12_09.05.
> 09_50
> > > is waiting. Cannot find any appendable volumes.
> > > > Please use the "label" command to create a new Volume for:
> > > > Storage:  "MidSwap" (/MidSwap)
> > > > Pool: OffsiteMid
> > > > Media type:   File
> > > > 12-Sep 15:36 kilchis JobId 35203: Wrote label to prelabeled Volume
> > > "homeMS-201" on File device "MidSwap" (/MidSwap)
> > > > 12-Sep 15:36 kilchis JobId 35203: New volume "homeMS-201" mounted on
> > > device "MidSwap" (/MidSwap) at 12-Sep-2017 15:36.
> > > > 12-Sep 19:54 kilchis JobId 35203: End of medium on Volume
> "homeMS-201"
> > > Bytes=1,932,735,281,790 Blocks=29,959,315 at 12-Sep-2017 19:54.
> > > > 12-Sep 19:54 kilchis JobId 35203: Job BackupUsers.2017-09-12_09.05.
> 09_50
> > > is waiting. Cannot find any appendable volumes.
> > > > Please use the "label" command to create a new Volume for:
> > > > Storage:  "MidSwap" (/MidSwap)
> > > > Pool: OffsiteMid
> > > > Media type:   File
> > > > 12-Sep 20:57 kilchis JobId 35203: Job BackupUsers.2017-09-12_09.05.
> 09_50
> > > is waiting. Cannot find any appendable volumes.
> > > > Please use the "label" command to create a new Volume for:
> > > > Storage:  "MidSwap" (/MidSwap)
> > > > Pool: OffsiteMid
> > > > Media type:   File
> > > > 12-Sep 23:03 kilchis JobId 35203: Job BackupUsers.2017-09-12_09.05.
> 09_50
> > > is waiting. Cannot find any appendable volumes.
> > > > Please use the "label" command to create a new Volume for:
> > > > Storage:  "MidSwap" (/MidSwap)
> > > > Pool: OffsiteMid
> > > > Media type:   File
> > > > 13-Sep 03:15 kilchis JobId 35203: Job BackupUsers.2017-09-12_09.05.
> 09_50
> > > is waiting. Cannot find any appendable volumes.
> > > > Please use the "label" command to create a new Volume for:
> > > > Storage:  "MidSwap" (/MidSwap)
> > > > Pool:

Re: [Bacula-users] Incomplete backup - due to bsock error

2017-09-22 Thread Martin Simmons 
That's odd -- the reading side looks normal to me until the error is detected.

Also, "Connection reset by peer" doesn't normally occur when connected to the
current machine.

Is kilchis a real computer (not a VM)?

Is this the only copy job that waits overnight for someone to label a new
volume?

Maybe something happens overnight on the system that causes networking to be
disrupted in some subtle way, causing "Connection reset by peer" when the
connection is closed cleanly?

__Martin


> On Tue, 19 Sep 2017 15:31:46 -0700, Jerry Lowry said:
> 
> The reading side is the same system.  It is a copy job setup to backup
> daily backups to the offsite backup disk.
> The attachment is the bacula jobid 35202.
> 
> jerry
> 
> On Tue, Sep 19, 2017 at 10:08 AM, Martin Simmons 
> wrote:
> 
> > The email below is from the writing side of the copy job and the message:
> >
> > 13-Sep 08:43 kilchis JobId 35203: Error: bsock.c:849 Read error from
> > Storage daemon:kilchis:9103: ERR=Connection reset by peer
> >
> > shows that the connection to the reading side of the job was closed
> > unexpectedly from the reading end.
> >
> > Do you have the corresponding email from the reading side?  It will have a
> > different JobId (but should mention JobId 35203) and should start with
> > something like "Using Device ... to read."
> >
> > __Martin
> >
> >
> > > On Mon, 18 Sep 2017 13:42:19 -0700, Jerry Lowry said:
> > >
> > > Martin,
> > > Here is the complete email that was sent just before the "Copy Error"
> > > message:
> > >
> > > 12-Sep 15:09 kilchis-dir JobId 35203: Using Device "MidSwap" to write.
> > > 12-Sep 15:09 kilchis JobId 35203: Volume "homeMS-200" previously
> > written, moving to end of data.
> > > 12-Sep 15:27 kilchis JobId 35203: End of medium on Volume "homeMS-200"
> > Bytes=1,932,735,274,146 Blocks=29,959,317 at 12-Sep-2017 15:27.
> > > 12-Sep 15:28 kilchis JobId 35203: Job BackupUsers.2017-09-12_09.05.09_50
> > is waiting. Cannot find any appendable volumes.
> > > Please use the "label" command to create a new Volume for:
> > > Storage:  "MidSwap" (/MidSwap)
> > > Pool: OffsiteMid
> > > Media type:   File
> > > 12-Sep 15:36 kilchis JobId 35203: Wrote label to prelabeled Volume
> > "homeMS-201" on File device "MidSwap" (/MidSwap)
> > > 12-Sep 15:36 kilchis JobId 35203: New volume "homeMS-201" mounted on
> > device "MidSwap" (/MidSwap) at 12-Sep-2017 15:36.
> > > 12-Sep 19:54 kilchis JobId 35203: End of medium on Volume "homeMS-201"
> > Bytes=1,932,735,281,790 Blocks=29,959,315 at 12-Sep-2017 19:54.
> > > 12-Sep 19:54 kilchis JobId 35203: Job BackupUsers.2017-09-12_09.05.09_50
> > is waiting. Cannot find any appendable volumes.
> > > Please use the "label" command to create a new Volume for:
> > > Storage:  "MidSwap" (/MidSwap)
> > > Pool: OffsiteMid
> > > Media type:   File
> > > 12-Sep 20:57 kilchis JobId 35203: Job BackupUsers.2017-09-12_09.05.09_50
> > is waiting. Cannot find any appendable volumes.
> > > Please use the "label" command to create a new Volume for:
> > > Storage:  "MidSwap" (/MidSwap)
> > > Pool: OffsiteMid
> > > Media type:   File
> > > 12-Sep 23:03 kilchis JobId 35203: Job BackupUsers.2017-09-12_09.05.09_50
> > is waiting. Cannot find any appendable volumes.
> > > Please use the "label" command to create a new Volume for:
> > > Storage:  "MidSwap" (/MidSwap)
> > > Pool: OffsiteMid
> > > Media type:   File
> > > 13-Sep 03:15 kilchis JobId 35203: Job BackupUsers.2017-09-12_09.05.09_50
> > is waiting. Cannot find any appendable volumes.
> > > Please use the "label" command to create a new Volume for:
> > > Storage:  "MidSwap" (/MidSwap)
> > > Pool: OffsiteMid
> > > Media type:   File
> > > 13-Sep 08:23 kilchis JobId 35203: Wrote label to prelabeled Volume
> > "homeMS-202" on File device "MidSwap" (/MidSwap)
> > > 13-Sep 08:23 kilchis JobId 35203: New volume "homeMS-202" mounted on
> > device "MidSwap" (/MidSwap) at 13-Sep-2017 08:23.
> > > 13-Sep 08:43 kilchis JobId 35203: Error: bsock.c:849 Read error from
> > Storage daemon:kilchis:9103: ERR=Connection reset by peer
> > > 13-Sep 08:43 kilchis JobId 35203: Fatal error: append.c:271 Network
> > error reading from FD. ERR=Connection reset by peer
> > > 13-Sep 08:43 kilchis JobId 35203: Elapsed time=04:56:15, Transfer
> > rate=125.6 M Bytes/second
> > > 13-Sep 08:43 kilchis JobId 35203: Sending spooled attrs to the Director.
> > Despooling 1,533,148,574 bytes ...
> > >
> > > I don't have the job log. Interestingly, I did not have any problems with
> > > this or any other copy job before I upgraded.  I went from 5.2.13 to
> > 9.0.3
> > > of Bacula and latest version of MySql to Mariadb.  Not saying that this
> > is
> > > a problem, because I have 5 other copy jobs that work without error
> > still.
> > > This one just happens to be the biggest one.
> > >
> > > thanks,
> > > jerry
> > >
> > > On Mon, Sep 18, 2017 at 7:55 A

Re: [Bacula-users] print numbers without thousands separators

2017-09-22 Thread Dimitri Maziuk 

On 2017-09-22 09:47, Heitor Faria wrote:

Hello, John,

"I'm not sure I see the utility of copying and pasting any of the
other formatted numbers here."

I do.  Running calculations within a script, a tally of number of
files, job bytes, averages, etc.

Can't you just workaround using sed (sed 's/,//')?


Why? AFAIK vast majority of humans use nothing or space for thousands 
separator and comma for decimal. Why can't bacula get it right?


Dima


--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

Re: [Bacula-users] print numbers without thousands separators

2017-09-22 Thread Heitor Faria 
Hello, John, 

> "I'm not sure I see the utility of copying and pasting any of the other
> formatted numbers here."

> I do. Running calculations within a script, a tally of number of files, job
> bytes, averages, etc.

Can't you just workaround using sed (sed 's/,//')? 

Regards, 

> On Fri, Sep 22, 2017 at 9:56 AM, Phil Stracchino < ph...@caerllewys.net > 
> wrote:

>> On 09/22/17 09:04, Christoph Litauer wrote:
>> > Dear bacula users,

>> > using bacula for many years without problems now. Great software!
>> > One little thing would be nice:
>> > In bconsole, querying e.g. "List alle backups for a client" results in

>> > +-+---++---+-+--+---++
>>> | jobid | client | fileset | level | starttime | jobfiles | jobbytes | 
>>> volumename
>> > | |
>> > +-+---++---+-+--+---++
>>> | 821,494 | printhost | client files-to-backup | F | 2016-11-19 00:10:53 | 
>>> 222,727
>> > | | 5,945,291,349 | LTO6-012 |

>>> As you can see, all numbers are printed using thousands separators. I think 
>>> this
>> > is independent on the current locale (tried with C and en).
>>> These numbers cannot be reused (by copy/paste) within bconsole or some other
>> > scripts without reformatting.
>> > Selecting information using psql seems to ommit those separators.

>> > Would be nice, if bconsole didn't print separators within numbers.

>> ...At least, not within JobIDs. I'm not sure I see the utility of
>> copying and pasting any of the other formatted numbers here.

>> --
>> Phil Stracchino
>> Babylon Communications
>> ph...@caerllewys.net
>> p...@co.ordinate.org
>> Landline: +1.603.293.8485
>> Mobile: +1.603.998.6958

>> --
>> Check out the vibrant tech community on one of the world's most
>> engaging tech sites, Slashdot.org! http://sdm.link/slashdot
>> ___
>> Bacula-users mailing list
>> Bacula-users@lists.sourceforge.net
>> https://lists.sourceforge.net/lists/listinfo/bacula-users

> --
> ---
> John M. Lockard | U of Michigan - School of Information
> Unix Sys Admin | Suite 205 | 309 Maynard Street
> jlock...@umich.edu | Ann Arbor, MI 48104-2211
> www.umich.edu/~jlockard | 734-615-8776 | 734-763-9677 FAX
> ---
> - The University of Michigan will never ask you for your password -

> --
> Check out the vibrant tech community on one of the world's most
> engaging tech sites, Slashdot.org! http://sdm.link/slashdot
> ___
> Bacula-users mailing list
> Bacula-users@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/bacula-users

-- 
=== 
Heitor Medrado de Faria | CEO Bacula do Brasil | Visto EB-1 | LPIC-III | EMC 
05-001 | ITIL-F 
• Não seja tarifado pelo tamanho dos seus backups, conheça o Bacula Enterprise: 
http://www.bacula.com.br/enterprise/ 
• Ministro treinamento e implementação in-company do Bacula Community: 
http://www.bacula.com.br/in-company/ 
+55 61 98268-4220 | www.bacula.com.br 
 
Indicamos também as capacitações complementares: 
• Shell básico e Programação em Shell com Julio Neves. 
• Zabbix com Adail Host. 
 
--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

Re: [Bacula-users] print numbers without thousands separators

2017-09-22 Thread John Lockard 
"I'm not sure I see the utility of copying and pasting any of the other
formatted numbers here."

I do.  Running calculations within a script, a tally of number of files,
job bytes, averages, etc.

On Fri, Sep 22, 2017 at 9:56 AM, Phil Stracchino 
wrote:

> On 09/22/17 09:04, Christoph Litauer wrote:
> > Dear bacula users,
> >
> > using bacula for many years without problems now. Great software!
> > One little thing would be nice:
> > In bconsole, querying e.g. "List alle backups for a client" results in
> >
> > +-+---++---+
> -+--+---++
> > | jobid   | client| fileset| level | starttime
>  | jobfiles | jobbytes  | volumename |
> > +-+---++---+
> -+--+---++
> > | 821,494 | printhost | client files-to-backup | F | 2016-11-19
> 00:10:53 |  222,727 | 5,945,291,349 | LTO6-012   |
> >
> > As you can see, all numbers are printed using thousands separators. I
> think this is independent on the current locale (tried with C and en).
> > These numbers cannot be reused (by copy/paste) within bconsole or some
> other scripts without reformatting.
> > Selecting information using psql seems to ommit those separators.
> >
> > Would be nice, if bconsole didn't print separators within numbers.
>
>
> ...At least, not within JobIDs.  I'm not sure I see the utility of
> copying and pasting any of the other formatted numbers here.
>
>
> --
>   Phil Stracchino
>   Babylon Communications
>   ph...@caerllewys.net
>   p...@co.ordinate.org
>   Landline: +1.603.293.8485
>   Mobile:   +1.603.998.6958
>
> 
> --
> Check out the vibrant tech community on one of the world's most
> engaging tech sites, Slashdot.org! http://sdm.link/slashdot
> ___
> Bacula-users mailing list
> Bacula-users@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/bacula-users
>



-- 
---
 John M. Lockard |  U of Michigan - School of Information
  Unix Sys Admin |  Suite 205 | 309 Maynard Street
  jlock...@umich.edu |Ann Arbor, MI  48104-2211
 www.umich.edu/~jlockard  |
734-615-8776 | 734-763-9677 FAX
---
- The University of Michigan will never ask you for your password -
--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

Re: [Bacula-users] print numbers without thousands separators

2017-09-22 Thread Phil Stracchino 
On 09/22/17 10:06, John Lockard wrote:
> "I'm not sure I see the utility of copying and pasting any of the other
> formatted numbers here."
> 
> I do.  Running calculations within a script, a tally of number of files,
> job bytes, averages, etc.


True.  Though if I were doing that, I'd probably wrap bconsole directly
with the script, because that sounds like the kind of thing that if
you're going to do it at all, you should automate it.  And at that point
it's simple to filter out the commas.



-- 
  Phil Stracchino
  Babylon Communications
  ph...@caerllewys.net
  p...@co.ordinate.org
  Landline: +1.603.293.8485
  Mobile:   +1.603.998.6958

--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

Re: [Bacula-users] print numbers without thousands separators

2017-09-22 Thread Phil Stracchino 
On 09/22/17 09:04, Christoph Litauer wrote:
> Dear bacula users,
> 
> using bacula for many years without problems now. Great software!
> One little thing would be nice: 
> In bconsole, querying e.g. "List alle backups for a client" results in
> 
> +-+---++---+-+--+---++
> | jobid   | client| fileset| level | starttime   
> | jobfiles | jobbytes  | volumename |
> +-+---++---+-+--+---++
> | 821,494 | printhost | client files-to-backup | F | 2016-11-19 00:10:53 
> |  222,727 | 5,945,291,349 | LTO6-012   |
> 
> As you can see, all numbers are printed using thousands separators. I think 
> this is independent on the current locale (tried with C and en).
> These numbers cannot be reused (by copy/paste) within bconsole or some other 
> scripts without reformatting.
> Selecting information using psql seems to ommit those separators.
> 
> Would be nice, if bconsole didn't print separators within numbers.


...At least, not within JobIDs.  I'm not sure I see the utility of
copying and pasting any of the other formatted numbers here.


-- 
  Phil Stracchino
  Babylon Communications
  ph...@caerllewys.net
  p...@co.ordinate.org
  Landline: +1.603.293.8485
  Mobile:   +1.603.998.6958

--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

[Bacula-users] print numbers without thousands separators

2017-09-22 Thread Christoph Litauer 
Dear bacula users,

using bacula for many years without problems now. Great software!
One little thing would be nice: 
In bconsole, querying e.g. "List alle backups for a client" results in

+-+---++---+-+--+---++
| jobid   | client| fileset| level | starttime   | 
jobfiles | jobbytes  | volumename |
+-+---++---+-+--+---++
| 821,494 | printhost | client files-to-backup | F | 2016-11-19 00:10:53 |  
222,727 | 5,945,291,349 | LTO6-012   |

As you can see, all numbers are printed using thousands separators. I think 
this is independent on the current locale (tried with C and en).
These numbers cannot be reused (by copy/paste) within bconsole or some other 
scripts without reformatting.
Selecting information using psql seems to ommit those separators.

Would be nice, if bconsole didn't print separators within numbers.

--
Kind regards
Christoph
_
Uni Koblenz, Computing Centre, Office A 022
Postfach 201602, 56016 Koblenz 
Fon: +49 261 287-1311, Fax: -100 1311





--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

[Bacula-users] malformed message error when testing new fileset

2017-09-22 Thread Adam Weremczuk 

Hello,

I have tried a couple of different ways of implementing "Exclude Dir 
Containing" but it was failing when restarting director with e.g:


Config error: Keyword DirContaining not permitted in this resource
or
Config error: ExcludeDirContaining directive not permitted in Exclude.

With this in client's config file:

FileSet {
  Name = testclient_fileset
  Enable VSS = yes
  Include {
    Options {
  signature = SHA1
    }
    File = "c:/backedup"
    Exclude Dir Containing = .excludeme
  }
}

director doesn't complain and starts fine but then a problem manifests 
itself as below:


Connecting to Director bacula:9101
1000 OK: bacula_dir Version: 5.2.6 (21 February 2012)
Enter a period to cancel a command.
*estimate job=testclient_backup listing client=testclient_fd 
fileset=testclient_fileset

Using Catalog "bacula_catalog"
Connecting to Client testclient_fd at testclient.domainname.com:9102
Error sending include list.

22-Sep 09:46 bacula_dir JobId 0: Error: getmsg.c:194 Malformed message: 
Invalid FileSet command: valid FileSet command: valid FileSet command: 
valid FileSet command: valid FileSet command: valid FileSet command: 
valid FileSet command: valid FileSet command: valid FileSet command: 
valid FileSet command: valid FileSet command: valid FileSet command: 
valid FileSet command: valid FileSet command: valid FileSet command: 
valid FileSet command: valid FileSet command: valid FileSet command: 
valid FileSet command: valid FileSet command: valid FileSet command: 
valid FileSet command: val
22-Sep 09:46 bacula_dir JobId 0: Fatal error: Socket error on Include 
command: ERR=No data available


Can somebody advise what's wrong with my config or how to troubleshoot 
it further?


Regards
Adam


--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users