subject:"\[Bacula\-users\] Backup from client with deduplication"

Re: [Bacula-users] Backup from client with deduplication

2020-10-08 Thread Josh Fisher



On 10/8/20 5:00 AM, Radosław Korzeniewski wrote:

Hello,

śr., 7 paź 2020 o 13:59 Рукавцов Дмитрий Геннадьевич 
mailto:m...@santel-navi.ru>> napisał(a):


"Enable VSS = no" as i said not solving this problem, and i dunno
why, i don't care about locks. Bacula can't just copy files?

AFAIK, Windows is limited in such a way that it prohibits opening in 
different processes an already opened file. To workaround this 
fundamental problem the one has to create a filesystem snapshot (VSS 
is a framework for it) where you can open every file. In my opinion it 
has nothing to do with a deduplication filesystem.



In Windows you must care about locks. Files are opened in Windows with 
both an access mode and a share mode (ie. lock mode). If share mode is 
set to zero, then no other process may open the file, even as superuser. 
Otherwise, for any other share mode, other processes may open the file. 
So, most opened files could be opened for read by the bacula-fd process 
and backed up normally. But there are a few, mostly system files, that 
cannot be backed up without VSS because they are opened in exclusive 
mode (with no sharing).


It is possible to turn off VSS and then exclude those system files in 
the FileSet definition.





best regards
--
Radosław Korzeniewski
rados...@korzeniewski.net 


___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

Re: [Bacula-users] Backup from client with deduplication

2020-10-08 Thread Radosław Korzeniewski

Hello,

śr., 7 paź 2020 o 13:59 Рукавцов Дмитрий Геннадьевич 
napisał(a):

> "Enable VSS = no" as i said not solving this problem, and i dunno why, i
> don't care about locks. Bacula can't just copy files?
>
AFAIK, Windows is limited in such a way that it prohibits opening in
different processes an already opened file. To workaround this fundamental
problem the one has to create a filesystem snapshot (VSS is a framework for
it) where you can open every file. In my opinion it has nothing to do with
a deduplication filesystem.

best regards
-- 
Radosław Korzeniewski
rados...@korzeniewski.net
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

Re: [Bacula-users] Backup from client with deduplication

2020-10-07 Thread Рукавцов Дмитрий Геннадьевич

No i didn't testing, unable to restore to machine without deduplication 
is already an issue. And i don't think it will work with another machine 
cause you need to backup " chunk store" as well.


https://docs.microsoft.com/en-us/previous-versions/windows/desktop/dedup/backup-and-restore-of-data-deduplication-enabled-volumes?redirectedfrom=MSDN

"Enable VSS = no" as i said not solving this problem, and i dunno why, i 
don't care about locks. Bacula can't just copy files?



07.10.2020 12:45, Žiga Žvan пишет:


Yes, probably the deduplicated amount (54) instead od full amount 
(700). First I thought that there is just an error in bacula reporting 
the correct size (eg. incorrectly interpreting windows size/size on 
disk columns), but it is actual this size on disk. I also thought that 
perhaps I need to turn off Snapshot/VSS (and deal with file locks) but 
then I stopped testing this, because my restores were ok.


Did you try to enable deduplication on different server and then 
restore to this drive?


Lets wait for some reply in communiy...

Regards,
Ziga



On 07.10.2020 11:16, Рукавцов Дмитрий Геннадьевич wrote:

You mean you have 700gb raw data but bacula backing it up as 54gb? If 
you will restore it to the same windows 2012 server it will work, but 
if you will restore it to server which not have deduplication think 
most files will be corrupted.


I have no idea how to force bacula backup all data.

07.10.2020 11:48, Žiga Žvan пишет:


Hi,
I'm able to backup Windows 2012 server with deduplication enabled. 
My bacula server is also centos, bacula version 9.6.5 (on client and 
server). I have tested only individual file restores but it always 
worked for me. Backup is very small (54 GB for app. 700 GB raw 
data). I'm not sure how bacula manages to backup and encrypt 
deduplicated amount (I would expect that there will be 700 GB on the 
wire). I'm having some performance problems (see thread 
performance&design&configuration challenges).


I'm at the the end of testing bacula sw, so I'm not an experienced 
user. I would also appreciate if someone confirms, that this should 
work. If it helps, there is log from my last backup.

Kind regards,
Ziga



02-Oct 23:05 bacula-dir JobId 692: Start Backup JobId 692, 
Job=dc1-monthly-backup.2020-10-02_23.05.00_09
02-Oct 23:05 bacula-dir JobId 692: Created new Volume="dc1-monthly-vol-0297", 
Pool="dc1-monthly-pool", MediaType="File" in catalog.
02-Oct 23:05 bacula-dir JobId 692: Using Device "FSLocalBackup" to write.
02-Oct 23:05 bacula-sd JobId 692: Labeled new Volume "dc1-monthly-vol-0297" on File 
device "FSLocalBackup" (/storage/bacula/backup).
02-Oct 23:05 bacula-sd JobId 692: Wrote label to prelabeled Volume "dc1-monthly-vol-0297" 
on File device "FSLocalBackup" (/storage/bacula/backup)
02-Oct 23:05 dc1.kranj.cetrtapot.si-fd JobId 692: Generate VSS snapshots. 
Driver="Win64 VSS"
02-Oct 23:05 dc1.kranj.cetrtapot.si-fd JobId 692: Snapshot mount point: D:\
03-Oct 13:42 dc1.kranj.cetrtapot.si-fd JobId 692: VSS Writer (BackupComplete): "Task 
Scheduler Writer", State: 0x1 (VSS_WS_STABLE)
03-Oct 13:42 dc1.kranj.cetrtapot.si-fd JobId 692: VSS Writer (BackupComplete): "VSS 
Metadata Store Writer", State: 0x1 (VSS_WS_STABLE)
03-Oct 13:42 dc1.kranj.cetrtapot.si-fd JobId 692: VSS Writer (BackupComplete): 
"Performance Counters Writer", State: 0x1 (VSS_WS_STABLE)
03-Oct 13:42 dc1.kranj.cetrtapot.si-fd JobId 692: VSS Writer (BackupComplete): 
"System Writer", State: 0x1 (VSS_WS_STABLE)
03-Oct 13:42 dc1.kranj.cetrtapot.si-fd JobId 692: VSS Writer (BackupComplete): "ASR 
Writer", State: 0x1 (VSS_WS_STABLE)
03-Oct 13:42 dc1.kranj.cetrtapot.si-fd JobId 692: VSS Writer (BackupComplete): "IIS 
Config Writer", State: 0x1 (VSS_WS_STABLE)
03-Oct 13:42 dc1.kranj.cetrtapot.si-fd JobId 692: VSS Writer (BackupComplete): "WMI 
Writer", State: 0x1 (VSS_WS_STABLE)
03-Oct 13:42 dc1.kranj.cetrtapot.si-fd JobId 692: VSS Writer (BackupComplete): "IIS 
Metabase Writer", State: 0x1 (VSS_WS_STABLE)
03-Oct 13:42 dc1.kranj.cetrtapot.si-fd JobId 692: VSS Writer (BackupComplete): 
"Dedup Writer", State: 0x1 (VSS_WS_STABLE)
03-Oct 13:42 dc1.kranj.cetrtapot.si-fd JobId 692: VSS Writer (BackupComplete): 
"Registry Writer", State: 0x1 (VSS_WS_STABLE)
03-Oct 13:42 dc1.kranj.cetrtapot.si-fd JobId 692: VSS Writer (BackupComplete): 
"Shadow Copy Optimization Writer", State: 0x1 (VSS_WS_STABLE)
03-Oct 13:42 dc1.kranj.cetrtapot.si-fd JobId 692: VSS Writer (BackupComplete): "DFS 
Replication service writer", State: 0x1 (VSS_WS_STABLE)
03-Oct 13:42 dc1.kranj.cetrtapot.si-fd JobId 692: VSS Writer (BackupComplete): "NPS 
VSS Writer", State: 0x1 (VSS_WS_STABLE)
03-Oct 13:42 dc1.kranj.cetrtapot.si-fd JobId 692: VSS Writer (BackupComplete): 
"Certificate Authority", State: 0x1 (VSS_WS_STABLE)
03-Oct 13:42 dc1.kranj.cetrtapot.si-fd JobId 692: VSS Writer (BackupComplete): "Dhcp 
Jet Writer", State: 0x1 (VSS_WS_STABLE)
03-Oct 13:42 dc1.kranj.cetrtapot.si-fd JobId 692: VSS Writer (BackupComplete): "COM+ 
REGDB W

Re: [Bacula-users] Backup from client with deduplication

2020-10-07 Thread Martin Simmons

I think you cannot do a Windows backup of just one subdirectory (D:/FS/) from
a deduplicated volume because that doesn't include the deduplication store.
It should work if you add portable=yes, as long as you don't need any
Windows-specific bits.

__Martin


> On Wed, 7 Oct 2020 07:37:18 +0300, 
> =?UTF-8?B?0KDRg9C60LDQstGG0L7QsiDQlNC80LjRgtGA0LjQuSDQk9C10L3QvdCw0LQ=?= 
> ьевич said:
> 
> So, there is no solution? Bacula can't backup such clients?
> 
> > Maybe i explained something wrong. I have bacula server on centos 
> > (server), i don't have any deduplication on the bacula server and i 
> > don't need it there. I have windows 2019 server (client), it has it's 
> > own deduplication on it's own volume which is enabled in windows 
> > settings. From this windows volume i need to backup files.
> >
> > Full backup size before i turn on deduplication (which is windows 
> > deduplication) on windows server (client) was like 200GB, full backup 
> > after i turn on deduplication with same files on the same server was 
> > 6.7GB, files count same, when i restoring it, most files is 
> > unreadable. I'm thinking cause bacula makes snapshot of the files when 
> > making backup and it's not working with deduplication on windows 
> > server (client), but maybe i'm wrong.
> >
> > I just want normal (recoverable) backup from this windows server 
> > (which is 200gb), i don't need any deduplication on my backups.
> >
> >
> > This is FileSet
> >
> > FileSet {
> >     Name = "winsrv"
> >     Enable VSS = no
> >     Include {
> >     Options {
> >
> >         compression = GZIP
> >             signature = MD5
> >     aclsupport = yes
> >    }
> >         File = D:/FS/
> >                   }
> > }
> >
> > this is log from backup:
> >
> > Sep 29 19:20:36 bacula bacula-dir:  Build OS: 
> > x86_64-redhat-linux-gnu-bacula redhat (Core)
> > Sep 29 19:20:36 bacula bacula-dir:  JobId:  3814
> > Sep 29 19:20:36 bacula bacula-dir:  Job: gc-main.2020-09-29_10.58.43_03
> > Sep 29 19:20:36 bacula bacula-dir:  Backup Level:   Full
> > Sep 29 19:20:36 bacula bacula-dir:  Client: "winsrv" 9.6.5 (11Jun20) 
> > Microsoft Standard Edition (build 9200), 64-bit,Cross-compile,Win64
> > Sep 29 19:20:36 bacula bacula-dir:  FileSet: "winsrv" 2020-09-15 22:00:00
> > Sep 29 19:20:36 bacula bacula-dir:  Pool: "main-pool" (From Job resource)
> > Sep 29 19:20:36 bacula bacula-dir:  Catalog: "default" (From Client 
> > resource)
> > Sep 29 19:20:36 bacula bacula-dir:  Storage: "backup-sd" (From Job 
> > resource)
> > Sep 29 19:20:36 bacula bacula-dir:  Scheduled time: 29-сен-2020 19:00:00
> > Sep 29 19:20:36 bacula bacula-dir:  Start time: 29-сен-2020 19:00:02
> > Sep 29 19:20:36 bacula bacula-dir:  End time: 29-сен-2020 19:20:36
> > Sep 29 19:20:36 bacula bacula-dir:  Elapsed time:   20 mins 34 
> > secs
> > Sep 29 19:20:36 bacula bacula-dir:  Priority:   1
> > Sep 29 19:20:36 bacula bacula-dir:  FD Files Written: 905,999
> > Sep 29 19:20:36 bacula bacula-dir:  SD Files Written: 905,999
> > Sep 29 19:20:36 bacula bacula-dir:  FD Bytes Written: 6,500,269,762 
> > (6.500 GB)
> > Sep 29 19:20:36 bacula bacula-dir:  SD Bytes Written: 6,708,284,663 
> > (6.708 GB)
> > Sep 29 19:20:36 bacula bacula-dir:  Rate: 5267.6 KB/s
> > Sep 29 19:20:36 bacula bacula-dir:  Software Compression: 15.7% 1.2:1
> > Sep 29 19:20:36 bacula bacula-dir:  Comm Line Compression:  0.6% 1.0:1
> > Sep 29 19:20:36 bacula bacula-dir:  Snapshot/VSS:   no
> > Sep 29 19:20:36 bacula bacula-dir:  Encryption: no
> > Sep 29 19:20:36 bacula bacula-dir:  Accurate:   no
> > Sep 29 19:20:36 bacula bacula-dir:  Volume name(s): All0121
> > Sep 29 19:20:36 bacula bacula-dir:  Volume Session Id:  1
> > Sep 29 19:20:36 bacula bacula-dir:  Volume Session Time: 1601366166
> > Sep 29 19:20:36 bacula bacula-dir:  Last Volume Bytes: 6,743,816,269 
> > (6.743 GB)
> > Sep 29 19:20:36 bacula bacula-dir:  Non-fatal FD errors:    2
> > Sep 29 19:20:36 bacula bacula-dir:  SD Errors:  0
> > Sep 29 19:20:36 bacula bacula-dir:  FD termination status:  OK
> > Sep 29 19:20:36 bacula bacula-dir:  SD termination status:  OK
> > Sep 29 19:20:36 bacula bacula-dir:  Termination: Backup OK -- with 
> > warnings
> >
> >
> > 30.09.2020 13:22, Radosław Korzeniewski пишет:
> >> Hello,
> >>
> >> wt., 29 wrz 2020 o 09:50 Рукавцов Дмитрий Геннадьевич 
> >> mailto:m...@santel-navi.ru>> napisał(a):
> >>
> >> Hello, i have bacula server 9.6.5 on Centos and i need to backup
> >> folders
> >> from Windows 2019 server client with deduplication on (optimized
> >> files).
> >> Problem if i'm making Full backup, the archive is incomplete
> >>
> >>
> >> Could you show the job terminate log?
> >>
> >> (files
> >> count match, but size is wrong)
> >>
> >>
> >> I do not understand how you judge that a size is wrong?
> >>
> >> Is there any way to tell Bacula server
> >>

Re: [Bacula-users] Backup from client with deduplication

2020-10-07 Thread Žiga Žvan


Hi,
I'm able to backup Windows 2012 server with deduplication enabled. My 
bacula server is also centos, bacula version 9.6.5 (on client and 
server). I have tested only individual file restores but it always 
worked for me. Backup is very small (54 GB for app. 700 GB raw data). 
I'm not sure how bacula manages to backup and encrypt deduplicated 
amount (I would expect that there will be 700 GB on the wire). I'm 
having some performance problems (see thread 
performance&design&configuration challenges).


I'm at the the end of testing bacula sw, so I'm not an experienced user. 
I would also appreciate if someone confirms, that this should work. If 
it helps, there is log from my last backup.

Kind regards,
Ziga



02-Oct 23:05 bacula-dir JobId 692: Start Backup JobId 692, 
Job=dc1-monthly-backup.2020-10-02_23.05.00_09
02-Oct 23:05 bacula-dir JobId 692: Created new Volume="dc1-monthly-vol-0297", 
Pool="dc1-monthly-pool", MediaType="File" in catalog.
02-Oct 23:05 bacula-dir JobId 692: Using Device "FSLocalBackup" to write.
02-Oct 23:05 bacula-sd JobId 692: Labeled new Volume "dc1-monthly-vol-0297" on File 
device "FSLocalBackup" (/storage/bacula/backup).
02-Oct 23:05 bacula-sd JobId 692: Wrote label to prelabeled Volume "dc1-monthly-vol-0297" 
on File device "FSLocalBackup" (/storage/bacula/backup)
02-Oct 23:05 dc1.kranj.cetrtapot.si-fd JobId 692: Generate VSS snapshots. 
Driver="Win64 VSS"
02-Oct 23:05 dc1.kranj.cetrtapot.si-fd JobId 692: Snapshot mount point: D:\
03-Oct 13:42 dc1.kranj.cetrtapot.si-fd JobId 692: VSS Writer (BackupComplete): "Task 
Scheduler Writer", State: 0x1 (VSS_WS_STABLE)
03-Oct 13:42 dc1.kranj.cetrtapot.si-fd JobId 692: VSS Writer (BackupComplete): "VSS 
Metadata Store Writer", State: 0x1 (VSS_WS_STABLE)
03-Oct 13:42 dc1.kranj.cetrtapot.si-fd JobId 692: VSS Writer (BackupComplete): 
"Performance Counters Writer", State: 0x1 (VSS_WS_STABLE)
03-Oct 13:42 dc1.kranj.cetrtapot.si-fd JobId 692: VSS Writer (BackupComplete): 
"System Writer", State: 0x1 (VSS_WS_STABLE)
03-Oct 13:42 dc1.kranj.cetrtapot.si-fd JobId 692: VSS Writer (BackupComplete): "ASR 
Writer", State: 0x1 (VSS_WS_STABLE)
03-Oct 13:42 dc1.kranj.cetrtapot.si-fd JobId 692: VSS Writer (BackupComplete): "IIS 
Config Writer", State: 0x1 (VSS_WS_STABLE)
03-Oct 13:42 dc1.kranj.cetrtapot.si-fd JobId 692: VSS Writer (BackupComplete): "WMI 
Writer", State: 0x1 (VSS_WS_STABLE)
03-Oct 13:42 dc1.kranj.cetrtapot.si-fd JobId 692: VSS Writer (BackupComplete): "IIS 
Metabase Writer", State: 0x1 (VSS_WS_STABLE)
03-Oct 13:42 dc1.kranj.cetrtapot.si-fd JobId 692: VSS Writer (BackupComplete): 
"Dedup Writer", State: 0x1 (VSS_WS_STABLE)
03-Oct 13:42 dc1.kranj.cetrtapot.si-fd JobId 692: VSS Writer (BackupComplete): 
"Registry Writer", State: 0x1 (VSS_WS_STABLE)
03-Oct 13:42 dc1.kranj.cetrtapot.si-fd JobId 692: VSS Writer (BackupComplete): 
"Shadow Copy Optimization Writer", State: 0x1 (VSS_WS_STABLE)
03-Oct 13:42 dc1.kranj.cetrtapot.si-fd JobId 692: VSS Writer (BackupComplete): "DFS 
Replication service writer", State: 0x1 (VSS_WS_STABLE)
03-Oct 13:42 dc1.kranj.cetrtapot.si-fd JobId 692: VSS Writer (BackupComplete): "NPS 
VSS Writer", State: 0x1 (VSS_WS_STABLE)
03-Oct 13:42 dc1.kranj.cetrtapot.si-fd JobId 692: VSS Writer (BackupComplete): 
"Certificate Authority", State: 0x1 (VSS_WS_STABLE)
03-Oct 13:42 dc1.kranj.cetrtapot.si-fd JobId 692: VSS Writer (BackupComplete): "Dhcp 
Jet Writer", State: 0x1 (VSS_WS_STABLE)
03-Oct 13:42 dc1.kranj.cetrtapot.si-fd JobId 692: VSS Writer (BackupComplete): "COM+ 
REGDB Writer", State: 0x1 (VSS_WS_STABLE)
03-Oct 13:42 dc1.kranj.cetrtapot.si-fd JobId 692: VSS Writer (BackupComplete): 
"NTDS", State: 0x1 (VSS_WS_STABLE)
03-Oct 13:42 bacula-sd JobId 692: Elapsed time=14:37:01, Transfer rate=1.081 M 
Bytes/second
03-Oct 13:42 bacula-sd JobId 692: Sending spooled attrs to the Director. 
Despooling 462,280,104 bytes ...
03-Oct 13:42 bacula-dir JobId 692: Bacula bacula-dir 9.6.5 (11Jun20):
  Build OS:   x86_64-redhat-linux-gnu-bacula redhat (Core)
  JobId:  692
  Job:dc1-monthly-backup.2020-10-02_23.05.00_09
  Backup Level:   Full
  Client: "dc1.kranj.cetrtapot.si-fd" 9.6.5 (11Jun20) Microsoft 
Standard Edition (build 9200), 64-bit,Cross-compile,Win64
  FileSet:"dc1-fileset" 2020-09-25 18:20:26
  Pool:   "dc1-monthly-pool" (From Job resource)
  Catalog:"MyCatalog" (From Client resource)
  Storage:"FSLocalBackup" (From Job resource)
  Scheduled time: 02-Oct-2020 23:05:00
  Start time: 02-Oct-2020 23:05:02
  End time:   03-Oct-2020 13:42:34
  Elapsed time:   14 hours 37 mins 32 secs
  Priority:   10
  FD Files Written:   1,706,949
  SD Files Written:   1,706,949
  FD Bytes Written:   55,666,483,955 (55.66 GB)
  SD Bytes Written:   56,910,661,569 (56.91 GB)
  Rate:   1057.3 KB/s
  Software Compression:

Re: [Bacula-users] Backup from client with deduplication

2020-10-06 Thread Radosław Korzeniewski

Hello,

śr., 7 paź 2020 o 06:37 Рукавцов Дмитрий Геннадьевич 
napisał(a):

> So, there is no solution? Bacula can't backup such clients?
>
I know nothing about windows filesystem deduplication capabilities, but I
assume to properly read contents of these files a standard win32
BackupRead() dedicated to backup function is not sufficient and requires
special handling.
I do not know if this special handling is available in Bacula, but as this
is not my area of interests in Bacula my answer could be incomplete.

best regards
-- 
Radosław Korzeniewski
rados...@korzeniewski.net
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

Re: [Bacula-users] Backup from client with deduplication

2020-10-06 Thread Рукавцов Дмитрий Геннадьевич


So, there is no solution? Bacula can't backup such clients?

Maybe i explained something wrong. I have bacula server on centos 
(server), i don't have any deduplication on the bacula server and i 
don't need it there. I have windows 2019 server (client), it has it's 
own deduplication on it's own volume which is enabled in windows 
settings. From this windows volume i need to backup files.


Full backup size before i turn on deduplication (which is windows 
deduplication) on windows server (client) was like 200GB, full backup 
after i turn on deduplication with same files on the same server was 
6.7GB, files count same, when i restoring it, most files is 
unreadable. I'm thinking cause bacula makes snapshot of the files when 
making backup and it's not working with deduplication on windows 
server (client), but maybe i'm wrong.


I just want normal (recoverable) backup from this windows server 
(which is 200gb), i don't need any deduplication on my backups.



This is FileSet

FileSet {
    Name = "winsrv"
    Enable VSS = no
    Include {
    Options {

        compression = GZIP
            signature = MD5
    aclsupport = yes
   }
        File = D:/FS/
                  }
}

this is log from backup:

Sep 29 19:20:36 bacula bacula-dir:  Build OS: 
x86_64-redhat-linux-gnu-bacula redhat (Core)

Sep 29 19:20:36 bacula bacula-dir:  JobId:  3814
Sep 29 19:20:36 bacula bacula-dir:  Job: gc-main.2020-09-29_10.58.43_03
Sep 29 19:20:36 bacula bacula-dir:  Backup Level:   Full
Sep 29 19:20:36 bacula bacula-dir:  Client: "winsrv" 9.6.5 (11Jun20) 
Microsoft Standard Edition (build 9200), 64-bit,Cross-compile,Win64

Sep 29 19:20:36 bacula bacula-dir:  FileSet: "winsrv" 2020-09-15 22:00:00
Sep 29 19:20:36 bacula bacula-dir:  Pool: "main-pool" (From Job resource)
Sep 29 19:20:36 bacula bacula-dir:  Catalog: "default" (From Client 
resource)
Sep 29 19:20:36 bacula bacula-dir:  Storage: "backup-sd" (From Job 
resource)

Sep 29 19:20:36 bacula bacula-dir:  Scheduled time: 29-сен-2020 19:00:00
Sep 29 19:20:36 bacula bacula-dir:  Start time: 29-сен-2020 19:00:02
Sep 29 19:20:36 bacula bacula-dir:  End time: 29-сен-2020 19:20:36
Sep 29 19:20:36 bacula bacula-dir:  Elapsed time:   20 mins 34 
secs

Sep 29 19:20:36 bacula bacula-dir:  Priority:   1
Sep 29 19:20:36 bacula bacula-dir:  FD Files Written: 905,999
Sep 29 19:20:36 bacula bacula-dir:  SD Files Written: 905,999
Sep 29 19:20:36 bacula bacula-dir:  FD Bytes Written: 6,500,269,762 
(6.500 GB)
Sep 29 19:20:36 bacula bacula-dir:  SD Bytes Written: 6,708,284,663 
(6.708 GB)

Sep 29 19:20:36 bacula bacula-dir:  Rate: 5267.6 KB/s
Sep 29 19:20:36 bacula bacula-dir:  Software Compression: 15.7% 1.2:1
Sep 29 19:20:36 bacula bacula-dir:  Comm Line Compression:  0.6% 1.0:1
Sep 29 19:20:36 bacula bacula-dir:  Snapshot/VSS:   no
Sep 29 19:20:36 bacula bacula-dir:  Encryption: no
Sep 29 19:20:36 bacula bacula-dir:  Accurate:   no
Sep 29 19:20:36 bacula bacula-dir:  Volume name(s): All0121
Sep 29 19:20:36 bacula bacula-dir:  Volume Session Id:  1
Sep 29 19:20:36 bacula bacula-dir:  Volume Session Time: 1601366166
Sep 29 19:20:36 bacula bacula-dir:  Last Volume Bytes: 6,743,816,269 
(6.743 GB)

Sep 29 19:20:36 bacula bacula-dir:  Non-fatal FD errors:    2
Sep 29 19:20:36 bacula bacula-dir:  SD Errors:  0
Sep 29 19:20:36 bacula bacula-dir:  FD termination status:  OK
Sep 29 19:20:36 bacula bacula-dir:  SD termination status:  OK
Sep 29 19:20:36 bacula bacula-dir:  Termination: Backup OK -- with 
warnings



30.09.2020 13:22, Radosław Korzeniewski пишет:

Hello,

wt., 29 wrz 2020 o 09:50 Рукавцов Дмитрий Геннадьевич 
mailto:m...@santel-navi.ru>> napisał(a):


Hello, i have bacula server 9.6.5 on Centos and i need to backup
folders
from Windows 2019 server client with deduplication on (optimized
files).
Problem if i'm making Full backup, the archive is incomplete


Could you show the job terminate log?

(files
count match, but size is wrong)


I do not understand how you judge that a size is wrong?

Is there any way to tell Bacula server
to ignore deduplication (optimized files) on client or any other way?


I do not understand your statement. Could you elaborate what you want 
to achieve and what configuration you use?


Deduplication in Bacula Enterprise Edition is called GED (Global 
Endpoint Deduplication) and it is performed on client side (endpoint) 
and on storage daemon. You can enable/disable deduplication for a 
single FileSet/Include resource.
A deduplication optimized volumes functionality (aligned plugin) 
available on Bacula Community requires an external deduplication 
solution, hardware or software level to perform the required 
computation and management. This solution does not involve a backup 
client in any way.



Tried to set "Enable VSS = no" at FileSet but it gives nothing.


VSS is a Microsoft Windo

Re: [Bacula-users] Backup from client with deduplication

2020-09-30 Thread Рукавцов Дмитрий Геннадьевич

Maybe i explained something wrong. I have bacula server on centos 
(server), i don't have any deduplication on the bacula server and i 
don't need it there. I have windows 2019 server (client), it has it's 
own deduplication on it's own volume which is enabled in windows 
settings. From this windows volume i need to backup files.


Full backup size before i turn on deduplication (which is windows 
deduplication) on windows server (client) was like 200GB, full backup 
after i turn on deduplication with same files on the same server was 
6.7GB, files count same, when i restoring it, most files is unreadable. 
I'm thinking cause bacula makes snapshot of the files when making backup 
and it's not working with deduplication on windows server (client), but 
maybe i'm wrong.


I just want normal (recoverable) backup from this windows server (which 
is 200gb), i don't need any deduplication on my backups.



This is FileSet

FileSet {
    Name = "winsrv"
    Enable VSS = no
    Include {
    Options {

        compression = GZIP
            signature = MD5
    aclsupport = yes
   }
        File = D:/FS/
                  }
}

this is log from backup:

Sep 29 19:20:36 bacula bacula-dir:  Build OS: 
x86_64-redhat-linux-gnu-bacula redhat (Core)

Sep 29 19:20:36 bacula bacula-dir:  JobId:  3814
Sep 29 19:20:36 bacula bacula-dir:  Job: gc-main.2020-09-29_10.58.43_03
Sep 29 19:20:36 bacula bacula-dir:  Backup Level:   Full
Sep 29 19:20:36 bacula bacula-dir:  Client: "winsrv" 9.6.5 (11Jun20) 
Microsoft Standard Edition (build 9200), 64-bit,Cross-compile,Win64

Sep 29 19:20:36 bacula bacula-dir:  FileSet: "winsrv" 2020-09-15 22:00:00
Sep 29 19:20:36 bacula bacula-dir:  Pool: "main-pool" (From Job resource)
Sep 29 19:20:36 bacula bacula-dir:  Catalog: "default" (From Client 
resource)

Sep 29 19:20:36 bacula bacula-dir:  Storage: "backup-sd" (From Job resource)
Sep 29 19:20:36 bacula bacula-dir:  Scheduled time: 29-сен-2020 19:00:00
Sep 29 19:20:36 bacula bacula-dir:  Start time: 29-сен-2020 19:00:02
Sep 29 19:20:36 bacula bacula-dir:  End time: 29-сен-2020 19:20:36
Sep 29 19:20:36 bacula bacula-dir:  Elapsed time:   20 mins 34 secs
Sep 29 19:20:36 bacula bacula-dir:  Priority:   1
Sep 29 19:20:36 bacula bacula-dir:  FD Files Written: 905,999
Sep 29 19:20:36 bacula bacula-dir:  SD Files Written: 905,999
Sep 29 19:20:36 bacula bacula-dir:  FD Bytes Written: 6,500,269,762 
(6.500 GB)
Sep 29 19:20:36 bacula bacula-dir:  SD Bytes Written: 6,708,284,663 
(6.708 GB)

Sep 29 19:20:36 bacula bacula-dir:  Rate:   5267.6 KB/s
Sep 29 19:20:36 bacula bacula-dir:  Software Compression:   15.7% 1.2:1
Sep 29 19:20:36 bacula bacula-dir:  Comm Line Compression:  0.6% 1.0:1
Sep 29 19:20:36 bacula bacula-dir:  Snapshot/VSS:   no
Sep 29 19:20:36 bacula bacula-dir:  Encryption: no
Sep 29 19:20:36 bacula bacula-dir:  Accurate:   no
Sep 29 19:20:36 bacula bacula-dir:  Volume name(s): All0121
Sep 29 19:20:36 bacula bacula-dir:  Volume Session Id:  1
Sep 29 19:20:36 bacula bacula-dir:  Volume Session Time: 1601366166
Sep 29 19:20:36 bacula bacula-dir:  Last Volume Bytes: 6,743,816,269 
(6.743 GB)

Sep 29 19:20:36 bacula bacula-dir:  Non-fatal FD errors:    2
Sep 29 19:20:36 bacula bacula-dir:  SD Errors:  0
Sep 29 19:20:36 bacula bacula-dir:  FD termination status:  OK
Sep 29 19:20:36 bacula bacula-dir:  SD termination status:  OK
Sep 29 19:20:36 bacula bacula-dir:  Termination:    Backup OK -- 
with warnings



30.09.2020 13:22, Radosław Korzeniewski пишет:

Hello,

wt., 29 wrz 2020 o 09:50 Рукавцов Дмитрий Геннадьевич 
mailto:m...@santel-navi.ru>> napisał(a):


Hello, i have bacula server 9.6.5 on Centos and i need to backup
folders
from Windows 2019 server client with deduplication on (optimized
files).
Problem if i'm making Full backup, the archive is incomplete


Could you show the job terminate log?

(files
count match, but size is wrong)


I do not understand how you judge that a size is wrong?

Is there any way to tell Bacula server
to ignore deduplication (optimized files) on client or any other way?


I do not understand your statement. Could you elaborate what you want 
to achieve and what configuration you use?


Deduplication in Bacula Enterprise Edition is called GED (Global 
Endpoint Deduplication) and it is performed on client side (endpoint) 
and on storage daemon. You can enable/disable deduplication for a 
single FileSet/Include resource.
A deduplication optimized volumes functionality (aligned plugin) 
available on Bacula Community requires an external deduplication 
solution, hardware or software level to perform the required 
computation and management. This solution does not involve a backup 
client in any way.



Tried to set "Enable VSS = no" at FileSet but it gives nothing.


VSS is a Microsoft Windows framework and API to manage fil

Re: [Bacula-users] Backup from client with deduplication

2020-09-30 Thread Radosław Korzeniewski

Hello,

wt., 29 wrz 2020 o 09:50 Рукавцов Дмитрий Геннадьевич 
napisał(a):

> Hello, i have bacula server 9.6.5 on Centos and i need to backup folders
> from Windows 2019 server client with deduplication on (optimized files).
> Problem if i'm making Full backup, the archive is incomplete


Could you show the job terminate log?


> (files
> count match, but size is wrong)


I do not understand how you judge that a size is wrong?


> Is there any way to tell Bacula server
> to ignore deduplication (optimized files) on client or any other way?


I do not understand your statement. Could you elaborate what you want to
achieve and what configuration you use?

Deduplication in Bacula Enterprise Edition is called GED (Global Endpoint
Deduplication) and it is performed on client side (endpoint) and on storage
daemon. You can enable/disable deduplication for a single FileSet/Include
resource.
A deduplication optimized volumes functionality (aligned plugin) available
on Bacula Community requires an external deduplication solution, hardware
or software level to perform the required computation and management. This
solution does not involve a backup client in any way.


>
> Tried to set "Enable VSS = no" at FileSet but it gives nothing.


VSS is a Microsoft Windows framework and API to manage filesystem level
storage snapshots. It is ignored on non Windows systems and has nothing to
do with deduplication.


> I read
> manuals but there is only info about how enable deduplication on server,
> not how to backup from such client.
>

When you enable a deduplication optimized volumes functionality (aligned
plugin) then deduplication could be performed externally using such
solutions like: EMC Data Domain, Quantum DXi or ZFS. No build in
deduplication is available in Bacula Community and aligned plugin. If you
want to enable client side (endpoint) deduplication then you have to use
Bacula Enterprise Edition.


> I need to make full backup to restore at another machine where i don't
> have deduplication, so backup chunk store container and optimized files
> separately is not an option.


Bacula is a network backup and restore solution. You do not need to "copy"
saved archives to another machine to do a restore. All you need is to
execute a restore job which will use an existing backup archive stored on a
backup server and not a client.


> I need to turn off
> FILE_FLAG_OPEN_REPARSE_POINT so i could make normal backup without using
> VSS but i don't understand why option "Enable VSS = no" not working
>

Bacula should handle reparse points very well. I'm not sure if it supports
all kinds of reparse points available. I think reparse points are not
related to VSS in any way, so switching VSS on or off does not help.
I do not understand what you want to achieve with this flag, could you
elaborate?

best regards
-- 
Radosław Korzeniewski
rados...@korzeniewski.net
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

[Bacula-users] Backup from client with deduplication

2020-09-29 Thread Рукавцов Дмитрий Геннадьевич

Hello, i have bacula server 9.6.5 on Centos and i need to backup folders 
from Windows 2019 server client with deduplication on (optimized files). 
Problem if i'm making Full backup, the archive is incomplete (files 
count match, but size is wrong) Is there any way to tell Bacula server 
to ignore deduplication (optimized files) on client or any other way? 
Tried to set "Enable VSS = no" at FileSet but it gives nothing. I read 
manuals but there is only info about how enable deduplication on server, 
not how to backup from such client.


I need to make full backup to restore at another machine where i don't 
have deduplication, so backup chunk store container and optimized files 
separately is not an option. I need to turn off 
FILE_FLAG_OPEN_REPARSE_POINT so i could make normal backup without using 
VSS but i don't understand why option "Enable VSS = no" not working




___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

Re: [Bacula-users] Backup from client with deduplication

Re: [Bacula-users] Backup from client with deduplication

Re: [Bacula-users] Backup from client with deduplication

Re: [Bacula-users] Backup from client with deduplication

Re: [Bacula-users] Backup from client with deduplication

Re: [Bacula-users] Backup from client with deduplication

Re: [Bacula-users] Backup from client with deduplication

Re: [Bacula-users] Backup from client with deduplication

Re: [Bacula-users] Backup from client with deduplication

[Bacula-users] Backup from client with deduplication

10 matches

Site Navigation

Mail list logo

Footer information