On 2022-05-16 3:36 pm, kjohn...@eclypse.org wrote:
'Radoslaw Korzeniewski' suggested that I could take the Comm Line
Compression report at face value; I appreciate this interest in my
problem.
Based on my 'feel' for the system usage, I thought there was a real
issue.
But today I collected numeric data.
Today I used "find <various paths> -mmin -$((60*24*7)) -printf '%s\n' |
awk
'{a+=$1;} END {printf "%.1f GB\n", a/2**30;}'"
on the various paths which are backed up by the nightly (5 per week)
backup
job to calculate the total file size that has changed since the last
backup
with the expected (%10% range) Comm Line Compression. I explicitly
excluded
a set of .zst and .gz files in /tmp/daily on the assumption that the
data
from those files would not be significantly compressed further by
Bacula.
All numbers in GB.
/home 3.2
/etc 0
/opt 0
/root 0
/usr/local 0
/var/lib 12.8
/var/log 0.2
/var/mail 1.8
/tmp/daily 0.1
Total 18.1GB
This is less than 2.5% of the total backup data. Even if all of this
data
had been compressed to zero bytes, it would not account for the change
in
compression ratio shown in the two reports below.
I believe that something is not right with the Comm Line Compression
report.
Whether it is the 6 reports in the ~10% range, or the (now) 4 reports
in the
~60% range, I cannot say.
Some may ask: if a fairly small part of the data changes each day, why
a
full backup each night? From my perspective backup is about the
restores,
and only needing a single tape cartridge with two jobs (the backup, the
catalog) on it is a significant simplification, in my view. YMMV.
Ken
-----Original Message-----
From: kjohn...@eclypse.org [mailto:kjohn...@eclypse.org]
Sent: Thursday, May 12, 2022 4:25 PM
To: 'bacula-users'
Subject: [Bacula-users] Comm Line Compression report
I use bacula to perform 5 backups per week of a server here running
Debian
10, and version 9.4.2-2+deb10u1 of bacula. Each backup is of the same
set
of data, so only the most recent tape is needed for a restore. Each
backup
is to an LT06 tape installed on the server, and about 800 GB is written
each
time.
Last week, and for the first backup this week, the reported Comm Line
Compression was in the 10-13% range.
The second and third backups this week had reported Comm Line
Compression in
the 60-70% range.
I do not believe there has been a significant change in the nature of
the
data that is backed up.
I would be grateful for any insight into what might be going on.
Ken
Minimal Comm Line Compression excerpt:
Elapsed time: 3 hours 12 mins 3 secs
Priority: 12
FD Files Written: 820,234
SD Files Written: 820,234
FD Bytes Written: 815,654,365,473 (815.6 GB)
SD Bytes Written: 815,806,065,214 (815.8 GB)
Rate: 70784.9 KB/s
Software Compression: None
Comm Line Compression: 10.8% 1.1:1
Snapshot/VSS: no
Encryption: no
Accurate: no
Volume name(s): Daily02
Volume Session Id: 52
Volume Session Time: 1648953913
Last Volume Bytes: 816,436,518,912 (816.4 GB)
Non-fatal FD errors: 0
SD Errors: 0
FD termination status: OK
SD termination status: OK
Termination: Backup OK
Surprising Comm Line Compression excerpt:
Elapsed time: 3 hours 12 mins 45 secs
Priority: 12
FD Files Written: 828,854
SD Files Written: 828,854
FD Bytes Written: 821,957,854,696 (821.9 GB)
SD Bytes Written: 822,111,208,734 (822.1 GB)
Rate: 71072.9 KB/s
Software Compression: None
Comm Line Compression: 61.6% 2.6:1
Snapshot/VSS: no
Encryption: no
Accurate: no
Volume name(s): Daily03
Volume Session Id: 56
Volume Session Time: 1648953913
Last Volume Bytes: 822,746,631,168 (822.7 GB)
Non-fatal FD errors: 0
SD Errors: 0
FD termination status: OK
SD termination status: OK
Termination: Backup OK
Com Line Compression, is the compression of the data transmitted to the
server about the backup, the file list for example, its not the data
that was backed up. Look at this from my Windows Workstation that ran a
full last night, the com line compression was 92%. The Software
compression was only 3.9%. That is the data compression, there is a
large chuck of this backup that is an iTunes library of both Music and
Movies which are already compressed. That makes the full backups
compression low, but the incremental backups throughout the week have
fairly high compression as the data that changes regularly usually does
compress well.
You must have a "Compression = " set under the options of the fileset in
order to get the software compression. I use the lzo option. I think the
com line compression is a default.
Build OS: amd64-portbld-freebsd13.1 freebsd 13.1-RC6
JobId: 20371
Job: Workstation-Backup.2022-05-16_22.05.00_07
Backup Level: Full
Client: "workstation-fd" 11.0.5 (03Jun21) Microsoft
Windows 8 Professional (build 9200), 64-bit,Cross-compile,Win64
FileSet: "Workstation-FileSet" 2013-09-27 13:12:01
Pool: "File" (From Run Pool override)
Catalog: "MyCatalog" (From Client resource)
Storage: "File" (From Pool resource)
Scheduled time: 16-May-2022 22:05:00
Start time: 16-May-2022 22:05:02
End time: 17-May-2022 03:48:35
Elapsed time: 5 hours 43 mins 33 secs
Priority: 15
FD Files Written: 1,061,603
SD Files Written: 1,061,603
FD Bytes Written: 1,269,611,875,453 (1.269 TB)
SD Bytes Written: 1,269,856,491,365 (1.269 TB)
Rate: 61592.8 KB/s
Software Compression: 3.9% 1.0:1
Comm Line Compression: 92.0% 12.5:1
Snapshot/VSS: yes
Encryption: no
Accurate: yes
Volume name(s):
BV115|BV116|BV117|BV118|BV119|BV120|BV121|BV122|BV123|BV124|BV125|BV126|BV131|BV127|BV100|BV101|BV102|BV103|BV104|BV105|BV106|BV107|BV108|BV109|BV110|BV111
Volume Session Id: 3
Volume Session Time: 1652711846
Last Volume Bytes: 41,041,537,114 (41.04 GB)
Non-fatal FD errors: 0
SD Errors: 0
FD termination status: OK
SD termination status: OK
Termination: Backup OK
--
Thanks,
Dean E. Weimer
http://www.dweimer.net/
_______________________________________________
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users
