Re: [Bacula-users] JobBytes
Hi Martin, I've just checked and data compression is enabled as expected: tapeinfo -f /dev/nst0 Product Type: Tape Drive Vendor ID: 'QUANTUM ' Product ID: 'ULTRIUM 4 ' Revision: '2210' Attached Changer API: No SerialNumber: 'PW1007AMJ50074' MinBlock: 1 MaxBlock: 16777215 SCSI ID: 0 SCSI LUN: 0 Ready: yes BufferedMode: yes Medium Type: Not Loaded Density Code: 0x46 BlockSize: 0 DataCompEnabled: yes DataCompCapable: yes DataDeCompEnabled: yes CompType: 0x1 DeCompType: 0x1 Block Position: 0 Partition 0 Remaining Kbytes: 172811 Partition 0 Size in Kbytes: 853060 ActivePartition: 0 EarlyWarningSize: 0 It required a tape to be inserted indeed. Thank you for all your help :) Adam On 14/02/18 16:00, Martin Simmons wrote: Ah, OK (mt shows compression on FreeBSD). You could try tapeinfo (part of the mtx package). I think you need to have a tape loaded to make it work. __Martin On Wed, 14 Feb 2018 14:47:59 +, Adam Weremczuk said: Hi Martin, This is what I'm getting from the command: mt -f /dev/nst0 status SCSI 2 tape drive: File number=0, block number=0, partition=0. Tape block size 0 bytes. Density code 0x46 (LTO-4). Soft error count since last status=0 General status bits on (4101): BOT ONLINE IM_REP_EN So compression is not mentioned here, just density code which is something different. I looked into "mt compression" and "mt datacompression" commands but both don't seem to have a view or status mode. Even when executed without a parameter they enable compression: https://wiki.zmanda.com/index.php/Hardware_compression So I'm reluctant to take the risk with our main backup system which I'm still learning how to use. Any ideas? Thanks Adam On 09/02/18 18:32, Martin Simmons wrote: Possibly "mt status" will show whether hardware compression is enabled? If you are getting close to 1.6TB per LTO-4 tape (according to JobBytes) then I think hardware compression must be enabled. The mt command also allows you to control compression (I'm not sure if you can change it in the middle of writing to a tape though). -- Check out the vibrant tech community on one of the world's most engaging tech sites, Slashdot.org! http://sdm.link/slashdot ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] JobBytes
Hi Martin, This is what I'm getting from the command: mt -f /dev/nst0 status SCSI 2 tape drive: File number=0, block number=0, partition=0. Tape block size 0 bytes. Density code 0x46 (LTO-4). Soft error count since last status=0 General status bits on (4101): BOT ONLINE IM_REP_EN So compression is not mentioned here, just density code which is something different. I looked into "mt compression" and "mt datacompression" commands but both don't seem to have a view or status mode. Even when executed without a parameter they enable compression: https://wiki.zmanda.com/index.php/Hardware_compression So I'm reluctant to take the risk with our main backup system which I'm still learning how to use. Any ideas? Thanks Adam On 09/02/18 18:32, Martin Simmons wrote: Possibly "mt status" will show whether hardware compression is enabled? If you are getting close to 1.6TB per LTO-4 tape (according to JobBytes) then I think hardware compression must be enabled. The mt command also allows you to control compression (I'm not sure if you can change it in the middle of writing to a tape though). -- Check out the vibrant tech community on one of the world's most engaging tech sites, Slashdot.org! http://sdm.link/slashdot ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] JobBytes
Ah, OK (mt shows compression on FreeBSD). You could try tapeinfo (part of the mtx package). I think you need to have a tape loaded to make it work. __Martin > On Wed, 14 Feb 2018 14:47:59 +, Adam Weremczuk said: > > Hi Martin, > > This is what I'm getting from the command: > > mt -f /dev/nst0 status > SCSI 2 tape drive: > File number=0, block number=0, partition=0. > Tape block size 0 bytes. Density code 0x46 (LTO-4). > Soft error count since last status=0 > General status bits on (4101): > BOT ONLINE IM_REP_EN > > So compression is not mentioned here, just density code which is > something different. > > I looked into "mt compression" and "mt datacompression" commands but > both don't seem to have a view or status mode. > > Even when executed without a parameter they enable compression: > > https://wiki.zmanda.com/index.php/Hardware_compression > > So I'm reluctant to take the risk with our main backup system which I'm > still learning how to use. > > Any ideas? > > Thanks > Adam > > > On 09/02/18 18:32, Martin Simmons wrote: > > Possibly "mt status" will show whether hardware compression is enabled? > > > > If you are getting close to 1.6TB per LTO-4 tape (according to JobBytes) > > then > > I think hardware compression must be enabled. > > > > The mt command also allows you to control compression (I'm not sure if you > > can > > change it in the middle of writing to a tape though). > > -- Check out the vibrant tech community on one of the world's most engaging tech sites, Slashdot.org! http://sdm.link/slashdot ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] JobBytes
Possibly "mt status" will show whether hardware compression is enabled? If you are getting close to 1.6TB per LTO-4 tape (according to JobBytes) then I think hardware compression must be enabled. The mt command also allows you to control compression (I'm not sure if you can change it in the middle of writing to a tape though). If you have hardware compression enabled, then using software compression will just waste time (assuming you have a fast enough network). __Martin > On Thu, 8 Feb 2018 09:51:55 +, Adam Weremczuk said: > > Thank you Martin. > > We are using a pretty old Bacula 5.2.6. > This version already appears to support compressions on clients. > But it's currently not defined for any. > > I'm assuming hardware compression is enabled on the LTO-4 tape drive. > I've checked: > - Device and Storage directives, > - "status storage" from bconsole, > - mtab, > - any place mentioning /dev/nts0 > but couldn't find any compression related options. > > Without it we would only be able to write 800GB to each tape and we > write close to 1.6TB, right? > Can we switch between these two modes? > Also - does it make sense to use more than one compression (i.e. > software on clients and hardware on tape drive) concurrently? > With the current settings does it make sense to track all big text files > on clients and compress them before a backup run? > > Cheers > Adam > > On 07/02/18 19:43, Martin Simmons wrote: > > JobBytes is the number of bytes sent from the FD (client) to the SD. If you > > are using Bacula's software compression (the compression option in the > > Fileset), then that will be the size after compression. > > > > Bacula's software compression is always done in the FD. > > > > You can control concurrency of backups using the various "Maximum Concurrent > > Jobs" options. > > > > __Martin > > > > > >> On Wed, 7 Feb 2018 12:02:56 +, Adam Weremczuk said: > >> Hi all, > >> > >> Last night a tape filled so I'm spending most of today investigating the > >> cause and ensuring the next run completes fine. > >> When executing "list jobname=" from the console I'm presented with 2 > >> columns: JobFiles and JobBytes. > >> > >> Is JobBytes expected to show the volume read from the client or written > >> to the tape? > >> In the latter the count is after compression, right? > >> > >> On that note: Is compression always performed on the server or can it be > >> done on clients? > >> It could make sense to iterate over all clients first and instruct them > >> to start compressing. > >> Then they start reporting as ready to the director and data transfers > >> begin. > >> > >> Please advise. > >> > >> Regards > >> Adam > >> > >> > >> -- > >> Check out the vibrant tech community on one of the world's most > >> engaging tech sites, Slashdot.org! http://sdm.link/slashdot > >> ___ > >> Bacula-users mailing list > >> Bacula-users@lists.sourceforge.net > >> https://lists.sourceforge.net/lists/listinfo/bacula-users > >> > -- Check out the vibrant tech community on one of the world's most engaging tech sites, Slashdot.org! http://sdm.link/slashdot ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] JobBytes
Thank you Martin. We are using a pretty old Bacula 5.2.6. This version already appears to support compressions on clients. But it's currently not defined for any. I'm assuming hardware compression is enabled on the LTO-4 tape drive. I've checked: - Device and Storage directives, - "status storage" from bconsole, - mtab, - any place mentioning /dev/nts0 but couldn't find any compression related options. Without it we would only be able to write 800GB to each tape and we write close to 1.6TB, right? Can we switch between these two modes? Also - does it make sense to use more than one compression (i.e. software on clients and hardware on tape drive) concurrently? With the current settings does it make sense to track all big text files on clients and compress them before a backup run? Cheers Adam On 07/02/18 19:43, Martin Simmons wrote: JobBytes is the number of bytes sent from the FD (client) to the SD. If you are using Bacula's software compression (the compression option in the Fileset), then that will be the size after compression. Bacula's software compression is always done in the FD. You can control concurrency of backups using the various "Maximum Concurrent Jobs" options. __Martin On Wed, 7 Feb 2018 12:02:56 +, Adam Weremczuk said: Hi all, Last night a tape filled so I'm spending most of today investigating the cause and ensuring the next run completes fine. When executing "list jobname=" from the console I'm presented with 2 columns: JobFiles and JobBytes. Is JobBytes expected to show the volume read from the client or written to the tape? In the latter the count is after compression, right? On that note: Is compression always performed on the server or can it be done on clients? It could make sense to iterate over all clients first and instruct them to start compressing. Then they start reporting as ready to the director and data transfers begin. Please advise. Regards Adam -- Check out the vibrant tech community on one of the world's most engaging tech sites, Slashdot.org! http://sdm.link/slashdot ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users -- Check out the vibrant tech community on one of the world's most engaging tech sites, Slashdot.org! http://sdm.link/slashdot ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] JobBytes
JobBytes is the number of bytes sent from the FD (client) to the SD. If you are using Bacula's software compression (the compression option in the Fileset), then that will be the size after compression. Bacula's software compression is always done in the FD. You can control concurrency of backups using the various "Maximum Concurrent Jobs" options. __Martin > On Wed, 7 Feb 2018 12:02:56 +, Adam Weremczuk said: > > Hi all, > > Last night a tape filled so I'm spending most of today investigating the > cause and ensuring the next run completes fine. > When executing "list jobname=" from the console I'm presented with 2 > columns: JobFiles and JobBytes. > > Is JobBytes expected to show the volume read from the client or written > to the tape? > In the latter the count is after compression, right? > > On that note: Is compression always performed on the server or can it be > done on clients? > It could make sense to iterate over all clients first and instruct them > to start compressing. > Then they start reporting as ready to the director and data transfers begin. > > Please advise. > > Regards > Adam > > > -- > Check out the vibrant tech community on one of the world's most > engaging tech sites, Slashdot.org! http://sdm.link/slashdot > ___ > Bacula-users mailing list > Bacula-users@lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/bacula-users > -- Check out the vibrant tech community on one of the world's most engaging tech sites, Slashdot.org! http://sdm.link/slashdot ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
[Bacula-users] JobBytes
Hi all, Last night a tape filled so I'm spending most of today investigating the cause and ensuring the next run completes fine. When executing "list jobname=" from the console I'm presented with 2 columns: JobFiles and JobBytes. Is JobBytes expected to show the volume read from the client or written to the tape? In the latter the count is after compression, right? On that note: Is compression always performed on the server or can it be done on clients? It could make sense to iterate over all clients first and instruct them to start compressing. Then they start reporting as ready to the director and data transfers begin. Please advise. Regards Adam -- Check out the vibrant tech community on one of the world's most engaging tech sites, Slashdot.org! http://sdm.link/slashdot ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
[Bacula-users] JobBytes
Hello, Is the value of JobBytes the number of compressed or un-compressed bytes backed up? Also, what is ReadBytes? I can't seem to find a reference in the manual. If it's there, could someone point me to the page in the 7.0.5 pdf manual. Warmest regards, -craig -- ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
