Re: [Bacula-users] Fwd: Spooling attrs takes forever
On 11/20/2013 12:29 AM, lst_ho...@kwsoft.de wrote: > Zitat von Charles Douglass : > >> On 11/13/2013 09:01 AM, Phil Stracchino wrote: >>> On 11/13/13 10:59, Charles Douglass wrote: Thanks for the reply. I know nothing about MySQL optimization. I downloaded the "mysqltuner.pl" script and I will run it after the next full backup to see if it has any recommendations. >>> The mysqltuner script is a pretty decent basic tool. The thing you need >>> to keep in mind regarding the "mysql-large", "mysql-huge" sample >>> configurations that are STILL widely distributed in MySQL packages (and >>> still widely used by the unsuspecting) is that most of those sample >>> configurations were originally written back when a "large" server was >>> one that might have as much as a whole 32MB of RAM. These days, that is >>> less than the compiled-in default size of some individual MySQL >>> *buffers*. So the unwary install the "suggested" configurations, and >>> can't understand why MySQL is performing like a geriatric tortoise. >>> >>> >> I ran mysqltuner before the backup and increased a couple of buffer >> sizes as well as taking this suggestion: >> >> innodb_flush_log_at_trx_commit=0 >> >> >> Then I ran a full backup and see this: >> >> 18-Nov 12:45 maple-sd JobId 46: Sending spooled attrs to the >> Director. Despooling 153,470,245 bytes ... >> 18-Nov 21:41 maple-dir JobId 46: Bacula maple-dir 5.2.6 (21Feb12): >> Build OS: x86_64-pc-linux-gnu ubuntu 13.04 >> >> So spooling/despooling took just about 9 hours. >> > Even for MySQL on a not too fast machine this is ridiculus slow. We > despool around 1.2GB attributes within around 2 minutes to our > Postgres server. Something is still wrong with your setup. Have you > checked with tools like iotop,top,vmstat what the server is actually > doing during the 9 hours. > > Regards > > Andreas > iostat during the attrs despooling shows: 334 be/3 root0.00 B/s0.00 B/s 0.00 % 99.99 % [jbd2/sda1-8] 24997 be/4 mysql 0.00 B/s 64.98 K/s 0.00 % 0.12 % mysqld Doing some research this appears to be a problem between MySQL and ext4 with file system journaling (jbd2/sda1-8). Thanks, everyone, for helping me get this far. My next step will be to create an ext3 partition and move all my MySQL Dbs there. Chas Douglass -- Shape the Mobile Experience: Free Subscription Software experts and developers: Be at the forefront of tech innovation. Intel(R) Software Adrenaline delivers strategic insight and game-changing conversations that shape the rapidly evolving mobile landscape. Sign up now. http://pubads.g.doubleclick.net/gampad/clk?id=63431311&iu=/4140/ostg.clktrk ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Fwd: Spooling attrs takes forever
On 11/20/2013 3:29 AM, lst_ho...@kwsoft.de wrote: > Zitat von Charles Douglass : > >> I ran mysqltuner before the backup and increased a couple of buffer >> sizes as well as taking this suggestion: >> innodb_flush_log_at_trx_commit=0 Then I ran a full backup and see >> this: 18-Nov 12:45 maple-sd JobId 46: Sending spooled attrs to the >> Director. Despooling 153,470,245 bytes ... 18-Nov 21:41 maple-dir >> JobId 46: Bacula maple-dir 5.2.6 (21Feb12): Build OS: >> x86_64-pc-linux-gnu ubuntu 13.04 So spooling/despooling took just >> about 9 hours. > Even for MySQL on a not too fast machine this is ridiculus slow. We > despool around 1.2GB attributes within around 2 minutes to our > Postgres server. Something is still wrong with your setup. Have you > checked with tools like iotop,top,vmstat what the server is actually > doing during the 9 hours. Yes. Something is wrong. Is there a script configured to run after the job? Is there a broken disk drive? Is the machine low on RAM and using swap space constantly? Are other processes slow on this machine while the despooling is in progress? Is there a messed up or missing index? Have you tried rebuilding the database from a dump file? -- Shape the Mobile Experience: Free Subscription Software experts and developers: Be at the forefront of tech innovation. Intel(R) Software Adrenaline delivers strategic insight and game-changing conversations that shape the rapidly evolving mobile landscape. Sign up now. http://pubads.g.doubleclick.net/gampad/clk?id=63431311&iu=/4140/ostg.clktrk ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Fwd: Spooling attrs takes forever
Zitat von Charles Douglass : > On 11/13/2013 09:01 AM, Phil Stracchino wrote: >> On 11/13/13 10:59, Charles Douglass wrote: >>> Thanks for the reply. >>> >>> I know nothing about MySQL optimization. I downloaded the >>> "mysqltuner.pl" script and I will run it after the next full backup to >>> see if it has any recommendations. >> The mysqltuner script is a pretty decent basic tool. The thing you need >> to keep in mind regarding the "mysql-large", "mysql-huge" sample >> configurations that are STILL widely distributed in MySQL packages (and >> still widely used by the unsuspecting) is that most of those sample >> configurations were originally written back when a "large" server was >> one that might have as much as a whole 32MB of RAM. These days, that is >> less than the compiled-in default size of some individual MySQL >> *buffers*. So the unwary install the "suggested" configurations, and >> can't understand why MySQL is performing like a geriatric tortoise. >> >> > I ran mysqltuner before the backup and increased a couple of buffer > sizes as well as taking this suggestion: > > innodb_flush_log_at_trx_commit=0 > > > Then I ran a full backup and see this: > > 18-Nov 12:45 maple-sd JobId 46: Sending spooled attrs to the > Director. Despooling 153,470,245 bytes ... > 18-Nov 21:41 maple-dir JobId 46: Bacula maple-dir 5.2.6 (21Feb12): > Build OS: x86_64-pc-linux-gnu ubuntu 13.04 > > So spooling/despooling took just about 9 hours. > Even for MySQL on a not too fast machine this is ridiculus slow. We despool around 1.2GB attributes within around 2 minutes to our Postgres server. Something is still wrong with your setup. Have you checked with tools like iotop,top,vmstat what the server is actually doing during the 9 hours. Regards Andreas -- Shape the Mobile Experience: Free Subscription Software experts and developers: Be at the forefront of tech innovation. Intel(R) Software Adrenaline delivers strategic insight and game-changing conversations that shape the rapidly evolving mobile landscape. Sign up now. http://pubads.g.doubleclick.net/gampad/clk?id=63431311&iu=/4140/ostg.clktrk ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Fwd: Spooling attrs takes forever
On 11/13/2013 09:01 AM, Phil Stracchino wrote: > On 11/13/13 10:59, Charles Douglass wrote: >> Thanks for the reply. >> >> I know nothing about MySQL optimization. I downloaded the >> "mysqltuner.pl" script and I will run it after the next full backup to >> see if it has any recommendations. > The mysqltuner script is a pretty decent basic tool. The thing you need > to keep in mind regarding the "mysql-large", "mysql-huge" sample > configurations that are STILL widely distributed in MySQL packages (and > still widely used by the unsuspecting) is that most of those sample > configurations were originally written back when a "large" server was > one that might have as much as a whole 32MB of RAM. These days, that is > less than the compiled-in default size of some individual MySQL > *buffers*. So the unwary install the "suggested" configurations, and > can't understand why MySQL is performing like a geriatric tortoise. > > I ran mysqltuner before the backup and increased a couple of buffer sizes as well as taking this suggestion: innodb_flush_log_at_trx_commit=0 Then I ran a full backup and see this: 18-Nov 12:45 maple-sd JobId 46: Sending spooled attrs to the Director. Despooling 153,470,245 bytes ... 18-Nov 21:41 maple-dir JobId 46: Bacula maple-dir 5.2.6 (21Feb12): Build OS: x86_64-pc-linux-gnu ubuntu 13.04 So spooling/despooling took just about 9 hours. And running mysqltuner after shows: Performance Metrics - [--] Up for: 2d 0h 15m 14s (765K q [4.407 qps], 203 conn, TX: 16M, RX: 158M) [--] Reads / Writes: 1% / 99% [--] Total buffers: 560.0M global + 2.7M per thread (100 max threads) [OK] Maximum possible memory usage: 828.8M (13% of installed RAM) [OK] Slow queries: 0% (3/765K) [OK] Highest usage of available connections: 4% (4/100) [OK] Key buffer size / total MyISAM indexes: 16.0M/6.0M [OK] Key buffer hit rate: 100.0% (5M cached / 3 reads) [OK] Query cache efficiency: 47.9% (11K cached / 23K selects) [OK] Query cache prunes per day: 0 [OK] Sorts requiring temporary tables: 0% (0 temp sorts / 32 sorts) [OK] Temporary tables created on disk: 17% (175 on disk / 976 total) [OK] Thread cache hit rate: 98% (4 created / 203 connections) [OK] Table cache hit rate: 24% (443 open / 1K opened) [OK] Open file limit used: 23% (261/1K) [OK] Table locks acquired immediately: 100% (742K immediate / 742K locks) [OK] InnoDB data size / buffer pool: 246.3M/256.0M Recommendations - General recommendations: Run OPTIMIZE TABLE to defragment tables for better performance I believe this only refers to other tables, since I just previously deleted and recreated the bacula database. There was one other suggestion to make the spool file on a different file system, but does this apply to me since I'm writing to tape? Thanks for any further help. Chas Douglass -- Shape the Mobile Experience: Free Subscription Software experts and developers: Be at the forefront of tech innovation. Intel(R) Software Adrenaline delivers strategic insight and game-changing conversations that shape the rapidly evolving mobile landscape. Sign up now. http://pubads.g.doubleclick.net/gampad/clk?id=63431311&iu=/4140/ostg.clktrk ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Fwd: Spooling attrs takes forever
On 11/15/13 17:21, James Harper wrote: >> >> When it does a full backup of one client (also running Mint 15) of >> around 40G it gets stuck on the "Sending spooled attrs to the Director. >> Despooling 151,437,267 bytes ...". This one step takes over eight hours >> to complete. For instance it started at 03:10 my time today and it's >> now 08:20 and it's still running. >> > > I had this spring up suddenly after a kernel upgrade. strace showed that > mysql was constantly doing 512 byte writes + fsync's, and I guess there was a > kernel update that handled fsync differently. I have a RAID1 of SATA disks, > and a 7200RPM SATA disk gets something like 75 IOPS, and if each op is 512 > bytes that's a very low throughput. > > I think I added: > > innodb_flush_log_at_trx_commit=0 > > to my config file and the performance went back up to what I was used to, at > the expense of a slight exposure to data loss in the event of a power failure. Actually, innodb_flush_log_at_trx_commit = 2 is the recommended setting for most cases, except where it is critical that no data be lost, ever. With that value, MySQL will attempt to flush the log once per second, which is a very good compromise between performance and risk of data loss. -- Phil Stracchino, CDK#2 DoD#299792458 ICBM: 43.5607, -71.355 ala...@caerllewys.net ala...@metrocast.net p...@co.ordinate.org Renaissance Man, Unix ronin, Perl hacker, SQL wrangler, Free Stater It's not the years, it's the mileage. -- DreamFactory - Open Source REST & JSON Services for HTML5 & Native Apps OAuth, Users, Roles, SQL, NoSQL, BLOB Storage and External API Access Free app hosting. Or install the open source package on any LAMP server. Sign up and see examples for AngularJS, jQuery, Sencha Touch and Native! http://pubads.g.doubleclick.net/gampad/clk?id=63469471&iu=/4140/ostg.clktrk ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Fwd: Spooling attrs takes forever
> > When it does a full backup of one client (also running Mint 15) of > around 40G it gets stuck on the "Sending spooled attrs to the Director. > Despooling 151,437,267 bytes ...". This one step takes over eight hours > to complete. For instance it started at 03:10 my time today and it's > now 08:20 and it's still running. > I had this spring up suddenly after a kernel upgrade. strace showed that mysql was constantly doing 512 byte writes + fsync's, and I guess there was a kernel update that handled fsync differently. I have a RAID1 of SATA disks, and a 7200RPM SATA disk gets something like 75 IOPS, and if each op is 512 bytes that's a very low throughput. I think I added: innodb_flush_log_at_trx_commit=0 to my config file and the performance went back up to what I was used to, at the expense of a slight exposure to data loss in the event of a power failure. James -- DreamFactory - Open Source REST & JSON Services for HTML5 & Native Apps OAuth, Users, Roles, SQL, NoSQL, BLOB Storage and External API Access Free app hosting. Or install the open source package on any LAMP server. Sign up and see examples for AngularJS, jQuery, Sencha Touch and Native! http://pubads.g.doubleclick.net/gampad/clk?id=63469471&iu=/4140/ostg.clktrk ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Fwd: Spooling attrs takes forever
On 11/15/2013 06:35 AM, Kern Sibbald wrote: > Hello, > > I would suggest one minor modification to what you > wrote. See below ... > > On 11/14/2013 04:29 PM, Josh Fisher wrote: >> >> On 11/12/2013 7:09 PM, Charles Douglass wrote: >>> This appears to have never posted. My apologies if I'm wrong and >>> this is duplicate. >>> >>> >>> Original Message >>> Subject:Spooling attrs takes forever >>> Date: Mon, 11 Nov 2013 08:30:59 -0800 >>> From: Charles Douglass >>> To: bacula-users@lists.sourceforge.net >>> >>> >>> >>> I am running : >>> Linux Mint 15 (based on Ubuntu 13.04) >>> Bacula "Version: 5.2.6 (21 February 2012)" >>> MySQL version "14.14 Distrib 5.5.32, for debian-linux-gnu (x86_64) using >>> readline 6.2" >>> >>> It's a network installation with several clients. >>> >>> When it does a full backup of one client (also running Mint 15) of >>> around 40G it gets stuck on the "Sending spooled attrs to the Director. >>> Despooling 151,437,267 bytes ...". This one step takes over eight hours >>> to complete. For instance it started at 03:10 my time today and it's >>> now 08:20 and it's still running. >> >> If backups are being written to fast local disk storage, then you >> would likely be better off with no spooling ( SpoolData=no and >> SpoolAttributes=no in bacula-dir.conf ). > > Yes, for disk storage, it does not make much sense to have data > spooling turned off. > I would suggest to always turn attribute spooling on (default off) so > that attributes > will be inserted in batch mode (much faster), and if possible ensure > that the > working directory, where attributes are spooled is on a different > drive from the > Archive Directory. Of course this last suggestion is most often not > possible. > > Kern > >> >> In any case, when spooling attributes, Bacula writes the attribute >> info to a temporary file in the Bacula work directory, then despools >> when the job is finished receiving data from the client by reading >> the temp file and updating the DB as a batch. My guess is that the >> problem you are having is the result of having the Bacula work >> directory and the DB storage on the same physical drive(s). Unless >> you have SSD drives, the constant movement of the read/write heads is >> killing performance (ie. disk thrashing). >> >>> There is no recent mention of this problem that I could find. One >>> suggestion from 2012 was to reinitialize the DB so I did that, with no >>> change to the problem. >>> >>> Job 1, which is a local backup of about 20G shows this on the console: >>> 11-Nov 00:35 maple-sd JobId 1: Sending spooled attrs to the Director. >>> Despooling 9,801,461 bytes ... >>> 11-Nov 01:11 maple-dir JobId 1: Bacula maple-dir 5.2.6 (21Feb12): >>> >>> Which seems long, but is way faster than Job 2. >>> >>> Chas Douglass >>> >>> >>> >>> > Sorry, I should have mentioned: I'm backing up to a 80/160 DAT drive. As a side note, spooling has never done what I think spooling should do. It says "spooling" and reads the disk for awhile, then says "writing to tape" and writes to tape for awhile. There doesn't seem to be any parallel processing going on. Chas Douglass -- DreamFactory - Open Source REST & JSON Services for HTML5 & Native Apps OAuth, Users, Roles, SQL, NoSQL, BLOB Storage and External API Access Free app hosting. Or install the open source package on any LAMP server. Sign up and see examples for AngularJS, jQuery, Sencha Touch and Native! http://pubads.g.doubleclick.net/gampad/clk?id=63469471&iu=/4140/ostg.clktrk___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Fwd: Spooling attrs takes forever
> Yes, for disk storage, it does not make much sense to have data spooling > turned off. > I would suggest to always turn attribute spooling on (default off) so > that attributes > will be inserted in batch mode (much faster), and if possible ensure > that the > working directory, where attributes are spooled is on a different drive > from the > Archive Directory. Of course this last suggestion is most often not > possible. One reason to turn on spooling even if you use disk storage for your volumes if you tend to have hosts that abruptly get pulled off the network during backups or otherwise have hiccups that cause backups to fail. With spooling, you shouldn't get volumes filling up with backup data from the partially completed failed backups. --tom -- DreamFactory - Open Source REST & JSON Services for HTML5 & Native Apps OAuth, Users, Roles, SQL, NoSQL, BLOB Storage and External API Access Free app hosting. Or install the open source package on any LAMP server. Sign up and see examples for AngularJS, jQuery, Sencha Touch and Native! http://pubads.g.doubleclick.net/gampad/clk?id=63469471&iu=/4140/ostg.clktrk ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Fwd: Spooling attrs takes forever
Hello, I would suggest one minor modification to what you wrote. See below ... On 11/14/2013 04:29 PM, Josh Fisher wrote: > > On 11/12/2013 7:09 PM, Charles Douglass wrote: >> This appears to have never posted. My apologies if I'm wrong and >> this is duplicate. >> >> >> Original Message >> Subject: Spooling attrs takes forever >> Date:Mon, 11 Nov 2013 08:30:59 -0800 >> From:Charles Douglass >> To: bacula-users@lists.sourceforge.net >> >> >> >> I am running : >> Linux Mint 15 (based on Ubuntu 13.04) >> Bacula "Version: 5.2.6 (21 February 2012)" >> MySQL version "14.14 Distrib 5.5.32, for debian-linux-gnu (x86_64) using >> readline 6.2" >> >> It's a network installation with several clients. >> >> When it does a full backup of one client (also running Mint 15) of >> around 40G it gets stuck on the "Sending spooled attrs to the Director. >> Despooling 151,437,267 bytes ...". This one step takes over eight hours >> to complete. For instance it started at 03:10 my time today and it's >> now 08:20 and it's still running. > > If backups are being written to fast local disk storage, then you > would likely be better off with no spooling ( SpoolData=no and > SpoolAttributes=no in bacula-dir.conf ). Yes, for disk storage, it does not make much sense to have data spooling turned off. I would suggest to always turn attribute spooling on (default off) so that attributes will be inserted in batch mode (much faster), and if possible ensure that the working directory, where attributes are spooled is on a different drive from the Archive Directory. Of course this last suggestion is most often not possible. Kern > > In any case, when spooling attributes, Bacula writes the attribute > info to a temporary file in the Bacula work directory, then despools > when the job is finished receiving data from the client by reading the > temp file and updating the DB as a batch. My guess is that the problem > you are having is the result of having the Bacula work directory and > the DB storage on the same physical drive(s). Unless you have SSD > drives, the constant movement of the read/write heads is killing > performance (ie. disk thrashing). > >> There is no recent mention of this problem that I could find. One >> suggestion from 2012 was to reinitialize the DB so I did that, with no >> change to the problem. >> >> Job 1, which is a local backup of about 20G shows this on the console: >> 11-Nov 00:35 maple-sd JobId 1: Sending spooled attrs to the Director. >> Despooling 9,801,461 bytes ... >> 11-Nov 01:11 maple-dir JobId 1: Bacula maple-dir 5.2.6 (21Feb12): >> >> Which seems long, but is way faster than Job 2. >> >> Chas Douglass >> >> >> >> >> >> -- >> DreamFactory - Open Source REST & JSON Services for HTML5 & Native Apps >> OAuth, Users, Roles, SQL, NoSQL, BLOB Storage and External API Access >> Free app hosting. Or install the open source package on any LAMP server. >> Sign up and see examples for AngularJS, jQuery, Sencha Touch and Native! >> http://pubads.g.doubleclick.net/gampad/clk?id=63469471&iu=/4140/ostg.clktrk >> >> >> ___ >> Bacula-users mailing list >> Bacula-users@lists.sourceforge.net >> https://lists.sourceforge.net/lists/listinfo/bacula-users > > > > -- > DreamFactory - Open Source REST & JSON Services for HTML5 & Native Apps > OAuth, Users, Roles, SQL, NoSQL, BLOB Storage and External API Access > Free app hosting. Or install the open source package on any LAMP server. > Sign up and see examples for AngularJS, jQuery, Sencha Touch and Native! > http://pubads.g.doubleclick.net/gampad/clk?id=63469471&iu=/4140/ostg.clktrk > > > ___ > Bacula-users mailing list > Bacula-users@lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/bacula-users -- DreamFactory - Open Source REST & JSON Services for HTML5 & Native Apps OAuth, Users, Roles, SQL, NoSQL, BLOB Storage and External API Access Free app hosting. Or install the open source package on any LAMP server. Sign up and see examples for AngularJS, jQuery, Sencha Touch and Native! http://pubads.g.doubleclick.net/gampad/clk?id=63469471&iu=/4140/ostg.clktrk___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Fwd: Spooling attrs takes forever
I've been running Bacula on MySQL for more than five years on Ubuntu. I don't know why it changed. I suppose there were big changes that came in the MySQL 5 line. Chas Douglass On 11/13/2013 09:01 AM, Phil Stracchino wrote: > On 11/13/13 10:59, Charles Douglass wrote: >> Thanks for the reply. >> >> I know nothing about MySQL optimization. I downloaded the >> "mysqltuner.pl" script and I will run it after the next full backup to >> see if it has any recommendations. > The mysqltuner script is a pretty decent basic tool. The thing you need > to keep in mind regarding the "mysql-large", "mysql-huge" sample > configurations that are STILL widely distributed in MySQL packages (and > still widely used by the unsuspecting) is that most of those sample > configurations were originally written back when a "large" server was > one that might have as much as a whole 32MB of RAM. These days, that is > less than the compiled-in default size of some individual MySQL > *buffers*. So the unwary install the "suggested" configurations, and > can't understand why MySQL is performing like a geriatric tortoise. > > -- DreamFactory - Open Source REST & JSON Services for HTML5 & Native Apps OAuth, Users, Roles, SQL, NoSQL, BLOB Storage and External API Access Free app hosting. Or install the open source package on any LAMP server. Sign up and see examples for AngularJS, jQuery, Sencha Touch and Native! http://pubads.g.doubleclick.net/gampad/clk?id=63469471&iu=/4140/ostg.clktrk ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Fwd: Spooling attrs takes forever
On 11/12/2013 7:09 PM, Charles Douglass wrote: This appears to have never posted. My apologies if I'm wrong and this is duplicate. Original Message Subject: Spooling attrs takes forever Date: Mon, 11 Nov 2013 08:30:59 -0800 From: Charles Douglass To: bacula-users@lists.sourceforge.net I am running : Linux Mint 15 (based on Ubuntu 13.04) Bacula "Version: 5.2.6 (21 February 2012)" MySQL version "14.14 Distrib 5.5.32, for debian-linux-gnu (x86_64) using readline 6.2" It's a network installation with several clients. When it does a full backup of one client (also running Mint 15) of around 40G it gets stuck on the "Sending spooled attrs to the Director. Despooling 151,437,267 bytes ...". This one step takes over eight hours to complete. For instance it started at 03:10 my time today and it's now 08:20 and it's still running. If backups are being written to fast local disk storage, then you would likely be better off with no spooling ( SpoolData=no and SpoolAttributes=no in bacula-dir.conf ). In any case, when spooling attributes, Bacula writes the attribute info to a temporary file in the Bacula work directory, then despools when the job is finished receiving data from the client by reading the temp file and updating the DB as a batch. My guess is that the problem you are having is the result of having the Bacula work directory and the DB storage on the same physical drive(s). Unless you have SSD drives, the constant movement of the read/write heads is killing performance (ie. disk thrashing). There is no recent mention of this problem that I could find. One suggestion from 2012 was to reinitialize the DB so I did that, with no change to the problem. Job 1, which is a local backup of about 20G shows this on the console: 11-Nov 00:35 maple-sd JobId 1: Sending spooled attrs to the Director. Despooling 9,801,461 bytes ... 11-Nov 01:11 maple-dir JobId 1: Bacula maple-dir 5.2.6 (21Feb12): Which seems long, but is way faster than Job 2. Chas Douglass -- DreamFactory - Open Source REST & JSON Services for HTML5 & Native Apps OAuth, Users, Roles, SQL, NoSQL, BLOB Storage and External API Access Free app hosting. Or install the open source package on any LAMP server. Sign up and see examples for AngularJS, jQuery, Sencha Touch and Native! http://pubads.g.doubleclick.net/gampad/clk?id=63469471&iu=/4140/ostg.clktrk ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users -- DreamFactory - Open Source REST & JSON Services for HTML5 & Native Apps OAuth, Users, Roles, SQL, NoSQL, BLOB Storage and External API Access Free app hosting. Or install the open source package on any LAMP server. Sign up and see examples for AngularJS, jQuery, Sencha Touch and Native! http://pubads.g.doubleclick.net/gampad/clk?id=63469471&iu=/4140/ostg.clktrk___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Fwd: Spooling attrs takes forever
On 11/13/13 10:59, Charles Douglass wrote: > Thanks for the reply. > > I know nothing about MySQL optimization. I downloaded the > "mysqltuner.pl" script and I will run it after the next full backup to > see if it has any recommendations. The mysqltuner script is a pretty decent basic tool. The thing you need to keep in mind regarding the "mysql-large", "mysql-huge" sample configurations that are STILL widely distributed in MySQL packages (and still widely used by the unsuspecting) is that most of those sample configurations were originally written back when a "large" server was one that might have as much as a whole 32MB of RAM. These days, that is less than the compiled-in default size of some individual MySQL *buffers*. So the unwary install the "suggested" configurations, and can't understand why MySQL is performing like a geriatric tortoise. -- Phil Stracchino, CDK#2 DoD#299792458 ICBM: 43.5607, -71.355 ala...@caerllewys.net ala...@metrocast.net p...@co.ordinate.org Renaissance Man, Unix ronin, Perl hacker, SQL wrangler, Free Stater It's not the years, it's the mileage. -- DreamFactory - Open Source REST & JSON Services for HTML5 & Native Apps OAuth, Users, Roles, SQL, NoSQL, BLOB Storage and External API Access Free app hosting. Or install the open source package on any LAMP server. Sign up and see examples for AngularJS, jQuery, Sencha Touch and Native! http://pubads.g.doubleclick.net/gampad/clk?id=63469471&iu=/4140/ostg.clktrk ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Fwd: Spooling attrs takes forever
Thanks for the reply. I know nothing about MySQL optimization. I downloaded the "mysqltuner.pl" script and I will run it after the next full backup to see if it has any recommendations. Chas Douglass On 11/12/2013 04:22 PM, John Drescher wrote: >> I am running : >> Linux Mint 15 (based on Ubuntu 13.04) >> Bacula "Version: 5.2.6 (21 February 2012)" >> MySQL version "14.14 Distrib 5.5.32, for debian-linux-gnu (x86_64) using >> readline 6.2" >> >> It's a network installation with several clients. >> >> When it does a full backup of one client (also running Mint 15) of >> around 40G it gets stuck on the "Sending spooled attrs to the Director. >> Despooling 151,437,267 bytes ...". This one step takes over eight hours >> to complete. For instance it started at 03:10 my time today and it's >> now 08:20 and it's still running. >> >> There is no recent mention of this problem that I could find. One >> suggestion from 2012 was to reinitialize the DB so I did that, with no >> change to the problem. >> >> Job 1, which is a local backup of about 20G shows this on the console: >> 11-Nov 00:35 maple-sd JobId 1: Sending spooled attrs to the Director. >> Despooling 9,801,461 bytes ... >> 11-Nov 01:11 maple-dir JobId 1: Bacula maple-dir 5.2.6 (21Feb12): >> >> Which seems long, but is way faster than Job 2. > > Did you optimize your my.cnf for your system? In many distributions > the default parameters are for a machine with a few MB of rm. Yes I > mean MBs of ram. > > John -- DreamFactory - Open Source REST & JSON Services for HTML5 & Native Apps OAuth, Users, Roles, SQL, NoSQL, BLOB Storage and External API Access Free app hosting. Or install the open source package on any LAMP server. Sign up and see examples for AngularJS, jQuery, Sencha Touch and Native! http://pubads.g.doubleclick.net/gampad/clk?id=63469471&iu=/4140/ostg.clktrk ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Fwd: Spooling attrs takes forever
> I am running : > Linux Mint 15 (based on Ubuntu 13.04) > Bacula "Version: 5.2.6 (21 February 2012)" > MySQL version "14.14 Distrib 5.5.32, for debian-linux-gnu (x86_64) using > readline 6.2" > > It's a network installation with several clients. > > When it does a full backup of one client (also running Mint 15) of > around 40G it gets stuck on the "Sending spooled attrs to the Director. > Despooling 151,437,267 bytes ...". This one step takes over eight hours > to complete. For instance it started at 03:10 my time today and it's > now 08:20 and it's still running. > > There is no recent mention of this problem that I could find. One > suggestion from 2012 was to reinitialize the DB so I did that, with no > change to the problem. > > Job 1, which is a local backup of about 20G shows this on the console: > 11-Nov 00:35 maple-sd JobId 1: Sending spooled attrs to the Director. > Despooling 9,801,461 bytes ... > 11-Nov 01:11 maple-dir JobId 1: Bacula maple-dir 5.2.6 (21Feb12): > > Which seems long, but is way faster than Job 2. Did you optimize your my.cnf for your system? In many distributions the default parameters are for a machine with a few MB of rm. Yes I mean MBs of ram. John -- DreamFactory - Open Source REST & JSON Services for HTML5 & Native Apps OAuth, Users, Roles, SQL, NoSQL, BLOB Storage and External API Access Free app hosting. Or install the open source package on any LAMP server. Sign up and see examples for AngularJS, jQuery, Sencha Touch and Native! http://pubads.g.doubleclick.net/gampad/clk?id=63469471&iu=/4140/ostg.clktrk ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
