[Bacula-users] How to troubleshoot a slow client, when there is no obvious answer
If you have checked disk I/O, CPU, memory, network, on both the client and the server, all seem great, both from a statistics look, showing all running normally, with low use. Verified with read/write test on local disks, remote through he network, using regular file copies (via SMB) and iperf on both client and server. However my backups which ran last weeks full backup in 3.5hrs at 26mbps, are now running at under 4mbps for this week. I have stopped and restarted things, no change. Every test I can think of shows everything is running great, but the backups are going to take more than a day at this rate. This isn't a new setup, I have been doing this for some time, using the same server / client combination. The only other client on the server ran normally, but it uses gzip (the one running slow doesn't) and never ran faster than about 6mbps, with between 55-65% compression. The local catalog backup ran at 60mpbs, which is almost as fast as dd if=/dev/zero runs against the drive I write my backup volumes to, as the dir and sd are on the same machine. Could I possibly be having a Database issue, I am using PostgreSQL, that could cause this, possibly waiting on records to be written to the catalog? If so how could I find out? Anything else anyone can think of to check? -- Thanks, Dean E. Weimer http://www.dweimer.net/ -- Master Visual Studio, SharePoint, SQL, ASP.NET, C# 2012, HTML5, CSS, MVC, Windows 8 Apps, JavaScript and much more. Keep your skills current with LearnDevNow - 3,200 step-by-step video tutorials by Microsoft MVPs and experts. ON SALE this month only -- learn more at: http://p.sf.net/sfu/learnnow-d2d ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] How to troubleshoot a slow client, when there is no obvious answer
On Tue, Jan 22, 2013 at 01:37:43PM -0600, dweimer wrote: If you have checked disk I/O, CPU, memory, network, on both the client and the server, all seem great, both from a statistics look, showing all What about many small files, possibly very many in single directories? I have seen these with huge impact on backup performance, especially on busy directories. Regards, Adrian -- LiHAS - Adrian Reyer - Hessenwiesenstraße 10 - D-70565 Stuttgart Fon: +49 (7 11) 78 28 50 90 - Fax: +49 (7 11) 78 28 50 91 Mail: li...@lihas.de - Web: http://lihas.de Linux, Netzwerke, Consulting Support - USt-ID: DE 227 816 626 Stuttgart -- Master Visual Studio, SharePoint, SQL, ASP.NET, C# 2012, HTML5, CSS, MVC, Windows 8 Apps, JavaScript and much more. Keep your skills current with LearnDevNow - 3,200 step-by-step video tutorials by Microsoft MVPs and experts. ON SALE this month only -- learn more at: http://p.sf.net/sfu/learnnow-d2d ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] How to troubleshoot a slow client, when there is no obvious answer
On 2013-01-22 15:34, Adrian Reyer wrote: On Tue, Jan 22, 2013 at 01:37:43PM -0600, dweimer wrote: If you have checked disk I/O, CPU, memory, network, on both the client and the server, all seem great, both from a statistics look, showing all What about many small files, possibly very many in single directories? I have seen these with huge impact on backup performance, especially on busy directories. Regards, Adrian There really hasn't been a huge change in the data, below are from the client status, the full I canceled this morning has the majority of files done, but not the bulk of the data. The biggest chuck of which is the around 30 iTunes movie files, which is where I should see the best performance in the backup, at the point I canceled this it was in the process of processing one of those 4+G movie files. Full 67,353284.8 G OK 15-Jan-13 03:30 Incr 0 0 OK 16-Jan-13 00:05 Incr 185.145 M OK 17-Jan-13 00:05 Incr 239.016 M OK 18-Jan-13 00:05 Incr 811.50 M OK 19-Jan-13 00:05 Incr 3116.09 M OK 21-Jan-13 00:08 Incr 3116.09 M OK 21-Jan-13 00:08 Full 53,174125.0 G Cancel 22-Jan-13 08:46 However you see there are multiple incremental backups ran on the 21st. I did have some failures Monday morning, I had rebooted the server on Sunday and forgot to remount my backup volume. It was in the server during reboot, but is encrypted so it doesn't mount automatically. Those jobs had marked all of my disk volumes with an error state, I hadn't noticed, so when I remounted the volume for its Monday backups most jobs failed, due to lack of usable online volumes but a couple completed, and got kicked off a second time after I marked the volumes back online. Its possible all of this had been caused by snapshots that had been stuck around since the failure of the backup job on the bacula-dir and bacula-sd server, the database volume was included in this, that few day old snap shot may have been slowing down writes to the database. I have cleared that and its running a dbcheck on the database now, I will kick off new backups when that finishes and see how things go. Hopefully this was the cause, however I may have to do some more performance searching because the snapshot shouldn't have caused the performance to degrade that much. -- Thanks, Dean E. Weimer http://www.dweimer.net/ -- Master Visual Studio, SharePoint, SQL, ASP.NET, C# 2012, HTML5, CSS, MVC, Windows 8 Apps, JavaScript and much more. Keep your skills current with LearnDevNow - 3,200 step-by-step video tutorials by Microsoft MVPs and experts. ON SALE this month only -- learn more at: http://p.sf.net/sfu/learnnow-d2d ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
