Re: [bareos-users] Super slow query after upgrade to 17.2 from 16.2
Hi again, > What can I do to analyze/fix this? would it be possible/reasonable to clean up all the "accurate mode" information and start from zero? If yes, how can I do this? Unfortunately I'm not that much experienced with mySQL databases... Thanks for your support and many greets Stephan Am 25.02.2018 um 00:49 schrieb Stephan Hermann: > Hi, > > it seems that I run into this issue, too. > I just upgraded from 16.2.7 to 17.2.5 (from subscription repo) and > updated the database scheme from 2004 to 2171. > > Since then incremental backups with a lot of files hang at 'Sending > Accurate information.' There is one mysql process at 100% cpu, but I > can't see any relevant disk activity (iotop -o). > > Killing the queries inside mySQL allow me to continue the jobs, but the > result looks like a full job, not an incremental. > > Some outputs: > > mysql> analyze table File; > +-+-+--+--+ > | Table | Op | Msg_type | Msg_text | > +-+-+--+--+ > | bareos.File | analyze | status | OK | > +-+-+--+--+ > 1 row in set (0.04 sec) > > mysql> optimize table File; > +-+--+--+---+ > | Table | Op | Msg_type | Msg_text >| > +-+--+--+---+ > | bareos.File | optimize | note | Table does not support optimize, > doing recreate + analyze instead | > | bareos.File | optimize | status | OK >| > +-+--+--+---+ > 2 rows in set (4 min 35.79 sec) > > What can I do to analyze/fix this? > > Thanks a lot and many greets > Stephan > > > Am 23.01.2018 um 22:26 schrieb Stephan Duehr: >> Hi, >> >> that looks like the query being run for accurate backup. >> I wonder why it's slower after the upgrade, as it's one table less for the >> join. >> This kind of problem with accurate has also been seen with the Bareos 16.2 >> DB schema >> in MySQL. >> >> Please check if the indexes on the File table match the ones defined in >> /usr/lib/bareos/scripts/ddl/creates/mysql.sql >> >> May be do >> ANALYZE TABLE File; >> in mysql again. >> >> If that does not help, disabling accurate for that job meanwhile may be >> better >> than killing the query. >> >> Regards, >> >> Stephan >> >> On 01/22/2018 02:30 PM, jsfre...@ludia.com wrote: >>> Hi, After the upgrade to 17.2.4 (21 Sep 2017) I noticed the following sql >>> query: >>> >>> SELECT Path.Path, T1.Name, T1.FileIndex, T1.JobId, LStat, DeltaSeq , >>> Fhinfo, Fhnode FROM ( SELECT FileId, Job.JobId AS JobId, FileIndex, >>> File.PathId AS PathId, File.Name AS Name, LStat , DeltaSeq, Fhinfo, Fhnode, >>> Job.JobTDate AS JobTDate FROM Job, File, (SELECT MAX(JobTDate) AS JobTDate, >>> PathId, FileName FROM (SELECT JobTDate, PathId, File.Name AS FileName FROM >>> File JOIN Job USING (JobId) WHERE File.JobId IN >>> (2897,2941,2963,3005,3023,3041) UNION ALL SELECT JobTDate, PathId, >>> File.Name AS FileName FROM BaseFiles JOIN File USING (FileId) JOIN Job ON >>> (BaseJobId = Job.JobId) WHERE BaseFiles.JobId IN >>> (2897,2941,2963,3005,3023,3041) ) AS tmp GROUP BY PathId, FileName) AS T1 >>> WHERE (Job.JobId IN (SELECT DISTINCT BaseJobId FROM BaseFiles WHERE JobId >>> IN (2897,2941,2963,3005,3023,3041)) OR Job.JobId IN >>> (2897,2941,2963,3005,3023,3041)) AND T1.JobTDate = Job.JobTDate AND >>> Job.JobId = File.JobId AND T1.PathId = File.PathId AND T1.FileName = >>> File.Name ) AS T1 JOIN Path ON (Path.PathId = T1.PathId) WHERE FileIndex > >>> 0 ORDER BY T1.JobTDate, FileIndex ASC >>> >>> This query is taking a very long time to complete. Now it's been running >>> over 13 hours and it is still running and taking 100% CPU. >>> >>> I'm using mysql 5.5, and the schema was upgraded when upgrading from 16.2 >>> to 17.2. >>> >>> For the last few days I had to kill the queries, but it would be great to >>> have this fixed. >>> >>> Any idea what is wrong here ? >>> >>> Thank you. >>> >> > -- You received this message because you are subscribed to the Google Groups "bareos-users" group. To unsubscribe from this group and stop receiving emails from it, send an email to bareos-users+unsubscr...@googlegroups.com. To post to this group, send email to bareos-users@googlegroups.com. For more options, visit https://groups.google.com/d/optout. signature.asc Description: OpenPGP digital signature
Re: [bareos-users] How to get parallel jobs working?
Hello Martin >> Probably a problem with the job priority? > Hmm can you elaborate? I have "Allow Mixed Priority" already set to yes > (tried the default "no" before, too). > Otherwise, all Jobs have the same priority 10. Do you mean with director the job definitions? Do you use different job definitions? If not do you use "Allow Duplicate Jobs"? Regards, Stefan -- *CaC, Computer and Communication* Inhaber Stefan Klatt End-2-End Senior Network Consultant CISSP / CISM / ISO27001 Lead Implementer / TOGAF9 Badges: https://www.youracclaim.com/users/stefan-klatt Triftstrasse 9 60528 Frankfurt Germany USt-IdNr.: DE260461592 Tel.: +49-(0)172-6807809 Tel.: +49-(0)69-67808-900 Fax: +49-(0)69-67808-837 Email: stefan.kl...@cac-netzwerk.de Profil: http://www.cac-netzwerk.de/profil -- You received this message because you are subscribed to the Google Groups "bareos-users" group. To unsubscribe from this group and stop receiving emails from it, send an email to bareos-users+unsubscr...@googlegroups.com. To post to this group, send email to bareos-users@googlegroups.com. For more options, visit https://groups.google.com/d/optout. smime.p7s Description: S/MIME Cryptographic Signature
Re: [bareos-users] How to get parallel jobs working?
Hi Stefan! Am Montag, 26. Februar 2018 19:20:21 UTC+1 schrieb Stefan Klatt: > > Probably a problem with the job priority? Hmm can you elaborate? I have "Allow Mixed Priority" already set to yes (tried the default "no" before, too). Otherwise, all Jobs have the same priority 10. Thanks, Martin -- You received this message because you are subscribed to the Google Groups "bareos-users" group. To unsubscribe from this group and stop receiving emails from it, send an email to bareos-users+unsubscr...@googlegroups.com. To post to this group, send email to bareos-users@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: [bareos-users] maximum volume size
Hi Stefan, Am 27.02.2018 um 12:27 schrieb 'Stefan Krüger' via bareos-users: > Hi Stefan ;) > > thanks for this. Our backup medium are harddisk in RAID6. > under this condition there are a few points from my point of view: - Raid6 are more stable than Raid5 or JBOD, but can have problems too - What about a pool of backup devices if your Raid is to small? - Copy jobs with such big files aren't handy All points show that volumes with a size of terrabytes can make live worse even if you have a little problem. Regards, Stefan Klatt -- *CaC, Computer and Communication* Inhaber Stefan Klatt End-2-End Senior Network Consultant CISSP / CISM / ISO27001 Lead Implementer / TOGAF9 Badges: https://www.youracclaim.com/users/stefan-klatt Triftstrasse 9 60528 Frankfurt Germany USt-IdNr.: DE260461592 Tel.: +49-(0)172-6807809 Tel.: +49-(0)69-67808-900 Fax: +49-(0)69-67808-837 Email: stefan.kl...@cac-netzwerk.de Profil: http://www.cac-netzwerk.de/profil -- You received this message because you are subscribed to the Google Groups "bareos-users" group. To unsubscribe from this group and stop receiving emails from it, send an email to bareos-users+unsubscr...@googlegroups.com. To post to this group, send email to bareos-users@googlegroups.com. For more options, visit https://groups.google.com/d/optout. smime.p7s Description: S/MIME Cryptographic Signature
[bareos-users] Re: Restore VMware VM backup if vCenter is down
I forgot to mention I'm running Bareos 17.2.4-6.1. -- You received this message because you are subscribed to the Google Groups "bareos-users" group. To unsubscribe from this group and stop receiving emails from it, send an email to bareos-users+unsubscr...@googlegroups.com. To post to this group, send email to bareos-users@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[bareos-users] Restore VMware VM backup if vCenter is down
My vCenter server is a Virtual Machine itself, and I've created backups of its VMDK using the Bareos VMware plugin. Now it's down and I need to restore the VMDK from a backup. This, however, seems not to be possible, as the plugins wants to connect to the vCenter server (my vCenter VM is called vc46-1): 27-Feb 12:10 bareos46-1-fd JobId 3976: Warning: python-fd: Could not retrieve SSL Cert from vc46-1.mobile.rz 27-Feb 12:10 bareos46-1-fd JobId 3976: Fatal error: python-fd: Traceback (most recent call last): File "/usr/lib/bareos/plugins/BareosFdWrapper.py", line 62, in plugin_io return bareos_fd_plugin_object.plugin_io(context, IOP) File "/usr/lib/bareos/plugins/BareosFdPluginVMware.py", line 358, in plugin_io self.vadp.keepalive() File "/usr/lib/bareos/plugins/BareosFdPluginVMware.py", line 693, in keepalive if int(time.time()) - self.si_last_keepalive > 900: TypeError: unsupported operand type(s) for -: 'int' and 'NoneType' 27-Feb 12:10 bareos-sd JobId 3976: Error: bsock_tcp.c:407 Write error sending 60131 bytes to client:10.46.22.80:9103: ERR=Connection reset by peer 27-Feb 12:10 bareos-sd JobId 3976: Fatal error: read.c:156 Error sending to File daemon. ERR=Connection reset by peer 27-Feb 12:10 bareos-sd JobId 3976: Error: bsock_tcp.c:452 Socket has errors=1 on call to client:10.46.22.80:9103 27-Feb 12:10 bareos-dir JobId 3976: Error: Bareos bareos-dir 17.2.4 (21Sep17): Build OS: x86_64-pc-linux-gnu debian Debian GNU/Linux 8.0 (jessie) JobId: 3976 Job:RestoreFiles.2018-02-27_12.10.37_04 Restore Client: bareos-fd Start time: 27-Feb-2018 12:10:39 End time: 27-Feb-2018 12:10:41 Elapsed time: 2 secs Files Expected: 2 Files Restored: 0 Bytes Restored: 0 Rate: 0.0 KB/s FD Errors: 1 FD termination status: Error SD termination status: Fatal Error Termination:*** Restore Error *** I would be happy if the plain VMDK file would just be extracted locally therefore I tried the plugin option "python:localvmdk=yes", but that didn't change the story. How can I extract a backup of this VM without connecting to the vCenter server? -- You received this message because you are subscribed to the Google Groups "bareos-users" group. To unsubscribe from this group and stop receiving emails from it, send an email to bareos-users+unsubscr...@googlegroups.com. To post to this group, send email to bareos-users@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: [bareos-users] maximum volume size
Hi Stefan ;) thanks for this. Our backup medium are harddisk in RAID6. -- You received this message because you are subscribed to the Google Groups "bareos-users" group. To unsubscribe from this group and stop receiving emails from it, send an email to bareos-users+unsubscr...@googlegroups.com. To post to this group, send email to bareos-users@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
