Hi Ugo,
Most likely your database is not corrupt, but only few entries had the duplicate keys (IDs). CloudStack should GC failed async jobs and mark jobs to be removed when mgmt server is shutdown. Unless you're seeing any errors you should be okay, however, do keep backups. - Rohit <https://cloudstack.apache.org> ________________________________ From: Ugo Vasi <[email protected]> Sent: Monday, October 1, 2018 3:11:39 PM To: [email protected] Subject: Fwd: Urgent: MySQLIntegrityConstraintViolationException: Duplicate entry Hi all, I found a problem in management server, now resolved, that rebooted the machine some times. Is it possible that the database was corrupted in this case? I don't found mysql server explicit errors. Regards Ugo -------- Messaggio Inoltrato -------- Oggetto: Urgent: MySQLIntegrityConstraintViolationException: Duplicate entry Data: Mon, 1 Oct 2018 08:59:32 +0200 Mittente: Ugo Vasi <[email protected]> Rispondi-a: [email protected] Organizzazione: PROCNE SRL A: users <[email protected]> Hi all, we had a problem with one of the hosts in an ACS 4.11.1-based cluster (ubuntu 16.04) and the high-availability system restored this system. After this, the VMs I was trying to restart manually gave me this message: Unable to schedule async job for command com.cloud.vm.VmWorkStart, unexpected exception. Repeating several times to restart the VM after a while it starts. In the /var/log/cloudstack/management/management-server.log files I find these messages at the boot command via the web interface: 2018-10-01 08:38:22,569 DEBUG [c.c.u.d.T.Transaction] (API-Job-Executor-10:ctx-7dbf3117 job-652 ctx-9ca1bda7) (logid:05d81b34) Rolling back the transaction: Time = 2 Name = API-Job-Executor-10; called by -TransactionLegacy.rollback:889-TransactionLegacy.removeUpTo:832-TransactionLegacy.close:656-Transaction.execute:43-Transaction.execute:47-AsyncJobManagerImpl.submitAsyncJob:231-VirtualMachineManagerImpl.startVmThroughJobQueue:4457-VirtualMachineManagerImpl.advanceStart:899-VirtualMachineManagerImpl.start:718-VMEntityManagerImpl.deployVirtualMachine:233-VirtualMachineEntityImpl.deploy:212-UserVmManagerImpl.startVirtualMachine:4495 2018-10-01 08:38:22,575 WARN [o.a.c.f.j.i.AsyncJobManagerImpl] (API-Job-Executor-10:ctx-7dbf3117 job-652 ctx-9ca1bda7) (logid:05d81b34) Unable to schedule async job for command com.cloud.vm.VmWorkStart, unexpected exception. javax.persistence.EntityExistsException: Entity already exists: at com.cloud.utils.db.GenericDaoBase.persist(GenericDaoBase.java:1434) at org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$1.doInTransaction(AsyncJobManagerImpl.java:235) .... at com.cloud.vm.VirtualMachineManagerImpl.advanceStart(VirtualMachineManagerImpl.java:899) at com.cloud.vm.VirtualMachineManagerImpl.start(VirtualMachineManagerImpl.java:718) at org.apache.cloudstack.engine.cloud.entity.api.VMEntityManagerImpl.deployVirtualMachine(VMEntityManagerImpl.java:233) at org.apache.cloudstack.engine.cloud.entity.api.VirtualMachineEntityImpl.deploy(VirtualMachineEntityImpl.java:212) at com.cloud.vm.UserVmManagerImpl.startVirtualMachine(UserVmManagerImpl.java:4495) .... at com.cloud.api.ApiAsyncJobDispatcher.runJob(ApiAsyncJobDispatcher.java:108) at org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.runInContext(AsyncJobManagerImpl.java:581) at org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedContextRunnable.java:49) at org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:56) at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:103) at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:53) at org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedContextRunnable.java:46) .... at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) Caused by: com.mysql.jdbc.exceptions.jdbc4.MySQLIntegrityConstraintViolationException: Duplicate entry '653' for key 'PRIMARY' at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) ... 60 more 2018-10-01 08:38:22,590 DEBUG [c.c.u.d.T.Transaction] (API-Job-Executor-10:ctx-7dbf3117 job-652 ctx-9ca1bda7) (logid:05d81b34) Rolling back the transaction: Time = 2 Name = API-Job-Executor-10; called by -TransactionLegacy.rollback:889-TransactionLegacy.removeUpTo:832-TransactionLegacy.close:656-Transaction.execute:43-Transaction.execute:47-AsyncJobManagerImpl.submitAsyncJob:231-VirtualMachineManagerImpl.startVmThroughJobQueue:4457-VirtualMachineManagerImpl.advanceStart:899-VirtualMachineManagerImpl.start:718-VMEntityManagerImpl.deployVirtualMachine:245-VirtualMachineEntityImpl.deploy:212-UserVmManagerImpl.startVirtualMachine:4495 2018-10-01 08:38:22,593 WARN [o.a.c.f.j.i.AsyncJobManagerImpl] (API-Job-Executor-10:ctx-7dbf3117 job-652 ctx-9ca1bda7) (logid:05d81b34) Unable to schedule async job for command com.cloud.vm.VmWorkStart, unexpected exception. javax.persistence.EntityExistsException: Entity already exists: at com.cloud.utils.db.GenericDaoBase.persist(GenericDaoBase.java:1434) at org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$1.doInTransaction(AsyncJobManagerImpl.java:235) .... at com.cloud.vm.VirtualMachineManagerImpl.advanceStart(VirtualMachineManagerImpl.java:899) at com.cloud.vm.VirtualMachineManagerImpl.start(VirtualMachineManagerImpl.java:718) at org.apache.cloudstack.engine.cloud.entity.api.VMEntityManagerImpl.deployVirtualMachine(VMEntityManagerImpl.java:245) at org.apache.cloudstack.engine.cloud.entity.api.VirtualMachineEntityImpl.deploy(VirtualMachineEntityImpl.java:212) .... at org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:56) at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:103) at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:53) at org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedContextRunnable.java:46) .... at java.lang.Thread.run(Thread.java:748) Caused by: com.mysql.jdbc.exceptions.jdbc4.MySQLIntegrityConstraintViolationException: Duplicate entry '654' for key 'PRIMARY' at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) ... 60 more 2018-10-01 08:38:22,603 ERROR [c.c.a.ApiAsyncJobDispatcher] (API-Job-Executor-10:ctx-7dbf3117 job-652) (logid:05d81b34) Unexpected exception while executing org.apache.cloudstack.api.command.admin.vm.StartVMCmdByAdmin com.cloud.utils.exception.CloudRuntimeException: Unable to schedule async job for command com.cloud.vm.VmWorkStart, unexpected exception. at org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl.submitAsyncJob(AsyncJobManagerImpl.java:247) at com.cloud.vm.VirtualMachineManagerImpl.startVmThroughJobQueue(VirtualMachineManagerImpl.java:4457) at com.cloud.vm.VirtualMachineManagerImpl.advanceStart(VirtualMachineManagerImpl.java:899) at com.cloud.vm.VirtualMachineManagerImpl.start(VirtualMachineManagerImpl.java:718) at org.apache.cloudstack.engine.cloud.entity.api.VMEntityManagerImpl.deployVirtualMachine(VMEntityManagerImpl.java:245) at org.apache.cloudstack.engine.cloud.entity.api.VirtualMachineEntityImpl.deploy(VirtualMachineEntityImpl.java:212) .... at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:103) at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:53) .... at java.lang.Thread.run(Thread.java:748) 2018-10-01 08:38:22,605 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl] (API-Job-Executor-10:ctx-7dbf3117 job-652) (logid:05d81b34) Complete async job-652, jobStatus: FAILED, resultCode: 530, result: org.apache.cloudstack.api.response.ExceptionResponse/null/{"uuidList":[],"errorcode":530,"errortext":"Unable to schedule async job for command com.cloud.vm.VmWorkStart, unexpected exception."} I'm investigating the incident but I would like to know if anyone has had this problem or if it can help me to figure out how to solve it. Thanks for any advise! -- *Ugo Vasi* / System Administrator [email protected] <mailto:[email protected]> *Procne S.r.l.* +39 0432 486 523 via Cotonificio, 45 33010 Tavagnacco (UD) www.procne.it<http://www.procne.it> <http://www.procne.it/> Le informazioni contenute nella presente comunicazione ed i relativi allegati possono essere riservate e sono, comunque, destinate esclusivamente alle persone od alla Società sopraindicati. La diffusione, distribuzione e/o copiatura del documento trasmesso da parte di qualsiasi soggetto diverso dal destinatario è proibita sia ai sensi dell'art. 616 c.p., che ai sensi del Decreto Legislativo n. 196/2003 "Codice in materia di protezione dei dati personali". Se avete ricevuto questo messaggio per errore, vi preghiamo di distruggerlo e di informare immediatamente Procne S.r.l. scrivendo all' indirizzo e-mail [email protected] <mailto:[email protected]>. [email protected] www.shapeblue.com Amadeus House, Floral Street, London WC2E 9DPUK @shapeblue
