On March 31, 2020 12:21 pm, Mikhail wrote:
> Hello,
>
> On one of our clusters we're seeing issues with VM backup task - the
> backup task fails with the following:
>
> ERROR: Node 'drive-scsi0' is busy: block device is in use by block job:
> mirror
> INFO: aborting backup job
> ERROR: Backup of VM 123 failed - Node 'drive-scsi0' is busy: block
> device is in use by block job: mirror
>
> I tried digging some information and it appears to be that there's
> "drive-mirror" job in QEMU/KVM for this particular VM is blocking backup
> process. It is clear to me that this problem started couple weeks ago
> when we attempted to change VM's disk underlying storage, however the
> "Move disk" operation was cancelled manually by administrator at the
> time and backup tasks started failing right after that. I'm not sure
> whether this is Proxmox issue or QEMU/KVM, but I suppose that stopping
> and starting VM from within Proxmox will remove this block, however in
> our case keeping this virtual machine up and running is critical and we
> should avoid even 1-2 min downtime.
yes, shutting the VM down and starting it again gets rid of any leftover
block-jobs for sure.
> The question is how to remove this drive-mirror block online and how to
> avoid this in the future.
you should be able to manually clean the messup using the QMP/monitor
interface:
`man qemu-qmp-ref` gives a detailed tour, you probably want
`query-block-jobs` and `query-block`, and then, depending on the output
`block-job-cancel` or `block-job-complete`.
the HMP interface accessible via 'qm monitor <VMID>' has slightly
different commands: `info block -v`, `info block-jobs` and
`block_job_cancel`/`block_job_complete` ('_' instead of '-').
feel free to post the output of the query/info commands before deciding
how to proceed. the complete task log of the failed 'move disk'
operation would also be interesting, if it is still available.
_______________________________________________
pve-user mailing list
pve-user@pve.proxmox.com
https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-user
