Re: [PATCH v2] scsi-disk: Fix crash for VM configured with USB CDROM after live migration
On Mon, Jun 24, 2024 at 6:06 PM Thomas Huth wrote: > On 10/06/2024 19.02, Paolo Bonzini wrote: > > From: Hyman Huang > > > > For VMs configured with the USB CDROM device: > > > > -drive > file=/path/to/local/file,id=drive-usb-disk0,media=cdrom,readonly=on... > > -device usb-storage,drive=drive-usb-disk0,id=usb-disk0... > > > > QEMU process may crash after live migration, to reproduce the issue, > > configure VM (Guest OS ubuntu 20.04 or 21.10) with the following XML: > > > > > > > > > > > > > > > > > > > > > > Do the live migration repeatedly, crash may happen after live migratoin, > > trace log at the source before live migration is as follows: > > > > 324808@1711972823.521945:usb_uhci_frame_start nr 319 > > 324808@1711972823.521978:usb_uhci_qh_load qh 0x35cb5400 > > 324808@1711972823.521989:usb_uhci_qh_load qh 0x35cb5480 > > 324808@1711972823.521997:usb_uhci_td_load qh 0x35cb5480, td 0x35cbe000, > ctrl 0x0, token 0xffe07f69 > > 324808@1711972823.522010:usb_uhci_td_nextqh qh 0x35cb5480, td 0x35cbe000 > > 324808@1711972823.522022:usb_uhci_qh_load qh 0x35cb5680 > > 324808@1711972823.522030:usb_uhci_td_load qh 0x35cb5680, td 0x75ac5180, > ctrl 0x1980, token 0x3c903e1 > > 324808@1711972823.522045:usb_uhci_packet_add token 0x103e1, td > 0x75ac5180 > > 324808@1711972823.522056:usb_packet_state_change bus 0, port 2, ep 2, > packet 0x559f9ba14b00, state undef -> setup > > 324808@1711972823.522079:usb_msd_cmd_submit lun 0, tag 0x472, flags > 0x0080, len 10, data-len 8 > > 324808@1711972823.522107:scsi_req_parsed target 0 lun 0 tag 1138 > command 74 dir 1 length 8 > > 324808@1711972823.522124:scsi_req_parsed_lba target 0 lun 0 tag 1138 > command 74 lba 4096 > > 324808@1711972823.522139:scsi_req_alloc target 0 lun 0 tag 1138 > > 324808@1711972823.522169:scsi_req_continue target 0 lun 0 tag 1138 > > 324808@1711972823.522181:scsi_req_data target 0 lun 0 tag 1138 len 8 > > 324808@1711972823.522194:usb_packet_state_change bus 0, port 2, ep 2, > packet 0x559f9ba14b00, state setup -> complete > > 324808@1711972823.522209:usb_uhci_packet_complete_success token > 0x103e1, td 0x75ac5180 > > 324808@1711972823.522219:usb_uhci_packet_del token 0x103e1, td > 0x75ac5180 > > 324808@1711972823.522232:usb_uhci_td_complete qh 0x35cb5680, td > 0x75ac5180 > > > > trace log at the destination after live migration is as follows: > > > > 3286206@1711972823.951646:usb_uhci_frame_start nr 320 > > 3286206@1711972823.951663:usb_uhci_qh_load qh 0x35cb5100 > > 3286206@1711972823.951671:usb_uhci_qh_load qh 0x35cb5480 > > 3286206@1711972823.951680:usb_uhci_td_load qh 0x35cb5480, td > 0x35cbe000, ctrl 0x100, token 0xffe07f69 > > 3286206@1711972823.951693:usb_uhci_td_nextqh qh 0x35cb5480, td > 0x35cbe000 > > 3286206@1711972823.951702:usb_uhci_qh_load qh 0x35cb5700 > > 3286206@1711972823.951709:usb_uhci_td_load qh 0x35cb5700, td > 0x75ac5240, ctrl 0x3980, token 0xe08369 > > 3286206@1711972823.951727:usb_uhci_queue_add token 0x8369 > > 3286206@1711972823.951735:usb_uhci_packet_add token 0x8369, td > 0x75ac5240 > > 3286206@1711972823.951746:usb_packet_state_change bus 0, port 2, ep 1, > packet 0x56066b2fb5a0, state undef -> setup > > 3286206@1711972823.951766:usb_msd_data_in 8/8 (scsi 8) > > 2024-04-01 12:00:24.665+: shutting down, reason=crashed > > > > The backtrace reveals the following: > > > > Program terminated with signal SIGSEGV, Segmentation fault. > > 0 __memmove_sse2_unaligned_erms () at > ../sysdeps/x86_64/multiarch/memmove-vec-unaligned-erms.S:312 > > 312movq-8(%rsi,%rdx), %rcx > > [Current thread is 1 (Thread 0x7f0a9025fc00 (LWP 3286206))] > > (gdb) bt > > 0 __memmove_sse2_unaligned_erms () at > ../sysdeps/x86_64/multiarch/memmove-vec-unaligned-erms.S:312 > > 1 memcpy (__len=8, __src=, __dest=) at > /usr/include/bits/string_fortified.h:34 > > 2 iov_from_buf_full (iov=, iov_cnt=, > offset=, buf=0x0, bytes=bytes@entry=8) at ../util/iov.c:33 > > 3 iov_from_buf (bytes=8, buf=, offset=, > iov_cnt=, iov=) > > at > /usr/src/debug/qemu-6-6.2.0-75.7.oe1.smartx.git.40.x86_64/include/qemu/iov.h:49 > > 4 usb_packet_copy (p=p@entry=0x56066b2fb5a0, ptr=, > bytes=bytes@entry=8) at ../hw/usb/core.c:636 > > 5 usb_msd_copy_data (s=s@entry=0x56066c62c770, p=p@entry=0x56066b2fb5a0) > at ../hw/usb/dev-storage.c:186 > > 6 usb_msd_handle_data (dev=0x56066c62c770, p=0x56066b2fb5a0) at > ../hw/usb/dev-storage.c:496 > > 7 usb_handle_packet (dev=0x56066c62c770, p=p@entry=0x56066b2fb5a0) at > ../hw/usb/core.c:455 > > 8 uhci_handle_td (s=s@entry=0x56066bd5f210, q=0x56066bb7fbd0, q@entry=0x0, > qh_addr=qh_addr@entry=902518530, td=td@entry=0x7fffe6e788f0, > td_addr=, > > int_mask=int_mask@entry=0x7fffe6e788e4) at ../hw/usb/hcd-uhci.c:885 > > 9 uhci_process_frame (s=s@entry=0x56066bd5f210) at > ../hw/usb/hcd-uhci.c:1061 > > 10 uhci_frame_timer (opaque=opaque@entry=0x56066bd5f210) at > ../hw/usb/hcd-uhci.c:1159 > > 11 timerlist_run_timers (timer_list=0x56066af26bd0) at > ../util/qemu-timer.c:64
Re: [PATCH v2] scsi-disk: Fix crash for VM configured with USB CDROM after live migration
On 10/06/2024 19.02, Paolo Bonzini wrote:
From: Hyman Huang
For VMs configured with the USB CDROM device:
-drive file=/path/to/local/file,id=drive-usb-disk0,media=cdrom,readonly=on...
-device usb-storage,drive=drive-usb-disk0,id=usb-disk0...
QEMU process may crash after live migration, to reproduce the issue,
configure VM (Guest OS ubuntu 20.04 or 21.10) with the following XML:
Do the live migration repeatedly, crash may happen after live migratoin,
trace log at the source before live migration is as follows:
324808@1711972823.521945:usb_uhci_frame_start nr 319
324808@1711972823.521978:usb_uhci_qh_load qh 0x35cb5400
324808@1711972823.521989:usb_uhci_qh_load qh 0x35cb5480
324808@1711972823.521997:usb_uhci_td_load qh 0x35cb5480, td 0x35cbe000, ctrl
0x0, token 0xffe07f69
324808@1711972823.522010:usb_uhci_td_nextqh qh 0x35cb5480, td 0x35cbe000
324808@1711972823.522022:usb_uhci_qh_load qh 0x35cb5680
324808@1711972823.522030:usb_uhci_td_load qh 0x35cb5680, td 0x75ac5180, ctrl
0x1980, token 0x3c903e1
324808@1711972823.522045:usb_uhci_packet_add token 0x103e1, td 0x75ac5180
324808@1711972823.522056:usb_packet_state_change bus 0, port 2, ep 2, packet
0x559f9ba14b00, state undef -> setup
324808@1711972823.522079:usb_msd_cmd_submit lun 0, tag 0x472, flags 0x0080,
len 10, data-len 8
324808@1711972823.522107:scsi_req_parsed target 0 lun 0 tag 1138 command 74 dir
1 length 8
324808@1711972823.522124:scsi_req_parsed_lba target 0 lun 0 tag 1138 command 74
lba 4096
324808@1711972823.522139:scsi_req_alloc target 0 lun 0 tag 1138
324808@1711972823.522169:scsi_req_continue target 0 lun 0 tag 1138
324808@1711972823.522181:scsi_req_data target 0 lun 0 tag 1138 len 8
324808@1711972823.522194:usb_packet_state_change bus 0, port 2, ep 2, packet
0x559f9ba14b00, state setup -> complete
324808@1711972823.522209:usb_uhci_packet_complete_success token 0x103e1, td
0x75ac5180
324808@1711972823.522219:usb_uhci_packet_del token 0x103e1, td 0x75ac5180
324808@1711972823.522232:usb_uhci_td_complete qh 0x35cb5680, td 0x75ac5180
trace log at the destination after live migration is as follows:
3286206@1711972823.951646:usb_uhci_frame_start nr 320
3286206@1711972823.951663:usb_uhci_qh_load qh 0x35cb5100
3286206@1711972823.951671:usb_uhci_qh_load qh 0x35cb5480
3286206@1711972823.951680:usb_uhci_td_load qh 0x35cb5480, td 0x35cbe000, ctrl
0x100, token 0xffe07f69
3286206@1711972823.951693:usb_uhci_td_nextqh qh 0x35cb5480, td 0x35cbe000
3286206@1711972823.951702:usb_uhci_qh_load qh 0x35cb5700
3286206@1711972823.951709:usb_uhci_td_load qh 0x35cb5700, td 0x75ac5240, ctrl
0x3980, token 0xe08369
3286206@1711972823.951727:usb_uhci_queue_add token 0x8369
3286206@1711972823.951735:usb_uhci_packet_add token 0x8369, td 0x75ac5240
3286206@1711972823.951746:usb_packet_state_change bus 0, port 2, ep 1, packet
0x56066b2fb5a0, state undef -> setup
3286206@1711972823.951766:usb_msd_data_in 8/8 (scsi 8)
2024-04-01 12:00:24.665+: shutting down, reason=crashed
The backtrace reveals the following:
Program terminated with signal SIGSEGV, Segmentation fault.
0 __memmove_sse2_unaligned_erms () at
../sysdeps/x86_64/multiarch/memmove-vec-unaligned-erms.S:312
312movq-8(%rsi,%rdx), %rcx
[Current thread is 1 (Thread 0x7f0a9025fc00 (LWP 3286206))]
(gdb) bt
0 __memmove_sse2_unaligned_erms () at
../sysdeps/x86_64/multiarch/memmove-vec-unaligned-erms.S:312
1 memcpy (__len=8, __src=, __dest=) at
/usr/include/bits/string_fortified.h:34
2 iov_from_buf_full (iov=, iov_cnt=,
offset=, buf=0x0, bytes=bytes@entry=8) at ../util/iov.c:33
3 iov_from_buf (bytes=8, buf=, offset=,
iov_cnt=, iov=)
at
/usr/src/debug/qemu-6-6.2.0-75.7.oe1.smartx.git.40.x86_64/include/qemu/iov.h:49
4 usb_packet_copy (p=p@entry=0x56066b2fb5a0, ptr=,
bytes=bytes@entry=8) at ../hw/usb/core.c:636
5 usb_msd_copy_data (s=s@entry=0x56066c62c770, p=p@entry=0x56066b2fb5a0) at
../hw/usb/dev-storage.c:186
6 usb_msd_handle_data (dev=0x56066c62c770, p=0x56066b2fb5a0) at
../hw/usb/dev-storage.c:496
7 usb_handle_packet (dev=0x56066c62c770, p=p@entry=0x56066b2fb5a0) at
../hw/usb/core.c:455
8 uhci_handle_td (s=s@entry=0x56066bd5f210, q=0x56066bb7fbd0, q@entry=0x0,
qh_addr=qh_addr@entry=902518530, td=td@entry=0x7fffe6e788f0, td_addr=,
int_mask=int_mask@entry=0x7fffe6e788e4) at ../hw/usb/hcd-uhci.c:885
9 uhci_process_frame (s=s@entry=0x56066bd5f210) at ../hw/usb/hcd-uhci.c:1061
10 uhci_frame_timer (opaque=opaque@entry=0x56066bd5f210) at
../hw/usb/hcd-uhci.c:1159
11 timerlist_run_timers (timer_list=0x56066af26bd0) at ../util/qemu-timer.c:642
12 qemu_clock_run_timers (type=QEMU_CLOCK_VIRTUAL) at ../util/qemu-timer.c:656
13 qemu_clock_run_all_timers () at ../util/qemu-timer.c:738
14 main_loop_wait (nonblocking=nonblocking@entry=0) at ../util/main-loop.c:542
15 qemu_main_loop () at ../softmmu/runstate.c:739
16 main (argc=, argv=, envp=) at
../softmmu/main.c:52
(gdb) frame 5
(gdb) p ((SCSIDiskReq *)s->req)->iov
$1 = {iov_base = 0x0
