On Tue, Nov 08, 2016 at 09:22:25PM +0000, Brian Candler wrote: > On 07/11/2016 10:42, Stefan Hajnoczi wrote: > > On Mon, Nov 07, 2016 at 08:42:17AM +0000, Brian Candler wrote: > > > >On 06/11/2016 18:04, Samuel Thibault wrote: > > > > > >Brian, could you run it with > > > > > > > > > > > >export MALLOC_CHECK_=2 > > > > > > > > > > > >and also this could be useful: > > > > > > > > > > > >export MALLOC_PERTURB_=1234 > > > > > > > > > > > >Also, to rule out the double-free scenario, and try to catch a buffer > > > > > >overflow coming from the socket structure itself, I have attached a > > > > > >patch which adds some debugging. > > > > > > > >Thanks. I've added the patch, and re-run the stress test. > > Back to the original setup, I can still get dumps. I notice I'm now getting > "malloc_printerr" in the backtrace, but unfortunately I don't get to see the > actual error message. It would seem that the malloc_check is being done and > finding an issue. I haven't been able to get one in tcp_close again though > :-(
Heap corruption. Valgrind's memcheck tool could be fruitful here: http://valgrind.org/docs/manual/quick-start.html#quick-start.mcrun Stefan > Regards, > > Brian. > > [Thread debugging using libthread_db enabled] > Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1". > Core was generated by `/usr/local/bin/qemu-system-x86_64 -m 4G -machine > type=pc,accel=kvm -device virt'. > Program terminated with signal SIGABRT, Aborted. > #0 0x00007eff4f3df428 in __GI_raise (sig=sig@entry=6) at > ../sysdeps/unix/sysv/linux/raise.c:54 > 54 ../sysdeps/unix/sysv/linux/raise.c: No such file or directory. > [Current thread is 1 (Thread 0x7eff50dffa80 (LWP 13616))] > (gdb) bt > #0 0x00007eff4f3df428 in __GI_raise (sig=sig@entry=6) at > ../sysdeps/unix/sysv/linux/raise.c:54 > #1 0x00007eff4f3e102a in __GI_abort () at abort.c:89 > #2 0x00007eff4f42bc1f in malloc_printerr (ar_ptr=<optimised out>, > ptr=<optimised out>, str=<optimised out>, > action=<optimised out>) at malloc.c:5008 > #3 _int_malloc (av=av@entry=0x7eff4f76db20 <main_arena>, > bytes=bytes@entry=89) at malloc.c:3384 > #4 0x00007eff4f42c409 in malloc_check (sz=88, caller=<optimised out>) at > hooks.c:295 > #5 0x00007eff50106729 in g_malloc () from > /lib/x86_64-linux-gnu/libglib-2.0.so.0 > #6 0x0000563ca16930cf in qemu_aio_get > (aiocb_info=aiocb_info@entry=0x563ca1841190 <blk_aio_em_aiocb_info>, > bs=0x563ca4132f40, cb=cb@entry=0x563ca14d85e0 <dma_blk_cb>, > opaque=opaque@entry=0x563ca5b78910) > at /home/nsrc/qemu-2.7.0/block/io.c:2231 > #7 0x0000563ca1687aa8 in blk_aio_get (opaque=0x563ca5b78910, > cb=0x563ca14d85e0 <dma_blk_cb>, blk=0x563ca4132d70, > aiocb_info=0x563ca1841190 <blk_aio_em_aiocb_info>) at > /home/nsrc/qemu-2.7.0/block/block-backend.c:1477 > #8 blk_aio_prwv (blk=0x563ca4132d70, offset=5278244864, bytes=4096, > qiov=0x563ca5b78968, > co_entry=co_entry@entry=0x563ca16872b0 <blk_aio_write_entry>, flags=0, > cb=0x563ca14d85e0 <dma_blk_cb>, > opaque=0x563ca5b78910) at > /home/nsrc/qemu-2.7.0/block/block-backend.c:941 > #9 0x0000563ca1687bc0 in blk_aio_pwritev (blk=<optimised out>, > offset=<optimised out>, qiov=<optimised out>, > flags=<optimised out>, cb=<optimised out>, opaque=<optimised out>) > at /home/nsrc/qemu-2.7.0/block/block-backend.c:1054 > #10 0x0000563ca14d8718 in dma_blk_cb (opaque=0x563ca5b78910, ret=<optimised > out>) > at /home/nsrc/qemu-2.7.0/dma-helpers.c:167 > #11 0x0000563ca14d8bf8 in dma_blk_io (ctx=0x563ca41184a0, > sg=sg@entry=0x563ca59a08f0, > offset=offset@entry=5278244864, io_func=io_func@entry=0x563ca15a58e0 > <scsi_dma_writev>, > io_func_opaque=io_func_opaque@entry=0x563ca5c8a350, > cb=cb@entry=0x563ca15a7250 <scsi_dma_complete>, > opaque=0x563ca5c8a350, dir=DMA_DIRECTION_TO_DEVICE) at > /home/nsrc/qemu-2.7.0/dma-helpers.c:222 > #12 0x0000563ca15a764e in scsi_write_data (req=0x563ca5c8a350) at > /home/nsrc/qemu-2.7.0/hw/scsi/scsi-disk.c:540 > #13 0x0000563ca15ac743 in scsi_req_continue (req=req@entry=0x563ca5c8a350) > at /home/nsrc/qemu-2.7.0/hw/scsi/scsi-bus.c:1680 > #14 0x0000563ca14381a2 in virtio_scsi_handle_cmd_req_submit > (s=0x563ca5abc1d0, req=<optimised out>) > at /home/nsrc/qemu-2.7.0/hw/scsi/virtio-scsi.c:565 > #15 virtio_scsi_handle_cmd_vq (s=0x563ca5abc1d0, vq=0x7eff4963f110) > at /home/nsrc/qemu-2.7.0/hw/scsi/virtio-scsi.c:583 > #16 0x0000563ca144a0d6 in virtio_queue_notify_vq (vq=0x7eff4963f110) > ---Type <return> to continue, or q <return> to quit--- > at /home/nsrc/qemu-2.7.0/hw/virtio/virtio.c:1113 > #17 0x0000563ca1654965 in aio_dispatch (ctx=0x563ca41184a0) at > /home/nsrc/qemu-2.7.0/aio-posix.c:330 > #18 0x0000563ca164a3ae in aio_ctx_dispatch (source=<optimised out>, > callback=<optimised out>, > user_data=<optimised out>) at /home/nsrc/qemu-2.7.0/async.c:234 > #19 0x00007eff501011a7 in g_main_context_dispatch () from > /lib/x86_64-linux-gnu/libglib-2.0.so.0 > #20 0x0000563ca16531db in glib_pollfds_poll () at > /home/nsrc/qemu-2.7.0/main-loop.c:213 > #21 os_host_main_loop_wait (timeout=<optimised out>) at > /home/nsrc/qemu-2.7.0/main-loop.c:258 > #22 main_loop_wait (nonblocking=<optimised out>) at > /home/nsrc/qemu-2.7.0/main-loop.c:506 > #23 0x0000563ca13be431 in main_loop () at /home/nsrc/qemu-2.7.0/vl.c:1908 > #24 main (argc=<optimised out>, argv=<optimised out>, envp=<optimised out>) > at /home/nsrc/qemu-2.7.0/vl.c:4604 > (gdb) >
signature.asc
Description: PGP signature
