Ack. vmovdqa64 is an aligned move [google it...] so it's no surprise whatsoever that it blew up. If you check the new/improved assembly code, you'll probably see that the compiler generated a 'u' flavor [unaligned] vector move.
Thanks... Dave -----Original Message----- From: vpp-dev@lists.fd.io <vpp-dev@lists.fd.io> On Behalf Of Elias Rudberg Sent: Wednesday, May 6, 2020 7:56 PM To: dmar...@me.com Cc: Dave Barach (dbarach) <dbar...@cisco.com>; vpp-dev@lists.fd.io Subject: Re: [vpp-dev] Segmentation fault in rdma_device_input_refill when using clang compiler Hi Dave and Damjan, Here is instruction and register info: (gdb) x/i $pc => 0x7fffabbbdd67 <rdma_device_input_refill+759>: vmovdqa64 -0x30a0(%rbp),%ymm0 (gdb) info registers rbp ymm0 rbp 0x7ffff417daf0 0x7ffff417daf0 ymm0 {v8_float = {0x0, 0x0, 0x0, 0x3, 0x0, 0x0, 0x0, 0xfffffffd}, v4_double = {0x0, 0x37, 0x0, 0xffffffffffffff85}, v32_int8 = {0x0, 0x0, 0x0, 0x10, 0x3f, 0xf6, 0x41, 0x80, 0x0, 0x0, 0x0, 0x10, 0x3f, 0xf6, 0x4b, 0x40, 0x0, 0x0, 0x0, 0x10, 0x3f, 0xf6, 0x55, 0x0, 0x0, 0x0, 0x0, 0x10, 0x3f, 0xf6, 0x5e, 0xc0}, v16_int16 = {0x0, 0x1000, 0xf63f, 0x8041, 0x0, 0x1000, 0xf63f, 0x404b, 0x0, 0x1000, 0xf63f, 0x55, 0x0, 0x1000, 0xf63f, 0xc05e}, v8_int32 = { 0x10000000, 0x8041f63f, 0x10000000, 0x404bf63f, 0x10000000, 0x55f63f, 0x10000000, 0xc05ef63f}, v4_int64 = {0x8041f63f10000000, 0x404bf63f10000000, 0x55f63f10000000, 0xc05ef63f10000000}, v2_int128 = {0x404bf63f100000008041f63f10000000, 0xc05ef63f100000000055f63f10000000}} Not sure if I understand all this but perhaps it means that the value in %rbp is used as a memory address, but that address 0x7ffff417daf0 is not 32-byte aligned as it needs to be. Adding __attribute__((aligned(32))) as Damjan suggests indeed seems to help. After that there was again a segfault in another place in the same file, where the same trick of adding __attribute__((aligned(32))) again helped. So it seems the problem can be fixed by adding that alignment attribute in two places, like this: diff --git a/src/plugins/rdma/input.c b/src/plugins/rdma/input.c index cf0b6bffe..324436f01 100644 --- a/src/plugins/rdma/input.c +++ b/src/plugins/rdma/input.c @@ -103,7 +103,7 @@ rdma_device_input_refill (vlib_main_t * vm, rdma_device_t * rd, if (is_mlx5dv) { - u64 va[8]; + u64 va[8] __attribute__((aligned(32))); mlx5dv_rwq_t *wqe = rxq->wqes + slot; while (n >= 1) @@ -488,7 +488,7 @@ rdma_device_input_inline (vlib_main_t * vm, vlib_node_runtime_t * node, rdma_rxq_t *rxq = vec_elt_at_index (rd->rxqs, qid); vlib_buffer_t *bufs[VLIB_FRAME_SIZE], **b = bufs; struct ibv_wc wc[VLIB_FRAME_SIZE]; - u32 byte_cnts[VLIB_FRAME_SIZE]; + u32 byte_cnts[VLIB_FRAME_SIZE] __attribute__((aligned(32))); vlib_buffer_t bt; u32 next_index, *to_next, n_left_to_next, n_rx_bytes = 0; int n_rx_packets, skip_ip4_cksum = 0; Many thanks for you help! Should I push the above as a patch to gerrit? / Elias On Wed, 2020-05-06 at 20:38 +0200, Damjan Marion wrote: > Can you try this: > > diff --git a/src/plugins/rdma/input.c b/src/plugins/rdma/input.c index > cf0b6bffe..b461ee27b 100644 > --- a/src/plugins/rdma/input.c > +++ b/src/plugins/rdma/input.c > @@ -103,7 +103,7 @@ rdma_device_input_refill (vlib_main_t * vm, > rdma_device_t * rd, > > if (is_mlx5dv) > { > - u64 va[8]; > + u64 va[8] __attribute__((aligned(32))); > mlx5dv_rwq_t *wqe = rxq->wqes + slot; > > while (n >= 1) > > > Thanks! > > > On 6 May 2020, at 19:45, Elias Rudberg <elias.rudb...@bahnhof.net> > > wrote: > > > > Hello VPP experts, > > > > When trying to use the current master branch, we get a segmentation > > fault error. Here is what it looks like in gdb: > > > > Thread 3 "vpp_wk_0" received signal SIGSEGV, Segmentation fault. > > [Switching to Thread 0x7fedf91fe700 (LWP 21309)] > > rdma_device_input_refill (vm=0x7ff8a5d2f4c0, rd=0x7fedd35ed5c0, > > rxq=0x7ffff7edea80, is_mlx5dv=1) > > at vpp/src/plugins/rdma/input.c:115 > > 115 *(u64x4 *) (va + 4) = u64x4_byte_swap (*(u64x4 *) (va > > + 4));
-=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#16269): https://lists.fd.io/g/vpp-dev/message/16269 Mute This Topic: https://lists.fd.io/mt/74033970/21656 Group Owner: vpp-dev+ow...@lists.fd.io Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-