Re: mellanox mlx4_core and SR-IOV

2012-11-26 Thread Albert Strasheim
Hello TLDR: get firmware 2.9.1200 and wait for 3.7. On Tue, Nov 27, 2012 at 4:30 AM, Jon Stanley wrote: > On Mon, Nov 26, 2012 at 8:05 PM, Zhangxiaofeng > wrote: > >> Hello, >> I tried to use SR-IOV virtualization for Mellanox ConnectX2 card with kernel >> 3.6.7. > > I'm not 100% sure, but I d

ibv_modify_qp to IBV_QPS_ERR returns EAGAIN

2012-08-28 Thread Albert Strasheim
Hello all I was wondering if anyone could shed some light about what kind conditions might cause ibv_modify_qp to IBV_QPS_ERR to return EAGAIN? The error occurred on a QP that probably already had some work completions for requests that failed. I've only seen it happen once in about 3 months, so

Re: memory region limit at 32 GB?

2012-08-02 Thread Albert Strasheim
Hello On Fri, Jan 20, 2012 at 8:00 PM, Roland Dreier wrote: > On Fri, Jan 20, 2012 at 9:39 AM, Albert Strasheim wrote: >> Is there some kind of limit that would prevent me from registering >> more than 32 GiB worth of memory regions with ibv_reg_mr in >> libibverbs? &

FDR HCA only doing QDR?

2012-08-02 Thread Albert Strasheim
Hello all I have a single machine setup with a dual port Mellanox FDR controller with a cable marked "FDR" connected between the two ports. OpenSM is running on the machine to make the ports active. However, as far as I can read the ibv_devinfo output, it seems to report that I'm only getting QD

Re: Work completion error: "transport retry counter exceeded"

2012-07-27 Thread Albert Strasheim
Hello On Fri, Jul 27, 2012 at 6:50 PM, Paul Grun wrote: > Ira, are you by any chance sending immediate data with your RDMA Write? Out of curiosity, what would be the significance if the answer to this question was yes? Regards Albert -- To unsubscribe from this list: send the line "unsubscribe

Re: Work completion error: "transport retry counter exceeded"

2012-07-26 Thread Albert Strasheim
Hello On Thu, Jul 26, 2012 at 9:15 AM, Roland Dreier wrote: > On Wed, Jul 25, 2012 at 7:07 PM, Ira Weiny wrote: >> attr.timeout = 14; > Is this timeout sufficient to account for the round trip on > the fabric and the ack delay on the remote HCA? > I don't think there are any othe

Re: IPoIB and RDMA CM

2012-06-15 Thread Albert Strasheim
Hello On Fri, Jun 15, 2012 at 3:56 PM, Yann Droneaud wrote: >> Hello all >> Can anyone recommend something I can read to find out more about how >> IPoIB bonding and librdmacm work together? > For status, you can have a look to "RDMACM APM Support" presentation > by Liran Liss, Mellanox Technolog

IPoIB and RDMA CM

2012-06-15 Thread Albert Strasheim
Hello all Can anyone recommend something I can read to find out more about how IPoIB bonding and librdmacm work together? Some googling seems to allude to the fact that patches in this area went in a few years ago, but I'd like to find some detailed information about how it works. I would also l

Re: kernel BUG at drivers/iommu/intel-iommu.c:1767 on F16 3.1.1-2.fc16.x86_64

2012-01-20 Thread Albert Strasheim
Hello On Fri, Jan 20, 2012 at 10:23 AM, Roland Dreier wrote: > On Thu, Jan 19, 2012 at 12:57 AM, Albert Strasheim wrote: >> Just checking up on this issue. Is there any further testing or >> information we can provide to help make a fix happen? > I'm not likely to be m

Re: memory region limit at 32 GB?

2012-01-20 Thread Albert Strasheim
Hello On Fri, Jan 20, 2012 at 8:03 PM, Roland Dreier wrote: > By the way, I wonder if we should auto-tune num_mtt so > we have enough MTTs to cover, say, 4X of the amount of > physical memory. > How much RAM do you have in your system? 192 GB. FYI, new Sandy Bridge motherboards will be out soon

memory region limit at 32 GB?

2012-01-20 Thread Albert Strasheim
Hello all Is there some kind of limit that would prevent me from registering more than 32 GiB worth of memory regions with ibv_reg_mr in libibverbs? >From strace I can see: open("/dev/infiniband/uverbs0", O_RDWR) = 8 ... write(8, "\t\0\0\0\f\0\3\0\340o\255\35\377\177\0\0\0P\211\336\26\177\0\0\0

Re: kernel BUG at drivers/iommu/intel-iommu.c:1767 on F16 3.1.1-2.fc16.x86_64

2012-01-19 Thread Albert Strasheim
Hello again On Tue, Dec 20, 2011 at 12:47 PM, Albert Strasheim wrote: > Any news on this one? > Regards > Albert > On Thu, Dec 8, 2011 at 8:31 PM, Albert Strasheim wrote: >> On Thu, Dec 8, 2011 at 8:29 PM, Roland Dreier wrote: >>> On Thu, Dec 8, 2011 at 9:56 AM, Albe

Re: kernel BUG at drivers/iommu/intel-iommu.c:1767 on F16 3.1.1-2.fc16.x86_64

2011-12-20 Thread Albert Strasheim
Any news on this one? Regards Albert On Thu, Dec 8, 2011 at 8:31 PM, Albert Strasheim wrote: > On Thu, Dec 8, 2011 at 8:29 PM, Roland Dreier wrote: >> On Thu, Dec 8, 2011 at 9:56 AM, Albert Strasheim wrote: >>> I think the BIOS has VT-d enabled. dmesg says: &g

Re: kernel BUG at drivers/iommu/intel-iommu.c:1767 on F16 3.1.1-2.fc16.x86_64

2011-12-08 Thread Albert Strasheim
Hello On Thu, Dec 8, 2011 at 8:29 PM, Roland Dreier wrote: > On Thu, Dec 8, 2011 at 9:56 AM, Albert Strasheim wrote: >> I think the BIOS has VT-d enabled. dmesg says: >> PCI-DMA: Intel(R) Virtualization Technology for Directed I/O > Yes, you're crashing in the VT-d co

Re: kernel BUG at drivers/iommu/intel-iommu.c:1767 on F16 3.1.1-2.fc16.x86_64

2011-12-08 Thread Albert Strasheim
Hello On Thu, Dec 8, 2011 at 7:47 PM, Roland Dreier wrote: >  > [  597.412843] kernel BUG at drivers/iommu/intel-iommu.c:1767! > So this is >        BUG_ON(addr_width < BITS_PER_LONG && (iov_pfn + nr_pages - 1) >>> addr_width); > > in __domain_mapping() I believe.  And we have: >        int addr_

kernel BUG at drivers/iommu/intel-iommu.c:1767 on F16 3.1.1-2.fc16.x86_64

2011-12-08 Thread Albert Strasheim
Hello all We've hit a kernel panic running 3.1.1-2.fc16.x86_64 on Fedora 16 when registering 144 buffers of 32 MB each. We're using libibverbs-1.1.5-5.fc16.x86_64. Any help would be appreciated. Would a firmware upgrade make a difference? ibv_devinfo output: hca_id: mlx4_0 transport: