Re: using same IP subnet on multiple interfaces

2010-08-17 Thread Or Gerlitz
Jason Gunthorpe wrote: As for the original issue we were discussing here, the conclusion is that with upstream 2.6.35 bits for the rdma connection to go from hca1 port1 to hca1 port2 (or from hca1 port1 to hca2 port1), the rdma-cm needs a neighbour, similarly to a ping -I ib0 to ib1 address. A

Re: using same IP subnet on multiple interfaces

2010-08-16 Thread Jason Gunthorpe
On Mon, Aug 16, 2010 at 06:30:04PM +0300, Or Gerlitz wrote: > As for the original issue we were discussing here, Sean - the > conclusion is that with upstream 2.6.35 bits for the rdma connection > to go from hca1 port1 to hca1 port2 (or from hca1 port1 to hca2 > port1), the rdma-cm needs a neighbo

Re: using same IP subnet on multiple interfaces

2010-08-16 Thread Or Gerlitz
Jason Gunthorpe wrote: > [...] The socket that is bound to a device will then use its device for > sending, > but other sockets not bound to devices will do route lookups and use the lo > device. > Do: [...] To see the difference in each side. sure, makes sense, the ping-reply code does route l

Re: using same IP subnet on multiple interfaces (was: dual HCAs with upstream kernel)

2010-08-15 Thread Jason Gunthorpe
On Sun, Aug 15, 2010 at 10:50:17AM +0300, Or Gerlitz wrote: > the echo requests go on the wire, the replies not, probably (...) > internally, Patrick? What all these settings do is let a socket that is bound to a device resolve the local host's address through ARP. The socket that is bound to a d

Re: using same IP subnet on multiple interfaces (was: dual HCAs with upstream kernel)

2010-08-15 Thread Or Gerlitz
Hefty, Sean wrote: > Does anyone have a system with multiple HCAs that's running a recent upstream > kernel? > Oracle has reported a bug connecting between two HCAs in the same system > using the rdma_cm Sean, With 2.6.35, I was hitting the reported failure (address error event, status -ETIME