Right now, RDS follows each RDMA write op with a Send op, which 1)
causes an interrupt and 2) includes the info we need to call
ib_dma_sync_sg_for_cpu() for the target of the rdma write.
We want to omit the Send. If we don't do the sync on the machine that is
the target of the RDMA write, the resu
On Tue, Feb 16, 2010 at 03:29:48PM -0800, Andy Grover wrote:
> Right now, RDS follows each RDMA write op with a Send op, which 1)
> causes an interrupt and 2) includes the info we need to call
> ib_dma_sync_sg_for_cpu() for the target of the rdma write.
>
> We want to omit the Send. If we don't do
er
Cc: linux-rdma@vger.kernel.org
Subject: Re: is it possible to avoid syncing after an rdma write?
On Tue, Feb 16, 2010 at 03:29:48PM -0800, Andy Grover wrote:
> Right now, RDS follows each RDMA write op with a Send op, which 1)
> causes an interrupt and 2) includes the info we n
On Tue, Feb 16, 2010 at 05:05:21PM -0800, Paul Grun wrote:
> Why not use an RDMA write w/ immed? That forces the consumption of a
> receive WQE and can be used to create a completion event. Since the
> immediate data is carried in the last packet of a multi-packet RDMA write,
> you are guaranteed
010 5:12 PM
To: Paul Grun
Cc: 'Andy Grover'; linux-rdma@vger.kernel.org
Subject: Re: is it possible to avoid syncing after an rdma write?
On Tue, Feb 16, 2010 at 05:05:21PM -0800, Paul Grun wrote:
> Why not use an RDMA write w/ immed? That forces the consumption of a
> receive W
Andy Grover wrote:
RDS follows each RDMA write op with a Send op [...] we want to omit the Send
Andy,
This way or another the side which isn't initiating the rdma write has
to be notified that the local buffer && rkey (stag) they advertised can
now invalidated from the HCA/RNIC IOMMU, its ma
On Tue, Feb 16, 2010 at 10:40:45PM -0800, Paul Grun wrote:
> Two advantages come to mind vs an RDMA Write followed by a SEND:
> Using a SEND will consume a second WQE on the send side, and the
> synchronizing SEND will cause an entire new transaction, which will
> consume a(n infinitesimally) smal
Jason Gunthorpe wrote:
> On Tue, Feb 16, 2010 at 03:29:48PM -0800, Andy Grover wrote:
>> Right now, RDS follows each RDMA write op with a Send op, which 1)
>> causes an interrupt and 2) includes the info we need to call
>> ib_dma_sync_sg_for_cpu() for the target of the rdma write.
>>
>> We want to
On Wed, Feb 17, 2010 at 11:54:44AM -0800, Andy Grover wrote:
> > What do you intend to replace the SEND with? spin on last byte? There
> > are other issues to consider like ordering within the PCI-E fabric..
>
> Well, hopefully nothing. What I'm looking for is to write to a target
> region multip
