Re: [openib-general] problem with 2.6.19?
> By the way, by userspace DMA do you mean DMA to userspace buffers, or > DMA initiated by userspace? I'm assuming the former, but if it's the > latter, how are the addresses returned from dma_map_sg communicated to > userspace? Actually, both. Userspace deals in virtual addresses (it asks the kernel to register memory in advance, and the kernel loads the IB device with the virtual -> bus mapping). - R. ___ openib-general mailing list openib-general@openib.org http://openib.org/mailman/listinfo/openib-general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
Re: [openib-general] problem with 2.6.19?
On Sat, Oct 28, 2006 at 01:10:54PM -0700, Roland Dreier wrote: > I guess the requirement for userspace RDMA to work is that no > further action is required after the dma_map_sg() for both the > device and the CPU to touch the region. Because userspace has no > way for calling dma_sync_xxx or anything like that. Ok, if there's no requirement on the address (i.e., the DMA address does not need to be the same as the machine physical address) then Calgary should be fine. By the way, by userspace DMA do you mean DMA to userspace buffers, or DMA initiated by userspace? I'm assuming the former, but if it's the latter, how are the addresses returned from dma_map_sg communicated to userspace? Cheers, Muli ___ openib-general mailing list openib-general@openib.org http://openib.org/mailman/listinfo/openib-general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
Re: [openib-general] problem with 2.6.19?
> You can also use the sync_{single|sg}_for_{device|cpu} calls to manually sync
> the buffers (this will cause a memcpy for swiotlb).
True, although in this case it's much simpler just to use coherent
memory. Also, the dma_sync_xxx calls don't _always_ work -- for
example if both the CPU and the device need to touch different parts
of a cacheline at the same time on a non-coherent architecture.
> Could you elaborate, what are the "requirements" for this to work with
> an arbitrary IOMMU (e.g., Calgary) than requires a mapping to exist
> before a DMA operation can go through?
Calgary should be fine, since pseries works Ok. I guess the
requirement for userspace RDMA to work is that no further action is
required after the dma_map_sg() for both the device and the CPU to
touch the region. Because userspace has no way for calling
dma_sync_xxx or anything like that.
- R.
___
openib-general mailing list
openib-general@openib.org
http://openib.org/mailman/listinfo/openib-general
To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
Re: [openib-general] problem with 2.6.19?
On Fri, Oct 27, 2006 at 11:15:40AM -0700, Roland Dreier wrote:
> > I must be misusing dma_map_single(). What I'm doing is allocating a
> > verb message reply queue for the adapter to DMA verb replies into. It
> > never gets unmapped. I kmalloc() it, then map it. I could use
> > dma_alloc_coherent() or something, and maybe that's what I need to do?
>
> Yeah, if you want to leave something mapped and have the device DMA
> into it, and the CPU look into the buffer too, then you need
> consistent/coherent memory -- either pci_alloc_consistent() or
> dma_alloc_coherent(). The dma_ variant is slightly better because you
> can pass in a GFP_ mask rather than having the kernel pick GFP_ATOMIC
> for you.
You can also use the sync_{single|sg}_for_{device|cpu} calls to manually sync
the buffers (this will cause a memcpy for swiotlb).
> > You're saying I must unmap it before the data is valid (cuz of the
> > bounce buffering).
No, sync_xxx should work as well to make the data valid.
> > If that's true, then how in sam hill does user mode
> > RDMA work since the user's memory isn't unmapped before the user looks
> > like memory that is the target of RDMA The uverbs code calls
> > dma_map_sg() which is roughly the same as dma_map_single, eh?
>
> It's a good point. We're kind of counting on the IOMMU situation not
> being too wacky,
Could you elaborate, what are the "requirements" for this to work with
an arbitrary IOMMU (e.g., Calgary) than requires a mapping to exist
before a DMA operation can go through?
Cheers,
Muli
___
openib-general mailing list
openib-general@openib.org
http://openib.org/mailman/listinfo/openib-general
To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
Re: [openib-general] problem with 2.6.19?
> But I guess I should still change this all to use > dma_alloc_coherent()... Yes, please fix at least this part the right way (even if we can't do anything about userspace). > Roland, Can we get this fix into 2.6.19? Yes, there's plenty of time before 2.6.19-final, and this is clearly a bugfix. - R. ___ openib-general mailing list openib-general@openib.org http://openib.org/mailman/listinfo/openib-general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
Re: [openib-general] problem with 2.6.19?
I found the bug. The ammasso driver was passing in the pointer c2_dev->ibdev.dma_device to dma_map_single(). However the driver hadn't yet registered with the ib core, so that field is zero. The dma map code uses bounce buffers if a map is done with a NULL device ptr passed in... The simple fix is to pass in the correct pointer, &c2dev->pcidev->dev. But I guess I should still change this all to use dma_alloc_coherent()... Roland, Can we get this fix into 2.6.19? Steve. On Fri, 2006-10-27 at 11:15 -0700, Roland Dreier wrote: > > I must be misusing dma_map_single(). What I'm doing is allocating a > > verb message reply queue for the adapter to DMA verb replies into. It > > never gets unmapped. I kmalloc() it, then map it. I could use > > dma_alloc_coherent() or something, and maybe that's what I need to do? > > Yeah, if you want to leave something mapped and have the device DMA > into it, and the CPU look into the buffer too, then you need > consistent/coherent memory -- either pci_alloc_consistent() or > dma_alloc_coherent(). The dma_ variant is slightly better because you > can pass in a GFP_ mask rather than having the kernel pick GFP_ATOMIC > for you. > > > You're saying I must unmap it before the data is valid (cuz of the > > bounce buffering). If that's true, then how in sam hill does user mode > > RDMA work since the user's memory isn't unmapped before the user looks > > like memory that is the target of RDMA The uverbs code calls > > dma_map_sg() which is roughly the same as dma_map_single, eh? > > It's a good point. We're kind of counting on the IOMMU situation not > being too wacky, and the device being able to DMA to arbitrary > addresses. So swiotlb won't work in this case actually -- but we > assume any RDMA device can do 64-bit DMA so it doesn't hurt us in > practice. But yes, DMA to userspace is slightly risky and won't work > everywhere. > > - R. ___ openib-general mailing list openib-general@openib.org http://openib.org/mailman/listinfo/openib-general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
Re: [openib-general] problem with 2.6.19?
> I must be misusing dma_map_single(). What I'm doing is allocating a > verb message reply queue for the adapter to DMA verb replies into. It > never gets unmapped. I kmalloc() it, then map it. I could use > dma_alloc_coherent() or something, and maybe that's what I need to do? Yeah, if you want to leave something mapped and have the device DMA into it, and the CPU look into the buffer too, then you need consistent/coherent memory -- either pci_alloc_consistent() or dma_alloc_coherent(). The dma_ variant is slightly better because you can pass in a GFP_ mask rather than having the kernel pick GFP_ATOMIC for you. > You're saying I must unmap it before the data is valid (cuz of the > bounce buffering). If that's true, then how in sam hill does user mode > RDMA work since the user's memory isn't unmapped before the user looks > like memory that is the target of RDMA The uverbs code calls > dma_map_sg() which is roughly the same as dma_map_single, eh? It's a good point. We're kind of counting on the IOMMU situation not being too wacky, and the device being able to DMA to arbitrary addresses. So swiotlb won't work in this case actually -- but we assume any RDMA device can do 64-bit DMA so it doesn't hurt us in practice. But yes, DMA to userspace is slightly risky and won't work everywhere. - R. ___ openib-general mailing list openib-general@openib.org http://openib.org/mailman/listinfo/openib-general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
Re: [openib-general] problem with 2.6.19?
On Fri, 2006-10-27 at 11:02 -0700, Roland Dreier wrote: > > So the issue is my maps are getting setup with bounce bufs. This > > shouldn't be the case, I think, because my device supports 64b > > addressing. I'm diggin' into this. > > Yes, it's a little strange that you're still bouncing with the > pci_set_dma_mask(DMA_64BIT_MASK). But the bounce buffering should > work -- the fact that it doesn't means you have a bug in how you're > using dma_map_single(), because dma_unmap_single() will copy things > back from the bounce buffer. > > So you have two issues: > - why is the bounce buffering happening? > - what's wrong with your dma mapping calls? (Because there are other >situations with an IOMMU where you have to get things right) > > - R. I must be misusing dma_map_single(). What I'm doing is allocating a verb message reply queue for the adapter to DMA verb replies into. It never gets unmapped. I kmalloc() it, then map it. I could use dma_alloc_coherent() or something, and maybe that's what I need to do? You're saying I must unmap it before the data is valid (cuz of the bounce buffering). If that's true, then how in sam hill does user mode RDMA work since the user's memory isn't unmapped before the user looks like memory that is the target of RDMA The uverbs code calls dma_map_sg() which is roughly the same as dma_map_single, eh? Slightly confused... Steve. ___ openib-general mailing list openib-general@openib.org http://openib.org/mailman/listinfo/openib-general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
Re: [openib-general] problem with 2.6.19?
> So the issue is my maps are getting setup with bounce bufs. This > shouldn't be the case, I think, because my device supports 64b > addressing. I'm diggin' into this. Yes, it's a little strange that you're still bouncing with the pci_set_dma_mask(DMA_64BIT_MASK). But the bounce buffering should work -- the fact that it doesn't means you have a bug in how you're using dma_map_single(), because dma_unmap_single() will copy things back from the bounce buffer. So you have two issues: - why is the bounce buffering happening? - what's wrong with your dma mapping calls? (Because there are other situations with an IOMMU where you have to get things right) - R. ___ openib-general mailing list openib-general@openib.org http://openib.org/mailman/listinfo/openib-general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
Re: [openib-general] problem with 2.6.19?
So the issue is my maps are getting setup with bounce bufs. This
shouldn't be the case, I think, because my device supports 64b
addressing. I'm diggin' into this.
On Fri, 2006-10-27 at 10:53 -0500, Steve Wise wrote:
> And I do have CONFIG_SWIOTLB set:
>
> CONFIG_SWIOTLB=y
>
> On Fri, 2006-10-27 at 10:48 -0500, Steve Wise wrote:
> > Also, if I drop the memory down to 1G (via mem=1G on the boot line),
> > then things work. But I think I'm just disabling the IOMMU. Note that
> > __pa() and dma_map_single() return the same thing:
> >
> >
> > c2_alloc_mqsp_chunk mqsp_chunk va 81003c4d2000 dma_addr 3c4d2000 __pa
> > 3c4d2000
> > c2_alloc_mqsp addr 81003c4d201a dma_addr 3c4d201a
> > c2_alloc_mqsp addr 81003c4d201c dma_addr 3c4d201c
> > c2_alloc_mqsp addr 81003c4d201e dma_addr 3c4d201e
> > c2_alloc_mqsp addr 81003c4d2020 dma_addr 3c4d2020
> > c2_rnic_init rep_vq va 81003cb08000 dma 3cb08000 __pa 3cb08000
> > c2_rnic_init aeq va 81003d4e dma 3d4e __pa 3d4e
> >
> >
> >
> > On Fri, 2006-10-27 at 09:43 -0500, Steve Wise wrote:
> > > On Thu, 2006-10-26 at 15:26 -0700, Roland Dreier wrote:
> > > > Steve> The adapter seems to be dma'ing into the wrong memory. The
> > > > Steve> patch below backs the usage of dma_map_single() back to
> > > > Steve> using __pa() for converting kernel virtual addresses (from
> > > > Steve> kmalloc) into bus addresses, and things work ok.
> > > >
> > > > Hmm. It might be interesting to hack the driver to print the result
> > > > of both dma_map_single() and __pa() and see if they're different.
> > > >
> > >
> > > Here's a dump. They are different. The __pa()'s are what I expect. I
> > > dunno if the dma_addr's returned from dma_map_single() look good or not
> > > (they certainly don't work :)
> > >
> > > c2_alloc_mqsp_chunk mqsp_chunk va 810148b1b000 dma_addr 3a56000 __pa
> > > 148b1b000
> > > c2_alloc_mqsp addr 810148b1b01a dma_addr 3a5601a
> > > c2_alloc_mqsp addr 810148b1b01c dma_addr 3a5601c
> > > c2_alloc_mqsp addr 810148b1b01e dma_addr 3a5601e
> > > c2_alloc_mqsp addr 810148b1b020 dma_addr 3a56020
> > > c2_rnic_init rep_vq va 810147e78000 dma 3a57000 __pa 147e78000
> > > c2_rnic_init aeq va 810147d48000 dma 3a5f000 __pa 147d48000
> > >
> > > > Are you running on a 32-bit (i386) or 64-bit (x86_64) kernel? How
> > > > much RAM do you have?
> > >
> > > 64b/X86_64. 4GB RAM. The CPUs are Dempsey class XEONs - Dual CPU, Dual
> > > core. So with HT on linux sees 8 CPUs.
> > >
> > > > Is the kernel using swiotlb? If so then you
> > > > need to make sure your DMA_{TO,FROM} directions and dma_unmap calls
> > > > are right, since otherwise the DMAed data won't be copied to/from the
> > > > bounce buffer at the right time.
> > >
> > > All these mappings are for the device to DMA into the host memory, and
> > > I'm using DMA_FROM_DEVICE in my calls to dma_map_single().
> > >
> > > How do I know if the kernel is using swiotlb?
> > >
> > >
> > > > Another thing to do if you're patient would be to use git-bisect and
> > > > figure out exactly which patch made amso1100 stop working.
> > > >
> > >
> > > I added these calls as part of the review for submission into the
> > > kernel, and I originally tested them on dual CPU opteron systems with
> > > 1GB of memory. But maybe they weren't using the IOMMU? Dunno.
> > >
> > >
> > >
> > >
> > > ___
> > > openib-general mailing list
> > > openib-general@openib.org
> > > http://openib.org/mailman/listinfo/openib-general
> > >
> > > To unsubscribe, please visit
> > > http://openib.org/mailman/listinfo/openib-general
> > >
> >
> >
> > ___
> > openib-general mailing list
> > openib-general@openib.org
> > http://openib.org/mailman/listinfo/openib-general
> >
> > To unsubscribe, please visit
> > http://openib.org/mailman/listinfo/openib-general
> >
>
>
> ___
> openib-general mailing list
> openib-general@openib.org
> http://openib.org/mailman/listinfo/openib-general
>
> To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
>
___
openib-general mailing list
openib-general@openib.org
http://openib.org/mailman/listinfo/openib-general
To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
Re: [openib-general] problem with 2.6.19?
And I do have CONFIG_SWIOTLB set:
CONFIG_SWIOTLB=y
On Fri, 2006-10-27 at 10:48 -0500, Steve Wise wrote:
> Also, if I drop the memory down to 1G (via mem=1G on the boot line),
> then things work. But I think I'm just disabling the IOMMU. Note that
> __pa() and dma_map_single() return the same thing:
>
>
> c2_alloc_mqsp_chunk mqsp_chunk va 81003c4d2000 dma_addr 3c4d2000 __pa
> 3c4d2000
> c2_alloc_mqsp addr 81003c4d201a dma_addr 3c4d201a
> c2_alloc_mqsp addr 81003c4d201c dma_addr 3c4d201c
> c2_alloc_mqsp addr 81003c4d201e dma_addr 3c4d201e
> c2_alloc_mqsp addr 81003c4d2020 dma_addr 3c4d2020
> c2_rnic_init rep_vq va 81003cb08000 dma 3cb08000 __pa 3cb08000
> c2_rnic_init aeq va 81003d4e dma 3d4e __pa 3d4e
>
>
>
> On Fri, 2006-10-27 at 09:43 -0500, Steve Wise wrote:
> > On Thu, 2006-10-26 at 15:26 -0700, Roland Dreier wrote:
> > > Steve> The adapter seems to be dma'ing into the wrong memory. The
> > > Steve> patch below backs the usage of dma_map_single() back to
> > > Steve> using __pa() for converting kernel virtual addresses (from
> > > Steve> kmalloc) into bus addresses, and things work ok.
> > >
> > > Hmm. It might be interesting to hack the driver to print the result
> > > of both dma_map_single() and __pa() and see if they're different.
> > >
> >
> > Here's a dump. They are different. The __pa()'s are what I expect. I
> > dunno if the dma_addr's returned from dma_map_single() look good or not
> > (they certainly don't work :)
> >
> > c2_alloc_mqsp_chunk mqsp_chunk va 810148b1b000 dma_addr 3a56000 __pa
> > 148b1b000
> > c2_alloc_mqsp addr 810148b1b01a dma_addr 3a5601a
> > c2_alloc_mqsp addr 810148b1b01c dma_addr 3a5601c
> > c2_alloc_mqsp addr 810148b1b01e dma_addr 3a5601e
> > c2_alloc_mqsp addr 810148b1b020 dma_addr 3a56020
> > c2_rnic_init rep_vq va 810147e78000 dma 3a57000 __pa 147e78000
> > c2_rnic_init aeq va 810147d48000 dma 3a5f000 __pa 147d48000
> >
> > > Are you running on a 32-bit (i386) or 64-bit (x86_64) kernel? How
> > > much RAM do you have?
> >
> > 64b/X86_64. 4GB RAM. The CPUs are Dempsey class XEONs - Dual CPU, Dual
> > core. So with HT on linux sees 8 CPUs.
> >
> > > Is the kernel using swiotlb? If so then you
> > > need to make sure your DMA_{TO,FROM} directions and dma_unmap calls
> > > are right, since otherwise the DMAed data won't be copied to/from the
> > > bounce buffer at the right time.
> >
> > All these mappings are for the device to DMA into the host memory, and
> > I'm using DMA_FROM_DEVICE in my calls to dma_map_single().
> >
> > How do I know if the kernel is using swiotlb?
> >
> >
> > > Another thing to do if you're patient would be to use git-bisect and
> > > figure out exactly which patch made amso1100 stop working.
> > >
> >
> > I added these calls as part of the review for submission into the
> > kernel, and I originally tested them on dual CPU opteron systems with
> > 1GB of memory. But maybe they weren't using the IOMMU? Dunno.
> >
> >
> >
> >
> > ___
> > openib-general mailing list
> > openib-general@openib.org
> > http://openib.org/mailman/listinfo/openib-general
> >
> > To unsubscribe, please visit
> > http://openib.org/mailman/listinfo/openib-general
> >
>
>
> ___
> openib-general mailing list
> openib-general@openib.org
> http://openib.org/mailman/listinfo/openib-general
>
> To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
>
___
openib-general mailing list
openib-general@openib.org
http://openib.org/mailman/listinfo/openib-general
To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
Re: [openib-general] problem with 2.6.19?
Also, if I drop the memory down to 1G (via mem=1G on the boot line),
then things work. But I think I'm just disabling the IOMMU. Note that
__pa() and dma_map_single() return the same thing:
c2_alloc_mqsp_chunk mqsp_chunk va 81003c4d2000 dma_addr 3c4d2000 __pa
3c4d2000
c2_alloc_mqsp addr 81003c4d201a dma_addr 3c4d201a
c2_alloc_mqsp addr 81003c4d201c dma_addr 3c4d201c
c2_alloc_mqsp addr 81003c4d201e dma_addr 3c4d201e
c2_alloc_mqsp addr 81003c4d2020 dma_addr 3c4d2020
c2_rnic_init rep_vq va 81003cb08000 dma 3cb08000 __pa 3cb08000
c2_rnic_init aeq va 81003d4e dma 3d4e __pa 3d4e
On Fri, 2006-10-27 at 09:43 -0500, Steve Wise wrote:
> On Thu, 2006-10-26 at 15:26 -0700, Roland Dreier wrote:
> > Steve> The adapter seems to be dma'ing into the wrong memory. The
> > Steve> patch below backs the usage of dma_map_single() back to
> > Steve> using __pa() for converting kernel virtual addresses (from
> > Steve> kmalloc) into bus addresses, and things work ok.
> >
> > Hmm. It might be interesting to hack the driver to print the result
> > of both dma_map_single() and __pa() and see if they're different.
> >
>
> Here's a dump. They are different. The __pa()'s are what I expect. I
> dunno if the dma_addr's returned from dma_map_single() look good or not
> (they certainly don't work :)
>
> c2_alloc_mqsp_chunk mqsp_chunk va 810148b1b000 dma_addr 3a56000 __pa
> 148b1b000
> c2_alloc_mqsp addr 810148b1b01a dma_addr 3a5601a
> c2_alloc_mqsp addr 810148b1b01c dma_addr 3a5601c
> c2_alloc_mqsp addr 810148b1b01e dma_addr 3a5601e
> c2_alloc_mqsp addr 810148b1b020 dma_addr 3a56020
> c2_rnic_init rep_vq va 810147e78000 dma 3a57000 __pa 147e78000
> c2_rnic_init aeq va 810147d48000 dma 3a5f000 __pa 147d48000
>
> > Are you running on a 32-bit (i386) or 64-bit (x86_64) kernel? How
> > much RAM do you have?
>
> 64b/X86_64. 4GB RAM. The CPUs are Dempsey class XEONs - Dual CPU, Dual
> core. So with HT on linux sees 8 CPUs.
>
> > Is the kernel using swiotlb? If so then you
> > need to make sure your DMA_{TO,FROM} directions and dma_unmap calls
> > are right, since otherwise the DMAed data won't be copied to/from the
> > bounce buffer at the right time.
>
> All these mappings are for the device to DMA into the host memory, and
> I'm using DMA_FROM_DEVICE in my calls to dma_map_single().
>
> How do I know if the kernel is using swiotlb?
>
>
> > Another thing to do if you're patient would be to use git-bisect and
> > figure out exactly which patch made amso1100 stop working.
> >
>
> I added these calls as part of the review for submission into the
> kernel, and I originally tested them on dual CPU opteron systems with
> 1GB of memory. But maybe they weren't using the IOMMU? Dunno.
>
>
>
>
> ___
> openib-general mailing list
> openib-general@openib.org
> http://openib.org/mailman/listinfo/openib-general
>
> To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
>
___
openib-general mailing list
openib-general@openib.org
http://openib.org/mailman/listinfo/openib-general
To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
Re: [openib-general] problem with 2.6.19?
On Thu, 2006-10-26 at 15:26 -0700, Roland Dreier wrote:
> Steve> The adapter seems to be dma'ing into the wrong memory. The
> Steve> patch below backs the usage of dma_map_single() back to
> Steve> using __pa() for converting kernel virtual addresses (from
> Steve> kmalloc) into bus addresses, and things work ok.
>
> Hmm. It might be interesting to hack the driver to print the result
> of both dma_map_single() and __pa() and see if they're different.
>
Here's a dump. They are different. The __pa()'s are what I expect. I
dunno if the dma_addr's returned from dma_map_single() look good or not
(they certainly don't work :)
c2_alloc_mqsp_chunk mqsp_chunk va 810148b1b000 dma_addr 3a56000 __pa
148b1b000
c2_alloc_mqsp addr 810148b1b01a dma_addr 3a5601a
c2_alloc_mqsp addr 810148b1b01c dma_addr 3a5601c
c2_alloc_mqsp addr 810148b1b01e dma_addr 3a5601e
c2_alloc_mqsp addr 810148b1b020 dma_addr 3a56020
c2_rnic_init rep_vq va 810147e78000 dma 3a57000 __pa 147e78000
c2_rnic_init aeq va 810147d48000 dma 3a5f000 __pa 147d48000
> Are you running on a 32-bit (i386) or 64-bit (x86_64) kernel? How
> much RAM do you have?
64b/X86_64. 4GB RAM. The CPUs are Dempsey class XEONs - Dual CPU, Dual
core. So with HT on linux sees 8 CPUs.
> Is the kernel using swiotlb? If so then you
> need to make sure your DMA_{TO,FROM} directions and dma_unmap calls
> are right, since otherwise the DMAed data won't be copied to/from the
> bounce buffer at the right time.
All these mappings are for the device to DMA into the host memory, and
I'm using DMA_FROM_DEVICE in my calls to dma_map_single().
How do I know if the kernel is using swiotlb?
> Another thing to do if you're patient would be to use git-bisect and
> figure out exactly which patch made amso1100 stop working.
>
I added these calls as part of the review for submission into the
kernel, and I originally tested them on dual CPU opteron systems with
1GB of memory. But maybe they weren't using the IOMMU? Dunno.
___
openib-general mailing list
openib-general@openib.org
http://openib.org/mailman/listinfo/openib-general
To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
Re: [openib-general] problem with 2.6.19?
Steve> The adapter seems to be dma'ing into the wrong memory. The
Steve> patch below backs the usage of dma_map_single() back to
Steve> using __pa() for converting kernel virtual addresses (from
Steve> kmalloc) into bus addresses, and things work ok.
Hmm. It might be interesting to hack the driver to print the result
of both dma_map_single() and __pa() and see if they're different.
Are you running on a 32-bit (i386) or 64-bit (x86_64) kernel? How
much RAM do you have? Is the kernel using swiotlb? If so then you
need to make sure your DMA_{TO,FROM} directions and dma_unmap calls
are right, since otherwise the DMAed data won't be copied to/from the
bounce buffer at the right time.
If you're not using swiotlb then I'm somewhat mystified. I guess
comparing what dma_map_single() and __pa() do might be enlightening.
Another thing to do if you're patient would be to use git-bisect and
figure out exactly which patch made amso1100 stop working.
- R.
___
openib-general mailing list
openib-general@openib.org
http://openib.org/mailman/listinfo/openib-general
To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
