New USB stack and Zero copy.
Hi, I want to get rid of the copying between DMA'able memory and non-DMA'able memory. Currently I allocate N memory-pages for each USB transfer like separate pages. The bus-dma system then assigns all of these pages each their virtual address. What I see is that when I allocate more than PAGE_SIZE bytes using bus-dma, I get physically contiguous memory. I don't need that for the USB stack. The question is: Should we change bus-dma to support so called scatter and gather allocations, where the physical allocation is non-contiguous, and the virtual allocation is contiguous accross all the scattered pages ? Also: How is the easiest way to load memory pages into DMA ? And I want that the loadig works like this, that when the page must be bounced it should not allocate a bounce buffer, hence I already have a bounce buffer. I only need to know which pages I can forward directly to the USB hardware, and the rest I will bounce somewhere else. --HPS ___ freebsd-usb@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-usb To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: New USB stack and Zero copy.
Hans Petter Selasky wrote this message on Wed, Jul 04, 2007 at 09:01 +0200: > I want to get rid of the copying between DMA'able memory and non-DMA'able > memory. > > Currently I allocate N memory-pages for each USB transfer like separate pages. How do you allocate these pages? With malloc(9) or w/ bus_dmamem_alloc(9)? > The bus-dma system then assigns all of these pages each their virtual > address. > > What I see is that when I allocate more than PAGE_SIZE bytes using bus-dma, I > get physically contiguous memory. I don't need that for the USB stack. That is a limitation of how bus_dma currently allocates memory... It calls contigmalloc, which doesn't handle multi-segment memory allocations yet.. > The question is: > > Should we change bus-dma to support so called scatter and gather allocations, > where the physical allocation is non-contiguous, and the virtual allocation > is contiguous accross all the scattered pages ? You can already support this by using malloc, and loading that buffer into your bus_dma map... or using the passing in userland buffer, and loading that into the map... > Also: How is the easiest way to load memory pages into DMA ? And I want that > the loadig works like this, that when the page must be bounced it should not > allocate a bounce buffer, hence I already have a bounce buffer. I only need > to know which pages I can forward directly to the USB hardware, and the rest > I will bounce somewhere else. Why do you not want to let bus_dma do the bouncing for you? If it's to save a copy to another buffer, why don't you load the final buffer into bus_dma? -- John-Mark Gurney Voice: +1 415 225 5579 "All that I will do, has been done, All that I have, has not." ___ freebsd-usb@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-usb To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: New USB stack and Zero copy.
On Wednesday 04 July 2007 19:35, John-Mark Gurney wrote: > Hans Petter Selasky wrote this message on Wed, Jul 04, 2007 at 09:01 +0200: > > I want to get rid of the copying between DMA'able memory and non-DMA'able > > memory. > > > > Currently I allocate N memory-pages for each USB transfer like separate > > pages. > > How do you allocate these pages? With malloc(9) or w/ bus_dmamem_alloc(9)? bus_dmamem_alloc(). > > > The bus-dma system then assigns all of these pages each their virtual > > address. > > > > What I see is that when I allocate more than PAGE_SIZE bytes using > > bus-dma, I get physically contiguous memory. I don't need that for the > > USB stack. > > That is a limitation of how bus_dma currently allocates memory... It > calls contigmalloc, which doesn't handle multi-segment memory allocations > yet.. > > > The question is: > > > > Should we change bus-dma to support so called scatter and gather > > allocations, where the physical allocation is non-contiguous, and the > > virtual allocation is contiguous accross all the scattered pages ? > > You can already support this by using malloc, and loading that buffer > into your bus_dma map... or using the passing in userland buffer, and > loading that into the map... > > > Also: How is the easiest way to load memory pages into DMA ? And I want > > that the loadig works like this, that when the page must be bounced it > > should not allocate a bounce buffer, hence I already have a bounce > > buffer. I only need to know which pages I can forward directly to the USB > > hardware, and the rest I will bounce somewhere else. > > Why do you not want to let bus_dma do the bouncing for you? If it's > to save a copy to another buffer, why don't you load the final buffer > into bus_dma? Because if I let bus_dma do the bounching, I cannot do this while holding a mutex, hence allocating DMA'able memory on the fly is not so good. --HPS ___ freebsd-usb@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-usb To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: New USB stack and Zero copy.
On Thursday 05 July 2007 03:31:59 am Hans Petter Selasky wrote: > On Wednesday 04 July 2007 19:35, John-Mark Gurney wrote: > > Hans Petter Selasky wrote this message on Wed, Jul 04, 2007 at 09:01 +0200: > > > Also: How is the easiest way to load memory pages into DMA ? And I want > > > that the loadig works like this, that when the page must be bounced it > > > should not allocate a bounce buffer, hence I already have a bounce > > > buffer. I only need to know which pages I can forward directly to the USB > > > hardware, and the rest I will bounce somewhere else. > > > > Why do you not want to let bus_dma do the bouncing for you? If it's > > to save a copy to another buffer, why don't you load the final buffer > > into bus_dma? > > Because if I let bus_dma do the bounching, I cannot do this while holding a > mutex, hence allocating DMA'able memory on the fly is not so good. This is not a hard problem to solve, every other driver using bus_dma solves it. Just make sure your driver is in a sane state and drop the lock while you let bus_dmamap_load() map/copy things for you. -- John Baldwin ___ freebsd-usb@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-usb To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: New USB stack and Zero copy.
On Thursday 05 July 2007 04:25:17 pm John Baldwin wrote: > On Thursday 05 July 2007 03:31:59 am Hans Petter Selasky wrote: > > On Wednesday 04 July 2007 19:35, John-Mark Gurney wrote: > > > Hans Petter Selasky wrote this message on Wed, Jul 04, 2007 at 09:01 > +0200: > > > > Also: How is the easiest way to load memory pages into DMA ? And I want > > > > that the loadig works like this, that when the page must be bounced it > > > > should not allocate a bounce buffer, hence I already have a bounce > > > > buffer. I only need to know which pages I can forward directly to the > USB > > > > hardware, and the rest I will bounce somewhere else. > > > > > > Why do you not want to let bus_dma do the bouncing for you? If it's > > > to save a copy to another buffer, why don't you load the final buffer > > > into bus_dma? > > > > Because if I let bus_dma do the bounching, I cannot do this while holding a > > mutex, hence allocating DMA'able memory on the fly is not so good. > > This is not a hard problem to solve, every other driver using bus_dma solves > it. Just make sure your driver is in a sane state and drop the lock while > you let bus_dmamap_load() map/copy things for you. Bah, backwards (was thinking of the fact that if you get EINPROGRESS you will have to drop the lock and just wait until the callback is called to make further progress on the request). bus_dmamap_load() already _requires_ you to hold your mutex when you call it, so I don't really see what the issue is, unless you are assuming BUS_DMA_NOWAIT behavior and can't properly handle deferred callbacks. You do have to drop the lock around bus_dmamem_alloc(), but not bus_dmamap_load(). -- John Baldwin ___ freebsd-usb@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-usb To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: New USB stack and Zero copy.
On Friday 06 July 2007 01:35, John Baldwin wrote: > On Thursday 05 July 2007 04:25:17 pm John Baldwin wrote: > > On Thursday 05 July 2007 03:31:59 am Hans Petter Selasky wrote: > > > On Wednesday 04 July 2007 19:35, John-Mark Gurney wrote: > > > > Hans Petter Selasky wrote this message on Wed, Jul 04, 2007 at 09:01 > > > > +0200: > > > > > Also: How is the easiest way to load memory pages into DMA ? And I > > want > > > > > > that the loadig works like this, that when the page must be bounced > > > > > it should not allocate a bounce buffer, hence I already have a > > > > > bounce buffer. I only need to know which pages I can forward > > > > > directly to the > > > > USB > > > > > > > hardware, and the rest I will bounce somewhere else. > > > > > > > > Why do you not want to let bus_dma do the bouncing for you? If it's > > > > to save a copy to another buffer, why don't you load the final buffer > > > > into bus_dma? > > > > > > Because if I let bus_dma do the bounching, I cannot do this while > > > holding > > a > > > > mutex, hence allocating DMA'able memory on the fly is not so good. > > > > This is not a hard problem to solve, every other driver using bus_dma > > solves it. Just make sure your driver is in a sane state and drop the > > lock while you let bus_dmamap_load() map/copy things for you. > > Bah, backwards (was thinking of the fact that if you get EINPROGRESS you > will have to drop the lock and just wait until the callback is called to > make further progress on the request). bus_dmamap_load() already > _requires_ you to hold your mutex when you call it, so I don't really see > what the issue is, unless you are assuming BUS_DMA_NOWAIT behavior and > can't properly handle deferred callbacks. You do have to drop the lock > around bus_dmamem_alloc(), but not bus_dmamap_load(). The thing is that allocating memory on the fly will be slow, and especially when the _load() functions will allocate contiguous memory. This only works fine when you load mbufs and things with size less than PAGE_SIZE bytes ?? --HPS ___ freebsd-usb@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-usb To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: New USB stack and Zero copy.
On Friday 06 July 2007 02:59:39 am Hans Petter Selasky wrote: > On Friday 06 July 2007 01:35, John Baldwin wrote: > > On Thursday 05 July 2007 04:25:17 pm John Baldwin wrote: > > > On Thursday 05 July 2007 03:31:59 am Hans Petter Selasky wrote: > > > > On Wednesday 04 July 2007 19:35, John-Mark Gurney wrote: > > > > > Hans Petter Selasky wrote this message on Wed, Jul 04, 2007 at 09:01 > > > > > > +0200: > > > > > > Also: How is the easiest way to load memory pages into DMA ? And I > > > > want > > > > > > > > that the loadig works like this, that when the page must be bounced > > > > > > it should not allocate a bounce buffer, hence I already have a > > > > > > bounce buffer. I only need to know which pages I can forward > > > > > > directly to the > > > > > > USB > > > > > > > > > hardware, and the rest I will bounce somewhere else. > > > > > > > > > > Why do you not want to let bus_dma do the bouncing for you? If it's > > > > > to save a copy to another buffer, why don't you load the final buffer > > > > > into bus_dma? > > > > > > > > Because if I let bus_dma do the bounching, I cannot do this while > > > > holding > > > > a > > > > > > mutex, hence allocating DMA'able memory on the fly is not so good. > > > > > > This is not a hard problem to solve, every other driver using bus_dma > > > solves it. Just make sure your driver is in a sane state and drop the > > > lock while you let bus_dmamap_load() map/copy things for you. > > > > Bah, backwards (was thinking of the fact that if you get EINPROGRESS you > > will have to drop the lock and just wait until the callback is called to > > make further progress on the request). bus_dmamap_load() already > > _requires_ you to hold your mutex when you call it, so I don't really see > > what the issue is, unless you are assuming BUS_DMA_NOWAIT behavior and > > can't properly handle deferred callbacks. You do have to drop the lock > > around bus_dmamem_alloc(), but not bus_dmamap_load(). > > The thing is that allocating memory on the fly will be slow, and especially > when the _load() functions will allocate contiguous memory. This only > works fine when you load mbufs and things with size less than PAGE_SIZE > bytes ?? Huh? The bounce pages are preallocated, so bus_dmamap_load() isn't going to be allocating things. -- John Baldwin ___ freebsd-usb@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-usb To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: New USB stack and Zero copy.
On Friday 06 July 2007 22:41, John Baldwin wrote: > On Friday 06 July 2007 02:59:39 am Hans Petter Selasky wrote: > > On Friday 06 July 2007 01:35, John Baldwin wrote: > > > On Thursday 05 July 2007 04:25:17 pm John Baldwin wrote: > > > > On Thursday 05 July 2007 03:31:59 am Hans Petter Selasky wrote: > > > > > On Wednesday 04 July 2007 19:35, John-Mark Gurney wrote: > > > > > > Hans Petter Selasky wrote this message on Wed, Jul 04, 2007 at > > Huh? The bounce pages are preallocated, so bus_dmamap_load() isn't going > to be allocating things. When are they allocated ? --HPS ___ freebsd-usb@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-usb To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: New USB stack and Zero copy.
On Saturday 07 July 2007 12:14:24 pm Hans Petter Selasky wrote: > On Friday 06 July 2007 22:41, John Baldwin wrote: > > On Friday 06 July 2007 02:59:39 am Hans Petter Selasky wrote: > > > On Friday 06 July 2007 01:35, John Baldwin wrote: > > > > On Thursday 05 July 2007 04:25:17 pm John Baldwin wrote: > > > > > On Thursday 05 July 2007 03:31:59 am Hans Petter Selasky wrote: > > > > > > On Wednesday 04 July 2007 19:35, John-Mark Gurney wrote: > > > > > > > Hans Petter Selasky wrote this message on Wed, Jul 04, 2007 at > > > > > Huh? The bounce pages are preallocated, so bus_dmamap_load() isn't going > > to be allocating things. > > When are they allocated ? UTSL. bus_dma_tag_create() (if you pass BUS_DMA_ALLOCNOW) or bus_dmamap_create() for the first map belonging to a tag. -- John Baldwin ___ freebsd-usb@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-usb To unsubscribe, send any mail to "[EMAIL PROTECTED]"
