Doing zero-copy stuff in drivers, or "is vm_fault_quick_hold_pages() enough" ?
Hi, gonzo@ committed a fix (r278615) to the videocore driver for the raspberry pi. The fix involved doing an explicit wire of pages that were about to be passed down to the hardware to send to the videobuffer hardware. It turns out that doing vm_fault_quick_hold_pages() wasn't enough - the pages weren't being wired, and they may disappear before the hardware gets to them. I looked at vmapbuf() and how it uses vm_fault_quick_hold_pages(), but I can't find anything that wires the pages down before it hands the addresses to the hardware. So, am I missing something about how/where that's done? Thanks, -adrian ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: Doing zero-copy stuff in drivers, or "is vm_fault_quick_hold_pages() enough" ?
On Mon, Mar 2, 2015 at 12:42 PM, Adrian Chadd wrote:
> Hi,
>
> gonzo@ committed a fix (r278615) to the videocore driver for the
> raspberry pi. The fix involved doing an explicit wire of pages that
> were about to be passed down to the hardware to send to the
> videobuffer hardware.
>
> It turns out that doing vm_fault_quick_hold_pages() wasn't enough -
> the pages weren't being wired, and they may disappear before the
> hardware gets to them.
Then your code is buggy or you've hit a bug in the VM. Holding a page
is a short-term wiring.
Right above vm_page_hold():
/*
* Keep page from being freed by the page daemon
* much of the same effect as wiring, except much lower
* overhead and should be used only for *very* temporary
* holding ("wiring").
*/
> I looked at vmapbuf() and how it uses vm_fault_quick_hold_pages(), but
> I can't find anything that wires the pages down before it hands the
> addresses to the hardware.
>
> So, am I missing something about how/where that's done?
Yes.
> Thanks,
>
>
> -adrian
> ___
> freebsd-current@freebsd.org mailing list
> http://lists.freebsd.org/mailman/listinfo/freebsd-current
> To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: Doing zero-copy stuff in drivers, or "is vm_fault_quick_hold_pages() enough" ?
On 2 March 2015 at 15:05, K. Macy wrote:
> On Mon, Mar 2, 2015 at 12:42 PM, Adrian Chadd wrote:
>> Hi,
>>
>> gonzo@ committed a fix (r278615) to the videocore driver for the
>> raspberry pi. The fix involved doing an explicit wire of pages that
>> were about to be passed down to the hardware to send to the
>> videobuffer hardware.
>>
>> It turns out that doing vm_fault_quick_hold_pages() wasn't enough -
>> the pages weren't being wired, and they may disappear before the
>> hardware gets to them.
>
>
> Then your code is buggy or you've hit a bug in the VM. Holding a page
> is a short-term wiring.
Well, you can look at what's going on in the vchiq code. gonzo made it
an explicit wire in order to avoid issues.
> Right above vm_page_hold():
> /*
> * Keep page from being freed by the page daemon
> * much of the same effect as wiring, except much lower
> * overhead and should be used only for *very* temporary
> * holding ("wiring").
> */
What's the definition of "very temporary holding" ? What's the
behavioural difference?
-adrian
___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: Doing zero-copy stuff in drivers, or "is vm_fault_quick_hold_pages() enough" ?
>> Right above vm_page_hold():
>> /*
>> * Keep page from being freed by the page daemon
>> * much of the same effect as wiring, except much lower
>> * overhead and should be used only for *very* temporary
>> * holding ("wiring").
>> */
>
> What's the definition of "very temporary holding" ? What's the
> behavioural difference?
Long enough to complete a DMA operation versus the lifetime of an
executing program.
___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: Doing zero-copy stuff in drivers, or "is vm_fault_quick_hold_pages() enough" ?
On 2 March 2015 at 16:12, K. Macy wrote:
>>> Right above vm_page_hold():
>>> /*
>>> * Keep page from being freed by the page daemon
>>> * much of the same effect as wiring, except much lower
>>> * overhead and should be used only for *very* temporary
>>> * holding ("wiring").
>>> */
>>
>> What's the definition of "very temporary holding" ? What's the
>> behavioural difference?
>
> Long enough to complete a DMA operation versus the lifetime of an
> executing program.
Ok, but is there a specific time length that this should be?
A DMA operation to a slow device could be up to hundreds of
milliseconds; or seconds if things are really backed up.
Using wire instead of hold definitely made things work without having
the page disappear from underneath it. Oleksander knows more about the
details of that.
-adrian
___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: Doing zero-copy stuff in drivers, or "is vm_fault_quick_hold_pages() enough" ?
On Mon, Mar 02, 2015 at 04:17:03PM -0800, Adrian Chadd wrote:
> On 2 March 2015 at 16:12, K. Macy wrote:
> >>> Right above vm_page_hold():
> >>> /*
> >>> * Keep page from being freed by the page daemon
> >>> * much of the same effect as wiring, except much lower
> >>> * overhead and should be used only for *very* temporary
> >>> * holding ("wiring").
> >>> */
> >>
> >> What's the definition of "very temporary holding" ? What's the
> >> behavioural difference?
> >
> > Long enough to complete a DMA operation versus the lifetime of an
> > executing program.
>
> Ok, but is there a specific time length that this should be?
Difference between hold and wire is effectively that held pages are
still kept on the page queues, providing potentially uneccessary work
for pagedaemon to find them and skip. Wired pages are removed from the
queues.
This means that holding a page is much cheaper, by the cost of leaving
slightly more work to the system. Also, holding a page only requires the
page lock, while wiring contend on the page queue lock, in addition to
the page lock.
>
> A DMA operation to a slow device could be up to hundreds of
> milliseconds; or seconds if things are really backed up.
>
> Using wire instead of hold definitely made things work without having
> the page disappear from underneath it. Oleksander knows more about the
> details of that.
Page cannot 'disappear'. The only thing which could happen with the
memory page is reuse, when the page is removed from the previous object
and re-purposed for some other object, loosing old content.
Your terminology suggests that something unrelated happen.
___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: Doing zero-copy stuff in drivers, or "is vm_fault_quick_hold_pages() enough" ?
On 2 March 2015 at 16:25, Konstantin Belousov wrote: >> Ok, but is there a specific time length that this should be? > Difference between hold and wire is effectively that held pages are > still kept on the page queues, providing potentially uneccessary work > for pagedaemon to find them and skip. Wired pages are removed from the > queues. > > This means that holding a page is much cheaper, by the cost of leaving > slightly more work to the system. Also, holding a page only requires the > page lock, while wiring contend on the page queue lock, in addition to > the page lock. Thanks for the description - that makes things a lot clearer! >> >> A DMA operation to a slow device could be up to hundreds of >> milliseconds; or seconds if things are really backed up. >> >> Using wire instead of hold definitely made things work without having >> the page disappear from underneath it. Oleksander knows more about the >> details of that. > > Page cannot 'disappear'. The only thing which could happen with the > memory page is reuse, when the page is removed from the previous object > and re-purposed for some other object, loosing old content. > > Your terminology suggests that something unrelated happen. Yup, and that's what I'm worried about :( -adrian ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
