kthread with kpause or callout
Hi, I'm writing a keyboard driver which doesn't support interrupts in some revisions, so I have to poll its status from time to time. What would be better suited for that task: - Running a kthread and calling kpause() between the polls. - Using a callout which reschedules itself after the poll. Or is there something even better suited, which I'm missing? -- Frank Wille
Re: kthread with kpause or callout
On Mon, Feb 08, 2010 at 11:11:04AM +0100, Frank Wille wrote: > - Running a kthread and calling kpause() between the polls. > - Using a callout which reschedules itself after the poll. The thread is quite a bit more heavyweight, but you have full freedom to do what you want. The callout is pretty lightweight but you are still subject of "interrupt context" rules. I ran into a similar case recently where I didn't want full thread context but needed it sometimes - I ended up using a a rescheduled callout plus sometimes adding a job to the sysmon workqueue (sysmon_task_queue_sched(), this task was in envsys scope, so it was an easy way out). Martin
Re: kthread with kpause or callout
On Mon, 8 Feb 2010 11:29:51 +0100 Martin Husemann wrote: > On Mon, Feb 08, 2010 at 11:11:04AM +0100, Frank Wille wrote: > > - Running a kthread and calling kpause() between the polls. > > - Using a callout which reschedules itself after the poll. > > The thread is quite a bit more heavyweight, but you have full freedom > to do what you want. The callout is pretty lightweight but you are > still subject of "interrupt context" rules. Ok, thanks. Then the callout should be fine for me. I'm just unsure about using mutexes during the callout. I have an IPL_NONE-kmutex which locks register access (my chip supports several register banks, so I need to make sure they are not switched). May I acquire this mutex during a callout (which is a softint, as I understand)? Will the softint sleep or busy-wait? -- Frank Wille
Re: kthread with kpause or callout
On Mon, Feb 08, 2010 at 03:04:07PM +0100, Frank Wille wrote: > I'm just unsure about using mutexes during the callout. I have an > IPL_NONE-kmutex which locks register access (my chip supports several > register banks, so I need to make sure they are not switched). May I > acquire this mutex during a callout (which is a softint, as I understand)? > Will the softint sleep or busy-wait? Depends on the mutex type, from mutex(9): IPL_NONE, or one of the IPL_SOFT* constants An adaptive mutex will be returned. Adaptive mutexes provide mutual exclusion between LWPs, and between LWPs and soft interrupt handlers. Adaptive mutexes cannot be acquired from a hardware interrupt handler. An LWP may either sleep or busy-wait when attempt- ing to acquire an adaptive mutex that is already held. IPL_VM, IPL_SCHED, IPL_HIGH A spin mutex will be returned. Spin mutexes provide mutual exclusion between LWPs, and between LWPs and interrupt han- dlers. The wording is not explicit, but a softint is not allowed to block on an adaptive mutex, you need a spin mutex for that (usually such mutexes are used by interrupt handlers, so you have a "natural" IPL to use here). A caller will always busy wait trying to aquire a spin mutex, but it might fall back to sleep on an adaptive mutex. Martin
Re: kthread with kpause or callout
On Mon, Feb 08, 2010 at 03:09:55PM +0100, Martin Husemann wrote: > The wording is not explicit, but a softint is not allowed to block on s/softint/callout/ of course, sorry for the confusion. Martin
Re: kthread with kpause or callout
On Mon, 8 Feb 2010 15:09:55 +0100 Martin Husemann wrote: > On Mon, Feb 08, 2010 at 03:04:07PM +0100, Frank Wille wrote: > > [...] May I acquire this mutex during a callout (which is a > > softint, as I understand)? Will the softint sleep or busy-wait? > > Depends on the mutex type, from mutex(9): > [...] > Adaptive mutexes cannot be acquired from a hardware > interrupt handler. > [...] > The wording is not explicit, but a softint is not allowed to block on > an adaptive mutex, Yes, this is what I thought as well, but the man page only mentions hardware interrupts. Are you sure about it? Hmmm... from softint(9): [...] Unlike hardware interrupts, software interrupts do have thread context. They may block on synchro- nization objects, sleep, and resume execution at a later time. [...] IMHO that would allow my callout to sleep on acquiring the mutex? -- Frank Wille
Re: kthread with kpause or callout
On Mon, 8 Feb 2010 15:12:10 +0100 Martin Husemann wrote: > On Mon, Feb 08, 2010 at 03:09:55PM +0100, Martin Husemann wrote: > > The wording is not explicit, but a softint is not allowed to block > > on > > s/softint/callout/ of course, sorry for the confusion. *Now* you confused me! :) A callout is executed as a softint, isn't it? -- Frank Wille
Re: kthread with kpause or callout
On Mon, Feb 08, 2010 at 03:35:35PM +0100, Frank Wille wrote: > IMHO that would allow my callout to sleep on acquiring the mutex? A softint can sleep, a callout can not. Martin
Re: kthread with kpause or callout
On Mon, Feb 08, 2010 at 03:36:07PM +0100, Martin Husemann wrote: > On Mon, Feb 08, 2010 at 03:35:35PM +0100, Frank Wille wrote: > > IMHO that would allow my callout to sleep on acquiring the mutex? > > A softint can sleep, a callout can not. s/can/should/ Joerg
Re: kthread with kpause or callout
On Mon, Feb 08, 2010 at 03:04:07PM +0100, Frank Wille wrote: > On Mon, 8 Feb 2010 11:29:51 +0100 > Martin Husemann wrote: > > > On Mon, Feb 08, 2010 at 11:11:04AM +0100, Frank Wille wrote: > > > - Running a kthread and calling kpause() between the polls. > > > - Using a callout which reschedules itself after the poll. > > > > The thread is quite a bit more heavyweight, but you have full freedom > > to do what you want. The callout is pretty lightweight but you are > > still subject of "interrupt context" rules. > > Ok, thanks. Then the callout should be fine for me. > > I'm just unsure about using mutexes during the callout. I have an > IPL_NONE-kmutex which locks register access (my chip supports several > register banks, so I need to make sure they are not switched). May I > acquire this mutex during a callout (which is a softint, as I understand)? > Will the softint sleep or busy-wait? Yes, you can take an IPL_NONE (adaptive) mutex from a softint / callout. Whether it sleeps or spins is a decision that the kernel makes at runtime. It tries to spin. If that would deadlock the system, it sleeps. This is the only time a callout/softint can sleep - for a mutex/rwlock. You have to be careful with such mutexes. For example, you have my_lock which is an IPL_NONE mutex. You can grab this from a softint, no problem. But you must NOT do something like this from thread context once that lock is known to be taken from a softint. mutex_enter(&my_lock); blargh = kmem_alloc(1024, KM_SLEEP); mutex_exit(&my_lock); .. as then you can cause the softint to wait on your memory allocation, simply because it wants the lock that you hold. This can deadlock the system in short order. See the various manual pages for a bit more discussion. FYI there is no difference between an IPL_NONE and IPL_SOFT* mutex. In this case the name is only a sprinkling of sugar.
Re: kthread with kpause or callout
Andrew Doran wrote: >> I'm just unsure about using mutexes during the callout. I have an >> IPL_NONE-kmutex which locks register access (my chip supports several >> register banks, so I need to make sure they are not switched). May I >> acquire this mutex during a callout (which is a softint, as I >> understand)? Will the softint sleep or busy-wait? > > Yes, you can take an IPL_NONE (adaptive) mutex from a softint / > callout. Whether it sleeps or spins is a decision that the kernel makes > at runtime. It tries to spin. If that would deadlock the system, it > sleeps. This is the only time a callout/softint can sleep - for a > mutex/rwlock. Thanks. Good to know. In the meantime I had to upgrade to an IPL_VM mutex, because the chip registers will have to be accessed from a VM-level interrupt too. I already discussed that with Martin in private and I guess it will have the effect that a process or softint holding the mutex will lock out any IPL_VM interrupt during this time? -- Frank Wille
Re: kthread with kpause or callout
On Mon, Feb 22, 2010 at 09:06:13PM +0100, Frank Wille wrote: > Andrew Doran wrote: > > >> I'm just unsure about using mutexes during the callout. I have an > >> IPL_NONE-kmutex which locks register access (my chip supports several > >> register banks, so I need to make sure they are not switched). May I > >> acquire this mutex during a callout (which is a softint, as I > >> understand)? Will the softint sleep or busy-wait? > > > > Yes, you can take an IPL_NONE (adaptive) mutex from a softint / > > callout. Whether it sleeps or spins is a decision that the kernel makes > > at runtime. It tries to spin. If that would deadlock the system, it > > sleeps. This is the only time a callout/softint can sleep - for a > > mutex/rwlock. > > Thanks. Good to know. > > In the meantime I had to upgrade to an IPL_VM mutex, because the chip > registers will have to be accessed from a VM-level interrupt too. I already > discussed that with Martin in private and I guess it will have the effect > that a process or softint holding the mutex will lock out any IPL_VM > interrupt during this time? In short, yes. There is a tradeoff to be made between: - how long you block interrupts - how much thread concurrency you want to support - the expense of adding additional concurrency controls like locks Note that the IPL_VM lock will block interrupts at that level, only on the CPU where the lock is held. The interrupt masking bit will not affect other CPUs (although of course, it's still a spinlock so another CPU will have to wait while its held). Depending on what you're doing, a IPL_VM lock alone may be fine, or it may make sense to have a two level setup for concurrency where you protect your hardware state and things touched from the hard interrupt handler with an IPL_VM lock, and then have an IPL_NONE lock to protect everything else. Or some other scheme. It's an "it depends".
