RE: Why can not processes switch in atomic context?
Date: Wed, 4 Jul 2012 08:12:25 -0700 Subject: Re: Why can not processes switch in atomic context? From: dhyla...@gmail.com To: bu...@live.cn CC: mobile.parmeni...@gmail.com; kernelnewbies@kernelnewbies.org Hi, On Wed, Jul 4, 2012 at 3:44 AM, 弋天 卜 bu...@live.cn wrote: 在 2012-7-3，22:26，Parmenides mobile.parmeni...@gmail.com 写道： ...snip... 1. For the spinlock case, it is easy to get if preemption is allowed in critical section, the purpose of protection provided by spinlock can not be achieved readily. i think disable preemption in spinlock is not for safe, it is For efficiency. Spinlock should exit as soon as possible. If tank1 get spinlock and goto sleep for 100 seconds before Release spinlock, task2 which requests the same spinlock Should wait for 100 seconds, for this example, mutex should be used instead of spinlock. Unless, of course, the interrupt that fired tried to acquire the spinlock it preempted, in which case you would have deadlock, even on an SMP system, if the same processor happened to be used for both. yes, i think you are right, suppose task1 use spin_lock_irqsave() to get a spinlock, then call schedule() to sleep for a long time, the interrupt on this cpu core will be enabled by kernel, and if there is an interrupt triggered on the same cpu and get the same spinlock, deadlock will happen. ___ Kernelnewbies mailing list Kernelnewbies@kernelnewbies.org http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies
Re: Why can not processes switch in atomic context?
On Tue, Jul 3, 2012 at 4:24 PM, Parmenides mobile.parmeni...@gmail.com wrote: Hi, It is said that kernel can not be preempted in interrupt context and when it is in a critical section protected by a spin lock. 1. For the spinlock case, it is easy to get if preemption is allowed in critical section, the purpose of protection provided by spinlock can not be achieved readily. A process cannot be preempted nor sleep while holding a spinlock due spinlocks behavior. If a process grabs a spinlock and goes to sleep before releasing it. A second process (or an interrupt handler) that to grab the spinlock will busy wait. On an uniprocessor machine the second process will lock the CPU not allowing the first process to wake up and release the spinlock so the second process can continue, it is basically a deadlock. This happens since grabbing an spinlocks also disables interrupts and this is required to synchronize threads with interrupt handlers. 2. For the interrupt context case, I think when processing interrupt, kernel can be preempted in principle. But, this really increases the interrupt processing time which further cause longer response time and data missing in device. Except that, is there any other reasons? The principal reason is quite simple, processes have an associated task_struct and get executed when the scheduler chose to run it but interrupt handlers are executed due an event (an interrupt happened on the registered IRQ line). So if you preempt an interrupt handler and schedule a process instead, how could you execute the interrupt handler again? they don't have an associated task_struct since they are not user-space process 3. Kernel is responsible for prohibiiting passive process switches, namely preemption, in the above cases. But, It seems that it does not take care of active process swtiches, namely yield. For example, some code in a critical section protected by a spinlock can invoke schedule() to switch process passively. Is this the case? Right, the kernel just avoid process switching by disabling preemption but you could still call schedule() or do a function call that sleeps like allocating big chunks of memory with GFP_KERNEL flag instead of GFP_ATOMIC, this is indeed a bug in the same way that is a bug dereferencing a NULL pointer. On the kernel hacking section of the Kbuild configuration you can find many kconfig options to enable different debug facilities that helps you detect these scenarios and avoid deadlock. Hope it helps, Javier ___ Kernelnewbies mailing list Kernelnewbies@kernelnewbies.org http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies
Re: Why can not processes switch in atomic context?
On Wed, Jul 4, 2012 at 1:51 PM, Javier Martinez Canillas martinez.jav...@gmail.com wrote: On Tue, Jul 3, 2012 at 4:24 PM, Parmenides mobile.parmeni...@gmail.com wrote: Hi, It is said that kernel can not be preempted in interrupt context and when it is in a critical section protected by a spin lock. 1. For the spinlock case, it is easy to get if preemption is allowed in critical section, the purpose of protection provided by spinlock can not be achieved readily. A process cannot be preempted nor sleep while holding a spinlock due spinlocks behavior. If a process grabs a spinlock and goes to sleep before releasing it. A second process (or an interrupt handler) that to grab the spinlock will busy wait. On an uniprocessor machine the second process will lock the CPU not allowing the first process to wake up and release the spinlock so the second process can continue, it is basically a deadlock. This happens since grabbing an spinlocks also disables interrupts and this is required to synchronize threads with interrupt handlers. 2. For the interrupt context case, I think when processing interrupt, kernel can be preempted in principle. But, this really increases the interrupt processing time which further cause longer response time and data missing in device. Except that, is there any other reasons? The principal reason is quite simple, processes have an associated task_struct and get executed when the scheduler chose to run it but interrupt handlers are executed due an event (an interrupt happened on the registered IRQ line). So if you preempt an interrupt handler and schedule a process instead, how could you execute the interrupt handler again? they don't have an associated task_struct since they are not user-space process In case of threaded handlers also? 3. Kernel is responsible for prohibiiting passive process switches, namely preemption, in the above cases. But, It seems that it does not take care of active process swtiches, namely yield. For example, some code in a critical section protected by a spinlock can invoke schedule() to switch process passively. Is this the case? Right, the kernel just avoid process switching by disabling preemption but you could still call schedule() or do a function call that sleeps like allocating big chunks of memory with GFP_KERNEL flag instead of GFP_ATOMIC, this is indeed a bug in the same way that is a bug dereferencing a NULL pointer. On the kernel hacking section of the Kbuild configuration you can find many kconfig options to enable different debug facilities that helps you detect these scenarios and avoid deadlock. Hope it helps, Javier ___ Kernelnewbies mailing list Kernelnewbies@kernelnewbies.org http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies ___ Kernelnewbies mailing list Kernelnewbies@kernelnewbies.org http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies
Re: Why can not processes switch in atomic context?
在 2012-7-3，22:26，Parmenides mobile.parmeni...@gmail.com 写道： Hi, It is said that kernel can not be preempted in interrupt context and when it is in a critical section protected by a spin lock. 1. For the spinlock case, it is easy to get if preemption is allowed in critical section, the purpose of protection provided by spinlock can not be achieved readily. i think disable preemption in spinlock is not for safe, it is For efficiency. Spinlock should exit as soon as possible. If tank1 get spinlock and goto sleep for 100 seconds before Release spinlock, task2 which requests the same spinlock Should wait for 100 seconds, for this example, mutex should be used instead of spinlock. 2. For the interrupt context case, I think when processing interrupt, kernel can be preempted in principle. But, this really increases the interrupt processing time which further cause longer response time and data missing in device. Except that, is there any other reasons? You are right, kernel can schedule in interrupt context, but that will delay the completion of interrupt, which does not make sense, so kernel will not do like this. Note! Kernel can, but should not schedule in interrupt context. 3. Kernel is responsible for prohibiiting passive process switches, namely preemption, in the above cases. But, It seems that it does not take care of active process swtiches, namely yield. For example, some code in a critical section protected by a spinlock can invoke schedule() to switch process passively. Is this the case? Kernel developer can do anything they want to. You are right, this can be done, but should not happen at all. ___ Kernelnewbies mailing list Kernelnewbies@kernelnewbies.org http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies ___ Kernelnewbies mailing list Kernelnewbies@kernelnewbies.org http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies
Re: Why can not processes switch in atomic context?
Hi, On Wed, Jul 4, 2012 at 3:44 AM, 弋天 卜 bu...@live.cn wrote: 在 2012-7-3，22:26，Parmenides mobile.parmeni...@gmail.com 写道： ...snip... 1. For the spinlock case, it is easy to get if preemption is allowed in critical section, the purpose of protection provided by spinlock can not be achieved readily. i think disable preemption in spinlock is not for safe, it is For efficiency. Spinlock should exit as soon as possible. If tank1 get spinlock and goto sleep for 100 seconds before Release spinlock, task2 which requests the same spinlock Should wait for 100 seconds, for this example, mutex should be used instead of spinlock. Unless, of course, the interrupt that fired tried to acquire the spinlock it preempted, in which case you would have deadlock, even on an SMP system, if the same processor happened to be used for both. 2. For the interrupt context case, I think when processing interrupt, kernel can be preempted in principle. But, this really increases the interrupt processing time which further cause longer response time and data missing in device. Except that, is there any other reasons? You are right, kernel can schedule in interrupt context, but that will delay the completion of interrupt, which does not make sense, so kernel will not do like this. Note! Kernel can, but should not schedule in interrupt context. Scheduling in an interrupt can hang the system. Lets suppose that A acquires a resource, disabled interrupts and calls schedule. Then B gets scheduled and wants the resource that A is holding so it blocks. But since interrupts were disabled, the timer tick will never fire and you're back into a deadlock situation again. -- Dave Hylands Shuswap, BC, Canada http://www.davehylands.com ___ Kernelnewbies mailing list Kernelnewbies@kernelnewbies.org http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies
RE: Why can not processes switch in atomic context?
Date: Wed, 4 Jul 2012 08:12:25 -0700 Subject: Re: Why can not processes switch in atomic context? From: dhyla...@gmail.com To: bu...@live.cn CC: mobile.parmeni...@gmail.com; kernelnewbies@kernelnewbies.org Hi, On Wed, Jul 4, 2012 at 3:44 AM, 弋天 卜 bu...@live.cn wrote: 在 2012-7-3，22:26，Parmenides mobile.parmeni...@gmail.com 写道： ...snip... 1. For the spinlock case, it is easy to get if preemption is allowed in critical section, the purpose of protection provided by spinlock can not be achieved readily. i think disable preemption in spinlock is not for safe, it is For efficiency. Spinlock should exit as soon as possible. If tank1 get spinlock and goto sleep for 100 seconds before Release spinlock, task2 which requests the same spinlock Should wait for 100 seconds, for this example, mutex should be used instead of spinlock. Unless, of course, the interrupt that fired tried to acquire the spinlock it preempted, in which case you would have deadlock, even on an SMP system, if the same processor happened to be used for both. we are talking about schedule in spinlock, but not the synchronization between normal process and interrupt. in your example, it is kernel developer's responsibility to make this correct, this is why kernel give us the API spin_lock_irqsave(). if normal process and interrupt handler will acquire the same spinlock, please use spin_lock_irqsave() instead of spin_lock(). 2. For the interrupt context case, I think when processing interrupt, kernel can be preempted in principle. But, this really increases the interrupt processing time which further cause longer response time and data missing in device. Except that, is there any other reasons? You are right, kernel can schedule in interrupt context, but that will delay the completion of interrupt, which does not make sense, so kernel will not do like this. Note! Kernel can, but should not schedule in interrupt context. Scheduling in an interrupt can hang the system. Lets suppose that A acquires a resource, disabled interrupts and calls schedule. Then B gets scheduled and wants the resource that A is holding so it blocks. But since interrupts were disabled, the timer tick will never fire and you're back into a deadlock situation again. in your example here, i think A is the interrupt handler, rather than a normal task, because we are talking about scheduleing in interrupt handler. if A calls schedule, the interrupted task, let's suppose it is usb task, will go to sleep. let's suppose cpu0 handles the interrupt handler A, and usb task runs on cpu0 also. after A go to sleep, which equals the usb task go to sleep, there must be another new task will be waked up by function switch_to(), let's suppose this lucky guy is BT task, the interrupt on cpu0 could be re-enalbed by function svc_exit() before BT task(or some other tasks) runs. i think disable interrupt on cpu0 inside A does not mean the interrupt can only be re-enabled by A. i am not saying it is good to schedule inside interrupt handler, i just want to make it clear that this can be done, but is not a good behavior and make no sense. -- Dave Hylands Shuswap, BC, Canada http://www.davehylands.com ___ Kernelnewbies mailing list Kernelnewbies@kernelnewbies.org http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies ___ Kernelnewbies mailing list Kernelnewbies@kernelnewbies.org http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies
Re: Why can not processes switch in atomic context?
Thanks for all responses to my question. As far as this quesiton be concerned, I am more interested in why we should do somthing rather than merely we should do it. The discussions have made it more and more clear. Thanks again. ___ Kernelnewbies mailing list Kernelnewbies@kernelnewbies.org http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies
Re: Why can not processes switch in atomic context?
Hi, On Tue, Jul 3, 2012 at 9:24 AM, Parmenides mobile.parmeni...@gmail.comwrote: Hi, It is said that kernel can not be preempted in interrupt context and when it is in a critical section protected by a spin lock. 1. For the spinlock case, it is easy to get if preemption is allowed in critical section, the purpose of protection provided by spinlock can not be achieved readily. 2. For the interrupt context case, I think when processing interrupt, kernel can be preempted in principle. But, this really increases the interrupt processing time which further cause longer response time and data missing in device. Except that, is there any other reasons? 3. Kernel is responsible for prohibiiting passive process switches, namely preemption, in the above cases. But, It seems that it does not take care of active process swtiches, namely yield. For example, some code in a critical section protected by a spinlock can invoke schedule() to switch process passively. Is this the case? Well one should not hold a spinlock and call schedule(). If at all you want to yield, release the spinlock and yield. Hope that answers your question. ___ Kernelnewbies mailing list Kernelnewbies@kernelnewbies.org http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies ___ Kernelnewbies mailing list Kernelnewbies@kernelnewbies.org http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies
Re: Why can not processes switch in atomic context?
On Jul 3, 2012 7:55 PM, Parmenides mobile.parmeni...@gmail.com wrote: Hi, It is said that kernel can not be preempted in interrupt context and when it is in a critical section protected by a spin lock. 1. For the spinlock case, it is easy to get if preemption is allowed in critical section, the purpose of protection provided by spinlock can not be achieved readily. I don't know about Linux kernel. But the very prime purpose of a spin lock is to synchronize in the presence of pre-emption. Are u talking about implementing spin lock in the absence of an atomic hardware instruction? 2. For the interrupt context case, I think when processing interrupt, kernel can be preempted in principle. But, this really increases the interrupt processing time which further cause longer response time and data missing in device. Except that, is there any other reasons? 3. Kernel is responsible for prohibiiting passive process switches, namely preemption, in the above cases. But, It seems that it does not take care of active process swtiches, namely yield. For example, some code in a critical section protected by a spinlock can invoke schedule() to switch process passively. Is this the case? ___ Kernelnewbies mailing list Kernelnewbies@kernelnewbies.org http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies ___ Kernelnewbies mailing list Kernelnewbies@kernelnewbies.org http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies
Re: Why can not processes switch in atomic context?
On Tue, 2012-07-03 at 22:24 +0800, Parmenides wrote: Hi, It is said that kernel can not be preempted in interrupt context and when it is in a critical section protected by a spin lock 1. For the spinlock case, it is easy to get if preemption is allowed in critical section, the purpose of protection provided by spinlock can not be achieved readily. I don't know what you mean here.Please clarify. 2. For the interrupt context case, I think when processing interrupt, kernel can be preempted in principle. But, this really increases the interrupt processing time which further cause longer response time and data missing in device. Except that, is there any other reasons? data missing in device.Can you elaborate that? Stack space is pretty limited in ISR context.Does that give you a clue? 3. Kernel is responsible for prohibiiting passive process switches, namely preemption, in the above cases. But, It seems that it does not take care of active process swtiches, namely yield. For example, some code in a critical section protected by a spinlock can invoke schedule() to switch process passively. Is this the case? I don't understand this question but we don't switch when we are holding spinlock as that will jeopardize the integrity of the system i.e. suppose you slept while holding spinlock.What would happen? ___ Kernelnewbies mailing list Kernelnewbies@kernelnewbies.org http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies ___ Kernelnewbies mailing list Kernelnewbies@kernelnewbies.org http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies
Re: Why can not processes switch in atomic context?
Always CC Kernelnewbies. On Wed, Jul 4, 2012 at 12:52 AM, Parmenides mobile.parmeni...@gmail.com wrote: 1. For the spinlock case, it is easy to get if preemption is allowed in critical section, the purpose of protection provided by spinlock can not be achieved readily. I don't know what you mean here.Please clarify. Sorry, this is not a problem really. :-) A critical section protected by a spinlock is atomic. So, preemption is prohibited when kernel enter this kind of critical section. I just want make sure it is the case. 2. For the interrupt context case, I think when processing interrupt, kernel can be preempted in principle. But, this really increases the interrupt processing time which further cause longer response time and data missing in device. Except that, is there any other reasons? data missing in device.Can you elaborate that? Stack space is pretty limited in ISR context.Does that give you a clue? data missing: For example, if a interrupt sent by Ether card can not processed for a long time, some frames received from the cable might be discarded owing to its limit buffer on card. Actually, every process has its kernel stack. If a preemption occurs during interrupt processing, the stack of another process is used. So, I think stack space is not a problem at all. https://lkml.org/lkml/2007/8/3/2 this thinks otherwise.Please read this as stack space is one of the reasons why sleeping is not allowed in ISR context. 3. Kernel is responsible for prohibiiting passive process switches, namely preemption, in the above cases. But, It seems that it does not take care of active process swtiches, namely yield. For example, some code in a critical section protected by a spinlock can invoke schedule() to switch process passively. Is this the case? I don't understand this question but we don't switch when we are holding spinlock as that will jeopardize the integrity of the system i.e. suppose you slept while holding spinlock.What would happen? So, kernel maintains its integrity relying on that we don't yield while in interrupt handler or critical section protected by a spinlock. If we do so, kernel do nothing at all. Why doesn't it refuse schedule when it finds out that we yield in a improper context. AFAIK it prints some kind of warning message.Why don't you try this? Call schedule in ISR context(try in threaded ISR) and share with us the results. ___ Kernelnewbies mailing list Kernelnewbies@kernelnewbies.org http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies
