Re: [PATCH v2 0/6] atmel_serial: Cleanups, irq handler splitup & DMA
On Wed, 19 Dec 2007 21:32:44 +0100 "Remy Bohmer" <[EMAIL PROTECTED]> wrote: > > We need to fix the break- and error handling though. But my vacation > > starts tomorrow, so I probably won't be able to fix it until next > > year. > > In that case, I wish you very good holiday, and a happy new-year. > It was very pleasant working with you on this driver. Thanks. It was a pleasure working with you too :-) > I probably see you again on the list next year. I think you can count on that. The oops fix needs to be folded into the patchset somehow, so I'll probably send out a new series before it's ready for merging (unless someone beats me to it.) We should probably check if the DMA RX path needs the same treatment as well. Haavard -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH v2 0/6] atmel_serial: Cleanups, irq handler splitup DMA
On Wed, 19 Dec 2007 21:32:44 +0100 Remy Bohmer [EMAIL PROTECTED] wrote: We need to fix the break- and error handling though. But my vacation starts tomorrow, so I probably won't be able to fix it until next year. In that case, I wish you very good holiday, and a happy new-year. It was very pleasant working with you on this driver. Thanks. It was a pleasure working with you too :-) I probably see you again on the list next year. I think you can count on that. The oops fix needs to be folded into the patchset somehow, so I'll probably send out a new series before it's ready for merging (unless someone beats me to it.) We should probably check if the DMA RX path needs the same treatment as well. Haavard -- To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH v2 0/6] atmel_serial: Cleanups, irq handler splitup & DMA
| The following patchset cleans up the atmel_serial driver a bit, | moves a significant portion of the interrupt handler into a tasklet, | and adds DMA support. This is the result of a combined effort by Chip | Coldwell, Remy Bohmer and me. The patches should apply cleanly onto | Linus' latest git tree. | | With DMA, I see transfer rates around 92 kbps when transferring a big | file using ZModem (both directions are roughly the same.) | The start and stop bits will use 20 % of the bit rate so the teoretical max throughput at 115,200 BAUD is 115,200 * 0,8 = 92160 bps. What would be interesting is to figure out if we can get reliable transmission at higher frequencies - 230/460/921 kbps. | Note that break and error handling doesn't work too well with DMA | enabled. This is a common problem with all the efforts I've seen | adding DMA support to this driver (including my own). The PDC error | handling also accesses icount without locking. I'm tempted to just | ignore the problem for now and hopefully come up with a solution | later. Have told the guys responsible for the USART IP block we need to improve the H/W for error control for the last 3-4 years. Why not at 12 gates/sq mm, add error counters??? Best Regards Ulf Samuelsson -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH v2 0/6] atmel_serial: Cleanups, irq handler splitup & DMA
Hello, > preempt-rt can disable interrupts for 300 us? I am not sure if it really is an interrupt lock that long, but on these ARM cores I still run into latencies that large. I tried using latency_trace to figure out where they come from, but that tool even reports interrupt locks up to 10 ms while those surely do not exist (the 2nd in size reported is about 300us). So, those tools are buggy on AT91 as well, and I do not trust them completely yet.. (They even do not compile cleanly, conflicts in headers, warnings etc. Maybe those tools are not used very much outside x86) So, I still have a long way to go to make Preempt-RT to run on these cores just as good as preempt-rt runs on X86, although I am sure that I never will reach the <30us as I have on x86. > If so, then I guess there's really no way to avoid a few overruns. And if DBGU is used as a terminal, it should be no problem at all, or someone has to slow down the baudrate a bit. I only get this overrun under very heavy system load (CPU load 99%, with one single thread running at RT-prio 99, taking chunks away of 99/100ms.) and then throw in a big file of many MBs in the terminal. But, as long as the read part still runs in hard-irq context, it should keep up, unless interrupts are locked too long. > > Notice that without these interrupt handler splitup, it was much, much > > much worse... > > Ok, that's good I guess. Yep, that _is_ good. > > So, for me it is not a big deal, because it is just a terminal, and > > with my shaky fingers I usually do not type that fast ;-)) > If you do, you just need to switch to one of the USARTs instead of the > DBGU so that you can use DMA :-) :-) > We need to fix the break- and error handling though. But my vacation > starts tomorrow, so I probably won't be able to fix it until next year. In that case, I wish you very good holiday, and a happy new-year. It was very pleasant working with you on this driver. I probably see you again on the list next year. Kind Regards, Remy -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH v2 0/6] atmel_serial: Cleanups, irq handler splitup & DMA
On Wed, 19 Dec 2007 17:59:09 +0100 "Remy Bohmer" <[EMAIL PROTECTED]> wrote: > > > but I noticed that I sometimes get an input overrun (ttyS0: 1 > > > input overrun(s) ) during stress conditions. > > > This is something I did not notice before, maybe it was already there, > > > or has something changed in this area that it is now more sensitive > > > for this? > > Hmm...is this with or without DMA? > > DBGU is without DMA. Right. > > If it's without DMA, something very strange is going on -- the non-DMA > > RX code is almost the only thing left in the hardirq handler, so I > > would really expect overruns to be much less likely to occur now than > > before... > > As mentioned, maybe it was already there, but I did not run into it > earlier. I have to figure that out. But, at 115200 and a 1 byte > receive-'fifo' on DBGU, and still interrupt locks somewhere in the > tree up to 300us, it is a simple calculation that we can run into > overrun conditions... preempt-rt can disable interrupts for 300 us? If so, then I guess there's really no way to avoid a few overruns. > Notice that without these interrupt handler splitup, it was much, much > much worse... Ok, that's good I guess. > So, for me it is not a big deal, because it is just a terminal, and > with my shaky fingers I usually do not type that fast ;-)) If you do, you just need to switch to one of the USARTs instead of the DBGU so that you can use DMA :-) We need to fix the break- and error handling though. But my vacation starts tomorrow, so I probably won't be able to fix it until next year. Haavard -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH v2 0/6] atmel_serial: Cleanups, irq handler splitup & DMA
> > but I noticed that I sometimes get an input overrun (ttyS0: 1 > > input overrun(s) ) during stress conditions. > > This is something I did not notice before, maybe it was already there, > > or has something changed in this area that it is now more sensitive > > for this? > Hmm...is this with or without DMA? DBGU is without DMA. > If it's without DMA, something very strange is going on -- the non-DMA > RX code is almost the only thing left in the hardirq handler, so I > would really expect overruns to be much less likely to occur now than > before... As mentioned, maybe it was already there, but I did not run into it earlier. I have to figure that out. But, at 115200 and a 1 byte receive-'fifo' on DBGU, and still interrupt locks somewhere in the tree up to 300us, it is a simple calculation that we can run into overrun conditions... Notice that without these interrupt handler splitup, it was much, much much worse... So, for me it is not a big deal, because it is just a terminal, and with my shaky fingers I usually do not type that fast ;-)) Remy Kind Regards, Remy -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH v2 0/6] atmel_serial: Cleanups, irq handler splitup & DMA
On Wed, 19 Dec 2007 17:40:44 +0100 "Remy Bohmer" <[EMAIL PROTECTED]> wrote: > But... I tried that patch, and it works a lot better, no oopses > anymore, Good :) > but I noticed that I sometimes get an input overrun (ttyS0: 1 > input overrun(s) ) during stress conditions. > This is something I did not notice before, maybe it was already there, > or has something changed in this area that it is now more sensitive > for this? Hmm...is this with or without DMA? If it's with DMA, perhaps we should increase the size of the RX buffer to compensate for moving the DMA RX code into tasklet context? If it's without DMA, something very strange is going on -- the non-DMA RX code is almost the only thing left in the hardirq handler, so I would really expect overruns to be much less likely to occur now than before... Haavard -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH v2 0/6] atmel_serial: Cleanups, irq handler splitup & DMA
Hello Haavard, > Could you try the patch below? It's a bit strange that you got an oops > though... It is not really strange... spinlocks are mutexes on preempt-rt, and recursive mutex locking is not allowed, this is one differences with the mainline spinlock. But... I tried that patch, and it works a lot better, no oopses anymore, but I noticed that I sometimes get an input overrun (ttyS0: 1 input overrun(s) ) during stress conditions. This is something I did not notice before, maybe it was already there, or has something changed in this area that it is now more sensitive for this? Kind Regards, Remy 2007/12/19, Haavard Skinnemoen <[EMAIL PROTECTED]>: > On Wed, 19 Dec 2007 16:57:04 +0100 > "Remy Bohmer" <[EMAIL PROTECTED]> wrote: > > > Hello Haavard, > > > > Sorry.. But I get an Oops on Preempt-RT with the latest set of > > patches. I did not see it earlier today with the other set of patches. > > Hmm...from the backtrace, it looks like lock recursion -- port->lock is > held for the whole duration of the tasklet, but we somehow end up in > uart_start(), which grabs the lock again. > > Could you try the patch below? It's a bit strange that you got an oops > though... > > Haavard > > diff --git a/drivers/serial/atmel_serial.c b/drivers/serial/atmel_serial.c > index 7967054..948c643 100644 > --- a/drivers/serial/atmel_serial.c > +++ b/drivers/serial/atmel_serial.c > @@ -666,7 +666,13 @@ static void atmel_rx_from_ring(struct uart_port *port) > uart_insert_char(port, status, ATMEL_US_OVRE, c.ch, flg); > } > > + /* > +* Drop the lock here since it might end up calling > +* uart_start(), which takes the lock. > +*/ > + spin_unlock(>lock); > tty_flip_buffer_push(port->info->tty); > + spin_lock(>lock); > } > > static void atmel_rx_from_dma(struct uart_port *port) > -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH v2 0/6] atmel_serial: Cleanups, irq handler splitup & DMA
On Wed, 19 Dec 2007 16:57:04 +0100 "Remy Bohmer" <[EMAIL PROTECTED]> wrote: > Hello Haavard, > > Sorry.. But I get an Oops on Preempt-RT with the latest set of > patches. I did not see it earlier today with the other set of patches. Hmm...from the backtrace, it looks like lock recursion -- port->lock is held for the whole duration of the tasklet, but we somehow end up in uart_start(), which grabs the lock again. Could you try the patch below? It's a bit strange that you got an oops though... Haavard diff --git a/drivers/serial/atmel_serial.c b/drivers/serial/atmel_serial.c index 7967054..948c643 100644 --- a/drivers/serial/atmel_serial.c +++ b/drivers/serial/atmel_serial.c @@ -666,7 +666,13 @@ static void atmel_rx_from_ring(struct uart_port *port) uart_insert_char(port, status, ATMEL_US_OVRE, c.ch, flg); } + /* +* Drop the lock here since it might end up calling +* uart_start(), which takes the lock. +*/ + spin_unlock(>lock); tty_flip_buffer_push(port->info->tty); + spin_lock(>lock); } static void atmel_rx_from_dma(struct uart_port *port) -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH v2 0/6] atmel_serial: Cleanups, irq handler splitup & DMA
Hello Haavard, Sorry.. But I get an Oops on Preempt-RT with the latest set of patches. I did not see it earlier today with the other set of patches. Here it is... Remy [EMAIL PROTECTED]:~ [ 13.760735] Unable to handle kernel NULL pointer dereference at virtual address [ 13.760735] pgd = c0004000 [ 13.760735] [] *pgd= [ 13.760735] stopped custom tracer. [ 13.760735] Internal error: Oops: 817 [#1] PREEMPT [ 13.760735] CPU: 0Not tainted (2.6.23.12-rt14 #76) [ 13.760735] PC is at rt_spin_lock_slowlock+0x84/0x1c8 [ 13.760735] LR is at rt_spin_lock_slowlock+0x54/0x1c8 [ 13.760735] pc : []lr : []psr: 6093 [ 13.760735] sp : c02e1d38 ip : c02dd5a0 fp : c02e1d94 [ 13.760735] r10: c3c3cc21 r9 : c001ec58 r8 : 0001 [ 13.760735] r7 : c001e800 r6 : c0212aec r5 : c02e r4 : a013 [ 13.760735] r3 : c02dd5a0 r2 : c02dd5a0 r1 : c0212afc r0 : [ 13.760735] Flags: nZCv IRQs off FIQs on Mode SVC_32 ISA ARM Segment kernel [ 13.760735] Control: c000717f Table: 23d98000 DAC: 0017 [ 13.760735] Process softirq-tasklet (pid: 9, stack limit = 0xc02e0268) [ 13.760735] Stack: (0xc02e1d38 to 0xc02e2000) [ 13.760735] 1d20: 0001 [ 13.760735] 1d40: c02e1d5c c02e1d50 001f c02dcd20 c02e1d94 c02e1d68 [ 13.760735] 1d60: c0029844 c02e1da4 c02e c0212aec c001e800 c001e800 0001 [ 13.760735] 1d80: c001ec58 c3c3cc21 c02e1dac c02e1d98 c01a4b28 c01a4680 c001e800 c0212aec [ 13.760735] 1da0: c02e1dbc c02e1db0 c01a4b44 c01a4ad8 c02e1dd4 c02e1dc0 c010e298 c01a4b44 [ 13.760735] 1dc0: 001b c3c3cc1c c02e1de4 c02e1dd8 c010e2bc c010e288 c02e1ea4 c02e1de8 [ 13.760735] 1de0: c01007e0 c010e2bc c3c3cc22 c3c3cd22 c0035d18 c00359b4 [ 13.760735] 1e00: 0001 c01f40a4 c02e1e2c c02e1e18 c0035f7c c0035cf4 0001 c01f40a4 [ 13.760735] 1e20: c02e1e3c c02e1e30 c0036230 c02e c02e1e54 c02e1e40 c01a4af8 c02e [ 13.760735] 1e40: c02e1e64 c02e1e50 c01a4af8 c002a4d8 c001e800 c001e92c c02e1e74 c02e1e68 [ 13.760735] 1e60: c01a4b44 c01a4ad8 c02e1e8c c02e c02e1e94 c02e1e80 c01a4af8 c3c3cd1c [ 13.760735] 1e80: c3c3cc1c c001e800 c3c3cc00 0005 c001e92c 0001 c02e1edc c02e1ea8 [ 13.760735] 1ea0: c00fb38c c00ffa64 c02e1ecc c001e80c c01a4af8 c001e800 c001e92c c001e800 [ 13.760735] 1ec0: 001b 0001 c02e1efe 0004 c02e1ef4 c02e1ee0 c00fb450 c00fb2a0 [ 13.760735] 1ee0: c0212aec 1a1b c02e1f2c c02e1ef8 c011188c c00fb424 0015 1b002da0 [ 13.760735] 1f00: c02e1f24 c0212bcc 000f4240 0004 0020 c0209e14 [ 13.760735] 1f20: c02e1f54 c02e1f30 c0035858 c01112b0 c0209c20 c0209c20 c0209c20 ffdf [ 13.760735] 1f40: c0209e20 c02e c02e1f64 c02e1f58 c003599c c00357a0 c02e1fd4 c02e1f68 [ 13.760735] 1f60: c00363b4 c0035974 c02cb580 61741f94 656c6b73 00302f74 c003e3e4 [ 13.760735] 1f80: c02dd9e0 0002 c02e1fbc c02e1f98 c01a31b0 c0051154 c02e [ 13.760735] 1fa0: c0209e14 001f c02e c0209e14 c003623c [ 13.760735] 1fc0: c02e1ff4 c02e1fd8 c004510c c003624c [ 13.760735] 1fe0: c02e1ff8 c00336d8 c00450bc 7475706e 74616420 [ 13.760735] Backtrace: [ 13.760735] [] (rt_spin_lock_slowlock+0x0/0x1c8) from [] (__rt_spin_lock+0x60/0x6c) [ 13.760735] [] (__rt_spin_lock+0x0/0x6c) from [] (rt_spin_lock+0x10/0x14) [ 13.760735] r5:c0212aec r4:c001e800 [ 13.760735] [] (rt_spin_lock+0x0/0x14) from [] (uart_start+0x20/0x34) [ 13.760735] [] (uart_start+0x0/0x34) from [] (uart_flush_chars+0x10/0x14) [ 13.760735] r5:c3c3cc1c r4:001b [ 13.760735] [] (uart_flush_chars+0x0/0x14) from [] (n_tty_receive_buf+0xd8c/0xe70) [ 13.760735] [] (n_tty_receive_buf+0x0/0xe70) from [] (flush_to_ldisc+0xfc/0x184) [ 13.760735] [] (flush_to_ldisc+0x0/0x184) from [] (tty_flip_buffer_push+0x3c/0x40) [ 13.760735] [] (tty_flip_buffer_push+0x0/0x40) from [] (atmel_tasklet_func+0x5ec/0x5fc) [ 13.760735] r5:1a1b r4:c0212aec [ 13.760735] [] (atmel_tasklet_func+0x0/0x5fc) from [] (__tasklet_action+0xc8/0x194) [ 13.760735] [] (__tasklet_action+0x0/0x194) from [] (tasklet_action+0x38/0x40) [ 13.760735] r8:c02e r7:c0209e20 r6:ffdf r5:c0209c20 r4:c0209c20 [ 13.760735] [] (tasklet_action+0x0/0x40) from [] (ksoftirqd+0x178/0x228) [ 13.760735] [] (ksoftirqd+0x0/0x228) from [] (kthread+0x60/0x94) [ 13.760735] [] (kthread+0x0/0x94) from [] (do_exit+0x0/0x7e0) [ 13.760735] r6: r5: r4: [ 13.760735] Code: e5963010 e595200c e3c33003 e1530002 (0580) [ 14.189445] note: softirq-tasklet[9] exited with preempt_count 1 [ 14.194328] BUG: sleeping function called from invalid context softirq-tasklet(9) at kernel/rtmutex.c:637 [ 14.204094] in_atomic():1 [0001], irqs_disabled():0 [ 14.208977] [] (dump_stack+0x0/0x14) from [] (__might_sleep+0xe0/0x104) [
Re: [PATCH v2 0/6] atmel_serial: Cleanups, irq handler splitup DMA
Hello Haavard, Sorry.. But I get an Oops on Preempt-RT with the latest set of patches. I did not see it earlier today with the other set of patches. Here it is... Remy [EMAIL PROTECTED]:~ [ 13.760735] Unable to handle kernel NULL pointer dereference at virtual address [ 13.760735] pgd = c0004000 [ 13.760735] [] *pgd= [ 13.760735] stopped custom tracer. [ 13.760735] Internal error: Oops: 817 [#1] PREEMPT [ 13.760735] CPU: 0Not tainted (2.6.23.12-rt14 #76) [ 13.760735] PC is at rt_spin_lock_slowlock+0x84/0x1c8 [ 13.760735] LR is at rt_spin_lock_slowlock+0x54/0x1c8 [ 13.760735] pc : [c01a46f4]lr : [c01a46c4]psr: 6093 [ 13.760735] sp : c02e1d38 ip : c02dd5a0 fp : c02e1d94 [ 13.760735] r10: c3c3cc21 r9 : c001ec58 r8 : 0001 [ 13.760735] r7 : c001e800 r6 : c0212aec r5 : c02e r4 : a013 [ 13.760735] r3 : c02dd5a0 r2 : c02dd5a0 r1 : c0212afc r0 : [ 13.760735] Flags: nZCv IRQs off FIQs on Mode SVC_32 ISA ARM Segment kernel [ 13.760735] Control: c000717f Table: 23d98000 DAC: 0017 [ 13.760735] Process softirq-tasklet (pid: 9, stack limit = 0xc02e0268) [ 13.760735] Stack: (0xc02e1d38 to 0xc02e2000) [ 13.760735] 1d20: 0001 [ 13.760735] 1d40: c02e1d5c c02e1d50 001f c02dcd20 c02e1d94 c02e1d68 [ 13.760735] 1d60: c0029844 c02e1da4 c02e c0212aec c001e800 c001e800 0001 [ 13.760735] 1d80: c001ec58 c3c3cc21 c02e1dac c02e1d98 c01a4b28 c01a4680 c001e800 c0212aec [ 13.760735] 1da0: c02e1dbc c02e1db0 c01a4b44 c01a4ad8 c02e1dd4 c02e1dc0 c010e298 c01a4b44 [ 13.760735] 1dc0: 001b c3c3cc1c c02e1de4 c02e1dd8 c010e2bc c010e288 c02e1ea4 c02e1de8 [ 13.760735] 1de0: c01007e0 c010e2bc c3c3cc22 c3c3cd22 c0035d18 c00359b4 [ 13.760735] 1e00: 0001 c01f40a4 c02e1e2c c02e1e18 c0035f7c c0035cf4 0001 c01f40a4 [ 13.760735] 1e20: c02e1e3c c02e1e30 c0036230 c02e c02e1e54 c02e1e40 c01a4af8 c02e [ 13.760735] 1e40: c02e1e64 c02e1e50 c01a4af8 c002a4d8 c001e800 c001e92c c02e1e74 c02e1e68 [ 13.760735] 1e60: c01a4b44 c01a4ad8 c02e1e8c c02e c02e1e94 c02e1e80 c01a4af8 c3c3cd1c [ 13.760735] 1e80: c3c3cc1c c001e800 c3c3cc00 0005 c001e92c 0001 c02e1edc c02e1ea8 [ 13.760735] 1ea0: c00fb38c c00ffa64 c02e1ecc c001e80c c01a4af8 c001e800 c001e92c c001e800 [ 13.760735] 1ec0: 001b 0001 c02e1efe 0004 c02e1ef4 c02e1ee0 c00fb450 c00fb2a0 [ 13.760735] 1ee0: c0212aec 1a1b c02e1f2c c02e1ef8 c011188c c00fb424 0015 1b002da0 [ 13.760735] 1f00: c02e1f24 c0212bcc 000f4240 0004 0020 c0209e14 [ 13.760735] 1f20: c02e1f54 c02e1f30 c0035858 c01112b0 c0209c20 c0209c20 c0209c20 ffdf [ 13.760735] 1f40: c0209e20 c02e c02e1f64 c02e1f58 c003599c c00357a0 c02e1fd4 c02e1f68 [ 13.760735] 1f60: c00363b4 c0035974 c02cb580 61741f94 656c6b73 00302f74 c003e3e4 [ 13.760735] 1f80: c02dd9e0 0002 c02e1fbc c02e1f98 c01a31b0 c0051154 c02e [ 13.760735] 1fa0: c0209e14 001f c02e c0209e14 c003623c [ 13.760735] 1fc0: c02e1ff4 c02e1fd8 c004510c c003624c [ 13.760735] 1fe0: c02e1ff8 c00336d8 c00450bc 7475706e 74616420 [ 13.760735] Backtrace: [ 13.760735] [c01a4670] (rt_spin_lock_slowlock+0x0/0x1c8) from [c01a4b28] (__rt_spin_lock+0x60/0x6c) [ 13.760735] [c01a4ac8] (__rt_spin_lock+0x0/0x6c) from [c01a4b44] (rt_spin_lock+0x10/0x14) [ 13.760735] r5:c0212aec r4:c001e800 [ 13.760735] [c01a4b34] (rt_spin_lock+0x0/0x14) from [c010e298] (uart_start+0x20/0x34) [ 13.760735] [c010e278] (uart_start+0x0/0x34) from [c010e2bc] (uart_flush_chars+0x10/0x14) [ 13.760735] r5:c3c3cc1c r4:001b [ 13.760735] [c010e2ac] (uart_flush_chars+0x0/0x14) from [c01007e0] (n_tty_receive_buf+0xd8c/0xe70) [ 13.760735] [c00ffa54] (n_tty_receive_buf+0x0/0xe70) from [c00fb38c] (flush_to_ldisc+0xfc/0x184) [ 13.760735] [c00fb290] (flush_to_ldisc+0x0/0x184) from [c00fb450] (tty_flip_buffer_push+0x3c/0x40) [ 13.760735] [c00fb414] (tty_flip_buffer_push+0x0/0x40) from [c011188c] (atmel_tasklet_func+0x5ec/0x5fc) [ 13.760735] r5:1a1b r4:c0212aec [ 13.760735] [c01112a0] (atmel_tasklet_func+0x0/0x5fc) from [c0035858] (__tasklet_action+0xc8/0x194) [ 13.760735] [c0035790] (__tasklet_action+0x0/0x194) from [c003599c] (tasklet_action+0x38/0x40) [ 13.760735] r8:c02e r7:c0209e20 r6:ffdf r5:c0209c20 r4:c0209c20 [ 13.760735] [c0035964] (tasklet_action+0x0/0x40) from [c00363b4] (ksoftirqd+0x178/0x228) [ 13.760735] [c003623c] (ksoftirqd+0x0/0x228) from [c004510c] (kthread+0x60/0x94) [ 13.760735] [c00450ac] (kthread+0x0/0x94) from [c00336d8] (do_exit+0x0/0x7e0) [ 13.760735] r6: r5: r4: [ 13.760735] Code: e5963010 e595200c e3c33003 e1530002 (0580) [ 14.189445] note: softirq-tasklet[9] exited with preempt_count 1 [ 14.194328] BUG: sleeping
Re: [PATCH v2 0/6] atmel_serial: Cleanups, irq handler splitup DMA
On Wed, 19 Dec 2007 16:57:04 +0100 Remy Bohmer [EMAIL PROTECTED] wrote: Hello Haavard, Sorry.. But I get an Oops on Preempt-RT with the latest set of patches. I did not see it earlier today with the other set of patches. Hmm...from the backtrace, it looks like lock recursion -- port-lock is held for the whole duration of the tasklet, but we somehow end up in uart_start(), which grabs the lock again. Could you try the patch below? It's a bit strange that you got an oops though... Haavard diff --git a/drivers/serial/atmel_serial.c b/drivers/serial/atmel_serial.c index 7967054..948c643 100644 --- a/drivers/serial/atmel_serial.c +++ b/drivers/serial/atmel_serial.c @@ -666,7 +666,13 @@ static void atmel_rx_from_ring(struct uart_port *port) uart_insert_char(port, status, ATMEL_US_OVRE, c.ch, flg); } + /* +* Drop the lock here since it might end up calling +* uart_start(), which takes the lock. +*/ + spin_unlock(port-lock); tty_flip_buffer_push(port-info-tty); + spin_lock(port-lock); } static void atmel_rx_from_dma(struct uart_port *port) -- To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH v2 0/6] atmel_serial: Cleanups, irq handler splitup DMA
Hello Haavard, Could you try the patch below? It's a bit strange that you got an oops though... It is not really strange... spinlocks are mutexes on preempt-rt, and recursive mutex locking is not allowed, this is one differences with the mainline spinlock. But... I tried that patch, and it works a lot better, no oopses anymore, but I noticed that I sometimes get an input overrun (ttyS0: 1 input overrun(s) ) during stress conditions. This is something I did not notice before, maybe it was already there, or has something changed in this area that it is now more sensitive for this? Kind Regards, Remy 2007/12/19, Haavard Skinnemoen [EMAIL PROTECTED]: On Wed, 19 Dec 2007 16:57:04 +0100 Remy Bohmer [EMAIL PROTECTED] wrote: Hello Haavard, Sorry.. But I get an Oops on Preempt-RT with the latest set of patches. I did not see it earlier today with the other set of patches. Hmm...from the backtrace, it looks like lock recursion -- port-lock is held for the whole duration of the tasklet, but we somehow end up in uart_start(), which grabs the lock again. Could you try the patch below? It's a bit strange that you got an oops though... Haavard diff --git a/drivers/serial/atmel_serial.c b/drivers/serial/atmel_serial.c index 7967054..948c643 100644 --- a/drivers/serial/atmel_serial.c +++ b/drivers/serial/atmel_serial.c @@ -666,7 +666,13 @@ static void atmel_rx_from_ring(struct uart_port *port) uart_insert_char(port, status, ATMEL_US_OVRE, c.ch, flg); } + /* +* Drop the lock here since it might end up calling +* uart_start(), which takes the lock. +*/ + spin_unlock(port-lock); tty_flip_buffer_push(port-info-tty); + spin_lock(port-lock); } static void atmel_rx_from_dma(struct uart_port *port) -- To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH v2 0/6] atmel_serial: Cleanups, irq handler splitup DMA
On Wed, 19 Dec 2007 17:40:44 +0100 Remy Bohmer [EMAIL PROTECTED] wrote: But... I tried that patch, and it works a lot better, no oopses anymore, Good :) but I noticed that I sometimes get an input overrun (ttyS0: 1 input overrun(s) ) during stress conditions. This is something I did not notice before, maybe it was already there, or has something changed in this area that it is now more sensitive for this? Hmm...is this with or without DMA? If it's with DMA, perhaps we should increase the size of the RX buffer to compensate for moving the DMA RX code into tasklet context? If it's without DMA, something very strange is going on -- the non-DMA RX code is almost the only thing left in the hardirq handler, so I would really expect overruns to be much less likely to occur now than before... Haavard -- To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH v2 0/6] atmel_serial: Cleanups, irq handler splitup DMA
but I noticed that I sometimes get an input overrun (ttyS0: 1 input overrun(s) ) during stress conditions. This is something I did not notice before, maybe it was already there, or has something changed in this area that it is now more sensitive for this? Hmm...is this with or without DMA? DBGU is without DMA. If it's without DMA, something very strange is going on -- the non-DMA RX code is almost the only thing left in the hardirq handler, so I would really expect overruns to be much less likely to occur now than before... As mentioned, maybe it was already there, but I did not run into it earlier. I have to figure that out. But, at 115200 and a 1 byte receive-'fifo' on DBGU, and still interrupt locks somewhere in the tree up to 300us, it is a simple calculation that we can run into overrun conditions... Notice that without these interrupt handler splitup, it was much, much much worse... So, for me it is not a big deal, because it is just a terminal, and with my shaky fingers I usually do not type that fast ;-)) Remy Kind Regards, Remy -- To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH v2 0/6] atmel_serial: Cleanups, irq handler splitup DMA
On Wed, 19 Dec 2007 17:59:09 +0100 Remy Bohmer [EMAIL PROTECTED] wrote: but I noticed that I sometimes get an input overrun (ttyS0: 1 input overrun(s) ) during stress conditions. This is something I did not notice before, maybe it was already there, or has something changed in this area that it is now more sensitive for this? Hmm...is this with or without DMA? DBGU is without DMA. Right. If it's without DMA, something very strange is going on -- the non-DMA RX code is almost the only thing left in the hardirq handler, so I would really expect overruns to be much less likely to occur now than before... As mentioned, maybe it was already there, but I did not run into it earlier. I have to figure that out. But, at 115200 and a 1 byte receive-'fifo' on DBGU, and still interrupt locks somewhere in the tree up to 300us, it is a simple calculation that we can run into overrun conditions... preempt-rt can disable interrupts for 300 us? If so, then I guess there's really no way to avoid a few overruns. Notice that without these interrupt handler splitup, it was much, much much worse... Ok, that's good I guess. So, for me it is not a big deal, because it is just a terminal, and with my shaky fingers I usually do not type that fast ;-)) If you do, you just need to switch to one of the USARTs instead of the DBGU so that you can use DMA :-) We need to fix the break- and error handling though. But my vacation starts tomorrow, so I probably won't be able to fix it until next year. Haavard -- To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH v2 0/6] atmel_serial: Cleanups, irq handler splitup DMA
Hello, preempt-rt can disable interrupts for 300 us? I am not sure if it really is an interrupt lock that long, but on these ARM cores I still run into latencies that large. I tried using latency_trace to figure out where they come from, but that tool even reports interrupt locks up to 10 ms while those surely do not exist (the 2nd in size reported is about 300us). So, those tools are buggy on AT91 as well, and I do not trust them completely yet.. (They even do not compile cleanly, conflicts in headers, warnings etc. Maybe those tools are not used very much outside x86) So, I still have a long way to go to make Preempt-RT to run on these cores just as good as preempt-rt runs on X86, although I am sure that I never will reach the 30us as I have on x86. If so, then I guess there's really no way to avoid a few overruns. And if DBGU is used as a terminal, it should be no problem at all, or someone has to slow down the baudrate a bit. I only get this overrun under very heavy system load (CPU load 99%, with one single thread running at RT-prio 99, taking chunks away of 99/100ms.) and then throw in a big file of many MBs in the terminal. But, as long as the read part still runs in hard-irq context, it should keep up, unless interrupts are locked too long. Notice that without these interrupt handler splitup, it was much, much much worse... Ok, that's good I guess. Yep, that _is_ good. So, for me it is not a big deal, because it is just a terminal, and with my shaky fingers I usually do not type that fast ;-)) If you do, you just need to switch to one of the USARTs instead of the DBGU so that you can use DMA :-) :-) We need to fix the break- and error handling though. But my vacation starts tomorrow, so I probably won't be able to fix it until next year. In that case, I wish you very good holiday, and a happy new-year. It was very pleasant working with you on this driver. I probably see you again on the list next year. Kind Regards, Remy -- To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH v2 0/6] atmel_serial: Cleanups, irq handler splitup DMA
| The following patchset cleans up the atmel_serial driver a bit, | moves a significant portion of the interrupt handler into a tasklet, | and adds DMA support. This is the result of a combined effort by Chip | Coldwell, Remy Bohmer and me. The patches should apply cleanly onto | Linus' latest git tree. | | With DMA, I see transfer rates around 92 kbps when transferring a big | file using ZModem (both directions are roughly the same.) | The start and stop bits will use 20 % of the bit rate so the teoretical max throughput at 115,200 BAUD is 115,200 * 0,8 = 92160 bps. What would be interesting is to figure out if we can get reliable transmission at higher frequencies - 230/460/921 kbps. | Note that break and error handling doesn't work too well with DMA | enabled. This is a common problem with all the efforts I've seen | adding DMA support to this driver (including my own). The PDC error | handling also accesses icount without locking. I'm tempted to just | ignore the problem for now and hopefully come up with a solution | later. Have told the guys responsible for the USART IP block we need to improve the H/W for error control for the last 3-4 years. Why not at 12 gates/sq mm, add error counters??? Best Regards Ulf Samuelsson -- To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
