Re: [PATCH v2] serial: stm32: optimize spin lock usage
Hi Johan,
On Fri, Apr 16, 2021 at 4:35 PM Johan Hovold wrote:
>
> On Tue, Apr 13, 2021 at 07:44:39AM +0800, dillon min wrote:
> > Hi Johan, Erwan
> >
> > It seems still a bit of a problem in the current version, not deadlock
> > but access register at the same time.
> >
> > For driver , we should consider it running under smp, let's think
> > about it for this case:
> >
> > static void stm32_usart_console_write(struct console *co, const char *s,
> > unsigned int cnt)
> > {
> > .
> > local_irq_save(flags);
> > if (port->sysrq)
> > locked = 0;
> > .
> > access register cr1, tdr, isr
> > .
> >
> > local_irq_restore(flags);
> > }
> >
> > if port->sysrq is 1, stm32_usart_console_write() just disable local
> > irq response by local_irq_save(), at the time of access register cr1,
> > tdr, isr. an TXE interrupt raised, for other cores(I know stm32
> > mpu/mcu do not have multi cores, just assume it has), it still has a
> > chance to handle interrupt. Then there is no lock to protect the uart
> > register.
>
> Right, the sysrq handling is a bit of a hack.
>
> > changes to below, should be more safe:
> >
> > .
> > if (port->sysrq || oops_in_progress)
> > locked = spin_trylock_irqsave(>lock, flags);
>
> Except that the lock debugging code would detect the attempt at
> recursive locking here and complain loudly on UP.
>
> If you really want to fix this, we have uart_unlock_and_check_sysrq()
> which can be used to defer sysrq processing until the interrupt handler
> has released the lock.
Great, uart_unlock_and_check_sysrq() is fit to fix this. you mean make
the flow like below:
stm32_usart_threaded_interrupt()
spin_lock(>lock);
uart_unlock_and_check_sysrq(port, flags);
...
uart_prepare_sysrq_char();
printk();
stm32_usart_console_write();
locked = spin_trylock_irqsave(>lock); //only
handle oops, normal case
If so, I will submit v3 as you suggested. thanks.
Best regards.
Dillon,
>
> > else
> > spin_lock_irqsave(>lock, flags);
> >
> >
> >
> > if (locked)
> > spin_unlock_irqrestore(>lock, flags);
>
> Johan
Re: [PATCH v2] serial: stm32: optimize spin lock usage
[ Please avoid top-posting. ] On Thu, Apr 15, 2021 at 07:09:14PM +0200, Erwan LE RAY wrote: > Hi Dillon, > > STM32MP151 is mono-core, but both STM32MP153 and STM32MP157 are > dual-core (see > https://www.st.com/content/st_com/en/products/microcontrollers-microprocessors/stm32-arm-cortex-mpus.html). > So your point is fully relevant, thanks. > > ST already fixed the same issue in st-asc.c driver in the past (see > ef49ffd8), because a systematic deadlock was detected with RT kernel. That's not the same issue. The above mentioned commit fixed an issue on *RT* where local_irq_save() should be avoided. > You proposed a first implementation in your patch, and a second one in > the discussion. It seems that your initial proposal (ie your V2 patch) > is the most standard one (implemented in 6 drivers). The second > implementation is implemented by only 1 company. > > It looks that the solution is to avoid locking in the sysrq case and > trylock in the oops_in_progress case (see detailed analysis in > 677fe555cbfb1). > > So your initial patch looks to the right proposal, but it would be safer > if Greg could confirm it. That would only fix the RT issue (and by making the sysrq one slightly worse). Using uart_unlock_and_check_sysrq() would address both issues. Johan
Re: [PATCH v2] serial: stm32: optimize spin lock usage
On Tue, Apr 13, 2021 at 07:44:39AM +0800, dillon min wrote:
> Hi Johan, Erwan
>
> It seems still a bit of a problem in the current version, not deadlock
> but access register at the same time.
>
> For driver , we should consider it running under smp, let's think
> about it for this case:
>
> static void stm32_usart_console_write(struct console *co, const char *s,
> unsigned int cnt)
> {
> .
> local_irq_save(flags);
> if (port->sysrq)
> locked = 0;
> .
> access register cr1, tdr, isr
> .
>
> local_irq_restore(flags);
> }
>
> if port->sysrq is 1, stm32_usart_console_write() just disable local
> irq response by local_irq_save(), at the time of access register cr1,
> tdr, isr. an TXE interrupt raised, for other cores(I know stm32
> mpu/mcu do not have multi cores, just assume it has), it still has a
> chance to handle interrupt. Then there is no lock to protect the uart
> register.
Right, the sysrq handling is a bit of a hack.
> changes to below, should be more safe:
>
> .
> if (port->sysrq || oops_in_progress)
> locked = spin_trylock_irqsave(>lock, flags);
Except that the lock debugging code would detect the attempt at
recursive locking here and complain loudly on UP.
If you really want to fix this, we have uart_unlock_and_check_sysrq()
which can be used to defer sysrq processing until the interrupt handler
has released the lock.
> else
> spin_lock_irqsave(>lock, flags);
>
>
>
> if (locked)
> spin_unlock_irqrestore(>lock, flags);
Johan
Re: [PATCH v2] serial: stm32: optimize spin lock usage
Hi Erwan,
Erwan LE RAY 于2021年4月16日周五 上午1:10写道：
>
> Hi Dillon,
>
> STM32MP151 is mono-core, but both STM32MP153 and STM32MP157 are
> dual-core (see
> https://www.st.com/content/st_com/en/products/microcontrollers-microprocessors/stm32-arm-cortex-mpus.html).
> So your point is fully relevant, thanks.
Thanks.
>
> ST already fixed the same issue in st-asc.c driver in the past (see
> ef49ffd8), because a systematic deadlock was detected with RT kernel.
>
> You proposed a first implementation in your patch, and a second one in
> the discussion. It seems that your initial proposal (ie your V2 patch)
> is the most standard one (implemented in 6 drivers). The second
> implementation is implemented by only 1 company.
>
> It looks that the solution is to avoid locking in the sysrq case and
> trylock in the oops_in_progress case (see detailed analysis in
> 677fe555cbfb1).
Thanks for the detail information. the V2 patch didn't cover this case:
stm32_usart_threaded_interrupt()
spin_lock(>lock);
...
uart_handle_sysrq_char();
sysrq_function();
printk();
stm32_usart_console_write();
locked = spin_trylock_irqsave(>lock); //better
than no lock(locked = 0) if other uart interrupt coming at this point
Find a same solution on fsl_lpuart.c
commit abf1e0a98083fd0a1069ce68ad8c92bfb97a57db
Thanks.
Best regards
Dillon
>
> So your initial patch looks to the right proposal, but it would be safer
> if Greg could confirm it.
>
> BR, Erwan.
>
>
> On 4/13/21 1:44 AM, dillon min wrote:
> > Hi Johan, Erwan
> >
> > It seems still a bit of a problem in the current version, not deadlock
> > but access register at the same time.
> >
> > For driver , we should consider it running under smp, let's think
> > about it for this case:
> >
> > static void stm32_usart_console_write(struct console *co, const char *s,
> >unsigned int cnt)
> > {
> > .
> > local_irq_save(flags);
> > if (port->sysrq)
> > locked = 0;
> > .
> > access register cr1, tdr, isr
> > .
> >
> > local_irq_restore(flags);
> > }
> >
> > if port->sysrq is 1, stm32_usart_console_write() just disable local
> > irq response by local_irq_save(), at the time of access register cr1,
> > tdr, isr. an TXE interrupt raised, for other cores(I know stm32
> > mpu/mcu do not have multi cores, just assume it has), it still has a
> > chance to handle interrupt. Then there is no lock to protect the uart
> > register.
> >
> > changes to below, should be more safe:
> >
> > .
> > if (port->sysrq || oops_in_progress)
> >locked = spin_trylock_irqsave(>lock, flags);
> > else
> >spin_lock_irqsave(>lock, flags);
> >
> >
> >
> > if (locked)
> > spin_unlock_irqrestore(>lock, flags);
> >
> > For current stm32 soc, it shouldn't happen. just a reminder for future.
> >
> > Thanks.
> >
> > Dillon
> >
> > On Mon, Apr 12, 2021 at 10:04 PM dillon min wrote:
> >>
> >> Hi Johan,
> >>
> >> Yes, there is no deadlock. my fault.
> >> I forget the local_irq_save() plus spin_lock() is spin_lock_irqsave().
> >>
> >> Thanks for your review. please ignore this patch.
> >>
> >> Best regards
> >>
> >> Dillon
> >>
> >> On Mon, Apr 12, 2021 at 9:08 PM Johan Hovold wrote:
> >>>
> >>> On Mon, Apr 12, 2021 at 05:31:38PM +0800, dillon.min...@gmail.com wrote:
> From: dillon min
>
> To avoid potential deadlock in spin_lock usage, use spin_lock_irqsave,
> spin_trylock_irqsave(), spin_unlock_irqrestore() in process context.
> >>>
> >>> This doesn't make much sense as console_write can be called in any
> >>> context. And where's the deadlock you claim to be fixing here?
> >>>
> remove unused local_irq_save/restore call.
>
> Cc: Alexandre Torgue
> Cc: Maxime Coquelin
> Cc: Gerald Baeza
> Cc: Erwan Le Ray
> Reported-by: kernel test robot
> Signed-off-by: dillon min
> ---
> v2: remove unused code from stm32_usart_threaded_interrupt() according
> from
> Greg's review.
>
> drivers/tty/serial/stm32-usart.c | 8 +++-
> 1 file changed, 3 insertions(+), 5 deletions(-)
>
> diff --git a/drivers/tty/serial/stm32-usart.c
> b/drivers/tty/serial/stm32-usart.c
> index b3675cf25a69..b1ba5e36e36e 100644
> --- a/drivers/tty/serial/stm32-usart.c
> +++ b/drivers/tty/serial/stm32-usart.c
> @@ -1354,13 +1354,12 @@ static void stm32_usart_console_write(struct
> console *co, const char *s,
> u32 old_cr1, new_cr1;
> int locked = 1;
>
> - local_irq_save(flags);
> if (port->sysrq)
> locked = 0;
> else if (oops_in_progress)
> - locked = spin_trylock(>lock);
> + locked = spin_trylock_irqsave(>lock, flags);
> else
> -
Re: [PATCH v2] serial: stm32: optimize spin lock usage
Hi Dillon,
STM32MP151 is mono-core, but both STM32MP153 and STM32MP157 are
dual-core (see
https://www.st.com/content/st_com/en/products/microcontrollers-microprocessors/stm32-arm-cortex-mpus.html).
So your point is fully relevant, thanks.
ST already fixed the same issue in st-asc.c driver in the past (see
ef49ffd8), because a systematic deadlock was detected with RT kernel.
You proposed a first implementation in your patch, and a second one in
the discussion. It seems that your initial proposal (ie your V2 patch)
is the most standard one (implemented in 6 drivers). The second
implementation is implemented by only 1 company.
It looks that the solution is to avoid locking in the sysrq case and
trylock in the oops_in_progress case (see detailed analysis in
677fe555cbfb1).
So your initial patch looks to the right proposal, but it would be safer
if Greg could confirm it.
BR, Erwan.
On 4/13/21 1:44 AM, dillon min wrote:
Hi Johan, Erwan
It seems still a bit of a problem in the current version, not deadlock
but access register at the same time.
For driver , we should consider it running under smp, let's think
about it for this case:
static void stm32_usart_console_write(struct console *co, const char *s,
unsigned int cnt)
{
.
local_irq_save(flags);
if (port->sysrq)
locked = 0;
.
access register cr1, tdr, isr
.
local_irq_restore(flags);
}
if port->sysrq is 1, stm32_usart_console_write() just disable local
irq response by local_irq_save(), at the time of access register cr1,
tdr, isr. an TXE interrupt raised, for other cores(I know stm32
mpu/mcu do not have multi cores, just assume it has), it still has a
chance to handle interrupt. Then there is no lock to protect the uart
register.
changes to below, should be more safe:
.
if (port->sysrq || oops_in_progress)
locked = spin_trylock_irqsave(>lock, flags);
else
spin_lock_irqsave(>lock, flags);
if (locked)
spin_unlock_irqrestore(>lock, flags);
For current stm32 soc, it shouldn't happen. just a reminder for future.
Thanks.
Dillon
On Mon, Apr 12, 2021 at 10:04 PM dillon min wrote:
Hi Johan,
Yes, there is no deadlock. my fault.
I forget the local_irq_save() plus spin_lock() is spin_lock_irqsave().
Thanks for your review. please ignore this patch.
Best regards
Dillon
On Mon, Apr 12, 2021 at 9:08 PM Johan Hovold wrote:
On Mon, Apr 12, 2021 at 05:31:38PM +0800, dillon.min...@gmail.com wrote:
From: dillon min
To avoid potential deadlock in spin_lock usage, use spin_lock_irqsave,
spin_trylock_irqsave(), spin_unlock_irqrestore() in process context.
This doesn't make much sense as console_write can be called in any
context. And where's the deadlock you claim to be fixing here?
remove unused local_irq_save/restore call.
Cc: Alexandre Torgue
Cc: Maxime Coquelin
Cc: Gerald Baeza
Cc: Erwan Le Ray
Reported-by: kernel test robot
Signed-off-by: dillon min
---
v2: remove unused code from stm32_usart_threaded_interrupt() according from
Greg's review.
drivers/tty/serial/stm32-usart.c | 8 +++-
1 file changed, 3 insertions(+), 5 deletions(-)
diff --git a/drivers/tty/serial/stm32-usart.c b/drivers/tty/serial/stm32-usart.c
index b3675cf25a69..b1ba5e36e36e 100644
--- a/drivers/tty/serial/stm32-usart.c
+++ b/drivers/tty/serial/stm32-usart.c
@@ -1354,13 +1354,12 @@ static void stm32_usart_console_write(struct console
*co, const char *s,
u32 old_cr1, new_cr1;
int locked = 1;
- local_irq_save(flags);
if (port->sysrq)
locked = 0;
else if (oops_in_progress)
- locked = spin_trylock(>lock);
+ locked = spin_trylock_irqsave(>lock, flags);
else
- spin_lock(>lock);
+ spin_lock_irqsave(>lock, flags);
/* Save and disable interrupts, enable the transmitter */
old_cr1 = readl_relaxed(port->membase + ofs->cr1);
@@ -1374,8 +1373,7 @@ static void stm32_usart_console_write(struct console *co,
const char *s,
writel_relaxed(old_cr1, port->membase + ofs->cr1);
if (locked)
- spin_unlock(>lock);
- local_irq_restore(flags);
+ spin_unlock_irqrestore(>lock, flags);
}
static int stm32_usart_console_setup(struct console *co, char *options)
Johan
Re: [PATCH v2] serial: stm32: optimize spin lock usage
Hi Johan, Erwan
It seems still a bit of a problem in the current version, not deadlock
but access register at the same time.
For driver , we should consider it running under smp, let's think
about it for this case:
static void stm32_usart_console_write(struct console *co, const char *s,
unsigned int cnt)
{
.
local_irq_save(flags);
if (port->sysrq)
locked = 0;
.
access register cr1, tdr, isr
.
local_irq_restore(flags);
}
if port->sysrq is 1, stm32_usart_console_write() just disable local
irq response by local_irq_save(), at the time of access register cr1,
tdr, isr. an TXE interrupt raised, for other cores(I know stm32
mpu/mcu do not have multi cores, just assume it has), it still has a
chance to handle interrupt. Then there is no lock to protect the uart
register.
changes to below, should be more safe:
.
if (port->sysrq || oops_in_progress)
locked = spin_trylock_irqsave(>lock, flags);
else
spin_lock_irqsave(>lock, flags);
if (locked)
spin_unlock_irqrestore(>lock, flags);
For current stm32 soc, it shouldn't happen. just a reminder for future.
Thanks.
Dillon
On Mon, Apr 12, 2021 at 10:04 PM dillon min wrote:
>
> Hi Johan,
>
> Yes, there is no deadlock. my fault.
> I forget the local_irq_save() plus spin_lock() is spin_lock_irqsave().
>
> Thanks for your review. please ignore this patch.
>
> Best regards
>
> Dillon
>
> On Mon, Apr 12, 2021 at 9:08 PM Johan Hovold wrote:
> >
> > On Mon, Apr 12, 2021 at 05:31:38PM +0800, dillon.min...@gmail.com wrote:
> > > From: dillon min
> > >
> > > To avoid potential deadlock in spin_lock usage, use spin_lock_irqsave,
> > > spin_trylock_irqsave(), spin_unlock_irqrestore() in process context.
> >
> > This doesn't make much sense as console_write can be called in any
> > context. And where's the deadlock you claim to be fixing here?
> >
> > > remove unused local_irq_save/restore call.
> > >
> > > Cc: Alexandre Torgue
> > > Cc: Maxime Coquelin
> > > Cc: Gerald Baeza
> > > Cc: Erwan Le Ray
> > > Reported-by: kernel test robot
> > > Signed-off-by: dillon min
> > > ---
> > > v2: remove unused code from stm32_usart_threaded_interrupt() according
> > > from
> > > Greg's review.
> > >
> > > drivers/tty/serial/stm32-usart.c | 8 +++-
> > > 1 file changed, 3 insertions(+), 5 deletions(-)
> > >
> > > diff --git a/drivers/tty/serial/stm32-usart.c
> > > b/drivers/tty/serial/stm32-usart.c
> > > index b3675cf25a69..b1ba5e36e36e 100644
> > > --- a/drivers/tty/serial/stm32-usart.c
> > > +++ b/drivers/tty/serial/stm32-usart.c
> > > @@ -1354,13 +1354,12 @@ static void stm32_usart_console_write(struct
> > > console *co, const char *s,
> > > u32 old_cr1, new_cr1;
> > > int locked = 1;
> > >
> > > - local_irq_save(flags);
> > > if (port->sysrq)
> > > locked = 0;
> > > else if (oops_in_progress)
> > > - locked = spin_trylock(>lock);
> > > + locked = spin_trylock_irqsave(>lock, flags);
> > > else
> > > - spin_lock(>lock);
> > > + spin_lock_irqsave(>lock, flags);
> > >
> > > /* Save and disable interrupts, enable the transmitter */
> > > old_cr1 = readl_relaxed(port->membase + ofs->cr1);
> > > @@ -1374,8 +1373,7 @@ static void stm32_usart_console_write(struct
> > > console *co, const char *s,
> > > writel_relaxed(old_cr1, port->membase + ofs->cr1);
> > >
> > > if (locked)
> > > - spin_unlock(>lock);
> > > - local_irq_restore(flags);
> > > + spin_unlock_irqrestore(>lock, flags);
> > > }
> > >
> > > static int stm32_usart_console_setup(struct console *co, char *options)
> >
> > Johan
Re: [PATCH v2] serial: stm32: optimize spin lock usage
Hi Johan, Yes, there is no deadlock. my fault. I forget the local_irq_save() plus spin_lock() is spin_lock_irqsave(). Thanks for your review. please ignore this patch. Best regards Dillon On Mon, Apr 12, 2021 at 9:08 PM Johan Hovold wrote: > > On Mon, Apr 12, 2021 at 05:31:38PM +0800, dillon.min...@gmail.com wrote: > > From: dillon min > > > > To avoid potential deadlock in spin_lock usage, use spin_lock_irqsave, > > spin_trylock_irqsave(), spin_unlock_irqrestore() in process context. > > This doesn't make much sense as console_write can be called in any > context. And where's the deadlock you claim to be fixing here? > > > remove unused local_irq_save/restore call. > > > > Cc: Alexandre Torgue > > Cc: Maxime Coquelin > > Cc: Gerald Baeza > > Cc: Erwan Le Ray > > Reported-by: kernel test robot > > Signed-off-by: dillon min > > --- > > v2: remove unused code from stm32_usart_threaded_interrupt() according from > > Greg's review. > > > > drivers/tty/serial/stm32-usart.c | 8 +++- > > 1 file changed, 3 insertions(+), 5 deletions(-) > > > > diff --git a/drivers/tty/serial/stm32-usart.c > > b/drivers/tty/serial/stm32-usart.c > > index b3675cf25a69..b1ba5e36e36e 100644 > > --- a/drivers/tty/serial/stm32-usart.c > > +++ b/drivers/tty/serial/stm32-usart.c > > @@ -1354,13 +1354,12 @@ static void stm32_usart_console_write(struct > > console *co, const char *s, > > u32 old_cr1, new_cr1; > > int locked = 1; > > > > - local_irq_save(flags); > > if (port->sysrq) > > locked = 0; > > else if (oops_in_progress) > > - locked = spin_trylock(>lock); > > + locked = spin_trylock_irqsave(>lock, flags); > > else > > - spin_lock(>lock); > > + spin_lock_irqsave(>lock, flags); > > > > /* Save and disable interrupts, enable the transmitter */ > > old_cr1 = readl_relaxed(port->membase + ofs->cr1); > > @@ -1374,8 +1373,7 @@ static void stm32_usart_console_write(struct console > > *co, const char *s, > > writel_relaxed(old_cr1, port->membase + ofs->cr1); > > > > if (locked) > > - spin_unlock(>lock); > > - local_irq_restore(flags); > > + spin_unlock_irqrestore(>lock, flags); > > } > > > > static int stm32_usart_console_setup(struct console *co, char *options) > > Johan
Re: [PATCH v2] serial: stm32: optimize spin lock usage
On Mon, Apr 12, 2021 at 05:31:38PM +0800, dillon.min...@gmail.com wrote: > From: dillon min > > To avoid potential deadlock in spin_lock usage, use spin_lock_irqsave, > spin_trylock_irqsave(), spin_unlock_irqrestore() in process context. This doesn't make much sense as console_write can be called in any context. And where's the deadlock you claim to be fixing here? > remove unused local_irq_save/restore call. > > Cc: Alexandre Torgue > Cc: Maxime Coquelin > Cc: Gerald Baeza > Cc: Erwan Le Ray > Reported-by: kernel test robot > Signed-off-by: dillon min > --- > v2: remove unused code from stm32_usart_threaded_interrupt() according from > Greg's review. > > drivers/tty/serial/stm32-usart.c | 8 +++- > 1 file changed, 3 insertions(+), 5 deletions(-) > > diff --git a/drivers/tty/serial/stm32-usart.c > b/drivers/tty/serial/stm32-usart.c > index b3675cf25a69..b1ba5e36e36e 100644 > --- a/drivers/tty/serial/stm32-usart.c > +++ b/drivers/tty/serial/stm32-usart.c > @@ -1354,13 +1354,12 @@ static void stm32_usart_console_write(struct console > *co, const char *s, > u32 old_cr1, new_cr1; > int locked = 1; > > - local_irq_save(flags); > if (port->sysrq) > locked = 0; > else if (oops_in_progress) > - locked = spin_trylock(>lock); > + locked = spin_trylock_irqsave(>lock, flags); > else > - spin_lock(>lock); > + spin_lock_irqsave(>lock, flags); > > /* Save and disable interrupts, enable the transmitter */ > old_cr1 = readl_relaxed(port->membase + ofs->cr1); > @@ -1374,8 +1373,7 @@ static void stm32_usart_console_write(struct console > *co, const char *s, > writel_relaxed(old_cr1, port->membase + ofs->cr1); > > if (locked) > - spin_unlock(>lock); > - local_irq_restore(flags); > + spin_unlock_irqrestore(>lock, flags); > } > > static int stm32_usart_console_setup(struct console *co, char *options) Johan
