Jerry Snitselaar @ 2020-12-02 17:02 MST:

> Jarkko Sakkinen @ 2020-12-02 09:49 MST:
>
>> On Tue, Dec 01, 2020 at 12:59:23PM -0700, Jerry Snitselaar wrote:
>>> 
>>> Jerry Snitselaar @ 2020-11-30 20:26 MST:
>>> 
>>> > Jerry Snitselaar @ 2020-11-30 19:58 MST:
>>> >
>>> >> When enabling the interrupt code for the tpm_tis driver we have
>>> >> noticed some systems have a bios issue causing an interrupt storm to
>>> >> occur. The issue isn't limited to a single tpm or system manufacturer
>>> >> so keeping a denylist of systems with the issue isn't optimal. Instead
>>> >> try to detect the problem occurring, disable interrupts, and revert to
>>> >> polling when it happens.
>>> >>
>>> >> Cc: Jarkko Sakkinen <jar...@kernel.org>
>>> >> Cc: Jason Gunthorpe <j...@ziepe.ca>
>>> >> Cc: Peter Huewe <peterhu...@gmx.de>
>>> >> Cc: James Bottomley <james.bottom...@hansenpartnership.com>
>>> >> Cc: Matthew Garrett <mj...@google.com>
>>> >> Cc: Hans de Goede <hdego...@redhat.com>
>>> >> Signed-off-by: Jerry Snitselaar <jsnit...@redhat.com>
>>> >> ---
>>> >> v2: drop tpm_tis specific workqueue and use just system_wq
>>> >>
>>> >> drivers/char/tpm/tpm_tis_core.c | 27 +++++++++++++++++++++++++++
>>> >>  drivers/char/tpm/tpm_tis_core.h |  2 ++
>>> >>  2 files changed, 29 insertions(+)
>>> >>
>>> >> diff --git a/drivers/char/tpm/tpm_tis_core.c 
>>> >> b/drivers/char/tpm/tpm_tis_core.c
>>> >> index 23b60583928b..72cc8a5a152c 100644
>>> >> --- a/drivers/char/tpm/tpm_tis_core.c
>>> >> +++ b/drivers/char/tpm/tpm_tis_core.c
>>> >> @@ -24,6 +24,8 @@
>>> >>  #include <linux/wait.h>
>>> >>  #include <linux/acpi.h>
>>> >>  #include <linux/freezer.h>
>>> >> +#include <linux/workqueue.h>
>>> >> +#include <linux/kernel_stat.h>
>>> >>  #include "tpm.h"
>>> >>  #include "tpm_tis_core.h"
>>> >>  
>>> >> @@ -745,9 +747,23 @@ static irqreturn_t tis_int_handler(int dummy, void 
>>> >> *dev_id)
>>> >>  {
>>> >>          struct tpm_chip *chip = dev_id;
>>> >>          struct tpm_tis_data *priv = dev_get_drvdata(&chip->dev);
>>> >> +        static bool check_storm = true;
>>> >> +        static unsigned int check_start;
>>> >>          u32 interrupt;
>>> >>          int i, rc;
>>> >>  
>>> >> +        if (unlikely(check_storm)) {
>>> >> +                if (!check_start) {
>>> >> +                        check_start = jiffies_to_msecs(jiffies);
>>> >> +                } else if ((kstat_irqs(priv->irq) > 1000) &&
>>> >> +                           (jiffies_to_msecs(jiffies) - check_start < 
>>> >> 500)) {
>>> >> +                        check_storm = false;
>>> >> +                        schedule_work(&priv->storm_work);
>>> >> +                } else if (jiffies_to_msecs(jiffies) - check_start >= 
>>> >> 500) {
>>> >> +                        check_storm = false;
>>> >> +                }
>>> >> +        }
>>> >> +
>>> >>          rc = tpm_tis_read32(priv, TPM_INT_STATUS(priv->locality), 
>>> >> &interrupt);
>>> >>          if (rc < 0)
>>> >>                  return IRQ_NONE;
>>> >> @@ -987,6 +1003,14 @@ static const struct tpm_class_ops tpm_tis = {
>>> >>          .clk_enable = tpm_tis_clkrun_enable,
>>> >>  };
>>> >>  
>>> >> +static void tpm_tis_storm_work(struct work_struct *work)
>>> >> +{
>>> >> +        struct tpm_tis_data *priv = container_of(work, struct 
>>> >> tpm_tis_data, storm_work);
>>> >> +
>>> >> +        disable_interrupts(priv->chip);
>>> >> +        dev_warn(&priv->chip->dev, "Interrupt storm detected, using 
>>> >> polling.\n");
>>> >> +}
>>> >> +
>>> >>  int tpm_tis_core_init(struct device *dev, struct tpm_tis_data *priv, 
>>> >> int irq,
>>> >>                        const struct tpm_tis_phy_ops *phy_ops,
>>> >>                        acpi_handle acpi_dev_handle)
>>> >> @@ -1003,6 +1027,9 @@ int tpm_tis_core_init(struct device *dev, struct 
>>> >> tpm_tis_data *priv, int irq,
>>> >>          if (IS_ERR(chip))
>>> >>                  return PTR_ERR(chip);
>>> >>  
>>> >> +        priv->chip = chip;
>>> >> +        INIT_WORK(&priv->storm_work, tpm_tis_storm_work);
>>> >> +
>>> >>  #ifdef CONFIG_ACPI
>>> >>          chip->acpi_dev_handle = acpi_dev_handle;
>>> >>  #endif
>>> >> diff --git a/drivers/char/tpm/tpm_tis_core.h 
>>> >> b/drivers/char/tpm/tpm_tis_core.h
>>> >> index edeb5dc61c95..5630f294dc0c 100644
>>> >> --- a/drivers/char/tpm/tpm_tis_core.h
>>> >> +++ b/drivers/char/tpm/tpm_tis_core.h
>>> >> @@ -95,6 +95,8 @@ struct tpm_tis_data {
>>> >>          u16 clkrun_enabled;
>>> >>          wait_queue_head_t int_queue;
>>> >>          wait_queue_head_t read_queue;
>>> >> +        struct work_struct storm_work;
>>> >> +        struct tpm_chip *chip;
>>> >>          const struct tpm_tis_phy_ops *phy_ops;
>>> >>          unsigned short rng_quality;
>>> >>  };
>>> >
>>> > I've tested this with the Intel platform that has an Infineon chip that
>>> > I found the other week. It works, but isn't the complete fix. With this
>>> > on top of James' patchset I sometimes see the message "Lost Interrupt
>>> > waiting for TPM stat", so I guess there needs to be a check in
>>> > wait_for_tpm_stat and request_locality to see if interrupts were
>>> > disabled when the wait_event_interruptible_timeout call times out.
>>> 
>>> As kernel test robot pointed out. kstat_irqs isn't visible when tpm_tis
>>> builds as a module. It looks like it is only called by kstat_irq_usrs,
>>> and that is only by the fs/proc code. I have a patch to export it, but
>>> the i915 driver open codes their own version instead of using it. Is
>>> there any reason not to export it?
>>
>> If you add a patch that exports it, then for coherency it'd be better to
>> also patch i915 driver. Jani?
>>
>> /Jarkko
>
> It looks like this might not solve all cases. I'm having Lenovo test
> another build to make sure I gave them the right code, but they reported
> with the L490 that the system hangs right when it is initializing
> tpm_tis. I'm working on getting a build on the T490s I have to try there
> as well. With the Intel system it spits out that it detects the
> interrupt storm, and continues on its way.

The interrupt storm detection code works on the T490s. I'm not sure what
is going on with the L490. I will see if I can get access to one.

Jerry

Reply via email to