Re: [PATCH v1 2/2] driver core: fw_devlink: Handle missing drivers for optional suppliers
On Tue, Feb 2, 2021 at 12:49 AM Geert Uytterhoeven wrote:
>
> Hi Saravana,
>
> On Mon, Feb 1, 2021 at 9:49 PM Saravana Kannan wrote:
> > On Mon, Feb 1, 2021 at 2:32 AM Geert Uytterhoeven
> > wrote:
> > > On Sat, Jan 30, 2021 at 5:03 AM Saravana Kannan
> > > wrote:
> > > > After a deferred probe attempt has exhaused all the devices that can be
> > > > bound, any device that remains unbound has one/both of these conditions
> > > > true:
> > > >
> > > > (1) It is waiting on its supplier to bind
> > > > (2) It does not have a matching driver
> > > >
> > > > So, to make fw_devlink=on more forgiving of missing drivers for optional
> > > > suppliers, after we've done a full deferred probe attempt, this patch
> > > > deletes all device links created by fw_devlink where the supplier hasn't
> > > > probed yet and the supplier itself is not waiting on any of its
> > > > suppliers. This allows consumers to probe during another deferred probe
> > > > attempt if they were waiting on optional suppliers.
> > > >
> > > > When modules are enabled, we can't differentiate between a driver
> > > > that'll never be registered vs a driver that'll be registered soon by
> > > > loading a module. So, this patch doesn't do anything for the case where
> > > > modules are enabled.
> > >
> > > For the modular case, can't you do a probe regardless? Or limit it
> > > to devices where the missing provider is a DMAC or IOMMU driver?
> > > Many drivers can handle missing DMAC controller drivers, and are even
> > > supposed to work that way. They may even retry obtaining DMA releases
> > > later.
> >
> > I don't want to handle this at a property/provider-type level. It'll
> > be a whack-a-mole that'll never end -- there'll be some driver that
> > would work without some resource. Letting it probe is not difficult (I
> > just need to drop these device links), but the problem is that a lot
> > of drivers are not written properly to be able to handle getting
> > deferred and then getting reattempted before the supplier. Either
> > because:
> >
> > 1. They were never built and tested as a module
> > 2. The supplier gets deferred and the consumer doesn't have proper
> > deferred probe implementation and when we drop the device links, the
> > consumer might be attempted before the supplier and things go bad.
>
> You may be a bit too pessimistic here: we had deferred probing for
> years. With devices with complex dependencies, it's not uncommon for a
> driver to be probed 3 or 4 times, before it succeeds (and FTR, would be
> happy to see this fixed). So most drivers should handle this already.
> And if they don't, they're already broken.
I fully agree the drivers are broken and they should be fixed (like
we've been doing so far). I'd happily say "fix your driver" and just
mark properties (iommu, dmas) as "optional" and be done with it.
Actually, handling optional properties is pretty simple -- we just
need to stop parsing them (delete code/make it a flag).
But my understanding is, you can't set fw_devlink=on by default and
break devices that used to boot fine before. But we can't find the
broken drivers without setting fw_devlink=on by default. So it's a
catch-22. I'm happy to continue helping debug the issues if we are
okay with leaving fw_devlink=on in 5.12, but I'm not sure everyone
will agree to that.
Also, there's also no way to selectively enable fw_devlink on a
DT/board level (Rob will have to agree to a property in the chosen {}
node). So, without that, I'm forced to go for the lowest common
denominator.
>
> > One hack I'm thinking of is that with CONFIG_MODULES, I can drop these
> > unmet device links after a N-second timeout, but having the timeout
> > extended everytime a new driver is registered. So as long as no two
> > modules are loaded further than N seconds apart during boot up, it
> > would all just work out fine. But it doesn't solve the problem fully
> > either. But maybe it'll be good enough? I haven't analyzed this fully
> > yet -- so apologies in advance if it's stupid.
>
> So you would introduce an additional delay when e.g. mounting the root
> file system, as the SDHI driver probe may be postponed due to the DMAC
> driver not being available?
Honestly I'm thinking of deleting/adding a conditional for iommu/dma
parsing. So boards that know all iommus/dmas are needed can set some
flag if they want.
In your example, if the DMAC driver is needed for SDHI to work (mount
root), then I'd expect it to be loaded(ramdisk)/builtin before mount
root. And it'd just work. If the DMA is optional and SDHI should have
mounted root without it, then the solution is what I said above.
> Let's consider the possible combinations here:
> 1. If both provider and consumer are built-in, there's no issue.
> 2. If the provider is built-in, and the consumer is modular, it should
> just work, too.
> 3. If the consumer is built-in, but the provider is modular (or not
> available), what to do?
> Wouldn't it be safe to
Re: [PATCH v1 2/2] driver core: fw_devlink: Handle missing drivers for optional suppliers
Hi Saravana,
On Mon, Feb 1, 2021 at 9:49 PM Saravana Kannan wrote:
> On Mon, Feb 1, 2021 at 2:32 AM Geert Uytterhoeven
> wrote:
> > On Sat, Jan 30, 2021 at 5:03 AM Saravana Kannan
> > wrote:
> > > After a deferred probe attempt has exhaused all the devices that can be
> > > bound, any device that remains unbound has one/both of these conditions
> > > true:
> > >
> > > (1) It is waiting on its supplier to bind
> > > (2) It does not have a matching driver
> > >
> > > So, to make fw_devlink=on more forgiving of missing drivers for optional
> > > suppliers, after we've done a full deferred probe attempt, this patch
> > > deletes all device links created by fw_devlink where the supplier hasn't
> > > probed yet and the supplier itself is not waiting on any of its
> > > suppliers. This allows consumers to probe during another deferred probe
> > > attempt if they were waiting on optional suppliers.
> > >
> > > When modules are enabled, we can't differentiate between a driver
> > > that'll never be registered vs a driver that'll be registered soon by
> > > loading a module. So, this patch doesn't do anything for the case where
> > > modules are enabled.
> >
> > For the modular case, can't you do a probe regardless? Or limit it
> > to devices where the missing provider is a DMAC or IOMMU driver?
> > Many drivers can handle missing DMAC controller drivers, and are even
> > supposed to work that way. They may even retry obtaining DMA releases
> > later.
>
> I don't want to handle this at a property/provider-type level. It'll
> be a whack-a-mole that'll never end -- there'll be some driver that
> would work without some resource. Letting it probe is not difficult (I
> just need to drop these device links), but the problem is that a lot
> of drivers are not written properly to be able to handle getting
> deferred and then getting reattempted before the supplier. Either
> because:
>
> 1. They were never built and tested as a module
> 2. The supplier gets deferred and the consumer doesn't have proper
> deferred probe implementation and when we drop the device links, the
> consumer might be attempted before the supplier and things go bad.
You may be a bit too pessimistic here: we had deferred probing for
years. With devices with complex dependencies, it's not uncommon for a
driver to be probed 3 or 4 times, before it succeeds (and FTR, would be
happy to see this fixed). So most drivers should handle this already.
And if they don't, they're already broken.
> One hack I'm thinking of is that with CONFIG_MODULES, I can drop these
> unmet device links after a N-second timeout, but having the timeout
> extended everytime a new driver is registered. So as long as no two
> modules are loaded further than N seconds apart during boot up, it
> would all just work out fine. But it doesn't solve the problem fully
> either. But maybe it'll be good enough? I haven't analyzed this fully
> yet -- so apologies in advance if it's stupid.
So you would introduce an additional delay when e.g. mounting the root
file system, as the SDHI driver probe may be postponed due to the DMAC
driver not being available?
Let's consider the possible combinations here:
1. If both provider and consumer are built-in, there's no issue.
2. If the provider is built-in, and the consumer is modular, it should
just work, too.
3. If the consumer is built-in, but the provider is modular (or not
available), what to do?
Wouldn't it be safe to assume the user did intend the consumer to
probe without the provider, and thus continue, if possible (e.g.
for an optional DMA channel)? Else the user would have configured
the provider driver built-in, too.
4. If both provider and consumer are modular, perhaps userspace should
try to load the modules in the right order, i.e. provider drivers
first?
For DMACs, it's the consumer that knows if it can work without the DMAC
driver present, and fall back to PIO, or not. So we could add a flag
for that to struct driver.
For IOMMUs, it's different: the consumer drivers are not aware of it,
only the driver/IOMMU subsystem is. And whether to work in the absence
of an IOMMU driver is more like a system (security) policy decision.
Gr{oetje,eeting}s,
Geert
--
Geert Uytterhoeven -- There's lots of Linux beyond ia32 -- ge...@linux-m68k.org
In personal conversations with technical people, I call myself a hacker. But
when I'm talking to journalists I just say "programmer" or something like that.
-- Linus Torvalds
Re: [PATCH v1 2/2] driver core: fw_devlink: Handle missing drivers for optional suppliers
On Mon, Feb 1, 2021 at 2:32 AM Geert Uytterhoeven wrote: > > Hi Saravana, > > On Sat, Jan 30, 2021 at 5:03 AM Saravana Kannan wrote: > > After a deferred probe attempt has exhaused all the devices that can be > > bound, any device that remains unbound has one/both of these conditions > > true: > > > > (1) It is waiting on its supplier to bind > > (2) It does not have a matching driver > > > > So, to make fw_devlink=on more forgiving of missing drivers for optional > > suppliers, after we've done a full deferred probe attempt, this patch > > deletes all device links created by fw_devlink where the supplier hasn't > > probed yet and the supplier itself is not waiting on any of its > > suppliers. This allows consumers to probe during another deferred probe > > attempt if they were waiting on optional suppliers. > > > > When modules are enabled, we can't differentiate between a driver > > that'll never be registered vs a driver that'll be registered soon by > > loading a module. So, this patch doesn't do anything for the case where > > modules are enabled. > > For the modular case, can't you do a probe regardless? Or limit it > to devices where the missing provider is a DMAC or IOMMU driver? > Many drivers can handle missing DMAC controller drivers, and are even > supposed to work that way. They may even retry obtaining DMA releases > later. I don't want to handle this at a property/provider-type level. It'll be a whack-a-mole that'll never end -- there'll be some driver that would work without some resource. Letting it probe is not difficult (I just need to drop these device links), but the problem is that a lot of drivers are not written properly to be able to handle getting deferred and then getting reattempted before the supplier. Either because: 1. They were never built and tested as a module 2. The supplier gets deferred and the consumer doesn't have proper deferred probe implementation and when we drop the device links, the consumer might be attempted before the supplier and things go bad. One hack I'm thinking of is that with CONFIG_MODULES, I can drop these unmet device links after a N-second timeout, but having the timeout extended everytime a new driver is registered. So as long as no two modules are loaded further than N seconds apart during boot up, it would all just work out fine. But it doesn't solve the problem fully either. But maybe it'll be good enough? I haven't analyzed this fully yet -- so apologies in advance if it's stupid. -Saravana
Re: [PATCH v1 2/2] driver core: fw_devlink: Handle missing drivers for optional suppliers
Hi Saravana,
On Sat, Jan 30, 2021 at 5:03 AM Saravana Kannan wrote:
> After a deferred probe attempt has exhaused all the devices that can be
> bound, any device that remains unbound has one/both of these conditions
> true:
>
> (1) It is waiting on its supplier to bind
> (2) It does not have a matching driver
>
> So, to make fw_devlink=on more forgiving of missing drivers for optional
> suppliers, after we've done a full deferred probe attempt, this patch
> deletes all device links created by fw_devlink where the supplier hasn't
> probed yet and the supplier itself is not waiting on any of its
> suppliers. This allows consumers to probe during another deferred probe
> attempt if they were waiting on optional suppliers.
>
> When modules are enabled, we can't differentiate between a driver
> that'll never be registered vs a driver that'll be registered soon by
> loading a module. So, this patch doesn't do anything for the case where
> modules are enabled.
For the modular case, can't you do a probe regardless? Or limit it
to devices where the missing provider is a DMAC or IOMMU driver?
Many drivers can handle missing DMAC controller drivers, and are even
supposed to work that way. They may even retry obtaining DMA releases
later.
Gr{oetje,eeting}s,
Geert
--
Geert Uytterhoeven -- There's lots of Linux beyond ia32 -- ge...@linux-m68k.org
In personal conversations with technical people, I call myself a hacker. But
when I'm talking to journalists I just say "programmer" or something like that.
-- Linus Torvalds
