Re: [PATCH v4 2/8] OF: Introduce DT overlay support.
On Tue, 27 May 2014 15:24:35 +0300, Pantelis Antoniou wrote: > Hi Grant, > > On May 27, 2014, at 3:12 PM, Grant Likely wrote: > > > On Mon, 26 May 2014 16:42:44 -0700, Guenter Roeck > > wrote: > >> On 05/26/2014 03:36 PM, Sebastian Reichel wrote: > >>> Hi, > >>> > >>> On Mon, May 26, 2014 at 10:33:03PM +0100, Grant Likely wrote: > After thinking about it more, I think it is very likely that removing > all the overlays is the correct thing to do in the kexec use-case. When > kexec-ing, it makes sense that we'd want the exact same behaviour from > the kexec'ed kernel. That means we want the device drivers to do the > same thing including loading whatever overlays they depend on. > > If the flattened tree was left applied, then the behaviour becomes > different. > > I say always remove the overlays unless explicitly told not to, but I'm > struggling to come up with use cases where keeping them applied is > desirable. > >>> > >>> I would assume, that I want them applied in most cases. DT describes > >>> the hardware. If I kexec into a new kernel I change software, not > >>> hardware. > >>> > >>> Maybe I'm missing the main purpose of the feature. I currently see > >>> two useful usecases for DT overlays: > >>> > >>> 1. The dtb the kernel is booted with cannot be changed for some > >>>reason, but the board has additional hardware attached (e.g. > >>>the user added a sensor on the i2c bus) > >>> 2. The hardware is changed on the fly (e.g. the user flashed the > >>>FPGA part of a zynq processor), sensors on i2c bus, ... > >>> > >>> In both cases the kernel should be booted with the additional > >>> overlay information IMHO. Though for the second case it should > >>> be possible to remove the "programmed" hardware information > >>> somehow. > >>> > >> > >> 3. Some hot-plug device or card is inserted or removed. > >> > >> I would argue that the kernel should _not_ be booted with the overlay in > >> place. > >> Otherwise the code handling overlays would have to have special handling > >> for the restart case, which is much more complex than just to re-insert > >> the overlay when it is determined that the device or card is still there. > > > > Exactly. > > > > Looks like we are levitating to the 'remove overlays on kexec' approach. > Is that correct? Yes. What form that takes is yet to be decided. Kexec traditionally has worked by userspace extracting the device tree from /proc/device-tree, getting a kernel image into memory, and then doing the kexec call. It wouldn't be very friendly to have kexec force all the overlays to be removed while the kernel is still running. We could solve this by making kexec use the original dtb blob instead of the livetree. Otherwise we'd need an interface for userspace to extract the original tree, or have the information to unwind the overlays. Not exactly simple. g. -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH v4 2/8] OF: Introduce DT overlay support.
Hi Geert, On Tue, May 27, 2014 at 07:52:57PM +0200, Geert Uytterhoeven wrote: > Hi Günther, > > On Tue, May 27, 2014 at 5:21 PM, Guenter Roeck wrote: > > On Tue, May 27, 2014 at 03:24:35PM +0300, Pantelis Antoniou wrote: > >> On May 27, 2014, at 3:12 PM, Grant Likely wrote: > >> > On Mon, 26 May 2014 16:42:44 -0700, Guenter Roeck > >> > wrote: > >> >> On 05/26/2014 03:36 PM, Sebastian Reichel wrote: > >> >>> On Mon, May 26, 2014 at 10:33:03PM +0100, Grant Likely wrote: > >> After thinking about it more, I think it is very likely that removing > >> all the overlays is the correct thing to do in the kexec use-case. > >> When > >> kexec-ing, it makes sense that we'd want the exact same behaviour from > >> the kexec'ed kernel. That means we want the device drivers to do the > >> same thing including loading whatever overlays they depend on. > >> > >> If the flattened tree was left applied, then the behaviour becomes > >> different. > >> > >> I say always remove the overlays unless explicitly told not to, but > >> I'm > >> struggling to come up with use cases where keeping them applied is > >> desirable. > >> >>> > >> >>> I would assume, that I want them applied in most cases. DT describes > >> >>> the hardware. If I kexec into a new kernel I change software, not > >> >>> hardware. > >> >>> > >> >>> Maybe I'm missing the main purpose of the feature. I currently see > >> >>> two useful usecases for DT overlays: > >> >>> > >> >>> 1. The dtb the kernel is booted with cannot be changed for some > >> >>>reason, but the board has additional hardware attached (e.g. > >> >>>the user added a sensor on the i2c bus) > >> >>> 2. The hardware is changed on the fly (e.g. the user flashed the > >> >>>FPGA part of a zynq processor), sensors on i2c bus, ... > >> >>> > >> >>> In both cases the kernel should be booted with the additional > >> >>> overlay information IMHO. Though for the second case it should > >> >>> be possible to remove the "programmed" hardware information > >> >>> somehow. > >> >>> > >> >> > >> >> 3. Some hot-plug device or card is inserted or removed. > >> >> > >> >> I would argue that the kernel should _not_ be booted with the overlay > >> >> in place. > >> >> Otherwise the code handling overlays would have to have special handling > >> >> for the restart case, which is much more complex than just to re-insert > >> >> the overlay when it is determined that the device or card is still > >> >> there. > >> > > >> > Exactly. > >> > > >> > >> Looks like we are levitating to the 'remove overlays on kexec' approach. > >> Is that correct? > >> > > > > Let's just assume for a minute that this is not the case, and that loaded > > overlays are passed on. > > > > This would be an interesting challenge for the overlay manager, as it would > > have to handle a number of startup conditions. After all, it can not take > > it as granted that the hardware state did not change after it was stopped > > in the old kernel, and before it was started in the new kernel. > > - overlay loaded, but hardware/device no longer present > > -> unload overlay > > - overlay loaded, but different hardware present > > -> unload old overlay, load new one > > - overlay not loaded, hardware present > > -> load overlay > > - overlay loaded and matches hardware > > -> do nothing > > > > In comparison, its task would be quite straightforward if loaded overlays > > are not passed on to the new kernel. > > - If hardware is present, load overlay > > > > Ultimately, I seem to be missing something, as I don't really see the > > benefit > > of passing on the loaded overlay(s) to the new kernel. Activation time, > > maybe, > > for the most common case (overlay loaded and matches hardware) ? > > All of the above assume you're using a system with overlays and an overlay > manager. I.e. you have add-on devices together with an overlay DT. > One way to handle this is via auto-detection of capes, that identify > themselves, > so the overlay manager knows which overlay to load. > Possibly auto-detected (and in our case that is so), but not necessarily. A user space overlay manager, which keeps state in user space, would be possible as well. > I'm trying to look at it in a more generic way: you add hardware which is not > in the DT, so the DT must be modified (in some way or another) to match the > hardware. > > If you run kexec, the added hardware is still there. In the absence of a > system > with auto-detection of capes, the newly booted kernel cannot know that > hardware has been added, compared to a pristine system. If it's in the > DT passed through kexec, everything will work. > Yes, but only for that case. You'd still have the problem that you'll need to keep the state somewhere to be able to remove the pre-loaded overlay(s) later on. If you don't have hardware auto-detection, I would assume that to be in user space. No matter if the system
Re: [PATCH v4 2/8] OF: Introduce DT overlay support.
Hi Günther,
On Tue, May 27, 2014 at 5:21 PM, Guenter Roeck wrote:
> On Tue, May 27, 2014 at 03:24:35PM +0300, Pantelis Antoniou wrote:
>> On May 27, 2014, at 3:12 PM, Grant Likely wrote:
>> > On Mon, 26 May 2014 16:42:44 -0700, Guenter Roeck
>> > wrote:
>> >> On 05/26/2014 03:36 PM, Sebastian Reichel wrote:
>> >>> On Mon, May 26, 2014 at 10:33:03PM +0100, Grant Likely wrote:
>> After thinking about it more, I think it is very likely that removing
>> all the overlays is the correct thing to do in the kexec use-case. When
>> kexec-ing, it makes sense that we'd want the exact same behaviour from
>> the kexec'ed kernel. That means we want the device drivers to do the
>> same thing including loading whatever overlays they depend on.
>>
>> If the flattened tree was left applied, then the behaviour becomes
>> different.
>>
>> I say always remove the overlays unless explicitly told not to, but I'm
>> struggling to come up with use cases where keeping them applied is
>> desirable.
>> >>>
>> >>> I would assume, that I want them applied in most cases. DT describes
>> >>> the hardware. If I kexec into a new kernel I change software, not
>> >>> hardware.
>> >>>
>> >>> Maybe I'm missing the main purpose of the feature. I currently see
>> >>> two useful usecases for DT overlays:
>> >>>
>> >>> 1. The dtb the kernel is booted with cannot be changed for some
>> >>>reason, but the board has additional hardware attached (e.g.
>> >>>the user added a sensor on the i2c bus)
>> >>> 2. The hardware is changed on the fly (e.g. the user flashed the
>> >>>FPGA part of a zynq processor), sensors on i2c bus, ...
>> >>>
>> >>> In both cases the kernel should be booted with the additional
>> >>> overlay information IMHO. Though for the second case it should
>> >>> be possible to remove the "programmed" hardware information
>> >>> somehow.
>> >>>
>> >>
>> >> 3. Some hot-plug device or card is inserted or removed.
>> >>
>> >> I would argue that the kernel should _not_ be booted with the overlay in
>> >> place.
>> >> Otherwise the code handling overlays would have to have special handling
>> >> for the restart case, which is much more complex than just to re-insert
>> >> the overlay when it is determined that the device or card is still there.
>> >
>> > Exactly.
>> >
>>
>> Looks like we are levitating to the 'remove overlays on kexec' approach.
>> Is that correct?
>>
>
> Let's just assume for a minute that this is not the case, and that loaded
> overlays are passed on.
>
> This would be an interesting challenge for the overlay manager, as it would
> have to handle a number of startup conditions. After all, it can not take
> it as granted that the hardware state did not change after it was stopped
> in the old kernel, and before it was started in the new kernel.
> - overlay loaded, but hardware/device no longer present
> -> unload overlay
> - overlay loaded, but different hardware present
> -> unload old overlay, load new one
> - overlay not loaded, hardware present
> -> load overlay
> - overlay loaded and matches hardware
> -> do nothing
>
> In comparison, its task would be quite straightforward if loaded overlays
> are not passed on to the new kernel.
> - If hardware is present, load overlay
>
> Ultimately, I seem to be missing something, as I don't really see the benefit
> of passing on the loaded overlay(s) to the new kernel. Activation time, maybe,
> for the most common case (overlay loaded and matches hardware) ?
All of the above assume you're using a system with overlays and an overlay
manager. I.e. you have add-on devices together with an overlay DT.
One way to handle this is via auto-detection of capes, that identify themselves,
so the overlay manager knows which overlay to load.
I'm trying to look at it in a more generic way: you add hardware which is not
in the DT, so the DT must be modified (in some way or another) to match the
hardware.
If you run kexec, the added hardware is still there. In the absence of a system
with auto-detection of capes, the newly booted kernel cannot know that
hardware has been added, compared to a pristine system. If it's in the
DT passed through kexec, everything will work.
Gr{oetje,eeting}s,
Geert
--
Geert Uytterhoeven -- There's lots of Linux beyond ia32 -- ge...@linux-m68k.org
In personal conversations with technical people, I call myself a hacker. But
when I'm talking to journalists I just say "programmer" or something like that.
-- Linus Torvalds
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH v4 2/8] OF: Introduce DT overlay support.
Hi Guenter, On May 27, 2014, at 6:21 PM, Guenter Roeck wrote: > On Tue, May 27, 2014 at 03:24:35PM +0300, Pantelis Antoniou wrote: >> Hi Grant, >> >> On May 27, 2014, at 3:12 PM, Grant Likely wrote: >> >>> On Mon, 26 May 2014 16:42:44 -0700, Guenter Roeck >>> wrote: On 05/26/2014 03:36 PM, Sebastian Reichel wrote: > Hi, > > On Mon, May 26, 2014 at 10:33:03PM +0100, Grant Likely wrote: >> After thinking about it more, I think it is very likely that removing >> all the overlays is the correct thing to do in the kexec use-case. When >> kexec-ing, it makes sense that we'd want the exact same behaviour from >> the kexec'ed kernel. That means we want the device drivers to do the >> same thing including loading whatever overlays they depend on. >> >> If the flattened tree was left applied, then the behaviour becomes >> different. >> >> I say always remove the overlays unless explicitly told not to, but I'm >> struggling to come up with use cases where keeping them applied is >> desirable. > > I would assume, that I want them applied in most cases. DT describes > the hardware. If I kexec into a new kernel I change software, not > hardware. > > Maybe I'm missing the main purpose of the feature. I currently see > two useful usecases for DT overlays: > > 1. The dtb the kernel is booted with cannot be changed for some > reason, but the board has additional hardware attached (e.g. > the user added a sensor on the i2c bus) > 2. The hardware is changed on the fly (e.g. the user flashed the > FPGA part of a zynq processor), sensors on i2c bus, ... > > In both cases the kernel should be booted with the additional > overlay information IMHO. Though for the second case it should > be possible to remove the "programmed" hardware information > somehow. > 3. Some hot-plug device or card is inserted or removed. I would argue that the kernel should _not_ be booted with the overlay in place. Otherwise the code handling overlays would have to have special handling for the restart case, which is much more complex than just to re-insert the overlay when it is determined that the device or card is still there. >>> >>> Exactly. >>> >> >> Looks like we are levitating to the 'remove overlays on kexec' approach. >> Is that correct? >> > > Let's just assume for a minute that this is not the case, and that loaded > overlays are passed on. > > This would be an interesting challenge for the overlay manager, as it would > have to handle a number of startup conditions. After all, it can not take > it as granted that the hardware state did not change after it was stopped > in the old kernel, and before it was started in the new kernel. > - overlay loaded, but hardware/device no longer present > -> unload overlay > - overlay loaded, but different hardware present > -> unload old overlay, load new one > - overlay not loaded, hardware present > -> load overlay > - overlay loaded and matches hardware > -> do nothing > > In comparison, its task would be quite straightforward if loaded overlays > are not passed on to the new kernel. > - If hardware is present, load overlay > Yeah, exactly. The only case where having the applied overlays present on the kexec-ed kernel is either some kind of virtualization environment, or an fpga that takes an awful lot amount of time to re-initializa. > Ultimately, I seem to be missing something, as I don't really see the benefit > of passing on the loaded overlay(s) to the new kernel. Activation time, maybe, > for the most common case (overlay loaded and matches hardware) ? > > Concern though is the other cases, with a mismatch between HW and loaded > overlays. I am not sure if it is even possible to ensure that there are > no race conditions if the devicetree is outdated at system startup. Sure, > that is unlikely to happen in 'normal' operating conditions, but that only > means that it _will_ happen a few hours after the first customer starts > playing with the system. > I concur. Accidents will happen, and if it's not a tightly controlled system breakage is guaranteed. > Guenter Regards -- Pantelis > -- > To unsubscribe from this list: send the line "unsubscribe devicetree" in > the body of a message to majord...@vger.kernel.org > More majordomo info at http://vger.kernel.org/majordomo-info.html -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH v4 2/8] OF: Introduce DT overlay support.
On Tue, May 27, 2014 at 03:24:35PM +0300, Pantelis Antoniou wrote: > Hi Grant, > > On May 27, 2014, at 3:12 PM, Grant Likely wrote: > > > On Mon, 26 May 2014 16:42:44 -0700, Guenter Roeck > > wrote: > >> On 05/26/2014 03:36 PM, Sebastian Reichel wrote: > >>> Hi, > >>> > >>> On Mon, May 26, 2014 at 10:33:03PM +0100, Grant Likely wrote: > After thinking about it more, I think it is very likely that removing > all the overlays is the correct thing to do in the kexec use-case. When > kexec-ing, it makes sense that we'd want the exact same behaviour from > the kexec'ed kernel. That means we want the device drivers to do the > same thing including loading whatever overlays they depend on. > > If the flattened tree was left applied, then the behaviour becomes > different. > > I say always remove the overlays unless explicitly told not to, but I'm > struggling to come up with use cases where keeping them applied is > desirable. > >>> > >>> I would assume, that I want them applied in most cases. DT describes > >>> the hardware. If I kexec into a new kernel I change software, not > >>> hardware. > >>> > >>> Maybe I'm missing the main purpose of the feature. I currently see > >>> two useful usecases for DT overlays: > >>> > >>> 1. The dtb the kernel is booted with cannot be changed for some > >>>reason, but the board has additional hardware attached (e.g. > >>>the user added a sensor on the i2c bus) > >>> 2. The hardware is changed on the fly (e.g. the user flashed the > >>>FPGA part of a zynq processor), sensors on i2c bus, ... > >>> > >>> In both cases the kernel should be booted with the additional > >>> overlay information IMHO. Though for the second case it should > >>> be possible to remove the "programmed" hardware information > >>> somehow. > >>> > >> > >> 3. Some hot-plug device or card is inserted or removed. > >> > >> I would argue that the kernel should _not_ be booted with the overlay in > >> place. > >> Otherwise the code handling overlays would have to have special handling > >> for the restart case, which is much more complex than just to re-insert > >> the overlay when it is determined that the device or card is still there. > > > > Exactly. > > > > Looks like we are levitating to the 'remove overlays on kexec' approach. > Is that correct? > Let's just assume for a minute that this is not the case, and that loaded overlays are passed on. This would be an interesting challenge for the overlay manager, as it would have to handle a number of startup conditions. After all, it can not take it as granted that the hardware state did not change after it was stopped in the old kernel, and before it was started in the new kernel. - overlay loaded, but hardware/device no longer present -> unload overlay - overlay loaded, but different hardware present -> unload old overlay, load new one - overlay not loaded, hardware present -> load overlay - overlay loaded and matches hardware -> do nothing In comparison, its task would be quite straightforward if loaded overlays are not passed on to the new kernel. - If hardware is present, load overlay Ultimately, I seem to be missing something, as I don't really see the benefit of passing on the loaded overlay(s) to the new kernel. Activation time, maybe, for the most common case (overlay loaded and matches hardware) ? Concern though is the other cases, with a mismatch between HW and loaded overlays. I am not sure if it is even possible to ensure that there are no race conditions if the devicetree is outdated at system startup. Sure, that is unlikely to happen in 'normal' operating conditions, but that only means that it _will_ happen a few hours after the first customer starts playing with the system. Guenter -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH v4 2/8] OF: Introduce DT overlay support.
Hi Grant, On May 27, 2014, at 3:12 PM, Grant Likely wrote: > On Mon, 26 May 2014 16:42:44 -0700, Guenter Roeck wrote: >> On 05/26/2014 03:36 PM, Sebastian Reichel wrote: >>> Hi, >>> >>> On Mon, May 26, 2014 at 10:33:03PM +0100, Grant Likely wrote: After thinking about it more, I think it is very likely that removing all the overlays is the correct thing to do in the kexec use-case. When kexec-ing, it makes sense that we'd want the exact same behaviour from the kexec'ed kernel. That means we want the device drivers to do the same thing including loading whatever overlays they depend on. If the flattened tree was left applied, then the behaviour becomes different. I say always remove the overlays unless explicitly told not to, but I'm struggling to come up with use cases where keeping them applied is desirable. >>> >>> I would assume, that I want them applied in most cases. DT describes >>> the hardware. If I kexec into a new kernel I change software, not >>> hardware. >>> >>> Maybe I'm missing the main purpose of the feature. I currently see >>> two useful usecases for DT overlays: >>> >>> 1. The dtb the kernel is booted with cannot be changed for some >>>reason, but the board has additional hardware attached (e.g. >>>the user added a sensor on the i2c bus) >>> 2. The hardware is changed on the fly (e.g. the user flashed the >>>FPGA part of a zynq processor), sensors on i2c bus, ... >>> >>> In both cases the kernel should be booted with the additional >>> overlay information IMHO. Though for the second case it should >>> be possible to remove the "programmed" hardware information >>> somehow. >>> >> >> 3. Some hot-plug device or card is inserted or removed. >> >> I would argue that the kernel should _not_ be booted with the overlay in >> place. >> Otherwise the code handling overlays would have to have special handling >> for the restart case, which is much more complex than just to re-insert >> the overlay when it is determined that the device or card is still there. > > Exactly. > Looks like we are levitating to the 'remove overlays on kexec' approach. Is that correct? > g. > Regards -- Pantelis -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH v4 2/8] OF: Introduce DT overlay support.
On Mon, 26 May 2014 16:42:44 -0700, Guenter Roeck wrote: > On 05/26/2014 03:36 PM, Sebastian Reichel wrote: > > Hi, > > > > On Mon, May 26, 2014 at 10:33:03PM +0100, Grant Likely wrote: > >> After thinking about it more, I think it is very likely that removing > >> all the overlays is the correct thing to do in the kexec use-case. When > >> kexec-ing, it makes sense that we'd want the exact same behaviour from > >> the kexec'ed kernel. That means we want the device drivers to do the > >> same thing including loading whatever overlays they depend on. > >> > >> If the flattened tree was left applied, then the behaviour becomes > >> different. > >> > >> I say always remove the overlays unless explicitly told not to, but I'm > >> struggling to come up with use cases where keeping them applied is > >> desirable. > > > > I would assume, that I want them applied in most cases. DT describes > > the hardware. If I kexec into a new kernel I change software, not > > hardware. > > > > Maybe I'm missing the main purpose of the feature. I currently see > > two useful usecases for DT overlays: > > > > 1. The dtb the kernel is booted with cannot be changed for some > > reason, but the board has additional hardware attached (e.g. > > the user added a sensor on the i2c bus) > > 2. The hardware is changed on the fly (e.g. the user flashed the > > FPGA part of a zynq processor), sensors on i2c bus, ... > > > > In both cases the kernel should be booted with the additional > > overlay information IMHO. Though for the second case it should > > be possible to remove the "programmed" hardware information > > somehow. > > > > 3. Some hot-plug device or card is inserted or removed. > > I would argue that the kernel should _not_ be booted with the overlay in > place. > Otherwise the code handling overlays would have to have special handling > for the restart case, which is much more complex than just to re-insert > the overlay when it is determined that the device or card is still there. Exactly. g. -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH v4 2/8] OF: Introduce DT overlay support.
On Mon, 26 May 2014 23:44:41 +0200, Geert Uytterhoeven wrote: > Hi Grant, > > On Mon, May 26, 2014 at 11:33 PM, Grant Likely > wrote: > > After thinking about it more, I think it is very likely that removing > > all the overlays is the correct thing to do in the kexec use-case. When > > kexec-ing, it makes sense that we'd want the exact same behaviour from > > the kexec'ed kernel. That means we want the device drivers to do the > > same thing including loading whatever overlays they depend on. > > Are the device drivers loading the overlays? > That sounds a bit backwards to me. In a lot of cases, yes. For example, the beaglebone capebus driver would be responsible for identifying and requesting or loading the correct overlay to describe the hardware. Having userspace provide an arbitrary overlay is only one of the use cases. g. -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH v4 2/8] OF: Introduce DT overlay support.
On Mon, 26 May 2014 23:44:41 +0200, Geert Uytterhoeven ge...@linux-m68k.org wrote: Hi Grant, On Mon, May 26, 2014 at 11:33 PM, Grant Likely grant.lik...@secretlab.ca wrote: After thinking about it more, I think it is very likely that removing all the overlays is the correct thing to do in the kexec use-case. When kexec-ing, it makes sense that we'd want the exact same behaviour from the kexec'ed kernel. That means we want the device drivers to do the same thing including loading whatever overlays they depend on. Are the device drivers loading the overlays? That sounds a bit backwards to me. In a lot of cases, yes. For example, the beaglebone capebus driver would be responsible for identifying and requesting or loading the correct overlay to describe the hardware. Having userspace provide an arbitrary overlay is only one of the use cases. g. -- To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH v4 2/8] OF: Introduce DT overlay support.
On Mon, 26 May 2014 16:42:44 -0700, Guenter Roeck li...@roeck-us.net wrote: On 05/26/2014 03:36 PM, Sebastian Reichel wrote: Hi, On Mon, May 26, 2014 at 10:33:03PM +0100, Grant Likely wrote: After thinking about it more, I think it is very likely that removing all the overlays is the correct thing to do in the kexec use-case. When kexec-ing, it makes sense that we'd want the exact same behaviour from the kexec'ed kernel. That means we want the device drivers to do the same thing including loading whatever overlays they depend on. If the flattened tree was left applied, then the behaviour becomes different. I say always remove the overlays unless explicitly told not to, but I'm struggling to come up with use cases where keeping them applied is desirable. I would assume, that I want them applied in most cases. DT describes the hardware. If I kexec into a new kernel I change software, not hardware. Maybe I'm missing the main purpose of the feature. I currently see two useful usecases for DT overlays: 1. The dtb the kernel is booted with cannot be changed for some reason, but the board has additional hardware attached (e.g. the user added a sensor on the i2c bus) 2. The hardware is changed on the fly (e.g. the user flashed the FPGA part of a zynq processor), sensors on i2c bus, ... In both cases the kernel should be booted with the additional overlay information IMHO. Though for the second case it should be possible to remove the programmed hardware information somehow. 3. Some hot-plug device or card is inserted or removed. I would argue that the kernel should _not_ be booted with the overlay in place. Otherwise the code handling overlays would have to have special handling for the restart case, which is much more complex than just to re-insert the overlay when it is determined that the device or card is still there. Exactly. g. -- To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH v4 2/8] OF: Introduce DT overlay support.
Hi Grant, On May 27, 2014, at 3:12 PM, Grant Likely wrote: On Mon, 26 May 2014 16:42:44 -0700, Guenter Roeck li...@roeck-us.net wrote: On 05/26/2014 03:36 PM, Sebastian Reichel wrote: Hi, On Mon, May 26, 2014 at 10:33:03PM +0100, Grant Likely wrote: After thinking about it more, I think it is very likely that removing all the overlays is the correct thing to do in the kexec use-case. When kexec-ing, it makes sense that we'd want the exact same behaviour from the kexec'ed kernel. That means we want the device drivers to do the same thing including loading whatever overlays they depend on. If the flattened tree was left applied, then the behaviour becomes different. I say always remove the overlays unless explicitly told not to, but I'm struggling to come up with use cases where keeping them applied is desirable. I would assume, that I want them applied in most cases. DT describes the hardware. If I kexec into a new kernel I change software, not hardware. Maybe I'm missing the main purpose of the feature. I currently see two useful usecases for DT overlays: 1. The dtb the kernel is booted with cannot be changed for some reason, but the board has additional hardware attached (e.g. the user added a sensor on the i2c bus) 2. The hardware is changed on the fly (e.g. the user flashed the FPGA part of a zynq processor), sensors on i2c bus, ... In both cases the kernel should be booted with the additional overlay information IMHO. Though for the second case it should be possible to remove the programmed hardware information somehow. 3. Some hot-plug device or card is inserted or removed. I would argue that the kernel should _not_ be booted with the overlay in place. Otherwise the code handling overlays would have to have special handling for the restart case, which is much more complex than just to re-insert the overlay when it is determined that the device or card is still there. Exactly. Looks like we are levitating to the 'remove overlays on kexec' approach. Is that correct? g. Regards -- Pantelis -- To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH v4 2/8] OF: Introduce DT overlay support.
On Tue, May 27, 2014 at 03:24:35PM +0300, Pantelis Antoniou wrote: Hi Grant, On May 27, 2014, at 3:12 PM, Grant Likely wrote: On Mon, 26 May 2014 16:42:44 -0700, Guenter Roeck li...@roeck-us.net wrote: On 05/26/2014 03:36 PM, Sebastian Reichel wrote: Hi, On Mon, May 26, 2014 at 10:33:03PM +0100, Grant Likely wrote: After thinking about it more, I think it is very likely that removing all the overlays is the correct thing to do in the kexec use-case. When kexec-ing, it makes sense that we'd want the exact same behaviour from the kexec'ed kernel. That means we want the device drivers to do the same thing including loading whatever overlays they depend on. If the flattened tree was left applied, then the behaviour becomes different. I say always remove the overlays unless explicitly told not to, but I'm struggling to come up with use cases where keeping them applied is desirable. I would assume, that I want them applied in most cases. DT describes the hardware. If I kexec into a new kernel I change software, not hardware. Maybe I'm missing the main purpose of the feature. I currently see two useful usecases for DT overlays: 1. The dtb the kernel is booted with cannot be changed for some reason, but the board has additional hardware attached (e.g. the user added a sensor on the i2c bus) 2. The hardware is changed on the fly (e.g. the user flashed the FPGA part of a zynq processor), sensors on i2c bus, ... In both cases the kernel should be booted with the additional overlay information IMHO. Though for the second case it should be possible to remove the programmed hardware information somehow. 3. Some hot-plug device or card is inserted or removed. I would argue that the kernel should _not_ be booted with the overlay in place. Otherwise the code handling overlays would have to have special handling for the restart case, which is much more complex than just to re-insert the overlay when it is determined that the device or card is still there. Exactly. Looks like we are levitating to the 'remove overlays on kexec' approach. Is that correct? Let's just assume for a minute that this is not the case, and that loaded overlays are passed on. This would be an interesting challenge for the overlay manager, as it would have to handle a number of startup conditions. After all, it can not take it as granted that the hardware state did not change after it was stopped in the old kernel, and before it was started in the new kernel. - overlay loaded, but hardware/device no longer present - unload overlay - overlay loaded, but different hardware present - unload old overlay, load new one - overlay not loaded, hardware present - load overlay - overlay loaded and matches hardware - do nothing In comparison, its task would be quite straightforward if loaded overlays are not passed on to the new kernel. - If hardware is present, load overlay Ultimately, I seem to be missing something, as I don't really see the benefit of passing on the loaded overlay(s) to the new kernel. Activation time, maybe, for the most common case (overlay loaded and matches hardware) ? Concern though is the other cases, with a mismatch between HW and loaded overlays. I am not sure if it is even possible to ensure that there are no race conditions if the devicetree is outdated at system startup. Sure, that is unlikely to happen in 'normal' operating conditions, but that only means that it _will_ happen a few hours after the first customer starts playing with the system. Guenter -- To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH v4 2/8] OF: Introduce DT overlay support.
Hi Guenter, On May 27, 2014, at 6:21 PM, Guenter Roeck wrote: On Tue, May 27, 2014 at 03:24:35PM +0300, Pantelis Antoniou wrote: Hi Grant, On May 27, 2014, at 3:12 PM, Grant Likely wrote: On Mon, 26 May 2014 16:42:44 -0700, Guenter Roeck li...@roeck-us.net wrote: On 05/26/2014 03:36 PM, Sebastian Reichel wrote: Hi, On Mon, May 26, 2014 at 10:33:03PM +0100, Grant Likely wrote: After thinking about it more, I think it is very likely that removing all the overlays is the correct thing to do in the kexec use-case. When kexec-ing, it makes sense that we'd want the exact same behaviour from the kexec'ed kernel. That means we want the device drivers to do the same thing including loading whatever overlays they depend on. If the flattened tree was left applied, then the behaviour becomes different. I say always remove the overlays unless explicitly told not to, but I'm struggling to come up with use cases where keeping them applied is desirable. I would assume, that I want them applied in most cases. DT describes the hardware. If I kexec into a new kernel I change software, not hardware. Maybe I'm missing the main purpose of the feature. I currently see two useful usecases for DT overlays: 1. The dtb the kernel is booted with cannot be changed for some reason, but the board has additional hardware attached (e.g. the user added a sensor on the i2c bus) 2. The hardware is changed on the fly (e.g. the user flashed the FPGA part of a zynq processor), sensors on i2c bus, ... In both cases the kernel should be booted with the additional overlay information IMHO. Though for the second case it should be possible to remove the programmed hardware information somehow. 3. Some hot-plug device or card is inserted or removed. I would argue that the kernel should _not_ be booted with the overlay in place. Otherwise the code handling overlays would have to have special handling for the restart case, which is much more complex than just to re-insert the overlay when it is determined that the device or card is still there. Exactly. Looks like we are levitating to the 'remove overlays on kexec' approach. Is that correct? Let's just assume for a minute that this is not the case, and that loaded overlays are passed on. This would be an interesting challenge for the overlay manager, as it would have to handle a number of startup conditions. After all, it can not take it as granted that the hardware state did not change after it was stopped in the old kernel, and before it was started in the new kernel. - overlay loaded, but hardware/device no longer present - unload overlay - overlay loaded, but different hardware present - unload old overlay, load new one - overlay not loaded, hardware present - load overlay - overlay loaded and matches hardware - do nothing In comparison, its task would be quite straightforward if loaded overlays are not passed on to the new kernel. - If hardware is present, load overlay Yeah, exactly. The only case where having the applied overlays present on the kexec-ed kernel is either some kind of virtualization environment, or an fpga that takes an awful lot amount of time to re-initializa. Ultimately, I seem to be missing something, as I don't really see the benefit of passing on the loaded overlay(s) to the new kernel. Activation time, maybe, for the most common case (overlay loaded and matches hardware) ? Concern though is the other cases, with a mismatch between HW and loaded overlays. I am not sure if it is even possible to ensure that there are no race conditions if the devicetree is outdated at system startup. Sure, that is unlikely to happen in 'normal' operating conditions, but that only means that it _will_ happen a few hours after the first customer starts playing with the system. I concur. Accidents will happen, and if it's not a tightly controlled system breakage is guaranteed. Guenter Regards -- Pantelis -- To unsubscribe from this list: send the line unsubscribe devicetree in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html -- To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH v4 2/8] OF: Introduce DT overlay support.
Hi Günther,
On Tue, May 27, 2014 at 5:21 PM, Guenter Roeck li...@roeck-us.net wrote:
On Tue, May 27, 2014 at 03:24:35PM +0300, Pantelis Antoniou wrote:
On May 27, 2014, at 3:12 PM, Grant Likely wrote:
On Mon, 26 May 2014 16:42:44 -0700, Guenter Roeck li...@roeck-us.net
wrote:
On 05/26/2014 03:36 PM, Sebastian Reichel wrote:
On Mon, May 26, 2014 at 10:33:03PM +0100, Grant Likely wrote:
After thinking about it more, I think it is very likely that removing
all the overlays is the correct thing to do in the kexec use-case. When
kexec-ing, it makes sense that we'd want the exact same behaviour from
the kexec'ed kernel. That means we want the device drivers to do the
same thing including loading whatever overlays they depend on.
If the flattened tree was left applied, then the behaviour becomes
different.
I say always remove the overlays unless explicitly told not to, but I'm
struggling to come up with use cases where keeping them applied is
desirable.
I would assume, that I want them applied in most cases. DT describes
the hardware. If I kexec into a new kernel I change software, not
hardware.
Maybe I'm missing the main purpose of the feature. I currently see
two useful usecases for DT overlays:
1. The dtb the kernel is booted with cannot be changed for some
reason, but the board has additional hardware attached (e.g.
the user added a sensor on the i2c bus)
2. The hardware is changed on the fly (e.g. the user flashed the
FPGA part of a zynq processor), sensors on i2c bus, ...
In both cases the kernel should be booted with the additional
overlay information IMHO. Though for the second case it should
be possible to remove the programmed hardware information
somehow.
3. Some hot-plug device or card is inserted or removed.
I would argue that the kernel should _not_ be booted with the overlay in
place.
Otherwise the code handling overlays would have to have special handling
for the restart case, which is much more complex than just to re-insert
the overlay when it is determined that the device or card is still there.
Exactly.
Looks like we are levitating to the 'remove overlays on kexec' approach.
Is that correct?
Let's just assume for a minute that this is not the case, and that loaded
overlays are passed on.
This would be an interesting challenge for the overlay manager, as it would
have to handle a number of startup conditions. After all, it can not take
it as granted that the hardware state did not change after it was stopped
in the old kernel, and before it was started in the new kernel.
- overlay loaded, but hardware/device no longer present
- unload overlay
- overlay loaded, but different hardware present
- unload old overlay, load new one
- overlay not loaded, hardware present
- load overlay
- overlay loaded and matches hardware
- do nothing
In comparison, its task would be quite straightforward if loaded overlays
are not passed on to the new kernel.
- If hardware is present, load overlay
Ultimately, I seem to be missing something, as I don't really see the benefit
of passing on the loaded overlay(s) to the new kernel. Activation time, maybe,
for the most common case (overlay loaded and matches hardware) ?
All of the above assume you're using a system with overlays and an overlay
manager. I.e. you have add-on devices together with an overlay DT.
One way to handle this is via auto-detection of capes, that identify themselves,
so the overlay manager knows which overlay to load.
I'm trying to look at it in a more generic way: you add hardware which is not
in the DT, so the DT must be modified (in some way or another) to match the
hardware.
If you run kexec, the added hardware is still there. In the absence of a system
with auto-detection of capes, the newly booted kernel cannot know that
hardware has been added, compared to a pristine system. If it's in the
DT passed through kexec, everything will work.
Gr{oetje,eeting}s,
Geert
--
Geert Uytterhoeven -- There's lots of Linux beyond ia32 -- ge...@linux-m68k.org
In personal conversations with technical people, I call myself a hacker. But
when I'm talking to journalists I just say programmer or something like that.
-- Linus Torvalds
--
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH v4 2/8] OF: Introduce DT overlay support.
Hi Geert, On Tue, May 27, 2014 at 07:52:57PM +0200, Geert Uytterhoeven wrote: Hi Günther, On Tue, May 27, 2014 at 5:21 PM, Guenter Roeck li...@roeck-us.net wrote: On Tue, May 27, 2014 at 03:24:35PM +0300, Pantelis Antoniou wrote: On May 27, 2014, at 3:12 PM, Grant Likely wrote: On Mon, 26 May 2014 16:42:44 -0700, Guenter Roeck li...@roeck-us.net wrote: On 05/26/2014 03:36 PM, Sebastian Reichel wrote: On Mon, May 26, 2014 at 10:33:03PM +0100, Grant Likely wrote: After thinking about it more, I think it is very likely that removing all the overlays is the correct thing to do in the kexec use-case. When kexec-ing, it makes sense that we'd want the exact same behaviour from the kexec'ed kernel. That means we want the device drivers to do the same thing including loading whatever overlays they depend on. If the flattened tree was left applied, then the behaviour becomes different. I say always remove the overlays unless explicitly told not to, but I'm struggling to come up with use cases where keeping them applied is desirable. I would assume, that I want them applied in most cases. DT describes the hardware. If I kexec into a new kernel I change software, not hardware. Maybe I'm missing the main purpose of the feature. I currently see two useful usecases for DT overlays: 1. The dtb the kernel is booted with cannot be changed for some reason, but the board has additional hardware attached (e.g. the user added a sensor on the i2c bus) 2. The hardware is changed on the fly (e.g. the user flashed the FPGA part of a zynq processor), sensors on i2c bus, ... In both cases the kernel should be booted with the additional overlay information IMHO. Though for the second case it should be possible to remove the programmed hardware information somehow. 3. Some hot-plug device or card is inserted or removed. I would argue that the kernel should _not_ be booted with the overlay in place. Otherwise the code handling overlays would have to have special handling for the restart case, which is much more complex than just to re-insert the overlay when it is determined that the device or card is still there. Exactly. Looks like we are levitating to the 'remove overlays on kexec' approach. Is that correct? Let's just assume for a minute that this is not the case, and that loaded overlays are passed on. This would be an interesting challenge for the overlay manager, as it would have to handle a number of startup conditions. After all, it can not take it as granted that the hardware state did not change after it was stopped in the old kernel, and before it was started in the new kernel. - overlay loaded, but hardware/device no longer present - unload overlay - overlay loaded, but different hardware present - unload old overlay, load new one - overlay not loaded, hardware present - load overlay - overlay loaded and matches hardware - do nothing In comparison, its task would be quite straightforward if loaded overlays are not passed on to the new kernel. - If hardware is present, load overlay Ultimately, I seem to be missing something, as I don't really see the benefit of passing on the loaded overlay(s) to the new kernel. Activation time, maybe, for the most common case (overlay loaded and matches hardware) ? All of the above assume you're using a system with overlays and an overlay manager. I.e. you have add-on devices together with an overlay DT. One way to handle this is via auto-detection of capes, that identify themselves, so the overlay manager knows which overlay to load. Possibly auto-detected (and in our case that is so), but not necessarily. A user space overlay manager, which keeps state in user space, would be possible as well. I'm trying to look at it in a more generic way: you add hardware which is not in the DT, so the DT must be modified (in some way or another) to match the hardware. If you run kexec, the added hardware is still there. In the absence of a system with auto-detection of capes, the newly booted kernel cannot know that hardware has been added, compared to a pristine system. If it's in the DT passed through kexec, everything will work. Yes, but only for that case. You'd still have the problem that you'll need to keep the state somewhere to be able to remove the pre-loaded overlay(s) later on. If you don't have hardware auto-detection, I would assume that to be in user space. No matter if the system auto-detects hardware or not, the same mechanism could be used: If, on startup, the hardware status (no matter if kept in some file or auto-detected) indicates that an overlay should be loaded, load it. If, for some reason, the overlay manager can not auto-detect hardware but keeps state in the kernel, it may make sense to have that state passed to
Re: [PATCH v4 2/8] OF: Introduce DT overlay support.
On Tue, 27 May 2014 15:24:35 +0300, Pantelis Antoniou pantelis.anton...@konsulko.com wrote: Hi Grant, On May 27, 2014, at 3:12 PM, Grant Likely wrote: On Mon, 26 May 2014 16:42:44 -0700, Guenter Roeck li...@roeck-us.net wrote: On 05/26/2014 03:36 PM, Sebastian Reichel wrote: Hi, On Mon, May 26, 2014 at 10:33:03PM +0100, Grant Likely wrote: After thinking about it more, I think it is very likely that removing all the overlays is the correct thing to do in the kexec use-case. When kexec-ing, it makes sense that we'd want the exact same behaviour from the kexec'ed kernel. That means we want the device drivers to do the same thing including loading whatever overlays they depend on. If the flattened tree was left applied, then the behaviour becomes different. I say always remove the overlays unless explicitly told not to, but I'm struggling to come up with use cases where keeping them applied is desirable. I would assume, that I want them applied in most cases. DT describes the hardware. If I kexec into a new kernel I change software, not hardware. Maybe I'm missing the main purpose of the feature. I currently see two useful usecases for DT overlays: 1. The dtb the kernel is booted with cannot be changed for some reason, but the board has additional hardware attached (e.g. the user added a sensor on the i2c bus) 2. The hardware is changed on the fly (e.g. the user flashed the FPGA part of a zynq processor), sensors on i2c bus, ... In both cases the kernel should be booted with the additional overlay information IMHO. Though for the second case it should be possible to remove the programmed hardware information somehow. 3. Some hot-plug device or card is inserted or removed. I would argue that the kernel should _not_ be booted with the overlay in place. Otherwise the code handling overlays would have to have special handling for the restart case, which is much more complex than just to re-insert the overlay when it is determined that the device or card is still there. Exactly. Looks like we are levitating to the 'remove overlays on kexec' approach. Is that correct? Yes. What form that takes is yet to be decided. Kexec traditionally has worked by userspace extracting the device tree from /proc/device-tree, getting a kernel image into memory, and then doing the kexec call. It wouldn't be very friendly to have kexec force all the overlays to be removed while the kernel is still running. We could solve this by making kexec use the original dtb blob instead of the livetree. Otherwise we'd need an interface for userspace to extract the original tree, or have the information to unwind the overlays. Not exactly simple. g. -- To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH v4 2/8] OF: Introduce DT overlay support.
On 05/26/2014 05:32 PM, Sebastian Reichel wrote: On Mon, May 26, 2014 at 04:42:44PM -0700, Guenter Roeck wrote: On 05/26/2014 03:36 PM, Sebastian Reichel wrote: On Mon, May 26, 2014 at 10:33:03PM +0100, Grant Likely wrote: After thinking about it more, I think it is very likely that removing all the overlays is the correct thing to do in the kexec use-case. When kexec-ing, it makes sense that we'd want the exact same behaviour from the kexec'ed kernel. That means we want the device drivers to do the same thing including loading whatever overlays they depend on. If the flattened tree was left applied, then the behaviour becomes different. I say always remove the overlays unless explicitly told not to, but I'm struggling to come up with use cases where keeping them applied is desirable. I would assume, that I want them applied in most cases. DT describes the hardware. If I kexec into a new kernel I change software, not hardware. Maybe I'm missing the main purpose of the feature. I currently see two useful usecases for DT overlays: 1. The dtb the kernel is booted with cannot be changed for some reason, but the board has additional hardware attached (e.g. the user added a sensor on the i2c bus) 2. The hardware is changed on the fly (e.g. the user flashed the FPGA part of a zynq processor), sensors on i2c bus, ... In both cases the kernel should be booted with the additional overlay information IMHO. Though for the second case it should be possible to remove the "programmed" hardware information somehow. 3. Some hot-plug device or card is inserted or removed. Can you give a more specific example? I guess most hot-plug devices are connected to busses, which are not described via DT, but support auto-identification (USB, PCI, ...) The card interface provides i2c and pcie busses, plus a number of gpio pins. Both I2C devices and PCIe devices depend on the inserted card type. There may be PCIe switches on some cards, or just PCIe devices on others. Auto-identification of PCIe devices does not help, as the device configuration depends on the card type inserted. A typical example is that the card may include a multi-function FPGA device with LED, GPIO, and I2C bus master functionality, and the specific configuration depends on the card type (even though the FPGA is the same). Other cards may include a PCIe switch with a number of ASICs connected to it. PCIe switch configuration depends on the card type, not on the PCIe switch type. I would argue that the kernel should _not_ be booted with the overlay in place. well the device is still attached to the system when you kexec into the new kernel, isn't it? So ? the code executed is ultimately the same, if the kernel is booted from rommon or from kexec. In one case we'll have to insert the overlay, in the other case not. Otherwise the code handling overlays would have to have special handling for the restart case, which is much more complex than just to re-insert the overlay when it is determined that the device or card is still there. I assume, that the kernel cannot auto-detect the attached hardware. Correct. Otherwise we don't need the DT entries, but can simply scan the bus. So the restart case (or restart + kexec case if kexec behaves like a restart) means, that userspace needs to provide the information about device existence. Exactly the same informatin we get if the kernel is booted from ROMMON or BIOS. No difference here. Removing the overlay is like dropping information supplied from the user. Not something, which should be done carelessly. In our case it is done when the card is removed. Guenter -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH v4 2/8] OF: Introduce DT overlay support.
On Mon, May 26, 2014 at 04:42:44PM -0700, Guenter Roeck wrote: > On 05/26/2014 03:36 PM, Sebastian Reichel wrote: > >On Mon, May 26, 2014 at 10:33:03PM +0100, Grant Likely wrote: > >>After thinking about it more, I think it is very likely that removing > >>all the overlays is the correct thing to do in the kexec use-case. When > >>kexec-ing, it makes sense that we'd want the exact same behaviour from > >>the kexec'ed kernel. That means we want the device drivers to do the > >>same thing including loading whatever overlays they depend on. > >> > >>If the flattened tree was left applied, then the behaviour becomes > >>different. > >> > >>I say always remove the overlays unless explicitly told not to, but I'm > >>struggling to come up with use cases where keeping them applied is > >>desirable. > > > >I would assume, that I want them applied in most cases. DT describes > >the hardware. If I kexec into a new kernel I change software, not > >hardware. > > > >Maybe I'm missing the main purpose of the feature. I currently see > >two useful usecases for DT overlays: > > > >1. The dtb the kernel is booted with cannot be changed for some > >reason, but the board has additional hardware attached (e.g. > >the user added a sensor on the i2c bus) > >2. The hardware is changed on the fly (e.g. the user flashed the > >FPGA part of a zynq processor), sensors on i2c bus, ... > > > >In both cases the kernel should be booted with the additional > >overlay information IMHO. Though for the second case it should > >be possible to remove the "programmed" hardware information > >somehow. > > > > 3. Some hot-plug device or card is inserted or removed. Can you give a more specific example? I guess most hot-plug devices are connected to busses, which are not described via DT, but support auto-identification (USB, PCI, ...) > I would argue that the kernel should _not_ be booted with the > overlay in place. well the device is still attached to the system when you kexec into the new kernel, isn't it? > Otherwise the code handling overlays would have to have special > handling for the restart case, which is much more complex than > just to re-insert the overlay when it is determined that the > device or card is still there. I assume, that the kernel cannot auto-detect the attached hardware. Otherwise we don't need the DT entries, but can simply scan the bus. So the restart case (or restart + kexec case if kexec behaves like a restart) means, that userspace needs to provide the information about device existence. Removing the overlay is like dropping information supplied from the user. Not something, which should be done carelessly. -- Sebastian signature.asc Description: Digital signature
Re: [PATCH v4 2/8] OF: Introduce DT overlay support.
On 05/26/2014 02:44 PM, Geert Uytterhoeven wrote: Hi Grant, On Mon, May 26, 2014 at 11:33 PM, Grant Likely wrote: After thinking about it more, I think it is very likely that removing all the overlays is the correct thing to do in the kexec use-case. When kexec-ing, it makes sense that we'd want the exact same behaviour from the kexec'ed kernel. That means we want the device drivers to do the same thing including loading whatever overlays they depend on. Are the device drivers loading the overlays? That sounds a bit backwards to me. Depends on what you mean with 'device driver'. In our case we have a driver dedicated to handle card status. Quite similar to what the connector subsystem does but with significantly more functionality (including devicetree overlay management). Guenter -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH v4 2/8] OF: Introduce DT overlay support.
On 05/26/2014 03:36 PM, Sebastian Reichel wrote: Hi, On Mon, May 26, 2014 at 10:33:03PM +0100, Grant Likely wrote: After thinking about it more, I think it is very likely that removing all the overlays is the correct thing to do in the kexec use-case. When kexec-ing, it makes sense that we'd want the exact same behaviour from the kexec'ed kernel. That means we want the device drivers to do the same thing including loading whatever overlays they depend on. If the flattened tree was left applied, then the behaviour becomes different. I say always remove the overlays unless explicitly told not to, but I'm struggling to come up with use cases where keeping them applied is desirable. I would assume, that I want them applied in most cases. DT describes the hardware. If I kexec into a new kernel I change software, not hardware. Maybe I'm missing the main purpose of the feature. I currently see two useful usecases for DT overlays: 1. The dtb the kernel is booted with cannot be changed for some reason, but the board has additional hardware attached (e.g. the user added a sensor on the i2c bus) 2. The hardware is changed on the fly (e.g. the user flashed the FPGA part of a zynq processor), sensors on i2c bus, ... In both cases the kernel should be booted with the additional overlay information IMHO. Though for the second case it should be possible to remove the "programmed" hardware information somehow. 3. Some hot-plug device or card is inserted or removed. I would argue that the kernel should _not_ be booted with the overlay in place. Otherwise the code handling overlays would have to have special handling for the restart case, which is much more complex than just to re-insert the overlay when it is determined that the device or card is still there. Guenter -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH v4 2/8] OF: Introduce DT overlay support.
Hi, On Mon, May 26, 2014 at 08:14:08AM -0700, Guenter Roeck wrote: > On 05/26/2014 08:09 AM, Sebastian Reichel wrote: > >On Mon, May 26, 2014 at 02:55:37PM +0300, Pantelis Antoniou wrote: > >>On May 26, 2014, at 2:23 PM, Grant Likely wrote: > >>>On Mon, 26 May 2014 12:57:32 +0200, Geert Uytterhoeven > >>> wrote: > >>>Heeheehee. We're back where we started. The original question is whether > >>>or not that is a valid approach. If the overlay represents something > >>>that can be hot plugged/unplugged, then passing it through to the second > >>>kernel would be the wrong thing to do. If it was a permenant addition, > >>>then it probably doesn't need to be removed. > >>> > >>>We do actually keep the overlay info in memory for the purpose of > >>>removal exactly so we can support hot unbinding of devices and drivers > >>>that make use of overlays. > >> > >>We can support either method. I am not feeling any wiser about which one > >>should be > >>the default TBH, so what about exporting a property and let the platform > >>figure out which is more appropriate? > > > >What about supporting "negative" overlays (so an overlay, that > >removes DT entries)? That way one could reverse apply an overlay. > >All the dependency stuff would basically be the users problem. The > >kernel only checks if it can apply an overlay (and return some error > >code if it can't). This this code is needed anyway to check the > >input from userspace. > > > > Does that mean that I would need to describe such a negative overlay > for each overlay to be able to get it removed ? > > This would introduce an endless source of problems with bad "reverse" > overlay descriptions. Sure, that would "be the users problem", > but I don't think that would make it better. I was thinking about supporting something like "patch --reverse". So you can try to undo the overlay by reverse applying it and you can do it in arbitrary order. Note: The dependency check must be done for all overlays coming from userspace, so that's not a problem __here__. The reverse method can "simply" reverse the overlay patch and apply it like a normal overlay from userspace (and thus using the same dependency checks). -- Sebastian signature.asc Description: Digital signature
Re: [PATCH v4 2/8] OF: Introduce DT overlay support.
Hi, On Mon, May 26, 2014 at 10:33:03PM +0100, Grant Likely wrote: > After thinking about it more, I think it is very likely that removing > all the overlays is the correct thing to do in the kexec use-case. When > kexec-ing, it makes sense that we'd want the exact same behaviour from > the kexec'ed kernel. That means we want the device drivers to do the > same thing including loading whatever overlays they depend on. > > If the flattened tree was left applied, then the behaviour becomes > different. > > I say always remove the overlays unless explicitly told not to, but I'm > struggling to come up with use cases where keeping them applied is > desirable. I would assume, that I want them applied in most cases. DT describes the hardware. If I kexec into a new kernel I change software, not hardware. Maybe I'm missing the main purpose of the feature. I currently see two useful usecases for DT overlays: 1. The dtb the kernel is booted with cannot be changed for some reason, but the board has additional hardware attached (e.g. the user added a sensor on the i2c bus) 2. The hardware is changed on the fly (e.g. the user flashed the FPGA part of a zynq processor), sensors on i2c bus, ... In both cases the kernel should be booted with the additional overlay information IMHO. Though for the second case it should be possible to remove the "programmed" hardware information somehow. -- Sebastian signature.asc Description: Digital signature
Re: [PATCH v4 2/8] OF: Introduce DT overlay support.
Hi Grant,
On Mon, May 26, 2014 at 11:33 PM, Grant Likely
wrote:
> After thinking about it more, I think it is very likely that removing
> all the overlays is the correct thing to do in the kexec use-case. When
> kexec-ing, it makes sense that we'd want the exact same behaviour from
> the kexec'ed kernel. That means we want the device drivers to do the
> same thing including loading whatever overlays they depend on.
Are the device drivers loading the overlays?
That sounds a bit backwards to me.
Gr{oetje,eeting}s,
Geert
--
Geert Uytterhoeven -- There's lots of Linux beyond ia32 -- ge...@linux-m68k.org
In personal conversations with technical people, I call myself a hacker. But
when I'm talking to journalists I just say "programmer" or something like that.
-- Linus Torvalds
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH v4 2/8] OF: Introduce DT overlay support.
On Mon, 26 May 2014 14:55:37 +0300, Pantelis Antoniou wrote: > Hi Grant, > > On May 26, 2014, at 2:23 PM, Grant Likely wrote: > > > On Mon, 26 May 2014 12:57:32 +0200, Geert Uytterhoeven > > wrote: > >> Hi Grant, > >> > >> On Mon, May 26, 2014 at 12:48 PM, Grant Likely > >> wrote: > >>> On Tue, 20 May 2014 09:38:49 +0200, Geert Uytterhoeven > >>> wrote: > On Tue, May 20, 2014 at 7:50 AM, Grant Likely > wrote: > >> Why has the overlay system been designed for plugging and unpluging > >> whole > >> overlays? > >> That means the kernel has to remember the full stack, causing issues > >> with > >> e.g. kexec. > > > > Mostly so that drivers don't see any difference in the livetree data > > structure. It also means that userspace sees a single representation of > > the hardware at any given time. > > Sorry, I don't follow the argument about the "single representation of > the > hardware". > >>> > >>> Er, s/of the hardware/of the tree/. Right now the overlay design > >>> modifies the live tree which at the same time modifies the tree > >>> representation in /sys/firmware/devicetree. If the design was changed to > >>> keep the overlay logically separate, then I would think we want to > >>> expose that information to usespace also. In fact, I think we would need > >>> to for usecases like kexec. > >> > >> OK, so it does modify the real tree, and doesn't keep the actual overlays. > >> > >> I was under the impression the overlay stack was also kept in memory, to > >> allow > >> reversal, so there was a misunderstanding. > >> > >> Hence for kexec, the tree in /sys/firmware/devicetree can just be passed > >> to the new kernel, as that's the current representation of the hardware? > > > > Heeheehee. We're back where we started. The original question is whether > > or not that is a valid approach. If the overlay represents something > > that can be hot plugged/unplugged, then passing it through to the second > > kernel would be the wrong thing to do. If it was a permenant addition, > > then it probably doesn't need to be removed. > > > > We do actually keep the overlay info in memory for the purpose of > > removal exactly so we can support hot unbinding of devices and drivers > > that make use of overlays. > > > > We can support either method. I am not feeling any wiser about which one > should be > the default TBH, so what about exporting a property and let the platform > figure out which is more appropriate? After thinking about it more, I think it is very likely that removing all the overlays is the correct thing to do in the kexec use-case. When kexec-ing, it makes sense that we'd want the exact same behaviour from the kexec'ed kernel. That means we want the device drivers to do the same thing including loading whatever overlays they depend on. If the flattened tree was left applied, then the behaviour becomes different. I say always remove the overlays unless explicitly told not to, but I'm struggling to come up with use cases where keeping them applied is desirable. g. -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH v4 2/8] OF: Introduce DT overlay support.
Hi Sebastian,
On May 26, 2014, at 6:09 PM, Sebastian Reichel wrote:
> Hi,
>
> On Mon, May 26, 2014 at 02:55:37PM +0300, Pantelis Antoniou wrote:
>> On May 26, 2014, at 2:23 PM, Grant Likely wrote:
>>> On Mon, 26 May 2014 12:57:32 +0200, Geert Uytterhoeven
>>> wrote:
>>> Heeheehee. We're back where we started. The original question is whether
>>> or not that is a valid approach. If the overlay represents something
>>> that can be hot plugged/unplugged, then passing it through to the second
>>> kernel would be the wrong thing to do. If it was a permenant addition,
>>> then it probably doesn't need to be removed.
>>>
>>> We do actually keep the overlay info in memory for the purpose of
>>> removal exactly so we can support hot unbinding of devices and drivers
>>> that make use of overlays.
>>
>> We can support either method. I am not feeling any wiser about which one
>> should be
>> the default TBH, so what about exporting a property and let the platform
>> figure out which is more appropriate?
>
> What about supporting "negative" overlays (so an overlay, that
> removes DT entries)? That way one could reverse apply an overlay.
> All the dependency stuff would basically be the users problem. The
> kernel only checks if it can apply an overlay (and return some error
> code if it can't). This this code is needed anyway to check the
> input from userspace.
>
> As a result the overlay handling would basically have the same
> behaviour as diff and patch :)
>
This is already supported; nodes and properties can be prefixed with a minus
sign '-' and operate as expected :)
i.e. "-property;" removes a property and "-node { };" removes a node.
> P.S.: Sorry if this has already been suggested. I have only read
> mails from PATCHv4.
>
> -- Sebastian
Regards
-- Pantelis
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH v4 2/8] OF: Introduce DT overlay support.
On 05/26/2014 08:09 AM, Sebastian Reichel wrote: Hi, On Mon, May 26, 2014 at 02:55:37PM +0300, Pantelis Antoniou wrote: On May 26, 2014, at 2:23 PM, Grant Likely wrote: On Mon, 26 May 2014 12:57:32 +0200, Geert Uytterhoeven wrote: Heeheehee. We're back where we started. The original question is whether or not that is a valid approach. If the overlay represents something that can be hot plugged/unplugged, then passing it through to the second kernel would be the wrong thing to do. If it was a permenant addition, then it probably doesn't need to be removed. We do actually keep the overlay info in memory for the purpose of removal exactly so we can support hot unbinding of devices and drivers that make use of overlays. We can support either method. I am not feeling any wiser about which one should be the default TBH, so what about exporting a property and let the platform figure out which is more appropriate? What about supporting "negative" overlays (so an overlay, that removes DT entries)? That way one could reverse apply an overlay. All the dependency stuff would basically be the users problem. The kernel only checks if it can apply an overlay (and return some error code if it can't). This this code is needed anyway to check the input from userspace. Does that mean that I would need to describe such a negative overlay for each overlay to be able to get it removed ? This would introduce an endless source of problems with bad "reverse" overlay descriptions. Sure, that would "be the users problem", but I don't think that would make it better. Guenter -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH v4 2/8] OF: Introduce DT overlay support.
Hi, On Mon, May 26, 2014 at 02:55:37PM +0300, Pantelis Antoniou wrote: > On May 26, 2014, at 2:23 PM, Grant Likely wrote: > > On Mon, 26 May 2014 12:57:32 +0200, Geert Uytterhoeven > > wrote: > > Heeheehee. We're back where we started. The original question is whether > > or not that is a valid approach. If the overlay represents something > > that can be hot plugged/unplugged, then passing it through to the second > > kernel would be the wrong thing to do. If it was a permenant addition, > > then it probably doesn't need to be removed. > > > > We do actually keep the overlay info in memory for the purpose of > > removal exactly so we can support hot unbinding of devices and drivers > > that make use of overlays. > > We can support either method. I am not feeling any wiser about which one > should be > the default TBH, so what about exporting a property and let the platform > figure out which is more appropriate? What about supporting "negative" overlays (so an overlay, that removes DT entries)? That way one could reverse apply an overlay. All the dependency stuff would basically be the users problem. The kernel only checks if it can apply an overlay (and return some error code if it can't). This this code is needed anyway to check the input from userspace. As a result the overlay handling would basically have the same behaviour as diff and patch :) P.S.: Sorry if this has already been suggested. I have only read mails from PATCHv4. -- Sebastian signature.asc Description: Digital signature
Re: [PATCH v4 2/8] OF: Introduce DT overlay support.
Hi Grant, On May 26, 2014, at 2:23 PM, Grant Likely wrote: > On Mon, 26 May 2014 12:57:32 +0200, Geert Uytterhoeven > wrote: >> Hi Grant, >> >> On Mon, May 26, 2014 at 12:48 PM, Grant Likely >> wrote: >>> On Tue, 20 May 2014 09:38:49 +0200, Geert Uytterhoeven >>> wrote: On Tue, May 20, 2014 at 7:50 AM, Grant Likely wrote: >> Why has the overlay system been designed for plugging and unpluging whole >> overlays? >> That means the kernel has to remember the full stack, causing issues with >> e.g. kexec. > > Mostly so that drivers don't see any difference in the livetree data > structure. It also means that userspace sees a single representation of > the hardware at any given time. Sorry, I don't follow the argument about the "single representation of the hardware". >>> >>> Er, s/of the hardware/of the tree/. Right now the overlay design >>> modifies the live tree which at the same time modifies the tree >>> representation in /sys/firmware/devicetree. If the design was changed to >>> keep the overlay logically separate, then I would think we want to >>> expose that information to usespace also. In fact, I think we would need >>> to for usecases like kexec. >> >> OK, so it does modify the real tree, and doesn't keep the actual overlays. >> >> I was under the impression the overlay stack was also kept in memory, to >> allow >> reversal, so there was a misunderstanding. >> >> Hence for kexec, the tree in /sys/firmware/devicetree can just be passed >> to the new kernel, as that's the current representation of the hardware? > > Heeheehee. We're back where we started. The original question is whether > or not that is a valid approach. If the overlay represents something > that can be hot plugged/unplugged, then passing it through to the second > kernel would be the wrong thing to do. If it was a permenant addition, > then it probably doesn't need to be removed. > > We do actually keep the overlay info in memory for the purpose of > removal exactly so we can support hot unbinding of devices and drivers > that make use of overlays. > We can support either method. I am not feeling any wiser about which one should be the default TBH, so what about exporting a property and let the platform figure out which is more appropriate? > g. > Regards -- Pantelis > -- > To unsubscribe from this list: send the line "unsubscribe devicetree" in > the body of a message to majord...@vger.kernel.org > More majordomo info at http://vger.kernel.org/majordomo-info.html -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH v4 2/8] OF: Introduce DT overlay support.
On Mon, 26 May 2014 12:57:32 +0200, Geert Uytterhoeven wrote: > Hi Grant, > > On Mon, May 26, 2014 at 12:48 PM, Grant Likely > wrote: > > On Tue, 20 May 2014 09:38:49 +0200, Geert Uytterhoeven > > wrote: > >> On Tue, May 20, 2014 at 7:50 AM, Grant Likely > >> wrote: > >> >> Why has the overlay system been designed for plugging and unpluging > >> >> whole > >> >> overlays? > >> >> That means the kernel has to remember the full stack, causing issues > >> >> with > >> >> e.g. kexec. > >> > > >> > Mostly so that drivers don't see any difference in the livetree data > >> > structure. It also means that userspace sees a single representation of > >> > the hardware at any given time. > >> > >> Sorry, I don't follow the argument about the "single representation of the > >> hardware". > > > > Er, s/of the hardware/of the tree/. Right now the overlay design > > modifies the live tree which at the same time modifies the tree > > representation in /sys/firmware/devicetree. If the design was changed to > > keep the overlay logically separate, then I would think we want to > > expose that information to usespace also. In fact, I think we would need > > to for usecases like kexec. > > OK, so it does modify the real tree, and doesn't keep the actual overlays. > > I was under the impression the overlay stack was also kept in memory, to allow > reversal, so there was a misunderstanding. > > Hence for kexec, the tree in /sys/firmware/devicetree can just be passed > to the new kernel, as that's the current representation of the hardware? Heeheehee. We're back where we started. The original question is whether or not that is a valid approach. If the overlay represents something that can be hot plugged/unplugged, then passing it through to the second kernel would be the wrong thing to do. If it was a permenant addition, then it probably doesn't need to be removed. We do actually keep the overlay info in memory for the purpose of removal exactly so we can support hot unbinding of devices and drivers that make use of overlays. g. -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH v4 2/8] OF: Introduce DT overlay support.
Hi Geert,
On May 26, 2014, at 1:57 PM, Geert Uytterhoeven wrote:
> Hi Grant,
>
> On Mon, May 26, 2014 at 12:48 PM, Grant Likely
> wrote:
>> On Tue, 20 May 2014 09:38:49 +0200, Geert Uytterhoeven
>> wrote:
>>> On Tue, May 20, 2014 at 7:50 AM, Grant Likely
>>> wrote:
> Why has the overlay system been designed for plugging and unpluging whole
> overlays?
> That means the kernel has to remember the full stack, causing issues with
> e.g. kexec.
Mostly so that drivers don't see any difference in the livetree data
structure. It also means that userspace sees a single representation of
the hardware at any given time.
>>>
>>> Sorry, I don't follow the argument about the "single representation of the
>>> hardware".
>>
>> Er, s/of the hardware/of the tree/. Right now the overlay design
>> modifies the live tree which at the same time modifies the tree
>> representation in /sys/firmware/devicetree. If the design was changed to
>> keep the overlay logically separate, then I would think we want to
>> expose that information to usespace also. In fact, I think we would need
>> to for usecases like kexec.
>
> OK, so it does modify the real tree, and doesn't keep the actual overlays.
>
It modifies the actual tree, and it keeps a log of each modification made to the
tree so that it will be able to revert the changes made.
> I was under the impression the overlay stack was also kept in memory, to allow
> reversal, so there was a misunderstanding.
>
Yep.
> Hence for kexec, the tree in /sys/firmware/devicetree can just be passed
> to the new kernel, as that's the current representation of the hardware?
>
Exactly.
> Gr{oetje,eeting}s,
>
>Geert
>
Regards
-- Pantelis
> --
> Geert Uytterhoeven -- There's lots of Linux beyond ia32 --
> ge...@linux-m68k.org
>
> In personal conversations with technical people, I call myself a hacker. But
> when I'm talking to journalists I just say "programmer" or something like
> that.
>-- Linus Torvalds
> --
> To unsubscribe from this list: send the line "unsubscribe devicetree" in
> the body of a message to majord...@vger.kernel.org
> More majordomo info at http://vger.kernel.org/majordomo-info.html
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH v4 2/8] OF: Introduce DT overlay support.
Hi Grant,
On Mon, May 26, 2014 at 12:48 PM, Grant Likely
wrote:
> On Tue, 20 May 2014 09:38:49 +0200, Geert Uytterhoeven
> wrote:
>> On Tue, May 20, 2014 at 7:50 AM, Grant Likely
>> wrote:
>> >> Why has the overlay system been designed for plugging and unpluging whole
>> >> overlays?
>> >> That means the kernel has to remember the full stack, causing issues with
>> >> e.g. kexec.
>> >
>> > Mostly so that drivers don't see any difference in the livetree data
>> > structure. It also means that userspace sees a single representation of
>> > the hardware at any given time.
>>
>> Sorry, I don't follow the argument about the "single representation of the
>> hardware".
>
> Er, s/of the hardware/of the tree/. Right now the overlay design
> modifies the live tree which at the same time modifies the tree
> representation in /sys/firmware/devicetree. If the design was changed to
> keep the overlay logically separate, then I would think we want to
> expose that information to usespace also. In fact, I think we would need
> to for usecases like kexec.
OK, so it does modify the real tree, and doesn't keep the actual overlays.
I was under the impression the overlay stack was also kept in memory, to allow
reversal, so there was a misunderstanding.
Hence for kexec, the tree in /sys/firmware/devicetree can just be passed
to the new kernel, as that's the current representation of the hardware?
Gr{oetje,eeting}s,
Geert
--
Geert Uytterhoeven -- There's lots of Linux beyond ia32 -- ge...@linux-m68k.org
In personal conversations with technical people, I call myself a hacker. But
when I'm talking to journalists I just say "programmer" or something like that.
-- Linus Torvalds
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH v4 2/8] OF: Introduce DT overlay support.
On Tue, 20 May 2014 09:38:49 +0200, Geert Uytterhoeven wrote: > Hi Grant, > > On Tue, May 20, 2014 at 7:50 AM, Grant Likely > wrote: > >> Why has the overlay system been designed for plugging and unpluging whole > >> overlays? > >> That means the kernel has to remember the full stack, causing issues with > >> e.g. kexec. > > > > Mostly so that drivers don't see any difference in the livetree data > > structure. It also means that userspace sees a single representation of > > the hardware at any given time. > > Sorry, I don't follow the argument about the "single representation of the > hardware". Er, s/of the hardware/of the tree/. Right now the overlay design modifies the live tree which at the same time modifies the tree representation in /sys/firmware/devicetree. If the design was changed to keep the overlay logically separate, then I would think we want to expose that information to usespace also. In fact, I think we would need to for usecases like kexec. > > >> Why not allowing the addition of removal of subtrees of the full device > >> tree? > > > > Overlays is more than just a subtree. A single overlay can make > > manipulations of multiple subtrees that should be handled as logically > > atomic. > > Sure, it's more complicated due to the atomicity of multiple changes. > > >> This is similar to other hotpluggable subsystems (which are not necessarily > >> DT-based), like PCI Express. That way the kernel can pass a > >> DT-representation of the actual current device tree to a kexec'ed kernel. > > > > I'm not following you argument. Hardware hotplug systems like PCIe don't > > manipulate the firmware data. The kernel detects the new device and > > populates the Linux device model directly. Firmware provided data (ACPI > > or FDT) isn't involved. > > I mean the kernel doesn't remember the exact order in a stack, to reverse > operations. E.g. I can add hotplug a PCIe bridge with multiple devices > behind it, and unplug a single device later. It's still one subtree, though. The problem comes when one overlay adds a node or configuration that is depended on by the second overlay, but there isn't a direct reference by the second overlay into the first. g. -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH v4 2/8] OF: Introduce DT overlay support.
On Tue, 20 May 2014 09:38:49 +0200, Geert Uytterhoeven ge...@linux-m68k.org wrote: Hi Grant, On Tue, May 20, 2014 at 7:50 AM, Grant Likely grant.lik...@secretlab.ca wrote: Why has the overlay system been designed for plugging and unpluging whole overlays? That means the kernel has to remember the full stack, causing issues with e.g. kexec. Mostly so that drivers don't see any difference in the livetree data structure. It also means that userspace sees a single representation of the hardware at any given time. Sorry, I don't follow the argument about the single representation of the hardware. Er, s/of the hardware/of the tree/. Right now the overlay design modifies the live tree which at the same time modifies the tree representation in /sys/firmware/devicetree. If the design was changed to keep the overlay logically separate, then I would think we want to expose that information to usespace also. In fact, I think we would need to for usecases like kexec. Why not allowing the addition of removal of subtrees of the full device tree? Overlays is more than just a subtree. A single overlay can make manipulations of multiple subtrees that should be handled as logically atomic. Sure, it's more complicated due to the atomicity of multiple changes. This is similar to other hotpluggable subsystems (which are not necessarily DT-based), like PCI Express. That way the kernel can pass a DT-representation of the actual current device tree to a kexec'ed kernel. I'm not following you argument. Hardware hotplug systems like PCIe don't manipulate the firmware data. The kernel detects the new device and populates the Linux device model directly. Firmware provided data (ACPI or FDT) isn't involved. I mean the kernel doesn't remember the exact order in a stack, to reverse operations. E.g. I can add hotplug a PCIe bridge with multiple devices behind it, and unplug a single device later. It's still one subtree, though. The problem comes when one overlay adds a node or configuration that is depended on by the second overlay, but there isn't a direct reference by the second overlay into the first. g. -- To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH v4 2/8] OF: Introduce DT overlay support.
Hi Grant,
On Mon, May 26, 2014 at 12:48 PM, Grant Likely
grant.lik...@secretlab.ca wrote:
On Tue, 20 May 2014 09:38:49 +0200, Geert Uytterhoeven ge...@linux-m68k.org
wrote:
On Tue, May 20, 2014 at 7:50 AM, Grant Likely grant.lik...@secretlab.ca
wrote:
Why has the overlay system been designed for plugging and unpluging whole
overlays?
That means the kernel has to remember the full stack, causing issues with
e.g. kexec.
Mostly so that drivers don't see any difference in the livetree data
structure. It also means that userspace sees a single representation of
the hardware at any given time.
Sorry, I don't follow the argument about the single representation of the
hardware.
Er, s/of the hardware/of the tree/. Right now the overlay design
modifies the live tree which at the same time modifies the tree
representation in /sys/firmware/devicetree. If the design was changed to
keep the overlay logically separate, then I would think we want to
expose that information to usespace also. In fact, I think we would need
to for usecases like kexec.
OK, so it does modify the real tree, and doesn't keep the actual overlays.
I was under the impression the overlay stack was also kept in memory, to allow
reversal, so there was a misunderstanding.
Hence for kexec, the tree in /sys/firmware/devicetree can just be passed
to the new kernel, as that's the current representation of the hardware?
Gr{oetje,eeting}s,
Geert
--
Geert Uytterhoeven -- There's lots of Linux beyond ia32 -- ge...@linux-m68k.org
In personal conversations with technical people, I call myself a hacker. But
when I'm talking to journalists I just say programmer or something like that.
-- Linus Torvalds
--
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH v4 2/8] OF: Introduce DT overlay support.
Hi Geert,
On May 26, 2014, at 1:57 PM, Geert Uytterhoeven wrote:
Hi Grant,
On Mon, May 26, 2014 at 12:48 PM, Grant Likely
grant.lik...@secretlab.ca wrote:
On Tue, 20 May 2014 09:38:49 +0200, Geert Uytterhoeven
ge...@linux-m68k.org wrote:
On Tue, May 20, 2014 at 7:50 AM, Grant Likely grant.lik...@secretlab.ca
wrote:
Why has the overlay system been designed for plugging and unpluging whole
overlays?
That means the kernel has to remember the full stack, causing issues with
e.g. kexec.
Mostly so that drivers don't see any difference in the livetree data
structure. It also means that userspace sees a single representation of
the hardware at any given time.
Sorry, I don't follow the argument about the single representation of the
hardware.
Er, s/of the hardware/of the tree/. Right now the overlay design
modifies the live tree which at the same time modifies the tree
representation in /sys/firmware/devicetree. If the design was changed to
keep the overlay logically separate, then I would think we want to
expose that information to usespace also. In fact, I think we would need
to for usecases like kexec.
OK, so it does modify the real tree, and doesn't keep the actual overlays.
It modifies the actual tree, and it keeps a log of each modification made to the
tree so that it will be able to revert the changes made.
I was under the impression the overlay stack was also kept in memory, to allow
reversal, so there was a misunderstanding.
Yep.
Hence for kexec, the tree in /sys/firmware/devicetree can just be passed
to the new kernel, as that's the current representation of the hardware?
Exactly.
Gr{oetje,eeting}s,
Geert
Regards
-- Pantelis
--
Geert Uytterhoeven -- There's lots of Linux beyond ia32 --
ge...@linux-m68k.org
In personal conversations with technical people, I call myself a hacker. But
when I'm talking to journalists I just say programmer or something like
that.
-- Linus Torvalds
--
To unsubscribe from this list: send the line unsubscribe devicetree in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
--
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH v4 2/8] OF: Introduce DT overlay support.
On Mon, 26 May 2014 12:57:32 +0200, Geert Uytterhoeven ge...@linux-m68k.org wrote: Hi Grant, On Mon, May 26, 2014 at 12:48 PM, Grant Likely grant.lik...@secretlab.ca wrote: On Tue, 20 May 2014 09:38:49 +0200, Geert Uytterhoeven ge...@linux-m68k.org wrote: On Tue, May 20, 2014 at 7:50 AM, Grant Likely grant.lik...@secretlab.ca wrote: Why has the overlay system been designed for plugging and unpluging whole overlays? That means the kernel has to remember the full stack, causing issues with e.g. kexec. Mostly so that drivers don't see any difference in the livetree data structure. It also means that userspace sees a single representation of the hardware at any given time. Sorry, I don't follow the argument about the single representation of the hardware. Er, s/of the hardware/of the tree/. Right now the overlay design modifies the live tree which at the same time modifies the tree representation in /sys/firmware/devicetree. If the design was changed to keep the overlay logically separate, then I would think we want to expose that information to usespace also. In fact, I think we would need to for usecases like kexec. OK, so it does modify the real tree, and doesn't keep the actual overlays. I was under the impression the overlay stack was also kept in memory, to allow reversal, so there was a misunderstanding. Hence for kexec, the tree in /sys/firmware/devicetree can just be passed to the new kernel, as that's the current representation of the hardware? Heeheehee. We're back where we started. The original question is whether or not that is a valid approach. If the overlay represents something that can be hot plugged/unplugged, then passing it through to the second kernel would be the wrong thing to do. If it was a permenant addition, then it probably doesn't need to be removed. We do actually keep the overlay info in memory for the purpose of removal exactly so we can support hot unbinding of devices and drivers that make use of overlays. g. -- To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH v4 2/8] OF: Introduce DT overlay support.
Hi Grant, On May 26, 2014, at 2:23 PM, Grant Likely wrote: On Mon, 26 May 2014 12:57:32 +0200, Geert Uytterhoeven ge...@linux-m68k.org wrote: Hi Grant, On Mon, May 26, 2014 at 12:48 PM, Grant Likely grant.lik...@secretlab.ca wrote: On Tue, 20 May 2014 09:38:49 +0200, Geert Uytterhoeven ge...@linux-m68k.org wrote: On Tue, May 20, 2014 at 7:50 AM, Grant Likely grant.lik...@secretlab.ca wrote: Why has the overlay system been designed for plugging and unpluging whole overlays? That means the kernel has to remember the full stack, causing issues with e.g. kexec. Mostly so that drivers don't see any difference in the livetree data structure. It also means that userspace sees a single representation of the hardware at any given time. Sorry, I don't follow the argument about the single representation of the hardware. Er, s/of the hardware/of the tree/. Right now the overlay design modifies the live tree which at the same time modifies the tree representation in /sys/firmware/devicetree. If the design was changed to keep the overlay logically separate, then I would think we want to expose that information to usespace also. In fact, I think we would need to for usecases like kexec. OK, so it does modify the real tree, and doesn't keep the actual overlays. I was under the impression the overlay stack was also kept in memory, to allow reversal, so there was a misunderstanding. Hence for kexec, the tree in /sys/firmware/devicetree can just be passed to the new kernel, as that's the current representation of the hardware? Heeheehee. We're back where we started. The original question is whether or not that is a valid approach. If the overlay represents something that can be hot plugged/unplugged, then passing it through to the second kernel would be the wrong thing to do. If it was a permenant addition, then it probably doesn't need to be removed. We do actually keep the overlay info in memory for the purpose of removal exactly so we can support hot unbinding of devices and drivers that make use of overlays. We can support either method. I am not feeling any wiser about which one should be the default TBH, so what about exporting a property and let the platform figure out which is more appropriate? g. Regards -- Pantelis -- To unsubscribe from this list: send the line unsubscribe devicetree in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html -- To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH v4 2/8] OF: Introduce DT overlay support.
Hi, On Mon, May 26, 2014 at 02:55:37PM +0300, Pantelis Antoniou wrote: On May 26, 2014, at 2:23 PM, Grant Likely wrote: On Mon, 26 May 2014 12:57:32 +0200, Geert Uytterhoeven ge...@linux-m68k.org wrote: Heeheehee. We're back where we started. The original question is whether or not that is a valid approach. If the overlay represents something that can be hot plugged/unplugged, then passing it through to the second kernel would be the wrong thing to do. If it was a permenant addition, then it probably doesn't need to be removed. We do actually keep the overlay info in memory for the purpose of removal exactly so we can support hot unbinding of devices and drivers that make use of overlays. We can support either method. I am not feeling any wiser about which one should be the default TBH, so what about exporting a property and let the platform figure out which is more appropriate? What about supporting negative overlays (so an overlay, that removes DT entries)? That way one could reverse apply an overlay. All the dependency stuff would basically be the users problem. The kernel only checks if it can apply an overlay (and return some error code if it can't). This this code is needed anyway to check the input from userspace. As a result the overlay handling would basically have the same behaviour as diff and patch :) P.S.: Sorry if this has already been suggested. I have only read mails from PATCHv4. -- Sebastian signature.asc Description: Digital signature
Re: [PATCH v4 2/8] OF: Introduce DT overlay support.
On 05/26/2014 08:09 AM, Sebastian Reichel wrote: Hi, On Mon, May 26, 2014 at 02:55:37PM +0300, Pantelis Antoniou wrote: On May 26, 2014, at 2:23 PM, Grant Likely wrote: On Mon, 26 May 2014 12:57:32 +0200, Geert Uytterhoeven ge...@linux-m68k.org wrote: Heeheehee. We're back where we started. The original question is whether or not that is a valid approach. If the overlay represents something that can be hot plugged/unplugged, then passing it through to the second kernel would be the wrong thing to do. If it was a permenant addition, then it probably doesn't need to be removed. We do actually keep the overlay info in memory for the purpose of removal exactly so we can support hot unbinding of devices and drivers that make use of overlays. We can support either method. I am not feeling any wiser about which one should be the default TBH, so what about exporting a property and let the platform figure out which is more appropriate? What about supporting negative overlays (so an overlay, that removes DT entries)? That way one could reverse apply an overlay. All the dependency stuff would basically be the users problem. The kernel only checks if it can apply an overlay (and return some error code if it can't). This this code is needed anyway to check the input from userspace. Does that mean that I would need to describe such a negative overlay for each overlay to be able to get it removed ? This would introduce an endless source of problems with bad reverse overlay descriptions. Sure, that would be the users problem, but I don't think that would make it better. Guenter -- To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH v4 2/8] OF: Introduce DT overlay support.
Hi Sebastian,
On May 26, 2014, at 6:09 PM, Sebastian Reichel wrote:
Hi,
On Mon, May 26, 2014 at 02:55:37PM +0300, Pantelis Antoniou wrote:
On May 26, 2014, at 2:23 PM, Grant Likely wrote:
On Mon, 26 May 2014 12:57:32 +0200, Geert Uytterhoeven
ge...@linux-m68k.org wrote:
Heeheehee. We're back where we started. The original question is whether
or not that is a valid approach. If the overlay represents something
that can be hot plugged/unplugged, then passing it through to the second
kernel would be the wrong thing to do. If it was a permenant addition,
then it probably doesn't need to be removed.
We do actually keep the overlay info in memory for the purpose of
removal exactly so we can support hot unbinding of devices and drivers
that make use of overlays.
We can support either method. I am not feeling any wiser about which one
should be
the default TBH, so what about exporting a property and let the platform
figure out which is more appropriate?
What about supporting negative overlays (so an overlay, that
removes DT entries)? That way one could reverse apply an overlay.
All the dependency stuff would basically be the users problem. The
kernel only checks if it can apply an overlay (and return some error
code if it can't). This this code is needed anyway to check the
input from userspace.
As a result the overlay handling would basically have the same
behaviour as diff and patch :)
This is already supported; nodes and properties can be prefixed with a minus
sign '-' and operate as expected :)
i.e. -property; removes a property and -node { }; removes a node.
P.S.: Sorry if this has already been suggested. I have only read
mails from PATCHv4.
-- Sebastian
Regards
-- Pantelis
--
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH v4 2/8] OF: Introduce DT overlay support.
On Mon, 26 May 2014 14:55:37 +0300, Pantelis Antoniou pantelis.anton...@konsulko.com wrote: Hi Grant, On May 26, 2014, at 2:23 PM, Grant Likely wrote: On Mon, 26 May 2014 12:57:32 +0200, Geert Uytterhoeven ge...@linux-m68k.org wrote: Hi Grant, On Mon, May 26, 2014 at 12:48 PM, Grant Likely grant.lik...@secretlab.ca wrote: On Tue, 20 May 2014 09:38:49 +0200, Geert Uytterhoeven ge...@linux-m68k.org wrote: On Tue, May 20, 2014 at 7:50 AM, Grant Likely grant.lik...@secretlab.ca wrote: Why has the overlay system been designed for plugging and unpluging whole overlays? That means the kernel has to remember the full stack, causing issues with e.g. kexec. Mostly so that drivers don't see any difference in the livetree data structure. It also means that userspace sees a single representation of the hardware at any given time. Sorry, I don't follow the argument about the single representation of the hardware. Er, s/of the hardware/of the tree/. Right now the overlay design modifies the live tree which at the same time modifies the tree representation in /sys/firmware/devicetree. If the design was changed to keep the overlay logically separate, then I would think we want to expose that information to usespace also. In fact, I think we would need to for usecases like kexec. OK, so it does modify the real tree, and doesn't keep the actual overlays. I was under the impression the overlay stack was also kept in memory, to allow reversal, so there was a misunderstanding. Hence for kexec, the tree in /sys/firmware/devicetree can just be passed to the new kernel, as that's the current representation of the hardware? Heeheehee. We're back where we started. The original question is whether or not that is a valid approach. If the overlay represents something that can be hot plugged/unplugged, then passing it through to the second kernel would be the wrong thing to do. If it was a permenant addition, then it probably doesn't need to be removed. We do actually keep the overlay info in memory for the purpose of removal exactly so we can support hot unbinding of devices and drivers that make use of overlays. We can support either method. I am not feeling any wiser about which one should be the default TBH, so what about exporting a property and let the platform figure out which is more appropriate? After thinking about it more, I think it is very likely that removing all the overlays is the correct thing to do in the kexec use-case. When kexec-ing, it makes sense that we'd want the exact same behaviour from the kexec'ed kernel. That means we want the device drivers to do the same thing including loading whatever overlays they depend on. If the flattened tree was left applied, then the behaviour becomes different. I say always remove the overlays unless explicitly told not to, but I'm struggling to come up with use cases where keeping them applied is desirable. g. -- To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH v4 2/8] OF: Introduce DT overlay support.
Hi Grant,
On Mon, May 26, 2014 at 11:33 PM, Grant Likely
grant.lik...@secretlab.ca wrote:
After thinking about it more, I think it is very likely that removing
all the overlays is the correct thing to do in the kexec use-case. When
kexec-ing, it makes sense that we'd want the exact same behaviour from
the kexec'ed kernel. That means we want the device drivers to do the
same thing including loading whatever overlays they depend on.
Are the device drivers loading the overlays?
That sounds a bit backwards to me.
Gr{oetje,eeting}s,
Geert
--
Geert Uytterhoeven -- There's lots of Linux beyond ia32 -- ge...@linux-m68k.org
In personal conversations with technical people, I call myself a hacker. But
when I'm talking to journalists I just say programmer or something like that.
-- Linus Torvalds
--
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH v4 2/8] OF: Introduce DT overlay support.
Hi, On Mon, May 26, 2014 at 10:33:03PM +0100, Grant Likely wrote: After thinking about it more, I think it is very likely that removing all the overlays is the correct thing to do in the kexec use-case. When kexec-ing, it makes sense that we'd want the exact same behaviour from the kexec'ed kernel. That means we want the device drivers to do the same thing including loading whatever overlays they depend on. If the flattened tree was left applied, then the behaviour becomes different. I say always remove the overlays unless explicitly told not to, but I'm struggling to come up with use cases where keeping them applied is desirable. I would assume, that I want them applied in most cases. DT describes the hardware. If I kexec into a new kernel I change software, not hardware. Maybe I'm missing the main purpose of the feature. I currently see two useful usecases for DT overlays: 1. The dtb the kernel is booted with cannot be changed for some reason, but the board has additional hardware attached (e.g. the user added a sensor on the i2c bus) 2. The hardware is changed on the fly (e.g. the user flashed the FPGA part of a zynq processor), sensors on i2c bus, ... In both cases the kernel should be booted with the additional overlay information IMHO. Though for the second case it should be possible to remove the programmed hardware information somehow. -- Sebastian signature.asc Description: Digital signature
Re: [PATCH v4 2/8] OF: Introduce DT overlay support.
Hi, On Mon, May 26, 2014 at 08:14:08AM -0700, Guenter Roeck wrote: On 05/26/2014 08:09 AM, Sebastian Reichel wrote: On Mon, May 26, 2014 at 02:55:37PM +0300, Pantelis Antoniou wrote: On May 26, 2014, at 2:23 PM, Grant Likely wrote: On Mon, 26 May 2014 12:57:32 +0200, Geert Uytterhoeven ge...@linux-m68k.org wrote: Heeheehee. We're back where we started. The original question is whether or not that is a valid approach. If the overlay represents something that can be hot plugged/unplugged, then passing it through to the second kernel would be the wrong thing to do. If it was a permenant addition, then it probably doesn't need to be removed. We do actually keep the overlay info in memory for the purpose of removal exactly so we can support hot unbinding of devices and drivers that make use of overlays. We can support either method. I am not feeling any wiser about which one should be the default TBH, so what about exporting a property and let the platform figure out which is more appropriate? What about supporting negative overlays (so an overlay, that removes DT entries)? That way one could reverse apply an overlay. All the dependency stuff would basically be the users problem. The kernel only checks if it can apply an overlay (and return some error code if it can't). This this code is needed anyway to check the input from userspace. Does that mean that I would need to describe such a negative overlay for each overlay to be able to get it removed ? This would introduce an endless source of problems with bad reverse overlay descriptions. Sure, that would be the users problem, but I don't think that would make it better. I was thinking about supporting something like patch --reverse. So you can try to undo the overlay by reverse applying it and you can do it in arbitrary order. Note: The dependency check must be done for all overlays coming from userspace, so that's not a problem __here__. The reverse method can simply reverse the overlay patch and apply it like a normal overlay from userspace (and thus using the same dependency checks). -- Sebastian signature.asc Description: Digital signature
Re: [PATCH v4 2/8] OF: Introduce DT overlay support.
On 05/26/2014 03:36 PM, Sebastian Reichel wrote: Hi, On Mon, May 26, 2014 at 10:33:03PM +0100, Grant Likely wrote: After thinking about it more, I think it is very likely that removing all the overlays is the correct thing to do in the kexec use-case. When kexec-ing, it makes sense that we'd want the exact same behaviour from the kexec'ed kernel. That means we want the device drivers to do the same thing including loading whatever overlays they depend on. If the flattened tree was left applied, then the behaviour becomes different. I say always remove the overlays unless explicitly told not to, but I'm struggling to come up with use cases where keeping them applied is desirable. I would assume, that I want them applied in most cases. DT describes the hardware. If I kexec into a new kernel I change software, not hardware. Maybe I'm missing the main purpose of the feature. I currently see two useful usecases for DT overlays: 1. The dtb the kernel is booted with cannot be changed for some reason, but the board has additional hardware attached (e.g. the user added a sensor on the i2c bus) 2. The hardware is changed on the fly (e.g. the user flashed the FPGA part of a zynq processor), sensors on i2c bus, ... In both cases the kernel should be booted with the additional overlay information IMHO. Though for the second case it should be possible to remove the programmed hardware information somehow. 3. Some hot-plug device or card is inserted or removed. I would argue that the kernel should _not_ be booted with the overlay in place. Otherwise the code handling overlays would have to have special handling for the restart case, which is much more complex than just to re-insert the overlay when it is determined that the device or card is still there. Guenter -- To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH v4 2/8] OF: Introduce DT overlay support.
On 05/26/2014 02:44 PM, Geert Uytterhoeven wrote: Hi Grant, On Mon, May 26, 2014 at 11:33 PM, Grant Likely grant.lik...@secretlab.ca wrote: After thinking about it more, I think it is very likely that removing all the overlays is the correct thing to do in the kexec use-case. When kexec-ing, it makes sense that we'd want the exact same behaviour from the kexec'ed kernel. That means we want the device drivers to do the same thing including loading whatever overlays they depend on. Are the device drivers loading the overlays? That sounds a bit backwards to me. Depends on what you mean with 'device driver'. In our case we have a driver dedicated to handle card status. Quite similar to what the connector subsystem does but with significantly more functionality (including devicetree overlay management). Guenter -- To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH v4 2/8] OF: Introduce DT overlay support.
On Mon, May 26, 2014 at 04:42:44PM -0700, Guenter Roeck wrote: On 05/26/2014 03:36 PM, Sebastian Reichel wrote: On Mon, May 26, 2014 at 10:33:03PM +0100, Grant Likely wrote: After thinking about it more, I think it is very likely that removing all the overlays is the correct thing to do in the kexec use-case. When kexec-ing, it makes sense that we'd want the exact same behaviour from the kexec'ed kernel. That means we want the device drivers to do the same thing including loading whatever overlays they depend on. If the flattened tree was left applied, then the behaviour becomes different. I say always remove the overlays unless explicitly told not to, but I'm struggling to come up with use cases where keeping them applied is desirable. I would assume, that I want them applied in most cases. DT describes the hardware. If I kexec into a new kernel I change software, not hardware. Maybe I'm missing the main purpose of the feature. I currently see two useful usecases for DT overlays: 1. The dtb the kernel is booted with cannot be changed for some reason, but the board has additional hardware attached (e.g. the user added a sensor on the i2c bus) 2. The hardware is changed on the fly (e.g. the user flashed the FPGA part of a zynq processor), sensors on i2c bus, ... In both cases the kernel should be booted with the additional overlay information IMHO. Though for the second case it should be possible to remove the programmed hardware information somehow. 3. Some hot-plug device or card is inserted or removed. Can you give a more specific example? I guess most hot-plug devices are connected to busses, which are not described via DT, but support auto-identification (USB, PCI, ...) I would argue that the kernel should _not_ be booted with the overlay in place. well the device is still attached to the system when you kexec into the new kernel, isn't it? Otherwise the code handling overlays would have to have special handling for the restart case, which is much more complex than just to re-insert the overlay when it is determined that the device or card is still there. I assume, that the kernel cannot auto-detect the attached hardware. Otherwise we don't need the DT entries, but can simply scan the bus. So the restart case (or restart + kexec case if kexec behaves like a restart) means, that userspace needs to provide the information about device existence. Removing the overlay is like dropping information supplied from the user. Not something, which should be done carelessly. -- Sebastian signature.asc Description: Digital signature
Re: [PATCH v4 2/8] OF: Introduce DT overlay support.
On 05/26/2014 05:32 PM, Sebastian Reichel wrote: On Mon, May 26, 2014 at 04:42:44PM -0700, Guenter Roeck wrote: On 05/26/2014 03:36 PM, Sebastian Reichel wrote: On Mon, May 26, 2014 at 10:33:03PM +0100, Grant Likely wrote: After thinking about it more, I think it is very likely that removing all the overlays is the correct thing to do in the kexec use-case. When kexec-ing, it makes sense that we'd want the exact same behaviour from the kexec'ed kernel. That means we want the device drivers to do the same thing including loading whatever overlays they depend on. If the flattened tree was left applied, then the behaviour becomes different. I say always remove the overlays unless explicitly told not to, but I'm struggling to come up with use cases where keeping them applied is desirable. I would assume, that I want them applied in most cases. DT describes the hardware. If I kexec into a new kernel I change software, not hardware. Maybe I'm missing the main purpose of the feature. I currently see two useful usecases for DT overlays: 1. The dtb the kernel is booted with cannot be changed for some reason, but the board has additional hardware attached (e.g. the user added a sensor on the i2c bus) 2. The hardware is changed on the fly (e.g. the user flashed the FPGA part of a zynq processor), sensors on i2c bus, ... In both cases the kernel should be booted with the additional overlay information IMHO. Though for the second case it should be possible to remove the programmed hardware information somehow. 3. Some hot-plug device or card is inserted or removed. Can you give a more specific example? I guess most hot-plug devices are connected to busses, which are not described via DT, but support auto-identification (USB, PCI, ...) The card interface provides i2c and pcie busses, plus a number of gpio pins. Both I2C devices and PCIe devices depend on the inserted card type. There may be PCIe switches on some cards, or just PCIe devices on others. Auto-identification of PCIe devices does not help, as the device configuration depends on the card type inserted. A typical example is that the card may include a multi-function FPGA device with LED, GPIO, and I2C bus master functionality, and the specific configuration depends on the card type (even though the FPGA is the same). Other cards may include a PCIe switch with a number of ASICs connected to it. PCIe switch configuration depends on the card type, not on the PCIe switch type. I would argue that the kernel should _not_ be booted with the overlay in place. well the device is still attached to the system when you kexec into the new kernel, isn't it? So ? the code executed is ultimately the same, if the kernel is booted from rommon or from kexec. In one case we'll have to insert the overlay, in the other case not. Otherwise the code handling overlays would have to have special handling for the restart case, which is much more complex than just to re-insert the overlay when it is determined that the device or card is still there. I assume, that the kernel cannot auto-detect the attached hardware. Correct. Otherwise we don't need the DT entries, but can simply scan the bus. So the restart case (or restart + kexec case if kexec behaves like a restart) means, that userspace needs to provide the information about device existence. Exactly the same informatin we get if the kernel is booted from ROMMON or BIOS. No difference here. Removing the overlay is like dropping information supplied from the user. Not something, which should be done carelessly. In our case it is done when the card is removed. Guenter -- To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH v4 2/8] OF: Introduce DT overlay support.
Hi Grant, On May 20, 2014, at 8:50 AM, Grant Likely wrote: > On Fri, 16 May 2014 13:52:42 +0200, Geert Uytterhoeven > wrote: >> Hi Grant, >> >> On Fri, May 16, 2014 at 12:58 PM, Grant Likely >> wrote: >>> On Thu, 15 May 2014 09:20:24 +0200, Geert Uytterhoeven >>> wrote: On Thu, May 15, 2014 at 9:12 AM, Pantelis Antoniou wrote: >> We also need to think about kexec. Kexec works by sucking the live tree >> out of the kernel and creating a .dtb from it to pass to the new kernel. >> What will the rules be when kexecing? Do all the overlays need to be >> removed, or does the kernel get the tree with all the overlays applied >> (in which case none of the overlays can be removed on the other side of >> kexec). > > We can add a sysfs attribute that configures whether overlays are > reverted before > kexec or not. I can't really tell which is the correct option, so let's > allow the > policy up to user-space. Kexec'ing into a new kernel doesn't change the hardware, so IMHO the in-kernel DT should not change. >>> >>> Conceptually though, if overlays are applied then the new kernel has a >>> different tree from the old one. If the overlay is hotpluggable, then >>> after the kexec, the new kernel would no longer be able to unplug the >>> overlay if it uses the current state of the tree instead of pulling the >>> overlays off first. >> >> Which brings me to another question... >> >> Why has the overlay system been designed for plugging and unpluging whole >> overlays? >> That means the kernel has to remember the full stack, causing issues with >> e.g. kexec. > > Mostly so that drivers don't see any difference in the livetree data > structure. It also means that userspace sees a single representation of > the hardware at any given time. > Yes. No change in semantics what-so-ever, when using the base DTS or an overlay. In a nutshell, you should be able to copy the overlay contents on the base DTS with equivalent results. >> Why not allowing the addition of removal of subtrees of the full device >> tree? > > Overlays is more than just a subtree. A single overlay can make > manipulations of multiple subtrees that should be handled as logically > atomic. > Right again. Drivers/subsystems are nowhere near ready to handle non-atomic changes in their properties. >> This is similar to other hotpluggable subsystems (which are not necessarily >> DT-based), like PCI Express. That way the kernel can pass a >> DT-representation of the actual current device tree to a kexec'ed kernel. > > I'm not following you argument. Hardware hotplug systems like PCIe don't > manipulate the firmware data. The kernel detects the new device and > populates the Linux device model directly. Firmware provided data (ACPI > or FDT) isn't involved. > Hotplugged busses in theory do not need DT. In practice... that will come later. >> >> I missed the initial design discussions, so forgive me if this has been >> beaten to death before. > > It's a good question. An alternative would be to keep the overlay tree > as a separate data structure and figure out how to make the core code > reference the overlay when iterating over nodes and properties. I don't > know how complex it would be to do that. We would definitely need to > adjust the data structure a bit, but that isn't an insurmountable > barrier. > The changes required would be mind-boggingly complex. At least with the overlay when you're done applying the changes you're done. Think about the cases where multiple overlays are present. You'll have to iterate over each and every one of them. > g. > Regards -- Pantelis -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH v4 2/8] OF: Introduce DT overlay support.
Hi Grant,
On Tue, May 20, 2014 at 7:50 AM, Grant Likely wrote:
>> Why has the overlay system been designed for plugging and unpluging whole
>> overlays?
>> That means the kernel has to remember the full stack, causing issues with
>> e.g. kexec.
>
> Mostly so that drivers don't see any difference in the livetree data
> structure. It also means that userspace sees a single representation of
> the hardware at any given time.
Sorry, I don't follow the argument about the "single representation of the
hardware".
>> Why not allowing the addition of removal of subtrees of the full device
>> tree?
>
> Overlays is more than just a subtree. A single overlay can make
> manipulations of multiple subtrees that should be handled as logically
> atomic.
Sure, it's more complicated due to the atomicity of multiple changes.
>> This is similar to other hotpluggable subsystems (which are not necessarily
>> DT-based), like PCI Express. That way the kernel can pass a
>> DT-representation of the actual current device tree to a kexec'ed kernel.
>
> I'm not following you argument. Hardware hotplug systems like PCIe don't
> manipulate the firmware data. The kernel detects the new device and
> populates the Linux device model directly. Firmware provided data (ACPI
> or FDT) isn't involved.
I mean the kernel doesn't remember the exact order in a stack, to reverse
operations. E.g. I can add hotplug a PCIe bridge with multiple devices
behind it, and unplug a single device later. It's still one subtree, though.
Gr{oetje,eeting}s,
Geert
--
Geert Uytterhoeven -- There's lots of Linux beyond ia32 -- ge...@linux-m68k.org
In personal conversations with technical people, I call myself a hacker. But
when I'm talking to journalists I just say "programmer" or something like that.
-- Linus Torvalds
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH v4 2/8] OF: Introduce DT overlay support.
Hi Grant,
On Tue, May 20, 2014 at 7:50 AM, Grant Likely grant.lik...@secretlab.ca wrote:
Why has the overlay system been designed for plugging and unpluging whole
overlays?
That means the kernel has to remember the full stack, causing issues with
e.g. kexec.
Mostly so that drivers don't see any difference in the livetree data
structure. It also means that userspace sees a single representation of
the hardware at any given time.
Sorry, I don't follow the argument about the single representation of the
hardware.
Why not allowing the addition of removal of subtrees of the full device
tree?
Overlays is more than just a subtree. A single overlay can make
manipulations of multiple subtrees that should be handled as logically
atomic.
Sure, it's more complicated due to the atomicity of multiple changes.
This is similar to other hotpluggable subsystems (which are not necessarily
DT-based), like PCI Express. That way the kernel can pass a
DT-representation of the actual current device tree to a kexec'ed kernel.
I'm not following you argument. Hardware hotplug systems like PCIe don't
manipulate the firmware data. The kernel detects the new device and
populates the Linux device model directly. Firmware provided data (ACPI
or FDT) isn't involved.
I mean the kernel doesn't remember the exact order in a stack, to reverse
operations. E.g. I can add hotplug a PCIe bridge with multiple devices
behind it, and unplug a single device later. It's still one subtree, though.
Gr{oetje,eeting}s,
Geert
--
Geert Uytterhoeven -- There's lots of Linux beyond ia32 -- ge...@linux-m68k.org
In personal conversations with technical people, I call myself a hacker. But
when I'm talking to journalists I just say programmer or something like that.
-- Linus Torvalds
--
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH v4 2/8] OF: Introduce DT overlay support.
Hi Grant, On May 20, 2014, at 8:50 AM, Grant Likely wrote: On Fri, 16 May 2014 13:52:42 +0200, Geert Uytterhoeven ge...@linux-m68k.org wrote: Hi Grant, On Fri, May 16, 2014 at 12:58 PM, Grant Likely grant.lik...@secretlab.ca wrote: On Thu, 15 May 2014 09:20:24 +0200, Geert Uytterhoeven ge...@linux-m68k.org wrote: On Thu, May 15, 2014 at 9:12 AM, Pantelis Antoniou pantelis.anton...@konsulko.com wrote: We also need to think about kexec. Kexec works by sucking the live tree out of the kernel and creating a .dtb from it to pass to the new kernel. What will the rules be when kexecing? Do all the overlays need to be removed, or does the kernel get the tree with all the overlays applied (in which case none of the overlays can be removed on the other side of kexec). We can add a sysfs attribute that configures whether overlays are reverted before kexec or not. I can't really tell which is the correct option, so let's allow the policy up to user-space. Kexec'ing into a new kernel doesn't change the hardware, so IMHO the in-kernel DT should not change. Conceptually though, if overlays are applied then the new kernel has a different tree from the old one. If the overlay is hotpluggable, then after the kexec, the new kernel would no longer be able to unplug the overlay if it uses the current state of the tree instead of pulling the overlays off first. Which brings me to another question... Why has the overlay system been designed for plugging and unpluging whole overlays? That means the kernel has to remember the full stack, causing issues with e.g. kexec. Mostly so that drivers don't see any difference in the livetree data structure. It also means that userspace sees a single representation of the hardware at any given time. Yes. No change in semantics what-so-ever, when using the base DTS or an overlay. In a nutshell, you should be able to copy the overlay contents on the base DTS with equivalent results. Why not allowing the addition of removal of subtrees of the full device tree? Overlays is more than just a subtree. A single overlay can make manipulations of multiple subtrees that should be handled as logically atomic. Right again. Drivers/subsystems are nowhere near ready to handle non-atomic changes in their properties. This is similar to other hotpluggable subsystems (which are not necessarily DT-based), like PCI Express. That way the kernel can pass a DT-representation of the actual current device tree to a kexec'ed kernel. I'm not following you argument. Hardware hotplug systems like PCIe don't manipulate the firmware data. The kernel detects the new device and populates the Linux device model directly. Firmware provided data (ACPI or FDT) isn't involved. Hotplugged busses in theory do not need DT. In practice... that will come later. I missed the initial design discussions, so forgive me if this has been beaten to death before. It's a good question. An alternative would be to keep the overlay tree as a separate data structure and figure out how to make the core code reference the overlay when iterating over nodes and properties. I don't know how complex it would be to do that. We would definitely need to adjust the data structure a bit, but that isn't an insurmountable barrier. The changes required would be mind-boggingly complex. At least with the overlay when you're done applying the changes you're done. Think about the cases where multiple overlays are present. You'll have to iterate over each and every one of them. g. Regards -- Pantelis -- To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH v4 2/8] OF: Introduce DT overlay support.
On Fri, 16 May 2014 13:52:42 +0200, Geert Uytterhoeven wrote: > Hi Grant, > > On Fri, May 16, 2014 at 12:58 PM, Grant Likely > wrote: > > On Thu, 15 May 2014 09:20:24 +0200, Geert Uytterhoeven > > wrote: > >> On Thu, May 15, 2014 at 9:12 AM, Pantelis Antoniou > >> wrote: > >> >> We also need to think about kexec. Kexec works by sucking the live tree > >> >> out of the kernel and creating a .dtb from it to pass to the new kernel. > >> >> What will the rules be when kexecing? Do all the overlays need to be > >> >> removed, or does the kernel get the tree with all the overlays applied > >> >> (in which case none of the overlays can be removed on the other side of > >> >> kexec). > >> > > >> > We can add a sysfs attribute that configures whether overlays are > >> > reverted before > >> > kexec or not. I can't really tell which is the correct option, so let's > >> > allow the > >> > policy up to user-space. > >> > >> Kexec'ing into a new kernel doesn't change the hardware, so IMHO the > >> in-kernel DT should not change. > > > > Conceptually though, if overlays are applied then the new kernel has a > > different tree from the old one. If the overlay is hotpluggable, then > > after the kexec, the new kernel would no longer be able to unplug the > > overlay if it uses the current state of the tree instead of pulling the > > overlays off first. > > Which brings me to another question... > > Why has the overlay system been designed for plugging and unpluging whole > overlays? > That means the kernel has to remember the full stack, causing issues with > e.g. kexec. Mostly so that drivers don't see any difference in the livetree data structure. It also means that userspace sees a single representation of the hardware at any given time. > Why not allowing the addition of removal of subtrees of the full device > tree? Overlays is more than just a subtree. A single overlay can make manipulations of multiple subtrees that should be handled as logically atomic. > This is similar to other hotpluggable subsystems (which are not necessarily > DT-based), like PCI Express. That way the kernel can pass a > DT-representation of the actual current device tree to a kexec'ed kernel. I'm not following you argument. Hardware hotplug systems like PCIe don't manipulate the firmware data. The kernel detects the new device and populates the Linux device model directly. Firmware provided data (ACPI or FDT) isn't involved. > > I missed the initial design discussions, so forgive me if this has been > beaten to death before. It's a good question. An alternative would be to keep the overlay tree as a separate data structure and figure out how to make the core code reference the overlay when iterating over nodes and properties. I don't know how complex it would be to do that. We would definitely need to adjust the data structure a bit, but that isn't an insurmountable barrier. g. -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH v4 2/8] OF: Introduce DT overlay support.
On Fri, 16 May 2014 13:52:42 +0200, Geert Uytterhoeven ge...@linux-m68k.org wrote: Hi Grant, On Fri, May 16, 2014 at 12:58 PM, Grant Likely grant.lik...@secretlab.ca wrote: On Thu, 15 May 2014 09:20:24 +0200, Geert Uytterhoeven ge...@linux-m68k.org wrote: On Thu, May 15, 2014 at 9:12 AM, Pantelis Antoniou pantelis.anton...@konsulko.com wrote: We also need to think about kexec. Kexec works by sucking the live tree out of the kernel and creating a .dtb from it to pass to the new kernel. What will the rules be when kexecing? Do all the overlays need to be removed, or does the kernel get the tree with all the overlays applied (in which case none of the overlays can be removed on the other side of kexec). We can add a sysfs attribute that configures whether overlays are reverted before kexec or not. I can't really tell which is the correct option, so let's allow the policy up to user-space. Kexec'ing into a new kernel doesn't change the hardware, so IMHO the in-kernel DT should not change. Conceptually though, if overlays are applied then the new kernel has a different tree from the old one. If the overlay is hotpluggable, then after the kexec, the new kernel would no longer be able to unplug the overlay if it uses the current state of the tree instead of pulling the overlays off first. Which brings me to another question... Why has the overlay system been designed for plugging and unpluging whole overlays? That means the kernel has to remember the full stack, causing issues with e.g. kexec. Mostly so that drivers don't see any difference in the livetree data structure. It also means that userspace sees a single representation of the hardware at any given time. Why not allowing the addition of removal of subtrees of the full device tree? Overlays is more than just a subtree. A single overlay can make manipulations of multiple subtrees that should be handled as logically atomic. This is similar to other hotpluggable subsystems (which are not necessarily DT-based), like PCI Express. That way the kernel can pass a DT-representation of the actual current device tree to a kexec'ed kernel. I'm not following you argument. Hardware hotplug systems like PCIe don't manipulate the firmware data. The kernel detects the new device and populates the Linux device model directly. Firmware provided data (ACPI or FDT) isn't involved. I missed the initial design discussions, so forgive me if this has been beaten to death before. It's a good question. An alternative would be to keep the overlay tree as a separate data structure and figure out how to make the core code reference the overlay when iterating over nodes and properties. I don't know how complex it would be to do that. We would definitely need to adjust the data structure a bit, but that isn't an insurmountable barrier. g. -- To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH v4 2/8] OF: Introduce DT overlay support.
Hi Grant,
On Fri, May 16, 2014 at 12:58 PM, Grant Likely
wrote:
> On Thu, 15 May 2014 09:20:24 +0200, Geert Uytterhoeven
> wrote:
>> On Thu, May 15, 2014 at 9:12 AM, Pantelis Antoniou
>> wrote:
>> >> We also need to think about kexec. Kexec works by sucking the live tree
>> >> out of the kernel and creating a .dtb from it to pass to the new kernel.
>> >> What will the rules be when kexecing? Do all the overlays need to be
>> >> removed, or does the kernel get the tree with all the overlays applied
>> >> (in which case none of the overlays can be removed on the other side of
>> >> kexec).
>> >
>> > We can add a sysfs attribute that configures whether overlays are reverted
>> > before
>> > kexec or not. I can't really tell which is the correct option, so let's
>> > allow the
>> > policy up to user-space.
>>
>> Kexec'ing into a new kernel doesn't change the hardware, so IMHO the
>> in-kernel DT should not change.
>
> Conceptually though, if overlays are applied then the new kernel has a
> different tree from the old one. If the overlay is hotpluggable, then
> after the kexec, the new kernel would no longer be able to unplug the
> overlay if it uses the current state of the tree instead of pulling the
> overlays off first.
Which brings me to another question...
Why has the overlay system been designed for plugging and unpluging whole
overlays?
That means the kernel has to remember the full stack, causing issues with
e.g. kexec.
Why not allowing the addition of removal of subtrees of the full device
tree?
This is similar to other hotpluggable subsystems (which are not necessarily
DT-based), like PCI Express. That way the kernel can pass a
DT-representation of the actual current device tree to a kexec'ed kernel.
I missed the initial design discussions, so forgive me if this has been
beaten to death before.
Thanks for your answers!
Gr{oetje,eeting}s,
Geert
--
Geert Uytterhoeven -- There's lots of Linux beyond ia32 -- ge...@linux-m68k.org
In personal conversations with technical people, I call myself a hacker. But
when I'm talking to journalists I just say "programmer" or something like that.
-- Linus Torvalds
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH v4 2/8] OF: Introduce DT overlay support.
On Thu, 15 May 2014 09:20:24 +0200, Geert Uytterhoeven wrote: > Hi Pantelis, > > On Thu, May 15, 2014 at 9:12 AM, Pantelis Antoniou > wrote: > >> We also need to think about kexec. Kexec works by sucking the live tree > >> out of the kernel and creating a .dtb from it to pass to the new kernel. > >> What will the rules be when kexecing? Do all the overlays need to be > >> removed, or does the kernel get the tree with all the overlays applied > >> (in which case none of the overlays can be removed on the other side of > >> kexec). > > > > We can add a sysfs attribute that configures whether overlays are reverted > > before > > kexec or not. I can't really tell which is the correct option, so let's > > allow the > > policy up to user-space. > > Kexec'ing into a new kernel doesn't change the hardware, so IMHO the > in-kernel DT should not change. Conceptually though, if overlays are applied then the new kernel has a different tree from the old one. If the overlay is hotpluggable, then after the kexec, the new kernel would no longer be able to unplug the overlay if it uses the current state of the tree instead of pulling the overlays off first. g. -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH v4 2/8] OF: Introduce DT overlay support.
On Thu, 15 May 2014 09:20:24 +0200, Geert Uytterhoeven ge...@linux-m68k.org wrote: Hi Pantelis, On Thu, May 15, 2014 at 9:12 AM, Pantelis Antoniou pantelis.anton...@konsulko.com wrote: We also need to think about kexec. Kexec works by sucking the live tree out of the kernel and creating a .dtb from it to pass to the new kernel. What will the rules be when kexecing? Do all the overlays need to be removed, or does the kernel get the tree with all the overlays applied (in which case none of the overlays can be removed on the other side of kexec). We can add a sysfs attribute that configures whether overlays are reverted before kexec or not. I can't really tell which is the correct option, so let's allow the policy up to user-space. Kexec'ing into a new kernel doesn't change the hardware, so IMHO the in-kernel DT should not change. Conceptually though, if overlays are applied then the new kernel has a different tree from the old one. If the overlay is hotpluggable, then after the kexec, the new kernel would no longer be able to unplug the overlay if it uses the current state of the tree instead of pulling the overlays off first. g. -- To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH v4 2/8] OF: Introduce DT overlay support.
Hi Grant,
On Fri, May 16, 2014 at 12:58 PM, Grant Likely
grant.lik...@secretlab.ca wrote:
On Thu, 15 May 2014 09:20:24 +0200, Geert Uytterhoeven ge...@linux-m68k.org
wrote:
On Thu, May 15, 2014 at 9:12 AM, Pantelis Antoniou
pantelis.anton...@konsulko.com wrote:
We also need to think about kexec. Kexec works by sucking the live tree
out of the kernel and creating a .dtb from it to pass to the new kernel.
What will the rules be when kexecing? Do all the overlays need to be
removed, or does the kernel get the tree with all the overlays applied
(in which case none of the overlays can be removed on the other side of
kexec).
We can add a sysfs attribute that configures whether overlays are reverted
before
kexec or not. I can't really tell which is the correct option, so let's
allow the
policy up to user-space.
Kexec'ing into a new kernel doesn't change the hardware, so IMHO the
in-kernel DT should not change.
Conceptually though, if overlays are applied then the new kernel has a
different tree from the old one. If the overlay is hotpluggable, then
after the kexec, the new kernel would no longer be able to unplug the
overlay if it uses the current state of the tree instead of pulling the
overlays off first.
Which brings me to another question...
Why has the overlay system been designed for plugging and unpluging whole
overlays?
That means the kernel has to remember the full stack, causing issues with
e.g. kexec.
Why not allowing the addition of removal of subtrees of the full device
tree?
This is similar to other hotpluggable subsystems (which are not necessarily
DT-based), like PCI Express. That way the kernel can pass a
DT-representation of the actual current device tree to a kexec'ed kernel.
I missed the initial design discussions, so forgive me if this has been
beaten to death before.
Thanks for your answers!
Gr{oetje,eeting}s,
Geert
--
Geert Uytterhoeven -- There's lots of Linux beyond ia32 -- ge...@linux-m68k.org
In personal conversations with technical people, I call myself a hacker. But
when I'm talking to journalists I just say programmer or something like that.
-- Linus Torvalds
--
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH v4 2/8] OF: Introduce DT overlay support.
Hi Pantelis, Thanks for writing this up. A few responses below... On Thu, 15 May 2014 00:12:17 -0700, Pantelis Antoniou wrote: > On May 14, 2014, at 3:08 AM, Grant Likely wrote: > > On Fri, 4 Apr 2014 15:43:55 +0300, Pantelis Antoniou > > wrote: > > The notification infrastructure bothers me. It duplicates the > > notification that the core DT code already performs. I do understand > > that the current notifications don't do what you need them to because > > you need it all deferred until the complete set of batch changes are > > applied. However, instead of creating a new infrastructure, the existing > > notifier should be reworked to be batch-change aware. > > > > If I understood correctly, you're asking of rolling in this per-bus notifier > mechanism in the standard DT notifier infrastructure already in place. > I can't be absolutely sure about the details right now, but seems possible. > > I don't know if the kernel notifier framework will be unmodified, but I hope > so. It should be. It will need to be the dt code that buffers up the notification events to be played out after the batch of changes has been applied. That shouldn't have any impact on core notifier framework. [...] > > Is it the base DT that needs the __symbols__ node, or the overlay tree? > > I had thought it was the overlay tree that contained the __symbols__ > > node. Regardless, this is the first mention in this file of __symbols__. > > It would be good to discuss briefly how it works. > > > > The __symbols__ usage is explained in the resolve patch. > Since target-path has been added the base DT no longer needs a __symbols__ > node. Can the target-phandle method be removed entirely then? > >> diff --git a/drivers/of/Kconfig b/drivers/of/Kconfig > >> index 4d39c88..cfb7ff8 100644 > >> --- a/drivers/of/Kconfig > >> +++ b/drivers/of/Kconfig > >> @@ -86,4 +86,14 @@ config OF_RESOLVE > >> Enable OF dynamic resolution support. This allows you to > >> load Device Tree object fragments are run time. > >> > >> +config OF_OVERLAY > >> + bool "OF overlay support" > >> + depends on OF > >> + select OF_DYNAMIC > >> + select OF_DEVICE > >> + select OF_RESOLVE > >> + help > >> +OpenFirmware overlay support. Allows you to modify on runtime the > >> +live tree using overlays. > > > > Should not be a user-visable option. Drivers using it should select it > > or otherwise cause it to be enabled. > > Hmm. I don't know; if I let it up to drivers, platform devices will select > it, in turn > making it always selected for 99.9% of the platforms out there. > > Some people might not want to incur the code size penalty. The only code ever selecting this function would be code that actually calls the overlay functions. g. -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH v4 2/8] OF: Introduce DT overlay support.
Hi Pantelis,
On Thu, May 15, 2014 at 9:12 AM, Pantelis Antoniou
wrote:
>> We also need to think about kexec. Kexec works by sucking the live tree
>> out of the kernel and creating a .dtb from it to pass to the new kernel.
>> What will the rules be when kexecing? Do all the overlays need to be
>> removed, or does the kernel get the tree with all the overlays applied
>> (in which case none of the overlays can be removed on the other side of
>> kexec).
>
> We can add a sysfs attribute that configures whether overlays are reverted
> before
> kexec or not. I can't really tell which is the correct option, so let's allow
> the
> policy up to user-space.
Kexec'ing into a new kernel doesn't change the hardware, so IMHO the
in-kernel DT should not change.
Gr{oetje,eeting}s,
Geert
--
Geert Uytterhoeven -- There's lots of Linux beyond ia32 -- ge...@linux-m68k.org
In personal conversations with technical people, I call myself a hacker. But
when I'm talking to journalists I just say "programmer" or something like that.
-- Linus Torvalds
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH v4 2/8] OF: Introduce DT overlay support.
Hi Guenter, On May 14, 2014, at 6:18 AM, Guenter Roeck wrote: > On 05/14/2014 06:03 AM, Geert Uytterhoeven wrote: >> On Wed, May 14, 2014 at 12:08 PM, Grant Likely >> wrote: +config OF_OVERLAY + bool "OF overlay support" + depends on OF + select OF_DYNAMIC + select OF_DEVICE + select OF_RESOLVE + help + OpenFirmware overlay support. Allows you to modify on runtime the + live tree using overlays. >>> >>> Should not be a user-visable option. Drivers using it should select it >> >> Why not? It's up to the (final) user to use it, or not. >> >>> or otherwise cause it to be enabled. >> >> Why should this be driver-specific? Shouldn't this just work with all >> drivers? >> > > It does once enabled. > > I think what Grant refers to is that there has to be a driver which implements > the actual overlay insertion and removal (ie calls the new API functions), > and that the Kconfig option for this driver should select OF_OVERLAY > instead of depending on it. > That makes sense. > Guenter > Regards -- Pantelis -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH v4 2/8] OF: Introduce DT overlay support.
Hi Michael, On May 14, 2014, at 5:11 AM, Michael Stickel wrote: > Hi Grant, > > Am 14.05.2014 12:08, schrieb Grant Likely: >> More generally I am concerned about whether or not overlays >> will introduce corner cases that can never be handled correctly, >> particularly in how multiple overlays will get handled. I want to see >> very clear rules on what happens when multiple overlays are applied, and >> then removed again. Is it possible to remove overlays out of order? If >> so, what are the conditions that would not be allowed? > > Yes, it is possible that an overlay depends on another. > > The problem is not, that an overlay is removed other overlays depend on, > but that nodes of an overlay may depend on the to-be-removed overlay and > the resulting devicetree can become inconsistent. > > > I have an SPI Bus with two slaves. The second slave is used only on one > of our boards. That is why we split the overlays the following way: > > _spi1.dts: > Pinmux for SPI-Bus and activation of spi-controller. > Pinmux for CS0 and definition of first slave. > > _spi1_cs1: > Pinmux for CS1 and definition of second slave. > > When the overlay for the bus is removed, the overlays for the second > slave does not make any sense anymore. > > It is even worse in a scenario we have with a test board. > One of the slaves is an spi-io-controller with a few bitbanging i2c > masters. In an extreme case, each component is defined in a separate > overlay and only the overlay with the master is removed. I know, that > this is completely sick. The devices are removed cleanly because of the > device dependency. > Well, shouldn't you be reverting the overlays in reverse sequence? As I see it, when proper subtree tracking is implemented this use case (of removing #0 before #1) will not be allowed. What is your ideal scenario for this use case? > Michael > Regards -- Pantelis -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH v4 2/8] OF: Introduce DT overlay support.
Hi Grant, On May 14, 2014, at 3:08 AM, Grant Likely wrote: > On Fri, 4 Apr 2014 15:43:55 +0300, Pantelis Antoniou > wrote: >> Introduce DT overlay support. >> Using this functionality it is possible to dynamically overlay a part of >> the kernel's tree with another tree that's been dynamically loaded. >> It is also possible to remove node and properties. >> >> The creation/destruction of the devices is handled by calling in to >> bus specific handlers which can deal with the peculiarities of each >> device. >> >> Signed-off-by: Pantelis Antoniou > > Hi Panto, > > I spent a bunch of time yesterday going over this code and I have some > comments and questions. First off I must apologize. I think I completely > misunderstood the approach you are taking for putting the overlay data > into the tree. I had assumed that you were unflattening the overlay tree > and then rearranging the new device_nodes to insert them into the > existing tree, but it looks like you're leaving the overlay tree alone > and merely using it's nodes as templates when allocating new nodes that > will get inserted into the tree. Sorry for being confused on this. A few > of my opinions have probably changed now that I understand what you're > trying to do. > No worries. It is a complicated subject after all. Let me reply with what we discussed on IRC, so it's recorded for posterity. > As I've said before, this is a complicated bit of code that touches into > the core of device tree behaviour, so I am being cautious. This patch > and the previous patch add a lot of functionality that is hard to review > all at once, and it conflates some concepts that I would like to be very > well defined. For example, this patch adds at least 3 distinct features: > > - Revertable batch modifications to the live tree. > - New notification infrastructure for informing other code when changes > take place > - Parsing a tree in the overlay format to create an of_overlay_info > array. > > The revertable batch feature should be a patch all on its own without > any of the change tracking or notification aside from what the core code > is already doing. That change could be applied independently even if I > still have issues with the notification or parsing code. > > I would even split up the first feature into two steps: batch > modifications to the tree, and then adding the logging feature so a > batch modification can be reverted. > OK. > The notification infrastructure bothers me. It duplicates the > notification that the core DT code already performs. I do understand > that the current notifications don't do what you need them to because > you need it all deferred until the complete set of batch changes are > applied. However, instead of creating a new infrastructure, the existing > notifier should be reworked to be batch-change aware. > If I understood correctly, you're asking of rolling in this per-bus notifier mechanism in the standard DT notifier infrastructure already in place. I can't be absolutely sure about the details right now, but seems possible. I don't know if the kernel notifier framework will be unmodified, but I hope so. > The parsing function is a specific user of the other two features. It > *may* be appropiate to be in a separate module, but it certainly should > be in a separate patch. > OK > More generally I am concerned about whether or not overlays > will introduce corner cases that can never be handled correctly, > particularly in how multiple overlays will get handled. I want to see > very clear rules on what happens when multiple overlays are applied, and > then removed again. Is it possible to remove overlays out of order? If > so, what are the conditions that would not be allowed? > As mentioned by Guenter, a single stack of overlays is no good; we need to support an arbitrary stacking method. One way to implement it would be to add an overlay use counter which we will increase whenever a node is 'touched' by an overlay. Removal is only going to be permitted if the counters of a subtree are all less or equal to 1. The equal to 0 is to account for non-overlay node attachements. > I'm also concerned about security. Turning on overlay support opens up > basically full access to the hardware of the system. It definitely needs > to be accessible only by root, but I think it goes farther than that. > When doing changes, there should be a mechanism to restrict which nodes > are allowed to be changed. > It's hard for me to think of all the security implications. For starters this is obviously for root only (or in-kernel users). We can implement some kind of attribute like 'mutable' that we will use to adorn a subtree that permits overlay application. > We also need to think about kexec. Kexec works by sucking the live tree > out of the kernel and creating a .dtb from it to pass to the new kernel. > What will the rules be when kexecing? Do all the overlays need to be > removed, or does the kernel get
Re: [PATCH v4 2/8] OF: Introduce DT overlay support.
Hi Grant, On May 14, 2014, at 3:08 AM, Grant Likely wrote: On Fri, 4 Apr 2014 15:43:55 +0300, Pantelis Antoniou pantelis.anton...@konsulko.com wrote: Introduce DT overlay support. Using this functionality it is possible to dynamically overlay a part of the kernel's tree with another tree that's been dynamically loaded. It is also possible to remove node and properties. The creation/destruction of the devices is handled by calling in to bus specific handlers which can deal with the peculiarities of each device. Signed-off-by: Pantelis Antoniou pantelis.anton...@konsulko.com Hi Panto, I spent a bunch of time yesterday going over this code and I have some comments and questions. First off I must apologize. I think I completely misunderstood the approach you are taking for putting the overlay data into the tree. I had assumed that you were unflattening the overlay tree and then rearranging the new device_nodes to insert them into the existing tree, but it looks like you're leaving the overlay tree alone and merely using it's nodes as templates when allocating new nodes that will get inserted into the tree. Sorry for being confused on this. A few of my opinions have probably changed now that I understand what you're trying to do. No worries. It is a complicated subject after all. Let me reply with what we discussed on IRC, so it's recorded for posterity. As I've said before, this is a complicated bit of code that touches into the core of device tree behaviour, so I am being cautious. This patch and the previous patch add a lot of functionality that is hard to review all at once, and it conflates some concepts that I would like to be very well defined. For example, this patch adds at least 3 distinct features: - Revertable batch modifications to the live tree. - New notification infrastructure for informing other code when changes take place - Parsing a tree in the overlay format to create an of_overlay_info array. The revertable batch feature should be a patch all on its own without any of the change tracking or notification aside from what the core code is already doing. That change could be applied independently even if I still have issues with the notification or parsing code. I would even split up the first feature into two steps: batch modifications to the tree, and then adding the logging feature so a batch modification can be reverted. OK. The notification infrastructure bothers me. It duplicates the notification that the core DT code already performs. I do understand that the current notifications don't do what you need them to because you need it all deferred until the complete set of batch changes are applied. However, instead of creating a new infrastructure, the existing notifier should be reworked to be batch-change aware. If I understood correctly, you're asking of rolling in this per-bus notifier mechanism in the standard DT notifier infrastructure already in place. I can't be absolutely sure about the details right now, but seems possible. I don't know if the kernel notifier framework will be unmodified, but I hope so. The parsing function is a specific user of the other two features. It *may* be appropiate to be in a separate module, but it certainly should be in a separate patch. OK More generally I am concerned about whether or not overlays will introduce corner cases that can never be handled correctly, particularly in how multiple overlays will get handled. I want to see very clear rules on what happens when multiple overlays are applied, and then removed again. Is it possible to remove overlays out of order? If so, what are the conditions that would not be allowed? As mentioned by Guenter, a single stack of overlays is no good; we need to support an arbitrary stacking method. One way to implement it would be to add an overlay use counter which we will increase whenever a node is 'touched' by an overlay. Removal is only going to be permitted if the counters of a subtree are all less or equal to 1. The equal to 0 is to account for non-overlay node attachements. I'm also concerned about security. Turning on overlay support opens up basically full access to the hardware of the system. It definitely needs to be accessible only by root, but I think it goes farther than that. When doing changes, there should be a mechanism to restrict which nodes are allowed to be changed. It's hard for me to think of all the security implications. For starters this is obviously for root only (or in-kernel users). We can implement some kind of attribute like 'mutable' that we will use to adorn a subtree that permits overlay application. We also need to think about kexec. Kexec works by sucking the live tree out of the kernel and creating a .dtb from it to pass to the new kernel. What will the rules be when kexecing? Do all the overlays need to be removed, or does the kernel get the tree with all the
Re: [PATCH v4 2/8] OF: Introduce DT overlay support.
Hi Michael, On May 14, 2014, at 5:11 AM, Michael Stickel wrote: Hi Grant, Am 14.05.2014 12:08, schrieb Grant Likely: More generally I am concerned about whether or not overlays will introduce corner cases that can never be handled correctly, particularly in how multiple overlays will get handled. I want to see very clear rules on what happens when multiple overlays are applied, and then removed again. Is it possible to remove overlays out of order? If so, what are the conditions that would not be allowed? Yes, it is possible that an overlay depends on another. The problem is not, that an overlay is removed other overlays depend on, but that nodes of an overlay may depend on the to-be-removed overlay and the resulting devicetree can become inconsistent. I have an SPI Bus with two slaves. The second slave is used only on one of our boards. That is why we split the overlays the following way: _spi1.dts: Pinmux for SPI-Bus and activation of spi-controller. Pinmux for CS0 and definition of first slave. _spi1_cs1: Pinmux for CS1 and definition of second slave. When the overlay for the bus is removed, the overlays for the second slave does not make any sense anymore. It is even worse in a scenario we have with a test board. One of the slaves is an spi-io-controller with a few bitbanging i2c masters. In an extreme case, each component is defined in a separate overlay and only the overlay with the master is removed. I know, that this is completely sick. The devices are removed cleanly because of the device dependency. Well, shouldn't you be reverting the overlays in reverse sequence? As I see it, when proper subtree tracking is implemented this use case (of removing #0 before #1) will not be allowed. What is your ideal scenario for this use case? Michael Regards -- Pantelis -- To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH v4 2/8] OF: Introduce DT overlay support.
Hi Guenter, On May 14, 2014, at 6:18 AM, Guenter Roeck wrote: On 05/14/2014 06:03 AM, Geert Uytterhoeven wrote: On Wed, May 14, 2014 at 12:08 PM, Grant Likely grant.lik...@secretlab.ca wrote: +config OF_OVERLAY + bool OF overlay support + depends on OF + select OF_DYNAMIC + select OF_DEVICE + select OF_RESOLVE + help + OpenFirmware overlay support. Allows you to modify on runtime the + live tree using overlays. Should not be a user-visable option. Drivers using it should select it Why not? It's up to the (final) user to use it, or not. or otherwise cause it to be enabled. Why should this be driver-specific? Shouldn't this just work with all drivers? It does once enabled. I think what Grant refers to is that there has to be a driver which implements the actual overlay insertion and removal (ie calls the new API functions), and that the Kconfig option for this driver should select OF_OVERLAY instead of depending on it. That makes sense. Guenter Regards -- Pantelis -- To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH v4 2/8] OF: Introduce DT overlay support.
Hi Pantelis,
On Thu, May 15, 2014 at 9:12 AM, Pantelis Antoniou
pantelis.anton...@konsulko.com wrote:
We also need to think about kexec. Kexec works by sucking the live tree
out of the kernel and creating a .dtb from it to pass to the new kernel.
What will the rules be when kexecing? Do all the overlays need to be
removed, or does the kernel get the tree with all the overlays applied
(in which case none of the overlays can be removed on the other side of
kexec).
We can add a sysfs attribute that configures whether overlays are reverted
before
kexec or not. I can't really tell which is the correct option, so let's allow
the
policy up to user-space.
Kexec'ing into a new kernel doesn't change the hardware, so IMHO the
in-kernel DT should not change.
Gr{oetje,eeting}s,
Geert
--
Geert Uytterhoeven -- There's lots of Linux beyond ia32 -- ge...@linux-m68k.org
In personal conversations with technical people, I call myself a hacker. But
when I'm talking to journalists I just say programmer or something like that.
-- Linus Torvalds
--
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH v4 2/8] OF: Introduce DT overlay support.
Hi Pantelis, Thanks for writing this up. A few responses below... On Thu, 15 May 2014 00:12:17 -0700, Pantelis Antoniou pantelis.anton...@konsulko.com wrote: On May 14, 2014, at 3:08 AM, Grant Likely wrote: On Fri, 4 Apr 2014 15:43:55 +0300, Pantelis Antoniou pantelis.anton...@konsulko.com wrote: The notification infrastructure bothers me. It duplicates the notification that the core DT code already performs. I do understand that the current notifications don't do what you need them to because you need it all deferred until the complete set of batch changes are applied. However, instead of creating a new infrastructure, the existing notifier should be reworked to be batch-change aware. If I understood correctly, you're asking of rolling in this per-bus notifier mechanism in the standard DT notifier infrastructure already in place. I can't be absolutely sure about the details right now, but seems possible. I don't know if the kernel notifier framework will be unmodified, but I hope so. It should be. It will need to be the dt code that buffers up the notification events to be played out after the batch of changes has been applied. That shouldn't have any impact on core notifier framework. [...] Is it the base DT that needs the __symbols__ node, or the overlay tree? I had thought it was the overlay tree that contained the __symbols__ node. Regardless, this is the first mention in this file of __symbols__. It would be good to discuss briefly how it works. The __symbols__ usage is explained in the resolve patch. Since target-path has been added the base DT no longer needs a __symbols__ node. Can the target-phandle method be removed entirely then? diff --git a/drivers/of/Kconfig b/drivers/of/Kconfig index 4d39c88..cfb7ff8 100644 --- a/drivers/of/Kconfig +++ b/drivers/of/Kconfig @@ -86,4 +86,14 @@ config OF_RESOLVE Enable OF dynamic resolution support. This allows you to load Device Tree object fragments are run time. +config OF_OVERLAY + bool OF overlay support + depends on OF + select OF_DYNAMIC + select OF_DEVICE + select OF_RESOLVE + help +OpenFirmware overlay support. Allows you to modify on runtime the +live tree using overlays. Should not be a user-visable option. Drivers using it should select it or otherwise cause it to be enabled. Hmm. I don't know; if I let it up to drivers, platform devices will select it, in turn making it always selected for 99.9% of the platforms out there. Some people might not want to incur the code size penalty. The only code ever selecting this function would be code that actually calls the overlay functions. g. -- To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH v4 2/8] OF: Introduce DT overlay support.
On Wed, May 14, 2014 at 04:49:07PM +0100, Grant Likely wrote: > On Wed, 14 May 2014 14:11:52 +0200, Michael Stickel wrote: > > Hi Grant, > > > > Am 14.05.2014 12:08, schrieb Grant Likely: > > > More generally I am concerned about whether or not overlays > > > will introduce corner cases that can never be handled correctly, > > > particularly in how multiple overlays will get handled. I want to see > > > very clear rules on what happens when multiple overlays are applied, and > > > then removed again. Is it possible to remove overlays out of order? If > > > so, what are the conditions that would not be allowed? > > > > Yes, it is possible that an overlay depends on another. > > > > The problem is not, that an overlay is removed other overlays depend on, > > but that nodes of an overlay may depend on the to-be-removed overlay and > > the resulting devicetree can become inconsistent. > > So what should the rule be then? It sounds to me that it should be a > hard and fast rule for overlays to always be removed in-order. If two > overlays are applied, and the first one needs to be removed again, then > that forces a removal of the second. The code needs to enforce it too. > > The question can be revisited if someone can find a way to validate > overlays do not conflict. > We'll need to find a way to determine if overlays are nested. Only nested overlays must be removed in order. Otherwise the entire concept falls apart. In our case, overlays are used for hot plugged cards. Obviously there can and will be more than one of those cards in the system, and they can and will be inserted and removed in any order. Maybe a nesting level counter in each property would do it ? Or a reference pointing to overlay specific objects / data ? Guenter -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH v4 2/8] OF: Introduce DT overlay support.
On Wed, 14 May 2014 14:11:52 +0200, Michael Stickel wrote: > Hi Grant, > > Am 14.05.2014 12:08, schrieb Grant Likely: > > More generally I am concerned about whether or not overlays > > will introduce corner cases that can never be handled correctly, > > particularly in how multiple overlays will get handled. I want to see > > very clear rules on what happens when multiple overlays are applied, and > > then removed again. Is it possible to remove overlays out of order? If > > so, what are the conditions that would not be allowed? > > Yes, it is possible that an overlay depends on another. > > The problem is not, that an overlay is removed other overlays depend on, > but that nodes of an overlay may depend on the to-be-removed overlay and > the resulting devicetree can become inconsistent. So what should the rule be then? It sounds to me that it should be a hard and fast rule for overlays to always be removed in-order. If two overlays are applied, and the first one needs to be removed again, then that forces a removal of the second. The code needs to enforce it too. The question can be revisited if someone can find a way to validate overlays do not conflict. g. -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH v4 2/8] OF: Introduce DT overlay support.
On Wed, 14 May 2014 15:03:35 +0200, Geert Uytterhoeven wrote: > On Wed, May 14, 2014 at 12:08 PM, Grant Likely > wrote: > >> +config OF_OVERLAY > >> + bool "OF overlay support" > >> + depends on OF > >> + select OF_DYNAMIC > >> + select OF_DEVICE > >> + select OF_RESOLVE > >> + help > >> + OpenFirmware overlay support. Allows you to modify on runtime the > >> + live tree using overlays. > > > > Should not be a user-visable option. Drivers using it should select it > > Why not? It's up to the (final) user to use it, or not. > > > or otherwise cause it to be enabled. > > Why should this be driver-specific? Shouldn't this just work with all drivers? This patch is just the infrastructure. The actual user-visible option is in another patch. g. -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH v4 2/8] OF: Introduce DT overlay support.
On 05/14/2014 06:03 AM, Geert Uytterhoeven wrote: On Wed, May 14, 2014 at 12:08 PM, Grant Likely wrote: +config OF_OVERLAY + bool "OF overlay support" + depends on OF + select OF_DYNAMIC + select OF_DEVICE + select OF_RESOLVE + help + OpenFirmware overlay support. Allows you to modify on runtime the + live tree using overlays. Should not be a user-visable option. Drivers using it should select it Why not? It's up to the (final) user to use it, or not. or otherwise cause it to be enabled. Why should this be driver-specific? Shouldn't this just work with all drivers? It does once enabled. I think what Grant refers to is that there has to be a driver which implements the actual overlay insertion and removal (ie calls the new API functions), and that the Kconfig option for this driver should select OF_OVERLAY instead of depending on it. Guenter -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH v4 2/8] OF: Introduce DT overlay support.
On Wed, May 14, 2014 at 12:08 PM, Grant Likely
wrote:
>> +config OF_OVERLAY
>> + bool "OF overlay support"
>> + depends on OF
>> + select OF_DYNAMIC
>> + select OF_DEVICE
>> + select OF_RESOLVE
>> + help
>> + OpenFirmware overlay support. Allows you to modify on runtime the
>> + live tree using overlays.
>
> Should not be a user-visable option. Drivers using it should select it
Why not? It's up to the (final) user to use it, or not.
> or otherwise cause it to be enabled.
Why should this be driver-specific? Shouldn't this just work with all drivers?
Gr{oetje,eeting}s,
Geert
--
Geert Uytterhoeven -- There's lots of Linux beyond ia32 -- ge...@linux-m68k.org
In personal conversations with technical people, I call myself a hacker. But
when I'm talking to journalists I just say "programmer" or something like that.
-- Linus Torvalds
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH v4 2/8] OF: Introduce DT overlay support.
Hi Grant, Am 14.05.2014 12:08, schrieb Grant Likely: > More generally I am concerned about whether or not overlays > will introduce corner cases that can never be handled correctly, > particularly in how multiple overlays will get handled. I want to see > very clear rules on what happens when multiple overlays are applied, and > then removed again. Is it possible to remove overlays out of order? If > so, what are the conditions that would not be allowed? Yes, it is possible that an overlay depends on another. The problem is not, that an overlay is removed other overlays depend on, but that nodes of an overlay may depend on the to-be-removed overlay and the resulting devicetree can become inconsistent. I have an SPI Bus with two slaves. The second slave is used only on one of our boards. That is why we split the overlays the following way: _spi1.dts: Pinmux for SPI-Bus and activation of spi-controller. Pinmux for CS0 and definition of first slave. _spi1_cs1: Pinmux for CS1 and definition of second slave. When the overlay for the bus is removed, the overlays for the second slave does not make any sense anymore. It is even worse in a scenario we have with a test board. One of the slaves is an spi-io-controller with a few bitbanging i2c masters. In an extreme case, each component is defined in a separate overlay and only the overlay with the master is removed. I know, that this is completely sick. The devices are removed cleanly because of the device dependency. Michael -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH v4 2/8] OF: Introduce DT overlay support.
On Fri, 4 Apr 2014 15:43:55 +0300, Pantelis Antoniou wrote: > Introduce DT overlay support. > Using this functionality it is possible to dynamically overlay a part of > the kernel's tree with another tree that's been dynamically loaded. > It is also possible to remove node and properties. > > The creation/destruction of the devices is handled by calling in to > bus specific handlers which can deal with the peculiarities of each > device. > > Signed-off-by: Pantelis Antoniou Hi Panto, I spent a bunch of time yesterday going over this code and I have some comments and questions. First off I must apologize. I think I completely misunderstood the approach you are taking for putting the overlay data into the tree. I had assumed that you were unflattening the overlay tree and then rearranging the new device_nodes to insert them into the existing tree, but it looks like you're leaving the overlay tree alone and merely using it's nodes as templates when allocating new nodes that will get inserted into the tree. Sorry for being confused on this. A few of my opinions have probably changed now that I understand what you're trying to do. As I've said before, this is a complicated bit of code that touches into the core of device tree behaviour, so I am being cautious. This patch and the previous patch add a lot of functionality that is hard to review all at once, and it conflates some concepts that I would like to be very well defined. For example, this patch adds at least 3 distinct features: - Revertable batch modifications to the live tree. - New notification infrastructure for informing other code when changes take place - Parsing a tree in the overlay format to create an of_overlay_info array. The revertable batch feature should be a patch all on its own without any of the change tracking or notification aside from what the core code is already doing. That change could be applied independently even if I still have issues with the notification or parsing code. I would even split up the first feature into two steps: batch modifications to the tree, and then adding the logging feature so a batch modification can be reverted. The notification infrastructure bothers me. It duplicates the notification that the core DT code already performs. I do understand that the current notifications don't do what you need them to because you need it all deferred until the complete set of batch changes are applied. However, instead of creating a new infrastructure, the existing notifier should be reworked to be batch-change aware. The parsing function is a specific user of the other two features. It *may* be appropiate to be in a separate module, but it certainly should be in a separate patch. More generally I am concerned about whether or not overlays will introduce corner cases that can never be handled correctly, particularly in how multiple overlays will get handled. I want to see very clear rules on what happens when multiple overlays are applied, and then removed again. Is it possible to remove overlays out of order? If so, what are the conditions that would not be allowed? I'm also concerned about security. Turning on overlay support opens up basically full access to the hardware of the system. It definitely needs to be accessible only by root, but I think it goes farther than that. When doing changes, there should be a mechanism to restrict which nodes are allowed to be changed. We also need to think about kexec. Kexec works by sucking the live tree out of the kernel and creating a .dtb from it to pass to the new kernel. What will the rules be when kexecing? Do all the overlays need to be removed, or does the kernel get the tree with all the overlays applied (in which case none of the overlays can be removed on the other side of kexec). More comments in-line... > --- > Documentation/devicetree/overlay-notes.txt | 187 ++ > drivers/of/Kconfig | 10 + > drivers/of/Makefile| 1 + > drivers/of/overlay.c | 895 > + > include/linux/of.h | 153 + > 5 files changed, 1246 insertions(+) > create mode 100644 Documentation/devicetree/overlay-notes.txt > create mode 100644 drivers/of/overlay.c > > diff --git a/Documentation/devicetree/overlay-notes.txt > b/Documentation/devicetree/overlay-notes.txt > new file mode 100644 > index 000..882d512 > --- /dev/null > +++ b/Documentation/devicetree/overlay-notes.txt > @@ -0,0 +1,187 @@ > +Device Tree Overlay Notes > +- > + > +This document describes the implementation of the in-kernel > +device tree overlay functionality residing in drivers/of/overlay.c and is a > +companion document to Documentation/devicetree/dt-object-internal.txt[1] & > +Documentation/devicetree/dynamic-resolution-notes.txt[2] > + > +How overlays work > +- > + > +A Device Tree's overlay purpose is to modify the kernel's
Re: [PATCH v4 2/8] OF: Introduce DT overlay support.
On Fri, 4 Apr 2014 15:43:55 +0300, Pantelis Antoniou pantelis.anton...@konsulko.com wrote: Introduce DT overlay support. Using this functionality it is possible to dynamically overlay a part of the kernel's tree with another tree that's been dynamically loaded. It is also possible to remove node and properties. The creation/destruction of the devices is handled by calling in to bus specific handlers which can deal with the peculiarities of each device. Signed-off-by: Pantelis Antoniou pantelis.anton...@konsulko.com Hi Panto, I spent a bunch of time yesterday going over this code and I have some comments and questions. First off I must apologize. I think I completely misunderstood the approach you are taking for putting the overlay data into the tree. I had assumed that you were unflattening the overlay tree and then rearranging the new device_nodes to insert them into the existing tree, but it looks like you're leaving the overlay tree alone and merely using it's nodes as templates when allocating new nodes that will get inserted into the tree. Sorry for being confused on this. A few of my opinions have probably changed now that I understand what you're trying to do. As I've said before, this is a complicated bit of code that touches into the core of device tree behaviour, so I am being cautious. This patch and the previous patch add a lot of functionality that is hard to review all at once, and it conflates some concepts that I would like to be very well defined. For example, this patch adds at least 3 distinct features: - Revertable batch modifications to the live tree. - New notification infrastructure for informing other code when changes take place - Parsing a tree in the overlay format to create an of_overlay_info array. The revertable batch feature should be a patch all on its own without any of the change tracking or notification aside from what the core code is already doing. That change could be applied independently even if I still have issues with the notification or parsing code. I would even split up the first feature into two steps: batch modifications to the tree, and then adding the logging feature so a batch modification can be reverted. The notification infrastructure bothers me. It duplicates the notification that the core DT code already performs. I do understand that the current notifications don't do what you need them to because you need it all deferred until the complete set of batch changes are applied. However, instead of creating a new infrastructure, the existing notifier should be reworked to be batch-change aware. The parsing function is a specific user of the other two features. It *may* be appropiate to be in a separate module, but it certainly should be in a separate patch. More generally I am concerned about whether or not overlays will introduce corner cases that can never be handled correctly, particularly in how multiple overlays will get handled. I want to see very clear rules on what happens when multiple overlays are applied, and then removed again. Is it possible to remove overlays out of order? If so, what are the conditions that would not be allowed? I'm also concerned about security. Turning on overlay support opens up basically full access to the hardware of the system. It definitely needs to be accessible only by root, but I think it goes farther than that. When doing changes, there should be a mechanism to restrict which nodes are allowed to be changed. We also need to think about kexec. Kexec works by sucking the live tree out of the kernel and creating a .dtb from it to pass to the new kernel. What will the rules be when kexecing? Do all the overlays need to be removed, or does the kernel get the tree with all the overlays applied (in which case none of the overlays can be removed on the other side of kexec). More comments in-line... --- Documentation/devicetree/overlay-notes.txt | 187 ++ drivers/of/Kconfig | 10 + drivers/of/Makefile| 1 + drivers/of/overlay.c | 895 + include/linux/of.h | 153 + 5 files changed, 1246 insertions(+) create mode 100644 Documentation/devicetree/overlay-notes.txt create mode 100644 drivers/of/overlay.c diff --git a/Documentation/devicetree/overlay-notes.txt b/Documentation/devicetree/overlay-notes.txt new file mode 100644 index 000..882d512 --- /dev/null +++ b/Documentation/devicetree/overlay-notes.txt @@ -0,0 +1,187 @@ +Device Tree Overlay Notes +- + +This document describes the implementation of the in-kernel +device tree overlay functionality residing in drivers/of/overlay.c and is a +companion document to Documentation/devicetree/dt-object-internal.txt[1] +Documentation/devicetree/dynamic-resolution-notes.txt[2] + +How overlays work +- + +A Device Tree's overlay purpose is to
Re: [PATCH v4 2/8] OF: Introduce DT overlay support.
Hi Grant, Am 14.05.2014 12:08, schrieb Grant Likely: More generally I am concerned about whether or not overlays will introduce corner cases that can never be handled correctly, particularly in how multiple overlays will get handled. I want to see very clear rules on what happens when multiple overlays are applied, and then removed again. Is it possible to remove overlays out of order? If so, what are the conditions that would not be allowed? Yes, it is possible that an overlay depends on another. The problem is not, that an overlay is removed other overlays depend on, but that nodes of an overlay may depend on the to-be-removed overlay and the resulting devicetree can become inconsistent. I have an SPI Bus with two slaves. The second slave is used only on one of our boards. That is why we split the overlays the following way: _spi1.dts: Pinmux for SPI-Bus and activation of spi-controller. Pinmux for CS0 and definition of first slave. _spi1_cs1: Pinmux for CS1 and definition of second slave. When the overlay for the bus is removed, the overlays for the second slave does not make any sense anymore. It is even worse in a scenario we have with a test board. One of the slaves is an spi-io-controller with a few bitbanging i2c masters. In an extreme case, each component is defined in a separate overlay and only the overlay with the master is removed. I know, that this is completely sick. The devices are removed cleanly because of the device dependency. Michael -- To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH v4 2/8] OF: Introduce DT overlay support.
On Wed, May 14, 2014 at 12:08 PM, Grant Likely
grant.lik...@secretlab.ca wrote:
+config OF_OVERLAY
+ bool OF overlay support
+ depends on OF
+ select OF_DYNAMIC
+ select OF_DEVICE
+ select OF_RESOLVE
+ help
+ OpenFirmware overlay support. Allows you to modify on runtime the
+ live tree using overlays.
Should not be a user-visable option. Drivers using it should select it
Why not? It's up to the (final) user to use it, or not.
or otherwise cause it to be enabled.
Why should this be driver-specific? Shouldn't this just work with all drivers?
Gr{oetje,eeting}s,
Geert
--
Geert Uytterhoeven -- There's lots of Linux beyond ia32 -- ge...@linux-m68k.org
In personal conversations with technical people, I call myself a hacker. But
when I'm talking to journalists I just say programmer or something like that.
-- Linus Torvalds
--
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH v4 2/8] OF: Introduce DT overlay support.
On 05/14/2014 06:03 AM, Geert Uytterhoeven wrote: On Wed, May 14, 2014 at 12:08 PM, Grant Likely grant.lik...@secretlab.ca wrote: +config OF_OVERLAY + bool OF overlay support + depends on OF + select OF_DYNAMIC + select OF_DEVICE + select OF_RESOLVE + help + OpenFirmware overlay support. Allows you to modify on runtime the + live tree using overlays. Should not be a user-visable option. Drivers using it should select it Why not? It's up to the (final) user to use it, or not. or otherwise cause it to be enabled. Why should this be driver-specific? Shouldn't this just work with all drivers? It does once enabled. I think what Grant refers to is that there has to be a driver which implements the actual overlay insertion and removal (ie calls the new API functions), and that the Kconfig option for this driver should select OF_OVERLAY instead of depending on it. Guenter -- To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH v4 2/8] OF: Introduce DT overlay support.
On Wed, 14 May 2014 15:03:35 +0200, Geert Uytterhoeven ge...@linux-m68k.org wrote: On Wed, May 14, 2014 at 12:08 PM, Grant Likely grant.lik...@secretlab.ca wrote: +config OF_OVERLAY + bool OF overlay support + depends on OF + select OF_DYNAMIC + select OF_DEVICE + select OF_RESOLVE + help + OpenFirmware overlay support. Allows you to modify on runtime the + live tree using overlays. Should not be a user-visable option. Drivers using it should select it Why not? It's up to the (final) user to use it, or not. or otherwise cause it to be enabled. Why should this be driver-specific? Shouldn't this just work with all drivers? This patch is just the infrastructure. The actual user-visible option is in another patch. g. -- To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH v4 2/8] OF: Introduce DT overlay support.
On Wed, 14 May 2014 14:11:52 +0200, Michael Stickel m...@mycable.de wrote: Hi Grant, Am 14.05.2014 12:08, schrieb Grant Likely: More generally I am concerned about whether or not overlays will introduce corner cases that can never be handled correctly, particularly in how multiple overlays will get handled. I want to see very clear rules on what happens when multiple overlays are applied, and then removed again. Is it possible to remove overlays out of order? If so, what are the conditions that would not be allowed? Yes, it is possible that an overlay depends on another. The problem is not, that an overlay is removed other overlays depend on, but that nodes of an overlay may depend on the to-be-removed overlay and the resulting devicetree can become inconsistent. So what should the rule be then? It sounds to me that it should be a hard and fast rule for overlays to always be removed in-order. If two overlays are applied, and the first one needs to be removed again, then that forces a removal of the second. The code needs to enforce it too. The question can be revisited if someone can find a way to validate overlays do not conflict. g. -- To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
[PATCH v4 2/8] OF: Introduce DT overlay support.
Introduce DT overlay support.
Using this functionality it is possible to dynamically overlay a part of
the kernel's tree with another tree that's been dynamically loaded.
It is also possible to remove node and properties.
The creation/destruction of the devices is handled by calling in to
bus specific handlers which can deal with the peculiarities of each
device.
Signed-off-by: Pantelis Antoniou
---
Documentation/devicetree/overlay-notes.txt | 187 ++
drivers/of/Kconfig | 10 +
drivers/of/Makefile| 1 +
drivers/of/overlay.c | 895 +
include/linux/of.h | 153 +
5 files changed, 1246 insertions(+)
create mode 100644 Documentation/devicetree/overlay-notes.txt
create mode 100644 drivers/of/overlay.c
diff --git a/Documentation/devicetree/overlay-notes.txt
b/Documentation/devicetree/overlay-notes.txt
new file mode 100644
index 000..882d512
--- /dev/null
+++ b/Documentation/devicetree/overlay-notes.txt
@@ -0,0 +1,187 @@
+Device Tree Overlay Notes
+-
+
+This document describes the implementation of the in-kernel
+device tree overlay functionality residing in drivers/of/overlay.c and is a
+companion document to Documentation/devicetree/dt-object-internal.txt[1] &
+Documentation/devicetree/dynamic-resolution-notes.txt[2]
+
+How overlays work
+-
+
+A Device Tree's overlay purpose is to modify the kernel's live tree, and
+have the modification affecting the state of the the kernel in a way that
+is reflecting the changes.
+Since the kernel mainly deals with devices, any new device node that result
+in an active device should have it created while if the device node is either
+disabled or removed all together, the affected device should be deregistered.
+
+Lets take an example where we have a foo board with the following base tree
+which is taken from [1].
+
+ foo.dts -
+ /* FOO platform */
+ / {
+ compatible = "corp,foo";
+
+ /* shared resources */
+ res: res {
+ };
+
+ /* On chip peripherals */
+ ocp: ocp {
+ /* peripherals that are always instantiated */
+ peripheral1 { ... };
+ }
+ };
+ foo.dts -
+
+The overlay bar.dts, when loaded (and resolved as described in [2]) should
+
+ bar.dts -
+/plugin/; /* allow undefined label references and record them */
+/ {
+ /* various properties for loader use; i.e. part id etc. */
+ fragment@0 {
+ target = <>;
+ __overlay__ {
+ /* bar peripheral */
+ bar {
+ compatible = "corp,bar";
+ ... /* various properties and child nodes */
+ }
+ };
+ };
+};
+ bar.dts -
+
+result in foo+bar.dts
+
+ foo+bar.dts -
+ /* FOO platform + bar peripheral */
+ / {
+ compatible = "corp,foo";
+
+ /* shared resources */
+ res: res {
+ };
+
+ /* On chip peripherals */
+ ocp: ocp {
+ /* peripherals that are always instantiated */
+ peripheral1 { ... };
+
+ /* bar peripheral */
+ bar {
+ compatible = "corp,bar";
+ ... /* various properties and child nodes */
+ }
+ }
+ };
+ foo+bar.dts -
+
+As a result of the the overlay, a new device node (bar) has been created
+so a bar platform device will be registered and if a matching device driver
+is loaded the device will be created as expected.
+
+Overlay in-kernel API
+-
+
+The steps typically required to get an overlay to work are as follows:
+
+1. Use of_build_overlay_info() to create an array of initialized and
+ready to use of_overlay_info structures.
+2. Call of_overlay() to apply the overlays declared in the array.
+3. If the overlay needs to be removed, call of_overlay_revert().
+4. Finally release the memory taken by the overlay info array by
+of_free_overlay_info().
+
+/**
+ * of_build_overlay_info - Build an overlay info array
+ * @tree: Device node containing all the overlays
+ * @cntp: Pointer to where the overlay info count will be help
+ * @ovinfop: Pointer to the pointer of an overlay info structure.
+ *
[PATCH v4 2/8] OF: Introduce DT overlay support.
Introduce DT overlay support.
Using this functionality it is possible to dynamically overlay a part of
the kernel's tree with another tree that's been dynamically loaded.
It is also possible to remove node and properties.
The creation/destruction of the devices is handled by calling in to
bus specific handlers which can deal with the peculiarities of each
device.
Signed-off-by: Pantelis Antoniou pantelis.anton...@konsulko.com
---
Documentation/devicetree/overlay-notes.txt | 187 ++
drivers/of/Kconfig | 10 +
drivers/of/Makefile| 1 +
drivers/of/overlay.c | 895 +
include/linux/of.h | 153 +
5 files changed, 1246 insertions(+)
create mode 100644 Documentation/devicetree/overlay-notes.txt
create mode 100644 drivers/of/overlay.c
diff --git a/Documentation/devicetree/overlay-notes.txt
b/Documentation/devicetree/overlay-notes.txt
new file mode 100644
index 000..882d512
--- /dev/null
+++ b/Documentation/devicetree/overlay-notes.txt
@@ -0,0 +1,187 @@
+Device Tree Overlay Notes
+-
+
+This document describes the implementation of the in-kernel
+device tree overlay functionality residing in drivers/of/overlay.c and is a
+companion document to Documentation/devicetree/dt-object-internal.txt[1]
+Documentation/devicetree/dynamic-resolution-notes.txt[2]
+
+How overlays work
+-
+
+A Device Tree's overlay purpose is to modify the kernel's live tree, and
+have the modification affecting the state of the the kernel in a way that
+is reflecting the changes.
+Since the kernel mainly deals with devices, any new device node that result
+in an active device should have it created while if the device node is either
+disabled or removed all together, the affected device should be deregistered.
+
+Lets take an example where we have a foo board with the following base tree
+which is taken from [1].
+
+ foo.dts -
+ /* FOO platform */
+ / {
+ compatible = corp,foo;
+
+ /* shared resources */
+ res: res {
+ };
+
+ /* On chip peripherals */
+ ocp: ocp {
+ /* peripherals that are always instantiated */
+ peripheral1 { ... };
+ }
+ };
+ foo.dts -
+
+The overlay bar.dts, when loaded (and resolved as described in [2]) should
+
+ bar.dts -
+/plugin/; /* allow undefined label references and record them */
+/ {
+ /* various properties for loader use; i.e. part id etc. */
+ fragment@0 {
+ target = ocp;
+ __overlay__ {
+ /* bar peripheral */
+ bar {
+ compatible = corp,bar;
+ ... /* various properties and child nodes */
+ }
+ };
+ };
+};
+ bar.dts -
+
+result in foo+bar.dts
+
+ foo+bar.dts -
+ /* FOO platform + bar peripheral */
+ / {
+ compatible = corp,foo;
+
+ /* shared resources */
+ res: res {
+ };
+
+ /* On chip peripherals */
+ ocp: ocp {
+ /* peripherals that are always instantiated */
+ peripheral1 { ... };
+
+ /* bar peripheral */
+ bar {
+ compatible = corp,bar;
+ ... /* various properties and child nodes */
+ }
+ }
+ };
+ foo+bar.dts -
+
+As a result of the the overlay, a new device node (bar) has been created
+so a bar platform device will be registered and if a matching device driver
+is loaded the device will be created as expected.
+
+Overlay in-kernel API
+-
+
+The steps typically required to get an overlay to work are as follows:
+
+1. Use of_build_overlay_info() to create an array of initialized and
+ready to use of_overlay_info structures.
+2. Call of_overlay() to apply the overlays declared in the array.
+3. If the overlay needs to be removed, call of_overlay_revert().
+4. Finally release the memory taken by the overlay info array by
+of_free_overlay_info().
+
+/**
+ * of_build_overlay_info - Build an overlay info array
+ * @tree: Device node containing all the overlays
+ * @cntp: Pointer to where the overlay info count will be help
+ * @ovinfop: Pointer to the pointer of an
