Re: [Ksummit-2013-discuss] DT bindings as ABI [was: Do we have people interested in device tree janitoring / cleanup?]
On 08/13/2013 06:49 PM, Guenter Roeck wrote: > >> That is one aspect (hardware standardization)... but it is more to it >> than that. > > I have to deal with lots of embedded / non-PC x86 based systems. Worst one > I encountered so far was a board where the VGA memory space was re-used > for an eeprom. The upcoming next generation hardware I'll have to support > is so far off-standard that I'll probably have to define a new platform > type (similar to OLPC or CE4100). > > No, it is not all PC. Not anymore. Intel has started to sell into > the embedded space, where PC compatibility is not a requirement. > We try to encourage vendors to do the right thing, where "the right thing" means, among other things, not to do gratuitously different things. With increasing amounts of the platform moving into the CPU and PCH this is of course also becoming more and more rare. Even on actual PCs we have seen some truly "special" facepalms. There was the Olivetti machine which used the GPIO for "fast A20" to control the screen backlight, for example. That one was fun to deal with. -hpa -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [Ksummit-2013-discuss] DT bindings as ABI [was: Do we have people interested in device tree janitoring / cleanup?]
On 08/13/2013 06:49 PM, Guenter Roeck wrote: That is one aspect (hardware standardization)... but it is more to it than that. I have to deal with lots of embedded / non-PC x86 based systems. Worst one I encountered so far was a board where the VGA memory space was re-used for an eeprom. The upcoming next generation hardware I'll have to support is so far off-standard that I'll probably have to define a new platform type (similar to OLPC or CE4100). No, it is not all PC. Not anymore. Intel has started to sell into the embedded space, where PC compatibility is not a requirement. We try to encourage vendors to do the right thing, where the right thing means, among other things, not to do gratuitously different things. With increasing amounts of the platform moving into the CPU and PCH this is of course also becoming more and more rare. Even on actual PCs we have seen some truly special facepalms. There was the Olivetti machine which used the GPIO for fast A20 to control the screen backlight, for example. That one was fun to deal with. -hpa -- To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [Ksummit-2013-discuss] DT bindings as ABI [was: Do we have people interested in device tree janitoring / cleanup?]
On 08/13/2013 04:32 PM, H. Peter Anvin wrote: On 08/01/2013 08:50 PM, David Gibson wrote: On Wed, Jul 31, 2013 at 05:26:47PM -0400, jonsm...@gmail.com wrote: On Wed, Jul 31, 2013 at 4:48 PM, Russell King - ARM Linux wrote: On Wed, Jul 31, 2013 at 04:37:36PM -0400, jonsm...@gmail.com wrote: [snip] Alternatively you may be of the belief that it is impossible to get rid of the board specific code. But x86 doesn't have any of it, why should ARM? Sure x86 has board specific code. It's just that x86 basically only has one board - PC. That is one aspect (hardware standardization)... but it is more to it than that. I have to deal with lots of embedded / non-PC x86 based systems. Worst one I encountered so far was a board where the VGA memory space was re-used for an eeprom. The upcoming next generation hardware I'll have to support is so far off-standard that I'll probably have to define a new platform type (similar to OLPC or CE4100). No, it is not all PC. Not anymore. Intel has started to sell into the embedded space, where PC compatibility is not a requirement. Guenter -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [Ksummit-2013-discuss] DT bindings as ABI [was: Do we have people interested in device tree janitoring / cleanup?]
On 08/01/2013 08:50 PM, David Gibson wrote: > On Wed, Jul 31, 2013 at 05:26:47PM -0400, jonsm...@gmail.com > wrote: >> On Wed, Jul 31, 2013 at 4:48 PM, Russell King - ARM Linux >> wrote: >>> On Wed, Jul 31, 2013 at 04:37:36PM -0400, jonsm...@gmail.com >> wrote: > [snip] >> Alternatively you may be of the belief that it is impossible to >> get rid of the board specific code. But x86 doesn't have any of >> it, why should ARM? > > Sure x86 has board specific code. It's just that x86 basically > only has one board - PC. > That is one aspect (hardware standardization)... but it is more to it than that. -hpa -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [Ksummit-2013-discuss] DT bindings as ABI [was: Do we have people interested in device tree janitoring / cleanup?]
On 08/01/2013 08:50 PM, David Gibson wrote: On Wed, Jul 31, 2013 at 05:26:47PM -0400, jonsm...@gmail.com wrote: On Wed, Jul 31, 2013 at 4:48 PM, Russell King - ARM Linux li...@arm.linux.org.uk wrote: On Wed, Jul 31, 2013 at 04:37:36PM -0400, jonsm...@gmail.com wrote: [snip] Alternatively you may be of the belief that it is impossible to get rid of the board specific code. But x86 doesn't have any of it, why should ARM? Sure x86 has board specific code. It's just that x86 basically only has one board - PC. That is one aspect (hardware standardization)... but it is more to it than that. -hpa -- To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [Ksummit-2013-discuss] DT bindings as ABI [was: Do we have people interested in device tree janitoring / cleanup?]
On 08/13/2013 04:32 PM, H. Peter Anvin wrote: On 08/01/2013 08:50 PM, David Gibson wrote: On Wed, Jul 31, 2013 at 05:26:47PM -0400, jonsm...@gmail.com wrote: On Wed, Jul 31, 2013 at 4:48 PM, Russell King - ARM Linux li...@arm.linux.org.uk wrote: On Wed, Jul 31, 2013 at 04:37:36PM -0400, jonsm...@gmail.com wrote: [snip] Alternatively you may be of the belief that it is impossible to get rid of the board specific code. But x86 doesn't have any of it, why should ARM? Sure x86 has board specific code. It's just that x86 basically only has one board - PC. That is one aspect (hardware standardization)... but it is more to it than that. I have to deal with lots of embedded / non-PC x86 based systems. Worst one I encountered so far was a board where the VGA memory space was re-used for an eeprom. The upcoming next generation hardware I'll have to support is so far off-standard that I'll probably have to define a new platform type (similar to OLPC or CE4100). No, it is not all PC. Not anymore. Intel has started to sell into the embedded space, where PC compatibility is not a requirement. Guenter -- To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [Ksummit-2013-discuss] DT bindings as ABI [was: Do we have people interested in device tree janitoring / cleanup?]
* Russell King - ARM Linux [130731 13:22]: > On Wed, Jul 31, 2013 at 10:00:17PM +0200, Richard Cochran wrote: > > On Wed, Jul 31, 2013 at 09:29:35PM +0200, Tomasz Figa wrote: > > > > > > I showed you two example solutions that could handle this use case > > > without > > > stable binding ABI, just to prove that b) is not the only option (even if > > > it's the best one, which I continue to agree on, don't get me wrong). > > > > You only showed *one* solution (b) that satisfies the use case. The > > use case was: > > > >User acquires a machine running ARM Linux version 3.x, with u-boot > >and dtb in a read only flash partition. The board boots and works just > > ^ > >fine. However, for his application, the user requires a new kernel > >feature that appeared in version 3.y where y > x. He compiles the new > >kernel, and it also works. > > > > But you suggested: > > > > a) using DT as direct replacement for board files - this means that you > > are free to say that DTSes are strictly coupled with kernel version > > and you are free to modify the bindings - see the analogy to board > > files, where you could modify the platform data structures and could > > not directly copy board file from one kernel version to another, > > > > In the use case, the kernel is upgraded, but not the DTB. So this > > solution makes no sense. > > That's also crap for another reason. DT on the whole is _supposed_ to > describe what the hardware is, and how it is wired up in a WELL DEFINED > and STABLE manner. Unfortunately, there's a *BIG* mistake that was made > - having this /chosen node in DT which gets used for "software" > configuration - eg, the command line and so forth. > > That was a mistake because it means that the DT isn't purely a > description of the hardware. Such stuff should have been left in ATAGs, > and DT should have been placed into its own ATAG entry. There would > not have been any conflict between ATAGs and DT, because ATAGs generally > don't deal with hardware configuration - the only real bit of hardware > configuration conveyed via ATAGs is the location of memory and size of > memory. This I completely agree with. And I'd go a bit further requiring the DT binding should describe the _types_ of registers the hardware has so the device driver does not have to contain data for each similar supported register instances for things like clocks and muxes and timers. In the worst case, platform_data is just being replaced by device tree and driver hacks in a confusing way that requires constant updating of both the .dts files and the device driver. Regards, Tony -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [Ksummit-2013-discuss] DT bindings as ABI [was: Do we have people interested in device tree janitoring / cleanup?]
* Russell King - ARM Linux li...@arm.linux.org.uk [130731 13:22]: On Wed, Jul 31, 2013 at 10:00:17PM +0200, Richard Cochran wrote: On Wed, Jul 31, 2013 at 09:29:35PM +0200, Tomasz Figa wrote: I showed you two example solutions that could handle this use case without stable binding ABI, just to prove that b) is not the only option (even if it's the best one, which I continue to agree on, don't get me wrong). You only showed *one* solution (b) that satisfies the use case. The use case was: User acquires a machine running ARM Linux version 3.x, with u-boot and dtb in a read only flash partition. The board boots and works just ^ fine. However, for his application, the user requires a new kernel feature that appeared in version 3.y where y x. He compiles the new kernel, and it also works. But you suggested: a) using DT as direct replacement for board files - this means that you are free to say that DTSes are strictly coupled with kernel version and you are free to modify the bindings - see the analogy to board files, where you could modify the platform data structures and could not directly copy board file from one kernel version to another, In the use case, the kernel is upgraded, but not the DTB. So this solution makes no sense. That's also crap for another reason. DT on the whole is _supposed_ to describe what the hardware is, and how it is wired up in a WELL DEFINED and STABLE manner. Unfortunately, there's a *BIG* mistake that was made - having this /chosen node in DT which gets used for software configuration - eg, the command line and so forth. That was a mistake because it means that the DT isn't purely a description of the hardware. Such stuff should have been left in ATAGs, and DT should have been placed into its own ATAG entry. There would not have been any conflict between ATAGs and DT, because ATAGs generally don't deal with hardware configuration - the only real bit of hardware configuration conveyed via ATAGs is the location of memory and size of memory. This I completely agree with. And I'd go a bit further requiring the DT binding should describe the _types_ of registers the hardware has so the device driver does not have to contain data for each similar supported register instances for things like clocks and muxes and timers. In the worst case, platform_data is just being replaced by device tree and driver hacks in a confusing way that requires constant updating of both the .dts files and the device driver. Regards, Tony -- To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [Ksummit-2013-discuss] DT bindings as ABI [was: Do we have people interested in device tree janitoring / cleanup?]
On Wed, Jul 31, 2013 at 05:26:47PM -0400, jonsm...@gmail.com wrote: > On Wed, Jul 31, 2013 at 4:48 PM, Russell King - ARM Linux > wrote: > > On Wed, Jul 31, 2013 at 04:37:36PM -0400, jonsm...@gmail.com > wrote: [snip] > Alternatively you may be of the belief that it is impossible to get > rid of the board specific code. But x86 doesn't have any of it, why > should ARM? Sure x86 has board specific code. It's just that x86 basically only has one board - PC. -- David Gibson| I'll have my music baroque, and my code david AT gibson.dropbear.id.au | minimalist, thank you. NOT _the_ _other_ | _way_ _around_! http://www.ozlabs.org/~dgibson pgpI6NfGqsPfk.pgp Description: PGP signature
Re: [Ksummit-2013-discuss] DT bindings as ABI [was: Do we have people interested in device tree janitoring / cleanup?]
On 08/01/2013 05:18 AM, David Woodhouse wrote: > On Wed, 2013-07-31 at 17:26 -0400, jonsm...@gmail.com wrote: >> Alternatively you may be of the belief that it is impossible to get >> rid of the board specific code. But x86 doesn't have any of it, why >> should ARM? > > The reason x86 doesn't have it is because it carries three decades worth > of legacy baggage so that it can still look like a 1980s IBM PC when > necessary. > > There *have* been some x86 platforms which abandon that legacy crap, and > for those we *do* have board-specific code. (Is James still maintaining > Voyager support? It feels very strange to talk about Voyager with it > *not* being the 'legacy crap' in question...) > > We've even seen *recent* attempts to abandon the legacy crap in the > embedded x86 space, which backtracked and added it all back again — in > part because x86 lacked any sane way to describe the hardware if it > wasn't pretending to be a PC. ACPI doesn't cut it, and DT "wasn't > invented here"... > > Unless you want the ARM world to settle on a strategy of "all the world > is an Assabet", I'd be careful what you wish for... There is some level of belief that ACPI will enable running this years OS on next years h/w. This idea is completely flawed as long as ARM vendors don't design for compatibility, spin the Si for compatibility issues, and have some mechanism to emulate legacy h/w. All the discussions and issues around DT bindings and processes will apply to ACPI bindings as well. Rob -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [Ksummit-2013-discuss] DT bindings as ABI [was: Do we have people interested in device tree janitoring / cleanup?]
On Thu, Aug 1, 2013 at 9:34 AM, jonsm...@gmail.com wrote: > On Thu, Aug 1, 2013 at 6:18 AM, David Woodhouse wrote: >> On Wed, 2013-07-31 at 17:26 -0400, jonsm...@gmail.com wrote: >>> Alternatively you may be of the belief that it is impossible to get >>> rid of the board specific code. But x86 doesn't have any of it, why >>> should ARM? >> >> The reason x86 doesn't have it is because it carries three decades worth >> of legacy baggage so that it can still look like a 1980s IBM PC when >> necessary. >> >> There *have* been some x86 platforms which abandon that legacy crap, and >> for those we *do* have board-specific code. (Is James still maintaining >> Voyager support? It feels very strange to talk about Voyager with it >> *not* being the 'legacy crap' in question...) >> >> We've even seen *recent* attempts to abandon the legacy crap in the >> embedded x86 space, which backtracked and added it all back again — in >> part because x86 lacked any sane way to describe the hardware if it >> wasn't pretending to be a PC. ACPI doesn't cut it, and DT "wasn't >> invented here"... >> >> Unless you want the ARM world to settle on a strategy of "all the world >> is an Assabet", I'd be careful what you wish for... > > So there should be shades for gray in this area. Try to eliminate all > of the board specific code you can, and then only if that fails add > board specific support to the kernel. > > But you take device trees pretty far. I believe Grant has even used > one to describe an FPGA that he can dynamically load code into and > change its function. Not sure how he did it, I wasn't paying too > close of attention when he was talking about it. > > I do believe a great deal of this simple plumbing can be eliminated. > Like how to hook up GPIO, I2C, SPI, etc. We're pretty far down that > road. > > A path like this seems like it would be beneficial... > 1) Implement DT schemas. Use those to enforce as much regularity as > possible into the device tree descriptions for common classes of > things (controllers especially - DMA, I2C, I2S, SPI, Uarts, etc).. > Form a group to review any changes to the common schema. > 2) Cleaning up the controller classes is going to cause some DT churn. > Hide backward compatibility in a quirks layer. > 3) Continue the process of removing all possible board specific code > that can be reasonably covered in device trees. > 4) There will be some board specific code left at the end of this > process. But anyone who looks at should agree that the functions > handled by the code are something that is unreasonable to address in > the DT system. 5) this scheme supports future improvements in the DT schema. Lets say initially we had punted power management to board specific code. Then in a later kernel version implemented it using device trees. The quirk system lets you delete the board specific code and replace it with a DT quirk. That DT quirk will see your deployed DTs at boot time and add in the new, fancy power management DT attributes. The new generic DT based power power management code will see these attributes added by the quirk and do the right thing. This gives us a way to slowly remove move board specific code if we choose to. > > >> >> -- >> dwmw2 >> > > > > -- > Jon Smirl > jonsm...@gmail.com -- Jon Smirl jonsm...@gmail.com -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [Ksummit-2013-discuss] DT bindings as ABI [was: Do we have people interested in device tree janitoring / cleanup?]
On Thu, Aug 1, 2013 at 6:18 AM, David Woodhouse wrote: > On Wed, 2013-07-31 at 17:26 -0400, jonsm...@gmail.com wrote: >> Alternatively you may be of the belief that it is impossible to get >> rid of the board specific code. But x86 doesn't have any of it, why >> should ARM? > > The reason x86 doesn't have it is because it carries three decades worth > of legacy baggage so that it can still look like a 1980s IBM PC when > necessary. > > There *have* been some x86 platforms which abandon that legacy crap, and > for those we *do* have board-specific code. (Is James still maintaining > Voyager support? It feels very strange to talk about Voyager with it > *not* being the 'legacy crap' in question...) > > We've even seen *recent* attempts to abandon the legacy crap in the > embedded x86 space, which backtracked and added it all back again — in > part because x86 lacked any sane way to describe the hardware if it > wasn't pretending to be a PC. ACPI doesn't cut it, and DT "wasn't > invented here"... > > Unless you want the ARM world to settle on a strategy of "all the world > is an Assabet", I'd be careful what you wish for... So there should be shades for gray in this area. Try to eliminate all of the board specific code you can, and then only if that fails add board specific support to the kernel. But you take device trees pretty far. I believe Grant has even used one to describe an FPGA that he can dynamically load code into and change its function. Not sure how he did it, I wasn't paying too close of attention when he was talking about it. I do believe a great deal of this simple plumbing can be eliminated. Like how to hook up GPIO, I2C, SPI, etc. We're pretty far down that road. A path like this seems like it would be beneficial... 1) Implement DT schemas. Use those to enforce as much regularity as possible into the device tree descriptions for common classes of things (controllers especially - DMA, I2C, I2S, SPI, Uarts, etc).. Form a group to review any changes to the common schema. 2) Cleaning up the controller classes is going to cause some DT churn. Hide backward compatibility in a quirks layer. 3) Continue the process of removing all possible board specific code that can be reasonably covered in device trees. 4) There will be some board specific code left at the end of this process. But anyone who looks at should agree that the functions handled by the code are something that is unreasonable to address in the DT system. > > -- > dwmw2 > -- Jon Smirl jonsm...@gmail.com -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [Ksummit-2013-discuss] DT bindings as ABI [was: Do we have people interested in device tree janitoring / cleanup?]
On Wed, 2013-07-31 at 17:26 -0400, jonsm...@gmail.com wrote: > Alternatively you may be of the belief that it is impossible to get > rid of the board specific code. But x86 doesn't have any of it, why > should ARM? The reason x86 doesn't have it is because it carries three decades worth of legacy baggage so that it can still look like a 1980s IBM PC when necessary. There *have* been some x86 platforms which abandon that legacy crap, and for those we *do* have board-specific code. (Is James still maintaining Voyager support? It feels very strange to talk about Voyager with it *not* being the 'legacy crap' in question...) We've even seen *recent* attempts to abandon the legacy crap in the embedded x86 space, which backtracked and added it all back again — in part because x86 lacked any sane way to describe the hardware if it wasn't pretending to be a PC. ACPI doesn't cut it, and DT "wasn't invented here"... Unless you want the ARM world to settle on a strategy of "all the world is an Assabet", I'd be careful what you wish for... -- dwmw2 smime.p7s Description: S/MIME cryptographic signature
Re: [Ksummit-2013-discuss] DT bindings as ABI [was: Do we have people interested in device tree janitoring / cleanup?]
On Thu, Aug 01, 2013 at 11:57:43AM +0200, Arend van Spriel wrote: > On 07/31/2013 11:26 PM, jonsm...@gmail.com wrote: > >Alternatively you may be of the belief that it is impossible to get > >rid of the board specific code. But x86 doesn't have any of it, why > >should ARM? > Well, I am curious whether that will stay that way once x86 is truly > moving into the embedded arena (if ever). There's shipping phones using x86, they do actually have board code due to their use of SFI. signature.asc Description: Digital signature
Re: [Ksummit-2013-discuss] DT bindings as ABI [was: Do we have people interested in device tree janitoring / cleanup?]
On 07/31/2013 11:26 PM, jonsm...@gmail.com wrote: Alternatively you may be of the belief that it is impossible to get rid of the board specific code. But x86 doesn't have any of it, why should ARM? Well, I am curious whether that will stay that way once x86 is truly moving into the embedded arena (if ever). Regards, Arend -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [Ksummit-2013-discuss] DT bindings as ABI [was: Do we have people interested in device tree janitoring / cleanup?]
On 07/31/2013 11:26 PM, jonsm...@gmail.com wrote: Alternatively you may be of the belief that it is impossible to get rid of the board specific code. But x86 doesn't have any of it, why should ARM? Well, I am curious whether that will stay that way once x86 is truly moving into the embedded arena (if ever). Regards, Arend -- To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [Ksummit-2013-discuss] DT bindings as ABI [was: Do we have people interested in device tree janitoring / cleanup?]
On Thu, Aug 01, 2013 at 11:57:43AM +0200, Arend van Spriel wrote: On 07/31/2013 11:26 PM, jonsm...@gmail.com wrote: Alternatively you may be of the belief that it is impossible to get rid of the board specific code. But x86 doesn't have any of it, why should ARM? Well, I am curious whether that will stay that way once x86 is truly moving into the embedded arena (if ever). There's shipping phones using x86, they do actually have board code due to their use of SFI. signature.asc Description: Digital signature
Re: [Ksummit-2013-discuss] DT bindings as ABI [was: Do we have people interested in device tree janitoring / cleanup?]
On Wed, 2013-07-31 at 17:26 -0400, jonsm...@gmail.com wrote: Alternatively you may be of the belief that it is impossible to get rid of the board specific code. But x86 doesn't have any of it, why should ARM? The reason x86 doesn't have it is because it carries three decades worth of legacy baggage so that it can still look like a 1980s IBM PC when necessary. There *have* been some x86 platforms which abandon that legacy crap, and for those we *do* have board-specific code. (Is James still maintaining Voyager support? It feels very strange to talk about Voyager with it *not* being the 'legacy crap' in question...) We've even seen *recent* attempts to abandon the legacy crap in the embedded x86 space, which backtracked and added it all back again — in part because x86 lacked any sane way to describe the hardware if it wasn't pretending to be a PC. ACPI doesn't cut it, and DT wasn't invented here... Unless you want the ARM world to settle on a strategy of all the world is an Assabet, I'd be careful what you wish for... -- dwmw2 smime.p7s Description: S/MIME cryptographic signature
Re: [Ksummit-2013-discuss] DT bindings as ABI [was: Do we have people interested in device tree janitoring / cleanup?]
On Thu, Aug 1, 2013 at 6:18 AM, David Woodhouse dw...@infradead.org wrote: On Wed, 2013-07-31 at 17:26 -0400, jonsm...@gmail.com wrote: Alternatively you may be of the belief that it is impossible to get rid of the board specific code. But x86 doesn't have any of it, why should ARM? The reason x86 doesn't have it is because it carries three decades worth of legacy baggage so that it can still look like a 1980s IBM PC when necessary. There *have* been some x86 platforms which abandon that legacy crap, and for those we *do* have board-specific code. (Is James still maintaining Voyager support? It feels very strange to talk about Voyager with it *not* being the 'legacy crap' in question...) We've even seen *recent* attempts to abandon the legacy crap in the embedded x86 space, which backtracked and added it all back again — in part because x86 lacked any sane way to describe the hardware if it wasn't pretending to be a PC. ACPI doesn't cut it, and DT wasn't invented here... Unless you want the ARM world to settle on a strategy of all the world is an Assabet, I'd be careful what you wish for... So there should be shades for gray in this area. Try to eliminate all of the board specific code you can, and then only if that fails add board specific support to the kernel. But you take device trees pretty far. I believe Grant has even used one to describe an FPGA that he can dynamically load code into and change its function. Not sure how he did it, I wasn't paying too close of attention when he was talking about it. I do believe a great deal of this simple plumbing can be eliminated. Like how to hook up GPIO, I2C, SPI, etc. We're pretty far down that road. A path like this seems like it would be beneficial... 1) Implement DT schemas. Use those to enforce as much regularity as possible into the device tree descriptions for common classes of things (controllers especially - DMA, I2C, I2S, SPI, Uarts, etc).. Form a group to review any changes to the common schema. 2) Cleaning up the controller classes is going to cause some DT churn. Hide backward compatibility in a quirks layer. 3) Continue the process of removing all possible board specific code that can be reasonably covered in device trees. 4) There will be some board specific code left at the end of this process. But anyone who looks at should agree that the functions handled by the code are something that is unreasonable to address in the DT system. -- dwmw2 -- Jon Smirl jonsm...@gmail.com -- To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [Ksummit-2013-discuss] DT bindings as ABI [was: Do we have people interested in device tree janitoring / cleanup?]
On Thu, Aug 1, 2013 at 9:34 AM, jonsm...@gmail.com jonsm...@gmail.com wrote: On Thu, Aug 1, 2013 at 6:18 AM, David Woodhouse dw...@infradead.org wrote: On Wed, 2013-07-31 at 17:26 -0400, jonsm...@gmail.com wrote: Alternatively you may be of the belief that it is impossible to get rid of the board specific code. But x86 doesn't have any of it, why should ARM? The reason x86 doesn't have it is because it carries three decades worth of legacy baggage so that it can still look like a 1980s IBM PC when necessary. There *have* been some x86 platforms which abandon that legacy crap, and for those we *do* have board-specific code. (Is James still maintaining Voyager support? It feels very strange to talk about Voyager with it *not* being the 'legacy crap' in question...) We've even seen *recent* attempts to abandon the legacy crap in the embedded x86 space, which backtracked and added it all back again — in part because x86 lacked any sane way to describe the hardware if it wasn't pretending to be a PC. ACPI doesn't cut it, and DT wasn't invented here... Unless you want the ARM world to settle on a strategy of all the world is an Assabet, I'd be careful what you wish for... So there should be shades for gray in this area. Try to eliminate all of the board specific code you can, and then only if that fails add board specific support to the kernel. But you take device trees pretty far. I believe Grant has even used one to describe an FPGA that he can dynamically load code into and change its function. Not sure how he did it, I wasn't paying too close of attention when he was talking about it. I do believe a great deal of this simple plumbing can be eliminated. Like how to hook up GPIO, I2C, SPI, etc. We're pretty far down that road. A path like this seems like it would be beneficial... 1) Implement DT schemas. Use those to enforce as much regularity as possible into the device tree descriptions for common classes of things (controllers especially - DMA, I2C, I2S, SPI, Uarts, etc).. Form a group to review any changes to the common schema. 2) Cleaning up the controller classes is going to cause some DT churn. Hide backward compatibility in a quirks layer. 3) Continue the process of removing all possible board specific code that can be reasonably covered in device trees. 4) There will be some board specific code left at the end of this process. But anyone who looks at should agree that the functions handled by the code are something that is unreasonable to address in the DT system. 5) this scheme supports future improvements in the DT schema. Lets say initially we had punted power management to board specific code. Then in a later kernel version implemented it using device trees. The quirk system lets you delete the board specific code and replace it with a DT quirk. That DT quirk will see your deployed DTs at boot time and add in the new, fancy power management DT attributes. The new generic DT based power power management code will see these attributes added by the quirk and do the right thing. This gives us a way to slowly remove move board specific code if we choose to. -- dwmw2 -- Jon Smirl jonsm...@gmail.com -- Jon Smirl jonsm...@gmail.com -- To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [Ksummit-2013-discuss] DT bindings as ABI [was: Do we have people interested in device tree janitoring / cleanup?]
On 08/01/2013 05:18 AM, David Woodhouse wrote: On Wed, 2013-07-31 at 17:26 -0400, jonsm...@gmail.com wrote: Alternatively you may be of the belief that it is impossible to get rid of the board specific code. But x86 doesn't have any of it, why should ARM? The reason x86 doesn't have it is because it carries three decades worth of legacy baggage so that it can still look like a 1980s IBM PC when necessary. There *have* been some x86 platforms which abandon that legacy crap, and for those we *do* have board-specific code. (Is James still maintaining Voyager support? It feels very strange to talk about Voyager with it *not* being the 'legacy crap' in question...) We've even seen *recent* attempts to abandon the legacy crap in the embedded x86 space, which backtracked and added it all back again — in part because x86 lacked any sane way to describe the hardware if it wasn't pretending to be a PC. ACPI doesn't cut it, and DT wasn't invented here... Unless you want the ARM world to settle on a strategy of all the world is an Assabet, I'd be careful what you wish for... There is some level of belief that ACPI will enable running this years OS on next years h/w. This idea is completely flawed as long as ARM vendors don't design for compatibility, spin the Si for compatibility issues, and have some mechanism to emulate legacy h/w. All the discussions and issues around DT bindings and processes will apply to ACPI bindings as well. Rob -- To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [Ksummit-2013-discuss] DT bindings as ABI [was: Do we have people interested in device tree janitoring / cleanup?]
On Wed, Jul 31, 2013 at 05:26:47PM -0400, jonsm...@gmail.com wrote: On Wed, Jul 31, 2013 at 4:48 PM, Russell King - ARM Linux li...@arm.linux.org.uk wrote: On Wed, Jul 31, 2013 at 04:37:36PM -0400, jonsm...@gmail.com wrote: [snip] Alternatively you may be of the belief that it is impossible to get rid of the board specific code. But x86 doesn't have any of it, why should ARM? Sure x86 has board specific code. It's just that x86 basically only has one board - PC. -- David Gibson| I'll have my music baroque, and my code david AT gibson.dropbear.id.au | minimalist, thank you. NOT _the_ _other_ | _way_ _around_! http://www.ozlabs.org/~dgibson pgpI6NfGqsPfk.pgp Description: PGP signature
Re: [Ksummit-2013-discuss] DT bindings as ABI [was: Do we have people interested in device tree janitoring / cleanup?]
On Wed, Jul 31, 2013 at 4:48 PM, Russell King - ARM Linux wrote: > On Wed, Jul 31, 2013 at 04:37:36PM -0400, jonsm...@gmail.com wrote: >> On Wed, Jul 31, 2013 at 4:14 PM, Russell King - ARM Linux >> wrote: >> > However, if we go back to the idea that DT is supposed to describe the >> > hardware, _and_ that the way to describe that hardware is well defined >> > and _stable_ (as it should be) then there should be no reason what so >> > ever that your old DT blob should not continue working in later kernels. >> > If it doesn't, then the contract that DT promised when we first moved >> > over to using DT has been _broken_. >> >> Suppose your DT predates PINCTRL and the CPU needs the pins set to >> function. But setting the pins up is a board specific function. How >> is this going to get implemented in a generic kernel that doesn't have >> any board specific code in it? >> >> I would propose that we only need to worry about DTs that got put into >> boot loaders and not worry about ones that were only used when >> appended to a kernel. > > That is - again - our mistake. We should have made it clear from the > start that the use of an _appended_ DT, or a DT provided with the kernel > being booted was more or less mandatory for the time being. We actually > did exactly the opposite - we advertised the appended DT as a stop-gap > measure just to allow a DT kernel to be booted with a boot loader which > didn't support DT. > > That in itself _encouraged_ boot loader support for DT on ARM (which in > itself is not a bad thing) but also leads people down the path of > potentially separating the DT from the kernel. > > Had we encouraged the use of an appended DT instead, but still encouraged > boot loaders to update, and also made a clear statement that DT is > unstable (which everyone can see - for example by making our DT stuff > depend on CONFIG_EXPERIMENTAL when it existed) then everyone would have > been clear about its status. > > The fact that we did not - the fact that we ignored this process means > that it is _our_ problem that people like Richard are blowing up such a > storm over this. We need to admit that we got it wrong, and commit to > doing it the right way in future. What that means is starting off today > with a commitment to keep as much of the stuff which works today working > _tomorrow_, and the day after, and so forth. What do you think about using a quirk layer to decouple deployed DTs from whatever is implemented in the kernel? I still think there is going to be volatility in the the kernel DT usage simply because we haven't figured out all of the issues with it yet. For example defining a global schema system is going to expose a lot of irregular DT syntax when you line up all of the platform DTs in a system where it is easy to compare them. One the plus side the kernel is certainly evolving to a point where this volatility will stop in the future, we just aren't there yet. If we don't use a quirk fixup system, then board specific code is going to continue to exist scattered around in the arch directories. My preference would be to contain the board specific DT fixup code inside a quirk system and have the goal of making the arch directories free of board specific code. Any new board would have a good enough DT that they don't need any board specific DT fixup code. Of course there's quite a ways to go before reaching that goal. Alternatively you may be of the belief that it is impossible to get rid of the board specific code. But x86 doesn't have any of it, why should ARM? -- Jon Smirl jonsm...@gmail.com -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [Ksummit-2013-discuss] DT bindings as ABI [was: Do we have people interested in device tree janitoring / cleanup?]
On Wed, Jul 31, 2013 at 04:37:36PM -0400, jonsm...@gmail.com wrote: > On Wed, Jul 31, 2013 at 4:14 PM, Russell King - ARM Linux > wrote: > > However, if we go back to the idea that DT is supposed to describe the > > hardware, _and_ that the way to describe that hardware is well defined > > and _stable_ (as it should be) then there should be no reason what so > > ever that your old DT blob should not continue working in later kernels. > > If it doesn't, then the contract that DT promised when we first moved > > over to using DT has been _broken_. > > Suppose your DT predates PINCTRL and the CPU needs the pins set to > function. But setting the pins up is a board specific function. How > is this going to get implemented in a generic kernel that doesn't have > any board specific code in it? > > I would propose that we only need to worry about DTs that got put into > boot loaders and not worry about ones that were only used when > appended to a kernel. That is - again - our mistake. We should have made it clear from the start that the use of an _appended_ DT, or a DT provided with the kernel being booted was more or less mandatory for the time being. We actually did exactly the opposite - we advertised the appended DT as a stop-gap measure just to allow a DT kernel to be booted with a boot loader which didn't support DT. That in itself _encouraged_ boot loader support for DT on ARM (which in itself is not a bad thing) but also leads people down the path of potentially separating the DT from the kernel. Had we encouraged the use of an appended DT instead, but still encouraged boot loaders to update, and also made a clear statement that DT is unstable (which everyone can see - for example by making our DT stuff depend on CONFIG_EXPERIMENTAL when it existed) then everyone would have been clear about its status. The fact that we did not - the fact that we ignored this process means that it is _our_ problem that people like Richard are blowing up such a storm over this. We need to admit that we got it wrong, and commit to doing it the right way in future. What that means is starting off today with a commitment to keep as much of the stuff which works today working _tomorrow_, and the day after, and so forth. -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [Ksummit-2013-discuss] DT bindings as ABI [was: Do we have people interested in device tree janitoring / cleanup?]
On Wed, Jul 31, 2013 at 4:14 PM, Russell King - ARM Linux wrote: > On Wed, Jul 31, 2013 at 10:00:17PM +0200, Richard Cochran wrote: >> On Wed, Jul 31, 2013 at 09:29:35PM +0200, Tomasz Figa wrote: >> > >> > I showed you two example solutions that could handle this use case without >> > stable binding ABI, just to prove that b) is not the only option (even if >> > it's the best one, which I continue to agree on, don't get me wrong). >> >> You only showed *one* solution (b) that satisfies the use case. The >> use case was: >> >>User acquires a machine running ARM Linux version 3.x, with u-boot >>and dtb in a read only flash partition. The board boots and works just >> ^ >>fine. However, for his application, the user requires a new kernel >>feature that appeared in version 3.y where y > x. He compiles the new >>kernel, and it also works. >> >> But you suggested: >> >> a) using DT as direct replacement for board files - this means that you >> are free to say that DTSes are strictly coupled with kernel version >> and you are free to modify the bindings - see the analogy to board >> files, where you could modify the platform data structures and could >> not directly copy board file from one kernel version to another, >> >> In the use case, the kernel is upgraded, but not the DTB. So this >> solution makes no sense. > > That's also crap for another reason. DT on the whole is _supposed_ to > describe what the hardware is, and how it is wired up in a WELL DEFINED > and STABLE manner. Unfortunately, there's a *BIG* mistake that was made > - having this /chosen node in DT which gets used for "software" > configuration - eg, the command line and so forth. > > That was a mistake because it means that the DT isn't purely a > description of the hardware. Such stuff should have been left in ATAGs, > and DT should have been placed into its own ATAG entry. There would > not have been any conflict between ATAGs and DT, because ATAGs generally > don't deal with hardware configuration - the only real bit of hardware > configuration conveyed via ATAGs is the location of memory and size of > memory. > > However, if we go back to the idea that DT is supposed to describe the > hardware, _and_ that the way to describe that hardware is well defined > and _stable_ (as it should be) then there should be no reason what so > ever that your old DT blob should not continue working in later kernels. > If it doesn't, then the contract that DT promised when we first moved > over to using DT has been _broken_. Suppose your DT predates PINCTRL and the CPU needs the pins set to function. But setting the pins up is a board specific function. How is this going to get implemented in a generic kernel that doesn't have any board specific code in it? I would propose that we only need to worry about DTs that got put into boot loaders and not worry about ones that were only used when appended to a kernel. For those deployed bootloader based DTs we'd use a quirk system to catch them at early boot time and add in the missing PINCTRL information. The function of this quirk system is to update deployed DTs with the needed information to allow a completely generic kernel to run. Now we can have a very clean generic kernel with all of the board specific fixups localized to a single spot - the quirk for those deployed DTs. And hopefully new boards won't need any quirks. > > Quite frankly, the fact that this discussion has gone on soo far, and > everyone is saying that the existing DT descriptions to date (for what, > two years) are all unstable is really extremely sad. > > I've stated in my previous postings what I think very clearly - and it > comes down to the fact that every DT binding which is in existence in > a released kernel _is_ by that very nature a stable binding. If it > wasn't a stable binding, it should have been clearly marked as such > and been subject to CONFIG_EXPERIMENTAL (when it existed) or > CONFIG_STAGING or similar. > > We even went as far as creating documentation for the bindings - directly > along side the documentation for PPC bindings, with no distinction. > > Given all that, it's down right insulting to those who have been using > ARM kernels to now start off a discussion about making these things > stable. Those people who think that these bindings are not stable need > to wake up to reality - we have USERS of this stuff over the last two > years. It's found its way into products. If we're going to keep this > stuff "unstable" then we are actively _hurting_ our users. > > I'll say it again: if a binding has been in a final kernel, it is by > definition a *stable* binding, and compatibility with that binding > *must* be maintained. > > If we're not going to do that, then we owe all the users of the ARM > kernel a VERY BIG apology. > > ___ > linux-arm-kernel mailing list >
Re: [Ksummit-2013-discuss] DT bindings as ABI [was: Do we have people interested in device tree janitoring / cleanup?]
On Wed, Jul 31, 2013 at 10:00:17PM +0200, Richard Cochran wrote: > On Wed, Jul 31, 2013 at 09:29:35PM +0200, Tomasz Figa wrote: > > > > I showed you two example solutions that could handle this use case without > > stable binding ABI, just to prove that b) is not the only option (even if > > it's the best one, which I continue to agree on, don't get me wrong). > > You only showed *one* solution (b) that satisfies the use case. The > use case was: > >User acquires a machine running ARM Linux version 3.x, with u-boot >and dtb in a read only flash partition. The board boots and works just > ^ >fine. However, for his application, the user requires a new kernel >feature that appeared in version 3.y where y > x. He compiles the new >kernel, and it also works. > > But you suggested: > > a) using DT as direct replacement for board files - this means that you > are free to say that DTSes are strictly coupled with kernel version > and you are free to modify the bindings - see the analogy to board > files, where you could modify the platform data structures and could > not directly copy board file from one kernel version to another, > > In the use case, the kernel is upgraded, but not the DTB. So this > solution makes no sense. That's also crap for another reason. DT on the whole is _supposed_ to describe what the hardware is, and how it is wired up in a WELL DEFINED and STABLE manner. Unfortunately, there's a *BIG* mistake that was made - having this /chosen node in DT which gets used for "software" configuration - eg, the command line and so forth. That was a mistake because it means that the DT isn't purely a description of the hardware. Such stuff should have been left in ATAGs, and DT should have been placed into its own ATAG entry. There would not have been any conflict between ATAGs and DT, because ATAGs generally don't deal with hardware configuration - the only real bit of hardware configuration conveyed via ATAGs is the location of memory and size of memory. However, if we go back to the idea that DT is supposed to describe the hardware, _and_ that the way to describe that hardware is well defined and _stable_ (as it should be) then there should be no reason what so ever that your old DT blob should not continue working in later kernels. If it doesn't, then the contract that DT promised when we first moved over to using DT has been _broken_. Quite frankly, the fact that this discussion has gone on soo far, and everyone is saying that the existing DT descriptions to date (for what, two years) are all unstable is really extremely sad. I've stated in my previous postings what I think very clearly - and it comes down to the fact that every DT binding which is in existence in a released kernel _is_ by that very nature a stable binding. If it wasn't a stable binding, it should have been clearly marked as such and been subject to CONFIG_EXPERIMENTAL (when it existed) or CONFIG_STAGING or similar. We even went as far as creating documentation for the bindings - directly along side the documentation for PPC bindings, with no distinction. Given all that, it's down right insulting to those who have been using ARM kernels to now start off a discussion about making these things stable. Those people who think that these bindings are not stable need to wake up to reality - we have USERS of this stuff over the last two years. It's found its way into products. If we're going to keep this stuff "unstable" then we are actively _hurting_ our users. I'll say it again: if a binding has been in a final kernel, it is by definition a *stable* binding, and compatibility with that binding *must* be maintained. If we're not going to do that, then we owe all the users of the ARM kernel a VERY BIG apology. -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [Ksummit-2013-discuss] DT bindings as ABI [was: Do we have people interested in device tree janitoring / cleanup?]
On Wed, Jul 31, 2013 at 09:29:35PM +0200, Tomasz Figa wrote: > > I showed you two example solutions that could handle this use case without > stable binding ABI, just to prove that b) is not the only option (even if > it's the best one, which I continue to agree on, don't get me wrong). You only showed *one* solution (b) that satisfies the use case. The use case was: User acquires a machine running ARM Linux version 3.x, with u-boot and dtb in a read only flash partition. The board boots and works just ^ fine. However, for his application, the user requires a new kernel feature that appeared in version 3.y where y > x. He compiles the new kernel, and it also works. But you suggested: a) using DT as direct replacement for board files - this means that you are free to say that DTSes are strictly coupled with kernel version and you are free to modify the bindings - see the analogy to board files, where you could modify the platform data structures and could not directly copy board file from one kernel version to another, In the use case, the kernel is upgraded, but not the DTB. So this solution makes no sense. > I also added that the use case is not fully valid, The use case *is* valid. I am not inventing this just to be a pain. There are plenty of people unable or unwilling to upgrade their boot loader or DTB. You can say that you won't support it, but it is a use case from actual real life. > because you can't > magically define bindings for all future hardware, which means you can't > support the hardware using a DTB made before stable bindings for that > hardware have ever been introduced. Surely it is possible to develop and release a stable kernel and DT ABI for a given hardware? Once this is present, it is reasonable for users to expect forward compatibility from the DT system. > With all of this, I agreed that a DTB made for kernel 3.x, when used with > kernel 3.y (y > x) should provide the same or greater feature set than > used with kernel 3.x, in other words, this should cause no regressions. > Still, for new features, you will likely need to update the DTB. Yes, that is the idea. > So, again, to summarize, my view on this is as follows: > - there is a list of best practices for binding design and existing >stable bindings that can be used to help for designing new bindings, > - new bindings go through review process, > - after positive review, such bindings gets staging status, i.e. they are >marked as something that could change, > - after some period of time (we need to define this precisely) they get >frozen and can't be changed in a way that breaks compatibility any >more. In other words, they become ABI. > > What do you think? Sounds okay to me, but why bother with this marking business? Why not just have staging or development trees like every other subsystem? Thanks, Richard -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [Ksummit-2013-discuss] DT bindings as ABI [was: Do we have people interested in device tree janitoring / cleanup?]
On Wednesday 31 of July 2013 21:12:09 Richard Cochran wrote: > On Wed, Jul 31, 2013 at 05:23:35PM +0200, Tomasz Figa wrote: > > I said it many, many times, that a) and b) I proposed are just two > > extremes. It is unlikely that an extreme solution will be the best > > option to choose. I am strongly for something in the middle, just > > like I wrote in several of my previous replies. > > > > This is something that should be commented, not those extreme options. > > We are saying that pursuing a) is useless because it adds pain and > complexity without adding benefit. I simply don't buy your argument > that DT makes a better platform data, but that is besides the point. > > I had said, think about the users. You said, what users? I wrote a > clear and concise use case. You said, lets think about a) and b) and > all the shades of gray in between. I showed you two example solutions that could handle this use case without stable binding ABI, just to prove that b) is not the only option (even if it's the best one, which I continue to agree on, don't get me wrong). I also added that the use case is not fully valid, because you can't magically define bindings for all future hardware, which means you can't support the hardware using a DTB made before stable bindings for that hardware have ever been introduced. With all of this, I agreed that a DTB made for kernel 3.x, when used with kernel 3.y (y > x) should provide the same or greater feature set than used with kernel 3.x, in other words, this should cause no regressions. Still, for new features, you will likely need to update the DTB. > In order to support the use case, you will have to provide a stable > ABI. You can't have a compromise solution. At the end of the day, > either you have a stable ABI, or you don't. > > It was apparent to me that the arm/dt thing has been meandering around > since its inception, but what was surprising is that people were doing > this on purpose, and now they are defending this. Why can't we get a > firm commitment on having a stable ABI? This is exactly what are we trying to do right now. But you don't get stable things right away. You need some kind of process, for design, review, staging, stabilization, etc. Without all of this we will again surely end up with something pretending to be good and stable, while being completely useless and only a burden to support in new code. So, again, to summarize, my view on this is as follows: - there is a list of best practices for binding design and existing stable bindings that can be used to help for designing new bindings, - new bindings go through review process, - after positive review, such bindings gets staging status, i.e. they are marked as something that could change, - after some period of time (we need to define this precisely) they get frozen and can't be changed in a way that breaks compatibility any more. In other words, they become ABI. What do you think? Best regards, Tomasz -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [Ksummit-2013-discuss] DT bindings as ABI [was: Do we have people interested in device tree janitoring / cleanup?]
On Wed, Jul 31, 2013 at 05:23:35PM +0200, Tomasz Figa wrote: > > I said it many, many times, that a) and b) I proposed are just two extremes. > It is unlikely that an extreme solution will be the best option to choose. I > am strongly for something in the middle, just like I wrote in several of my > previous replies. > > This is something that should be commented, not those extreme options. We are saying that pursuing a) is useless because it adds pain and complexity without adding benefit. I simply don't buy your argument that DT makes a better platform data, but that is besides the point. I had said, think about the users. You said, what users? I wrote a clear and concise use case. You said, lets think about a) and b) and all the shades of gray in between. In order to support the use case, you will have to provide a stable ABI. You can't have a compromise solution. At the end of the day, either you have a stable ABI, or you don't. It was apparent to me that the arm/dt thing has been meandering around since its inception, but what was surprising is that people were doing this on purpose, and now they are defending this. Why can't we get a firm commitment on having a stable ABI? Thanks, Richard -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [Ksummit-2013-discuss] DT bindings as ABI [was: Do we have people interested in device tree janitoring / cleanup?]
On Wednesday 31 of July 2013 17:07:19 Richard Cochran wrote: > On Wed, Jul 31, 2013 at 12:59:59PM +0200, Tomasz Figa wrote: > > On Wednesday 31 of July 2013 12:37:37 Maxime Bizon wrote: > > > Board files are C code anyone has the skill to edit/understand/refactor. > > > Moving to DT and keep them in tree tightly coupled with the kernel > > > version just adds another layer of indirection for *no purpose*. > > +1 > > That is exactly what I tried to say. I agree with you to some extent. Don't be so extreme though. As I already said, this is not entirely "no purpose", as there are more benefits of having device tree than just separation from kernel tree. > > > Linus started the whole thing some years ago by refusing to pull ARM > > > tree [1]. Reread his post, what he wants is clearly b). > > > > > > Going a) does not solve any problem. You are just moving churn to > > > somewhere else. We had board files churn, then defconfigs churn, DTS > > > files (and associated drivers) will be next. > > And at this rate, we are headed for another Linus ultimatum, sooner or > later. (see end of the message) > > > DT is self inflicted pain. It has to be for the greater good. > > > > It has several benefits over board files that I mentioned above, possible > > without fully separating them from kernel tree. > > Every time a criticism is voiced about DT, the DT people stick their > fingers in their ears and say, "NAH, NAH, NAH, I CAN'T HEAR YOU!" I won't comment this... > WRT to DT-as-platform-device, we would rather stick with the C code, > please. Just pushing the configuration tables into an external form > does not simplify the problem. In fact, it creates new problems by > inviting the possibility of a bootloader/DT/kernel mismatch. Care to stop ignoring my points other than those defending ideas (nowhere stated as good or bad) from extreme opinions? I said it many, many times, that a) and b) I proposed are just two extremes. It is unlikely that an extreme solution will be the best option to choose. I am strongly for something in the middle, just like I wrote in several of my previous replies. This is something that should be commented, not those extreme options. Best regards, Tomasz -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [Ksummit-2013-discuss] DT bindings as ABI [was: Do we have people interested in device tree janitoring / cleanup?]
On Wed, Jul 31, 2013 at 12:59:59PM +0200, Tomasz Figa wrote: > On Wednesday 31 of July 2013 12:37:37 Maxime Bizon wrote: > > > > Board files are C code anyone has the skill to edit/understand/refactor. > > Moving to DT and keep them in tree tightly coupled with the kernel > > version just adds another layer of indirection for *no purpose*. +1 That is exactly what I tried to say. > > Linus started the whole thing some years ago by refusing to pull ARM > > tree [1]. Reread his post, what he wants is clearly b). > > > > Going a) does not solve any problem. You are just moving churn to > > somewhere else. We had board files churn, then defconfigs churn, DTS > > files (and associated drivers) will be next. And at this rate, we are headed for another Linus ultimatum, sooner or later. > > DT is self inflicted pain. It has to be for the greater good. > > It has several benefits over board files that I mentioned above, possible > without fully separating them from kernel tree. Every time a criticism is voiced about DT, the DT people stick their fingers in their ears and say, "NAH, NAH, NAH, I CAN'T HEAR YOU!" WRT to DT-as-platform-device, we would rather stick with the C code, please. Just pushing the configuration tables into an external form does not simplify the problem. In fact, it creates new problems by inviting the possibility of a bootloader/DT/kernel mismatch. Thanks, Richard -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: DT bindings as ABI [was: Do we have people interested in device tree janitoring / cleanup?]
On Thu, 2013-07-25 at 18:57 +0100, Mark Rutland wrote: > > A possible way to handle this is to have exactly that: A group of > > people that essentially constitute the "standards committee" that meet > > on a regular basis to review and approve new bindings. They should be > > people not just doing ARM Linux work, but other stakeholders in these > > bindings too. One of the things they should probably do is sift > > through our current in-kernel bindings and select those who seem ready > > to be locked in, review/discuss/decide upon that and once the decision > > is made, that specific binding does become part of the static, > > never-ever-change ABI of firmware-to-kernel interfaces. That might > > also be the time that the binding is moved from the kernel to a > > separate repo, but that's a technicality that we'll let the DT > > maintainers decide among themselves, IMHO. > > We're going to need input from other OSs too, or the bindings will > remain Linux-specific regardless of how far away the bindings and dts > repo(s) is/are. FWIW my primary interest in stable DT ABIs is because Xen would like to consume the same device trees on boot, as well as be able to do other clever things with DT at runtime. Specifically: * Take the host device tree passed to Xen at boot, filter out the stuff which Xen uses for itself (which is for the most part core architectural stuff) and pass the remainder to the dom0 Linux kernel (or perhaps in the future another kernel) to drive the remainder of the hardware (mostly the SoC specific stuff, but some architectural and some Xen defined stuff too) * Generate a suitable DT for a guest domain on the fly depending on the guest configuration. Obviously both of those need a stable ABI for us to understand, manipulate and generate. Ian. -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [Ksummit-2013-discuss] DT bindings as ABI [was: Do we have people interested in device tree janitoring / cleanup?]
On Wednesday 31 of July 2013 12:37:37 Maxime Bizon wrote: > On Sat, 2013-07-27 at 11:51 -0700, Tomasz Figa wrote: > > Well, it depends on how we use the DT. There are (at least) two possible > > > > usage scenarios: > > a) using DT as direct replacement for board files - this means that you > > > > are free to say that DTSes are strictly coupled with kernel version > > and you are free to modify the bindings - see the analogy to board > > files, where you could modify the platform data structures and could > > not directly copy board file from one kernel version to another, > > I'm shocked to see this as a possible option. > > Board files are C code anyone has the skill to edit/understand/refactor. > Moving to DT and keep them in tree tightly coupled with the kernel > version just adds another layer of indirection for *no purpose*. Well, I agree that it's not the most fortunate scenario, but it is not that bad as it might seem. It still has the benefit of separating hardware data (including board-specific data) from kernel code and has all the benefits of DT, e.g. easy way to specify relations between devices (phandles), no need to manually register any resources, platform data or platform devices, in other words, when describing hardware you just care about the hardware, not how the kernel works. > The fact that we loose compiler syntax/type checking (as highlighted > somewhere else in this thread) even looks like a regression. Syntax checking is already there. We don't have semantic checking yet, but we are working on this. > > b) using DT as an ABI - this is the original way, i.e. define stable > > > > bindings and make sure that anu DTB built for older kernel will > > > > work, with equal or greater set of functionality on newer kernels. > > Linus started the whole thing some years ago by refusing to pull ARM > tree [1]. Reread his post, what he wants is clearly b). > > Going a) does not solve any problem. You are just moving churn to > somewhere else. We had board files churn, then defconfigs churn, DTS > files (and associated drivers) will be next. > > DT is self inflicted pain. It has to be for the greater good. It has several benefits over board files that I mentioned above, possible without fully separating them from kernel tree. > Going b) *might* allow what some people here dream about, a kernel free > of hardware knowledge. A new board could boot a kernel compiled before > it was even designed. This is very unlikely to happen. You can already see problems with defining bindings for existing hardware and what to say about hardware that is not even designed yet? Unless all the board uses is a set of already supported components, that have bindings defined and drivers merged, you would still need to provide remaining drivers and bindings for them. > Now since I do "embedded" stuff everyday, I don't think b) can apply to > the whole ARM world. There is just to much hardware peculiarity. That's why I think we need a hybrid solution. Staging bindings could use a) and those stable one would use b). Staging bindings would be fixed over time if needed and if they turn out to be fine, they can be stabilized and move to b). So basically, you can get all the stable functionality with one DTB on any kernel released after the functionality got stable, but during development of some functionality you might need to change the DTB to test the development. Best regards, Tomasz > [1] https://lkml.org/lkml/2011/3/30/525 -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [Ksummit-2013-discuss] DT bindings as ABI [was: Do we have people interested in device tree janitoring / cleanup?]
On Sat, 2013-07-27 at 11:51 -0700, Tomasz Figa wrote: > Well, it depends on how we use the DT. There are (at least) two possible > usage scenarios: > > a) using DT as direct replacement for board files - this means that you > are free to say that DTSes are strictly coupled with kernel version > and you are free to modify the bindings - see the analogy to board > files, where you could modify the platform data structures and could > not directly copy board file from one kernel version to another, I'm shocked to see this as a possible option. Board files are C code anyone has the skill to edit/understand/refactor. Moving to DT and keep them in tree tightly coupled with the kernel version just adds another layer of indirection for *no purpose*. The fact that we loose compiler syntax/type checking (as highlighted somewhere else in this thread) even looks like a regression. > b) using DT as an ABI - this is the original way, i.e. define stable > bindings and make sure that anu DTB built for older kernel will > work, with equal or greater set of functionality on newer kernels. > Linus started the whole thing some years ago by refusing to pull ARM tree [1]. Reread his post, what he wants is clearly b). Going a) does not solve any problem. You are just moving churn to somewhere else. We had board files churn, then defconfigs churn, DTS files (and associated drivers) will be next. DT is self inflicted pain. It has to be for the greater good. Going b) *might* allow what some people here dream about, a kernel free of hardware knowledge. A new board could boot a kernel compiled before it was even designed. Now since I do "embedded" stuff everyday, I don't think b) can apply to the whole ARM world. There is just to much hardware peculiarity. [1] https://lkml.org/lkml/2011/3/30/525 -- Maxime -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [Ksummit-2013-discuss] DT bindings as ABI [was: Do we have people interested in device tree janitoring / cleanup?]
On Sat, 2013-07-27 at 11:51 -0700, Tomasz Figa wrote: Well, it depends on how we use the DT. There are (at least) two possible usage scenarios: a) using DT as direct replacement for board files - this means that you are free to say that DTSes are strictly coupled with kernel version and you are free to modify the bindings - see the analogy to board files, where you could modify the platform data structures and could not directly copy board file from one kernel version to another, I'm shocked to see this as a possible option. Board files are C code anyone has the skill to edit/understand/refactor. Moving to DT and keep them in tree tightly coupled with the kernel version just adds another layer of indirection for *no purpose*. The fact that we loose compiler syntax/type checking (as highlighted somewhere else in this thread) even looks like a regression. b) using DT as an ABI - this is the original way, i.e. define stable bindings and make sure that anu DTB built for older kernel will work, with equal or greater set of functionality on newer kernels. Linus started the whole thing some years ago by refusing to pull ARM tree [1]. Reread his post, what he wants is clearly b). Going a) does not solve any problem. You are just moving churn to somewhere else. We had board files churn, then defconfigs churn, DTS files (and associated drivers) will be next. DT is self inflicted pain. It has to be for the greater good. Going b) *might* allow what some people here dream about, a kernel free of hardware knowledge. A new board could boot a kernel compiled before it was even designed. Now since I do embedded stuff everyday, I don't think b) can apply to the whole ARM world. There is just to much hardware peculiarity. [1] https://lkml.org/lkml/2011/3/30/525 -- Maxime -- To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [Ksummit-2013-discuss] DT bindings as ABI [was: Do we have people interested in device tree janitoring / cleanup?]
On Wednesday 31 of July 2013 12:37:37 Maxime Bizon wrote: On Sat, 2013-07-27 at 11:51 -0700, Tomasz Figa wrote: Well, it depends on how we use the DT. There are (at least) two possible usage scenarios: a) using DT as direct replacement for board files - this means that you are free to say that DTSes are strictly coupled with kernel version and you are free to modify the bindings - see the analogy to board files, where you could modify the platform data structures and could not directly copy board file from one kernel version to another, I'm shocked to see this as a possible option. Board files are C code anyone has the skill to edit/understand/refactor. Moving to DT and keep them in tree tightly coupled with the kernel version just adds another layer of indirection for *no purpose*. Well, I agree that it's not the most fortunate scenario, but it is not that bad as it might seem. It still has the benefit of separating hardware data (including board-specific data) from kernel code and has all the benefits of DT, e.g. easy way to specify relations between devices (phandles), no need to manually register any resources, platform data or platform devices, in other words, when describing hardware you just care about the hardware, not how the kernel works. The fact that we loose compiler syntax/type checking (as highlighted somewhere else in this thread) even looks like a regression. Syntax checking is already there. We don't have semantic checking yet, but we are working on this. b) using DT as an ABI - this is the original way, i.e. define stable bindings and make sure that anu DTB built for older kernel will work, with equal or greater set of functionality on newer kernels. Linus started the whole thing some years ago by refusing to pull ARM tree [1]. Reread his post, what he wants is clearly b). Going a) does not solve any problem. You are just moving churn to somewhere else. We had board files churn, then defconfigs churn, DTS files (and associated drivers) will be next. DT is self inflicted pain. It has to be for the greater good. It has several benefits over board files that I mentioned above, possible without fully separating them from kernel tree. Going b) *might* allow what some people here dream about, a kernel free of hardware knowledge. A new board could boot a kernel compiled before it was even designed. This is very unlikely to happen. You can already see problems with defining bindings for existing hardware and what to say about hardware that is not even designed yet? Unless all the board uses is a set of already supported components, that have bindings defined and drivers merged, you would still need to provide remaining drivers and bindings for them. Now since I do embedded stuff everyday, I don't think b) can apply to the whole ARM world. There is just to much hardware peculiarity. That's why I think we need a hybrid solution. Staging bindings could use a) and those stable one would use b). Staging bindings would be fixed over time if needed and if they turn out to be fine, they can be stabilized and move to b). So basically, you can get all the stable functionality with one DTB on any kernel released after the functionality got stable, but during development of some functionality you might need to change the DTB to test the development. Best regards, Tomasz [1] https://lkml.org/lkml/2011/3/30/525 -- To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: DT bindings as ABI [was: Do we have people interested in device tree janitoring / cleanup?]
On Thu, 2013-07-25 at 18:57 +0100, Mark Rutland wrote: A possible way to handle this is to have exactly that: A group of people that essentially constitute the standards committee that meet on a regular basis to review and approve new bindings. They should be people not just doing ARM Linux work, but other stakeholders in these bindings too. One of the things they should probably do is sift through our current in-kernel bindings and select those who seem ready to be locked in, review/discuss/decide upon that and once the decision is made, that specific binding does become part of the static, never-ever-change ABI of firmware-to-kernel interfaces. That might also be the time that the binding is moved from the kernel to a separate repo, but that's a technicality that we'll let the DT maintainers decide among themselves, IMHO. We're going to need input from other OSs too, or the bindings will remain Linux-specific regardless of how far away the bindings and dts repo(s) is/are. FWIW my primary interest in stable DT ABIs is because Xen would like to consume the same device trees on boot, as well as be able to do other clever things with DT at runtime. Specifically: * Take the host device tree passed to Xen at boot, filter out the stuff which Xen uses for itself (which is for the most part core architectural stuff) and pass the remainder to the dom0 Linux kernel (or perhaps in the future another kernel) to drive the remainder of the hardware (mostly the SoC specific stuff, but some architectural and some Xen defined stuff too) * Generate a suitable DT for a guest domain on the fly depending on the guest configuration. Obviously both of those need a stable ABI for us to understand, manipulate and generate. Ian. -- To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [Ksummit-2013-discuss] DT bindings as ABI [was: Do we have people interested in device tree janitoring / cleanup?]
On Wed, Jul 31, 2013 at 12:59:59PM +0200, Tomasz Figa wrote: On Wednesday 31 of July 2013 12:37:37 Maxime Bizon wrote: Board files are C code anyone has the skill to edit/understand/refactor. Moving to DT and keep them in tree tightly coupled with the kernel version just adds another layer of indirection for *no purpose*. +1 That is exactly what I tried to say. Linus started the whole thing some years ago by refusing to pull ARM tree [1]. Reread his post, what he wants is clearly b). Going a) does not solve any problem. You are just moving churn to somewhere else. We had board files churn, then defconfigs churn, DTS files (and associated drivers) will be next. And at this rate, we are headed for another Linus ultimatum, sooner or later. DT is self inflicted pain. It has to be for the greater good. It has several benefits over board files that I mentioned above, possible without fully separating them from kernel tree. Every time a criticism is voiced about DT, the DT people stick their fingers in their ears and say, NAH, NAH, NAH, I CAN'T HEAR YOU! WRT to DT-as-platform-device, we would rather stick with the C code, please. Just pushing the configuration tables into an external form does not simplify the problem. In fact, it creates new problems by inviting the possibility of a bootloader/DT/kernel mismatch. Thanks, Richard -- To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [Ksummit-2013-discuss] DT bindings as ABI [was: Do we have people interested in device tree janitoring / cleanup?]
On Wednesday 31 of July 2013 17:07:19 Richard Cochran wrote: On Wed, Jul 31, 2013 at 12:59:59PM +0200, Tomasz Figa wrote: On Wednesday 31 of July 2013 12:37:37 Maxime Bizon wrote: Board files are C code anyone has the skill to edit/understand/refactor. Moving to DT and keep them in tree tightly coupled with the kernel version just adds another layer of indirection for *no purpose*. +1 That is exactly what I tried to say. I agree with you to some extent. Don't be so extreme though. As I already said, this is not entirely no purpose, as there are more benefits of having device tree than just separation from kernel tree. Linus started the whole thing some years ago by refusing to pull ARM tree [1]. Reread his post, what he wants is clearly b). Going a) does not solve any problem. You are just moving churn to somewhere else. We had board files churn, then defconfigs churn, DTS files (and associated drivers) will be next. And at this rate, we are headed for another Linus ultimatum, sooner or later. (see end of the message) DT is self inflicted pain. It has to be for the greater good. It has several benefits over board files that I mentioned above, possible without fully separating them from kernel tree. Every time a criticism is voiced about DT, the DT people stick their fingers in their ears and say, NAH, NAH, NAH, I CAN'T HEAR YOU! I won't comment this... WRT to DT-as-platform-device, we would rather stick with the C code, please. Just pushing the configuration tables into an external form does not simplify the problem. In fact, it creates new problems by inviting the possibility of a bootloader/DT/kernel mismatch. Care to stop ignoring my points other than those defending ideas (nowhere stated as good or bad) from extreme opinions? I said it many, many times, that a) and b) I proposed are just two extremes. It is unlikely that an extreme solution will be the best option to choose. I am strongly for something in the middle, just like I wrote in several of my previous replies. This is something that should be commented, not those extreme options. Best regards, Tomasz -- To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [Ksummit-2013-discuss] DT bindings as ABI [was: Do we have people interested in device tree janitoring / cleanup?]
On Wed, Jul 31, 2013 at 05:23:35PM +0200, Tomasz Figa wrote: I said it many, many times, that a) and b) I proposed are just two extremes. It is unlikely that an extreme solution will be the best option to choose. I am strongly for something in the middle, just like I wrote in several of my previous replies. This is something that should be commented, not those extreme options. We are saying that pursuing a) is useless because it adds pain and complexity without adding benefit. I simply don't buy your argument that DT makes a better platform data, but that is besides the point. I had said, think about the users. You said, what users? I wrote a clear and concise use case. You said, lets think about a) and b) and all the shades of gray in between. In order to support the use case, you will have to provide a stable ABI. You can't have a compromise solution. At the end of the day, either you have a stable ABI, or you don't. It was apparent to me that the arm/dt thing has been meandering around since its inception, but what was surprising is that people were doing this on purpose, and now they are defending this. Why can't we get a firm commitment on having a stable ABI? Thanks, Richard -- To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [Ksummit-2013-discuss] DT bindings as ABI [was: Do we have people interested in device tree janitoring / cleanup?]
On Wednesday 31 of July 2013 21:12:09 Richard Cochran wrote: On Wed, Jul 31, 2013 at 05:23:35PM +0200, Tomasz Figa wrote: I said it many, many times, that a) and b) I proposed are just two extremes. It is unlikely that an extreme solution will be the best option to choose. I am strongly for something in the middle, just like I wrote in several of my previous replies. This is something that should be commented, not those extreme options. We are saying that pursuing a) is useless because it adds pain and complexity without adding benefit. I simply don't buy your argument that DT makes a better platform data, but that is besides the point. I had said, think about the users. You said, what users? I wrote a clear and concise use case. You said, lets think about a) and b) and all the shades of gray in between. I showed you two example solutions that could handle this use case without stable binding ABI, just to prove that b) is not the only option (even if it's the best one, which I continue to agree on, don't get me wrong). I also added that the use case is not fully valid, because you can't magically define bindings for all future hardware, which means you can't support the hardware using a DTB made before stable bindings for that hardware have ever been introduced. With all of this, I agreed that a DTB made for kernel 3.x, when used with kernel 3.y (y x) should provide the same or greater feature set than used with kernel 3.x, in other words, this should cause no regressions. Still, for new features, you will likely need to update the DTB. In order to support the use case, you will have to provide a stable ABI. You can't have a compromise solution. At the end of the day, either you have a stable ABI, or you don't. It was apparent to me that the arm/dt thing has been meandering around since its inception, but what was surprising is that people were doing this on purpose, and now they are defending this. Why can't we get a firm commitment on having a stable ABI? This is exactly what are we trying to do right now. But you don't get stable things right away. You need some kind of process, for design, review, staging, stabilization, etc. Without all of this we will again surely end up with something pretending to be good and stable, while being completely useless and only a burden to support in new code. So, again, to summarize, my view on this is as follows: - there is a list of best practices for binding design and existing stable bindings that can be used to help for designing new bindings, - new bindings go through review process, - after positive review, such bindings gets staging status, i.e. they are marked as something that could change, - after some period of time (we need to define this precisely) they get frozen and can't be changed in a way that breaks compatibility any more. In other words, they become ABI. What do you think? Best regards, Tomasz -- To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [Ksummit-2013-discuss] DT bindings as ABI [was: Do we have people interested in device tree janitoring / cleanup?]
On Wed, Jul 31, 2013 at 09:29:35PM +0200, Tomasz Figa wrote: I showed you two example solutions that could handle this use case without stable binding ABI, just to prove that b) is not the only option (even if it's the best one, which I continue to agree on, don't get me wrong). You only showed *one* solution (b) that satisfies the use case. The use case was: User acquires a machine running ARM Linux version 3.x, with u-boot and dtb in a read only flash partition. The board boots and works just ^ fine. However, for his application, the user requires a new kernel feature that appeared in version 3.y where y x. He compiles the new kernel, and it also works. But you suggested: a) using DT as direct replacement for board files - this means that you are free to say that DTSes are strictly coupled with kernel version and you are free to modify the bindings - see the analogy to board files, where you could modify the platform data structures and could not directly copy board file from one kernel version to another, In the use case, the kernel is upgraded, but not the DTB. So this solution makes no sense. I also added that the use case is not fully valid, The use case *is* valid. I am not inventing this just to be a pain. There are plenty of people unable or unwilling to upgrade their boot loader or DTB. You can say that you won't support it, but it is a use case from actual real life. because you can't magically define bindings for all future hardware, which means you can't support the hardware using a DTB made before stable bindings for that hardware have ever been introduced. Surely it is possible to develop and release a stable kernel and DT ABI for a given hardware? Once this is present, it is reasonable for users to expect forward compatibility from the DT system. With all of this, I agreed that a DTB made for kernel 3.x, when used with kernel 3.y (y x) should provide the same or greater feature set than used with kernel 3.x, in other words, this should cause no regressions. Still, for new features, you will likely need to update the DTB. Yes, that is the idea. So, again, to summarize, my view on this is as follows: - there is a list of best practices for binding design and existing stable bindings that can be used to help for designing new bindings, - new bindings go through review process, - after positive review, such bindings gets staging status, i.e. they are marked as something that could change, - after some period of time (we need to define this precisely) they get frozen and can't be changed in a way that breaks compatibility any more. In other words, they become ABI. What do you think? Sounds okay to me, but why bother with this marking business? Why not just have staging or development trees like every other subsystem? Thanks, Richard -- To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [Ksummit-2013-discuss] DT bindings as ABI [was: Do we have people interested in device tree janitoring / cleanup?]
On Wed, Jul 31, 2013 at 10:00:17PM +0200, Richard Cochran wrote: On Wed, Jul 31, 2013 at 09:29:35PM +0200, Tomasz Figa wrote: I showed you two example solutions that could handle this use case without stable binding ABI, just to prove that b) is not the only option (even if it's the best one, which I continue to agree on, don't get me wrong). You only showed *one* solution (b) that satisfies the use case. The use case was: User acquires a machine running ARM Linux version 3.x, with u-boot and dtb in a read only flash partition. The board boots and works just ^ fine. However, for his application, the user requires a new kernel feature that appeared in version 3.y where y x. He compiles the new kernel, and it also works. But you suggested: a) using DT as direct replacement for board files - this means that you are free to say that DTSes are strictly coupled with kernel version and you are free to modify the bindings - see the analogy to board files, where you could modify the platform data structures and could not directly copy board file from one kernel version to another, In the use case, the kernel is upgraded, but not the DTB. So this solution makes no sense. That's also crap for another reason. DT on the whole is _supposed_ to describe what the hardware is, and how it is wired up in a WELL DEFINED and STABLE manner. Unfortunately, there's a *BIG* mistake that was made - having this /chosen node in DT which gets used for software configuration - eg, the command line and so forth. That was a mistake because it means that the DT isn't purely a description of the hardware. Such stuff should have been left in ATAGs, and DT should have been placed into its own ATAG entry. There would not have been any conflict between ATAGs and DT, because ATAGs generally don't deal with hardware configuration - the only real bit of hardware configuration conveyed via ATAGs is the location of memory and size of memory. However, if we go back to the idea that DT is supposed to describe the hardware, _and_ that the way to describe that hardware is well defined and _stable_ (as it should be) then there should be no reason what so ever that your old DT blob should not continue working in later kernels. If it doesn't, then the contract that DT promised when we first moved over to using DT has been _broken_. Quite frankly, the fact that this discussion has gone on soo far, and everyone is saying that the existing DT descriptions to date (for what, two years) are all unstable is really extremely sad. I've stated in my previous postings what I think very clearly - and it comes down to the fact that every DT binding which is in existence in a released kernel _is_ by that very nature a stable binding. If it wasn't a stable binding, it should have been clearly marked as such and been subject to CONFIG_EXPERIMENTAL (when it existed) or CONFIG_STAGING or similar. We even went as far as creating documentation for the bindings - directly along side the documentation for PPC bindings, with no distinction. Given all that, it's down right insulting to those who have been using ARM kernels to now start off a discussion about making these things stable. Those people who think that these bindings are not stable need to wake up to reality - we have USERS of this stuff over the last two years. It's found its way into products. If we're going to keep this stuff unstable then we are actively _hurting_ our users. I'll say it again: if a binding has been in a final kernel, it is by definition a *stable* binding, and compatibility with that binding *must* be maintained. If we're not going to do that, then we owe all the users of the ARM kernel a VERY BIG apology. -- To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [Ksummit-2013-discuss] DT bindings as ABI [was: Do we have people interested in device tree janitoring / cleanup?]
On Wed, Jul 31, 2013 at 4:14 PM, Russell King - ARM Linux li...@arm.linux.org.uk wrote: On Wed, Jul 31, 2013 at 10:00:17PM +0200, Richard Cochran wrote: On Wed, Jul 31, 2013 at 09:29:35PM +0200, Tomasz Figa wrote: I showed you two example solutions that could handle this use case without stable binding ABI, just to prove that b) is not the only option (even if it's the best one, which I continue to agree on, don't get me wrong). You only showed *one* solution (b) that satisfies the use case. The use case was: User acquires a machine running ARM Linux version 3.x, with u-boot and dtb in a read only flash partition. The board boots and works just ^ fine. However, for his application, the user requires a new kernel feature that appeared in version 3.y where y x. He compiles the new kernel, and it also works. But you suggested: a) using DT as direct replacement for board files - this means that you are free to say that DTSes are strictly coupled with kernel version and you are free to modify the bindings - see the analogy to board files, where you could modify the platform data structures and could not directly copy board file from one kernel version to another, In the use case, the kernel is upgraded, but not the DTB. So this solution makes no sense. That's also crap for another reason. DT on the whole is _supposed_ to describe what the hardware is, and how it is wired up in a WELL DEFINED and STABLE manner. Unfortunately, there's a *BIG* mistake that was made - having this /chosen node in DT which gets used for software configuration - eg, the command line and so forth. That was a mistake because it means that the DT isn't purely a description of the hardware. Such stuff should have been left in ATAGs, and DT should have been placed into its own ATAG entry. There would not have been any conflict between ATAGs and DT, because ATAGs generally don't deal with hardware configuration - the only real bit of hardware configuration conveyed via ATAGs is the location of memory and size of memory. However, if we go back to the idea that DT is supposed to describe the hardware, _and_ that the way to describe that hardware is well defined and _stable_ (as it should be) then there should be no reason what so ever that your old DT blob should not continue working in later kernels. If it doesn't, then the contract that DT promised when we first moved over to using DT has been _broken_. Suppose your DT predates PINCTRL and the CPU needs the pins set to function. But setting the pins up is a board specific function. How is this going to get implemented in a generic kernel that doesn't have any board specific code in it? I would propose that we only need to worry about DTs that got put into boot loaders and not worry about ones that were only used when appended to a kernel. For those deployed bootloader based DTs we'd use a quirk system to catch them at early boot time and add in the missing PINCTRL information. The function of this quirk system is to update deployed DTs with the needed information to allow a completely generic kernel to run. Now we can have a very clean generic kernel with all of the board specific fixups localized to a single spot - the quirk for those deployed DTs. And hopefully new boards won't need any quirks. Quite frankly, the fact that this discussion has gone on soo far, and everyone is saying that the existing DT descriptions to date (for what, two years) are all unstable is really extremely sad. I've stated in my previous postings what I think very clearly - and it comes down to the fact that every DT binding which is in existence in a released kernel _is_ by that very nature a stable binding. If it wasn't a stable binding, it should have been clearly marked as such and been subject to CONFIG_EXPERIMENTAL (when it existed) or CONFIG_STAGING or similar. We even went as far as creating documentation for the bindings - directly along side the documentation for PPC bindings, with no distinction. Given all that, it's down right insulting to those who have been using ARM kernels to now start off a discussion about making these things stable. Those people who think that these bindings are not stable need to wake up to reality - we have USERS of this stuff over the last two years. It's found its way into products. If we're going to keep this stuff unstable then we are actively _hurting_ our users. I'll say it again: if a binding has been in a final kernel, it is by definition a *stable* binding, and compatibility with that binding *must* be maintained. If we're not going to do that, then we owe all the users of the ARM kernel a VERY BIG apology. ___ linux-arm-kernel mailing list linux-arm-ker...@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel -- Jon Smirl
Re: [Ksummit-2013-discuss] DT bindings as ABI [was: Do we have people interested in device tree janitoring / cleanup?]
On Wed, Jul 31, 2013 at 04:37:36PM -0400, jonsm...@gmail.com wrote: On Wed, Jul 31, 2013 at 4:14 PM, Russell King - ARM Linux li...@arm.linux.org.uk wrote: However, if we go back to the idea that DT is supposed to describe the hardware, _and_ that the way to describe that hardware is well defined and _stable_ (as it should be) then there should be no reason what so ever that your old DT blob should not continue working in later kernels. If it doesn't, then the contract that DT promised when we first moved over to using DT has been _broken_. Suppose your DT predates PINCTRL and the CPU needs the pins set to function. But setting the pins up is a board specific function. How is this going to get implemented in a generic kernel that doesn't have any board specific code in it? I would propose that we only need to worry about DTs that got put into boot loaders and not worry about ones that were only used when appended to a kernel. That is - again - our mistake. We should have made it clear from the start that the use of an _appended_ DT, or a DT provided with the kernel being booted was more or less mandatory for the time being. We actually did exactly the opposite - we advertised the appended DT as a stop-gap measure just to allow a DT kernel to be booted with a boot loader which didn't support DT. That in itself _encouraged_ boot loader support for DT on ARM (which in itself is not a bad thing) but also leads people down the path of potentially separating the DT from the kernel. Had we encouraged the use of an appended DT instead, but still encouraged boot loaders to update, and also made a clear statement that DT is unstable (which everyone can see - for example by making our DT stuff depend on CONFIG_EXPERIMENTAL when it existed) then everyone would have been clear about its status. The fact that we did not - the fact that we ignored this process means that it is _our_ problem that people like Richard are blowing up such a storm over this. We need to admit that we got it wrong, and commit to doing it the right way in future. What that means is starting off today with a commitment to keep as much of the stuff which works today working _tomorrow_, and the day after, and so forth. -- To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [Ksummit-2013-discuss] DT bindings as ABI [was: Do we have people interested in device tree janitoring / cleanup?]
On Wed, Jul 31, 2013 at 4:48 PM, Russell King - ARM Linux li...@arm.linux.org.uk wrote: On Wed, Jul 31, 2013 at 04:37:36PM -0400, jonsm...@gmail.com wrote: On Wed, Jul 31, 2013 at 4:14 PM, Russell King - ARM Linux li...@arm.linux.org.uk wrote: However, if we go back to the idea that DT is supposed to describe the hardware, _and_ that the way to describe that hardware is well defined and _stable_ (as it should be) then there should be no reason what so ever that your old DT blob should not continue working in later kernels. If it doesn't, then the contract that DT promised when we first moved over to using DT has been _broken_. Suppose your DT predates PINCTRL and the CPU needs the pins set to function. But setting the pins up is a board specific function. How is this going to get implemented in a generic kernel that doesn't have any board specific code in it? I would propose that we only need to worry about DTs that got put into boot loaders and not worry about ones that were only used when appended to a kernel. That is - again - our mistake. We should have made it clear from the start that the use of an _appended_ DT, or a DT provided with the kernel being booted was more or less mandatory for the time being. We actually did exactly the opposite - we advertised the appended DT as a stop-gap measure just to allow a DT kernel to be booted with a boot loader which didn't support DT. That in itself _encouraged_ boot loader support for DT on ARM (which in itself is not a bad thing) but also leads people down the path of potentially separating the DT from the kernel. Had we encouraged the use of an appended DT instead, but still encouraged boot loaders to update, and also made a clear statement that DT is unstable (which everyone can see - for example by making our DT stuff depend on CONFIG_EXPERIMENTAL when it existed) then everyone would have been clear about its status. The fact that we did not - the fact that we ignored this process means that it is _our_ problem that people like Richard are blowing up such a storm over this. We need to admit that we got it wrong, and commit to doing it the right way in future. What that means is starting off today with a commitment to keep as much of the stuff which works today working _tomorrow_, and the day after, and so forth. What do you think about using a quirk layer to decouple deployed DTs from whatever is implemented in the kernel? I still think there is going to be volatility in the the kernel DT usage simply because we haven't figured out all of the issues with it yet. For example defining a global schema system is going to expose a lot of irregular DT syntax when you line up all of the platform DTs in a system where it is easy to compare them. One the plus side the kernel is certainly evolving to a point where this volatility will stop in the future, we just aren't there yet. If we don't use a quirk fixup system, then board specific code is going to continue to exist scattered around in the arch directories. My preference would be to contain the board specific DT fixup code inside a quirk system and have the goal of making the arch directories free of board specific code. Any new board would have a good enough DT that they don't need any board specific DT fixup code. Of course there's quite a ways to go before reaching that goal. Alternatively you may be of the belief that it is impossible to get rid of the board specific code. But x86 doesn't have any of it, why should ARM? -- Jon Smirl jonsm...@gmail.com -- To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [Ksummit-2013-discuss] DT bindings as ABI [was: Do we have people interested in device tree janitoring / cleanup?]
On Tue, Jul 30, 2013 at 10:30:45AM -0600, Stephen Warren wrote: > On 07/29/2013 08:15 PM, jonsm...@gmail.com wrote: > > On Mon, Jul 29, 2013 at 9:44 PM, David Gibson > > wrote: > ... > >> I also think we should consider the option of having a simple and > >> straightforward schema language which handles, say, 80% of cases with > >> a fall back to C for the 20% of curly cases. That might actually be > >> simpler to work with in practice than a schema language which can > >> express absolutely anything, at the cost of being awkward for simple > >> cases or difficult to get your head around. > > > > Would C++ work? You can use operating overloading and templates to > > change the syntax into something that doesn't even resemble C any > > more. > > From my perspective, that's precisely why C++ should /not/ be used. Amen. -- John W. LinvilleSomeday the world will need a hero, and you linvi...@tuxdriver.com might be all we have. Be ready. -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [Ksummit-2013-discuss] DT bindings as ABI [was: Do we have people interested in device tree janitoring / cleanup?]
On 07/29/2013 07:44 PM, David Gibson wrote: ... > So, by way of investigation, let me propose an alternative > expression of schemas, that I'm also not convinced we should do, > but is possible and expressive. It's illustrative, because it's > kind of the polar opposite approach to XSD: just use C. That actually sounds reasonable. But, I think there's a difference between a schema, which defines what's legal in an abstract fashion, and a validator, which defines that the data conforms to the schema. I can see that codding a validator in C would be pretty easy, and even potentially quite simple/clean given a suitable set of library functions as you say. However, I'm not so convinced that expressing the schema itself as C would work out so well. I think the code/data-structures would end up being pretty stylized so they could actually be read as a schema. At that point, a specific schema language would probably be simpler? Of course, this is all somewhat conjecture; perhaps the only way to tell would be to prototype various ideas and see how well they work. Equally, I'm not sure I want to bind the schema definition to a particular language. I'd like to be able to programatically generate or manipulate *.dts files in arbitrary languages. Integrating a C interpreter into those languages in order to handle the schema would be annoying, whereas directly handling a special-purpose schema language seems like it'd be more tangible. As an aside, in the past I toyed with the idea of using Python data structures as a *.dts syntax, and hence allowing macros/auto-generation using custom Python code, and I'm sure Forth will come up sometime in this thread:-) -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [Ksummit-2013-discuss] DT bindings as ABI [was: Do we have people interested in device tree janitoring / cleanup?]
On 07/29/2013 08:15 PM, jonsm...@gmail.com wrote: > On Mon, Jul 29, 2013 at 9:44 PM, David Gibson > wrote: ... >> I also think we should consider the option of having a simple and >> straightforward schema language which handles, say, 80% of cases with >> a fall back to C for the 20% of curly cases. That might actually be >> simpler to work with in practice than a schema language which can >> express absolutely anything, at the cost of being awkward for simple >> cases or difficult to get your head around. > > Would C++ work? You can use operating overloading and templates to > change the syntax into something that doesn't even resemble C any > more. >From my perspective, that's precisely why C++ should /not/ be used. -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [Ksummit-2013-discuss] DT bindings as ABI [was: Do we have people interested in device tree janitoring / cleanup?]
On Mon, Jul 29, 2013 at 10:35:03PM -0600, Grant Likely wrote: > >> User acquires a machine running ARM Linux version 3.x, with u-boot > >> and dtb in a read only flash partition. The board boots and works just > >> fine. However, for his application, the user requires a new kernel > >> feature that appeared in version 3.y where y > x. He compiles the new > >> kernel, and it also works. > > > > I'm afraid this kind of use case will never be properly supported, DT > > stable ABI or not. > > Why? New kernel features should be no problem at all. > > New driver features /might/ not be available, but only if the new > feature requires additional data that isn't present in the tree and > cannot be obtained elsewhere. > > > > > Think about this: what kernel will actually be shipped in that board? > > Most likely, it will be a BSP kernel from the vendor. Does the vendor > > will have made that commitment to have a stable ABI for the DT? Will it > > use the same bindings than mainline? Do we want to support all the crazy > > bindings every vendor will come up with? > > That's not a DT issue. That an out-of-tree board/SoC support issue. DT > doesn't make that any better or worse. Yet, with the DT switch, the two are bound together. Before the DT, the only convention we had basically was that we had to be loaded in memory and executed. Now, the bootloader has to pass a complex data structure to the kernel. If this data structure doesn't follow the same convention than we do, we are screwed, and if we can't update either the bootloader or the DT that is loaded by it, we end up screwed. One example that comes to my mind right now is this one: we've been arguing over spidev probing for quite some time. The easy solution would be to add a "linux,spidev" compatible. Obviously, this has been ruled out numerous time. Yet, in the beaglebone black out-of-tree kernel, there is actually a patch that adds this compatible, that is later used in the DTs. These spidev part of the DTs will never work with vanilla. And if you can't update the DT and want to use mainline, you'll never ever have the same feature set. That's of course just a small example, with not much impact on a system, but that could be way worse. But saying that vendors will follow the same convention we do is just unrealistic. I mean, even us haven't be able to work out a stable ABI for more than 2 years now, yet we should expect anyone else to do it? Maxime -- Maxime Ripard, Free Electrons Embedded Linux, Kernel and Android engineering http://free-electrons.com signature.asc Description: Digital signature
Re: [Ksummit-2013-discuss] DT bindings as ABI [was: Do we have people interested in device tree janitoring / cleanup?]
On Mon, Jul 29, 2013 at 10:35:03PM -0600, Grant Likely wrote: User acquires a machine running ARM Linux version 3.x, with u-boot and dtb in a read only flash partition. The board boots and works just fine. However, for his application, the user requires a new kernel feature that appeared in version 3.y where y x. He compiles the new kernel, and it also works. I'm afraid this kind of use case will never be properly supported, DT stable ABI or not. Why? New kernel features should be no problem at all. New driver features /might/ not be available, but only if the new feature requires additional data that isn't present in the tree and cannot be obtained elsewhere. Think about this: what kernel will actually be shipped in that board? Most likely, it will be a BSP kernel from the vendor. Does the vendor will have made that commitment to have a stable ABI for the DT? Will it use the same bindings than mainline? Do we want to support all the crazy bindings every vendor will come up with? That's not a DT issue. That an out-of-tree board/SoC support issue. DT doesn't make that any better or worse. Yet, with the DT switch, the two are bound together. Before the DT, the only convention we had basically was that we had to be loaded in memory and executed. Now, the bootloader has to pass a complex data structure to the kernel. If this data structure doesn't follow the same convention than we do, we are screwed, and if we can't update either the bootloader or the DT that is loaded by it, we end up screwed. One example that comes to my mind right now is this one: we've been arguing over spidev probing for quite some time. The easy solution would be to add a linux,spidev compatible. Obviously, this has been ruled out numerous time. Yet, in the beaglebone black out-of-tree kernel, there is actually a patch that adds this compatible, that is later used in the DTs. These spidev part of the DTs will never work with vanilla. And if you can't update the DT and want to use mainline, you'll never ever have the same feature set. That's of course just a small example, with not much impact on a system, but that could be way worse. But saying that vendors will follow the same convention we do is just unrealistic. I mean, even us haven't be able to work out a stable ABI for more than 2 years now, yet we should expect anyone else to do it? Maxime -- Maxime Ripard, Free Electrons Embedded Linux, Kernel and Android engineering http://free-electrons.com signature.asc Description: Digital signature
Re: [Ksummit-2013-discuss] DT bindings as ABI [was: Do we have people interested in device tree janitoring / cleanup?]
On 07/29/2013 08:15 PM, jonsm...@gmail.com wrote: On Mon, Jul 29, 2013 at 9:44 PM, David Gibson da...@gibson.dropbear.id.au wrote: ... I also think we should consider the option of having a simple and straightforward schema language which handles, say, 80% of cases with a fall back to C for the 20% of curly cases. That might actually be simpler to work with in practice than a schema language which can express absolutely anything, at the cost of being awkward for simple cases or difficult to get your head around. Would C++ work? You can use operating overloading and templates to change the syntax into something that doesn't even resemble C any more. From my perspective, that's precisely why C++ should /not/ be used. -- To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [Ksummit-2013-discuss] DT bindings as ABI [was: Do we have people interested in device tree janitoring / cleanup?]
On 07/29/2013 07:44 PM, David Gibson wrote: ... So, by way of investigation, let me propose an alternative expression of schemas, that I'm also not convinced we should do, but is possible and expressive. It's illustrative, because it's kind of the polar opposite approach to XSD: just use C. That actually sounds reasonable. But, I think there's a difference between a schema, which defines what's legal in an abstract fashion, and a validator, which defines that the data conforms to the schema. I can see that codding a validator in C would be pretty easy, and even potentially quite simple/clean given a suitable set of library functions as you say. However, I'm not so convinced that expressing the schema itself as C would work out so well. I think the code/data-structures would end up being pretty stylized so they could actually be read as a schema. At that point, a specific schema language would probably be simpler? Of course, this is all somewhat conjecture; perhaps the only way to tell would be to prototype various ideas and see how well they work. Equally, I'm not sure I want to bind the schema definition to a particular language. I'd like to be able to programatically generate or manipulate *.dts files in arbitrary languages. Integrating a C interpreter into those languages in order to handle the schema would be annoying, whereas directly handling a special-purpose schema language seems like it'd be more tangible. As an aside, in the past I toyed with the idea of using Python data structures as a *.dts syntax, and hence allowing macros/auto-generation using custom Python code, and I'm sure Forth will come up sometime in this thread:-) -- To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [Ksummit-2013-discuss] DT bindings as ABI [was: Do we have people interested in device tree janitoring / cleanup?]
On Tue, Jul 30, 2013 at 10:30:45AM -0600, Stephen Warren wrote: On 07/29/2013 08:15 PM, jonsm...@gmail.com wrote: On Mon, Jul 29, 2013 at 9:44 PM, David Gibson da...@gibson.dropbear.id.au wrote: ... I also think we should consider the option of having a simple and straightforward schema language which handles, say, 80% of cases with a fall back to C for the 20% of curly cases. That might actually be simpler to work with in practice than a schema language which can express absolutely anything, at the cost of being awkward for simple cases or difficult to get your head around. Would C++ work? You can use operating overloading and templates to change the syntax into something that doesn't even resemble C any more. From my perspective, that's precisely why C++ should /not/ be used. Amen. -- John W. LinvilleSomeday the world will need a hero, and you linvi...@tuxdriver.com might be all we have. Be ready. -- To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [Ksummit-2013-discuss] DT bindings as ABI [was: Do we have people interested in device tree janitoring / cleanup?]
On Mon, Jul 29, 2013 at 1:31 AM, Maxime Ripard wrote: > Hi, > > On Sun, Jul 28, 2013 at 03:19:03PM +0200, Richard Cochran wrote: >> On Sun, Jul 28, 2013 at 11:12:53AM +0200, Tomasz Figa wrote: >> >> > I'm not really sure what effect on users this has. Maybe you should define >> > "users". >> >> ... >> >> > Care to explain this reasoning? >> >> Use Case >> >> >> User acquires a machine running ARM Linux version 3.x, with u-boot >> and dtb in a read only flash partition. The board boots and works just >> fine. However, for his application, the user requires a new kernel >> feature that appeared in version 3.y where y > x. He compiles the new >> kernel, and it also works. > > I'm afraid this kind of use case will never be properly supported, DT > stable ABI or not. Why? New kernel features should be no problem at all. New driver features /might/ not be available, but only if the new feature requires additional data that isn't present in the tree and cannot be obtained elsewhere. > > Think about this: what kernel will actually be shipped in that board? > Most likely, it will be a BSP kernel from the vendor. Does the vendor > will have made that commitment to have a stable ABI for the DT? Will it > use the same bindings than mainline? Do we want to support all the crazy > bindings every vendor will come up with? That's not a DT issue. That an out-of-tree board/SoC support issue. DT doesn't make that any better or worse. g. -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [Ksummit-2013-discuss] DT bindings as ABI [was: Do we have people interested in device tree janitoring / cleanup?]
On Mon, Jul 29, 2013 at 10:15:12PM -0400, jonsm...@gmail.com wrote: > On Mon, Jul 29, 2013 at 9:44 PM, David Gibson > wrote: > > On Sat, Jul 27, 2013 at 10:11:16PM -0700, James Bottomley wrote: > >> On Sat, 2013-07-27 at 21:28 -0600, Grant Likely wrote: > >> > On Sat, Jul 27, 2013 at 2:25 PM, Grant Likely > >> > wrote: > >> > > On Sat, Jul 27, 2013 at 2:01 PM, jonsm...@gmail.com > >> > > wrote: > >> > >> On Sat, Jul 27, 2013 at 3:45 PM, Grant Likely > >> > >> wrote: > >> > >>> On Sat, Jul 27, 2013 at 4:59 AM, Arend van Spriel > >> > >>> wrote: > >> > Let's see how many people go and scream if I say this: Too bad .dts > >> > files > >> > are not done using XML format as DT bindings could be described > >> > using XML > >> > Schema. > >> > >>> > >> > >>> Draft an example and show us how it would look! :-) There is > >> > >>> absolutely nothing preventing us from expressing a DT in XML format, > >> > >>> or even using XSLT to define DT schema while still using our current > >> > >>> .dts syntax. It would be trivial to do lossless translation between > >> > >>> .dts syntax and xml. > >> > >>> > >> > >>> The problem that I have with XML and XSLT is that it is very verbose > >> > >>> and not entirely friendly to mere-mortals. However, I'm more than > >> > >>> willing to be proved wrong on this point. > >> > >> > >> > >> I considered this approach a while ago and discarded it. It would work > >> > >> but it is just too much of a Frankenstein monster. > >> > >> > >> > >> Much cleaner to modify dtc to take a schema as part of the compilation > >> > >> process. The schema language itself has no requirement to look like > >> > >> DTS syntax. Whoever wrote dtc probably has a favorite language that > >> > >> would be good for writing schemas in. > >> > > > >> > > Making it part of dtc is a required feature as far as I'm concerned. > >> > > Using XML/XSLT and dtc-integration are not mutually exclusive, but I > >> > > digress. > >> > > >> > Oops, ignore the XSLT bit. XSLT isn't schema and has no bearing on the > >> > discussion of schema. Sorry for the noise. > >> > >> XSLT is a transform language ... you'd use it say to transform xml to > >> dtc, so it would be an integral component of an xml/xslt based schema. > >> > >> If you want actually to describe and have validated the xml schema > >> itself, then you'd use xsd (XML schema description language) and its > >> associated tools. > >> > >> I'm not saying you *should* do this, just that it's possible (plus I've > >> just blown my kernel cred by knowing about xml, sigh). > > > > Heh. So, it was said in jest, but that actually raises an important > > point. > > > > There are basically two criteria to keep in mind for our > > representation of schemas: > >1) Adequate expressiveness to validate a sufficiently large part, > > of a sufficiently large number of bindings to be useful. > >2) Ease of use and ease of learning **for the target audience**. > > > > To the best of my knowledge xsd would do well on (1), but I'm not > > convinced it does very well on (2). In an environment where XML was > > already widely used, XSD would make perfect sense. Here, I think it > > would be pretty ugly to wire onto the existing DT tools and > > infrastructure, and unpleasantly unfamiliar for many kernel and board > > developers trying to work with DT schemas. > > > > > > So, by way of investigation, let me propose an alternative expression > > of schemas, that I'm also not convinced we should do, but is possible > > and expressive. It's illustrative, because it's kind of the polar > > opposite approach to XSD: just use C. > > > > dtc already has a (so far limited) "checks" mechanism which verifies > > various aspects of DT content. These are implemented by C functions > > in checks.c. There's obviously ample expressiveness - you can express > > any constraint you want that way. It can be pretty verbose, and > > fiddly. A good library of helper functions can mitigate that, but > > it's not clear how much. On the other hand, a very good fraction of > > people working with this will already be familiar with C, which is a > > big plus. This is, after all, the reason that the dts syntax is > > chiefly C inspired. > > > > Now, in practice, I think we will want a more convenient schema > > language (just as we wanted dts, rather than manually constructing > > FDTs as C structures). But I absolutely do think, that the schema > > handling should be handled as plugins to the checks mechanism - > > basically we'd have a validate_schemas() check function. > > > > I also think we should consider the option of having a simple and > > straightforward schema language which handles, say, 80% of cases with > > a fall back to C for the 20% of curly cases. That might actually be > > simpler to work with in practice than a schema language which can > > express absolutely anything, at the cost of being awkward for simple > > cases or difficult to get
Re: [Ksummit-2013-discuss] DT bindings as ABI [was: Do we have people interested in device tree janitoring / cleanup?]
On Mon, Jul 29, 2013 at 9:44 PM, David Gibson wrote: > On Sat, Jul 27, 2013 at 10:11:16PM -0700, James Bottomley wrote: >> On Sat, 2013-07-27 at 21:28 -0600, Grant Likely wrote: >> > On Sat, Jul 27, 2013 at 2:25 PM, Grant Likely >> > wrote: >> > > On Sat, Jul 27, 2013 at 2:01 PM, jonsm...@gmail.com >> > > wrote: >> > >> On Sat, Jul 27, 2013 at 3:45 PM, Grant Likely >> > >> wrote: >> > >>> On Sat, Jul 27, 2013 at 4:59 AM, Arend van Spriel >> > >>> wrote: >> > Let's see how many people go and scream if I say this: Too bad .dts >> > files >> > are not done using XML format as DT bindings could be described using >> > XML >> > Schema. >> > >>> >> > >>> Draft an example and show us how it would look! :-) There is >> > >>> absolutely nothing preventing us from expressing a DT in XML format, >> > >>> or even using XSLT to define DT schema while still using our current >> > >>> .dts syntax. It would be trivial to do lossless translation between >> > >>> .dts syntax and xml. >> > >>> >> > >>> The problem that I have with XML and XSLT is that it is very verbose >> > >>> and not entirely friendly to mere-mortals. However, I'm more than >> > >>> willing to be proved wrong on this point. >> > >> >> > >> I considered this approach a while ago and discarded it. It would work >> > >> but it is just too much of a Frankenstein monster. >> > >> >> > >> Much cleaner to modify dtc to take a schema as part of the compilation >> > >> process. The schema language itself has no requirement to look like >> > >> DTS syntax. Whoever wrote dtc probably has a favorite language that >> > >> would be good for writing schemas in. >> > > >> > > Making it part of dtc is a required feature as far as I'm concerned. >> > > Using XML/XSLT and dtc-integration are not mutually exclusive, but I >> > > digress. >> > >> > Oops, ignore the XSLT bit. XSLT isn't schema and has no bearing on the >> > discussion of schema. Sorry for the noise. >> >> XSLT is a transform language ... you'd use it say to transform xml to >> dtc, so it would be an integral component of an xml/xslt based schema. >> >> If you want actually to describe and have validated the xml schema >> itself, then you'd use xsd (XML schema description language) and its >> associated tools. >> >> I'm not saying you *should* do this, just that it's possible (plus I've >> just blown my kernel cred by knowing about xml, sigh). > > Heh. So, it was said in jest, but that actually raises an important > point. > > There are basically two criteria to keep in mind for our > representation of schemas: >1) Adequate expressiveness to validate a sufficiently large part, > of a sufficiently large number of bindings to be useful. >2) Ease of use and ease of learning **for the target audience**. > > To the best of my knowledge xsd would do well on (1), but I'm not > convinced it does very well on (2). In an environment where XML was > already widely used, XSD would make perfect sense. Here, I think it > would be pretty ugly to wire onto the existing DT tools and > infrastructure, and unpleasantly unfamiliar for many kernel and board > developers trying to work with DT schemas. > > > So, by way of investigation, let me propose an alternative expression > of schemas, that I'm also not convinced we should do, but is possible > and expressive. It's illustrative, because it's kind of the polar > opposite approach to XSD: just use C. > > dtc already has a (so far limited) "checks" mechanism which verifies > various aspects of DT content. These are implemented by C functions > in checks.c. There's obviously ample expressiveness - you can express > any constraint you want that way. It can be pretty verbose, and > fiddly. A good library of helper functions can mitigate that, but > it's not clear how much. On the other hand, a very good fraction of > people working with this will already be familiar with C, which is a > big plus. This is, after all, the reason that the dts syntax is > chiefly C inspired. > > Now, in practice, I think we will want a more convenient schema > language (just as we wanted dts, rather than manually constructing > FDTs as C structures). But I absolutely do think, that the schema > handling should be handled as plugins to the checks mechanism - > basically we'd have a validate_schemas() check function. > > I also think we should consider the option of having a simple and > straightforward schema language which handles, say, 80% of cases with > a fall back to C for the 20% of curly cases. That might actually be > simpler to work with in practice than a schema language which can > express absolutely anything, at the cost of being awkward for simple > cases or difficult to get your head around. Would C++ work? You can use operating overloading and templates to change the syntax into something that doesn't even resemble C any more. > > Remember, a schema language is only a win if it is - for the target > audience - more convenient to
Re: [Ksummit-2013-discuss] DT bindings as ABI [was: Do we have people interested in device tree janitoring / cleanup?]
On Sat, Jul 27, 2013 at 10:11:16PM -0700, James Bottomley wrote: > On Sat, 2013-07-27 at 21:28 -0600, Grant Likely wrote: > > On Sat, Jul 27, 2013 at 2:25 PM, Grant Likely > > wrote: > > > On Sat, Jul 27, 2013 at 2:01 PM, jonsm...@gmail.com > > > wrote: > > >> On Sat, Jul 27, 2013 at 3:45 PM, Grant Likely > > >> wrote: > > >>> On Sat, Jul 27, 2013 at 4:59 AM, Arend van Spriel > > >>> wrote: > > Let's see how many people go and scream if I say this: Too bad .dts > > files > > are not done using XML format as DT bindings could be described using > > XML > > Schema. > > >>> > > >>> Draft an example and show us how it would look! :-) There is > > >>> absolutely nothing preventing us from expressing a DT in XML format, > > >>> or even using XSLT to define DT schema while still using our current > > >>> .dts syntax. It would be trivial to do lossless translation between > > >>> .dts syntax and xml. > > >>> > > >>> The problem that I have with XML and XSLT is that it is very verbose > > >>> and not entirely friendly to mere-mortals. However, I'm more than > > >>> willing to be proved wrong on this point. > > >> > > >> I considered this approach a while ago and discarded it. It would work > > >> but it is just too much of a Frankenstein monster. > > >> > > >> Much cleaner to modify dtc to take a schema as part of the compilation > > >> process. The schema language itself has no requirement to look like > > >> DTS syntax. Whoever wrote dtc probably has a favorite language that > > >> would be good for writing schemas in. > > > > > > Making it part of dtc is a required feature as far as I'm concerned. > > > Using XML/XSLT and dtc-integration are not mutually exclusive, but I > > > digress. > > > > Oops, ignore the XSLT bit. XSLT isn't schema and has no bearing on the > > discussion of schema. Sorry for the noise. > > XSLT is a transform language ... you'd use it say to transform xml to > dtc, so it would be an integral component of an xml/xslt based schema. > > If you want actually to describe and have validated the xml schema > itself, then you'd use xsd (XML schema description language) and its > associated tools. > > I'm not saying you *should* do this, just that it's possible (plus I've > just blown my kernel cred by knowing about xml, sigh). Heh. So, it was said in jest, but that actually raises an important point. There are basically two criteria to keep in mind for our representation of schemas: 1) Adequate expressiveness to validate a sufficiently large part, of a sufficiently large number of bindings to be useful. 2) Ease of use and ease of learning **for the target audience**. To the best of my knowledge xsd would do well on (1), but I'm not convinced it does very well on (2). In an environment where XML was already widely used, XSD would make perfect sense. Here, I think it would be pretty ugly to wire onto the existing DT tools and infrastructure, and unpleasantly unfamiliar for many kernel and board developers trying to work with DT schemas. So, by way of investigation, let me propose an alternative expression of schemas, that I'm also not convinced we should do, but is possible and expressive. It's illustrative, because it's kind of the polar opposite approach to XSD: just use C. dtc already has a (so far limited) "checks" mechanism which verifies various aspects of DT content. These are implemented by C functions in checks.c. There's obviously ample expressiveness - you can express any constraint you want that way. It can be pretty verbose, and fiddly. A good library of helper functions can mitigate that, but it's not clear how much. On the other hand, a very good fraction of people working with this will already be familiar with C, which is a big plus. This is, after all, the reason that the dts syntax is chiefly C inspired. Now, in practice, I think we will want a more convenient schema language (just as we wanted dts, rather than manually constructing FDTs as C structures). But I absolutely do think, that the schema handling should be handled as plugins to the checks mechanism - basically we'd have a validate_schemas() check function. I also think we should consider the option of having a simple and straightforward schema language which handles, say, 80% of cases with a fall back to C for the 20% of curly cases. That might actually be simpler to work with in practice than a schema language which can express absolutely anything, at the cost of being awkward for simple cases or difficult to get your head around. Remember, a schema language is only a win if it is - for the target audience - more convenient to express schemas in than C. -- David Gibson| I'll have my music baroque, and my code david AT gibson.dropbear.id.au | minimalist, thank you. NOT _the_ _other_ | _way_ _around_! http://www.ozlabs.org/~dgibson pgpB1pAGHvvqM.pgp Description: PGP signature
Re: [Ksummit-2013-discuss] DT bindings as ABI [was: Do we have people interested in device tree janitoring / cleanup?]
On Mon, Jul 29, 2013 at 05:14:25PM -0600, Jason Gunthorpe wrote: > On Tue, Jul 30, 2013 at 08:20:39AM +1000, David Gibson wrote: > > > > Clearly general purpose systems (eg servers, workstations, etc) with > > > *full featured firmware* fall into category b. Linux already basically > > > has stable DT for those systems - but the firmware is expected to do > > > lots of work and hide all the low level details from the > > > kernel. Basically, the DT should stick to approximately ePAR and > > > everything else is hidden by the firmware. > > > > No. With the exception of the hypervisor/virtualization extensions, > > ePAPR describes (for now) an entirely passive firmware interface. > > That is, once the handover to the OS has happened, there is *no* > > further firmware interaction. It is not capable of hiding any details > > from the OS, except those which can be done by one-time > > initialization. > > Well, one-time initialization details are actually exactly one of the > areas I am thinking about. Some of the embedded SOCs have extensive > one time initization code in the kernel that is highly SOC specific > and on x86 it would live in the firmware. > > But I see what you mean, ePAPR was the wrong reference, I didn't > carefully check it. I ment the kind of DT use we've seen in SPARC, > POWER servers, Apple stuff, etc - systems explicitly designed so that > new hardware will boot old OSs. Yeah, and at least for POWER servers and Apples, like every other attempt at this, ever, it at best sorta-kinda worked. It's not *as* bad as the mess of broken BIOSes on x86 (mostly due to smaller variety), but there's still plenty of broken crap in Apple workstation and IBM server firmwares and device trees. You see a clear line between "full featured" and "minimal" firmwares where none exists. -- David Gibson| I'll have my music baroque, and my code david AT gibson.dropbear.id.au | minimalist, thank you. NOT _the_ _other_ | _way_ _around_! http://www.ozlabs.org/~dgibson pgpR_NbTqfEwq.pgp Description: PGP signature
Re: [Ksummit-2013-discuss] DT bindings as ABI [was: Do we have people interested in device tree janitoring / cleanup?]
On Tue, Jul 30, 2013 at 08:20:39AM +1000, David Gibson wrote: > > Clearly general purpose systems (eg servers, workstations, etc) with > > *full featured firmware* fall into category b. Linux already basically > > has stable DT for those systems - but the firmware is expected to do > > lots of work and hide all the low level details from the > > kernel. Basically, the DT should stick to approximately ePAR and > > everything else is hidden by the firmware. > > No. With the exception of the hypervisor/virtualization extensions, > ePAPR describes (for now) an entirely passive firmware interface. > That is, once the handover to the OS has happened, there is *no* > further firmware interaction. It is not capable of hiding any details > from the OS, except those which can be done by one-time > initialization. Well, one-time initialization details are actually exactly one of the areas I am thinking about. Some of the embedded SOCs have extensive one time initization code in the kernel that is highly SOC specific and on x86 it would live in the firmware. But I see what you mean, ePAPR was the wrong reference, I didn't carefully check it. I ment the kind of DT use we've seen in SPARC, POWER servers, Apple stuff, etc - systems explicitly designed so that new hardware will boot old OSs. Jason -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [Ksummit-2013-discuss] DT bindings as ABI [was: Do we have people interested in device tree janitoring / cleanup?]
On Mon, Jul 29, 2013 at 12:05:13PM -0600, Jason Gunthorpe wrote: > On Sat, Jul 27, 2013 at 11:51:06AM -0700, Tomasz Figa wrote: > > > Well, it depends on how we use the DT. There are (at least) two possible > > usage scenarios: > > > > a) using DT as direct replacement for board files - this means that you > > are free to say that DTSes are strictly coupled with kernel version > > and you are free to modify the bindings - see the analogy to board > > files, where you could modify the platform data structures and could > > not directly copy board file from one kernel version to another, > > > > b) using DT as an ABI - this is the original way, i.e. define stable > > bindings and make sure that anu DTB built for older kernel will work, > > with equal or greater set of functionality on newer kernels. > > > > Now I believe in this thread the point whether we should use a) or b) or a > > combination of both has been raised. > > Right, and I think it is very important to consider that different > systems can legitimately fall into those categories. > > Clearly general purpose systems (eg servers, workstations, etc) with > *full featured firmware* fall into category b. Linux already basically > has stable DT for those systems - but the firmware is expected to do > lots of work and hide all the low level details from the > kernel. Basically, the DT should stick to approximately ePAR and > everything else is hidden by the firmware. No. With the exception of the hypervisor/virtualization extensions, ePAPR describes (for now) an entirely passive firmware interface. That is, once the handover to the OS has happened, there is *no* further firmware interaction. It is not capable of hiding any details from the OS, except those which can be done by one-time initialization. In fact, a guiding principle of ePAPR's design was that except in cases where it's *much* easier for the firmware to do things, the OS should be expected to do it, because the OS is usually easier to fix than the firmware. > This is essentially how x86 > works and achieves its compatibility. > > Systems that have minimalist firmware, where firmware functions are > pushed into the kernel and the DT is now required to describe > intricate and unique SOC specific functions are clearly very > different, and I think it makes sense to encourage those environments > to be case 'a'. > > Basically, minimalist firmware should have a boot process design that > *can* couple the DT and kernel, while full featured firmware should > keep them seperate. IMHO that should be the clear message to people > implementing this stuff. > > After enough time the DT for 'a' should become stable and churn free, > but expecting/demanding that from day 0 is not helping anyone, IMHO. > > Jason -- David Gibson| I'll have my music baroque, and my code david AT gibson.dropbear.id.au | minimalist, thank you. NOT _the_ _other_ | _way_ _around_! http://www.ozlabs.org/~dgibson pgpmLdvucEcrS.pgp Description: PGP signature
Re: [Ksummit-2013-discuss] DT bindings as ABI [was: Do we have people interested in device tree janitoring / cleanup?]
On Mon, Jul 29, 2013 at 08:38:52PM +0200, Richard Cochran wrote: > > Right, we can and should do better. I got the beaglebone Ethernet > working in mainline (not by writing the driver, but by complaining > over and over again). I except that it will continue to work and not ^^ expect > fall victim to some random DT change. Thanks, Richard -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: DT bindings as ABI [was: Do we have people interested in device tree janitoring / cleanup?]
On Mon, Jul 29, 2013 at 07:16:07PM +0100, Russell King - ARM Linux wrote: > What does it take? Good practice, care, thought and planning. All > the qualities which should already be present for kernel _engineers_. > Not an "lets create something for me, I don't care about anyone else" > attitude. I agree with what you've written, but we are looking at this from different ends of the problem. I fully agree you can create a main line kernel GIT tree that has a stable DT ABI. However, I as an ODM, with time pressure, cannot wait for the kernel folks to finish this work. So from my perspective the DT will not be stable, as I will put whatever interm stuff I choose to have a shippable product. Thus I have to design my systems for an unstable DT, and the message from the kernel community to people in my posistion should be: When you ship early with non-mainlined DT schema, design your boot system around an unstable DT. Plan to migrate your DT to upstream once it becomes finalized. Here is the rub: Once I design for an unstable DT I simply don't derive value from the kernel communities work to create a stable DT. So who is getting the benefit of this work, and is it worth the cost? Jason -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [Ksummit-2013-discuss] DT bindings as ABI [was: Do we have people interested in device tree janitoring / cleanup?]
On Mon, Jul 29, 2013 at 09:31:23AM +0200, Maxime Ripard wrote: > > I'm afraid this kind of use case will never be properly supported, DT > stable ABI or not. > > Think about this: what kernel will actually be shipped in that board? > Most likely, it will be a BSP kernel from the vendor. Does the vendor > will have made that commitment to have a stable ABI for the DT? Will it > use the same bindings than mainline? Do we want to support all the crazy > bindings every vendor will come up with? > > I'm afraid the answer to these three questions will most of the time be > "no.". Yes, I know, and it is sad but true. We can't stop the vendors from shipping half-baked BSPs. I really don't mind that they do that. After all, they want to get *something* working when they launch their chips. > That doesn't mean we shouldn't aim for *mainline* having a stable DT > ABI, but that kind of use case doesn't seem very realistic to me. Right, we can and should do better. I got the beaglebone Ethernet working in mainline (not by writing the driver, but by complaining over and over again). I except that it will continue to work and not fall victim to some random DT change. Thanks, Richard -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [Ksummit-2013-discuss] DT bindings as ABI [was: Do we have people interested in device tree janitoring / cleanup?]
On Mon, Jul 29, 2013 at 12:05:13PM -0600, Jason Gunthorpe wrote: > Clearly general purpose systems (eg servers, workstations, etc) with > *full featured firmware* fall into category b. Linux already basically > has stable DT for those systems - but the firmware is expected to do > lots of work and hide all the low level details from the > kernel. Basically, the DT should stick to approximately ePAR and > everything else is hidden by the firmware. This is essentially how x86 > works and achieves its compatibility. and the downside of placing that trust in firmware is that it very often leads to bugs which can't be solved. I have a set of patches for my laptop/docking station which gets the serial/printer ports working. I've had them for many years now - and I thought I had a problem sorted. I was wrong - the nice helpful ACPI reports on dock that the serial port is enabled, but the port is actually disabled in hardware. If I have ACPI/PNP stuff setup to always believe that this port is disabled, then on boot with the docking station connected, it comes up initially as ttyS0, and then, because the port is "disabled", the resources are busy so it gets reassigned to a different IO port. Undocking/redocking works because the port is now properly re-enabled after a dock event. If I report the ACPI status to the kernel, then on boot, it correctly remains as ttyS0. However, after an undock/redock event, Linux believes that the port is back, but any attempt to use it gets the "LSR safety check engaged" message, because the port is actually disabled. The only way to get it working again is to manually unbind the device and rebind it - so the ACPI disable + enable methods get called. I've had this problem for years, and it's basically unfixable for me - it's only fixable if you are IBM and can reflash the BIOS with alternative ACPI tables which are correct. And I won't submit these patches all the time that I don't know they work correctly. That's the problem with "compatible systems" - you give away trust that someone else will do the right thing, and if they don't, you may be screwed in a way that is can not be worked around by quirks or similar - just like the above. Firmware is swings and roundabouts. You give up some control to a third party so you don't have to think about XYZ, and you hope that they get it right. -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: DT bindings as ABI [was: Do we have people interested in device tree janitoring / cleanup?]
On Mon, Jul 29, 2013 at 11:54:00AM -0600, Jason Gunthorpe wrote: > There is no way I can possibly ship a product with a DT that is > finished. I can't tie my company's product release cycles to the whims > of the kernel community. > > So embedded people are going to ship with unfinished DT and upgrade > later. They have to. There is no choice. Stable DT doesn't change > anything unless you can create perfect stable bindings for a new SOC > instantaneously. > > This is where I see the whole disconnect in this > discussion. General-purpose and embedded are *DIFFERENT* users, and > they have different use-cases and different needs. We have had for the last 20 odd years a stable kernel syscall ABI, in that syscalls which were around 20 years ago are still around today, and still function according to how they're meant to. However, some syscalls were found to be insufficient for todays needs, so they got augmented. For example, the uid/gid IDs used to be 16-bit. We now have 32-bit versions, but the 16-bit versions are still there. (Since ARM started out from early 1.0 times, it too has the 16-bit syscalls, and _still_ does.) So, "stable" is possible. What has to be realised to achieve that it requires nothing more than a "keep existing stuff working" attitude towards it. When you need to change the interface, you supplement it, leaving the old version to work in the same manner that it used to (yes, you should choose to implement the old interfaces using the new stuff, just like we support the 16-bit UID calls but internally deal with 32-bit UIDs.) What does it take? Good practice, care, thought and planning. All the qualities which should already be present for kernel _engineers_. Not an "lets create something for me, I don't care about anyone else" attitude. We can draw the line at an interface becoming stable in exactly the same way that we do every other "stable" interface like syscalls - if it's in a -final kernel, then it has been released at that point as a stable interface to the world. That's how syscalls are dealt with - if a syscall is supported by a -final kernel, then that interface immediately becomes part of the userland ABI and can't be changed or deleted without some really serious thought, a migration path, and sufficient (which means many years) warning to users that it's obsolete. If that is followed, then there is absolutely no reason why a "Stable DT" is not possible - one which it's possible to write a DT file today, and it should still work in 20 years time with updated kernels. That's what a stable interface _should_ allow, and this is what DT _should_ be. It should be possible to say "I have a XYZ ethernet device, it is hooked up like this, uses this interrupt" etc and that same way work in 20 years time to describe exactly the same connections. Sure, we can create a "staging" directory for stuff which we want to merge but is not deemed to be properly fit for the kernel, and deem _that_ stuff to be unstable - just like drivers/stable. Maybe some of it can live in drivers/stable too... -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [Ksummit-2013-discuss] DT bindings as ABI [was: Do we have people interested in device tree janitoring / cleanup?]
On Sat, Jul 27, 2013 at 11:51:06AM -0700, Tomasz Figa wrote: > Well, it depends on how we use the DT. There are (at least) two possible > usage scenarios: > > a) using DT as direct replacement for board files - this means that you > are free to say that DTSes are strictly coupled with kernel version > and you are free to modify the bindings - see the analogy to board > files, where you could modify the platform data structures and could > not directly copy board file from one kernel version to another, > > b) using DT as an ABI - this is the original way, i.e. define stable > bindings and make sure that anu DTB built for older kernel will work, > with equal or greater set of functionality on newer kernels. > > Now I believe in this thread the point whether we should use a) or b) or a > combination of both has been raised. Right, and I think it is very important to consider that different systems can legitimately fall into those categories. Clearly general purpose systems (eg servers, workstations, etc) with *full featured firmware* fall into category b. Linux already basically has stable DT for those systems - but the firmware is expected to do lots of work and hide all the low level details from the kernel. Basically, the DT should stick to approximately ePAR and everything else is hidden by the firmware. This is essentially how x86 works and achieves its compatibility. Systems that have minimalist firmware, where firmware functions are pushed into the kernel and the DT is now required to describe intricate and unique SOC specific functions are clearly very different, and I think it makes sense to encourage those environments to be case 'a'. Basically, minimalist firmware should have a boot process design that *can* couple the DT and kernel, while full featured firmware should keep them seperate. IMHO that should be the clear message to people implementing this stuff. After enough time the DT for 'a' should become stable and churn free, but expecting/demanding that from day 0 is not helping anyone, IMHO. Jason -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: DT bindings as ABI [was: Do we have people interested in device tree janitoring / cleanup?]
On Sat, Jul 27, 2013 at 10:48:26AM +0200, Richard Cochran wrote: > > Of course all this could have been done in C, but it wasn't/hasn't been.. > > You have identified the real issue: poor quality of the ARM board > support process within the kernel. Churn is still churn, whether in DT > or in platform devices, but the DT people promised to end the churn. Actually my experience in this area was mainly from PPC, and granted it spanned the PPC32/PPC64 merge. > [ I disagree about the "more thought" part. The current discussion, > coming years too late after the introduction of DT to ARM Linux, is > contrary evidence enough. ] The current discussion has little to do with the quality of the bindings, look at the new kirkwood bindings - they have had much more thought put into them than the old board.c stuff they are replacing. > > I can't delay shipping while upstream sorts this out - so I know > > *absolutely* that the DT schema will change. This has been planned for > > and designed into the boot system and won't be a problem. > > > > Why would anyone do embedded any other way? > > Yep, that is the embedded way: ship it! > > We can do better than that. I think you missed the point. There is no way I can possibly ship a product with a DT that is finished. I can't tie my company's product release cycles to the whims of the kernel community. So embedded people are going to ship with unfinished DT and upgrade later. They have to. There is no choice. Stable DT doesn't change anything unless you can create perfect stable bindings for a new SOC instantaneously. This is where I see the whole disconnect in this discussion. General-purpose and embedded are *DIFFERENT* users, and they have different use-cases and different needs. Jason -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [Ksummit-2013-discuss] DT bindings as ABI [was: Do we have people interested in device tree janitoring / cleanup?]
On Fri, Jul 26, 2013 at 08:49:43AM -0700, Olof Johansson wrote: > On Fri, Jul 26, 2013 at 7:10 AM, Mark Brown wrote: > > On Fri, Jul 26, 2013 at 03:09:29PM +0200, Richard Cochran wrote: > > > >> Unless I totally misunderstood, the thread is talking about letting > >> established bindings change with each new kernel version. I am > >> opposed to that. > > > > No, nobody is really saying that is a particularly good idea. There is > > some debate about how we work out what an established binding is but > > there's no serious suggestion that we don't want stable bindings. > > Yes, what Mark said -- _today_ all bindings are subject to change and > can be changed in lockstep with the kernel. This has been necessary as > part of development to sort out all of the various bootstrapping > issues across platforms. However, we still have people arguing that we can't (or should not) change a binding right now even to fix inconsistency issues that are discovered after the fact. I'm hearing a different story depending on who is telling it at the moment. Getting quickly to a definitive answer on the criteria for an "established" binding is will help end ongoing discussions as to whether we can fix a currently broken one or just have to leave it. > What we're talking about is to end that mode of operation, and moving > over to locking in bindings. Device tree contents, as mentioned > elsewhere, might still be changed just like code is -- bugs are fixed, > etc. But it's time to start locking down the bindings, in particular > no longer change the established ones. > > Long term, final goal is likely to be close to what Russell is saying > -- nothing should go into the kernel tree unless the binding is in a > fully stable state. However, we have a transitional period between now > and then, and even when we're at the final state there will be need to > have some sort of sandbox for development and test of future bindings. > Dealing with all that, as well as the actual process for locking in > bindings, is what needs to be sorted out. > > I think we're all in agreement that bindings that change over time are > nothing but pain, but we're arguing that in circles anyway. +1 -Matt -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [Ksummit-2013-discuss] DT bindings as ABI [was: Do we have people interested in device tree janitoring / cleanup?]
On Fri, Jul 26, 2013 at 03:40:47PM +0100, Mark Brown wrote: > On Fri, Jul 26, 2013 at 03:21:40PM +0100, Russell King - ARM Linux wrote: > > > So this is why I'm seeing patches just a short time ago removing existing > > compatible strings from the DT descriptions and associated driver, and > > replacing them with new ones... meaning that the old DT files won't work > > with newer kernels. > > The big question is if you're seeing such patches merged. People making > mistakes on submissions is totally normal. Over in the bcm53xx thread, we've discussed such a patch to fix inconsistencies. The problem here is that there is no canonical answer to what a mistake is. I can make a strong argument that the support for these parts is in such an early stage that the bindings (in this case specifically the two different compatible strings for one vendor) should be considered unstable and ok to fix the consistency issue. Mark Rutland suggests we should change nothing and possibly add a third vendor prefix for new bindings. I'm having trouble accepting that just because these bindings made it into a final kernel during a period where scrutinization of these things was not very high that we need to forever carry this inconsistency in the "specification". http://www.spinics.net/lists/arm-kernel/msg262059.html > > If it's the case I think you mean TBH I'm not sure anyone cares, I don't > think anyone is using that stuff in production yet as those chips go > almost exclusively into Android phones. At least the bcm281xx stuff falls into this category as you describe. There's simply nobody that would be upset if its bindings changed from bcm-> as there's just nobody using the DT-based upstream work except for internal Broadcom and a couple people external working closely with them. This situation seems to illustrate the strong need for an unstable binding period that's longer than just inclusion in a final kernel release. -Matt -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [Ksummit-2013-discuss] DT bindings as ABI [was: Do we have people interested in device tree janitoring / cleanup?]
On 07/29/2013 11:19 AM, Arend van Spriel wrote: On 07/27/2013 10:01 PM, jonsm...@gmail.com wrote: On Sat, Jul 27, 2013 at 3:45 PM, Grant Likely wrote: On Sat, Jul 27, 2013 at 4:59 AM, Arend van Spriel wrote: Let's see how many people go and scream if I say this: Too bad .dts files are not done using XML format as DT bindings could be described using XML Schema. Draft an example and show us how it would look! :-) There is absolutely nothing preventing us from expressing a DT in XML format, or even using XSLT to define DT schema while still using our current .dts syntax. It would be trivial to do lossless translation between .dts syntax and xml. The problem that I have with XML and XSLT is that it is very verbose and not entirely friendly to mere-mortals. However, I'm more than willing to be proved wrong on this point. I considered this approach a while ago and discarded it. It would work but it is just too much of a Frankenstein monster. Ah, but he is so cute. At least I do not think it is more monstrous as the bindings files. I just browsed through a couple of arm binding files as I felt challenged to come up with an example. I did not get the impression that there is some kind of template in place to get consitent bindings descriptions. Much cleaner to modify dtc to take a schema as part of the compilation process. The schema language itself has no requirement to look like DTS syntax. Whoever wrote dtc probably has a favorite language that would be good for writing schemas in. Not sure if I can follow here. I guess you mean the dts compilation, right? There are tools freely available to validate XML files against XSD specification files. As an example libxml2 has support for it. I suspect it is not desired to have a dependency for DTC with an out-of-tree library, but it could be incorporated and have DTC spew the validation results. crap. Probably not as libxml2 has MIT-license. Regards, Arend -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [Ksummit-2013-discuss] DT bindings as ABI [was: Do we have people interested in device tree janitoring / cleanup?]
On 07/27/2013 10:01 PM, jonsm...@gmail.com wrote: On Sat, Jul 27, 2013 at 3:45 PM, Grant Likely wrote: On Sat, Jul 27, 2013 at 4:59 AM, Arend van Spriel wrote: Let's see how many people go and scream if I say this: Too bad .dts files are not done using XML format as DT bindings could be described using XML Schema. Draft an example and show us how it would look! :-) There is absolutely nothing preventing us from expressing a DT in XML format, or even using XSLT to define DT schema while still using our current .dts syntax. It would be trivial to do lossless translation between .dts syntax and xml. The problem that I have with XML and XSLT is that it is very verbose and not entirely friendly to mere-mortals. However, I'm more than willing to be proved wrong on this point. I considered this approach a while ago and discarded it. It would work but it is just too much of a Frankenstein monster. Ah, but he is so cute. At least I do not think it is more monstrous as the bindings files. I just browsed through a couple of arm binding files as I felt challenged to come up with an example. I did not get the impression that there is some kind of template in place to get consitent bindings descriptions. Much cleaner to modify dtc to take a schema as part of the compilation process. The schema language itself has no requirement to look like DTS syntax. Whoever wrote dtc probably has a favorite language that would be good for writing schemas in. Not sure if I can follow here. I guess you mean the dts compilation, right? There are tools freely available to validate XML files against XSD specification files. As an example libxml2 has support for it. I suspect it is not desired to have a dependency for DTC with an out-of-tree library, but it could be incorporated and have DTC spew the validation results. Regards, Arend -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [Ksummit-2013-discuss] DT bindings as ABI [was: Do we have people interested in device tree janitoring / cleanup?]
Hi, On Sun, Jul 28, 2013 at 03:19:03PM +0200, Richard Cochran wrote: > On Sun, Jul 28, 2013 at 11:12:53AM +0200, Tomasz Figa wrote: > > > I'm not really sure what effect on users this has. Maybe you should define > > "users". > > ... > > > Care to explain this reasoning? > > Use Case > > > User acquires a machine running ARM Linux version 3.x, with u-boot > and dtb in a read only flash partition. The board boots and works just > fine. However, for his application, the user requires a new kernel > feature that appeared in version 3.y where y > x. He compiles the new > kernel, and it also works. I'm afraid this kind of use case will never be properly supported, DT stable ABI or not. Think about this: what kernel will actually be shipped in that board? Most likely, it will be a BSP kernel from the vendor. Does the vendor will have made that commitment to have a stable ABI for the DT? Will it use the same bindings than mainline? Do we want to support all the crazy bindings every vendor will come up with? I'm afraid the answer to these three questions will most of the time be "no.". That doesn't mean we shouldn't aim for *mainline* having a stable DT ABI, but that kind of use case doesn't seem very realistic to me. Maxime -- Maxime Ripard, Free Electrons Embedded Linux, Kernel and Android engineering http://free-electrons.com signature.asc Description: Digital signature
Re: [Ksummit-2013-discuss] DT bindings as ABI [was: Do we have people interested in device tree janitoring / cleanup?]
Hi, On Sun, Jul 28, 2013 at 03:19:03PM +0200, Richard Cochran wrote: On Sun, Jul 28, 2013 at 11:12:53AM +0200, Tomasz Figa wrote: I'm not really sure what effect on users this has. Maybe you should define users. ... Care to explain this reasoning? Use Case User acquires a machine running ARM Linux version 3.x, with u-boot and dtb in a read only flash partition. The board boots and works just fine. However, for his application, the user requires a new kernel feature that appeared in version 3.y where y x. He compiles the new kernel, and it also works. I'm afraid this kind of use case will never be properly supported, DT stable ABI or not. Think about this: what kernel will actually be shipped in that board? Most likely, it will be a BSP kernel from the vendor. Does the vendor will have made that commitment to have a stable ABI for the DT? Will it use the same bindings than mainline? Do we want to support all the crazy bindings every vendor will come up with? I'm afraid the answer to these three questions will most of the time be no.. That doesn't mean we shouldn't aim for *mainline* having a stable DT ABI, but that kind of use case doesn't seem very realistic to me. Maxime -- Maxime Ripard, Free Electrons Embedded Linux, Kernel and Android engineering http://free-electrons.com signature.asc Description: Digital signature
Re: [Ksummit-2013-discuss] DT bindings as ABI [was: Do we have people interested in device tree janitoring / cleanup?]
On 07/27/2013 10:01 PM, jonsm...@gmail.com wrote: On Sat, Jul 27, 2013 at 3:45 PM, Grant Likely grant.lik...@secretlab.ca wrote: On Sat, Jul 27, 2013 at 4:59 AM, Arend van Spriel ar...@broadcom.com wrote: Let's see how many people go and scream if I say this: Too bad .dts files are not done using XML format as DT bindings could be described using XML Schema. Draft an example and show us how it would look! :-) There is absolutely nothing preventing us from expressing a DT in XML format, or even using XSLT to define DT schema while still using our current .dts syntax. It would be trivial to do lossless translation between .dts syntax and xml. The problem that I have with XML and XSLT is that it is very verbose and not entirely friendly to mere-mortals. However, I'm more than willing to be proved wrong on this point. I considered this approach a while ago and discarded it. It would work but it is just too much of a Frankenstein monster. Ah, but he is so cute. At least I do not think it is more monstrous as the bindings files. I just browsed through a couple of arm binding files as I felt challenged to come up with an example. I did not get the impression that there is some kind of template in place to get consitent bindings descriptions. Much cleaner to modify dtc to take a schema as part of the compilation process. The schema language itself has no requirement to look like DTS syntax. Whoever wrote dtc probably has a favorite language that would be good for writing schemas in. Not sure if I can follow here. I guess you mean the dts compilation, right? There are tools freely available to validate XML files against XSD specification files. As an example libxml2 has support for it. I suspect it is not desired to have a dependency for DTC with an out-of-tree library, but it could be incorporated and have DTC spew the validation results. Regards, Arend -- To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [Ksummit-2013-discuss] DT bindings as ABI [was: Do we have people interested in device tree janitoring / cleanup?]
On 07/29/2013 11:19 AM, Arend van Spriel wrote: On 07/27/2013 10:01 PM, jonsm...@gmail.com wrote: On Sat, Jul 27, 2013 at 3:45 PM, Grant Likely grant.lik...@secretlab.ca wrote: On Sat, Jul 27, 2013 at 4:59 AM, Arend van Spriel ar...@broadcom.com wrote: Let's see how many people go and scream if I say this: Too bad .dts files are not done using XML format as DT bindings could be described using XML Schema. Draft an example and show us how it would look! :-) There is absolutely nothing preventing us from expressing a DT in XML format, or even using XSLT to define DT schema while still using our current .dts syntax. It would be trivial to do lossless translation between .dts syntax and xml. The problem that I have with XML and XSLT is that it is very verbose and not entirely friendly to mere-mortals. However, I'm more than willing to be proved wrong on this point. I considered this approach a while ago and discarded it. It would work but it is just too much of a Frankenstein monster. Ah, but he is so cute. At least I do not think it is more monstrous as the bindings files. I just browsed through a couple of arm binding files as I felt challenged to come up with an example. I did not get the impression that there is some kind of template in place to get consitent bindings descriptions. Much cleaner to modify dtc to take a schema as part of the compilation process. The schema language itself has no requirement to look like DTS syntax. Whoever wrote dtc probably has a favorite language that would be good for writing schemas in. Not sure if I can follow here. I guess you mean the dts compilation, right? There are tools freely available to validate XML files against XSD specification files. As an example libxml2 has support for it. I suspect it is not desired to have a dependency for DTC with an out-of-tree library, but it could be incorporated and have DTC spew the validation results. crap. Probably not as libxml2 has MIT-license. Regards, Arend -- To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [Ksummit-2013-discuss] DT bindings as ABI [was: Do we have people interested in device tree janitoring / cleanup?]
On Fri, Jul 26, 2013 at 03:40:47PM +0100, Mark Brown wrote: On Fri, Jul 26, 2013 at 03:21:40PM +0100, Russell King - ARM Linux wrote: So this is why I'm seeing patches just a short time ago removing existing compatible strings from the DT descriptions and associated driver, and replacing them with new ones... meaning that the old DT files won't work with newer kernels. The big question is if you're seeing such patches merged. People making mistakes on submissions is totally normal. Over in the bcm53xx thread, we've discussed such a patch to fix inconsistencies. The problem here is that there is no canonical answer to what a mistake is. I can make a strong argument that the support for these parts is in such an early stage that the bindings (in this case specifically the two different compatible strings for one vendor) should be considered unstable and ok to fix the consistency issue. Mark Rutland suggests we should change nothing and possibly add a third vendor prefix for new bindings. I'm having trouble accepting that just because these bindings made it into a final kernel during a period where scrutinization of these things was not very high that we need to forever carry this inconsistency in the specification. http://www.spinics.net/lists/arm-kernel/msg262059.html If it's the case I think you mean TBH I'm not sure anyone cares, I don't think anyone is using that stuff in production yet as those chips go almost exclusively into Android phones. At least the bcm281xx stuff falls into this category as you describe. There's simply nobody that would be upset if its bindings changed from bcm-something_else as there's just nobody using the DT-based upstream work except for internal Broadcom and a couple people external working closely with them. This situation seems to illustrate the strong need for an unstable binding period that's longer than just inclusion in a final kernel release. -Matt -- To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [Ksummit-2013-discuss] DT bindings as ABI [was: Do we have people interested in device tree janitoring / cleanup?]
On Fri, Jul 26, 2013 at 08:49:43AM -0700, Olof Johansson wrote: On Fri, Jul 26, 2013 at 7:10 AM, Mark Brown broo...@kernel.org wrote: On Fri, Jul 26, 2013 at 03:09:29PM +0200, Richard Cochran wrote: Unless I totally misunderstood, the thread is talking about letting established bindings change with each new kernel version. I am opposed to that. No, nobody is really saying that is a particularly good idea. There is some debate about how we work out what an established binding is but there's no serious suggestion that we don't want stable bindings. Yes, what Mark said -- _today_ all bindings are subject to change and can be changed in lockstep with the kernel. This has been necessary as part of development to sort out all of the various bootstrapping issues across platforms. However, we still have people arguing that we can't (or should not) change a binding right now even to fix inconsistency issues that are discovered after the fact. I'm hearing a different story depending on who is telling it at the moment. Getting quickly to a definitive answer on the criteria for an established binding is will help end ongoing discussions as to whether we can fix a currently broken one or just have to leave it. What we're talking about is to end that mode of operation, and moving over to locking in bindings. Device tree contents, as mentioned elsewhere, might still be changed just like code is -- bugs are fixed, etc. But it's time to start locking down the bindings, in particular no longer change the established ones. Long term, final goal is likely to be close to what Russell is saying -- nothing should go into the kernel tree unless the binding is in a fully stable state. However, we have a transitional period between now and then, and even when we're at the final state there will be need to have some sort of sandbox for development and test of future bindings. Dealing with all that, as well as the actual process for locking in bindings, is what needs to be sorted out. I think we're all in agreement that bindings that change over time are nothing but pain, but we're arguing that in circles anyway. +1 -Matt -- To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: DT bindings as ABI [was: Do we have people interested in device tree janitoring / cleanup?]
On Sat, Jul 27, 2013 at 10:48:26AM +0200, Richard Cochran wrote: Of course all this could have been done in C, but it wasn't/hasn't been.. You have identified the real issue: poor quality of the ARM board support process within the kernel. Churn is still churn, whether in DT or in platform devices, but the DT people promised to end the churn. Actually my experience in this area was mainly from PPC, and granted it spanned the PPC32/PPC64 merge. [ I disagree about the more thought part. The current discussion, coming years too late after the introduction of DT to ARM Linux, is contrary evidence enough. ] The current discussion has little to do with the quality of the bindings, look at the new kirkwood bindings - they have had much more thought put into them than the old board.c stuff they are replacing. I can't delay shipping while upstream sorts this out - so I know *absolutely* that the DT schema will change. This has been planned for and designed into the boot system and won't be a problem. Why would anyone do embedded any other way? Yep, that is the embedded way: ship it! We can do better than that. I think you missed the point. There is no way I can possibly ship a product with a DT that is finished. I can't tie my company's product release cycles to the whims of the kernel community. So embedded people are going to ship with unfinished DT and upgrade later. They have to. There is no choice. Stable DT doesn't change anything unless you can create perfect stable bindings for a new SOC instantaneously. This is where I see the whole disconnect in this discussion. General-purpose and embedded are *DIFFERENT* users, and they have different use-cases and different needs. Jason -- To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [Ksummit-2013-discuss] DT bindings as ABI [was: Do we have people interested in device tree janitoring / cleanup?]
On Sat, Jul 27, 2013 at 11:51:06AM -0700, Tomasz Figa wrote: Well, it depends on how we use the DT. There are (at least) two possible usage scenarios: a) using DT as direct replacement for board files - this means that you are free to say that DTSes are strictly coupled with kernel version and you are free to modify the bindings - see the analogy to board files, where you could modify the platform data structures and could not directly copy board file from one kernel version to another, b) using DT as an ABI - this is the original way, i.e. define stable bindings and make sure that anu DTB built for older kernel will work, with equal or greater set of functionality on newer kernels. Now I believe in this thread the point whether we should use a) or b) or a combination of both has been raised. Right, and I think it is very important to consider that different systems can legitimately fall into those categories. Clearly general purpose systems (eg servers, workstations, etc) with *full featured firmware* fall into category b. Linux already basically has stable DT for those systems - but the firmware is expected to do lots of work and hide all the low level details from the kernel. Basically, the DT should stick to approximately ePAR and everything else is hidden by the firmware. This is essentially how x86 works and achieves its compatibility. Systems that have minimalist firmware, where firmware functions are pushed into the kernel and the DT is now required to describe intricate and unique SOC specific functions are clearly very different, and I think it makes sense to encourage those environments to be case 'a'. Basically, minimalist firmware should have a boot process design that *can* couple the DT and kernel, while full featured firmware should keep them seperate. IMHO that should be the clear message to people implementing this stuff. After enough time the DT for 'a' should become stable and churn free, but expecting/demanding that from day 0 is not helping anyone, IMHO. Jason -- To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: DT bindings as ABI [was: Do we have people interested in device tree janitoring / cleanup?]
On Mon, Jul 29, 2013 at 11:54:00AM -0600, Jason Gunthorpe wrote: There is no way I can possibly ship a product with a DT that is finished. I can't tie my company's product release cycles to the whims of the kernel community. So embedded people are going to ship with unfinished DT and upgrade later. They have to. There is no choice. Stable DT doesn't change anything unless you can create perfect stable bindings for a new SOC instantaneously. This is where I see the whole disconnect in this discussion. General-purpose and embedded are *DIFFERENT* users, and they have different use-cases and different needs. We have had for the last 20 odd years a stable kernel syscall ABI, in that syscalls which were around 20 years ago are still around today, and still function according to how they're meant to. However, some syscalls were found to be insufficient for todays needs, so they got augmented. For example, the uid/gid IDs used to be 16-bit. We now have 32-bit versions, but the 16-bit versions are still there. (Since ARM started out from early 1.0 times, it too has the 16-bit syscalls, and _still_ does.) So, stable is possible. What has to be realised to achieve that it requires nothing more than a keep existing stuff working attitude towards it. When you need to change the interface, you supplement it, leaving the old version to work in the same manner that it used to (yes, you should choose to implement the old interfaces using the new stuff, just like we support the 16-bit UID calls but internally deal with 32-bit UIDs.) What does it take? Good practice, care, thought and planning. All the qualities which should already be present for kernel _engineers_. Not an lets create something for me, I don't care about anyone else attitude. We can draw the line at an interface becoming stable in exactly the same way that we do every other stable interface like syscalls - if it's in a -final kernel, then it has been released at that point as a stable interface to the world. That's how syscalls are dealt with - if a syscall is supported by a -final kernel, then that interface immediately becomes part of the userland ABI and can't be changed or deleted without some really serious thought, a migration path, and sufficient (which means many years) warning to users that it's obsolete. If that is followed, then there is absolutely no reason why a Stable DT is not possible - one which it's possible to write a DT file today, and it should still work in 20 years time with updated kernels. That's what a stable interface _should_ allow, and this is what DT _should_ be. It should be possible to say I have a XYZ ethernet device, it is hooked up like this, uses this interrupt etc and that same way work in 20 years time to describe exactly the same connections. Sure, we can create a staging directory for stuff which we want to merge but is not deemed to be properly fit for the kernel, and deem _that_ stuff to be unstable - just like drivers/stable. Maybe some of it can live in drivers/stable too... -- To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [Ksummit-2013-discuss] DT bindings as ABI [was: Do we have people interested in device tree janitoring / cleanup?]
On Mon, Jul 29, 2013 at 12:05:13PM -0600, Jason Gunthorpe wrote: Clearly general purpose systems (eg servers, workstations, etc) with *full featured firmware* fall into category b. Linux already basically has stable DT for those systems - but the firmware is expected to do lots of work and hide all the low level details from the kernel. Basically, the DT should stick to approximately ePAR and everything else is hidden by the firmware. This is essentially how x86 works and achieves its compatibility. and the downside of placing that trust in firmware is that it very often leads to bugs which can't be solved. I have a set of patches for my laptop/docking station which gets the serial/printer ports working. I've had them for many years now - and I thought I had a problem sorted. I was wrong - the nice helpful ACPI reports on dock that the serial port is enabled, but the port is actually disabled in hardware. If I have ACPI/PNP stuff setup to always believe that this port is disabled, then on boot with the docking station connected, it comes up initially as ttyS0, and then, because the port is disabled, the resources are busy so it gets reassigned to a different IO port. Undocking/redocking works because the port is now properly re-enabled after a dock event. If I report the ACPI status to the kernel, then on boot, it correctly remains as ttyS0. However, after an undock/redock event, Linux believes that the port is back, but any attempt to use it gets the LSR safety check engaged message, because the port is actually disabled. The only way to get it working again is to manually unbind the device and rebind it - so the ACPI disable + enable methods get called. I've had this problem for years, and it's basically unfixable for me - it's only fixable if you are IBM and can reflash the BIOS with alternative ACPI tables which are correct. And I won't submit these patches all the time that I don't know they work correctly. That's the problem with compatible systems - you give away trust that someone else will do the right thing, and if they don't, you may be screwed in a way that is can not be worked around by quirks or similar - just like the above. Firmware is swings and roundabouts. You give up some control to a third party so you don't have to think about XYZ, and you hope that they get it right. -- To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [Ksummit-2013-discuss] DT bindings as ABI [was: Do we have people interested in device tree janitoring / cleanup?]
On Mon, Jul 29, 2013 at 09:31:23AM +0200, Maxime Ripard wrote: I'm afraid this kind of use case will never be properly supported, DT stable ABI or not. Think about this: what kernel will actually be shipped in that board? Most likely, it will be a BSP kernel from the vendor. Does the vendor will have made that commitment to have a stable ABI for the DT? Will it use the same bindings than mainline? Do we want to support all the crazy bindings every vendor will come up with? I'm afraid the answer to these three questions will most of the time be no.. Yes, I know, and it is sad but true. We can't stop the vendors from shipping half-baked BSPs. I really don't mind that they do that. After all, they want to get *something* working when they launch their chips. That doesn't mean we shouldn't aim for *mainline* having a stable DT ABI, but that kind of use case doesn't seem very realistic to me. Right, we can and should do better. I got the beaglebone Ethernet working in mainline (not by writing the driver, but by complaining over and over again). I except that it will continue to work and not fall victim to some random DT change. Thanks, Richard -- To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: DT bindings as ABI [was: Do we have people interested in device tree janitoring / cleanup?]
On Mon, Jul 29, 2013 at 07:16:07PM +0100, Russell King - ARM Linux wrote: What does it take? Good practice, care, thought and planning. All the qualities which should already be present for kernel _engineers_. Not an lets create something for me, I don't care about anyone else attitude. I agree with what you've written, but we are looking at this from different ends of the problem. I fully agree you can create a main line kernel GIT tree that has a stable DT ABI. However, I as an ODM, with time pressure, cannot wait for the kernel folks to finish this work. So from my perspective the DT will not be stable, as I will put whatever interm stuff I choose to have a shippable product. Thus I have to design my systems for an unstable DT, and the message from the kernel community to people in my posistion should be: When you ship early with non-mainlined DT schema, design your boot system around an unstable DT. Plan to migrate your DT to upstream once it becomes finalized. Here is the rub: Once I design for an unstable DT I simply don't derive value from the kernel communities work to create a stable DT. So who is getting the benefit of this work, and is it worth the cost? Jason -- To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [Ksummit-2013-discuss] DT bindings as ABI [was: Do we have people interested in device tree janitoring / cleanup?]
On Mon, Jul 29, 2013 at 08:38:52PM +0200, Richard Cochran wrote: Right, we can and should do better. I got the beaglebone Ethernet working in mainline (not by writing the driver, but by complaining over and over again). I except that it will continue to work and not ^^ expect fall victim to some random DT change. Thanks, Richard -- To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [Ksummit-2013-discuss] DT bindings as ABI [was: Do we have people interested in device tree janitoring / cleanup?]
On Mon, Jul 29, 2013 at 12:05:13PM -0600, Jason Gunthorpe wrote: On Sat, Jul 27, 2013 at 11:51:06AM -0700, Tomasz Figa wrote: Well, it depends on how we use the DT. There are (at least) two possible usage scenarios: a) using DT as direct replacement for board files - this means that you are free to say that DTSes are strictly coupled with kernel version and you are free to modify the bindings - see the analogy to board files, where you could modify the platform data structures and could not directly copy board file from one kernel version to another, b) using DT as an ABI - this is the original way, i.e. define stable bindings and make sure that anu DTB built for older kernel will work, with equal or greater set of functionality on newer kernels. Now I believe in this thread the point whether we should use a) or b) or a combination of both has been raised. Right, and I think it is very important to consider that different systems can legitimately fall into those categories. Clearly general purpose systems (eg servers, workstations, etc) with *full featured firmware* fall into category b. Linux already basically has stable DT for those systems - but the firmware is expected to do lots of work and hide all the low level details from the kernel. Basically, the DT should stick to approximately ePAR and everything else is hidden by the firmware. No. With the exception of the hypervisor/virtualization extensions, ePAPR describes (for now) an entirely passive firmware interface. That is, once the handover to the OS has happened, there is *no* further firmware interaction. It is not capable of hiding any details from the OS, except those which can be done by one-time initialization. In fact, a guiding principle of ePAPR's design was that except in cases where it's *much* easier for the firmware to do things, the OS should be expected to do it, because the OS is usually easier to fix than the firmware. This is essentially how x86 works and achieves its compatibility. Systems that have minimalist firmware, where firmware functions are pushed into the kernel and the DT is now required to describe intricate and unique SOC specific functions are clearly very different, and I think it makes sense to encourage those environments to be case 'a'. Basically, minimalist firmware should have a boot process design that *can* couple the DT and kernel, while full featured firmware should keep them seperate. IMHO that should be the clear message to people implementing this stuff. After enough time the DT for 'a' should become stable and churn free, but expecting/demanding that from day 0 is not helping anyone, IMHO. Jason -- David Gibson| I'll have my music baroque, and my code david AT gibson.dropbear.id.au | minimalist, thank you. NOT _the_ _other_ | _way_ _around_! http://www.ozlabs.org/~dgibson pgpmLdvucEcrS.pgp Description: PGP signature
Re: [Ksummit-2013-discuss] DT bindings as ABI [was: Do we have people interested in device tree janitoring / cleanup?]
On Tue, Jul 30, 2013 at 08:20:39AM +1000, David Gibson wrote: Clearly general purpose systems (eg servers, workstations, etc) with *full featured firmware* fall into category b. Linux already basically has stable DT for those systems - but the firmware is expected to do lots of work and hide all the low level details from the kernel. Basically, the DT should stick to approximately ePAR and everything else is hidden by the firmware. No. With the exception of the hypervisor/virtualization extensions, ePAPR describes (for now) an entirely passive firmware interface. That is, once the handover to the OS has happened, there is *no* further firmware interaction. It is not capable of hiding any details from the OS, except those which can be done by one-time initialization. Well, one-time initialization details are actually exactly one of the areas I am thinking about. Some of the embedded SOCs have extensive one time initization code in the kernel that is highly SOC specific and on x86 it would live in the firmware. But I see what you mean, ePAPR was the wrong reference, I didn't carefully check it. I ment the kind of DT use we've seen in SPARC, POWER servers, Apple stuff, etc - systems explicitly designed so that new hardware will boot old OSs. Jason -- To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [Ksummit-2013-discuss] DT bindings as ABI [was: Do we have people interested in device tree janitoring / cleanup?]
On Mon, Jul 29, 2013 at 05:14:25PM -0600, Jason Gunthorpe wrote: On Tue, Jul 30, 2013 at 08:20:39AM +1000, David Gibson wrote: Clearly general purpose systems (eg servers, workstations, etc) with *full featured firmware* fall into category b. Linux already basically has stable DT for those systems - but the firmware is expected to do lots of work and hide all the low level details from the kernel. Basically, the DT should stick to approximately ePAR and everything else is hidden by the firmware. No. With the exception of the hypervisor/virtualization extensions, ePAPR describes (for now) an entirely passive firmware interface. That is, once the handover to the OS has happened, there is *no* further firmware interaction. It is not capable of hiding any details from the OS, except those which can be done by one-time initialization. Well, one-time initialization details are actually exactly one of the areas I am thinking about. Some of the embedded SOCs have extensive one time initization code in the kernel that is highly SOC specific and on x86 it would live in the firmware. But I see what you mean, ePAPR was the wrong reference, I didn't carefully check it. I ment the kind of DT use we've seen in SPARC, POWER servers, Apple stuff, etc - systems explicitly designed so that new hardware will boot old OSs. Yeah, and at least for POWER servers and Apples, like every other attempt at this, ever, it at best sorta-kinda worked. It's not *as* bad as the mess of broken BIOSes on x86 (mostly due to smaller variety), but there's still plenty of broken crap in Apple workstation and IBM server firmwares and device trees. You see a clear line between full featured and minimal firmwares where none exists. -- David Gibson| I'll have my music baroque, and my code david AT gibson.dropbear.id.au | minimalist, thank you. NOT _the_ _other_ | _way_ _around_! http://www.ozlabs.org/~dgibson pgpR_NbTqfEwq.pgp Description: PGP signature
Re: [Ksummit-2013-discuss] DT bindings as ABI [was: Do we have people interested in device tree janitoring / cleanup?]
On Sat, Jul 27, 2013 at 10:11:16PM -0700, James Bottomley wrote: On Sat, 2013-07-27 at 21:28 -0600, Grant Likely wrote: On Sat, Jul 27, 2013 at 2:25 PM, Grant Likely grant.lik...@secretlab.ca wrote: On Sat, Jul 27, 2013 at 2:01 PM, jonsm...@gmail.com jonsm...@gmail.com wrote: On Sat, Jul 27, 2013 at 3:45 PM, Grant Likely grant.lik...@secretlab.ca wrote: On Sat, Jul 27, 2013 at 4:59 AM, Arend van Spriel ar...@broadcom.com wrote: Let's see how many people go and scream if I say this: Too bad .dts files are not done using XML format as DT bindings could be described using XML Schema. Draft an example and show us how it would look! :-) There is absolutely nothing preventing us from expressing a DT in XML format, or even using XSLT to define DT schema while still using our current .dts syntax. It would be trivial to do lossless translation between .dts syntax and xml. The problem that I have with XML and XSLT is that it is very verbose and not entirely friendly to mere-mortals. However, I'm more than willing to be proved wrong on this point. I considered this approach a while ago and discarded it. It would work but it is just too much of a Frankenstein monster. Much cleaner to modify dtc to take a schema as part of the compilation process. The schema language itself has no requirement to look like DTS syntax. Whoever wrote dtc probably has a favorite language that would be good for writing schemas in. Making it part of dtc is a required feature as far as I'm concerned. Using XML/XSLT and dtc-integration are not mutually exclusive, but I digress. Oops, ignore the XSLT bit. XSLT isn't schema and has no bearing on the discussion of schema. Sorry for the noise. XSLT is a transform language ... you'd use it say to transform xml to dtc, so it would be an integral component of an xml/xslt based schema. If you want actually to describe and have validated the xml schema itself, then you'd use xsd (XML schema description language) and its associated tools. I'm not saying you *should* do this, just that it's possible (plus I've just blown my kernel cred by knowing about xml, sigh). Heh. So, it was said in jest, but that actually raises an important point. There are basically two criteria to keep in mind for our representation of schemas: 1) Adequate expressiveness to validate a sufficiently large part, of a sufficiently large number of bindings to be useful. 2) Ease of use and ease of learning **for the target audience**. To the best of my knowledge xsd would do well on (1), but I'm not convinced it does very well on (2). In an environment where XML was already widely used, XSD would make perfect sense. Here, I think it would be pretty ugly to wire onto the existing DT tools and infrastructure, and unpleasantly unfamiliar for many kernel and board developers trying to work with DT schemas. So, by way of investigation, let me propose an alternative expression of schemas, that I'm also not convinced we should do, but is possible and expressive. It's illustrative, because it's kind of the polar opposite approach to XSD: just use C. dtc already has a (so far limited) checks mechanism which verifies various aspects of DT content. These are implemented by C functions in checks.c. There's obviously ample expressiveness - you can express any constraint you want that way. It can be pretty verbose, and fiddly. A good library of helper functions can mitigate that, but it's not clear how much. On the other hand, a very good fraction of people working with this will already be familiar with C, which is a big plus. This is, after all, the reason that the dts syntax is chiefly C inspired. Now, in practice, I think we will want a more convenient schema language (just as we wanted dts, rather than manually constructing FDTs as C structures). But I absolutely do think, that the schema handling should be handled as plugins to the checks mechanism - basically we'd have a validate_schemas() check function. I also think we should consider the option of having a simple and straightforward schema language which handles, say, 80% of cases with a fall back to C for the 20% of curly cases. That might actually be simpler to work with in practice than a schema language which can express absolutely anything, at the cost of being awkward for simple cases or difficult to get your head around. Remember, a schema language is only a win if it is - for the target audience - more convenient to express schemas in than C. -- David Gibson| I'll have my music baroque, and my code david AT gibson.dropbear.id.au | minimalist, thank you. NOT _the_ _other_ | _way_ _around_! http://www.ozlabs.org/~dgibson pgpB1pAGHvvqM.pgp Description: PGP signature
Re: [Ksummit-2013-discuss] DT bindings as ABI [was: Do we have people interested in device tree janitoring / cleanup?]
On Mon, Jul 29, 2013 at 9:44 PM, David Gibson da...@gibson.dropbear.id.au wrote: On Sat, Jul 27, 2013 at 10:11:16PM -0700, James Bottomley wrote: On Sat, 2013-07-27 at 21:28 -0600, Grant Likely wrote: On Sat, Jul 27, 2013 at 2:25 PM, Grant Likely grant.lik...@secretlab.ca wrote: On Sat, Jul 27, 2013 at 2:01 PM, jonsm...@gmail.com jonsm...@gmail.com wrote: On Sat, Jul 27, 2013 at 3:45 PM, Grant Likely grant.lik...@secretlab.ca wrote: On Sat, Jul 27, 2013 at 4:59 AM, Arend van Spriel ar...@broadcom.com wrote: Let's see how many people go and scream if I say this: Too bad .dts files are not done using XML format as DT bindings could be described using XML Schema. Draft an example and show us how it would look! :-) There is absolutely nothing preventing us from expressing a DT in XML format, or even using XSLT to define DT schema while still using our current .dts syntax. It would be trivial to do lossless translation between .dts syntax and xml. The problem that I have with XML and XSLT is that it is very verbose and not entirely friendly to mere-mortals. However, I'm more than willing to be proved wrong on this point. I considered this approach a while ago and discarded it. It would work but it is just too much of a Frankenstein monster. Much cleaner to modify dtc to take a schema as part of the compilation process. The schema language itself has no requirement to look like DTS syntax. Whoever wrote dtc probably has a favorite language that would be good for writing schemas in. Making it part of dtc is a required feature as far as I'm concerned. Using XML/XSLT and dtc-integration are not mutually exclusive, but I digress. Oops, ignore the XSLT bit. XSLT isn't schema and has no bearing on the discussion of schema. Sorry for the noise. XSLT is a transform language ... you'd use it say to transform xml to dtc, so it would be an integral component of an xml/xslt based schema. If you want actually to describe and have validated the xml schema itself, then you'd use xsd (XML schema description language) and its associated tools. I'm not saying you *should* do this, just that it's possible (plus I've just blown my kernel cred by knowing about xml, sigh). Heh. So, it was said in jest, but that actually raises an important point. There are basically two criteria to keep in mind for our representation of schemas: 1) Adequate expressiveness to validate a sufficiently large part, of a sufficiently large number of bindings to be useful. 2) Ease of use and ease of learning **for the target audience**. To the best of my knowledge xsd would do well on (1), but I'm not convinced it does very well on (2). In an environment where XML was already widely used, XSD would make perfect sense. Here, I think it would be pretty ugly to wire onto the existing DT tools and infrastructure, and unpleasantly unfamiliar for many kernel and board developers trying to work with DT schemas. So, by way of investigation, let me propose an alternative expression of schemas, that I'm also not convinced we should do, but is possible and expressive. It's illustrative, because it's kind of the polar opposite approach to XSD: just use C. dtc already has a (so far limited) checks mechanism which verifies various aspects of DT content. These are implemented by C functions in checks.c. There's obviously ample expressiveness - you can express any constraint you want that way. It can be pretty verbose, and fiddly. A good library of helper functions can mitigate that, but it's not clear how much. On the other hand, a very good fraction of people working with this will already be familiar with C, which is a big plus. This is, after all, the reason that the dts syntax is chiefly C inspired. Now, in practice, I think we will want a more convenient schema language (just as we wanted dts, rather than manually constructing FDTs as C structures). But I absolutely do think, that the schema handling should be handled as plugins to the checks mechanism - basically we'd have a validate_schemas() check function. I also think we should consider the option of having a simple and straightforward schema language which handles, say, 80% of cases with a fall back to C for the 20% of curly cases. That might actually be simpler to work with in practice than a schema language which can express absolutely anything, at the cost of being awkward for simple cases or difficult to get your head around. Would C++ work? You can use operating overloading and templates to change the syntax into something that doesn't even resemble C any more. Remember, a schema language is only a win if it is - for the target audience - more convenient to express schemas in than C. -- David Gibson| I'll have my music baroque, and my code david AT gibson.dropbear.id.au | minimalist,
Re: [Ksummit-2013-discuss] DT bindings as ABI [was: Do we have people interested in device tree janitoring / cleanup?]
On Mon, Jul 29, 2013 at 10:15:12PM -0400, jonsm...@gmail.com wrote: On Mon, Jul 29, 2013 at 9:44 PM, David Gibson da...@gibson.dropbear.id.au wrote: On Sat, Jul 27, 2013 at 10:11:16PM -0700, James Bottomley wrote: On Sat, 2013-07-27 at 21:28 -0600, Grant Likely wrote: On Sat, Jul 27, 2013 at 2:25 PM, Grant Likely grant.lik...@secretlab.ca wrote: On Sat, Jul 27, 2013 at 2:01 PM, jonsm...@gmail.com jonsm...@gmail.com wrote: On Sat, Jul 27, 2013 at 3:45 PM, Grant Likely grant.lik...@secretlab.ca wrote: On Sat, Jul 27, 2013 at 4:59 AM, Arend van Spriel ar...@broadcom.com wrote: Let's see how many people go and scream if I say this: Too bad .dts files are not done using XML format as DT bindings could be described using XML Schema. Draft an example and show us how it would look! :-) There is absolutely nothing preventing us from expressing a DT in XML format, or even using XSLT to define DT schema while still using our current .dts syntax. It would be trivial to do lossless translation between .dts syntax and xml. The problem that I have with XML and XSLT is that it is very verbose and not entirely friendly to mere-mortals. However, I'm more than willing to be proved wrong on this point. I considered this approach a while ago and discarded it. It would work but it is just too much of a Frankenstein monster. Much cleaner to modify dtc to take a schema as part of the compilation process. The schema language itself has no requirement to look like DTS syntax. Whoever wrote dtc probably has a favorite language that would be good for writing schemas in. Making it part of dtc is a required feature as far as I'm concerned. Using XML/XSLT and dtc-integration are not mutually exclusive, but I digress. Oops, ignore the XSLT bit. XSLT isn't schema and has no bearing on the discussion of schema. Sorry for the noise. XSLT is a transform language ... you'd use it say to transform xml to dtc, so it would be an integral component of an xml/xslt based schema. If you want actually to describe and have validated the xml schema itself, then you'd use xsd (XML schema description language) and its associated tools. I'm not saying you *should* do this, just that it's possible (plus I've just blown my kernel cred by knowing about xml, sigh). Heh. So, it was said in jest, but that actually raises an important point. There are basically two criteria to keep in mind for our representation of schemas: 1) Adequate expressiveness to validate a sufficiently large part, of a sufficiently large number of bindings to be useful. 2) Ease of use and ease of learning **for the target audience**. To the best of my knowledge xsd would do well on (1), but I'm not convinced it does very well on (2). In an environment where XML was already widely used, XSD would make perfect sense. Here, I think it would be pretty ugly to wire onto the existing DT tools and infrastructure, and unpleasantly unfamiliar for many kernel and board developers trying to work with DT schemas. So, by way of investigation, let me propose an alternative expression of schemas, that I'm also not convinced we should do, but is possible and expressive. It's illustrative, because it's kind of the polar opposite approach to XSD: just use C. dtc already has a (so far limited) checks mechanism which verifies various aspects of DT content. These are implemented by C functions in checks.c. There's obviously ample expressiveness - you can express any constraint you want that way. It can be pretty verbose, and fiddly. A good library of helper functions can mitigate that, but it's not clear how much. On the other hand, a very good fraction of people working with this will already be familiar with C, which is a big plus. This is, after all, the reason that the dts syntax is chiefly C inspired. Now, in practice, I think we will want a more convenient schema language (just as we wanted dts, rather than manually constructing FDTs as C structures). But I absolutely do think, that the schema handling should be handled as plugins to the checks mechanism - basically we'd have a validate_schemas() check function. I also think we should consider the option of having a simple and straightforward schema language which handles, say, 80% of cases with a fall back to C for the 20% of curly cases. That might actually be simpler to work with in practice than a schema language which can express absolutely anything, at the cost of being awkward for simple cases or difficult to get your head around. Would C++ work? You can use operating overloading and templates to change the syntax into something that doesn't even resemble C any more. Well, in theory. But given that dtc and the kernel are both in plain C, I don't think
Re: [Ksummit-2013-discuss] DT bindings as ABI [was: Do we have people interested in device tree janitoring / cleanup?]
On Mon, Jul 29, 2013 at 1:31 AM, Maxime Ripard maxime.rip...@free-electrons.com wrote: Hi, On Sun, Jul 28, 2013 at 03:19:03PM +0200, Richard Cochran wrote: On Sun, Jul 28, 2013 at 11:12:53AM +0200, Tomasz Figa wrote: I'm not really sure what effect on users this has. Maybe you should define users. ... Care to explain this reasoning? Use Case User acquires a machine running ARM Linux version 3.x, with u-boot and dtb in a read only flash partition. The board boots and works just fine. However, for his application, the user requires a new kernel feature that appeared in version 3.y where y x. He compiles the new kernel, and it also works. I'm afraid this kind of use case will never be properly supported, DT stable ABI or not. Why? New kernel features should be no problem at all. New driver features /might/ not be available, but only if the new feature requires additional data that isn't present in the tree and cannot be obtained elsewhere. Think about this: what kernel will actually be shipped in that board? Most likely, it will be a BSP kernel from the vendor. Does the vendor will have made that commitment to have a stable ABI for the DT? Will it use the same bindings than mainline? Do we want to support all the crazy bindings every vendor will come up with? That's not a DT issue. That an out-of-tree board/SoC support issue. DT doesn't make that any better or worse. g. -- To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [Ksummit-2013-discuss] DT bindings as ABI [was: Do we have people interested in device tree janitoring / cleanup?]
On Sun, Jul 28, 2013 at 05:35:46PM +0200, Richard Cochran wrote: > On Sun, Jul 28, 2013 at 10:09:57AM -0400, jonsm...@gmail.com wrote: > > > > 3.z kernel is free to alter the schema. But it will have to supply the > > necessary quirks needed to keep those old dtb's functioning. > > The quirks idea sounds okay to me, if it can really provide forward > compatibility. In practice, I doubt anyone will really spend the > effort to make this work. I think it would be much easier to make sure > the bindings are "future proof" in the first place. I should clarify. The idea of DT quirks is not to remove the need to properly design and review bindings. It's to limit the damage when there are, inevitably, failings in that process. And when, also inevitably, firmware vendors ship DTs that don't follow the bindings correctly, even when there is a good one available. I think it's more likely that people will create, and get right, a well localized bit of quirk code, than they will get backwards compat code correct for every place in which a driver wants info from the device tree. -- David Gibson| I'll have my music baroque, and my code david AT gibson.dropbear.id.au | minimalist, thank you. NOT _the_ _other_ | _way_ _around_! http://www.ozlabs.org/~dgibson pgpXkvbcj28C1.pgp Description: PGP signature
Re: [Ksummit-2013-discuss] DT bindings as ABI [was: Do we have people interested in device tree janitoring / cleanup?]
On Sun, Jul 28, 2013 at 11:50:19AM -0400, jonsm...@gmail.com wrote: > "furture proof" is much easier to say that it is to do. We've been > messing around with the audio bindings for three years and still don't > have a really good scheme. It is pretty easy to come up with the first > 90% of a device tree. It is really hard to work out that last 10%. I wouldn't say that for audio - we've got a pretty solid idea of what's going on and little if any churn. If some more of the subsystems that we rely on were working well we could do a bit more but it's mostly just extensions of where we're at now. signature.asc Description: Digital signature
Re: [Ksummit-2013-discuss] DT bindings as ABI [was: Do we have people interested in device tree janitoring / cleanup?]
On Sun, Jul 28, 2013 at 11:35 AM, Richard Cochran wrote: > On Sun, Jul 28, 2013 at 10:09:57AM -0400, jonsm...@gmail.com wrote: >> >> 3.z kernel is free to alter the schema. But it will have to supply the >> necessary quirks needed to keep those old dtb's functioning. > > The quirks idea sounds okay to me, if it can really provide forward > compatibility. In practice, I doubt anyone will really spend the > effort to make this work. I think it would be much easier to make sure > the bindings are "future proof" in the first place. "furture proof" is much easier to say that it is to do. We've been messing around with the audio bindings for three years and still don't have a really good scheme. It is pretty easy to come up with the first 90% of a device tree. It is really hard to work out that last 10%. You can easily get the chips into the tree. Doing that will load the correct device drivers. But now how are these chips wired together? Is the appropriate button, LED, etc attached to all the IO pins offered by the chip? Those answers vary by the PCB the chip was used in.. Trying to figure out a scheme for this has lead to some volatility in the device trees. The whole concept of pin mapping was missing from the early device trees. > > Thanks, > Richard -- Jon Smirl jonsm...@gmail.com -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [Ksummit-2013-discuss] DT bindings as ABI [was: Do we have people interested in device tree janitoring / cleanup?]
On Sun, Jul 28, 2013 at 10:09:57AM -0400, jonsm...@gmail.com wrote: > > 3.z kernel is free to alter the schema. But it will have to supply the > necessary quirks needed to keep those old dtb's functioning. The quirks idea sounds okay to me, if it can really provide forward compatibility. In practice, I doubt anyone will really spend the effort to make this work. I think it would be much easier to make sure the bindings are "future proof" in the first place. Thanks, Richard -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [Ksummit-2013-discuss] DT bindings as ABI [was: Do we have people interested in device tree janitoring / cleanup?]
On Sun, Jul 28, 2013 at 03:39:56PM +0200, Tomasz Figa wrote: > There are many possible options: ... Wow, you totally ignored the use case. > Please note, though, I'm _not_ trying to convince you that this kind of > solutions is good, as I'm not convinced either. That's why we are > discussing this. And next you are going to convince me that updating my BIOS is a normal, expected, and necessary step in upgrading my kernel. No, thank you. Richard -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [Ksummit-2013-discuss] DT bindings as ABI [was: Do we have people interested in device tree janitoring / cleanup?]
On Sun, Jul 28, 2013 at 9:39 AM, Tomasz Figa wrote: > On Sunday 28 of July 2013 15:19:03 Richard Cochran wrote: >> On Sun, Jul 28, 2013 at 11:12:53AM +0200, Tomasz Figa wrote: >> > I'm not really sure what effect on users this has. Maybe you should >> > define "users". >> >> ... >> >> > Care to explain this reasoning? >> >> Use Case >> >> >> User acquires a machine running ARM Linux version 3.x, with u-boot >> and dtb in a read only flash partition. The board boots and works just >> fine. However, for his application, the user requires a new kernel >> feature that appeared in version 3.y where y > x. He compiles the new >> kernel, and it also works. > > Generally the user does not care where the dtb is stored. He just want to > upgrade the kernel without thinking about internals. > > There are many possible options: > > a) The BSP packaging script he received from board vendor, or even kernel > build system, builds dtb from kernel sources and appends it to built > zImage. He just flashes the zImage and everything is working correctly. > > This is pretty common case today, as many boards still use legacy > bootloaders without native support for DT. See the analogy to board > files, being compiled as a part of kernel sources. > > b) The user always compiles the kernel and dtb and flashes both at the > same time. > > This does not differ at all to flashing the kernel alone, except two > files, not one, need to be flashed. c) Use a quirk system to map ROM based DTB's. 3.x kernel was matched to dtb that is flashed into boot rom. Everything was initially ok. The 3.y kernel is built with init time quirks that map the dtb from 3.x format into the 3.y format. These quirks can be easily built since we know the generic schema from both 3.x and 3.y. Mapping ten year old power PC dtbs may involve more code. These quirks aren't going to be large, it's not like the entire schema is going to change on each kernel release. They are just going to make a best effort to map the dtb in ROM into something the 3.y kernel can understand. Owners of these ROMs will complain loudly if the quirks aren't right. 3.z kernel is free to alter the schema. But it will have to supply the necessary quirks needed to keep those old dtb's functioning. > > By the way, in use case you are describing, changes that dtb wouldn't have > to be updated are very low, unless the one present in read only memory of > the board is complete, i.e. fully describes all the available hardware, > which is unlikely for a dtb built at time of 3.x availability, whenever > the feature showed up in 3.y (y > x). The user will most likely have to > update the dtb anyway. > > Please note, though, I'm _not_ trying to convince you that this kind of > solutions is good, as I'm not convinced either. That's why we are > discussing this. > > Best regards, > Tomasz > > -- > To unsubscribe from this list: send the line "unsubscribe linux-kernel" in > the body of a message to majord...@vger.kernel.org > More majordomo info at http://vger.kernel.org/majordomo-info.html > Please read the FAQ at http://www.tux.org/lkml/ -- Jon Smirl jonsm...@gmail.com -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [Ksummit-2013-discuss] DT bindings as ABI [was: Do we have people interested in device tree janitoring / cleanup?]
On Sunday 28 of July 2013 15:19:03 Richard Cochran wrote: > On Sun, Jul 28, 2013 at 11:12:53AM +0200, Tomasz Figa wrote: > > I'm not really sure what effect on users this has. Maybe you should > > define "users". > > ... > > > Care to explain this reasoning? > > Use Case > > > User acquires a machine running ARM Linux version 3.x, with u-boot > and dtb in a read only flash partition. The board boots and works just > fine. However, for his application, the user requires a new kernel > feature that appeared in version 3.y where y > x. He compiles the new > kernel, and it also works. Generally the user does not care where the dtb is stored. He just want to upgrade the kernel without thinking about internals. There are many possible options: a) The BSP packaging script he received from board vendor, or even kernel build system, builds dtb from kernel sources and appends it to built zImage. He just flashes the zImage and everything is working correctly. This is pretty common case today, as many boards still use legacy bootloaders without native support for DT. See the analogy to board files, being compiled as a part of kernel sources. b) The user always compiles the kernel and dtb and flashes both at the same time. This does not differ at all to flashing the kernel alone, except two files, not one, need to be flashed. By the way, in use case you are describing, changes that dtb wouldn't have to be updated are very low, unless the one present in read only memory of the board is complete, i.e. fully describes all the available hardware, which is unlikely for a dtb built at time of 3.x availability, whenever the feature showed up in 3.y (y > x). The user will most likely have to update the dtb anyway. Please note, though, I'm _not_ trying to convince you that this kind of solutions is good, as I'm not convinced either. That's why we are discussing this. Best regards, Tomasz -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [Ksummit-2013-discuss] DT bindings as ABI [was: Do we have people interested in device tree janitoring / cleanup?]
On Sun, Jul 28, 2013 at 11:12:53AM +0200, Tomasz Figa wrote: > I'm not really sure what effect on users this has. Maybe you should define > "users". ... > Care to explain this reasoning? Use Case User acquires a machine running ARM Linux version 3.x, with u-boot and dtb in a read only flash partition. The board boots and works just fine. However, for his application, the user requires a new kernel feature that appeared in version 3.y where y > x. He compiles the new kernel, and it also works. Thanks, Richard -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
