RE: [RFC 00/15] Device Tree schemas and validation
Rob Herring in devicetree/msg06598.html > This is interesting approach using the dts syntax, Benoit Cousson in devicetree/msg06617.html > it has the big advantage of re-using the parser already included in DTC for > free. In term or readability, it avoids to re-defining a brand new syntax for > people who are already familiar with the DTS one. Stephen Warren wrote in devicetree/msg06676.html: > DT is a language for representing data. > The validation checks described by schemas are rules, or code, and not static > data. The syntax of the existing device-tree source is strikingly similar to that of the widely used JavaScript Object Notation, better known as JSON. JSON has many parsers, validators and schemata already in existence. Assuredly as many coders know how to program JSON as know C. JSON makes some sense for representation of device-trees, because, as the authoritative json.org explains, "JSON is a text format that is completely language independent but uses conventions that are familiar to programmers of the C-family of languages . . . JSON is a natural representation of data for the C family of programming languages." Benoit Cousson remarks in devicetree/msg06617.html: > The bindings definition being quite open, there is no easy way to > ensure proper schema / bindings without careful review of the > schema. Stephen Warren continues in devicetree/msg06676.html: > Overall, I believe perhaps the single most important aspect of any DT > schema is schema inheritance or instancing, David Gibson comments in devicetree/msg06725.html: > define the notion of dt "patterns" or "templates". A dt > pattern is to a dt node or subtree as a regex is to a string - it > provides a reasonably expressive way of defining a family of dt > nodes. These would be defined in an extension / superset of dt > syntax. I violently agree with Stephen and David and believe that inheritance offers a partial solution to the problem Benoit describes. What about improving compliance by explicitly making use of inheritance with device-tree include files? Suppose we consider supplementing the ARM tree's skeleton.dtsi tree-root with board.dtsi, cpu.dtsi, daughtercard.dtsi . . . Suppose then we require board-level dts files to include board.dtsi, and furthermore that the CPU node for the board to be described in a DTSI file that must itself include cpu.dtsi. Then the dtc itself could in effect check perform some constraint checking. Device vendors could offer .dtsi and .dtsi that users of those products would be required to include. A related question: what DTS files will a validator compare against a schema? Assuredly given the existing ability of nodes and properties in a hierarchy of include files to override and modify one another, the post C-preprocessed and compiled single-file is the one wanted, that is, the output from $CC $CPP_FLAGS -o foo.tmp foo.dts dtc -O dts -i arch/arm/boot/dts foo.tmp Some other suggestions: Let's not make the documentation derived from schemata and DTS files tree-structured.Otherwise we end up with GNU info, ahem. Stephen Warren previously contributed a useful bindings checklist. We should try to roll that checklist into any validator. Benoit promises in devicetree/msg06617.html: > Being the very first one, you might get a free beer... meaning there > might be such thing as a free beer :-) I'm presenting a talk about device-tree for Embedded Linux Conference Europe, http://sched.co/17ozhPE, and hope that some of you come flame me in person there. If so, I will actually buy you a free beer, but not until after the second seminar I must give that afternoon! -- Alison Chaiken Mentor Embedded Software Division Fremont, CA GMT-8 -- To unsubscribe from this list: send the line "unsubscribe linux-omap" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [RFC 00/15] Device Tree schemas and validation
Hi David, On 02/10/2013 16:29, David Gibson wrote: On Tue, Oct 01, 2013 at 04:22:24PM -0600, Stephen Warren wrote: On 09/24/2013 10:52 AM, Benoit Cousson wrote: Hi All, Following the discussion that happened during LCE-2013 and the email thread started by Tomasz few months ago [1], here is a first attempt to introduce: - a schema language to define the bindings accurately - DTS validation during device tree compilation in DTC itself Sorry, this is probably going to sound a bit negative. Hopefully you find it constructive though. The syntax for a schema is the same as the one for dts. This choice has been made to simplify its development, to maximize the code reuse and finally because the format is human-readable. I'm not convinced that's a good decision. DT is a language for representing data. The validation checks described by schemas are rules, or code, and not static data. So, while I'm sure it's possible to shoe-horn at least some reasonable subset of DT validation into DT syntax itself, I feel it's unlikely to yield something that's scalable enough. I tend to agree. For example, it's easy to specify that a property must be 2 cells long. What if it could be any multiple of two? That's a lot of numbers to explicitly enumerate as data. Sure, you can then invent syntax to represent that specific rule (parameterized by 2), but what about the next similar-but-different rule? The only approach I can think of to that is to allow the schema to contain arbitrary expressions, which would likely need to morph into arbitary statements not just expressions. Once you're there, I think the schema would be better represented as a programming language rather than as a data structure that could have code hooked into it. How to: * Associate a schema to one or several nodes As said earlier a schema can be used to validate one or several nodes from a dts. To do this the "compatible" properties from the nodes which should be validated must be present in the schema. timer1: timer@4a318000 { compatible = "ti,omap3430-timer"; ... To write a schema which will validate OMAP Timers like the one above, one may write the following schema: /dts-v1/; / { compatible = "ti,omap[0-9]+-timer"; What about DT nodes that don't have a compatible value? We certainly have some of those already like /memory and /chosen. We should be able to validate their schema too. This probably doesn't invalidate being able to look things up by compatible value though; it just means we need some additional mechanisms too. More to the point, what about the properties of a node whose format is defined not by this node's binding but by some other nodes binding. e.g. the exact format of reg and ranges is at least partially determined by the parent bus's binding, and interrupts is defined partially by the interrupt parent's binding. gpio properties are defined by a combination of a global binding and the gpio parent, IIRC. Yeah, that's a general concern that Stephen raised several time as well. We need to figure out some way to handle that. * Define constraints on properties To define constraints on a property one has to create a node in a schema which has as name the name of the property that one want to validate. To specify constraints on the property "ti,hwmods" of OMAP Timers one can write this schema: /dts-v1/; / { compatible = "ti,omap[0-9]+-timer"; ti,hwmods { ... }; compatible and ti,hwmods are both properties in the DT file. However, in the schema above, one appears as a property, and one as a node. I don't like that inconsistency. It'd be better if compatible was a node too. Essentially what's going on here is that to describe the constraint on a property, a node with corresponding name is defined to encode the parameters of that constraint. It kind of works, but it's forced. It also hits problems since nodes and properties are technically in different namespaces, although they rarely collide in real cases. OK, so would you suggest keeping mapping between node / attribute in DTS and in the schema? If one want to use a regular as property name one can write this schema: /dts-v1/; / { compatible = "abc"; def { name = "def[0-9]"; Isn't it valid to have a property named "name" within the node itself? How do you differentiate between specifying the node name and the name property? Or to look at it another way, how do you differentiate between nodes representing encoded constraints for a property, and nodes representing nodes directly. What if the node name needs more validation than just a regex. For example, suppose we want to validate the unit-name-must-match-reg-address rule. We need to write some complex expression using data extracted from reg to calculate the unit address. Equally
Re: [RFC 00/15] Device Tree schemas and validation
Hi Stephen, On 02/10/2013 00:22, Stephen Warren wrote: On 09/24/2013 10:52 AM, Benoit Cousson wrote: Hi All, Following the discussion that happened during LCE-2013 and the email thread started by Tomasz few months ago [1], here is a first attempt to introduce: - a schema language to define the bindings accurately - DTS validation during device tree compilation in DTC itself Sorry, this is probably going to sound a bit negative. Hopefully you find it constructive though. Well, I hope so too, let's see at the end of the email :-) The syntax for a schema is the same as the one for dts. This choice has been made to simplify its development, to maximize the code reuse and finally because the format is human-readable. I'm not convinced that's a good decision. Me neither :-), but I gave the current rational... DT is a language for representing data. The validation checks described by schemas are rules, or code, and not static data. I will not be that strict with what DTS is supposed to do. In that aspect DTS is just a way to represent information in a structured hierarchical way. It is for my point of view no different than XML. I know that everybody hate XML, including me, but that shows at least what is doable with such language. The fact that we like it or not is a different topic. So, while I'm sure it's possible to shoe-horn at least some reasonable subset of DT validation into DT syntax itself, I feel it's unlikely to yield something that's scalable enough. I don't think we have any limit with such representation. My concern will be more the readability. To be honest, a language choice is by nature completely subjective, and nobody will have the same taste. So we can spend weeks arguing about that :-) For example, it's easy to specify that a property must be 2 cells long. What if it could be any multiple of two? That's a lot of numbers to explicitly enumerate as data. Sure, you can then invent syntax to represent that specific rule (parameterized by 2), but what about the next similar-but-different rule? The only approach I can think of to that is to allow the schema to contain arbitrary expressions, which would likely need to morph into arbitary statements not just expressions. Once you're there, I think the schema would be better represented as a programming language rather than as a data structure that could have code hooked into it. Sure, but how many complex cases like that do we have? I guess, we can handle all the use-cases required by Rob with the current syntax. Let's assume we cover 99% of the use-cases with such language, do we really want to have a super complex language just for the corner cases? Potentially, writing a C extension to DTC is still possible for that specific case. Not ideal, I do agree, but we have to be pragmatic as well. We really need to understand how scalable we have to be before deciding that the current representation is not good enough. How to: * Associate a schema to one or several nodes As said earlier a schema can be used to validate one or several nodes from a dts. To do this the "compatible" properties from the nodes which should be validated must be present in the schema. timer1: timer@4a318000 { compatible = "ti,omap3430-timer"; ... To write a schema which will validate OMAP Timers like the one above, one may write the following schema: /dts-v1/; / { compatible = "ti,omap[0-9]+-timer"; What about DT nodes that don't have a compatible value? We certainly have some of those already like /memory and /chosen. We should be able to validate their schema too. This probably doesn't invalidate being able to look things up by compatible value though; it just means we need some additional mechanisms too. Yes, that's a good point and easy to add as well. * Define constraints on properties To define constraints on a property one has to create a node in a schema which has as name the name of the property that one want to validate. To specify constraints on the property "ti,hwmods" of OMAP Timers one can write this schema: /dts-v1/; / { compatible = "ti,omap[0-9]+-timer"; ti,hwmods { ... }; compatible and ti,hwmods are both properties in the DT file. However, in the schema above, one appears as a property, and one as a node. I don't like that inconsistency. It'd be better if compatible was a node too. That's already possible, you can check the timer.schema. The point is to simplify the representation for simple case and use a attribute instead of a node. But that will make 2 different representation for the same case, which might not be that good. If one want to use a regular as property name one can write this schema: /dts-v1/; / { compatible = "abc"; def { name = "def[0-9]"; Isn't it val
Re: [RFC 00/15] Device Tree schemas and validation
On 02/10/2013 15:54, David Gibson wrote: On Tue, Oct 01, 2013 at 03:54:20PM -0500, Rob Herring wrote: On Tue, Oct 1, 2013 at 10:06 AM, Benoit Cousson wrote: Hi Rob, On 01/10/2013 15:17, Rob Herring wrote: On 10/01/2013 03:06 AM, Benoit Cousson wrote: + more DT maintainers folks Hi all, I know this is mostly boring user space code, but I was expecting a little bit of comments about at least the bindings syntax:-( I'd like to know if this is the right direction and if it worth pursuing in that direction. The idea was to have at least some base for further discussion during ARM KS 2013. I feel alone :-( If you have any comment, go ahead! Thanks for taking this on! This is interesting approach using the dts syntax, Well, this was discussed a little bit on the list, and it has the big advantage of re-using the parser already included in DTC for free. In term or readability, it avoids to re-defining a brand new syntax for people who are already familiar with the DTS one. but I worry that the validation will only be as good as the schema written and the review of the schema. Well, sure, but unfortunately, that will always be the case :-( The bindings definition being quite open, there is no easy way to ensure proper schema / bindings without careful review of the schema. There is no such thing as a free beer... Unfortunately :-) I think the schema needs to define the binding rather than define the checks. Then the schema can feed the validation checks. This format does not seem to me as easily being able to generate documentation from the schema which I believe is one of the goals. Indeed, but I think is it easy to generate any kind of readable format for the documentation purpose if needed from the actual format. Otherwise, we should consider a schema format based on kerneldoc type of syntax to improve readability. I'm just afraid it will become harder then to define complex schema. BTW, what kind of documentation are you expecting here? Is is a text that can be added on top of each schema? I would expect the schema to replace Documentation/devicetree/bindings/* over time. I think the thing that needs to be worked out here is how to add free form multi-line text. I'm not convinced that's a realistic goal. As I see it, the fundamental difference between a binding document and a formal schema is that a binding defines both the syntax required of a node, and its semantics, whereas a schema defines only syntax - the semantics still need to be defined somewhere. Mmm, I do not really understand what is missing in the following schema in term of semantic compared to the original interrupt-controller bindings? interrupt-controller { description = "Identifies the node as an interrupt controller"; is-required; type = "bool"; }; #interrupt-cells { description = "Specifies the number of cells needed to encode an interrupt source. The type shall be a and the value shall be 1. The cell contains the interrupt number in the range [0-128]."; is-required; type = "integer"; value = <1>; }; As soon as you have a description you can keep the content of the original binding. But on top of that, you do have the schema validation information. I'm not sure to understand your point. Could you elaborate? Regards, Benoit -- To unsubscribe from this list: send the line "unsubscribe linux-omap" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [RFC 00/15] Device Tree schemas and validation
On Wed, Oct 02, 2013 at 07:08:41PM +0100, Mark Brown wrote: > On Wed, Oct 02, 2013 at 11:54:50PM +1000, David Gibson wrote: > > On Tue, Oct 01, 2013 at 03:54:20PM -0500, Rob Herring wrote: > > > > I would expect the schema to replace > > > Documentation/devicetree/bindings/* over time. I think the thing that > > > needs to be worked out here is how to add free form multi-line text. > > > I'm not convinced that's a realistic goal. As I see it, the > > fundamental difference between a binding document and a formal schema > > is that a binding defines both the syntax required of a node, and its > > semantics, whereas a schema defines only syntax - the semantics still > > need to be defined somewhere. > > So long as the schema lets you include free form text to define the > semantics I'm not sure there's an incompatibility there - the same > document can cover both. True, there's no reason the machine-readable schema and human-readable documentation can't be contained in the same file. -- David Gibson| I'll have my music baroque, and my code david AT gibson.dropbear.id.au | minimalist, thank you. NOT _the_ _other_ | _way_ _around_! http://www.ozlabs.org/~dgibson pgpv8VA08IhkS.pgp Description: PGP signature
Re: [RFC 00/15] Device Tree schemas and validation
On Wed, Oct 02, 2013 at 11:54:50PM +1000, David Gibson wrote: > On Tue, Oct 01, 2013 at 03:54:20PM -0500, Rob Herring wrote: > > I would expect the schema to replace > > Documentation/devicetree/bindings/* over time. I think the thing that > > needs to be worked out here is how to add free form multi-line text. > I'm not convinced that's a realistic goal. As I see it, the > fundamental difference between a binding document and a formal schema > is that a binding defines both the syntax required of a node, and its > semantics, whereas a schema defines only syntax - the semantics still > need to be defined somewhere. So long as the schema lets you include free form text to define the semantics I'm not sure there's an incompatibility there - the same document can cover both. signature.asc Description: Digital signature
Re: [RFC 00/15] Device Tree schemas and validation
On Tue, Oct 01, 2013 at 04:22:24PM -0600, Stephen Warren wrote: > On 09/24/2013 10:52 AM, Benoit Cousson wrote: > > Hi All, > > > > Following the discussion that happened during LCE-2013 and the email > > thread started by Tomasz few months ago [1], here is a first attempt > > to introduce: > > - a schema language to define the bindings accurately > > - DTS validation during device tree compilation in DTC itself > > Sorry, this is probably going to sound a bit negative. Hopefully you > find it constructive though. > > > The syntax for a schema is the same as the one for dts. This choice has > > been made to simplify its development, to maximize the code reuse and > > finally because the format is human-readable. > > I'm not convinced that's a good decision. > > DT is a language for representing data. > > The validation checks described by schemas are rules, or code, and not > static data. > > So, while I'm sure it's possible to shoe-horn at least some reasonable > subset of DT validation into DT syntax itself, I feel it's unlikely to > yield something that's scalable enough. I tend to agree. > For example, it's easy to specify that a property must be 2 cells long. > What if it could be any multiple of two? That's a lot of numbers to > explicitly enumerate as data. Sure, you can then invent syntax to > represent that specific rule (parameterized by 2), but what about the > next similar-but-different rule? The only approach I can think of to > that is to allow the schema to contain arbitrary expressions, which > would likely need to morph into arbitary statements not just > expressions. Once you're there, I think the schema would be better > represented as a programming language rather than as a data structure > that could have code hooked into it. > > > How to: > > * Associate a schema to one or several nodes > > > > As said earlier a schema can be used to validate one or several nodes > > from a dts. To do this the "compatible" properties from the nodes which > > should be validated must be present in the schema. > > > > timer1: timer@4a318000 { > > compatible = "ti,omap3430-timer"; > ... > > To write a schema which will validate OMAP Timers like the one above, > > one may write the following schema: > > > > /dts-v1/; > > / { > > compatible = "ti,omap[0-9]+-timer"; > > What about DT nodes that don't have a compatible value? We certainly > have some of those already like /memory and /chosen. We should be able > to validate their schema too. This probably doesn't invalidate being > able to look things up by compatible value though; it just means we need > some additional mechanisms too. More to the point, what about the properties of a node whose format is defined not by this node's binding but by some other nodes binding. e.g. the exact format of reg and ranges is at least partially determined by the parent bus's binding, and interrupts is defined partially by the interrupt parent's binding. gpio properties are defined by a combination of a global binding and the gpio parent, IIRC. > > * Define constraints on properties > > > > To define constraints on a property one has to create a node in a schema > > which has as name the name of the property that one want to validate. > > > > To specify constraints on the property "ti,hwmods" of OMAP Timers one > > can write this schema: > > > > /dts-v1/; > > / { > > compatible = "ti,omap[0-9]+-timer"; > > ti,hwmods { > > ... > > }; > > compatible and ti,hwmods are both properties in the DT file. However, in > the schema above, one appears as a property, and one as a node. I don't > like that inconsistency. It'd be better if compatible was a node too. Essentially what's going on here is that to describe the constraint on a property, a node with corresponding name is defined to encode the parameters of that constraint. It kind of works, but it's forced. It also hits problems since nodes and properties are technically in different namespaces, although they rarely collide in real cases. > > If one want to use a regular as property name one can write this schema: > > > > /dts-v1/; > > / { > > compatible = "abc"; > > def { > > name = "def[0-9]"; > > Isn't it valid to have a property named "name" within the node itself? > How do you differentiate between specifying the node name and the name > property? Or to look at it another way, how do you differentiate between nodes representing encoded constraints for a property, and nodes representing nodes directly. > What if the node name needs more validation than just a regex. For > example, suppose we want to validate the > unit-name-must-match-reg-address rule. We need to write some complex > expression using data extracted from reg to calculate the unit address. > Equally, the node name perhaps has to exist in some global list of > acceptable node n
Re: [RFC 00/15] Device Tree schemas and validation
On Tue, Oct 01, 2013 at 03:54:20PM -0500, Rob Herring wrote: > On Tue, Oct 1, 2013 at 10:06 AM, Benoit Cousson wrote: > > Hi Rob, > > > > > > On 01/10/2013 15:17, Rob Herring wrote: > >> > >> On 10/01/2013 03:06 AM, Benoit Cousson wrote: > >>> > >>> + more DT maintainers folks > >>> > >>> Hi all, > >>> > >>> I know this is mostly boring user space code, but I was expecting a > >>> little bit of comments about at least the bindings syntax:-( > >>> > >>> I'd like to know if this is the right direction and if it worth pursuing > >>> in that direction. > >>> > >>> The idea was to have at least some base for further discussion during > >>> ARM KS 2013. > >>> > >>> I feel alone :-( > >>> > >>> If you have any comment, go ahead! > >> > >> > >> Thanks for taking this on! > >> > >> This is interesting approach using the dts syntax, > > > > > > Well, this was discussed a little bit on the list, and it has the big > > advantage of re-using the parser already included in DTC for free. > > In term or readability, it avoids to re-defining a brand new syntax for > > people who are already familiar with the DTS one. > > > > > >> but I worry that the > >> validation will only be as good as the schema written and the review of > >> the schema. > > > > > > Well, sure, but unfortunately, that will always be the case :-( > > The bindings definition being quite open, there is no easy way to ensure > > proper schema / bindings without careful review of the schema. There is no > > such thing as a free beer... Unfortunately :-) > > > > > >> I think the schema needs to define the binding rather than > >> define the checks. Then the schema can feed the validation checks. > > > > > >> This format does not seem to me as easily being able to generate > >> documentation from the schema which I believe is one of the goals. > > > > > > Indeed, but I think is it easy to generate any kind of readable format for > > the documentation purpose if needed from the actual format. > > Otherwise, we should consider a schema format based on kerneldoc type of > > syntax to improve readability. I'm just afraid it will become harder then to > > define complex schema. > > > > BTW, what kind of documentation are you expecting here? Is is a text that > > can be added on top of each schema? > > I would expect the schema to replace > Documentation/devicetree/bindings/* over time. I think the thing that > needs to be worked out here is how to add free form multi-line text. I'm not convinced that's a realistic goal. As I see it, the fundamental difference between a binding document and a formal schema is that a binding defines both the syntax required of a node, and its semantics, whereas a schema defines only syntax - the semantics still need to be defined somewhere. -- David Gibson| I'll have my music baroque, and my code david AT gibson.dropbear.id.au | minimalist, thank you. NOT _the_ _other_ | _way_ _around_! http://www.ozlabs.org/~dgibson pgp0yD2z4Q1JA.pgp Description: PGP signature
Re: [RFC 00/15] Device Tree schemas and validation
On Tue, Oct 01, 2013 at 08:17:42AM -0500, Rob Herring wrote: > On 10/01/2013 03:06 AM, Benoit Cousson wrote: > > + more DT maintainers folks > > > > Hi all, > > > > I know this is mostly boring user space code, but I was expecting a > > little bit of comments about at least the bindings syntax:-( > > > > I'd like to know if this is the right direction and if it worth pursuing > > in that direction. > > > > The idea was to have at least some base for further discussion during > > ARM KS 2013. > > > > I feel alone :-( > > > > If you have any comment, go ahead! > > Thanks for taking this on! > > This is interesting approach using the dts syntax, but I worry that the > validation will only be as good as the schema written and the review of > the schema. I think the schema needs to define the binding rather than > define the checks. Then the schema can feed the validation checks. This > format does not seem to me as easily being able to generate > documentation from the schema which I believe is one of the goals. I for > one don't care to review the documentation and the schema for every binding. Hrm. I'm less optimistic about entirely replacing human-readable bindings with machine-readable schemas. But I do think the schema language needs to be substantially more flexible than the draft presented here. While I think a schema syntax which mirrors dts syntax makes a lot of sense, actually defining schemas as "device" trees doesn't seem quite right to me. -- David Gibson| I'll have my music baroque, and my code david AT gibson.dropbear.id.au | minimalist, thank you. NOT _the_ _other_ | _way_ _around_! http://www.ozlabs.org/~dgibson pgps4GNtO_6_u.pgp Description: PGP signature
Re: [RFC 00/15] Device Tree schemas and validation
> > > > Benoit, > > > > Sorry, I meant to ask earlier but forgot. > > Shouldn't this development be based on the > > upstream DTC repository and not the in-kernel > > copy of the DTC? > > Eventually, yes, We should *start* there, though. > but here the main point is to discuss the schema that > will be used to defined bindings. > In that case, the DTC patches code are mostly a proof of concept Sure; all that is fine. > using the Linux kernel as example. Example?(*) We shouldn't churn the kernel. jdl (*) In the sense that *this* Universe is just one example from all known Universes? :-) -- To unsubscribe from this list: send the line "unsubscribe linux-omap" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [RFC 00/15] Device Tree schemas and validation
Hi Jon, On 01/10/2013 17:17, Jon Loeliger wrote: Hi Rob, On 01/10/2013 15:17, Rob Herring wrote: On 10/01/2013 03:06 AM, Benoit Cousson wrote: + more DT maintainers folks Hi all, I know this is mostly boring user space code, but I was expecting a little bit of comments about at least the bindings syntax:-( I'd like to know if this is the right direction and if it worth pursuing in that direction. The idea was to have at least some base for further discussion during ARM KS 2013. I feel alone :-( If you have any comment, go ahead! Benoit, Sorry, I meant to ask earlier but forgot. Shouldn't this development be based on the upstream DTC repository and not the in-kernel copy of the DTC? Eventually, yes, but here the main point is to discuss the schema that will be used to defined bindings. In that case, the DTC patches code are mostly a proof of concept using the Linux kernel as example. Regards, Benoit -- To unsubscribe from this list: send the line "unsubscribe linux-omap" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [RFC 00/15] Device Tree schemas and validation
On Tue, Oct 01, 2013 at 10:17:53AM -0500, Jon Loeliger wrote: > > Hi Rob, > > > > On 01/10/2013 15:17, Rob Herring wrote: > > > On 10/01/2013 03:06 AM, Benoit Cousson wrote: > > >> + more DT maintainers folks > > >> > > >> Hi all, > > >> > > >> I know this is mostly boring user space code, but I was expecting a > > >> little bit of comments about at least the bindings syntax:-( > > >> > > >> I'd like to know if this is the right direction and if it worth pursuing > > >> in that direction. > > >> > > >> The idea was to have at least some base for further discussion during > > >> ARM KS 2013. > > >> > > >> I feel alone :-( > > >> > > >> If you have any comment, go ahead! > > > Benoit, > > Sorry, I meant to ask earlier but forgot. > Shouldn't this development be based on the > upstream DTC repository and not the in-kernel > copy of the DTC? Absolutely. Please work against upstream dtc. -- David Gibson| I'll have my music baroque, and my code david AT gibson.dropbear.id.au | minimalist, thank you. NOT _the_ _other_ | _way_ _around_! http://www.ozlabs.org/~dgibson pgpvQSOHC9Pwv.pgp Description: PGP signature
Re: [RFC 00/15] Device Tree schemas and validation
On 09/24/2013 10:52 AM, Benoit Cousson wrote: > Hi All, > > Following the discussion that happened during LCE-2013 and the email > thread started by Tomasz few months ago [1], here is a first attempt > to introduce: > - a schema language to define the bindings accurately > - DTS validation during device tree compilation in DTC itself Sorry, this is probably going to sound a bit negative. Hopefully you find it constructive though. > The syntax for a schema is the same as the one for dts. This choice has > been made to simplify its development, to maximize the code reuse and > finally because the format is human-readable. I'm not convinced that's a good decision. DT is a language for representing data. The validation checks described by schemas are rules, or code, and not static data. So, while I'm sure it's possible to shoe-horn at least some reasonable subset of DT validation into DT syntax itself, I feel it's unlikely to yield something that's scalable enough. For example, it's easy to specify that a property must be 2 cells long. What if it could be any multiple of two? That's a lot of numbers to explicitly enumerate as data. Sure, you can then invent syntax to represent that specific rule (parameterized by 2), but what about the next similar-but-different rule? The only approach I can think of to that is to allow the schema to contain arbitrary expressions, which would likely need to morph into arbitary statements not just expressions. Once you're there, I think the schema would be better represented as a programming language rather than as a data structure that could have code hooked into it. > How to: > * Associate a schema to one or several nodes > > As said earlier a schema can be used to validate one or several nodes > from a dts. To do this the "compatible" properties from the nodes which > should be validated must be present in the schema. > > timer1: timer@4a318000 { > compatible = "ti,omap3430-timer"; ... > To write a schema which will validate OMAP Timers like the one above, > one may write the following schema: > > /dts-v1/; > / { > compatible = "ti,omap[0-9]+-timer"; What about DT nodes that don't have a compatible value? We certainly have some of those already like /memory and /chosen. We should be able to validate their schema too. This probably doesn't invalidate being able to look things up by compatible value though; it just means we need some additional mechanisms too. > * Define constraints on properties > > To define constraints on a property one has to create a node in a schema > which has as name the name of the property that one want to validate. > > To specify constraints on the property "ti,hwmods" of OMAP Timers one > can write this schema: > > /dts-v1/; > / { > compatible = "ti,omap[0-9]+-timer"; > ti,hwmods { > ... > }; compatible and ti,hwmods are both properties in the DT file. However, in the schema above, one appears as a property, and one as a node. I don't like that inconsistency. It'd be better if compatible was a node too. > If one want to use a regular as property name one can write this schema: > > /dts-v1/; > / { > compatible = "abc"; > def { > name = "def[0-9]"; Isn't it valid to have a property named "name" within the node itself? How do you differentiate between specifying the node name and the name property? What if the node name needs more validation than just a regex. For example, suppose we want to validate the unit-name-must-match-reg-address rule. We need to write some complex expression using data extracted from reg to calculate the unit address. Equally, the node name perhaps has to exist in some global list of acceptable node names. It would be extremely tricky if not impossible to do that with a regex. > ... > }; > }; > > Above one can see that the "name" property override the node name. Override implies that dtc would change the node name during compilation. I think s/override/validate/ or s/override/overrides the validation rules for/? > * Require the presence of a property inside a node or inside one of its > parents ... > /dts-v1/; > / { > compatible = "ti,twl[0-9]+-rtc"; > interrupt-controller { > is-required; > can-be-inherited; interrupt-controller isn't a good example here, since it isn't a property that would typically be inherited. Why not use interrupt-parent instead? > One can check if 'node' has the following subnode 'subnode1', 'subnode2', > and 'abc' with the schema below: > > /dts-v1/; > / { > compatible = "comp"; > children = "abc", "subnode[0-9]"; > }; How is the schema for each sub-node specified? What if some nodes are optional and some required? The conditions where a sub-node is required might be complex, and I think we'd always want to be able
Re: [RFC 00/15] Device Tree schemas and validation
On Tue, Oct 1, 2013 at 10:06 AM, Benoit Cousson wrote: > Hi Rob, > > > On 01/10/2013 15:17, Rob Herring wrote: >> >> On 10/01/2013 03:06 AM, Benoit Cousson wrote: >>> >>> + more DT maintainers folks >>> >>> Hi all, >>> >>> I know this is mostly boring user space code, but I was expecting a >>> little bit of comments about at least the bindings syntax:-( >>> >>> I'd like to know if this is the right direction and if it worth pursuing >>> in that direction. >>> >>> The idea was to have at least some base for further discussion during >>> ARM KS 2013. >>> >>> I feel alone :-( >>> >>> If you have any comment, go ahead! >> >> >> Thanks for taking this on! >> >> This is interesting approach using the dts syntax, > > > Well, this was discussed a little bit on the list, and it has the big > advantage of re-using the parser already included in DTC for free. > In term or readability, it avoids to re-defining a brand new syntax for > people who are already familiar with the DTS one. > > >> but I worry that the >> validation will only be as good as the schema written and the review of >> the schema. > > > Well, sure, but unfortunately, that will always be the case :-( > The bindings definition being quite open, there is no easy way to ensure > proper schema / bindings without careful review of the schema. There is no > such thing as a free beer... Unfortunately :-) > > >> I think the schema needs to define the binding rather than >> define the checks. Then the schema can feed the validation checks. > > >> This format does not seem to me as easily being able to generate >> documentation from the schema which I believe is one of the goals. > > > Indeed, but I think is it easy to generate any kind of readable format for > the documentation purpose if needed from the actual format. > Otherwise, we should consider a schema format based on kerneldoc type of > syntax to improve readability. I'm just afraid it will become harder then to > define complex schema. > > BTW, what kind of documentation are you expecting here? Is is a text that > can be added on top of each schema? I would expect the schema to replace Documentation/devicetree/bindings/* over time. I think the thing that needs to be worked out here is how to add free form multi-line text. >> I for >> one don't care to review the documentation and the schema for every >> binding. >> >> I would like to ensure we can start with the basics and have some >> generic rules. Some examples: >> >> - flag compatible strings in dts files that are not documented > > > This is almost done with the last patch of the series. Nodes are already > checked, properties are missing. > >> - check required properties > > > Done as well. > > >> - check properties against cell sizes > > > Yeah, that one is still to be done. > > >> - a node with reg property must have a unit address > > > Easy to add. > > >> - flag properties with "_" > > > Easy to add as well. > > >> - check properties are the correct data type > > > Already done for a couple of data type. Good to know. It wasn't really clear what common items are checked. An example schema for clocks or gpio (or any other common) binding would be nice. Do you have a run with all the warnings found? > Thanks for your comments. Being the very first one, you might get a free > beer... meaning there might be such thing as a free beer :-) In that case, it all looks perfect. :) Rob -- To unsubscribe from this list: send the line "unsubscribe linux-omap" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [RFC 00/15] Device Tree schemas and validation
> Hi Rob, > > On 01/10/2013 15:17, Rob Herring wrote: > > On 10/01/2013 03:06 AM, Benoit Cousson wrote: > >> + more DT maintainers folks > >> > >> Hi all, > >> > >> I know this is mostly boring user space code, but I was expecting a > >> little bit of comments about at least the bindings syntax:-( > >> > >> I'd like to know if this is the right direction and if it worth pursuing > >> in that direction. > >> > >> The idea was to have at least some base for further discussion during > >> ARM KS 2013. > >> > >> I feel alone :-( > >> > >> If you have any comment, go ahead! Benoit, Sorry, I meant to ask earlier but forgot. Shouldn't this development be based on the upstream DTC repository and not the in-kernel copy of the DTC? Thanks, jdl -- To unsubscribe from this list: send the line "unsubscribe linux-omap" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [RFC 00/15] Device Tree schemas and validation
Hi Rob, On 01/10/2013 15:17, Rob Herring wrote: On 10/01/2013 03:06 AM, Benoit Cousson wrote: + more DT maintainers folks Hi all, I know this is mostly boring user space code, but I was expecting a little bit of comments about at least the bindings syntax:-( I'd like to know if this is the right direction and if it worth pursuing in that direction. The idea was to have at least some base for further discussion during ARM KS 2013. I feel alone :-( If you have any comment, go ahead! Thanks for taking this on! This is interesting approach using the dts syntax, Well, this was discussed a little bit on the list, and it has the big advantage of re-using the parser already included in DTC for free. In term or readability, it avoids to re-defining a brand new syntax for people who are already familiar with the DTS one. but I worry that the validation will only be as good as the schema written and the review of the schema. Well, sure, but unfortunately, that will always be the case :-( The bindings definition being quite open, there is no easy way to ensure proper schema / bindings without careful review of the schema. There is no such thing as a free beer... Unfortunately :-) I think the schema needs to define the binding rather than define the checks. Then the schema can feed the validation checks. This format does not seem to me as easily being able to generate documentation from the schema which I believe is one of the goals. Indeed, but I think is it easy to generate any kind of readable format for the documentation purpose if needed from the actual format. Otherwise, we should consider a schema format based on kerneldoc type of syntax to improve readability. I'm just afraid it will become harder then to define complex schema. BTW, what kind of documentation are you expecting here? Is is a text that can be added on top of each schema? I for one don't care to review the documentation and the schema for every binding. I would like to ensure we can start with the basics and have some generic rules. Some examples: - flag compatible strings in dts files that are not documented This is almost done with the last patch of the series. Nodes are already checked, properties are missing. - check required properties Done as well. - check properties against cell sizes Yeah, that one is still to be done. - a node with reg property must have a unit address Easy to add. - flag properties with "_" Easy to add as well. - check properties are the correct data type Already done for a couple of data type. Thanks for your comments. Being the very first one, you might get a free beer... meaning there might be such thing as a free beer :-) Thanks, Benoit Thanks, Benoit On 24/09/2013 18:52, Benoit Cousson wrote: Hi All, Following the discussion that happened during LCE-2013 and the email thread started by Tomasz few months ago [1], here is a first attempt to introduce: - a schema language to define the bindings accurately - DTS validation during device tree compilation in DTC itself [1] http://www.spinics.net/lists/arm-kernel/msg262224.html === What it does? === For now device-tree bindings are defined in a not standardized text-based format and thus any one can write the documentation for a binding as he wish. In addition to this there is no automated way to check if a dts is conform to the available bindings. The goal of this series of patch is to fix this situation by adding a well defined way to write bindings in a human-readable format. These bindings will be written through files called "schemas". === What is a schema? === A schema is a file describing the constraints that one or several nodes of a dts must conform to. Schemas must use the file extension ".schema". A schema can check that: - A node has a given property - An array has a valid size - A node contains the required children. - A property has the correct type. - A property has the correct value. A schema can as well recursively check the constraints for parent nodes. The syntax for a schema is the same as the one for dts. This choice has been made to simplify its development, to maximize the code reuse and finally because the format is human-readable. === How to defined a schema? === A binding directory has been added at the root of repository. Users can add schema files anywhere in it but a nice way would be to keep the same structure as the binding directory in the documentation. To demonstrate how to write a schema and its capability I will use the OMAP DTS schemas when applicable. How to: * Associate a schema to one or several nodes As said earlier a schema can be used to validate one or several nodes from a dts. To do this the "compatible" properties from the nodes which should be validated must be present in the schema. timer1: timer@4a318000 { compatible = "ti,omap3430-timer"; reg = <0x4a318000 0x80>; interrupt
Re: [RFC 00/15] Device Tree schemas and validation
On 10/01/2013 03:06 AM, Benoit Cousson wrote: > + more DT maintainers folks > > Hi all, > > I know this is mostly boring user space code, but I was expecting a > little bit of comments about at least the bindings syntax:-( > > I'd like to know if this is the right direction and if it worth pursuing > in that direction. > > The idea was to have at least some base for further discussion during > ARM KS 2013. > > I feel alone :-( > > If you have any comment, go ahead! Thanks for taking this on! This is interesting approach using the dts syntax, but I worry that the validation will only be as good as the schema written and the review of the schema. I think the schema needs to define the binding rather than define the checks. Then the schema can feed the validation checks. This format does not seem to me as easily being able to generate documentation from the schema which I believe is one of the goals. I for one don't care to review the documentation and the schema for every binding. I would like to ensure we can start with the basics and have some generic rules. Some examples: - flag compatible strings in dts files that are not documented - check required properties - check properties against cell sizes - a node with reg property must have a unit address - flag properties with "_" - check properties are the correct data type Rob > > Thanks, > Benoit > > On 24/09/2013 18:52, Benoit Cousson wrote: >> Hi All, >> >> Following the discussion that happened during LCE-2013 and the email >> thread started by Tomasz few months ago [1], here is a first attempt >> to introduce: >> - a schema language to define the bindings accurately >> - DTS validation during device tree compilation in DTC itself >> >> [1] http://www.spinics.net/lists/arm-kernel/msg262224.html >> >> >> === What it does? === >> >> For now device-tree bindings are defined in a not standardized >> text-based format and thus any one can write the documentation for a >> binding as he wish. In addition to this there is no automated way to >> check if a dts is conform to the available bindings. >> >> The goal of this series of patch is to fix this situation by adding a >> well defined way to write bindings in a human-readable format. These >> bindings will be written through files called "schemas". >> >> >> === What is a schema? === >> >> A schema is a file describing the constraints that one or several nodes >> of a dts must conform to. Schemas must use the file extension ".schema". >> A schema can check that: >> - A node has a given property >> - An array has a valid size >> - A node contains the required children. >> - A property has the correct type. >> - A property has the correct value. >> >> A schema can as well recursively check the constraints for parent nodes. >> >> The syntax for a schema is the same as the one for dts. This choice has >> been made to simplify its development, to maximize the code reuse and >> finally because the format is human-readable. >> >> >> === How to defined a schema? === >> >> A binding directory has been added at the root of repository. Users can >> add schema files anywhere in it but a nice way would be to keep the same >> structure as the binding directory in the documentation. >> >> To demonstrate how to write a schema and its capability I will use the >> OMAP DTS schemas when applicable. >> >> How to: >> * Associate a schema to one or several nodes >> >> As said earlier a schema can be used to validate one or several nodes >> from a dts. To do this the "compatible" properties from the nodes which >> should be validated must be present in the schema. >> >> timer1: timer@4a318000 { >> compatible = "ti,omap3430-timer"; >> reg = <0x4a318000 0x80>; >> interrupts = <0x0 0x25 0x4>; >> ti,hwmods = "timer1"; >> ti,timer-alwon; >> }; >> >> To write a schema which will validate OMAP Timers like the one above, >> one may write the following schema: >> >> /dts-v1/; >> / { >> compatible = "ti,omap[0-9]+-timer"; >> ... >> }; >> >> The schema above will be used to validate every node in a dts which has >> a compatible matching the following regular expression: >> "ti,omap[0-9]+-timer". >> >> It is possible to specify multiple compatible inside a schema using this >> syntax: >> compatible = "ti,omap[0-9]+-timer", "ti,am[0-9]+-timer"; >> >> This time the schema will be application for both OMAP Timers and AM >> Timers. >> >> >> * Define constraints on properties >> >> To define constraints on a property one has to create a node in a schema >> which has as name the name of the property that one want to validate. >> >> To specify constraints on the property "ti,hwmods" of OMAP Timers one >> can write this schema: >> >> /dts-v1/; >> / { >> compatible = "ti,omap[0-9]+-timer"; >> ti,hwmods { >> ... >> }; >> }; >> >> If one want to use a regular as property name one can write thi
Re: [RFC 00/15] Device Tree schemas and validation
+ more DT maintainers folks Hi all, I know this is mostly boring user space code, but I was expecting a little bit of comments about at least the bindings syntax:-( I'd like to know if this is the right direction and if it worth pursuing in that direction. The idea was to have at least some base for further discussion during ARM KS 2013. I feel alone :-( If you have any comment, go ahead! Thanks, Benoit On 24/09/2013 18:52, Benoit Cousson wrote: Hi All, Following the discussion that happened during LCE-2013 and the email thread started by Tomasz few months ago [1], here is a first attempt to introduce: - a schema language to define the bindings accurately - DTS validation during device tree compilation in DTC itself [1] http://www.spinics.net/lists/arm-kernel/msg262224.html === What it does? === For now device-tree bindings are defined in a not standardized text-based format and thus any one can write the documentation for a binding as he wish. In addition to this there is no automated way to check if a dts is conform to the available bindings. The goal of this series of patch is to fix this situation by adding a well defined way to write bindings in a human-readable format. These bindings will be written through files called "schemas". === What is a schema? === A schema is a file describing the constraints that one or several nodes of a dts must conform to. Schemas must use the file extension ".schema". A schema can check that: - A node has a given property - An array has a valid size - A node contains the required children. - A property has the correct type. - A property has the correct value. A schema can as well recursively check the constraints for parent nodes. The syntax for a schema is the same as the one for dts. This choice has been made to simplify its development, to maximize the code reuse and finally because the format is human-readable. === How to defined a schema? === A binding directory has been added at the root of repository. Users can add schema files anywhere in it but a nice way would be to keep the same structure as the binding directory in the documentation. To demonstrate how to write a schema and its capability I will use the OMAP DTS schemas when applicable. How to: * Associate a schema to one or several nodes As said earlier a schema can be used to validate one or several nodes from a dts. To do this the "compatible" properties from the nodes which should be validated must be present in the schema. timer1: timer@4a318000 { compatible = "ti,omap3430-timer"; reg = <0x4a318000 0x80>; interrupts = <0x0 0x25 0x4>; ti,hwmods = "timer1"; ti,timer-alwon; }; To write a schema which will validate OMAP Timers like the one above, one may write the following schema: /dts-v1/; / { compatible = "ti,omap[0-9]+-timer"; ... }; The schema above will be used to validate every node in a dts which has a compatible matching the following regular expression: "ti,omap[0-9]+-timer". It is possible to specify multiple compatible inside a schema using this syntax: compatible = "ti,omap[0-9]+-timer", "ti,am[0-9]+-timer"; This time the schema will be application for both OMAP Timers and AM Timers. * Define constraints on properties To define constraints on a property one has to create a node in a schema which has as name the name of the property that one want to validate. To specify constraints on the property "ti,hwmods" of OMAP Timers one can write this schema: /dts-v1/; / { compatible = "ti,omap[0-9]+-timer"; ti,hwmods { ... }; }; If one want to use a regular as property name one can write this schema: /dts-v1/; / { compatible = "abc"; def { name = "def[0-9]"; ... }; }; Above one can see that the "name" property override the node name. * Require the presence of a property One can require the presence of a property by using the "is-required" constraint. /dts-v1/; / { compatible = "ti,omap[0-9]+-timer"; ti,hwmods { is-required; }; }; The "ti,hwmods" property above is set as required and its presence will be checked in every OMAP timer. * Require the presence of a property inside a node or inside one of its parents Sometimes a property required is not directly present inside a node but is present in one of its parents. To check this, one can use "can-be-inherited" in addition to "is-required". twl { interrupt-controller; rtc { compatible = "ti,twl4030-rtc"; interrupts = <0xc>; }; } In the case of the rtc abov
[RFC 00/15] Device Tree schemas and validation
Hi All, Following the discussion that happened during LCE-2013 and the email thread started by Tomasz few months ago [1], here is a first attempt to introduce: - a schema language to define the bindings accurately - DTS validation during device tree compilation in DTC itself [1] http://www.spinics.net/lists/arm-kernel/msg262224.html === What it does? === For now device-tree bindings are defined in a not standardized text-based format and thus any one can write the documentation for a binding as he wish. In addition to this there is no automated way to check if a dts is conform to the available bindings. The goal of this series of patch is to fix this situation by adding a well defined way to write bindings in a human-readable format. These bindings will be written through files called "schemas". === What is a schema? === A schema is a file describing the constraints that one or several nodes of a dts must conform to. Schemas must use the file extension ".schema". A schema can check that: - A node has a given property - An array has a valid size - A node contains the required children. - A property has the correct type. - A property has the correct value. A schema can as well recursively check the constraints for parent nodes. The syntax for a schema is the same as the one for dts. This choice has been made to simplify its development, to maximize the code reuse and finally because the format is human-readable. === How to defined a schema? === A binding directory has been added at the root of repository. Users can add schema files anywhere in it but a nice way would be to keep the same structure as the binding directory in the documentation. To demonstrate how to write a schema and its capability I will use the OMAP DTS schemas when applicable. How to: * Associate a schema to one or several nodes As said earlier a schema can be used to validate one or several nodes from a dts. To do this the "compatible" properties from the nodes which should be validated must be present in the schema. timer1: timer@4a318000 { compatible = "ti,omap3430-timer"; reg = <0x4a318000 0x80>; interrupts = <0x0 0x25 0x4>; ti,hwmods = "timer1"; ti,timer-alwon; }; To write a schema which will validate OMAP Timers like the one above, one may write the following schema: /dts-v1/; / { compatible = "ti,omap[0-9]+-timer"; ... }; The schema above will be used to validate every node in a dts which has a compatible matching the following regular expression: "ti,omap[0-9]+-timer". It is possible to specify multiple compatible inside a schema using this syntax: compatible = "ti,omap[0-9]+-timer", "ti,am[0-9]+-timer"; This time the schema will be application for both OMAP Timers and AM Timers. * Define constraints on properties To define constraints on a property one has to create a node in a schema which has as name the name of the property that one want to validate. To specify constraints on the property "ti,hwmods" of OMAP Timers one can write this schema: /dts-v1/; / { compatible = "ti,omap[0-9]+-timer"; ti,hwmods { ... }; }; If one want to use a regular as property name one can write this schema: /dts-v1/; / { compatible = "abc"; def { name = "def[0-9]"; ... }; }; Above one can see that the "name" property override the node name. * Require the presence of a property One can require the presence of a property by using the "is-required" constraint. /dts-v1/; / { compatible = "ti,omap[0-9]+-timer"; ti,hwmods { is-required; }; }; The "ti,hwmods" property above is set as required and its presence will be checked in every OMAP timer. * Require the presence of a property inside a node or inside one of its parents Sometimes a property required is not directly present inside a node but is present in one of its parents. To check this, one can use "can-be-inherited" in addition to "is-required". twl { interrupt-controller; rtc { compatible = "ti,twl4030-rtc"; interrupts = <0xc>; }; } In the case of the rtc above the interrupt-controller is not present, but it is present in its parent. If inheriting the property from the parent makes senses like here one can specify in the schema that interrupt-controller is required in the rtc node and that the property can be inherited from a parent. /dts-v1/; / { compatible = "ti,twl[0-9]+-rtc"; interrupt-controller { is-required; can-be-inherited; }; }; * Require a node to contains a given list of children O