Re: [RFC] yamldt v0.5, now a DTS compiler too
Hi all, I’ve just put up a blog-entry explaining the rationale behind yaml and device tree. https://www.konsulko.com/yaml-and-device-tree/ Please give it a quick glance and give feedback if you can. Regards — Pantelis
Re: [RFC] yamldt v0.5, now a DTS compiler too
Hi all, I’ve just put up a blog-entry explaining the rationale behind yaml and device tree. https://www.konsulko.com/yaml-and-device-tree/ Please give it a quick glance and give feedback if you can. Regards — Pantelis
Re: [RFC] yamldt v0.5, now a DTS compiler too
On Sun, Oct 22, 2017 at 7:23 PM, Pantelis Antoniouwrote: > Hi Grant, > >> On Oct 22, 2017, at 19:54 , Grant Likely wrote: >> >> On Fri, Oct 20, 2017 at 8:16 PM, Pantelis Antoniou >> wrote: >>> Hi Grant, >>> On Oct 20, 2017, at 20:46 , Grant Likely wrote: On Thu, Sep 28, 2017 at 8:58 PM, Pantelis Antoniou wrote: [...] >>> Yes. This is expected. I don’t think pure YAML form is a good match for all >>> the crazy things that are possible (and actually used in practice) with DTS. >> >> I don’t think it is as dire as that. The DTS structure is not complex >> and I think can easily be mapped into pure YAML. But, it does require >> treating the DTS structure separately from its semantic meaning. For >> example, the grammar of nodes, properties and labels easily maps to >> pure YAML, but phandles and overlay trees have semantic meaning that >> must be resolved by DT specific code. I’ll respond to you’re specific >> examples below… >> > > We are in complete agreement here. Single nodes and properties map to YAML > perfectly. > It’s the complex way that we build the complete DTB that stress the YAML > structures. Okay, good. >>> If we were to force YAML/DT file to be regular YAML files parseable by >>> everything >>> we’d have to rethink a lot of conventions that DT files currently use and >>> I’m afraid >>> a machine translation from DTS to YAML might not be feasible then. >>> >>> OTOH moving to that model might make it possible to use YAML constructs >>> that are not >>> mapped at all to DTS. For instance >>> >>> - foo: true >>> bar: “string” >>> >>> - foo: false >>> bar: “another-string” >>> >>> is not possible to be mapped to a DT node/property structure right now. >> >> I’m not following. Are you saying DT has no way to encode a list of >> nodes? What use case are you imagining? >> > > This threw me off too at first. > > The problem is when you try to convert this into a live tree structure. > > The YAML encoding is > > +SEQ > +MAP > =VAL :foo > =VAL :true > =VAL :bar > =VAL "string > -MAP > +MAP > =VAL :foo > =VAL :false > =VAL :bar > =VAL "another-string > -MAP > -SEQ > > So it’s a sequence of MAPs. > > In a live tree DTS form would be > > noname-property = { foo=true; bar = “string”; }, { foo = false; > bar=“another-string”; }; > > We don’t have the capability as right now to express something like this. > > Namely properties containing nodes, and the root of the live tree not being a > node. Ah, so you're proposing property data that encodes key/value pairs, correct? Effectively adding another layer of namespace under a property, but that namespace is invisible to and generic DT parsing code in an OS (for how DT is defined today). I would straight out make that structure illegal in the YAML DT metaschema until such time as the semantics are defined. For now properties should only be allowed to contain scalars, or lists of scalars/lists which can be nested. [...] The encoding differentiates between nodes and properties implicitly base on whether the contents are a map, or a scalar/list. This does mean any parser needs to do a bit more work and it may impact what can be done with validation (I'm not going to talk about validation in this email though. We'll talk next week.) >>> >>> Hmm, I’m not sure I follow this. Is that related to the metadata problem. >> >> It has to do with iterating over properties or nodes. Nodes and props >> are mixed together, and any code processing them has to explicitly >> check the type to differentiate between the two. The alternative would >> be to collect all properties under a single key: >> >> parent: >> $props: >>foo: hello >>bar: world >> child1: {} >> child2: {} >> > > That’s no problem. It does look weird though. I am using the same parser > for parsing bindings and this might make the files a little bit weird. That it does. I don’t feel strongly about either approach, but I want to see how the different encodings affect doing the schema bits. > > Keep in mind that we have two YAML formats to parse; the hardware description > and the bindings describing them. Three! Hw description, schema & metaschema. :-) But metaschema should be a normal jsonschema file. > >>> >>> BTW, a blog post explaining the rationale behind yamldt is going to come >>> up soon at our website, I’ll post the link when it does. >> >> Cool. I see it got posted, but didn’t get a chance to read it before I >> got on this airplane. Will try to read this evening. >> >> (BTW, I’m arriving at about 11:00 tonight) >> > > Have a safe flight, you’ll get to beat me up tomorrow :) See you tomorrow. It will be good to catch up. g.
Re: [RFC] yamldt v0.5, now a DTS compiler too
On Sun, Oct 22, 2017 at 7:23 PM, Pantelis Antoniou wrote: > Hi Grant, > >> On Oct 22, 2017, at 19:54 , Grant Likely wrote: >> >> On Fri, Oct 20, 2017 at 8:16 PM, Pantelis Antoniou >> wrote: >>> Hi Grant, >>> On Oct 20, 2017, at 20:46 , Grant Likely wrote: On Thu, Sep 28, 2017 at 8:58 PM, Pantelis Antoniou wrote: [...] >>> Yes. This is expected. I don’t think pure YAML form is a good match for all >>> the crazy things that are possible (and actually used in practice) with DTS. >> >> I don’t think it is as dire as that. The DTS structure is not complex >> and I think can easily be mapped into pure YAML. But, it does require >> treating the DTS structure separately from its semantic meaning. For >> example, the grammar of nodes, properties and labels easily maps to >> pure YAML, but phandles and overlay trees have semantic meaning that >> must be resolved by DT specific code. I’ll respond to you’re specific >> examples below… >> > > We are in complete agreement here. Single nodes and properties map to YAML > perfectly. > It’s the complex way that we build the complete DTB that stress the YAML > structures. Okay, good. >>> If we were to force YAML/DT file to be regular YAML files parseable by >>> everything >>> we’d have to rethink a lot of conventions that DT files currently use and >>> I’m afraid >>> a machine translation from DTS to YAML might not be feasible then. >>> >>> OTOH moving to that model might make it possible to use YAML constructs >>> that are not >>> mapped at all to DTS. For instance >>> >>> - foo: true >>> bar: “string” >>> >>> - foo: false >>> bar: “another-string” >>> >>> is not possible to be mapped to a DT node/property structure right now. >> >> I’m not following. Are you saying DT has no way to encode a list of >> nodes? What use case are you imagining? >> > > This threw me off too at first. > > The problem is when you try to convert this into a live tree structure. > > The YAML encoding is > > +SEQ > +MAP > =VAL :foo > =VAL :true > =VAL :bar > =VAL "string > -MAP > +MAP > =VAL :foo > =VAL :false > =VAL :bar > =VAL "another-string > -MAP > -SEQ > > So it’s a sequence of MAPs. > > In a live tree DTS form would be > > noname-property = { foo=true; bar = “string”; }, { foo = false; > bar=“another-string”; }; > > We don’t have the capability as right now to express something like this. > > Namely properties containing nodes, and the root of the live tree not being a > node. Ah, so you're proposing property data that encodes key/value pairs, correct? Effectively adding another layer of namespace under a property, but that namespace is invisible to and generic DT parsing code in an OS (for how DT is defined today). I would straight out make that structure illegal in the YAML DT metaschema until such time as the semantics are defined. For now properties should only be allowed to contain scalars, or lists of scalars/lists which can be nested. [...] The encoding differentiates between nodes and properties implicitly base on whether the contents are a map, or a scalar/list. This does mean any parser needs to do a bit more work and it may impact what can be done with validation (I'm not going to talk about validation in this email though. We'll talk next week.) >>> >>> Hmm, I’m not sure I follow this. Is that related to the metadata problem. >> >> It has to do with iterating over properties or nodes. Nodes and props >> are mixed together, and any code processing them has to explicitly >> check the type to differentiate between the two. The alternative would >> be to collect all properties under a single key: >> >> parent: >> $props: >>foo: hello >>bar: world >> child1: {} >> child2: {} >> > > That’s no problem. It does look weird though. I am using the same parser > for parsing bindings and this might make the files a little bit weird. That it does. I don’t feel strongly about either approach, but I want to see how the different encodings affect doing the schema bits. > > Keep in mind that we have two YAML formats to parse; the hardware description > and the bindings describing them. Three! Hw description, schema & metaschema. :-) But metaschema should be a normal jsonschema file. > >>> >>> BTW, a blog post explaining the rationale behind yamldt is going to come >>> up soon at our website, I’ll post the link when it does. >> >> Cool. I see it got posted, but didn’t get a chance to read it before I >> got on this airplane. Will try to read this evening. >> >> (BTW, I’m arriving at about 11:00 tonight) >> > > Have a safe flight, you’ll get to beat me up tomorrow :) See you tomorrow. It will be good to catch up. g.
Re: [RFC] yamldt v0.5, now a DTS compiler too
Hi Grant, > On Oct 22, 2017, at 19:54 , Grant Likelywrote: > > On Fri, Oct 20, 2017 at 8:16 PM, Pantelis Antoniou > wrote: >> Hi Grant, >> >>> On Oct 20, 2017, at 20:46 , Grant Likely wrote: >>> >>> On Thu, Sep 28, 2017 at 8:58 PM, Pantelis Antoniou >>> wrote: Hello again, Significant progress has been made on yamldt and is now capable of not only generating yaml from DTS source but also compiling DTS sources and being almost fully compatible with DTC. Compiling the kernel's DTBs using yamldt is as simple as using a DTC=yamldt. Error reporting is accurate and validation against a YAML based schema works as well. In a short while I will begin posting patches with fixes on bindings and DTS files in the kernel. Please try it on your platform and report if you encounter any problems. https://github.com/pantoniou/yamldt I am eagerly awaiting for your comments. >>> >>> Hi Pantelis, >>> >>> This is good work. I've played around with it and I'm looking forward >>> to chatting next week. >>> >> >> Thanks. I’m looking forward to it too. >> >>> One thing I've done is tried loading the output YAML files into >>> another YAML interpreter and the current encoding causes problems. >>> Specifically, in yamldt anchors/aliases are being used as a >>> replacement for labels/phandles, but that conflicts with the YAML data >>> model which defines a reference as a way to make a copy of the data >>> appear in another part of the tree. For example, for the following >>> snippit: >>> >>> intc: intc@1 { >>> #interrupt-cells = <1>; >>> compatible = "acme,intc"; >>> reg = <0x1 0x1000>; >>> gpio-controller; >>> }; >>> >>> serial@2 { >>> compatible = "acme,uart"; >>> reg = <0x2 0x1000>; >>> interrupt-parent = <>; >>> interrupts = <5>; >>> }; >>> >>> yamldt will encode this as: >>> >>> intc@1: >>> "#interrupt-cells": 1 >>> compatible: acme,intc >>> reg: [0x1, 0x1000] >>> gpio-controller: >>> >>> serial@2: >>> compatible: acme,uart >>> reg: [0x2, 0x1000] >>> interrupt-parent: *intc >>> interrupts: 5 >>> >>> But, the expected behaviour for a YAML parser is expand the alias >>> '*intc' which results in the following structure: >>> >>> intc@1: >>> "#interrupt-cells": 1 >>> compatible: acme,intc >>> reg: [0x1, 0x1000] >>> gpio-controller: >>> >>> serial@2: >>> compatible: acme,uart >>> reg: [0x2, 0x1000] >>> interrupt-parent: >>> "#interrupt-cells": 1 >>> compatible: acme,intc >>> reg: [0x1, 0x1000] >>> gpio-controller: >>> interrupts: 5 >>> >>> See? It results in the entire interrupt controller node to appear as >>> an instance under the interrupt-parent property, when the intention is >>> only to create a phandle. yamldt should not redefine the behaviour of >>> '*' aliases. Instead, it should use a different indicator, either >>> using an explicit !phandle tag, or by replacing '*' with something >>> else. I worked around it in my tests by replacing '*' with '$’. >> >> Yes. This is expected. I don’t think pure YAML form is a good match for all >> the crazy things that are possible (and actually used in practice) with DTS. > > I don’t think it is as dire as that. The DTS structure is not complex > and I think can easily be mapped into pure YAML. But, it does require > treating the DTS structure separately from its semantic meaning. For > example, the grammar of nodes, properties and labels easily maps to > pure YAML, but phandles and overlay trees have semantic meaning that > must be resolved by DT specific code. I’ll respond to you’re specific > examples below… > We are in complete agreement here. Single nodes and properties map to YAML perfectly. It’s the complex way that we build the complete DTB that stress the YAML structures. >> >> For instance there’s no way a normal YAML parser won’t choke with something >> like >> >> / { >> foo; >> }; >> >> / { >> bar; >> }; >> >> Which is a common idiom in DTS files. > > That’s only true when the top level of nodes is encoded as a map, but > it shouldn’t be. It should be a list instead, for two reasons. First, > order matters for the top level. Subsequent top level trees have to be > applied in order to fully resolve the tree, but a map doesn’t preserve > the ordering. Second, as you rightly point out, the same name can be > used for multiple top level trees. So, the encoding should be a list > with each list entry containing the node path {/path,}, and the > node data. One possible way to do this is with a tuple: > > treedata: > - - / > - foo: > - - / > - bar: > > Another option is to add a special property in the node to contain the name: > > treedata: > - $path: / > foo: > - $path: / > bar: > > Personally, I prefer
Re: [RFC] yamldt v0.5, now a DTS compiler too
Hi Grant, > On Oct 22, 2017, at 19:54 , Grant Likely wrote: > > On Fri, Oct 20, 2017 at 8:16 PM, Pantelis Antoniou > wrote: >> Hi Grant, >> >>> On Oct 20, 2017, at 20:46 , Grant Likely wrote: >>> >>> On Thu, Sep 28, 2017 at 8:58 PM, Pantelis Antoniou >>> wrote: Hello again, Significant progress has been made on yamldt and is now capable of not only generating yaml from DTS source but also compiling DTS sources and being almost fully compatible with DTC. Compiling the kernel's DTBs using yamldt is as simple as using a DTC=yamldt. Error reporting is accurate and validation against a YAML based schema works as well. In a short while I will begin posting patches with fixes on bindings and DTS files in the kernel. Please try it on your platform and report if you encounter any problems. https://github.com/pantoniou/yamldt I am eagerly awaiting for your comments. >>> >>> Hi Pantelis, >>> >>> This is good work. I've played around with it and I'm looking forward >>> to chatting next week. >>> >> >> Thanks. I’m looking forward to it too. >> >>> One thing I've done is tried loading the output YAML files into >>> another YAML interpreter and the current encoding causes problems. >>> Specifically, in yamldt anchors/aliases are being used as a >>> replacement for labels/phandles, but that conflicts with the YAML data >>> model which defines a reference as a way to make a copy of the data >>> appear in another part of the tree. For example, for the following >>> snippit: >>> >>> intc: intc@1 { >>> #interrupt-cells = <1>; >>> compatible = "acme,intc"; >>> reg = <0x1 0x1000>; >>> gpio-controller; >>> }; >>> >>> serial@2 { >>> compatible = "acme,uart"; >>> reg = <0x2 0x1000>; >>> interrupt-parent = <>; >>> interrupts = <5>; >>> }; >>> >>> yamldt will encode this as: >>> >>> intc@1: >>> "#interrupt-cells": 1 >>> compatible: acme,intc >>> reg: [0x1, 0x1000] >>> gpio-controller: >>> >>> serial@2: >>> compatible: acme,uart >>> reg: [0x2, 0x1000] >>> interrupt-parent: *intc >>> interrupts: 5 >>> >>> But, the expected behaviour for a YAML parser is expand the alias >>> '*intc' which results in the following structure: >>> >>> intc@1: >>> "#interrupt-cells": 1 >>> compatible: acme,intc >>> reg: [0x1, 0x1000] >>> gpio-controller: >>> >>> serial@2: >>> compatible: acme,uart >>> reg: [0x2, 0x1000] >>> interrupt-parent: >>> "#interrupt-cells": 1 >>> compatible: acme,intc >>> reg: [0x1, 0x1000] >>> gpio-controller: >>> interrupts: 5 >>> >>> See? It results in the entire interrupt controller node to appear as >>> an instance under the interrupt-parent property, when the intention is >>> only to create a phandle. yamldt should not redefine the behaviour of >>> '*' aliases. Instead, it should use a different indicator, either >>> using an explicit !phandle tag, or by replacing '*' with something >>> else. I worked around it in my tests by replacing '*' with '$’. >> >> Yes. This is expected. I don’t think pure YAML form is a good match for all >> the crazy things that are possible (and actually used in practice) with DTS. > > I don’t think it is as dire as that. The DTS structure is not complex > and I think can easily be mapped into pure YAML. But, it does require > treating the DTS structure separately from its semantic meaning. For > example, the grammar of nodes, properties and labels easily maps to > pure YAML, but phandles and overlay trees have semantic meaning that > must be resolved by DT specific code. I’ll respond to you’re specific > examples below… > We are in complete agreement here. Single nodes and properties map to YAML perfectly. It’s the complex way that we build the complete DTB that stress the YAML structures. >> >> For instance there’s no way a normal YAML parser won’t choke with something >> like >> >> / { >> foo; >> }; >> >> / { >> bar; >> }; >> >> Which is a common idiom in DTS files. > > That’s only true when the top level of nodes is encoded as a map, but > it shouldn’t be. It should be a list instead, for two reasons. First, > order matters for the top level. Subsequent top level trees have to be > applied in order to fully resolve the tree, but a map doesn’t preserve > the ordering. Second, as you rightly point out, the same name can be > used for multiple top level trees. So, the encoding should be a list > with each list entry containing the node path {/path,}, and the > node data. One possible way to do this is with a tuple: > > treedata: > - - / > - foo: > - - / > - bar: > > Another option is to add a special property in the node to contain the name: > > treedata: > - $path: / > foo: > - $path: / > bar: > > Personally, I prefer the special property approach. That would also be > a good way to encode labels > The YAML bare sequence is
Re: [RFC] yamldt v0.5, now a DTS compiler too
On Fri, Oct 20, 2017 at 8:16 PM, Pantelis Antoniouwrote: > Hi Grant, > >> On Oct 20, 2017, at 20:46 , Grant Likely wrote: >> >> On Thu, Sep 28, 2017 at 8:58 PM, Pantelis Antoniou >> wrote: >>> Hello again, >>> >>> Significant progress has been made on yamldt and is now capable of >>> not only generating yaml from DTS source but also compiling DTS sources >>> and being almost fully compatible with DTC. >>> >>> Compiling the kernel's DTBs using yamldt is as simple as using a >>> DTC=yamldt. >>> >>> Error reporting is accurate and validation against a YAML based schema >>> works as well. In a short while I will begin posting patches with >>> fixes on bindings and DTS files in the kernel. >>> >>> Please try it on your platform and report if you encounter any problems. >>> >>> https://github.com/pantoniou/yamldt >>> >>> I am eagerly awaiting for your comments. >> >> Hi Pantelis, >> >> This is good work. I've played around with it and I'm looking forward >> to chatting next week. >> > > Thanks. I’m looking forward to it too. > >> One thing I've done is tried loading the output YAML files into >> another YAML interpreter and the current encoding causes problems. >> Specifically, in yamldt anchors/aliases are being used as a >> replacement for labels/phandles, but that conflicts with the YAML data >> model which defines a reference as a way to make a copy of the data >> appear in another part of the tree. For example, for the following >> snippit: >> >> intc: intc@1 { >>#interrupt-cells = <1>; >>compatible = "acme,intc"; >>reg = <0x1 0x1000>; >>gpio-controller; >> }; >> >> serial@2 { >>compatible = "acme,uart"; >>reg = <0x2 0x1000>; >>interrupt-parent = <>; >>interrupts = <5>; >> }; >> >> yamldt will encode this as: >> >> intc@1: >>"#interrupt-cells": 1 >>compatible: acme,intc >>reg: [0x1, 0x1000] >>gpio-controller: >> >> serial@2: >>compatible: acme,uart >>reg: [0x2, 0x1000] >>interrupt-parent: *intc >>interrupts: 5 >> >> But, the expected behaviour for a YAML parser is expand the alias >> '*intc' which results in the following structure: >> >> intc@1: >>"#interrupt-cells": 1 >>compatible: acme,intc >>reg: [0x1, 0x1000] >>gpio-controller: >> >> serial@2: >>compatible: acme,uart >>reg: [0x2, 0x1000] >>interrupt-parent: >>"#interrupt-cells": 1 >>compatible: acme,intc >>reg: [0x1, 0x1000] >>gpio-controller: >>interrupts: 5 >> >> See? It results in the entire interrupt controller node to appear as >> an instance under the interrupt-parent property, when the intention is >> only to create a phandle. yamldt should not redefine the behaviour of >> '*' aliases. Instead, it should use a different indicator, either >> using an explicit !phandle tag, or by replacing '*' with something >> else. I worked around it in my tests by replacing '*' with '$’. > > Yes. This is expected. I don’t think pure YAML form is a good match for all > the crazy things that are possible (and actually used in practice) with DTS. I don’t think it is as dire as that. The DTS structure is not complex and I think can easily be mapped into pure YAML. But, it does require treating the DTS structure separately from its semantic meaning. For example, the grammar of nodes, properties and labels easily maps to pure YAML, but phandles and overlay trees have semantic meaning that must be resolved by DT specific code. I’ll respond to you’re specific examples below... > > For instance there’s no way a normal YAML parser won’t choke with something > like > > / { > foo; > }; > > / { > bar; > }; > > Which is a common idiom in DTS files. That’s only true when the top level of nodes is encoded as a map, but it shouldn’t be. It should be a list instead, for two reasons. First, order matters for the top level. Subsequent top level trees have to be applied in order to fully resolve the tree, but a map doesn’t preserve the ordering. Second, as you rightly point out, the same name can be used for multiple top level trees. So, the encoding should be a list with each list entry containing the node path {/path,}, and the node data. One possible way to do this is with a tuple: treedata: - - / - foo: - - / - bar: Another option is to add a special property in the node to contain the name: treedata: - $path: / foo: - $path: / bar: Personally, I prefer the special property approach. That would also be a good way to encode labels > The decision to use the YAML anchors and references was more borne out of a > desire > to sort of match conceptually the labels and references of DTS. It’s not a big > issue to switch to something different. I think it would be a good idea. One reason for defining a YAML encoding is to be able to use existing tools to work with the data. Changing the
Re: [RFC] yamldt v0.5, now a DTS compiler too
On Fri, Oct 20, 2017 at 8:16 PM, Pantelis Antoniou wrote: > Hi Grant, > >> On Oct 20, 2017, at 20:46 , Grant Likely wrote: >> >> On Thu, Sep 28, 2017 at 8:58 PM, Pantelis Antoniou >> wrote: >>> Hello again, >>> >>> Significant progress has been made on yamldt and is now capable of >>> not only generating yaml from DTS source but also compiling DTS sources >>> and being almost fully compatible with DTC. >>> >>> Compiling the kernel's DTBs using yamldt is as simple as using a >>> DTC=yamldt. >>> >>> Error reporting is accurate and validation against a YAML based schema >>> works as well. In a short while I will begin posting patches with >>> fixes on bindings and DTS files in the kernel. >>> >>> Please try it on your platform and report if you encounter any problems. >>> >>> https://github.com/pantoniou/yamldt >>> >>> I am eagerly awaiting for your comments. >> >> Hi Pantelis, >> >> This is good work. I've played around with it and I'm looking forward >> to chatting next week. >> > > Thanks. I’m looking forward to it too. > >> One thing I've done is tried loading the output YAML files into >> another YAML interpreter and the current encoding causes problems. >> Specifically, in yamldt anchors/aliases are being used as a >> replacement for labels/phandles, but that conflicts with the YAML data >> model which defines a reference as a way to make a copy of the data >> appear in another part of the tree. For example, for the following >> snippit: >> >> intc: intc@1 { >>#interrupt-cells = <1>; >>compatible = "acme,intc"; >>reg = <0x1 0x1000>; >>gpio-controller; >> }; >> >> serial@2 { >>compatible = "acme,uart"; >>reg = <0x2 0x1000>; >>interrupt-parent = <>; >>interrupts = <5>; >> }; >> >> yamldt will encode this as: >> >> intc@1: >>"#interrupt-cells": 1 >>compatible: acme,intc >>reg: [0x1, 0x1000] >>gpio-controller: >> >> serial@2: >>compatible: acme,uart >>reg: [0x2, 0x1000] >>interrupt-parent: *intc >>interrupts: 5 >> >> But, the expected behaviour for a YAML parser is expand the alias >> '*intc' which results in the following structure: >> >> intc@1: >>"#interrupt-cells": 1 >>compatible: acme,intc >>reg: [0x1, 0x1000] >>gpio-controller: >> >> serial@2: >>compatible: acme,uart >>reg: [0x2, 0x1000] >>interrupt-parent: >>"#interrupt-cells": 1 >>compatible: acme,intc >>reg: [0x1, 0x1000] >>gpio-controller: >>interrupts: 5 >> >> See? It results in the entire interrupt controller node to appear as >> an instance under the interrupt-parent property, when the intention is >> only to create a phandle. yamldt should not redefine the behaviour of >> '*' aliases. Instead, it should use a different indicator, either >> using an explicit !phandle tag, or by replacing '*' with something >> else. I worked around it in my tests by replacing '*' with '$’. > > Yes. This is expected. I don’t think pure YAML form is a good match for all > the crazy things that are possible (and actually used in practice) with DTS. I don’t think it is as dire as that. The DTS structure is not complex and I think can easily be mapped into pure YAML. But, it does require treating the DTS structure separately from its semantic meaning. For example, the grammar of nodes, properties and labels easily maps to pure YAML, but phandles and overlay trees have semantic meaning that must be resolved by DT specific code. I’ll respond to you’re specific examples below... > > For instance there’s no way a normal YAML parser won’t choke with something > like > > / { > foo; > }; > > / { > bar; > }; > > Which is a common idiom in DTS files. That’s only true when the top level of nodes is encoded as a map, but it shouldn’t be. It should be a list instead, for two reasons. First, order matters for the top level. Subsequent top level trees have to be applied in order to fully resolve the tree, but a map doesn’t preserve the ordering. Second, as you rightly point out, the same name can be used for multiple top level trees. So, the encoding should be a list with each list entry containing the node path {/path,}, and the node data. One possible way to do this is with a tuple: treedata: - - / - foo: - - / - bar: Another option is to add a special property in the node to contain the name: treedata: - $path: / foo: - $path: / bar: Personally, I prefer the special property approach. That would also be a good way to encode labels > The decision to use the YAML anchors and references was more borne out of a > desire > to sort of match conceptually the labels and references of DTS. It’s not a big > issue to switch to something different. I think it would be a good idea. One reason for defining a YAML encoding is to be able to use existing tools to work with the data. Changing the meaning of anchors and aliases breaks those tools immediately. > If we were to force YAML/DT
Re: [RFC] yamldt v0.5, now a DTS compiler too
Hi Grant, > On Oct 20, 2017, at 20:46 , Grant Likelywrote: > > On Thu, Sep 28, 2017 at 8:58 PM, Pantelis Antoniou > wrote: >> Hello again, >> >> Significant progress has been made on yamldt and is now capable of >> not only generating yaml from DTS source but also compiling DTS sources >> and being almost fully compatible with DTC. >> >> Compiling the kernel's DTBs using yamldt is as simple as using a >> DTC=yamldt. >> >> Error reporting is accurate and validation against a YAML based schema >> works as well. In a short while I will begin posting patches with >> fixes on bindings and DTS files in the kernel. >> >> Please try it on your platform and report if you encounter any problems. >> >> https://github.com/pantoniou/yamldt >> >> I am eagerly awaiting for your comments. > > Hi Pantelis, > > This is good work. I've played around with it and I'm looking forward > to chatting next week. > Thanks. I’m looking forward to it too. > One thing I've done is tried loading the output YAML files into > another YAML interpreter and the current encoding causes problems. > Specifically, in yamldt anchors/aliases are being used as a > replacement for labels/phandles, but that conflicts with the YAML data > model which defines a reference as a way to make a copy of the data > appear in another part of the tree. For example, for the following > snippit: > > intc: intc@1 { >#interrupt-cells = <1>; >compatible = "acme,intc"; >reg = <0x1 0x1000>; >gpio-controller; > }; > > serial@2 { >compatible = "acme,uart"; >reg = <0x2 0x1000>; >interrupt-parent = <>; >interrupts = <5>; > }; > > yamldt will encode this as: > > intc@1: >"#interrupt-cells": 1 >compatible: acme,intc >reg: [0x1, 0x1000] >gpio-controller: > > serial@2: >compatible: acme,uart >reg: [0x2, 0x1000] >interrupt-parent: *intc >interrupts: 5 > > But, the expected behaviour for a YAML parser is expand the alias > '*intc' which results in the following structure: > > intc@1: >"#interrupt-cells": 1 >compatible: acme,intc >reg: [0x1, 0x1000] >gpio-controller: > > serial@2: >compatible: acme,uart >reg: [0x2, 0x1000] >interrupt-parent: >"#interrupt-cells": 1 >compatible: acme,intc >reg: [0x1, 0x1000] >gpio-controller: >interrupts: 5 > > See? It results in the entire interrupt controller node to appear as > an instance under the interrupt-parent property, when the intention is > only to create a phandle. yamldt should not redefine the behaviour of > '*' aliases. Instead, it should use a different indicator, either > using an explicit !phandle tag, or by replacing '*' with something > else. I worked around it in my tests by replacing '*' with '$’. Yes. This is expected. I don’t think pure YAML form is a good match for all the crazy things that are possible (and actually used in practice) with DTS. For instance there’s no way a normal YAML parser won’t choke with something like / { foo; }; / { bar; }; Which is a common idiom in DTS files. The decision to use the YAML anchors and references was more borne out of a desire to sort of match conceptually the labels and references of DTS. It’s not a big issue to switch to something different. If we were to force YAML/DT file to be regular YAML files parseable by everything we’d have to rethink a lot of conventions that DT files currently use and I’m afraid a machine translation from DTS to YAML might not be feasible then. OTOH moving to that model might make it possible to use YAML constructs that are not mapped at all to DTS. For instance - foo: true bar: “string” - foo: false bar: “another-string” is not possible to be mapped to a DT node/property structure right now. > > Plus, it would be useful to use normal YAML anchors/aliases for > creating node templates. For example: > > serial-template: . # The anchor for the template >compatible: acme,uart >interrupt-parent: *intc > > root: >serial@2: ><<: *acme-uart # Alias node merged into serial@2 >interrupts: 5 >reg: [0x2, 0x1000] >serial@3: ><<: *acme-uart # Alias node merged into serial@3 >interrupts: 5 >reg: [0x3, 0x1000] > Yes, I’m aware of this and I planned to talk to you about it :) It can be a very powerful way to cut down the churn of DT right now. It’s not going to be a problem for yamldt to support it (perhaps adding a new editing tags to allow more fine grained control of the substitions). > Another problem with anchors/references is YAML seems to require the > anchor to be defined before the reference, or at least that's what > pyyaml and ruamel both expect. Regardless, The chosen YAML encoding > should be readily consumable by existing yaml implementations without > having to do a
Re: [RFC] yamldt v0.5, now a DTS compiler too
Hi Grant, > On Oct 20, 2017, at 20:46 , Grant Likely wrote: > > On Thu, Sep 28, 2017 at 8:58 PM, Pantelis Antoniou > wrote: >> Hello again, >> >> Significant progress has been made on yamldt and is now capable of >> not only generating yaml from DTS source but also compiling DTS sources >> and being almost fully compatible with DTC. >> >> Compiling the kernel's DTBs using yamldt is as simple as using a >> DTC=yamldt. >> >> Error reporting is accurate and validation against a YAML based schema >> works as well. In a short while I will begin posting patches with >> fixes on bindings and DTS files in the kernel. >> >> Please try it on your platform and report if you encounter any problems. >> >> https://github.com/pantoniou/yamldt >> >> I am eagerly awaiting for your comments. > > Hi Pantelis, > > This is good work. I've played around with it and I'm looking forward > to chatting next week. > Thanks. I’m looking forward to it too. > One thing I've done is tried loading the output YAML files into > another YAML interpreter and the current encoding causes problems. > Specifically, in yamldt anchors/aliases are being used as a > replacement for labels/phandles, but that conflicts with the YAML data > model which defines a reference as a way to make a copy of the data > appear in another part of the tree. For example, for the following > snippit: > > intc: intc@1 { >#interrupt-cells = <1>; >compatible = "acme,intc"; >reg = <0x1 0x1000>; >gpio-controller; > }; > > serial@2 { >compatible = "acme,uart"; >reg = <0x2 0x1000>; >interrupt-parent = <>; >interrupts = <5>; > }; > > yamldt will encode this as: > > intc@1: >"#interrupt-cells": 1 >compatible: acme,intc >reg: [0x1, 0x1000] >gpio-controller: > > serial@2: >compatible: acme,uart >reg: [0x2, 0x1000] >interrupt-parent: *intc >interrupts: 5 > > But, the expected behaviour for a YAML parser is expand the alias > '*intc' which results in the following structure: > > intc@1: >"#interrupt-cells": 1 >compatible: acme,intc >reg: [0x1, 0x1000] >gpio-controller: > > serial@2: >compatible: acme,uart >reg: [0x2, 0x1000] >interrupt-parent: >"#interrupt-cells": 1 >compatible: acme,intc >reg: [0x1, 0x1000] >gpio-controller: >interrupts: 5 > > See? It results in the entire interrupt controller node to appear as > an instance under the interrupt-parent property, when the intention is > only to create a phandle. yamldt should not redefine the behaviour of > '*' aliases. Instead, it should use a different indicator, either > using an explicit !phandle tag, or by replacing '*' with something > else. I worked around it in my tests by replacing '*' with '$’. Yes. This is expected. I don’t think pure YAML form is a good match for all the crazy things that are possible (and actually used in practice) with DTS. For instance there’s no way a normal YAML parser won’t choke with something like / { foo; }; / { bar; }; Which is a common idiom in DTS files. The decision to use the YAML anchors and references was more borne out of a desire to sort of match conceptually the labels and references of DTS. It’s not a big issue to switch to something different. If we were to force YAML/DT file to be regular YAML files parseable by everything we’d have to rethink a lot of conventions that DT files currently use and I’m afraid a machine translation from DTS to YAML might not be feasible then. OTOH moving to that model might make it possible to use YAML constructs that are not mapped at all to DTS. For instance - foo: true bar: “string” - foo: false bar: “another-string” is not possible to be mapped to a DT node/property structure right now. > > Plus, it would be useful to use normal YAML anchors/aliases for > creating node templates. For example: > > serial-template: . # The anchor for the template >compatible: acme,uart >interrupt-parent: *intc > > root: >serial@2: ><<: *acme-uart # Alias node merged into serial@2 >interrupts: 5 >reg: [0x2, 0x1000] >serial@3: ><<: *acme-uart # Alias node merged into serial@3 >interrupts: 5 >reg: [0x3, 0x1000] > Yes, I’m aware of this and I planned to talk to you about it :) It can be a very powerful way to cut down the churn of DT right now. It’s not going to be a problem for yamldt to support it (perhaps adding a new editing tags to allow more fine grained control of the substitions). > Another problem with anchors/references is YAML seems to require the > anchor to be defined before the reference, or at least that's what > pyyaml and ruamel both expect. Regardless, The chosen YAML encoding > should be readily consumable by existing yaml implementations without > having to do a lot of customization. > I think this is only true for
Re: [RFC] yamldt v0.5, now a DTS compiler too
On Thu, Sep 28, 2017 at 8:58 PM, Pantelis Antoniouwrote: > Hello again, > > Significant progress has been made on yamldt and is now capable of > not only generating yaml from DTS source but also compiling DTS sources > and being almost fully compatible with DTC. > > Compiling the kernel's DTBs using yamldt is as simple as using a > DTC=yamldt. > > Error reporting is accurate and validation against a YAML based schema > works as well. In a short while I will begin posting patches with > fixes on bindings and DTS files in the kernel. > > Please try it on your platform and report if you encounter any problems. > > https://github.com/pantoniou/yamldt > > I am eagerly awaiting for your comments. Hi Pantelis, This is good work. I've played around with it and I'm looking forward to chatting next week. One thing I've done is tried loading the output YAML files into another YAML interpreter and the current encoding causes problems. Specifically, in yamldt anchors/aliases are being used as a replacement for labels/phandles, but that conflicts with the YAML data model which defines a reference as a way to make a copy of the data appear in another part of the tree. For example, for the following snippit: intc: intc@1 { #interrupt-cells = <1>; compatible = "acme,intc"; reg = <0x1 0x1000>; gpio-controller; }; serial@2 { compatible = "acme,uart"; reg = <0x2 0x1000>; interrupt-parent = <>; interrupts = <5>; }; yamldt will encode this as: intc@1: "#interrupt-cells": 1 compatible: acme,intc reg: [0x1, 0x1000] gpio-controller: serial@2: compatible: acme,uart reg: [0x2, 0x1000] interrupt-parent: *intc interrupts: 5 But, the expected behaviour for a YAML parser is expand the alias '*intc' which results in the following structure: intc@1: "#interrupt-cells": 1 compatible: acme,intc reg: [0x1, 0x1000] gpio-controller: serial@2: compatible: acme,uart reg: [0x2, 0x1000] interrupt-parent: "#interrupt-cells": 1 compatible: acme,intc reg: [0x1, 0x1000] gpio-controller: interrupts: 5 See? It results in the entire interrupt controller node to appear as an instance under the interrupt-parent property, when the intention is only to create a phandle. yamldt should not redefine the behaviour of '*' aliases. Instead, it should use a different indicator, either using an explicit !phandle tag, or by replacing '*' with something else. I worked around it in my tests by replacing '*' with '$'. Plus, it would be useful to use normal YAML anchors/aliases for creating node templates. For example: serial-template: . # The anchor for the template compatible: acme,uart interrupt-parent: *intc root: serial@2: <<: *acme-uart # Alias node merged into serial@2 interrupts: 5 reg: [0x2, 0x1000] serial@3: <<: *acme-uart # Alias node merged into serial@3 interrupts: 5 reg: [0x3, 0x1000] Another problem with anchors/references is YAML seems to require the anchor to be defined before the reference, or at least that's what pyyaml and ruamel both expect. Regardless, The chosen YAML encoding should be readily consumable by existing yaml implementations without having to do a lot of customization. I'm slightly concerned about using & anchors for labels because it seems only one anchor can be defined per node, but DTC allows multiple labels for a single node. This might not be an issue in practice though. Another implementation issue related to using & anchors is the YAML spec defines them as an encoding artifact, and parsers can discard the anchor names after parsing the YAML structure, which is a problem if we use something like $name to reference an anchor. The solution might just be that labels need to go into a special property so they don't disappear from the data stream. There appears to be no place to put metadata. The root of the tree is the top level of the YAML structure. There isn't any provision for having a top level object to hold things like the memreserve map. We may need a namespace to use for special properties that aren't nodes or properties. The encoding differentiates between nodes and properties implicitly base on whether the contents are a map, or a scalar/list. This does mean any parser needs to do a bit more work and it may impact what can be done with validation (I'm not going to talk about validation in this email though. We'll talk next week.) Cheers, g.
Re: [RFC] yamldt v0.5, now a DTS compiler too
On Thu, Sep 28, 2017 at 8:58 PM, Pantelis Antoniou wrote: > Hello again, > > Significant progress has been made on yamldt and is now capable of > not only generating yaml from DTS source but also compiling DTS sources > and being almost fully compatible with DTC. > > Compiling the kernel's DTBs using yamldt is as simple as using a > DTC=yamldt. > > Error reporting is accurate and validation against a YAML based schema > works as well. In a short while I will begin posting patches with > fixes on bindings and DTS files in the kernel. > > Please try it on your platform and report if you encounter any problems. > > https://github.com/pantoniou/yamldt > > I am eagerly awaiting for your comments. Hi Pantelis, This is good work. I've played around with it and I'm looking forward to chatting next week. One thing I've done is tried loading the output YAML files into another YAML interpreter and the current encoding causes problems. Specifically, in yamldt anchors/aliases are being used as a replacement for labels/phandles, but that conflicts with the YAML data model which defines a reference as a way to make a copy of the data appear in another part of the tree. For example, for the following snippit: intc: intc@1 { #interrupt-cells = <1>; compatible = "acme,intc"; reg = <0x1 0x1000>; gpio-controller; }; serial@2 { compatible = "acme,uart"; reg = <0x2 0x1000>; interrupt-parent = <>; interrupts = <5>; }; yamldt will encode this as: intc@1: "#interrupt-cells": 1 compatible: acme,intc reg: [0x1, 0x1000] gpio-controller: serial@2: compatible: acme,uart reg: [0x2, 0x1000] interrupt-parent: *intc interrupts: 5 But, the expected behaviour for a YAML parser is expand the alias '*intc' which results in the following structure: intc@1: "#interrupt-cells": 1 compatible: acme,intc reg: [0x1, 0x1000] gpio-controller: serial@2: compatible: acme,uart reg: [0x2, 0x1000] interrupt-parent: "#interrupt-cells": 1 compatible: acme,intc reg: [0x1, 0x1000] gpio-controller: interrupts: 5 See? It results in the entire interrupt controller node to appear as an instance under the interrupt-parent property, when the intention is only to create a phandle. yamldt should not redefine the behaviour of '*' aliases. Instead, it should use a different indicator, either using an explicit !phandle tag, or by replacing '*' with something else. I worked around it in my tests by replacing '*' with '$'. Plus, it would be useful to use normal YAML anchors/aliases for creating node templates. For example: serial-template: . # The anchor for the template compatible: acme,uart interrupt-parent: *intc root: serial@2: <<: *acme-uart # Alias node merged into serial@2 interrupts: 5 reg: [0x2, 0x1000] serial@3: <<: *acme-uart # Alias node merged into serial@3 interrupts: 5 reg: [0x3, 0x1000] Another problem with anchors/references is YAML seems to require the anchor to be defined before the reference, or at least that's what pyyaml and ruamel both expect. Regardless, The chosen YAML encoding should be readily consumable by existing yaml implementations without having to do a lot of customization. I'm slightly concerned about using & anchors for labels because it seems only one anchor can be defined per node, but DTC allows multiple labels for a single node. This might not be an issue in practice though. Another implementation issue related to using & anchors is the YAML spec defines them as an encoding artifact, and parsers can discard the anchor names after parsing the YAML structure, which is a problem if we use something like $name to reference an anchor. The solution might just be that labels need to go into a special property so they don't disappear from the data stream. There appears to be no place to put metadata. The root of the tree is the top level of the YAML structure. There isn't any provision for having a top level object to hold things like the memreserve map. We may need a namespace to use for special properties that aren't nodes or properties. The encoding differentiates between nodes and properties implicitly base on whether the contents are a map, or a scalar/list. This does mean any parser needs to do a bit more work and it may impact what can be done with validation (I'm not going to talk about validation in this email though. We'll talk next week.) Cheers, g.
Re: [RFC] yamldt v0.5, now a DTS compiler too
On Tue, Oct 10, 2017 at 06:19:03PM +0300, Pantelis Antoniou wrote: > Hi David, > > > On Oct 10, 2017, at 04:50 , David Gibson> > wrote: > > > > On Mon, Oct 09, 2017 at 06:07:28PM +0300, Pantelis Antoniou wrote: > >> Hi David, > >> > >>> On Oct 9, 2017, at 03:00 , David Gibson > >>> wrote: > >>> > >>> On Sun, Oct 08, 2017 at 04:08:03PM -0700, Frank Rowand wrote: > On 10/07/17 03:23, Pantelis Antoniou wrote: > > Hi Rob, > > > >> On Oct 6, 2017, at 16:55 , Rob Herring wrote: > >> > >> On Tue, Oct 3, 2017 at 12:39 PM, Pantelis Antoniou > >> wrote: > >>> Hi Rob, > > < snip > > > >>> eBPF is portable, can be serialized after compiling in the schema file > >>> and can be executed in the kernel. > >> > >> Executing in the kernel is a non-goal for me. > > Executing in the kernel is an anti-goal for me. > > We are trying to reduce the device tree footprint inside the kernel, > not increase it. > > 99.99% of the validation should be possible statically, in the compile > phase. > > > >>> By stripping out all documentation related properties and nodes > >>> keeping > >>> only the compiled filters you can generate a dtb blob that passed to > >>> kernel can be used for verification of all runtime changes in the > >>> kernel's live tree. eBPF is enforcing an execution model that is > >>> 'safe' > >>> so we can be sure that no foul play is possible. > > Run time changes can be assumed correct (short of bugs in the overlay > application code), if the base tree is validated, the overlay is > validated, > and the interface between the live tree and the overlay is a > connector. > >>> > >>> In addition, no amount of schema validation can really protect the > >>> kernel from a bad DT. Even if the schemas can 100% verify that the DT > >>> is "syntactically" correct, which is ambitious, it can't protect > >>> against a DT which is in the right form, but contains information that > >>> is simply wrong for the hardware in question. That can stuff the > >>> kernel at least as easily as an incorrectly formatted DT. > >>> > >> > >> I disagree. > >> > >> There are multiple levels of validation. For now we’re only talking about > >> binding validation. There can be SoC level validation, board level > >> validation, > >> revision level validation and finally application specific validation. > >> > >> Binding validation is making sure properties/nodes follow the binding > >> document. > >> For instance that for a foo device there’s a mandatory interrupt property. > >> > >> Simplified > >> > >> interrupt = ; > >> > >> Binding validation would ‘catch’ errors like assigning a string or not > >> having the > >> interrupt property available. > >> > >> SoC level validation would list the available interrupt number that a given > >> SoC would support for that device. > >> > >> For example that interrupt can only take the values 10 or 99 in a given > >> SoC. > >> > >> Board level validation would narrow this down even further to a value of > >> 10 for > >> a given board model. > > > >> Similar revision level validation would place further restriction on the > >> allowed > >> configuration. > >> > >> Finally application specific validation could place restriction based on > >> the intended > >> application that piece of hardware is used for. For instance devices that > >> should not > >> exceed a given power budget would have restrictions on the clock frequency > >> of the processor > >> or bus frequencies etc. > > > > This doesn't help. In order to do this, the validator would need > > information that's essentially equivalent to the content of DT, at > > which point there's no point to the DT at all - and you're left with > > the problem of validating the information that the validator has. > > That would be the case if hardware IP only has a single way to be configured. Right, and if if there's more than one way, then the validator can't possibly tell whether the DT has the right one. DTs must always come from a trusted source, because if they don't, then you don't need the DT in the first place (you could build your own). > The industry standard nowadays is picking reusable IP blocks and integrating > them > together in an SoC. The driver and the binding is common for every platform > that uses > that block, but the allowed configuration varies according to what the > hardware > people uses in a given instance. > > Fundamentally a validator that's useful *cannot* tell the difference > > between a correct tree and one which _could_ be correct for some > > theoretical hardware, but isn't for this particular hardware. > > That’s why there’s reason for a nested hierarchy of bindings IMO. Nothing about how
Re: [RFC] yamldt v0.5, now a DTS compiler too
On Tue, Oct 10, 2017 at 06:19:03PM +0300, Pantelis Antoniou wrote: > Hi David, > > > On Oct 10, 2017, at 04:50 , David Gibson > > wrote: > > > > On Mon, Oct 09, 2017 at 06:07:28PM +0300, Pantelis Antoniou wrote: > >> Hi David, > >> > >>> On Oct 9, 2017, at 03:00 , David Gibson > >>> wrote: > >>> > >>> On Sun, Oct 08, 2017 at 04:08:03PM -0700, Frank Rowand wrote: > On 10/07/17 03:23, Pantelis Antoniou wrote: > > Hi Rob, > > > >> On Oct 6, 2017, at 16:55 , Rob Herring wrote: > >> > >> On Tue, Oct 3, 2017 at 12:39 PM, Pantelis Antoniou > >> wrote: > >>> Hi Rob, > > < snip > > > >>> eBPF is portable, can be serialized after compiling in the schema file > >>> and can be executed in the kernel. > >> > >> Executing in the kernel is a non-goal for me. > > Executing in the kernel is an anti-goal for me. > > We are trying to reduce the device tree footprint inside the kernel, > not increase it. > > 99.99% of the validation should be possible statically, in the compile > phase. > > > >>> By stripping out all documentation related properties and nodes > >>> keeping > >>> only the compiled filters you can generate a dtb blob that passed to > >>> kernel can be used for verification of all runtime changes in the > >>> kernel's live tree. eBPF is enforcing an execution model that is > >>> 'safe' > >>> so we can be sure that no foul play is possible. > > Run time changes can be assumed correct (short of bugs in the overlay > application code), if the base tree is validated, the overlay is > validated, > and the interface between the live tree and the overlay is a > connector. > >>> > >>> In addition, no amount of schema validation can really protect the > >>> kernel from a bad DT. Even if the schemas can 100% verify that the DT > >>> is "syntactically" correct, which is ambitious, it can't protect > >>> against a DT which is in the right form, but contains information that > >>> is simply wrong for the hardware in question. That can stuff the > >>> kernel at least as easily as an incorrectly formatted DT. > >>> > >> > >> I disagree. > >> > >> There are multiple levels of validation. For now we’re only talking about > >> binding validation. There can be SoC level validation, board level > >> validation, > >> revision level validation and finally application specific validation. > >> > >> Binding validation is making sure properties/nodes follow the binding > >> document. > >> For instance that for a foo device there’s a mandatory interrupt property. > >> > >> Simplified > >> > >> interrupt = ; > >> > >> Binding validation would ‘catch’ errors like assigning a string or not > >> having the > >> interrupt property available. > >> > >> SoC level validation would list the available interrupt number that a given > >> SoC would support for that device. > >> > >> For example that interrupt can only take the values 10 or 99 in a given > >> SoC. > >> > >> Board level validation would narrow this down even further to a value of > >> 10 for > >> a given board model. > > > >> Similar revision level validation would place further restriction on the > >> allowed > >> configuration. > >> > >> Finally application specific validation could place restriction based on > >> the intended > >> application that piece of hardware is used for. For instance devices that > >> should not > >> exceed a given power budget would have restrictions on the clock frequency > >> of the processor > >> or bus frequencies etc. > > > > This doesn't help. In order to do this, the validator would need > > information that's essentially equivalent to the content of DT, at > > which point there's no point to the DT at all - and you're left with > > the problem of validating the information that the validator has. > > That would be the case if hardware IP only has a single way to be configured. Right, and if if there's more than one way, then the validator can't possibly tell whether the DT has the right one. DTs must always come from a trusted source, because if they don't, then you don't need the DT in the first place (you could build your own). > The industry standard nowadays is picking reusable IP blocks and integrating > them > together in an SoC. The driver and the binding is common for every platform > that uses > that block, but the allowed configuration varies according to what the > hardware > people uses in a given instance. > > Fundamentally a validator that's useful *cannot* tell the difference > > between a correct tree and one which _could_ be correct for some > > theoretical hardware, but isn't for this particular hardware. > > That’s why there’s reason for a nested hierarchy of bindings IMO. Nothing about how you structure the validation can change the basic fact that there are only two possibilities. Either: a) You
Re: [RFC] yamldt v0.5, now a DTS compiler too
Hi David, > On Oct 10, 2017, at 04:50 , David Gibsonwrote: > > On Mon, Oct 09, 2017 at 06:07:28PM +0300, Pantelis Antoniou wrote: >> Hi David, >> >>> On Oct 9, 2017, at 03:00 , David Gibson wrote: >>> >>> On Sun, Oct 08, 2017 at 04:08:03PM -0700, Frank Rowand wrote: On 10/07/17 03:23, Pantelis Antoniou wrote: > Hi Rob, > >> On Oct 6, 2017, at 16:55 , Rob Herring wrote: >> >> On Tue, Oct 3, 2017 at 12:39 PM, Pantelis Antoniou >> wrote: >>> Hi Rob, < snip > >>> eBPF is portable, can be serialized after compiling in the schema file >>> and can be executed in the kernel. >> >> Executing in the kernel is a non-goal for me. Executing in the kernel is an anti-goal for me. We are trying to reduce the device tree footprint inside the kernel, not increase it. 99.99% of the validation should be possible statically, in the compile phase. >>> By stripping out all documentation related properties and nodes keeping >>> only the compiled filters you can generate a dtb blob that passed to >>> kernel can be used for verification of all runtime changes in the >>> kernel's live tree. eBPF is enforcing an execution model that is 'safe' >>> so we can be sure that no foul play is possible. Run time changes can be assumed correct (short of bugs in the overlay application code), if the base tree is validated, the overlay is validated, and the interface between the live tree and the overlay is a connector. >>> >>> In addition, no amount of schema validation can really protect the >>> kernel from a bad DT. Even if the schemas can 100% verify that the DT >>> is "syntactically" correct, which is ambitious, it can't protect >>> against a DT which is in the right form, but contains information that >>> is simply wrong for the hardware in question. That can stuff the >>> kernel at least as easily as an incorrectly formatted DT. >>> >> >> I disagree. >> >> There are multiple levels of validation. For now we’re only talking about >> binding validation. There can be SoC level validation, board level >> validation, >> revision level validation and finally application specific validation. >> >> Binding validation is making sure properties/nodes follow the binding >> document. >> For instance that for a foo device there’s a mandatory interrupt property. >> >> Simplified >> >> interrupt = ; >> >> Binding validation would ‘catch’ errors like assigning a string or not >> having the >> interrupt property available. >> >> SoC level validation would list the available interrupt number that a given >> SoC would support for that device. >> >> For example that interrupt can only take the values 10 or 99 in a given SoC. >> >> Board level validation would narrow this down even further to a value of 10 >> for >> a given board model. > >> Similar revision level validation would place further restriction on the >> allowed >> configuration. >> >> Finally application specific validation could place restriction based on the >> intended >> application that piece of hardware is used for. For instance devices that >> should not >> exceed a given power budget would have restrictions on the clock frequency >> of the processor >> or bus frequencies etc. > > This doesn't help. In order to do this, the validator would need > information that's essentially equivalent to the content of DT, at > which point there's no point to the DT at all - and you're left with > the problem of validating the information that the validator has. > That would be the case if hardware IP only has a single way to be configured. The industry standard nowadays is picking reusable IP blocks and integrating them together in an SoC. The driver and the binding is common for every platform that uses that block, but the allowed configuration varies according to what the hardware people uses in a given instance. > Fundamentally a validator that's useful *cannot* tell the difference > between a correct tree and one which _could_ be correct for some > theoretical hardware, but isn't for this particular hardware. > That’s why there’s reason for a nested hierarchy of bindings IMO. Completeness of validation schemes can be a differentiating factor when choosing parts for hardware design. They would sure cut down development time. > -- > David Gibson | I'll have my music baroque, and my code > david AT gibson.dropbear.id.au| minimalist, thank you. NOT _the_ > _other_ > | _way_ _around_! > http://www.ozlabs.org/~dgibson Regards — Pantelis
Re: [RFC] yamldt v0.5, now a DTS compiler too
Hi David, > On Oct 10, 2017, at 04:50 , David Gibson wrote: > > On Mon, Oct 09, 2017 at 06:07:28PM +0300, Pantelis Antoniou wrote: >> Hi David, >> >>> On Oct 9, 2017, at 03:00 , David Gibson wrote: >>> >>> On Sun, Oct 08, 2017 at 04:08:03PM -0700, Frank Rowand wrote: On 10/07/17 03:23, Pantelis Antoniou wrote: > Hi Rob, > >> On Oct 6, 2017, at 16:55 , Rob Herring wrote: >> >> On Tue, Oct 3, 2017 at 12:39 PM, Pantelis Antoniou >> wrote: >>> Hi Rob, < snip > >>> eBPF is portable, can be serialized after compiling in the schema file >>> and can be executed in the kernel. >> >> Executing in the kernel is a non-goal for me. Executing in the kernel is an anti-goal for me. We are trying to reduce the device tree footprint inside the kernel, not increase it. 99.99% of the validation should be possible statically, in the compile phase. >>> By stripping out all documentation related properties and nodes keeping >>> only the compiled filters you can generate a dtb blob that passed to >>> kernel can be used for verification of all runtime changes in the >>> kernel's live tree. eBPF is enforcing an execution model that is 'safe' >>> so we can be sure that no foul play is possible. Run time changes can be assumed correct (short of bugs in the overlay application code), if the base tree is validated, the overlay is validated, and the interface between the live tree and the overlay is a connector. >>> >>> In addition, no amount of schema validation can really protect the >>> kernel from a bad DT. Even if the schemas can 100% verify that the DT >>> is "syntactically" correct, which is ambitious, it can't protect >>> against a DT which is in the right form, but contains information that >>> is simply wrong for the hardware in question. That can stuff the >>> kernel at least as easily as an incorrectly formatted DT. >>> >> >> I disagree. >> >> There are multiple levels of validation. For now we’re only talking about >> binding validation. There can be SoC level validation, board level >> validation, >> revision level validation and finally application specific validation. >> >> Binding validation is making sure properties/nodes follow the binding >> document. >> For instance that for a foo device there’s a mandatory interrupt property. >> >> Simplified >> >> interrupt = ; >> >> Binding validation would ‘catch’ errors like assigning a string or not >> having the >> interrupt property available. >> >> SoC level validation would list the available interrupt number that a given >> SoC would support for that device. >> >> For example that interrupt can only take the values 10 or 99 in a given SoC. >> >> Board level validation would narrow this down even further to a value of 10 >> for >> a given board model. > >> Similar revision level validation would place further restriction on the >> allowed >> configuration. >> >> Finally application specific validation could place restriction based on the >> intended >> application that piece of hardware is used for. For instance devices that >> should not >> exceed a given power budget would have restrictions on the clock frequency >> of the processor >> or bus frequencies etc. > > This doesn't help. In order to do this, the validator would need > information that's essentially equivalent to the content of DT, at > which point there's no point to the DT at all - and you're left with > the problem of validating the information that the validator has. > That would be the case if hardware IP only has a single way to be configured. The industry standard nowadays is picking reusable IP blocks and integrating them together in an SoC. The driver and the binding is common for every platform that uses that block, but the allowed configuration varies according to what the hardware people uses in a given instance. > Fundamentally a validator that's useful *cannot* tell the difference > between a correct tree and one which _could_ be correct for some > theoretical hardware, but isn't for this particular hardware. > That’s why there’s reason for a nested hierarchy of bindings IMO. Completeness of validation schemes can be a differentiating factor when choosing parts for hardware design. They would sure cut down development time. > -- > David Gibson | I'll have my music baroque, and my code > david AT gibson.dropbear.id.au| minimalist, thank you. NOT _the_ > _other_ > | _way_ _around_! > http://www.ozlabs.org/~dgibson Regards — Pantelis
Re: [RFC] yamldt v0.5, now a DTS compiler too
On Mon, Oct 09, 2017 at 06:07:28PM +0300, Pantelis Antoniou wrote: > Hi David, > > > On Oct 9, 2017, at 03:00 , David Gibsonwrote: > > > > On Sun, Oct 08, 2017 at 04:08:03PM -0700, Frank Rowand wrote: > >> On 10/07/17 03:23, Pantelis Antoniou wrote: > >>> Hi Rob, > >>> > On Oct 6, 2017, at 16:55 , Rob Herring wrote: > > On Tue, Oct 3, 2017 at 12:39 PM, Pantelis Antoniou > wrote: > > Hi Rob, > >> > >> < snip > > >> > > eBPF is portable, can be serialized after compiling in the schema file > > and can be executed in the kernel. > > Executing in the kernel is a non-goal for me. > >> > >> Executing in the kernel is an anti-goal for me. > >> > >> We are trying to reduce the device tree footprint inside the kernel, > >> not increase it. > >> > >> 99.99% of the validation should be possible statically, in the compile > >> phase. > >> > >> > > By stripping out all documentation related properties and nodes keeping > > only the compiled filters you can generate a dtb blob that passed to > > kernel can be used for verification of all runtime changes in the > > kernel's live tree. eBPF is enforcing an execution model that is 'safe' > > so we can be sure that no foul play is possible. > >> > >> Run time changes can be assumed correct (short of bugs in the overlay > >> application code), if the base tree is validated, the overlay is validated, > >> and the interface between the live tree and the overlay is a > >> connector. > > > > In addition, no amount of schema validation can really protect the > > kernel from a bad DT. Even if the schemas can 100% verify that the DT > > is "syntactically" correct, which is ambitious, it can't protect > > against a DT which is in the right form, but contains information that > > is simply wrong for the hardware in question. That can stuff the > > kernel at least as easily as an incorrectly formatted DT. > > > > I disagree. > > There are multiple levels of validation. For now we’re only talking about > binding validation. There can be SoC level validation, board level validation, > revision level validation and finally application specific validation. > > Binding validation is making sure properties/nodes follow the binding > document. > For instance that for a foo device there’s a mandatory interrupt property. > > Simplified > > interrupt = ; > > Binding validation would ‘catch’ errors like assigning a string or not having > the > interrupt property available. > > SoC level validation would list the available interrupt number that a given > SoC would support for that device. > > For example that interrupt can only take the values 10 or 99 in a given SoC. > > Board level validation would narrow this down even further to a value of 10 > for > a given board model. > Similar revision level validation would place further restriction on the > allowed > configuration. > > Finally application specific validation could place restriction based on the > intended > application that piece of hardware is used for. For instance devices that > should not > exceed a given power budget would have restrictions on the clock frequency of > the processor > or bus frequencies etc. This doesn't help. In order to do this, the validator would need information that's essentially equivalent to the content of DT, at which point there's no point to the DT at all - and you're left with the problem of validating the information that the validator has. Fundamentally a validator that's useful *cannot* tell the difference between a correct tree and one which _could_ be correct for some theoretical hardware, but isn't for this particular hardware. -- David Gibson| I'll have my music baroque, and my code david AT gibson.dropbear.id.au | minimalist, thank you. NOT _the_ _other_ | _way_ _around_! http://www.ozlabs.org/~dgibson signature.asc Description: PGP signature
Re: [RFC] yamldt v0.5, now a DTS compiler too
On Mon, Oct 09, 2017 at 06:07:28PM +0300, Pantelis Antoniou wrote: > Hi David, > > > On Oct 9, 2017, at 03:00 , David Gibson wrote: > > > > On Sun, Oct 08, 2017 at 04:08:03PM -0700, Frank Rowand wrote: > >> On 10/07/17 03:23, Pantelis Antoniou wrote: > >>> Hi Rob, > >>> > On Oct 6, 2017, at 16:55 , Rob Herring wrote: > > On Tue, Oct 3, 2017 at 12:39 PM, Pantelis Antoniou > wrote: > > Hi Rob, > >> > >> < snip > > >> > > eBPF is portable, can be serialized after compiling in the schema file > > and can be executed in the kernel. > > Executing in the kernel is a non-goal for me. > >> > >> Executing in the kernel is an anti-goal for me. > >> > >> We are trying to reduce the device tree footprint inside the kernel, > >> not increase it. > >> > >> 99.99% of the validation should be possible statically, in the compile > >> phase. > >> > >> > > By stripping out all documentation related properties and nodes keeping > > only the compiled filters you can generate a dtb blob that passed to > > kernel can be used for verification of all runtime changes in the > > kernel's live tree. eBPF is enforcing an execution model that is 'safe' > > so we can be sure that no foul play is possible. > >> > >> Run time changes can be assumed correct (short of bugs in the overlay > >> application code), if the base tree is validated, the overlay is validated, > >> and the interface between the live tree and the overlay is a > >> connector. > > > > In addition, no amount of schema validation can really protect the > > kernel from a bad DT. Even if the schemas can 100% verify that the DT > > is "syntactically" correct, which is ambitious, it can't protect > > against a DT which is in the right form, but contains information that > > is simply wrong for the hardware in question. That can stuff the > > kernel at least as easily as an incorrectly formatted DT. > > > > I disagree. > > There are multiple levels of validation. For now we’re only talking about > binding validation. There can be SoC level validation, board level validation, > revision level validation and finally application specific validation. > > Binding validation is making sure properties/nodes follow the binding > document. > For instance that for a foo device there’s a mandatory interrupt property. > > Simplified > > interrupt = ; > > Binding validation would ‘catch’ errors like assigning a string or not having > the > interrupt property available. > > SoC level validation would list the available interrupt number that a given > SoC would support for that device. > > For example that interrupt can only take the values 10 or 99 in a given SoC. > > Board level validation would narrow this down even further to a value of 10 > for > a given board model. > Similar revision level validation would place further restriction on the > allowed > configuration. > > Finally application specific validation could place restriction based on the > intended > application that piece of hardware is used for. For instance devices that > should not > exceed a given power budget would have restrictions on the clock frequency of > the processor > or bus frequencies etc. This doesn't help. In order to do this, the validator would need information that's essentially equivalent to the content of DT, at which point there's no point to the DT at all - and you're left with the problem of validating the information that the validator has. Fundamentally a validator that's useful *cannot* tell the difference between a correct tree and one which _could_ be correct for some theoretical hardware, but isn't for this particular hardware. -- David Gibson| I'll have my music baroque, and my code david AT gibson.dropbear.id.au | minimalist, thank you. NOT _the_ _other_ | _way_ _around_! http://www.ozlabs.org/~dgibson signature.asc Description: PGP signature
Re: [RFC] yamldt v0.5, now a DTS compiler too
Hi David, > On Oct 9, 2017, at 03:00 , David Gibsonwrote: > > On Sun, Oct 08, 2017 at 04:08:03PM -0700, Frank Rowand wrote: >> On 10/07/17 03:23, Pantelis Antoniou wrote: >>> Hi Rob, >>> On Oct 6, 2017, at 16:55 , Rob Herring wrote: On Tue, Oct 3, 2017 at 12:39 PM, Pantelis Antoniou wrote: > Hi Rob, >> >> < snip > >> > eBPF is portable, can be serialized after compiling in the schema file > and can be executed in the kernel. Executing in the kernel is a non-goal for me. >> >> Executing in the kernel is an anti-goal for me. >> >> We are trying to reduce the device tree footprint inside the kernel, >> not increase it. >> >> 99.99% of the validation should be possible statically, in the compile >> phase. >> >> > By stripping out all documentation related properties and nodes keeping > only the compiled filters you can generate a dtb blob that passed to > kernel can be used for verification of all runtime changes in the > kernel's live tree. eBPF is enforcing an execution model that is 'safe' > so we can be sure that no foul play is possible. >> >> Run time changes can be assumed correct (short of bugs in the overlay >> application code), if the base tree is validated, the overlay is validated, >> and the interface between the live tree and the overlay is a >> connector. > > In addition, no amount of schema validation can really protect the > kernel from a bad DT. Even if the schemas can 100% verify that the DT > is "syntactically" correct, which is ambitious, it can't protect > against a DT which is in the right form, but contains information that > is simply wrong for the hardware in question. That can stuff the > kernel at least as easily as an incorrectly formatted DT. > I disagree. There are multiple levels of validation. For now we’re only talking about binding validation. There can be SoC level validation, board level validation, revision level validation and finally application specific validation. Binding validation is making sure properties/nodes follow the binding document. For instance that for a foo device there’s a mandatory interrupt property. Simplified interrupt = ; Binding validation would ‘catch’ errors like assigning a string or not having the interrupt property available. SoC level validation would list the available interrupt number that a given SoC would support for that device. For example that interrupt can only take the values 10 or 99 in a given SoC. Board level validation would narrow this down even further to a value of 10 for a given board model. Similar revision level validation would place further restriction on the allowed configuration. Finally application specific validation could place restriction based on the intended application that piece of hardware is used for. For instance devices that should not exceed a given power budget would have restrictions on the clock frequency of the processor or bus frequencies etc. > -- > David Gibson | I'll have my music baroque, and my code > david AT gibson.dropbear.id.au| minimalist, thank you. NOT _the_ > _other_ > | _way_ _around_! > http://www.ozlabs.org/~dgibson Regards — Pantelis
Re: [RFC] yamldt v0.5, now a DTS compiler too
Hi David, > On Oct 9, 2017, at 03:00 , David Gibson wrote: > > On Sun, Oct 08, 2017 at 04:08:03PM -0700, Frank Rowand wrote: >> On 10/07/17 03:23, Pantelis Antoniou wrote: >>> Hi Rob, >>> On Oct 6, 2017, at 16:55 , Rob Herring wrote: On Tue, Oct 3, 2017 at 12:39 PM, Pantelis Antoniou wrote: > Hi Rob, >> >> < snip > >> > eBPF is portable, can be serialized after compiling in the schema file > and can be executed in the kernel. Executing in the kernel is a non-goal for me. >> >> Executing in the kernel is an anti-goal for me. >> >> We are trying to reduce the device tree footprint inside the kernel, >> not increase it. >> >> 99.99% of the validation should be possible statically, in the compile >> phase. >> >> > By stripping out all documentation related properties and nodes keeping > only the compiled filters you can generate a dtb blob that passed to > kernel can be used for verification of all runtime changes in the > kernel's live tree. eBPF is enforcing an execution model that is 'safe' > so we can be sure that no foul play is possible. >> >> Run time changes can be assumed correct (short of bugs in the overlay >> application code), if the base tree is validated, the overlay is validated, >> and the interface between the live tree and the overlay is a >> connector. > > In addition, no amount of schema validation can really protect the > kernel from a bad DT. Even if the schemas can 100% verify that the DT > is "syntactically" correct, which is ambitious, it can't protect > against a DT which is in the right form, but contains information that > is simply wrong for the hardware in question. That can stuff the > kernel at least as easily as an incorrectly formatted DT. > I disagree. There are multiple levels of validation. For now we’re only talking about binding validation. There can be SoC level validation, board level validation, revision level validation and finally application specific validation. Binding validation is making sure properties/nodes follow the binding document. For instance that for a foo device there’s a mandatory interrupt property. Simplified interrupt = ; Binding validation would ‘catch’ errors like assigning a string or not having the interrupt property available. SoC level validation would list the available interrupt number that a given SoC would support for that device. For example that interrupt can only take the values 10 or 99 in a given SoC. Board level validation would narrow this down even further to a value of 10 for a given board model. Similar revision level validation would place further restriction on the allowed configuration. Finally application specific validation could place restriction based on the intended application that piece of hardware is used for. For instance devices that should not exceed a given power budget would have restrictions on the clock frequency of the processor or bus frequencies etc. > -- > David Gibson | I'll have my music baroque, and my code > david AT gibson.dropbear.id.au| minimalist, thank you. NOT _the_ > _other_ > | _way_ _around_! > http://www.ozlabs.org/~dgibson Regards — Pantelis
Re: [RFC] yamldt v0.5, now a DTS compiler too
Hi Frank, > On Oct 9, 2017, at 02:08 , Frank Rowandwrote: > > On 10/07/17 03:23, Pantelis Antoniou wrote: >> Hi Rob, >> >>> On Oct 6, 2017, at 16:55 , Rob Herring wrote: >>> >>> On Tue, Oct 3, 2017 at 12:39 PM, Pantelis Antoniou >>> wrote: Hi Rob, > > < snip > > eBPF is portable, can be serialized after compiling in the schema file and can be executed in the kernel. >>> >>> Executing in the kernel is a non-goal for me. > > Executing in the kernel is an anti-goal for me. > > We are trying to reduce the device tree footprint inside the kernel, > not increase it. > > 99.99% of the validation should be possible statically, in the compile > phase. > That’s not possible when you dynamically alter the tree at runtime. > By stripping out all documentation related properties and nodes keeping only the compiled filters you can generate a dtb blob that passed to kernel can be used for verification of all runtime changes in the kernel's live tree. eBPF is enforcing an execution model that is 'safe' so we can be sure that no foul play is possible. > > Run time changes can be assumed correct (short of bugs in the overlay > application code), if the base tree is validated, the overlay is validated, > and the interface between the live tree and the overlay is a connector. > You can validate the base tree statically but not the overlays. In fact a large percentage of overlay usage simply modifies a status property to turn on or off a device. There is nothing to validate in such case. The portable connector is still a long ways off and I haven’t seen anything that could handle something trickier that a few GPIOs and I2C or SPI busses. My goal is something that works covering all the cases without any surprising gotchas. > >>> Humm, if you wanted to ensure dtb's are safe, I'd think that we just >>> sign them like you would for the kernel or modules. >>> >> >> That’s a problem when deploying; the runtime validation is for making sure >> developers don’t get bogged down chasing problems when working on their >> own platforms/drivers. >> >> We have absolutely zero checks for stopping badly configured DT blobs >> hanging the kernel. With runtime validation a bug that might take a few >> days to figure out can be cut down to a few minutes. > > Same reply as above. > > That means that you can a) run it at boot-time, verifying the dtb blob passed by the bootloader for errors (potentially disabling devices that their nodes fail) and b) run it when applying overlays to reject any that result in an invalid tree. >>> >>> Let's get verification at build time working first, then we can worry >>> about something like this. > > < snip > > Right, let’s get build verification working first. > -Frank Regards — Pantelis
Re: [RFC] yamldt v0.5, now a DTS compiler too
Hi Frank, > On Oct 9, 2017, at 02:08 , Frank Rowand wrote: > > On 10/07/17 03:23, Pantelis Antoniou wrote: >> Hi Rob, >> >>> On Oct 6, 2017, at 16:55 , Rob Herring wrote: >>> >>> On Tue, Oct 3, 2017 at 12:39 PM, Pantelis Antoniou >>> wrote: Hi Rob, > > < snip > > eBPF is portable, can be serialized after compiling in the schema file and can be executed in the kernel. >>> >>> Executing in the kernel is a non-goal for me. > > Executing in the kernel is an anti-goal for me. > > We are trying to reduce the device tree footprint inside the kernel, > not increase it. > > 99.99% of the validation should be possible statically, in the compile > phase. > That’s not possible when you dynamically alter the tree at runtime. > By stripping out all documentation related properties and nodes keeping only the compiled filters you can generate a dtb blob that passed to kernel can be used for verification of all runtime changes in the kernel's live tree. eBPF is enforcing an execution model that is 'safe' so we can be sure that no foul play is possible. > > Run time changes can be assumed correct (short of bugs in the overlay > application code), if the base tree is validated, the overlay is validated, > and the interface between the live tree and the overlay is a connector. > You can validate the base tree statically but not the overlays. In fact a large percentage of overlay usage simply modifies a status property to turn on or off a device. There is nothing to validate in such case. The portable connector is still a long ways off and I haven’t seen anything that could handle something trickier that a few GPIOs and I2C or SPI busses. My goal is something that works covering all the cases without any surprising gotchas. > >>> Humm, if you wanted to ensure dtb's are safe, I'd think that we just >>> sign them like you would for the kernel or modules. >>> >> >> That’s a problem when deploying; the runtime validation is for making sure >> developers don’t get bogged down chasing problems when working on their >> own platforms/drivers. >> >> We have absolutely zero checks for stopping badly configured DT blobs >> hanging the kernel. With runtime validation a bug that might take a few >> days to figure out can be cut down to a few minutes. > > Same reply as above. > > That means that you can a) run it at boot-time, verifying the dtb blob passed by the bootloader for errors (potentially disabling devices that their nodes fail) and b) run it when applying overlays to reject any that result in an invalid tree. >>> >>> Let's get verification at build time working first, then we can worry >>> about something like this. > > < snip > > Right, let’s get build verification working first. > -Frank Regards — Pantelis
Re: [RFC] yamldt v0.5, now a DTS compiler too
On Sun, Oct 08, 2017 at 04:08:03PM -0700, Frank Rowand wrote: > On 10/07/17 03:23, Pantelis Antoniou wrote: > > Hi Rob, > > > >> On Oct 6, 2017, at 16:55 , Rob Herringwrote: > >> > >> On Tue, Oct 3, 2017 at 12:39 PM, Pantelis Antoniou > >> wrote: > >>> Hi Rob, > > < snip > > > >>> eBPF is portable, can be serialized after compiling in the schema file > >>> and can be executed in the kernel. > >> > >> Executing in the kernel is a non-goal for me. > > Executing in the kernel is an anti-goal for me. > > We are trying to reduce the device tree footprint inside the kernel, > not increase it. > > 99.99% of the validation should be possible statically, in the compile > phase. > > > >>> By stripping out all documentation related properties and nodes keeping > >>> only the compiled filters you can generate a dtb blob that passed to > >>> kernel can be used for verification of all runtime changes in the > >>> kernel's live tree. eBPF is enforcing an execution model that is 'safe' > >>> so we can be sure that no foul play is possible. > > Run time changes can be assumed correct (short of bugs in the overlay > application code), if the base tree is validated, the overlay is validated, > and the interface between the live tree and the overlay is a > connector. In addition, no amount of schema validation can really protect the kernel from a bad DT. Even if the schemas can 100% verify that the DT is "syntactically" correct, which is ambitious, it can't protect against a DT which is in the right form, but contains information that is simply wrong for the hardware in question. That can stuff the kernel at least as easily as an incorrectly formatted DT. -- David Gibson| I'll have my music baroque, and my code david AT gibson.dropbear.id.au | minimalist, thank you. NOT _the_ _other_ | _way_ _around_! http://www.ozlabs.org/~dgibson signature.asc Description: PGP signature
Re: [RFC] yamldt v0.5, now a DTS compiler too
On Sun, Oct 08, 2017 at 04:08:03PM -0700, Frank Rowand wrote: > On 10/07/17 03:23, Pantelis Antoniou wrote: > > Hi Rob, > > > >> On Oct 6, 2017, at 16:55 , Rob Herring wrote: > >> > >> On Tue, Oct 3, 2017 at 12:39 PM, Pantelis Antoniou > >> wrote: > >>> Hi Rob, > > < snip > > > >>> eBPF is portable, can be serialized after compiling in the schema file > >>> and can be executed in the kernel. > >> > >> Executing in the kernel is a non-goal for me. > > Executing in the kernel is an anti-goal for me. > > We are trying to reduce the device tree footprint inside the kernel, > not increase it. > > 99.99% of the validation should be possible statically, in the compile > phase. > > > >>> By stripping out all documentation related properties and nodes keeping > >>> only the compiled filters you can generate a dtb blob that passed to > >>> kernel can be used for verification of all runtime changes in the > >>> kernel's live tree. eBPF is enforcing an execution model that is 'safe' > >>> so we can be sure that no foul play is possible. > > Run time changes can be assumed correct (short of bugs in the overlay > application code), if the base tree is validated, the overlay is validated, > and the interface between the live tree and the overlay is a > connector. In addition, no amount of schema validation can really protect the kernel from a bad DT. Even if the schemas can 100% verify that the DT is "syntactically" correct, which is ambitious, it can't protect against a DT which is in the right form, but contains information that is simply wrong for the hardware in question. That can stuff the kernel at least as easily as an incorrectly formatted DT. -- David Gibson| I'll have my music baroque, and my code david AT gibson.dropbear.id.au | minimalist, thank you. NOT _the_ _other_ | _way_ _around_! http://www.ozlabs.org/~dgibson signature.asc Description: PGP signature
Re: [RFC] yamldt v0.5, now a DTS compiler too
On 10/07/17 03:23, Pantelis Antoniou wrote: > Hi Rob, > >> On Oct 6, 2017, at 16:55 , Rob Herringwrote: >> >> On Tue, Oct 3, 2017 at 12:39 PM, Pantelis Antoniou >> wrote: >>> Hi Rob, < snip > >>> eBPF is portable, can be serialized after compiling in the schema file >>> and can be executed in the kernel. >> >> Executing in the kernel is a non-goal for me. Executing in the kernel is an anti-goal for me. We are trying to reduce the device tree footprint inside the kernel, not increase it. 99.99% of the validation should be possible statically, in the compile phase. >>> By stripping out all documentation related properties and nodes keeping >>> only the compiled filters you can generate a dtb blob that passed to >>> kernel can be used for verification of all runtime changes in the >>> kernel's live tree. eBPF is enforcing an execution model that is 'safe' >>> so we can be sure that no foul play is possible. Run time changes can be assumed correct (short of bugs in the overlay application code), if the base tree is validated, the overlay is validated, and the interface between the live tree and the overlay is a connector. >> Humm, if you wanted to ensure dtb's are safe, I'd think that we just >> sign them like you would for the kernel or modules. >> > > That’s a problem when deploying; the runtime validation is for making sure > developers don’t get bogged down chasing problems when working on their > own platforms/drivers. > > We have absolutely zero checks for stopping badly configured DT blobs > hanging the kernel. With runtime validation a bug that might take a few > days to figure out can be cut down to a few minutes. Same reply as above. >>> That means that you can a) run it at boot-time, verifying the dtb blob >>> passed by the bootloader for errors (potentially disabling devices >>> that their nodes fail) and b) run it when applying overlays to reject >>> any that result in an invalid tree. >> >> Let's get verification at build time working first, then we can worry >> about something like this. < snip > -Frank
Re: [RFC] yamldt v0.5, now a DTS compiler too
On 10/07/17 03:23, Pantelis Antoniou wrote: > Hi Rob, > >> On Oct 6, 2017, at 16:55 , Rob Herring wrote: >> >> On Tue, Oct 3, 2017 at 12:39 PM, Pantelis Antoniou >> wrote: >>> Hi Rob, < snip > >>> eBPF is portable, can be serialized after compiling in the schema file >>> and can be executed in the kernel. >> >> Executing in the kernel is a non-goal for me. Executing in the kernel is an anti-goal for me. We are trying to reduce the device tree footprint inside the kernel, not increase it. 99.99% of the validation should be possible statically, in the compile phase. >>> By stripping out all documentation related properties and nodes keeping >>> only the compiled filters you can generate a dtb blob that passed to >>> kernel can be used for verification of all runtime changes in the >>> kernel's live tree. eBPF is enforcing an execution model that is 'safe' >>> so we can be sure that no foul play is possible. Run time changes can be assumed correct (short of bugs in the overlay application code), if the base tree is validated, the overlay is validated, and the interface between the live tree and the overlay is a connector. >> Humm, if you wanted to ensure dtb's are safe, I'd think that we just >> sign them like you would for the kernel or modules. >> > > That’s a problem when deploying; the runtime validation is for making sure > developers don’t get bogged down chasing problems when working on their > own platforms/drivers. > > We have absolutely zero checks for stopping badly configured DT blobs > hanging the kernel. With runtime validation a bug that might take a few > days to figure out can be cut down to a few minutes. Same reply as above. >>> That means that you can a) run it at boot-time, verifying the dtb blob >>> passed by the bootloader for errors (potentially disabling devices >>> that their nodes fail) and b) run it when applying overlays to reject >>> any that result in an invalid tree. >> >> Let's get verification at build time working first, then we can worry >> about something like this. < snip > -Frank
Re: [RFC] yamldt v0.5, now a DTS compiler too
Hi Rob, > On Oct 6, 2017, at 16:55 , Rob Herringwrote: > > On Tue, Oct 3, 2017 at 12:39 PM, Pantelis Antoniou > wrote: >> Hi Rob, >> >> On Tue, 2017-10-03 at 12:13 -0500, Rob Herring wrote: >>> On Tue, Oct 3, 2017 at 9:13 AM, Pantelis Antoniou >>> wrote: Hi Rob, On Tue, 2017-10-03 at 08:18 -0500, Rob Herring wrote: > On Mon, Oct 2, 2017 at 2:46 PM, Pantelis Antoniou > wrote: >> Hi Rob, >> >> On Sun, 2017-10-01 at 17:00 -0500, Rob Herring wrote: >>> On Thu, Sep 28, 2017 at 2:58 PM, Pantelis Antoniou >>> wrote: Hello again, Significant progress has been made on yamldt and is now capable of not only generating yaml from DTS source but also compiling DTS sources and being almost fully compatible with DTC. >>> >>> Can you quantify "almost"? >>> Compiling the kernel's DTBs using yamldt is as simple as using a DTC=yamldt. >>> >>> Good. >>> Error reporting is accurate and validation against a YAML based schema works as well. In a short while I will begin posting patches with fixes on bindings and DTS files in the kernel. >>> >>> What I would like to see is the schema format posted for review. >>> >> >> I'm including the skeleton.yaml binding which is the template for >> the bindings and a board-example.yaml binding for a top level binding. >> >>> I would also like to see the bindings for top-level compatible strings >>> (aka boards) as an example. That's something that's simple enough that >>> I'd think we could agree on a format and start moving towards defining >>> board bindings that way. >>> >> >> Note there is some line wrapping I'm including a link >> to the github repo of the files: >> >> >> The skeleton.yaml >> >> https://raw.githubusercontent.com/pantoniou/yamldt/master/validate/bindings/skeleton.yaml >> >> %YAML 1.1 >> --- >> # The name of the binding is first >> # The anchor is put there for use by others >> skeleton: > > This and "id" seem redundant. > Indeed. >>> >>> One other thing, "skeleton" is a bit weird as a key. It can't be >>> validated. All the other keys are standard words. I could write >>> "skeloton" by mistake and I guess I'd only find the mistake when >>> something inherits it. That's somewhat true with id, but we can at >>> least check "id" is present and that it's value is unique among all >>> other id values. >>> >> >> We can keep id and check that it matches the name of the enclosing node. >> That way you can be sure that there's no error. >> >> But it's a bit weird cause this is similar to declaring a function name >> with a typo. You won't find out until you use it. >> >> version: 1 >> >> id: skel-device >> >> title: > >>Skeleton Device >> >> maintainer: >>name: Skeleton Person >> >> description: > >>The Skeleton Device binding represents the SK11 device produced by >>the Skeleton Corporation. The binding can also support compatible >>clones made by second source vendors. >> >> # The class is an optional property that declares this >> # binding as part of a larger set >> # Multiple definitions are possible >> class: [ device, spi-device ] >> >> # This binding inherits property characteristics from the generic >> # spi-slave binding >> # Note that the notation is standard yaml reference >> inherits: *spi-slave >> >> # virtual bindings do not generate checkers >> virtual: true > > virtual is an overloaded term. > OK, what term should I use that this binding should not be instantiated as a checker, only be used by other bindings when inherited? >>> >>> checks: true? >>> >>> I'd really like to avoid having to decide and drop this, but I don't >>> really get why it is needed. >>> >> >> It is needed because otherwise checker filters will be generated for >> the template bindings that while they won't be executed they will be >> compiled and take up space in the schema. > > Size is not a problem I have. That can come later if needed. > >> # each property is defined by each name >> properties: >> >># The compatible property is a reserved name. The type is always >> "string" >># and should not be repeated device binding. >>compatible: >> category: required# required property >> type: strseq # is a sequence of strings >> >> description: > >>FX11 is a clone of the original SK11 device >> >> # v
Re: [RFC] yamldt v0.5, now a DTS compiler too
Hi Rob, > On Oct 6, 2017, at 16:55 , Rob Herring wrote: > > On Tue, Oct 3, 2017 at 12:39 PM, Pantelis Antoniou > wrote: >> Hi Rob, >> >> On Tue, 2017-10-03 at 12:13 -0500, Rob Herring wrote: >>> On Tue, Oct 3, 2017 at 9:13 AM, Pantelis Antoniou >>> wrote: Hi Rob, On Tue, 2017-10-03 at 08:18 -0500, Rob Herring wrote: > On Mon, Oct 2, 2017 at 2:46 PM, Pantelis Antoniou > wrote: >> Hi Rob, >> >> On Sun, 2017-10-01 at 17:00 -0500, Rob Herring wrote: >>> On Thu, Sep 28, 2017 at 2:58 PM, Pantelis Antoniou >>> wrote: Hello again, Significant progress has been made on yamldt and is now capable of not only generating yaml from DTS source but also compiling DTS sources and being almost fully compatible with DTC. >>> >>> Can you quantify "almost"? >>> Compiling the kernel's DTBs using yamldt is as simple as using a DTC=yamldt. >>> >>> Good. >>> Error reporting is accurate and validation against a YAML based schema works as well. In a short while I will begin posting patches with fixes on bindings and DTS files in the kernel. >>> >>> What I would like to see is the schema format posted for review. >>> >> >> I'm including the skeleton.yaml binding which is the template for >> the bindings and a board-example.yaml binding for a top level binding. >> >>> I would also like to see the bindings for top-level compatible strings >>> (aka boards) as an example. That's something that's simple enough that >>> I'd think we could agree on a format and start moving towards defining >>> board bindings that way. >>> >> >> Note there is some line wrapping I'm including a link >> to the github repo of the files: >> >> >> The skeleton.yaml >> >> https://raw.githubusercontent.com/pantoniou/yamldt/master/validate/bindings/skeleton.yaml >> >> %YAML 1.1 >> --- >> # The name of the binding is first >> # The anchor is put there for use by others >> skeleton: > > This and "id" seem redundant. > Indeed. >>> >>> One other thing, "skeleton" is a bit weird as a key. It can't be >>> validated. All the other keys are standard words. I could write >>> "skeloton" by mistake and I guess I'd only find the mistake when >>> something inherits it. That's somewhat true with id, but we can at >>> least check "id" is present and that it's value is unique among all >>> other id values. >>> >> >> We can keep id and check that it matches the name of the enclosing node. >> That way you can be sure that there's no error. >> >> But it's a bit weird cause this is similar to declaring a function name >> with a typo. You won't find out until you use it. >> >> version: 1 >> >> id: skel-device >> >> title: > >>Skeleton Device >> >> maintainer: >>name: Skeleton Person >> >> description: > >>The Skeleton Device binding represents the SK11 device produced by >>the Skeleton Corporation. The binding can also support compatible >>clones made by second source vendors. >> >> # The class is an optional property that declares this >> # binding as part of a larger set >> # Multiple definitions are possible >> class: [ device, spi-device ] >> >> # This binding inherits property characteristics from the generic >> # spi-slave binding >> # Note that the notation is standard yaml reference >> inherits: *spi-slave >> >> # virtual bindings do not generate checkers >> virtual: true > > virtual is an overloaded term. > OK, what term should I use that this binding should not be instantiated as a checker, only be used by other bindings when inherited? >>> >>> checks: true? >>> >>> I'd really like to avoid having to decide and drop this, but I don't >>> really get why it is needed. >>> >> >> It is needed because otherwise checker filters will be generated for >> the template bindings that while they won't be executed they will be >> compiled and take up space in the schema. > > Size is not a problem I have. That can come later if needed. > >> # each property is defined by each name >> properties: >> >># The compatible property is a reserved name. The type is always >> "string" >># and should not be repeated device binding. >>compatible: >> category: required# required property >> type: strseq # is a sequence of strings >> >> description: > >>FX11 is a clone of the original SK11 device >> >> # v is always the name of the value of the property >> # np is passed to the checker and is the current >> # node pointer. We can access properties and
Re: [RFC] yamldt v0.5, now a DTS compiler too
On Tue, Oct 3, 2017 at 12:39 PM, Pantelis Antoniouwrote: > Hi Rob, > > On Tue, 2017-10-03 at 12:13 -0500, Rob Herring wrote: >> On Tue, Oct 3, 2017 at 9:13 AM, Pantelis Antoniou >> wrote: >> > Hi Rob, >> > >> > On Tue, 2017-10-03 at 08:18 -0500, Rob Herring wrote: >> >> On Mon, Oct 2, 2017 at 2:46 PM, Pantelis Antoniou >> >> wrote: >> >> > Hi Rob, >> >> > >> >> > On Sun, 2017-10-01 at 17:00 -0500, Rob Herring wrote: >> >> >> On Thu, Sep 28, 2017 at 2:58 PM, Pantelis Antoniou >> >> >> wrote: >> >> >> > Hello again, >> >> >> > >> >> >> > Significant progress has been made on yamldt and is now capable of >> >> >> > not only generating yaml from DTS source but also compiling DTS >> >> >> > sources >> >> >> > and being almost fully compatible with DTC. >> >> >> >> >> >> Can you quantify "almost"? >> >> >> >> >> >> > Compiling the kernel's DTBs using yamldt is as simple as using a >> >> >> > DTC=yamldt. >> >> >> >> >> >> Good. >> >> >> >> >> >> > >> >> >> > Error reporting is accurate and validation against a YAML based >> >> >> > schema >> >> >> > works as well. In a short while I will begin posting patches with >> >> >> > fixes on bindings and DTS files in the kernel. >> >> >> >> >> >> What I would like to see is the schema format posted for review. >> >> >> >> >> > >> >> > I'm including the skeleton.yaml binding which is the template for >> >> > the bindings and a board-example.yaml binding for a top level binding. >> >> > >> >> >> I would also like to see the bindings for top-level compatible strings >> >> >> (aka boards) as an example. That's something that's simple enough that >> >> >> I'd think we could agree on a format and start moving towards defining >> >> >> board bindings that way. >> >> >> >> >> > >> >> > Note there is some line wrapping I'm including a link >> >> > to the github repo of the files: >> >> > >> >> > >> >> > The skeleton.yaml >> >> > >> >> > https://raw.githubusercontent.com/pantoniou/yamldt/master/validate/bindings/skeleton.yaml >> >> > >> >> > %YAML 1.1 >> >> > --- >> >> > # The name of the binding is first >> >> > # The anchor is put there for use by others >> >> > skeleton: >> >> >> >> This and "id" seem redundant. >> >> >> > >> > Indeed. >> >> One other thing, "skeleton" is a bit weird as a key. It can't be >> validated. All the other keys are standard words. I could write >> "skeloton" by mistake and I guess I'd only find the mistake when >> something inherits it. That's somewhat true with id, but we can at >> least check "id" is present and that it's value is unique among all >> other id values. >> > > We can keep id and check that it matches the name of the enclosing node. > That way you can be sure that there's no error. > > But it's a bit weird cause this is similar to declaring a function name > with a typo. You won't find out until you use it. > >> > >> >> > version: 1 >> >> > >> >> > id: skel-device >> >> > >> >> > title: > >> >> > Skeleton Device >> >> > >> >> > maintainer: >> >> > name: Skeleton Person >> >> > >> >> > description: > >> >> > The Skeleton Device binding represents the SK11 device produced by >> >> > the Skeleton Corporation. The binding can also support compatible >> >> > clones made by second source vendors. >> >> > >> >> > # The class is an optional property that declares this >> >> > # binding as part of a larger set >> >> > # Multiple definitions are possible >> >> > class: [ device, spi-device ] >> >> > >> >> > # This binding inherits property characteristics from the generic >> >> > # spi-slave binding >> >> > # Note that the notation is standard yaml reference >> >> > inherits: *spi-slave >> >> > >> >> > # virtual bindings do not generate checkers >> >> > virtual: true >> >> >> >> virtual is an overloaded term. >> >> >> > >> > OK, what term should I use that this binding should not be instantiated >> > as a checker, only be used by other bindings when inherited? >> >> checks: true? >> >> I'd really like to avoid having to decide and drop this, but I don't >> really get why it is needed. >> > > It is needed because otherwise checker filters will be generated for > the template bindings that while they won't be executed they will be > compiled and take up space in the schema. Size is not a problem I have. That can come later if needed. >> >> > # each property is defined by each name >> >> > properties: >> >> > >> >> > # The compatible property is a reserved name. The type is always >> >> > "string" >> >> > # and should not be repeated device binding. >> >> > compatible: >> >> > category: required# required property >> >> > type: strseq # is a sequence of strings >> >> > >> >> > description: > >> >> > FX11 is a clone of the original SK11 device >> >> > >> >> > # v is always the
Re: [RFC] yamldt v0.5, now a DTS compiler too
On Tue, Oct 3, 2017 at 12:39 PM, Pantelis Antoniou wrote: > Hi Rob, > > On Tue, 2017-10-03 at 12:13 -0500, Rob Herring wrote: >> On Tue, Oct 3, 2017 at 9:13 AM, Pantelis Antoniou >> wrote: >> > Hi Rob, >> > >> > On Tue, 2017-10-03 at 08:18 -0500, Rob Herring wrote: >> >> On Mon, Oct 2, 2017 at 2:46 PM, Pantelis Antoniou >> >> wrote: >> >> > Hi Rob, >> >> > >> >> > On Sun, 2017-10-01 at 17:00 -0500, Rob Herring wrote: >> >> >> On Thu, Sep 28, 2017 at 2:58 PM, Pantelis Antoniou >> >> >> wrote: >> >> >> > Hello again, >> >> >> > >> >> >> > Significant progress has been made on yamldt and is now capable of >> >> >> > not only generating yaml from DTS source but also compiling DTS >> >> >> > sources >> >> >> > and being almost fully compatible with DTC. >> >> >> >> >> >> Can you quantify "almost"? >> >> >> >> >> >> > Compiling the kernel's DTBs using yamldt is as simple as using a >> >> >> > DTC=yamldt. >> >> >> >> >> >> Good. >> >> >> >> >> >> > >> >> >> > Error reporting is accurate and validation against a YAML based >> >> >> > schema >> >> >> > works as well. In a short while I will begin posting patches with >> >> >> > fixes on bindings and DTS files in the kernel. >> >> >> >> >> >> What I would like to see is the schema format posted for review. >> >> >> >> >> > >> >> > I'm including the skeleton.yaml binding which is the template for >> >> > the bindings and a board-example.yaml binding for a top level binding. >> >> > >> >> >> I would also like to see the bindings for top-level compatible strings >> >> >> (aka boards) as an example. That's something that's simple enough that >> >> >> I'd think we could agree on a format and start moving towards defining >> >> >> board bindings that way. >> >> >> >> >> > >> >> > Note there is some line wrapping I'm including a link >> >> > to the github repo of the files: >> >> > >> >> > >> >> > The skeleton.yaml >> >> > >> >> > https://raw.githubusercontent.com/pantoniou/yamldt/master/validate/bindings/skeleton.yaml >> >> > >> >> > %YAML 1.1 >> >> > --- >> >> > # The name of the binding is first >> >> > # The anchor is put there for use by others >> >> > skeleton: >> >> >> >> This and "id" seem redundant. >> >> >> > >> > Indeed. >> >> One other thing, "skeleton" is a bit weird as a key. It can't be >> validated. All the other keys are standard words. I could write >> "skeloton" by mistake and I guess I'd only find the mistake when >> something inherits it. That's somewhat true with id, but we can at >> least check "id" is present and that it's value is unique among all >> other id values. >> > > We can keep id and check that it matches the name of the enclosing node. > That way you can be sure that there's no error. > > But it's a bit weird cause this is similar to declaring a function name > with a typo. You won't find out until you use it. > >> > >> >> > version: 1 >> >> > >> >> > id: skel-device >> >> > >> >> > title: > >> >> > Skeleton Device >> >> > >> >> > maintainer: >> >> > name: Skeleton Person >> >> > >> >> > description: > >> >> > The Skeleton Device binding represents the SK11 device produced by >> >> > the Skeleton Corporation. The binding can also support compatible >> >> > clones made by second source vendors. >> >> > >> >> > # The class is an optional property that declares this >> >> > # binding as part of a larger set >> >> > # Multiple definitions are possible >> >> > class: [ device, spi-device ] >> >> > >> >> > # This binding inherits property characteristics from the generic >> >> > # spi-slave binding >> >> > # Note that the notation is standard yaml reference >> >> > inherits: *spi-slave >> >> > >> >> > # virtual bindings do not generate checkers >> >> > virtual: true >> >> >> >> virtual is an overloaded term. >> >> >> > >> > OK, what term should I use that this binding should not be instantiated >> > as a checker, only be used by other bindings when inherited? >> >> checks: true? >> >> I'd really like to avoid having to decide and drop this, but I don't >> really get why it is needed. >> > > It is needed because otherwise checker filters will be generated for > the template bindings that while they won't be executed they will be > compiled and take up space in the schema. Size is not a problem I have. That can come later if needed. >> >> > # each property is defined by each name >> >> > properties: >> >> > >> >> > # The compatible property is a reserved name. The type is always >> >> > "string" >> >> > # and should not be repeated device binding. >> >> > compatible: >> >> > category: required# required property >> >> > type: strseq # is a sequence of strings >> >> > >> >> > description: > >> >> > FX11 is a clone of the original SK11 device >> >> > >> >> > # v is always the name of the value of the property >> >> > # np is passed to the checker and is the current >> >> > # node pointer. We can access
Re: [RFC] yamldt v0.5, now a DTS compiler too
Hi Rob, On Tue, 2017-10-03 at 12:13 -0500, Rob Herring wrote: > On Tue, Oct 3, 2017 at 9:13 AM, Pantelis Antoniou >wrote: > > Hi Rob, > > > > On Tue, 2017-10-03 at 08:18 -0500, Rob Herring wrote: > >> On Mon, Oct 2, 2017 at 2:46 PM, Pantelis Antoniou > >> wrote: > >> > Hi Rob, > >> > > >> > On Sun, 2017-10-01 at 17:00 -0500, Rob Herring wrote: > >> >> On Thu, Sep 28, 2017 at 2:58 PM, Pantelis Antoniou > >> >> wrote: > >> >> > Hello again, > >> >> > > >> >> > Significant progress has been made on yamldt and is now capable of > >> >> > not only generating yaml from DTS source but also compiling DTS > >> >> > sources > >> >> > and being almost fully compatible with DTC. > >> >> > >> >> Can you quantify "almost"? > >> >> > >> >> > Compiling the kernel's DTBs using yamldt is as simple as using a > >> >> > DTC=yamldt. > >> >> > >> >> Good. > >> >> > >> >> > > >> >> > Error reporting is accurate and validation against a YAML based schema > >> >> > works as well. In a short while I will begin posting patches with > >> >> > fixes on bindings and DTS files in the kernel. > >> >> > >> >> What I would like to see is the schema format posted for review. > >> >> > >> > > >> > I'm including the skeleton.yaml binding which is the template for > >> > the bindings and a board-example.yaml binding for a top level binding. > >> > > >> >> I would also like to see the bindings for top-level compatible strings > >> >> (aka boards) as an example. That's something that's simple enough that > >> >> I'd think we could agree on a format and start moving towards defining > >> >> board bindings that way. > >> >> > >> > > >> > Note there is some line wrapping I'm including a link > >> > to the github repo of the files: > >> > > >> > > >> > The skeleton.yaml > >> > > >> > https://raw.githubusercontent.com/pantoniou/yamldt/master/validate/bindings/skeleton.yaml > >> > > >> > %YAML 1.1 > >> > --- > >> > # The name of the binding is first > >> > # The anchor is put there for use by others > >> > skeleton: > >> > >> This and "id" seem redundant. > >> > > > > Indeed. > > One other thing, "skeleton" is a bit weird as a key. It can't be > validated. All the other keys are standard words. I could write > "skeloton" by mistake and I guess I'd only find the mistake when > something inherits it. That's somewhat true with id, but we can at > least check "id" is present and that it's value is unique among all > other id values. > We can keep id and check that it matches the name of the enclosing node. That way you can be sure that there's no error. But it's a bit weird cause this is similar to declaring a function name with a typo. You won't find out until you use it. > > > >> > version: 1 > >> > > >> > id: skel-device > >> > > >> > title: > > >> > Skeleton Device > >> > > >> > maintainer: > >> > name: Skeleton Person > >> > > >> > description: > > >> > The Skeleton Device binding represents the SK11 device produced by > >> > the Skeleton Corporation. The binding can also support compatible > >> > clones made by second source vendors. > >> > > >> > # The class is an optional property that declares this > >> > # binding as part of a larger set > >> > # Multiple definitions are possible > >> > class: [ device, spi-device ] > >> > > >> > # This binding inherits property characteristics from the generic > >> > # spi-slave binding > >> > # Note that the notation is standard yaml reference > >> > inherits: *spi-slave > >> > > >> > # virtual bindings do not generate checkers > >> > virtual: true > >> > >> virtual is an overloaded term. > >> > > > > OK, what term should I use that this binding should not be instantiated > > as a checker, only be used by other bindings when inherited? > > checks: true? > > I'd really like to avoid having to decide and drop this, but I don't > really get why it is needed. > It is needed because otherwise checker filters will be generated for the template bindings that while they won't be executed they will be compiled and take up space in the schema. > > > >> > > >> > # each property is defined by each name > >> > properties: > >> > > >> > # The compatible property is a reserved name. The type is always > >> > "string" > >> > # and should not be repeated device binding. > >> > compatible: > >> > category: required# required property > >> > type: strseq # is a sequence of strings > >> > > >> > description: > > >> > FX11 is a clone of the original SK11 device > >> > > >> > # v is always the name of the value of the property > >> > # np is passed to the checker and is the current > >> > # node pointer. We can access properties and call > >> > # methods that operate on them. > >> > # There can be multiple constraints, just put them > >> > # into a
Re: [RFC] yamldt v0.5, now a DTS compiler too
Hi Rob, On Tue, 2017-10-03 at 12:13 -0500, Rob Herring wrote: > On Tue, Oct 3, 2017 at 9:13 AM, Pantelis Antoniou > wrote: > > Hi Rob, > > > > On Tue, 2017-10-03 at 08:18 -0500, Rob Herring wrote: > >> On Mon, Oct 2, 2017 at 2:46 PM, Pantelis Antoniou > >> wrote: > >> > Hi Rob, > >> > > >> > On Sun, 2017-10-01 at 17:00 -0500, Rob Herring wrote: > >> >> On Thu, Sep 28, 2017 at 2:58 PM, Pantelis Antoniou > >> >> wrote: > >> >> > Hello again, > >> >> > > >> >> > Significant progress has been made on yamldt and is now capable of > >> >> > not only generating yaml from DTS source but also compiling DTS > >> >> > sources > >> >> > and being almost fully compatible with DTC. > >> >> > >> >> Can you quantify "almost"? > >> >> > >> >> > Compiling the kernel's DTBs using yamldt is as simple as using a > >> >> > DTC=yamldt. > >> >> > >> >> Good. > >> >> > >> >> > > >> >> > Error reporting is accurate and validation against a YAML based schema > >> >> > works as well. In a short while I will begin posting patches with > >> >> > fixes on bindings and DTS files in the kernel. > >> >> > >> >> What I would like to see is the schema format posted for review. > >> >> > >> > > >> > I'm including the skeleton.yaml binding which is the template for > >> > the bindings and a board-example.yaml binding for a top level binding. > >> > > >> >> I would also like to see the bindings for top-level compatible strings > >> >> (aka boards) as an example. That's something that's simple enough that > >> >> I'd think we could agree on a format and start moving towards defining > >> >> board bindings that way. > >> >> > >> > > >> > Note there is some line wrapping I'm including a link > >> > to the github repo of the files: > >> > > >> > > >> > The skeleton.yaml > >> > > >> > https://raw.githubusercontent.com/pantoniou/yamldt/master/validate/bindings/skeleton.yaml > >> > > >> > %YAML 1.1 > >> > --- > >> > # The name of the binding is first > >> > # The anchor is put there for use by others > >> > skeleton: > >> > >> This and "id" seem redundant. > >> > > > > Indeed. > > One other thing, "skeleton" is a bit weird as a key. It can't be > validated. All the other keys are standard words. I could write > "skeloton" by mistake and I guess I'd only find the mistake when > something inherits it. That's somewhat true with id, but we can at > least check "id" is present and that it's value is unique among all > other id values. > We can keep id and check that it matches the name of the enclosing node. That way you can be sure that there's no error. But it's a bit weird cause this is similar to declaring a function name with a typo. You won't find out until you use it. > > > >> > version: 1 > >> > > >> > id: skel-device > >> > > >> > title: > > >> > Skeleton Device > >> > > >> > maintainer: > >> > name: Skeleton Person > >> > > >> > description: > > >> > The Skeleton Device binding represents the SK11 device produced by > >> > the Skeleton Corporation. The binding can also support compatible > >> > clones made by second source vendors. > >> > > >> > # The class is an optional property that declares this > >> > # binding as part of a larger set > >> > # Multiple definitions are possible > >> > class: [ device, spi-device ] > >> > > >> > # This binding inherits property characteristics from the generic > >> > # spi-slave binding > >> > # Note that the notation is standard yaml reference > >> > inherits: *spi-slave > >> > > >> > # virtual bindings do not generate checkers > >> > virtual: true > >> > >> virtual is an overloaded term. > >> > > > > OK, what term should I use that this binding should not be instantiated > > as a checker, only be used by other bindings when inherited? > > checks: true? > > I'd really like to avoid having to decide and drop this, but I don't > really get why it is needed. > It is needed because otherwise checker filters will be generated for the template bindings that while they won't be executed they will be compiled and take up space in the schema. > > > >> > > >> > # each property is defined by each name > >> > properties: > >> > > >> > # The compatible property is a reserved name. The type is always > >> > "string" > >> > # and should not be repeated device binding. > >> > compatible: > >> > category: required# required property > >> > type: strseq # is a sequence of strings > >> > > >> > description: > > >> > FX11 is a clone of the original SK11 device > >> > > >> > # v is always the name of the value of the property > >> > # np is passed to the checker and is the current > >> > # node pointer. We can access properties and call > >> > # methods that operate on them. > >> > # There can be multiple constraints, just put them > >> > # into a sequence. > >> > # Note that the BASE("skel,sk11") form from the previous > >> > # binding will have to be
Re: [RFC] yamldt v0.5, now a DTS compiler too
On Tue, Oct 3, 2017 at 9:13 AM, Pantelis Antoniouwrote: > Hi Rob, > > On Tue, 2017-10-03 at 08:18 -0500, Rob Herring wrote: >> On Mon, Oct 2, 2017 at 2:46 PM, Pantelis Antoniou >> wrote: >> > Hi Rob, >> > >> > On Sun, 2017-10-01 at 17:00 -0500, Rob Herring wrote: >> >> On Thu, Sep 28, 2017 at 2:58 PM, Pantelis Antoniou >> >> wrote: >> >> > Hello again, >> >> > >> >> > Significant progress has been made on yamldt and is now capable of >> >> > not only generating yaml from DTS source but also compiling DTS sources >> >> > and being almost fully compatible with DTC. >> >> >> >> Can you quantify "almost"? >> >> >> >> > Compiling the kernel's DTBs using yamldt is as simple as using a >> >> > DTC=yamldt. >> >> >> >> Good. >> >> >> >> > >> >> > Error reporting is accurate and validation against a YAML based schema >> >> > works as well. In a short while I will begin posting patches with >> >> > fixes on bindings and DTS files in the kernel. >> >> >> >> What I would like to see is the schema format posted for review. >> >> >> > >> > I'm including the skeleton.yaml binding which is the template for >> > the bindings and a board-example.yaml binding for a top level binding. >> > >> >> I would also like to see the bindings for top-level compatible strings >> >> (aka boards) as an example. That's something that's simple enough that >> >> I'd think we could agree on a format and start moving towards defining >> >> board bindings that way. >> >> >> > >> > Note there is some line wrapping I'm including a link >> > to the github repo of the files: >> > >> > >> > The skeleton.yaml >> > >> > https://raw.githubusercontent.com/pantoniou/yamldt/master/validate/bindings/skeleton.yaml >> > >> > %YAML 1.1 >> > --- >> > # The name of the binding is first >> > # The anchor is put there for use by others >> > skeleton: >> >> This and "id" seem redundant. >> > > Indeed. One other thing, "skeleton" is a bit weird as a key. It can't be validated. All the other keys are standard words. I could write "skeloton" by mistake and I guess I'd only find the mistake when something inherits it. That's somewhat true with id, but we can at least check "id" is present and that it's value is unique among all other id values. > >> > version: 1 >> > >> > id: skel-device >> > >> > title: > >> > Skeleton Device >> > >> > maintainer: >> > name: Skeleton Person >> > >> > description: > >> > The Skeleton Device binding represents the SK11 device produced by >> > the Skeleton Corporation. The binding can also support compatible >> > clones made by second source vendors. >> > >> > # The class is an optional property that declares this >> > # binding as part of a larger set >> > # Multiple definitions are possible >> > class: [ device, spi-device ] >> > >> > # This binding inherits property characteristics from the generic >> > # spi-slave binding >> > # Note that the notation is standard yaml reference >> > inherits: *spi-slave >> > >> > # virtual bindings do not generate checkers >> > virtual: true >> >> virtual is an overloaded term. >> > > OK, what term should I use that this binding should not be instantiated > as a checker, only be used by other bindings when inherited? checks: true? I'd really like to avoid having to decide and drop this, but I don't really get why it is needed. > >> > >> > # each property is defined by each name >> > properties: >> > >> > # The compatible property is a reserved name. The type is always >> > "string" >> > # and should not be repeated device binding. >> > compatible: >> > category: required# required property >> > type: strseq # is a sequence of strings >> > >> > description: > >> > FX11 is a clone of the original SK11 device >> > >> > # v is always the name of the value of the property >> > # np is passed to the checker and is the current >> > # node pointer. We can access properties and call >> > # methods that operate on them. >> > # There can be multiple constraints, just put them >> > # into a sequence. >> > # Note that the BASE("skel,sk11") form from the previous >> > # binding will have to be reworked. >> > constraint: | >> > anystreq(v, "skel,sk11") || >> > anystreq(v, "faux,fx11") >> >> Constraints and logic ops were certainly not decided in the last >> attempt and I think will be the hardest part to define. I see you are >> using eBPF in the checker. Is that where anystreq comes from? >> > > Yes. The ebpf environment declares a number of methods that are executed > outside the ebpf sandbox. Check out > > https://raw.githubusercontent.com/pantoniou/yamldt/master/validate/schema/codegen.yaml > https://github.com/pantoniou/yamldt/blob/master/ebpf_dt.c I looked at this some a while back. It wasn't clear to me
Re: [RFC] yamldt v0.5, now a DTS compiler too
On Tue, Oct 3, 2017 at 9:13 AM, Pantelis Antoniou wrote: > Hi Rob, > > On Tue, 2017-10-03 at 08:18 -0500, Rob Herring wrote: >> On Mon, Oct 2, 2017 at 2:46 PM, Pantelis Antoniou >> wrote: >> > Hi Rob, >> > >> > On Sun, 2017-10-01 at 17:00 -0500, Rob Herring wrote: >> >> On Thu, Sep 28, 2017 at 2:58 PM, Pantelis Antoniou >> >> wrote: >> >> > Hello again, >> >> > >> >> > Significant progress has been made on yamldt and is now capable of >> >> > not only generating yaml from DTS source but also compiling DTS sources >> >> > and being almost fully compatible with DTC. >> >> >> >> Can you quantify "almost"? >> >> >> >> > Compiling the kernel's DTBs using yamldt is as simple as using a >> >> > DTC=yamldt. >> >> >> >> Good. >> >> >> >> > >> >> > Error reporting is accurate and validation against a YAML based schema >> >> > works as well. In a short while I will begin posting patches with >> >> > fixes on bindings and DTS files in the kernel. >> >> >> >> What I would like to see is the schema format posted for review. >> >> >> > >> > I'm including the skeleton.yaml binding which is the template for >> > the bindings and a board-example.yaml binding for a top level binding. >> > >> >> I would also like to see the bindings for top-level compatible strings >> >> (aka boards) as an example. That's something that's simple enough that >> >> I'd think we could agree on a format and start moving towards defining >> >> board bindings that way. >> >> >> > >> > Note there is some line wrapping I'm including a link >> > to the github repo of the files: >> > >> > >> > The skeleton.yaml >> > >> > https://raw.githubusercontent.com/pantoniou/yamldt/master/validate/bindings/skeleton.yaml >> > >> > %YAML 1.1 >> > --- >> > # The name of the binding is first >> > # The anchor is put there for use by others >> > skeleton: >> >> This and "id" seem redundant. >> > > Indeed. One other thing, "skeleton" is a bit weird as a key. It can't be validated. All the other keys are standard words. I could write "skeloton" by mistake and I guess I'd only find the mistake when something inherits it. That's somewhat true with id, but we can at least check "id" is present and that it's value is unique among all other id values. > >> > version: 1 >> > >> > id: skel-device >> > >> > title: > >> > Skeleton Device >> > >> > maintainer: >> > name: Skeleton Person >> > >> > description: > >> > The Skeleton Device binding represents the SK11 device produced by >> > the Skeleton Corporation. The binding can also support compatible >> > clones made by second source vendors. >> > >> > # The class is an optional property that declares this >> > # binding as part of a larger set >> > # Multiple definitions are possible >> > class: [ device, spi-device ] >> > >> > # This binding inherits property characteristics from the generic >> > # spi-slave binding >> > # Note that the notation is standard yaml reference >> > inherits: *spi-slave >> > >> > # virtual bindings do not generate checkers >> > virtual: true >> >> virtual is an overloaded term. >> > > OK, what term should I use that this binding should not be instantiated > as a checker, only be used by other bindings when inherited? checks: true? I'd really like to avoid having to decide and drop this, but I don't really get why it is needed. > >> > >> > # each property is defined by each name >> > properties: >> > >> > # The compatible property is a reserved name. The type is always >> > "string" >> > # and should not be repeated device binding. >> > compatible: >> > category: required# required property >> > type: strseq # is a sequence of strings >> > >> > description: > >> > FX11 is a clone of the original SK11 device >> > >> > # v is always the name of the value of the property >> > # np is passed to the checker and is the current >> > # node pointer. We can access properties and call >> > # methods that operate on them. >> > # There can be multiple constraints, just put them >> > # into a sequence. >> > # Note that the BASE("skel,sk11") form from the previous >> > # binding will have to be reworked. >> > constraint: | >> > anystreq(v, "skel,sk11") || >> > anystreq(v, "faux,fx11") >> >> Constraints and logic ops were certainly not decided in the last >> attempt and I think will be the hardest part to define. I see you are >> using eBPF in the checker. Is that where anystreq comes from? >> > > Yes. The ebpf environment declares a number of methods that are executed > outside the ebpf sandbox. Check out > > https://raw.githubusercontent.com/pantoniou/yamldt/master/validate/schema/codegen.yaml > https://github.com/pantoniou/yamldt/blob/master/ebpf_dt.c I looked at this some a while back. It wasn't clear to me what eBPF gives us and what you have defined on top of it. What I'm really after is documentation of the syntax
Re: [RFC] yamldt v0.5, now a DTS compiler too
Hi Rob, On Tue, 2017-10-03 at 08:18 -0500, Rob Herring wrote: > On Mon, Oct 2, 2017 at 2:46 PM, Pantelis Antoniou >wrote: > > Hi Rob, > > > > On Sun, 2017-10-01 at 17:00 -0500, Rob Herring wrote: > >> On Thu, Sep 28, 2017 at 2:58 PM, Pantelis Antoniou > >> wrote: > >> > Hello again, > >> > > >> > Significant progress has been made on yamldt and is now capable of > >> > not only generating yaml from DTS source but also compiling DTS sources > >> > and being almost fully compatible with DTC. > >> > >> Can you quantify "almost"? > >> > >> > Compiling the kernel's DTBs using yamldt is as simple as using a > >> > DTC=yamldt. > >> > >> Good. > >> > >> > > >> > Error reporting is accurate and validation against a YAML based schema > >> > works as well. In a short while I will begin posting patches with > >> > fixes on bindings and DTS files in the kernel. > >> > >> What I would like to see is the schema format posted for review. > >> > > > > I'm including the skeleton.yaml binding which is the template for > > the bindings and a board-example.yaml binding for a top level binding. > > > >> I would also like to see the bindings for top-level compatible strings > >> (aka boards) as an example. That's something that's simple enough that > >> I'd think we could agree on a format and start moving towards defining > >> board bindings that way. > >> > > > > Note there is some line wrapping I'm including a link > > to the github repo of the files: > > > > > > The skeleton.yaml > > > > https://raw.githubusercontent.com/pantoniou/yamldt/master/validate/bindings/skeleton.yaml > > > > %YAML 1.1 > > --- > > # The name of the binding is first > > # The anchor is put there for use by others > > skeleton: > > This and "id" seem redundant. > Indeed. > > version: 1 > > > > id: skel-device > > > > title: > > > Skeleton Device > > > > maintainer: > > name: Skeleton Person > > > > description: > > > The Skeleton Device binding represents the SK11 device produced by > > the Skeleton Corporation. The binding can also support compatible > > clones made by second source vendors. > > > > # The class is an optional property that declares this > > # binding as part of a larger set > > # Multiple definitions are possible > > class: [ device, spi-device ] > > > > # This binding inherits property characteristics from the generic > > # spi-slave binding > > # Note that the notation is standard yaml reference > > inherits: *spi-slave > > > > # virtual bindings do not generate checkers > > virtual: true > > virtual is an overloaded term. > OK, what term should I use that this binding should not be instantiated as a checker, only be used by other bindings when inherited? > > > > # each property is defined by each name > > properties: > > > > # The compatible property is a reserved name. The type is always > > "string" > > # and should not be repeated device binding. > > compatible: > > category: required# required property > > type: strseq # is a sequence of strings > > > > description: > > > FX11 is a clone of the original SK11 device > > > > # v is always the name of the value of the property > > # np is passed to the checker and is the current > > # node pointer. We can access properties and call > > # methods that operate on them. > > # There can be multiple constraints, just put them > > # into a sequence. > > # Note that the BASE("skel,sk11") form from the previous > > # binding will have to be reworked. > > constraint: | > > anystreq(v, "skel,sk11") || > > anystreq(v, "faux,fx11") > > Constraints and logic ops were certainly not decided in the last > attempt and I think will be the hardest part to define. I see you are > using eBPF in the checker. Is that where anystreq comes from? > Yes. The ebpf environment declares a number of methods that are executed outside the ebpf sandbox. Check out https://raw.githubusercontent.com/pantoniou/yamldt/master/validate/schema/codegen.yaml https://github.com/pantoniou/yamldt/blob/master/ebpf_dt.c > How would you express the ordering requirement (most significant > compatible first)? > Err, there might be new bpf API needed there. For the first stab at the bindings problem I concentrated on getting things working and be as clear as possible. You could do something like that: orderedstrseq(v, (const char *[]){ "skel,sk11", "faux,fx11", NULL }) Which would check ordering too. Obviously you'd hide the weird syntax using a macro. #define ORDEREDSTRSEQ(_v, ...) \ orderedstrseq(_v, (const char *[]){ __VA_ARGS__ , NULL }) So you'd write the above as: ORDEREDSTRSEQ(v, "skel,sk11", "faux,fx11") > > > > # The reg property is a reserved name. The type is always "int" and > > # should not be repeated
Re: [RFC] yamldt v0.5, now a DTS compiler too
Hi Rob, On Tue, 2017-10-03 at 08:18 -0500, Rob Herring wrote: > On Mon, Oct 2, 2017 at 2:46 PM, Pantelis Antoniou > wrote: > > Hi Rob, > > > > On Sun, 2017-10-01 at 17:00 -0500, Rob Herring wrote: > >> On Thu, Sep 28, 2017 at 2:58 PM, Pantelis Antoniou > >> wrote: > >> > Hello again, > >> > > >> > Significant progress has been made on yamldt and is now capable of > >> > not only generating yaml from DTS source but also compiling DTS sources > >> > and being almost fully compatible with DTC. > >> > >> Can you quantify "almost"? > >> > >> > Compiling the kernel's DTBs using yamldt is as simple as using a > >> > DTC=yamldt. > >> > >> Good. > >> > >> > > >> > Error reporting is accurate and validation against a YAML based schema > >> > works as well. In a short while I will begin posting patches with > >> > fixes on bindings and DTS files in the kernel. > >> > >> What I would like to see is the schema format posted for review. > >> > > > > I'm including the skeleton.yaml binding which is the template for > > the bindings and a board-example.yaml binding for a top level binding. > > > >> I would also like to see the bindings for top-level compatible strings > >> (aka boards) as an example. That's something that's simple enough that > >> I'd think we could agree on a format and start moving towards defining > >> board bindings that way. > >> > > > > Note there is some line wrapping I'm including a link > > to the github repo of the files: > > > > > > The skeleton.yaml > > > > https://raw.githubusercontent.com/pantoniou/yamldt/master/validate/bindings/skeleton.yaml > > > > %YAML 1.1 > > --- > > # The name of the binding is first > > # The anchor is put there for use by others > > skeleton: > > This and "id" seem redundant. > Indeed. > > version: 1 > > > > id: skel-device > > > > title: > > > Skeleton Device > > > > maintainer: > > name: Skeleton Person > > > > description: > > > The Skeleton Device binding represents the SK11 device produced by > > the Skeleton Corporation. The binding can also support compatible > > clones made by second source vendors. > > > > # The class is an optional property that declares this > > # binding as part of a larger set > > # Multiple definitions are possible > > class: [ device, spi-device ] > > > > # This binding inherits property characteristics from the generic > > # spi-slave binding > > # Note that the notation is standard yaml reference > > inherits: *spi-slave > > > > # virtual bindings do not generate checkers > > virtual: true > > virtual is an overloaded term. > OK, what term should I use that this binding should not be instantiated as a checker, only be used by other bindings when inherited? > > > > # each property is defined by each name > > properties: > > > > # The compatible property is a reserved name. The type is always > > "string" > > # and should not be repeated device binding. > > compatible: > > category: required# required property > > type: strseq # is a sequence of strings > > > > description: > > > FX11 is a clone of the original SK11 device > > > > # v is always the name of the value of the property > > # np is passed to the checker and is the current > > # node pointer. We can access properties and call > > # methods that operate on them. > > # There can be multiple constraints, just put them > > # into a sequence. > > # Note that the BASE("skel,sk11") form from the previous > > # binding will have to be reworked. > > constraint: | > > anystreq(v, "skel,sk11") || > > anystreq(v, "faux,fx11") > > Constraints and logic ops were certainly not decided in the last > attempt and I think will be the hardest part to define. I see you are > using eBPF in the checker. Is that where anystreq comes from? > Yes. The ebpf environment declares a number of methods that are executed outside the ebpf sandbox. Check out https://raw.githubusercontent.com/pantoniou/yamldt/master/validate/schema/codegen.yaml https://github.com/pantoniou/yamldt/blob/master/ebpf_dt.c > How would you express the ordering requirement (most significant > compatible first)? > Err, there might be new bpf API needed there. For the first stab at the bindings problem I concentrated on getting things working and be as clear as possible. You could do something like that: orderedstrseq(v, (const char *[]){ "skel,sk11", "faux,fx11", NULL }) Which would check ordering too. Obviously you'd hide the weird syntax using a macro. #define ORDEREDSTRSEQ(_v, ...) \ orderedstrseq(_v, (const char *[]){ __VA_ARGS__ , NULL }) So you'd write the above as: ORDEREDSTRSEQ(v, "skel,sk11", "faux,fx11") > > > > # The reg property is a reserved name. The type is always "int" and > > # should not be repeated in a device binding. Constraints are > > defined > > # only in the context of
Re: [RFC] yamldt v0.5, now a DTS compiler too
On Mon, Oct 2, 2017 at 2:46 PM, Pantelis Antoniouwrote: > Hi Rob, > > On Sun, 2017-10-01 at 17:00 -0500, Rob Herring wrote: >> On Thu, Sep 28, 2017 at 2:58 PM, Pantelis Antoniou >> wrote: >> > Hello again, >> > >> > Significant progress has been made on yamldt and is now capable of >> > not only generating yaml from DTS source but also compiling DTS sources >> > and being almost fully compatible with DTC. >> >> Can you quantify "almost"? >> >> > Compiling the kernel's DTBs using yamldt is as simple as using a >> > DTC=yamldt. >> >> Good. >> >> > >> > Error reporting is accurate and validation against a YAML based schema >> > works as well. In a short while I will begin posting patches with >> > fixes on bindings and DTS files in the kernel. >> >> What I would like to see is the schema format posted for review. >> > > I'm including the skeleton.yaml binding which is the template for > the bindings and a board-example.yaml binding for a top level binding. > >> I would also like to see the bindings for top-level compatible strings >> (aka boards) as an example. That's something that's simple enough that >> I'd think we could agree on a format and start moving towards defining >> board bindings that way. >> > > Note there is some line wrapping I'm including a link > to the github repo of the files: > > > The skeleton.yaml > > https://raw.githubusercontent.com/pantoniou/yamldt/master/validate/bindings/skeleton.yaml > > %YAML 1.1 > --- > # The name of the binding is first > # The anchor is put there for use by others > skeleton: This and "id" seem redundant. > version: 1 > > id: skel-device > > title: > > Skeleton Device > > maintainer: > name: Skeleton Person > > description: > > The Skeleton Device binding represents the SK11 device produced by > the Skeleton Corporation. The binding can also support compatible > clones made by second source vendors. > > # The class is an optional property that declares this > # binding as part of a larger set > # Multiple definitions are possible > class: [ device, spi-device ] > > # This binding inherits property characteristics from the generic > # spi-slave binding > # Note that the notation is standard yaml reference > inherits: *spi-slave > > # virtual bindings do not generate checkers > virtual: true virtual is an overloaded term. > > # each property is defined by each name > properties: > > # The compatible property is a reserved name. The type is always > "string" > # and should not be repeated device binding. > compatible: > category: required# required property > type: strseq # is a sequence of strings > > description: > > FX11 is a clone of the original SK11 device > > # v is always the name of the value of the property > # np is passed to the checker and is the current > # node pointer. We can access properties and call > # methods that operate on them. > # There can be multiple constraints, just put them > # into a sequence. > # Note that the BASE("skel,sk11") form from the previous > # binding will have to be reworked. > constraint: | > anystreq(v, "skel,sk11") || > anystreq(v, "faux,fx11") Constraints and logic ops were certainly not decided in the last attempt and I think will be the hardest part to define. I see you are using eBPF in the checker. Is that where anystreq comes from? How would you express the ordering requirement (most significant compatible first)? > > # The reg property is a reserved name. The type is always "int" and > # should not be repeated in a device binding. Constraints are > defined > # only in the context of the parent node's address, size, and ranges > # cells. The description is inherited from the spi-slave binding. > # Note that if inheriting from a base binding this declaration may > # be omitted. > reg: > category: required# required property > type: intseq # is a sequence of integers > > # spi-max-frequency needs the device-specific constraint to be > supplied > spi-max-frequency: > # this constraint is dependent on the compatible property > # property containing "skel,sk11" > constraint: | > v <= anystreq(get_strseq(np, "compatible"), "skel,sk11") ? > 1000 : 100 > > # This property overrides the generic binding description with > # a device specific description in order to mention the chip's > # h/w cfg strapping pins. > spi-cs-high: > description: > > Set if skeleton device configuration pins are set for chip > select polarity high > > # Device specific properties don't inherit characteristic from a > generic > # binding so category, type, constraint, and description must be > specified > # if needed. >
Re: [RFC] yamldt v0.5, now a DTS compiler too
On Mon, Oct 2, 2017 at 2:46 PM, Pantelis Antoniou wrote: > Hi Rob, > > On Sun, 2017-10-01 at 17:00 -0500, Rob Herring wrote: >> On Thu, Sep 28, 2017 at 2:58 PM, Pantelis Antoniou >> wrote: >> > Hello again, >> > >> > Significant progress has been made on yamldt and is now capable of >> > not only generating yaml from DTS source but also compiling DTS sources >> > and being almost fully compatible with DTC. >> >> Can you quantify "almost"? >> >> > Compiling the kernel's DTBs using yamldt is as simple as using a >> > DTC=yamldt. >> >> Good. >> >> > >> > Error reporting is accurate and validation against a YAML based schema >> > works as well. In a short while I will begin posting patches with >> > fixes on bindings and DTS files in the kernel. >> >> What I would like to see is the schema format posted for review. >> > > I'm including the skeleton.yaml binding which is the template for > the bindings and a board-example.yaml binding for a top level binding. > >> I would also like to see the bindings for top-level compatible strings >> (aka boards) as an example. That's something that's simple enough that >> I'd think we could agree on a format and start moving towards defining >> board bindings that way. >> > > Note there is some line wrapping I'm including a link > to the github repo of the files: > > > The skeleton.yaml > > https://raw.githubusercontent.com/pantoniou/yamldt/master/validate/bindings/skeleton.yaml > > %YAML 1.1 > --- > # The name of the binding is first > # The anchor is put there for use by others > skeleton: This and "id" seem redundant. > version: 1 > > id: skel-device > > title: > > Skeleton Device > > maintainer: > name: Skeleton Person > > description: > > The Skeleton Device binding represents the SK11 device produced by > the Skeleton Corporation. The binding can also support compatible > clones made by second source vendors. > > # The class is an optional property that declares this > # binding as part of a larger set > # Multiple definitions are possible > class: [ device, spi-device ] > > # This binding inherits property characteristics from the generic > # spi-slave binding > # Note that the notation is standard yaml reference > inherits: *spi-slave > > # virtual bindings do not generate checkers > virtual: true virtual is an overloaded term. > > # each property is defined by each name > properties: > > # The compatible property is a reserved name. The type is always > "string" > # and should not be repeated device binding. > compatible: > category: required# required property > type: strseq # is a sequence of strings > > description: > > FX11 is a clone of the original SK11 device > > # v is always the name of the value of the property > # np is passed to the checker and is the current > # node pointer. We can access properties and call > # methods that operate on them. > # There can be multiple constraints, just put them > # into a sequence. > # Note that the BASE("skel,sk11") form from the previous > # binding will have to be reworked. > constraint: | > anystreq(v, "skel,sk11") || > anystreq(v, "faux,fx11") Constraints and logic ops were certainly not decided in the last attempt and I think will be the hardest part to define. I see you are using eBPF in the checker. Is that where anystreq comes from? How would you express the ordering requirement (most significant compatible first)? > > # The reg property is a reserved name. The type is always "int" and > # should not be repeated in a device binding. Constraints are > defined > # only in the context of the parent node's address, size, and ranges > # cells. The description is inherited from the spi-slave binding. > # Note that if inheriting from a base binding this declaration may > # be omitted. > reg: > category: required# required property > type: intseq # is a sequence of integers > > # spi-max-frequency needs the device-specific constraint to be > supplied > spi-max-frequency: > # this constraint is dependent on the compatible property > # property containing "skel,sk11" > constraint: | > v <= anystreq(get_strseq(np, "compatible"), "skel,sk11") ? > 1000 : 100 > > # This property overrides the generic binding description with > # a device specific description in order to mention the chip's > # h/w cfg strapping pins. > spi-cs-high: > description: > > Set if skeleton device configuration pins are set for chip > select polarity high > > # Device specific properties don't inherit characteristic from a > generic > # binding so category, type, constraint, and description must be > specified > # if needed. > skel,deprecated1: > # note that the category may be declare more than
Re: [RFC] yamldt v0.5, now a DTS compiler too
Hi Geert, On Tue, 2017-10-03 at 09:17 +0200, Geert Uytterhoeven wrote: > Hi Pantelis, > > On Mon, Oct 2, 2017 at 9:46 PM, Pantelis Antoniou >wrote: > > # Note that the YAML example must be validated against this binding > > # to be an accepted entry > > Indeed ;-) > > > yaml: | > > sk11@0: > > compatible: "skel,sk11" > > reg: 0 > > sip-max-frequency: 100 > > s/sip-max-frequency/spi-max-frequency/ > Heh, obviously the auto-check of binding examples is not working yet :) > Gr{oetje,eeting}s, > > Geert > > -- > Geert Uytterhoeven -- There's lots of Linux beyond ia32 -- > ge...@linux-m68k.org > > In personal conversations with technical people, I call myself a hacker. But > when I'm talking to journalists I just say "programmer" or something like > that. > -- Linus Torvalds
Re: [RFC] yamldt v0.5, now a DTS compiler too
Hi Geert, On Tue, 2017-10-03 at 09:17 +0200, Geert Uytterhoeven wrote: > Hi Pantelis, > > On Mon, Oct 2, 2017 at 9:46 PM, Pantelis Antoniou > wrote: > > # Note that the YAML example must be validated against this binding > > # to be an accepted entry > > Indeed ;-) > > > yaml: | > > sk11@0: > > compatible: "skel,sk11" > > reg: 0 > > sip-max-frequency: 100 > > s/sip-max-frequency/spi-max-frequency/ > Heh, obviously the auto-check of binding examples is not working yet :) > Gr{oetje,eeting}s, > > Geert > > -- > Geert Uytterhoeven -- There's lots of Linux beyond ia32 -- > ge...@linux-m68k.org > > In personal conversations with technical people, I call myself a hacker. But > when I'm talking to journalists I just say "programmer" or something like > that. > -- Linus Torvalds
Re: [RFC] yamldt v0.5, now a DTS compiler too
Hi Pantelis, On Mon, Oct 2, 2017 at 9:46 PM, Pantelis Antoniouwrote: > # Note that the YAML example must be validated against this binding > # to be an accepted entry Indeed ;-) > yaml: | > sk11@0: > compatible: "skel,sk11" > reg: 0 > sip-max-frequency: 100 s/sip-max-frequency/spi-max-frequency/ Gr{oetje,eeting}s, Geert -- Geert Uytterhoeven -- There's lots of Linux beyond ia32 -- ge...@linux-m68k.org In personal conversations with technical people, I call myself a hacker. But when I'm talking to journalists I just say "programmer" or something like that. -- Linus Torvalds
Re: [RFC] yamldt v0.5, now a DTS compiler too
Hi Pantelis, On Mon, Oct 2, 2017 at 9:46 PM, Pantelis Antoniou wrote: > # Note that the YAML example must be validated against this binding > # to be an accepted entry Indeed ;-) > yaml: | > sk11@0: > compatible: "skel,sk11" > reg: 0 > sip-max-frequency: 100 s/sip-max-frequency/spi-max-frequency/ Gr{oetje,eeting}s, Geert -- Geert Uytterhoeven -- There's lots of Linux beyond ia32 -- ge...@linux-m68k.org In personal conversations with technical people, I call myself a hacker. But when I'm talking to journalists I just say "programmer" or something like that. -- Linus Torvalds
Re: [RFC] yamldt v0.5, now a DTS compiler too
Hi Rob, On Sun, 2017-10-01 at 17:00 -0500, Rob Herring wrote: > On Thu, Sep 28, 2017 at 2:58 PM, Pantelis Antoniou >wrote: > > Hello again, > > > > Significant progress has been made on yamldt and is now capable of > > not only generating yaml from DTS source but also compiling DTS sources > > and being almost fully compatible with DTC. > > Can you quantify "almost"? > > > Compiling the kernel's DTBs using yamldt is as simple as using a > > DTC=yamldt. > > Good. > > > > > Error reporting is accurate and validation against a YAML based schema > > works as well. In a short while I will begin posting patches with > > fixes on bindings and DTS files in the kernel. > > What I would like to see is the schema format posted for review. > I'm including the skeleton.yaml binding which is the template for the bindings and a board-example.yaml binding for a top level binding. > I would also like to see the bindings for top-level compatible strings > (aka boards) as an example. That's something that's simple enough that > I'd think we could agree on a format and start moving towards defining > board bindings that way. > Note there is some line wrapping I'm including a link to the github repo of the files: The skeleton.yaml https://raw.githubusercontent.com/pantoniou/yamldt/master/validate/bindings/skeleton.yaml %YAML 1.1 --- # The name of the binding is first # The anchor is put there for use by others skeleton: version: 1 id: skel-device title: > Skeleton Device maintainer: name: Skeleton Person description: > The Skeleton Device binding represents the SK11 device produced by the Skeleton Corporation. The binding can also support compatible clones made by second source vendors. # The class is an optional property that declares this # binding as part of a larger set # Multiple definitions are possible class: [ device, spi-device ] # This binding inherits property characteristics from the generic # spi-slave binding # Note that the notation is standard yaml reference inherits: *spi-slave # virtual bindings do not generate checkers virtual: true # each property is defined by each name properties: # The compatible property is a reserved name. The type is always "string" # and should not be repeated device binding. compatible: category: required# required property type: strseq # is a sequence of strings description: > FX11 is a clone of the original SK11 device # v is always the name of the value of the property # np is passed to the checker and is the current # node pointer. We can access properties and call # methods that operate on them. # There can be multiple constraints, just put them # into a sequence. # Note that the BASE("skel,sk11") form from the previous # binding will have to be reworked. constraint: | anystreq(v, "skel,sk11") || anystreq(v, "faux,fx11") # The reg property is a reserved name. The type is always "int" and # should not be repeated in a device binding. Constraints are defined # only in the context of the parent node's address, size, and ranges # cells. The description is inherited from the spi-slave binding. # Note that if inheriting from a base binding this declaration may # be omitted. reg: category: required# required property type: intseq # is a sequence of integers # spi-max-frequency needs the device-specific constraint to be supplied spi-max-frequency: # this constraint is dependent on the compatible property # property containing "skel,sk11" constraint: | v <= anystreq(get_strseq(np, "compatible"), "skel,sk11") ? 1000 : 100 # This property overrides the generic binding description with # a device specific description in order to mention the chip's # h/w cfg strapping pins. spi-cs-high: description: > Set if skeleton device configuration pins are set for chip select polarity high # Device specific properties don't inherit characteristic from a generic # binding so category, type, constraint, and description must be specified # if needed. skel,deprecated1: # note that the category may be declare more than one option category: [ deprecated, optional ] type: int constraint: | v >= 10 && v <= 20 description: > First of two deprecated properties. # There are no constraints for properties of empty type skel,deprecated2: category: deprecated type: empty description: > Second of two deprecated properties. # This example could be auto-generated rather than explicitly included # in the yaml source. # Note that the YAML example must be validated against this binding # to be an
Re: [RFC] yamldt v0.5, now a DTS compiler too
Hi Rob, On Sun, 2017-10-01 at 17:00 -0500, Rob Herring wrote: > On Thu, Sep 28, 2017 at 2:58 PM, Pantelis Antoniou > wrote: > > Hello again, > > > > Significant progress has been made on yamldt and is now capable of > > not only generating yaml from DTS source but also compiling DTS sources > > and being almost fully compatible with DTC. > > Can you quantify "almost"? > > > Compiling the kernel's DTBs using yamldt is as simple as using a > > DTC=yamldt. > > Good. > > > > > Error reporting is accurate and validation against a YAML based schema > > works as well. In a short while I will begin posting patches with > > fixes on bindings and DTS files in the kernel. > > What I would like to see is the schema format posted for review. > I'm including the skeleton.yaml binding which is the template for the bindings and a board-example.yaml binding for a top level binding. > I would also like to see the bindings for top-level compatible strings > (aka boards) as an example. That's something that's simple enough that > I'd think we could agree on a format and start moving towards defining > board bindings that way. > Note there is some line wrapping I'm including a link to the github repo of the files: The skeleton.yaml https://raw.githubusercontent.com/pantoniou/yamldt/master/validate/bindings/skeleton.yaml %YAML 1.1 --- # The name of the binding is first # The anchor is put there for use by others skeleton: version: 1 id: skel-device title: > Skeleton Device maintainer: name: Skeleton Person description: > The Skeleton Device binding represents the SK11 device produced by the Skeleton Corporation. The binding can also support compatible clones made by second source vendors. # The class is an optional property that declares this # binding as part of a larger set # Multiple definitions are possible class: [ device, spi-device ] # This binding inherits property characteristics from the generic # spi-slave binding # Note that the notation is standard yaml reference inherits: *spi-slave # virtual bindings do not generate checkers virtual: true # each property is defined by each name properties: # The compatible property is a reserved name. The type is always "string" # and should not be repeated device binding. compatible: category: required# required property type: strseq # is a sequence of strings description: > FX11 is a clone of the original SK11 device # v is always the name of the value of the property # np is passed to the checker and is the current # node pointer. We can access properties and call # methods that operate on them. # There can be multiple constraints, just put them # into a sequence. # Note that the BASE("skel,sk11") form from the previous # binding will have to be reworked. constraint: | anystreq(v, "skel,sk11") || anystreq(v, "faux,fx11") # The reg property is a reserved name. The type is always "int" and # should not be repeated in a device binding. Constraints are defined # only in the context of the parent node's address, size, and ranges # cells. The description is inherited from the spi-slave binding. # Note that if inheriting from a base binding this declaration may # be omitted. reg: category: required# required property type: intseq # is a sequence of integers # spi-max-frequency needs the device-specific constraint to be supplied spi-max-frequency: # this constraint is dependent on the compatible property # property containing "skel,sk11" constraint: | v <= anystreq(get_strseq(np, "compatible"), "skel,sk11") ? 1000 : 100 # This property overrides the generic binding description with # a device specific description in order to mention the chip's # h/w cfg strapping pins. spi-cs-high: description: > Set if skeleton device configuration pins are set for chip select polarity high # Device specific properties don't inherit characteristic from a generic # binding so category, type, constraint, and description must be specified # if needed. skel,deprecated1: # note that the category may be declare more than one option category: [ deprecated, optional ] type: int constraint: | v >= 10 && v <= 20 description: > First of two deprecated properties. # There are no constraints for properties of empty type skel,deprecated2: category: deprecated type: empty description: > Second of two deprecated properties. # This example could be auto-generated rather than explicitly included # in the yaml source. # Note that the YAML example must be validated against this binding # to be an accepted entry example: dts: |
Re: [RFC] yamldt v0.5, now a DTS compiler too
> On Oct 2, 2017, at 01:00 , Rob Herringwrote: > > On Thu, Sep 28, 2017 at 2:58 PM, Pantelis Antoniou > wrote: >> Hello again, >> >> Significant progress has been made on yamldt and is now capable of >> not only generating yaml from DTS source but also compiling DTS sources >> and being almost fully compatible with DTC. > > Can you quantify "almost”? The —sort option and /include-bin/ doesn’t work yet. It’s a day or so to add them but there are no kernel dtb files that rely on them. > >> Compiling the kernel's DTBs using yamldt is as simple as using a >> DTC=yamldt. > > Good. > >> >> Error reporting is accurate and validation against a YAML based schema >> works as well. In a short while I will begin posting patches with >> fixes on bindings and DTS files in the kernel. > > What I would like to see is the schema format posted for review. > It is based on your binding schema you’ve posted but tweaked slightly to be parseable by the YAML subset that matches the DTS style. In particular bare sequences do not map to DT at all, so: - name: foo is changed to name: foo And so on. > I would also like to see the bindings for top-level compatible strings > (aka boards) as an example. That's something that's simple enough that > I'd think we could agree on a format and start moving towards defining > board bindings that way. > I’ll see what I can do. > Rob Regards — Pantelis
Re: [RFC] yamldt v0.5, now a DTS compiler too
> On Oct 2, 2017, at 01:00 , Rob Herring wrote: > > On Thu, Sep 28, 2017 at 2:58 PM, Pantelis Antoniou > wrote: >> Hello again, >> >> Significant progress has been made on yamldt and is now capable of >> not only generating yaml from DTS source but also compiling DTS sources >> and being almost fully compatible with DTC. > > Can you quantify "almost”? The —sort option and /include-bin/ doesn’t work yet. It’s a day or so to add them but there are no kernel dtb files that rely on them. > >> Compiling the kernel's DTBs using yamldt is as simple as using a >> DTC=yamldt. > > Good. > >> >> Error reporting is accurate and validation against a YAML based schema >> works as well. In a short while I will begin posting patches with >> fixes on bindings and DTS files in the kernel. > > What I would like to see is the schema format posted for review. > It is based on your binding schema you’ve posted but tweaked slightly to be parseable by the YAML subset that matches the DTS style. In particular bare sequences do not map to DT at all, so: - name: foo is changed to name: foo And so on. > I would also like to see the bindings for top-level compatible strings > (aka boards) as an example. That's something that's simple enough that > I'd think we could agree on a format and start moving towards defining > board bindings that way. > I’ll see what I can do. > Rob Regards — Pantelis
Re: [RFC] yamldt v0.5, now a DTS compiler too
On Thu, Sep 28, 2017 at 2:58 PM, Pantelis Antoniouwrote: > Hello again, > > Significant progress has been made on yamldt and is now capable of > not only generating yaml from DTS source but also compiling DTS sources > and being almost fully compatible with DTC. Can you quantify "almost"? > Compiling the kernel's DTBs using yamldt is as simple as using a > DTC=yamldt. Good. > > Error reporting is accurate and validation against a YAML based schema > works as well. In a short while I will begin posting patches with > fixes on bindings and DTS files in the kernel. What I would like to see is the schema format posted for review. I would also like to see the bindings for top-level compatible strings (aka boards) as an example. That's something that's simple enough that I'd think we could agree on a format and start moving towards defining board bindings that way. Rob
Re: [RFC] yamldt v0.5, now a DTS compiler too
On Thu, Sep 28, 2017 at 2:58 PM, Pantelis Antoniou wrote: > Hello again, > > Significant progress has been made on yamldt and is now capable of > not only generating yaml from DTS source but also compiling DTS sources > and being almost fully compatible with DTC. Can you quantify "almost"? > Compiling the kernel's DTBs using yamldt is as simple as using a > DTC=yamldt. Good. > > Error reporting is accurate and validation against a YAML based schema > works as well. In a short while I will begin posting patches with > fixes on bindings and DTS files in the kernel. What I would like to see is the schema format posted for review. I would also like to see the bindings for top-level compatible strings (aka boards) as an example. That's something that's simple enough that I'd think we could agree on a format and start moving towards defining board bindings that way. Rob
[RFC] yamldt v0.5, now a DTS compiler too
Hello again, Significant progress has been made on yamldt and is now capable of not only generating yaml from DTS source but also compiling DTS sources and being almost fully compatible with DTC. Compiling the kernel's DTBs using yamldt is as simple as using a DTC=yamldt. Error reporting is accurate and validation against a YAML based schema works as well. In a short while I will begin posting patches with fixes on bindings and DTS files in the kernel. Please try it on your platform and report if you encounter any problems. https://github.com/pantoniou/yamldt I am eagerly awaiting for your comments. Regards -- Pantelis
[RFC] yamldt v0.5, now a DTS compiler too
Hello again, Significant progress has been made on yamldt and is now capable of not only generating yaml from DTS source but also compiling DTS sources and being almost fully compatible with DTC. Compiling the kernel's DTBs using yamldt is as simple as using a DTC=yamldt. Error reporting is accurate and validation against a YAML based schema works as well. In a short while I will begin posting patches with fixes on bindings and DTS files in the kernel. Please try it on your platform and report if you encounter any problems. https://github.com/pantoniou/yamldt I am eagerly awaiting for your comments. Regards -- Pantelis