Re: [netmod] string in when and must statement in YANG ABNF Grammar
William Lupton writes: > It might also be worth noting (see > https://github.com/mbj4668/pyang/wiki/InstanceValidation) that "DSDL > schemas can be used with generic off-the-shelf XML tools for both > syntactic and semantic validation of XML instance documents”. This > includes validating “must” statements and so on. William. Right, although XPath expressions as they appear in YANG modules need to be massaged somewhat before they can be used in Schematron - e.g. explicit prefixes need to be added where they are missing. Lada > >> On 29 Mar 2016, at 08:22, Martin Bjorklund wrote: >> >> "Bogaert, Bart (Nokia - BE)" wrote: >>> Is there a reason why the ABNF for example the when and must statement has >>> been "restricted" to when-keyword sep string optsep >>> >>> As "string" can't be tokenized no errors are generated by YANG compilers if >>> this string contains an error, e.g. referred leaf does not exit due to a >>> typo. This problem only exposes itself at run-time. I was wondering why >>> this string was not broken down into a number of specific parts that define >>> the when statement so that these kind of errors can be trapped early in the >>> development process? I am rather new in this so I did not follow all >>> discussions that led to the definition of YANG and hence have no idea >>> whether this was discussed and what were the reasons not to do it this way. >> >> The arguments to the "when" and "must" statements are XPath 1.0 >> expressions. The syntax of XPath 1.0 is not defined by YANG, but by >> the XPath 1.0 spec. This is the reason that the YANG grammar isn't >> more specific. >> >> YANG compilers differ in their ability to detect errors in these XPath >> expressions. Some perform more checks than others (unfortunately, >> pyang is pretty bad in this regard... (patches are always welcome:)). >> >> [For your particular example, referencing a leaf that doesn't exist is >> not an error per se; it is perfectly valid XPath. But it probably >> warrants a warning by the compiler.] >> >> /martin >> >> ___ >> netmod mailing list >> netmod@ietf.org >> https://www.ietf.org/mailman/listinfo/netmod >> > > ___ > netmod mailing list > netmod@ietf.org > https://www.ietf.org/mailman/listinfo/netmod -- Ladislav Lhotka, CZ.NIC Labs PGP Key ID: E74E8C0C ___ netmod mailing list netmod@ietf.org https://www.ietf.org/mailman/listinfo/netmod
Re: [netmod] string in when and must statement in YANG ABNF Grammar
It might also be worth noting (see https://github.com/mbj4668/pyang/wiki/InstanceValidation) that "DSDL schemas can be used with generic off-the-shelf XML tools for both syntactic and semantic validation of XML instance documents”. This includes validating “must” statements and so on. William. > On 29 Mar 2016, at 08:22, Martin Bjorklund wrote: > > "Bogaert, Bart (Nokia - BE)" wrote: >> Is there a reason why the ABNF for example the when and must statement has >> been "restricted" to when-keyword sep string optsep >> >> As "string" can't be tokenized no errors are generated by YANG compilers if >> this string contains an error, e.g. referred leaf does not exit due to a >> typo. This problem only exposes itself at run-time. I was wondering why >> this string was not broken down into a number of specific parts that define >> the when statement so that these kind of errors can be trapped early in the >> development process? I am rather new in this so I did not follow all >> discussions that led to the definition of YANG and hence have no idea >> whether this was discussed and what were the reasons not to do it this way. > > The arguments to the "when" and "must" statements are XPath 1.0 > expressions. The syntax of XPath 1.0 is not defined by YANG, but by > the XPath 1.0 spec. This is the reason that the YANG grammar isn't > more specific. > > YANG compilers differ in their ability to detect errors in these XPath > expressions. Some perform more checks than others (unfortunately, > pyang is pretty bad in this regard... (patches are always welcome:)). > > [For your particular example, referencing a leaf that doesn't exist is > not an error per se; it is perfectly valid XPath. But it probably > warrants a warning by the compiler.] > > /martin > > ___ > netmod mailing list > netmod@ietf.org > https://www.ietf.org/mailman/listinfo/netmod > ___ netmod mailing list netmod@ietf.org https://www.ietf.org/mailman/listinfo/netmod
Re: [netmod] string in when and must statement in YANG ABNF Grammar
Hi, "Bogaert, Bart (Nokia - BE)" wrote: > Martin, > > Thanks for this feedback. A run-time crash of the NETCONF server is a bad > "outcome" of a typo like this... Sure; but that's a tool issue, not a specification issue. > I checked the W3C pages and there is grammar defined for an XPATH expression > so ultimately the string could be parsed as an XPATH (I guess it is > currently not done). It all depends on the tool you are using. > Any other programming language ensures to eliminate typing errors during the > compilation phase allowing early detection of issues like this. It is > becoming a problem when the NETCONF server internally stores values (e.g. > identityref) in a format different from what is specified in the string of > the when statement resulting in unexpected options when trying to use the > CLI generated from the model. In this specific case we were advised to > prefix the leaf (as the leaf was stored in this way...) even though the leaf > was in the local module itself. This makes development even less > transparent... When we flagged this we received the following feedback: > > Quote: > For YANG 1.1 that is underway it will be defined in the following way: > " The string value of a node of type identityref in a "must" or "when" XPath > > expression is the referred identity's qualified name with the prefix > present. > If the referred identity is defined in an imported module, the prefix in the > > string value is the prefix defined in the corresponding "import" statement. > > Otherwise, the prefix in the string value is the prefix for the current > module."" > > But as this is within a string that is not parsed (depending on the > compiler) this is something that people need to do and if not done will > still result in run-time problems. Right; for this particular example, it is better to use the new (YANG 1.1) functions derived-from() and derived-from-or-self(). /martin > > Best regards - Vriendelijke groeten, > Bart Bogaert > System Architect Data-Centric SW Architectures > Fixed Networks - Broadband Access BU, Nokia > Contact number +32 3 2408310 (+32 477 673952) > > -Original Message----- > From: EXT Martin Bjorklund [mailto:m...@tail-f.com] > Sent: 29 March 2016 09:23 > To: Bogaert, Bart (Nokia - BE) > Cc: netmod@ietf.org > Subject: Re: [netmod] string in when and must statement in YANG ABNF Grammar > > "Bogaert, Bart (Nokia - BE)" wrote: > > Is there a reason why the ABNF for example the when and must statement > > has been "restricted" to when-keyword sep string optsep > > > > As "string" can't be tokenized no errors are generated by YANG > > compilers if this string contains an error, e.g. referred leaf does > > not exit due to a typo. This problem only exposes itself at run-time. > > I was wondering why this string was not broken down into a number of > > specific parts that define the when statement so that these kind of errors > can be trapped early in the > > development process? I am rather new in this so I did not follow all > > discussions that led to the definition of YANG and hence have no idea > > whether this was discussed and what were the reasons not to do it this > way. > > The arguments to the "when" and "must" statements are XPath 1.0 expressions. > The syntax of XPath 1.0 is not defined by YANG, but by the XPath 1.0 spec. > This is the reason that the YANG grammar isn't more specific. > > YANG compilers differ in their ability to detect errors in these XPath > expressions. Some perform more checks than others (unfortunately, pyang is > pretty bad in this regard... (patches are always welcome:)). > > [For your particular example, referencing a leaf that doesn't exist is not > an error per se; it is perfectly valid XPath. But it probably warrants a > warning by the compiler.] > > > /martin ___ netmod mailing list netmod@ietf.org https://www.ietf.org/mailman/listinfo/netmod
Re: [netmod] string in when and must statement in YANG ABNF Grammar
Hi Bart, Maybe you need to use a different NETCONF server if your current one is crashing with badly formed XPATH expressions? Our YANG compiler validates XPATH expressions when we load in YANG files, and anything that slips through should certainly not cause a crash at runtime (unless there's a bug of course). I had assumed others would do likewise - as far as I'm concerned XPATH is an 'extension' of YANG and we validate it with the same level of scrutiny as the YANG itself. So, I don't think this is anything to do with the 'programming language' - it's all about individual implementations of the YANG / XPATH compiler. I don't quite follow your point about prefixing a leaf - if you don't explicitly prefix a leaf in an XPATH statement then there are clear (well, after the third or fourth reading!) rules that specify what prefix will be used, depending on whether the must statement is inside a grouping or an augment statement, or in 'plain' YANG. Is your NETCONF server doing something unexpected? Regards, William -Original Message- From: netmod [mailto:netmod-boun...@ietf.org] On Behalf Of Bogaert, Bart (Nokia - BE) Sent: 29 March 2016 08:57 To: EXT Martin Bjorklund Cc: netmod@ietf.org Subject: Re: [netmod] string in when and must statement in YANG ABNF Grammar Martin, Thanks for this feedback. A run-time crash of the NETCONF server is a bad "outcome" of a typo like this... I checked the W3C pages and there is grammar defined for an XPATH expression so ultimately the string could be parsed as an XPATH (I guess it is currently not done). Any other programming language ensures to eliminate typing errors during the compilation phase allowing early detection of issues like this. It is becoming a problem when the NETCONF server internally stores values (e.g. identityref) in a format different from what is specified in the string of the when statement resulting in unexpected options when trying to use the CLI generated from the model. In this specific case we were advised to prefix the leaf (as the leaf was stored in this way...) even though the leaf was in the local module itself. This makes development even less transparent... When we flagged this we received the following feedback: Quote: For YANG 1.1 that is underway it will be defined in the following way: " The string value of a node of type identityref in a "must" or "when" XPath expression is the referred identity's qualified name with the prefix present. If the referred identity is defined in an imported module, the prefix in the string value is the prefix defined in the corresponding "import" statement. Otherwise, the prefix in the string value is the prefix for the current module."" But as this is within a string that is not parsed (depending on the compiler) this is something that people need to do and if not done will still result in run-time problems. Best regards - Vriendelijke groeten, Bart Bogaert System Architect Data-Centric SW Architectures Fixed Networks - Broadband Access BU, Nokia Contact number +32 3 2408310 (+32 477 673952) -Original Message- From: EXT Martin Bjorklund [mailto:m...@tail-f.com] Sent: 29 March 2016 09:23 To: Bogaert, Bart (Nokia - BE) Cc: netmod@ietf.org Subject: Re: [netmod] string in when and must statement in YANG ABNF Grammar "Bogaert, Bart (Nokia - BE)" wrote: > Is there a reason why the ABNF for example the when and must statement > has been "restricted" to when-keyword sep string optsep > > As "string" can't be tokenized no errors are generated by YANG > compilers if this string contains an error, e.g. referred leaf does > not exit due to a typo. This problem only exposes itself at run-time. > I was wondering why this string was not broken down into a number of > specific parts that define the when statement so that these kind of > errors can be trapped early in the > development process? I am rather new in this so I did not follow all > discussions that led to the definition of YANG and hence have no idea > whether this was discussed and what were the reasons not to do it this way. The arguments to the "when" and "must" statements are XPath 1.0 expressions. The syntax of XPath 1.0 is not defined by YANG, but by the XPath 1.0 spec. This is the reason that the YANG grammar isn't more specific. YANG compilers differ in their ability to detect errors in these XPath expressions. Some perform more checks than others (unfortunately, pyang is pretty bad in this regard... (patches are always welcome:)). [For your particular example, referencing a leaf that doesn't exist is not an error per se; it is perfectly valid XPath. But it probably warrants a warning by the compiler.] /martin ___ netmod mailing list netmod@ietf.org https://www.ietf.org/mailman/listinfo/netmod
Re: [netmod] string in when and must statement in YANG ABNF Grammar
Martin, Thanks for this feedback. A run-time crash of the NETCONF server is a bad "outcome" of a typo like this... I checked the W3C pages and there is grammar defined for an XPATH expression so ultimately the string could be parsed as an XPATH (I guess it is currently not done). Any other programming language ensures to eliminate typing errors during the compilation phase allowing early detection of issues like this. It is becoming a problem when the NETCONF server internally stores values (e.g. identityref) in a format different from what is specified in the string of the when statement resulting in unexpected options when trying to use the CLI generated from the model. In this specific case we were advised to prefix the leaf (as the leaf was stored in this way...) even though the leaf was in the local module itself. This makes development even less transparent... When we flagged this we received the following feedback: Quote: For YANG 1.1 that is underway it will be defined in the following way: " The string value of a node of type identityref in a "must" or "when" XPath expression is the referred identity's qualified name with the prefix present. If the referred identity is defined in an imported module, the prefix in the string value is the prefix defined in the corresponding "import" statement. Otherwise, the prefix in the string value is the prefix for the current module."" But as this is within a string that is not parsed (depending on the compiler) this is something that people need to do and if not done will still result in run-time problems. Best regards - Vriendelijke groeten, Bart Bogaert System Architect Data-Centric SW Architectures Fixed Networks - Broadband Access BU, Nokia Contact number +32 3 2408310 (+32 477 673952) -Original Message- From: EXT Martin Bjorklund [mailto:m...@tail-f.com] Sent: 29 March 2016 09:23 To: Bogaert, Bart (Nokia - BE) Cc: netmod@ietf.org Subject: Re: [netmod] string in when and must statement in YANG ABNF Grammar "Bogaert, Bart (Nokia - BE)" wrote: > Is there a reason why the ABNF for example the when and must statement > has been "restricted" to when-keyword sep string optsep > > As "string" can't be tokenized no errors are generated by YANG > compilers if this string contains an error, e.g. referred leaf does > not exit due to a typo. This problem only exposes itself at run-time. > I was wondering why this string was not broken down into a number of > specific parts that define the when statement so that these kind of errors can be trapped early in the > development process? I am rather new in this so I did not follow all > discussions that led to the definition of YANG and hence have no idea > whether this was discussed and what were the reasons not to do it this way. The arguments to the "when" and "must" statements are XPath 1.0 expressions. The syntax of XPath 1.0 is not defined by YANG, but by the XPath 1.0 spec. This is the reason that the YANG grammar isn't more specific. YANG compilers differ in their ability to detect errors in these XPath expressions. Some perform more checks than others (unfortunately, pyang is pretty bad in this regard... (patches are always welcome:)). [For your particular example, referencing a leaf that doesn't exist is not an error per se; it is perfectly valid XPath. But it probably warrants a warning by the compiler.] /martin smime.p7s Description: S/MIME cryptographic signature ___ netmod mailing list netmod@ietf.org https://www.ietf.org/mailman/listinfo/netmod
Re: [netmod] string in when and must statement in YANG ABNF Grammar
"Bogaert, Bart (Nokia - BE)" wrote: > Is there a reason why the ABNF for example the when and must statement has > been "restricted" to when-keyword sep string optsep > > As "string" can't be tokenized no errors are generated by YANG compilers if > this string contains an error, e.g. referred leaf does not exit due to a > typo. This problem only exposes itself at run-time. I was wondering why > this string was not broken down into a number of specific parts that define > the when statement so that these kind of errors can be trapped early in the > development process? I am rather new in this so I did not follow all > discussions that led to the definition of YANG and hence have no idea > whether this was discussed and what were the reasons not to do it this way. The arguments to the "when" and "must" statements are XPath 1.0 expressions. The syntax of XPath 1.0 is not defined by YANG, but by the XPath 1.0 spec. This is the reason that the YANG grammar isn't more specific. YANG compilers differ in their ability to detect errors in these XPath expressions. Some perform more checks than others (unfortunately, pyang is pretty bad in this regard... (patches are always welcome:)). [For your particular example, referencing a leaf that doesn't exist is not an error per se; it is perfectly valid XPath. But it probably warrants a warning by the compiler.] /martin ___ netmod mailing list netmod@ietf.org https://www.ietf.org/mailman/listinfo/netmod
[netmod] string in when and must statement in YANG ABNF Grammar
Is there a reason why the ABNF for example the when and must statement has been "restricted" to when-keyword sep string optsep As "string" can't be tokenized no errors are generated by YANG compilers if this string contains an error, e.g. referred leaf does not exit due to a typo. This problem only exposes itself at run-time. I was wondering why this string was not broken down into a number of specific parts that define the when statement so that these kind of errors can be trapped early in the development process? I am rather new in this so I did not follow all discussions that led to the definition of YANG and hence have no idea whether this was discussed and what were the reasons not to do it this way. Best regards - Vriendelijke groeten, Bart Bogaert smime.p7s Description: S/MIME cryptographic signature ___ netmod mailing list netmod@ietf.org https://www.ietf.org/mailman/listinfo/netmod
