Hi, now that I'm sort of allmost finished with adjusting the mspec parsers and model types, I'm planning up update the code generation templates.
I know that if I do it the normal way, that this would excessivley blow up the complexity and size of the template files as well as that of the produced code. As this code has become more and more complex over the last year anyway, I would like to radically clean up the code here. For that I had an idea, that I formalized a bit in the Confluence page: https://cwiki.apache.org/confluence/display/PLC4X/MSPEC+improvements TL/DR: I would create static methods that handle the reading/writing of every field type and simply use these instead of generating code for every field. Feel free to comment, Chris -----Ursprüngliche Nachricht----- Von: Christofer Dutz <christofer.d...@c-ware.de> Gesendet: Montag, 4. Oktober 2021 08:55 An: dev@plc4x.apache.org Betreff: AW: Changes on mspec: parameterized type refs, assert, try, const Hi all, so over the weekend Sebastian and I have been working hard on an updated mspec version. In the "feature/mspec-ng" is where all of this is happening. So far the changes: - We introduced a concept of "attributes" which can be added to fields and types (These are name=expression) tuples. These are added before any argument-block. These can currently be used for setting the encoding for string types (possibly even for other types in case we need them). And for the upcomming "endianess" attribute which should allow switching endianess. - We introduced a new simple type "vstring" for variable length strings. The expression however is optional. - The previous "string" now only accepts fixed number of bits. - The encoding for String fields (string and vstring) defaults to "UTF-8" but you can override it with an attribute with: encoding='"UTF-16"' - The previous encoding string has been removed from string types) - We added a new type of "field": "batchSet" which takes a list of attributes. These are automatically added to all fields it contains. (Currently this is only available in "type", "complexTyte" elements or inside typeSwitch case elements. I think this was about it for now ... still need to implement the attribute functionality ... it's currently parsed, but not processed yet. Chris -----Ursprüngliche Nachricht----- Von: Christofer Dutz <christofer.d...@c-ware.de> Gesendet: Dienstag, 28. September 2021 14:57 An: dev@plc4x.apache.org Betreff: AW: Changes on mspec: parameterized type refs, assert, try, const Another thought I had last night ... while sort of not being able to sleep (at least it hat something good) So I think a "try" field should always be an optional field. As it might be set or not, just with the difference that the condition is different. This got me thinking even further ... how about making the condition of an "optional" field optional? ... So in general what Sebastian was proposing with a: [try simple SomeType 'coolField'] Would actually be: [optional SomeType 'coolField'] This would simplify things even more... Chris -----Ursprüngliche Nachricht----- Von: Christofer Dutz <christofer.d...@c-ware.de> Gesendet: Sonntag, 26. September 2021 16:03 An: dev@plc4x.apache.org Betreff: AW: Changes on mspec: parameterized type refs, assert, try, const Bringing the thought of the other discussion here too ... How about adding name-value paris to the type declarations as well as to the fields? Then the "try" flag could be replaced with a name-value pair or even a name=expression pair ... sort of like "parse-behaviour=try" or something similar. Then we wouldn't be starting to add all sorts of "flags". Chris -----Ursprüngliche Nachricht----- Von: Sebastian Rühl <sru...@apache.org> Gesendet: Sonntag, 26. September 2021 15:44 An: dev@plc4x.apache.org Betreff: Re: Changes on mspec: parameterized type refs, assert, try, const Hey Lukaz, the optional after the try is just a coincidence from the bac spec. Like it say the low should always appear with a high and if the id is set then no name should not be set. But in itself the try is self sufficient and behaves indeed like an optional without an expression. Other than that you can always group dependended fields as a complex... but let's see how this evolves. These blocks could be indeed come in handy as an option to define things... - Sebastian On 2021/09/25 20:23:08, Łukasz Dywicki <l...@code-house.org> wrote: > Hey Sebastian, > Not that far ago we had a similar discussion. Yet then it was mainly > about optionals. See "[DISCUSS] Change the way we use "optional". > > Back then we did not do any changes to mspec, just ranted about > possible approach to make better use of optional fields. > I remember that current mspec/code can't for example handle ie. a > bacnet whois without all fields. Looking at your example I guess you > try to solve that puzzle. > > I don't have any strong feelings here, but why not using optional > field with additional "reset" flag? > To me what rings a bell in your mail is an assumption that try is > always followed by optional field. This tells me that 'try' should > rather be a block than a field. By this way we will be able to pair > these visually as well as handle situation when there is one try > statement but multiple optional fields. > > Best, > Łukasz > > > On 24.09.2021 23:52, Sebastian Rühl wrote: > > Hi together, > > > > I have some exciting changes in the pipeline regarding the mspec: > > > > 1. parameters on type refs > > with that change it is now possible to target a discriminated child in > > advance. > > 2. assert keyword > > with that change it is possible to throw a > > ParserAssertException (in java, or errors in other languages). This field > > is similar to a const but instead of a ParseException a > > ParserAssertException is thrown. In contrast to a const the check > > expression can be dynamic (e.g. virtual fields now working on develop) 3. > > try keyword to prefix fields: > > with that change it is possible to try to parse some content and in > > case an assert fails it resets the buffer. > > 4. const is now extended to type reference > > this change allows enums to be used as const values. > > > > All theses changes allow to encapsulate behavior in complex types so you > > don't need to DRY. > > > > Here is a example working with bacnet: > > ['0x07' BACnetUnconfirmedServiceRequestWhoHas > > [try simple BACnetComplexTagUnsignedInteger ['0', > > 'BACnetDataType.UNSIGNED_INTEGER' ] 'deviceInstanceRangeLowLimit' ] > > [optional BACnetComplexTagUnsignedInteger ['1', > > 'BACnetDataType.UNSIGNED_INTEGER' ] 'deviceInstanceRangeHighLimit' > > 'deviceInstanceRangeLowLimit != null'] > > [try simple BACnetComplexTagOctetString ['2', > > 'BACnetDataType.OCTET_STRING' ] 'objectIdentifier' ] > > [optional BACnetComplexTagOctetString ['3', > > 'BACnetDataType.OCTET_STRING' ] 'objectName' 'objectIdentifier == null' > > ] > > ] > > > > The logic if a type matches is asserted in the type itself. The second > > optional implies when the first element appears the second must be present. > > The last one tries to read and if it fails it uses the second type. > > > > Here is the snippet from the parent type: > > > > [discriminatedType 'BACnetComplexTag' [uint 4 'tagNumberArgument', > > BACnetDataType 'dataType'] > > [assert uint 4 'tagNumber' > > 'tagNumberArgument' ] > > [const TagClass 'tagClass' > > 'TagClass.CONTEXT_SPECIFIC_TAGS' ] > > [simple uint 3 'lengthValueType' > > ] > > ..... > > [virtual uint 32 'actualLength' 'lengthValueType == > > 5 ....'] > > [typeSwitch 'dataType' > > .... > > ['OCTET_STRING' BACnetComplexTagOctetString [uint 32 > > 'actualLength'] > > // TODO: The reader expects int but uint32 get's mapped to > > long so even uint32 would easily overflow... > > [virtual uint 16 > > 'actualLengthInBit' 'actualLength * 8'] > > [simple string 'actualLengthInBit' 'ASCII' 'theString'] > > ] > > > > Would love to hear some opinions! If there are no objections I would push > > this change to develop soon. > > > > - Sebastian > > > > PatchContent: > > Index: > > code-generation/protocol-base-mspec/src/main/antlr4/org/apache/plc4x > > /plugins/codegenerator/language/mspec/MSpec.g4 > > <+>UTF-8 > > =================================================================== > > diff --git > > a/code-generation/protocol-base-mspec/src/main/antlr4/org/apache/plc4x/plugins/codegenerator/language/mspec/MSpec.g4 > > > > b/code-generation/protocol-base-mspec/src/main/antlr4/org/apache/plc4x/plugins/codegenerator/language/mspec/MSpec.g4 > > --- > > a/code-generation/protocol-base-mspec/src/main/antlr4/org/apache/plc4x/plugins/codegenerator/language/mspec/MSpec.g4 > > (revision ef35531d5a872f29dccddb3a11a135b166958185) > > +++ > > b/code-generation/protocol-base-mspec/src/main/antlr4/org/apache/plc4x/plugins/codegenerator/language/mspec/MSpec.g4 > > (date 1632518519426) > > @@ -34,7 +34,7 @@ > > ; > > > > fieldDefinition > > - : LBRACKET field (LBRACKET params=multipleExpressions RBRACKET)? > > RBRACKET > > + : LBRACKET tryParse? field (LBRACKET params=multipleExpressions > > + RBRACKET)? RBRACKET > > ; > > > > dataIoDefinition > > @@ -49,6 +49,7 @@ > > | discriminatorField > > | enumField > > | implicitField > > + | assertField > > | manualArrayField > > | manualField > > | optionalField > > @@ -73,7 +74,7 @@ > > ; > > > > constField > > - : 'const' type=dataType name=idExpression expected=expression > > + : 'const' type=typeReference name=idExpression expected=expression > > ; > > > > discriminatorField > > @@ -88,6 +89,10 @@ > > : 'implicit' type=dataType name=idExpression > > serializeExpression=expression > > ; > > > > +assertField > > + : 'assert' type=typeReference name=idExpression > > +condition=expression ; > > + > > manualArrayField > > : 'manualArray' type=typeReference name=idExpression > > loopType=ARRAY_LOOP_TYPE loopExpression=expression > > parseExpression=expression serializeExpression=expression > > lengthExpression=expression > > ; > > @@ -129,7 +134,7 @@ > > ; > > > > typeReference > > - : complexTypeReference=IDENTIFIER_LITERAL > > + : complexTypeReference=IDENTIFIER_LITERAL (LBRACKET > > params=multipleExpressions RBRACKET)? > > | simpleTypeReference=dataType > > ; > > > > @@ -150,6 +155,10 @@ > > | base='dateTime' > > ; > > > > +tryParse > > + : 'try' > > + ; > > + > > argument > > : type=typeReference name=idExpression > > ; > > >
