I think we should just remove the choiceElement entirely. A new
UnorderedSequenceCombinatorParser should be able to handle all the logic
that is currently handled by converting unordered sequences to a
sequence of choices (which looks to be done in the lazy val unorderedSeq
in SequenceGroup.scala). It does seem like this expressions accessing
other branches of an unordered sequences should be an SDE.
On 11/30/2017 03:39 PM, Taylor Wise wrote:
> By this same token then, any assertions/expressions that navigate to a
> different
> branch of the unordered sequence/choice are invalid.
>
>
> "When processing a choice group the parser validates any contained path
> expressions. If a path expression contained inside a choice branch refers to
> any
> other branch of the choice, then it is a schema definition error. Note that
> this
> rule handles nested choices also. A path that navigates outward from an inner
> choice to another alternative of an outer choice is violating this rule with
> respect to the outer choice."
>
>
> I'm currently running into an issue where there's an existing test that has
> an
> expression in the initiator of an element in an unordered sequence. The
> expression refers to a prior element. Because of the spec, this should SDE.
>
>
> However, it's currently failing stating that the element does not exist when
> it
> should have already been added. This is leading me to believe that there is
> either something funky going on with the expression steps due to the
> additional
> choiceElement or that somehow the element (e3) isn't being kept around in the
> infoset (unlikely as tests without expressions run fine).
>
>
> The existing test:
>
>
> <xs:element name="USG_01">
> <xs:complexType>
> <xs:sequence dfdl:sequenceKind="unordered"
> dfdl:separator=",">
> <xs:element name="e1" type="xs:int" dfdl:lengthKind="delimited"
> dfdl:initiator="a"/>
>
> <xs:element name="e2" type="xs:int" dfdl:lengthKind="delimited"
> dfdl:initiator="b"/>
>
> <xs:element name="e3" type="xs:int" dfdl:lengthKind="delimited"
> dfdl:initiator="c"/>
>
> <xs:element name="e4" type="xs:string" dfdl:lengthKind="delimited"
> dfdl:initiator="{ ../ex:e3 }"/>
> </xs:sequence>
> </xs:complexType>
> </xs:element>
>
> <tdml:parserTestCase name="test_simple_invalid_path_to_branch"
> model="simple_invalid_path_to_branch" description="Section 14 Sequence group
> with left over data - DFDL-14-001R"
> root="USG_01" roundTrip="false">
> <tdml:document><![CDATA[b2,a1,a4,c3,ruh-oh!]]></tdml:document>
> <tdml:errors>
> <tdml:error>Schema Definition Error</tdml:error>
> <tdml:error>Expression</tdml:error>
> <tdml:error>../e3</tdml:error>
> <tdml:error>navigates to another branch</tdml:error>
> <tdml:error>e3</tdml:error>
> </tdml:errors>
> </tdml:parserTestCase>
> edu.illinois.ncsa.daffodil.tdml.TDMLException: Did not find diagnostic
> message
> "../e3" in any of the actual diagnostic messages:
> Runtime Schema Definition Error: Expression Evaluation Error: Child element
> {http://example.com}e3 does not exist.
> Schema context: e4 Location line 199 column 22 in
> file:/tmp/simple_invalid_path_to_branch1343627752058513822.dfdl.xsd
> Data location was preceding byte 12 limit(bytes) 19
>
> The source-to-source transformation:
> <xs:element name="USG_01">
> <xs:complexType>
> <xs:sequence dfdl:sequenceKind="ordered"
> dfdl:separator=",">
> <xs:element name="choiceElement" minOccurs="0" maxOccurs="unbounded"
> occursCountKind="parsed">
> <xs:complexType>
> <xs:choice dfdl:choiceLengthKind="implicit">
> <xs:element name="e1" type="xs:int" dfdl:lengthKind="delimited"
> dfdl:initiator="a"/>
>
> <xs:element name="e2" type="xs:int" dfdl:lengthKind="delimited"
> dfdl:initiator="b"/>
>
> <xs:element name="e3" type="xs:int" dfdl:lengthKind="delimited"
> dfdl:initiator="c"/>
> <xs:element name="e4" type="xs:string" dfdl:lengthKind="delimited"
> dfdl:initiator="{ ../ex:e3 }"/>
> </xs:choice>
> </xs:complexType>
> </xs:element>
> </xs:sequence>
> </xs:complexType>
> </xs:element>
>
> In this case, e3 should definitely exist by the time the choice alternative
> for
> e4 is run. Stepping through, I can see each alternative succeed for each
> element
> until it tries e4.
> At which time, it doesn't appear that e3 is in the infoset (though it should
> be). What appears to be happening is that each time an alternative succeeds,
> a choiceElement (containing the parsed element) is added to USG_01. So I
> think
> what's happening is that these choiceElement's are interfering with the
> expression evaluations.
>
> You end up with something like this in the Infoset:
> <USG_01>
> <choiceElement><e2>2</e2></choiceElement>
> <choiceElement><e1>1</e1></choiceElement>
> <choiceElement><e1>4</e1></choiceElement>
> <choiceElement><e3>3</e3></choiceElement>
> <choiceElement> ....Let's look for the Initiator from e3... </choiceElement>
> </USG_01>
>
> The expression is expecting to be able to go UP to USG_01 and ask for a child
> named e3. But, it would appear that it's going UP to its choiceElement and
> asking for e3.
> Which does not exist.
>
> Which brings me to two possible solutions:
>
> 1. Somehow detect that we're running an expression from within a choice and
> 'hide/skip' choiceElements. Likely meaning we'll have to prepend an extra
> ../ somewhere. This adds some unnecessary complexity to the expression
> code. Which I kind of want to avoid.
> 2. Remove the choiceElement entirely.
>
>
>
>
>
>
> --------------------------------------------------------------------------------
> *From:* Steve Lawrence <slawre...@apache.org>
> *Sent:* Friday, November 17, 2017 1:37:46 PM
> *To:* dev@daffodil.apache.org; Taylor Wise; Mike Beckerle
> *Subject:* Re: Unordered Sequence and Choice Branch Violations
> On 11/17/2017 01:13 PM, Taylor Wise wrote:
>> While executing some existing tests I've come across a case that I need
>> guidance on.
>>
>>
>> Given the following test case:
>>
>> <tdml:defineSchema name="simple_invalid_path_to_branch">
>> <dfdl:format ref="ex:daffodilTest1" lengthUnits="characters"
>> lengthKind="delimited" />
>> <xs:element name="USG_01">
>> <xs:complexType>
>> <xs:sequence dfdl:sequenceKind="unordered"
>> dfdl:separator=",">
>>
>> <xs:element name="e1" type="xs:int" dfdl:lengthKind="delimited"
>> dfdl:initiator="a">
>> </xs:element>
>>
>> <xs:element name="e2" type="xs:int" dfdl:lengthKind="delimited"
>> dfdl:initiator="b">
>> </xs:element>
>>
>> <xs:element name="e3" type="xs:int" dfdl:lengthKind="delimited"
>> dfdl:initiator="c">
>> </xs:element>
>>
>> <xs:element name="e4" type="xs:string" dfdl:lengthKind="delimited"
>> dfdl:inputValueCalc="{ ../ex:e3 }">
>> </xs:element>
>>
>> </xs:sequence>
>> </xs:complexType>
>> </xs:element>
>>
>>
>> <tdml:parserTestCase name="test_simple_invalid_path_to_branch"
>> model="simple_invalid_path_to_branch" description="Section 14 Sequence group
>> with left over data - DFDL-14-001R"
>> root="USG_01" roundTrip="false">
>> <tdml:document><![CDATA[b2,a1,a4,c3,ruh-oh!]]></tdml:document>
>> <tdml:errors>
>> <tdml:error>Schema Definition Error</tdml:error>
>> <tdml:error>Expression</tdml:error>
>> <tdml:error>../e3</tdml:error>
>> <tdml:error>navigates to another branch</tdml:error>
>> <tdml:error>e3</tdml:error>
>> </tdml:errors>
>> </tdml:parserTestCase>
>>
>> The result of the test is the following:
>>
>> edu.illinois.ncsa.daffodil.tdml.TDMLException: Did not find diagnostic
>> message "Expression" in any of the actual diagnostic messages:
>> Schema Definition Error: Branch of choice
>> ex:USG_01::LocalComplexTypeDef::sequence::choiceElement::LocalComplexTypeDef::choice::e4
>> cannot have the dfdl:inputValueCalc property.
>> Schema context: choice Location line 184 in
>> file:/tmp/simple_invalid_path_to_branch4208585747499063196.dfdl.xsd
>>
>> Now, in the test you'll notice that there's no choice listed in the schema.
>> But because it's an unordered sequence, it gets transformed into a sequence
>> of choice.
>>
>> Something like:
>>
>> <xs:sequence dfdl:sequenceKind="ordered" dfdl:separator=",">
>>
>> <xs:element name="choiceElement" minOccurs="0" maxOccurs="unbounded"
>> dfdl:occursCountKind="parsed">
>>
>> <xs:complexType>
>>
>> <xs:choice dfdl:choiceLengthKind="implicit">
>>
>> <xs:element name="e1" type="xs:string" dfdl:initiator="a" />
>>
>> <xs:element name="e2" type="xs:int" dfdl:initiator="b" />
>>
>> <xs:element name="e3" type="xs:string" dfdl:initiator="c" />
>>
>> <xs:element name="e4" type="xs:string" dfdl:inputValueCalc="{ ../ex:e3 }" />
>>
>> </xs:choice>
>>
>> </xs:complexType>
>>
>> </xs:element>
>>
>> </xs:sequence>
>>
>> So does this now mean that this schema has to obey all the rules of choice?
>> That's not immediately obvious.
>> The spec states that IVC is "not allowed to appear on a local element or
>> element reference that is the root of a choice branch". e4 is now the root
>> of a choice branch.
>>
>> Offending code:
>> final def branchesAreNotIVCElements = LV('branchesAreNotIVCElements) {
>> val branchesOk = groupMembersNoRefs map { branch =>
>> if (!branch.isRepresented) {
>> schemaDefinitionErrorButContinue("Branch of choice %s cannot have
>> the dfdl:inputValueCalc property.".format(branch.path))
>> false
>> } else true
>> }
>> assuming(branchesOk.forall { x => x })
>> }.value
>>
>>
>> About IVC from the spec:
>> It is a schema definition error if this property is specified on an element
>> which has an XSDL fixed or default property.
>> It is a schema definition error if dfdl:inputValueCalc and
>> dfdl:outputValueCalc are specified on the same element.
>> It is not possible to place this property in scope on a dfdl:format
>> annotation.
>> This property is not allowed to appear on a local element or element
>> reference that is the root of a choice branch.
>> If this property appears on an element declaration or element reference
>> schema component, the appearance of any other DFDL properties on that
>> component is a schema definition error.
>> If this property appears on an element reference, then DFDL properties
>> expressed on the referenced global element declaration or its type are
>> ignored.
>> If this property appears on an element declaration, then DFDL properties
>> expressed on its type are ignored.
>>
>
> Seems to me like this shouldn't be allowed to me. I don't think
> inputValueCalc elements even consume any data, so it doesn't make sense
> to be in an unordered sequence. Otherwise you could parse
> inputValueCalc's forever as part of that sequence.
>
> I'll also add that if we add a new UnordredSequenceCombinator, I would
> think we could get rid of the code that creates a fake sequence of
> choices. That always seemed a bit of a hack and made things awkward, for
> example the "choiceElement" showing up in the path.
>
