Interesting. I think this is a bug related to
occursCountKind="expression". It seems even if the count is zero we
still output a separator.
Fortuntately, I think I've found a workaround. Instead of using
occursCountKind="expression", use occursCountKind="implicit" (your
default) and put a discriminator on the headerType sequence, e.g.:
<xs:element name="header" type="headerType" minOccurs="0" />
...
<xs:complexType name="headerType">
<xs:sequence dfdl:ref="fieldSeparator">
<xs:annotation>
<xs:appinfo source="http://www.ogf.org/dfdl/";>
<dfdl:discriminator test="{ $header eq 'present' }" />
</xs:appinfo>
</xs:annotation>
<xs:element name="title" maxOccurs="unbounded" type="xs:string" />
</xs:sequence>
</xs:complexType>
So before we start to parse any header fields we check the $header
element and cause a discriminator to fail if it's not "present". This
seems to unparse as expected.
Also note that you no longer need the [1] index on the header element in
the assert expression.
- Steve
On 11/11/19 1:54 PM, Costello, Roger L. wrote:
> > you've made you accidentally removed the newline separator.
>
> Ah! Yes, you are right. Thanks.
>
> Okay, now parsing is working great, but unparsing produces something strange:
> when the CSV file doesn't have a header, it parses fine, but unparsing
> produces
> a blank line for the header:
>
> Why am I getting a blank line at the top? How do I fix it? /Roger
>
> <xs:elementname="csv">
> <xs:complexType>
> <xs:sequence>
> <xs:sequencedfdl:separator="%NL;"dfdl:separatorPosition="infix">
> <xs:elementname="header"type="headerType"minOccurs="0"
> dfdl:occursCountKind="expression"
> dfdl:occursCount="{ if ($header eq 'present') then 1
> else 0
> }"/>
> <xs:elementname="record"type="recordType"maxOccurs="unbounded">
> <xs:annotation>
> <xs:appinfosource="http://www.ogf.org/dfdl/";>
> <dfdl:asserttest="{
> if ($header eq 'present')
> then fn:count(field) eq
> fn:count(../header[1]/title)
> else fn:count(field) eq
> fn:count(../record[1]/field)
> }"
> message="{'Each record should contain the
> same
> number of fields.'}"/>
> </xs:appinfo>
> </xs:annotation>
> </xs:element>
> </xs:sequence>
> <xs:sequencedfdl:hiddenGroupRef="hidden-newline"/>
> </xs:sequence>
> </xs:complexType>
> </xs:element>
>
> -----Original Message-----
> From: Steve Lawrence <[email protected]>
> Sent: Monday, November 11, 2019 12:46 PM
> To: [email protected]
> Subject: [EXT] Re: Statically ambiguous or query-style paths not supported in
> step path ... huh?
>
> Looks like with all the changes you've made you accidentally removed the
> newline
> separator. I think you just need to add back dfdl:separator="%NL;", or
> however
> you were handling the newline before.
>
> - Steve
>
> On 11/11/19 12:23 PM, Costello, Roger L. wrote:
>
> > Thanks for offering to take a look at my schema Steve. I have attached
>
> > my TFDL file. /Roger
>
> >
>
> > -----Original Message-----
>
> > From: Steve Lawrence <[email protected] <mailto:[email protected]>>
>
> > Sent: Monday, November 11, 2019 11:00 AM
>
> > To: [email protected] <mailto:[email protected]>
>
> > Subject: [EXT] Re: Statically ambiguous or query-style paths not supported
> in
> step path ... huh?
>
> >
>
> > I'm not sure based on what you've provided. Can you include the full
> schema?
> Does you data have a trailing EOL or not?
>
> >
>
> > On 11/11/19 10:39 AM, Costello, Roger L. wrote:
>
> >> Thanks Steve. That is a wicked cool approach.
>
> >>
>
> >> Okay, I made the suggested changes. See below. Unfortunately, with
>
> >> this simple input CSV file:
>
> >>
>
> >> Year,Make,Model,Description,Price
>
> >> 1997,Chevy,E350,"ac, abs, moon",2999.99
>
> >>
>
> >> I get this error message:
>
> >>
>
> >> [error] Parse Error: Failed to populate record[1]. Cause: Parse Error:
>
> >> Assertion
>
> >> failed: Each record should contain the same number of fields.
>
> >>
>
> >> Why am I getting that error? I checked the input and the number of
>
> >> field elements in the first record matches the number of title
>
> >> elements in the header. /Roger
>
> >>
>
> >> <xs:elementname="csv">
>
> >> <xs:complexType>
>
> >> <xs:sequence>
>
> >> <xs:elementname="header"type="headerType"minOccurs="0"
>
> >> dfdl:occursCountKind="expression"
>
> >> dfdl:occursCount="{ if ($header eq 'present') then 1
>
> >> else 0 }"/>
>
> >> <xs:elementname="record"type="recordType"maxOccurs="unbounded">
>
> >> <xs:annotation>
>
> >> <xs:appinfosource="http://www.ogf.org/dfdl/";>
>
> >> <dfdl:asserttest="{
>
> >> if ($header eq 'present')
>
> >> then fn:count(field) eq
> fn:count(../header[1]/title)
>
> >> else fn:count(field) eq
> fn:count(../record[1]/field)
>
> >> }"
>
> >> message="{'Each record should contain
>
> >> the same number of fields.'}"/> </xs:appinfo> </xs:annotation>
>
> >> </xs:element> <xs:sequencedfdl:hiddenGroupRef="hidden-newline"/>
>
> >> </xs:sequence>
>
> >> </xs:complexType>
>
> >> </xs:element>
>
> >>
>
> >> -----Original Message-----
>
> >> From: Steve Lawrence <[email protected] <mailto:[email protected]>>
>
> >> Sent: Monday, November 11, 2019 10:08 AM
>
> >> To: [email protected] <mailto:[email protected]>
>
> >> Subject: [EXT] Re: Statically ambiguous or query-style paths not
>
> >> supported in step path ... huh?
>
> >>
>
> >> It is legal to have more than one assertion. Pattern assertions are
>
> >> always evaluated before the data is parsed and expression assertions
>
> >> are evaluated after data is parsed.
>
> >>
>
> >> Regarding the warning, the issue here is the second assert you have:
>
> >>
>
> >> { fn:count(field) eq fn:count(../record[1]/field) }
>
> >>
>
> >> This expression tries to access the first record element with
>
> >> ../record[1]. So it goes up to the parent element (csv) and then
>
> >> steps down to the record array and gets the first element. But there
>
> >> are actually two record arrays that are children of the csv element,
>
> >> one in each branch of the choice. So depending on how things are
>
> >> parsed, this expression could access either of those arrays, which
>
> >> could cause Daffodil to cause an SDE in some cases. (Daffodil is
>
> >> strict about knowing exactly which element will be access, in this case
> it's
> ambiguous so there is a warning).
>
> >>
>
> >> Now, technically Daffodil probably could know that because one record
>
> >> array is in a different choice branch than the expression that it
>
> >> could never actually reference that one, but our expression compilation
> doesn't currently know that.
>
> >> But as schema authors, we do know that the second expression will
>
> >> never get confused and reference the wrong thing, and so the warning
> could
> be ignored.
>
> >>
>
> >> That said, there are two ways to get rid of this warning.
>
> >>
>
> >> 1) Just give your record arrays different names. That's kindof
>
> >> unfortunately because you really do want them to have the same name.
>
> >>
>
> >> 2) Do not use a choice for determining header optionality, but make
>
> >> the header an optional element. Then you truly only have a single
>
> >> record array. So remove the choice, you header element becomes something
> like this:
>
> >>
>
> >> <xs:element name="header" type="headerType" minOccurs="0"
>
> >>
>
> >> dfdl:occursCountKind="expression"
>
> >>
>
> >> dfdl:occursCount="{ if ($header eq 'present') then 1 else 0 }"
>
> >> />
>
> >>
>
> >> And then your record assert becomes something like this:
>
> >>
>
> >> <dfdl:assert test="{
>
> >>
>
> >> if ($header eq 'present')
>
> >>
>
> >> then fn:count(field) eq fn:count(../header[1]/title)
>
> >>
>
> >> else fn:count(field) eq fn:count(../record[1]/field) }" />
>
> >>
>
> >> Note that because header is now optional, Daffodil treats it like an
>
> >> array so you need to access it with the [1] index.
>
> >>
>
> >> - Steve
>
> >>
>
> >> On 11/11/19 7:30 AM, Costello, Roger L. wrote:
>
> >>
>
> >> > Hi Folks,
>
> >>
>
> >> >
>
> >>
>
> >> > Per Brandon's (excellent) suggestion, I added an assertion on each
>
> >>
>
> >> > record element to test that it contains the same number of field
>
> >>
>
> >> > elements as the first record. See below. Notice that I have a
>
> >> choice
>
> >>
>
> >> > to deal with the case where the CSV file does and doesn't have a
>
> >>
>
> >> > header. Notice in the second branch of the choice I have two
>
> >>
>
> >> > assertions - is it legal to have multiple assertions? When I run
>
> >> my schema I get this error message:
>
> >>
>
> >> >
>
> >>
>
> >> > *[warning] Schema Definition Warning: Statically ambiguous or
>
> >>
>
> >> > query-style paths not supported in step path: '{}record'.*
>
> >>
>
> >> >
>
> >>
>
> >> > What does that mean? How to fix it? /Roger
>
> >>
>
> >> >
>
> >>
>
> >> > <xs:elementname="csv">
>
> >>
>
> >> > <xs:complexType>
>
> >>
>
> >> > <xs:sequence>
>
> >>
>
> >> > <xs:choicedfdl:choiceDispatchKey="{$header}">
>
> >>
>
> >> > <xs:sequencedfdl:choiceBranchKey="present">
>
> >>
>
> >> > <xs:sequencedfdl:separator="%NL;"dfdl:separatorPosition="infix">
>
> >>
>
> >> > <xs:elementname="header"type="headerType"/>
>
> >>
>
> >> > <xs:elementname="record"type="recordType"maxOccurs="unbounded">
>
> >>
>
> >> > <xs:annotation>
>
> >>
>
> >> > <xs:appinfosource="http://www.ogf.org/dfdl/";>
>
> >>
>
> >> > <dfdl:asserttest="{ fn:count(field) eq fn:count(../header/title) }"
>
> >>
>
> >> > message="{'Each record should
>
> >>
>
> >> > contain the same number of fields as the header.'}"/>
>
> >> </xs:appinfo>
>
> >>
>
> >> > </xs:annotation> </xs:element> </xs:sequence> </xs:sequence>
>
> >>
>
> >> > <xs:sequencedfdl:choiceBranchKey="absent">
>
> >>
>
> >> > <xs:sequencedfdl:separator="%NL;"dfdl:separatorPosition="infix">
>
> >>
>
> >> > <xs:elementname="record"type="recordType"maxOccurs="unbounded">
>
> >>
>
> >> > <xs:annotation>
>
> >>
>
> >> > <xs:appinfosource="http://www.ogf.org/dfdl/";>
>
> >>
>
> >> > <dfdl:asserttest="{ fn:count(field) eq fn:count(../record[1]/field) }"
>
> >>
>
> >> > message="{'Each record should
>
> >>
>
> >> > contain the same number of fields.'}"/>
>
> >>
>
> >> > <dfdl:asserttestKind="pattern"testPattern="."
>
> >>
>
> >> > message="{'If the last record
>
> >>
>
> >> > is empty, then no fields should be generated'}"/> </xs:appinfo>
>
> >>
>
> >> > </xs:annotation> </xs:element> </xs:sequence> </xs:sequence>
>
> >>
>
> >> > </xs:choice> <xs:sequencedfdl:hiddenGroupRef="hidden-newline"/>
>
> >>
>
> >> > </xs:sequence>
>
> >>
>
> >> > </xs:complexType>
>
> >>
>
> >> > </xs:element>
>
> >>
>
> >> >
>
> >>
>
> >
>
