Fantastic! Thank you Steve! Now both parsing and unparsing is working perfectly.
/Roger -----Original Message----- From: Steve Lawrence <[email protected]> Sent: Monday, November 11, 2019 2:16 PM To: [email protected] Subject: [EXT] Re: Statically ambiguous or query-style paths not supported in step path ... huh? Interesting. I think this is a bug related to occursCountKind="expression". It seems even if the count is zero we still output a separator. Fortuntately, I think I've found a workaround. Instead of using occursCountKind="expression", use occursCountKind="implicit" (your default) and put a discriminator on the headerType sequence, e.g.: <xs:element name="header" type="headerType" minOccurs="0" /> ... <xs:complexType name="headerType"> <xs:sequence dfdl:ref="fieldSeparator"> <xs:annotation> <xs:appinfo source="http://www.ogf.org/dfdl/";> <dfdl:discriminator test="{ $header eq 'present' }" /> </xs:appinfo> </xs:annotation> <xs:element name="title" maxOccurs="unbounded" type="xs:string" /> </xs:sequence> </xs:complexType> So before we start to parse any header fields we check the $header element and cause a discriminator to fail if it's not "present". This seems to unparse as expected. Also note that you no longer need the [1] index on the header element in the assert expression. - Steve On 11/11/19 1:54 PM, Costello, Roger L. wrote: > > you've made you accidentally removed the newline separator. > > Ah! Yes, you are right. Thanks. > > Okay, now parsing is working great, but unparsing produces something strange: > when the CSV file doesn't have a header, it parses fine, but unparsing > produces a blank line for the header: > > Why am I getting a blank line at the top? How do I fix it? /Roger > > <xs:elementname="csv"> > <xs:complexType> > <xs:sequence> > <xs:sequencedfdl:separator="%NL;"dfdl:separatorPosition="infix"> > <xs:elementname="header"type="headerType"minOccurs="0" > dfdl:occursCountKind="expression" > dfdl:occursCount="{ if ($header eq 'present') > then 1 else 0 }"/> > <xs:elementname="record"type="recordType"maxOccurs="unbounded"> > <xs:annotation> > <xs:appinfosource="http://www.ogf.org/dfdl/";> > <dfdl:asserttest="{ > if ($header eq 'present') > then fn:count(field) eq > fn:count(../header[1]/title) > else fn:count(field) eq > fn:count(../record[1]/field) > }" > message="{'Each record should contain > the same number of fields.'}"/> </xs:appinfo> </xs:annotation> > </xs:element> </xs:sequence> > <xs:sequencedfdl:hiddenGroupRef="hidden-newline"/> > </xs:sequence> > </xs:complexType> > </xs:element> > > -----Original Message----- > From: Steve Lawrence <[email protected]> > Sent: Monday, November 11, 2019 12:46 PM > To: [email protected] > Subject: [EXT] Re: Statically ambiguous or query-style paths not > supported in step path ... huh? > > Looks like with all the changes you've made you accidentally removed > the newline separator. I think you just need to add back > dfdl:separator="%NL;", or however you were handling the newline before. > > - Steve > > On 11/11/19 12:23 PM, Costello, Roger L. wrote: > > > Thanks for offering to take a look at my schema Steve. I have > attached > > > my TFDL file. /Roger > > > > > > -----Original Message----- > > > From: Steve Lawrence <[email protected] > <mailto:[email protected]>> > > > Sent: Monday, November 11, 2019 11:00 AM > > > To: [email protected] <mailto:[email protected]> > > > Subject: [EXT] Re: Statically ambiguous or query-style paths not > supported in step path ... huh? > > > > > > I'm not sure based on what you've provided. Can you include the full > schema? > Does you data have a trailing EOL or not? > > > > > > On 11/11/19 10:39 AM, Costello, Roger L. wrote: > > >> Thanks Steve. That is a wicked cool approach. > > >> > > >> Okay, I made the suggested changes. See below. Unfortunately, with > > >> this simple input CSV file: > > >> > > >> Year,Make,Model,Description,Price > > >> 1997,Chevy,E350,"ac, abs, moon",2999.99 > > >> > > >> I get this error message: > > >> > > >> [error] Parse Error: Failed to populate record[1]. Cause: Parse Error: > > >> Assertion > > >> failed: Each record should contain the same number of fields. > > >> > > >> Why am I getting that error? I checked the input and the number of > > >> field elements in the first record matches the number of title > > >> elements in the header. /Roger > > >> > > >> <xs:elementname="csv"> > > >> <xs:complexType> > > >> <xs:sequence> > > >> <xs:elementname="header"type="headerType"minOccurs="0" > > >> dfdl:occursCountKind="expression" > > >> dfdl:occursCount="{ if ($header eq 'present') then 1 > > >> else 0 }"/> > > >> <xs:elementname="record"type="recordType"maxOccurs="unbounded"> > > >> <xs:annotation> > > >> <xs:appinfosource="http://www.ogf.org/dfdl/";> > > >> <dfdl:asserttest="{ > > >> if ($header eq 'present') > > >> then fn:count(field) eq > fn:count(../header[1]/title) > > >> else fn:count(field) eq > fn:count(../record[1]/field) > > >> }" > > >> message="{'Each record should contain > > >> the same number of fields.'}"/> </xs:appinfo> </xs:annotation> > > >> </xs:element> <xs:sequencedfdl:hiddenGroupRef="hidden-newline"/> > > >> </xs:sequence> > > >> </xs:complexType> > > >> </xs:element> > > >> > > >> -----Original Message----- > > >> From: Steve Lawrence <[email protected] > <mailto:[email protected]>> > > >> Sent: Monday, November 11, 2019 10:08 AM > > >> To: [email protected] <mailto:[email protected]> > > >> Subject: [EXT] Re: Statically ambiguous or query-style paths not > > >> supported in step path ... huh? > > >> > > >> It is legal to have more than one assertion. Pattern assertions > are > > >> always evaluated before the data is parsed and expression > assertions > > >> are evaluated after data is parsed. > > >> > > >> Regarding the warning, the issue here is the second assert you have: > > >> > > >> { fn:count(field) eq fn:count(../record[1]/field) } > > >> > > >> This expression tries to access the first record element with > > >> ../record[1]. So it goes up to the parent element (csv) and then > > >> steps down to the record array and gets the first element. But > there > > >> are actually two record arrays that are children of the csv > element, > > >> one in each branch of the choice. So depending on how things are > > >> parsed, this expression could access either of those arrays, which > > >> could cause Daffodil to cause an SDE in some cases. (Daffodil is > > >> strict about knowing exactly which element will be access, in this > case it's ambiguous so there is a warning). > > >> > > >> Now, technically Daffodil probably could know that because one > record > > >> array is in a different choice branch than the expression that it > > >> could never actually reference that one, but our expression > compilation doesn't currently know that. > > >> But as schema authors, we do know that the second expression will > > >> never get confused and reference the wrong thing, and so the > warning could be ignored. > > >> > > >> That said, there are two ways to get rid of this warning. > > >> > > >> 1) Just give your record arrays different names. That's kindof > > >> unfortunately because you really do want them to have the same name. > > >> > > >> 2) Do not use a choice for determining header optionality, but > make > > >> the header an optional element. Then you truly only have a single > > >> record array. So remove the choice, you header element becomes > something like this: > > >> > > >> <xs:element name="header" type="headerType" minOccurs="0" > > >> > > >> dfdl:occursCountKind="expression" > > >> > > >> dfdl:occursCount="{ if ($header eq 'present') then 1 else 0 }" > > >> /> > > >> > > >> And then your record assert becomes something like this: > > >> > > >> <dfdl:assert test="{ > > >> > > >> if ($header eq 'present') > > >> > > >> then fn:count(field) eq fn:count(../header[1]/title) > > >> > > >> else fn:count(field) eq fn:count(../record[1]/field) }" /> > > >> > > >> Note that because header is now optional, Daffodil treats it like > an > > >> array so you need to access it with the [1] index. > > >> > > >> - Steve > > >> > > >> On 11/11/19 7:30 AM, Costello, Roger L. wrote: > > >> > > >> > Hi Folks, > > >> > > >> > > > >> > > >> > Per Brandon's (excellent) suggestion, I added an assertion on > each > > >> > > >> > record element to test that it contains the same number of > field > > >> > > >> > elements as the first record. See below. Notice that I have a > > >> choice > > >> > > >> > to deal with the case where the CSV file does and doesn't have > a > > >> > > >> > header. Notice in the second branch of the choice I have two > > >> > > >> > assertions - is it legal to have multiple assertions? When I > run > > >> my schema I get this error message: > > >> > > >> > > > >> > > >> > *[warning] Schema Definition Warning: Statically ambiguous or > > >> > > >> > query-style paths not supported in step path: '{}record'.* > > >> > > >> > > > >> > > >> > What does that mean? How to fix it? /Roger > > >> > > >> > > > >> > > >> > <xs:elementname="csv"> > > >> > > >> > <xs:complexType> > > >> > > >> > <xs:sequence> > > >> > > >> > <xs:choicedfdl:choiceDispatchKey="{$header}"> > > >> > > >> > <xs:sequencedfdl:choiceBranchKey="present"> > > >> > > >> > > <xs:sequencedfdl:separator="%NL;"dfdl:separatorPosition="infix"> > > >> > > >> > <xs:elementname="header"type="headerType"/> > > >> > > >> > <xs:elementname="record"type="recordType"maxOccurs="unbounded"> > > >> > > >> > <xs:annotation> > > >> > > >> > <xs:appinfosource="http://www.ogf.org/dfdl/";> > > >> > > >> > <dfdl:asserttest="{ fn:count(field) eq fn:count(../header/title) }" > > >> > > >> > message="{'Each record should > > >> > > >> > contain the same number of fields as the header.'}"/> > > >> </xs:appinfo> > > >> > > >> > </xs:annotation> </xs:element> </xs:sequence> </xs:sequence> > > >> > > >> > <xs:sequencedfdl:choiceBranchKey="absent"> > > >> > > >> > > <xs:sequencedfdl:separator="%NL;"dfdl:separatorPosition="infix"> > > >> > > >> > <xs:elementname="record"type="recordType"maxOccurs="unbounded"> > > >> > > >> > <xs:annotation> > > >> > > >> > <xs:appinfosource="http://www.ogf.org/dfdl/";> > > >> > > >> > <dfdl:asserttest="{ fn:count(field) eq fn:count(../record[1]/field) }" > > >> > > >> > message="{'Each record should > > >> > > >> > contain the same number of fields.'}"/> > > >> > > >> > <dfdl:asserttestKind="pattern"testPattern="." > > >> > > >> > message="{'If the last record > > >> > > >> > is empty, then no fields should be generated'}"/> </xs:appinfo> > > >> > > >> > </xs:annotation> </xs:element> </xs:sequence> </xs:sequence> > > >> > > >> > </xs:choice> <xs:sequencedfdl:hiddenGroupRef="hidden-newline"/> > > >> > > >> > </xs:sequence> > > >> > > >> > </xs:complexType> > > >> > > >> > </xs:element> > > >> > > >> > > > >> > > > >
