This is an automated email from the ASF dual-hosted git repository.
slawrence pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/daffodil.git
The following commit(s) were added to refs/heads/main by this push:
new d83a86086 Fix property scoping of default properties
d83a86086 is described below
commit d83a86086dec799bb4acb7e31a8617b5a597754a
Author: Steve Lawrence <[email protected]>
AuthorDate: Fri Oct 31 11:34:01 2025 -0400
Fix property scoping of default properties
Section 8.1.4, Rule 5 of the DFDL specification describes the
scoping/resolution of default properties, saying:
Obtain applicable "default" properties from a dfdl:format annotation
on the xs:schema that contains the component (if such annotation is
present). Combine these with the current working set of "default"
properties, the latter overriding the former (that is, inner wins).
Result is a new working set of "default" properties.
So if a schema component does not explicitly define a property, we
should look for default properties on the innermost components first.
But Daffodil currently does the reverse, looking at the default
properties of the outermost components first. This happens because we
build the default component property chain list recursively, prepending
the outermost components default chain to the list. This means wen we
scan for default values we look at the outermost chain first.
The solution to this is to reverse the default component chain so that
when we scan it for properties the innermost components appear first and
are given precedence.
Note that it is possible that some schemas unintentionally relied on
this buggy behavior, and it can sometimes be very difficult to figure
out which properties changed due to the change in property resolution.
To aid in debugging, we keep the old reversed list and when a property
is found in the default list, we also find it in the reversed default
list. If the property value found in the reversed list is different than
found using the correct list, this indicates the new logic could have
changed a relevant property value and we output a warning. In many cases
this can actually be ignored since the property values might not
actually matter, but when things break this can be a useful diagnostic.
Deprecation/Compatibility:
When resolving default properties, Daffodil now gives precedence to
innermost referenced schema components rather than the outermost, as
described by section 8.1.4 of the DFDL specification. If this new logic
results in a different property value than previous versions of
Daffodil then a schema definition warning is issued.
DAFFODIL-2855
---
.../core/dsom/AnnotatedSchemaComponent.scala | 39 ++++++++++++++--
...xample_a02_targetnamespace_unqualified.dfdl.xsd | 2 +-
.../resources/org/apache/daffodil/xsd/dafext.xsd | 1 +
.../daffodil/section06/namespaces/namespaces.tdml | 16 +++----
.../subfolder/multi_C_06_nons_valid.dfdl.xsd | 2 +-
.../property_scoping/PropertyScoping_01.tdml | 14 ++++++
.../property_scoping/PropertyScoping_06a.dfdl.xsd | 50 +++++++++------------
.../property_scoping/PropertyScoping_06b.dfdl.xsd | 52 +++++++++-------------
.../property_scoping/TestPropertyScoping.scala | 1 +
9 files changed, 103 insertions(+), 74 deletions(-)
diff --git
a/daffodil-core/src/main/scala/org/apache/daffodil/core/dsom/AnnotatedSchemaComponent.scala
b/daffodil-core/src/main/scala/org/apache/daffodil/core/dsom/AnnotatedSchemaComponent.scala
index b69324a48..4f3e1f60c 100644
---
a/daffodil-core/src/main/scala/org/apache/daffodil/core/dsom/AnnotatedSchemaComponent.scala
+++
b/daffodil-core/src/main/scala/org/apache/daffodil/core/dsom/AnnotatedSchemaComponent.scala
@@ -107,10 +107,24 @@ trait ResolvesScopedProperties extends FindPropertyMixin
{ self: Term =>
private def findDefaultProperty(pname: String): PropertyLookupResult = {
val result = findPropertyInSources(pname, defaultPropertySources)
val fixup = result match {
- case Found(value, loc, pname, _) =>
+ case Found(value, loc, pname, _) => {
+ // we found the property in default format annotation. Daffodil 4.1.0
correct the order
+ // we resolve default property sources to match the DFDL
specification. Fortunately,
+ // this does not affect most schemas, but for those that it does, it
can be very
+ // difficult to determine what the old value was. So we lookup
property sources in the
+ // old incorrect reversed order and see if we find the same value and
warn if not
+ val oldResult = findPropertyInSources(pname,
defaultPropertySourcesReversed)
+ val oldValue = oldResult.toOption.get
+ schemaDefinitionWarningWhen(
+ WarnID.ChangedDefaultPropertyResolution,
+ oldValue != value,
+ s"""Value of property $pname changed from "$oldValue" to "$value"
due to corrections in default property resolution."""
+ )
+
// found as a default property.
// supply constructor's last arg is boolean indicating it's a default
property
Found(value, loc, pname, true)
+ }
case NotFound(nd, d, pn) =>
Assert.invariant(d.isEmpty)
NotFound(
@@ -357,16 +371,33 @@ trait AnnotatedSchemaComponent
}
}.value
- final protected lazy val defaultPropertySources: Seq[ChainPropProvider] =
- LV(Symbol("defaultPropertySources")) {
+ /**
+ * The default property sources we should use for resolving property values
are calculated by
+ * combining the default chain of referred components with our own default
chain. We do this
+ * by prepending our own defaultFormatChain to the chain of referred
components. Prepends are
+ * efficient, but this creates the chain in the reverse order, since we want
to give
+ * precedence to the innermost referred components (see section 8.1.4 of the
DFDL
+ * specification). We could reverse it now, but the reversed list is
actually useful to keep
+ * around. For example, Daffodil 4.1.0 fixed a bug to correctly use the
non-reversed order,
+ * but some schemas relied on this buggy behavior. By storing the old
reversed order, we can
+ * use this to detect and warn if schemas are relying on the buggy behavior.
+ */
+ final protected lazy val defaultPropertySourcesReversed:
Seq[ChainPropProvider] =
+ LV(Symbol("defaultPropertySourcesReversed")) {
val refTo = refersToForPropertyCombining
- val chainFromReferredTo = refTo.toSeq.map { _.defaultPropertySources
}.distinct.flatten
+ val chainFromReferredTo =
+ refTo.toSeq.map { _.defaultPropertySourcesReversed }.distinct.flatten
val completeDefaultFormatChain =
defaultFormatChain +: chainFromReferredTo
val seq = completeDefaultFormatChain.distinct
seq
}.value
+ final protected lazy val defaultPropertySources: Seq[ChainPropProvider] =
+ LV(Symbol("defaultPropertySources")) {
+ defaultPropertySourcesReversed.reverse
+ }.value
+
final protected lazy val nonDefaultFormatChain: ChainPropProvider = {
val fa = formatAnnotation
val fc = fa.formatChain
diff --git
a/daffodil-core/src/test/resources/test/example_a02_targetnamespace_unqualified.dfdl.xsd
b/daffodil-core/src/test/resources/test/example_a02_targetnamespace_unqualified.dfdl.xsd
index 5ebc23108..040ad788d 100644
---
a/daffodil-core/src/test/resources/test/example_a02_targetnamespace_unqualified.dfdl.xsd
+++
b/daffodil-core/src/test/resources/test/example_a02_targetnamespace_unqualified.dfdl.xsd
@@ -43,7 +43,7 @@
</xs:annotation>
<xs:element name="inty" type="xs:int" dfdl:lengthKind="delimited"/>
- <xs:element name="intx" type="xs:int" nillable="true"
dfdl:nilKind="literalValue" dfdl:nilValue="^"/>
+ <xs:element name="intx" type="xs:int" dfdl:lengthKind="delimited"
nillable="true" dfdl:nilKind="literalValue" dfdl:nilValue="^"/>
</xs:schema>
diff --git
a/daffodil-propgen/src/main/resources/org/apache/daffodil/xsd/dafext.xsd
b/daffodil-propgen/src/main/resources/org/apache/daffodil/xsd/dafext.xsd
index befb240b6..f82ea9b06 100644
--- a/daffodil-propgen/src/main/resources/org/apache/daffodil/xsd/dafext.xsd
+++ b/daffodil-propgen/src/main/resources/org/apache/daffodil/xsd/dafext.xsd
@@ -713,6 +713,7 @@
<xs:documentation>Deprecated.</xs:documentation>
</xs:annotation>
</xs:enumeration>
+ <xs:enumeration value="changedDefaultPropertyResolution" />
<xs:enumeration value="deprecatedBuiltInFormats" />
<xs:enumeration value="deprecatedEncodingNameUSASCII7BitPacked" />
<xs:enumeration value="deprecatedExpressionResultCoercion" />
diff --git
a/daffodil-test/src/test/resources/org/apache/daffodil/section06/namespaces/namespaces.tdml
b/daffodil-test/src/test/resources/org/apache/daffodil/section06/namespaces/namespaces.tdml
index 0052c6adf..f8c8552d0 100644
---
a/daffodil-test/src/test/resources/org/apache/daffodil/section06/namespaces/namespaces.tdml
+++
b/daffodil-test/src/test/resources/org/apache/daffodil/section06/namespaces/namespaces.tdml
@@ -744,18 +744,18 @@
as a restriction base.
- In Schema B, an element is declared of the simpleType in C.
- In Schema A, a sequence of B elements is declared
- In this case, each Schema file in the chain, starting at F, imposes its
- own terminator value, named
- after the schema itself (Schema E's terminator is "E", B's is "B", etc.).
- Each time a new schema is used
- in the chain, the separator is overwritten. Schema B is the last schema
- in the chain to set this value,
- so the terminator for our data set will be "B".
+ In this case, each Schema file in the chain, starting at F, sets a default
+ terminator value, named after the schema itself (Schema E's terminator is
+ "E", B's is "B", etc.).
+ Each time a new schema is used in the chain, the separator is overwritten.
+ Default properties are resolved from the defaults of the innermost schema
+ components, which in this case is schema E, so the terminator for our data
+ set will be "E"
-->
<tdml:parserTestCase name="long_chain_06" root="nestSequence6"
model="multi_A_03.dfdl.xsd" description="import a schema - DFDL-6-007R">
- <tdml:document><![CDATA[5632B]]></tdml:document>
+ <tdml:document><![CDATA[5632E]]></tdml:document>
<tdml:infoset>
<tdml:dfdlInfoset>
<nestSequence6>
diff --git
a/daffodil-test/src/test/resources/org/apache/daffodil/section06/namespaces/subfolder/multi_C_06_nons_valid.dfdl.xsd
b/daffodil-test/src/test/resources/org/apache/daffodil/section06/namespaces/subfolder/multi_C_06_nons_valid.dfdl.xsd
index c6925cf5b..e19ab0bb9 100644
---
a/daffodil-test/src/test/resources/org/apache/daffodil/section06/namespaces/subfolder/multi_C_06_nons_valid.dfdl.xsd
+++
b/daffodil-test/src/test/resources/org/apache/daffodil/section06/namespaces/subfolder/multi_C_06_nons_valid.dfdl.xsd
@@ -33,6 +33,6 @@
</xs:appinfo>
</xs:annotation>
- <xs:element name="cElem" type="xs:string" dfdl:initiator="c:"
dfdl:outputValueCalc="{ 'done' }" />
+ <xs:element name="cElem" type="xs:string" dfdl:lengthKind="delimited"
dfdl:initiator="c:" dfdl:outputValueCalc="{ 'done' }" />
</xs:schema>
diff --git
a/daffodil-test/src/test/resources/org/apache/daffodil/section08/property_scoping/PropertyScoping_01.tdml
b/daffodil-test/src/test/resources/org/apache/daffodil/section08/property_scoping/PropertyScoping_01.tdml
index 9d5bf5851..9f9811ccf 100644
---
a/daffodil-test/src/test/resources/org/apache/daffodil/section08/property_scoping/PropertyScoping_01.tdml
+++
b/daffodil-test/src/test/resources/org/apache/daffodil/section08/property_scoping/PropertyScoping_01.tdml
@@ -686,4 +686,18 @@
</tdml:errors>
</tdml:parserTestCase>
+ <tdml:parserTestCase name="property_scoping_12"
+ model="PropertyScoping_06a.dfdl.xsd"
+ description="Section 8 Default properties inherited from referenced
components">
+
+ <tdml:document>
+ <tdml:documentPart type="byte">00000001</tdml:documentPart>
+ </tdml:document>
+ <tdml:infoset>
+ <tdml:dfdlInfoset>
+ <tns:int01>1</tns:int01>
+ </tdml:dfdlInfoset>
+ </tdml:infoset>
+ </tdml:parserTestCase>
+
</tdml:testSuite>
diff --git
a/daffodil-core/src/test/resources/test/example_a02_targetnamespace_unqualified.dfdl.xsd
b/daffodil-test/src/test/resources/org/apache/daffodil/section08/property_scoping/PropertyScoping_06a.dfdl.xsd
similarity index 51%
copy from
daffodil-core/src/test/resources/test/example_a02_targetnamespace_unqualified.dfdl.xsd
copy to
daffodil-test/src/test/resources/org/apache/daffodil/section08/property_scoping/PropertyScoping_06a.dfdl.xsd
index 5ebc23108..8cb900dd7 100644
---
a/daffodil-core/src/test/resources/test/example_a02_targetnamespace_unqualified.dfdl.xsd
+++
b/daffodil-test/src/test/resources/org/apache/daffodil/section08/property_scoping/PropertyScoping_06a.dfdl.xsd
@@ -16,34 +16,24 @@
limitations under the License.
-->
-<xs:schema targetNamespace="http://a02.com";
- xmlns:xs="http://www.w3.org/2001/XMLSchema";
+<schema xmlns="http://www.w3.org/2001/XMLSchema";
xmlns:dfdl="http://www.ogf.org/dfdl/dfdl-1.0/";
- xmlns:a2="http://a02.com";>
-
- <xs:include
schemaLocation="/org/apache/daffodil/xsd/DFDLGeneralFormat.dfdl.xsd"/>
-
- <xs:annotation>
- <xs:appinfo source="http://www.ogf.org/dfdl/";>
- <dfdl:format ref="a2:GeneralFormat"
- separator=""
- initiator=""
- separatorPosition="infix"
- ignoreCase="no"
- separatorSuppressionPolicy="anyEmpty"
- terminator=""
- occursCountKind="parsed"
- initiatedContent="no"
- representation="text"
- textNumberRep="standard"
- encoding="ASCII"
- textTrimKind="none"
- leadingSkip='0'/>
- </xs:appinfo>
- </xs:annotation>
-
- <xs:element name="inty" type="xs:int" dfdl:lengthKind="delimited"/>
- <xs:element name="intx" type="xs:int" nillable="true"
dfdl:nilKind="literalValue" dfdl:nilValue="^"/>
-
-
-</xs:schema>
+ xmlns:xs="http://www.w3.org/2001/XMLSchema";
+ xmlns:ex="http://example.com";
+ targetNamespace="http://example.com";>
+
+ <include
schemaLocation="/org/apache/daffodil/xsd/DFDLGeneralFormat.dfdl.xsd" />
+
+ <annotation>
+ <appinfo source="http://www.ogf.org/dfdl/";>
+ <dfdl:format ref="ex:GeneralFormat"
+ representation="binary"
+ byteOrder="littleEndian" />
+ </appinfo>
+ </annotation>
+
+ <include schemaLocation="PropertyScoping_06b.dfdl.xsd" />
+
+ <element name="int01" type="ex:int" />
+
+</schema>
diff --git
a/daffodil-core/src/test/resources/test/example_a02_targetnamespace_unqualified.dfdl.xsd
b/daffodil-test/src/test/resources/org/apache/daffodil/section08/property_scoping/PropertyScoping_06b.dfdl.xsd
similarity index 51%
copy from
daffodil-core/src/test/resources/test/example_a02_targetnamespace_unqualified.dfdl.xsd
copy to
daffodil-test/src/test/resources/org/apache/daffodil/section08/property_scoping/PropertyScoping_06b.dfdl.xsd
index 5ebc23108..e798a39a4 100644
---
a/daffodil-core/src/test/resources/test/example_a02_targetnamespace_unqualified.dfdl.xsd
+++
b/daffodil-test/src/test/resources/org/apache/daffodil/section08/property_scoping/PropertyScoping_06b.dfdl.xsd
@@ -16,34 +16,26 @@
limitations under the License.
-->
-<xs:schema targetNamespace="http://a02.com";
- xmlns:xs="http://www.w3.org/2001/XMLSchema";
+<schema xmlns="http://www.w3.org/2001/XMLSchema";
xmlns:dfdl="http://www.ogf.org/dfdl/dfdl-1.0/";
- xmlns:a2="http://a02.com";>
-
- <xs:include
schemaLocation="/org/apache/daffodil/xsd/DFDLGeneralFormat.dfdl.xsd"/>
-
- <xs:annotation>
- <xs:appinfo source="http://www.ogf.org/dfdl/";>
- <dfdl:format ref="a2:GeneralFormat"
- separator=""
- initiator=""
- separatorPosition="infix"
- ignoreCase="no"
- separatorSuppressionPolicy="anyEmpty"
- terminator=""
- occursCountKind="parsed"
- initiatedContent="no"
- representation="text"
- textNumberRep="standard"
- encoding="ASCII"
- textTrimKind="none"
- leadingSkip='0'/>
- </xs:appinfo>
- </xs:annotation>
-
- <xs:element name="inty" type="xs:int" dfdl:lengthKind="delimited"/>
- <xs:element name="intx" type="xs:int" nillable="true"
dfdl:nilKind="literalValue" dfdl:nilValue="^"/>
-
-
-</xs:schema>
+ xmlns:xs="http://www.w3.org/2001/XMLSchema";
+ xmlns:ex="http://example.com";
+ targetNamespace="http://example.com";>
+
+ <include
schemaLocation="/org/apache/daffodil/xsd/DFDLGeneralFormat.dfdl.xsd" />
+
+ <annotation>
+ <appinfo source="http://www.ogf.org/dfdl/";>
+ <dfdl:format ref="ex:GeneralFormat"
+ representation="binary"
+ byteOrder="bigEndian" />
+ </appinfo>
+ </annotation>
+
+ <include schemaLocation="PropertyScoping_06b.dfdl.xsd" />
+
+ <simpleType name="int">
+ <restriction base="xs:int" />
+ </simpleType>
+
+</schema>
diff --git
a/daffodil-test/src/test/scala/org/apache/daffodil/section08/property_scoping/TestPropertyScoping.scala
b/daffodil-test/src/test/scala/org/apache/daffodil/section08/property_scoping/TestPropertyScoping.scala
index 7f39399f8..98e5c62ee 100644
---
a/daffodil-test/src/test/scala/org/apache/daffodil/section08/property_scoping/TestPropertyScoping.scala
+++
b/daffodil-test/src/test/scala/org/apache/daffodil/section08/property_scoping/TestPropertyScoping.scala
@@ -79,6 +79,7 @@ class TestPropertyScoping01 extends TdmlTests {
@Test def unparse_property_scoping_11 = test
@Test def unparse_property_scoping_12 = test
@Test def NearestEnclosingSequenceElementRef = test
+ @Test def property_scoping_12 = test
@Test def refElementFormFail = test
}
