Yes, commenting out the outputValueCalc is another way to avoid this
circular deadlock problem. The drawback to that approach is if you
modify or remove a DataExtensionSegment from the infoset and then
unparse it, then the length will be incorrect and the unparsed file will
likely be invalid.
If you don't plan on having anything modify the infoset, then it
shouldn't be an issue and commenting it out is a reasonable alternative
workaround, just be aware of the limitation.
But we don't know how the public nitf schema will be used--maybe some
users will wan to modify the infoset and unparse--so we probably should
update it to use something like what I suggested below.
On 2024-02-15 03:49 PM, Lara Blatchford wrote:
Steve - thanks for this suggestion, it did the trick. I see that the
outputValueCalc in question is commented out in the latest version of the
schema, is this issue avoided by other schema changes in that version? I don't
have easy access to get the latest version and try it, but I wanted to make
sure an issue didn't need to be entered against the latest.
Thanks again for your help,
Lara
-----Original Message-----
From: Steve Lawrence <[email protected]>
Sent: Wednesday, February 14, 2024 3:21 PM
To: [email protected]
Subject: Re: circular deadlock on NITF unparse
This looks like a schema bug, and there are circular deadlocks. One common way
to fix this issue is to break up an outputCalc expression so that instead of
calculating the length of an element, you calculate the length of its children
and add them together. This makes for a more complicated expression, but often
times avoids circular references. In this particular case the dfdl:outputCalc
property on the DataExtensionSegmentLengths/DataLength element can be changed
from this:
dfdl:outputValueCalc="{
dfdl:valueLength(../../../DataExtensionSegment[dfdl:occursIndex()]/Data,
'bytes')
}
To this:
dfdl:outputValueCalc="{
if
(fn:exists(../../../DataExtensionSegment[dfdl:occursIndex()]/Data/nitf:TaggedRecordExtensions))
then
dfdl:valueLength(../../../DataExtensionSegment[dfdl:occursIndex()]/Data/nitf:TaggedRecordExtensions,
'bytes')
else
dfdl:valueLength(../../../DataExtensionSegment[dfdl:occursIndex()]/Data/DESUserDefinedData,
'bytes')
}
So instead of evaluating the length of just the Data element, it is changed to
get the length of the TaggedRecordExtensions element or the DESUserDefinedData
element depending on which one exists (since they are in a choice).
You can see we do similar things to avoid deadlocks when calculating the
lengths of various HeaderLength elements in the schema.
On 2024-02-14 02:33 PM, Lara Blatchford wrote:
I’m using an older version of the NITF schema from github (attached) and
can parse the attached NITF file successfully. On unparse I get the
circular deadlock error below. I’m using Daffodil 3.5.0.
How can I determine whether this is an issue with the data itself or a
bug in Daffodil as I see other open issues regarding circular deadlocks?
Runtime Schema Definition Error: Expressions/Unparsers are circularly
deadlocked (mutually defined): - target length for Data expr
<EvaluatableExpression eName='DataExtensionSegment/Data' expr='{
../../Header/DataExtensionSegmentLengths[dfdl:occursIndex()]/DataLength
}' /> - target length for DESUserDefinedData expr
<EvaluatableExpression
eName='DataExtensionSegment/Data/DESUserDefinedData' expr='{
../../../Header/DataExtensionSegmentLengths[dfdl:occursIndex()]/DataLength }'
/> - SuspendableExpression(DataLength, expr={
dfdl:valueLength(../../../DataExtensionSegment[dfdl:occursIndex()]/Data, 'bytes')
}) Schema context: Data Location line 781 column 16 in nitf.dfdl.xsd Data location
was preceding byte 1074
Thank you!
Lara Blatchford
Principal Engineer
240-341-1421
