Well, to sanity check ideas, here's my thoughts on how unparsers and variable instances and suspensions/expressions ought to interact.
There may be naive assumptions in here. If so let's find them. So unparsers call each other in a recursive walk, and variable instances go in/out of scope as the unparsers are walked. That's on the variable-map structure stacks in UStateMain. This creates the variable instances, and those specific variable instances should be the ones that are frozen into a suspension. I.e., the suspension shouldn't contain the variable map with things going into/out of scope, but just a single association of variable name to variable instance object. So I think creating a suspension should create a snapshot of the top-of-stack for each variable as part of that UStateForSuspension. It's important that these variable instances are; however, shared with other expressions/unparser actions that are for that same scope. So we can't deep copy the variable instances themselves or we'll disconnect them from their producers and consumers. We could in principle copy the variable stacks with all their pointers to variable instances, but we'll only ever address the top-of-stack variable instances from a suspension. These variable instances are then sort of floating in air, connected to expressions that produce/consume them by the suspension/expression system, but they're not being stack-maintained any more. They're heap objects at that point, disconnected from the stacks that controlled when they were scope-visible. They have to eventually be reclaimed by the garbage collector. The variable going out of scope should only affect the variable map in the UStateMain object, not the suspensions, and it should remove a variable from scope, but not otherwise frob the variable-instance, which may be referenced by suspensions. Now, all that said, I bet there's a flaw in there. ________________________________ From: Adams, Joshua <jad...@owlcyberdefense.com> Sent: Tuesday, February 2, 2021 11:12 AM To: dev@daffodil.apache.org <dev@daffodil.apache.org> Subject: Suspensions and NewVariableInstance I've been running into a lot of headaches trying to get newVariableInstance to correctly handle suspensions. Currently when a newVariableInstance statement is found, a NewVariableInstanceStart and End unparsers are created. NewVariableInstanceStart will immediately create the newVariableInstance with no value and, if applicable, will create a SuspendableExpression that will calculate the default value. This works as expected but as the NVI's go out of scope we start running into some issues. The NewVariableIsntanceEnd unparser simply removes the variable instance that was created in NVIStart. It is not performing or checking for any sort of suspension, so NVIEnd is called while the NVIStart suspension is still active. Since the UStateForSuspension object uses the same VariableMap as the main UState object, this results in the variable's value not being correct after it goes out of scope. I'm not sure what a good solution for this would be. I've attempted adding a SuspendableOperation to the NVIEnd unparser to wait until the variable has a value before removing, but this results in a SuspensionDeadlock. I've also attempted doing a deep copy of all the variables for each UStateForSuspension object, but this too results in a SuspensionDeadlock, not to mention that any changes made in one suspension wouldn't be visible to other suspensions. One other thought I had, which I'm not even sure would even address the suspension issue, is to have whatever sequence is containing the NVI statement handle calling the NVIEnd unparser by simply adding it to the end of its sequence child unparsers. Any thoughts on how best to handle this sticky situation of dealing with newVariableInstance and unparsing suspensions?