I envisioned that the Mark's would still be owned by the parsers that created them. Perhaps I should say that rather than a discriminator "discarding" a mark, it should really just "clear" the mark. This clear does things like decrease the refcounter so the InfosetWalker can continue walking, clears marks associate with the data input stream so that buckets can be freed, etc. But this does not return the Mark to the MarkPool. When we unwind back to the parser that created the Mark, the parse can check if the Mark was cleared (thus, was discriminator) and discard it to return it back to the MarkPool.
I hadn't thought too in detail about Unparse aspect, but what you mention was similar to how I thought these it would be handled. I'll figure out some details and update the proposal for unparse. - Steve On 6/2/20 3:26 PM, Beckerle, Mike wrote: > I'm a little unclear on who owns the mark objects. You have the discriminator > stack of marks, and also infoset nodes with referenced counters to marks. The > discriminator stack of marks is used to determine when to call the walk() as > well as to keep track of discrimination, and the infoset nodes with reference > counters to marks are to enable the walker to know how much it can consume. > > You did not discuss unparsing yet of course. It may be illustrative to design > all of the parse and unparse capability together. > > There is no backtracking, but we still have the issue of wanting to evaluate > suspensions asap, not saving all of them to the end of unparsing (what it > does now). The concept here I think is that the suspensions, when created, > should be hung on the infoset nodes whose values they depend on, so that when > those nodes are attached or the parent element is closed, that is when the > expressions are retried to see if they succeed or suspend again. > > The expressions can reference into the future when unparsing, that is, > element N may depend on element N+1. That causes suspensions typically. But I > think the trim-away logic is pretty much the same as parser. When you are > done unparsing an element, then you can release it unless it is referenced by > expressions. If the referenced element is an array element at index N, then > you can release prior array element N-1 generally, based on the same control > property you suggest for parsing. > > > ________________________________ > From: Steve Lawrence <[email protected]> > Sent: Tuesday, June 2, 2020 9:18 AM > To: [email protected] <[email protected]> > Subject: Proposal: Infoset Streaming and Memory Reduction > > I've added a new proposal to allow Daffodil to use a more stream-like > behavior when creating infoset events, and to remove infoset nodes when > possible to reduce memory usage. The proposal is available here: > > https://cwiki.apache.org/confluence/display/DAFFODIL/Proposal%3A+Infoset+Streaming+and+Memory+Reduction > > I welcome any thoughts on ways to improve this. >
