Thanks Mike! Continuations seems like a better alternative, at least from a support point of view. Though, it's a little concerning that no one is really stepping up to port it to 2.13, but I don't think we're in any rush to get to 2.13. And I personally find the reset/shift concept a bit harder to wrap my head around than the co-routine resume/yield, but ultimately it's not too bad.
To see how it would work with our DataProcessor/InfosetInputter, I forked and updated your gist to include things like InfosetInputters, DataProcessor, ContentHandler, etc. and added a bunch of println's and comments to make sure things were behaving the way I thought they should. https://gist.github.com/stevedlawrence/5e16081f4690448de6131af02daacea9 I think it came out pretty straightforward. I also modified this so that there isn't as much back and forth between hasNext/next like I have in the current proposal. The only time we go back the to ContentHandler/producer is when next() is called, and we only go back to the InfosetInputter/consumer when a complete event is found, including hasNext. I do have one concern with this approach. Scala required the @suspendable annotation on the unparse() method of the DataProcessor and on the next() method of the InfosetInputter for both the abstract class and concrete SAX implementation. I'm not sure if that annotation causes any problems when not used inside a reset block (i.e. old API style), or if that annotation will end up cascading throughout the codebase. Seems like there's a possibility for that to happen. Maybe I just need to reorganize the code a bit, but it's not clear to me how. On 4/22/20 7:18 PM, Beckerle, Mike wrote: > scala continuations is supported on 2.11 and 2.12, but work in progress for > 2.13. The main web page for it says it is looking for a lead developer and > without that typesafe/lightbeam is doing bare minimum maintenance. > > A producer/consumer idiom like what we need is easily expressed using this > shift/reset thing. > > Here's a gist that does a control turnaround from a handler to a > pull-oriented while loop. Took me a bit of research to get the build.sbt > right so this would "just work" > > https://gist.github.com/mbeckerle/4c1d8f8c365958ef7d01bf770fa6317c > > > ________________________________ > From: Beckerle, Mike <mbecke...@tresys.com> > Sent: Wednesday, April 22, 2020 5:01 PM > To: dev@daffodil.apache.org <dev@daffodil.apache.org> > Subject: Re: Daffodil SAX API Proposal > > Another possibility is scala-asynch which I think can do what we want. > ________________________________ > From: Beckerle, Mike <mbecke...@tresys.com> > Sent: Wednesday, April 22, 2020 4:34 PM > To: dev@daffodil.apache.org <dev@daffodil.apache.org> > Subject: Re: Daffodil SAX API Proposal > > The alternative is probably scala.util.continuations aka "shift and reset". > > It's much harder to understand and use, but at least its in the standard > library so is supported. (I think.) > > ________________________________ > From: Steve Lawrence <slawre...@apache.org> > Sent: Wednesday, April 22, 2020 3:40 PM > To: dev@daffodil.apache.org <dev@daffodil.apache.org> > Subject: Re: Daffodil SAX API Proposal > > I responded. > > I checked the license to make sure it's compatible (BSD-3), but I didn't > actually check what versions of Scala it works with. > > Looks like it is only published for 2.11, and the repo hasn't been > updated for at least 3 years. There is a 2.12.x branch in their repo, > but it too hasn't been updated in a long time. We might have to see how > much effort it would take to update that library, or perhaps find > another library. > > > On 4/22/20 3:28 PM, Beckerle, Mike wrote: >> I reviewed this and added a comment about the only significant issue, which >> I think just boils down to trying to keep the coroutining back and forth as >> simple as possible. >> >> Another thought: Is the scala coroutines library supported in 2.11 and 2.12 >> (and 2.13 for being future-safe?) >> >> >> ________________________________ >> From: Steve Lawrence <slawre...@apache.org> >> Sent: Wednesday, April 22, 2020 1:06 PM >> To: dev@daffodil.apache.org <dev@daffodil.apache.org> >> Subject: Daffodil SAX API Proposal >> >> I've added a proposal to add a SAX API support to Daffodil. >> >> https://cwiki.apache.org/confluence/display/DAFFODIL/Proposal%3A+SAX+API >> >> Many libraries and applications already support SAX, so this should >> provide a means for more seamless integration into different toolsuites, >> opening up the places where Daffodil could be easily integrated. >> >> SAX is also generally viewed as having a lower memory overhead, though >> this does not attempt to solve the memory issues related to Daffodil and >> the internal infoset representation. This essentially just adds a SAX >> compatible API around our existing API. Other changes are needed to >> reduce our memory overhead and truly support a streaming model. >> >> - Steve >> >> > >
