Robert - I've implemented a version of the traversal using a stack of iterators. Conceivably less memory consumption depending on the underlying implementation, but it shouldn't be more then what I had.
I'll leave the pull request open for two more days, and then merge and do the release. Unless someone else has input. - Jason On Tue, Jun 5, 2018, at 9:07 AM, Jason E Bailey wrote: > Realized I wasn't clear in how the method declaration would be problematic. > > example: > > > > - stream(Predicate<Resource>, Set<TraversalOption>... ) > > take the invocation > > - stream(predicate, CHILDREN_ONLY); > > since the first predicate is a structural predicate in the manner that > it determines what child resources to traverse on, it's now not used in > the above invocation. I don't like methods where one parameter is > ignored based on the value of the second parameter. I find it can lead > to unpredictable outcomes because people aren't cognizant of it. > > Saying that, there may be Traversal Options that aren't necessarily > conflicting with the predicate and it may make sense to incorporate it. > > My methodology tends to be release early and often and garner feedback > so I'm not adverse to adding that, I just want to get some wider usage > of the feature so that I'm making useful choices in it's evolution. > > > > > I think that the 'hasNext' method is not lazy, eagerly loading the > > > direct children of a resource when there are no more resources to load. > > > > > > If the resource has a large number of children we can end up taking > > > lots of memory, so I think we need to be more lazy with it, since the > > > streams framework can take advantage of that (IIRC). Is it possible to > > > ajust the implementation this way? > > > > So one thing I discovered is problematic is probably related to the JCR > and that is the multi threading in the stream. When I ran tests where I > made the stream capable of being splitted I had worse results then > single threading it. I didn't dive to much into it but my assumption at > this point is that when you are requesting multiple resources using the > same session on multiple threads, the underlying JCR doesn't like it. > > I do have an idea to make it potentially more lazy which I'll do on my > lunch break. Where I replace the list of resources with a list of > iterators which references the depth of the tree I'm iterating through. > I can't really guarantee lazy behavior because I would be relying on the > underlying resource provider to provide a Lazy iterator. Which can't be > guaranteed. >
