On Tue, May 26, 2015 at 11:29 PM Alex Miller <a...@puredanger.com> wrote:
> The point I was getting at is really whether you should consider this to > be broken with the old behavior too. > Such APIs are tricky to use correctly from Clojure via seqs, but it is possible to do so with the "normal" automatic clojure.lang.RT seq adapters in Clojure <=1.6. My point is that existing Clojure code can and does depend on non-chunked realization of iterator seqs, and that for such code this is a breaking change. > Can you point to code for "the original behavior allowed room to transform > the mutated object into an object which *could* be safely cached in a > 'downstream' seq"? By what means does this transformation occur? It sounds > to me like you are starting with an Iterator, creating a seq, then walking > the seq exactly once, one element at a time, and producing a new > transformed seq or other output. > Exactly -- the unfortunately Java <=1.6-only snippet I posted earlier is just such an example of this. > If you did reuse that IteratorSeq, all of the elements of the sequence > would point to the same object which would be in the "last" state like the > java 1.6 example you gave. Thus, the "caching" capability of the seq can't > possibly be something you're using. And if that's true, then why are you > paying the allocation and synchronization costs of making the seq at all? > Why not just use the iterator directly, thus skipping all the extra > allocation that these object-reusing high-performance iterators are working > so hard to avoid in the first place? In 1.7, transducers would give you > exactly the capability to walk the source iterator, apply a transducer > version of your transformation, and output to a collection (via into), a > value (via transduce), or a lazy sequence (via sequence). I think you would > find this faster as well due to reduced allocation (possibly greatly > reduced depending on the transformation). > I've personally used reducers wherever possible since they were introduced, and for the Hadoop case Parkour's primary recommended API is in terms of reducers [1]. For new code, the transducer-based facilities in Clojure 1.7 will certainly provide more options for functional-safe handling of the iterators at question. But to repeat my main point, those don't help with existing code which depends on the one-at-a-time realization semantics of Iterators being reflected in one-at-a-time realization in iterator-backed seqs. [1] https://github.com/damballa/parkour/blob/master/doc/reducers-vs-seqs.md Thanks, -Marshall -- You received this message because you are subscribed to the Google Groups "Clojure" group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups "Clojure" group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
