Thanks so much for your input Alex! It was a very helpful confirmation of the key conclusions arrived at in this thread, and I appreciate the additional elaborations you gave, especially the insight you passed on about the stateful transducers using `ArrayList`. I'm glad that I wasn't the only one wondering about the apparent lack of parity between its unsynchronized mutability and the volatile boxes used for e.g. `map-indexed` and others.
As an aside about the stateful `take` transducer, Tesser uses the equivalent of one but skirts the issue by not guaranteeing that the first n items of the collection will be returned, but rather, n items of the collection in no particular order and starting at no particular index. This is achievable without Tesser by simply replacing the `volatile` in the `core/take` transducer with an `atom` and using it with `fold`. But yes, `take`'s contract is broken with this and so still follows the rule of thumb you established that `fold` can't use stateful transducers (at least, not without weird things like reordering of the indices in `map-indexed` and so on). That's interesting that `fold` can use transducers directly! I haven't tried that yet — I've just been wrapping them in an `r/folder`. On Sunday, April 9, 2017 at 10:22:13 PM UTC-4, Alex Miller wrote: > > Hey all, just catching up on this thread after the weekend. Rich and I > discussed the thread safety aspects of transducers last fall and the > intention is that transducers are expected to only be used in a single > thread at a time, but that thread can change throughout the life of the > transducing process (for example when a go block is passed over threads in > a pool in core.async). While transducing processes may provide locking to > cover the visibility of state updates in a stateful transducer, transducers > should still use stateful constructs that ensure visibility (by using > volatile, atoms, etc). > > The major transducing processes provided in core are transduce, into, > sequence, eduction, and core.async. All but core.async are single-threaded. > core.async channel transducers may occur on many threads due to interaction > with the go processing threads, but never happen on more than one thread at > a time. These operations are covered by the channel lock which should > guarantee visibility. Transducers used within a go block (via something > like transduce or into) occur eagerly and don't incur any switch in threads > so just fall back to the same old expectations of single-threaded use and > visibility. > > Note that there are a couple of stateful transducers that use ArrayList > (partition-by and partition-all). From my last conversation with Rich, he > said those should really be changed to protect themselves better with > volatile or something else. I thought I wrote up a ticket for this but > looks like maybe I didn't, so I will take care of that. > > Reducer fold is interesting in that each "bucket" is reduced via its > reduce function, which can actually use a transducer (since that produces a > reduce function), however, it can't be a stateful transducer (something > like take, etc). > > Hope that helps with respect to intent. > > -- You received this message because you are subscribed to the Google Groups "Clojure" group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups "Clojure" group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
