Pipeline must have a way of controlling on which thread the execution happens, otherwise it would parallize very little.
It does so by creating a new channel for each put with a buffer of 1 and an xf. It makes a blocking >!! and only after it unblocks it puts that new channel to the consumer that takes the value from there. So at least in this limited case where there is space in the buffer and there is no pending take - Is it safe to assume that the xf is applied immediately? On Thursday, December 17, 2015 at 4:28:27 PM UTC+1, Herwig Hochleitner wrote: > > 2015-12-16 16:22 GMT+01:00 Leon Grapenthin <grapent...@gmail.com > <javascript:>>: > >> The blocking put is made on a separate thread (channel named res), then >> later a blocking take from that same channel is made in the second go-loop. >> > >> Or are you saying "if it takes too long, parallelize via pipeline"? In my >> case I can't because the transducer is stateful. I can run multiple >> pipelines with multiple transducer instances if I split up the inputs >> manually and later merge the outputs. But since the transducer does all of >> the relevant work, I need to determine where it runs. >> > >> The only alternative I see is to re-implement the transducer process >> within a thread/loop body and run multiple of those. >> > > If you don't care about message ordering, you don't need pipline. I > suppose you could solve the "where does it run" problem by using a > ThreadLocal, but that feels pretty dirty. Running parallel loops taking > from the same source channel and putting into the same target channel, is > pretty easy to implement by hand. I have some code doing batch-indexing, > where I used this paradigm, if you're interested. > > I am not familiar with the JVM locking mechanism of core.async so what >> should I be worried about when the "transducer is executed inside the >> channel lock"? Are other channels going to have to wait if it is too slow? >> > > The channel lock only affects put! and take! on a channel, as well as > transducers attached to that channel, so no need to worry about this for > your regular go-blocks. > > -- You received this message because you are subscribed to the Google Groups "Clojure" group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups "Clojure" group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
