Re: core.async buffers alternative backend

[Alex Miller]

I don't actually remember now, but it's possible that when core.async was 
created we were still trying to accommodate an older version of Java before 
some of those existed. That's not an issue now as we only need to support 
Java 1.8+. So, I don't know of any reason these wouldn't be an option. 
Would be happy to see a core.async issue and/or patch (may need to sign the 
CA and become the contributor first).

On Thursday, September 10, 2020 at 8:14:53 AM UTC-5 Jim foo.bar wrote:

> Hi folks, 
>
> `LinkedList` is used as the underlying data-structure for all core.async 
> buffers. However, looking closer reveals that what is really needed is a 
> `Deque` (for its .addFirst/.removeLast methods). So naturally then it 
> begs the question - why not `ArrayDeque` [1]? It should offer superior 
> insertion/removal/traversing performance, and in the context of 
> core.async it will never be resized (if initialised with `n`). 
>
> And since we're on the subject, why not even go crazy with a 
> (thread-safe) `ConcurrentLinkedDeque` [2] ? Leaving the counting 
> complication aside for a moment (something which I wouldn't mind 
> feedback on), having a thread-safe buffer opens up the door to safe 
> buffer snapshots. 
>
> Anything wrong with either of those approaches? Many thanks in advance... 
>
> Kind regards, 
>
> Dimitris 
>
> [1]: 
>
> https://github.com/jimpil/asynctopia/blob/master/src/asynctopia/buffers/array.clj
>  
>
> [2]: 
>
> https://github.com/jimpil/asynctopia/blob/master/src/asynctopia/buffers/thread_safe.clj
>  
>
>
>
>

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/clojure/7ab00670-4768-4738-8f18-77406bd1fdcfn%40googlegroups.com.

Re: core.async: Unbound channels

[Ernesto Garcia]

Thanks Alex!

Correct, the channel implementation takes care that "transduced" channels 
always pass elements through the transducer and the buffer. Also, a 
FixedBuffer allows running out of limit for those cases, see this example 
with a FixedBuffer of size 1 making space for 4 elements:

(def c (chan 1 (mapcat seq)))
=> #'psdk.hack-config/c

(put! c "hola")
=> true

(-> c .buf .buf)
=> (\a \l \o \h)

However, the blocking semantics of a channel change if a buffer is 
enforced. You can't have a "transduced" channel for which all puts will 
block for a take. Not sure if this is too limiting for any practical case.

For the case of expanding transducers, I am not sure if there would be 
sensible semantics for channel operations without a buffer.

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/clojure/a652870b-59c2-40f9-b4e0-7387e87de204%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: core.async: Unbound channels

[Alex Miller]

Expanding transducers (like mapcat) can produce multiple output values per 
input value, and those have to have someplace to go.

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/clojure/1be8a455-f7c9-4e79-8906-d49ab6a09aef%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: core.async: Unbound channels

[Ernesto Garcia]

I see. Bufferless channels are meant to be used within the core.async 
threading architecture, where there will be a limited number of blocked 
puts and takes. At the boundaries, channels with dropping or sliding 
windows can be used for limiting work.

So, my original question actually turns into: Why do channels with a 
transducer need a buffer? Is it just a limitation of the implementation, or 
does it have a conceptual reason behind it?

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/clojure/a4201a4b-7d84-4457-a857-ed6e252f6b0e%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: core.async: Unbound channels

[Matching Socks]

"Effective" is in the eye of the beholder.  The 1024 limit helps surface 
bugs wherein more than a thousand threads are blocked for lack of a certain 
channel's buffer space.  But the 1024 limit does not pertain if 1 thread 
would like to do thousands of puts for which there is no buffer space.  In 
the latter case, that thread is in a loop that does 1 put at a time, and 
therefore only 1 put, at any given moment, counts against the 1024 limit.

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/clojure/844b72c4-71a3-43a9-b3bc-41c56922f3b2%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: core.async: Unbound channels

[Ernesto Garcia]

On Thursday, July 4, 2019 at 4:24:33 PM UTC+2, Matching Socks wrote:
>
> Ernesto, you may be interested in the informative response to this 
> enhancement request, https://clojure.atlassian.net/browse/ASYNC-23 
> ,
>  
> "Support channel buffers of unlimited size".
>

An example: Here a function that makes a request, and returns a channel 
with responses with the same referenceId as the request. The function taps 
a mult-channel, but I don't now what buffer to use.I would expect that the 
original mult-channel already implements some buffering limits.

(defn request [client events-mult-channel req]
  (let [ref-id (.getReference req)
result-channel (chan 16 (filter #(= ref-id (.getReference %]
(tap events-mult-channel result-channel)
(.send client req)
result-channel))

 

> Anyway, if you do not want to think very hard about buffer size, you can 
> specify a size of 1.  It does not limit the number of items the producer 
> may put on the channel, but it affects how soon (and how often) the 
> producer's puts must wait for a consumer.
>

I see, puts start to block when the buffer is full or when no buffer. Isn't 
the 1024-limit on blocked puts an effective limit, like a buffer size?

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/clojure/205bb066-7014-41ec-b552-153b1331143c%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: core.async: Unbound channels

[Matching Socks]

Ernesto, you may be interested in the informative response to this 
enhancement request, https://clojure.atlassian.net/browse/ASYNC-23, 
"Support channel buffers of unlimited size".  Anyway, if you do not want to 
think very hard about buffer size, you can specify a size of 1.  It does 
not limit the number of items the producer may put on the channel, but it 
affects how soon (and how often) the producer's puts must wait for a 
consumer.

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/clojure/f79addd3-794d-4a6a-ba45-550a2b75e389%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: core.async: Unbound channels

[Ernesto Garcia]

Thanks for your response, it is important to know. (Sorry for my lexical 
typo: *unbound**ed*. I didn't realize it derives from the verb *bound*, not 
*bind*!)

My question on channel boundaries still holds though. Why the enforcement 
of boundaries *always*?

On Wednesday, July 3, 2019 at 5:16:31 PM UTC+2, Ghadi Shayban wrote:
>
> (chan) is not a channel with an unbounded buffer. It is a channel with 
> *no* buffer and needs to rendezvous putters and takers 1-to-1.  
> (Additionally it will throw an exception if more than 1024 takers or 
> putters are enqueued waiting)
>
> On Wednesday, July 3, 2019 at 7:14:46 AM UTC-4, Ernesto Garcia wrote:
>>
>> You can create a unbound channel with (chan), but not if you use a 
>> transducer; (chan nil (filter odd?)) will raise an error that no buffer 
>> is provided. Why is this the case?
>>
>> Why the enforcement of all channels to be bound? In a program, there will 
>> be channels that propagate to other channels, so only channels at the 
>> boundaries would require to be bound?
>>
>> Channel limits are also much dependent on the particular process and 
>> environment. How would we write generic code that creates channels, if 
>> those need to be bound to limits unknown?
>>
>> Thanks,
>> Ernesto
>>
>

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/clojure/7f9f5efe-883f-47e4-b2a5-a457fb7444ed%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: core.async: Unbound channels

[Ghadi Shayban]

(chan) is not a channel with an unbounded buffer. It is a channel with *no* 
buffer and needs to rendezvous putters and takers 1-to-1.  (Additionally it 
will throw an exception if more than 1024 takers or putters are enqueued 
waiting)

On Wednesday, July 3, 2019 at 7:14:46 AM UTC-4, Ernesto Garcia wrote:
>
> You can create a unbound channel with (chan), but not if you use a 
> transducer; (chan nil (filter odd?)) will raise an error that no buffer 
> is provided. Why is this the case?
>
> Why the enforcement of all channels to be bound? In a program, there will 
> be channels that propagate to other channels, so only channels at the 
> boundaries would require to be bound?
>
> Channel limits are also much dependent on the particular process and 
> environment. How would we write generic code that creates channels, if 
> those need to be bound to limits unknown?
>
> Thanks,
> Ernesto
>

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/clojure/12cf0bf8-dafd-429b-a2d2-3875f579f1c1%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: core.async buffered channel behavior

[Timothy Baldridge]

The best way to understand how/why this happens is to look at the source of
>!!. In short, the thread making the call doesn't block on the channel. It
starts an async put, then waits on a promise that is delivered by the async
put. So it works something like this:

1) Calling thread creates a promise
2) Calling thread calls put! telling it to deliver the promise once the put
completes
3) Calling thread tries an immediate put! but fails since the channel is
blocked. So the put is enqueued into the channel.
4) Calling thread is canceled (maybe? not sure how this works in nrepl or
whatever is being used here)
5) put! completes, so a different thread executes the callback and delivers
the promise
6) If the thread hasn't been completely killed yet it's possible that it
may get the value from the delivered promise and continue

In short, core.async doesn't support any sort of cancellation without use
of alt!



On Wed, Jun 27, 2018 at 11:18 AM, Justin Smith  wrote:

> I should be more precise there, by "consumed" I meant buffered or consumed.
>
> On Wed, Jun 27, 2018 at 10:17 AM Justin Smith 
> wrote:
>
>> I doubt core.async would ever make promises about the behavior of a
>> blocking put that gets forcibly cancelled. It promises that the blocking
>> put doesn't return until the message is consumed, but that's not the same
>> as promising that the message isn't consumed if the blocking put is
>> forcibly cancelled.
>>
>> On Wed, Jun 27, 2018 at 9:47 AM Didier  wrote:
>>
>>> I think its due to ctrl+c, not sure what it actually does, but maybe it
>>> didn't actually kill the blocked thread?
>>>
>>> --
>>> You received this message because you are subscribed to the Google
>>> Groups "Clojure" group.
>>> To post to this group, send email to clojure@googlegroups.com
>>> Note that posts from new members are moderated - please be patient with
>>> your first post.
>>> To unsubscribe from this group, send email to
>>> clojure+unsubscr...@googlegroups.com
>>> For more options, visit this group at
>>> http://groups.google.com/group/clojure?hl=en
>>> ---
>>> You received this message because you are subscribed to the Google
>>> Groups "Clojure" group.
>>> To unsubscribe from this group and stop receiving emails from it, send
>>> an email to clojure+unsubscr...@googlegroups.com.
>>> For more options, visit https://groups.google.com/d/optout.
>>>
>> --
> You received this message because you are subscribed to the Google
> Groups "Clojure" group.
> To post to this group, send email to clojure@googlegroups.com
> Note that posts from new members are moderated - please be patient with
> your first post.
> To unsubscribe from this group, send email to
> clojure+unsubscr...@googlegroups.com
> For more options, visit this group at
> http://groups.google.com/group/clojure?hl=en
> ---
> You received this message because you are subscribed to the Google Groups
> "Clojure" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to clojure+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.
>



-- 
“One of the main causes of the fall of the Roman Empire was that–lacking
zero–they had no way to indicate successful termination of their C
programs.”
(Robert Firth)

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: core.async buffered channel behavior

[Justin Smith]

I should be more precise there, by "consumed" I meant buffered or consumed.

On Wed, Jun 27, 2018 at 10:17 AM Justin Smith  wrote:

> I doubt core.async would ever make promises about the behavior of a
> blocking put that gets forcibly cancelled. It promises that the blocking
> put doesn't return until the message is consumed, but that's not the same
> as promising that the message isn't consumed if the blocking put is
> forcibly cancelled.
>
> On Wed, Jun 27, 2018 at 9:47 AM Didier  wrote:
>
>> I think its due to ctrl+c, not sure what it actually does, but maybe it
>> didn't actually kill the blocked thread?
>>
>> --
>> You received this message because you are subscribed to the Google
>> Groups "Clojure" group.
>> To post to this group, send email to clojure@googlegroups.com
>> Note that posts from new members are moderated - please be patient with
>> your first post.
>> To unsubscribe from this group, send email to
>> clojure+unsubscr...@googlegroups.com
>> For more options, visit this group at
>> http://groups.google.com/group/clojure?hl=en
>> ---
>> You received this message because you are subscribed to the Google Groups
>> "Clojure" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to clojure+unsubscr...@googlegroups.com.
>> For more options, visit https://groups.google.com/d/optout.
>>
>

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: core.async buffered channel behavior

[Justin Smith]

I doubt core.async would ever make promises about the behavior of a
blocking put that gets forcibly cancelled. It promises that the blocking
put doesn't return until the message is consumed, but that's not the same
as promising that the message isn't consumed if the blocking put is
forcibly cancelled.

On Wed, Jun 27, 2018 at 9:47 AM Didier  wrote:

> I think its due to ctrl+c, not sure what it actually does, but maybe it
> didn't actually kill the blocked thread?
>
> --
> You received this message because you are subscribed to the Google
> Groups "Clojure" group.
> To post to this group, send email to clojure@googlegroups.com
> Note that posts from new members are moderated - please be patient with
> your first post.
> To unsubscribe from this group, send email to
> clojure+unsubscr...@googlegroups.com
> For more options, visit this group at
> http://groups.google.com/group/clojure?hl=en
> ---
> You received this message because you are subscribed to the Google Groups
> "Clojure" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to clojure+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: core.async buffered channel behavior

[craig worrall]

I guess the interrupt doesn't really obliterate the fourth put attempt, and 
that put proceeds in background when you first take. 

On Wednesday, June 27, 2018 at 5:12:45 AM UTC+10, jonah wrote:
>
> Hi folks,
>
> It's been a while since I've used core.async. Documentation suggests that
>
> (chan n) 
>
> where n is a number creates a fixed size channel buffer supporting n 
> elements.
>
> The below clj repl session seems to indicate that I can put 4 items into a 
> 3-sized buffer:
>
> user=> (def c (async/chan 3))
> #'user/c
> user=> (async/>!! c :a)
> true
> user=> (async/>!! c :b)
> true
> user=> (async/>!! c :c)
> true
> user=> (async/>!! c :d)
> ;; correctly blocks, ctrl-c to break
> user=> (async/ :a
> user=> (async/ :b
> user=> (async/ :c
> ;; why do i get :d ? shouldn't it block?
> user=> (async/ :d
>
> This is with:
>
> [org.clojure/core.async "0.4.474"]
>
> Thanks,
>
> Jonah
>
>
>

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: core.async consumer + producer working by chunk?

[Brian J. Rubinton]

https://dev.clojure.org/jira/browse/ASYNC-210 


> On Jan 6, 2018, at 12:11 PM, Brian J. Rubinton  
> wrote:
> 
> Thanks! I will. Just signed the CA.
> 
> 
> On Sat, Jan 6, 2018, 12:10 PM Alex Miller  > wrote:
> 
> 
> On Saturday, January 6, 2018 at 10:56:06 AM UTC-6, Brian J. Rubinton wrote:
> Alex - it makes sense to me that the buffer temporarily expands beyond its 
> normal size with the content of the expanding transducer. What does not make 
> sense to me is the buffer also accepts puts even though its buffer is full.
> 
> Why would the take! process puts when the channel's buffer is full?
> 
> I was experimenting with the implementation and I think the patch in this 
> gist leads to more intuitive behavior — though it’s possible it breaks 
> something else. 
> https://gist.github.com/brianru/d30f0319e7a14d875a80762937cccb9c 
> 
> 
> Possibly, feel free to file a ticket and I will take a closer look.
>  
> 
> -- 
> You received this message because you are subscribed to the Google
> Groups "Clojure" group.
> To post to this group, send email to clojure@googlegroups.com 
> 
> Note that posts from new members are moderated - please be patient with your 
> first post.
> To unsubscribe from this group, send email to
> clojure+unsubscr...@googlegroups.com 
> 
> For more options, visit this group at
> http://groups.google.com/group/clojure?hl=en 
> 
> --- 
> You received this message because you are subscribed to the Google Groups 
> "Clojure" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to clojure+unsubscr...@googlegroups.com 
> .
> For more options, visit https://groups.google.com/d/optout 
> .

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: core.async consumer + producer working by chunk?

[Brian J. Rubinton]

Thanks! I will. Just signed the CA.

On Sat, Jan 6, 2018, 12:10 PM Alex Miller  wrote:

>
>
> On Saturday, January 6, 2018 at 10:56:06 AM UTC-6, Brian J. Rubinton wrote:
>>
>> Alex - it makes sense to me that the buffer temporarily expands beyond
>> its normal size with the content of the expanding transducer. What does not
>> make sense to me is the buffer also accepts puts even though its buffer is
>> full.
>>
>> Why would the take! process puts when the channel's buffer is full?
>>
>> I was experimenting with the implementation and I think the patch in this
>> gist leads to more intuitive behavior — though it’s possible it breaks
>> something else.
>> https://gist.github.com/brianru/d30f0319e7a14d875a80762937cccb9c
>>
>
> Possibly, feel free to file a ticket and I will take a closer look.
>
>
> --
> You received this message because you are subscribed to the Google
> Groups "Clojure" group.
> To post to this group, send email to clojure@googlegroups.com
> Note that posts from new members are moderated - please be patient with
> your first post.
> To unsubscribe from this group, send email to
> clojure+unsubscr...@googlegroups.com
> For more options, visit this group at
> http://groups.google.com/group/clojure?hl=en
> ---
> You received this message because you are subscribed to the Google Groups
> "Clojure" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to clojure+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: core.async consumer + producer working by chunk?

[Alex Miller]

On Saturday, January 6, 2018 at 10:56:06 AM UTC-6, Brian J. Rubinton wrote:
>
> Alex - it makes sense to me that the buffer temporarily expands beyond its 
> normal size with the content of the expanding transducer. What does not 
> make sense to me is the buffer also accepts puts even though its buffer is 
> full.
>
> Why would the take! process puts when the channel's buffer is full?
>
> I was experimenting with the implementation and I think the patch in this 
> gist leads to more intuitive behavior — though it’s possible it breaks 
> something else. 
> https://gist.github.com/brianru/d30f0319e7a14d875a80762937cccb9c
>

Possibly, feel free to file a ticket and I will take a closer look.
 

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: core.async consumer + producer working by chunk?

[Brian J. Rubinton]

Typo — I meant to say the channel executes puts during a take! even though the 
buffer is full before executing the puts. This is clearer in code (please see 
the gist).

> On Jan 6, 2018, at 11:55 AM, Brian J. Rubinton  
> wrote:
> 
> Alex - it makes sense to me that the buffer temporarily expands beyond its 
> normal size with the content of the expanding transducer. What does not make 
> sense to me is the buffer also accepts puts even though its buffer is full.
> 
> Why would the take! process puts when the channel's buffer is full?
> 
> I was experimenting with the implementation and I think the patch in this 
> gist leads to more intuitive behavior — though it’s possible it breaks 
> something else. 
> https://gist.github.com/brianru/d30f0319e7a14d875a80762937cccb9c 
> 
> 
> 
> 
>> On Jan 6, 2018, at 11:43 AM, Alex Miller > > wrote:
>> 
>> 
>> 
>> On Saturday, January 6, 2018 at 10:27:20 AM UTC-6, Rob Nikander wrote:
>> 
>> On Jan 5, 2018, at 8:01 PM, Gary Verhaegen > wrote:
>> What about simply having the producer put items one by one on the channel?
>> 
>> I will do that. My current producer is doing too many other things, but if I 
>> break it up into separate threads or go blocks for each work queue, then 
>> that should work. Thank you.
>>  
>> 
>> On Saturday, January 6, 2018 at 8:22:34 AM UTC-5, Brian J. Rubinton wrote:
>> I think the behavior in our examples differ because the blocking puts will 
>> complete whenever there is a take and the buffer is not full, ignoring 
>> whether the transducer is still outputting values. This bug may be relevant, 
>> though there it arises in a less common scenario (fixed buffer of size 0, 
>> which is now disallowed) https://dev.clojure.org/jira/browse/ASYNC-140 
>> 
>> 
>> So, should I report this as a bug? If you have channel with a buffer and a 
>> (mapcat identity) transducer, the number of items in the channel can grow 
>> without bound. I thought channels were supposed to prevent that.
>> 
>> This is by design. Because the transducer executes in the channel, it's 
>> possible for it to temporarily expand the buffer beyond its normal size. 
>> This is certainly something to be aware of when using an expanding 
>> transducer (mapcat is probably the most common case).
>> 
>> -- 
>> You received this message because you are subscribed to the Google
>> Groups "Clojure" group.
>> To post to this group, send email to clojure@googlegroups.com 
>> 
>> Note that posts from new members are moderated - please be patient with your 
>> first post.
>> To unsubscribe from this group, send email to
>> clojure+unsubscr...@googlegroups.com 
>> 
>> For more options, visit this group at
>> http://groups.google.com/group/clojure?hl=en 
>> 
>> --- 
>> You received this message because you are subscribed to the Google Groups 
>> "Clojure" group.
>> To unsubscribe from this group and stop receiving emails from it, send an 
>> email to clojure+unsubscr...@googlegroups.com 
>> .
>> For more options, visit https://groups.google.com/d/optout 
>> .
> 

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: core.async consumer + producer working by chunk?

[Brian J. Rubinton]

Alex - it makes sense to me that the buffer temporarily expands beyond its 
normal size with the content of the expanding transducer. What does not make 
sense to me is the buffer also accepts puts even though its buffer is full.

Why would the take! process puts when the channel's buffer is full?

I was experimenting with the implementation and I think the patch in this gist 
leads to more intuitive behavior — though it’s possible it breaks something 
else. https://gist.github.com/brianru/d30f0319e7a14d875a80762937cccb9c 




> On Jan 6, 2018, at 11:43 AM, Alex Miller  wrote:
> 
> 
> 
> On Saturday, January 6, 2018 at 10:27:20 AM UTC-6, Rob Nikander wrote:
> 
> On Jan 5, 2018, at 8:01 PM, Gary Verhaegen > wrote:
> What about simply having the producer put items one by one on the channel?
> 
> I will do that. My current producer is doing too many other things, but if I 
> break it up into separate threads or go blocks for each work queue, then that 
> should work. Thank you.
>  
> 
> On Saturday, January 6, 2018 at 8:22:34 AM UTC-5, Brian J. Rubinton wrote:
> I think the behavior in our examples differ because the blocking puts will 
> complete whenever there is a take and the buffer is not full, ignoring 
> whether the transducer is still outputting values. This bug may be relevant, 
> though there it arises in a less common scenario (fixed buffer of size 0, 
> which is now disallowed) https://dev.clojure.org/jira/browse/ASYNC-140 
> 
> 
> So, should I report this as a bug? If you have channel with a buffer and a 
> (mapcat identity) transducer, the number of items in the channel can grow 
> without bound. I thought channels were supposed to prevent that.
> 
> This is by design. Because the transducer executes in the channel, it's 
> possible for it to temporarily expand the buffer beyond its normal size. This 
> is certainly something to be aware of when using an expanding transducer 
> (mapcat is probably the most common case).
> 
> -- 
> You received this message because you are subscribed to the Google
> Groups "Clojure" group.
> To post to this group, send email to clojure@googlegroups.com
> Note that posts from new members are moderated - please be patient with your 
> first post.
> To unsubscribe from this group, send email to
> clojure+unsubscr...@googlegroups.com
> For more options, visit this group at
> http://groups.google.com/group/clojure?hl=en 
> 
> --- 
> You received this message because you are subscribed to the Google Groups 
> "Clojure" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to clojure+unsubscr...@googlegroups.com 
> .
> For more options, visit https://groups.google.com/d/optout 
> .

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: core.async consumer + producer working by chunk?

[Alex Miller]

On Saturday, January 6, 2018 at 10:27:20 AM UTC-6, Rob Nikander wrote:
>
>
> On Jan 5, 2018, at 8:01 PM, Gary Verhaegen  wrote:
>
>> What about simply having the producer put items one by one on the channel?
>
>
> I will do that. My current producer is doing too many other things, but if 
> I break it up into separate threads or go blocks for each work queue, then 
> that should work. Thank you.
>  
>
> On Saturday, January 6, 2018 at 8:22:34 AM UTC-5, Brian J. Rubinton wrote:
>>
>> I think the behavior in our examples differ because the blocking puts 
>> will complete whenever there is a take and the buffer is not full, ignoring 
>> whether the transducer is still outputting values. This bug may be 
>> relevant, though there it arises in a less common scenario (fixed buffer of 
>> size 0, which is now disallowed) 
>> https://dev.clojure.org/jira/browse/ASYNC-140 
>> 
>>
>
> So, should I report this as a bug? If you have channel with a buffer and a 
> (mapcat identity) transducer, the number of items in the channel can grow 
> without bound. I thought channels were supposed to prevent that.
>

This is by design. Because the transducer executes in the channel, it's 
possible for it to temporarily expand the buffer beyond its normal size. 
This is certainly something to be aware of when using an expanding 
transducer (mapcat is probably the most common case).

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: core.async consumer + producer working by chunk?

[Rob Nikander]

On Jan 5, 2018, at 8:01 PM, Gary Verhaegen  wrote:

> What about simply having the producer put items one by one on the channel?


I will do that. My current producer is doing too many other things, but if 
I break it up into separate threads or go blocks for each work queue, then 
that should work. Thank you.
 

On Saturday, January 6, 2018 at 8:22:34 AM UTC-5, Brian J. Rubinton wrote:
>
> I think the behavior in our examples differ because the blocking puts will 
> complete whenever there is a take and the buffer is not full, ignoring 
> whether the transducer is still outputting values. This bug may be 
> relevant, though there it arises in a less common scenario (fixed buffer of 
> size 0, which is now disallowed) 
> https://dev.clojure.org/jira/browse/ASYNC-140
>

So, should I report this as a bug? If you have channel with a buffer and a 
(mapcat identity) transducer, the number of items in the channel can grow 
without bound. I thought channels were supposed to prevent that.

 

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: core.async consumer + producer working by chunk?

[Brian J. Rubinton]

Rob - I’d go with Gary's approach, which essentially moves the splitting up of 
the chunk of results from the core.async channel’s transducer to the producing 
function. You can do that using a channel with a fixed buffer of 50 and >!!. As 
long as the next db query is blocked until each of the results from the 
previous query are put onto the channel it’ll work as you want.

I think the behavior in our examples differ because the blocking puts will 
complete whenever there is a take and the buffer is not full, ignoring whether 
the transducer is still outputting values. This bug may be relevant, though 
there it arises in a less common scenario (fixed buffer of size 0, which is now 
disallowed) https://dev.clojure.org/jira/browse/ASYNC-140 


> On Jan 5, 2018, at 8:01 PM, Gary Verhaegen  wrote:
> 
> On 5 January 2018 at 19:44, Rob Nikander  > wrote:
> Hi,
> 
> I’m wondering if there is a core.async design idiom for this situation...
> 
> - A buffered channel 
> - One producer feeding it 
> - A bunch of consumers pulling from it.
> - Producer should wake up and fill the channel only when it’s empty. In other 
> words, the producer should work in chunks.
> 
> My first idea is to have two channels. The second will be used by consumers 
> to signal the producer that the primary channel is empty. But I'm wondering 
> if there is a better way.
> 
> The motive for this is that the producer is doing a DB query that is more 
> efficient in bulk. `select ... limit 50` rather than `select ... limit 1` 50 
> times.
> 
> Rob
> 
> What about simply having the producer put items one by one on the channel?
> 
> (ns t.core
>   (:require [clojure.core.async :as async]))
> 
> (defn ap
>   "atomic print"
>   [m]
>   (print (str (pr-str m) "\n")))
> 
> (defn produce-next-batch
>   [s]
>   (let [m (+ s 10)]
> [(range s m) m]))
> 
> (defn chunked-producer
>   [init-state]
>   (let [result-chan (async/chan)]
> (async/go
>   (loop [[batch cursor] (produce-next-batch init-state)]
> (ap {:produced batch})
> (doseq [elem batch]
>   (async/>! result-chan elem))
> (recur (produce-next-batch cursor
> result-chan))
> 
> (defn run-consumer
>   [ch id n]
>   (async/go
> (dotimes [_ n]
>   (ap {:id id :received (async/   (async/ 
> (defn run
>   []
>   (let [c (chunked-producer 0)
> c1 (run-consumer c 1 10)
> c2 (run-consumer c 2 10)
> c3 (run-consumer c 3 10)]
> (->> [c1 c2 c3]
>  (mapv async/ (flush)))
> Here produce-next-batch has been deliberately written to evoke the idea that 
> you have some sort of state or cursor that lets you produce the next batch. 
> Real code would obviously need to account for exceptions, handle channel 
> closing, etc., but hopefully this illustrate the idea.
> 
> 
> 
> -- 
> You received this message because you are subscribed to the Google
> Groups "Clojure" group.
> To post to this group, send email to clojure@googlegroups.com
> Note that posts from new members are moderated - please be patient with your 
> first post.
> To unsubscribe from this group, send email to
> clojure+unsubscr...@googlegroups.com
> For more options, visit this group at
> http://groups.google.com/group/clojure?hl=en 
> 
> --- 
> You received this message because you are subscribed to the Google Groups 
> "Clojure" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to clojure+unsubscr...@googlegroups.com 
> .
> For more options, visit https://groups.google.com/d/optout 
> .

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: core.async consumer + producer working by chunk?

[Gary Verhaegen]

On 5 January 2018 at 19:44, Rob Nikander  wrote:

> Hi,
>
> I’m wondering if there is a core.async design idiom for this situation...
>
> - A buffered channel
> - One producer feeding it
> - A bunch of consumers pulling from it.
> - Producer should wake up and fill the channel only when it’s empty. In
> other words, the producer should work in chunks.
>
> My first idea is to have two channels. The second will be used by
> consumers to signal the producer that the primary channel is empty. But I'm
> wondering if there is a better way.
>
> The motive for this is that the producer is doing a DB query that is more
> efficient in bulk. `select ... limit 50` rather than `select ... limit 1`
> 50 times.
>
> Rob
>

What about simply having the producer put items one by one on the channel?

(ns t.core
  (:require [clojure.core.async :as async]))
(defn ap
  "atomic print"
  [m]
  (print (str (pr-str m) "\n")))
(defn produce-next-batch
  [s]
  (let [m (+ s 10)]
[(range s m) m]))
(defn chunked-producer
  [init-state]
  (let [result-chan (async/chan)]
(async/go
  (loop [[batch cursor] (produce-next-batch init-state)]
(ap {:produced batch})
(doseq [elem batch]
  (async/>! result-chan elem))
(recur (produce-next-batch cursor
result-chan))
(defn run-consumer
  [ch id n]
  (async/go
(dotimes [_ n]
  (ap {:id id :received (async/> [c1 c2 c3]
 (mapv async/

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: core.async consumer + producer working by chunk?

[Brian J. Rubinton]

I don’t know; I don’t fully understand the implementation differences of >!! 
and offer!. The behavior of offer! makes me think the buffer is not empty until 
all the outputs of the transducer are consumed, but the behavior of >!! makes 
me think otherwise.

Moritz - is the buffer cleared if:
- it’s size is 1
- you’ve put 1 item onto the channel 
- that 1 item is transformed into 3 by the channel’s transducer
- and only 1 of the 3 items are taken from the channel?

Per the docstrings and the behavior it’s both not possible to put another value 
onto the channel immediately (so offer! returns nil) yet buffer space is 
available after only 1 of the 3 values on the channel are taken (so >!! doesn't 
block).
https://github.com/clojure/core.async/blob/master/src/main/clojure/clojure/core/async.clj#L138
 

https://github.com/clojure/core.async/blob/master/src/main/clojure/clojure/core/async.clj#L391
 


This seems like a weird corner case made possible by transducers. Maybe the 
collection put into the buffer is removed from the buffer and passed to the 
transducer on the first take, so the buffer is empty and that’s all >!! cares 
about, yet there are still values to be taken from the channel (from the 
transducer — not the buffer) and somehow that affects offer!’s behavior.


> On Jan 5, 2018, at 4:07 PM, Rob Nikander  wrote:
> 
> 
> 
> On Friday, January 5, 2018 at 4:00:25 PM UTC-5, Moritz Ulrich wrote:
> 
> You have a channel with a buffer-size of one. You clear the buffer by 
> taking one item from it, making room for another one. Therefore the put 
> succeeds. Try just `(async/chan nil xform)` to create a channel without 
> a backing buffer (a rendezvouz channel) where puts only succeed if 
> there's a matching consumer. 
> 
> Then why does Brian's code work the way it does? See how he takes 2 things 
> off the channel, and offer! still fails and returns nil. If offer! is 
> returning nil, shouldn't >!! be blocking?
> 
> 
> 
> 
> -- 
> You received this message because you are subscribed to the Google
> Groups "Clojure" group.
> To post to this group, send email to clojure@googlegroups.com
> Note that posts from new members are moderated - please be patient with your 
> first post.
> To unsubscribe from this group, send email to
> clojure+unsubscr...@googlegroups.com
> For more options, visit this group at
> http://groups.google.com/group/clojure?hl=en 
> 
> --- 
> You received this message because you are subscribed to the Google Groups 
> "Clojure" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to clojure+unsubscr...@googlegroups.com 
> .
> For more options, visit https://groups.google.com/d/optout 
> .

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: core.async consumer + producer working by chunk?

[Rob Nikander]

On Friday, January 5, 2018 at 4:00:25 PM UTC-5, Moritz Ulrich wrote:
>
>
> You have a channel with a buffer-size of one. You clear the buffer by 
> taking one item from it, making room for another one. Therefore the put 
> succeeds. Try just `(async/chan nil xform)` to create a channel without 
> a backing buffer (a rendezvouz channel) where puts only succeed if 
> there's a matching consumer. 
>

Then why does Brian's code work the way it does? See how he takes 2 things 
off the channel, and offer! still fails and returns nil. If offer! is 
returning nil, shouldn't >!! be blocking?



-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: core.async consumer + producer working by chunk?

[Moritz Ulrich]

Rob Nikander  writes:

> Thanks for the explanation! This is very close to what I want. I see some 
> confusing behavior though. See below.
>
> On Friday, January 5, 2018 at 2:40:14 PM UTC-5, Brian J. Rubinton wrote:
>>
>>
>> The work-queue channel has a fixed buffer size of 1. A collection (range 
>> 50) is put on the channel. While consumers can take items off the channel — 
>> note the individual contents of (range 50) are returned — a producer cannot 
>> put another value onto the channel until the entire initial collection is 
>> consumed. 
>>
>
> I tried your example code works for me, but then I tried using `>!!` 
> instead of `offer!` and it behaves differently.
>
> user> (def c (chan 1 (mapcat identity)))
> => #'user/c
> user> (do 
>(async/thread (async/>!! c [1 2 3]) (println "put #1"))
>(async/thread (async/>!! c [4 5 6]) (println "put #2"))
>(async/thread (async/>!! c [7 8 9]) (println "put #3")))
> put #1
>
> As expected it prints "put #1" right away, and other threads are blocked. 
> But...
>
> user> (poll! c)
> => 1
> put #2
> user> (poll! c)
> => 2
> put #3
> 
> As soon as your read anything from it, other puts succeed. Shouldn't they 
> be blocked?

You have a channel with a buffer-size of one. You clear the buffer by
taking one item from it, making room for another one. Therefore the put
succeeds. Try just `(async/chan nil xform)` to create a channel without
a backing buffer (a rendezvouz channel) where puts only succeed if
there's a matching consumer.

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: core.async consumer + producer working by chunk?

[Rob Nikander]

Thanks for the explanation! This is very close to what I want. I see some 
confusing behavior though. See below.

On Friday, January 5, 2018 at 2:40:14 PM UTC-5, Brian J. Rubinton wrote:
>
>
> The work-queue channel has a fixed buffer size of 1. A collection (range 
> 50) is put on the channel. While consumers can take items off the channel — 
> note the individual contents of (range 50) are returned — a producer cannot 
> put another value onto the channel until the entire initial collection is 
> consumed. 
>

I tried your example code works for me, but then I tried using `>!!` 
instead of `offer!` and it behaves differently.

user> (def c (chan 1 (mapcat identity)))
=> #'user/c
user> (do 
   (async/thread (async/>!! c [1 2 3]) (println "put #1"))
   (async/thread (async/>!! c [4 5 6]) (println "put #2"))
   (async/thread (async/>!! c [7 8 9]) (println "put #3")))
put #1

As expected it prints "put #1" right away, and other threads are blocked. 
But...
   
user> (poll! c)
=> 1
put #2
user> (poll! c)
=> 2
put #3

As soon as your read anything from it, other puts succeed. Shouldn't they 
be blocked?

Rob 

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: core.async consumer + producer working by chunk?

[Brian J. Rubinton]

The `mapcat` transducer takes a collection as its input and outputs each of its 
items individually. This example might be helpful:

user> (use ‘[clojure.core.async])
nil
user> (def work-queue (chan 1 (mapcat identity)))
#’user/work-queue
user> (offer! work-queue (range 50))
true
user> ( ( (offer! work-queue (range 50))
nil
user> (dotimes [_ 48] ( (offer! work-queue (range 50))
true
user> ( On Jan 5, 2018, at 2:10 PM, Rob Nikander  wrote:
> 
> 
> 
> On Friday, January 5, 2018 at 2:03:00 PM UTC-5, Brian J. Rubinton wrote:
> 
> What is the buffered channel’s buffer used for? If that’s set to 1 and the 
> channel’s transducer is `(mapcat identity)` then the producer should be able 
> to continuously put chunks of work onto the channel with the puts only 
> completing when the previous chunk is completely consumed.
> 
> The buffer is sort of a work queue, used to break up and distribute one 
> "chunk" to a pool of consumers. I was imaging the buffer would be size 50. So 
> the producer would grab 50 rows (one "chunk") and put them all in the 
> channel. Consumers would pick them out one by one.
> 
> I'm not familiar with the transducer concept so I need to go read a bit 
> before I can understand your point about the channel's transducer being 
> `(mapcat identity)`.
> 
>  
> 
> -- 
> You received this message because you are subscribed to the Google
> Groups "Clojure" group.
> To post to this group, send email to clojure@googlegroups.com
> Note that posts from new members are moderated - please be patient with your 
> first post.
> To unsubscribe from this group, send email to
> clojure+unsubscr...@googlegroups.com
> For more options, visit this group at
> http://groups.google.com/group/clojure?hl=en 
> 
> --- 
> You received this message because you are subscribed to the Google Groups 
> "Clojure" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to clojure+unsubscr...@googlegroups.com 
> .
> For more options, visit https://groups.google.com/d/optout 
> .

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: core.async consumer + producer working by chunk?

[Rob Nikander]

On Friday, January 5, 2018 at 2:03:00 PM UTC-5, Brian J. Rubinton wrote:
>
>
> What is the buffered channel’s buffer used for? If that’s set to 1 and the 
> channel’s transducer is `(mapcat identity)` then the producer should be 
> able to continuously put chunks of work onto the channel with the puts only 
> completing when the previous chunk is completely consumed.
>

The buffer is sort of a work queue, used to break up and distribute one 
"chunk" to a pool of consumers. I was imaging the buffer would be size 50. 
So the producer would grab 50 rows (one "chunk") and put them all in the 
channel. Consumers would pick them out one by one.

I'm not familiar with the transducer concept so I need to go read a bit 
before I can understand your point about the channel's transducer being 
`(mapcat identity)`.

 

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: core.async consumer + producer working by chunk?

[Brian J. Rubinton]

Hi Rob,

What is the buffered channel’s buffer used for? If that’s set to 1 and the 
channel’s transducer is `(mapcat identity)` then the producer should be able to 
continuously put chunks of work onto the channel with the puts only completing 
when the previous chunk is completely consumed.

That would make the producer block/park until the previous chunk is consumed.

For your use case I think an interesting result of this approach (maybe a 
downside?) is initially 2 db queries would be performed, 1 whose result makes 
it onto the channel while 2’s result will be parked until the previous query’s 
result is consumed.

Brian

> On Jan 5, 2018, at 1:44 PM, Rob Nikander  wrote:
> 
> Hi,
> 
> I’m wondering if there is a core.async design idiom for this situation...
> 
> - A buffered channel 
> - One producer feeding it 
> - A bunch of consumers pulling from it.
> - Producer should wake up and fill the channel only when it’s empty. In other 
> words, the producer should work in chunks.
> 
> My first idea is to have two channels. The second will be used by consumers 
> to signal the producer that the primary channel is empty. But I'm wondering 
> if there is a better way.
> 
> The motive for this is that the producer is doing a DB query that is more 
> efficient in bulk. `select ... limit 50` rather than `select ... limit 1` 50 
> times.
> 
> Rob
> 
> -- 
> You received this message because you are subscribed to the Google
> Groups "Clojure" group.
> To post to this group, send email to clojure@googlegroups.com
> Note that posts from new members are moderated - please be patient with your 
> first post.
> To unsubscribe from this group, send email to
> clojure+unsubscr...@googlegroups.com
> For more options, visit this group at
> http://groups.google.com/group/clojure?hl=en 
> 
> --- 
> You received this message because you are subscribed to the Google Groups 
> "Clojure" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to clojure+unsubscr...@googlegroups.com 
> .
> For more options, visit https://groups.google.com/d/optout 
> .

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: core.async got in a bad state?

[Matching Socks]

Special behavior built into only certain blocking operations (e.g., 
core.async's, but not java.io's) could instill a false sense of security.  

It could be complemented by a watchdog thread to poll the core.async pool 
threads and call a given fn if a thread was blocked when it shouldn't have 
been.  The watchdog, too, would be a leaky sieve, but it would be likely to 
detect egregious violations, whatever their cause.

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: core.async got in a bad state?

[Didier]

>
> No code called by a go block should ever call the blocking variants of 
> core.async functions (!!, alts!!, etc.). So I'd start at the code 
> redacted in those lines and go from there.
>

Seems like a good use case for a static code analyser.  Maybe a 
contribution to https://github.com/jonase/eastwood

On Tuesday, 29 August 2017 14:40:08 UTC-7, Alex Miller wrote:
>
> We did actually discuss doing something like this a long time ago. The 
> worry that comes to mind is whether it should actually be forbidden (an 
> invariant) or merely strong frowned upon. It is possible to arrange a 
> situation where you know (based on other knowledge) that a put on a channel 
> will succeed. Should that be allowed? I'd have to hammock on that one for a 
> while.
>
> On Tuesday, August 29, 2017 at 3:19:51 PM UTC-5, Aaron Iba wrote:
>>
>> Ahh that makes a lot of sense.  Indeed, I'm guilty of doing a blocking 
>> >!! inside a go-block.  I was so careful to avoid other kinds of blocking 
>> calls (like IO) that I forgot that blocking variants of core.async calls 
>> themselves were forbidden.
>>
>> Thank you for pointing this out!  I will rewire things to not do this.
>>
>> Per Gary's suggestion, I also think it'd be useful if core.async blocking 
>> ops checked a dynamic var (or a property of the thread itself) and at least 
>> warned if they are being called from a forbidden context.  To resolve my 
>> original issue, I'm considering doing this in my dev environment:
>>
>> (doseq [v '[!!]]
>>   (alter-var-root (ns-resolve 'clojure.core.async v)
>>   (fn [f]
>> (fn [& args]
>>   (if (.startsWith (.getName (Thread/currentThread))
>>"async-dispatch-")
>> (throw (Exception. (str v " called inside 
>> async-dispatch")))
>> (apply f args))
>>
>>
>>
>>
>> On Tuesday, August 29, 2017 at 1:43:53 PM UTC-4, Gary Trakhman wrote:
>>>
>>> Hm, I came across a similar ordering invariant (No code called by a go 
>>> block should ever call the blocking variants of core.async functions) while 
>>> wrapping an imperative API, and I thought it might be useful to use 
>>> vars/binding to enforce it.  Has this or other approaches been considered 
>>> in core.async?  I could see a *fixed-thread-pool* var being set and >!! 
>>> checking for false.
>>>
>>> An analogy in existing clojure.core would be the STM commute's 'must be 
>>> running in a transaction' check that uses a threadlocal. 
>>> https://github.com/clojure/clojure/blob/master/src/jvm/clojure/lang/LockingTransaction.java#L205
>>>
>>> On Tue, Aug 29, 2017 at 1:30 PM Timothy Baldridge  
>>> wrote:
>>>
 To add to what Alex said, look at this trace: 
 https://gist.github.com/anonymous/65049ffdd37d43df8f23630928e8fed0#file-thread-dump-out-L1337-L1372

 Here we see a go block calling mapcat, and inside the inner map 
 something is calling >!!. As Alex mentioned this can be a source of 
 deadlocks. No code called by a go block should ever call the blocking 
 variants of core.async functions (!!, alts!!, etc.). So I'd start at 
 the code redacted in those lines and go from there. 



 On Tue, Aug 29, 2017 at 11:09 AM, Alex Miller  
 wrote:

> go blocks are multiplexed over a thread pool which has (by default) 8 
> threads. You should never perform any kind of blocking activity inside a 
> go 
> block, because if every go block in work happens to end up blocked, you 
> will prevent all go blocks from making any further progress. It sounds to 
> me like that's what has happened here. The go block threads are named 
> "async-dispatch-" and it looks like there are 8 blocked ones in your 
> thread dump.
>
> It also looks like they are all blocking on a >!!, which is a blocking 
> call. So I would look for a go block that contains a >!! and convert that 
> to a >! or do something else to avoid blocking there.
>
>
> On Tuesday, August 29, 2017 at 11:48:25 AM UTC-5, Aaron Iba wrote:
>>
>> My company has a production system that uses core.async extensively. 
>> We've been running it 24/7 for over a year with occasional restarts to 
>> update things and add features, and so far core.async has been working 
>> great.
>>
>> The other day, during a particularly high workload, the whole system 
>> got locked up. All the channels seemed blocked at once.  I was able to 
>> connect with a REPL and poke around, and noticed strange behavior of 
>> core.async. Specifically, the following code, when evaluated in the 
>> REPL, 
>> blocked on the put (third expression):
>>
>> (def c (async/chan))
>> (go-loop []
>>   (when-some [x (> (println x)
>> (recur)))
>> (>!! c true)
>>
>> Whereas on any fresh system, the above 

Re: core.async got in a bad state?

[Alex Miller]

We did actually discuss doing something like this a long time ago. The 
worry that comes to mind is whether it should actually be forbidden (an 
invariant) or merely strong frowned upon. It is possible to arrange a 
situation where you know (based on other knowledge) that a put on a channel 
will succeed. Should that be allowed? I'd have to hammock on that one for a 
while.

On Tuesday, August 29, 2017 at 3:19:51 PM UTC-5, Aaron Iba wrote:
>
> Ahh that makes a lot of sense.  Indeed, I'm guilty of doing a blocking >!! 
> inside a go-block.  I was so careful to avoid other kinds of blocking calls 
> (like IO) that I forgot that blocking variants of core.async calls 
> themselves were forbidden.
>
> Thank you for pointing this out!  I will rewire things to not do this.
>
> Per Gary's suggestion, I also think it'd be useful if core.async blocking 
> ops checked a dynamic var (or a property of the thread itself) and at least 
> warned if they are being called from a forbidden context.  To resolve my 
> original issue, I'm considering doing this in my dev environment:
>
> (doseq [v '[!!]]
>   (alter-var-root (ns-resolve 'clojure.core.async v)
>   (fn [f]
> (fn [& args]
>   (if (.startsWith (.getName (Thread/currentThread))
>"async-dispatch-")
> (throw (Exception. (str v " called inside 
> async-dispatch")))
> (apply f args))
>
>
>
>
> On Tuesday, August 29, 2017 at 1:43:53 PM UTC-4, Gary Trakhman wrote:
>>
>> Hm, I came across a similar ordering invariant (No code called by a go 
>> block should ever call the blocking variants of core.async functions) while 
>> wrapping an imperative API, and I thought it might be useful to use 
>> vars/binding to enforce it.  Has this or other approaches been considered 
>> in core.async?  I could see a *fixed-thread-pool* var being set and >!! 
>> checking for false.
>>
>> An analogy in existing clojure.core would be the STM commute's 'must be 
>> running in a transaction' check that uses a threadlocal. 
>> https://github.com/clojure/clojure/blob/master/src/jvm/clojure/lang/LockingTransaction.java#L205
>>
>> On Tue, Aug 29, 2017 at 1:30 PM Timothy Baldridge  
>> wrote:
>>
>>> To add to what Alex said, look at this trace: 
>>> https://gist.github.com/anonymous/65049ffdd37d43df8f23630928e8fed0#file-thread-dump-out-L1337-L1372
>>>
>>> Here we see a go block calling mapcat, and inside the inner map 
>>> something is calling >!!. As Alex mentioned this can be a source of 
>>> deadlocks. No code called by a go block should ever call the blocking 
>>> variants of core.async functions (!!, alts!!, etc.). So I'd start at 
>>> the code redacted in those lines and go from there. 
>>>
>>>
>>>
>>> On Tue, Aug 29, 2017 at 11:09 AM, Alex Miller  
>>> wrote:
>>>
 go blocks are multiplexed over a thread pool which has (by default) 8 
 threads. You should never perform any kind of blocking activity inside a 
 go 
 block, because if every go block in work happens to end up blocked, you 
 will prevent all go blocks from making any further progress. It sounds to 
 me like that's what has happened here. The go block threads are named 
 "async-dispatch-" and it looks like there are 8 blocked ones in your 
 thread dump.

 It also looks like they are all blocking on a >!!, which is a blocking 
 call. So I would look for a go block that contains a >!! and convert that 
 to a >! or do something else to avoid blocking there.


 On Tuesday, August 29, 2017 at 11:48:25 AM UTC-5, Aaron Iba wrote:
>
> My company has a production system that uses core.async extensively. 
> We've been running it 24/7 for over a year with occasional restarts to 
> update things and add features, and so far core.async has been working 
> great.
>
> The other day, during a particularly high workload, the whole system 
> got locked up. All the channels seemed blocked at once.  I was able to 
> connect with a REPL and poke around, and noticed strange behavior of 
> core.async. Specifically, the following code, when evaluated in the REPL, 
> blocked on the put (third expression):
>
> (def c (async/chan))
> (go-loop []
>   (when-some [x ( (println x)
> (recur)))
> (>!! c true)
>
> Whereas on any fresh system, the above expressions obviously succeed.
>
> Puts succeeded if they went onto the channel's buffer, but not when 
> they should go through to a consumer. For example with the following 
> expressions, evaluated in the REPL, the first put succeeded (presumably 
> because it went on the buffer), but subsequent puts blocked:
>
> (def c (async/chan 1))
> (def m (async/mult c))
> (def out (async/chan (async/sliding-buffer 3)))
> (async/tap m out)
> (>!! c 

Re: core.async got in a bad state?

[Aaron Iba]

Ahh that makes a lot of sense.  Indeed, I'm guilty of doing a blocking >!! 
inside a go-block.  I was so careful to avoid other kinds of blocking calls 
(like IO) that I forgot that blocking variants of core.async calls 
themselves were forbidden.

Thank you for pointing this out!  I will rewire things to not do this.

Per Gary's suggestion, I also think it'd be useful if core.async blocking 
ops checked a dynamic var (or a property of the thread itself) and at least 
warned if they are being called from a forbidden context.  To resolve my 
original issue, I'm considering doing this in my dev environment:

(doseq [v '[!!]]
  (alter-var-root (ns-resolve 'clojure.core.async v)
  (fn [f]
(fn [& args]
  (if (.startsWith (.getName (Thread/currentThread))
   "async-dispatch-")
(throw (Exception. (str v " called inside 
async-dispatch")))
(apply f args))




On Tuesday, August 29, 2017 at 1:43:53 PM UTC-4, Gary Trakhman wrote:
>
> Hm, I came across a similar ordering invariant (No code called by a go 
> block should ever call the blocking variants of core.async functions) while 
> wrapping an imperative API, and I thought it might be useful to use 
> vars/binding to enforce it.  Has this or other approaches been considered 
> in core.async?  I could see a *fixed-thread-pool* var being set and >!! 
> checking for false.
>
> An analogy in existing clojure.core would be the STM commute's 'must be 
> running in a transaction' check that uses a threadlocal. 
> https://github.com/clojure/clojure/blob/master/src/jvm/clojure/lang/LockingTransaction.java#L205
>
> On Tue, Aug 29, 2017 at 1:30 PM Timothy Baldridge  > wrote:
>
>> To add to what Alex said, look at this trace: 
>> https://gist.github.com/anonymous/65049ffdd37d43df8f23630928e8fed0#file-thread-dump-out-L1337-L1372
>>
>> Here we see a go block calling mapcat, and inside the inner map something 
>> is calling >!!. As Alex mentioned this can be a source of deadlocks. No 
>> code called by a go block should ever call the blocking variants of 
>> core.async functions (!!, alts!!, etc.). So I'd start at the code 
>> redacted in those lines and go from there. 
>>
>>
>>
>> On Tue, Aug 29, 2017 at 11:09 AM, Alex Miller > > wrote:
>>
>>> go blocks are multiplexed over a thread pool which has (by default) 8 
>>> threads. You should never perform any kind of blocking activity inside a go 
>>> block, because if every go block in work happens to end up blocked, you 
>>> will prevent all go blocks from making any further progress. It sounds to 
>>> me like that's what has happened here. The go block threads are named 
>>> "async-dispatch-" and it looks like there are 8 blocked ones in your 
>>> thread dump.
>>>
>>> It also looks like they are all blocking on a >!!, which is a blocking 
>>> call. So I would look for a go block that contains a >!! and convert that 
>>> to a >! or do something else to avoid blocking there.
>>>
>>>
>>> On Tuesday, August 29, 2017 at 11:48:25 AM UTC-5, Aaron Iba wrote:

 My company has a production system that uses core.async extensively. 
 We've been running it 24/7 for over a year with occasional restarts to 
 update things and add features, and so far core.async has been working 
 great.

 The other day, during a particularly high workload, the whole system 
 got locked up. All the channels seemed blocked at once.  I was able to 
 connect with a REPL and poke around, and noticed strange behavior of 
 core.async. Specifically, the following code, when evaluated in the REPL, 
 blocked on the put (third expression):

 (def c (async/chan))
 (go-loop []
   (when-some [x (>>> (println x)
 (recur)))
 (>!! c true)

 Whereas on any fresh system, the above expressions obviously succeed.

 Puts succeeded if they went onto the channel's buffer, but not when 
 they should go through to a consumer. For example with the following 
 expressions, evaluated in the REPL, the first put succeeded (presumably 
 because it went on the buffer), but subsequent puts blocked:

 (def c (async/chan 1))
 (def m (async/mult c))
 (def out (async/chan (async/sliding-buffer 3)))
 (async/tap m out)
 (>!! c true) ;; succeeds
 (>!! c true) ;; blocks forever

 This leads me to wonder if core.async itself somehow got into a bad 
 state. It's entirely possible I caused this by misusing the API somewhere 
 in the codebase, but we use core.async so extensively that I wouldn't know 
 where to begin looking.

 I'm wondering if someone more familiar with core.async internals has an 
 idea about what could cause the above situation. Or if we notice it 
 happening again, what could I do to gather more helpful information.

 I also 

Re: core.async got in a bad state?

[Gary Trakhman]

Hm, I came across a similar ordering invariant (No code called by a go
block should ever call the blocking variants of core.async functions) while
wrapping an imperative API, and I thought it might be useful to use
vars/binding to enforce it.  Has this or other approaches been considered
in core.async?  I could see a *fixed-thread-pool* var being set and >!!
checking for false.

An analogy in existing clojure.core would be the STM commute's 'must be
running in a transaction' check that uses a threadlocal.
https://github.com/clojure/clojure/blob/master/src/jvm/clojure/lang/LockingTransaction.java#L205

On Tue, Aug 29, 2017 at 1:30 PM Timothy Baldridge 
wrote:

> To add to what Alex said, look at this trace:
> https://gist.github.com/anonymous/65049ffdd37d43df8f23630928e8fed0#file-thread-dump-out-L1337-L1372
>
> Here we see a go block calling mapcat, and inside the inner map something
> is calling >!!. As Alex mentioned this can be a source of deadlocks. No
> code called by a go block should ever call the blocking variants of
> core.async functions (!!, alts!!, etc.). So I'd start at the code
> redacted in those lines and go from there.
>
>
>
> On Tue, Aug 29, 2017 at 11:09 AM, Alex Miller  wrote:
>
>> go blocks are multiplexed over a thread pool which has (by default) 8
>> threads. You should never perform any kind of blocking activity inside a go
>> block, because if every go block in work happens to end up blocked, you
>> will prevent all go blocks from making any further progress. It sounds to
>> me like that's what has happened here. The go block threads are named
>> "async-dispatch-" and it looks like there are 8 blocked ones in your
>> thread dump.
>>
>> It also looks like they are all blocking on a >!!, which is a blocking
>> call. So I would look for a go block that contains a >!! and convert that
>> to a >! or do something else to avoid blocking there.
>>
>>
>> On Tuesday, August 29, 2017 at 11:48:25 AM UTC-5, Aaron Iba wrote:
>>>
>>> My company has a production system that uses core.async extensively.
>>> We've been running it 24/7 for over a year with occasional restarts to
>>> update things and add features, and so far core.async has been working
>>> great.
>>>
>>> The other day, during a particularly high workload, the whole system got
>>> locked up. All the channels seemed blocked at once.  I was able to connect
>>> with a REPL and poke around, and noticed strange behavior of core.async.
>>> Specifically, the following code, when evaluated in the REPL, blocked on
>>> the put (third expression):
>>>
>>> (def c (async/chan))
>>> (go-loop []
>>>   (when-some [x (>> (println x)
>>> (recur)))
>>> (>!! c true)
>>>
>>> Whereas on any fresh system, the above expressions obviously succeed.
>>>
>>> Puts succeeded if they went onto the channel's buffer, but not when they
>>> should go through to a consumer. For example with the following
>>> expressions, evaluated in the REPL, the first put succeeded (presumably
>>> because it went on the buffer), but subsequent puts blocked:
>>>
>>> (def c (async/chan 1))
>>> (def m (async/mult c))
>>> (def out (async/chan (async/sliding-buffer 3)))
>>> (async/tap m out)
>>> (>!! c true) ;; succeeds
>>> (>!! c true) ;; blocks forever
>>>
>>> This leads me to wonder if core.async itself somehow got into a bad
>>> state. It's entirely possible I caused this by misusing the API somewhere
>>> in the codebase, but we use core.async so extensively that I wouldn't know
>>> where to begin looking.
>>>
>>> I'm wondering if someone more familiar with core.async internals has an
>>> idea about what could cause the above situation. Or if we notice it
>>> happening again, what could I do to gather more helpful information.
>>>
>>> I also have a redacted thread dump, in case it's useful:
>>>
>>> https://gist.github.com/anonymous/65049ffdd37d43df8f23630928e8fed0
>>>
>>> Any help would be much appreciated,
>>>
>>> Aaron
>>>
>>> P.S. core.async has been a godsend in terms of helping us structure and
>>> modularize our large system.  Thank you to all those who contributed to
>>> this wonderful library!
>>>
>>> --
>> You received this message because you are subscribed to the Google
>> Groups "Clojure" group.
>> To post to this group, send email to clojure@googlegroups.com
>> Note that posts from new members are moderated - please be patient with
>> your first post.
>> To unsubscribe from this group, send email to
>> clojure+unsubscr...@googlegroups.com
>> For more options, visit this group at
>> http://groups.google.com/group/clojure?hl=en
>> ---
>> You received this message because you are subscribed to the Google Groups
>> "Clojure" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to clojure+unsubscr...@googlegroups.com.
>> For more options, visit https://groups.google.com/d/optout.
>>
>
>
>
> --
> “One of the main causes of the fall of the Roman Empire was that–lacking
> zero–they had no 

Re: core.async got in a bad state?

[Timothy Baldridge]

To add to what Alex said, look at this trace:
https://gist.github.com/anonymous/65049ffdd37d43df8f23630928e8fed0#file-thread-dump-out-L1337-L1372

Here we see a go block calling mapcat, and inside the inner map something
is calling >!!. As Alex mentioned this can be a source of deadlocks. No
code called by a go block should ever call the blocking variants of
core.async functions (!!, alts!!, etc.). So I'd start at the code
redacted in those lines and go from there.



On Tue, Aug 29, 2017 at 11:09 AM, Alex Miller  wrote:

> go blocks are multiplexed over a thread pool which has (by default) 8
> threads. You should never perform any kind of blocking activity inside a go
> block, because if every go block in work happens to end up blocked, you
> will prevent all go blocks from making any further progress. It sounds to
> me like that's what has happened here. The go block threads are named
> "async-dispatch-" and it looks like there are 8 blocked ones in your
> thread dump.
>
> It also looks like they are all blocking on a >!!, which is a blocking
> call. So I would look for a go block that contains a >!! and convert that
> to a >! or do something else to avoid blocking there.
>
>
> On Tuesday, August 29, 2017 at 11:48:25 AM UTC-5, Aaron Iba wrote:
>>
>> My company has a production system that uses core.async extensively.
>> We've been running it 24/7 for over a year with occasional restarts to
>> update things and add features, and so far core.async has been working
>> great.
>>
>> The other day, during a particularly high workload, the whole system got
>> locked up. All the channels seemed blocked at once.  I was able to connect
>> with a REPL and poke around, and noticed strange behavior of core.async.
>> Specifically, the following code, when evaluated in the REPL, blocked on
>> the put (third expression):
>>
>> (def c (async/chan))
>> (go-loop []
>>   (when-some [x (> (println x)
>> (recur)))
>> (>!! c true)
>>
>> Whereas on any fresh system, the above expressions obviously succeed.
>>
>> Puts succeeded if they went onto the channel's buffer, but not when they
>> should go through to a consumer. For example with the following
>> expressions, evaluated in the REPL, the first put succeeded (presumably
>> because it went on the buffer), but subsequent puts blocked:
>>
>> (def c (async/chan 1))
>> (def m (async/mult c))
>> (def out (async/chan (async/sliding-buffer 3)))
>> (async/tap m out)
>> (>!! c true) ;; succeeds
>> (>!! c true) ;; blocks forever
>>
>> This leads me to wonder if core.async itself somehow got into a bad
>> state. It's entirely possible I caused this by misusing the API somewhere
>> in the codebase, but we use core.async so extensively that I wouldn't know
>> where to begin looking.
>>
>> I'm wondering if someone more familiar with core.async internals has an
>> idea about what could cause the above situation. Or if we notice it
>> happening again, what could I do to gather more helpful information.
>>
>> I also have a redacted thread dump, in case it's useful:
>>
>> https://gist.github.com/anonymous/65049ffdd37d43df8f23630928e8fed0
>>
>> Any help would be much appreciated,
>>
>> Aaron
>>
>> P.S. core.async has been a godsend in terms of helping us structure and
>> modularize our large system.  Thank you to all those who contributed to
>> this wonderful library!
>>
>> --
> You received this message because you are subscribed to the Google
> Groups "Clojure" group.
> To post to this group, send email to clojure@googlegroups.com
> Note that posts from new members are moderated - please be patient with
> your first post.
> To unsubscribe from this group, send email to
> clojure+unsubscr...@googlegroups.com
> For more options, visit this group at
> http://groups.google.com/group/clojure?hl=en
> ---
> You received this message because you are subscribed to the Google Groups
> "Clojure" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to clojure+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.
>



-- 
“One of the main causes of the fall of the Roman Empire was that–lacking
zero–they had no way to indicate successful termination of their C
programs.”
(Robert Firth)

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: core.async got in a bad state?

[Alex Miller]

go blocks are multiplexed over a thread pool which has (by default) 8 
threads. You should never perform any kind of blocking activity inside a go 
block, because if every go block in work happens to end up blocked, you 
will prevent all go blocks from making any further progress. It sounds to 
me like that's what has happened here. The go block threads are named 
"async-dispatch-" and it looks like there are 8 blocked ones in your 
thread dump.

It also looks like they are all blocking on a >!!, which is a blocking 
call. So I would look for a go block that contains a >!! and convert that 
to a >! or do something else to avoid blocking there.


On Tuesday, August 29, 2017 at 11:48:25 AM UTC-5, Aaron Iba wrote:
>
> My company has a production system that uses core.async extensively. We've 
> been running it 24/7 for over a year with occasional restarts to update 
> things and add features, and so far core.async has been working great.
>
> The other day, during a particularly high workload, the whole system got 
> locked up. All the channels seemed blocked at once.  I was able to connect 
> with a REPL and poke around, and noticed strange behavior of core.async. 
> Specifically, the following code, when evaluated in the REPL, blocked on 
> the put (third expression):
>
> (def c (async/chan))
> (go-loop []
>   (when-some [x ( (println x)
> (recur)))
> (>!! c true)
>
> Whereas on any fresh system, the above expressions obviously succeed.
>
> Puts succeeded if they went onto the channel's buffer, but not when they 
> should go through to a consumer. For example with the following 
> expressions, evaluated in the REPL, the first put succeeded (presumably 
> because it went on the buffer), but subsequent puts blocked:
>
> (def c (async/chan 1))
> (def m (async/mult c))
> (def out (async/chan (async/sliding-buffer 3)))
> (async/tap m out)
> (>!! c true) ;; succeeds
> (>!! c true) ;; blocks forever
>
> This leads me to wonder if core.async itself somehow got into a bad state. 
> It's entirely possible I caused this by misusing the API somewhere in the 
> codebase, but we use core.async so extensively that I wouldn't know where 
> to begin looking.
>
> I'm wondering if someone more familiar with core.async internals has an 
> idea about what could cause the above situation. Or if we notice it 
> happening again, what could I do to gather more helpful information.
>
> I also have a redacted thread dump, in case it's useful:
>
> https://gist.github.com/anonymous/65049ffdd37d43df8f23630928e8fed0
>
> Any help would be much appreciated,
>
> Aaron
>
> P.S. core.async has been a godsend in terms of helping us structure and 
> modularize our large system.  Thank you to all those who contributed to 
> this wonderful library!
>
>

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: core.async vs continuations

[Timothy Baldridge]

Yes, calls to ! could be written with continuations, or call/cc,
but they are not implemented that way. Instead the code inside the body of
the `go` macro is rewritten into a statemachine. This sort of rewrite is a
lot like the rewrites that C# does for yield, and Cython does much of the
same sort of thing for Python generators.

This blog post goes into many of the details:
http://hueypetersen.com/posts/2013/08/02/the-state-machines-of-core-async/

But the gist is this:
1) Chop up the code into blocks normally via a variant of Single Static
Assignment compilation
2) Move all locals that need to exist across block transitions into a
array, or a object field
3) Store the current state ID in a field.
4) To "pause" the machine, store the id of the next block and exit the
function. Later on you can pick up again by grabbing the correct block id
from the object field and then jumping back into that block.

Timothy

On Mon, Jul 10, 2017 at 2:41 PM, Răzvan Rotaru 
wrote:

> Hi,
>
> Pardon the ignorance, I was just rewatching Rich Hickeys talk about
> core.async, and realized that the await call is actually a continuation
> (like call/cc from scheme). And if I am not mistaken, you can't implement
> call/cc with macros, so I suspect there is a difference, but I fail to see
> it. Hence, my message here.
>
> Thanks,
> Razvan
>
> --
> You received this message because you are subscribed to the Google
> Groups "Clojure" group.
> To post to this group, send email to clojure@googlegroups.com
> Note that posts from new members are moderated - please be patient with
> your first post.
> To unsubscribe from this group, send email to
> clojure+unsubscr...@googlegroups.com
> For more options, visit this group at
> http://groups.google.com/group/clojure?hl=en
> ---
> You received this message because you are subscribed to the Google Groups
> "Clojure" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to clojure+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.
>



-- 
“One of the main causes of the fall of the Roman Empire was that–lacking
zero–they had no way to indicate successful termination of their C
programs.”
(Robert Firth)

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: core.async/close! locks on chans with pending transforms

[Vitalie Spinu]

>> On Mon, Jul 03 2017 21:18, Timothy Baldridge wrote:

> This means, if you want to execute take correctly you must ensure that only
> one thread executes the take instance at one time, since channels already
> operate via a channel-level lock, it makes sense to run the transducer
> inside the channels' lock.

I looked at the code and got somewhat surprised that transducer can be
potentially called within all 3 methods - take!, put! and close!. Not sure if
this could be done differently, but given this implementation it indeed makes
sense to block all 3 methods.

I think the docs regarding the blocking behavior and the timing of transducer's
execution could be improved. I always naively assumed that transducers are
executed on the same thread within put!, but in fact xform is called only when
the value is added to the buffer and can happen both on current thread and on
the thread pool.

This is what I dug so far regarding put!:

1) (half-full buf and pending takes)
 -> transform and add to buf on the same thread

2) (nil buf or full buf) and pending takes
 -> dispatch to pending take on the thread pool
   
3) (half full buf and no pending takes)
 -> transform and add to buf on the same thread
 
4) (nil of full buf)
 -> append to puts list


I find 2 troublesome. It means that this line:

  
https://github.com/clojure/core.async/blob/master/src/main/clojure/clojure/core/async/impl/channels.clj#L135

can be reached in the corner case when buffer is full and there are pending
takes. If so then the current val is dispatched to the leading pending take
bypassing buf completely. I think that line is intended to be reached only for
unbuffered chans. Am I mistaken here?


  Vitalie

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: core.async/close! locks on chans with pending transforms

[Timothy Baldridge]

A big reason they have to be run inside the lock is that they have to
operate in the context of the channel.

For example:

(chan (take 10))

Firstly we must recognize that transducer instances are *not* thread safe.
They should only ever be executed by one thread at a time. But channels
allow multiple puts and takes. Also channels are not processes, they are
simply rather complex lists with locks around them.

This means, if you want to execute take correctly you must ensure that only
one thread executes the take instance at one time, since channels already
operate via a channel-level lock, it makes sense to run the transducer
inside the channels' lock.

But that's also why it's always been recommended to stick with simple, non
cpu-intensive, non blocking operations in channel transducers.

Timothy

On Mon, Jul 3, 2017 at 4:30 PM, Kevin Downey  wrote:

> On 07/03/2017 03:12 PM, Vitalie Spinu wrote:
>
>>
>>
>>
>> On Monday, 3 July 2017 22:48:40 UTC+2, red...@gmail.com wrote:
>>
>>
>> Discussion of locks aside, doing blocking operations like io or >   >!! or basically anything that looks like it blocks and isn't >!
>> or > is a very bad idea in a transducer on a channel. You will (eventually)
>> block the threadpool and get deadlocks.
>>
>>
>>
>> Is this the limitation in general or only when I use the chan with
>> blocking transducer withing a go block?  Transducers are not run on a go
>> block, are they?
>>
>>
> It is kind of complicated, and I haven't traced through everything in
> channels.clj recently, but assume transducers are not magic, the
> computation has to occur on some real thread some where, and the best
> threads available to do that are the consumer thread and the producer
> thread. If those threads are both executing go blocks, and you have a
> transducer doing blocking stuff on a channel they operate on, you may not
> be technically "blocking" in the go block, but the channel machinery is now
> blocking before it yields control back to the go  block. Putting blocking
> operations in a transducer then operating on the channel from go blocks
> turns `!>` and ' blocks), in to operations that do block a whole OS thread, of which the
> core.async threadpool that go blocks execute on only has 8 by default.
>
> There is this sort of dance where control is handed back and forth between
> channels and go blocks without tying up a thread. When you introduce
> blocking in to the channel machinery (by making the channel execute a
> blocking operation as things are produced or consumed) then the dance stops
> and the thread waits.
>
> --
>> You received this message because you are subscribed to the Google
>> Groups "Clojure" group.
>> To post to this group, send email to clojure@googlegroups.com
>> Note that posts from new members are moderated - please be patient with
>> your first post.
>> To unsubscribe from this group, send email to
>> clojure+unsubscr...@googlegroups.com
>> For more options, visit this group at
>> http://groups.google.com/group/clojure?hl=en
>> ---
>> You received this message because you are subscribed to the Google Groups
>> "Clojure" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to clojure+unsubscr...@googlegroups.com > clojure+unsubscr...@googlegroups.com>.
>> For more options, visit https://groups.google.com/d/optout.
>>
>
>
> --
> And what is good, Phaedrus,
> And what is not good—
> Need we ask anyone to tell us these things?
>
> --
> You received this message because you are subscribed to the Google
> Groups "Clojure" group.
> To post to this group, send email to clojure@googlegroups.com
> Note that posts from new members are moderated - please be patient with
> your first post.
> To unsubscribe from this group, send email to
> clojure+unsubscr...@googlegroups.com
> For more options, visit this group at
> http://groups.google.com/group/clojure?hl=en
> --- You received this message because you are subscribed to the Google
> Groups "Clojure" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to clojure+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.
>



-- 
“One of the main causes of the fall of the Roman Empire was that–lacking
zero–they had no way to indicate successful termination of their C
programs.”
(Robert Firth)

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to 

Re: core.async/close! locks on chans with pending transforms

[Kevin Downey]

On 07/03/2017 03:12 PM, Vitalie Spinu wrote:




On Monday, 3 July 2017 22:48:40 UTC+2, red...@gmail.com wrote:


Discussion of locks aside, doing blocking operations like io or !! or basically anything that looks like it blocks and isn't >!
or Is this the limitation in general or only when I use the chan with 
blocking transducer withing a go block?  Transducers are not run on a go 
block, are they?




It is kind of complicated, and I haven't traced through everything in 
channels.clj recently, but assume transducers are not magic, the 
computation has to occur on some real thread some where, and the best 
threads available to do that are the consumer thread and the producer 
thread. If those threads are both executing go blocks, and you have a 
transducer doing blocking stuff on a channel they operate on, you may 
not be technically "blocking" in the go block, but the channel machinery 
is now blocking before it yields control back to the go  block. Putting 
blocking operations in a transducer then operating on the channel from 
go blocks turns `!>` and '(just park go blocks), in to operations that do block a whole OS thread, 
of which the core.async threadpool that go blocks execute on only has 8 
by default.


There is this sort of dance where control is handed back and forth 
between channels and go blocks without tying up a thread. When you 
introduce blocking in to the channel machinery (by making the channel 
execute a blocking operation as things are produced or consumed) then 
the dance stops and the thread waits.



--
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with 
your first post.

To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
---
You received this message because you are subscribed to the Google 
Groups "Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send 
an email to clojure+unsubscr...@googlegroups.com 
.

For more options, visit https://groups.google.com/d/optout.



--
And what is good, Phaedrus,
And what is not good—
Need we ask anyone to tell us these things?

--
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups "Clojure" group.

To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: core.async/close! locks on chans with pending transforms

[Vitalie Spinu]

On Monday, 3 July 2017 22:48:40 UTC+2, red...@gmail.com wrote:
>
>
> Discussion of locks aside, doing blocking operations like io or   >!! or basically anything that looks like it blocks and isn't >! or  is a very bad idea in a transducer on a channel. You will (eventually) 
> block the threadpool and get deadlocks. 
>


Is this the limitation in general or only when I use the chan with blocking 
transducer withing a go block?  Transducers are not run on a go block, are 
they?

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: core.async/close! locks on chans with pending transforms

[Vitalie Spinu]

> the side-effect of this means that no other operation (puts, takes or 
closes)

Is there a deeper reason for this beside the ease of implementation?

 If chan is buffered I still fail to see why should close and take block. 

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: core.async/close! locks on chans with pending transforms

[Kevin Downey]

On 07/03/2017 11:03 AM, Vitalie Spinu wrote:


Hi,

Async/close! causes deadlocks if its reducer is stalled (e.g. waits for 
an event from

  another chan).

Consider:

   (let [d (chan)
 s (chan 1 (map (fn [v]
  (println "this:" v)
  (println "from d:" (

Discussion of locks aside, doing blocking operations like io or >!! or basically anything that looks like it blocks and isn't >! or is a very bad idea in a transducer on a channel. You will (eventually) 
block the threadpool and get deadlocks. If you must use a transducer 
that does blocking operations, you should checkout pipeline-blocking. 
pipeline-blocking has a slightly convoluted internal structure exactly 
to avoid blocking operations running on the go block threadpool.


The transducer will be executed when publishing to a channel(this is a 
little bit squishy, sometimes publishing to a channel doesn't actually 
publish because the channel is full and a callback is registered to do 
the publishing the next time a consumer takes), if you publish to a 
channel with a transducer that blocks from a go-block, then you are 
blocking that go block which is bad.



  v)))]
 (go (>! s 1))
 (Thread/sleep 100)
 (println "closing s")
 (async/close! s))

   ;; =>
   ;; this: 1
   ;; closing s
   ;; .. [lock]

This is caused by (.lock mutex) in close! method here:


https://github.com/clojure/core.async/blob/master/src/main/clojure/clojure/core/async/impl/channels.clj#L247


I wonder if it is there "by design" or a bug. IMO it makes little sense 
to lock
external code simply because chan's reducer function is stalled. Just as 
close!

doesn't lock on pending puts it shouldn't stall on pending "half-puts".

I need this for systems with inter-dependent chans. In the above example
component 'd' is a dependency of 's', then system will halt in reverse 
order of

dependencies closing `s` first.

Thank you,

Vitalie

--
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with 
your first post.

To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
---
You received this message because you are subscribed to the Google 
Groups "Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send 
an email to clojure+unsubscr...@googlegroups.com 
.

For more options, visit https://groups.google.com/d/optout.



--
And what is good, Phaedrus,
And what is not good—
Need we ask anyone to tell us these things?

--
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups "Clojure" group.

To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: core.async/close! locks on chans with pending transforms

[Timothy Baldridge]

Transducers on channels lock the channel while they are running. This is by
design. So yes, the side-effect of this means that no other operation
(puts, takes or closes) can succeed while the transducer is running.

So the short answer is: If you have code that can take awhile to run, don't
put it in a channel transducer, put it in `async/pipeline(-blocking)`
instead.

On Mon, Jul 3, 2017 at 12:03 PM, Vitalie Spinu  wrote:

>
> Hi,
>
> Async/close! causes deadlocks if its reducer is stalled (e.g. waits for an
> event from
>  another chan).
>
> Consider:
>
>   (let [d (chan)
> s (chan 1 (map (fn [v]
>  (println "this:" v)
>  (println "from d:" (  v)))]
> (go (>! s 1))
> (Thread/sleep 100)
> (println "closing s")
> (async/close! s))
>
>   ;; =>
>   ;; this: 1
>   ;; closing s
>   ;; .. [lock]
>
> This is caused by (.lock mutex) in close! method here:
>
>https://github.com/clojure/core.async/blob/master/src/
> main/clojure/clojure/core/async/impl/channels.clj#L247
>
> I wonder if it is there "by design" or a bug. IMO it makes little sense to
> lock
> external code simply because chan's reducer function is stalled. Just as
> close!
> doesn't lock on pending puts it shouldn't stall on pending "half-puts".
>
> I need this for systems with inter-dependent chans. In the above example
> component 'd' is a dependency of 's', then system will halt in reverse
> order of
> dependencies closing `s` first.
>
> Thank you,
>
>Vitalie
>
> --
> You received this message because you are subscribed to the Google
> Groups "Clojure" group.
> To post to this group, send email to clojure@googlegroups.com
> Note that posts from new members are moderated - please be patient with
> your first post.
> To unsubscribe from this group, send email to
> clojure+unsubscr...@googlegroups.com
> For more options, visit this group at
> http://groups.google.com/group/clojure?hl=en
> ---
> You received this message because you are subscribed to the Google Groups
> "Clojure" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to clojure+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.
>



-- 
“One of the main causes of the fall of the Roman Empire was that–lacking
zero–they had no way to indicate successful termination of their C
programs.”
(Robert Firth)

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: core.async top use cases

[Gordon Syme]

I've used agents to wrap thread-unsafe mutable Java objects with a defined life 
cycle, so that they could be used from multiple threads whilst respecting the 
life cycle.

My particular case was server-side gRPC StreamObservers for long lived client 
connections.
These are either usable, closed, or errored and an exception is thrown if you 
try to send on a closed or errored observer.

Using the agent state to model the life cycle, and the natural queueing of 
agent actions gave me thread-safe fire-and-forget semantics (which was 
appropriate for the messages involved).

To be honest it was the first time I've found a good use for agents, but they 
fit really well.

-Gordon

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: core.async top use cases

[Timothy Baldridge]

Yeah, I used to do that, but once core.async came out I started to desire
the back pressure aspects of channels. I don't think I've used agents for
logging since. You always run the risk of something backing up the queue of
the agent and causing your thread to crash when it runs out of memory.

On Thu, Oct 13, 2016 at 8:04 PM, Mark Engelberg 
wrote:

> I always found it a bit ironic that my main use case for agents doesn't
> really at all make use of the "mutable ref" aspect of the agent, only the
> queue piece.  I usually hold the name of the log file in the mutable ref to
> emphasize that the agent is "guarding" this particular log file, but I
> don't actually mutate it, so the mutability doesn't really matter for this
> purpose.
>
> On Thu, Oct 13, 2016 at 7:02 PM, Mark Engelberg 
> wrote:
>
>> My primary use case for agents has always been when I want to coordinate
>> multiple threads writing to a log file.  The agent effectively serializes
>> all the write requests with a minimum of fuss.
>>
>
> --
> You received this message because you are subscribed to the Google
> Groups "Clojure" group.
> To post to this group, send email to clojure@googlegroups.com
> Note that posts from new members are moderated - please be patient with
> your first post.
> To unsubscribe from this group, send email to
> clojure+unsubscr...@googlegroups.com
> For more options, visit this group at
> http://groups.google.com/group/clojure?hl=en
> ---
> You received this message because you are subscribed to the Google Groups
> "Clojure" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to clojure+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.
>



-- 
“One of the main causes of the fall of the Roman Empire was that–lacking
zero–they had no way to indicate successful termination of their C
programs.”
(Robert Firth)

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: core.async top use cases

[Mark Engelberg]

I always found it a bit ironic that my main use case for agents doesn't
really at all make use of the "mutable ref" aspect of the agent, only the
queue piece.  I usually hold the name of the log file in the mutable ref to
emphasize that the agent is "guarding" this particular log file, but I
don't actually mutate it, so the mutability doesn't really matter for this
purpose.

On Thu, Oct 13, 2016 at 7:02 PM, Mark Engelberg 
wrote:

> My primary use case for agents has always been when I want to coordinate
> multiple threads writing to a log file.  The agent effectively serializes
> all the write requests with a minimum of fuss.
>

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: core.async top use cases

[Mark Engelberg]

My primary use case for agents has always been when I want to coordinate
multiple threads writing to a log file.  The agent effectively serializes
all the write requests with a minimum of fuss.

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: core.async top use cases

[Alex Miller]

The other other special feature of agents is that the stm knows about them so 
it's a safe way to have a side effect occur in an stm transaction (all agent 
sends are delayed till the txn succeeds). I've found that to be pretty handy in 
advanced usage.

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: core.async top use cases

[Timothy Baldridge]

Agents combine two things 1) a queue of functions, 2) mutable state. The
key thing about agents is that they still respect Clojure's concept of
"instant deref". That is to say, you can always deref an agent even if the
queue is backlogged. This is one of the key differences between agents and
actors. You have to send a message to an actor to deref it, an agent is
always deref-able.

Now it's true that you could build an agent via a core.async channel and an
atom, but there's really no need since agents already exist in the runtime.
Agents also support the lesser known features of Clojure's mutable refs,
namely, validators and watchers. But the short answer is: agents existed
before core.async. I've used them once in the past 2 years, but that's not
to say they aren't useful sometimes. And it's nice being able to have a
queue attached to a mutable reference without having to import core.async.

On Thu, Oct 13, 2016 at 7:28 PM, Timothy Baldridge 
wrote:

> >> When using clojurescript, adding async really increases the load time.
> That's one place where you might want to use agents when you can.
>
> But Clojurescript doesn't support agents.
>
> On Thu, Oct 13, 2016 at 7:16 PM, William la Forge 
> wrote:
>
>> On Thursday, October 13, 2016 at 3:38:16 PM UTC-4, larry google groups
>> wrote:
>>>
>>> So when to use agents? I've looked through Clojure repos on Github,
>>> looking for uses of agents, and I found very few. (I was writing a blog
>>> post about concurrency in Clojure, and I found that agents are among the
>>> least used tools for concurrency). I found a lot of uses of futures and
>>> promises and channels and core.async, and certainly atoms, but I didn't
>>> find many uses of agents. When are agents best used?
>>>

>
>> When using clojurescript, adding async really increases the load time.
>> That's one place where you might want to use agents when you can.
>>
>>
>> --
>> You received this message because you are subscribed to the Google
>> Groups "Clojure" group.
>> To post to this group, send email to clojure@googlegroups.com
>> Note that posts from new members are moderated - please be patient with
>> your first post.
>> To unsubscribe from this group, send email to
>> clojure+unsubscr...@googlegroups.com
>> For more options, visit this group at
>> http://groups.google.com/group/clojure?hl=en
>> ---
>> You received this message because you are subscribed to the Google Groups
>> "Clojure" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to clojure+unsubscr...@googlegroups.com.
>> For more options, visit https://groups.google.com/d/optout.
>>
>
>
>
> --
> “One of the main causes of the fall of the Roman Empire was that–lacking
> zero–they had no way to indicate successful termination of their C
> programs.”
> (Robert Firth)
>



-- 
“One of the main causes of the fall of the Roman Empire was that–lacking
zero–they had no way to indicate successful termination of their C
programs.”
(Robert Firth)

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: core.async top use cases

[Timothy Baldridge]

>> When using clojurescript, adding async really increases the load time.
That's one place where you might want to use agents when you can.

But Clojurescript doesn't support agents.

On Thu, Oct 13, 2016 at 7:16 PM, William la Forge 
wrote:

> On Thursday, October 13, 2016 at 3:38:16 PM UTC-4, larry google groups
> wrote:
>>
>> So when to use agents? I've looked through Clojure repos on Github,
>> looking for uses of agents, and I found very few. (I was writing a blog
>> post about concurrency in Clojure, and I found that agents are among the
>> least used tools for concurrency). I found a lot of uses of futures and
>> promises and channels and core.async, and certainly atoms, but I didn't
>> find many uses of agents. When are agents best used?
>>
>>>

> When using clojurescript, adding async really increases the load time.
> That's one place where you might want to use agents when you can.
>
>
> --
> You received this message because you are subscribed to the Google
> Groups "Clojure" group.
> To post to this group, send email to clojure@googlegroups.com
> Note that posts from new members are moderated - please be patient with
> your first post.
> To unsubscribe from this group, send email to
> clojure+unsubscr...@googlegroups.com
> For more options, visit this group at
> http://groups.google.com/group/clojure?hl=en
> ---
> You received this message because you are subscribed to the Google Groups
> "Clojure" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to clojure+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.
>



-- 
“One of the main causes of the fall of the Roman Empire was that–lacking
zero–they had no way to indicate successful termination of their C
programs.”
(Robert Firth)

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: core.async top use cases

[William la Forge]

On Thursday, October 13, 2016 at 3:38:16 PM UTC-4, larry google groups 
wrote:
>
> So when to use agents? I've looked through Clojure repos on Github, 
> looking for uses of agents, and I found very few. (I was writing a blog 
> post about concurrency in Clojure, and I found that agents are among the 
> least used tools for concurrency). I found a lot of uses of futures and 
> promises and channels and core.async, and certainly atoms, but I didn't 
> find many uses of agents. When are agents best used? 
>
>>
>>>
When using clojurescript, adding async really increases the load time. 
That's one place where you might want to use agents when you can.
 

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: core.async top use cases

[larry google groups]

> It is not just convenience. For example agents don't provide 
functionality like buffering, back pressure 
> and select aka alts. If you send an action to an agent you don't get to 
know when it's done or 
> to choose what to do if it is currently busy. 


So when to use agents? I've looked through Clojure repos on Github, looking 
for uses of agents, and I found very few. (I was writing a blog post about 
concurrency in Clojure, and I found that agents are among the least used 
tools for concurrency). I found a lot of uses of futures and promises and 
channels and core.async, and certainly atoms, but I didn't find many uses 
of agents. When are agents best used? 





On Monday, September 19, 2016 at 1:19:55 PM UTC-4, Leon Grapenthin wrote:
>
> It is not just convenience. For example agents don't provide functionality 
> like buffering, backpressure and select aka alts. If you send an action to 
> an agent you don't get to know when it's done or to choose what to do if it 
> is currently busy. 
>
> On Monday, September 19, 2016 at 11:49:13 AM UTC+2, Matan Safriel wrote:
>>
>> Thanks, and I put the blog post on my reading list. 
>> Although I can't avoid thinking that we already have asynchronous idioms 
>> in the core language itself, like agents. I think the crux for server-side 
>> is more about the convenient piping, rather than the mere asynchronism 
>> itself, but I might be wrong in any of this.
>>
>> On Mon, Sep 19, 2016 at 9:14 AM, Mond Ray  wrote:
>>
>>> Pushing asynchrony further into the stack is useful for reliability and 
>>> fault tolerance. We can also use it as a basis for Complex Event Processing 
>>> using time series windows. 
>>>
>>> I wrote up a few examples in my blog 
>>> 
>>>  
>>> if you have the time to check out a longer explanation with code.
>>>
>>> I recently wrote a small set of functions to enable HTML5 Server Sent 
>>> Events from any Kafka topic which also uses core.async (with an example 
>>> using Aleph and Compojure). You might like to check that repo 
>>>  out too.
>>>
>>> Ray
>>>
>>> On Sunday, 18 September 2016 08:37:38 UTC+2, Matan Safriel wrote:

 Hi,

 It's very easy to see how core.async solves callback hell for front-end 
 development with clojurescript.
 In what use cases would you use it for server-side? we already have 
 non-blocking IO from Java, and we have clojure agents. So what's a bunch 
 of 
 salient use cases?
 Are there prominent clojure http server implementations which rely on 
 it for transcending the threaded web service paradigm?

 Thanks,
 Matan

 -- 
>>> You received this message because you are subscribed to the Google
>>> Groups "Clojure" group.
>>> To post to this group, send email to clo...@googlegroups.com
>>> Note that posts from new members are moderated - please be patient with 
>>> your first post.
>>> To unsubscribe from this group, send email to
>>> clojure+u...@googlegroups.com
>>> For more options, visit this group at
>>> http://groups.google.com/group/clojure?hl=en
>>> --- 
>>> You received this message because you are subscribed to a topic in the 
>>> Google Groups "Clojure" group.
>>> To unsubscribe from this topic, visit 
>>> https://groups.google.com/d/topic/clojure/peJXvE0nBZs/unsubscribe.
>>> To unsubscribe from this group and all its topics, send an email to 
>>> clojure+u...@googlegroups.com.
>>> For more options, visit https://groups.google.com/d/optout.
>>>
>>
>>

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: core.async top use cases

[Derek Troy-West]

Fine grained control of parallelism is a superb aspect of core.async.

Cassandra is a distributed database, often a query requires you 
resolve-on-read denormalised data partitioned multiple ways (semantically, 
by time, etc). You can think of it like a grid I guess.

Lets say I have a query that I want to execute against Cassandra that 
covers a week of data, where the data is partitioned chronologically by the 
hour, and by modulo 24 to spread data around the cluster, that's going to 
require 4032 queries. Judicious use of core.async/pipeline-async allows me 
to execute those queries in parallel with granular control of the 
chronological / modulo parallelism independent of one another, meaning I 
can play with that parallelism in the REPL by just adjusting a couple of 
numbers and reviewing the results. Twiddling knobs, measuring output.

That allows me to execute those 4032 queries in parallel, sub-second, with 
non-blocking requiring none of the overhead of thread-per-request.

My services are Netty end-to-end (server and driver interface to Cassandra) 
so there are always a fixed number of threads. If I wanted to tune the 
buffers or thread-pool sizes I could probably get an even higher degree of 
throughput / lower-latency on each monolithic cassandra query, the 
constraints are known, quantifiable, and easily configurable.

On Sunday, September 18, 2016 at 6:37:38 PM UTC+12, Matan Safriel wrote:
>
> Hi,
>
> It's very easy to see how core.async solves callback hell for front-end 
> development with clojurescript.
> In what use cases would you use it for server-side? we already have 
> non-blocking IO from Java, and we have clojure agents. So what's a bunch of 
> salient use cases?
> Are there prominent clojure http server implementations which rely on it 
> for transcending the threaded web service paradigm?
>
> Thanks,
> Matan
>
>

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: core.async top use cases

[Beau Fabry]

You're probably right, I was confusing actors with agents.

On Tuesday, September 20, 2016 at 7:05:19 PM UTC-7, Matan Safriel wrote:
>
> Actually, I am not sure clojure implements the actor model, which I can 
> only construe as the Erlang actor model here. I am almost certain the core 
> language explicitly does not: http://clojure.org/about/state
>
> It can be shoehorned somehow (see okku) but I would probably not venture 
> saying clojure supports the actor model.
>
> Sent from my mobile
>
>  Original Message 
> From:Beau Fabry 
> Sent:Wed, 21 Sep 2016 03:47:29 +0300
> To:Clojure 
> Subject:Re: core.async top use cases
>
> I'm no expert on this, but the Actor model and the CSP model seem to be 
> two different ways to model a concurrent system. Clojure supports them 
> both. Personally I find the CSP model a simpler and easier to understand 
> one than Actors, and so pretty much default to it. You might find 
> non-clojure related sources comparing the tradeoffs between the two though?
>
>
> On Monday, September 19, 2016 at 11:50:53 PM UTC-7, Matan Safriel wrote:
>>
>> Thanks but I'm not entirely sure about this. I could use agents for side 
>> effects too, or at least I thought so. Care to further clarify?
>>
>>
>>  Original Message 
>> From:William la Forge 
>> Sent:Tue, 20 Sep 2016 02:37:20 +0300
>> To:Clojure 
>> Subject:Re: core.async top use cases
>>
>> The really nice thing to me is that async handles side-effects while 
>> agents do not.
>>
>> -- 
>> You received this message because you are subscribed to the Google
>> Groups "Clojure" group.
>> To post to this group, send email to clo...@googlegroups.com
>> Note that posts from new members are moderated - please be patient with 
>> your first post.
>> To unsubscribe from this group, send email to
>> clojure+u...@googlegroups.com
>> For more options, visit this group at
>> http://groups.google.com/group/clojure?hl=en
>> --- 
>> You received this message because you are subscribed to a topic in the 
>> Google Groups "Clojure" group.
>> To unsubscribe from this topic, visit 
>> https://groups.google.com/d/topic/clojure/peJXvE0nBZs/unsubscribe.
>> To unsubscribe from this group and all its topics, send an email to 
>> clojure+u...@googlegroups.com.
>> For more options, visit https://groups.google.com/d/optout.
>>
> -- 
> You received this message because you are subscribed to the Google
> Groups "Clojure" group.
> To post to this group, send email to clo...@googlegroups.com 
> Note that posts from new members are moderated - please be patient with 
> your first post.
> To unsubscribe from this group, send email to
> clojure+u...@googlegroups.com 
> For more options, visit this group at
> http://groups.google.com/group/clojure?hl=en
> --- 
> You received this message because you are subscribed to a topic in the 
> Google Groups "Clojure" group.
> To unsubscribe from this topic, visit 
> https://groups.google.com/d/topic/clojure/peJXvE0nBZs/unsubscribe.
> To unsubscribe from this group and all its topics, send an email to 
> clojure+u...@googlegroups.com .
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: core.async top use cases

[Matan Safriel]

Actually, I am not sure clojure implements the actor model, which I can only 
construe as the Erlang actor model here. I am almost certain the core language 
explicitly does not: http://clojure.org/about/state

It can be shoehorned somehow (see okku) but I would probably not venture saying 
clojure supports the actor model.

Sent from my mobile

 Original Message 
From:Beau Fabry 
Sent:Wed, 21 Sep 2016 03:47:29 +0300
To:Clojure 
Subject:Re: core.async top use cases

>I'm no expert on this, but the Actor model and the CSP model seem to be two 
>different ways to model a concurrent system. Clojure supports them both. 
>Personally I find the CSP model a simpler and easier to understand one than 
>Actors, and so pretty much default to it. You might find non-clojure related 
>sources comparing the tradeoffs between the two though?
>
>
>
>On Monday, September 19, 2016 at 11:50:53 PM UTC-7, Matan Safriel wrote:
>
>Thanks but I'm not entirely sure about this. I could use agents for side 
>effects too, or at least I thought so. Care to further clarify?
>
>
> Original Message 
>From:William la Forge 
>Sent:Tue, 20 Sep 2016 02:37:20 +0300
>To:Clojure 
>Subject:Re: core.async top use cases
>
>The really nice thing to me is that async handles side-effects while agents do 
>not.
>
>-- 
>You received this message because you are subscribed to the Google
>Groups "Clojure" group.
>To post to this group, send email to clo...@googlegroups.com
>Note that posts from new members are moderated - please be patient with your 
>first post.
>To unsubscribe from this group, send email to
>clojure+u...@googlegroups.com
>For more options, visit this group at
>http://groups.google.com/group/clojure?hl=en
>--- 
>You received this message because you are subscribed to a topic in the Google 
>Groups "Clojure" group.
>To unsubscribe from this topic, visit 
>https://groups.google.com/d/topic/clojure/peJXvE0nBZs/unsubscribe.
>To unsubscribe from this group and all its topics, send an email to 
>clojure+u...@googlegroups.com.
>For more options, visit https://groups.google.com/d/optout.
>
>-- 
>You received this message because you are subscribed to the Google
>Groups "Clojure" group.
>To post to this group, send email to clojure@googlegroups.com
>Note that posts from new members are moderated - please be patient with your 
>first post.
>To unsubscribe from this group, send email to
>clojure+unsubscr...@googlegroups.com
>For more options, visit this group at
>http://groups.google.com/group/clojure?hl=en
>--- 
>You received this message because you are subscribed to a topic in the Google 
>Groups "Clojure" group.
>To unsubscribe from this topic, visit 
>https://groups.google.com/d/topic/clojure/peJXvE0nBZs/unsubscribe.
>To unsubscribe from this group and all its topics, send an email to 
>clojure+unsubscr...@googlegroups.com.
>For more options, visit https://groups.google.com/d/optout.

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: core.async top use cases

[Beau Fabry]

I'm no expert on this, but the Actor model and the CSP model seem to be two 
different ways to model a concurrent system. Clojure supports them both. 
Personally I find the CSP model a simpler and easier to understand one than 
Actors, and so pretty much default to it. You might find non-clojure 
related sources comparing the tradeoffs between the two though?


On Monday, September 19, 2016 at 11:50:53 PM UTC-7, Matan Safriel wrote:
>
> Thanks but I'm not entirely sure about this. I could use agents for side 
> effects too, or at least I thought so. Care to further clarify?
>
>
>  Original Message 
> From:William la Forge 
> Sent:Tue, 20 Sep 2016 02:37:20 +0300
> To:Clojure 
> Subject:Re: core.async top use cases
>
> The really nice thing to me is that async handles side-effects while 
> agents do not.
>
> -- 
> You received this message because you are subscribed to the Google
> Groups "Clojure" group.
> To post to this group, send email to clo...@googlegroups.com 
> Note that posts from new members are moderated - please be patient with 
> your first post.
> To unsubscribe from this group, send email to
> clojure+u...@googlegroups.com 
> For more options, visit this group at
> http://groups.google.com/group/clojure?hl=en
> --- 
> You received this message because you are subscribed to a topic in the 
> Google Groups "Clojure" group.
> To unsubscribe from this topic, visit 
> https://groups.google.com/d/topic/clojure/peJXvE0nBZs/unsubscribe.
> To unsubscribe from this group and all its topics, send an email to 
> clojure+u...@googlegroups.com .
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: core.async top use cases

[William la Forge]

My bad. I was thinking of atomic. Swap! doesn't work with side effects, but 
send does.

On Tuesday, September 20, 2016 at 2:50:53 AM UTC-4, Matan Safriel wrote:
>
> Thanks but I'm not entirely sure about this. I could use agents for side 
> effects too, or at least I thought so. Care to further clarify?
>
>
>  Original Message 
> From:William la Forge 
> Sent:Tue, 20 Sep 2016 02:37:20 +0300
> To:Clojure 
> Subject:Re: core.async top use cases
>
> The really nice thing to me is that async handles side-effects while 
> agents do not.
>
> -- 
> You received this message because you are subscribed to the Google
> Groups "Clojure" group.
> To post to this group, send email to clo...@googlegroups.com 
> Note that posts from new members are moderated - please be patient with 
> your first post.
> To unsubscribe from this group, send email to
> clojure+u...@googlegroups.com 
> For more options, visit this group at
> http://groups.google.com/group/clojure?hl=en
> --- 
> You received this message because you are subscribed to a topic in the 
> Google Groups "Clojure" group.
> To unsubscribe from this topic, visit 
> https://groups.google.com/d/topic/clojure/peJXvE0nBZs/unsubscribe.
> To unsubscribe from this group and all its topics, send an email to 
> clojure+u...@googlegroups.com .
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: core.async top use cases

[Matan Safriel]

Thanks but I'm not entirely sure about this. I could use agents for side 
effects too, or at least I thought so. Care to further clarify?


 Original Message 
From:William la Forge 
Sent:Tue, 20 Sep 2016 02:37:20 +0300
To:Clojure 
Subject:Re: core.async top use cases

>The really nice thing to me is that async handles side-effects while agents do 
>not.
>
>-- 
>You received this message because you are subscribed to the Google
>Groups "Clojure" group.
>To post to this group, send email to clojure@googlegroups.com
>Note that posts from new members are moderated - please be patient with your 
>first post.
>To unsubscribe from this group, send email to
>clojure+unsubscr...@googlegroups.com
>For more options, visit this group at
>http://groups.google.com/group/clojure?hl=en
>--- 
>You received this message because you are subscribed to a topic in the Google 
>Groups "Clojure" group.
>To unsubscribe from this topic, visit 
>https://groups.google.com/d/topic/clojure/peJXvE0nBZs/unsubscribe.
>To unsubscribe from this group and all its topics, send an email to 
>clojure+unsubscr...@googlegroups.com.
>For more options, visit https://groups.google.com/d/optout.

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: core.async top use cases

[Ken Restivo]

On Sat, Sep 17, 2016 at 11:37:38PM -0700, Matan Safriel wrote:
> Hi,
> 
> It's very easy to see how core.async solves callback hell for front-end 
> development with clojurescript.
> In what use cases would you use it for server-side? we already have 
> non-blocking IO from Java, and we have clojure agents. So what's a bunch of 
> salient use cases?
> Are there prominent clojure http server implementations which rely on it 
> for transcending the threaded web service paradigm?
> 

Decoupling components within a system, similar to what one would use queues or 
a messaging architecture for.

Used it extensively in two projects (one open source, one for a client) some 
years ago.

Rather than coupling the components via namespace dependencies and a call API, 
I just had the components passing data via async channels (queues), including 
pub/sub and topic subscription.

Worked great for its intended purpose.

-ken

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: core.async top use cases

[William la Forge]

The really nice thing to me is that async handles side-effects while agents 
do not.

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: core.async top use cases

[Matan Safriel]

Right!

 Original Message 
From:Leon Grapenthin 
Sent:Mon, 19 Sep 2016 20:19:55 +0300
To:Clojure 
Subject:Re: core.async top use cases

>It is not just convenience. For example agents don't provide functionality 
>like buffering, backpressure and select aka alts. If you send an action to an 
>agent you don't get to know when it's done or to choose what to do if it is 
>currently busy. 
>
>On Monday, September 19, 2016 at 11:49:13 AM UTC+2, Matan Safriel wrote:
>
>Thanks, and I put the blog post on my reading list. 
>
>Although I can't avoid thinking that we already have asynchronous idioms in 
>the core language itself, like agents. I think the crux for server-side is 
>more about the convenient piping, rather than the mere asynchronism itself, 
>but I might be wrong in any of this.
>
>
>On Mon, Sep 19, 2016 at 9:14 AM, Mond Ray  wrote:
>
>Pushing asynchrony further into the stack is useful for reliability and fault 
>tolerance. We can also use it as a basis for Complex Event Processing using 
>time series windows. 
>
>
>I wrote up a few examples in my blog if you have the time to check out a 
>longer explanation with code.
>
>
>I recently wrote a small set of functions to enable HTML5 Server Sent Events 
>from any Kafka topic which also uses core.async (with an example using Aleph 
>and Compojure). You might like to check that repo out too.
>
>
>Ray
>
>
>On Sunday, 18 September 2016 08:37:38 UTC+2, Matan Safriel wrote:
>
>Hi,
>
>
>It's very easy to see how core.async solves callback hell for front-end 
>development with clojurescript.
>
>In what use cases would you use it for server-side? we already have 
>non-blocking IO from Java, and we have clojure agents. So what's a bunch of 
>salient use cases?
>
>Are there prominent clojure http server implementations which rely on it for 
>transcending the threaded web service paradigm?
>
>
>Thanks,
>
>Matan
>
>
>-- 
>You received this message because you are subscribed to the Google
>Groups "Clojure" group.
>To post to this group, send email to clo...@googlegroups.com
>Note that posts from new members are moderated - please be patient with your 
>first post.
>To unsubscribe from this group, send email to
>clojure+u...@googlegroups.com
>For more options, visit this group at
>http://groups.google.com/group/clojure?hl=en
>--- 
>You received this message because you are subscribed to a topic in the Google 
>Groups "Clojure" group.
>To unsubscribe from this topic, visit 
>https://groups.google.com/d/topic/clojure/peJXvE0nBZs/unsubscribe.
>To unsubscribe from this group and all its topics, send an email to 
>clojure+u...@googlegroups.com.
>For more options, visit https://groups.google.com/d/optout.
>
>
>-- 
>You received this message because you are subscribed to the Google
>Groups "Clojure" group.
>To post to this group, send email to clojure@googlegroups.com
>Note that posts from new members are moderated - please be patient with your 
>first post.
>To unsubscribe from this group, send email to
>clojure+unsubscr...@googlegroups.com
>For more options, visit this group at
>http://groups.google.com/group/clojure?hl=en
>--- 
>You received this message because you are subscribed to a topic in the Google 
>Groups "Clojure" group.
>To unsubscribe from this topic, visit 
>https://groups.google.com/d/topic/clojure/peJXvE0nBZs/unsubscribe.
>To unsubscribe from this group and all its topics, send an email to 
>clojure+unsubscr...@googlegroups.com.
>For more options, visit https://groups.google.com/d/optout.

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: core.async top use cases

[Leon Grapenthin]

It is not just convenience. For example agents don't provide functionality 
like buffering, backpressure and select aka alts. If you send an action to 
an agent you don't get to know when it's done or to choose what to do if it 
is currently busy. 

On Monday, September 19, 2016 at 11:49:13 AM UTC+2, Matan Safriel wrote:
>
> Thanks, and I put the blog post on my reading list. 
> Although I can't avoid thinking that we already have asynchronous idioms 
> in the core language itself, like agents. I think the crux for server-side 
> is more about the convenient piping, rather than the mere asynchronism 
> itself, but I might be wrong in any of this.
>
> On Mon, Sep 19, 2016 at 9:14 AM, Mond Ray  > wrote:
>
>> Pushing asynchrony further into the stack is useful for reliability and 
>> fault tolerance. We can also use it as a basis for Complex Event Processing 
>> using time series windows. 
>>
>> I wrote up a few examples in my blog 
>> 
>>  
>> if you have the time to check out a longer explanation with code.
>>
>> I recently wrote a small set of functions to enable HTML5 Server Sent 
>> Events from any Kafka topic which also uses core.async (with an example 
>> using Aleph and Compojure). You might like to check that repo 
>>  out too.
>>
>> Ray
>>
>> On Sunday, 18 September 2016 08:37:38 UTC+2, Matan Safriel wrote:
>>>
>>> Hi,
>>>
>>> It's very easy to see how core.async solves callback hell for front-end 
>>> development with clojurescript.
>>> In what use cases would you use it for server-side? we already have 
>>> non-blocking IO from Java, and we have clojure agents. So what's a bunch of 
>>> salient use cases?
>>> Are there prominent clojure http server implementations which rely on it 
>>> for transcending the threaded web service paradigm?
>>>
>>> Thanks,
>>> Matan
>>>
>>> -- 
>> You received this message because you are subscribed to the Google
>> Groups "Clojure" group.
>> To post to this group, send email to clo...@googlegroups.com 
>> 
>> Note that posts from new members are moderated - please be patient with 
>> your first post.
>> To unsubscribe from this group, send email to
>> clojure+u...@googlegroups.com 
>> For more options, visit this group at
>> http://groups.google.com/group/clojure?hl=en
>> --- 
>> You received this message because you are subscribed to a topic in the 
>> Google Groups "Clojure" group.
>> To unsubscribe from this topic, visit 
>> https://groups.google.com/d/topic/clojure/peJXvE0nBZs/unsubscribe.
>> To unsubscribe from this group and all its topics, send an email to 
>> clojure+u...@googlegroups.com .
>> For more options, visit https://groups.google.com/d/optout.
>>
>
>

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: core.async top use cases

[Matan Safriel]

Thanks, and I put the blog post on my reading list.
Although I can't avoid thinking that we already have asynchronous idioms in
the core language itself, like agents. I think the crux for server-side is
more about the convenient piping, rather than the mere asynchronism itself,
but I might be wrong in any of this.

On Mon, Sep 19, 2016 at 9:14 AM, Mond Ray  wrote:

> Pushing asynchrony further into the stack is useful for reliability and
> fault tolerance. We can also use it as a basis for Complex Event Processing
> using time series windows.
>
> I wrote up a few examples in my blog
> 
> if you have the time to check out a longer explanation with code.
>
> I recently wrote a small set of functions to enable HTML5 Server Sent
> Events from any Kafka topic which also uses core.async (with an example
> using Aleph and Compojure). You might like to check that repo
>  out too.
>
> Ray
>
> On Sunday, 18 September 2016 08:37:38 UTC+2, Matan Safriel wrote:
>>
>> Hi,
>>
>> It's very easy to see how core.async solves callback hell for front-end
>> development with clojurescript.
>> In what use cases would you use it for server-side? we already have
>> non-blocking IO from Java, and we have clojure agents. So what's a bunch of
>> salient use cases?
>> Are there prominent clojure http server implementations which rely on it
>> for transcending the threaded web service paradigm?
>>
>> Thanks,
>> Matan
>>
>> --
> You received this message because you are subscribed to the Google
> Groups "Clojure" group.
> To post to this group, send email to clojure@googlegroups.com
> Note that posts from new members are moderated - please be patient with
> your first post.
> To unsubscribe from this group, send email to
> clojure+unsubscr...@googlegroups.com
> For more options, visit this group at
> http://groups.google.com/group/clojure?hl=en
> ---
> You received this message because you are subscribed to a topic in the
> Google Groups "Clojure" group.
> To unsubscribe from this topic, visit https://groups.google.com/d/
> topic/clojure/peJXvE0nBZs/unsubscribe.
> To unsubscribe from this group and all its topics, send an email to
> clojure+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: core.async top use cases

[Mond Ray]

Pushing asynchrony further into the stack is useful for reliability and 
fault tolerance. We can also use it as a basis for Complex Event Processing 
using time series windows. 

I wrote up a few examples in my blog 

 
if you have the time to check out a longer explanation with code.

I recently wrote a small set of functions to enable HTML5 Server Sent 
Events from any Kafka topic which also uses core.async (with an example 
using Aleph and Compojure). You might like to check that repo 
 out too.

Ray

On Sunday, 18 September 2016 08:37:38 UTC+2, Matan Safriel wrote:
>
> Hi,
>
> It's very easy to see how core.async solves callback hell for front-end 
> development with clojurescript.
> In what use cases would you use it for server-side? we already have 
> non-blocking IO from Java, and we have clojure agents. So what's a bunch of 
> salient use cases?
> Are there prominent clojure http server implementations which rely on it 
> for transcending the threaded web service paradigm?
>
> Thanks,
> Matan
>
>

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: core.async top use cases

[Rangel Spasov]

http://aleph.io/aleph/literate.html

"Alternately, we can use a core.async 
 goroutine to create our response, 
and convert the channel it returns using manifold.deferred/->source, and 
then take the first message from it. This is entirely equivalent to the 
previous implementation."

"Returns a streamed HTTP response, consisting of newline-delimited numbers 
every 100 milliseconds. While this would typically be represented by a lazy 
sequence, instead we use a Manifold stream. Similar to the use of the 
deferred above, *this means we don't need to allocate a thread per-request.*
"

I have tried the examples, it all works. Haven't done any benchmarks 
against using thread per-request though - you should think if your use-case 
can really benefit from this approach.

Rangel

On Saturday, September 17, 2016 at 11:37:38 PM UTC-7, Matan Safriel wrote:
>
> Hi,
>
> It's very easy to see how core.async solves callback hell for front-end 
> development with clojurescript.
> In what use cases would you use it for server-side? we already have 
> non-blocking IO from Java, and we have clojure agents. So what's a bunch of 
> salient use cases?
> Are there prominent clojure http server implementations which rely on it 
> for transcending the threaded web service paradigm?
>
> Thanks,
> Matan
>
>

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: core.async and channel flushing

[Jeremy Vuillermet]

I'm also wondering how this could be done.
I have a similar use case

On Tuesday, October 22, 2013 at 9:30:08 AM UTC+2, Alexander L. wrote:
>
> Sorry for bringing this back up, but I was wondering if anyone figured out 
> something better...
>
> On Saturday, September 14, 2013 10:49:08 PM UTC+3, Alexander L. wrote:
>>
>>
>> I am developing an application and I use core.async to push data from 
>> multiple threads within an infinite 
>>
>> (go (while true 
>>>   (let [data (>> (do processing here...)))
>>
>>
>> in order to be processed. As you probably already figured out, I store my 
>> channel inside an atom defined at the start of my app.
>>
>> If an exception occurs in one of the threads, then I am calling a global 
>> exception handler which will reset the atom that holds the channel in order 
>> to flush it and make sure the processing of the data it contains will stop.
>>
>> Now, my question is, does anyone know a better way to flush a core.async 
>> channel or the only way is to re-create the channel?
>>
>> Thank you for your time.
>>
>> Regards,
>>
>> Alexander
>>
>

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Core.async performance with many channels

[JvJ]

It seems to do rather well.

On Wednesday, 6 April 2016 08:29:07 UTC-7, Francis Avila wrote:
>
> On Monday, April 4, 2016 at 6:30:07 PM UTC-5, Howard M. Lewis Ship wrote:
>>
>> David Nolen had an early ClojureScript core.async demo with thousands of 
>> channels, controlling individual pixels.
>>
>
> This is the demo you are referring to: 
> http://swannodette.github.io/2013/08/02/10-processes/
>  
>

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Core.async performance with many channels

[Francis Avila]

On Monday, April 4, 2016 at 6:30:07 PM UTC-5, Howard M. Lewis Ship wrote:
>
> David Nolen had an early ClojureScript core.async demo with thousands of 
> channels, controlling individual pixels.
>

This is the demo you are referring to: 
http://swannodette.github.io/2013/08/02/10-processes/
 

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Core.async performance with many channels

[Howard Lewis Ship]

You should perhaps look at github.com/walmartlabs/active-status if you want
a way to figure out what all those processes are doing.

I'd say the things we've worked on have had dozens of channels, and often
common core.async primitives (such as into, pipe, etc.) create additional
channels and CSPs behind the scenes.

David Nolen had an early ClojureScript core.async demo with thousands of
channels, controlling individual pixels.

On Sun, Apr 3, 2016 at 11:46 PM, JvJ  wrote:

> This thing just an idea at this point.  Basically, your typical game loop
> will consist of iterating over a collection of objects and calling some
> kind of update operation on each.  I would like to replace this with
> asynchronous signaling.  Signaling could include messages like "update this
> object".  These messages would trigger certain async processes to start in
> the object.  This may allow for more flexible and asynchronous programming,
> since not everything has to fit into an update cycle.
>
>
> On Sunday, 3 April 2016 21:10:16 UTC-7, Rangel Spasov wrote:
>>
>> Without knowing too much about the internals, but having used
>> core.async/channels a lot, I don't think "hundreds" of channels will be a
>> problem ever. However, as always the devil is in the details. People might
>> be able to give better feedback if you give more details about your use
>> case.
>>
>> On Saturday, April 2, 2016 at 7:59:50 PM UTC-7, JvJ wrote:
>>>
>>>
>>> Lately, I've been working on a game in Clojure, and I've been trying out
>>> various ways of modelling the game state and objects.
>>>
>>> One of these ideas is to give each object its own channel and make a
>>> fully asynchronous architecture.
>>>
>>> I would like to know if having potentially hundreds of channels
>>> operating at once would be a significant performance issue.
>>>
>>> Thanks.
>>>
>> --
> You received this message because you are subscribed to the Google
> Groups "Clojure" group.
> To post to this group, send email to clojure@googlegroups.com
> Note that posts from new members are moderated - please be patient with
> your first post.
> To unsubscribe from this group, send email to
> clojure+unsubscr...@googlegroups.com
> For more options, visit this group at
> http://groups.google.com/group/clojure?hl=en
> ---
> You received this message because you are subscribed to the Google Groups
> "Clojure" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to clojure+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.
>



-- 
Howard M. Lewis Ship

Senior Mobile Developer at Walmart Labs

Creator of Apache Tapestry

(971) 678-5210
http://howardlewisship.com
@hlship

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Core.async performance with many channels

[JvJ]

This thing just an idea at this point.  Basically, your typical game loop 
will consist of iterating over a collection of objects and calling some 
kind of update operation on each.  I would like to replace this with 
asynchronous signaling.  Signaling could include messages like "update this 
object".  These messages would trigger certain async processes to start in 
the object.  This may allow for more flexible and asynchronous programming, 
since not everything has to fit into an update cycle.

On Sunday, 3 April 2016 21:10:16 UTC-7, Rangel Spasov wrote:
>
> Without knowing too much about the internals, but having used 
> core.async/channels a lot, I don't think "hundreds" of channels will be a 
> problem ever. However, as always the devil is in the details. People might 
> be able to give better feedback if you give more details about your use 
> case. 
>
> On Saturday, April 2, 2016 at 7:59:50 PM UTC-7, JvJ wrote:
>>
>>
>> Lately, I've been working on a game in Clojure, and I've been trying out 
>> various ways of modelling the game state and objects.
>>
>> One of these ideas is to give each object its own channel and make a 
>> fully asynchronous architecture.
>>
>> I would like to know if having potentially hundreds of channels operating 
>> at once would be a significant performance issue.
>>
>> Thanks.
>>
>

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Core.async performance with many channels

[Rangel Spasov]

Without knowing too much about the internals, but having used 
core.async/channels a lot, I don't think "hundreds" of channels will be a 
problem ever. However, as always the devil is in the details. People might 
be able to give better feedback if you give more details about your use 
case. 

On Saturday, April 2, 2016 at 7:59:50 PM UTC-7, JvJ wrote:
>
>
> Lately, I've been working on a game in Clojure, and I've been trying out 
> various ways of modelling the game state and objects.
>
> One of these ideas is to give each object its own channel and make a fully 
> asynchronous architecture.
>
> I would like to know if having potentially hundreds of channels operating 
> at once would be a significant performance issue.
>
> Thanks.
>

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: core.async mult "damaged" on reload

[Terje Dahl]

Forgive me for taking so long to reply to your very informative feedback.
Only this evening have I had time to experiment and study. These are my 
findings so far:

'defprotocol' is a macro with side-effects, so any attempt at holding on to 
a copy and then re-inserting it into the var in the other namespace are 
futile.
Even worse, 'reify' is also a macro with side-effects.  
The side-effects seem to be that they emit (byte-)code, and then one builds 
on the current instance of the others emitted code.
A fun hack would have been to insert a reader-macro which would prevent 
specific macros to run a second time by simply removing them at read-time.  
But alas, Clojure does not (yet) support insertion of reader-macros.  

But, it seems that if your call the instance-method stead of the 
protocol-method by the same name on your defonce-ed object, then everything 
works fine:
(.protocol-function object)  ;; do
(protocol-function object)  ;; don't.



However, since these calls are often wrapped in a second layer of 
functions, then either calling the protocol-method directly (as an 
instance-emthod), using your own copy of the calling function will have to 
be the work-around.  In the case of core.async/mult:
(.untap-all* saved-mult-instance)  ;; do
(defn my-untap-all [mult] (.untap-all* mult)) (my-untap-all* saved-mult-
instance)  ;; do
(untap-all saved-mult-instance)  ;; don't





On Saturday, November 14, 2015 at 11:01:20 PM UTC+1, James Elliott wrote:
>
> If I understand what you’re saying, then it sounds like you are reloading 
> a protocol definition, which creates a new protocol with the same name as 
> the old protocol, but they are different things. Old objects which 
> implement the old protocol do not implement the new protocol, even though 
> it looks identical and has the same name. This kind of issue has bit me 
> when working in iterative development on files, so I tend to protect my 
> protocol definitions inside defonce forms, so they do not get redefined 
> when I reload the file. It looks awkward, but helps. The example which 
> inspired me to do this was in Overtone:
>
> https://github.com/overtone/overtone/blob/master/src/overtone/music/rhythm.clj#L7-L28
>
>   -James
>
> On Friday, November 13, 2015 at 2:50:10 PM UTC-6, Terje Dahl wrote:
>>
>> I put a core.async/mult instance in a map structure in a map.
>> When i do load-file ... (from La Clojure in IntelliJ) the structure with 
>> the object instance is still there, but it isn't possible to use.
>> I get:
>>
>>> CompilerException java.lang.IllegalArgumentException: No implementation 
>>> of method: :tap* of protocol: #'clojure.core.async/Mult found for class: 
>>> clojure.core.async$mult$reify__7262
>>>
>>
>> Is this something to do with core.async itself, or perhaps a more general 
>> issue with protocol?
>> Any thoughts? 
>>
>

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: core.async channel /w transducer which thread?

[Herwig Hochleitner]

2015-12-16 16:22 GMT+01:00 Leon Grapenthin :

> The blocking put is made on a separate thread (channel named res), then
> later a blocking take from that same channel is made in the second go-loop.
>

> Or are you saying "if it takes too long, parallelize via pipeline"? In my
> case I can't because the transducer is stateful. I can run multiple
> pipelines with multiple transducer instances if I split up the inputs
> manually and later merge the outputs. But since the transducer does all of
> the relevant work, I need to determine where it runs.
>

> The only alternative I see is to re-implement the transducer process
> within a thread/loop body and run multiple of those.
>

If you don't care about message ordering, you don't need pipline. I suppose
you could solve the "where does it run" problem by using a ThreadLocal, but
that feels pretty dirty. Running parallel loops taking from the same source
channel and putting into the same target channel, is pretty easy to
implement by hand. I have some code doing batch-indexing, where I used this
paradigm, if you're interested.

I am not familiar with the JVM locking mechanism of core.async so what
> should I be worried about when the "transducer is executed inside the
> channel lock"? Are other channels going to have to wait if it is too slow?
>

The channel lock only affects put! and take! on a channel, as well as
transducers attached to that channel, so no need to worry about this for
your regular go-blocks.

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: core.async channel /w transducer which thread?

[Leon Grapenthin]

Pipeline must have a way of controlling on which thread the execution 
happens, otherwise it would parallize very little.

It does so by creating a new channel for each put with a buffer of 1 and an 
xf. It makes a blocking >!! and only after it unblocks it puts that new 
channel to the consumer that takes the value from there.

So at least in this limited case where there is space in the buffer and 
there is no pending take - Is it safe to assume that the xf is applied 
immediately?

On Thursday, December 17, 2015 at 4:28:27 PM UTC+1, Herwig Hochleitner 
wrote:
>
> 2015-12-16 16:22 GMT+01:00 Leon Grapenthin  >:
>
>> The blocking put is made on a separate thread (channel named res), then 
>> later a blocking take from that same channel is made in the second go-loop.
>>
>
>> Or are you saying "if it takes too long, parallelize via pipeline"? In my 
>> case I can't because the transducer is stateful. I can run multiple 
>> pipelines with multiple transducer instances if I split up the inputs 
>> manually and later merge the outputs. But since the transducer does all of 
>> the relevant work, I need to determine where it runs. 
>>
>
>> The only alternative I see is to re-implement the transducer process 
>> within a thread/loop body and run multiple of those.
>>
>
> If you don't care about message ordering, you don't need pipline. I 
> suppose you could solve the "where does it run" problem by using a 
> ThreadLocal, but that feels pretty dirty. Running parallel loops taking 
> from the same source channel and putting into the same target channel, is 
> pretty easy to implement by hand. I have some code doing batch-indexing, 
> where I used this paradigm, if you're interested.
>
> I am not familiar with the JVM locking mechanism of core.async so what 
>> should I be worried about when the "transducer is executed inside the 
>> channel lock"? Are other channels going to have to wait if it is too slow?
>>
>
> The channel lock only affects put! and take! on a channel, as well as 
> transducers attached to that channel, so no need to worry about this for 
> your regular go-blocks.
>
>

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: core.async channel /w transducer which thread?

[Timothy Baldridge]

When the transducer code was originally added to core.async I asked Rich
this very question. He declined to specify where the transducer is run. The
reasoning is simple: since the transducer is executed inside the channel
lock, your transducers should always be fast enough that you don't care
where they are run. If the transducer is going to take long enough that you
care about where it executes, you should use async/pipeline instead.

Timothy

On Wed, Dec 16, 2015 at 4:16 AM, Leon Grapenthin 
wrote:

> Is it safe to say that a transducer in a channel always executes on the
> producer thread?
>
> Kind regards,
> Leon.
>
> --
> You received this message because you are subscribed to the Google
> Groups "Clojure" group.
> To post to this group, send email to clojure@googlegroups.com
> Note that posts from new members are moderated - please be patient with
> your first post.
> To unsubscribe from this group, send email to
> clojure+unsubscr...@googlegroups.com
> For more options, visit this group at
> http://groups.google.com/group/clojure?hl=en
> ---
> You received this message because you are subscribed to the Google Groups
> "Clojure" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to clojure+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.
>



-- 
“One of the main causes of the fall of the Roman Empire was that–lacking
zero–they had no way to indicate successful termination of their C
programs.”
(Robert Firth)

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: core.async channel /w transducer which thread?

[Leon Grapenthin]

Thanks for your reply. I just studied the pipeline code and am now 
wondering how it can affect on which thread the transducer runs if the 
answer to the question is unspecified.

The blocking put is made on a separate thread (channel named res), then 
later a blocking take from that same channel is made in the second go-loop.

Or are you saying "if it takes too long, parallelize via pipeline"? In my 
case I can't because the transducer is stateful. I can run multiple 
pipelines with multiple transducer instances if I split up the inputs 
manually and later merge the outputs. But since the transducer does all of 
the relevant work, I need to determine where it runs.

The only alternative I see is to re-implement the transducer process within 
a thread/loop body and run multiple of those.

I am not familiar with the JVM locking mechanism of core.async so what 
should I be worried about when the "transducer is executed inside the 
channel lock"? Are other channels going to have to wait if it is too slow?




On Wednesday, December 16, 2015 at 2:06:24 PM UTC+1, tbc++ wrote:
>
> When the transducer code was originally added to core.async I asked Rich 
> this very question. He declined to specify where the transducer is run. The 
> reasoning is simple: since the transducer is executed inside the channel 
> lock, your transducers should always be fast enough that you don't care 
> where they are run. If the transducer is going to take long enough that you 
> care about where it executes, you should use async/pipeline instead. 
>
> Timothy
>
> On Wed, Dec 16, 2015 at 4:16 AM, Leon Grapenthin  > wrote:
>
>> Is it safe to say that a transducer in a channel always executes on the 
>> producer thread?
>>
>> Kind regards, 
>> Leon.
>>
>> -- 
>> You received this message because you are subscribed to the Google
>> Groups "Clojure" group.
>> To post to this group, send email to clo...@googlegroups.com 
>> 
>> Note that posts from new members are moderated - please be patient with 
>> your first post.
>> To unsubscribe from this group, send email to
>> clojure+u...@googlegroups.com 
>> For more options, visit this group at
>> http://groups.google.com/group/clojure?hl=en
>> --- 
>> You received this message because you are subscribed to the Google Groups 
>> "Clojure" group.
>> To unsubscribe from this group and stop receiving emails from it, send an 
>> email to clojure+u...@googlegroups.com .
>> For more options, visit https://groups.google.com/d/optout.
>>
>
>
>
> -- 
> “One of the main causes of the fall of the Roman Empire was that–lacking 
> zero–they had no way to indicate successful termination of their C 
> programs.”
> (Robert Firth) 
>

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: core.async mult "damaged" on reload

[James Elliott]

If I understand what you’re saying, then it sounds like you are reloading a 
protocol definition, which creates a new protocol with the same name as the 
old protocol, but they are different things. Old objects which implement 
the old protocol do not implement the new protocol, even though it looks 
identical and has the same name. This kind of issue has bit me when working 
in iterative development on files, so I tend to protect my protocol 
definitions inside defonce forms, so they do not get redefined when I 
reload the file. It looks awkward, but helps. The example which inspired me 
to do this was in Overtone:
https://github.com/overtone/overtone/blob/master/src/overtone/music/rhythm.clj#L7-L28

  -James

On Friday, November 13, 2015 at 2:50:10 PM UTC-6, Terje Dahl wrote:
>
> I put a core.async/mult instance in a map structure in a map.
> When i do load-file ... (from La Clojure in IntelliJ) the structure with 
> the object instance is still there, but it isn't possible to use.
> I get:
>
>> CompilerException java.lang.IllegalArgumentException: No implementation 
>> of method: :tap* of protocol: #'clojure.core.async/Mult found for class: 
>> clojure.core.async$mult$reify__7262
>>
>
> Is this something to do with core.async itself, or perhaps a more general 
> issue with protocol?
> Any thoughts? 
>

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: core.async multi-channel?

[Alex Miller]

You could use mult or pub (bit different capabilities depending on your 
needs).

http://clojure.github.io/core.async/#clojure.core.async/mult
http://clojure.github.io/core.async/#clojure.core.async/pub


On Tuesday, October 27, 2015 at 3:56:32 PM UTC-5, JvJ wrote:
>
> Is it possible to create a core.async channel that has one input and 
> simultaneously delivers the same message to multiple endpoints?
>

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: core.async: implementing pi.go

[Divyansh Prakash]

 does maya automatically handle operator precedence?


maya always evals from left to right. 

(maya 1 + 5 :as six, six * 2 :as twelve, twelve / 3 * 2) ;= 8


- Divyansh 

-- 
You received this message because you are subscribed to the Google
Groups Clojure group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
Clojure group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: core.async: implementing pi.go

[Mark Engelberg]

On Sun, Aug 2, 2015 at 4:38 AM, Divyansh Prakash 
divyanshprakas...@gmail.com wrote:

 I have one more question, though: how does one work in ClojureScript
 without !! ?



This use case is a little weird because the !! is being done to block the
function until the reduction is complete, thus making pi appear externally
like a pure function.

In Clojurescript, you really don't want to block your main thread to read a
channel, because browsers are single-threaded and this will lock up your
whole page.  So in Clojurescript, you'd probably only use this channel
architecture if you wanted to compute pi, and then when it is ready, render
the answer to the webpage.  In that case, you'd just do:

(def browser-state-atom (atom {}))

(defn pi [n]
  (let [ch (to-chan (map term (range (inc n]
(go (swap! browser-state-atom assoc :pi (! (async/reduce + 0.0 ch))

and then use something like reagent to detect the change in the :pi portion
of the atom and render the result to the page.  Or you could get clever,
and render the results as they are being added up:
(defn pi [n]
  (let [ch (to-chan (map term (range (inc n]
(go-loop []
 (when-let [v (! ch)]
   (swap! browser-state-atom update :pi (fnil (partial + v)
0.0))
   (recur)

If, for some reason, you really wanted to sit and spin in a loop, blocking
for the result to complete, so you can mimic the behavior of the clojure
version, I think you could sit in a loop and use the async function take!
with the variant that calls a callback function only if a value is
immediately available on the channel.  So in that callback you mark some
variable with the final result of the reduction, and you only exit the loop
once that result is available.

-- 
You received this message because you are subscribed to the Google
Groups Clojure group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
Clojure group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: core.async: implementing pi.go

[Mark Engelberg]

Clojure's async is built around the opinion that you, the programmer,
should be required to think about what sort of buffer you want to have on
your channel, and think about what should happen if that buffer overflows.

Your code spins off 5000 little go blocks that are each trying to write to
a channel of size 1.  This effectively forces Clojure to buffer up all
those blocking puts.  Clojure provides a fairly limited buffer (size 1024)
for the blocked puts so that you can't easily circumvent its opinionated
policy that *you* should be supplying the right size buffer with the
appropriate policy.

So, short answer: change your (chan) to something like (chan 1) and
your code works fine.

But that's probably not what you're looking for.  We really don't want to
have to size the buffer based on the input to the pi function.  We want to
be able to deal with arbitrarily large inputs with some reasonably smallish
buffer.  Because this is a toy problem, it glosses over something that
would be critical for us to think about in a real application: Which runs
faster, the term computation or the additions in the accumulator?  The
answer to that question leads to two a couple different solutions.

Before I show some possible solutions, I should mention that I'm not a big
fan of the way you wrote the term function.  I would argue that it is
much cleaner to write it as a pure function (something you can test).
Don't inject notions of channels into a pure arithmetic computation.  Also,
you've made the math unnecessarily convoluted with your overuse of -.
Embrace infix math.  If you think the -1^k expression is awkward, you could
do something like this:

(defn term [k]
  (/ (if (even? k) 4 -4)
 (inc (* 2 k

There, a nice clean pure function that makes it clear that -1^k is just a
trick for putting the correct sign on the expression.  You can change 2
to 2.0 if you want to return a double rather than a rational number from
this function (or use this version and just coerce to the double in the
consumer, depending on the behavior you want).

Or, if you want to use pow:
(defn term [k]
  (/ (* 4 (Math/pow -1 k))
 (inc (* 2 k

Or, you can leverage the fact that multiplying by 1 or -1 is the same as
dividing and eliminate the multiplication:
(defn term [k]
  (/ 4 (Math/pow -1 k)
 (inc (* 2 k

Any of these would be preferable to the - version which is harder to read.

Now, back to the key question.

Case 1: term is fast, addition is slow
In this case, we really just need one thread to generate all the term
expressions, and we simply want to let the producer get ahead of the
consumer by some bounded amount.  So something like this would work:

(defn pi [n]
  (let [ch (to-chan (map term (range (inc n]
(!! (async/reduce + 0.0 ch

to-chan uses a bounded buffer of size 100, and we're also making use of the
way map works with chunked sequences to efficiently do fast operations.  If
you don't want chunked sequence behavior, you can use `sequence` with a
transducer:

(defn pi [n]
  (let [ch (to-chan (sequence (map term) (range (inc n]
(!! (async/reduce + 0.0 ch

Case 2: term is slow, addition is fast
The above code will still work if term is slower than addition, but if we
know term is slower, we can improve performance by running the (map term)
transducer on multiple processors.  async's pipeline lets you do just
that.  Let's say we want to use 20 go blocks.  We'll put all the numbers
0 through n on one channel, pipe it to another channel with the (map term)
transducer using a parallelism count of 20, and reduce the result:

(defn pi [n]
  (let [ch (chan 100)]
(pipeline 20 ch (map term) (to-chan (range (inc n
(!! (async/reduce + 0.0 ch

Of course, you can get computational parallelism without using channels at
all:
(defn pi [n]
  (clojure.core.reducers/fold + (clojure.core.reducers/map term (vec (range
(inc n))

which runs several times faster (depending on your number of processors)
than the naive:
(defn pi [n]
  (reduce + (map term (range (inc n)



Summary: core.async forces you to make some choices, because of its
philosophy that unlimited buffers results in error-prone programs that will
crash unexpectedly when resources are exceeded, but once you make those
choices, it gives you some nice high-level constructs to express these
ideas succinctly.

-- 
You received this message because you are subscribed to the Google
Groups Clojure group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
Clojure group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to 

Re: core.async: implementing pi.go

[Divyansh Prakash]

Hey, puzzler!
Thanks for the detailed response. Just changing (chan) to (chan n) actually 
worked!
 
I get your displeasure with how 'term' is implemented. 
That's not how I generally code. I'm very new to core.async and was aiming 
for a direct translation of the Go code.

I do get a little carried away with - at times, though.

Also - though optimization wasn't a priority, thanks for taking out the 
time to show alternate approaches along with their use cases. 
I haven't tried reducers yet, so that was new.

I have one more question, though: how does one work in ClojureScript 
without !! ?

- Divyansh

-- 
You received this message because you are subscribed to the Google
Groups Clojure group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
Clojure group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: core.async: implementing pi.go

[Divyansh Prakash]

Makes sense.
By the way - I've refactored my code 
https://github.com/divs1210/go-play/blob/master/src/go_play/pi.clj to not 
use a go block inside 'term', and made it much more readable (IMO) than 
before using my maya library https://github.com/divs1210/maya.
I'm telling you this because I'm not a big fan of writing mathematical 
expressions as s-exps but you seem all for it.
I had written the macros defined in maya quite a long time ago, but you 
just inspired me to push it to clojars, so thanks for that! :)

- Divyansh

-- 
You received this message because you are subscribed to the Google
Groups Clojure group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
Clojure group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: core.async: implementing pi.go

[Mark Engelberg]

I agree that there's value to making math expressions look like the way
they are written in actual math -- it is much easier to tell at a glance
that you've entered the correct formula.  I think your maya library is a
nice contribution.  (I think Incanter has a similar macro, but it is nice
to have it in a lightweight standalone library.
https://github.com/joyofclojure/unfix is another one, I haven't figured out
exactly how they all compare.  It's not obvious to me from your
documentation -- does maya automatically handle operator precedence?).

My point is mainly that people who read and write Clojure are used to using
infix notation, so for that particular audience, it should be no big deal
to read the infix math unless the expression is particularly complex.
Specifically, I don't think that reordering math with - gains you any
readability.  But I'm all for judicious use of infix DSLs.



On Sun, Aug 2, 2015 at 5:59 AM, Divyansh Prakash 
divyanshprakas...@gmail.com wrote:

 Makes sense.
 By the way - I've refactored my code
 https://github.com/divs1210/go-play/blob/master/src/go_play/pi.clj to
 not use a go block inside 'term', and made it much more readable (IMO) than
 before using my maya library https://github.com/divs1210/maya.
 I'm telling you this because I'm not a big fan of writing mathematical
 expressions as s-exps but you seem all for it.
 I had written the macros defined in maya quite a long time ago, but you
 just inspired me to push it to clojars, so thanks for that! :)

 - Divyansh

 --
 You received this message because you are subscribed to the Google
 Groups Clojure group.
 To post to this group, send email to clojure@googlegroups.com
 Note that posts from new members are moderated - please be patient with
 your first post.
 To unsubscribe from this group, send email to
 clojure+unsubscr...@googlegroups.com
 For more options, visit this group at
 http://groups.google.com/group/clojure?hl=en
 ---
 You received this message because you are subscribed to the Google Groups
 Clojure group.
 To unsubscribe from this group and stop receiving emails from it, send an
 email to clojure+unsubscr...@googlegroups.com.
 For more options, visit https://groups.google.com/d/optout.


-- 
You received this message because you are subscribed to the Google
Groups Clojure group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
Clojure group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: core.async status?

[Michael Blume]

Looking through the tickets at http://dev.clojure.org/jira/browse/ASYNC
might give you a better idea of what's planned.

On Sat, Jul 4, 2015 at 8:52 PM Martin Raison martinrai...@gmail.com wrote:

 thanks!

 Le samedi 4 juillet 2015 20:38:22 UTC-7, Alex Miller a écrit :

 Oh just busy. We will get to a new release at some point.

  --
 You received this message because you are subscribed to the Google
 Groups Clojure group.
 To post to this group, send email to clojure@googlegroups.com
 Note that posts from new members are moderated - please be patient with
 your first post.
 To unsubscribe from this group, send email to
 clojure+unsubscr...@googlegroups.com
 For more options, visit this group at
 http://groups.google.com/group/clojure?hl=en
 ---
 You received this message because you are subscribed to the Google Groups
 Clojure group.
 To unsubscribe from this group and stop receiving emails from it, send an
 email to clojure+unsubscr...@googlegroups.com.
 For more options, visit https://groups.google.com/d/optout.


-- 
You received this message because you are subscribed to the Google
Groups Clojure group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
Clojure group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: core.async status?

[Martin Raison]

thanks!

Le samedi 4 juillet 2015 20:38:22 UTC-7, Alex Miller a écrit :

 Oh just busy. We will get to a new release at some point.

-- 
You received this message because you are subscribed to the Google
Groups Clojure group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
Clojure group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: core.async pub/sub closing source channel issue

[Leon Grapenthin]

The reason for the behavior you are observing is a race condition. The 
effects of close! on the pub aren't synchronous. Even though the channel is 
immediately closed before return, consumers need time to determine that it 
has been closed. At the point in time the pub determines that the source 
channel has been closed, it does indeed close all subscribed channels, as 
your last form demonstrates. However, in your second last form, all 
subscribed channels may have already been unsubscribed, due to unsub-all.

Thus, the solution is to not call unsubs-all if you want all subscribed 
channels to be closed as a consequence of closing the source channel.


On Sunday, March 1, 2015 at 2:17:20 PM UTC+1, Jonas wrote:

 Hi all!

 I’m working with core.async pub/sub and ran into an issue which I don’t 
 quite understand. According to the clojure.core.async/sub docstring:

  By default the channel will be closed when the source closes

 This is not the behaviour I’m seeing when I call 
 clojure.core.async/unsub-all immediately after I close the source channel.

 Example:

 (def source (async/chan))
 (def sub-chan (async/chan))
 (def pub (async/pub source (constantly foo)))
 
 ;; by default sub-chan should close when source closes
 (async/sub pub foo sub-chan) 
 
 ;; This will not close sub-chan
 (async/thread
   (async/close! source)
   (async/unsub-all pub))

 ;; This will close sub-chan
 (async/thread
   (async/close! source))

 What is the reason for this behaviour? Is it unnecessary to call unsub-all 
 if I close the publications source channel?

 Thanks,
 Jonas


-- 
You received this message because you are subscribed to the Google
Groups Clojure group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
Clojure group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: core.async why 42 extra threads?

[Timothy Baldridge]

The  number of threads is meant to be semi-tolerant to usage of blocking IO
in go blocks. You can't be certain that some go block won't call a function
that calls a function in some library that blocks for just a few
milliseconds. So we tried to make it high enough to be tolerant of mis-use,
but low enough to be bounded somewhat.

So yes, the number + 42 is arbitrary (and somewhat of a joke) but the fact
that it's more than 2 * CPUs is intentional.

Timothy

On Thu, Feb 19, 2015 at 5:02 AM, Robin Heggelund Hansen 
skinney...@gmail.com wrote:

 From the source of core.async, I see that it is started with a threadpool
 of number of processors*2 + 42.

 I can understand the number of processors * 2 part, erlang does the same
 thing. But Hitchicker's references aside, why add 42 to this? Won't that
 many threads do more harm than good, in terms of overhead related to
 context switching?

 --
 You received this message because you are subscribed to the Google
 Groups Clojure group.
 To post to this group, send email to clojure@googlegroups.com
 Note that posts from new members are moderated - please be patient with
 your first post.
 To unsubscribe from this group, send email to
 clojure+unsubscr...@googlegroups.com
 For more options, visit this group at
 http://groups.google.com/group/clojure?hl=en
 ---
 You received this message because you are subscribed to the Google Groups
 Clojure group.
 To unsubscribe from this group and stop receiving emails from it, send an
 email to clojure+unsubscr...@googlegroups.com.
 For more options, visit https://groups.google.com/d/optout.




-- 
“One of the main causes of the fall of the Roman Empire was that–lacking
zero–they had no way to indicate successful termination of their C
programs.”
(Robert Firth)

-- 
You received this message because you are subscribed to the Google
Groups Clojure group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
Clojure group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: core.async why 42 extra threads?

[Robin Heggelund Hansen]

 Ahh ok, makes sense :)

-- 
You received this message because you are subscribed to the Google
Groups Clojure group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
Clojure group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: core.async: Deprecated - this function will be removed. Use transducer instead

[Ben Smith-Mannschott]

I'm unclear on one thing: what's the purpose of core.async/pipe? In your
blog article, you write:

(- source (pipe (chan)) payload-decoder payload-json-decoder)

(pipe source destination) just copies elements from source to destination.
How is that any different than just using source here directly?:

(- source payload-decoder payload-json-decoder)

What have I missed?

// Ben

On Thu, Feb 19, 2015 at 3:06 AM, Malcolm Sparks malc...@juxt.pro wrote:

 I have recently written a blog article which explains how to use
 transducers with core.async.

 You can find it here: http://malcolmsparks.com/posts/transducers.html


 On Wednesday, 18 February 2015 21:48:05 UTC, bsmith.occs wrote:

 I'm probably just especially dense today, but perhaps someone can give me
 a poke in the right direction.

 I'm trying to wrap my head around transducers.

 (1) For debugging purposes I'd like to be able to consume the values on a
 channel and put them in a collection to be printed.

 I'm doing this at the REPL:

 (async/!! (async/into [] channel))

 This seems needlessly clunky. Is this really the best way to accomplish
 this?

 (2) async/map is deprecated, what's the alternative, exactly?

 (let [ch (async/to-chan [1 2 3])]
   (async/map inc ch))

 AFAICT this would produce a channel, which when consumed would yield the
 values 2 3 4, correct?

 Now I suppose if I didn't already have the channel ch, I could create one
 with an associated transformation function, and then stuff the values 1 2 3
 into it somehow, right?

 (let [ch (async/chan nil (map inc))]
   (async/onto-chan ch [1 2 3])
   ch)
 ;; returns a channel yielding items 2, 3, 4

 But, I'm not clear at all how I can apply the operation (inc) to the all
 values yielded by a channel I already have in my hand.

 Any help would be appreciated. I feel like I'm missing some critical bit
 of insight and that async and transducers might just fall into place for me
 if I could only find out what that bit is.

 // Ben

  --
 You received this message because you are subscribed to the Google
 Groups Clojure group.
 To post to this group, send email to clojure@googlegroups.com
 Note that posts from new members are moderated - please be patient with
 your first post.
 To unsubscribe from this group, send email to
 clojure+unsubscr...@googlegroups.com
 For more options, visit this group at
 http://groups.google.com/group/clojure?hl=en
 ---
 You received this message because you are subscribed to the Google Groups
 Clojure group.
 To unsubscribe from this group and stop receiving emails from it, send an
 email to clojure+unsubscr...@googlegroups.com.
 For more options, visit https://groups.google.com/d/optout.


-- 
You received this message because you are subscribed to the Google
Groups Clojure group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
Clojure group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: core.async: Deprecated - this function will be removed. Use transducer instead

[Malcolm Sparks]

You're right - thanks for that! I've updated the blog article to remove it.

On 19 February 2015 at 17:37, Ben Smith-Mannschott bsmith.o...@gmail.com
wrote:

 I'm unclear on one thing: what's the purpose of core.async/pipe? In your
 blog article, you write:

 (- source (pipe (chan)) payload-decoder payload-json-decoder)

 (pipe source destination) just copies elements from source to destination.
 How is that any different than just using source here directly?:

 (- source payload-decoder payload-json-decoder)

 What have I missed?

 // Ben

 On Thu, Feb 19, 2015 at 3:06 AM, Malcolm Sparks malc...@juxt.pro wrote:

 I have recently written a blog article which explains how to use
 transducers with core.async.

 You can find it here: http://malcolmsparks.com/posts/transducers.html


 On Wednesday, 18 February 2015 21:48:05 UTC, bsmith.occs wrote:

 I'm probably just especially dense today, but perhaps someone can give
 me a poke in the right direction.

 I'm trying to wrap my head around transducers.

 (1) For debugging purposes I'd like to be able to consume the values on
 a channel and put them in a collection to be printed.

 I'm doing this at the REPL:

 (async/!! (async/into [] channel))

 This seems needlessly clunky. Is this really the best way to accomplish
 this?

 (2) async/map is deprecated, what's the alternative, exactly?

 (let [ch (async/to-chan [1 2 3])]
   (async/map inc ch))

 AFAICT this would produce a channel, which when consumed would yield the
 values 2 3 4, correct?

 Now I suppose if I didn't already have the channel ch, I could create
 one with an associated transformation function, and then stuff the values 1
 2 3 into it somehow, right?

 (let [ch (async/chan nil (map inc))]
   (async/onto-chan ch [1 2 3])
   ch)
 ;; returns a channel yielding items 2, 3, 4

 But, I'm not clear at all how I can apply the operation (inc) to the all
 values yielded by a channel I already have in my hand.

 Any help would be appreciated. I feel like I'm missing some critical bit
 of insight and that async and transducers might just fall into place for me
 if I could only find out what that bit is.

 // Ben

  --
 You received this message because you are subscribed to the Google
 Groups Clojure group.
 To post to this group, send email to clojure@googlegroups.com
 Note that posts from new members are moderated - please be patient with
 your first post.
 To unsubscribe from this group, send email to
 clojure+unsubscr...@googlegroups.com
 For more options, visit this group at
 http://groups.google.com/group/clojure?hl=en
 ---
 You received this message because you are subscribed to the Google Groups
 Clojure group.
 To unsubscribe from this group and stop receiving emails from it, send an
 email to clojure+unsubscr...@googlegroups.com.
 For more options, visit https://groups.google.com/d/optout.


  --
 You received this message because you are subscribed to the Google
 Groups Clojure group.
 To post to this group, send email to clojure@googlegroups.com
 Note that posts from new members are moderated - please be patient with
 your first post.
 To unsubscribe from this group, send email to
 clojure+unsubscr...@googlegroups.com
 For more options, visit this group at
 http://groups.google.com/group/clojure?hl=en
 ---
 You received this message because you are subscribed to a topic in the
 Google Groups Clojure group.
 To unsubscribe from this topic, visit
 https://groups.google.com/d/topic/clojure/z1B-42npgaI/unsubscribe.
 To unsubscribe from this group and all its topics, send an email to
 clojure+unsubscr...@googlegroups.com.
 For more options, visit https://groups.google.com/d/optout.


-- 
You received this message because you are subscribed to the Google
Groups Clojure group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
Clojure group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: core.async: Deprecated - this function will be removed. Use transducer instead

[Erik Price]

(let [out (async/chan 0 (map inc))]
  (async/pipe in out)
  out)

Earlier in your email you mention printing, however. If you have I/O to
perform (like printing), I’m told that you don’t want to do it in a
transducer. You can use pipeline-async for this instead:

(defn f [v ch]
  (async/go
(println v)
(async/! ch v)))

(let [out (async/chan)]
  (async/pipeline-async 1 out f in)
  out)

(If I’m doing this wrong, someone please steer me in the right direction.)

e

On Wed, Feb 18, 2015 at 4:47 PM, Ben Smith-Mannschott bsmith.o...@gmail.com
wrote:

I'm probably just especially dense today, but perhaps someone can give me a
 poke in the right direction.

 I'm trying to wrap my head around transducers.

 (1) For debugging purposes I'd like to be able to consume the values on a
 channel and put them in a collection to be printed.

 I'm doing this at the REPL:

 (async/!! (async/into [] channel))

 This seems needlessly clunky. Is this really the best way to accomplish
 this?

 (2) async/map is deprecated, what's the alternative, exactly?

 (let [ch (async/to-chan [1 2 3])]
   (async/map inc ch))

 AFAICT this would produce a channel, which when consumed would yield the
 values 2 3 4, correct?

 Now I suppose if I didn't already have the channel ch, I could create one
 with an associated transformation function, and then stuff the values 1 2 3
 into it somehow, right?

 (let [ch (async/chan nil (map inc))]
   (async/onto-chan ch [1 2 3])
   ch)
 ;; returns a channel yielding items 2, 3, 4

 But, I'm not clear at all how I can apply the operation (inc) to the all
 values yielded by a channel I already have in my hand.

 Any help would be appreciated. I feel like I'm missing some critical bit
 of insight and that async and transducers might just fall into place for me
 if I could only find out what that bit is.

 // Ben

 --
 You received this message because you are subscribed to the Google
 Groups Clojure group.
 To post to this group, send email to clojure@googlegroups.com
 Note that posts from new members are moderated - please be patient with
 your first post.
 To unsubscribe from this group, send email to
 clojure+unsubscr...@googlegroups.com
 For more options, visit this group at
 http://groups.google.com/group/clojure?hl=en
 ---
 You received this message because you are subscribed to the Google Groups
 Clojure group.
 To unsubscribe from this group and stop receiving emails from it, send an
 email to clojure+unsubscr...@googlegroups.com.
 For more options, visit https://groups.google.com/d/optout.

​

-- 
You received this message because you are subscribed to the Google
Groups Clojure group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
Clojure group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: core.async: Deprecated - this function will be removed. Use transducer instead

[Ben Smith-Mannschott]

Thanks Malcolm, you're blog post was a great help to me.

On Thu, Feb 19, 2015 at 3:06 AM, Malcolm Sparks malc...@juxt.pro wrote:

 I have recently written a blog article which explains how to use
 transducers with core.async.

 You can find it here: http://malcolmsparks.com/posts/transducers.html


 On Wednesday, 18 February 2015 21:48:05 UTC, bsmith.occs wrote:

 I'm probably just especially dense today, but perhaps someone can give me
 a poke in the right direction.

 I'm trying to wrap my head around transducers.

 (1) For debugging purposes I'd like to be able to consume the values on a
 channel and put them in a collection to be printed.

 I'm doing this at the REPL:

 (async/!! (async/into [] channel))

 This seems needlessly clunky. Is this really the best way to accomplish
 this?

 (2) async/map is deprecated, what's the alternative, exactly?

 (let [ch (async/to-chan [1 2 3])]
   (async/map inc ch))

 AFAICT this would produce a channel, which when consumed would yield the
 values 2 3 4, correct?

 Now I suppose if I didn't already have the channel ch, I could create one
 with an associated transformation function, and then stuff the values 1 2 3
 into it somehow, right?

 (let [ch (async/chan nil (map inc))]
   (async/onto-chan ch [1 2 3])
   ch)
 ;; returns a channel yielding items 2, 3, 4

 But, I'm not clear at all how I can apply the operation (inc) to the all
 values yielded by a channel I already have in my hand.

 Any help would be appreciated. I feel like I'm missing some critical bit
 of insight and that async and transducers might just fall into place for me
 if I could only find out what that bit is.

 // Ben

  --
 You received this message because you are subscribed to the Google
 Groups Clojure group.
 To post to this group, send email to clojure@googlegroups.com
 Note that posts from new members are moderated - please be patient with
 your first post.
 To unsubscribe from this group, send email to
 clojure+unsubscr...@googlegroups.com
 For more options, visit this group at
 http://groups.google.com/group/clojure?hl=en
 ---
 You received this message because you are subscribed to the Google Groups
 Clojure group.
 To unsubscribe from this group and stop receiving emails from it, send an
 email to clojure+unsubscr...@googlegroups.com.
 For more options, visit https://groups.google.com/d/optout.


-- 
You received this message because you are subscribed to the Google
Groups Clojure group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
Clojure group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: core.async: Deprecated - this function will be removed. Use transducer instead

[Malcolm Sparks]

I have recently written a blog article which explains how to use 
transducers with core.async. 

You can find it here: http://malcolmsparks.com/posts/transducers.html

On Wednesday, 18 February 2015 21:48:05 UTC, bsmith.occs wrote:

 I'm probably just especially dense today, but perhaps someone can give me 
 a poke in the right direction.

 I'm trying to wrap my head around transducers.

 (1) For debugging purposes I'd like to be able to consume the values on a 
 channel and put them in a collection to be printed.

 I'm doing this at the REPL:

 (async/!! (async/into [] channel))

 This seems needlessly clunky. Is this really the best way to accomplish 
 this?

 (2) async/map is deprecated, what's the alternative, exactly?

 (let [ch (async/to-chan [1 2 3])]
   (async/map inc ch))

 AFAICT this would produce a channel, which when consumed would yield the 
 values 2 3 4, correct?

 Now I suppose if I didn't already have the channel ch, I could create one 
 with an associated transformation function, and then stuff the values 1 2 3 
 into it somehow, right?

 (let [ch (async/chan nil (map inc))]
   (async/onto-chan ch [1 2 3])
   ch)
 ;; returns a channel yielding items 2, 3, 4

 But, I'm not clear at all how I can apply the operation (inc) to the all 
 values yielded by a channel I already have in my hand.

 Any help would be appreciated. I feel like I'm missing some critical bit 
 of insight and that async and transducers might just fall into place for me 
 if I could only find out what that bit is.

 // Ben


-- 
You received this message because you are subscribed to the Google
Groups Clojure group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
Clojure group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: core.async -- lazy evaluation on take

[janpaulbultmann]

Make the channel unbuffered, that way it turns into a rondevouz a la ada and 
every producer will block until a consumer takes something from it.

cheers Jan

 On 16.02.2015, at 21:45, Huey Petersen eys...@gmail.com wrote:
 
 Hello,
 
 I was playing around with having a lazy sequence abstracting over a paged 
 http request.  First off, maybe this is dumb, I dunno, lemme know ;p
 
 So (resources url) returns a seq and no http request is made, but (take 1 
 (resources url)) makes a request and (take 50 (resources url)) may make 
 multiple requests (depending on how many per page).
 
 I'm wondering if it is possible to implement the same thing as a channel -- 
 no http request is made until someone tries to take from it.  More of a 
 curiosity.
 
 Thanks!
 -- 
 You received this message because you are subscribed to the Google
 Groups Clojure group.
 To post to this group, send email to clojure@googlegroups.com
 Note that posts from new members are moderated - please be patient with your 
 first post.
 To unsubscribe from this group, send email to
 clojure+unsubscr...@googlegroups.com
 For more options, visit this group at
 http://groups.google.com/group/clojure?hl=en
 --- 
 You received this message because you are subscribed to the Google Groups 
 Clojure group.
 To unsubscribe from this group and stop receiving emails from it, send an 
 email to clojure+unsubscr...@googlegroups.com.
 For more options, visit https://groups.google.com/d/optout.

-- 
You received this message because you are subscribed to the Google
Groups Clojure group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
Clojure group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: core.async -- lazy evaluation on take

[Erik Price]

Yes, the producer’s put will block until the consumer takes, but doesn’t
this still involve an eager initial request (so that the producer will have
something to put in the first place, so that it can block)?

e
​

On Mon, Feb 16, 2015 at 5:52 PM, janpaulbultm...@googlemail.com wrote:

 Make the channel unbuffered, that way it turns into a rondevouz a la ada
 and every producer will block until a consumer takes something from it.

 cheers Jan

 On 16.02.2015, at 21:45, Huey Petersen eys...@gmail.com wrote:

 Hello,

 I was playing around with having a lazy sequence abstracting over a paged
 http request.  First off, maybe this is dumb, I dunno, lemme know ;p

 So (resources url) returns a seq and no http request is made, but (take
 1 (resources url)) makes a request and (take 50 (resources url)) may
 make multiple requests (depending on how many per page).

 I'm wondering if it is possible to implement the same thing as a channel
 -- no http request is made until someone tries to take from it.  More of a
 curiosity.

 Thanks!

 --
 You received this message because you are subscribed to the Google
 Groups Clojure group.
 To post to this group, send email to clojure@googlegroups.com
 Note that posts from new members are moderated - please be patient with
 your first post.
 To unsubscribe from this group, send email to
 clojure+unsubscr...@googlegroups.com
 For more options, visit this group at
 http://groups.google.com/group/clojure?hl=en
 ---
 You received this message because you are subscribed to the Google Groups
 Clojure group.
 To unsubscribe from this group and stop receiving emails from it, send an
 email to clojure+unsubscr...@googlegroups.com.
 For more options, visit https://groups.google.com/d/optout.

  --
 You received this message because you are subscribed to the Google
 Groups Clojure group.
 To post to this group, send email to clojure@googlegroups.com
 Note that posts from new members are moderated - please be patient with
 your first post.
 To unsubscribe from this group, send email to
 clojure+unsubscr...@googlegroups.com
 For more options, visit this group at
 http://groups.google.com/group/clojure?hl=en
 ---
 You received this message because you are subscribed to the Google Groups
 Clojure group.
 To unsubscribe from this group and stop receiving emails from it, send an
 email to clojure+unsubscr...@googlegroups.com.
 For more options, visit https://groups.google.com/d/optout.


-- 
You received this message because you are subscribed to the Google
Groups Clojure group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
Clojure group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: core.async -- lazy evaluation on take

[Fluid Dynamics]

On Monday, February 16, 2015 at 9:42:31 PM UTC-5, Erik Price wrote:

 Yes, the producer’s put will block until the consumer takes, but doesn’t 
 this still involve an eager initial request (so that the producer will have 
 something to put in the first place, so that it can block)?

The producer can put a pair of a token and the result of an HTTP request 
on, waiting to perform the HTTP request until the blocking put of the token 
has completed. The consumer can, when it wants the results of an HTTP 
request, perform a take to get a token, discard that, and do another take 
to get the result of the HTTP request. The first take unblocks the producer 
to perform the request in the first place.
 

-- 
You received this message because you are subscribed to the Google
Groups Clojure group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
Clojure group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: core.async chan ex-handler

[coltnz]

On Friday, January 23, 2015 at 12:59:40 AM UTC+13, Derek Troy-West wrote:

 From the documentation:

 (chan buf-or-n xform ex-handler)

 If a transducer is supplied a
 buffer must be specified. ex-handler must be a fn of one argument -
 if an exception occurs during transformation it will be called with
 the Throwable as an argument


 Is there a reason the ex-handler fn doesn't have a second arity which also 
 takes the val which caused the transformation error?



No reason I can see. Seems worthy of a ticket on the async project to me.

You could construct your own ManyToManyChannel as a workaround for now. 
Copy chan's 3rd arity and call your own handle fn taking buf.

Colin 

-- 
You received this message because you are subscribed to the Google
Groups Clojure group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
Clojure group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: core.async go-loop questions

[Jonathon McKitrick]

Well, my goal is to start a go-loop (if possible) at the root level of the
code that simply parks and waits for a group of emails to be sent.  When
that happens, it would wake up and broadcast the result of the send
operation via web socket back to the browser.  I'd like to avoid starting
that loop every time I send the emails, and I was under the impression that
could be done with a go-loop, which would park until the channel had values
to consume.  But you are saying that might be 'unhealthy'?


--
Jonathon McKitrick

On Sat, Dec 20, 2014 at 10:56 PM, Chris Freeman cwfree...@gmail.com wrote:

 I'm a little uncertain exactly what your code is trying to do, but I
 believe you're trying to notify a bunch of connections after your speaker
 notification emails are sent.

 In which case, I'd do something like this:

 (defn send-notifications []
   (try
 (mailer/send-speaker-confirmation-notification 1 http://localhost;)
 true
 (catch Exception e
   (println (.getMessage e))
   false)))

 (defn test-mailer []
   (let [done (async/thread-call send-notifications)]
 (when-let [status (async/! done)]
   (doseq [chan @connections]
 (async/! chan (pr-str Done status))

 I've replaced the string Success with an explicit true and added an
 explicit false. I'd prefer if send-speaker-confirmation-notification
 returned a truthy value, but I don't know that it does.

 In your original, the doseq call was done in a separate thread. If you'd
 still like that, wrap the when-let in a async/go call, as followings:

 (defn test-mailer []
   (let [done (async/thread-call send-notifications)]
 (async/go
   (when-let [status (async/! done)]
 (doseq [chan @connections]
   (async/! chan (pr-str Done status)))

 But if you do, keep in mind that the main thread will end before the other
 two threads do, and that's probably unhealthy.

 Chris

 On Sat, Dec 20, 2014 at 9:17 PM, Jonathon McKitrick jmckitr...@gmail.com
 wrote:

 I'd like to implement a thread that will send an email, then send a
 response via websocket to the client when the send completes.

 (defn my-wait-loop []
   (async/go-loop [status (async/! @mailer-status)]
 (if status
   (do
 (println Ready to send  status)
 (doseq [chan @connections]
   (println Channel chan)
   (send! chan (pr-str Done status)))
 (recur (async/! @mailer-status)))
   (println Go away

 (defn test-mailer []
   ;;(my-wait-loop)
   (reset! mailer-status
   (async/thread
 (try
   (mailer/send-speaker-confirmation-notification 1 
 http://localhost;)
   (catch Exception e
 (println (.getMessage e
 Success)))

 I would like to have the go-loop inside my-wait-loop run at all times,
 waiting for mailer-status to have a value.
 But I believe that can never happen, since the go-loop is waiting on an
 empty channel, and the reset! with the mailer will replace the channel with
 a new one after the emails are sent.

 Is there a batter way to do this, without needing to call my-wait-loop
 before the email thread is dispatched?

 --
 You received this message because you are subscribed to the Google
 Groups Clojure group.
 To post to this group, send email to clojure@googlegroups.com
 Note that posts from new members are moderated - please be patient with
 your first post.
 To unsubscribe from this group, send email to
 clojure+unsubscr...@googlegroups.com
 For more options, visit this group at
 http://groups.google.com/group/clojure?hl=en
 ---
 You received this message because you are subscribed to the Google Groups
 Clojure group.
 To unsubscribe from this group and stop receiving emails from it, send an
 email to clojure+unsubscr...@googlegroups.com.
 For more options, visit https://groups.google.com/d/optout.


  --
 You received this message because you are subscribed to the Google
 Groups Clojure group.
 To post to this group, send email to clojure@googlegroups.com
 Note that posts from new members are moderated - please be patient with
 your first post.
 To unsubscribe from this group, send email to
 clojure+unsubscr...@googlegroups.com
 For more options, visit this group at
 http://groups.google.com/group/clojure?hl=en
 ---
 You received this message because you are subscribed to a topic in the
 Google Groups Clojure group.
 To unsubscribe from this topic, visit
 https://groups.google.com/d/topic/clojure/f9J_CBPoc5U/unsubscribe.
 To unsubscribe from this group and all its topics, send an email to
 clojure+unsubscr...@googlegroups.com.
 For more options, visit https://groups.google.com/d/optout.


-- 
You received this message because you are subscribed to the Google
Groups Clojure group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to