Re: An Error spec?

[Didier]

Interesting. I understand that, I guess it's because of my usage of spec.

I tend to use spec exclusively at the boundaries. Think serialization. 
Sometimes, I have a producer which can't handle the error, so it must be 
serialized and exchanged to the consumer.

So say the producer couldn't produce the value, for some reason. In some cases, 
that's a bug, and I throw an exception, and fail the producer itself. But 
sometimes, its an expected possibility, in which case, tbe value is an error. 
And the producer must communicate that out.

Unfortunatly, exceptions are not friendly to serialization or spec. So I need 
to roll my own error structure, serialize that, and spec it.

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Mail queue thread

[Erik Assum]

Not knowing your app nor your requirements, but it might also be an idea to 
separate concerns:

Your web-app could store “emails to be sent” in a database table, and you could 
have a different process reading this table at regular intervals and sending 
mails, and also marking the emails as sent, failed, or what have you. Maybe 
even a time stamp to indicate when an email was sent. 

This gives you a couple of benefits:
1) a simpler web app
2) a chance to resend emails
3) easier testing, as you can see that the web-app stores the mails it should 
in the db, without actually sending emails

HTH, 

Erik. 
-- 
i farta

> 25. okt. 2018 kl. 17:46 skrev brj...@gmail.com:
> 
> Hi,
> 
> First, I would like to briefly present myself to the list. I'm a psychologist 
> and researcher at the Karolinska Institutet in Stockholm, Sweden. I do 
> research within the field of internet-based psychological treatment - which 
> is a field that has grown a lot during the last 10-15 years. I have developed 
> my own web platform to deliver these treatments to patients with different 
> mental and medical conditions. I wrote the first version in PHP, and not 
> being a professional programmer, the code base is a big mess after > 10 years 
> of development.
> 
> Two years ago, a friend introduced me to Clojure. I had never worked with a 
> lisp before, but I quickly fell completely in love with it. So many of the 
> issues of programming in PHP completely disappeared! Words cannot express my 
> gratitude to Rich for inventing Clojure, the core team, all developers out 
> there who write extremely useful libraries, and my friend for introducing 
> Clojure to me.
> 
> I decided to completely rewrite my web platform in Clojure. I've chosen to do 
> a gradual move from PHP to Clojure, meaning that I replace PHP code with 
> Clojure code component by component and continuously test in production. Like 
> I said, I'm not a professional programmer, and this venture poses a lot of 
> challenges. I try to read everything I find in books and on the web, but when 
> it comes to more complex issues, such as threads and async programming, I 
> feel that I end up almost guessing and with a lot of trial and error. I have 
> no idea how cautious one should be when launching a new thread (while it 
> completely occupy the server???) and am often surprised when my go blocks 
> suddenly freeze. I feel that I am at the mercy of the Clojure community if I 
> want to understand these (and many other!) issues.
> 
> This leads me to the subject of this email. I've decided to migrate my mail 
> queue from PHP to Clojure. In PHP, it's just a cron job that executes every 
> five minutes to send all emails (and actually also SMS-messages, but not 
> really relevant) that have been queued.
> 
> I've written a basic mock Clojure implementation with the following goals
> - All messages should be sent async, i..e, the web user should not have to 
> wait while the email is being sent. -> I'm sending them in a separate thread.
> - I have a fear that if I have a thread dedicated only to sending emails, I'm 
> wasting server resources. -> I want the thread to die 5 seconds after the 
> last email in the queue has been sent.
> 
> My implementation basically consists of
> - An eternal go loop that receives a notification through a channel if new 
> messages have been queued
> - The go loop checks if the mail sender thread is running. If not, it starts 
> it.
> - The mail sender thread dies 5 secs after the last email was sent
> - The state of the thread (running / not running) is stored in an agent to 
> avoid race conditions (i.e., starting multiple threads or not starting a 
> thread because it is running when its status is checked but stops right 
> after).
> 
> My code is here
> https://gist.github.com/brjann/2aef16849b9bd445374cb6b31efece60
> 
> If any of you have had the time and energy to read this far (including the 
> code), I would be very grateful for your input.
> - Is there a risk that my go block will hang?
> - Have I eliminated the risk for race conditions?
> - Do I really need to kill the thread or is there no risk for thread 
> starvation on the server (I will probably 
> - Could I use send instead of send-off? I guess that I am now using two 
> threads, one for the sender and one each time I send a message using send-off.
> - Any newbie mistakes / coding style issues?
> - Could this be done in a better/simpler way???
> 
> (Btw, I would be very grateful for feedback on the form of my question if you 
> have any. Are there better/other forums? Could I present the question or code 
> in a another manner to help you understand better?)
> 
> I am happy for any replies - on the list or backchannel. And I hope that you 
> feel that I have not misused your inbox (I may repeat this behavior...).
> 
> Thanks,
> Brjánn
> -- 
> You received this message because you are subscribed to the Google
> Groups "Clojure" group.
> To post to this group, send email to c

Re: An Error spec?

[Alex Miller]

This is not a pattern that we encourage in Clojure. The standard mechanism 
for error reporting is exceptions, which are not spec'ed.

On Thursday, October 25, 2018 at 8:46:54 PM UTC-5, Didier wrote:
>
> I've started to see a pattern in my spec like this:
>
> (s/or :success string?
>   :error ::error)
>
> And I've been tempted to create my own spec macro for this. But I also 
> thought, maybe Spec itself should have such a spec.
>
> (s/error  )
>
> What do people think?
>

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

An Error spec?

[Didier]

I've started to see a pattern in my spec like this:

(s/or :success string?
  :error ::error)

And I've been tempted to create my own spec macro for this. But I also 
thought, maybe Spec itself should have such a spec.

(s/error  )

What do people think?

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Mail queue thread

[James Reeves]

On Thu, 25 Oct 2018 at 21:47,  wrote:

> It seems that I somewhat over-engineered my queue :-) I don't know any
> Java so this is very helpful!
>

When it comes to Java concurrency, ExecutorService and BlockingQueue cover
a lot of bases.


> If I want to know the result of the email send within the worker
> thread, would you recommend sending it back through a channel?
>

You can actually get back a return value directly, though it requires a
wrapper function:

(defn submit [^java.util.concurrent.ExecutorService executor, ^Callable f]
  (.submit executor f))

(let [executor (java.util.concurrent.Executors/newFixedThreadPool 32)]
  result   (submit executor #(+ 1 1))]
  (prn @result))

In the above example we're using ".submit" rather than ".execute", which
tells the executor we care about the return value from the function and
want it back. It returns it as a future, result, which we can deref as
@result. When a future is dereferenced in Clojure, the current threat
blocks until it's receives a value.

The purpose of the "submit" function is to force the executor through type
hints to treat the function as a Callable with a return value. I wasn't
able to get it working without this wrapper.

One purpose of my queue was to be able to put urgent messages at the front
> of the queue. I guess that using a threadpool with workers does not give me
> any guarantee that the email is delivered immediately, since X emails may
> already have been added but not yet sent. I guess that one one use a
> dedicated thread(pool) for urgent messages?
>

Yes, you could do that. You could also use a priority queue with the
PriorityBlockingQueue class. A priority queue is a queue with a comparator
that allows you to control ordering, shunting certain items to the front.

There's a library called Claypoole
 that packages all
these Java classes into nice Clojure functions, and it includes a priority
queue so I'd advise looking into it. I haven't tried it myself, but it
looks like you can do something like:

(require '[com.climate.claypoole :as cp])

(let [pool(cp/priority-threadpool 32)
  result1 (cp/future (cp/with-priority pool 1000) (send-email
important-mail))
  result2 (cp/future (cp/with-priority pool 0)(send-email
unimportant-mail))]
  (prn @result1)
  (prn @result2))

Which is probably easier than messing around with the executor classes
directly.

-- 
James Reeves
booleanknot.com

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Mail queue thread

[brjann]

It seems that I somewhat over-engineered my queue :-) I don't know any Java
so this is very helpful!

If I want to know the result of the email send within the worker
thread, would you recommend sending it back through a channel?

One purpose of my queue was to be able to put urgent messages at the front
of the queue. I guess that using a threadpool with workers does not give me
any guarantee that the email is delivered immediately, since X emails may
already have been added but not yet sent. I guess that one one use a
dedicated thread(pool) for urgent messages?

Thanks,
Brjánn

On Thu, 25 Oct 2018 at 20:16, James Reeves  wrote:

> On Thu, 25 Oct 2018 at 19:01,  wrote:
>
>> Thank you for the thorough explanation! So if I understand you correctly,
>> if no calls are made to the executor thread, it will just rest silently
>> until queue-email is called again - not consuming any server resources.
>>
>
> That's essentially correct.
>
> Threads do take up a small amount of memory even when idle (IIRC its
> around 1M per thread), but unless you're dealing with thousands of threads
> or you're dealing with very constrained memory, this can be ignored.
> Creating a thread is relatively expensive, which is why they're often kept
> around for reuse in thread pools.
>
>
>> Are there any recommendations for the number of worker threads when each
>> does IO?
>>
>
> It depends entirely on the external service you're communicating with. My
> advice would be to choose a relatively small default, say 32 or under, and
> then make it configurable from an environment variable or system property.
> That way you can tune the service without changing the code.
>
> (def mail-pool-size
>   (Integer/parseInt (or (System/getenv "MAIL_POOL_SIZE") "32")))
>
> --
> James Reeves
> booleanknot.com
>
> --
> You received this message because you are subscribed to the Google
> Groups "Clojure" group.
> To post to this group, send email to clojure@googlegroups.com
> Note that posts from new members are moderated - please be patient with
> your first post.
> To unsubscribe from this group, send email to
> clojure+unsubscr...@googlegroups.com
> For more options, visit this group at
> http://groups.google.com/group/clojure?hl=en
> ---
> You received this message because you are subscribed to the Google Groups
> "Clojure" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to clojure+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

RE: [ANN] 1.10.0-beta4

[Sean Corfield]

We have had this in production for all processes for about 24 hours so far. No 
problems found.

Sean Corfield -- (970) FOR-SEAN -- (904) 302-SEAN
An Architect's View -- http://corfield.org/

"If you're not annoying somebody, you're not really alive."
-- Margaret Atwood


From: clojure@googlegroups.com  on behalf of Alex 
Miller 
Sent: Monday, October 22, 2018 2:52:52 PM
To: Clojure
Subject: [ANN] 1.10.0-beta4

1.10.0-beta4 is now available.

You can try it with clj using:

  clj -Sdeps '{:deps {org.clojure/clojure {:mvn/version "1.10.0-beta4"}}}'

Changes in 1.10.0-beta4:

  *   CLJ-2417 sort and sort-by 
should retain meta

You can read the full 1.10 changelog here: 
https://github.com/clojure/clojure/blob/master/changes.md

--
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
---
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to 
clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Mail queue thread

[James Reeves]

On Thu, 25 Oct 2018 at 19:01,  wrote:

> Thank you for the thorough explanation! So if I understand you correctly,
> if no calls are made to the executor thread, it will just rest silently
> until queue-email is called again - not consuming any server resources.
>

That's essentially correct.

Threads do take up a small amount of memory even when idle (IIRC its around
1M per thread), but unless you're dealing with thousands of threads or
you're dealing with very constrained memory, this can be ignored. Creating
a thread is relatively expensive, which is why they're often kept around
for reuse in thread pools.


> Are there any recommendations for the number of worker threads when each
> does IO?
>

It depends entirely on the external service you're communicating with. My
advice would be to choose a relatively small default, say 32 or under, and
then make it configurable from an environment variable or system property.
That way you can tune the service without changing the code.

(def mail-pool-size
  (Integer/parseInt (or (System/getenv "MAIL_POOL_SIZE") "32")))

-- 
James Reeves
booleanknot.com

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Mail queue thread

[brjann]

Hi James,

Thank you for the thorough explanation! So if I understand you correctly,
if no calls are made to the executor thread, it will just rest silently
until queue-email is called again - not consuming any server resources. Are
there any recommendations for the number of worker threads when each does
IO?

Thanks,
Brjánn

On Thu, 25 Oct 2018 at 19:45, James Reeves  wrote:

> The Java executor classes are essentially a queue backed by a thread pool
> of workers.
>
> So suppose we create a new executor:
>
> (def executor
>   (java.util.concurrent.Executors/newFixedThreadPool 32))
>
> This creates an ExecutorService object with a pool of 32 worker threads.
> We can pass it a zero-argument function to execute:
>
> (.execute executor (fn [] (prn "Hello World")))
>
> This function is placed on a queue until one of the 32 worker threads is
> available to process it. In this case the function prints a message, but
> you could write something to send an email instead:
>
> (defn queue-email [email]
>   (.execute executor (fn [] (send-email email)))
>
> If we execute "queue-email" 10,000 times, then 10,000 "send-email" jobs
> will be passed to the executor. The executor will pass them onto its 32
> workers, effectively processing 32 at a time until it exhausts the queue.
>
> If you want fewer emails to be sent at once, you can reduce the worker
> thread pool size. In my earlier example, I used an executor with only one
> worker thread.
>
> On Thu, 25 Oct 2018 at 18:17,  wrote:
>
>> Hi James,
>>
>> Thanks! How would one create a thread that continuously monitors a mail
>> queue and sends any mails in it? Or do you mean that I would create one
>> thread per mail that I want to send? Wouldn't that create a problem if I'm
>> about to send 1 mails in one go?
>>
>> And thanks for the heads up regarding indefinite blocking - I'll make
>> sure to guard against that.
>>
>> Thanks,
>> Brjánn
>>
>> On Thu, 25 Oct 2018 at 18:57, James Reeves  wrote:
>>
>>> Hi Brjánn,
>>>
>>> Executing queued jobs in a background thread is a common task, and as it
>>> happens there's a set of Java classes to do just that.
>>>
>>> (let [e (java.util.concurrent.Executors/newSingleThreadExecutor)]
>>>   (.execute e #(prn "foo"))
>>>   (.execute e #(prn "bar"))
>>>   (Thread/sleep 1000)
>>>   (.execute e #(prn "baz")))
>>>
>>> Don't worry about cleaning up a single thread. An inactive thread takes
>>> up a minimal amount of VM memory and no CPU. You're going to use up more OS
>>> resources deleting and recreating the thread.
>>>
>>> You do need to be careful that sending an email doesn't block
>>> indefinitely, but usually I/O operations come with timeouts. Just ensure
>>> you set a reasonable timeout and do something sensible when you can't send.
>>>
>>> On Thu, 25 Oct 2018 at 16:46,  wrote:
>>>
 Hi,

 First, I would like to briefly present myself to the list. I'm a
 psychologist and researcher at the Karolinska Institutet in Stockholm,
 Sweden. I do research within the field of internet-based psychological
 treatment - which is a field that has grown a lot during the last 10-15
 years. I have developed my own web platform to deliver these treatments to
 patients with different mental and medical conditions. I wrote the first
 version in PHP, and not being a professional programmer, the code base is a
 big mess after > 10 years of development.

 Two years ago, a friend introduced me to Clojure. I had never worked
 with a lisp before, but I quickly fell completely in love with it. So many
 of the issues of programming in PHP completely disappeared! Words cannot
 express my gratitude to Rich for inventing Clojure, the core team, all
 developers out there who write extremely useful libraries, and my friend
 for introducing Clojure to me.

 I decided to completely rewrite my web platform in Clojure. I've chosen
 to do a gradual move from PHP to Clojure, meaning that I replace PHP code
 with Clojure code component by component and continuously test in
 production. Like I said, I'm not a professional programmer, and this
 venture poses a lot of challenges. I try to read everything I find in books
 and on the web, but when it comes to more complex issues, such as threads
 and async programming, I feel that I end up almost guessing and with a lot
 of trial and error. I have no idea how cautious one should be when
 launching a new thread (while it completely occupy the server???) and am
 often surprised when my go blocks suddenly freeze. I feel that I am at the
 mercy of the Clojure community if I want to understand these (and many
 other!) issues.

 This leads me to the subject of this email. I've decided to migrate my
 mail queue from PHP to Clojure. In PHP, it's just a cron job that executes
 every five minutes to send all emails (and actually also SMS-messages, but
 not really relevant) that have been queued.

Re: Mail queue thread

[James Reeves]

The Java executor classes are essentially a queue backed by a thread pool
of workers.

So suppose we create a new executor:

(def executor
  (java.util.concurrent.Executors/newFixedThreadPool 32))

This creates an ExecutorService object with a pool of 32 worker threads. We
can pass it a zero-argument function to execute:

(.execute executor (fn [] (prn "Hello World")))

This function is placed on a queue until one of the 32 worker threads is
available to process it. In this case the function prints a message, but
you could write something to send an email instead:

(defn queue-email [email]
  (.execute executor (fn [] (send-email email)))

If we execute "queue-email" 10,000 times, then 10,000 "send-email" jobs
will be passed to the executor. The executor will pass them onto its 32
workers, effectively processing 32 at a time until it exhausts the queue.

If you want fewer emails to be sent at once, you can reduce the worker
thread pool size. In my earlier example, I used an executor with only one
worker thread.

On Thu, 25 Oct 2018 at 18:17,  wrote:

> Hi James,
>
> Thanks! How would one create a thread that continuously monitors a mail
> queue and sends any mails in it? Or do you mean that I would create one
> thread per mail that I want to send? Wouldn't that create a problem if I'm
> about to send 1 mails in one go?
>
> And thanks for the heads up regarding indefinite blocking - I'll make sure
> to guard against that.
>
> Thanks,
> Brjánn
>
> On Thu, 25 Oct 2018 at 18:57, James Reeves  wrote:
>
>> Hi Brjánn,
>>
>> Executing queued jobs in a background thread is a common task, and as it
>> happens there's a set of Java classes to do just that.
>>
>> (let [e (java.util.concurrent.Executors/newSingleThreadExecutor)]
>>   (.execute e #(prn "foo"))
>>   (.execute e #(prn "bar"))
>>   (Thread/sleep 1000)
>>   (.execute e #(prn "baz")))
>>
>> Don't worry about cleaning up a single thread. An inactive thread takes
>> up a minimal amount of VM memory and no CPU. You're going to use up more OS
>> resources deleting and recreating the thread.
>>
>> You do need to be careful that sending an email doesn't block
>> indefinitely, but usually I/O operations come with timeouts. Just ensure
>> you set a reasonable timeout and do something sensible when you can't send.
>>
>> On Thu, 25 Oct 2018 at 16:46,  wrote:
>>
>>> Hi,
>>>
>>> First, I would like to briefly present myself to the list. I'm a
>>> psychologist and researcher at the Karolinska Institutet in Stockholm,
>>> Sweden. I do research within the field of internet-based psychological
>>> treatment - which is a field that has grown a lot during the last 10-15
>>> years. I have developed my own web platform to deliver these treatments to
>>> patients with different mental and medical conditions. I wrote the first
>>> version in PHP, and not being a professional programmer, the code base is a
>>> big mess after > 10 years of development.
>>>
>>> Two years ago, a friend introduced me to Clojure. I had never worked
>>> with a lisp before, but I quickly fell completely in love with it. So many
>>> of the issues of programming in PHP completely disappeared! Words cannot
>>> express my gratitude to Rich for inventing Clojure, the core team, all
>>> developers out there who write extremely useful libraries, and my friend
>>> for introducing Clojure to me.
>>>
>>> I decided to completely rewrite my web platform in Clojure. I've chosen
>>> to do a gradual move from PHP to Clojure, meaning that I replace PHP code
>>> with Clojure code component by component and continuously test in
>>> production. Like I said, I'm not a professional programmer, and this
>>> venture poses a lot of challenges. I try to read everything I find in books
>>> and on the web, but when it comes to more complex issues, such as threads
>>> and async programming, I feel that I end up almost guessing and with a lot
>>> of trial and error. I have no idea how cautious one should be when
>>> launching a new thread (while it completely occupy the server???) and am
>>> often surprised when my go blocks suddenly freeze. I feel that I am at the
>>> mercy of the Clojure community if I want to understand these (and many
>>> other!) issues.
>>>
>>> This leads me to the subject of this email. I've decided to migrate my
>>> mail queue from PHP to Clojure. In PHP, it's just a cron job that executes
>>> every five minutes to send all emails (and actually also SMS-messages, but
>>> not really relevant) that have been queued.
>>>
>>> I've written a basic mock Clojure implementation with the following goals
>>> - All messages should be sent async, i..e, the web user should not have
>>> to wait while the email is being sent. -> I'm sending them in a separate
>>> thread.
>>> - I have a fear that if I have a thread dedicated only to sending
>>> emails, I'm wasting server resources. -> I want the thread to die 5 seconds
>>> after the last email in the queue has been sent.
>>>
>>> My implementation basically consists of
>>

Re: Mail queue thread

[brjann]

Hi James,

Thanks! How would one create a thread that continuously monitors a mail
queue and sends any mails in it? Or do you mean that I would create one
thread per mail that I want to send? Wouldn't that create a problem if I'm
about to send 1 mails in one go?

And thanks for the heads up regarding indefinite blocking - I'll make sure
to guard against that.

Thanks,
Brjánn

On Thu, 25 Oct 2018 at 18:57, James Reeves  wrote:

> Hi Brjánn,
>
> Executing queued jobs in a background thread is a common task, and as it
> happens there's a set of Java classes to do just that.
>
> (let [e (java.util.concurrent.Executors/newSingleThreadExecutor)]
>   (.execute e #(prn "foo"))
>   (.execute e #(prn "bar"))
>   (Thread/sleep 1000)
>   (.execute e #(prn "baz")))
>
> Don't worry about cleaning up a single thread. An inactive thread takes up
> a minimal amount of VM memory and no CPU. You're going to use up more OS
> resources deleting and recreating the thread.
>
> You do need to be careful that sending an email doesn't block
> indefinitely, but usually I/O operations come with timeouts. Just ensure
> you set a reasonable timeout and do something sensible when you can't send.
>
> On Thu, 25 Oct 2018 at 16:46,  wrote:
>
>> Hi,
>>
>> First, I would like to briefly present myself to the list. I'm a
>> psychologist and researcher at the Karolinska Institutet in Stockholm,
>> Sweden. I do research within the field of internet-based psychological
>> treatment - which is a field that has grown a lot during the last 10-15
>> years. I have developed my own web platform to deliver these treatments to
>> patients with different mental and medical conditions. I wrote the first
>> version in PHP, and not being a professional programmer, the code base is a
>> big mess after > 10 years of development.
>>
>> Two years ago, a friend introduced me to Clojure. I had never worked with
>> a lisp before, but I quickly fell completely in love with it. So many of
>> the issues of programming in PHP completely disappeared! Words cannot
>> express my gratitude to Rich for inventing Clojure, the core team, all
>> developers out there who write extremely useful libraries, and my friend
>> for introducing Clojure to me.
>>
>> I decided to completely rewrite my web platform in Clojure. I've chosen
>> to do a gradual move from PHP to Clojure, meaning that I replace PHP code
>> with Clojure code component by component and continuously test in
>> production. Like I said, I'm not a professional programmer, and this
>> venture poses a lot of challenges. I try to read everything I find in books
>> and on the web, but when it comes to more complex issues, such as threads
>> and async programming, I feel that I end up almost guessing and with a lot
>> of trial and error. I have no idea how cautious one should be when
>> launching a new thread (while it completely occupy the server???) and am
>> often surprised when my go blocks suddenly freeze. I feel that I am at the
>> mercy of the Clojure community if I want to understand these (and many
>> other!) issues.
>>
>> This leads me to the subject of this email. I've decided to migrate my
>> mail queue from PHP to Clojure. In PHP, it's just a cron job that executes
>> every five minutes to send all emails (and actually also SMS-messages, but
>> not really relevant) that have been queued.
>>
>> I've written a basic mock Clojure implementation with the following goals
>> - All messages should be sent async, i..e, the web user should not have
>> to wait while the email is being sent. -> I'm sending them in a separate
>> thread.
>> - I have a fear that if I have a thread dedicated only to sending emails,
>> I'm wasting server resources. -> I want the thread to die 5 seconds after
>> the last email in the queue has been sent.
>>
>> My implementation basically consists of
>> - An eternal go loop that receives a notification through a channel if
>> new messages have been queued
>> - The go loop checks if the mail sender thread is running. If not, it
>> starts it.
>> - The mail sender thread dies 5 secs after the last email was sent
>> - The state of the thread (running / not running) is stored in an agent
>> to avoid race conditions (i.e., starting multiple threads or not starting a
>> thread because it is running when its status is checked but stops right
>> after).
>>
>> My code is here
>> https://gist.github.com/brjann/2aef16849b9bd445374cb6b31efece60
>>
>> If any of you have had the time and energy to read this far (including
>> the code), I would be very grateful for your input.
>> - Is there a risk that my go block will hang?
>> - Have I eliminated the risk for race conditions?
>> - Do I really need to kill the thread or is there no risk for thread
>> starvation on the server (I will probably
>> - Could I use send instead of send-off? I guess that I am now using two
>> threads, one for the sender and one each time I send a message using
>> send-off.
>> - Any newbie mistakes / coding style issues?
>> - Cou

Re: Mail queue thread

[James Reeves]

Hi Brjánn,

Executing queued jobs in a background thread is a common task, and as it
happens there's a set of Java classes to do just that.

(let [e (java.util.concurrent.Executors/newSingleThreadExecutor)]
  (.execute e #(prn "foo"))
  (.execute e #(prn "bar"))
  (Thread/sleep 1000)
  (.execute e #(prn "baz")))

Don't worry about cleaning up a single thread. An inactive thread takes up
a minimal amount of VM memory and no CPU. You're going to use up more OS
resources deleting and recreating the thread.

You do need to be careful that sending an email doesn't block indefinitely,
but usually I/O operations come with timeouts. Just ensure you set a
reasonable timeout and do something sensible when you can't send.

On Thu, 25 Oct 2018 at 16:46,  wrote:

> Hi,
>
> First, I would like to briefly present myself to the list. I'm a
> psychologist and researcher at the Karolinska Institutet in Stockholm,
> Sweden. I do research within the field of internet-based psychological
> treatment - which is a field that has grown a lot during the last 10-15
> years. I have developed my own web platform to deliver these treatments to
> patients with different mental and medical conditions. I wrote the first
> version in PHP, and not being a professional programmer, the code base is a
> big mess after > 10 years of development.
>
> Two years ago, a friend introduced me to Clojure. I had never worked with
> a lisp before, but I quickly fell completely in love with it. So many of
> the issues of programming in PHP completely disappeared! Words cannot
> express my gratitude to Rich for inventing Clojure, the core team, all
> developers out there who write extremely useful libraries, and my friend
> for introducing Clojure to me.
>
> I decided to completely rewrite my web platform in Clojure. I've chosen to
> do a gradual move from PHP to Clojure, meaning that I replace PHP code with
> Clojure code component by component and continuously test in production.
> Like I said, I'm not a professional programmer, and this venture poses a
> lot of challenges. I try to read everything I find in books and on the web,
> but when it comes to more complex issues, such as threads and async
> programming, I feel that I end up almost guessing and with a lot of trial
> and error. I have no idea how cautious one should be when launching a new
> thread (while it completely occupy the server???) and am often surprised
> when my go blocks suddenly freeze. I feel that I am at the mercy of the
> Clojure community if I want to understand these (and many other!) issues.
>
> This leads me to the subject of this email. I've decided to migrate my
> mail queue from PHP to Clojure. In PHP, it's just a cron job that executes
> every five minutes to send all emails (and actually also SMS-messages, but
> not really relevant) that have been queued.
>
> I've written a basic mock Clojure implementation with the following goals
> - All messages should be sent async, i..e, the web user should not have to
> wait while the email is being sent. -> I'm sending them in a separate
> thread.
> - I have a fear that if I have a thread dedicated only to sending emails,
> I'm wasting server resources. -> I want the thread to die 5 seconds after
> the last email in the queue has been sent.
>
> My implementation basically consists of
> - An eternal go loop that receives a notification through a channel if new
> messages have been queued
> - The go loop checks if the mail sender thread is running. If not, it
> starts it.
> - The mail sender thread dies 5 secs after the last email was sent
> - The state of the thread (running / not running) is stored in an agent to
> avoid race conditions (i.e., starting multiple threads or not starting a
> thread because it is running when its status is checked but stops right
> after).
>
> My code is here
> https://gist.github.com/brjann/2aef16849b9bd445374cb6b31efece60
>
> If any of you have had the time and energy to read this far (including the
> code), I would be very grateful for your input.
> - Is there a risk that my go block will hang?
> - Have I eliminated the risk for race conditions?
> - Do I really need to kill the thread or is there no risk for thread
> starvation on the server (I will probably
> - Could I use send instead of send-off? I guess that I am now using two
> threads, one for the sender and one each time I send a message using
> send-off.
> - Any newbie mistakes / coding style issues?
> - Could this be done in a better/simpler way???
>
> (Btw, I would be very grateful for feedback on the form of my question if
> you have any. Are there better/other forums? Could I present the question
> or code in a another manner to help you understand better?)
>
> I am happy for any replies - on the list or backchannel. And I hope that
> you feel that I have not misused your inbox (I may repeat this behavior...).
>
> Thanks,
> Brjánn
>
> --
> You received this message because you are subscribed to the Google
> Groups "Clojure" group.
> To

Mail queue thread

[brjann]

Hi,

First, I would like to briefly present myself to the list. I'm a
psychologist and researcher at the Karolinska Institutet in Stockholm,
Sweden. I do research within the field of internet-based psychological
treatment - which is a field that has grown a lot during the last 10-15
years. I have developed my own web platform to deliver these treatments to
patients with different mental and medical conditions. I wrote the first
version in PHP, and not being a professional programmer, the code base is a
big mess after > 10 years of development.

Two years ago, a friend introduced me to Clojure. I had never worked with a
lisp before, but I quickly fell completely in love with it. So many of the
issues of programming in PHP completely disappeared! Words cannot express
my gratitude to Rich for inventing Clojure, the core team, all developers
out there who write extremely useful libraries, and my friend for
introducing Clojure to me.

I decided to completely rewrite my web platform in Clojure. I've chosen to
do a gradual move from PHP to Clojure, meaning that I replace PHP code with
Clojure code component by component and continuously test in production.
Like I said, I'm not a professional programmer, and this venture poses a
lot of challenges. I try to read everything I find in books and on the web,
but when it comes to more complex issues, such as threads and async
programming, I feel that I end up almost guessing and with a lot of trial
and error. I have no idea how cautious one should be when launching a new
thread (while it completely occupy the server???) and am often surprised
when my go blocks suddenly freeze. I feel that I am at the mercy of the
Clojure community if I want to understand these (and many other!) issues.

This leads me to the subject of this email. I've decided to migrate my mail
queue from PHP to Clojure. In PHP, it's just a cron job that executes every
five minutes to send all emails (and actually also SMS-messages, but not
really relevant) that have been queued.

I've written a basic mock Clojure implementation with the following goals
- All messages should be sent async, i..e, the web user should not have to
wait while the email is being sent. -> I'm sending them in a separate
thread.
- I have a fear that if I have a thread dedicated only to sending emails,
I'm wasting server resources. -> I want the thread to die 5 seconds after
the last email in the queue has been sent.

My implementation basically consists of
- An eternal go loop that receives a notification through a channel if new
messages have been queued
- The go loop checks if the mail sender thread is running. If not, it
starts it.
- The mail sender thread dies 5 secs after the last email was sent
- The state of the thread (running / not running) is stored in an agent to
avoid race conditions (i.e., starting multiple threads or not starting a
thread because it is running when its status is checked but stops right
after).

My code is here
https://gist.github.com/brjann/2aef16849b9bd445374cb6b31efece60

If any of you have had the time and energy to read this far (including the
code), I would be very grateful for your input.
- Is there a risk that my go block will hang?
- Have I eliminated the risk for race conditions?
- Do I really need to kill the thread or is there no risk for thread
starvation on the server (I will probably
- Could I use send instead of send-off? I guess that I am now using two
threads, one for the sender and one each time I send a message using
send-off.
- Any newbie mistakes / coding style issues?
- Could this be done in a better/simpler way???

(Btw, I would be very grateful for feedback on the form of my question if
you have any. Are there better/other forums? Could I present the question
or code in a another manner to help you understand better?)

I am happy for any replies - on the list or backchannel. And I hope that
you feel that I have not misused your inbox (I may repeat this behavior...).

Thanks,
Brjánn

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Releasing d2q, a simple, efficient, generally applicable engine for implementing graph-pulling API servers in Clojure. Feedback wanted!

[Tom Connors]

I scanned through the readme and the tutorial and I'm definitely interested 
to give this lib a try in my own applications.

To answer your questions:
- The value prop is clear an appealing, but you might want to add a 
comparison directly against Om-next in addition to the Pathom comparison
- No criticism of the design choices, but the resolver functions in the 
tutorial are a bit busy at a glance, especially since they are responsible 
for the wrapping w/ mfd/future and d2q/result-cell. I suppose some helper 
functions/macros could make that look cleaner. I also think namespacing the 
d2q keywords would make it easier to scan.
- I'd put query expressiveness and real-world examples (authorization, more 
complex schemas, etc) as tied for most important future work.

I'm eager to see where you take this library.

On Thursday, October 25, 2018 at 4:45:57 AM UTC-4, Val Waeselynck wrote:
>
> I'm happy to release d2q , a library 
> for implementing graph-pulling server backends, playing in the same space 
> as Lacinia or Pathom 
> , with an emphasis on simplicity, 
> generality and performance:
>
> https://github.com/alvalval/d2q
>
> This library has been used internally on a previous project for months, 
> I've just removed the application-specific parts and added documentation.
>
> Before developing this any further, I'm now looking for feedback:
>
>- 
>
>Do you find the value proposition clear and appealing?
>- 
>
>Any criticism of the design choices?
>- 
>
>I see many potential directions for future development, which are 
>important to you? (enhance query expressiveness with new features / better 
>docs / add specs / integration with GraphQL or Om.Next / more helpers for 
>integrating data sources)
>
> Cheers,
>

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[ANN] flow 0.3.1

[alex]

flow is a tiny library for declarative errors handling without monads
https://github.com/dawcs/flow

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: (type ...) vs (class ...)

[Rick Moynihan]

I think the main intention of type is to be used as a more generic version
of class, that lets clojure data participate alongside java objects etc
with multi-methods etc.

i.e.

(defmulti foo type)

(defmethod foo java.lang.String [x]
  :string-dispatched)

(defmethod foo :my-clojure-type [x]
  :my-clojure-type-dispatched)

(foo ^{:type :my-clojure-type} {:blah :blah}) ;; =>
:my-clojure-type-dispatched

derive et al also let you use keywords to augment java class hierarchies
with new types, provide multiple inheritance and priorities with
prefer-method etc.

R.

On Fri, 12 Feb 2016 at 21:37, Alan Thompson  wrote:

> Hey - Just saw something on the clojure.core/type function:
>
>
>
> (defn type
> "Returns the :type metadata of x, or its Class if none"
> {:added "1.0"
> :static true}
> [x]
> (or (get (meta x) :type) (class x)))
>
>
> I have never seen this before, and it appears the :type metadata is not
> used in the clojure.core source code itself.  Is this ever used for
> anything or is it just a vestigial remnant from long ago?
>
> Alan
>
> --
> You received this message because you are subscribed to the Google
> Groups "Clojure" group.
> To post to this group, send email to clojure@googlegroups.com
> Note that posts from new members are moderated - please be patient with
> your first post.
> To unsubscribe from this group, send email to
> clojure+unsubscr...@googlegroups.com
> For more options, visit this group at
> http://groups.google.com/group/clojure?hl=en
> ---
> You received this message because you are subscribed to the Google Groups
> "Clojure" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to clojure+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Releasing d2q, a simple, efficient, generally applicable engine for implementing graph-pulling API servers in Clojure. Feedback wanted!

[Val Waeselynck]

I'm happy to release d2q , a library 
for implementing graph-pulling server backends, playing in the same space 
as Lacinia or Pathom 
, with an emphasis on simplicity, 
generality and performance:

https://github.com/alvalval/d2q

This library has been used internally on a previous project for months, 
I've just removed the application-specific parts and added documentation.

Before developing this any further, I'm now looking for feedback:

   - 
   
   Do you find the value proposition clear and appealing?
   - 
   
   Any criticism of the design choices?
   - 
   
   I see many potential directions for future development, which are 
   important to you? (enhance query expressiveness with new features / better 
   docs / add specs / integration with GraphQL or Om.Next / more helpers for 
   integrating data sources)
   
Cheers,

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.