Re: How to Process Files in Background

2013-02-11 Thread Max Penet 
Hi,

I made a small library a while ago to help with this. It wraps different 
java executors and has a version of clojure future and future-call that 
allows you to pass an executor as argument:

https://github.com/mpenet/knit

(require '[qbits.knit :as k]) 

(def x (k/executor :fixed :num-threads 10)

(k/future x (do-something) (do-more))

;; or you can just use k/execute with a callable:
(k/execute x (fn [] ...))


- Max


On Monday, February 11, 2013 5:50:03 PM UTC+1, Ari wrote:
>
> Hi, 
>
> I'd like my web application to process uploaded text files in the 
> background; in which ways can I accomplish this? I thought of using a 
> message queue like 0mq to push jobs to workers within the web app itself. 
> While relatively straight forward this option seems a bit involved. 
> Suggestions/recommendations appreciated.
>
> Best,
> Ari 
>

-- 
-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: How to Process Files in Background

2013-02-11 Thread Timothy Baldridge 
For that, look up Executors in Java. They'll have exactly what you are
looking for. Create a new fixed sized executor:
http://docs.oracle.com/javase/1.5.0/docs/api/java/util/concurrent/Executors.html

They expect Runnable objects, but clojure Fns implement Runnable, so you
can simply pass the executor a (fn []) that wraps up all the data needed
for the task.

Timothy


On Mon, Feb 11, 2013 at 2:48 PM, Ari  wrote:

> From what I understand, futures would work. But is there a way to limit
> thread creation to a predetermined pool size? I'm concerned as more files
> are uploaded the continual future calls would soon overwhelm the system.
>
> -Ari
>
>
> On Monday, February 11, 2013 11:54:19 AM UTC-5, Zack Maril wrote:
>>
>> Is there any reason why a future wouldn't work?
>> http://clojuredocs.org/**clojure_core/clojure.core/**future
>> -Zack
>>
>> On Monday, February 11, 2013 8:50:03 PM UTC+4, Ari wrote:
>>>
>>> Hi,
>>>
>>> I'd like my web application to process uploaded text files in the
>>> background; in which ways can I accomplish this? I thought of using a
>>> message queue like 0mq to push jobs to workers within the web app itself.
>>> While relatively straight forward this option seems a bit involved.
>>> Suggestions/recommendations appreciated.
>>>
>>> Best,
>>> Ari
>>>
>>  --
> --
> You received this message because you are subscribed to the Google
> Groups "Clojure" group.
> To post to this group, send email to clojure@googlegroups.com
> Note that posts from new members are moderated - please be patient with
> your first post.
> To unsubscribe from this group, send email to
> clojure+unsubscr...@googlegroups.com
> For more options, visit this group at
> http://groups.google.com/group/clojure?hl=en
> ---
> You received this message because you are subscribed to the Google Groups
> "Clojure" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to clojure+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/groups/opt_out.
>
>
>



-- 
“One of the main causes of the fall of the Roman Empire was that–lacking
zero–they had no way to indicate successful termination of their C
programs.”
(Robert Firth)

-- 
-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: How to Process Files in Background

2013-02-11 Thread Ari 
>From what I understand, futures would work. But is there a way to limit 
thread creation to a predetermined pool size? I'm concerned as more files 
are uploaded the continual future calls would soon overwhelm the system.

-Ari

On Monday, February 11, 2013 11:54:19 AM UTC-5, Zack Maril wrote:
>
> Is there any reason why a future wouldn't work? 
> http://clojuredocs.org/clojure_core/clojure.core/future
> -Zack
>
> On Monday, February 11, 2013 8:50:03 PM UTC+4, Ari wrote:
>>
>> Hi, 
>>
>> I'd like my web application to process uploaded text files in the 
>> background; in which ways can I accomplish this? I thought of using a 
>> message queue like 0mq to push jobs to workers within the web app itself. 
>> While relatively straight forward this option seems a bit involved. 
>> Suggestions/recommendations appreciated.
>>
>> Best,
>> Ari 
>>
>

-- 
-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: How to Process Files in Background

2013-02-11 Thread AtKaaZ 
someone posted this [1] earlier, I think it might help you

[1]
http://clojure-doc.org/articles/language/concurrency_and_parallelism.html


On Mon, Feb 11, 2013 at 5:50 PM, Ari  wrote:

> Hi,
>
> I'd like my web application to process uploaded text files in the
> background; in which ways can I accomplish this? I thought of using a
> message queue like 0mq to push jobs to workers within the web app itself.
> While relatively straight forward this option seems a bit involved.
> Suggestions/recommendations appreciated.
>
> Best,
> Ari
>
> --
> --
> You received this message because you are subscribed to the Google
> Groups "Clojure" group.
> To post to this group, send email to clojure@googlegroups.com
> Note that posts from new members are moderated - please be patient with
> your first post.
> To unsubscribe from this group, send email to
> clojure+unsubscr...@googlegroups.com
> For more options, visit this group at
> http://groups.google.com/group/clojure?hl=en
> ---
> You received this message because you are subscribed to the Google Groups
> "Clojure" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to clojure+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/groups/opt_out.
>
>
>



-- 
Please correct me if I'm wrong or incomplete,
even if you think I'll subconsciously hate it.

-- 
-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: How to Process Files in Background

2013-02-11 Thread Zack Maril 
Is there any reason why a future wouldn't work? 
http://clojuredocs.org/clojure_core/clojure.core/future
-Zack

On Monday, February 11, 2013 8:50:03 PM UTC+4, Ari wrote:
>
> Hi, 
>
> I'd like my web application to process uploaded text files in the 
> background; in which ways can I accomplish this? I thought of using a 
> message queue like 0mq to push jobs to workers within the web app itself. 
> While relatively straight forward this option seems a bit involved. 
> Suggestions/recommendations appreciated.
>
> Best,
> Ari 
>

-- 
-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

How to Process Files in Background

2013-02-11 Thread Ari 
Hi, 

I'd like my web application to process uploaded text files in the 
background; in which ways can I accomplish this? I thought of using a 
message queue like 0mq to push jobs to workers within the web app itself. 
While relatively straight forward this option seems a bit involved. 
Suggestions/recommendations appreciated.

Best,
Ari 

-- 
-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.