How to Process Files in Background

2013-02-11 Thread Ari
Hi, I'd like my web application to process uploaded text files in the background; in which ways can I accomplish this? I thought of using a message queue like 0mq to push jobs to workers within the web app itself. While relatively straight forward this option seems a bit involved.

Re: How to Process Files in Background

2013-02-11 Thread Zack Maril
Is there any reason why a future wouldn't work? http://clojuredocs.org/clojure_core/clojure.core/future -Zack On Monday, February 11, 2013 8:50:03 PM UTC+4, Ari wrote: Hi, I'd like my web application to process uploaded text files in the background; in which ways can I accomplish this? I

Re: How to Process Files in Background

2013-02-11 Thread AtKaaZ
someone posted this [1] earlier, I think it might help you [1] http://clojure-doc.org/articles/language/concurrency_and_parallelism.html On Mon, Feb 11, 2013 at 5:50 PM, Ari ari.brandeis.k...@gmail.com wrote: Hi, I'd like my web application to process uploaded text files in the background;

Re: How to Process Files in Background

2013-02-11 Thread Ari
From what I understand, futures would work. But is there a way to limit thread creation to a predetermined pool size? I'm concerned as more files are uploaded the continual future calls would soon overwhelm the system. -Ari On Monday, February 11, 2013 11:54:19 AM UTC-5, Zack Maril wrote: Is

Re: How to Process Files in Background

2013-02-11 Thread Timothy Baldridge
For that, look up Executors in Java. They'll have exactly what you are looking for. Create a new fixed sized executor: http://docs.oracle.com/javase/1.5.0/docs/api/java/util/concurrent/Executors.html They expect Runnable objects, but clojure Fns implement Runnable, so you can simply pass the

Re: How to Process Files in Background

2013-02-11 Thread Max Penet
Hi, I made a small library a while ago to help with this. It wraps different java executors and has a version of clojure future and future-call that allows you to pass an executor as argument: https://github.com/mpenet/knit (require '[qbits.knit :as k]) (def x (k/executor :fixed