Once again, my lack of knowledge of Java trips me up. Manifold relies on Dirigiste, which relies on Java's Executor Service. I see a bit here:
http://www.nurkiewicz.com/2014/11/executorservice-10-tips-and-tricks.html Nurkiewicz writes: "I got bitten by that too many times: it won't print *anything*. No sign of java.lang.ArithmeticException: / by zero, nothing. Thread pool just swallows this exception, as if it never happened. If it was a good'ol java.lang.Thread created from scratch, UncaughtExceptionHandler <https://docs.oracle.com/javase/8/docs/api/java/lang/Thread.UncaughtExceptionHandler.html> could work. " I suspect I'm facing something like that. On Monday, July 10, 2017 at 8:28:03 PM UTC-4, lawrence...@gmail.com wrote: > > By the way, this code works fine if I go into a (doseq) a level above > enqueue, and then put each individual row onto the stream. Then the code > loops over all of the rows. But that seems to defeat the whole point of > using something like Manifold. I want to be able to put the whole lazy-seq > on the stream. That is supposed to work, yes? > > > On Monday, July 10, 2017 at 8:18:07 PM UTC-4, lawrence...@gmail.com wrote: >> >> I'm using Zach Tellman's excellent Manifold library, though I admit I >> don't fully understand it. >> >> My code queries a MySQL database and then needs to do some processing on >> each row retrieved. I copy-and-pasted some code from the documentation for >> Manifold: >> >> >> ;; 2017-07-10 -- we want a thread pool. I'm arbitrarily choosing 200 >> threads. >> ;; query_database/start will pull data from the database and dump it onto >> a >> ;; stream below, at which point each row should be assigned to one of the >> rows >> ;; on our thread pool. >> (def executor (me/fixed-thread-executor 200)) >> >> >> (defn enqueue >> [sequence-from-database] >> (slingshot/try+ >> (println "the type of the object from the database: " (type >> sequence-from-database)) >> (->> (ms/->source sequence-from-database) >> (ms/onto executor) >> (ms/map api/query)) >> (catch Object o >> (println " message queue is not happy about the message we were >> given") >> (errors/error o "" " we tried to put something on the message queue, >> but we got an error ")))) >> >> The line where I print out the type assures that I'm dealing with a >> LazySeq. >> >> The code prints out the type and the first row: >> >> >> the type of the object from the database: clojure.lang.LazySeq >> >> in query_api/query {:profile_id 2, :profile_name Mike Shaw Automotive >> Group, :headquarters_addr1 90 Madison St., :headquarters_city Denver, >> :headquarters_state_code CO, :headquarters_country_code US, :url >> mikeshawauto.com} >> >> and then it stops. I assume there must be an Exception or Error >> happening, but I can't find it. I've added as many general Catch clauses as >> I could: >> >> >> (defn query >> [row & args] >> >> (println " in query_api/query " row) >> >> (let [config (if args >> (first args)) >> ] >> ;; 2017-03-30 -- the API is overwhelmed and all I get is Socket >> Timeout errors >> (Thread/sleep 300) >> >> (slingshot/try+ >> (call-api row) >> (catch Object o >> (println " error : " o) >> >> ;;(errors/error o row " we tried to call-api, but we got this >> error ") >> >> >> ) >> >> (catch Error e >> (println " there Error: " e)) >> ))) >> >> So if I do: >> >> java -jar scan-database-find-similar-items-standalone.jar >> >> >> The code runs till it prints out the first row from the database, and >> then it stops. Nothing else happens. There are no error messages. >> >> What did I miss? >> >> >> >> >> >> >> >> -- You received this message because you are subscribed to the Google Groups "Clojure" group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups "Clojure" group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
