> Is the problem possibly a difference between your > compilation environment and your deploy env?
This seems to be proven by the fact that upgrading Java on the EC2 instance fixed the problem. On Wednesday, January 16, 2019 at 3:49:41 PM UTC-5, Chris Nuernberger wrote: > > Are you using aot? > > We have used that durable queue with 1.8. In fact, we have a > compatibility later that allows you to move from the durable queue to an > Aws queue: > (mostly undocumented) > > https://github.com/techascent/tech.queue > > Is the problem possibly a difference between your compilation environment > and your deploy env? > > > > On Wed, Jan 16, 2019, 1:29 PM <lawrence...@gmail.com <javascript:> wrote: > >> So, I upgraded to Java 11, and now everything works. So I guess this was >> a version conflict. >> >> Just curious, but is there a way for Factual to make durable-queue to >> tell Leiningen that Java 11 is necessary? >> >> >> >> On Wednesday, January 16, 2019 at 3:17:49 PM UTC-5, lawrence...@gmail.com >> wrote: >>> >>> On the new EC2 instance, running Ubuntu: >>> >>> java -version >>> openjdk version "1.8.0_191" >>> OpenJDK Runtime Environment (build >>> 1.8.0_191-8u191-b12-0ubuntu0.18.04.1-b12) >>> OpenJDK 64-Bit Server VM (build 25.191-b12, mixed mode) >>> >>> Is it possible this version does not have the checksum signature that >>> durable-queue is looking for? >>> >>> I'm thinking this is some kind of version problem. >>> >>> >>> >>> >>> >>> >>> On Wednesday, January 16, 2019 at 2:50:14 PM UTC-5, >>> lawrence...@gmail.com wrote: >>>> >>>> Sorry, I'm an idiot. The real error was when I called put! >>>> >>>> I don't understand this error: >>>> >>>> INFO: java.lang.NoSuchMethodError: java.util.zip.Checksum.update([B)V >>>> java.lang.NoSuchMethodError: java.util.zip.Checksum.update([B)V >>>> at durable_queue$checksum.invokeStatic (durable_queue.clj:64) >>>> durable_queue$checksum.invokePrim (durable_queue.clj:-1) >>>> durable_queue.TaskSlab.append_to_slab_BANG_ (durable_queue.clj:314) >>>> durable_queue$queues$reify__6549$slab_BANG___6570.invoke >>>> (durable_queue.clj:702) >>>> durable_queue$queues$reify__6549$fn__6575.invoke >>>> (durable_queue.clj:719) >>>> durable_queue$queues$reify__6549.put_BANG_ (durable_queue.clj:717) >>>> durable_queue$queues$reify__6549.put_BANG_ (durable_queue.clj:732) >>>> humongorous_nlp.core$cycle_to_database$fn__13673$fn__13693.invoke >>>> (core.clj:665) >>>> humongorous_nlp.core$cycle_to_database$fn__13673.invoke >>>> (core.clj:663) >>>> clojure.lang.AFn.run (AFn.java:22) >>>> java.util.concurrent.Executors$RunnableAdapter.call >>>> (Executors.java:511) >>>> java.util.concurrent.FutureTask.runAndReset (FutureTask.java:308) >>>> >>>> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301 >>>> >>>> (ScheduledThreadPoolExecutor.java:180) >>>> >>>> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run >>>> (ScheduledThreadPoolExecutor.java:294) >>>> java.util.concurrent.ThreadPoolExecutor.runWorker >>>> (ThreadPoolExecutor.java:1149) >>>> java.util.concurrent.ThreadPoolExecutor$Worker.run >>>> (ThreadPoolExecutor.java:624) >>>> java.lang.Thread.run (Thread.java:748) >>>> >>>> >>>> >>>> >>>> On Wednesday, January 16, 2019 at 1:47:21 PM UTC-5, >>>> lawrence...@gmail.com wrote: >>>>> >>>>> I was away from Clojure for a year and I missed it. I am pleased to be >>>>> back. But I've forgotten certain common errors. I feel like this is >>>>> something I used to know but now I've lost the knowledge. >>>>> >>>>> I'm using Factual's durable-queue to put a step inbetween the import >>>>> of large JSON files, and their writes to the database. This works fine on >>>>> my local MacBook Pro, but when I move to an EC2 instance, I'm instead >>>>> getting time-outs when durable-queue tries to read from the queue. >>>>> >>>>> At start up the app creates a few of these workers, which run for as >>>>> long as the app is running: >>>>> >>>>> (defn worker >>>>> [from-topics-to-persistence-queue current-database-connection] >>>>> (slingshot/try+ >>>>> (loop [message (durable/take! from-topics-to-persistence-queue >>>>> :message 60000 :timed-out!)] >>>>> (slingshot/try+ >>>>> (log-seq " the message in the work function " message) >>>>> (when (= (type message) durable_queue.Task) >>>>> (advance message current-database-connection)) >>>>> (catch Object o >>>>> (log-seq "error in worker function") >>>>> (durable/retry! message) >>>>> (log-seq o))) >>>>> (recur (durable/take! from-topics-to-persistence-queue :message >>>>> 60000 :timed-out!))) >>>>> (catch Object o >>>>> (error o) >>>>> (slingshot/throw+ { >>>>> :type worker >>>>> :error o >>>>> :from-topics-to-persistence-queue >>>>> from-topics-to-persistence-queue >>>>> :current-database-connection >>>>> current-database-connection >>>>> } >>>>> )))) >>>>> >>>>> On the EC2 instance, I see in the output: >>>>> >>>>> Jan 16, 2019 6:43:23 PM humongorous-nlp.core invoke >>>>> INFO: the message in the work function >>>>> >>>>> Jan 16, 2019 6:43:23 PM humongorous-nlp.core invoke >>>>> INFO: >>>>> >>>>> Jan 16, 2019 6:43:23 PM humongorous-nlp.core invoke >>>>> INFO: :timed-out! >>>>> >>>>> I'm using an environment var to set the path to the queue: >>>>> >>>>> PATH_TO_DURABLE_QUEUE_S3_TO_DATABASE=/tmp/s3_to_database_queue >>>>> >>>>> I feel like I ran into this problem 2 years ago, but I can't recall >>>>> the solution. Is the problem specific to AWS? >>>>> >>>>> An important clue, I think, is that I'm not getting an error on the >>>>> put!, only on the take! Why would that be? >>>>> >>>>> >>>>> --- lawrence >>>>> >>>>> >>>>> >>>>> -- >> You received this message because you are subscribed to the Google >> Groups "Clojure" group. >> To post to this group, send email to clo...@googlegroups.com >> <javascript:> >> Note that posts from new members are moderated - please be patient with >> your first post. >> To unsubscribe from this group, send email to >> clojure+u...@googlegroups.com <javascript:> >> For more options, visit this group at >> http://groups.google.com/group/clojure?hl=en >> --- >> You received this message because you are subscribed to the Google Groups >> "Clojure" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to clojure+u...@googlegroups.com <javascript:>. >> For more options, visit https://groups.google.com/d/optout. >> > -- You received this message because you are subscribed to the Google Groups "Clojure" group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups "Clojure" group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
