I'm glad that sort of thing doesn't just happen to me :) Sent from my iPhone
> On 10 Aug 2015, at 00:44, Lawrence Krubner <[email protected]> wrote: > > > Okay, I am an idiot. I had the "directory watch" setup to watch the directory > where the uberjar was built, so of course it was triggered thousands of time > while the uberjar was built, since I guess every new byte added to the > uberjar triggers the change event. > > So I set my app to listen on the final directory, where the final uberjar is > moved, just once, when the uberjar is complete, and now the app behaves the > way I was expecting it to. > > >> On Sunday, August 9, 2015 at 1:13:56 PM UTC-4, Lawrence Krubner wrote: >> >> Let's assume for a moment that, for some strange reason, the "directory >> watching" strategy won't work here. Does anyone have any other suggestions? >> I am thinking there must be a simple way to setup Jenkins and Supervisord to >> run these apps, but what I'm doing keeps getting more and more complex, >> which is typically a warning sign that I'm on the wrong track. >> >> >> >> >>> On Saturday, August 8, 2015 at 5:24:28 PM UTC-4, Lawrence Krubner wrote: >>> I feel stupid, but I have not been able to track this down. >>> >>> The background is that I have Jenkins running on the server, and when >>> triggered it pulls code from Github, compiles it, and then moves the final >>> uberjar to the directory where I keep all the uberjars that run on this >>> server. Then I have an app that should kill all currently running >>> instances. Then Supervisord should step in and restart the instances, using >>> the new uberjars for the new instances. >>> >>> All of this works, but I have been doing one bit by hand: "kill all >>> currently running instances". I log in as root and run a script that kills >>> the old instances. But just today I wrote a quick Clojure app to automate >>> this for me (the whole app is 50 lines of code, so this is a very small >>> app). >>> >>> To trigger the main action, I thought the app should set a watcher on the >>> final directory where Jenkins moves the uberjars to. When Jenkins moves a >>> new uberjar to the directory, the watch would be triggered and then it >>> could run the "kill all currently running instances" script. However, I >>> assumed that the ""kill all currently running instances"" script would be >>> triggered once each time the uberjar was updated, but instead it seems to >>> be triggered infinitely. Does a JVM app, or a Clojure app, change the dir >>> its in, on launch? Or is the problem in my own code? I'm using Chris >>> Zheng's excellent Hara library for the Watch. This is the main function of >>> my app, which is called on startup: >>> >>> (defn establish-watchers [] >>> "The config contains a hashmap which has a 'watchers' key, which contains >>> a vector that holds hashmaps with 2 keys, one pointing to the directory >>> that should be watched and the other pointing to the command that should be >>> run when the watch is triggered." >>> (let [config (read-string (slurp "/etc/fiit.edn")) >>> watchers (:watchers config)] >>> (doseq [w watchers] >>> (common-watch/add (clojure.java.io/file (:dir-to-watch w)) (keyword >>> (:dir-to-watch w)) >>> (fn [f k _ [cmd file]] >>> (pprint/pprint (:out (sh/sh (:action-to-do w))))) >>> {:types #{:create :modify} >>> :recursive false >>> :async false})))) >>> >>> So if I log in as root and start the app like this: >>> >>> /usr/bin/java -jar /home/jenkins/fiit/fiit-1-standalone.jar >>> >>> Then at the terminal I see this, which I expect: >>> >>> [/home/jenkins/sarr] >>> [/home/jenkins/food_for_all] >>> >>> Those are the 2 directories that it is watching. >>> >>> Then if I trigger Jenkins for the sarr app, Jenkins will pull the sarr code >>> from Github, rebuild the sarr uberjar, and move the uberjar to >>> /home/jenkins/sarr. >>> >>> All of that works just great. And then my app sees there has been a change >>> in /home/jenkins/sarr, and so it calls the command to kill all existing >>> instances of the sarr app. However, instead of doing this once, it starts >>> doing so an infinite number of times. At the terminal I see: >>> >>> (sarr is a Java app, not a Clojure app) >>> >>> "We will kill these processes: \n" >>> "We will kill these processes: \n" >>> "We will kill these processes: \n" >>> "We will kill these processes: \n" >>> "We will kill these processes: \n" >>> "We will kill these processes: \n" >>> "We will kill these processes: \n" >>> "We will kill these processes: \nroot 25593 0.0 0.1 5813460 19452 ? >>> Sl 20:28 0:00 /usr/bin/java -cp /home/jenkins/sarr/sarr.jar >>> com.candle.sarr.Main\nroot 25595 0.0 0.1 5813460 19564 ? Sl >>> 20:28 0:00 /usr/bin/java -cp /home/jenkins/sarr/sarr.jar >>> com.candle.sarr.Main\n25593\n25595\n" >>> "We will kill these processes: \n" >>> "We will kill these processes: \n" >>> "We will kill these processes: \n25694\n" >>> "We will kill these processes: \n" >>> "We will kill these processes: \n" >>> "We will kill these processes: \n" >>> "We will kill these processes: \n" >>> "We will kill these processes: \n" >>> "We will kill these processes: \n" >>> "We will kill these processes: \n" >>> "We will kill these processes: \n" >>> "We will kill these processes: \n" >>> "We will kill these processes: \n" >>> "We will kill these processes: \n" >>> "We will kill these processes: \n" >>> "We will kill these processes: \n" >>> "We will kill these processes: \n" >>> "We will kill these processes: \n" >>> "We will kill these processes: \n" >>> "We will kill these processes: \n" >>> "We will kill these processes: \n" >>> "We will kill these processes: \n" >>> "We will kill these processes: \n" >>> "We will kill these processes: \n" >>> "We will kill these processes: \n" >>> "We will kill these processes: \n" >>> "We will kill these processes: \n" >>> "We will kill these processes: \nroot 26239 0.0 0.1 5813460 19452 ? >>> Sl 20:28 0:00 /usr/bin/java -cp /home/jenkins/sarr/sarr.jar >>> com.candle.sarr.Main\n26239\n" >>> "We will kill these processes: \n" >>> "We will kill these processes: \n" >>> "We will kill these processes: \nroot 26319 0.0 0.1 5811252 15484 ? >>> Sl 20:28 0:00 /usr/bin/java -cp /home/jenkins/sarr/sarr.jar >>> com.candle.sarr.Main\n26319\n" >>> "We will kill these processes: \n" >>> "We will kill these processes: \n" >>> "We will kill these processes: \n" >>> "We will kill these processes: \n" >>> "We will kill these processes: \n" >>> "We will kill these processes: \n" >>> "We will kill these processes: \n" >>> "We will kill these processes: \n" >>> "We will kill these processes: \n" >>> >>> >>> I have Supervisord set to run 3 instances of the sarr app, so you can see >>> at first it it is killing these PIDs: >>> >>> 25593 >>> 25595 >>> 25694 >>> >>> Supervisord restarts these almost instantly, so you can see these processes >>> are launched and instantly killed, all on the same line of output: >>> >>> "We will kill these processes: \nroot 26239 0.0 0.1 5813460 19452 ? >>> Sl 20:28 0:00 /usr/bin/java -cp /home/jenkins/sarr/sarr.jar >>> com.candle.sarr.Main\n26239\n" >>> >>> "We will kill these processes: \nroot 26319 0.0 0.1 5811252 15484 ? >>> Sl 20:28 0:00 /usr/bin/java -cp /home/jenkins/sarr/sarr.jar >>> com.candle.sarr.Main\n26319\n" >>> >>> This goes on for as long as I allow it (it takes Supervisord a long time to >>> quit, because when I am developing new software, I set startretries=500, in >>> Supervisord, exactly so I can experiment like this). >>> >>> I looked here to try to figure out what was going on: >>> >>> https://github.com/zcaudate/hara/blob/55b4eda688a4e616f03bf3740419a69b5c5dd658/src/hara/io/watch.clj >>> >>> As near as I can tell, the main action happens here: >>> >>> (defn run-watcher [watcher] >>> (let [^java.nio.file.WatchKey wkey >>> (.take ^java.nio.file.WatchService (:service watcher))] >>> (doseq [^java.nio.file.WatchEvent event (.pollEvents wkey) >>> :when (not= (.kind event) >>> StandardWatchEventKinds/OVERFLOW)] >>> (let [kind (.kind event) >>> ^java.nio.file.Path path (.watchable wkey) >>> ^java.nio.file.Path context (.context event) >>> ^java.nio.file.Path res-path (.resolve path context) >>> ^java.io.File file (.toFile res-path)] >>> (if (and (= kind StandardWatchEventKinds/ENTRY_CREATE) >>> (.isDirectory file) >>> (-> watcher :options :recursive)) >>> (register-sub-directory watcher (.getPath file))) >>> (if (.isFile file) >>> (process-event watcher kind file)))) >>> (.reset wkey) >>> (recur watcher))) >>> >>> >>> Again, maybe I am being stupid, but it looks to me like the callback is >>> called once per event triggered on that directory. So why would the >>> callback seem to be called an infinite number of times? Does starting or >>> stopping the app trigger a change in the directory (thus triggering the >>> callback again)? > > -- > You received this message because you are subscribed to the Google > Groups "Clojure" group. > To post to this group, send email to [email protected] > Note that posts from new members are moderated - please be patient with your > first post. > To unsubscribe from this group, send email to > [email protected] > For more options, visit this group at > http://groups.google.com/group/clojure?hl=en > --- > You received this message because you are subscribed to the Google Groups > "Clojure" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected]. > For more options, visit https://groups.google.com/d/optout. -- You received this message because you are subscribed to the Google Groups "Clojure" group. To post to this group, send email to [email protected] Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to [email protected] For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups "Clojure" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. For more options, visit https://groups.google.com/d/optout.
