Re: What do Clojure developers use for recurring functions, other than at-at
+1 for at-at. I've used it recently and it works well. It uses ScheduledThreadPoolExecutor etc under the hood, and it's only 350 lines of readable code if you want to dig in. I have found its interface easy to use. As well as scheduling at specific times (the at function) or at regular intervals (every), it has the function interspaced which will schedule tasks after a specified delay after the previous call to that task has finished. It also has a way of viewing and reseting tasks in the task pool. Rowan On Monday, December 17, 2018 at 8:31:07 PM UTC, Tim Visher wrote: > > On Mon, Dec 17, 2018 at 2:54 PM > > wrote: > >> But at-at has not been updated in 6 years, so I assume it is abandoned. I >> have two questions about this: >> > > A common bit of wisdom here in the Clojure community is that time since > last update is not always (or even often) a sign of abandonment but instead > of stability. If there are many and recent open issues on at-at then maybe > it's been abandoned. Or it could just be stable. > > Disclaimer, I don't know much about at-at. > -- You received this message because you are subscribed to the Google Groups "Clojure" group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups "Clojure" group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: What do Clojure developers use for recurring functions, other than at-at
Hi, On Mon, Dec 17, 2018 at 3:46 PM wrote: > Laurens Van Houtven, good ideas, but then I'd also have to write some code > to catch documents that got lost when a process died while trying to fetch > a document from S3. If I simply check every 15 minutes, and grab everything > that has not already been stored in the database, then I automatically > fetch documents that were dropped due to error. It seems simple. In my > current case, there is no penalty for fetching the same document twice, but > it is important that we avoid missing a document. > I'm not sure why that makes my suggestions not work: you create a CloudWatch Event that just fires every 15 minutes :-) Heck, you can write one that fires on S3 write, and then another that fires on cloudwatch errors for when that process dies :-) But I'd probably just use a CloudWatch event that fires every 15 minute sand run your code as an ECS Fargate job. lvh > > > On Monday, December 17, 2018 at 2:59:22 PM UTC-5, Laurens Van Houtven > wrote: >> >> Honestly I'd use CloudWatch Timed Events to kick off a Lambda or ECS >> Fargate job (which of course you can write in Clojure) assuming you're >> using AWS yourself anyway. If you don't care about batching maybe even just >> attach a Lambda to the write-to-S3 bucket itself instead of checking every >> 15m? >> >> If you want to do it in-process, my tool of choice is ztellman's >> manifold: >> https://github.com/ztellman/manifold/blob/d67a8c1b9f1268c094895d70dbbf146521f5774b/src/manifold/time.clj >> >> On Mon, Dec 17, 2018 at 1:54 PM wrote: >> >>> I'm coming back to Clojure development after a year away. This is a fast >>> moving community and it is hard to keep up when one is not working on it >>> full time. I'm dusting off some code I wrote 2 years ago, and trying to >>> bring all the dependencies up to their current versions. >>> >>> I have a function that fetches files from an AWS S3 bucket, every 15 >>> minutes. I had previously used the at-at library for this: >>> >>> https://github.com/overtone/at-at >>> >>> But at-at has not been updated in 6 years, so I assume it is abandoned. >>> I have two questions about this: >>> >>> 1.) how else do Clojure programmers usually call recurring >>> functionality? >>> >>> 2.) I am ignorant about the JVM, so I'm afraid I have to ask, at a >>> fundamental level, how does at-at work? I know that if a function calls >>> itself recurringly, on the JVM, one eventually gets stackoverflow. So how >>> does at-at make its magic work? >>> >>> -- >>> You received this message because you are subscribed to the Google >>> Groups "Clojure" group. >>> To post to this group, send email to clo...@googlegroups.com >>> Note that posts from new members are moderated - please be patient with >>> your first post. >>> To unsubscribe from this group, send email to >>> clojure+u...@googlegroups.com >>> For more options, visit this group at >>> http://groups.google.com/group/clojure?hl=en >>> --- >>> You received this message because you are subscribed to the Google >>> Groups "Clojure" group. >>> To unsubscribe from this group and stop receiving emails from it, send >>> an email to clojure+u...@googlegroups.com. >>> For more options, visit https://groups.google.com/d/optout. >>> >> -- > You received this message because you are subscribed to the Google > Groups "Clojure" group. > To post to this group, send email to clojure@googlegroups.com > Note that posts from new members are moderated - please be patient with > your first post. > To unsubscribe from this group, send email to > clojure+unsubscr...@googlegroups.com > For more options, visit this group at > http://groups.google.com/group/clojure?hl=en > --- > You received this message because you are subscribed to the Google Groups > "Clojure" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to clojure+unsubscr...@googlegroups.com. > For more options, visit https://groups.google.com/d/optout. > -- You received this message because you are subscribed to the Google Groups "Clojure" group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups "Clojure" group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: What do Clojure developers use for recurring functions, other than at-at
I think iirc that at-at uses scheduledexecutor and is a very simple and stable library which only does one thing, but does it well and has no feature requests. I wrote schejulure which does a very similar job but with cron style time specifications rather than periodic and haven't touched it for years because it just works and even though it's used in production nobody has found any issues or needed it to do anything it doesn't already do. I think the same is probably true of at-at. Adam On Mon, 17 Dec 2018, 9:46 pm James Reeves, that does sound like the right way to go. I'll do that. > > > On Monday, December 17, 2018 at 3:31:01 PM UTC-5, James Reeves wrote: >> >> >> I'd use an executor: >> >> (ns example.main >> (:import [java.util.concurrent Executors TimeUnit])) >> >> (def scheduler >> (Executors/newScheduledThreadPool 32)) >> >> (defn fetch-files [] >> (println "Fetching files...)) >> >> (defn -main [] >> (.scheduleAtFixedRate scheduler ^Runnable fetch-files 15 15 >> TimeUnit/MINUTES)) >> >> On Mon, 17 Dec 2018 at 20:14, wrote: >> >>> I'm coming back to Clojure development after a year away. This is a fast >>> moving community and it is hard to keep up when one is not working on it >>> full time. I'm dusting off some code I wrote 2 years ago, and trying to >>> bring all the dependencies up to their current versions. >>> >>> I have a function that fetches files from an AWS S3 bucket, every 15 >>> minutes. I had previously used the at-at library for this: >>> >>> https://github.com/overtone/at-at >>> >>> But at-at has not been updated in 6 years, so I assume it is abandoned. >>> I have two questions about this: >>> >>> 1.) how else do Clojure programmers usually call recurring >>> functionality? >>> >>> 2.) I am ignorant about the JVM, so I'm afraid I have to ask, at a >>> fundamental level, how does at-at work? I know that if a function calls >>> itself recurringly, on the JVM, one eventually gets stackoverflow. So how >>> does at-at make its magic work? >>> >>> -- >>> You received this message because you are subscribed to the Google >>> Groups "Clojure" group. >>> To post to this group, send email to clo...@googlegroups.com >>> Note that posts from new members are moderated - please be patient with >>> your first post. >>> To unsubscribe from this group, send email to >>> clojure+u...@googlegroups.com >>> For more options, visit this group at >>> http://groups.google.com/group/clojure?hl=en >>> --- >>> You received this message because you are subscribed to the Google >>> Groups "Clojure" group. >>> To unsubscribe from this group and stop receiving emails from it, send >>> an email to clojure+u...@googlegroups.com. >>> For more options, visit https://groups.google.com/d/optout. >>> >> >> >> -- >> James Reeves >> booleanknot.com >> > -- > You received this message because you are subscribed to the Google > Groups "Clojure" group. > To post to this group, send email to clojure@googlegroups.com > Note that posts from new members are moderated - please be patient with > your first post. > To unsubscribe from this group, send email to > clojure+unsubscr...@googlegroups.com > For more options, visit this group at > http://groups.google.com/group/clojure?hl=en > --- > You received this message because you are subscribed to the Google Groups > "Clojure" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to clojure+unsubscr...@googlegroups.com. > For more options, visit https://groups.google.com/d/optout. > -- You received this message because you are subscribed to the Google Groups "Clojure" group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups "Clojure" group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: What do Clojure developers use for recurring functions, other than at-at
Laurens Van Houtven, good ideas, but then I'd also have to write some code to catch documents that got lost when a process died while trying to fetch a document from S3. If I simply check every 15 minutes, and grab everything that has not already been stored in the database, then I automatically fetch documents that were dropped due to error. It seems simple. In my current case, there is no penalty for fetching the same document twice, but it is important that we avoid missing a document. On Monday, December 17, 2018 at 2:59:22 PM UTC-5, Laurens Van Houtven wrote: > > Honestly I'd use CloudWatch Timed Events to kick off a Lambda or ECS > Fargate job (which of course you can write in Clojure) assuming you're > using AWS yourself anyway. If you don't care about batching maybe even just > attach a Lambda to the write-to-S3 bucket itself instead of checking every > 15m? > > If you want to do it in-process, my tool of choice is ztellman's manifold: > https://github.com/ztellman/manifold/blob/d67a8c1b9f1268c094895d70dbbf146521f5774b/src/manifold/time.clj > > On Mon, Dec 17, 2018 at 1:54 PM > > wrote: > >> I'm coming back to Clojure development after a year away. This is a fast >> moving community and it is hard to keep up when one is not working on it >> full time. I'm dusting off some code I wrote 2 years ago, and trying to >> bring all the dependencies up to their current versions. >> >> I have a function that fetches files from an AWS S3 bucket, every 15 >> minutes. I had previously used the at-at library for this: >> >> https://github.com/overtone/at-at >> >> But at-at has not been updated in 6 years, so I assume it is abandoned. I >> have two questions about this: >> >> 1.) how else do Clojure programmers usually call recurring functionality? >> >> 2.) I am ignorant about the JVM, so I'm afraid I have to ask, at a >> fundamental level, how does at-at work? I know that if a function calls >> itself recurringly, on the JVM, one eventually gets stackoverflow. So how >> does at-at make its magic work? >> >> -- >> You received this message because you are subscribed to the Google >> Groups "Clojure" group. >> To post to this group, send email to clo...@googlegroups.com >> >> Note that posts from new members are moderated - please be patient with >> your first post. >> To unsubscribe from this group, send email to >> clojure+u...@googlegroups.com >> For more options, visit this group at >> http://groups.google.com/group/clojure?hl=en >> --- >> You received this message because you are subscribed to the Google Groups >> "Clojure" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to clojure+u...@googlegroups.com . >> For more options, visit https://groups.google.com/d/optout. >> > -- You received this message because you are subscribed to the Google Groups "Clojure" group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups "Clojure" group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: What do Clojure developers use for recurring functions, other than at-at
James Reeves, that does sound like the right way to go. I'll do that. On Monday, December 17, 2018 at 3:31:01 PM UTC-5, James Reeves wrote: > > > I'd use an executor: > > (ns example.main > (:import [java.util.concurrent Executors TimeUnit])) > > (def scheduler > (Executors/newScheduledThreadPool 32)) > > (defn fetch-files [] > (println "Fetching files...)) > > (defn -main [] > (.scheduleAtFixedRate scheduler ^Runnable fetch-files 15 15 > TimeUnit/MINUTES)) > > On Mon, 17 Dec 2018 at 20:14, > wrote: > >> I'm coming back to Clojure development after a year away. This is a fast >> moving community and it is hard to keep up when one is not working on it >> full time. I'm dusting off some code I wrote 2 years ago, and trying to >> bring all the dependencies up to their current versions. >> >> I have a function that fetches files from an AWS S3 bucket, every 15 >> minutes. I had previously used the at-at library for this: >> >> https://github.com/overtone/at-at >> >> But at-at has not been updated in 6 years, so I assume it is abandoned. I >> have two questions about this: >> >> 1.) how else do Clojure programmers usually call recurring functionality? >> >> 2.) I am ignorant about the JVM, so I'm afraid I have to ask, at a >> fundamental level, how does at-at work? I know that if a function calls >> itself recurringly, on the JVM, one eventually gets stackoverflow. So how >> does at-at make its magic work? >> >> -- >> You received this message because you are subscribed to the Google >> Groups "Clojure" group. >> To post to this group, send email to clo...@googlegroups.com >> >> Note that posts from new members are moderated - please be patient with >> your first post. >> To unsubscribe from this group, send email to >> clojure+u...@googlegroups.com >> For more options, visit this group at >> http://groups.google.com/group/clojure?hl=en >> --- >> You received this message because you are subscribed to the Google Groups >> "Clojure" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to clojure+u...@googlegroups.com . >> For more options, visit https://groups.google.com/d/optout. >> > > > -- > James Reeves > booleanknot.com > -- You received this message because you are subscribed to the Google Groups "Clojure" group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups "Clojure" group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: What do Clojure developers use for recurring functions, other than at-at
I'd use an executor: (ns example.main (:import [java.util.concurrent Executors TimeUnit])) (def scheduler (Executors/newScheduledThreadPool 32)) (defn fetch-files [] (println "Fetching files...)) (defn -main [] (.scheduleAtFixedRate scheduler ^Runnable fetch-files 15 15 TimeUnit/MINUTES)) On Mon, 17 Dec 2018 at 20:14, wrote: > I'm coming back to Clojure development after a year away. This is a fast > moving community and it is hard to keep up when one is not working on it > full time. I'm dusting off some code I wrote 2 years ago, and trying to > bring all the dependencies up to their current versions. > > I have a function that fetches files from an AWS S3 bucket, every 15 > minutes. I had previously used the at-at library for this: > > https://github.com/overtone/at-at > > But at-at has not been updated in 6 years, so I assume it is abandoned. I > have two questions about this: > > 1.) how else do Clojure programmers usually call recurring functionality? > > 2.) I am ignorant about the JVM, so I'm afraid I have to ask, at a > fundamental level, how does at-at work? I know that if a function calls > itself recurringly, on the JVM, one eventually gets stackoverflow. So how > does at-at make its magic work? > > -- > You received this message because you are subscribed to the Google > Groups "Clojure" group. > To post to this group, send email to clojure@googlegroups.com > Note that posts from new members are moderated - please be patient with > your first post. > To unsubscribe from this group, send email to > clojure+unsubscr...@googlegroups.com > For more options, visit this group at > http://groups.google.com/group/clojure?hl=en > --- > You received this message because you are subscribed to the Google Groups > "Clojure" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to clojure+unsubscr...@googlegroups.com. > For more options, visit https://groups.google.com/d/optout. > -- James Reeves booleanknot.com -- You received this message because you are subscribed to the Google Groups "Clojure" group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups "Clojure" group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: What do Clojure developers use for recurring functions, other than at-at
On Mon, Dec 17, 2018 at 2:54 PM wrote: > But at-at has not been updated in 6 years, so I assume it is abandoned. I > have two questions about this: > A common bit of wisdom here in the Clojure community is that time since last update is not always (or even often) a sign of abandonment but instead of stability. If there are many and recent open issues on at-at then maybe it's been abandoned. Or it could just be stable. Disclaimer, I don't know much about at-at. -- You received this message because you are subscribed to the Google Groups "Clojure" group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups "Clojure" group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: What do Clojure developers use for recurring functions, other than at-at
Honestly I'd use CloudWatch Timed Events to kick off a Lambda or ECS Fargate job (which of course you can write in Clojure) assuming you're using AWS yourself anyway. If you don't care about batching maybe even just attach a Lambda to the write-to-S3 bucket itself instead of checking every 15m? If you want to do it in-process, my tool of choice is ztellman's manifold: https://github.com/ztellman/manifold/blob/d67a8c1b9f1268c094895d70dbbf146521f5774b/src/manifold/time.clj On Mon, Dec 17, 2018 at 1:54 PM wrote: > I'm coming back to Clojure development after a year away. This is a fast > moving community and it is hard to keep up when one is not working on it > full time. I'm dusting off some code I wrote 2 years ago, and trying to > bring all the dependencies up to their current versions. > > I have a function that fetches files from an AWS S3 bucket, every 15 > minutes. I had previously used the at-at library for this: > > https://github.com/overtone/at-at > > But at-at has not been updated in 6 years, so I assume it is abandoned. I > have two questions about this: > > 1.) how else do Clojure programmers usually call recurring functionality? > > 2.) I am ignorant about the JVM, so I'm afraid I have to ask, at a > fundamental level, how does at-at work? I know that if a function calls > itself recurringly, on the JVM, one eventually gets stackoverflow. So how > does at-at make its magic work? > > -- > You received this message because you are subscribed to the Google > Groups "Clojure" group. > To post to this group, send email to clojure@googlegroups.com > Note that posts from new members are moderated - please be patient with > your first post. > To unsubscribe from this group, send email to > clojure+unsubscr...@googlegroups.com > For more options, visit this group at > http://groups.google.com/group/clojure?hl=en > --- > You received this message because you are subscribed to the Google Groups > "Clojure" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to clojure+unsubscr...@googlegroups.com. > For more options, visit https://groups.google.com/d/optout. > -- You received this message because you are subscribed to the Google Groups "Clojure" group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups "Clojure" group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
What do Clojure developers use for recurring functions, other than at-at
I'm coming back to Clojure development after a year away. This is a fast moving community and it is hard to keep up when one is not working on it full time. I'm dusting off some code I wrote 2 years ago, and trying to bring all the dependencies up to their current versions. I have a function that fetches files from an AWS S3 bucket, every 15 minutes. I had previously used the at-at library for this: https://github.com/overtone/at-at But at-at has not been updated in 6 years, so I assume it is abandoned. I have two questions about this: 1.) how else do Clojure programmers usually call recurring functionality? 2.) I am ignorant about the JVM, so I'm afraid I have to ask, at a fundamental level, how does at-at work? I know that if a function calls itself recurringly, on the JVM, one eventually gets stackoverflow. So how does at-at make its magic work? -- You received this message because you are subscribed to the Google Groups "Clojure" group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups "Clojure" group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
