Re: "GC overhead limit exceeded": Deceptive message?

2017-08-08 Thread Paulus Esterhazy
For background on "holding onto the head of a sequence" type problems, see https://stuartsierra.com/2015/04/26/clojure-donts-concat and https://stackoverflow.com/questions/15994316/clojure-head-retention On Tue, Aug 8, 2017 at 6:19 PM, Nathan Smutz wrote: > The one thing I'm aware of holding o

Re: "GC overhead limit exceeded": Deceptive message?

2017-08-08 Thread Gary Trakhman
@Nathan the top-level (def requirement-seq ..) is probably the thing holding on to all the objects. Try removing the def and calling (last (sequence (comp ..))) and see if it returns? The purpose of a lazy sequence is to allow processing to happen one item or chunk at a time, if there are still p

Re: "GC overhead limit exceeded": Deceptive message?

2017-08-08 Thread Nathan Smutz
The one thing I'm aware of holding on to is a filtered file-seq: (def the-files (filter #(s/ends-with? (.getName %) ".xml" ) (rest (file-seq (io/file dw-path) There are 7,000+ files; but I'm assuming the elements there are just file-references and shouldn't take much space. The rest of the

Re: Converting json to work with clojure.spec

2017-08-08 Thread jmckitrick
Hmm, that might be useful. I assume it would make sense for an API called from other apps, but not so much for internal web services called from your own client app. On Mon, Aug 7, 2017 at 4:04 AM Peter Hull wrote: > > On Sunday, 4 December 2016 15:11:55 UTC, Jonathon McKitrick wrote: >> >> That

Re: "GC overhead limit exceeded": Deceptive message?

2017-08-08 Thread Peter Hull
On Tuesday, 8 August 2017 06:20:56 UTC+1, Nathan Smutz wrote: > Does this message sometimes present because the non-garbage data is > getting too big? > Yes, it's when most of your heap is non-garbage, so the GC has to keep running but doesn't succeed in freeing much memory each time. See http