I think my monkey-patch of jdbc/transaction didn't take. Doing
user/sourcery on it shows your version, but doing a macroexpand-all shows
the original.
My code calls via korma as kdb:
=> (clojure.walk/macroexpand-all '(kdb/transaction nil))
(if (clojure.java.jdbc/find-connection) (clojure.java.j
The latest java.jdbc snapshot (of 0.3.0) includes this fix so you can
either get it via Leiningen from the sonatype snapshots repo or git
clone it and do mvn install (or lein install since java.jdbc has a
project.clj file now).
I'll try to cut an official alpha5 release shortly once I've reviewed
After patching both transaction and with-connection (which is also used by
korma.db/transaction, and also creates a closure), I can pass in the seq
w/o leaking.
Thanks, Christophe.
On Thursday, September 12, 2013 11:20:11 AM UTC-7, Brian Craft wrote:
>
> or perhaps it's that macroexpand drops t
or perhaps it's that macroexpand drops the ^:once
On Thursday, September 12, 2013 10:55:28 AM UTC-7, Brian Craft wrote:
>
> I think my monkey-patch of jdbc/transaction didn't take. Doing
> user/sourcery on it shows your version, but doing a macroexpand-all shows
> the original.
>
> My code calls
I don't get the same results:
$ LEIN_JVM_OPTS=-Xmx20M lein repl
nREPL server started on port 61221 on host 127.0.0.1
REPL-y 0.2.1
Clojure 1.5.1
Docs: (doc function-name-here)
(find-doc "part-of-name-here")
Source: (source function-name-here)
Javadoc: (javadoc java-object-or-class-
Just to confirm, (t2 0 (range 100)) -- using doseq instead of
dorun -- does NOT run out of memory, correct?
On Wed, Sep 11, 2013 at 8:59 AM, Brian Craft wrote:
> This appears to have no effect on the problem. I tested with
> jdbc/transaction, and with Sean's simple example:
>
> user=> (defn f
Attaching a screenshot. Is this reference chain useful somehow? I can
recognize the contents of the string from the input file. The input file is
4G, and apparently lines from line-seq are lingering, so I'm still blowing
the heap.
On Wednesday, September 11, 2013 11:20:18 AM UTC-7, Brian Craft
Correct, I forgot to paste that part. ;)
On Wednesday, September 11, 2013 10:57:09 AM UTC-7, Sean Corfield wrote:
>
> Just to confirm, (t2 0 (range 100)) -- using doseq instead of
> dorun -- does NOT run out of memory, correct?
>
> On Wed, Sep 11, 2013 at 8:59 AM, Brian Craft
> >
> wrote:
I did start with visualvm. I posted a screenshot in an earlier thread.
However I'm unable to make sense of its output. jhat pointed straight to
the closure with the reference. visualvm gave me a thousand cascading
widgets to expand with names that were meaningless to me, none of which
pointed b
This appears to have no effect on the problem. I tested with
jdbc/transaction, and with Sean's simple example:
user=> (defn f [g] (g))
#'user/f
user=> (defn t1 [n c] (f (fn [] (dorun (map identity c)
#'user/t1
user=> (t1 0 (range 100))
java.lang.OutOfMemoryError: Java heap space (NO_SOURC
I have nothing to add for the problem itself (sorry) but am very interested
in the *process* of answering this question. Presuming there are things to
document here, I would love to see someone create a wiki page (on
http://dev.clojure.org/display/community/Home) or a clojure-doc note or
some
(started lein with LEIN_JVM_OPTS=-Xmx10M lein repl, to make it easy to see
a full heap)
On Wednesday, September 11, 2013 8:59:50 AM UTC-7, Brian Craft wrote:
>
> This appears to have no effect on the problem. I tested with
> jdbc/transaction, and with Sean's simple example:
>
> user=> (defn f [g
^:once on fn* (not fn, the fn macro doesn't propagate metadata) instructs
the commielr to clear closed-overs ASAP. It follows that you can't call
such a function twice because it forgets its closed-overs.
On Wed, Sep 11, 2013 at 4:36 PM, Brian Craft wrote:
> ugh. Can't find documentation for th
ugh. Can't find documentation for this. What does it do?
On Wednesday, September 11, 2013 2:22:56 AM UTC-7, Christophe Grand wrote:
>
>
> On Wed, Sep 11, 2013 at 6:00 AM, Brian Craft
> > wrote:
>
>> (defmacro transaction
>> [& body]
>> `(transaction* (fn [] ~@body)))
>>
>> I'm not sure how to
On Wed, Sep 11, 2013 at 6:00 AM, Brian Craft wrote:
> (defmacro transaction
> [& body]
> `(transaction* (fn [] ~@body)))
>
> I'm not sure how to avoid that. The anonymous function created here
> doesn't take parameters.
>
The fix for this is:
(defmacro transaction
[& body]
`(transaction*
jvisualvm has an innocuous button called "Dump Memory" or something.
You'd expect it to write out a core dump or something, but actually it
opens up a GUI which lets you interactively explore all of the objects on
the heap. It is pretty amazing. Much better than jhat, which I've found
to be reall
>
> I also suspected the closure over the matrix argument as being the
> root cause but was puzzled when using doseq instead made the problem
> go away...
>
>
Right, it doesn't seem to be a hold in the closure, unless the compiler
could tell when to release it, which is the case when the code
(defmacro transaction
[& body]
`(transaction* (fn [] ~@body)))
I'm not sure how to avoid that. The anonymous function created here doesn't
take parameters.
On Tuesday, September 10, 2013 8:55:42 PM UTC-7, Brian Craft wrote:
>
> Ah. So the root of my problem is that the jdbc/transaction macro
Ah. So the root of my problem is that the jdbc/transaction macro creates a
closure in my function..
(defmacro transaction
On Tuesday, September 10, 2013 7:34:32 PM UTC-7, Armando Blancas wrote:
>
> Can anyone explain to me what's happening here? Something about creating a
>> anonymous function?
FWIW, Brian and I were looking at this off-list and we changed (dorun
(map identity matrix)) to (doseq [x matrix] (identity x)) and that
seemed to work just fine - even in Brian's more complicated case.
Given that dorun specifically says it doesn't hold on to the head, I
would have expected the two
>
> Can anyone explain to me what's happening here? Something about creating a
> anonymous function?
>
The problem is not creating the anonymous function but that it closes over
the matrix argument. The closure passed on to blah will keep a reference to
matrix until blah returns. This won't ha
Copying the pattern of jdbc/transaction*, I tried this:
(defn blah [func] (func))
(defn load-exp [file timestamp filehash matrix]
(blah
(fn [] (dorun (map identity matrix)
which also consumes the heap.
Can anyone explain to me what's happening here? Something about creating a
anonymo
It seems to be something about the jdbc/transaction macro. If I rewrite
load_exp so it does nothing but walk the seq, it still consumes the heap:
(defn load-exp [file timestamp filehash matrix]
(jdbc/transaction
(dorun (map identity matrix
If I remove the jdbc/transaction call, it's f
Trying jhat now, reference chain here:
--> cavm.h2$load_exp$fn__165@0x2aaab4b04660 (56 bytes) (field matrix:)
--> clojure.lang.LazySeq@0x2aaab4b05388 (48 bytes) (field s:)
--> clojure.lang.Cons@0x2aaab4b0fe08 (48 bytes) (field _more:)
--> clojure.lang.LazySeq@0x2aaab4b10330 (48 bytes) (field s:)
-
>From jmap output it's pretty clear I'm retaining the head of a seq somehow,
but I have no idea how, or where. Is there any way to find the reference,
besides staring at code waiting for enlightenment?
--
--
You received this message because you are subscribed to the Google
Groups "Clojure" gr
25 matches
Mail list logo
