Re: why am I hitting the 64k method limit on a var?
On Tuesday, March 26, 2013 6:54:12 PM UTC+1, David Powell wrote: > > > On Tue, Mar 26, 2013 at 5:48 PM, larry google groups > > > wrote: > >> >> > I'd guess that this would be a pretty big map to run to over 64k though >> - though >> > I think Clojure has a single initialiser method that initialises all >> vars in the >> > namespace, so it is tha total that is important. >> >> >> It certainly seems as if it is the whole namespace that is being >> compiled. The startup file was fine for a while, when I had maybe 3k or 4k >> of data that I was holding in a map, which told the software about the >> different types of data it needed to work on. But then my company gave me a >> list of all search node numbers that we have used to structure the way we >> make calls to Solr. The most obvious place for me to put this was in that >> same map, so I could use it to build search queries on the fly, but, as I >> said, I then ran into the 64k limit. >> >> I guess, moving forward, I will store the data in MongoDb and I will read >> into that var once the app is running. >> > > Yeah, top-level defs get compiled into a *__init.class* class > - you can verify this by AOT compiling then using *javap -c* on the class > file. > There is no equivalent to this in the case without AOT compilation. In that case the evaluation of each def form individually compiles and executes its initialization code. -marko -- -- You received this message because you are subscribed to the Google Groups "Clojure" group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups "Clojure" group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out.
Re: why am I hitting the 64k method limit on a var?
On Tue, Mar 26, 2013 at 5:48 PM, larry google groups < lawrencecloj...@gmail.com> wrote: > > > I'd guess that this would be a pretty big map to run to over 64k though > - though > > I think Clojure has a single initialiser method that initialises all > vars in the > > namespace, so it is tha total that is important. > > > It certainly seems as if it is the whole namespace that is being compiled. > The startup file was fine for a while, when I had maybe 3k or 4k of data > that I was holding in a map, which told the software about the different > types of data it needed to work on. But then my company gave me a list of > all search node numbers that we have used to structure the way we make > calls to Solr. The most obvious place for me to put this was in that same > map, so I could use it to build search queries on the fly, but, as I said, > I then ran into the 64k limit. > > I guess, moving forward, I will store the data in MongoDb and I will read > into that var once the app is running. > Yeah, top-level defs get compiled into a *__init.class* class - you can verify this by AOT compiling then using *javap -c* on the class file. Something like: (def config (read (clojure.java.io/resource "my_file.clj"))) will overcome the limitation -- Dave -- -- You received this message because you are subscribed to the Google Groups "Clojure" group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups "Clojure" group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out.
Re: why am I hitting the 64k method limit on a var?
> I'd guess that this would be a pretty big map to run to over 64k though - though > I think Clojure has a single initialiser method that initialises all vars in the > namespace, so it is tha total that is important. It certainly seems as if it is the whole namespace that is being compiled. The startup file was fine for a while, when I had maybe 3k or 4k of data that I was holding in a map, which told the software about the different types of data it needed to work on. But then my company gave me a list of all search node numbers that we have used to structure the way we make calls to Solr. The most obvious place for me to put this was in that same map, so I could use it to build search queries on the fly, but, as I said, I then ran into the 64k limit. I guess, moving forward, I will store the data in MongoDb and I will read into that var once the app is running. On Tuesday, March 26, 2013 7:13:54 AM UTC-4, David Powell wrote: > > > > On Tue, Mar 26, 2013 at 10:40 AM, Marko Topolnik > > > wrote: > >> What exactly is getting compiled into a class here? Not the map itself, I >> believe; that wouldn't even cause this error. >> >> From general clues, the code that builds your config map is being >> compiled into a method. Without AOT compilation this shouldn't be >> happening, but I'm surprised it's happening even with AOT. >> > > Java classes can contain code, and primitive constants, everything else, > even array initialisers in Java, have to be compiled into initialiser code > that builds the data-structure. > > I'd guess that this would be a pretty big map to run to over 64k though - > though I think Clojure has a single initialiser method that initialises all > vars in the namespace, so it is tha total that is important. > > I think AOT is irrelevant - the same classes get compiled whether they are > AOT'd or not. > -- -- You received this message because you are subscribed to the Google Groups "Clojure" group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups "Clojure" group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out.
Re: why am I hitting the 64k method limit on a var?
On Tuesday, March 26, 2013 12:13:54 PM UTC+1, David Powell wrote: > > On Tue, Mar 26, 2013 at 10:40 AM, Marko Topolnik > > > wrote: > >> What exactly is getting compiled into a class here? Not the map itself, I >> believe; that wouldn't even cause this error. >> >> From general clues, the code that builds your config map is being >> compiled into a method. Without AOT compilation this shouldn't be >> happening, but I'm surprised it's happening even with AOT. >> > > I'd guess that this would be a pretty big map to run to over 64k though - > though I think Clojure has a single initialiser method that initialises all > vars in the namespace, so it is tha total that is important. > > I think AOT is irrelevant - the same classes get compiled whether they are > AOT'd or not. > Each form is compiled individually and nothing is compiled for the namespace as a whole. Whether or not this is true for AOT I'm not sure; for :gen-class it is definitely not true. -marko -- -- You received this message because you are subscribed to the Google Groups "Clojure" group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups "Clojure" group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out.
Re: why am I hitting the 64k method limit on a var?
On Tue, Mar 26, 2013 at 10:40 AM, Marko Topolnik wrote: > What exactly is getting compiled into a class here? Not the map itself, I > believe; that wouldn't even cause this error. > > From general clues, the code that builds your config map is being compiled > into a method. Without AOT compilation this shouldn't be happening, but I'm > surprised it's happening even with AOT. > Java classes can contain code, and primitive constants, everything else, even array initialisers in Java, have to be compiled into initialiser code that builds the data-structure. I'd guess that this would be a pretty big map to run to over 64k though - though I think Clojure has a single initialiser method that initialises all vars in the namespace, so it is tha total that is important. I think AOT is irrelevant - the same classes get compiled whether they are AOT'd or not. -- -- You received this message because you are subscribed to the Google Groups "Clojure" group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups "Clojure" group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out.
Re: why am I hitting the 64k method limit on a var?
What exactly is getting compiled into a class here? Not the map itself, I
believe; that wouldn't even cause this error.
>From general clues, the code that builds your config map is being compiled
into a method. Without AOT compilation this shouldn't be happening, but I'm
surprised it's happening even with AOT.
-marko
On Monday, March 25, 2013 10:35:33 PM UTC+1, David Powell wrote:
>
> You might be better off putting the config into a file, and then
> "read"-ing it. That way it never gets compiled into a class, and just
> stays as a data structure.
>
> On Mon, Mar 25, 2013 at 9:23 PM, larry google groups
>
> > wrote:
>
>> Are vars subject to Java's 64k limit on methods? I started an app a few
>> months ago and I was putting all the config into into a var:
>>
>> (def app-config
>> {
>> :cites {
>>;; more here
>>}
>>
>> :slides {
>>;; more here
>>}
>> :questions {
>>;; more here
>>}
>> }
>>
>>
>> Then I guess I added too much and I started getting the error regarding
>> the 64k limit. Is this expected?
>>
>> Why can I add more than 64k to a var, but I can not define it that way at
>> compile time?
>>
>>
>> --
>> --
>> You received this message because you are subscribed to the Google
>> Groups "Clojure" group.
>> To post to this group, send email to clo...@googlegroups.com
>> Note that posts from new members are moderated - please be patient with
>> your first post.
>> To unsubscribe from this group, send email to
>> clojure+u...@googlegroups.com
>> For more options, visit this group at
>> http://groups.google.com/group/clojure?hl=en
>> ---
>> You received this message because you are subscribed to the Google Groups
>> "Clojure" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to clojure+u...@googlegroups.com .
>> For more options, visit https://groups.google.com/groups/opt_out.
>>
>>
>>
>
>
--
--
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
---
You received this message because you are subscribed to the Google Groups
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
Re: why am I hitting the 64k method limit on a var?
You might be better off putting the config into a file, and then "read"-ing
it. That way it never gets compiled into a class, and just stays as a data
structure.
On Mon, Mar 25, 2013 at 9:23 PM, larry google groups <
lawrencecloj...@gmail.com> wrote:
> Are vars subject to Java's 64k limit on methods? I started an app a few
> months ago and I was putting all the config into into a var:
>
> (def app-config
> {
> :cites {
>;; more here
>}
>
> :slides {
>;; more here
>}
> :questions {
>;; more here
>}
> }
>
>
> Then I guess I added too much and I started getting the error regarding
> the 64k limit. Is this expected?
>
> Why can I add more than 64k to a var, but I can not define it that way at
> compile time?
>
>
> --
> --
> You received this message because you are subscribed to the Google
> Groups "Clojure" group.
> To post to this group, send email to clojure@googlegroups.com
> Note that posts from new members are moderated - please be patient with
> your first post.
> To unsubscribe from this group, send email to
> clojure+unsubscr...@googlegroups.com
> For more options, visit this group at
> http://groups.google.com/group/clojure?hl=en
> ---
> You received this message because you are subscribed to the Google Groups
> "Clojure" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to clojure+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/groups/opt_out.
>
>
>
--
--
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
---
You received this message because you are subscribed to the Google Groups
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
