Re: [Chicken-users] Declare large expressions as read-only and not gc-able?
On Sun, Jan 05, 2014 at 06:07:33PM +0100, Sven Hartrumpf wrote: > Hi all. > > Is there a recommended way to declare some large expressions (> 100 MB) > in a compiled program as read-only and more importantly > as not gc-able (the garbage collector should be saved from traversing > these large structures again and again)? > Should the constant value be compiled in or should it be read from a file? > Is define-constant the right way to go? > (How do other Scheme implementation handle such cases?) Hi Sven, I don't know if define-constant helps. I don't think it does, but I might be wrong. Perhaps you can use object-evict!, but that means it won't ever be reclaimed by the GC. I haven't used this myself yet, so I don't know if there's a way to "un-evict" the object when it can be GC'ed. Depending on the task, you could also just malloc some foreign object and provide foreign-lambda or foreign-primitive accessors to reach into the object. Cheers, Peter -- http://www.more-magic.net ___ Chicken-users mailing list Chicken-users@nongnu.org https://lists.nongnu.org/mailman/listinfo/chicken-users
Re: [Chicken-users] Declare large expressions as read-only and not gc-able?
I'll bet if you define a 100MB literal, the resulting C file would be at least a gigabyte and neither CHICKEN nor gcc would be able to handle it. You could try it and let us know. You could evict it theoretically, but for something that big, you may have to create your own data structure in C and malloc it. I'm curious what this expression is for. On Jan 5, 2014, at 11:07, Sven Hartrumpf wrote: > Hi all. > > Is there a recommended way to declare some large expressions (> 100 MB) > in a compiled program as read-only and more importantly > as not gc-able (the garbage collector should be saved from traversing > these large structures again and again)? > Should the constant value be compiled in or should it be read from a file? > Is define-constant the right way to go? > (How do other Scheme implementation handle such cases?) > > Ciao > Sven > > ___ > Chicken-users mailing list > Chicken-users@nongnu.org > https://lists.nongnu.org/mailman/listinfo/chicken-users ___ Chicken-users mailing list Chicken-users@nongnu.org https://lists.nongnu.org/mailman/listinfo/chicken-users
Re: [Chicken-users] Declare large expressions as read-only and not gc-able?
On Jan 5, 2014, at 11:48, Peter Bex wrote: > Perhaps you can use object-evict!, but >> > that means it won't ever be reclaimed by the GC. I haven't used > this myself yet, so I don't know if there's a way to "un-evict" the > object when it can be GC'ed. > http://api.call-cc.org/doc/lolevel/object-unevict ___ Chicken-users mailing list Chicken-users@nongnu.org https://lists.nongnu.org/mailman/listinfo/chicken-users
Re: [Chicken-users] Declare large expressions as read-only and not gc-able?
Not sure why this message got truncated, but it was in reference to moving an object back into the heap, in other words unevicting it. Felix says object-evict! is considered harmful, so keep that in mind. (It works fine in the limited context I have used it.) On Jan 5, 2014, at 15:27, Jim Ursetto wrote: > On Jan 5, 2014, at 11:48, Peter Bex wrote: > >> Perhaps you can use object-evict!, but [message truncated by my mail client] > > http://api.call-cc.org/doc/lolevel/object-unevict > ___ Chicken-users mailing list Chicken-users@nongnu.org https://lists.nongnu.org/mailman/listinfo/chicken-users
Re: [Chicken-users] Declare large expressions as read-only and not gc-able?
Sun, 5 Jan 2014 15:25:38 -0600, zbigniewsz wrote: > I'll bet if you define a 100MB literal, the resulting C file would be at > least a gigabyte and neither CHICKEN nor gcc would be able to handle it. You > could try it and let us know. You are right. chicken (and others like bigloo) does not compile such a beast and crashes with an error message. > You could evict it theoretically, but for something that big, you may have to > create your own data structure in C and malloc it. With object-evict, the program runs much slower without any progress, so Felix' warning about object-evict seems to apply in my use case. > I'm curious what this expression is for. It's a letter tree for a fullform lexicon of a natural language (more than 1 million entries). Access times are short, hence I abstained from switching to a key-value store on disk. But the time wasted in GC makes me think - from time to time :-) Ciao Sven ___ Chicken-users mailing list Chicken-users@nongnu.org https://lists.nongnu.org/mailman/listinfo/chicken-users
Re: [Chicken-users] Declare large expressions as read-only and not gc-able?
Sven Hartrumpf scripsit: > It's a letter tree for a fullform lexicon of a natural language > (more than 1 million entries). > Access times are short, hence I abstained from switching to a key-value > store on disk. I would design a C layout for such a structure and write C code that creates it in global memory (not the malloc'd heap), and then compile that directly with gcc. -- Real FORTRAN programmers can program FORTRANJohn Cowan in any language. --Ed Post co...@ccil.org ___ Chicken-users mailing list Chicken-users@nongnu.org https://lists.nongnu.org/mailman/listinfo/chicken-users
