Re: [racket-users] Persisting large values for debugging long-running programs

[Jon Zeppieri]

> On Sep 14, 2016, at 4:24 PM, 'John Clements' via Racket Users 
>  wrote:
> 
> Just in case this information is useful:
> 
> The “classical” hack here—Eli showed me this, I believe—is to write the data 
> out in the form of a file that “provide”s the specified datum, then compile 
> it to a “.zo” file. It seems plausible to me that loading a compiled .zo file 
> might be the fastest way of reading in data. Then again, if you use a 
> database as others have suggested you might be able to bypass racket’s 
> loading altogether.
> 
> John

Isn't fasl the format used in .zo files? 

-- 
You received this message because you are subscribed to the Google Groups 
"Racket Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to racket-users+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [racket-users] Persisting large values for debugging long-running programs

['John Clements' via Racket Users]

> On Sep 14, 2016, at 12:38 PM, Jonathan Schuster  wrote:
> 
> I have some large (several GB worth) sets of values I'd like to persist 
> across debugging runs of a program, rather than recomputing them each time. 
> I'm currently doing this with the built-in "read" and "write", but is there a 
> more efficient method, especially for reading the data back in?

Just in case this information is useful:

The “classical” hack here—Eli showed me this, I believe—is to write the data 
out in the form of a file that “provide”s the specified datum, then compile it 
to a “.zo” file. It seems plausible to me that loading a compiled .zo file 
might be the fastest way of reading in data. Then again, if you use a database 
as others have suggested you might be able to bypass racket’s loading 
altogether.

John



> 
> I could of course come up with some kind of custom encoding, but that's 
> likely not worth the effort in my case, so I'm wondering if there's any 
> general purpose method already available in Racket (or in a package).
> 
> Thanks,
> Jon
> 
> -- 
> You received this message because you are subscribed to the Google Groups 
> "Racket Users" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to racket-users+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.



-- 
You received this message because you are subscribed to the Google Groups 
"Racket Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to racket-users+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [racket-users] Persisting large values for debugging long-running programs

[Jon Zeppieri]

'Course you already mentioned a custom encoding, so, yeah, fasl: 
https://docs.racket-lang.org/reference/fasl.html

> On Sep 14, 2016, at 3:47 PM, Jon Zeppieri  wrote:
> 
> I think reading and writing fasl should be faster, but don't expect the 
> format to be compatible across Racket versions.
> 
> Or, if you know some super-efficient encoding for your data, wrap it in a new 
> struct type and implement your own serialization.
> 
>> On Sep 14, 2016, at 3:38 PM, Jonathan Schuster  wrote:
>> 
>> I have some large (several GB worth) sets of values I'd like to persist 
>> across debugging runs of a program, rather than recomputing them each time. 
>> I'm currently doing this with the built-in "read" and "write", but is there 
>> a more efficient method, especially for reading the data back in?
>> 
>> I could of course come up with some kind of custom encoding, but that's 
>> likely not worth the effort in my case, so I'm wondering if there's any 
>> general purpose method already available in Racket (or in a package).
>> 
>> Thanks,
>> Jon
>> -- 
>> You received this message because you are subscribed to the Google Groups 
>> "Racket Users" group.
>> To unsubscribe from this group and stop receiving emails from it, send an 
>> email to racket-users+unsubscr...@googlegroups.com.
>> For more options, visit https://groups.google.com/d/optout.

-- 
You received this message because you are subscribed to the Google Groups 
"Racket Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to racket-users+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [racket-users] Persisting large values for debugging long-running programs

[Jon Zeppieri]

I think reading and writing fasl should be faster, but don't expect the format 
to be compatible across Racket versions.

Or, if you know some super-efficient encoding for your data, wrap it in a new 
struct type and implement your own serialization.

> On Sep 14, 2016, at 3:38 PM, Jonathan Schuster  wrote:
> 
> I have some large (several GB worth) sets of values I'd like to persist 
> across debugging runs of a program, rather than recomputing them each time. 
> I'm currently doing this with the built-in "read" and "write", but is there a 
> more efficient method, especially for reading the data back in?
> 
> I could of course come up with some kind of custom encoding, but that's 
> likely not worth the effort in my case, so I'm wondering if there's any 
> general purpose method already available in Racket (or in a package).
> 
> Thanks,
> Jon
> -- 
> You received this message because you are subscribed to the Google Groups 
> "Racket Users" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to racket-users+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.

-- 
You received this message because you are subscribed to the Google Groups 
"Racket Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to racket-users+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [racket-users] Persisting large values for debugging long-running programs

[David Storrs]

Would a database work for you?  SQLite (
https://docs.racket-lang.org/db/connect.html#%28def._%28%28lib._db%2Fmain..rkt%29._sqlite3-connect%29%29)
is dead easy and very convenient.  Their maximum DB size is 140TB, so they
won't have an issue with a few gigs.

Dave


On Wed, Sep 14, 2016 at 12:38 PM, Jonathan Schuster 
wrote:

> I have some large (several GB worth) sets of values I'd like to persist
> across debugging runs of a program, rather than recomputing them each time.
> I'm currently doing this with the built-in "read" and "write", but is there
> a more efficient method, especially for reading the data back in?
>
> I could of course come up with some kind of custom encoding, but that's
> likely not worth the effort in my case, so I'm wondering if there's any
> general purpose method already available in Racket (or in a package).
>
> Thanks,
> Jon
>
> --
> You received this message because you are subscribed to the Google Groups
> "Racket Users" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to racket-users+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google Groups 
"Racket Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to racket-users+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[racket-users] Persisting large values for debugging long-running programs

[Jonathan Schuster]

I have some large (several GB worth) sets of values I'd like to persist
across debugging runs of a program, rather than recomputing them each time.
I'm currently doing this with the built-in "read" and "write", but is there
a more efficient method, especially for reading the data back in?

I could of course come up with some kind of custom encoding, but that's
likely not worth the effort in my case, so I'm wondering if there's any
general purpose method already available in Racket (or in a package).

Thanks,
Jon

-- 
You received this message because you are subscribed to the Google Groups 
"Racket Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to racket-users+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.