On Thursday, January 17, 2019 at 3:37:36 PM UTC-8, Enrique Artal wrote: > > I made some computations, I skip the details for now, but the result was a > rational function with rational coefficients and 13 indeterminates. The > computation took around three hours and used a lot of memory so I made it > using a script. The first time I saved the rational function in a pickle > file, the second time in a sobj file. >
Both are actually pickles, but the standard python "pickle" defaults to an older version of the protocol. Sage's "sobj" wrappers select a more modern version of the protocol and a more compact (binary) file representation. > The first one was 1.2Gb, the second one around 300Mb. The main issue is > that opening a sage session (or using a script) takes much more time than > computing (with pickle more than 12 hours, I stopped it, with sobj 10 hours > and counting). > The pickle format basically consists of a simple programming language with instructions to build data structures. To get an idea what it does, you can use, for instance, sage: R.<x,y>=QQ[] sage: f=(x^2+y^2)/x sage: import pickletools sage: pickletools.dis(sage.misc.explainpickle.comp.decompress(dumps(f))) or, to get a more or less equivalent piece of sage code: sage: explain_pickle(dumps(f)) > I did not try a text file, but I wonder which is the best strategy to save > a heavy result and being able to load it using less time than the actual > computation. > The main thing you learn from the above exercise is that pickle in principle does something fairly sensible for rational functions: It constructs numerator and denominator via a dictionary. It may be that this happens with a lot of overhead, so perhaps you can gain something if you do it via another route, but it may well be that the bottleneck is in the sage-to-libsingular interface, and other construction methods won't get around that. It's not always the case that reconstructing something from a file store is faster than computing it. A silly example: a=2**(10**10) completes quite quickly, but writing the bit representation to a file and reading it in will be considerable work. It's unlikely your example is as extreme as the one above, but your experience indicates that the size of the polynomials involved in your case (the size of the "sobj" file is a poor measure for that) is beyond what sage has been tested with regularly. It looks like there's significant room for improvement here. Pickle really should be able to do this. > Ideas would be appreciated. Thanks, Enrique Artal. > -- You received this message because you are subscribed to the Google Groups "sage-support" group. To unsubscribe from this group and stop receiving emails from it, send an email to sage-support+unsubscr...@googlegroups.com. To post to this group, send email to sage-support@googlegroups.com. Visit this group at https://groups.google.com/group/sage-support. For more options, visit https://groups.google.com/d/optout.
