Here is the Python documentation for this: https://docs.python.org/2/library/base64.html
Regards, Edward On 8 December 2014 at 10:09, Edward d'Auvergne <edw...@nmr-relax.com> wrote: > Hi Troels, > > This is a standard problem of using text files and XML for universal > compatibility. However built into the XML format is the ability to > store any text between the tags. For example in this case we probably > need Base64: > > http://www.xml.com/pub/a/98/07/binary/binary.html > https://commons.apache.org/proper/commons-codec/apidocs/org/apache/commons/codec/binary/Base64.html > > This is many orders of magnitude quicker. This is the text format > used for all binary email attachments, for example. The only problem > here is converting the nmrglue numpy format to Base64, and back again. > Maybe you could ask Jonathan about it at > https://groups.google.com/forum/#!forum/nmrglue-discuss ? Although a > quick Google search finds this very simple Python solution: > > https://stackoverflow.com/questions/6485790/numpy-array-to-base64-and-back-to-numpy-array-python > > The relax data store to_xml() and from_xml() methods would need to be > modified to catch this data type. The to_xml() would have to set the > XML type attribute to 'base64', or '%s, base64'%repr(value.dtype). > See lib.xml.object_to_xml(). And the from_xml() method modified to > catch this and decode the Base64 string. This will require shifting > the lib.spectrum.object.Assignment class into > data_store.nmrglue.Assignment and then writing a custom to_xml() > method (maybe by a helper function in lib.xml). The decoding of the > Base64 numpy ndarray data could be part of the general function in > lib.xml.xml_to_object(). We should not make this general for all > numpy data, as the relax data store Python and XML structures are > designed for power users to be able read, and to be able to parse and > manipulate. However Base64 can be used for certain large data > structures for speed and size efficiency, with nmrglue data being the > prime example. > > Regards, > > Edward > > > On 7 December 2014 at 23:43, <tlin...@nmr-relax.com> wrote: >> Author: tlinnet >> Date: Sun Dec 7 23:43:06 2014 >> New Revision: 27007 >> >> URL: http://svn.gna.org/viewcvs/relax?rev=27007&view=rev >> Log: >> Change to systemtest Nmrglue.test_save_state. >> >> This now tries to save a 8MB numpy array to a state file. >> The numpy array has shape 512x4096. >> >> numpy.save store the array instantly, while saving the state, hungs up the >> computer for a long time. >> >> This shows that relax is really in-efficient in storing data. >> This both accounts for simple numpy array, and for storing class instances. >> >> Task #7873 (https://gna.org/task/index.php?7873): Write wrapper function to >> nmrglue, to read .ft2 files and process them. >> Homepage: http://www.nmrglue.com/ >> Link to nmrglue discussion: >> https://groups.google.com/forum/#!forum/nmrglue-discuss >> The code is develop at Github: https://github.com/jjhelmus/nmrglue/ >> Documentation: http://nmrglue.readthedocs.org/en/latest/index.html >> >> Modified: >> branches/nmrglue/test_suite/system_tests/nmrglue.py >> >> Modified: branches/nmrglue/test_suite/system_tests/nmrglue.py >> URL: >> http://svn.gna.org/viewcvs/relax/branches/nmrglue/test_suite/system_tests/nmrglue.py?rev=27007&r1=27006&r2=27007&view=diff >> ============================================================================== >> --- branches/nmrglue/test_suite/system_tests/nmrglue.py (original) >> +++ branches/nmrglue/test_suite/system_tests/nmrglue.py Sun Dec 7 23:43:06 >> 2014 >> @@ -373,7 +373,7 @@ >> print("Filesize of .npy file is: %i"%(data_numpy_size) ) >> >> # Delete the large data array, for faster saving. >> - cdp.nmrglue_data[sp_id] = 0 >> + #cdp.nmrglue_data[sp_id] = 0 >> >> # Store the directory path, before reset of the controller. >> dirpath = ds.tmpdir >> >> >> _______________________________________________ >> relax (http://www.nmr-relax.com) >> >> This is the relax-commits mailing list >> relax-comm...@gna.org >> >> To unsubscribe from this list, get a password >> reminder, or change your subscription options, >> visit the list information page at >> https://mail.gna.org/listinfo/relax-commits _______________________________________________ relax (http://www.nmr-relax.com) This is the relax-devel mailing list relax-devel@gna.org To unsubscribe from this list, get a password reminder, or change your subscription options, visit the list information page at https://mail.gna.org/listinfo/relax-devel
