Re: [Pytables-users] Importing from Matlab's sparse format
Antonio Valentino tiscali.it> writes: > > Hi Tom, > > If my understanding is correct before calling csc_matrix you should > actually read data from disk > > > data = f2.root.M.data[...] > > ir = f2.root.M.ir[...] > > jc = f2.root.M.jc[...] > > Please note that f3.root.M.data in a pytables object, and not a numpy array > > Not sure to understand the second question but I guess the answer is > yes, unless it is a trivial element-by-element operation. > > Best regards > Hi Antonio, Thank-you for your reply. Yes that solved my problem! Thanks, Tom -- Learn Windows Azure Live! Tuesday, Dec 13, 2011 Microsoft is holding a special Learn Windows Azure training event for developers. It will provide a great way to learn Windows Azure and what it provides. You can attend the event by watching it streamed LIVE online. Learn more at http://p.sf.net/sfu/ms-windowsazure ___ Pytables-users mailing list Pytables-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/pytables-users
Re: [Pytables-users] Importing from Matlab's sparse format
Hi Tom, Il 09/12/2011 14:12, Tom Diethe ha scritto: > I have files stored using Matlab's sparse format (HDF5, csc I > believe), and I'm trying to use Pytables to operate on them directly, > but haven't succeeded yet. Using h5py I can do the following: > > # Method 1: uses h5py (WORKS) > f1 = h5py.File(fname) > data = f1['M']['data] > ir = f1['M]['ir'] > jc = f1['M']['jc'] > M = scipy.sparse.csc_matrix( (data,ir,jc) ) > > but if I try to do the equivalent in Pytables: > > # Method 2: uses pyTables (DOESN'T WORK) > f2 = tables.openFile(fname) > data = f2.root.M.data > ir = f2.root.M.ir > jc = f2.root.M.jc > M = scipy.sparse.csc_matrix( (data,ir,jc) ) > If my understanding is correct before calling csc_matrix you should actually read data from disk > data = f2.root.M.data[...] > ir = f2.root.M.ir[...] > jc = f2.root.M.jc[...] Please note that f3.root.M.data in a pytables object, and not a numpy array In [23]: f2.root.M.data Out[23]: /M/data (Array(20,)) '' atom := Float64Atom(shape=(), dflt=0.0) maindim := 0 flavor := 'numpy' byteorder := 'little' chunkshape := None > this fails (after a long wait) with the error: > > TypeError Traceback (most recent call last) > > /home/tdiethe/BMJ/ in () > > /usr/lib/python2.6/dist-packages/scipy/sparse/compressed.pyc in > __init__(self, arg1, shape, dtype, copy, dims, nzmax) > 56 self.indices = np.array(indices, copy=copy) > 57 self.indptr = np.array(indptr, copy=copy) > ---> 58 self.data= np.array(data, copy=copy, > dtype=getdtype(dtype, data)) > 59 else: > 60 raise ValueError, "unrecognized %s_matrix > constructor usage" %\ > > /usr/lib/python2.6/dist-packages/scipy/sparse/sputils.pyc in > getdtype(dtype, a, default) > 69 canCast = False > 70 else: > ---> 71 raise TypeError, "could not interpret data type" > 72 else: > 73 newdtype = np.dtype(dtype) > > TypeError: could not interpret data type > > > I have two questions: > > - how do I load this file in > - do I need to perform the conversion to a scipy sparse matrix in > order to be able to perform operations on it, or can I perform > operations directly on the disk files (matrix multiplication etc)? Not sure to understand the second question but I guess the answer is yes, unless it is a trivial element-by-element operation. Best regards -- Antonio Valentino -- Learn Windows Azure Live! Tuesday, Dec 13, 2011 Microsoft is holding a special Learn Windows Azure training event for developers. It will provide a great way to learn Windows Azure and what it provides. You can attend the event by watching it streamed LIVE online. Learn more at http://p.sf.net/sfu/ms-windowsazure ___ Pytables-users mailing list Pytables-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/pytables-users
[Pytables-users] Importing from Matlab's sparse format
I have files stored using Matlab's sparse format (HDF5, csc I believe), and I'm trying to use Pytables to operate on them directly, but haven't succeeded yet. Using h5py I can do the following: # Method 1: uses h5py (WORKS) f1 = h5py.File(fname) data = f1['M']['data] ir = f1['M]['ir'] jc = f1['M']['jc'] M = scipy.sparse.csc_matrix( (data,ir,jc) ) but if I try to do the equivalent in Pytables: # Method 2: uses pyTables (DOESN'T WORK) f2 = tables.openFile(fname) data = f2.root.M.data ir = f2.root.M.ir jc = f2.root.M.jc M = scipy.sparse.csc_matrix( (data,ir,jc) ) this fails (after a long wait) with the error: TypeError Traceback (most recent call last) /home/tdiethe/BMJ/ in () /usr/lib/python2.6/dist-packages/scipy/sparse/compressed.pyc in __init__(self, arg1, shape, dtype, copy, dims, nzmax) 56 self.indices = np.array(indices, copy=copy) 57 self.indptr = np.array(indptr, copy=copy) ---> 58 self.data= np.array(data, copy=copy, dtype=getdtype(dtype, data)) 59 else: 60 raise ValueError, "unrecognized %s_matrix constructor usage" %\ /usr/lib/python2.6/dist-packages/scipy/sparse/sputils.pyc in getdtype(dtype, a, default) 69 canCast = False 70 else: ---> 71 raise TypeError, "could not interpret data type" 72 else: 73 newdtype = np.dtype(dtype) TypeError: could not interpret data type I have two questions: - how do I load this file in - do I need to perform the conversion to a scipy sparse matrix in order to be able to perform operations on it, or can I perform operations directly on the disk files (matrix multiplication etc)? -- Cloud Services Checklist: Pricing and Packaging Optimization This white paper is intended to serve as a reference, checklist and point of discussion for anyone considering optimizing the pricing and packaging model of a cloud services business. Read Now! http://www.accelacomm.com/jaw/sfnl/114/51491232/ ___ Pytables-users mailing list Pytables-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/pytables-users
