Re: [Numpy-discussion] record data previous to Numpy use

On 07 Jul 2017, at 4:24 PM, paul.carr...@free.fr wrote: > > ps : I'd like to use the following code that is much more familiar for me :-) > > COMP_list = np.asarray(COMP_list, dtype = np.float64) > i = np.arange(1,NumberOfRecords,2) > COMP_list = np.delete(COMP_list,i) > Not sure about the

Re: [Numpy-discussion] record data previous to Numpy use

Hi (all) Ounce again I would like to thanks the community for the supports. I progressing in moving my code to Python .. In my mind some parts remains quite hugly (and burns me the eyes), but it works and I'll optimized it in the future ; so far I can work with the data in a single reading

Re: [Numpy-discussion] record data previous to Numpy use

On 7 Jul 2017, at 1:59 am, Chris Barker wrote: > > On Thu, Jul 6, 2017 at 10:55 AM, wrote: > It's is just a reflexion, but for huge files one solution might be to > split/write/build first the array in a dedicated file (2x o(n) iterations - > one

Re: [Numpy-discussion] record data previous to Numpy use

On Thu, Jul 6, 2017 at 10:55 AM, wrote: > > It's is just a reflexion, but for huge files one solution might be to > split/write/build first the array in a dedicated file (2x o(n) iterations - > one to identify the blocks size - additional one to get and write), and > then to

Re: [Numpy-discussion] record data previous to Numpy use

OK, you have two performance "issues" 1) memory use: IF yu need to read a file to build a numpy array, and dont know how big it is when you start, you need to accumulate the values first, and then make an array out of them. And numpy arrays are fixed size, so they can not efficiently accumulate

Re: [Numpy-discussion] record data previous to Numpy use

Thanks Rober for your effort - I'll have a look on it ... the goal is be guide in how to proceed (and to understand), and not to have a "ready-made solution" ... but I appreciate honnestly :-) Paul Le 2017-07-06 11:51, Robert Kern a écrit : > On Thu, Jul 6, 2017 at 1:49 AM,

Re: [Numpy-discussion] record data previous to Numpy use

On Thu, Jul 6, 2017 at 1:49 AM, wrote: > > Dear All > > First of all thanks for the answers and the information’s (I’ll ding into it) and let me trying to add comments on what I want to : > > My asci file mainly contains data (float and int) in a single column > (it is not

Re: [Numpy-discussion] record data previous to Numpy use

Dear All First of all thanks for the answers and the information's (I'll ding into it) and let me trying to add comments on what I want to : * My asci file mainly contains data (float and int) in a single column * (it is not always the case but I can easily manage it - as well I

Re: [Numpy-discussion] record data previous to Numpy use

While I'm going to bet that the fastest way to build a ndarray from ascii is with a 'io.ByteIO` stream, NumPy does have a function to load from text, `numpy.loadtxt` that works well enough for most purposes. https://docs.scipy.org/doc/numpy/reference/generated/numpy.loadtxt.html It's hard to

Re: [Numpy-discussion] record data previous to Numpy use

Hi Paul, > ascii file is an input format (and the only one I can deal with) > > HDF5 one might be an export one (it's one of the options) in order to speed > up the post-processing stage > > > > Paul > > > > > > Le 2017-07-05 20:19, Thomas Caswell a écrit : > >> Are you tied to ASCII

Re: [Numpy-discussion] record data previous to Numpy use

On Wed, Jul 5, 2017 at 5:41 AM, wrote: > > Dear all > > I’m sorry if my question is too basic (not fully in relation to Numpy – while it is to build matrices and to work with Numpy afterward), but I’m spending a lot of time and effort to find a way to record data from an

Re: [Numpy-discussion] record data previous to Numpy use

Hi Thanks for the answer: ascii file is an input format (and the only one I can deal with) HDF5 one might be an export one (it's one of the options) in order to speed up the post-processing stage Paul Le 2017-07-05 20:19, Thomas Caswell a écrit : > Are you tied to ASCII files? HDF5

Re: [Numpy-discussion] record data previous to Numpy use

Are you tied to ASCII files? HDF5 (via h5py or pytables) might be a better storage format for what you are describing. Tom On Wed, Jul 5, 2017 at 8:42 AM wrote: > Dear all > > > I’m sorry if my question is too basic (not fully in relation to Numpy – > while it is to

[Numpy-discussion] record data previous to Numpy use

Dear all I'm sorry if my question is too basic (not fully in relation to Numpy - while it is to build matrices and to work with Numpy afterward), but I'm spending a lot of time and effort to find a way to record data from an asci while, and reassign it into a matrix/array … with unsuccessfully!