Re: [Pytables-users] Pytables bulk loading data

Thanks. I will try it out and post any findings. Pushkar On Thu, Jul 18, 2013 at 12:36 AM, Andreas Hilboll wrote: > > > > You could use pandas_ and the read_table function. There, you have nrows > and skiprows parameters with which you can easily do your own 'streaming'. > > .. _pandas: http://

Re: [Pytables-users] Storing small files in the filenodes

Thank you, Anthony, I will try VLArray as you suggested =) On Thu, Jul 18, 2013 at 3:39 AM, Anthony Scopatz wrote: > Hello Valeriy, > > For better or worse, the is exactly the performance I would expect. The > thing that you are running up against is that every HDF5 data set has 64 Kb > of he

Re: [Pytables-users] Pytables-users Digest, Vol 86, Issue 8

Hi Pushkar, Il 18/07/2013 08:45, Pushkar Raj Pande ha scritto: > Both loadtxt and genfromtxt read the entire data into memory which is not > desirable. Is there a way to achieve streaming writes? > OK, probably fromfile [1] can help you to cook something that works without loading the entire fil

Re: [Pytables-users] Pytables-users Digest, Vol 86, Issue 8

On 18.07.2013 08:45, Pushkar Raj Pande wrote: > Both loadtxt and genfromtxt read the entire data into memory which is > not desirable. Is there a way to achieve streaming writes? > > Thanks, > Pushkar > > > On Wed, Jul 17, 2013 at 7:04 PM, Pushkar Raj Pande > wrote: