On Mon, Sep 28, 2009 at 12:41 PM, Christopher Barker <chris.bar...@noaa.gov> wrote: > Skipper Seabold wrote: >> FWIW, I have a script that creates and savez arrays from several text >> files in total about 1.5 GB of text. >> >> without the incrementing in genfromtxt >> >> Run time: 122.043943 seconds >> >> with the incrementing in genfromtxt >> >> Run time: 131.698873 seconds >> >> If we just want to always keep track of things, I would be willing to >> take a poorly measured 8 % slowdown, > > I also think 8% is worth it, but I'm still surprised it's that much. > What addition code is inside the inner loop? (or , I guess, the each > line loop...) > > -Chris >
This was probably due to the way that I timed it, honestly. I only did it once. The only differences I made for that part were in the first post of the thread. Two incremented scalars for line numbers and column numbers and a try/except block. I'm really not against a debug mode if someone wants to do it, and it's deemed necessary. If it could be made to log all of the errors that would be extremely helpful. I still need to post some of my use cases though. Anything to help make data cleaning less of a chore... Skipper _______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
