On Thu, May 28, 2009 at 5:57 AM, Phil Dawes <p...@phildawes.net> wrote: > Yep you're right- not sure why I didn't see that. > I made a start on adding dialects to the csv module: > > http://github.com/phildawes/factor/commit/752e6a5e075dfc07a692a0ae797164c5da85d2d7 > > I'll try and do a little more tonight. >
I used csv last night and found it to be wicked fast even for my 306 column monster. It parsed it all in about 32 seconds and I could run a combinator function on the whole dataset in 10 ms. > Using a tuple for config feels a little clunky - is there a better way > that I'm missing? > with-delimiter seemed a little awkward to me. I thought I should be setting up a config object and passing it to file>csv or whatever, but that could just be the OO in me talking. I haven't really grasped the factor way yet. One thing that would have been helpful would have been a doc string on the row function to describe what happens when you call row at the end of the dataset. I think what I really wanted was some lazy sequence protocol-y thing. But anyway, wow. In optimized python I was happy to go through the dataset in a few minutes and not blow out of a half gigabyte of ram. In factor I naively loaded the whole of the data into ram in half a minute and I never bothered to check ram usage. (there was no need) I didn't have a lot of time last night but I was able to determine, thanks to factor, that I had 15.8K rows in my file and all but around 160 rows had 306 columns. -- Darrin ------------------------------------------------------------------------------ Register Now for Creativity and Technology (CaT), June 3rd, NYC. CaT is a gathering of tech-side developers & brand creativity professionals. Meet the minds behind Google Creative Lab, Visual Complexity, Processing, & iPhoneDevCamp as they present alongside digital heavyweights like Barbarian Group, R/GA, & Big Spaceship. http://p.sf.net/sfu/creativitycat-com _______________________________________________ Factor-talk mailing list Factor-talk@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/factor-talk
