On Tue, Dec 13, 2011 at 10:07 PM, Chris Barker <chris.bar...@noaa.gov>wrote:
> On Tue, Dec 13, 2011 at 11:29 AM, Bruce Southey <bsout...@gmail.com>wrote: > >> ** >> Reading data is hard and writing code that suits the diversity in the >> Numerical Python community is even harder! >> >> > yup > > Both loadtxt and genfromtxt functions (other functions are perhaps less >> important) perhaps need an upgrade to incorporate the new NA object. >> > > yes, if we are satisfiedthat the new NA object is, in fact, the way of the > future. > > >> Here I think loadtxt is a better target than genfromtxt because, as I >> understand it, it assumes the user really knows the data. Whereas >> genfromtxt can ask the data for the appropriatye format. >> >> So I agree that new 'superfast custom CSV reader for well-behaved data' >> function would be rather useful especially as an replacement for loadtxt. >> By that I mean reading data using a user specified format that essentially >> follows the CSV format ( >> http://en.wikipedia.org/wiki/Comma-separated_values) - it needs are to >> allow for NA object, skipping lines and user-defined delimiters. >> >> > I think that ideally, there could be one interface to reading tabular data > -- hopefully, it would be easy for the user to specify what the want, and > if they don't the code tries to figure it out. Also, under the hood, the > "easy" cases are special-cased to high-performing versions. > > genfromtxt sure looks close for an API > This I don't agree with. It has a huge amount of keywords that just confuse or intimidate a beginning user. There should be a dead simple interface, even the loadtxt API is on the heavy side. Ralf > -- it just needs the "high performance special cases" under the hood. It > may be that the way it's designed makes it very difficult to do that, > though -- I haven't looked closely enough to tell. > > At least that's what I'm thinking at the moment. > > -Chris > > >
_______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
