Hi, I think writing a Python script that convert your txt file to one netcdf file, reading the txt file one line at a time, and then use the netcdf file normally would be a good solution!
Best, David Excerpts from Chao YUE's message of mar. mars 20 13:33:56 +0100 2012: > Dear all, > > I received a file from others which contains ~30 million lines and in size > of ~500M. > I try read it with numpy.genfromtxt in ipython interactive mode. Then > ipython crashed. > The data contains lat,lon,var1,year, the year ranges from 1001 to 2006. > Finally I want to write the > data to netcdf for separate years and feed them into the model. I guess I > need a better way to do this? > anyone would be any idea is highly appreciated. > > > lon,lat,year,area_burned > -180.0,65.0,1001,0 > -180.0,65.0,1002,0 > -180.0,65.0,1003,0 > -180.0,65.0,1004,0 > -180.0,65.0,1005,0 > -180.0,65.0,1006,0 > -180.0,65.0,1007,0 > > thanks and cheers, > > Chao > _______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
