Memory Error while working with pandas dataframe. Description of Environment Windows 7 python 3.4.2 32-bit version pandas 0.16.0
We are running into the error described below. Any help provided will be sincerely appreciated. We are able to read a 300MB Csv file into a dataframe using the read_csv function. While working with the dataframe we ran into memory error. We used the pd.Concat function to concatenate two dataframes. So we decided to use chunksize for lazy reading. Chunking returns an object of type TextFileReader. http://pandas.pydata.org/pandas-docs/stable/io.html#iterating-through-files-chunk-by-chunk We are able to iterate over this object once as a debugging measure. The iterator gets exhausted after iterating once. So we are not able to convert the TextFileReader object back into a dataframe, using the pd.concat function. Error Traceback (most recent call last): File "psindia.py", line 60, in <module> data=pd.concat(tp,ignore_index=True) File "C:\Python34\lib\site-packages\pandas\tools\merge.py", line 754, in conca t copy=copy) File "C:\Python34\lib\site-packages\pandas\tools\merge.py", line 799, in __ini t__ raise ValueError('All objects passed were None') ValueError: All objects passed were None Thanks for your time. -- https://mail.python.org/mailman/listinfo/python-list
