On 08/10/13 21:46, Leena Gupta wrote:

I need to process a large csv file every 5-10 minutes. The file could
contain 3mill to 10 mill rows and size could be 6MB to 10MB(+).

Assuming you have a modern multi-core PC with >2G RAM then those sizes should not be a problem. If you have an older PC then you may need to be a bit more clever about how you load the files.

... I need to sum up a number value by grouping on
certain attributes and store the output in a datastore.

Again that sounds fairly normal. There are modules that can
help with that too.

I wanted to know if Python is recommended and can it be used
> for processing data in csv files of this size?

Yes, for sure.

Python has a csv library as well.

Yes and it is quite well documented but if you have specific
questions we can try to help.

--
Alan G
Author of the Learn to Program web site
http://www.alan-g.me.uk/
http://www.flickr.com/photos/alangauldphotos

_______________________________________________
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Reply via email to