Ayushi Dalmia <ayushidalmia2...@gmail.com> Wrote in message: >> getsizeof() gives you the size of the list only; to complete the picture you >> >> have to add the sizes of the lines. >> >> >> >> However, why do you want to keep track of the actual memory used by >> >> variables in your script? You should instead concentrate on the algorithm, >> >> and as long as either the size of the dataset is manageable or you can limit >> >> the amount of data accessed at a given time you are golden. > > As I said, I need to merge large files and I cannot afford more I/O > operations. So in order to minimise the I/O operation I am writing in chunks. > Also, I need to use the merged files as indexes later which should be loaded > in the memory for fast access. Hence the concern. > > Can you please elaborate on the point of taking lines into consideration? >
Please don't doublespace your quotes. If you must use googlegroups, fix its bugs before posting. There's usually no net gain in trying to 'chunk' your output to a text file. The python file system already knows how to do that for a sequential file. For list of strings just add the getsizeof for the list to the sum of the getsizeof of all the list items. -- DaveA -- https://mail.python.org/mailman/listinfo/python-list