On Sat, 19 Feb 2022 11:28:31 +0000, Shaozhong SHI <shishaozh...@gmail.com> declaimed the following:
>I have a cvs file of 932956 row and have to have time.sleep in a Python >script. It takes a long time to process. > I'd echo the others... Unless you better explain WHY you have .sleep() (along with how often it is called, and what duration you sleep) the first recommendation would be to remove it. The most common justification for .sleep() is that one has CPU-BOUND processing and needs to force context switches to let other operations proceed more often than the system quantum time. Not normally a concern given Python's GIL and the presence of multi-core chips. How are you processing the (near) million rows of that CSV? If you are loading all of them into a large list you could be running Python list reallocations, or OS page swapping (though I wouldn't expect that on most modern systems -- maybe on a Raspberry-Pi/BeagleBone Black). Note: >>> import sys >>> sys.getsizeof("a") 50 >>> even a one-character string expands to 50 bytes. An EMPTY string takes up 51 bytes... Except for the empty string, that comes to about 49+<#chars> IF all characters fit an 8-bit encoding -- if any non 8-bit characters are in the string, the #chars needs to be multiplied by either 2 or 4 depending upon the widest representation needed. If you are doing read-one-record, process-one-record, repeat -- and have the .sleep() inside that loop... definitely remove the .sleep(). That loop is already I/O bound, the fastest you can obtain is determined by how rapidly the OS can transfer records from the file system to your program. -- Wulfraed Dennis Lee Bieber AF6VN wlfr...@ix.netcom.com http://wlfraed.microdiversity.freeddns.org/ -- https://mail.python.org/mailman/listinfo/python-list