Re: How to read such file and sumarize the data?

2010-11-18 Thread huisky
thank you Martin. You are right. But the elapsed time is also okay for me. And i would like to assume that the total CPU time equals to the number of CPUs multiply the elapsed time. As to the number you mentioned, it is the 'process id', so it will be no problem to identify each job. Huiksy On

Re: How to read such file and sumarize the data?

2010-11-18 Thread huisky
the number before @ is the process id in the linux server and it is identical. So i do NOT think distinguish each job's starting and ending time is difficult in this case. Huisky On Nov 17, 11:38 pm, Tim Harig user...@ilthio.net wrote: On 2010-11-17, huisky hui...@gmail.com wrote: I want to

Re: How to read such file and sumarize the data?

2010-11-18 Thread huisky
Thank you very much for your reply. I think you just count the total number of NCPU used for each user. And it does NOT show how much time used for each user. Huisky On Nov 18, 12:10 am, Steve Holden st...@holdenweb.com wrote: On 11/17/2010 4:45 PM, huisky wrote: Say I have following log file,

How to read such file and sumarize the data?

2010-11-17 Thread huisky
Say I have following log file, which records the code usage. I want to read this file and do the summarize how much total CPU time consumed for each user. Is Python able to do so or say easy to achieve this?, anybody can give me some hints, appricate very much! Example log file.

Re: How to read such file and sumarize the data?

2010-11-17 Thread MRAB
On 17/11/2010 21:45, huisky wrote: Say I have following log file, which records the code usage. I want to read this file and do the summarize how much total CPU time consumed for each user. Is Python able to do so or say easy to achieve this?, anybody can give me some hints, appricate very much!

Re: How to read such file and sumarize the data?

2010-11-17 Thread Tim Harig
On 2010-11-17, huisky hui...@gmail.com wrote: I want to read this file and do the summarize how much total CPU time consumed for each user. Is Python able to do so or say easy to achieve this?, anybody can give me some hints, appricate very much! The question is, is the information you want

Re: How to read such file and sumarize the data?

2010-11-17 Thread Tim Harig
On 2010-11-17, MRAB pyt...@mrabarnett.plus.com wrote: When a user starts, save the info in the first dict, and when a user finishes, calculate the elapsed time and add it to the total for that user. Perhaps you know more about the structure of this data. It seems to me that a user might have

Re: How to read such file and sumarize the data?

2010-11-17 Thread Steve Holden
On 11/17/2010 4:45 PM, huisky wrote: Say I have following log file, which records the code usage. I want to read this file and do the summarize how much total CPU time consumed for each user. Is Python able to do so or say easy to achieve this?, anybody can give me some hints, appricate very

Re: How to read such file and sumarize the data?

2010-11-17 Thread MRAB
On 17/11/2010 22:49, Tim Harig wrote: On 2010-11-17, MRABpyt...@mrabarnett.plus.com wrote: When a user starts, save the info in the first dict, and when a user finishes, calculate the elapsed time and add it to the total for that user. Perhaps you know more about the structure of this data.

Re: How to read such file and sumarize the data?

2010-11-17 Thread Martin Gregorie
On Wed, 17 Nov 2010 13:45:58 -0800, huisky wrote: Say I have following log file, which records the code usage. I want to read this file and do the summarize how much total CPU time consumed for each user. Two points you should think about: - I don't think you can extract CPU time from this

Re: How to read such file and sumarize the data?

2010-11-17 Thread Terry Reedy
On 11/17/2010 6:10 PM, Steve Holden wrote: $ cat data.py lines = open(data.txt).readlines() Since you iterate through the file just once, there is no reason I can think of to make a complete in-memory copy. That would be a problem with a multi-gigabyte log file ;=). In 3.x at least, open

Re: How to read such file and sumarize the data?

2010-11-17 Thread Steve Holden
On 11/17/2010 7:51 PM, Terry Reedy wrote: On 11/17/2010 6:10 PM, Steve Holden wrote: $ cat data.py lines = open(data.txt).readlines() Since you iterate through the file just once, there is no reason I can think of to make a complete in-memory copy. That would be a problem with a