Re: Any Clue about Devel::Cover Error Message Corrupted storable file (binary v2.7) at ../../lib/Storable.pm
On Sat, Jul 08, 2006 at 03:44:55PM -0700, Scott Wang wrote: Thanks Paul! (1) Yes, we do send SIGKILL (9) to kill the parent process even the child processes are still running and our purpose is to have a clean kill from root, so, do you think send SIGKILL (2) will be better? or, we could consider to send SIGKILL (2) to kill all the child processes before send SIGKILL (2) to kill parent process, do you think this may help? (2) Maybe you could send it something a little nicer which might allow to process to clean up., any more detail suggestion and example will be appreciated! I'm not quite sure what your environment looks like, so I can only give some general suggestions at best. If you can avoid signals completely, I would try to do that. Maybe you have some other method of IPC going on that you could use to use to send a kill yourself command. Singals in perl used not to be safe, that is they could lead to corruption or crashes. Since 5.8 they have been safe (providing you haven't explicitly made them unsafe). (Hopefully you are not using 5.6.) So if you can send a signal which can be caught, and then set up a signal handler which cleans up (if necessary) and then exists the program, that should allow things to work better. You might like to consider USR1 or USR2 for this, if you are not already using them. Or you might prefer TERM. In any case, you should ensure that it is a signal which can be caught, which is not the case with KILL. (3) We might also consider to give some sleep time before destroy the test object, which causes process killing and cleaning up, but, is there a way we can know Devel::Cover has doen its work to write data? (4) Devel::Cover does its work in the very last END block., does this mean that Devel::Cover will not write data until the main test process run to its end? Sleeping, then killing won't work. Although Devel::Cover collects data the entire time a program is running (unless you turn it off at some point), the data will not be saved until its END block is run. This will normally be the very last END block. This means that unless you let the program run to completion the coverage data will not be fully saved. You might hinder this process by calling _exit, calling exec, sending SIGKILL or doing anything else that stops normal completion of the program. For example, a perl test script loads the perl module that is under testing and excise the functions in that module, sometime the test script needs use system to call some other perl scripts (that also is under testing), Devel::Cover will not write data until the main test script reach its end point or exit? Correct. But the scripts you are calling with system will not have any coverage data collected unless you have arranged for that in some way. And if you do that, the coverage data for those scripts will be written as soon as the scripts exit. (5) Is there a way we can tell which data files (under structure folder?) have been corrupted? Just try reading them with the retrieve method from Storable. Better yet, send me patches for Devel::Cover::DB::validate_db and Devel::Cover::DB::is_valid to do something more than just check that one file exists there ;-) (Those methods should probably be merged too.) (And sorry they are not already written. I suspect something a little more robust there might have saved you a bit of pain.) -- Paul Johnson - [EMAIL PROTECTED] http://www.pjcj.net
Re: Any Clue about Devel::Cover Error Message Corrupted storable file (binary v2.7) at ../../lib/Storable.pm
Thanks Paul! (1) Yes, we do send SIGKILL (9) to kill the parent process even the child processes are still running and our purpose is to have a clean kill from root, so, do you think send SIGKILL (2) will be better? or, we could consider to send SIGKILL (2) to kill all the child processes before send SIGKILL (2) to kill parent process, do you think this may help? (2) Maybe you could send it something a little nicer which might allow to process to clean up., any more detail suggestion and example will be appreciated! (3) We might also consider to give some sleep time before destroy the test object, which causes process killing and cleaning up, but, is there a way we can know Devel::Cover has doen its work to write data? (4) Devel::Cover does its work in the very last END block., does this mean that Devel::Cover will not write data until the main test process run to its end? For example, a perl test script loads the perl module that is under testing and excise the functions in that module, sometime the test script needs use system to call some other perl scripts (that also is under testing), Devel::Cover will not write data until the main test script reach its end point or exit? (5) Is there a way we can tell which data files (under structure folder?) have been corrupted? We are trying to integrate Devel::Cover into our regular test driven development process, to make Devel::Cover, the amazing code coverage tool in Perl world, work correctly with our test to improve the data accuracy is very critial for us to move forward with the code coverage tool. Thanks a lot for your kind helps! Scott --- Paul Johnson [EMAIL PROTECTED] wrote: On Fri, Jul 07, 2006 at 10:26:13AM -0700, Scott Wang wrote: Hi Paul, Even, currently, there is no any zero size data file in structure folder in my regression code coverage run (lots of suites), I still got lots of messages like below: --- Corrupted storable file (binary v2.7) at ../../lib/Storable.pm (autosplit into ../../lib/auto/Storable/_retrieve.al) line 331, at //Devel/Cover/DB/Structure.pm line 269 END failed--call queue aborted. - Seems some data files got corrupted somehow. The process killing (kill parent process before killing child process and make chid process be terminated abnormally) is one possibility, I also noticed that useing system or other folk method to execute command could also cause this problem. Any clue about above issue Corrupted storable file (binary v2.7) at ... and does anybody also experience this issue and know how to deal with it? Also, does anybody have similar experience on killing process and folking process cause Magic number ... issue and orrupted storable file ... issue? In lieu of discussing either licences or alligators, let me try to answer this. Killing the process sounds like the most likely cause of this problem. If you manage to kill the process while it is writing out a storable file the file will very probably be corrupted. How are you killing the process? Are you sending it SIGKILL (9) for example? Maybe you could send it something a little nicer which might allow to process to clean up. Devel::Cover does its work in the very last END block. You really need to let it run to completion. -- Paul Johnson - [EMAIL PROTECTED] http://www.pjcj.net __ Do You Yahoo!? Tired of spam? Yahoo! Mail has the best spam protection around http://mail.yahoo.com
Any Clue about Devel::Cover Error Message Corrupted storable file (binary v2.7) at ../../lib/Storable.pm
Hi Paul, Even, currently, there is no any zero size data file in structure folder in my regression code coverage run (lots of suites), I still got lots of messages like below: --- Corrupted storable file (binary v2.7) at ../../lib/Storable.pm (autosplit into ../../lib/auto/Storable/_retrieve.al) line 331, at //Devel/Cover/DB/Structure.pm line 269 END failed--call queue aborted. - Seems some data files got corrupted somehow. The process killing (kill parent process before killing child process and make chid process be terminated abnormally) is one possibility, I also noticed that useing system or other folk method to execute command could also cause this problem. Any clue about above issue Corrupted storable file (binary v2.7) at ... and does anybody also experience this issue and know how to deal with it? Also, does anybody have similar experience on killing process and folking process cause Magic number ... issue and orrupted storable file ... issue? Thanks in advance, Scott --- Paul Johnson [EMAIL PROTECTED] wrote: On Sat, Jun 10, 2006 at 08:14:09PM -0700, Scott Wang wrote: Hi Paul, You are right, I found a 0 size file under the cover_db/structure folder. After I removed that 0 size file, my cover worked fine to merge all the data and generated HTML file. The 0 size file, I think, is because some test was killed for timeout because of the slow-down of the test process since we instrumented all test and product scripts with Devel::Cover. When the test process was killed and terminated abnormal, I think, Devel::Cover was still trying to grab the test process to generate data, so there was no any data actually got generated because the test processes had been killed. Is this a reasonable explaination to the 0 size data file? It sounds plausible, but there could be any number of reasons. Perhaps the file system filled up, or you have some rogue process running around truncating Devel::Cover databases. But the test process being killed probably has something to do with it. The files in the structure directory contain information about the structure of files. That is, the statements, branches and conditions in the file, and other similar data. -- Paul Johnson - [EMAIL PROTECTED] http://www.pjcj.net __ Do You Yahoo!? Tired of spam? Yahoo! Mail has the best spam protection around http://mail.yahoo.com
Re: Any Clue about Devel::Cover Error Message Corrupted storable file (binary v2.7) at ../../lib/Storable.pm
On Fri, Jul 07, 2006 at 10:26:13AM -0700, Scott Wang wrote: Hi Paul, Even, currently, there is no any zero size data file in structure folder in my regression code coverage run (lots of suites), I still got lots of messages like below: --- Corrupted storable file (binary v2.7) at ../../lib/Storable.pm (autosplit into ../../lib/auto/Storable/_retrieve.al) line 331, at //Devel/Cover/DB/Structure.pm line 269 END failed--call queue aborted. - Seems some data files got corrupted somehow. The process killing (kill parent process before killing child process and make chid process be terminated abnormally) is one possibility, I also noticed that useing system or other folk method to execute command could also cause this problem. Any clue about above issue Corrupted storable file (binary v2.7) at ... and does anybody also experience this issue and know how to deal with it? Also, does anybody have similar experience on killing process and folking process cause Magic number ... issue and orrupted storable file ... issue? In lieu of discussing either licences or alligators, let me try to answer this. Killing the process sounds like the most likely cause of this problem. If you manage to kill the process while it is writing out a storable file the file will very probably be corrupted. How are you killing the process? Are you sending it SIGKILL (9) for example? Maybe you could send it something a little nicer which might allow to process to clean up. Devel::Cover does its work in the very last END block. You really need to let it run to completion. -- Paul Johnson - [EMAIL PROTECTED] http://www.pjcj.net
