[Koha-bugs] [Bug 33898] background_jobs_worker.pl may leave defunct children processes for extended periods of time
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=33898 Alessandro Tedesco changed: What|Removed |Added CC||alessandro.tedesco.it@gmail ||.com --- Comment #12 from Alessandro Tedesco --- Hi, I encounter the same issue. In my case, Koha 22.11.15 (Debian Package) on Ubuntu 22.04.1(MariaDB + RabbitMQ), the issue occurs when I do: From Koha Staff Interface 1) Stage records for import (everything goes fine) 2) Manage staged MARC records (everything goes fine) FromShell 3) # ps aux | grep 'Z' find the "background_jobs" as Zombie with a PID (e.g. 54679) From Koha Staff Interface 4) Stage records for import (everything goes fine) 5) Manage staged MARC records (everything goes well) FromShell 6) # ps aux | grep 'Z' find the "background_jobs" as Zombie with another PID (e.g. 54679) So, the fact that "background_jobs" remains a zombie does not actually prevent other processes from operating in Koha (simply once the new process has started, that zombie is killed and a new zombie is created). A reboot of the server kills the zombie process. Since it seems like a problem that doesn't affect everyone, could it be something related to the Server settings? Thanks -- You are receiving this mail because: You are watching all bug changes. ___ Koha-bugs mailing list Koha-bugs@lists.koha-community.org https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 33898] background_jobs_worker.pl may leave defunct children processes for extended periods of time
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=33898 --- Comment #13 from David Cook --- (In reply to Asymar Riu from comment #12) > A reboot of the server kills the zombie process. That's overkill. To get child zombie processes reaped, you'd just need to stop the parent process. Then the child process gets inherited by PID 1 and they'll get reaped. > Since it seems like a problem that doesn't affect everyone, could it be > something related to the Server settings? No, it's a real issue. It's more likely to happen to systems that use background jobs more, especially sporadically. This is just how parent/child processes work. Parent processes are responsible for reaping their child processes. As Kyle pointed out, currently child processes are only reaped under certain conditions. It's just about tweaking the code to reap them in a more responsible way. -- You are receiving this mail because: You are watching all bug changes. ___ Koha-bugs mailing list Koha-bugs@lists.koha-community.org https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 33898] background_jobs_worker.pl may leave defunct children processes for extended periods of time
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=33898 --- Comment #14 from Victor Grousset/tuxayo --- (In reply to David Cook from comment #13) > To get child zombie processes reaped, you'd just need to stop the parent > process. Then the child process gets inherited by PID 1 and they'll get > reaped. In other words @Asymar just restart the koha services (plack more specifically). I think it's here: https://wiki.koha-community.org/wiki/Commands_provided_by_the_Debian_packages#koha-plack -- You are receiving this mail because: You are watching all bug changes. ___ Koha-bugs mailing list Koha-bugs@lists.koha-community.org https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 33898] background_jobs_worker.pl may leave defunct children processes for extended periods of time
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=33898 Victor Grousset/tuxayo changed: What|Removed |Added Attachment #153194|0 |1 is obsolete|| --- Comment #15 from Victor Grousset/tuxayo --- Created attachment 162941 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=162941&action=edit Bug 33898: background_jobs_worker.pl may leave defunct children processes for extended periods of time It appears that the background jobs worker can leave defunct processes for periods of time. Though it is mostly harmless, it would be nice if that did not happen. Children are reaped automatically when start or wait_all_children are called. We only call start when a new job is found, and wait_all_children after exiting our while loop. The solution is to simply call reap_all_children after we sleep. This is a non-blocking call that will clean up those defunct processes. Test Plan: 1) Disable Rabbit 2) Set background_jobs_worker/max_processes to something like 5 3) Restart all the things! 4) Run a bunch of elastic index updates 5) Verify you have defunct processes 6) Apply this patch 7) Run more elastic index updates 8) Defunct processes should disappear every 10 seconds or so! If you do not see defunct processes, the test plan is to simply verify everything continues to work as expected. Signed-off-by: Emily Lamancusa Signed-off-by: Victor Grousset/tuxayo Used Asymar's test plan from comment 12. (based on record import) -- You are receiving this mail because: You are watching all bug changes. ___ Koha-bugs mailing list Koha-bugs@lists.koha-community.org https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 33898] background_jobs_worker.pl may leave defunct children processes for extended periods of time
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=33898 --- Comment #16 from Victor Grousset/tuxayo --- Thanks Asymar for your test plan, it's very simple and I was able to reproduce the issue and confirm that the patch prevents it. So with comment 12's test plan, there one zombie at the time. It's killed when the next staging/import starts. And a new zombie will stay at the end of the operation. So that blocking point is no more. (unless the elastic index update based test plan shows something more subtle or impactful that would benefit from being tested) Remaining is «there seems to be some discussion about implementation». -- You are receiving this mail because: You are watching all bug changes. ___ Koha-bugs mailing list Koha-bugs@lists.koha-community.org https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 33898] background_jobs_worker.pl may leave defunct children processes for extended periods of time
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=33898 --- Comment #17 from Marcel de Rooy --- In the above test plan Elastic is used, but note that those updates do not go via the regular worker script. There is another one (with more ore less the same code). Still to be merged :) -- You are receiving this mail because: You are watching all bug changes. ___ Koha-bugs mailing list Koha-bugs@lists.koha-community.org https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 33898] background_jobs_worker.pl may leave defunct children processes for extended periods of time
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=33898 --- Comment #18 from Marcel de Rooy --- Created attachment 162942 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=162942&action=edit Bug 33898: Alternative approach with receive frame timeout See bug 33898 comment6. -- You are receiving this mail because: You are watching all bug changes. ___ Koha-bugs mailing list Koha-bugs@lists.koha-community.org https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 33898] background_jobs_worker.pl may leave defunct children processes for extended periods of time
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=33898 Marcel de Rooy changed: What|Removed |Added Patch complexity|--- |Small patch -- You are receiving this mail because: You are watching all bug changes. ___ Koha-bugs mailing list Koha-bugs@lists.koha-community.org https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 33898] background_jobs_worker.pl may leave defunct children processes for extended periods of time
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=33898 Marcel de Rooy changed: What|Removed |Added Attachment #162942|0 |1 is obsolete|| --- Comment #19 from Marcel de Rooy --- Created attachment 162943 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=162943&action=edit Bug 33898: Alternative approach with receive frame timeout See bug 33898 comment6. Test plan: Based on comment12: Stage MARC import and manage. Look at the same time to the results of ps aux|grep Z. Verify that the lines with [background_jobs] disappear within 10 seconds. Signed-off-by: Marcel de Rooy -- You are receiving this mail because: You are watching all bug changes. ___ Koha-bugs mailing list Koha-bugs@lists.koha-community.org https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 33898] background_jobs_worker.pl may leave defunct children processes for extended periods of time
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=33898 --- Comment #20 from Marcel de Rooy --- The alternative patch addresses the issues raised earlier on the use of signals and follows the suggestions of David. Kyle, Victor and Asymar: Would this work for you? Note that you should apply only the second patch as an alternative for the first one. -- You are receiving this mail because: You are watching all bug changes. ___ Koha-bugs mailing list Koha-bugs@lists.koha-community.org https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 33898] background_jobs_worker.pl may leave defunct children processes for extended periods of time
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=33898 --- Comment #21 from Marcel de Rooy --- Created attachment 162944 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=162944&action=edit Bug 33898: (follow-up) Apply same solution to es_indexer_daemon Test plan: Similar as first patch with Elastic index jobs. -- You are receiving this mail because: You are watching all bug changes. ___ Koha-bugs mailing list Koha-bugs@lists.koha-community.org https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 33898] background_jobs_worker.pl may leave defunct children processes for extended periods of time
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=33898 Marcel de Rooy changed: What|Removed |Added Attachment #162944|0 |1 is obsolete|| --- Comment #22 from Marcel de Rooy --- Comment on attachment 162944 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=162944 Bug 33898: (follow-up) Apply same solution to es_indexer_daemon Oops this still needs a bit of attention -- You are receiving this mail because: You are watching all bug changes. ___ Koha-bugs mailing list Koha-bugs@lists.koha-community.org https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 33898] background_jobs_worker.pl may leave defunct children processes for extended periods of time
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=33898 --- Comment #23 from Marcel de Rooy --- (In reply to Marcel de Rooy from comment #22) > Comment on attachment 162944 [details] [review] > Bug 33898: (follow-up) Apply same solution to es_indexer_daemon > > Oops this still needs a bit of attention Ah I see $pm is not even used in the es_indexer_daemon.. -- You are receiving this mail because: You are watching all bug changes. ___ Koha-bugs mailing list Koha-bugs@lists.koha-community.org https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 33898] background_jobs_worker.pl may leave defunct children processes for extended periods of time
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=33898 Marcel de Rooy changed: What|Removed |Added Blocks||35920 Referenced Bugs: https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=35920 [Bug 35920] Centralize code from workers -- You are receiving this mail because: You are watching all bug changes. ___ Koha-bugs mailing list Koha-bugs@lists.koha-community.org https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 33898] background_jobs_worker.pl may leave defunct children processes for extended periods of time
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=33898 Kyle M Hall changed: What|Removed |Added Status|Failed QA |Signed Off -- You are receiving this mail because: You are watching all bug changes. ___ Koha-bugs mailing list Koha-bugs@lists.koha-community.org https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 33898] background_jobs_worker.pl may leave defunct children processes for extended periods of time
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=33898 Kyle M Hall changed: What|Removed |Added Attachment #162941|0 |1 is obsolete|| Attachment #162943|0 |1 is obsolete|| --- Comment #24 from Kyle M Hall --- Created attachment 162953 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=162953&action=edit Bug 33898: Alternative approach with receive frame timeout See bug 33898 comment6. Test plan: Based on comment12: Stage MARC import and manage. Look at the same time to the results of ps aux|grep Z. Verify that the lines with [background_jobs] disappear within 10 seconds. Signed-off-by: Marcel de Rooy Signed-off-by: Kyle M Hall -- You are receiving this mail because: You are watching all bug changes. ___ Koha-bugs mailing list Koha-bugs@lists.koha-community.org https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 33898] background_jobs_worker.pl may leave defunct children processes for extended periods of time
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=33898 --- Comment #25 from Kyle M Hall --- Created attachment 162954 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=162954&action=edit Bug 33898: Implement reaping for database polling Signed-off-by: Kyle M Hall -- You are receiving this mail because: You are watching all bug changes. ___ Koha-bugs mailing list Koha-bugs@lists.koha-community.org https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 33898] background_jobs_worker.pl may leave defunct children processes for extended periods of time
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=33898 Victor Grousset/tuxayo changed: What|Removed |Added Attachment #162953|0 |1 is obsolete|| --- Comment #26 from Victor Grousset/tuxayo --- Created attachment 163056 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=163056&action=edit Bug 33898: Alternative approach with receive frame timeout See bug 33898 comment6. Test plan: Based on comment12: Stage MARC import and manage. Look at the same time to the results of ps aux|grep Z. Verify that the lines with [background_jobs] disappear within 10 seconds. Signed-off-by: Marcel de Rooy Signed-off-by: Kyle M Hall Signed-off-by: Victor Grousset/tuxayo -- You are receiving this mail because: You are watching all bug changes. ___ Koha-bugs mailing list Koha-bugs@lists.koha-community.org https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 33898] background_jobs_worker.pl may leave defunct children processes for extended periods of time
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=33898 Victor Grousset/tuxayo changed: What|Removed |Added Attachment #162954|0 |1 is obsolete|| --- Comment #27 from Victor Grousset/tuxayo --- Created attachment 163057 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=163057&action=edit Bug 33898: Implement reaping for database polling Signed-off-by: Kyle M Hall Signed-off-by: Victor Grousset/tuxayo -- You are receiving this mail because: You are watching all bug changes. ___ Koha-bugs mailing list Koha-bugs@lists.koha-community.org https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 33898] background_jobs_worker.pl may leave defunct children processes for extended periods of time
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=33898 --- Comment #28 from Victor Grousset/tuxayo --- (In reply to Marcel de Rooy from comment #20) > The alternative patch addresses the issues raised earlier on the use of > signals and follows the suggestions of David. Thanks :) > Kyle, Victor and Asymar: Would this work for you? It works, so signing off. As for the signal vs no signals I don't know the good practices to QA this. Same for how this plays with when using database polling or the message queue. (I would have missed that the 1st alternate patch missed this) -- You are receiving this mail because: You are watching all bug changes. ___ Koha-bugs mailing list Koha-bugs@lists.koha-community.org https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 33898] background_jobs_worker.pl may leave defunct children processes for extended periods of time
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=33898 Kyle M Hall changed: What|Removed |Added Status|Signed Off |Passed QA --- Comment #29 from Kyle M Hall --- (In reply to Victor Grousset/tuxayo from comment #28) > (In reply to Marcel de Rooy from comment #20) > > The alternative patch addresses the issues raised earlier on the use of > > signals and follows the suggestions of David. > > Thanks :) > > > Kyle, Victor and Asymar: Would this work for you? > > It works, so signing off. > As for the signal vs no signals I don't know the good practices to QA this. > Same for how this plays with when using database polling or the message > queue. (I would have missed that the 1st alternate patch missed this) I think we can move this to PQA with the sign-offs we have! -- You are receiving this mail because: You are watching all bug changes. ___ Koha-bugs mailing list Koha-bugs@lists.koha-community.org https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 33898] background_jobs_worker.pl may leave defunct children processes for extended periods of time
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=33898 --- Comment #30 from Asymar Riu --- Hi, sorry for the delay in replying: I confirm that it works. Thank you very much! :) -- You are receiving this mail because: You are watching all bug changes. ___ Koha-bugs mailing list Koha-bugs@lists.koha-community.org https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 33898] background_jobs_worker.pl may leave defunct children processes for extended periods of time
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=33898 Katrin Fischer changed: What|Removed |Added Status|Passed QA |Pushed to master Version(s)||24.05.00 released in|| -- You are receiving this mail because: You are watching all bug changes. ___ Koha-bugs mailing list Koha-bugs@lists.koha-community.org https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 33898] background_jobs_worker.pl may leave defunct children processes for extended periods of time
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=33898 --- Comment #31 from Katrin Fischer --- Pushed for 24.05! Well done everyone, thank you! -- You are receiving this mail because: You are watching all bug changes. ___ Koha-bugs mailing list Koha-bugs@lists.koha-community.org https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 33898] background_jobs_worker.pl may leave defunct children processes for extended periods of time
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=33898 Fridolin Somers changed: What|Removed |Added Status|Pushed to master|Pushed to stable Version(s)|24.05.00|24.05.00,23.11.04 released in|| CC||fridolin.som...@biblibre.co ||m --- Comment #32 from Fridolin Somers --- Pushed to 23.11.x for 23.11.04 -- You are receiving this mail because: You are watching all bug changes. ___ Koha-bugs mailing list Koha-bugs@lists.koha-community.org https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 33898] background_jobs_worker.pl may leave defunct children processes for extended periods of time
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=33898 Lucas Gass changed: What|Removed |Added Version(s)|24.05.00,23.11.04 |24.05.00,23.11.04,23.05.10 released in|| Status|Pushed to stable|Pushed to oldstable CC||lu...@bywatersolutions.com --- Comment #33 from Lucas Gass --- Backported to 23.05.x for upcoming 23.05.10 -- You are receiving this mail because: You are watching all bug changes. ___ Koha-bugs mailing list Koha-bugs@lists.koha-community.org https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 33898] background_jobs_worker.pl may leave defunct children processes for extended periods of time
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=33898 Katrin Fischer changed: What|Removed |Added Status|Signed Off |Failed QA --- Comment #11 from Katrin Fischer --- Taking a closer look at the comments here, there seems to be some discussion about implementation, also question about how to reproduce the issue this is trying to fix. Truth is this has had comments from a lot of QA team members already and we are a little stuck. Kyle, could you review the last few comments for a start please? -- You are receiving this mail because: You are watching all bug changes. ___ Koha-bugs mailing list Koha-bugs@lists.koha-community.org https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 33898] background_jobs_worker.pl may leave defunct children processes for extended periods of time
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=33898 Marcel de Rooy changed: What|Removed |Added Blocks||35092 Referenced Bugs: https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=35092 [Bug 35092] [OMNIBUS] Remaining background job/worker issues -- You are receiving this mail because: You are watching all bug changes. ___ Koha-bugs mailing list Koha-bugs@lists.koha-community.org https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 33898] background_jobs_worker.pl may leave defunct children processes for extended periods of time
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=33898 Emily Lamancusa changed: What|Removed |Added Attachment #151997|0 |1 is obsolete|| --- Comment #4 from Emily Lamancusa --- Created attachment 153194 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=153194&action=edit Bug 33898: background_jobs_worker.pl may leave defunct children processes for extended periods of time It appears that the background jobs worker can leave defunct processes for periods of time. Though it is mostly harmless, it would be nice if that did not happen. Children are reaped automatically when start or wait_all_children are called. We only call start when a new job is found, and wait_all_children after exiting our while loop. The solution is to simply call reap_all_children after we sleep. This is a non-blocking call that will clean up those defunct processes. Test Plan: 1) Disable Rabbit 2) Set background_jobs_worker/max_processes to something like 5 3) Restart all the things! 4) Run a bunch of elastic index updates 5) Verify you have defunct processes 6) Apply this patch 7) Run more elastic index updates 8) Defunct processes should disappear every 10 seconds or so! If you do not see defunct processes, the test plan is to simply verify everything continues to work as expected. Signed-off-by: Emily Lamancusa -- You are receiving this mail because: You are watching all bug changes. ___ Koha-bugs mailing list Koha-bugs@lists.koha-community.org https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 33898] background_jobs_worker.pl may leave defunct children processes for extended periods of time
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=33898 Emily Lamancusa changed: What|Removed |Added CC||emily.lamancusa@montgomeryc ||ountymd.gov Status|Needs Signoff |Signed Off -- You are receiving this mail because: You are watching all bug changes. ___ Koha-bugs mailing list Koha-bugs@lists.koha-community.org https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 33898] background_jobs_worker.pl may leave defunct children processes for extended periods of time
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=33898 Marcel de Rooy changed: What|Removed |Added CC||m.de.r...@rijksmuseum.nl --- Comment #5 from Marcel de Rooy --- You did not see https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=32558#c36 btw? Signal handlers are not the most elegant way to address problems. Inherited by children too, altough no big deal here (not the pending signals). P::F allows you to define a run_on_finish callback. Could that be used? -- You are receiving this mail because: You are watching all bug changes. ___ Koha-bugs mailing list Koha-bugs@lists.koha-community.org https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 33898] background_jobs_worker.pl may leave defunct children processes for extended periods of time
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=33898 --- Comment #6 from David Cook --- (In reply to Marcel de Rooy from comment #5) > You did not see > https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=32558#c36 btw? > > Signal handlers are not the most elegant way to address problems. Inherited > by children too, altough no big deal here (not the pending signals). > P::F allows you to define a run_on_finish callback. Could that be used? I agree that signal handlers aren't the way to go. It looks like run_on_finish is called via wait_all_children, so probably won't work in this case. An alternative could be to add a timeout (e.g. 10 seconds) to $conn->receive_frame, and to call $pm->reap_finished_children() if it returns undef (before calling next()). That would be pretty lightweight, since receive_frame is just calling the can_read () on the select loop. In effect, it's doing a 10 second sleep which can be interrupted by an incoming frame. An incoming frame reaps via start(), or we'd reap after that timeout sleep. That should do the trick. -- You are receiving this mail because: You are watching all bug changes. ___ Koha-bugs mailing list Koha-bugs@lists.koha-community.org https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 33898] background_jobs_worker.pl may leave defunct children processes for extended periods of time
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=33898 --- Comment #7 from Kyle M Hall --- (In reply to David Cook from comment #6) > (In reply to Marcel de Rooy from comment #5) > > You did not see > > https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=32558#c36 btw? I'm not sure what you are trying to indicate. Your patch was included in the bug push and does not solve this problem. > An alternative could be to add a timeout (e.g. 10 seconds) to > $conn->receive_frame, and to call $pm->reap_finished_children() if it > returns undef (before calling next()). That would only work for Koha's using Rabbit and 10 seconds is quite a long delay between calls to reap all. -- You are receiving this mail because: You are watching all bug changes. ___ Koha-bugs mailing list Koha-bugs@lists.koha-community.org https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 33898] background_jobs_worker.pl may leave defunct children processes for extended periods of time
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=33898 --- Comment #8 from David Cook --- (In reply to Kyle M Hall from comment #7) > (In reply to David Cook from comment #6) > > An alternative could be to add a timeout (e.g. 10 seconds) to > > $conn->receive_frame, and to call $pm->reap_finished_children() if it > > returns undef (before calling next()). > > That would only work for Koha's using Rabbit and 10 seconds is quite a long > delay between calls to reap all. You could easily add it to the database polling block too. I suggested 10 seconds since that's the sleep used for the database polling block, but we could use shorter intervals for reaping. That said, why would 10 seconds be a long delay between calls to reap? If you have frequent jobs, then start() will be reaping previous ones. If you have infrequent jobs, then you probably won't have a lot of zombie children around, so 10 seconds seems all right to me to wait for reaping them. I suppose the problem compounds if you have many instances on a server, but I don't think they'd fill up the process table. -- You are receiving this mail because: You are watching all bug changes. ___ Koha-bugs mailing list Koha-bugs@lists.koha-community.org https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 33898] background_jobs_worker.pl may leave defunct children processes for extended periods of time
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=33898 Victor Grousset/tuxayo changed: What|Removed |Added CC||vic...@tuxayo.net --- Comment #9 from Victor Grousset/tuxayo --- > 4) Run a bunch of elastic index updates > 5) Verify you have defunct processes > If you do not see defunct processes, the test plan is to simply verify > everything continues to work as expected. Any idea what factors can help having the issue reproduced? Also, I renamed multiple times at record, that causes ES updates with a background worker, right? -- You are receiving this mail because: You are watching all bug changes. ___ Koha-bugs mailing list Koha-bugs@lists.koha-community.org https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 33898] background_jobs_worker.pl may leave defunct children processes for extended periods of time
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=33898 --- Comment #10 from Victor Grousset/tuxayo --- at record => a record -- You are receiving this mail because: You are watching all bug changes. ___ Koha-bugs mailing list Koha-bugs@lists.koha-community.org https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 33898] background_jobs_worker.pl may leave defunct children processes for extended periods of time
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=33898 Kyle M Hall changed: What|Removed |Added Status|NEW |Needs Signoff -- You are receiving this mail because: You are watching all bug changes. You are the assignee for the bug. ___ Koha-bugs mailing list Koha-bugs@lists.koha-community.org https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 33898] background_jobs_worker.pl may leave defunct children processes for extended periods of time
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=33898 --- Comment #1 from Kyle M Hall --- Created attachment 151993 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=151993&action=edit Bug 33898: background_jobs_worker.pl may leave defunct children processes for extended periods of time It appears that the background jobs worker can leave defunct processes for periods of time. Though it is mostly harmless, it would be nice if that did not happen. Children are reaped automatically when start or wait_all_children are called. We only call start when a new job is found, and wait_all_children after exiting our while loop. The solution is to simply call reap_all_children after we sleep. This is a non-blocking call that will clean up those defunct processes. Test Plan: 1) Disable Rabbit 2) Set background_jobs_worker/max_processes to something like 5 3) Restart all the things! 4) Run a bunch of elastic index updates 5) Verify you have defunct processes 6) Apply this patch 7) Run more elastic index updates 8) Defunct processes should disappear every 10 seconds or so! If you do not see defunct processes, the test plan is to simply verify everything continues to work as expected. -- You are receiving this mail because: You are watching all bug changes. You are the assignee for the bug. ___ Koha-bugs mailing list Koha-bugs@lists.koha-community.org https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 33898] background_jobs_worker.pl may leave defunct children processes for extended periods of time
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=33898 Kyle M Hall changed: What|Removed |Added Attachment #151993|0 |1 is obsolete|| --- Comment #2 from Kyle M Hall --- Created attachment 151994 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=151994&action=edit Bug 33898: background_jobs_worker.pl may leave defunct children processes for extended periods of time It appears that the background jobs worker can leave defunct processes for periods of time. Though it is mostly harmless, it would be nice if that did not happen. Children are reaped automatically when start or wait_all_children are called. We only call start when a new job is found, and wait_all_children after exiting our while loop. The solution is to simply call reap_all_children after we sleep. This is a non-blocking call that will clean up those defunct processes. Test Plan: 1) Disable Rabbit 2) Set background_jobs_worker/max_processes to something like 5 3) Restart all the things! 4) Run a bunch of elastic index updates 5) Verify you have defunct processes 6) Apply this patch 7) Run more elastic index updates 8) Defunct processes should disappear every 10 seconds or so! If you do not see defunct processes, the test plan is to simply verify everything continues to work as expected. -- You are receiving this mail because: You are the assignee for the bug. You are watching all bug changes. ___ Koha-bugs mailing list Koha-bugs@lists.koha-community.org https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 33898] background_jobs_worker.pl may leave defunct children processes for extended periods of time
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=33898 Kyle M Hall changed: What|Removed |Added Attachment #151994|0 |1 is obsolete|| --- Comment #3 from Kyle M Hall --- Created attachment 151997 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=151997&action=edit Bug 33898: background_jobs_worker.pl may leave defunct children processes for extended periods of time It appears that the background jobs worker can leave defunct processes for periods of time. Though it is mostly harmless, it would be nice if that did not happen. Children are reaped automatically when start or wait_all_children are called. We only call start when a new job is found, and wait_all_children after exiting our while loop. The solution is to simply call reap_all_children after we sleep. This is a non-blocking call that will clean up those defunct processes. Test Plan: 1) Disable Rabbit 2) Set background_jobs_worker/max_processes to something like 5 3) Restart all the things! 4) Run a bunch of elastic index updates 5) Verify you have defunct processes 6) Apply this patch 7) Run more elastic index updates 8) Defunct processes should disappear every 10 seconds or so! If you do not see defunct processes, the test plan is to simply verify everything continues to work as expected. -- You are receiving this mail because: You are the assignee for the bug. You are watching all bug changes. ___ Koha-bugs mailing list Koha-bugs@lists.koha-community.org https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 33898] background_jobs_worker.pl may leave defunct children processes for extended periods of time
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=33898 Kyle M Hall changed: What|Removed |Added QA Contact|testo...@bugs.koha-communit |k...@bywatersolutions.com |y.org | -- You are receiving this mail because: You are the assignee for the bug. You are watching all bug changes. ___ Koha-bugs mailing list Koha-bugs@lists.koha-community.org https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 33898] background_jobs_worker.pl may leave defunct children processes for extended periods of time
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=33898 Kyle M Hall changed: What|Removed |Added QA Contact|k...@bywatersolutions.com |testo...@bugs.koha-communit ||y.org Assignee|koha-b...@lists.koha-commun |k...@bywatersolutions.com |ity.org | -- You are receiving this mail because: You are watching all bug changes. You are the assignee for the bug. ___ Koha-bugs mailing list Koha-bugs@lists.koha-community.org https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 33898] background_jobs_worker.pl may leave defunct children processes for extended periods of time
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=33898 David Cook changed: What|Removed |Added CC||dc...@prosentient.com.au -- You are receiving this mail because: You are watching all bug changes. ___ Koha-bugs mailing list Koha-bugs@lists.koha-community.org https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
