[Koha-bugs] [Bug 29507] Speed up auto renew cronjob via parallel processing
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=29507 Emily Lamancusa (emlam) changed: What|Removed |Added Blocks||38156 Referenced Bugs: https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=38156 [Bug 38156] Auto renew cron job mangles digest notices when parallel processing is enabled -- You are receiving this mail because: You are watching all bug changes. ___ Koha-bugs mailing list Koha-bugs@lists.koha-community.org https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 29507] Speed up auto renew cronjob via parallel processing
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=29507 Martin Renvoize changed: What|Removed |Added CC|martin.renvoize@ptfs-europe | |.com| -- You are receiving this mail because: You are watching all bug changes. ___ Koha-bugs mailing list Koha-bugs@lists.koha-community.org https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 29507] Speed up auto renew cronjob via parallel processing
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=29507 David Nind changed: What|Removed |Added Text to go in the||This enhancement enables release notes||using parallel processing ||to speed up the running of ||the automatic renewals cron ||job ||(misc/cronjobs/automatic_re ||newals.pl). This cron job ||can take hours to run for ||libraries with thousands of ||items to renew. || ||To use ||this enhancement, add this ||setting to the instance's ||koha-conf.xml - adjusting ||the value depending on the ||system resources available: || || || ||1 || || CC||da...@davidnind.com --- Comment #25 from David Nind --- I've had a go at adding a release note - please update if I haven't described how to use this enhancement correctly! -- You are receiving this mail because: You are watching all bug changes. ___ Koha-bugs mailing list Koha-bugs@lists.koha-community.org https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 29507] Speed up auto renew cronjob via parallel processing
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=29507 --- Comment #24 from David Cook --- (In reply to Nick Clemens (kidclamp) from comment #23) > (In reply to Jonathan Druart from comment #22) > > Why didn't we enqueue several background jobs instead? > > > > Isn't the point of the workers to deal with parallelization and > > configuration of how to process jobs? > > I think we didn't have as much progress on the background jobs when I > started this. Would it be the goal now to have most cronjobs submit > background jobs? I see pros and cons with both approaches I think. If we did enqueue them as background jobs and there were a large number of jobs across multiple Koha instances, that would produce a higher load on the server than doing the work in the cronjob itself. While parallel processing in the cronjob would produce a higher load as well, it would still be smaller than the load using background jobs (although the impact will vary depending on the number of instances per system and the amount of resources available) But... we'd get through the cronjobs faster. (We have found the cronjobs can take a very long time to finish, because each cronjob has to run for each library one by one.) I'm intrigued by both approaches... -- You are receiving this mail because: You are watching all bug changes. ___ Koha-bugs mailing list Koha-bugs@lists.koha-community.org https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 29507] Speed up auto renew cronjob via parallel processing
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=29507 --- Comment #23 from Nick Clemens (kidclamp) --- (In reply to Jonathan Druart from comment #22) > Why didn't we enqueue several background jobs instead? > > Isn't the point of the workers to deal with parallelization and > configuration of how to process jobs? I think we didn't have as much progress on the background jobs when I started this. Would it be the goal now to have most cronjobs submit background jobs? -- You are receiving this mail because: You are watching all bug changes. ___ Koha-bugs mailing list Koha-bugs@lists.koha-community.org https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 29507] Speed up auto renew cronjob via parallel processing
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=29507 --- Comment #22 from Jonathan Druart --- Why didn't we enqueue several background jobs instead? Isn't the point of the workers to deal with parallelization and configuration of how to process jobs? -- You are receiving this mail because: You are watching all bug changes. ___ Koha-bugs mailing list Koha-bugs@lists.koha-community.org https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 29507] Speed up auto renew cronjob via parallel processing
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=29507 --- Comment #21 from Katrin Fischer --- Pushed for 24.11! Well done everyone, thank you! -- You are receiving this mail because: You are watching all bug changes. ___ Koha-bugs mailing list Koha-bugs@lists.koha-community.org https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 29507] Speed up auto renew cronjob via parallel processing
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=29507 Katrin Fischer changed: What|Removed |Added Version(s)||24.11.00 released in|| Status|Passed QA |Pushed to main -- You are receiving this mail because: You are watching all bug changes. ___ Koha-bugs mailing list Koha-bugs@lists.koha-community.org https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 29507] Speed up auto renew cronjob via parallel processing
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=29507 Katrin Fischer changed: What|Removed |Added Keywords||additional_work_needed --- Comment #20 from Katrin Fischer --- How can we best document this new addition to the koha-conf.xml file? -- You are receiving this mail because: You are watching all bug changes. ___ Koha-bugs mailing list Koha-bugs@lists.koha-community.org https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 29507] Speed up auto renew cronjob via parallel processing
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=29507 Martin Renvoize changed: What|Removed |Added Status|Signed Off |Passed QA --- Comment #19 from Martin Renvoize --- Nice little performance enhancement. Passing QA -- You are receiving this mail because: You are watching all bug changes. ___ Koha-bugs mailing list Koha-bugs@lists.koha-community.org https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 29507] Speed up auto renew cronjob via parallel processing
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=29507 Martin Renvoize changed: What|Removed |Added Attachment #168363|0 |1 is obsolete|| --- Comment #18 from Martin Renvoize --- Created attachment 168382 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=168382&action=edit Bug 29507: Tidy Signed-off-by: Martin Renvoize -- You are receiving this mail because: You are watching all bug changes. ___ Koha-bugs mailing list Koha-bugs@lists.koha-community.org https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 29507] Speed up auto renew cronjob via parallel processing
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=29507 Martin Renvoize changed: What|Removed |Added Attachment #168362|0 |1 is obsolete|| --- Comment #17 from Martin Renvoize --- Created attachment 168381 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=168381&action=edit Bug 29507: Speed up auto renew cronjob via parallel processing The cron can take a very long time to run on systems with many issues. For example, a partner with ~250k auto_renew issues is taking about 9 hours to run. If we run that same number of issues in 5 parallel chunks ( splitting the number of issues as evenly as possible ), it could take under 2 hours. Test Plan: 1) Generate a number of issues marked for auto_renew 2) Run the automatic_renewals.pl, use the `time` utility to track how much time it took to run 3) Set parallel_loops to 10 in auto_renew_cronjob section of config in koha-conf 4) Repeat step 2, note the improvement in speed 5) Experiment with other values Signed-off-by: Matt Blenkinsop Signed-off-by: Martin Renvoize -- You are receiving this mail because: You are watching all bug changes. ___ Koha-bugs mailing list Koha-bugs@lists.koha-community.org https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 29507] Speed up auto renew cronjob via parallel processing
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=29507 --- Comment #16 from Nick Clemens (kidclamp) --- Created attachment 168363 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=168363&action=edit Bug 29507: Tidy -- You are receiving this mail because: You are watching all bug changes. ___ Koha-bugs mailing list Koha-bugs@lists.koha-community.org https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 29507] Speed up auto renew cronjob via parallel processing
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=29507 Nick Clemens (kidclamp) changed: What|Removed |Added Attachment #165091|0 |1 is obsolete|| Attachment #165092|0 |1 is obsolete|| --- Comment #15 from Nick Clemens (kidclamp) --- Created attachment 168362 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=168362&action=edit Bug 29507: Speed up auto renew cronjob via parallel processing The cron can take a very long time to run on systems with many issues. For example, a partner with ~250k auto_renew issues is taking about 9 hours to run. If we run that same number of issues in 5 parallel chunks ( splitting the number of issues as evenly as possible ), it could take under 2 hours. Test Plan: 1) Generate a number of issues marked for auto_renew 2) Run the automatic_renewals.pl, use the `time` utility to track how much time it took to run 3) Set parallel_loops to 10 in auto_renew_cronjob section of config in koha-conf 4) Repeat step 2, note the improvement in speed 5) Experiment with other values Signed-off-by: Matt Blenkinsop -- You are receiving this mail because: You are watching all bug changes. ___ Koha-bugs mailing list Koha-bugs@lists.koha-community.org https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 29507] Speed up auto renew cronjob via parallel processing
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=29507 Martin Renvoize changed: What|Removed |Added QA Contact|testo...@bugs.koha-communit |martin.renvoize@ptfs-europe |y.org |.com -- You are receiving this mail because: You are watching all bug changes. ___ Koha-bugs mailing list Koha-bugs@lists.koha-community.org https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 29507] Speed up auto renew cronjob via parallel processing
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=29507 --- Comment #14 from Nick Clemens --- Created attachment 165092 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=165092&action=edit Bug 29507: Tidy -- You are receiving this mail because: You are watching all bug changes. ___ Koha-bugs mailing list Koha-bugs@lists.koha-community.org https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 29507] Speed up auto renew cronjob via parallel processing
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=29507 Nick Clemens changed: What|Removed |Added Attachment #161326|0 |1 is obsolete|| Attachment #161327|0 |1 is obsolete|| --- Comment #13 from Nick Clemens --- Created attachment 165091 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=165091&action=edit Bug 29507: Speed up auto renew cronjob via parallel processing The cron can take a very long time to run on systems with many issues. For example, a partner with ~250k auto_renew issues is taking about 9 hours to run. If we run that same number of issues in 5 parallel chunks ( splitting the number of issues as evenly as possible ), it could take under 2 hours. Test Plan: 1) Generate a number of issues marked for auto_renew 2) Run the automatic_renewals.pl, use the `time` utility to track how much time it took to run 3) Set parallel_loops to 10 in auto_renew_cronjob section of config in koha-conf 4) Repeat step 2, note the improvement in speed 5) Experiment with other values Signed-off-by: Matt Blenkinsop -- You are receiving this mail because: You are watching all bug changes. ___ Koha-bugs mailing list Koha-bugs@lists.koha-community.org https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 29507] Speed up auto renew cronjob via parallel processing
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=29507 Nick Clemens changed: What|Removed |Added Status|Patch doesn't apply |Signed Off -- You are receiving this mail because: You are watching all bug changes. ___ Koha-bugs mailing list Koha-bugs@lists.koha-community.org https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 29507] Speed up auto renew cronjob via parallel processing
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=29507 Marcel de Rooy changed: What|Removed |Added Status|Signed Off |Patch doesn't apply --- Comment #12 from Marcel de Rooy --- Applying: Bug 29507: Tidy error: sha1 information is lacking or useless (misc/cronjobs/automatic_renewals.pl). error: could not build fake ancestor -- You are receiving this mail because: You are watching all bug changes. ___ Koha-bugs mailing list Koha-bugs@lists.koha-community.org https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 29507] Speed up auto renew cronjob via parallel processing
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=29507 Matt Blenkinsop changed: What|Removed |Added Attachment #161315|0 |1 is obsolete|| --- Comment #11 from Matt Blenkinsop --- Created attachment 161327 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=161327&action=edit Bug 29507: Tidy Signed-off-by: Matt Blenkinsop -- You are receiving this mail because: You are watching all bug changes. ___ Koha-bugs mailing list Koha-bugs@lists.koha-community.org https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 29507] Speed up auto renew cronjob via parallel processing
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=29507 Matt Blenkinsop changed: What|Removed |Added Attachment #161314|0 |1 is obsolete|| --- Comment #10 from Matt Blenkinsop --- Created attachment 161326 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=161326&action=edit Bug 29507: Speed up auto renew cronjob via parallel processing The cron can take a very long time to run on systems with many issues. For example, a partner with ~250k auto_renew issues is taking about 9 hours to run. If we run that same number of issues in 5 parallel chunks ( splitting the number of issues as evenly as possible ), it could take under 2 hours. Test Plan: 1) Generate a number of issues marked for auto_renew 2) Run the automatic_renewals.pl, use the `time` utility to track how much time it took to run 3) Set parallel_loops to 10 in auto_renew_cronjob section of config in koha-conf 4) Repeat step 2, note the improvement in speed 5) Experiment with other values Signed-off-by: Matt Blenkinsop -- You are receiving this mail because: You are watching all bug changes. ___ Koha-bugs mailing list Koha-bugs@lists.koha-community.org https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 29507] Speed up auto renew cronjob via parallel processing
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=29507 Matt Blenkinsop changed: What|Removed |Added Status|Needs Signoff |Signed Off -- You are receiving this mail because: You are watching all bug changes. ___ Koha-bugs mailing list Koha-bugs@lists.koha-community.org https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 29507] Speed up auto renew cronjob via parallel processing
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=29507 --- Comment #9 from Nick Clemens --- Created attachment 161315 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=161315&action=edit Bug 29507: Tidy -- You are receiving this mail because: You are watching all bug changes. ___ Koha-bugs mailing list Koha-bugs@lists.koha-community.org https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 29507] Speed up auto renew cronjob via parallel processing
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=29507 Nick Clemens changed: What|Removed |Added Attachment #157772|0 |1 is obsolete|| --- Comment #8 from Nick Clemens --- Created attachment 161314 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=161314&action=edit Bug 29507: Speed up auto renew cronjob via parallel processing The cron can take a very long time to run on systems with many issues. For example, a partner with ~250k auto_renew issues is taking about 9 hours to run. If we run that same number of issues in 5 parallel chunks ( splitting the number of issues as evenly as possible ), it could take under 2 hours. Test Plan: 1) Generate a number of issues marked for auto_renew 2) Run the automatic_renewals.pl, use the `time` utility to track how much time it took to run 3) Set parallel_loops to 10 in auto_renew_cronjob section of config in koha-conf 4) Repeat step 2, note the improvement in speed 5) Experiment with other values -- You are receiving this mail because: You are watching all bug changes. ___ Koha-bugs mailing list Koha-bugs@lists.koha-community.org https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 29507] Speed up auto renew cronjob via parallel processing
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=29507 Nick Clemens changed: What|Removed |Added Status|Patch doesn't apply |Needs Signoff -- You are receiving this mail because: You are watching all bug changes. ___ Koha-bugs mailing list Koha-bugs@lists.koha-community.org https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 29507] Speed up auto renew cronjob via parallel processing
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=29507 Matt Blenkinsop changed: What|Removed |Added Status|Needs Signoff |Patch doesn't apply CC||matt.blenkinsop@ptfs-europe ||.com -- You are receiving this mail because: You are watching all bug changes. ___ Koha-bugs mailing list Koha-bugs@lists.koha-community.org https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 29507] Speed up auto renew cronjob via parallel processing
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=29507 Nick Clemens changed: What|Removed |Added Attachment #150692|0 |1 is obsolete|| --- Comment #7 from Nick Clemens --- Created attachment 157772 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=157772&action=edit Bug 29507: Speed up auto renew cronjob via parallel processing The cron can take a very long time to run on systems with many issues. For example, a partner with ~250k auto_renew issues is taking about 9 hours to run. If we run that same number of issues in 5 parallel chunks ( splitting the number of issues as evenly as possible ), it could take under 2 hours. Test Plan: 1) Generate a number of issues marked for auto_renew 2) Run the automatic_renewals.pl, use the `time` utility to track how much time it took to run 3) Set parallel_loops to 10 in auto_renew_cronjob section of config in koha-conf 4) Repeat step 2, note the improvement in speed 5) Experiment with other values -- You are receiving this mail because: You are watching all bug changes. ___ Koha-bugs mailing list Koha-bugs@lists.koha-community.org https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 29507] Speed up auto renew cronjob via parallel processing
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=29507 Nick Clemens changed: What|Removed |Added Status|Patch doesn't apply |Needs Signoff -- You are receiving this mail because: You are watching all bug changes. ___ Koha-bugs mailing list Koha-bugs@lists.koha-community.org https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 29507] Speed up auto renew cronjob via parallel processing
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=29507 Sam Lau changed: What|Removed |Added CC||sama...@gmail.com Status|Needs Signoff |Patch doesn't apply -- You are receiving this mail because: You are watching all bug changes. ___ Koha-bugs mailing list Koha-bugs@lists.koha-community.org https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 29507] Speed up auto renew cronjob via parallel processing
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=29507 David Cook changed: What|Removed |Added CC||dc...@prosentient.com.au -- You are receiving this mail because: You are watching all bug changes. ___ Koha-bugs mailing list Koha-bugs@lists.koha-community.org https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 29507] Speed up auto renew cronjob via parallel processing
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=29507 --- Comment #6 from Nick Clemens --- (In reply to Jonathan Druart from comment #4) > I am wondering how this can work with the digest option. There's a loop to split based on borrowers. We push all the issues from the first borrower to chunk 0, when we complete one borrower we push the next on to the next chunk, then the next borrower to next chunk When we have pushed to as many chunks as we need (as many loops) we start again. So a single borrower, no matter how many checkouts, will always be processed by a single fork. It means the chunks won't be necessarily the same size, but they will work with digest -- You are receiving this mail because: You are watching all bug changes. ___ Koha-bugs mailing list Koha-bugs@lists.koha-community.org https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 29507] Speed up auto renew cronjob via parallel processing
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=29507 Nick Clemens changed: What|Removed |Added Attachment #132056|0 |1 is obsolete|| --- Comment #5 from Nick Clemens --- Created attachment 150692 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=150692&action=edit Bug 29507: Speed up auto renew cronjob via parallel processing The cron can take a very long time to run on systems with many issues. For example, a partner with ~250k auto_renew issues is taking about 9 hours to run. If we run that same number of holds in 5 parallel chunks ( splitting the number of issues as evenly as possible ), it could take under 2 hours. Test Plan: 1) Generate a number of issues marked for auto_renew 2) Run the automatic_renewals.pl, use the `time` utility to track how much time it took to run 3) Set parallel_loops to 10 in auto_renew_cronjob section of config in koha-conf 4) Repeat step 2, note the improvement in speed 5) Experiment with other values -- You are receiving this mail because: You are watching all bug changes. ___ Koha-bugs mailing list Koha-bugs@lists.koha-community.org https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 29507] Speed up auto renew cronjob via parallel processing
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=29507 Nick Clemens changed: What|Removed |Added Status|Patch doesn't apply |Needs Signoff -- You are receiving this mail because: You are watching all bug changes. ___ Koha-bugs mailing list Koha-bugs@lists.koha-community.org https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 29507] Speed up auto renew cronjob via parallel processing
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=29507 Jonathan Druart changed: What|Removed |Added Status|Needs Signoff |Patch doesn't apply CC||jonathan.druart+koha@gmail. ||com --- Comment #4 from Jonathan Druart --- Does not apply. I am wondering how this can work with the digest option. -- You are receiving this mail because: You are watching all bug changes. ___ Koha-bugs mailing list Koha-bugs@lists.koha-community.org https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 29507] Speed up auto renew cronjob via parallel processing
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=29507 Nick Clemens changed: What|Removed |Added CC||martin.renvoize@ptfs-europe ||.com -- You are receiving this mail because: You are watching all bug changes. ___ Koha-bugs mailing list Koha-bugs@lists.koha-community.org https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 29507] Speed up auto renew cronjob via parallel processing
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=29507 Katrin Fischer changed: What|Removed |Added Assignee|koha-b...@lists.koha-commun |n...@bywatersolutions.com |ity.org | -- You are receiving this mail because: You are watching all bug changes. You are the assignee for the bug. ___ Koha-bugs mailing list Koha-bugs@lists.koha-community.org https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 29507] Speed up auto renew cronjob via parallel processing
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=29507 Nick Clemens changed: What|Removed |Added Attachment #127744|0 |1 is obsolete|| --- Comment #3 from Nick Clemens --- Created attachment 132056 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=132056&action=edit Bug 29507: Speed up auto renew cronjob via parallel processing The cron can take a very long time to run on systems with many issues. For example, a partner with ~250k auto_renew issues is taking about 9 hours to run. If we run that same number of holds in 5 parallel chunks ( splitting the number of issues as evenly as possible, it could take under 2 hours. Test Plan: 0) Install the Perl library Parallel::ForkManager 1) Generate a number of issues marked for auto_renew 2) Run the automatic_renewals.pl, use the `time` utility to track how much time it took to run 3) Set parallel_loops to 10 in auto_renew_cronjob section of config in koha-conf 4) Repeat step 2, note the improvement in speed 5) Experiment with other values -- You are receiving this mail because: You are the assignee for the bug. You are watching all bug changes. ___ Koha-bugs mailing list Koha-bugs@lists.koha-community.org https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 29507] Speed up auto renew cronjob via parallel processing
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=29507 Nick Clemens changed: What|Removed |Added Attachment #127743|0 |1 is obsolete|| --- Comment #2 from Nick Clemens --- Created attachment 127744 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=127744&action=edit Bug 29057: Speed up auto renew cronjob via parallel processing The cron can take a very long time to run on systems with many issues. For example, a partner with ~250k auto_renew issues is taking about 9 hours to run. If we run that same number of holds in 5 parallel chunks ( splitting the number of issues as evenly as possible, it could take under 2 hours. Test Plan: 0) Install the Perl library Parallel::ForkManager 1) Generate a number of issues marked for auto_renew 2) Run the automatic_renewals.pl, use the `time` utility to track how much time it took to run 3) Set parallel_loops to 10 in auto_renew_cronjob section of config in koha-conf 4) Repeat step 2, note the improvement in speed 5) Experiment with other values -- You are receiving this mail because: You are the assignee for the bug. You are watching all bug changes. ___ Koha-bugs mailing list Koha-bugs@lists.koha-community.org https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 29507] Speed up auto renew cronjob via parallel processing
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=29507 --- Comment #1 from Nick Clemens --- Created attachment 127743 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=127743&action=edit Bug 28833: Speed up auto renew cronjob via parallel processing The cron can take a very long time to run on systems with many issues. For example, a partner with ~250k auto_renew issues is taking about 9 hours to run. If we run that same number of holds in 5 parallel chunks ( splitting the number of issues as evenly as possible, it could take under 2 hours. Test Plan: 0) Install the Perl library Parallel::ForkManager 1) Generate a number of issues marked for auto_renew 2) Run the automatic_renewals.pl, use the `time` utility to track how much time it took to run 3) Set parallel_loops to 10 in auto_renew_cronjob section of config in koha-conf 4) Repeat step 2, note the improvement in speed 5) Experiment with other values -- You are receiving this mail because: You are watching all bug changes. You are the assignee for the bug. ___ Koha-bugs mailing list Koha-bugs@lists.koha-community.org https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 29507] Speed up auto renew cronjob via parallel processing
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=29507 Nick Clemens changed: What|Removed |Added Status|NEW |Needs Signoff -- You are receiving this mail because: You are watching all bug changes. You are the assignee for the bug. ___ Koha-bugs mailing list Koha-bugs@lists.koha-community.org https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
