Hi, A short summary of the current discussion. I further recommend to read [0] which tries to explain as to when the `UpdateJob` is triggered.
Krabina Bernhard: Using SMW 1.8.0.5 / MW 1.22 with "Suddenly, without anybody knowing why, ~ 5000 have been done, but > 7000 are still sitting there ", "our problem is not only that we cannot get the hanging jobs to run, but that even new jobs being created (by changing a template or CSV import with data transfer) will not run.... " Yaron Koren Explains that "each job is only attempted a certain number of times before MediaWiki basically just gives up and ignores it" and by executing "UPDATE job SET job_attempts = 0" allows to reset the "job_attempts" field. James Montalvo / Daren Welsh Upgrading to MW 1.23 and SMW 2.0 from 1.22/1.8.0.5 seeing "stuck" jobs for "job_cmd of refreshLinks in job ... other four have job_cmd of SMW\UpdateJob", "Those "stuck" jobs just keep repeating over and over.", "runJobs evidently will eventually stop (or time-out) but only after repeating certain jobs hundreds of times." Phil Legault Explained that "same issue a while ago after I restored a database." but "ran another backup from my production database after I cleared the runjobs, and this worked out fine after". Neill Mitchell I can't think of any reason why the current thread and [1] are related based on the information given in [0]. [0] http://semantic-mediawiki.org/wiki/Help:UpdateJob [1] https://github.com/SemanticMediaWiki/SemanticMediaWiki/issues/518#issuecomment-54269008 Related discussions - https://github.com/SemanticMediaWiki/SemanticMediaWiki/issues/212 - https://github.com/SemanticMediaWiki/SemanticMediaWiki/pull/307 - https://github.com/SemanticMediaWiki/SemanticMediaWiki/issues/330 - https://github.com/SemanticMediaWiki/SemanticMediaWiki/issues/297 - http://wikimedia.7.x6.nabble.com/Re-Job-queue-affecting-semantic-queries-in-MW-1-22-tp5029433.html - http://wikimedia.7.x6.nabble.com/SMW-UpdateJob-error-tp5036300.html Cheers On 9/25/14, Neill Mitchell <ne...@nlkmitchell.com> wrote: > Hi Bernhard. > > Perhaps you are seeing the same issue as me. I have this logged on github: > https://github.com/SemanticMediaWiki/SemanticMediaWiki/issues/518#issuecomment-54269008 > > Please let me know if you think your issue may be the same as mine. > > Best regards > Neill. > > On 25/09/14 12:02, Krabina Bernhard wrote: >> our problem is not only that we cannot get the hanging jobs to run, but >> that even new jobs being created (by changing a template or CSV import >> with data transfer) will not run.... >> >> regards, >> Bernhard >> >> ----- Ursprüngliche Mail ----- >>> I had the same issue a while ago after I restored a database. >>> I figured it was because when I ran the backup there where jobs in the >>> queue >>> and after the restore they kept running over and over and etc. >>> >>> I ran another backup from my production database after I cleared the >>> runjobs, >>> and this worked out fine after. >>> >>> >>> >>> -----Original Message----- >>> From: James Montalvo [mailto:jamesmontal...@gmail.com] >>> Sent: Wednesday, September 24, 2014 5:55 PM >>> To: Daren Welsh >>> Cc: Semantic MediaWiki developers; Yaron Koren; Semantic MediaWiki users >>> Subject: Re: [Semediawiki-user] [SMW-devel] MediaWiki Job queue problem >>> >>> Daren and I work together, so we have the same issue. One thing to add: >>> Run >>> jobs never finishes. Those "stuck" jobs just keep repeating over and >>> over. >>> >>> On Wed, Sep 24, 2014 at 4:51 PM, Daren Welsh <darenwe...@gmail.com> >>> wrote: >>> >>>> I have executed runJobs several times and the job_attempts remains at >>>> 1 for those five jobs. We were thinking of doing a database backup >>>> today, then delete those five jobs from the table, then run the SMW >>>> "repair and upgrade" via the admin special page. >>>> >>>> Even if this clears the job queue, we'd like to understand what caused >>>> this in the first place. I realize that's a very open-ended question >>>> :) >>>> >>>> Daren >>>> >>>> >>>> On Wed, Sep 24, 2014 at 4:30 PM, James HK >>>> <jamesin.hongkon...@gmail.com> >>>> wrote: >>>> >>>>> Hi, >>>>> >>>>>> We currently have five jobs that are "stuck". All of them have 1 >>>>>> for job_attempts. >>>>>> >>>>>> One has job_cmd of refreshLinks in job namespace 10 and it is for a >>>>>> template page. >>>>>> The other four have job_cmd of SMW\UpdateJob in job namespace 0 and >>>>>> are >>>>> for >>>>>> "standard" pages. These pages do not seem to be related based on >>>>> category >>>>>> or template. >>>>> Just to make sure that I interpret the meaning of "stuck" correctly, >>>>> after finishing `runJobs` those four jobs (five with the >>>>> `refreshLinks` jobs) are still visible in the job table with an >>>>> "job_attempts" of 1. When running `runJobs` again the same four >>>>> `SMW\UpdateJob` (same as in the same title and same Id) jobs are >>>>> executed and increment the "job_attempts" to 2? >>>>> >>>>> If you empty the job table and execute `runJobs` does the same five >>>>> jobs appear again after the run with "job_attempts" = 1? >>>>> >>>>> Cheers >>>>> >>>>> On 9/25/14, Daren Welsh <darenwe...@gmail.com> wrote: >>>>>> We currently have five jobs that are "stuck". All of them have 1 >>>>>> for job_attempts. >>>>>> >>>>>> One has job_cmd of refreshLinks in job namespace 10 and it is for a >>>>>> template page. >>>>>> The other four have job_cmd of SMW\UpdateJob in job namespace 0 and >>>>>> are >>>>> for >>>>>> "standard" pages. These pages do not seem to be related based on >>>>> category >>>>>> or template. >>>>>> >>>>>> On Wed, Sep 24, 2014 at 3:37 PM, James HK >>>>>> <jamesin.hongkon...@gmail.com >>>>>> >>>>>> wrote: >>>>>> >>>>>>> Hi, >>>>>>> >>>>>>>> runJobs.php will literally run forever. After the non-offending >>>>>>>> jobs are cleared it's easy to see which are the offenders. Thus >>>>>>>> far I think >>>>> all >>>>>>>> offenders have been of type SMW::UpdateJob. >>>>>>> I don't think the problem is with the `SMW\UpdateJob` because it >>>>>>> does a simple "shallow update" of the store while the management >>>>>>> of job status (including how many attempts, id's etc.) are done by >>>>>>> the MW JobQueue (which has first change in 1.22 and then again in >>>>>>> 1.23). >>>>>>> >>>>>>> It does beg the question whether all `SMW\UpdateJob`'s are "stuck" >>>>>>> or only certain jobs belonging to a group of pages or single page? >>>>>>> >>>>>>>> runJobs.php, but for some reason they keep attempting to run >>>>>>>> over and >>>>>>> over. >>>>>>> >>>>>>> How do you know that the same job is run over and over again >>>>>>> because based and above discussion ("job_attempts") a job with too >>>>>>> many attempts is retired after some time. >>>>>>> >>>>>>> If the same job is run over and over again, what is displayed for >>>>>>> the "job_attempts" counter? >>>>>>> >>>>>>> [0] went into SMW 2.0 to counteract any possible job duplicates >>>>>>> for the same `root title`. >>>>>>> >>>>>>> [0] >>>>>>> https://github.com/SemanticMediaWiki/SemanticMediaWiki/pull/307 >>>>>>> >>>>>>> Cheers >>>>>>> >>>>>>> On 9/25/14, James Montalvo <jamesmontal...@gmail.com> wrote: >>>>>>>> I'm not sure if this is related, but on my wiki I'm occasionally >>>>>>>> getting "stuck" jobs. I've only noticed this since upgrading to >>>>>>>> MW 1.23 and >>>>> SMW >>>>>>> 2.0 >>>>>>>> from 1.22/1.8.0.5. >>>>>>>> >>>>>>>> What I mean by "stuck" is that the jobs don't get executed when >>>>>>>> I do runJobs.php, but for some reason they keep attempting to >>>>>>>> run over and >>>>>>> over. >>>>>>>> runJobs.php will literally run forever. After the non-offending >>>>>>>> jobs are cleared it's easy to see which are the offenders. Thus >>>>>>>> far I think >>>>> all >>>>>>>> offenders have been of type SMW::UpdateJob. >>>>>>>> >>>>>>>> Is there some way to debug runJobs.php so I can provide better info? >>>>>>>> >>>>>>>> --James >>>>>>>> On Sep 24, 2014 10:55 AM, "Yaron Koren" <ya...@wikiworks.com> wrote: >>>>>>>> >>>>>>>>> I certainly hope so too - or that there's some other standard >>>>>>>>> way to get previously-attempted jobs to be run again. I only >>>>>>>>> know that I tried that SQL trick once, and it worked. Perhaps >>>>>>>>> this is another reason why >>>>> the >>>>>>>>> question should have instead been sent to the mediawiki-l >>>>>>>>> mailing list. >>>>>>>>> :) >>>>>>>>> >>>>>>>>> On Wed, Sep 24, 2014 at 11:35 AM, James HK < >>>>>>> jamesin.hongkon...@gmail.com> >>>>>>>>> wrote: >>>>>>>>> >>>>>>>>>> Hi, >>>>>>>>>> >>>>>>>>>>> column is greater than 0 for all the rows in the table; I >>>>>>>>>>> think >>>>> if >>>>>>>>>>> you >>>>>>>>>> just >>>>>>>>>>> go into the database and call something like "UPDATE job >>>>>>>>>>> SET >>>>>>>>>> job_attempts = >>>>>>>>>>> 0", they will get run again. >>>>>>>>>> In case this solves the issue, I sincerely hope there is a >>>>> different >>>>>>>>>> way (a more standard way) to reset the "job_attempts" field >>>>>>>>>> other than by using a SQL statement to manipulate the job >>>>>>>>>> table. >>>>>>>>>> >>>>>>>>>> Cheers >>>>>>>>>> >>>>>>>>>> On 9/25/14, Yaron Koren <ya...@wikiworks.com> wrote: >>>>>>>>>>> Hi, >>>>>>>>>>> >>>>>>>>>>> I believe the issue is the "job_attempts" field in the "job" >>>>>>>>>>> table. >>>>>>> I >>>>>>>>>>> believe each job is only attempted a certain number of >>>>>>>>>>> times before MediaWiki basically just gives up and ignores >>>>>>>>>>> it. My guess is >>>>> that >>>>>>>>>>> that >>>>>>>>>>> column is greater than 0 for all the rows in the table; I >>>>>>>>>>> think >>>>> if >>>>>>>>>>> you >>>>>>>>>> just >>>>>>>>>>> go into the database and call something like "UPDATE job >>>>>>>>>>> SET >>>>>>>>>> job_attempts = >>>>>>>>>>> 0", they will get run again. >>>>>>>>>>> >>>>>>>>>>> -Yaron >>>>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> -- >>>>>>>>> WikiWorks · MediaWiki Consulting · http://wikiworks.com >>>>>>>>> >>>>>>>>> >>>>> --------------------------------------------------------------------- >>>>> --------- >>>>>>>>> Meet PCI DSS 3.0 Compliance Requirements with EventLog Analyzer >>>>>>>>> Achieve PCI DSS 3.0 Compliant Status with Out-of-the-box PCI >>>>>>>>> DSS Reports Are you Audit-Ready for PCI DSS 3.0 Compliance? >>>>>>>>> Download White paper Comply to PCI DSS 3.0 Requirement 10 and >>>>>>>>> 11.5 with EventLog Analyzer >>>>>>>>> >>>>>>>>> >>>>> http://pubads.g.doubleclick.net/gampad/clk?id=154622311&iu=/4140/ostg >>>>> .clktrk >>>>>>>>> _______________________________________________ >>>>>>>>> Semediawiki-user mailing list >>>>>>>>> semediawiki-u...@lists.sourceforge.net >>>>>>>>> https://lists.sourceforge.net/lists/listinfo/semediawiki-user >>>>>>>>> >>>>>>> >>>>>>> >>>>> --------------------------------------------------------------------- >>>>> --------- >>>>>>> Meet PCI DSS 3.0 Compliance Requirements with EventLog Analyzer >>>>>>> Achieve PCI DSS 3.0 Compliant Status with Out-of-the-box PCI DSS >>>>> Reports >>>>>>> Are you Audit-Ready for PCI DSS 3.0 Compliance? Download White >>>>>>> paper Comply to PCI DSS 3.0 Requirement 10 and 11.5 with EventLog >>>>>>> Analyzer >>>>>>> >>>>>>> >>>>> http://pubads.g.doubleclick.net/gampad/clk?id=154622311&iu=/4140/ostg >>>>> .clktrk >>>>>>> _______________________________________________ >>>>>>> Semediawiki-devel mailing list >>>>>>> Semediawiki-devel@lists.sourceforge.net >>>>>>> https://lists.sourceforge.net/lists/listinfo/semediawiki-devel >>>>>>> >>>>>> >>>>>> >>>>>> -- >>>>>> __________________ >>>>>> http://mixcloud.com/darenwelsh >>>>>> http://www.beatportfolio.com >>>>>> >>>> >>>> >>>> -- >>>> __________________ >>>> http://mixcloud.com/darenwelsh >>>> http://www.beatportfolio.com >>>> >>> ------------------------------------------------------------------------------ >>> Meet PCI DSS 3.0 Compliance Requirements with EventLog Analyzer Achieve >>> PCI >>> DSS 3.0 Compliant Status with Out-of-the-box PCI DSS Reports Are you >>> Audit-Ready for PCI DSS 3.0 Compliance? Download White paper Comply to >>> PCI >>> DSS 3.0 Requirement 10 and 11.5 with EventLog Analyzer >>> http://pubads.g.doubleclick.net/gampad/clk?id=154622311&iu=/4140/ostg.clktrk >>> _______________________________________________ >>> Semediawiki-user mailing list >>> semediawiki-u...@lists.sourceforge.net >>> https://lists.sourceforge.net/lists/listinfo/semediawiki-user >>> ------------------------------------------------------------------------------ >>> Meet PCI DSS 3.0 Compliance Requirements with EventLog Analyzer >>> Achieve PCI DSS 3.0 Compliant Status with Out-of-the-box PCI DSS Reports >>> Are you Audit-Ready for PCI DSS 3.0 Compliance? Download White paper >>> Comply to PCI DSS 3.0 Requirement 10 and 11.5 with EventLog Analyzer >>> http://pubads.g.doubleclick.net/gampad/clk?id=154622311&iu=/4140/ostg.clktrk >>> _______________________________________________ >>> Semediawiki-user mailing list >>> semediawiki-u...@lists.sourceforge.net >>> https://lists.sourceforge.net/lists/listinfo/semediawiki-user >>> >> ------------------------------------------------------------------------------ >> Meet PCI DSS 3.0 Compliance Requirements with EventLog Analyzer >> Achieve PCI DSS 3.0 Compliant Status with Out-of-the-box PCI DSS Reports >> Are you Audit-Ready for PCI DSS 3.0 Compliance? Download White paper >> Comply to PCI DSS 3.0 Requirement 10 and 11.5 with EventLog Analyzer >> http://pubads.g.doubleclick.net/gampad/clk?id=154622311&iu=/4140/ostg.clktrk >> _______________________________________________ >> Semediawiki-devel mailing list >> Semediawiki-devel@lists.sourceforge.net >> https://lists.sourceforge.net/lists/listinfo/semediawiki-devel > > > ------------------------------------------------------------------------------ > Meet PCI DSS 3.0 Compliance Requirements with EventLog Analyzer > Achieve PCI DSS 3.0 Compliant Status with Out-of-the-box PCI DSS Reports > Are you Audit-Ready for PCI DSS 3.0 Compliance? Download White paper > Comply to PCI DSS 3.0 Requirement 10 and 11.5 with EventLog Analyzer > http://pubads.g.doubleclick.net/gampad/clk?id=154622311&iu=/4140/ostg.clktrk > _______________________________________________ > Semediawiki-devel mailing list > Semediawiki-devel@lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/semediawiki-devel > ------------------------------------------------------------------------------ Meet PCI DSS 3.0 Compliance Requirements with EventLog Analyzer Achieve PCI DSS 3.0 Compliant Status with Out-of-the-box PCI DSS Reports Are you Audit-Ready for PCI DSS 3.0 Compliance? Download White paper Comply to PCI DSS 3.0 Requirement 10 and 11.5 with EventLog Analyzer http://pubads.g.doubleclick.net/gampad/clk?id=154622311&iu=/4140/ostg.clktrk _______________________________________________ Semediawiki-devel mailing list Semediawiki-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/semediawiki-devel
