Re: [pgadmin-hackers] RFC: pgAgent Scheduler Design
> -Original Message- > From: Andreas Pflug [mailto:[EMAIL PROTECTED] > Sent: 05 March 2005 23:16 > To: Dave Page > Cc: [email protected] > Subject: Re: [pgadmin-hackers] RFC: pgAgent Scheduler Design > > Sorry, this won't work. > > It is mandatory that a "next run" schedule is calculated > after a job has > run, and FireSchedule will run when nextRun is >= current_timestamp. Yes, and that is how it will run - the code snippet I gave was simply to illustrate how the schedule storage will work, not how the scheduler will actually run. I fully intend to retain the current method of pre-calculating the next run time. > Imagine two jobs scheduled for the very same minute, and only one > pgAgent running. It will run the first job, which will run > for lets say > 2 minutes. After that, the fire time for the second job is > not due any > more, so it will run somewhat later, if ever. Yes, I'm aware of that issue - and even the current design will have problems in that jobs may stil run late. I think the correct way to address this is to grab all due jobs, and spawn a separate thread to handle each. This should allow jobs to actually start when they are supposed to. > That's what pga_next_schedule is good for. You'll have quite > a hard time > to calulate it from your way of storing schedules, I'm afraid... It's > somewhat the difference between cron and anacron. Yes, I realise it's not the easiest thing to do, but I do not believe it's a major problem that cannot be overcome with a little thought. I think the trick is to treat each element of the schedule seperately, and find the next month, then the next day, and lastly the hour and minute in turn. Regards, Dave. ---(end of broadcast)--- TIP 7: don't forget to increase your free space map settings
Re: [pgadmin-hackers] RFC: pgAgent Scheduler Design
Dave Page wrote: Yes, I'm aware of that issue - and even the current design will have problems in that jobs may stil run late. Better late than never. I think the correct way to address this is to grab all due jobs, and spawn a separate thread to handle each. This should allow jobs to actually start when they are supposed to. Well, yes, hm... Threading is nice, but not a guarantee to be exactly on-time for job starts. The design is basically able to be run by multiple process instances of pgAgent (on different machines), this would get quite impossible without further control. A combination seems the best: - a job that is due to run will be running as soon as possible - any instance of pgAgent might be configured to run a job threaded - multiple instance may share the pool of due tasks - all together, instances and threads try to execute the schedule asap. IIRC, that was my original intention, those days. Regards, Andreas ---(end of broadcast)--- TIP 3: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
[pgadmin-hackers] Missing files in current snapshots (?)
Hi Andreas, Dave, it seems the current snapshots lack two files required for the build: /cvsroot/pgadmin3/src/slony/include/dlgRepProperty.h /cvsroot/pgadmin3/src/slony/include/dlgRepTable.h After I checked them out, I was able to get it running. Or maybe I did something wrong ? Cheers, Raphaël ---(end of broadcast)--- TIP 2: you can get off all lists at once with the unregister command (send "unregister YourEmailAddressHere" to [EMAIL PROTECTED])
Re: [pgadmin-hackers] RFC: pgAgent Scheduler Design
-Original Message- From: Andreas Pflug [mailto:[EMAIL PROTECTED] Sent: Sun 3/6/2005 1:41 PM To: Dave Page Cc: [email protected] Subject: Re: [pgadmin-hackers] RFC: pgAgent Scheduler Design > >Yes, I'm aware of that issue - and even the current design will have > >problems in that jobs may stil run late. > > Better late than never. Yup - but, an exception... say I have a daily job that sends me a simple report via email. If the system is down for a few days, then I don't want it to run all the old instances of the job on restart (cron won't, the MS Task Scheduler won't - in fact, I can't think of any I've used that do would). Anyway, I was thinking that when the agent first starts, it should do something like 'update pga_schedule set nextrun = nextrun where jscactive = true and jscrunning = false' to nudge the update trigger to recalculate the next run dates from that point. What I'm not so sure about is how to log the failed jobs in that instance. This should be multi-agent safe. > Threading is nice, but not a guarantee to be exactly on-time for job starts. Nope, but except on the most overloaded of systems each job should start within a minute of it's schedule. With the current design, one 6 hour job could completely screw things up for other jobs. > -a job that is due to run will be running as soon as possible Yup. > - any instance of pgAgent might be configured to run a job threaded > - multiple instance may share the pool of due tasks Eh? No, I'm advocating 1 thread per job. The main thread queries the db, find 5 jobs due and spawns 5 threads to run them. 1 minute later, regardless of the state of the 5 threads, the main thread checks for new tasks to run and spawns more threads if required. As jobs are finished, their threads simply die. The use of multiple agents by the vast majority of people seems unlikely to me - especially given the lack of control over what runs on what agent. In particular, the majority of jobs are likely to be SQL jobs, the distribution of which is pretty much irrelevant anyway as all the hard work is done by the server. I'm not convinced that many people will want to run resource hungry batch jobs that may run on random agent machines. Can we get some third party opinions on what usage models will be useful please? Regards, Dave ---(end of broadcast)--- TIP 7: don't forget to increase your free space map settings
Re: [pgadmin-hackers] RFC: pgAgent Scheduler Design
Dave Page wrote: -Original Message- From: Andreas Pflug [mailto:[EMAIL PROTECTED] Sent: Sun 3/6/2005 1:41 PM To: Dave Page Cc: [email protected] Subject: Re: [pgadmin-hackers] RFC: pgAgent Scheduler Design Yes, I'm aware of that issue - and even the current design will have problems in that jobs may stil run late. Better late than never. Yup - but, an exception... say I have a daily job that sends me a simple report via email. If the system is down for a few days, then I don't want it to run all the old instances of the job on restart (cron won't, the MS Task Scheduler won't - in fact, I can't think of any I've used that do would). I don't completely agree. If a job was stuck unexecuted in the queue for a while, it should run asap, and the next schedule should be calculated in the future, i.e. a daily job not executed for 5 days shouldn't be executed 5x, but only once after pgAgent is up again. Anyway, I was thinking that when the agent first starts, it should do something like 'update pga_schedule set nextrun = nextrun where jscactive = true and jscrunning = false' to nudge the update trigger to recalculate the next run dates from that point. No, it should run *once* to get in sync again, if it has become due in the meantime. What I'm not so sure about is how to log the failed jobs in that instance. This should be multi-agent safe. Actually they don't log. Think of a weekdaily job, pgAgent down since Sunday, now up on Friday morningagain. The job was due on Monday evening, and will run on Friday immediately, new schedule Friday night . After an agent failure, you'll have to check anyway in which state your jobs are. I don't think something like "monday job ran on friday, tue/wed/thur was skipped" would help here. The logging "Job due on Monday at 2200 was run on Friday 830" should be enough. Threading is nice, but not a guarantee to be exactly on-time for job starts. Nope, but except on the most overloaded of systems each job should start within a minute of it's schedule. With the current design, one 6 hour job could completely screw things up for other jobs. -a job that is due to run will be running as soon as possible Yup. - any instance of pgAgent might be configured to run a job threaded - multiple instance may share the pool of due tasks Eh? No, I'm advocating 1 thread per job. The main thread queries the db, find 5 jobs due and spawns 5 threads to run them. 1 minute later, regardless of the state of the 5 threads, the main thread checks for new tasks to run and spawns more threads if required. As jobs are finished, their threads simply die. This doesn't enable multi agents. There should be a limit on threads per agent to give other instances a chance. The use of multiple agents by the vast majority of people seems unlikely to me - especially given the lack of control over what runs on what agent. In particular, the majority of jobs are likely to be SQL jobs, the distribution of which is pretty much irrelevant anyway as all the hard work is done by the server. I'm not convinced that many people will want to run resource hungry batch jobs that may run on random agent machines. Binary jobs (shell jobs) need a system qualifier. Can we get some third party opinions on what usage models will be useful please? Yesplease. Regards, Andreas ---(end of broadcast)--- TIP 7: don't forget to increase your free space map settings
