Re: [v9.3] Extra Daemons (Re: [HACKERS] elegant and effective way for running jobs inside a database)

2012-11-24 Thread Kohei KaiGai
2012/11/21 Alvaro Herrera : > Alvaro Herrera escribió: >> FWIW I have pushed this to github; see >> https://github.com/alvherre/postgres/compare/bgworker >> >> It's also attached. >> >> The UnBlockSig stuff is the main stumbling block as I see it because it >> precludes compilation on Windows. May

Re: [v9.3] Extra Daemons (Re: [HACKERS] elegant and effective way for running jobs inside a database)

2012-11-23 Thread Tom Lane
Alvaro Herrera writes: > If the bgworker developer gets really tense about this stuff (or > anything at all, really), they can create a completely new sigmask and > do sigaddset() etc. Since this is all C code, we cannot keep them from > doing anything, really; I think what we need to provide her

Re: [v9.3] Extra Daemons (Re: [HACKERS] elegant and effective way for running jobs inside a database)

2012-11-22 Thread Alvaro Herrera
Heikki Linnakangas escribió: > On 22.11.2012 19:18, Alvaro Herrera wrote: > >Heikki Linnakangas escribió: > >>On 21.11.2012 23:29, Alvaro Herrera wrote: > >>>Alvaro Herrera escribió: > The UnBlockSig stuff is the main stumbling block as I see it because it > precludes compilation on Windows

Re: [v9.3] Extra Daemons (Re: [HACKERS] elegant and effective way for running jobs inside a database)

2012-11-22 Thread Heikki Linnakangas
On 22.11.2012 19:18, Alvaro Herrera wrote: Heikki Linnakangas escribió: On 21.11.2012 23:29, Alvaro Herrera wrote: Alvaro Herrera escribió: The UnBlockSig stuff is the main stumbling block as I see it because it precludes compilation on Windows. Maybe we should fix that by providing another f

Re: [v9.3] Extra Daemons (Re: [HACKERS] elegant and effective way for running jobs inside a database)

2012-11-22 Thread Alvaro Herrera
Heikki Linnakangas escribió: > On 21.11.2012 23:29, Alvaro Herrera wrote: > >Alvaro Herrera escribió: > >>FWIW I have pushed this to github; see > >>https://github.com/alvherre/postgres/compare/bgworker > >> > >>It's also attached. > >> > >>The UnBlockSig stuff is the main stumbling block as I see

Re: [v9.3] Extra Daemons (Re: [HACKERS] elegant and effective way for running jobs inside a database)

2012-11-22 Thread Heikki Linnakangas
On 21.11.2012 23:29, Alvaro Herrera wrote: Alvaro Herrera escribió: FWIW I have pushed this to github; see https://github.com/alvherre/postgres/compare/bgworker It's also attached. The UnBlockSig stuff is the main stumbling block as I see it because it precludes compilation on Windows. Maybe

Re: [v9.3] Extra Daemons (Re: [HACKERS] elegant and effective way for running jobs inside a database)

2012-11-19 Thread Tom Lane
Alvaro Herrera writes: > Kohei KaiGai escribió: >> StartOneBackgroundWorker always scan the BackgroundWorkerList from >> the head. Isn't it available to save the current position at static variable? >> If someone tries to manage thousand of bgworkers, it makes a busy loop. :( > Seems messy; we wo

Re: [v9.3] Extra Daemons (Re: [HACKERS] elegant and effective way for running jobs inside a database)

2012-11-19 Thread Alvaro Herrera
Kohei KaiGai escribió: > 2012/10/22 Alvaro Herrera : > > Here's an updated version of this patch, which also works in > > an EXEC_BACKEND environment. (I haven't tested this at all on Windows, > > but I don't see anything that would create a portability problem there.) > > > I also tried to check

Re: [v9.3] Extra Daemons (Re: [HACKERS] elegant and effective way for running jobs inside a database)

2012-11-16 Thread Alvaro Herrera
Kohei KaiGai escribió: > 2012/10/22 Alvaro Herrera : > > Here's an updated version of this patch, which also works in > > an EXEC_BACKEND environment. (I haven't tested this at all on Windows, > > but I don't see anything that would create a portability problem there.) > > > I also tried to check

Re: [v9.3] Extra Daemons (Re: [HACKERS] elegant and effective way for running jobs inside a database)

2012-11-16 Thread Kohei KaiGai
2012/10/22 Alvaro Herrera : > Here's an updated version of this patch, which also works in > an EXEC_BACKEND environment. (I haven't tested this at all on Windows, > but I don't see anything that would create a portability problem there.) > I also tried to check the latest patch "briefly". Let me

Re: [v9.3] Extra Daemons (Re: [HACKERS] elegant and effective way for running jobs inside a database)

2012-11-16 Thread Simon Riggs
On 15 November 2012 10:10, Alvaro Herrera wrote: > I am unsure about the amount of pre-cooked stuff we need to provide. > For instance, do we want some easy way to let the user code run > transactions? That sounds like a basic requirement. There will be a few non-transactional bgworkers but most

Re: [v9.3] Extra Daemons (Re: [HACKERS] elegant and effective way for running jobs inside a database)

2012-11-15 Thread Heikki Linnakangas
On 15.11.2012 17:10, Alvaro Herrera wrote: Heikki Linnakangas escribió: On 23.10.2012 00:29, Alvaro Herrera wrote: Here's an updated version of this patch, which also works in an EXEC_BACKEND environment. (I haven't tested this at all on Windows, but I don't see anything that would create a po

Re: [v9.3] Extra Daemons (Re: [HACKERS] elegant and effective way for running jobs inside a database)

2012-11-15 Thread Alvaro Herrera
Heikki Linnakangas escribió: > On 23.10.2012 00:29, Alvaro Herrera wrote: > >Here's an updated version of this patch, which also works in > >an EXEC_BACKEND environment. (I haven't tested this at all on Windows, > >but I don't see anything that would create a portability problem there.) > > Looks

Re: [v9.3] Extra Daemons (Re: [HACKERS] elegant and effective way for running jobs inside a database)

2012-11-15 Thread Heikki Linnakangas
On 23.10.2012 00:29, Alvaro Herrera wrote: Here's an updated version of this patch, which also works in an EXEC_BACKEND environment. (I haven't tested this at all on Windows, but I don't see anything that would create a portability problem there.) Looks good at first glance. Fails on Windows,

Re: [v9.3] Extra Daemons (Re: [HACKERS] elegant and effective way for running jobs inside a database)

2012-09-27 Thread Alvaro Herrera
Excerpts from Kohei KaiGai's message of jue sep 27 01:06:41 -0300 2012: > Hi Alvaro, > > Let me volunteer for reviewing, of course, but now pgsql_fdw is in my queue... Sure, thanks -- keep in mind I entered this patch in the next commitfest, so please do invest more effort in the ones in the comm

Re: [v9.3] Extra Daemons (Re: [HACKERS] elegant and effective way for running jobs inside a database)

2012-09-26 Thread Kohei KaiGai
Hi Alvaro, Let me volunteer for reviewing, of course, but now pgsql_fdw is in my queue... If some other folks can also volunteer it soon, it is welcome. 2012/9/26 Alvaro Herrera : > Excerpts from Alvaro Herrera's message of mié sep 26 13:04:34 -0300 2012: >> Excerpts from Kohei KaiGai's message

Re: [v9.3] Extra Daemons (Re: [HACKERS] elegant and effective way for running jobs inside a database)

2012-09-23 Thread Amit Kapila
> On Monday, September 24, 2012 12:24 AM Alvaro Herrera wrote: > Excerpts from Amit kapila's message of sáb sep 22 01:14:40 -0300 2012: > > On Friday, September 21, 2012 6:50 PM Alvaro Herrera wrote: > > Excerpts from Amit Kapila's message of vie sep 21 02:26:49 -0300 > 2012: > > > On Thursday, Sep

Re: [v9.3] Extra Daemons (Re: [HACKERS] elegant and effective way for running jobs inside a database)

2012-09-23 Thread Alvaro Herrera
Excerpts from Amit kapila's message of sáb sep 22 01:14:40 -0300 2012: > On Friday, September 21, 2012 6:50 PM Alvaro Herrera wrote: > Excerpts from Amit Kapila's message of vie sep 21 02:26:49 -0300 2012: > > On Thursday, September 20, 2012 7:13 PM Alvaro Herrera wrote: > > > > > Well, there is a

Re: [v9.3] Extra Daemons (Re: [HACKERS] elegant and effective way for running jobs inside a database)

2012-09-21 Thread Amit kapila
On Friday, September 21, 2012 6:50 PM Alvaro Herrera wrote: Excerpts from Amit Kapila's message of vie sep 21 02:26:49 -0300 2012: > On Thursday, September 20, 2012 7:13 PM Alvaro Herrera wrote: > > > Well, there is a difficulty here which is that the number of processes > >> connected to database

Re: [v9.3] Extra Daemons (Re: [HACKERS] elegant and effective way for running jobs inside a database)

2012-09-21 Thread Alvaro Herrera
Excerpts from Amit Kapila's message of vie sep 21 02:26:49 -0300 2012: > On Thursday, September 20, 2012 7:13 PM Alvaro Herrera wrote: > > Well, there is a difficulty here which is that the number of processes > > connected to databases must be configured during postmaster start > > (because it de

Re: [v9.3] Extra Daemons (Re: [HACKERS] elegant and effective way for running jobs inside a database)

2012-09-20 Thread Amit Kapila
On Thursday, September 20, 2012 7:35 PM Kohei KaiGai wrote: 2012/9/20 Amit Kapila : > On Thursday, September 20, 2012 1:44 AM Simon Riggs wrote: > On 12 September 2012 04:30, Amit Kapila wrote: >> On Tuesday, September 11, 2012 9:09 PM Alvaro Herrera wrote: >> Excerpts from Boszormenyi Zoltan's me

Re: [v9.3] Extra Daemons (Re: [HACKERS] elegant and effective way for running jobs inside a database)

2012-09-20 Thread Amit Kapila
On Thursday, September 20, 2012 7:13 PM Alvaro Herrera wrote: Excerpts from Amit Kapila's message of jue sep 20 02:10:23 -0300 2012: >> Why can't worker tasks be also permanent, which can be controlled through >> configuration. What I mean to say is that if user has need for parallel >> opera

Re: [v9.3] Extra Daemons (Re: [HACKERS] elegant and effective way for running jobs inside a database)

2012-09-20 Thread Kohei KaiGai
2012/9/20 Amit Kapila : > On Thursday, September 20, 2012 1:44 AM Simon Riggs wrote: > On 12 September 2012 04:30, Amit Kapila wrote: >> On Tuesday, September 11, 2012 9:09 PM Alvaro Herrera wrote: >> Excerpts from Boszormenyi Zoltan's message of vie jun 29 09:11:23 -0400 > 2012: >> > We have

Re: [v9.3] Extra Daemons (Re: [HACKERS] elegant and effective way for running jobs inside a database)

2012-09-20 Thread Alvaro Herrera
Excerpts from Amit Kapila's message of jue sep 20 02:10:23 -0300 2012: > Why can't worker tasks be also permanent, which can be controlled through > configuration. What I mean to say is that if user has need for parallel > operations > he can configure max_worker_tasks and those many worker

Re: [v9.3] Extra Daemons (Re: [HACKERS] elegant and effective way for running jobs inside a database)

2012-09-19 Thread Amit Kapila
On Thursday, September 20, 2012 1:44 AM Simon Riggs wrote: On 12 September 2012 04:30, Amit Kapila wrote: > On Tuesday, September 11, 2012 9:09 PM Alvaro Herrera wrote: > Excerpts from Boszormenyi Zoltan's message of vie jun 29 09:11:23 -0400 2012: > We have some use cases for this patch, whe

Re: [v9.3] Extra Daemons (Re: [HACKERS] elegant and effective way for running jobs inside a database)

2012-09-19 Thread Simon Riggs
On 12 September 2012 04:30, Amit Kapila wrote: > On Tuesday, September 11, 2012 9:09 PM Alvaro Herrera wrote: > Excerpts from Boszormenyi Zoltan's message of vie jun 29 09:11:23 -0400 2012: > >>> We have some use cases for this patch, when can you post >>> a new version? I would test and review it

Re: [v9.3] Extra Daemons (Re: [HACKERS] elegant and effective way for running jobs inside a database)

2012-09-11 Thread Alvaro Herrera
Excerpts from Amit Kapila's message of mié sep 12 00:30:40 -0300 2012: > On Tuesday, September 11, 2012 9:09 PM Alvaro Herrera wrote: > Excerpts from Boszormenyi Zoltan's message of vie jun 29 09:11:23 -0400 2012: > > >> We have some use cases for this patch, when can you post > >> a new version?

Re: [v9.3] Extra Daemons (Re: [HACKERS] elegant and effective way for running jobs inside a database)

2012-09-11 Thread Amit Kapila
On Tuesday, September 11, 2012 9:09 PM Alvaro Herrera wrote: Excerpts from Boszormenyi Zoltan's message of vie jun 29 09:11:23 -0400 2012: >> We have some use cases for this patch, when can you post >> a new version? I would test and review it. > What use cases do you have in mind? Wouldn't it

Re: [v9.3] Extra Daemons (Re: [HACKERS] elegant and effective way for running jobs inside a database)

2012-09-11 Thread Boszormenyi Zoltan
2012-09-11 17:58 keltezéssel, Alvaro Herrera írta: Excerpts from Kohei KaiGai's message of mar sep 11 12:46:34 -0300 2012: 2012/9/11 Alvaro Herrera : Excerpts from Boszormenyi Zoltan's message of vie jun 29 09:11:23 -0400 2012: We have some use cases for this patch, when can you post a new ve

Re: [v9.3] Extra Daemons (Re: [HACKERS] elegant and effective way for running jobs inside a database)

2012-09-11 Thread Alvaro Herrera
Excerpts from Kohei KaiGai's message of mar sep 11 13:25:18 -0300 2012: > 2012/9/11 Alvaro Herrera : > >> > - an SQL-driven scheduler, similar to pgAgent, it's generic enough, > >> > we might port it to this scheme and publish it > > > > Hm, this would benefit from a direct backend connection to

Re: [v9.3] Extra Daemons (Re: [HACKERS] elegant and effective way for running jobs inside a database)

2012-09-11 Thread Kohei KaiGai
2012/9/11 Alvaro Herrera : > Excerpts from Kohei KaiGai's message of mar sep 11 12:46:34 -0300 2012: >> 2012/9/11 Alvaro Herrera : >> > Excerpts from Boszormenyi Zoltan's message of vie jun 29 09:11:23 -0400 >> > 2012: >> > >> >> We have some use cases for this patch, when can you post >> >> a new

Re: [v9.3] Extra Daemons (Re: [HACKERS] elegant and effective way for running jobs inside a database)

2012-09-11 Thread Alvaro Herrera
Excerpts from Kohei KaiGai's message of mar sep 11 12:46:34 -0300 2012: > 2012/9/11 Alvaro Herrera : > > Excerpts from Boszormenyi Zoltan's message of vie jun 29 09:11:23 -0400 > > 2012: > > > >> We have some use cases for this patch, when can you post > >> a new version? I would test and review i

Re: [v9.3] Extra Daemons (Re: [HACKERS] elegant and effective way for running jobs inside a database)

2012-09-11 Thread Kohei KaiGai
2012/9/11 Alvaro Herrera : > Excerpts from Boszormenyi Zoltan's message of vie jun 29 09:11:23 -0400 2012: > >> We have some use cases for this patch, when can you post >> a new version? I would test and review it. > > What use cases do you have in mind? > I'm motivated with this feature to impleme

Re: [v9.3] Extra Daemons (Re: [HACKERS] elegant and effective way for running jobs inside a database)

2012-09-11 Thread Alvaro Herrera
Excerpts from Boszormenyi Zoltan's message of vie jun 29 09:11:23 -0400 2012: > We have some use cases for this patch, when can you post > a new version? I would test and review it. What use cases do you have in mind? -- Álvaro Herrerahttp://www.2ndQuadrant.com/ PostgreSQL Devel

Re: [v9.3] Extra Daemons (Re: [HACKERS] elegant and effective way for running jobs inside a database)

2012-08-31 Thread Kohei KaiGai
2012/6/21 Simon Riggs : > On 21 June 2012 19:13, Jaime Casanova wrote: >> On Sun, Jun 10, 2012 at 4:15 AM, Kohei KaiGai wrote: >>> 2012/6/8 Simon Riggs : >>> I have a prototype that has some of these characteristics, so I see our work as complementary. At present, I don't thin

Re: [v9.3] Extra Daemons (Re: [HACKERS] elegant and effective way for running jobs inside a database)

2012-06-29 Thread Jaime Casanova
On Fri, Jun 29, 2012 at 9:44 AM, Kohei KaiGai wrote: > > The auth_counter is just an proof-of-concept patch, so, it is helpful if you > could provide another use case that can make sense. > what about pgbouncer? -- Jaime Casanova         www.2ndQuadrant.com Professional PostgreSQL: Soporte 24x7

Re: [v9.3] Extra Daemons (Re: [HACKERS] elegant and effective way for running jobs inside a database)

2012-06-29 Thread Boszormenyi Zoltan
2012-06-29 16:44 keltezéssel, Kohei KaiGai írta: 2012/6/29 Boszormenyi Zoltan : 2012-04-25 11:40 keltezéssel, Kohei KaiGai írta: 2012/3/10 Simon Riggs : On Fri, Mar 9, 2012 at 6:51 PM, Andrew Dunstan wrote: On 03/09/2012 01:40 PM, Robert Haas wrote: On Fri, Mar 9, 2012 at 12:02 PM, David

Re: [v9.3] Extra Daemons (Re: [HACKERS] elegant and effective way for running jobs inside a database)

2012-06-29 Thread Kohei KaiGai
2012/6/29 Boszormenyi Zoltan : > 2012-04-25 11:40 keltezéssel, Kohei KaiGai írta: > >> 2012/3/10 Simon Riggs : >>> >>> On Fri, Mar 9, 2012 at 6:51 PM, Andrew Dunstan >>> wrote: On 03/09/2012 01:40 PM, Robert Haas wrote: > > On Fri, Mar 9, 2012 at 12:02 PM, David E. > Whe

Re: [v9.3] Extra Daemons (Re: [HACKERS] elegant and effective way for running jobs inside a database)

2012-06-29 Thread Boszormenyi Zoltan
2012-04-25 11:40 keltezéssel, Kohei KaiGai írta: 2012/3/10 Simon Riggs : On Fri, Mar 9, 2012 at 6:51 PM, Andrew Dunstan wrote: On 03/09/2012 01:40 PM, Robert Haas wrote: On Fri, Mar 9, 2012 at 12:02 PM, David E. Wheeler wrote: On Mar 9, 2012, at 7:55 AM, Merlin Moncure wrote: 100% agree

Re: [v9.3] Extra Daemons (Re: [HACKERS] elegant and effective way for running jobs inside a database)

2012-06-29 Thread Boszormenyi Zoltan
2012-06-21 23:53 keltezéssel, Simon Riggs írta: On 21 June 2012 19:13, Jaime Casanova wrote: On Sun, Jun 10, 2012 at 4:15 AM, Kohei KaiGai wrote: 2012/6/8 Simon Riggs : I have a prototype that has some of these characteristics, so I see our work as complementary. At present, I don't think

Re: [v9.3] Extra Daemons (Re: [HACKERS] elegant and effective way for running jobs inside a database)

2012-06-21 Thread Simon Riggs
On 21 June 2012 19:13, Jaime Casanova wrote: > On Sun, Jun 10, 2012 at 4:15 AM, Kohei KaiGai wrote: >> 2012/6/8 Simon Riggs : >> >>> I have a prototype that has some of these characteristics, so I see >>> our work as complementary. >>> >>> At present, I don't think this patch would be committable

Re: [v9.3] Extra Daemons (Re: [HACKERS] elegant and effective way for running jobs inside a database)

2012-06-21 Thread Jaime Casanova
On Sun, Jun 10, 2012 at 4:15 AM, Kohei KaiGai wrote: > 2012/6/8 Simon Riggs : > >> I have a prototype that has some of these characteristics, so I see >> our work as complementary. >> >> At present, I don't think this patch would be committable in CF1, but >> I'd like to make faster progress with

Re: [v9.3] Extra Daemons (Re: [HACKERS] elegant and effective way for running jobs inside a database)

2012-06-10 Thread Kohei KaiGai
2012/6/8 Simon Riggs : > On 25 April 2012 10:40, Kohei KaiGai wrote: > >> I tried to implement a patch according to the idea. It allows extensions >> to register an entry point of the self-managed daemon processes, >> then postmaster start and stop them according to the normal manner. > > The patc

Re: [v9.3] Extra Daemons (Re: [HACKERS] elegant and effective way for running jobs inside a database)

2012-06-08 Thread Simon Riggs
On 25 April 2012 10:40, Kohei KaiGai wrote: > I tried to implement a patch according to the idea. It allows extensions > to register an entry point of the self-managed daemon processes, > then postmaster start and stop them according to the normal manner. The patch needs much work yet, but has m

Re: [v9.3] Extra Daemons (Re: [HACKERS] elegant and effective way for running jobs inside a database)

2012-04-25 Thread Simon Riggs
On Wed, Apr 25, 2012 at 10:40 AM, Kohei KaiGai wrote: > I tried to implement a patch according to the idea. It allows extensions > to register an entry point of the self-managed daemon processes, > then postmaster start and stop them according to the normal manner. I've got a provisional version

[v9.3] Extra Daemons (Re: [HACKERS] elegant and effective way for running jobs inside a database)

2012-04-25 Thread Kohei KaiGai
2012/3/10 Simon Riggs : > On Fri, Mar 9, 2012 at 6:51 PM, Andrew Dunstan wrote: >> >> >> On 03/09/2012 01:40 PM, Robert Haas wrote: >>> >>> On Fri, Mar 9, 2012 at 12:02 PM, David E. Wheeler >>>  wrote: On Mar 9, 2012, at 7:55 AM, Merlin Moncure wrote: > > 100% agree  (having re-r

[HACKERS] elegant and effective way for running jobs inside a database

2012-03-12 Thread Artur Litwinowicz
Dear Developers, I am looking for elegant and effective way for running jobs inside a database or cluster - for now I can not find that solution. OK if You say "use cron" or "pgAgent" I say I know that solutions, but the are not effective and elegant. Compilation of "pgAgent" is a pain (especial

Re: [HACKERS] elegant and effective way for running jobs inside a database

2012-03-12 Thread Simon Riggs
On Sat, Mar 10, 2012 at 2:59 PM, Andrew Dunstan wrote: > The devil is in the details, though, pace Mies van der Rohe. > > In particular, it's the "tight integration" piece I'm worried about. > > What is the postmaster supposed to do if the daemon start fails? What if it > gets a flood of failures

Re: [HACKERS] elegant and effective way for running jobs inside a database

2012-03-11 Thread Dimitri Fontaine
Tom Lane writes: > I don't want to have a server-side ticker at all, especially not one > that exists only for a client that might or might not be there. We've > been doing what we can to reduce PG's idle-power consumption, which is > an important consideration for large-data-center applications.

Re: [HACKERS] elegant and effective way for running jobs inside a database

2012-03-10 Thread Andrew Dunstan
On 03/10/2012 07:11 AM, Simon Riggs wrote: On Fri, Mar 9, 2012 at 6:51 PM, Andrew Dunstan wrote: On 03/09/2012 01:40 PM, Robert Haas wrote: On Fri, Mar 9, 2012 at 12:02 PM, David E. Wheeler wrote: On Mar 9, 2012, at 7:55 AM, Merlin Moncure wrote: 100% agree (having re-read the thread a

Re: [HACKERS] elegant and effective way for running jobs inside a database

2012-03-10 Thread Artur Litwinowicz
W dniu 2012-03-09 16:55, Merlin Moncure pisze: > On Fri, Mar 9, 2012 at 9:36 AM, Kohei KaiGai > wrote: >> 2012/3/6 Alvaro Herrera : >>> It seems to me that the only thing that needs core support is >>> the ability to start up the daemon when postmaster is ready to >>> accept queries, and shut the

Re: [HACKERS] elegant and effective way for running jobs inside a database

2012-03-10 Thread Simon Riggs
On Fri, Mar 9, 2012 at 6:51 PM, Andrew Dunstan wrote: > > > On 03/09/2012 01:40 PM, Robert Haas wrote: >> >> On Fri, Mar 9, 2012 at 12:02 PM, David E. Wheeler >>  wrote: >>> >>> On Mar 9, 2012, at 7:55 AM, Merlin Moncure wrote: 100% agree  (having re-read the thread and Alvaro's idea hav

Re: [HACKERS] elegant and effective way for running jobs inside a database

2012-03-09 Thread Andrew Dunstan
On 03/09/2012 01:40 PM, Robert Haas wrote: On Fri, Mar 9, 2012 at 12:02 PM, David E. Wheeler wrote: On Mar 9, 2012, at 7:55 AM, Merlin Moncure wrote: 100% agree (having re-read the thread and Alvaro's idea having sunk in). Being able to set up daemon processes side by side with the postmas

Re: [HACKERS] elegant and effective way for running jobs inside a database

2012-03-09 Thread Robert Haas
On Fri, Mar 9, 2012 at 12:02 PM, David E. Wheeler wrote: > On Mar 9, 2012, at 7:55 AM, Merlin Moncure wrote: >> 100% agree  (having re-read the thread and Alvaro's idea having sunk >> in).  Being able to set up daemon processes side by side with the >> postmaster would fit the bill nicely.  It's p

Re: [HACKERS] elegant and effective way for running jobs inside a database

2012-03-09 Thread David E. Wheeler
On Mar 9, 2012, at 7:55 AM, Merlin Moncure wrote: > 100% agree (having re-read the thread and Alvaro's idea having sunk > in). Being able to set up daemon processes side by side with the > postmaster would fit the bill nicely. It's pretty interesting to > think of all the places you could go wi

Re: [HACKERS] elegant and effective way for running jobs inside a database

2012-03-09 Thread Merlin Moncure
On Fri, Mar 9, 2012 at 9:36 AM, Kohei KaiGai wrote: > 2012/3/6 Alvaro Herrera : >> It seems to me that the only thing that needs core support is the >> ability to start up the daemon when postmaster is ready to accept >> queries, and shut the daemon down when postmaster kills backends (either >> b

Re: [HACKERS] elegant and effective way for running jobs inside a database

2012-03-09 Thread Kohei KaiGai
2012/3/6 Alvaro Herrera : > It seems to me that the only thing that needs core support is the > ability to start up the daemon when postmaster is ready to accept > queries, and shut the daemon down when postmaster kills backends (either > because one crashed, or because it's shutting down). > +10

Re: [HACKERS] elegant and effective way for running jobs inside a database

2012-03-07 Thread Tom Lane
Merlin Moncure writes: > sure, I get that, especially in regards to procedures. a server > ticker though is a pretty small thing and it's fair to ask if maybe > that should be exposed instead of (or perhaps in addition to) a job > scheduling system. I don't want to have a server-side ticker at a

Re: [HACKERS] elegant and effective way for running jobs inside a database

2012-03-07 Thread Merlin Moncure
On Wed, Mar 7, 2012 at 2:14 PM, Simon Riggs wrote: > The stored procedure route sounds attractive but its a long way off > and doesn't address all of the states needs people have voiced. I'm > not against doing both, I just want to do the quickest and easiest. sure, I get that, especially in rega

Re: [HACKERS] elegant and effective way for running jobs inside a database

2012-03-07 Thread Simon Riggs
On Wed, Mar 7, 2012 at 7:55 PM, Merlin Moncure wrote: > On Wed, Mar 7, 2012 at 2:15 AM, Simon Riggs wrote: >> We talked about this at last year's Dev meeting. And we got >> sidetracked into "what we really want is stored procedures". Maybe we >> want that, but its a completely separate thing. Ple

Re: [HACKERS] elegant and effective way for running jobs inside a database

2012-03-07 Thread Merlin Moncure
On Wed, Mar 7, 2012 at 2:15 AM, Simon Riggs wrote: > We talked about this at last year's Dev meeting. And we got > sidetracked into "what we really want is stored procedures". Maybe we > want that, but its a completely separate thing. Please lets not get > distracted from a very simple thing becau

Re: [HACKERS] elegant and effective way for running jobs inside a database

2012-03-07 Thread Alvaro Herrera
Excerpts from Simon Riggs's message of mié mar 07 05:15:03 -0300 2012: > We talked about this at last year's Dev meeting. And we got > sidetracked into "what we really want is stored procedures". Maybe we > want that, but its a completely separate thing. Please lets not get > distracted from a ve

Re: [HACKERS] elegant and effective way for running jobs inside a database

2012-03-07 Thread Pavel Stehule
2012/3/7 Simon Riggs : > On Tue, Mar 6, 2012 at 3:21 PM, Tom Lane wrote: > >> But having said that, it's not apparent to me why such a thing would >> need to live "inside the database" at all.  It's very easy to visualize >> a task scheduler that runs as a client and requires nothing new from the

Re: [HACKERS] elegant and effective way for running jobs inside a database

2012-03-07 Thread Simon Riggs
On Tue, Mar 6, 2012 at 3:21 PM, Tom Lane wrote: > But having said that, it's not apparent to me why such a thing would > need to live "inside the database" at all.  It's very easy to visualize > a task scheduler that runs as a client and requires nothing new from the > core code.  Approaching the

Re: [HACKERS] elegant and effective way for running jobs inside a database

2012-03-06 Thread Daniel Farina
On Tue, Mar 6, 2012 at 3:31 PM, Andrew Dunstan wrote:> > We don't slavishly need to reproduce every piece of cron. In any case, on my > Linux machine at least, batch is part of the "at" package, not the "cron" > package. If you want anything at all done, then I'd suggest starting with a > simple s

Re: [HACKERS] elegant and effective way for running jobs inside a database

2012-03-06 Thread Andrew Dunstan
On 03/06/2012 06:12 PM, Christopher Browne wrote: On Tue, Mar 6, 2012 at 5:01 PM, Alvaro Herrera wrote: Why do we need a ticker? Just fetch the time of the task closest in the future, and sleep till that time or a notify arrives (meaning schedule change). Keep in mind that cron functionali

Re: [HACKERS] elegant and effective way for running jobs inside a database

2012-03-06 Thread Christopher Browne
On Tue, Mar 6, 2012 at 5:01 PM, Alvaro Herrera wrote: > Why do we need a ticker?  Just fetch the time of the task closest in the > future, and sleep till that time or a notify arrives (meaning schedule > change). Keep in mind that cron functionality also includes "batch", which means that the pro

Re: [HACKERS] elegant and effective way for running jobs inside a database

2012-03-06 Thread Tom Lane
Alvaro Herrera writes: > I was thinking that the connection would be kept open but no query would > be running. Does this preclude reception of notifies? I mean, you > don't sleep via "SELECT pg_sleep()" but rather a select/poll in the > daemon. No. If you're not inside a transaction, notifies

Re: [HACKERS] elegant and effective way for running jobs inside a database

2012-03-06 Thread Alvaro Herrera
Excerpts from Merlin Moncure's message of mar mar 06 19:07:51 -0300 2012: > > On Tue, Mar 6, 2012 at 4:01 PM, Alvaro Herrera > wrote: > > Why do we need a ticker?  Just fetch the time of the task closest in the > > future, and sleep till that time or a notify arrives (meaning schedule > > change

Re: [HACKERS] elegant and effective way for running jobs inside a database

2012-03-06 Thread Merlin Moncure
On Tue, Mar 6, 2012 at 4:01 PM, Alvaro Herrera wrote: > Why do we need a ticker?  Just fetch the time of the task closest in the > future, and sleep till that time or a notify arrives (meaning schedule > change). Because that can't be done in userland (at least, not without stored procedures) sin

Re: [HACKERS] elegant and effective way for running jobs inside a database

2012-03-06 Thread Alvaro Herrera
Excerpts from Dimitri Fontaine's message of mar mar 06 18:44:18 -0300 2012: > Josh Berkus writes: > > Activity and discretion beyond that could be defined in PL code, > > including run/don't run conditions, activities, and dependancies. The > > only thing Postgres doesn't currently have is a clo

Re: [HACKERS] elegant and effective way for running jobs inside a database

2012-03-06 Thread Merlin Moncure
On Tue, Mar 6, 2012 at 3:44 PM, Dimitri Fontaine wrote: > Josh Berkus writes: >> Activity and discretion beyond that could be defined in PL code, >> including run/don't run conditions, activities, and dependancies.  The >> only thing Postgres doesn't currently have is a clock which fires >> event

Re: [HACKERS] elegant and effective way for running jobs inside a database

2012-03-06 Thread Dimitri Fontaine
Josh Berkus writes: > Activity and discretion beyond that could be defined in PL code, > including run/don't run conditions, activities, and dependancies. The > only thing Postgres doesn't currently have is a clock which fires > events. Anything we try to implement which is more complex than the

Re: [HACKERS] elegant and effective way for running jobs inside a database

2012-03-06 Thread Josh Berkus
>> It seems to me that the only thing that needs core support is the >> ability to start up the daemon when postmaster is ready to accept >> queries, and shut the daemon down when postmaster kills backends (either >> because one crashed, or because it's shutting down). I think this could be addre

Re: [HACKERS] elegant and effective way for running jobs inside a database

2012-03-06 Thread Jaime Casanova
On Tue, Mar 6, 2012 at 1:14 PM, Alvaro Herrera wrote: > > It seems to me that the only thing that needs core support is the > ability to start up the daemon when postmaster is ready to accept > queries, and shut the daemon down when postmaster kills backends (either > because one crashed, or becau

Re: [HACKERS] elegant and effective way for running jobs inside a database

2012-03-06 Thread Christopher Browne
On Tue, Mar 6, 2012 at 12:47 PM, Robert Haas wrote: > And also some interface.  It'd be useful to have background jobs that > executed either immediately or at a certain time or after a certain > delay, as well as repeating jobs that execute at a certain interval or > on a certain schedule.  Figur

Re: [HACKERS] elegant and effective way for running jobs inside a database

2012-03-06 Thread Merlin Moncure
On Tue, Mar 6, 2012 at 9:37 AM, Robert Haas wrote: >> But having said that, it's not apparent to me why such a thing would >> need to live "inside the database" at all.  It's very easy to visualize >> a task scheduler that runs as a client and requires nothing new from the >> core code.  Approachi

Re: [HACKERS] elegant and effective way for running jobs inside a database

2012-03-06 Thread Alvaro Herrera
Excerpts from Pavel Stehule's message of mar mar 06 14:57:30 -0300 2012: > 2012/3/6 Robert Haas : > > On Tue, Mar 6, 2012 at 12:37 PM, Christopher Browne > > wrote: > >> On Tue, Mar 6, 2012 at 12:20 PM, Artur Litwinowicz wrote: > >>> Algorithm for first loop: > >>> check jobs exists and is time

Re: [HACKERS] elegant and effective way for running jobs inside a database

2012-03-06 Thread Pavel Stehule
2012/3/6 Robert Haas : > On Tue, Mar 6, 2012 at 12:37 PM, Christopher Browne > wrote: >> On Tue, Mar 6, 2012 at 12:20 PM, Artur Litwinowicz wrote: >>> Algorithm for first loop: >>> check jobs exists and is time to run it >>>   run job as other sql statements (some validity check may be done) >>>

Re: [HACKERS] elegant and effective way for running jobs inside a database

2012-03-06 Thread Robert Haas
On Tue, Mar 6, 2012 at 12:37 PM, Christopher Browne wrote: > On Tue, Mar 6, 2012 at 12:20 PM, Artur Litwinowicz wrote: >> Algorithm for first loop: >> check jobs exists and is time to run it >>   run job as other sql statements (some validity check may be done) >>   get next job >> no jobs - dela

Re: [HACKERS] elegant and effective way for running jobs inside a database

2012-03-06 Thread Christopher Browne
On Tue, Mar 6, 2012 at 12:20 PM, Artur Litwinowicz wrote: > Algorithm for first loop: > check jobs exists and is time to run it >   run job as other sql statements (some validity check may be done) >   get next job > no jobs - delay There are crucial things missing here, namely the need to establ

Re: [HACKERS] elegant and effective way for running jobs inside a database

2012-03-06 Thread Artur Litwinowicz
With all the respect to all into this Community... I have no experience enough rich with C or C++ to say yes I can do that alone. I do not know the internals of PostgreSQL at all. But I have quite long experience with other languages. I imagine if you have a piece of code which can run function lik

Re: [HACKERS] elegant and effective way for running jobs inside a database

2012-03-06 Thread Alvaro Herrera
Excerpts from Tom Lane's message of mar mar 06 12:47:46 -0300 2012: > Robert Haas writes: > > But since you brought it up, I think there is a lot of value to having > > a scheduler that's integrated with the database. There are many > > things that the database does which could also be done out

Re: [HACKERS] elegant and effective way for running jobs inside a database

2012-03-06 Thread Tom Lane
Robert Haas writes: > On Tue, Mar 6, 2012 at 10:21 AM, Tom Lane wrote: >> But having said that, it's not apparent to me why such a thing would >> need to live "inside the database" at all.  It's very easy to visualize >> a task scheduler that runs as a client and requires nothing new from the >>

Re: [HACKERS] elegant and effective way for running jobs inside a database

2012-03-06 Thread Robert Haas
On Tue, Mar 6, 2012 at 10:21 AM, Tom Lane wrote: > Robert Haas writes: >> On Mon, Mar 5, 2012 at 5:03 PM, Artur Litwinowicz wrote: >>> Regarding a functional area I can help... but I can not understand why >>> this idea is so unappreciated? > >> I think it's a bit unfair to say that this idea is

Re: [HACKERS] elegant and effective way for running jobs inside a database

2012-03-06 Thread Pavel Stehule
2012/3/6 Tom Lane : > Robert Haas writes: >> On Mon, Mar 5, 2012 at 5:03 PM, Artur Litwinowicz wrote: >>> Regarding a functional area I can help... but I can not understand why >>> this idea is so unappreciated? > >> I think it's a bit unfair to say that this idea is unappreciated. > > Well, ther

Re: [HACKERS] elegant and effective way for running jobs inside a database

2012-03-06 Thread Tom Lane
Robert Haas writes: > On Mon, Mar 5, 2012 at 5:03 PM, Artur Litwinowicz wrote: >> Regarding a functional area I can help... but I can not understand why >> this idea is so unappreciated? > I think it's a bit unfair to say that this idea is unappreciated. Well, there is the question of why we sh

Re: [HACKERS] elegant and effective way for running jobs inside a database

2012-03-06 Thread Robert Haas
On Mon, Mar 5, 2012 at 5:03 PM, Artur Litwinowicz wrote: > Regarding a functional area I can help... but I can not understand why > this idea is so unappreciated? I think it's a bit unfair to say that this idea is unappreciated. There are LOTS of good features that we don't have yet simply becaus

Re: [HACKERS] elegant and effective way for running jobs inside a database

2012-03-05 Thread David Johnston
> > > > Keep in mind that it's not about coding in C but mostly about figuring > > out what a sane design out to look like. > > > While I can straddle the fence pretty my first reaction is that we are talking about "application" functionality that falls outside what belongs in "core" PostgreSQL

Re: [HACKERS] elegant and effective way for running jobs inside a database

2012-03-05 Thread Daniel Farina
On Mon, Mar 5, 2012 at 12:17 PM, Pavel Stehule wrote: > Hello > > 2012/3/5 Alvaro Herrera : >> >> Excerpts from Artur Litwinowicz's message of lun mar 05 16:18:56 -0300 2012: >>> Dear Developers, >>>    I am looking for elegant and effective way for running jobs inside a >>> database or cluster -

Re: [HACKERS] elegant and effective way for running jobs inside a database

2012-03-05 Thread Christopher Browne
On Mon, Mar 5, 2012 at 4:44 PM, Alvaro Herrera wrote: > > Excerpts from Artur Litwinowicz's message of lun mar 05 18:32:44 -0300 2012: > >> Ouch... "in next 2-4 years" - it broke my heart like a bullet - You >> should not write it... ;) >> I feel that I need to set aside SQL, Python, PHP and so on

Re: [HACKERS] elegant and effective way for running jobs inside a database

2012-03-05 Thread Artur Litwinowicz
W dniu 2012-03-05 23:09, Jaime Casanova pisze: > On Mon, Mar 5, 2012 at 5:03 PM, Artur Litwinowicz wrote: >> >> I understand it... (I meant if you wanna something... do it for your >> self - it is the fastest way). > > other way is to fund the work so someone can use his/her time to do it > >> R

Re: [HACKERS] elegant and effective way for running jobs inside a database

2012-03-05 Thread Jaime Casanova
On Mon, Mar 5, 2012 at 5:03 PM, Artur Litwinowicz wrote: > > I understand it... (I meant if you wanna something... do it for your > self - it is the fastest way). other way is to fund the work so someone can use his/her time to do it > Regarding a functional area I can help... but I can not unde

Re: [HACKERS] elegant and effective way for running jobs inside a database

2012-03-05 Thread Artur Litwinowicz
W dniu 2012-03-05 22:44, Alvaro Herrera pisze: > > Excerpts from Artur Litwinowicz's message of lun mar 05 18:32:44 -0300 2012: > >> Ouch... "in next 2-4 years" - it broke my heart like a bullet - You >> should not write it... ;) >> I feel that I need to set aside SQL, Python, PHP and so on and t

Re: [HACKERS] elegant and effective way for running jobs inside a database

2012-03-05 Thread Alvaro Herrera
Excerpts from Artur Litwinowicz's message of lun mar 05 18:32:44 -0300 2012: > Ouch... "in next 2-4 years" - it broke my heart like a bullet - You > should not write it... ;) > I feel that I need to set aside SQL, Python, PHP and so on and take to > my hands old book about C programming language

Re: [HACKERS] elegant and effective way for running jobs inside a database

2012-03-05 Thread Artur Litwinowicz
W dniu 2012-03-05 22:09, Pavel Stehule pisze: > 2012/3/5 Artur Litwinowicz : >> W dniu 2012-03-05 20:56, Alvaro Herrera pisze: >>> >>> Excerpts from Artur Litwinowicz's message of lun mar 05 16:18:56 -0300 2012: Dear Developers, I am looking for elegant and effective way for running jo

Re: [HACKERS] elegant and effective way for running jobs inside a database

2012-03-05 Thread Pavel Stehule
2012/3/5 Artur Litwinowicz : > W dniu 2012-03-05 20:56, Alvaro Herrera pisze: >> >> Excerpts from Artur Litwinowicz's message of lun mar 05 16:18:56 -0300 2012: >>> Dear Developers, >>>    I am looking for elegant and effective way for running jobs inside a >>> database or cluster - for now I can n

Re: [HACKERS] elegant and effective way for running jobs inside a database

2012-03-05 Thread Artur Litwinowicz
W dniu 2012-03-05 20:56, Alvaro Herrera pisze: > > Excerpts from Artur Litwinowicz's message of lun mar 05 16:18:56 -0300 2012: >> Dear Developers, >>I am looking for elegant and effective way for running jobs inside a >> database or cluster - for now I can not find that solution. > > Yeah, i

Re: [HACKERS] elegant and effective way for running jobs inside a database

2012-03-05 Thread Pavel Stehule
Hello 2012/3/5 Alvaro Herrera : > > Excerpts from Artur Litwinowicz's message of lun mar 05 16:18:56 -0300 2012: >> Dear Developers, >>    I am looking for elegant and effective way for running jobs inside a >> database or cluster - for now I can not find that solution. > > Yeah, it'd be good to h

Re: [HACKERS] elegant and effective way for running jobs inside a database

2012-03-05 Thread Alvaro Herrera
Excerpts from Artur Litwinowicz's message of lun mar 05 16:18:56 -0300 2012: > Dear Developers, >I am looking for elegant and effective way for running jobs inside a > database or cluster - for now I can not find that solution. Yeah, it'd be good to have something. Many people say it's not n

  1   2   >