[Puppet-dev] Re: Queuing some server-side operations

2009-03-31 Thread Luke Kanies
On Mar 30, 2009, at 11:49 PM, David Lutterkort wrote: > > On Thu, 2009-03-19 at 23:06 -0400, Ethan Rowe wrote: >> I'm a newcomer, so feel free to ignore me. But, if RESTful design >> has >> become an increasingly important consideration in interactions >> between >> client and server (and it

[Puppet-dev] Re: Queuing some server-side operations

2009-03-30 Thread Clay McClure
On Tue, Mar 31, 2009 at 12:49 AM, David Lutterkort wrote: > I've spent some time recently banging my head against the limitations of > REST for oVirt - the assumption that REST makes that everything is a > resource on which you execute a fixed number of actions is really nice > if you are dealin

[Puppet-dev] Re: Queuing some server-side operations

2009-03-30 Thread David Lutterkort
On Thu, 2009-03-19 at 23:06 -0400, Ethan Rowe wrote: > I'm a newcomer, so feel free to ignore me. But, if RESTful design has > become an increasingly important consideration in interactions between > client and server (and it's my impression that it has), I think you > might find that it fits bet

[Puppet-dev] Re: Queuing some server-side operations

2009-03-24 Thread Luke Kanies
On Mar 24, 2009, at 9:27 PM, Ohad Levy wrote: > > Steven posted this picture already: > > http://reductivelabs.com/images/Puppet_Parsing_Architecture_0.25.x.png > > And here's what the queueing does: > > http://reductivelabs.com/images/Puppet_Parsing_Architecture_With_Queueing_0.25.x.png > > Do t

[Puppet-dev] Re: Queuing some server-side operations

2009-03-24 Thread Nigel Kersten
On Tue, Mar 24, 2009 at 7:27 PM, Ohad Levy wrote: >> >> Steven posted this picture already: >> >> http://reductivelabs.com/images/Puppet_Parsing_Architecture_0.25.x.png >> >> And here's what the queueing does: >> >> >> http://reductivelabs.com/images/Puppet_Parsing_Architecture_With_Queueing_0.25

[Puppet-dev] Re: Queuing some server-side operations

2009-03-24 Thread Ohad Levy
> > > Steven posted this picture already: > > http://reductivelabs.com/images/Puppet_Parsing_Architecture_0.25.x.png > > And here's what the queueing does: > > > http://reductivelabs.com/images/Puppet_Parsing_Architecture_With_Queueing_0.25.x.png > > Do they help? > Yes, a few questions: 1. Can w

[Puppet-dev] Re: Queuing some server-side operations

2009-03-24 Thread Luke Kanies
On Mar 18, 2009, at 3:50 PM, James Turnbull wrote: > Ethan Rowe wrote: >> * puppetqd runs as a separate process, and is listening to the >> relevant >> queue (though the degree to which it uses the indirection system >> natively for queue subscription needs to be worked out a bit more); >> as

[Puppet-dev] Re: Queuing some server-side operations

2009-03-19 Thread Luke Kanies
On Mar 19, 2009, at 10:06 PM, Ethan Rowe wrote: > > Luke Kanies wrote: >> On Mar 19, 2009, at 9:52 PM, Sam Rowe wrote: >> >>> On Wed, Mar 18, 2009 at 1:16 PM, Luke Kanies >>> wrote: Hi all, As mentioned in an email yesterday, I'm working with someone to add a queuein

[Puppet-dev] Re: Queuing some server-side operations

2009-03-19 Thread Ethan Rowe
Luke Kanies wrote: > On Mar 19, 2009, at 9:52 PM, Sam Rowe wrote: > >> On Wed, Mar 18, 2009 at 1:16 PM, Luke Kanies wrote: >>> Hi all, >>> >>> As mentioned in an email yesterday, I'm working with someone to add a >>> queueing service to Puppet so that some server-side operations are >>> queued a

[Puppet-dev] Re: Queuing some server-side operations

2009-03-19 Thread Sam Rowe
On Thu, Mar 19, 2009 at 10:57 PM, Luke Kanies wrote: > The queueing would only help with reducing client contention when the > contention was caused by StoreConfigs, unfortunately. > > Although... > > I could see having a ping-back model, where a client sends a request > for a compilation but the

[Puppet-dev] Re: Queuing some server-side operations

2009-03-19 Thread Luke Kanies
On Mar 19, 2009, at 9:52 PM, Sam Rowe wrote: > > On Wed, Mar 18, 2009 at 1:16 PM, Luke Kanies wrote: >> >> Hi all, >> >> As mentioned in an email yesterday, I'm working with someone to add a >> queueing service to Puppet so that some server-side operations are >> queued and executed as cpu time

[Puppet-dev] Re: Queuing some server-side operations

2009-03-19 Thread Sam Rowe
On Wed, Mar 18, 2009 at 1:16 PM, Luke Kanies wrote: > > Hi all, > > As mentioned in an email yesterday, I'm working with someone to add a > queueing service to Puppet so that some server-side operations are > queued and executed as cpu time is available - generally those that > aren't on the crit

[Puppet-dev] Re: Queuing some server-side operations

2009-03-19 Thread Luke Kanies
On Mar 18, 2009, at 2:36 PM, Larry Ludwig wrote: > > > On Mar 18, 2009, at 2:01 PM, Paul Nasrat wrote: > >> >>> I'm working with someone to add a >>> queueing service to Puppet so that some server-side operations are >>> queued and executed as cpu time is available - generally those that >>> aren

[Puppet-dev] Re: Queuing some server-side operations

2009-03-19 Thread Luke Kanies
On Mar 18, 2009, at 1:01 PM, Paul Nasrat wrote: > >> As mentioned in an email yesterday, I'm working with someone to add a >> queueing service to Puppet so that some server-side operations are >> queued and executed as cpu time is available - generally those that >> aren't on the critical path bu

[Puppet-dev] Re: Queuing some server-side operations

2009-03-19 Thread Steven Jenkins
Ethan Rowe wrote: ... > A diagram would presumably help quite a lot. Words can express the > design, but the abstractions are of the sort that will tend to lose a > reader. Luke recently pointed out : http://reductivelabs.com/trac/puppet/wiki/ParsingArchitecture, which should probably be the s

[Puppet-dev] Re: Queuing some server-side operations

2009-03-19 Thread James Turnbull
Ethan Rowe wrote: >> 1. Configuration should all run out of puppet.conf - no new files for >> this (I am assuming that was the intent). > > Since the message service particulars will need to be shared by at least > daemons (puppetmaster, puppetqd), it makes sense to place those > particulars in a

[Puppet-dev] Re: Queuing some server-side operations

2009-03-19 Thread Ethan Rowe
James Turnbull wrote: > Ethan Rowe wrote: >> * puppetqd runs as a separate process, and is listening to the relevant >> queue (though the degree to which it uses the indirection system >> natively for queue subscription needs to be worked out a bit more); as >> messages come in, it serializes them

[Puppet-dev] Re: Queuing some server-side operations

2009-03-18 Thread James Turnbull
Ethan Rowe wrote: > * puppetqd runs as a separate process, and is listening to the relevant > queue (though the degree to which it uses the indirection system > natively for queue subscription needs to be worked out a bit more); as > messages come in, it serializes them and stores them to the datab

[Puppet-dev] Re: Queuing some server-side operations

2009-03-18 Thread Ethan Rowe
Luke Kanies wrote: [snip] > So, I'm looking for input on what people think the right tools and > architecture are for this, and just generally how we should go about > it. I don't want to have a six week discussion on it, but I do want > to make sure we get as much feedback as possible duri

[Puppet-dev] Re: Queuing some server-side operations

2009-03-18 Thread Larry Ludwig
On Mar 18, 2009, at 2:01 PM, Paul Nasrat wrote: > >> I'm working with someone to add a >> queueing service to Puppet so that some server-side operations are >> queued and executed as cpu time is available - generally those that >> aren't on the critical path but particularly the storeconfigs sav

[Puppet-dev] Re: Queuing some server-side operations

2009-03-18 Thread Paul Nasrat
> As mentioned in an email yesterday, I'm working with someone to add a > queueing service to Puppet so that some server-side operations are > queued and executed as cpu time is available - generally those that > aren't on the critical path but particularly the storeconfigs save > operation. Soun