Re: proposal: make config parsing non-blocking.

2006-04-26 Thread Gonzalo Arana
On 4/26/06, Robert Collins [EMAIL PROTECTED] wrote:
 On Wed, 2006-04-26 at 07:23 -0300, Gonzalo Arana wrote:
  Hi,

 perhaps we should discuss this on list ? (You took it offlist, I just
 followed suit).

Absolutely! My mistake, sory (I'm used to have 'Reply-To' headers set
in mails I receive from mailing lists).

  On 4/25/06, Robert Collins [EMAIL PROTECTED] wrote:
   On Tue, 2006-04-25 at 09:40 -0300, Gonzalo Arana wrote:
I may be wrong, but the only signifficant difference (in wall clock
time of blocking squid) between checking configuration and applying it
are the external helpers (external_acl, auth helpers, piped loggers).
   
Getting configuration data from the network could be a nice thing on
the other hand, but external_acl provide the means for doing something
similar.
   
Rather than providing a 'slow migration' for every configuration
directive, making 'slow migration' for 'expensive' directives (like
external_acl, auth helpers, etc.) would have the same result but with
less work, right?
  
   Nope.
  
   Any non blocking task can only be used by other non blocking tasks, all
   the way out the main event loop.
  
   Put another way, if you imagine a call stack:
   main()
 event_loop()
   FUNCTION()
  
   if FUNCTION is a non blocking task, such as doEvents(), or
   comm_select(), then it can call either blocking calls (for a performance
   penalty), or other non blocking calls (which is ideal).
  
   if FUNCTION is a blocking task like squidReconfigure(), then it can only
   call other blocking calls.
  
   So the *first* blocking call means that all other calls down from there
   need to be blocking.
  
   We cannot use any non blocking calls in the reconfiguration process
   while the configuration is a blocking event.
 
  While I understand your point, I don't understand why does this
  contradicts what I suggested.  My proposal was something like this:
 
  1) reconfigure is requested
  2) parse configuration  apply non-slow-migration directives.
  3) open new passive socket (if port is changed).
  4) assign new callbacks to that port.
  5) mark all helpers as 'die  create a new one when idle' (support for
  this flag has to be added, I think).
 
  This way, applying new configuration is a blocking procedure, but
  should be much faster than waiting for all external helpers to warm
  up.  This is a way to achieve what you suggested, am I right?

 Not really. For instance: one common configuration of squid is to have
 the ACL's configured outside of the squid configuration, and reading
 these can take some time.

right.

 Anyhow, what I thought you were saying was to have some slow sections
 within the parsing, which is where my statements did contradict you -
 because the parsing then has to be slow (as opposed to grouping the
 actions to be taken after parsing as fast/slow).

I see, and I agree with you.

It would be just great to have squid reconfiguration without closing
passive sockets, but any deep rework on squid3 will delay it's stable
release.  My counterproposal was just to suggest a way of achieving
this with minimum changes to squid3.

Regards,

--
Gonzalo A. Arana


Re: proposal: make config parsing non-blocking.

2006-04-25 Thread Gonzalo Arana
I may be wrong, but the only signifficant difference (in wall clock
time of blocking squid) between checking configuration and applying it
are the external helpers (external_acl, auth helpers, piped loggers).

Getting configuration data from the network could be a nice thing on
the other hand, but external_acl provide the means for doing something
similar.

Rather than providing a 'slow migration' for every configuration
directive, making 'slow migration' for 'expensive' directives (like
external_acl, auth helpers, etc.) would have the same result but with
less work, right?

Regards,

On 4/25/06, Robert Collins [EMAIL PROTECTED] wrote:
 This is just a long term thought: should we make config parsing non
 blocking.

 This would allow reconfiguration to on a running system with less
 disruption than it has now, but...

 would require that all the configuration data be much more dynamic than
 it is now - for instance the closure of listening ports, conversion of a
 port that is currently listening as http to listen as https etc.

 The nice thing about this is that we could configure up a new
 configuration that is not 'active', make sure its all koscher and so
 forth, then let it slide into place by a pivot-like process - change the
 callbacks for all the listening ports, and update any other global
 variables - but not removing the prior config - and then let the old
 config free as references to it die.

 This isn't very tied into async/sync behaviour, but making it explicitly
 async would allow for parsing very big configs, or geting configuration
 data from over the network etc.

 Rob
 --
 GPG key available at: http://www.robertcollins.net/keys.txt.



--
Gonzalo A. Arana