[Zope-dev] ZConfig keys
Hello, I would like to confirm that ZConfig keys are case insensitive and that the corresponding attributes on the config object returned by the 'loadConfig' call are always lower case. The only thing that seems relevant about keys and case sensitivity in the documentation does not state it explicitely : '''When this schema is loaded, a set of defaults for the derived section type is computed. Since basic-key is case-insensitive (everything is converted to lower case), foo and Foo are both converted to foo, which clashes since key only allows one value for each key.''' Thanks -- Godefroid Chapelle (aka __gotcha)http://bubblenet.be ___ Zope-Dev maillist - [EMAIL PROTECTED] http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
[Zope-dev] Possible Windows Service improvements.
Hi all, I am starting to venture into the wonderful world of Zope! With the benefit of a complete lack of Zope experience, I have been able to look at the Windows service support from a fairly clean slate. However, I also realize this lack of experience means my ideas may be naive - hence I have attempted to split them into discrete issues for discrete rejection wink. 1) startup error redirection. I've noticed that the main Zope service driver for Windows seems to work fine when everything is setup correctly, but when things go wrong it offers no clues as to what. This is reflected in collector item 1020 (poor error reporting on product initialisation failure under windows). Issue 1408 (Configuration file imports don't see INSTANCE_HOME when running Zope as a windows service), via the referenced thread, has evidence of someone burning a day due to this. It cost me alot of time too :) I propose: Each time the child process terminates with a non-zero return code, the tail x-bytes of the child output be written to the Windows event log, where x~2k. 2) reporting of successful start and backoff strategy. A trivial startup error (eg, PYTHONPATH not set) will cause the Zope service to hopelessly retry for a number of minutes, and not respond to shutdown requests during a retry. At the moment, as soon as the service starts it reports successful startup to Windows. It then begins an attempt to start the child. If the child immediately fails, the code immediately begins the backoff strategy. This strategy appears to have 2 main purposes: * Startup may fail due to other 'services' not having yet started, so retry in the hope they become available. * The process may die due to some obscure error - restart it. On windows, assuming we install the service to depend on the tcpip service, I see no reason that the first reason is valid. If the process fails quickly the first time we attempt to launch it, it is almost certainly going to fail every time we try and launch it. The current strategy also means that 3rd party services could not themselves depend on the Zope service - the Zope service will report successful startup before it really has (and therefore the dependent service may itself fail). This isn't a known requirement today, but who knows! net start and other front ends also fail to detect fatal errors - they all say Zope started OK. I propose: We insist the child process can be created and continues to run for x seconds (where x~5). If that fails, we report an error (never reporting to Windows that we started successfully). If the child process stays alive for this period, we report success to Windows, and then use the existing backoff strategy should it die. If the machine is heavily loaded, this 5 seconds may expire before the fatal error is hit in the child - in that case, we are simply doing what we do now - using the backoff strategy to hopelessly attempt a restart - ie, a win in most cases, and no loss in the others. 3) environment setting The service process should set a number of environment variables before spawning the child - PYTHONPATH at a minimum, and according to issue #1408, INSTANCE_HOME. It already knows these values thanks to mkzopeinstance. I'm yet to determine where these values comes from for in binary build, but I see no reason not to fix this (and possibly remove whatever magic the binary does) I propose: A few trivial os.environ insertions based on the substitutions done by mkzopeinstance, before we create the child process(es). Alternatively, we create an explicit new environment we pass to CreateProcess, but I see no good reason for that.) 4) Currently, when the process is stopped, we immediately terminate the child process. This seems dangerous. We should find a way to gracefully terminate the child, and try that before we simply kill it. I propose: That someone help me work out how to do this wink. I've already worked out how if the service knows the username/password of a Zope administrator, but it doesn't! Sending a Ctrl_C 'signal' doesn't work without hacks to run.py (and I'm yet to confirm it will even with such hacks). I welcome any feedback on these issues. Obviously I am willing to back each of these proposals up (except 4!) with code that seems to work :) I would also welcome feedback on the best way to proceed (ie, create a new collector for each issue? thrash it out here? give up?wink, etc) Note that none of these issues would require a win32all/pywin32 update. If anyone was really upset by issue 1423 (Zope 2.7.1 won't run as service under NT), and also able to test, I'd be willing to fix it - but that *would* require a pywin32 upgrade. Tim has already kindly filled me in on that background, so it may not be trivial (ie, I would need help!) Thanks! Mark ___ Zope-Dev maillist - [EMAIL PROTECTED] http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! **
Re: [Zope-dev] Problems with session on ZEO
On Tue, 2004-08-03 at 03:34, Gerhard Schmidt wrote:
Hi,
we have some problems with our tempstorage on out ZEO client.
We are running 6 zope Frontend server and one ZEO server. Serving
Both main and temp storage.
I runs well for some days. But than it stops working an reports the
following error
Traceback (innermost last):
Module ZPublisher.Publish, line 101, in publish
Module ZPublisher.mapply, line 88, in mapply
Module ZPublisher.Publish, line 39, in call_object
Module Shared.DC.Scripts.Bindings, line 306, in __call__
Module Shared.DC.Scripts.Bindings, line 343, in _bindAndExec
Module Products.PageTemplates.ZopePageTemplate, line 222, in _exec
Module Products.PageTemplates.PageTemplate, line 96, in pt_render
- ZopePageTemplate at /TUM_Portal/login_form
Module TAL.TALInterpreter, line 189, in __call__
Module TAL.TALInterpreter, line 218, in interpret
Module TAL.TALInterpreter, line 233, in interpret
Module TAL.TALInterpreter, line 670, in do_useMacro
Module TAL.TALInterpreter, line 233, in interpret
Module TAL.TALInterpreter, line 693, in do_defineSlot
Module TAL.TALInterpreter, line 233, in interpret
Module TAL.TALInterpreter, line 636, in do_condition
Module TAL.TALInterpreter, line 233, in interpret
Module TAL.TALInterpreter, line 483, in do_insertText_tal
Module Products.PageTemplates.TALES, line 227, in evaluateText
Module Products.PageTemplates.TALES, line 221, in evaluate
Module Products.PageTemplates.ZRPythonExpr, line 47, in __call__
- __traceback_info__: here.REQUEST.SESSION.set('callerURL',
here.REQUEST.HTTP_REFERER)
Module Python expression here.REQUEST.SESSION.set('callerURL',
here.REQUEST.HTTP_REFERER), line 1, in expression
Module ZPublisher.HTTPRequest, line 1218, in __getattr__
Module ZPublisher.HTTPRequest, line 1178, in get
Module Products.Sessions.SessionDataManager, line 94, in getSessionData
Module Products.Sessions.SessionDataManager, line 183, in _getSessionDataObject
SystemError: error return without exception set
Wow! You appear to be the first person to get this symptom who does not
use Plone. This is a great data point. This error is being tracked at
http://zope.org/Collectors/Zope/1401 . There is no solution currently,
we need volunteers to help reproduce and/or debug it.
Restarting the zeo server and one frontend fixes the problem for some
time.
i am running Zope 2.7.2 and Python 2.3.4
Here is the zeo config
snip config
I suspect the fact that you're running sessioning over ZEO doesn't much
matter here as people who are using the default config also get the
error... (which is a good data point in itself).
- C
___
Zope-Dev maillist - [EMAIL PROTECTED]
http://mail.zope.org/mailman/listinfo/zope-dev
** No cross posts or HTML encoding! **
(Related lists -
http://mail.zope.org/mailman/listinfo/zope-announce
http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Problems with session on ZEO
On Wed, Aug 04, 2004 at 08:37:28AM -0400, Chris McDonough wrote: Wow! You appear to be the first person to get this symptom who does not use Plone. This is a great data point. This error is being tracked at http://zope.org/Collectors/Zope/1401 . There is no solution currently, we need volunteers to help reproduce and/or debug it. I am running plone2. Restarting the zeo server and one frontend fixes the problem for some time. i am running Zope 2.7.2 and Python 2.3.4 Here is the zeo config snip config I suspect the fact that you're running sessioning over ZEO doesn't much matter here as people who are using the default config also get the error... (which is a good data point in itself). Im getting this error since i have updated to Zope 2.7.1 and switched from filestorage to tempstorage for sessioning. Bye Estartu - Gerhard Schmidt | E-Mail: [EMAIL PROTECTED] TU-München| WWW Online Services | Tel: 089/289-25270| Fax: 089/289-22253| PGP-Publickey auf Anfrage pgpCKqh85YehB.pgp Description: PGP signature ___ Zope-Dev maillist - [EMAIL PROTECTED] http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Possible Windows Service improvements.
On Wed, 2004-08-04 at 07:35, Mark Hammond wrote: Hi all, I am starting to venture into the wonderful world of Zope! With the benefit of a complete lack of Zope experience, I have been able to look at the Windows service support from a fairly clean slate. However, I also realize this lack of experience means my ideas may be naive - hence I have attempted to split them into discrete issues for discrete rejection wink. 1) startup error redirection. I've noticed that the main Zope service driver for Windows seems to work fine when everything is setup correctly, but when things go wrong it offers no clues as to what. This is reflected in collector item 1020 (poor error reporting on product initialisation failure under windows). Issue 1408 (Configuration file imports don't see INSTANCE_HOME when running Zope as a windows service), via the referenced thread, has evidence of someone burning a day due to this. It cost me alot of time too :) Yes, sorry about that! (I was the fool who checked in the NameError referenced in 1408). I propose: Each time the child process terminates with a non-zero return code, the tail x-bytes of the child output be written to the Windows event log, where x~2k. This is a good idea. FWIW, I believe the Zope HEAD already has some work done towards this (in lib/python/nt_svcutils/service.py), although the child output goes to a logfile instead of the event log. It would be nice to make the output go to the event log and then backport this to 2.7. 2) reporting of successful start and backoff strategy. A trivial startup error (eg, PYTHONPATH not set) will cause the Zope service to hopelessly retry for a number of minutes, and not respond to shutdown requests during a retry. Yup. The reason it retry-restarts is because it's simple and stupid and the reason it doesn't respond to shutdown requests during a retry is because the service code sleeps for the backoff interval after an unsuccessful startup. Any async requests that happen in the meantime are blocked waiting for this sleep to end. I'm not quite sure how to do that better. At the moment, as soon as the service starts it reports successful startup to Windows. It then begins an attempt to start the child. If the child immediately fails, the code immediately begins the backoff strategy. This strategy appears to have 2 main purposes: * Startup may fail due to other 'services' not having yet started, so retry in the hope they become available. * The process may die due to some obscure error - restart it. One concrete example of the obscure error is that the Zope process handles a restart request from its Control Panel web interface. On windows, assuming we install the service to depend on the tcpip service, I see no reason that the first reason is valid. If the process fails quickly the first time we attempt to launch it, it is almost certainly going to fail every time we try and launch it. The current strategy also means that 3rd party services could not themselves depend on the Zope service - the Zope service will report successful startup before it really has (and therefore the dependent service may itself fail). This isn't a known requirement today, but who knows! net start and other front ends also fail to detect fatal errors - they all say Zope started OK. I propose: We insist the child process can be created and continues to run for x seconds (where x~5). If that fails, we report an error (never reporting to Windows that we started successfully). If the child process stays alive for this period, we report success to Windows, and then use the existing backoff strategy should it die. If the machine is heavily loaded, this 5 seconds may expire before the fatal error is hit in the child - in that case, we are simply doing what we do now - using the backoff strategy to hopelessly attempt a restart - ie, a win in most cases, and no loss in the others. That sounds good. 3) environment setting The service process should set a number of environment variables before spawning the child - PYTHONPATH at a minimum, and according to issue #1408, INSTANCE_HOME. It already knows these values thanks to mkzopeinstance. I'm yet to determine where these values comes from for in binary build, but I see no reason not to fix this (and possibly remove whatever magic the binary does) I propose: A few trivial os.environ insertions based on the substitutions done by mkzopeinstance, before we create the child process(es). Alternatively, we create an explicit new environment we pass to CreateProcess, but I see no good reason for that.) Note that the Zope Python install also has a sitecustomize.py that munges sys.path in order to get things set up properly. Others have claimed this is unnecessary and that the work that gets done in there could be done in the service code. It's a bit of a mess. At one point I flailed trying to make the child process inherit its
Re: [Zope-dev] Problems with session on ZEO
On Wed, 2004-08-04 at 08:47, Gerhard Schmidt wrote: On Wed, Aug 04, 2004 at 08:37:28AM -0400, Chris McDonough wrote: Wow! You appear to be the first person to get this symptom who does not use Plone. This is a great data point. This error is being tracked at http://zope.org/Collectors/Zope/1401 . There is no solution currently, we need volunteers to help reproduce and/or debug it. I am running plone2. Ah. Well, at least it continues to be consistent then... Restarting the zeo server and one frontend fixes the problem for some time. i am running Zope 2.7.2 and Python 2.3.4 Here is the zeo config snip config I suspect the fact that you're running sessioning over ZEO doesn't much matter here as people who are using the default config also get the error... (which is a good data point in itself). Im getting this error since i have updated to Zope 2.7.1 and switched from filestorage to tempstorage for sessioning. I suspect the first change (updating to Zope 2.7.1) is to blame, and the second change (switching from filestorage to tempstorage) is unrelated. I wish I had a solution other than move back to 2.7.0 for you, but I don't. Someone needs to help make this error trivially repeatable in order to get it fixed. - C ___ Zope-Dev maillist - [EMAIL PROTECTED] http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] ZConfig keys
On Wed, 04 Aug 2004 10:24:30 +0200, Godefroid Chapelle [EMAIL PROTECTED] wrote: I would like to confirm that ZConfig keys are case insensitive and that the corresponding attributes on the config object returned by the 'loadConfig' call are always lower case. It sounds like I need to clarify the documentation a bit. ZConfig schema allow you to change the case-sensitivity by changing the keytype of the schema or of a sectiontype. If you don't use this feature, the basic-key type is used, which converts to lower case. The effect is case-insensitivity for the keys. The attributes on the configuration objects, both for the top-level schema and individual sections, are computed by taking the key (after conversion by the keytype) and replacing hyphens with underscores. So a key of My-Key is the same as my-key, and the attribute name will be my_key. -Fred -- Fred L. Drake, Jr.fdrake at gmail.com Zope Corporation ___ Zope-Dev maillist - [EMAIL PROTECTED] http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Possible Windows Service improvements.
Mark Hammond wrote: ... I propose: We insist the child process can be created and continues to run for x seconds (where x~5). I recommend against this strategy. It's too implicit and has bitten us on Unix in the past. The problem with the strategy if the program starts very fast and then dies, we don't restart it even though we should. Similarly, if the program starts very slowly, we don't recognize a startup failure and keep retrying even though we shouldn't. Is there some sort of IPC that can be used here? On Unix, we use a reserved exit code to indicate a startup failure and arrange for Zope to exit with that. On another note, I'd really prefer to work out a general facility that can be used with any Python program, including both Zope 2 and Zope 3. Jim -- Jim Fulton mailto:[EMAIL PROTECTED] Python Powered! CTO (540) 361-1714http://www.python.org Zope Corporation http://www.zope.com http://www.zope.org ___ Zope-Dev maillist - [EMAIL PROTECTED] http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Problems with session on ZEO
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Andreas Jung wrote: ~ The whole problem might be related to some version of | PlacelessTranslationService. | Christian Heimes' hint was to upgrade to the latest PTS version from the | CVS which | seems to solve the problem although the traceback indicates some deeper | error | in some C-extension. | Well its releated to PTS so far that the PTS session negotiator accessed the session on evey request. Without this this session load is gone and therefore the error does not pop up. I assume no matter which code accesses the session _very_ frequent leads to this exception in the end. Cheers, ~ Simon - -- Simon Eisenmann [ mailto:[EMAIL PROTECTED] ] [ struktur AG | Junghansstraße 5 | 70469 Stuttgart ] [ T. +49.711.896656.68 | F.+49.711.89665610 ] [ http://www.struktur.de | mailto:[EMAIL PROTECTED] ] -BEGIN PGP SIGNATURE- Version: GnuPG v1.2.4 (GNU/Linux) Comment: Using GnuPG with Thunderbird - http://enigmail.mozdev.org iD8DBQFBEP5YcKXkdvrBGTARAm4EAJ45iGHKh15CbTDGJ0tLpOUNoMfPQgCg1NgZ +6soN9zEQcwh8W7WdqOFhVI= =/TFT -END PGP SIGNATURE- ___ Zope-Dev maillist - [EMAIL PROTECTED] http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Problems with session on ZEO
Gerhard Schmidt wrote at 2004-8-3 09:34 +0200: ... Module Products.Sessions.SessionDataManager, line 94, in getSessionData Module Products.Sessions.SessionDataManager, line 183, in _getSessionDataObject SystemError: error return without exception set I had an issue in the same line: it called an incredibly expensive __len__ function. Maybe, in your case, it calls some broken __len__ function... I replaced the line by: # hasattr hides conflicts # DM 2004-07-09: and boolean tests may be very expensive! if getattr(ob, '__of__', None) is not None and getattr(ob, 'aq_parent', None) is not None: See whether this helps you... -- Dieter ___ Zope-Dev maillist - [EMAIL PROTECTED] http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Problems with session on ZEO
On Wed, 2004-08-04 at 17:42, Dieter Maurer wrote: Gerhard Schmidt wrote at 2004-8-3 09:34 +0200: ... Module Products.Sessions.SessionDataManager, line 94, in getSessionData Module Products.Sessions.SessionDataManager, line 183, in _getSessionDataObject SystemError: error return without exception set I had an issue in the same line: it called an incredibly expensive __len__ function. Yikes! Good find! - C ___ Zope-Dev maillist - [EMAIL PROTECTED] http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
RE: [Zope-dev] Possible Windows Service improvements.
[Me] I propose: We insist the child process can be created and continues to run for x seconds (where x~5). [Jim] I recommend against this strategy. It's too implicit and has bitten us on Unix in the past. The problem with the strategy if the program starts very fast and then dies, we don't restart it even though we should. Similarly, if the program starts very slowly, we don't recognize a startup failure and keep retrying even though we shouldn't. Thanks Jim. I agree with those concerns. Note that already we do not recognise startup failure and keep retyring even though we shouldn't, and is exactly what I am trying to solve. Is there some sort of IPC that can be used here? On Unix, we use a reserved exit code to indicate a startup failure and arrange for Zope to exit with that. The Unix situation is a little different - if runzope itself has immediate startup failure, then everything immediately fails - ie, as far as Unix is concerned, the service itself failed. On Windows however, runzope is executed as a *child* of the service. If the child fatally fails, the service itself is still reporting success, and hopelessly attempting a restart. The service needs to know if the child fatally failed. This doesn't apply on Unix as runzope *is* the 'service', not the child of the service. By adding a layer around run.py, I believe we could arrange for these fatal errors to be handled with a special return code. Alternatively, if Zope itself never returns an error code of 1 (one), then we could use that - Python itself returns this for unhandled exceptions. That seems dangerous though. Can you offer some advice here? * Is an exit-code of 1 suitable for a fatal error? If so, this requires no changes to the child process. However, I assume it is not suitable. * Is a special exit code, generated by a wrapper to run.py (eg, run_svc.py) suitable? If so, what value do you recommend? * Should some Win32 specific, robust IPC mechanism be investigated? This would cut deeper into a run.py wrapper, and obviously is not a general solution. On another note, I'd really prefer to work out a general facility that can be used with any Python program, including both Zope 2 and Zope 3. The problem at the moment is that our facilities are *too* general - ie, without some coordination between the parent and child, the parent must guess. The simplest coordination does seem to be process exit code, but that seems fragile. But whatever coodination is chosen, any Python program that was willing to play the coordinate game could use it. The simpler this game to play, the more fragile the system is (ie, just using exit codes is simple, but fragile; using other IPC mechanisms could be made robust, but is not simple - especially not in a platform independent way.) Mark. ___ Zope-Dev maillist - [EMAIL PROTECTED] http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
RE: [Zope-dev] Possible Windows Service improvements.
I propose: Each time the child process terminates with a non-zero return code, the tail x-bytes of the child output be written to the Windows event log, where x~2k. This is a good idea. FWIW, I believe the Zope HEAD already has some work done towards this (in lib/python/nt_svcutils/service.py), although the child output goes to a logfile instead of the event log. It would be nice to make the output go to the event log and then backport this to 2.7. I started from HEAD, and indeed did find a good attempt at making this work. However, it was disabled by default, it writes to a file, and had issues with blocking reads. I fixed this to capure the output in memory (but not all of it - just the tail), and to use a single pipe for both stdout and stderr. 2) reporting of successful start and backoff strategy. A trivial startup error (eg, PYTHONPATH not set) will cause the Zope service to hopelessly retry for a number of minutes, and not respond to shutdown requests during a retry. Yup. The reason it retry-restarts is because it's simple and stupid and the reason it doesn't respond to shutdown requests during a retry is because the service code sleeps for the backoff interval after an unsuccessful startup. Any async requests that happen in the meantime are blocked waiting for this sleep to end. I'm not quite sure how to do that better. FWIW, the way to do that better was to use WaitForSingleObject(hStopEvent, timeout_period*1000). Note that the Zope Python install also has a sitecustomize.py that munges sys.path in order to get things set up properly. Right! That is the magic I was referring to and had not yet found. Others have claimed this is unnecessary and that the work that gets done in there could be done in the service code. It's a bit of a mess. I agree with the others :) mkzopeinstance goes to lengths to provide all relevent information, and the service code does not take full advantage of that. It should be possible for Zope to work with a standard, external Python/pywin32 installation. I'm not suggesting this become a distribution option, but still a worthy goal; for developers, and to keep us honest wink. Note that with a few tweaks, Zope *does* build and work with an external Python/pywin32. At one point I flailed trying to make the child process inherit its environment from the parent, and plastered over the problem with various sys.path and PYTHONPATH and other environment variable settings. The current situation is a result. Some guidance here would be helpful. The child process *does* inherit the parent, service environment. Hence adding os.environ[] entries in the service does set them for subsequently created children. ie, setting os.environ[PYTHONPATH]=SOFTWARE_HOME in the service main code appears to avoid the sitecustomize.py requirement - the child process *does* see the new PYTHONPATH. I'm a Windows signal idiot. No - Windows is a signal idiot :) Is there a way that we can make the Zope process capture Windows signals and when the Windows equivalent of SIGTERM is sent to the process to shut it down cleanly? This is how it works on UNIX That makes sense, but: but we circumvent trying to listen for signals on Windows entirely at startup. Can you explain the above? Do you mean that on Windows, you take no special signal actions, as demonstrated in WindowsZopeStarter.registerSignals? I'll see what I can come up with though. Note that the UNIX environment has a lot of additional niceties due to responses to signals (like logfile rotation) that Windows doesn't now, which tends to have the effect of relegating Windows to a second-class platform on which to run a production Zope instance. Windows certainly has these features available - they are just not always spelt the same as they are on Unix. Sometimes they are even better wink. So there seems to be a chicken-and-egg problem - users will always consider it second class until Zope itself starts considering it first class. This is an observation, not a critisism :) Mark. ___ Zope-Dev maillist - [EMAIL PROTECTED] http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
RE: [Zope-dev] Possible Windows Service improvements.
On Wed, 2004-08-04 at 19:56, Mark Hammond wrote: On Windows however, runzope is executed as a *child* of the service. If the child fatally fails, the service itself is still reporting success, and hopelessly attempting a restart. The service needs to know if the child fatally failed. This doesn't apply on Unix as runzope *is* the 'service', not the child of the service. FWIW, the equivalent of the service manager code under UNIX is zopectl rather than runzope. zopectl has its own failure detection and backoff algorithm that's a bit more complex than the Windows service code of the same ilk. Actually that's not entirely true: zopectl is a client that attempts to communicate with a separate daemonizing process via a UNIX domain socket. The daemonizing process is really the parent of the Zope process (it just invokes runzope). The majority of code for this is in lib/python/zdaemon/zdrun.py. The mainloop that impleents the backoff algorithm is in the runforever method of the Daemonizer class in that file, and the thing that decides not to restart it if it exits with a known error code is in the reportstatus method of the same class. You probably care about none of this, but it's there if you do. ;-) By adding a layer around run.py, I believe we could arrange for these fatal errors to be handled with a special return code. Alternatively, if Zope itself never returns an error code of 1 (one), then we could use that - Python itself returns this for unhandled exceptions. That seems dangerous though. When you say these errors above, do you mean any unhandled exception? If so (and any nonzero exit code indicated a startup failure), would we really need cooperation from run.py for this? It seems like it could be done entirely inside of SvcDoRun. OTOH, if the real problem is that you can't stop the service from fruitlessly restarting itself in the face of an insoluble error because of the blocking sleep, it seems like you already solved that. Would it be a reasonable strategy to leave the backoff stupidty as-is if you were able to stop the service from flailing via the service manager CP applet and if it didn't report successful startup until the child actually starts successfully? Can you offer some advice here? * Is an exit-code of 1 suitable for a fatal error? If so, this requires no changes to the child process. However, I assume it is not suitable. * Is a special exit code, generated by a wrapper to run.py (eg, run_svc.py) suitable? If so, what value do you recommend? * Should some Win32 specific, robust IPC mechanism be investigated? This would cut deeper into a run.py wrapper, and obviously is not a general solution. I guess I'll need to wait for the confusion evidenced by my last question to clear up before I'd be able to venture an answer to that. On another note, I'd really prefer to work out a general facility that can be used with any Python program, including both Zope 2 and Zope 3. The problem at the moment is that our facilities are *too* general - ie, without some coordination between the parent and child, the parent must guess. The simplest coordination does seem to be process exit code, but that seems fragile. But whatever coodination is chosen, any Python program that was willing to play the coordinate game could use it. The simpler this game to play, the more fragile the system is (ie, just using exit codes is simple, but fragile; using other IPC mechanisms could be made robust, but is not simple - especially not in a platform independent way.) Mark. ___ Zope-Dev maillist - [EMAIL PROTECTED] http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope ) ___ Zope-Dev maillist - [EMAIL PROTECTED] http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
RE: [Zope-dev] Possible Windows Service improvements.
On Wed, 2004-08-04 at 20:24, Mark Hammond wrote: This is a good idea. FWIW, I believe the Zope HEAD already has some work done towards this (in lib/python/nt_svcutils/service.py), although the child output goes to a logfile instead of the event log. It would be nice to make the output go to the event log and then backport this to 2.7. I started from HEAD, and indeed did find a good attempt at making this work. However, it was disabled by default, it writes to a file, and had issues with blocking reads. I fixed this to capure the output in memory (but not all of it - just the tail), and to use a single pipe for both stdout and stderr. Great! Yup. The reason it retry-restarts is because it's simple and stupid and the reason it doesn't respond to shutdown requests during a retry is because the service code sleeps for the backoff interval after an unsuccessful startup. Any async requests that happen in the meantime are blocked waiting for this sleep to end. I'm not quite sure how to do that better. FWIW, the way to do that better was to use WaitForSingleObject(hStopEvent, timeout_period*1000). Is this a more-or-less direct replacement for the usage of time.sleep() in the current incarnation of SvcDoRun? Do you think you might be able to attach your patched service.py (and related files) to a future mail? Others have claimed this is unnecessary and that the work that gets done in there could be done in the service code. It's a bit of a mess. I agree with the others :) mkzopeinstance goes to lengths to provide all relevent information, and the service code does not take full advantage of that. It should be possible for Zope to work with a standard, external Python/pywin32 installation. I'm not suggesting this become a distribution option, but still a worthy goal; for developers, and to keep us honest wink. Note that with a few tweaks, Zope *does* build and work with an external Python/pywin32. Yup, it should be possible; there's really nothing special about the Zope-supplied Python, we just have a history of attempting to make Zope install under Windows without requiring that the user know anything about Python, and the easiest way to do this is to ship our own. It actually impossible to *stop* Zope from using pieces of another system-installed Python, given that it apparently looks unconditionally to the registry to find some things. (See the thread revolving around http://mail.zope.org/pipermail/zope-dev/2004-March/021979.html for the details). This is actually not a desirable feature. At one point I flailed trying to make the child process inherit its environment from the parent, and plastered over the problem with various sys.path and PYTHONPATH and other environment variable settings. The current situation is a result. Some guidance here would be helpful. The child process *does* inherit the parent, service environment. Hence adding os.environ[] entries in the service does set them for subsequently created children. ie, setting os.environ[PYTHONPATH]=SOFTWARE_HOME in the service main code appears to avoid the sitecustomize.py requirement - the child process *does* see the new PYTHONPATH. Yup, that confirms what the thread above says too. I'm a Windows signal idiot. No - Windows is a signal idiot :) Actually, signals are idiotic, but... ;-) Is there a way that we can make the Zope process capture Windows signals and when the Windows equivalent of SIGTERM is sent to the process to shut it down cleanly? This is how it works on UNIX That makes sense, but: but we circumvent trying to listen for signals on Windows entirely at startup. Can you explain the above? Do you mean that on Windows, you take no special signal actions, as demonstrated in WindowsZopeStarter.registerSignals? That's just what I mean. The signaling mechanism doesn't necessarily need to literally be signals, I just don't know of another way to asynchronously influence the state of a running process (UNIX or Windows). Maybe Zope needs to cooperate a bit, which would be fine. I'll see what I can come up with though. Note that the UNIX environment has a lot of additional niceties due to responses to signals (like logfile rotation) that Windows doesn't now, which tends to have the effect of relegating Windows to a second-class platform on which to run a production Zope instance. Windows certainly has these features available - they are just not always spelt the same as they are on Unix. Sometimes they are even better wink. So there seems to be a chicken-and-egg problem - users will always consider it second class until Zope itself starts considering it first class. This is an observation, not a critisism :) Sure. It probably won't take much work to match the features that are available under UNIX. FWIW, Tim (most recently), I (second-most-recently), and Brian Lloyd (least recently) are
RE: [Zope-dev] Possible Windows Service improvements.
FWIW, the equivalent of the service manager code under UNIX is zopectl rather than runzope. Ahh, ok. That makes more sense now, thanks. When you say these errors above, do you mean any unhandled exception? If so (and any nonzero exit code indicated a startup failure), would we really need cooperation from run.py for this? It seems like it could be done entirely inside of SvcDoRun. Yes, that is what I tried to say here: * Is an exit-code of 1 suitable for a fatal error? If so, this requires no changes to the child process. However, I assume it is not suitable. As the exit code of 1 is the only way I see to detect an unhandled exception in the child. OTOH, if the real problem is that you can't stop the service from fruitlessly restarting itself in the face of an insoluble error because of the blocking sleep, it seems like you already solved that. Yeah, it really isn't a huge issue. It leaves us with the fact that Zope always reports startup success, even in hopeless cases. This isn't a real problem - just a nicety - along the lines of making Zope a first-class service on Windows wink. I'm happy to let that idea rest. To your other mail: Do you think you might be able to attach your patched service.py (and related files) to a future mail? I'll attach the patch in private mail. I just tried attaching it to collector 1020, but I don't have permission. It has the redirection and the sleep fix, but no changes to the startup strategy. It actually impossible to *stop* Zope from using pieces of another system-installed Python, given that it apparently looks unconditionally to the registry to find some things. Yes, and it is my fault that code even exists. As far as I know, win32all was the only product that registered modules in this way, and later versions no longer do. I'd be +1 on getting this removed from Python. Thanks, Mark. ___ Zope-Dev maillist - [EMAIL PROTECTED] http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
