Re: apache 2.2 support
Jim Gallacher wrote ..
> I'd like to checkin my patch to support apache 2.2. It doesn't add any
> new functionality. Any objections?
If I recollect, the only real issue was the implications on Win32 platform
of changing the APR_STATUS_IS_SUCCESS(s) test to (s == APR_SUCCESS),
because of Win32 the definition was originally:
#define APR_STATUS_IS_SUCCESS(s) ((s) == APR_SUCCESS \
|| (s) == APR_OS_START_SYSERR + ERROR_SUCCESS)
Anyway, this prompted be to look at cases where APR_STATUS_IS_SUCCESS()
is used. They are in connobject.c and filterobject.c. I noticed something that
may need to be looked at, not strictly related to Apache 2.2 support.
This is that we just made change in connobject.c to check for initial empty
bucket brigade and to loop when that occurred. This could have come about
because of EGAIN on socket read. The EGAIN wasn't able to be seen as a
distinct error as the lower level read routine squashed the error returning
success with empty bucket brigade. The changed code was:
while (APR_BRIGADE_EMPTY(bb)) {
Py_BEGIN_ALLOW_THREADS;
rc = ap_get_brigade(c->input_filters, bb, mode, APR_BLOCK_READ,
bufsize);
Py_END_ALLOW_THREADS;
if (! APR_STATUS_IS_SUCCESS(rc)) {
PyErr_SetObject(PyExc_IOError,
PyString_FromString("Connection read error"));
return NULL;
}
}
Now looking at instances in filterobject.c where ap_get_brigade() is used,
have checks such as:
Py_BEGIN_ALLOW_THREADS;
self->rc = ap_get_brigade(self->f->next, self->bb_in, self->mode,
APR_BLOCK_READ, self->readbytes);
Py_END_ALLOW_THREADS;
if (!APR_STATUS_IS_EAGAIN(self->rc) &&
!APR_STATUS_IS_SUCCESS(self->rc)) {
PyErr_SetObject(PyExc_IOError,
PyString_FromString("Input filter read error"));
return NULL;
}
}
The code is similar to what was in connobject.c except that it checks for EGAIN
case. Already know that EGAIN may actually be squashed, but then, it doesn't
raise an error is sees EGAIN anyway.
The next section of code has:
b = APR_BRIGADE_FIRST(self->bb_in);
if (b == APR_BRIGADE_SENTINEL(self->bb_in))
return PyString_FromString("");
Now I am assuming here that the check with APR_BRIGADE_SENTINEL() is
equivalent to APR_BRIGADE_EMPTY(bb). Yes/No? Would be nice to know
that they are. If it isn't then is an empty bucket brigade being handled?
Anyway, back to Win32 platform. I can't see that the APR_OS_START_SYSERR
offset check is important. This is because any function calls would only
return the error status if the call itself failed. Take for example socket
calls.
#ifndef _WIN32_WCE
rv = WSARecv(sock->socketdes, &wsaData, 1, &dwBytes, &flags, NULL, NULL);
#else
rv = recv(sock->socketdes, wsaData.buf, wsaData.len, 0);
dwBytes = rv;
#endif
if (rv == SOCKET_ERROR) {
lasterror = apr_get_netos_error();
*len = 0;
return lasterror;
}
*len = dwBytes;
return dwBytes == 0 ? APR_EOF : APR_SUCCESS;
It only returns lasterror when it fails, otherwise APR_SUCCESS is returned
explicitly. I am going to guess that Apache would be consistent about
that. Thus highly doubt that non zero error status which indicated
success would occur and so extra check used before redundant.
In that context, see no problems with changes being commited.
Graham
Re: apache 2.2 support
Jim Gallacher wrote ..
> Graham Dumpleton wrote:
> > The next section of code has:
> >
> > b = APR_BRIGADE_FIRST(self->bb_in);
> >
> > if (b == APR_BRIGADE_SENTINEL(self->bb_in))
> > return PyString_FromString("");
> >
> > Now I am assuming here that the check with APR_BRIGADE_SENTINEL() is
> > equivalent to APR_BRIGADE_EMPTY(bb). Yes/No? Would be nice to know
> > that they are. If it isn't then is an empty bucket brigade being handled?
>
> I'm a little fuzzy right now, so I can't really say. I think we should
> either add a comment to MODPYTHON-102 or create a new issue so we don't
> loose track of this potential problem.
Looking at it a bit more, it should be okay.
#define APR_BRIGADE_EMPTY(b)APR_RING_EMPTY(&(b)->list, apr_bucket, link)
#define APR_BRIGADE_FIRST(b)APR_RING_FIRST(&(b)->list)
#define APR_BRIGADE_SENTINEL(b) APR_RING_SENTINEL(&(b)->list, apr_bucket,
link)
#define APR_RING_EMPTY(hp, elem, link) \
(APR_RING_FIRST((hp)) == APR_RING_SENTINEL((hp), elem, link))
Ie., the macro APR_BRIGADE_EMPTY() expands to the exact same test of
checking the first element against the sentinel.
Graham
Re: Last modified times
Mike Looijmans wrote ..
> I've read through the documentation. I like the interface - just call
> ap_update_mtime for every object your response depends on (and another
> great feature: It starts with the mtime of the file, so that updates to
> the script also invalidate the cached response).
>
> Exposing those three functions would make a very good addition. A little
> Pythonic wrapping around ap_update_mtime to accept reasonable things
> like datetime objects would be helpful.
Okay,I will create JIRA issue for adding it in some way.
> ap_discard_request_body() might help resolving the issue with HEAD
> request writing out a body, maybe that change was related to
> http://issues.apache.org/jira/browse/MODPYTHON-71 but I'm not sure.
The outcome of the HEAD issue was that data must still be written, else you
can muck up output filters that do things like add content length headers.
Thus, not relevant to that situation. It might though be useful where writing
buffer output and an error occurs half way. You might be able to discard
what has been written so far, allowing you to write an error response. Not
sure, will need to test out that idea.
> Mike Looijmans
> Philips Natlab / Topic Automation
>
>
> Graham Dumpleton wrote:
> > Mike, I have a feeling that Apache has ways of generating those date/time
> > strings for last modified for you. Thus, the routine shouldn't be required.
> > The real problem is that mod_python doesn't expose the methods which
> > may actually be useful to you.
> >
> > Can you look through the Apache functions described in:
> >
> > http://162.105.203.19/apache-doc/138.htm
> >
> > and indicate whether if functions like:
> >
> > ap_set_etag()
> > ap_update_mtime()
> > ap_set_last_modified()
> >
> > you could more easily do the same thing.
> >
> > The request object already provides equivalent to ap_meets_conditions(),
> > but not these others.
> >
> > I also find the function:
> >
> > ap_discard_request_body()
> >
> > to be interesting. That might be interesting to have access to as well.
> >
> > Graham
> >
> > Mike Looijmans wrote ..
> >
> >>>We probably want to defer until after 3.2.7 (final) is released to have
> >>>any serious discussion about what should constitute version 3.3, but
> >>>am still curious to know at this point where your interests in 3.3 lie.
> >>>Is it simply to help finish up eliminating all these known issues/bugs
> >>>or do you have other ideas in mind as to the direction of mod_python?
> >>
> >>Well, since you asked...
> >>
> >>I'd like to see more HTTP level helper functions. For one thing, I keep
> >>writing the same stuff for handling if-modified-since headers, accept,
> >>and similar construct. Having one library to handle these with would
> be
> >>very helpful.
> >>
> >>For example, a function to format a datetime tuple into a HTTP style
> >>date string (this one should *definitely* go into mod_python), as well
> >>as its counterpart. Note that this implementation is not correct as it
> >>does not handle timezones...
> >>
> >>##
> >>
> >>weekdayname = ('Mon', 'Tue', 'Wed', 'Thu', 'Fri', 'Sat', 'Sun')
> >>
> >>monthname = (None,
> >> 'Jan', 'Feb', 'Mar', 'Apr', 'May', 'Jun',
> >> 'Jul', 'Aug', 'Sep', 'Oct', 'Nov', 'Dec')
> >>
> >>def httpdate(dt):
> >> """Return the current date and time formatted for a
> >> message header."""
> >> now = time.time()
> >> year, month, day, hh, mm, ss, wd, y, z = dt
> >> s = "%s, %02d %3s %4d %02d:%02d:%02d GMT" % (
> >> weekdayname[wd],
> >> day, monthname[month], year,
> >> hh, mm, ss)
> >> return s
> >>
> >>##
> >>
> >>Another thing I use very often (when writing DB apps) is the
> >>if-modified-since mechanism. This helps reducing the load on the server
> >>since most clients will be visiting the same page a number of times in
> >>a
> >>session.
> >>Calling checkLastModified in my code quickly aborts rendering the page
> >>if it hasn't changed since the last visit.
> >>(This one is incomplete
Re: Refactoring of the test suite
My only comment is that, especially if tests are being split into separate files, that the access/error log files be distinct for each test with a name incorporating the name of the test. I always found it a pain to try and dig through a big log of multiple tests trying to work out which was the output from the one that was failing. Graham On 11/02/2006, at 2:23 AM, Jim Gallacher wrote: Nicolas Lehuen wrote: Hi, There is something I'd like to do for the 3.3 version : it is to refactor the test suite. It's more a chore than real development, but the current test suite is slowly becoming big and quite difficult to maintain. Also, I think the size and complexity may be intimidating to new contributors. I don't mind so much now that I'm familiar with it but when you first dive in it looks like a lot of code. We want the tests themselves to be as robust as possible and if refactoring helps I'm all for it. We've had a few problems where the a unit test was incorrect and as a result was missing mod_python problems. What I'd like to do is simply split the test runner and the published tests in various parts, matching the different functionalities that are tested. What would be great, then, is to introduce platform-specific tests, of even MPM-specific tests, so that we could exercise some parts of the code that are only available with a threaded MPM (for example, the MemorySession implementation, or the publisher reload test which I erroneously implemented with a threaded MPM in mind). +1 One thing that I'd like to settle with you is whether you are OK to split the big PerRequestTestCase class into multiple classes, to ease maintenance and configuration switches. This has the downside that Apache server is likely to be stopped and restarted a few more times during the tests, so the tests will take more time to run. On the other hand, I'd like to introduce a way to select the tests to run when launching the test suite, so that we won't have to wait for the whole test suite to pass while debugging a given problem. I think this is a good idea, and I've been thinking along the same lines in light of the changes that are necessary to support apache 2.2 (ie, different test configurations required for 2.0 and 2.2). I'm not too concerned about taking more time for the tests to complete. Anything we can do to make the tests more reliable and easier to write will be worth the extra testing time and the initial refactoring effort. Ah, and one thing I'd definitely get rid of - usage of backquotes like in `rsp`. I really don't like this magic quoting thingy, it reminds me all too much of PHP :). Well, we can't have that now can we? :) I don't know if this would be practical, but one idea I have is to put each test in it's own file in a test_modules direcotry. This directory would be scanned on test startup and the tests added dynamically. That way if someone contributes some new code, they can also provide the corresponding unit test without worrying about editing the unwieldy test.py and htdocs/tests.py files. All they need to do is create test_modules/mytest.py and htdocs/mytest.py files which get dropped into the test framework with no additional configuration required. Oh and if we are refactoring the tests, I want a "make tests" rule. I'm tired of doing: ./configure; make; sudo make install; make tests; DOH! cd test; python test.py. :) Jim
Re: [DRAFT] ANNOUNCE: Mod_python 3.2.7
On 11/02/2006, at 7:22 AM, Daniel J. Popowich wrote: Gregory (Grisha) Trubetskoy writes: On Thu, 9 Feb 2006, Jim Gallacher wrote: Depending on Grisha's preference, I think we should either put the content in svn and have a cron job on modpython.org do a nightly update, or start using the Apache infrastructure for the website. See http://www.apache.org/dev/project-site.html for more info. We've been talking about moving modpython.org to apache infrastructure for years, it's just always ended up being a lower priority thing that never got done. modpython.org isn't much of a site at this point, but if someone would be willing to step up and dedicate some serious time to web design and content, perhaps it would make it worth it to actually do the work of moving it... Could we run mod_python on apache.org's site, like, say, the current version? ;-) It'd be nice to get live demo's running of various handlers. The only problem with that I see is that since automatic module reloading doesn't work as it good as it should at the moment, updating of examples and code could be a problem as it would require Apache to be restarted. So, good idea, but until we get the importer fixed, might not be totally practical. :-( Graham
Remembering directory Apache configuration applies to.
I know the subject line doesn't mean much, but I want to outline an
idea I have for an addition to mod_python which would help solve
a few problems. The mail is likely to be long, but if people can
understand
what I am going on about, I would appreciate some feedback.
Some background information first.
In order to define which module contains a handler to be executed,
or even a specific handler within the module, the Python*Handler
directives are used. Thus you might say:
SetHandler mod_python
PythonHandler mymodule
or:
AddHandler mod_python .py
PythonHandler mymodule::handler_py
In the case where the module is not installed into a standard location
such as Python "site-packages" directory, but in the document tree,
mod_python will modify "sys.path", prepending it with what it believes
is the directory for which the Python*Handler directory is being
applied.
Unfortunately, mod_python doesn't really get this right, or can't, for a
few cases.
The main cases which mod_python has to deal with are, use of the
directive within a Directory directive or htaccess file, or inside of a
Location/LocationMatch or Files/FilesMatch directive. The file based
directives may also be nested within a Directory directive or htaccess
file.
When Python*Handler directive appears immediately inside of a
Directory directive or htaccess file:
SetHandler mod_python
PythonHandler mymodule
mod_python is able to get hold of the name of the directory and it
uses this to setup the value of "req.hlist.directory". It is the
value of
"req.hlist.directory" that mod_python prepends to "sys.path". Having
added it to "sys.path", if the module resides in that directory it
will be
able to be found.
Now consider case of file based directives, specifically Files
directive inside of a Directory directive:
SetHandler mod_python
PythonHandler mymodule::handler_py
PythonHandler mymodule::handler_csv
As logged in:
http://issues.apache.org/jira/browse/MODPYTHON-126
this does not work. The problem is that the value that mod_python is
accessing and trying to use for the directory are the strings "*.py" or
"*.csv".
In the case of a location directive:
AddHandler mod_python .py
PythonHandler mod_python.publisher | .py
what mod_python is possibly going to use (haven't confirmed it) is
the URL.
Because this is actually directory like, this could actually be an
issue because
you get something that looks like a directory being added to "sys.path".
Ultimately, this is why Python*Handler only works immediately inside of
Directory directive or htaccess file, where you want the module to be
picked up from the directory.
FWIW, I haven't yet sat down and tried to work out if Python*Handler
code in
mod_python can tell when it is in a Location/File directive and
ignore the
value it is currently using at that point and set
"req.hlist.directory" to None
instead if that is appropriate.
Now on to a prelude of what I really want to talk about.
In the new module importer which I have implemented and will be
proposing
be added to mod_python to replace the existing version, the first
argument
to the "apache.import_module()" has been overloaded so that instead of a
module name being supplied, you can supply a full path name instead.
Thus, you can say:
__here__ = os.path.dirname(__file__)
module = apache.import_module(os.path.join(__here__,"_common.py"))
When supplying the full path name, the ".py" extension has to be
specified.
Ie., it truly has to be an exact and full path name.
As well as when calling "apache.import_module()" explicitly, one can
supply
a full path name in the Apache configuration as well.
AddHandler mod_python .py
PythonHandler /some/directory/handlers/mymodule.py::handler_py | .py
Because having to specify full path names in a configuration file
like this is
a pain, especially when the directory can change when files are moved
around, a short cut is implemented.
Specifically, you can say:
AddHandler mod_python .py
PythonHandler ~/mymodule.py::handler_py | .py
The "~/" at the start of the path provided to the directive will be
replaced with
the actual directory the Python*Handler directive is being used in.
In this case
the "/some/directory" directory.
The "~/" prefix can also be used when calling "apache.import_module()"
explicitly.
config = apache.import_module("~/common/config.py")
This may be done for a handler module in the actual directory or a
subdirectory.
In other words, the directory that the Python*Handler directive was
used in
becomes a context point or base directory from which files can be
addressed
using relative paths. This is good because you don't have to go
around changing
lots of hard coded paths when you reorganise directory structures.
You can still use relative paths relative to the directory the module
is in, but
instead of having to say:
__here__ = os.path.dirna
Re: Remembering directory Apache configuration applies to.
On 12/02/2006, at 2:40 PM, Jim Gallacher wrote:
Graham Dumpleton wrote:
Thus we come to my actual idea that I want some feedback on.
The idea is to provide a new directive in mod_python that allows
you to mark
an arbitrary point in the directory hierarchy as a context point
or base directory
from which files can then be addressed using relative paths.
That all said, specific feedback I am interested in is:
1. Is this a good idea and something worthwhile putting in
mod_python?
I like it.
2. If it is, what are better names for the directives? Ie.,
instead of PythonSetDirectory
and PythonAddDirectory.
Perhaps they should be more strongly associated with the module
import idea, such PythonSetModuleDirectory and
PythonAddModuleDirectory?
3. Again if it is, what is a better name for "req.get_directories()"?
get_directories could have several meanings in the context of a
request. Would req.get_module_directories() be a better
alternative? Does that capture the concept that you are trying to
achieve?
Except that it isn't really specific to just module importing. I gave
the example
of using it to work out a relative URL to a base URL. You might also
use it to
work out a base directory for where to load a non module file such as a
ConfigParser compatible config file.
My other option was:
PythonSetBaseDirectory
PythonAddBaseDirectory
req.get_base_directories()
or:
PythonBaseDirectory
PythonSetDirectory
req.get_directories()
or:
PythonSetContext
PythonAddContent
req.get_contexts()
4. Lastly, if you think the idea is good, but how it is done is
not, how do you
think it should be done and how do you think the Python*Handler in
FIles
directive should be solved, if at all?
I'll need to read your proposal a more few times before commenting
further.
My one minor quibble is the special meaning given to the '~'
character. My brain automatically reads this as $HOME. I suspect
that other people casually looking at an apache config file may do
the same, which could lead to some confusion.
Yeah, I know and I was going to put on my list of questions as to an
alternative
to using "~/". The other option was whether to use something like:
PythonHandler ${__handler_root__}/handlers/mymodule.py
PythonHandler ${myapproot}/handlers/mymodule.py
If you start going down arbitrary variable replacement though, do you
start
to allow variable references anywhere and allow stuff like:
PythonHandler ${myapproot}/handlers/mymodule.py::${__phase__}_html
PythonOption myapp.variant production
PythonHandler ${myapproot}/handlers/${myapp.variant}/mymodule.py::$
{__phase__}_html
Ie., introduce other magic variables which refer to other attributes
associated
with the request, such as the phase, or even the value of variables
specified
using PythonOption.
Also, what variable interpolation scheme do you use, something like
above
which would need a special parser, or something like builtin system
used with
mod operator for strings.
PythonHandler %(myapproot)s/handlers/%(myapp.variant)s/
mymodule.py::%(__phase__)s_html
Need to see what conventions other parts of Apache use as should be
the same
if one went this far.
Graham
Getting Started on mod_python 3.3.
As Jim pointed out a while back, we need to get going on mod_python 3.3 before I fill up JIRA with another page of bug reports or suggestions. That said, how do we want to proceed on this? Do we want to draw up an initial list of things to do with priorities, discuss them to make sure all are okay with the fix or what is proposed, possibly assign them to individuals, and then get started? Or even before that, do we want to state general aims about what we want to address in mod_python 3.3? Do we want to focus only on addressing bugs again, or look at some new features as well? By then I might have got my SVN access sorted out. Have account, but haven't as yet tried a check out using it. BTW, I still don't have priviledges in JIRA to administer entries, ie., assign etc. Do I need/want that? How do I get that set up? Graham
Re: Getting Started on mod_python 3.3.
Jim Gallacher wrote .. > Jorey Bump wrote: > > Jim Gallacher wrote: > > > >> This is how I would set priorities: > > > > > >> Try and assign most of the issues to someone. This is a bit of PR > >> spin, but I think it looks bad when there are a large number of open > >> issues with no assignee. To the public it may look like the project > is > >> not being actively maintained. > > > > > > I think the same can be said for the lack of Apache 2.2 support. > > Personally, I would put this (as well as backporting 2.2 support to > > mod_python 3.2) as the number one priority, for both PR and pragmatic > > reasons. > > > > The need for compatibility with Apache 2.0 & 2.2 is going to be an issue > > for quite a while, and should be addressed before mod_python undergoes > > some of the significant changes that have been discussed. > > Apache 2.2 support has already been checked into svn trunk. It's just a > question of doing the backport to the 3.2.x branch once we've seen some > testing. I think we should plan on doing regular 3.2.x bugfix releases > so that the 3.3 dev branch can mature without the pressure making a > release just to fix bugs. If we want to go down the path of having interim 3.2 bug rollup releases while 3.3 is being developed, might I suggest that we target the following for such a release in the near future. MODPYTHON-77 The Simplified GIL Aquisition patches. MODPYTHON-78 Apache 2.2 patches. MODPYTHON-94 Support for optional mod_ssl functions on request object. MODPYTHON-113 Make PythonImport use apache.import_module() via CallBack method. MODPYTHON-119 DBM Session test patches. MODPYTHON-122 Bash 3.1.X configure patches. I know that MODPYTHON-94 isn't a bug fix, but a few people have been after this one. Also MODPYTHON-113 may not seem important, but will mean that any test package I make available for new importer will work properly in all cases where module imports occur. Anyway, after trolling through JIRA, these seemed to be the important ones to me, but other might have other suggestions. Now, the question is how we manage this. Do we concentrate on these only in the trunk and get them out of the way first as a 3.2.X release, holding back any changes to test framework? Or do we merge such changes from trunk on a case by case basis in 3.2.X branch? Graham
Re: Getting Started on mod_python 3.3.
Grisha wrote .. > > On Tue, 14 Feb 2006, Nicolas Lehuen wrote: > > > 2006/2/14, Graham Dumpleton <[EMAIL PROTECTED]>: > > [...] > >> If we want to go down the path of having interim 3.2 bug rollup releases > >> while 3.3 is being developed, might I suggest that we target the following > >> for such a release in the near future. > >> > >> MODPYTHON-77 > >> > >> The Simplified GIL Aquisition patches. > > If this is the one where you get "Restriction Execution" errors upon > launching a thread, then I'm kinda keen on seeing this fixed sooner than > later. Just my $0.02. :-) Yes it is. Note that there is one caveat on this fix because of how simple GIL works when there are multiple interpreters. Namely, if using third party code which uses simple GIL API, then "PythonInterpreter" must be explicitly set to "main_interpreter" for context the code is used in. This interpreter is now gauranteed to be the first interpreter which was created, thus satisfying simple GIL requirements. No real drama, we just need to document this fact. Graham
Re: mutex dir?
Grisha wrote .. > > On Tue, 14 Feb 2006, Jim Gallacher wrote: > > > I wonder if we should generalize this, so rather than PythonMutexDir, > we have > > PythonModuleConfig. Usage might look like: > > > > PythonModuleConfig mutex_dir /path/to/mutexs > > PythonModuleConfig max_mutex_locks 8 > > I may be wrong, but I think the reason this was never configurable was > because the mutex is created before the apache config is read. Correct, is actually done from the mod_python module init function. The only way one could have it dynamically changing is through an Apache -D option checked by using ap_exists_config_define(). Not too useful for general use, but may be necessary when running test suite to avoid clashes if it is that big a problem. Graham
Re: Getting Started on mod_python 3.3.
Is my list of suggested JIRA items then seen as being reasonable for such a projected release? Do we want to create a new JIRA issue for the mutex directory issue and also include a fix for that, but limit it to just an option to configure to override the directory? Graham Nicolas Lehuen wrote .. > Based on today's traffic on the mailing lists, I think that we should > go for a short-term 3.2.8 release of mod_python, with certified Apache > 2.2 support on multiple platforms. The code is only there but I > suppose we'll need a lot of testing, so maybe we could expect to > release this in a month or two (given that the Win32 source code is > not even available right now). > > Regards, > Nicolas > > 2006/2/14, Nicolas Lehuen <[EMAIL PROTECTED]>: > > 2006/2/14, Graham Dumpleton <[EMAIL PROTECTED]>: > > [...] > > > If we want to go down the path of having interim 3.2 bug rollup releases > > > while 3.3 is being developed, might I suggest that we target the following > > > for such a release in the near future. > > > > > > MODPYTHON-77 > > > > > > The Simplified GIL Aquisition patches. > > > > > > MODPYTHON-78 > > > > > > Apache 2.2 patches. > > > > > > MODPYTHON-94 > > > > > > Support for optional mod_ssl functions on request object. > > > > > > MODPYTHON-113 > > > > > > Make PythonImport use apache.import_module() via CallBack method. > > > > > > MODPYTHON-119 > > > > > > DBM Session test patches. > > > > > > MODPYTHON-122 > > > > > > Bash 3.1.X configure patches. > > > > > > I know that MODPYTHON-94 isn't a bug fix, but a few people have been > after > > > this one. Also MODPYTHON-113 may not seem important, but will mean > > > that any test package I make available for new importer will work properly > > > in all cases where module imports occur. > > > > > > Anyway, after trolling through JIRA, these seemed to be the important > ones > > > to me, but other might have other suggestions. > > > > > > Now, the question is how we manage this. Do we concentrate on these > only > > > in the trunk and get them out of the way first as a 3.2.X release, > holding back > > > any changes to test framework? Or do we merge such changes from trunk > on > > > a case by case basis in 3.2.X branch? > > > > > > Graham > > > > > > > I was thinking about working on the new test framework in parallel of > > "real work", away from the trunk (in my /branches/nlehuen directory). > > I don't think it will be too hard to track down the changes in the > > trunk tests and bring them back in the new test framework, but I may > > be wrong. One the new tests are available, I'll merge them back in the > > trunk. > > > > So I guess it's not necessary to hold back the next release do to the > > tests, and it may be a good exercise to due a few 3.2.x releases in a > > short period of time before doing the 3.3.x release. > > > > Regards, > > Nicolas > >
Re: Getting Started on mod_python 3.3.
Jim Gallacher wrote .. > Graham Dumpleton wrote: > > Is my list of suggested JIRA items then seen as being reasonable for > > such a projected release? > > This seems reasonable, since we have either already committed the fixes > or have patches available (I think). I don't have changes ready for MODPYTHON-113, having PythonImport use apache.import_module(). I am quite happy to drop that from the list as would need to be sure that one worked okay. Might also be a good idea to defer that anyway to also address MODPYTHON-117 and MODPYTHON-118 at the same time as changing code in same area. All the rest in my list we have code ready for. > > Do we want to create a new JIRA issue for the mutex directory issue and > > also include a fix for that, but limit it to just an option to configure > > to override the directory? > > I can live with that. I don't like the idea of adding alot of new > functionality into a bugfix release, but a configure option is a pretty > minor change. The configure option is probably still good to have even if later we decide to also allow it to be overridden in Apache configuration. Possibly doesn't help to solve any conflict issues with test suite though. > Jim > > > > Graham > > > > Nicolas Lehuen wrote .. > > > >>Based on today's traffic on the mailing lists, I think that we should > >>go for a short-term 3.2.8 release of mod_python, with certified Apache > >>2.2 support on multiple platforms. The code is only there but I > >>suppose we'll need a lot of testing, so maybe we could expect to > >>release this in a month or two (given that the Win32 source code is > >>not even available right now). > >> > >>Regards, > >>Nicolas > >> > >>2006/2/14, Nicolas Lehuen <[EMAIL PROTECTED]>: > >> > >>>2006/2/14, Graham Dumpleton <[EMAIL PROTECTED]>: > >>>[...] > >>> > >>>>If we want to go down the path of having interim 3.2 bug rollup releases > >>>>while 3.3 is being developed, might I suggest that we target the following > >>>>for such a release in the near future. > >>>> > >>>>MODPYTHON-77 > >>>> > >>>> The Simplified GIL Aquisition patches. > >>>> > >>>>MODPYTHON-78 > >>>> > >>>> Apache 2.2 patches. > >>>> > >>>>MODPYTHON-94 > >>>> > >>>> Support for optional mod_ssl functions on request object. > >>>> > >>>>MODPYTHON-113 > >>>> > >>>> Make PythonImport use apache.import_module() via CallBack method. > >>>> > >>>>MODPYTHON-119 > >>>> > >>>> DBM Session test patches. > >>>> > >>>>MODPYTHON-122 > >>>> > >>>> Bash 3.1.X configure patches. > >>>> > >>>>I know that MODPYTHON-94 isn't a bug fix, but a few people have been > >> > >>after > >> > >>>>this one. Also MODPYTHON-113 may not seem important, but will mean > >>>>that any test package I make available for new importer will work properly > >>>>in all cases where module imports occur. > >>>> > >>>>Anyway, after trolling through JIRA, these seemed to be the important > >> > >>ones > >> > >>>>to me, but other might have other suggestions. > >>>> > >>>>Now, the question is how we manage this. Do we concentrate on these > >> > >>only > >> > >>>>in the trunk and get them out of the way first as a 3.2.X release, > >> > >>holding back > >> > >>>>any changes to test framework? Or do we merge such changes from trunk > >> > >>on > >> > >>>>a case by case basis in 3.2.X branch? > >>>> > >>>>Graham > >>>> > >>> > >>>I was thinking about working on the new test framework in parallel of > >>>"real work", away from the trunk (in my /branches/nlehuen directory). > >>>I don't think it will be too hard to track down the changes in the > >>>trunk tests and bring them back in the new test framework, but I may > >>>be wrong. One the new tests are available, I'll merge them back in the > >>>trunk. > >>> > >>>So I guess it's not necessary to hold back the next release do to the > >>>tests, and it may be a good exercise to due a few 3.2.x releases in > a > >>>short period of time before doing the 3.3.x release. > >>> > >>>Regards, > >>>Nicolas > >>> > > > >
Re: mutex dir?
Hmmm, somehow I managed to vapourise an email, didn't even get to my sent mail box. Let me try this again. Jim Gallacher wrote .. > Graham Dumpleton wrote: > > Correct, is actually done from the mod_python module init function. > > > > The only way one could have it dynamically changing is through an > > Apache -D option checked by using ap_exists_config_define(). > > This is incorrect. You can get at the directives from python_init. Proof > of concept patch attached. Apply the patch and add the following > directives in your apache config after the LoadModule. > > LoadModule python_module /usr/lib/apache2/modules/mod_python.so > PythonModuleConfig max_locks 911 > PythonModuleConfig mutex_dir /some/place/safe > > Check the error.log for the results. > > I'm sure I don't need to tell everyone that this is intend as Proof of > Concept only. Don't blame me if your hair catches on fire and your teeth > fall out as a result of using this patch. :) I don't have much hair left, so I better believe you. Thus I stand corrected. I also should look more closely at the code before I send off email. :-) If the settings are going to be a generic key/value like in PythonOption, but only for purposes of the mod_python system itself, maybe it should be called PythonSystemOption. Prefer PythonSystemOption as "Module" is too confusing to me given you have both Apache modules and Python modules and a module importer. Also wary of "Config" because of confusion with "req.get_config()". Other than that, probably is reasonable. What other configuration things are there like this that could be handled in the same way? Would it be possible using such a thing to specify a directory that could be prefixed to sys.path to say where mod_python Apache module should look for mod_python Python modules? It is always a pain to debug sometimes when people have multiple versions of Python and for various reasons their Apache environment means it is finding wrong mod_python modules. If had way of saying where it was, could override it as way of proving one way or another it is a problem. Yes/No? Graham
Re: Testing mod_python SVN trunk with Apache 2.2 on Win32
On 15/02/2006, at 5:29 PM, Nicolas Lehuen wrote: Hi, I've built Apache 2.2 and tested mod_python SVN trunk with it. The two register_cleanup tests fail. Apparently it's because the test code registers a cleanup function giving the current request as parameter. Of course when the cleanup function is called, the request object is no longer valid, and Apache segfaults. Fixing this is only a matter of fixing the test code, yet I wonder how this code could properly run on Apache 2.0.55. Maybe the request object was not properly destroyed and this has been fixed in Apache 2.2 ? This also shows that we should document the fact that the current request object should not be passed directly or indirectly to the *.register_cleanup functions. Maybe we could implement a little test in those function to make sure it is not passed directly ? Those two failures aside, the rest of the tests are OK. I have had a quick go on Mac OS X with Apache 2.2 and SVN trunk and I get test_global_lock failing. Will have to wait until tomorrow for me to look at it further. Graham
Re: Testing mod_python SVN trunk with Apache 2.2 on Win32
I have already noted that apache.register_cleanup() and req.server.register_cleanup() are prone to failure. The problem is that they are run in the context of a signal handler. See: http://issues.apache.org/jira/browse/MODPYTHON-109 I suggested in the JIRA issue that these functions may need to be removed unless an equivalent can be found that is safe. Graham On 16/02/2006, at 6:55 AM, Nicolas Lehuen wrote: 2006/2/15, Jim Gallacher <[EMAIL PROTECTED]>: Nicolas Lehuen wrote: 2006/2/15, Jim Gallacher <[EMAIL PROTECTED]>: Nicolas Lehuen wrote: Hi, I've built Apache 2.2 and tested mod_python SVN trunk with it. The two register_cleanup tests fail. Apparently it's because the test code registers a cleanup function giving the current request as parameter. Of course when the cleanup function is called, the request object is no longer valid, and Apache segfaults. Fixing this is only a matter of fixing the test code, yet I wonder how this code could properly run on Apache 2.0.55. Maybe the request object was not properly destroyed and this has been fixed in Apache 2.2 ? The tests pass as-is on Debian linux, so this seems to be Win32 specific. Jim Yet there is no way it should pass anywhere : the test is de facto using an old request object during the shutdown of the server. Either this old request object is still valid, and we have to ask ourselves why it hasn't be properly destroyed, or the old request object isn't valid and we are just lucky under Linux and unlucky under Win32. BTW, I've launched the debugger and Apache segfaults when dereferencing the request object to get its server object, during an ap_log_rerror() call. Just to make sure I didn't introduce any regressions with the code cleanup I recently checked in, could you retest with revision 376545? The only diffence between that rev and 3.2.7 is the Apache 2.2 support and bash 3.1 configure problem. svn co -r 376545 https://svn.apache.org/repos/asf/httpd/mod_python/ trunk mp.3765545 Jim I've corrected the tests and the documentation, which were both wrong about the usage of apache.register_cleanup and server.register_cleanup. I still wonder how those two test could possibly have passed before. The usage of apache.register_cleanup was plain wrong, it should never have passed. This is not particularly reassuring about the legitimity of our test suite. I've double checked the code, and I don't see what's wrong here, so if anyone want to give a try... Anyway, FWIW, now the whole unit test suite passes with Apache 2.2 under Windows XP SP2. Regards, Nicolas
Re: mutex dir?
Jim Gallacher wrote .. > > If the settings are going to be a generic key/value like in > > PythonOption, but only for purposes of the mod_python system itself, > > maybe it should be called PythonSystemOption. Prefer PythonSystemOption > > as "Module" is too confusing to me given you have both Apache modules > > and Python modules and a module importer. > > Fair enough. PythonSystemOption is a better. The directive definition is > such that it cannot be used in VirtualHost, Directory, Location and so > on which should help emphasize that it's use in only for configuring > mod_python.so initialization. > > > Also wary of "Config" because > > of confusion with "req.get_config()". > > Except that PythonSystemOption just stuffs strings into > server->module_config->directives table, which can be accessed via > req.server.get_config(). I can't see any reason application code will > ever need the settings PythonSystemOption might set, so it's not a big > deal if it's not obvious from the name that the settings can be > retrieved with get_config. I like the new name as it signals our > intention of how PythonSystemOption should be used. This name is also > consistent with PythonOption, which is a good idea. I have a better option (pun intended). :-) We do not need a new directive. Instead use existing "PythonOption" directive. In the handler code for the directive, it can look at the value of the "cmd_parms->path" and determine if it is being used outside of all VirtualHost, Directory, Location, File directives, ie., at global scope "path" will be NULL. If it is, then stick it in a table object associated with the server level config for mod_python. This would then be accessible as: "req.server.get_options()". Because "PythonOption" as used now at global scope results in options being seen in "req.get_options()", so that still works, we merge the server options into the request one before overlaying it with the request specific options. In other words "req.get_options()" is just like now, but "req.server.get_options()" becomes the subset of options which were defined at global server scope. Anything used by mod_python itself would use a namespace prefix of "mod_python." as discussed before. Eg. PythonOption mod_python.mutex_directory Thus, existing directive is used and just needs to be stated that option is only used if at global server scope. Graham
Re: mutex dir?
Jim Gallacher wrote ..
> > I have a better option (pun intended). :-)
> >
> > We do not need a new directive. Instead use existing "PythonOption"
> > directive.
>
> That could work.
>
> > In the handler code for the directive, it can look at the
> > value of the "cmd_parms->path" and determine if it is being used outside
> > of all VirtualHost, Directory, Location, File directives, ie., at global
> > scope "path" will be NULL.
>
> The disadvantage is that every request which triggers the
> directive_PythonOption call would require a number of string
> comparisons. Granted this is done in C, but there is a penalty to be paid.
No. No string comparisons at all. Check that cmd->path == NULL only.
> > If it is, then stick it in a table object associated
> > with the server level config for mod_python. This would then be accessible
> > as: "req.server.get_options()".
>
> Do you see us adding a new apr_table_t field to the py_config structure
> (define in mod_python.h)? Or would these server options go in either the
> directives or options table?
Hmmm, this is interesting. It seems that the code may have intended to
implement the exact feature I was talking about but it wasn't completed.
The code has:
static const char *directive_PythonOption(cmd_parms *cmd, void * mconfig,
const char *key, const char *val)
{
py_config *conf;
conf = (py_config *) mconfig;
if(val!=NULL) {
apr_table_set(conf->options, key, val);
conf = ap_get_module_config(cmd->server->module_config,
&python_module);
apr_table_set(conf->options, key, val);
}
else {
/** We don't remove the value, but set it
to an empty string. There is no possibility
of colliding with an actual value, since
an entry string precisely means 'remove the value' */
apr_table_set(conf->options, key, "");
conf = ap_get_module_config(cmd->server->module_config,
&python_module);
apr_table_set(conf->options, key, "");
}
return NULL;
}
It was already setting the options table object in the server config
object, but was doing it no matter where in config. Thus no different
to request object. If instead it is written as:
static const char *directive_PythonOption(cmd_parms *cmd, void * mconfig,
const char *key, const char *val)
{
py_config *conf;
conf = (py_config *) mconfig;
if(val!=NULL) {
apr_table_set(conf->options, key, val);
if (cmd->path == NULL) {
conf = ap_get_module_config(cmd->server->module_config,
&python_module);
apr_table_set(conf->options, key, val);
}
}
else {
/** We don't remove the value, but set it
to an empty string. There is no possibility
of colliding with an actual value, since
an entry string precisely means 'remove the value' */
apr_table_set(conf->options, key, "");
if (cmd->path == NULL) {
conf = ap_get_module_config(cmd->server->module_config,
&python_module);
apr_table_set(conf->options, key, "");
}
}
return NULL;
}
Then it then correctly separates the global from the non global.
The merging code already merges the two back together to
construct the actual request object.
Code will actually run quicker than before, as have avoided inserting
values into two tables when not global config.
The only thing then missing is in serverobject.c, which needs:
/**
** server.get_options(server self)
**
* Returns the options set through PythonOption directives.
* unlike req.get_options, this one returns the per-server config
*/
static PyObject * server_get_options(serverobject *self)
{
py_config *conf =
(py_config *) ap_get_module_config(self->server->module_config,
&python_module);
return MpTable_FromTable(conf->options);
}
static PyMethodDef server_methods[] = {
{"get_config", (PyCFunction) server_get_config,
METH_NOARGS},
{"get_options", (PyCFunction) server_get_options,
METH_NOARGS},
{"register_cleanup", (PyCFunction) server_register_cleanup,
METH_VARARGS},
{ NULL, NULL } /* sentinel */
};
If I then have at global config scope:
PythonOption global 1
PythonOption override 1
and in a .htaccess file:
PythonOption local 1
PythonOption override 2
The end result is:
req.get_options()={'local': '1', 'override': '2', 'global': '1'}
req.server.get_options()={'override': '1', 'global': '1'}
> How does req.server.get_options() differ from req.server.get_config(),
> which already exists?
I still see what is in get_config() as special, ie., the values for
actual directives. Just don't think it
Re: mutex dir?
Graham Dumpleton wrote ..
> > How does req.server.get_options() differ from req.server.get_config(),
> > which already exists?
>
> I still see what is in get_config() as special, ie., the values for
> actual directives. Just don't think it is good to mix them.
Looking at this further, the distinction between req.get_config() and
req.server.get_config() seems to be all broken. The code for PythonDebug
is:
/**
** directive_PythonDebug
**
* This function called whenever PythonDebug directive
* is encountered.
*/
static const char *directive_PythonDebug(cmd_parms *cmd, void *mconfig,
int val) {
const char *rc = python_directive_flag(mconfig, "PythonDebug", val, 0);
if (!rc) {
py_config *conf = ap_get_module_config(cmd->server->module_config,
&python_module);
return python_directive_flag(conf, "PythonDebug", val, 0);
}
return rc;
}
The python_directive_flag() function always returns NULL and so it adds
the config value to both table objects. This means that local values for
the directives in a directory/location/files/host context are going to
overwrite the global ones in req.server.
This code should perhaps similarly be looking to see if cmd->path is
NULL. Before we decide that, we probably need to clarify what is
actually mean't by the req.server.get_config() documentation. Ie.,
what is it supposed to return in the first place.
get_config( )
Similar to req.get_config(), but returns a config pointed to by
server->module_config Apache config vector.
My assumption would be like in example in previous email for the
PythonOption directive, is that it is the original values for the directives
when defined outside of all containers. What the documentation
says doesn't mean much and doesn't really say this.
Thus, what do people think that req.server.get_config() is mean't
to return in comparison to req.get_config().
Graham
Re: mutex dir?
Graham Dumpleton wrote ..
> Graham Dumpleton wrote ..
> > > How does req.server.get_options() differ from req.server.get_config(),
> > > which already exists?
> >
> > I still see what is in get_config() as special, ie., the values for
> > actual directives. Just don't think it is good to mix them.
>
> Looking at this further, the distinction between req.get_config() and
> req.server.get_config() seems to be all broken. The code for PythonDebug
> is:
>
> /**
> ** directive_PythonDebug
> **
> * This function called whenever PythonDebug directive
> * is encountered.
> */
> static const char *directive_PythonDebug(cmd_parms *cmd, void *mconfig,
> int val) {
> const char *rc = python_directive_flag(mconfig, "PythonDebug", val,
> 0);
>
> if (!rc) {
> py_config *conf = ap_get_module_config(cmd->server->module_config,
>&python_module);
>
> return python_directive_flag(conf, "PythonDebug", val, 0);
> }
> return rc;
> }
>
> The python_directive_flag() function always returns NULL and so it adds
> the config value to both table objects. This means that local values for
> the directives in a directory/location/files/host context are going to
> overwrite the global ones in req.server.
>
> This code should perhaps similarly be looking to see if cmd->path is
> NULL.
Thus, FWIW, I get what I would expect when I change the code to:
/**
** directive_PythonDebug
**
* This function called whenever PythonDebug directive
* is encountered.
*/
static const char *directive_PythonDebug(cmd_parms *cmd, void *mconfig,
int val) {
const char *rc = python_directive_flag(mconfig, "PythonDebug", val, 0);
if (!cmd->path) {
py_config *conf = ap_get_module_config(cmd->server->module_config,
&python_module);
return python_directive_flag(conf, "PythonDebug", val, 0);
}
return rc;
}
It isn't just this directive processor though, all of them should have:
if (!rc) {
changed to:
if (!cmd->path) {
The actual return values from the function are really meaningless as
they are always NULL.
Graham
Constructing of a URL for location redirect.
Just sending this while I remember ... A while back there was long discussion on constructing a URL for location redirect. While digging through Apache stuff, I stumbled across the function: ap_construct_url() This takes an absolute path and constructs a full URL based on information Apache knows about request. Does this Apache function perhaps do what was wanted, or is it going to have the same issues as was discussed before whereby certain cases wouldn't work? If it works, just need to expose it in the request object. See notes about thus function in: http://www.kuzbass.ru/docs/apache_c_mod_perl/158.htm Graham
Re: Python 2.x; what's x?
Daniel J. Popowich wrote .. > > Graham Dumpleton (JIRA) writes: > > If mod_python is to still support Python 2.2, which it looks like we > > are still because of Nokia work, then can't use the Python bool type > > yet as that was only added to Python 2.3. > > But can't a decision be made? I think deciding by not deciding is > less than an ideal design process. It sure would be nice to have a > decisive: We support python versions 2.x and greater. > > So, what's x? > > Seems like a "core developer" vote, but my 2-cents is I'm hoping it's > 3, for the same reasons I outlined in a previous thread: > > http://www.mail-archive.com/python-dev@httpd.apache.org/msg01127.html Version 3.2.7 was made to be able to work with Python 2.2. http://www.mail-archive.com/python-dev@httpd.apache.org/msg01123.html Your right though. I'm not sure if a final policy was stated that 2.2 and later would be supported, there was a sort of consensus, but no actual official statement unless I missed it. FWIW, the last production system I have to deal with that has Python 2.2 will be upgraded to Python 2.3 in the next month, so on that point it makes no difference to me. At the moment, I don't think we loose anything by still supporting 2.2. That is, there is nothing specific in 2.3 which would be of huge benefit. Thus, no harm still supporting 2.2 at least for mod_python 3.2.X stream. We may well want to reconsider that for mod_python 3.3 stream. The particular patch I was talking about is going to go into 3.2.X stream, so should still work for Python 2.2. Thus, let us at least state that mod_python 3.2.X will retain support for Python 2.2. As to the mod_python 3.3 stream, there would be no gaurantees that Python 2.2 will be supported. Yes I realise this is deferring the decision again, but at the moment we don't know how long before 3.3 is done. If it takes a while, as it probably will, then by then, saying that Python 2.3 only is supported will probably be a quite reasonable thing to do. Graham
Re: mutex dir?
On 17/02/2006, at 10:01 AM, Jim Gallacher wrote:
The ap_check_cmd_context function may also be of interest.
http://docx.webperf.org/group__ap__check__cmd__context.html
For example, lets say we wanted to disallow the use of PythonDebug
in
const char *err;
if ((err = ap_check_cmd_context(cmd, NOT_IN_VIRTUALHOST))) {
return err;
} else {
/* do some processing */
}
The one I can't find a good way of doing is, how do I determine if a
directive
was used in a .htaccess file.
If you use a directive in a container, you will know
because you
can search back through parent containers of command and find it. For a
.htaccess file, I would have assumed there would have been an implicit
container created or some other indicator, but I can't
find any.
The only kludge I can even dream of at the moment is to rely on the fact
that the filename attribute will identify the full path to
the .htaccess file. If I
can be sure that it is a .htaccess file, I can take the dirname of it
and use that
as the req.hlist.directory.
The particular problem I am trying to solve is the PythonHandler in
container inside of a .htaccess file. Ie., MODPYTHON-126.
Don't know how to work that one out do you? :-(
Graham
mod_python 3.2.8, simplified GIL patch etc
I have finally got myself sorted out to start committing changes into the repository. The list as it stands for mod_python 3.2.8 bug rollup release is: MODPYTHON-77 The Simplified GIL Aquisition patches. MODPYTHON-78 Apache 2.2 patches. MODPYTHON-94 Support for optional mod_ssl functions on request object. MODPYTHON-119 DBM Session test patches. MODPYTHON-122 Bash 3.1.X configure patches. MODPYTHON-131 Make mutex directory configurable. MODPYTHON-135 FileSession security patch. I have dropped MODPYTHON-113 (PythonImport) for now and have added MODPYTHON-131 and MODPYTHON-135 which have come up since the list was made and are also probably reasonable additions at this stage. If we can get through these quick, then maybe we can look at some of the others I have added to JIRA in the last day. Right now I have patch for MODPYTHON-77 (Simplified GIL) ready to commit and after that I'll move onto MODPYTHON-94 (mod_ssl functions). Any objections to me starting to commit these changes. Just want to make sure given the GIL patch is significant even if changes not huge. BTW, my test for the GIL problem requires some C code to be compiled, so not sure how one could integrate it into the tests. Any ideas? Also, I still get failure of test suite for Apache 2.2 on Mac OS X (10.4). Fails in test_global_lock. I have managed to overwrite the log messages when I went back to Apache 2.0.55 so don't have it handy, but the error was from deep inside of Apache and looks perhaps like an Apache problem and not mod_python itself. I want to try and get some of the above out out the way before I go back and look at that problem. Graham
How mod_python treats apache.OK/apache.DECLINED response from handlers.
Nicolas, Jim, Grisha (and others who might understand this issue) When you get a chance, can you read through the JIRA issue: http://issues.apache.org/jira/browse/MODPYTHON-129 and provide some feedback on whether you think my analysis is correct and thus whether mod_python really needs to be changed. I would like to perhaps progress a fix for this issue sooner rather than later as part of changes addressing authentication. This would be MODPYTHON-124, MODPYTHON-94 plus MODPYTHON-129. If I can fix these, then feel I can give a proper solution to the SSL/ Basic auth handler question on the main list at the moment. Bit frustrating that functionality is incomplete at the moment. Thanks. Graham
Re: How mod_python treats apache.OK/apache.DECLINED response from handlers.
Thanks for the feedback. At least I am not wacko. Further comments below. On 18/02/2006, at 5:44 PM, Deron Meranda wrote: I guess the positive thing is that because of this I don't have any existing code which actually uses multiple mod_python handlers in the same phase; so this change in behavior is only positive for me. So for me it's +1 on Graham's suggested change in logic (but any incompatible change in behavior is something to think hard about). As for an example where this could even be benificial to the content phase, consider having different content handlers, one which outputs application/xhtml+xml+SVG+MathML and one for text/html+GIF. Then the first handler would look at the Accept header, etc, and if the user-agent can't support it, it returns a defer and then the text/html handler gets a chance. However, I still think even this new proposal is somewhat restrictive. For example there's no way to interleave C and python handlers in the list-to-try (this is because to apache all the mod_python handlers look just like one handler). Of course a necessary thing to go along with this, would be a way to specify the hook order (one of APR_HOOK_REALLY_FIRST, _FIRST, _MIDDLE, _LAST, or _REALLY_LAST). Perhaps mod_python can actually add multiple hooks to itself in for each phase (at each order position), and then have a way for the python-handlers to optionally specify a position. If mod_python had no handlers for a particilar hook position it would just by default return a decline. Yeah, have thought in the past about how mod_python could be made to register handlers as really first, first, last and really last. Besides how one would denote that in the configuration file, was concerned about the overhead that might result. This is because mod_python would need to register a permanent hook against every one of them and thus no matter what the request is for, you get multiple calls into mod_python for each phase just so it can check its configuration to see if it has to do anything or not. Even if it decides not to do anything, it is an overhead on every request made to the server as far as I understand it. In respect of interleaving and ordering of handlers within a phase, am not sure that Apache provides much guarantees about order within a section such as middle etc. Therefore the only way to ensure that a handler is run in the right situation is not to have its condition check for running overlap with any other handler in that section. Graham
Re: How mod_python treats apache.OK/apache.DECLINED response from handlers.
I'll post more later when I have a chance to check a few things, but I am going to backtrack a bit. How far I don't know yet. In short for now, mod_perl has a a concept called stacked handlers. You have stacked handlers when you list more than one handler with the handler directive on the same line. Mod_python equivalent would be: PythonHandler page::header page::body page::footer When handlers are stacked in this way, how status is treated would probably be like mod_python works now. Ie., keep going while OK, status of last handler is what is returned from the stack of handlers. If you list the handlers on separate lines: PythonAuthenHandler mysessionauth PythonAuthenHandler mybasicauth then the Apache rules for status would be applied against handlers for each separate handler directive as specified by Apache for that phase. What I have to check is whether mod_python does use Apache rules in the latter case or not or whether it is treating combination of all across multiple lines as stacked handlers. If mod_python isn't quite doing it right, then what mod_perl does seems to be good way of doing it. That is stacked only if all listed on same line. Still need work out what should happen with req.add_handler(). Anyway, for information of stacked handlers in mod_perl, see: http://www.oreilly.com/catalog/wrapmod/chapter/ch04.html#3948 Note that don't need chained handlers from mod_perl as that is now doable using filters. Graham On 19/02/2006, at 2:16 AM, Jim Gallacher wrote: I agree with Deron's summary of your summary. :) If we make this change (and that is a +1 from me) I suggest the following path, assuming that it is possible to control this behaviour with a PythonOption flag: 1. mp 3.3 - New behaviour is off by default, but can be turned on using a PythonOption. The old behaviour is deprecated and our user education phase begins. Perhaps we can log a message at Apache startup that the PythonOption whatever_we_call_it was not found? 2. mp 3.4 - New behaviour is *on* by default, but can still be turned off with the PythonOption. A deprecation warning goes into the Apache log if the new behaviour is turned off. 3. mp 3.5 - New behaviour only. The old way is deprecated and cannot be enabled. This means maintaining some cruft in the code during the transition, but I think this is a big enough change that we need to either jump the version to 4.0 or offer an easy transition path. Jim Deron Meranda wrote: On 2/17/06, Graham Dumpleton <[EMAIL PROTECTED]> wrote: When you get a chance, can you read through the JIRA issue: http://issues.apache.org/jira/browse/MODPYTHON-129 and provide some feedback on whether you think my analysis is correct and thus whether mod_python really needs to be changed. Yes, I think your summary is pretty accurate. I never did really understand why mod_python works the way it does in this area (I'm sure there was a reason but I wasn't around then).. I always thought the Apache logic was a lot more useful. I am one of those who do make somewhat-significant use of the other apache phases. Actually what I've been doing is to essentially write my own "looper" wrapper, which looks like just one handler to mod_python, but which then implements more of an Apache-style loop/dispatch itself in sub-handlers. So at least in my case, I've been working around mod_python rather than letting it work for me. I guess the positive thing is that because of this I don't have any existing code which actually uses multiple mod_python handlers in the same phase; so this change in behavior is only positive for me. So for me it's +1 on Graham's suggested change in logic (but any incompatible change in behavior is something to think hard about). As for an example where this could even be benificial to the content phase, consider having different content handlers, one which outputs application/xhtml+xml+SVG+MathML and one for text/html+GIF. Then the first handler would look at the Accept header, etc, and if the user-agent can't support it, it returns a defer and then the text/html handler gets a chance. However, I still think even this new proposal is somewhat restrictive. For example there's no way to interleave C and python handlers in the list-to-try (this is because to apache all the mod_python handlers look just like one handler). Of course a necessary thing to go along with this, would be a way to specify the hook order (one of APR_HOOK_REALLY_FIRST, _FIRST, _MIDDLE, _LAST, or _REALLY_LAST). Perhaps mod_python can actually add multiple hooks to itself in for each phase (at each order position), and then have a way for the python-handlers to optionally specify a position. If mod_python had no handlers for a particilar hook position it would just by default return a decline. -- Deron Meranda
What is test_req_get_basic_auth_pw meant to test.
On 19/02/2006, at 9:35 AM, Jim Gallacher wrote:
I just noticed that "write" is declared twice in request_methods
[] . What's up with that??
src/requestobject.c
static PyMethodDef request_methods[] = {
...
... line 1075
{"write", (PyCFunction) req_write, METH_VARARGS},
...
... line 1087
{"write", (PyCFunction) req_write, METH_VARARGS},
...
For my WTF moment, have a look at test_req_get_basic_auth_pw in the
test suite. I guess it is supposed to be test req.get_basic_auth_pw
(), but
that function isn't even mentioned anywhere. Plus the authenhandler
phase
will not even be run since there is no Require directive in the
configuration.
Even if you add the Require directive, it seems to be testing the
mod_auth
module and not the ability to call req.get_basic_auth_pw().
Would it perhaps be better as:
def test_req_get_basic_auth_pw_conf(self):
c = VirtualHost("*",
ServerName("test_req_get_basic_auth_pw"),
DocumentRoot(DOCUMENT_ROOT),
Directory(DOCUMENT_ROOT,
SetHandler("mod_python"),
AuthName("blah"),
AuthType("basic"),
#PythonAuthenHandler
("tests::req_get_basic_auth_pw"),
PythonHandler
("tests::req_get_basic_auth_pw"),
PythonDebug("On")))
return str(c)
def req_get_basic_auth_pw(req):
req.get_basic_auth_pw()
if req.user != "spam":
req.write("test failed")
else:
req.write("test ok")
return apache.OK
Graham
Re: What is test_req_get_basic_auth_pw meant to test.
I already fixed the test and checked it in. :-)
On 20/02/2006, at 5:15 AM, Jim Gallacher wrote:
Yes, this test seems to be broken. I'll create a JIRA issue for it.
We need unit tests for the unit tests. :)
Jim
For my WTF moment, have a look at test_req_get_basic_auth_pw in the
test suite. I guess it is supposed to be test
req.get_basic_auth_pw (), but
that function isn't even mentioned anywhere. Plus the
authenhandler phase
will not even be run since there is no Require directive in the
configuration.
Even if you add the Require directive, it seems to be testing the
mod_auth
module and not the ability to call req.get_basic_auth_pw().
Would it perhaps be better as:
def test_req_get_basic_auth_pw_conf(self):
c = VirtualHost("*",
ServerName("test_req_get_basic_auth_pw"),
DocumentRoot(DOCUMENT_ROOT),
Directory(DOCUMENT_ROOT,
SetHandler("mod_python"),
AuthName("blah"),
AuthType("basic"),
#PythonAuthenHandler
("tests::req_get_basic_auth_pw"),
PythonHandler
("tests::req_get_basic_auth_pw"),
PythonDebug("On")))
return str(c)
def req_get_basic_auth_pw(req):
req.get_basic_auth_pw()
if req.user != "spam":
req.write("test failed")
else:
req.write("test ok")
return apache.OK
Graham
Maually adding notes about commits to JIRA.
Jim Gallacher wrote .. > Graham, > > I don't think it's necessary to add an additional JIRA comment when you > commit some code. JIRA will pickup the svn commit as long as the issue > number is mentioned in the svn commit message. People can subscribe to > python-cvs if they want notification of the commits. > > This should save you a bit of work. :) > > Jim Looking at the bigger picture, I believe that adding the comment to JIRA explicitly saves me work down the track. Where I am coming from is the perspective of a user who has started using mod_python recently and is having a problem. For myself, I find it quite annoying when looking at other third party packages when investigating an issue and find the problem mentioned in a issue tracking system, but then find that it hasn't been updated in a while and thus looks like the software is still broken and isn't about to be fixed. Often one digs further though and finds that it was actually fixed some time back, but because the issue tracking system was never updated to at least mention that it was fixed in the repository you never actually know this. The average person is not going to want to subscribe to some special mailing list to find out about repository commits for a package they have only just started to use. Even if one did this, you aren't going to find out about what has happened in the past from the mailing list. Thus to me, the JIRA system is perfect for recording a full and complete history of what has happened that people can refer to. The only problem at the moment is that the JIRA system still isn't linked from the mod_python home page so people never find it in the first place. So overall I feel it saves me work as people will either find the information themselves and know that if they simply check out latest source code that they will get the fix, or at worst I can respond to questions on the mailing list with a quick link to the JIRA issue and not have to reiterate the same information all over again. I also feel that identifying the revision may be helpful when someone wants to perhaps backport changes to an older version. This may be us, or someone who needs to do it for themselves. Graham > Graham Dumpleton (JIRA) wrote: > > [ > > http://issues.apache.org/jira/browse/MODPYTHON-124?page=comments#action_12366957 > ] > > > > Graham Dumpleton commented on MODPYTHON-124: > > > > > > Changes to add req.auth_name(), req.auth_type() and make req.ap_auth_type > writable commited into mod_python SVN trunk in revsion 378864. > >
Re: JIRA Housekeeping
Jim Gallacher wrote .. > Jim Gallacher wrote: > > Now that 3.2.7 is out, should we be changing the status resolved issues > > to closed in JIRA. If that is what closed implies. Is there somewhere which states what we should interprete the different status as meaning? I don't recollect seeing anything unless I am missing the obvious. BTW, as far as I know I still don't have full JIRA access otherwise I would help with closing them off. I noticed a few on the weekend that weren't even in resolved status yet, even though fixed in 3.2.7. Grisha, if you are reading this, what do I need to do to get admin JIRA access on mod_python stuff? I recollect asking you but can't remember what you said. I thought perhaps you were going to organise it, but also can't remember. > Other JIRA thoughts: > > Should we have a "unit test" component for bugs in the actual unit test > code? > > Since we plan on having 3.2.x bugfix releases, should create new JIRA > versions starting with 3.2.7? No harm in doing so. Probably would only be used if reported by someone else or change is not simple. For the simple stuff, like basic auth, easier to just fix it on the spot, although I am tending towards thinking having a JIRA issue for all changes is a good goal. Graham
Re: JIRA Housekeeping
Jim Gallacher wrote .. > >>Other JIRA thoughts: > >> > >>Should we have a "unit test" component for bugs in the actual unit test > >>code? > >> > >>Since we plan on having 3.2.x bugfix releases, should create new JIRA > >>versions starting with 3.2.7? > > > > > > No harm in doing so. Probably would only be used if reported by someone > > else or change is not simple. For the simple stuff, like basic auth, > easier to > > just fix it on the spot, although I am tending towards thinking having > a > > JIRA issue for all changes is a good goal. > > I guess I'm just looking for a bit of clarity. Let's say we make a > 3.2.10 bugfix release. A JIRA issue which is fixed in 3.2.10 gets marked > as fixed in 3.2. A user looks at the issue and says "Great, I'm using > 3.2.7, and it says it's fixed in 3.2, so I must be ok". Hmmm, I can see why you might be confused by my response. I was only referring to the "unit test" question. Totally missed the 3.2.7 question. On the latter I'm not sure. This sort of confusion is why I like to add final comments to issues when I put stuff back in a repository. I was remiss this time in not mentioning that the fixes would be expected to first be available in 3.3.0. BTW, are we going to put up your developer guidelines anywhere for viewing? http://www.modpython.org/pipermail/mod_python/2005-December/019712.html Just spent a while trying to find out how to subscribe to python-cvs. Couldn't find it in an obvious spot unless I missed the obvious. Graham
Re: Maually adding notes about commits to JIRA.
Jim Gallacher wrote .. > All very interesting, except that JIRA does record the svn commit info, > and with great detail. It just doesn't treat the commit as a comment. > > For example: > http://issues.apache.org/jira/browse/MODPYTHON-124?page=all > > Personally I think the Subversion commit information should be included > in the comments by default, rather than requiring a separate view, but > I > guess that's between JIRA and myself. :) This is showing my ignorance then. That the commit appeared there was a fluke on my part. I was not aware that it was able to tie the commit message to the JIRA issue. I do not recollect seeing the commits there when I added the comment, perhaps because I had the issue up already before doing the commit. :-( What is the magic then? Is it purely because I had listed MODPYTHON-124 in the commit message. Where is that documented? Did a quick scan of dev.apache.org I haven't found mention of it yet. Graham
Re: How mod_python treats apache.OK/apache.DECLINED response fromhandlers.
Jim Gallacher wrote .. > I don't have alot more to say on these last 2 emails. I think you are on > the right path here. Okay, I think I have a good plan now. To summarise the whole issue, the way Apache treats multiple handlers in a single phase for non content handler phases is as follows: PostReadRequestHandler RUN_ALL TransHandler RUN_FIRST MapToStorageHandler RUN_FIRST InitHandler RUN_ALL HeaderParserHandler RUN_ALL AccessHandlerRUN_ALL AuthenHandlerRUN_FIRST AuthzHandler RUN_FIRST TypeHandler RUN_FIRST FixupHandler RUN_ALL LogHandler RUN_ALL RUN_ALL means run all handlers until one returns something other than OK or DECLINED. Thus, handler needs to return DONE or an error to have it stop processing for that phase. RUN_FIRST means run all handlers while they return DECLINED. Thus, needs handler to return OK, DONE or error to have it stop processing for that phase. Where multiple handlers are registered within mod_python for a single phase it doesn't behave like either of these. In mod_python it will keep running the handlers only while OK is returned. Returning DECLINED causes it to stop. This existing behaviour can be described (like mod_perl) as stacked handlers. Having non content handler phases behave differently to how Apache does it causes problems. For example things like a string of authentication handlers which only say OK when they handle the authentication type, can't be implemented properly. In Apache, it should stop the first time one returns OK, but in mod_python it will keep running the handlers in that phase. In summary, it needs to behave more like Apache for the non content handler phases. In respect of the content handler phase itself, in practice only one handler module is supposed to implement it. At the Apache level there is no concept of different Apache modules having goes at the content handler phase and returning DECLINED if they don't want to handle it. This is reflected in how in the type handler phase, selection of the module to deliver content is usually done by setting the single valued req.handler string. Although, when using mod_python this is done implicitly by setting the SetHandler/AddHandler directives and mod_negotiation then in turn setting req.handler to be mod_python for you. Because mod_python when executed for the content handler phase is the only thing generating the content, the existing mechanism of stacked handlers and how the status is handled is fine within just the content handler phase. Can thus keep that as is and no chance of stuffing up existing systems. Where another phase calls req.add_handler() to add a handler or multiple handlers for the "PythonHandler" (content) phase, these will be added in a stacked manner within that phase. This also is the same as it works now. There would be non need to have a new function to add stacked handlers as that behaviour would be dictated by phase being "PythonHandler". For all the non content handler phases though, the current stacked handlers algorithm used by mod_python would be replaced with how Apache does it. That is, within multiple handlers registered with mod_python for non content handler phase, it would use RUN_FIRST or RUN_ALL algorithm as appropriate for the phase. For those which use RUN_ALL, this wouldn't be much different than what mod_python does now except that returning DECLINED would cause it to go to next mod_python handler in that phase instead of stopping. It is highly unlikely that this change would have an impact as returning DECLINED in RUN_ALL phases for how mod_python currently implements it, tends not to be useful and can't see that anyone would have been using it. For those which use RUN_FIRST, the difference would be significant as reurning OK will now cause it to stop instead of going to next mod_python handler in the phase. Personally I don't think this would be a drama as not many people would be using these phases and highly unlikely that someone would have listed multiple handlers for such phases. If they had and knew what they were doing, they should have long ago realised that the current behaviour was a bit broken and it even probably stopped them from doing what they wanted unless they fudged it. As to use of req.add_handler() for non content handler phases, each call would create a distinct handler, ie., no stacking of handlers at all. No separate function is required though, as slight change in behaviour determine form phase specified. To sum up, I think these changes would have minimal if no impact as where changes are significant, it isn't likely to overlap with existing code as shortcomings in current system would have mean't people wouldn't have been doing the sorts of things that may have been impacted. Therefore, I don't see a need for this to be switch enabled and the change could just be made and merely documente
Re: 3.2.8 summary / core group vote
+1 core vote Nicolas Lehuen wrote .. > +1 core vote > > 2006/2/20, Jim Gallacher <[EMAIL PROTECTED]>: > > +1 core vote > > > > Jim > > > > Gregory (Grisha) Trubetskoy wrote: > > > > > > Based on summary below, +1 from for putting it out there. > > > > > > Grisha > > > > > > > > > Graham Dumpleton <[EMAIL PROTECTED]> > > > +1 Mac OS X 10.4, apache 2.0.55 mpm-worker, python 2.3.5 (3.2.x SVN > > > trunk) > > > > > > Nicolas Lehuen <[EMAIL PROTECTED]> > > > +1 mod_python 3.2.8 + Apache/2.0.55 + Python/2.2.3 + Windows 2000 > SP4 > > > +1 mod_python 3.2.8 + Apache/2.0.55 + ActivePython/2.3.5 + Windows > > > 2000 SP4 > > > +1 mod_python 3.2.8 + Apache/2.0.55 + ActivePython/2.4.2 + Windows > XP SP2 > > > > > > Barry Pederson <[EMAIL PROTECTED]> > > > +1 FreeBSD 6.0, apache 2.0.55 mpm-prefork, python 2.4.2 > > > > > > Jim Gallacher <[EMAIL PROTECTED]> > > > +1 Debian sid, apache 2.0.55 mpm-prefork, python 2.3.5 > > > > > > > > > >
Re: mod_python 3.2.8 available for testing
On 21/02/2006, at 7:08 AM, Jim Gallacher wrote: The Apache 2.2 support will likely go into the 3.2.9 bugfix release. We just wanted to get the security problem out of the way first. Jim, if we are again going to aim for a bug rollup release for 3.2.9 do I need to stop or hold off on doing any substantial changes to trunk, or are you going to selectively backport changes from trunk which would be appropriate, meaning that I can keep going for it. Got about half a dozen reasonably major changes sitting in my old play source directory that I was starting to look at pushing into the repository before I loose where I was up to. This includes support for dynamic filters, setting of req.handler, mod_include/ssi, fixes for server.get_config () and new server.get_options(), plus other fixes. Graham
Re: How mod_python treats apache.OK/apache.DECLINED responsefromhandlers.
Grisha wrote ..
>
> If I understand this correctly, then +1.
>
> ...though I'm wondering if anyone will actually try to do something as
> arcane as dynamicaly registering non-content handers? :-)
I agree, it might not be a totally realistic scenario, but then now that
I have checked in a change to make req.handler writable, the system
is becoming flexible enough that it may actually be reasonable to do
it for some reason.
Specifically, with the change to make req.handler writable, instead of
using SetHandler/AddHandler to have mod_mime internally set
req.handler to "mod_python", you could define your own type handler
which did it.
def typehandler(req):
if os.path.splitext(req.filename)[1] == ".py":
req.handler = "mod_python"
req.add_handler("PythonHandler","mod_python.publisher")
return apache.OK
return apache.DECLINED
You might even at the same time want to register a fixup handler
to do stuff prior to the response phase being run:
def typehandler(req):
if os.path.splitext(req.filename)[1] == ".py":
req.handler = "mod_python"
req.add_handler("PythonFixupHandler","manage_session_object")
req.add_handler("PythonHandler","mod_python.publisher")
return apache.OK
return apache.DECLINED
For example, you might introduce a fixup handler which ensures that
a session object is created and put in req.session. This is a lot cleaner
than what most people do, which is to put a call to the session manager
code in every single published function.
Graham
> On Tue, 21 Feb 2006, Jim Gallacher wrote:
>
> > Nice summary.
> > +1 for the change.
> >
> > Jim
> >
> > Graham Dumpleton wrote:
> >> Jim Gallacher wrote ..
> >>
> >>> I don't have alot more to say on these last 2 emails. I think you are
> on
> >>> the right path here.
> >>
> >>
> >> Okay, I think I have a good plan now.
> >>
> >> To summarise the whole issue, the way Apache treats multiple handlers
> in
> >> a single phase for non content handler phases is as follows:
> >>
> >> PostReadRequestHandler RUN_ALL
> >> TransHandler RUN_FIRST
> >> MapToStorageHandler RUN_FIRST
> >> InitHandler RUN_ALL
> >> HeaderParserHandler RUN_ALL
> >> AccessHandlerRUN_ALL
> >> AuthenHandlerRUN_FIRST
> >> AuthzHandler RUN_FIRST
> >> TypeHandler RUN_FIRST
> >> FixupHandler RUN_ALL
> >>
> >> LogHandler RUN_ALL
> >>
> >> RUN_ALL means run all handlers until one returns something other than
> OK
> >> or DECLINED. Thus, handler needs to return DONE or an error to have
> it
> >> stop
> >> processing for that phase.
> >>
> >> RUN_FIRST means run all handlers while they return DECLINED. Thus, needs
> >> handler to return OK, DONE or error to have it stop processing for that
> >> phase.
> >>
> >> Where multiple handlers are registered within mod_python for a single
> >> phase it doesn't behave like either of these. In mod_python it will
> keep
> >> running the handlers only while OK is returned. Returning DECLINED
> >> causes it to stop. This existing behaviour can be described (like
> >> mod_perl)
> >> as stacked handlers.
> >>
> >> Having non content handler phases behave differently to how Apache does
> >> it causes problems. For example things like a string of authentication
> >> handlers which only say OK when they handle the authentication type,
> >> can't be implemented properly. In Apache, it should stop the first time
> >> one returns OK, but in mod_python it will keep running the handlers
> >> in that phase.
> >>
> >> In summary, it needs to behave more like Apache for the non content
> >> handler phases.
> >>
> >> In respect of the content handler phase itself, in practice only one
> >> handler
> >> module is supposed to implement it. At the Apache level there is no
> >> concept of different Apache modules having goes at the content handler
> >> phase and returning DECLINED if they don't want to handle it. This is
> >> reflected in how in the type handler phase, selection of the module
> to
> >> deliver content is usually done by setting the single valued req.handler
> >> string. Although, when using mod_python this is done implici
Re: mod_python license
Jim Gallacher wrote .. > Justin Erenkrantz wrote: > > On 2/19/06, Jim Gallacher <[EMAIL PROTECTED]> wrote: > > > >>I just notice that a few files still say that mod_python uses Apache > >>License 1.1 (eg htdocs/tests.py, src/psp_string.c). Can I assume this > is > >>an error and that *everything* should be version 2.0? > > > > > > Yes. -- justin > > Should the dates be updated to 2006 as well? The copyright notice in > each file currently shows 2004. eg. > > * Copyright 2004 Apache Software Foundation Technically speaking, if you make a change to the file, you should be ensuring you add the current years date. Ie., not replace it. Thus, presuming no changes were made in 2005, you would have: * Copyright 2004, 2006 Apache Software Foundation If you have a string of consecutive years, ie., changes were also made in 2005, you would use the following it you didn't want to enumerate all the years: * Copyright 2004-2006 Apache Software Foundation At least that is how I understand copyright stuff. Apache folks may want it done differently. Graham
Re: [jira] Updated: (MODPYTHON-112) If using filters value of req.phase only valid up till first req.read()/req.write().
On 26/02/2006, at 5:58 AM, Jim Gallacher wrote:
Since we are discussing Python*Filter, can someone explain why it
is only allowed in the server config context, whereas
SetInputFilter and related directives are allowed in any context?
Is this a mod_python feature or a bug, or is it just the way
filters work?
There are two parts to it. First is the registration of a filter.
Second is saying that
that filter should be used within a particular configuration
container context.
The table used to hold the names of the registered filters is global.
Ie., shared
by the whole server (virtual server??). Because of this, it isn't
possible to say
that the registration of a filter can only be valid within a certain
context.
Although the registration is global, the filter still needs to be
associated with
specific requests using the Set*Filter directives. It is the latter
second step which
constraints it to a specific context.
Not being able to dynamically register filters for only a specific
context, is a bit
limiting, which is why I implemented:
http://issues.apache.org/jira/browse/MODPYTHON-103
With this, in 3.3 you can dynamically register filters implemented in
mod_python.
from mod_python import apache
def uppercase(filter):
s = filter.read()
while s:
filter.write(s.upper())
s = filter.read()
if s is None:
filter.close()
def fixuphandler(req):
req.register_output_filter("UPPERCASE","example::uppercase")
req.add_output_filter("UPPERCASE")
return apache.OK
You might as shown use a fixuphandler() to register a filter to
convert all content
to uppercase and then activate it for the current request.
Even if only static files are being served, they will still pass
through the filter.
The dynamic registration works by mod_python itself being registered
as a
filter under the name MOD_PYTHON on initialisation of mod_python. When a
mod_python filter is dynamically registered using
req.register_*_filter(), the
details are remembered in the per request config. When
req.add_*_filter() is called,
the MOD_PYTHON filter is attached to the request, with some context
being
supplied such that when it is called, it knows to actually delegate
to the
filter registered within the mod_python per request config.
Graham
My plans for mod_python changes (260206).
One of the problems when I am looking at making changes to mod_python is knowing that there is some consensus that changes are a good thing, or at least that there is no objection. So far if a change was trivial, I would commit it without consultation. I have also committed some changes which were non trivial but which did not overlap with existing functionality and have been documented on JIRA for some time. Other cases I will try and get a response out of list members as to whether it is a good idea before proceeding, this may be in the form of a general discussion or by posting possible patches against JIRA issue. This has sort of been working okay, but doing it in an ad-hoc manner does make it a bit harder for me to plan ahead as to what issues in JIRA I am going to work on next, especially where one change may depend on sorting another issue out first. Now that I have got developer access to JIRA and can assign/resolve issues etc, I can better indicate what issues I am currently looking at. I thought though that it may also be a good idea if I regularly post an email to the list indicating what batch of issues I would be intending to look at in the coming week or so. If I do this, I feel that the warning may give time for people to have a quick look at the issues and comment on the issue or raise an objection to the proposed changes because of problems it may cause, before I actually start proper on dealing with it. Overall I feel that this will make me more productive and allow me to get through the issues quicker, as I will not at the last moment just prior to doing a commit be waiting for some sort of consensus as whether it was a good change to make in the first place. Does this sound helpful, or does everyone just trust that I am not going to screw things up? :-) That all said, below are the list of JIRA issues I intend to address in the coming week and commit changes back into the repository for. As I do get around to each I'll assign it to myself in JIRA so everyone knows my progress and when I am ready to do a commit will also attach patches for review and then wait for a bit for any last minute objections. If any objections can be expressed early on based on just the proposal for change, that would be appreciated. https://issues.apache.org/jira/browse/MODPYTHON-133 req.server.get_config() table object populated wrongly. https://issues.apache.org/jira/browse/MODPYTHON-134 Setting PythonDebug to Off, doesn't override On setting in parent scope. https://issues.apache.org/jira/browse/MODPYTHON-137 Add req.server.get_options() for obtain PythonOption values set at global level. https://issues.apache.org/jira/browse/MODPYTHON-104 Allow Python code callouts with mod_include (SSI). https://issues.apache.org/jira/browse/MODPYTHON-109 Signal handler calling Py_Finalize() when child processes being killed. https://issues.apache.org/jira/browse/MODPYTHON-129 HandlerDispatch doesn't treat OK/DECLINED result properly for all phases. Of these, main one I am concerned about is any perceived implications of changes suggested for MODPYTHON-109. Graham
Re: mod_python license
On 20/02/2006, at 4:22 AM, Jim Gallacher wrote: I just notice that a few files still say that mod_python uses Apache License 1.1 (eg htdocs/tests.py, src/psp_string.c). Can I assume this is an error and that *everything* should be version 2.0? Jim, I think you managed to delete the Copyright line from src/ mod_python.c when you made these changes. Or was it intentional for some reason: http://svn.apache.org/viewcvs.cgi/httpd/mod_python/trunk/src/ mod_python.c?rev=379638&r1=378803&r2=379638&diff_format=h
Re: My plans for mod_python changes (260206).
Jim Gallacher wrote .. > > https://issues.apache.org/jira/browse/MODPYTHON-104 > > Allow Python code callouts with mod_include (SSI). > > The unit test fails for the applied patch. It looks like it's missing > LoadModule in the config section. I've attached a patch. I'd attach it > to the issue, but JIRA is down right... again. :( Okay, I have incorporated that into my private code workspace so it will be checked in when I do get to do it. Hadn't thought about the issue of how different Apache installations are built. Useful to know for the future. > Since this is a new feature I wonder if we should have a vote to see if > everyone agrees it should be incorporated? Do we have a policy for > adding features, other than committing and then seeing if Grisha gets > upset? ;) Good thing we are in different countries, it can't actually come to fisticuffs if I do do something wrong. Graham
Re: My plans for mod_python changes (260206).
On 04/03/2006, at 4:59 AM, Jim Gallacher wrote: More in the way of a general observation rather than on these specific issues, but I wouldn't necessarily mark things as resolved just on the basis of the fix being committed. For the big changes at least, I think we should see some testing on a couple of different platforms. Maybe we could announce development milestones and ask the community for a round of testing? Issues would be marked as resolved after each milestone test cycle. That way we are more likely to catch problems early and hopefully reduce the time it takes for the next beta cycle. We should do whatever we can to avoid the 7 months it took to get 3.2 out. (Actually, it was even longer than that, as Grisha first mentioned a 3.2 release last spring). Ideally trunk would always be in a stable enough condition that we could do a release within a month of making a decision. Okay. Pity there isn't a status that a issue can be parked in meaning "waiting to be reviewed/tested", or did I just miss it. BTW, shall I still start closing off issues fixed in 3.2.8 and earlier. Graham
Re: My plans for mod_python changes (260206).
On 04/03/2006, at 4:59 AM, Jim Gallacher wrote: More in the way of a general observation rather than on these specific issues, but I wouldn't necessarily mark things as resolved just on the basis of the fix being committed. For the big changes at least, I think we should see some testing on a couple of different platforms. Maybe we could announce development milestones and ask the community for a round of testing? Issues would be marked as resolved after each milestone test cycle. That way we are more likely to catch problems early and hopefully reduce the time it takes for the next beta cycle. We should do whatever we can to avoid the 7 months it took to get 3.2 out. (Actually, it was even longer than that, as Grisha first mentioned a 3.2 release last spring). Ideally trunk would always be in a stable enough condition that we could do a release within a month of making a decision. Jim, have read through the JIRA documentation page you once referred me to about status values and what they mean and for "Resolved" it states: http://www.atlassian.com/software/jira/docs/v3.3.2/images/docs/ config/statuses-addstatus.png A resolution has been taken, and it is awaiting verification by reporter. From here issues are either reopened and or closed. The issue for us is whether one expects a reporter to verify a fix for an issue before it has even been committed to the repository. For them it would probably be a pain to have to first checkout latest version from repository and then also apply a patch. If they do say it is okay, then by rights should verify it again after it has then been committed as who is to say the final commit isn't stuffed up somehow. They way I read all the status values is that it would be quite reasonable once a fix has been committed to mark it as resolved. If the reporter then disagrees it can be reopened. If they agree it is okay, then it should be closed there and then and not wait until some milestone. If a problem is later found, then a new issue can be created and linked to the original. Without moving issues through resolved/closed in a timely fashion, too me it just makes it harder to work out where one is up to amongst the large list of issues. Although changing the workflow to add new status values is interesting, in reality not sure this could be done as workflow is probably global across all projects. Note, marking issues as resolved as soon as committed doesn't mean that we can't do what I have been doing so far which is to attach patch to issue while "In Progress" and wait for any feedback before finally committing it and marking it "Resolved". In reality it is probably only going to be us core developers who might even comment at that time, especially since it is us who are raising most of the issues. Anyway, I now have to go assign a whole bunch of issues I fixed before I had proper JIRA access to me and mark as in progress so I at least partly know where I am up to. As you see this list grow, you might see why being able to mark them as resolved and closing off 3.2.X issues as well will help clean things up. Graham
Re: My plans for mod_python changes (260206).
On 05/03/2006, at 12:55 PM, Graham Dumpleton wrote:
Although changing the workflow to add new status values is
interesting, in reality
not sure this could be done as workflow is probably global across
all projects.
Actually, if want to get that pedantic and formalised, it does look
like one can select
an alternate workflow. The level of administration access we have
probably does
not allow us to change it, let alone see what options exist.
Anyway, digging around with the XML-RPC interface into JIRA, found
the list of
issue codes with descriptions at end of this email. You can see that
some other
project must have a customised work flow, don't know which project
though.
My purpose of playing with XML-RPC interface was to find a way of
closing off all
those 3.2.7 issues. The bulk actions mechanism in the web pages
doesn't seem
to allow a bulk closing of selected issues, only changing certain
information about
them. I'd rather not go through 60+ issues closing each one. :-(
{'id': '1', 'description': 'The issue is open and ready for the
assignee to start work on it.', 'name': 'Open', 'icon': 'http://
issues.apache.org/jira/images/icons/status_open.gif'}
{'id': '3', 'description': 'This issue is being actively worked on at
the moment by the assignee.', 'name': 'In Progress', 'icon': 'http://
issues.apache.org/jira/images/icons/status_inprogress.gif'}
{'id': '4', 'description': 'This issue was once resolved, but the
resolution was deemed incorrect. From here issues are either marked
assigned or resolved.', 'name': 'Reopened', 'icon': 'http://
issues.apache.org/jira/images/icons/status_reopened.gif'}
{'id': '5', 'description': 'A resolution has been taken, and it is
awaiting verification by reporter. From here issues are either
reopened, or are closed.', 'name': 'Resolved', 'icon': 'http://
issues.apache.org/jira/images/icons/status_resolved.gif'}
{'id': '6', 'description': 'The issue is considered finished, the
resolution is correct. Issues which are not closed can be reopened.',
'name': 'Closed', 'icon': 'http://issues.apache.org/jira/images/icons/
status_closed.gif'}
{'id': '1', 'description': 'The issue is open but work on it is
temporarily suspended until further feedback is received determining
its resolution.', 'name': 'On Hold', 'icon': 'http://
issues.apache.org/jira/images/icons/status_needinfo.gif'}
{'id': '10001', 'description': 'The issue was on hold, and feedback
was received indicating that further action are required towards its
resolution.', 'name': 'Continued', 'icon': 'http://issues.apache.org/
jira/images/icons/status_generic.gif'}
{'id': '10002', 'description': 'A patch for this issue has been
uploaded to JIRA by a contributor.', 'name': 'Patch Available',
'icon': 'http://issues.apache.org/jira/images/icons/
status_document.gif'}
{'id': '10003', 'description': 'A patch attached to this issue has
been reviewed by another contributor.', 'name': 'Patch Reviewed',
'icon': 'http://issues.apache.org/jira/images/icons/status_visible.gif'}
{'id': '10004', 'description': 'A patch attached to this issue has
been revised by the contributor.', 'name': 'Patch Revised', 'icon':
'http://issues.apache.org/jira/images/icons/status_information.gif'}
{'id': '10005', 'description': 'A patch attached to this issue is
ready to be committed.', 'name': 'Patch Finalized', 'icon': 'http://
issues.apache.org/jira/images/icons/status_up.gif'}
Re: Apache 2.2 failure on Mac OS X 10.4.5.
A bit more information. This may actually be a problem with the "ab"
tool
used for this test. I have actually seen the "ab" program give
problems on
Mac OS X before even with Apache 2.0.
On 06/03/2006, at 9:02 PM, Graham Dumpleton wrote:
Traceback (most recent call last):
File "test.py", line 2247, in test_global_lock
self.fail("global_lock is broken (too quick)")
File "/System/Library/Frameworks/Python.framework/Versions/2.3/
lib/python2.3/unittest.py", line 270, in fail
raise self.failureException, msg
AssertionError: global_lock is broken (too quick)
What the test does is to run "ab" to fire off five concurrent
requests. The
handler the request is calling acquires a global lock, sleeps for a
second
and then releases the lock. Because all requests need to acquire the
lock but are blocked out for a second, the execution of the five
requests
should therefore take five seconds because they will in effect be
serialised.
The problem is that "ab" is giving errors on 3 out of 5 of the requests:
Concurrency Level: 5
Time taken for tests: 3.514548 seconds
Complete requests: 5
Failed requests:0
Write errors: 3
Total transferred: 540 bytes
HTML transferred: 21 bytes
Requests per second:1.42 [#/sec] (mean)
Time per request: 3514.548 [ms] (mean)
Time per request: 702.910 [ms] (mean, across all concurrent
requests)
Transfer rate: 0.00 [Kbytes/sec] received
As example, when "ab" is run in verbose mode:
LOG: header received:
HTTP/1.1 200 OK^M
Date: Mon, 06 Mar 2006 10:28:42 GMT^M
Server: Apache/2.2.0 (Unix) mod_python/3.3.0-dev-20060305 Python/2.3.5^M
Connection: close^M
Content-Type: text/plain^M
^M
test ok
LOG: Response code = 200
Send request failed!
LOG: header received:
HTTP/1.1 200 OK^M
Date: Mon, 06 Mar 2006 10:28:42 GMT^M
Server: Apache/2.2.0 (Unix) mod_python/3.3.0-dev-20060305 Python/2.3.5^M
Connection: close^M
Content-Type: text/plain^M
^M
Note the "Send request failed!" message.
Although "ab" is giving the error, the requests are in fact received
by Apache
and handled. This time output is from prefork and not worker.
[Mon Mar 06 21:28:41 2006] [error] [client 127.0.0.1] global_lock 1
1141640921.61 0
[Mon Mar 06 21:28:41 2006] [error] [client 127.0.0.1] global_lock 2
1141640921.61 0
[Mon Mar 06 21:28:42 2006] [error] [client 127.0.0.1] global_lock 3
1141640922.61 0
[Mon Mar 06 21:28:42 2006] [error] [client 127.0.0.1] global_lock 4
1141640922.61 0
[Mon Mar 06 21:28:42 2006] [error] [client 127.0.0.1] global_lock 5
1141640922.61 0
[Mon Mar 06 21:28:43 2006] [notice] mod_python: (Re)importing module
'tests'
[Mon Mar 06 21:28:43 2006] [notice] mod_python: (Re)importing module
'tests'
[Mon Mar 06 21:28:43 2006] [error] [client 127.0.0.1] global_lock 1
1141640923.14 4
[Mon Mar 06 21:28:43 2006] [error] [client 127.0.0.1] global_lock 2
1141640923.14 4
[Mon Mar 06 21:28:43 2006] [notice] mod_python: (Re)importing module
'tests'
[Mon Mar 06 21:28:43 2006] [error] [client 127.0.0.1] global_lock 1
1141640923.22 1
[Mon Mar 06 21:28:43 2006] [notice] mod_python: (Re)importing module
'tests'
[Mon Mar 06 21:28:43 2006] [notice] mod_python: (Re)importing module
'tests'
[Mon Mar 06 21:28:43 2006] [error] [client 127.0.0.1] global_lock 1
1141640923.26 3
[Mon Mar 06 21:28:43 2006] [error] [client 127.0.0.1] global_lock 1
1141640923.26 2
[Mon Mar 06 21:28:43 2006] [error] [client 127.0.0.1] global_lock 1
1141640923.26 5
[Mon Mar 06 21:28:44 2006] [error] [client 127.0.0.1] global_lock 3
1141640924.14 4
[Mon Mar 06 21:28:44 2006] [error] [client 127.0.0.1] global_lock 4
1141640924.14 4
[Mon Mar 06 21:28:44 2006] [error] [client 127.0.0.1] global_lock 2
1141640924.14 1
[Mon Mar 06 21:28:44 2006] [error] [client 127.0.0.1] global_lock 5
1141640924.14 4
[Mon Mar 06 21:28:45 2006] [error] [client 127.0.0.1] global_lock 3
1141640925.14 1
[Mon Mar 06 21:28:45 2006] [error] [client 127.0.0.1] global_lock 4
1141640925.14 1
[Mon Mar 06 21:28:45 2006] [error] [client 127.0.0.1] global_lock 2
1141640925.14 3
[Mon Mar 06 21:28:45 2006] [error] [client 127.0.0.1] global_lock 5
1141640925.14 1
[Mon Mar 06 21:28:46 2006] [error] [client 127.0.0.1] global_lock 3
1141640926.14 3
[Mon Mar 06 21:28:46 2006] [error] [client 127.0.0.1] global_lock 4
1141640926.14 3
[Mon Mar 06 21:28:46 2006] [error] [client 127.0.0.1] global_lock 2
1141640926.14 2
[Mon Mar 06 21:28:46 2006] [error] [client 127.0.0.1] global_lock 5
1141640926.14 3
[Mon Mar 06 21:28:49 2006] [warn] child process 902 still did not
exit, sending a SIGTERM
[Mon Mar 06 21:28:51 2006] [warn] child process 902 still did not
exit, sending a SIGTERM
[Mon Mar 06 21:28:53 2006] [warn] child process 902 still did not
exit, sending a SIGTERM
[Mon Mar 06 21:28:55 2006] [erro
Re: Apache 2.2 failure on Mac OS X 10.4.5.
Sorry, last for the night. Time for me to go to sleep.
If I modify "ab" source code in Apache 2.2 to output error number
causing it to fail, get:
Send request failed 9!
The code where modification was done was:
if (e != APR_SUCCESS) {
/*
* Let's hope this traps EWOULDBLOCK too !
*/
if (!APR_STATUS_IS_EAGAIN(e)) {
epipe++;
printf("Send request failed %d!\n", e);
close_connection(c);
}
return;
}
On Mac OS X this equates to:
#define EBADF 9 /* Bad file descriptor */
Maybe when I am really bored I'll pursue further as to why.
Graham
On 06/03/2006, at 10:27 PM, Graham Dumpleton wrote:
Don't even need to rewrite test to use threads to fire off
requests. If
I hardwire test to use "ab" from Apache 1.3 or Apache 2.0, then it
works okay. It is only the version of "ab" in Apache 2.0 which causes
the test to fail.
Graham
On 06/03/2006, at 9:41 PM, Graham Dumpleton wrote:
A bit more information. This may actually be a problem with the
"ab" tool
used for this test. I have actually seen the "ab" program give
problems on
Mac OS X before even with Apache 2.0.
On 06/03/2006, at 9:02 PM, Graham Dumpleton wrote:
Traceback (most recent call last):
File "test.py", line 2247, in test_global_lock
self.fail("global_lock is broken (too quick)")
File "/System/Library/Frameworks/Python.framework/Versions/2.3/
lib/python2.3/unittest.py", line 270, in fail
raise self.failureException, msg
AssertionError: global_lock is broken (too quick)
What the test does is to run "ab" to fire off five concurrent
requests. The
handler the request is calling acquires a global lock, sleeps for
a second
and then releases the lock. Because all requests need to acquire the
lock but are blocked out for a second, the execution of the five
requests
should therefore take five seconds because they will in effect be
serialised.
The problem is that "ab" is giving errors on 3 out of 5 of the
requests:
Concurrency Level: 5
Time taken for tests: 3.514548 seconds
Complete requests: 5
Failed requests:0
Write errors: 3
Total transferred: 540 bytes
HTML transferred: 21 bytes
Requests per second:1.42 [#/sec] (mean)
Time per request: 3514.548 [ms] (mean)
Time per request: 702.910 [ms] (mean, across all concurrent
requests)
Transfer rate: 0.00 [Kbytes/sec] received
As example, when "ab" is run in verbose mode:
LOG: header received:
HTTP/1.1 200 OK^M
Date: Mon, 06 Mar 2006 10:28:42 GMT^M
Server: Apache/2.2.0 (Unix) mod_python/3.3.0-dev-20060305 Python/
2.3.5^M
Connection: close^M
Content-Type: text/plain^M
^M
test ok
LOG: Response code = 200
Send request failed!
LOG: header received:
HTTP/1.1 200 OK^M
Date: Mon, 06 Mar 2006 10:28:42 GMT^M
Server: Apache/2.2.0 (Unix) mod_python/3.3.0-dev-20060305 Python/
2.3.5^M
Connection: close^M
Content-Type: text/plain^M
^M
Note the "Send request failed!" message.
Although "ab" is giving the error, the requests are in fact
received by Apache
and handled. This time output is from prefork and not worker.
[Mon Mar 06 21:28:41 2006] [error] [client 127.0.0.1] global_lock
1 1141640921.61 0
[Mon Mar 06 21:28:41 2006] [error] [client 127.0.0.1] global_lock
2 1141640921.61 0
[Mon Mar 06 21:28:42 2006] [error] [client 127.0.0.1] global_lock
3 1141640922.61 0
[Mon Mar 06 21:28:42 2006] [error] [client 127.0.0.1] global_lock
4 1141640922.61 0
[Mon Mar 06 21:28:42 2006] [error] [client 127.0.0.1] global_lock
5 1141640922.61 0
[Mon Mar 06 21:28:43 2006] [notice] mod_python: (Re)importing
module 'tests'
[Mon Mar 06 21:28:43 2006] [notice] mod_python: (Re)importing
module 'tests'
[Mon Mar 06 21:28:43 2006] [error] [client 127.0.0.1] global_lock
1 1141640923.14 4
[Mon Mar 06 21:28:43 2006] [error] [client 127.0.0.1] global_lock
2 1141640923.14 4
[Mon Mar 06 21:28:43 2006] [notice] mod_python: (Re)importing
module 'tests'
[Mon Mar 06 21:28:43 2006] [error] [client 127.0.0.1] global_lock
1 1141640923.22 1
[Mon Mar 06 21:28:43 2006] [notice] mod_python: (Re)importing
module 'tests'
[Mon Mar 06 21:28:43 2006] [notice] mod_python: (Re)importing
module 'tests'
[Mon Mar 06 21:28:43 2006] [error] [client 127.0.0.1] global_lock
1 1141640923.26 3
[Mon Mar 06 21:28:43 2006] [error] [client 127.0.0.1] global_lock
1 1141640923.26 2
[Mon Mar 06 21:28:43 2006] [error] [client 127.0.0.1] global_lock
1 1141640923.26 5
[Mon Mar 06 21:28:44 2006] [error] [client 127.0.0.1] global_lock
3 1141640924.14 4
[Mon Mar 06 21:28:44 2006] [error] [client 127.0.0.1] global_lock
4 1141640924.14 4
[Mon Mar 06 21:28:44 2006] [error] [client 127.0.0.1
Changes made to apache.register_cleanup()
Nicolas
A while back you made the following change:
r378072 | nlehuen | 2006-02-16 06:41:25 +1100 (Thu, 16 Feb 2006) | 5
lines
- Fixed the unit tests for apache.register_cleanup
server.register_cleanup. Ther
e is not way it could have passed before, yet it did ???
- Corrected the documentation about those two functions, it was badly
broken.
- Added a warning so that users don't try to pass a request object as
the argume
nt to the callable.
You say that you don't understand how old test failed, but I don't
understand how the new test wouldn't fail.
The apache.register_cleanup() function is defined as:
def register_cleanup(handler,data=None):
_apache.register_cleanup(_interpreter,_server,handler,data)
Ie., the visible method takes two arguments. You changed the test to
call it with 4.
def apache_register_cleanup(req):
apache.register_cleanup(req.interpreter, req.server,
server_cleanup, "apache_register_cleanup test ok")
req.write("registered server cleanup that will write to log")
return apache.OK
It is only the hidden version of register_cleanup() in the _apache
module which takes 4.
As a result, the changes made to the documentation aren't correct
either.
I'll try and work out why the test harness doesn't fail with code as now
written, although it is probably all academic given that as described
in MODPYTHON-109, the server based register cleanup functions will not
reliably run anyway.
Most strange.
Graham
Re: Changes made to apache.register_cleanup()
Okay, the reason that the failing code wasn't detected was because
the test searched for the presence of a specific string in the
error log file. Problems was that that string appeared in the
Python exception generated by the incorrect code.
[Wed Mar 08 21:01:12 2006] [notice] mod_python: (Re)importing module
'tests'
[Wed Mar 08 21:01:13 2006] [error] [client 127.0.0.1] PythonHandler
tests::apache_register_cleanup: Traceback (most recent call last):
[Wed Mar 08 21:01:13 2006] [error] [client 127.0.0.1] PythonHandler
tests::apache_register_cleanup: File
"/System/Library/Frameworks/Python.framework/Versions/2.3/lib/
python2.3/site-packages/mod_python/apache.py", line 300, in
HandlerDispatch\nresult = object(req)
[Wed Mar 08 21:01:13 2006] [error] [client 127.0.0.1] PythonHandler
tests::apache_register_cleanup: File
"/Users/grahamd/Workspaces/mod_python-trunk-dev-5/test/htdocs/
tests.py", line 844, in apache_register_cleanup\n
apache.register_cleanup(req.interpreter, req.server, server_cleanup,
"apache_register_cleanup test ok")
[Wed Mar 08 21:01:13 2006] [error] [client 127.0.0.1] PythonHandler
tests::apache_register_cleanup: TypeError: register_cleanup() takes at
most 2 arguments (4 given)
The string in this case was:
apache_register_cleanup test ok
When you are restructuring the test harnesses, you might be mindful
as to whether there are other situations like this where the log
file is searched for a string. In these situations the string should
not perhaps appear in the actual line that is being tested. Thus instead
of:
def apache_register_cleanup(req):
apache.register_cleanup(req.interpreter, req.server,
server_cleanup, "apache_register_cleanup test ok")
req.write("registered server cleanup that will write to log")
return apache.OK
Use:
def apache_register_cleanup(req):
data = "apache_register_cleanup test ok"
apache.register_cleanup(req.interpreter, req.server,
server_cleanup, data)
req.write("registered server cleanup that will write to log")
return apache.OK
In this case, it would possibly also have been picked up if the response
content was still checked to see if it matched what was written. Ie.,
the
other string:
"registered server cleanup that will write to log
Anyway, I'll fix this all up as am going to do some stuff to fix at
least
the Py_Finalize() issue anyway, so working in related code.
Graham
On 08/03/2006, at 8:50 PM, Graham Dumpleton wrote:
Nicolas
A while back you made the following change:
r378072 | nlehuen | 2006-02-16 06:41:25 +1100 (Thu, 16 Feb 2006) | 5
lines
- Fixed the unit tests for apache.register_cleanup
server.register_cleanup. Ther
e is not way it could have passed before, yet it did ???
- Corrected the documentation about those two functions, it was badly
broken.
- Added a warning so that users don't try to pass a request object as
the argume
nt to the callable.
You say that you don't understand how old test failed, but I don't
understand how the new test wouldn't fail.
The apache.register_cleanup() function is defined as:
def register_cleanup(handler,data=None):
_apache.register_cleanup(_interpreter,_server,handler,data)
Ie., the visible method takes two arguments. You changed the test to
call it with 4.
def apache_register_cleanup(req):
apache.register_cleanup(req.interpreter, req.server,
server_cleanup, "apache_register_cleanup test ok")
req.write("registered server cleanup that will write to log")
return apache.OK
It is only the hidden version of register_cleanup() in the _apache
module which takes 4.
As a result, the changes made to the documentation aren't correct
either.
I'll try and work out why the test harness doesn't fail with code as
now
written, although it is probably all academic given that as described
in MODPYTHON-109, the server based register cleanup functions will not
reliably run anyway.
Most strange.
Graham
__auth__/__access_ methods in publisher
Jim Gallacher (JIRA) wrote .. > I think part of the problem with process_auth() is the uncertainty of meaning > associated with auth and __auth__. Does it mean authenticate or authorize? > If it's authorize, then there is no reason to call __auth__ with the password. > Likewise, you shouldn't need the user name when calling __access__. One > could certainly come up with access rules which are not tied to a specific > user. > > Maybe we should consider refactoring process_auth() to look for 2 new > attributes > such as __authen__ and __author__, and deprecate the use of __auth__. If > somebody wants to do authentication in publisher then their code would > be completely responsible for returning either a user or raising an error. > (Further discussion of this should be shifted out of this particular issue). Yes __auth__ is effectively equivalent to authenhandler, but __access__ is actually equivalent to authzhandler and not accesshandler as it suggests. In other words I see __access__ to be there to allow doing the equivalent of the "Require" directive where a user name would actually be required. Given the fragility of __auth__/__access__ functions nested inside of a function, I would suggest just leave it as is and perhaps steer people to using an actual acesshandler/authhandler/authzhandler instead. That is, promote the Apache way of doing things. This doesn't even mean that these steps need be done in Python as mod_auth or other authentication and authorisation modules written in other languages could just as easily be used. I'll have a lot more to say about this at a future date when I have finished some examples I am writing of authentication/authorisation handlers in mod_python implemented in the proper Apache way. This wasn't actually possible to do until I committed some of the changes I did in 3.3 trunk. Graham
Vote on whether to integrate server side include (SSI) support.
I have had patches for adding server side include support into mod_python ready for a while now. See: https://issues.apache.org/jira/browse/MODPYTHON-104 In short, it would add the ability to add Python code into files being served up through the INCLUDES output filter. More commonly this is known as server side include (SSI). For example: One could say that there is an overlap between this and the mod_python.psp handler, but the SSI feature is a builtin feature of Apache and it make sense to support it. Using SSI, if one was mad enough, you could even have Python and Perl code appearing in the one file. Anyway, the point of this email is to get a decision on whether this major new feature should or should not be added into mod_python. Core developer votes obviously matter the most, but others are more than welcome to voice an opinion. So, is it a Yes or a No? Graham
Re: [jira] Commented: (MODPYTHON-131) Make name of mutex directory configurable.
This bit is going to change anyway when I add the PythonOption mod_python.mutex_directory support. I have the changes ready, but I think I'll review them in the morning rather than committing now. I decide to do this stuff in 2 steps: 1. configure option 2. PythonOption mod_python.mutex_directory PythonOption mod_python.mutex_locks BTW, the commit seemed to miss newly generated src/include/mod_python.h and any notes added to Doc/appendixc.tex. Graham
Re: [jira] Assigned: (MODPYTHON-59) Add get_session() method to request object
I would rather we not go ahead with adding req.get_session() at this time. At least not how it was envisaged to be done previously. I'll come back with a bit of analysis after I review where we were up to previously. Graham On 12/03/2006, at 8:47 AM, Jim Gallacher (JIRA) wrote: [ http://issues.apache.org/jira/browse/MODPYTHON-59?page=all ] Jim Gallacher reassigned MODPYTHON-59: -- Assign To: Jim Gallacher Add get_session() method to request object -- Key: MODPYTHON-59 URL: http://issues.apache.org/jira/browse/MODPYTHON-59 Project: mod_python Type: New Feature Components: session Versions: 3.1.4, 3.1.3, 3.2.7 Environment: All Reporter: Jim Gallacher Assignee: Jim Gallacher Fix For: 3.3 Attachments: Session.py.diff.txt Users will get session instances by calling req.get_session(). If a session already exists it will be returned, otherwise a new session instance will be created. Session configuration will be handled using apache directives rather than within their code. Using this scheme means only one session instance will be created per request, which will eliminate the deadlock problems many people experience. Also, using this scheme makes it possible for sessions to be properly handled within psp pages and across req.internal_redirect() calls. Code will be commited to svn shortly. -- This message is automatically generated by JIRA. - If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa - For more information on JIRA, see: http://www.atlassian.com/software/jira
Re: [jira] Resolved: (MODPYTHON-131) Make name of mutex directory configurable.
In the area of code where changes for this was made, there is: char fname[255]; /* XXX What happens if len(mutex_dir) > 255 - len(mpmtx%d%d)? */ snprintf(fname, 255, "%s/mpmtx%d%d", mutex_dir, glb->parent_pid, n); The value 255 should really be MAXPATHLEN macro. On UNIX platforms, it is often 1024 or 4096 from memory. This is defined in system header file on UNIX systems. Two places in src/mod_python.c where this occurs. This would at least be a bit safer than 255. Graham On 12/03/2006, at 8:45 AM, Jim Gallacher (JIRA) wrote: [ http://issues.apache.org/jira/browse/MODPYTHON-131?page=all ] Jim Gallacher resolved MODPYTHON-131: - Fix Version: 3.3 Resolution: Fixed Make name of mutex directory configurable. -- Key: MODPYTHON-131 URL: http://issues.apache.org/jira/browse/MODPYTHON-131 Project: mod_python Type: Improvement Components: core Versions: 3.2.7 Reporter: Graham Dumpleton Assignee: Jim Gallacher Fix For: 3.3 Creating an issue for this so it can be tracked. Been pointed out in: http://www.mail-archive.com/python-dev@httpd.apache.org/msg01271.html that on Mandriva Linux, that is is necessary to manually change the mutex directory in mod_python.c source code. Area of code is: #if !defined(OS2) && !defined(WIN32) && !defined(BEOS) && !defined(NETWARE) char fname[255]; snprintf(fname, 255, "/tmp/mpmtx%d%d", glb->parent_pid, n); #else char *fname = NULL; #endif There should be an option to configure program to allow this to be more easily changed. Way of changing the value through Apache configuration to also be investigated further. -- This message is automatically generated by JIRA. - If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa - For more information on JIRA, see: http://www.atlassian.com/software/jira
Re: Vote on whether to integrate server side include (SSI) support.
Not seeing any negatives, I am going to go ahead and commit the SSI stuff. Comments that this is just another way to skin a cat are true, even if a small cat. I guess the reason for doing it is to fill out those basic features that can be filled out by using just what Apache provides. Anyway, I'll try and get around to writing one of my mini articles about the feature and also use some of that for inclusion into the mod_python documentation itself. Maybe from writing a mini article it might become more apparent how its simplicity can be just as useful as any other approach for getting small tasks down quickly. Graham On 11/03/2006, at 1:43 AM, Gregory (Grisha) Trubetskoy wrote: I don't understand this enough to have an opinion on it, seems like another way to skin a cat, but to really form an opinion would require some thinking on my part at least (may be I'm getting thick with age). I think it'd be great if those who send in +1's (or -1's) would explain why they think this is good, and even if it's not so useful, then is it worth being supported and maintained in the future. Grisha On Fri, 10 Mar 2006, Jim Gallacher wrote: +1 Graham Dumpleton wrote: I have had patches for adding server side include support into mod_python ready for a while now. See: https://issues.apache.org/jira/browse/MODPYTHON-104 In short, it would add the ability to add Python code into files being served up through the INCLUDES output filter. More commonly this is known as server side include (SSI). For example: One could say that there is an overlap between this and the mod_python.psp handler, but the SSI feature is a builtin feature of Apache and it make sense to support it. Using SSI, if one was mad enough, you could even have Python and Perl code appearing in the one file. Anyway, the point of this email is to get a decision on whether this major new feature should or should not be added into mod_python. Core developer votes obviously matter the most, but others are more than welcome to voice an opinion. So, is it a Yes or a No? Graham
Re: Vote on whether to integrate server side include (SSI) support.
On 12/03/2006, at 8:25 PM, André Malo wrote: * Graham Dumpleton wrote: Not seeing any negatives, I am going to go ahead and commit the SSI stuff. Comments that this is just another way to skin a cat are true, even if a small cat. I guess the reason for doing it is to fill out those basic features that can be filled out by using just what Apache provides. If that's a point, you know, what would be really great in this case? To be able to register own SSI handlers using mod_python instead of (or in addition to) this generic #python thingy, which nobody really seems to be able to classify/justify. Like registering a name and a callback function with a fixed signature. What do you think? I did think about it but deferred the idea in preference to at least getting some basic support for use of Python code with SSI in place. In other words, to take this initial step and then see whether people even consider using server side includes a viable way of doing stuff. These days there is so much focus on full blown frameworks that I am not sure that SSI even gets used much. I could well be wrong on that point. Do you have examples of SSI tag handlers that you might implement this way if such a feature were available? I ask as it all good to speculate on such a feature, but like this generic "#python" tag, would it in itself be used either? Graham
Re: Vote on whether to integrate server side include (SSI) support.
On 12/03/2006, at 9:04 PM, Graham Dumpleton wrote: On 12/03/2006, at 8:25 PM, André Malo wrote: * Graham Dumpleton wrote: Not seeing any negatives, I am going to go ahead and commit the SSI stuff. Comments that this is just another way to skin a cat are true, even if a small cat. I guess the reason for doing it is to fill out those basic features that can be filled out by using just what Apache provides. If that's a point, you know, what would be really great in this case? To be able to register own SSI handlers using mod_python instead of (or in addition to) this generic #python thingy, which nobody really seems to be able to classify/justify. Like registering a name and a callback function with a fixed signature. What do you think? I did think about it but deferred the idea in preference to at least getting some basic support for use of Python code with SSI in place. In other words, to take this initial step and then see whether people even consider using server side includes a viable way of doing stuff. These days there is so much focus on full blown frameworks that I am not sure that SSI even gets used much. I could well be wrong on that point. Do you have examples of SSI tag handlers that you might implement this way if such a feature were available? I ask as it all good to speculate on such a feature, but like this generic "#python" tag, would it in itself be used either? BTW, I should point out that one of the reasons for not rushing straight into allowing tag handler registration is that the same can be achieved with what is provided, albeit just more verbose and still exposing the fact that Python is used. All you need to do is put your common code that the tag handler would do in a Python function, import that module and call it. That is, you don't have to enumerate the complete code within the actual file being processed by mod_includes. The function can either query filter.req.subprocess_env for input parameters, or the point in the file where it is called can extract out the required values and pass them as explicit arguments to the function being called. All you thus get by having an explicit registered tag is that that Python is used can be hidden and a user would be none the wiser. Graham
get_session(), req.session, req.form and MODPYTHON-38
That get_session() was being added at the C code level was also one of the things that worried me about this change. Thus in part why my suggestion for an alternate approach didn't touch the request object at all. While we are talking about this issue of additions to the request object, the documentation says: The request object is a Python mapping to the Apache request_rec structure. When a handler is invoked, it is always passed a single argument - the request object. You can dynamically assign attributes to it as a way to communicate between handlers. The bit I want to highlight is the last line about handlers dynamically assigning attributes to the request object. Being able to do this is quite useful and is already used within mod_python.publisher. Specifically, it assigns the instance of the FieldStorage object to req.form. This can then be accessed by the published function. As I describe in: https://issues.apache.org/jira/browse/MODPYTHON-38 I want to formalise a couple of current conventions regarding attributes being assigned to the request object by handlers. I have mentioned req.form. The other I want to formalise is req.session. Thus I want a documented convention that if a handler is going to use util.FieldStorage, that it should before doing so, first check whether an existing instance resides as req.form and use that instead. Similarly, if a handler is going to create a Session object, that it look for an existing instance as req.session and again use that instead. Both mod_python.psp and mod_python.publisher would be modified to follow the convention, which would then avoid the problems which sometimes come up when people try and use the two together. Ie., both trying to parse form arguments, or both trying to create session objects and locking each other out. Is there support for doing this? If so I'll up it on my priority list. Note, this isn't addressing some of what the get_session() changes were intended to address, specifically issues of internal redirects, but I think it is a good start to at least address this, with any final solution building around this convention. Graham On 13/03/2006, at 2:39 PM, Gregory (Grisha) Trubetskoy wrote: I'm -1 on get_session() too. The request object is supposed to be a representation of Apache's request, and get_session() just does not belong there. Grisha On Sun, 12 Mar 2006, Jim Gallacher wrote: I handn't really intended to start working on an implementation. I just don't like seeing all those issues in JIRA marked as unassigned. :) Since I created it I figured I should take some responsibility for it. Plus, it's a gentle reminder when I list my assigned issues - resolve it one way or another. I still think we need some sort of solution to the problem of people trying to create 2 session instances in the same request, but I agree that the original concept of req.get_session() was not quite right. Jim Graham Dumpleton wrote: I would rather we not go ahead with adding req.get_session() at this time. At least not how it was envisaged to be done previously. I'll come back with a bit of analysis after I review where we were up to previously. Graham On 12/03/2006, at 8:47 AM, Jim Gallacher (JIRA) wrote: [ http://issues.apache.org/jira/browse/MODPYTHON-59?page=all ] Jim Gallacher reassigned MODPYTHON-59: -- Assign To: Jim Gallacher Add get_session() method to request object -- Key: MODPYTHON-59 URL: http://issues.apache.org/jira/browse/MODPYTHON-59 Project: mod_python Type: New Feature Components: session Versions: 3.1.4, 3.1.3, 3.2.7 Environment: All Reporter: Jim Gallacher Assignee: Jim Gallacher Fix For: 3.3 Attachments: Session.py.diff.txt Users will get session instances by calling req.get_session(). If a session already exists it will be returned, otherwise a new session instance will be created. Session configuration will be handled using apache directives rather than within their code. Using this scheme means only one session instance will be created per request, which will eliminate the deadlock problems many people experience. Also, using this scheme makes it possible for sessions to be properly handled within psp pages and across req.internal_redirect() calls. Code will be commited to svn shortly. -- This message is automatically generated by JIRA. - If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa - For more information on JIRA, see: http://www.atlassian.com/software/jira
Re: get_session(), req.session, req.form and MODPYTHON-38
Grisha wrote .. > > On Mon, 13 Mar 2006, Graham Dumpleton wrote: > > > Thus I want a documented convention that if a handler is going to use > > util.FieldStorage, that it should before doing so, first check whether > > an existing instance resides as req.form and use that instead. > > I'm not sure if this is a good example - req.form is something specific > to > the publisher. Rather than perhaps documenting it as you suggest, > util.FieldStorage can take it upon itself to create a req.form so that > subsequent attempts to instantiate it just return req.form. (This is just > an example, I'm not 100% sure that I having FS do this makes sense - seems > like a good idea). > > > Similarly, if a handler is going to create a Session object, that it > > look for an existing instance as req.session and again use that instead. > > OR, the Session module would know to look for a req.session, in which case > the handlers wouldn't need to worry about it. > > (One thing to watch out for would be that mutliple concurrent sessions > in the same request is a possibility) Hmmm, having a look at the code, at some point the check for req.session got added and I didn't realise or forgot that it had been done. # does this code use session? session = None if "session" in code.co_names: if hasattr(req, 'session'): session = req.session else: session = Session.Session(req) It didn't get added for form though, which means that accessing form arguments from within a PSP page will mean only those in the query string of the URL will be available as the content of the request has already been consumed. Looks like a audit of both: https://issues.apache.org/jira/browse/MODPYTHON-38 https://issues.apache.org/jira/browse/MODPYTHON-59 need to be done to work out what has and hasn't been done related to this so we know where we are up to. Looks a bit like when the req.get_session() changes got rolled back that it got introduced at that point: http://svn.apache.org/viewcvs.cgi//httpd/mod_python/trunk/lib/python/mod_python/psp.py?rev=226320&view=diff&r1=226320&r2=226319&p1=/httpd/mod_python/trunk/lib/python/mod_python/psp.py&p2=/httpd/mod_python/trunk/lib/python/mod_python/psp.py Before the req.get_session() change it didn't exist: http://svn.apache.org/viewcvs.cgi//httpd/mod_python/trunk/lib/python/mod_python/psp.py?rev=191745&view=diff&r1=191745&r2=191744&p1=/httpd/mod_python/trunk/lib/python/mod_python/psp.py&p2=/httpd/mod_python/trunk/lib/python/mod_python/psp.py I can't remember if this was a conscious decision to check for req.session based on suggestions or otherwise. Graham
Re: get_session(), req.session, req.form and MODPYTHON-38
Jim Gallacher wrote .. > The idea of something like req.get_session() is to give users an obvious > way to grab a session object without the deadlock concerns. How many > times have we seen this kind of problem-code on the mailing list? > > def index(req): > sess = Session.Session(req) > do_stuff(req) > ... > > def do_stuff(req): > sess = Session.Session(req) > ... do other stuff. > > Having the session constructor check for the existence of req.session is > of no use here. We need a way to make sure only *one* session instance > is created per request. (Bonus marks for making it work with internal > redirect). FWIW, I use the following in a class based authenhandler. thread = threading.currentThread() if self.__cache.has_key(thread): req.session = self.__cache[thread] else: req.session = Session.Session(req) self.__cache[thread] = req.session def cleanup(data): del self.__cache[thread] req.register_cleanup(cleanup) In short, store it in a cache outside of the request object keys by the thread ID. Works for both internal redirects and also for fast redirects as used by DirectoryIndex matching algorithm, although for the latter there are other issues as I have documented in MODPYTHON-146. My thinking keeps changing a bit, but overall I have been leaning towards not having a get_session() like function explicitly provided and instead documenting how to correctly write a authenhandler which can handle form based login with sessions. Ie., use Apache phases properly rather than pushing authentication into content handler phase as most do. Unfortunately, I keep finding things in mod_python that need to be improved or fixed to avoiding having to fudge things, thus haven't presented my code for others to look at yet. :-( Graham
Re: get_session(), req.session, req.form and MODPYTHON-38
Jim Gallacher wrote .. > Which is all good, but you are assuming that people are only using > sessions for authentication purposes. Consider a shopping cart > implemented as session: the user may not be authenticated until *after* > they have filled their cart and are ready to checkout. Perhaps the cache > code would be better off in Session.py? In the case where a session is required across both public and private areas of a web site where login is required for the private area, the example I am working on handles that. I am still working out what presents as being the more sensible way, but Apache configuration would be something like: AuthType Session::Private Require valid-user PythonAuthenHandler _sessionmanager AuthType Session::Public or: AuthType Session AuthName "Private Area" Require valid-user PythonAuthenHandler _sessionmanager AuthName "Public Area" The benefit of using an authenhandler in this case is that it doesn't particularly matter what is being used for the content handler phase as longs as access to resources can be controlled based on filename matched by Apache for the URL, or possibly by the URL itself. Thus, you could be using publisher or PSP, or even a mix. Whatever is used, it just uses the req.session object created for it by the authenhandler. Sure the code might not handle every single possible use case, but its purpose was an example, not something to be embodied into any package. Thus, it could by all means be customised. The important thing is illustrates how to do all the hard stuff that people don't generally get correct. Graham
Re: Cross-platform query: _FILE_OFFSET_BITS in python and httpd
On 15/03/2006, at 8:45 AM, Gregory (Grisha) Trubetskoy wrote: Could folks with access to different OS's try the following: Compare output of "apxs -q CPPFLAGS" with the value of _FILE_OFFSET_BITS in pyconfig.h. For example, on my Fedora Core 4 i386 system (stock httpd and python): $ /usr/sbin/apxs -q CPPFLAGS -DSSL_EXPERIMENTAL_ENGINE [note - no mention of _FILE_OFFSET_BITS above] $ grep _FILE_OFFSET_BITS /usr/include/python2.4/pyconfig.h #define _FILE_OFFSET_BITS 64 Apache 2.0.55/Python 2.3/Mac OS X 10.3.9 ~ [507]$ /usr/local/apache-2.0/bin/apxs -q CPPFLAGS ~ [508]$ grep _FILE_OFFSET_BITS /System/Library/Frameworks/Python.framework/Versions/Current/include/ python2.3/pyconfig.h /* #undef _FILE_OFFSET_BITS */
Re: Vote on whether to integrate server side include (SSI) support.
On 14/03/2006, at 7:33 PM, André Malo wrote:
* Graham Dumpleton <[EMAIL PROTECTED]> wrote:
Do you have examples of SSI tag handlers that you might implement
this
way if such a feature were available? I ask as it all good to
speculate
on such a feature, but like this generic "#python" tag, would it in
itself be used either?
Actually, I'd hardly use the the #python tag by itself, but specific
functions, provided by my application as needed (like the SSI templates
are provided by my application but designed by someone else!).
All you thus get by having an explicit registered tag is that that
Python
is used can be hidden and a user would be none the wiser.
Exactly that's the point. Separation of concerns - I'm not a friend of
a
raw programming language in a template.
Therefore - if you really want to pass httpd features to mod_python, it
would be nice to consider this one :)
I would agree that excessive raw programming language code in a template
is not good either. I however don't see minimal callouts, which
effectively what a custom tag is doing anyway, as that bad.
What I am about to describe can only be done using a backdoor with the
current SSI support. This is the case because of bug described in:
https://issues.apache.org/jira/browse/MODPYTHON-146
which means it will not work where DirectoryIndex is matching files.
When that is fixed, a notionally private variable reference just needs
to be made part of the public API and this will all work. You could use
the notionally private variable name now if you wanted to play, it just
will not work on DirectoryIndex matched files. In other words, have
simply hidden fact this can be done until that other issue is fixed.
First off, in my .htaccess file I have:
PythonFixupHandler _dispatch
I have not turned on INCLUDES output filter in the configuration as I
do it dynamically, but you could do it in the configuration instead.
My fixup handler then contains:
from mod_python import apache
def header():
return "HEADER"
def footer():
return "FOOTER"
def other(filter):
print >> filter, "OTHER"
def fixuphandler(req):
if req.content_type == "text/html":
# Enable INCLUDES output filter for request.
req.add_output_filter("INCLUDES")
# Setup globals visible to INCLUDES files.
req.ssi_globals = {}
req.ssi_globals["header"] = header
req.ssi_globals["footer"] = footer
req.ssi_globals["other"] = other
return apache.OK
In my HTML file being processed by INCLUDES output filter, I can then
use:
TEXT
TEXT
What I am doing is using the fixup handler to enabled the INCLUDES
output filter based on type of file matched by request, but am also
creating a dictionary object preloaded with data accessible to the
code executing within context of file being processed by INCLUDES
output filter. This dictionary actual becomes the globals of the code.
That is, equivalent to saying:
eval "header()" in req.ssi_globals
Thus one can avoid having a whole lot of excessive code in the HTML
to perform imports, setup data etc. This can all be done in the
fixup handler for all files in the directory to be processed by the
INCLUDES output filter. The result is that the HTML is quite simple
and not really that much more complicated than having custom tags.
In respect of customised tag handlers, this is more flexible, as a
customised tag handler resides in the global Apache server namespace.
That is, it is accessible to everything. With the above mechanism, what
is available and callable can be specific to a particular part of the
URL namespace, specifically, whatever context the fixup handler is
specified for. With a bit of extra work, you could even allow fixup
handlers specific to each resource and allow file level customisations
as well as directory level.
That customised tag handlers are in the global namespace presents
other issues as well. The first is that they have to be registered at
the time that mod_python is being initialised. To do this, custom
directives would need to be created and they must be used only in
main sever configuration context. That is, outside of all VirtualHost,
Location, Directory or Files directive containers. As mentioned before,
means they cant apply to a specific context and a user who only has
access to a .htaccess file cant define them.
The next issue would be coming up with a simple API on the Python side
for processing the callback when a registered tag were called. Doing
this in C code is quite fiddly and not sure how one could present a nice
API to do the same if you wanted to have the same abilities for
processing tag arguments and values etc.
What might be a better approach is to harness some other changes I have
in mind for mod_python down the track. That i
Bug in Apache ap_internal_fast_redirect() function????
I know this is the wrong list to be asking this, but thought I
would ask before I go and get my self subscribed to some Apache
server list just to ask the question as I know some more involved
in Apache core lurk here. :-)
I have been looking at a way of solving:
https://issues.apache.org/jira/browse/MODPYTHON-146
The basic problem described in this issue is that when DirectoryIndex
directive is used, data added to the Python request object by a fixup
handler for the actual file doesn't get propogated back into the main
request object pertaining to the request against the directory. This
is because mod_dir uses ap_internal_fast_redirect() to do some of the
nasty work and Python stuff isn't part of the request_rec so does not
get copied from subrequest request_rec to main request_rec.
In investigating this, am starting to question whether what the function
ap_internal_fast_redirect() is doing is even sensible in some parts
anyway. Specifically, it does:
r->headers_out = apr_table_overlay(r->pool, rr->headers_out,
r->headers_out);
r->err_headers_out = apr_table_overlay(r->pool, rr->err_headers_out,
r->err_headers_out);
r->subprocess_env = apr_table_overlay(r->pool, rr->subprocess_env,
r->subprocess_env);
In this code "r" is the main request_rec and "rr" is that for the sub
request which matched actual file in DirectoryIndex directive file list.
The problem here is that apr_table_overlay() merges the contents of two
Apache tables. What this means is that if the same key exists in both
tables, that from "rr" doesn't replace that in "r", instead the result
contains both key/value pairs even if they have the same value for that
key.
Now I tend to work out my home directory, thus mod_userdir comes into
play. One of the things it does is add an entry to req.notes recording
what my username is. This is just one example, there are others entries
added as well by other stuff. For example, if I access
/~grahamd/index.html
the req.notes table contains:
{'no-etag': '', 'ap-mime-exceptions-list': 'Ϣ', 'mod_userdir_user':
'grahamd', 'python_init_ran': '1'}
I could therefore access the username using:
req.notes["mod_userdir_user"]
this would yield a string containing "grahamd".
If now I access the directory itself as /~grahamd/ and rely on
DirectoryIndex
directive to map it to index.html file, I get:
{'no-etag': '', 'mod_userdir_user': 'grahamd', 'python_init_ran': '1',
'ap-mime-exceptions-list': 'œ\', 'mod_userdir_user': 'grahamd'}
Note how there are two entries for "mod_userdir_user". When I try and
get
the username now, I will actually get an array:
['grahamd', 'grahamd']
If my handler had assumed that that was always going to be a string, it
would promptly generate some sort of exception when used in a way that
wasn't going to work for an array.
The question I guess is whether ap_internal_fast_redirect() is wrong for
merging the tables, or whether any handler (including those not written
in mod_python) are supposed to handle this strangeness?
I have only talked about req.notes here, but same issue applies to the
req.headers_out, req.err_headers_out and req.subprocess_env table
objects,
as the same issue can come up with them as well.
Graham
Re: mod-python error
Following may or may not be relevant.
BTW, have you tried your sample code outside of context of mod_python.
Ie., extract out main bit and run it as command line script.
Patty wrote ..
> This is the piece of code:
>
> # Connect to the database
> conn = MySQLdb.connect(host = "localhost", user = "root", passwd = "",
> db ="ants_control_center")
> cursor = conn.cursor()
>
>
> def ask(req, host, target):
>
> gtarget = "%s" % target
> ghost = "%s" % host
> cursor.execute(""" #This is line 18
> SELECT date FROM targets WHERE target_name = %s""",(gtarget))
In Python, saying:
(gtarget)
is the same as:
gtarget
That is, a tuple will only be created for a single item if written as:
(gtarget,)
Thus, if cursor.execute() expects a tuple, use latter form.
> result = cursor.fetchone()
> conn.commit()
>
> if ((gtarget == 'bl') | (gtarget == 'bs') | (gtarget == 'nl') |
> (gtarget == 'bx')):
Traditionally one uses "or" and not "|" for logical or. The "|" operator is
bitwise or and has special meaning when arguments are integers. It
might work like "or" for booleans, but best to use "or" as that is what
most people will be used to.
> cursor.execute("UPDATE targets SET date = %s WHERE target_name
> = %s",
>(datetime.date.today(),gtarget))
> conn.commit()
> return compare_hosts(ghost,gtarget)
>
> # This one takes care of the rest
> if (datetime.date.today() == result[0]):
> return compare_hosts(ghost,gtarget)
> else:
> return set_host_percentage(ghost,gtarget)
>
> ***
>
> This is the error I'm getting:
>
>
> Mod_python error: "PythonHandler mod_python.publisher"
>
> Traceback (most recent call last):
>
> File "/usr/lib/python2.3/site-packages/mod_python/apache.py", line 299,
> in
> HandlerDispatch
> result = object(req)
>
> File "/usr/lib/python2.3/site-packages/mod_python/publisher.py", line
> 136, in
> handler
> result = util.apply_fs_data(object, req.form, req=req)
>
> File "/usr/lib/python2.3/site-packages/mod_python/util.py", line 361,
> in
> apply_fs_data
> return object(**args)
>
> File "/var/www/html/mptest/ask.py", line 18, in ask
> cursor.execute("""
>
> File "/usr/lib/python2.3/site-packages/MySQLdb/cursors.py", line 137,
> in execute
> self.errorhandler(self, exc, value)
>
> File "/usr/lib/python2.3/site-packages/MySQLdb/connections.py", line
> 33, in
> defaulterrorhandler
> raise errorclass, errorvalue
>
> AttributeError: 'NoneType' object has no attribute 'literal'
>
>
Re: [jira] Resolved: (MODPYTHON-118) Allow PythonImport to optionally call function in module.
Note that the changes committed for this created a new file:
test/htdocs/dummymodule.py
Previously, this was created on demand by test/test.py.
Because the "clean" target doesn't remove this old version
created by test/test.py, when you do a "svn update" it will
complain the file already exists as a non versioned file.
You will need to remove the file before doing "svn update".
Graham
On 17/03/2006, at 3:17 PM, Graham Dumpleton (JIRA) wrote:
[ http://issues.apache.org/jira/browse/MODPYTHON-118?page=all ]
Graham Dumpleton resolved MODPYTHON-118:
Fix Version: 3.3
Resolution: Fixed
Allow PythonImport to optionally call function in module.
-
Key: MODPYTHON-118
URL: http://issues.apache.org/jira/browse/MODPYTHON-118
Project: mod_python
Type: Wish
Components: core
Versions: 3.3
Reporter: Graham Dumpleton
Assignee: Graham Dumpleton
Fix For: 3.3
PythonImport can currently be used to specify that a module be
imported into a named interpreter at the time that an Apache child
process is initiated. Because all it does is import the module, if
any specific action needs to be triggered, it has to be done as a
side effect of the module import.
Triggering actions as a side effect of a module import is
generally not a good idea as failure of the side effect action
will cause the import of the module itself to fail if the code
doesn't properly handle this situation. It is generally preferable
to import the module and when that has suceeded only then call a
specific function contained in the module to initiate the action.
Thus proposed that PythonImport be able to take an optional
function to be called upon successful import of the name module.
The syntax would be like that for Python*Handler directives.
PythonImport mymodule::myfunc myinterpreter
This would have the effect of loading module "mymodule" in the
interpreter called "myinterpreter" and then calling
"mymodule.myfunc()". No arguments would be supplied to the
function when called.
Another benefit of this feature would be that it would allow a
single module to be able to contain a number of special
initialisation functions that might be triggerable. The user could
selectively call those that might be required.
PythonImport mymodule::enable_caching myinterpreter
PythonImport mymodule::disable_logging myinterpreter
At the moment to do that, a distinct module would need to be
created for each where the only thing in the module is the call of
the function.
Note that in using something similar to mod_python option/config
values, am talking here about options that must be able to only be
enabled/disable in one spot. The problem with mod_python option/
config values in Apache is that different parts of the document
tree can set them to different values, which for some things is
actually a problem, such as the case with PythonAutoReload.
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the
administrators:
http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see:
http://www.atlassian.com/software/jira
New module importer. Was: Re: mod_python roadmap
On 14/03/2006, at 12:23 PM, Jim Gallacher wrote: I find I work more effectively when I have deadlines to worry about (being a procrastinator by nature), so I thought I'd propose the following roadmap. Mar 20: 3.3-dev - snapshot for testing Apr 1: 3.2.9 - bugfix release May 1: 3.3-dev - snapshot for testing Jun 15: 3.3-dev - snapshot for testing Jul 15: 3.3 - feature freeze Aug 1: 3.3.0 - first 3.3 beta - branches/3.3.x created - work on trunk resumes - beta cycle proceeds independent of dev work Sep 15: 3.3.y - 3.3 final released (hopefully) For the development snapshots I'd just roll a tarball from trunk and make a call to the community for testing help. Hopefully we'll catch new bugs and regressions early so that the actual beta cycle will be much shorter. There would be *no* freeze during the snapshot tests. Work on trunk can continue while we wait for the test feedback. With the plan being to roll a tar ball on the 20th March, do people want me to incorporate the new module importer or not, such that it will be included in this snapshot and be available for testing? For background on the new importer see: https://issues.apache.org/jira/browse/MODPYTHON-143 and follow links given there to articles I have written or started writing and all the JIRA issues. The code for this is all ready, it just needs to be committed into the subversion repository. Note that just because the code would be part of the source code does not mean it will be used. Specifically, the code has been set up at the moment so the existing importer will still be used unless you explicitly configure mod_python to use the new importer. If you want to try the new module importer, you will be able to enable it for all Python interpreter instances created, or selected ones. Only after sufficient testing and tweaking as necessary, and after it has been deemed an acceptable solution would it be properly integrated into mod_python as the default. If people feel it isn't acceptable, it would be stripped out of code and someone else can have a go with coming up with a better alternative. Graham
Re: cookies generation by session, patch
Now can you explain why one would want to do this?
Unless you provide some justification of why it is necessary it is
less likely
to be accepted as although the reasons may be obvious to you, it may not
be to us. There also may be better ways of achieving the same end.
Also, describe why this would be better than simply deleting the cookie
that is being created from the outgoing headers.
del req.headers_out["Set-Cookie"]
Graham
On 21/03/2006, at 7:39 PM, Stanislav Ershov wrote:
Hi,
I wrote a simple patch for 'Session.py'. Patch adds possibility to
disable cookies generation by session. And it's optional.
By default cookies generation enabled.
Add Apache directive 'Python Option sessin_cookie_generation 0' for
disabling.
--- mod_python-3.2.8.orig/lib/python/mod_python/Session.py Mon Feb
20 00:51:18 2006
+++ mod_python-3.2.8/lib/python/mod_python/Session.py Tue Mar 21
09:50:46 2006
@@ -138,17 +138,19 @@
dict.__init__(self)
session_cookie_name = req.get_options().get
("session_cookie_name",COOKIE_NAME)
+session_cookie_generation = int(req.get_options().get
("session_cookie_generation",1))
if not self._sid:
-# check to see if cookie exists
-if secret:
-cookies = Cookie.get_cookies(req,
Class=Cookie.SignedCookie,
- secret=self._secret)
-else:
-cookies = Cookie.get_cookies(req)
+if session_cookie_generation:
+# check to see if cookie exists
+if secret:
+cookies = Cookie.get_cookies(req,
Class=Cookie.SignedCookie,
+ secret=self._secret)
+ else:
+cookies = Cookie.get_cookies(req)
-if cookies.has_key(session_cookie_name):
-self._sid = cookies[session_cookie_name].value
+if cookies.has_key(session_cookie_name):
+self._sid = cookies[session_cookie_name].value
if self._sid:
# Validate the sid *before* locking the session
@@ -171,7 +173,8 @@
if self._sid: self.unlock() # unlock old sid
self._sid = _new_sid(self._req)
self.lock() # lock new sid
-Cookie.add_cookie(self._req, self.make_cookie())
+if session_cookie_generation:
+Cookie.add_cookie(self._req, self.make_cookie())
self._created = time.time()
if timeout:
self._timeout = timeout
Re: mod_python directory index error
Firat KUCUK wrote .. > Hi, > > i have a little problem about Directory Index. > > this is our .htaccess file: > > Allow from All > > AddHandler mod_python .py > PythonHandlerwepy.handler > PythonDebug On > > DirectoryIndex index.htm index.html index.php index.py index.pl > > wepy is a new PHP like web python library. > > http://www.wepy.org/ > > we just type http://blablabla.com/wepy/ > > and our main file is index.py > > but req.path_info is None > > so in mod_python/apache.py/build_cgi_env function: > > *if* req.path_info *and* len(req.path_info) > 0: > env[*"SCRIPT_NAME"*] = req.uri[:-len(req.path_info)] > *else*: > env[*"SCRIPT_NAME"*] = req.uri > > > i think should be like this. > > if req.path_info: > env["SCRIPT_NAME"] = req.uri[:-len(req.path_info)] > else: > env["SCRIPT_NAME"] = req.uri What is the actual problem you are trying to solve? The "len(req.path_info) > 0" is actually redundant because when req.path_info is a string and has length 0, the "req.path_info" boolean check will fail anyway. In other words, the change you made wouldn't make any difference that I can see to the actual outcome. Is the redundancy all you were wanting to point out??? BTW, you should be careful about what SCRIPT_NAME gets set to by Apache and by this code. See discussion of strange things that happen at: https://issues.apache.org/jira/browse/MODPYTHON-68 Graham
Re: cookies generation by session, patch
Now that I have some time, I'll explain why I want your reasoning. I
didn't have the time when I sent original email.
The only reason I can think of for Session not to generate a cookie is
because the SID is being extracted from the URL or is being passed by
some mechanism other than as a cookie.
In this case the SID would need to be supplied explicitly when the
Session object is being created:
session = Session(req, sid=value)
When a SID is supplied in this way, the Session object does not attempt
to parse any cookies to get it.
if not self._sid:
# check to see if cookie exists
if secret:
cookies = Cookie.get_cookies(req, Class=Cookie.SignedCookie,
secret=self._secret)
else:
cookies = Cookie.get_cookies(req)
if cookies.has_key(session_cookie_name):
self._sid = cookies[session_cookie_name].value
Ie. only uses cookies to get it when self._sid evaluates False.
Since if not using cookies but supplying the SID, the fact that
this happens means that the change:
> > if not self._sid:
> > -# check to see if cookie exists
> > -if secret:
> > -cookies = Cookie.get_cookies(req,
> > Class=Cookie.SignedCookie,
> > - secret=self._secret)
> > -else:
> > -cookies = Cookie.get_cookies(req)
> > +if session_cookie_generation:
> > +# check to see if cookie exists
> > +if secret:
> > +cookies = Cookie.get_cookies(req,
> > Class=Cookie.SignedCookie,
> > + secret=self._secret)
> > + else:
> > +cookies = Cookie.get_cookies(req)
is possibly redundant. I can't see any sense why if not supplying
the SID that you would want to stop it reading the cookies as
it probably wouldn't be useful.
In respect of writing out a cookie, it could be argued that if you
were supplying your own SID that it shouldn't assume that it should
write the cookie. In that case though, rather than:
> > -Cookie.add_cookie(self._req, self.make_cookie())
> > +if session_cookie_generation:
> > +Cookie.add_cookie(self._req, self.make_cookie())
it possibly should be:
if not sid:
Cookie.add_cookie(self._req, self.make_cookie())
In other words, don't write out cookie if SID was supplied as input
parameter.
Thus, there wouldn't need to be a reason for a specific Python option
to disable writing of cookie.
So, can you explain what the original problem is you are trying to
solve. On first appearances, your solution would seem to be going
about it the wrong way.
A question for others. Would it be reasonable that a cookie is not
written out if SID was supplied explicitly?
Graham
Graham Dumpleton wrote ..
> Now can you explain why one would want to do this?
>
> Unless you provide some justification of why it is necessary it is
> less likely
> to be accepted as although the reasons may be obvious to you, it may not
> be to us. There also may be better ways of achieving the same end.
>
> Also, describe why this would be better than simply deleting the cookie
> that is being created from the outgoing headers.
>
>del req.headers_out["Set-Cookie"]
>
> Graham
>
> On 21/03/2006, at 7:39 PM, Stanislav Ershov wrote:
>
> > Hi,
> > I wrote a simple patch for 'Session.py'. Patch adds possibility to
> > disable cookies generation by session. And it's optional.
> >
> > By default cookies generation enabled.
> > Add Apache directive 'Python Option sessin_cookie_generation 0' for
> > disabling.
> >
> > --- mod_python-3.2.8.orig/lib/python/mod_python/Session.py Mon Feb
> > 20 00:51:18 2006
> > +++ mod_python-3.2.8/lib/python/mod_python/Session.py Tue Mar 21
> > 09:50:46 2006
> > @@ -138,17 +138,19 @@
> > dict.__init__(self)
> >
> > session_cookie_name = req.get_options().get
> > ("session_cookie_name",COOKIE_NAME)
> > +session_cookie_generation = int(req.get_options().get
> > ("session_cookie_generation",1))
> >
> > if not self._sid:
> > -# check to see if cookie exists
> > -if secret:
> > -cookies = Cookie.get_cookies(req,
> > Class=Cookie.SignedCookie,
> > - secret=self._secret)
> > -else:
> > -
Re: mod_python directory index error
Firat KUCUK wrote ..
> Graham Dumpleton yazmýþ:
> >What is the actual problem you are trying to solve?
> >
> >The "len(req.path_info) > 0" is actually redundant because when
> >req.path_info is a string and has length 0, the "req.path_info"
> >boolean check will fail anyway.
> >
> >In other words, the change you made wouldn't make any difference
> >that I can see to the actual outcome. Is the redundancy all you
> >were wanting to point out???
> >
> >BTW, you should be careful about what SCRIPT_NAME gets set
> >to by Apache and by this code. See discussion of strange things
> >that happen at:
> >
> > https://issues.apache.org/jira/browse/MODPYTHON-68
>
> briefly:
>
> print len(None)
>
> TypeError: len() of unsized object
>
>
> if we use:
>
> if req.path_info *and* len(req.path_info) > 0:
>
> the same error will be occur.
>
> please try
> DirectoryIndex index.py
> and use cgihandler.
>
> req.path_info will be None.
> and we cannot calculate length of none object
If req.path_info is None, the RHS of the "and" is not executed
so len() is never performed on a None object.
Python 2.3 (#1, Sep 13 2003, 00:49:11)
[GCC 3.3 20030304 (Apple Computer, Inc. build 1495)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> path_info = None
>>> if path_info and len(path_info):
.. print "hi"
..
>>> path_info = ""
>>> if path_info and len(path_info):
.. print "hi"
..
>>>
I really can't see how this can be a source of a problem.
I had already written a separate test handler to check.
AddHandler mod_python .py
PythonHandler index
PythonDebug On
DirectoryIndex index.py
# ---
from mod_python import apache
def handler(req):
req.content_type = 'text/plain'
req.write("%s\n"%req.path_info)
req.write("%s\n"%apache.build_cgi_env(req))
return apache.OK
Output shows:
None
{.., 'SCRIPT_NAME': '/~grahamd/script_name/index.py', 'REQUEST_URI':
'/~grahamd/script_name/', ..}
No error occurred.
If you are getting an actual error of some sort, then post what
the full traceback is, as at the moment this isn't making a great
deal of sense to me.
Graham
Re: mod_python 3.3.0-dev-20060321 available for testing
Nicolas Lehuen wrote .. > 2006/3/22, Nicolas Lehuen <[EMAIL PROTECTED]>: > > However I have a -1 on Python 2.2 with a LOT of test failures, but I > > guess we won't support Python 2.2 for mod_python 3.3 ? > > Sorry, my -1 was due to a configuration problem, everything works on Python > 2.2. > > +1 for mod_python 3.3.0-dev-20060321 on Windows 2000 SP4 + > ActivePython 2.2.3 + Apache 2.0.55 If you run the tests with the new importer, I would not have expected it to get very far with Python 2.2. This is because at one point it does: sys.meta_path.insert(0, _ModuleImporter()) Our understanding so far had been that sys.meta_path would only have appeared in Python 2.3, thus the import should have failed when that attribute was used. Graham
Re: mod_python 3.3.0-dev-20060321 available for testing
Graham Dumpleton wrote .. > > On 23/03/2006, at 5:06 AM, Jim Gallacher wrote: > > That's another reason to rewrite the unit tests. It's too hard to > > sort out the wheat from the chaff. > > > > I don't think this is related to your failing test, but I noticed > > the following, repeated numerous times throughout your log: > > > > [Wed Mar 22 09:09:30 2006] [info] [client 127.0.0.1] (32)Broken > > pipe: core_output_filter: writing data to the network > > > > I think we need to investigate this one. > > That is the error I was getting from ab in Apache 2.2. Change test.py to > explicitly use ab from Apache 2.0 and see if they go away. Here is the previous information I posted about core_input_filter errors caused by ab test tool in Apache 2.2. http://www.mail-archive.com/python-dev@httpd.apache.org/msg01535.html I only had it occur once for each test run and specifically the global lock test, since that is the only place ab is used. Thus cause may be different this time. Graham
Auto updating of req.finfo when req.filename changed.
Now the mailing list is a bit quiet, I would like to see if I can get some explicit feedback on some issues related to the inability to update the req.finfo attribute. Grisha, would be nice if you could respond on this issue and give some guidance else I fear I'll never be able to progress a solution to this issue. :-( As explained in: http://issues.apache.org/jira/browse/MODPYTHON-128 although it is possible to assign a new value to req.filename, there is no way to update req.finfo to the file stats associated with that new value of req.filename. If one had access to the low level C API this would normally be achieved using: apr_stat(&r->finfo, r->filename, APR_FINFO_MIN, r->pool); In mod_python though, there is no way to access the function and affect that outcome. In mod_perl 1.0 they implemented the behaviour whereby the "finfo" attribute was automatically updated when the "filename" attribute was updated by a handler. In mod_perl 2.0 they dropped this though, as they wanted to preserve the idea that in mod_perl everything behaved exactly like the C API they were trying to provide a 1 to 1 mapping for. Thus in mod_perl 2.0 you need to write: use Apache2::RequestRec (); use APR::Finfo (); use APR::Const -compile => qw(FINFO_NORM); $r->filename($newfile); $r->finfo(APR::Finfo::stat($newfile, APR::Const::FINFO_NORM, $r->pool)); As mod_python isn't attempting to provide a strict 1 to 1 mapping, it might be argued that it could do what mod_perl 1.0 did and automatically updated the "finfo" attribute when "filename" is updated. The only other alternative is to add a new method to the Python request object for which there isn't strictly speaking a direct equivalent to in the Apache C API. That is, a method that calls apr_stat() but which only performs it in relation to the "filename" and "finfo" attributes in the request object itself and is not a generic routine. Since it isn't likely that mod_python will ever provide a lower level API for use of finfo related structures and functions and even if it did they most likely would be distinct to the request object, the name of the function added to the request object could still be called "stat()". Thus mod_python equivalent to what mod_perl 2.0 does would be: req.filename = newfile req.stat() This though doesn't really convey a sense of what it occurring. Thus a more descriptive name would probably be more appropriate. For example: req.filename = newfile req.update_finfo() There is no ap_update_finfo() function now, but if they did later implement one, this would shadow it and prevent it being added to the request object if it was pertinent for that be done. The next problem is that apr_stat() actually takes an argument indicating what fields in the "finfo" attribute should be updated. In mod_perl 1.0 the value used when the automatic update was done was APR_FINFO_MIN which results in type, mtime, ctime, atime, size being updated. In the documentation for mod_perl 2.0 it suggests use of APR_FINFO_NORM instead which is described as intended to be used when an atomic unix apr_stat() is required whatever that means. Important to note though is that is that the ap_directory_walk() function in Apache which is used to map a URL against a file in the filesystem uses APR_FINFO_MIN. Now if a function were to be provided, it seems to make sense that it have a default whereby it uses APR_FINO_MIN, much as would be the case if the "finfo" attribute were updated automatically when "filename" is updated. Should though a function if provided allow the ability to supply an alternate for this value so as to be selective as to what attributes of "finfo" are updated? If it were allowed, have the problem that there are already attributes in mod_python for: FINFO_MODE = 0 FINFO_INO = 1 FINFO_DEV = 2 FINFO_NLINK = 3 FINFO_UID = 4 FINFO_GID = 5 FINFO_SIZE = 6 FINFO_ATIME = 7 FINFO_MTIME = 8 FINFO_CTIME = 9 FINFO_FNAME = 10 FINFO_NAME = 11 FINFO_FILETYPE = 12 Rather than these being equivalents to the APR constants: #define APR_FINFO_LINK 0x0001 #define APR_FINFO_MTIME 0x0010 #define APR_FINFO_CTIME 0x0020 #define APR_FINFO_ATIME 0x0040 #define APR_FINFO_SIZE 0x0100 #define APR_FINFO_CSIZE 0x0200 #define APR_FINFO_DEV 0x1000 #define APR_FINFO_INODE 0x2000 #define APR_FINFO_NLINK 0x4000 #define APR_FINFO_TYPE 0x8000 #define APR_FINFO_USER 0x0001 #define APR_FINFO_GROUP 0x0002 #define APR_FINFO_UPROT 0x0010 #define APR_FINFO_GPROT 0x0020 #define APR_FINFO_WPROT 0x0040 #define APR_FINFO_ICASE 0x0100 #define APR_FINFO_NAME 0x0200 #define APR_FINFO_MIN 0x8170 #define APR_FINFO_IDENT 0x3000 #define APR_FINFO_OWNER 0x0003 #define A
Re: Auto updating of req.finfo when req.filename changed.
that one can't use the mtime stored in req.finfo as input to req.set_last_modified(). Ie.,set last modified header based on the modification time of Python code file as stored in req.finfo. If one really wanted to do it, one would have to stat the file once again to get the mtime and call req.update_mtime() explicitly. I could possibly drag up other cases, but hopefully this time I have been able to more clearly show that there are some legitimate uses for being able to update req.finfo to reflect newer value of req.filename. If this is accepted, the question still remains is how to enable that ability. Graham Grisha wrote .. > > I'm -1 on updating req.finfo when req.filename is updated. I don't have > the time to explain it in detail, but I think it flows from Graham's > explanation below. > > Basically, httpd does a stat() for you, and its result is stored in finfo. > Why make it any more complicated and magical? > > If you alter req.filename - should that imply another stat()? I don't > think so, because the stat() is the most expensive operations in the whole > request service process and unnecessary stats should be avoided. > > I also don't see any good use case for it. If you need a stat - Python > provides way for you to do it, there is no need to get httpd involved in > this process. > > I, of course, may be missing something obvious here, but this is my > initial reaction. Let httpd/apr behave the way they behave - there is no > need to "pythonize" them, especially when the Python standard library > provides the same functionality in a "pythonic" way. > > Grisha > > On Sun, 26 Mar 2006, Jim Gallacher wrote: > > > Hi Graham, > > > > +1 auto update req.finfo when req.filename changed. > > > > Is there a use case where the user might change filename but not want > finfo > > to change? I can't think of one, so let's save the user some work and > make > > their code more robust to boot. > > > > A point I'd like to address is your concern about mod_python differing > from > > the Apache C api in implementing certain features. Personally I think > our > > general concern about this is somewhat misguided. I suspect that the > majority > > of our users are more concerned about the "python-ness" of mod_python > rather > > than its "apache-ness", and most would never notice if we deviate slightly > > from the apache C api. Adding a method or attribute to the request object > for > > example, if it's useful to *our* users, shouldn't be rejected out of > hand > > just because it does not exist in the underlying api. > > > > Jim > > > > Graham Dumpleton wrote: > >> Now the mailing list is a bit quiet, I would like to see if I can get > >> some explicit feedback on some issues related to the inability to update > >> the req.finfo attribute. > >> > >> Grisha, would be nice if you could respond on this issue and give some > >> guidance else I fear I'll never be able to progress a solution to this > >> issue. :-( > >> > >> As explained in: > >> > >> http://issues.apache.org/jira/browse/MODPYTHON-128 > >> > >> although it is possible to assign a new value to req.filename, there > is > >> no way to update req.finfo to the file stats associated with that new > >> value of req.filename. If one had access to the low level C API this > >> would normally be achieved using: > >> > >> apr_stat(&r->finfo, r->filename, APR_FINFO_MIN, r->pool); > >> > >> In mod_python though, there is no way to access the function and affect > >> that outcome. > >> > >> In mod_perl 1.0 they implemented the behaviour whereby the "finfo" > >> attribute was automatically updated when the "filename" attribute was > >> updated by a handler. > >> > >> In mod_perl 2.0 they dropped this though, as they wanted to preserve > >> the idea that in mod_perl everything behaved exactly like the C API > >> they were trying to provide a 1 to 1 mapping for. Thus in mod_perl > >> 2.0 you need to write: > >> > >> use Apache2::RequestRec (); > >> use APR::Finfo (); > >> use APR::Const -compile => qw(FINFO_NORM); > >> $r->filename($newfile); > >> $r->finfo(APR::Finfo::stat($newfile, APR::Const::FINFO_NORM, $r->pool)); > >> > >> As mod_python isn't attempting to provide a strict 1 to 1 mapping, it > >>
PythonImport that works for any interpreter.
In: http://issues.apache.org/jira/browse/MODPYTHON-117 I describe the idea of having a means of using PythonImport to define a module to be imported into any interpreter that may be created. For some cases where there are a lot of virtual hosts, this may be simpler than having to list a directive for every virtual host explicitly. Is there any interest in such a feature? If of interest, for a simple implementation, the only issue is one of ordering when for an interpreter there are both imports for all interpreters and an interpreter specific imports. Does one import the module specified to be imported in all interpreters first before the interpreter specific ones or vice versa. My feeling has been that the modules to be imported in all intrepreters should be done first. Feedback? Am I wasting my time implementing this one? Graham
Pickling/unpickling top-level functions, classes etc.
Nicolas Are you okay with: http://issues.apache.org/jira/browse/MODPYTHON-81 Pickling/unpickling top-level functions defined in published module no longer works in mod_python 3.2 being resolved as "Won't Fix" and then closed? As I describe in: http://www.dscpl.com.au/articles/modpython-005.html there are going to be various issues with pickling with any new importer which doesn't keep stuff in sys.modules. I don't see any solution for the issue as far as modules managed by the mod_python importer. Simply means that if you want to pickle stuff like that, has to be in module on standard sys.path and managed by normal Python module import system. Anyone else want to comment? Graham
Re: Auto updating of req.finfo when req.filename changed.
Grisha wrote .. > > On Sun, 26 Mar 2006, Graham Dumpleton wrote: > > > One use for it that I already have is to get around the DirectoryIndex > > problems in mod_python caused by Apache's use of the > > ap_internal_fast_redirect() function to implement that feature. The > > specifics of this particular issue are documented under: > > > > http://issues.apache.org/jira/browse/MODPYTHON-146 > > > Could we zoom in this a little bit. I've read the description, but not > quite sure I understand it quite yet. Is "the problem" that if I set > req.notes['foo'] = 'bar' in a phase prior to fixup, by the time we get > to > the content handler, it will be gone because notes would be overwritten > by > mod_dir? Fixup phase or earlier actually. In the case of req.notes though, it isn't that the value in req.notes vanishes, it is that it gets duplicated. Consider .htaccess file containing: AddHandler mod_python .py PythonHandler mod_python.publisher PythonDebug On DirectoryIndex index.py PythonFixupHandler _fixup In _fixup.py in the same directory, have: from mod_python import apache import time def fixuphandler(req): time.sleep(0.1) req.notes['time'] = str(time.time()) return apache.OK In index.py have: def index(req): return req.notes['time'] When I use a URL: http://localhost:8080/~grahamd/fast_redirect/index.py the result I get is: 1143667522.23 Ie., a single float value holding the time the request was made. If I now instead access the directory using the URL: http://localhost:8080/~grahamd/fast_redirect/ I instead get: ['1143667680.57', '1143667680.47'] In other words, instead of getting the single value I now get two values contained in a list. It wouldn't matter if the the two values were the same they would both still be included. Where a content handler was expecting a single string value, it would die when it gets a list. What is happening is that when the request is made against the directory it runs through the phases up to and including the fixup handler phase. As a consequence it runs _fixup::fixuphandler() with req.notes['time'] being set to be the time at that point. At the end of the fixup phase a mod_dir handler kicks in and it sees that the file type of request_rec->filename as indicated by request_rec->finfo->filetype is APR_DIR. As a consequence it will apply the DirectoryIndex directive, looping through listed files to find a candidate it can redirect the request too. In finding a candidate it reapplies phases up to and including the fixup handler phase on the new candidate filename. This is done so that access and authorisation checks etc are still performed on the candidate file. Because it has run the fixup handlers on the candidate file, the _fixup::fixuphandler() will be run again. This results in req.notes being set. At that stage the req.notes is separate as it is in effect run as a sub request to the main request against the directory. If after checking through the candidates it finds one that matches, to avoid having to run phases up to and including the fixup handler phase on the candidate again, mod_dir tries to fake a redirect. This is what ap_internal_fast_redirect() is being used for. What the method does is to copy details from the request_rec structure of the sub request for the candidate into the request_rec of the main request. When the mod_dir fixup handler returns, the main request then continues on to execute the content handler phase, with the details of the sub request. The problem with this is that rather than simply using req.notes from the sub request, or overlapping the contents from the sub request onto that of the main request, it merges them together. You therefore end up with multiple entries for the 'time' value which was added. To emphasise the problem, change the fixup handler to be: from mod_python import apache def fixuphandler(req): req.notes['filename'] = req.filename return apache.OK and index.py to: def index(req): return req.notes['filename'] The result when using URL against the directory is used is: ['/Users/grahamd/Sites/fast_redirect/index.py', '/Users/grahamd/Sites/fast_redirect/'] Now it isn't just req.notes that is going to see this merging as the code in ap_internal_fast_redirect() is: r->notes = apr_table_overlay(r->pool, rr->notes, r->notes); r->headers_out = apr_table_overlay(r->pool, rr->headers_out, r->headers_out); r->err_headers_out = apr_table_overlay(r->pool, rr->err_headers_out, r->err_headers_out); r->subprocess_env = apr_table_overlay(r->pool, rr->subpr
Re: mod_python/sax bug.. (<3.1.4, 3.2.8)
As pointed out by someone else, it is probably a versioning issue
with expat.
See the following article for how to debug which versions of expat
are being
used and thus whether there might be a conflict.
http://www.dscpl.com.au/articles/modpython-006.html
Graham
On 30/03/2006, at 9:00 PM, Dom. Orchard wrote:
Hi,
I'm in the middle of re-writing an application that uses mod_python
for serving to the web, and have decided to switch to an XML model
to store some information,
however, I encountered an issue that I don't quite understand, see
this short example that isolates the problem:
from xml.sax import make_parser, SAXException
def index(req):
req.write("Hello...")
try :
test = make_parser()
except SAXException:
req.write("Failed")
req.write("\n Can you see me?")
When I try to view this page I just get the "Hello..." text and
then nothing else, the program seems to terminate at the make_parser
() call. Which is most confusing. Run locally from the command
prompt there are no problems. I was using mod_python 3.1.4 in a
hope to fix the problem I downloaded 3.2.8, but this did not change
anything.
Here is my Apache info (Apache/2.0.54 (Unix) mod_python/3.2.8
Python/2.4 PHP/5.0.4)
Does anyone else get this problem?
Could this be a potential bug, I'm not quite sure what would be
causing it though, some kind of serialization problem?
Regards,
Dominic Orchard
mod_python and Apache 2.2.1 release candidate.
FWIW, mod_python (from subversion) passes all tests with the Apache 2.2.1 release candidate on Mac OS X 10.4. Previously the test_global_lock test failed with Apache 2.2.0 because of a bug in the APR library on Mac OS X. Apache 2.2.1 has newer version of APR library which fixes problem. Graham
GET request content and mod_python.publisher/psp.
I have just added to mod_python in subversion a req.discard_request_body() method. This is a direct wrapper for underlying ap_discard_request_body() function in C API. The purpose of the underlying function is as described in documentation attached to prorotype in headers. /** * In HTTP/1.1, any method can have a body. However, most GET handlers * wouldn't know what to do with a request body if they received one. * This helper routine tests for and reads any message body in the request, * simply discarding whatever it receives. We need to do this because * failing to read the request body would cause it to be interpreted * as the next request on a persistent connection. * @param r The current request * @return error status if request is malformed, OK otherwise */ AP_DECLARE(int) ap_discard_request_body(request_rec *r); In other words, the function should be used in GET handlers to get rid of any content in the request. I appreciate that web clients are generally well behaved, but if such content is sent and isn't discarded and keep alives are in use on a connection and requests are pipelined, that content can be wrongly interpreted as being the next request and a failure would thus occur. A robust handler would therefore always call req.discard_request_body() if the request is a GET request. My question is, should mod_python.publisher and mod_python.psp be enhanced and call req.discard_request_body() for a GET request to avoid the posibilities of any problems arising due to a client sending content for a GET request? With the rise of AJAX applications where Javascript is explicitly used to send custom requests, even if it is done inadvertantly and POST should have been used, the risk of having GET requests with content is also probably on the rise. Note that although mod_python.publisher will always apply util.FieldStorage to a request, it only consumes content for a POST request. Same deal with mod_python.psp, although it only processes forms when "form" variable accessed from PSP page. Thoughts??? Graham
Re: GET request content and mod_python.publisher/psp.
On 03/04/2006, at 4:53 PM, Mike Looijmans wrote: My question is, should mod_python.publisher and mod_python.psp be enhanced and call req.discard_request_body() for a GET request to avoid the posibilities of any problems arising due to a client sending content for a GET request? -1 on that particular way of implementing it. If the GET request has a body, that body probably serves some purpose. The right thing to do for any handler that does not know how to handle the request is to return a 'bad request' error to the client. Just throwing away what is not understood is not very nice to developers and users - you'll get unexpected behaviour because the server is only handling a part of the request. The trouble here is of course that publisher or PSP cannot tell forehand that the handler will read the body data. So the only way to determine this is to have the handler handle the request, and after that, check if it did read all of the request. If not, you're too late to report this to the client, because the headers have already been sent out. Putting some message in an error log that no- one will ever read (in particular not the one who caused that problem) does not make sense either. To fix this, the handler should somehow advertise its capability to read the body. I guess you can't really solve the problem. Which is the lesser evil? Digging further, Apache will always ensure that ap_discard_request_body() is executed at the end of the request being processed, so there is no real problem to be solved. Although caution is good, I have read to much into the combination of the statements: The first step we take upon entering the handler() function is to call the discard_request_body() method. Unlike HTTP/1.0, where only POST and PUT requests may contain a request body, in HTTP/1.1 any method may include a body. We have no use for it, so we throw it away to avoid potential problems. and: * In HTTP/1.1, any method can have a body. However, most GET handlers * wouldn't know what to do with a request body if they received one. * This helper routine tests for and reads any message body in the request, * simply discarding whatever it receives. We need to do this because * failing to read the request body would cause it to be interpreted * as the next request on a persistent connection. The bit I was missing was that Apache calls the function already. :-) So, nothing to see, move along Graham
FieldStorage and multiline headers in multipart/form.
With FieldStorage being discussed on main user mailing list, came across this old post of the mailing list: http://www.modpython.org/pipermail/mod_python/2001-November/012256.html What it is saying is that some HTTP clients use multi line headers in sections of multipart/form postings and that mod_python doesn't handle this. Looking at FieldStorage code, which I don't grok right at this minute because of an intensive coding frenzy today on the back of not enough sleep last night, I can't see that it has ever been modified to accomodate such multiline headers if indeed it needs to. Anyone who is more intimate with FieldStorage code want to have a better look at validity of original mailing list post and whether multiline headers are legitimate and whether there indeed could be a problem in mod_python. Graham
Re: Progressing 3.2.9.
Graham Dumpleton wrote .. > Jim Gallacher wrote .. > > WRT to 3.2.9, I've been bogged down with other stuff and will likely > be > > fairly busy this week as well. How be we aim for a release somewhere > > around April 22? I'd like to sort out why the apache 2.2 auth test fails > > on some platforms before the release. It's a defect in the test I'm > > sure, but I don't think we should make a release when failing tests. > > In terms of the list we drew up, what still needs to be done? Has > anything new come up since that we should also consider? > > I can help progress things a bit, although stuff like these logging > additions would prefer someone else to do purely so they get reviewed by > someone else in the process to ensure I didn't stuff up. > > BTW, you don't happen to have a reference to the Apache 2.2 auth test > failures to save me tracking it down in the archive do you. :-) All I can find from previous 3.3 snapshot testing on Apache 2.2 auth problems was: http://www.mail-archive.com/python-dev@httpd.apache.org/msg01703.html Specifically: [Wed Mar 22 07:16:03 2006] [warn] mod_python (pid=5140,interpreter='test_req_auth_type'): Module directory listed in "sys.path". This may cause problems. Please check code. Code file being imported is "C:\\projets\\mod_python\\test\\htdocs\\tests.py". [Wed Mar 22 07:16:03 2006] [notice] mod_python (pid=5140,interpreter='test_req_auth_type'): Importing module 'C:\\projets\\mod_python\\test\\htdocs\\tests.py' [Wed Mar 22 07:16:03 2006] [crit] [client 127.0.0.1] configuration error: couldn't check access. No groups file?: /tests.py [Wed Mar 22 07:16:03 2006] [error] [client 127.0.0.1] No Authn provider configured Nicolas, what is the latest on this, I couldn't see any followup to say it had been resolved as a configuration issue or otherwise. The "No Authn provider configured" error almost suggests Apache can't load up separate Apache 2.2 auth provider modules and that it is getting itself in a know over that. Graham
Re: Progressing 3.2.9.
Is there any of these you would like me to do, perhaps 77, 94 or 137 since I am familiar with the code? Not knowing what you are up to, wary of just going ahead and doing any in case you are already doing it. Any hints on how to use subversion to merge diff automatically from the other branch? Graham On 11/04/2006, at 12:47 PM, Jim Gallacher wrote: Here is the list of things I still need to backport. Fixes have already been committed to trunk. MODPYTHON-77 The Simplified GIL Aquisition patches. MODPYTHON-94 Support for optional mod_ssl functions on request object. MODPYTHON-131 Make mutex directory configurable. MODPYTHON-145 Make number of mutex locks configurable at apache startup MODPYTHON-137 Add req.server.get_options() for obtain PythonOption values set at global level. MODPYTHON-93 Improved FieldStorage The backports shouldn't take more than an hour and I can likely get to them tommorow. I'd do it now but my head is not entirely clean (too sleepy) so I'd better wait until morning. That just leaves the review and backporting of the logging stuff, and deciding if we need to worry about the auth test failure issue for apache 2.2. Once we sort those out I'll be able to roll a tarball that we can offer as a release candidate. I'm heading out of town on Thursday so if I don't get to it by then it'll have to wait until next Monday. Jim
Re: Progressing 3.2.9.
On 11/04/2006, at 12:47 PM, Jim Gallacher wrote: Here is the list of things I still need to backport. Fixes have already been committed to trunk. MODPYTHON-77 The Simplified GIL Aquisition patches. Jim, you must be coding in your sleep and thus not knowing what you are doing as MODPYTHON-77 was back ported into 3.2.X branch back in March. http://svn.apache.org/viewcvs?view=rev&rev=383731 One less thing to do at least. ;-) Graham
Re: Progressing 3.2.9.
On 11/04/2006, at 12:47 PM, Jim Gallacher wrote: Here is the list of things I still need to backport. Fixes have already been committed to trunk. MODPYTHON-77 The Simplified GIL Aquisition patches. Was already done. MODPYTHON-94 Support for optional mod_ssl functions on request object. MODPYTHON-131 Make mutex directory configurable. MODPYTHON-145 Make number of mutex locks configurable at apache startup MODPYTHON-137 Add req.server.get_options() for obtain PythonOption values set at global level All the above now done. MODPYTHON-93 Improved FieldStorage Not yet done. Also haven't done logging additions listed under MODPYTHON-158. Time for me to sleep, so I will pass the baton on for the moment. ;-) At this point, if there is anything else that anyone feels should be back ported from the 3.3 development tree to 3.2.x branch, speak now or forever hold your peace. If you are unsure of what work has been done on 3.3 so far, see resolved issues: http://issues.apache.org/jira/secure/IssueNavigator.jspa? reset=true&pid=10640&status=5 At this point, we are probably only after changes which have some impact and would be better addressed now rather than later. Thus, feel free to put your case, but don't get too carried away. Graham
Re: mysqldb error
On 13/04/2006, at 5:26 AM, Firat KUCUK wrote: Hi, i wrote a simple connection script. It works as a command line script or cgi script. But does not work in mod_python. import MySQLdb conn = MySQLdb.connect( host = '127.0.0.1', user = 'pismikrop', passwd = 'pass', db = 'db') print conn conn.close() command-line and cgi output: <_mysql.connection open to '127.0.0.1' at 81a304c> mod_python output: <_mysql.connection open to '(null)' at 82ac504> i cannot execute MySQL queries. What should i do? Can you supply the remainder of your mod_python handler code so we can see the context in which this used? Minimal but complete as possible is preferable. If the above is truly exactly how you are using it, you probably aren't using mod_python but a CGI scripts as "print" statement will not output anywhere in mod_python so you wouldn't be able to capture it. Graham
Re: [mod_python] Anyone have an idea when mod_python will be available for Apache 2.2
Nicolas Lehuen wrote ..
> Just as a note, I'm still struggling with Apache 2.2 to make the test suite
> run with the latest svn version (everything seems OK with Apache 2.0.55).
>
> I still have a problem while testing req.auth_type(). It looks like the
> authentication system was changed in Apache 2.2, and that the current test
> framework does not configure it correctly.
>
> If I understood the new framework correctly, the authentication is handled
> by three different layers, in which you can select whatever implementation
> you want.
>
> The first layer is the authentication method, for now you can choose the
> Basic (mod_auth_basic.c) or the Digest (mod_auth_digest.c) method.
>
> The second layer is the authentication database, which is used to
> authenticate the credentials provided by the authentication method against
> a
> database. There are different mod_authn_* modules for different database
> storage, for instance mod_authn_file for flat file storage and mod_authn_dbm
> for DBM file storage.
>
> The last layer is the authorization database, which is used to check whether
> the given authenticated user may access the request resource. There are
> different mod_authz_* modules for different database storage, too.
>
> The current problem is that the test suite does not configure correctly
> the
> authentication framework. Either I get a 500 error because the
> authentication database is not properly configured, hence causing the
> "configuration error: couldn't check access. No groups file?" in the
> error
> log, or if I set up properly the authn and authz modules (adding directives
> to load mod_authn_default and mod_authz_default in test.py line 338) then
> I
> get a 401 error because the test users are not in the database.
>
> The best thing I guess would be to disable authentication & authorisation
> entirely because it seems that Apache 2.2 is hijacking the entire
> authn/authz process, even when mod_python tries to do it itself.
>
> I tried playing with the AuthBasicAuthoritative directive, even mimicking
> in
> test_req_auth_type_conf() the same condition that was put in
> test_req_requires_conf(). Unfortunately this doesn't lead me anywhere.
>
> What I don't understand is why the problem is only showing on Win32 and
> Jim's platform. Is it really OK on other platforms ?
>
> Any thoughts on this problem ?
I had a look at this a while back and couldn't work out why it would be
failing. I at least have no problems on Mac OS X. In looking through
again, one thought was to ask whether you tried it with both the new
module importer and old and got the same results. The original error
messages you posted indicated it was with new importer.
[Wed Mar 22 07:16:03 2006] [warn] mod_python
(pid=5140,interpreter='test_req_auth_type'): Module directory listed
in "sys.path". This may cause problems. Please check code. Code file
being imported is "C:\\projets\\mod_python\\test\\htdocs\\tests.py".
[Wed Mar 22 07:16:03 2006] [notice] mod_python
(pid=5140,interpreter='test_req_auth_type'): Importing module
'C:\\projets\\mod_python\\test\\htdocs\\tests.py'
[Wed Mar 22 07:16:03 2006] [crit] [client 127.0.0.1] configuration
error: couldn't check access. No groups file?: /tests.py
[Wed Mar 22 07:16:03 2006] [error] [client 127.0.0.1] No Authn
provider configured
Can you also augment the req_auth_type() in test/htdocs/tests.py
with some calls to req.log_error() to capture whether the function
is in fact being called and what path through the function it goes
through. Ie.,
def req_auth_type(req):
req.log_error("req_auth_type()")
req.log_error("phase=%s" % req.phase)
if (req.phase == "PythonAuthenHandler"):
if req.auth_type() != "dummy":
req.log_error("auth_type check failed")
req.write("test failed")
return apache.DONE
if req.auth_name() != "blah":
req.log_error("auth_name check failed")
req.write("test failed")
return apache.DONE
req.log_error("set user/ap_auth_type")
req.user = "dummy"
req.ap_auth_type = req.auth_type()
else:
if req.ap_auth_type != "dummy":
req.log_error("ap_auth_type check failed")
req.write("test failed")
return apache.OK
if req.user != "dummy":
req.log_error("user check failed")
req.write("test failed")
return apache.OK
req.log_error("return test ok")
req.write("test ok")
return apache.OK
I would expect to see in the tests/logs/error_log file:
[Thu Apr 13 11:09:37 2006] [error] [client 127.0.0.1] req_auth_type()
[Thu Apr 13 11:09:37 2006] [error] [client 127.0.0.1] phase=PythonAuthenHandler
[Thu Apr 13 11:09:37 2006] [error] [client 127.0.0.1] set user/ap_auth_type
[Thu Apr 13 11:09:37 2006] [error] [client 127.0.0.1] req_auth_type()
[Thu Apr 13 11:09:37 2006] [error] [client 127.0.0.1] phase=PythonHandler
[Thu Apr 13 11:09:37 2006]
Re: [mod_python] Anyone have an idea when mod_python will be available for Apache 2.2
Graham Dumpleton wrote ..
> What is interesting now is that when doing that, I note that on Mac OS
> X there are some worrying error messages which follow that:
>
> [Thu Apr 13 11:09:37 2006] [error] Internal error: pcfg_openfile() called
> with NULL filename
> [Thu Apr 13 11:09:37 2006] [error] [client 127.0.0.1] (9)Bad file descriptor:
> Could not open password file: (null)
>
> A bit of checking though suggests that that is from test_req_requires.
>
> I'll dig into these latter errors, maybe they might help to uncover
> something.
The reason for the above errors is that the req_requires configuration
is in practice probably wrong. The configuration for the test is:
c = VirtualHost("*",
ServerName("test_req_requires"),
DocumentRoot(DOCUMENT_ROOT),
Directory(DOCUMENT_ROOT,
SetHandler("mod_python"),
AuthName("blah"),
AuthType("basic"),
Require("valid-user"),
AuthBasicAuthoritative("Off"),
PythonAuthenHandler("tests::req_requires"),
PythonDebug("On")))
By saying:
AuthType("basic"),
you are literally saying that it should be passed to provider for "Basic"
authentication, but in doing that no AuthFile directive has been supplied
and so it is obviously going to fail. Apache 2.2 is probably at fault for
not giving better error messages.
To fix the problem, it should be set to anything but "basic" or "digest".
AuthType("dummy"),
In your original email you said your problem was with the auth_type test.
I've tested with and without the new importer on Windows XP SP2 +
Python 2.4.2 + Apache 2.2.0 and everything works except the
test_req_auth_type test, which signals a 500 error.
Are you sure it was the auth_type test and not the req_requires test
that immediately follows?
Graham
Re: mysqldb error
Apache doesn't probably run as any of the users which your database allows access to. Add lines in your CGI which says: import os print os.getuid() and then see what user that UID actually is and give it access. User may be something like "apache", "www", "wwwroot" or possibly even "nobody" depending on the system configuration. You can also check your Apache configuration for lines: User www Group www to see what it runs as: Graham Firat KUCUK wrote .. > my distro is ubuntu breezy, > I used cgi handler. And text mime type > so i can view the print statement. > > MySQL server 4.0.24 > apache 2.0.54 > python2.4-mysqldb 1.2.1 > > php, console python, cgi python works fine. > > my .htaccess file > > Allow from All > > AddHandler mod_python .py > PythonHandlermod_python.cgihandler > PythonDebug On > > DirectoryIndex index.htm index.html index.php index.py index.pl > > - > > #! /usr/bin/python > # -*- coding: UTF-8 -*- > > print 'Content-Type: text/plain\n' > > import MySQLdb > conn = MySQLdb.connect( >host = '127.0.0.1', >user = 'pismikrop', >passwd = 'pass', >db = 'gate') > print conn > conn.close() > > > --- > > output: > <_mysql.connection open to '(null)' at 82a97e4> > > > if host = '10.0.0.6' > > Mod_python error: "PythonHandler mod_python.cgihandler" > > Traceback (most recent call last): > > File "/usr/lib/python2.4/site-packages/mod_python/apache.py", line 299, > in HandlerDispatch > result = object(req) > > File "/usr/lib/python2.4/site-packages/mod_python/cgihandler.py", line > 96, in handler > imp.load_module(module_name, fd, path, desc) > > File "/home/pismikrop/vhosts/mikropyuvasi/content/tests/mpcgi/firat.py", > line 11, in ? > db = 'gate') > > File "/usr/lib/python2.4/site-packages/MySQLdb/__init__.py", line 66, > in Connect > return Connection(*args, **kwargs) > > File "/usr/lib/python2.4/site-packages/MySQLdb/connections.py", line > 134, in __init__ > super(Connection, self).__init__(*args, **kwargs2) > > OperationalError: (2003, "Can't connect to MySQL server on '10.0.0.6' (111)") > > -- > > mysql> SELECT User, Host FROM user; > +--+--+ > | User | Host | > +--+--+ > | pismikrop| %| > | debian-sys-maint | localhost| > | root | localhost| > | root | mikropyuvasi | > +--+--+ > > pismikrop user has all priviliges to all databases.
Re: mysqldb error
Whoops. I could be talking nonsense here. But then I missed that your code says 127.0.0.1 yet the error says 10.0.0.6. FWIW, the reason that I thought to suggest to look at this was that I was using a database once where using 127.0.0.1 made it use a local database connection rather than full IP connection and in that situation, for whatever reason it actually ignored the user name in the login and was using the user ID of the process connecting to the database to determine privileges. I rarely use databases, so could though be completely wrong and misunderstood what I saw at the time. :-( Someone who knows what they are talking about should step in and save me now. :-) Graham Graham Dumpleton wrote .. > Apache doesn't probably run as any of the users which your database > allows access to. Add lines in your CGI which says: > > import os > print os.getuid() > > and then see what user that UID actually is and give it access. User > may be something like "apache", "www", "wwwroot" or possibly even > "nobody" depending on the system configuration. > > You can also check your Apache configuration for lines: > > User www > Group www > > to see what it runs as: > > Graham > > Firat KUCUK wrote .. > > my distro is ubuntu breezy, > > I used cgi handler. And text mime type > > so i can view the print statement. > > > > MySQL server 4.0.24 > > apache 2.0.54 > > python2.4-mysqldb 1.2.1 > > > > php, console python, cgi python works fine. > > > > my .htaccess file > > > > Allow from All > > > > AddHandler mod_python .py > > PythonHandlermod_python.cgihandler > > PythonDebug On > > > > DirectoryIndex index.htm index.html index.php index.py index.pl > > > > - > > > > #! /usr/bin/python > > # -*- coding: UTF-8 -*- > > > > print 'Content-Type: text/plain\n' > > > > import MySQLdb > > conn = MySQLdb.connect( > >host = '127.0.0.1', > >user = 'pismikrop', > >passwd = 'pass', > >db = 'gate') > > print conn > > conn.close() > > > > > > --- > > > > output: > > <_mysql.connection open to '(null)' at 82a97e4> > > > > > > if host = '10.0.0.6' > > > > Mod_python error: "PythonHandler mod_python.cgihandler" > > > > Traceback (most recent call last): > > > > File "/usr/lib/python2.4/site-packages/mod_python/apache.py", line > 299, > > in HandlerDispatch > > result = object(req) > > > > File "/usr/lib/python2.4/site-packages/mod_python/cgihandler.py", line > > 96, in handler > > imp.load_module(module_name, fd, path, desc) > > > > File "/home/pismikrop/vhosts/mikropyuvasi/content/tests/mpcgi/firat.py", > > line 11, in ? > > db = 'gate') > > > > File "/usr/lib/python2.4/site-packages/MySQLdb/__init__.py", line 66, > > in Connect > > return Connection(*args, **kwargs) > > > > File "/usr/lib/python2.4/site-packages/MySQLdb/connections.py", line > > 134, in __init__ > > super(Connection, self).__init__(*args, **kwargs2) > > > > OperationalError: (2003, "Can't connect to MySQL server on '10.0.0.6' > (111)") > > > > -- > > > > mysql> SELECT User, Host FROM user; > > +--+--+ > > | User | Host | > > +--+--+ > > | pismikrop| %| > > | debian-sys-maint | localhost| > > | root | localhost| > > | root | mikropyuvasi | > > +--+--+ > > > > pismikrop user has all priviliges to all databases.
Re: [mod_python] Anyone have an idea when mod_python will be available for Apache 2.2
Nicolas
Can you check out latest code from trunk in subversion and retest to
see if problem has gone away. I have made the req_auth_type and
req_requires tests both a bit more exact/resilient whereas before
they left out things which otherwise might need to be present in
a properly crafted handler which made use of such features.
Let me know if things look better and I will back port req_requires
changes to 3.2.x branch.
Thanks.
Graham
Graham Dumpleton wrote ..
> Graham Dumpleton wrote ..
> > What is interesting now is that when doing that, I note that on Mac OS
> > X there are some worrying error messages which follow that:
> >
> > [Thu Apr 13 11:09:37 2006] [error] Internal error: pcfg_openfile() called
> > with NULL filename
> > [Thu Apr 13 11:09:37 2006] [error] [client 127.0.0.1] (9)Bad file
> > descriptor:
> > Could not open password file: (null)
> >
> > A bit of checking though suggests that that is from test_req_requires.
> >
> > I'll dig into these latter errors, maybe they might help to uncover
> > something.
>
> The reason for the above errors is that the req_requires configuration
> is in practice probably wrong. The configuration for the test is:
>
> c = VirtualHost("*",
> ServerName("test_req_requires"),
> DocumentRoot(DOCUMENT_ROOT),
> Directory(DOCUMENT_ROOT,
> SetHandler("mod_python"),
> AuthName("blah"),
> AuthType("basic"),
> Require("valid-user"),
> AuthBasicAuthoritative("Off"),
> PythonAuthenHandler("tests::req_requires"),
> PythonDebug("On")))
>
> By saying:
>
> AuthType("basic"),
>
> you are literally saying that it should be passed to provider for "Basic"
> authentication, but in doing that no AuthFile directive has been supplied
> and so it is obviously going to fail. Apache 2.2 is probably at fault for
> not giving better error messages.
>
> To fix the problem, it should be set to anything but "basic" or "digest".
>
> AuthType("dummy"),
>
> In your original email you said your problem was with the auth_type test.
>
> I've tested with and without the new importer on Windows XP SP2 +
> Python 2.4.2 + Apache 2.2.0 and everything works except the
> test_req_auth_type test, which signals a 500 error.
>
> Are you sure it was the auth_type test and not the req_requires test
> that immediately follows?
>
> Graham
strict_parsing parameter in forms handling.
The util.FieldStorage class, plus parse_qs and parse_qsl functions
take a
parameter called strict_parsing. All the documentation says is:
The \var{strict_parsing} argument is not yet implemented.
Ie., it doesn't even say what it is meant to be for.
Does anyone know what it is meant to be for. The closest I could find as
to what it may be for is from cgi.FieldStorage documentation:
strict_parsing: flag indicating what to do with parsing errors.
If false (the default), errors are silently ignored.
If true, errors raise a ValueError exception.
I don't know though what constitutes an error though.
This argument has been there back from mod_python 2.X. If it is never
going
to be implemented, is there any point really keeping it in the API?
Graham
Re: mysqldb error
On 13/04/2006, at 8:33 PM, Firat KUCUK wrote: i think it is related with: http://www.modpython.org/FAQ/faqw.py?req=show&file=faq02.013.htp but i didn't understand. If that is the case, it is easy to check. This is done by disabling the loading into Apache of PHP support. If after commenting out the PHP lines and doing a "stop/start" of Apache (best not to use "restart" just in case it doesn't unloaded shared libraries properly) your problem doesn't go away, then that FAQ entry is not relevant. Graham
Re: Progressing 3.2.9.
On 12/04/2006, at 10:13 PM, Graham Dumpleton wrote: On 11/04/2006, at 12:47 PM, Jim Gallacher wrote: MODPYTHON-93 Improved FieldStorage Not yet done. Also haven't done logging additions listed under MODPYTHON-158. Now also both done. At this point, if there is anything else that anyone feels should be back ported from the 3.3 development tree to 3.2.x branch, speak now or forever hold your peace. If you are unsure of what work has been done on 3.3 so far, see resolved issues: http://issues.apache.org/jira/secure/IssueNavigator.jspa? reset=true&pid=10640&status=5 At this point, we are probably only after changes which have some impact and would be better addressed now rather than later. Thus, feel free to put your case, but don't get too carried away. I can see maybe a few little things that could also be done, but I'll post about that tomorrow as getting late again and time for me to get some sleep. :-) Graham
Re: mysqldb error [solved]
On 13/04/2006, at 9:30 PM, Firat KUCUK wrote: Hi Guys, php5 uses libmysqlclient12 as default i made symbolic link like this: ln -s libmysqlclient.so.14.0.0 libmysqlclient.so.12 problem fixed. I would strongly recommend against doing this as any difference in the API or object layouts will now possibly cause PHP scripts to crash. You really should rebuild one or the other of Python MySQL module or PHP so they have both been compiled against the same version library. Graham
Form of req.filename/req.hlist.directory on Win32 systems.
I am sure I asked this a long time ago, but have forgotten all the details. On Win32 systems does req.filename set by Apache always use POSIX style forward slashes, ie., '/', to separate components of a directory? Thus: /some/path How does Apache indicate a drive letter when one is necessary? Is it: c:/some/path Does any of the above change based on whether forward or backward slashes are used in a Directory directive? Ie., ... ... Or does Apache not allow the latter anyway? If Apache does allow the latter, does that mean that req.hlist.directory is coming through set including backslashes rather than forward slashes. I want to get my head around this all again as at different times the values of req.filename and req.hlist.directory are used to determine the Python interpreter name. As highlighted in: http://issues.apache.org/jira/browse/MODPYTHON-161 If there is a mix of conventions, with user code also being able to affect these values, there may be no consistency and thus could end up with scenarios where a different interpreter to one than was expected will be used. Any help from Win32 users in understanding all this would be much appreciated. Thanks. Graham
Re: Form of req.filename/req.hlist.directory on Win32 systems.
Was this with mod_python from subversion or 3.2.8? Want to qualify whether latest set of changes I checked in to support Files directive has caused it to behave differently as how it determines req.hlist.directory is different to before. Thanks. Graham On 18/04/2006, at 4:33 AM, Nicolas Lehuen wrote: Hi Graham, Here is the test handler I've used : from mod_python import apache def handler(req): req.content_type = 'text/plain' req.write(req.hlist.directory+'\n') req.write(req.filename+'\n' ) return apache.OK If I use : DocumentRoot "c:\\apache22\\htdocs" # ... SetHandler mod_python PythonHandler test_handler I get, when calling http://localhost/index.html: c:\apache22\htdocs/ C:/apache22/htdocs/index.htmlNote that the drive letter has been uppercased and req.filename normalized to POSIX path names. req.hlist.directory, though supported by Win32, looks weird. Now with : DocumentRoot "c:/apache22/htdocs" # ... SetHandler mod_python PythonHandler test_handler I get : c:/apache22/htdocs/ C:/apache22/htdocs/index.html With : DocumentRoot "c:/apache22/htdocs" # ... SetHandler mod_python PythonHandler test_handler I get : c:\apache22\htdocs/ C:/apache22/htdocs/index.html And finally with : DocumentRoot "c:\\apache22\\htdocs" # ... SetHandler mod_python PythonHandler test_handler I get : c:/apache22/htdocs/ C:/apache22/htdocs/index.html So req.filename seems always normalized while req.hlist.directory reflects what was entered in the Directory tag. Both POSIX and Windows forms are allowed, unfortunately, but the backslash forms needs C-style escaping, and IIRC the Apache documentation recommends using forward slashes. Regards, Nicolas 2006/4/16, Graham Dumpleton <[EMAIL PROTECTED]>: I am sure I asked this a long time ago, but have forgotten all the details. On Win32 systems does req.filename set by Apache always use POSIX style forward slashes, ie., '/', to separate components of a directory? Thus: /some/path How does Apache indicate a drive letter when one is necessary? Is it: c:/some/path Does any of the above change based on whether forward or backward slashes are used in a Directory directive? Ie., ... ... Or does Apache not allow the latter anyway? If Apache does allow the latter, does that mean that req.hlist.directory is coming through set including backslashes rather than forward slashes. I want to get my head around this all again as at different times the values of req.filename and req.hlist.directory are used to determine the Python interpreter name. As highlighted in: http://issues.apache.org/jira/browse/MODPYTHON-161 If there is a mix of conventions, with user code also being able to affect these values, there may be no consistency and thus could end up with scenarios where a different interpreter to one than was expected will be used. Any help from Win32 users in understanding all this would be much appreciated. Thanks. Graham
Re: svn commit: r394455 - in /httpd/mod_python/trunk: Doc/appendixc.tex src/hlist.c src/include/hlist.h src/include/mod_python.h src/include/mod_python.h.in src/mod_python.c src/requestobject.c test/h
On 20/04/2006, at 12:39 AM, Jim Gallacher wrote: [EMAIL PROTECTED] wrote: Author: grahamd Date: Sun Apr 16 03:49:39 2006 New Revision: 394455 > URL: http://svn.apache.org/viewcvs?rev=394455&view=rev +1 Debian Sid, apache 2.2.0, python 2.4.2 -1 Debian Sid, apache 2.0.55, python 2.3.5 Compilation fails with this output: make[1]: Entering directory `/tmp/mod_python/src' Compiling for DSO. /usr/bin/apxs2 -I/tmp/mod_python/src/include -I/usr/include/apache2 -I/usr/include/python2.3 -c mod_python.c _apachemodule.c requestobject.c tableobject.c util.c serverobject.c connobject.c filterobject.c hlist.c hlistobject.c -L/usr/lib/python2.3/config -Xlinker -export-dynamic -lm -lpython2.3 -lpthread -ldl -lutil -lm /usr/bin/libtool --silent --mode=compile gcc -prefer-pic -pipe -I/usr/include/xmltok -I/usr/include/openssl -Wall -g -O2 -DAP_HAVE_DESIGNATED_INITIALIZER -DLINUX=2 -D_REENTRANT -D_XOPEN_SOURCE=500 -D_BSD_SOURCE -D_SVID_SOURCE -D_GNU_SOURCE -pipe -I/usr/include/xmltok -I/usr/include/openssl -Wall -g -O2 -pthread -I/usr/include/apache2 -I/usr/include/apr-0 -I/usr/include/apr-0 -I/usr/include -I/tmp/mod_python/src/include -I/usr/include/apache2 -I/usr/include/python2.3 -c -o mod_python.lo mod_python.c && touch mod_python.slo In file included from mod_python.c:28: /tmp/mod_python/src/include/mod_python.h:65: error: syntax error before 'ap_regex_t' /tmp/mod_python/src/include/mod_python.h:65: warning: useless type name in empty declaration In file included from mod_python.c:28: /tmp/mod_python/src/include/mod_python.h:66:1: warning: "AP_REG_EXTENDED" redefined The AP_REG_EXTENDED macro shouldn't exist in Apache 2.0.55. It, along with ap_regex_t only came along in Apache 2.1.? sometime. The code in mod_python.h which tries to accommodate the change is: #if !AP_MODULE_MAGIC_AT_LEAST(20050127,0) typedef regex_t ap_regex_t; #define AP_REG_EXTENDED REG_EXTENDED #define AP_REG_ICASE REG_ICASE #endif The regex_t type comes from pcreposix.h which is include by httpd.h which is included in mod_python.h prior to this point. Either how I am using AP_MODULE_MAGIC_AT_LEAST is wrong, of you must be picking up a wrong header file somehow. Bar ensuring you did a make distclean between builds, not sure what else to suggest. Strange. Graham In file included from /usr/include/apache2/httpd.h:44, from /tmp/mod_python/src/include/mod_python.h:42, from mod_python.c:28: /usr/include/apache2/ap_regex.h:44:1: warning: this is the location of the previous definition In file included from mod_python.c:28: /tmp/mod_python/src/include/mod_python.h:67:1: warning: "AP_REG_ICASE" redefined In file included from /usr/include/apache2/httpd.h:44, from /tmp/mod_python/src/include/mod_python.h:42, from mod_python.c:28: /usr/include/apache2/ap_regex.h:32:1: warning: this is the location of the previous definition In file included from /usr/include/python2.3/Python.h:8, from /tmp/mod_python/src/include/mod_python.h:75, from mod_python.c:28: /usr/include/python2.3/pyconfig.h:853:1: warning: "_POSIX_C_SOURCE" redefined In file included from /usr/include/sys/types.h:27, from /usr/include/apr-0/apr.h:113, from /usr/include/apache2/ap_config.h:20, from /usr/include/apache2/httpd.h:30, from /tmp/mod_python/src/include/mod_python.h:42, from mod_python.c:28: /usr/include/features.h:150:1: warning: this is the location of the previous definition mod_python.c: In function 'determine_context': mod_python.c:938: error: 'REG_EXTENDED' undeclared (first use in this function) mod_python.c:938: error: (Each undeclared identifier is reported only once mod_python.c:938: error: for each function it appears in.) mod_python.c: In function 'directive_PythonHandlerModule': mod_python.c:2264: warning: unused variable 'srv_conf' mod_python.c: In function 'PythonChildInitHandler': mod_python.c:2504: warning: unused variable 'ppath' apxs:Error: Command failed with rc=65536 No time to dig in to this right now. Will investigate later. Jim
