Re: 3.2

[Nicolas Lehuen]

Hi,

I've marked #45 as resolved, since the FileSession class has been
checked in for a while now. We only need a bit of documentation, but a
few seconds of source browsing can help an adventurous user to see how
to use FileSession. Now, what should we do about the new
req.get_session() API ? Is it ready for release now ?

This leaves use with #8 and #54, which are both related to importing
issues. Therefore I suggest we move those issues to mod_python 3.3 and
try to release 3.2 ASAP... A lot of bug fixes have been waiting 9
months or so to be fixed "officially".

Regards,
Nicolas2005/7/28, Gregory (Grisha) Trubetskoy <[EMAIL PROTECTED]>:
I just looked on JIRA, and there are 3 issues marked for 3.2.
http://issues.apache.org/jira/secure/IssueNavigator.jspa?reset=true&mode=hide&sorter/order=DESC&sorter/field=priority&resolutionIds=-1&pid=10640&fixfor=11060I don't see any of them as show stopping really, IMHO we should be able to
release 3.2 beta as it is now.Could we have a little vote on this?Thanks!

Re: 3.2

[Nicolas Lehuen]

Note that there are 29 unscheduled issues :

http://issues.apache.org/jira/secure/IssueNavigator.jspa?reset=true&mode=hide&sorter/order=DESC&sorter/field=priority&resolutionIds=-1&pid=10640&fixfor=-1


Maybe some of them should be included in the 3.2 release ?

Regards,
Nicolas2005/7/28, Nicolas Lehuen <[EMAIL PROTECTED]>:
Hi,

I've marked #45 as resolved, since the FileSession class has been
checked in for a while now. We only need a bit of documentation, but a
few seconds of source browsing can help an adventurous user to see how
to use FileSession. Now, what should we do about the new
req.get_session() API ? Is it ready for release now ?

This leaves use with #8 and #54, which are both related to importing
issues. Therefore I suggest we move those issues to mod_python 3.3 and
try to release 3.2 ASAP... A lot of bug fixes have been waiting 9
months or so to be fixed "officially".

Regards,
Nicolas2005/7/28, Gregory (Grisha) Trubetskoy <[EMAIL PROTECTED]>:

I just looked on JIRA, and there are 3 issues marked for 3.2.

http://issues.apache.org/jira/secure/IssueNavigator.jspa?reset=true&mode=hide&sorter/order=DESC&sorter/field=priority&resolutionIds=-1&pid=10640&fixfor=11060I don't see any of them as show stopping really, IMHO we should be able to
release 3.2 beta as it is now.Could we have a little vote on this?Thanks!

Re: 3.2

[Gregory (Grisha) Trubetskoy]

On Thu, 28 Jul 2005, Nicolas Lehuen wrote:


Note that there are 29 unscheduled issues :

http://issues.apache.org/jira/secure/IssueNavigator.jspa?reset=true&mode=hide&sorter/order=DESC&sorter/field=priority&resolutionIds=-1&pid=10640&fixfor=-1

Maybe some of them should be included in the 3.2 release ?


My inclanation is to just release whatever we have, and mark it as a beta 
release. The last "true" release was 3.1.3 in Feb 2004, which makes it 18 
months (if my math is correct)


Grisha

Re: 3.2

[Nicolas Lehuen]

OK, +1 for me.

Regards,
Nicolas2005/7/28, Gregory (Grisha) Trubetskoy <[EMAIL PROTECTED]>:
On Thu, 28 Jul 2005, Nicolas Lehuen wrote:> Note that there are 29 unscheduled issues :>> 
http://issues.apache.org/jira/secure/IssueNavigator.jspa?reset=true&mode=hide&sorter/order=DESC&sorter/field=priority&resolutionIds=-1&pid=10640&fixfor=-1>> Maybe some of them should be included in the 
3.2 release ?My inclanation is to just release whatever we have, and mark it as a betarelease. The last "true" release was 3.1.3 in Feb 2004, which makes it 18months (if my math is correct)
Grisha

Re: 3.2

[dharana]

As a non-developer, just as an user:

+1

Nicolas Lehuen wrote:

OK, +1 for me.

Regards,
Nicolas

2005/7/28, Gregory (Grisha) Trubetskoy <[EMAIL PROTECTED] 
<mailto:[EMAIL PROTECTED]>>:



On Thu, 28 Jul 2005, Nicolas Lehuen wrote:

 > Note that there are 29 unscheduled issues :
 >
 >

http://issues.apache.org/jira/secure/IssueNavigator.jspa?reset=true&mode=hide&sorter/order=DESC&sorter/field=priority&resolutionIds=-1&pid=10640&fixfor=-1

<http://issues.apache.org/jira/secure/IssueNavigator.jspa?reset=true&mode=hide&sorter/order=DESC&sorter/field=priority&resolutionIds=-1&pid=10640&fixfor=-1>
 >
 > Maybe some of them should be included in the 3.2 release ?

My inclanation is to just release whatever we have, and mark it as a
beta
release. The last "true" release was 3.1.3 in Feb 2004, which makes
it 18
months (if my math is correct)

Grisha




--
dharana

Re: 3.2

[Jim Gallacher]

Gregory (Grisha) Trubetskoy wrote:


On Thu, 28 Jul 2005, Nicolas Lehuen wrote:


Note that there are 29 unscheduled issues :

http://issues.apache.org/jira/secure/IssueNavigator.jspa?reset=true&mode=hide&sorter/order=DESC&sorter/field=priority&resolutionIds=-1&pid=10640&fixfor=-1 



Maybe some of them should be included in the 3.2 release ?



My inclanation is to just release whatever we have, and mark it as a 
beta release. The last "true" release was 3.1.3 in Feb 2004, which makes 
it 18 months (if my math is correct)


Grisha



I've either commited fixes or have fixes ready for 6 or 8 of those 
issues. Also there some that don't apply to 3.2 (eg website or mailing 
list issues). Must run right now but will make a list this evening of 
issues which can be closed.


Jim

Re: 3.2

[Gregory (Grisha) Trubetskoy]

On Thu, 28 Jul 2005, Jim Gallacher wrote:

I've either commited fixes or have fixes ready for 6 or 8 of those issues. 
Also there some that don't apply to 3.2 (eg website or mailing list issues). 
Must run right now but will make a list this evening of issues which can be 
closed.


Do you think you'll be able to create a tgz file? It would really be 
fantastic if someone other than me could do it.


Grisha

Re: 3.2

[Jim Gallacher]

Gregory (Grisha) Trubetskoy wrote:


On Thu, 28 Jul 2005, Jim Gallacher wrote:

I've either commited fixes or have fixes ready for 6 or 8 of those 
issues. Also there some that don't apply to 3.2 (eg website or mailing 
list issues). Must run right now but will make a list this evening of 
issues which can be closed.



Do you think you'll be able to create a tgz file? It would really be 
fantastic if someone other than me could do it.


Grisha



Yes I can do that. I've already done a couple of dry runs with good 
success.


Jim

Re: 3.2

[Jim Gallacher]

Nicolas Lehuen wrote:

Hi,

I've marked #45 as resolved, since the FileSession class has been 
checked in for a while now. We only need a bit of documentation, but a 
few seconds of source browsing can help an adventurous user to see how 
to use FileSession. Now, what should we do about the new 
req.get_session() API ? Is it ready for release now ?


No. Graham has raised some valid concerns about the current 
implentation. See http://issues.apache.org/jira/browse/MODPYTHON-59


I haven't had time to address these issues properly, and I'd rather see 
it pushed into 3.3 rather than release a half-baked solution in 3.2.


I don't want to leave req_get_session in as-is since people may start 
using it which would make things difficult if the final solution ends up 
being something completely different. Rather than delete 
req_get_session() completely I'd like to change it to
raise a NotImplemented error. That way there is starting point for 3.3 
but nobody will depend on it actually being there. Does this seem 
reasonable?


Regards,
Jim

Re: 3.2

[Gregory (Grisha) Trubetskoy]

On Thu, 28 Jul 2005, Jim Gallacher wrote:

I don't want to leave req_get_session in as-is since people may start using 
it which would make things difficult if the final solution ends up being 
something completely different. Rather than delete req_get_session() 
completely I'd like to change it to
raise a NotImplemented error. That way there is starting point for 3.3 but 
nobody will depend on it actually being there. Does this seem reasonable?


+1

Grisha

Re: 3.2

[Jim Gallacher]

Gregory (Grisha) Trubetskoy wrote:


On Thu, 28 Jul 2005, Jim Gallacher wrote:

I don't want to leave req_get_session in as-is since people may start 
using it which would make things difficult if the final solution ends 
up being something completely different. Rather than delete 
req_get_session() completely I'd like to change it to
raise a NotImplemented error. That way there is starting point for 3.3 
but nobody will depend on it actually being there. Does this seem 
reasonable?



+1

Grisha



I've commited this change. get_session implementation will be deferred 
to 3.3.


Jim

Re: 3.2

[Jim Gallacher]

Jim Gallacher wrote:

Gregory (Grisha) Trubetskoy wrote:



On Thu, 28 Jul 2005, Nicolas Lehuen wrote:


Note that there are 29 unscheduled issues :

http://issues.apache.org/jira/secure/IssueNavigator.jspa?reset=true&mode=hide&sorter/order=DESC&sorter/field=priority&resolutionIds=-1&pid=10640&fixfor=-1 



Maybe some of them should be included in the 3.2 release ?




My inclanation is to just release whatever we have, and mark it as a 
beta release. The last "true" release was 3.1.3 in Feb 2004, which 
makes it 18 months (if my math is correct)


Grisha



I've either commited fixes or have fixes ready for 6 or 8 of those 
issues. Also there some that don't apply to 3.2 (eg website or mailing 
list issues). Must run right now but will make a list this evening of 
issues which can be closed.


Jim



Here is my list. I think you can close all of these JIRA issues except 
MODPYTHON-52.


http://issues.apache.org/jira/browse/MODPYTHON-45
Implement a file-based session manager
Resolved but waiting for documentation. Working on it now - will commit 
in the next 12 hours.


http://issues.apache.org/jira/browse/MODPYTHON-58
_apache._global_lock results in segfault when index > number of mutexes
Fix has been commited

http://issues.apache.org/jira/browse/MODPYTHON-62
local_ip and local_host in connection object returns remote_ip and 
remote_host instead

This issue only applies to 3.1.4. It's already been fixed in 3.2.

http://issues.apache.org/jira/browse/MODPYTHON-65
3.2 working version will not install on Mac OS X (10.3.7)
Fix has been commited.

http://issues.apache.org/jira/browse/MODPYTHON-66
install_dso target also tries to install Python code files
Fix has been commited.

http://issues.apache.org/jira/browse/MODPYTHON-59
Add get_session() method to request object
Let's defer this to 3.3. I've changed current implementation to raise a 
NotImplemented error.


Related to get_session, I've made a small change to Session.Session(). 
It now checks PythonOption session for the default session type before 
using the hard coded default. For reasons that escape me I put this in a 
separate function, create_session(), but it really belongs in Session(). 
This is useful outside of get_session, so I've kept the change for 3.2.


Regards,
Jim

Re: 3.2

[Jim Gallacher]

Gregory (Grisha) Trubetskoy wrote:


I just looked on JIRA, and there are 3 issues marked for 3.2.

http://issues.apache.org/jira/secure/IssueNavigator.jspa?reset=true&mode=hide&sorter/order=DESC&sorter/field=priority&resolutionIds=-1&pid=10640&fixfor=11060 



I don't see any of them as show stopping really, IMHO we should be able 
to release 3.2 beta as it is now.


Could we have a little vote on this?

Thanks!



+1

Re: 3.2

[Graham Dumpleton]

Sorry, getting a bit overwhelmed with all these rapid changes in state
of things as bit busy today. Will probably will only know what is going
on when I look at latest code in repository. Then I'll probably pipe in
with some more pertinent comments about 3.2.

One report I would like to get some confirmation about is:

  http://issues.apache.org/jira/browse/MODPYTHON-34

This was about a potential security hole. Nicolas has indicated that
changes he had been working on which involved a different module
loader addressed it, but if various things are getting pushed to 3.3,
is then this addressed or not in what would make it to 3.2?

Since it was security related, just wanted to highlight it even if
exposure risk is low.

BTW, did anyone see an issue with my proposal for making the
req.path_info attribute writable. Ie.,

  http://issues.apache.org/jira/browse/MODPYTHON-67

I have been working on some code which would depend on such a
change and the code is of a nature that I think would be a nice
candidate for rolling back into the mod_python core. If path_info
was writable in 3.2, would allow people to experiment with the code
I am working on, or even turn it into a common project, without
needing to patch the mod_python source code.

I know Jim gave a +1 for path_info to be writable.

Graham

Jim Gallacher wrote ..
> Jim Gallacher wrote:
> > Gregory (Grisha) Trubetskoy wrote:
> > 
> >>
> >> On Thu, 28 Jul 2005, Nicolas Lehuen wrote:
> >>
> >>> Note that there are 29 unscheduled issues :
> >>>
> >>> http://issues.apache.org/jira/secure/IssueNavigator.jspa?reset=true&mode=hide&sorter/order=DESC&sorter/field=priority&resolutionIds=-1&pid=10640&fixfor=-1
> >>>
> >>>
> >>> Maybe some of them should be included in the 3.2 release ?
> >>
> >>
> >>
> >> My inclanation is to just release whatever we have, and mark it as a
> >> beta release. The last "true" release was 3.1.3 in Feb 2004, which 
> >> makes it 18 months (if my math is correct)
> >>
> >> Grisha
> >>
> > 
> > I've either commited fixes or have fixes ready for 6 or 8 of those 
> > issues. Also there some that don't apply to 3.2 (eg website or mailing
> > list issues). Must run right now but will make a list this evening of
> > issues which can be closed.
> > 
> > Jim
> > 
> 
> Here is my list. I think you can close all of these JIRA issues except
> MODPYTHON-52.
> 
> http://issues.apache.org/jira/browse/MODPYTHON-45
> Implement a file-based session manager
> Resolved but waiting for documentation. Working on it now - will commit
> in the next 12 hours.
> 
> http://issues.apache.org/jira/browse/MODPYTHON-58
> _apache._global_lock results in segfault when index > number of mutexes
> Fix has been commited
> 
> http://issues.apache.org/jira/browse/MODPYTHON-62
> local_ip and local_host in connection object returns remote_ip and 
> remote_host instead
> This issue only applies to 3.1.4. It's already been fixed in 3.2.
> 
> http://issues.apache.org/jira/browse/MODPYTHON-65
> 3.2 working version will not install on Mac OS X (10.3.7)
> Fix has been commited.
> 
> http://issues.apache.org/jira/browse/MODPYTHON-66
> install_dso target also tries to install Python code files
> Fix has been commited.
> 
> http://issues.apache.org/jira/browse/MODPYTHON-59
> Add get_session() method to request object
> Let's defer this to 3.3. I've changed current implementation to raise a
> NotImplemented error.
> 
> Related to get_session, I've made a small change to Session.Session().
> It now checks PythonOption session for the default session type before
> using the hard coded default. For reasons that escape me I put this in
> a 
> separate function, create_session(), but it really belongs in Session().
> This is useful outside of get_session, so I've kept the change for 3.2.
> 
> Regards,
> Jim

Re: 3.2

[Jim Gallacher]

Jim Gallacher wrote:

Jim Gallacher wrote:


Gregory (Grisha) Trubetskoy wrote:



On Thu, 28 Jul 2005, Nicolas Lehuen wrote:


Note that there are 29 unscheduled issues :

http://issues.apache.org/jira/secure/IssueNavigator.jspa?reset=true&mode=hide&sorter/order=DESC&sorter/field=priority&resolutionIds=-1&pid=10640&fixfor=-1 



Maybe some of them should be included in the 3.2 release ?





My inclanation is to just release whatever we have, and mark it as a 
beta release. The last "true" release was 3.1.3 in Feb 2004, which 
makes it 18 months (if my math is correct)


Grisha



I've either commited fixes or have fixes ready for 6 or 8 of those 
issues. Also there some that don't apply to 3.2 (eg website or mailing 
list issues). Must run right now but will make a list this evening of 
issues which can be closed.


Jim



Here is my list. I think you can close all of these JIRA issues except 
MODPYTHON-52.


That should read:
"Here is my list. I think you can close all of these JIRA issues except 
MODPYTHON-59."


Not sure how *that* typo crept in. 2 is nowhere near 9 on the keyboard.

Jim



http://issues.apache.org/jira/browse/MODPYTHON-45
Implement a file-based session manager
Resolved but waiting for documentation. Working on it now - will commit 
in the next 12 hours.


http://issues.apache.org/jira/browse/MODPYTHON-58
_apache._global_lock results in segfault when index > number of mutexes
Fix has been commited

http://issues.apache.org/jira/browse/MODPYTHON-62
local_ip and local_host in connection object returns remote_ip and 
remote_host instead

This issue only applies to 3.1.4. It's already been fixed in 3.2.

http://issues.apache.org/jira/browse/MODPYTHON-65
3.2 working version will not install on Mac OS X (10.3.7)
Fix has been commited.

http://issues.apache.org/jira/browse/MODPYTHON-66
install_dso target also tries to install Python code files
Fix has been commited.

http://issues.apache.org/jira/browse/MODPYTHON-59
Add get_session() method to request object
Let's defer this to 3.3. I've changed current implementation to raise a 
NotImplemented error.


Related to get_session, I've made a small change to Session.Session(). 
It now checks PythonOption session for the default session type before 
using the hard coded default. For reasons that escape me I put this in a 
separate function, create_session(), but it really belongs in Session(). 
This is useful outside of get_session, so I've kept the change for 3.2.


Regards,
Jim

Re: 3.2

[Gregory (Grisha) Trubetskoy]

Just thought I'd ask if we're making any progress towards a 3.2 tarball to 
test. No pressure, just curious :-)


Grisha

Re: 3.2

[Nicolas Lehuen]

MODPYTHON-34 has been fixed in the current version of the publisher,
with the new importing system. As I've written before, I can roll back
the part regarding the import system if you really want that, all the
while maintaining the fix for MODPYTHON-34.

Regards,
Nicolas2005/7/29, Graham Dumpleton <[EMAIL PROTECTED]>:
Sorry, getting a bit overwhelmed with all these rapid changes in stateof things as bit busy today. Will probably will only know what is goingon when I look at latest code in repository. Then I'll probably pipe in
with some more pertinent comments about 3.2.One report I would like to get some confirmation about is:  http://issues.apache.org/jira/browse/MODPYTHON-34
This was about a potential security hole. Nicolas has indicated thatchanges he had been working on which involved a different moduleloader addressed it, but if various things are getting pushed to 3.3,
is then this addressed or not in what would make it to 3.2?Since it was security related, just wanted to highlight it even ifexposure risk is low.BTW, did anyone see an issue with my proposal for making the
req.path_info attribute writable. Ie.,  http://issues.apache.org/jira/browse/MODPYTHON-67I have been working on some code which would depend on such a
change and the code is of a nature that I think would be a nicecandidate for rolling back into the mod_python core. If path_infowas writable in 3.2, would allow people to experiment with the codeI am working on, or even turn it into a common project, without
needing to patch the mod_python source code.I know Jim gave a +1 for path_info to be writable.GrahamJim Gallacher wrote ..> Jim Gallacher wrote:> > Gregory (Grisha) Trubetskoy wrote:
> >> >>> >> On Thu, 28 Jul 2005, Nicolas Lehuen wrote:> >>> >>> Note that there are 29 unscheduled issues :> >>>> >>> 
http://issues.apache.org/jira/secure/IssueNavigator.jspa?reset=true&mode=hide&sorter/order=DESC&sorter/field=priority&resolutionIds=-1&pid=10640&fixfor=-1> >>>> >>>
> >>> Maybe some of them should be included in the 3.2 release ?> >>> >>> >>> >> My inclanation is to just release whatever we have, and mark it as a
> >> beta release. The last "true" release was 3.1.3 in Feb 2004, which> >> makes it 18 months (if my math is correct)> >>> >> Grisha> >>> >
> > I've either commited fixes or have fixes ready for 6 or 8 of those> > issues. Also there some that don't apply to 3.2 (eg website or mailing> > list issues). Must run right now but will make a list this evening of
> > issues which can be closed.> >> > Jim> >>> Here is my list. I think you can close all of these JIRA issues except> MODPYTHON-52.>> 
http://issues.apache.org/jira/browse/MODPYTHON-45> Implement a file-based session manager> Resolved but waiting for documentation. Working on it now - will commit> in the next 12 hours.>
> http://issues.apache.org/jira/browse/MODPYTHON-58> _apache._global_lock results in segfault when index > number of mutexes> Fix has been commited
>> http://issues.apache.org/jira/browse/MODPYTHON-62> local_ip and local_host in connection object returns remote_ip and> remote_host instead
> This issue only applies to 3.1.4. It's already been fixed in 3.2.>> http://issues.apache.org/jira/browse/MODPYTHON-65> 3.2 working version will not install on Mac OS X (
10.3.7)> Fix has been commited.>> http://issues.apache.org/jira/browse/MODPYTHON-66> install_dso target also tries to install Python code files
> Fix has been commited.>> http://issues.apache.org/jira/browse/MODPYTHON-59> Add get_session() method to request object> Let's defer this to 
3.3. I've changed current implementation to raise a> NotImplemented error.>> Related to get_session, I've made a small change to Session.Session().> It now checks PythonOption session for the default session type before
> using the hard coded default. For reasons that escape me I put this in> a> separate function, create_session(), but it really belongs in Session().> This is useful outside of get_session, so I've kept the change for 
3.2.>> Regards,> Jim

Re: 3.2

[Nicolas Lehuen]

As for MODPYTHON-67, I've just reviewed your patch, +1ed it and checked
it in. The only problem I could imagine about it is that there is no
downward compatibility (code for 3.2.x would not run on 3.1.x), but I
think this is not a big problem.

Regards,
Nicolas2005/7/29, Graham Dumpleton <[EMAIL PROTECTED]>:
Sorry, getting a bit overwhelmed with all these rapid changes in stateof things as bit busy today. Will probably will only know what is goingon when I look at latest code in repository. Then I'll probably pipe in
with some more pertinent comments about 3.2.One report I would like to get some confirmation about is:  http://issues.apache.org/jira/browse/MODPYTHON-34
This was about a potential security hole. Nicolas has indicated thatchanges he had been working on which involved a different moduleloader addressed it, but if various things are getting pushed to 3.3,
is then this addressed or not in what would make it to 3.2?Since it was security related, just wanted to highlight it even ifexposure risk is low.BTW, did anyone see an issue with my proposal for making the
req.path_info attribute writable. Ie.,  http://issues.apache.org/jira/browse/MODPYTHON-67I have been working on some code which would depend on such a
change and the code is of a nature that I think would be a nicecandidate for rolling back into the mod_python core. If path_infowas writable in 3.2, would allow people to experiment with the codeI am working on, or even turn it into a common project, without
needing to patch the mod_python source code.I know Jim gave a +1 for path_info to be writable.GrahamJim Gallacher wrote ..> Jim Gallacher wrote:> > Gregory (Grisha) Trubetskoy wrote:
> >> >>> >> On Thu, 28 Jul 2005, Nicolas Lehuen wrote:> >>> >>> Note that there are 29 unscheduled issues :> >>>> >>> 
http://issues.apache.org/jira/secure/IssueNavigator.jspa?reset=true&mode=hide&sorter/order=DESC&sorter/field=priority&resolutionIds=-1&pid=10640&fixfor=-1> >>>> >>>
> >>> Maybe some of them should be included in the 3.2 release ?> >>> >>> >>> >> My inclanation is to just release whatever we have, and mark it as a
> >> beta release. The last "true" release was 3.1.3 in Feb 2004, which> >> makes it 18 months (if my math is correct)> >>> >> Grisha> >>> >
> > I've either commited fixes or have fixes ready for 6 or 8 of those> > issues. Also there some that don't apply to 3.2 (eg website or mailing> > list issues). Must run right now but will make a list this evening of
> > issues which can be closed.> >> > Jim> >>> Here is my list. I think you can close all of these JIRA issues except> MODPYTHON-52.>> 
http://issues.apache.org/jira/browse/MODPYTHON-45> Implement a file-based session manager> Resolved but waiting for documentation. Working on it now - will commit> in the next 12 hours.>
> http://issues.apache.org/jira/browse/MODPYTHON-58> _apache._global_lock results in segfault when index > number of mutexes> Fix has been commited
>> http://issues.apache.org/jira/browse/MODPYTHON-62> local_ip and local_host in connection object returns remote_ip and> remote_host instead
> This issue only applies to 3.1.4. It's already been fixed in 3.2.>> http://issues.apache.org/jira/browse/MODPYTHON-65> 3.2 working version will not install on Mac OS X (
10.3.7)> Fix has been commited.>> http://issues.apache.org/jira/browse/MODPYTHON-66> install_dso target also tries to install Python code files
> Fix has been commited.>> http://issues.apache.org/jira/browse/MODPYTHON-59> Add get_session() method to request object> Let's defer this to 
3.3. I've changed current implementation to raise a> NotImplemented error.>> Related to get_session, I've made a small change to Session.Session().> It now checks PythonOption session for the default session type before
> using the hard coded default. For reasons that escape me I put this in> a> separate function, create_session(), but it really belongs in Session().> This is useful outside of get_session, so I've kept the change for 
3.2.>> Regards,> Jim

Re: 3.2

[Nicolas Lehuen]

Hi Jim,

Do you want me to mark the issues resolved ? Don't you have the rights
to do this ? If you don't have the right, then maybe Scott Sanders can
give them to you (and to Graham while we're at it...). Scott, are you
reading this ?

Regards,
Nicolas2005/7/29, Jim Gallacher <[EMAIL PROTECTED]>:
Jim Gallacher wrote:> Gregory (Grisha) Trubetskoy wrote:>>>>> On Thu, 28 Jul 2005, Nicolas Lehuen wrote:>>>>> Note that there are 29 unscheduled issues :>>>
>>> http://issues.apache.org/jira/secure/IssueNavigator.jspa?reset=true&mode=hide&sorter/order=DESC&sorter/field=priority&resolutionIds=-1&pid=10640&fixfor=-1
>>>>>>>>> Maybe some of them should be included in the 3.2 release ?>>>>>>>> My inclanation is to just release whatever we have, and mark it as a
>> beta release. The last "true" release was 3.1.3 in Feb 2004, which>> makes it 18 months (if my math is correct)>>>> Grisha>>>> I've either commited fixes or have fixes ready for 6 or 8 of those
> issues. Also there some that don't apply to 3.2 (eg website or mailing> list issues). Must run right now but will make a list this evening of> issues which can be closed.>> Jim>
Here is my list. I think you can close all of these JIRA issues exceptMODPYTHON-52.http://issues.apache.org/jira/browse/MODPYTHON-45Implement a file-based session manager
Resolved but waiting for documentation. Working on it now - will commitin the next 12 hours.http://issues.apache.org/jira/browse/MODPYTHON-58
_apache._global_lock results in segfault when index > number of mutexesFix has been commitedhttp://issues.apache.org/jira/browse/MODPYTHON-62
local_ip and local_host in connection object returns remote_ip andremote_host insteadThis issue only applies to 3.1.4. It's already been fixed in 3.2.
http://issues.apache.org/jira/browse/MODPYTHON-653.2 working version will not install on Mac OS X (10.3.7)Fix has been commited.http://issues.apache.org/jira/browse/MODPYTHON-66
install_dso target also tries to install Python code filesFix has been commited.http://issues.apache.org/jira/browse/MODPYTHON-59Add get_session() method to request object
Let's defer this to 3.3. I've changed current implementation to raise aNotImplemented error.Related to get_session, I've made a small change to Session.Session().It now checks PythonOption session for the default session type before
using the hard coded default. For reasons that escape me I put this in aseparate function, create_session(), but it really belongs in Session().This is useful outside of get_session, so I've kept the change for 
3.2.Regards,Jim

Re: 3.2

[Graham Dumpleton]

A prerequisite for full JIRA access is probably going to be sending in  
the
ASF committers agreement. I haven't done that as yet and so don't have  
SVN

access either. I am hoping that my work load will drop after I take a
holiday in September and I'll be able to finally start taking a more  
direct

hands on role and get the committers agreement submitted etc.

On 10/08/2005, at 8:58 PM, Nicolas Lehuen wrote:


Hi Jim,

 Do you want me to mark the issues resolved ? Don't you have the  
rights to do this ? If you don't have the right, then maybe Scott  
Sanders can give them to you (and to Graham while we're at it...).  
Scott, are you reading this ?


 Regards,
 Nicolas

2005/7/29, Jim Gallacher <[EMAIL PROTECTED]>:
 Jim Gallacher wrote:
> Gregory (Grisha) Trubetskoy wrote:
>
>>
>> On Thu, 28 Jul 2005, Nicolas Lehuen wrote:
>>
>>> Note that there are 29 unscheduled issues :
>>>
>>>  
http://issues.apache.org/jira/secure/IssueNavigator.jspa? 
reset=true&mode=hide&sorter/order=DESC&sorter/ 
field=priority&resolutionIds=-1&pid=10640&fixfor=-1

>>>
>>>
>>> Maybe some of them should be included in the 3.2 release ?
>>
>>
>>
>> My inclanation is to just release whatever we have, and mark it as a
>> beta release. The last "true" release was 3.1.3 in Feb 2004, which
>> makes it 18 months (if my math is correct)
>>
>> Grisha
>>
>
> I've either commited fixes or have fixes ready for 6 or 8 of those
> issues. Also there some that don't apply to 3.2 (eg website or  
mailing

> list issues). Must run right now but will make a list this evening of
> issues which can be closed.
>
> Jim
>

Here is my list. I think you can close all of these JIRA issues except
MODPYTHON-52.

http://issues.apache.org/jira/browse/MODPYTHON-45
Implement a file-based session manager
Resolved but waiting for documentation. Working on it now - will commit
in the next 12 hours.

http://issues.apache.org/jira/browse/MODPYTHON-58
_apache._global_lock results in segfault when index > number of mutexes
Fix has been commited

http://issues.apache.org/jira/browse/MODPYTHON-62
local_ip and local_host in connection object returns remote_ip and
remote_host instead
This issue only applies to 3.1.4. It's already been fixed in 3.2.

 http://issues.apache.org/jira/browse/MODPYTHON-65
3.2 working version will not install on Mac OS X (10.3.7)
Fix has been commited.

http://issues.apache.org/jira/browse/MODPYTHON-66
install_dso target also tries to install Python code files
Fix has been commited.

http://issues.apache.org/jira/browse/MODPYTHON-59
Add get_session() method to request object
Let's defer this to 3.3. I've changed current implementation to raise a
NotImplemented error.

Related to get_session, I've made a small change to Session.Session().
It now checks PythonOption session for the default session type before
using the hard coded default. For reasons that escape me I put this in  
a
separate function, create_session(), but it really belongs in  
Session().

This is useful outside of get_session, so I've kept the change for 3.2.

Regards,
Jim

3.2.x branch

[Gregory (Grisha) Trubetskoy]

On Thu, 9 Feb 2006, Jim Gallacher wrote:

Looks good. As soon as you make the official announcement I'll create 
branches/3.2.x in svn and we can start on 3.3.


I don't think we need to wait for the announcement - 3.2.7 is already out 
if you look on the download page, the announcement is more about 
"marketing" if you will.


Grisha

mod_python.publisher, HEAD and 3.2.

[Graham Dumpleton]

In a hurry, so a quick note. In 3.2, mod_python.publisher was changed 
to read:


if req.method!='HEAD':

# TODO : the problem is that a handler can still use 
req.write
# and break the assumption that nothing should be written 
with the

# HEAD method.
req.write(result)

I think before I said that not writing output for HEAD case was 
probably not
the best idea if Apache was truncated output later anyway. Now have a 
good
reason why mod_python.publisher shouldn't do this and why it should 
always

write output.

The reason is that output filters are run even if HEAD is used. This 
means
that if an output filter was being used which was doing something 
special
which required the full output, maybe a caching system, it isn't going 
to
see the full output because mod_python.publisher is restricting output 
for

HEAD.

Haven't been able to check JIRA as access to it unreliable at the moment
for me, so can't see what comments I made about this before or add to 
them.


Graham

Re: 3.2.x branch

[Jim Gallacher]

I've created the a 3.2.x branch. Just so we are all clear, the 
branches/3.2.x branch gets bug fixes *only*. This will be the source 
repository for future 3.2 bug fix releases. New features and major code 
modifications continue to go into trunk.


I think any other new branches we create such as a 3.3 experimental 
module import branch should start with an alpha character so the stable 
3.x.x branches  don't get lost in the noise. eg.


experimental-3.3

but not:

3.3-experimental

Jim

Gregory (Grisha) Trubetskoy wrote:



On Thu, 9 Feb 2006, Jim Gallacher wrote:

Looks good. As soon as you make the official announcement I'll create 
branches/3.2.x in svn and we can start on 3.3.



I don't think we need to wait for the announcement - 3.2.7 is already 
out if you look on the download page, the announcement is more about 
"marketing" if you will.


Grisha

Re: import issues for 3.2 (Was mod_python docs appendix A - changes in 3.2)

[Jim Gallacher]

I don't have a really have a clear understanding of all the issues with 
the import mechanism, but I'd rather see a correct fix rather than a 
quick fix.


Nicolas Lehuen wrote:

4) Graham, I don't know what it takes to be a member of the
development team, but to me you are one :).


+1 on that. I'm in awe of the amount of time Graham devotes to answering 
questions on the mailing lists and for me his input on python-dev has 
the same weight as any developer.


Thank you Graham. I've been meaning to say that for a while. :)

Regards,
Jim

Re: 3.2 beta release today?

[Gregory (Grisha) Trubetskoy]

I think the best thing to do is just to go ahead and tag and a create a 
tarball. Whether everyone was ready will become apparent during the 
testing/voting.


Grisha


On Mon, 15 Aug 2005, Jim Gallacher wrote:


Are we on track to release a 3.2.0beta tarball today?

Regards,
Jim

Re: 3.2 beta release today?

[Jim Gallacher]

Gregory (Grisha) Trubetskoy wrote:


I think the best thing to do is just to go ahead and tag and a create a 
tarball. Whether everyone was ready will become apparent during the 
testing/voting.




OK, I'll throw together some release notes (ie, changes from 3.1.x). Do 
you want me to tag it as well?


Subversion branch: tags/3-2-0-BETA1
mpversion.h: #define MPV_STRING "3.2.0-beta-1"

Jim



On Mon, 15 Aug 2005, Jim Gallacher wrote:


Are we on track to release a 3.2.0beta tarball today?

Regards,
Jim

Re: 3.2 beta release today?

[Gregory (Grisha) Trubetskoy]

On Mon, 15 Aug 2005, Jim Gallacher wrote:

OK, I'll throw together some release notes (ie, changes from 3.1.x). Do you 
want me to tag it as well?


Sure.


Subversion branch: tags/3-2-0-BETA1
mpversion.h: #define MPV_STRING "3.2.0-beta-1"


There shouldn't be a number after BETA (I did this before, which is where 
you may have gotten the idea, but it was wrong).


Tag it as 3-2-0-BETA. "BETA" simply denotes the quality of this release.

If it gets approved, then this is how we will release it. If problems are 
found, then they will be fixed and depending on the extent of the fixes 
another tarball may be released by pushing the tag forward, or creating 
another tag 3-2-1-BETA, and so on, until the public BETA release.


Grisha

Re: 3.2 beta release today?

[Jim Gallacher]

I was putting together the 3.2 beta release notes but ran out of gas. 
I'll finish tomorrow when I'm more awake.


In the mean time, are there any 3.1.4 > 3.2.0 upgrade issues which need 
to be documented? I can't think of any, but then I am half asleep. :)


Regards,
Jim

Re: 3.2 beta release today?

[Graham Dumpleton]

Apologies if this is a duplicate, mail delivery is screwing up real bad for me.

Grisha wrote ..
> > Subversion branch: tags/3-2-0-BETA1
> > mpversion.h: #define MPV_STRING "3.2.0-beta-1"
> 
> There shouldn't be a number after BETA (I did this before, which is where
> you may have gotten the idea, but it was wrong).
> 
> Tag it as 3-2-0-BETA. "BETA" simply denotes the quality of this release.
> 
> If it gets approved, then this is how we will release it. If problems are
> found, then they will be fixed and depending on the extent of the fixes
> another tarball may be released by pushing the tag forward, or creating
> another tag 3-2-1-BETA, and so on, until the public BETA release.

Does this imply no more fixes for outstanding problems even though
the necessary patches have been posted up on JIRA. Eg,.

  http://issues.apache.org/jira/browse/MODPYTHON-12

The patch for this one would have eliminated the last workaround
that I keep having to propagate in my code, even though the
minimum requirement for my code was going to be to have 3.2.

Guess I'll have to go back and put the workaround in again until
3.2.X. :-(

Graham

Re: 3.2 beta release today?

[Gregory (Grisha) Trubetskoy]

On Mon, 15 Aug 2005, Graham Dumpleton wrote:


Does this imply no more fixes for outstanding problems even though
the necessary patches have been posted up on JIRA. Eg,.


If we stick to the schedule, then yes.


 http://issues.apache.org/jira/browse/MODPYTHON-12

The patch for this one would have eliminated the last workaround
that I keep having to propagate in my code, even though the
minimum requirement for my code was going to be to have 3.2.

Guess I'll have to go back and put the workaround in again until
3.2.X. :-(


But you have an opportunity to vote -1 on the new tarball because of this 
issue (the rules are that anyone on this list can vote).


Grisha

Re: 3.2 beta release today?

[Gregory (Grisha) Trubetskoy]

On Mon, 15 Aug 2005, Jim Gallacher wrote:

In the mean time, are there any 3.1.4 > 3.2.0 upgrade issues which need to be 
documented? I can't think of any, but then I am half asleep. :)


Let that not slow you down (the upgrade issues part, not the half asleep 
one :-) ) - such a document can be put together separately and posted on 
the website, or included in the final 3.2. We won't know what all the 
issues are anyway until a few people actually try it.


Grisha

Re: 3.2 beta release today?

[Jim Gallacher]

Gregory (Grisha) Trubetskoy wrote:


On Mon, 15 Aug 2005, Jim Gallacher wrote:

In the mean time, are there any 3.1.4 > 3.2.0 upgrade issues which 
need to be documented? I can't think of any, but then I am half 
asleep. :)



Let that not slow you down (the upgrade issues part, not the half asleep 
one :-) ) - such a document can be put together separately and posted on 
the website, or included in the final 3.2. We won't know what all the 
issues are anyway until a few people actually try it.


I wasn't going to wait on it. I just thought if anything turned up in my 
inbox today I would include it. I need to look after a couple of 
customers after lunch, but should be able to package the beta 3 or 4 
hours from now.


Jim

Re: 3.2 beta release today?

[Jim Gallacher]

Gregory (Grisha) Trubetskoy wrote:


On Mon, 15 Aug 2005, Graham Dumpleton wrote:


Does this imply no more fixes for outstanding problems even though
the necessary patches have been posted up on JIRA. Eg,.



If we stick to the schedule, then yes.


 http://issues.apache.org/jira/browse/MODPYTHON-12

The patch for this one would have eliminated the last workaround
that I keep having to propagate in my code, even though the
minimum requirement for my code was going to be to have 3.2.

Guess I'll have to go back and put the workaround in again until
3.2.X. :-(



But you have an opportunity to vote -1 on the new tarball because of 
this issue (the rules are that anyone on this list can vote).




It does not look like a major patch. The diff is only a few lines. I can 
commit it as the last act before shoving the beta out the door today.


Jim

Re: 3.2 beta release today?

[Jim Gallacher]

Gregory (Grisha) Trubetskoy wrote:


On Mon, 15 Aug 2005, Graham Dumpleton wrote:


Does this imply no more fixes for outstanding problems even though
the necessary patches have been posted up on JIRA. Eg,.



If we stick to the schedule, then yes.


 http://issues.apache.org/jira/browse/MODPYTHON-12

The patch for this one would have eliminated the last workaround
that I keep having to propagate in my code, even though the
minimum requirement for my code was going to be to have 3.2.

Guess I'll have to go back and put the workaround in again until
3.2.X. :-(




I've applied Graham's patch so he can turn that :-( upside down. :-)

I'm just doing my last revision of the release notes, and then it's tag 
and tar.


A couple of questions:

What is the correct directory name? mod_python-3.2.0-b, 
mod_python-3.2.0-BETA, or something else.


What is the correct name for the tarball?
mod_python-3.2.0-b.tgz, mod_python-3.2.0-BETA.tgz or something else?

Once I've got the tarball, do I just email it to you Grisha?

Jim

Re: 3.2 beta release today?

[Gregory (Grisha) Trubetskoy]

On Tue, 16 Aug 2005, Jim Gallacher wrote:


What is the correct name for the tarball?
mod_python-3.2.0-b.tgz, mod_python-3.2.0-BETA.tgz or something else?


I like "mod_python-3.2.0b" and the tarball would be mod_python-3.2.0b.tgz

Grisha

Publisher bug in 3.2 BETA.

[Graham Dumpleton]

In 3.2 beta, if one uses:

  SetHandler mod_python
  PythonHandler mod_python.publisher
  PythonDebug On

and you DO NOT have an index.py file and use a URL where the first
URL component doesn't map explicitly to a .py file, you will get
a Python error rather than a 404 NOT FOUND response. For example,
for URL "http://localhost:8080/~grahamd/mp32/index";, I get:

Mod_python error: "PythonHandler mod_python.publisher"

Traceback (most recent call last):

  File  
"/System/Library/Frameworks/Python.framework/Versions/2.3/lib/ 
python2.3/site-packages/mod_python/apache.py", line 299, in  
HandlerDispatch

result = object(req)

  File  
"/System/Library/Frameworks/Python.framework/Versions/2.3/lib/ 
python2.3/site-packages/mod_python/publisher.py", line 192, in handler

module = page_cache[req]

  File  
"/System/Library/Frameworks/Python.framework/Versions/2.3/lib/ 
python2.3/site-packages/mod_python/cache.py", line 77, in __getitem__

return self._checkitem(name)[2]

  File  
"/System/Library/Frameworks/Python.framework/Versions/2.3/lib/ 
python2.3/site-packages/mod_python/cache.py", line 118, in _checkitem

opened = self.check(key, name, entry)

  File  
"/System/Library/Frameworks/Python.framework/Versions/2.3/lib/ 
python2.3/site-packages/mod_python/publisher.py", line 67, in check

return ModuleCache.check(self, key, req, entry)

  File  
"/System/Library/Frameworks/Python.framework/Versions/2.3/lib/ 
python2.3/site-packages/mod_python/cache.py", line 249, in check

opened = file(key, self.mode)

IOError: [Errno 2] No such file or directory:  
'/Users/grahamd/Sites/mp32/index.py'

Re: 3.2 (import and publisher issues)

[Jim Gallacher]

Nicolas Lehuen wrote:
MODPYTHON-34 has been fixed in the current version of the publisher, 
with the new importing system. As I've written before, I can roll back 
the part regarding the import system if you really want that, all the 
while maintaining the fix for MODPYTHON-34.


I haven't had much to say on this topic (because I don't have a solid 
grasp of all the implications), but it seems like it's a real stumbling 
block for the release.


I'd say if the current changes won't break any current code, or cause 
users unexpected behaviour, leave them in. Otherwise roll them back. 
It's better for the user to have to deal with old, documentented issues 
rather than discover brand new ones in their code, only to find we've 
changed the behaviour again in 3.3.


Whichever route we take, we will obviously work on this further in 3.3 
but for now let's get 3.2.0b out the door.


Regards,
Jim

Re: 3.2 (import and publisher issues)

[Nicolas Lehuen]

I'd like to stress the fact that a lot of issues currently in JIRA are
related to the publisher. We have worked on it to solve some of these
issues. I have made sure that both the old version and the new version
pass a series of unit tests. I can't be sure that those unit tests
reflect the whole range of usage patterns people could have of the
publisher, but anyway, it's better than nothing.

So, I think we should move forward on the 3.2 release. The new
publisher code is meant to make it work better, not worse, while
retaining the compatibility with the current code. It's not a new
publisher, it's a set of bug fixes. I mean, what is the purpose of
releasing a new version of mod_python if we don't fix the dozen of bugs
that are related to the publisher ?

Regards,
Nicolas2005/8/10, Jim Gallacher <[EMAIL PROTECTED]>:
Nicolas Lehuen wrote:> MODPYTHON-34 has been fixed in the current version of the publisher,> with the new importing system. As I've written before, I can roll back> the part regarding the import system if you really want that, all the
> while maintaining the fix for MODPYTHON-34.I haven't had much to say on this topic (because I don't have a solidgrasp of all the implications), but it seems like it's a real stumblingblock for the release.
I'd say if the current changes won't break any current code, or causeusers unexpected behaviour, leave them in. Otherwise roll them back.It's better for the user to have to deal with old, documentented issues
rather than discover brand new ones in their code, only to find we'vechanged the behaviour again in 3.3.Whichever route we take, we will obviously work on this further in 3.3but for now let's get 3.2.0b
 out the door.Regards,Jim

Re: 3.2 (import and publisher issues)

[Jim Gallacher]

Nicolas Lehuen wrote:
I'd like to stress the fact that a lot of issues currently in JIRA are 
related to the publisher. We have worked on it to solve some of these 
issues. I have made sure that both the old version and the new version 
pass a series of unit tests. I can't be sure that those unit tests 
reflect the whole range of usage patterns people could have of the 
publisher, but anyway, it's better than nothing.


So, I think we should move forward on the 3.2 release.


+1

The new publisher 
code is meant to make it work better, not worse, while retaining the 
compatibility with the current code. It's not a new publisher, it's a 
set of bug fixes. I mean, what is the purpose of releasing a new version 
of mod_python if we don't fix the dozen of bugs that are related to the 
publisher ?


Regards,
Nicolas

2005/8/10, Jim Gallacher <[EMAIL PROTECTED] 
<mailto:[EMAIL PROTECTED]>>:


Nicolas Lehuen wrote:
 > MODPYTHON-34 has been fixed in the current version of the publisher,
 > with the new importing system. As I've written before, I can roll
back
 > the part regarding the import system if you really want that, all
the
 > while maintaining the fix for MODPYTHON-34.

I haven't had much to say on this topic (because I don't have a solid
grasp of all the implications), but it seems like it's a real stumbling
block for the release.

I'd say if the current changes won't break any current code, or cause
users unexpected behaviour, leave them in. Otherwise roll them back.
It's better for the user to have to deal with old, documentented issues
rather than discover brand new ones in their code, only to find we've
changed the behaviour again in 3.3.

Whichever route we take, we will obviously work on this further in 3.3
but for now let's get 3.2.0b out the door.

Regards,
Jim

3.2 Compile problems with gcc 4.0

[Jim Gallacher]

I updated dated my workstation today which included pulling in gcc 4.0.1 
and thought I'd give mod_python a whirl and how if they'd play together.


Compilation fails with the following:

requestobject.c: In function 'request_tp_dealloc':
requestobject.c:1482: warning: implicit declaration of function 
'request_tp_clear'

requestobject.c: At top level:
requestobject.c:1514: error: static declaration of 'request_tp_clear' 
follows non-static declaration
requestobject.c:1482: error: previous implicit declaration of 
'request_tp_clear' was here

requestobject.c: In function 'request_tp_clear':
requestobject.c:1528: warning: assignment from incompatible pointer type
apxs:Error: Command failed with rc=65536

Since gcc 3.3 sails right through, I'm going to assume gcc 4 is 
stricter.  It's easy enough to fix (and I will) by adding a function 
prototype for request_tp_clear before it is called in 
request_tp_dealloc(). eg.

 static int request_tp_clear(requestobject *self);

I've noticed that in general functions are implicitly declared in 
mod_python. Is there a technical reason for this, or is it "just one of 
those things"?


Also, since request_tp_clear is only used in requestobject.c, is it 
correct to put the declaration there or should it go in requestobject.h? 
 Ya, I know, my c-skills suck. :)


Regards,
Jim

Getting ready for 3.2 beta 2

[Jim Gallacher]

I think we should aim for the second beta release in the next couple of 
days. I have a few questions and a list of outstanding issues.


Name of tarball: mod_python-3.2.1b.tgz?
Also, I assume a new branch called tags/release-3.2.1-BETA will be 
created in subversion, correct?


Outstanding issues:

1. flex
   Fix is ready to commit pending some feedback on the warning message 
generated by configure.


2. MacOS compile
   Fixed in svn trunk.

3. Eliminate creation of mod_python_so.so in non-windows environments.
   Fix is ready to commit.

4. Fix segfault + memory leaks detailed in:
  http://issues.apache.org/jira/browse/MODPYTHON-75
  http://issues.apache.org/jira/browse/MODPYTHON-60
  Boyan's patch detailed in MODPYTHON-75 seems to fix both of these. 
Fix is ready to commit.


5. http://issues.apache.org/jira/browse/MODPYTHON-72
   Fix still required.

6. Publisher bug in 3.2 BETA, detailed by Graham in python-dev message 
posted 2005-08-21.

   Fix still required.

I haven't looked at the code involved in items 5 and 6, but hopefully 
the fixes will be fairly trivial.


Regards,
Jim

Re: Publisher bug in 3.2 BETA.

[Jim Gallacher]

I was hoping there would be a simple fix for this but a quick glance at 
the code makes me think that will not be the case. Also, I don't think 
this is a new bug as 3.1.4 does not generate a 404 NOT FOUND response 
either:


Mod_python error: "PythonHandler mod_python.publisher"

Traceback (most recent call last):

  File "/usr/lib/python2.3/site-packages/mod_python/apache.py", line 
299, in HandlerDispatch

result = object(req)

  File "/usr/lib/python2.3/site-packages/mod_python/publisher.py", line 
98, in handler

path=[path])

  File "/usr/lib/python2.3/site-packages/mod_python/apache.py", line 
454, in import_module

f, p, d = imp.find_module(parts[i], path)

ImportError: No module named index

We should create a new JIRA issue. If it turns out that the fix is not 
easy, is this a big enough problem to hold up the 3.2 release?


Regards,
Jim


Graham Dumpleton wrote:

In 3.2 beta, if one uses:

  SetHandler mod_python
  PythonHandler mod_python.publisher
  PythonDebug On

and you DO NOT have an index.py file and use a URL where the first
URL component doesn't map explicitly to a .py file, you will get
a Python error rather than a 404 NOT FOUND response. For example,
for URL "http://localhost:8080/~grahamd/mp32/index";, I get:

Mod_python error: "PythonHandler mod_python.publisher"

Traceback (most recent call last):

  File  "/System/Library/Frameworks/Python.framework/Versions/2.3/lib/ 
python2.3/site-packages/mod_python/apache.py", line 299, in  
HandlerDispatch

result = object(req)

  File  "/System/Library/Frameworks/Python.framework/Versions/2.3/lib/ 
python2.3/site-packages/mod_python/publisher.py", line 192, in handler

module = page_cache[req]

  File  "/System/Library/Frameworks/Python.framework/Versions/2.3/lib/ 
python2.3/site-packages/mod_python/cache.py", line 77, in __getitem__

return self._checkitem(name)[2]

  File  "/System/Library/Frameworks/Python.framework/Versions/2.3/lib/ 
python2.3/site-packages/mod_python/cache.py", line 118, in _checkitem

opened = self.check(key, name, entry)

  File  "/System/Library/Frameworks/Python.framework/Versions/2.3/lib/ 
python2.3/site-packages/mod_python/publisher.py", line 67, in check

return ModuleCache.check(self, key, req, entry)

  File  "/System/Library/Frameworks/Python.framework/Versions/2.3/lib/ 
python2.3/site-packages/mod_python/cache.py", line 249, in check

opened = file(key, self.mode)

IOError: [Errno 2] No such file or directory:  
'/Users/grahamd/Sites/mp32/index.py'

Re: Publisher bug in 3.2 BETA.

[Graham Dumpleton]

Unless I am missing something, the fix is quite trivial as it specially
affects just the case where "index.py" is explicitly added to 
req.filename

within the publisher code.

Index: lib/python/mod_python/publisher.py
===
--- lib/python/mod_python/publisher.py  (revision 240397)
+++ lib/python/mod_python/publisher.py  (working copy)
@@ -166,6 +166,9 @@
 # we'll just insert the module name index.py in the middle
 path, func_path = split(req.filename)
 req.filename = join(path, 'index.py')
+
+if not exists(req.filename):
+raise apache.SERVER_RETURN, apache.HTTP_NOT_FOUND

 # I don't know if it's still possible to have a path_info
 # but if we have one, we append it to the filename which



On 27/08/2005, at 12:37 AM, Jim Gallacher wrote:

I was hoping there would be a simple fix for this but a quick glance 
at the code makes me think that will not be the case. Also, I don't 
think this is a new bug as 3.1.4 does not generate a 404 NOT FOUND 
response either:


Mod_python error: "PythonHandler mod_python.publisher"

Traceback (most recent call last):

  File "/usr/lib/python2.3/site-packages/mod_python/apache.py", line 
299, in HandlerDispatch

result = object(req)

  File "/usr/lib/python2.3/site-packages/mod_python/publisher.py", 
line 98, in handler

path=[path])

  File "/usr/lib/python2.3/site-packages/mod_python/apache.py", line 
454, in import_module

f, p, d = imp.find_module(parts[i], path)

ImportError: No module named index

We should create a new JIRA issue. If it turns out that the fix is not 
easy, is this a big enough problem to hold up the 3.2 release?


Regards,
Jim


Graham Dumpleton wrote:

In 3.2 beta, if one uses:
  SetHandler mod_python
  PythonHandler mod_python.publisher
  PythonDebug On
and you DO NOT have an index.py file and use a URL where the first
URL component doesn't map explicitly to a .py file, you will get
a Python error rather than a 404 NOT FOUND response. For example,
for URL "http://localhost:8080/~grahamd/mp32/index";, I get:
Mod_python error: "PythonHandler mod_python.publisher"
Traceback (most recent call last):
  File  
"/System/Library/Frameworks/Python.framework/Versions/2.3/lib/ 
python2.3/site-packages/mod_python/apache.py", line 299, in  
HandlerDispatch

result = object(req)
  File  
"/System/Library/Frameworks/Python.framework/Versions/2.3/lib/ 
python2.3/site-packages/mod_python/publisher.py", line 192, in 
handler

module = page_cache[req]
  File  
"/System/Library/Frameworks/Python.framework/Versions/2.3/lib/ 
python2.3/site-packages/mod_python/cache.py", line 77, in __getitem__

return self._checkitem(name)[2]
  File  
"/System/Library/Frameworks/Python.framework/Versions/2.3/lib/ 
python2.3/site-packages/mod_python/cache.py", line 118, in _checkitem

opened = self.check(key, name, entry)
  File  
"/System/Library/Frameworks/Python.framework/Versions/2.3/lib/ 
python2.3/site-packages/mod_python/publisher.py", line 67, in check

return ModuleCache.check(self, key, req, entry)
  File  
"/System/Library/Frameworks/Python.framework/Versions/2.3/lib/ 
python2.3/site-packages/mod_python/cache.py", line 249, in check

opened = file(key, self.mode)
IOError: [Errno 2] No such file or directory:  
'/Users/grahamd/Sites/mp32/index.py'

Name suggestions svn 3.2 stable branch

[Jim Gallacher]

Gregory (Grisha) Trubetskoy wrote:


On Thu, 22 Sep 2005, Jim Gallacher wrote:
On another note, if we are starting to roll with new features for 3.3 
I would suggest we need to immediately create a new svn branch for 3.2 
bugfixes.



Yes


Any preference for a branch name? Some suggestions:

branches/release-3.2
branches/release-3.2-stable
branches/release-3.2-bugfixes

Jim

Does 3.2 still support python 2.2?

[Jim Gallacher]

From the 3.2.5b doc:

(http://www.modpython.org/live/mod_python-3.2.5b/doc-html/inst-prerequisites.html)

2.1 Prerequisites

* Python 2.2.1 or later. Earlier versions of Python will not work.


Is this still true or have we dropped support for python < 2.3? Has 
anybody tested using python 2.2.1?


Jim

mod_python docs appendix A - changes in 3.2

[Jim Gallacher]

I'm working on putting the docs in order for the 3.2 release, and 
thought I'd update appendix A as I go.


ie, http://www.modpython.org/live/current/doc-html/app-changes.html

So, what are the changes from 3.1? Should major bug fixes be listed or 
just new features/changed behaviour? Feel free to flesh these out to 
full line or add more.


Changed import mechanism.
(I wasn't paying too much attention to this whole discussion, so maybe 
someone else could comment on how this changed and the possible impact 
on user code?)


New version attribute.

Added publisher.get_page() which allows the import of a published page.

Added new session class FileSession which stores the data for each 
session in a separate file.


More??

Regards,
Jim

Re: 3.2 Compile problems with gcc 4.0

[Jim Gallacher]

Gregory (Grisha) Trubetskoy wrote:



On Wed, 10 Aug 2005, Jim Gallacher wrote:


Compilation fails with the following:

requestobject.c: In function 'request_tp_dealloc':
requestobject.c:1482: warning: implicit declaration of function 
'request_tp_clear'



This looks like a bug - I guess GCC 3 defaulted to static for 
implicitely declared functions, or perhaps didn't mind a non-static 
declaration followed by a static declaration.


gcc 3.3 just generates a warning:

requestobject.c: In function `request_tp_dealloc':
requestobject.c:1479: warning: implicit declaration of function 
`request_tp_clear'

requestobject.c: At top level:
requestobject.c:1511: warning: `request_tp_clear' was declared 
implicitly `extern' and later `static'

requestobject.c:1479: warning: previous declaration of `request_tp_clear'

As a sidenote - (wearing my dictator hat) - we really should be 
following the Apache coding style - having no spaces after if and 
several statements per line makes it difficult to read. Also, repetitive 
code is probably a good case for using an extra function or a macro 
(macro's should be last resort though because they can be hard to read 
and debug - but in this particular case a macro may be just fine)


OK, chief. :) I don't mind doing an audit for Apache coding style 
compliance, but in a post 3.2.0 time frame. Let's not start mucking with 
working code just before the release.


It's easy enough to fix (and I will) by adding a function prototype 
for request_tp_clear before it is called in request_tp_dealloc(). eg.

static int request_tp_clear(requestobject *self);



I think it can be fixed by simply moving tp_dealloc after tp_clear.


This is true, but I wasn't sure if it that was the preferred way.

I've noticed that in general functions are implicitly declared in 
mod_python. Is there a technical reason for this, or is it "just one 
of those things"?



Not really - implicit declaration is when something is referenced before 
being declared which in most cases results in compilation errors. What 
you probably mean to say is that they do not have forward declarations, 
but you generally don't need them if code is ordered correctly - there 
is no reason to have a forward declaration for every function.


OK. I'll fix the bug by moving request_tp_clear ahead of request_tp_dealloc.

Jim

Re: 3.2 Compile problems with gcc 4.0

[Gregory (Grisha) Trubetskoy]

On Thu, 11 Aug 2005, Jim Gallacher wrote:

OK, chief. :) I don't mind doing an audit for Apache coding style 
compliance, but in a post 3.2.0 time frame. Let's not start mucking with 
working code just before the release.


I agree, it's just a note for the future, whatever code we check in should 
always be the best, well spaced, well commented, read like a poem, etc.


:-)

Grisha

Re: Getting ready for 3.2 beta 2

[Nicolas Lehuen]

Hi Jim,

The fix for MODPYTHON-72
should be easy, unfortunately I'm quite busy right now, since my first
daughter was born three days ago... I'll do my best to have a look at
it, but if someone feels like doing it, I'll understand.

Regards,
Nicolas2005/8/26, Jim Gallacher <[EMAIL PROTECTED]>:
I think we should aim for the second beta release in the next couple ofdays. I have a few questions and a list of outstanding issues.Name of tarball: mod_python-3.2.1b.tgz?Also, I assume a new branch called tags/release-
3.2.1-BETA will becreated in subversion, correct?Outstanding issues:1. flexFix is ready to commit pending some feedback on the warning messagegenerated by configure.2. MacOS compile
Fixed in svn trunk.3. Eliminate creation of mod_python_so.so in non-windows environments.Fix is ready to commit.4. Fix segfault + memory leaks detailed in:   
http://issues.apache.org/jira/browse/MODPYTHON-75   http://issues.apache.org/jira/browse/MODPYTHON-60   Boyan's patch detailed in MODPYTHON-75 seems to fix both of these.
Fix is ready to commit.5. http://issues.apache.org/jira/browse/MODPYTHON-72Fix still required.6. Publisher bug in 3.2 BETA, detailed by Graham in python-dev message
posted 2005-08-21.Fix still required.I haven't looked at the code involved in items 5 and 6, but hopefullythe fixes will be fairly trivial.Regards,Jim

Re: Getting ready for 3.2 beta 2

[Jim Gallacher]

Nicolas Lehuen wrote:

Hi Jim,

The fix for MODPYTHON-72 
 should be easy, 
unfortunately I'm quite busy right now, since my first daughter was born 
three days ago... 


Congratulations Nicolas!

I'll do my best to have a look at it, but if someone

feels like doing it, I'll understand.


I have nothing planned for this weekend, so don't worry if you don't 
have the time. I should be able to handle it.


Best Regards,
Jim

Re: Getting ready for 3.2 beta 2

[Gregory (Grisha) Trubetskoy]

On Thu, 25 Aug 2005, Jim Gallacher wrote:

I think we should aim for the second beta release in the next couple of days. 
I have a few questions and a list of outstanding issues.


Name of tarball: mod_python-3.2.1b.tgz?


yep, 3.2.1b

Also, I assume a new branch called tags/release-3.2.1-BETA will be created in 
subversion, correct?


yup


Regards,
Jim


Thanks Jim!

Grisha

Re: Getting ready for 3.2 beta 2

[Gregory (Grisha) Trubetskoy]

On Fri, 26 Aug 2005, Nicolas Lehuen wrote:

unfortunately I'm quite busy right now, since my first daughter was born 
three days ago...


Wow! Congratulations Is this a first child? Does she have a name?

Anyway, scratch that word "unfortunately", you didn't mean it! :-)

Grisha

Re: Getting ready for 3.2 beta 2

[Nicolas Lehuen]

It's my first child (but the second for my wife), and her name is
Violette. And I totally crazy of her ! The problem for you guys is that
I've been preparing my house for her coming back (she came a bit
earlier than expected so I had to hurry things a little bit), so I had
no time to work on mod_python. I feel quite bad about this since it
looks like some problem are found about the publisher. I'll try to give
it a shot in the next few days.

Regards,
Nicolas2005/8/26, Gregory (Grisha) Trubetskoy <[EMAIL PROTECTED]>:
On Fri, 26 Aug 2005, Nicolas Lehuen wrote:> unfortunately I'm quite busy right now, since my first daughter was born> three days ago...Wow! Congratulations Is this a first child? Does she have a name?
Anyway, scratch that word "unfortunately", you didn't mean it! :-)Grisha

Re: Getting ready for 3.2 beta 2

[Jim Gallacher]

Hey Gang,

I think we are ready for the 3.2.1b release. If there are no objections 
in the next 24 hours I'll create the package and make the announcement 
on python-dev.


Jim Gallacher wrote:
I think we should aim for the second beta release in the next couple of 
days. I have a few questions and a list of outstanding issues.


Name of tarball: mod_python-3.2.1b.tgz?
Also, I assume a new branch called tags/release-3.2.1-BETA will be 
created in subversion, correct?


Outstanding issues:

1. flex
   Fix is ready to commit pending some feedback on the warning message 
generated by configure.


 Done


2. MacOS compile
   Fixed in svn trunk.


 Done


3. Eliminate creation of mod_python_so.so in non-windows environments.
   Fix is ready to commit.


  Not Done. I decided to defer this for reasons I won't go into just 
now. It is not a show stopper anyway.



4. Fix segfault + memory leaks detailed in:
  http://issues.apache.org/jira/browse/MODPYTHON-75
  http://issues.apache.org/jira/browse/MODPYTHON-60
  Boyan's patch detailed in MODPYTHON-75 seems to fix both of these. Fix 
is ready to commit.


Done. These can be marked as fixed in JIRA.


5. http://issues.apache.org/jira/browse/MODPYTHON-72
   Fix still required.


Done

6. Publisher bug in 3.2 BETA, detailed by Graham in python-dev message 
posted 2005-08-21.

   Fix still required.


Done (fix for item 5 fixed this as well).

Regards,
Jim

Re: Getting ready for 3.2 beta 2

[Graham Dumpleton]

On 01/09/2005, at 6:19 AM, Jim Gallacher wrote:Hey Gang,I think we are ready for the 3.2.1b release. If there are no objections in the next 24 hours I'll create the package and make the announcement on python-dev.Sounds good.I'll always be hoping to sneak in just one more change (eg. MODPYTHON-73),but realities are that I have to stop at some point. :-)BTW, I will be traveling for a few weeks from this weekend and at timeswill be disconnected from the Internet and at other times will only havebasic dial-up access, so you might not hear from me too much duringthat period.Maybe I'll use the time to dream about writing a book. ;-)Graham

Re: Getting ready for 3.2 beta 2

[Gregory (Grisha) Trubetskoy]

Or speaking in diff (not tested):

--- setup.py.in.orig2005-09-01 11:42:09.082202944 -0400
+++ setup.py.in 2005-09-01 11:44:35.969872624 -0400
@@ -140,18 +140,24 @@
 # this is a hack to prevent build_ext from trying to append 
"initmod_python" to the export symbols
 self.export_symbols = finallist(self.export_symbols)

-ModPyModule = ModPyExtension(getmp_srcdir(), [getmp_includedir(), 
getapache_includedir()], [getapache_libdir()])

 if winbuild:
+
+# build mod_python.so
+ModPyModule = ModPyExtension(getmp_srcdir(), [getmp_includedir(), 
getapache_includedir()], [getapache_libdir()])
+
 scripts = ["win32_postinstall.py"]
 # put the mod_python.so file in the Python root ...
 # win32_postinstall.py will pick it up from there...
 # data_files = [("", [(os.path.join(getmp_srcdir(), 'Release', 
'mod_python.so'))])]
 data_files = []
+ext_modules = [ModPyModule, PSPModule]
+
 else:
-# mpso = "../src/mod_python.so"
+
 scripts = []
 data_files = []
+ext_modules = [PSPModule]

 import string
 from distutils import sysconfig
@@ -174,7 +180,7 @@
   package_dir={'mod_python': os.path.join(getmp_rootdir(), 'lib', 
'python', 'mod_python')},
   scripts=scripts,
   data_files=data_files,
-  ext_modules=[ModPyModule, PSPModule])
+  ext_modules=ext_modules)

 # makes emacs go into python mode
 ### Local Variables:



On Thu, 1 Sep 2005, Gregory (Grisha) Trubetskoy wrote:



On Wed, 31 Aug 2005, Jim Gallacher wrote:


3. Eliminate creation of mod_python_so.so in non-windows environments.
   Fix is ready to commit.


 Not Done. I decided to defer this for reasons I won't go into just now. 
It is not a show stopper anyway.


Isn't the fix basically just placing the ModPyModule and setup() with 
ModPyModule inside the "if winbuild" block and then having another set() 
without the ModPyModule in the else clause?


Unless there is some good reason for it, I think it is a show stopper because 
it makes the build process look a bit on the bizzare side on Unix.


Grisha

Re: Getting ready for 3.2 beta 2

[Gregory (Grisha) Trubetskoy]

On Wed, 31 Aug 2005, Jim Gallacher wrote:


3. Eliminate creation of mod_python_so.so in non-windows environments.
   Fix is ready to commit.


 Not Done. I decided to defer this for reasons I won't go into just now. It 
is not a show stopper anyway.


Isn't the fix basically just placing the ModPyModule and setup() with 
ModPyModule inside the "if winbuild" block and then having another set() 
without the ModPyModule in the else clause?


Unless there is some good reason for it, I think it is a show stopper 
because it makes the build process look a bit on the bizzare side on Unix.


Grisha

Re: Getting ready for 3.2 beta 2

[Jim Gallacher]

I've tested this patch and checked it into svn. Should probably be 
tested for winbuild and MacOS X.


I think we are good to go if Graham and Nicolas are happy with their 
MODPYTHON-73 changes.


Jim

Gregory (Grisha) Trubetskoy wrote:


Or speaking in diff (not tested):

--- setup.py.in.orig2005-09-01 11:42:09.082202944 -0400
+++ setup.py.in 2005-09-01 11:44:35.969872624 -0400
@@ -140,18 +140,24 @@
 # this is a hack to prevent build_ext from trying to append 
"initmod_python" to the export symbols

 self.export_symbols = finallist(self.export_symbols)

-ModPyModule = ModPyExtension(getmp_srcdir(), [getmp_includedir(), 
getapache_includedir()], [getapache_libdir()])


 if winbuild:
+
+# build mod_python.so
+ModPyModule = ModPyExtension(getmp_srcdir(), [getmp_includedir(), 
getapache_includedir()], [getapache_libdir()])

+
 scripts = ["win32_postinstall.py"]
 # put the mod_python.so file in the Python root ...
 # win32_postinstall.py will pick it up from there...
 # data_files = [("", [(os.path.join(getmp_srcdir(), 'Release', 
'mod_python.so'))])]

 data_files = []
+ext_modules = [ModPyModule, PSPModule]
+
 else:
-# mpso = "../src/mod_python.so"
+
 scripts = []
 data_files = []
+ext_modules = [PSPModule]

 import string
 from distutils import sysconfig
@@ -174,7 +180,7 @@
   package_dir={'mod_python': os.path.join(getmp_rootdir(), 'lib', 
'python', 'mod_python')},

   scripts=scripts,
   data_files=data_files,
-  ext_modules=[ModPyModule, PSPModule])
+  ext_modules=ext_modules)

 # makes emacs go into python mode
 ### Local Variables:



On Thu, 1 Sep 2005, Gregory (Grisha) Trubetskoy wrote:



On Wed, 31 Aug 2005, Jim Gallacher wrote:


3. Eliminate creation of mod_python_so.so in non-windows environments.
   Fix is ready to commit.



 Not Done. I decided to defer this for reasons I won't go into just 
now. It is not a show stopper anyway.



Isn't the fix basically just placing the ModPyModule and setup() with 
ModPyModule inside the "if winbuild" block and then having another 
set() without the ModPyModule in the else clause?


Unless there is some good reason for it, I think it is a show stopper 
because it makes the build process look a bit on the bizzare side on 
Unix.


Grisha

Re: Getting ready for 3.2 beta 2

[Nicolas Lehuen]

Well I for one am happy woth MODPYTHON-73, I've integrated Graham's
patch and made unit test to check if everything was OK. Graham should
be happy too :).

Regards,
Nicolas2005/9/1, Jim Gallacher <[EMAIL PROTECTED]>:
I've tested this patch and checked it into svn. Should probably betested for winbuild and MacOS X.I think we are good to go if Graham and Nicolas are happy with theirMODPYTHON-73 changes.Jim
Gregory (Grisha) Trubetskoy wrote:>> Or speaking in diff (not tested):>> --- setup.py.in.orig2005-09-01 11:42:09.082202944 -0400> +++ setup.py.in 2005-09-01 11:44:
35.969872624 -0400> @@ -140,18 +140,24 @@>  # this is a hack to prevent build_ext from trying to append> "initmod_python" to the export symbols>  self.export_symbols = finallist(
self.export_symbols)>> -ModPyModule = ModPyExtension(getmp_srcdir(), [getmp_includedir(),> getapache_includedir()], [getapache_libdir()])>>  if winbuild:> +> +# build mod_python.so
> +ModPyModule = ModPyExtension(getmp_srcdir(), [getmp_includedir(),> getapache_includedir()], [getapache_libdir()])> +>  scripts = ["win32_postinstall.py"]>  # put the mod_python.so file in the Python root ...
>  # win32_postinstall.py will pick it up from there...>  # data_files = [("", [(os.path.join(getmp_srcdir(), 'Release',> 'mod_python.so'))])]>  data_files = []> +ext_modules = [ModPyModule, PSPModule]
> +>  else:> -# mpso = "../src/mod_python.so"> +>  scripts = []>  data_files = []> +ext_modules = [PSPModule]>>  import string>  from distutils import sysconfig
> @@ -174,7 +180,7 @@>package_dir={'mod_python': os.path.join(getmp_rootdir(), 'lib',> 'python', 'mod_python')},>scripts=scripts,>data_files=data_files,> -  ext_modules=[ModPyModule, PSPModule])
> +  ext_modules=ext_modules)>>  # makes emacs go into python mode>  ### Local Variables: On Thu, 1 Sep 2005, Gregory (Grisha) Trubetskoy wrote:>>>
>> On Wed, 31 Aug 2005, Jim Gallacher wrote:>> 3. Eliminate creation of mod_python_so.so in non-windows environments.Fix is ready to commit.>>>
>>  Not Done. I decided to defer this for reasons I won't go into just>>> now. It is not a show stopper anyway.>> Isn't the fix basically just placing the ModPyModule and setup() with
>> ModPyModule inside the "if winbuild" block and then having another>> set() without the ModPyModule in the else clause? Unless there is some good reason for it, I think it is a show stopper
>> because it makes the build process look a bit on the bizzare side on>> Unix. Grisha>>>

Re: Getting ready for 3.2 beta 2

[Gregory (Grisha) Trubetskoy]

I've been away this weekend - just got back, but I'm too busy to try to 
read all the multiple-interpreter related comments. I guess my question is 
- can someone provide a quick summary of how far we are from 3.2.1b test 
tarbal?


Thanks!

Grisha

On Thu, 1 Sep 2005, Graham Dumpleton wrote:


Nicolas Lehuen wrote ..

Well I for one am happy woth MODPYTHON-73, I've integrated Graham's patch
and made unit test to check if everything was OK. Graham should be happy
too
:).


As troublesome as I am, even I am happy at this point. :-)

Unfortunately, probably will not be able to do any last build checks on
MacOSX. I think I'll get killed tonight if I start working on the
computer tonight since I fly off quite early tomorrow morning. If I am
lucky I'll get just enough time to sync from the svn repository and then
I'll play with it on the plane. I'll then be off the Internet for about
4-5 days so if there are any problems you will not hear about them in
time anyway.

Graham

Re: Getting ready for 3.2 beta 2

[Nicolas Lehuen]

Well, if I've understood Jim's mail, apart from the new MODPYTHON-77, we're all set.

Regards,
Nicolas2005/9/6, Gregory (Grisha) Trubetskoy <[EMAIL PROTECTED]>:
I've been away this weekend - just got back, but I'm too busy to try toread all the multiple-interpreter related comments. I guess my question is- can someone provide a quick summary of how far we are from 3.2.1b
 testtarbal?Thanks!GrishaOn Thu, 1 Sep 2005, Graham Dumpleton wrote:> Nicolas Lehuen wrote ..>> Well I for one am happy woth MODPYTHON-73, I've integrated Graham's patch
>> and made unit test to check if everything was OK. Graham should be happy>> too>> :).>> As troublesome as I am, even I am happy at this point. :-)>> Unfortunately, probably will not be able to do any last build checks on
> MacOSX. I think I'll get killed tonight if I start working on the> computer tonight since I fly off quite early tomorrow morning. If I am> lucky I'll get just enough time to sync from the svn repository and then
> I'll play with it on the plane. I'll then be off the Internet for about> 4-5 days so if there are any problems you will not hear about them in> time anyway.>> Graham>

Re: Getting ready for 3.2 beta 2

[Jim Gallacher]

Gregory (Grisha) Trubetskoy wrote:


I've been away this weekend - just got back, but I'm too busy to try to 
read all the multiple-interpreter related comments. I guess my question 
is - can someone provide a quick summary of how far we are from 3.2.1b 
test tarbal?


I've also been away for the weekend. The patch for MODPYTHON-77 from 
last Thursday was causing apache to segfault and I have not had a chance 
to try the lastest changes from Boyan.


As Graham stated on the weekend, "the use of thread states can be very 
tricky". I think we should proceed with the 3.2.1b without the fix. That 
way we can take the time to make sure we understand the issues and fix 
it in 3.3.


If that seems reasonable, I'll make the tarball today.

Jim



Thanks!

Grisha

On Thu, 1 Sep 2005, Graham Dumpleton wrote:


Nicolas Lehuen wrote ..

Well I for one am happy woth MODPYTHON-73, I've integrated Graham's 
patch

and made unit test to check if everything was OK. Graham should be happy
too
:).



As troublesome as I am, even I am happy at this point. :-)

Unfortunately, probably will not be able to do any last build checks on
MacOSX. I think I'll get killed tonight if I start working on the
computer tonight since I fly off quite early tomorrow morning. If I am
lucky I'll get just enough time to sync from the svn repository and then
I'll play with it on the plane. I'll then be off the Internet for about
4-5 days so if there are any problems you will not hear about them in
time anyway.

Graham

Re: Getting ready for 3.2 beta 2

[Gregory (Grisha) Trubetskoy]

On Tue, 6 Sep 2005, Jim Gallacher wrote:

As Graham stated on the weekend, "the use of thread states can be very 
tricky". I think we should proceed with the 3.2.1b without the fix. That way 
we can take the time to make sure we understand the issues and fix it in 3.3.


If that seems reasonable, I'll make the tarball today.


+1

Grisha

Re: Getting ready for 3.2 beta 2

[Graham Dumpleton]

Jim Gallacher wrote ..
> Gregory (Grisha) Trubetskoy wrote:
> > 
> > I've been away this weekend - just got back, but I'm too busy to try
> to 
> > read all the multiple-interpreter related comments. I guess my question
> > is - can someone provide a quick summary of how far we are from 3.2.1b
> > test tarbal?
> 
> I've also been away for the weekend. The patch for MODPYTHON-77 from 
> last Thursday was causing apache to segfault and I have not had a chance
> to try the lastest changes from Boyan.
> 
> As Graham stated on the weekend, "the use of thread states can be very
> tricky". I think we should proceed with the 3.2.1b without the fix. That
> way we can take the time to make sure we understand the issues and fix
> it in 3.3.

If we feel that 3.3 could be a while in coming, I'm tending to think that this
change to support use of extensions using the simplified GIL interface
should be incorporated into 3.2. This would depend though on how many
more beta snapshots we think we might go through.

Graham

Re: Getting ready for 3.2 beta 2

[Nicolas Lehuen]

Well, why not keep our plan of releasing 3.2 ASAP and save this
problem for a later 3.2.x as a bug fix ?

Making subsequent bug-fix releases should be fast and easy. We cannot
afford to repeat the long hiatus between 3.1.3 and 3.2, with a long
period of time without any official bug fix.

I agree that 3.3 may come later, but we definitely should be able to
release 3.2 bugfixes version as often as possible. This will save us
and our users a lot of time, allowing us to stop writing "yeah, we
know this bug, it's already fixed in SVN but you'll have to wait an
undefinite time for the fix to go public".

Releasing often may need a bit of work on the web site side, however.
I feel that updating the web site is the current bottleneck, since
only Grisha can do it. Could it be possible to make the web site
writable by the commiters ? Maybe have a "web" directory in
subversion, so that we can edit the site and make update on the web
server side when we want to release new content ?

Regards,
Nicolas

2005/9/8, Graham Dumpleton <[EMAIL PROTECTED]>:
> Jim Gallacher wrote ..
> > Gregory (Grisha) Trubetskoy wrote:
> > >
> > > I've been away this weekend - just got back, but I'm too busy to try
> > to
> > > read all the multiple-interpreter related comments. I guess my question
> > > is - can someone provide a quick summary of how far we are from 3.2.1b
> > > test tarbal?
> >
> > I've also been away for the weekend. The patch for MODPYTHON-77 from
> > last Thursday was causing apache to segfault and I have not had a chance
> > to try the lastest changes from Boyan.
> >
> > As Graham stated on the weekend, "the use of thread states can be very
> > tricky". I think we should proceed with the 3.2.1b without the fix. That
> > way we can take the time to make sure we understand the issues and fix
> > it in 3.3.
> 
> If we feel that 3.3 could be a while in coming, I'm tending to think that this
> change to support use of extensions using the simplified GIL interface
> should be incorporated into 3.2. This would depend though on how many
> more beta snapshots we think we might go through.
> 
> Graham
>

Re: Getting ready for 3.2 beta 2

[Gregory (Grisha) Trubetskoy]

On Thu, 8 Sep 2005, Nicolas Lehuen wrote:


Releasing often may need a bit of work on the web site side, however.


with respect to modpython.org, no, not at all. with httpd.apache.org - 
it's a little bit of work, but not more than 15 minutes.



I feel that updating the web site is the current bottleneck, since
only Grisha can do it. Could it be possible to make the web site
writable by the commiters ?


the httpd.apache.org website _is_ writable by committers.

the plan is to move the modpython.org website to the apache infrastructure 
eventually, it's just been low priority and it would entail some work (the 
mailing list migration being the most complex task), but it will happen, 
eventually.


grisha

Re: Getting ready for 3.2 beta 2

[Jim Gallacher]

Nicolas Lehuen wrote:

Well, why not keep our plan of releasing 3.2 ASAP and save this
problem for a later 3.2.x as a bug fix ?
Making subsequent bug-fix releases should be fast and easy. We cannot
afford to repeat the long hiatus between 3.1.3 and 3.2, with a long
period of time without any official bug fix.

I agree that 3.3 may come later, but we definitely should be able to
release 3.2 bugfixes version as often as possible. This will save us
and our users a lot of time, allowing us to stop writing "yeah, we
know this bug, it's already fixed in SVN but you'll have to wait an
undefinite time for the fix to go public".


+1

It's always tempting to make one last change, fix one more bug, but then 
the release never happens. I think everyone has the will to move 
mod_python forward, we just need a little more discipline. There are 
lots of things we can do in 3.3, but I for one am not motivated to work 
on these until 3.2 is out. Lets get this puppy out the door and then 
have a discussion on plans and priorities for 3.3 with a view to 
reducing the time between bug fixes and major releases.


Jim


Releasing often may need a bit of work on the web site side, however.
I feel that updating the web site is the current bottleneck, since
only Grisha can do it. Could it be possible to make the web site
writable by the commiters ? Maybe have a "web" directory in
subversion, so that we can edit the site and make update on the web
server side when we want to release new content ?

Regards,
Nicolas

2005/9/8, Graham Dumpleton <[EMAIL PROTECTED]>:


Jim Gallacher wrote ..


Gregory (Grisha) Trubetskoy wrote:


I've been away this weekend - just got back, but I'm too busy to try


to


read all the multiple-interpreter related comments. I guess my question
is - can someone provide a quick summary of how far we are from 3.2.1b
test tarbal?


I've also been away for the weekend. The patch for MODPYTHON-77 from
last Thursday was causing apache to segfault and I have not had a chance
to try the lastest changes from Boyan.

As Graham stated on the weekend, "the use of thread states can be very
tricky". I think we should proceed with the 3.2.1b without the fix. That
way we can take the time to make sure we understand the issues and fix
it in 3.3.


If we feel that 3.3 could be a while in coming, I'm tending to think that this
change to support use of extensions using the simplified GIL interface
should be incorporated into 3.2. This would depend though on how many
more beta snapshots we think we might go through.

Graham

Re: Getting ready for 3.2 beta 2

[Jorey Bump]

Jim Gallacher wrote:

Nicolas Lehuen wrote:


Well, why not keep our plan of releasing 3.2 ASAP and save this
problem for a later 3.2.x as a bug fix ?
Making subsequent bug-fix releases should be fast and easy. We cannot
afford to repeat the long hiatus between 3.1.3 and 3.2, with a long
period of time without any official bug fix.

I agree that 3.3 may come later, but we definitely should be able to
release 3.2 bugfixes version as often as possible. This will save us
and our users a lot of time, allowing us to stop writing "yeah, we
know this bug, it's already fixed in SVN but you'll have to wait an
undefinite time for the fix to go public".



+1

It's always tempting to make one last change, fix one more bug, but then 
the release never happens. I think everyone has the will to move 
mod_python forward, we just need a little more discipline. There are 
lots of things we can do in 3.3, but I for one am not motivated to work 
on these until 3.2 is out. Lets get this puppy out the door and then 
have a discussion on plans and priorities for 3.3 with a view to 
reducing the time between bug fixes and major releases.


Would it help to adopt a naming convention where odd minor versions are 
for development, and even minor versions are stable/bug-fix-only? This 
would be a convenient time to adopt it. In some environments, this gives 
developers a place to add new features (3.3.x) while the first stable 
release (3.2.0) is getting bug squashed. As a user, it makes things a 
lot clearer that a certain version is still in development when you lust 
after a new feature it offers.


Just a thought...

Re: Getting ready for 3.2 beta 2

[Nicolas Lehuen]

2005/9/8, Jorey Bump <[EMAIL PROTECTED]>:
> Jim Gallacher wrote:
> > Nicolas Lehuen wrote:
> >
> >> Well, why not keep our plan of releasing 3.2 ASAP and save this
> >> problem for a later 3.2.x as a bug fix ?
> >> Making subsequent bug-fix releases should be fast and easy. We cannot
> >> afford to repeat the long hiatus between 3.1.3 and 3.2, with a long
> >> period of time without any official bug fix.
> >>
> >> I agree that 3.3 may come later, but we definitely should be able to
> >> release 3.2 bugfixes version as often as possible. This will save us
> >> and our users a lot of time, allowing us to stop writing "yeah, we
> >> know this bug, it's already fixed in SVN but you'll have to wait an
> >> undefinite time for the fix to go public".
> >
> >
> > +1
> >
> > It's always tempting to make one last change, fix one more bug, but then
> > the release never happens. I think everyone has the will to move
> > mod_python forward, we just need a little more discipline. There are
> > lots of things we can do in 3.3, but I for one am not motivated to work
> > on these until 3.2 is out. Lets get this puppy out the door and then
> > have a discussion on plans and priorities for 3.3 with a view to
> > reducing the time between bug fixes and major releases.
> 
> Would it help to adopt a naming convention where odd minor versions are
> for development, and even minor versions are stable/bug-fix-only? This
> would be a convenient time to adopt it. In some environments, this gives
> developers a place to add new features (3.3.x) while the first stable
> release (3.2.0) is getting bug squashed. As a user, it makes things a
> lot clearer that a certain version is still in development when you lust
> after a new feature it offers.
> 
> Just a thought...
> 

Yeah, why not.

In any case, we should maintain a separate 3.2 branch with only bug
fixes while developing the 3.3 version on the trunk (and merge the
bugfixes from the 3.2 version into the 3.3 trunk, of course).

We haven't done this for the 3.1 and 3.2 version, so everybody will
need to upgrade to 3.2 even if they want a single bugfix. This is not
a really bad thing this time, but next time, if we start to "fool
around" and break some compatibility (think "new import system" here
:), we should make sure we don't force our users to upgrade just to
get one bugfix.

Regards,
Nicolas

Re: Getting ready for 3.2 beta 2

[Jim Gallacher]

Nicolas Lehuen wrote:

2005/9/8, Jorey Bump <[EMAIL PROTECTED]>:


Jim Gallacher wrote:


Nicolas Lehuen wrote:



Well, why not keep our plan of releasing 3.2 ASAP and save this
problem for a later 3.2.x as a bug fix ?
Making subsequent bug-fix releases should be fast and easy. We cannot
afford to repeat the long hiatus between 3.1.3 and 3.2, with a long
period of time without any official bug fix.

I agree that 3.3 may come later, but we definitely should be able to
release 3.2 bugfixes version as often as possible. This will save us
and our users a lot of time, allowing us to stop writing "yeah, we
know this bug, it's already fixed in SVN but you'll have to wait an
undefinite time for the fix to go public".



+1

It's always tempting to make one last change, fix one more bug, but then
the release never happens. I think everyone has the will to move
mod_python forward, we just need a little more discipline. There are
lots of things we can do in 3.3, but I for one am not motivated to work
on these until 3.2 is out. Lets get this puppy out the door and then
have a discussion on plans and priorities for 3.3 with a view to
reducing the time between bug fixes and major releases.


Would it help to adopt a naming convention where odd minor versions are
for development, and even minor versions are stable/bug-fix-only? This
would be a convenient time to adopt it. In some environments, this gives
developers a place to add new features (3.3.x) while the first stable
release (3.2.0) is getting bug squashed. As a user, it makes things a
lot clearer that a certain version is still in development when you lust
after a new feature it offers.

Just a thought...




Yeah, why not.

In any case, we should maintain a separate 3.2 branch with only bug
fixes while developing the 3.3 version on the trunk (and merge the
bugfixes from the 3.2 version into the 3.3 trunk, of course).

We haven't done this for the 3.1 and 3.2 version, so everybody will
need to upgrade to 3.2 even if they want a single bugfix. This is not
a really bad thing this time, but next time, if we start to "fool
around" and break some compatibility (think "new import system" here
:), we should make sure we don't force our users to upgrade just to
get one bugfix.



As Jorey points out, this is a good time to discuss this.

I'm not opposed to the even/odd numbering, but I'm not sure it's really 
necessary. Does anyone really envisage making development snapshots 
available as tarballs? I'm a big fan of subversion. If anyone wants to 
play around with the development branch it's just as easy to grab a copy 
of mod_python/trunk with svn as it is to download a tarball. If we are 
not making development releases then we don't need to number them.


Beyond that, I agree we should have a 3.2 branch for bugfixes, and I 
think we need it now. It's just too tempting to mess around with code in 
mod_python/trunk which could screw up our close-to-golden 3.2 release. 
I'd like to suggest "branches/release-3.2" as the name. (Note the lack 
of a patch level in the name). Any further beta's and the final release 
would be generated from this branch, with mod_python/tags reserved for 
snapshots.


As far as some future version breaking compatibility, I favour a bigger 
jump in the major number: 3.2 -> 4.0. This is server software after all, 
and some people may prefer to maintain an older version for a longer 
period, foregoing new features in favour of the tried and true. 
Incrementing the major number makes it more obvious that an upgrade may 
cause some problems. But I guess that discussion is sometime *way* in 
the future. :)


Jim

Re: Getting ready for 3.2 beta 2

[Graham Dumpleton]

On 09/09/2005, at 10:02 AM, Jim Gallacher wrote:
As far as some future version breaking compatibility, I favour a 
bigger jump in the major number: 3.2 -> 4.0. This is server software 
after all, and some people may prefer to maintain an older version for 
a longer period, foregoing new features in favour of the tried and 
true. Incrementing the major number makes it more obvious that an 
upgrade may cause some problems. But I guess that discussion is 
sometime *way* in the future. :)


I also have been thinking that a jump to version 4.0 would be better
for what is being speculated on for the next release.

Graham

Re: Getting ready for 3.2 beta 2

[Jim Gallacher]

I thought I'd it a shot on minotaur as well.

Poking around a bit reveals that the default apache is indeed 1.3. It 
looks like there might be a viable apache2 hiding in 
/usr/local/apache2-install/www.apache.org/current.


eg ./configure 
--with-apxs=/usr/local/apache2-install/www.apache.org/current/apxs


Unfortunately, I'm getting the same errors as Grisha reported starting with:
mod_python.c:34: error: syntax error before '*' token

Regards,
Jim

Nicolas Lehuen wrote:

I tried to build it under minotaur as well, but "./configure" only
finds a 1.3.33 version of Apache, so I can't go further. I can't help
much here since I'm not used to FreeBSD...

Regards,
Nicolas

2005/9/9, Gregory (Grisha) Trubetskoy <[EMAIL PROTECTED]>:


Just tried compiling it on minotaur and I get the same error. minotaur is
FreeBSD 5.4, so it looks like we have a -1. I don't know how much time
I'll have this weekend, so I might or might not look into the cause of
this - but anyone else with access to a FreeBSD box, you're more than
welcome to dig in... :-)

Grisha


On Fri, 9 Sep 2005, Jim Gallacher wrote:



Gregory (Grisha) Trubetskoy wrote:


Don't know about versions, but I'd _really_ like to see a FreeBSD +1 at
this point :-) Graham - don't you have FreeBSD access somewhere?


If Graham can't help out maybe we could recruit a volunteer on the mod_python
list?



On the versioning discussion - I don't like 4.0, I think 3.3 should be the
next version after 3.2.x. As far as even/odd stable/unstable - the Linux
kernel folks have abandoned it because it didn't work for them. The
fallacy is that you cannot know ahead of time what is stable and what is
not.

My preference is to just follow versions incrementally, and making it
known which version is stable or not independently of the version number,
which is what the HTTPD folks have been doing.


I can't get worked up one way or another wrt to a version numbering scheme,
as long as we release *something*. ;)

Regards,
Jim

Re: Getting ready for 3.2 beta 2

[Nicolas Lehuen]

I've tried to build 3.1.4 from the tarball on minotaur and of course
it works. Could it be possible that the recent changes in the
configure script cause the problem ?

Regards,

Nicolas

2005/9/10, Jim Gallacher <[EMAIL PROTECTED]>:
> I thought I'd it a shot on minotaur as well.
> 
> Poking around a bit reveals that the default apache is indeed 1.3. It
> looks like there might be a viable apache2 hiding in
> /usr/local/apache2-install/www.apache.org/current.
> 
> eg ./configure
> --with-apxs=/usr/local/apache2-install/www.apache.org/current/apxs
> 
> Unfortunately, I'm getting the same errors as Grisha reported starting with:
> mod_python.c:34: error: syntax error before '*' token
> 
> Regards,
> Jim
> 
> Nicolas Lehuen wrote:
> > I tried to build it under minotaur as well, but "./configure" only
> > finds a 1.3.33 version of Apache, so I can't go further. I can't help
> > much here since I'm not used to FreeBSD...
> >
> > Regards,
> > Nicolas
> >
> > 2005/9/9, Gregory (Grisha) Trubetskoy <[EMAIL PROTECTED]>:
> >
> >>Just tried compiling it on minotaur and I get the same error. minotaur is
> >>FreeBSD 5.4, so it looks like we have a -1. I don't know how much time
> >>I'll have this weekend, so I might or might not look into the cause of
> >>this - but anyone else with access to a FreeBSD box, you're more than
> >>welcome to dig in... :-)
> >>
> >>Grisha
> >>
> >>
> >>On Fri, 9 Sep 2005, Jim Gallacher wrote:
> >>
> >>
> >>>Gregory (Grisha) Trubetskoy wrote:
> >>>
> >>>>Don't know about versions, but I'd _really_ like to see a FreeBSD +1 at
> >>>>this point :-) Graham - don't you have FreeBSD access somewhere?
> >>>
> >>>If Graham can't help out maybe we could recruit a volunteer on the 
> >>>mod_python
> >>>list?
> >>>
> >>>
> >>>>On the versioning discussion - I don't like 4.0, I think 3.3 should be the
> >>>>next version after 3.2.x. As far as even/odd stable/unstable - the Linux
> >>>>kernel folks have abandoned it because it didn't work for them. The
> >>>>fallacy is that you cannot know ahead of time what is stable and what is
> >>>>not.
> >>>>
> >>>>My preference is to just follow versions incrementally, and making it
> >>>>known which version is stable or not independently of the version number,
> >>>>which is what the HTTPD folks have been doing.
> >>>
> >>>I can't get worked up one way or another wrt to a version numbering scheme,
> >>>as long as we release *something*. ;)
> >>>
> >>>Regards,
> >>>Jim
> >>>
> >>
> >
> 
>

Re: Getting ready for 3.2 beta 2

[Graham Dumpleton]

On 10/09/2005, at 2:08 AM, Gregory (Grisha) Trubetskoy wrote:



Don't know about versions, but I'd _really_ like to see a FreeBSD +1 
at this point :-) Graham - don't you have FreeBSD access somewhere?


Sorry, don't have access to FreeBSD anymore. Any comments I make about
FreeBSD are from experiences with battling with mod_python 2.7 under
Apache 1.3 on my old web hosting provider. Now that I am with the much
superior OpenHosting (which I am sure Grisha will agree), don't have
FreeBSD access.

Graham

mod_python.util.StorageField.read_to_boundary has problems in 3.1 and 3.2

[Alexis Marrero]

All, The current 3.1 mod_python implementation of mod_python.util.StorageField.read_to_boudary reads as follows:   203      def read_to_boundary(self, req, boundary, file):   204          delim = ""   205          line = req.readline()   206          sline = line.strip()   207          last_bound = boundary + "--"   208          while line and sline != boundary and sline != last_bound:   209              odelim = delim   210              if line[-2:] == "\r\n":   211                  delim = "\r\n"   212                  line = line[:-2]   213              elif line[-1:] == "\n":   214                  delim = "\n"   215                  line = line[:-1]   216              file.write(odelim + line)   217              line = req.readline()   218              sline = line.strip()As we have discussed previously: http://www.modpython.org/pipermail/mod_python/2005-March/017754.htmlhttp://www.modpython.org/pipermail/mod_python/2005-March/017756.htmlhttp://www.modpython.org/pipermail/mod_python/2005-November/019460.htmlThis triggered couple of changes in mod_python 3.2 Beta which reads as follows:    33  # Fixes memory error when upload large files such as 700+MB ISOs.    34  readBlockSize = 65368    35...   225     def read_to_boundary(self, req, boundary, file):...   234         delim = ''   235         lastCharCarried = False   236         last_bound = boundary + '--'   237         roughBoundaryLength = len(last_bound) + 128   238         line = req.readline(readBlockSize)   239         lineLength = len(line)   240         if lineLength < roughBoundaryLength:   241             sline = line.strip()   242         else:   243             sline = ''   244         while lineLength > 0 and sline != boundary and sline != last_bound:   245             if not lastCharCarried:   246                 file.write(delim)   247                 delim = ''   248             else:   249                 lastCharCarried = False   250             cutLength = 0   251             if lineLength == readBlockSize:   252                 if line[-1:] == '\r':   253                     delim = '\r'   254                     cutLength = -1   255                     lastCharCarried = True   256             if line[-2:] == '\r\n':   257                 delim += '\r\n'   258                 cutLength = -2   259             elif line[-1:] == '\n':   260                 delim += '\n'   261                 cutLength = -1   262             if cutLength != 0:   263                 file.write(line[:cutLength])   264             else:   265                 file.write(line)   266             line = req.readline(readBlockSize)   267             lineLength = len(line)   268             if lineLength < roughBoundaryLength:   269                 sline = line.strip()   270             else:   271                 sline = ''This function has a mysterious bug in it... For some files which I could disclose (one of them been the PDF file for Apple's Pages User Manual in Italian) the uploaded file in the server ends up with the same length but different sha512 (the only digest that I'm using).  The problem is a '\r' in the middle of a chunk of data that is much larger than readBlockSize.Anyhow, I wrote a new function, which I believe is much simpler, and test it with thousands and thousands of different files and so far it seems to work fine.  It reads as follows:def read_to_boundary(self, req, boundary, file):    ''' read from the request object line by line with a maximum size,        until the new line starts with boundary    '''    previous_delimiter = ''    while 1:        line = req.readline(1<<16)        if line.startswith(boundary):            break                if line.endswith('\r\n'):            file.write(previous_delimiter + line[:-2])            previous_delimiter = '\r\n'                elif line.endswith('\r') or line.endswith('\n'):            file.write(previous_delimiter + line[:-1])               previous_delimiter = line[-1:]        else:            file.write(previous_delimiter + line)            previous_delimiter = ''Let me know any comments on it and if you test it and fails please also let me know. I don't have subversion account neither I don't know how to use it thus this email./amn___Mod_python mailing list[EMAIL PROTECTED]http://mailman.modpython.org/mailman/listinfo/mod_python 

Re: Does 3.2 still support python 2.2?

[Nick]

I'm pretty sure we've had a few discussions about being able to use certain 
functions and modules because they became available in 2.3, and that's what 
mod_python was going to require.  Like the bsddb database version for your 
session code, for example.


Nick

Jim Gallacher wrote:

 From the 3.2.5b doc:

(http://www.modpython.org/live/mod_python-3.2.5b/doc-html/inst-prerequisites.html) 



2.1 Prerequisites

* Python 2.2.1 or later. Earlier versions of Python will not work.


Is this still true or have we dropped support for python < 2.3? Has 
anybody tested using python 2.2.1?


Jim

Re: Does 3.2 still support python 2.2?

[Jim Gallacher]

I figured we had moved on to 2.3, but I wanted to make sure I wasn't 
missing something before I changed the docs. I'm not sure if there was a 
formal decision on this or everyone just assumed it was true. Perhaps a 
pronoucement from Grisha to make it offical?


If python 2.2 support has been dropped then it needs needs to be 
mentioned in changes section of the docs, and the README as well.


Jim

Nick wrote:
I'm pretty sure we've had a few discussions about being able to use 
certain functions and modules because they became available in 2.3, and 
that's what mod_python was going to require.  Like the bsddb database 
version for your session code, for example.


Nick

Jim Gallacher wrote:


 From the 3.2.5b doc:

(http://www.modpython.org/live/mod_python-3.2.5b/doc-html/inst-prerequisites.html) 



2.1 Prerequisites

* Python 2.2.1 or later. Earlier versions of Python will not work.


Is this still true or have we dropped support for python < 2.3? Has 
anybody tested using python 2.2.1?


Jim

Re: Does 3.2 still support python 2.2?

[Nick]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

http://article.gmane.org/gmane.comp.apache.mod-python.devel/865

Jim Gallacher wrote:
| I figured we had moved on to 2.3, but I wanted to make sure I wasn't
| missing something before I changed the docs. I'm not sure if there was a
| formal decision on this or everyone just assumed it was true. Perhaps a
| pronoucement from Grisha to make it offical?
|
| If python 2.2 support has been dropped then it needs needs to be
| mentioned in changes section of the docs, and the README as well.
|
| Jim
|
| Nick wrote:
|
|> I'm pretty sure we've had a few discussions about being able to use
|> certain functions and modules because they became available in 2.3,
|> and that's what mod_python was going to require.  Like the bsddb
|> database version for your session code, for example.
|>
|> Nick
|>
|> Jim Gallacher wrote:
|>
|>>  From the 3.2.5b doc:
|>>
|>>
(http://www.modpython.org/live/mod_python-3.2.5b/doc-html/inst-prerequisites.html)

|>>
|>>
|>> 2.1 Prerequisites
|>>
|>> * Python 2.2.1 or later. Earlier versions of Python will not work.
|>>
|>>
|>> Is this still true or have we dropped support for python < 2.3? Has
|>> anybody tested using python 2.2.1?
|>>
|>> Jim
|>
|>
|>
|>
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.1 (GNU/Linux)

iD8DBQFDmj4Cv4zJ7LQ+i84RAh9VAJ9rWpumf/Bdky9NuK0bvX96NHrmQQCeKSDD
JAF18Qqe3CvDezgOww9599A=
=tlHN
-END PGP SIGNATURE-

Re: Does 3.2 still support python 2.2?

[Jim Gallacher]

OK then, good enough.

Next question. Should we bump the Apache version requirement as well. 
Currently the docs state that Apache 2.0.40 or later is required. I 
don't recall seeing anyone testing mod_python 3.2 on anything less than 
apache 2.0.53. I don't know if there are any changes between 40 and 53 
that may have a negative impact, but if we haven't actually tested on 
the earlier versions are we just asking for trouble?


Jim

Nick wrote:

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

http://article.gmane.org/gmane.comp.apache.mod-python.devel/865

Jim Gallacher wrote:
| I figured we had moved on to 2.3, but I wanted to make sure I wasn't
| missing something before I changed the docs. I'm not sure if there was a
| formal decision on this or everyone just assumed it was true. Perhaps a
| pronoucement from Grisha to make it offical?
|
| If python 2.2 support has been dropped then it needs needs to be
| mentioned in changes section of the docs, and the README as well.
|
| Jim
|
| Nick wrote:
|
|> I'm pretty sure we've had a few discussions about being able to use
|> certain functions and modules because they became available in 2.3,
|> and that's what mod_python was going to require.  Like the bsddb
|> database version for your session code, for example.
|>
|> Nick
|>
|> Jim Gallacher wrote:
|>
|>>  From the 3.2.5b doc:
|>>
|>>
(http://www.modpython.org/live/mod_python-3.2.5b/doc-html/inst-prerequisites.html) 



|>>
|>>
|>> 2.1 Prerequisites
|>>
|>> * Python 2.2.1 or later. Earlier versions of Python will not work.
|>>
|>>
|>> Is this still true or have we dropped support for python < 2.3? Has
|>> anybody tested using python 2.2.1?
|>>
|>> Jim
|>
|>
|>
|>
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.1 (GNU/Linux)

iD8DBQFDmj4Cv4zJ7LQ+i84RAh9VAJ9rWpumf/Bdky9NuK0bvX96NHrmQQCeKSDD
JAF18Qqe3CvDezgOww9599A=
=tlHN
-END PGP SIGNATURE-

Re: Does 3.2 still support python 2.2?

[Gregory (Grisha) Trubetskoy]

On Fri, 9 Dec 2005, Jim Gallacher wrote:

Next question. Should we bump the Apache version requirement as well. 
Currently the docs state that Apache 2.0.40 or later is required. I 
don't recall seeing anyone testing mod_python 3.2 on anything less than 
apache 2.0.53. I don't know if there are any changes between 40 and 53 
that may have a negative impact, but if we haven't actually tested on 
the earlier versions are we just asking for trouble?


Probably :)

I don't remember where the 2.0.40 came from. There may have been a 
technical reason for it, or may be it was just the current version at the 
time. I think the most honest thing to would be to say that we've tested 
it with 2.0.53, but it will most likely work with a few minor versions 
earlier just as well. Since we're part of HTTP project, we should anyhow 
encourage folks to use the most recent versions.


Grisha

Re: mod_python docs appendix A - changes in 3.2

[Nicolas Lehuen]

Hi Jim,

I don't have much time now but there should be a complete list here :

http://issues.apache.org/jira/secure/ReleaseNote.jspa?version=11060&styleName=Html&projectId=10640&Create=Create

What you should add to your list are :

- a fix/improvement for FieldStorage allowing for better file upload management
- lots of bugfixes ;)

BTW, there's a new feature in JIRA, now : it looks like if a
MODPYTHON-xx bug number is inserted into a subversion commit message,
the commit is reference in a "Subversion commit" tab in the bug page.
Example : http://issues.apache.org/jira/browse/MODPYTHON-48

Regards,
Nicoas



2005/6/25, Jim Gallacher <[EMAIL PROTECTED]>:
> I'm working on putting the docs in order for the 3.2 release, and
> thought I'd update appendix A as I go.
> 
> ie, http://www.modpython.org/live/current/doc-html/app-changes.html
> 
> So, what are the changes from 3.1? Should major bug fixes be listed or
> just new features/changed behaviour? Feel free to flesh these out to
> full line or add more.
> 
> Changed import mechanism.
> (I wasn't paying too much attention to this whole discussion, so maybe
> someone else could comment on how this changed and the possible impact
> on user code?)
> 
> New version attribute.
> 
> Added publisher.get_page() which allows the import of a published page.
> 
> Added new session class FileSession which stores the data for each
> session in a separate file.
> 
> More??
> 
> Regards,
> Jim
> 
>

Re: mod_python docs appendix A - changes in 3.2

[Graham Dumpleton]

On 26/06/2005, at 1:28 AM, Jim Gallacher wrote:

Changed import mechanism.
(I wasn't paying too much attention to this whole discussion, so
maybe someone else could comment on how this changed and the possible
impact on user code?)

New version attribute.

Added publisher.get_page() which allows the import of a published page.


I know I am not part of the development team, so can only make 
suggestions,
but I personally don't believe that the new import mechanism which has 
been

developed just for publisher should be included.

There are already problems which can arise due to the mixing of the 
"import"

statement and "apache.import_module()", and now a third system is being
brought into the picture which behaves different enough that in itself 
may
cause confusion, but when wrongly mixed with the others could result in 
a

mess. I know I don't want to have to be answering questions about it and
if it did come down to it, I probably wouldn't.

Thus, I would be suggesting leaving this feature out. You shouldn't 
however
let this delay 3.2. People understand that module importing is broken 
and

how to work around it, leaving it that way a bit longer is not going to
cause any new harm. At least, less harm than if this new scheme has to 
be

ripped out at a later date.

Instead, fixing of the main module importer should be carried out. I 
know
that others disagree with some of my ideas on that, but I do believe it 
can
be fixed and the only features you would loose would be the ability to 
import
parts of packages when using "apache.import_module()" explicitly and 
this
only because to support packages properly when autoreloading is 
supported

overly complicates the design to the point that it isn't practical. This
ability of the current importer to deal with packages is actually broken
in various ways anyway and I don't think dropping it would cause any 
real
problem. Note that this wouldn't prevent use of packages or parts of 
packages

from standard Python locations from any of the Handler directives.

Anyway, the point of this mail is simply to register my viewpoint that 
the

new publisher specific import mechanism should not be included. I am not
at this time going to be drawn into further discussions about how to fix
the current import mechanism though. I am working on a document which
describes the basics of how the module importer works and all its 
problems

and issues. When I am finished that, then we might start discussing it
again. I am about half done on this document and hopefully in another
week it will be ready to be used as the basis of some discussions on how
to progress this issue.

Graham

Re: mod_python docs appendix A - changes in 3.2

[Gregory (Grisha) Trubetskoy]

On Sun, 26 Jun 2005, Graham Dumpleton wrote:


Anyway, the point of this mail is simply to register my viewpoint that the
new publisher specific import mechanism should not be included.


What does everyone think? I'm inclined to agree, especially given that 
Graham is working on documenting the issue (see below) which may affect 
the ultimate design.


Grisha

I am not at this time going to be drawn into further discussions about 
how to fix the current import mechanism though. I am working on a 
document which describes the basics of how the module importer works and 
all its problems and issues. When I am finished that, then we might 
start discussing it again. I am about half done on this document and 
hopefully in another week it will be ready to be used as the basis of 
some discussions on how to progress this issue.


Graham

Re: mod_python docs appendix A - changes in 3.2

[Nicolas Lehuen]

I agree with Graham on the whole, but I'd like to point out a few things.

1) The current publisher implementation, with its own dynamic loading
mechanism, already fixes
http://issues.apache.org/jira/browse/MODPYTHON-9
http://issues.apache.org/jira/browse/MODPYTHON-10
http://issues.apache.org/jira/browse/MODPYTHON-11 and a bit of
http://issues.apache.org/jira/browse/MODPYTHON-8 . So I think we
should keep it until we have a "unified importing theory".

2) The only thing I'm not too proud of, retrospectively, is
publisher.get_page. Like Graham wrote, this introduces a third way for
a developer to handle imports, which is bad. So I agree that even if
we keep the current implementation of the publisher, we should remove
publisher.get_page.

3) My suggestion for an "unified importing theory" is :
a) static importing should be done with import and end up in sys.modules
b) dynamic importing should be done with apache.import_module and end
up in its own module cache, not polluting sys.modules.
c) root level handlers should be imported statically, published pages
and application specific code should be imported dynamically
d) to be practical in a shared hosting environment, dynamic importing
should support reloading, of course, with dependencies, so that
restarting the server whenever published pages or application-specific
code is modified would not be needed.

4) Graham, I don't know what it takes to be a member of the
development team, but to me you are one :).

Regards,
Nicolas

2005/6/26, Gregory (Grisha) Trubetskoy <[EMAIL PROTECTED]>:
> 
> On Sun, 26 Jun 2005, Graham Dumpleton wrote:
> 
> > Anyway, the point of this mail is simply to register my viewpoint that the
> > new publisher specific import mechanism should not be included.
> 
> What does everyone think? I'm inclined to agree, especially given that
> Graham is working on documenting the issue (see below) which may affect
> the ultimate design.
> 
> Grisha
> 
> > I am not at this time going to be drawn into further discussions about
> > how to fix the current import mechanism though. I am working on a
> > document which describes the basics of how the module importer works and
> > all its problems and issues. When I am finished that, then we might
> > start discussing it again. I am about half done on this document and
> > hopefully in another week it will be ready to be used as the basis of
> > some discussions on how to progress this issue.
> >
> > Graham
> >
> >
> >
>

Re: mod_python docs appendix A - changes in 3.2

[Jim Gallacher]

Nicolas Lehuen wrote:

Hi Jim,

I don't have much time now but there should be a complete list here :

http://issues.apache.org/jira/secure/ReleaseNote.jspa?version=11060&styleName=Html&projectId=10640&Create=Create


Nice. That will certainly make things easier.


What you should add to your list are :

- a fix/improvement for FieldStorage allowing for better file upload management
- lots of bugfixes ;)


OK.


BTW, there's a new feature in JIRA, now : it looks like if a
MODPYTHON-xx bug number is inserted into a subversion commit message,
the commit is reference in a "Subversion commit" tab in the bug page.
Example : http://issues.apache.org/jira/browse/MODPYTHON-48


You pointed that out previously, so I've been making a point of doing it 
with my commits. However, JIRA does not seem to send these commit 
comments to the mailing list in the same way it does regular comments.


Jim

Re: mod_python docs appendix A - changes in 3.2

[Gregory (Grisha) Trubetskoy]

On Sun, 26 Jun 2005, Nicolas Lehuen wrote:

2) The only thing I'm not too proud of, retrospectively, is 
publisher.get_page.


One thing we can do is leave it undocumented and put a comment in the code 
warning people not to rely on it as it is not a "sure thing".


Grisha

Re: mod_python.util.StorageField.read_to_boundary has problems in 3.1 and 3.2

[Jim Gallacher]

Alexis,

Do you a have a small file which shows this behaviour and could be used 
for testing? Even better would be a function which would generate a test 
file. This could be included in the mod_python unit tests.


Jim


Alexis Marrero wrote:
All, 

The current 3.1 mod_python implementation of 
mod_python.util.StorageField.read_to_boudary reads as follows:


   203  def read_to_boundary(self, req, boundary, file):
   204  delim = ""
   205  line = req.readline()
   206  sline = line.strip()
   207  last_bound = boundary + "--"
   208  while line and sline != boundary and sline != last_bound:
   209  odelim = delim
   210  if line[-2:] == "\r\n":
   211  delim = "\r\n"
   212  line = line[:-2]
   213  elif line[-1:] == "\n":
   214  delim = "\n"
   215  line = line[:-1]
   216  file.write(odelim + line)
   217  line = req.readline()
   218  sline = line.strip()

As we have discussed previously: 
http://www.modpython.org/pipermail/mod_python/2005-March/017754.html

http://www.modpython.org/pipermail/mod_python/2005-March/017756.html
http://www.modpython.org/pipermail/mod_python/2005-November/019460.html

This triggered couple of changes in mod_python 3.2 Beta which reads as 
follows:

33  # Fixes memory error when upload large files such as 700+MB ISOs.
34  readBlockSize = 65368
35
*...*
   225 def read_to_boundary(self, req, boundary, file):
...
   234 delim = ''
   235 lastCharCarried = False
   236 last_bound = boundary + '--'
   237 roughBoundaryLength = len(last_bound) + 128
   238 line = req.readline(readBlockSize)
   239 lineLength = len(line)
   240 if lineLength < roughBoundaryLength:
   241 sline = line.strip()
   242 else:
   243 sline = ''
   244 while lineLength > 0 and sline != boundary and sline != 
last_bound:

   245 if not lastCharCarried:
   246 file.write(delim)
   247 delim = ''
   248 else:
   249 lastCharCarried = False
   250 cutLength = 0
   251 if lineLength == readBlockSize:
   252 if line[-1:] == '\r':
   253 delim = '\r'
   254 cutLength = -1
   255 lastCharCarried = True
   256 if line[-2:] == '\r\n':
   257 delim += '\r\n'
   258 cutLength = -2
   259 elif line[-1:] == '\n':
   260 delim += '\n'
   261 cutLength = -1
   262 if cutLength != 0:
   263 file.write(line[:cutLength])
   264 else:
   265 file.write(line)
   266 line = req.readline(readBlockSize)
   267 lineLength = len(line)
   268 if lineLength < roughBoundaryLength:
   269 sline = line.strip()
   270 else:
   271 sline = ''

This function has a mysterious bug in it... For some files which I could 
disclose (one of them been the PDF file for Apple's Pages User Manual in 
Italian) the uploaded file in the server ends up with the same length 
but different sha512 (the only digest that I'm using).  The problem is a 
'\r' in the middle of a chunk of data that is much larger 
than readBlockSize.


Anyhow, I wrote a new function, which I believe is much simpler, and 
test it with thousands and thousands of different files and so far it 
seems to work fine.  It reads as follows:


def read_to_boundary(self, req, boundary, file):
''' read from the request object line by line with a maximum size,
until the new line starts with boundary
'''
previous_delimiter = ''
while 1:
line = req.readline(1<<16)
if line.startswith(boundary):
break
   
if line.endswith('\r\n'):

file.write(previous_delimiter + line[:-2])
previous_delimiter = '\r\n'
   
elif line.endswith('\r') or line.endswith('\n'):
file.write(previous_delimiter + line[:-1])  
previous_delimiter = line[-1:]


else:
file.write(previous_delimiter + line)
previous_delimiter = ''

Let me know any comments on it and if you test it and fails please also 
let me know. I don't have subversion account neither I don't know how to 
use it thus this email.


/amn

___
Mod_python mailing list
[EMAIL PROTECTED] <mailto:[EMAIL PROTECTED]>
http://mailman.modpython.org/mailman/listinfo/mod_python

Re: mod_python.util.StorageField.read_to_boundary has problems in 3.1 and 3.2

[Jim Gallacher]

Alexis,

I wanted to add that I'm testing your code.

Alexis Marrero wrote:


Let me know any comments on it and if you test it and fails please also 
let me know. I don't have subversion account neither I don't know how to 
use it thus this email.


You don't need an account to use subversion anonymously. Just install 
subversion and grab a mod_python working copy.


$ svn co http://svn.apache.org/repos/asf/httpd/mod_python/trunk trunk

This will checkout a working copy into a new directory called "trunk" on 
 your machine. All of the following commands assume you are working in 
trunk/.


Make your changes in your working copy, and then create a diff with:

$ svn diff lib/python/mod_python/util.py > your-patch.diff

The other commands which you'll find immediately useful are:

svn update  - update your working copy from the repository
svn status  - shows status of changes in your working copy
svn -u status   - shows status of your copy against the repository

I've found "Version Control with Subverion" is an excellent resource and 
is available online.

http://svnbook.red-bean.com/en/1.1/index.html

Jim

Re: mod_python.util.StorageField.read_to_boundary has problems in 3.1 and 3.2

[Alexis Marrero]

I don't have a function that creates the files but the I can point  
you to a file that has the problem, ironically is "Unix Haters  
Handbook" :) Well, at least is not the Python HH


http://research.microsoft.com/~daniel/uhh-download.html

It's MD5 is 9e8c42be55aac825e7a34d448044d0fe. I don't know what it  
ends up been after upload with read_to_boundary().


When you use the function to copy the file you will see that the  
digest will be e45979254297b0ece9c182a789d7966e.


I have other 5 files out of 78546 files that I'm testing it against  
that have the same issues, coincidentally there are all PDF files.  
Here is the script that I was testing it with.


def read_to_boundary(self, req, boundary, file):
''' read from the request object line by line with a maximum size,
until the new line starts with boundary
'''
previous_delimiter = ''
while 1:
line = req.readline(1<<16)
if line.startswith(boundary):
break

if line.endswith('\r\n'):
file.write(previous_delimiter + line[:-2])
previous_delimiter = '\r\n'

elif line.endswith('\r') or line.endswith('\n'):
file.write(previous_delimiter + line[:-1])
previous_delimiter = line[-1:]

else:
file.write(previous_delimiter + line)
previous_delimiter = ''

#f = file('Perl Bookshelf [4th Ed]/mre/final/ch06.pdf.new', 'a+')
#f = file('Pages User Guide.app/Contents/Resources/Italian.lproj/ 
Pages Manuale Utente.pdf', 'a+')

f = file('ugh.pdf.new', 'a+')
f.write('\r\n--myboundary--\r\n')
f.seek(0)
o = file('test.bin', 'wb')
read_to_boundary(None, f, '--myboundary', o)
o.close()

/amn


On Nov 6, 2005, at 11:58 AM, Jim Gallacher wrote:


Alexis,

I wanted to add that I'm testing your code.

Alexis Marrero wrote:

Let me know any comments on it and if you test it and fails please  
also let me know. I don't have subversion account neither I don't  
know how to use it thus this email.




You don't need an account to use subversion anonymously. Just  
install subversion and grab a mod_python working copy.


$ svn co http://svn.apache.org/repos/asf/httpd/mod_python/trunk trunk

This will checkout a working copy into a new directory called  
"trunk" on  your machine. All of the following commands assume you  
are working in trunk/.


Make your changes in your working copy, and then create a diff with:

$ svn diff lib/python/mod_python/util.py > your-patch.diff

The other commands which you'll find immediately useful are:

svn update- update your working copy from the repository
svn status- shows status of changes in your working copy
svn -u status- shows status of your copy against the repository

I've found "Version Control with Subverion" is an excellent  
resource and is available online.

http://svnbook.red-bean.com/en/1.1/index.html

Jim

Re: mod_python.util.StorageField.read_to_boundary has problems in 3.1 and 3.2

[Nicolas Lehuen]

Hi guys,

In the pure "if it ain't tested, it ain't fixed" fashion, I've added a
unit test for file upload to the test suite. It uploads a randomly
generated 1 MB file to the server, and check that the MD5 digest
returned by the server is correct. I could not reproduce Alexis' bug
report this way, however. But I then I added a test with the
UNIX-HATERS handbook file ugh.pdf, and bang, here comes the bug.

I've checked in both unit tests into subversion, so that you can play with them. I'm now going to test Alexis' fix.

Regards,
Nicolas2005/11/6, Alexis Marrero <[EMAIL PROTECTED]>:
I don't have a function that creates the files but the I can pointyou to a file that has the problem, ironically is "Unix HatersHandbook" :) Well, at least is not the Python HH
http://research.microsoft.com/~daniel/uhh-download.htmlIt's MD5 is 9e8c42be55aac825e7a34d448044d0fe. I don't know what itends up been after upload with read_to_boundary().When you use the function to copy the file you will see that the
digest will be e45979254297b0ece9c182a789d7966e.I have other 5 files out of 78546 files that I'm testing it againstthat have the same issues, coincidentally there are all PDF files.Here is the script that I was testing it with.
def read_to_boundary(self, req, boundary, file): ''' read from the request object line by line with a maximum size, until the new line starts with boundary ''' previous_delimiter = ''
 while 1: line = req.readline(1<<16) if line.startswith(boundary): break if line.endswith('\r\n'): file.write(previous_delimiter + line[:-2])
 previous_delimiter = '\r\n' elif line.endswith('\r') or line.endswith('\n'): file.write(previous_delimiter + line[:-1]) previous_delimiter = line[-1:]
 else: file.write(previous_delimiter + line) previous_delimiter = ''#f = file('Perl Bookshelf [4th Ed]/mre/final/ch06.pdf.new', 'a+')#f = file('Pages User Guide.app/Contents/Resources/Italian.lproj/
Pages Manuale Utente.pdf', 'a+')f = file('ugh.pdf.new', 'a+')f.write('\r\n--myboundary--\r\n')f.seek(0)o = file('test.bin', 'wb')read_to_boundary(None, f, '--myboundary', o)o.close()/amn
On Nov 6, 2005, at 11:58 AM, Jim Gallacher wrote:> Alexis,>> I wanted to add that I'm testing your code.>> Alexis Marrero wrote:>>> Let me know any comments on it and if you test it and fails please
>> also let me know. I don't have subversion account neither I don't>> know how to use it thus this email. You don't need an account to use subversion anonymously. Just> install subversion and grab a mod_python working copy.
>> $ svn co http://svn.apache.org/repos/asf/httpd/mod_python/trunk trunk>> This will checkout a working copy into a new directory called
> "trunk" on  your machine. All of the following commands assume you> are working in trunk/.>> Make your changes in your working copy, and then create a diff with:>> $ svn diff lib/python/mod_python/util.py > 
your-patch.diff>> The other commands which you'll find immediately useful are:>> svn update- update your working copy from the repository> svn status- shows status of changes in your working copy
> svn -u status- shows status of your copy against the repository>> I've found "Version Control with Subverion" is an excellent> resource and is available online.> 
http://svnbook.red-bean.com/en/1.1/index.html>> Jim>

Re: mod_python.util.StorageField.read_to_boundary has problems in 3.1 and 3.2

[Nicolas Lehuen]

OK, it looks like Alexis' fix solves the problem with ugh.pdf without
breaking the other unit tests. So I think we can safely integrate his
patch. Shall I do it ?

Regards,
Nicolas2005/11/6, Nicolas Lehuen <[EMAIL PROTECTED]>:
Hi guys,

In the pure "if it ain't tested, it ain't fixed" fashion, I've added a
unit test for file upload to the test suite. It uploads a randomly
generated 1 MB file to the server, and check that the MD5 digest
returned by the server is correct. I could not reproduce Alexis' bug
report this way, however. But I then I added a test with the
UNIX-HATERS handbook file ugh.pdf, and bang, here comes the bug.

I've checked in both unit tests into subversion, so that you can play with them. I'm now going to test Alexis' fix.

Regards,
Nicolas2005/11/6, Alexis Marrero <[EMAIL PROTECTED]>:

I don't have a function that creates the files but the I can pointyou to a file that has the problem, ironically is "Unix HatersHandbook" :) Well, at least is not the Python HH

http://research.microsoft.com/~daniel/uhh-download.htmlIt's MD5 is 9e8c42be55aac825e7a34d448044d0fe. I don't know what itends up been after upload with read_to_boundary().When you use the function to copy the file you will see that the
digest will be e45979254297b0ece9c182a789d7966e.I have other 5 files out of 78546 files that I'm testing it againstthat have the same issues, coincidentally there are all PDF files.Here is the script that I was testing it with.
def read_to_boundary(self, req, boundary, file): ''' read from the request object line by line with a maximum size, until the new line starts with boundary ''' previous_delimiter = ''
 while 1: line = req.readline(1<<16) if line.startswith(boundary): break if line.endswith('\r\n'): file.write(previous_delimiter + line[:-2])
 previous_delimiter = '\r\n' elif line.endswith('\r') or line.endswith('\n'): file.write(previous_delimiter + line[:-1]) previous_delimiter = line[-1:]

 else: file.write(previous_delimiter + line) previous_delimiter = ''#f = file('Perl Bookshelf [4th Ed]/mre/final/ch06.pdf.new', 'a+')#f = file('Pages User Guide.app/Contents/Resources/Italian.lproj/
Pages Manuale Utente.pdf', 'a+')f = file('ugh.pdf.new', 'a+')f.write('\r\n--myboundary--\r\n')f.seek(0)o = file('test.bin', 'wb')read_to_boundary(None, f, '--myboundary', o)o.close()/amn
On Nov 6, 2005, at 11:58 AM, Jim Gallacher wrote:> Alexis,>> I wanted to add that I'm testing your code.>> Alexis Marrero wrote:>>> Let me know any comments on it and if you test it and fails please
>> also let me know. I don't have subversion account neither I don't>> know how to use it thus this email. You don't need an account to use subversion anonymously. Just
> install subversion and grab a mod_python working copy.
>> $ svn co http://svn.apache.org/repos/asf/httpd/mod_python/trunk
 trunk>> This will checkout a working copy into a new directory called
> "trunk" on  your machine. All of the following commands assume you> are working in trunk/.>> Make your changes in your working copy, and then create a diff with:>> $ svn diff lib/python/mod_python/util.py > 
your-patch.diff>> The other commands which you'll find immediately useful are:>> svn update- update your working copy from the repository> svn status- shows status of changes in your working copy
> svn -u status- shows status of your copy against the repository>> I've found "Version Control with Subverion" is an excellent> resource and is available online.> 

http://svnbook.red-bean.com/en/1.1/index.html>> Jim>

Re: mod_python.util.StorageField.read_to_boundary has problems in 3.1 and 3.2

[Alexis Marrero]

Nicolas,Not that I'm the one to give permission whether to integrate things or not, but just to let you know I don't even have svn installed so I won't do it. At least not for a while...BTW, if there are some cherrypy developers in this mailing list, the CherryPy function that handles file uploads DOES also has the same issue. I'm not subscribe to CherryPy dev group thus the cross posting.From http://svn.cherrypy.org/trunk/cherrypy/_cpcgifs.py     24      def read_lines_to_outerboundary(self):    25          """Internal: read lines until outerboundary."""    26          next = "--" + self.outerboundary    27          last = next + "--"    28          delim = ""    29          last_line_lfend = True    30          while 1:    31              line = self.fp.readline(1<<16)    32              if not line:    33                  self.done = -1    34                  break    35              if line[:2] == "--" and last_line_lfend:    36                  strippedline = line.strip()    37                  if strippedline == next:    38                      break    39                  if strippedline == last:    40                      self.done = 1    41                      break    42              odelim = delim    43              if line[-2:] == "\r\n":    44                  delim = "\r\n"    45                  line = line[:-2]    46                  last_line_lfend = True    47              elif line[-1] == "\n":    48                  delim = "\n"    49                  line = line[:-1]    50                  last_line_lfend = True    51              else:    52                  delim = ""    53                  last_line_lfend = False    54              self.__write(odelim + line)/amnOn Nov 6, 2005, at 4:20 PM, Nicolas Lehuen wrote:OK, it looks like Alexis' fix solves the problem with ugh.pdf without breaking the other unit tests. So I think we can safely integrate his patch. Shall I do it ?  Regards, Nicolas2005/11/6, Nicolas Lehuen <[EMAIL PROTECTED]>: Hi guys,  In the pure "if it ain't tested, it ain't fixed" fashion, I've added a unit test for file upload to the test suite. It uploads a randomly generated 1 MB file to the server, and check that the MD5 digest returned by the server is correct. I could not reproduce Alexis' bug report this way, however. But I then I added a test with the UNIX-HATERS handbook file ugh.pdf, and bang, here comes the bug.  I've checked in both unit tests into subversion, so that you can play with them. I'm now going to test Alexis' fix.  Regards, Nicolas2005/11/6, Alexis Marrero <[EMAIL PROTECTED]>:  I don't have a function that creates the files but the I can pointyou to a file that has the problem, ironically is "Unix HatersHandbook" :) Well, at least is not the Python HH http://research.microsoft.com/~daniel/uhh-download.htmlIt's MD5 is 9e8c42be55aac825e7a34d448044d0fe. I don't know what itends up been after upload with read_to_boundary().When you use the function to copy the file you will see that the digest will be e45979254297b0ece9c182a789d7966e.I have other 5 files out of 78546 files that I'm testing it againstthat have the same issues, coincidentally there are all PDF files.Here is the script that I was testing it with. def read_to_boundary(self, req, boundary, file): ''' read from the request object line by line with a maximum size, until the new line starts with boundary ''' previous_delimiter = ''  while 1: line = req.readline(1<<16) if line.startswith(boundary): break if line.endswith('\r\n'): file.write(previous_delimiter + line[:-2])  previous_delimiter = '\r\n' elif line.endswith('\r') or line.endswith('\n'): file.write(previous_delimiter + line[:-1]) previous_delimiter = line[-1:]  else: file.write(previous_delimiter + line) previous_delimiter = ''#f = file('Perl Bookshelf [4th Ed]/mre/final/ch06.pdf.new', 'a+')#f = file('Pages User Guide.app/Contents/Resources/Italian.lproj/ Pages Manuale Utente.pdf', 'a+')f = file('ugh.pdf.new', 'a+')f.write('\r\n--myboundary--\r\n')f.seek(0)o = file('test.bin', 'wb')read_to_boundary(None, f, '--myboundary', o)o.close()/amn On Nov 6, 2005, at 11:58 AM, Jim Gallacher wrote:> Alexis,>> I wanted to add that I'm testing your code.>> Alexis Marrero wrote:>>> Let me know any comments on it and if you test it and fails please >> also let me know. I don't have subversion account neither I don't>> know how to use it thus this email. You don't need an account to use subversion anonymously. Just > install subversion and grab a mod_python working copy. >> $ svn co http://svn.apache.org/repos/asf/httpd/mod_python/trunk trunk>> This will checkout a working copy into a new directory called > "trunk" on  your machine. All of the following commands assume you> are working in trunk/.>> Make your changes in your working copy, and then create a diff with:>> $ svn diff lib/python/mod_py

Re: mod_python.util.StorageField.read_to_boundary has problems in 3.1 and 3.2

[Jim Gallacher]

I've been spending some quality time with hexedit, vim and a little bit 
of python. I can now generate a file which can be used in the unit test.


The problem seems to occur when a '\r' character is right at 
readBlockSize boundary, which is 65368 in the current mod_python.util.


I have not yet tested Alexis's fix yet. It's possible that it doesn't 
fix the problem but is just masking it since his code uses a 
readBlockSize of 65536, which will not be a problem for ugh.pdf.


The attached script will generate a file that can be used for testing. I 
haven't tried to integrate it into Nicolas's unit test as I wanted to 
share my findings with everyone asap. I'll spend some more time on this 
tommorrow.


Happy Bug Hunting,
Jim

Nicolas Lehuen wrote:

Hi guys,

In the pure "if it ain't tested, it ain't fixed" fashion, I've added a 
unit test for file upload to the test suite. It uploads a randomly 
generated 1 MB file to the server, and check that the MD5 digest 
returned by the server is correct. I could not reproduce Alexis' bug 
report this way, however. But I then I added a test with the UNIX-HATERS 
handbook file ugh.pdf, and bang, here comes the bug.


I've checked in both unit tests into subversion, so that you can play 
with them. I'm now going to test Alexis' fix.



#!/usr/bin/env python

import sys

def generate_file(offset=-1, readBlockSize=65368, fname='testfile'):
""" Generate a file which causes the error with file upload
The default offset of -1 should generate a file which will
be corrupted by the file upload.

Test Results:

offset = -2
235a26d377f0b49ba2fd8706f27dc9e5  testfile.-2.bin
235a26d377f0b49ba2fd8706f27dc9e5  testfile.-2.bin.res

offset = -1
** THIS ONE FAILS **
39626fae4bd0812a2522ebe45c23e186  testfile.-1.bin
e1fc1008bfa9a98dde6c9481e3c83c43  testfile.-1.bin.res

offset = 0
8bd673f325f250a90d623defe1fff11d  testfile.0.bin
8bd673f325f250a90d623defe1fff11d  testfile.0.bin.res

offset = 1
d0f4adf904cdc058f0ac7013baa83998  testfile.1.bin
d0f4adf904cdc058f0ac7013baa83998  testfile.1.bin.res

"""

# readBlockSize is the same as the value from mod_python.util
f = open('%s.%d.src' % (fname, offset), 'w')
f.write('a'*100)
f.write('\r\n')

block_size =  readBlockSize + offset
f.write('b'*block_size)
f.write('\rccc')

f.write('d'*100)
f.write('\r\n')

f.close()

if __name__ == '__main__':
generate_file(offset=int(sys.argv[1]))

Re: mod_python.util.StorageField.read_to_boundary has problems in 3.1 and 3.2

[Gregory (Grisha) Trubetskoy]

So I guess this means we roll and vote on a 3.2.5b?

Grisah

On Sun, 6 Nov 2005, Nicolas Lehuen wrote:


OK, it looks like Alexis' fix solves the problem with ugh.pdf without
breaking the other unit tests. So I think we can safely integrate his patch.
Shall I do it ?

Regards,
Nicolas

Re: mod_python.util.StorageField.read_to_boundary has problems in 3.1 and 3.2

[Jim Gallacher]

Gregory (Grisha) Trubetskoy wrote:


So I guess this means we roll and vote on a 3.2.5b?



As much as it pains me to say it, but yes, this is a must fixm so it's 
on to 3.2.5b.


I think we need to do some more extensive testing on Alexis's fix before 
we roll 3.2.5b. His read_to_boundary is much simpler than the current 
one. This makes me wonder if there is some magic happening in the 
current version which is solving some weird problems, or is his code 
just that much better?


I'm feeling a little dull right now so all the code just blurs together. 
It also doesn't help that util.py uses *3 spaces* for the indent. Yikes. 
How the heck did that happen? :(


I'll take another look tomorrow.

Jim

Re: mod_python.util.StorageField.read_to_boundary has problems in 3.1 and 3.2

[Nicolas Lehuen]

Well, I've re-read the previous code and it looks like it does almost
the same thing except it is bugged :). CherryPy's implementation is
almost the same except it ought to work.

Jim, I've integrated your tricky file into the unit test. Alexis'
version passes all tests, whereas the current version fails on your
file and ugh.pdf. So I guess we can call it a day and integrate Alexis'
fix.
One thing which is different is the use of 1<<16 = 65536 as
read block size instead of 65368. It was Barry Pearce who contributed
the latest version of read_to_boundary, he must have known why this
block size was good (or was it his lucky number ?). Anyway, I've
changed Alexis' code to use the readBlockSize variable and changed its
value to 1<<16. I've done tests with both 1<<16 and 65638
on the server side and on the client side (changing the tricky file
generation), and the new code works.

What we should have done, while we were at spending a lot of time on a
trivial string search problem, is at least to implement a fast
Boyer-Moore search algorithm, and drop the use of readline. But I guess
this will be for 3.4 ;).

And yes, this 3 spaces indent is dreadful.

Regards,
Nicolas

2005/11/7, Jim Gallacher <[EMAIL PROTECTED]
>:Gregory (Grisha) Trubetskoy wrote:>> So I guess this means we roll and vote on a 
3.2.5b?>As much as it pains me to say it, but yes, this is a must fixm so it'son to 3.2.5b.I think we need to do some more extensive testing on Alexis's fix beforewe roll 3.2.5b. His read_to_boundary is much simpler than the current
one. This makes me wonder if there is some magic happening in thecurrent version which is solving some weird problems, or is his codejust that much better?I'm feeling a little dull right now so all the code just blurs together.
It also doesn't help that util.py uses *3 spaces* for the indent. Yikes.How the heck did that happen? :(I'll take another look tomorrow.Jim

Re: mod_python.util.StorageField.read_to_boundary has problems in 3.1 and 3.2

[Jim Gallacher]

Nicolas Lehuen wrote:
Well, I've re-read the previous code and it looks like it does almost 
the same thing except it is bugged :). CherryPy's implementation is 
almost the same except it ought to work.


Jim, I've integrated your tricky file into the unit test. Alexis' 
version passes all tests, whereas the current version fails on your file 
and ugh.pdf. So I guess we can call it a day and integrate Alexis' fix.


One thing which is different is the use of 1<<16 = 65536 as read block 
size instead of 65368. It was Barry Pearce who contributed the latest 
version of read_to_boundary, he must have known why this block size was 
good (or was it his lucky number ?). Anyway, I've changed Alexis' code 
to use the readBlockSize variable and changed its value to 1<<16. I've 
done tests with both 1<<16 and 65638 on the server side and on the 
client side (changing the tricky file generation), and the new code works.


There is a failure mode in Alexis' code. If the final line in the file 
has length == readBlockSize - 1, the boundary (eg. '\r\n--myboundary') 
gets split after the '\r'. The read terminates properly, but a stray 
'\r' gets appended to the file. Interestingly, the original code 
mentions that this is unlikely to happen, but seems to handle it anyway.


This bit of code will generate a file which will cause a problem for 
Alexis' read_to_boundary.


def generate_split_file(offset=-1,
  readBlockSize=65368,
  fname='testfile'):
f = open(fname, 'w')
f.write('a'*50)
f.write('\r\n')
block_size =  readBlockSize + offset
f.write('b'*block_size)
f.close()

I'll write a new unit test to cover this situation.

I'm also thinking that the unit test should not hard code the 
readBlockSize right in the test. It would be better to have a global 
variable at the top of test.py, with a note in both test.py and util.py 
that it's value must be the same in both files. Otherwise, if the value 
of readBlockSize changes in util.py, we may end up with regressions that 
the unit test will not catch.


As far as Nicolas' test_fileupload unit test is concerned I'd be 
inclined to remove the test for ugh.pdf. I'm pretty confident that the 
simulated 'tricky' file is an adequate test, plus we can't be sure that 
ugh.pdf will not change in the future.


Alexis mentioned that he had 5 pdf files that were corrupted. If the 
other files are publicly accessible I could take a look and see if they 
are failing the same way, with a '\r' character right at a readBlockSize 
boundary point. Alternatively, I could polish my test script and he can 
run it against his files. If we do that then I'm confident we can 
generate a synthetic file for testing and not depend on ugh.pdf


Jim

Re: mod_python.util.StorageField.read_to_boundary has problems in 3.1 and 3.2

[Jim Gallacher]

Alexis Marrero wrote:

Jim,

Nicolas,

Thanks for sending the function that creates the test file. However I 
ran it to create the test file, and after uploading the file the MD5 
still the same.


Did you call it with the same block size as you are using in your code? 
The '\r' character must appear in the file right at the readBlockSize 
boundary.


ie.

generate_file(offset=-1, readBlockSize=1<<16, fname='testfile')

Jim

PS. Please make sure you cc. python-dev when replying. It's best to keep 
discussion on the list.

Re: mod_python.util.StorageField.read_to_boundary has problems in 3.1 and 3.2

[Jim Gallacher]

Jim Gallacher wrote:

Alexis Marrero wrote:


Jim,
Thanks for sending the function that creates the test file. However I 
ran it to create the test file, and after uploading the file the MD5 
still the same.


Just to clarify, is this for your new read_to_boundary or the one in 
3.2? If it's for yours then the MD5 sum *should* be the same, since 
that's what you fixed. :)


Did you call it with the same block size as you are using in your code? 
The '\r' character must appear in the file right at the readBlockSize 
boundary.


ie.

generate_file(offset=-1, readBlockSize=1<<16, fname='testfile')

Re: mod_python.util.StorageField.read_to_boundary has problems in 3.1 and 3.2

[Alexis Marrero]

Sorry for all this emails, but my system depends 100% on mod_python  
specially file uploading. :)


On Nov 7, 2005, at 2:04 PM, Jim Gallacher wrote:


Alexis Marrero wrote:


Jim,
Nicolas,
Thanks for sending the function that creates the test file.  
However I ran it to create the test file, and after uploading the  
file the MD5 still the same.




Did you call it with the same block size as you are using in your  
code? The '\r' character must appear in the file right at the  
readBlockSize boundary.
YES I ran it with generate_file(offset=-1, readBlockSize=1<<16,  
fname='testfile')

and the MD5 of the input and output files match.



ie.

generate_file(offset=-1, readBlockSize=1<<16, fname='testfile')

Jim

PS. Please make sure you cc. python-dev when replying. It's best to  
keep discussion on the list.