Hello everybody,
I just recently started looking at supporting Python 3 with one of my libraries
(Werkzeug), mainly because the MoinMoin projects considers using it which uses
the library in question. Right now what Werkzeug does is consider HTTP being
Unicode aware in the sense that everything t
Hi,
This is my first reply in a list of replies for Grahams lengthy blog
post about WSGI 3 [1]. I break it up into multiple separate threads so
that this can be discussed easier.
> What should be highlighted is that for Jython, as I understand it at
> least, when reading from a socket connection
Hi,
Graham mentioned that the WSGI development might further drift apart
based on the changes Ian Bicking and I did on DjangoCon in a separate hg
repository [1] for the WSGI PEP.
I just want to point out that these are in no way final and are further
intended to only clarify some of the wrong wor
Hi,
Graham currently proposes[1] the following behaviors for Strings in WSGI
(Python version independent). However this mail only covers the Python
3 part which I assume becomes a separate section in the PEP or even WSGI
version.
Terminology:
byte string == contains bytes
unicode string ==
Hi,
For this topic I would love to remember everybody that the web is
currently changing and will even more change in the future which will
probably also mean that a lot of what we're doing currently might not be
common practise in the near future.
WSGI is currently not doing to well for asyncron
Hi,
René Dudfield schrieb:
> I don't like yours and Ians changes with regard to cgi. cgi exists.
> Breaking wsgi apps on cgi is silly.
Can you give an example on where we break CGI compatibility?
> I think you mean pre-2.2 support, not python 2.2? iterators came
> about in python 2.2.
That migh
Hi,
René Dudfield schrieb:
> It says "Because of this future revisions of WSGI will most likely
> switch away from a raw CGI environment to require the server to
> provide these values to be quoted and available on a different key."
This information would be additional information of course!
> Al
Hi,
Graham Dumpleton schrieb:
> I believe it does matter and that it contains ASCII possibly doesn't
> mean it is somehow simpler. The reason is that URL reconstruction
> recipe as per WSGI PEP has to work. Ie.
> *snip*
That of course will not work and is not something we should aim for.
There is
Hi,
Graham Dumpleton schrieb:
> On top of the issues above, Armin believes 2to3 gives better results
> where bytes everywhere interpretation is used. Has anyone else
> actually tried 2to3 and have the experience with it?
You slightly misquoted me. I said that 2to3 gives good results on high
level
Hi,
Let me backup a bit here.
We have to focus on two difference use cases for WSGI on Python 3. The
one is the application that should continue to work on Python 3, the
other one is the application that was designed for Python 3.
In both cases let's just assume that this application is using
W
Hi,
René Dudfield schrieb:
> Perhaps a good way to test that, is to make a smallish example wsgi
> program to port to python3, using the various proposals... or the
> proposal most liked.
Not a good idea. Because a small WSGI application directly on top of
WSGI behaves completely different than a
Hi,
P.J. Eby schrieb:
> newer spec. On CPython at least, this can be implemented using
> greenlets, and on other Python implementations it could be done with
> threads. Here's a quick and dirty, untested sketch (no error
> checking, no version handling) of how it could be done with greenlets:
Hi,
Ian Bicking schrieb:
> What's wrong with this simpler approach to the conversion?
It buffers, you can no longer do this:
request.write('processing data')
request.flush()
...
request.write('data processed')
request.flush()
But that's not too common and people should rather rewr
Hi,
Massimo Di Pierro schrieb:
> I liked your idea very much Rene' , so I made this
Can you please stop that before you do any more damage? Your code is
not even anywhere close to what was discussed and has tons of errors and
ugly bits and pieces in there.
Again. An example does not bring us an
Hi,
I spent the last few hours now figuring out what decisions Python took
in the standard library to get a better understanding of unicode in
Python 3 and how it affects web applications.
Let's sum up the current state of encodings in the web world:
RFC 2616 specifies the header encoding as "la
Hi,
Armin Ronacher schrieb:
> urllib.parse appears to be buggy with bytestrings:
>
> I'm pretty sure the latter is a bug and I will file one, however if
> there is broken behavior with bytestrings in Python 3.1 that's another
> thing we have to keep in mind.
I have to
Hi,
Another observation from the HTTP server that comes with the Python 3
Standard Library: it does not support non-ASCII headers:
def send_header(self, keyword, value):
"""Send a MIME header."""
if self.request_version != 'HTTP/0.9':
self.wfile.write(("%s: %s\r\n"
Hi,
Armin Ronacher schrieb:
> I have to correct myself, there are separate functions for byte quoting.
> (parse.unquote_to_bytes, parse.quote_from_bytes).
However, urlencode and urldecode are string only.
Regards,
Armin
___
Web-SIG mailing li
Hi,
Armin Ronacher schrieb:
> http://paste.pocoo.org/show/140501/
Corrected version without Werkzeug leftovers:
http://paste.pocoo.org/show/140502/
Regards,
Armin
___
Web-SIG mailing list
Web-SIG@python.org
Web SIG: http://www.python.org/sigs/web-
Hi,
René Dudfield schrieb:
> I think that shows that they are being handled differently depending
> on type. Which is against polymorphism... but some people prefer to
> have separate functions for different types(in and out). I don't
> think other python functions do this though. So maybe this
Hi,
Graham Dumpleton schrieb:
> So, no strict need to make the WSGI adapter do it differently. You may
> want to only do that if concerned about overhead of transcoding.
>
> Transcoding just these is most probably going to be less overhead than
> the WSGI adapter having to set up both unicode and
Hi,
René Dudfield schrieb:
> Rather than using a 2to3 tool - which then makes you have two versions
> of your code, making the code work in python 2.x and 3.x. 2to3
> outputs python2.x incompatible code - when it doesn't have to.
2to3 is intended to be run automatically for each release. You wou
Hi,
René Dudfield schrieb:
> What is proposed:
Where was that proposed?
> 1. Default utf-8 to be used.
That's a possibility yes, but it has to be carefully be considered.
> 2. A buffer to be used for raw data.
What is raw data? If you mean we keep the unencoded data around, I
would stro
Hi,
I know I pretty much SPAM the list here now which is why I added all the
changes of WSGI 1.0 and what could become WSGI 1.1 into a repo on
bitbucket as two PEPS:
http://bitbucket.org/ianb/wsgi-peps/src/
pep-0333.txt
This is basically just a new revision for PEP 333 changing the following
t
Hi,
Graham Dumpleton schrieb:
> Regardless of the details of changes being made to the PEP and the
> creation of any new ones, do we need to first agree on the overall
> direction we are going to take. Ie., the grand plan at a high level.
Indeed. The 0333 changes are mostly uncontroversial and ca
Hi,
Armin Ronacher schrieb:
> WSGI 1.1 as currently specified in would be pretty uncontroversial
> on Python 2.x because of the str/unicode coercion that Python implicitly
> applies and that this is basically the only change.
Based on the table, is 2.0 now.
> That would
Hello everybody,
Thanks to Graham Dumpleton and Robert Brewer there is some serious
progress on WSGI currently. I proposed a roadmap with some PEP changes
now that need some input.
Summary:
WSGI 1.0 stays the same as PEP 0333 currently is
WSGI 1.1 becomes what Ian and I added to
Hi,
P.J. Eby schrieb:
> This discussion has been going on for so long that I've already
> forgotten what the problem was with just using the original 1.0 spec
> for 3.X, i.e., using native strings for everything, using latin-1
> encoding. The only things I can recall off the top of my head are
Hi,
Chris McDonough schrieb:
> Personally, I find it a bit hard to get excited about Python 3 as a web
> application deployment platform.
Everybody feels that way currently. But if we don't fix WSGI that will
never change.
> Given this point of view, it would be extremely helpful if someone cou
Hi,
Robert Brewer schrieb:
> urllib.unquote, for one. We had to make a version which accepts bytes
> (and outputs bytes). But it's only 8 lines of code.
Here a patch for urllib.parse that restores Python 2.x behavior.
Because it also changes behavior for Python 3.x I have not yet submitted
it for
Hi,
James Bennett schrieb:
> Well, ordinarily I'd be inclined to agree: HTTP deals in bytes, so an
> interface to HTTP should deal in bytes as well.
If it was just that I would be happy to stay with bytes. But unless the
standard library changes in the way it works on Python 3 there is not
much b
Hi,
René Dudfield wrote:
> That's all the arguing and explaining I'll do on this - I'm not going
> to rewrite cherrypy for you as proof.
If it just puts a burden on implementors on the client and server side
and there is no proof for it to be faster for real world applications we
can probably just
Hi,
And Clover schrieb:
> Seems like a mistake. If the middleware knows iso-8859-7 is in use, it
> would need to transcode the charset regardless of whether the
> initially-submitted bytes were a valid UTF-8 sequence or not. Otherwise
> the application would break when fed with eg. Greek words
Hi,
Henry Precheur schrieb:
> surrogateescape isn't implemented (yet?) for Python 2. That's not an
> issue if the 'new' WSGI sticks to native strings.
So the same standard should have different behavior on different Python
versions? That would make framework code a lot more complicated.
Regards
Hi,
P.J. Eby schrieb:
> Actually, latin-1 bytes encoding is the *simplest* thing that could
> possibly work, since it works already in e.g. Jython, and is actually
> in the spec already... and any framework that wants unicode URIs
> already has to decode them, so the code is already written.
Hi,
Mark Nottingham schrieb:
> HTTP headers *are* ASCII; RFC2616 defined them to be ISO-8859-1, but
> HTTPbis currently takes the stance that they're ASCII, as in practice
> Latin-1 isn't used and may introduce interop problems.
In practise non-ascii data ends up in headers.
> What does it me
Hi,
Ian Bicking schrieb:
> Request headers, which you didn't split out... those I'm not sure. I'd
> *like* them to be native. But damn, I'm just not sure quite how.
> surrogateescape? Latin1? Latin1 as a kind of poor man's surrogateescape
> isn't so bad. And the headers *should* be ASCII for
Hi,
Alan Kennedy schrieb:
> So, if nobody implements that, then why are we trying to standardise it?
I think that was just one of the ideas that were discussed.
Just to sum it up a bit where we already went:
- my initial plan was going bytes everywhere. Turns out, on Python 3
this is nearly i
Hi,
Ian Bicking schrieb:
> Tell doesn't have particular overhead except to keep track of how many bytes
> have been read. That would allow libraries to at least detect contention
> for wsgi.input. I wish seek were detectable, though I agree it shouldn't be
> required at all.
Ah right. Thought t
Hi,
Alan Kennedy schrieb:
> 2. Give the programmer (possibly mojibake) unicode strings in the WSGI
> environ anyway
> 3. And let them solve their problems themselves, using server
> configuration or bespoke middleware
Because that problem was solved a long ago in applications themselves.
Webob, We
Hi,
Alan Kennedy schrieb:
> from miscellaneous unknown character sets into unicode, with out any
> mistakes, under all possible WSGI environments, e.g.
No, they know the character sets. You tell them what character set you
want to use. For example you can specify "utf-8", and they will
decode/en
Hi,
Alan Kennedy schrieb:
> Hmmm, define "know" ;-)
The charset of incoming data, the charset of URLs, the charset of
outgoing data, the charset of whatever the application uses, is what the
application decides it to be. Most new applications go with utf-8 for
everything these days.
> I see this
Hi,
Alan Kennedy schrieb:
> So you don't like the way that Django, Werkzeug, WebOb, etc, do it
> now, even though they appear to be mostly successful, and you're happy
> to cite them as such?
Server != Application.
> From the applications point of view, a framework-level configuration
> variable
Hi,
P.J. Eby schrieb:
> If they are putting objects of type 'unicode' under WSGI-defined
> environ keys on Python 2.x, they are *not WSGI compliant*.
Who is doing that? They are using bytestrings when talking to WSGI and
only expose unicode to the user of the framework. That happens on top
of WS
Hi,
P.J. Eby schrieb:
> What roundtrips? If they're operating on unicode, either they're in
> violation of the spec (in which case, f*** them), or they're already
> running a decode every time they pull something out of the
> environ... and using latin-1 or surrogates is only one encoding cal
Hi,
And Clover schrieb:
> This is absolutely the opposite of what I want as an application author.
> I want to hand out my WSGI application that uses UTF-8 and know that
> wherever it is deployed the non-ASCII characters will go through without
> getting mangled.
I could not agree more.
Probab
Hey,
After all that discussions about unicode and path info and all related
problems I would love to remind everybody how well we are doing.
I just had a brief discussion with Christian Neukirchen (The Rack
developer) about the state of URL quoting and unicode and this is how it
looks in Ruby lan
Hi,
Ian Bicking schrieb:
> I propose we switch primarily to "native" strings: str on both Python 2 and
> 3.
I'm starting to think that this is the best idea.
> I then propose that we eliminate SCRIPT_NAME and PATH_INFO. Instead we
> have:
IMO they should stick around for compatibility with older
Hi,
Graham Dumpleton schrieb:
> So, rather than throw away completely the idea of bytes everywhere,
> and rewrite the WSGI specification, we could instead say that the
> existing conceptual idea of WSGI 1.0 is still valid, and just build on
> top of it a translation interface to present that as un
Hi,
On 7/17/10 1:20 AM, Chris McDonough wrote:
> Let me know if I'm missing something.
The only thing you miss is that the bytes type of Python 3 is badly
supported in the stdlib (not an issue if we reimplement everything in
our libraries, not an issue for me) and that the bytes type has no
st
Hi,
On 7/17/10 9:15 AM, Ian Bicking wrote:
This is an Apache-specific issue. It definitely doesn't apply to
paste.httpserver, I doubt CherryPy or wsgiref. I don't really know how
Nginx or other servers work.
This will be an issue for every server that...
* supports unicode filesystems
* de
Hi,
On 7/17/10 12:57 PM, Armin Ronacher wrote:
In fact, this will be an issue for things like middlewares that want to
map applications to paths. In fact, this already is an issue on Python 2
already, just that nobody cares.
s/applications/serving static files from folders/
Regards,
Armin
Hi,
Is there a status update on that now I missed? Did something decide on
bytes for the environment values or are we still unsure about that?
From a discussion lately I had with Graham on #pocoo it seems like he
lost interest on supporting WSGI on Python 3 for the time being due to
lack of
Hi,
On 2010-08-27 5:45 AM, P.J. Eby wrote:
> At 01:37 AM 8/27/2010 +0200, Armin Ronacher wrote:
> To the extent we're "unsure", I think the holdup is simply that nobody
> has tried doing an all-bytes WSGI implementation -- unless of course you
> count all our Python 2
Hi,
On 2010-08-27 6:05 PM, Christoph Zwerschke wrote:
> Btw, another problem with this is that the lower() method does not know
> that it has to use latin1 when lowercasing.
That is not a problem insofar that case insensitive HTTP tokens are
limited to ASCII only.
Regards,
Armin
_
Hi,
On 2010-08-28 1:04 PM, Georg Brandl wrote:
Let me just throw in here that it's *NOT* too late to do something about
Python 3.2. It is not even in beta state yet, and I am very willing to
introduce the changes to make web programming work again, or even hold
up 3.2 for a bit if you need more
Hi,
On 2010-09-16 2:05 AM, P.J. Eby wrote:
The first thing I notice is that web3.async appears to force all
existing middleware to delete it from the environment if it wishes to
remain compatible, unless it adapts to support receiving callables itself.
In terms of backwards compatibility, we hav
Hi,
On 2010-09-16 3:33 AM, James Mills wrote:
As a developer of an asynchronous framework myself, I'm actually not
really sure what to think of
the whole web3.async "thing" yet... My feeling(s) are that other web
frameworks are just doing to
do their own thing anyway...
Any chances of finding s
Hi,
On 9/16/10 1:23 PM, Dirkjan Ochtman wrote:
I find the order of the application return arguments really annoying,
could it just be status, headers, body? Mirrors the actual structure
of the request, which is easier to remember IMO.
The motivation is that you can pass that to constructors of r
Hi,
On 9/16/10 1:44 PM, Tarek Ziadé wrote:
I propose to write in the PEP that a middleware should provide an
"app" attribute to get the wrapped application or middleware.
It seems to be the most common name used out there.
What about middlewares that encapsulate more than one application?
Reg
Hi,
On 9/16/10 2:38 PM, Tarek Ziadé wrote:
True... I don't know what's the best option here.. I guess we need to
provide all children so one may visit the whole graph.
Another gripe I have with WSGI is that if you attempt to combine
applications together with a dispatcher middleware, the inner
Hi,
On 9/16/10 6:19 PM, Robert Brewer wrote:
1. Hooray for all-byte output.
Hooray for agreeing :)
3. +1 on (status, headers, body) in that order. Your own example code
composed them in that order, and then re-arranged them for output!
One of the benefits of a new spec is the op
Hi,
On 9/16/10 7:56 PM, Ty Sarna wrote:
Agreed. Among many other reasons, it seems poor from a Python 3
marketing perspective to introduce a name change that implies
something totally different from WSGI that will require major
rewrites to port to. It's also a poor choice as a rebranding even if
Hi,
Here some things comments summarized and how things will change:
- The order of the response tuple.
The majority of this list wants it to be changed to the standard
(status, headers, body) format, and we agree. The original
motivation was passing it to the constructor of a common
re
Hi,
On 9/17/10 3:43 AM, Ian Bicking wrote:
Not if you are working with the URL-encoded paths.
SCRIPT_NAME / PATH_INFO will always stay unencoded and the current spec
requires the web3.script_name thing to only be provided if the server
can safely provide that. So at least for the fallback, we
Hi,
On 9/17/10 4:21 AM, Ian Bicking wrote:
The Title header (in Atompub) also suggests 2047, but that's essentially
an ASCII conversion like URL quoting. It looks something like
=?iso-8859-1?q?p=F6stal?=
Yep. That was mere a fun fact I wanted to share. Was not aware of HTTP
specifying a non l
Hi,
On 9/17/10 11:40 AM, And Clover wrote:
This is why I am continuing to plead for a 'script_name/path_info are
authoritative' flag in environ that applications can use to detect
situations where it is safe to go ahead and rely on them. I want to say
"Unicode paths are supported if your server/
Hi,
On 9/17/10 5:42 PM, P.J. Eby wrote:
So, do you have an example of what some real-world code is going to *do*
with this information? i.e., what's the use case for knowing the precise
degree of messed-uppedness of the path? ;-)
Actually, I can see a couple of use cases. I have a blog that by
Hi,
On 9/17/10 7:43 PM, Ian Bicking wrote:
I'm also not sure what motivated this particular change, but I don't
have any opinion one way or the other.
Motivation is that WSGI wants servers to do something like this:
if len(iterable) == 1 and content_length_header_missing:
headers.app
Hi,
On 9/20/10 6:31 PM, Matt Goodall wrote:
Servers should definitely not transform a HEAD to a GET.
There are some good reasons why it currently has to. I haven't read the
link in question but I had a discussion with Graham a few days ago on
Skype and he outlined the issue in detail.
I wil
Hi,
On 10/22/10 2:35 AM, Graham Dumpleton wrote:
has said:
"""Hopefully not. WSGI could do better and there is a proposal for
that (444)."""
Just to give this some more context: I think WSGI (even in Python 2) is
faulty and we have the possibility now to fix it. Nobody in the web
communit
Hi,
On 10/23/10 7:43 PM, P.J. Eby wrote:
I don't think it's an either-or case. PEP just means that there's a
clear path to port WSGI 1 apps. If somebody wants to champion a WSGI
1.1, a 2.0, and whatever else, that's great!
Oh, I was not denying that. The original post on reddit to which I
Hi,
I personally probably do not want to participate in this discussion much
but I want to leave some thoughts in case someone finds them useful.
I personally think that fundamentally "concurrent programming" and just
getting access to a socket is not something that fits into a generically
d
Hi,
On 04/01/2016 16:15, Cory Benfield wrote:
I don’t believe that will work.
Correct. This cannot be done except for very simplistic servers.
Regards,
Armin
___
Web-SIG mailing list
Web-SIG@python.org
Web SIG: http://www.python.org/sigs/web-sig
Un
Hi,
On 04/01/2016 16:30, Cory Benfield wrote:
Your core question seems to be: “why do we need a spec that specifies
concurrency?” I think this is reasonable. One way out might be to
take the route of ASGI[0], which essentially uses a message broker to
act as the interface between server and appl
Hi,
I just want to reply to this because I think many people seem to be
missing why things are done in a certain way. Especially if the appear
to be odd.
On 05/01/2016 12:26, Cory Benfield wrote:
1. WSGI is prone to header injection vulnerabilities issues by
designdue to the conversion of H
Hi,
On 05/01/2016 13:09, Luke Plant wrote:
Just to add my 2c - as another Django developer, I agree completely with
Aymeric here. My own experience was that the HTTP handling done by WSGI
(especially URL handing, HTTP header mangling, os.environ as a
destination - all due to CGI compatibility -
77 matches
Mail list logo
