[issue45807] Strange SyntaxError message / suggestions for "@x = 123"
New submission from Pierre Quentel : In CPython 3.10 : Python 3.10.0 (tags/v3.10.0:b494f59, Oct 4 2021, 19:00:18) [MSC v.1929 64 bit (AMD64)] on win32 Type "help", "copyright", "credits" or "license" for more information. >>> @x = 123 File "", line 1 @x = 123 ^^^ SyntaxError: invalid syntax. Maybe you meant '==' or ':=' instead of '='? but both suggestions lead to a SyntaxError : >>> @x == 123 ... File "", line 2 ^ SyntaxError: invalid syntax >>> @x := 123 ... File "", line 2 ^ SyntaxError: invalid syntax >>> Maybe an error message such as "cannot assign to decorator" would be more appropriate ? -- components: Parser messages: 406351 nosy: lys.nikolaou, pablogsal, quentel priority: normal severity: normal status: open title: Strange SyntaxError message / suggestions for "@x = 123" type: enhancement versions: Python 3.10 ___ Python tracker <https://bugs.python.org/issue45807> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue44741] Pattern Matching - star subpattern with a subject derived from collections.abc.Sequence
Pierre Quentel added the comment: I found why len() is required, it's to avoid trying to match the subject (thus consuming a part of it) if its length is less than the number of non-star patterns, as explained in the PEP. My mistake, sorry. -- ___ Python tracker <https://bugs.python.org/issue44741> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue44741] Pattern Matching - star subpattern with a subject derived from collections.abc.Sequence
Pierre Quentel added the comment: Oh, I did not invent this class, it is in the test script for pattern matching : https://github.com/python/cpython/blob/6948964ecf94e858448dd28eea634317226d2913/Lib/test/test_patma.py#L1932 With this class, [x, *_, y] matches, but not [x, *w, y] : this is what made me create this issue. Maybe it would be a good idea to change this class in test_patma.py ? OTOH, if the current implementation remains the same, why does the PEP insist on subjects having a len() ? Could sequence patterns match a wider range of subjects that can be unpacked ? -- ___ Python tracker <https://bugs.python.org/issue44741> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue44741] Pattern Matching - star subpattern with a subject derived from collections.abc.Sequence
Pierre Quentel added the comment:
Thanks for the explanations, but I feel unconfortable with the fact that
variable-length sequence patterns are implemented the same as unpacking. (sorry
if this has been discussed before, I can't find references to the discussions
that lead to the current version of the PEP).
There are obvious similarities between
[x, *y, z] = A
and
match A:
case [x, *y, z]:
print('ok')
but a big difference, put forward in PEP 634 : the classes supported for
pattern matching are limited (unpacking a generator expression is possible, but
they are not supported as subjects for sequence patterns), and the PEP
explicitely refers to them having a len().
It seems to me that this implies that the algorithms should be different:
- for unpacking, the iterable must be completely consumed before binding the
names on the left-hand side. If it is infinite, unpacking fails
- for variable-length sequence pattern matching (this is how I understand the
last paragraph about them in PEP 634):
. get the subject length
. iterate one by one before the star pattern
. iterate (len(subject) - number of non-star patterns) times for the star
pattern
. iterate one by one after the star pattern
In the second case, even if the subject never raises StopIteration, the match
succeeds.
Does this make sense ?
--
___
Python tracker
<https://bugs.python.org/issue44741>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue44741] Pattern Matching - star subpattern with a subject derived from collections.abc.Sequence
New submission from Pierre Quentel :
This code
match range(42):
case [x, *w, y]:
z = 0
sets w to a list with 40 items : the length of the subject, minus the number of
non-star subpatterns.
But this code (adapted from test_patma_186) enters an infinite loop
import collections.abc
class Seq(collections.abc.Sequence):
def __getitem__(self, i):
print('get item', i)
return i
def __len__(self):
return 42
match Seq():
case [x, *w, y]:
z = 0
__getitem__ gets called forever, instead of stopping when the expected number
of items in w is reached.
--
components: Interpreter Core
messages: 398226
nosy: quentel
priority: normal
severity: normal
status: open
title: Pattern Matching - star subpattern with a subject derived from
collections.abc.Sequence
type: crash
versions: Python 3.10, Python 3.11
___
Python tracker
<https://bugs.python.org/issue44741>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue44589] Pattern Matching - duplicate keys in mapping patterns
Pierre Quentel added the comment: Sorry, I don't know C so I can't write a PR for this change. -- ___ Python tracker <https://bugs.python.org/issue44589> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue44589] Pattern Matching - duplicate keys in mapping patterns
New submission from Pierre Quentel :
PEP 634 specifies that
"A mapping pattern may not contain duplicate key values. (If all key patterns
are literal patterns this is considered a syntax error; otherwise this is a
runtime error and will raise ValueError.)"
but this is not what happens with the latest release:
Python 3.10.0b3 (tags/v3.10.0b3:865714a, Jun 17 2021, 20:39:25) [MSC v.1929 64
bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> x = {'a': 1}
>>> match x:
... case {'a': 1, 'a': 2}: # (A)
... print('ok')
...
>>> x = {'a': 3}
>>> match x:
... case {'a': 1, 'a': 2}: # (B)
... print('ok')
...
>>> x = {'a': 1, 'b': 2}
>>> match x:
... case {'a': 1, 'a': 2}: # (C)
... print('ok')
...
Traceback (most recent call last):
File "", line 2, in
ValueError: mapping pattern checks duplicate key ('a')
>>>
If I understand the PEP correctly, all these examples should raise a
SyntaxError for the line
case {'a': 1, 'a': 2}:
since all key patterns are literal patterns, and the key 'a' is duplicated.
Cases (A) where the subject matches one of the key-value patterns, and (B) when
it doesn't, fail without raising SyntaxError.
Case (C) where one of the keys in the subject is not present in the mapping
pattern raises a ValueError at runtime instead of SyntaxError.
This behaviour is tested in test_patma.py:
def test_patma_251(self):
x = {"a": 0, "b": 1}
w = y = z = None
with self.assertRaises(ValueError):
match x:
case {"a": y, "a": z}:
w = 0
self.assertIs(w, None)
self.assertIs(y, None)
self.assertIs(z, None)
but this doesn't seem compliant with the specification.
BTW, it's not clear to me why the SyntaxError should be limited to the case
when all keys are literal patterns; it could be raised whenever a literal
pattern is repeated, even when there are value patterns or a double-star
pattern, like in
case {'a': 1, 'a': 2, c.imag, **rest}:
--
components: Interpreter Core
messages: 397195
nosy: quentel
priority: normal
severity: normal
status: open
title: Pattern Matching - duplicate keys in mapping patterns
versions: Python 3.10
___
Python tracker
<https://bugs.python.org/issue44589>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue44085] Remaining invalid rules in simplified grammar
New submission from Pierre Quentel :
In the simplified version of Python grammar at
https://docs.python.org/3.10/reference/grammar.html, most 'invalid_' from
the complete grammar at
https://github.com/python/cpython/blob/3.10/Grammar/python.gram have been
removed, but 2 of them remain :
primary:
| invalid_primary # must be before 'primay genexp' because of
invalid_genexp
dict:
| '{' invalid_double_starred_kvpairs '}'
I suppose that the simplified version is extracted from the complete grammar
with a program, and this program doesn't detect the 'invalid_' that don't
end the line, since these 2 occurrences correspond to the only such lines in
the complete grammar
primary[expr_ty]:
| invalid_primary # must be before 'primay genexp' because of
invalid_genexp
dict[expr_ty]:
| '{' invalid_double_starred_kvpairs '}'
Also note the typo in the comment : 'primay genexp' instead of 'primary genexp'
--
assignee: docs@python
components: Documentation
messages: 393306
nosy: docs@python, quentel
priority: normal
severity: normal
status: open
title: Remaining invalid rules in simplified grammar
versions: Python 3.10
___
Python tracker
<https://bugs.python.org/issue44085>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue38469] PEP 572 : assignment expression to a global variable in a comprehension
Pierre Quentel added the comment: That was a quick fix, thanks ! -- ___ Python tracker <https://bugs.python.org/issue38469> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue38469] PEP 572 : assignment expression to a global variable in a comprehension
New submission from Pierre Quentel : PEP 572 says that "an assignment expression occurring in a (...) comprehension (...) binds the target in the containing scope, honoring a nonlocal or global declaration for the target in that scope, if one exists." In Appendix B, the PEP shows this example : def f(): global TARGET a = [TARGET := EXPR for VAR in ITERABLE] So I don't understand why this fails: Python 3.8.0rc1 (tags/v3.8.0rc1:34214de, Oct 1 2019, 18:42:37) [MSC v.1916 64 bit (AMD64)] on win32 Type "help", "copyright", "credits" or "license" for more information. >>> x = 0 >>> def f(): ... global x ... [x := i for i in range(5)] ... File "", line 3 SyntaxError: no binding for nonlocal 'x' found >>> Is this a bug or am I missing something ? -- components: Interpreter Core messages: 354601 nosy: quentel priority: normal severity: normal status: open title: PEP 572 : assignment expression to a global variable in a comprehension type: behavior versions: Python 3.8 ___ Python tracker <https://bugs.python.org/issue38469> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue20504] cgi.FieldStorage, multipart, missing Content-Length
Pierre Quentel added the comment: Now that the PR has been merged, can someone close the issue ? -- ___ Python tracker <https://bugs.python.org/issue20504> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue21705] cgi.py: Multipart with more than one file is misparsed
Pierre Quentel added the comment: @ethan.furman Yes, in test_cgi.py, the method test_fieldstorage_multipart_w3c https://github.com/python/cpython/blob/master/Lib/test/test_cgi.py#L316) uses a multipart content with 2 files in it (https://github.com/python/cpython/blob/master/Lib/test/test_cgi.py#L579) -- ___ Python tracker <https://bugs.python.org/issue21705> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue21705] cgi.py: Multipart with more than one file is misparsed
Pierre Quentel added the comment: The patch has been applied some time ago (I couldn't find the exact commit), cf. https://github.com/python/cpython/blob/master/Lib/cgi.py#L750 I think we can close the issue. -- nosy: +quentel ___ Python tracker <https://bugs.python.org/issue21705> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue27777] cgi.FieldStorage can't parse simple body with Content-Length and no Content-Disposition
Pierre Quentel added the comment: I have submitted another Pull Request (10771) that seems to fix the bug while passing all the tests in test_cgi.py -- nosy: +quentel ___ Python tracker <https://bugs.python.org/issue2> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue27777] cgi.FieldStorage can't parse simple body with Content-Length and no Content-Disposition
Change by Pierre Quentel : -- pull_requests: +10015 ___ Python tracker <https://bugs.python.org/issue2> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue20504] cgi.FieldStorage, multipart, missing Content-Length
Pierre Quentel added the comment: I have submitted PR #10638 to fix this issue. -- ___ Python tracker <https://bugs.python.org/issue20504> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue20504] cgi.FieldStorage, multipart, missing Content-Length
Change by Pierre Quentel : -- pull_requests: +9885 ___ Python tracker <https://bugs.python.org/issue20504> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue20504] cgi.FieldStorage, multipart, missing Content-Length
Change by Pierre Quentel : -- pull_requests: +9881 ___ Python tracker <https://bugs.python.org/issue20504> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue20504] cgi.FieldStorage, multipart, missing Content-Length
Change by Pierre Quentel : -- nosy: +quentel ___ Python tracker <https://bugs.python.org/issue20504> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue10486] http.server doesn't set all CGI environment variables
Pierre Quentel added the comment:
The QUERY_STRING value is always set by the code at lines 1135-1137 of
http.server:
for k in ('QUERY_STRING', 'REMOTE_HOST', 'CONTENT_LENGTH',
'HTTP_USER_AGENT', 'HTTP_COOKIE', 'HTTP_REFERER'):
env.setdefault(k, "")
The RFC for CGI has not evolved since 2004, probably because the technology is
stable, and also because other, more efficient protocols have been defined to
avoid the "CGI overhead" (FastCGI for instance).
I think that http.server should only implement the "meta-variables" defined in
RFC 3875:
- AUTH_TYPE CONTENT_LENGTH CONTENT_TYPE GATEWAY_INTERFACE PATH_INFO
PATH_TRANSLATED QUERY_STRING REMOTE_ADDR REMOTE_HOST REMOTE_IDENT REMOTE_USER
REQUEST_METHOD SCRIPT_NAME SERVER_NAME SERVER_PORT SERVER_PROTOCOL
SERVER_SOFTWARE. Some of these must always be set (eg QUERY_STRING,
REQUEST_METHOD, SERVER_NAME...) but for other ones, there are conditions (for
instance for CONTENT_LENGTH: "The server MUST set this meta-variable if and
only if the request is accompanied by a message-body entity")
- "protocol-specific meta variables" : for HTTP, variables determined by the
HTTP request headers such as HTTP_COOKIE (cf section 4.1.18. Protocol-Specific
Meta-Variables)
Other meta variables are probably beyond the scope of a module in the standard
library.
In short, in my opinion the issue can be closed.
--
___
Python tracker
<https://bugs.python.org/issue10486>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue10486] http.server doesn't set all CGI environment variables
Change by Pierre Quentel : -- nosy: +quentel ___ Python tracker <https://bugs.python.org/issue10486> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue30576] http.server should support HTTP compression (gzip)
Pierre Quentel added the comment: I have released the module as httpcompressionserver on PyPI : https://pypi.org/project/httpcompressionserver/ -- ___ Python tracker <https://bugs.python.org/issue30576> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue30576] http.server should support HTTP compression (gzip)
Pierre Quentel added the comment: Brett, Thanks for taking the time, you and other core devs, to review the PR and to explain why you took this decision. I am disappointed by the result, but I understand the reasons, having to face the same maintenance issues, on a much smaller scale, with the project I manage (Brython, an implementation of Python for the browser). I will follow your advice to publish the server on PyPI and cross fingers that it gains enough support to be included in a future version. -- ___ Python tracker <https://bugs.python.org/issue30576> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue30576] http.server should support HTTP compression (gzip)
Pierre Quentel <pierre.quen...@gmail.com> added the comment: On the Github site Raymond Hettinger is mentioned as the reviewer. I don't know how to contact him, can anyone ask him if he can review the PR on time for inclusion in Python 3.7 ? I understand that it's difficult to find time for this and that there are many pending PRs ; I can't judge if this one is more or less important than the others, but the issue has been discussed for a few months now, the code has been improved after many iterations, so hopefully it should not be too far from approval. -- ___ Python tracker <rep...@bugs.python.org> <https://bugs.python.org/issue30576> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue30576] http.server should support HTTP compression (gzip)
Pierre Quentel <pierre.quen...@gmail.com> added the comment: I think I have made all the changes requested in the code review (many thanks to the reviewer !). I see that the Pull Request has been flagged "awaiting core review". With the deadline for Python 3.7 coming soon (11 weeks), is there a chance that the feature will be added on time ? Is there something I should do or someone I should contact for that ? -- ___ Python tracker <rep...@bugs.python.org> <https://bugs.python.org/issue30576> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue30406] async and await should be keywords in 3.7
Pierre Quentel <pierre.quen...@gmail.com> added the comment: According to PEP 492, async and await should have been deprecated in 3.5 and 3.6, but I don't think they have been : await = 1 def f(async=True): ... don't raise any deprecation warning in 3.6. Since version 3.7 will break existing code with a SyntaxError, could it be possible to have something explicit in the SyntaxError message for these new keywords, for instance SyntaxError : 'async' is a keyword I don't speack C so I can't provide a patch, sorry. -- nosy: +quentel ___ Python tracker <rep...@bugs.python.org> <https://bugs.python.org/issue30406> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue30576] http.server should support HTTP compression (gzip)
Pierre Quentel added the comment: On Python-ideas someone asked if other compressions could be supported besides gzip. The latest version of the PR adds a mechanism for that : SimpleHTTPRequestHandler has a new attribute "compressions", a dictionary that maps compression encodings (eg "gzip") to a "compressed data generator". The generator takes a file object as argument, yields non-empty chunks of compressed data and ends by yielding b'' for compliance with Chunked Transfer Encoding protocol. To support other compression algorithms, "compressions" can be extended with another key (eg "brotli") mapped to the appropriate generator. A test has been added with the non-standard "bzip2" encoding, using the bz2 module in the standard distribution. I also added support for "deflate" by default (it's very close to "gzip"). -- ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue30576> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue30576] http.server should support HTTP compression (gzip)
Pierre Quentel added the comment: In the latest version of the PR, following Martin's comments : - apply Chunk Transfer for HTTP/1.1 only, change implementation of compression for previous protocols (send gzipped data without Content-Length) - use http.cookiejar to parse the Accept-Encoding header - fix a bug with chunk length (conversion to hex) - support x-gzip besides gzip - handle Python builds without zlib / gzip Headers parsing is done in several places in the standard distribution. It should probably be done in a single module, but I think it would be better to open a new issue for that, as it would impact more modules than just http.server. I couldn't find a simple way to reuse code from http.client to generate HTTP chunks (it's in HTTPConnection._send_output()), but I'm not sure it's worth it, the code to generate a chunk is a one-liner. -- ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue30576> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue29654] SimpleHTTPRequestHandler should support browser cache
Pierre Quentel added the comment: Thanks for telling me. I must have run the test hundreds of times now, on a Windows 7 PC, and this bug never occured. Just for my information, why do you add temp.flush() in the "with" block ? I thought the context manager took care of this. 2017-07-28 18:19 GMT+02:00 STINNER Victor <rep...@bugs.python.org>: > > STINNER Victor added the comment: > > FYI there was a race condition, impacting Windows. I just fixed it: > bpo-31066, commit 28ce07ae9e34c70eea6b52515c7e1cefd41e. > > -- > nosy: +haypo > > ___ > Python tracker <rep...@bugs.python.org> > <http://bugs.python.org/issue29654> > ___ > -- ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue29654> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue30576] http.server should support HTTP compression (gzip)
Pierre Quentel added the comment: @martin.panter Please forget my previous message. There is a 3rd solution, and you gave it : no Content-Length and close the connection when all (compressed) data has been sent. -- ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue30576> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue30576] http.server should support HTTP compression (gzip)
Pierre Quentel added the comment: @martin.panter For HTTP/1.0, since chunked transfer is not supported, and storage in a temporary file is also not an option, I see 2 possible solutions : - give up compressing big files - it would be a pity, compression is actually made for them... - compress the file 2 times : a first time just to compute the content length, without storing or sending anything, and a second time to send the gzipped data after all headers have been sent If there is a 3rd solution I'd be glad to know ; otherwise I prefer the second one, in spite of the waste of CPU. -- ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue30576> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue30576] http.server should support HTTP compression (gzip)
Pierre Quentel added the comment: In the latest version of the Pull Request (https://github.com/python/cpython/pull/2078/commits/6466c93555bec521860c57e832b691fe7f0c6c20) : - compression is disabled by default (compressed_types is set to []) - as suggested by Chris Barker in the discussion on python-ideas, I added a list of commonly compressed types that can be used as the value of compressed_types in subclasses of SimpleHTTPRequestHandler. For want of official sources I took the list from https://github.com/h5bp/server-configs-apache. - a command-line option --gzip has been introduced, when it is set, the list of commonly compressed types is used - the implementation of compression for "big files" has been changed : as suggested by Victor and Christian, no more temporary file, compressed data is now sent as chunks using Chunked Transfer Encoding. The file object returned by send_head() is a generator that produces the chunks of compressed data -- ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue30576> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue29512] regrtest refleak: implement bisection feature
Pierre Quentel added the comment: Maybe it's me who is doing things wrong, but when I run Lib/test/test_httpservers.py I get this strange error : Traceback (most recent call last): File "C:\cpython\Lib\test\test_httpservers.py", line 7, in from http.server import BaseHTTPRequestHandler, HTTPServer, \ File "c:\python36\lib\http\server.py", line 92, in import email.utils File "c:\python36\lib\email\utils.py", line 28, in import random File "c:\python36\lib\random.py", line 48, in import bisect as _bisect File "C:\cpython\Lib\test\bisect.py", line 27, in import tempfile File "c:\python36\lib\tempfile.py", line 45, in from random import Random as _Random ImportError: cannot import name 'Random' I see that the bisect module is loaded from location /test/bisect.py because of the script introduced in the resolution of this issue. Is it intentional to give it the same name as a module of the standard library ? -- nosy: +quentel ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue29512> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue30576] http.server should support HTTP compression (gzip)
Pierre Quentel added the comment: Thank you Terry and Victor for your comments. I understand that you agree on adding HTTP compression to http.server, but don't want it to be enabled by default. @terry.reedy With the implementation proposed in the Pull Request, to disable compression by default, all it takes is to set the attribute SimpleHTTPRequestHandler.compressed_types to the empty list. Users who want to enable compression for some types would create subclasses of Simple... that set compressed_types. For instance : import http.server class CompressedHandler(http.server.SimpleHTTPRequestHandler): compressed_types = ["text/html", "text/plain"] http.server.test(HandlerClass=CompressedHandler) Would that be ok ? For a command line argument --gzip, compressed_types could be set to a list of commonly compressed types - I don't think we want to pass a list of types on the command line. For CGI scripts, I may be missing something but for me it is up to the script to determine the response headers and content, I don't think a mixin class in http.server could help. @haypo I fully agree on your comment about content-length, the implementation in the PR is not optimal. The alternative would be to send the answer using chunked transfer encoding, which is currently not used in http.server, and would also be relevant for other use cases than gzipped files. It shouldn't be difficult to add, and I volunteer to propose an implementation, but it's yet another feature to add to the server. Do you think it's relevant ? -- ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue30576> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue30576] http.server should support HTTP compression (gzip)
Pierre Quentel added the comment: Is Python-ideas the appropriate place to get input from other core devs ? -- ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue30576> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue30824] Add mimetype for extension .json
New submission from Pierre Quentel: I propose to add a mapping of file extension .json to mime type "application/json". This is registered in https://www.iana.org/assignments/media-types -- components: Library (Lib) messages: 297494 nosy: quentel, r.david.murray priority: normal severity: normal status: open title: Add mimetype for extension .json type: enhancement versions: Python 3.7 ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue30824> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue30576] http.server should support HTTP compression (gzip)
Pierre Quentel added the comment:
Thanks for the comments. I agree with some of them, and have improved the PR
accordingly, but I don't agree on the opinion that HTTP compression is beyond
the scope of http.server : like browser cache (which also implies a negociation
between client and server) it's a basic, normalized feature of HTTP servers ;
it doesn't change the intention of "mapping requests to the directory
structure", it just changes the way some files are served, improving transfer
speed and network load, which is especially useful on mobile networks.
The implementation makes HTTP compression the default for the types defined in
SimpleHTTPRequestHandler.compressed_types, but it can be disabled if the system
can't handle it, like in the case you mention : just by setting
compressed_types to the empty list. I have made it more clear in the
documentation.
I understand the concern for HEAD requests, but they must return the same
headers as GET, so the compression must be done to determine the content-length
header.
To address the case when the tmp zone is limited, I have also added a
try/except block that checks if the temporary gzipped file can be created ; if
not, the file is returned uncompressed.
I don't understand Martin's suggestion to use HTTP compression for .gz files :
they are already compressed, there wouldn't be any benefit to compress them
again. It's the same for audio or video files. Compression is only useful for
uncompressed data such as text files, scripts, css files, etc. that are not
already compressed. All those files are sent unmodified and with their own
content type.
I have also improved the PR for the alternative forms of Accept-Encoding
("gzip;q=0", "*", "GZIP", etc.).
--
___
Python tracker <rep...@bugs.python.org>
<http://bugs.python.org/issue30576>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue30576] http.server should support HTTP compression (gzip)
Changes by Pierre Quentel <pierre.quen...@gmail.com>: -- pull_requests: +2142 ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue30576> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue30576] http.server should support HTTP compression (gzip)
Pierre Quentel added the comment: The compression is done on the fly : if compression criteria are satisfied, the original content is gzipped, either in memory or on a temporary file on disk, depending on the file size. The gzipped content is not cached, but since the server now supports browser cache, on the next requests for the same file a 304 response will be sent. -- ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue30576> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue30576] http.server should support HTTP compression (gzip)
Pierre Quentel added the comment: I propose this as a minor improvement to the built-in server, like the support of browser cache that will be included in Python 3.7 (issue #29654, PR #298). I understand that the server is not supposed to be full-featured, but HTTP compression is widespread, reduces network load and is easy to implement (the code will be very similar to SimpleXMLRPCRequestHandler). Content-Encoding is used because it's the most simple to implement. Chunked transfer for large amount of data seems to me to be out of the scope of the built-in server. Content-Length is set to the length of the compressed data. I don't understand your question about persistent connections : the proposal covers a single request / response sequence, it doesn't depend on the underlying TCP connection being reused or not. >From what I understand, issue #1508475 refers to the http client, not server ; >and #4733 refers to another meaning of encoding (conversion from characters to >bytes with a charset such as utf-8), not to HTTP compression, which >unfortunately also uses "encoding" in headers names. -- ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue30576> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue30576] http.server should support HTTP compression (gzip)
Changes by Pierre Quentel <pierre.quen...@gmail.com>: -- type: -> enhancement ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue30576> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue30576] http.server should support HTTP compression (gzip)
New submission from Pierre Quentel: The server in http.server currently doesn't support HTTP compression. I propose to implement it in the method send_head() of SimpleHTTPRequestHandler this way : for each GET request, if the request header "Accept-Encoding" is present and includes "gzip" among the possible compression schemes, and if the Content-Type determined by the file extension is in a list compressed_types, then the server sends the "Content-Encoding" response header to "gzip" and send_head() returns a file object with the gzipped value. compressed_types is an attribute of the SimpleHTTPRequestHandler class and is set by default to ["text/plain", "text/html", "text/css", "text/xml", "text/javascript", "application/javascript", "application/json"]. The implementation is very simple (a few lines of code). I also propose to modify mimetypes to add the mapping of extension ".json" to "application/json". I will make a Pull Request on the CPython Github site with these changes. -- components: Library (Lib) messages: 295207 nosy: quentel priority: normal severity: normal status: open title: http.server should support HTTP compression (gzip) versions: Python 3.7 ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue30576> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue29979] cgi.parse_multipart is not consistent with FieldStorage
Pierre Quentel added the comment: Senthil, Can you take a look at the Pull Request when you have time ? The correct PR is #991, not #990. -- ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue29979> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue29979] cgi.parse_multipart is not consistent with FieldStorage
Changes by Pierre Quentel <pierre.quen...@gmail.com>: -- pull_requests: +1163 ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue29979> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue29979] cgi.parse_multipart is not consistent with FieldStorage
Changes by Pierre Quentel <pierre.quen...@gmail.com>: -- pull_requests: +1162 ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue29979> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue29979] cgi.parse_multipart is not consistent with FieldStorage
New submission from Pierre Quentel: In the cgi module, the parse_multipart() function duplicates code from FieldStorage, and the result is not compliant with that of FieldStorage for requests sent with multipart/form-data : for non-file fields, the value associated with a key is a list of *bytes* in parse_multipart() and a list of *strings* for FieldStorage (the bytes decoded with the argument "encoding" passed to FieldStorage()). I will propose a PR on the Github repo with a version of parse_multipart that uses FieldStorage and returns the same result (values as strings). The function will take an additional argument "encoding". -- components: Library (Lib) messages: 291117 nosy: quentel priority: normal severity: normal status: open title: cgi.parse_multipart is not consistent with FieldStorage type: behavior versions: Python 3.7 ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue29979> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue11066] cgi.py proposals : sys.stdout encoding + rewriting of parsing functions
Pierre Quentel added the comment: I close this issue and will open a more specific one for the rewriting of parse_multipart() -- resolution: -> out of date stage: -> resolved status: open -> closed ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue11066> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue29654] SimpleHTTPRequestHandler should support browser cache
New submission from Pierre Quentel: SimpleHTTPServer send a Last-Modified response header, but doesn't take into account the If-Modified-Since header if it was sent by the user agent. If a url matches a file and this file was not modified after the value of the If-Modified-Since header, the server should return HTTP status 304 (Not Modified). -- components: Library (Lib) messages: 288581 nosy: quentel priority: normal severity: normal status: open title: SimpleHTTPRequestHandler should support browser cache type: enhancement versions: Python 3.7 ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue29654> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue24764] cgi.FieldStorage can't parse multipart part headers with Content-Length and no filename in Content-Disposition
Pierre Quentel added the comment: Victor, you can apply the patch and close the issue. Le 7 août 2015 17:12, Peter Landry rep...@bugs.python.org a écrit : Peter Landry added the comment: A new patch that simply removes Content-Length from part headers when present. -- Added file: http://bugs.python.org/file40145/cgi_multipart.patch ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue24764 ___ -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue24764 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue24764] cgi.FieldStorage can't parse multipart part headers with Content-Length and no filename in Content-Disposition
Pierre Quentel added the comment: I don't really see why there is a Content-Length in the headers of a multipart form data. The specification at http://www.w3.org/TR/html401/interact/forms.html#h-17.13.4.2 doesn't mention it, and it is absent in the example that looks like the one tested by Peter : Content-Type: multipart/form-data; boundary=AaB03x --AaB03x Content-Disposition: form-data; name=submit-name Larry --AaB03x Content-Disposition: form-data; name=files; filename=file1.txt Content-Type: text/plain ... contents of file1.txt ... --AaB03x-- In case a user agent would insert it, I think the best would be to ignore it. That is, inside read_multi(), after headers = parser.close() add these lines : if 'content-length' in headers: del headers['content-length'] It's hard to see the potential side effects but I think it's cleaner than the proposed patch, which is not correct anyway for another reason : the attribute value is set to a bytes objects, instead of a string. Peter, does this make sense ? If so, can you submit another patch ? -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue24764 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue24764] cgi.FieldStorage can't parse multipart part headers with Content-Length and no filename in Content-Disposition
Pierre Quentel added the comment: Yes, I will be able to review the patch next week 2015-07-31 18:13 GMT+02:00 STINNER Victor rep...@bugs.python.org: STINNER Victor added the comment: @Pierre Quentel: Hi! Are you still working on CGI? Can you please review this patch? Thanks. -- Previous large change in the cgi module: issue #4953. Pierre helped a lot on this one. -- nosy: +quentel ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue24764 ___ -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue24764 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue11352] Update cgi module doc
Pierre Quentel pierre.quen...@gmail.com added the comment: I attach a new version after sharing thought with Glenn CGI scripts are still unable to define which encoding to use to print the strings received from the user agent. A patch was proposed #11066 but the issue is still pending. The new version documents this issue -- Added file: http://bugs.python.org/file25519/cgi-doc.patch ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue11352 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue11352] Update cgi module doc
Pierre Quentel pierre.quen...@gmail.com added the comment: Hi, I started working on a revised version of the whole cgi documentation. I mostly changed paragraphs 2 3 (Using the CGI module and Higher level interface) and replaced them by a paragraph still called Using the CGI module + 2 other paragraphs for special cases : Multiple fields with the same name and File uploads The content is basically the same but the new presentation is hopefully more clear The patch is attached as file cgi-doc.patch -- Added file: http://bugs.python.org/file25512/cgi-doc.patch ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue11352 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue14565] is_cgi doesn't function as documented for cgi_directories
Pierre Quentel pierre.quen...@gmail.com added the comment: Hi Glenn, My proposal was not about optimization, I just thought that if x==y is simpler than if len(x)==len(y) and x==y. Since we don't expect that there will be many directories in the list, I don't think optimizing is so important. But it doesn't matter to me really, the most important is to have the bug fixed Can you propose a diff file so that the committers can review it ? -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue14565 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue14565] is_cgi doesn't function as documented for cgi_directories
Pierre Quentel pierre.quen...@gmail.com added the comment: Thanks for the explanation I still think that the patch can be simplified, not using path lengths and the found flag collapsed_path = _url_collapse_path(self.path) for head in self.cgi_directories: if head==collapsed_path: self.cgi_info = (head,'') return True elif collapsed_path.startswith(head) \ and collapsed_path[len(head)]=='/': self.cgi_info = head, collapsed_path[len(head)+1:] return True return False BTW the last return False is rather useless since is_cgi() is only used in tests like if is_cgi() -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue14565 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue14565] is_cgi doesn't function as documented for cgi_directories
Pierre Quentel pierre.quen...@gmail.com added the comment:
Hi Glenn, good to hear from you ;-)
I think the fix can be simplified replacing
dir_sep = collapsed_path.find('/', 1)
by
dir_sep = collapsed_path.rfind('/', 1)
--
nosy: +quentel
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue14565
___
___
Python-bugs-list mailing list
Unsubscribe:
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue8077] cgi handling of POSTed files is broken
Pierre Quentel pierre.quen...@gmail.com added the comment: There are 2 different problems : - handling of data by cgi.FieldStorage (issue 4953) : fixed since version 3.2 - in http.server.CGIHTTPRequestHandler, for POST requests on Windows, before opening the subprocess in run_cgi() all data is read by a *single* call to self.rfile.read(). If not all bytes are read by this single call, which is the case for a large file upload, only the read data are processed The attached patch modifies http.server : - if all data are read in the first call to self.rfile.read() (that is, if its length is equal to the Content-length header), process them - if not, store all data in a temporary file (not in memory) and set the stdin argument of subprocess.Popen to this temporary file With this patch, the tests provided by Mitchell all work on my PC (Windows XP Pro SP3) -- keywords: +patch Added file: http://bugs.python.org/file25440/http-server.patch ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue8077 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue8077] cgi handling of POSTed files is broken
Changes by Pierre Quentel pierre.quen...@gmail.com: -- nosy: +quentel ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue8077 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue11352] Update cgi module doc
Pierre Quentel pierre.quen...@gmail.com added the comment: Thanks Hynek for raising this issue from the dead Patch proposal attached. Sorry if there are markup errors, it's my first contact with rst -- Added file: http://bugs.python.org/file25416/cgi.rst ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue11352 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue11352] Update cgi module doc
Pierre Quentel pierre.quen...@gmail.com added the comment: Sorry about that. I didn't dare to say I was also a Mercurial newbie -- Added file: http://bugs.python.org/file25421/cgi-doc-update.patch ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue11352 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue11352] Update cgi module doc
Changes by Pierre Quentel pierre.quen...@gmail.com: Removed file: http://bugs.python.org/file25416/cgi.rst ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue11352 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue11352] Update cgi module doc
Pierre Quentel pierre.quen...@gmail.com added the comment: Thanks Senthil I spot a typo in the first modified paragraph : cet instead of set -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue11352 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue12922] StringIO and seek()
Changes by Pierre Quentel pierre.quen...@gmail.com: -- nosy: +quentel ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue12922 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue12411] cgi.parse_multipart is broken on 3.x
Pierre Quentel pierre.quen...@gmail.com added the comment: When the FieldStorage class was fixed there was a discussion in issue 4953 about the module-level functions parse() and parse_multipart(). The code was very similar to methods of the FieldStorage class so the idea was to use FieldStorage inside the functions The patch proposed in issue 11066 replaced the code in parse_multipart by just : def parse_multipart(fp, pdict): return FieldStorage(fp,environ=pdict) Did anyone test it ? -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue12411 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue11352] Buf in cgi module doc
New submission from Pierre Quentel pierre.quen...@gmail.com: Hi, I wrote a patch for the cgi module in version 3.2rc1 (#4953). Small changes should be done to the documentation of this module to reflect the changes in the module API : - in section 20.2.2. Using the cgi module original text : If a field represents an uploaded file, accessing the value via the value attribute or the getvalue() method reads the entire file in memory as a string. This may not be what you want. You can test for an uploaded file by testing either the filename attribute or the file attribute. You can then read the data at leisure from the file attribute: proposed new text (for files, value is bytes, not string, and the read() method on file also returns bytes) : If a field represents an uploaded file, accessing the value via the value attribute or the getvalue() method reads the entire file in memory as bytes. This may not be what you want. You can test for an uploaded file by testing either the filename attribute or the file attribute. You can then read the data at leisure from the file attribute (the read() and readline() methods will return bytes) : - version 3.2 introduced a parameter encoding for the FieldStorage constructor, used to decode the bytes received on the HTTP connection for fields other than files. This encoding must the one defined in the HTML document holding the form submitted to the CGI script ; it is usually defined by a meta tag : meta http-equiv=Content-type content=text/html;charset=latin-1 or by the Content-Type header for this document I'm not sure where this should be mentioned in the module documentation. Maybe in 20.2.9. Common problems and solutions for the moment. But there are plans (#11066) to introduce another interface to change the encoding of sys.stdout in the CGI script itself, so another option would be to open a specific section about encodings Hope it's clear enough Pierre -- assignee: docs@python components: Documentation messages: 129691 nosy: docs@python, quentel priority: normal severity: normal status: open title: Buf in cgi module doc type: behavior versions: Python 3.2 ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue11352 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue11352] Bug in cgi module doc
Pierre Quentel pierre.quen...@gmail.com added the comment: bug, not buf... -- title: Buf in cgi module doc - Bug in cgi module doc ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue11352 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue10911] cgi: add more tests
Pierre Quentel pierre.quen...@gmail.com added the comment: I opened issue #11066 for the code refactoring -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue10911 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue10911] cgi: add more tests
Pierre Quentel pierre.quen...@gmail.com added the comment: Here is the diff file for test_cgi.py I added a test for a multipart/form-data form with non ASCII data to test the encoding parameter of FieldStorage -- keywords: +patch Added file: http://bugs.python.org/file20611/test_cgi.diff ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue10911 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue11066] cgi.py proposals : sys.stdout encoding + rewriting of parsing functions
New submission from Pierre Quentel pierre.quen...@gmail.com: Python 3.2rc1 introduced a new version of cgi.py that handles correctly file uploads In this version, the FieldStorage constructor receives an argument encoding which is the encoding used by the document holding the submitted form On the CGI script side, there is currently no easy way to print the received form fields with another encoding than sys.stdout.encoding. The proposed version introduces a function, set_stdout_encoding(charset), which can be used in the CGI script to set sys.stdout to an instance of a class that uses the charset encoding. This way, print() will use this encoding. charset must be the encoding defined in the content-type header sent by the CGI scrpt This class (IOMix) was written by Glen Linderman and proposed in the 3.2rc1 version, but no consensus could be reached on time for the release Another proposed change is a rewriting of the module-level functions parse() and parse_multipart() : they now use the FieldStorage methods instead -- components: Library (Lib) files: cgi_20110129.diff keywords: patch messages: 127487 nosy: quentel priority: normal severity: normal status: open title: cgi.py proposals : sys.stdout encoding + rewriting of parsing functions type: behavior versions: Python 3.3 Added file: http://bugs.python.org/file20612/cgi_20110129.diff ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue11066 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue10911] cgi: add more tests
Pierre Quentel pierre.quen...@gmail.com added the comment: Hi, I have written more tests, but also propose changes to cgi.py : - rewrite the parse_qs() and parse_multipart() functions so that they use FieldStorage methods instead of duplicating them - add a function set_stdout_encoding(encoding), using the IOMix class proposed by Glen Linderman in issue #4953 Should I post the new version of test_cgi.py here and open another issue for the other proposed changes to cgi.py, or use this issue to discuss both ? -- nosy: +quentel ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue10911 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue4953] cgi module cannot handle POST with multipart/form-data in 3.x
Pierre Quentel pierre.quen...@gmail.com added the comment: @Victor Thanks for the comments - I don't understand why FieldStorage changes sys.stdout and sys.stderr (see remarks about IOMix above): please remove the charset argument (it is also confusing to have two encoding arguments). it should be done somewhere else done please remove the O_BINARY hack: the patch is for Python 3.2 and I closed issue #10841. If you would like a backport, another patch should be written later done encoding = 'latin-1' # ?: write a real comment or remove it I removed this part 'self.fp.read(...) # bytes': you should add a test on the type if you are not sure that fp.read() gives bytes added tests in read_urlencoded(), read_multi() and read_binary() file: the file(-like) object from which you can read the data *as bytes*: you should mention that TextIOWrapper is also tolerated (accepted?) not done : here file is not the argument passed to the FieldStorage constructor, but the attribute of values returned from calls to FieldStorage. In the new implementation, its read() method always returns bytes you may set fp directly to sys.stdin.buffer (instead of sys.stdin) if fp is None (it will be easier after removing the O_BINARY thing) done the patch adds a tab in an empty line, please don't do that :-) done (hopefully :-) you should add a (private?) attribute to FieldStorage to decide if it works on bytes or unicode, instead of using self.filename is not None test (eg. self._use_bytes = (self.filename is not None) done i don't like the idea of having a generic self.__write() method supporting bytes and unicode. it would prefer two methods, eg. self.__write_text() and self.__write_binary() (they can share a third private method) not done, the argument of __write is always bytes i don't like stream_encoding name: what is the stream here? do you process a file, a string or a stream? why not just self.encoding? done - import email.parser,email.feedparser one import is useless here. I prefer from email.feedparser import FeedParser because you get directly a ImportError if the symbol is missing. And it's already faster to get FeedParser instead of email.feedparser.FeedParser in a loop (dummy micro-optimization) done even I like the following change, please do it in a separated patch: -if type(value) is type([]): +if isinstance(value,list): not done I really don't like the IOMix thing: removed I vote +0 to commit the patch now (if the release manager agrees), and +1 if all of my remarks are fixed. should be close to +0.8 now ;-) -- Added file: http://bugs.python.org/file20402/cgi_32.patch ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue4953 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue4953] cgi module cannot handle POST with multipart/form-data in 3.x
Changes by Pierre Quentel pierre.quen...@gmail.com: Removed file: http://bugs.python.org/file20387/cgi_32.patch ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue4953 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue4953] cgi module cannot handle POST with multipart/form-data in 3.x
Pierre Quentel pierre.quen...@gmail.com added the comment: Glenn, you read my mind ;-) Thanks for mentioning the O_BINARY thing. New (last !) patch attached -- Added file: http://bugs.python.org/file20403/cgi_32.patch ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue4953 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue4953] cgi module cannot handle POST with multipart/form-data in 3.x
Changes by Pierre Quentel pierre.quen...@gmail.com: Removed file: http://bugs.python.org/file20402/cgi_32.patch ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue4953 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue4953] cgi module cannot handle POST with multipart/form-data in 3.x
Pierre Quentel pierre.quen...@gmail.com added the comment: Thanks a lot Victor ! I wrote the patch : Pierre Quentel (pierre.quen...@gmail.com) with many inputs by Glenn Linderman 2011/1/14 STINNER Victor rep...@bugs.python.org STINNER Victor victor.stin...@haypocalc.com added the comment: Oh, I forgot to credit the author(s): who wrote the patch? -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue4953 ___ -- title: cgi module cannot handle POST with multipart/form-data in3.x - cgi module cannot handle POST with multipart/form-data in 3.x Added file: http://bugs.python.org/file20405/unnamed ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue4953 ___Thanks a lot Victor !brbrI wrote the patch : Pierre Quentel (a href=mailto:pierre.quen...@gmail.com;pierre.quen...@gmail.com/a) with many inputs by Glenn Lindermanbrbrdiv class=gmail_quote2011/1/14 STINNER Victor span dir=ltrlt;a href=mailto:rep...@bugs.python.org;rep...@bugs.python.org/agt;/spanbr blockquote class=gmail_quote style=margin: 0pt 0pt 0pt 0.8ex; border-left: 1px solid rgb(204, 204, 204); padding-left: 1ex;div class=imbr STINNER Victor lt;a href=mailto:victor.stin...@haypocalc.com;victor.stin...@haypocalc.com/agt; added the comment:br br /divOh, I forgot to credit the author(s): who wrote the patch?br br --br divdiv/divdiv class=h5br ___br Python tracker lt;a href=mailto:rep...@bugs.python.org;rep...@bugs.python.org/agt;br lt;a href=http://bugs.python.org/issue4953; target=_blankhttp://bugs.python.org/issue4953/agt;br ___br /div/div/blockquote/divbrdiv class=ogc-tooltip-class id=ogc_tooltip_id/div ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue4953] cgi module cannot handle POST with multipart/form-data in 3.x
Changes by Pierre Quentel pierre.quen...@gmail.com: Removed file: http://bugs.python.org/file20405/unnamed ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue4953 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue4953] cgi module cannot handle POST with multipart/form-data in 3.x
Pierre Quentel pierre.quen...@gmail.com added the comment: My latest patch for test_cgi is in cgi_32.patch I will try to add more tests later -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue4953 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue4953] cgi module cannot handle POST with multipart/form-data in 3.0
Pierre Quentel pierre.quen...@gmail.com added the comment: I knew the builtins hack was terrible, thanks for the replies... I changed cgi.py with Glenn's IOMix class, and included the changes in make_file(). The patch is attached to this message Is it really too late to include it in 3.2 ? Missing a working cgi module is really a problem for a wider 3.x adoption -- Added file: http://bugs.python.org/file20383/cgi_20110113.diff ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue4953 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue4953] cgi module cannot handle POST with multipart/form-data in 3.0
Pierre Quentel pierre.quen...@gmail.com added the comment: diff for the updated version of test_cgi.py, compatible with cgi.py -- Added file: http://bugs.python.org/file20384/test_cgi_20111013.diff ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue4953 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue4953] cgi module cannot handle POST with multipart/form-data in 3.0
Pierre Quentel pierre.quen...@gmail.com added the comment: zip file with the updated cgi_test.py and associated files -- Added file: http://bugs.python.org/file20385/cgi_tests.zip ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue4953 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue4953] cgi module cannot handle POST with multipart/form-data in 3.x
Changes by Pierre Quentel pierre.quen...@gmail.com: Removed file: http://bugs.python.org/file20229/cgi_diff.txt ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue4953 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue4953] cgi module cannot handle POST with multipart/form-data in 3.x
Changes by Pierre Quentel pierre.quen...@gmail.com: Removed file: http://bugs.python.org/file20235/cgi_diff.txt ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue4953 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue4953] cgi module cannot handle POST with multipart/form-data in 3.x
Changes by Pierre Quentel pierre.quen...@gmail.com: Removed file: http://bugs.python.org/file20322/cgi_diff_20110109.txt ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue4953 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue4953] cgi module cannot handle POST with multipart/form-data in 3.x
Changes by Pierre Quentel pierre.quen...@gmail.com: Removed file: http://bugs.python.org/file20323/cgi_tests.zip ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue4953 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue4953] cgi module cannot handle POST with multipart/form-data in 3.x
Changes by Pierre Quentel pierre.quen...@gmail.com: Removed file: http://bugs.python.org/file20356/cgi_diff_20110111.txt ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue4953 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue4953] cgi module cannot handle POST with multipart/form-data in 3.x
Changes by Pierre Quentel pierre.quen...@gmail.com: Removed file: http://bugs.python.org/file20382/cgi_diff_20110112.txt ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue4953 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue4953] cgi module cannot handle POST with multipart/form-data in 3.x
Changes by Pierre Quentel pierre.quen...@gmail.com: Removed file: http://bugs.python.org/file20383/cgi_20110113.diff ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue4953 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue4953] cgi module cannot handle POST with multipart/form-data in 3.x
Changes by Pierre Quentel pierre.quen...@gmail.com: Removed file: http://bugs.python.org/file20384/test_cgi_20111013.diff ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue4953 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue4953] cgi module cannot handle POST with multipart/form-data in 3.x
Changes by Pierre Quentel pierre.quen...@gmail.com: Removed file: http://bugs.python.org/file20244/cgi_diff.txt ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue4953 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue4953] cgi module cannot handle POST with multipart/form-data in 3.x
Pierre Quentel pierre.quen...@gmail.com added the comment: Ok Eric, thanks for the tips I attach the diff for the 2 modified modules (cgi.py and test_cgi.py). For the other tests, they are not in the branch and there are many test files so I leave the zip file I removed outdated diffs -- Added file: http://bugs.python.org/file20387/cgi_32.patch ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue4953 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue4953] cgi module cannot handle POST with multipart/form-data in 3.x
Pierre Quentel pierre.quen...@gmail.com added the comment: Ok, thanks. Here is a summary of the API changes : - the argument fp passed to FieldStorage is either an instance of (a subclass of) io.TextIOBase with a buffer attribute for the underlying binary layer (thus, it can't be a StringIO instance) ; or an object with read() and readline() methods that return bytes Defaults to sys.stdin - 2 additional arguments can be passed to the FieldStorage constructor : . stream_encoding : the encoding used by the user agent to encode submitted data. It must be the same as the content-type of the HTML page where the form stands. Defaults to utf-8 . charset : the encoding used by the CGI script (the one used by the print() function to encode and send to sys.stdout). It must be the same as the charset in the content-type header sent by this script. Defaults to None, in which case the default encoding of sys.stdout is used - the only change in the object returned by FieldStorage() is that, if a field represents a file (its argument filename is not None), the read() method on this field returns bytes, and its attribute value is a bytestring, not a string -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue4953 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue4953] cgi module cannot handle POST with multipart/form-data in 3.x
Pierre Quentel pierre.quen...@gmail.com added the comment: Comment ça, no up to date patch ? cgi_32.patch is up to date, the API changes are documented, the unittests work, what else do you want ? -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue4953 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue4953] cgi module cannot handle POST with multipart/form-data in 3.0
Pierre Quentel pierre.quen...@gmail.com added the comment: Many thoughts and tests after... Glenn, the both of us were wrong : the encoding to use in FieldStorage is neither latin-1, nor sys.stdin.encoding : I tested form fields with characters whose utf-8 encoding has bytes that map to undefined in cp1252, the calls to the decode() method with sys.stdin.encoding failed The encoding used by the browser is defined in the Content-Type meta tag, or the content-type header ; if not, the default seems to vary for different browsers. So it's definitely better to define it The argument stream_encoding used in FieldStorage *must* be this encoding ; in this version, it is set to utf-8 by default But this raises another problem, when the CGI script has to print the data received. The built-in print() function encodes the string with sys.stdout.encoding, and this will fail if the string can't be encoded with it. It is the case on my PC, where sys.stdout.encoding is cp1252 : it can't handle Arabic or Chinese characters The solution I have tried is to pass another argument, charset, to the FieldStorage contructor, defaulting to utf-8. It must be the same as the charset defined in the CGI script in the Content-Type header FieldStorage uses this argument to override the built-in print() function : - flush the text layer of sys.stdin, in case calls to print() have been made before calling FieldStorage - get the binary layer of stdout : out = sys.stdout.detach() - define a function _print this way: def _print(*strings): for item in strings: out.write(str(item).encode(charset)) out.write(b'\r\n') - override print() : import builtins builtins.print = _print The function print() in the CGI script now sends the strings encoded with charset to the binary layer of sys.stdout. All the tests I made with Arabic or Chinese input fileds, or file names, succed when using this patch ; so do test_cgi and cgi_test (slightly modified) -- Added file: http://bugs.python.org/file20382/cgi_diff_20110112.txt ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue4953 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue4953] cgi module cannot handle POST with multipart/form-data in 3.0
Pierre Quentel pierre.quen...@gmail.com added the comment: @Glenn I'm curious what your system (probably Windows since you mention cp-) and browser, and HTTP server is, that you used for that test. Is it possible to capture the data stream for that test? Describe how, and at what stage the data stream was captured, if you can capture it. Most interesting would be on the interface between browser and HTTP server. I tested it on Windows XP Family Edition 2020, Service Pack 3, with Python 3.2b2 Browsers : Mozilla Firefox 3.6.13 and Internet Explorer 7.0 Servers : Apache 2.2, and the built-in server started by : import http.server http.server.test(HandlerClass=http.server.CGIHTTPRequestHandler) I print the bytes received in the multipart/form-data part by print(odelim+line) at the end of method read_lines_to_outerboundary() of FieldStorage. The bytes sent when I enter the string a+n tilde + the euro sign are : b'a\xf1\x80' - that is, the cp-1252 encoding of the string Since it works the same with 2 browsers and 2 web servers, I'm almost sure it's not dependant on the configuration - but if others can tests on different configurations I'd like to know the result Basically, this behaviour is not surprising : if sys.stdin.encoding is set to a certain value, it's natural that the bytes sent on the binary layer are encoded with this encoding, not with latin-1 I attach the diff file for an updated version of cgi.py : - new argument stream_encoding instead of setting an attribute encoding to fp - use locale.getpreferredencoding() to decode the query string -- Added file: http://bugs.python.org/file20356/cgi_diff_20110111.txt ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue4953 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue4953] cgi module cannot handle POST with multipart/form-data in 3.0
Pierre Quentel pierre.quen...@gmail.com added the comment: @Glenn Also, the use of FeedParser could be replaced by BytesFeedParser, thus eliminating the need to decode header lines in that loop. BytesFeedParser only uses the ascii codec ; if the header has non ASCII characters (filename in a multipart/form-data), they are replaced by ? : the original file name is lost. So for the moment I leave the text version of FeedParser @Victor : you should use qs.encode(locale.getpreferredencoding(), 'surrogateescape') Ok, I changed the code to that Maybe a DeprecationWarning if we would like to drop support of TextIOWrapper later :-) Maybe I'm missing something here, but sys.stdin is always a TextIOWrapper instance, even if set to binary mode For the else case: you should maybe add a strict test on the type, eg. check for RawIOBase or BufferedIOBase subclass, isinstance(fp, (io.RawIOBase, io.BufferedIOBase)). It would avoid to check that fp.read() returns a bytes object (or get an ugly error later). Rejecting non-instances of RawIOBase or BufferedIOBase is too much, I think. Any class whose instances have a read() method that return bytes should be accepted, like the TestReadLine class in test_cgi.py Set sys.stdin.buffer.encoding attribute is not a good idea. Why do you modify fp, instead of using a separated attribute on FieldStorage (eg. self.fp_encoding)? I set an attribute encoding to self.fp because, for each part of a multipart/form-data, a new instance of FieldStorage is created, and this instance needs to know how to decode bytes. So, either an attribute must be set to one of the arguments of the FieldStorage constructor, and fp comes to mind, or an extra argument has to be passed to this constructor, i.e. the encoding of the original stream -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue4953 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue4953] cgi module cannot handle POST with multipart/form-data in 3.0
Pierre Quentel pierre.quen...@gmail.com added the comment: @Glenn The _defined_ encoding of the original stream is irrelevant, in the same manner that if it is a text stream, that is irrelevant. The stream is binary, and latin-1, or it is non-standard I wish it could be as simple, but I'm afraid it's not. On my PC, sys.stdin.encoding is cp-1252. I tested a multipart/form-data with an INPUT field, and I entered the euro character, which is encoded \x80 in cp-1252 If I use the encoding defined for sys.stdin (cp-1252) to decode the bytes received on sys.stdin.buffer, I get the correct value in the cgi script ; if I set the encoding to latin-1 in FieldStorage, since \x80 maps to undefined in latin-1, I get a UnicodeEncodeError if I try to print the value (character maps to undefined) -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue4953 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue4953] cgi module cannot handle POST with multipart/form-data in 3.0
Pierre Quentel pierre.quen...@gmail.com added the comment: Here is the diff file for the revised version of cgi.py FieldStorage tests if the stream is an instance of (a subclass of) io.TextIOBase. If true, data is read from its attribute buffer ; if it hasn't one (eg for StringIO instances), an AttributeException is raised. Should we have a more specific exception ? If false, the stream's method read() is supposed to return bytes ; an exception will be raised if it's not the case The encoding used to decode keys and values to strings is the attribute encoding of the stream, or latin-1 if this attribute doesn't exist Besides FieldStorage, I modified the parse() function at module level, but not parse_multipart (should it be kept at all ?) I leave the code to set sys.stdin to binary on Windows for the moment, but it can be removed in the final version thanks to Victor's fix of issue 10841 I modified cgi_test.py and test_cgi.py (sent in a next post), all the tests pass with the revised version of cgi.py on my PC While testing the patch I found other related things that I suppose should be changed (but need to check again - perhaps there are already tracker issues about them) : - in http.server.CGIHTPPRequestHandler, the -u option should be removed (line 1123) - on Windows, http.server.SimpleHTTPRequestHandler.list_directory() fails with Arabic characters (mbcs encoding fails, utf-8 works) - in urllib.parse.unquote(), default encoding should be latin-1, not utf-8 (submitting a simple form with French accented characters raises a UnicodeEncodeError when trying to print the submitted value) -- Added file: http://bugs.python.org/file20322/cgi_diff_20110109.txt ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue4953 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue4953] cgi module cannot handle POST with multipart/form-data in 3.0
Pierre Quentel pierre.quen...@gmail.com added the comment: cgi tests -- Added file: http://bugs.python.org/file20323/cgi_tests.zip ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue4953 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue4953] cgi module cannot handle POST with multipart/form-data in 3.0
Pierre Quentel pierre.quen...@gmail.com added the comment: Option 1 is impossible, because the CGI script sometimes has no control on the stream : for instance on a shared web host, it will receive sys.stdin as a text stream I also vote for option 3 ; explaining that if no argument is passed, the program will use sys.stdin.buffer (or the result of sys.stdin.detach() : I guess it's the same ?), and that if an argument is passed, it must provide an attribute buffer (or a method detach() ?) as the binary layer of the stream BTW, I didn't have time to finish the versions of cgi.py and tests, my next slot is this week-end -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue4953 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue4953] cgi module cannot handle POST with multipart/form-data in 3.0
Pierre Quentel pierre.quen...@gmail.com added the comment:
I agree that the only consistent solution is to impose that the attribute
self.fp must read bytes in all cases, all required conversions should occur
inside FieldStorage, using some encoding (not sure how to define it...)
If no argument fp is passed to __init__(), the instance uses the binary version
of sys.stdin. In my patch I use sys.stdin.buffer, but it also works if I set it
to sys.stdin.detach()
In all cases the interpreter must be launched with the -u option. As stated in
the documentation, the effect of this option is to force the binary layer of
the stdin, stdout and stderr streams (which is available as their buffer
attribute) to be unbuffered. The text I/O layer will still be line-buffered..
On my PC (Windows XP) this is required to be able to read all the data stream ;
otherwise, only the beginning is read. I tried Glenn's suggestion with mscvrt,
with no effect
I am working on the cgi.py module so that all tests (test_cgi and cgi_test)
pass with binary streams. It's almost finished ; I had to adapt the tests, and
sometimes fix bugs in them
Problems in test_cgi.py :
- in testQSAndFormData() string data should not begin with a line feed
- in testQSAndFormDataFile() : same thing as above + the argument to update
result should be {'upload': b'this is the content of the fake file\n'} : bytes,
ending with a line feed as in the string data
- in do_test(), for POST method, fp must be a BytesIO
- in test_fieldstorage_multipart(), expected value should be b'Testing 123.\n'
for the third case (filename is not None, bytes expected, there is a line feed
in string data)
Problems in cgi_test.py
- data files mix headers (which should be strings) and POST data which should
be read as bytes. In setup(), the file is opened in binary mode, the first two
lines are read to initialize Content-Length and Content-Type, and an attribute
encoding = 'latin-1' is set
- the tests showed warnings ResourceWarning: unclosed file _io.BufferedReader
name='zenASCII.txt', I changed the code to avoid these warnings
I will send the results (diff for new version of cgi + tests) hopefully tomorrow
--
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue4953
___
___
Python-bugs-list mailing list
Unsubscribe:
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
