[issue24827] round(1.65, 1) return 1.6 with decimal
Merlijn van Deen added the comment:
As Zachary explained, the behavior is correct. There are three issues in play
here.
1) The rounding method. With the ROUND_HALF_EVEN rounding mode, .5 is rounded
to the nearest *even* number, so 1.65 is rounded to 1.6, while 1.75 is rounded
to 1.8.
2) Rounding of floats. Floats cannot represent every number, and numbers are
therefore rounded.
- round(2.675, 2) = round(2.6748, 2) and is thus rounded to 2.67
- round(1.65, 1) = round(1.6499, 1) and is thus rounded to 1.6
3a) In Python 2, round returns a float, so Decimal(round(Decimal(1.65))) =
Decimal(1.6) =
Decimal('1.600088817841970012523233890533447265625') !=
Decimal('1.6')
3b) In Python 3, Decimal.__round__ is implemented, so round(D(1.65), 1) ==
D(1.6) as expected.
--
nosy: +valhallasw
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue24827
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue24108] fnmatch.translate('*.txt') fails
Merlijn van Deen added the comment: As far as I can see, the regex is correct: \Z Matches only at the end of the string. (?iLmsux) The group matches the empty string; the letters set the corresponding flags: (...) - re.M (multi-line), - re.S (dot matches all) See https://docs.python.org/3.4/library/re.html Do you have an example where the regex does not match a file it should match (or matches a file it shouldn't)? -- nosy: +valhallasw ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue24108 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue24108] fnmatch.translate('*.txt') fails
Merlijn van Deen added the comment: They are subtly different; the new regex also matches filenames with newlines, the old one doesn't (see Issue #6665 [1]). Patch (although crazily minor) attached. With some fuzz, it applies on everything from 2.6..default. [1] https://bugs.python.org/issue6665 -- assignee: - docs@python components: +Documentation -asyncio keywords: +patch nosy: +docs@python Added file: http://bugs.python.org/file39261/default.diff ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue24108 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue21300] Docs (incorrectly) suggest email.policy.default is the default policy
Merlijn van Deen added the comment: Small typo that slipped in: 'udpate' instead of 'update' on the following lines: http://hg.python.org/cpython/rev/63fa945119cb#l2.18 http://hg.python.org/cpython/rev/63fa945119cb#l2.43 http://hg.python.org/cpython/rev/63fa945119cb#l2.66 -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue21300 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue21315] email._header_value_parser does not recognise in-line encoding changes
New submission from Merlijn van Deen:
Bugzilla sends e-mail in a format where =?UTF-8 is not preceded by whitespace.
This makes email.headerregistry.UnstructuredHeader (and
email._header_value_parser on the background) not recognise the structure.
import email.headerregistry, pprint
x = {}; email.headerregistry.UnstructuredHeader.parse('[Bug
64155]\tNew:=?UTF-8?Q?=20non=2Dascii=20bug=20t=C3=A9st?=;\trussian
text:=?UTF-8?Q?=20=D0=90=D0=91=D0=92=D0=93=D2=90=D0=94', x);
pprint.pprint(x)
{'decoded': '[Bug 64155]\tNew:=?UTF-8?Q?=20non=2Dascii=20bug=20t=C3=A9st?=;\t'
'russian text:=?UTF-8?Q?=20=D0=90=D0=91=D0=92=D0=93=D2=90=D0=94',
'parse_tree': UnstructuredTokenList([ValueTerminal('[Bug'),
WhiteSpaceTerminal(' '), ValueTerminal('64155]'), WhiteSpaceTerminal('\t'),
ValueTerminal('New:=?UTF-8?Q?=20non=2Dascii=20bug=20t=C3=A9st?=;'),
WhiteSpaceTerminal('\t'), ValueTerminal('russian'), WhiteSpaceTerminal(' '),
ValueTerminal('text:=?UTF-8?Q?=20=D0=90=D0=91=D0=92=D0=93=D2=90=D0=94')])}
versus
x = {}; email.headerregistry.UnstructuredHeader.parse('[Bug 64155]\tNew:
=?UTF-8?Q?=20non=2Dascii=20bug=20t=C3=A9st?=;\trussian text:
=?UTF-8?Q?=20=D0=90=D0=91=D0=92=D0=93=D2=90=D0=94', x); pprint.pprint(x)
{'decoded': '[Bug 64155]\tNew: non-ascii bug tést;\trussian text: АБВГҐД',
'parse_tree': UnstructuredTokenList([ValueTerminal('[Bug'),
WhiteSpaceTerminal(' '), ValueTerminal('64155]'), WhiteSpaceTerminal('\t'),
ValueTerminal('New:'), WhiteSpaceTerminal(' '),
EncodedWord([WhiteSpaceTerminal(' '), ValueTerminal('non-ascii'),
WhiteSpaceTerminal(' '), ValueTerminal('bug'), WhiteSpaceTerminal(' '),
ValueTerminal('tést')]), ValueTerminal(';'), WhiteSpaceTerminal('\t'),
ValueTerminal('russian'), WhiteSpaceTerminal(' '), ValueTerminal('text:'),
WhiteSpaceTerminal(' '), EncodedWord([WhiteSpaceTerminal(' '),
ValueTerminal('АБВГҐД')])])}
I have attached the raw e-mail as attachment.
Judging by the code, this is supposed to work (while raising a Defect --
missing whitespace before encoded word), but the code splits by whitespace:
tok, *remainder = _wsp_splitter(value, 1)
which swallows the encoded section in one go. In a second attachment, I added a
patch which 1) adds a test case for this and 2) implements a solution, but the
solution is unfortunately not in the style of the rest of the module.
In the meanwhile, I've chosen a monkey-patching approach to work around the
issue:
import email._header_value_parser, email.headerregistry
def get_unstructured(value):
value = value.replace(=?UTF-8?Q?=20, =?UTF-8?Q?)
return email._header_value_parser.get_unstructured(value)
email.headerregistry.UnstructuredHeader.value_parser =
staticmethod(get_unstructured)
--
components: email
files: 000359.raw
messages: 216908
nosy: barry, r.david.murray, valhallasw
priority: normal
severity: normal
status: open
title: email._header_value_parser does not recognise in-line encoding changes
versions: Python 3.3, Python 3.4, Python 3.5
Added file: http://bugs.python.org/file34984/000359.raw
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue21315
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue21315] email._header_value_parser does not recognise in-line encoding changes
Changes by Merlijn van Deen valhall...@gmail.com: -- keywords: +patch type: - behavior Added file: http://bugs.python.org/file34985/unstructured_ew_without_whitespace.diff ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue21315 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue21300] Docs (incorrectly) suggest email.policy.default is the default policy
New submission from Merlijn van Deen: Which would make sense, but email.policy.Compat32 is *actually* the default policy. This patch adapts the documentation to reflect this. -- assignee: docs@python components: Documentation files: defaultpolicy.diff keywords: patch messages: 216783 nosy: docs@python, r.david.murray, valhallasw priority: normal severity: normal status: open title: Docs (incorrectly) suggest email.policy.default is the default policy versions: Python 3.3, Python 3.4, Python 3.5 Added file: http://bugs.python.org/file34964/defaultpolicy.diff ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue21300 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue19456] ntpath doesn't join paths correctly when a drive is present
Merlijn van Deen added the comment: I'm not sure whether that question was aimed at me -- I think both options have their merits, but I'd suggest to adopt the .NET semantics. The semantics are also explicitly defined [1] and the behavior seems to be acceptable for the .NET world. [1] http://msdn.microsoft.com/en-us/library/fyy7a5kt(v=vs.110).aspx -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue19456 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue19456] ntpath doesn't join paths correctly when a drive is present
Merlijn van Deen added the comment:
so perhaps ntpath.join('c:/x', 'd:/y', 'c:z') should return 'c:/x\\z', not
'c:/z'.
'c:z' is consistent with what .NET's System.IO.Path.Combine does:
via http://ironpython.net/try/ :
import System.IO.Path; print System.IO.Path.Combine('c:/x', 'd:/y', 'c:z')
returns 'c:z'
Could anyone please check it? Create directory x/z on drive c: and directory
y on drive d:, then execute following commands:
cd c:/x
cd d:/y
cd c:z
What is resulting current working directory?
c:\cd c:/x
c:\xcd e:\y
c:\xcd c:z
Het systeem kan het opgegeven pad niet vinden. # file not found, in Dutch
c:\xcd c:\z
Yes, there is a seperate current directory for each drive, but cd does not
switch drives. (cd e:\f does not mean you actually go to e:\f - it just changes
the current directory on the e:-drive). I don't think those semantics are
sensible for joining paths...
--
nosy: +valhallasw
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue19456
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue19456] ntpath doesn't join paths correctly when a drive is present
Merlijn van Deen added the comment: Sorry, I was a bit too quick - I forgot to create c:\x\z. Now this is the result: c:\x\zcd c:/x c:\xcd e:/y c:\xcd c:z c:\x\z However, the behavior does not work in, for example, a 'Save as...' window, where c:z will always return illegal filename -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue19456 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue19919] SSL: test_connect_ex_error fails with EWOULDBLOCK
Merlijn van Deen added the comment:
My error is slightly different:
$ ./python -i -c from test.test_ssl import *;
support.run_unittest(NetworkedTests)
(...)
==
FAIL: test_connect_ex_error (test.test_ssl.NetworkedTests)
--
Traceback (most recent call last):
File /home/valhallasw/src/cpython/Lib/test/test_ssl.py, line 1205, in
test_connect_ex_error
s.connect_ex((svn.python.org, 444)))
AssertionError: 111 != 11
--
Ran 15 tests in 33.590s
FAILED (failures=1, skipped=1)
Traceback (most recent call last):
File string, line 1, in module
File /home/valhallasw/src/cpython/Lib/test/support/__init__.py, line 1719,
in run_unittest
_run_suite(suite)
File /home/valhallasw/src/cpython/Lib/test/support/__init__.py, line 1694,
in _run_suite
raise TestFailed(err)
test.support.TestFailed: Traceback (most recent call last):
File /home/valhallasw/src/cpython/Lib/test/test_ssl.py, line 1205, in
test_connect_ex_error
s.connect_ex((svn.python.org, 444)))
AssertionError: 111 != 11
errno.errorcode[11]
'EAGAIN'
errno.errorcode[111]
'ECONNREFUSED'
This is on Ubuntu x64 12.04 LTS, under VirtualBox 4.3.4 (r91027), running on
Windows 7 Professional (6.1.7601).
OpenSSL versions as returned by test_ssl:
$ ./python -m Lib.test.test_ssl
test_ssl: testing with 'OpenSSL 1.0.1 14 Mar 2012' (1, 0, 1, 0, 15)
under Linux ('debian', 'wheezy/sid', '')
HAS_SNI = True
OP_ALL = 0x83ff
OP_NO_TLSv1_1 = 0x1000
--
nosy: +valhallasw
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue19919
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue19919] SSL: test_connect_ex_error fails with EWOULDBLOCK
Merlijn van Deen added the comment:
Yes, they are.
errno.EWOULDBLOCK
11
EAGAIN and EWOULDBLOCK are the only two with that errno:
[(k,v) for (k,v) in errno.__dict__.items() if v==11]
[('EWOULDBLOCK', 11), ('EAGAIN', 11)]
111 is just ECONNREFUSED:
[(k,v) for (k,v) in errno.__dict__.items() if v==111]
[('ECONNREFUSED', 111)]
--
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue19919
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue19919] SSL: test_connect_ex_error fails with EWOULDBLOCK
Merlijn van Deen added the comment: OK. I did some network sniffing; inside the VM, this is what I see: $ sudo tshark host 82.94.164.164 tshark: Lua: Error during loading: [string /usr/share/wireshark/init.lua]:45: dofile has been disabled Running as user root and group root. This could be dangerous. Capturing on eth0 0.0010.0.2.15 - 82.94.164.164 TCP 74 52719 snpp [SYN] Seq=0 Win=14600 Len=0 MSS=1460 SACK_PERM=1 TSval=10175140 TSecr=0 WS=16 while on the host, this is what I see: 1 0.0 192.168.1.122 82.94.164.164 TCP 66 49909 snpp [SYN] Seq=0 Win=8192 Len=0 MSS=1460 WS=4 SACK_PERM=1 2 0.029908000 82.94.164.164 192.168.1.122 TCP 54 snpp 49909 [RST, ACK] Seq=1 Ack=1 Win=0 Len=0 Basically, the RST sent by the server never reaches the VM. This might actually happen in other NAT-ed situations, too, I guess. But even if this is a VM issue, it *does* make the tests misbehave. Maybe receiving EAGAIN should result in a test skip instead of an error? -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue19919 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue6784] byte/unicode pickle incompatibilities between python2 and python3
Changes by Merlijn van Deen valhall...@gmail.com: Removed file: http://bugs.python.org/file24640/BytestrPickler_c.diff ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue6784 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue6784] byte/unicode pickle incompatibilities between python2 and python3
Changes by Merlijn van Deen valhall...@gmail.com: Removed file: http://bugs.python.org/file24688/test_pickle.diff ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue6784 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue6784] byte/unicode pickle incompatibilities between python2 and python3
Changes by Merlijn van Deen valhall...@gmail.com: Removed file: http://bugs.python.org/file24719/pickle_bytestr.patch ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue6784 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue6784] byte/unicode pickle incompatibilities between python2 and python3
Changes by Merlijn van Deen valhall...@gmail.com: Removed file: http://bugs.python.org/file24906/pickle_bytes_code.diff ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue6784 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue6784] byte/unicode pickle incompatibilities between python2 and python3
Changes by Merlijn van Deen valhall...@gmail.com: Removed file: http://bugs.python.org/file24907/pickle_bytes_tests.diff ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue6784 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue6784] byte/unicode pickle incompatibilities between python2 and python3
Changes by Merlijn van Deen valhall...@gmail.com: Removed file: http://bugs.python.org/file24568/pickle.py.patch ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue6784 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue6784] byte/unicode pickle incompatibilities between python2 and python3
Merlijn van Deen added the comment: Hi Alexandre, Attached is a diff based on r87793:0c508d87f80b. Merlijn -- Added file: http://bugs.python.org/file33011/bytestrpickle.diff ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue6784 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue6784] byte/unicode pickle incompatibilities between python2 and python3
Merlijn van Deen added the comment: I have fixed most of the nits in this patch, except for: 1) the intermediate bytes object being created; inlining is an option, as storchaka suggested, but I'd rather have you decide what it should become before implementing it; 2) make clinic gives me ./python -E ./Tools/clinic/clinic.py --make Error in file ./Modules/_pickle.c on line 6611: Checksum mismatch! Expected: bed0d8bbe1c647960ccc6f997b33bf33935fa56f Computed: 58dcccb705487695fec30980f566027bc68d9c69 make: *** [clinic] Error 255 and I have no clue how to fix that -- the clinic docs are sparse, to say the least; 3) The tests are still in their own test case; please decide between the two of you what is the best solution; 4) I have grouped the test cases: test_load_python2_str_as_bytes (which checks protocols 0, 1, and 2), test_load_python2_unicode_as_str and test_load_long_python2_str_as_bytes; 5) I have moved the commands to create the shown pickled versions from docstrings to comments. If you think they are not useful, I'll remove them, but I found them pretty useful while shortening the strings. -- Added file: http://bugs.python.org/file33016/bytestrpickle.diff ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue6784 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue18870] eval() uses latin-1 to decode str
Merlijn van Deen added the comment: On the lowest level, this affects exec, eval(), compile() and input() (!). On a higher level, more modules are affected: modules ast, codeop, compiler, cProfile, dis, distutils (not sure), doctest, idlelib, ihooks, pdb, pkgutil, plat-mac, py_compile, rexec, runpy and timeit all call compile() modules dbd, compiler, gettext, idlelib, lib2to3, lib-tk.turtle, logging, mhlib, pdb, plat-irix5, plat-mac, rexec, rlcompleter and warnings all call eval() and modules Bastion, bdb, code, collections, cProfile, distutils, doctest, idlelib, ihooks, imputil, pdb, plat-irix5, plat-irix6, plat-mac, profile, rexec, site, timeit and trace all call exec. Not all of them necessarily take user-supplied code - I haven't checked that. After checking tests/test_pep263.py, it seems the behavior is a bit more complicated than I initially thought: a str parameter is considered latin-1 unless either a) an utf-8 bom is present, in which case it is considered utf-8 b) an # encoding: XXX line is present, in which case it is considered to be in that encoding In any case, I have attached a doc patch for exec, eval(), compile(), and ast.literal_eval(), because I think these are the most widely used. I think input() does not need a doc change because it explicitly refers to eval(). I ignored the subtleties noted above for the doc patch, simplifying to 'pass either a Unicode or a latin-1 encoded string'. -- keywords: +patch Added file: http://bugs.python.org/file31513/doc_18870.patch ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue18870 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue18870] eval() uses latin-1 to decode str
New submission from Merlijn van Deen:
Steps to reproduce:
---
eval(u'ä')
# in an utf-8 console, so this is equivalent to
eval(u'\xc3\xa4')
Actual result:
u'\xc3\xa4'
# i.e.: u'ä'
Expected result:
-
SyntaxError: Non-ASCII character '\xc3' in file string on line 1, but no
encoding declared; see http://www.python.org/peps/pep-0263.html for details
(which is what would happen if it was in a source file)
Or, alternatively:
UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in position 2: ordinal
not in range(128)
(which is what results from decoding the str with sys.getdefaultencoding())
Instead, the string is interpreted as latin-1. The same happens for
ast.literal_eval - even calling compile() directly.
In python 3.2, this is the result, as utf-8 is used as default source encoding:
eval(b'\xc3\xa4')
'ä'
Workarounds
--
eval(# encoding: utf-8\nu'\xc3\xa4')
u'\xe4'
eval(u'\xc3\xa4'.decode('utf-8'))
u'\xe4'
I understand this might be considered a WONTFIX, as it would change behavior
some people might depend on. Nonetheless, documenting this explicitly seems a
sensible thing to do.
--
messages: 196398
nosy: valhallasw
priority: normal
severity: normal
status: open
title: eval() uses latin-1 to decode str
versions: Python 2.7
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue18870
___
___
Python-bugs-list mailing list
Unsubscribe:
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue15097] Improving wording on the thread-safeness of import
New submission from Merlijn van Deen valhall...@gmail.com: http://docs.python.org/library/threading.html#importing-in-threaded-code Currently, the documentation states Firstly, other than in the main module, an import should not have the side effect of spawning a new thread and then waiting for that thread in any way. Failing to abide by this restriction can lead to a deadlock if the spawned thread directly or indirectly attempts to import a module. which, I think, fails to make the main point: a call to import acquires the import lock. A call to import from a second thread will thus block. As such, I would suggest rephrasing it to something like: Firstly, an import acquires the import lock for that thread. Therefore, the import should not have the side effect of waiting for a different thread in any way, as this can lead to a deadlock if that thread directly or indirectly attempts to import a module. There are two additional points that might be interesting to note: (1) Any module can be imported. If the import causes a deadlock, that is a bad thing. Every module *will* be imported by tools such as nosetests. (1b) so: never, ever, have code that causes locks in a different thread in module level code witout 'if __name__==__main__ ' blocks? (2) The lock is also acquired if a module has already been imported. For instance, in import sys # (1) def f(): import sys # (2) the import lock is acquired in (1) /and/ (2). Adding example code and/or a flow diagram might be a bit too much, but it does clarify how easy it is to make this mistake. See the attached for an example (both a simple example script, as well as a flow diagram explaining what happens). -- assignee: docs@python components: Documentation files: deadlock.py messages: 163068 nosy: docs@python, valhallasw priority: normal severity: normal status: open title: Improving wording on the thread-safeness of import type: enhancement Added file: http://bugs.python.org/file26037/deadlock.py ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue15097 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue15097] Improving wording on the thread-safeness of import
Changes by Merlijn van Deen valhall...@gmail.com: Removed file: http://bugs.python.org/file26037/deadlock.py ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue15097 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue15097] Improving wording on the thread-safeness of import
Changes by Merlijn van Deen valhall...@gmail.com: Added file: http://bugs.python.org/file26038/deadlock.py ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue15097 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue15097] Improving wording on the thread-safeness of import
Merlijn van Deen valhall...@gmail.com added the comment: First off, thank you for your response. The existence of an import lock is deliberately omitted from the text, and the reader is supposed to abide by the restriction as written regardless of the motivation behind it. The entire notion of an import lock is obsolete. Python 3.3 does not have that anymore. This warning is still valid but for a different reason or this warning is no longer valid in 3.3 ? Assuming the first (which is what I guess based on the fact the deadlock still occurs in 3.3), I think the text can still be improved; the current wording suggests to me a) it's OK to wait for a thread as long as you did not create it and b) it's OK to import something that waits for a thread as long as you do it from the main module - while both cases can still lead to a deadlock. so, leaving the implementation details out, this is my suggestion: Firstly, an import should not have the side effect of waiting for a thread in any way. This can lead to a deadlock if that thread directly or indirectly attempts to import a module. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue15097 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue14873] Windows devguide: clarification for build errors due to missing optional dependencies
Merlijn van Deen valhall...@gmail.com added the comment: Thanks for your responses! @Ezio: 1) I can see your point. On the other hand, I also think it is helpful to have a prominent note on this - the first thing people see after following the instructions is a huge list of build errors (see the bottom of this post) - build errors that are not really 'build errors' in the normal sense, because they just mean several optional subprojects were not built. In any case, I've removed the text from this patch. Maybe a better solution would be to add a '(details)' link to each step in the Quick Start that refers to the relevant section in the devguide? 2) Sorry; it looks that way from the patch - but it's actually in 'Windows' section - which is about a screen down from the 'Build dependencies' section on my computer. The patch in this post is a .diff instead of the hg patch format, hoping that Rietveld will pick it up then. 3) Fixed. @Éric: That should have been 'project', to use the VS 'solution' with 'projects' (that each build one library/executable) terminology - in this case, it refers to the project that builds python.exe (called 'python') inside the solution (called 'PCBuild'). In this patch 'project' is used consistently, although I welcome alternative wordings. --/ error output /-- Error 3 error C1083: Cannot open include file: 'bzlib.h': No such file or directory C:\Users\valhallasw\Documents\Visual Studio 2010\cpython\Modules\_bz2module.c 12 1 _bz2 Error 5 error C1083: Cannot open source file: '..\..\bzip2-1.0.6\blocksort.c': No such file or directory C:\Users\valhallasw\Documents\Visual Studio 2010\cpython\PCbuild\c1 _bz2 Error 6 error C1083: Cannot open source file: '..\..\bzip2-1.0.6\bzlib.c': No such file or directory C:\Users\valhallasw\Documents\Visual Studio 2010\cpython\PCbuild\c1 _bz2 Error 7 error C1083: Cannot open source file: '..\..\bzip2-1.0.6\compress.c': No such file or directory C:\Users\valhallasw\Documents\Visual Studio 2010\cpython\PCbuild\c1 _bz2 Error 8 error C1083: Cannot open source file: '..\..\bzip2-1.0.6\crctable.c': No such file or directory C:\Users\valhallasw\Documents\Visual Studio 2010\cpython\PCbuild\c1 _bz2 Error 9 error C1083: Cannot open source file: '..\..\bzip2-1.0.6\decompress.c': No such file or directory C:\Users\valhallasw\Documents\Visual Studio 2010\cpython\PCbuild\c1 _bz2 Error 10 error C1083: Cannot open source file: '..\..\bzip2-1.0.6\huffman.c': No such file or directory C:\Users\valhallasw\Documents\Visual Studio 2010\cpython\PCbuild\c1 _bz2 Error 11 error C1083: Cannot open source file: '..\..\bzip2-1.0.6\randtable.c': No such file or directory C:\Users\valhallasw\Documents\Visual Studio 2010\cpython\PCbuild\c1 _bz2 Error 21 error C1083: Cannot open include file: 'openssl/evp.h': No such file or directory C:\Users\valhallasw\Documents\Visual Studio 2010\cpython\Modules\_hashopenssl.c 40 1 _hashlib Error 4 error C1083: Cannot open include file: 'lzma.h': No such file or directory C:\Users\valhallasw\Documents\Visual Studio 2010\cpython\Modules\_lzmamodule.c 19 1 _lzma Error 15 error C1083: Cannot open include file: 'sqlite3.h': No such file or directory c:\users\valhallasw\documents\visual studio 2010\cpython\modules\_sqlite\connection.h 33 1 _sqlite3 Error 16 error C1083: Cannot open include file: 'sqlite3.h': No such file or directory c:\users\valhallasw\documents\visual studio 2010\cpython\modules\_sqlite\connection.h 33 1 _sqlite3 Error 17 error C1083: Cannot open include file: 'sqlite3.h': No such file or directory c:\users\valhallasw\documents\visual studio 2010\cpython\modules\_sqlite\connection.h 33 1 _sqlite3 Error 18 error C1083: Cannot open include file: 'sqlite3.h': No such file or directory c:\users\valhallasw\documents\visual studio 2010\cpython\modules\_sqlite\connection.h 33 1 _sqlite3 Error 19 error C1083: Cannot open include file: 'sqlite3.h': No such file or directory c:\users\valhallasw\documents\visual studio 2010\cpython\modules\_sqlite\connection.h 33 1 _sqlite3 Error 20 error C1083: Cannot open include file: 'sqlite3.h': No such file or directory c:\users\valhallasw\documents\visual studio 2010\cpython\modules\_sqlite\connection.h 33 1 _sqlite3 Error 22 error C1083: Cannot open include file: 'sqlite3.h': No such file or directory c:\users\valhallasw\documents\visual studio 2010\cpython\modules\_sqlite\connection.h 33 1 _sqlite3 Error 14 error C1083: Cannot open include file: 'openssl/rsa.h': No such file or directory C:\Users\valhallasw\Documents\Visual Studio 2010\cpython\Modules\_ssl.c 91 1 _ssl Error 12
[issue4198] os.path.normcase gets fooled on windows with mapped linux network drive
Merlijn van Deen valhall...@gmail.com added the comment: OK, I did some quick tests. Given a samba share with the following files A (contents: 'test: A') a (contents: 'test: a') B (contents: 'test: B') 1) opening \\share\files\A or \\share\files\a opens the same file - in my case 'test: A' 2) opening \\share\files\B or \\share\files\b both work. As such, the behaviour is at least not incorrect. In addition, it means that the if normcase(path1) == normcase(path2): option to check if two files are the same is actually correct (if normcase(path1) == normcase(path2) then open(path1).read() == open(path2).read() - at least on the share I tested). Interestingly, os.path.samefile suggests this would not be the case: True os.path.samefile(u:\\a, u:\\A) False On 22 May 2012 03:59, Ezio Melotti rep...@bugs.python.org wrote: Changes by Ezio Melotti ezio.melo...@gmail.com: -- nosy: +brian.curtin, tim.golden stage: - needs patch versions: -Python 3.4 ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue4198 ___ -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue4198 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue4198] os.path.normcase gets fooled on windows with mapped linux network drive
Merlijn van Deen valhall...@gmail.com added the comment: To confirm this behaviour is still current: Python 3.3.0a1+ (default:958a98bf924e+, May 21 2012, 22:18:16) [GCC 4.5.2] on linux Type help, copyright, credits or license for more information. import ntpath ntpath.normcase(rc:\mixedCASE) 'c:\\mixedcase' In general, I'm not so sure of the use case of the function /at all/ - possibly something like if normcase(path1) == normcase(path2): ... however, this can give the false impression that two files are the same - I think, given a Samba share with two files 'A' and 'a', it's possible to access both files from windows. -- nosy: +valhallasw versions: +Python 3.3, Python 3.4 -Python 2.7 ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue4198 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue14873] Windows devguide: clarification for build errors due to missing optional dependencies
New submission from Merlijn van Deen valhall...@gmail.com: The amount of errors in the solution during the windows build was surprising to me - I interpreted it as if python was not built correctly. Upon further inspection, just some extension modules were not built due to missing dependencies, and the interpreter runs without problems. To clarify this in the devguide, I suggest the attached changes. -- components: Devguide files: devguide_windows_build_errors_notes.patch keywords: patch messages: 161299 nosy: ezio.melotti, valhallasw priority: normal severity: normal status: open title: Windows devguide: clarification for build errors due to missing optional dependencies Added file: http://bugs.python.org/file25663/devguide_windows_build_errors_notes.patch ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue14873 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue14361] No link to issue tracker on Python home page
Merlijn van Deen valhall...@gmail.com added the comment: Maybe, but python.org also is the host of CPython itself (and this issue tracker is also for issues on the programming language). I think the Core development page makes sense, but having it in a sidebar instead of somewhere at the bottom of the page would be helpful, or an introduction that notes there are useful links at the bottom. Maybe just move the quick links section to above the quick start section? To be more specific- the devguide made me think 'oh, so this is the page on how I compile CPython, but I'm looking for the issue tracker. *clicks back*' -- nosy: +valhallasw ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue14361 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue6784] byte/unicode pickle incompatibilities between python2 and python3
Merlijn van Deen valhall...@gmail.com added the comment: Based on the discussion on python-dev [1], this is an updated implementation that uses encoding='bytes' to signal str-bytes behaviour. http://mail.python.org/pipermail/python-dev/2012-March/117536.html -- Added file: http://bugs.python.org/file24906/pickle_bytes_code.diff ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue6784 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue6784] byte/unicode pickle incompatibilities between python2 and python3
Merlijn van Deen valhall...@gmail.com added the comment: ...and the tests to go with that. -- Added file: http://bugs.python.org/file24907/pickle_bytes_tests.diff ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue6784 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue6566] json.dumps converts None to null (not null)
Merlijn van Deen valhall...@gmail.com added the comment:
JSON does not have the distinction between bytes and unicode; py2 strings are
coerced into unicode.
I think it would be better to speak about 'JSON string' or 'JSON string
element' if it's JSON and about 'unicode' (2.7) or 'str' (3.x) when speaking
about the data that has been decoded again.
json.loads(json.dumps(boo))
u'boo'
json.loads(json.dumps({a: b}))
{u'a': u'b'}
(the keys will always be unicode in 2.7)
--
nosy: +valhallasw
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue6566
___
___
Python-bugs-list mailing list
Unsubscribe:
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue829370] math.signum(int)
Merlijn van Deen valhall...@gmail.com added the comment: I'm not quite sure why that formula would be elegant in the first place, and I most certainly don't understand why 0.5*sign((100*YY)+MM-190002.5) + 0.5 is more elegant than ((100*YY)+MM 190002.5) or (((YY = 1900) and (MM 2.5)) or (YY 1900)) or rather: implementing leap years correctly in the first place, so the formula also works outside of the 1800-2099 range. And, in general, I don't understand the problem. Everyone who does scientific computing has numpy *anyway*, so there is no gain for them. As a last note, the C math.h also doesn't have a sign() function, and only a copysign() function: http://en.wikipedia.org/wiki/C_mathematical_functions -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue829370 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue829370] math.signum(int)
Merlijn van Deen valhall...@gmail.com added the comment: Although there is one use case which I now realise due to your post: easier 1-on-1 implementation of existing algorithms. Another possible reason to implement it is that it's not that hard to implement the sign() function wrongly, if it also has to work with nans. This implementation: def signum(x): return (x 0) - (x 0) returns 0 for nan, which is wrong (it should return nan). Another naive implementation def signum(x): return math.copysign(1, x) also fails for nan, and gives a result for +/- 0 that could be right or wrong, depending on context. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue829370 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue829370] math.signum(int)
Merlijn van Deen valhall...@gmail.com added the comment: numpy.sign does this: http://docs.scipy.org/doc/numpy/reference/generated/numpy.sign.html -- nosy: +valhallasw ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue829370 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue6147] multithreading.Pool.map() crashes Windows computer
Merlijn van Deen valhall...@gmail.com added the comment: Two questions: (1) can this be at least be added as a big fat warning in the documentation? (2) would it be a reasonable option to let either (a) the creation of a Pool (b) executing something using the Pool cause an exception when it happens during the import of the function to run? I think it makes sense to prevent any accidental forkbombs, especially if they are /this/ easy to create. Untested (for obvious reasons...), but this should be enough: import multiprocessing def x(val): return val multiprocessing.Pool().map(x, range(10)) -- nosy: +valhallasw ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue6147 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue6784] byte/unicode pickle incompatibilities between python2 and python3
Changes by Merlijn van Deen valhall...@gmail.com: Removed file: http://bugs.python.org/file24567/test_bytestrpickle.py ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue6784 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue6784] byte/unicode pickle incompatibilities between python2 and python3
Merlijn van Deen valhall...@gmail.com added the comment:
OK, and now a version that's not broken... I forgot to initialize self-bytestr
for PicklerObject/UnpicklerObject. *puts on the you-broke-the-build-hat*
Except for test_packaging.test_caches, this version passes all tests --
test_packaging.test_caches, which seems to fail because I make install'd python
and installed {distribute,pip,setuptools,virtualenv}.
--
Added file: http://bugs.python.org/file24719/pickle_bytestr.patch
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue6784
___
___
Python-bugs-list mailing list
Unsubscribe:
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue6784] byte/unicode pickle incompatibilities between python2 and python3
Changes by Merlijn van Deen valhall...@gmail.com: Removed file: http://bugs.python.org/file24714/pickle_bytestr.patch ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue6784 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue6784] byte/unicode pickle incompatibilities between python2 and python3
Merlijn van Deen valhall...@gmail.com added the comment: And a complete patch that implements the tests, the python implementation and the C implementation. I'm not completely happy with the code duplication in read_string/read_binstring/read_short_binstring C implementation, so that might be an improvement (however, there is already a lot of code duplication there at the moment). Again: comments would be very welcome... -- Added file: http://bugs.python.org/file24714/pickle_bytestr.patch ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue6784 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue6784] byte/unicode pickle incompatibilities between python2 and python3
Merlijn van Deen valhall...@gmail.com added the comment: Added tests in Lib/test format. After applying pickle.py.patch and BytestrPickler_c.diff, ./python -m test -v -m PyPicklerBytestrTests test_pickle returns 12 tests, no errors, while ./python -m test -v -m CPicklerBytestrTests test_pickle only passes test_dump_bytes_protocol_0 (test.test_pickle.CPicklerBytestrTests) ... ok test_dump_bytes_protocol_1 (test.test_pickle.CPicklerBytestrTests) ... ok test_dump_bytes_protocol_2 (test.test_pickle.CPicklerBytestrTests) ... ok test_dump_bytes_protocol_3 (test.test_pickle.CPicklerBytestrTests) ... ok and has 8 errors (as expected). -- Added file: http://bugs.python.org/file24688/test_pickle.diff ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue6784 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue14118] _pickle.c structure cleanup
Merlijn van Deen valhall...@gmail.com added the comment: See https://bitbucket.org/valhallasw/cpython/src/ee0d2beaf6a4/Modules/_pickle.c for a rough structure overview - which maybe also explains why I thought restructuring made sense in the first place. However, I'm not the person who has to maintain the module. If you're not interested, I'll just split out the module until I feel comfortable editing stuff, make my patch for #6784 and backport it to the 6500-line version. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue14118 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue6784] byte/unicode pickle incompatibilities between python2 and python3
Merlijn van Deen valhall...@gmail.com added the comment: Ok, this is my first attempt at the Pickler part of the C implementation. I'll have to adapt the python implementation to match this one. All BytestrPicklerTests in test_bytestrpickle.py pass, and ./python -m test -G -v test_pickle passes. Comments on style etc. are very welcome. -- Added file: http://bugs.python.org/file24640/BytestrPickler_c.diff ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue6784 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue14124] _pickle.c comment/documentation improvement
New submission from Merlijn van Deen valhall...@gmail.com: As suggested by loewis in msg154233, I created some documentation to help people get started with _pickle.c. -- assignee: docs@python components: Documentation, Extension Modules files: _pickle_c_doc.diff keywords: patch messages: 154284 nosy: docs@python, valhallasw priority: normal severity: normal status: open title: _pickle.c comment/documentation improvement type: enhancement versions: Python 3.3 Added file: http://bugs.python.org/file24641/_pickle_c_doc.diff ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue14124 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue14124] _pickle.c comment/documentation improvement
Changes by Merlijn van Deen valhall...@gmail.com: -- nosy: +loewis ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue14124 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue14118] _pickle.c structure cleanup
New submission from Merlijn van Deen valhall...@gmail.com: While working on #6784, I've looked at _pickle.c and found it quite... daunting: 6500 lines and 185kB. I have been working on a bit of cleanup, and I'd like some comments on this. I'm working on adapting _pickle.c into the following structure: _pickle.c takes care of the module initialization _pickle/*.c are helper files for this (global functions/definitions) _pickle/PicklerObject contains all files that relate to the Pickler class - initialization, all functions, etc, and _pickle/PicklerObject/picklers/*.c contains all pickling functions, split out into groups (corresponding to pickletools.StackObjects) This should end in a tree where every .c module lists the relevant dependencies (and as such should be compilable on itself). Currently, I'm at the point where PicklerObject roughly has the structure I want (although not every file is compilable on itself yet). As such, I feel this is the right moment to ask if it would be seen as an useful improvement in trunk, and to see if anyone has suggestions for improvements. hg repos: https://bitbucket.org/valhallasw/cpython/src/0810ffadffa3/Modules/_pickle/PicklerObject (_pickle_clean branch) -- components: Extension Modules hgrepos: 114 messages: 154165 nosy: valhallasw priority: normal severity: normal status: open title: _pickle.c structure cleanup type: enhancement versions: Python 3.3, Python 3.4 ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue14118 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue14118] _pickle.c structure cleanup
Merlijn van Deen valhall...@gmail.com added the comment: That makes sense. The goal was not so much cleaning up the module per se; rather, it was a result of trying to understand the general structure of _pickler.c specifically. However, is there an intermediate level of 'modularization' you would propose? The idea is of course not to have 30 files open to make a small change, but rather have one or two reasonably small files open. My current approach was fine-grained on purpose - it's not that much work to recombine files. I think it makes sense to split the PicklerObject into several parts: * the Pickler object (initialization/type/etc, maybe functions split off) * the actualy pickler implementation (dump/save/save_*) (but maybe in less files - see below) * the PicklerMemoProxy for the picker implementation - specifically the picklers directory - my approach was to have a 'mirrored' directory for PicklerObject and UnPicklerObject: the methods to pickle and unpickle will be in the same files in the two directories (i.e. picklers/unicode.c will pickle str, unpicklers/unicode.c will unpickle them). -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue14118 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue6784] byte/unicode pickle incompatibilities between python2 and python3
Merlijn van Deen valhall...@gmail.com added the comment: Last night, I hacked together a wrapper to do what loewis suggested [1]. It pickles bytes to str (for protocol = 2), and unpickles str to bytes. If I (ever) get the build system and tests of python itself to work, I'll try and see if I can implement a nicer solution - at least for pickle.py. [1] https://github.com/valhallasw/py2/blob/master/bytestrpickle.py -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue6784 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue6784] byte/unicode pickle incompatibilities between python2 and python3
Changes by Merlijn van Deen valhall...@gmail.com: Added file: http://bugs.python.org/file24567/test_bytestrpickle.py ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue6784 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue6784] byte/unicode pickle incompatibilities between python2 and python3
Merlijn van Deen valhall...@gmail.com added the comment: OK, this is the pickle.py patch. A new parameter 'bytestr' has been added to both _Pickler and _Unpickler to toggle the pickle.string=bytes behaviour: _Pickler: IF protocol = 2 AND bytestr=True THEN bytes are stored as STRING/SHORT_BINSTRING/BINSTRING ELSE (the old behaviour; obj for protocol =2, else BINARY) _Unpickler: IF bytestr=True THEN STRING/SHORT_BINSTRING/BINSTRING are read as bytes ELSE they are read as str (old behaviour) I also extracted the decoding stuff from the three string reading functions to a single one. -- keywords: +patch Added file: http://bugs.python.org/file24568/pickle.py.patch ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue6784 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue6784] byte/unicode pickle incompatibilities between python2 and python3
Merlijn van Deen valhall...@gmail.com added the comment: P.S. (sorry for forgetting this in the original post ;-)) Both ./python -m test -G -v test_pickle and ./python test_bytestrpickle.py pass, but I have not run the entire test suite, as that takes ~90 minutes on my laptop The test script should of course be merged with test_pickle.py at some time -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue6784 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue6784] byte/unicode pickle incompatibilities between python2 and python3
Merlijn van Deen valhall...@gmail.com added the comment: Any news on this? Just as a note, pickletools.py also does not reflect the current behaviour; pickle types STRING, BINSTRING and SHORT_BINSTRING are all defined with stack_after=[pystring]: [1, line 992] I(name='STRING', code='S', arg=stringnl, stack_before=[], stack_after=[pystring], proto=0, doc=(...) ) although the doc=... does describe it will be decoded, the object type of pystring is still defined as bytes: [1, line 747] pystring = StackObject( name='string', obtype=bytes, doc=A Python (8-bit) string object.) [1] http://hg.python.org/cpython/file/98df29d51e12/Lib/pickletools.py -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue6784 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue6784] byte/unicode pickle incompatibilities between python2 and python3
Changes by Merlijn van Deen valhall...@gmail.com: -- nosy: +valhallasw ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue6784 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue11236] getpass.getpass does not respond to ctrl-c or ctrl-z
Merlijn van Deen valhall...@gmail.com added the comment: @orsenthil Close similarity with getpass.c 's behavior had lent some to support to this change in 2.6. Changing now in older codeline has some chances of breaking others code. Someone who has been affected by this change in behavior should provide some insights if back-porting would make sense. Most code will probably have been updated their getpass code with a line like if '\x03' in text: raise KeyboardInterrupt() ( http://www.mediawiki.org/wiki/Special:Code/pywikipedia/8978 ) However, people might have changed their code from try: pass = getpass.getpass() except KeyboardInterrupt: print Ctrl-C! to: pass = getpass.getpass() if \x03' in pass: print Ctrl-C! which will break with this change. The first workaround makes more sense, though, so I suspect very little code will be broken by reverting the ISIG flag. Overall, I think most people are not aware of the removal, either because they still use python 2.5 or because they don't press ctrl-c. They are still in for a surprise if the ISIG flag is not removed (although it will probably stay in the 2.6 branch, anyway?). -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue11236 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue11236] getpass.getpass does not respond to ctrl-c or ctrl-z
Merlijn van Deen valhall...@gmail.com added the comment: No, I do not, for several reasons. First of all, this is not a change *from* previous behaviour, but a change *back to* previous behaviour. And sensible behaviour, too. Secondly, I have tested what getpass does on windows (Python 2.7.1 (r271:86832, Nov 27 2010, 18:30:46) [MSC v.1500 32 bit (Intel)] on win32). There, I get a KeyboardInterrupt when pressing ctrl-c. So for the simple reason of having consistend cross-platform behaviour, either the linux or the windows implementation should change. Thirdly, the reason for adding the ISIG flag was that it is 'a possible security hole' - but no-one actually mentions *what* that hole would be. As Guido notes (in that same thread!): Sorry, but this just doesn't make sense. There are so many differences between C and Python that you can't just compare a C and a Python version of a function and pointing at the differences as possible security holes or bugs.. Lastly, I see no reason to mimic POSIX behaviour per se. Why not use the POSIX getpass function in the first place, then? By the way - even with ISIG on, it should be possible to support KeyboardInterrupts - just read per-character instead of per-line (cf. the windows getpass). [1] http://mail.python.org/pipermail/python-dev/2003-December/040583.html -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue11236 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue11236] getpass.getpass does not respond to ctrl-c or ctrl-z
Merlijn van Deen valhall...@gmail.com added the comment: Sorry, I'm not going to run my patch through the entire test suite, I've got better things to do with my time than setting up a working python-development-test-bench. Especially for a one-line-revert. The result now is: valhallasw@dorthonion:~/src/pythonpatch$ python -c import getpass; print getpass.getpass().__repr__() Password: Traceback (most recent call last): File string, line 1, in module File getpass.py, line 71, in unix_getpass passwd = _raw_input(prompt, stream, input=input) File getpass.py, line 133, in _raw_input line = input.readline() KeyboardInterrupt As for a patch - see attachment. It reverts one change from r76000 http://svn.python.org/view/python/trunk/Lib/getpass.py?r1=74860r2=76000pathrev=76000 line 65 The ISIG flag is copied from getpass.c (http://www.koders.com/c/fid3C9D79A0C31256E7875CB8930CF8B9E49BDA8C12.aspx line 122). According to the r76000 commit message: This change also incorporates some additional getpass implementation suggestions for security based on an analysis of getpass.c linked to from the issue. 'The issue' should probably be issue7208, but I cannot find any reference to getpass.c there. -- keywords: +patch Added file: http://bugs.python.org/file20789/issue11236.patch ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue11236 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue11236] getpass.getpass does not respond to ctrl-c or ctrl-z
Merlijn van Deen valhall...@gmail.com added the comment: Linux-2.6.22.18-co-0.7.4-i686-with-Ubuntu-10.04-lucid (the 'on linux2' versions) and Solaris-2.10-i86pc-i386-32bit-ELF (the 'on sunos5'). -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue11236 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue11236] getpass.getpass does not respond to ctrl-c or ctrl-z
Merlijn van Deen valhall...@gmail.com added the comment:
To allow people to cancel at the password prompt, we added a manual post-check.
Although this check runs after return is pressed, it's better than nothing.
Index: branches/rewrite/pywikibot/userinterfaces/terminal_interface.py
===
--- branches/rewrite/pywikibot/userinterfaces/terminal_interface.py
(revision 8977)
+++ branches/rewrite/pywikibot/userinterfaces/terminal_interface.py
(revision 8978)
@@ -175,6 +175,11 @@
if password:
import getpass
text = getpass.getpass('')
+# See PYWP-13 / http://bugs.python.org/issue11236
+# getpass does not always raise an KeyboardInterrupt when ^C
+# is pressed.
+if '\x03' in text:
+raise KeyboardInterrupt()
else:
text = raw_input()
finally:
http://www.mediawiki.org/wiki/Special:Code/pywikipedia/8978
--
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue11236
___
___
Python-bugs-list mailing list
Unsubscribe:
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue10254] unicodedata.normalize('NFC', s) regression
New submission from Merlijn van Deen valhall...@gmail.com:
Summary: Somewhere between 2.6.5 r79063 and 3.1 r79147 a regression in the
unicode NFC normalization has been introduces. This regression leads to bot
edit wars on wikipedia [1]. It is reproducable with a simple script [2].
Mediawiki/PHP [3] and C# [4] test scripts both show the old behaviour, which
leads me to believe this is a python bug.
A search for older bugs shows bug #1054943 [5] which has commits in the
suspected region.
The regression causes certain NFC-normalized strings to become mangled. Because
of the wide range of unicode strings on wikipedia, this causes several
problems. Details of those can be found at [1].
Example strings include: (these strings have been NFC-normalized by mediawiki)
* u'Li\u030dt-s\u1e73\u0301'
* u'\u092e\u093e\u0930\u094d\u0915
\u091c\u093c\u0941\u0915\u0947\u0930\u092c\u0930\u094d\u0917'
*
u'\u0915\u093f\u0930\u094d\u0917\u093f\u091c\u093c\u0938\u094d\u0924\u093e\u0928'
The bug can be shown simply with
unicodedata.normalize('NFC', s) == s
where s is one of the strings above. This will return True on older python
versions, False on newer versions. There is a script available that does this
[2].
The bug has been tested on the following machines and python versions. OK
indicates the bug is not present, FAIL indicates the bug is present.
Host: SunOS willow 5.10 Generic_142910-17 i86pc i386 i86pc Solaris
'2.3.3 (#1, Dec 16 2004, 14:38:56) [C]' OK
'2.6.5 (r265:79063, Jul 10 2010, 17:50:38) [C]' OK
'2.7 (r27:82500, Aug 5 2010, 04:28:45) [C]' FAIL
'3.1.2 (r312:79147, Sep 24 2010, 05:34:04) [C]' FAIL
Host: Linux nightshade 2.6.26-2-amd64 #1 SMP Thu Sep 16 15:56:38 UTC 2010
x86_64 GNU/Linux
'2.4.6 (#2, Jan 24 2010, 12:20:41) \n[GCC 4.3.2]' OK
'2.5.2 (r252:60911, Jan 24 2010, 17:44:40) \n[GCC 4.3.2]' OK
'2.6.4+ (r264:75706, Feb 16 2010, 05:11:28) \n[GCC 4.4.3]' OK
Host: Linux dorthonion 2.6.22.18-co-0.7.4 #1 PREEMPT Wed Apr 15 18:57:39 UTC
2009 i686 GNU/Linux
'2.5.4 (r254:67916, Jan 20 2010, 21:44:03) \n[GCC 4.3.3]' OK
'2.6.2 (release26-maint, Apr 19 2009, 01:56:41) \n[GCC 4.3.3]' OK
'3.0.1+ (r301:69556, Apr 15 2009, 15:59:22) \n[GCC 4.3.3]' OK
[1]
https://sourceforge.net/tracker/index.php?func=detailaid=3081100group_id=93107atid=603138#
;
http://fr.wikipedia.org/w/index.php?title=Mark_Zuckerbergaction=historysubmitdiff=57753004oldid=57751674
[2] http://pastebin.ca/1977285 (py2.x), http://pastebin.ca/1977287 (py3.x)
[3] http://pastebin.ca/1977292 (PHP, placed in
http://svn.wikimedia.org/svnroot/mediawiki/trunk/phase3/normal/),
[4] http://pastebin.ca/1977261 (C#)
[5] http://bugs.python.org/issue1054943#
--
components: Unicode
messages: 119995
nosy: valhallasw
priority: normal
severity: normal
status: open
title: unicodedata.normalize('NFC', s) regression
type: behavior
versions: Python 2.7, Python 3.1
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue10254
___
___
Python-bugs-list mailing list
Unsubscribe:
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue10254] unicodedata.normalize('NFC', s) regression
Merlijn van Deen valhall...@gmail.com added the comment: Please note: The bug might very well be present in python 3.2 and 3.3. However, I do not have these versions installed, so I cannot confirm this. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue10254 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
