Antoine Pitrou added the comment:
The patch needs a test, a proper doc, and reviewing.
--
nosy: +pitrou
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue18233
___
Changes by Arfrever Frehtes Taifersar Arahesis arfrever@gmail.com:
--
nosy: +Arfrever
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue3982
___
Arfrever Frehtes Taifersar Arahesis added the comment:
Is there any reason why the order of characters matters here?
builtins.open() supports them in any order (br==rb, bw==wb, ba==ab,
bx==xb).
--
nosy: +Arfrever
___
Python tracker
Changes by Arfrever Frehtes Taifersar Arahesis arfrever@gmail.com:
--
nosy: +Arfrever
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue19222
___
Changes by Arfrever Frehtes Taifersar Arahesis arfrever@gmail.com:
--
nosy: +Arfrever
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue19223
___
Arfrever Frehtes Taifersar Arahesis added the comment:
Also tarfile.open() could support x mode.
--
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue19201
___
Antoine Pitrou added the comment:
Sorry for the incorrect answer. I just noticed there was a test in the patch!
Further looking at it, I notice the new function is returning a tuple. Wouldn't
it be better to return a list here?
--
___
Python tracker
Dustin Oprea added the comment:
My two-cents is to leave it a tuple (why not?).
Dustin
--
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue18233
___
Antoine Pitrou added the comment:
Is this something we actually want to support officially? Many other types have
non-repeatable hashes, e.g.:
$ PYTHONHASHSEED=1 python3 -c print(hash((lambda: 0)))
8771754605115
$ PYTHONHASHSEED=1 python3 -c print(hash((lambda: 0)))
8791504743739
$
Antoine Pitrou added the comment:
-1 on such hacks. I much prefer the _abcoll approach.
--
nosy: +pitrou
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue19218
___
STINNER Victor added the comment:
The collections module is loaded by the io module. You removed from
collections.abc import MutableMapping from os.py. To be useful, you have also
to rewrite the whole io module to remove all references to the collections.abc
module, right?
--
nosy:
Antoine Pitrou added the comment:
Why adding ASCII strings, whereas you can add Latin1 (UCS1, U+-U+00FF)
strings?
Reasons:
- most strings in pyc files are pure ASCII (it's like 99% in the stdlib)
- unmarshalling ASCII strings is faster: you can pass 127 to PyUnicode_New
without scanning
STINNER Victor added the comment:
In the same Python version, hash(None) always give me the same value. I cannot
reproduced your issue on Linux, I tested Python 2.7, 3.3 and 3.4.
$ python2.7 -c print(hash(None))
17171842026
$ python2.7 -c print(hash(None))
17171842026
$ python2.7 -c
STINNER Victor added the comment:
It's wired and make difficulty for distributed systems partitioning data
according hash of keys if the system wants the keys support None.
How you handle the randomization of hash(str)? (python2.7 -R, enabled by
default in Python 3.3).
--
STINNER Victor added the comment:
unmarshalling ASCII strings is faster: you can pass 127 to PyUnicode_New
without scanning for non-ASCII chars
Oh, I forgot this pain of the PEP 393. Don't tell me more, it's enough :-)
--
___
Python tracker
Kristján Valur Jónsson added the comment:
I imagine that the test for ASCII is cheaper. It corresponds to the new
compact internal unicode representation (one byte characters).
This looks fine. Can you quantify where the speedup comes from? Reading the
code, I see we now maintain a small
Antoine Pitrou added the comment:
This looks fine. Can you quantify where the speedup comes from?
From all changes, but mainly the ASCII special-casing and the new
buffering.
Reading the code, I see we now maintain a small internal buffer in
the file object, rather than using stack
Antoine Pitrou added the comment:
Reading the code, I see we now maintain a small internal buffer in
the file object, rather than using stack allocation at the call
sites. It is unclear to me how this saves memory, since the amount
of memory copying should be the same.
No, memory
New submission from hiroaki itoh:
http://docs.python.org/2.7/c-api/exceptions.html#standard-exceptions
Python2.7 (at least 2.7.5) has PyExc_BufferError, but the document does not
tell it.
--
assignee: docs@python
components: Documentation
messages: 199458
nosy: docs@python,
Kristján Valur Jónsson added the comment:
Right, in this case, memory copying is avoided.
Regarding the memoing of 0, empty tuple, etc:
Special opcode may still benefit because it takes only one byte. So, you save
four bytes for each such case. I think it might be worth investigating.
hiroaki itoh added the comment:
also:
* GeneratorExit
* StopIteration
* VMSError (#ifdef __VMS)
* UnboundLocalError
* IndentationError
* TabError
* UnicodeError
* UnicodeDecodeError
* UnicodeEncodeError
* UnicodeTranslateError
* Warning;
* UserWarning;
* DeprecationWarning;
*
Martin v. Löwis added the comment:
It's at https://mail.python.org/mailman/listinfo/python-list
--
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue19210
___
Serhiy Storchaka added the comment:
- unmarshalling ASCII strings is faster: you can pass 127 to PyUnicode_New
without scanning for non-ASCII chars
You should ensure that loaded bytes are ASCII-only. Otherwise broken or
malicious marshalled data will compromise you program. Decoding UTF-8
STINNER Victor added the comment:
You should ensure that loaded bytes are ASCII-only. Otherwise broken or
malicious marshalled data will compromise you program.
This is not new, see the red warning in marshal doc:
Warning
The marshal module is not intended to be secure against erroneous or
Kristján Valur Jónsson added the comment:
We have to make two distinctions here:
1) Loading data and then running it. This is a bad idea if your data is not
trusted. This is what is meant by marshal being unsafe.
2) Loading data and then not running it. This is perfectly fine, because
STINNER Victor added the comment:
As for output, we could use cached UTF-8 representation of string (always
exists for ASCII only strings) before calling PyUnicode_AsUTF8String().
PyUnicode_AsEncodedString(v, utf8, surrogatepass) is expensive. I proposed
an optimization for the pickle module,
Michael Foord added the comment:
Ouch. Looking.
--
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue19217
___
___
Python-bugs-list mailing list
Antoine Pitrou added the comment:
In fact, using 'marshal' as a cheap and fast pickler for builtin types
is actually a good idea because it has no side effects like invoking
code.
It's an unsupported use case. The marshal docs are quite clear:
Therefore, the Python maintainers reserve the
STINNER Victor added the comment:
marshal and pickle are unsafe, even without the patch attached to the issue. If
you consider that it is an issue that should be fixed, please open a new issue.
Antoine's patch doesn't make the module less secure, since it was already not
secure :)
Loading
STINNER Victor added the comment:
please discuss it elsewhere.
Hum, I'm not sure that this word exist, I mean: somethere else :-)
--
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue19219
___
Changes by STINNER Victor victor.stin...@gmail.com:
--
nosy: +haypo
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue19217
___
___
Python-bugs-list
Kristján Valur Jónsson added the comment:
Therefore, the Python maintainers reserve the right to modify
the marshal format in backward incompatible ways
sure, don't expect such things to survive version changes. (Actually, they
have been hitherto, and my version 3 I actually changed, to be
Kristján Valur Jónsson added the comment:
Loading untrusted data ... is not supported by Python.
This is a pretty bold claim. Is this, indeed, a fact? (and yes, elsewhere
is a word)
--
___
Python tracker rep...@bugs.python.org
Kristján Valur Jónsson added the comment:
Anyway, whether or not Pyhon guarantees this and that wrt. untrusted is
beside the point and offtopic, as Victor poitns out.
However: We are, and have always been, careful to fail gracefully if we detect
data corruption. Never should the flipping of
Changes by Ronald Oussoren ronaldousso...@mac.com:
--
nosy: +ronaldoussoren
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue19219
___
___
Serhiy Storchaka added the comment:
The marshal module is not intended to be secure against erroneous or
maliciously constructed data. Never unmarshal data received from an untrusted
or unauthenticated source.
Then we can simplify the marshal module by dropping all error handling:
Antoine Pitrou added the comment:
Sorry for sarcasm.
Well, indeed, the sarcasm is undeserved here, if the interpreter cannot
crash because of the change.
It's exactly what you suggest: reuse PyUnicode_AsUTF8String().
Actually _PyUnicode_UTF8(). PyUnicode_AsUTF8String() creates UTF8
Antoine Pitrou added the comment:
That said, I'll try out the patch with _PyUnicode_FromUCS1 instead of
_PyUnicode_FromASCII, to see if it affects performance.
--
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue19219
Kristján Valur Jónsson added the comment:
Just for the record, I want to say that this is great stuff, Antoine! It's
great when this sort of stuff gets some attention.
--
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue19219
Serhiy Storchaka added the comment:
I don't understand how _PyUnicode_UTF8() can be used for *unmarshalling*.
I say about marshalling.
That said, I'll try out the patch with _PyUnicode_FromUCS1 instead of
_PyUnicode_FromASCII, to see if it affects performance.
Could you try out the patch
Kristján Valur Jónsson added the comment:
This will save you two opcodes and perhaps several lines of code.
Just bear in mind that without other changes, version 4 needs to be backwards
compatible with version 3.
I ran into this when developing version 3. The reason is that while the
marshal
Changes by Guido van Rossum gu...@python.org:
--
nosy: -gvanrossum
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue3982
___
___
Python-bugs-list
Changes by Berker Peksag berker.pek...@gmail.com:
--
keywords: +needs review
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue13477
___
___
Antoine Pitrou added the comment:
That said, I'll try out the patch with _PyUnicode_FromUCS1 instead
of _PyUnicode_FromASCII, to see if it affects performance.
Could you try out the patch with PyUnicode_DecodeUTF8()? This will
save you two opcodes and perhaps several lines of code.
That
Antoine Pitrou added the comment:
I ran into this when developing version 3. The reason is that while
the marshal format includes the version information in its header,
it isn't actually verified on loading. IIRC. You specify the
expected format to the function, or something like that.
Kristján Valur Jónsson added the comment:
How about adding a version opcode? This is a backwards compatible change, and
allows us to reject unsupported versions in the future, as long as they are not
very old unsupported versions.
--
___
Python
Serhiy Storchaka added the comment:
This will save you two opcodes and perhaps several lines of code.
Just bear in mind that without other changes, version 4 needs to be backwards
compatible with version 3.
I meant two of new proposed opcodes: TYPE_ASCII and TYPE_ASCII_INTERNED.
Serhiy Storchaka added the comment:
That would be a change in behaviour, since currently surrogatepass
is the error handler.
I don't propose any change in behaviour.
--
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue19219
Antoine Pitrou added the comment:
I meant two of new proposed opcodes: TYPE_ASCII and
TYPE_ASCII_INTERNED.
You cannot change the meaning of TYPE_UNICODE (it uses surrogatepass).
Therefore, you have to use new opcodes with other semantics.
Besides, opcodes are cheap.
--
Serhiy Storchaka added the comment:
You cannot change the meaning of TYPE_UNICODE (it uses surrogatepass).
Therefore, you have to use new opcodes with other semantics.
You don't need new semantic. Use old semantic. The set of ASCII encoded strings
is a subset of valid UTF8 encoded strings,
Antoine Pitrou added the comment:
You cannot change the meaning of TYPE_UNICODE (it uses
surrogatepass).
Therefore, you have to use new opcodes with other semantics.
You don't need new semantic. Use old semantic. The set of ASCII
encoded strings is a subset of valid UTF8 encoded
Roundup Robot added the comment:
New changeset a4515186bf9c by R David Murray in branch 'default':
#19192: Give up on time.xmlrpc.com as an xmlrpc network test.
http://hg.python.org/cpython/rev/a4515186bf9c
--
nosy: +python-dev
___
Python tracker
R. David Murray added the comment:
Thanks, Vajrasky. I simplified your test a bit further still.
It occurs to me that nowadays (unlike when the file was written), we can use
resources to skip individual test classes or even individual tests. So we
could open a new issue to move the last
Changes by Tshepang Lekhonkhobe tshep...@gmail.com:
--
nosy: +tshepang
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue19180
___
___
New submission from W. Trevor King:
Avoid:
Traceback (most recent call last):
File setup.py, line 61, in module
'html2text (=3.0.1)',
File /.../python3.2/distutils/core.py, line 148, in setup
dist.run_commands()
File /.../python3.2/distutils/dist.py, line
Serhiy Storchaka added the comment:
See also http://comments.gmane.org/gmane.comp.python.devel/139953 .
--
nosy: +serhiy.storchaka
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue19180
___
Changes by Giampaolo Rodola' g.rod...@gmail.com:
--
nosy: +giampaolo.rodola
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue15806
___
___
Georg Brandl added the comment:
Is this still an issue?
--
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue5388
___
___
Python-bugs-list mailing
Ned Deily added the comment:
This problem has been reported previously as Issue17354, a duplicate of
Issue12853 which is still open at the moment.
--
nosy: +ned.deily
resolution: - duplicate
stage: - committed/rejected
status: open - closed
superseder: - global name 'r' is not
Changes by Brett Cannon br...@python.org:
--
nosy: +brett.cannon
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue19219
___
___
Python-bugs-list
Changes by Ned Deily n...@acm.org:
--
versions: +Python 3.3, Python 3.4
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue12853
___
___
W. Trevor King added the comment:
On Fri, Oct 11, 2013 at 05:21:15PM +, Ned Deily wrote:
This problem has been reported previously as Issue17354, a duplicate
of Issue12853 which is still open at the moment.
Ah, I searched for a fwe possible subject lines, but didn't hit those
:p. I
W. Trevor King added the comment:
I just posted a patch fixing this in the current master branch [1]. Ned Deily
pointed out that my Issue19226 duplicated an earlier Issue17354, which also has
a patch fixing this problem. Merging either one of these patches should close
this issue. I like
Antoine Pitrou added the comment:
Updated patch using PyUnicode_FromKindAndData for ASCII strings, to ensure that
corrupt marshal bytecode doesn't provide corrupt unicode objects. Performance
is within 2% of the previous patch.
(however, a quick test suggests that PyUnicode_DecodeUTF8 is
Antoine Pitrou added the comment:
From a quick glance, the patch looks ok. Serhiy, do you want to review it any
further?
--
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue13477
___
New submission from Antoine Pitrou:
test_multiprocessing has started to hang frequently on the Gentoo buildbots.
It seems it has started happening with the OpenSSL re-seeding commits:
http://hg.python.org/cpython/rev/8e1194c39beddb83337c0acb9e4c2922a02a36cf
David, could you try to investigate
Terry J. Reedy added the comment:
Yes. On win 7, html help control 6.1, the horizontal scroll bar is still put in
or on top of the box, but it is thinner so it only cuts off half of the bottom
line instead of hiding it completely. There is no longer a vertical scroll bar,
which means one can
Changes by Terry J. Reedy tjre...@udel.edu:
--
versions: +Python 3.3, Python 3.4 -Python 3.1, Python 3.2
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue5388
___
Serhiy Storchaka added the comment:
Doc/whatsnew/3.4.rst
--
nosy: +serhiy.storchaka
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue19221
___
___
Serhiy Storchaka added the comment:
Updated patch addresses Victor's comments on Rietveld. Thank you Victor. The
surrogatepass error handler now works with different spellings of encodings
(utf_32le, UTF-32-LE, etc).
I tested utf_16_32_surrogates_4.patch: surrogateescape with as encoder does
Serhiy Storchaka added the comment:
Yes, this is in my plans.
--
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue13477
___
___
Python-bugs-list
Changes by Christian Heimes li...@cheimes.de:
Removed file: http://bugs.python.org/file32023/sysconfig_delay_re.patch
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue19205
___
Changes by Christian Heimes li...@cheimes.de:
Removed file: http://bugs.python.org/file32017/site_no_re.patch
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue19205
___
Christian Heimes added the comment:
Here is a new patch with unit test and patch for the locale module. The locale
modules is loaded by the _io module when any standard stream is not connected
to a terminal (or so).
--
Added file: http://bugs.python.org/file32048/startup_no_re.patch
Antoine Pitrou added the comment:
Review posted on Rietveld.
--
nosy: +pitrou
stage: needs patch - patch review
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue19205
___
Serhiy Storchaka added the comment:
Let a code say instead me.
--
Added file: http://bugs.python.org/file32049/marshal_opts5a.patch
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue19219
___
Changes by STINNER Victor victor.stin...@gmail.com:
--
nosy: +haypo
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue19227
___
___
Python-bugs-list
Antoine Pitrou added the comment:
Let a code say instead me.
I don't understand your patch. The macros you define aren't used
anywhere.
Also, your patch keeps the slow call to PyUnicode_DecodeUTF8.
--
___
Python tracker rep...@bugs.python.org
STINNER Victor added the comment:
I accept hacks to speedup Python is the site module, but it becomes more
surprising in the locale module. The issue #9548 proposes to a more generic
solution for the locale module at startup.
-CONFIG_LINE =
Christian Heimes added the comment:
The locale module uses only collections.abc.Mapping. The import of the entire
collections module can be avoided if collections.abc is renamed to _abcoll
again. functools.wrap() can be replaced with:
localeconv.__doc__ = _localeconv.__doc__
The other
STINNER Victor added the comment:
(however, a quick test suggests that PyUnicode_DecodeUTF8 is quite slower)
It's surprising that PyUnicode_DecodeUTF8() is quite slower than
_PyUnicode_FromUCS1(). _PyUnicode_FromUCS1() calls ucs1lib_find_max_char() and
then memcpy(). PyUnicode_DecodeUTF8()
R. David Murray added the comment:
Running test_socket test_ssl test_multiprocessing_fork
test_multiprocessing_forkserver test_multiprocessing_spaw under -F on the
buildbot, I got the following failure during the second loop:
[ 10] test_multiprocessing_spawn
Traceback (most recent call last):
STINNER Victor added the comment:
... the _bootlocale can be simplified to a few lines: ...
Here is the patch implementing my proposition: bootlocale3.patch.
--
Added file: http://bugs.python.org/file32050/bootlocale3.patch
___
Python tracker
STINNER Victor added the comment:
Does anyone know if Python does still support systems where CODESET is not
available? Which OS does not support CODESET?
I checked my VMs with Python, nl_langinfo(CODESET) works on:
- Linux (Fedora 18, kernel 3.9)
- OpenBSD 5.2
- OpenIndiana 148 (SunOS 5.11)
Christian Heimes added the comment:
You could raise an error and wait until somebody files a complain. :)
--
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue9548
___
Roundup Robot added the comment:
New changeset fbbf8b160e8d by Antoine Pitrou in branch 'default':
Issue #9548: Add a minimal _bootlocale module that is imported by the _io
module instead of the full locale module.
http://hg.python.org/cpython/rev/fbbf8b160e8d
--
nosy: +python-dev
Roundup Robot added the comment:
New changeset 406529adf156 by Christian Heimes in branch 'default':
Issue #19205: Don't import the 're' module in site and sysconfig module to
http://hg.python.org/cpython/rev/406529adf156
--
nosy: +python-dev
___
Christian Heimes added the comment:
Thanks for your input!
--
resolution: - fixed
stage: patch review - commit review
status: open - closed
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue19205
Roundup Robot added the comment:
New changeset 2cd1b28d1666 by Christian Heimes in branch 'default':
Issue #19205 fix 406529adf156
http://hg.python.org/cpython/rev/2cd1b28d1666
--
___
Python tracker rep...@bugs.python.org
Christian Heimes added the comment:
os_stat_statvfs_pickle.patch with comments and tests.
--
Added file: http://bugs.python.org/file32051/os_stat_statvfs_pickle2.patch
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue19209
New submission from STINNER Victor:
type.__qualname__ getter type_qualname() calls type_name() which strips the
module name from the type name (type-tp_name).
Attached patch fixes this.
--
files: type_qualname.patch
keywords: patch
messages: 199518
nosy: haypo
priority: normal
Antoine Pitrou added the comment:
__qualname__ strips the module name by design. If you want the module name,
look up __name__ on the module.
--
nosy: +pitrou
resolution: - rejected
status: open - closed
___
Python tracker rep...@bugs.python.org
Christian Heimes added the comment:
The io module no longer imports collections.abc through the locale module.
Eric, I'm with Antoine. Your patch is too much of a clever hack and uses tricks
that are dark magic. _abcoll is much simpler and easier to understand. Plus it
can be used from other
Eric Snow added the comment:
Not a problem. It is most definitely a hack. :) I put in up as an alternative
to rearranging the collections module, but agree that doing so is preferable.
I image that Raymond will make a decision on that one.
--
Roundup Robot added the comment:
New changeset 29c4a6a11e76 by Christian Heimes in branch 'default':
Issue #19209: Remove import of copyreg from the os module to speed up
http://hg.python.org/cpython/rev/29c4a6a11e76
--
nosy: +python-dev
___
Python
Christian Heimes added the comment:
Thanks for your help!
Python is down to 43 modules on Linux.
--
resolution: - fixed
stage: patch review - committed/rejected
status: open - closed
___
Python tracker rep...@bugs.python.org
Roundup Robot added the comment:
New changeset 89e405e6a7a9 by Christian Heimes in branch 'default':
Issue #19209: fix structseq test
http://hg.python.org/cpython/rev/89e405e6a7a9
--
___
Python tracker rep...@bugs.python.org
New submission from STINNER Victor:
To speedup Python startup, it may be interesting to not create useless many
functions and classes in operator.py: from _operator import * will remove
them a few line later.
What do you think of moving the Python implementation of the operator module
inside
Terry J. Reedy added the comment:
I agree that this could be improved. It took a few readings to convince myself
that True == Vulnerable. (Assuming that this is correct ;-) the table needs a
title like Vulernable or a lead in sentence: The following table indicates
which modules are
Changes by Terry J. Reedy tjre...@udel.edu:
--
status: open - closed
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue19191
___
___
Python-bugs-list
1 - 100 of 114 matches
Mail list logo
