[issue43323] UnicodeEncodeError: surrogates not allowed when parsing invalid charset
Anders Kaseorg added the comment: It could and does, as quoted in my original report. Content-Type: text/plain; charset*=utf-8”''utf-8%E2%80%9D That’s a U+201D right double quotation mark. This is not a valid charset for the charset of course, but it seems like the code was intended to handle an invalid charset value without crashing, so it should also handle an invalid charset charset (despite the absurdity of the entire concept of a charset charset). -- ___ Python tracker <https://bugs.python.org/issue43323> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue46637] Incorrect error message: "missing 1 required positional argument"
Anders Hovmöller added the comment: For `foo(a, /, b)`, it could be: "TypeError: foo() missing 1 required argument 'a', and one required positional argument 'b'. If we start on this road there are some more, like for `def foo(a, *, b)` you get the error "TypeError: foo() missing 1 required positional argument: 'a'" which leaves out that the keyword only argument is also required. Another solution would be something like: TypeError: foo() missing 3 required arguments: 'a' (positional only), 'b', 'c' (keyword only) This solution scales to the worst complex cases, and is a lot clearer imo. Could even be further improved with some nicer formatting: TypeError: foo() missing 3 required arguments: 'a' (positional only) 'b' 'c' (keyword only) But that might be a bit overkill :) -- ___ Python tracker <https://bugs.python.org/issue46637> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue46637] Incorrect error message: "missing 1 required positional argument"
Anders Hovmöller added the comment: Just dropping the word "positional" is very good. That word is a lie, and just removing it makes it true. -- ___ Python tracker <https://bugs.python.org/issue46637> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue46637] Incorrect error message: "missing 1 required positional argument"
New submission from Anders Hovmöller : >>> def foo(a): ... pass ... >>> foo() Traceback (most recent call last): File "", line 1, in TypeError: foo() missing 1 required positional argument: 'a' This error is incorrect. It says "positional argument", but it's just "argument". The proof is that if you call it with foo(a=3) it works fine. -- components: Interpreter Core messages: 412510 nosy: Anders.Hovmöller priority: normal severity: normal status: open title: Incorrect error message: "missing 1 required positional argument" type: enhancement versions: Python 3.7, Python 3.8, Python 3.9 ___ Python tracker <https://bugs.python.org/issue46637> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue9334] argparse does not accept options taking arguments beginning with dash (regression from optparse)
Anders Kaseorg added the comment: > While optparse that it isn't being developed further, therebut will not > be taken away. IIRC the reason for this was that it too had become > difficult to build out and that is what necessitated the creation of > argparse -- there wasn't clean way to add the desired features > (subparsers, actions, etc). My concern is not that optparse will be taken away. My concern is that the documentation incorrectly discourages its use. https://docs.python.org/3/library/optparse.html “Deprecated since version 3.2: The optparse module is deprecated and will not be developed further; development will continue with the argparse module.” Given that the apparent conclusion of this bug is that argparse has also become too difficult to fix, either argparse should be deprecated for exactly the same reason, or optparse should be un-deprecated. Most programs don’t need the extra features of argparse, and optparse doesn’t have this bug, so optparse is a better default choice; the documentation should not be encouraging argparse over it. -- ___ Python tracker <https://bugs.python.org/issue9334> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue9334] argparse does not accept options taking arguments beginning with dash (regression from optparse)
Anders Kaseorg added the comment: If argparse will not be developed further to fix this bug, then we should undo the deprecation of optparse in the documentation (https://bugs.python.org/issue37103), since the stated justification for that deprecation was that optparse will not be developed further. The documentation should encourage programmers to use correct libraries, and optparse is correct here where argparse is not. People who need the extra features of argparse and aren’t bothered by its incorrectness are welcome to decide to use it, but this is not the right default decision for the documentation to promote. -- ___ Python tracker <https://bugs.python.org/issue9334> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue44967] pydoc should return non-zero exit code when a query is not found
Change by Gregory Anders : -- keywords: +patch pull_requests: +26323 stage: -> patch review pull_request: https://github.com/python/cpython/pull/27868 ___ Python tracker <https://bugs.python.org/issue44967> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue44967] pydoc should return non-zero exit code when a query is not found
New submission from Gregory Anders : Currently pydoc returns an exit code of zero no matter what, even with e.g. pydoc lsjdfkdfj However, the ability to know whether or not pydoc successfully found a result is useful in tools that embed pydoc in some way. Here's one use case: Vim and Neovim have a feature that allows a user to run an external command for the keyword under the cursor (keywordprg). In Python files, this defaults to pydoc. In Neovim, we would like to automatically close the PTY buffers that we create for these processes when they finish without any errors, but if it returns a non-zero exit code we want to keep the PTY buffer open so the user can see what went wrong. Because pydoc returns immediately when it fails to find a match and does not indicate that it failed via a return code, the PTY buffer is closed immediately with no indication to the user that anything went wrong. I have a patch prepared for this that I will link to the issue. -- components: Demos and Tools messages: 400012 nosy: gpanders priority: normal severity: normal status: open title: pydoc should return non-zero exit code when a query is not found type: enhancement ___ Python tracker <https://bugs.python.org/issue44967> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue44680] Reference cycles from a WeakKeyDictionary value to its key aren’t collected
Anders Kaseorg added the comment: > extra_states[o] = ExtraState(obj) (Typo for extra_states[obj] = ExtraState(obj), obviously.) -- ___ Python tracker <https://bugs.python.org/issue44680> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue44680] Reference cycles from a WeakKeyDictionary value to its key aren’t collected
New submission from Anders Kaseorg :
Because WeakKeyDictionary unconditionally maintains strong references to its
values, the garbage collector fails to collect a reference cycle from a
WeakKeyDictionary value to its key. For example, the following program
unexpectedly leaks memory:
from weakref import WeakKeyDictionary
class C: pass
d = WeakKeyDictionary()
while True:
c = C()
d[c] = [c]
I would expect a WeakKeyDictionary value to be marked live _if_ its key is
marked live, not unconditionally. This could be implemented with garbage
collector support for ephemerons
(https://www.researchgate.net/publication/221320677_Ephemerons_A_New_Finalization_Mechanism).
To motivate this issue, a typical use of WeakKeyDictionary is as a hygienic
replacement for patching extra properties into third-party objects:
# before:
obj._extra_state = ExtraState(obj)
# after:
extra_states = WeakKeyDictionary()
extra_states[o] = ExtraState(obj)
However, such a conversion will introduce this memory leak if ExtraState(obj)
has any transitive references to obj.
This leak does not occur in JavaScript:
class C {}
const d = new WeakMap();
while (true) {
const c = new C();
d[c] = [c];
}
--
components: Library (Lib)
messages: 397841
nosy: andersk
priority: normal
severity: normal
status: open
title: Reference cycles from a WeakKeyDictionary value to its key aren’t
collected
type: resource usage
versions: Python 3.9
___
Python tracker
<https://bugs.python.org/issue44680>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue44283] Add jump table for certain safe match-case statements
Anders Munch added the comment: Are you sure you want to do this? This optimisation is not applicable if the matched values are given symbolic names. You would be encouraging people to write bad code with lots of literals, for speed. -- nosy: +AndersMunch ___ Python tracker <https://bugs.python.org/issue44283> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue43323] UnicodeEncodeError: surrogates not allowed when parsing invalid charset
New submission from Anders Kaseorg :
We ran into a UnicodeEncodeError exception using email.parser to parse this
email
<https://lists.cam.ac.uk/pipermail/cl-isabelle-users/2021-February/msg00135.html>,
with full headers available in the raw archive
<https://lists.cam.ac.uk/pipermail/cl-isabelle-users/2021-February.txt>. The
offending header is hilariously invalid:
Content-Type: text/plain; charset*=utf-8”''utf-8%E2%80%9D
but I’m filing an issue since the parser is intended to be robust against
invalid input. Minimal reproduction:
>>> import email, email.policy
>>> email.message_from_bytes(b"Content-Type: text/plain;
>>> charset*=utf-8\xE2\x80\x9D''utf-8%E2%80%9D", policy=email.policy.default)
Traceback (most recent call last):
File "", line 1, in
File "/usr/local/lib/python3.10/email/__init__.py", line 46, in
message_from_bytes
return BytesParser(*args, **kws).parsebytes(s)
File "/usr/local/lib/python3.10/email/parser.py", line 123, in parsebytes
return self.parser.parsestr(text, headersonly)
File "/usr/local/lib/python3.10/email/parser.py", line 67, in parsestr
return self.parse(StringIO(text), headersonly=headersonly)
File "/usr/local/lib/python3.10/email/parser.py", line 57, in parse
return feedparser.close()
File "/usr/local/lib/python3.10/email/feedparser.py", line 187, in close
self._call_parse()
File "/usr/local/lib/python3.10/email/feedparser.py", line 180, in _call_parse
self._parse()
File "/usr/local/lib/python3.10/email/feedparser.py", line 256, in _parsegen
if self._cur.get_content_type() == 'message/delivery-status':
File "/usr/local/lib/python3.10/email/message.py", line 578, in
get_content_type
value = self.get('content-type', missing)
File "/usr/local/lib/python3.10/email/message.py", line 471, in get
return self.policy.header_fetch_parse(k, v)
File "/usr/local/lib/python3.10/email/policy.py", line 163, in
header_fetch_parse
return self.header_factory(name, value)
File "/usr/local/lib/python3.10/email/headerregistry.py", line 608, in
__call__
return self[name](name, value)
File "/usr/local/lib/python3.10/email/headerregistry.py", line 196, in __new__
cls.parse(value, kwds)
File "/usr/local/lib/python3.10/email/headerregistry.py", line 453, in parse
kwds['decoded'] = str(parse_tree)
File "/usr/local/lib/python3.10/email/_header_value_parser.py", line 126, in
__str__
return ''.join(str(x) for x in self)
File "/usr/local/lib/python3.10/email/_header_value_parser.py", line 126, in
return ''.join(str(x) for x in self)
File "/usr/local/lib/python3.10/email/_header_value_parser.py", line 798, in
__str__
for name, value in self.params:
File "/usr/local/lib/python3.10/email/_header_value_parser.py", line 783, in
params
value = value.decode(charset, 'surrogateescape')
UnicodeEncodeError: 'utf-8' codec can't encode characters in position 5-7:
surrogates not allowed
--
components: email
messages: 387685
nosy: andersk, barry, r.david.murray
priority: normal
severity: normal
status: open
title: UnicodeEncodeError: surrogates not allowed when parsing invalid charset
versions: Python 3.10, Python 3.6, Python 3.7, Python 3.8, Python 3.9
___
Python tracker
<https://bugs.python.org/issue43323>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue43115] locale.getlocale fails if locale is set
Anders Munch added the comment: >> What does use getlocale is time.strptime and datetime.datetime.strptime, so >> when getlocale fails, strptime fails. > Would they work with getlocale() returning None for the encoding ? Yes. All getlocale is used for in _strptime.py is comparing the value returned to the previous value returned. I expect changing _strptime._getlang to return the unnormalized value locale._setlocale(locale.LC_TIME, None) would work as well and more robustly. -- ___ Python tracker <https://bugs.python.org/issue43115> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue43115] locale.getlocale fails if locale is set
Anders Munch added the comment: > BTW: What is wxWidgets doing with the returned values ? wxWidgets doesn't call getlocale, it's a C++ library (wrapped by wxPython) that uses C setlocale. What does use getlocale is time.strptime and datetime.datetime.strptime, so when getlocale fails, strptime fails. > We could enhance this to return None for the encoding instead > of raising an exception, but would this really help ? Very much so. Frankly, I don't get the impression that the current locale preferred encoding is used for *anything*. Other than possibly having a role in implementing getpreferredencoding. > Alternatively, we could add "en_DE" to the alias table and set > a default encoding to use. Where would you get a complete list of all the new aliases that would need be to be added? As the "en_DE" example shows, you'd need all combinations of languages and regions. That's going to be a very long list. -- ___ Python tracker <https://bugs.python.org/issue43115> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue43115] locale.getlocale fails if locale is set
Anders Munch added the comment: getlocale is documented to return None for the encoding if no encoding can be determined. There's no need to guess. I can't change the locale.setlocale call, because where I'm actually having the problem, I'm not even calling locale.setlocale: wxWidgets is calling C setlocale, that's where it comes from. wxWidgets aside, it doesn't seem right that getlocale fails when setlocale succeeded. -- ___ Python tracker <https://bugs.python.org/issue43115> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue43115] locale.getlocale fails if locale is set
Anders Munch added the comment:
I discovered that this can happen with underscores as well:
Python 3.8.7 (tags/v3.8.7:6503f05, Dec 21 2020, 17:59:51) [MSC v.1928 64 bit
(AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import locale
>>> locale.setlocale(locale.LC_ALL, 'en_DE')
'en_DE'
>>> locale.getlocale()
Traceback (most recent call last):
File "", line 1, in
File "C:\flonidan\env\Python38-64\lib\locale.py", line 591, in getlocale
return _parse_localename(localename)
File "C:\flonidan\env\Python38-64\lib\locale.py", line 499, in
_parse_localename
raise ValueError('unknown locale: %s' % localename)
ValueError: unknown locale: en_DE
locale.setlocale does validate input - if you write nonsense in the second
argument then you get an exception, "locale.Error: unsupported locale setting".
So I'm guessing the en_DE locale is actually being set here, and the problem
is solely about getlocale making unfounded assumptions about the format.
Same thing happens in 3.10.0a4.
--
___
Python tracker
<https://bugs.python.org/issue43115>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue43115] locale.getlocale fails if locale is set
New submission from Anders Munch :
getlocale fails with an exception when the string returned by _setlocale is
already an RFC 1766 language tag.
Example:
Python 3.10.0a4 (tags/v3.10.0a4:445f7f5, Jan 4 2021, 19:55:53) [MSC v.1928 64
bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import locale
>>> locale.setlocale(locale.LC_ALL, 'en-US')
'en-US'
>>> locale.getlocale()
Traceback (most recent call last):
File "", line 1, in
File "C:\flonidan\env\Python310\lib\locale.py", line 593, in getlocale
return _parse_localename(localename)
File "C:\flonidan\env\Python310\lib\locale.py", line 501, in _parse_localename
raise ValueError('unknown locale: %s' % localename)
ValueError: unknown locale: en-US
Expected result:
('en-US', None)
See https://github.com/wxWidgets/Phoenix/issues/1637 for an example of the
ensuing problems. wx.Locale calls C setlocale directly, but, as far as I can
see, correctly, using dashes in the language code which is consistent with not
only RFC 1766 but also the examples in Microsoft's documentation
(https://docs.microsoft.com/en-us/cpp/c-runtime-library/reference/setlocale-wsetlocale?view=msvc-160).
CPython seems to assume underscored names such as 'en_US' instead, as shown
by getdefaultlocale inserting an underscore.
--
components: Library (Lib), Windows
messages: 386203
nosy: AndersMunch, paul.moore, steve.dower, tim.golden, zach.ware
priority: normal
severity: normal
status: open
title: locale.getlocale fails if locale is set
versions: Python 3.10, Python 3.8, Python 3.9
___
Python tracker
<https://bugs.python.org/issue43115>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue37909] Thread pool return ref hold memory
Anders added the comment:
Note: due to a change in Python 3.8 this example would be a lot less noticeable
if tested. The problem remains the same though.
If you run this snippet with Python 3.7, which is before the thread reuse was
introduced into the ThreadPoolExecutor, each thread will keep around 600mb of
memory in use.
This can be solved by shutting down the ThreadPoolExecutor which this example
does.
Now, the big problem is that asyncio uses a long-running ThreadPoolExecutor,
per default, for run_in_executor. Those threads will stay around forever and
consume memory until the application is shut down.
If you have a job that consumes a lot of memory for a short period of time and
use any long-running ThreadPoolExecutor then the memory will just keep growing
as the job hits various threads that are never cleaned up.
--
import asyncio
import concurrent
import threading
def prepare_a_giant_list():
d = {}
for i in range(1000):
d[i] = {}
for j in range(1000):
d[i][j] = {}
for k in range(30):
d[i][j][k] = 'a' * 1000
del d
th_num = threading.active_count()
print("Thread number is {}".format(th_num))
async def main():
loop = asyncio.get_running_loop()
with concurrent.futures.ThreadPoolExecutor(max_workers=20) as
async_executor:
await loop.run_in_executor(async_executor, prepare_a_giant_list)
await asyncio.sleep(5)
await loop.run_in_executor(async_executor, prepare_a_giant_list)
await asyncio.sleep(5)
await loop.run_in_executor(async_executor, prepare_a_giant_list)
await asyncio.sleep(5)
await loop.run_in_executor(async_executor, prepare_a_giant_list)
await asyncio.sleep(5)
print('Done!')
await asyncio.sleep(15)
if __name__ == "__main__":
asyncio.run(main())
--
--
nosy: +johndoee
versions: +Python 3.6, Python 3.7
___
Python tracker
<https://bugs.python.org/issue37909>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue32308] Replace empty matches adjacent to a previous non-empty match in re.sub()
Anders Hovmöller added the comment: We were also bitten by this. In fact we still run a compatibility shim in production where we log if the new and old behavior are different. We also didn't think this "bug fix" made sense or was treated with the appropriate gravity in the release notes. I understand the logic in the bug tracker and they it matches other languages is good. But the bahvior also makes no sense for the .* case unfortunately. > On 21 Jan 2020, at 05:56, David Barnett wrote: > > > David Barnett added the comment: > > We were also bitten by this behavior change in > https://github.com/google/vroom/issues/110. I'm kinda baffled by the new > behavior and assumed it had to be an accidental regression, but I guess not. > If you have any other context on the BDFL conversation and reasoning for > calling this behavior correct, I'd love to see additional info. > > -- > nosy: +mu_mind > > ___ > Python tracker > <https://bugs.python.org/issue32308> > ___ -- ___ Python tracker <https://bugs.python.org/issue32308> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue38673] REPL shows continuation prompt (...) when comment or space entered
Anders Lorentsen added the comment: As a person without much experience, it sounded like a simple enough task, but having dug a bit, I found it quite complicated. It seems to me that the interpreter loop (in the standard REPL, that you get when you start ./python, blocks for input somewhere inside a massive function called 'parsetok' (in Parser/parsetok.c). Now, I could maybe investigate further, to have it return to the interpreter loop if it reads a comment (or empty line), but I'm afraid to mess up something. >From my understanding, there aren't that many other choices, because >parsetok() doesn't return before you finish the statement (in other words, it >does not return if you type a comment line or a blank line - it instead waits >for more input, as indicated by the '... '). Am I way off in concluding that this would be a change to the parser? -- nosy: +Phaqui ___ Python tracker <https://bugs.python.org/issue38673> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue31956] Add start and stop parameters to the array.index()
Anders Lorentsen added the comment: I have actually managed to lost my local branch of this fix, though I assume I can just start another one, manually copy over the changes, somehow mark this current PR as cancelled, aborted, or in my option the best: "replaced/superseeded by: [new PR]". In any case, there were discussions that seem to be unresolved, allow me to summarize: * Document that index() raises ValueError when *value* is not found? > vstinner: We don't do this, remove this addition. > serhiy: Other index() methods does this. ---> My patch current does this. Who has final saying here? * 'start' and 'stop' arguments are not keyword arguments, and also not shown in the signature as '.. start=0 ..' for this very reason (it may make them look as keyword arguments). Also, this lines up with list.index() for consistency. Wishes about changing this for all index()-methods has been expressed, but it seems to be consensus on doing this in unison for all index()-methods at once, in a bigger change... So, what is currently in the PR is good enough for now, or? * Wording in documentation: Clarify that "the returned index is still relative to the start of the array, not the searched sub sequence" or not? * Comment in the code about checking the length of the array on each iteration? There were comments about it being "confusing" - and while I agree, the other index()-code for lists, does not comment on this. Again I followed the path of most consistency, but I did receive comments about this. Yes to descriptive comments, or not? Generally speaking: In the end, all I really did was mimic how list.index() is both written and documented, and that's when discussions about issues related to that started occurring, and so I now remember that I halted my PR, waiting for these issues to be resolved. -- ___ Python tracker <https://bugs.python.org/issue31956> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue31956] Add start and stop parameters to the array.index()
Anders Lorentsen added the comment: As far as I can recall, the patch is generally speaking good to go. A number of discussions arose on various details, however. In any event, I'll take a look at it during the next few days. -- ___ Python tracker <https://bugs.python.org/issue31956> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue37103] Undo deprecation of optparse
New submission from Anders Kaseorg : The optparse library is currently marked in the documentation as deprecated in favor of argparse. However, argparse uses a nonstandard reinterpretation of Unix command line grammars that makes certain arguments impossible to express, and causes scripts to break when ported from optparse to argparse. See the bug report I filed nine years ago: https://bugs.python.org/issue9334 argparse does not accept options taking arguments beginning with dash (regression from optparse) The conclusion of the core developers (e.g. msg309691) seems to have been that although it’s a valid bug, it can’t or won’t be fixed with the current argparse architecture. I was asked by another core developer to file a bug report for the de-deprecation of optparse (https://discuss.python.org/t/pep-594-removing-dead-batteries-from-the-standard-library/1704/20), so here it is. -- assignee: docs@python components: Documentation, Library (Lib) messages: 343997 nosy: andersk, docs@python priority: normal severity: normal status: open title: Undo deprecation of optparse versions: Python 3.8 ___ Python tracker <https://bugs.python.org/issue37103> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue32308] Replace empty matches adjacent to a previous non-empty match in re.sub()
Anders Hovmöller added the comment: That might be true, but that seems like a weak argument. If anything, it means those others are broken. What is the logic behind "(.*)" returning the entire string (which is what you asked for) and exactly one empty string? Why not two empty strings? 3? 4? 5? Why not an empty string at the beginning? It makes no practical sense. We will have to spend considerable effort to work around this change and adapt our code to 3.7. The lack of a discussion about backwards compatibility in this, and the other, thread before making this change is also a problem I think. -- ___ Python tracker <https://bugs.python.org/issue32308> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue32308] Replace empty matches adjacent to a previous non-empty match in re.sub()
Anders Hovmöller added the comment: Just as a comparison, sed does the 3.6 thing: > echo foo | sed 's/\(.*\)/x\1y/g' xfooy -- ___ Python tracker <https://bugs.python.org/issue32308> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue32308] Replace empty matches adjacent to a previous non-empty match in re.sub()
Anders Hovmöller added the comment:
This was a really bad idea in my opinion. We just found this and we have no way
to know how this will impact production. It's really absurd that
re.sub('(.*)', r'foo', 'asd')
is "foo" in python 1 to 3.6 but 'foofoo' in python 3.7.
--
nosy: +Anders.Hovmöller
___
Python tracker
<https://bugs.python.org/issue32308>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue36485] Add a way to clear all caches
Anders Hovmöller added the comment: I think this is a great idea. We would have needed this many times for tests over the years. -- nosy: +Anders.Hovmöller ___ Python tracker <https://bugs.python.org/issue36485> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue22490] Using realpath for __PYVENV_LAUNCHER__ makes Homebrew installs fragile
Anders Hovmöller added the comment: I just discovered this ticket again and see that it's stuck! I have read through the thread but it's still a bit unclear what would be required to test this with homebrew like Guido says is needed for this to go forward. Is there anyone who can explain or better yet test? -- ___ Python tracker <https://bugs.python.org/issue22490> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue9334] argparse does not accept options taking arguments beginning with dash (regression from optparse)
Anders Kaseorg added the comment: porton: Please don’t steal someone else’s issue to report a different bug. Open a new issue instead. -- title: argparse: add a full fledged parser as a subparser -> argparse does not accept options taking arguments beginning with dash (regression from optparse) ___ Python tracker <https://bugs.python.org/issue9334> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue34861] Improve cProfile standard output
Anders Hovmöller added the comment: Output before this patch: 3666 function calls (3556 primitive calls) in 0.005 seconds Ordered by: standard name ncalls tottime percall cumtime percall filename:lineno(function) 20.0000.0000.0020.001 :1009(_handle_fromlist) 70.0000.0000.0000.000 :103(release) 50.0000.0000.0000.000 :143(__init__) 50.0000.0000.0000.000 :147(__enter__) 50.0000.0000.0000.000 :151(__exit__) 70.0000.0000.0000.000 :157(_get_module_lock) 50.0000.0000.0000.000 :176(cb) 20.0000.0000.0000.000 :194(_lock_unlock_module) 7/10.0000.0000.0030.003 :211(_call_with_frames_removed) 530.0000.0000.0000.000 :222(_verbose_message) 50.0000.0000.0000.000 :307(__init__) 50.0000.0000.0000.000 :311(__enter__) 50.0000.0000.0000.000 :318(__exit__) 200.0000.0000.0000.000 :321() 40.0000.0000.0000.000 :35(_new_module) 50.0000.0000.0000.000 :369(__init__) 90.0000.0000.0000.000 :403(cached) 70.0000.0000.0000.000 :416(parent) 50.0000.0000.0000.000 :424(has_location) 50.0000.0000.0000.000 :504(_init_module_attrs) 50.0000.0000.0000.000 :576(module_from_spec) 50.0000.0000.0000.000 :58(__init__) 5/10.0000.0000.0040.004 :663(_load_unlocked) 50.0000.0000.0000.000 :719(find_spec) 70.0000.0000.0000.000 :78(acquire) 50.0000.0000.0000.000 :792(find_spec) 150.0000.0000.0000.000 :855(__enter__) 150.0000.0000.0000.000 :859(__exit__) 50.0000.0000.0010.000 :882(_find_spec) 5/10.0000.0000.0040.004 :948(_find_and_load_unlocked) 5/10.0000.0000.0040.004 :978(_find_and_load) 10.0000.0000.0000.000 :1072(__init__) 10.0000.0000.0000.000 :1083(create_module) 10.0000.0000.0000.000 :1091(exec_module) 10.0000.0000.0000.000 :1233(_path_hooks) 120.0000.0000.0000.000 :1246(_path_importer_cache) 50.0000.0000.0010.000 :1283(_get_spec) 50.0000.0000.0010.000 :1315(find_spec) 10.0000.0000.0000.000 :1362(__init__) 80.0000.0000.0000.000 :1368() 50.0000.0000.0000.000 :1394(_get_spec) 100.0000.0000.0010.000 :1399(find_spec) 10.0000.0000.0000.000 :1447(_fill_cache) 10.0000.0000.0000.000 :1476() 10.0000.0000.0000.000 :1488(path_hook_for_FileFinder) 80.0000.0000.0000.000 :282(cache_from_source) 100.0000.0000.0000.000 :36(_relax_case) 50.0000.0000.0000.000 :412(_get_cached) 40.0000.0000.0000.000 :444(_check_name_wrapper) 40.0000.0000.0000.000 :481(_classify_pyc) 120.0000.0000.0000.000 :51(_r_long) 40.0000.0000.0000.000 :514(_validate_timestamp_pyc) 510.0000.0000.0000.000 :56(_path_join) 40.0000.0000.0000.000 :566(_compile_bytecode) 510.0000.0000.0000.000 :58() 50.0000.0000.0000.000 :617(spec_from_file_location) 80.0000.0000.0000.000 :62(_path_split) 230.0000.0000.0000.000 :74(_path_stat) 40.0000.0000.0000.000 :762(create_module) 4/10.0000.0000.0040.004 :765(exec_module) 40.0000.0000.0010.000 :836(get_code) 90.0000.0000.0000.000 :84(_path_is_mode_type) 40.0000.0000.0000.000 :927(__init__) 80.0000.0000.0000.000 :93(_path_isfile) 40.0000.0000.0000.000 :952(get_filename) 40.0000.0000.0000.000 :957(get_data) 10.0000.0000.0000.000 :98(_path_isdir) 40.0000.0000.0000.000 :994(path_stats) 1000.0000.0000.0010.000 __init__.py:183(dumps) 10.0000.0000.0030.003 __init__.py:97() 10.0000.0000.0020.002 decoder.py:2() 10.0000.0000.0000.000 decoder.py:20(JSONDecodeError) 10.0000.0000.0000.000 decoder.py:254(JSONDecoder) 10.0000.0000.0000.000
[issue34861] Improve cProfile standard output
Anders Hovmöller added the comment: There is an example output on github. Should I paste it here too? I can do it once I get home if you want. -- ___ Python tracker <https://bugs.python.org/issue34861> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue34861] Improve cProfile standard output
Change by Anders Hovmöller : -- keywords: +patch pull_requests: +9046 stage: -> patch review ___ Python tracker <https://bugs.python.org/issue34861> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue34861] Improve cProfile standard output
New submission from Anders Hovmöller : The standard output for cProfile when run from a command line is not very useful. It has two main flaws: - Default sort order is by name of function - It strips the full path of source files The first makes it very hard to look at the output. The second is very annoying when you get a bunch of __init__.py in the output. Suggested solution: - Default cumulative time sort order - Show one additional folder level when filename is __main__ or __init__ -- components: Library (Lib) messages: 326793 nosy: Anders.Hovmöller priority: normal severity: normal status: open title: Improve cProfile standard output type: enhancement versions: Python 2.7, Python 3.4, Python 3.5, Python 3.6, Python 3.7, Python 3.8 ___ Python tracker <https://bugs.python.org/issue34861> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue33143] encode UTF-16 generates unexpected results
Anders Rundgren added the comment: Thanx for the superquick response! I really appreciate it. I'm obviously a Python n00b Anders -- ___ Python tracker <https://bugs.python.org/issue33143> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue33143] encode UTF-16 generates unexpected results
New submission from Anders Rundgren :
Python 3.5.1 (v3.5.1:37a07cee5969, Dec 6 2015, 01:54:25) [MSC v.1900 64 bit
(AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> v = '\u20ac'
>>> print (v)
€
>>> v.encode('utf-16')
b'\xff\xfe\xac '
>>> v.encode('utf-16_be')
b' \xac'
I had expected to get pair of bytes with 20 AC for the € symbol
--
components: Unicode
messages: 314443
nosy: anders.rundgren@gmail.com, ezio.melotti, vstinner
priority: normal
severity: normal
status: open
title: encode UTF-16 generates unexpected results
type: behavior
versions: Python 3.5
___
Python tracker
<https://bugs.python.org/issue33143>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue31961] subprocess._execute_child doesn't accept a single PathLike argument for args
Anders Lorentsen added the comment: What do you mean "is a bug", and "the PR would encourage this"? Can't it be fixed? Are you saying that just because it is a bug now, we should be discouraged from making it work in the way you'd expect it to work? If `exe` is a path, these two lines of code should do exactly the same, right? It seems unintuitive to me if they produce different results. run(exe) run([exe]) -- ___ Python tracker <https://bugs.python.org/issue31961> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue31961] subprocess._execute_child doesn't accept a single PathLike argument for args
Anders Lorentsen added the comment: > # runs this weird file > subprocess.run([bin]) > # Currently an error; if this is implemented, would run > # /bin/ls, and pass it the -l argument. Refers to something > # completely different than our .exists() call above. I do not understand where it is stated that this is the behavior. My understanding was that if run() is passed a single argument, it will interpret the string "/bin/ls -l" to mean "run the `/bin/ls` program, and pass it one argument `-l`, whereas if instead run() is given a list, it will literally treat the first element as the program to run, regardless of how many spaces or whatever else it contains ... And as such, if `bin` is a Path-like object, this issue tries to resolve that run([bin]) works as excepted, but run(bin) fails horribly. Sure, I can understand that for strings it may not feel natural how these two things are interpreted differently, but if `bin` is a Path-like object, it at least feels very natural to me that then it should be run as the program name itself (as "paths" does not have arguments). -- ___ Python tracker <https://bugs.python.org/issue31961> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue32764] Popen doesn't work on Windows when args is a list
Anders Lorentsen added the comment: Wait a minute. The failing test is test_nonexisting_with_pipes, and it fails because args[0] is a tuple - how can that be? Nobody is supposed to pass cmd=sequence-where-first-element-is-a-tuple! Is everything all right with the test itself? -- ___ Python tracker <https://bugs.python.org/issue32764> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue32764] Popen doesn't work on Windows when args is a list
Anders Lorentsen added the comment: Also, isn't there continuous integration testing? Everything passed on the PR, so where does this come from? -- ___ Python tracker <https://bugs.python.org/issue32764> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue32764] Popen doesn't work on Windows when args is a list
Anders Lorentsen added the comment: This is strange, because _execute_child calls os.fsdecode with `args` as the argument, which may be a list. os.fsdecode calls fspath. Now, the python docstring of _fspath, as defined in Lib/os.py on line 1031, clearly states that it will raise a TypeError if the argument is not of type bytes, str or is a os.PathLike object, and that's probably why I wrote the initial code the way I did (catching TypeError from os.fsdecode). Doesn't the try-except block actually catch this TypeError? I don't understand off the top of my head why my code doesn't catch this exception.. -- ___ Python tracker <https://bugs.python.org/issue32764> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue32601] PosixPathTest.test_expanduser fails in NixOS build sandbox
New submission from Anders Kaseorg :
PosixPathTest.test_expanduser fails in the NixOS build sandbox, where every
user has home directory /, so it falls off the end of the for pwdent in
pwd.getpwall() loop.
nixbld:x:30001:3:Nix build user:/:/noshell
nobody:x:65534:65534:Nobody:/:/noshell
==
FAIL: test_expanduser (__main__.PosixPathTest)
--
Traceback (most recent call last):
File "/nix/store/mdak9gcy16dc536ws08rshyakd1l7srj-test_pathlib.py", line
2162, in test_expanduser
self.assertEqual(p3.expanduser(), P(otherhome) / 'Documents')
AssertionError: PosixPath('/Documents') != PosixPath('Documents')
--
components: Tests
messages: 310282
nosy: andersk
priority: normal
pull_requests: 5091
severity: normal
status: open
title: PosixPathTest.test_expanduser fails in NixOS build sandbox
type: behavior
versions: Python 3.7
___
Python tracker
<https://bugs.python.org/issue32601>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue22490] Using realpath for __PYVENV_LAUNCHER__ makes Homebrew installs fragile
Change by Anders Hovmöller : -- nosy: +Anders.Hovmöller ___ Python tracker <https://bugs.python.org/issue22490> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue22490] Using realpath for __PYVENV_LAUNCHER__ makes Homebrew installs fragile
Change by Anders Hovmöller : -- versions: +Python 3.6 ___ Python tracker <https://bugs.python.org/issue22490> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue31956] Add start and stop parameters to the array.index()
Anders Lorentsen added the comment: Writing my tests, I originally looked at Lib/test/seq_tests.py. One test case uses indexes that are (+-)4*sys.maxsize. This does not fit in Py_ssize_t, and so these tests cause my array implementation to raise an overflow exception. A solution is of course to have the function take general objects instead, and then truncate them down to a number that can fit in Py_ssize_t if it's too negative or positive). But I concur. It seems more reasonable to stay consistent with the rest of the module, too. I'll look over the test code to make sure I test for every given scenario (or as many as I can think of), and prepare a PR for this, then :) -- ___ Python tracker <https://bugs.python.org/issue31956> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue31956] Add start and stop parameters to the array.index()
Anders Lorentsen added the comment: I decided to work on this, and I would like some review, as this would be my second contribution to cpython. Also, a general question: As I defined the start and end arguments Py_ssize_t, bigger indexes (more negative or more positive) than what can fit in Py_ssize_t will trigger an overflow error. This should be OK, though, as other functions with index arguments has them as Py_ssize_t - and getarrayitem() itself accepts a Py_ssize_t. Or? -- nosy: +Phaqui Added file: https://bugs.python.org/file47261/WIP-issue-31956 ___ Python tracker <https://bugs.python.org/issue31956> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue31961] subprocess._execute_child doesn't accept a single PathLike argument for args
Anders Lorentsen added the comment: While researching this, I discovered that on MS Windows >>> subprocess.run([pathlike_object, additional_arguments]) did not run like it did on Posix. My PR includes this problem and it's fix. -- ___ Python tracker <https://bugs.python.org/issue31961> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue31961] subprocess._execute_child doesn't accept a single PathLike argument for args
Anders Lorentsen added the comment: I was able to make a test that reproduces your code, and expectedly fails. Also implemented a fix for it. See a temporary diff here: https://pastebin.com/C9JWkg0i However, there is also a specific MS Windows version of _execute_child() (a phrase only computer-folks can say out loud without feeling very...wrong...). It uses a different method to extract and parse arguments, and it should be corrected and tested under windows, before I submit a PR for this. Also, my test messes up the formatting. Instead of saying "ok", they both say "... total 0\nok". I have no idea why. -- nosy: +Phaqui ___ Python tracker <https://bugs.python.org/issue31961> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue31843] sqlite3.connect() should accept PathLike objects
Anders Lorentsen added the comment: Had my first go at a python patch. Added a test case for it, and all tests passing when I test with `./python -bb -E -Wd -m test -v test.test_sqlite -r -w -uall -R 3:2` -- nosy: +Phaqui ___ Python tracker <https://bugs.python.org/issue31843> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue15873] datetime: add ability to parse RFC 3339 dates and times
Anders Hovmöller added the comment: @larsonreever That lib is pretty limited, in that it doesn't handle dates or deltas. Again: my lib that is linked above does and has comprehensive tests. -- ___ Python tracker <http://bugs.python.org/issue15873> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue8376] Tutorial offers dangerous advice about iterators: “__iter__() can just return self”
Anders Kaseorg added the comment: Usui, this is a tutorial intended for beginners. Even if the change from “most” to “built-in” were a relevant one (and I don’t see how it is), beginners cannot possibly be expected to parse that level of meaning out of a single word. The difference between iterators and containers deserves at least a couple of sentences and preferably an example that includes both, as I proposed in http://bugs.python.org/issue8376#msg102966. Do you disapprove of that proposal? -- ___ Python tracker <http://bugs.python.org/issue8376> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue27623] int.to_bytes() and int.from_bytes(): raise ValueError when bytes count is zero
Anders Lorentsen added the comment: I updated my patch to account for that second corner case. But ideally, shouldn't it rather be accounted for in the function that does the actual conversion, that is, in _PyLong_AsByteArray? -- Added file: http://bugs.python.org/file43942/int_to_bytes_overflow_cornercase2.patch ___ Python tracker <http://bugs.python.org/issue27623> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue27623] int.to_bytes() and int.from_bytes(): raise ValueError when bytes count is zero
Anders Lorentsen added the comment: So, am I to understand that the only corner case we should fix is that >>> (-1).to_bytes(0, 'big', signed=True) should raise an overflow error (currently it returns b'') ? -- Added file: http://bugs.python.org/file43925/int_to_bytes_overflow_cornercase.patch ___ Python tracker <http://bugs.python.org/issue27623> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue27623] int.to_bytes() and int.from_bytes(): raise ValueError when bytes count is zero
Anders Lorentsen added the comment: Isn't it possible to just add a small line of code that checks if length is less than or equal to 0, and if it is, call the necessary c functions to have python raise a valueerror...? Sorry if this is giving a solution without actually submitting the patch - but this is all very new to me. I have never contributed to anything yet (fourth year CS-student), and I am as fresh to the process as can be. I registered here just now. This seems like an issue I could handle.. I just need to take the time to learn about the process of how things are done around here. -- ___ Python tracker <http://bugs.python.org/issue27623> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue27623] int.to_bytes() and int.from_bytes(): raise ValueError when bytes count is zero
Changes by Anders Lorentsen : -- nosy: +Phaqui ___ Python tracker <http://bugs.python.org/issue27623> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue15873] datetime: add ability to parse RFC 3339 dates and times
Anders Hovmöller added the comment: Hmm, ok. I guess I was confused by "dates and times" part of the subject. Ok, so only datetimes. My other comments still apply though. > On 19 Jul 2016, at 16:20, Mathieu Dupuy wrote: > > > Mathieu Dupuy added the comment: > > because it limits itself to only support the RFC 3339 subset, as > explained in the begining of the discussion. > > 2016-07-19 16:07 GMT+02:00 Anders Hovmöller : >> >> Anders Hovmöller added the comment: >> >> The tests attached to this ticket seem pretty bare. Issues that I can spot >> directly: >> >> - only tests for datetimes, not times or dates >> - only tests for zulu and "-8:00” timezones >> - no tests for invalid input (parsing a valid date as a datetime for example) >> - only tests for -MM-DDTHH:MM:SSZ, but ISO8601 supports: >>- Naive times >>- Timezone information (specified as offsets or as Z for 0 offset) >>- Year >>- Year-month >>- Year-month-date >>- Year-week >>- Year-week-weekday >>- Year-ordinal day >>- Hour >>- Hour-minute >>- Hour-minute >>- Hour-minute-second >>- Hour-minute-second-microsecond >>- All combinations of the three "families" above! >> (the above list is a copy paste from my project that implements all ISO8601 >> that fits into native python: https://github.com/boxed/iso8601 >> <https://github.com/boxed/iso8601>) >> >> This is a more reasonable test suite: >> https://github.com/boxed/iso8601/blob/master/iso8601.py#L166 >> <https://github.com/boxed/iso8601/blob/master/iso8601.py#L166> although it >> lacks the tests for bogus inputs. >> >>> On 2016-07-16, at 03:41, Alexander Belopolsky >>> wrote: >>> >>> >>> Alexander Belopolsky added the comment: >>> >>> I would very much like to see this ready before the feature cut-off for >>> Python 3.6. Could someone post a summary on python-ideas to get a show of >>> hands on some of the remaining wrinkles? >>> >>> I would not worry about a C implementation at this point. We can put >>> python implementation in _strptime.py and call it from C as we do for the >>> strptime method. >>> >>> -- >>> >>> ___ >>> Python tracker >>> <http://bugs.python.org/issue15873> >>> ___ >> >> -- >> >> ___ >> Python tracker >> <http://bugs.python.org/issue15873> >> ___ > > -- > > ___ > Python tracker > <http://bugs.python.org/issue15873> > ___ -- ___ Python tracker <http://bugs.python.org/issue15873> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue15873] datetime: add ability to parse RFC 3339 dates and times
Anders Hovmöller added the comment: The tests attached to this ticket seem pretty bare. Issues that I can spot directly: - only tests for datetimes, not times or dates - only tests for zulu and "-8:00” timezones - no tests for invalid input (parsing a valid date as a datetime for example) - only tests for -MM-DDTHH:MM:SSZ, but ISO8601 supports: - Naive times - Timezone information (specified as offsets or as Z for 0 offset) - Year - Year-month - Year-month-date - Year-week - Year-week-weekday - Year-ordinal day - Hour - Hour-minute - Hour-minute - Hour-minute-second - Hour-minute-second-microsecond - All combinations of the three "families" above! (the above list is a copy paste from my project that implements all ISO8601 that fits into native python: https://github.com/boxed/iso8601 <https://github.com/boxed/iso8601>) This is a more reasonable test suite: https://github.com/boxed/iso8601/blob/master/iso8601.py#L166 <https://github.com/boxed/iso8601/blob/master/iso8601.py#L166> although it lacks the tests for bogus inputs. > On 2016-07-16, at 03:41, Alexander Belopolsky wrote: > > > Alexander Belopolsky added the comment: > > I would very much like to see this ready before the feature cut-off for > Python 3.6. Could someone post a summary on python-ideas to get a show of > hands on some of the remaining wrinkles? > > I would not worry about a C implementation at this point. We can put python > implementation in _strptime.py and call it from C as we do for the strptime > method. > > -- > > ___ > Python tracker > <http://bugs.python.org/issue15873> > ___ -- ___ Python tracker <http://bugs.python.org/issue15873> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue24954] No way to generate or parse timezone as produced by datetime.isoformat()
Anders Hovmöller added the comment: > The `arrow` library depends on the supposed "strict" behaviour of strptime > that has never been guaranteed, which often results in very buggy behaviour > under some conditions. Well… the arrow library accepts all sorts of broken input and gives you a date back. I think they even think that’s a feature and not a bug so no use in trying to help people who are adamant that they don’t want to be helped :/ -- ___ Python tracker <http://bugs.python.org/issue24954> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue15873] datetime: add ability to parse RFC 3339 dates and times
Anders Hovmöller added the comment: > > By the way, I just discovered, that the way we treat microseconds differs > from the strptime one : we are smarter read every digits and smartly round to > six, strptime doesn't go that far and just *truncate* to this. Should go that > way, for consistency with what strptime does, maybe ? I'm strongly against silently throwing away data and calling it even. If we want compatibility with strptime then it should be a separate flag like silent_truncate_milliseconds=True. On another matter: does the latest proposed code pass the tests in my ISO8601 implementation that I mentioned 2013 (! time flies)? -- ___ Python tracker <http://bugs.python.org/issue15873> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue26229] Make number serialization ES6/V8 compatible
Anders Rundgren added the comment: In ES6/V8-compatible implementations which include "node.js", Chrome, Firefox, Safari and (of course) my Java reference implementation you can take a cryptographic hash of a JSON object with a predictable result. That is, this request is in no way limited to JCS. Other solutions to this problem has been to create something like XML's canonicalization which is much more complex. The JSON RFC is still valid, it just isn't very useful for people who are interested in security solutions. The predictable property order introduced in ES6 makes a huge difference! Now it is just the number thing left... The other alternative is dressing your JSON objects in Base64 to maintain a predictable signature like in IETF's JOSE. I doubt that this is going to be mainstream except for OpenID/OAuth which JOSE stems from. -- ___ Python tracker <http://bugs.python.org/issue26229> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue26229] Make number serialization ES6/V8 compatible
Anders Rundgren added the comment: An easier fix than mucking around in the pretty complex number serializer code would be adding an "ES6Format" option to the "json.dump*" methods which could use the supplied conversion code as is. For JSON parsing in an ES6-compatible way you must anyway use an "OrderedDict" hook option to get the right (=original) property order. -- ___ Python tracker <http://bugs.python.org/issue26229> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue26241] repr() and str() are identical for floats in 3.5
Anders Rundgren added the comment: Apparently the docs have changed since 2.7: https://docs.python.org/3.5/tutorial/floatingpoint.html However, the documentation still "sort of" mentions repr() as the most accurate form which isn't entirely correct since it nowadays is identical to str() for floats. No big deal, I just thought I was doing something wrong :-) related: http://bugs.python.org/issue26229 -- ___ Python tracker <http://bugs.python.org/issue26241> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue26229] Make number serialization ES6/V8 compatible
Anders Rundgren added the comment:
As I said, the problem is close to fixed in 3.5.
You should not consider the JCS specification as the [sole] target but the
ability to creating a normalized JSON object which has many uses including
calculating a hash of such objects.
##
# Convert a Python double/float into an ES6/V8 compatible string #
##
def convert2Es6Format(value):
# Convert double/float to str using the native Python formatter
pyDouble = str(value)
pySign = ''
if pyDouble.find('-') == 0:
#
# Save sign separately, it doesn't have any role in the rest
#
pySign = '-'
pyDouble = pyDouble[1:]
pyExpStr = ''
pyExpVal = 0
q = pyDouble.find('e')
if q > 0:
#
# Grab the exponent and remove it from the number
#
pyExpStr = pyDouble[q:]
if pyExpStr[2:3] == '0':
#
# Supress leading zero on exponents
#
pyExpStr = pyExpStr[0:2] + pyExpStr[3:]
pyDouble = pyDouble[0:q]
pyExpVal = int(pyExpStr[1:])
#
# Split number in pyFirst + pyDot + pyLast
#
pyFirst = pyDouble
pyDot = ''
pyLast = ''
q = pyDouble.find('.')
if q > 0:
pyDot = '.'
pyFirst = pyDouble[0:q]
pyLast = pyDouble[q + 1:]
#
# Now the string is split into: pySign + pyFirst + pyDot + pyLast + pyExpStr
#
if pyLast == '0':
#
# Always remove trailing .0
#
pyDot = ''
pyLast = ''
if pyExpVal > 0 and pyExpVal < 21:
#
# Integers are shown as is with up to 21 digits
#
pyFirst += pyLast
pyLast = ''
pyDot = ''
pyExpStr = ''
q = pyExpVal - len(pyFirst)
while q >= 0:
q -= 1;
pyFirst += '0'
elif pyExpVal < 0 and pyExpVal > -7:
#
# Small numbers are shown as 0.etc with e-6 as lower limit
#
pyLast = pyFirst + pyLast
pyFirst = '0'
pyDot = '.'
pyExpStr = ''
q = pyExpVal
while q < -1:
q += 1;
pyLast = '0' + pyLast
#
# The resulting sub-strings are concatenated
#
return pySign + pyFirst + pyDot + pyLast + pyExpStr
--
___
Python tracker
<http://bugs.python.org/issue26229>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue26241] repr() and str() are identical for floats in 3.5
New submission from Anders Rundgren: According to the documentation repr() and str() are different when it comes to number formatting. A test with a 100 million random and selected IEEE 64-bit values returned no differences -- components: Interpreter Core messages: 259244 nosy: anders.rundgren@gmail.com priority: normal severity: normal status: open title: repr() and str() are identical for floats in 3.5 versions: Python 3.5 ___ Python tracker <http://bugs.python.org/issue26241> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue26229] Make number serialization ES6/V8 compatible
New submission from Anders Rundgren:
ECMA has in their latest release defined that JSON elements must be ordered
during serialization. This is easy to accomplish using Python's OrderedDict.
What is less trivial is that numbers have to be formatted in a certain way as
well. I have tested 100 millions specific and random values and found out that
Python 3.5.1 is mathematically identical to ES6/V8 but has some differences in
formatting:
IEEE DoubleECMAScript 6/V8Python 3.5.1
c43211ede4974a35, -0,-3.333e+20
c3fce97ca0f21056, -6000, -3.3336e+19
c3c7213080c1a6ac, -3334000, -3.334e+18
c39280f39a348556, -333400, -3.334e+17
c35d9b1f5d20d557, -33340,-3.334e+16
c327af4c4a80aaac, -3334, -3334.0
bf0179ec9cbd821e, -0.5, -3.3335e-05
becbf647612f3696, -0.03, -3.e-06
4024, 10,10.0
, 0, 0.0
4014, 5, 5.0
3f0a36e2eb1c432d, 0.5, 5e-05
3ed4f8b588e368f1, 0.05, 5e-06
3ea0c6f7a0b5ed8d, 5e-7, 5e-07
Why could this be important?
https://github.com/Microsoft/ChakraCore/issues/149
# Python test program
import binascii
import struct
import json
f = open('c:\\es6\\numbers\\es6testfile100m.txt','rb')
l = 0;
string = '';
while True:
byte = f.read(1);
if len(byte) == 0:
exit(0)
if byte == b'\n':
l = l + 1;
i = string.find(',')
if i <= 0 or i >= len(string) - 1:
print('Bad string: ' + str(i))
exit(0)
hex = string[:i]
while len(hex) < 16:
hex = '0' + hex
o = dict()
o['double'] = struct.unpack('>d',binascii.a2b_hex(hex))[0]
py3Double = json.dumps(o)[11:-1]
es6Double = string[i + 1:]
if es6Double != py3Double:
es6Dpos = es6Double.find('.')
py3Dpos = py3Double.find('.')
es6Epos = es6Double.find('e')
py3Epos = py3Double.find('e')
if py3Epos > 0:
py3Exp = int(py3Double[py3Epos + 1:])
if es6Dpos < 0 and py3Dpos > 0:
if es6Epos < 0 and py3Epos > 0:
py3New = py3Double[:py3Dpos] + py3Double[py3Dpos + 1:py3Epos -
len(py3Double)]
q = py3Exp - py3Epos + py3Dpos
while q >= 0:
py3New += '0'
q -= 1
if py3New != es6Double:
print('E1: ' + py3New)
exit(0)
elif py3Epos < 0:
py3New = py3Double[:-2]
if py3New != es6Double:
print('E2: ' + py3New)
exit(0)
else:
print (error + hex + '#' + es6Double + '#' + py3Double)
exit(0)
elif es6Dpos > 0 and py3Dpos > 0 and py3Epos > 0 and es6Epos < 0:
py3New = py3Double[py3Dpos - 1:py3Dpos] + py3Double[py3Dpos + 1:py3Epos
- len(py3Double)]
q = py3Exp + 1
while q < 0:
q += 1
py3New = '0' + py3New
py3New = py3Double[0:py3Dpos - 1] + '0.' + py3New
if py3New != es6Double:
print('E3: ' + py3New + '#' + es6Double)
exit(0)
elif es6Dpos == py3Dpos and py3Epos > 0 and es6Epos > 0:
py3New = py3Double[:py3Epos + 2] + str(abs(py3Exp))
if py3New != es6Double:
print('E4: ' + py3New + '#' + es6Double)
exit(0)
elif es6Dpos > 0 and py3Dpos < 0 and py3Epos > 0 and es6Epos < 0:
py3New = py3Double[:py3Epos - len(py3Double)]
q = py3Exp + 1
while q < 0:
q += 1
py3New = '0' + py3New
py3New = '0.' + py3New
if py3New != es6Double:
print('E5: ' + py3New + '#' + es6Double)
exit(0)
else:
print ('Unexpected: ' + hex + '#' + es6Double + '#' + py3Double)
exit(0)
string = ''
if l % 1 == 0:
print(l)
else:
string += byte.decode(encoding='UTF-8')
--
components: Interpreter Core
messages: 259105
nosy: anders.rundgren@gmail.com
priority: normal
severity: normal
status: open
title: Make number serialization ES6/V8 compatible
type: enhancement
versions: Python 3.5
___
Python tracker
<http://bugs.python.org/issue26229>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue26191] pip on Windows doesn't honor Case
Anders Rundgren added the comment: You are right. Pardon me for erring :-( Thanks for the quick response BTW! Anders -- ___ Python tracker <http://bugs.python.org/issue26191> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue26191] pip on Windows doesn't honor Case
New submission from Anders Rundgren: pip install Crypto Terminates correctly and the package is there as well. Unfortunately the directory is named "crypto" rather than "Crypto" so when I perform >>>import Crypto the interpreter fails. >>>import crypto seems to work but is incompatible over platforms. If this is a problem with pycrypto or pip is beyond my knowledge of python. -- components: Installation messages: 258887 nosy: anders.rundgren@gmail.com priority: normal severity: normal status: open title: pip on Windows doesn't honor Case type: compile error versions: Python 3.5 ___ Python tracker <http://bugs.python.org/issue26191> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue23123] Only READ support for Decimal in json
Anders Rundgren added the comment:
> Antoine Pitrou added the comment:
>
> I won't claim to know/understand the specifics, but "message payload in
> base64" actually sounds reasonable to me, if far from optimal (both from
> readibility and space overhead POV) :-).
It is indeed a working solution. I do though think that communities that
previously used XML would accept base64-encoded messages. It becomes really
messy when applied to counter-signed messages like the following:
{
"@context": "http://xmlns.webpki.org/wcpp-payment-demo";,
"@qualifier": "AuthData",
"paymentRequest":
{
"commonName": "Demo Merchant",
"amount": 8600550,
"currency": "USD",
"referenceId": "#100",
"dateTime": "2014-12-18T13:39:35Z",
"signature":
{
"algorithm": "RS256",
"signerCertificate":
{
"issuer": "CN=Merchant Network Sub CA5,C=DE",
"serialNumber": "1413983542582",
"subject": "CN=Demo Merchant,2.5.4.5=#1306383936333235,C=DE"
},
"certificatePath":
[
"MIIDQzCCAiugAwIBAgIGAUk3_J02M...eMGlY734U3NasQfAhTUhxrdDbphEvsWTc",
"MIIEPzCCAiegAwIBAgIBBTANBgkqh...gU1IyRGA7IbdHOeDB2RUpsXloU2QKfLrk"
],
"value":
"Ny4Qe6FQhd5_qcSc3xiH8Kt7tIZ9Z...9LEjC6_Rulg_G20fGxJ-wzezFpsAGbmuFQg"
}
},
"domainName": "merchant.com",
"cardType": "SuperCard",
"pan": "1618342124919252",
"dateTime": "2014-12-18T13:40:59Z",
"signature":
{
"algorithm": "RS256",
"signerCertificate":
{
"issuer": "CN=Mybank Client Root CA1,C=US",
"serialNumber": "1413983550045",
"subject": "CN=The Cardholder,2.5.4.5=#13083935363733353232"
},
"certificatePath":
["MIIENzCCAh-gAwIBAgIGAUk3_LpdM...IGcN1md5feo5DndNnV8D0UM-oBRkUDDFiWlhCU"],
"value":
"wyUcFcHmvM5ZozZKOEwOQkYic0D7M...S_HbaPGau5KfZjCaksvb5U1lYZaXxP8kAbuGPQ"
}
}
--
___
Python tracker
<http://bugs.python.org/issue23123>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue23123] Only READ support for Decimal in json
Anders Rundgren added the comment: > Antoine Pitrou added the comment: > > "To cope with this potential problem, compliant parsers must preserve the > original textual representation of properties internally in order to support > JCS normalization requirements" > > That sounds ridiculous. Did someone try to reason the "IETF guys"?:) The alternative is either doing what Bob suggested which is almost the same as writing a new parser or take the IETF route and shroud the message payload in base64. So all solutions are "by definition" bd :-) FWIW my super-bad solution has the following compatibility issues: - Whitespace: None, all parsers can "stringify", right? - Escaping: None, all parsers MUST do it to follow the JSON spec. - Property order: A problem in some parsers. If you take a look on stackoverflow lots of folks request that insertion/reader order should be honored since computers <> humans. Fixed in Python. Works in browsers as well. - Floating point: an almost useless JSON feature anyway, it doesn't work for crypto-numbers or money. It is "only" a validation problem though. Now fixed in Python. http://www.ietf.org/mail-archive/web/acme/current/msg00200.html -- ___ Python tracker <http://bugs.python.org/issue23123> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue23123] Only READ support for Decimal in json
Anders Rundgren added the comment: Ethan Furman added the comment: > I am not a regular json user, but my impression is the format is > pretty basic, and we would be overloading it to try and keep numbers > with three decimal places as Decimal, and anything else as float. > Isn't json's main purpose to support data exchange between different > programs of different languages? Not between different Python > programs? Right, unfortunately the need to support non-native data types like big decimals, dates and blobs have lead to a certain amount of confusion and innovation among JSON tool designers. I (FWIW) do actually NOT want to extend a single bit from the RFC, I just want serializing to be "non-invasive". If the parse_float option stays "as is" it seems that both the people who want big (non-standard) numbers and I who want somewhat non-standard serialization would be happy. I.e. a documentation snippet would be sufficient as far as I can tell. Serialization order of objects is apparently a hot topic https://code.google.com/p/v8/issues/detail?id=164 but Python has no problem with that. -- ___ Python tracker <http://bugs.python.org/issue23123> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue23123] Only READ support for Decimal in json
Anders Rundgren added the comment: The current JCS validator is only 150 lines and does both RSA and EC signatures: https://code.google.com/p/openkeystore/source/browse/python/trunk/src/org/webpki/json/JCSValidator.py My Java-version is much more advanced but this is quite useful anyway -- ___ Python tracker <http://bugs.python.org/issue23123> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue23123] Only READ support for Decimal in json
Anders Rundgren added the comment: Bob, I'm not sure I understand why you say that JCS requires *almost* full normalization. Using browsers you can generate fully compliant JCS objects using like 20 lines of javascript/webcrypto (here excluding base64 support). No normalization step is needed. But sure, the IETF JOSE WG has taken an entirely different approach and require JSON objects to be serialized and Base64-encoded. Then the Base64 is signed. Boring. And in conflict with complex messaging like: https://openkeystore.googlecode.com/svn/wcpp-payment-demo/trunk/docs/messages.html#UserAuthorizesTransaction Thanx anyway, I'm pretty happy with how it works now! Well, if Decimal didn't manipulate its argument I would be even happier :-) because then there wouldn't even be a hack. -- ___ Python tracker <http://bugs.python.org/issue23123> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue23123] Only READ support for Decimal in json
Anders Rundgren added the comment:
Using simplejson I got it to work!!!
I just wonder what you think of the solution:
import collections
import simplejson as json
from decimal import Decimal
class EnhancedDecimal(Decimal):
def __str__ (self):
return self.saved_string
def __new__(cls, value="0", context=None):
obj = Decimal.__new__(cls,value,context)
obj.saved_string = value
return obj;
jsonString = '{"t":6,"h":4.50, "g":"text","j":1.40e450}'
jsonObject = json.loads(jsonString,
object_pairs_hook=collections.OrderedDict,parse_float=EnhancedDecimal)
for item in jsonObject:
print jsonObject[item]
print json.dumps(jsonObject)
6
4.50
text
1.40e450
{"t": 6, "h": 4.50, "g": "text", "j": 1.40e450}
--
___
Python tracker
<http://bugs.python.org/issue23123>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue23123] Only READ support for Decimal in json
Anders Rundgren added the comment: Well, I could have insisted on canonicalization of floating-point data but that's so awkward that outlawing such data is a cleaner approach. Since the target for JCS is security- and payment-protocols, I don't think the absence of floating-point support will be a show-stopper. I does though make the IETF folks unhappy. Another reason for still wanting it to work as currently specified is because it would be nice to have JCS running on three fully compatible platforms, including one which I haven't designed :-) -- ___ Python tracker <http://bugs.python.org/issue23123> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue23123] Only READ support for Decimal in json
Anders Rundgren added the comment: Bob, Your'e right, I have put up a requirement for JSON serializing that may be "over the top". OTOH, there are (AFAICT...) only two possible solutions: 1. Outlaw floating point data from the plot 2. Insist that serializers conform to the spec As a pragmatic I have settled on something in between :-) https://openkeystore.googlecode.com/svn/resources/trunk/docs/jcs.html#Interoperability I don't think that the overhead in Decimal would be a problem but I'm not a Python platform maintainer so I leave it to you guys. -- ___ Python tracker <http://bugs.python.org/issue23123> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue23123] Only READ support for Decimal in json
Anders Rundgren added the comment: I guess my particular requirement/wish is unusual (keeping the original textual representation of a floating point number intact) while using Decimal should be fairly universal. If these things could be combined in a Decimal support option I would (of course) be extremely happy. They do not appear to be in conflict. Currently I'm a bit bogged down by the crypto-stuff since it is spread over different and incompatible modules which makes it awkward creating a nice unified RSA/EC solution. I may end-up writing a wrapper... -- ___ Python tracker <http://bugs.python.org/issue23123> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue23123] Only READ support for Decimal in json
Anders Rundgren added the comment: It would be great if I could use a sub-classed Decimal during parsing but since it doesn't appear to be a way to serialize the result using the "json" package I'm probably stuck with the current "99%" solution. I have solved this in Java and JavaScript by writing my own JSON stuff http://webpki.org/papers/keygen2/doc/org/webpki/json/package-summary.html but that method obviously doesn't scale and I'm a real n00b when it comes to Python although it was more fun than I had expected :-) A minor patch addressing serialization of Decimal would probably do fine (after sub-classing) and would be generally useful. -- ___ Python tracker <http://bugs.python.org/issue23123> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue23123] Only READ support for Decimal in json
Anders Rundgren added the comment: I was actually hoping to implement the final part of this: https://openkeystore.googlecode.com/svn/resources/trunk/docs/jcs.html#Normalization_and_Signature_Validation It seems that the current Decimal implementation wouldn't save me anyway since it modifies the input :-( Anyway, floats in JSON have rather little use so maybe my existing Pyhton (PoC) solution will be "good enough": https://code.google.com/p/openkeystore/source/browse/python/trunk/src/org/webpki/json/JCSValidator.py -- ___ Python tracker <http://bugs.python.org/issue23123> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue23123] Only READ support for Decimal in json
New submission from Anders Rundgren:
jsonString = '{"t":6,"h":4.50, "g":"text","j":1.40e450}'
jsonObject = json.loads(jsonString,
object_pairs_hook=collections.OrderedDict,parse_float=Decimal)
for item in jsonObject:
print jsonObject[item]
6
4.50
text
1.40E+450
Works as expected.
However, there seems to be no way to get back to the original JSON string as
far as I can tell since you have to convert Decimal to str in cls when using
json.dumps which adds "" around the arguments
--
components: Extension Modules
messages: 233139
nosy: anders.rundgren@gmail.com
priority: normal
severity: normal
status: open
title: Only READ support for Decimal in json
type: behavior
versions: Python 2.7
___
Python tracker
<http://bugs.python.org/issue23123>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue20288] HTMLParse handing of non-numeric charrefs broken
New submission from Anders Hammarquist:
Python 2.7 HTMLParse.py lines 185-199 (similar lines still exist in Python 3.4)
match = charref.match(rawdata, i)
if match:
...
else:
if ";" in rawdata[i:]: #bail by consuming &#
self.handle_data(rawdata[0:2])
i = self.updatepos(i, 2)
break
if you feed a broken charref, that is non-numeric, it will pass whatever random
string that happened to be at the start of rawdata to handle_data(). Eg:
p = HTMLParser()
p.handle_data = lambda x: sys.stdout.write(x)
p.feed('&#foo;')
will print '
<http://bugs.python.org/issue20288>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue15873] "datetime" cannot parse ISO 8601 dates and times
Anders Hovmöller added the comment: Éric Araujo: absolutely. Although I think my code can be improved (speed wise, elegance, etc) since I just wrote it quickly a weekend :) -- ___ Python tracker <http://bugs.python.org/issue15873> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue15873] "datetime" cannot parse ISO 8601 dates and times
Anders Hovmöller added the comment: I've written a parser for ISO 8601: https://github.com/boxed/iso8601 Some basic tests are included and it supports most of the standard. Haven't gotten around to the more obscure parts like durations and intervals, but those are trivial to add... -- nosy: +Anders.Hovmöller ___ Python tracker <http://bugs.python.org/issue15873> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue15429] types.NoneType missing
New submission from Anders Kaseorg : http://docs.python.org/py3k/library/constants.html#None says that None is the sole value type types.NoneType. However, NoneType was removed from the types module with Python 3. -- ___ Python tracker <http://bugs.python.org/issue15429> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue15429] types.NoneType missing
Changes by Anders Kaseorg : -- assignee: docs@python components: Documentation nosy: andersk, docs@python priority: normal severity: normal status: open title: types.NoneType missing type: behavior versions: Python 3.2 ___ Python tracker <http://bugs.python.org/issue15429> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue15292] import hook behavior documentation improvement
New submission from Anders Hammarquist : When testing Eutaxia on PyPy (1.9) I discovered a discrepancy in the path_hooks import hook implementation. In CPython (2.7), if the find_module() method raises ImportError (as imp.find_module() does when it does not find a module in the given path), will cause the search to continue, whereas PyPy would propagate the ImportError. PyPy has now been changed to behave like CPython. The documentation is not entirely clear, but it does not explicitly document the import hook mechanism as eating an ImportError in find_module(). It should probably be made explicit, which ever way it should be. It is not obvious what is the correct behaviour, given the implicit relative imports, where the ImportError simply means that the import hook cannot find the module. Quick testing on CPython 3.3 indicates that it behaves like PyPy did, but as it doesn't do implicit relative imports my test case didn't work as it was. For 3.3, without implicit relative imports, propagating the ImportError feels like the correct behaviour. The attached demonstration needs a file /tmp/test/foo.py that does a top-level import, e.g. "import errno" to demonstrate the discrepancy. -- assignee: docs@python components: Documentation files: testimport.py messages: 164998 nosy: docs@python, iko priority: normal severity: normal status: open title: import hook behavior documentation improvement type: behavior versions: Python 2.7 Added file: http://bugs.python.org/file26315/testimport.py ___ Python tracker <http://bugs.python.org/issue15292> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue10513] sqlite3.InterfaceError after commit
Anders Blomdell added the comment: > So my suggestion is to remove in "pysql_connection_commit" the call to : > pysqlite_do_all_statements(self, ACTION_RESET, 0); > to bring back the correct old behavior. That's what I have been running for years, now... > And also eventually to remove in "pysqlite_connection_rollback" the > call to : > pysqlite_do_all_statements(self, ACTION_RESET, 1); Have no opinion on this Would be really nice to not have to fix this in ech new release :-) -- ___ Python tracker <http://bugs.python.org/issue10513> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue9334] argparse does not accept options taking arguments beginning with dash (regression from optparse)
Anders Kaseorg added the comment: James: That’s not related to this issue. This issue is about options taking arguments beginning with dash (such as a2x --asciidoc-opts --safe, where --safe is the argument to --asciidoc-opts), not positional arguments beginning with dash. Your observation isn’t a bug. In all getopt-like parsers, -- is the only way to pass positional arguments beginning with -. (Whether you shell-quoted the argument is irrelevant; the - is interpreted by the program, not the shell, after the shell has already stripped off the shell quoting.) If your program doesn’t take any options and you’d like to parse positional arguments without requiring --, don’t use a getopt-like parser; use sys.argv directly. If you still think your example is a bug, please file a separate report. -- ___ Python tracker <http://bugs.python.org/issue9334> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue12844] Support more than 255 arguments
Anders Kaseorg added the comment:
I guess the desugaring is slightly more complicated in the case where the
original function call already used *args or **kwargs:
f(arg0, …, arg999, *args, k0=v0, …, k999=v999, **kwargs)
becomes something like
f(*((arg0, …, arg999) + args),
**dict({'k0': 'v0', …, 'k999': 'v999'}, **kwargs))
--
___
Python tracker
<http://bugs.python.org/issue12844>
___
___
Python-bugs-list mailing list
Unsubscribe:
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue12844] Support more than 255 arguments
New submission from Anders Kaseorg :
This feels like an arbitrary restriction (obvious sequences have been replaced
with ‘…’ to save space in this report):
>>> zip([0], [1], [2], …, [1999])
File "", line 1
SyntaxError: more than 255 arguments
especially when this works:
>>> zip(*[[0], [1], [2], …, [1999]])
[(0, 1, 2, …, 1999)]
Apparently that limit bites some people:
https://docs.djangoproject.com/en/1.3/topics/http/urls/#module-django.conf.urls.defaults
The bytecode format doesn’t support directly calling a function with more than
255 arguments. But, it should still be pretty easy to compile such function
calls by desugaring
f(arg0, …, arg999, k0=v0, …, k999=v999)
into
f(*(arg0, …, arg999), **{'k0': 'v0', …, 'k999': 'v999'})
--
components: Interpreter Core
messages: 142995
nosy: andersk
priority: normal
severity: normal
status: open
title: Support more than 255 arguments
type: feature request
___
Python tracker
<http://bugs.python.org/issue12844>
___
___
Python-bugs-list mailing list
Unsubscribe:
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue9334] argparse does not accept options taking arguments beginning with dash (regression from optparse)
Anders Kaseorg added the comment: > @andersk: Would the restriction to only having flags with a fixed > number of arguments be acceptable for your use case? I think that’s fine. Anyone coming from optparse won’t need options with optional arguments. However, FWIW, GNU getopt_long() supports options with an optional argument under the restrictions that: • the option must be a long option, • the optional argument must be the only argument for the option, and • the argument, if present, must be supplied using the ‘--option=argument’ form, not the ‘--option argument’ form. This avoids all parsing ambiguity. It would be useful to have feature parity with getopt_long(), to facilitate writing Python wrapper scripts for C programs. -- ___ Python tracker <http://bugs.python.org/issue9334> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue11202] Win32: shutil.move does not inherit permissions
Anders Østhus added the comment: Thank you for taking the time to explain it to me, but it still seems inconsistent to me. I did a test with the functions copy, copy2, move, os.rename, copyfile, both on the same filesystem and another filesystem, and the result is: Same filesystem: shutil.copy:Inherit shutil.copy2: Inherit shutil.move:Original os.rename: Original shutil.copyfile:Inherit Different filesystem: shutil.copy:Inherit shutil.copy2: Inherit shutil.move:Inherit os.rename: Inherit shutil.copyfile:Inherit On the same system, the result will differ if you move the file to a different location on the same filesystem, or if you move it to another filesystem. But still, I'm happy. I'll just use copy/copy2 and then delete the original file afterwards :) -- ___ Python tracker <http://bugs.python.org/issue11202> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue11202] Win32: shutil.move does not inherit permissions
Anders Østhus added the comment: On my system (Win Server 2008 R2 64-Bit, Python 2.7.1), when I use copy, copy2 or move(to another filesystem), the file _will_ get the ACL of the DST folder, and remove any ACL in SRC file that the DST folder does not have. Thus, it doesn't copy it from the source file, but inherits from the DST folder. -- ___ Python tracker <http://bugs.python.org/issue11202> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue11202] Win32: shutil.move does not inherit permissions
Anders Østhus added the comment: Ok. But that makes the whole method inconsistent. Basically, if it's on the same filesystem, rename the file, and thus not inheriting ACL. If it's on another use copy2, and inherit ACL. That makes no sense, atleast not to me :) -- ___ Python tracker <http://bugs.python.org/issue11202> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue11202] Win32: shutil.move does not inherit permissions
Anders Østhus added the comment: Ok, but the whole page you linked to (http://docs.python.org/library/shutil) confuses me then. It states at the top: "Warning Even the higher-level file copying functions (copy(), copy2()) can’t copy all file metadata. On POSIX platforms, this means that file owner and group are lost as well as ACLs. On Mac OS, the resource fork and other metadata are not used. This means that resources will be lost and file type and creator codes will not be correct. On Windows, file owners, ACLs and alternate data streams are not copied." Then, under shutil.copy: "Permission bits are copied". I'm assuming this is UGO permissions on POSIX systems, and thus correct according to the top text. shutil.copy2 says: Similar to shutil.copy, but with metadata. Files copied with both shutil.copy and shutil.copy2 both inherits the permissions from their destination, but shutil.move does not. According to the shutil doc page, neither copy or copy2 should do this. And since they do, and you say shutil.move is implemented using shutil.copy2, shouldn't files moved with shutil.move also then inherit the permissions? -- status: closed -> open ___ Python tracker <http://bugs.python.org/issue11202> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue11202] Win32: shutil.move does not inherit permissions
New submission from Anders Østhus : Hi I'm running Python 2.7.1 (r271:86832, Nov 27 2010, 17:19:03) [MSC v.1500 64 bit (AMD64)] on win32 (Server 2008 R2). I've discovered that when moving files with shutil.move, the file won't inherit the security settings as it should. When using shutil.copy, it does get the right permissions. -- components: IO messages: 128452 nosy: Anders.Østhus priority: normal severity: normal status: open title: Win32: shutil.move does not inherit permissions versions: Python 2.7 ___ Python tracker <http://bugs.python.org/issue11202> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue9334] argparse does not accept options taking arguments beginning with dash (regression from optparse)
Anders Kaseorg added the comment: > (1) It's only deprecated in the documentation Which is why I suggested un-deprecating it in the documentation. (I want to avoid encouraging programmers to switch away from optparse until this bug is fixed.) > # proposed behavior > parser = ArgumentParser(error_on_unknown_options=False) Perhaps you weren’t literally proposing “error_on_unknown_options=False” as the name of the new flag, but note that neither the current nor proposed behaviors have nothing to do with whether arguments look like known or unknown options. Under the proposed behavior, anything in argument position (--asciidoc-opts ___) is parsed as an argument, no matter what it looks like. So a more accurate name might be “refuse_dashed_args=False”, or more generally (in case prefix_chars != '-'), “refuse_prefixed_args=False”? -- ___ Python tracker <http://bugs.python.org/issue9334> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue9334] argparse does not accept options taking arguments beginning with dash (regression from optparse)
Anders Kaseorg added the comment: That would be a good first step. I continue to advocate making that mode the default, because it’s consistent with how every other command line program works[1], and backwards compatible with the current argparse behavior. As far as documentation for older versions, would it be reasonable to un-deprecate optparse until argparse becomes a suitable replacement? There are still lots of programmers working in Python 2.7. [1] bethard’s msg128047 is confusing positional arguments with option arguments. All UNIX commands that accept option arguments have no trouble accepting option arguments that begin with -. For example, ‘grep -e -pattern file’ is commonly used to search for patterns beginning with -. -- ___ Python tracker <http://bugs.python.org/issue9334> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue9334] argparse does not accept options taking arguments beginning with dash (regression from optparse)
Anders Kaseorg added the comment:
There are some problems that ‘=’ can’t solve, such as options with nargs ≥ 2.
optparse has no trouble with this:
>>> parser = optparse.OptionParser()
>>> parser.add_option('-a', nargs=2)
>>> parser.parse_args(['-a', '-first', '-second'])
(, [])
But inputting those arguments is _not possible_ with argparse.
>>> parser = argparse.ArgumentParser()
>>> parser.add_argument('-a', nargs=2)
>>> parser.parse_args(['-a', '-first', '-second'])
usage: [-h] [-a A A]
: error: argument -a: expected 2 argument(s)
--
___
Python tracker
<http://bugs.python.org/issue9334>
___
___
Python-bugs-list mailing list
Unsubscribe:
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1571170] Some numeric characters are still not recognized
Anders Chrigström added the comment: This is indeed a duplicate of #1571184 -- resolution: -> duplicate status: open -> closed ___ Python tracker <http://bugs.python.org/issue1571170> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue10513] sqlite3.InterfaceError after commit
Anders Blomdell added the comment:
The culprit seems to be 'pysqlite_do_all_statements(self, ACTION_RESET, 0)' in
pysqlite_connection_commit, which resets all active statements, but subsequent
fetch/fetchall seems to trash the sqlite3 state in the statements. Removing the
ACTION_RESET seems to bring back old behaviour (if it's the correct fix is,
however, beyond me).
Slightly modified testprogram that shows more wierdness; output from:
c = cursor.execute(' select k from t where k == ?;', (0,))
conn.commit()
print c.fetchall()
is:
[(0,), (0,)]
which is not what I would expect with a primary key...
--
Added file: http://bugs.python.org/file19794/sqlite_bug.py
___
Python tracker
<http://bugs.python.org/issue10513>
___
___
Python-bugs-list mailing list
Unsubscribe:
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue10513] sqlite3.InterfaceError after commit
New submission from Anders Blomdell :
With version 2.7 (and 2.7.1rc1), the following sequence (see attached test):
c = cursor.execute(' select k from t where k == ?;', (1,))
conn.commit()
r = c.fetchone()
Traceback (most recent call last):
File "/bugs/sqlite_bug.py", line 22, in
c = cursor.execute(' select k from t where k == ?;', (2,))
sqlite3.InterfaceError: Error binding parameter 0 - probably unsupported type.
The program works with 2.6.2
--
components: Extension Modules
files: sqlite_bug.py
messages: 122213
nosy: anders.blomd...@control.lth.se
priority: normal
severity: normal
status: open
title: sqlite3.InterfaceError after commit
type: crash
versions: Python 2.7
Added file: http://bugs.python.org/file19784/sqlite_bug.py
___
Python tracker
<http://bugs.python.org/issue10513>
___
___
Python-bugs-list mailing list
Unsubscribe:
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
