[issue46455] Deprecate / remove os.name=java
Xavier Morel added the comment: PS: "platform.system()" also documents `Java` as a value which doesn't seem to make that much sense, however it's an open set so probably less of an issue / source of confusion. -- ___ Python tracker <https://bugs.python.org/issue46455> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue46455] Deprecate / remove os.name=java
New submission from Xavier Morel : os.name is defined as: > The following names have currently been registered: 'posix', 'nt', 'java'. In my understanding, the value `'java'` is for the benefit of jython, which is rather poorly. Other third-party implementations which may or may not have a "full os module" (e.g. ironpython) do not -- as far as I can tell -- get to be registered against this value, and Python 3.3's addition of `sys.implementation` seems like a more reliable (and better supported) way to perform implementation-specific checks. Therefore I feel `os.name == 'java'` only exists to confuse readers of the documentation, but doesn't really provide any value, and should be removed. -- components: Library (Lib) messages: 411123 nosy: xmorel priority: normal severity: normal status: open title: Deprecate / remove os.name=java versions: Python 3.10, Python 3.11, Python 3.7, Python 3.8, Python 3.9 ___ Python tracker <https://bugs.python.org/issue46455> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue45368] ~(True) and ~(False) gives incorrect result
Xavier Morel added the comment: > True is a boolean so ~True should return False according to me. That's be a BC break for no reason: if you want to invert a boolean you can just `not` it. > True is not the same as 1 For historical reasons, in Python it is: >>> bool.mro() [, , ] >>> True == 1 True >>> False == 0 True So when you call ~True, you're calling `int.__invert__(True)`, which behaves as what it is: the bitwise inverse of a two's-complement signed integer. -- nosy: +xmorel ___ Python tracker <https://bugs.python.org/issue45368> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue45365] concurrent.futures.Future should be suitable for use outside of executors
New submission from Xavier Morel : concurrent.futures.Future currently has the note: > Future instances are created by Executor.submit() and should not be created > directly except for testing. That seems like a shame as futures are useful concurrency construct and having to rebuild them "by hand" seems like a waste. What are the issues which prevent safely using futures outside of executors, and is there a way they could be fixed / lifted? -- components: Library (Lib) messages: 403181 nosy: xmorel priority: normal severity: normal status: open title: concurrent.futures.Future should be suitable for use outside of executors type: enhancement versions: Python 3.10 ___ Python tracker <https://bugs.python.org/issue45365> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue43526] Programmatic management of BytesWarning doesn't work for native triggers.
Xavier Morel added the comment: > If working Python 3 program suddenly became emitting BytesWarning it will > confuse users. Oh yeah no I meant making it a normal warning, without needing the `-b` flag, not enabling it by default. Because controlling / configuring warnings can be done programmatically and dynamically from inside the program, but in my understanding -b is either set or unset when calling it, if you're not setting it you're SOL. Or so I understand, I didn't find any switch accessible from inside the Python program. > I think it would work. But it is significant amount of work (add > corresponding C API, parsing and copying code, documentation, tests) Wouldn't it just be removing the check on `-b` in the tp_str slot of bytes and bytearray? > and for small benefit. Would not be better to filter out warnings by message? Well yes and no, my issue remains the one from the beginning: it's quite easy to unexpectedly stringify bytes, which usually isn't what's desired. Ensuring that doesn't happen requires making sure every developer and environment (non-production at least) does run python with -b to enable the warning, which is not as easy as the software being able to set that internally (which is doable for various deprecation warnings and friends). -- ___ Python tracker <https://bugs.python.org/issue43526> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue43526] Programmatic management of BytesWarning doesn't work for native triggers.
Xavier Morel added the comment: And though I did not check, I expect the `-b` flag exists mostly because of the performance impact of the warning any time bytes are checked for equality, but surely that impact would be limited and probably not very relevant for the stringification of bytes and that could (eventually) be moved out of the `-b` flag and to the regular warnings system? I recall Inada-san tends to be quite involved in BytesWarning, is adding people to the nosy list something that's done or is it in bad taste? -- ___ Python tracker <https://bugs.python.org/issue43526> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue43526] Programmatic management of BytesWarning doesn't work for native triggers.
Xavier Morel added the comment: > I am not against documenting the behavior of -b and BytesWarning clearly. I > don't think that anyone would be against. Just somebody have to provide a PR. Right but what about the ability to enable warning on stringification without enabling the warning on comparison? There are packages / situations where comparing strings and bytes makes a lot of sense (e.g. use string and byte versions of data as keys in the same dict), but I think stringifying bytes rarely if ever does. > Explicitly repr-ing a bytes instance does not produce a warning, and never > produced, so I don't understand what do you mean. Well that's absolutely ideal as I did not check if it did, just wanted to be clear that I wasn't looking for a warning in that case. -- ___ Python tracker <https://bugs.python.org/issue43526> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue43526] Programmatic management of BytesWarning doesn't work for native triggers.
Xavier Morel added the comment: Serhiy an other question (because I've encountered it *again*), do you think it'd be possible to split out the specific warning of stringifying (but *not* explicitely repr-ing) a bytes instance from everything else? There are use-cases for it, but I think it's rather error prone when a `bytes` unexpectedly finds itself mixed into either string or non-string data which stringify "normally" and generates what is often garbage output. -- ___ Python tracker <https://bugs.python.org/issue43526> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue44178] Add an interpreter-level critical section construct
New submission from Xavier Morel : Python code uses a fair amount of globals, and sometimes there's no good choice but to do dodgy things and temporarily update global state for one reason or another e.g. redirect a standard fd or stream (cf redirect_stdout), monkey-patch a builtin to intercept some behaviour, etc... One issue with this is, if the program is multithreaded there is no real way to prevent the interpreter from suspending the current thread *while doing the dodgy thing*. The only interpreter-level global lock would be the GIL, and while native extension can just not release the GIL this is not an option for Python code (as far as I know). `sys` does provide getswitchinterval()/setswitchinterval() (and under the old GIL had getcheckinterval()/setcheckinterval()), but the semantics of these functions is a bit obscure (e.g. does `sys.setcheckinterval` update the *current* time slice os is the current thread doomed? can it fail? what happens if it does?) and they feel extremely uncomfortable. As such, having a context manager which would explicitly but only prevent thread switching while it's held would be valuable. Assuming `setswitchinginterval` operates on the "current" timeslice, this would not provide any more misbehaving powers as any code can already call `sys.setswitchinterval(sys.maxsize)` (unless there are already mitigation systems in-place), and the contextmanager could just do that internally if that is the "correct" way to prevent thread switching under the current scheme. The one thing which is not great is that this scheme might not really work under GILectomy, but then neither does switchinterval I think. -- components: Interpreter Core messages: 393951 nosy: xmorel priority: normal severity: normal status: open title: Add an interpreter-level critical section construct type: enhancement versions: Python 3.9 ___ Python tracker <https://bugs.python.org/issue44178> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue12056] "…" (HORIZONTAL ELLIPSIS) should be an alternative syntax for "..." (FULL STOP FULL STOP FULL STOP)
Xavier Morel added the comment:
> But if we allow for ellipsis, then would we not also have to start allowing
> characters like ≥ and ≤ in Python?
No, they're not defined as canonically equivalent to >= and <= by the Unicode
specification:
>>> unicodedata.normalize('NFKD', '…')
...
>>> unicodedata.normalize('NFKD', '≤')
≤
--
___
Python tracker
<https://bugs.python.org/issue12056>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue44157] redirect_* should also redirect C-level streams
New submission from Xavier Morel : In 3.4 (resp. 3.5), `redirect_stdout` and `redirect_stderr` were added to provide easy and reentrant stream redirection. Although that is documented, it seems like a waste that they only redirect the high-level `sys.std*` streams: those are already pretty easy to redirect by simply setting the corresponding attribute, which is essentially what the contextmanager does. However, * that does not work if the writer has imported the original stream object (`from sys import stderr`) * that does not work if the writer bypasses or is not aware of the Python layer e.g. a native library In that case, the user still has to deal with dup/dup2 dances in order to redirect the underlying fds, which is significantly more error-prone (and verbose) than intercepting the high-level `sys.std*` ever was. On the other hand, it's also less expressive as it requires an actual fd, rather than a Python-level file-like object. So I can understand not doing it by default. But still, I think I feel like it'd be very useful to either expand the existing context managers to perform fd redirection, or have a separate context manager able to redirect arbitrary fds. -- components: Library (Lib) messages: 393810 nosy: xmorel priority: normal severity: normal status: open title: redirect_* should also redirect C-level streams type: enhancement versions: Python 3.9 ___ Python tracker <https://bugs.python.org/issue44157> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue43526] Programmatic management of BytesWarning doesn't work for native triggers.
Xavier Morel added the comment: > In normal circumstances you should never deal with BytesWarning. The -b > option is only used for testing your program for some possible bugs caused by > migration from Python 2. If your program always worked only with Python 3, > the -b option has no use for you. That's technically true for perfect programs, but practically if the program is not tested enough it's possible for it to contain unexpected instances of stringifying bytes which pass by mostly unnoticed, just with garbage bits of output which may or may not get reported. Noticing such cases in an existing codebase is exactly what prompted me to try and enable BytesWarning (and then try and filter them to just conversion warnings, as the category is quite noisy with somewhat uninteresting warnings), and then spend a while trying to understand why programmatically enabling BytesWarning didn't do anything. As noted in the last paragraph it's fine if `-b` is required to enable the feature, I'd just like to know if it could be documented clearly as it's somewhat frustrating to pore over the documentation of `warnings`, `BytesWarning`, and `-b`, diff the state of the filters configuration, etc… and end up finding out from a bpo comment whether the behaviour you're observing is intentional or not. -- ___ Python tracker <https://bugs.python.org/issue43526> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue43527] Support full stack trace extraction in warnings.
New submission from Xavier Morel : When triggering warnings, it's possible to pass in a `stacklevel` in order to point to a more informative cause than the `warnings.warn` call. For instance `stacklevel=2` is a common one for DeprecationWarning in order to mark the call itself as deprecated in the caller's codebase. The problem with this is that it's not transitive, so when a dependency triggers a warning it can be hard to know where that comes from in the codebase (at least without `-Werror` which can prevent reaching the interesting warning entirely), and whether this is an issue in the codebase (e.g. passing bytes where the library really works in terms of strings) or whether it would be possible to work around the warning by using some other API. In that case, the ability to show a full stack trace from the `stacklevel` down is very useful to diagnose such issues. Not quite sure how it would be managed though: I'd think this should be part of the warnings filter information, but the `stacklevel` currently isn't stored there, and it might be risky to extend the warnings filter with a 6th field). -- components: Library (Lib) messages: 388914 nosy: xmorel priority: normal severity: normal status: open title: Support full stack trace extraction in warnings. type: enhancement versions: Python 3.10 ___ Python tracker <https://bugs.python.org/issue43527> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue43526] Programmatic management of BytesWarning doesn't work for native triggers.
Xavier Morel added the comment: Addendum: is there a way to force `-b` from within the running Python program? -- ___ Python tracker <https://bugs.python.org/issue43526> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue43526] Programmatic management of BytesWarning doesn't work for native triggers.
New submission from Xavier Morel :
When setting `BytesWarning` programmatically (via the warnings API), though the
`warnings.filters` value matches what's obtained via `python -b` and an
explicit `warnings.warn` trigger will trigger, "native" triggers of the warning
fail to trigger properly:
import warnings
warnings.simplefilter('default', category=BytesWarning)
str(b'')
warnings.warn('test', category=BytesWarning)
If run using `python`, this will print:
test.py:4: BytesWarning: test
warnings.warn('test', category=BytesWarning)
There is no warning for the string-ification of the bytes instance.
If run using `python -b`, the behaviour is as one would expect:
test.py:3: BytesWarning: str() on a bytes instance
str(b'')
test.py:4: BytesWarning: test
warnings.warn('test', category=BytesWarning)
Inspecting `warnings.filters` shows now difference in their contents, in both
cases it is:
[('default', None, , None, 0), ('default', None,
, '__main__', 0), ('ignore', None, , None, 0), ('ignore', None, , None, 0), ('ignore', None, , None, 0), ('ignore', None, , None,
0)]
(in Python 3.9).
The warning module's own suggestion:
import sys
if not sys.warnoptions:
import warnings
warnings.simplefilter("default") # Change the filter in this process
also fails to enable BytesWarning.
If this is intended behaviour, which seems to be the case according to
ncoghlan's comment https://bugs.python.org/issue32230#msg307721, it should be
clearly documented, as it's rather frustrating.
--
components: Library (Lib)
messages: 388912
nosy: xmorel
priority: normal
severity: normal
status: open
title: Programmatic management of BytesWarning doesn't work for native triggers.
type: behavior
versions: Python 3.6, Python 3.7, Python 3.8, Python 3.9
___
Python tracker
<https://bugs.python.org/issue43526>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue43036] TOS-behaviour documentation is inconsistent
Change by Xavier Morel : -- title: TOS-behaviour documentation is -> TOS-behaviour documentation is inconsistent ___ Python tracker <https://bugs.python.org/issue43036> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue43036] TOS-behaviour documentation is
New submission from Xavier Morel :
I was looking at the disassembly of a fairly straightforward listcomp:
[e for e in s if e[0]]
1 0 BUILD_LIST 0
2 LOAD_FAST0 (.0)
>>4 FOR_ITER16 (to 22)
6 STORE_FAST 1 (e)
8 LOAD_FAST1 (e)
10 LOAD_CONST 0 (0)
12 BINARY_SUBSCR
14 POP_JUMP_IF_FALSE4
16 LOAD_FAST1 (e)
18 LIST_APPEND 2
20 JUMP_ABSOLUTE4
>> 22 RETURN_VALUE
6, 8 bothered me because STORE_FAST is documented as
> Stores TOS into the local co_varnames[var_num].
So it seems like it leaves TOS and thus the LOAD is unnecessary, However
looking at ceval.c:
case TARGET(STORE_FAST): {
PREDICTED(STORE_FAST);
PyObject *value = POP();
SETLOCAL(oparg, value);
FAST_DISPATCH();
}
so STORE_FAST does pop the TOS and the LOAD_FAST is necessary. This is
confusing because there are instructions which literally have POP in their name
whose stack behaviour is documented explicitly.
Should all bytecode instructions have their stack behaviour explicitly
documented, or only those with an *odd* stack behaviour (e.g.
JUMP_IF_FALSE_OR_POP) and the rest maybe covered by a note saying that they
will pop their parameters and push back their result or somesuch?
--
Furthermore, maybe optimising `STORE_LOCAL a; LOAD_LOCAL a` to `DUP_TOP;
STORE_LOCAL a` would be useful? It obviously would have no effect on bytecode
size since wordcode, and `fastlocals[i]` would be in cache and the conditional
check likely predicted, but it seems like skipping them entirely would still be
more reliable? This idea is somewhat supported by expression assignments
already generating the latter:
>>> @dis.dis
... def foo():
... if a := thing():
... do(a)
...
3 0 LOAD_GLOBAL 0 (thing)
2 CALL_FUNCTION0
4 DUP_TOP
6 STORE_FAST 0 (a)
8 POP_JUMP_IF_FALSE 18
4 10 LOAD_GLOBAL 1 (do)
12 LOAD_FAST0 (a)
14 CALL_FUNCTION1
16 POP_TOP
>> 18 LOAD_CONST 0 (None)
20 RETURN_VALUE
This optimisation would also trigger for e.g.
[x[i] for x in xs]
or
a = foo()
if a:
# do thing
making the latter generate bytecode identical to walrus assignments at least
for the trivial case: currently the only difference (aside from line numbers)
is that the normal assignment generates STORE_FAST;LOAD_FAST while expression
assignments generate DUP_TOP;STORE_FAST.
--
assignee: docs@python
components: Documentation
messages: 385762
nosy: docs@python, xmorel
priority: normal
severity: normal
status: open
title: TOS-behaviour documentation is
type: enhancement
versions: Python 3.9
___
Python tracker
<https://bugs.python.org/issue43036>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue42644] logging.disable('WARN') breaks AsyncIO
Xavier Morel added the comment: Oh I now see you've created a PR to do essentially that, nm. -- ___ Python tracker <https://bugs.python.org/issue42644> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue42644] logging.disable('WARN') breaks AsyncIO
Xavier Morel added the comment: > I think that patching logging.disable to raise a type error immediately would > be welcome FWIW `logging` has a built-in checker / converter[0] which is already used in a bunch of places (e.g. the aforementioned setLevel), it could just be added inside `disable` in the same way it's used in setLevel[1] and would uniformly convert "level names" to proper levels or raise an error. That seems like a really easy patch / contribution. [0] https://github.com/python/cpython/blob/3.9/Lib/logging/__init__.py#L189 [1] https://github.com/python/cpython/blob/3.9/Lib/logging/__init__.py#L906 -- ___ Python tracker <https://bugs.python.org/issue42644> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue42644] logging.disable('WARN') breaks AsyncIO
Xavier Morel added the comment: The problem seems to be in the user code? As you were told by "Carreau", loggin.disable takes a logging level (an integer), since you're giving it a string which it dutifully stores, it blows up at the next logging call which happens to be in asyncio. This is not an asyncio bug, nor is it a crash anymore than passing a broken key function to `sorted` is a *Python* crash. At most it's an enancement request: `logging.disable` could raise a TypeError immediately or convert the value to a loglevel the way `setLevel` does instead of storing the broken value as-is (all it does currently is store the value it receives on the root logger without any check). -- nosy: +xmorel ___ Python tracker <https://bugs.python.org/issue42644> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue42572] Better path handling with argparse
Xavier Morel added the comment: > What exactly do you do with a path argument? Because they mention "convert[ing] the string to a path", I would expect an output of `pathlib.Path`, optionally checked for existence / non-existence and / or kind (file, directory, symlink, ...). Obviously it is rather easy to bring your own, but OP's expectation is that paths input is common enough (especially in smaller standalone scripts I would expect) that having a helper in a standard library would be... helpful. -- nosy: +xmorel ___ Python tracker <https://bugs.python.org/issue42572> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue42470] DeprecationWarning triggers for sequences which happen to be sets as well
Xavier Morel added the comment: Tried patterning the PR after the one which originally added the warning. Wasn't too sure how the news item was supposed to be generated, and grepping the repository didn't reveal any clear script doing that, so I made up a date and copied an existing random bit (which I expect is just a deduplicator in case multiple NEWS items are created at the same instant?) -- ___ Python tracker <https://bugs.python.org/issue42470> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue42470] DeprecationWarning triggers for sequences which happen to be sets as well
Change by Xavier Morel : -- pull_requests: +22532 pull_request: https://github.com/python/cpython/pull/23665 ___ Python tracker <https://bugs.python.org/issue42470> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue42470] DeprecationWarning triggers for sequences which happen to be sets as well
Xavier Morel added the comment: I was preparing to open the PR but now I'm doubting: should I open the PR against master and miss islington will backport it, or should I open the PR against 3.9? -- ___ Python tracker <https://bugs.python.org/issue42470> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue42470] DeprecationWarning triggers for sequences which happen to be sets as well
Xavier Morel added the comment: > Do you want to submit a PR for this? Sure. Do you think the code I proposed would be suitable? > * The current logic matches the logic before the warning was added. > * The proposed logic matches what the code will do after the > deprecation period (it will only check for non-sequences). Yes, that was my basis for it as it seemed sensible, but you're right that it's a bit of a behavioural change as you note: > * There is some value in the warning in that it lets you know an > inefficient code path is being used (i.e. the conversion to a tuple). > * The proposed logic doesn't just change the warning, it changes > what actually happens to the data. IMO the change is for the > better, but it is a behavior change and could potentially cause > a failure in someone's code. Aye, and also I guess the "sequence" implementation of the input collection might be less efficient than one-shot converting to a set and sampling from the set. > * The case of an object that is both a sequence and a set is > likely very rare. Chances are you're right, but it's what got me to stumble upon it ($dayjob makes significant use of a "unique list / ordered set" smart-ish collection, that collection was actually registered against Set and Sequence specifically because Python 3's random.sample typechecks those, we registered against both as the collection technically implements both interfaces so that seemed like a good idea at the time). -- ___ Python tracker <https://bugs.python.org/issue42470> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue42470] DeprecationWarning triggers for sequences which happen to be sets as well
New submission from Xavier Morel :
In 3.9, using `random.sample` on sets triggers
DeprecationWarning: Sampling from a set deprecated
since Python 3.9 and will be removed in a subsequent version.
*However* it also triggers on types which implement *both* Sequence and Set,
despite Sequence on its own being fine.
The issue is that it first checks for Set and triggers a warning, and only then
checks that the input is a sequence:
if isinstance(population, _Set):
_warn('Sampling from a set deprecated\n'
'since Python 3.9 and will be removed in a subsequent
version.',
DeprecationWarning, 2)
population = tuple(population)
if not isinstance(population, _Sequence):
raise TypeError("Population must be a sequence. For dicts or sets,
use sorted(d).")
the check should rather be:
if not isinstance(population, _Sequence):
if isinstance(population, _Set):
_warn('Sampling from a set deprecated\n'
'since Python 3.9 and will be removed in a subsequent
version.',
DeprecationWarning, 2)
population = tuple(population)
else:
raise TypeError("Population must be a sequence. For dicts or
sets, use sorted(d).")
this also only incurs a single instance check for `_Sequence` types instead of
two.
--
messages: 381885
nosy: xmorel
priority: normal
severity: normal
status: open
title: DeprecationWarning triggers for sequences which happen to be sets as well
versions: Python 3.9
___
Python tracker
<https://bugs.python.org/issue42470>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue42106] docs.python.org prioritises search horribly
Xavier Morel added the comment: Apparently it's at least a possibility on DDG's side (https://duckduckgo.com/search_box), don't know how easy it'd be to integrate in sphinx, or whether a hard dependency on an external search engine is acceptable / desirable. -- ___ Python tracker <https://bugs.python.org/issue42106> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue42106] docs.python.org prioritises search horribly
New submission from Xavier Morel : I expect it simply uses sphinx and I don't know if sphinx's search is easily customisable but the experience is really terrible when looking for the doc of a specific thing, *especially* when that thing is or is related to a builtin, which I'd expect to be extremely common. Though I expect part of the issue might also be that builtin or ABC methods are not documented *as such*, they're just written as code in tables. For instance if you search "append": https://docs.python.org/3/search.html?q=append On my end, neither list nor MutableSequence appear anywhere on this page, even scrolling down. Searching for "list": https://docs.python.org/3/search.html?q=list The documentation for the builtin "list" object also doesn't appear on the page. "Data Structures"[0] and "built-in types"[1] appear below the fold and the former is genuinely useful but also very easy to miss (I had not actually noticed it before going back in order to get the various links and try to extensively describe the issue). Neither actually links to the `list` builtin type though. Above the fold we find various "list" methods and classes from the stdlib as well as the PDB `list` comment, none of which seems like the best match for the query. And even Google doesn't help much there, most of the hits are for third-party documentation, and the one docs.python.org link is to the Data Structures page *of Python 2.7*. [0] https://docs.python.org/3/tutorial/datastructures.html?highlight=list [1] https://docs.python.org/3/library/stdtypes.html?highlight=list -- assignee: docs@python components: Documentation messages: 379203 nosy: docs@python, xmorel priority: normal severity: normal status: open title: docs.python.org prioritises search horribly type: enhancement versions: Python 3.6, Python 3.7, Python 3.8, Python 3.9 ___ Python tracker <https://bugs.python.org/issue42106> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue33387] Simplify bytecodes for try-finally, try-except and with blocks.
Xavier Morel added the comment: The 3.9 changelog mentions WITH_EXCEPT_FINISH but that seems to have ultimately been called WITH_EXCEPT_START, maybe it shoudl be updated also? -- nosy: +xmorel ___ Python tracker <https://bugs.python.org/issue33387> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue24828] Segfault when using store-context AST node in a load context
Xavier Morel added the comment: Should I close this since I believe 2.7 is not supported anymore? -- ___ Python tracker <https://bugs.python.org/issue24828> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue40325] Random.seed does not affect string hash randomization leading to non-intuitive results
Xavier Morel added the comment: @rhettinger checking software against 3.9 there's a little issue with the way the check is done: if passed something which is *both* a sequence and a set (e.g. an ordered set), `random.sample` will trigger a warning, which I don't think is correct. Should I open a new issue for that? Fix seems simple: just move the check for _Set inside the check for _Sequence, and raise if that doesn't pass either. -- nosy: +xmorel ___ Python tracker <https://bugs.python.org/issue40325> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue42033] Seemingly unnecessary complexification of foo(**kw)
Xavier Morel added the comment: I have not noticed anything, I was just looking at the bytecode changes and stumbled upon this oddity. Though I would expect a small slowdown as every fn(**kw) would now incur an extra dict copy, unless there’s something in call_function_ex which copies the input dict iff its ref count is not one? For whatever that’s worth, the 3.8 bytecode has been there since call_function_ex was added in 3.6 and before that call_function_kw looks identical (load_global foo, load_local var, call_function_kw) -- ___ Python tracker <https://bugs.python.org/issue42033> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue42033] Seemingly unnecessary complexification of foo(**kw)
New submission from Xavier Morel : Following bpo-39320 the highly specialised bytecode for vararg calls were replaced by simpler ones, but there seems to be at least one area where the generated bytecode regressed for possibly no reason? In Python 3.8, foo(**var) compiles to: 0 LOAD_GLOBAL 0 (foo) 2 BUILD_TUPLE 0 4 LOAD_FAST2 (var) 6 CALL_FUNCTION_EX 1 In Python 3.9, it compiles to: 0 LOAD_GLOBAL 0 (foo) 2 BUILD_TUPLE 0 4 BUILD_MAP0 6 LOAD_FAST2 (var) 8 DICT_MERGE 1 0 CALL_FUNCTION_EX 1 The PR 18141 does not seem to change the implementation of CALL_FUNCTION_EX so I would expect that if it was fine with taking the `var` arbitrary mapping before it stil is now, and the extra two opcodes (and creation of a dict) is unnecessary? -- messages: 378613 nosy: Mark.Shannon, xmorel priority: normal severity: normal status: open title: Seemingly unnecessary complexification of foo(**kw) versions: Python 3.9 ___ Python tracker <https://bugs.python.org/issue42033> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue41641] Add a "message" action to warnings, to trigger for every *unique* message
New submission from Xavier Morel :
Warning actions allow deduplicating warning triggers based on category
("once"), category + file ("module") and category + exact location ("default").
One thing which is missing is support for a single location generating warnings
*on behalf* of other pieces of code.
`warnings.warn` provides some support via the "stacklevel" parameter, but it is
difficult to compute if the warning is generated from somewhat nested generic
code (or even impossible if the warning is generated from completely generic
code checking over e.g. data files or the like, or if the invoker could be
invoked from multiple places with different depth difference from the code
they're working for).
But in that case, the warning messages will often be unique per functional
unit, so it could be a useful base for deduplication, even though the warning
could dynamically be invoked multiple times (per functional unit) because e.g.
code is reloaded or whatever.
--
components: Library (Lib)
messages: 375924
nosy: xmorel
priority: normal
severity: normal
status: open
title: Add a "message" action to warnings, to trigger for every *unique* message
type: enhancement
versions: Python 3.9
___
Python tracker
<https://bugs.python.org/issue41641>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue30951] Documentation error in inspect module
Xavier Morel added the comment: Maybe something along the lines of "names other than arguments and function locals", or "names of the symbols used in the code object, other than arguments and function locals"? This is still slightly confusing because in the case of an import the name is present in both mappings, but less so than the outright lie of the current version. Serhiy, is there a way to access the class's code object from Python? Also does that mean varnames is not used by class code objects despite it stating unambiguously that it stores locals? -- nosy: +xmorel ___ Python tracker <https://bugs.python.org/issue30951> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue30535] Warn that meta_path is not empty
Xavier Morel added the comment: > Fair enough. So we can just add a sentence informing readers that `meta_path`, by default, holds entries to handle the standard kinds of modules (.py files, extension modules…). Yeah that's basically what I meant, talking about a "warning" in python 3 may have been the wrong wording or given an incorrect impression, sorry about that. -- ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue30535> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue30535] Warn that meta_path is not empty
Xavier Morel added the comment: > A warning is probably too strong. Also, it's easy to check sys.meta_path at > the interpreter prompt, so I'm not sure it's worth mentioning at all. It's easy if you think of it and did not miss a small bit of the Python 3.3 release note indicating that the *documented guarantees* of Python 2.x, 3.0, 3.1 and 3.2 had been dropped, even as they were incorrectly still present in the official documentation itself. I spent a few hours trying to understand why our import hooks did not work correctly anymore in 3.5, and I never even considered printing sys.meta_path in an open interpreter, I realised the behavioural change because I ended up printing meta_path after the addition of our custom hooks to make sure they were actually there. -- ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue30535> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue30535] Warn that meta_path is not empty
Xavier Morel added the comment: > I'm fine with adding a note to the Python 2 docs, but putting it in Python 3 > seems to be going in the wrong direction as having sys.meta_path not be empty > is how Python will be going forward. I don't think putting a note is any hint that the Python 3 behaviour (which I appreciate) would change, it would simply tell the reader that the list is not empty by default and they ought decide whether they want their finders to take precedence over (and insert at head) or be fallback to (and append) the builtin finders. It could also link to the standard/builtin finders. -- ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue30535> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue29920] Document cgitb.text and cgitb.html
Xavier Morel added the comment: Should I close this since the PR was merged? -- ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue29920> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue30535] Warn that meta_path is not empty
Xavier Morel added the comment: And it turns out the change was noted in the Python 3.3 release notes[0] though not all the consequences were spelled out (and the meta_path and path_hooks documentations were not changed) [0] https://docs.python.org/3/whatsnew/3.3.html#visible-changes -- ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue30535> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue30535] Warn that meta_path is not empty
Xavier Morel added the comment: Addendum: the warning was present (in the documentation) until Python 3.5, even though Python 3.3 is apparently where the "default finders" were moved to meta_path: > python3.2 -c 'import sys;print(sys.meta_path)' [] > python3.3 -c 'import sys;print(sys.meta_path)' [, , ] -- ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue30535> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue30535] Warn that meta_path is not empty
New submission from Xavier Morel: Encountered this issue porting Python 2 code manipulating meta_path to Python 3. In Python 2, meta_path is empty by default and its documentation specifically notes that: > sys.meta_path is searched before any implicit default finders or sys.path. As a result, sys.meta_path.append() works perfectly well and is overwhelmingly common[0]. In Python 3, this note was removed but the documentation does not specifically state default finders are present in meta_path, and the old code using sys.meta_path.append may now break as the default finders will silently take precedence on the custom ones if they believe they can do the job. I'd like to add a warning to the Python 3 documentation (3.5+?) noting the presence of existing default loaders, and to the Python 2 documentation suggesting `sys.meta_path.insert(0, finder)` for future-proofing, would there be objections or issues? [0] https://github.com/search?q=meta_path.append=Code=✓ -- assignee: docs@python components: Documentation messages: 294919 nosy: docs@python, xmorel priority: normal severity: normal status: open title: Warn that meta_path is not empty type: behavior versions: Python 2.7, Python 3.5, Python 3.6 ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue30535> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue29920] Document cgitb.text and cgitb.html
Changes by Xavier Morel <xavier.mo...@masklinn.net>: -- pull_requests: -747 ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue29920> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue29920] Document cgitb.text and cgitb.html
Xavier Morel added the comment: PR targetted to master rather than 2.7 -- pull_requests: +748 ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue29920> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue29920] Document cgitb.text and cgitb.html
New submission from Xavier Morel: Currently, cgitb documents the hook (enable) and somewhat unclearly the ability to dump the HTML traceback to stdout, but despite that being technically available it does not document the ability to dump the traceback to a string as either text or html. Possible further improvement: make ``cgitb.html`` and ``cgitb.text`` implicitly call `sys.exc_info()` if not given a parameter (much like `cgitb.handler` does). -- assignee: docs@python components: Documentation messages: 290608 nosy: docs@python, xmorel priority: normal pull_requests: 747 severity: normal status: open title: Document cgitb.text and cgitb.html type: enhancement versions: Python 2.7, Python 3.6, Python 3.7 ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue29920> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue24828] Segfault when using store-context AST node in a load context
New submission from Xavier Morel: It looks to be fixed in 3.3 and up, but in Python 2.7 import ast m = ast.Module(body=[ ast.Expr(value=ast.Name(id='foo', ctx=ast.Store())) ]) ast.fix_missing_locations(m) code = compile(m, '', mode='exec') eval(code) will segfault on eval. So will a similarly incorrect ast.Attribute node. Version tested: Python 2.7.10 (default, May 28 2015, 12:02:55) [GCC 4.2.1 Compatible Apple LLVM 6.0 (clang-600.0.57)] on darwin -- components: Library (Lib) messages: 248269 nosy: xmorel priority: normal severity: normal status: open title: Segfault when using store-context AST node in a load context type: crash versions: Python 2.7, Python 3.2 ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue24828 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue13405] Add DTrace probes
Changes by Xavier Morel xavier.mo...@masklinn.net: -- nosy: +xmorel ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue13405 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue14776] Add SystemTap static markers
Changes by Xavier Morel xavier.mo...@masklinn.net: -- nosy: +xmorel ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue14776 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue21590] Systemtap and DTrace support
Changes by Xavier Morel xavier.mo...@masklinn.net: -- nosy: +xmorel ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue21590 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue15639] csv.Error description is incorrectly broad
Xavier Morel added the comment: Correction: csv also seems to raise csv.Error if the file contains NUL bytes: Error: line contains NULL byte -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue15639 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue15639] csv.Error description is incorrectly broad
New submission from Xavier Morel: In both Python 2.7 and Python 3.x, csv.Error is documented as: Raised by any of the functions when an error is detected. As far as I can tell from using the module and looking at the code, this is completely incorrect. There is actually a single instance of csv.Error being used: the instantiation of csv.Dialect (which converts TypeError raised from _csv._Dialect() into csv.Error, a comment notes that this is for compatibility with py 2.3). And the only way to hit that code paths seems to be subclassing `Dialect` and putting incorrect values in the various attributes (providing them to `csv.reader` raises a TypeError). I believe the documentation to csv.Error should be changed to: 1. Mark it as effectively deprecated 2. Indicate that the only situation in which it it may be raised is when initializing a subclass of csv.Dialect -- assignee: docs@python components: Documentation messages: 168096 nosy: docs@python, xmorel priority: normal severity: normal status: open title: csv.Error description is incorrectly broad versions: Python 2.7, Python 3.3 ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue15639 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue13850] Summary tables for argparse add_argument options
Xavier Morel xavier.mo...@masklinn.net added the comment: Had some time to play with this today, here's a draft matrix of actions and add_argument parameters which is pretty readable, but: * It's incredibly not helpful for people who don't know argparse * I tried adding effects descriptions in the cells instead of mere tick marks, the table becomes completely unreadable. I added a note directive below the table but it only lists a few really important/weird things, and it really won't scale beyond the current 3 items (which might already be too much) * I completely removed the ``help`` action from the table as it's unlikely anyone will want to override it (and its row was completely blank) * Hyperlinking and cross-linking (to the params, the actions, footnotes) would probably be a good idea, although it would definitely make the raw text (in-rst) I also tried my hand at formatting ``nargs``, but I don't see it as much clearer than http://docs.python.org/library/argparse.html#nargs without the examples, it's still just a mapping from a value to a behavior. I think the result would be just as good if the current ``nargs`` description was made into a definition list (in effect, it already is one emulated through an unordered list) and the examples were moved or removed. -- Added file: http://bugs.python.org/file25018/argparse-actions-matrix ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue13850 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue13850] Summary tables for argparse add_argument options
Xavier Morel xavier.mo...@masklinn.net added the comment: completion for list item 4: although it would definitely make the raw text (in-rst) much harder to read compared to the current table (which can be used from the rst source without compiling) -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue13850 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue13850] Summary tables for argparse add_argument options
Xavier Morel xavier.mo...@masklinn.net added the comment: Creating the tables should not be too hard, especially using e.g. org-mode, but: 1. Where should those tables live? The argparse documentation is pretty big and there's no completely obvious place. I would guess table 1. could just replace the list of arguments in http://docs.python.org/py3k/library/argparse.html#the-add-argument-method but things are harder for `action` (as many actions have examples as well, so the listing can't just be replaced) and for `nargs` 2. If the current lists of `argument: role` (in ArgumentParser and add_argument) are not sufficient, why would a table somehow be considering it would *add* visual clutter (table borders, for instance)? Maybe a good alternative would be to build a table style for info fields lists and to use :param: wherever that belongs? Or the doc could be split into a quickstart with just a listing of arguments and a *very simple* example or two, and then the exhaustive documentation, which could even be a separate document? -- nosy: +xmorel ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue13850 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue13850] Summary tables for argparse add_argument options
Xavier Morel xavier.mo...@masklinn.net added the comment: My specific suggestion is to have a dedicated Quick Reference section before the first example. OK, that looks like a good plan. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue13850 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue13850] Summary tables for argparse add_argument options
Xavier Morel xavier.mo...@masklinn.net added the comment: The Parameters column would span multiple lines, with one parameter and a brief description of the parameter on each line. I started looking into that, and it turns out that's more annoying than expected: a bunch of parameters are shared by many (to all) actions, leading to lots of duplications in the table. And the full matrix of actions to parameters is not really explained in the doc. In fact, I'm coming around to thinking a matrix of the interaction between actions and arguments could be better and clearer than a table of actions with parameters bunched together at the end. In any case, it would certainly be more maintainable... except for rST not really having support for attributes on data tables, and (as far as I can tell) can't handle horizontal headers. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue13850 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue12056] … (HORIZONTAL ELLIPSIS) should be an alternative syntax for ... (FULL STOP FULL STOP FULL STOP)
New submission from Xavier Morel xavier.mo...@masklinn.net: In Python 3, ... became useable as a normal expression, and translates into an ellipsis instance. Unicode defines an ellipsis character … (U+2026 HORIZONTAL ELLIPSIS) which is canonically equivalent to a 3-sequence of FULL STOP [U+002E U+002E U+002E] I think it would be nice if Python supported … as an alternative to ... -- components: Interpreter Core, Unicode messages: 135771 nosy: xmorel priority: normal severity: normal status: open title: … (HORIZONTAL ELLIPSIS) should be an alternative syntax for ... (FULL STOP FULL STOP FULL STOP) type: feature request versions: Python 3.2 ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue12056 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue11979] Minor improvements to the Sockets readme: typos, wording and sphinx features usage
New submission from Xavier Morel xavier.mo...@masklinn.net: First patch fixes a typo (the reads a reply - then reads a reply) and — I believe — improves the wording of a pair of sentences. Second patch makes use of Sphinx's ``:abbr:`` role, and removes some period which I think are redundant with the use of ``::`` -- assignee: docs@python components: Documentation files: wording messages: 134970 nosy: docs@python, xmorel priority: normal severity: normal status: open title: Minor improvements to the Sockets readme: typos, wording and sphinx features usage versions: Python 3.3 Added file: http://bugs.python.org/file21853/wording ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue11979 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue11979] Minor improvements to the Sockets readme: typos, wording and sphinx features usage
Changes by Xavier Morel xavier.mo...@masklinn.net: Added file: http://bugs.python.org/file21854/sphinx-features ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue11979 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue11260] smtpd-as-a-script feature should be documented and should use argparse
Xavier Morel xavier.mo...@masklinn.net added the comment: Do tests currently exist for smtpd run as a script? I have to confess I didn't think to check. If not, our experience with converting compileall to argparse indicates a thorough test suite is needed (and even so we missed some things we hadn't thought to test). OK, so if there is no test suite currently I should create one, and if there is one I should ensure it's complete? I guess I should use compileall as an example of how to test modules-as-scripts if the former? Overall, smtpd-as-a-script is really pretty simple, it totals 28 lines apart from the argument parsing itself (which is a bit under 60 lines ignoring the help text/pattern and gets a bit under 50 including it post-patch), and as I mentioned the only part which actually needed changing is the arguments parsing, which was very well factored out. In other words, if the current code works, is updating it a sufficient reason to change it, considering the chances of introducing new bugs? I'm not sure, but one of the ways I see standard libraries is not just as ready to use code, it's also as a repository of how to do things. And to that end, if argparse is now considered the canonical way to parse command-line arguments (which I would expect), there are very few examples of how to do things in the stdlib itself (though there are examples outside of it, due to the life argparse had pre-stdlib). It also rose to the occasion as I was wondering about the numerous standard library modules-as-scripts which are either undocumented or under-documented: because it had a good command-line documentation and a clean separation between the configuration (options parsing) and the actual execution, but no module documentation (in Doc/) it seemed like a good starting point: if it's not feasible to correctly convert best cases (and smtpd is probably one, short of modules using optparse probably) then the whole idea is stillborn: I do not see how it would be possible to fare better on some of the fully undocumented modules using manual options parsing, yet it would have to be expected. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue11260 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue11260] smtpd-as-a-script feature should be documented and should use argparse
Xavier Morel xavier.mo...@masklinn.net added the comment: any of the undocumented command-line interfaces are intentionally undocumented -- they were there for the convenience of the developer for exercising the module as it was being developed and are not part of the official API. I can understand that, but it's not clear from the source code which is which, and for several of them third-parties have taken up the convenience. Maybe a more formal process of defining which tools are to be considered official and which are easter-eggs is needed? In any case, even for easter eggs surely a good command-line documentation is a good idea (even if the module documentation side is not added due to the easter egg status) isn't it? The standard library's primary purpose is to be a library for use in user programs, not to serve as a suite of applications. Sure, and I don't think most of the module-as-scripts have what it takes to be seen as applications, but many are useful and/or interesting helpers/shortcuts for day to day tasks. http.server and smtpd are good examples of useful modules-as-scripts (http.server being the exact opposite of smtpd in that it has a module-as-script documentation but no documentation whatsoever on the command-line) Real apps need more features than we usually offer and they need much faster development cycles than supported by our 18-24 month release cycle. I think my suggestion has been misunderstood: I don't want to turn these into real apps, they're fine as basic scripts/helpers to spend 5mn on a task instead of half an hour. I think they deserve better than to be documented and known through StackOverflow or Reddit what are the -m stdlib features lists. To the extent the standard library serves as a set of examples, that is just side benefit. There are other ways to make code specifically for examples (we include tons of examples in the documentation; we had a Demos directory; there is the wiki; and there is the ASPN Cookbook site; etc) Sure, but I think it's still an important driver of how things are done in that they're *concrete* examples, not abstract (of course the Cookbook is generally concrete as well). I'm not discounting the importance or quality of the rest of the documentation at all, or at least that was not my intention. All that being said, there are some exceptions and it make may sense to document the interface in some where we really do want a command-line app. I'll look at any patches you want to submit, but try to not go wild turning the library into a suite of applications. For the most part, that is not what the standard library is about. As I said, my only intention here is be to document (and argparsify/formalize) what is already there. I considered doing more (e.g. for the specific case of smtpd-as-a-script making ports optional even when hosts are specified, that kind of things) but decided against this distraction, I really just want to make existing module-as-script features simpler to discover and use. That said, do you have guidelines of which areas this idea would be most interestingly/efficiently applied? Maybe a list of modules-as-scripts you know are used regularly and could benefit from some improvements in their interface or documentation? -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue11260 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue11260] smtpd-as-a-script feature should be documented and should use argparse
Xavier Morel xavier.mo...@masklinn.net added the comment: One more note I forgot previously: the conversion of as much scripts as possible to argparse would be for three reasons: * Make behavior consistent across the board (e.g. -h/--help) * Make CLI documentation format consistent across the board either so users know what to expect and when * Provide easy to reach (for users) examples of using argparse -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue11260 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue11260] smtpd-as-a-script feature should be documented and should use argparse
Xavier Morel xavier.mo...@masklinn.net added the comment: Barry, do I correctly understand your comment to mean I should write end-to-end tests of the CLI (until reaching the already tested meat of smtpd), not just the CLI options parsing? -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue11260 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue11260] smtpd-as-a-script feature should be documented and should use argparse
Xavier Morel xavier.mo...@masklinn.net added the comment: Only document and formalize the parts that are well thought out. I don't believe I have the knowledge, right or ability to make that call for any module or package but a pair of extremely obvious ones (http.server seems a pretty good candidate as it's documented in the module doc, and is just missing a CLI help). Should I create a bug and nosy the people who have been involved in the module to get their opinion for each one? (as with this one, but maybe with a different wording/initial proposal) For this one, David expresses concerns on the stability on the interface (and the point of the idea), you express even bigger concerns with the idea itself and more generally modules-as-scripts in the stdlib (not their existence so much as their official support, which full documentation would imply, if I didn't misunderstand you), and Barry seems to okay the idea as long as an extensive test suite is created beforehand to ensure there is no regression, is that sufficient to go ahead with more work and defer the final decision for when that will be done? Also, if this is to become an actual project if the smtpd stuff ever reaches fruition (though not necessarily a big or impactful one), should there be a meta-bug? (I know they're used in many bugzilla projects and I see the tracker handles dependencies) There's no harm in adding --help or a usage example, but please avoid making behavior guarantees that we really don't want to have to live with. I'm not sure what that results in: for quick hacks which are not to be officially supported, does this mean fixing the CLI (adding a help for tools missing one) is OK but no more? Is an argparse switch still acceptable? Tests for the CLI tool? I'm guessing documentation in the module would definitely be off limits for those, is my interpretation correct? And again, who would be allowed to make the call on which modules-as-scripts may be considered supported, and can't be? With this one, I created separate patches for the documentation and the CLI parsing alterations, which would allow merging the CLI without adding it to the official documentation (which would I think imply a lack of official support), would that be OK for future works, if this one pans out? -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue11260 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue11260] smtpd-as-a-script feature should be documented and should use argparse
New submission from Xavier Morel xavier.mo...@masklinn.net: argparse has been merged to the standard library in 3.2, and (tell me if I'm wrong) would therefore be the best-practices way to parse command-line arguments. Numerous stdlib modules can be used as scripts, but they tend to have ad-hoc documentation (if they are at all documented) and arguments parsing (using any of the 4 available methods in Python: straight from sys.argv, getopt, optparse and argparse). I picked smtpd as a first shot since it does something useful (SMTP proxy) and has a pretty good (if ad-hoc) command-line documentation. smtpd is currently using getopt for its options parsing and the argument parsing is very cleanly factored: the port only had to replace the implementation of the `parseargs` function (and add and remove some helpers). * The port keeps the existing arguments semantics (including the mandatory host:port syntax for the local and remote specs if overridden) * The port tries to maintain the old error messages, but due to the way argparse works (or the way I used it for the specs) the parity is not perfect when providing incorrect specs * The CLI help uses argparse's formatting, and the documentation for the local and remote specs was set on these arguments rather than as an epilog. The version string was also removed from the help screen * Because they are set by argparse's arguments validation, the status codes on incorrect arguments are slightly different: - running smtpd.py as a regular user without the `--nosetuid` flag still exits with status 1 - providing incorrect spec formats (missing or non-int port) or providing too many positional arguments (3 or more) now exits with status 2 (formerly 1) -- assignee: docs@python components: Documentation, Library (Lib) files: smtpd-to-argparse.diff keywords: patch messages: 128917 nosy: barry, docs@python, xmorel priority: normal severity: normal status: open title: smtpd-as-a-script feature should be documented and should use argparse versions: Python 3.3 Added file: http://bugs.python.org/file20812/smtpd-to-argparse.diff ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue11260 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue11260] smtpd-as-a-script feature should be documented and should use argparse
Xavier Morel xavier.mo...@masklinn.net added the comment: Second patch: documenting smtpd-as-a-script in the module's rst -- Added file: http://bugs.python.org/file20813/smtpd-as-script-doc.diff ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue11260 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
