Re: [Python-Dev] PEP 457: Syntax For Positional-Only Parameters
On Tue, Oct 8, 2013 at 9:02 PM, Terry Reedy tjre...@udel.edu wrote: I am for having a way to succintly properly describe the signature of C in the manual and docstrings and help output. As it is now, the only safe thing to do, without trial and exception, is to assume positional only unless one knows otherwise. But the syntax should only be as complicated as needed for 'most' C functions. If there are a few complicated oddballs, perhaps they could be simplified, or given a special-case explanation. +1 -eric ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 457: Syntax For Positional-Only Parameters
On Wed, 09 Oct 2013 01:33:26 +0200 Larry Hastings la...@hastings.org wrote: I've contributed a new PEP to humanity. I include the RST for your reading pleasure below, but you can also read it online here: http://www.python.org/dev/peps/pep-0457/ Only skimmed through the PEP, but -1 on the whole idea. I'm also -1 on the / separator for docs, it's obscure and brings nothing to the table but pointless complication. Regards Antoine. ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 457: Syntax For Positional-Only Parameters
On 10/09/2013 03:57 AM, Joao S. O. Bueno wrote:
def a([b],/,**kw):
print (b)
does calling a(b=5) should raise a TypeError, or put 5 in
kw[b] and leave
b as undefined ?
The latter. This is how they are similarly to *args and **kwargs:
def foo(*a, **kw):
... print(a, kw)
...
foo(a=3, kw=5)
() {'kw': 5, 'a': 3}
//arry/
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Reduce memory footprint of Python
Le Tue, 8 Oct 2013 15:43:40 -0400, Benjamin Peterson benja...@python.org a écrit : 2013/10/8 R. David Murray rdmur...@bitdance.com: In this context, if we'd been *really* smart-lazy in CPython development, we'd have kept the memory and startup-time and...well, we probably do pretty well on CPU actually...smaller, so that when smartphones came along Python would have been the first high level language used on them, because it fit. Then we'd all be able to be *much* lazier now :) Even on desktop, startup time leaves a lot to be desired. That's true. Anyone have any ideas to improve it? ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Reduce memory footprint of Python
Le Wed, 09 Oct 2013 11:01:01 +1300, Greg Ewing greg.ew...@canterbury.ac.nz a écrit : R. David Murray wrote: I can give you one data point: a mobile platform that (currently) uses Python3, and does not use linecache because of how much memory it consumes. Wouldn't a sensible approach be to discard the linecache when you've finished generating a traceback? You're not likely to be generating tracebacks often enough to be worth keeping it from one to the next. The linecache is also used by the warnings module. I think the sensiblest approach would be to limit its size (LRU-style). Regards Antoine. ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 457: Syntax For Positional-Only Parameters
The PEP needs to state more clearly up front (preferably in the title) that it's about *reserving* a Python level syntax that matches the syntax we worked out for Argument Clinic at PyCon US. Explicitly stating that the requirements that drive the Argument Clinic design are to support the signature of all current CPython builtins and extension modules would also be helpful. Generally, it needs to be a bit clearer that the intent of the PEP isn't to say let's do this, it's to be explicit that acceptance of the Argument Clinic PEP severely constrains the design space for possible solutions if we ever *did* implement Python level support for positional only arguments. A few other specific points: - parameters in optional groups should just have an implied =None that can be overriden if desired. - parsing positional only args in Python is actually pretty easy in most cases, as you can just use a second private function to do the heavy lifting. - a simpler variant that omits the grouping support and only allows optional parameters on the right should also be reserved - explicitly delegate to the argument clinic PEP for the impact on inspect.Signature Cheers, Nick. ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 457: Syntax For Positional-Only Parameters
On 9 October 2013 10:46, Nick Coghlan ncogh...@gmail.com wrote: The PEP needs to state more clearly up front (preferably in the title) that it's about *reserving* a Python level syntax that matches the syntax we worked out for Argument Clinic at PyCon US. Explicitly stating that the requirements that drive the Argument Clinic design are to support the signature of all current CPython builtins and extension modules would also be helpful. Generally, it needs to be a bit clearer that the intent of the PEP isn't to say let's do this, it's to be explicit that acceptance of the Argument Clinic PEP severely constrains the design space for possible solutions if we ever *did* implement Python level support for positional only arguments. I'm confused about the expectations here. To be specific, I hate the syntax, I find the proliferation of punctuation messy and unhelpful. But if it's only for documentation/internal use, I don't really mind - I can see that we need some way to express this stuff, and TBH I don't care enough to make a fuss either way. But if the intention is that any attempt to add the functionality to Python will be constrained to use the syntax here, surely that means we need to have all of the usability arguments and bikeshedding *now*. otherwise the opportunity is lost? Personally, I'm -1 on the idea that this acts as any sort of precedent or design constraint for a Python-level implementation of the functionality, and -0 on it as a syntax for Argument Clinic, and as a documentation convention. And either way, I agree with Nick that the PEP needs to be very clear about its scope and impact. Paul ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] type.__subclasses__() doesn't work
Hello, Just noticed the following quirk: type.__subclasses__() Traceback (most recent call last): File stdin, line 1, in module TypeError: descriptor '__subclasses__' of 'type' object needs an argument Yet it would be nice to know about the subclasses of type. Regards Antoine. ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Python startup time
Le Wed, 9 Oct 2013 10:29:30 +0200,
Antoine Pitrou solip...@pitrou.net a écrit :
Le Tue, 8 Oct 2013 15:43:40 -0400,
Benjamin Peterson benja...@python.org a écrit :
2013/10/8 R. David Murray rdmur...@bitdance.com:
In this context, if we'd been *really* smart-lazy in CPython
development, we'd have kept the memory and startup-time
and...well, we probably do pretty well on CPU actually...smaller,
so that when smartphones came along Python would have been the
first high level language used on them, because it fit. Then
we'd all be able to be *much* lazier now :)
Even on desktop, startup time leaves a lot to be desired.
That's true. Anyone have any ideas to improve it?
It's difficult to identify significant contributors but some possible
factors:
- marshal.loads() has become twice slower in 3.x (compared to 2.7)
- instantiating a class is slow (type('foo', (), {}) takes around 25ms
here)
Regards
Antoine.
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] type.__subclasses__() doesn't work
On Wed, Oct 09, 2013 at 12:20:18PM +0200, Antoine Pitrou wrote: Hello, Just noticed the following quirk: type.__subclasses__() Traceback (most recent call last): File stdin, line 1, in module TypeError: descriptor '__subclasses__' of 'type' object needs an argument Yet it would be nice to know about the subclasses of type. py type.__subclasses__(type) [class 'abc.ABCMeta', class 'string._TemplateMetaclass'] -- Steven ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] type.__subclasses__() doesn't work
Steven D'Aprano wrote: On Wed, Oct 09, 2013 at 12:20:18PM +0200, Antoine Pitrou wrote: Hello, Just noticed the following quirk: type.__subclasses__() Traceback (most recent call last): File stdin, line 1, in module TypeError: descriptor '__subclasses__' of 'type' object needs an argument Yet it would be nice to know about the subclasses of type. py type.__subclasses__(type) [class 'abc.ABCMeta', class 'string._TemplateMetaclass'] The underlying problem seems to be that there is no helper function to bypass the instance attribute. Compare: class T(type): ... def __len__(self): return 0 ... class A(metaclass=T): ... def __len__(self): return 1 ... A.__len__() Traceback (most recent call last): File stdin, line 1, in module TypeError: __len__() missing 1 required positional argument: 'self' len(A) 0 So should there be a subclasses() function, in the operator module perhaps? ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] type.__subclasses__() doesn't work
On 10/09/2013 02:22 PM, Peter Otten wrote: py type.__subclasses__(type) [class 'abc.ABCMeta', class 'string._TemplateMetaclass'] The underlying problem seems to be that there is no helper function to bypass the instance attribute. Note that the problem is specific to the type type, which is its own metatype. With other types that get __subclasses__ from type, is no problem with just calling __subclasses__(): int.__subclasses__() [type 'bool'] ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 457: Syntax For Positional-Only Parameters
2013/10/9 Nick Coghlan ncogh...@gmail.com: The PEP needs to state more clearly up front (preferably in the title) that it's about *reserving* a Python level syntax that matches the syntax we worked out for Argument Clinic at PyCon US. Explicitly stating that the requirements that drive the Argument Clinic design are to support the signature of all current CPython builtins and extension modules would also be helpful. Generally, it needs to be a bit clearer that the intent of the PEP isn't to say let's do this, it's to be explicit that acceptance of the Argument Clinic PEP severely constrains the design space for possible solutions if we ever *did* implement Python level support for positional only arguments. Why does a syntax need to be reserved? Documentation conventions and the syntax of internal tools like argument clinic may be changed any time we like. -- Regards, Benjamin ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Python startup time
2013/10/9 Antoine Pitrou solip...@pitrou.net:
Le Wed, 9 Oct 2013 10:29:30 +0200,
Antoine Pitrou solip...@pitrou.net a écrit :
Le Tue, 8 Oct 2013 15:43:40 -0400,
Benjamin Peterson benja...@python.org a écrit :
2013/10/8 R. David Murray rdmur...@bitdance.com:
In this context, if we'd been *really* smart-lazy in CPython
development, we'd have kept the memory and startup-time
and...well, we probably do pretty well on CPU actually...smaller,
so that when smartphones came along Python would have been the
first high level language used on them, because it fit. Then
we'd all be able to be *much* lazier now :)
Even on desktop, startup time leaves a lot to be desired.
That's true. Anyone have any ideas to improve it?
It's difficult to identify significant contributors but some possible
factors:
- marshal.loads() has become twice slower in 3.x (compared to 2.7)
- instantiating a class is slow (type('foo', (), {}) takes around 25ms
here)
There's also the increasing number of modules (Python or otherwise)
that have to be loaded on startup.
--
Regards,
Benjamin
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Python startup time
Le Wed, 9 Oct 2013 08:39:40 -0400,
Benjamin Peterson benja...@python.org a écrit :
2013/10/9 Antoine Pitrou solip...@pitrou.net:
Le Wed, 9 Oct 2013 10:29:30 +0200,
Antoine Pitrou solip...@pitrou.net a écrit :
Le Tue, 8 Oct 2013 15:43:40 -0400,
Benjamin Peterson benja...@python.org a écrit :
2013/10/8 R. David Murray rdmur...@bitdance.com:
In this context, if we'd been *really* smart-lazy in CPython
development, we'd have kept the memory and startup-time
and...well, we probably do pretty well on CPU
actually...smaller, so that when smartphones came along Python
would have been the first high level language used on them,
because it fit. Then we'd all be able to be *much* lazier
now :)
Even on desktop, startup time leaves a lot to be desired.
That's true. Anyone have any ideas to improve it?
It's difficult to identify significant contributors but some
possible factors:
- marshal.loads() has become twice slower in 3.x (compared to 2.7)
- instantiating a class is slow (type('foo', (), {}) takes around
25ms here)
There's also the increasing number of modules (Python or otherwise)
that have to be loaded on startup.
See e.g. http://bugs.python.org/issue9548
Regards
Antoine.
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Python startup time
Am 09.10.2013 14:39, schrieb Benjamin Peterson: There's also the increasing number of modules (Python or otherwise) that have to be loaded on startup. I have gathered some stats: Python 3.4: len(sys.modules) 56 len(list(name for name in sys.modules if name not in sys.builtin_module_names)) 36 sorted(sys.modules) ['__main__', '_codecs', '_collections', '_frozen_importlib', '_functools', '_heapq', '_imp', '_io', '_operator', '_sitebuiltins', '_sre', '_stat', '_sysconfigdata', '_thread', '_warnings', '_weakref', '_weakrefset', 'abc', 'atexit', 'builtins', 'codecs', 'collections', 'collections.abc', 'copyreg', 'encodings', 'encodings.aliases', 'encodings.latin_1', 'encodings.utf_8', 'errno', 'functools', 'genericpath', 'heapq', 'io', 'itertools', 'keyword', 'marshal', 'operator', 'os', 'os.path', 'posix', 'posixpath', 're', 'readline', 'reprlib', 'rlcompleter', 'signal', 'site', 'sre_compile', 'sre_constants', 'sre_parse', 'stat', 'sys', 'sysconfig', 'types', 'weakref', 'zipimport'] Python does quite alot of syscalls, too: $ strace ./python -E -c '' 21 | grep stat | wc -l 236 $ strace ./python -E -c '' 21 | grep open | wc -l 53 Python 2.7 is a bit better: len(sys.modules) 41 len(list(name for name in sys.modules if name not in sys.builtin_module_names)) 29 ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 457: Syntax For Positional-Only Parameters
On 10/09/2013 02:38 PM, Benjamin Peterson wrote: Why does a syntax need to be reserved? Documentation conventions and the syntax of internal tools like argument clinic may be changed any time we like. If Argument Clinic is accepted, and ships with Python, my expectation is that it would cease being an internal tool, and would start being used by external developers. //arry/ ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 457: Syntax For Positional-Only Parameters
Le Wed, 09 Oct 2013 15:32:10 +0200, Larry Hastings la...@hastings.org a écrit : On 10/09/2013 02:38 PM, Benjamin Peterson wrote: Why does a syntax need to be reserved? Documentation conventions and the syntax of internal tools like argument clinic may be changed any time we like. If Argument Clinic is accepted, and ships with Python, my expectation is that it would cease being an internal tool, and would start being used by external developers. I think it would be better to exercise it first as an internal tool, eliminate any possible quirks and issues, and *then* perhaps expose it as a public API. By the way, do you think it'll be ready for 3.4? Regards Antoine. ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 457: Syntax For Positional-Only Parameters
Am 09.10.2013 15:32, schrieb Larry Hastings: On 10/09/2013 02:38 PM, Benjamin Peterson wrote: Why does a syntax need to be reserved? Documentation conventions and the syntax of internal tools like argument clinic may be changed any time we like. If Argument Clinic is accepted, and ships with Python, my expectation is that it would cease being an internal tool, and would start being used by external developers. Definitely it should be treated as internal for at least one minor version, so that we can iron out whatever weaknesses we may find. If others start using it during that time and complain, tough luck. Georg ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 457: Syntax For Positional-Only Parameters
On 10/09/2013 03:31 AM, Benjamin Peterson wrote: 2013/10/8 Larry Hastings la...@hastings.org: This PEP proposes a backwards-compatible syntax that should permit implementing any builtin in pure Python code. This is rather too strong. You can certainly implement them; you just have to implement the argument parsing yourself. Can you suggest an alternative wording? What I was trying to say was, you cannot express many builtin's signatures in pure Python code. I guess implement is the bad word here--you can implement those semantics, but you cannot express it syntactically. So how about This PEP proposes a backwards-compatible syntax that should permit expressing the signature of any builtin as a Python function. Python's call/signature syntax is already extremely expressive, and resolving call arguments to formal parameters is already a complicated (and slow) process. Well, I did propose that the syntax not actually be implemented. Implementing functions with such strange argument semantics is hardly common enough to justify the whole grouping syntax proposed in this PEP. -1 to that. I think I can live with /, but YANGTNI still. On the contrary, I can tell you exactly when I'm going to need it. What happens when you run help(foo)? pydoc examines the signature of the function and generates the first line, the function's prototype. For builtins, pydoc currently sees that there is no introspection information and skips the generation, which is why docstrings for builtins by convention have a handwritten prototype as their first line. Let's assume that the Argument Clinic PEP is accepted, and my current implementation goes in, and I add introspection metadata for builtins. Once we have this metadata, pydoc will use it and generate that first line for builtins itself. Now: what happens when you run help(curses.window.addch)? * My first problem: Argument Clinic has to allow the core C developer to express the signature of addch. The syntax is Python-like, if you change the commas into newlines and enforce a few extra rules regarding leading whitespace. But Python's existing syntax can't communicate the semantics of addch. Nobody fought me on Clinic's need to express this sort of function, though I had plenty of bikeshedding on the syntax. Then Guido dictated a syntax at PyCon US 2013, and that was that. My second problem: I have to communicate the metadata in Argument Clinic's output to the CPython runtime. Almost nobody cares what this looks like, as it will be an implementation detail. But I suspect there will be one or two places where you can't use Clinic yet, so you'll be writing this metadata by hand. So it needs to be cogent and human readable. Barry Warsaw suggested that the best syntax--not only most readable, but most compact--was to just use Python's own syntax. Again, existing Python syntax can't communicate the semantics of addch. So I expect to use this syntax for the metadata of builtins. My third problem: I have to represent the signature of these functions with inspect.Signature. Parameter objects can be marked as having a kind of inspect.Parameter.POSITIONAL_ONLY, so inspect has at least tacit, vestigial support for positional-only parameters. In practice this is insufficient to express the semantics of addch. IIRC Brett said I could add a group field to inspect.Parameter indicating the option group of a parameter. (There's no way I'm modifying inspect.getargspec() or inspect.getfullargspec() to represent positional-only parameters.) My fourth problem: I have to modify pydoc so it uses inspect.signature() instead of inspect.getfullargspec(), then understands the new semantics of positional-only parameters. And then it needs to generate that first line. But what should that line look like--what should be its syntax? It's written in Python syntax, but Python existing syntax can't communicate the semantics of addch. So I expect to use this syntax for the first line generated by pydoc. And there you have it. I have to represent the semantics of addch in four places, and three of those are textual. Two are implementation details, but in practice they will use Python syntax. The third is documentation but should also be Python syntax. My goal with the PEP is to codify existing practice. I find it a bit perverse that we have functions callable in Python with call signatures one cannot express syntactically in Python, and with undocumented semantics (their parameters are positional-only but this is not mentioned). I understand it's all kind of a historical accident, but it is absolutely part of Python and it's not going away. My proposed syntax is a little complex--but complex is better than complicated and inconsistent and undocumented and inconvenient, which is what we have now. //arr//y/ * The callable is not simply curses.window.addch; window is a class or something. But
Re: [Python-Dev] PEP 457: Syntax For Positional-Only Parameters
On 10/09/2013 07:48 AM, Eric Snow wrote: The PEP doesn't mention anything about inspect.Signature, which already supports positional-only parameters. It should. Well, inspect.Signature support for positional-only parameters is currently insufficient to express all of Python's builtins. As I mentioned in my long reply to Benjamin just now, I got Brett to agree that I could add a group field to inspect.Parameter to indicate a positional parameter's option group. I guess this could be part of the PEP. This also has an impact on other Python implementations, where maintaining the same positional-only semantics for builtins is complicated (at least a little) by the lack of a positional-only syntax in Python. Again, I don't expect this syntax to be implemented any time soon. But this does raise a mild sore point: Maciej and Armin Rigo tell me that PyPy's implementation of range() looks like this: def range(x, y=None, step=None): The None above is not the Python None, it's RPython's internal None, so it's not available to Python programs. But this does mean range() in PyPy accepts keyword parameters, and in fact this works: range(9, step=2) That's way illegal in CPython. This is for the sake of only a couple unfortunate builtins, right? Parameter grouping will undoubtedly be the most controversial part of this PEP. I'm not convinced this is a capability we need to encourage in any way. Is there some alternate one-off we can use for range, addch, et al.? I look forward to an alternate suggestion. This is the least-bad thing I could come up with. What are the names bound to for unfulfilled optional groups? Not bound (yuck)? What is wrong with default values within optional groups (other than how messy the syntax starts to become)? Please see my reply to Steven D'Aprano from about twelve hours ago. What about this: def spam([a,] b, [c,] d, /): Quoting the PEP: All required positional-only parameters must be contiguous. b and d are discontinuous. So this is illegal syntax. //arry/ ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 457: Syntax For Positional-Only Parameters
On 10/09/2013 07:15 AM, Georg Brandl wrote: We have quite a large amount of C functions with positional-only parameters. Adding a / to each of those is a) a tedious task and b) probably not helpful for a lot of people: I volunteer to ensure that the /s are added if this PEP is accepted. (I expect I'd do it myself, but who knows, maybe a volunteer would appear out of the woodwork.) ... and the undefined singleton just smells wrong. Another way to spell None is just asking for trouble. It has to be a different value from None, for the same reasons that all those other different-spellings-for-None (e.g. inspect.Parameter.empty) exist. I realize you are -1 on the proposal in general, but I'd be very interested if you could propose an alternate approach where I didn't need a new spelling for None as you put it. //arry/ ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 457: Syntax For Positional-Only Parameters
On Wed, Oct 9, 2013 at 8:15 AM, Georg Brandl g.bra...@gmx.net wrote: [...] Rather, I would try to make as many C functions as possible regular, See http://bugs.python.org/issue8706 and http://bugs.python.org/issue8350 Best Regards, Ezio Melotti ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 457: Syntax For Positional-Only Parameters
On 10/09/2013 11:46 AM, Nick Coghlan wrote: Generally, it needs to be a bit clearer that the intent of the PEP isn't to say let's do this, it's to be explicit that acceptance of the Argument Clinic PEP severely constrains the design space for possible solutions if we ever *did* implement Python level support for positional only arguments. Can you suggest an edit that would make you happy? - parameters in optional groups should just have an implied =None that can be overriden if desired. No no no! You can't have a default value, and you definitely couldn't use None. range() decides what its arguments mean based on how many arguments it receives. If I understand your proposal correctly, you suggest that range(None, 5) would return the same result as range(5) But that's simply not how it works. If you want to propose changing the semantics of range, go ahead, I'll stand back. - a simpler variant that omits the grouping support and only allows optional parameters on the right should also be reserved If that syntax is a subset of this syntax, and this syntax is reserved, then by extension we would automatically reserve that syntax too. In any other circumstance (this PEP is rejected, the simpler variant uses a different syntax) the simpler syntax should get a new PEP. I'll just say that that syntax is insufficient to implement existing functions we all know and love (addch, range). - explicitly delegate to the argument clinic PEP for the impact on inspect.Signature I'd actually rather do it the other way 'round. This PEP is a better place for it. Even though I think the Clinic PEP has a higher chance of being accepted ;-) //arry/ ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 457: Syntax For Positional-Only Parameters
2013/10/9 Larry Hastings la...@hastings.org: On 10/09/2013 03:31 AM, Benjamin Peterson wrote: 2013/10/8 Larry Hastings la...@hastings.org: This PEP proposes a backwards-compatible syntax that should permit implementing any builtin in pure Python code. This is rather too strong. You can certainly implement them; you just have to implement the argument parsing yourself. Can you suggest an alternative wording? What I was trying to say was, you cannot express many builtin's signatures in pure Python code. I guess implement is the bad word here--you can implement those semantics, but you cannot express it syntactically. So how about This PEP proposes a backwards-compatible syntax that should permit expressing the signature of any builtin as a Python function. That's fine. Python's call/signature syntax is already extremely expressive, and resolving call arguments to formal parameters is already a complicated (and slow) process. Well, I did propose that the syntax not actually be implemented. Yes, that's good. Implementing functions with such strange argument semantics is hardly common enough to justify the whole grouping syntax proposed in this PEP. -1 to that. I think I can live with /, but YANGTNI still. On the contrary, I can tell you exactly when I'm going to need it. What happens when you run help(foo)? pydoc examines the signature of the function and generates the first line, the function's prototype. For builtins, pydoc currently sees that there is no introspection information and skips the generation, which is why docstrings for builtins by convention have a handwritten prototype as their first line. Let's assume that the Argument Clinic PEP is accepted, and my current implementation goes in, and I add introspection metadata for builtins. Once we have this metadata, pydoc will use it and generate that first line for builtins itself. Now: what happens when you run help(curses.window.addch)? * My first problem: Argument Clinic has to allow the core C developer to express the signature of addch. The syntax is Python-like, if you change the commas into newlines and enforce a few extra rules regarding leading whitespace. But Python's existing syntax can't communicate the semantics of addch. Nobody fought me on Clinic's need to express this sort of function, though I had plenty of bikeshedding on the syntax. Then Guido dictated a syntax at PyCon US 2013, and that was that. My second problem: I have to communicate the metadata in Argument Clinic's output to the CPython runtime. Almost nobody cares what this looks like, as it will be an implementation detail. But I suspect there will be one or two places where you can't use Clinic yet, so you'll be writing this metadata by hand. So it needs to be cogent and human readable. Barry Warsaw suggested that the best syntax--not only most readable, but most compact--was to just use Python's own syntax. Again, existing Python syntax can't communicate the semantics of addch. So I expect to use this syntax for the metadata of builtins. My third problem: I have to represent the signature of these functions with inspect.Signature. Parameter objects can be marked as having a kind of inspect.Parameter.POSITIONAL_ONLY, so inspect has at least tacit, vestigial support for positional-only parameters. In practice this is insufficient to express the semantics of addch. IIRC Brett said I could add a group field to inspect.Parameter indicating the option group of a parameter. (There's no way I'm modifying inspect.getargspec() or inspect.getfullargspec() to represent positional-only parameters.) My fourth problem: I have to modify pydoc so it uses inspect.signature() instead of inspect.getfullargspec(), then understands the new semantics of positional-only parameters. And then it needs to generate that first line. But what should that line look like--what should be its syntax? It's written in Python syntax, but Python existing syntax can't communicate the semantics of addch. So I expect to use this syntax for the first line generated by pydoc. And there you have it. I have to represent the semantics of addch in four places, and three of those are textual. Two are implementation details, but in practice they will use Python syntax. The third is documentation but should also be Python syntax. My goal with the PEP is to codify existing practice. I find it a bit perverse that we have functions callable in Python with call signatures one cannot express syntactically in Python, and with undocumented semantics (their parameters are positional-only but this is not mentioned). I understand it's all kind of a historical accident, but it is absolutely part of Python and it's not going away. You can express any wild argument semantics you want in Python with *args and **kwargs. If the strange C functions like range() and addch() get signature objects, I don't think
Re: [Python-Dev] PEP 457: Syntax For Positional-Only Parameters
Am 09.10.2013 15:56, schrieb Larry Hastings: On 10/09/2013 07:15 AM, Georg Brandl wrote: We have quite a large amount of C functions with positional-only parameters. Adding a / to each of those is a) a tedious task and b) probably not helpful for a lot of people: I volunteer to ensure that the /s are added if this PEP is accepted. (I expect I'd do it myself, but who knows, maybe a volunteer would appear out of the woodwork.) Yes, the much bigger concern is the second one. ... and the undefined singleton just smells wrong. Another way to spell None is just asking for trouble. It has to be a different value from None, for the same reasons that all those other different-spellings-for-None (e.g. inspect.Parameter.empty) exist. I know. Still, as a new builtin it will get used for all kinds of other purposes, especially when it has the name that other languages use for our None. Maybe you could get away with NotGiven or NotPassed (in parallel to NotImplemented). I realize you are -1 on the proposal in general, but I'd be very interested if you could propose an alternate approach where I didn't need a new spelling for None as you put it. I think I would make Steven's proposed syntax mandatory: let the implementor of the function decide which value stands for not given -- just like we do in the C version, BTW. cheers, Georg ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 457: Syntax For Positional-Only Parameters
On 10/09/2013 03:36 PM, Antoine Pitrou wrote: I think it would be better to exercise [Argument Clinc] first as an internal tool, eliminate any possible quirks and issues, and *then* perhaps expose it as a public API. By the way, do you think it'll be ready for 3.4? Only if I can stop writing other PEPs and replying to their discussion threads! The implementation of Argument Clinic is done enough to be ready to go in. It doesn't support /everything/ it should. The main thing: it doesn't have enough predefined converters and return converters. But those are pretty quick to add, so that problem wouldn't last long. It also doesn't allow mixing positional-only parameters and any other types of parameters, so for example the dict constructor is currently inexpressable. And (as always) it needs more tests. But I'm finally happy with the internal interfaces, so that's something. The next step: polish the PEP, get final feedback, and submit it for judgment. Then start writing / polishing the documentation. If the PEP is accepted, get the implementation reviewed, and (fingers crossed) checked in. The implementation is available here for your reviewing pleasure: https://bitbucket.org/larry/python-clinic/ It got a /cursory/ high-level code review from Brett; we spent six or eight hours going over it at PyCon CA. He gave it a thumbs up but definitely wanted more coverage in the unit tests. //arry/ ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Python startup time
Am 09.10.2013 14:54, schrieb Christian Heimes: Am 09.10.2013 14:39, schrieb Benjamin Peterson: There's also the increasing number of modules (Python or otherwise) that have to be loaded on startup. We can easily peel off about 11 modules (re, sysconfig and their dependencies) with some changes to site.py: http://bugs.python.org/issue19205 Another low hanging fruit is the copyreg module: http://bugs.python.org/issue19209 The os module imports MutableMapping from collections.abc. That import adds collections, collections.abc and eight more modules. I'm not sure if we can do anything about it, though. Christian ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 457: Syntax For Positional-Only Parameters
On Oct 9, 2013 4:04 AM, Paul Moore p.f.mo...@gmail.com wrote: But if the intention is that any attempt to add the functionality to Python will be constrained to use the syntax here, surely that means we need to have all of the usability arguments and bikeshedding *now*. otherwise the opportunity is lost? I have the same concern. -eric ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 457: Syntax For Positional-Only Parameters
On 9 October 2013 13:38, Benjamin Peterson benja...@python.org wrote: Generally, it needs to be a bit clearer that the intent of the PEP isn't to say let's do this, it's to be explicit that acceptance of the Argument Clinic PEP severely constrains the design space for possible solutions if we ever *did* implement Python level support for positional only arguments. Why does a syntax need to be reserved? Documentation conventions and the syntax of internal tools like argument clinic may be changed any time we like. For that matter, why does the syntax used by Argument Clinic have to be the same as whatever future syntax is used in Python? If indeed, any ever is? What benefit do we get given the cost (rushing in a syntax that nobody is actually convinced we even need in Python yet). In other words, why does PEP this need to be separated out from the Argument Clinic PEP at all? Paul ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 457: Syntax For Positional-Only Parameters
On 9 October 2013 15:30, Larry Hastings la...@hastings.org wrote: Only if I can stop writing other PEPs and replying to their discussion threads! So once again, why is this new PEP needed? :-) Paul ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 457: Syntax For Positional-Only Parameters
Le Wed, 9 Oct 2013 15:40:08 +0100, Paul Moore p.f.mo...@gmail.com a écrit : On 9 October 2013 15:30, Larry Hastings la...@hastings.org wrote: Only if I can stop writing other PEPs and replying to their discussion threads! So once again, why is this new PEP needed? :-) Procrastination perhaps ;-) Paul ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 457: Syntax For Positional-Only Parameters
On 10/09/2013 04:26 PM, Georg Brandl wrote: I realize you are -1 on the proposal in general, but I'd be very interested if you could propose an alternate approach where I didn't need a new spelling for None as you put it. I think I would make Steven's proposed syntax mandatory: let the implementor of the function decide which value stands for not given -- just like we do in the C version, BTW. But that's not how addch works. addch counts how many arguments it received; if it is called with one or two, it does one thing, and if it's called with three or four it does something else. You can't duplicate these semantics with Similarly, you can't accurately express the semantics of range's arguments using default values. PyPy's approach is approximately like this: def range(x, y=None, step=None): step = 1 if step is None else step if y is not None: start, stop = x, y else: start, stop = 0, x But now introspection information on range() is inaccurate and unhelpful. (Not to mention, they allow specifying step without specifying y, by using keyword arguments.) My goal in writing the PEP was to codify existing practice, which meant reflecting these (annoying!) corner cases accurately. //arry/ ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 457: Syntax For Positional-Only Parameters
Larry Hastings wrote: I look forward to an alternate suggestion. This is the least-bad thing I could come up with. How about a naming convention instead, where using a leading underscore in a parameter name is a hint that it is positional-only. For example, the docstring of sorted: sorted(iterable, key=None, reverse=False) -- new sorted list would become: sorted(_iterable, key=None, reverse=False) -- new sorted list It seems more intuitive than the slash, and requires no change to the toolchain. Although, it may collide slightly with code that uses a leading underscore to indicate implementation detail for a parameter with a default value, but a quick scan of the std. lib. suggests that that's very rare. regards, Anders ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 457: Syntax For Positional-Only Parameters
On 10/09/2013 04:24 PM, Benjamin Peterson wrote: My proposed syntax is a little complex--but complex is better than complicated and inconsistent and undocumented and inconvenient, which is what we have now. Certainly the argument conventions of these functions are not undocumented, so wonder what is. Also, inconvenient for what? What inconsistency problem does this PEP solve? Whether or not a particular function accepts keyword arguments is undocumented. I have in the past been inconvenienced by this not being clear. The documentation uses two approaches for documenting option groups, effectively at random, which is inconsistent. //arry/ ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 457: Syntax For Positional-Only Parameters
On 10/09/2013 04:38 PM, Paul Moore wrote: For that matter, why does the syntax used by Argument Clinic have to be the same as whatever future syntax is used in Python? If indeed, any ever is? What benefit do we get given the cost (rushing in a syntax that nobody is actually convinced we even need in Python yet). In other words, why does PEP this need to be separated out from the Argument Clinic PEP at all? I propose to use this syntax in three places: 1. As input to Argument Clinic. 2. As input to the Python C API, to express the metadata for builtins. 3. As the first line of output of pydoc (aka help()) when reviewing a function. Of these, 3 is visible to users. Theoretically all of them could use different syntaxes. But consistency is desirable, so it would be better if these syntaxes were as similar as possible. Python syntax works well for all three syntaxes, except it does not support positional-only parameters. It seemed reasonable to codify twenty-year old practice in a syntax extension that I could use in all of the above. //arry/ ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 457: Syntax For Positional-Only Parameters
Am 09.10.2013 16:53, schrieb Larry Hastings: On 10/09/2013 04:26 PM, Georg Brandl wrote: I realize you are -1 on the proposal in general, but I'd be very interested if you could propose an alternate approach where I didn't need a new spelling for None as you put it. I think I would make Steven's proposed syntax mandatory: let the implementor of the function decide which value stands for not given -- just like we do in the C version, BTW. But that's not how addch works. addch counts how many arguments it received; if it is called with one or two, it does one thing, and if it's called with three or four it does something else. You can't duplicate these semantics with Well, why should a function that requires counting arguments even in C code not be implemented using *args? Documentation can cope fine, in the case of range() with two signatures. Similarly, you can't accurately express the semantics of range's arguments using default values. PyPy's approach is approximately like this: def range(x, y=None, step=None): step = 1 if step is None else step if y is not None: start, stop = x, y else: start, stop = 0, x But now introspection information on range() is inaccurate and unhelpful. (Not to mention, they allow specifying step without specifying y, by using keyword arguments.) My goal in writing the PEP was to codify existing practice, which meant reflecting these (annoying!) corner cases accurately. These are only two, one of them being exceedingly obscure. The hobgoblin comes to mind :) cheers, Georg ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 457: Syntax For Positional-Only Parameters
On 9 October 2013 15:53, Larry Hastings la...@hastings.org wrote: My goal in writing the PEP was to codify existing practice, which meant reflecting these (annoying!) corner cases accurately. That's fair. But I would argue that we very much don't want to encourage anyone ever duplicating that practice with new functions, which is why I don't think that this syntax should ever be implemented in the form described in Python. OTOH, there *may* be reasons in the future for wanting to support positional-only syntax in Python - that design should not be constrained by the need (which is Argument Clinic specific, AIUI) to cater for existing builtins' practice. Paul ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 457: Syntax For Positional-Only Parameters
On 10/09/2013 04:45 PM, Anders J. Munch wrote: Larry Hastings wrote: I look forward to an alternate suggestion. This is the least-bad thing I could come up with. How about a naming convention instead, where using a leading underscore in a parameter name is a hint that it is positional-only. For example, the docstring of sorted: sorted(iterable, key=None, reverse=False) -- new sorted list would become: sorted(_iterable, key=None, reverse=False) -- new sorted list It seems more intuitive than the slash, and requires no change to the toolchain. And the spelling for range()? curses.window.addch()? I suggest your proposed syntax is insufficient to codify existing practice. In any case, boy am I -1 on this. If you wish to continue this proposal further please write your own PEP. //arry/ ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 457: Syntax For Positional-Only Parameters
2013/10/9 Larry Hastings la...@hastings.org: On 10/09/2013 04:24 PM, Benjamin Peterson wrote: My proposed syntax is a little complex--but complex is better than complicated and inconsistent and undocumented and inconvenient, which is what we have now. Certainly the argument conventions of these functions are not undocumented, so wonder what is. Also, inconvenient for what? What inconsistency problem does this PEP solve? Whether or not a particular function accepts keyword arguments is undocumented. I have in the past been inconvenienced by this not being clear. The documentation uses two approaches for documenting option groups, effectively at random, which is inconsistent. I suppose the question is whether people will find it easier to learn your syntax for documenting it, or just relearning it for particular functions whenever they attempt to use a kwarg for a position argument. I don't expect it will show up on many functions. -- Regards, Benjamin ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 457: Syntax For Positional-Only Parameters
On 10/09/2013 08:09 AM, Larry Hastings wrote: On 10/09/2013 04:45 PM, Anders J. Munch wrote: How about a naming convention instead, where using a leading underscore in a parameter name is a hint that it is positional-only. For example, the docstring of sorted: sorted(iterable, key=None, reverse=False) -- new sorted list would become: sorted(_iterable, key=None, reverse=False) -- new sorted list In any case, boy am I -1 on this. If you wish to continue this proposal further please write your own PEP. A leading underscore indicates a private implementation detail. The proposed signature now looks like we can provide a key and reverse, but only riskily provide the actual thing we want sorted. No warm fuzzy feelings there! ;) -1 -- ~Ethan~ ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 457: Syntax For Positional-Only Parameters
On 9 October 2013 16:07, Larry Hastings la...@hastings.org wrote: On 10/09/2013 04:38 PM, Paul Moore wrote: For that matter, why does the syntax used by Argument Clinic have to be the same as whatever future syntax is used in Python? If indeed, any ever is? What benefit do we get given the cost (rushing in a syntax that nobody is actually convinced we even need in Python yet). In other words, why does PEP this need to be separated out from the Argument Clinic PEP at all? I propose to use this syntax in three places: As input to Argument Clinic. As input to the Python C API, to express the metadata for builtins. As the first line of output of pydoc (aka help()) when reviewing a function. Of these, 3 is visible to users. Theoretically all of them could use different syntaxes. But consistency is desirable, so it would be better if these syntaxes were as similar as possible. OK, all of those are reasonable points. I'm still -0 because I find the proposed syntax ugly, but I can't offer a better syntactic solution. My proposal is to hide the ugly syntax internally, and use prose documentation where users can see. You may disagree, that's fine. Python syntax works well for all three syntaxes, except it does not support positional-only parameters. It seemed reasonable to codify twenty-year old practice in a syntax extension that I could use in all of the above. I remain -1 on forcing Python syntax to support all of these odd corner cases (and positional-only is already a corner case, range/addch are seriously so). I don't see any reason for the proposal to mandate that the arguments made here should apply to any future changes to core Python syntax. Either we have the debate now, or we agree that the proposal doesn't apply to Python. Anyway, thanks for clarifying your position. I'm happy that I understand what you're proposing now, even if I don't agree with you :-) I presume you'll incorporate the various pieces of feedback into a revised version of the PEP that explains the purpose a bit more clearly. Paul ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] inspect() and dir()
On Tue, Oct 8, 2013 at 5:29 PM, Ethan Furman et...@stoneleaf.us wrote: Greetings, Currently, inspect() is dependent on dir(). What's inspect()? I don't know of any global with that name and you are obviously not talking about the module. ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 457: Syntax For Positional-Only Parameters
On 10/9/2013 9:51 AM, Larry Hastings wrote: Again, I don't expect this syntax to be implemented any time soon. But this does raise a mild sore point: Maciej and Armin Rigo tell me that PyPy's implementation of range() looks like this: def range(x, y=None, step=None): The None above is not the Python None, it's RPython's internal None, so it's not available to Python programs. But this does mean range() in PyPy accepts keyword parameters, and in fact this works: range(9, step=2) That's way illegal in CPython. But Georg's point is that it does not have to be illegal in CPython. Range, in particular, does not have to be blazingly fast since it wraps loops rather than being in loops. I quite agree that it would be better to fix most functions. -- Terry Jan Reedy ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Reduce memory footprint of Python
On Oct 06, 2013, at 11:41 AM, Benjamin Peterson wrote: What is the usecase for minimizing the memory usage of Python in the middle of a program? There are environments where Python is being actively discouraged in part due to its memory footprint. For example, we can't use Python on Ubuntu Touch for long-running processes because the target platforms (phones, tablets) are memory constrained. Python will never be as conservative as C++, but it's also unfortunate that the issue is bad enough for its use to be prohibited. FWIW, Python 3 is acceptable for some short-lived processes on these platforms. -Barry ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 457: Syntax For Positional-Only Parameters
On 10/09/2013 06:48 PM, Terry Reedy wrote: On 10/9/2013 9:51 AM, Larry Hastings wrote: range() in PyPy accepts keyword parameters, and in fact this works: range(9, step=2) That's way illegal in CPython. But Georg's point is that it does not have to be illegal in CPython. Range, in particular, does not have to be blazingly fast since it wraps loops rather than being in loops. I quite agree that it would be better to fix most functions. First, if you're proposing to change the signature of range()... well, good luck. Any proposals I made like that got shot down almost immediately. I /think/ Guido said somewhere range isn't changing. deal with it., though I admit I don't have a reference for that handy. Also, if you're suggesting change the semantics of builtins to get rid of positional-only parameters, I've got a different example for you: the dict constructor, which /must/ accept a single optional positional-only parameter. The only way to simulate that in Python code is with *args, but now introspection on that function will be misleading, because the dict constructor doesn't accept more than one positional-only argument. And, as Georg suggests, these niggling inconsistencies are the hobgoblins of my small mind. //arry/ ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 457: Syntax For Positional-Only Parameters
Sorry if I meddle, but wouldn't be better to include a module in the standard library that provides functions and classes to help parsing *args and **kwargs? This way you can avoid defining a new syntax, new builtin objects and so on. ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 457: Syntax For Positional-Only Parameters
On 10/9/2013 7:45 AM, Anders J. Munch wrote: Larry Hastings wrote: I look forward to an alternate suggestion. This is the least-bad thing I could come up with. How about a naming convention instead, where using a leading underscore in a parameter name is a hint that it is positional-only. For example, the docstring of sorted: sorted(iterable, key=None, reverse=False) -- new sorted list would become: sorted(_iterable, key=None, reverse=False) -- new sorted list It seems more intuitive than the slash, and requires no change to the toolchain. Although, it may collide slightly with code that uses a leading underscore to indicate implementation detail for a parameter with a default value, but a quick scan of the std. lib. suggests that that's very rare. regards, Anders So, to avoid that collision, use two leading underscores... a private name for the parameter which can't be specified in the call, similar to private class members that can't (without extreme difficulty) be specified in a reference outside the class. ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] inspect() and dir()
On 10/09/2013 09:40 AM, Brett Cannon wrote: On Tue, Oct 8, 2013 at 5:29 PM, Ethan Furman wrote: Currently, inspect() is dependent on dir(). What's inspect()? I don't know of any global with that name and you are obviously not talking about the module. My apologies. I am actually talking about the module. I meant inspect.get_members() and inspect.classify_class_attrs(), which, as near as I can tell, are the only two functions in inspect that attempt to retrieve/look at all of an object's attributes. -- ~Ethan~ ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 457: Syntax For Positional-Only Parameters
On Wed, 09 Oct 2013 17:05:12 +0100, Paul Moore p.f.mo...@gmail.com wrote: On 9 October 2013 16:07, Larry Hastings la...@hastings.org wrote: On 10/09/2013 04:38 PM, Paul Moore wrote: For that matter, why does the syntax used by Argument Clinic have to be the same as whatever future syntax is used in Python? If indeed, any ever is? What benefit do we get given the cost (rushing in a syntax that nobody is actually convinced we even need in Python yet). In other words, why does PEP this need to be separated out from the Argument Clinic PEP at all? I propose to use this syntax in three places: As input to Argument Clinic. As input to the Python C API, to express the metadata for builtins. As the first line of output of pydoc (aka help()) when reviewing a function. Of these, 3 is visible to users. Theoretically all of them could use different syntaxes. But consistency is desirable, so it would be better if these syntaxes were as similar as possible. OK, all of those are reasonable points. I'm still -0 because I find the proposed syntax ugly, but I can't offer a better syntactic solution. My proposal is to hide the ugly syntax internally, and use prose documentation where users can see. You may disagree, that's fine. Python syntax works well for all three syntaxes, except it does not support positional-only parameters. It seemed reasonable to codify twenty-year old practice in a syntax extension that I could use in all of the above. I remain -1 on forcing Python syntax to support all of these odd corner cases (and positional-only is already a corner case, range/addch are seriously so). I don't see any reason for the proposal to mandate that the arguments made here should apply to any future changes to core Python syntax. Either we have the debate now, or we agree that the proposal doesn't apply to Python. It seems to me, then, that the solution for the handle-the-ugly-existing-practice-groups issue is to make case (3) (pydoc prototype) match the convention that we have arrived at for the documentation: multiple signature lines to represent what Argument Clinic represents with the group notation. We don't currently have a doc solution for 'positional only' arguments. How about a variant on the '_' markup idea: range(stop) range(start, stop[, step]) This takes advantage of the fact that the notation is commonly used in various kinds of programming documentation to indicate put your specific value here. --David ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 457: Syntax For Positional-Only Parameters
On 10/09/2013 09:05 AM, Paul Moore wrote: I remain -1 on forcing Python syntax to support all of these odd corner cases (and positional-only is already a corner case, range/addch are seriously so). Considering the prevalence of positional-only functions and methods, I don't think they can be called a corner-case. (Or maybe I've just had the bad luck to run into all of the few there are.) -- ~Ethan~ ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 457: Syntax For Positional-Only Parameters
On Tue, Oct 8, 2013 at 4:33 PM, Larry Hastings la...@hastings.org wrote: I've contributed a new PEP to humanity. I include the RST for your reading pleasure below, but you can also read it online here: http://www.python.org/dev/peps/pep-0457/ Hi Larry, My comments below. TL;DR I'm -1, but trying to figure out your rationale and motivation in more detail. Discuss, */arry* - PEP: 457 Title: Syntax For Positional-Only Parameters Version: $Revision$ Last-Modified: $Date$ Author: Larry Hastings la...@hastings.org la...@hastings.org Discussions-To: Python-Dev python-dev@python.org python-dev@python.org Status: Draft Type: Informational Content-Type: text/x-rst Created: 08-Oct-2013 Overview This PEP proposes a syntax for positional-only parameters in Python. Positional-only parameters are parameters without an externally-usable name; when a function accepting positional-only parameters is called, positional arguments are mapped to these parameters based solely on their position. = Rationale = Python has always supported positional-only parameters. Early versions of Python lacked the concept of specifying parameters by name, so naturally all parameters were positional-only. This changed around Python 1.0, when all parameters suddenly became positional-or-keyword. But, even in current versions of Python, many CPython builtin functions still only accept positional-only arguments. Functions implemented in modern Python can accept an arbitrary number of positional-only arguments, via the variadic ``*args`` parameter. However, there is no Python syntax to specify accepting a specific number of positional-only parameters. Put another way, there are many builtin functions whose signatures are simply not expressable with Python syntax. This PEP proposes a backwards-compatible syntax that should permit implementing any builtin in pure Python code. - Positional-Only Parameter Semantics In Current Python - There are many, many examples of builtins that only accept positional-only parameters. The resulting semantics are easily experienced by the Python programmer--just try calling one, specifying its arguments by name:: pow(x=5, y=3) Traceback (most recent call last): File stdin, line 1, in module TypeError: pow() takes no keyword arguments You mean, like: def mypow(*args): return pow(args[0], args[1]) ... mypow(2, 3) 8 mypow(x=2, y=3) Traceback (most recent call last): File stdin, line 1, in module TypeError: mypow() got an unexpected keyword argument 'x' def myfoo(*args): a, b = *args; print a, b File stdin, line 1 def myfoo(*args): a, b = *args; print a, b ^ ? In addition, there are some functions with particularly interesting semantics: * ``range()``, which accepts an optional parameter to the *left* of its required parameter. [#RANGE]_ * ``dict()``, whose mapping/iterator parameter is optional and semantically must be positional-only. Any externally visible name for this parameter would occlude that name going into the ``**kwarg`` keyword variadic parameter dict! [#DICT]_ Obviously one can simulate any of these in pure Python code by accepting ``(*args, **kwargs)`` and parsing the arguments by hand. But this results in a disconnect between the Python function's signature and what it actually accepts, not to mention the work of implementing said argument parsing. I'm not sure what you call parsing. This? def myfoo(*args): a, b = args; print(%s, then %s % (a, b)) ? == Motivation == This PEP does not propose we implement positional-only parameters in Python. The goal of this PEP is simply to define the syntax, so that: * Documentation can clearly, unambiguously, and consistently express exactly how the arguments for a function will be interpreted. Can't we do this now? Is the problem only with those existing built-ins, or for new functions too? * The syntax is reserved for future use, in case the community decides someday to add positional-only parameters to the language. This doesn't seem like a valid motivation. YAGNI * Argument Clinic can use a variant of the syntax as part of its input when defining the arguments for built-in functions. Isn't one of the goals of Argument Clinic to provide a DSL anyhow? I'm not sure how this help. Eli ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 457: Syntax For Positional-Only Parameters
Am 09.10.2013 19:00, schrieb Ethan Furman: On 10/09/2013 09:05 AM, Paul Moore wrote: I remain -1 on forcing Python syntax to support all of these odd corner cases (and positional-only is already a corner case, range/addch are seriously so). Considering the prevalence of positional-only functions and methods, I don't think they can be called a corner-case. (Or maybe I've just had the bad luck to run into all of the few there are.) No, but their number can be decreased substantially without much additional work while switching to Argument Clinic. Georg ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 457: Syntax For Positional-Only Parameters
On Wed, Oct 9, 2013 at 8:53 AM, Larry Hastings la...@hastings.org wrote: But that's not how addch works. addch counts how many arguments it received; if it is called with one or two, it does one thing, and if it's called with three or four it does something else. You can't duplicate these semantics with Similarly, you can't accurately express the semantics of range's arguments using default values. PyPy's approach is approximately like this: def range(x, y=None, step=None): step = 1 if step is None else step if y is not None: start, stop = x, y else: start, stop = 0, x But now introspection information on range() is inaccurate and unhelpful. (Not to mention, they allow specifying step without specifying y, by using keyword arguments.) My goal in writing the PEP was to codify existing practice, which meant reflecting these (annoying!) corner cases accurately. The optional group syntax is in the PEP to support a small number of builtin functions, right? Is it just range() and addch()? If so, adding the complexity of optional groups (as syntax) isn't worth it. I'm still +0 on adding positional-only arguments (with '/'). As others have said, you can already achieve all this using *args. For me the allure of positional-only arguments lies in the following: 1. not having to roll my own *args handling; 2. not having to clutter up my code with the *args handling; 3. not having to handle positional-or-keyword params with *args when also using positional-only args; 4. documentation and help() can be more clear/accurate. As far as documentation goes, I'll defer to Georg and other Doc/library-savvy folks about the downside of using the syntax in this PEP. However, I agree it would be nice to be able to indicate in the docs that a parameter is positional-only. Perhaps it is as simple as always listing the positional-only parameters right after the signature. For example: type(object) type(name, bases, dict) positional-only: object, name, bases, dict With one argument, return the type of ... range(stop) range(start, stop, step) positional-only: start, stop, step Rather than being a function, ... Regarding help(), getting pydoc to make use of inspect.Signature objects would be great. The idea of supporting optional groups in inspect.Parameter sounds good to me and pydoc could make use of that. -eric ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 457: Syntax For Positional-Only Parameters
On 10/9/2013 1:05 PM, Larry Hastings wrote: First, if you're proposing to change the signature of range()... well, good luck. Any proposals I made like that got shot down almost immediately. I /think/ Guido said somewhere range isn't changing. deal with it., though I admit I don't have a reference for that handy. I am sure that he has rejected any backward-incompatible change, as would I. Allowing keyword passing *is* compatible, so I would not guess about that. Also, if you're suggesting change the semantics of builtins to get rid of positional-only parameters, Georg, and I agree, said *most*. I've got a different example for you: the dict constructor, which /must/ accept a single optional positional-only parameter. An *optional* positional-only parameter can be and sometimes is or has been denoted by [], as with dir([object]). In Python 3, [] is no longer used for normal optional params, which always have defaults. The dict signature could be (and maybe once was) given as dict([mapping-or-iterable], *kwds). There is a definite problem, though with required positional-only params. If these are rarer than at present, then it may matter less to people the indicator is. If they think it obnoxious, they will not have to see it so often, Unlike Georg, I think '/' is plausibly ok. You should, however, explicitly say in the PEP that it was selected not randomly but by rough analogy. Division (/) is the inverse of multiplication (*), and positional-only is sort of the 'inverse' of keyword-only. (If I am wrong, and the above is just an accident, it is at least a way to remember ;-). Another approach would be to make parameter names that are illegal to use in calls illegal as Python names. form follows function or in this case, bad form indicates no function. For instance, always put a *hyphen* in the displayed parameter name, as I intentionally did above. If nothing else, add a '-p' suffix. A hyphen is not legal in a name, so 'object-p', for instance, cannot possibly be used to make a call such as: dir(object-p=int) SyntaxError: keyword can't be an expression Note that I am talking about the doc and help results, not necessarily about the CPython C code most people never see. -- Terry Jan Reedy ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 457: Syntax For Positional-Only Parameters
On Wed, Oct 9, 2013 at 11:38 AM, R. David Murray rdmur...@bitdance.com wrote: It seems to me, then, that the solution for the handle-the-ugly-existing-practice-groups issue is to make case (3) (pydoc prototype) match the convention that we have arrived at for the documentation: multiple signature lines to represent what Argument Clinic represents with the group notation. +1 We don't currently have a doc solution for 'positional only' arguments. How about a variant on the '_' markup idea: range(stop) range(start, stop[, step]) This takes advantage of the fact that the notation is commonly used in various kinds of programming documentation to indicate put your specific value here. +1 I was thinking of exactly this same notation. -eric ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 457: Syntax For Positional-Only Parameters
On 9 October 2013 18:00, Ethan Furman et...@stoneleaf.us wrote: On 10/09/2013 09:05 AM, Paul Moore wrote: I remain -1 on forcing Python syntax to support all of these odd corner cases (and positional-only is already a corner case, range/addch are seriously so). Considering the prevalence of positional-only functions and methods, I don't think they can be called a corner-case. (Or maybe I've just had the bad luck to run into all of the few there are.) You're probably right - I was exaggerating, sorry. I do however consider most positional-only arguments in builtins to be an implementation artifact - if they could have been positional-or-keyword when they were introduced, they may well have have been (keyword args in C were a later addition to the C API, I can't recall exactly when but a good chunk of the existing builtins were already there when they arrived). Paul. ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] cpython (3.3): Fix macro expansion of _PyErr_OCCURRED(), and make sure to use it in at least
Am 07.10.2013 20:41, schrieb antoine.pitrou: http://hg.python.org/cpython/rev/2aee9c41bb79 changeset: 86140:2aee9c41bb79 branch: 3.3 parent: 86136:bd16e333 user:Antoine Pitrou solip...@pitrou.net date:Mon Oct 07 20:38:51 2013 +0200 summary: Fix macro expansion of _PyErr_OCCURRED(), and make sure to use it in at least one place so as to avoid regressions. If it was broken and never used anyway (and is an internal macro), why not remove it completely? Georg ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 457: Syntax For Positional-Only Parameters
On 10/09/2013 07:50 PM, Eli Bendersky wrote: I'm not sure what you call parsing. This? def myfoo(*args): a, b = args; print(%s, then %s % (a, b)) From the Argument Clinic PEP (436): The primary implementation of Python, CPython, is written in a mixture of Python and C. One implementation detail of CPython is what are called built-in functions -- functions available to Python programs but written in C. When a Python program calls a built-in function and passes in arguments, those arguments must be translated from Python values into C values. This process is called parsing arguments. I should probably amend that, because deciding how to map arguments to parameters is also part of parsing arguments. //arry/ ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 457: Syntax For Positional-Only Parameters
On 10/09/2013 08:25 PM, Terry Reedy wrote: I am sure that he has rejected any backward-incompatible change, as would I. Allowing keyword passing *is* compatible, so I would not guess about that. I don't know where to look for this discussion, and I read it quite some time ago. So I this is only dimly remembered. If it's crucial to the outcome of this discussion maybe we can ask Guido for a fresh ruling. But. As I recall the discussion, he was specifically against supporting keyword arguments for range, and he was adamant that range argument parsing was not to change in the slightest degree, backwards-incompatible or not. Again, my almost-certainly-misremembered quote: range isn't changing. deal with it. An *optional* positional-only parameter can be and sometimes is or has been denoted by [], as with dir([object]). In Python 3, [] is no longer used for normal optional params, which always have defaults. The dict signature could be (and maybe once was) given as dict([mapping-or-iterable], *kwds). There is a definite problem, though with required positional-only params. If these are rarer than at present, then it may matter less to people the indicator is. If they think it obnoxious, they will not have to see it so often, Most of the positional-only parameters in CPython are of the required variety. Some examples? Almost all of the built-in functions: http://docs.python.org/3/library/functions.html only accept positional-only parameters. So, almost anywhere you see a required parameter on that page, it is a required positional-only parameter. Also, many (most?) of the really old C extensions exclusively use PyArg_ParseTuple, so those too. My go-to example is curses simply because that one gave me the most psychological scarring. But there are loads of them. Anything that parses arguments solely with PyArg_ParseTuple only accepts positional-only arguments, and anything that calls it more than once probably has some optional arguments. Run from a fresh Python trunk: % fgrep -w PyArg_ParseTuple */*.c | wc 8614728 69716 //arry/ ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 457: Syntax For Positional-Only Parameters
On 09/10/13 00:33, Larry Hastings wrote:
I've contributed a new PEP to humanity. I include the RST for your
reading pleasure below, but you can also read it online here:
http://www.python.org/dev/peps/pep-0457/
Overall I'm in favour.
As a motivation for positional only parameters, consider:
Python 3.2:
from decimal import Decimal
d = Decimal(4)
d.__add__(other=d)
Decimal('8')
Python 3.3:
from decimal import Decimal
d = Decimal(4)
d.__add__(other=d)
Traceback (most recent call last):
File stdin, line 1, in module
TypeError: wrapper __add__ doesn't take keyword arguments
[snip]
The obvious solution: add a new singleton constant to Python
that is passed in when a parameter is not mapped to an argument.
I propose that the value be called called ``undefined``,
and be a singleton of a special class called ``Undefined``.
If a positional-only parameter did not receive an argument
when called, its value would be set to ``undefined``.
There is no need to create an undefined value.
Rather than define a parameter by assigning a fake value, just don't
define it. We already do this for non-parameter locals and it could be
extended to parameters.
'range' would be defined thus:
def range([start,] stop, [step], /):
try:
start
except UnboundLocalError:
start = 0
try:
step
except UnboundLocalError:
step = 1
...
Cheers,
Mark.
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 457: Syntax For Positional-Only Parameters
On 10/09/2013 10:46 PM, Mark Shannon wrote: There is no need to create an undefined value. Rather than define a parameter by assigning a fake value, just don't define it. We already do this for non-parameter locals and it could be extended to parameters. 'range' would be defined thus: def range([start,] stop, [step], /): try: start except UnboundLocalError: start = 0 try: step except UnboundLocalError: step = 1 ... That's a clever, and intuitive, idea. Though the try/excepts look awfully clumsy. Perhaps we could add (egad, no, I can't believe I'm saying this) a new built-in function that tells you whether or not a local variable has been assigned to yet? def range([start,] stop, [step], /): if not bound(start): start = 0 if not bound(step): step = 1 ... Anyway, this section is in the Notes For Future Implementors section, so as long as we never actually implement the syntax we don't need to solve the problem. //arry/ ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] cpython (3.3): Fix macro expansion of _PyErr_OCCURRED(), and make sure to use it in at least
On Wed, 09 Oct 2013 20:38:55 +0200 Georg Brandl g.bra...@gmx.net wrote: Am 07.10.2013 20:41, schrieb antoine.pitrou: http://hg.python.org/cpython/rev/2aee9c41bb79 changeset: 86140:2aee9c41bb79 branch: 3.3 parent: 86136:bd16e333 user:Antoine Pitrou solip...@pitrou.net date:Mon Oct 07 20:38:51 2013 +0200 summary: Fix macro expansion of _PyErr_OCCURRED(), and make sure to use it in at least one place so as to avoid regressions. If it was broken and never used anyway (and is an internal macro), why not remove it completely? Because I used it in a patch :-) Regards Antoine. ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 455: TransformDict
On 9 Oct 2013 01:07, Antoine Pitrou solip...@pitrou.net wrote: Le Tue, 8 Oct 2013 22:49:28 +1000, Nick Coghlan ncogh...@gmail.com a écrit : Well, you could ask the same question about OrderedDict, defaultdict or Weak*Dictionary since neither of them use composition :-) We *did* ask the same question about those (except the Weak* variants, simply due to age). Each time, the point that each new dict variant would be used to justify yet *more* variants was a cause for concern. Could you explain? I don't understand. Just that the comment I replied to is getting very close to the argument we have so many non-composable mapping variants already, what's the harm in adding one more?. I believe potentially enabling that argument in the future was cited as a downside for all of defaultdict, OrderedDict and Counter. Composition doesn't work with some mappings isn't an adequate answer to the criticism that a composition based design could work with more mappings than just the builtin dict. Fair enough. Regards Antoine. ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/ncoghlan%40gmail.com ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] inspect() and dir()
On 10 Oct 2013 03:39, Ethan Furman et...@stoneleaf.us wrote: On 10/09/2013 09:40 AM, Brett Cannon wrote: On Tue, Oct 8, 2013 at 5:29 PM, Ethan Furman wrote: Currently, inspect() is dependent on dir(). What's inspect()? I don't know of any global with that name and you are obviously not talking about the module. My apologies. I am actually talking about the module. I meant inspect.get_members() and inspect.classify_class_attrs(), which, as near as I can tell, are the only two functions in inspect that attempt to retrieve/look at all of an object's attributes. Those have to depend on __dir__ so classes (especially proxies) can accurately report *extra* dynamically accessible names. It's also the mechanism by which custom metaclasses, descriptors and __getattribute__ implementations can report the right names to the introspection machinery. However, it may make sense to offer a separate introspection utility to help debug custom __dir__ implementations. Cheers, Nick. -- ~Ethan~ ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/ncoghlan%40gmail.com ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 457: Syntax For Positional-Only Parameters
On Wed, Oct 09, 2013 at 02:25:18PM -0400, Terry Reedy wrote: Unlike Georg, I think '/' is plausibly ok. You should, however, explicitly say in the PEP that it was selected not randomly but by rough analogy. Division (/) is the inverse of multiplication (*), and positional-only is sort of the 'inverse' of keyword-only. (If I am wrong, and the above is just an accident, it is at least a way to remember ;-). I recall that was exactly the analogy Guido used when he first suggested the syntax. -- Steven ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 457: Syntax For Positional-Only Parameters
On Wed, Oct 09, 2013 at 09:46:00PM +0100, Mark Shannon wrote: There is no need to create an undefined value. Rather than define a parameter by assigning a fake value, just don't define it. We already do this for non-parameter locals and it could be extended to parameters. 'range' would be defined thus: def range([start,] stop, [step], /): try: start except UnboundLocalError: start = 0 try: step except UnboundLocalError: step = 1 ... If there is even the tiniest, faintest chance that positional arguments will be available to functions written in Python some day, -1 to this suggestion. There is no way I want to write code like the above instead of: def range([start,] stop, [step,] /): if start is Undefined: start = 0 if step is Undefined: step = 1 But what I really want to write is this: def range(start=0, stop, step=1, /): ... although maybe we don't want to encourage odd calling signatures by making them too easy to write. -- Steven ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] inspect() and dir()
On 10/09/2013 02:46 PM, Nick Coghlan wrote: On 10 Oct 2013 03:39, Ethan Furman wrote: My apologies. I am actually talking about the module. I meant inspect.get_members() and inspect.classify_class_attrs(), which, as near as I can tell, are the only two functions in inspect that attempt to retrieve/look at all of an object's attributes. Those have to depend on __dir__ so classes (especially proxies) can accurately report *extra* dynamically accessible names. Indeed, my rough idea is to use the results of the new dir() and combining that with the results of the old dir(). The use case being that some classes *ahem* Enum *ahem* may report *less* than is actually available, but introspection should reveal whatever is there even if dir() is not reporting it. -- ~Ethan~ ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] inspect() and dir()
On 10 Oct 2013 08:07, Ethan Furman et...@stoneleaf.us wrote: On 10/09/2013 02:46 PM, Nick Coghlan wrote: On 10 Oct 2013 03:39, Ethan Furman wrote: My apologies. I am actually talking about the module. I meant inspect.get_members() and inspect.classify_class_attrs(), which, as near as I can tell, are the only two functions in inspect that attempt to retrieve/look at all of an object's attributes. Those have to depend on __dir__ so classes (especially proxies) can accurately report *extra* dynamically accessible names. Indeed, my rough idea is to use the results of the new dir() and combining that with the results of the old dir(). The use case being that some classes *ahem* Enum *ahem* may report *less* than is actually available, but introspection should reveal whatever is there even if dir() is not reporting it. Not necessarily. For autocompletion, for example, you probably only want the public stuff. That's why I'm inclined to suggest the existing functions should continue to only report advertised attributes, with a separate introspection API that tries harder to find all accessible attributes (potentially including those from the metaclass). That way users can choose the one most appropriate to their use case, as well as being able to use the latter to help test custom dir support. Cheers, Nick. -- ~Ethan~ ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 457: Syntax For Positional-Only Parameters
On Wed, Oct 9, 2013 at 12:20 PM, Eric Snow ericsnowcurren...@gmail.com wrote: For me the allure of positional-only arguments lies in the following: 1. not having to roll my own *args handling; 2. not having to clutter up my code with the *args handling; 3. not having to handle positional-or-keyword params with *args when also using positional-only args; 4. documentation and help() can be more clear/accurate. [snip] Regarding help(), getting pydoc to make use of inspect.Signature objects would be great. The idea of supporting optional groups in inspect.Parameter sounds good to me and pydoc could make use of that. It may work better to facilitate the syntax in this PEP through inspect.Signature and friends, rather than as a pseudo-addition to the Python syntax. Here is some additional API that could help: * inspect.Signature.from_string(sig) - a factory that parses a signature string (including / from this PEP or perhaps param). Could Argument Clinic make use of this directly? * inspect.Signature.__str__() - updated to also display / for positional-only (or the param syntax that RDM described). * inspect.MultiSignature - a Sequence(?) containing one or more Signature objects (proxies the first one), which facilitates grouping. * inspect.MultiSignature.from_string(sig) - a factory that parses a signature string (including the full syntax from this PEP), adding a Signature for each derivable signature. These, along with a few other additions, would certainly help address the 4 things I listed above. I have a rough implementation of the above APIs (ignore anything else :) [1]. -eric [1] https://bitbucket.org/ericsnowcurrently/odds_and_ends/src/default/signature.py ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 457: Syntax For Positional-Only Parameters
On 10 Oct 2013 00:17, Larry Hastings la...@hastings.org wrote: On 10/09/2013 11:46 AM, Nick Coghlan wrote: Generally, it needs to be a bit clearer that the intent of the PEP isn't to say let's do this, it's to be explicit that acceptance of the Argument Clinic PEP severely constrains the design space for possible solutions if we ever *did* implement Python level support for positional only arguments. Can you suggest an edit that would make you happy? PEP title: Representation of positional-only parameters (Removes the loaded word Syntax for) The abstract would then draw from your reply to Benjamin: adding introspection support for builtins and extension modules means we need a way to represent positional-only parameters that can't be expressed using the normal function definition syntax. - parameters in optional groups should just have an implied =None that can be overriden if desired. No no no! You can't have a default value, and you definitely couldn't use None. range() decides what its arguments mean based on how many arguments it receives. If I understand your proposal correctly, you suggest that range(None, 5) would return the same result as range(5) But that's simply not how it works. No, you'd use range([start=0,] stop, [step=1,] /) to represent that case. Remember that this is about representing how the function effectively works from the user's point of view, not how it actually implements argument unpacking internally. Allowing default values and having =None implied for optional groups should cover all the cases we have to handle. If there are exceptions, I'd prefer to fix them over considering the future introduction of a new singleton (even if fixing them means offering better ways to handle None in PyArg_ParseTuple). If you want to propose changing the semantics of range, go ahead, I'll stand back. Not necessary, it just needs to be possible to express default values. - a simpler variant that omits the grouping support and only allows optional parameters on the right should also be reserved If that syntax is a subset of this syntax, and this syntax is reserved, then by extension we would automatically reserve that syntax too. In any other circumstance (this PEP is rejected, the simpler variant uses a different syntax) the simpler syntax should get a new PEP. I'll just say that that syntax is insufficient to implement existing functions we all know and love (addch, range). Allowing a single optional parameter on the left is enough to handle range, though. - explicitly delegate to the argument clinic PEP for the impact on inspect.Signature I'd actually rather do it the other way 'round. This PEP is a better place for it. Even though I think the Clinic PEP has a higher chance of being accepted ;-) I thought the group attribute idea was already in the argument clinic PEP. Since it isn't, documenting it here is fine. Cheers, Nick. /arry ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/ncoghlan%40gmail.com ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 457: Syntax For Positional-Only Parameters
On Wed, Oct 09, 2013 at 04:15:36PM +0200, Larry Hastings wrote: - parameters in optional groups should just have an implied =None that can be overriden if desired. No no no! You can't have a default value, and you definitely couldn't use None. range() decides what its arguments mean based on how many arguments it receives. If I understand your proposal correctly, you suggest that range(None, 5) would return the same result as range(5) But that's simply not how it works. Hmmm... I'm not entirely sure that I understand, but let me see if I've got it. range is defined something like this, only in C: def range(*args): if len(args) == 1: start, stop, step = 0, args[0], 1 elif len(args) == 2: start, stop, step = args[0], args[1], 1 elif len(args) == 3: start, stop, step = args else: raise TypeError ... So we can't document range as having the signature: range(start=None, stop, range=None) (modulo choice of defaults). Even if some future implementation of positional-only arguments allowed functions to set defaults, you would still have to deal with cases like range that don't. Do I understand you correctly? If so, then I ask that this PEP should not rule out setting defaults in the the future, but make it clear that even if defaults are allowed, there needs to be some way to handle defaultless-but-still-optional arguments, for cases like range. -- Steven ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 454: Add a new tracemalloc module (final version)
2013/10/3 Victor Stinner victor.stin...@gmail.com: I worked on the implementation of the tracemalloc module and its PEP 454. I consider that this third version of the PEP is ready for a final review. What should be done to finish the PEP? I splitted the module into two parts. Do you prefer the new API? Module: http://www.haypocalc.com/tmp/tracemalloc.split/library/tracemalloc.html Tool: http://www.haypocalc.com/tmp/tracemalloc.split/library/tracemalloctext.html Full story: Charles-François Natali made me some suggestions: * why using a Task class instead of a thread? * the module should be splitted into two parts: module (low-level functions getting data), tool (display/compare data with colors) I modified the implementation in a new split branch: * I moved DisplayTop, DisplayTopTask, TakeSnapshotTask and Task classes and the command line interface to a new tracemalloctext module * I rewrote Task into Python using a thread The tool would not be integrated in Python, but hosted on PyPI. I just kept it to get its documentation and show clearly how the split was made. tracemalloc API is simpler: http://www.haypocalc.com/tmp/tracemalloc.split/library/tracemalloc.html API of the tool (using a temporary name tracemalloctext which may change later): http://www.haypocalc.com/tmp/tracemalloc.split/library/tracemalloctext.html I chose to keep Snapshot.top_by(), GroupedStats and StatsDiff in the (tracemalloc) module. I prefer to hide internal structures of tracemalloc (tracemalloc.get_stats(), tracemalloc.get_traces()), and suggest to use higher level structures (GroupedStats, StatsDiff). Internal structures are not hidden, they are even documented. But they might change later, whereas GroupedStats structure should not change. Or do you think that computing top N allocations is something specific to a tool? Or each tool may implement this function differently? Using a thread has a drawback. Checking the memory threshold is implemented using a short sleep (100 ms). If the memory peak is shorter than the sleep, it will not be noticed. A workaround is to use a threshold on the maximum traced memory. The advantage of the thread is to have a better resolution than 1 second for the timer, and to not have to get the current time at each memory allocation when a task is scheduled. The current implement (not splitted) uses Py_AddPendingCall(), function already used by the C signal handler (signalmodule.c) to call later the Python signal handler from the bytecode evaluation loop (ceval.c). Victor ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Python startup time
On Wed, Oct 9, 2013 at 8:30 AM, Christian Heimes christ...@python.org wrote: The os module imports MutableMapping from collections.abc. That import adds collections, collections.abc and eight more modules. I'm not sure if we can do anything about it, though. Well, that depends on how much we want to eliminate those 10 imports. :) Both environ and environb could be turned into lazy wrappers around an _Environ-created-when-needed. If we used a custom module type for os [1], then adding descriptors for the two attributes is a piece of cake. As it is, with a little metaclass magic (or even with explicit wrapping of the various dunder methods), we could drop those 10 imports from startup. -eric [1] This probably wouldn't be a great idea considering that undoubtedly some code depends on type(os) is types.ModuleType. ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] PyCon US 2014
Registration is now open for PyCon US 2014. Are there any plans yet for the language summit? Just the day (e.g. Thursday April 10) will suffice. Then we can make arrangements appropriately. Thanks. -eric ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 457: Syntax For Positional-Only Parameters
On 10/10/13 09:51, Larry Hastings wrote: Perhaps we could add (egad, no, I can't believe I'm saying this) a new built-in function that tells you whether or not a local variable has been assigned to yet? def range([start,] stop, [step], /): if not bound(start): start = 0 if not bound(step): step = 1 ... It couldn't be a function in the usual sense, because attempting to evaluate the unbound parameter would raise an exception before the function was called. It would have to be something magical, probably a new piece of syntax understood by the compiler. -- Greg ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 457: Syntax For Positional-Only Parameters
On 10/10/13 11:57, Nick Coghlan wrote: PEP title: Representation of positional-only parameters Something like Notation for documenting positional-only parameters would make it even clearer that this is not a proposal for adding to the syntax of the language. -- Greg ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] inspect() and dir()
On 10/09/2013 03:36 PM, Nick Coghlan wrote: On 10 Oct 2013 08:07, Ethan Furman wrote: On 10/09/2013 02:46 PM, Nick Coghlan wrote: On 10 Oct 2013 03:39, Ethan Furman wrote: My apologies. I am actually talking about the module. I meant inspect.get_members() and inspect.classify_class_attrs(), which, as near as I can tell, are the only two functions in inspect that attempt to retrieve/look at all of an object's attributes. Those have to depend on __dir__ so classes (especially proxies) can accurately report *extra* dynamically accessible names. Indeed, my rough idea is to use the results of the new dir() and combining that with the results of the old dir(). The use case being that some classes *ahem* Enum *ahem* may report *less* than is actually available, but introspection should reveal whatever is there even if dir() is not reporting it. Not necessarily. For autocompletion, for example, you probably only want the public stuff. That's why I'm inclined to suggest the existing functions should continue to only report advertised attributes, with a separate introspection API that tries harder to find all accessible attributes (potentially including those from the metaclass). That way users can choose the one most appropriate to their use case, as well as being able to use the latter to help test custom dir support. That makes sense. So what should the new functions be called? get_all_members and classify_all_class_attrs ? -- ~Ethan~ ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Python startup time
I'm not sure Droping imports is the best way to go, since every python script/app will import common modules right on the start and it will still seem like the interpeter boot is slow. making modules load faster seems like a better approch On Thu, Oct 10, 2013 at 3:18 AM, Eric Snow ericsnowcurren...@gmail.comwrote: On Wed, Oct 9, 2013 at 8:30 AM, Christian Heimes christ...@python.org wrote: The os module imports MutableMapping from collections.abc. That import adds collections, collections.abc and eight more modules. I'm not sure if we can do anything about it, though. Well, that depends on how much we want to eliminate those 10 imports. :) Both environ and environb could be turned into lazy wrappers around an _Environ-created-when-needed. If we used a custom module type for os [1], then adding descriptors for the two attributes is a piece of cake. As it is, with a little metaclass magic (or even with explicit wrapping of the various dunder methods), we could drop those 10 imports from startup. -eric [1] This probably wouldn't be a great idea considering that undoubtedly some code depends on type(os) is types.ModuleType. ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/yoavglazner%40gmail.com ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
