[Python-ideas] constants in Python: Starting simple and gradually adding more features, was: Re: Pattern Matching controversy
Hello, On Mon, 16 Nov 2020 08:39:30 +1100 Steven D'Aprano wrote: [] > > The baseline of my version is much simpler: > > > > # This makes "const" a kind of hard keyword for this module > > from __future__ import const > > > > FOO: const = 1 # obviously, this is constant > Oh, well, To start with, in the original thread I wanted to concentrate on issues of the PEP634/PEP635, and whether these issues are prominent enough to hope for them being addressed. So, I changed the subject (and we'd soon be excused to python-ideas, indeed, I cc: there). > if all it takes is to add a new keyword, then constants are > easy! A new annotation. And the new subject is "constants in Python: Starting simple and gradually adding more", which hopefully should set the theme. Please remember how we arrived here: it's from the fact that PEP634 doesn't allow for the following trivial code: MY_CONST = 1 match foo: case MY_CONST: We're considering (another alternative) how to address that. Please don't make a leap towards supporting (at once) const-ness equivalent to statically typed languages. > No need to worry about how constantness affects name resolution, > or how the syntax interacts with type-hints: "const" is *the* type hint. > spam: const: Widget = build_widget(*args) That syntax is clearly invalid. And composability of type annotations (aka type hints) is a known, looming issue. We now have https://www.python.org/dev/peps/pep-0593/ , but in all fairness, it seems like a stopgap measure rather than an elegant solution. (In full fairness, entire "typing" module's annotations aren't very elegant, but as we know, it's not a part of language core, but part of CPython's stdlib. That means it's only *one* of possible annotation schemes for *Python*). So, under PEP593, the above would be written spam: Annotated[Widget, const] = build_widget(*args) If you want a glimpse of what alternatives might look, then: given that "|" is going to be used for union types, why not try "&" for composition? spam: Widget & const = build_widget(*args) But again, for "initial support of constants in Python, prompted by the introduction of pattern matching facilities", we don't need to worry about all that right away. > # Maybe this is better? > # let spam: Widget = build_widget(*args) As you remember, at the beginning of my proposal, I wrote "The baseline of my version ...". Baseline == level 0. For "level 2", I considered const spam: Widget = ... I don't think that "let" should be used. We're putting emphasis on *constantness* (not lexical scoping of variables [immutable by functional tradition, though that's "minor"]). JavaScript (now) has both "const" and "let", and I think that's pretty reasonable approach. So, let's save "let" for possible later uses. So, what's "level 1"? As I mentioned, under "level 0", "from __future__ import const" has a magic meaning (akin to other "from __future__"'s). Under "level 1", "const" is just a normal annotation symbol imported from a module. So, following would be possible: = from __future__ import const FOO: const = 1 def fun(): const = 1 # Just a variable in function scope def sub(): # Gimme powerz back from __future__ import const BAR: const = 2 # Back to annotation in global scope BAZ: const = 3 = "level 0" should be implementable in CPython in ~1hr. "level 1" is realistically what we should shoot for. "level 2" (the dedicated keyword), I'm still not sure about. "const" is very baseline annotation, and deserves a dedicated keyword. But the underlying problem is composability of (arbitrary) annotations. I'd rather keep searching for graal in that regard, before giving up and introduce a dedicated thing just for "const". > or how "constant" interacts with mutability: > > spam: const = [] > spam.append(99) # Allowed? > spam.extend(items) > spam.sort() "const" is an annotation just like any other. And it affects *runtime* in the same as any other annotation in CPython affects it: in no way. "const" is however introduced as a hint for *compile-time*, so compiler could make some simple inferences and maybe even optimizations based on it. > or for that matter, whether or not constants are actually constant. > > spam: const = 1 > spam = 2 Compiler can, and thus would, catch that as an error (warning for a beta version?). > If constants aren't actually constant, but just a tag on symbols, > then you would be right, it probably would be trivially easy to add > "constants" to Python. Right. > But I think most people agree making them behave as constants is a > pretty important feature. As mentioned, "const" is just an annotation like any other, except compiler has some insight into it. Dealing with runtime is distant goal for CPython. (Which is for now offloaded to static type checkers and libraries/alternative Python implementations.) > *The* most critical feature of
[Python-ideas] Re: Use __bytes__ to access buffer-protocol from "user-land"
I don't think __bytes__ is necessarily a bad idea, but I want to point out a couple of things you may be unaware of. First, slicing a memoryview object creates a subview, so you can wrap your mmap object in a memoryview and then create slices for each partition, cluster run, etc. without wasting any memory (but not for fragmented files). Second, your iterator example works in Python as it stands if you substitute __iter__ for __bytes__ and writelines for write. ___ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-le...@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/BM7KEUPKKU2DM7OP7VDSWJJJKHDJBQT3/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-ideas] Use __bytes__ to access buffer-protocol from "user-land"
I am working on a toolbox for computer-archaeology where old data media are
"excavated" and presented on a web-page.
(https://github.com/Datamuseum-DK/AutoArchaeologist for anybody who cares).
Since these data-media can easily sum tens of gigabytes, mmap and virtual
memory is my weapons of choice and that has brought me into an obscure corner
of python where few people seem to venture: I want to access the
buffer-protocol from "userland".
The fundamental problem is that if I have a image of a disk and it has 2
partitions, I end up with the "mmap.mmap" object that mapped the raw disk
image, and two "bytes" or "bytearray" objects, each containing one partition,
for a total memory footprint of twice the size of the disk.
As the tool dives into the filesystems in the partitions and creates objects
for the individual files in the filesystem, that grows to three times the size
of the disk etc.
To avoid this, I am writing a "bytes-like" scatter-gather class (not yet
committed), and that is fine as far as it goes.
If I want to write one of my scatter-gather objects to disk, I have to:
fd.write(bytes(myobj))
As a preliminary point, I think that is just wrong: A class with a __bytes__
method should satisfy any needs the buffer-protocol might have, so this should
work:
fd.write(myobj)
But taking this a little bit further, I think __bytes__ should be allowed to be
an iterator, provided the object also offers __len__, so that this would work:
class bar():
def __len__(self):
return 3
def __bytes__(self):
yield b'0'
yield b'1'
yield b'2'
open("/tmp/_", "wb").write(foo())
This example is of course trivial, but hav the yield statements hand out
hundreds of megabytes, and the savings in time and malloc-space becomes very
tangible.
Poul-Henning
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at
https://mail.python.org/archives/list/python-ideas@python.org/message/LPXGGCU2UG7Q7P4EYDKCR2XKH7HVYPB7/
Code of Conduct: http://python.org/psf/codeofconduct/
[Python-ideas] Re: PEP 634-636: Mapping patterns and extra keys
On 2020-11-15 at 19:11:15 +1100, Steven D'Aprano wrote: > On Sat, Nov 14, 2020 at 10:17:34PM -0800, Guido van Rossum wrote: > > > It’s a usability issue; mappings are used quite differently than sequences. > > Compare to class patterns rather than sequence patterns. > > I'm keeping an open mind on this question, but I think David is right > to raise it. I think that most people are going to see this as dict > matching as "ignoring errors by default" and going against the Zen of > Python, and I expect that we'll be answering questions about it for > years to come. > > "Why did my match statement match the wrong case?" > > Naively, I too would expect that dicts should only match if the keys > match with no left overs, and I would like to see the choice to ignore > left overs justified in the PEP. > > It would be good if the PEP gave a survey of the practical experience of > other languages with pattern matching: > > - are there languages which require an exact match, with no left over > keys? what issues, if any, do users have with that choice? > > - which languages ignore extra keys? do users of those languages > consider this feature a bug, a wart, or a feature? In Erlang, "mappings" tend to be generalized collections of (key, value) pairs in which the keys are not nailed down at design time, or the keys evolve over time (think about adding a new field to an existing message). Pattern matching ignores extra keys, so that old code can continue to handle the messages it knows how to handle and simply ignore data it doesn't know about (yes, you have to think carefully about extending messages in this way, but it has worked well over decades). This is definitely a feature. Also in Erlang, "records" are very similar to Python's named tuples. Pattern matching on records also ignores extra keys, so that I can match records that meet certain criteria and not have to list every attribute in every pattern. IMO, ignoring extra keys allows for extensibility when you don't always have control over which versions of which code is actually running (which is the case in the typical distributed system). Not ignoring extra keys may work better inside a monolithic application where all the data comes from within or is already parsed/decoded. IMO, this is going to come down to your use case (I'm *shocked*). If I receive HTML/XML/JSON/TCP/whatever messages, and I want to use pattern matching to decode or dispatch on the message type (e.g., login, logout, attack, connect), then *not* having to write **rest on every pattern reduces clutter. But if I have to handle 2D points separately from 3D points, then a more strict matching (i.e., not ignoring extra keys) relieves me of having to think about which case is more or less specific, and may be easier for beginners to use. (IMO, making things easier for beginners is only a means to an end. If making things easier for beginners makes things harder for experts, then don't do it. But I'm not in charge around here.) As an analogy, when you write a command line utility, do you accept or reject extraneous command line arguments? Is "spam --version ham eggs" the same as "spam --version"? I'm going to guess that it depends on your personality and your background and not anything else inside the utility (note that your choice of command line parser also depends on your personality and your backround...). IOW, the answer to the question of ignoring extra keys is going to aggravate half the users and half the use cases no matter what. ___ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-le...@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/B6BPSIH42FVCXUVGX37RGJZAM2ONYGDI/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-ideas] Re: PEP 634-636: Mapping patterns and extra keys
On Sat, Nov 14, 2020 at 10:17:34PM -0800, Guido van Rossum wrote: > It’s a usability issue; mappings are used quite differently than sequences. > Compare to class patterns rather than sequence patterns. I'm keeping an open mind on this question, but I think David is right to raise it. I think that most people are going to see this as dict matching as "ignoring errors by default" and going against the Zen of Python, and I expect that we'll be answering questions about it for years to come. "Why did my match statement match the wrong case?" Naively, I too would expect that dicts should only match if the keys match with no left overs, and I would like to see the choice to ignore left overs justified in the PEP. It would be good if the PEP gave a survey of the practical experience of other languages with pattern matching: - are there languages which require an exact match, with no left over keys? what issues, if any, do users have with that choice? - which languages ignore extra keys? do users of those languages consider this feature a bug, a wart, or a feature? -- Steve ___ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-le...@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/OVJKX3SY525EO62JBPSW2REZ66CK5RBU/ Code of Conduct: http://python.org/psf/codeofconduct/
