[Python-ideas] Re: Unpack operator "**" and Mapping
26.12.20 13:23, Anton Abrosimov пише: > I am trying to release comfortable dataclass unpacking using `**` operator. > Now I have 5 different ways to do it. > But not a single good one. Confused by the implementation of the unpacking > operator. > > So when I try to unpack any custom class, I get the error: > > `type object argument after ** must be a mapping, not MyClass` > > Ok, nothing special. I need to use `collections.abc.Mapping` right? > Now I need to implement: `__getitem__`, `__iter__`, `__len__`. Not a problem. > But additionally I get: `keys`, `items`, `values`. > Hey, I don't need them. I don't need the full mapping functionality. I only > need the double asterisk to work. > > Right, we have a duck typing! > We throw out `abc.Mapping`. What do we need to implement? > It's `__getitem__` and `keys`. Wtf `keys`? > > I am looking at Python Data model: > https://docs.python.org/3/reference/datamodel.html > There many operators, and they depend on special double underscore methods. > Hmm, I don't see unpack operators there, it's strange. > But why it's `keys`? Because the historical is `dict`? > I think a dependency on `__iter__` is more preferable and expectable over a > userspace named `keys`. > Actually, `items()` is more predictable. See thread "Add __keys__ or __items__ protocol". https://mail.python.org/archives/list/python-ideas@python.org/thread/A3CK7Y2ICIIVYKMAXSCUVYLK7IC7CH6G/#LCWMA6WMFLLSWXPKYC6FEUDHZBR3MYPE ___ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-le...@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/3AJOVRCP5J2YXS4GH66D6YQS52JHXF2T/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-ideas] Re: built in to clear terminal
On Sun, Dec 27, 2020 at 4:26 PM Steven D'Aprano wrote: > > On Sun, Dec 27, 2020 at 02:00:00PM +1100, Chris Angelico wrote: > > > This is actually the exact part that I think should *not* happen, for > > several reasons: > > > > 1) Stuff that exists interactively and doesn't exist in a script is a > > barrier to that migration. I don't think this feature justifies that > > barrier. > > It will still exist in a script, you can still import it from the os > module if you care about supporting cases where site.py doesn't get run. > > This is no different from `exit`: > > [steve ~]$ python3.9 -c "print(exit)" > Use exit() or Ctrl-D (i.e. EOF) to exit > > [steve ~]$ python3.9 -S -c "print(exit)" > Traceback (most recent call last): > File "", line 1, in > NameError: name 'exit' is not defined Oh, sorry, my bad. Withdraw point 1 then. > > 2) The builtins are already incredibly busy, and this, again, doesn't > > justify being one > > It's not an *actual* builtin, it's just monkey-patched into the > builtins by site.py. But how's that different? Either it's a builtin or it's not. How it gets there is mostly irrelevant, other than to make a separate monkeypatching layer (such as would need to be done by Idle) more complicated. > I don't think there is sufficient justification for this to make it a > genuine builtin like `len` or `iter`, but I think loading it into the > builtin namespace for the convenience of the user is plenty justified. Ohh, do you mean how some functions are intrinsic parts of the interpreter? I'm not talking about that; I'm talking about how it's accessed, ie the namespacing. > Having clear available immediately in the interactive interpreter is > certainly more useful than having the licence, credits and copyright > notice, and yet there they are. Probably the three least used builtins > ever. Perhaps, but they have legal reasons for existing. > > 3) Beginners already have to learn about importing (eg "from math > > import sin") and this is just one more importable. > > As far as I can tell, every language that supports a clear/cls command, > from old BASIC to modern shells, makes it available immediately. You > don't have to adjust the PATH in bash to get access to clear, it just > works. Old BASIC didn't *have* an import feature (at least, the BASICs that I used didn't), and modern shells have a ton of shell-related stuff in their "builtins" but nothing else, because that's their focus. You can't, for instance, do trig functions in the basic shell, nor even arithmetic, because those aren't builtins. Different environments focus on different things. > Think of this as the companion to print. If you think about Python as an > interactive interpreter, its weird to have print, help, quit available > but not clear. Its a fairly fundamental interactive command. I don't actually agree. Having the ability to print? Yes, that's fundamental. Being able to quit? Of course you need that. But there's no cursor movement function in the builtins, nor is there a "show this image" function, nor "display this web page", or various things like that. In terms of console actions, the two most fundamental are print and input, and we get both of those. Everything else has to justify itself in terms of utility and convenience; getting documentation, for instance, could easily have been buried away in a module, but it is *incredibly* helpful to be able to pull up help(x) without an import. We don't need a full set of cursor movement and screen manipulation features as builtins - that's the job of curses. I don't think we really need screen clearing that way either, particularly as the obvious name "clear" is also used in various other contexts, such as lists and dicts. (How confusing would it be to say clear(lst) instead of lst.clear() and then try to figure out what's going on?) > > > So that's the easy part: just agree with me and import it in site.py, > > > wherever it happens to come from. *wink* > > > > This is python-ideas! We don't "just agree" here, we argue! Even if we > > don't have to! > > I disagree! > > *wink* > Is this one of those cases where we "agree to disagree"? *wink* ChrisA ___ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-le...@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/GNCXENABIDMRKCH45DSPCOWBCPMTC7HG/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-ideas] Re: Unpack operator "**" and Mapping
On Sun, Dec 27, 2020 at 4:30 PM Brendan Barnwell wrote: > That said. . . I'm starting to wonder why not just create a new dunder > called __items__ and have dict alias that to .items(). Then the > **-unpacking protocol could use that and everything would be fine, right? > +0.95. If we could borrow the time machine and create this protocol way back in Python's past, I think it'd be the best. The semantics of maintaining backward compatibility MAY complicate things... but then again, iterator protocol is a bit complicated too, and everyone's fine with just defining __iter__. ChrisA ___ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-le...@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/M5XJF4SI2FRTOEUIXVXJXD6IXWWBISR7/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-ideas] Re: Unpack operator "**" and Mapping
On 2020-12-26 21:02, Steven D'Aprano wrote: On Sat, Dec 26, 2020 at 06:52:46PM -0800, Brendan Barnwell wrote: On 2020-12-26 18:44, Steven D'Aprano wrote: >I think if we were designing mapping protocols now, that would be an >excellent idea, but we aren't, we have decades of history from `dict` >behind us. And protocols from dict use `keys()` and getitem. E.g. >update. >>> >>>What do you mean by "protocols from dict"? What are these protocols? >"And protocols from dict use `keys()` and getitem. E.g. update." If I understand you right, that's not a protocol, that's just the behavior of the dict type specifically. As far as I can tell, it's not even documented behavior, so it doesn't constrain anything. Yes it is documented: help(dict.update) and it was intentionally the inspiration for the behaviour of dict augmented assignment. I see. It's rather disturbing that that isn't mentioned in the docs on python.org. If you want to argue it's not a protocol, just an interface, okay, it's an interface. That's a difference that makes no difference. No, it does make a difference. What you're describing is the interface to a single existing type. A protocol is a framework that defines behavior for USER-DEFINED types to hook into, as the descriptor protocol lets you define __get__ or the iterator protocol lets you define __iter__. The fact that dict uses a method with a particular name to do this or that should not constrain the creation of future protocols that define behavior for methods to be defined in user-created classes. That said. . . I'm starting to wonder why not just create a new dunder called __items__ and have dict alias that to .items(). Then the **-unpacking protocol could use that and everything would be fine, right? -- Brendan Barnwell "Do not follow where the path may lead. Go, instead, where there is no path, and leave a trail." --author unknown ___ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-le...@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/5UA5OOOCQIGAUKVLPQQI3Y2U2HV5FYAZ/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-ideas] Re: built in to clear terminal
On Sun, Dec 27, 2020 at 02:00:00PM +1100, Chris Angelico wrote: > This is actually the exact part that I think should *not* happen, for > several reasons: > > 1) Stuff that exists interactively and doesn't exist in a script is a > barrier to that migration. I don't think this feature justifies that > barrier. It will still exist in a script, you can still import it from the os module if you care about supporting cases where site.py doesn't get run. This is no different from `exit`: [steve ~]$ python3.9 -c "print(exit)" Use exit() or Ctrl-D (i.e. EOF) to exit [steve ~]$ python3.9 -S -c "print(exit)" Traceback (most recent call last): File "", line 1, in NameError: name 'exit' is not defined Well, actually, it will be slightly different: the pseudo-builtin `exit` is a special object created for interactive use, it's not the same as `sys.exit`. Whereas I invisage that the interactive `clear` will be identical (same object) as `os.clear`. > 2) The builtins are already incredibly busy, and this, again, doesn't > justify being one It's not an *actual* builtin, it's just monkey-patched into the builtins by site.py. I don't think there is sufficient justification for this to make it a genuine builtin like `len` or `iter`, but I think loading it into the builtin namespace for the convenience of the user is plenty justified. Having clear available immediately in the interactive interpreter is certainly more useful than having the licence, credits and copyright notice, and yet there they are. Probably the three least used builtins ever. > 3) Beginners already have to learn about importing (eg "from math > import sin") and this is just one more importable. As far as I can tell, every language that supports a clear/cls command, from old BASIC to modern shells, makes it available immediately. You don't have to adjust the PATH in bash to get access to clear, it just works. Think of this as the companion to print. If you think about Python as an interactive interpreter, its weird to have print, help, quit available but not clear. Its a fairly fundamental interactive command. > > So that's the easy part: just agree with me and import it in site.py, > > wherever it happens to come from. *wink* > > This is python-ideas! We don't "just agree" here, we argue! Even if we > don't have to! I disagree! *wink* -- Steve ___ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-le...@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/XBXNLWGXCTSM5BQCNZQNU5DFXVI7L4TK/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-ideas] Re: Unpack operator "**" and Mapping
On Sat, Dec 26, 2020 at 06:52:46PM -0800, Brendan Barnwell wrote: > On 2020-12-26 18:44, Steven D'Aprano wrote: > >I think if we were designing mapping protocols now, that would be an > >excellent idea, but we aren't, we have decades of history from `dict` > >behind us. And protocols from dict use `keys()` and getitem. E.g. > >update. > >>> > >>> What do you mean by "protocols from dict"? What are these protocols? > >"And protocols from dict use `keys()` and getitem. E.g. update." > > If I understand you right, that's not a protocol, that's just the > behavior of the dict type specifically. As far as I can tell, it's not > even documented behavior, so it doesn't constrain anything. Yes it is documented: help(dict.update) and it was intentionally the inspiration for the behaviour of dict augmented assignment. If you want to argue it's not a protocol, just an interface, okay, it's an interface. That's a difference that makes no difference. -- Steve ___ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-le...@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/2RBFMYAZ4QA2W4ZITAZVECBOF6WZST2E/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-ideas] Re: built in to clear terminal
On 2020-12-27 at 13:24:34 +1100, Steven D'Aprano wrote: > But the ctrl-L trick has no discoverability. It took me close to > twenty years of using Linux before I discovered it, and I still don't > remember to use it when I need it. The first time a sysadmin added readline to one of our work computers, it took me *minutes* to be stunned that my terminal window was being cleared for no reason. It took me longer to discover that the reason was accidental Ctrl-L's (no doubt an off-by-one error trying to press Ctrl-K), and way longer to figure out what piece of [software] was intercepting that keystroke and clearning my screen. The very first line in my .inputrc file is as follows: Control-L: redraw-current-line > Beginners and casual users aren't going to know ctrl-L, or stumble > across it through experimentation. People like me aren't going to > remember to use it. For interactive use, os.clear is barely any > better. Technically, I was a professional, and I stumbled on it through fat fingers and not knowing that readline had been installed and did horrible things like that out of the box. And then I disabled it as quickly as I could figure out how. Call me 0 (because I don't care enough to be +0 or -0) on the whole thing, as long as it's spelled with more than one keystroke (to prevent accidental invocation). ___ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-le...@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/TPJFVV3ZI2IBCYL6DABKSQMGSQQVSYNF/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-ideas] Re: Unpack operator "**" and Mapping
On Sun, Dec 27, 2020 at 1:45 PM Greg Ewing wrote: > > On 27/12/20 3:03 pm, Chris Angelico wrote: > > But that would mean that a lot of iterables would look like mappings > > when they're not. > > In the context of ** you're expecting a mapping, not a sequence. > Exactly; and under the proposal I was replying to, ANY object would look like a mapping if you can iterate over it and subscript it - which means all sequences look like mappings, just broken ones. ChrisA ___ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-le...@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/SRQFBK65CGYETKVHZPFCY6NNA4N73NBW/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-ideas] Re: built in to clear terminal
On Sun, Dec 27, 2020 at 1:25 PM Steven D'Aprano wrote: > But the ctrl-L trick has no discoverability. It took me close to twenty > years of using Linux before I discovered it, and I still don't remember > to use it when I need it. > > Beginners and casual users aren't going to know ctrl-L, or stumble > across it through experimentation. People like me aren't going to > remember to use it. For interactive use, os.clear is barely any better. > > (Me, I'll be looking in shutils, not os, because this is a shell > feature, not an os feature.) True. Plus, despite my misgivings about how it'd be misused, this IS a useful feature to have within a program, and making a program press Ctrl-L isn't reliable. > > So I think the reason to have it in the std lib is for scripting, and I > > think putting it in os or sys is just fine. > > Putting it in sys requires it to be built into the interpreter, which > means it has to be written in C (for CPython). Putting it in os is fine, > that can easily delegate to platform specific code in posix.py, nt.py > etc, with a pure-Python lowest common denominator fallback for weird > platforms that don't support anything sensible. os or shutil would be fine with me, and I know for sure that a lot of people will look in the wrong one. My inclination would be for os, since shutil is described as "high level FILE operations", and is mainly focused on file system rather than console. But that's bikeshedding. > I just want site.py to import it at startup for the sake of beginners. > Is that so bad? It adds one line to site.py, four if you want to make it > conditional: > > try: > from os import clear > except: > pass > > I can easily add it to my own personal PYTHONSTARTUP file, but the > average beginner can't. Let's throw newbies a bone and pre-import it for > them, they're the ones who seem to expect it the most. This is actually the exact part that I think should *not* happen, for several reasons: 1) Stuff that exists interactively and doesn't exist in a script is a barrier to that migration. I don't think this feature justifies that barrier. 2) The builtins are already incredibly busy, and this, again, doesn't justify being one 3) Beginners already have to learn about importing (eg "from math import sin") and this is just one more importable. > So that's the easy part: just agree with me and import it in site.py, > wherever it happens to come from. *wink* This is python-ideas! We don't "just agree" here, we argue! Even if we don't have to! > The hard part is deciding exactly what it should do, e.g. clear the > scrollback or not. I think Chris makes a good point that real beginners > will misuse this to hide the evidence whenever they get an error, so I > think the default should be to *not* clear the scrollback, for the sake > of educators, teachers and trainers. People who know what the scrollback > is and definitely want to clear it can just provide an extra argument to > the function. > > So let's just all agree I'm right *wink* and move on to the *really* > hard part, which is ensuring it works on Windows and weird terminals. Okay, so I think there are a few things we can agree on. * Having a function to clear the screen can be abused, but is still worth having * It shouldn't be based on os.system, which top SO results are advocating. * It should work on all OSes, consoles, etc, if possible * It doesn't have to work in a GUI; Idle can monkeypatch it My personal shed colour preferences: * os module, not sys, not shutil, not builtin * clearscreen(), not cls(), although I'd accept clear() * Suppressing scrollback is a downside, but better than not having the feature at all * Beginners should NOT be encouraged to use this too much The last one is hard to define, but I'm generally disinclined towards any proposal that makes it too easy for a beginner to clear the screen all the time. Life is SO much easier as an educator if you can do post-mortem analysis, not just of the one most recent action, but of all previous ones. Even among more advanced students, I see far too many cases where they try to "tidy up" (by clearing the screen, or closing the terminal and opening a new one, or whatever), and in doing so, throw away valuable error messages or indicative commands (like if someone wrote "os = 'Linux'" and then broke all os.x() calls). ChrisA ___ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-le...@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/7AUANHJPTZAIMWEGV4AAISDCAACWI2PZ/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-ideas] Re: Unpack operator "**" and Mapping
On 2020-12-26 18:44, Steven D'Aprano wrote: > >I think if we were designing mapping protocols now, that would be an > >excellent idea, but we aren't, we have decades of history from `dict` > >behind us. And protocols from dict use `keys()` and getitem. E.g. > >update. > >What do you mean by "protocols from dict"? What are these protocols? "And protocols from dict use `keys()` and getitem. E.g. update." If I understand you right, that's not a protocol, that's just the behavior of the dict type specifically. As far as I can tell, it's not even documented behavior, so it doesn't constrain anything. -- Brendan Barnwell "Do not follow where the path may lead. Go, instead, where there is no path, and leave a trail." --author unknown ___ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-le...@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/KBUBD4DE4D4EFRMRJDRYLQJXFXTYXY2V/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-ideas] Re: Unpack operator "**" and Mapping
On 27/12/20 3:03 pm, Chris Angelico wrote: But that would mean that a lot of iterables would look like mappings when they're not. In the context of ** you're expecting a mapping, not a sequence. -- Greg ___ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-le...@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/7HSMEYDRGANI2XX6QBAMGP6IDNFK4JXK/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-ideas] Re: Unpack operator "**" and Mapping
On Sat, Dec 26, 2020 at 06:09:42PM -0800, Brendan Barnwell wrote:
> On 2020-12-26 18:00, Steven D'Aprano wrote:
> >I think if we were designing mapping protocols now, that would be an
> >excellent idea, but we aren't, we have decades of history from `dict`
> >behind us. And protocols from dict use `keys()` and getitem. E.g.
> >update.
>
> What do you mean by "protocols from dict"? What are these protocols?
"And protocols from dict use `keys()` and getitem. E.g. update."
The dict in-place union assignment operator also uses the same protocol:
>>> class A:
... def keys(self):
... return iter('abc')
... def __getitem__(self, key):
... return key.upper()
...
>>> d = {}
>>> d |= A()
>>> d
{'a': 'A', 'b': 'B', 'c': 'C'}
(Regular union operator does not, it requires an actual dict.)
There may be others. I know I have written code that followed the same
interface as update, although I don't have it easily at hand.
--
Steve
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at
https://mail.python.org/archives/list/python-ideas@python.org/message/PO3EOV3Z6XADC4WIHO2XHQL4COOBLEVQ/
Code of Conduct: http://python.org/psf/codeofconduct/
[Python-ideas] Re: Unpack operator "**" and Mapping
On Sun, Dec 27, 2020 at 1:15 PM Brendan Barnwell wrote:
>
> On 2020-12-26 18:03, Chris Angelico wrote:
> > On Sun, Dec 27, 2020 at 11:36 AM Greg Ewing
> > wrote:
> >>
> >> On 27/12/20 10:15 am, Christopher Barker wrote:
> >> > It does seem like ** could be usable with any iterable that returns
> >> > pairs of objects. However the trick is that when you iterate a dict, you
> >> > get the keys, not the items, which makes me think that the only thing
> >> > you should *need* is an items() method that returns an iterable (pf
> >> > pairs of objects).
> >>
> >> It seems to me it would be more fundamental to use iteration to get
> >> the keys and indexing to get the corresponding values. You're only
> >> relying on dunder methods then.
> >>
> >
> > But that would mean that a lot of iterables would look like mappings
> > when they're not. Consider:
> >
> def naive_items(x):
> > ... return [(key, x[key]) for key in x]
> > ...
> naive_items(range(9, -1, -1))
> > [(9, 0), (8, 1), (7, 2), (6, 3), (5, 4), (4, 5), (3, 6), (2, 7), (1, 8),
> > (0, 9)]
>
> I don't see that as a major problem. It is no more "surprising" than
> doing something like list('abc') and getting ['a', 'b', 'c']. If you do
> {**range(9, -1, -1)} you may get a result that looks strange or isn't
> useful, but as long as the result is consistent with the protocol,
> that's fine. Just don't use **-unpacking with ranges if you don't want to.
>
Perhaps, but that means you can't raise TypeError for anything
iterable. Instead, you'd have to raise ValueError, because it could
potentially be valid. Are mappings really just iterables with indexing
(which most iterables support), or are they distinctly different?
Remember, most things iterate over their *values*, but a dict iterates
over its *keys*.
On the plus side, you COULD convert some objects into sparse lists by
basically just dictifying them:
>>> {**range(5)} # with this proposal
{0: 0, 1: 1, 2: 2, 3: 3, 4: 4}
But on the downside, you could only do this if they iterate the 'right
way', which most iterables won't. It would be MUCH more useful if
there were a dedicated way to request the valid keys or items, and use
that instead. Then you could convert *any* iterable that way.
At the moment, you can create an actual dict from any iterable by
enumerating, and that will give the correct items:
>>> def smarter_items(x):
... return list(dict(enumerate(x)).items())
...
>>> smarter_items(range(9, -1, -1))
[(0, 9), (1, 8), (2, 7), (3, 6), (4, 5), (5, 4), (6, 3), (7, 2), (8, 1), (9, 0)]
>>> smarter_items(range(10, 20))
[(0, 10), (1, 11), (2, 12), (3, 13), (4, 14), (5, 15), (6, 16), (7,
17), (8, 18), (9, 19)]
If you want to dictify something, that'd be the more normal way to do
it, IMO. Instead of something with lots of false positives, wouldn't
it be cleaner to have a protocol that specifically returns the
equivalent of dict.items()?
ChrisA
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at
https://mail.python.org/archives/list/python-ideas@python.org/message/S24DU6DYBNFHMWIIDQU23YW5MI5OUU7U/
Code of Conduct: http://python.org/psf/codeofconduct/
[Python-ideas] Re: built in to clear terminal
On Sat, Dec 26, 2020 at 05:58:16PM -0800, Christopher Barker wrote: > > > > On 2020-12-22 14:01, Steven D'Aprano wrote: > > > It seems to me that the primary use for this will be in the interactive > > > interpreter. People are used to something like "cls" or "clear" from > > > various shells. Using it in scripting is, I think, a secondary use-case. > > > ' > I disagree. yes, I want to clear the screen a lot during interactive use, > but I'd rather do that with a keystroke (cmd+K on the Mac) -- and indeed, > do that with iPython all the time, even though iPython has the command > built in. And nobody is going to take that away from you, unless you choose to go to Windows, where ctrl-L doesn't work. But the ctrl-L trick has no discoverability. It took me close to twenty years of using Linux before I discovered it, and I still don't remember to use it when I need it. Beginners and casual users aren't going to know ctrl-L, or stumble across it through experimentation. People like me aren't going to remember to use it. For interactive use, os.clear is barely any better. (Me, I'll be looking in shutils, not os, because this is a shell feature, not an os feature.) > So I think the reason to have it in the std lib is for scripting, and I > think putting it in os or sys is just fine. Putting it in sys requires it to be built into the interpreter, which means it has to be written in C (for CPython). Putting it in os is fine, that can easily delegate to platform specific code in posix.py, nt.py etc, with a pure-Python lowest common denominator fallback for weird platforms that don't support anything sensible. I just want site.py to import it at startup for the sake of beginners. Is that so bad? It adds one line to site.py, four if you want to make it conditional: try: from os import clear except: pass I can easily add it to my own personal PYTHONSTARTUP file, but the average beginner can't. Let's throw newbies a bone and pre-import it for them, they're the ones who seem to expect it the most. So that's the easy part: just agree with me and import it in site.py, wherever it happens to come from. *wink* The hard part is deciding exactly what it should do, e.g. clear the scrollback or not. I think Chris makes a good point that real beginners will misuse this to hide the evidence whenever they get an error, so I think the default should be to *not* clear the scrollback, for the sake of educators, teachers and trainers. People who know what the scrollback is and definitely want to clear it can just provide an extra argument to the function. So let's just all agree I'm right *wink* and move on to the *really* hard part, which is ensuring it works on Windows and weird terminals. -- Steve ___ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-le...@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/RAU2WORPY2E7IVPYBWM76YXSG43KACAE/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-ideas] Re: Unpack operator "**" and Mapping
On 2020-12-26 16:34, Greg Ewing wrote: On 27/12/20 10:15 am, Christopher Barker wrote: It does seem like ** could be usable with any iterable that returns pairs of objects. However the trick is that when you iterate a dict, you get the keys, not the items, which makes me think that the only thing you should *need* is an items() method that returns an iterable (pf pairs of objects). It seems to me it would be more fundamental to use iteration to get the keys and indexing to get the corresponding values. You're only relying on dunder methods then. This is what I was thinking as well. I don't like the idea of relying on a non-dunder method like .keys() to implement syntax like ** unpacking. -- Brendan Barnwell "Do not follow where the path may lead. Go, instead, where there is no path, and leave a trail." --author unknown ___ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-le...@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/DTPGJ4HOZSHIEFKBINHMNW3XXXSD3OZS/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-ideas] Re: Unpack operator "**" and Mapping
On 2020-12-26 18:03, Chris Angelico wrote:
On Sun, Dec 27, 2020 at 11:36 AM Greg Ewing wrote:
On 27/12/20 10:15 am, Christopher Barker wrote:
> It does seem like ** could be usable with any iterable that returns
> pairs of objects. However the trick is that when you iterate a dict, you
> get the keys, not the items, which makes me think that the only thing
> you should *need* is an items() method that returns an iterable (pf
> pairs of objects).
It seems to me it would be more fundamental to use iteration to get
the keys and indexing to get the corresponding values. You're only
relying on dunder methods then.
But that would mean that a lot of iterables would look like mappings
when they're not. Consider:
def naive_items(x):
... return [(key, x[key]) for key in x]
...
naive_items(range(9, -1, -1))
[(9, 0), (8, 1), (7, 2), (6, 3), (5, 4), (4, 5), (3, 6), (2, 7), (1, 8), (0, 9)]
I don't see that as a major problem. It is no more "surprising" than
doing something like list('abc') and getting ['a', 'b', 'c']. If you do
{**range(9, -1, -1)} you may get a result that looks strange or isn't
useful, but as long as the result is consistent with the protocol,
that's fine. Just don't use **-unpacking with ranges if you don't want to.
--
Brendan Barnwell
"Do not follow where the path may lead. Go, instead, where there is no
path, and leave a trail."
--author unknown
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at
https://mail.python.org/archives/list/python-ideas@python.org/message/B6DRJ5YVCMXTY3E4R2RYJD6BSYV33PZL/
Code of Conduct: http://python.org/psf/codeofconduct/
[Python-ideas] Re: Unpack operator "**" and Mapping
On 2020-12-26 18:00, Steven D'Aprano wrote: On Sun, Dec 27, 2020 at 01:34:02PM +1300, Greg Ewing wrote: On 27/12/20 10:15 am, Christopher Barker wrote: >It does seem like ** could be usable with any iterable that returns >pairs of objects. However the trick is that when you iterate a dict, you >get the keys, not the items, which makes me think that the only thing >you should *need* is an items() method that returns an iterable (pf >pairs of objects). It seems to me it would be more fundamental to use iteration to get the keys and indexing to get the corresponding values. You're only relying on dunder methods then. I think if we were designing mapping protocols now, that would be an excellent idea, but we aren't, we have decades of history from `dict` behind us. And protocols from dict use `keys()` and getitem. E.g. update. What do you mean by "protocols from dict"? What are these protocols? -- Brendan Barnwell "Do not follow where the path may lead. Go, instead, where there is no path, and leave a trail." --author unknown ___ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-le...@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/4HOWZPELGXVBP7UKKNUUZUMH7C7SA6ZZ/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-ideas] Re: Unpack operator "**" and Mapping
On Sun, Dec 27, 2020 at 11:36 AM Greg Ewing wrote: > > On 27/12/20 10:15 am, Christopher Barker wrote: > > It does seem like ** could be usable with any iterable that returns > > pairs of objects. However the trick is that when you iterate a dict, you > > get the keys, not the items, which makes me think that the only thing > > you should *need* is an items() method that returns an iterable (pf > > pairs of objects). > > It seems to me it would be more fundamental to use iteration to get > the keys and indexing to get the corresponding values. You're only > relying on dunder methods then. > But that would mean that a lot of iterables would look like mappings when they're not. Consider: >>> def naive_items(x): ... return [(key, x[key]) for key in x] ... >>> naive_items(range(9, -1, -1)) [(9, 0), (8, 1), (7, 2), (6, 3), (5, 4), (4, 5), (3, 6), (2, 7), (1, 8), (0, 9)] ChrisA ___ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-le...@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/7BO62PFYBMXYPJFB6V4YBMMK774SGBY7/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-ideas] Re: Unpack operator "**" and Mapping
On Sun, Dec 27, 2020 at 01:34:02PM +1300, Greg Ewing wrote: > On 27/12/20 10:15 am, Christopher Barker wrote: > >It does seem like ** could be usable with any iterable that returns > >pairs of objects. However the trick is that when you iterate a dict, you > >get the keys, not the items, which makes me think that the only thing > >you should *need* is an items() method that returns an iterable (pf > >pairs of objects). > > It seems to me it would be more fundamental to use iteration to get > the keys and indexing to get the corresponding values. You're only > relying on dunder methods then. I think if we were designing mapping protocols now, that would be an excellent idea, but we aren't, we have decades of history from `dict` behind us. And protocols from dict use `keys()` and getitem. E.g. update. I think it would be confusing to have dict protocols use keys and double star use items, so I think it would be better to follow the API of dict.update: D.update([E, ]**F) -> None. Update D from dict/iterable E and F. If E is present and has a .keys() method, then does: for k in E: D[k] = E[k] If E is present and lacks a .keys() method, then does: for k, v in E: D[k] = v So to have an object E usable with double star dict unpacking, perhaps it needs to either: * have a keys() method which iterates over keys (assumed to all be strings), and `__getitem__`; or * support iteration, yielding (key, value) pairs. -- Steve ___ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-le...@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/5VXR3ZDKGF6BQDEGQXELSEKIDVVLPLGG/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-ideas] Re: built in to clear terminal
> > On 2020-12-22 14:01, Steven D'Aprano wrote: > > It seems to me that the primary use for this will be in the interactive > > interpreter. People are used to something like "cls" or "clear" from > > various shells. Using it in scripting is, I think, a secondary use-case. > ' I disagree. yes, I want to clear the screen a lot during interactive use, but I'd rather do that with a keystroke (cmd+K on the Mac) -- and indeed, do that with iPython all the time, even though iPython has the command built in. So I think the reason to have it in the std lib is for scripting, and I think putting it in os or sys is just fine. -CHB > > Given that, I think that like other conveniences for interactive use > > like `help`, it should be available without an import. > > > Agreed in general, however Ctrl+L is mostly available for this purpose. > There > are two major exceptions unfortunately. > > One is when using cmd.exe. However, every other combination of shell and > console/terminal on Windows does support it. Not to mention everywhere > Unix has > an influence. > > Second is the Python REPL console on Windows. I'm guessing it wouldn't be > too > hard to add Ctrl+L handling to it. Would be useful I think, regardless of > some > of the other decisions that need to be made. > > -Mike > ___ > Python-ideas mailing list -- python-ideas@python.org > To unsubscribe send an email to python-ideas-le...@python.org > https://mail.python.org/mailman3/lists/python-ideas.python.org/ > Message archived at > https://mail.python.org/archives/list/python-ideas@python.org/message/FU7MZEY6NPUN54W7QMFNABUQB6LSPP2Z/ > Code of Conduct: http://python.org/psf/codeofconduct/ > -- Christopher Barker, PhD Python Language Consulting - Teaching - Scientific Software Development - Desktop GUI and Web Development - wxPython, numpy, scipy, Cython ___ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-le...@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/MMVBLUS5XDORIFCK7XYBVVYVL7GLZXP5/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-ideas] Re: Unpack operator "**" and Mapping
On Sat, Dec 26, 2020 at 4:35 PM Greg Ewing wrote: > On 27/12/20 10:15 am, Christopher Barker wrote: > > ... the only thing > > you should *need* is an items() method that returns an iterable (pf > > pairs of objects). > > It seems to me it would be more fundamental to use iteration to get > the keys and indexing to get the corresponding values. You're only > relying on dunder methods then. > And that seems to work as well, I made a "Mapping" with a do-nothing .items(), but a functional __iter__ and __getitem__ and that seems to work as well. Though it seems that .items() could be more efficient, if it's there. Maybe not significant, but still ... In any case, I'd like to see the protocol be duck-typed, like, as far as I know, every other protocol in Python, rather than doing actual type checking. I'd live to hear why it currently checks for Mapping, rather than simply calling the methods it needs. -CHB > -- > Greg > ___ > Python-ideas mailing list -- python-ideas@python.org > To unsubscribe send an email to python-ideas-le...@python.org > https://mail.python.org/mailman3/lists/python-ideas.python.org/ > Message archived at > https://mail.python.org/archives/list/python-ideas@python.org/message/Z4UL4NZQ7JS3JLVMYOZZVN2N3FL6UKCW/ > Code of Conduct: http://python.org/psf/codeofconduct/ > -- Christopher Barker, PhD Python Language Consulting - Teaching - Scientific Software Development - Desktop GUI and Web Development - wxPython, numpy, scipy, Cython ___ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-le...@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/RTFZ554U72DFJPT3QXUQOPZA5AZXWTHX/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-ideas] Re: fsync-on-close io object
On Sat, Dec 26, 2020 at 07:39:42PM -0500, Michael Smith wrote: > I love context managers when they're alone, but I dislike stacking > them. It is less clear how we can ensure the fsync happens exactly > between flush and close with a context manager than a keyword argument > to open. That is, if open is the only context manager, everything is > great. But if is up to users to stack context managers including open > and some fsync, I think correct ordering will be a problem. Indeed. I started playing with this idea, and even though I intellectually know that you can't sync a closed file (fsync needs to know the file descriptor ID) I keep wanting to write: with sync(): with open(...) as f: ... I dunno, maybe it's just me, and I'm sure that if I get a sensible exception on that I would soon learn to write it as: with open(...) as f: with sync(f): ... But what concerns me more is what happens in more complex cases, for example with two or more files being written: with open(..., 'w') as f: with open(..., 'w') as g: with sync(g): with sync(f): ... Aside from the deeper nesting, its not at all obvious to me whether the syncs need to be in the same or opposite order to the opens, or if it even matters. And what if people insert code around the sync? with open(...) as f: # code writing to the file here is probably safe with sync(f): ... # but code writing to the file here is not If I have to remember all these fiddly little rules for the order of context managers and what can go where, I'd rather than stick to syncing manually: with open(...) as f: write stuff sync even though I have no idea of the pros and cons of fdatasync versus fsync and will probably just forget and use os.sync. Even though I don't usually like functions with a million arguments, open is an exception. I'm +1 on adding a keyword-only "sync=False" parameter to open: - if the file is opened in read-only mode, has no effect; - if the file is opened for writing, guarantees that fdatasync or fsync will be called after flush and before closing. (I'm not entirely sure fdatasync is better than fsync, it's only available on non-OS-X Unixes and I don't know why I would want to update the file data but not the metadata.) -- Steve ___ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-le...@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/FCITSUEHTFWK2ZYYUOE2WF64DVXHCSY3/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-ideas] Re: fsync-on-close io object
On Thu, Dec 24, 2020 at 06:06:32PM -0600, re:fi.64 wrote: > At least on Linux, this wouldn't be that ideal. These LWN articles go over > the general fsync mess and the lack of viable alternatives: > > https://lwn.net/Articles/351422/ But isn't that a trade-off application developers can make for themselves? Performance over reliability? We're not suggesting that fsync before close becomes mandatory or even the default. People have to opt in to the behaviour, just as they do now. -- Steve ___ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-le...@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/GCPH7P7GREEGKSHUPBQ5WLGZEKVWQVHN/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-ideas] Re: fsync-on-close io object
On Thu, Dec 24, 2020 at 6:18 PM Cameron Simpson wrote:
>
> On 25Dec2020 09:29, Steven D'Aprano wrote:
> >On Thu, Dec 24, 2020 at 12:15:08PM -0500, Michael A. Smith wrote:
> >
> >> With all the buffering that modern disks and filesystems do, a
> >> specific question has come up a few times with respect to whether or
> >> not data was actually written after flush. I think it would be pretty
> >> useful for the standard library to have a variant in the io module
> >> that would explicitly fsync on close.
> >
> >One argument against this idea is that "disks and file systems buffer
> >for a reason, you should trust them, explicitly calling sync after every
> >written file is just going to slow I/O down".
> >
> >Personally I don't believe this argument, I've been bitten many, many
> >times until I learned to explicitly sync files, but its an argument you
> >should counter.
>
> By contrast, I support this argument. The _vast_ majority of things
> don't need to sync their data all the way to the hardware base substrate
> (eg magnetic fields on spinning rust).
>
> And on the whole, if I do care, I issue a single sync() call at the end
> of a large task (typically interactively, at a prompt!) rather than
> forcing a heap of performance impairing stutters all the way through
> some process because many per-file syncs force that.
>
> IMO, per-file syncs fall into the "policy" arena: aside from low level
> tools (example: fdisk, a disc partition editor), to my mind the purpose
> of the kernel is to accept responsibility for my data when I hand it
> off.
>
> Perhaps for you that isn't enough; for me it normally is. And when it
> isn't, I'll take steps myself, _outside_ the programme, to ensure the
> sync or commit or off site backup is complete when it matters. Thus the
> policy is in my hands.
>
> The tool which causes a per-file sync all on every close, or even after
> every write, is a performance killer. The faster our hardware, the less
> that may seem to matter (and, conversely, the less the risk as the
> ordinary kernel I/O flushing will catch up faster). But when the
> hardware slowness _is_ relevant, if I can't turn that off I have a
> needlessly unperformant task.
>
>
> The example which stands out in my own mind is when I was using firefox
> on a laptop with a spinning rust hard drive (and being a laptop
> hardware, a low power physically slow piece of spinning rust). There was
> once a setting to turn off the synchronous-write sqlite setting (used
> for history and bookmarks). That was _visibly obvious_ in the user
> experience. And I turned it off. As a matter of policy, those data
> didn't need such care.
> So I'm resistant to this kind of thing because IMO it leads to an
> attractive nuisance: over use of sync or fsync for everything. And it
> will usually not be exposed as policy the user can adjust/disable.
>
> My rule of thumb:
>
> If it can't be turned off, it's not a feature. - Karl Heuer
Are you arguing that if something is a bad idea to overuse, even if
it's a good idea sometimes, then it shouldn't be allowed into Python,
because someone might write a program that abuses that feature, you
might end up with that program, and it would be irksome to deal with
it?
I'm not trying to present a straw man, but that is my genuine
impression of what you said. If I got it wrong, I apologize and please
help me understand what you meant.
> >Another argument is that even syncing your data doesn't mean that the
> >data is actually written to disk, since the hardware can lie. On the
> >other hand, I don't know what anyone can do, not even the kernel, in the
> >face of deceitful hardware.
>
> Aye.
>
> But in principle, after a sync() or fsync() the kernel at least believes
> that. Hardware which lies, or which claims saved data without having the
> rresources to guarrentee it (eg a small battery to complete the writes
> if there's a power out) is indeed nasty.
>
> >> You might be tempted to argue that this can be done very easily in
> >> Python already, so why include it in the standard io module?
>
> I would indeed. There _should_ be a small bar which at least causes the
> programmer to think "do I really need this here"? I suppose a
> "fsync=False" default parameter is a visible bar.
>
> [...]
> >I mean, the obvious way is:
> >
> >try:
> >with open(..., 'w') as f:
> >f.write("stuff")
> >finally:
> >os.sync()
>
> An os.fsync(f.fileno()) is lower impact - os.sync() requests a sync of
> all filesystems.
>
> >so maybe all we really need is a "sync file" context manager.
>
> Aye. Fully agree here, and frankly think this is a "write your own"
> situation. Except, of course, that like all "write your own" one/few
> liners there will be suboptimal or buggy ones released. Such as the
> "overly wide sync" from your os.sync() above.
>
> Personally I'm -1 on this. A context manager while goes f.flush()
> os.fsync(f.fileno()) seems plenty, and easy to roll your own.
There
[Python-ideas] Re: Unpack operator "**" and Mapping
On 27/12/20 10:15 am, Christopher Barker wrote: It does seem like ** could be usable with any iterable that returns pairs of objects. However the trick is that when you iterate a dict, you get the keys, not the items, which makes me think that the only thing you should *need* is an items() method that returns an iterable (pf pairs of objects). It seems to me it would be more fundamental to use iteration to get the keys and indexing to get the corresponding values. You're only relying on dunder methods then. -- Greg ___ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-le...@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/Z4UL4NZQ7JS3JLVMYOZZVN2N3FL6UKCW/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-ideas] Re: fsync-on-close io object
Also sorry for the duplicate, Steven.
On Thu, Dec 24, 2020 at 5:31 PM Steven D'Aprano wrote:
> On Thu, Dec 24, 2020 at 12:15:08PM -0500, Michael A. Smith wrote:
>
> > With all the buffering that modern disks and filesystems do, a
> > specific question has come up a few times with respect to whether or
> > not data was actually written after flush. I think it would be pretty
> > useful for the standard library to have a variant in the io module
> > that would explicitly fsync on close.
>
> One argument against this idea is that "disks and file systems buffer
> for a reason, you should trust them, explicitly calling sync after every
> written file is just going to slow I/O down".
>
> Personally I don't believe this argument, I've been bitten many, many
> times until I learned to explicitly sync files, but its an argument you
> should counter.
>
Thanks for playing devil's advocate. I can see this argument came up in
some other branches on this thread. I'll address it there.
Another argument is that even syncing your data doesn't mean that the
> data is actually written to disk, since the hardware can lie. On the
> other hand, I don't know what anyone can do, not even the kernel, in the
> face of deceitful hardware.
>
Death and taxes. The best we could do here is address it in the
documentation as something else to be aware of.
That would be fine with me.
> 3. There are many ways to do this, and I think several of them could
> > be subtly incorrect.
>
> Can you elaborate?
>
> I mean, the obvious way is:
>
> try:
> with open(..., 'w') as f:
> f.write("stuff")
> finally:
> os.sync()
>
I think this illustrates my point. Aside from os.sync syncing more than the
user might expect, I believe the sync here happens after `f` is closed.
IIUC, it's safest to fsync between flush and close.
Reference:
https://stackoverflow.com/questions/37288453/calling-fsync2-after-close2
We'd also probably want to fdatasync() instead of fsync or os.sync if we
can.
so maybe all we really need is a "sync file" context manager.
>
I think that since open is already a context manager, putting another
context manager in play is not as desirable as another keyword argument to
open.
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at
https://mail.python.org/archives/list/python-ideas@python.org/message/PQGBPCJCJH2LZQIA62SFOMCO46RMR5UG/
Code of Conduct: http://python.org/psf/codeofconduct/
[Python-ideas] Re: fsync-on-close io object
Sorry for the duplicate, Barry. I got bit by the "I don't reply-all by default" spider. On Sat, Dec 26, 2020 at 12:30 PM Barry Scott wrote: > > > > On 24 Dec 2020, at 17:15, Michael A. Smith wrote: > > > > With all the buffering that modern disks and filesystems do, a > > specific question has come up a few times with respect to whether or > > not data was actually written after flush. I think it would be pretty > > useful for the standard library to have a variant in the io module > > that would explicitly fsync on close. > > > > You might be tempted to argue that this can be done very easily in > > Python already, so why include it in the standard io module? > > > > 1. It seems to me that it would be better to do this in C, so for the > > folks who need to make a consistency > performance kind of choice, > > they don't have to sacrifice any additional performance. > > 2. Having it in the io library will call attention to this issue, > > which I think is something a lot of folks don't consider. Assuming > > that `close` or `flush` are sufficient for consistency has always been > > wrong (at its limits), but it was less likely to be a stumbling block > > in the past, when buffering was less aggressive and less layered and > > the peak size and continuous-ness of data streams was a more niche > > concern. > > 3. There are many ways to do this, and I think several of them could > > be subtly incorrect. In other words, maybe it can't be done very > > easily and correctly after all. Providing "obviously right" ways to do > > things is the baileywick of the standard library, isn't it? > > I have used rename to make a new file appear atomically after it is closed > and I have used fsync to ensure records are on disk before a file is > closed. > > I've not needed fsync on close yet. > I'm confused -- did you not just describe fsync-on-close? Using fsync to ensure records are on disk before a file is closed is what we're talking about. Perhaps you think I mean to fsync after close? What I really mean is 1. flush if applicable 2. (fdatasync or fsync) if indicated 3. close in that order. What is the use case that needs this? One use case is atomically writing a file by first writing to a temporary file, then renaming the temporary file to its real destination, such as the problem described by He Chen in https://issues.apache.org/jira/browse/AVRO-3013. ___ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-le...@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/FX7OTNULBCLR22ZVUYGW4GWR33JGUDLI/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-ideas] Re: built in to clear terminal
On 2020-12-22 14:01, Steven D'Aprano wrote: It seems to me that the primary use for this will be in the interactive interpreter. People are used to something like "cls" or "clear" from various shells. Using it in scripting is, I think, a secondary use-case. Given that, I think that like other conveniences for interactive use like `help`, it should be available without an import. Agreed in general, however Ctrl+L is mostly available for this purpose. There are two major exceptions unfortunately. One is when using cmd.exe. However, every other combination of shell and console/terminal on Windows does support it. Not to mention everywhere Unix has an influence. Second is the Python REPL console on Windows. I'm guessing it wouldn't be too hard to add Ctrl+L handling to it. Would be useful I think, regardless of some of the other decisions that need to be made. -Mike ___ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-le...@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/FU7MZEY6NPUN54W7QMFNABUQB6LSPP2Z/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-ideas] Re: Unpack operator "**" and Mapping
My first thought is that for dataclasses, you can use the asdict() method, and you're done. But sure -- why not make it more generic. It does seem like ** could be usable with any iterable that returns pairs of objects. However the trick is that when you iterate a dict, you get the keys, not the items, which makes me think that the only thing you should *need* is an items() method that returns an iterable (pf pairs of objects). Indeed, if I make a subclass of Mapping, and define everything but items() with dummy methods, then it does work, so only items() is being used. So all we need to do is not check for a Mapping, but rather simply try to call `.items()`, which seems in the spirit of Python Duck Typing to anyway. And sequence unpacking seems to be only calling __iter__ -- so why not something similar for ** unpacking? In [35]: class Seq: ...: def __iter__(self): ...: return iter([3, 4]) ...: In [36]: s = Seq() In [37]: x, y = s In [38]: x Out[38]: 3 In [39]: y Out[39]: 4 But in the spirit of Chesterton’s Fence: Why DOES the unpacking operator type check for Mapping? -CHB On Sat, Dec 26, 2020 at 6:18 AM Anton Abrosimov wrote: > I am trying to release comfortable dataclass unpacking using `**` > operator. Now I have 5 different ways to do it. > But not a single good one. Confused by the implementation of the unpacking > operator. > > So when I try to unpack any custom class, I get the error: > > `type object argument after ** must be a mapping, not MyClass` > > Ok, nothing special. I need to use `collections.abc.Mapping` right? > Now I need to implement: `__getitem__`, `__iter__`, `__len__`. Not a > problem. > But additionally I get: `keys`, `items`, `values`. > Hey, I don't need them. I don't need the full mapping functionality. I > only need the double asterisk to work. > > Right, we have a duck typing! > We throw out `abc.Mapping`. What do we need to implement? > It's `__getitem__` and `keys`. Wtf `keys`? > > I am looking at Python Data model: > https://docs.python.org/3/reference/datamodel.html > There many operators, and they depend on special double underscore methods. > Hmm, I don't see unpack operators there, it's strange. > But why it's `keys`? Because the historical is `dict`? > I think a dependency on `__iter__` is more preferable and expectable over > a userspace named `keys`. > Actually, `items()` is more predictable. > > But this is not the end. > The `__getitem__` overload is often used for additional checking. > I think `__iter__` and `keys` should only return a valid keys. > Therefore, we don't need to further check them when unpacking. At the very > least, we must control this. > > And in the end. > `Mapping` keys can be `Any` type. `Unpack` keys must be `str` type. > Some `Mapping` can be unpackable and some `Unpack` can be mappable. > > My suggestion: > * Add new `collections.abc.Unpack` abstract layout for `**` unpack. > * Add new special method like: > > def __unpack__(self): > if issubclass(self, collections.abc.Mapping): # Really overload this > method in `Mapping` and `dict`. > keys = self.keys() # or return self.items()? > else: > keys = iter(self) > return ((k, self[k]) for k in keys) > > * Update the implementation of the unpack operator to use the `__unpack__` > function. > > As a result: > * We can make the class unpackable without the advanced `Mapping` > functionality. > * We can control the unpacking process separately. > * We throw away userspace named dependencies. > * I think we are making behavior more predictable. > > What do you think about it? > ___ > Python-ideas mailing list -- python-ideas@python.org > To unsubscribe send an email to python-ideas-le...@python.org > https://mail.python.org/mailman3/lists/python-ideas.python.org/ > Message archived at > https://mail.python.org/archives/list/python-ideas@python.org/message/2HMRGJ672NDZJZ5PVLMNVW6KP7OHMQDI/ > Code of Conduct: http://python.org/psf/codeofconduct/ > -- Christopher Barker, PhD Python Language Consulting - Teaching - Scientific Software Development - Desktop GUI and Web Development - wxPython, numpy, scipy, Cython ___ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-le...@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/FXASC76PDP64H7FWJVM2ERVKQQTZ44EK/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-ideas] Re: Allow return value in asynchronous generator
I guess this can be extended to "allow `yield from` with asynchronous generators", maybe something like `async yield from`. ___ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-le...@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/WT6EPXR5WZ6CW4L2ZBHRJWTTEYJLM64U/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-ideas] Allow return value in asynchronous generator
Currently, asynchronous generators don't allow a non-empty `return` statement: ```python import asyncio async def async_generator(): x = yield 1 await asyncio.sleep(1) if x: ... return "result A" # SyntaxError else: ... return "result B" ``` unlike regular generators which communicate their return value via the `value` attribute on the `StopIteration` exception: ```python def synchronous_generator(): x = yield 1 if x: ... return "result A" # raises StopIteration with value="result A" else: ... return "result B" ``` I think that `StopAsyncIteration` should behave likewise and have a `value` attribute containing the return value. I'm not aware of any reason to maintain this asymmetry. ___ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-le...@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/E2J4FTVGOZDQCBN2SZSY6VTWF2GE4EKK/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-ideas] Re: fsync-on-close io object
> On 24 Dec 2020, at 17:15, Michael A. Smith wrote: > > With all the buffering that modern disks and filesystems do, a > specific question has come up a few times with respect to whether or > not data was actually written after flush. I think it would be pretty > useful for the standard library to have a variant in the io module > that would explicitly fsync on close. > > You might be tempted to argue that this can be done very easily in > Python already, so why include it in the standard io module? > > 1. It seems to me that it would be better to do this in C, so for the > folks who need to make a consistency > performance kind of choice, > they don't have to sacrifice any additional performance. > 2. Having it in the io library will call attention to this issue, > which I think is something a lot of folks don't consider. Assuming > that `close` or `flush` are sufficient for consistency has always been > wrong (at its limits), but it was less likely to be a stumbling block > in the past, when buffering was less aggressive and less layered and > the peak size and continuous-ness of data streams was a more niche > concern. > 3. There are many ways to do this, and I think several of them could > be subtly incorrect. In other words, maybe it can't be done very > easily and correctly after all. Providing "obviously right" ways to do > things is the baileywick of the standard library, isn't it? I have used rename to make a new file appear atomically after it is closed and I have used fsync to ensure records are on disk before a file is closed. I've not needed fsync on close yet. What is the use case that needs this? Without a compelling use case I'm -1 on this. Barry > > Thanks for your consideration and feedback, > Michael Smith > ___ > Python-ideas mailing list -- python-ideas@python.org > To unsubscribe send an email to python-ideas-le...@python.org > https://mail.python.org/mailman3/lists/python-ideas.python.org/ > Message archived at > https://mail.python.org/archives/list/python-ideas@python.org/message/UOUS4BNSI6WFBPWMXHJF5IEAV2PQ47VN/ > Code of Conduct: http://python.org/psf/codeofconduct/ > ___ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-le...@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/CNIIRBIEXL46FEUTVHSDU63ASBMHE72X/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-ideas] Unpack operator "**" and Mapping
I am trying to release comfortable dataclass unpacking using `**` operator. Now I have 5 different ways to do it. But not a single good one. Confused by the implementation of the unpacking operator. So when I try to unpack any custom class, I get the error: `type object argument after ** must be a mapping, not MyClass` Ok, nothing special. I need to use `collections.abc.Mapping` right? Now I need to implement: `__getitem__`, `__iter__`, `__len__`. Not a problem. But additionally I get: `keys`, `items`, `values`. Hey, I don't need them. I don't need the full mapping functionality. I only need the double asterisk to work. Right, we have a duck typing! We throw out `abc.Mapping`. What do we need to implement? It's `__getitem__` and `keys`. Wtf `keys`? I am looking at Python Data model: https://docs.python.org/3/reference/datamodel.html There many operators, and they depend on special double underscore methods. Hmm, I don't see unpack operators there, it's strange. But why it's `keys`? Because the historical is `dict`? I think a dependency on `__iter__` is more preferable and expectable over a userspace named `keys`. Actually, `items()` is more predictable. But this is not the end. The `__getitem__` overload is often used for additional checking. I think `__iter__` and `keys` should only return a valid keys. Therefore, we don't need to further check them when unpacking. At the very least, we must control this. And in the end. `Mapping` keys can be `Any` type. `Unpack` keys must be `str` type. Some `Mapping` can be unpackable and some `Unpack` can be mappable. My suggestion: * Add new `collections.abc.Unpack` abstract layout for `**` unpack. * Add new special method like: def __unpack__(self): if issubclass(self, collections.abc.Mapping): # Really overload this method in `Mapping` and `dict`. keys = self.keys() # or return self.items()? else: keys = iter(self) return ((k, self[k]) for k in keys) * Update the implementation of the unpack operator to use the `__unpack__` function. As a result: * We can make the class unpackable without the advanced `Mapping` functionality. * We can control the unpacking process separately. * We throw away userspace named dependencies. * I think we are making behavior more predictable. What do you think about it? ___ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-le...@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/2HMRGJ672NDZJZ5PVLMNVW6KP7OHMQDI/ Code of Conduct: http://python.org/psf/codeofconduct/
