Re: Terminal Emulator (Posting On Python-List Prohibited)
Modern debian (ubuntu) and fedora block users installing using pip. > Even if you're telling it to install in ~/.local? I could see not allowing to run it as root. I honestly haven't tried. Maybe I should... 🤔 I have an old laptop running XUbuntu 22.04 which I generally only use to compile the most recent branches on GitHub (main, 3.12, & 3.13 at the moment). Skip -- https://mail.python.org/mailman/listinfo/python-list
Re: PyCon
> > > I’m at PyCon in Pittsburgh and I’m haven’t an amazing time! > > s/haven’t/having/ > No need to explain/correct. We understand you are excited. Many of us have been in the same state before. ;-) Enjoy, Skip > -- https://mail.python.org/mailman/listinfo/python-list
Re: xkcd.com/353 ( Flying with Python )
> > > https://xkcd.com/1306/ > > what does SIGIL mean? > > I think its' a Perl term, referring to the $/@/# symbols in front of > identifiers. > I had a vague recollection of hearing it elsewhere (*Game of Thrones,* on the armies' battle flags?), but didn't know what it meant. Google tells me: *an inscribed or painted symbol considered to have magical power.* So, they're more than just line noise. They confer power on their users... Perhaps '@' in the context of decorators is the most prominent example in Python, since decorators technically don't allow the programmer to do something they couldn't before, but are now are used everywhere, a key feature of many applications and modules. Magical-ly, y'rs, Skip > -- https://mail.python.org/mailman/listinfo/python-list
Re: Testing (sorry)
> > Here is a typical bounce message that I get: > > : host mail.python.org[188.166.95.178] said: > 450-4.3.2 > Service currently unavailable 450 4.3.2 > > Some time after I get one of these messages I re-send the post. Usually > it gets through then. > Looks kinda like greylisting to me. I'm pretty sure that's one of the tool in the mail.python.org chain. Skip > -- https://mail.python.org/mailman/listinfo/python-list
Re: Testing (sorry)
I can't explain the delays, but will note that the gate-news program on the server runs every 5 minutes via cron. There are multiple moving parts in the overall system. You'll probably get a more useful answer from postmas...@python.org. Skip -- https://mail.python.org/mailman/listinfo/python-list
Re: test-ignore
> > True, but did the poster really need to send another one to say "yes, > that worked"? > Maybe to test the bidirectionality of the gateway? 🤷 If the messages stop I think we can let it die. It's not like this sort of activity is a regular occurrence. (A bigger problem for me was always Usenet posters who used fake email addresses.) Skip -- https://mail.python.org/mailman/listinfo/python-list
Re: test-ignore
> > > Test post to see if my Newsgroup post program is working. > > Aim your test messages at alt.test, please. > I agree that basic Usenet connectivity messages should go to alt.test. It's not clear from the original post, but if the poster's aim was to see if posts to comp.lang.python traverse the gateway and show up on this list, then alt.test won't help. Skip > -- https://mail.python.org/mailman/listinfo/python-list
Re: Question about garbage collection
> I do have several circular references. My experience is that if I do not > take some action to break the references when closing the session, the > objects remain alive. Below is a very simple program to illustrate this. > > Am I missing something? All comments appreciated. Python has normal reference counting, but also has a cyclic garbage collector. Here's plenty of detail about how it works: https://devguide.python.org/internals/garbage-collector/index.html Skip -- https://mail.python.org/mailman/listinfo/python-list
Re: Python 3.11.5 Pip Issue
> I downloaded Python 3.11.5, and there was nothing in the “Scripts” file, > and there was no Pip. I would like to know why. > Can't help with the empty/missing Scripts folder. Does running python -m ensurepip get you a working pip? Which you should then run as python -m pip ... Skip > -- https://mail.python.org/mailman/listinfo/python-list
Re: What to use instead of nntplib?
> > My understanding is that nntplib isn't being erased from reality, > it's merely being removed from the set of modules that are provided > by default. > > I presume that once it's removed from the core, it will still be > possible to install it via pip or some other mechanism. > It won't magically be available via pip unless someone steps up to maintain it as a PyPI package Skip > -- https://mail.python.org/mailman/listinfo/python-list
Re: What to use instead of nntplib?
> I got a nice warning today from the inews utility I use daily: > > DeprecationWarning: 'nntplib' is deprecated and slated for removal in > Python 3.13 > > What should I use in place of nntplib? I'd recommend creating a PyPI project with the existing 3.12 code, then using that from 3.13 onward. Skip -- https://mail.python.org/mailman/listinfo/python-list
Re: replacement for nntplib on Python3
You could create a git repo with just nntplib and generate a package on PyPI, then use that when running a version of Python which lacks nntplib. Skip On Wed, Apr 26, 2023, 8:42 PM Retrograde wrote: > I used to use a script that relied on nntplib, which is currently still > available in Python3 but scheduled for deprecation in the near future. > What would be a suitable replacement? > > -- > Retrograde > -- > https://mail.python.org/mailman/listinfo/python-list > -- https://mail.python.org/mailman/listinfo/python-list
Small lament...
Just wanted to throw this out there... I lament the loss of waking up on April 1st to see a creative April Fool's Day joke on one or both of these lists, often from our FLUFL... Maybe such frivolity still happens, just not in the Python ecosystem? I know you can still import "this" or "antigravity", but those are now old (both introduced before 2010). When was the last time a clever easter egg was introduced or an April Fool's Day joke played? ¯\_(ツ)_/¯ Skip -- https://mail.python.org/mailman/listinfo/python-list
Re: JOB | Lead Linux Sysadmin (Edinburgh/London)
There's a link at the bottom of each message to the list info pager. Follow the directions on that page to unsubscribe. Skip On Wed, Mar 8, 2023, 5:38 PM Thomas Gregg wrote: > Is there any way to be removed from this list? > Thank you, Tom > > On Wed, Mar 8, 2023 at 3:51 PM Skip Montanaro > wrote: > > > > Hello, I'm working with an employer that is looking to hire someone in > > > (Edinburgh or London) that can administer on-prem and vmware > > > platforms. > > > > > > > James, > > > > If you haven't already, please post to the Phone Jobs Board: > > > > https://www.python.org/jobs/ > > > > Skip > > > > > > > -- > > https://mail.python.org/mailman/listinfo/python-list > > > -- > https://mail.python.org/mailman/listinfo/python-list > -- https://mail.python.org/mailman/listinfo/python-list
Re: JOB | Lead Linux Sysadmin (Edinburgh/London)
> Hello, I'm working with an employer that is looking to hire someone in > (Edinburgh or London) that can administer on-prem and vmware > platforms. > James, If you haven't already, please post to the Phone Jobs Board: https://www.python.org/jobs/ Skip > -- https://mail.python.org/mailman/listinfo/python-list
Re: Python 3.10 Fizzbuzz
Dang auto-correct... Should read ... double quotes around "strings" and single quotes around 'c'haracters ... On Sun, Feb 26, 2023, 6:28 PM Skip Montanaro wrote: > is there any reason to prefer"over' ? >> > > Not really. As an old C programmer for many years I used double > quotes"around "strings" and single word around 'c'haracters, because that's > what I was used to. (This was long before triple quoted strings appeared in > the language.) > > Aside: Given all the various ways to quote strings for display, it irks me > a bit to see repr() still use single quotes in all cases, which requires > display of single quotes to be escaped. (In similar fashion, it would be a > minor improvement in my mind if the repr() code used raw strings where they > would simplify the display.) > > Skip > >> -- https://mail.python.org/mailman/listinfo/python-list
Re: Python 3.10 Fizzbuzz
> > is there any reason to prefer"over' ? > Not really. As an old C programmer for many years I used double quotes"around "strings" and single word around 'c'haracters, because that's what I was used to. (This was long before triple quoted strings appeared in the language.) Aside: Given all the various ways to quote strings for display, it irks me a bit to see repr() still use single quotes in all cases, which requires display of single quotes to be escaped. (In similar fashion, it would be a minor improvement in my mind if the repr() code used raw strings where they would simplify the display.) Skip > -- https://mail.python.org/mailman/listinfo/python-list
Re: Is there a more efficient threading lock?
> And yet, it appears that *something* changed between Python 2 and Python 3 such that it *is* atomic: I haven't looked, but something to check in the source is opcode prediction. It's possible that after the BINARY_OP executes, opcode prediction jumps straight to the STORE_FAST opcode, avoiding the transfer to the top of the virtual machine loop. That would (I think) avoid checks related to GIL release and thread switches. I don't guarantee that's what's going on, and even if I'm correct, I don't think you can rely on it. Skip -- https://mail.python.org/mailman/listinfo/python-list
Re: Is there a more efficient threading lock?
Thanks for the responses. Peter wrote: > Which OS is this? MacOS Ventura 13.1, M1 MacBook Pro (eight cores). Thomas wrote: > I'm no expert on locks, but you don't usually want to keep a lock while > some long-running computation goes on. You want the computation to be > done by a separate thread, put its results somewhere, and then notify > the choreographing thread that the result is ready. In this case I'm extracting the noun phrases from the body of an email message (returned as a list). I have a collection of email messages organized by month (typically 1000 to 3000 messages per month). I'm using concurrent.futures.ThreadPoolExecutor() with the default number of workers ( os.cpu_count() * 1.5, or 12 threads on my system) to process each month, so 12 active threads at a time. Given that the process is pretty much CPU bound, maybe reducing the number of workers to the CPU count would make sense. Processing of each email message enters that with block once. That's about as minimal as I can make it. I thought for a bit about pushing the textblob stuff into a separate worker thread, but it wasn't obvious how to set up queues to handle the communication between the threads created by ThreadPoolExecutor() and the worker thread. Maybe I'll think about it harder. (I have a related problem with SQLite, since an open database can't be manipulated from multiple threads. That makes much of the program's end-of-run processing single-threaded.) > This link may be helpful - > > https://anandology.com/blog/using-iterators-and-generators/ I don't think that's where my problem is. The lock protects the generation of the noun phrases. My loop which does the yielding operates outside of that lock's control. The version of the code is my latest, in which I tossed out a bunch of phrase-processing code (effectively dead end ideas for processing the phrases). Replacing the for loop with a simple return seems not to have any effect. In any case, the caller which uses the phrases does a fair amount of extra work with the phrases, populating a SQLite database, so I don't think the amount of time it takes to process a single email message is dominated by the phrase generation. Here's timeit output for the noun_phrases code: % python -m timeit -s 'text = """`python -m timeit --help`""" ; from textblob import TextBlob ; from textblob.np_extractors import ConllExtractor ; ext = ConllExtractor() ; phrases = TextBlob(text, np_extractor=ext).noun_phrases' 'phrases = TextBlob(text, np_extractor=ext).noun_phrases' 5000 loops, best of 5: 98.7 usec per loop I process the output of timeit's help message which looks to be about the same length as a typical email message, certainly the same order of magnitude. Also, note that I call it once in the setup to eliminate the initial training of the ConllExtractor instance. I don't know if ~100us qualifies as long running or not. I'll keep messing with it. Skip -- https://mail.python.org/mailman/listinfo/python-list
Is there a more efficient threading lock?
I have a multi-threaded program which calls out to a non-thread-safe library (not mine) in a couple places. I guard against multiple threads executing code there using threading.Lock. The code is straightforward: from threading import Lock # Something in textblob and/or nltk doesn't play nice with no-gil, so just # serialize all blobby accesses. BLOB_LOCK = Lock() def get_terms(text): with BLOB_LOCK: phrases = TextBlob(text, np_extractor=EXTRACTOR).noun_phrases for phrase in phrases: yield phrase When I monitor the application using py-spy, that with statement is consuming huge amounts of CPU. Does threading.Lock.acquire() sleep anywhere? I didn't see anything obvious poking around in the C code which implements this stuff. I'm no expert though, so could easily have missed something. Thx, Skip -- https://mail.python.org/mailman/listinfo/python-list
Re: Upgrading Python on Ubuntu 22.04.1 LTS
> > > I have a server with Ubuntu 22.04.1 LTS and the Python version there > is Python 3.10.6. Is there a safe way to upgrade to the latest version of > Python 3.10 (3.10.9)? I tried with the OS update and upgrade but the Python > version remained the same. > > I have an Ubuntu 22.04 system. The installed Python 3.10 version is 3.10.7: $ apt search python3 | egrep '^python3.10.*installed' WARNING: apt does not have a stable CLI interface. Use with caution in scripts. python3.10/now 3.10.7-1+focal1 amd64 [installed,local] python3.10-distutils/now 3.10.7-1+focal1 all [installed,local] python3.10-doc/jammy-updates,jammy-security,now 3.10.6-1~22.04.2 all [installed] python3.10-gdbm/now 3.10.7-1+focal1 amd64 [installed,local] python3.10-lib2to3/now 3.10.7-1+focal1 all [installed,local] python3.10-minimal/now 3.10.7-1+focal1 amd64 [installed,local] python3.10-venv/now 3.10.7-1+focal1 amd64 [installed,local] Off the top of my head, I can't recall if it's LTS or not. If you want to go beyond 3.10.6, it should be possible. As Grant indicated though, upgrading packages on an Ubuntu system (of any flavor) is the province of the Ubuntu community. Skip -- https://mail.python.org/mailman/listinfo/python-list
Re: Android APK
> > Is there a good python library for converting python3 to android APK > I'm not an app type of person, so don't watch that space. I've heard of Kivy: https://kivy.org/ I also something called python-for-android on PyPI with a recent release: https://pypi.org/project/python-for-android/ I'd be interested to see what else turns up. Skip > -- https://mail.python.org/mailman/listinfo/python-list
Re: argparse — adding a --version flag in the face of positional args
Thanks. It occurs to me that instead of providing two special actions
("help" and "version"), it might be worthwhile to provide a standard way of
saying, "if present, process this option and exit before considering other
details of the command line." Matt's example action works well enough for
my needs, but it would be nice if more than one such option could be given.
For example:
import argparse
parser = argparse.ArgumentParser()
parser.add_argument("url")
parser.add_argument("--version", version="777", action="version")
args = parser.parse_args(["--help", "--version"])
Which option is processed depends on their order on the command line. I
don't believe it's possible to run the script and see them both processed.
That's probably a secondary consideration though. My script is working well
enough in this regard now.
Skip
--
https://mail.python.org/mailman/listinfo/python-list
Re: argparse — adding a --version flag in the face of positional args
> > ummm, hate to say this, but have you checked the documentation? this > case is supported using an action named 'version' without doing very much. > Thanks, Mats. I actually searched all over the argparse docs. (There's a lot to digest. Honestly, if I wasn't attempting to be sort of up-to-date I'd just continue using getopt.) It never occurred to me that "version" would be special, so I didn't specifically search for it, or realize there would be a specific action devoted to it. I knew (the default) "help" was special, so I focused my search for it. Obviously, "--help" is a pretty bad search term. Skip > -- https://mail.python.org/mailman/listinfo/python-list
Re: argparse — adding a --version flag in the face of positional args
> class VersionAction(argparse.Action):
> def __call__(self, parser, namespace, values, option_string):
> print(VERSION)
> exit()
...
> parser.add_argument("-v", "--version", nargs=0, action=VersionAction)
Thanks. An action class didn't occur to me. I looked briefly at the
code for argparse to see how it handled --help. The added argument
seemed normal, so gave up, figuring there was some special handling of
that option.
Skip
--
https://mail.python.org/mailman/listinfo/python-list
argparse — adding a --version flag in the face of positional args
I have a script to which I'd like to add a --version flag. It should print the version number then exit, much in the same way --help prints the help text then exits. I haven't been able to figure that out. I always get a complaint about the required positional argument. I think I could use something like nargs='*', but that would push off detection of the presence of the positional arg to the application. Shouldn't I be able to tell argparse I'm going to process --verbose, then exit? Thx, Skip -- https://mail.python.org/mailman/listinfo/python-list
Re: Ref-strings in logging messages (was: Performance issue with CPython 3.10 + Cython)
Dang autocorrect. Subject first word was supposed to be "f-strings" not "ref-strings." Sorry about that. S On Fri, Oct 7, 2022, 10:45 AM Skip Montanaro wrote: > > > On Fri, Oct 7, 2022 at 9:42 AM Andreas Ames > wrote: > >> 1. The culprit was me. As lazy as I am, I have used f-strings all over the >> place in calls to `logging.logger.debug()` and friends, evaluating all >> arguments regardless of whether the logger was enabled or not. >> > > I thought there was some discussion about whether and how to efficiently > admit f-strings to the logging package. I'm guessing that's not gone > anywhere (yet). > > Skip > -- https://mail.python.org/mailman/listinfo/python-list
Ref-strings in logging messages (was: Performance issue with CPython 3.10 + Cython)
On Fri, Oct 7, 2022 at 9:42 AM Andreas Ames wrote: > 1. The culprit was me. As lazy as I am, I have used f-strings all over the > place in calls to `logging.logger.debug()` and friends, evaluating all > arguments regardless of whether the logger was enabled or not. > I thought there was some discussion about whether and how to efficiently admit f-strings to the logging package. I'm guessing that's not gone anywhere (yet). Skip -- https://mail.python.org/mailman/listinfo/python-list
Python gets a mention on Reddit - in r/tifu
Not exactly where I expected to see Python mentioned on Reddit, but I found this amusing (yes, despite the subreddit, it should be safe for work): https://www.reddit.com/r/tifu/comments/wibmkh/tifu_by_teaching_my_13yo_brother_python/ Just a little diversion for a Sunday afternoon. I was particularly amused, as my son is in dev ops, writes Python from time-to-time, and my grandson is 13. I could definitely see the pattern in the story transferring over to Chris and Carmine. Skip -- https://mail.python.org/mailman/listinfo/python-list
Re: Dictionary order?
> > So I decided to write a little test program to run on a variety of > CPythons, to confirm what I was thinking. > > And instead I got a surprise. > > On 1.4 through 2.1 I got descending key order. I expected the keys to be > scattered, but they weren't. > > On 2.2 through 3.5 I got ascending key order. I expected the keys to be > scattered, but they weren't. > > On 3.6 through 3.10 I got insertion order, as expected. > > But why are 1.4 through 3.5 ordering so much? > That's long in the past, but I seem to recall that key order was unspecified. That would give the implementer (likely Tim Peters much of the time) the freedom to do whatever worked best for performance or simplicity of implementation. Skip > -- https://mail.python.org/mailman/listinfo/python-list
Re: Resolving Weekday Schedules to Dates
> Do you know of a library that resolves schedules like every Wednesday > at 3:00pm to absolute time, that is return the datetime of the next > occurrence? Take a look at the `rrule` module in the `dateutil` package: https://dateutil.readthedocs.io/en/stable/rrule.html Skip -- https://mail.python.org/mailman/listinfo/python-list
Now for something completely different...
This isn't really Python-related, but I don't wander around any more general software engineering/computer science places on the web, so I will throw this out here. I start a framebuilding class next week (lugged steel construction for those into bikes). After about three weeks of work, I should have a custom-made frame, which will also happen to have been built by me. It will need paint or powder coat and graphics. I'm looking for ideas about the graphics. Though I spent most of my professional life noodling around with Python (I'm now retired), I don't really want the graphics to be terribly Python-centric. Consequently, "import this" is probably not going to work. I have decided to go ahead and use my first name in lower case Courier as the core piece of the downtube graphic: skip That's not too informative (other than its relationship to moi), and I have room for probably four or five more characters. (I have a graphic artist in mind, so the space need not strictly be text either.) So, how would you fill the remaining space? I've considered "n++" or "n+1", but not thought of anything more than that. For those who are not into bikes, you may not have heard of the Velominati Rules <https://www.velominati.com/>. I quote Rule #12: *The correct number of bikes to own is n+1.* While the minimum number of bikes one should own is three, the correct number is n+1, where n is the number of bikes currently owned. This equation may also be re-written as s-1, where s is the number of bikes owned that would result in separation from your partner. I currently have six to ten, depending on how demanding you are that they be ready to ride with nothing more than air in the tires. This project, while definitely n+1 might well also be approaching s-1. :-) Ideas about head tube graphics are also welcome. I have no idea there at all yet. Again, some subtle reference to software engineering would be nice. So, if you'd like to play along, toss out some ideas. I'll consider everything as long as it's not NSFW. Skip -- https://mail.python.org/mailman/listinfo/python-list
Re: Do projects exist to audit PyPI-hosted packages?
> > A related problem is that even if a package is maintained by somebody with > good intentions, the account might be hijacked by a malicious actor and > since PyPi is separate from source control, people might not be able to > find out easily and malware could spread through PyPi. > I hadn't considered that. Some sort of authenticated connection between the source code hosting service and the PyPI user posting the package would be nice. Some other (only tangentially related) stuff occurs to me as I search for useful bits... I'd kinda be curious what hosting services other than GitHub or GitLab are in common use. GNU Savannah? SourceForge? PyPI relevance isn't a terrific indicator (I assume it uses Libraries.io's SourceRank to get a relevance score), but it's still some kind of indicator how useful a package is. Perhaps the PyPI BigQuery stuff has hosting info. I've not dug into it. (Thinking that obscure hosting service might be a small knock against a package, but that's just a thought. I realize not everyone is happy with corporate hosting services.) Having a decent idea what functional alternatives are out there to a particular package would be nice as well. Again, considering pynput, I hit Google up for "python packages similar to pynput" which led me here: https://www.libhunt.com/r/pynput I was unaware of its existence before. I have no idea how useful it might be for narrowly focused packages like pynput. Something with application to a much wider community, like numpy, returns a bunch more: https://www.libhunt.com/r/numpy Skip -- https://mail.python.org/mailman/listinfo/python-list
Do projects exist to audit PyPI-hosted packages?
I woke with a start in what amounted to the middle of the night (I really need to get about three more hours of sleep, but you'll understand why I was awake to write this). Many years ago, so as to preserve my wrists, I wrote a tool <https://github.com/smontanaro/python-bits/blob/main/src/watch.py> to monitor mouse and keyboard activity. It tells me when to rest. I use it when I have problems, then put it away until it's needed again. I have resurrected it a few times over the years, most recently a month or two ago. Having never been all that fond of how I tracked keyboard and mouse activity, I was happy when I stumbled upon pynput <https://pypi.org/project/pynput/>. "Yay!", I thought. My worries are over. Then extremely early this morning I woke thinking, "Damn, this runs on my computer and it can see my mouse and keyboard activity. How do I know it's not stealing my keystrokes?" Not going back to sleep after that. So, I'm going through the code (and the Xlib package on which it relies) to make myself more comfortable that there are no issues. Note: I am *most certainly not* accusing the pynput author of any mischief. In fact, I suspect there's no problem with the package. It's got a bunch of stars and plenty of forks on GitHub (for what that's worth). I suspect the code has had plenty of eyeballs looking at it. Still, I don't really know how well vetted it might be, so I have no assurances of that. I saw it mentioned somewhere (discuss I think?), checked it out, and thought it would solve my activity tracking in a cross-platform way. (I currently only use an Xorg environment, so while I am looking at the code, I'm not paying attention to the Windows or MacOS bits either.) This got me thinking. If I'm curious about pynput, might other people be as well? What about other packages? I'm actually not worried about Python proper or vulnerabilities which have already been found <https://github.com/pypa/advisory-database>. PyPI currently advertises that it hosts over 373k packages. With that many hosted packages, it is almost certainly a haven for some undetected vulnerabilities. Knowing which packages have been audited — at least in a cursory fashion — could be used as a further criterion to use when deciding which packages to consider using on a project. So, does something already exist (pointers appreciated)? Thx... Skip -- https://mail.python.org/mailman/listinfo/python-list
Re: Receive a signal when waking or suspending?
> https://duckduckgo.com/?t=ffab&q=python+up+time pointed-out the > "uptime — Cross-platform uptime library" > - TLDR; I'm not sure if "uptime" and "boot" relate only to a > 'cold-start' or if they include bringing the machine out of 'stand-by' > or 'hibernation' Yeah, that won't help. uptime(1) gives the time since the last boot. Wake/suspend doesn't change the system's notion of boot time. > https://duckduckgo.com/?t=ffab&q=python+battery+status pointed-out > several ideas which require MS-Windows, but apparently the psutil > library could be bent to your will: > https://www.geeksforgeeks.org/python-script-to-shows-laptop-battery-percentage/ I actually already have psutil installed. It gives you details about various system bits (like battery status, which is why I installed it). It doesn't seem to have anything that will notify about state changes, just the current state (or — for example — continuous values like the battery state of charge). I didn't see anything regarding wake/suspend, but I might have missed it. Since my tool can't run while the system is suspended, it would always see the system as "awake" but not know for how long it was in that state. (I'm sure I could track in my tick routine and pay attention to unusually long gaps.) I think Marco's response gives me a bit better handle on what I need(*). Maybe just an easier way to do things. For example, just before suspending and just after waking, all scripts in /usr/lib/systemd/system-sleep/ are executed. The only difference is the arguments with which they are called. A script to suit my needs can just write a stream of records with timestamps. It took just a minute or so: #!/bin/sh echo "`date --iso-8601=seconds` $1 $2" >> /tmp/suspensions chmod 0644 /tmp/suspensions Here's what /tmp/suspensions looks like after a couple cycles: root@montanaro:/usr/lib/systemd/system-sleep# cat /tmp/suspensions 2022-04-23T19:17:41-05:00 pre suspend 2022-04-23T19:17:54-05:00 post suspend 2022-04-23T19:18:48-05:00 pre suspend 2022-04-23T19:19:04-05:00 post suspend My tool can monitor that file for changes and keep track of the times of state changes. It can thus track rest intervals across system suspensions and know if I've been resting for a long enough period of time to allow me to subject my wrists to further agony <https://github.com/smontanaro/python-bits/blob/master/src/watch.py> with the keyboard and mouse. Skip (*) man systemd-sleep contains this admonition: Note that scripts or binaries dropped in /lib/systemd/system-sleep/ are intended for local use only and *should be considered hacks*. If applications want to react to system suspend/hibernation and resume, they should rather use the Inhibitor interface[1]. A hack should suit my needs just fine. I'll leave more elegant solutions to others. :-) -- https://mail.python.org/mailman/listinfo/python-list
Re: Receive a signal when waking or suspending?
> I don't know in Python, but maybe you can create a script that writes > on a named pipe and read it from Python? > https://askubuntu.com/questions/226278/run-script-on-wakeup Thanks, that gives me something to munch on. -- https://mail.python.org/mailman/listinfo/python-list
Receive a signal when waking or suspending?
It's not clear there is a straightforward way to catch a signal or get an event notification when my computer (Dell running XUbuntu 20.04) is about to sleep or when it's just awakened. The app uses tkinter. Is there some more-or-less easy way to do this? Mac support would be nice (I have my eye on a new MacBook one of these days). I suppose bonus points for something which works on Windows, but that's not a platform I actually care about. Thx, Skip -- https://mail.python.org/mailman/listinfo/python-list
Re: asyncio+tkinter
> > > Given that both asyncio & tkinter are modules in the standard lib and > both > > have event loops, I would have expected to find some "best practice" > > solution to mixing the two. > > Agreed. For GTK, you can use a dedicated loop policy like this: > > import asyncio_glib > asyncio.set_event_loop_policy(asyncio_glib.GLibEventLoopPolicy()) > > It should be possible to make it this easy for asyncio + tkinter. > I haven't used GTK in ages and ages, but I do still have a tkinter program that I drag out every now and then when my wrists start acting up. After opining about this on the PyIdeas discuss forum, I spent a few minutes and came up with a simple AsyncTk subclass of tkinter.Tk. Here's its hello world: https://gist.github.com/smontanaro/5e12c557602a76c609e46ca59387ad1c I modified my activity watcher to use it: https://github.com/smontanaro/python-bits/blob/master/src/watch.py A few warnings: 1. copy/paste reuse of that AsyncTk class 2. the code is a mess (it's probably 20-25 years old, I make no apologies for the mess) 3. as of about an hour ago it now uses pynput to watch for mouse/kbd watching So, Tk+asyncio turns out to be fairly easy to do, at least for simple stuff. Skip -- https://mail.python.org/mailman/listinfo/python-list
Re: Set tkinter top-level window to "always on visible workspace"
> Also, wmctrl accepts window names instead of ids if you omit the -i > option. In case winfo_id() is not returning the id you need. Nice. That worked. FWIW, Tk never gave me the very outermost window as the parent of the tkinter.Tk instance. Here's the start of hierarchy: % xwininfo -tree xwininfo: Please select the window about which you would like information by clicking the mouse in that window. xwininfo: Window id: 0x5a0001f "Typing Watcher" Root window id: 0x7a2 (the root window) (has no name) Parent window id: 0xc73880 (has no name) 1 child: 0x5a0001e (has no name): () 178x126+0+0 +2+43 1 child: 0x5a00020 (has no name): () 178x126+0+0 +2+43 5 children: ... Tk thought 0x5a0001e was the top, but as you can see 0x5a0001f is the named window. Calling wmctrl using "Typing Watcher" as the target window did the trick. Thanks, Skip -- https://mail.python.org/mailman/listinfo/python-list
Re: Set tkinter top-level window to "always on visible workspace"
> Would you accept a solution that involves a subprocess call?
>
> wmctrl -ir {id} -b add,sticky
I'm already checking idle time with xprintidle(1), so what's one more
(one-time) subprocess call?
Small amount of history, I'm referring to this mouse/typing watcher:
https://github.com/smontanaro/python-bits/blob/master/src/watch.py
I wrote this years and years ago when I first started having RSI problems.
So long ago that I just deleted a commented out piece of code which was
there in case I ever ran it with Python 1.4(!).
Over the years I've needed it on Linux, Linux+WIndows, Mac, etc. I'd put it
away for a couple years, then suffer a flare-up and pull it back out. I
never thought I'd need it after I retired, but here I am again. I've tried
all sorts of things to monitor mouse and keyboard activity. At the moment
I'm just using a Linux laptop, so have tossed out any sort of attempted
cross-platform tracking functionality.
> Now, the only problem is... figuring out your window ID. Worst case,
> parse wmctrl -lG to get that info, but it might be possible to get the
> window ID from Tkinter itself.
I'm struggling to get the outermost window (not sure what's going on), but
I will keep messing around.
Skip
--
https://mail.python.org/mailman/listinfo/python-list
Re: Set tkinter top-level window to "always on visible workspace"
> I think you must be using xfce4, not fvwm4 (there's an fvwm3 in > development). See https://xfce.org/ Both? % pgrep -fla xfwm4 1803 xfwm4 --replace 539426 xfwm4-settings % pgrep -fla xfce 1599 xfce4-session 1755 /usr/bin/ssh-agent /usr/bin/im-launch startxfce4 1782 /usr/lib/x86_64-linux-gnu/xfce4/xfconf/xfconfd ... I kind of assume xfce4 is the session manager sort of thing, while xfwm4 is the actual window manager. Skip -- https://mail.python.org/mailman/listinfo/python-list
Re: Set tkinter top-level window to "always on visible workspace"
> I might be misguided, but on modern desktops that should be possible > with a few mouseclicks. E.g. in KDE, there is a little pin icon > displayed in every title bar of a toplevel window on the left side. Correct, and that's what I'm currently doing. I'm lazy though. I want the program to start always visible on the active virtual desktop. As Cameron showed, this is possible. I have vague recollection of doing this with the same program probably 15-20 years ago in an age when window managers weren't largely configured with the mouse. Unfortunately, I no longer recall what window manager(s) I used at the time (probably twm or fvwm). Now I use fvwm4 and can't find squat online about configuration files. I do have a ~/.config/xfce4/xfwm4/ directory, but it is completely empty. Skip -- https://mail.python.org/mailman/listinfo/python-list
Re: Set tkinter top-level window to "always on visible workspace"
> So you might tell your window manager to keep that window on the main workspace. Thanks. I'd forgotten about the possibility of doing this sort of thing in the window manager config. That would certainly be fine in this case. (It's been ages since I messed with this sort of thing.) Skip -- https://mail.python.org/mailman/listinfo/python-list
Set tkinter top-level window to "always on visible workspace"
I have a tkinter app (Ubuntu/X11 env) whose main window should always be displayed on the currently visible workspace. Is there some way to set that attribute programmatically? I see that there is a tkinter.Wm class and that Toplevel widgets have a wm_attributes method, but haven't found any examples in the library doc or on the wider net which demonstrate control of this particular window manager interaction. (I don't care about Windows or Mac, at least for the time being.) Thx, Skip -- https://mail.python.org/mailman/listinfo/python-list
Re: How to detect an undefined method?
> Let's say I have a Python app and have used an undefined method somewhere. > Let > us further assume I have not detected it thru my tests. > > Is there a way to detect it before deploying the app? pylint doesn't > notice it. > This is maybe not exactly what you're looking for, but writing a test to exercise that function call (if possible) would be top of my list. Next on my list would be to use coverage to get a good idea of what code is not tested. Writing more test cases to reduce the number of uncovered lines will also make it easier to manually examine the rest of your (untested) code to find calls to missing functions or methods. Skip > -- https://mail.python.org/mailman/listinfo/python-list
asyncio+tkinter
Given that both asyncio & tkinter are modules in the standard lib and both have event loops, I would have expected to find some "best practice" solution to mixing the two. I've not used asyncio, but might find it useful with the pynput module in the context of a Tk app. I see a few solutions out in the wild, but this seems like something which might best be addressed in either the asyncio or tkinter documentation, or better yet, implemented in one or the other. Skip Montanaro -- https://mail.python.org/mailman/listinfo/python-list
Re: Gunicorn - HTTP and HTTPS in the same instance?
Thanks all. I was hoping to get away without something more sophisticated like NGINX. This is just a piddly little archive of an old mailing list running on a single-core Ubuntu VM somewhere on the East Coast. Speed is not a real requirement. Load balancing seemed like overkill to me. Still, I guess if it has to be, then it has to be. Skip -- https://mail.python.org/mailman/listinfo/python-list
Gunicorn - HTTP and HTTPS in the same instance?
Hopefully some Pythonistas are also Gunicornistas. I've had little success finding help with a small dilemma in the docs or in other more specific sources. I'm testing out a new, small website. It is just Gunicorn+Flask. I'd like to both listen for HTTP and HTTPS connections. Accordingly, in my config, I have the Gunicorn process bind to both ports 80 and 443 if running as root: if IAM_ROOT: bind = [ '0.0.0.0:443', '0.0.0.0:80', ] else: bind = [ '0.0.0.0:8080', ] Gunicorn listens on both ports, but insists on SSL/TLS chit chat over port 80, not just port 443 (which seems to work okay). Is there some magic incantation to get it to just talk HTTP on port 80, or will I need to spin up two instances? (The non-root config works fine - plain old HTTP over port 8080.) Thx, Skip -- https://mail.python.org/mailman/listinfo/python-list
Re: email.message_from_file & quoted printable
> That sometimes you want the raw content as it was in the file? That if > you have that choice (decoded or raw) the default should be False, as it > does less? Yeah, thinking back later, the round-trip possibility seemed plausible. I'll stop overthinking this now... > That we my _first_ instinct - I've never used get_payload(), myself, and > I've written my own complete mailfiler :-) I'm only trying to display email through a web browser, possibly with a link to the raw message. > Really, if a function isn't doing what you want you should always read > the docs for that function specificly, even if you don't want to read > the entire module docs (which is a big ask). Yeah, my mental model was simply off. I frequently pull up docs in my browser. I was thinking the message contents would be normalized when reading. It didn't occur to me the transformation would occur on the way out. I think another way I might have saved myself was if I was using a modern IDE where I might have gotten some hints hovering over the method call. But, I'm an End user. I'm sure there is some add-on package I could install, but I haven't looked. Skip -- https://mail.python.org/mailman/listinfo/python-list
Re: email.message_from_file & quoted printable
Cameron> Try decode=True. Skip> :dopeslap: Thanks. Never been all that consistent reading documentation. The more I think about it, the more I think maybe my lack of documentation reading wasn't all that unreasonable. The content transfer encoding and charset are properties of the message as a whole. Iexpected the internal form of the message body to be already decoded. Am I missing some key bit of perspective here? Skip -- https://mail.python.org/mailman/listinfo/python-list
Re: email.message_from_file & quoted printable
> > From the docs: > > get_payload(i=None, decode=False) ... Try decode=True. :dopeslap: Thanks. Never been all that consistent reading documentation. Skip -- https://mail.python.org/mailman/listinfo/python-list
email.message_from_file & quoted printable
I have a bunch of old email archives I'm messing around with, for example,
rendering them on-demand in HTML. Some of the files use quoted printable
content transfer encoding. Here's one (with a number of headers elided):
>From classicrendezvous-admin Mon Dec 4 15:29:22 2000
Message-ID: <027801bfcdaa$fa175230$268cbcd4@stewart>
Reply-To: "Stewart Barrie"
From: "Stewart Barrie"
To:
Date: Sat, 3 Jun 2000 23:28:00 +0100
MIME-Version: 1.0
X-StripMime: Non-text section removed by stripmime
Content-Transfer-Encoding: quoted-printable
Content-Type: text/plain;charset="iso-8859-1"
Subject: {ClassicRend]Frame dent
Just inherited a nice 531 Claud Butler from the 60's I think. No =
eylelets and pencil stays. Unfortunately there is nice dent in the =
middle of the seat tube just above front mech clip height. Otherwise it = seems
fine.=20
Any thoughts on restoration? Are there any techniques for fixing dents =
in situ (I can hear the screams already) or are we definitely looking at =
a new seat tube? In which case, is it worth it?
Cheers
Stewart
--- StripMime Report -- processed MIME parts ---
multipart/alternative
text/plain (text body -- kept)
text/html
---
This message is stored in a file. I read it using email.message_from_file.
In the example above, the payload still retains the quoted printable bits.
Maybe it's not correctly encoded (no "=20" at the end of the second
paragraph, for example), but I would have thought the email package would
do what it could to decode things, but nope:
>>> print(msg.get_payload())
Just inherited a nice 531 Claud Butler from the 60's I think. No =
eylelets and pencil stays. Unfortunately there is nice dent in the =
middle of the seat tube just above front mech clip height. Otherwise it =
seems fine.=20
Any thoughts on restoration? Are there any techniques for fixing dents =
in situ (I can hear the screams already) or are we definitely looking at =
a new seat tube? In which case, is it worth it?
Cheers
Stewart
--- StripMime Report -- processed MIME parts ---
multipart/alternative
text/plain (text body -- kept)
text/html
---
Am I expecting too much from the email package when munching on crufty
20+yo archived email messages?
Thx,
Skip
--
https://mail.python.org/mailman/listinfo/python-list
Re: Sad news: Fredrik Lundh ("Effbot") has passed away
Like many others, I'm saddened to hear of Fredrik Lundh's passing. I vaguely recall meeting him just once, probably at a Python workshop, before they grew big enough to be called conferences. Effbot.org was my Tkinter, ElemenTree, and PIL reference and cheat sheet. My attention to Python development has waxed and waned over the years. Most of the time, the trip through the Python folder in my mail program was generally pretty quick, hitting the 'd' key far more often than I'd stop to read a message. There are only a few people whose messages I'd always read. Effbot was one. In my opinion, Fredrik ranks up there with Guido, Tim Peters and Barry Warsaw. I went to effbot.org and saw the "on hiatus" message. Searching through The Wayback Machine, it seems it went on hiatus in late November, 2020. The 11 November 2020 snapshot appears to be the last usable version: https://web.archive.org/web/2020145627/http://effbot.org/ Probably worth a bookmark in your browser. Rest easy /F ... Skip -- https://mail.python.org/mailman/listinfo/python-list
Proliferation of Python packaging formats
Is the proliferation of packaging formats in Python as nutzo as this author believes? https://drewdevault.com/2021/11/16/Python-stop-screwing-distros-over.html Asking because I've never been in the business of releasing "retail" Python applications or packages. Skip -- https://mail.python.org/mailman/listinfo/python-list
Re: Package conflicts trying to install jpegdupes package
Woo hoo! It's installed. The ultimate error was a missing turbojpeg.h file. Thank goodness for the apt-file command. I was able to track that down to the libturbojpeg0-dev package, install that, and after a bit more fussing around now have jpegdupes installed. Thanks for the help, Skip -- https://mail.python.org/mailman/listinfo/python-list
Re: Package conflicts trying to install jpegdupes package
> > For the most part, a proper colorizing compiler can make the > errors stand out among the spam of warnings, but it's hard when people > copy and paste just the text. > In my defense, the entire traceback was red. :-) It's quite possible that GCC colorized its error/warning messages, but colorizing might have been suppressed by stderr being fed into a pipe, or by distutils tossing it out. Skip -- https://mail.python.org/mailman/listinfo/python-list
Re: Package conflicts trying to install jpegdupes package
> > Those are all warnings. Are there any errors that follow them? > Maybe I just missed the actual errors, but the compiler exit status was 1, so there must have been. I'll give it another try. The package doesn't seem to have had a release since 2018, but the > GitHub repository has had changes as recently as a month ago. You > might have better luck cloning the repo and building that. > Thanks, good suggestion. FWIW, I'm trying to whittle down at least 12,000 images to a more manageable number for the sister of a good friend who recently passed away. I've got a straightforward dedupe program, but need something which can compare just the data chunk of JPEGs, ignoring the metadata. This program apparently does that. Is like to avoid reinventing that wheel. Skip -- https://mail.python.org/mailman/listinfo/python-list
Package conflicts trying to install jpegdupes package
I'd like to use the jpegdupes package <https://pypi.org/project/jpegdupes/> but can't seem to get past compilation issued related to jpegtrans-cffi. Attempts to install any available versions on my XUbuntu system crap out with this C compiler error message: In file included from jpegtran/__pycache__/_cffi__xd2d84bdexcdb1023.c:267: src/epeg_private.h:71:17: warning: ‘_jpeg_init_source’ declared ‘static’ but never defined [-Wunused-function] 71 | METHODDEF(void) _jpeg_init_source(j_decompress_ptr cinfo); | ^ src/epeg_private.h:72:20: warning: ‘_jpeg_fill_input_buffer’ declared ‘static’ but never defined [-Wunused-function] 72 | METHODDEF(boolean) _jpeg_fill_input_buffer(j_decompress_ptr cinfo); |^~~ src/epeg_private.h:73:17: warning: ‘_jpeg_skip_input_data’ declared ‘static’ but never defined [-Wunused-function] 73 | METHODDEF(void) _jpeg_skip_input_data(j_decompress_ptr cinfo, long num_bytes); | ^ src/epeg_private.h:74:17: warning: ‘_jpeg_term_source’ declared ‘static’ but never defined [-Wunused-function] 74 | METHODDEF(void) _jpeg_term_source(j_decompress_ptr cinfo); | ^ src/epeg_private.h:76:17: warning: ‘_jpeg_init_destination’ declared ‘static’ but never defined [-Wunused-function] 76 | METHODDEF(void) _jpeg_init_destination(j_compress_ptr cinfo); | ^~ src/epeg_private.h:77:20: warning: ‘_jpeg_empty_output_buffer’ declared ‘static’ but never defined [-Wunused-function] 77 | METHODDEF(boolean) _jpeg_empty_output_buffer (j_compress_ptr cinfo); |^ src/epeg_private.h:78:17: warning: ‘_jpeg_term_destination’ declared ‘static’ but never defined [-Wunused-function] 78 | METHODDEF(void) _jpeg_term_destination (j_compress_ptr cinfo); | ^~ ... This seems like a rather odd error message for a presumably sort-of-stable PyPI package. Looking at the version dependencies I see 2.6, 2.7, 3.3 or PyPy. I thought nmybe I needed to try with Python2... Create a Conda environment, attempt to install jpegdupes, only to discover it's Python 3 only. So, I'm kind of stuck. Maybe I need to install Python 3.3 and try that? Any other ideas? Thx, Skip -- https://mail.python.org/mailman/listinfo/python-list
Track 3.9 instead of 3.10?
I have a development branch in my fork of python/cpython, the register2 branch of https://github.com/smontanaro/cpython. As I am dealing with virtual machine internals I've found the changes to the virtual machine between 3.9 and 3.10 too disruptive. I'd like to track 3.9 instead. How would I go about making the switch while minimizing the number of (inevitable) conflicts? I was thinking of diffing my current state against 3.10, then creating a new branch off 3.9 and applying the diff to that. That makes most of the effort outside of the view of git though, and won't necessarily minimize conflicts. Is there some way to do this totally within the git infrastructure? Thx, Skip -- https://mail.python.org/mailman/listinfo/python-list
Re: imaplib: is this really so unwieldy?
> It's working (with Cyrus), but I have the feeling I'm doing it all wrong > because it seems so unwieldy. I have a program, Polly <https://github.com/smontanaro/polly>, which I wrote to generate XKCD 936 passphrases. (I got the idea - and the name - from Chris Angelico. See the README.) It builds its dictionary from emails in my Gmail account which are tagged "polly" by a Gmail filter. I had put it away for a few years, at which time it was still using Python 2. When I came back to it, I wanted to update it to Python 3. As with so many 2-to-3 ports, the whole bytes/str problem was my stumbling block. Imaplib's API (as you've discovered) is not the most Pythonic. I didn't spend much time horsing around with it. Instead, I searched for higher-level packages, eventually landing on IMAPClient <https://pypi.org/project/IMAPClient/>. Once I made the switch, things came together pretty quickly, due in large part, I think, to its more sane API. YMMV, but you're more than welcome to steal code from Polly. Skip -- https://mail.python.org/mailman/listinfo/python-list
Re: Bloody rubbish
> > Machine language is so much simpler, and you can code with just a hexpad. > Pshaa... All you need are front panel switches. ;-) (Yes, I had a professor who required is to 'key' in our programs on the front panel, of a rack mounted PDP-11 as I recall. Needless to say, we didn't use an assembler either. We just wrote raw opcodes and their arguments on paper. This was in the late 70s.) Skip > -- https://mail.python.org/mailman/listinfo/python-list
Re: Proposal: Disconnect comp.lang.python from python-list
> > Not that I support shutting down the Usenet/email gateway -- the > > signal/noise ration seems fine to me. > > > On that I quite agree. :-) I remind people that the existing Usenet<->Mail gateway keeps the mailing list software on mail.python.org from migrating to Mailman 3. I'm sure Mark Sapiro and other Mailman maintainers would like to keep moving away from Mailman 2. If the gateway (easier anonymity) and Mailman 3 (maybe better list archives) are of interest to you, you might check in with the Mailman dev list and see what would be involved in porting the gateway code to MM3. Skip -- https://mail.python.org/mailman/listinfo/python-list
Re: Proposal: Disconnect comp.lang.python from python-list
> Are you saying that the messages that appear occasionally from people > such as Ethan Furman claiming to be moderators and alleging that > particular people have been banned or suspended are lies? And that the > message I received once saying that my comp.lang.python post had been > rejected from the list was also a lie? I'm sorry, but I don't recall accusing you of lying, neither in my words nor in my thoughts. If you think I was doing that, you must be reading everything with a very jaundiced eye. My issue was that we seem to disagree with the definition of "moderation." Earlier in this thread it was suggested that comp.lang.python.moderated be created and that some as-yet-unnamed moderators would vet every single message to that group. I think of moderation as things which humans do, not tool chains. I suppose the python.org postmasters could disable all elements of that tool chain and let the admins/moderators do it all manually. Still, I don't think of the mailing list as moderated. Yes, Ethan is one of three listed admins of the list. I wouldn't have mentioned him by name, but you did. (I suppose his role common knowledge anyway.) I see three held messages in the queue right now with received times between 00:33 to 08:15 today (timezone isn't mentioned). I seriously doubt Ethan or the other admins approved all the other messages which have reached the list in that interval. Despite the existing tool chain, a number of messages which shouldn't probably leak through without being vetted by Ethan or the other admins. I will offer one other bit of information. As I understand it, the only reason python-list@python.org can't be migrated to Mailman 3 is because it doesn't support the gateway between Usenet and mail. It's worth considering for people thinking about whether or not to disconnect the two. (I have no opinion on that subject. Clearly others do.) Skip -- https://mail.python.org/mailman/listinfo/python-list
Re: Proposal: Disconnect comp.lang.python from python-list
> Are you unaware that the group is already gatewayed to a moderated > mailing list for which all of that work is already done? What is this moderation of which you speak? I'm one of the Python postmasters (I maintain the SpamBayes setup) and am aware of a multi-part tool chain (black hole lists, SpamBayes, greylisting, etc), but not of human moderation on a grand scale, just of the relatively few posts which are held for the admins/moderators by SpamBayes. My definition of "moderation" is that a human vets every post. That's certainly not the case for python-list@python.org. Posts gated from comp.lang.python to the mailing list only get passed through SpamBayes. All other elements of the tool chain occur ahead of the gateway. If we are using two different definitions of "moderation" I think it is important to be clear what we mean. Skip -- https://mail.python.org/mailman/listinfo/python-list
Re: About \033[m
>Porque quando se usa formatac,ao de cores, o python nao consegue
>
centralizar dentro da cadeia de 40 caracteres ?
>
>Tive que colocar 54 no parametro pois de alguma forma esta sendo
>considerado os caracteres de dentro do comando \033[m
Python doesn't know there is anything special about escape sequences (that
they take up no space on the screen). I think you will have better results
if you place the escape sequences in the format specifier:
print("\033[7;30;43m{:^40}\033[m".format("Programac,ao Python"))
( haven't tried this, but I think it will work as you expect.)
Skip
--
https://mail.python.org/mailman/listinfo/python-list
Re: Code Formatter Questions
> 1) Are there any major other formatters that I can/should look at? I see some > "online" pretty printers, but I'm after something I can run on whole > recursive directories of code. I use Emacs, so I'm kind of biased, but I find its python-mode does a good job, good enough that I wouldn't use a batch formatter to conform to some other conventions, then wind up having a mixed set of conventions after my next edit. I presume vim and all IDEs worth their salt also do a suitable job of formatting code on-the-fly. Skip -- https://mail.python.org/mailman/listinfo/python-list
Re: [Python-ideas] Looking for people interested in a Python register virtual machine project
t the end of each frame, so (in theory, I suppose), > > you could have instructions from both executed in the same frame. > > I hope you have a plan of how to deal with more than 256 registers, > etc. Register VM adds a lot of accidental implementation complexity ;-). One of the reasons I just reused the current stack space as my register space harkens back to a thread with Tim Peters back in the earliest days of my original work. He indicated that it was possible to use no more space than the allocated stack space. That worked for me then. (I spent a few hours one day looking for that thread, but never found it.) > > > > ## Motivation > > > > > > I'm not sure the content of the section corresponds much to its > > > title. It jumps from background survey of the different Python VM > > > optimizations to (some) implementation details of register VM - > > > leaving "motivation" somewhere "between the lines". > > > > > > > Despite all that effort, opcodes which do nothing more than move > > > > data onto or off of the stack (LOAD_FAST, LOAD_GLOBAL, etc) still > > > > account for nearly half of all opcodes executed. > > > > > > ... And - you intend to change that with a register VM? In which > > > way and how? As an example, LOAD_GLOBAL isn't going anywhere - it > > > loads a variable by *symbolic* name into a register. > > > > Certainly, if you have data which isn't already on the stack, you are > > going to have to move data. > > Even if you have data in registers, you still may need to move it > around to accommodate special conventions of some instructions. Yes. In particular, function calling and construction of lists (and similar collections) requires arguments to be in order in contiguous locations. > > As the appendix shows though, a fairly > > large chunk of the current virtual machine does nothing more than > > manipulate the stack (LOAD_FAST, STORE_FAST, POP_TOP, etc). > > > > > > Running Pyperformance using a development version of Python 3.9 > > > > showed that the five most frequently executed pure stack opcodes > > > > (LOAD_FAST, STORE_FAST, POP_TOP, DUP_TOP and ROT_TWO) accounted > > > > for 35% of all executed instructions. > > > > > > And you intend to change that with a register VM? How? > > > > I modified the frame so that (once again) the current local variable > > space adjoins the current stack space (which, again, I treat as > > registers). The virtual machine can thus access local variables in > > place. > > For simple instructions, yes. What about instructions with multiple > (arbitrarily number) arguments, like CALL_FUNCTION*/CALL_METHOD*, > BUILD_*, etc. (That's the same question as asked in the first mail and > not answered here.) CALL_FUNCTION and one or two variants as well as BUILD_* are done and work. I have to be careful when removing unneeded LOAD/STORE instructions that I pay attention to these implicit references (e.g., "%r1 and the next three slots" instead of "%r1, %r2, %r3, %r4"). > > In retrospect, I suspect it might not have been necessary. > > During the current phase where I've yet to implement any *_DEREF_REG > > instructions, it would be a moot point though. Still, I'm not sure the > > cell/free slots have the same semantics as locals/stack (an area of my > > expertise which is lacking). Isn't there an extra level of indirection > > there? In any case, if the cell/free slots are semantically just like > > locals, then it would be straightforward for me to restore the order > > of the data blocks in frames. > > Yes, that's the point - semantically they're just locals, even though > they are usually accessed with an extra level of indirection. > > > > Quick google search leads to > > > https://www.strchr.com/x86_machine_code_statistics (yeah, that's not > > > VM, it's RM (real machine), stats over different VMs would be > > > definitely welcome): > > > > > > > The most popular instruction is MOV (35% of all instructions). > > > > > > So, is the plan to replace 35% of "five most frequently executed > > > pure stack opcodes" with 35% of register-register move > > > instructions? If not, why it would be different and how would you > > > achieve that? > > > > I have clearly not explained myself very well in the "PEP". > > Well, it seems to be written with an idea that a reader is already > familiar with the benefits of register-based VMs. As a fresh reader, I
Re: [Python-ideas] Looking for people interested in a Python register virtual machine project
t; variable by *symbolic* name into a register.
Certainly, if you have data which isn't already on the stack, you are
going to have to move data. As the appendix shows though, a fairly
large chunk of the current virtual machine does nothing more than
manipulate the stack (LOAD_FAST, STORE_FAST, POP_TOP, etc).
> > Running Pyperformance using a development version of Python 3.9
> > showed that the five most frequently executed pure stack opcodes
> > (LOAD_FAST, STORE_FAST, POP_TOP, DUP_TOP and ROT_TWO) accounted for
> > 35% of all executed instructions.
>
> And you intend to change that with a register VM? How?
I modified the frame so that (once again) the current local variable
space adjoins the current stack space (which, again, I treat as
registers). The virtual machine can thus access local variables in
place. In retrospect, I suspect it might not have been necessary.
During the current phase where I've yet to implement any *_DEREF_REG
instructions, it would be a moot point though. Still, I'm not sure the
cell/free slots have the same semantics as locals/stack (an area of my
expertise which is lacking). Isn't there an extra level of indirection
there? In any case, if the cell/free slots are semantically just like
locals, then it would be straightforward for me to restore the order
of the data blocks in frames.
> Quick google search leads to
> https://www.strchr.com/x86_machine_code_statistics (yeah, that's not
> VM, it's RM (real machine), stats over different VMs would be
> definitely welcome):
>
> > The most popular instruction is MOV (35% of all instructions).
>
> So, is the plan to replace 35% of "five most frequently executed pure
> stack opcodes" with 35% of register-register move instructions? If not,
> why it would be different and how would you achieve that?
I have clearly not explained myself very well in the "PEP". I will
rework that section. Still though, local variables and stack space are
adjacent in my implementation, so local variables can be addressed
directly without needing to first copy them onto the stack (or into a
register). Clearly, global variables must still be copied into and out
of registers to manipulate. I may well replicate the code object's
constants in the frame as well so they can also be treated as
(read-only) registers. Since FrameObjects are cached for reuse with
the same code object, the cost to copy them should be bearable.
> > They are low-cost instructions (compared with CALL_FUNCTION for
> > example), but still eat up time and space in the virtual machine
>
> But that's the problem of any VM - it's slow by definition. There can
> be less slow and more slow VMs, but VMs can't be fast. So, what's the
> top-level motivation - is it "making CPython fast" or "making CPython a
> little bit less slow"? By how much?
The top-level motivation is to have fun. Need there be anything more
ambitious? Still, point taken. Had I continued to pursue this back in
the early 2000s, or had Victor succeeded in getting his implementation
into the core, we might be much further along. The current problem is
made all the more difficult by the fact that the virtual machine has
grown so much in the past 20+ years.
> > Consider the layout of the data section of a Frame object:
> > All those LOAD_FAST and STORE_FAST instructions just copy pointers
> > between chunks of RAM which are just a few bytes away from each other
> > in memory.
>
> Ok, but LOAD_DEREF and STORE_DEREF instructions also just copy pointers
> (with extra dereferencing, but that's a detail). It's unclear why you
> try to ignore them ("cell" registers), putting ahead "locals" and
> "stack" registers. The actual register instructions implementation would
> just treat any frame slot as a register with continuous numbering,
> allowing to access all of locals, cells, and stack locs in the same
> way. In that regard, trying to rearrange 3 groups at this stage seems
> like rather unneeded implementation complexity with no clear motivation.
I haven't even looked at LOAD_DEREF or STORE_DEREF yet. I think that
extra dereferencing will be more than a simple detail though. That
makes the semantics of cell/free slots different than locals/registers
slots (more like globals). If true, then my reordering of the frame
data is worthwhile, I think.
Skip
--
https://mail.python.org/mailman/listinfo/python-list
Re: [Python-ideas] Looking for people interested in a Python register virtual machine project
> Yeah, that is old writing, so is probably less clear (no pun intended) > than it should be. In frame_dealloc, Py_CLEAR is called for > stack/register slots instead of just Py_XDECREF. Might not be > necessary. Also, the intent is not to change any semantics here. The implementation of RETURN_VALUE_REG still Py_INCREFs the to-be-returned value. It's not like the data can get reclaimed before the caller receives it. S -- https://mail.python.org/mailman/listinfo/python-list
Re: [Python-ideas] Looking for people interested in a Python register virtual machine project
> In the "Object Lifetime" section you say "registers should be cleared upon > last reference". That isn't safe, since there can be hidden dependencies on > side effects of __del__, e.g.: > > process_objects = create_pipeline() > output_process = process_objects[-1] > return output_process.wait() > > If the process class terminates the process in __del__ (PyQt5's QProcess > does), then implicitly deleting process_objects after the second line will > break the code. Yeah, that is old writing, so is probably less clear (no pun intended) than it should be. In frame_dealloc, Py_CLEAR is called for stack/register slots instead of just Py_XDECREF. Might not be necessary. Skip -- https://mail.python.org/mailman/listinfo/python-list
Re: [Python-ideas] Looking for people interested in a Python register virtual machine project
Yes, I remember Parrot. As I understand it their original goal was a language-agnostic virtual machine, which might have complicated things. I will do a bit of reading and add some text to the "PEP." Skip On Sat, Mar 20, 2021, 11:36 AM David Mertz wrote: > The Parrot project was also intended to be the same thing, and for a while > had a fair number of contributors. Unfortunately, it never obtained the > performance wins that were good for. > > On Sat, Mar 20, 2021, 11:55 AM Skip Montanaro > wrote: > >> Back in the late 90s (!) I worked on a reimagining of the Python >> virtual machine as a register-based VM based on 1.5.2. I got part of >> the way with that, but never completed it. In the early 2010s, Victor >> Stinner got much further using 3.4 as a base. The idea (and dormant >> code) has been laying around in my mind (and computers) these past >> couple decades, so I took another swing at it starting in late 2019 >> after retirement, mostly as a way to keep my head in the game. While I >> got a fair bit of the way, it stalled. I've picked it up and put it >> down a number of times in the past year, often needing to resolve >> conflicts because of churn in the current Python virtual machine. >> Though I kept getting things back in sync, I realize this is not a >> one-person project, at least not this one person. There are several >> huge chunks of Python I've ignored over the past 20 years, and not >> just the internals. (I've never used async anything, for example.) If >> it is ever to truly be a viable demonstration of the concept, I will >> need help. I forked the CPython repo and have a branch (register2) of >> said fork which is currently synced up with the 3.10 (currently >> master) branch: >> >> https://github.com/smontanaro/cpython/tree/register2 >> >> I started on what could only very generously be called a PEP which you >> can read here. It includes some of the history of this work as well as >> details about what I've managed to do so far: >> >> https://github.com/smontanaro/cpython/blob/register2/pep-.rst >> >> If you think any of this is remotely interesting (whether or not you >> think you'd like to help), please have a look at the "PEP". Because >> this covers a fair bit of the CPython implementation, chances to >> contribute in a number of areas exist, even if you have never delved >> into Python's internals. Questions/comments/pull requests welcome. >> >> Skip Montanaro >> ___ >> Python-ideas mailing list -- python-id...@python.org >> To unsubscribe send an email to python-ideas-le...@python.org >> https://mail.python.org/mailman3/lists/python-ideas.python.org/ >> Message archived at >> https://mail.python.org/archives/list/python-id...@python.org/message/IUKZPH4ZSZ22RZFKMITQ3Q6A22P4BXWX/ >> Code of Conduct: http://python.org/psf/codeofconduct/ >> > -- https://mail.python.org/mailman/listinfo/python-list
Looking for people interested in a Python register virtual machine project
Back in the late 90s (!) I worked on a reimagining of the Python virtual machine as a register-based VM based on 1.5.2. I got part of the way with that, but never completed it. In the early 2010s, Victor Stinner got much further using 3.4 as a base. The idea (and dormant code) has been laying around in my mind (and computers) these past couple decades, so I took another swing at it starting in late 2019 after retirement, mostly as a way to keep my head in the game. While I got a fair bit of the way, it stalled. I've picked it up and put it down a number of times in the past year, often needing to resolve conflicts because of churn in the current Python virtual machine. Though I kept getting things back in sync, I realize this is not a one-person project, at least not this one person. There are several huge chunks of Python I've ignored over the past 20 years, and not just the internals. (I've never used async anything, for example.) If it is ever to truly be a viable demonstration of the concept, I will need help. I forked the CPython repo and have a branch (register2) of said fork which is currently synced up with the 3.10 (currently master) branch: https://github.com/smontanaro/cpython/tree/register2 I started on what could only very generously be called a PEP which you can read here. It includes some of the history of this work as well as details about what I've managed to do so far: https://github.com/smontanaro/cpython/blob/register2/pep-.rst If you think any of this is remotely interesting (whether or not you think you'd like to help), please have a look at the "PEP". Because this covers a fair bit of the CPython implementation, chances to contribute in a number of areas exist, even if you have never delved into Python's internals. Questions/comments/pull requests welcome. Skip Montanaro -- https://mail.python.org/mailman/listinfo/python-list
Re: .title() - annoying mistake
> > That's annoying. You have to roll your own solution! > Certainly seems like a known issue: https://bugs.python.org/issue12737 That issue was opened in 2011. Skip -- https://mail.python.org/mailman/listinfo/python-list
Re: [Python-Dev] Re: Python 0.9.1
> Also mind
> http://www.dalkescientific.com/writings/diary/archive/2009/03/27/python_0_9_1p1.html
> for result comparison.
Thanks, Paul. I had lost track of Andrew. Good to know he's still out
there. I wonder why his tar file was never sucked up into the
historical releases page.
Whew! My stupid little extraction script did a reasonable job. I see
plenty of differences, but a cursory examination shows they are only
in leading whitespace. Where I translated "\t" to TAB, it seems Andrew
used a suitable number of spaces. Python modules/scripts seem more
plausibly indented, and the couple I tried worked, so I'm a bit more
confident I have things right:
% PYTHONPATH=lib ./src/python
>>> import string
>>> print string.upper('hello world!')
HELLO WORLD!
>>>
% ./src/python lib/fact.py
9
[3, 3, 41, 271]
4096
[2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2]
The tests don't pass though. 1 * 1 raises an integer overflow exception:
>>> 1 * 1
Unhandled exception: run-time error: integer overflow
Stack backtrace (innermost last):
File "", line 1
I'll let someone figure that out. :-)
At any rate, the git repo has been updated.
Skip
--
https://mail.python.org/mailman/listinfo/python-list
Re: [Python-Dev] Re: Python 0.9.1
> If someone knows how to get the original Usenet messages from what Google > published, let me know. Seems the original shar is there buried in a Javascript string toward the end of the file. I think I've got a handle on it, though it will take a Python script to massage back into correct format. Skip -- https://mail.python.org/mailman/listinfo/python-list
Re: [Python-Dev] Re: Python 0.9.1
> > Wow. Was white-space not significant in this release of Python? I see the >> lack of indentation in the first Python programs. >> > > Indentation most certainly was significant from day 0. I suspect what > happened is that these files got busted somehow by the extraction process > used by Skip or Hiromi. > Yes, that's certainly possible. While it's nice that Google has archived this stuff, their faithfulness to the original formats leaves a bit to be desired (and gmane still doesn't work for me, eliminating that option). Guido's messages are displayed as HTML, and I saw no way to get at the raw Usenet messages. I just copied the shar data and saved the result. It seems clear that tabs copied as spaces. The Makefile indentation was hosed up. It should have dawned on me that the .py, .c and .h files would be messed up as well. I was only concerned with building the interpreter. If someone knows how to get the original Usenet messages from what Google published, let me know. Skip > -- https://mail.python.org/mailman/listinfo/python-list
Python 0.9.1
A note to webmas...@python.org from an astute user named Hiromi in
Japan* referred
us to Guido's shell archives for the 0.9.1 release from 1991. As that
wasn't listed in the historical releases README file:
https://legacy.python.org/download/releases/src/README
I pulled the shar files (and a patch), then made a few tweaks to get it to
build:
% ./python
>>> print 'hello world!'
hello world!
>>> import sys
>>> dir(sys)
['argv', 'exit', 'modules', 'path', 'ps1', 'ps2', 'stderr', 'stdin',
'stdout']
>>> sys.modules
{'builtin': ; 'sys': ; '__main__': }
>>> sys.exit(0)
I then pushed the result to a Github repo:
https://github.com/smontanaro/python-0.9.1
There is a new directory named "shar" with the original files, a small
README file and a compile.patch file between the original code and the
runnable code.
It was a pleasant diversion for a couple hours. I was tired of shovelling
snow anyway... Thank you, Hiromi.
Skip
* Hiromi is bcc'd on this note in case he cares to comment. I didn't want
to publish his email beyond the bounds of the webmaster alias without his
permission.
--
https://mail.python.org/mailman/listinfo/python-list
Re: UTF-16 or something else?
> > It's UTF-8 with a UTF-16 BOM prepended, which is not uncommon when you > have a file that's been converted to UTF-8 from UTF-16 or has been > produced by shitty Microsoft software. You can tell instantly at a > glance that it's not UTF-16 because the ascii dump would l.o.o.k. > .l.i.k.e. .t.h.i.s. > Ah, right. Been a long, long while (well before Unicode was a thing) since I needed to use od(1) and don't remember dealing with UTF-16 before. Skip -- https://mail.python.org/mailman/listinfo/python-list
Re: UTF-16 or something else?
> > Try setting encoding to: "utf-8-sig". > > 'eb bb bf' is the byte order mark for UTF8 (most systems do not include > this in UTF-8 encoded files) > > Python will correctly read UTF8 BOMs if you use the 'utf-8-sig' encoding > when reading files > Excellent, thanks. That worked like a charm. Knowing what its called also allowed me to look up more info. Skip -- https://mail.python.org/mailman/listinfo/python-list
UTF-16 or something else?
I downloaded US hospital ICU capacity data this morning from this page: https://healthdata.gov/dataset/covid-19-reported-patient-impact-and-hospital-capacity-facility (The download link is about halfway down the page.) Trying to read it using my personal CSV tools without specifying an encoding, it failed to understand the first column, hospital_pk. That is apparently because the file isn't simply ASCII or UTF-8. There are a few bytes ahead of the "h". However, if I open the file using "utf-16" as the encoding, Python complains there is no BOM. od(1) suggests there is *something* ahead of the first column name, but it's three bytes, not two: % od -A x -t x1z -v < reported_hospital_capacity_admissions_facility_level_weekly_average_timeseries_20210207.csv | head 00 *ef bb bf* 68 6f 73 70 69 74 61 6c 5f 70 6b 2c 63 >...hospital_pk,c< 10 6f 6c 6c 65 63 74 69 6f 6e 5f 77 65 65 6b 2c 73 >ollection_week,s< 20 74 61 74 65 2c 63 63 6e 2c 68 6f 73 70 69 74 61 >tate,ccn,hospita< ... I'm opening the file like so: inf = open(args[0], "r", encoding=encoding) where encoding is passed on the command line. I know I can simply edit out those bytes and probably be good-to-go, but I'd prefer not to. What should I be passing for the encoding? Skip, who thought everybody had effectively settled on utf-8 at this point, but apparently not... -- https://mail.python.org/mailman/listinfo/python-list
Log exception so traceback contains timestamp and level?
The logging package can log exceptions and call stacks, but it does
(in my opinion) a suboptimal job of it. Consider this simple example:
>>> import logging
>>> FORMAT = '%(asctime)-15s %(levelname)s %(message)s'
>>> logging.basicConfig(format=FORMAT, force=True)
>>> log.warning("msg", stack_info=True)
2021-02-06 20:46:52,399 WARNING msg
Stack (most recent call last):
File "", line 1, in
It formats the warning message just fine, but simply dumps the
traceback onto the stream with no timestamp or level. For my purposes,
this is more important for exceptions. (I'm just using the stack trace
feature as it's easier in a small example.) I would like to call
something like
log.exception("Some message...")
and find something like this in the output stream:
2021-02-06 20:46:52,399 ERROR Some message...
2021-02-06 20:46:52,400 ERROR Traceback (most recent call last):
2021-02-06 20:46:52,402 ERROR File "", line 1, in
That way I can more easily grep log files for errors and get the
entire detail, including the traceback. It seems I have to subclass
logging.Formatter to override formatStack or formatException, then
construct the individual lines of output, set handler and formatter,
blah blah blah.
I'm pretty sure I've done this in the past by writing my own little
log_exception method which formats the exception and calls log.error
for each row of the traceback. That seems easier than the "right" way.
That seems harder than it ought to be. Hopefully I'm just missing
something simple.
Skip
--
https://mail.python.org/mailman/listinfo/python-list
Best practice for handling exceptions raised at lower levels?
I'm curious if there are best practices for handling exceptions raised by lower level modules and packages. For example, I wrote an application called polly[1] which constructs a personalized dictionary from email messages using IMAP4. That dictionary is then used as input to a password generator based on the famous XKCD 936 comic[2]. The idea is based on a similar tool Chris Angelico mentioned here several years ago[3], though as not a D&D player, I didn't have player chat available as user-specific input from which to construct a dictionary and chose email instead. So much for background. After a long period of stasis, I decided to update polly to Python 3. While the 2-to-3 process was straightforward, I had problems after the update which looked like IMAP4 issues. I really didn't want to get knee deep in a pond full of IMAP alligators, so rather than bookmarking the IMAP4 RFC[4], I cast about for something higher level than imaplib, eventually settling on IMAPClient[5]. I thus have a collection of modules and packages: imapclient, imaplib, ssl, socket, and many other non-network bits. IMAPClient did what I wanted, for the most part abstracting away IMAP4 details. I still have to craft a couple IMAP4 parameter strings, but the rest of the interface is pretty Pythonic. However... Network applications being what they are, hiccups are going to happen. In the time since I swapped in the imapclient package, I've also had to catch exceptions raised by lower level modules/packages I wasn't using directly, discovering them only as they occurred. (Think of it as lazy exception discovery. I suppose if I was developing the application for others, I'd have been more proactive about uncovering the exceptions.) In addition to a couple imapclient exceptions, I also now need to catch socket.gaierror, ConnectionError and imaplib.IMAP4.error. Here's the crux of the problem. Where does responsibility generally fall for low level exception handling? I don't mean to pick on IMAPClient. It's just a recent example and got me thinking about the problem. Is it (generally) the responsibility of the application author or the package author? This isn't an issue just for network applications with a number of moving parts (socket, ssl, higher level packages, etc). It happens in almost all applications or packages, even if it's just to deal with exceptions raised by internal data structures. Skip [1] https://github.com/smontanaro/polly [2] https://xkcd.com/936/ [3] https://mail.python.org/pipermail/python-list/2014-August/827854.html [4] https://tools.ietf.org/html/rfc3501 [5] https://pypi.org/project/IMAPClient/ -- https://mail.python.org/mailman/listinfo/python-list
Re: IDE tools to debug in Python?
> > > Python is an interactive language. You can develop a lot while working > > on a Python console. Then copy and paste into a program. > > Absolutely, the humble interactive prompt is often overlooked > as a development tool. It's not as good as the "evaluate > expression" tool in the Smalltalk workspace, say, but it's close. > In an earlier reply I failed to mention Jupyter. It's a pretty awesome environment. Sort of takes the whole concept of "just insert prints" to a new level. I started using it a bit at my last job for financial stuff, and still mostly use it for that, but it's another viable option. Skip > -- https://mail.python.org/mailman/listinfo/python-list
Re: How do you debug in Python? Coming from a Matlab and R user. I'm already aware of pdb.
CW> How do you troubleshooting/debugging in Python? GE> Mostly I read exception trace and read the code and think about it. GE> If that doesn't work, I add some print() or syslog() calls. CW> I know hardcore computer scientists would tell me about Python debugger. GE> I've been writing Python for 20+ years. I've never tried a debugger. Agree with Grant on these points. I certainly use gdb to debug C code (like the interpreter), but for Python code, tracebacks and print statements pretty much take care of things. I know pdb has been around for a long while and many people like IDEs, but I've not generally found them all that useful. I'm stuck in the 90s I guess. Skip -- https://mail.python.org/mailman/listinfo/python-list
Re: why sqrt is not a built-in function?
> I don't understand why sqrt is not a built-in function. > Why do we have to first import the function from the math module? > I use it ALL THE TIME! For one, it's specific to numeric types. You might use it all the time, but I (for example) almost never need to use it, or to import the math module for my Python work/play. OTOH, I use stuff from the sys and csv modules (and lately, pandas) all the time. I import those modules when necessary. If you don't need them, I suspect you might grumble that they are useless for you. Requiring explicit module import gives programmers more control over the content of their module namespaces if the builtins don't include the kitchen sink. If you skim the output of import builtins dir(builtins) you should see that the two largest classes of builtin identifiers are exceptions (ZeroDivisionError, etc) and types (list, set, etc). Other classes include singleton constants (True, False, None) and general sequence operations (useful for loop control - reversed, iter, zip...). math.sqrt doesn't fit into those object classes. The remainder are a mixture of things, but generally aren't quite as special purpose as that. Granted, there are a couple which might be better left out (IMO), like round and pow, but I suspect their presence might simply be more a matter of them being present since Python's earliest days and reflect a tendency later to avoid gratuitous breakage of existing code. Finally, should have never considered it, I think you might want to study the output of import this Think on the second and last lines in particular. Skip -- https://mail.python.org/mailman/listinfo/python-list
Re: Python3 change logs
Dang... I'm having very incomplete thoughts. Apologies for the multiple replies when one would have sufficed. https://docs.python.org/3/whatsnew/3.9.html Skip -- https://mail.python.org/mailman/listinfo/python-list
Re: Python3 change logs
Also, check www.python.org for "What's New" pages. I believe one is generated for every release. It will be less detailed than change logs in GitHub, but more reader friendly. Skip -- https://mail.python.org/mailman/listinfo/python-list
Re: Python3 change logs
Change logs are kept as part of the source, I believe. Try browsing the cpython GitHub repo: https://github.com/python/cpython Skip On Tue, Dec 15, 2020, 10:05 AM Rich Shepard wrote: > I've upgraded from Python-3.7.x to Python-3.9.x and want to learn about > differences (if any) in tkinter between the two versions. > > Looking on the python.org web site I did not find change logs with the > sources or under the other menus I checked. Please point me to any > available > logs. > > Regards, > > Rich > -- > https://mail.python.org/mailman/listinfo/python-list > -- https://mail.python.org/mailman/listinfo/python-list
Re: How to run Jupyter notebook in command line and get full error message?
> > My VPN keeps dropping and can not run Jupyter Notebook as it is. > You don't provide a lot of detail, but this seems similar to the kind of flaky networking we used to deal with in the Before Times. Simply connecting directly to a host over the Internet was often plagued by disconnects. For that we used screen(1). You don't mention your operating system. It's available for most/all Linux systems and Macs. If you're on Windows I imagine you could get it using WSL. Of course, I might be way off-base focusing on just the above statement from your post. If so, you might want to provide some more details about your problem. Skip -- https://mail.python.org/mailman/listinfo/python-list
Re: Cannot marshal objects
> Because the marshaling is happening in the guts of xmlrpc.
Okay, I misunderstood your original message. You used the word
"marshal" (and the xmlrpc error did as well). I thought you were
referring to the marshal module. I didn't understand (and ignored) the
references to the xmlrpc module. In Python 3, the Marshaller class has
migrated into the xmlrpc.client module.
Your code worked for me in 3.8. Note, however, that this is not going
to round trip properly. You will stuff in a Decimal object at one end
and get out a double at the other end. Recent versions support
unmarshalling to Decimal objects in the Unmarshaller class. I don't
know why the Marshaller class doesn't accept Decimal objects for
serialization. You could try this:
def dump_decimal(self, value, write):
write("")
write(str(value))
write("\n")
xmlrpc.client.Marshaller.dispatch[decimal.Decimal] = dump_decimal
I used the "ex:" prefix based on this document:
http://ws.apache.org/xmlrpc/types.html
YMMV. The tag name understood by the Unmarshaller class doesn't
include that prefix.
Skip
--
https://mail.python.org/mailman/listinfo/python-list
Re: Cannot marshal objects
> I am getting this error.
I assume you mean the email subject. It doesn't work in 3.8 either:
>>> import decimal
>>> d = decimal.Decimal(3.5)
>>> d
Decimal('3.5')
>>> import marshal
>>> marshal.dumps(d)
Traceback (most recent call last):
File "", line 1, in
ValueError: unmarshallable object
but that's not surprising to me. The marshal module is more-or-less
meant to serialize Python byte code. Pickle is more generally used for
object serialization. Why not use it?
>>> import pickle, decimal
>>> d = decimal.Decimal(3.5)
>>> pickle.dumps(d)
b'\x80\x04\x95!\x00\x00\x00\x00\x00\x00\x00\x8c\x07decimal\x94\x8c\x07Decimal\x94\x93\x94\x8c\x033.5\x94\x85\x94R\x94.'
That's meant more for serialization of data objects.
Skip
--
https://mail.python.org/mailman/listinfo/python-list
Re: constant folding - why not more
>
> On the contrary, comparison remains for runtime:
> >>> dis.dis(compile('1 < 2', filename='', mode='eval',
> >>> optimize=2))
> 1 0 LOAD_CONST 0 (1)
> 2 LOAD_CONST 1 (2)
> 4 COMPARE_OP 0 (<)
> 6 RETURN_VALUE
> In function fold_unaryop (though comparison is a binary operation) in
> Python/ast_opt.c is remark: /* Fold not into comparison */
>
> Is there a reason why comparison (== != < > <= >=) is not folded?
>
I can think of two reasons. One, this kind of comparison will almost never
appear in production code (maybe in unit tests?). Unlike the C family of
languages, Python doesn't have a macro processor which would give symbolic
names to numeric constants or string literals. Code generators might
conceivably generate constant comparisons, but they might be able to easily
do constant folding of comparisons themselves.
Two, given that this sort of construct will almost never be found in the
wild, folding constant comparisons in the compiler would increase the
maintenance burden of the compiler (just slightly, but still...) with no
clear benefit.
Skip
>
--
https://mail.python.org/mailman/listinfo/python-list
Re: Help
> > When I run this, my answer is 8 but it should be 336 can some help ._. > Looks like you are returning from inside the loop. Skip > -- https://mail.python.org/mailman/listinfo/python-list
Re: Artie 3000 the coding robot
> > Sounds like an expensive way to do turtle graphics... > > > Really expensive -- From what I could make out, the "robot" has no > independent functioning. It is tethered (WiFi, maybe Bluetooth) to the > controlling computer. The programs run on the computer and send command > packets to the robot... Thanks Dennis. My initial thought was that it probably used the turtle module, but I didn't see it mentioned anywhere. Seems the "value add" is just the communication module. I guess I'll pass. Skip -- https://mail.python.org/mailman/listinfo/python-list
Artie 3000 the coding robot
I'm looking for feedback on Artie 3000 the Coding Robot. It's recommended by wirecutter.com for introductory programming for ages seven and up (though stemfinity.com <https://www.stemfinity.com/Artie-3000-The-Coding-Robot> says 7-12). I was thinking about getting it for my grandson who's turning 12 next month. I worry the introductory lessons might be kind of boring for him (though he's very into Minecraft, so who knows), and I've seen nothing online about transitioning to programming Artie using Python. His dad's a techie, so I'm not worried if he gets stuck. Here's the manufacturer's page <https://www.educationalinsights.com/artie-3000-8482-the-coding-robot> for the tool/toy. Any feedback appreciated... Skip -- https://mail.python.org/mailman/listinfo/python-list
Re: Python Pandas split Date in day month year and hour
> I would like know how can I get for this array the values for day, month > and hour: > > 2017-11-01 03:00:00 year = 2017 month = 11day = 1and > hour = 3 Pandas has a datetime type. You should probably be using it. It's been awhile (a year at least), but if your datetime column is filled with datetime types, you should be able to append columns to your DataFrame which correspond to individual components of the datetime instances. Don't be looking outside of Pandas to Python's time or datetime module. Skip -- https://mail.python.org/mailman/listinfo/python-list
Re: Syntax question
> Typing is not required by > Python. However, you may find the extra error-checking helpful... I haven't used type hints much, if at all, but my understanding is that the "extra error-checking" of which you speak is gotten through other static checkers, correct? I know the syntax was developed with the MyPy folks (http://mypy-lang.org/), but I'm not sure what other tools currently make use of it. Any pointers? Skip -- https://mail.python.org/mailman/listinfo/python-list
Re: Explicit is better than Implicit
> > When Excel reads a file, it looks for stuff and decides to upgrade its > type. Eg dates etc (particularly pernicious with US-style dates versus > the rest of the planet). Mojibake for data ensues. > > As always, I am reminded of Heuer's Razor: > > If it can't be turned off, it's not a feature. - Karl Heuer > Good one. I always remember the start-up days (late 90s) when I developed and maintained an online concert calendar (Musi-Cal) written in Python. The technology got bought by another start-up (Mojam) who used Perl for their web stuff. Both front end systems talked to my Python-based back end (communication between both front ends and the one back end was via XML-RPC). I was sometimes frustrated by the stuff Perl did. The one which stuck with me all these years was its silent conversion of the band name " 311 <https://en.wikipedia.org/wiki/311_(band)>" to the integer 311 on which my Python backend obligingly barfed. I eventually had to put in data type checks for all fields in my back end (my front end already had that sort of input validation) as I could no longer assume a sentient front end was handing it data. Skip -- https://mail.python.org/mailman/listinfo/python-list
Explicit is better than Implicit
Hmmm... Rename genes, fix Excel, or dump Excel in favor of Python? I know what my choice would have been. :-) https://www.theverge.com/2020/8/6/21355674/human-genes-rename-microsoft-excel-misreading-dates Skip -- https://mail.python.org/mailman/listinfo/python-list
Re: Strings: double versus single quotes
> > > May I ask why not simply like this: > > stmt = """ > select foo from bar > where a = 'bag' > and c = 'dog' > """ > Sorry, I don't recall. I wouldn't be at all surprised that it has something to do with Emacs's Python mode behavior. Emacs wouldn't know what to do with spaces in the string, but knows where to put string literals within the open parens. I'm pretty sure I was doing this before triple quoted strings existed. Thankfully, I don't need to mess around with SQL anymore. :-) Skip > -- https://mail.python.org/mailman/listinfo/python-list
