Re: Help with Streaming and Chunk Processing for Large JSON Data (60 GB) from Kenna API
On Thu, 3 Oct 2024 at 08:48, Left Right wrote: > > > You can't validate an IP packet without having all of it. Your notion > > of "streaming" is nonsensical. > > Whoa, whoa, hold your horses! "nonsensical" needs a little bit of > justification :) > > It seems you don't understand the difference between words and > languages! In my examples, IP _protocol_ is the language, sequences of > IP packets are the words in the language. A language is amenable to > streaming if the words of the language are repetition of sequences of > symbols of the alphabet of fixed length. This is, essentially, like > saying that the words themselves are regular. One single IP packet is all you can parse. You're playing shenanigans with words the way Humpty Dumpty does. IP packets are not sequences, they are individuals. ChrisA -- https://mail.python.org/mailman/listinfo/python-list
Re: Help with Streaming and Chunk Processing for Large JSON Data (60 GB) from Kenna API
On Wed, 2 Oct 2024 at 23:53, Left Right via Python-list wrote: > In the same email you replied to, I gave examples of languages for > which parsers can be streaming (in general): SCSI or IP. You can't validate an IP packet without having all of it. Your notion of "streaming" is nonsensical. ChrisA -- https://mail.python.org/mailman/listinfo/python-list
Re: Help with Streaming and Chunk Processing for Large JSON Data (60 GB) from Kenna API
On Tue, 1 Oct 2024 at 08:56, Grant Edwards via Python-list wrote: > > On 2024-09-30, Dan Sommers via Python-list wrote: > > > In Common Lisp, integers can be written in any integer base from two > > to thirty six, inclusive. So knowing the last digit doesn't tell > > you whether an integer is even or odd until you know the base > > anyway. > > I had to think about that for an embarassingly long time before it > clicked. The only part I'm not clear on is what identifies the base. If you're going to write numbers little-endian, it's not that hard to also write them with a base indicator before the digits. But, whatever. This is a typical tangent and people are argumentative for no reason. I was just trying to add some explanatory notes to why little-endian does make more sense than big-endian. ChrisA -- https://mail.python.org/mailman/listinfo/python-list
Re: Help with Streaming and Chunk Processing for Large JSON Data (60 GB) from Kenna API
On Tue, 1 Oct 2024 at 04:30, Dan Sommers via Python-list wrote: > > But why do I need to start with the least > significant digit? If you start from the most significant, you don't know anything about the number until you finish parsing it. There's almost nothing you can say about a number given that it starts with a particular sequence (since you don't know how MANY digits there are). However, if you know the LAST digits, you can make certain statements about it (trivial examples being whether it's odd or even). It's not very, well, significant. But there's something to it. And it extends nicely to p-adic numbers, which can have an infinite number of nonzero digits to the left of the decimal: https://en.wikipedia.org/wiki/P-adic_number ChrisA -- https://mail.python.org/mailman/listinfo/python-list
Re: Help with Streaming and Chunk Processing for Large JSON Data (60 GB) from Kenna API
On Tue, 1 Oct 2024 at 02:20, Thomas Passin via Python-list
wrote:
>
> On 9/30/2024 11:30 AM, Barry via Python-list wrote:
> >
> >
> >> On 30 Sep 2024, at 06:52, Abdur-Rahmaan Janhangeer via Python-list
> >> wrote:
> >>
> >>
> >> import polars as pl
> >> pl.read_json("file.json")
> >>
> >>
> >
> > This is not going to work unless the computer has a lot more the 60GiB of
> > RAM.
> >
> > As later suggested a streaming parser is required.
>
> Streaming won't work because the file is gzipped. You have to receive
> the whole thing before you can unzip it. Once unzipped it will be even
> larger, and all in memory.
Streaming gzip is perfectly possible. You may be thinking of PKZip
which has its EOCD at the end of the file (although it may still be
possible to stream-decompress if you work at it).
ChrisA
--
https://mail.python.org/mailman/listinfo/python-list
Re: python3 package import difference?
On Thu, 8 Aug 2024 at 03:40, Tobiah via Python-list
wrote:
> The one under rcs.dbi contains:
>
> from dbi import *
> from regos import *
>
You probably want these to be package-relative now:
from .dbi import *
from .regos import *
Or, since you're using namespaced imports anyway ("rcs.dbi.feature"),
you may prefer this form:
from . import dbi, regos
ChrisA
--
https://mail.python.org/mailman/listinfo/python-list
Re: python C-api and thread
On Tue, 6 Aug 2024 at 08:48, aotto1968 via Python-list wrote: > > hi, > > Is it possible to run two completely independent Python interpreters in one > process, each using a thread? > > By independent, I mean that no data is shared between the interpreters and > thus the C API can be used without any other > "lock/GIL" etc. > You're probably thinking of subinterpreters: https://peps.python.org/pep-0734/ ChrisA -- https://mail.python.org/mailman/listinfo/python-list
Re: Anonymous email users
On Wed, 26 Jun 2024 at 03:40, Anton Shepelev via Python-list wrote: > > Chris Angelico to dn: > > > > Python mailing-lists are covered by the Code of Conduct > > > and monitored by ListAdmins. Thus, there are controls > > > which limit the impact which advertisers and others with > > > non-pythonic aims might otherwise exert! > > > > So long as there's a newsgroup gateway, those controls are > > toothless. > > The gateway operator can have the usual anti-spam software > installed Anti-spam is not the same as CoC and admins, though. Without putting an actual moderation barrier in there, it's still toothless. (Yes, there are a scant few posters who've been blocked from the gateway, but it's rare.) ChrisA -- https://mail.python.org/mailman/listinfo/python-list
Re: Anonymous email users
On Tue, 25 Jun 2024 at 11:41, Grant Edwards via Python-list wrote: > I've been using the same e-mail address for about 20 years. I've use > that e-mail address with probably close to 100 retailers, charities, > open-source projects, media sites, and various other organizations. Mostly the same, although in my case, I've had multiple email addresses for different purposes (and still kept all of them for decades). > I get at most a few spam emails per week [I just checked my spam > folder: 8 in the past 30 days]. And Gmail is very, very close to 100% > accurate at filtering them out. I can't remember the last time I > actually got a spam message in my inbox. > > > A few years ago the spam count was greater than a 1,000 a month. > > I'm baffled. Is Gmail silently rejecting that much junk before it > even gets to the filter that puts stuff into my "spam" folder? > It really depends on how you count. On my mail server (can't get stats for Gmail), I have a number of anti-spam and anti-abuse rules that apply prior to the Bayesian filtering (for example, protocol violations), and any spam that gets blocked by those rules isn't shown in my stats. And then I have a further set of rules that nuke some of the most blatant spam, and finally the regular trainable filter. I should probably keep better stats on the stuff I don't keep, but at the moment, all I actually track is the ones that the filter sees - which is roughly 25-50 a day. So yeah, Gmail is probably rejecting that much junk, but most of it for protocol violations. ChrisA -- https://mail.python.org/mailman/listinfo/python-list
Re: Anonymous email users
On Tue, 25 Jun 2024 at 08:31, dn via Python-list wrote: > Python mailing-lists are covered by the Code of Conduct and monitored by > ListAdmins. Thus, there are controls which limit the impact which > advertisers and others with non-pythonic aims might otherwise exert! > So long as there's a newsgroup gateway, those controls are toothless. ChrisA -- https://mail.python.org/mailman/listinfo/python-list
Re: Decoding bytes to text strings in Python 2
On Mon, 24 Jun 2024 at 10:18, MRAB via Python-list wrote: > Tkinter in recent versions of Python can handle astral characters, at > least back to Python 3.8, the oldest I have on my Windows PC. Good to know, thanks! I was hoping that would be the case, but I don't have a Windows system to check on, so I didn't want to speak without facts. ChrisA -- https://mail.python.org/mailman/listinfo/python-list
Re: Decoding bytes to text strings in Python 2
On Mon, 24 Jun 2024 at 08:20, Rayner Lucas via Python-list wrote: > > In article , > ros...@gmail.com says... > > > > If you switch to a Linux system, it should work correctly, and you'll > > be able to migrate the rest of the way onto Python 3. Once you achieve > > that, you'll be able to operate on Windows or Linux equivalently, > > since Python 3 solved this problem. At least, I *think* it will; my > > current system has a Python 2 installed, but doesn't have tkinter > > (because I never bothered to install it), and it's no longer available > > from the upstream Debian repos, so I only tested it in the console. > > But the decoding certainly worked. > > Thank you for the idea of trying it on a Linux system. I did so, and my > example code generated the error: > > _tkinter.TclError: character U+1f40d is above the range (U+-U+) > allowed by Tcl > > So it looks like the problem is ultimately due to a limitation of > Tcl/Tk. Yep, that seems to be the case. Not sure if that's still true on a more recent Python, but it does look like you won't get astral characters in tkinter on the one you're using. > I'm still not sure why it doesn't give an error on Windows and Because of the aforementioned weirdness of old (that is: pre-3.3) Python versions on Windows. They were built to use a messy, buggy hybrid of UCS-2 and UTF-16. Sometimes this got you around problems, or at least masked them; but it wouldn't be reliable. That's why, in Python 3.3, all that was fixed :) > instead either works (when UTF-8 encoding is specified) or converts the > out-of-range characters to ones it can display (when the encoding isn't > specified). But now I know what the root of the problem is, I can deal > with it appropriately (and my curiosity is at least partly satisfied). Converting out-of-range characters is fairly straightforward, at least as long as your Python interpreter is correctly built (so, Python 3, or a Linux build of Python 2). "".join(c if ord(c) < 65536 else "?" for c in text) > This has given me a much better understanding of what I need to do in > order to migrate to Python 3 and add proper support for non-ASCII > characters, so I'm very grateful for your help! > Excellent. Hopefully all this mess is just a transitional state and you'll get to something that REALLY works, soon! ChrisA -- https://mail.python.org/mailman/listinfo/python-list
Re: Decoding bytes to text strings in Python 2
On Sat, 22 Jun 2024 at 03:28, Rayner Lucas via Python-list wrote: > I'm curious about something I've encountered while updating a very old > Tk app (originally written in Python 1, but I've ported it to Python 2 > as a first step towards getting it running on modern systems). > > I am using Python 2.7.18 on a Windows 10 system. If there's any other > relevant information I should provide please let me know. Unfortunately, you're running into one of the most annoying problems from Python 2 and Windows: "narrow builds". You don't actually have proper Unicode support. You have a broken implementation that works for UCS-2 but doesn't actually support astral characters. If you switch to a Linux system, it should work correctly, and you'll be able to migrate the rest of the way onto Python 3. Once you achieve that, you'll be able to operate on Windows or Linux equivalently, since Python 3 solved this problem. At least, I *think* it will; my current system has a Python 2 installed, but doesn't have tkinter (because I never bothered to install it), and it's no longer available from the upstream Debian repos, so I only tested it in the console. But the decoding certainly worked. ChrisA -- https://mail.python.org/mailman/listinfo/python-list
Re: Anonymous email users
On Sat, 15 Jun 2024 at 08:32, dn via Python-list wrote: > These mailing-lists all run under the Python Code of Conduct. > The newsgroup, however, is not. Which means that anyone who posts on the newsgroup is subject to no such restrictions - and that might explain the, shall we say, quite different signal-to-noise ratio of newsgroup posters compared to mailing list posters. ChrisA -- https://mail.python.org/mailman/listinfo/python-list
Re: Couldn't install numpy on Python 2.7
On Thu, 13 Jun 2024 at 10:58, wrote: > > Chris, > > You seem to have perceived an insult that I remain unaware of. If you're not aware that you're saying this, then don't say it. > I looked up FUD and sharply disagree with suggestions I am trying to somehow > cause Fear, Uncertainty or Doubt. I simply asked if another such update ... > as a hypothetical. Had I asked what impact Quantum Computers might have on > existing languages, would that also be FUD, or just a speculation in a > discussion. What DID you intend by your comments? Were you trying to imply that work spent upgrading to Python 3 would have to be redone any day now when this hypothetical massively-incompatible Python 4 is released? Or what? What WERE you trying to say? If you don't understand how damaging it can be to say that sort of thing, **don't say it**. Otherwise, expect responses like this. I *detest* the attitude that you can make vague disparaging comments and then hide behind claims that you had no idea how damaging you were being. ChrisA -- https://mail.python.org/mailman/listinfo/python-list
Re: Couldn't install numpy on Python 2.7
On Thu, 13 Jun 2024 at 09:20, wrote: > My point was that version 4 COULD HAPPEN one day and I meant INCOMPATIBLE > version not 4. Obviously we can make a version 4 that is not incompatible > too. This is still FUD. Back your words with something, or stop trying to imply that there's another incompatible change just around the corner. Do you realise how insulting you are being to the developers of Python by these implications? ChrisA -- https://mail.python.org/mailman/listinfo/python-list
Re: Couldn't install numpy on Python 2.7
On Thu, 13 Jun 2024 at 08:46, Oscar Benjamin via Python-list wrote: > I don't know much about SSL and related networking things especially > on Windows. I would be surprised if pip on old Python can't install > from current PyPI though. I imagine that something strange has > happened like a new version of pip running on an old version of Python > or old Python on new OS (or old PyCharm...). > > There is no problem using Python 2.7 with pip and PyPI on this Linux > machine but I guess it has a newer SSL library provided by the OS: Sadly, I would NOT be surprised if this is indeed a problem on Windows. You're exactly right - on Linux, it can use a newer SSL library from the OS. Of course, this does assume that you've updated your OS, which is far from a guarantee, but since this has security implications there's a good chance you can update it while retaining a legacy system. On Thu, 13 Jun 2024 at 08:51, Greg Ewing via Python-list wrote: > On 13/06/24 10:09 am, Chris Angelico wrote: > > So if anyone > > actually does need to use pip with Python 2.7, they probably need to > > set up a local server > > You should also be able to download a .tar.gz from PyPI and use pip > to install that. Although you'll have to track down the dependencies > yourself in that case. Also a possibility; in my opinion, losing dependency management is too big a cost, so I would be inclined to set up a local server. But then, I would be using a newer SSL library and not have the problem in the first place. ChrisA -- https://mail.python.org/mailman/listinfo/python-list
Re: Couldn't install numpy on Python 2.7
On Thu, 13 Jun 2024 at 07:57, Oscar Benjamin via Python-list wrote: > They are seeing a warning that explicitly says "You can upgrade to a > newer version of Python to solve this". I don't know whether that SSL > warning is directly connected to pip not finding any versions of numpy > but with the available information so far that seems like the first > thing to consider. I think it is; AIUI, with an ancient SSL library, pip is unable to download packages safely from the current pypi server. So if anyone actually does need to use pip with Python 2.7, they probably need to set up a local server, using older encryption protocols (which should therefore NOT be made accessible to the internet). Since pip can't contact the upstream pypi, there's no available numpy for it to install. ChrisA -- https://mail.python.org/mailman/listinfo/python-list
Re: Couldn't install numpy on Python 2.7
On Thu, 13 Jun 2024 at 07:36, wrote: > But if the goal was to deprecate python 2 and in some sense phase it out, it > is perhaps not working well for some. Frankly, issuing so many updates like > 2.7 and including backporting of new features has helped make it possible to > delay any upgrade. The goal was to improve Python. I don't think anyone's ever tried to "kill off" Python 2 - not in the sense of stopping people from using it - but there haven't been any changes, not even security fixes, in over four years. > And, yes, I was KIDDING about python 4. I am simply suggesting that there > may well be a time that another shift happens that may require another > effort to get people on board a new and perhaps incompatible setup. Kidding, eh? It sure sounded like you were trying to imply that there would inevitably be another major breaking change. It definitely smelled like FUD. Maybe your jokes just aren't funny. ChrisA -- https://mail.python.org/mailman/listinfo/python-list
Re: Couldn't install numpy on Python 2.7
On Thu, 13 Jun 2024 at 06:55, Thomas Passin via Python-list wrote: > The project cannot move to a Python-3 compatible version because Jython > 3.xx doesn't exist and may never exist. The saving grace is that my > project doesn't have to use packages like numpy, scipy, and so forth. Exactly. If you don't need to ALSO use something newer, there's nothing stopping you from continuing with the old version. And that's fine! As long as you're okay with not getting updates, you're welcome to do whatever you like - including running Windows 98 on an ancient PC and editing your documents on that. (Yes, I know someone who did that, long after Win 98 was dead to most of us.) > Thunderbird and everything else worked perfectly for me during that > week. True, there were a few Windows-only programs I missed, but I used > other similar programs even if I didn't like them as much. It's true. And there ARE solutions to that, although it's a bit rough trying to run them on low hardware (Wine works nicely for some programs, less so for others; VirtualBox is really not gonna be happy with a small fraction of your limited RAM). But if your needs are simple, even a crazily low-end system is sufficient. > It's amazing > how little resources Linux installs need, even with a GUI. Of course, > 4GB RAM is limiting whether you are on Linux or Windows - you can't > avoid shuffling all those GUI bits around - but with a little care it > worked great. And with the external SSD the laptop was a lot snappier > than it ever was when it was new. One of the big differences with Linux is that you have a choice of desktop environments, from "none" (just boot straight into a terminal) on up. Some of them are a bit of a compromise in terms of how well you can get your work done, but let's say you had an even MORE ancient system with maybe one gig of memory... I'd rather have a super-light desktop environment even if it doesn't have everything I'm normally accustomed to! ChrisA -- https://mail.python.org/mailman/listinfo/python-list
Re: Couldn't install numpy on Python 2.7
On Thu, 13 Jun 2024 at 03:41, AVI GROSS via Python-list wrote: > > Change is hard even when it may be necessary. > > The argument often is about whether some things are necessary or not. > > Python made a decision but clearly not a unanimous one. What decision? To not release any new versions of Python 2? That isn't actually the OP's problem here - the Python interpreter runs just fine. But there's no numpy build for the OP's hardware and Python 2.7. So if you want to complain about Python 2.7 being dead, all you have to do is go through all of the popular packages and build binaries for all modern computers. If that sounds easy, go ahead and do it; if it sounds hard, realise that open source is not a democracy, and you can't demand that other people do more and more and more unpaid work just because you can't be bothered upgrading your code. > My current PC was not upgradable because of the new hardware requirement > Microsoft decided was needed for Windows 11. Yes, and that's a good reason to switch to Linux for the older computer. > I mention this in the context of examples of why even people who are fairly > knowledgeable do not feel much need to fix what does not feel broken. It doesn't feel broken, right up until it does. The OP has discovered that it *IS* broken. Whining that it doesn't "feel broken" is nonsense when it is, in fact, not working. > When is Python 4 coming? Is this just another content-free whine, or are you actually curious about the planned future of Python? If the latter, there is **PLENTY** of information out there and I don't need to repeat it here. Please don't FUD. ChrisA -- https://mail.python.org/mailman/listinfo/python-list
Re: Couldn't install numpy on Python 2.7
On Wed, 12 Jun 2024 at 21:32, marc nicole via Python-list wrote: > > I am trying to install numpy library on Python 2.7.15 in PyCharm but the > error message I get is: > > You can upgrade to a newer version of Python to solve this. The answer is right there in the error message. ChrisA -- https://mail.python.org/mailman/listinfo/python-list
Re: From JoyceUlysses.txt -- words occurring exactly once
On Wed, 5 Jun 2024 at 02:49, Edward Teach via Python-list wrote: > > On Mon, 03 Jun 2024 14:58:26 -0400 (EDT) > Grant Edwards wrote: > > > On 2024-06-03, Edward Teach via Python-list > > wrote: > > > > > The Gutenburg Project publishes "plain text". That's another > > > problem, because "plain text" means UTF-8and that means > > > unicode...and that means running some sort of unicode-to-ascii > > > conversion in order to get something like "words". A couple of > > > hoursa couple of hundred lines of Cproblem solved! > > > > I'm curious. Why does it need to be converted frum Unicode to ASCII? > > > > When you read it into Python, it gets converted right back to > > Unicode... > > > > Well.when using the file linux.words as a useful master list of > "words".linux.words is strict ASCII > Whatever gave you that idea? I have a large number of dictionaries in /usr/share/dict, all of them encoded UTF-8 except one (and I don't know why that is). Even the English ones aren't entirely ASCII. There is no need to "convert from Unicode to ASCII", which makes no sense. ChrisA -- https://mail.python.org/mailman/listinfo/python-list
Re: Formatted Output and Argument Parsing (was: Re: Flubbed it in the second interation through the string: range error... HOW?)
On Wed, 29 May 2024 at 23:06, Dan Sommers via Python-list wrote: > (For the history-impaired, getopt existed long before Python and will > likely exist long after it, but getopt's "replacement" optparse lasted > only from 2003 until 2011.) Depends on your definition of "lasted". It's not getting developed further, but it's still there. If you started using it, you're welcome to keep going. https://docs.python.org/3/library/optparse.html ChrisA -- https://mail.python.org/mailman/listinfo/python-list
Re: Flubbed it in the second interation through the string: range error... HOW?
On Wed, 29 May 2024 at 16:03, Cameron Simpson via Python-list
wrote:
> By which Thomas means stuff like this:
>
> print(f'if block {name[index]} and index {index}')
>
> Notice the leading "f'". Personally I wouldn't even go that far, just:
>
> print('if block', name[index], 'and index', index)
>
> But there are plenty of places where f-strings are very useful.
I wouldn't replace str.format() everywhere, nor would I replace
percent encoding everywhere - but in this case, I think Thomas is
correct. Not because it's 2024 (f-strings were brought in back in
2015, so they're hardly chronologically special), but because most of
this looks like debugging output that can take advantage of this
feature:
print(f"if block {name[index]=} {index=}")
ChrisA
--
https://mail.python.org/mailman/listinfo/python-list
Re: PyCon
On Sun, 19 May 2024 at 04:10, Abdur-Rahmaan Janhangeer via Python-list wrote: > > Yes, this year's pretty exciting, great keynotes, great lightnings, great > location, great even sponsor talks (thought they would be pumping a lot of > marketing, but the ones i went were pretty awesome technically) > > Organization side pretty smooth as well đź‘Ť I missed out on the first day's lightning talks session due to conflict with work, and it didn't seem to have been recorded. What did I miss? ChrisA -- https://mail.python.org/mailman/listinfo/python-list
Re: PyCon
On Sat, 18 May 2024 at 21:44, Skip Montanaro via Python-list wrote: > > > > > > I’m at PyCon in Pittsburgh and I’m haven’t an amazing time! > > > > s/haven’t/having/ > > > > No need to explain/correct. We understand you are excited. Many of us have > been in the same state before. ;-) > We're fluent in autocorrupt :) If you're connecting to the online parts of PyCon, I'll see you there - I'm about to hang out in the lightning talks! Geography's a bit of a barrier for me, but thanks to livestreams, I can participate too! ChrisA -- https://mail.python.org/mailman/listinfo/python-list
Re: Python Dialogs
On Tue, 7 May 2024 at 03:42, jak via Python-list wrote: > > Loris Bennett ha scritto: > > r...@zedat.fu-berlin.de (Stefan Ram) writes: > > > >>Me (indented by 2) and the chatbot (flush left). Lines lengths > 72! > > > > Is there a name for this kind of indentation, i.e. the stuff you are > > writing not being flush left? It is sort of contrary to > > what I think of as "normal" indentation. You seem to use it in all your > > postings, too, which hurts my brain, but I guess that's my problem :-) > > > > [snip (40 lines)] > > > > It's not just your problem. When the texts are particularly long and > complex, I happen to use Google-Translator. It makes bad translations if > the lines are interrupted by the newline, so I wrote a tool dedicated to > Stefan and to all those who indent like him. This tool takes the text > from the clipboard, removes all the superfluous spaces, combines all the > lines in one and puts the result back into the clipboard. :-D > Fun fact: His posts are completely irrelevant to people who follow the mailing list. Due to a dispute over permissions, his posts are blocked at the gateway. So all of us folks who use the mailing list never need to see the wonky indentation. ChrisA -- https://mail.python.org/mailman/listinfo/python-list
Re: how to discover what values produced an exception?
On Tue, 7 May 2024 at 03:38, Alan Bawden via Python-list wrote: > A good error message shouldn't withhold any information that can > _easily_ be included. Debugging is more art than science, so there is > no real way to predict what information might prove useful in solving > the crime. I emphasized "easily" because of course you have to draw the > line somewhere. Emphasis on "easily" is correct here. How easy is it to report the value of something that could be arbitrarily large? ChrisA -- https://mail.python.org/mailman/listinfo/python-list
Re: Using a background thread with asyncio/futures with flask
On Fri, 22 Mar 2024 at 18:35, Lars Liedtke via Python-list wrote: > > Hey, > > As far as I know (might be old news) flask does not support asyncio. > > You would have to use a different framework, like e.g. FastAPI or similar. > Maybe someone has already written "flask with asyncio" but I don't know about > that. > Did you try searching their documentation? https://flask.palletsprojects.com/en/3.0.x/async-await/ ChrisA -- https://mail.python.org/mailman/listinfo/python-list
Re: GIL-Removal Project Takes Another Step (Posting On Python-List Prohibited)
On Wed, 20 Mar 2024 at 18:31, Greg Ewing via Python-list wrote: > > On 20/03/24 4:14 pm, Lawrence D'Oliveiro wrote: > > not to > > mention the latency when there isn’t quite enough memory for an allocation > > and you have to wait until the next GC run to proceed. Run the GC a > > thousand times a second, and the latency is still 1 millisecond. > > That's not the way it usually works. If you run out of memory, you > run a GC there and then. You don't have to wait for GCs to occur on > a time schedule. > > Also, as a previous poster pointed out, GCs are typically scheduled > by number of allocations, not by time. > FYI you're violating someone's request by responding to them in a way that results in it getting onto python-list, so it's probably safest to just ignore cranks and trolls and let them stew in their own juices. But normally the GC doesn't need to be scheduled at all. In CPython, the only reason to "run garbage collection" is to detect cycles, so you would have to be generating inordinate amounts of cyclic garbage for this to matter at all. ChrisA -- https://mail.python.org/mailman/listinfo/python-list
Re: A Single Instance of an Object?
On Tue, 12 Mar 2024 at 08:04, Ivan "Rambius" Ivanov wrote: > > A Singleton is just a global variable. Why do this? Did someone tell > > you "global variables are bad, don't use them"? > > I have bad experience with global variables because it is hard to > track what and when modifies them. I don't consider them bad, but if I > can I avoid them. > If you have a singleton, how will you track "what and when modifies" it? How is it any different from a global? ChrisA -- https://mail.python.org/mailman/listinfo/python-list
Re: A Single Instance of an Object?
On Tue, 12 Mar 2024 at 07:54, Ivan "Rambius" Ivanov via Python-list wrote: > I am refactoring some code and I would like to get rid of a global > variable. Here is the outline: > > ... > > I have never done that in Python because I deliberately avoided such > complicated situations up to now. I know about the Singleton pattern, > but I have never implemented it in Python and I don't know if it is > Pythonish. > A Singleton is just a global variable. Why do this? Did someone tell you "global variables are bad, don't use them"? ChrisA -- https://mail.python.org/mailman/listinfo/python-list
Re: Variable scope inside and outside functions - global statement being overridden by assignation unless preceded by reference
On Sat, 9 Mar 2024 at 03:42, Grant Edwards via Python-list wrote: > > On 2024-03-08, Chris Angelico via Python-list wrote: > > On Sat, 9 Mar 2024 at 00:51, Grant Edwards via Python-list > > wrote: > > > >> One might argue that "global" isn't a good choice for what to call the > >> scope in question, since it's not global. It's limited to that source > >> file. It doesn't make sense to me to call a binding "global", when > >> there can be multile different "global" bindings of the same name. > > > > Most "globals" aren't global either, since you can have different > > globals in different running applications. > > To me, "global" has always been limited to within a single > process/address space, but that's probably just bias left over from > C/Pascal/FORTRAN/assembly/etc. It never occurred to me that a global > called "X" in one program on one computer would be the same as a > global called "X" in a different program on a different computer > somewhere else on the "globe". > Yeah. My point is, though, the name "global" is a bit of a hack anyway, so it's not THAT big a deal if it has other caveats too. For example, let's say you always "import globals" at the top of every script, and then assign "globals.x = 123" etc. Now you have a concept of globals that spans the entire application, right? Well, no, not if you use multiprocessing. So, go ahead and call them globals, but people will always have to learn about exactly what that means. ChrisA -- https://mail.python.org/mailman/listinfo/python-list
Re: Variable scope inside and outside functions - global statement being overridden by assignation unless preceded by reference
On Sat, 9 Mar 2024 at 00:51, Grant Edwards via Python-list wrote: > One might argue that "global" isn't a good choice for what to call the > scope in question, since it's not global. It's limited to that source > file. It doesn't make sense to me to call a binding "global", when > there can be multile different "global" bindings of the same name. > Most "globals" aren't global either, since you can have different globals in different running applications. ChrisA -- https://mail.python.org/mailman/listinfo/python-list
Re: Can one output something other than 'nan' for not a number values?
On Mon, 19 Feb 2024 at 06:47, Grant Edwards via Python-list wrote: > I would be tempted to try monkey-patching the float class to override > the __format__ method. I have no idea what side effects that might > have, or if it's even used by the various formatting mechanisms, so > you might end up scraping bits off the walls... > You can try, but you'd have to do it in C - the float type is immutable in Python. ChrisA -- https://mail.python.org/mailman/listinfo/python-list
Re: Is there a way to implement the ** operator on a custom object
On Fri, 9 Feb 2024 at 17:03, Cameron Simpson via Python-list wrote: > > On 08Feb2024 12:21, tony.fl...@btinternet.com > wrote: > >I know that mappings by default support the ** operator, to unpack the > >mapping into key word arguments. > > > >Has it been considered implementing a dunder method for the ** > >operator so you could unpack an object into a key word argument, and > >the developer could choose which keywords would be generated (or could > >even generate 'virtual' attributes). > > Can you show us why you think that would look like in code? > > Note that Python already has `a ** b` to raise `a` to the power of `b`, > and it has a bunder method `__pow__` which you can define. I presume this is more like: obj = SomeObject() func(**obj) ie making the object behave in a dict-like way. I can't remember how this is implemented, but you can create the necessary methods to have your object produce whatever it likes. ChrisA -- https://mail.python.org/mailman/listinfo/python-list
Re: Await expressions (Posting On Python-List Prohibited)
On Sat, 27 Jan 2024 at 11:01, Greg Ewing via Python-list wrote: > > If it helps at all, you can think of an async function as being > very similar to a generator, and "await" as being very similar to > "yield from". In the current implementation they're almost exactly > the same thing underneath. > Don't bother responding to people who say that they don't want responses on a public list. ChrisA -- https://mail.python.org/mailman/listinfo/python-list
Re: Question about garbage collection
On Wed, 17 Jan 2024 at 01:45, Frank Millman via Python-list wrote: > > On 2024-01-16 2:15 PM, Chris Angelico via Python-list wrote: > > > > Where do you tend to "leave a reference dangling somewhere"? How is > > this occurring? Is it a result of an incomplete transaction (like an > > HTTP request that never finishes), or a regular part of the operation > > of the server? > > > > I have a class that represents a database table, and another class that > represents a database column. There is a one-to-many relationship and > they maintain references to each other. > > In another part of the app, there is a class that represents a form, and > another class that represents the gui elements on the form. Again there > is a one-to-many relationship. I don't know when you'd be "done" with the table, so I won't try to give an example, but I'll try this one and maybe it'll give some ideas that could apply to both. When you open the form, you initialize it, display it, etc, etc. This presumably includes something broadly like this: class Form: def __init__(self): self.elements = [] class Element: def __init__(self, form): self.form = form form.elements.append(self) frm = Form(...) Element(frm, ...) # as many as needed frm.show() # present it to the user This is a pretty classic refloop. I don't know exactly what your setup is, but most likely it's going to look something like this. Feel free to correct me if it doesn't. The solution here would be to trap the "form is no longer being displayed" moment. That'll be some sort of GUI event like a "close" or "delete" signal. When that comes through (and maybe after doing other processing), you no longer need the form, and can dispose of it. The simplest solution here is: Empty out frm.elements. That immediately leaves the form itself as a leaf (no references to anything relevant), and the elements still refer back to it, but once nothing ELSE refers to the form, everything can be disposed of. > A gui element that represents a piece of data has to maintain a link to > its database column object. There can be a many-to-one relationship, as > there could be more than one gui element referring to the same column. Okay, so the Element also refers to the corresponding Column. If the Form and Element aren't in a refloop, this shouldn't be a problem. However, if this is the same Table and Column that you referred to above, that might be the answer to my question. Are you "done" with the Table at the same time that the form is no longer visible? If so, you would probably have something similar where the Form refers to the Table, and the Table and Columns refer to each other... so the same solution hopefully should work: wipe out the Table's list of columns. > There are added complications which I won't go into here. The bottom > line is that on some occasions a form which has been closed does not get > gc'd. > > I have been trying to reproduce the problem in my toy app, but I cannot > get it to fail. There is a clue there! I think I have just > over-complicated things. Definitely possible. > I will start with a fresh approach tomorrow. If you don't hear from me > again, you will know that I have solved it! > > Thanks for the input, it definitely helped. Cool cool, happy to help. ChrisA -- https://mail.python.org/mailman/listinfo/python-list
Re: Question about garbage collection
On Tue, 16 Jan 2024 at 23:08, Frank Millman via Python-list wrote: > > On 2024-01-15 3:51 PM, Frank Millman via Python-list wrote: > > Hi all > > > > I have read that one should not have to worry about garbage collection > > in modern versions of Python - it 'just works'. > > > > I don't want to rely on that. My app is a long-running server, with > > multiple clients logging on, doing stuff, and logging off. They can > > create many objects, some of them long-lasting. I want to be sure that > > all objects created are gc'd when the session ends. > > > > I did not explain myself very well. Sorry about that. > > My problem is that my app is quite complex, and it is easy to leave a > reference dangling somewhere which prevents an object from being gc'd. > > This can create (at least) two problems. The obvious one is a memory > leak. The second is that I sometimes need to keep a reference from a > transient object to a more permanent structure in my app. To save myself > the extra step of removing all these references when the transient > object is deleted, I make them weak references. This works, unless the > transient object is kept alive by mistake and the weak ref is never removed. > > I feel it is important to find these dangling references and fix them, > rather than wait for problems to appear in production. The only method I > can come up with is to use the 'delwatcher' class that I used in my toy > program in my original post. > > I am surprised that this issue does not crop up more often. Does nobody > else have these problems? > It really depends on how big those dangling objects are. My personal habit is to not worry about a few loose objects, by virtue of ensuring that everything either has its reference loops deliberately broken at some point in time, or by keeping things small. An example of deliberately breaking a refloop would be when I track websockets. Usually I'll tag the socket object itself with some kind of back-reference to my own state, but I also need to be able to iterate over all of my own state objects (let's say they're dictionaries for simplicity) and send a message to each socket. So there'll be a reference loop between the socket and the state. But at some point, I will be notified that the socket has been disconnected, and that's when I go to its state object and wipe out its back-reference. It can then be disposed of promptly, since there's no loop. It takes a bit of care, but in general, large state objects won't have these kinds of loops, and dangling references haven't caused me any sort of major issues in production. Where do you tend to "leave a reference dangling somewhere"? How is this occurring? Is it a result of an incomplete transaction (like an HTTP request that never finishes), or a regular part of the operation of the server? ChrisA -- https://mail.python.org/mailman/listinfo/python-list
Re: Question about garbage collection
On Tue, 16 Jan 2024 at 13:49, Akkana Peck via Python-list wrote: > > I wrote: > > > Also be warned that some modules (particularly if they're based on > > > libraries not written in Python) might not garbage collect, so you may > > > need to use other methods of cleaning up after those objects. > > Chris Angelico writes: > > Got any examples of that? > > The big one for me was gdk-pixbuf, part of GTK. When you do something like > gtk.gdk.pixbuf_new_from_file(), there's a Python object that gets created, > but there's also the underlying C code that allocates memory for the pixbuf. > When the object went out of scope, the Python object was automatically > garbage collected, but the pixbuf data leaked. Calling gc.collect() caused > the pixbuf data to be garbage collected too. > > There used to be a post explaining this on the pygtk mailing list: the link > was > http://www.daa.com.au/pipermail/pygtk/2003-December/006499.html > but that page is gone now and I can't seem to find any other archives of that > list (it's not on archive.org either). And this was from GTK2; I never > checked whether the extra gc.collect() is still necessary in GTK3, but I > figure leaving it in doesn't hurt anything. I use pixbufs in a tiled map > application, so there are a lot of small pixbufs being repeatedly read and > then deallocated. > Okay, so to clarify: the Python object will always be garbage collected correctly, but a buggy third-party module might have *external* resources (in that case, the pixbuf) that aren't properly released. Either that, or there is a reference loop, which doesn't necessarily mean you NEED to call gc.collect(), but it can help if you want to get rid of them more promptly. (Python will detect such loops at some point, but not always immediately.) But these are bugs in the module, particularly the first case, and should be considered as such. 2003 is fully two decades ago now, and I would not expect that a serious bug like that has been copied into PyGObject (the newer way of using GTK from Python). So, Python's garbage collection CAN be assumed to "just work", unless you find evidence to the contrary. ChrisA -- https://mail.python.org/mailman/listinfo/python-list
Re: Question about garbage collection
On Tue, 16 Jan 2024 at 06:32, Akkana Peck via Python-list wrote: > > > Frank Millman wrote at 2024-1-15 15:51 +0200: > > >I have read that one should not have to worry about garbage collection > > >in modern versions of Python - it 'just works'. > > Dieter Maurer via Python-list writes: > > There are still some isolated cases when not all objects > > in an unreachable cycle are destroyed > > (see e.g. step 2 of > > "https://devguide.python.org/internals/garbage-collector/index.html#destroying-unreachable-objects";). > > Also be warned that some modules (particularly if they're based on libraries > not written in Python) might not garbage collect, so you may need to use > other methods of cleaning up after those objects. > Got any examples of that? ChrisA -- https://mail.python.org/mailman/listinfo/python-list
Re: Extract lines from file, add to new files
On Mon, 15 Jan 2024 at 19:26, Greg Ewing via Python-list wrote: > > On 15/01/24 9:07 pm, Chris Angelico wrote: > > The grammar *can't* specify everything. If it did, it would have to > > have rules for combining individual letters into a NAME and individual > > characters into a string literal. > > The lexical level of a grammar can be, and often is, described > formally using regular expressions. > > Although some might consider that this doesn't contradict > your statement about readability. :-) > Not even slightly :) ChrisA -- https://mail.python.org/mailman/listinfo/python-list
Re: Extract lines from file, add to new files
On Mon, 15 Jan 2024 at 18:56, Greg Ewing via Python-list wrote: > > On 15/01/24 1:28 am, Left Right wrote: > > Python isn't a context-free language, so the grammar that is used to > > describe it doesn't actually describe the language > > Very few languages have a formal grammar that *fully* describes > the set of strings that constitute valid programs, including all > the rules about things having to be declared, types matching up, > etc. The only one I know of which attempted that is Algol 68, > and it seems to be regarded as a technical success but a practical > failure. > > > ... so, it's a "pretend grammar" that ignores indentation. > > Indentation isn't ignored, it appears in the grammar by means of > INDENT and DEDENT lexical tokens. > > It's true that the meaning of these tokens is described informally > elsewhere, but that's true of all the lexical features. > I've recently been doing a bit of work with grammar parsers, and to be quite honest, the grammar is only about one third of the overall parser. There are three sections with roughly equal importance: 1. Tokenizer 2. Grammar 3. What to DO with that grammar (actions) INDENT and DEDENT are being handled at the tokenizer stage, and so are a lot of other rules like backslashes in quoted strings. On the flip side, string prefixes (like b"...") seem to be handled in the third phase, and the grammar actually doesn't concern itself with those either. The grammar *can't* specify everything. If it did, it would have to have rules for combining individual letters into a NAME and individual characters into a string literal. The grammar would be completely unreadable. (I tried, and even just building up a decimal literal in that style was quite a pain.) ChrisA -- https://mail.python.org/mailman/listinfo/python-list
Re: Extract lines from file, add to new files
On Mon, 15 Jan 2024 at 12:42, dn via Python-list wrote: > > On 15/01/24 14:33, Chris Angelico via Python-list wrote: > > On Mon, 15 Jan 2024 at 12:12, dn via Python-list > > wrote: > >> Here's another witticism I'll often toss at trainees (in many languages, > >> and especially in UX): just because we can do it, doesn't make it a good > >> idea! > >> > > > > Programming. We were so busy with whether we COULD that we didn't stop > > to think if we SHOULD. > > > > I think that's basically my whole life. > > Don't be too hard on yourself. > > My life-advice has always been "don't do anything I wouldn't do" > - which pretty-much gives you carte blanche. > I would NEVER give that advice to anyone. They would leave the dishes unwashed, and a number of other problems :) ChrisA -- https://mail.python.org/mailman/listinfo/python-list
Re: Extract lines from file, add to new files
On Mon, 15 Jan 2024 at 12:12, dn via Python-list wrote: > Here's another witticism I'll often toss at trainees (in many languages, > and especially in UX): just because we can do it, doesn't make it a good > idea! > Programming. We were so busy with whether we COULD that we didn't stop to think if we SHOULD. I think that's basically my whole life. ChrisA -- https://mail.python.org/mailman/listinfo/python-list
Re: Extract lines from file, add to new files
On Mon, 15 Jan 2024 at 09:40, dn via Python-list wrote: > The basic challenge came from my earlier (and blasé) repetition of the > Python refrain "everything in Python is an object". Which led to: > > <<< > For example, you may say "functions in Python are > objects", but you cannot put a function definition in the head of the > for loop clause. > >>> > > Which is logical - to some degree, and in-isolation. > > for def a_function( etc )... in iterable/iterator: > > does not make sense. The 'head' (a more generic name, where Python says > "target_list", that refines down to 'something which can identify the > generated-value'. > > So, no, there's an "everything" which (might be) an object but which > cannot be used in that scenario. More accurately, every VALUE in Python is an object. This does not mean that syntax is an object. Very few languages would say that every single grammatical element is a value. Yes, it's sloppy to say "everything" is an object, but it's also rather nonintuitive to claim that, therefore, syntax elements are all objects. It's like claiming that everything that this dealership sells is a car (or "everything in this dealership is a car"), and therefore the dealership's name must itself be a car. > That said, does anyone think that something like: > > for a_function( etc ) in iterable/iterator: > > is acceptable? > - see both Python definition and (full-)quotation. > > I've not come-across a language which does allow such - YMMV/mea culpa; > and am struggling to see how it could possibly be useful. You could do something close to that: for a_function(etc)[0] in iterable: ... because an assignment target can contain an arbitrary expression followed by the subscript. > * Looking at the correspondent's email-address (cf 'handle') - and as an > unfair stereotype, raises the question of issues related to (English) > language-skills - which, arrogantly implies/assumes that native > English-speakers are all highly-capable. (?) A negative-interpretation > is to note his apparent intelligence, but wonder if failing to represent > others' comments fairly is deliberate, or carelessness. Is there an > irony in behaving/failing in such, whilst attempting to hold Python's > structure to some golden-ideal? Seems likely. ChrisA -- https://mail.python.org/mailman/listinfo/python-list
Re: Extract lines from file, add to new files
On Mon, 15 Jan 2024 at 08:15, Left Right wrote: > Python grammar rules prevent function definition from > appearing in left-hand side of the head of the for loop. However, a > variable declaration, which is also a statement, is allowed there. What is a "variable declaration" in Python? Please elaborate. ChrisA -- https://mail.python.org/mailman/listinfo/python-list
Re: Extract lines from file, add to new files
On Mon, 15 Jan 2024 at 00:27, Left Right wrote: > > > What do you mean? > > > > for x in lambda: ...: > > ... > > > > Perfectly grammatical. > > 1. You put the lambda definition in the wrong place (it should be in > the left-hand side, or as Python calls it "star_targets", but you put > it into "star_expressions", which would be where the right-hand side > is drawn from). > 2. You used what Python calls "lambdadef" in place of what Python > calls "function_def". I.e. lambda definition and function definition > are two different things, at least as far as grammar is considered. > > So, you solved a different problem. You said function. I made a function. You said "head of a for loop clause". I put it there. Problem was underspecified. But if you're trying to tell me that a def statement should be a valid assignment target, I don't know what you're smoking, but I want you to keep it a long way away from me. Can you name ANY language in which that would make the slightest bit of sense? ChrisA -- https://mail.python.org/mailman/listinfo/python-list
Re: Extract lines from file, add to new files
On Sun, 14 Jan 2024 at 23:28, Left Right wrote: > Having worked with a bunch of different grammar languages, the one > used for Python isn't a recognizable BNF derivative. That might possibly be because it isn't? It's not BNF. It's PEG. Or are you a long way behind the times? > For example, you may say "functions in Python are > objects", but you cannot put a function definition in the head of the > for loop clause. What do you mean? for x in lambda: ...: ... Perfectly grammatical. ChrisA -- https://mail.python.org/mailman/listinfo/python-list
Re: Extract lines from file, add to new files
On Sun, 14 Jan 2024 at 14:43, dn via Python-list wrote: > Similarly, whilst we could write: > > a, b, c = 1, 2, 3 > I would only do this when it aligns particularly well with the algorithm being implemented. For example, you could start a Fibonacci evaluator with "a, b = 0, 1". Otherwise, there's not all that much reason to unpack three constants in this way. (Though I am much more likely to use multiple initialization to set a bunch of things to the SAME value, lilke "a = b = c = 0".) -- https://mail.python.org/mailman/listinfo/python-list
Re: Extract lines from file, add to new files
On Sat, 13 Jan 2024 at 13:11, Left Right via Python-list wrote: > > Very few > languages allow arbitrary complex expressions in the same place they > allow variable introduction. What do you mean by this? Most languages I've worked with allow variables to be initialized with arbitrary expressions, and a lot of languages allow narrowly-scoped variables. ChrisA -- https://mail.python.org/mailman/listinfo/python-list
Re: Extract lines from file, add to new files
On Fri, 12 Jan 2024 at 08:56, Left Right via Python-list wrote: > > By the way, in an attempt to golf this problem, I discovered this, > which seems like a parser problem: When you jump immediately to "this is a bug", all you do is make yourself look like an idiot. Unsurprisingly, this is NOT a bug, this is simply that you didn't understand what was going on. The grammar isn't easy to read, and it's usually better to read the documentation instead. (Plus, golfing isn't really a goal in Python, and you didn't shorten the code by much at all. Good job.) ChrisA -- https://mail.python.org/mailman/listinfo/python-list
Re: Aw: Re: mypy question
On Sun, 31 Dec 2023 at 03:38, Thomas Passin via Python-list wrote: > I am not very expert in Python type hints. In working up the example > program I just posted, I got an error message from mypy that remarked > that "list" is invariant, and to try Sequence which is "covariant". I > don't know what that means (and I haven't looked into it yet), but when > I changed from list to Sequence as suggested, mypy stopped complaining. > Ah, I think you've hit on the problem there. Consider this: def add_item(stuff: dict[str: str | int]): stuff["spam"] = "ham" stuff["vooom"] = 1_000_000 Is it valid to pass this function a dict[str: str]? No, because it's going to add an integer into it. Hopefully that helps explain what's going on a bit. ChrisA -- https://mail.python.org/mailman/listinfo/python-list
Re: Python 3.12.1, Windows 11: shebang line #!/usr/bin/env python3 doesn't work any more
On Sat, 30 Dec 2023 at 14:06, Mike Dewhirst via Python-list wrote: > > On 29/12/2023 12:09 pm, FĂ©lix An via Python-list wrote: > > On 2023-12-25 12:36, Mike Dewhirst wrote: > >> > >> 3. You cannot trust Microsoft. You can trust Python Software > >> Foundation. Python from PSF works the same in all environments - or > >> if not it is a bug. Python from Microsoft is tweaked to satisfy their > >> aforementioned strategy of locking in users to Windows. > >> > > > > I strongly disagree with this. > > Not sure which part of the above you strongly disagree with. I might > seem a bit OTT with "You cannot trust Microsoft" but I did put it in a > specific context. > > PSF does try to make Python work identically in all operating systems it > supports. The OP was using py.exe which I discovered (just now - and it > is why I'm writing this) exists on my Windows 10 machine. I have never > installed any Python other than personally downloaded from the > python.org website - therefore py.exe must have come from PSF! > > I had assumed the OP had installed Python from the Microsoft shop and > that's where py.exe must have come from. > > I learn something every day. If you'd done a little research, you might have found this: https://peps.python.org/pep-0397/ Yes, it is from the official launcher. I don't see why you'd expect Microsoft to go to the effort of building their own launcher when it can be available for every Python user. > > I don't get all the irrational hate for Microsoft that exists within > > the Linux community. > > Perhaps you are too young to remember when Steve Ballmer was head of > Microsoft? How relevant is this? I remember the days when OS/2 was the big target, and Microsoft was the greatest anti-OS/2 voice out there. Does that affect the way I use tools today? Is it even slightly relevant? I remember when Intel had floating-point division issues in their FPUs. Do I tell people "don't do floating-point math on Intel processors"? Is it relevant? ChrisA -- https://mail.python.org/mailman/listinfo/python-list
Re: What is Install-Paths-To in WHEEL file?
On Sat, 30 Dec 2023 at 06:58, Left Right wrote: > My understanding is that "welcome and encourage participation by > everyone" is in stark contradiction to banning someone disagreeing > with you. Then your understanding is flat-out wrong. Encouraging participation by everyone DOES mean deleting what is unproductive, offensive, and likely to discourage participation. Your entire argument is based on misconceptions. Go play in your own sandbox somewhere, see if you can make something where everyone is welcome, including the toxic AND the people who dislike toxicity. ChrisA -- https://mail.python.org/mailman/listinfo/python-list
Re: What is Install-Paths-To in WHEEL file?
On Sat, 30 Dec 2023 at 01:37, Left Right wrote: > > > Yeah, because you have the God-given RIGHT to be able to say anything > > you like, on anyone's web site, and nobody's allowed to delete > > anything you say! That's how it goes, right? > > I don't believe in god, and I don't believe he / she can give me > rights. What I believe in is that Python is a public good, and its > status is enshrined in the license it uses. Is it? I'm not a lawyer, but I really don't think that that's what the license entitles you to. Can you quote the relevant parts of it? > > Don't let the door hit you on the way out. > > Oh, great. Here we go again. You don't even know what this discussion > is about, but decided to be rude. Oh trust me, I saw the discussion previously. I know what it is about. And when it comes to rudeness, let's just say, you reap what you sow. ChrisA -- https://mail.python.org/mailman/listinfo/python-list
Re: What is Install-Paths-To in WHEEL file?
On Sat, 30 Dec 2023 at 01:16, Left Right via Python-list wrote: > > That's not the discussion that was toxic. But the one that was -- > doesn't exist anymore since the forum owners deleted it. > > The part where the forum owners delete whatever they disagree with is > the toxic part. Yeah, because you have the God-given RIGHT to be able to say anything you like, on anyone's web site, and nobody's allowed to delete anything you say! That's how it goes, right? You're most welcome to avoid the Python Discourse if you dislike moderated forums, but do be aware that python-list is ALSO moderated, and that completely unmoderated forums are far far more toxic than anything I've seen on the Python Discourse. Don't let the door hit you on the way out. ChrisA -- https://mail.python.org/mailman/listinfo/python-list
Re: Python 3.12.1, Windows 11: shebang line #!/usr/bin/env python3 doesn't work any more
On Fri, 29 Dec 2023 at 12:23, FĂ©lix An via Python-list wrote: > > On 2023-12-25 12:36, Mike Dewhirst wrote: > > > > 3. You cannot trust Microsoft. You can trust Python Software Foundation. > > Python from PSF works the same in all environments - or if not it is a bug. > > Python from Microsoft is tweaked to satisfy their aforementioned strategy > > of locking in users to Windows. > > > > I strongly disagree with this. I don't get all the irrational hate for > Microsoft that exists within the Linux community. It's worth noting that Mike Dewhirst is NOT a spokesman for the Linux community. One of the cool benefits freedom brings is that anyone's allowed to be wrong :) Not ALL of us hate Microsoft. I store the vast majority of my code on GitHub, and it didn't make any difference when MS bought that company (it was already a company, and their interests were always to make money, and that was okay with me). ChrisA -- https://mail.python.org/mailman/listinfo/python-list
Re: >>> %matplotlib inline results in SyntaxError: invalid syntax
On Tue, 26 Dec 2023 at 07:27, Chris Grace via Python-list wrote: > I'd also recommend a newer version of python. Python 3.4 reached end of > life almost 5 years ago. Uhh, putting this in perspective... until a spammer revived the thread just now, it was asked, answered, and finished with, all back in 2016. I don't think anyone's still waiting on answers here. ChrisA -- https://mail.python.org/mailman/listinfo/python-list
Re: Python 3.12.1, Windows 11: shebang line #!/usr/bin/env python3 doesn't work any more
On Mon, 25 Dec 2023 at 15:42, Mike Dewhirst via Python-list wrote: > > Apologies for top posting - my phone seems unable to do otherwise. > > Here's my view - which may not be popular. You're right about that part, anyhow :) > 4. Shebang lines are pretty much redundant now that most python interpreters > are run from venvs Strongly dispute that. The rest. you're entitled to your opinions (they happen to be wrong, but you're entitled to them :) ), but this is a statement of fact that I would need to see some evidence for. ChrisA -- https://mail.python.org/mailman/listinfo/python-list
Re: argparse argument post-processing
On Mon, 27 Nov 2023 at 22:31, Dom Grigonis via Python-list
wrote:
>
> Hi all,
>
> I have a situation, maybe someone can give some insight.
>
> Say I want to have input which is comma separated array (e.g.
> paths='path1,path2,path3') and convert it to the desired output - list:
This is a single argument.
> Now the second case. I want input to be space separated array - bash array.
> And I want space-separated string returned. My current approach is:
> import argparse
> parser = argparse.ArgumentParser()
> parser.add_argument('paths', nargs='+')
> args = parser.parse_args()
> paths = ' '.join(args.paths)
> But what I am looking for is a way to do this, which is intrinsic to
> `argparse` module. Reason being I have a fair amount of such cases and I
> don’t want to do post-processing, where post-post-processing happens (after
> `parser.parse_args()`).
>
This is parsing multiple arguments.
If you want it space-separated, you can do that, just as a single
argument. Otherwise, what you're doing is taking multiple arguments
and joining them. I'm not sure what you expect to see, but your
examples here pretty clearly show that you are taking multiple
arguments, and joining them with spaces.
ChrisA
--
https://mail.python.org/mailman/listinfo/python-list
Re: Newline (NuBe Question)
On Mon, 27 Nov 2023 at 13:52, AVI GROSS via Python-list wrote: > Be that as it > may, and I have no interest in this topic, in the future I may use the ever > popular names of Primus, Secundus and Tertius and get blamed for using > Latin. > Imperious Prima flashes forth her edict to "begin it". In gentler tone Secunda hopes there will be nonsense in it. While Tertia interrupts the tale not more than once a minute. ChrisA -- https://mail.python.org/mailman/listinfo/python-list
Re: RE: Newline (NuBe Question)
On Mon, 27 Nov 2023 at 06:15, wrote: > But I learn from criticism. If I ever write a program like that and do not > feel like typing, will this do? > > dents = [ ...] > > Or will that not include students who happen to be edentulous? > If they're learning to drive, this variable name would make complete sense. ChrisA -- https://mail.python.org/mailman/listinfo/python-list
Re: RE: Newline (NuBe Question)
On Sun, 26 Nov 2023 at 21:08, Michael F. Stemper via Python-list wrote: > > On 24/11/2023 21.45, avi.e.gr...@gmail.com wrote: > > Grizz[l]y, > > > > I think the point is not about a sorted list or sorting in general It is > > about reasons why maintaining a data structure such as a list in a program > > can be useful beyond printing things once. There are many possible examples > > such as having a list of lists containing a record where the third item is a > > GPA for the student and writing a little list comprehension that selects a > > smaller list containing only students who are Magna Cum Laude or Summa Cum > > Laude. > > > > studs = [ > >["Peter", 82, 3.53], > >["Paul", 77, 2.83], > >["Mary", 103, 3.82] > > ] > > I've seen Mary, and she didn't look like a "stud" to me. > That's what happens when you abbreviate "student" though :) Don't worry, there's far FAR worse around the place, and juvenile brains will always find things to snigger at, usually in mathematical libraries with "cumulative" functions. ChrisA -- https://mail.python.org/mailman/listinfo/python-list
Re: xor operator
On Tue, 14 Nov 2023 at 11:40, Chris Angelico wrote: > Here's a couple of excellent videos on error correction, and you'll > see XOR showing up as a crucial feature: > > https://www.youtube.com/watch?v=X8jsijhllIA > https://www.youtube.com/watch?v=h0jloehRKas > I just flipped through that 3blue1brown video again, and as it turns out, the actual part where he talks about multi-argument XOR happens in part two (it's barely touched on at the very end as he's introducing part two). https://www.youtube.com/watch?v=b3NxrZOu_CE Honestly though, the full video is well worth watching. More modern error correction systems are slightly different, but will still be built on many of the same principles. ChrisA -- https://mail.python.org/mailman/listinfo/python-list
Re: xor operator
On Tue, 14 Nov 2023 at 12:02, Dom Grigonis via Python-list wrote: > As I am here, I will dare to ask if there is no way that `sign` function is > going to be added to `math` or `builtins`. > https://docs.python.org/3/library/math.html#math.copysign ChrisA -- https://mail.python.org/mailman/listinfo/python-list
Re: xor operator
On Tue, 14 Nov 2023 at 11:29, Dom Grigonis via Python-list wrote: > > > > Except the 'any' and 'all' builtins are _exactly_ the same as bitwise > > or and and applided to many bits. To do something "in line" with that > > using the 'xor' operator would return True for an odd number of True > > values and False for an even Number of True values. > > Fair point. > > Have you ever encountered the need for xor for many bits (the one that I am > NOT referring to)? Would be interested in what sort of case it could be > useful. Yes, parity checking. That's why I namedropped that in my prior response - it's an easy thing to search the web for if you want to know more about it. Here's a couple of excellent videos on error correction, and you'll see XOR showing up as a crucial feature: https://www.youtube.com/watch?v=X8jsijhllIA https://www.youtube.com/watch?v=h0jloehRKas ChrisA -- https://mail.python.org/mailman/listinfo/python-list
Re: xor operator
On Tue, 14 Nov 2023 at 10:00, Dom Grigonis via Python-list wrote: > > I am not asking. Just inquiring if the function that I described could be > useful for more people. > > Which is: a function with API that of `all` and `any` and returns `True` if > specified number of elements is True. > > It is not a generalised `xor` in strict programatic space. I.e. NOT bitwise > xor applied to many bits. > This is more in line with cases that `any` and `all` builtins are used. > A generalization of XOR is exactly what Grant and I said, though: a parity check. See for example: https://en.wikipedia.org/wiki/Exclusive_or https://reference.wolfram.com/language/ref/Xor.html It tells you whether you have an odd or even number of true values. Now, if you want something that short-circuits a counting function, that's definitely doable, but it's a sum-and-compare, not xor. Also, it's quite specialized so it's unlikely to end up in the stdlib. ChrisA -- https://mail.python.org/mailman/listinfo/python-list
Re: xor operator
On Tue, 14 Nov 2023 at 08:57, Axel Reichert via Python-list wrote: > > Barry writes: > > > I do not understand how xor(iterator) works. > > I thought xor takes exactly 2 args. > > See > > https://mathworld.wolfram.com/XOR.html > > for some background (I was not aware of any generalizations for more > than 2 arguments either). > Generalization to more arguments usually means calculating parity - that is, it tells you whether you have an odd or even number of true results. Which means that short-circuiting is nonsensical. ChrisA -- https://mail.python.org/mailman/listinfo/python-list
Re: Beep on WIndows 11
On Mon, 13 Nov 2023 at 04:13, MRAB via Python-list wrote: > In the old days, with a BBC micro, that was simple. It had 3 tone > channels and 1 white noise channel, with control over frequency, > duration and volume, beeps on different channels could be synchronised > to start at the same time, there was a sound queue so that the SOUND > command returned immediately, and there was an ENVELOPE command for > controlling the attack, decay, sustain and release. All this on an 8-bit > machine! > > My current PC is way more powerful. 64-bit processor, GBs of RAM, etc. > Python offers winsound.Beep. 1 tone, no volume control, and it blocks > while beeping. I learned to make a computer beep using the programmable timer chip (8254?). Send it signals on the I/O port saying "timer chip, speaker signal, counter = N" (where N is the fundamental clock divided by the frequency I wanted), and then "speaker, respond to timer chip". Then you wait the right length of time, then send "speaker, stop responding to timer chip". Of course, I had a bug in the "wait the right length of time" part, and my program stopped running. My family was not amused. ChrisA -- https://mail.python.org/mailman/listinfo/python-list
Re: Beep on WIndows 11
On Sun, 12 Nov 2023 at 21:27, Y Y via Python-list wrote: > > I am curious and humble to ask: What is the purpose of a BEEP? > There are several purposes. I can't say which of these are relevant to the OP, but some or all of them could easily be. * A very very simple notification that can be triggered by any program * An audio cue that is not routed to your regular audio system (good if you use headphones but are AFK) * An extremely low level signal that requires little-to-no processing power * An emergency signal that does not even require a CPU (probably not in this instance though!) * Imitating a pre-existing audio signal that works by beeping Depending on what's needed, a more complex system might suffice (for example, I cover the first two points by having an entire separate audio subsystem with its own dedicated speakers, which I can invoke using VLC in a specific configuration); but a basic beep is definitely of value. I suspect in this situation that the first point is important here, but it's up to the OP to elaborate. (Note that the "no CPU emergency sound" option usually requires a motherboard-mounted speaker or speaker header, which not all have these days. Sad.) ChrisA -- https://mail.python.org/mailman/listinfo/python-list
Re: Checking if email is valid
On Tue, 7 Nov 2023 at 10:11, Greg Ewing via Python-list wrote: > > On 7/11/23 7:45 am, Mats Wichmann wrote: > > Continuing with the example, if you have a single phone number field, or > > let a mobile number be entered in a field marked for landline, you will > > probably assume you can text to that number. > > But if the site can detect that you've entered a mobile number into > the landline field or vice versa and reject it, then it can figure out > whether it can text to a given numner or not without you having > to tell it! > Oh yes, it totally can. Never mind that some mobile numbers are able to accept text messages but not calls, nor that some landline numbers can accept text messages as well as calls :) ChrisA -- https://mail.python.org/mailman/listinfo/python-list
Re: Checking if email is valid
On Tue, 7 Nov 2023 at 07:10, Mats Wichmann via Python-list wrote: > Suggests maybe labeling should be something like: > > * Number you want to be called on > * Number for texted security messages, if different > > Never seen that, though :-) > My responses would be: * Don't. * That's what cryptographic keys are for. :) ChrisA -- https://mail.python.org/mailman/listinfo/python-list
Re: Checking if email is valid
On Tue, 7 Nov 2023 at 02:05, Jon Ribbens via Python-list wrote: > That was another thing that I used to find ridiculous, but seems to have > improved somewhat in recent years - website error pages that said "please > contact us to let us know about this error". I'm sorry, what? You want > me to contact you to tell you about what your own website is doing? How > does that make any sense? Websites should be self-reporting problems. Actually, I think those serve a very important purpose. The reports are almost certainly being discarded unread; the value of such a reporting system is to give the user the sensation that they've "done something" to "help". It makes some people feel better about running into a bug. But you're right, they serve little purpose in solving web site problems. ChrisA -- https://mail.python.org/mailman/listinfo/python-list
Re: Checking if email is valid
On Fri, 3 Nov 2023 at 12:21, AVI GROSS via Python-list wrote: > My guess is that a first test of an email address might be to see if a decent > module of that kind fills out the object to your satisfaction. You can then > perhaps test parts of the object, rather than everything at once, to see if > it is obviously invalid. As an example, what does u...@alpha...com with > what seems to be lots of meaningless periods, get parsed into? > What do you mean by "obviously invalid"? Have you read the RFC? ChrisA -- https://mail.python.org/mailman/listinfo/python-list
Re: Checking if email is valid
On Fri, 3 Nov 2023 at 07:17, Jon Ribbens via Python-list wrote: > > On 2023-11-02, Simon Connah wrote: > > Valid as in conforms to the standard. Although having looked at the > > standard that might be more difficult than originally planned. > > Yes. Almost nobody actually implements "the standard" as in RFC 2822 > section 3.4.1 (which can contain, for example, non-printable control > characters, and comments), nor is it particularly clear that they > should. So while checking against "the spec" might sound right, it's > highly unlikely that it's what you actually want. Would you really > want to allow: > > (jam today) "chris @ \"home\""@ (Chris's host.)public.example > > for example? And would you be able to do anything with it if you did? If by checking against the spec you mean "sending an email to it with a code or magic link", then sure, allow that! It's still short enough to fit on one line, even. Seems fine to me. Of course, since that one is in the .example TLD, it's not actually going to succeed, but now I'm curious whether you could craft a mail server that mandates the Queen's rule of "jam tomorrow, jam yesterday, but never jam today". That part is technically a comment, but it's a clear violation of a royal decree, so that should cause the email to bounce. It's jam every OTHER day, and today isn't any OTHER day. ChrisA -- https://mail.python.org/mailman/listinfo/python-list
Re: Checking if email is valid
On Thu, 2 Nov 2023 at 17:47, Cameron Simpson via Python-list wrote: > > On 02Nov2023 17:04, Chris Angelico wrote: > >On Thu, 2 Nov 2023 at 15:20, AVI GROSS via Python-list > > wrote: > >> Yes, it would be nice if there was a syntax for sending a test > >> message sort > >> of like an ACK that is not delivered to the recipient but merely results in > >> some status being sent back such as DELIVERABLE or NO SUCH USER or even > >> MAILBOX FULL. > > > >Yes, it would! Spammers would be able to use this syntax to figure out > >exactly which addresses actually have real people connected to it. It > >would save them so much trouble! Brilliant idea. > > Hmm. IIRC... > > https://datatracker.ietf.org/doc/html/rfc2821#section-4.1.1.6 > > I think a lot of mail receivers don't honour this one, for exactly the > reasons above. Yeah, and it also won't tell you if the mailbox is full, or unattended, or if the email would be rejected or discarded for any other reason. Which means it's not even all that useful if it IS implemented. ChrisA -- https://mail.python.org/mailman/listinfo/python-list
Re: Checking if email is valid
On Thu, 2 Nov 2023 at 17:47, Simon Connah wrote: > > My goal is to make a simple mailing list platform. I guess I could just send > email to an address and if it bounces then I can remove it from the database. > Thing is I'm not sure how close to a real email address an email has to be in > order to be bounced. If it was completely wrong it might just swallowed up. > Every address is completely separate. There is no "closeness". Just send email to an address. ChrisA -- https://mail.python.org/mailman/listinfo/python-list
Re: Checking if email is valid
On Thu, 2 Nov 2023 at 15:20, AVI GROSS via Python-list wrote: > > Yes, it would be nice if there was a syntax for sending a test message sort > of like an ACK that is not delivered to the recipient but merely results in > some status being sent back such as DELIVERABLE or NO SUCH USER or even > MAILBOX FULL. > Yes, it would! Spammers would be able to use this syntax to figure out exactly which addresses actually have real people connected to it. It would save them so much trouble! Brilliant idea. ChrisA -- https://mail.python.org/mailman/listinfo/python-list
Re: Checking if email is valid
On Thu, 2 Nov 2023 at 08:52, Grant Edwards via Python-list wrote: > > On 2023-11-01, Chris Angelico via Python-list wrote: > > On Thu, 2 Nov 2023 at 08:09, Grant Edwards via Python-list > > wrote: > > >> Make sure it has an '@' in it. Possibly require at least one '.' > >> after the '@'. > > > > No guarantee that there'll be a dot after the at. > > Ah, I forgot about defaulting to a local domain if one is > omitted. Will MTAs do that these days? Yeah they will; but that'll depend on the exact server you send to, whereas if you have a domain part, all you need is some server that accepts mail for forwarding. > > (Technically there's no guarantee of an at sign either, but email > > addresses without at signs are local-only, so in many contexts, you > > can assume there needs to be an at.) > > > > So the regex to match all valid email addresses that aren't > > local-only is... drumroll please... > > > > r"@" > > Unless you want to support UUCP or X400 addresses... > > :) Yyyyeah I think we can assume SMTP these days :) ChrisA -- https://mail.python.org/mailman/listinfo/python-list
Re: Checking if email is valid
On Thu, 2 Nov 2023 at 08:09, Grant Edwards via Python-list wrote: > Make sure it has an '@' in it. Possibly require at least one '.' > after the '@'. No guarantee that there'll be a dot after the at. (Technically there's no guarantee of an at sign either, but email addresses without at signs are local-only, so in many contexts, you can assume there needs to be an at.) So the regex to match all valid email addresses that aren't local-only is... drumroll please... r"@" ChrisA -- https://mail.python.org/mailman/listinfo/python-list
Re: Checking if email is valid
On Thu, 2 Nov 2023 at 06:02, Jon Ribbens via Python-list wrote: > > On 2023-11-01, Chris Angelico wrote: > > On Thu, 2 Nov 2023 at 05:21, Simon Connah via Python-list > > wrote: > >> Could someone push me in the right direction please? I just want to > >> find out if a string is a valid email address. > > > > There is only one way to know that a string is a valid email address, > > and that's to send an email to it. > > > > What is your goal though? For example, if you're trying to autolink > > email addresses in text, you don't really care whether it's valid, > > only that it looks like an address. > > There's often value in even only partially-effective checks though. > With an email address you can easily check to see if it has an "@", > and if the stuff after the "@" is a syntactically valid domain name. > You can also go a bit further and check to see if the domain has an > MX record, and if it doesn't then it is extremely unlikely that the > address is valid. Yeah, which is why I asked about the goal. (You may also note that I worded the prior statement very carefully: You cannot know that it IS valid without sending email to it, but there can be ways to know that it CANNOT BE valid.) In the vast majority of contexts, local addresses can be ignored, so an email address will have to include an at sign. I wouldn't bother with a syntax check on the domain portion, though; just check for a couple of possible formats (IP literal), and if it looks like a domain name, do the MX lookup. That said, though, there are certain contexts where you can be a LOT more restrictive, such as the autolinking example I mentioned; it's fine to exclude some unusual email addresses when all you're doing is offering a small convenience. ChrisA -- https://mail.python.org/mailman/listinfo/python-list
Re: Checking if email is valid
On Thu, 2 Nov 2023 at 05:21, Simon Connah via Python-list wrote: > > Could someone push me in the right direction please? I just want to find out > if a string is a valid email address. There is only one way to know that a string is a valid email address, and that's to send an email to it. What is your goal though? For example, if you're trying to autolink email addresses in text, you don't really care whether it's valid, only that it looks like an address. ChrisA -- https://mail.python.org/mailman/listinfo/python-list
Re: Question(s)
On Wed, 25 Oct 2023 at 22:46, o1bigtenor via Python-list wrote: > > On Wed, Oct 25, 2023 at 6:24 AM Dieter Maurer wrote: > > > > o1bigtenor wrote at 2023-10-24 07:22 -0500: > > > ... > > >Is there a way to verify that a program is going to do what it is > > >supposed to do even > > >before all the hardware has been assembled and installed and tested? > > > > Others have already noted that "verify" is a very strong aim. > > I have worked in environments where everything was 100% tested. Errors > were either corrected or one's attendance was uninvited. Powerful impetus > to do a good job. Or powerful impetus to deny that the error was yours. Remember, 100% test coverage does NOT mean the code is perfect. It just means it's tested. ChrisA -- https://mail.python.org/mailman/listinfo/python-list
Re: Question(s)
On Wed, 25 Oct 2023 at 21:53, o1bigtenor wrote: > > Hmm - - - - now how can I combine 'Hamming codes' > and a raid array? > > TIA Normally you wouldn't. But let's say you're worried that a file might get randomly damaged. (I don't think single-bit errors are really a significant issue with mass storage, as you'd be more likely to have an entire sector unreadable, but this can certainly happen in transmission.) What you do is take a set of data bits, add an error correction code, and send them on their way. The more data bits per block, the more efficient, but if there are too many errors you will lose data. So there's a tradeoff. ChrisA -- https://mail.python.org/mailman/listinfo/python-list
Re: Question(s)
On Wed, 25 Oct 2023 at 21:46, o1bigtenor wrote: > > 2. Catch the failure as you save. We have a lot of tools that can help > > you to spot bugs. > > Tools like this for python please. Various ones. Type checkers like MyPy fall into this category if you set your system up to run them when you save. Some editors do basic syntax checks too. > > 3. Catch the failure before you commit and push. Unit tests are great for > > this. > > Where might I find such please. The same tools, but run as a pre-commit hook. Any tool that can help you find bugs has the potential to be of value. It's all a question of how much time it saves with earlier detection of bugs versus how much it costs you in pacifying the tool. Some are better than others. ChrisA -- https://mail.python.org/mailman/listinfo/python-list
Re: Simple webserver
On Wed, 25 Oct 2023 at 19:00, Frank Millman via Python-list wrote: > 2. Instead of running as a stand-alone server, run my app as a > reverse-proxy using Nginx. I tested this a few years ago using Apache, > and it 'just worked', so I am fairly sure that it will work with Nginx > as well. Nginx can then provide the additional functionality that Dieter > has mentioned. This, I would recommend. In general, tools like Django and Flask and such are aimed at the creation of web apps, but not necessarily web servers; it's convenient to have a high-performance web server like Nginx or Apache, and then it passes along requests to the application server. This provides a number of benefits: 1. Static files can be served directly, without involving the app 2. Need to scale horizontally? Run two or more copies of your app and have the web server share requests among them. 3. App crashed? The web server can return an HTTP 503 error so end users aren't left wondering what's going on. 4. DOS protection can be done in the web server (although it could also be done in a firewall, or any other level that's appropriate) > My main concern is that, if I do release my app, I want it to be taken > seriously and not dismissed as 'Mickey Mouse'. Do you think the above > changes would assist with that? You shouldn't be dismissed as Still-In-Copyright-Decades-Old-Rodent even if you don't make those changes. I've been hosting a number of servers, some fairly home-grade, and nobody's ever told me that it looks bad. If you want to be respected, the main thing is to have something that people find interesting - everything else is secondary. But there are a few points to consider: * Performance. People respect something that's snappy and responsive more than they respect something where a single HTTP request takes anywhere from 2.89 seconds to nearly 30 seconds. And no, I'm totally not still mindblown at having seen this kind of appalling performance from a published and very expensive API. * Have your own domain name. https://rosuav.github.io/AntiSocialMedia/ is a toy; https://devicat.art/ is a professional web site. (They're not related, and the first one really is just a toy that I whipped up in like half an hour.) * Have a proper SSL certificate. It looks *really bad* to have a broken or outdated certificate (or none at all, these days). LetsEncrypt can do that for you, no charge. * Put some effort into styling. Yeah, I know, most of my web sites are ugly, so I shouldn't talk. But things definitely look more professional if you take the time to style them up a bit. > When I talk about releasing it, it is already available on Github here - > https://github.com/FrankMillman/AccInABox. > > You are welcome to look at it, but it needs a lot of tidying up before > it will be ready for a wider audience. Cool! This is particularly of note to me personally. Back in the 90s, I was working in the family business, and we used a fairly clunky piece of professionally-written software (and if you want stories, ask me about overnight runs of report generation, or Ctrl-Alt-Shift and old mouse pedal importing, or 32-bit installers for 16-bit applications, or a bunch of other things). There was, for a while, a theory of us designing our own accounting system, but it turns out, that's a really REALLY big job, and it's less effort to keep using the thing you already have. Your README, dating from 9 years ago, says that you support/require Python 3.4 - that's kinda ancient now. If you want to take advantage of newer features, I think you should be safe bumping that up a long way. :) I wouldn't recommend sqlite3 for any production work here, but it's good for a demo. Postgres is a far better choice if you're going to be storing your vital information in this. You can list vital packages in a file called requirements.txt - this will be recognized by both people and automated tooling. huh. I'm listed as a contributor. I'll be quite honest, I do not remember this, but presumably you shared this previously! Looks like all I contributed was a minor suggestion and commit, but still, I have absolutely no memory. LOL. Looks pretty good there. I don't have time right now to download and install it for a proper test, but based on flipping through the code, looks like you have something decent going on. ChrisA -- https://mail.python.org/mailman/listinfo/python-list
Re: How to sort this without 'cmp=' in python 3?
On Wed, 25 Oct 2023 at 13:02, Mike H via Python-list wrote: > Is it possible to use lambda expression instead of defining a `Key` class? > Something like `sorted(my_list, key = lambda x, y: x+y > y+x)`? Look up functools.cmp_to_key. ChrisA -- https://mail.python.org/mailman/listinfo/python-list
Re: Question(s)
On Wed, 25 Oct 2023 at 12:20, AVI GROSS via Python-list wrote: > Consider an example of bit rot. I mean what if your CPU or hard disk has a > location where you can write a byte and read it back multiple times and > sometimes get the wrong result. To be really cautions, you might need your > software to write something in multiple locations and when it reads it back > in, check all of them and if most agree, ignore the one or two that don't > while blocking that memory area off and moving your data elsewhere. Or > consider a memory leak that happens rarely but if a program runs for years or > decades, may end up causing an unanticipated error. > True, but there are FAR more efficient ways to do error correction :) Hamming codes give you single-bit correction and two-bit detection at a cost of log N bits, which is incredibly cheap - even if you were to go for a granularity of 64 bytes (one cache line in a modern Intel CPU), you would need just 11 bits of Hamming code for every 512 bits of data and you can guarantee to fix any single-bit error in any cache line. The "if most agree, ignore the one or two that don't" design implies that you're writing to an absolute minimum of three places, and in order to be able to ignore two that disagree, you'd probably need five copies of everything - that is to say, to store 512 bits of data, you would need 2560 bits of storage. But with a Hamming code, you need just 523 bits to store 512 reliably. Here's a great run-down on how efficiently this can be done, and how easily. https://www.youtube.com/watch?v=X8jsijhllIA Side note: If we assume that random bit flips occur at a rate of one every X storage bits, having redundant copies of data will increase the chances of a failure happening. For example, using a naive and horrendously wasteful "store 256 copies of everything" strategy, you would be 256 times more likely to have a random bitflip, which is insane :) You would also be able to guarantee detection of up to 128 random bitflips. But as you can see, this puts a maximum on your storage ratio. ChrisA -- https://mail.python.org/mailman/listinfo/python-list
Re: Question(s)
On Wed, 25 Oct 2023 at 12:11, Thomas Passin via Python-list wrote: > This doesn't mean that no program can ever be proven to halt, nor that > no program can never be proven correct by formal means. Will your > program be one of those? The answer may never come ... Indeed, and I would go further and say that, in any non-trivial system, it is impossible to completely 100% prove that it is perfectly correct. Sometimes you might have perfect mathematics and software, but only subject to certain assumptions about the environment. Or about the users. More commonly, you build a system so that failure becomes vanishingly unlikely. Take space flight as an example. Computers have been vital to the safety of human lives in space pretty much since humans have been going to space at all. How do you make sure that the Apollo Guidance Computer works correctly when you need it to? Multiple layers of protection. Error correcting memory, redundant systems, and human monitoring, plus the ability to rewrite the guidance software on the fly if they needed to. Even when people are being sent to the moon, you can't completely guarantee that the software is perfect, so you add other layers to give greater protection. (And more recently, both India's "Chandrayaan 2" and Japan's "Hakuto-R" unmanned moon missions crash-landed due to software issues. A half century of improvements hasn't changed the fundamental fact that building a perfect system is basically impossible.) So is all hope lost? No. We learn from our mistakes, we add more layers. And ultimately, we test until we're reasonably confident, and then go with it, knowing that failures WILL happen. Your goal as a programmer isn't to prevent failure altogether - if it were, you would never be able to achieve anything. Your goal is to catch those failures before they cause major issues. 1. Catch the failure as you're typing in code. Done, fixed, that's what the Backspace key is for. 2. Catch the failure as you save. We have a lot of tools that can help you to spot bugs. 3. Catch the failure before you commit and push. Unit tests are great for this. 4. Catch the failure collaboratively. Other developers can help. Or you can use automated tests that run on a bot farm, checking your code on a variety of different systems (see for example Python's buildbots). 5. Catch the failure in alpha. Release to a small number of willing users first. They get rewarded with cool new features before everyone else does, in return for having fewer guarantees. 6. If all else fails, catch the failure before it kills someone. Design your system so that failures are contained. That's easier for some than others, but it's part of why I've been saying "system" here rather than "program". Eff up like it's your job. https://thedailywtf.com/articles/eff-up-like-it-s-your-job ChrisA -- https://mail.python.org/mailman/listinfo/python-list
Re: Simple webserver
On Sun, 22 Oct 2023 at 04:13, Janis Papanagnou via Python-list wrote: > I have a couple decades experience with about a dozen programming > languages (not counting assemblers). Asynchronous processing, IPC, > multi-processing, client/server architectures, multi-threading, > semaphores, etc. etc. are concepts that are not new to me. Oh, sweet, sweet, then you should be fine with the library I suggested. It's certainly served me well (and I have similar experience, having learned networking mainly on OS/2 in the 1990s). > My expectation would be that any sophistically designed socket/ > web-socket library would not impose any risk. And the intended > server by itself has only very limited requirements; listening to > incoming request, storing some client information, broadcasting > to the attached clients. Basically just (informally written): > > init server > forever: > wait for request(s) -> queue > handle requests from queue (sequentially): > store specific information from new registered clients > broadcast some information to all registered clients > > It seems to me that multi-threading or async I/O aren't necessary. Technically that's true, but "wait for request(s)" has to handle (a) new incoming sockets, (b) messages from currently-connected sockets, and possibly (c) sockets now being writable when previously they blocked. So you have most of the work of async I/O. Since the library's been built specifically for asyncio, that's the easiest. > I'd like to ask; where do you see the specific risks with Python > (as language per se) and it's (web-socket-)libraries here? > > If the web-socket IPC is well supported the algorithmic parts in > Python seem trivial to learn and implement. - Or am I missing > something? Pretty trivial, yeah. You shouldn't have too much trouble here I expect. > (A brief search gave me the impression that for JS communication > web-sockets would be the method to use. Otherwise I'd just use > basic Unix domain sockets for the purpose and write it, say, in > C or C++ that I already know. But I don't know whether (or how) > plain sockets are used from JS running in a browser. Here I'm > lacking experience. And that lead me to have a look at Python, > since the web-sockets/server examples that I found looked simple.) Yes, that's correct. You can't use plain sockets from inside a web browser, mainly because they offer way way too much flexibility (JS code is untrusted and is now running on your computer, do you really want that to be able to telnet to anything on your LAN?). So websockets are the way to go. There are other similar technologies, but for this sort of "broadcast to connected clients" messaging system, websockets rule. ChrisA -- https://mail.python.org/mailman/listinfo/python-list
Re: Simple webserver
On Fri, 20 Oct 2023 at 22:31, Janis Papanagnou via Python-list
wrote:
>
> On 19.10.2023 01:23, Chris Angelico wrote:
> >
> > Broadly speaking, your ideas are great. Any programming language CAN
> > be used for the server (and I've used several, not just Python).
>
> Out of curiosity; what where these languages? - If there's one I
> already know I might save some time implementing the server. :-)
>
I've done websocket servers in Python, Node.js, and Pike, and possibly
others but I can't recall at the moment. Might have done one in Ruby,
but that would have just been part of playing around and comparing
features ("how easy is it to do in Ruby").
ChrisA
--
https://mail.python.org/mailman/listinfo/python-list
Re: Re: Re: Re: Any possible type alias that can also set a default value for a function arg?
On Thu, 19 Oct 2023 at 19:34, Karsten Hilbert wrote:
>
> > > As per my recent foray into abusing existence-checking for Singleton
> > > assurance
> > > along such lines as
> > >
> > > >>> try: self.initialized
> > > >>> except AttributeError: print('first instantiation'); self.initialized
> > > >>> = True
> > >
> > > and then changing that to
> > >
> > > >>> try: self.initialized:bool
> >
> > But that's not equivalent code.
>
> I learned as much (RHS vs LHS).
>
> But it did not _intuitively_ resonate with the sentiment
> "type annotation does not change the running of code".
Unfortunately, that simply means that your intuition was wrong. It
doesn't change my prior statement.
ChrisA
--
https://mail.python.org/mailman/listinfo/python-list
Re: Re: Re: Any possible type alias that can also set a default value for a function arg?
On Thu, 19 Oct 2023 at 18:25, Karsten Hilbert wrote:
>
> > > > Fundamentally no, at least not without some shenanigans. Type hints do
> > > > not affect the regular running of the code,
> > >
> > > Except when they do ;-)
> > >
> > > ... depending on what counts as (valid) code ...
> > >
> > > In Python a distinction can be made between "runnable" and "valid" :-D
> > >
> >
> > Can you give a counter-example?
>
> As per my recent foray into abusing existence-checking for Singleton assurance
> along such lines as
>
> >>> try: self.initialized
> >>> except AttributeError: print('first instantiation'); self.initialized =
> >>> True
>
> and then changing that to
>
> >>> try: self.initialized:bool
But that's not equivalent code. You might just as well say that the
ellipsis here suddenly changes the code:
self.initialized
self.initialized = ...
These are completely different, and they behave differently. Both are
valid, but they mean different things.
ChrisA
--
https://mail.python.org/mailman/listinfo/python-list
Re: Re: Any possible type alias that can also set a default value for a function arg?
On Thu, 19 Oct 2023 at 18:04, Karsten Hilbert wrote: > > > > or something like that. Basically, any way to avoid writing `= None` over > > > and over again. > > > > Fundamentally no, at least not without some shenanigans. Type hints do > > not affect the regular running of the code, > > Except when they do ;-) > > ... depending on what counts as (valid) code ... > > In Python a distinction can be made between "runnable" and "valid" :-D > Can you give a counter-example? I mean, yes, any code can be written that inspects the annotations at runtime and makes whatever changes it likes, but that's part of what I described as "shenanigans". ChrisA -- https://mail.python.org/mailman/listinfo/python-list
Re: Any possible type alias that can also set a default value for a function arg?
On Thu, 19 Oct 2023 at 10:11, Matthew Carruth via Python-list
wrote:
>
> We have the `Optional[T]` type as a short-hand for Union[T | None] and
> telling us that said argument may not be present.
>
> However, I find that a majority of the time, we also want to set a default
> value of None on the argument so that it can be evaluated without doing a
> getattr() check first.
>
> iow, a lot of `foo: Optional[str] = None` in method signatures.
>
> I'd love to see a companion to the Optional type, I'll call it Default, so
> that it can take a default value as a second arg, with a default of that
> being None.
>
> For example:
>
> foo: Default[str] would be equivalent to foo: Optional[str] = None
> foo: Default[str, "bar"] would be equivalent to foo: Optional[str] = "bar"
>
> or something like that. Basically, any way to avoid writing `= None` over and
> over again.
Fundamentally no, at least not without some shenanigans. Type hints do
not affect the regular running of the code, so they can't add
defaults. You could do it the other way around and have the default
imply that it is optional, and I believe that used to be the way that
MyPy calculated things, but it was ultimately rejected. (I may have
the details wrong on that though, don't quote me.)
Ahh, but shenanigans? What kind of shenanigans is that? Glad you
asked! So, uhh, you could decorate a function and mess with its
defaults.
>>> from typing import Optional
>>> def spam(n: Optional[int]):
... if n is None: print("Default spamminess")
... else: print("Spam " * n)
...
>>> spam(5)
Spam Spam Spam Spam Spam
>>> spam(None)
Default spamminess
>>> spam()
Traceback (most recent call last):
File "", line 1, in
TypeError: spam() missing 1 required positional argument: 'n'
>>> spam.__defaults__ = (None,)
>>> spam()
Default spamminess
So you could design a decorator that goes through all the arguments,
finds the ones that say "Optional", and adds a default of None if one
wasn't specified. Good luck with it though. First, you'll have to deal
with the difficulties of aligning arguments (not insurmountable but a
lot of work; don't forget that there are posonly and kwonly args to
consider). Then, you'll have to deal with the much bigger difficulties
of convincing people that this is a good thing.
BTW, rather than a decorator, you could do this by iterating over
every function in a module or class. That might work out easier. Not
sure.
Just be aware that, while Python provides you with all the tools
necessary to shoot yourself in the foot, that isn't a guarantee that
holes in feet are worthwhile.
ChrisA
--
https://mail.python.org/mailman/listinfo/python-list
Re: Simple webserver
On Thu, 19 Oct 2023 at 10:07, Janis Papanagnou via Python-list wrote: > > I am pondering about writing a client/server software with > websockets as communication protocol. The clients will run > in browser as Javascript programs and the server may be in > any (any sensible) programming language running standalone > to be connected remotely by the browser-based JS clients. > > I found a Python sample[*] but I am neither familiar with > Python nor with the 'simple_websocket_server' package that > is used in that sample code. But the code looks so simple > that I'm considering to learn and use Python for the task. > > The requirements I have are quite simple; I want to get the > client "address"/identifier from an incoming message, store > it in a list, and send responses to all active clients for > which addresses have been stored. > > Can anyone tell me whether a simple extension of that "echo > incoming message" sample[*] would be easily possible with > Python and with that 'simple_websocket_server' package used? > > Thanks for any hints (or search keywords, or code samples)! Oooh you've touched on one of my favourite topics. I *love* networking and communication, and websockets are one of my well-used technologies. Let's do this!! I've never used the "simple_websocket_server" you mentioned, but I've used this one in a few projects: https://pypi.org/project/websockets/ Be aware that it is designed with asyncio in mind, so if you prefer different concurrency models, you may need to look elsewhere. But I've had good success with this one. Broadly speaking, your ideas are great. Any programming language CAN be used for the server (and I've used several, not just Python). My personal preference is to build a protocol on top of websockets, for example: * All messages are "text", and are JSON-encoded * All messages represent objects (in Python, dictionaries) with a "cmd" attribute * The first message sent by the client has cmd "init" and specifies a "type" and "group". * The server tracks all connected clients by their groups, and can broadcast messages to everyone in a group. Here's one example, actually one of my brother's projects but I contributed to the websocket aspects: https://github.com/stephenangelico/BioBox/blob/master/browser.py And here's one that doesn't actually use Python, but uses all the same ideas; this is the JS end: https://github.com/Rosuav/StilleBot/blob/master/httpstatic/ws_sync.js (It has quite a bit more sophistication than you'll need to get started with, but shows how the protocol can expand as needed.) So! Tying this back in with your goals: > The requirements I have are quite simple; I want to get the > client "address"/identifier from an incoming message, store > it in a list, and send responses to all active clients for > which addresses have been stored. Sounds to me like the best way would be for the socket group to be the identifier of the client. You could augment the "init" message to include some sort of authentication, or alternatively, rely on other forms of authentication; a websocket established to the same origin as the page itself can take advantage of regular browser credentials. Once that's established, you can have a message from one client result in the server sending out that message to all clients for the recipient's address. The reason I'm talking about "groups" here instead of simply having one client per address is that it scales well to one person having multiple tabs open, or having the app on their phone as well as their computer, or anything like that. Messages will arrive on all of them. Hope that's enough to get you started! I'd be delighted to help further if you run into difficulties. ChrisA -- https://mail.python.org/mailman/listinfo/python-list
Re: Where I do ask for a new feature
On Tue, 17 Oct 2023 at 12:55, Bongo Ferno via Python-list wrote: > > Where I can ask python developers for a new feature? > > This feature would allow us to create short aliases for long object paths, > similar to the with statement. This would make code more readable and > maintainable. > > For example, if we have a long object like > "MyObject.stuff.longStuff.SubObject", we could create a short alias for it > like this: > > > aliasView my_object.stuff.long_stuff.sub_object as short_view > #Now, we can operate with the nested object using the short alias: > print(short_view.some_method()) > > This is much more concise and readable than having to write out the full > object path every time. > You can actually just do that with simple assignment! short_view = my_object.stuff.long_stuff.sub_object print(short_view.some_method()) It'll work! ChrisA -- https://mail.python.org/mailman/listinfo/python-list
Re: Is a Python event polled or interrupt driven?
On Fri, 13 Oct 2023 at 01:48, Chris Green via Python-list wrote: > > In the following code is the event polled by the Python process > running the code or is there something cleverer going on such that > Python sees an interrupt when the input goes high (or low)? > This isn't something inherent to Python; it's the specific behaviour of the library you're using. So I dug through that library a bit, and ended up here: https://github.com/adafruit/adafruit-beaglebone-io-python/blob/cf306ed7f9f24111d0949dd60ac232e81241bffe/source/event_gpio.c#L753 which starts a thread: https://github.com/adafruit/adafruit-beaglebone-io-python/blob/cf306ed7f9f24111d0949dd60ac232e81241bffe/source/event_gpio.c#L662 which appears to make use of epoll for efficient event handling. Edge detection itself seems to be done here: https://github.com/adafruit/adafruit-beaglebone-io-python/blob/cf306ed7f9f24111d0949dd60ac232e81241bffe/source/event_gpio.c#L522 I don't know enough about the architecture of the BeagleBone to be certain, but my reading of it is that most of the work of edge detection is done by the OS kernel, which then sends the Adafruit handler a notification via a file descriptor. The secondary thread waits for those messages (which can be done very efficiently), and in turn calls the Python callbacks. In other words, the "something cleverer" is all inside the OS kernel, and yes, in effect, it's an interrupt. ChrisA -- https://mail.python.org/mailman/listinfo/python-list
