Re: How to have python 2 and 3 both on windows?
On Sun, 24 Apr 2022 01:19:38 + (UTC), Sunil KR declaimed the following: > >-- Why are my strings being sent to python3, so that I get the unicode related >error? >-- in other cases I see error pertaining to the print function In python2, the default for strings is BYTES -- you must explicitly ask for unicode (for literals, using u'literal' notation). Python3 strings are, by default, interpreted as unicode (with the encoding for source code [and hence, literals] specified somewhere via a special comment). Getting a normal python2 string requires using the b'literal' notation to indicate /bytes/. Also, in Python2, print is a language statement, not a function. If you have any print statements that do not have ( ) surrounding the output items, it WILL fail in Python3. >In my case, I don't own the python2 scripts and so I am not allowed to change >any part of them. And I wouldn't need to either, if I can make python 2 and 3 >coexist on my system > Even if you are not "allowed to change" those scripts, have you tried feeding them through the 2to3 conversion script just to see what type of changes would be required? https://docs.python.org/3/library/2to3.html -- Wulfraed Dennis Lee Bieber AF6VN wlfr...@ix.netcom.comhttp://wlfraed.microdiversity.freeddns.org/ -- https://mail.python.org/mailman/listinfo/python-list
Re: How to have python 2 and 3 both on windows?
I am happy with how the python starts up. When I use python I get python 2. I am ok with using py -3 for my new scripts, even using the shebang like #!py -3 I don't want to put a unix (or for that matter windows) path in the shebang, as it is not platform portable But the real question/s for me is/are -- Why are my strings being sent to python3, so that I get the unicode related error? -- in other cases I see error pertaining to the print function In my case, I don't own the python2 scripts and so I am not allowed to change any part of them. And I wouldn't need to either, if I can make python 2 and 3 coexist on my system > On 22 Apr 2022, at 17:10, Sunil KR via Python-list wrote: > > I have some scripts that are old and won't work under python2 and at the > same time I am writing new scripts which will use python3. However, if python > 2 and 3 cannot co-exist in a windows box it will be impossible to transition > What I try:- remove all pythons and launchers- Use windows installer and > install python2 in python27 directory- Use windows installer and install > python3 in python310 directory- When installing python3 I opt in to install > the launcher- Test with py -2 and py -3 and see that I get the expected > prompt- just typing python gets me python2 As you have proved you can install many versions of python at the same time on windows. In your scripts set the shebang to run the python version you want. E.g #!/usr/bin/python2 Or #!/usr/bin/python3 The python launcher py.exe will then do the right thing after looking at en shebang line. Also you can edit the INI file of the py.exe launcher to set the defaults the way you want them. Do a search for “py.exe ini” to file the path to the file, I do not have it my finger tips. Tip “py.exe -0” will list the state of installed pythons. Barry > -- > https://mail.python.org/mailman/listinfo/python-list > -- https://mail.python.org/mailman/listinfo/python-list
Re: tail
On Sun, 24 Apr 2022 at 10:04, Cameron Simpson wrote: > > On 24Apr2022 08:21, Chris Angelico wrote: > >On Sun, 24 Apr 2022 at 08:18, Cameron Simpson wrote: > >> An approach I think you both may have missed: mmap the file and use > >> mmap.rfind(b'\n') to locate line delimiters. > >> https://docs.python.org/3/library/mmap.html#mmap.mmap.rfind > > > >Yeah, I made a vague allusion to use of mmap, but didn't elaborate > >because I actually have zero idea of how efficient this would be. > >Would it be functionally equivalent to the chunking, but with the > >chunk size defined by the system as whatever's most optimal? It would > >need to be tested. > > True. I'd expect better than single byte seek/read though. > Yeah, I think pretty much *anything* would be better than single byte seeks. ChrisA -- https://mail.python.org/mailman/listinfo/python-list
Re: tail
On 24Apr2022 08:21, Chris Angelico wrote: >On Sun, 24 Apr 2022 at 08:18, Cameron Simpson wrote: >> An approach I think you both may have missed: mmap the file and use >> mmap.rfind(b'\n') to locate line delimiters. >> https://docs.python.org/3/library/mmap.html#mmap.mmap.rfind > >Yeah, I made a vague allusion to use of mmap, but didn't elaborate >because I actually have zero idea of how efficient this would be. >Would it be functionally equivalent to the chunking, but with the >chunk size defined by the system as whatever's most optimal? It would >need to be tested. True. I'd expect better than single byte seek/read though. >I've never used mmap for this kind of job, so it's not something I'm >comfortable predicting the performance of. Fair. But it would be much easier to read code. Cheers, Cameron Simpson -- https://mail.python.org/mailman/listinfo/python-list
Re: Style for docstring
On 23/04/2022 08.35, Michael F. Stemper wrote: > On 22/04/2022 14.59, Chris Angelico wrote: >> On Sat, 23 Apr 2022 at 05:56, Michael F. Stemper >> wrote: >>> >>> I'm writing a function that is nearly self-documenting by its name, >>> but still want to give it a docstring. Which of these would be >>> best from a stylistic point of view: >>> >>> >>> Tells caller whether or not a permutation is even. >>> >>> Determines if a permutation is even. (Alternative is that it's odd.) >>> >>> Returns True if permutation is even, False if it is odd. > > >> >> I'd go with the third one, but "Return" rather than "Returns". Or >> possibly "Test whether a permutation is even". > > "So let it be written. So let it be done." > >> That's just one opinion though, others may disagree :) > > Two for two. Thanks. I've always taken the PEP somewhat for granted (with belated thanks and respect to the authors and others who exerted effort (or would that be, efforts?) to publish the finished work. One of the things that I've taken-as-read, is WHY we use docstrings. That rationale is wider than Python. Like many others, I just brought that sort thinking with me. Have you noticed that the PEP doesn't discuss this? It has taken this thread to make me realise such. The elephant in the style room is PEP-008. It's quite old in 'Python-years'! As time went by, PEP-257 became relevant - and I don't recall which was 'chicken' and which 'egg', between the PEP and the docutils/help/etc tools which depend upon a level of docstring uniformity. No matter, together they represent a step of progress, after PEP-008. Since then we have added typing. Where once there was an extensive documentation system to clarify the use of parameters and the definition of return-values, now much of that can be done on the def-line (or equivalent). Another 'development' (I'd prefer 'natural progression') is that as computers have become more powerful and storage cheaper, we have been able to first justify, and now habitually, use longer identifier-names. This step of progress should enable more descriptive and readable code. So, we can consider that there are (now) three aspects which overlap, considerably: 1 the name of the function - descriptive and readable (see OP's assurance on this point) 2 typing - description and distinction of parameters and return-values 3 docstring - even more information about the function, how the author intends it be used, embedded assumptions, etc Thus, aren't there situations where a short, sharp, DRY, and SRP, function; once well-named and accurately typed, could stand on its own merits without a docstring. (yes, others will suck in their breath and start preparing a bon-fire, at the utterance of such 'heresy'...) We all hate 'boiler-plate', ie feeling forced to type stuff 'just because', and particularly when it seems like duplication/repetition (see recent thread(s) 'here' about __init__() and dataclasses). In fact, one reason why we have functions is to reduce/remove (code) duplication! So, why would we indulge in documental 'duplication' just-because we should?must have a docstring? While I'm tilting at windmills, wasn't option-2 the best? (ooh, those flames are starting to warm my feet...) Why? First, see other discussion about "imperatives". My habit is to take the Statement of Requirements and copy it into a brand-new __main__. As development proceeds, various details will be extracted and copied into function and class docstrings. The language of a specification should be imperative, ie you will *do* 'this'. A second reason, is that the docstring should document the function, which is subtly-different from describing the return-value (and that has become a task for typing anyway). (OK, the flames are getting-good, and it's time to break-out the marsh-mallows and crumpets...) So, 'nr2' describes what the function DOES! PS would you mind passing-over the fire-extinguisher? -- Regards, =dn -- https://mail.python.org/mailman/listinfo/python-list
Re: tail
On Sun, 24 Apr 2022 at 08:18, Cameron Simpson wrote: > > On 24Apr2022 07:15, Chris Angelico wrote: > >On Sun, 24 Apr 2022 at 07:13, Marco Sulla > >wrote: > >> Emh, why chunks? My function simply reads byte per byte and compares > >> it to b"\n". When it find it, it stops and do a readline(): > [...] > >> This is only for one line and in utf8, but it can be generalised. > > For some encodings that generalisation might be hard. But mostly, yes. > > >Ah. Well, then, THAT is why it's inefficient: you're seeking back one > >single byte at a time, then reading forwards. That is NOT going to > >play nicely with file systems or buffers. > > An approach I think you both may have missed: mmap the file and use > mmap.rfind(b'\n') to locate line delimiters. > https://docs.python.org/3/library/mmap.html#mmap.mmap.rfind Yeah, I made a vague allusion to use of mmap, but didn't elaborate because I actually have zero idea of how efficient this would be. Would it be functionally equivalent to the chunking, but with the chunk size defined by the system as whatever's most optimal? It would need to be tested. I've never used mmap for this kind of job, so it's not something I'm comfortable predicting the performance of. ChrisA -- https://mail.python.org/mailman/listinfo/python-list
Re: tail
On Sun, 24 Apr 2022 at 08:06, dn wrote: > > On 24/04/2022 09.15, Chris Angelico wrote: > > On Sun, 24 Apr 2022 at 07:13, Marco Sulla > > wrote: > >> > >> On Sat, 23 Apr 2022 at 23:00, Chris Angelico wrote: > > This is quite inefficient in general. > > Why inefficient? I think that readlines() will be much slower, not > only more time consuming. > >>> > >>> It depends on which is more costly: reading the whole file (cost > >>> depends on size of file) or reading chunks and splitting into lines > >>> (cost depends on how well you guess at chunk size). If the lines are > >>> all *precisely* the same number of bytes each, you can pick a chunk > >>> size and step backwards with near-perfect efficiency (it's still > >>> likely to be less efficient than reading a file forwards, on most file > >>> systems, but it'll be close); but if you have to guess, adjust, and > >>> keep going, then you lose efficiency there. > >> > >> Emh, why chunks? My function simply reads byte per byte and compares it to > >> b"\n". When it find it, it stops and do a readline(): > ... > > > Ah. Well, then, THAT is why it's inefficient: you're seeking back one > > single byte at a time, then reading forwards. That is NOT going to > > play nicely with file systems or buffers. > > > > Compare reading line by line over the file with readlines() and you'll > > see how abysmal this is. > > > > If you really only need one line (which isn't what your original post > > suggested), I would recommend starting with a chunk that is likely to > > include a full line, and expanding the chunk until you have that > > newline. Much more efficient than one byte at a time. > > > Disagreeing with @Chris in the sense that I use tail very frequently, > and usually in the context of server logs - but I'm talking about the > Linux implementation, not Python code! tail(1) doesn't read a single byte at a time. It works in chunks, more-or-less the way I described. (That's where I borrowed the technique from.) It finds one block of lines, and displays those; it doesn't iterate backwards. (By the way, the implementation of "tail -f" is actually less complicated than you might think, as long as inotify is available. That's been much more useful to me than reading a file backwards.) > Agree with @Chris' assessment of the (in)efficiency. It is more likely > than not, that you will have a good idea of the length of each line. > Even if the line-length is highly-variable (thinking of some of my > applications of the Python logging module!), one can still 'take a stab > at it' (a "thumb suck" as an engineer-colleague used to say - apparently > not an electrical engineer!) by remembering that lines exceeding > 80-characters become less readable and thus have likely?hopefully been > split into two. > > Thus, > > N*(80+p) > > where N is the number of lines desired and p is a reasonable > 'safety'/over-estimation percentage, would give a good chunk size. > Binar-ily grab that much of the end of the file, split on line-ending, > and take the last N elements from that list. (with 'recovery code' just > in case the 'chunk' wasn't large-enough). Yup, that's the broad idea of the chunked read. If you know how many lines you're going to need, that's not too bad. If you need to iterate backwards over the file (as the original question suggested), that gets complicated fast. > Adding to the efficiency (of the algorithm, but not the dev-time), > consider that shorter files are likely to be more easily--handled by > reading serially from the beginning. To side-step @Chris' criticism, use > a generator to produce the individual lines (lazy evaluation and low > storage requirement) and feed them into a circular-queue which is > limited to N-entries. QED, as fast as the machine's I/O, and undemanding > of storage-space! Still needs to read the whole file though. > Running a few timing trials should reveal the 'sweet spot', at which one > algorithm takes-over from the other! Unfortunately, the sweet spot will depend on a lot of things other than just the file size. Is it stored contiguously? Is it on hard drive or SSD? Can it be read from the disk cache? How frequently do the chunk size guesses fail, and how often do they grab more than is necessary? It's basically unknowable, so the best you can do is judge your own app, pick one, and go with it. > NB quite a few of IBM's (extensively researched) algorithms which formed > utility program[me]s on mainframes, made similar such algorithmic > choices, in the pursuit of efficiencies. Indeed. Just make a choice and run with it, and accept that there will be situations where it'll be inefficient. ChrisA -- https://mail.python.org/mailman/listinfo/python-list
Re: tail
On 24Apr2022 07:15, Chris Angelico wrote: >On Sun, 24 Apr 2022 at 07:13, Marco Sulla wrote: >> Emh, why chunks? My function simply reads byte per byte and compares >> it to b"\n". When it find it, it stops and do a readline(): [...] >> This is only for one line and in utf8, but it can be generalised. For some encodings that generalisation might be hard. But mostly, yes. >Ah. Well, then, THAT is why it's inefficient: you're seeking back one >single byte at a time, then reading forwards. That is NOT going to >play nicely with file systems or buffers. An approach I think you both may have missed: mmap the file and use mmap.rfind(b'\n') to locate line delimiters. https://docs.python.org/3/library/mmap.html#mmap.mmap.rfind Avoids sucking the whole file into memory in the usualy sense, instead the file is paged in as needed. Far more efficient that a seek/read single byte approach. If the file's growing you can do this to start with, then do a normal file open from your end point to follow accruing text. (Or reuse the descriptor you sues for the mmap, but using s.read().) Cheers, Cameron Simpson -- https://mail.python.org/mailman/listinfo/python-list
Re: tail
On Sun, 24 Apr 2022 at 08:03, Peter J. Holzer wrote: > > On 2022-04-24 04:57:20 +1000, Chris Angelico wrote: > > On Sun, 24 Apr 2022 at 04:37, Marco Sulla > > wrote: > > > What about introducing a method for text streams that reads the lines > > > from the bottom? Java has also a ReversedLinesFileReader with Apache > > > Commons IO. > > > > It's fundamentally difficult to get precise. In general, there are > > three steps to reading the last N lines of a file: > > > > 1) Find out the size of the file (currently, if it's being grown) > > 2) Seek to the end of the file, minus some threshold that you hope > > will contain a number of lines > > 3) Read from there to the end of the file, split it into lines, and > > keep the last N > [...] > > This is quite inefficient in general. It would be far FAR easier to do > > this instead: > > > > 1) Read the entire file and decode bytes to text > > 2) Split into lines > > 3) Iterate backwards over the lines > > Which one is more efficient depends very much on the size of the file. > For a file of a few kilobytes, the second solution is probably more > efficient. But for a few gigabytes, that's almost certainly not the > case. Yeah. I said "easier", not necessarily more efficient. Which is more efficient is a virtually unanswerable question (will you need to iterate over the whole file or stop part way? Is the file stored contiguously? Can you memory map it in some way?), so it's going to depend a lot on your use-case. > > Tada! Done. And in Python, quite easy. The downside, of course, is > > that you have to store the entire file in memory. > > Not just memory. You have to read the whole file in the first place. Which is > hardly efficient if you only need a tiny fraction. Right - if that's the case, then the chunked form, even though it's harder, would be worth doing. > > Personally, unless the file is tremendously large and I know for sure > > that I'm not going to end up iterating over it all, I would pay the > > memory price. > > Me, too. Problem with a library function (as Marco proposes) is that you > don't know how it will be used. > Yup. And there may be other options worth considering, like maintaining an index (a bunch of "line 142857 is at byte position 3141592" entries) which would allow random access... but at some point, if your file is that big, you probably shouldn't be storing it as a file of lines of text. Use a database instead. Reading a text file backwards by lines is, by definition, hard. Every file format I know of that involves starting at the end of the file is defined in binary, so you can actually seek, and is usually defined with fixed-size structures (so you just go "read the last 768 bytes of the file" or something). ChrisA -- https://mail.python.org/mailman/listinfo/python-list
Re: tail
On 24/04/2022 09.15, Chris Angelico wrote: > On Sun, 24 Apr 2022 at 07:13, Marco Sulla > wrote: >> >> On Sat, 23 Apr 2022 at 23:00, Chris Angelico wrote: > This is quite inefficient in general. Why inefficient? I think that readlines() will be much slower, not only more time consuming. >>> >>> It depends on which is more costly: reading the whole file (cost >>> depends on size of file) or reading chunks and splitting into lines >>> (cost depends on how well you guess at chunk size). If the lines are >>> all *precisely* the same number of bytes each, you can pick a chunk >>> size and step backwards with near-perfect efficiency (it's still >>> likely to be less efficient than reading a file forwards, on most file >>> systems, but it'll be close); but if you have to guess, adjust, and >>> keep going, then you lose efficiency there. >> >> Emh, why chunks? My function simply reads byte per byte and compares it to >> b"\n". When it find it, it stops and do a readline(): ... > Ah. Well, then, THAT is why it's inefficient: you're seeking back one > single byte at a time, then reading forwards. That is NOT going to > play nicely with file systems or buffers. > > Compare reading line by line over the file with readlines() and you'll > see how abysmal this is. > > If you really only need one line (which isn't what your original post > suggested), I would recommend starting with a chunk that is likely to > include a full line, and expanding the chunk until you have that > newline. Much more efficient than one byte at a time. Disagreeing with @Chris in the sense that I use tail very frequently, and usually in the context of server logs - but I'm talking about the Linux implementation, not Python code! Agree with @Chris' assessment of the (in)efficiency. It is more likely than not, that you will have a good idea of the length of each line. Even if the line-length is highly-variable (thinking of some of my applications of the Python logging module!), one can still 'take a stab at it' (a "thumb suck" as an engineer-colleague used to say - apparently not an electrical engineer!) by remembering that lines exceeding 80-characters become less readable and thus have likely?hopefully been split into two. Thus, N*(80+p) where N is the number of lines desired and p is a reasonable 'safety'/over-estimation percentage, would give a good chunk size. Binar-ily grab that much of the end of the file, split on line-ending, and take the last N elements from that list. (with 'recovery code' just in case the 'chunk' wasn't large-enough). Adding to the efficiency (of the algorithm, but not the dev-time), consider that shorter files are likely to be more easily--handled by reading serially from the beginning. To side-step @Chris' criticism, use a generator to produce the individual lines (lazy evaluation and low storage requirement) and feed them into a circular-queue which is limited to N-entries. QED, as fast as the machine's I/O, and undemanding of storage-space! Running a few timing trials should reveal the 'sweet spot', at which one algorithm takes-over from the other! NB quite a few of IBM's (extensively researched) algorithms which formed utility program[me]s on mainframes, made similar such algorithmic choices, in the pursuit of efficiencies. -- Regards, =dn -- https://mail.python.org/mailman/listinfo/python-list
Re: tail
On 2022-04-24 04:57:20 +1000, Chris Angelico wrote: > On Sun, 24 Apr 2022 at 04:37, Marco Sulla > wrote: > > What about introducing a method for text streams that reads the lines > > from the bottom? Java has also a ReversedLinesFileReader with Apache > > Commons IO. > > It's fundamentally difficult to get precise. In general, there are > three steps to reading the last N lines of a file: > > 1) Find out the size of the file (currently, if it's being grown) > 2) Seek to the end of the file, minus some threshold that you hope > will contain a number of lines > 3) Read from there to the end of the file, split it into lines, and > keep the last N [...] > This is quite inefficient in general. It would be far FAR easier to do > this instead: > > 1) Read the entire file and decode bytes to text > 2) Split into lines > 3) Iterate backwards over the lines Which one is more efficient depends very much on the size of the file. For a file of a few kilobytes, the second solution is probably more efficient. But for a few gigabytes, that's almost certainly not the case. > Tada! Done. And in Python, quite easy. The downside, of course, is > that you have to store the entire file in memory. Not just memory. You have to read the whole file in the first place. Which is hardly efficient if you only need a tiny fraction. > Personally, unless the file is tremendously large and I know for sure > that I'm not going to end up iterating over it all, I would pay the > memory price. Me, too. Problem with a library function (as Marco proposes) is that you don't know how it will be used. hp -- _ | Peter J. Holzer| Story must make more sense than reality. |_|_) || | | | h...@hjp.at |-- Charles Stross, "Creative writing __/ | http://www.hjp.at/ | challenge!" signature.asc Description: PGP signature -- https://mail.python.org/mailman/listinfo/python-list
Re: Receive a signal when waking or suspending?
> I don't know in Python, but maybe you can create a script that writes > on a named pipe and read it from Python? > https://askubuntu.com/questions/226278/run-script-on-wakeup Thanks, that gives me something to munch on. -- https://mail.python.org/mailman/listinfo/python-list
Re: Receive a signal when waking or suspending?
On 24/04/2022 07.36, Skip Montanaro wrote: > It's not clear there is a straightforward way to catch a signal or get > an event notification when my computer (Dell running XUbuntu 20.04) is > about to sleep or when it's just awakened. The app uses tkinter. Is > there some more-or-less easy way to do this? Mac support would be nice > (I have my eye on a new MacBook one of these days). I suppose bonus > points for something which works on Windows, but that's not a platform > I actually care about. https://duckduckgo.com/?t=ffab=python+up+time pointed-out the "uptime — Cross-platform uptime library" - TLDR; I'm not sure if "uptime" and "boot" relate only to a 'cold-start' or if they include bringing the machine out of 'stand-by' or 'hibernation' https://duckduckgo.com/?t=ffab=python+battery+status pointed-out several ideas which require MS-Windows, but apparently the psutil library could be bent to your will: https://www.geeksforgeeks.org/python-script-to-shows-laptop-battery-percentage/ Please let us know how you get on... -- Regards, =dn -- https://mail.python.org/mailman/listinfo/python-list
Re: tail
On Sun, 24 Apr 2022 at 07:13, Marco Sulla wrote: > > On Sat, 23 Apr 2022 at 23:00, Chris Angelico wrote: > > > > This is quite inefficient in general. > > > > > > Why inefficient? I think that readlines() will be much slower, not > > > only more time consuming. > > > > It depends on which is more costly: reading the whole file (cost > > depends on size of file) or reading chunks and splitting into lines > > (cost depends on how well you guess at chunk size). If the lines are > > all *precisely* the same number of bytes each, you can pick a chunk > > size and step backwards with near-perfect efficiency (it's still > > likely to be less efficient than reading a file forwards, on most file > > systems, but it'll be close); but if you have to guess, adjust, and > > keep going, then you lose efficiency there. > > Emh, why chunks? My function simply reads byte per byte and compares it to > b"\n". When it find it, it stops and do a readline(): > > def tail(filepath): > """ > @author Marco Sulla > @date May 31, 2016 > """ > > try: > filepath.is_file > fp = str(filepath) > except AttributeError: > fp = filepath > > with open(fp, "rb") as f: > size = os.stat(fp).st_size > start_pos = 0 if size - 1 < 0 else size - 1 > > if start_pos != 0: > f.seek(start_pos) > char = f.read(1) > > if char == b"\n": > start_pos -= 1 > f.seek(start_pos) > > if start_pos == 0: > f.seek(start_pos) > else: > for pos in range(start_pos, -1, -1): > f.seek(pos) > > char = f.read(1) > > if char == b"\n": > break > > return f.readline() > > This is only for one line and in utf8, but it can be generalised. > Ah. Well, then, THAT is why it's inefficient: you're seeking back one single byte at a time, then reading forwards. That is NOT going to play nicely with file systems or buffers. Compare reading line by line over the file with readlines() and you'll see how abysmal this is. If you really only need one line (which isn't what your original post suggested), I would recommend starting with a chunk that is likely to include a full line, and expanding the chunk until you have that newline. Much more efficient than one byte at a time. ChrisA -- https://mail.python.org/mailman/listinfo/python-list
Re: tail
On Sat, 23 Apr 2022 at 23:00, Chris Angelico wrote: > > > This is quite inefficient in general. > > > > Why inefficient? I think that readlines() will be much slower, not > > only more time consuming. > > It depends on which is more costly: reading the whole file (cost > depends on size of file) or reading chunks and splitting into lines > (cost depends on how well you guess at chunk size). If the lines are > all *precisely* the same number of bytes each, you can pick a chunk > size and step backwards with near-perfect efficiency (it's still > likely to be less efficient than reading a file forwards, on most file > systems, but it'll be close); but if you have to guess, adjust, and > keep going, then you lose efficiency there. Emh, why chunks? My function simply reads byte per byte and compares it to b"\n". When it find it, it stops and do a readline(): def tail(filepath): """ @author Marco Sulla @date May 31, 2016 """ try: filepath.is_file fp = str(filepath) except AttributeError: fp = filepath with open(fp, "rb") as f: size = os.stat(fp).st_size start_pos = 0 if size - 1 < 0 else size - 1 if start_pos != 0: f.seek(start_pos) char = f.read(1) if char == b"\n": start_pos -= 1 f.seek(start_pos) if start_pos == 0: f.seek(start_pos) else: for pos in range(start_pos, -1, -1): f.seek(pos) char = f.read(1) if char == b"\n": break return f.readline() This is only for one line and in utf8, but it can be generalised. -- https://mail.python.org/mailman/listinfo/python-list
Re: tail
On Sun, 24 Apr 2022 at 06:41, Marco Sulla wrote: > > On Sat, 23 Apr 2022 at 20:59, Chris Angelico wrote: > > > > On Sun, 24 Apr 2022 at 04:37, Marco Sulla > > wrote: > > > > > > What about introducing a method for text streams that reads the lines > > > from the bottom? Java has also a ReversedLinesFileReader with Apache > > > Commons IO. > > > > It's fundamentally difficult to get precise. In general, there are > > three steps to reading the last N lines of a file: > > > > 1) Find out the size of the file (currently, if it's being grown) > > 2) Seek to the end of the file, minus some threshold that you hope > > will contain a number of lines > > 3) Read from there to the end of the file, split it into lines, and > > keep the last N > > > > Reading the preceding N lines is basically a matter of repeating the > > same exercise, but instead of "end of the file", use the byte position > > of the line you last read. > > > > The problem is, seeking around in a file is done by bytes, not > > characters. So if you know for sure that you can resynchronize > > (possible with UTF-8, not possible with some other encodings), then > > you can do this, but it's probably best to build it yourself (opening > > the file in binary mode). > > Well, indeed I have an implementation that does more or less what you > described for utf8 only. The only difference is that I just started > from the end of file -1. I'm just wondering if this will be useful in > the stdlib. I think it's not too difficult to generalise for every > encoding. > > > This is quite inefficient in general. > > Why inefficient? I think that readlines() will be much slower, not > only more time consuming. It depends on which is more costly: reading the whole file (cost depends on size of file) or reading chunks and splitting into lines (cost depends on how well you guess at chunk size). If the lines are all *precisely* the same number of bytes each, you can pick a chunk size and step backwards with near-perfect efficiency (it's still likely to be less efficient than reading a file forwards, on most file systems, but it'll be close); but if you have to guess, adjust, and keep going, then you lose efficiency there. I don't think this is necessary in the stdlib. If anything, it might be good on PyPI, but I for one have literally never wanted this. ChrisA -- https://mail.python.org/mailman/listinfo/python-list
Re: tail
On Sat, 23 Apr 2022 at 20:59, Chris Angelico wrote: > > On Sun, 24 Apr 2022 at 04:37, Marco Sulla > wrote: > > > > What about introducing a method for text streams that reads the lines > > from the bottom? Java has also a ReversedLinesFileReader with Apache > > Commons IO. > > It's fundamentally difficult to get precise. In general, there are > three steps to reading the last N lines of a file: > > 1) Find out the size of the file (currently, if it's being grown) > 2) Seek to the end of the file, minus some threshold that you hope > will contain a number of lines > 3) Read from there to the end of the file, split it into lines, and > keep the last N > > Reading the preceding N lines is basically a matter of repeating the > same exercise, but instead of "end of the file", use the byte position > of the line you last read. > > The problem is, seeking around in a file is done by bytes, not > characters. So if you know for sure that you can resynchronize > (possible with UTF-8, not possible with some other encodings), then > you can do this, but it's probably best to build it yourself (opening > the file in binary mode). Well, indeed I have an implementation that does more or less what you described for utf8 only. The only difference is that I just started from the end of file -1. I'm just wondering if this will be useful in the stdlib. I think it's not too difficult to generalise for every encoding. > This is quite inefficient in general. Why inefficient? I think that readlines() will be much slower, not only more time consuming. -- https://mail.python.org/mailman/listinfo/python-list
Re: Receive a signal when waking or suspending?
I don't know in Python, but maybe you can create a script that writes on a named pipe and read it from Python? https://askubuntu.com/questions/226278/run-script-on-wakeup -- https://mail.python.org/mailman/listinfo/python-list
Receive a signal when waking or suspending?
It's not clear there is a straightforward way to catch a signal or get an event notification when my computer (Dell running XUbuntu 20.04) is about to sleep or when it's just awakened. The app uses tkinter. Is there some more-or-less easy way to do this? Mac support would be nice (I have my eye on a new MacBook one of these days). I suppose bonus points for something which works on Windows, but that's not a platform I actually care about. Thx, Skip -- https://mail.python.org/mailman/listinfo/python-list
Re: Style for docstring
"return true iff this".
I like this.
jan
On 23/04/2022, Stefan Ram wrote:
> Rob Cliffe writes:
>>I'm curious as to why so many people prefer "Return" to "Returns".
>
> The commands, er, names of functions, use the imperative mood
> ("print", not "prints"). So, "return" aligns with that mood
> as a paraphrase of such names.
>
> In Java, at one point, they decided to start to use the
> third person at one point and then half-heartedly converted
> all the documentation, as in
>
> |void println(boolean x)
> |Prints a boolean and then terminate the line.
>
> , where they modified "print" but did not bother do modify
> "terminate".
>
> And instead of "return true if this, false if not this",
> I might be inclined to write "return true iff this".
>
> BTW: As a language element that helps to construct a boolean
> expression from a file name, some languages, like SQL, use
> "EXISTS", while others, like MS-DOS-batch, use "EXIST".
>
>
> --
> https://mail.python.org/mailman/listinfo/python-list
>
--
https://mail.python.org/mailman/listinfo/python-list
Re: tail
On Sun, 24 Apr 2022 at 04:37, Marco Sulla wrote: > > What about introducing a method for text streams that reads the lines > from the bottom? Java has also a ReversedLinesFileReader with Apache > Commons IO. It's fundamentally difficult to get precise. In general, there are three steps to reading the last N lines of a file: 1) Find out the size of the file (currently, if it's being grown) 2) Seek to the end of the file, minus some threshold that you hope will contain a number of lines 3) Read from there to the end of the file, split it into lines, and keep the last N Reading the preceding N lines is basically a matter of repeating the same exercise, but instead of "end of the file", use the byte position of the line you last read. The problem is, seeking around in a file is done by bytes, not characters. So if you know for sure that you can resynchronize (possible with UTF-8, not possible with some other encodings), then you can do this, but it's probably best to build it yourself (opening the file in binary mode). This is quite inefficient in general. It would be far FAR easier to do this instead: 1) Read the entire file and decode bytes to text 2) Split into lines 3) Iterate backwards over the lines Tada! Done. And in Python, quite easy. The downside, of course, is that you have to store the entire file in memory. So it's up to you: pay the memory price, or pay the complexity price. Personally, unless the file is tremendously large and I know for sure that I'm not going to end up iterating over it all, I would pay the memory price. ChrisA -- https://mail.python.org/mailman/listinfo/python-list
Re: Style for docstring
On Fri, Apr 22, 2022 at 12:56 PM Michael F. Stemper < michael.stem...@gmail.com> wrote: > I'm writing a function that is nearly self-documenting by its name, > but still want to give it a docstring. Which of these would be > best from a stylistic point of view: > > >Tells caller whether or not a permutation is even. > >Determines if a permutation is even. (Alternative is that it's odd.) > >Returns True if permutation is even, False if it is odd. > > > (Before somebody suggests it, I'm not going to put six weeks' worth > of a course in group theory in there to help somebody who doesn't > know what those standard terms mean.) > Maybe try pydocstyle? -- https://mail.python.org/mailman/listinfo/python-list
tail
What about introducing a method for text streams that reads the lines from the bottom? Java has also a ReversedLinesFileReader with Apache Commons IO. -- https://mail.python.org/mailman/listinfo/python-list
Re: Style for docstring
Given what you added, Michael, your function is part of a larger collection of functions and being compatible with the others is a valid consideration. Whatever you decide, would ideally be done consistently with all or most of them. And, of course, it others in the collection also can handle multiple ways to specify a permutation, it may be simpler to have each call something like as.permutation() that handlesmultiple forms and converts to the one easiest for you to use. I am not sure that is needed as I suspect the simplest storage is something like a list: [0,3,2,4,5,6,7,1,9,8] but could also be shown with each cycle as a sub-list or something like anumpy vector or a customized class. Clearly if you control the package and how it is used, errors from bad data may not be a concern. But like many Boolean return(s) it is always a problem how to deal with a third possibility. -Original Message- From: Michael F. Stemper To: python-list@python.org Sent: Sat, Apr 23, 2022 8:57 am Subject: Re: Style for docstring On 22/04/2022 21.58, Avi Gross wrote: > Python does have a concept of "truthy" that includes meaning for not just the > standard Booleans but for 0 and non-zero and the empty string and many more > odd things such as an object that defines __bool__ (). > But saying it returns a Boolean True/False valuesounds direct and simple and > informative enough if that is True. > What bothers me is the assumption that anyone knows not so muchjust group > theory but what the argument to the function looks like as a Python object > of some kind. > Does the function accept only some permutation object managed by a specific > module? Will it accept some alternate representation such as a list structure > or other iterator? That's a fair point. However, this function will be the 22nd one in a module for dealing with permutations and groups of permutations. The module has a lengthy docstring explaining the several ways provided to specify a permutation. That way, the same information doesn't need to be written twenty-plus times. > Obviously deeper details would normally be in a manual page or other > documentation but as "permutations" are likely not to be what most people > think about before breakfast, or even after, odd as that may seem, ... I see what you did there :-> -- Michael F. Stemper Psalm 94:3-6 -- https://mail.python.org/mailman/listinfo/python-list -- https://mail.python.org/mailman/listinfo/python-list
Re: Style for docstring
On 22/04/2022 16.12, alister wrote: On Fri, 22 Apr 2022 14:36:27 -0500, Michael F. Stemper wrote: I'm writing a function that is nearly self-documenting by its name, but still want to give it a docstring. Which of these would be best from a stylistic point of view: for guidance I would sugest Pep257 as a start point which would suggest "Return True if permutation is even" I'll take a look at the PEP. Thanks. -- Michael F. Stemper This email is to be read by its intended recipient only. Any other party reading is required by the EULA to send me $500.00. -- https://mail.python.org/mailman/listinfo/python-list
Re: Style for docstring
On 22/04/2022 21.58, Avi Gross wrote: Python does have a concept of "truthy" that includes meaning for not just the standard Booleans but for 0 and non-zero and the empty string and many more odd things such as an object that defines __bool__ (). But saying it returns a Boolean True/False valuesounds direct and simple and informative enough if that is True. What bothers me is the assumption that anyone knows not so muchjust group theory but what the argument to the function looks like as a Python object of some kind. Does the function accept only some permutation object managed by a specific module? Will it accept some alternate representation such as a list structure or other iterator? That's a fair point. However, this function will be the 22nd one in a module for dealing with permutations and groups of permutations. The module has a lengthy docstring explaining the several ways provided to specify a permutation. That way, the same information doesn't need to be written twenty-plus times. Obviously deeper details would normally be in a manual page or other documentation but as "permutations" are likely not to be what most people think about before breakfast, or even after, odd as that may seem, ... I see what you did there :-> -- Michael F. Stemper Psalm 94:3-6 -- https://mail.python.org/mailman/listinfo/python-list
[Python-announce] Nikola v8.2.1 is out!
On behalf of the Nikola team, I am pleased to announce the immediate availability of Nikola v8.2.1. This is a minor release with a couple new features, as well as fixes for compatibility with the latest version of doit. What is Nikola? === Nikola is a static site and blog generator, written in Python. It can use Mako and Jinja2 templates, and input in many popular markup formats, such as reStructuredText and Markdown — and can even turn Jupyter Notebooks into blog posts! It also supports image galleries, and is multilingual. Nikola is flexible, and page builds are extremely fast, courtesy of doit (which is rebuilding only what has been changed). Find out more at the website: https://getnikola.com/ Downloads = Install using `pip install Nikola`. Changes === Features * Add ``emphasize_lines`` directive to code blocks (Issue #3607) * Gallery index pages support the ``status`` flag (Issue #3598) * Add ``start_at`` option to youtube directive (Issue #3603) Bugfixes * Add data files to dependency file (Issue #3608) * Compatibility with doit 0.36.0 (Issue #3612) Note: ``nikola doit_auto`` is not available if using ``doit>=0.36.0``. You can still use ``nikola auto``. OpenPGP_signature Description: OpenPGP digital signature ___ Python-announce-list mailing list -- python-announce-list@python.org To unsubscribe send an email to python-announce-list-le...@python.org https://mail.python.org/mailman3/lists/python-announce-list.python.org/ Member address: arch...@mail-archive.com
[Python-announce] pytest-7.1.2
pytest 7.1.2 has just been released to PyPI. This is a bug-fix release, being a drop-in replacement. To upgrade:: pip install --upgrade pytest The full changelog is available at https://docs.pytest.org/en/stable/changelog.html. Thanks to all of the contributors to this release: * Anthony Sottile * Bruno Oliveira * Hugo van Kemenade * Kian Eliasi * Ran Benita * Zac Hatfield-Dodds Happy testing, The pytest Development Team ___ Python-announce-list mailing list -- python-announce-list@python.org To unsubscribe send an email to python-announce-list-le...@python.org https://mail.python.org/mailman3/lists/python-announce-list.python.org/ Member address: arch...@mail-archive.com
Re: upgrade pip
On 4/22/22, Tola Oj wrote: > im trying to upgrade my pip so i can install openpyxl. i though i had > successfully upgraded pip, and then I was trying to install openpyxl, but I > was getting this: > > C:\Users\ojomo>"C:\Program Files\Python310\python.exe" -m pip install > --upgrade > > [...] > > "C:\Users\ojomo\AppData\Roaming\Python\Python310\site-packages\pip\__main__.py", > line 29, in You have pip installed for the current user, but Python is installed for all users. I'd delete the package directory "%AppData%\Python\Python310\site-packages\pip". Then run ensurepip and upgrade from an administrator shell. This will install and update pip for all users. https://docs.python.org/3/library/ensurepip.html -- https://mail.python.org/mailman/listinfo/python-list
