Re: How to stop a specific thread in Python 2.7?

[Cameron Simpson via Python-list]

On 25Sep2024 22:56, marc nicole  wrote:

How to create a per-thread event in Python 2.7?


Every time you make a Thread, make an Event. Pass it to the thread 
worker function and keep it to hand for your use outside the thread.

--
https://mail.python.org/mailman/listinfo/python-list

Re: How to stop a specific thread in Python 2.7?

[Cameron Simpson via Python-list]

On 25Sep2024 19:24, marc nicole  wrote:

I want to know how to kill a specific running thread (say by its id)

for now I run and kill a thread like the following:
# start thread
thread1 = threading.Thread(target= self.some_func(), args=( ...,), )
thread1.start()
# kill the thread
event_thread1 = threading.Event()
event_thread1.set()

I know that set() will kill all running threads, but if there was thread2
as well and I want to kill only thread1?


No, `set()` doesn't kill a thread at all. It sets the `Event`, and each 
thread must be checking that event regularly, and quitting if it becomes 
set.


You just need a per-thred vent instead of a single Event for all the 
threads.


Cheers,
Cameron Simpson 
--
https://mail.python.org/mailman/listinfo/python-list

Re: Bug in 3.12.5

[Cameron Simpson via Python-list]

On 20Sep2024 12:52, Martin Nilsson  wrote:

The attached program doesn’t work in 3.12.5, but in 3.9 it worked.


This mailing list discards attachments.

Please include your code inline in the message text.

Thanks,
Cameron Simpson 
--
https://mail.python.org/mailman/listinfo/python-list

Re: python3 package import difference?

[Cameron Simpson via Python-list]

On 08Aug2024 21:55, Gilmeh Serda  wrote:
I guess in a sense Py2 was smarter figuring out what whent where and 
where

it came from. Or it was a bad hack that has been removed.


No, Python 2 offered less control.
--
https://mail.python.org/mailman/listinfo/python-list

Re: python3 package import difference?

[Cameron Simpson via Python-list]

On 07Aug2024 08:35, Tobiah  wrote:

When I do the same import with python3, I get:

   Traceback (most recent call last):
 File "/home/toby/me", line 1, in 
   import rcs.dbi
 File "/usr/regos-1.0/lib/python/rcs/dbi/__init__.py", line 1, in 
   from dbi import *
   ModuleNotFoundError: No module named 'dbi'


Python 3 imports are absolute (they start from the top of the package 
tree and you have no `dbi` at the top). You want a relative import i.e.:


    from .dbi import *

Cheers,
Cameron Simpson 
--
https://mail.python.org/mailman/listinfo/python-list

Re: Help needed - - running into issues with python and its tools

[Cameron Simpson via Python-list]

On 03Aug2024 16:34, o1bigtenor  wrote:
So please - - - how do I set up a venv so that I can install and run 
python

3.12
(and other needed programs related to 3.12) inside?


Maybe this github comment will help with this:
https://github.com/orgs/micropython/discussions/10255#discussioncomment-671
--
https://mail.python.org/mailman/listinfo/python-list

Re: Best use of "open" context manager

[Cameron Simpson via Python-list]

On 07Jul2024 22:22, Rob Cliffe  wrote:
it's legal, but doesn't work (trying to access the file after "with f" 
raises the same

    ValueError: I/O operation on closed file.


Just to this: of course. The with closes the file. But my version runs 
the with after the try/except.

--
https://mail.python.org/mailman/listinfo/python-list

Re: Best use of "open" context manager

[Cameron Simpson via Python-list]

On 07Jul2024 22:22, Rob Cliffe  wrote:
Remember, the `open()` call returns a file object _which can be used 
as a context manager_. It is separate from the `with` itself.

Did you test this?
    f = open(FileName) as f:
is not legal syntax.


No. You're right, remove the "as f:".

it's legal, but doesn't work (trying to access the file after "with f" 
raises the same

    ValueError: I/O operation on closed file.


This astounds me. Code snippet to demo this?

Here's a test script which I've just run now:

FileName = 'foo.txt'
try:
  f = open(FileName)
except FileNotFoundError:
  print(f"File {FileName} not found")
  sys.exit()
with f:
  for line in f:
print("line:", line.rstrip())

Here's the foo.txt file:

here are
some lines of text

Here's the run:

% python3 p.py
line: here are
line: some lines of text
--
https://mail.python.org/mailman/listinfo/python-list

Re: Best use of "open" context manager

[Cameron Simpson via Python-list]

On 06Jul2024 11:49, Rob Cliffe  wrote:

try:
    f = open(FileName) as f:
    FileLines = f.readlines()
except FileNotFoundError:
    print(f"File {FileName} not found")
    sys.exit()
# I forgot to put "f.close()" here -:)
for ln in File Lines:
        print("I do a lot of processing here")
        # Many lines of code here .


What about this:

try:
    f = open(FileName) as f:
except FileNotFoundError:
    print(f"File {FileName} not found")
    sys.exit()
with f:
... process the lines here ...

Remember, the `open()` call returns a file object _which can be used as 
a context manager_. It is separate from the `with` itself.

--
https://mail.python.org/mailman/listinfo/python-list

Re: Suggested python feature: allowing except in context maneger

[Cameron Simpson via Python-list]

On 14Jun2024 09:07, Yair Eshel  wrote:

Cameron, I'm not really sure I got your point. I've used the "file not
found" exception as an example for a behavior typical on context managers.
This could be a failure to connect to DB, or threads. It also applies to
any kind of possible exception, whether cased by the context manager itself
or the lines inside it. Long story short, this syntax change is as useful
as context managers are


The example exception is not what bothers me. The syntax change is 
nowhere near as useful as `with` and context managers. They provide an 
excellent idiom for resource usage and release.


Your suggestion complicates the `with` statement and brings only a tiny 
indentation reduction over the `with`-inside-`try` idiom. It brings no 
semantic changes or new features.


That is why I'm -1: the benefit is triviailly small to my eye.
--
https://mail.python.org/mailman/listinfo/python-list

Re: Anonymous email users

[Cameron Simpson via Python-list]

On 14Jun2024 18:00, avi.e.gr...@gmail.com  wrote:

I notice that in some recent discussions, we have users who cannot be
replied to directly as their email addresses are not valid ones, and I
believe on purpose. Examples in the thread I was going to reply to are:

 henha...@devnull.tb

[...]
I know some here suggest that we only reply to the wider community and 
they

have a point. But I think there is a role for having some conversations
offline and especially when they are not likely to be wanted, or even
tolerated, by many in the community.

Using such fake or invalid emails makes it hard to answer the person
directly or perhaps politely ask them for more info on their request or
discuss unrelated common interests. Worse, when I reply, unless I use
reply-all, my mailer sends to them futilely. When I do the reply-all, I have
to edit out their name or get a rejection.


I often reply-all (meaning to the list and to the author). And edit the 
headers (frankly, often just to discard anything @gmail.com which has 
very stupid spam poolicies).  If I miss an @invalid.com or whatever, 
then whoops.


If I want to reply directly (eg for some kind of feedback rather than a 
list type reply) and they've got a bogus address, well then I don't.  
Supposedly my reply would be of benefit for them or I shouldn't be doing 
it, so their loss. But equally, if they don't want personal off-list 
contact, they've expressed their preference. I should respect that.


Plenty of people have reasons to post anonymously, even to a list like 
this one. Just assume they've got their reasons and move on.

--
https://mail.python.org/mailman/listinfo/python-list

Re: Suggested python feature: allowing except in context maneger

[Cameron Simpson via Python-list]

On 13Jun2024 19:44, dieter.mau...@online.de  wrote:

Why not use:
```
try:
 with open()...
   ...
except FileNotFoundError:
 ...
```


This is exactly what the OP was expressing dissatisfaction with.

I'm -1 on the idea myself - not every combination of things needs 
additional syntactic support, and doing stuff like merging an `except` 
with a `wtih` is bound to introduce some weird corner case, complicating 
its semantics.


Cheers,
Cameron Simpson 
--
https://mail.python.org/mailman/listinfo/python-list

Re: Fwd: IDLE: clearing the screen

[Cameron Simpson via Python-list]

On 04Jun2024 22:43, Rob Cliffe  wrote:

import os
def cls(): x=os.system("cls")

Now whenever you type
cls()
it will clear the screen and show the prompt at the top of the screen.

(The reason for the "x=" is: os.system returns a result, in this case 
0.  When you evaluate an expression in the IDE, the IDE prints the 
result.  So without the "x=" you get an extra line at the top of the 
screen containing "0".)


Not if it's in a function, because the IDLE prints the result if it 
isn't None, and your function returns None. So:


def cls():
os.system("cls")

should be just fine.
--
https://mail.python.org/mailman/listinfo/python-list

Re: Flubbed it in the second interation through the string: range error... HOW?

[Cameron Simpson via Python-list]

On 29May2024 01:14, Thomas Passin  wrote:
Also, it's 2024 ... time to start using f-strings (because they are 
more readable than str.format())


By which Thomas means stuff like this:

print(f'if block {name[index]} and index {index}')

Notice the leading "f'". Personally I wouldn't even go that far, just:

print('if block', name[index], 'and index', index)

But there are plenty of places where f-strings are very useful.
--
https://mail.python.org/mailman/listinfo/python-list

Re: Terminal Emulator

[Cameron Simpson via Python-list]

On 14May2024 18:44, Gordinator  wrote:
I wish to write a terminal emulator in Python. I am a fairly competent 
Python user, and I wish to try a new project idea. What references can 
I use when writing my terminal emulator? I wish for it to be a true 
terminal emulator as well, not just a Tk text widget or something like 
that.


Start with the `pty` module.
--
https://mail.python.org/mailman/listinfo/python-list

Re: First two bytes of 'stdout' are lost

[Cameron Simpson via Python-list]

On 11Apr2024 14:42, Olivier B.  wrote:

I am trying to use StringIO to capture stdout, in code that looks like this:

import sys
from io import StringIO
old_stdout = sys.stdout
sys.stdout = mystdout = StringIO()
print( "patate")
mystdout.seek(0)
sys.stdout = old_stdout
print(mystdout.read())

Well, it is not exactly like this, since this works properly


Aye, I just tried that. All good.


This code is actually run from C++ using the C Python API.
This worked quite well, so the code was right at some point. But now,
two things changed:
- Now using python 3.11.7 instead of 3.7.12
- Now using only the python limited C API


Maybe you should post the code then: the exact Python code and the exact 
C++ code.



And it seems that now, mystdout.read() always misses the first two
characters that have been written to stdout.

My first ideas was something related to the BOM improperly truncated
at some point, but i am manipulating UTF-8, so the bom would be 3
bytes, not 2.


I didn't think UTF-8 needed a BOM. Somone will doubtless correct me.

However, does the `mystdout.read()` code _know_ you're using UTF-8? I 
have the vague impression that eg some Windows systems default to UTF-16 
of some flavour, possibly _with_ a BOM.


I'm suggesting that you rigorously check that the bytes->text bits know 
what text encoding they're using. If you've left an encoding out 
anywhere, put it in explicitly.



Hopefully someone has a clue on what would have changed in Python for
this to stop working compared to python 3.7?


None at all, alas. My experience with the Python C API is very limited.
--
https://mail.python.org/mailman/listinfo/python-list

Re: How to Add ANSI Color to User Response

[Cameron Simpson via Python-list]

On 10Apr2024 23:41, Alan Gauld  wrote:

Normally, for any kind of fancy terminal work, I'd say use curses.


My problem with curses is that it takes over the whole terminal - you 
need to manage everything from that point on. Great if you want it (eg 
some full-terminal tool like `top`) but complex overkill for small 
interactive (or not interactive) commands which are basicly printing 
lines of text. Which is what many of my scripts are.


That said, you don't _have_ to use curses to run the whole terminal. You 
can use it to just look up the terminal capabilities and use those 
strings. I haven't tried that for colours, but here's some same code 
from my `cs.upd` module using curses to look up various cursor motion 
type things:


... up the top ...
try:
  import curses
except ImportError as curses_e:
  warning("cannot import curses: %s", curses_e)
  curses = None

... later we cache the available motions ...
try:
  # pylint: disable=no-member
  curses.setupterm(fd=backend_fd)
except TypeError:
  pass
else:
  for ti_name in (
  'vi',  # cursor invisible
  'vs',  # cursor visible
  'cuu1',  # cursor up one line
  'dl1',  # delete one line
  'il1',  # insert one line
  'el',  # clear to end of line
  ):
# pylint: disable=no-member
s = curses.tigetstr(ti_name)
if s is not None:
  s = s.decode('ascii')
self._ti_strs[ti_name] = s

... then a method to access the cache ...
def ti_str(self, ti_name):
  ''' Fetch the terminfo capability string named `ti_name`.
  Return the string or `None` if not available.
  '''
  return self._ti_strs.get(ti_name, None)

... and even later, use the method ...
# emit cursor_up
cursor_up = self.ti_str('cuu1')
movetxts.append(cursor_up * (to_slot - from_slot))

Generally, when I add ANSI colours I do it via a little module of my 
own, `cs.ansi_colour`, which you can get from PyPI using `pip`.


The two most useful items in it for someone else are probably 
`colourise` and `colourise_patterns`. Link:

https://github.com/cameron-simpson/css/blob/26504f1df55e1bbdef00c3ff7f0cb00b2babdc01/lib/python/cs/ansi_colour.py#L96

I particularly use it to automatically colour log messages on a 
terminal, example code:

https://github.com/cameron-simpson/css/blob/26504f1df55e1bbdef00c3ff7f0cb00b2babdc01/lib/python/cs/logutils.py#L824
--
https://mail.python.org/mailman/listinfo/python-list

Re: Variable scope inside and outside functions - global statement being overridden by assignation unless preceded by reference

[Cameron Simpson via Python-list]

On 06Mar2024 15:12, Jacob Kruger  wrote:
So, this does not make sense to me in terms of the following snippet 
from the official python docs page:

https://docs.python.org/3/faq/programming.html

"In Python, variables that are only referenced inside a function are 
implicitly global. If a variable is assigned a value anywhere within 
the function’s body, it’s assumed to be a local unless explicitly 
declared as global."


So, I would then assume that if I explicitly include a variable name 
inside the global statement, then even just assigning it a new value 
should update the variable in the global context, outside the 
function?


Yes. Note that the "global" namespace is the module in which the 
function is defined.


x = 1

def f(n):
global x
x += n

This updates the `x` global variable in the module where `f` was 
defined.


If you import `f` and use it in another module it will _still_ update 
`x` in the original module namespace.

--
https://mail.python.org/mailman/listinfo/python-list

Re: Variable scope inside and outside functions - global statement being overridden by assignation unless preceded by reference

[Cameron Simpson via Python-list]

On 05Mar2024 20:13, Jacob Kruger  wrote:
Now, what almost seems to be occurring, is that while just manipulating 
the contents of a referenced variable is fine in this context, the 
moment I try to reassign it, that's where the issue is occurring .


Because there are no variable definitions in Python, when you write a 
function Python does a static analysis of it to decide which variables 
are local and which are not. If there's an assignment to a variable, it 
is a local variable.  _Regardless_ of whether that assignment has been 
executed, or gets executed at all (eg in an if-statement branch which 
doesn't fire).


You can use `global` or `nonlocal` to change where Python looks for a 
particular name.


In the code below, `f1` has no local variables and `f2` has an `x` and 
`l1` local variable.


x = 1
l1 = [1, 2, 3]

def f1():
print("f1 ...")
l1[1] = 5   # _not_ an assignment to "l1"
print("in f1, x =", x, "l1 =", l1)

def f2():
print("f2 ...")
x = 3
l1 = [6, 7, 9]  # assignment to "l1"
print("in f2, x =", x, "l1 =", l1)

print("outside, x =", x, "l1 =", l1)
f1()
    print("outside after f1, x =", x, "l1 =", l1)
f2()
print("outside after f2, x =", x, "l1 =", l1)

Cheers,
Cameron Simpson 
--
https://mail.python.org/mailman/listinfo/python-list

Re: Can one output something other than 'nan' for not a number values?

[Cameron Simpson via Python-list]

On 16Feb2024 22:12, Chris Green  wrote:

I'm looking for a simple way to make NaN values output as something
like '-' or even just a space instead of the string 'nan'.  This would
then make it much easier to handle outputting values from sensors when
not all sensors are present.

So, for example, my battery monitoring program outputs:-

   Battery Voltages and Currents
   Leisure Battery - 12.42 volts  -0.52 Amps
   Starter Battery - 12.34 volts  -0.01 Amps

If the starter battery sensor has failed, or is disconnected, I see:-

   Battery Voltages and Currents
   Leisure Battery - 12.42 volts  -0.52 Amps
   Starter Battery -   nan voltsnan Amps


What I would like is for those 'nan' strings to be just a '-' or
something similar.

Obviously I can write conditional code to check for float('nan')
values but is there a neater way with any sort of formatting string or
other sort of cleverness?


The simplest thing is probably just a function writing it how you want 
it:


def float_s(f):
if isnan(f):
return "-"
return str(f)

and then use eg:

print(f'value is {float_s(value)}')

or whatever fits your code.

Cheers,
Cameron Simpson 

--
https://mail.python.org/mailman/listinfo/python-list

Re: A question about import

[Cameron Simpson via Python-list]

On 16Feb2024 20:32, MRAB  wrote:

On 2024-02-16 20:07, Gabor Urban via Python-list wrote:

I need something about modules to be clarified.

Suppose I have written a module eg: ModuleA which imports an other
module, let us say the datetime.

If I import ModuleA in a script, will be datetime imported automatically?


Yes. When a module is imported it can import other modules.


But note that `datetime` does not magicly get put in the script's 
namespace.


Module A:

import datetime

Script:

import A

In the code in module A the name datetime is known and can be used.

In the code in the script the name A is known and can be used. Importing 
A does not magicly set the name datetime in the script's namespace - 
imagine the the pollution!


You _can_ access it as A.datetime because it is in the A module's 
namespace. But really if you just wanted datetime for direct use in the 
script you would import it there too:


import datetime
import A

Note that the datetime module is only actually loaded once. The import 
binds the name into your local namespace like any other variable.


Cheers,
Cameron Simpson 
--
https://mail.python.org/mailman/listinfo/python-list

Re: Is there a way to implement the ** operator on a custom object

[Cameron Simpson via Python-list]

On 09Feb2024 18:56, Left Right  wrote:

But, more to the point: extending collections.abc.Mapping may or may
not be possible in OP's case.


We don't yet know if that's what the OP had in mind yet, anyway.


Also, if you are doing this through inheritance, this seems really
convoluted: why not just inherit from dict? -- less methods to
implement, less stuff to import etc.


There's a rule of thumb that we _tend_ not to subclass the builtins; it 
certainly has its pitfalls to do with object creation/initialisation.  
That said, I have some classes which subclass dict, int, str and 
namedtuple.


Cheers,
Cameron Simpson 
--
https://mail.python.org/mailman/listinfo/python-list

Re: Is there a way to implement the ** operator on a custom object

[Cameron Simpson via Python-list]

On 08Feb2024 12:21, tony.fl...@btinternet.com  wrote:
I know that mappings by default support the ** operator, to unpack the 
mapping into key word arguments.


Has it been considered implementing a dunder method for the ** 
operator so you could unpack an object into a key word argument, and 
the developer could choose which keywords would be generated (or could 
even generate 'virtual' attributes).


Can you show us why you think that would look like in code?

Note that Python already has `a ** b` to raise `a` to the power of `b`, 
and it has a bunder method `__pow__` which you can define.

--
https://mail.python.org/mailman/listinfo/python-list

Re: How would you name this dictionary?

[Cameron Simpson via Python-list]

On 21Jan2024 23:39, bagra...@live.com  wrote:

class NameMe(dict):
   def __missing__(self, key):
   return key


I would need to know more about what it might be used for. What larger 
problem led you to writing a `dict` subclass with this particular 
`__missing__` implementation?

--
https://mail.python.org/mailman/listinfo/python-list

Re: Type hints - am I doing it right?

[Cameron Simpson via Python-list]

On 13Dec2023 09:19, Frank Millman  wrote:

I am adding type hints to my code base.

[...]

In the other module I have this -

 def config_database(db_params):

To add a type hint, I now have this -

 def config_database(db_params: configparser.SectionProxy):

To get this to work, I have to add 'import configparser' at the top of 
the module.


I have three separate modules, one for each database, with a subclass 
containing the methods and attributes specific to that database. Each 
one has a connect() method which receives db_params as a parameter. 
Now I have to add 'import configparser' at the top of each of these 
modules in order to type hint the method.


This seems verbose. If it is the correct way of doing it I can live 
with it, but I wondered if there was an easier way.


Not really. It's like any other name - it needs importing if you're 
going to use it.


You can make the hint itself more compact:

from configparser import SectionProxy
...
def config_database(db_params: SectionProxy):

Or you could be a bit more agnostic:

from typing import Mapping
...
def config_database(db_params: Mapping):

since I imagine config_database() would accept any kind of mapping 
(dicts, etc etc).


Cheers,
Cameron Simpson 
--
https://mail.python.org/mailman/listinfo/python-list

Re: Newline (NuBe Question)

[Cameron Simpson via Python-list]

On 15Nov2023 07:25, Grizzy Adams  wrote:

Have this (from an online "classes" tutorial)


Response inline below.


students = []
grades = []
for s in geographyClass:
students.append(geographyStudent(s))
for s in students:
   grades.append(s.school)
   grades.append(s.name)
   grades.append(s.finalGrade())
   if s.finalGrade()>82:
   grades.append("Pass")
   else:
   grades.append("Fail")
print(grades)

--- End Code Snippit  ---

I have extended (from tutorial) it a bit, I would really like to have a newline
at end of each record, I have searched (and tested) but cant get "\n" 
to give a newline, I get "Mydata\n"


It would be useful to:
- see some of the things you've tried
- what their output actually was
- what output you actually want to achieve


Do I need to replace "append" with "print", or is there a way to get the
newline in as I append to list?


I think you're confusing output (print) with your data (the list of 
grades).


Is the code above genuinely what you're running?

I ask because it looks to me that you're:
- appending all the individual grade fields (school, name, ...) to one 
  long grades list containing all the data from all the students

- you're printing that single enormous list in one go at the end

What I'm imagine you want is one line of grade information per student.

Remember that indentation is important in Python. You're grades code 
looks like this:


grades = []
for s in students:
grades.append(s.school)
grades.append(s.name)
grades.append(s.finalGrade())
if s.finalGrade()>82:
grades.append("Pass")
else:
grades.append("Fail")
print(grades)

This:
- makes an empty list
- gathers up all of the student data
- prints the data in one go

I think you may want to do the first and last steps on a per student 
basis, not just once at the start and the end. So you might want to 
rearrange things:


for s in students:
grades = []
grades.append(s.school)
grades.append(s.name)
grades.append(s.finalGrade())
if s.finalGrade()>82:
grades.append("Pass")
else:
grades.append("Fail")
print(grades)

Cheers,
Cameron Simpson 
--
https://mail.python.org/mailman/listinfo/python-list

Re: Checking if email is valid

[Cameron Simpson via Python-list]

On 02Nov2023 17:04, Chris Angelico  wrote:

On Thu, 2 Nov 2023 at 15:20, AVI GROSS via Python-list
 wrote:
Yes, it would be nice if there was a syntax for sending a test 
message sort

of like an ACK that is not delivered to the recipient but merely results in
some status being sent back such as DELIVERABLE or NO SUCH USER or even
MAILBOX FULL.


Yes, it would! Spammers would be able to use this syntax to figure out
exactly which addresses actually have real people connected to it. It
would save them so much trouble! Brilliant idea.


Hmm. IIRC...

https://datatracker.ietf.org/doc/html/rfc2821#section-4.1.1.6

I think a lot of mail receivers don't honour this one, for exactly the 
reasons above.


Cheers,
Cameron Simpson 
--
https://mail.python.org/mailman/listinfo/python-list

Re: Checking if email is valid

[Cameron Simpson via Python-list]

On 01Nov2023 14:08, Grant Edwards  wrote:

On 2023-11-01, Simon Connah via Python-list  wrote:

I'm building a simple project using smtplib and have a
question. I've been doing unit testing but I'm not sure how to check
if an email message is valid.

[...]

Could someone push me in the right direction please? I just want to
find out if a string is a valid email address.


I confess that I punt "syntactically valid" to email.utils.getaddresses:
https://docs.python.org/3/library/email.utils.html#email.utils.getaddresses

"Deliverable"? I'm prepared to just send to it and hope not to get a 
bounce; delivery of email is, after all, asynchronous. (Even if it were 
going direct to the primary MX, I still hand it to the local mail system 
to deal with.)


"A real person or entity"? A lot of systems do the round trip thing: 
here's a special unique and opaue URL,please visit it to confirm receipt 
of this email (implying email is 'real"). You see this a lot when 
signing up for things. And for plenty of things I generate a random 
throw away address at mailinator.com (looking at you, every "catch up" 
free online TV streaming service who still wants me to log in).


Cheers,
Cameron Simpson 
--
https://mail.python.org/mailman/listinfo/python-list

Re: return type same as class gives NameError.

[Cameron Simpson via Python-list]

On 22Oct2023 17:50, Antoon Pardon  wrote:

I have the following small module:
=-=-=-=-=-=-=-=-=-=-=-= 8< =-=-=-=-=-=-=-=-=-=-=-=-=
class Pnt (NamedTuple):
   x: float
   y: float

   def __add__(self, other: PNT) -> Pnt:
   return Pnt(self[0] + other[0], self[1] + other[1])


When this function is defined, the class "Pnt" has not yet been defined.  
That happens afterwards.


You want a forward reference, eg:

def __add__(self, other: PNT) -> "Pnt":

A type checker will resolve this after the fact, when it encounters the 
string in the type annotation.


This message:

NameError: name 'Pnt' is not defined. Did you mean: 'PNT'?

is unfortunate, because you have a very similar "PNT" name in scope. But 
it isn't what you want.


Cheers,
Cameron Simpson 
--
https://mail.python.org/mailman/listinfo/python-list

Re: Where I do ask for a new feature

[Cameron Simpson via Python-list]

On 19Oct2023 20:16, Bongo Ferno  wrote:
A with statement makes clear that the alias is an alias and is local, 
and it automatically clears the variable after the block code is used.


No it doesn't:

>>> with open('/dev/null') as f:
...   print(f)
...
<_io.TextIOWrapper name='/dev/null' mode='r' encoding='UTF-8'>
>>> print(f)
<_io.TextIOWrapper name='/dev/null' mode='r' encoding='UTF-8'>
--
https://mail.python.org/mailman/listinfo/python-list

Re: Why doc call `__init__` as a method rather than function?

[Cameron Simpson via Python-list]

On 15Sep2023 10:49, scruel tao  wrote:

```python

class A:

...   def __init__(self):
... pass
...
```

On many books and even the official documents, it seems that many authors prefer to call `__init__` 
as a "method" rather than a "function".
The book PYTHON CRASH COURSE  mentioned that "A function that’s part of a class is a 
method.", however, ` A.__init__` tells that `__init__` is a function...


As mentioned, methods in Python _are_ functions for use with a class.


A.__init__



a = A()
a.__init__


>
I wonder how can I call `__init__` as? Consider the output above.
Maybe both are OK?


As you can see, they're both legal expressions.

The thing about `__init__` is that it's usually called automatically 
which you make a new object. Try putting a `print()` call in your 
`__init__` method, then make a new instance of `A`.


The purpose of `__init__` is to initialise the object's attribute/state 
after the basic, empty-ish, object is made.


Like other dunder methods (methods named with double underscores front 
and back) it is called automatically for you. In a subclass the 
`__init__` method calls the subperclass `__init__` and then does 
whatever additional things might be wanted by the subclass.


Let's look at what you used above:

>>> A.__init__


Here's we've just got a reference to the function you supposlied with 
the class definition for class `A`.


This:

>>> a = A()
>>> a.__init__


Here's you've accessed the name `__init__` via an existing instance of 
`A`, your variable `a`. At this point you haven't called it. So you've 
got a callable thing which is a binding of the function to the object 
`a` i.e. when you call it, the "bound method" knows that t is associated 
with `a` and puts that in as the first argument (usually named `self`).


Cheers,
Cameron Simpson 
--
https://mail.python.org/mailman/listinfo/python-list

Re: Imports and dot-notation

[Cameron Simpson via Python-list]

On 09Aug2023 12:30, Oliver Schinagl  wrote:
Looking at a python projects code and repository layout, we see the 
following directory structure.



/project/core
/project/components/module1
...
/project/components/moduleN
/projects/util

(this is far from complete, but enough to help paint a picture.

Some modules import other modules, and I see (at the very least) two 
(kind of three?) different ways of doing so.


`from project.components.module1 import function1, function2 as func, 
CONST1, CONST2 as CONST`


or maybe even (which has as an advantage that it becomes clear which 
namespace something belongs to


`from project.components.module1 import function1, function2 as 
module1_function2, CONST1, CONST2 as MODULE1_CONST2`


but then it really just becomes personal preference, as the number of 
characters saved on typing is almost negative (due to having a more 
complex import).



but also the dot notation being used

`from project.components import module1`

where obviously the functions are invoked as `module1.function1` etc

I hope the above is clear enough from an example.


Simply put, which of the two would be considered cleaner and more pythonic?


This is almost entirely a programmer preference thing.

The Zen, as usual, offers guideance but not dictates.

As you say, the module1.func form is very clear and has its beauty.  
Personally I lean towards the single name flavour, sometimes with a 
rename. FOr example, with os.path I often go:


from os.path import join as joinpath

and likewise for several other names from there, because the bare names 
are very generic (eg "join"). I see to be alone here. Many other people 
use:


import os.path

and use the full os.path.join in their code, which I find jarring and 
visually noisy.


Now for a bit more thought, looking at PEP8, we notes about imports, 
but sadly PEP8 does not state which method is better/preferred. While 
obviously in the end, it's always the maintainers choice in what they 
prefer, so are tabs vs spaces and PEP8 is opinionated about that too 
(in a good way, even though I'm not a fan :p).


PEP8 is for the stdlib source code; that said it is also the basis for 
most Python coding styles in the wild.


For me, the Zen's "readability counts" is the most important guideline; 
and to this end, your "module1.function" is an entirely reasonable 
response, particularly if you've got a lot of imports. But it does make 
for more verbose code, and that works against readability to a degree.


This is why it's all subjective.

Particular workplaces may mandate particular styles, but in your 
personal code? Do what seem best, and experience will cause that to 
evolve over time.


Also, the style can depend on the code you're working on. For a small 
module with few imports the:


from module1 import funcname

can be a win because the code is short, and this short form of funcname 
is both clear and readable. But in a very long piece of code with many 
imports you might go for module1.funcname for clarity, particularly if 
funcname is generic or overlaps with another similar imported name.


Cheers,
Cameron Simpson 
--
https://mail.python.org/mailman/listinfo/python-list

Re: Where is the error?

[Cameron Simpson via Python-list]

On 07Aug2023 08:02, Barry  wrote:

On 7 Aug 2023, at 05:28, Cameron Simpson via Python-list 
 wrote:
Used to use a Pascal compiler once which was uncannily good at 
suggesting where you'd missing a semicolon.


Was that on DEC VMS? It was a goal at DEC for its compilers to do this well.


No, a PDP-11 using V7 UNIX.


They could output the errors in a machine readable format to allow editors to 
auto fix.
I am learning rust and it is very good at suggesting fixes.
There is a command to apply fixes automatically, cargo fix.


Neat.

Cheers,
Cameron Simpson 
--
https://mail.python.org/mailman/listinfo/python-list

Re: Where is the error?

[Cameron Simpson via Python-list]

On 06Aug2023 22:41, Peter J. Holzer  wrote:

Mostly, error messages got a lot better in Python 3.10, but this one had
me scratching my head for a few minutes.

Consider this useless and faulty script:

r = {
   "x": (1 + 2 + 3)
   "y": (4 + 5 + 6)
   "z": (7 + 8 + 9)
}


[...]

Python 3.10 and 3.11 report:


 File "/home/hjp/tmp/foo", line 2
   "x": (1 + 2 + 3)
 ^^
SyntaxError: invalid syntax. Perhaps you forgot a comma?


The error message is now a lot better, of course, but the fact that it
points at the expression *before* the error completely threw me. The
underlined expression is clearly not missing a comma, nor is there an
error before that.


Well, it's hard to underline a token which isn't present. But maybe the 
message could be more evocative:


SyntaxError: invalid syntax. Perhaps you forgot a comma after the 
underlined code?


Is this "clairvoyant" behaviour a side-effect of the new parser or was
that a deliberate decision?


I have the vague impression the new parser enabled the improved 
reporting.


Used to use a Pascal compiler once which was uncannily good at 
suggesting where you'd missing a semicolon.


Cheers,
Cameron Simpson 
--
https://mail.python.org/mailman/listinfo/python-list

Re: isinstance()

[Cameron Simpson via Python-list]

On 03Aug2023 10:14, dn  wrote:
Can you please explain why a multi-part second-argument must be a 
tuple and not any other form of collection-type?


The signature is: isinstance(object, classinfo)
leading to "classinfo" of:

1/ a single class/type, eg int
2/ a tuple of same, eg ( int, str, )
3/ a union type, eg int | str (v3.10+)


help(isinstance) (in 3.8) says class_or_tuple.

I would speculate that requiring a tuple has two advantages:
- it may lend itself to promotion to a set for comparison with the MRO 
  of object (IIRC the "foo in (a,b,c)" is optimised this way also), and 
  this in turn may hint at the Hashable requirement
- it is _frugal_ in what we expect there, leaving open a future meaning 
  for something else in that place (for example, isinstance well 
  predates type annotations, and a looser kind of argument might have 
  precluded this new usage)


There's similar language in this try/except documentation:
file:///Users/cameron/var/doc/python/3.8.0/reference/compound_stmts.html#index-10

 For an except clause with an expression, that expression is 
 evaluated, and the clause matches the exception if the resulting 
 object is “compatible” with the exception. An object is compatible 
 with an exception if it is the class or a base class of the 
 exception object or a tuple containing an item compatible with the 
 exception.


To my mind the tuple requirement lends itself to a distinct syntax (the 
brackets) and frugal use of the meaning of what values can occur there.


Cheers,
Cameron Simpson 
--
https://mail.python.org/mailman/listinfo/python-list

Re: Should NoneType be iterable?

[Cameron Simpson via Python-list]

On 21Jun2023 10:09, Chris Angelico  wrote:

On Wed, 21 Jun 2023 at 09:59, Cameron Simpson via Python-list
 wrote:
I wasted some time the other evening on an API which returned a 
string

or None. My own API, and the pain it caused tells me that that API
design choice isn't good (it's an automatic attribute based on a tag,
returning None if the tag isn't there). My experience so far is that it
_looks_ handy so that you can safely say "foo.bar" all the time, but as
soon a you do something with the value you're in for a world of
None-checking.


https://peps.python.org/pep-0505/



Hmm. Thanks. - Cameron Simpson 
--
https://mail.python.org/mailman/listinfo/python-list

Re: Should NoneType be iterable?

[Cameron Simpson via Python-list]

On 21Jun2023 03:01, Greg Ewing  wrote:

On 20/06/23 7:36 pm, Barry wrote:

I have some APIs that do return None or a list.
The None says that a list is not available and that the caller is
responsible with dealing in a application-domain specific with
with that situation.


In that case, the caller should probably be checking for
None rather than blindly trying to iterate over the result.


I wasted some time the other evening on an API which returned a string 
or None. My own API, and the pain it caused tells me that that API 
design choice isn't good (it's an automatic attribute based on a tag, 
returning None if the tag isn't there). My experience so far is that it 
_looks_ handy so that you can safely say "foo.bar" all the time, but as 
soon a you do something with the value you're in for a world of 
None-checking.


I'm rethinking that choice right now. Just the other day I removed a 
setting in a related class which provided an automatic default value 
because, again, while handy for careless use it caused hard to debug 
problems because the default would flow out the call chain until it was 
unsuitable, making the cause hard to trace.


And of course I'm very -1 on None acquiring iteration or other features.  
Fail early, fail often!


Cheers,
Cameron Simpson 
--
https://mail.python.org/mailman/listinfo/python-list

Re: Tkinter docs?

[Cameron Simpson]

On 24May2023 02:18, Rob Cliffe  wrote:

    There doesn't seem to be any decent documentation for it anywhere.


Already mentioned in the replies, I use this: 
https://tkdocs.com/shipman/index.html

quite a lot.
--
https://mail.python.org/mailman/listinfo/python-list

Re: Tkinter (related)~

[Cameron Simpson]

On 18May2023 12:06, Jack Dangler  wrote:
I thought the OP of the tkinter thread currently running may have 
needed to install the tkinter package (since I had the same missing 
component error message), so I tried to install the package on to my 
Ubu laptop -


pip install tkinter
Defaulting to user installation because normal site-packages is not 
writeable
ERROR: Could not find a version that satisfies the requirement tkinter 
(from versions: none)

ERROR: No matching distribution found for tkinter

Is there an alternate path to installing this?


Usually tkinter ships with Python because it is part of the stdlib.

On some platforms eg Ubuntu Linux the stdlib doesn't come in completely 
unless you ask - a lot of stdlib packages are apt things you need to ask 
for. On my Ubunut here tkinter comes from python3-tk. So:


$ sudo apt-get install python3-tk

Cheers,
Cameron Simpson 
--
https://mail.python.org/mailman/listinfo/python-list

Re: What to use instead of nntplib?

[Cameron Simpson]

On 16May2023 09:26, Alan Gauld  wrote:

On 15/05/2023 22:11, Grant Edwards wrote:

I got a nice warning today from the inews utility I use daily:

DeprecationWarning: 'nntplib' is deprecated and slated for removal in 
Python 3.13

What should I use in place of nntplib?


I'm curious as to why nntplib is deprecated? Surely there are still a
lot of nntp servers around, both inside and outside corporate firewalls?
Is there a problem with the module or is it just perceived as no longer
required?


See PEP 594: https://peps.python.org/pep-0594/
--
https://mail.python.org/mailman/listinfo/python-list

Re: What do these '=?utf-8?' sequences mean in python?

[Cameron Simpson]

On 08May2023 12:19, jak  wrote:
In reality you should also take into account the fact that if the 
header

contains a 'b' instead of a 'q' as a penultimate character, then the
rest of the package is converted on the basis64

"=?utf-8?Q?"  --> "=?utf-8?B?"


Aye. Specification:

https://datatracker.ietf.org/doc/html/rfc2047

You should reach for jak's suggested email.header suggestion _before_ 
parsing the subject line. Details:


https://docs.python.org/3/library/email.header.html#module-email.header

Cheers,
Cameron Simpson 
--
https://mail.python.org/mailman/listinfo/python-list

Re: double bracket integer index in pandas; Is this a legal syntax

[Cameron Simpson]

On 03May2023 17:52, Artie Ziff  wrote:

The code came from a video course, "Pandas Data Analysis with Python
Fundamentals" by Daniel Chen.

I am curious why the author may have said this. To avoid attaching
screenshots, I'll describe this section of the content. Perhaps someone can
say, "oh that's how it used to work"... haha


Unlikely; Python indices (and by implication Pandas indices) have 
counted from 0 since forever.  I suspect just a typo/braino.



D.CHEN:
"You can also subset the columns by number. If we wanted to get the first
column from our data set, we would use zero":

df = pandas.read_csv('./data/gapminder.tsv', sep='\t')

subset = df[[0]]
print(subset.head())

  country
0  Afghanistan
1  Afghanistan
2  Afghanistan
3  Afghanistan
4  Afghanistan

Data for the course:
https://github.com/chendaniely/pandas_for_everyone.git

"df[[0]]" is being described to the course student as selecting the first
column of data.  :-)


Well, I would say it makes a new dataframe with just the first column.

So:

df[ 0 ] # spaces for clarity

would (probably, need to check) return the Series for the first column.  
versus:


df[ [0] ]   # spaces for clarity

makes a new dataframe with only the first column.

A dataframe can be thought of as an array of Series (one per column).

Cheers,
Cameron Simpson 
--
https://mail.python.org/mailman/listinfo/python-list

Re: double bracket integer index in pandas; Is this a legal syntax

[Cameron Simpson]

On 03May2023 03:41, Artie Ziff  wrote:

I am hope that pandas questions are OK here.


There are some pandas users here.


In a panda lecture, I did not get the expected result.
I tried this on two different platforms
(old macOS distro and up-to-date Ubuntu Linux distro, 22.04)

The Linux distro has:
python3.10.11
pandas1.5.2
conda 23.3.1

Is this double bracket form, df[[1]], deprecated... maybe?

There is data in a dataframe, df.


subset = df[[1]]


Whether this works depends on the contents of the dataframe. You're 
supplying this index:


[1]

which is a list of ints (with just one int).

Have a look at this page:
https://pandas.pydata.org/docs/user_guide/indexing.html

If you suppply a list, it expects a list of labels. Is 1 a valid label 
for your particular dataframe?


Cheers,
Cameron Simpson 
--
https://mail.python.org/mailman/listinfo/python-list

Re: How to 'ignore' an error in Python?

[Cameron Simpson]

On 28Apr2023 10:39, Mats Wichmann  wrote:

For this specific case, you can use os.makedirs:
os.makedirs(dirname, exist_ok=True)


I'm not a great fan of makedirs because it will make all the missing 
components, not just the final one. So as an example, if you've got a 
NAS mounted backup area at eg:


/mnt/nas/backups/the-thing/backups-subdir

and your config file has this mistyped as:

/mn/nas/backups/the-thing/backups-subdir

and makedirs is used, then it will make the backup area on eg the root 
drive where there's no room. (I'm looking at you, Docker, grr). There 
are plenty of similar situations.


Because of this I usually am prepared to make a missing final component 
with mkdir(), but not a potentially deep path with makedirs().


Cheers,
Cameron Simpson 
--
https://mail.python.org/mailman/listinfo/python-list

Re: How to 'ignore' an error in Python?

[Cameron Simpson]

On 28Apr2023 16:55, Chris Green  wrote:

   for dirname in listofdirs:
   try:
   os.mkdir(dirname)
   except FileExistsError:
   # so what can I do here that says 'carry on regardless'
   except:
   # handle any other error, which is really an error

   # I want code here to execute whether or not dirname exists

Do I really have to use a finally: block?  It feels rather clumsy.


You don't. Provided the "handle any other error" part reraises or does a 
break/continue, so as to skip the bottom of the loop body.


ok = true
for dirname in listofdirs:
try:
os.mkdir(dirname)
except FileExistsError:
pass
except Exception as e:
warning("mkdir(%r): %s", dirname, e)
ok = False
continue
rest of the loop body ...
if not ok:
... not all directories made ...

2 notes on the above:
- catching Exception, not a bare except (which catches a rather broader 
  suit of things)

- reporting the other exception

Cheers,
Cameron Simpson 
--
https://mail.python.org/mailman/listinfo/python-list

Re: Question regarding unexpected behavior in using __enter__ method

[Cameron Simpson]

On 21Apr2023 00:44, Lorenzo Catoni  wrote:

I am writing to seek your assistance in understanding an unexpected
behavior that I encountered while using the __enter__ method. I have
provided a code snippet below to illustrate the problem:

```

class X:

... __enter__ = int
... __exit__ = lambda *_: None
...

with X() as x:

... pass
...

x

0
```
As you can see, the __enter__ method does not throw any exceptions and
returns the output of "int()" correctly. However, one would normally expect
the input parameter "self" to be passed to the function.


My descriptor fu is weak, but I believe this is because `int` is not a 
plain function but a type.


Consider this class definition:

class X:
   x = 1
   def y(self):
   return "y"

When you define a class, the body of the class is run in a namespace, 
and on completion, the namespace is _used_ to construct the class.  
During that process, the various names are considered. Here we've got 2 
names: "x" and "y".


"x" refers to an int and is just stored as a class attribute, unchanged.

"y" refers to a function, and is promoted to a descriptor of an unbound 
method.


So later: X.x return 1 but X.y returns a unbound method. If we make an 
instance:


objx = X()

then obj.x returns 1 (by not fining an "x" on "obj", but finding one on 
"type(obj)" i.e. the class attribute.


By contrast, obj.y returns a bound method, a function already curried 
with a leading parameter "obj" (which will be "self"). There's no "y" 
attribute directly on "obj" but there's an unbound method on 
"type(obj).y", which gets bound by saying "obj.y".


The means that what happens to a name when you define the class depends 
on the typeof the value bound to the name.


A plain function gets turned into an unbound instance method, but other 
things are left alone.


When you went:

__enter__ = int

That's not a plain function and so "obj.__enter__" doesn't turn into a 
bound method - it it just `int`.


Cheers,
Cameron Simpson 
--
https://mail.python.org/mailman/listinfo/python-list

Re: Fwd: pip is not installed

[Cameron Simpson]

On 16Apr2023 16:54, Khoinguyen Nguyen  wrote:
I have tried to reinstall, repair, and run in command prompt, but it 
seems

as though pip won't install. I am using Windows 10 and Python 3.11.3. Are
there any other suggestions for troubleshooting?


Have you tried invoking pip _via_ Python? This is actually the 
recommended approach because it uses the pip for that specific Python.  
Example:


py -m pip install foo

This invokes the "pip" module for thePython invoked by "py" (which I 
believe is how Python is normally invoked from a Windows command line), 
to install some package "foo".


Cheers,
Cameron Simpson 
--
https://mail.python.org/mailman/listinfo/python-list

Re: Weak Type Ability for Python

[Cameron Simpson]

On 13Apr2023 03:36, MRAB  wrote:
I thought that in Java you can, in fact, concatenate a string and an 
int, so I did a quick search online and it appears that you can.


I stand corrected. I could have sworn it didn't, but it has been a long 
time. - Cameron Simpson 

--
https://mail.python.org/mailman/listinfo/python-list

Re: Weak Type Ability for Python

[Cameron Simpson]

On 12Apr2023 22:12, avi.e.gr...@gmail.com  wrote:
I suspect the OP is thinking of languages like PERL or JAVA which guess 
for

you and make such conversions when it seems to make sense.


JavaScript guesses. What a nightmare. Java acts like Python and will 
forbid it on type grounds (at compile time with Java, being staticly 
typed).


Cheers,
Cameron Simpson 
--
https://mail.python.org/mailman/listinfo/python-list

Re: Looking for package/library to extract MP4 metadata

[Cameron Simpson]

On 10Apr2023 13:10, Chris Green  wrote:

command:
ffprobe -v warning -i "input.mp4" -show_streams -of json
python:
Popen(command, stderr=STDOUT, stdout=PIPE, encoding='utf8')
json:
json.loads(''.join(p.stdout.readlines()))

It's easy to find a version of ffmpeg/ffprobe for every platform.


Thank you, that worked straight away, ffprobe is installed on my
systems already and I can probably just grep for the tag I want as all
I'm looking for is the date of its creation which appears (twice) with
the tag "creation_time".

This is just to handle the occasional MP4 that a python program which
basically manages JPEGs can't handle.  It throws an exception so I can
just get that to run a simple bash script to get the creation date.


Yes, ffprobe is great, particularly the JSON output mode, very parsable.

If you want to get into the knitty gritty you could try my `cs.iso14496` 
package, which has a full MP4/MOV parser and a hook for getting the 
metadata.


Not as convenient as ffprobe, but if you care about the innards...

Cheers,
Cameron Simpson 
--
https://mail.python.org/mailman/listinfo/python-list

Re: Python file location

[Cameron Simpson]

On 29Mar2023 13:47, windhorn  wrote:
I have an older laptop I use for programming, particularly Python and 
Octave, running a variety of Debian Linux, and I am curious if there is 
a "standard" place in the file system to store this type of program 
file. OK, I know they should go in a repository and be managed by an 
IDE but this seems like way overkill for the kind of programming that I 
do, normally a single file. Any suggestions welcome, thanks.


Are you writing personal scripts? ~/bin then.

Person modules you import in your personal scripts? I put these in 
~/lib/python/cs/ for my modules, which are all named "cs.*" (avoids 
conflict). But that's just me.


Cheers,
Cameron Simpson 
--
https://mail.python.org/mailman/listinfo/python-list

Re: Standard class for time *period*?

[Cameron Simpson]

On 29Mar2023 08:17, Loris Bennett  wrote:

I am glad to hear that I am not alone :-) However, my use-case is fairly
trivial, indeed less complicated than yours.  So, in truth I don't
really need a Period class.  I just thought it might be a sufficiently
generic itch that someone else with a more complicated use-case could
have already scratched.


Well, my attitude to time spans is to get the start and end (or start 
and length, but I prefer the former) as UNIX timestamps and work from 
that. I try to stay as far from datetimes as possible, because of their 
foibles (particularly timezones, but really the whole calendar 
decomposition etc) and use them only for presentation.


I do in fact have a `TimePartition` in my timeseries module; it 
presently doesn't do comparisons because I'm not comparing them - I'm 
just using them as slices into the timeseries data on the whole.


https://github.com/cameron-simpson/css/blob/0ade6d191833b87cab8826d7ecaee4d114992c45/lib/python/cs/timeseries.py#L2163

But it would be easy to give that class `__lt__` etc methods.

You're welcome to use it, or anything from the module (it's on PyPI).

Cheers,
Cameron Simpson 
--
https://mail.python.org/mailman/listinfo/python-list

Re: Standard class for time *period*?

[Cameron Simpson]

On 30Mar2023 10:13, Cameron Simpson  wrote:
I do in fact have a `TimePartition` in my timeseries module; it 
presently doesn't do comparisons because I'm not comparing them - I'm 
just using them as slices into the timeseries data on the whole.


https://github.com/cameron-simpson/css/blob/0ade6d191833b87cab8826d7ecaee4d114992c45/lib/python/cs/timeseries.py#L2163


On review it isn't a great match for a simple time range; it's aimed at 
expressing the time ranges into which my time series data files are 
broken up.


I think most of the code using this particular class just starts with a 
`start_unixtime` and `end_unixtime` and indexes whichever timeseries 
it's using, and these come into play internally to access the relevant 
files. There are a bunch of methods to take a pair of start/stop 
unixtimes and return time slot indices etc.


So even I don't have a purely time period class, right now anyway; 
passing around a pair of floats is so simple that I probably haven't 
elaborated on it. Just made ways to make that pair from what I'm given.


Cheers,
Cameron Simpson 
--
https://mail.python.org/mailman/listinfo/python-list

Re: Standard class for time *period*?

[Cameron Simpson]

On 28Mar2023 08:05, Dennis Lee Bieber  wrote:
	So far, you seem to be the only person who has ever asked for a 
	single

entity incorporating an EPOCH (datetime.datetime) + a DURATION
(datetime.timedelta).


But not the only person to want one. I've got a timeseries data format 
where (within a file) time slots are represented as a seconds offset, 
and the file has an associated epoch starting point. Dual to that is 
that a timeslot has an offset from the file start, and that is 
effectively a (file-epoch, duration) notion.


I've got plenty of code using that which passes around UNIX timestamp 
start/stop pairs. Various conversions happen to select the appropriate 
file (this keeps the files of bounded size while supporting an unbounded 
time range).


Even a UNIX timestamp has an implied epoch, and therefore kind of 
represents that epoch plus the timestamp as a duration.


I'm not sure I understand Loris' other requirements though. It might be 
hard to write a general thing which was also still useful.


Cheers,
Cameron Simpson 
--
https://mail.python.org/mailman/listinfo/python-list

Re: How does a method of a subclass become a method of the base class?

[Cameron Simpson]

On 27Mar2023 12:03, Cameron Simpson  wrote:

On 27Mar2023 01:53, Jen Kris  wrote:
But that brings up a new question.  I can create a class instance with 
x = BinaryConstraint(),


That makes an instance of EqualityConstraint.


Copy/paste mistake on my part. This makes an instance of 
BinaryConstraint.


Apologies,
Cameron Simpson 
--
https://mail.python.org/mailman/listinfo/python-list

Re: How does a method of a subclass become a method of the base class?

[Cameron Simpson]

On 27Mar2023 01:53, Jen Kris  wrote:
Thanks for your reply.  You are correct about the class definition 
lines – e.g. class EqualityConstraint(BinaryConstraint).  I didn’t post 
all of the code because this program is over 600 lines long.  It's 
DeltaBlue in the Python benchmark suite. 


Doubtless. But the "class ...:" line is the critical thing.


I’ve done some more work since this morning, and now I see what’s happening.  
But it gave rise to another question, which I’ll ask at the end. 

The call chain starts at

    EqualityConstraint(prev, v, Strength.REQUIRED) 

The class EqualityConstraint is a subclass of BinaryConstraint.  The entire 
class code is:

    class EqualityConstraint(BinaryConstraint):
    def execute(self):
    self.output().value = self.input().value


Ok, so it is effectively a BinaryConstraint with a single customised 
execute() method.



Because EqualityConstraint is a subclass of BinaryConstraint, the init method 
of BinaryConstraint is called first.


That is courtesy of the MRO: EqualityConstraint has no __init__, so the 
one found in BinaryConstraint is found and used.


During that initialization (I showed the call chain in my previous 
message), it calls choose_method.  When I inspect the code at 
"self.choose_method(mark):" in PyCharm, it shows:


    >


This says that it found found the choose_method method in the 
BinaryConstraint class definition, but that it is _bound_ to the 
EqualityConstraint instance you're using (self).



As EqualityConstraint is a subclass of BinaryConstraint it has bound the choose 
method from BinaryConstraint, apparently during the BinaryConstraint init 
process, and that’s the one it uses.  So that answers my original question. 


Incorrectly, I think.

__init__ does _nothing_ about other methods, if you mean "__init__" in 
the sentence above.


The class definition results in an EqualityConstraint.__mro__ list of 
classes, which is where _every_ method is looked for _at call time_.


But that brings up a new question.  I can create a class instance with 
x = BinaryConstraint(),


That makes an instance of EqualityConstraint.

but what happens when I have a line like "EqualityConstraint(prev, v, 
Strength.REQUIRED)"?


That makes an instance of EqualityConstraint.


Is it because the only method of EqualityConstraint is execute(self)?


I don't know what you're asking here.


Is execute a special function like a class __init__?


No. It's just a method.

In a sense, __init__ isn't very special either. It is just _called_ 
during new instance setup as the final step. By that time, self is a 
fully set up instance of... whatever and the __init__ function is just 
being called to do whatever initial attribute setup every instance of 
that class needs.


Python doesn't magicly call _all_ the __init__ methods of all the 
superclasses; it calls the first found. So for an EqualityConstraint it 
finds the one from BinaryConstraint because EqualityConstraint does not 
provide an overriding one of its own.


If EqualityConstraint _did_ provide one, it might typically be 
structured as:


class EqualityConstraint(BinaryConstraint):
def __init__(self, prev, v, strength, something_extra):
super().__init__(prev, v, strength)
self.extra = something_extra

So:
- Python only calls _one_ __init__ method
- in the example above, it finds one in EqualityConstraint and uses it
- because Python calls _only_ that, in order to _also_ do the normal 
  setup a BinaryConstraint needs, we pass the parameters used by 
  BinaryConstraint to the new __init__ up the chain by calling 
  super().__init__(BinaryConstraint-parameters-here)

- we do our own special something with the something_extra

That super().__init__() call is the _only_ thing which arranges that 
superclass inits get to run. This givens you full control to sompletely 
replace some superclass' init with a custom one. By calling 
super().__init__() we're saying we not replacing that stuff, we're 
running the old stuff and just doing something additional for our 
subclass.


Cheers,
Cameron Simpson 
--
https://mail.python.org/mailman/listinfo/python-list

Re: How does a method of a subclass become a method of the base class?

[Cameron Simpson]

On 26Mar2023 22:36, Jen Kris  wrote:
At the final line it calls "satisfy" in the Constraint class, and that 
line calls choose_method in the BinaryConstraint class.  Just as Peter 
Holzer said, it requires a call to "satisfy." 


My only remaining question is, did it select the choose_method in the 
BinaryConstraint class instead of the choose_method in the 
UrnaryConstraint class because of "super(BinaryConstraint, 
self).__init__(strength)" in step 2 above? 


Basicly, no.

You've omitting the "class" lines of the class definitions, and they 
define the class inheritance, _not "__init__". The "__init__" method 
just initialises the state of the new objects (which has already been 
created). The:


super(BinaryConstraint,_ self).__init__(strength)

line simply calls the appropriate superclass "__init__" with the 
"strength" parameter to do that aspect of the initialisation.


You haven't cited the line which calls the "choose_method" method, but 
I'm imagining it calls "choose_method" like this:


self.choose_method(...)

That searchs for the "choose_method" method based on the method 
resolution order of the object "self". So if "self" was an instance of 
"EqualityConstraint", and I'm guessing abut its class definition, 
assuming this:


class EqualityConstraint(BinaryConstraint):

Then a call to "self.choose_method" would look for a "choose_method" 
method first in the EqualityConstraint class and then via the 
BinaryConstraint class. I'm also assuming UrnaryConstraint is not in 
that class ancestry i.e. not an ancestor of BinaryConstraint, for 
example.


The first method found is used.

In practice, when you define a class like:

class EqualityConstraint(BinaryConstraint):

the complete class ancestry (the addition classes from which 
BinaryConstraint inherits) gets flatterned into a "method resultion 
order" list of classes to inspect in order, and that is stored as the 
".__mro__" field on the new class (EqualityConstraint). You can look at 
it directly as "EqualityConstraint.__mro__".


So looking up:

self.choose_method()

looks for a "choose_method" method on the classes in 
"type(self).__mro__".


Cheers,
Cameron Simpson 
--
https://mail.python.org/mailman/listinfo/python-list

Re: 转发: How to exit program with custom code and custom message?

[Cameron Simpson]

On 13Mar2023 10:18, scruel tao  wrote:

Chris:

but for anything more complicated, just print and then exit.
It's worth noting, by the way, that sys.exit("error message") will
print that to STDERR, not to STDOUT, which mean that the equivalent
is:


Yes, I know, but don’t you think if `sys.exit` can take more parameters 
and have a default output channel selection strategy will be better?


That kind of thing is an open ended can of worms. You're better off 
writing your own `aort()` function such as:


def abort(message, *a, output=None, exit_code=1):
if a:
message = message %a
if output is None:
output = sys.stderr
print(message, file=output)
exit(exit_code)

which can be extended with whatever (potentially _many_) custom options 
you like. `sys.exit()` itself is just to abort your programme, and 
doesn't want a lot of knobs. The more knobs, the more things which have 
to be implemented, and if the knobs are not independent that gets 
complicated very quickly. Not to mention agreeing on what knobs to 
provide.


BTW, `sys.exit()` actually raises a `SystemExit` exception which is 
handled by the `sys.excepthook` callback which handles any exception 
which escapes from the programme uncaught.


Finally, in larger programs it is uncommon to call `sys.exit()` except 
in the very outermost "main function", and even there I tend to just 
`return`, myself. Example:


def main(argv=None):
if argv is None:
argv = sys.argv
xit = 0
 handle the command line arguments, run code ...
... errors set xit=1 or maybe xit="message" ...
return xit

if __name__ == '__main__':
sys.exit(main(sys.argv))

Notice that the only call to `sys.exit()` is right at the bottom.  
Everything else is just regularfunction returns.


Cheers,
Cameron Simpson 
--
https://mail.python.org/mailman/listinfo/python-list

Re: Baffled by readline module

[Cameron Simpson]

On 10Mar2023 23:11, Weatherby,Gerard  wrote:

This is the implementation of cmd.Cmd (Python 3.9). As you can see in 
cmdloop(), the import is already wrapped, and the readline feature can be 
turned off explicitly by passing None to the completekey in the constructor.


This isn't strictly true, as I read it.

This says that if you supply a `completekey`, _then_ `cmdloop` will try 
to import `readline` and set up working completion for that key.


It _doesn't_ say that readline is or is not automatically active for its 
other features (command line editing, history etc).


Having a gander further down the `cmdloop` function we see:

while not stop:
if self.cmdqueue:
line = self.cmdqueue.pop(0)
else:
if self.use_rawinput:
try:
line = input(self.prompt)
except EOFError:
line = 'EOF'

and on a Python 3.10 here we see:

>>> help(input)
Help on built-in function input in module builtins:

input(prompt=None, /)
Read a string from standard input.  The trailing newline is stripped.

The prompt string, if given, is printed to standard output without a
trailing newline before reading input.

If the user hits EOF (*nix: Ctrl-D, Windows: Ctrl-Z+Return), raise 
EOFError.
On *nix systems, readline is used if available.

which says to me that readline is used if available, regardless of 
whether it is already imported.


Cheers,
Cameron Simpson 
--
https://mail.python.org/mailman/listinfo/python-list

Re: Baffled by readline module

[Cameron Simpson]

On 10Mar2023 09:12, Grant Edwards  wrote:

On 2023-03-10, Weatherby,Gerard  wrote:

On our Linux systems, I can up-arrow to go back to prior commands
and use the left and right arrows to navigate a line. The
functionality may be provided internally by readline. I’ve never had
to dig into it because it meets my needs out of the box.


Apparently the cmd.Cmd docs are wrong. It says:

 If the readline module is loaded, input will automatically
 inherit bash-like history-list editing (e.g. Control-P scrolls
 back to the last command, Control-N forward to the next one,
 Control-F moves the cursor to the right non-destructively,
 Control-B moves the cursor to the left non-destructively, etc.).

On my Python 3.10.10 Linux system, cmd.Com itself is importing the
readline module unconditionally when I call cmdloop(). There's no 'if'
about it.


I was wondering about that myself, whether this is an accident of 
phrasing. It doesn't say "is imported", so maybe the author was thinking 
"if readline's part of the install" here.


Anyway, I've got a try/import-readline/except-importerror/pass in my 
cmd.Cmd wrapper, because I read this the way you read it.


Cheers,
Cameron Simpson 
--
https://mail.python.org/mailman/listinfo/python-list

Re: Lambda returning tuple question, multi-expression

[Cameron Simpson]

On 09Mar2023 17:55, aapost  wrote:

On 3/9/23 16:37, Cameron Simpson wrote:
Just a note that some code formatters use a trailing comma on the last 
element to make the commas fold points. Both yapf (my preference) and 
black let you write a line like (and, indeed, flatten if short 
enough):


    ( a, b, c )

but if you write:

    ( a, b, c, )

they'll fold the lines like:

    ( a,
  b,
  c,
    )
Cameron Simpson 



Thanks for the info, good to know, I actually do like the idea of 
trailing commas for tuples (helps prevent things like the difference 
between ("abc") and ("abc",) and makes swapping things around nicer.


Just keep in mind that if you restructure code with copy/paste it can be 
a small issue (not really tied to the trailing comma but commas in 
general. More than once I've been bitten by doing roughly this:


f( a,
   b=x+y,
)

shuffled to:

b=x+y,
f( a,
   b=b,
)

Whoops! It shows up almost immediately, but the first time it took me a 
while to see that stray comma.


I've just been using a lot of json lately and it has been 
subconsciously training me different, lol.


Yes, it hates the trailing comma. So annoying.

Cheers,
Cameron Simpson 
--
https://mail.python.org/mailman/listinfo/python-list

Re: Baffled by readline module

[Cameron Simpson]

On 09Mar2023 13:08, Grant Edwards  wrote:

Having a module modify the behavior of a built-in makes me cringe.


Maybe. But input(), like print(), is one of those funky conveniences for 
human interaction. I'm somewhat ok with that. If for no other reason 
than to make keyboard arrow keys act as normal humans expect them to (as 
opposed to old nerds who predate such fancies). General improvement of 
the user experience.


Another thing affected by readline is the cmd module, which will use it 
during the input step for completion _if the module is present_.  And 
use it for the input itself if it is loaded. (These might not mean 
different things and instead be an accident of phrasing.) Again, a human 
interaction convenience and I'm good with it.


Good enough to have written this code recently:

try:
  import readline
except ImportError:
  pass

just to turn it on if available.

Cheers,
Cameron Simpson 
--
https://mail.python.org/mailman/listinfo/python-list

Re: Baffled by readline module

[Cameron Simpson]

On 09Mar2023 13:08, Grant Edwards  wrote:

On 2023-03-09, Chris Angelico  wrote:

Not sure about the history file, and I would assume that if you don't
configure one, history is simply lost when you restart. But with tab
completion, unless you need to be able to input a tab character, it
should be safe to ignore the feature and leave it at the defaults.


Indeed, that seems to be how it works (though I never found that
stated anywhere in the docs).

What's really weird about the docs is that when it is described it
doesn't even _mention_ that it provides command-line recall and
editing:

[...]

I think this might be the common case of a module which wraps another 
library: there's a tension between describing everything in pointless 
detail and the trite "we're just shimming this library, go read its 
docs".


The module documentation needs to:
- not insult the reader by saying nearly nothing ("this is just GNU 
  readline, hooked to Python!") I'm looking at you, many "man" pages on 
  a GNU based system which say "oh just go and read GNI info, it's all 
  over there"
- be detailed enough to enumerate the API calls and at least summarise 
  their semantics
- be general enough to not overprescribe so that small shifts in the 
  library as it evolves don't cause falsehoods in the module docs (and a 
  nightmarish maintenance burden)


Similar example: the "os" modules, which largely shims the OS POSIX 
calls. It lists them and has a paragraph on their purpose and the 
meaning of the flags (for example). But the platform specifics and fine 
grained stuff is off in "man 2 foo" for the Python "os.foo" function.


[...]

It finally dawned on me after seeing an example I found elsewhere that
you don't call some module method to fetch the next user-entered line.

You call the input() built-in.


Ah. That's not overtly stated? [...reads...] Ah, there it is in the last 
sentence of the opening paragraph. Not quite as in-your-face as I'd have 
liked it. That paragraph could do with being a bullet list of use cases.


Cheers,
Cameron Simpson 
--
https://mail.python.org/mailman/listinfo/python-list

Re: Lambda returning tuple question, multi-expression

[Cameron Simpson]

On 09Mar2023 09:06, Alan Gauld  wrote:

Also layout is all important here. It could get very messy to read if
indentation isn't clear. You only have to look at some Javascript code
with function definitions as arguments to functions to see how clunky
that can be.


Just a note that some code formatters use a trailing comma on the last 
element to make the commas fold points. Both yapf (my preference) and 
black let you write a line like (and, indeed, flatten if short enough):


( a, b, c )

but if you write:

( a, b, c, )

they'll fold the lines like:

( a,
  b,
  c,
)

in varying flavours of indentation depending on tuning. The point being 
that if, like me, you often have a code formatter active-on-save it can 
be hinted to nicely present complex tuples (or parameter lists and 
imports).


It isn't magic, but can be quite effective.

Cheers,
Cameron Simpson 
--
https://mail.python.org/mailman/listinfo/python-list

Re: Lambda returning tuple question, multi-expression

[Cameron Simpson]

On 09Mar2023 09:06, Alan Gauld  wrote:

On 08/03/2023 21:56, aapost wrote:

When making a UI there are a lot of binding/trace operations that need
to occur that lead to a lot of annoying 1 use function definitions. I
don't really see lambda use like below.


Lambdas are very common in GUI callbacks but I admit I've never seen
tuples used to create multiple expressions. That's a neat trick I
hadn't thought of and will probably use.


I often uses as a debugging hack. Given:

foo=lambda: expr

I'll often write:

foo=lambda: (X("foo happening here!"), ...maybe more..., expr)[-1]

to embed some debug tracing in a lambda defined expression.

Cheers,
Cameron Simpson 
--
https://mail.python.org/mailman/listinfo/python-list

Re: Lambda returning tuple question, multi-expression

[Cameron Simpson]

On 08Mar2023 16:56, aapost  wrote:
When making a UI there are a lot of binding/trace operations that need 
to occur that lead to a lot of annoying 1 use function definitions. I 
don't really see lambda use like below.


Giving 2 working lambda examples using a returned tuple to accomplish 
multiple expressions - what sort of gotchas, if any, might make the 
following bad practice if I am missing something?


There're no real gotchas: tuples are evaluated left to right, so you 
should have things happen in the order you've expressed them (if the 
order matters). What you lose with a lambda is control constructs like 
loops and if-statements (well, there's the `x if foo else y` but that 
gets cumbersome quickly). Once things get complicated you may want to 
define a more complication function using `def`:


def callback1():
... do complicated stuff ...

b.config(command=callback1)

The only other issue, which applies across the board with GUIs and is 
nothing specific to lambdas is that the GUI only renders and operates 
while the main loop is running. When your callbacks do trivial stuff 
you're fine. If they block (eg waiting for user input or network calls 
etc) the GUI is also blocked. You need threads or other concurrent 
approaches if the GUI is to stay responsive.


The flip side of that is that a concurrent context like a Thread should 
not interact with the GUI directly. In things like Qt I've actually had 
that mistake crash the app because the Qt framework is (was?) not thread 
safe. You need to arrange that GUI actions occur in the main programme 
thread. I think the same applies with tk, and is anyway probably good 
practice for any GUI. It's not as hard as it sounds - typically when 
something happens asynchronously you arrange to issue an "event", and 
the GUI mainloop will process that as it happens - the event callback 
will be fired (called) by the main loop itself and thus the callback 
gets to do its thing in the main loop.


Cheers,
Cameron Simpson 
--
https://mail.python.org/mailman/listinfo/python-list

Re: Bug 3.11.x behavioral, open file buffers not flushed til file closed.

[Cameron Simpson]

On 05Mar2023 10:38, aapost  wrote:

Additionally (not sure if this still applies):
flush() does not necessarily write the file’s data to disk. Use flush() 
followed by os.fsync() to ensure this behavior.


Yes. You almost _never_ need or want this behaviour. A database tends to 
fsync at the end of a transaction and at other critical points.


However, once you've `flush()`ed the file the data are then in the hands 
of the OS, to get to disc in a timely but efficient fashion. Calling 
fsync(), like calling flush(), affects writing _efficiency_ by depriving 
the OS (or for flush(), the Python I/O buffering system) the opportunity 
to bundle further data efficiency. It will degrade the overall 
performance.


Also, fsync() need not expedite the data getting to disc. It is equally 
valid that it just blocks your programme _until_ the data have gone to 
disc. I practice it probably does expedite things slightly, but the real 
world effect is that your pogramme will gratuitously block anyway, when 
it could just get on with its work, secure in the knowledge that the OS 
has its back.


flush() is for causality - ensuring the data are on their way so that 
some external party _will_ see them rather than waiting forever for data 
with are lurking in the buffer.  If that external party, for you, is an 
end user tailing a log file, then you might want to flush(0 at the end 
of every line.  Note that there is a presupplied line-buffering mode you 
can choose which will cause a file to flush like that for you 
automatically.


So when you flush is a policy decision which you can make either during 
the programme flow or to a less flexible degree when you open the file.


As an example of choosing-to-flush, here's a little bit of code in a 
module I use for writing packet data to a stream (eg a TCP connection):

https://github.com/cameron-simpson/css/blob/00ab1a8a64453dc8a39578b901cfa8d1c75c3de2/lib/python/cs/packetstream.py#L624

Starting at line 640: `if Q.empty():` it optionally pauses briefly to 
see if more packets are coming on the source queue. If another arrives, 
the flush() is _skipped_, and the decision to flush made again after the 
next packet is transcribed. In this way a busy source of packets can 
write maximally efficient data (full buffers) as long as there's new 
data coming from the queue, but if the queue is empty and stays empty 
for more that `grace` seconds we flush anyway so that the receiver 
_will_ still see the latest packet.


Cheers,
Cameron Simpson 
--
https://mail.python.org/mailman/listinfo/python-list

Re: Bug 3.11.x behavioral, open file buffers not flushed til file closed.

[Cameron Simpson]

On 05Mar2023 09:35, aapost  wrote:
I have run in to this a few times and finally reproduced it. Whether 
it is as expected I am not sure since it is slightly on the user, but 
I can think of scenarios where this would be undesirable behavior.. 
This occurs on 3.11.1 and 3.11.2 using debian 12 testing, in case the 
reasoning lingers somewhere else.


If a file is still open, even if all the operations on the file have 
ceased for a time, the tail of the written operation data does not get 
flushed to the file until close is issued and the file closes cleanly.


Yes, because files are _buffered_ by default. See the `buffering` 
parameter to the open() function in the docs.



2 methods to recreate - 1st run from interpreter directly:

f = open("abc", "w")
for i in range(5):
 f.write(str(i) + "\n")

you can cat the file and see it stops at 49626 until you issue an f.close()


Or until you issue an `f.flush()`. hich is what flush is for.
cat out the file and same thing, stops at 49626. a ctrl-c exit closes 
the files cleanly, but if the file exits uncleanly, i.e. a kill command 
or something else catastrophic. the remaining buffer is lost.


Yes, because of bfufering. This is normal and IMO correct. You can turn 
it off, or catch-and-flush these circumstances (SIGKILL excepted, 
because SIGKILL's entire purpose it to be uncatchable).


Of course one SHOULD manage the closing of their files and this is 
partially on the user, but if by design something is hanging on to a 
file while it is waiting for something, then a crash occurs, they lose 
a portion of what was assumed already complete...


f.flush()

Cheers,
Cameron Simpson 
--
https://mail.python.org/mailman/listinfo/python-list

Re: Regular Expression bug?

[Cameron Simpson]

On 02Mar2023 20:06, jose isaias cabrera  wrote:
This re is a bit different than the one I am used. So, I am trying to 
match

everything after 'pn=':

import re
s = "pm=jose pn=2017"
m0 = r"pn=(.+)"
r0 = re.compile(m0)
s0 = r0.match(s)


`match()` matches at the start of the string. You want r0.search(s).
- Cameron Simpson 
--
https://mail.python.org/mailman/listinfo/python-list

Re: Python 3.10 Fizzbuzz

[Cameron Simpson]

On 26Feb2023 13:07, Hen Hanna  wrote:

is there any reason to prefer"over'   ?


Not inherently.

It is only important if you want to embed one of those characters in a 
string, eg:


x = "That's silly."

versus:

x = 'That\'s silly.'

and the converse for the other quote character.

Cheers,
Cameron Simpson 
--
https://mail.python.org/mailman/listinfo/python-list

Re: Python 3.10 Fizzbuzz

[Cameron Simpson]

On 28Feb2023 12:54, Greg Ewing  wrote:

I guess this means I can't use Black. :-(


Black's treatment of quotes and docstrings is one of the largest reasons 
why I won't let it touch my personal code. yapf is far better behaved, 
and can be tuned as well!


Cheers,
Cameron Simpson 
--
https://mail.python.org/mailman/listinfo/python-list

Re: How to escape strings for re.finditer?

[Cameron Simpson]

On 28Feb2023 18:57, Jen Kris  wrote:
One question:  several people have made suggestions other than regex 
(not your terser example with regex you shown below).  Is there a 
reason why regex is not preferred to, for example, a list comp?  


These are different things; I'm not sure a comparison is meaningful.


Performance?  Reliability? 


Regexps are:
- cryptic and error prone (you can make them more readable, but the 
  notation is deliberately both terse and powerful, which means that 
  small changes can have large effects in behaviour); the "error prone" 
  part does not mean that a regexp is unreliable, but that writing one 
  which is _correct_ for your task can be difficult, and also difficult 
  to debug

- have a compile step, which slows things down
- can be slower to execute as well, as a regexp does a bunch of 
  housekeeping for you


The more complex the tool the more... indirection between your solution 
using that tool and the smallest thing which needs to be done, and often 
the slower the solution. This isn't absolute;  there are times for the 
complex tool.


Common opinion here is often that if you're doing simple fixed-string 
things such as your task, which was finding instances of a fixed string, 
just use the existing str methods. You'll end up writing what you need 
directly and overtly.


I've a personal maxim that one should use the "smallest" tool which 
succinctly solves the problem. I usually use it to choose a programming 
language (eg sed vs awk vs shell vs python in loose order of problem 
difficulty), but it applies also to choosing tools within a language.


Cheers,
Cameron Simpson 
--
https://mail.python.org/mailman/listinfo/python-list

Re: How to escape strings for re.finditer?

[Cameron Simpson]

On 28Feb2023 00:57, Jen Kris  wrote:
Yes, that's it.  I don't know how long it would have taken to find that 
detail with research through the voluminous re documentation.  Thanks 
very much. 


You find things like this by printing out the strings you're actually 
working with. Not the original strings, but the strings when you're 
invoking `finditer` i.e. in your case, escaped strings.


Then you might have seen that what you were searching no longer 
contained what you were searching for.


Don't underestimate the value of the debugging print call. It lets you 
see what your programme is actually working with, instead of what you 
thought it was working with.


Cheers,
Cameron Simpson 
--
https://mail.python.org/mailman/listinfo/python-list

Re: How to escape strings for re.finditer?

[Cameron Simpson]

On 28Feb2023 01:13, Jen Kris  wrote:
I went to the re module because the specified string may appear more 
than once in the string (in the code I'm writing).


Sure, but writing a `finditer` for plain `str` is pretty easy 
(untested):


pos = 0
while True:
found = s.find(substring, pos)
if found < 0:
break
start = found
end = found + len(substring)
... do whatever with start and end ...
pos = end

Many people go straight to the `re` module whenever they're looking for 
strings. It is often cryptic error prone overkill. Just something to 
keep in mind.


Cheers,
Cameron Simpson 
--
https://mail.python.org/mailman/listinfo/python-list

Re: How to escape strings for re.finditer?

[Cameron Simpson]

On 28Feb2023 00:11, Jen Kris  wrote:

When matching a string against a longer string, where both strings have spaces 
in them, we need to escape the spaces. 

This works (no spaces):

import re
example = 'abcdefabcdefabcdefg'
find_string = "abc"
for match in re.finditer(find_string, example):
    print(match.start(), match.end())

That gives me the start and end character positions, which is what I want. 

However, this does not work:

import re
example = re.escape('X - cty_degrees + 1 + qq')
find_string = re.escape('cty_degrees + 1')
for match in re.finditer(find_string, example):
    print(match.start(), match.end())

I’ve tried several other attempts based on my reseearch, but still no 
match. 


You need to print those strings out. You're escaping the _example_ 
string, which would make it:


X - cty_degrees \+ 1 \+ qq

because `+` is a special character in regexps and so `re.escape` escapes 
it. But you don't want to mangle the string you're searching! After all, 
the text above does not contain the string `cty_degrees + 1`.


My secondary question is: if you're escaping the thing you're searching 
_for_, then you're effectively searching for a _fixed_ string, not a 
pattern/regexp. So why on earth are you using regexps to do your 
searching?


The `str` type has a `find(substring)` function. Just use that! It'll be 
faster and the code simpler!


Cheers,
Cameron Simpson 
--
https://mail.python.org/mailman/listinfo/python-list

Re: Programming by contract.

[Cameron Simpson]

On 26Feb2023 02:44, Weatherby,Gerard  wrote:

The discussion of asserts got me thinking about Programming by Contract. Back 
in the 90s, I had occasion to learn Eiffel programming language. ( 
https://en.wikipedia.org/wiki/Eiffel_(programming_language) The concepts are 
intriguing, although Eiffel itself had barriers to widespread adoption.

Reviewing the topic, I found Wikipedia mentioned some Python implementations. I 
kicked the tires of “deal” and found it straightforward. I’m wondering if 
anyone on the list has experience using any of the Programming by Contract 
modules?


I've been using the icontract package happily.

from icontract import require, ensure

@typechecked
@require(
lambda name: is_valid_basename(name),  # pylint: 
disable=unnecessary-lambda
"name should be a clean file basename"
)
def TagSetClass(self, name: str) -> TaggedPath:
  ''' factory to create a `TaggedPath` from a `name`.
  '''
  fspath = joinpath(dirname(self.fspath), name)
  return TaggedPath(fspath, fstags=self.fstags)

@ensure(lambda result: result == normpath(result))
def keypath(self, fspath):
  ''' Compute the absolute path used to index a `TaggedPath` instance.

  This returns `realpath(fspath)` if `self.config.physical`,
  otherwise `abspath(fspath)`.
  '''
  return realpath(fspath) if self.config.physical else abspath(fspath)

You can stack the decorators just like deal.

Cheers,
Cameron Simpson 
--
https://mail.python.org/mailman/listinfo/python-list

Re: Error-Msg Jeannie's charming, teasing ways

[Cameron Simpson]

On 23Feb2023 14:58, Hen Hanna  wrote:
Python's Error-Msg  genie  (Jeannie)  is cute and fickle...   She 
sometimes teases me by not telling me what the VALUE of the   "int"  is

( "That's for me to know, and for you to find out!" )
as in:
  TypeError: can only concatenate str (not 
  "int") to str


This is a runtime error< not a syntax error. Valid code using invalid 
values. It can only be seen at runtime.



Other times, she (Jeannie)  gives me a helpful comment:
ABCD (above and beyond... her job desc.)
 if (x=0):
   ^^^
SyntaxError: invalid syntax. Maybe you meant '==' or ':=' instead of '='?


Syntax error can be seen at compile time. Since Python's interpreted, 
the compilation step i usually implicit in trying to run things. But 
technically this happens before any of _your_ code runs.


The helpfulness of this warning is a _very_ recent upgrade, with Python 
3.11 I think, maybe 3.10.  The syntax errors used to be a lot less 
helpful.


is there a Tool that can scan my code and tell me   such   (wink,wink)  
type suggestions


There are several type checking programs for Python, with mypy probably 
being the best known. I seem to recall seeing some mention of tools 
which will aid inferring types from partially types programmes, usually 
as an aid to completing the type annotations.


Cheers,
Cameron Simpson 
--
https://mail.python.org/mailman/listinfo/python-list

Re: Line continuation and comments

[Cameron Simpson]

On 24/02/2023 12.45, Weatherby,Gerard wrote:
NB my PyCharm-settings grumble whenever I create an identifier which 
is

only used once (and perhaps, soon after it was established). I
understand the (space) optimisation, but prefer to trade that for
'readability'.


It isn't "space". Got an example for that warning? I don't use PyCharm, 
but the main linter warning I get is an _unused_ variable, eg:


def f():
x = 3

I set x and never use it. Likely brain fade on my part, and worth a 
warning.


On 24Feb2023 15:01, dn  wrote:

I haven’t seen that one. What I get is warnings about:

def is_adult( self )->bool:
    LEGAL_AGE_US = 21
    return LEGAL_AGE

It doesn’t like LEGAL_AGE_US being all caps if declared in a function.


Yes, I suffered this one too.

The rationale comes from PEP-008 (Constants):

Constants are usually defined on a module level and written in all 
capital letters with underscores separating words.


Yeah. The above looks like a method. I'd probably have something like:

class C:

LEGAL_AGE_US = 21

def is_adult(self) -> bool:
return self.age >= self.LEGAL_AGE_US

Variables used (self). Constant a class attribute.

Cheers,
Cameron Simpson 
--
https://mail.python.org/mailman/listinfo/python-list

Re: Python + Vim editor

[Cameron Simpson]

On 21Feb2023 18:00, Hen Hanna  wrote:

what editor do you (all) use to write Python code?  (i use Vim)


vim
--
https://mail.python.org/mailman/listinfo/python-list

Re: Line continuation and comments

[Cameron Simpson]

On 22Feb2023 11:27, Thomas Passin  wrote:

On 2/22/2023 10:02 AM, Weatherby,Gerard wrote:

That’s a neat tip. End of line comments work, too

x = (3 > 4  #never
 and 7 == 7  # hopefully
 or datetime.datetime.now().day > 15 # sometimes
 )
print(x)


I find myself doing this more and more often.  It can also help to 
make the code more readable and the intention more clear. Here's one 
example:


   return (getTerminalFromProcess()
   or getTerminalFromDirectory('/usr/bin')
   or getTerminalFromDirectory('/bin')
   or getCommonTerminal(('konsole', 'xterm'))
   )


Aye, me too.

I autoformat my code using `yapf` (I _hate_ `black`) and append my 
personal code style below. In particular, note the 
`split_before_logical_operator` which does the above automatically when 
it reflows a logical expression (you need the brackets, but you need 
them anyway for multiline stuff if you're eschewing the backslash).


[style]
based_on_style = pep8
align_closing_bracket_with_visual_indent = True
arithmetic_precedence_indication = False
blank_line_before_module_docstring = True
blank_line_before_nested_class_or_def = True
blank_lines_around_top_level_definition = 1
dedent_closing_brackets = True
indent_dictionary_value = False
indent_width = 2
space_between_ending_comma_and_closing_bracket = False
spaces_before_comment = 2
split_before_dot = True
split_before_expression_after_opening_paren = True
split_before_first_argument = True
split_before_logical_operator = True
split_complex_comprehension = True
use_tabs = False

So basicly PEP8 with some tweaks.

Cheers,
Cameron Simpson 
--
https://mail.python.org/mailman/listinfo/python-list

Re: Tuple Comprehension ???

[Cameron Simpson]

On 20Feb2023 19:36, Hen Hanna  wrote:

For a while,  i've been curious about a  [Tuple   Comprehension]

So  finally   i tried it, and the result was a bit surprising...

X= [ x for x in range(10) ]


This is a list comprehension, resulting in a list as its result.


X= ( x for x in range(10) )


This is not a tuple comprehension - Python does not have one.

Instead, it is a generator expression:

x for x in range(10)

inside some brackets, which are just group as you might find in an 
expression like:


(3 + 4) * 7

If you want a tuple, you need to write:

X = tuple( x for x in range(10) )

which makes a tuple from an iterable (such as a list, but anything 
iterable will do). Here the iterable is the generator expression:


x for x in range(10)

Cheers,
Cameron Simpson 
--
https://mail.python.org/mailman/listinfo/python-list

Re: Change the identation base value?

[Cameron Simpson]

On 16Feb2023 04:20, Chris Angelico  wrote:

On Thu, 16 Feb 2023 at 04:18, scruel tao  wrote:

Currently, we have following PEP:
PEP 8: E114 indentation is not a multiple of 4 (comment)

However, I wonder how many people are using 2 spaces as their code indentation 
in projects? Should it be nesserary for us to change this PEP from “multiple of 
4” to “multiple of 2”?


Read PEP 8 itself before making a decision.

1) It is NOT rules for all Python code, only the standard library.
2) 
https://peps.python.org/pep-0008/#a-foolish-consistency-is-the-hobgoblin-of-little-minds

It is not necessary to change the PEP; it is necessary to fix tools
that claim more authority than they actually have.


Speaking as a 2-space indenter myself (in my personal code):

The tool in question is likely `pycodestyle`, whose _default_ linting 
measures PEP8 criteria among other things.


I run a few different linters (via a script named "lint") and turn off 
quite a few of the `pycodestyle` checks, `E114` included.


`pycodestyle` doesn't clean any special "authority", and exlicitly 
offers ways to turn off whichever checks you find unsuitable to your 
code.


Cheers,
Cameron Simpson 
--
https://mail.python.org/mailman/listinfo/python-list

Re: Am I banned from Discuss forum?

[Cameron Simpson]

On 11Feb2023 13:21, Roel Schroeven  wrote:

Michael Torrie schreef op 11/02/2023 om 4:59:
I didn't know there was a Discourse forum.  Is it supposed to be 
sync  > with the mailing list and USENET? Or is it intended to 
replace this
mailing list? I rarely see Python devs on this list, so maybe > 
they've chosen to hang out exclusively in Discourse, which would be > 
unfortunate.
I don't think it's meant to replace this mailing list; it's still 
mentioned on the Python website. At least initially the forum was 
intended mostly for core dev discussions, as far as I understand it.


The initial announcement (which only talks about python-comitters 
being migrated): https://www.mail-archive.com/python-committers@python.org/msg06151.html

The site with the discussions: https://discuss.python.org/


There are several sections on the Discourse forum; the "Help" section 
broadly has the same purpose as this mailing list. I use both, and use 
Discourse in email mode - the forum's there, but >90% of my interaction 
is via email - I file it and python-list to the same "python" folder 
here and lossely treat them the same.


Cheers,
Cameron Simpson 
--
https://mail.python.org/mailman/listinfo/python-list

Re: How to make argparse accept "-4^2+5.3*abs(-2-1)/2" string argument?

[Cameron Simpson]

On 29Jan2023 08:10, avi.e.gr...@gmail.com  wrote:

You are technically correct but perhaps off the mark.

Yes, a python program only sees what is handed to it by some shell if invoked a 
certain way.

The issue here is what you tell people using your program about what they need 
to type to get it to work. That means if their shell is going to make changes 
in what they typed, they need to know how to avoid unintended changes. As one 
example not mentioned, whitespace disappears if not somehow protected as in 
quotes.


Hmm, there is that. But the OP needs clarity on what happens in a shell 
and what happens in a programme once the shell has invoked it for a 
user.


Cheers,
Cameron Simpson 
--
https://mail.python.org/mailman/listinfo/python-list

Re: How to make argparse accept "-4^2+5.3*abs(-2-1)/2" string argument?

[Cameron Simpson]

On 29Jan2023 07:12, 2qdxy4rzwzuui...@potatochowder.com 
<2qdxy4rzwzuui...@potatochowder.com> wrote:

On 2023-01-29 at 16:51:20 +1100,
Cameron Simpson  wrote:
They're unrelated. As others have mentioned, "--" is _extremely_ 
common;

almost _all_ UNIX command like programmes which handle -* style options
honour the "--" convention. _argparse_ itself honours that convention, as
does getopt etc.


And why do UNIX programs behave this way?

Because POSIX says they should:

   https://pubs.opengroup.org/onlinepubs/9699919799/basedefs/V1_chap12.html


Aye, but POSIX says they should because POSIX formalises a lot of 
existing practices so that people have a reference to let them _also_ 
use that practice in a consistent way.


I used to lurk on the POSIX discussion list, and the length of the 
discussions around corner cases was quite wearing, even just to read.


Cheers,
Cameron Simpson 
--
https://mail.python.org/mailman/listinfo/python-list

Re: How to make argparse accept "-4^2+5.3*abs(-2-1)/2" string argument?

[Cameron Simpson]

On 28Jan2023 18:55, Jach Feng  wrote:

Mark Bourne 在 2023年1月28日 星期六晚上10:00:01 [UTC+8] 的信中寫道：

I notice you explain the need to enclose the equation in quotes if it
contains spaces. That's not even a feature of your application, but of
the shell used to call it. So why so much objection to explaining the
need for "--"?

Depending on the shell, there are other cases where quoting might be
needed, e.g. if the equation includes a "*" or "?" and happens to look
like a pattern matching files in the current directory (most shells I've
used pass the pattern unchanged if it doesn't match any files). In
bash, if a "$" is used I'd need to enclose that in 'single quotes'
(can't even use "double quotes" for that one). You can't really expect
to document all that sort of thing, because it depends on which shell
the user happens to run your application from - you just have to trust
the user to know or learn how to use their shell.


Thank you for detail explanation of the role the shell is involved in this 
problem. I'm very appreciated!


The shell has basicly _nothing_ to do with your problems. By the time 
you've got sys.argv in your Python programme you will have no idea 
whether quotes were used with an argument. (UNIX/POSIX, not Windows, 
where things are ... more complex.) This means you don't know if the use 
typed:


-4.5

or

"-4.5"

You'll just get a string '4.5' in your Python programme both ways.

All the quotes in the shell do is delimit what things should be kept 
together as a single argument versus several, or where variables should 
be interpolated when computing arguments etc. It's just _shell_ 
punctuation and the invoked programme doesn't see it.



It seems that a CLI app may become very complex when dealing with different 
kind of shell, and may not be possible to solve its problem.


It doesn't matter what shell is used. The just controls what punctuation 
the end user may need to use to invoke your programme. You programme 
doesn't need to care (and can't because it doesn't get the quotes etc, 
only their result).



So why so much objection to explaining the need for "--"?

Because of using " to enclose a space separated string is a common convention, and adding 
a "--" is not:-)


They're unrelated. As others have mentioned, "--" is _extremely_ common; 
almost _all_ UNIX command like programmes which handle -* style options 
honour the "--" convention. _argparse_ itself honours that convention, 
as does getopt etc.


The "--" convention has nothing to do with the shell.

Cheers,
Cameron Simpson 
--
https://mail.python.org/mailman/listinfo/python-list

Re: Custom help format for a choice argparse argument

[Cameron Simpson]

On 27Jan2023 15:31, Ivan "Rambius" Ivanov  wrote:

I am developing a script that accepts a time zone as an option. The
time zone can be any from pytz.all_timezones. I have

def main():
   parser = argparse.ArgumentParser()
   parser.add_argument("-z", "--zone", choices=pytz.all_timezones)

[...]


It works, but when I run it with the -h option it dumps all entries in
pytz.all_timezones.


What happens if you just presupply a `help=` parameter in 
`add_argument`?


Cheers,
Cameron Simpson 
--
https://mail.python.org/mailman/listinfo/python-list

Re: How to make argparse accept "-4^2+5.3*abs(-2-1)/2" string argument?

[Cameron Simpson]

On 25Jan2023 16:15, Chris Angelico  wrote:

On Wed, 25 Jan 2023 at 14:42, Jach Feng  wrote:

I was happy working with argparse during implement my script. To save the 
typing, I used a default equation for testing.


Sure, but what benefit was it bringing you? Just the usage (help) 
message?  Did you have many options to handle?



sample = "-4^2+5.3*abs(-2-1)/2, abs(Abc)*(B+C)/D, (-3) * sqrt(1-(x1/7)*(y1/7)) * 
sqrt(abs((x0-4.5)/(y0-4)))"
parser = argparse.ArgumentParser(description='Convert infix notation to 
postfix')
parser.add_argument('infix', nargs='?', default=sample, help="")


If this was the whole thing, I don't see what argparse was doing for you 
which was better than just handling the arguments yourself - there's 
only one after all.



You're still not really using argparse as an argument parser. Why not
just do your own -h checking? Stop trying to use argparse for what
it's not designed for, and then wondering why it isn't doing what you
expect it to magically know.


I'm with Chris here.

As someone who pretty well _never_ uses argparse, and occasionally uses 
getopt, I'd do something like this:


usage = '''Usage: infix
Parse the argument infix as an expression, print the result.'''

badopts = False
cmd = argv.pop(0)
if not argv:
print(f'{cmd}: missing infix argument', file=sys.stderr)
badopts = True
else:
infix = argv.pop(0)
if infix in ('-h', '--help'):
print(usage)
exit(0)
if argv:
print(f'{cmd}: extra arguments after infix: {argv!r}', 
file=sys.stderr)
badopts = True
if badopts:
print(usage, file=sys.stderr)
exit(2)
... work with infix as desired ...

This:
- avoids trying to shoehorn argparse into behaving in a way it was not 
  designed for
- avoids users needing to know the standard UNIX/POSIX "--" idiom for 
  marking off the end of options

- supports a -h or --help leading option

Cheers,
Cameron Simpson 
--
https://mail.python.org/mailman/listinfo/python-list

Re: How to make argparse accept "-4^2+5.3*abs(-2-1)/2" string argument?

[Cameron Simpson]

On 23Jan2023 17:58, Jach Feng  wrote:

parser.parse_args(['--', 'infix2postfix.py', '-4.3+5'])

usage: [-h] infix
: error: unrecognized arguments: -4.3+5


This error doesn't look like "-4.3+5 looks like an option" but instead 
"we don't expect any arguments after "infix".


Not personally a fan of argparse myself, but then I have my own 
elaborate command line framework which generally uses getopt for the 
option stuff.


Cheers,
Cameron Simpson 
--
https://mail.python.org/mailman/listinfo/python-list

Re: How to make argparse accept "-4^2+5.3*abs(-2-1)/2" string argument?

[Cameron Simpson]

On 23Jan2023 13:46, 2qdxy4rzwzuui...@potatochowder.com 
<2qdxy4rzwzuui...@potatochowder.com> wrote:

On 2023-01-22 at 18:19:13 -0800,
Jach Feng  wrote:
1) Modify the sys.argv by inserting an item '--' before parsing it, 
ie.

sys.argv.insert(1, '--')
args = parser.parse_args()


Please don't do that.  :-)


Well... We routine mmodify argv when parsing a command line. It's just a 
list. It does assume I'm the only user of it.



In my mind, sys.argv belongs to Python, not the application.  Instead,
pass a newly created argument list to parse_args:

args = parser.parse_args(['--'] + sys.argv)


I do sometimes copy it:

def main(argv):
argv = list(argv)

for exactly the reasons you're alluding to. Then I'm free to modify the 
copy.


But for Jach Feng: the "--" is really expected as something the user 
does when they invoke your programme, _explicitly_ saying that what 
follows from here is not an argument. So the user is expected to type:


your_script -x -y -- "-4^2+5.3*abs(-2-1)/2"

where there are -x and -y options, then end of options, then an 
argument, which would look like an option if there wasn't the "--" 
argument.


Cheers,
Cameron Simpson 
--
https://mail.python.org/mailman/listinfo/python-list

Re: How to make argparse accept "-4^2+5.3*abs(-2-1)/2" string argument?

[Cameron Simpson]

On 21Jan2023 19:11, Jach Feng  wrote:

Fail on command line,

e:\Works\Python>py infix2postfix.py "-4^2+5.3*abs(-2-1)/2"
usage: infix2postfix.py [-h] [infix]
infix2postfix.py: error: unrecognized arguments: -4^2+5.3*abs(-2-1)/2


The usual convention for having "nonoption" arguments beginning with a 
dash/minus is to explicitly _end_ the option arguments, eg:


py infix2postfix.py -- "-4^2+5.3*abs(-2-1)/2"

That "--" indicates the end of the options, and that what follows should 
not be treated as an option. _However_, it requires support in the 
script parsing the options. I'm pretty sure argparse gives that support 
for free, as does getopt and probably any others implementing "normal 
UNIXish options".


SO try adding a "--" argument and see how it behaves.


Also fail in REPL,

e:\Works\Python>py
Python 3.8.8 (tags/v3.8.8:024d805, Feb 19 2021, 13:08:11) [MSC v.1928 32 bit 
(Intel)] on win32
Type "help", "copyright", "credits" or "license" for more information.

import argparse
parser = argparse.ArgumentParser(description='Convert infix notation to 
postfix')
parser.parse_args("-4^2+5.3*abs(-2-1)/2")

usage: [-h]
: error: unrecognized arguments: - 4 ^ 2 + 5 . 3 * a b s ( - 2 - 1 ) / 2


This is a different error. `parse_args()` expects a list of arguments, 
not a single argument. So it has iterated over what you gave it, which 
produces a series of single character strings which it has taken to be 
individual arguments. Try this:


parser.parse_args(["-4^2+5.3*abs(-2-1)/2"])

and of course:

parser.parse_args(["--", "-4^2+5.3*abs(-2-1)/2"])

You can see this behaviour of strings as:

print(list("abc"))

or:

for s in "abc":
print("s =", repr(s))

Cheers,
Cameron Simpson 
--
https://mail.python.org/mailman/listinfo/python-list

Re: PEP-8, Line Length, And All That

[Cameron Simpson]

On 21Jan2023 01:32, Thomas Passin  wrote:
In another thread ("Improvement to imports, what is a better way ?") 
there was a lot of talk about line length, PEP-8, etc.  I realized 
that one subject did not really come up, yet it can greatly affect the 
things we were talking about.


I'm referring to the design of the functions, methods, and classes. 
When they are well designed, or more likely, refactored over and over 
again, they can lead to code that reads almost like pseudo-code.  [...]


Yeah. Personally, I summarise this as the maxim: "ergonomics matter".  
Code should be easy to use and easy to read.


I've got a heap of code I've refactored after the initial implementation 
to make it easy to use, if only to make it easier for _me_ to use to do 
more things with it.


I've also got decorators and context managers and special classes all 
aimed at making all the things easier to use and easier to _write_ 
(because some decorators (a) provide flexibility for the caller - easier 
to use and (b) implement standard boilerplate logic for common types of 
parameters, making the function/method implementation easier to write.)


And increasingly I've got docstrings with _example_ code which doubles 
as a doctest. Making it easier to understand how to easily use this 
thing.


Cheers,
Cameron Simpson 
--
https://mail.python.org/mailman/listinfo/python-list

Re: Improvement to imports, what is a better way ?

[Cameron Simpson]

On 19Jan2023 07:34, Dan Kolis  wrote:

On Thursday, January 19, 2023 at 12:09:02 AM UTC-5, cameron wrote:

I know this is vague. Once you find its stalling in a particular
function (if it is) you may be able to run that function directly. Also,
a print() at the top abd bottom/return of the stalling function. And so
on.


Dan says:
After lots of iterations, I changed one line in a call used to refresh 
windows from .update() to .update_idle_tasks()


Yeah. See the bottom of this page:
https://tkdocs.com/tutorial/eventloop.html


Some other dudes were so nice they tested it before that, it worked perfectly 
on their computers anyway.


This kind of thing can be timing and user interaction sensitive.


Now it seems to work 'all the way' too after this change on my favroritish 
computer.
Since a CTRL C often started it going again I had a hint there, reading 
other peoples various troubles all over the web made me think it was 
'someplace in this part'.


The `update_idle_tasks()` call runs pending display updates (i.e.  
redraws) but does not process any events (mostly user actions). So it 
generally returns promptly.


The `update()` call processes events as well. If those events make 
callbacks, the callbacks can block (depending what they do). Or they 
themselves might call update() if you've written your code that way.


There's a note on the page mentioned above.

Calling update() effectively runs the main event loop _here_, at the 
call. This is also how things like interactive dialogue boxes work: draw 
the dialogue, then internally call update() to process user interaction 
until the dialogue is done, then return.


This description is lousy, and we'd need to see the code to pin point 
exactly how your interface is deadlocking (or, possibly, just pausing 
for a long time while something happens).


But the key points are:
- there's no threading in the stuff above, so if you need GUI updates 
  _during_ some process you need to _call_ the GUI main loop for it to 
  do some work - you've blocked until that call comes back, like any 
  other function call
- the update_idle_tasks just does redraws; because it doesn't process 
  events those events' callbacks do not run, therefore those callbacks 
  cannot block your current code
- the update function does the full mainloop, including processing 
  events, and processing events can make callbacks, whcih can themselves 
  call update etc and ...


Cheers,
Cameron Simpson 
--
https://mail.python.org/mailman/listinfo/python-list

Re: Improvement to imports, what is a better way ?

[Cameron Simpson]

On 18Jan2023 16:10, Dan Kolis  wrote:

I have written a very sizable and elaborate program that uses tKinter for X11 
displays of genomics.
Yet maybe 1 of 6 times it freezes, so I decided to extract the minimum 
that works perfectly and add back big pieces. It does it both running 
.pyc and in VSCode.


The other thing you can do is put in a heap of print() calls in 
strategic places. See if you can get a handle on what its doing when it 
freezes. This might be iterative, zooming in on where this happens, if 
it's a specific thing.


I know this is vague. Once you find its stalling in a particular 
function (if it is) you may be able to run that function directly. Also, 
a print() at the top abd bottom/return of the stalling function. And so 
on.


If the GUI freezes, maybe you're running something time consuming inline 
in the event loop, such that it blocks reaching the "idle" point where 
the tk gui updates widgets and polls for user actions.


Cheers,
Cameron Simpson 
--
https://mail.python.org/mailman/listinfo/python-list

Re: Improvement to imports, what is a better way ?

[Cameron Simpson]

On 18Jan2023 12:57, Dan Kolis  wrote:
I'm not sure what to do. Do either / both know if there is a way to 
make it parse each import list to bytecode in one shot ??


Python usually compiles files to bytecode and leaves those around as 
.pyc files to be used until the original source is modified. And anyway, 
an imported module is only loaded a ready once; imports are kept track 
of in the sys.modules dict, and just pulled directly from there when the 
same module is requests by another import.


The hop around read keeps making me worry it migth leave some memory 
leak or something.


I don't think you've described this. I don't know what you mean here.

Cheers,
Cameron Simpson 
--
https://mail.python.org/mailman/listinfo/python-list

Re: Mailing-Lists (pointer)

[Cameron Simpson]

On 11Jan2023 19:10, Dieter Maurer  wrote:

Cameron Simpson wrote at 2023-1-11 08:37 +1100:

...
There's a Discourse forum over at discuss.python.org. I use it in
"mailing list mode" and do almost all my interactions via email, exactly
as I do for python-list. [...]


I am also using the Plone `Discourse` forum in "mailing list mode".
It now works quite well but it took some years before reaching this state.


Some of this kind of thing will be because the Doscourse devs, as you 
might imagine, are forum/web-first people while you and I are 
email-first people. So they won't notice email shortcomings as readily.


That said, they do seem very engaged and willing to chase and fix bugs 
if they can be identified.



For a very long time, my mail replies did not reach the forum reliably.
My latest complaint (more than half a year ago): when I had visited
the forum via `http` (I did this occasionally to verify
my reply has reached the forum), it sometimes thought, I had
seen a new message and did not inform me about it via mail.


There's certainly still an issue where some messages are not reliably 
sent via email when the inbound needs-spam-review filter flags a 
message/post, particularly the first post; they're fixing that right now 
:-)



Meanwhile, all replies seem to arrive reliably and I no longer
use `http` for access. Therefore, I do not know whether
the behavior described above still persists.


One day everything will be perfect!

Cheers,
Cameron Simpson 
--
https://mail.python.org/mailman/listinfo/python-list

Re: Tkinter GUI freezing, used Thread then encountered RuntimeError: threads can only be started once

[Cameron Simpson]

On 10Jan2023 18:32, MRAB  wrote:
I don't like how you're passing Thread...start as an argument. IMHO, it 
would be better/cleaner to pass a plain function, even if the only 
thing that function does is to start the thread.


Yes, and this is likely the thing causing the cited exception "threads 
can only be started once". Your setup of the button with the action 
defined as:


Thread().start

creates a _single_ new Thread _when you define the button_, and makes 
hte button callback try to start it. On the second and following 
callback, you're trying to start the _same_ single Thread again.


Do as MRAB suggests and have the callback create-and-start a Thread 
instead of just starting an _existing_ Thread.


Also, for simple quick things there's no need to use a Thread at all. If 
the final version of the programme is going to do something long running 
at that point, then sure.


I can't tell what 'change_flag' is doing because of the formatting 
issue. Is it doing GUI stuff? In a thread? If yes, don't do that. The 
GUI doesn't like that. Only the main thread should do GUI stuff.


Aye. This is very important in almost all GUI toolkits.

Bit me very badly with Qt once, badly in that the segfaults (yes!  
segfaults! in a Python app!) were erratic and very timing dependent, 
making them hard to reproduce and understand. It wasn't until I 
_realised_ it was thread/concurrency related that I could fix it.


Note that in Tk you can have a callback do GUI work, just not in a 
separate thread.


Cheers,
Cameron Simpson 
--
https://mail.python.org/mailman/listinfo/python-list

Re: Mailing-Lists (pointer)

[Cameron Simpson]

On 10Jan2023 08:45, Chris Green  wrote:

dn  wrote:

See also the wisdom of enabling comp.lang.python and python-list as
'mirrors', enabling those who prefer one mechanism/client to another,
yet maintaining a single 'community'.


Yes, this is important I think.  Plus, if possible, if it's decided to
move to a forum format make that accessible by E-Mail.


There's a Discourse forum over at discuss.python.org. I use it in 
"mailing list mode" and do almost all my interactions via email, exactly 
as I do for python-list. Posts come to me and land in the same local 
mail folder I use for python-list. My replies land on the forum as 
expected (and of course also go by email to those members who have 
turned that mode on).


So I'm using both the new forum and the currently mailing list still, 
and broadly in exactly the same way.


Cheers,
Cameron Simpson 
--
https://mail.python.org/mailman/listinfo/python-list

Re: Suggestion: Regex string specifier like r and f

[Cameron Simpson]

On 08Jan2023 12:44, Raphael Santiago  wrote:

Maybe something like re""
It should behave exactly like a raw string but would be useful for syntax
highlighting and debugging. Perhaps also for type hinting expected regex
input (don't know if this is feasible).


A nice idea. (Though I'm personally reluctant to make regexps any 
"easier" to use than they are already, a discussion for another thread.)


But the IDE/syntax help is cool.

There was discussion of essentially this idea just recently here:
https://discuss.python.org/t/allow-for-arbitrary-string-prefix-of-strings/19740
which you will find interesting.

I think it fizzled for 2 main reasons:
- string prefixes current have semantic meaning for the Python 
  parser/interpreter, while an artbitrary prefix ... doesn't

- "too hard"? to get IDEs to use it?

Of these, I only find the former compelling, and one could argue that 
type annotations already form a "has no effect" syntax extension to 
Python, so we're already in that playground :-)


Cheers,
Cameron Simpson 
--
https://mail.python.org/mailman/listinfo/python-list

Re: What should go to stdout/stderr and why Python logging write everything to stderr?

[Cameron Simpson]

On Jan 3, 2023, at 10:38 AM, c.bu...@posteo.jp wrote:
this posting isn't about asking for a technical solution. My 
intention is to understand the design decision Python's core developers made in

context of that topic.

The logging module write everything to stderr no matter which logging
level is used.


Sure. They're logs.


The argparse module does it more differentiated. If arguments are
mandatory but none are given then argparse produce a short version of
the usage info; on stderr. If the user request the usage info via "-h"
it goes to stdout. This is the behavior I would expect.


Error messages to stderr. _Requested output_ to stdout. Good with me.


Why does logging behave different? DEBUG and INFO imho should go to
stdout not stderr.


I entirely disagree.

Consider a shell pipeline:

cmd1 | cmd2 | cmd3

where `cmd2` and `cmd3` parse data coming down the pipes. Letting 
logging info into those data streams would produce garbage. This 
separation between core programme output and error/logging output is the 
basic reason that the two streams exist, and the reason logging goes to 
stderr by default.


You're conflating "normal output" with INFO, which is not a substitute 
for eg `print()`.



Of course I could modify the handlers etc. to workaround this. But I
assume that there was something in mind of the Python developers when
they decided that.


Basicly, logging data is not the normal output. So it does not get sent 
to stdout. The log level doesn't change that characteristic.



My goal is not to divide between the use of print() or logging.info()
in my own code. This would mess up a lot.


I think you should, or you should fiddle the handlers as you previous 
resist doing if it is your goal.


But keep in mind that if you do that globally (eg in the root logger) it 
will affect _all_ logging calls, not merely your own, and other 
libraries will assume they can log at whatever level and not pollute 
stdout with their logs.


Basicly, logging isn't "output".

Cheers,
Cameron Simpson 
--
https://mail.python.org/mailman/listinfo/python-list