Re: PyGILState_Ensure() deadlocks, why?

[Barry Scott via Python-list]

> On 7 Jul 2024, at 23:21, Barry via Python-list  wrote:
> 
> 
> 
>> On 7 Jul 2024, at 22:09, Tomas Ukkonen via Python-list 
>>  wrote:
>> 
>>Py_Initialize();
> 
> You also need to tell python to init threading.

I'm in front of my dev machine now and checking up on threading.

There is no longer any extra init for threads required.

> I think you are missing more python setup code before you can use threads.
> Also i think you need to tell python that your thread wants to call into 
> python.
> But I an not near my dev system to research this for you.

You are right to use PyGILState_Ensure()

But as MRAB says the main thread is holding the GIL.

> 
> I have code to use python from C++ in my pysvn project.
> See the code starting a line 354 in 
> https://sourceforge.net/p/pysvn/code/HEAD/tree/trunk/pysvn/Extension/Source/pysvn.cpp
> That saves the thread state and restores it.

You still might find the classes I wrong to manage GIL acquire and release 
interesting.
I have the C++ type system enforcing the rules of acquire and release.
As well as RAII ensuring never to leave a block with the GIL in the wrong state.

Barry

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: fCONV_AUSRICHTG is not defined - Why?

[Egon Frerich via Python-list]

Am 07.11.23 um 20:10 schrieb MRAB via Python-list:

On 2023-11-07 18:30, dn via Python-list wrote:

On 08/11/2023 06.47, Egon Frerich via Python-list wrote:
I've no idea why this happens. In a module there are lists and 
definitions:

...

 ["%s%s%s " % (i[fCONV_AUSRICHTG], i[fLG2], i[fTYP]) for i in 
Felder])
   File "/home/egon/Entw/Geldspur/geldspur/gui/GUI_Konfig.py", line 
90, in 
 ["%s%s%s " % (i[fCONV_AUSRICHTG], i[fLG2], i[fTYP]) for i in 
Felder])

NameError: name 'fCONV_AUSRICHTG' is not defined

You see "Felder" and with "0 0 3 4" the correct value 4 for 
fCONV_AUSRICHTG. But there is the NameError.


What does  mean? Is there a change from python2 to python3?


Works for me (Python 3.11 on Fedora-Linux 37)
- both as a script, and simple/single import.

What happens when you extract the second dimension's definitions into a
module of their own, and import that (with/out less-sophisticated join)?


The missing detail is this line from the traceback:

   File "/home/egon/Entw/Geldspur/geldspur/gui/GUI_Konfig.py", line 11,
in 
 class GUIcfg:

You are right. The list comprehension has to be outside the class. The 
scope rules have been changed python2 and python3.


Egon




Here's a small example that shows the problem:

8<
#!python3.11
# -*- encoding: utf-8 -*-

class Test:
    hello = "hello"
    print(hello)
    print([[zero] for _ in range(4)])
8<

and its traceback:

8<
hello
Traceback (most recent call last):
  File "C:\Projects\regex3\test_clipboard.py", line 4, in 
    class Test:
  File "C:\Projects\regex3\test_clipboard.py", line 7, in Test
    print([zero for _ in range(4)])
 ^^
  File "C:\Projects\regex3\test_clipboard.py", line 7, in 
    print([zero for _ in range(4)])
   
NameError: name 'zero' is not defined
8<

'zero' is visible in:

    print(hello)

but not in:

    print([zero for _ in range(4)])

Something to do with how scoping is implemented in comprehensions?



--
https://mail.python.org/mailman/listinfo/python-list

Re: fCONV_AUSRICHTG is not defined - Why?

[Greg Ewing via Python-list]

On 8/11/23 8:10 am, MRAB wrote:

Something to do with how scoping is implemented in comprehensions?


Yes, together with the way class scopes work during class construction.

Behind the scenes, the body of a listcomp happens to be implemented
as a nested function.

Usually you don't notice this, but while a class is being built,
its scope doesn't count as an enlosing scope for functions defined
within the class.

This is necessary, otherwise all of a class's attributes would be 
visible inside its methods, which isn't what we want. However, it

leads to some odd corner cases, such as this one.

There are various ways you could work around this. I would suggest
moving the offending code outside the class and qualifying the
constants it uses with the class name.

--
Greg
--
https://mail.python.org/mailman/listinfo/python-list

Re: fCONV_AUSRICHTG is not defined - Why?

[MRAB via Python-list]

On 2023-11-07 20:56, Thomas Passin via Python-list wrote:

On 11/7/2023 3:29 PM, MRAB via Python-list wrote:

On 2023-11-07 19:20, Jim Schwartz via Python-list wrote:
Where do you define fCONV_AUSRICHTG? It must be initialized or defined 
somewhere. Did you leave out a statement from the python 2 version?



It's given its value here:

     (
     fNAME,
     fLG1,
     fLG2,
     fTYP,
     fCONV_AUSRICHTG,
     fENTRY_AUSRICHTG,
     fTEXT_AUSRICHTUNG,
     fHOLFUNKT,
     fPRUEFFUNKT,
     fPRUEF_ARG,
     ) = list(range(10))



This construction is a sneaky way to assign index numbers to list
entries.  A simplified example:

  >>> S1 = 'string 1'
  >>> S2 = 'string 2'
  >>> (fS1, fS2) = list(range(2))
  >>> fS1
0
  >>>
  >>> fS2
1


You don't need the 'list', though: range(...) will work on its own.

[snip]

--
https://mail.python.org/mailman/listinfo/python-list

Re: fCONV_AUSRICHTG is not defined - Why?

[Thomas Passin via Python-list]

On 11/7/2023 3:29 PM, MRAB via Python-list wrote:

On 2023-11-07 19:20, Jim Schwartz via Python-list wrote:
Where do you define fCONV_AUSRICHTG? It must be initialized or defined 
somewhere. Did you leave out a statement from the python 2 version?



It's given its value here:

     (
     fNAME,
     fLG1,
     fLG2,
     fTYP,
     fCONV_AUSRICHTG,
     fENTRY_AUSRICHTG,
     fTEXT_AUSRICHTUNG,
     fHOLFUNKT,
     fPRUEFFUNKT,
     fPRUEF_ARG,
     ) = list(range(10))



This construction is a sneaky way to assign index numbers to list 
entries.  A simplified example:


>>> S1 = 'string 1'
>>> S2 = 'string 2'
>>> (fS1, fS2) = list(range(2))
>>> fS1
0
>>>
>>> fS2
1





On Nov 7, 2023, at 1:06 PM, Thomas Passin via Python-list 
 wrote:


On 11/7/2023 12:47 PM, Egon Frerich via Python-list wrote:
I've no idea why this happens. In a module there are lists and 
definitions:

    Felder = [
    # Name   lg1  lg2 typ   Ausrichtung Holen Prüfen Prüfvorg
    ["Jahr", 4, 5, "u", "", "right", "center"],
    ["Monat", 2, 5, "u", "", "right", "center"],
    ["Tag", 2, 3, "u", "", "right", "center"],
    ["Belegnr", 5, 7, "s", "", "right", "center"],
    ["Bank", 2, 4, "u", "", "center", "center"],
    ["Art", 2, 3, "u", "", "center", "center"],
    ["Aufg", 2, 4, "u", "", "center", "center"],
    ["Text", 25, 25, "s", "-", "left", "left"],
    ["Ergänzung", 12, 12, "s", "-", "left", "left"],
    ["Betrag", 13, 13, "s", "", "right", "right"],
    ["W", 1, 2, "s", "", "center", "center"],
    ["WBetrag", 7, 7, "s", "", "right", "right"],
    ["Kurs", 6, 6, "s", "", "right", "right"],
    ]
    "Reihenfolge in der Dimension 1"
    (
    fJAHR,
    fMONAT,
    fTAG,
    fBELEGNR,
    fBANK,
    fART,
    fAUFGABE,
    fTEXT,
    fTEXTERG,
    fBETRAG,
    fWAEHRUNG,
    fBETRAGinWAEHRUNG,
    fUMRECHNUNGSKURS,
    ) = list(range(13))
    "Reihenfolge in der Dimension 2"
    (
    fNAME,
    fLG1,
    fLG2,
    fTYP,
    fCONV_AUSRICHTG,
    fENTRY_AUSRICHTG,
    fTEXT_AUSRICHTUNG,
    fHOLFUNKT,
    fPRUEFFUNKT,
    fPRUEF_ARG,
    ) = list(range(10))
Two lines with  test statements follow and the statement which 
produces an error:

    print(Felder)
    print(fJAHR, fNAME, fTYP, fCONV_AUSRICHTG)
    akette = "%" + "%".join(
    ["%s%s%s " % (i[fCONV_AUSRICHTG], i[fLG2], i[fTYP]) for i in 
Felder])

The traceback shows:
$ python3 testGeldspurGUI.py
[['Jahr', 4, 5, 'u', '', 'right', 'center'], ['Monat', 2, 5, 'u', 
'', 'right', 'center'], ['Tag', 2, 3, 'u', '', 'right', 'center'], 
['Belegnr', 5, 7, 's', '', 'right', 'center'], ['Bank', 2, 4, 'u', 
'', 'center', 'center'], ['Art', 2, 3, 'u', '', 'center', 'center'], 
['Aufg', 2, 4, 'u', '', 'center', 'center'], ['Text', 25, 25, 's', 
'-', 'left', 'left'], ['Ergänzung', 12, 12, 's', '-', 'left', 
'left'], ['Betrag', 13, 13, 's', '', 'right', 'right'], ['W', 1, 2, 
's', '', 'center', 'center'], ['WBetrag', 7, 7, 's', '', 'right', 
'right'], ['Kurs', 6, 6, 's', '', 'right', 'right']]

0 0 3 4
Traceback (most recent call last):
  File "/home/egon/Entw/Geldspur/geldspur/testGeldspurGUI.py", line 
15, in 

    from tests.testU2 import testU2
  File "/home/egon/Entw/Geldspur/geldspur/tests/testU2.py", line 9, 
in 

    from gui.GUI_Konfig import GUIcfg
  File "/home/egon/Entw/Geldspur/geldspur/gui/GUI_Konfig.py", line 
11, in 

    class GUIcfg:
  File "/home/egon/Entw/Geldspur/geldspur/gui/GUI_Konfig.py", line 
90, in GUIcfg
    ["%s%s%s " % (i[fCONV_AUSRICHTG], i[fLG2], i[fTYP]) for i in 
Felder])
  File "/home/egon/Entw/Geldspur/geldspur/gui/GUI_Konfig.py", line 
90, in 
    ["%s%s%s " % (i[fCONV_AUSRICHTG], i[fLG2], i[fTYP]) for i in 
Felder])

NameError: name 'fCONV_AUSRICHTG' is not defined
You see "Felder" and with "0 0 3 4" the correct value 4 for 
fCONV_AUSRICHTG. But there is the NameError.

What does  mean? Is there a change from python2 to python3?


You are using a syntax that I don't understand, but "listcomp" means 
a list comprehenson.






--
https://mail.python.org/mailman/listinfo/python-list

Re: fCONV_AUSRICHTG is not defined - Why?

[MRAB via Python-list]

On 2023-11-07 19:20, Jim Schwartz via Python-list wrote:

Where do you define fCONV_AUSRICHTG? It must be initialized or defined 
somewhere. Did you leave out a statement from the python 2 version?


It's given its value here:

(
fNAME,
fLG1,
fLG2,
fTYP,
fCONV_AUSRICHTG,
fENTRY_AUSRICHTG,
fTEXT_AUSRICHTUNG,
fHOLFUNKT,
fPRUEFFUNKT,
fPRUEF_ARG,
) = list(range(10))




On Nov 7, 2023, at 1:06 PM, Thomas Passin via Python-list 
 wrote:

On 11/7/2023 12:47 PM, Egon Frerich via Python-list wrote:

I've no idea why this happens. In a module there are lists and definitions:
Felder = [
# Name   lg1  lg2 typ   Ausrichtung Holen Prüfen Prüfvorg
["Jahr", 4, 5, "u", "", "right", "center"],
["Monat", 2, 5, "u", "", "right", "center"],
["Tag", 2, 3, "u", "", "right", "center"],
["Belegnr", 5, 7, "s", "", "right", "center"],
["Bank", 2, 4, "u", "", "center", "center"],
["Art", 2, 3, "u", "", "center", "center"],
["Aufg", 2, 4, "u", "", "center", "center"],
["Text", 25, 25, "s", "-", "left", "left"],
["Ergänzung", 12, 12, "s", "-", "left", "left"],
["Betrag", 13, 13, "s", "", "right", "right"],
["W", 1, 2, "s", "", "center", "center"],
["WBetrag", 7, 7, "s", "", "right", "right"],
["Kurs", 6, 6, "s", "", "right", "right"],
]
"Reihenfolge in der Dimension 1"
(
fJAHR,
fMONAT,
fTAG,
fBELEGNR,
fBANK,
fART,
fAUFGABE,
fTEXT,
fTEXTERG,
fBETRAG,
fWAEHRUNG,
fBETRAGinWAEHRUNG,
fUMRECHNUNGSKURS,
) = list(range(13))
"Reihenfolge in der Dimension 2"
(
fNAME,
fLG1,
fLG2,
fTYP,
fCONV_AUSRICHTG,
fENTRY_AUSRICHTG,
fTEXT_AUSRICHTUNG,
fHOLFUNKT,
fPRUEFFUNKT,
fPRUEF_ARG,
) = list(range(10))
Two lines with  test statements follow and the statement which produces an 
error:
print(Felder)
print(fJAHR, fNAME, fTYP, fCONV_AUSRICHTG)
akette = "%" + "%".join(
["%s%s%s " % (i[fCONV_AUSRICHTG], i[fLG2], i[fTYP]) for i in Felder])
The traceback shows:
$ python3 testGeldspurGUI.py
[['Jahr', 4, 5, 'u', '', 'right', 'center'], ['Monat', 2, 5, 'u', '', 'right', 
'center'], ['Tag', 2, 3, 'u', '', 'right', 'center'], ['Belegnr', 5, 7, 's', 
'', 'right', 'center'], ['Bank', 2, 4, 'u', '', 'center', 'center'], ['Art', 2, 
3, 'u', '', 'center', 'center'], ['Aufg', 2, 4, 'u', '', 'center', 'center'], 
['Text', 25, 25, 's', '-', 'left', 'left'], ['Ergänzung', 12, 12, 's', '-', 
'left', 'left'], ['Betrag', 13, 13, 's', '', 'right', 'right'], ['W', 1, 2, 
's', '', 'center', 'center'], ['WBetrag', 7, 7, 's', '', 'right', 'right'], 
['Kurs', 6, 6, 's', '', 'right', 'right']]
0 0 3 4
Traceback (most recent call last):
  File "/home/egon/Entw/Geldspur/geldspur/testGeldspurGUI.py", line 15, in 

from tests.testU2 import testU2
  File "/home/egon/Entw/Geldspur/geldspur/tests/testU2.py", line 9, in 
from gui.GUI_Konfig import GUIcfg
  File "/home/egon/Entw/Geldspur/geldspur/gui/GUI_Konfig.py", line 11, in 

class GUIcfg:
  File "/home/egon/Entw/Geldspur/geldspur/gui/GUI_Konfig.py", line 90, in GUIcfg
["%s%s%s " % (i[fCONV_AUSRICHTG], i[fLG2], i[fTYP]) for i in Felder])
  File "/home/egon/Entw/Geldspur/geldspur/gui/GUI_Konfig.py", line 90, in 

["%s%s%s " % (i[fCONV_AUSRICHTG], i[fLG2], i[fTYP]) for i in Felder])
NameError: name 'fCONV_AUSRICHTG' is not defined
You see "Felder" and with "0 0 3 4" the correct value 4 for fCONV_AUSRICHTG. 
But there is the NameError.
What does  mean? Is there a change from python2 to python3?


You are using a syntax that I don't understand, but "listcomp" means a list 
comprehenson.



--
https://mail.python.org/mailman/listinfo/python-list

Re: fCONV_AUSRICHTG is not defined - Why?

[Jim Schwartz via Python-list]

Where do you define fCONV_AUSRICHTG? It must be initialized or defined 
somewhere. Did you leave out a statement from the python 2 version?  

Sent from my iPhone

> On Nov 7, 2023, at 1:06 PM, Thomas Passin via Python-list 
>  wrote:
> 
> On 11/7/2023 12:47 PM, Egon Frerich via Python-list wrote:
>> I've no idea why this happens. In a module there are lists and definitions:
>> Felder = [
>> # Name   lg1  lg2 typ   Ausrichtung Holen Prüfen Prüfvorg
>> ["Jahr", 4, 5, "u", "", "right", "center"],
>> ["Monat", 2, 5, "u", "", "right", "center"],
>> ["Tag", 2, 3, "u", "", "right", "center"],
>> ["Belegnr", 5, 7, "s", "", "right", "center"],
>> ["Bank", 2, 4, "u", "", "center", "center"],
>> ["Art", 2, 3, "u", "", "center", "center"],
>> ["Aufg", 2, 4, "u", "", "center", "center"],
>> ["Text", 25, 25, "s", "-", "left", "left"],
>> ["Ergänzung", 12, 12, "s", "-", "left", "left"],
>> ["Betrag", 13, 13, "s", "", "right", "right"],
>> ["W", 1, 2, "s", "", "center", "center"],
>> ["WBetrag", 7, 7, "s", "", "right", "right"],
>> ["Kurs", 6, 6, "s", "", "right", "right"],
>> ]
>> "Reihenfolge in der Dimension 1"
>> (
>> fJAHR,
>> fMONAT,
>> fTAG,
>> fBELEGNR,
>> fBANK,
>> fART,
>> fAUFGABE,
>> fTEXT,
>> fTEXTERG,
>> fBETRAG,
>> fWAEHRUNG,
>> fBETRAGinWAEHRUNG,
>> fUMRECHNUNGSKURS,
>> ) = list(range(13))
>> "Reihenfolge in der Dimension 2"
>> (
>> fNAME,
>> fLG1,
>> fLG2,
>> fTYP,
>> fCONV_AUSRICHTG,
>> fENTRY_AUSRICHTG,
>> fTEXT_AUSRICHTUNG,
>> fHOLFUNKT,
>> fPRUEFFUNKT,
>> fPRUEF_ARG,
>> ) = list(range(10))
>> Two lines with  test statements follow and the statement which produces an 
>> error:
>> print(Felder)
>> print(fJAHR, fNAME, fTYP, fCONV_AUSRICHTG)
>> akette = "%" + "%".join(
>> ["%s%s%s " % (i[fCONV_AUSRICHTG], i[fLG2], i[fTYP]) for i in Felder])
>> The traceback shows:
>> $ python3 testGeldspurGUI.py
>> [['Jahr', 4, 5, 'u', '', 'right', 'center'], ['Monat', 2, 5, 'u', '', 
>> 'right', 'center'], ['Tag', 2, 3, 'u', '', 'right', 'center'], ['Belegnr', 
>> 5, 7, 's', '', 'right', 'center'], ['Bank', 2, 4, 'u', '', 'center', 
>> 'center'], ['Art', 2, 3, 'u', '', 'center', 'center'], ['Aufg', 2, 4, 'u', 
>> '', 'center', 'center'], ['Text', 25, 25, 's', '-', 'left', 'left'], 
>> ['Ergänzung', 12, 12, 's', '-', 'left', 'left'], ['Betrag', 13, 13, 's', '', 
>> 'right', 'right'], ['W', 1, 2, 's', '', 'center', 'center'], ['WBetrag', 7, 
>> 7, 's', '', 'right', 'right'], ['Kurs', 6, 6, 's', '', 'right', 'right']]
>> 0 0 3 4
>> Traceback (most recent call last):
>>   File "/home/egon/Entw/Geldspur/geldspur/testGeldspurGUI.py", line 15, in 
>> 
>> from tests.testU2 import testU2
>>   File "/home/egon/Entw/Geldspur/geldspur/tests/testU2.py", line 9, in 
>> 
>> from gui.GUI_Konfig import GUIcfg
>>   File "/home/egon/Entw/Geldspur/geldspur/gui/GUI_Konfig.py", line 11, in 
>> 
>> class GUIcfg:
>>   File "/home/egon/Entw/Geldspur/geldspur/gui/GUI_Konfig.py", line 90, in 
>> GUIcfg
>> ["%s%s%s " % (i[fCONV_AUSRICHTG], i[fLG2], i[fTYP]) for i in Felder])
>>   File "/home/egon/Entw/Geldspur/geldspur/gui/GUI_Konfig.py", line 90, in 
>> 
>> ["%s%s%s " % (i[fCONV_AUSRICHTG], i[fLG2], i[fTYP]) for i in Felder])
>> NameError: name 'fCONV_AUSRICHTG' is not defined
>> You see "Felder" and with "0 0 3 4" the correct value 4 for fCONV_AUSRICHTG. 
>> But there is the NameError.
>> What does  mean? Is there a change from python2 to python3?
> 
> You are using a syntax that I don't understand, but "listcomp" means a list 
> comprehenson.
> 
> -- 
> https://mail.python.org/mailman/listinfo/python-list

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: fCONV_AUSRICHTG is not defined - Why?

[MRAB via Python-list]

On 2023-11-07 18:30, dn via Python-list wrote:

On 08/11/2023 06.47, Egon Frerich via Python-list wrote:

I've no idea why this happens. In a module there are lists and definitions:

...


     ["%s%s%s " % (i[fCONV_AUSRICHTG], i[fLG2], i[fTYP]) for i in Felder])
   File "/home/egon/Entw/Geldspur/geldspur/gui/GUI_Konfig.py", line 90, 
in 

     ["%s%s%s " % (i[fCONV_AUSRICHTG], i[fLG2], i[fTYP]) for i in Felder])
NameError: name 'fCONV_AUSRICHTG' is not defined

You see "Felder" and with "0 0 3 4" the correct value 4 for 
fCONV_AUSRICHTG. But there is the NameError.


What does  mean? Is there a change from python2 to python3?


Works for me (Python 3.11 on Fedora-Linux 37)
- both as a script, and simple/single import.

What happens when you extract the second dimension's definitions into a
module of their own, and import that (with/out less-sophisticated join)?


The missing detail is this line from the traceback:

   File "/home/egon/Entw/Geldspur/geldspur/gui/GUI_Konfig.py", line 11,
in 
 class GUIcfg:

Here's a small example that shows the problem:

8<
#!python3.11
# -*- encoding: utf-8 -*-

class Test:
hello = "hello"
print(hello)
print([[zero] for _ in range(4)])
8<

and its traceback:

8<
hello
Traceback (most recent call last):
  File "C:\Projects\regex3\test_clipboard.py", line 4, in 
class Test:
  File "C:\Projects\regex3\test_clipboard.py", line 7, in Test
print([zero for _ in range(4)])
 ^^
  File "C:\Projects\regex3\test_clipboard.py", line 7, in 
print([zero for _ in range(4)])
   
NameError: name 'zero' is not defined
8<

'zero' is visible in:

print(hello)

but not in:

print([zero for _ in range(4)])

Something to do with how scoping is implemented in comprehensions?

--
https://mail.python.org/mailman/listinfo/python-list

Re: fCONV_AUSRICHTG is not defined - Why?

[Thomas Passin via Python-list]

On 11/7/2023 12:47 PM, Egon Frerich via Python-list wrote:

I've no idea why this happens. In a module there are lists and definitions:

     Felder = [
     # Name   lg1  lg2 typ   Ausrichtung Holen Prüfen Prüfvorg
     ["Jahr", 4, 5, "u", "", "right", "center"],
     ["Monat", 2, 5, "u", "", "right", "center"],
     ["Tag", 2, 3, "u", "", "right", "center"],
     ["Belegnr", 5, 7, "s", "", "right", "center"],
     ["Bank", 2, 4, "u", "", "center", "center"],
     ["Art", 2, 3, "u", "", "center", "center"],
     ["Aufg", 2, 4, "u", "", "center", "center"],
     ["Text", 25, 25, "s", "-", "left", "left"],
     ["Ergänzung", 12, 12, "s", "-", "left", "left"],
     ["Betrag", 13, 13, "s", "", "right", "right"],
     ["W", 1, 2, "s", "", "center", "center"],
     ["WBetrag", 7, 7, "s", "", "right", "right"],
     ["Kurs", 6, 6, "s", "", "right", "right"],
     ]
     "Reihenfolge in der Dimension 1"
     (
     fJAHR,
     fMONAT,
     fTAG,
     fBELEGNR,
     fBANK,
     fART,
     fAUFGABE,
     fTEXT,
     fTEXTERG,
     fBETRAG,
     fWAEHRUNG,
     fBETRAGinWAEHRUNG,
     fUMRECHNUNGSKURS,
     ) = list(range(13))
     "Reihenfolge in der Dimension 2"
     (
     fNAME,
     fLG1,
     fLG2,
     fTYP,
     fCONV_AUSRICHTG,
     fENTRY_AUSRICHTG,
     fTEXT_AUSRICHTUNG,
     fHOLFUNKT,
     fPRUEFFUNKT,
     fPRUEF_ARG,
     ) = list(range(10))


Two lines with  test statements follow and the statement which produces 
an error:


     print(Felder)
     print(fJAHR, fNAME, fTYP, fCONV_AUSRICHTG)
     akette = "%" + "%".join(
     ["%s%s%s " % (i[fCONV_AUSRICHTG], i[fLG2], i[fTYP]) for i in 
Felder])


The traceback shows:

$ python3 testGeldspurGUI.py
[['Jahr', 4, 5, 'u', '', 'right', 'center'], ['Monat', 2, 5, 'u', '', 
'right', 'center'], ['Tag', 2, 3, 'u', '', 'right', 'center'], 
['Belegnr', 5, 7, 's', '', 'right', 'center'], ['Bank', 2, 4, 'u', '', 
'center', 'center'], ['Art', 2, 3, 'u', '', 'center', 'center'], 
['Aufg', 2, 4, 'u', '', 'center', 'center'], ['Text', 25, 25, 's', '-', 
'left', 'left'], ['Ergänzung', 12, 12, 's', '-', 'left', 'left'], 
['Betrag', 13, 13, 's', '', 'right', 'right'], ['W', 1, 2, 's', '', 
'center', 'center'], ['WBetrag', 7, 7, 's', '', 'right', 'right'], 
['Kurs', 6, 6, 's', '', 'right', 'right']]

0 0 3 4
Traceback (most recent call last):
   File "/home/egon/Entw/Geldspur/geldspur/testGeldspurGUI.py", line 15, 
in 

     from tests.testU2 import testU2
   File "/home/egon/Entw/Geldspur/geldspur/tests/testU2.py", line 9, in 


     from gui.GUI_Konfig import GUIcfg
   File "/home/egon/Entw/Geldspur/geldspur/gui/GUI_Konfig.py", line 11, 
in 

     class GUIcfg:
   File "/home/egon/Entw/Geldspur/geldspur/gui/GUI_Konfig.py", line 90, 
in GUIcfg

     ["%s%s%s " % (i[fCONV_AUSRICHTG], i[fLG2], i[fTYP]) for i in Felder])
   File "/home/egon/Entw/Geldspur/geldspur/gui/GUI_Konfig.py", line 90, 
in 

     ["%s%s%s " % (i[fCONV_AUSRICHTG], i[fLG2], i[fTYP]) for i in Felder])
NameError: name 'fCONV_AUSRICHTG' is not defined

You see "Felder" and with "0 0 3 4" the correct value 4 for 
fCONV_AUSRICHTG. But there is the NameError.


What does  mean? Is there a change from python2 to python3?


You are using a syntax that I don't understand, but "listcomp" means a 
list comprehenson.


--
https://mail.python.org/mailman/listinfo/python-list

Re: fCONV_AUSRICHTG is not defined - Why?

[dn via Python-list]

On 08/11/2023 06.47, Egon Frerich via Python-list wrote:

I've no idea why this happens. In a module there are lists and definitions:

...


     ["%s%s%s " % (i[fCONV_AUSRICHTG], i[fLG2], i[fTYP]) for i in Felder])
   File "/home/egon/Entw/Geldspur/geldspur/gui/GUI_Konfig.py", line 90, 
in 

     ["%s%s%s " % (i[fCONV_AUSRICHTG], i[fLG2], i[fTYP]) for i in Felder])
NameError: name 'fCONV_AUSRICHTG' is not defined

You see "Felder" and with "0 0 3 4" the correct value 4 for 
fCONV_AUSRICHTG. But there is the NameError.


What does  mean? Is there a change from python2 to python3?


Works for me (Python 3.11 on Fedora-Linux 37)
- both as a script, and simple/single import.

What happens when you extract the second dimension's definitions into a 
module of their own, and import that (with/out less-sophisticated join)?


--
Regards,
=dn
--
https://mail.python.org/mailman/listinfo/python-list

fCONV_AUSRICHTG is not defined - Why?

[Egon Frerich via Python-list]

I've no idea why this happens. In a module there are lists and definitions:

    Felder = [
    # Name   lg1  lg2 typ   Ausrichtung Holen Prüfen Prüfvorg
    ["Jahr", 4, 5, "u", "", "right", "center"],
    ["Monat", 2, 5, "u", "", "right", "center"],
    ["Tag", 2, 3, "u", "", "right", "center"],
    ["Belegnr", 5, 7, "s", "", "right", "center"],
    ["Bank", 2, 4, "u", "", "center", "center"],
    ["Art", 2, 3, "u", "", "center", "center"],
    ["Aufg", 2, 4, "u", "", "center", "center"],
    ["Text", 25, 25, "s", "-", "left", "left"],
    ["Ergänzung", 12, 12, "s", "-", "left", "left"],
    ["Betrag", 13, 13, "s", "", "right", "right"],
    ["W", 1, 2, "s", "", "center", "center"],
    ["WBetrag", 7, 7, "s", "", "right", "right"],
    ["Kurs", 6, 6, "s", "", "right", "right"],
    ]
    "Reihenfolge in der Dimension 1"
    (
    fJAHR,
    fMONAT,
    fTAG,
    fBELEGNR,
    fBANK,
    fART,
    fAUFGABE,
    fTEXT,
    fTEXTERG,
    fBETRAG,
    fWAEHRUNG,
    fBETRAGinWAEHRUNG,
    fUMRECHNUNGSKURS,
    ) = list(range(13))
    "Reihenfolge in der Dimension 2"
    (
    fNAME,
    fLG1,
    fLG2,
    fTYP,
    fCONV_AUSRICHTG,
    fENTRY_AUSRICHTG,
    fTEXT_AUSRICHTUNG,
    fHOLFUNKT,
    fPRUEFFUNKT,
    fPRUEF_ARG,
    ) = list(range(10))


Two lines with  test statements follow and the statement which produces 
an error:


    print(Felder)
    print(fJAHR, fNAME, fTYP, fCONV_AUSRICHTG)
    akette = "%" + "%".join(
    ["%s%s%s " % (i[fCONV_AUSRICHTG], i[fLG2], i[fTYP]) for i in 
Felder])


The traceback shows:

$ python3 testGeldspurGUI.py
[['Jahr', 4, 5, 'u', '', 'right', 'center'], ['Monat', 2, 5, 'u', '', 
'right', 'center'], ['Tag', 2, 3, 'u', '', 'right', 'center'], 
['Belegnr', 5, 7, 's', '', 'right', 'center'], ['Bank', 2, 4, 'u', '', 
'center', 'center'], ['Art', 2, 3, 'u', '', 'center', 'center'], 
['Aufg', 2, 4, 'u', '', 'center', 'center'], ['Text', 25, 25, 's', '-', 
'left', 'left'], ['Ergänzung', 12, 12, 's', '-', 'left', 'left'], 
['Betrag', 13, 13, 's', '', 'right', 'right'], ['W', 1, 2, 's', '', 
'center', 'center'], ['WBetrag', 7, 7, 's', '', 'right', 'right'], 
['Kurs', 6, 6, 's', '', 'right', 'right']]

0 0 3 4
Traceback (most recent call last):
  File "/home/egon/Entw/Geldspur/geldspur/testGeldspurGUI.py", line 15, 
in 

    from tests.testU2 import testU2
  File "/home/egon/Entw/Geldspur/geldspur/tests/testU2.py", line 9, in 


    from gui.GUI_Konfig import GUIcfg
  File "/home/egon/Entw/Geldspur/geldspur/gui/GUI_Konfig.py", line 11, 
in 

    class GUIcfg:
  File "/home/egon/Entw/Geldspur/geldspur/gui/GUI_Konfig.py", line 90, 
in GUIcfg

    ["%s%s%s " % (i[fCONV_AUSRICHTG], i[fLG2], i[fTYP]) for i in Felder])
  File "/home/egon/Entw/Geldspur/geldspur/gui/GUI_Konfig.py", line 90, 
in 

    ["%s%s%s " % (i[fCONV_AUSRICHTG], i[fLG2], i[fTYP]) for i in Felder])
NameError: name 'fCONV_AUSRICHTG' is not defined

You see "Felder" and with "0 0 3 4" the correct value 4 for 
fCONV_AUSRICHTG. But there is the NameError.


What does  mean? Is there a change from python2 to python3?

Egon



--
https://mail.python.org/mailman/listinfo/python-list

Re: Why doc call `__init__` as a method rather than function?

[scruel tao via Python-list]

Thanks for all repliers:
@Tony & @Cameron, I do know related stuffs about the dunder methods, and your 
explanation just make it to be more clear, thank you!
@Roel, you just caught everyone here, we do miss it even though we know it and 
use it regularly!

@Clara
> its both, depending on how you're getting it.
Might can be more clear: its both, depending on how you're using/getting it.

And I think I can mark this question as resolved, and with the following 
conclusions:
As @Clara mentioned, we need to know that "all methods are functions", so we do 
can call `__init__` as a method or a function, or we can be avoid to have such 
discussion like Dan, and call it "the initializer" (However, you will need to 
think about “what is this is” for other functions :). ).
As @Alan mentioned, and according to the Wikipedia, in computer programming 
field, "method" is:
> A method in object-oriented programming (OOP) is a procedure associated with 
> an object, and generally also a message. An object consists of state data and 
> behavior; these compose an interface, which specifies how the object may be 
> used. A method is a behavior of an object parametrized by a user.

For `__init__` and other functions  in classes, we usually use them by writing 
code `obj.action()`, so we usually will call them as methods, so here, we call 
`action` or `__init__` as a method.
However, if you use them by writing code `Clz.action(obj)`, then you'd better 
(or must?) to call them as functions, and it is not a "daily use case" in daily 
development, and in some languages, this behavior won't even be possible.
**So, its kinda a "Majority rule" to call `__init__` (functions in classes) as 
a method.**

===
BTW, in Wikipedia, the "static methods" section is a very interesting:
> Static methods are meant to be relevant to all the instances of a class 
> rather than to any specific instance.
This explanation might can "group" some functions back to "methods" :) However, 
let's still remember:
All methods are functions, but not every function is a method.

Thanks again for helping, you guys are really nice!

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Why doc call `__init__` as a method rather than function?

[Roel Schroeven via Python-list]

Op 15/09/2023 om 15:05 schreef anthony.flury via Python-list:
Like all of the other  methods you shouldn't ever need to 
call them directly : these are called dunder methods and represent 
functions and features which are called by other operators.


The only recommended way to call A.__init__ is to create an instance 
of A : obj = A() - the __init__ method gets called automatically with 
a newly created object.

There is an exception:

  super().__init__()

to call the base class's __init__ (normally from the derived class's 
__init__)


--
"Human beings, who are almost unique in having the ability to learn from the
experience of others, are also remarkable for their apparent disinclination
to do so."
-- Douglas Adams

--
https://mail.python.org/mailman/listinfo/python-list

Re: Why doc call `__init__` as a method rather than function?

[Cameron Simpson via Python-list]

On 15Sep2023 10:49, scruel tao  wrote:

```python

class A:

...   def __init__(self):
... pass
...
```

On many books and even the official documents, it seems that many authors prefer to call `__init__` 
as a "method" rather than a "function".
The book PYTHON CRASH COURSE  mentioned that "A function that’s part of a class is a 
method.", however, ` A.__init__` tells that `__init__` is a function...


As mentioned, methods in Python _are_ functions for use with a class.


A.__init__



a = A()
a.__init__


>
I wonder how can I call `__init__` as? Consider the output above.
Maybe both are OK?


As you can see, they're both legal expressions.

The thing about `__init__` is that it's usually called automatically 
which you make a new object. Try putting a `print()` call in your 
`__init__` method, then make a new instance of `A`.


The purpose of `__init__` is to initialise the object's attribute/state 
after the basic, empty-ish, object is made.


Like other dunder methods (methods named with double underscores front 
and back) it is called automatically for you. In a subclass the 
`__init__` method calls the subperclass `__init__` and then does 
whatever additional things might be wanted by the subclass.


Let's look at what you used above:

>>> A.__init__


Here's we've just got a reference to the function you supposlied with 
the class definition for class `A`.


This:

>>> a = A()
>>> a.__init__


Here's you've accessed the name `__init__` via an existing instance of 
`A`, your variable `a`. At this point you haven't called it. So you've 
got a callable thing which is a binding of the function to the object 
`a` i.e. when you call it, the "bound method" knows that t is associated 
with `a` and puts that in as the first argument (usually named `self`).


Cheers,
Cameron Simpson 
--
https://mail.python.org/mailman/listinfo/python-list

Re: Why doc call `__init__` as a method rather than function?

[Chris Angelico via Python-list]

On Mon, 18 Sept 2023 at 13:49, anthony.flury via Python-list
 wrote:
>
>
>
> To me __init__ is a method, but that is implemented internally as
> function associated to a class
>

What is a method, if it is not a function associated with a class?

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Why doc call `__init__` as a method rather than function?

[anthony.flury via Python-list]

To me __init__ is a method, but that is implemented internally as 
function associated to a class



When you use A.__init__ on it's own and inspect it, then it will show 
that it is a function object - this is expected. The implementation 
internals of the runtime don't need to have a special implementation for 
a method.


Like all of the other  methods you shouldn't ever need to call 
them directly : these are called dunder methods and represent functions 
and features which are called by other operators.


The only recommended way to call A.__init__ is to create an instance of 
A : obj = A() - the __init__ method gets called automatically with a 
newly created object.


If you did call A.__init__() directly on a an already existing object :

 obj = A()
 A.__init__(obj)

for example - all that would happen is that the object itself would be 
reset : ie the obj's attributes would be reset to whatever the __init__ 
sets them to - if you need to do that it might be better to have a reset 
method.



 Regards,

Tony


-- Original Message --
From: "scruel tao via Python-list" 
To: "python-list@python.org" 
Sent: Friday, 15 Sep, 23 At 11:49
Subject: Why doc call `__init__` as a method rather than function?
```python
class A:
...   def __init__(self):
... pass
...
A.__init__

a = A()
a.__init__
>
```
On many books and even the official documents, it seems that many 
authors prefer to call `__init__` as a "method" rather than a 
"function".
The book PYTHON CRASH COURSE  mentioned that "A function that’s part of 
a class is a method.", however, ` A.__init__` tells that `__init__` is a 
function...

I wonder how can I call `__init__` as? Consider the output above.
Maybe both are OK? If you prefer or think that we must use one of the 
two, please explain the why, I really want to know, thanks!

--
https://mail.python.org/mailman/listinfo/python-list 
<https://mail.python.org/mailman/listinfo/python-list>


-- Anthony Fluryanthony.fl...@btinternet.com
--
https://mail.python.org/mailman/listinfo/python-list

Re: Why doc call `__init__` as a method rather than function?

[Alan Gauld via Python-list]

On 15/09/2023 11:49, scruel tao via Python-list wrote:
> ```python
 class A:
> ...   def __init__(self):
> ... pass

> On many books and even the official documents, it seems that 
> many authors prefer to call `__init__` as a "method" rather 
> than a "function".

That' because in OOP terminology methods are traditionally
implemented as functions defined inside a class (There are
other ways to define methods in other languages, but the
class/function duology is by far the most common.)

What is a method? It is the way that an object handles a
message. OOP is all about objects sending messages to each
other. Many different objects can receive the same message
but they each have their own method of handling that message.
(This is called polymorphism.)

Over time the most common way to implememt OOP in a language
is to invent a "class" structure and to allow functions to
either be defined within it or added to it later. In either
case these functions are what define how a class (and its
object instances) handle a given message. So the function
describes the method for that message. And over time that
has become shortened to the function inside a class *is*
its method.

In practice, the message looks like a function call. And
the method consists of the function body (and/or any
inherited function body). Thus every method is a function
(or set of functions) but not every function is a  method
(or part of one).

-- 
Alan G
Author of the Learn to Program web site
http://www.alan-g.me.uk/
http://www.amazon.com/author/alan_gauld
Follow my photo-blog on Flickr at:
http://www.flickr.com/photos/alangauldphotos


-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Why doc call `__init__` as a method rather than function?

[Clara Spealman via Python-list]

All methods are functions, but not all functions are methods. All methods
are functions in the namespace of a class and when resolved from the class
directly, you'll get the original function. Its only when resolved from an
*instance* of the class, as in self.__init__ or self.any_other_method(),
that the function is wrapped in a method object, which is callable and
binds the function to the particular instance passed as "self".

So the simple answer is "its both". The only slightly less simple answer is
"its both, depending on how you're getting it."

On Fri, Sep 15, 2023 at 6:53 AM scruel tao via Python-list <
python-list@python.org> wrote:

> ```python
> >>> class A:
> ...   def __init__(self):
> ... pass
> ...
> >>> A.__init__
> 
> >>> a = A()
> >>> a.__init__
> >
> ```
>
> On many books and even the official documents, it seems that many authors
> prefer to call `__init__` as a "method" rather than a "function".
> The book PYTHON CRASH COURSE  mentioned that "A function that’s part of a
> class is a method.", however, ` A.__init__` tells that `__init__` is a
> function...
>
> I wonder how can I call `__init__` as? Consider the output above.
> Maybe both are OK? If you prefer or think that we must use one of the two,
> please explain the why, I really want to know, thanks!
> --
> https://mail.python.org/mailman/listinfo/python-list
>
>

-- 

CALVIN SPEALMAN

SENIOR QUALITY ENGINEER

calvin.speal...@redhat.com  M: +1.336.210.5107
[image: https://red.ht/sig] <https://red.ht/sig>
TRIED. TESTED. TRUSTED. <https://redhat.com/trusted>
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Why doc call `__init__` as a method rather than function?

[Dan Sommers via Python-list]

On 2023-09-15 at 10:49:10 +,
scruel tao via Python-list  wrote:

> ```python
> >>> class A:
> ...   def __init__(self):
> ... pass
> ...
> >>> A.__init__
> 
> >>> a = A()
> >>> a.__init__
> >
> ```
> 
> On many books and even the official documents, it seems that many authors 
> prefer to call `__init__` as a "method" rather than a "function".
> The book PYTHON CRASH COURSE  mentioned that "A function that’s part of a 
> class is a method.", however, ` A.__init__` tells that `__init__` is a 
> function...

I always call __init__ "the initializer."  YMMV.

> I wonder how can I call `__init__` as? Consider the output above.
> Maybe both are OK? If you prefer or think that we must use one of the two, 
> please explain the why, I really want to know, thanks!

Usually, you don't call (or even refer to) __init__ from your
application.  One __init__ can call another one in the case of
initializing superclasses.

When you evaluate A(), Python calls __init__ for you.  You can see this
if you add something "visible" to __init__, like a print statement.
-- 
https://mail.python.org/mailman/listinfo/python-list

Why doc call `__init__` as a method rather than function?

[scruel tao via Python-list]

```python
>>> class A:
...   def __init__(self):
... pass
...
>>> A.__init__

>>> a = A()
>>> a.__init__
>
```

On many books and even the official documents, it seems that many authors 
prefer to call `__init__` as a "method" rather than a "function".
The book PYTHON CRASH COURSE  mentioned that "A function that’s part of a class 
is a method.", however, ` A.__init__` tells that `__init__` is a function...

I wonder how can I call `__init__` as? Consider the output above.
Maybe both are OK? If you prefer or think that we must use one of the two, 
please explain the why, I really want to know, thanks!
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Why do I always get an exception raised in this __init__()?

[Chris Green via Python-list]

Alan Gauld  wrote:
> On 31/08/2023 22:15, Chris Green via Python-list wrote:
> 
> > class Gpiopin:
> > 
> > def __init__(self, pin):
> > # 
> > #  
> > # scan through the GPIO chips to find the line/pin we want 
> > # 
> > for c in ['gpiochip0', 'gpiochip1', 'gpiochip2', 'gpiochip3']:
> >  
> > chip = gpiod.Chip(c)
> > for l in range(32):
> > line = chip.get_line(l)
> > if pin in line.name():
> > print("Found: ", line.name())
> > return
> > else:
> > raise ValueError("Can't find pin '" + pin + "'")
> 
> You don't store the line anywhere.
> You need to use self.line
> self.line = chip.get_line(l)
> if pin...
> 
> > def print_name(self): 
> > print (self.line.name()) 
> >  
> > def set(self): 
> > self.line.set_value(1) 
> >
> > def clear(self): 
> > self.line.set_value(0) 
> 
> As you do here.
> 
Yes, OK, absolutely.  However that wasn't my original rather basic
problem which was, as I said, that I wasn't running the code I was
looking at.

The above was just a quick hack from some even cruder code doing the
same job, trying to develop it into something better and more general.

It's all on a headless Beaglebone Black (bit like a Raspberry Pi) so
I'm doing everything via multiple ssh connections and sometimes this
results in "the left hand not knowing what the right hand is doing"!

-- 
Chris Green
·
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Why do I always get an exception raised in this __init__()?

[Alan Gauld via Python-list]

On 31/08/2023 22:15, Chris Green via Python-list wrote:

> class Gpiopin:
> 
> def __init__(self, pin):
> # 
> #  
> # scan through the GPIO chips to find the line/pin we want 
> # 
> for c in ['gpiochip0', 'gpiochip1', 'gpiochip2', 'gpiochip3']:
>  
> chip = gpiod.Chip(c)
> for l in range(32):
> line = chip.get_line(l)
> if pin in line.name():
> print("Found: ", line.name())
> return
> else:
> raise ValueError("Can't find pin '" + pin + "'")

You don't store the line anywhere.
You need to use self.line
self.line = chip.get_line(l)
if pin...

> def print_name(self): 
> print (self.line.name()) 
>  
> def set(self): 
> self.line.set_value(1) 
>
> def clear(self): 
> self.line.set_value(0) 

As you do here.

-- 
Alan G
Author of the Learn to Program web site
http://www.alan-g.me.uk/
http://www.amazon.com/author/alan_gauld
Follow my photo-blog on Flickr at:
http://www.flickr.com/photos/alangauldphotos


-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Why do I always get an exception raised in this __init__()?

[Larry Martell via Python-list]

On Thu, Aug 31, 2023 at 3:19 PM Chris Green via Python-list
 wrote:
>
> I'm obviously doing something very silly here but at the moment I
> can't see what.
>
> Here's the code:-
>
> #!/usr/bin/python3
> #
> #
> # GPIO
> #
> import gpiod
> #
> #
> # Simple wrapper class for gpiod to make set and clearing outputs
> easier
> #
> class Gpiopin:
>
> def __init__(self, pin):
> #
> #
> # scan through the GPIO chips to find the line/pin we want
> #
> for c in ['gpiochip0', 'gpiochip1', 'gpiochip2', 'gpiochip3']:
>
> chip = gpiod.Chip(c)
> for l in range(32):
> line = chip.get_line(l)
> if pin in line.name():
> print("Found: ", line.name())
> return
> else:
> raise ValueError("Can't find pin '" + pin + "'")
>
> def print_name(self):
> print (self.line.name())
>
> def set(self):
> self.line.set_value(1)
>
> def clear(self):
> self.line.set_value(0)
>
>
> This is by no means the final code, the print() in the __init__() is
> just a diagnostic for example. However I really can't understand why I
> see the following when I try it:-
>
> >>> import ngp
> >>> ngp.Gpiopin("P9_23")
> Found:  P9_23
> Traceback (most recent call last):
>   File "", line 1, in 
>   File "/home/chris/.cfg/hosts/bbb/bin/ngp.py", line 24, in __init__
> return
> ValueError: Can't find pin 'P9_23'
> >>>
>
> Does a return in __init__() not do what I think it does?
>
> How else could/should I do this?

Change the return to a break
-- 
https://mail.python.org/mailman/listinfo/python-list

Why do I always get an exception raised in this __init__()?

[Chris Green via Python-list]

I'm obviously doing something very silly here but at the moment I
can't see what.

Here's the code:-

#!/usr/bin/python3 
# 
# 
# GPIO  
# 
import gpiod
# 
# 
# Simple wrapper class for gpiod to make set and clearing outputs
easier 
# 
class Gpiopin:

def __init__(self, pin):
# 
#  
# scan through the GPIO chips to find the line/pin we want 
# 
for c in ['gpiochip0', 'gpiochip1', 'gpiochip2', 'gpiochip3']:
 
chip = gpiod.Chip(c)
for l in range(32):
line = chip.get_line(l)
if pin in line.name():
print("Found: ", line.name())
return
else:
raise ValueError("Can't find pin '" + pin + "'")
 
def print_name(self): 
print (self.line.name()) 
 
def set(self): 
self.line.set_value(1) 
 
def clear(self): 
self.line.set_value(0) 


This is by no means the final code, the print() in the __init__() is
just a diagnostic for example. However I really can't understand why I
see the following when I try it:-

>>> import ngp
>>> ngp.Gpiopin("P9_23")
Found:  P9_23
Traceback (most recent call last):
  File "", line 1, in 
  File "/home/chris/.cfg/hosts/bbb/bin/ngp.py", line 24, in __init__
return
ValueError: Can't find pin 'P9_23'
>>> 

Does a return in __init__() not do what I think it does?

How else could/should I do this?





-- 
Chris Green
·
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Why do I always get an exception raised in this __init__()?

[Chris Green via Python-list]

Chris Green  wrote:

[snip code and question]

Sorry folks, it was a caching problem, I wasn't running the code I
thought I was running!  When I made sure I had cleared everything out
and tried again it all worked as I expected.

-- 
Chris Green
·
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Why does IDLE use a subprocess?

[James Schaffler via Python-list]

On Tuesday, May 30th, 2023 at 10:18 PM, Chris Angelico wrote:
> Yep, what you're seeing there is the namespace and nothing else. But
> if you mess with an actual builtin object, it'll be changed for the
> other interpreter too.
> 
> > > > import ctypes
> > > > ctypes.cast(id(42), ctypes.POINTER(ctypes.c_int))[6] = 43
> > > > 41+1
> 
> 43
> 
> > > > from code import InteractiveInterpreter
> > > > interp = InteractiveInterpreter()
> > > > interp.runcode("print(41+1)")
> 
> 43
> 
> (Note that this only works in CPython and only with integers small
> enough to be in the cache, meaning that there is only one such object
> representing that integer.)
> 
> The same is true of C extensions, which often have their own internal
> state, and that state isn't isolated to a single interpreter.
> 
> Better isolation is coming with PEP 554
> https://peps.python.org/pep-0554/ which also has some great
> information about what currently is NOT isolated. (Also, even then,
> some things won't be fully isolated; I think that the ctypes trick
> above might still affect a subinterpreter even in a post-PEP554
> world.)

Amazing example! Thank you everyone for the detailed responses - will be sure 
to check out the PEP as well.

Jim
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Why does IDLE use a subprocess?

[Chris Angelico]

On Wed, 31 May 2023 at 12:03, James Schaffler via Python-list
 wrote:
>
> On Tuesday, May 30th, 2023 at 9:14 PM, Greg Ewing wrote:
> > Globals you create by executing code in the REPL have their own
> > namespace. But everything else is shared -- builtins, imported
> > Python modules, imported C extension modules, etc. etc.
>
> Thanks for the explanation. Could you elaborate on precisely how "everything 
> else is shared"? As far as I understand, if you run the following code:
>
> from code import InteractiveInterpreter
> interp = InteractiveInterpreter()
> import numpy as np
> interp.runcode("np.__name__")
>
> this will result in the error
> Traceback (most recent call last):
>   File "", line 1, in 
> NameError: name 'np' is not defined
>
> which seems to imply that imports in the parent shell are not shared with 
> interpreters and vice versa (if you swap the places of the import and the 
> __name__ call).
>

Yep, what you're seeing there is the namespace and nothing else. But
if you mess with an actual builtin object, it'll be changed for the
other interpreter too.

>>> import ctypes
>>> ctypes.cast(id(42), ctypes.POINTER(ctypes.c_int))[6] = 43
>>> 41+1
43
>>> from code import InteractiveInterpreter
>>> interp = InteractiveInterpreter()
>>> interp.runcode("print(41+1)")
43

(Note that this only works in CPython and only with integers small
enough to be in the cache, meaning that there is only one such object
representing that integer.)

The same is true of C extensions, which often have their own internal
state, and that state isn't isolated to a single interpreter.

Better isolation is coming with PEP 554
https://peps.python.org/pep-0554/ which also has some great
information about what currently is NOT isolated. (Also, even then,
some things won't be fully isolated; I think that the ctypes trick
above might still affect a subinterpreter even in a post-PEP554
world.)

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Why does IDLE use a subprocess?

[James Schaffler via Python-list]

On Tuesday, May 30th, 2023 at 9:14 PM, Greg Ewing wrote:
> Globals you create by executing code in the REPL have their own
> namespace. But everything else is shared -- builtins, imported
> Python modules, imported C extension modules, etc. etc.

Thanks for the explanation. Could you elaborate on precisely how "everything 
else is shared"? As far as I understand, if you run the following code:

from code import InteractiveInterpreter
interp = InteractiveInterpreter()
import numpy as np
interp.runcode("np.__name__")

this will result in the error
Traceback (most recent call last):
  File "", line 1, in 
NameError: name 'np' is not defined

which seems to imply that imports in the parent shell are not shared with 
interpreters and vice versa (if you swap the places of the import and the 
__name__ call).

Thanks,
Jim
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Why does IDLE use a subprocess?

[Greg Ewing via Python-list]

On 29/05/23 8:10 am, James Schaffler wrote:

However, some minimal testing of InteractiveInterpreter leads me to believe 
that the Interpreter object has its own view of local/global variables and 
therefore shouldn't be able to affect the calling interpreter


Globals you create by executing code in the REPL have their own
namespace. But everything else is shared -- builtins, imported
Python modules, imported C extension modules, etc. etc.

There's a long-running project to make it possible to have
multiple fully-isolated Python interpreters in one process, but
the way CPython is structured makes that very difficult to achieve,
and as far as I know it's not there yet.

In the case of IDLE, there's really no reason not to use a
subprocess[1]. It's easy and guarantees 100% isolation.

[1] Well, mostly. There used to be a small hitch on Windows with
the default firewall settings not letting you connect to a local
socket (nice one, Microsoft). I don't know whether that's still
an issue.

--
Greg
--
https://mail.python.org/mailman/listinfo/python-list

Re: Why does IDLE use a subprocess?

[Chris Angelico]

On Wed, 31 May 2023 at 08:16, Barry  wrote:
> I don’t think it security but robustness that needs the subprocess.
>
> Also if your code use tk then it would conflict with idle’s use of tk.
>

From my memory, it's precisely this - it's much MUCH easier to allow
you to use Tk in your own program without conflicting with Idle's own
use of it. I'm sure it would be possible to make everything work in
one process, but the cost of doing so would be greater than the costs
of juggling two processes.

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Why does IDLE use a subprocess?

[Barry]

> On 30 May 2023, at 21:10, James Schaffler via Python-list 
>  wrote:
> 
> Originally posted to idle-dev, but thought this might be a better place. Let 
> me know if it isn't.
> 
> Hi,
> 
> I was curious about the internals of IDLE, and noticed that IDLE uses 
> executes user code in a "subprocess" that's separate from the Python 
> interpreter that is running IDLE itself (which does tasks such as making the 
> window and coloring the text).
> 
> As far as I understand, IDLE runs a modified version of 
> code.InteractiveInterpreter by sending user code through a socket. Even the 
> IDLE documentation says that without a subprocess, "user code is not isolated 
> from IDLE itself." However, some minimal testing of InteractiveInterpreter 
> leads me to believe that the Interpreter object has its own view of 
> local/global variables and therefore shouldn't be able to affect the calling 
> interpreter (please correct me if I'm wrong).
> 
> So my question is a combination of "Why does IDLE use a subprocess?" and "Why 
> is InteractiveInterpreter not secureuldenough?" What possible security 
> vulnerabilities exist if one uses IDLE without the subprocess? If anyone 
> knows (or could point me to information on) why IDLE is designed this way, 
> I'd really appreciate it. Thank you!

I don’t think it security but robustness that needs the subprocess.

You can crash idle with bugs in the code that you are developing.
By running your code in a subprocess idle protects itself, and your edits from 
bugs in your code.

Also if your code use tk then it would conflict with idle’s use of tk.

That is my assumption on why the subprocess is required.

Barry

> 
> Jim
> -- 
> https://mail.python.org/mailman/listinfo/python-list
> 

-- 
https://mail.python.org/mailman/listinfo/python-list

Why does IDLE use a subprocess?

[James Schaffler via Python-list]

Originally posted to idle-dev, but thought this might be a better place. Let me 
know if it isn't.

Hi,

I was curious about the internals of IDLE, and noticed that IDLE uses executes 
user code in a "subprocess" that's separate from the Python interpreter that is 
running IDLE itself (which does tasks such as making the window and coloring 
the text).

As far as I understand, IDLE runs a modified version of 
code.InteractiveInterpreter by sending user code through a socket. Even the 
IDLE documentation says that without a subprocess, "user code is not isolated 
from IDLE itself." However, some minimal testing of InteractiveInterpreter 
leads me to believe that the Interpreter object has its own view of 
local/global variables and therefore shouldn't be able to affect the calling 
interpreter (please correct me if I'm wrong).

So my question is a combination of "Why does IDLE use a subprocess?" and "Why 
is InteractiveInterpreter not secure enough?" What possible security 
vulnerabilities exist if one uses IDLE without the subprocess? If anyone knows 
(or could point me to information on) why IDLE is designed this way, I'd really 
appreciate it. Thank you!

Jim
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Why doesn't Python (error msg) tell me WHAT the actual (arg) values are ?

[Weatherby,Gerard]

“So the case where the assumption fails may not be easily
reproducable and the more information you can get post-mortem the
better”

That’s true for rare corner cases or esoteric race conditions. Usually, when I 
see asserts it's just because I was just plain stupid.

From: Python-list  on 
behalf of Peter J. Holzer 
Date: Saturday, February 25, 2023 at 5:21 PM
To: python-list@python.org 
Subject: Re: Why doesn't Python (error msg) tell me WHAT the actual (arg) 
values are ?
On 2023-02-25 21:58:18 +, Weatherby,Gerard wrote:
> I only use asserts for things I know to be true.

Yeah, that's what assers are for. Or rather for things that you *think*
are true.

> In other words, a failing assert means I have a hole in my program
> logic.

Yes, if you include your assumptions in your definition of "logic".


> For that use, the default behavior –telling me which line the assert
> is on, is more than sufficient. Depending on the circumstance, I’ll
> re-run the code with a breakpoint or replace the assert with an
> informative f-string Exception.

That may not always be practical. Things that we know (or think) are
true often have *are* true in most cases (otherwise we wouldn't think
so). So the case where the assumption fails may not be easily
reproducable and the more information you can get post-mortem the
better. For example, in C on Linux a failed assertion causes a core
dump. So you can inspect the complete state of the program.

hp

--
   _  | Peter J. Holzer| Story must make more sense than reality.
|_|_) ||
| |   | h...@hjp.at |-- Charles Stross, "Creative writing
__/   | http://www.hjp.at/ |   challenge!"
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Why doesn't Python (error msg) tell me WHAT the actual (arg) values are ?

[Peter J. Holzer]

On 2023-02-25 21:58:18 +, Weatherby,Gerard wrote:
> I only use asserts for things I know to be true.

Yeah, that's what assers are for. Or rather for things that you *think*
are true.

> In other words, a failing assert means I have a hole in my program
> logic.

Yes, if you include your assumptions in your definition of "logic".


> For that use, the default behavior –telling me which line the assert
> is on, is more than sufficient. Depending on the circumstance, I’ll
> re-run the code with a breakpoint or replace the assert with an
> informative f-string Exception.

That may not always be practical. Things that we know (or think) are
true often have *are* true in most cases (otherwise we wouldn't think
so). So the case where the assumption fails may not be easily
reproducable and the more information you can get post-mortem the
better. For example, in C on Linux a failed assertion causes a core
dump. So you can inspect the complete state of the program.

hp

-- 
   _  | Peter J. Holzer| Story must make more sense than reality.
|_|_) ||
| |   | h...@hjp.at |-- Charles Stross, "Creative writing
__/   | http://www.hjp.at/ |   challenge!"


signature.asc
Description: PGP signature
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Why doesn't Python (error msg) tell me WHAT the actual (arg) values are ?

[Weatherby,Gerard]

I only use asserts for things I know to be true. Nothing is harder to debug 
than when something you know to be true turns out to be… not True. Because I’ll 
check everything else instead of the cause of the bug.

In other words, a failing assert means I have a hole in my program logic.

For that use, the default behavior –telling me which line the assert is on, is 
more than sufficient. Depending on the circumstance, I’ll re-run the code with 
a breakpoint or replace the assert with an informative f-string Exception.



From: Python-list  on 
behalf of Peter J. Holzer 
Date: Saturday, February 25, 2023 at 9:22 AM
To: python-list@python.org 
Subject: Re: Why doesn't Python (error msg) tell me WHAT the actual (arg) 
values are ?
On 2023-02-25 09:10:06 -0500, Thomas Passin wrote:
> On 2/25/2023 1:13 AM, Peter J. Holzer wrote:
> > On 2023-02-24 18:19:52 -0500, Thomas Passin wrote:
> > > Sometimes you can use a second parameter to assert if you know what kind 
> > > of
> > > error to expect:
[...]
> > > With type errors, assert may actually give you the information needed:
> > >
> > > > > > c = {"a": a, "b": 2}
> > > > > > assert a > c
> > > Traceback (most recent call last):
> > >File "", line 1, in 
> > > TypeError: '>' not supported between instances of 'list' and 'dict'
> >
> > Actually in this case it isn't assert which gives you the information,
> > it's evaluating the expression itself. You get the same error with just
> >  a > c
> > on a line by its own.
>
> In some cases.  For my example with an explanatory string, you wouldn't want
> to write code like that after an ordinary line of code, at least not very
> often.  The assert statement allows it syntactically.

Yes, but if an error in the expression triggers an exception (as in this
case) the explanatory string will never be displayed.

hp

--
   _  | Peter J. Holzer| Story must make more sense than reality.
|_|_) ||
| |   | h...@hjp.at |-- Charles Stross, "Creative writing
__/   | http://www.hjp.at/ |   challenge!"
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Why doesn't Python (error msg) tell me WHAT the actual (arg) values are ?

[Peter J. Holzer]

On 2023-02-25 09:10:06 -0500, Thomas Passin wrote:
> On 2/25/2023 1:13 AM, Peter J. Holzer wrote:
> > On 2023-02-24 18:19:52 -0500, Thomas Passin wrote:
> > > Sometimes you can use a second parameter to assert if you know what kind 
> > > of
> > > error to expect:
[...]
> > > With type errors, assert may actually give you the information needed:
> > > 
> > > > > > c = {"a": a, "b": 2}
> > > > > > assert a > c
> > > Traceback (most recent call last):
> > >File "", line 1, in 
> > > TypeError: '>' not supported between instances of 'list' and 'dict'
> > 
> > Actually in this case it isn't assert which gives you the information,
> > it's evaluating the expression itself. You get the same error with just
> >  a > c
> > on a line by its own.
> 
> In some cases.  For my example with an explanatory string, you wouldn't want
> to write code like that after an ordinary line of code, at least not very
> often.  The assert statement allows it syntactically.

Yes, but if an error in the expression triggers an exception (as in this
case) the explanatory string will never be displayed.

hp

-- 
   _  | Peter J. Holzer| Story must make more sense than reality.
|_|_) ||
| |   | h...@hjp.at |-- Charles Stross, "Creative writing
__/   | http://www.hjp.at/ |   challenge!"


signature.asc
Description: PGP signature
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Why doesn't Python (error msg) tell me WHAT the actual (arg) values are ?

[Thomas Passin]

On 2/25/2023 1:13 AM, Peter J. Holzer wrote:

On 2023-02-24 18:19:52 -0500, Thomas Passin wrote:

On 2/24/2023 2:47 PM, dn via Python-list wrote:

On 25/02/2023 08.12, Peter J. Holzer wrote:

On 2023-02-24 16:12:10 +1300, dn via Python-list wrote:

In some ways, providing this information seems appropriate.
Curiously, this does not even occur during an assert exception -
despite the value/relationship being the whole point of using
the command!

  x = 1
  assert x == 2

AssertionError (and that's it)


Sometimes you can use a second parameter to assert if you know what kind of
error to expect:


a = [1,2,3]
b = [4,5]
assert len(a) == len(b), f'len(a): {len(a)} != len(b): {len(b)}'

Traceback (most recent call last):
   File "", line 1, in 
AssertionError: len(a): 3 != len(b): 2


Yup. That's very useful (but I tend to forget that).



With type errors, assert may actually give you the information needed:


c = {"a": a, "b": 2}
assert a > c

Traceback (most recent call last):
   File "", line 1, in 
TypeError: '>' not supported between instances of 'list' and 'dict'


Actually in this case it isn't assert which gives you the information,
it's evaluating the expression itself. You get the same error with just
 a > c
on a line by its own.


In some cases.  For my example with an explanatory string, you wouldn't 
want to write code like that after an ordinary line of code, at least 
not very often.  The assert statement allows it syntactically.


--
https://mail.python.org/mailman/listinfo/python-list

Re: Why doesn't Python (error msg) tell me WHAT the actual (arg) values are ?

[Peter J. Holzer]

On 2023-02-24 18:19:52 -0500, Thomas Passin wrote:
> On 2/24/2023 2:47 PM, dn via Python-list wrote:
> > On 25/02/2023 08.12, Peter J. Holzer wrote:
> > > On 2023-02-24 16:12:10 +1300, dn via Python-list wrote:
> > > > In some ways, providing this information seems appropriate.
> > > > Curiously, this does not even occur during an assert exception -
> > > > despite the value/relationship being the whole point of using
> > > > the command!
> > > > 
> > > >  x = 1
> > > >  assert x == 2
> > > > 
> > > > AssertionError (and that's it)
> 
> Sometimes you can use a second parameter to assert if you know what kind of
> error to expect:
> 
> >>> a = [1,2,3]
> >>> b = [4,5]
> >>> assert len(a) == len(b), f'len(a): {len(a)} != len(b): {len(b)}'
> Traceback (most recent call last):
>   File "", line 1, in 
> AssertionError: len(a): 3 != len(b): 2

Yup. That's very useful (but I tend to forget that).


> With type errors, assert may actually give you the information needed:
> 
> >>> c = {"a": a, "b": 2}
> >>> assert a > c
> Traceback (most recent call last):
>   File "", line 1, in 
> TypeError: '>' not supported between instances of 'list' and 'dict'

Actually in this case it isn't assert which gives you the information,
it's evaluating the expression itself. You get the same error with just
a > c
on a line by its own.

hp

-- 
   _  | Peter J. Holzer| Story must make more sense than reality.
|_|_) ||
| |   | h...@hjp.at |-- Charles Stross, "Creative writing
__/   | http://www.hjp.at/ |   challenge!"


signature.asc
Description: PGP signature
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Why doesn't Python (error msg) tell me WHAT the actual (arg) values are ?

[Thomas Passin]

On 2/24/2023 2:47 PM, dn via Python-list wrote:

On 25/02/2023 08.12, Peter J. Holzer wrote:

On 2023-02-24 16:12:10 +1300, dn via Python-list wrote:
In some ways, providing this information seems appropriate. 
Curiously, this

does not even occur during an assert exception - despite the
value/relationship being the whole point of using the command!

 x = 1
 assert x == 2

AssertionError (and that's it)


Sometimes you can use a second parameter to assert if you know what kind 
of error to expect:


>>> a = [1,2,3]
>>> b = [4,5]
>>> assert len(a) == len(b), f'len(a): {len(a)} != len(b): {len(b)}'
Traceback (most recent call last):
  File "", line 1, in 
AssertionError: len(a): 3 != len(b): 2

With type errors, assert may actually give you the information needed:

>>> c = {"a": a, "b": 2}
>>> assert a > c
Traceback (most recent call last):
  File "", line 1, in 
TypeError: '>' not supported between instances of 'list' and 'dict'

So now we know that a is a list and c is a dictionary.


Pytest is great there. If an assertion in a test case fails it analyzes
the expression to give you various levels of details:

 test session starts 


platform linux -- Python 3.10.6, pytest-6.2.5, py-1.10.0, pluggy-0.13.0
rootdir: /home/hjp/tmp/t
plugins: cov-3.0.0, anyio-3.6.1
collected 1 item

test_a.py 
F   [100%]


= FAILURES 
==
__ test_a 
___


 def test_a():
 a = [1, 2, 3]
 b = {"a": a, "b": 2}


   assert len(a) == len(b)

E   AssertionError: assert 3 == 2
E    +  where 3 = len([1, 2, 3])
E    +  and   2 = len({'a': [1, 2, 3], 'b': 2})

test_a.py:7: AssertionError
== short test summary info 
==

FAILED test_a.py::test_a - AssertionError: assert 3 == 2
= 1 failed in 0.09s 
=


+1
and hence the tone of slight surprise in the observation - because only 
ever use assert within pytests, and as observed, pytest amplifies the 
report-back to provide actionable-intelligence. See also: earlier 
contribution about using a debugger.



That said, have observed coders 'graduating' from other languages, 
making wider use of assert - assumed to be more data (value) 
sanity-checks than typing, but ...


Do you use assert frequently?


The OP seems wedded to his?her ways, complaining that Python does not 
work the way it 'should'. In turn, gives rise to the impression that 
expounding the advantages of TDD, and thus anticipating such unit and 
integration error-possibilities, might be considered an insult or 
unhelpful.

(sigh!)

Personally, I struggled a bit to adapt from the more-strictured (if not 
more-structured) languages of my past, to Python - particularly the 
different philosophies or emphases of what happens at 'compile-time' cf 
'execution-time'; and how such required marked changes in attitudes to 
design, time-allocation, work-flow, and tool-set. Two related-activities 
which made the language-change more workable and unleashed greater than 
step-change advantage, were: increased use of TDD, and actively learning 
the facilities within Python-oriented IDEs.




--
https://mail.python.org/mailman/listinfo/python-list

Re: Why doesn't Python (error msg) tell me WHAT the actual (arg) values are ?

[Peter J. Holzer]

On 2023-02-25 08:47:00 +1300, dn via Python-list wrote:
> That said, have observed coders 'graduating' from other languages, making
> wider use of assert - assumed to be more data (value) sanity-checks than
> typing, but ...
> 
> Do you use assert frequently?

Not very often, but I do use it. Sometimes for its intended purpose
(i.e. to guard against bugs or wrong assumptions), sometimes just to
guard incomplete or sloppy code (e.g. browsing through some projects I
find
assert len(data["structure"]["dimensions"]["observation"]) == 1
(incomplete code - I didn't bother to implement multiple observations)
and
assert(header[0] == "Monat (MM)")
(the code below is sloppy. Instead of fixing it I just made the original
programmer's assumptions explicit)
and of course
assert False
(this point should never be reached)).

hp

-- 
   _  | Peter J. Holzer| Story must make more sense than reality.
|_|_) ||
| |   | h...@hjp.at |-- Charles Stross, "Creative writing
__/   | http://www.hjp.at/ |   challenge!"


signature.asc
Description: PGP signature
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Why doesn't Python (error msg) tell me WHAT the actual (arg) values are ?

[dn via Python-list]

On 25/02/2023 08.12, Peter J. Holzer wrote:

On 2023-02-24 16:12:10 +1300, dn via Python-list wrote:

In some ways, providing this information seems appropriate. Curiously, this
does not even occur during an assert exception - despite the
value/relationship being the whole point of using the command!

 x = 1
 assert x == 2

AssertionError (and that's it)


Pytest is great there. If an assertion in a test case fails it analyzes
the expression to give you various levels of details:

 test session starts 

platform linux -- Python 3.10.6, pytest-6.2.5, py-1.10.0, pluggy-0.13.0
rootdir: /home/hjp/tmp/t
plugins: cov-3.0.0, anyio-3.6.1
collected 1 item

test_a.py F 
  [100%]

= FAILURES 
==
__ test_a 
___

 def test_a():
 a = [1, 2, 3]
 b = {"a": a, "b": 2}


   assert len(a) == len(b)

E   AssertionError: assert 3 == 2
E+  where 3 = len([1, 2, 3])
E+  and   2 = len({'a': [1, 2, 3], 'b': 2})

test_a.py:7: AssertionError
== short test summary info 
==
FAILED test_a.py::test_a - AssertionError: assert 3 == 2
= 1 failed in 0.09s 
=


+1
and hence the tone of slight surprise in the observation - because only 
ever use assert within pytests, and as observed, pytest amplifies the 
report-back to provide actionable-intelligence. See also: earlier 
contribution about using a debugger.



That said, have observed coders 'graduating' from other languages, 
making wider use of assert - assumed to be more data (value) 
sanity-checks than typing, but ...


Do you use assert frequently?


The OP seems wedded to his?her ways, complaining that Python does not 
work the way it 'should'. In turn, gives rise to the impression that 
expounding the advantages of TDD, and thus anticipating such unit and 
integration error-possibilities, might be considered an insult or unhelpful.

(sigh!)

Personally, I struggled a bit to adapt from the more-strictured (if not 
more-structured) languages of my past, to Python - particularly the 
different philosophies or emphases of what happens at 'compile-time' cf 
'execution-time'; and how such required marked changes in attitudes to 
design, time-allocation, work-flow, and tool-set. Two related-activities 
which made the language-change more workable and unleashed greater than 
step-change advantage, were: increased use of TDD, and actively learning 
the facilities within Python-oriented IDEs.


--
Regards,
=dn
--
https://mail.python.org/mailman/listinfo/python-list

Re: Why doesn't Python (error msg) tell me WHAT the actual (arg) values are ?

[Peter J. Holzer]

On 2023-02-24 16:12:10 +1300, dn via Python-list wrote:
> In some ways, providing this information seems appropriate. Curiously, this
> does not even occur during an assert exception - despite the
> value/relationship being the whole point of using the command!
> 
> x = 1
> assert x == 2
> 
> AssertionError (and that's it)

Pytest is great there. If an assertion in a test case fails it analyzes
the expression to give you various levels of details:

 test session starts 

platform linux -- Python 3.10.6, pytest-6.2.5, py-1.10.0, pluggy-0.13.0
rootdir: /home/hjp/tmp/t
plugins: cov-3.0.0, anyio-3.6.1
collected 1 item

test_a.py F 
  [100%]

= FAILURES 
==
__ test_a 
___

def test_a():
a = [1, 2, 3]
b = {"a": a, "b": 2}

>   assert len(a) == len(b)
E   AssertionError: assert 3 == 2
E+  where 3 = len([1, 2, 3])
E+  and   2 = len({'a': [1, 2, 3], 'b': 2})

test_a.py:7: AssertionError
== short test summary info 
==
FAILED test_a.py::test_a - AssertionError: assert 3 == 2
= 1 failed in 0.09s 
=

hp

-- 
   _  | Peter J. Holzer| Story must make more sense than reality.
|_|_) ||
| |   | h...@hjp.at |-- Charles Stross, "Creative writing
__/   | http://www.hjp.at/ |   challenge!"


signature.asc
Description: PGP signature
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Why doesn't Python (error msg) tell me WHAT the actual (arg) values are ?

[Peter J. Holzer]

On 2023-02-23 20:32:26 -0700, Michael Torrie wrote:
> On 2/23/23 01:08, Hen Hanna wrote:
> >  Python VM  is seeing an "int" object (123)   (and telling me that)
> >  ...   so it should be easy to print that "int" object What does
> >  Python VMknow ?   and when does it know it ?
> It knows there is an object and its name and type.  It knows this from
> the first moment you create the object and bind a name to it.
> > it seems like  it is being playful, teasing (or mean),and
> > hiding  the ball from me
> 
> Sorry you aren't understanding.  Whenever you print() out an object,
> python calls the object's __repr__() method to generate the string to
> display.  For built-in objects this is obviously trivial. But if you
> were dealing an object of some arbitrary class, there may not be a
> __repr__() method

Is this even possible? object has a __repr__ method, so all other
classes would inherit that if they don't define one themselves. I guess
it's possible to explicitely remove it ...

> which would cause an exception, or if the __repr__()
> method itself raised an exception,

Yup. That is possible and has happened to me several times - of course
always in a situation where I really needed that output ...

hp

-- 
   _  | Peter J. Holzer| Story must make more sense than reality.
|_|_) ||
| |   | h...@hjp.at |-- Charles Stross, "Creative writing
__/   | http://www.hjp.at/ |   challenge!"


signature.asc
Description: PGP signature
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Why doesn't Python (error msg) tell me WHAT the actual (arg) values are ?

[Peter J. Holzer]

On 2023-02-22 15:46:09 -0800, Hen Hanna wrote:
> On Wednesday, February 22, 2023 at 12:05:34 PM UTC-8, Hen Hanna wrote:
> > > py bug.py 
> > Traceback (most recent call last): 
> > File "C:\Usenet\bug.py", line 5, in  
> > print( a + 12 ) 
> > TypeError: can only concatenate str (not "int") to str 
> > 
> > 
> > Why doesn't Python (error msg) do the obvious thing and tell me 
> > WHAT the actual (offending, arg) values are ? 
> > 
> > In many cases, it'd help to know what string the var A had , when the error 
> > occurred. 
> >  i wouldn't have to put print(a) just above, to see. 
> > 
> > ( pypy doesn't do that either, but Python makes programming
> > (debugging) so easy that i hardly feel any inconvenience.)

That seems like a non-sequitur to me. If you hardly feel any
inconvenience, why argue so forcefully?
And why is pypy relevant here?


> i  see that my example   would be clearER  with this one-line  change:
> 
> 
>   >  py   bug.py
> 
>Traceback (most recent call last):
> 
>   File "C:\Usenet\bug.py", line 5, in 
>  map( Func,fooBar(  X,  Y,  X +  
> Y  ))
>  
>TypeError: can only concatenate str (not "int") to str
> 
> 
> i hope that   NOW   a few of you  can  see this as a genuine,  (reasonable)  
> question.

That doesn't seem a better example to me. There is still only one
subexpression (X + Y) where that error can come from, so I know that X
is a str and Y is an int.

A better example would be something like 

x = (a + b) * (c + d)

In this case it could be either (a + b) or (c + d) which caused the
error. But what I really want to know here is the names of the involved
variables, NOT the values. If the error message told me that the values
were 'foo' and 12.3, I still wouldn't be any wiser. The problem here of
course is that the operands aren't necessarily simple variables as in
this example - they may be arbitrarily complex expressions. However, it
might be sufficient to mark the operator which caused the exception:

|   ...
|   File "/home/hjp/tmp/./foo", line 4, in f
| return (a + b) * (c + d)
| ^
| TypeError: can only concatenate str (not "int") to str

would tell me that (c + d) caused the problem and therefore that c must
be a str which it obviously shouldn't be.

hp

-- 
   _  | Peter J. Holzer| Story must make more sense than reality.
|_|_) ||
| |   | h...@hjp.at |-- Charles Stross, "Creative writing
__/   | http://www.hjp.at/ |   challenge!"


signature.asc
Description: PGP signature
-- 
https://mail.python.org/mailman/listinfo/python-list

RE: Why doesn't Python (error msg) tell me WHAT the actual (arg) values are ?

[avi.e.gross]

We have been supplying many possible reasons or consequences for why the
implementation of python does not do what the OP wants and even DEMANDS.

I am satisfied with knowing it was because they CHOSE NOT TO in some places
and maybe not in others. It is nice to see some possible reasons, but
something as simple as efficiency or needing to complicate the code in
something used regularly, might be enough for now.

But to comment on what Michael T. and Dave N. have been saying, newcomers
often have no clue of what can happen so their questions may sound quite
reasonable.

So what happens if you create a large data structure, so some operation that
fails, catch the error and save the variables involved in an exception and
throw that onward and perhaps the program keeps running? There is now a
pointer to the large data structure in the exception object, or even a copy.
If that exception is not discarded or garbage collected, it can remain in
memory indefinitely even if the original string was expected to be removed,
replaced, or garbage collected. Some modern features in R such as generators
will stay alive infinitely and retain their state in between calls for a
next item.

You can end up with memory leaks that are not trivial to solve or that may
mysteriously disappear when an iterable has finally been consumed and all
the storage it used or pointed at can be retrieved, as one example.

A more rational approach is to realize that python has multiple levels of
debugging and exceptions are one among many. They are not meant to solve the
entire problem but just enough to be helpful or point you in some direction.
Yes, they can do more.

And, FYI, I too pointed this person at the Tutor list and I see no sign they
care how many people they make waste their time with so many mainly gripes.
I personally now ignore any post by them.
-Original Message-
From: Python-list  On
Behalf Of Michael Torrie
Sent: Thursday, February 23, 2023 10:32 PM
To: python-list@python.org
Subject: Re: Why doesn't Python (error msg) tell me WHAT the actual (arg)
values are ?

On 2/23/23 01:08, Hen Hanna wrote:
>  Python VM  is seeing an "int" object (123)   (and telling me that)   ...
so it should be easy to print that "int" object 
> What does  Python VMknow ?   and when does it know it ?
It knows there is an object and its name and type.  It knows this from the
first moment you create the object and bind a name to it.
> it seems like  it is being playful, teasing (or mean),and   hiding
the ball from me

Sorry you aren't understanding.  Whenever you print() out an object, python
calls the object's __repr__() method to generate the string to display.  For
built-in objects this is obviously trivial. But if you were dealing an
object of some arbitrary class, there may not be a
__repr__() method which would cause an exception, or if the __repr__()
method itself raised an exception, you'd lose the original error message and
the stack trace would be all messed up and of no value to you.  Does that
make sense?  Remember that Python is a very dynamic language and what might
be common sense for a built-in type makes no sense at all for a custom type.
Thus there's no consistent way for Python to print out the information you
think is so simple.
--
https://mail.python.org/mailman/listinfo/python-list

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Why doesn't Python (error msg) tell me WHAT the actual (arg) values are ?

[Michael Torrie]

On 2/23/23 01:08, Hen Hanna wrote:
>  Python VM  is seeing an "int" object (123)   (and telling me that)   ...   
> so it should be easy to print that "int" object 
> What does  Python VMknow ?   and when does it know it ?
It knows there is an object and its name and type.  It knows this from
the first moment you create the object and bind a name to it.
> it seems like  it is being playful, teasing (or mean),and   hiding  the 
> ball from me

Sorry you aren't understanding.  Whenever you print() out an object,
python calls the object's __repr__() method to generate the string to
display.  For built-in objects this is obviously trivial. But if you
were dealing an object of some arbitrary class, there may not be a
__repr__() method which would cause an exception, or if the __repr__()
method itself raised an exception, you'd lose the original error message
and the stack trace would be all messed up and of no value to you.  Does
that make sense?  Remember that Python is a very dynamic language and
what might be common sense for a built-in type makes no sense at all for
a custom type.  Thus there's no consistent way for Python to print out
the information you think is so simple.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Why doesn't Python (error msg) tell me WHAT the actual (arg) values are ?

[dn via Python-list]

On 23/02/2023 09.05, Hen Hanna wrote:


   >  py   bug.py
Traceback (most recent call last):
  File "C:\Usenet\bug.py", line 5, in 
  print( a + 12 )
   TypeError: can only concatenate str (not "int") to str


Why doesn't  Python (error msg) do the obvious thing and tell me
 WHAT   the actual   (offending,  arg)  values are ?

In many cases, it'd help to know what string the var  A  had  ,   when the 
error occurred.
    i wouldn't have to put  print(a) just 
above,  to see.


In some ways, providing this information seems appropriate. Curiously, 
this does not even occur during an assert exception - despite the 
value/relationship being the whole point of using the command!


x = 1
assert x == 2

AssertionError (and that's it)

Then again, remember that exceptions can be 'caught'. So, such data 
would need to be added to the exception-instance. This could become 
quite costly.




What are the appropriate tools for the job?


Don't add an extra print(), use a debugger.

Not only does this allow you to breakpoint critical points in the code, 
but identifiers can be watch-ed and changes noted. The other handy 
feature is being able to correct the current erroneous value of the 
identifier and continue execution.


For us, memory-challenged coders, there is no need to remember to remove 
the print() again, afterwards.



The TypeError indicates a problem between the programmer's ears. What 
was thought to be a string or an integer was the opposite. This seems to 
be playing fast-and-loose with Python's dynamic-typing. To quote: "we're 
all adults here". Thus, I wouldn't recommend 're-cycling' an identifier 
to represent two different (types of) data-point in the same code - but 
there's nothing to stop you/anyone!


The other possibility is that it was an accident. Sounds more like 
something I would do, but... In this case, the tool which is your/my 
friend is typing. The IDE should catch most of the situations where an 
int would be used as an str, or v-v. Remember though, Python's typing is 
(a) not part of the language, and (b) probably won't help at run-time.



PS are you aware that there is a Python-Tutor list for the use of people 
learning Python?

--
Regards,
=dn
--
https://mail.python.org/mailman/listinfo/python-list

RE: Why doesn't Python (error msg) tell me WHAT the actual (arg) values are ?

[avi.e.gross]

or but a
temporary drawback. Only when int fails too is there a likely error but int
is not programmed to do anything lie reporting in __iadd__() and nor should
it except if you are debugging new functionality.

Can we get better error messages in many cases? Sure. Clearly the
interpreter level sees a call like func(x, y, x+y) and if evaluating it
causes an error, it could type out what all the variables were. 

But if you are running code at the console and it aborts like this, you
usually have the variables at your fingertips and can say print(x) or
print(y) but you cannot properly say print(x+y) yet. The error message did
specify you were trying to work with incompatible objects as one was of type
this and the other of type that. That suggests the values of both could have
been accessed and shown in this case ad perhaps the code could be modified
so it returns some semblance of the values at least for built-in objects.
All that though adds code and complexity and often slows things down. 



-Original Message-
From: Python-list  On
Behalf Of Rob Cliffe via Python-list
Sent: Wednesday, February 22, 2023 4:31 PM
To: python-list@python.org
Subject: Re: Why doesn't Python (error msg) tell me WHAT the actual (arg)
values are ?



On 22/02/2023 20:05, Hen Hanna wrote:
> Python makes programming (debugging) so easy
I agree with that!
Rob Cliffe
-- 
https://mail.python.org/mailman/listinfo/python-list

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Why doesn't Python (error msg) tell me WHAT the actual (arg) values are ?

[Rob Cliffe via Python-list]

On 22/02/2023 20:05, Hen Hanna wrote:

Python makes programming (debugging) so easy

I agree with that!
Rob Cliffe
--
https://mail.python.org/mailman/listinfo/python-list

Re: Why doesn't Python (error msg) tell me WHAT the actual (arg) values are ?

[Hen Hanna]

On Wednesday, February 22, 2023 at 11:57:45 PM UTC-8, Barry wrote:
> > On 23 Feb 2023, at 01:39, Hen Hanna  wrote:
> > 
> > On Wednesday, February 22, 2023 at 3:46:21 PM UTC-8, Hen Hanna wrote: 
> >> On Wednesday, February 22, 2023 at 12:05:34 PM UTC-8, Hen Hanna wrote: 
> >>>> py bug.py 
> >>> Traceback (most recent call last): 
> >>> File "C:\Usenet\bug.py", line 5, in  
> >>> print( a + 12 ) 
> >>> TypeError: can only concatenate str (not "int") to str 
> >>> 
> >>> 
> >>> Why doesn't Python (error msg) do the obvious thing and tell me 
> >>> WHAT the actual (offending, arg) values are ? 
> >>> 
> >>> In many cases, it'd help to know what string the var A had , when the 
> >>> error occurred. 
> >>>  i wouldn't have to put print(a) just above, to see. 
> >>> 
> >>> 
> >>> 
> >>> 
> >>> ( pypy doesn't do that either, but Python makes programming (debugging) 
> >>> so easy that i hardly feel any inconvenience.) 
> > 
> > 
> > i see that my example would be (even) clearER with this one-line change: 
> > 
> > py bug.py 
> > 
> > Traceback (most recent call last): 
> > 
> > File "C:\Usenet\bug.py", line 5, in  
> > map( Func, fooBar( X, Y, X + Y )) 
> > 
> > TypeError: can only concatenate str (not "int") to str 
> >   attempt to call +  with   'abc',  123 
> >   <-- 
> > 
> >> i hope that NOW a few of you can see this as a genuine, (reasonable) 
> >> question. 
> > 
> > Python seems so perfectly User-friendly that 
> > i 'm so curious (puzzled) that it doesn't do the very obvious and easy 
> > thing 
> > of giving me this info: 
> > 
> > attempt to call +with   'abc',   
> > 123 <--

> It is not easy to do that in a robust and reliable way for any object. 
> You can end up in the code to generate the error message itself breaking. 
> For example using unbounded CPU time when attempting to get the string repr 
> of the variable. 
> 
> Barry 
> 


 Python VM  is seeing an "int" object (123)   (and telling me that)   ...   so 
it should be easy to print that "int" object 


What does  Python VMknow ?   and when does it know it ?


it seems like  it is being playful, teasing (or mean),and   hiding  the 
ball from me
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Why doesn't Python (error msg) tell me WHAT the actual (arg) values are ?

[Barry]

> On 23 Feb 2023, at 01:39, Hen Hanna  wrote:
> 
> On Wednesday, February 22, 2023 at 3:46:21 PM UTC-8, Hen Hanna wrote:
>> On Wednesday, February 22, 2023 at 12:05:34 PM UTC-8, Hen Hanna wrote: 
>>>> py bug.py 
>>> Traceback (most recent call last): 
>>> File "C:\Usenet\bug.py", line 5, in  
>>> print( a + 12 ) 
>>> TypeError: can only concatenate str (not "int") to str 
>>> 
>>> 
>>> Why doesn't Python (error msg) do the obvious thing and tell me 
>>> WHAT the actual (offending, arg) values are ? 
>>> 
>>> In many cases, it'd help to know what string the var A had , when the error 
>>> occurred. 
>>>  i wouldn't have to put print(a) just above, to see. 
>>> 
>>> 
>>> 
>>> 
>>> ( pypy doesn't do that either, but Python makes programming (debugging) so 
>>> easy that i hardly feel any inconvenience.)
> 
> 
> i see that my example would be (even)  clearER with this one-line change:
> 
>   py bug.py 
> 
> Traceback (most recent call last): 
> 
>   File "C:\Usenet\bug.py", line 5, in 
> map( Func, fooBar( X, Y, X + Y ))
> 
> TypeError: can only concatenate str (not "int") to str
>attempt to call +  with  'abc'  ,   123.45  
> <--
> 
>> i hope that NOW a few of you can see this as a genuine, (reasonable) 
>> question.
> 
> Python  seems so perfectly  User-friendly that
>   i 'm  so curious (puzzled)  that it doesn't do the very  
> obvious and easy thing 
> of giving me this info:
> 
>attempt to call  + with  'abc'  ,   
> 123.45  <--

It is not easy to do that in a robust and reliable way for any object.
You can end up in the code to generate the error message itself breaking.
For example using unbounded CPU time when attempting to get the string repr of 
the variable.

Barry

> -- 
> https://mail.python.org/mailman/listinfo/python-list
> 

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Why doesn't Python (error msg) tell me WHAT the actual (arg) values are ?

[Hen Hanna]

On Wednesday, February 22, 2023 at 3:46:21 PM UTC-8, Hen Hanna wrote:
> On Wednesday, February 22, 2023 at 12:05:34 PM UTC-8, Hen Hanna wrote: 
> > > py bug.py 
> > Traceback (most recent call last): 
> > File "C:\Usenet\bug.py", line 5, in  
> > print( a + 12 ) 
> > TypeError: can only concatenate str (not "int") to str 
> > 
> > 
> > Why doesn't Python (error msg) do the obvious thing and tell me 
> > WHAT the actual (offending, arg) values are ? 
> > 
> > In many cases, it'd help to know what string the var A had , when the error 
> > occurred. 
> >  i wouldn't have to put print(a) just above, to see. 
> > 
> > 
> > 
> > 
> > ( pypy doesn't do that either, but Python makes programming (debugging) so 
> > easy that i hardly feel any inconvenience.)


 i see that my example would be (even)  clearER with this one-line change:

   py bug.py 
   
 Traceback (most recent call last): 
  
   File "C:\Usenet\bug.py", line 5, in 
 map( Func, fooBar( X, Y, X + Y ))

 TypeError: can only concatenate str (not "int") to str
attempt to call +  with  'abc'  ,   123.45  
<--

> i hope that NOW a few of you can see this as a genuine, (reasonable) question.

Python  seems so perfectly  User-friendly that
   i 'm  so curious (puzzled)  that it doesn't do the very  
obvious and easy thing 
 of giving me this info:

attempt to call  + with  'abc'  ,   
123.45  <--
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Why doesn't Python (error msg) tell me WHAT the actual (arg) values are ?

[Thomas Passin]

On 2/22/2023 6:46 PM, Hen Hanna wrote:

On Wednesday, February 22, 2023 at 12:05:34 PM UTC-8, Hen Hanna wrote:

py bug.py

Traceback (most recent call last):
File "C:\Usenet\bug.py", line 5, in 
print( a + 12 )
TypeError: can only concatenate str (not "int") to str


Why doesn't Python (error msg) do the obvious thing and tell me
WHAT the actual (offending, arg) values are ?

In many cases, it'd help to know what string the var A had , when the error 
occurred.
 i wouldn't have to put print(a) just above, to see.




( pypy doesn't do that either, but Python makes programming (debugging) so easy 
that i hardly feel any inconvenience.)




i  see that my example   would be clearER  with this one-line  change:


   >  py   bug.py

Traceback (most recent call last):

   File "C:\Usenet\bug.py", line 5, in 
  map( Func,fooBar(  X,  Y,  X +  Y 
 ))
  
TypeError: can only concatenate str (not "int") to str



i hope that   NOW   a few of you  can  see this as a genuine,  (reasonable)  
question.


It tells me to go look at the function definition and how it's being 
invoked.  Even if I knew which of (X, Y) was an int and which a str, I'd 
still need to do that.


Or you could add type annotations to your code and run mypy on it...


--
https://mail.python.org/mailman/listinfo/python-list

Re: Why doesn't Python (error msg) tell me WHAT the actual (arg) values are ?

[Hen Hanna]

On Wednesday, February 22, 2023 at 12:05:34 PM UTC-8, Hen Hanna wrote:
> > py bug.py 
> Traceback (most recent call last): 
> File "C:\Usenet\bug.py", line 5, in  
> print( a + 12 ) 
> TypeError: can only concatenate str (not "int") to str 
> 
> 
> Why doesn't Python (error msg) do the obvious thing and tell me 
> WHAT the actual (offending, arg) values are ? 
> 
> In many cases, it'd help to know what string the var A had , when the error 
> occurred. 
>  i wouldn't have to put print(a) just above, to see. 
> 
> 
> 
> 
> ( pypy doesn't do that either, but Python makes programming (debugging) so 
> easy that i hardly feel any inconvenience.)



i  see that my example   would be clearER  with this one-line  change:


  >  py   bug.py

   Traceback (most recent call last):

  File "C:\Usenet\bug.py", line 5, in 
 map( Func,fooBar(  X,  Y,  X +  Y  
))
 
   TypeError: can only concatenate str (not "int") to str


i hope that   NOW   a few of you  can  see this as a genuine,  (reasonable)  
question.

-- 
https://mail.python.org/mailman/listinfo/python-list

RE: Why doesn't Python (error msg) tell me WHAT the actual (arg) values are ?

[avi.e.gross]

Hen or Hanna,

You keep asking WHY which may be reasonable but hard or irrelevant in many
cases.

I find the traceback perfectly informative.

It says you asked it to print NOT just "a" but "a + 12" and the error is
coming not from PRINT but from trying to invoke addition between two objects
that have not provided instructions on how to do so. Specifically, an object
of type str has not specified anything to do if asked to concatenate an
object of type int to it. And, an object of type int has not specified what
to do if asked to add itself to an object of type str to the left of it.
Deeper in python, the objects have dunder methods like __ADD__() and
___RADD__() to invoke for those situations that do some logic and decide
they cannot handle it and return an exception of sorts that ends up
generating your message.

If you want to know what "a" has at the moment, ask for just it, not adding
twelve to it. Perhaps you should add a line above your print asking to just
print(a).

Before you suggest what might be helpful, consider what it might mean in a
complex case with lots of variables and what work the interpreter might have
to do to dump the current values of anything relevant or just ANYTHING.

The way forward is less about asking why but asking what to do to get what
you want, or realize it is not attained the way you thought.

Avi

-Original Message-
From: Python-list  On
Behalf Of Hen Hanna
Sent: Wednesday, February 22, 2023 3:05 PM
To: python-list@python.org
Subject: Why doesn't Python (error msg) tell me WHAT the actual (arg) values
are ?


  >  py   bug.py
   Traceback (most recent call last):
 File "C:\Usenet\bug.py", line 5, in 
 print( a + 12 )
  TypeError: can only concatenate str (not "int") to str


Why doesn't  Python (error msg) do the obvious thing and tell me
WHAT   the actual   (offending,  arg)  values are ?

In many cases, it'd help to know what string the var  A  had  ,   when the
error occurred.
   i wouldn't have to put  print(a) just
above,  to see.




( pypydoesn't do that either,   but Python makes programming (debugging)
so easy that i hardly feel any inconvenience.)
-- 
https://mail.python.org/mailman/listinfo/python-list

-- 
https://mail.python.org/mailman/listinfo/python-list

Why doesn't Python (error msg) tell me WHAT the actual (arg) values are ?

[Hen Hanna]

  >  py   bug.py
   Traceback (most recent call last):
 File "C:\Usenet\bug.py", line 5, in 
 print( a + 12 )
  TypeError: can only concatenate str (not "int") to str


Why doesn't  Python (error msg) do the obvious thing and tell me
WHAT   the actual   (offending,  arg)  values are ?

In many cases, it'd help to know what string the var  A  had  ,   when the 
error occurred.
   i wouldn't have to put  print(a) just above, 
 to see.




( pypydoesn't do that either,   but Python makes programming (debugging) so 
easy that i hardly feel any inconvenience.)
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: why is a search thru a Tuple slower ? ---- (meaningless indentations)

[Michael Torrie]

On 2/20/23 18:01, Hen Hanna wrote:
> is Comp.Lang.Python very active
Fairly.  Apparently the cool kids are using the Python Discourse forum.

> why is a linear search  thru  a Tuple  slower 
>  (than  thru a  (corresponding)   List ) ???

I cannot say, unfortunately.  Perhaps doing some analysis of the byte
code with the disasm module could tell you what the interpreter is doing
and why it is slower.  Since tuples are read only, I cannot think of any
reason to use them for large, generated structures. A list is far better
in my opinion.  I use tuples mainly for bundling small amounts of
information together, such as coordinates, or returning multiple values
from a function.

> sometimes,  i 'd  like to put  meaningless indentations
> like  i do (twice)  below
>  ( how can it do this ?)

Fortunately you cannot.  Such indents are syntax errors.

And I have to say it makes your emails very hard to read and understand
when you indent your sentences as you do.  Looks poetic but hard to read.

Also your python example code was not run-able either thanks to those
two extra indents which are syntax errors.  It's always helpful to post
complete and working code examples when asking for help or wanting to
get discussion on a piece of code.
-- 
https://mail.python.org/mailman/listinfo/python-list

why is a search thru a Tuple slower ? ---- (meaningless indentations)

[Hen Hanna]

is Comp.Lang.Python very active


why is a linear search  thru  a Tuple  slower 
 (than  thru a  (corresponding)   List ) ???


sometimes,  i 'd  like to put  meaningless indentations
like  i do (twice)  below
 ( how can it do this ?)

___

import time

X= [x for x in range(1) ]
  TupX= tuple(X)
  SetX= set(X)

Items=[20, 30, 100, 200, 1000,   5000,7000,   8000, 9000 ]
Count=1000

Time = time.time() 
for _ in range(Count):
for i in X:test= i*i; test= i*i *i
print( ' \t ({0:.4f})  X'.format(time.time()-Time) )

Time = time.time() 
for _ in range(Count):
for i in TupX:test= i*i; test= i*i *i
print( ' \t ({0:.4f})  TupX'.format(time.time()-Time) )



Time = time.time() 
for _ in range(Count):
for i in Items:   test= i in X
print( ' \t ({0:.4f}) i in X'.format(time.time()-Time) )

Time = time.time() 
for _ in range(Count):
for i in Items:   test= i in TupX
print( ' \t ({0:.4f}) i in TupX'.format(time.time()-Time) )

-- 
https://mail.python.org/mailman/listinfo/python-list

Python logging (was Re: What should go to stdout/stderr and why Python logging write everything to stderr?)

[Weatherby,Gerard]

FWIW, it wasn’t too difficult to build a logging handler to send messages to 
Slack.

https://pypi.org/project/slack-webclient-logger/
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: What should go to stdout/stderr and why Python logging write everything to stderr?

[Peter J. Holzer]

On 2023-01-05 12:29:33 -0800, Grant Edwards wrote:
> On 2023-01-05, Thomas Passin  wrote:
> 
> > The logging system is so configurable that...
> 
> I find it almost impossible to use unless I copy a working example I
> find somewhere. ;)

I usually copy the working example from my previous project ;-).

I think the general structure is clear enough. You need a formatter to
format your log messages, a handler to actually write them and finally a
logger to determine what goes where. I can look up the details in the
docs but generally the logging is always the same (except for path names
and log levels), so I can just copy the config from last time and adjust
those.

So I might have a config.py like this:

# ...
logging = {
"version": 1,
"disable_existing_loggers": False,
"formatters": {
"standard": {
"format": "%(asctime)s %(levelname)s %(name)s %(funcName)s 
%(lineno)d | %(message)s"
}
},
"handlers": {
"file": {
"class": "switchinglogfilehandlers.TimeoutSwitchingFileHandler",
"formatter": "standard",
"filename": "/var/log/www/XXX.hjp.at/XXX.",
},
},
"loggers": {
"": {
"handlers": ["file"],
"level": "INFO"
}
}
}
# ...

And then my application would start like this:

import logging
import logging.config
import config
logging.config.dictConfig(config.logging)
log = logging.getLogger(__name__)

Plus typically every other source file contains

import logging
log = logging.getLogger(__name__)

somewhere near the start.

Then I can just scatter my log.debug(...) or whatever whereever I want.
When I decide that I need debug output from one module, I'll just add a
logger. Or if some library is too chatty I'll add another logger to shut
it up - no programming, just a few extra lines in a dict.

(Instead of a config.py I might use a json file, especially if I expect
the config to change often.)

> I'm not at all surprised that the OP didn't understand how it
> works.

It probably helps to have worked with log4j before that. The structure
is very similar, although I find Python logging easier to use (but then
I never did much Java programming so that's probably just a matter of
familiarity.

hp

PS: The TimeoutSwitchingFileHandler mentioned above is one I wrote
myself. You can find it on PyPI, but be warned that the
documentation is (still) quite lacking.

-- 
   _  | Peter J. Holzer| Story must make more sense than reality.
|_|_) ||
| |   | h...@hjp.at |-- Charles Stross, "Creative writing
__/   | http://www.hjp.at/ |   challenge!"


signature.asc
Description: PGP signature
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: asyncio and tkinter (was: What should go to stdout/stderr and why Python logging write everything to stderr?)

[Thomas Passin]

On 1/5/2023 7:52 PM, Stefan Ram wrote:

Thomas Passin  writes:

On 1/5/2023 4:24 PM, Stefan Ram wrote:

You often can replace threads in tkinter by coroutines using
asyncio when you write a replacement for the mainloop of
tkinter that uses asyncio. Now, try to read only the official
documentation of asyncio and tkinter and figure out only from
this how to get such a program to work!

Cool! Can we have a link, please?


   I do not post links, but tried to create a minimal example app.


Thanks!  That's not exactly obvious, is it?  Congratulations on getting 
it working.



import asyncio
import logging
import tkinter

# This program was tested on Windows, it is experimental.
# It should open a tkinter root window.
# Ctrl-E should create a task. You should be able to create
# a new task even while one task is already running.
# Each task will end after about 10 seconds.
# Ctrl-X should exit the process.
# Or, use the menu "Action".

# I (Stefan Ram) designed and wrote the program, but especially
# the code in "app_async_mainloop" and "terminate_tasks" was
# written following educational material from the World-Wide Web.

class app_class( tkinter.Tk ):
 def __init__( self, *args, **kwargs ):
 self.is_running = 1_000_000
 super().__init__( *args, **kwargs )
 root = self
 root.protocol( "WM_DELETE_WINDOW", self.app_exit_macro )
 async def app_example_task( self, example_label ):
 try:
 for i in range( 10 ):
 example_label[ 'text' ]=str( i )
 await asyncio.sleep( 1 )
 except asyncio.CancelledError:
 pass
 example_label.destroy()
 def app_example_macro( self ):
 root = self
 example_label=tkinter.Label( root )
 example_label.pack()
 asyncio.get_running_loop().create_task\
 ( self.app_example_task( example_label ))
 root = self
 def terminate_tasks( self ):
 loop = asyncio.get_running_loop()
 pending = asyncio.all_tasks( loop=loop )
 label_tasks = []
 for task in pending:
 if 'app_example_task' in str( task ):
 label_tasks.append( task )
 task.cancel()
 group = asyncio.gather( *label_tasks, return_exceptions=True )
 try:
 loop.run_until_complete( group )
 except AssertionError:
 # ignoring an assertion error on Windows I do not understand
 pass
 def app_exit_macro( self ):
 self.terminate_tasks()
 self.is_running = 99
 root = self
 root.destroy()
 def app_add_basemenu( self, example=False ):
 root = self
 menubar = tkinter.Menu( root )
 menu = tkinter.Menu( menubar, tearoff=0 )
 name = "Actions"; menu = tkinter.Menu( menubar, tearoff=0 )
 if example:
 text = "Example";
 callback = self.app_example_macro; menu.add_command\
 ( label=text, underline=0, command=callback,
   accelerator="Control-e" );
 root.bind\
 ( "", lambda event, callback=callback:callback() )
 text = "Exit";
 callback = self.app_exit_macro
 menu.add_command\
 ( label=text, underline=1,
   command=callback, accelerator="Control-x" )
 root.bind\
 ( "", lambda event,callback=callback: callback() )
 menubar.add_cascade( label=name, underline=0, menu=menu )
 root.config( menu=menubar )
 async def app_async_mainloop( self ):
 root = self
 root.willdispatch()
 root.tk.dooneevent( tkinter._tkinter.DONT_WAIT )
 while self.is_running > 0:
 if self.is_running < 1_000_000: self.is_running -= 1
 try:
 await asyncio.sleep\
 ( tkinter._tkinter.getbusywaitinterval() / 10_000 )
 root.tk.dooneevent( tkinter._tkinter.DONT_WAIT )
 root.update()
 except tkinter.TclError:
 break
 async def app_async_main( self ):
 try:
 await self.app_async_mainloop()
 except asyncio.CancelledError:
 logging.debug( f'asyncio.CancelledError' )
 def app_async_run( self ):
 asyncio.run( self.app_async_main() )

app = app_class()
app.app_add_basemenu(example=True)
app.app_async_run()





--
https://mail.python.org/mailman/listinfo/python-list

Re: What should go to stdout/stderr and why Python logging write everything to stderr?

[Thomas Passin]

On 1/5/2023 4:24 PM, Stefan Ram wrote:

You often can replace threads in tkinter by coroutines using
   asyncio when you write a replacement for the mainloop of
   tkinter that uses asyncio. Now, try to read only the official
   documentation of asyncio and tkinter and figure out only from
   this how to get such a program to work!


Cool! Can we have a link, please?

--
https://mail.python.org/mailman/listinfo/python-list

Re: What should go to stdout/stderr and why Python logging write everything to stderr?

[Thomas Passin]

On 1/5/2023 4:28 PM, Weatherby,Gerard wrote:

logging.basicConfig()
logging.info(“Nice to know”)
logging.debug(“Details for when things are funky”)
logging.warn(“Trouble is brewing”)


Not quite -

>>> import logging
>>> logging.basicConfig()
>>> logging.info("Nice to know")
>>> logging.debug("Details for when things are funky")
>>> logging.warn("Trouble is brewing")
:1: DeprecationWarning: The 'warn' function is deprecated, use 
'warning' instead

WARNING:root:Trouble is brewing

Only warning level messages are displayed, unless you do some 
configuration, as mentioned in the logging Howto page:


"The default level is WARNING, which means that only events of this 
level and above will be tracked, unless the logging package is 
configured to do otherwise."


So following the Howto:
>>> logging.basicConfig(level=logging.DEBUG)
>>> logging.info('So should this')
INFO:root:So should this

: Finally!

Not quite straightforward, though it is in the Howto.  Now about those 
handlers ...



From: Python-list  on behalf of 
Grant Edwards 
Date: Thursday, January 5, 2023 at 3:31 PM
To: python-list@python.org 
Subject: Re: What should go to stdout/stderr and why Python logging write 
everything to stderr?
*** Attention: This is an external email. Use caution responding, opening 
attachments or clicking on links. ***

On 2023-01-05, Thomas Passin  wrote:


The logging system is so configurable that...


I find it almost impossible to use unless I copy a working example I
find somewhere. ;)

I'm not at all surprised that the OP didn't understand how it
works. I've been writing Python programs for over 20 years, and it's
beyond me.

--
Grant




--
https://urldefense.com/v3/__https://mail.python.org/mailman/listinfo/python-list__;!!Cn_UX_p3!l6vSXQFKppEuLS0R5gYeLYaiHyVFfs2Rapqm1oPGEtvnZ5ivQyApZcnJyNTnnH9zEVY80ajNb-HfYkNwgw8fMtsnlSOT$<https://urldefense.com/v3/__https:/mail.python.org/mailman/listinfo/python-list__;!!Cn_UX_p3!l6vSXQFKppEuLS0R5gYeLYaiHyVFfs2Rapqm1oPGEtvnZ5ivQyApZcnJyNTnnH9zEVY80ajNb-HfYkNwgw8fMtsnlSOT$>


--
https://mail.python.org/mailman/listinfo/python-list

Re: What should go to stdout/stderr and why Python logging write everything to stderr?

[Grant Edwards]

On 2023-01-05, Weatherby,Gerard  wrote:
> logging.basicConfig()
> logging.info(“Nice to know”)
> logging.debug(“Details for when things are funky”)
> logging.warn(“Trouble is brewing”)

I always seem to need something slightly more complex. Usually something like:

 * Specify a log level on the command line.
 * Allow logging to a file as specified on the command line.
 * Optional timestamps for log messages.

I know, without doubt, that it can easily do all those things. I just
never seem to be able to figure out how unless I can find example code
to look at.

--
Grant

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: What should go to stdout/stderr and why Python logging write everything to stderr?

[Weatherby,Gerard]

logging.basicConfig()
logging.info(“Nice to know”)
logging.debug(“Details for when things are funky”)
logging.warn(“Trouble is brewing”)

From: Python-list  on 
behalf of Grant Edwards 
Date: Thursday, January 5, 2023 at 3:31 PM
To: python-list@python.org 
Subject: Re: What should go to stdout/stderr and why Python logging write 
everything to stderr?
*** Attention: This is an external email. Use caution responding, opening 
attachments or clicking on links. ***

On 2023-01-05, Thomas Passin  wrote:

> The logging system is so configurable that...

I find it almost impossible to use unless I copy a working example I
find somewhere. ;)

I'm not at all surprised that the OP didn't understand how it
works. I've been writing Python programs for over 20 years, and it's
beyond me.

--
Grant




--
https://urldefense.com/v3/__https://mail.python.org/mailman/listinfo/python-list__;!!Cn_UX_p3!l6vSXQFKppEuLS0R5gYeLYaiHyVFfs2Rapqm1oPGEtvnZ5ivQyApZcnJyNTnnH9zEVY80ajNb-HfYkNwgw8fMtsnlSOT$<https://urldefense.com/v3/__https:/mail.python.org/mailman/listinfo/python-list__;!!Cn_UX_p3!l6vSXQFKppEuLS0R5gYeLYaiHyVFfs2Rapqm1oPGEtvnZ5ivQyApZcnJyNTnnH9zEVY80ajNb-HfYkNwgw8fMtsnlSOT$>
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: What should go to stdout/stderr and why Python logging write everything to stderr?

[Thomas Passin]

On 1/5/2023 3:29 PM, Grant Edwards wrote:

On 2023-01-05, Thomas Passin  wrote:


The logging system is so configurable that...


I find it almost impossible to use unless I copy a working example I
find somewhere. ;)

I'm not at all surprised that the OP didn't understand how it
works. I've been writing Python programs for over 20 years, and it's
beyond me.


I know what you mean!  I have never used the Python logging package, but 
I inherited a java project that uses Log4j.  Talk about obscure!  The 
python docs are a wonder of clarity beside Log4j.  So most of my Log4j 
configuration has been by copy-pasta, with occasional surprises and 
Log4j bug bites.  Like a handler that deleted its own previous log file 
when it rolled over at midnight - a "feature" that wasn't advertised.


I will say, though, that having used a bit of Log4j must have been 
helpful, since much of what is in the Python logging package docs seemed 
somewhat familiar and even seemed to make sense.


BTW, I was able to re-create that java project using Jython for about 
90% of it.  Way better!


--
https://mail.python.org/mailman/listinfo/python-list

Re: What should go to stdout/stderr and why Python logging write everything to stderr?

[Grant Edwards]

On 2023-01-05, Thomas Passin  wrote:

> The logging system is so configurable that...

I find it almost impossible to use unless I copy a working example I
find somewhere. ;)

I'm not at all surprised that the OP didn't understand how it
works. I've been writing Python programs for over 20 years, and it's
beyond me.

--
Grant




-- 
https://mail.python.org/mailman/listinfo/python-list

Re: What should go to stdout/stderr and why Python logging write everything to stderr?

[Thomas Passin]

On 1/5/2023 2:18 PM, Peter J. Holzer wrote:

On 2023-01-05 08:31:40 -0500, Thomas Passin wrote:

The logging system is so configurable that a user could set a different
destination for each level of logging.  So it seems that the O.P.'s original
question about why the package's developers choose stderr for all levels can
be answered: "They didn't".


Which is almost exactly what I wrote in
<20230104162154.uzljittbs6xwt...@hjp.at> ;-)


Yup, two minds are better than one :)

--
https://mail.python.org/mailman/listinfo/python-list

Re: What should go to stdout/stderr and why Python logging write everything to stderr?

[Peter J. Holzer]

On 2023-01-05 08:31:40 -0500, Thomas Passin wrote:
> The logging system is so configurable that a user could set a different
> destination for each level of logging.  So it seems that the O.P.'s original
> question about why the package's developers choose stderr for all levels can
> be answered: "They didn't".

Which is almost exactly what I wrote in
<20230104162154.uzljittbs6xwt...@hjp.at> ;-)

hp

-- 
   _  | Peter J. Holzer| Story must make more sense than reality.
|_|_) ||
| |   | h...@hjp.at |-- Charles Stross, "Creative writing
__/   | http://www.hjp.at/ |   challenge!"


signature.asc
Description: PGP signature
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: What should go to stdout/stderr and why Python logging write everything to stderr?

[Thomas Passin]

On 1/5/2023 6:27 AM, Peter J. Holzer wrote:

On 2023-01-04 12:32:40 -0500, Thomas Passin wrote:

On 1/3/2023 10:35 AM, c.bu...@posteo.jp wrote:

The logging module write everything to stderr no matter which logging
level is used.


The OP wrote this, but it's not so by default.


By default it is - sort of.

That is all log messages go to stderr, but not all log levels are
logged.


Actually, I misread something else that made me think that non-warning 
messages would go to stdout, but I was mistaken.  As it happens, the 
page in the Python docs only mention stderr twice, both in the same 
paragraph:


"logging.lastResort
A “handler of last resort” is available through this attribute. This is 
a StreamHandler writing to sys.stderr with a level of WARNING, and is 
used to handle logging events in the absence of any logging 
configuration.  The end result is to just print the message to sys.stderr."


Nowhere else on this page is the printed output destination mentioned, 
except that it gets printed on the console, which could be either stderr 
or stdout.


The logging system is so configurable that a user could set a different 
destination for each level of logging.  So it seems that the O.P.'s 
original question about why the package's developers choose stderr for 
all levels can be answered: "They didn't".


--
https://mail.python.org/mailman/listinfo/python-list

Re: What should go to stdout/stderr and why Python logging write everything to stderr?

[Peter J. Holzer]

On 2023-01-04 12:32:40 -0500, Thomas Passin wrote:
> On 1/3/2023 10:35 AM, c.bu...@posteo.jp wrote:
> > The logging module write everything to stderr no matter which logging
> > level is used.
> 
> The OP wrote this, but it's not so by default.

By default it is - sort of. 

That is all log messages go to stderr, but not all log levels are
logged.

hp

-- 
   _  | Peter J. Holzer| Story must make more sense than reality.
|_|_) ||
| |   | h...@hjp.at |-- Charles Stross, "Creative writing
__/   | http://www.hjp.at/ |   challenge!"


signature.asc
Description: PGP signature
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: What should go to stdout/stderr and why Python logging write everything to stderr?

[Chris Angelico]

On Thu, 5 Jan 2023 at 05:06, Barry Scott  wrote:
>
>
> On 03/01/2023 21:24, c.bu...@posteo.jp wrote:
> > Am 03.01.2023 17:51 schrieb r...@zedat.fu-berlin.de:
> >> logging.getLogger().addHandler( logging.StreamHandler( sys.stdout ))
> >
> > But I don't want to make all log levels go to stdout. Just DEBUG and
> > INFO. But this would be a workaround.
> >
> > The main question here is why does Python deciecded to make all logs
> > go to stderr?
> > Maybe I totally misunderstood the intention of logging.info()?! Isn't
> > this the "usual applicaton output"?
> >
> > If not, what messages should go into logging.info()? Can you name me
> > some examples?
>
> Example:
>
> write an app that prints the contents of a file.
>
> The application output is the contents of the file.
>
> The logging might be this when all is working:
>
> INFO About to open 
> INFO Wrote  bytes from 
>
> The logging might be this when there is a problem:
>
> INFO About to open 
> ERROR Failed to open  - 
>
> Does that help?

Notably, the info lines can provide context for an error. For example:

INFO: Loading module XYZ
WARN: Unrecognized module option frobnicate=1
INFO: Using default spamination of 4
ERROR: Insufficient ham to spaminate

This tells a hypothetical (but only slightly hypothetical) story of a
module that was given an incorrect or misspelled option, and which
then failed to load as a consequence. Without the INFO lines, it would
be harder to figure out what the problem was, but they are still part
of logging, not output, and belong in the same place as the warnings
and errors.

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: What should go to stdout/stderr and why Python logging write everything to stderr?

[Barry Scott]

On 03/01/2023 21:24, c.bu...@posteo.jp wrote:

Am 03.01.2023 17:51 schrieb r...@zedat.fu-berlin.de:

logging.getLogger().addHandler( logging.StreamHandler( sys.stdout ))


But I don't want to make all log levels go to stdout. Just DEBUG and 
INFO. But this would be a workaround.


The main question here is why does Python deciecded to make all logs 
go to stderr?
Maybe I totally misunderstood the intention of logging.info()?! Isn't 
this the "usual applicaton output"?


If not, what messages should go into logging.info()? Can you name me 
some examples?


Example:

write an app that prints the contents of a file.

The application output is the contents of the file.

The logging might be this when all is working:

INFO About to open 
INFO Wrote  bytes from 

The logging might be this when there is a problem:

INFO About to open 
ERROR Failed to open  - 

Does that help?

Barry


--
https://mail.python.org/mailman/listinfo/python-list

Re: What should go to stdout/stderr and why Python logging write everything to stderr?

[Barry Scott]

On 03/01/2023 21:24, c.bu...@posteo.jp wrote:

Am 03.01.2023 17:51 schrieb r...@zedat.fu-berlin.de:

logging.getLogger().addHandler( logging.StreamHandler( sys.stdout ))


But I don't want to make all log levels go to stdout. Just DEBUG and 
INFO. But this would be a workaround.


The main question here is why does Python deciecded to make all logs 
go to stderr?
Maybe I totally misunderstood the intention of logging.info()?! Isn't 
this the "usual applicaton output"?


If not, what messages should go into logging.info()? Can you name me 
some examples?


It is up to you, the designer of an application, to decide how it works.
You will take into account conventions that your users will expect you 
to follow.


If the logging module helps you then use it, but you are not forced by 
logging to
design your app is a particular way. The default behavior of the logging 
module is
a generally useful default, but its only a default. You are free to 
setup logging to

meet your needs.

Barry


--
https://mail.python.org/mailman/listinfo/python-list

Re: What should go to stdout/stderr and why Python logging write everything to stderr?

[Thomas Passin]

On 1/3/2023 10:35 AM, c.bu...@posteo.jp wrote:

The logging module write everything to stderr no matter which logging
level is used.


The OP wrote this, but it's not so by default.  There is a HOW-TO page 
that explains this and how to get the logging package to log everything 
to a file, along with many other useful bits of information -


https://docs.python.org/3/howto/logging.html

--
https://mail.python.org/mailman/listinfo/python-list

Re: What should go to stdout/stderr and why Python logging write everything to stderr?

[Peter J. Holzer]

On 2023-01-03 21:24:20 +, c.bu...@posteo.jp wrote:
> The main question here is why does Python deciecded to make all logs go to
> stderr?

It doesn't.

The Python logging system provides a plethora of handlers to log to lots
of different places. Only the StreamHandler defaults to writing to
stderr, and even that can log to any stream - just pass it as an
argument.

If none of existing handlers does what you want you can always write
your own.

hp

-- 
   _  | Peter J. Holzer| Story must make more sense than reality.
|_|_) ||
| |   | h...@hjp.at |-- Charles Stross, "Creative writing
__/   | http://www.hjp.at/ |   challenge!"


signature.asc
Description: PGP signature
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: What should go to stdout/stderr and why Python logging write everything to stderr?

[Peter J. Holzer]

On 2023-01-04 16:46:37 +1100, Chris Angelico wrote:
> On Wed, 4 Jan 2023 at 15:11, Eryk Sun  wrote:
> > On 1/3/23, Chris Angelico  wrote:
> > > writing the FD is the same as using stdout
> >
> > Except stdout may be buffered.

Maybe I'm overly lax in my terminology, but I frequently use the term
"stdout" as a shorthand for "standard output" which refers to file
descriptor 1, not just to the C FILE* of that name. Or Python's
sys.stdout which is a different thing again (but also is a frontend to
FD 1).

> > One should probably flush the buffer
> > before each raw write to the file descriptor.
> 
> FDs can also be buffered.

That depends on what you mean by "buffered". A write to an fd referring
to a file on a disk will not usually affect the disk, true. But it will
be immediately visible to other processes reading that file.

A write to a terminal attached via a serial cable will also be written
to a buffer first (in the OS or maybe inside the UART), but it will
be sent to the terminal as fast as the hardware allows.

So. sure, the writes are "buffered", but is that meaningful in this
context?

Reading from a terminal also often involves a buffer, but in a very
different sense: Here the OS buffers each line to implement line editing
functions. If the kernel didn't do that each application which could
possibly read from a terminal would have to implement that itself. and
it was decided back in the 1970's that this wasn't a good use of
resources. (Of course these days many programs which didn't do that
originally (like shells) do implement line editing (often via
readline).)

> If it's buffering you want to avoid, don't
> mess around with exactly which one you're writing to, just flush.

I don't think fsync() will have an effect on an FD connected to a
terminal (or a pipe or a socket).

hp

-- 
   _  | Peter J. Holzer| Story must make more sense than reality.
|_|_) ||
| |   | h...@hjp.at |-- Charles Stross, "Creative writing
__/   | http://www.hjp.at/ |   challenge!"


signature.asc
Description: PGP signature
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: What should go to stdout/stderr and why Python logging write everything to stderr?

[Barry Scott]

On 04/01/2023 06:46, Chris Angelico wrote:

I've known some systems to have a trigger of "reading on FD 0 flushes
FD 1"


C++ has this feature:

Quote from https://en.cppreference.com/w/cpp/io/cin

"Once |std::cin| is constructed, std::cin.tie() returns ::cout 
, and likewise, 
std::wcin.tie() returns ::wcout 
. This means that any 
formatted input operation on |std::cin| forces a call to std::cout 
.flush() if any characters are 
pending for output."


Barry

--
https://mail.python.org/mailman/listinfo/python-list

Re: What should go to stdout/stderr and why Python logging write everything to stderr?

[Barry Scott]

On 04/01/2023 02:26, Chris Angelico wrote:

Reading/writing the FD is the same as using stdout (technically you'd
write to fd 1 and read from fd 0), but yes, can use /dev/tty to reach
for the console directly.


I think the logic is more like checking that stdin is a tty then using
the tty it to read and write the tty. This works with stdout redirected.
Also stdin is likely only open with read access.

For example the way to prevent ssh-add from prompting in the terminal is 
this:


$ ssh-add ~/.ssh/id_rsa https://mail.python.org/mailman/listinfo/python-list

Re: What should go to stdout/stderr and why Python logging write everything to stderr?

[2QdxY4RzWzUUiLuE]

On 2023-01-04 at 12:02:55 +,
"Weatherby,Gerard"  wrote:

> Dealing with stdout / stderr is bash is just syntax. I don’t always
> remember it off the top of my head but … stackoverflow.com.

https://tldp.org/LDP/abs/html/io-redirection.html

> On Linux at least it’s possible to pipe to arbitrary streams, it
> seems. The following uses bash to write “Hi” to the file “third”
> opened by Python. I determined the file was 5 empirically.

Empirically?

> import os
> import subprocess
> command= 'echo Hi >/dev/fd/5'
> fd = os.open("third",os.O_WRONLY|os.O_CREAT)

command = f"echo Hi >/dev/fd/{fd}"

Or:

command = f"echo Hi >&{fd}"

> os.set_inheritable(fd,True)
> 
> subprocess.run(('/usr/bin/bash','-c',command),close_fds=False)

By the time I'm that far down that path, I usually write a separate
function to call fork, open, close, and exec myself (old habits die
hard).  In the end, there's no reward for writing and maintaining shell
programs in Python.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: What should go to stdout/stderr and why Python logging write everything to stderr?

[Weatherby,Gerard]

Dealing with stdout / stderr is bash is just syntax. I don’t always remember it 
off the top of my head but … stackoverflow.com.

On Linux at least it’s possible to pipe to arbitrary streams, it seems. The 
following uses bash to write “Hi” to the file “third” opened by Python. I 
determined the file was 5 empirically.


import os
import subprocess
command= 'echo Hi >/dev/fd/5'
fd = os.open("third",os.O_WRONLY|os.O_CREAT)
os.set_inheritable(fd,True)

subprocess.run(('/usr/bin/bash','-c',command),close_fds=False)

From: Python-list  on 
behalf of Michael Torrie 
Date: Tuesday, January 3, 2023 at 5:18 PM
To: python-list@python.org 
Subject: Re: What should go to stdout/stderr and why Python logging write 
everything to stderr?
*** Attention: This is an external email. Use caution responding, opening 
attachments or clicking on links. ***


Maybe some day an interface and shell syntax will be developed to
interact with an arbitrary number of standard streams.  Current piping
syntax really only works well with one stream and even trying to use
stderr and stdout with pipes and filters in a shell is awkward.




--
https://urldefense.com/v3/__https://mail.python.org/mailman/listinfo/python-list__;!!Cn_UX_p3!j8qV9yx1DJI_G2F-q1fQz2LfnVYoMi40Qpk_h8bxrOcw50rVXpwScpFJSyZ212Tm9rj6T7vKgJjaIEgLRw$<https://urldefense.com/v3/__https:/mail.python.org/mailman/listinfo/python-list__;!!Cn_UX_p3!j8qV9yx1DJI_G2F-q1fQz2LfnVYoMi40Qpk_h8bxrOcw50rVXpwScpFJSyZ212Tm9rj6T7vKgJjaIEgLRw$>
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: What should go to stdout/stderr and why Python logging write everything to stderr?

[Chris Angelico]

On Wed, 4 Jan 2023 at 17:26, Eryk Sun  wrote:
>
> On 1/3/23, Chris Angelico  wrote:
> >
> > FDs can also be buffered. If it's buffering you want to avoid, don't
> > mess around with exactly which one you're writing to, just flush.
>
> I meant to flush a C FILE stream or Python file before writing
> directly to the file descriptor, in order to avoid out-of-sequence and
> interlaced writes that make no sense. Any OS buffering on the file
> doesn't matter in this regard.

True, that can help. If you're about to prompt for a password, though,
OS buffering can most certainly make a difference. And it doesn't hurt
to explicitly flush when you need it flushed.

I've known some systems to have a trigger of "reading on FD 0 flushes
FD 1", but that's not going to work when you use certain
keyboard-input routines (ISTR readline was one of the few things that
*didn't* have that problem, but I may be misremembering), so the
general strategy was: print your prompt, flush stdout, THEN request
input.

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: What should go to stdout/stderr and why Python logging write everything to stderr?

[Eryk Sun]

On 1/3/23, Chris Angelico  wrote:
>
> FDs can also be buffered. If it's buffering you want to avoid, don't
> mess around with exactly which one you're writing to, just flush.

I meant to flush a C FILE stream or Python file before writing
directly to the file descriptor, in order to avoid out-of-sequence and
interlaced writes that make no sense. Any OS buffering on the file
doesn't matter in this regard.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: What should go to stdout/stderr and why Python logging write everything to stderr?

[Eryk Sun]

On 1/3/23, Grant Edwards  wrote:
>
> That's definitely a better option, but I'm pretty sure I've seen
> multiple examples over the decades where fd 0 was used instead. Was
> /dev/tty a "recent" enough invention that I would have seen
> application code that was written before it existed? Or maybe I was
> just looking at code written by people who weren't aware of /dev/tty?

POSIX has required "/dev/tty" for 20 years, or longer. It's in the
2004 edition of the spec, which was a minor update to the 2001
edition.

https://pubs.opengroup.org/onlinepubs/95399/basedefs/xbd_chap10.html

> No, the whole point is _not_ to write to stdout (which would corrupt
> the program's output). The way I remember it was that you wrote the
> prompt to fd 0, and then read the password from fd 0. That way it
> worked even when fd 1 (stdout) was redirected. It still failed if
> stdin was redirected...

FYI, that wouldn't work on Windows. The "\\.\CONIN$" and "\\.\CONOUT$"
files can be opened with read-write access, but it's for a different
purpose.

A console input buffer can be read from via read() or WinAPI
ReadFile(). It can also be read from using IOCTLs that work with
16-bit wide-character strings, which is the basis of WinAPI
ReadConsoleW() and ReadConsoleInputW(). A console input buffer can be
*written to* via the IOCTL-based function WriteConsoleInputW().

A console screen buffer can be written to via write() or WinAPI
WriteFile(). It can also be written to using IOCTLs that work with
16-bit wide-character strings, which is the basis of WriteConsoleW()
and WriteConsoleOutputW(). A console screen buffer can be *read from*
via the IOCTL-based function ReadConsoleOutputW().
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: What should go to stdout/stderr and why Python logging write everything to stderr?

[Chris Angelico]

On Wed, 4 Jan 2023 at 16:21, Grant Edwards  wrote:
>
> On 2023-01-04, Chris Angelico  wrote:
> > On Wed, 4 Jan 2023 at 09:52, Grant Edwards  
> > wrote:
> >>
> >>> I can't think of a specific example, but I know I have piped the output
> >>> of a program while at the same time interacting with a prompt on stderr.
> >>> A rare thing, though.
> >>
> >> Programs that ask for passwords often do it by reading/writing to
> >> fd 0 or /dev/tty so that stdout is unaffected.
> >
> > Reading/writing the FD is the same as using stdout.
>
> No, stdout is fd 1.
>
> > (technically you'd write to fd 1 and read from fd 0),

I assumed that you were using a shorthand. My bad.

> No, the whole point is _not_ to write to stdout (which would corrupt
> the program's output). The way I remember it was that you wrote the
> prompt to fd 0, and then read the password from fd 0. That way it
> worked even when fd 1 (stdout) was redirected. It still failed if
> stdin was redirected...

Writing to FD 0 assumes that (a) it was originally connected to the
TTY, and (b) it hasn't been redirected. Not a reliable assumption.

rosuav@sikorsky:~/tmp$ cat testfds.py
import os
for fd in range(3):
os.write(fd, b"This is on FD %d\n" % fd)
rosuav@sikorsky:~/tmp$ python3 testfds.py
This is on FD 0
This is on FD 1
This is on FD 2
rosuav@sikorsky:~/tmp$ echo | python3 testfds.py
Traceback (most recent call last):
  File "/home/rosuav/tmp/testfds.py", line 3, in 
os.write(fd, b"This is on FD %d\n" % fd)
OSError: [Errno 9] Bad file descriptor

Writing to fd 0 might work if stdout is redirected, but it won't work
if stdin is redirected, so honestly, I don't think this hack is at all
reliable.

> > but yes, can use /dev/tty to reach for the console directly.
>
> That's definitely a better option, but I'm pretty sure I've seen
> multiple examples over the decades where fd 0 was used instead. Was
> /dev/tty a "recent" enough invention that I would have seen
> application code that was written before it existed? Or maybe I was
> just looking at code written by people who weren't aware of /dev/tty?

No idea. Probably. It's definitely possible that someone tries it with
stdout redirected and goes "oh hey, writing to fd 0 still works", and
never tries it with stdin redirected. It's very common for programmers
to see that something works without knowing WHY it works :)

By the way, I can reinstate the behaviour of "FD 0 writes to the TTY"
in bash by redirecting it:

rosuav@sikorsky:~/tmp$ python3 testfds.py >/dev/null 0<>/dev/tty
This is on FD 0
This is on FD 2

which is another neat demonstration that bash has more power than you
will ever need...

(though redirecting FDs above 2 in read/write mode can be *incredibly* useful.)

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: What should go to stdout/stderr and why Python logging write everything to stderr?

[Chris Angelico]

On Wed, 4 Jan 2023 at 15:11, Eryk Sun  wrote:
>
> On 1/3/23, Chris Angelico  wrote:
> >
> > writing the FD is the same as using stdout
>
> Except stdout may be buffered. One should probably flush the buffer
> before each raw write to the file descriptor.

FDs can also be buffered. If it's buffering you want to avoid, don't
mess around with exactly which one you're writing to, just flush.

> > use /dev/tty to reach for the console directly.
>
> On Windows, open "CON" (read or write), "CONIN$" (read-write), or
> "CONOUT$" (read-write). Or use the more explicit device paths
> "\\.\CON", "\\.\CONIN$", and "\\.\CONOUT$". If the process has no
> console, one can call WinAPI AllocConsole() or
> AttachConsole(process_id) to obtain one.

Yeah, /dev/tty is the Unix option, Windows has its own way, but
whatever platform you're on, there's a way to directly reach the
console.

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: What should go to stdout/stderr and why Python logging write everything to stderr?

[Grant Edwards]

On 2023-01-04, Chris Angelico  wrote:
> On Wed, 4 Jan 2023 at 09:52, Grant Edwards  wrote:
>>
>>> I can't think of a specific example, but I know I have piped the output
>>> of a program while at the same time interacting with a prompt on stderr.
>>> A rare thing, though.
>>
>> Programs that ask for passwords often do it by reading/writing to
>> fd 0 or /dev/tty so that stdout is unaffected.
>
> Reading/writing the FD is the same as using stdout.

No, stdout is fd 1.

> (technically you'd write to fd 1 and read from fd 0),

No, the whole point is _not_ to write to stdout (which would corrupt
the program's output). The way I remember it was that you wrote the
prompt to fd 0, and then read the password from fd 0. That way it
worked even when fd 1 (stdout) was redirected. It still failed if
stdin was redirected...

IIRC, you can't usually write to the stdin stream in C (or Python), so
you had to write(0,promptstr,strlen(promptstr)) instead of
printf("%s",promptstr) or fputs(promptstr,stdin).

> but yes, can use /dev/tty to reach for the console directly.

That's definitely a better option, but I'm pretty sure I've seen
multiple examples over the decades where fd 0 was used instead. Was
/dev/tty a "recent" enough invention that I would have seen
application code that was written before it existed? Or maybe I was
just looking at code written by people who weren't aware of /dev/tty?

--
Grant
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: What should go to stdout/stderr and why Python logging write everything to stderr?

[Eryk Sun]

On 1/3/23, Chris Angelico  wrote:
>
> writing the FD is the same as using stdout

Except stdout may be buffered. One should probably flush the buffer
before each raw write to the file descriptor.

> use /dev/tty to reach for the console directly.

On Windows, open "CON" (read or write), "CONIN$" (read-write), or
"CONOUT$" (read-write). Or use the more explicit device paths
"\\.\CON", "\\.\CONIN$", and "\\.\CONOUT$". If the process has no
console, one can call WinAPI AllocConsole() or
AttachConsole(process_id) to obtain one.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: What should go to stdout/stderr and why Python logging write everything to stderr?

[Chris Angelico]

On Wed, 4 Jan 2023 at 10:28, Stefan Ram  wrote:
>
> c.bu...@posteo.jp writes:
> >But I don't want to make all log levels go to stdout. Just DEBUG and
> >INFO.
>
>   The following was stripped down from something some guy
>   posted on a web site, maybe it was "rkachach".

Yet another shining example of bad code that does what someone asks,
but will make life harder for the next person to come along and try to
work with that program. It's like if someone asked for a way to make
True equal to -1 instead of 1, for compatibility with BASIC. Bad idea.

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: What should go to stdout/stderr and why Python logging write everything to stderr?

[Chris Angelico]

On Wed, 4 Jan 2023 at 09:52, Grant Edwards  wrote:
>
> On 2023-01-03, Michael Torrie  wrote:
> > On 1/3/23 11:45, Keith Thompson wrote:
> >> MRAB  writes:
> >> [...]
> >>> The purpose of stderr is to display status messages, logging and error
> >>> messages, even user prompts, and not mess up the program's actual
> >>> output. This is important on a *nix system where you might be piping
> >>> the output of one program into the input of another.
> >>
> >> I would expect user prompts to be written to stdout, or perhaps to some
> >> system-specific stream like the current tty, not to stderr.  If a
> >> program has user prompts, it probably doesn't make sense to pipe its
> >> output to the input of another.
> >
> > I can't think of a specific example, but I know I have piped the output
> > of a program while at the same time interacting with a prompt on stderr.
> > A rare thing, though.
>
> Programs that ask for passwords often do it by reading/writing to fd 0
> or /dev/tty so that stdout is unaffected.
>

Reading/writing the FD is the same as using stdout (technically you'd
write to fd 1 and read from fd 0), but yes, can use /dev/tty to reach
for the console directly.

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: What should go to stdout/stderr and why Python logging write everything to stderr?

[Grant Edwards]

On 2023-01-03, Michael Torrie  wrote:
> On 1/3/23 11:45, Keith Thompson wrote:
>> MRAB  writes:
>> [...]
>>> The purpose of stderr is to display status messages, logging and error
>>> messages, even user prompts, and not mess up the program's actual 
>>> output. This is important on a *nix system where you might be piping
>>> the output of one program into the input of another.
>> 
>> I would expect user prompts to be written to stdout, or perhaps to some
>> system-specific stream like the current tty, not to stderr.  If a
>> program has user prompts, it probably doesn't make sense to pipe its
>> output to the input of another.
>
> I can't think of a specific example, but I know I have piped the output
> of a program while at the same time interacting with a prompt on stderr.
> A rare thing, though.

Programs that ask for passwords often do it by reading/writing to fd 0
or /dev/tty so that stdout is unaffected.

--
Grant

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: What should go to stdout/stderr and why Python logging write everything to stderr?

[Chris Angelico]

On Wed, 4 Jan 2023 at 09:17, Michael Torrie  wrote:
>
> On 1/3/23 11:45, Keith Thompson wrote:
> > MRAB  writes:
> > [...]
> >> The purpose of stderr is to display status messages, logging and error
> >> messages, even user prompts, and not mess up the program's actual
> >> output. This is important on a *nix system where you might be piping
> >> the output of one program into the input of another.
> >
> > I would expect user prompts to be written to stdout, or perhaps to some
> > system-specific stream like the current tty, not to stderr.  If a
> > program has user prompts, it probably doesn't make sense to pipe its
> > output to the input of another.
>
> I can't think of a specific example, but I know I have piped the output
> of a program while at the same time interacting with a prompt on stderr.
>  A rare thing, though.

That can certainly happen, but you may more often see things that go
directly to the TTY rather than using the standard streams at all. For
example, ssh and sudo will prompt on the terminal if they need a
password, to avoid interfering with the standard streams (which can
and often will be redirected).

> Maybe some day an interface and shell syntax will be developed to
> interact with an arbitrary number of standard streams.  Current piping
> syntax really only works well with one stream and even trying to use
> stderr and stdout with pipes and filters in a shell is awkward.

Yeah, maybe some day. And then maybe someone will use a time machine
to backport it into bash. You have at least ten available, with more
being possible but potentially causing conflicts:

https://www.gnu.org/software/bash/manual/html_node/Redirections.html

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: What should go to stdout/stderr and why Python logging write everything to stderr?

[2QdxY4RzWzUUiLuE]

On 2023-01-03 at 21:24:20 +,
c.bu...@posteo.jp wrote:

> The main question here is why does Python deciecded to make all logs
> go to stderr?

It makes sense to send all logging to one destination, and stderr is
that place.

> Maybe I totally misunderstood the intention of logging.info()?! Isn't
> this the "usual applicaton output"?

Aha.  This seems to be the misconception.  "Usual" in the sense of "the
application is running as usual," but not in the sense of "the output
the users usually see."

Logging is the process where applications write details that ordinarily
aren't visible to the user.  Logs are for forensics, after the fact.
There is an entire universe of software for managing (writing, reading,
archiving, retrieving, searching, etc.) log files.

At my last job, we wrote gigabytes of compressed logs every hour, and
read roughly two or three out of a million entries to track down
questions from our users.  Users wouldn't have understood 99% of what
was in the logs anyway.

I also worked a lot in embedded systems, like telephone switches.  Users
(i.e., the people with the telephones) didn't have access to the
switches, but when things went wrong, I could look at the switches' logs
to get the their view of what happened.

> If not, what messages should go into logging.info()? Can you name me some
> examples?

INFO level messages usually track "regular" happy-path progress through
your application:  reading config files, calculating results, writing
report.  Again, not meant for users (who would have their own progress
meters, textual or graphical), but for tracking down problems when there
are problems.  For one example, the name of the config file, even if you
only tell the users that you found a config file.

HTH
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: What should go to stdout/stderr and why Python logging write everything to stderr?

[Michael Torrie]

On 1/3/23 11:45, Keith Thompson wrote:
> MRAB  writes:
> [...]
>> The purpose of stderr is to display status messages, logging and error
>> messages, even user prompts, and not mess up the program's actual 
>> output. This is important on a *nix system where you might be piping
>> the output of one program into the input of another.
> 
> I would expect user prompts to be written to stdout, or perhaps to some
> system-specific stream like the current tty, not to stderr.  If a
> program has user prompts, it probably doesn't make sense to pipe its
> output to the input of another.

I can't think of a specific example, but I know I have piped the output
of a program while at the same time interacting with a prompt on stderr.
 A rare thing, though.

Maybe some day an interface and shell syntax will be developed to
interact with an arbitrary number of standard streams.  Current piping
syntax really only works well with one stream and even trying to use
stderr and stdout with pipes and filters in a shell is awkward.




-- 
https://mail.python.org/mailman/listinfo/python-list

Re: What should go to stdout/stderr and why Python logging write everything to stderr?

[Chris Angelico]

On Wed, 4 Jan 2023 at 08:25,  wrote:
>
> Am 03.01.2023 17:51 schrieb r...@zedat.fu-berlin.de:
> > logging.getLogger().addHandler( logging.StreamHandler( sys.stdout ))
>
> But I don't want to make all log levels go to stdout. Just DEBUG and
> INFO. But this would be a workaround.

But why are DEBUG logs part of stdout? That doesn't make any sense.

> The main question here is why does Python deciecded to make all logs go
> to stderr?
> Maybe I totally misunderstood the intention of logging.info()?! Isn't
> this the "usual applicaton output"?
>
> If not, what messages should go into logging.info()? Can you name me
> some examples?

Reports of what the program is doing.

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: What should go to stdout/stderr and why Python logging write everything to stderr?

[Cameron Simpson]

On Jan 3, 2023, at 10:38 AM, c.bu...@posteo.jp wrote:
this posting isn't about asking for a technical solution. My 
intention is to understand the design decision Python's core developers made in

context of that topic.

The logging module write everything to stderr no matter which logging
level is used.


Sure. They're logs.


The argparse module does it more differentiated. If arguments are
mandatory but none are given then argparse produce a short version of
the usage info; on stderr. If the user request the usage info via "-h"
it goes to stdout. This is the behavior I would expect.


Error messages to stderr. _Requested output_ to stdout. Good with me.


Why does logging behave different? DEBUG and INFO imho should go to
stdout not stderr.


I entirely disagree.

Consider a shell pipeline:

cmd1 | cmd2 | cmd3

where `cmd2` and `cmd3` parse data coming down the pipes. Letting 
logging info into those data streams would produce garbage. This 
separation between core programme output and error/logging output is the 
basic reason that the two streams exist, and the reason logging goes to 
stderr by default.


You're conflating "normal output" with INFO, which is not a substitute 
for eg `print()`.



Of course I could modify the handlers etc. to workaround this. But I
assume that there was something in mind of the Python developers when
they decided that.


Basicly, logging data is not the normal output. So it does not get sent 
to stdout. The log level doesn't change that characteristic.



My goal is not to divide between the use of print() or logging.info()
in my own code. This would mess up a lot.


I think you should, or you should fiddle the handlers as you previous 
resist doing if it is your goal.


But keep in mind that if you do that globally (eg in the root logger) it 
will affect _all_ logging calls, not merely your own, and other 
libraries will assume they can log at whatever level and not pollute 
stdout with their logs.


Basicly, logging isn't "output".

Cheers,
Cameron Simpson 
--
https://mail.python.org/mailman/listinfo/python-list

Re: What should go to stdout/stderr and why Python logging write everything to stderr?

[c . buhtz]

Am 03.01.2023 17:51 schrieb r...@zedat.fu-berlin.de:

logging.getLogger().addHandler( logging.StreamHandler( sys.stdout ))


But I don't want to make all log levels go to stdout. Just DEBUG and 
INFO. But this would be a workaround.


The main question here is why does Python deciecded to make all logs go 
to stderr?
Maybe I totally misunderstood the intention of logging.info()?! Isn't 
this the "usual applicaton output"?


If not, what messages should go into logging.info()? Can you name me 
some examples?

--
https://mail.python.org/mailman/listinfo/python-list

Re: What should go to stdout/stderr and why Python logging write everything to stderr?

[Eryk Sun]

On 1/3/23, Weatherby,Gerard  wrote:
> If sys.stdout is a tty, it typically flushes on newline. e. g.

Sorry if I wasn't clear. Yes, a tty file will be buffered, but it's
line buffering, which isn't an issue as long as lines are written to
stdout. I was referring to full buffering of pipe and disk files. This
can be a problem when printing diagnostics. We went the information
written to the file immediately, not whenever the buffer happens to
fill up. Thus sys.stderr is unbuffered.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: What should go to stdout/stderr and why Python logging write everything to stderr?

[Weatherby,Gerard]

If sys.stdout is a tty, it typically flushes on newline. e. g.

!/usr/bin/env python3
import time
import sys
print("No flush",end='',file=sys.stdout)
time.sleep(2)
print("flushed",file=sys.stdout)
time.sleep(5)

will print the “flushed” 5 seconds before the script ends

From: Python-list  on 
behalf of Eryk Sun 
Date: Tuesday, January 3, 2023 at 1:33 PM
To: c.bu...@posteo.jp 
Cc: python-list@python.org 
Subject: Re: What should go to stdout/stderr and why Python logging write 
everything to stderr?
*** Attention: This is an external email. Use caution responding, opening 
attachments or clicking on links. ***

On 1/3/23, c.bu...@posteo.jp  wrote:
>
> If the user request the usage info via "-h" it goes to stdout.

The standard file for application output is sys.stdout. Note that
sys.stdout may be buffered, particularly if it isn't a tty. When a
file is buffered, writes are aggregated and only written to the OS
file when the buffer fills up or is manually flushed.

> Why does logging behave different? DEBUG and INFO imho should go to
> stdout not stderr.

The standard file for error messages and other diagnostic information
is sys.stderr. This file should never be buffered.
--
https://urldefense.com/v3/__https://mail.python.org/mailman/listinfo/python-list__;!!Cn_UX_p3!jGBVNrRuUThZCKMmShbuvBgggwv7FBDL9h2vW-vvehPnBHdfkrJUhohhZCgsCAqlRrDluk9c526jABrLjg$<https://urldefense.com/v3/__https:/mail.python.org/mailman/listinfo/python-list__;!!Cn_UX_p3!jGBVNrRuUThZCKMmShbuvBgggwv7FBDL9h2vW-vvehPnBHdfkrJUhohhZCgsCAqlRrDluk9c526jABrLjg$>
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: What should go to stdout/stderr and why Python logging write everything to stderr?

[Thomas Passin]

On 1/3/2023 11:51 AM, Stefan Ram wrote:

Thomas Passin  writes:

On 1/3/2023 10:35 AM, c.bu...@posteo.jp wrote:
Also, I think it would be confusing to sometimes have logging output go
to stdout and other times go to stderr.


   In UNIX, the output of a program can be redirected,
   so error messages written to the output might get appended
   to some result file and might not be seen by the user.


Yes, and in Windows, too.  Note that stderr messages can be written or 
redirected to a file, too, and then would not be seen on the terminal.



   Therefore, UNIX and C provide the unbuffered "stderr" stream
   that can be redirected separately.

   The following example shows how logging might write to stdout.

import logging
import sys
logging.getLogger().addHandler( logging.StreamHandler( sys.stdout ))
logging.error( 'example' )


This is starting to look too much like java.  Yuck!

--
https://mail.python.org/mailman/listinfo/python-list