Re: shutil.rmtree() fails when used in Fedora (rpm) "mock" environment

[Christian Buhtz via Python-list]

As you can see in the linked issue it seems it was an incompatibility
between the version of Python and PyFakeFS.

In the end it was a Fedora packaging bug because that pyfakefs version
was not compatible with Python 3.13.

Thanks in advance for helping out.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: shutil.rmtree() fails when used in Fedora (rpm) "mock" environment

[Christian Buhtz via Python-list]

Am 25.10.2024 09:06 schrieb Christian Buhtz via Python-list:

On a "regular" system all tests are running.


To clarify: "regular" does not exclude PyFakeFS. It means on my own 
local development machine and on the TravsCI machines (Ubuntu 22 with 
Python 3.9 up to 3.13) and using PyFakeFS in that test, everything is 
fine.

Only when mock/chroot is involved that happens.
--
https://mail.python.org/mailman/listinfo/python-list

Re: shutil.rmtree() fails when used in Fedora (rpm) "mock" environment

[Christian Buhtz via Python-list]

Hello Barry,

thank you for your reply and clarifying the Fedora aspects.

Am 25.10.2024 00:44 schrieb Barry:

What do you mean by the real file sustem?

You cannot write to the /usr file system. Is that what your tests do?
If so that needs changing.


Asking the right questions brings up to important details. While I was 
writing and trying to explain that the relevant test does use 
""tempfile.TemporaryDirectory" as a context, I realized that "PyFakeFS" 
is used in the back [1].


But that makes me wonder. On a "regular" system all tests are running. 
So the issue might exist because of a combination of 3 factors: 
shutil.rmtree(), PyFakeFS in a chroot/mock build environment.


[1] -- 


--
https://mail.python.org/mailman/listinfo/python-list

Re: shutil.rmtree() fails when used in Fedora (rpm) "mock" environment

[Christian Buhtz via Python-list]

Thank you very much for all your responses.

Am 24.10.2024 17:17 schrieb Left Right:

To investigate this, I'd edit the file with the assertion and make it
print the actual value found in os.lstat and func.  My guess is that
they are both somehow "lstat", but with different memory addresses.


My reporter provided this [1]. I think this is the relevant output:

=== short test summary info 

FAILED test/test_plugin_usercallback.py::SystemTest::test_local_snapshot 
- As...
FAILED test/test_uniquenessset.py::General::test_ctor_defaults - 
AssertionError
FAILED test/test_uniquenessset.py::General::test_deep_check - 
AssertionError
FAILED 
test/test_uniquenessset.py::General::test_fail_equal_without_equal_to
FAILED test/test_uniquenessset.py::General::test_size_mtime - 
AssertionError
FAILED test/test_uniquenessset.py::General::test_unique_myself - 
AssertionError
FAILED 
test/test_uniquenessset.py::General::test_unique_size_but_different_mtime
== 7 failed, 267 passed, 16 skipped in 20.79s 
==

os.lstat=
os.lstat=
os.lstat=
os.lstat=
os.lstat=
os.lstat=
os.lstat=
os.lstat=
os.lstat=
os.lstat=
os.lstat=
os.lstat=
os.lstat=
os.lstat=
os.lstat=
os.lstat=
os.lstat=
os.lstat=
os.lstat=
os.lstat=
os.lstat=
os.lstat=
os.lstat=
make: Leaving directory 
'/home/johannes/rpmbuild/BUILD/backintime-1.5.3-build/backintime-1.5.3-rc1/common'


RPM build errors:



[1] -- 





--
https://mail.python.org/mailman/listinfo/python-list

shutil.rmtree() fails when used in Fedora (rpm) "mock" environment

[Christian Buhtz via Python-list]

Hello,
I am upstream maintainer of "Back In Time" [1] investigating an issue a 
distro maintainer from Fedora reported [2] to me.


On one hand Fedora seems to use a tool called "mock" to build packages 
in a chroot environment.
On the other hand the test suite of "Back In Time" does read and write 
to the real file system.
One test fails because a temporary directory is cleaned up using 
shutil.rmtree(). Please see the output below.


I am not familiar with Fedora and "mock". So I am not able to reproduce 
this on my own.
It seems the Fedora maintainer also has no clue how to solve it or why 
it happens.


Can you please have a look (especially at the line "assert func is 
os.lstat").
Maybe you have an idea what is the intention behind this error raised by 
an "assert" statement inside "shutil.rmtree()".


Thanks in advance,
Christian Buhtz

[1] -- <https://github.com/bit-team/backintime>
[2] -- <https://github.com/bit-team/backintime/issues/1911>

__ General.test_ctor_defaults 
__

self = 
def test_ctor_defaults(self):
"""Default values in constructor."""

  with TemporaryDirectory(prefix='bit.') as temp_name:

test/test_uniquenessset.py:47:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
_ _ _ _

/usr/lib64/python3.13/tempfile.py:946: in __exit__
self.cleanup()
/usr/lib64/python3.13/tempfile.py:950: in cleanup
self._rmtree(self.name, ignore_errors=self._ignore_cleanup_errors)
/usr/lib64/python3.13/tempfile.py:930: in _rmtree
_shutil.rmtree(name, onexc=onexc)
/usr/lib64/python3.13/shutil.py:763: in rmtree
_rmtree_safe_fd(stack, onexc)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
_ _ _ _

stack = []
onexc = .onexc at 
0xb39bc860>

def _rmtree_safe_fd(stack, onexc):
# Each stack item has four elements:
# * func: The first operation to perform: os.lstat, os.close or 
os.rmdir.
#   Walking a directory starts with an os.lstat() to detect 
symlinks; in
#   this case, func is updated before subsequent operations and 
passed to

#   onexc() if an error occurs.
# * dirfd: Open file descriptor, or None if we're processing the 
top-level
#   directory given to rmtree() and the user didn't supply 
dir_fd.
# * path: Path of file to operate upon. This is passed to 
onexc() if an

#   error occurs.
# * orig_entry: os.DirEntry, or None if we're processing the 
top-level
#   directory given to rmtree(). We used the cached stat() of 
the entry to

#   save a call to os.lstat() when walking subdirectories.
func, dirfd, path, orig_entry = stack.pop()
name = path if orig_entry is None else orig_entry.name
try:
if func is os.close:
os.close(dirfd)
return
if func is os.rmdir:
os.rmdir(name, dir_fd=dirfd)
return

# Note: To guard against symlink races, we use the standard
# lstat()/open()/fstat() trick.

  assert func is os.lstat

E   AssertionError
/usr/lib64/python3.13/shutil.py:663: AssertionError

--
https://mail.python.org/mailman/listinfo/python-list

Re: Python 3.12.1, Windows 11: shebang line #!/usr/bin/env python3 doesn't work any more

[Christian Buhtz via Python-list]

What is the "command line" on your Windows 11?

On Windows 10 it usually is "cmd.exe" (Windows Command Prompt).
On Windows 11 it usually is the "Terminal" which is different from 
cmd.exe.

--
https://mail.python.org/mailman/listinfo/python-list

Request to Review: Tutorial about Python Packaging offering different use case

[Christian Buhtz via Python-list]

Hello,

I would like to point to my Python Packaging Tutorial explaining several 
common use cases using minimal demo projects.


<https://codeberg.org/buhtz/tech-demo-python-packaging>

I am not an expert and assume that some of my solutions might not be the 
best. So I would appreciate if you can review them.


Background of that project: I am member of FOSS maintainer team and 
prepare the migration of Back In Time 
(https://github.com/bit-team/backintime) from a makefile based 
build-system to a modern Python Build system using pyproject.toml & Co. 
To explore some expect able problems and possible solutions I created 
this minimal examples. I also do plan a tutorial repo about Debian 
Python Packaging using the same approach with minimal examples 
illustrating different use cases.


Thanks in advance,
Christian
--
https://mail.python.org/mailman/listinfo/python-list

Re: Detect naming typos (AttributeError) in function names

[Christian Buhtz via Python-list]

Hello Dieter,

thanks for your reply.

Am 06.11.2023 19:11 schrieb Dieter Maurer:

One option is a test suite (--> Python's "unittest" package)
with a sufficiently high coverage (near 100 %).


Yes, that is the primary goal. But it is far away in the related 
project.


I got a hint that "pylint" is able to detect problems like this.
--
https://mail.python.org/mailman/listinfo/python-list

Detect naming typos (AttributeError) in function names

[Christian Buhtz via Python-list]

Hello,

I would like to know how to detect (e.g. via a linter) typos in function 
names imported from another module.


Let's assume this given Python code snippet.

import foo
foo.baR()

The package "foo" do contain a function named "bar()" (all lower case 
letters). The function "baR()" does not exist in "foo". This cause an 
AttributeError when run with a Python interpreter.


The described error is not detected in my IDE (Emacs with eglot, pylsp 
and flake8) and not by flake8 on the shell. Because the involved tools 
do not look inside the "foo" package if "baR()" really exist.


Can I fix this somehow?

I am aware that this would get detected by a unit test. That is the way 
I do prefer in most cases. But sometimes not all code segments are 
covered by tests. I'm more interested in using a linter or something 
else for that.


Kind
Christian
--
https://mail.python.org/mailman/listinfo/python-list

Re: unable to run the basic Embedded Python example

[Christian Gollwitzer via Python-list]

Hi Dave,

I can tell you where the error comes from, but I don't know how to fix 
it correctly:


Am 24.06.23 um 19:29 schrieb Dave Ohlsson:

9. And now, when I ran embedded_python.exe:

20:14:06: Starting
C:\temp\build-embedded_python-Desktop_Qt_6_1_3_MSVC2019_64bit-Debug\debug\embedded_python.exe...
Could not find platform independent libraries 
Python path configuration:
   PYTHONHOME = (not set)
   PYTHONPATH = (not set)
   program name =



Python consists of the DLL that you have lined it with plus a large 
number of files (standard library) which contain, e.g., the encoding 
data and builtin functions etc. It needs to find these in order to run. 
You could set PYTHONHOME as an environment variable to point to that 
folder. If you don't use the regular Pyton installer, then you need to 
install these files yourself. Maybe there is also a way to set it from 
the C code.


    Christian
--
https://mail.python.org/mailman/listinfo/python-list

Re: Tkinter docs?

[Christian Gollwitzer]

Am 24.05.23 um 03:18 schrieb Rob Cliffe:
I have recently started converting a large project to tkinter, starting 
with zero knowledge of tkinter.  (You are free to think: BAD IDEA. 😁)


Welcome to the awesome world of GUI development.

     I was writing a subclass of the Checkbutton class (tkinter's name 
for what I call a checkbox).  I needed to be able to (1) set (2) clear 
(3) interrogate the checked state.  This is easy to do if you associate 
a "variable" with each Checkbutton instance, as seems to be usual 
tkinter practice.  ("variable" is not a variable in the usual sense, but 
an object provided by tkinter that is linked to a GUI object (such as a 
Checkbutton), so that when the GUI object changes, the value of the 
"variable" changes, and vice versa.) However, I decided that I wanted to 
dispense with the variable, if possible.  


As you found out the hard way, it is possible, but then you dive into 
the internals of how the widgets work - usually you don't want to know 
that. Also, this bit differs between Tk and Ttk widgets.




[...] lengthe description of Checkbutton internals



In GUI programming, you will meet a bag of concepts that you haven't 
seen in sequential programming before. To make it worse, every GUI 
toolkit brings its own new set of concepts to the table. Maybe it is 
helpful to work through a Tk tutorial first. This is a good one:


http://tkdocs.com/tutorial/index.html

Similarly to the tutorial I would suggest to stick with the ttk widgets. 
Those are an "update" (> 10 years ago) to the tk widgets and are 
supposed to provide sensible defaults matching the users experience with 
native GUI elements. There are only 3 widgets where you must use a Tk 
widget, this is Text (for multiline formatted text entry), Canvas (for 
2D drawings), and Toplevel (for new windows/ popups etc.)


Christian

--
https://mail.python.org/mailman/listinfo/python-list

Re: How to add clickable url links to 3D Matplotlib chart ?

[Christian Gollwitzer]

Am 30.03.23 um 01:11 schrieb a a:

On Wednesday, 29 March 2023 at 22:51:15 UTC+2, Greg Ewing wrote:

On 30/03/23 8:39 am, a a wrote:

How to add clickable url links to the following 3D Matplotlib chart to make it 
knowledge representation 3D chart, make of 1,000+ open Tabs in Firefox ?

It seems that matplotlib can be made to generate SVG images with
hyperlinks in them:

https://matplotlib.org/stable/gallery/misc/hyperlinks_sgskip.html

--
Greg

thank you
but I need mouse hover-on, mouse click events to be handled by a simple 
algorithm to calculate which ball/circle has been selected (in the Matplotlib 
3D animated chart above) to make the selected ball to flash and have label 
opened made of url icon, name of url, followed by url (exactly what Firefox 
makes with Tabs)

For knowledge representation, 1,000+  Tabs open in Firefox,  earthquakes 3D 
live chart by Giuseppe is a nice tool.

One axis can represent time (timeline), two other axis can represent features 
attributed to to opened Tabs, like frequency of visits, ranking.

Ok, balls should overlayed with a respective url icon, as done in Firefox (Tabs 
row) : url icon + label's name abbreviated



It doesn't sound as if there is a "one-line" solution to this problem. 
It sounds more like you want a video game engine to interact with a 3D 
world.


There used to be a 3D version of HTML, called VRML, with the successor 
of X3D that could show such a thing in the browser, but I doubt that 
there is easy support for it any more in recent browsers. Therefore it 
would be difficult to post this to the internet, unless you invest in 
some JS programming. In case you want to run this on your local 
computer, as opposed to in the browser, you can check out Python game 
engines.


Christian
--
https://mail.python.org/mailman/listinfo/python-list

Re: Cutting slices

[Christian Gollwitzer]

Am 05.03.23 um 23:43 schrieb Stefan Ram:

   The following behaviour of Python strikes me as being a bit
   "irregular". A user tries to chop of sections from a string,
   but does not use "split" because the separator might become
   more complicated so that a regular expression will be required
   to find it. 


OK, so if you want to use an RE for splitting, can you not use 
re.split() ? It basically works like the built-in splitting in AWK


>>> s='alphaAbetaBgamma'
>>> import re
>>> re.split(r'A|B|C', s)
['alpha', 'beta', 'gamma']
>>>


Christian
--
https://mail.python.org/mailman/listinfo/python-list

Re: Evaluation of variable as f-string

[Christian Gollwitzer]

Am 28.01.23 um 02:56 schrieb Thomas Passin:

On 1/27/2023 5:10 PM, Christian Gollwitzer wrote:

Am 27.01.23 um 21:43 schrieb Johannes Bauer:
I don't understand why you fully ignore literally the FIRST example I 
gave in my original post and angrily claim that you solution works 
when it does not:


x = { "y": "z" }
s = "-> {x['y']}"
print(s.format(x = x))
Traceback (most recent call last):
   File "", line 1, in 
KeyError: "'y'"

This. Does. Not. Work.


It's because "you're holding it wrong!". Notice the error message; it 
says that the key 'y' does not exist.



(base) Apfelkiste:Abschlussmeeting chris$ ipython
Python 3.8.8 (default, Apr 13 2021, 12:59:45)
Type 'copyright', 'credits' or 'license' for more information
IPython 7.22.0 -- An enhanced Interactive Python. Type '?' for help.

In [1]: x = { "y": "z" }

In [2]: s = "-> {x[y]}"

In [3]: print(s.format(x = x))
-> z

In [4]:

 Christian


Oops, that's not quite what he wrote.

You: s = "-> {x[y]}"    # Works
Him: s = "-> {x['y']}"  # Fails

You might want to reconsider why I could have possibly written this 
message


  Christian
--
https://mail.python.org/mailman/listinfo/python-list

Re: Evaluation of variable as f-string

[Christian Gollwitzer]

Am 27.01.23 um 21:43 schrieb Johannes Bauer:
I don't understand why you fully ignore literally the FIRST example I 
gave in my original post and angrily claim that you solution works when 
it does not:


x = { "y": "z" }
s = "-> {x['y']}"
print(s.format(x = x))
Traceback (most recent call last):
   File "", line 1, in 
KeyError: "'y'"

This. Does. Not. Work.


It's because "you're holding it wrong!". Notice the error message; it 
says that the key 'y' does not exist.



(base) Apfelkiste:Abschlussmeeting chris$ ipython
Python 3.8.8 (default, Apr 13 2021, 12:59:45)
Type 'copyright', 'credits' or 'license' for more information
IPython 7.22.0 -- An enhanced Interactive Python. Type '?' for help.

In [1]: x = { "y": "z" }

In [2]: s = "-> {x[y]}"

In [3]: print(s.format(x = x))
-> z

In [4]:

Christian
--
https://mail.python.org/mailman/listinfo/python-list

Re: How to turn this Matlab code into python

[Christian Gollwitzer]

Am 08.11.22 um 09:01 schrieb Dioumacor FAYE:

From datenum to data_mjja=precp(:,ind_mjja);


I think the code filters data based on calendar dates. I'd guess that 
you can do that most easily with pandas.


https://pandas.pydata.org/docs/user_guide/timeseries.html

    Christian


Le lun. 7 nov. 2022 à 21:42, Christian Gollwitzer  a
écrit :


Am 07.11.22 um 18:12 schrieb Dioumacor FAYE:

hi everyone
I wanted to transform this matlab code into python. If anyone can help me
please let me know.
load /media/lpaosf/Dioumss/maman/data/precip_chirps_SEN_1981-2018.mat
prcp = reshape(precip,[140*100,13879]);
dates = datenum(1981,1,1):datenum(2018,12,31);
date = datevec(dates);
ind_mjja=find(date(:,2)>=5 & date(:,2)<=8);% May to Aug (May(31)+

June(30)+

July(31)+ August(31)= 123 days)
data_mjja=precp(:,ind_mjja);
data_mjja_res=reshape(data_mjja,14000,123,38);


which part do you have trouble with?


     Christian
--
https://mail.python.org/mailman/listinfo/python-list






--
https://mail.python.org/mailman/listinfo/python-list

Re: How to turn this Matlab code into python

[Christian Gollwitzer]

Am 07.11.22 um 18:12 schrieb Dioumacor FAYE:

hi everyone
I wanted to transform this matlab code into python. If anyone can help me
please let me know.
load /media/lpaosf/Dioumss/maman/data/precip_chirps_SEN_1981-2018.mat
prcp = reshape(precip,[140*100,13879]);
dates = datenum(1981,1,1):datenum(2018,12,31);
date = datevec(dates);
ind_mjja=find(date(:,2)>=5 & date(:,2)<=8);% May to Aug (May(31)+ June(30)+
July(31)+ August(31)= 123 days)
data_mjja=precp(:,ind_mjja);
data_mjja_res=reshape(data_mjja,14000,123,38);


which part do you have trouble with?


    Christian
--
https://mail.python.org/mailman/listinfo/python-list

Re: How to manage python shebang on mixed systems?

[Christian Gollwitzer]

Am 07.11.22 um 10:28 schrieb Chris Green:

Chris Green  wrote:

3: with your pseudo "python3" script in place, make all the scripts use
the "#!/usr/bin/env python3" shebang suggested above.


Yes, that sounds a good plan to me, thanks Cameron.


Doesn't '#!/usr/bin/env python3' suffer from the same problem as
'#!/usr/bin/python3' in the sense that the env executable might not be
in /usr/bin?

Wouldn't '#! env python3' be better?


Does this even work? I thought that, since the #! is resolved by the 
kernel, an absolute path is required, and that the whole purpose of the 
/usr/bin/env thing is to search the path (because it is universally 
installed in /usr/bin) and to possibly set other environment variables




Christian
--
https://mail.python.org/mailman/listinfo/python-list

Re: setup.py + cython == chicken and the egg problem

[Christian Gollwitzer]

Am 16.08.22 um 23:03 schrieb Dan Stromberg:

I'm attempting to package up a python package that uses Cython.

Rather than build binaries for everything under the sun, I've been focusing
on including the .pyx file and running cython on it at install time.  This
requires a C compiler, but I'm OK with that.

However, when I try to install the package from test.pypi.org, I get:
$ python3 -m pip install -i https://test.pypi.org/simple/ pyx-treap
below cmd output started 2022 Tue Aug 16 01:55:16 PM PDT
Looking in indexes: https://test.pypi.org/simple/
Collecting pyx-treap
   Downloading
https://test-files.pythonhosted.org/packages/3a/41/af5360934adccfc086a39e1f720323895144b53454ff6dacc0f06267db55/pyx_treap-2.0.15.tar.gz
(125 kB)

  

125.9/125.9 kB 1.9 MB/s eta 0:00:00
   Installing build dependencies ... error
   error: subprocess-exited-with-error

   ×? pip subprocess to install build dependencies did not run successfully.
   ?? exit code: 1
   > [3 lines of output]
   Looking in indexes: https://test.pypi.org/simple/
   ERROR: Could not find a version that satisfies the requirement
setuptools (from versions: none)
   ERROR: No matching distribution found for setuptools
   [end of output]

   note: This error originates from a subprocess, and is likely not a
problem with pip.
error: subprocess-exited-with-error



I looked at your code and I think you are trying too hard. As far as I 
understand, you need Cython to be installed before the build process 
begins. Your entry in pyproject.toml should take care of that.

But you also have these lines in your setup.py

subprocess.check_call('%s -m pip install cython' % (sys.executable, ), 
shell=True)
subprocess.check_call('%s -m cython pyx_treap.pyx' % (sys.executable, ), 
shell=True)


The first one calls out to pip while pip is already running, I'm not 
sure that this will work, but judging from the error message it is 
looking for the requirements also from test.pypi. Maybe this is the 
reason that it fails (the error message says that it can't find 
setuptools). So jut delete this line and it might already work


The second line, which compiles the Cython code, also runs *at every 
invocation of setup.py*, even if you'd do just


python3 setup.py --help

It may still work, but the correct way to do it is to create a build 
extension for setuptools. In my project you can see this here:


https://github.com/j-from-b/CDEF/blob/main/setup.py#L88

OTOH, I would be surprised if Cython did not have this already, indeed 
you imported cythonize from Cython.Build. So maybe just deleting these 
two lines and it might work?


Christian
--
https://mail.python.org/mailman/listinfo/python-list

Re: Which linux distro is more conducive for learning the Python programming language?

[Christian Heimes]

On 04/08/2022 20.12, Barry wrote:

Noted with thanks Kushal. Since I can download FREE copies of RHEL
9.0, I will use it then.


I consider rhel 9 is an old os. I would suggest using fedora over rhel.
Fedora 36 has python 3.10 and the when fedora 37 is released it will have 
python 3.11.
And fedora is free as well.


Fedora is an excellent choice for Python users. Fedora 36 already comes 
with Python 3.11.0b5 in its main repository. In fact you have Python 
2.7, 3.5-3.11, PyPy 2.7, PyPy 3.7-3.9, and MicroPython at your fingertips.


Christian
--
https://mail.python.org/mailman/listinfo/python-list

Re: "CPython"

[Christian Gollwitzer]

Am 20.06.22 um 22:47 schrieb Roel Schroeven:
indication that www.analyticsinsight.net is wrong on that point. Frankly 
that website seems very low quality in general. In that same article 
they say:


"CPython is a descendant of Pyscript built on Pyodide, a port of 
CPython, or a Python distribution for the browser and Node.js that is 
based on Webassembly and Emscripten."


CPython is definitely not a descendant of Pyscript! Looks like someone 
found something (semi-) interesting and tried to write something 
insightful about it, but without really understanding any of it. Other 
articles don't seem to be any better.


To me, this sentence is so badly cobbled together that it could be the 
output of a KI of some sort (GPT-3) trying to summarize stuff from the 
web. It doesn't make any sense at all on a semantic level.


Christian
--
https://mail.python.org/mailman/listinfo/python-list

Re: How to test characters of a string

[Christian Gollwitzer]

Am 08.06.22 um 19:57 schrieb De ongekruisigde:

On 2022-06-08, 2qdxy4rzwzuui...@potatochowder.com 
<2qdxy4rzwzuui...@potatochowder.com> wrote:

On 2022-06-09 at 04:15:46 +1000,
Chris Angelico  wrote:

If you insist:

 >>> s = 'nm-iodine:x:996:57::/var/empty:/run/current-system/sw/bin/nologin'
 >>> print(s.split(':'))
 ['nm-iodine', 'x', '996', '57', '', '/var/empty', 
'/run/current-system/sw/bin/nologin']

Hesitantly, because this is the Python mailing list, I claim (a) ':' is
simpler than r'([^:]*):([^:]*):(\d+):(\d+):([^:]*):([^:]*):(.*)$', and
(b) string.split covers pretty much the same edge cases as re.search.


Ah, but you don't catch the be numeric of fields (0-based) 2 and 3! But
agreed, it's not the best of examples.



Yes, that is a simplistic example, since the : can't even be quoted to 
appear in that format (which would require higher-order parsing than a 
simple split)


Fortunately, the OP has just given another requirement to recognise 
different patterns, and now it went into a direction where it will 
become quite ugly if you avoid REs or any other pattern matching tools.


This is also the main reason I suggested REs initially - often if you 
see other patterns in the data, you can easily adapt a RE solution, 
whereas you'll have to write the thing from ground up anew if you do it 
manually.


Christian

--
https://mail.python.org/mailman/listinfo/python-list

Re: How to test characters of a string

[Christian Gollwitzer]

Am 09.06.22 um 07:50 schrieb Dave:

Hi,

I’ve found you also need to take care of multiple disk CD releases. These have 
a format of

“1-01 Track Name”
“2-02  Trackl Name"

Meaning Disk 1 Track1, Disk 2, Track 2.

Also A and B Sides (from Vinyl LPs)

“A1-Track Name”
“B2-Track Name”

Side A, Track 1, etc.



Now you're getting into the complexity that is better handled by REs 
than by individual character examination.

The first of your formats matches the RE

\d-\d{2}

(one digit, - two digits). Anchor that to check for a match at the 
beginning.


The second one matches

(A|B)\d-

As long as one digit is enough. What is your goal, to extract these 
numbers or to strip them? Regexes can do both relatively easily.


    Christian
--
https://mail.python.org/mailman/listinfo/python-list

Re: How to test characters of a string

[Christian Gollwitzer]

Am 07.06.22 um 23:01 schrieb Christian Gollwitzer:


In [3]: re.sub(r'^\d+\s*', '', s) Out[3]: 'Trinket'



that RE does match what you intended to do, but not exactly what you 
wrote in the OP. that would be '^\d\d.'  start with exactly two digits 
followed by any character.


Christian
--
https://mail.python.org/mailman/listinfo/python-list

Re: How to test characters of a string

[Christian Gollwitzer]

Am 07.06.22 um 21:56 schrieb Dave:

It depends on the language I’m using, in Objective C, I’d use isNumeric, just 
wanted to know what the equivalent is in Python.



Your problem is also a typical case for regular expressions. You can 
create an expression for "starts with any number of digits plus optional 
whitespace" and then replace this with nothing:



chris@linux-tb9f:~> ipython
Python 3.6.15 (default, Sep 23 2021, 15:41:43) [GCC]
Type 'copyright', 'credits' or 'license' for more information
IPython 7.13.0 -- An enhanced Interactive Python. Type '?' for help.

In [1]: import re

In [2]: s='05 Trinket'   

In [3]: re.sub(r'^\d+\s*', '', s)
Out[3]: 'Trinket'




If it doesn't match, it will do nothing:

In [4]: s='Es geht los'  

In [5]: re.sub(r'^\d+\s*', '', s)
Out[5]: 'Es geht los'


Some people on this list don't like regexes but for tasks like this they 
are made and working well.


^ is "starts with"
\d is any digit
\s is any space
+ is at least one
* is nothing or one of

Christian




--
https://mail.python.org/mailman/listinfo/python-list

Re: Automatic Gain Control in Python?

[Christian Gollwitzer]

Am 29.05.22 um 00:45 schrieb Stefan Ram:

"Steve GS"  writes:

Subject: Automatic Gain Control in Python?


   Automatic Gain Control in Python is trivial. You have a list
   of samples and normalize them, i.e., divide by max. Slightly
   simplified

[ s/max( samples )for s in samples ]

   (where sample values are normalized to the range -1,+1.)


No, that's not it. Loudness is perceived in a different way, the crudest 
approximation is the standard deviation averaged over small frames, 
better measures take into account that the ear has a strongly frequency 
dependent response. For similar sound files, like podcasts which are 
voice-heavy, the stddev works reasoably well.


If the normalization only reduces the volume, then dividing by the max 
is sufficient. However if you also want to raise the volume, then you 
need dynamic range compression. If you have ever tried to record some 
music with a simple microphone and a computer, you would have noticed 
that the recording is very soft, when normalized to the max. Commercial 
music is incredibly loud, and you might have wondered, how they do that.


Google for "Loudness war" and "dynamic range compression" if you want to 
understand it in detail.


Christian


--
https://mail.python.org/mailman/listinfo/python-list

Re: Issue sending data from C++ to Python

[Christian Gollwitzer]

Am 18.05.22 um 16:08 schrieb Pablo Martinez Ulloa:

I have been using your C++ Python API, in order to establish a bridge from
C++ to Python. We want to do this, as we have a tactile sensor, which only
has a library developed in C++, but we want to obtain the data in real time
in Python to perform tests with a robotic arm and gripper. The problem we
are facing is with transmitting the data into Python. We are using the
function Py_BuildValue, and it seems to be working, but only for sending
one value at a time, and we want to transmit 54 numbers


The usual way to pass an array of numbers into Python is by means of a 
Numpy Array. In order to do that, you need to include arrayobject.h and 
then use PyArray_SimpleNew to create an array of numbers as a Python 
object which you can return 
https://numpy.org/devdocs/reference/c-api/array.html#c.PyArray_SimpleNew



 Christian
--
https://mail.python.org/mailman/listinfo/python-list

Re: No shortcut Icon on Desktop

[Christian Gollwitzer]

Am 15.04.22 um 02:49 schrieb Mats Wichmann:

On 4/14/22 18:06, Grant Edwards wrote:

On 2022-04-14, Richard Damon  wrote:


I think the issue is that the 'python' interpreter/compiler isn't the
sort of program that makes sense to make a desktop icon for, as it is a
command line utility.


Yes, it is a command line utility. Why does that mean you shouldn't
have a desktop shortcut for it?

I start up a python REPL prompt in a terminal often enough that were I
a windows users, I would probably want a desktop shortcut for it.

It would at least let people know that something got installed and
show them what a Python is. If they don't want/use that shortcut, it's
trivial to delete it.


easy to add - it's a windows thing, not a python thing.  you can
navigate to the install directory and create a shortcut and drag that
out of that directiory in explorer and drop it on the desktop.  or you
can navigate through the start menu, and when you get to the thing you
want, pick open folder and then you can create a shortcut and drag off
to the desktop.


Yes, you *can* do that of course and it is not a Python thing - but the 
point is, that typical Windows installers create these shortcuts during 
the installation process for you - typically there is a pre-selected 
checkbox "Create desktop icons" or similar. I agree with Grant that this 
is what users expect from the installer.


Christian

--
https://mail.python.org/mailman/listinfo/python-list

Re: Comparing sequences with range objects

[Christian Gollwitzer]

Am 08.04.22 um 09:21 schrieb Antoon Pardon:

The first is really hard. Not only may information be missing, no single
single piece of information is unique or immutable. Two people may have
the same name (I know about several other "Peter Holzer"s), a single
person might change their name (when I was younger I went by my middle
name - how would you know that "Peter Holzer" and "Hansi Holzer" are the
same person?), they will move (= change their address), change jobs,
etc. Unless you have a unique immutable identifier that's enforced by
some authority (like a social security number[1]), I don't think there
is a chance to do that reliably in a program (although with enough data,
a heuristic may be good enough).


Yes I know all that. That is why I keep a bucket of possible duplicates
per "identifying" field that is examined and use some heuristics at the
end of all the comparing instead of starting to weed out the duplicates
at the moment something differs.

The problem is, that when an identifying field is judged to be unusable,
the bucket to be associated with it should conceptually contain all other
records (which in this case are the indexes into the population list).
But that will eat a lot of memory. So I want some object that behaves as
if it is a (immutable) list of all these indexes without actually 
containing

them. A range object almost works, with the only problem it is not
comparable with a list.



Then write your own comparator function?

Also, if the only case where this actually works is the index of all 
other records, then a simple boolean flag "all" vs. "these items in the 
index list" would suffice - doesn't it?


Christian

--
https://mail.python.org/mailman/listinfo/python-list

Re: Exchange OWA using Python?

[Christian Gollwitzer]

Am 01.04.22 um 01:26 schrieb Grant Edwards:

On 2022-03-31, Christian Gollwitzer  wrote:

Davmail is written in Java, not Python, but basically this should
not matter if you only use it.


Have you used it with OWA as the protocol?



At least I thought so - this was in 2016 - 2017 and there was external 
webmail access allowed to our mail server. I've used davmail to connect 
to it. I vaguely remember that there was one day a sudden change, 
webmail was disabled and davmail then connected using EWS. Maybe the OWA 
protocol also evolved, I now see it on the roadmap.


Good that you found a solution now with owl!

I'm having a similar issue at my current position, we're using Lotus 
Notes (yes! The package from the 90s, it's outright horrible). There is 
only one thing available, the iNotes exporter, which allows to read the 
mail using thunderbird, sending is not implemented. Always keeping 
fingers crossed that the IT department does not update the server 
protocol. Sometimes they do minor changes like internal redirection URL 
changes, which costs me half a day to fix then.


Christian
--
https://mail.python.org/mailman/listinfo/python-list

Re: Exchange OWA using Python?

[Christian Gollwitzer]

Am 31.03.22 um 16:41 schrieb Grant Edwards:

Is anybody aware of any Python code for the Exchange OWA protocol/API?

The OWA e-mail client I've been using has stopped working. It was a
commerical Python application named Hiri, wich has been abandoned by
the developer.

So, for now, I'm stuck with the OWA web client. It's clumsy and
everything takes 3X as long to accomplish as it would with a real
e-mail client, but I can survive with it if I have to. 


I can recommend davmail as a workaround:

http://davmail.sourceforge.net/

Davmail converts OWA into regular mail protocols like POP3, IMAP, SMTP. 
You can use any email/calender client (I've used thunderbird) and most 
of it worked well.


Davmail is written in Java, not Python, but basically this should not 
matter if you only use it.


Christian
--
https://mail.python.org/mailman/listinfo/python-list

Re: Set tkinter top-level window to "always on visible workspace"

[Christian Gollwitzer]

Am 28.03.22 um 20:03 schrieb Chris Angelico:

Would you accept a solution that involves a subprocess call?

wmctrl -ir {id} -b add,sticky

Now, the only problem is... figuring out your window ID. Worst case,
parse wmctrl -lG to get that info, but it might be possible to get the
window ID from Tkinter itself.


Sure: Call "winfo_id()" on the toplevel. You might want to reformat in 
it in hex format, which is the usual way to pass these IDs around. Tk 
actually returns it in hex format, but Tkinter reformats it as an integer.


    Christian
--
https://mail.python.org/mailman/listinfo/python-list

Re: Set tkinter top-level window to "always on visible workspace"

[Christian Gollwitzer]

Am 28.03.22 um 00:55 schrieb Skip Montanaro:

So you might tell your window manager to keep that window on the main

workspace.

Thanks. I'd forgotten about the possibility of doing this sort of thing in
the window manager config. That would certainly be fine in this case. (It's
been ages since I messed with this sort of thing.)


I might be misguided, but on modern desktops that should be possible 
with a few mouseclicks. E.g. in KDE, there is a little pin icon 
displayed in every title bar of a toplevel window on the left side. If 
you click it, the window is shown on every workspace. There is also a 
way to set properties for a window permanently, by right-clicking on the 
title bar and then "enhanced attributes" (or similar, don't have KDE to 
check it roght now here) which gives a menu with multiple options to 
force the geometry. I don't know for GNOME desktop, but suspect it 
should be similarly easy.


Christian

--
https://mail.python.org/mailman/listinfo/python-list

Re: Compiling and Linking pre-built Windows Python libraries with C++ files on Linux for Windows

[Christian Gollwitzer]

Am 19.03.22 um 01:08 schrieb Ankit Agarwal:

This is a very specific question. I am trying to figure out whether or not
I can use pre-built python libraries and headers on Windows in a MinGW
build on Linux.


With the mingw cross-compiler on Linux that should be possible, however 
I guess it might be difficult to set it up. Setuptools is not good for 
crosscompiling. The easiest way to build Python extensions across 
multiple OSes is cibuildwheel: https://github.com/pypa/cibuildwheel


Especially if your code lives on Github, then you can simply add 
cibuildwheel to your workflow and it will build for multiple OSes and 
Python versions on every commit. For OpenSource projects, Githubs build 
servers are  even for free! Otherwise, the prices are moderate as well.


If you happen to have your own build server farm (as in a commercial 
setting) then you can also use other CI tools with cibuildwheel.


Christian
--
https://mail.python.org/mailman/listinfo/python-list

Re: strange problem building non-pure wheel for apple M1 arm64

[Christian Gollwitzer]

Am 07.03.22 um 17:22 schrieb Robin Becker:


I use brew to install freetype version 2.11.1.



gcc -bundle -undefined dynamic_lookup -g -arch arm64
 build/temp.macosx-11.0-arm64-3.9/src/rl_addons/renderPM/_renderPM.o
 build/temp.macosx-11.0-arm64-3.9/src/rl_addons/renderPM/gt1/gt1-dict.o
 
build/temp.macosx-11.0-arm64-3.9/src/rl_addons/renderPM/gt1/gt1-namecontext.o
 '''''''other compiled code
 
build/temp.macosx-11.0-arm64-3.9/src/rl_addons/renderPM/libart_lgpl/art_vpath_dash.o
 -L/usr/local/lib
 -L/usr/lib
 -L/Library/Frameworks/Python.framework/Versions/3.9/lib
 -lfreetype -o 
build/lib.macosx-11.0-arm64-3.9/reportlab/graphics/_renderPM.cpython-39-darwin.so 



ld: warning: ignoring file /usr/local/lib/libfreetype.dylib, building 
for macOS-arm64 but attempting to link with file built for macOS-x86_64


The above message seems bizarre; everything is compiled for arm64, but 
gcc doesn't want to use an arm64 dylib.


I would interpret this as: the host is runnig in intel, by doing the 
"brew install" it installs the intel version of freetype into 
/usr/local/lib/ ; then you cross-compile the python extension for ARM, 
but -lfreetype picks up the host version.


On macOS it is not mandatory that all symbols in a .dylib are resolved. 
I would guess that the resulting file will not work on M1 macs. YOu can 
check by


otool -L /path/toyour.dylib

which libraries are referenced. If you use freetype functions in your C 
code and do not see it referenced, then the lib will not work.


Maybe you can install an ARM-version of freetype, or compile it from 
source during your build process?


Christian

--
https://mail.python.org/mailman/listinfo/python-list

Re: Why does not Python accept functions with no names?

[Christian Gollwitzer]

Am 20.02.22 um 16:48 schrieb Python:

Abdur-Rahmaan Janhangeer wrote:

Greetings list.

Out of curiosity, why doesn't Python accept
def ():
 return '---'

()

Where the function name is ''?


For the same reason an empty sequence of characters cannot
be a variable name. Do you know any language (or formal
theory) that allows that?


Tcl allows that:

Main console display active (Tcl8.6.9 / Tk8.6.9)
(CDEF) 49 % set "" Hallo
Hallo
(CDEF) 50 % puts ${}
Hallo
(CDEF) 51 % proc "" {} { puts "I'm empty" }
(CDEF) 52 % ""
I'm empty
(CDEF) 53 %

Any string can be a variable or command name, only :: is special as a 
namespace separator.


This only works because of the sparse syntax; to retrieve a variable's 
content, $ is used. For "strange" names quoting is required, therefore I 
had to use "" in the example.


It's a different matter how useful this actually is. One of the object 
systems in Tcl uses the empty variable to represent "self" as an array, 
so that you can write $(prop) for self.prop as it is in Python.


Christian


--
https://mail.python.org/mailman/listinfo/python-list

Re: PermissionError: [Errno 13] Permission denied: 'Abc.xlsx'

[Christian Gollwitzer]

Am 10.02.22 um 20:43 schrieb Chris Angelico:

On Fri, 11 Feb 2022 at 06:41, Dennis Lee Bieber  wrote:

 While not tested with Excel, I /have/ encountered cases where an
application has locked the file for writing, but multiple readers are
permitted. Those would fail then if one attempts to write. {The other view
point is a library that does a complete open/read\write-all/close to memory
-- such an application might open/read/close, then Excel opens/locks, with
the application only learning of the change when it attempts the
open/write/close cycle}



Yeah, I doubt Excel is that sophisticated. It's built on an assumption
of single-user operation.



It guards against multiple user opening the same file over network 
drives. All MS applications create lock files with weird names like 
~.original-name.xlsx etc. If you open a file on the network share, and a 
colleague tries to open it from a second machine, then he will get the 
message "File locked by user xy from machine z". See here for word: 
https://support.microsoft.com/en-us/topic/-the-document-is-locked-for-editing-by-another-user-error-message-when-you-try-to-open-a-document-in-word-10b92aeb-2e23-25e0-9110-370af6edb638


I believe (haven't tested) that this is cooperative locking only and it 
doesn't help if you alter the file from another program. On the same 
machine though, I think that Excel opens the file with a flag to lock it 
from other processes. At least that was my observation and also what the 
OP has described.


Hence it is impossible to concurrently write from Python into an open 
Excel file. One might ask what the real problem is the user is trying to 
solve. Is Excel a requirement, can it be swapped by a database engine?


Best regards,

Christian
--
https://mail.python.org/mailman/listinfo/python-list

Re: How to solve the given problem?

[Christian Gollwitzer]

Am 10.02.22 um 11:26 schrieb NArshad:

-ChrisA:
You don't reply if you have problems.
When I don't find any solution elsewhere then only I place in this group


-Christian:
One problem of different type requires the same elaboration.


No it doesn't


Q. What technique of statistics or numerical computation or general mathematics 
to use to solve this problem. Can you tell one example

Find the values of 𝑥subscript(𝑖) for i from 1 to n (n =100).



WTF? Go study maths yourself instead of asking here (Offf-topic as well)

    Christian

--
https://mail.python.org/mailman/listinfo/python-list

Re: How to solve the given problem?

[Christian Gollwitzer]

Am 10.02.22 um 07:40 schrieb NArshad:


Assume that there is a pattern of feeding for a special fish in a day (10 hours 
a day) as below:
150100303030202010  
  55
Today, the fish is fed in the second hour 60 unit instead of 100 unit 
Accidently. Implement some methods to distribute the remaining 40 unit in the 
rest of the day and propose the new patterns. Try to keep the distribution 
similar to the current feeding pattern.
Note: pay attention that the total feeding amounts should be fix in a day.


This is not a Python problem, it's a math problem and most probably a 
homework problem. Actually the question already tells you how to solve 
it. There are 40 units of fish-food left and you should distribute them 
proportionally to the rest of the day. Sum up the numbers from 3rd to 
last, add 40, and then distribute this to proportionally to each day.


You'll end up with fractinoal numbers in the general case, so you'll 
have to find a method to fairly distribute the units, if you wan to 
stick with integers.


You can also check out the various algorithms for distributing seats in 
a parliament, it is almost the same problem.


    Christian
--
https://mail.python.org/mailman/listinfo/python-list

Re: PermissionError: [Errno 13] Permission denied: 'Abc.xlsx'

[Christian Gollwitzer]

Am 09.02.22 um 08:46 schrieb NArshad:

When I enter data using Tkinter form in an Excel file when the excel file is 
closed there is no error but when I enter data using Tkinter form when the 
excel is already open following error comes:



PermissionError: [Errno 13] Permission denied: 'Abc.xlsx'



What to do to correct this error? I have already searched on google search many 
times but no solution was found.


It's impossible. Excel locks the file deliberately when it is open, so 
that you can't overwrite it from a different program. Otherwise, the 
file could become inconsistent.


The correct way to handle it in the GUI is to tell the user via a 
message box that the file is open and can't be written.


An alternative to writing the file directly would be that you remote 
control Excel; I think it provides a DDE API: 
https://support.microsoft.com/en-us/office/dde-function-79e8b21c-2054-4b48-9ceb-d2cf38dc17f9


Christian
--
https://mail.python.org/mailman/listinfo/python-list

Re: Correct way to setup a package with both compiled C code and Python code?

[Christian Gollwitzer]

Am 08.02.22 um 18:57 schrieb Dieter Maurer:

Christian Gollwitzer wrote at 2022-2-7 20:33 +0100:

we've developed a Python pacakge which consists of both a compiled
extension module and some helper functions in Python. Is there a
tutorial on how to package such an extension?


Look at "https://package.python.org";,
especially 
"https://packaging.python.org/en/latest/guides/packaging-binary-extensions/";.


Thank you, but that page is more like a high-level description, it talks 
a lot about the differences between C code and Python and how it can be 
combined using SWIG etc, but I already have a working extension.


My question was more targeted at how to write the setup.py file in this 
case such that both the compiled code and the Python code gets loaded.


In the meantime, I found a similar example for C++ code with CMake and 
pybind11 here:


https://github.com/benjaminjack/python_cpp_example

That gave me enough to create a similar thing for a pure CPython extension:

https://github.com/auriocus/python_setuptools_sandbox

Now, I still have to figure out how to get this on PyPI and how to 
enable OpenMP during compilation.


Christian
--
https://mail.python.org/mailman/listinfo/python-list

Correct way to setup a package with both compiled C code and Python code?

[Christian Gollwitzer]

Hi all,

we've developed a Python pacakge which consists of both a compiled 
extension module and some helper functions in Python. Is there a 
tutorial on how to package such an extension?


Most resources I found for distutils describe either building an 
extension or pure python modules. Currently I have a structure like this:


ABCD/__init__.py
ABCD/main.py
ccode/some.c
ccode/some.h
ccode/Makefile

The Makefile compiles the C code and creates ABCD/some.so, which 
"main.py" then imports. This works, but not e.g. on Windows and it's not 
integrated into pip, of course.


We're soon going to publish the package as open source code and it would 
be great to do "pip install ABCD" ultimately. Is there a simple example 
out there how to achieve this?


Additionally, we use OpenMP in the C code for parallelism. This is easy 
in the Makefile, one has to pass "-fopenmp" to gcc and "/openmp" to 
msvc. Is there a way to set this flag automatically depending on the 
compiler?


Best regards,

 Christian
--
https://mail.python.org/mailman/listinfo/python-list

Re: ssl server: how to disable client cert verfication?

[Christian Heimes]

On 04/02/2022 19.24, Grant Edwards wrote:

The problem is _getting_ the client certificate that was provided
during the client/server handshake. That's trivial if the handshake
was successful. The problem is obtaining the client certificate when
the handshake fails. I was hoping there was a way to disable client
certificate validation so that the handshake will succeed and then
allow me to get the client certificate from the connection object.


FYI, it's more complicated in TLS 1.3. Post-handshake authentication 
(PHA) can happen out-of-bounce. Only TLS 1.2 performs client cert auth 
during handshake or renegotiation.


Christian
--
https://mail.python.org/mailman/listinfo/python-list

Re: ssl server: how to disable client cert verfication?

[Christian Heimes]

On 03/02/2022 19.57, Grant Edwards wrote:

I've got a small ssl server app. I want to require a certificate from
the client, so I'm using a context with

context.verify_mode = ssl.CERT_REQUIRED

But, I want all certificates accepted. How do I disable client
certificate verification?


You can't. Python's ssl module does not expose the necessary feature to 
override the verification callback SSL_CTX_set_verify(). PyOpenSSL lets 
you set a callback and ignore any and all errors.


Christian
--
https://mail.python.org/mailman/listinfo/python-list

Re: About Python Compressed Archive or Binaries

[Christian Heimes]

On 17/01/2022 18.49, Kirill Ratkin wrote:
It would be nice to have just zip file with python interpreter (not 
executable installer), unpack it anywhere, add path  to this 'anywhere' 
to PATH, and use it.


Java/DotNet/Go have this option. But python - not.

And question is - why?


Java is developed by Oracle (revenue 49 billion USD)
.NET is developed by Microsoft (revenue 161 billion USD)
Go is developed by Google (revenue of parent company is 182 billion USD)
Python is developed by a bunch of (mostly unpoaid) volunteers under the 
umbrella of the PSF (revenue 3.1 million USD).


Do you see a pattern? :)

A distributable binary format for CPython is high on our list. Brett 
Cannon, Nathaniel Smith, and I have been discussion a proposal recently. 
I landed a bunch of changes in 3.11-dev that are both useful for binary 
distribution and for Web Assembly builds. Because all work is done in 
our free time, there is only little progress.


Christian
--
https://mail.python.org/mailman/listinfo/python-list

Re: Failure to Display Top menu

[Christian Gollwitzer]

Am 26.11.21 um 21:38 schrieb Peter Mwale:

Hello, my python 3.10 shell is not displaying the top menu. What should I
do?



You should explain, what you do exactly. The Python interpreter does not 
have a menu.


a) What platform are you on? Windows, macOS, Linux?

b) How did ou start Python and what was the expectation?

Christian
--
https://mail.python.org/mailman/listinfo/python-list

Re: One line sort

[Christian Gollwitzer]

Am 15.11.21 um 14:10 schrieb ast:

A curiosity:

q = lambda x: x and q([i for i in x[1:] if i < x[0]]) + [x[0]] + q([i 
for i in x[1:] if i >= x[0]])


 >>> q([7, 5, 9, 0])
[0, 5, 7, 9]


That seems to be a translation of the classic Haskell quicksort example:

qsort [] = []
qsort (x:xs) = qsort [a | a <- xs, a < x]
  ++ [x] ++
   qsort [b | b <- xs, b >= x]


The Haskell version is a bit clearer IMHO due to the pattern matching, 
but that results in a 2 liner :)


Christian
--
https://mail.python.org/mailman/listinfo/python-list

Re: Create a contact book

[Christian Gollwitzer]

Am 26.10.21 um 07:40 schrieb anders Limpan:

i would like to create a contact book were you can keep track of your friends. 
With this contact book you will both be able to add friends and view which 
friends that you have added. anyone interested in helping me out with this one 
?=)


Here is how to do it: https://facebook.com/


Christian
--
https://mail.python.org/mailman/listinfo/python-list

Re: sum() vs. loop

[Christian Gollwitzer]

Am 12.10.21 um 05:41 schrieb Dan Stromberg:

On Mon, Oct 11, 2021 at 2:54 PM Steve Keller  wrote:


I have found the sum() function to be much slower than to loop over the
operands myself:

def sum_products(seq1, seq2):
 return sum([a * b for a, b in zip(seq1, seq2)])

def sum_products2(seq1, seq2):
 sum = 0
 for a, b in zip(seq1, seq2):
 sum += a * b
 return sum



It seems like the generator expression should be the fastest to me.  But
writing for speed instead of writing for clarity is usually not a great
idea.



Maybe, unless this was just a test, it can be fastest AND clearest with

   numpy.dot(seq1, seq2)

- in case that the sequence consists of floats and, in the best case, is 
already stored in a numpy array. Then you cannot beat this with Python 
code.


Christian
--
https://mail.python.org/mailman/listinfo/python-list

Re: sum() vs. loop

[Christian Gollwitzer]

Am 10.10.21 um 10:49 schrieb Steve Keller:

I have found the sum() function to be much slower than to loop over the
operands myself:

def sum_products(seq1, seq2):
 return sum([a * b for a, b in zip(seq1, seq2)])

def sum_products2(seq1, seq2):
 sum = 0
 for a, b in zip(seq1, seq2):
 sum += a * b
 return sum

In a program I generate about 14 million pairs of sequences of ints each
of length 15 which need to be summed.  The first version with sum() needs
44 seconds while the second version runs in 37 seconds.


The first version constructs a list, sums it up and throws the list 
away, while the second version only keeps the running sum in memory. How 
about a generator expression instead, i.e.



sum((a * b for a, b in zip(seq1, seq2)))

(untested) ?

Christian
--
https://mail.python.org/mailman/listinfo/python-list

Re: NUmpy

[Christian Gollwitzer]

Am 29.09.21 um 18:16 schrieb Jorge Conforte:



Hi,

I have a netcdf file "uwnd_850_1981.nc" and I'm using the commands to 
read it:


Your code is incomplete:

from numpy import dtype
  fileu ='uwnd_850_1981.nc'
ncu = Dataset(fileu,'r')


Where is "Dataset" defined?


uwnd=ncu.variables['uwnd'][:]


and I had:

:1: DeprecationWarning: `np.bool` is a deprecated alias for the 
builtin `bool`. To silence this warning, use `bool` by itself. Doing 
this will not modify any behavior and is safe. If you specifically 
wanted the numpy scalar type, use `np.bool_` here.
Deprecated in NumPy 1.20; for more details and guidance: 
https://numpy.org/devdocs/release/1.20.0-notes.html#deprecations


I didn't how I have this message. My numpy verison is 1.21.2.


Please, how can I solve this.


First, it is only a warning, therefore it should still work. Second, the 
problem is not in the code that you posted. POssibly in the definition 
of "Dataset". Maybe the netCDF-File contains boolean values and the 
package you use to read it should be updated?


Christian
--
https://mail.python.org/mailman/listinfo/python-list

Re: XML Considered Harmful

[Christian Gollwitzer]

Am 22.09.21 um 16:52 schrieb Michael F. Stemper:

On 21/09/2021 19.30, Eli the Bearded wrote:

Yes, CSV files can model that. But it would not be my first choice of
data format. (Neither would JSON.) I'd probably use XML.


Okay. 'Go not to the elves for counsel, for they will say both no
and yes.' (I'm not actually surprised to find differences of opinion.)


It is wrong, CSV has no model of hierarchical data. A CSV file is a 2d 
table, just like a database table or an Excel sheet.


You can /layer/ high-dimensional data on top of a 2D table, there is the 
relational algebra theory behind this, but it is wrong (or misleading at 
best) to say that CSV can model hierarchical data.


It's the same as saying "CSV supports images". Of course it doesn't, its 
a textfile, but you could encode a JPEG as base64 and then put this 
string into the cell of a CSV table. That definitely isn't what a sane 
person would understand as "support".


Christian

--
https://mail.python.org/mailman/listinfo/python-list

Re: on floating-point numbers

[Christian Gollwitzer]

Am 04.09.21 um 14:48 schrieb Hope Rouselle:

Christian Gollwitzer  writes:


Am 02.09.21 um 15:51 schrieb Hope Rouselle:

Just sharing a case of floating-point numbers.  Nothing needed to be
solved or to be figured out.  Just bringing up conversation.
(*) An introduction to me
I don't understand floating-point numbers from the inside out, but I
do
know how to work with base 2 and scientific notation.  So the idea of
expressing a number as
mantissa * base^{power}
is not foreign to me. (If that helps you to perhaps instruct me on
what's going on here.)
(*) A presentation of the behavior


import sys
sys.version

'3.8.10 (tags/v3.8.10:3d8993a, May 3 2021, 11:48:03) [MSC v.1928 64
bit (AMD64)]'


ls = [7.23, 8.41, 6.15, 2.31, 7.73, 7.77]
sum(ls)

39.594


ls = [8.41, 6.15, 2.31, 7.73, 7.77, 7.23]
sum(ls)

39.61
All I did was to take the first number, 7.23, and move it to the
last
position in the list.  (So we have a violation of the commutativity of
addition.)


I believe it is not commutativity, but associativity, that is
violated.


Shall we take this seriously?  (I will disagree, but that doesn't mean I
am not grateful for your post.  Quite the contary.)  It in general
violates associativity too, but the example above couldn't be referring
to associativity because the second sum above could not be obtained from
associativity alone.  Commutativity is required, applied to five pairs
of numbers.  How can I go from

   7.23 + 8.41 + 6.15 + 2.31 + 7.73 + 7.77

to

   8.41 + 6.15 + 2.31 + 7.73 + 7.77 + 7.23?

Perhaps only through various application of commutativity, namely the
ones below. (I omit the parentheses for less typing.  I suppose that
does not create much trouble.  There is no use of associativity below,
except for the intented omission of parentheses.)


With the parens it will become more obvious.



  7.23 + 8.41 + 6.15 + 2.31 + 7.73 + 7.77
= 8.41 + 7.23 + 6.15 + 2.31 + 7.73 + 7.77


The sum is evaluated as

(((7.23 + 8.41) + 6.15 + ...)

For the first shift, you are correct that commutativity will result in

(((8.41 + 7.23) + 6.15 + ...)

But you can't go in one step to

(((8.41 + 6.15) + 7.23 + ...)

with  the commutativity law alone. Instead, a sequence of associativity 
and commutativity is required to move the 7.23 out of the first pair of 
parentheses.


And what I was trying to say, the commutative steps *are* equal in 
floating point arithmetics, whereas the associative steps are not.


Christian

--
https://mail.python.org/mailman/listinfo/python-list

Re: on floating-point numbers

[Christian Gollwitzer]

Am 02.09.21 um 21:02 schrieb Julio Di Egidio:

On Thursday, 2 September 2021 at 20:43:36 UTC+2, Chris Angelico wrote:

On Fri, Sep 3, 2021 at 4:29 AM Hope Rouselle  wrote:



All I did was to take the first number, 7.23, and move it to the last
position in the list. (So we have a violation of the commutativity of
addition.)


It's not about the commutativity of any particular pair of operands -
that's always guaranteed.


Nope, that is rather *not* guaranteed, as I have quite explained up thread.



No, you haven't explained that. You linked to the famous Goldberg paper. 
Where in the paper does it say that operations on floats are not 
commutative?


I'd be surprised because it is generally wrong.
Unless you have special numbers like NaN or signed zeros etc., a+b=b+a 
and a*b=b*a holds also for floats.


Christiah
--
https://mail.python.org/mailman/listinfo/python-list

Re: on floating-point numbers

[Christian Gollwitzer]

Am 02.09.21 um 15:51 schrieb Hope Rouselle:

Just sharing a case of floating-point numbers.  Nothing needed to be
solved or to be figured out.  Just bringing up conversation.

(*) An introduction to me

I don't understand floating-point numbers from the inside out, but I do
know how to work with base 2 and scientific notation.  So the idea of
expressing a number as

   mantissa * base^{power}

is not foreign to me. (If that helps you to perhaps instruct me on
what's going on here.)

(*) A presentation of the behavior


import sys
sys.version

'3.8.10 (tags/v3.8.10:3d8993a, May  3 2021, 11:48:03) [MSC v.1928 64 bit 
(AMD64)]'


ls = [7.23, 8.41, 6.15, 2.31, 7.73, 7.77]
sum(ls)

39.594


ls = [8.41, 6.15, 2.31, 7.73, 7.77, 7.23]
sum(ls)

39.61

All I did was to take the first number, 7.23, and move it to the last
position in the list.  (So we have a violation of the commutativity of
addition.)


I believe it is not commutativity, but associativity, that is violated. 
Even for floating point, a+b=b+a except for maybe some extreme cases 
like denormliazed numbers etc.


But in general (a+b)+c != a+ (b+c)

Consider decimal floating point with 2 digits.
a=1
b=c=0.04

Then you get LHS;
 (1 + 0.04) + 0.04 = 1 + 0.04 = 1

RHS:

1 + (0.04 + 0.04) = 1 + 0.08 = 1.1


Your sum is evaluated like (((a + b) + c) + ) and hence, if you 
permute the numbers, it can be unequal. If you need better accuracy, 
there is the Kahan summation algorithm and other alternatives: 
https://en.wikipedia.org/wiki/Kahan_summation_algorithm



Christian
--
https://mail.python.org/mailman/listinfo/python-list

Re: on floating-point numbers

[Christian Gollwitzer]

Am 02.09.21 um 16:49 schrieb Julio Di Egidio:

On Thursday, 2 September 2021 at 16:41:38 UTC+2, Peter Pearson wrote:

On Thu, 02 Sep 2021 10:51:03 -0300, Hope Rouselle wrote:



39.61


Welcome to the exciting world of roundoff error:


Welcome to the exiting world of Usenet.

*Plonk*


Pretty harsh, isn't it? He gave a concise example of the same inaccuracy 
right afterwards.


    Christian
--
https://mail.python.org/mailman/listinfo/python-list

Re: Create a real-time interactive TUI using Python.

[Christian Gollwitzer]

Am 31.08.21 um 18:49 schrieb Chris Angelico:

On Wed, Sep 1, 2021 at 1:59 AM hongy...@gmail.com  wrote:


I want to know whether python can be used to create real-time interactive TUI, 
as hstr [1] does.

[1] https://github.com/dvorka/hstr



Yes.


I think he also would like to know, how to achieve this ;)

This kind of interface is usually done with the "curses"-library. There 
is a Python version of it, see e.g. here


https://docs.python.org/3/howto/curses.html


    Christian
--
https://mail.python.org/mailman/listinfo/python-list

Re: Searching pypi.org, is there an 'advanced search'?

[Christian Heimes]

On 18/07/2021 10.15, Chris Green wrote:
> Mark Lawrence  wrote:
>> On Saturday, July 17, 2021 at 1:03:21 PM UTC+1, Chris Green wrote:
>>> Every time I go to pypi.org to look for a neat utility or something I 
>>> curse the stupid search. 
>>>
>>> Is there really no better search available? Apart from anything else 
>>> it appears to OR the elements together so, for example, I wanted to 
>>> search for programs/libraries that will allow me to access Google 
>>> Contacts:- 
>>>
>>> Putting the words Google and Contacts in the search box produces 
>>> more than 1 hits. 
>>>
>>> Putting just Google in the search box produces 8,629 projects. 
>>>
>>> So, it would appear the terms are OR'ed together, surely OR'ing terms 
>>> in a search (by default) went out with the ark! 
>>>
>>> When I put 'Google contacts' in the search box I want projects that 
>>> have both 'Google' AND 'Contacts' in their description somewhere! 
>>> (... and, no, I tried 'Google AND Contacts', still more than 1 
>>> results). 
>>>
>>> pypi.org is a wonderful resource but its size now demands a better 
>>> search engine. 
>>>
>>
>> As this is an open source project I'm looking forward to the huge amount 
>> of work that you personally will be putting in to remedy this situation. 
>> In summary put up or shut up :) 
> 
> I'm quite happy to help, however I was just asking (maybe rather more
> agressively than I should have done) if I was missing something
> obvious.  E.g. is there some syntax that I don't know about that I can
> use in the search box, or maybe I should use an external search engine?
> 
> ... and Open Source doesn't mean one can't criticise surely, rather
> the opposite! :-)   (I have, by the way, contributed in a minor way to
> quite a few open source projects over the years, and that's a lot of
> years!  I even have a small credit in the 'bible' for the Kermit file
> transfer protocol, remember that?)

PyPI used to have a search API. The XML-RPC endpoints were disabled last
year due to flooding. Our infrastructure could not cope with abusive
queries. You can read the log of events at https://status.python.org/

Christian


-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Subpixel positioning on Tk canvas

[Christian Gollwitzer]

Am 20.06.21 um 01:49 schrieb Terry Reedy:

On 6/19/2021 12:42 AM, Christian Gollwitzer wrote:
Sorry for that answer, but Tkinter does not support many of the most 
useful extensions for Tcl/Tk, because someone has to write the 
wrappers. It only supports what is provided by base Tk. Among those I 
consider useful and use in almost any application are:


Are these extensions included with the tcl/tk distribution, or otherwise 
available from active state?  Are this extensions included with Linux 
installations of tcl/tk?  Or easily installed?


Since ActiveState has pulled out the developers of Tcl a few years ago, 
I haven't used ActiveTcl anymore. I was surprised to see that they 
actually offer a fairly recent version, but it also cannot be simply 
downloaded, one has to register. It was unclear to me if it costs money.


Other people have stepped in to provide Tcl distributions where tese 
extensions are included; notable exanples are BAWT by Paul Obermeier 
http://www.bawt.tcl3d.org/download.html which offers all of the 
mentioned packages (and many more), Androwish/Undroidwish by Christian 
Werner which was originally developed for Android, but now works on te 
major desktop platforms, http://androwish.org/home/wiki?name=undroidwish 
and even kbskit can be mentioned, started by Rene Zaumseil and now 
updated in irregular intervals by me https://github.com/auriocus/kbskit


I haven't checked the major linux distros, but they also might ship with 
some of these extensions.


Concerning installation, it differs. Tablelist (also part of tklib) and 
pdf4tcl are pure-Tcl packages and therefore easily installed.
TkDnD, TkTable and tkTreeCtrl are compiled extensions and therefore more 
difficult - however, due to the stubs mechanism of Tcl, the version 
number of Tcl and C compiler do NOT need to match. Typically a binary 
downloaded for the right OS and bitness will work, and compilation from 
source works with an autoconf-based configure script.


Due to ActiveState's failure with the teapot, the Tcl world does now not 
any longer have a central repository tool like "pip" which works for 
everyone. This has just recently been discussed on comp.lang.tcl, but it 
is unlikely to happen in the near future.


It is of course unrealistic to expect that Tkinter supports every odd Tk 
extension fron the Tcl world, which might not even be maintained any 
longer. OTOH there are extensions that are well-maintained, that could 
as well be part of the core Tk, but aren't for political reasons. If 
Tkinter continues to be the shipped "first choice" GUI for Python, then 
a few could be included - or otherwise Tkinter will lack Drag'n'drop for 
ever and a reasonable Tree widget (the core one is verrrry basic).


Best regards,

Christian


--
https://mail.python.org/mailman/listinfo/python-list

Re: Tkinter problem

[Christian Gollwitzer]

Am 19.06.21 um 08:48 schrieb Jach Feng:

Christian Gollwitzer 在 2021年6月19日 星期六下午1:54:46 [UTC+8] 的信中寫道：

I guess you wanted to post another question? Then please open a new
thread. In addition, the question is unclear, you just posted a
transcript of three lines of Python.



I posted to point out there is an error in Liya's script. It's not related to 
Colab, but it may relate to his problem.



Ah! Now I get it, sorry, missed that earlier.

    Christian
--
https://mail.python.org/mailman/listinfo/python-list

Re: Tkinter problem

[Christian Gollwitzer]

Am 19.06.21 um 07:16 schrieb Jach Feng:

Christian Gollwitzer 在 2021年6月19日 星期六下午12:27:54 [UTC+8] 的信中寫道：

Am 19.06.21 um 05:59 schrieb Jach Feng:

import tkinter as Tk
Tk



from tkinter import *
Tk



tkinter

Traceback (most recent call last):
File "", line 1, in 
NameError: name 'tkinter' is not defined



What's the point? That has no relation to the question.

"import A as B" does not define A. That's a feature, not a bug.

Christian

No, it's not. It's only because this line triggers my response:-)

label1 = tkinter.Label(master, text='Hello')


You have posted this as an answer to Liya's question about Google Colab. 
What you wrote has nothing to do with it and does not help with Liya's 
problem.


I guess you wanted to post another question? Then please open a new 
thread. In addition, the question is unclear, you just posted a 
transcript of three lines of Python.


Christian
--
https://mail.python.org/mailman/listinfo/python-list

Re: Subpixel positioning on Tk canvas

[Christian Gollwitzer]

Am 19.06.21 um 06:26 schrieb George Furbish:

On Saturday, June 19, 2021 at 12:22:31 AM UTC-4, Christian Gollwitzer wrote:

Am 19.06.21 um 02:03 schrieb George Furbish:

Does Tk support interpolation/subpixel positioning of canvas elements? (e.g. 
images, text.) I have moving elements on my canvas, but the movement isn't very 
smooth and it's especially obvious when I have text moving alongside an image, 
since the two elements tend to jump to the next pixel at different times, 
creating a little judder between the two.

There is an "improved canvas" available, tkpath, which supports
antialiasing on all platforms. It is part of, e.g. undroidwish, if you
want to experiment with it. Last time I tested it had problems on macOS
though.


How can I enable or access the improved canvas via Tkinter?



Probably by writing the wrapper for it ;)

Sorry for that answer, but Tkinter does not support many of the most 
useful extensions for Tcl/Tk, because someone has to write the wrappers. 
It only supports what is provided by base Tk. Among those I consider 
useful and use in almost any application are:


* TkDnD for native drag'n'drop support (there is an inferior python 
package of the same name which implements local DnD only)


* tablelist - complete widget for displaying trees and tables like 
ttk::treeview, but with almost every feature one could imagine


* pdf4tcl - create a PDF from a canvas content, e.g. for printing


Basically you call Tcl via the eval() method of tkinter; in principle 
you could do



import tkinter as tk
root=tk()

root.eval('package require tkpath')
root.eval('...here comes your tkpath code...')
root.call('.tkp', 'create', 'oval', )


tkpath is described here: https://wiki.tcl-lang.org/page/tkpath

For the wrapping, look at the implementation files of Tkinter, for say, 
the original canvas, and modify accordingly.


Christian

--
https://mail.python.org/mailman/listinfo/python-list

Re: Tkinter problem

[Christian Gollwitzer]

Am 19.06.21 um 05:59 schrieb Jach Feng:

import tkinter as Tk
Tk



from tkinter import *
Tk



tkinter

Traceback (most recent call last):
   File "", line 1, in 
NameError: name 'tkinter' is not defined




What's the point? That has no relation to the question.

"import A as B" does not define A. That's a feature, not a bug.

Christian
--
https://mail.python.org/mailman/listinfo/python-list

Re: Tkinter problem

[Christian Gollwitzer]

Am 18.06.21 um 08:28 schrieb Liya Ann Sunny:

I am using Colab. How could  solve this problem.
TclError: couldn't connect to display ":0.0"


You're either not running an X server, or having problems to connect to it.

Are you sure that Google Colab supports X11 at all? This link doesn't 
seem to support that idea:


https://stackoverflow.com/questions/61168210/is-there-any-way-to-use-tkinter-with-google-colaboratory

Christian


--
https://mail.python.org/mailman/listinfo/python-list

Re: Is there a way to get the following result in Python?

[Christian Gollwitzer]

Am 12.06.21 um 04:02 schrieb Jach Feng:

def foo():

... # do something
...

a = []
for i in range(3):

... a.append(foo())
...

a

[]




How about having "foo" return a list of things? Then you can append that 
list and return an empty list if you want nothing added:


In [1]: def foo():
   ...: return [1,2,3]
   ...:

In [2]: def bar():
   ...: return []
   ...:

In [3]: a=[]

In [4]: a += foo()

In [5]: a
Out[5]: [1, 2, 3]

In [6]: a += bar()

In [7]: a
Out[7]: [1, 2, 3]


    Christian
--
https://mail.python.org/mailman/listinfo/python-list

Re: learning python ...

[Christian Gollwitzer]

Am 25.05.21 um 06:08 schrieb hw:

On 5/25/21 12:37 AM, Greg Ewing wrote:

Python does have references to *objects*. All objects live on
the heap and are kept alive as long as there is at least one
reference to them.

If you rebind a name, and it held the last reference to an
object, there is no way to get that object back.


Are all names references?  When I pass a name as a parameter to a 
function, does the object the name is referring to, when altered by the 
function, still appear altered after the function has returned?  I 
wouldn't expect that ...




Yes, it does. It is a common pitfall for newbie Python programmers.

def f(a):
a.append(2)

l=[1, 2, 3]

f(l)
print(l)


==> [1, 2, 3, 2]

The strange thing, coming from a different language, is the apparent 
difference, if instead of a list, you pass an integer:


def f(a):
a=5

l=3

f(l)
print(l)

> 3

Here, the "l" is not changed. The reason is that the statement "a=5" 
does NOT modify the object in a, but instead creates a new one and binds 
it to a. l still points to the old one. Whereas a.append() tells the 
object pointed to by a to change.


Christian

--
https://mail.python.org/mailman/listinfo/python-list

Re: Bloody rubbish

[Christian Gollwitzer]

Am 06.05.21 um 19:54 schrieb Skip Montanaro:


Machine language is so much simpler, and you can code with just a hexpad.



Pshaa... All you need are front panel switches. ;-) (Yes, I had a professor
who required is to 'key' in our programs on the front panel, of a rack
mounted PDP-11 as I recall. Needless to say, we didn't use an assembler
either. We just wrote raw opcodes and their arguments on paper. This was in
the late 70s.)


Pure luxury! https://xkcd.com/378/

    Christian
--
https://mail.python.org/mailman/listinfo/python-list

Re: Not found in the documentation

[Christian Gollwitzer]

Am 29.04.21 um 08:54 schrieb elas tica:

Le mercredi 28 avril 2021 à 17:36:32 UTC+2, Chris Angelico a écrit :


In what sense of the word "token" are you asking? The parser? You can
play around with the low-level tokenizer with the aptly-named
tokenizer module.


It was a good suggestion, and the PLR doesn't mention the tokeniser module. It 
should, this goes very well with the conversional style it has.



# --
from tokenize import tokenize
from io import BytesIO

s="""42 not\
  in [42]"""
g = tokenize(BytesIO(s.encode('utf-8')).readline)
print(*(g), sep='\n')
# --

the docs are wrong when they say:

..
using a backslash). A backslash is illegal elsewhere on a line outside a string 
literal.
..



You're not passing a backslash. Try print(s).
It would be different with a raw string

s=r"""42 not\
   in [42]"""

Christian

--
https://mail.python.org/mailman/listinfo/python-list

Re: question about basics of creating a PROXY to MONITOR network activity

[Christian Seberino]

>
>
> a) your reverse proxy must be colocated with the service it fronts on the
> same machine;
> b) your network infrastructure transparently encrypts traffic between your
> proxy and the service; or
> c) your proxy must negotiate its own TLS connection(s) with the service.
>

Paul

Thanks. I'm curious, do you know which of your options CloudFlare uses?  It
has to stand in between
you and all the sites you visit while allowing encryption right?

cs
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Retrieving non-/etc/passwd users with Python 3?

[Christian Heimes]

On 31/03/2021 14.45, Loris Bennett wrote:
> Chris Angelico  writes:
> 
>> On Wed, Mar 31, 2021 at 11:21 PM Loris Bennett
>>  wrote:
>>>
>>> Hi,
>>>
>>> I want to get a list of users on a Linux system using Python 3.6.  All
>>> the users I am interested in are just available via LDAP and are not in
>>> /etc/passwd.  Thus, in a bash shell I can use 'getent' to display them.
>>>
>>> When I try to install the PyPi package
>>>
>>>   getent
>>>
>>> I get the error
>>>
>>> File "/tmp/pip-build-vu4lziex/getent/setup.py", line 9, in 
>>>   long_description = file('README.rst').read(),
>>>   NameError: name 'file' is not defined
>>>
>>> I duckduckwent a bit and the problem seems to be that 'file' from Python
>>> 2 has been replaced by 'open' in Python 3.
>>>
>>> So what's the standard way of getting a list of users in this case?
>>>
>>
>> I don't have LDAP experience so I don't know for sure, but is the
>> stdlib "pwd" module suitable, or does it only read /etc/passwd?
>>
>> https://docs.python.org/3/library/pwd.html
>>
>> Failing that, one option - and not as bad as you might think - is
>> simply to run getent using the subprocess module, and parse its
>> output. Sometimes that's easier than finding (or porting!) a library.
> 
> D'oh!  Thanks, 'pwd' is indeed exactly what I need.  When I read the
> documentation here
> 
>   https://docs.python.org/3.6/library/pwd.html 
> 
> I mistakenly got the impression that it was only going to give me the
> local users.  It doesn't actually say that, but it mentions /etc/shadow
> and not getent.  However, it does talk about the "account and password
> database", which is a clue (although our passwords are on an other
> system entirely), since "database" is more getent terminology.
> 
> In any case, I think 'pwd' is hiding its light under a bushel a bit
> here.

Please open a documentation bug :)

The pwd and grp module use the libc API to get users from the local
account database. On Linux and glibc the account database is handled by
NSS and nsswitch.conf.

By the way I recommend that you use SSSD instead of talking to LDAP
directly. You'll have a much more pleasant experience.

Christian

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: convert script awk in python

[Christian Gollwitzer]

Am 25.03.21 um 00:30 schrieb Avi Gross:

It [awk] is, as noted, a great tool and if you only had one or a few tools like 
it
available, it can easily be bent and twisted to do much of what the others
do as it is more programmable than most. But following that line of
reasoning, fairly simple python scripts can be written with python -c "..."
or by pointing to a script


The thing with awk is that lots of useful text processing is directly 
built into the main syntax; whereas in Python, you can certainly do it 
as well, but it requires to load a library. The simple column summation 
mentioned before by Cameron would be


   awk ' {sum += $2 } END {print sum}'

which can be easily typed into a command line, with the benefit that it 
skips every line where the 2nd col is not a valid number. This is 
important because often there are empty lines, often there is an empty 
line at the end, some ascii headers whatever.


The closest equivalent I can come up with in Python is this:

==
import sys

s=0
for line in sys.stdin:
try:
s += float(line.split()[1])
except:
pass
print(s)
===


I don't want to cram this into a python -c " "  line, if it even is 
possible; how do you handle indentation levels and loops??


Of course, for big fancy programs Python is a much better choice than 
awk, no questions asked - but awk has a place for little things which 
fit the special programming model, and there are surprisingly many 
applications where this is just the easiest and fastest way to do the job.


It's like regexes - a few simple characters can do the job which 
otherwise requires a bulky program, but once the parsing gets to certain 
complexity, a true parsing language, or even just handcoded Python is 
much more maintainable.


Christian

PS: Exercise - handle lines commented out with a '#', i.e. skip those. 
In awk:


gawk '!/^\s*#/ {sum += $2 } END {print sum}'

--
https://mail.python.org/mailman/listinfo/python-list

Re: port to PDOS (especially mainframe)

[Christian Heimes]

On 23/03/2021 22.22, Paul Edwards wrote:
> "long long" is not C90. I don't have support for that. The
> "i370" target of GCC 3.2.3 doesn't have DI instructions
> working. Just 32 bits. Everything is 32 bits.

Which version of Python are you using as baseline? CPython requires C99
since 3.6,
https://docs.python.org/3/whatsnew/3.6.html#build-and-c-api-changes

Christian

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: .title() - annoying mistake

[Christian Gollwitzer]

Am 22.03.21 um 16:03 schrieb Robert Latest:

Chris Angelico wrote:

Cool thing is, nobody in Python needs to maintain anything here.


That's great because I'm actually having trouble with sending log messages over
the socket conection you helped me with, would you mind having a look?


You misunderstood the "here".

(native German as well, but I think it means "for keeping .title() up to 
date)


I agree with Chris that .title() can be useful for "title-casing" a 
single character, whatever that means. It should be documented, though, 
that it is not suitable to title-case a string, not even in English.


Christian
--
https://mail.python.org/mailman/listinfo/python-list

Re: neonumeric - C++ arbitrary precision arithmetic library

[Christian Gollwitzer]

Am 07.03.21 um 20:42 schrieb Peter J. Holzer:

The second part is converting a parse tree into code. I am quite sure
that it is possible to devise a formal language to specify the semantics
of any programming language and then to use this to generate the code.
However, I strongly suspect that such a language would be comparable in
expressiveness and ease of use to other programming languages - or in
other worlds it would be just another programming language. 


As far as I understand the idea, Leigh (aka Mr Flibble) thinks that he 
can create a "library" of code translators that translate the individual 
pieces of the parse tree into some intermediate code, and by specifying 
these codelets within the grammar the semantics of a language can be 
fully described.


My argument against this whole thing is that the library would be 
enormous. He seems to think that big chunks can be reused for different 
languages, but I don't believe this. A simple example:


int a = -3
unsigned int b = 5;

in C: a < b is false. This is insane but follows from the type casting 
rules in C.


in Python or any other language with sane integers, -3 < 5 will always 
be true. For large enough values you must convert them to big integers. 
CPython simply uses big integers for everything, but that obviously 
slows down the whole thing. If you want to compile that to fast machine 
code, you need to do the computation in fixed integers and check for 
overflow *after every single step*.


And there are myriads of differences. Introducing JavaScript again will 
bring myriads of differences and so on. The closest thing to such a 
project I can think of is LLVM. It still requires you to write the 
actual code generator, but features many intrinsics and libriaries that 
could be reused and therefore it is easy to add a new language. The 
approximate value of LLVM is estimated here:


  https://chriscummins.cc/2019/llvm-cost/

It's over 5000 man years of work. There is no way a single developer, 
regardless how brilliant he may be, can recreate this in his lifetime. 
Maybe focussed to a single language or two, but not "universal" in any 
sense.



Christian
--
https://mail.python.org/mailman/listinfo/python-list

Re: Tkinter long-running window freezes

[Christian Gollwitzer]

Am 26.02.21 um 06:15 schrieb John O'Hagan:

On Thu, 25 Feb 2021 21:57:19 +0100
Christian Gollwitzer  wrote:

I think it is not yet clear, if this is a bug in Tkinter or in
Tcl/Tk, the underlying scripting language. It might also be platform
dependent. Are you on Windows? Here is an equivalent Tcl program:

==
package require Tk

proc randint {} {
expr {int(rand()*1000)}
}

proc display {label} {
destroy $label
set id [randint]
set label [label .l$id -text [randint]]
pack $label
after 100 [list display $label]
}

display [label .l]



Can you run this and check that the freeze also occurs? If you can't
execute the Tcl that is used by Python directly, you may also do
something like


root = Tk()
root.eval('Here comes the Tcl code')
root.mainloop()

Can you also find out what version of Tcl/Tk you are using? Try

root.eval('info patchlevel')

Christian



I've followed your suggestions as per my last post, and can confirm
the same freezing behaviour when running your code directly as a tclsh
script on Debian Testing, Tcl 8.6.11.


You might report this as a bug to the Tcl bugtracker 
https://core.tcl-lang.org/tk/ticket


I guess the problem is with the steady creation of widgets. Tk was not 
meant to be used like that. Tkinter creates new widget names for each 
widget with random numbers, just like the Tcl code above does, whereas 
in a usual Tcl/Tk program the names are given by the programmer.


Can you also check this program, which reuses the same widget path name, 
albeit does the creation/destruction in cycles:


==
package require Tk

proc randint {} {
expr {int(rand()*1000)}
}

proc display {label} {
destroy $label
set label [label .l -text [randint]]
pack $label
after 100 [list display $label]
}

display [label .l]


As mentioned by others, typically you wouldn't continuously recreate new 
widgets, but either update the text of the widget (label['text']="New 
text") or attaching a StringVar() )


or, if you must rearrange the widgets, you pack_forget() them and then 
repack them.


Christian
--
https://mail.python.org/mailman/listinfo/python-list

Re: error of opening Python

[Christian Gollwitzer]

Am 25.02.21 um 18:22 schrieb Botao Liu:

Dear Python team,

This is my first time using Python, I tried to launch Python and it showed
"Python 3.9.2 (tags/v3.9.2:1a79785, Feb 19 2021, 13:44:55) [MSC v.1928 64
bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information." I
don't know what this meant and how to fix this. Could you please help me?
Thank you very much.


This is as it should be. What have you expected, that you think this is 
wrong?


Maybe you are looking for an IDE like IDLE (should be installed along 
with Python on Windows), or something very different?


Christian
--
https://mail.python.org/mailman/listinfo/python-list

Re: Tkinter long-running window freezes

[Christian Gollwitzer]

Am 24.02.21 um 12:35 schrieb John O'Hagan:

Hi list

I have a 3.9 tkinter interface that displays data from an arbitrary
number of threads, each of which runs for an arbitrary period of time.
A frame opens in the root window when each thread starts and closes
when it stops. Widgets in the frame and the root window control the
thread and how the data is displayed.

This works well for several hours, but over time the root window
becomes unresponsive and eventually freezes and goes grey. No error
messages are produced in the terminal.

Here is some minimal, non-threaded code that reproduces the problem on
my system (Xfce4 on Debian testing):

from tkinter import *
from random import randint

root = Tk()

def display(label):
 label.destroy()
 label = Label(text=randint(0, 9))
 label.pack()
 root.after(100, display, label)

display(Label())
mainloop()
  
This opens a tiny window that displays a random digit on a new label

every .1 second. (Obviously I could do this by updating the text rather
than recreating the label, but my real application has to destroy
widgets and create new ones).

This works for 3-4 hours, but eventually the window freezes.



I think it is not yet clear, if this is a bug in Tkinter or in Tcl/Tk, 
the underlying scripting language. It might also be platform dependent. 
Are you on Windows? Here is an equivalent Tcl program:


==
package require Tk

proc randint {} {
expr {int(rand()*1000)}
}

proc display {label} {
destroy $label
set id [randint]
set label [label .l$id -text [randint]]
pack $label
after 100 [list display $label]
}

display [label .l]



Can you run this and check that the freeze also occurs? If you can't 
execute the Tcl that is used by Python directly, you may also do 
something like



root = Tk()
root.eval('Here comes the Tcl code')
root.mainloop()

Can you also find out what version of Tcl/Tk you are using? Try

root.eval('info patchlevel')

Christian

--
https://mail.python.org/mailman/listinfo/python-list

Re: Python 0.9.1

[Christian Gollwitzer]

Am 16.02.21 um 22:57 schrieb Skip Montanaro:

A note to webmas...@python.org from an astute user named Hiromi in
Japan* referred
us to Guido's shell archives for the 0.9.1 release from 1991.



I then pushed the result to a Github repo:

https://github.com/smontanaro/python-0.9.1



That's a nice find!

Reading the README, it occured to me that it refers to the shell, awk 
and perl as contenders for Python[1], but not to Tcl. I would have 
thought that in 1991, Tcl was one of the obvious choices as an extension 
language. At that time, it had already progressed to version 6 [2]


Was Guido not aware of Tcl, or did he not think that it is a contender 
in the same area?


    Christian




[1] "Python, an extensible interpreted programming language [...] can be 
used instead of shell, Awk or Perl scripts, to write prototypes of real 
applications, or as an extension language of large systems, you name it."


[2] https://wiki.tcl-lang.org/page/Tcl+chronology

--
https://mail.python.org/mailman/listinfo/python-list

Re: New Python implementation

[Christian Gollwitzer]

I agree to all the rest of your post, but this:

Am 16.02.21 um 09:57 schrieb Tarjei Bærland:

I am not sure I agree that a language like Scheme or Logo or Brainfuck, with
their small number of building blocks, would be harder to learn.



is strange. I'm not sure, have you actually looked at Brainfuck? Maybe 
there is also confusion what means "learning" a programming language. 
For me, learning a language does not mean to remember the rules and 
keywords, but to be able to write useful programs. Indeed, Brainfuck 
with its 8 commands is easy to remember, but it comes at a very high 
price: you can't do anything useful with it with reasonable effort. It 
is unusable even for pure computer science stuff. It is easy to see that 
BF is Turing complete, so please write a BF program to compute the 
ackermann function. Should be easy, just three rules ;) I'd definitely 
choose Python to do it here.


In that sense, Scheme also appears to be the Brainfuck of functional 
programming to me. It is not much more than the pure untyped lambda 
calculus, and by definition this allows you to compute anything, just 
like Brainfuck is a Turing machine. Actually it is impressive that you 
can write actual useful code with such a minimalist language (infix 
math? Pure bloat!). OTOH it feels like "assembly" compared to more 
evolved functional languages like, e.g. Haskell.


  Christian
--
https://mail.python.org/mailman/listinfo/python-list

Re: New Python implementation

[Christian Gollwitzer]

Am 16.02.21 um 06:36 schrieb dn:

Pascal's value as a teaching language was that it embodied many aspects
of structured programming, and like Python, consisted of a limited range
of items which could be learned very quickly (in contrast to PL/I's many
'bells and whistles'). 


ROFL. Maybe that was true for Python when it was first invented. Today 
it is not "a few simple things". Even just the core language, anything 
that's built into the interpreter if you leave out any standard 
function, is enormous. To name a few: List comprehension, format 
strings, iterator protocol, asynchronous programming, everything called 
__dunderland. A minimal language with only very few basic rules, that 
would be Scheme e.g. Of course, it doesn't mean that Scheme is easier to 
program, but it is easier to write a compiler for it than for Python.


That is a misundestanding often presented - a language that is simple in 
the sense of having a few simple rules, is usually hard to use. (e.g. 
Brainfuck). A language which is easy to use, often comes with a large 
variety of building blocks, to give you the right tool to choose for the 
job at hands (e.g. Python), and therefore is "complex".


Christian

--
https://mail.python.org/mailman/listinfo/python-list

Re: New Python implementation

[Christian Gollwitzer]

Am 15.02.21 um 21:37 schrieb Roel Schroeven:


So your claim is that your compiler is able to, or will be able to, 
compile any language just by specifying a small schema file. Great!


Do you maybe have a proof-of-concept? A simple language with a simple 
schema file to test the basic workings of your compiler, 


Here is the git repo:

https://github.com/i42output/neos

under languages/ you'll find different schema files.

    Christian
--
https://mail.python.org/mailman/listinfo/python-list

Re: New Python implementation

[Christian Gollwitzer]

Am 14.02.21 um 11:12 schrieb Paul Rubin:

Christian Gollwitzer  writes:

He wants that neoGFX is scriptable in Python, but instead of linking
with CPython, he will write his own Python implementation instead,
because CPython is slow/not clean/ whatever. He doesn't seem to
understand that this is an enormous task on its own, because the
interesting part of a scripting language is the standard library with
many decade-years of work.


I wonder how big an issue the stdlib really is.  Lots of it is written
in Python and can port to another interpreter.  Lots more is CPython C
API bindings to external C libraries (e.g. OpenSSL) so porting those
would be a matter of rebinding those libraries to libffi in the cases
where that hasn't been done already.


I'm not saying that it is unfeasible or very difficult. I'm saying that 
it is a lot of work, and for a single developer who has this as a side 
project / support for his graphics engine and who wants to beat existing 
implementations wrt. speed, I'm saying it is going to take a lot of 
time. It'definitely impossible by "defining a few JSON schema files", as 
Leigh claims with his "universal compiler". There definitely IS a lot of 
stuff in a baseline CPython interpreter - a (seemingly) simple thing 
like "print" will have an implementation of 1000 lines in C with all the 
formatting library, file I/O etc. Arbitrary precision integers - another 
library, networking - yet another and so on.



CPython really is pretty slow, maybe because it uses so much boxed data,
derps around with refcounts all the time, suffers memory fragmentation
from not having a relocating GC, etc.  And that is before we even
mention the GIL.


I don't argue with that. CPython is slow. But I'm arguing that you can't 
make it faster by multiple orders of magnitude unless you change the 
language. For numerical code, straight-forward C is usually 100x to 
1000x faster than Python, and you can reach the same performance by 
giving up dynamic typing - Cython demonstrates this quite convincingly. 
If you don't want static typing (with types close to the machine like 
fixed-width integers) than you'll have to resort to a LOT of black magic 
to even come close (PyPy, modern JS engines, )


Christian

--
https://mail.python.org/mailman/listinfo/python-list

Re: New Python implementation

[Christian Gollwitzer]

Am 14.02.21 um 01:19 schrieb Chris Angelico:

On Sun, Feb 14, 2021 at 11:14 AM Mr Flibble
 wrote:


On 13/02/2021 23:30, Igor Korot wrote:

Hi,
But most importantly - what is the reason for this ?
I mean - what problems the actual python compiler produce?

Thank you.


I am creating neos as I need a performant scripting engine for my other major project 
"neoGFX" and I want to be able to support multiple popular scripting languages 
including Python.



Until you have actually produced a (mostly) compatible Python
implementation, can you please stop making these repeated and baseless
jabs at CPython's performance? You keep stating or hinting that
CPython is somehow unnecessarily slow, but unless you have some code
to back your claims, this is nothing but mudslinging.


This is a message to all commentators on this list: Mr Flibble is known 
on comp.lang.c++ for making very confident claims and announcements of 
the greatest software product since the invention of the wheel.


He is indeed a capable C++ programmer, but he frequently underestimates 
the work needed for such big projects so that you can easily count them 
as vaporware for now. For example, he created his own "cross-platform, 
modern" GUI toolkit neoGFX[*] just because he thinks that QT is bad (it 
uses non-standard C++). Of course, cross-platform for now means it runs 
on Windows and anything else is far into the future, also the optical 
design looks modern / OK right now, but will look outdated soon. There 
is no evidence that any other designer will join the project to keep the 
look up to date.


He wants that neoGFX is scriptable in Python, but instead of linking 
with CPython, he will write his own Python implementation instead, 
because CPython is slow/not clean/ whatever. He doesn't seem to 
understand that this is an enormous task on its own, because the 
interesting part of a scripting language is the standard library with 
many decade-years of work.


So my advice is to waste your time in this discussion if you find it 
entertaining, but don't expect to have a usable product soon.


Best regards,

  Christian

[*] https://github.com/i42output/neoGFX
--
https://mail.python.org/mailman/listinfo/python-list

Re: Python subinterpreters with separate GILs

[Christian Heimes]

On 11/02/2021 03.51, James Lu wrote:
> Directly removing the Global Interpreter Lock (GIL) would break a lot
> of libraries that implicitly assume it is there. What if Python had
> "realms" that each had separate GILs?
> 
> The "realms" (not sure if "subinterpreter" is the correct term here)
> could share objects.

Eric Snow and others have been working on that since 2017,
https://www.python.org/dev/peps/pep-0554/

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Convert MBOX thunderbird to PST outlook

[Christian Gollwitzer]

Am 08.02.21 um 11:08 schrieb J.O. Aho:


On 08/02/2021 10.22, Paul Rubin wrote:

"J.O. Aho"  writes:

I think most migrated to use IMAP like 30 years ago


That handles email but not calendar or the other stuff.  I think there
are starting to be standards for that, but no idea whether either
Thunderbird or Outlook follows them.


Yes, that is true that the IMAP don't handle the calendar, for those you 
need caldav/webcal and then maybe carddav for address books too and I do 
think those are well supported in Thunderbird, the question is more what 
the web based Outlook handles, 


For thatm you can run radicale:

https://radicale.org/3.0.html

A Python-based calendar server. Then upload your calendars there from 
Thunderbird and download them from Outlook.


    Christian
--
https://mail.python.org/mailman/listinfo/python-list

Re: idlelib re-use

[Christian Gollwitzer]

Am 28.01.21 um 20:57 schrieb Terry Reedy:

On 1/28/2021 5:53 AM, Robin Becker wrote:
I googled in vain for instances where parts of idlelib are re-used in 
a simplistic way. I would like to use the editor functionality in a 
tkinter window and also probably run code in a subprocess.


Are there any examples around that do these sorts of things?


turtledemo reuses IDLE's colorizer and read-only textviews.  I have seen 
occasional hints on stackoverflow of other such uses.


One barrier to reuse is that the parts are highly interconnected, with 
numerous import loops.  (Changing the form of some imports from 'import 
x' to 'from x import y' can make IDLE startup fail.)  Some objects, like 
EditorWindow, are too monolithic.  You cannot put a toplevel inside 
another toplevel.


Yes, you can. There are two possiblities, the one is converting the 
toplevel into a frame with wm_forget(), after which you can simply pack 
it into another frame. The second one is configuring a frame as a 
container, which allows to even embed a window from a foreign 
application. I'm not sure this works on all platforms, though.


Christian
--
https://mail.python.org/mailman/listinfo/python-list

Re: A beginning beginner's question about input, output and . . .

[Christian Gollwitzer]

Am 13.01.21 um 06:24 schrieb Greg Ewing:

On 13/01/21 4:18 am, Grant Edwards wrote:


AFAIK, Python can't be used to write device drivers for any popular OS


At least not until some crazy person embeds Python in the
Linux kernel...



 What do you mean, "until" ?

https://medium.com/@yon.goldschmidt/running-python-in-the-linux-kernel-7cbcbd44503c

http://www.kplugs.org/

    Christian
--
https://mail.python.org/mailman/listinfo/python-list

Re: dayofyear is not great when going into a new year

[Christian Gollwitzer]

Am 05.01.21 um 23:56 schrieb Eli the Bearded:

Elijah
--
also finds "week starts on Monday" to be oddball about ISO-8601



In Europe, the week starts on Monday - hence, Saturday and Sunday are 
the last days of the week or the "weekend". Starting on Sunday is weird 
for us, because then the weekend is split into the first and last day of 
the week (?) - even if, historically, the week was the time from Sunday 
to Sunday.



Christian
--
https://mail.python.org/mailman/listinfo/python-list

Re: tkinter: creating/attaching menubar to top level window

[Christian Gollwitzer]

Am 08.01.21 um 22:47 schrieb Rich Shepard:

I'm using Chapter 9 in Mark Roseman's "Modern Tkinter for Busy Python
Developers" to learn how to write a top level menu. MWE code is attached.

Python3 tells me there's invalid syntax on line 42:
     self.['menu'] = menubar # attach it to the top level window
  ^
yet that's the syntax he prints on page 84 (and has in the book's code
supplement).


It is a simple typo, remove the dot.

self['menu'] = menubar

It will then stop at the add_cascade, fix it like this:

 menubar.add_cascade(menu=self.menu_file, label='File')

and then it works,


Why am I getting an invalid syntax error here?


Because the dot would indicate the access of an attribute. but no name 
follows. What it does here, instead, is indexing - the correct line is 
similar to setting a dict entry.


Christian
--
https://mail.python.org/mailman/listinfo/python-list

Re: Class and tkinter problem

[Christian Gollwitzer]

Am 07.01.21 um 08:29 schrieb Paulo da Silva:


Does anybody know why cmd method isn't called when I change the button
state (clicking on it) in this example?
I know that this seems a weird class use. But why doesn't it work?
Thanks.

class C:
 from tkinter import Checkbutton
 import tkinter

 @staticmethod

^^it works if you remove the staticmethod here


 def cmd():
 print("Test")



Maybe there is a bug in tkinter, that it doesn't work with static methods?

Christian
--
https://mail.python.org/mailman/listinfo/python-list

Re: I'm finally disentangled from Python 2, thank you everyone

[Christian Gollwitzer]

Am 30.12.20 um 11:58 schrieb Chris Green:

Could I ask you to write up a post on what you did here? I've never used
cx-freeze but it sounds like a useful thing for keeping legacy stuff
functioning. A writeup from someone who's actually used it for that
would be welcome.


Of course, here is what I wrote in my 'self help' Dokuwiki wiki about it. It 
refers
specifically to the OKI software I wanted to keep using but it should be fairly
easy to apply a similar process to other software.


 I asked on the Python newsgroup and the one suggestion that seemed 
feasible was to
 package the OKI software with all its dependencies on a system which still 
has Gtk2
 and Python 2 and then install the whole package on esprimo.

 After a bit of looking around at Snap, Appimage and such I found cx_freeze 
which is
 aimed specifically at Python.  The latest version doesn't support Python 2 
but 5.1.1
 does, so this is how I've have done it

   * Install xubuntu 18.04 on my old Revo system (has to be 64-bit), 18.04 
is still in support. (has to be 64-bit simply because the final target is 
64-bit)
   * Install cx_freeze 5.1.1 on Revo
   * Install the Oki software on Revo, check that it works, pulls in lots 
of libraries and packages.
   * Run 'cxfreeze /usr/libexec/okimfputl/scantool.py' (scantool.py is the 
utility I want to run on systems without Python 2)
   * Copy the resulting 'dist' directory to the target system, name isn't 
critical

 Then the fun starts.  There's quite a few more libraries and packages are 
required
 and the scan daemon needs to be runnable.  (The scan daemon was, 
fortunately, just
 a compiled program so ran 'as is' without issues)

 Files needed in /usr/libexec/okimfputl and /usr/libexec/okimfpdrv
 """""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""
 There are a lot of hard coded references to /usr/libexec/okimfputl and
 /usr/libexec/okimfpdrv, it **might** be possible to change all these but
 I decided it would be less painful to use a couple of symbolic links
 to locations in /usr/local/libexec and put the required files in
 /usr/local/libexec/okimfputl and /usr/local/libexec/okimfpdrv.

 I discovered what files are needed in these directories by simply running
 scantool on the target and fixing each error as it arose.

 Other Python package and library files
 """""""""""""""""""""""""""""""""""""
 I have installed the "dist" created by cxfreeze in 
/usr/local/libexec/okicxfreeze. The executable to run is thus 
/usr/local/libexec/okicxfreeze/scantool.

 There are also a few .so libraries and Python packages needed, as above I
 just found these by running scantool and fixing each error as it appeared.
 The system library files are put in /usr/local/lib, you have to run 
ldconfig
 after each file is put there.  The Python packages (including the dreaded
 pyscand.so) have to be put in the working directory when scantool is run,
 so I have put them in /usr/local/libexec/okicxfreeze where the scantool
 executable is.


I hope the above is useful.  As I said it refers specifically to the scantool.py
that I needed for my Oki scanner but I think the description is general enough 
to
be useful.

The basic process is:-

 Find (or create) a system where the software you want to run works OK.

 Install cx-freeze 5.1.1 on that system and run 'cxfreeze '

 Check that the executable created by cxfreeze works on the system you 
built it on

 Copy the executable (and its 'dist' environment) to the target where you 
want to run it

 Try and run it on the target

 Iteratively work through the errors it throws up when you run it, in my 
case these were:-

 Missing .so system library files, copy them from the build system to 
somewhere
 they will be found on the target.  You could put them in a 'private to 
the
 package' directory and set LD_LIBRARY_PATH or do as I did and put them 
in
 a standard library location (and run ldconfig after adding each).



I've used pyinstaller in the past, and it seems to do a better job with 
that. It usually copies all the sytem libraries, too, but would fail 
with /usr/libexec/okimfputl  & friends


Christian

--
https://mail.python.org/mailman/listinfo/python-list

Re: Debian testing and virtual environment error message

[Christian Heimes]

On 27/12/2020 04.05, songbird wrote:
> songbird wrote:
> ...
>>   The package mentioned is installed:
>>
>>
>> =
>>
>> $ dpkg -l | grep python3-venv
>> ii  python3-venv 3.9.0-4
>> amd64pyvenv-3 binary for python3 (default python3 version)
>>
> 
>   here is something i missed including in my first post, but i don't
> know if this matters:
> 
> ii  python3.9-venv   3.9.1-1amd64 
>Interactive high-level object-oriented language (pyvenv binary, 
> version 3.9)

Don't worry, you are not the first person that ran into this issue.

Debian's packaging of Python has caused multiple issues. I have reported
some issues to downstream Debian/Ubuntu. You can find a list of all
known issues at
https://gist.github.com/tiran/2dec9e03c6f901814f6d1e8dad09528e

Christian

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Function returns old value

[Christian Gollwitzer]

Am 17.12.20 um 21:22 schrieb Michael F. Stemper:

On 17/12/2020 03.57, Peter J. Holzer wrote:

On 2020-12-17 03:06:32 -, Bischoop wrote:

pasting from my IDE to vim/slrn was messing syntax,


You can

:set paste

in vim to prevent it from messing with pasted content (don't forget to
set nopaste afterwards).


What's the difference between that and
:set ai
and
:set noai


With newer vims that's rarely necessary though since they can distinguish
between input that was pasted and input that was typed.


I thought that I was going nuts when I encountered that. Any idea how to
defeat such a so-called 'feature"?


Use "gvim" instead of "vim" in a Terminal. The problems arises because 
vim in the terminal simply gets the input and "thinks" that you typed 
that stuff in, for which it does auto-indenting. gvim, OTOH, doe sthe 
pasting by itself. It also has other useful features like menus and 
popup-dialogs for searching etc.


Christian

--
https://mail.python.org/mailman/listinfo/python-list

Re: linear algebric equations

[Christian Gollwitzer]

Am 07.12.20 um 17:59 schrieb Tito Sanò:

Regarding the solution of linear algebraic equations I noticed a big
difference in the computation

time in Python compared to the old fortran language.

I have compared both the linelg and lapack.dgesv-lapack.zgesv modules with
the fortan: dgelg and f04adf.

The difference in computation time is enormous:

for example for 430 degrees of freedom it is about 24 min in Python versus
about 1 sec in fortran.


There must be something seriously wrong. If I understand correctly, you 
want to solve an 430x430 matrix with LU decompositino (that is what 
dgesv from LAPACK does). The following code takes less than a second on 
my machine:


==430.py==
import numpy as np

# create a 430x430 random matrix
A = np.random.randn(430,430)

# right hand side
b = np.ones(430)

# solve it

x = np.linalg.solve(A, b)

print(x)



time python3 430.py
real0m0.318s
user0m0.292s
sys 0m0.046s

If it takes longer than 1s, there is something wrong with your system.

Christian
--
https://mail.python.org/mailman/listinfo/python-list

Re: how to plot the FFT of a list of values

[Christian Gollwitzer]

Am 05.12.20 um 18:16 schrieb Boris Dorestand:

I have 16 values of the period sequence 1, 2, 4, 8, 1, 2, 4, 8, ...  I
compute its fourier transform using


from scipy import fft, ifft
x = [1,2,4,8,1,2,4,8]
fft(x)

array([ 30. +0.j,   0. +0.j,  -6.+12.j,   0. +0.j, -10. +0.j,   0. +0.j,
 -6.-12.j,   0. +0.j])

Now how can I plot these values?  I would like to plot 16 values.  What
do I need to do here?  Can you show an example?



Usually, for the FFT of real input data, you plot only the magnitude or 
square of the complex array, and usually on a logscale. So:


import pylab
import numpy as np

fx = fft(x)

pylab.semilogy(np.abs(fx))
pylab.show()



Christian



--
https://mail.python.org/mailman/listinfo/python-list

Re: Why can't numpy array be restored to saved value?

[Christian Gollwitzer]

Am 25.11.20 um 07:47 schrieb pjfarl...@earthlink.net:

Why isn't the final value of the numpy array npary in the following code the
same as the initial value before some but not all elements of the array were
changed to a new value?

I know I am missing something basic here.  I thought I understood the
concepts of immutable vs mutable values but obviously I missed something.

My environment is Win10-64, Python 3.8.5, numpy 1.19.2.

Code and output follows.  TIA for any help you can provide to cure my
ignorance.

Peter

--- nptest.py ---
import numpy as np
import sys

if len(sys.argv) > 0:
 try:
 asz = int(sys.argv[1]) + 0
 except:
 asz = 4

npary = np.full([asz, asz, asz], 0, dtype=np.int32)
print("Array before change=\n{}".format(npary))
svary = npary[:, :, :]


Because this does not copy the array, rather it creates a view into the 
original array. This is an optimization to avoid copying. If you want a 
copy, do svary = npary.copy()


Christian
--
https://mail.python.org/mailman/listinfo/python-list

Re: Strange terminal behavior after quitting Tkinter application

[Christian Gollwitzer]

Am 03.11.20 um 23:34 schrieb Dennis Lee Bieber:



Out of curiosity, does Python on Linux honor the .pyw extension?

On Windows, .pyw indicates a Python program that implements a GUI and
will NOT make use of console (stdin/stdout/stderr).


On Linux, there is no such distinction. On Windows it is only needed 
because, if you connect stdin/out, a terminal window pops up. >For a 
true GUI program that is notr acceptable, the user will be puzzled what 
this ugly useless window wants to do, and therefore a flag in the EXE 
file format indicates to Windows if it should pop up the console or not.


On Linux, stdin/out is always connected. You must run your program from 
a terminal window to see it, otherwise it is silently connected to some 
channel in the background by the desktop environment. It can happen that 
the standard channels are closed, if you run a program in the terminal 
and then close the terminal (which sends SIGHUP to the program). In this 
case the program might later on throw I/O errors, when printing to stdout.


    Christian

--
https://mail.python.org/mailman/listinfo/python-list

Re: GUI (tkinter) popularity and job prospects for

[Christian Gollwitzer]

Am 22.10.20 um 23:52 schrieb Paul Rubin:

Michael Torrie  writes:

I doubt you'll find any jobs connected a particular Python GUI toolkit.



It would be really nice if there was a way to straightforwardly run
Tkinter applications on Android.  You'd install a single .apk and that
would let you run your Tkinter apps and make launchers for them.


It's not unrealistic - there is AndroWish to run Tcl/Tk on Android 
including GUI and connection to Android specific services (subcommand 
"borg"). Someone in the Python community would need to make the Tkinter 
bridge with a native Python for Android.



https://www.androwish.org/home/home

Christian

--
https://mail.python.org/mailman/listinfo/python-list

Re: Covariance matrix syntax

[Christian Gollwitzer]

Am 19.10.20 um 07:23 schrieb Meghna Karkera:

I am unable to find the *formula for covariance* used in np.cov syntax in
PYTHON given in link
https://numpy.org/doc/stable/reference/generated/numpy.cov.html.  Could you
please help me out.



As said, you should click on the link [source] just at the right hand 
side of the page, at the top. There is


numpy.cov(m, y=None, rowvar=True, bias=False, ddof=None, fweights=None, 
aweights=None)


and just next to it there is the link. It takes you there:

https://github.com/numpy/numpy/blob/v1.19.0/numpy/lib/function_base.py#L2270-L2481

Which is the source code of the function.

Best regards,

Christian



Thanks
Meghna

On Tue, Oct 13, 2020 at 11:46 AM Christian Gollwitzer 
wrote:


Am 13.10.20 um 06:52 schrieb Meghna Karkera:

Could you let me know what is the back end calculation of this covariance
matrix syntax np.cov



You can look it up yourself: Go to the docs
https://numpy.org/doc/stable/reference/generated/numpy.cov.html

At the right hand side, just right of the function signature, there is a
link [source]. Click there and it takes you to the implementation.

Apparently it is done in straightforward Python.

 Christian

PS: Snipped a lot of unrelated citation at the bottom


On Tue, Oct 13, 2020, 10:14 Bruno P. Kinoshita <

brunodepau...@yahoo.com.br>

wrote:
[...]

I think the np.cov is from the numpy module (imported/aliased as np?).

--
https://mail.python.org/mailman/listinfo/python-list



--
https://mail.python.org/mailman/listinfo/python-list

Re: Covariance matrix syntax

[Christian Gollwitzer]

Am 13.10.20 um 06:52 schrieb Meghna Karkera:

Could you let me know what is the back end calculation of this covariance
matrix syntax np.cov



You can look it up yourself: Go to the docs
https://numpy.org/doc/stable/reference/generated/numpy.cov.html

At the right hand side, just right of the function signature, there is a 
link [source]. Click there and it takes you to the implementation.


Apparently it is done in straightforward Python.

Christian

PS: Snipped a lot of unrelated citation at the bottom


On Tue, Oct 13, 2020, 10:14 Bruno P. Kinoshita 
wrote:
[...]

I think the np.cov is from the numpy module (imported/aliased as np?).

--
https://mail.python.org/mailman/listinfo/python-list