Re: How to package a Python command line app?
On 12/9/21 11:35, Manfred Lotz wrote: I played with pyinstaller which worked fine. However, it builds a dynamic executable and thus it is glibc version dependent. Means, I have to build different executables for differen glibc versions. So, it seems I will have to check how executable zip archives are supposed to work. For me at least, I'm still not sure what you are trying to accomplish. "A Python command line app which requires some packages which are not in the standard library." If your code and its dependencies are "pure python" then you only need to build a wheel package and you should be good to go. If you think you have Python version issues, most of the good checkers will take a version argument, so if you think you need to be compatible with 3.6-3.10 you give it a version of 3.6 to check against, and it should spot things that don't work all the way back to that base version, and then you can find ways to code around it. If the things you call on are binary, it gets a little more complicated. Wheels that contain compiled bits need to match the version of Python that's going to run them (unless they use the stable ABI, but that's not terribly common yet). You shouldn't run into glibc versioning problems. Most of glibc has been extraordinarily stable for nearly two decades, and within the range of distributions you've mentioned there should not be problems unless something is reaching into very esoteric areas. Other system libraries - maybe not so much. If you really think you're going to have this level of binary-compatibility problem, the flavor-of-the-month technique is to build a self-contained bundle, such as a Snap or Flatpak, or... Quote: > New packaging formats like Snap, Flatpak and AppImage are providing distribution agnostic packages that work on most Linux distributions. But that's no longer a Python-specific question at that point... As I said, I can't really deduce the details of what you're trying to accomplish, just hoping you don't buy yourself more trouble than you actually need to - this might be easier than you think. -- https://mail.python.org/mailman/listinfo/python-list
Re: Return
Hello,
On 12/8/21 11:29, vani arul wrote:
Thanks for your help.
I am not good at programming.My code works perfectly.But I am bit
confused about the return values.
Binary Search Program
/def Binarysearch(a,key):
l=0
r=len(a)-1
while l<=r:
med = (l+r)//2
if key==a[med]:
return med
elif key>a[med]:
l=med+1/
/ elif keyBut in the following block,how does it return values if there is no
return specified?
First of all please reply always to the list address!
If Python does not encounter a return statement it will continue to run
the function.
In your example the while loop will continue until the condition 'l<=r'
is true. If Python exits the while loop then there is a return -1.
BR,
Roland
Regards
Vani
On Wed, Dec 8, 2021 at 2:15 AM Roland Mueller
wrote:
Hello
ti 7. jouluk. 2021 klo 19.47 vani arul (arulvan...@gmail.com)
kirjoitti:
Hey There,
Can someone help to understand how a python function can
return value with
using return in the code?
It is not not about explicit or implicit function call.
Not sure whether I understood your question: I have a simple
example about return.
* f() and h() explicitely return something
* g() and h() return None
BTW, also Python documentation tool pydoc has an article about return.
#!/usr/bin/python
def f():
return 42
def g():
pass
def h():
return
if __name__ == "__main__":
print(f"f(): {f()}")
print(f"g(): {g()}")
print(f"h(): {h()}")
Result:
f(): 42
g(): None
h(): None
Pydoc:
$ pydoc return
BR,
Roland
Thanks
Vani
--
https://mail.python.org/mailman/listinfo/python-list
--
https://mail.python.org/mailman/listinfo/python-list
Re: How to package a Python command line app?
On 2021-12-09 19:35:37 +0100, Manfred Lotz wrote: > I played with pyinstaller which worked fine. However, it builds a > dynamic executable and thus it is glibc version dependent. Means, I > have to build different executables for differen glibc versions. Just build it for the oldest version. Even if you don't, it may not matter. I just compiled a hello world program on Ubuntu 20 and ran it on Debian 6 (which is 10 years old). Obviusly a real program has more opportunities to run into compatibility problems, but glibc doesn't change all that much. hp -- _ | Peter J. Holzer| Story must make more sense than reality. |_|_) || | | | h...@hjp.at |-- Charles Stross, "Creative writing __/ | http://www.hjp.at/ | challenge!" signature.asc Description: PGP signature -- https://mail.python.org/mailman/listinfo/python-list
Re: Odd locale error that has disappeared on reboot.
On 2021-12-09 10:38:58 +, Chris Green wrote: > Julio Di Egidio wrote: > > Still your code wouldn't pass review: you do need some exception > > handling there [...] > However catching and re-trying isn't going to help at all. It happily > produced the same arror every 10 minutes throughout the night until I > rebooted the system. I agree. Only catch exceptions which you can reasonably handle. For those which you can't handle (and "some part of my Python installation has just gone AWOL" is one of them) crashing with a stack trace is almost certainly your best option. > I suppose I *could* reboot after (say) three failures Maybe, but I wouldn't do that in the python program itself, but in a separate monitor process. > but it seems a bit drastic! :-) Yup. I'd do that only if I've run out of other options (which has happened in the past and probably will happen in the future). hp -- _ | Peter J. Holzer| Story must make more sense than reality. |_|_) || | | | h...@hjp.at |-- Charles Stross, "Creative writing __/ | http://www.hjp.at/ | challenge!" signature.asc Description: PGP signature -- https://mail.python.org/mailman/listinfo/python-list
Re: How to package a Python command line app?
On Thu, 9 Dec 2021 18:18:26 +0100
"Dieter Maurer" wrote:
> Manfred Lotz wrote at 2021-12-8 12:31 +0100:
> >The are many possibilities to package a Python app, and I have to
> >admit I am pretty confused.
> >
> >Here is what I have:
> >
> >A Python command line app which requires some packages which are not
> >in the standard library.
>
> Are they on PyPI or can they be downloaded to PyPI?
> In this case, you could install it via `pip` and
> a "requirements" file (describing what is necessary).
>
Acutally, those packages are on pypi.
I created a minimal example where I have two files
hello.py
#!/usr/bin/python3
from message import hello
import typer
def main():
hello()
if __name__ == "__main__":
typer.run(main)
message.py
==
def hello():
print("hello world")
pyinstaller worked fine taking care of message.py and typer module. But
as said in my other reply it is glibc version dependent.
> You could also build an archive (e.g. `tar`, `zip`)
> and create small installer script which unpacks and installs as
> necessary. You could put the archive at the end of the script (such
> that you have a single file which gets executed to do everything).
This I will try next.
--
Manfred
--
https://mail.python.org/mailman/listinfo/python-list
Re: How to package a Python command line app?
On Thu, 9 Dec 2021 17:34:03 + Barry wrote: > > On 8 Dec 2021, at 18:27, Manfred Lotz wrote: > > > > The are many possibilities to package a Python app, and I have to > > admit I am pretty confused. > > > > Here is what I have: > > > > A Python command line app which requires some packages which are > > not in the standard library. > > > > I am on Linux and like to have an executable (perhaps a zip file > > with a shebang; whatever) which runs on different Linux systems. > > > > Different mean > > - perhaps different glibc versions > > - perhaps different Python versions > > > > In my specific case this is: > > - RedHat 8.4 with Python 3.6.8 > > - Ubuntu 20.04 LTS with Python 3.8.10 > > - and finally Fedora 33 with Python 3.9.9 > > > > > > Is this possible to do? If yes which tool would I use for this? > > Have a look at pyinstaller it knows how to collect up all your > dependancies and build a folder of files you can distribute. It’s > also got a single file mode that you might find useful. The end user > does not need to install python. > > I am looking at using it to build gui apps a macOs. > > You will have to experiment to find out if it solves you glib case > concerns. > I played with pyinstaller which worked fine. However, it builds a dynamic executable and thus it is glibc version dependent. Means, I have to build different executables for differen glibc versions. So, it seems I will have to check how executable zip archives are supposed to work. -- Manfred -- https://mail.python.org/mailman/listinfo/python-list
Re: How to package a Python command line app?
> On 8 Dec 2021, at 18:27, Manfred Lotz wrote: > > The are many possibilities to package a Python app, and I have to admit > I am pretty confused. > > Here is what I have: > > A Python command line app which requires some packages which are not in > the standard library. > > I am on Linux and like to have an executable (perhaps a zip file with a > shebang; whatever) which runs on different Linux systems. > > Different mean > - perhaps different glibc versions > - perhaps different Python versions > > In my specific case this is: > - RedHat 8.4 with Python 3.6.8 > - Ubuntu 20.04 LTS with Python 3.8.10 > - and finally Fedora 33 with Python 3.9.9 > > > Is this possible to do? If yes which tool would I use for this? Have a look at pyinstaller it knows how to collect up all your dependancies and build a folder of files you can distribute. It’s also got a single file mode that you might find useful. The end user does not need to install python. I am looking at using it to build gui apps a macOs. You will have to experiment to find out if it solves you glib case concerns. Barry > > > -- > Manfred > > -- > https://mail.python.org/mailman/listinfo/python-list > -- https://mail.python.org/mailman/listinfo/python-list
Re: How to package a Python command line app?
Manfred Lotz wrote at 2021-12-8 12:31 +0100: >The are many possibilities to package a Python app, and I have to admit >I am pretty confused. > >Here is what I have: > >A Python command line app which requires some packages which are not in >the standard library. Are they on PyPI or can they be downloaded to PyPI? In this case, you could install it via `pip` and a "requirements" file (describing what is necessary). You could also build an archive (e.g. `tar`, `zip`) and create small installer script which unpacks and installs as necessary. You could put the archive at the end of the script (such that you have a single file which gets executed to do everything). -- https://mail.python.org/mailman/listinfo/python-list
Re: HTML extraction
Pieter van Oostrum wrote at 2021-12-8 11:00 +0100: > ... >bs4 can do it, but lxml wants correct XML. Use `lxml's the `HTMLParser` to parse HTML (--> "see https://lxml.de/parsing.html#parsing-html";). -- https://mail.python.org/mailman/listinfo/python-list
Re: help
Hi, On Thu, Dec 9, 2021 at 10:31 AM smita wrote: > > > >I am not able to open python on my laptop plzz help What do you mean by saying "open python"? Thank you. > > > >Sent from [1]Mail for Windows > > > > References > >Visible links >1. https://go.microsoft.com/fwlink/?LinkId=550986 > -- > https://mail.python.org/mailman/listinfo/python-list -- https://mail.python.org/mailman/listinfo/python-list
Re: help
On 12/9/21 03:39, smita wrote: I am not able to open python on my laptop plzz help You're going to have to provide some details. -- https://mail.python.org/mailman/listinfo/python-list
Re: Odd locale error that has disappeared on reboot.
Inada Naoki wrote: > On Wed, Dec 8, 2021 at 2:52 AM Chris Green wrote: > > > > > > At 03:40 last night it suddenly started throwing the following error every > > time it ran:- > > > > Fatal Python error: initfsencoding: Unable to get the locale encoding > > LookupError: unknown encoding: UTF-8 > > > > Current thread 0xb6f8db40 (most recent call first): > > Aborted > > > > Running the program from the command line produced the same error. > > Restarting the Pi system has fixed the problem. > > > > This error means Python can not find its standard libraries. There are > some possibilities. > > * You set the wrong PYTHONHOME > PYTHONHOME is very rarely useful. It shouldn't be used if you can > not solve this kind of problem. > > * Your Python installation is broken. > Some files are deleted or overwritten. You need to *clean* install > Python again. > So how was it that rebooting the system fixed the error? It has been working perfectly OK now for a few days since the above failure and I've changed nothing. I think it was probably a bit of power supply noise or something that corrupted a memory read or write, or something of that sort anyway. -- Chris Green · -- https://mail.python.org/mailman/listinfo/python-list
Re: How to package a Python command line app?
Hi Manfred, Manfred Lotz writes: > Hi Loris, > > On Wed, 08 Dec 2021 15:38:48 +0100 > "Loris Bennett" wrote: > >> Hi Manfred, >> >> Manfred Lotz writes: >> >> > The are many possibilities to package a Python app, and I have to >> > admit I am pretty confused. >> > >> > Here is what I have: >> > >> > A Python command line app which requires some packages which are >> > not in the standard library. >> > >> > I am on Linux and like to have an executable (perhaps a zip file >> > with a shebang; whatever) which runs on different Linux systems. >> > >> > Different mean >> > - perhaps different glibc versions >> > - perhaps different Python versions >> > >> > In my specific case this is: >> > - RedHat 8.4 with Python 3.6.8 >> > - Ubuntu 20.04 LTS with Python 3.8.10 >> > - and finally Fedora 33 with Python 3.9.9 >> > >> > >> > Is this possible to do? If yes which tool would I use for this? >> >> I use poetry[1] on CentOS 7 to handle all the dependencies and create >> a wheel which I then install to a custom directory with pip3. >> >> You would checkout the repository with your code on the target system, >> start a poetry shell using the Python version required, and then build >> the wheel. From outside the poetry shell you can set PYTHONUSERBASE >> and then install with pip3. You then just need to set PYTHONPATH >> appropriately where ever you want to use your software. >> > > In my case it could happen that I do not have access to the target > system but wants to handover the Python app to somebody else. This > person wants just to run it. For what ever reasons, there does not seem to be much focus on this kind of deployment for Python. Similar to the way things are with Perl, the assumption seems to be that you have a working environment and install any dependencies needed to get the program you have been given working. I have never used it, but you might want to look at something like pyinstaller https://pyinstaller.readthedocs.io However, it looks as if it is aimed towards bundling a single script. I don't know how it would work if you have a more complex program consisting of a number of modules. >> Different Python versions shouldn't be a problem. If some module >> depends on a specific glibc version, then you might end up in standard >> dependency-hell territory, but you can pin module versions of >> dependencies in poetry, and you could also possibly use different >> branches within your repository to handle that. >> > > I try to avoid using modules which depeng on specific glibc. So would I, but you mentioned it above in your definition of 'different'. > Although, it seems that it doesn't really help for my use case I will > play with poetry to get a better understanding of its capabilities. You're right, poetry doesn't seem to address your main problem. Nevertheless, it might be useful for developing your program before you get to the question of how to distribute it Cheers, Loris -- This signature is currently under construction. -- https://mail.python.org/mailman/listinfo/python-list
Re: Odd locale error that has disappeared on reboot.
Julio Di Egidio wrote: > On 08/12/2021 10:50, Chris Green wrote: > > Julio Di Egidio wrote: > >> On 07/12/2021 16:28, Chris Green wrote: > >>> What could have caused this? I certainly wasn't around at 03:40! :-) > >>> There aren't any automatic updates enabled on the system, the only > >>> thing that might have been going on was a backup as that Pi is also > >>> my 'NAS' with a big USB drive connected to it. The backups have been > >>> running without problems for more than a year. Looking at the system > >>> logs shows that a backup was started at 03:35 so I suppose that *could* > >>> have provoked something but I fail to understand how. > >> > >> Since it's a one-off, doesn't sound like a system problem. The easiest > >> might be that you try-catch that call and retry when needed, and I'd > >> also check that 'ftxt' is what it should be: "external devices" may > >> fail, including when they do produce output... > >> > > Well it repeated every ten minutes through the night until I rebooted > > the system, so it wasn't really a "one off". I hasn't repeated since > > though. > > Still your code wouldn't pass review: you do need some exception > handling there, that it's not failing right now is no excuse. Moreover, > next time it fails, you won't be any wiser since you keep not handling > it... Unless this is all just for fun, in which case of course never mind. > It is basically 'just for fun' we wanted to see how warm/cold the cats' home above our garage was now that winter has arrived. However catching and re-trying isn't going to help at all. It happily produced the same arror every 10 minutes throughout the night until I rebooted the system. I suppose I *could* reboot after (say) three failures but it seems a bit drastic! :-) -- Chris Green · -- https://mail.python.org/mailman/listinfo/python-list
help
I am not able to open python on my laptop plzz help Sent from [1]Mail for Windows References Visible links 1. https://go.microsoft.com/fwlink/?LinkId=550986 -- https://mail.python.org/mailman/listinfo/python-list
Re: Python child process in while True loop blocks parent
> On 8 Dec 2021, at 17:11, Jen Kris wrote:
>
> I started this post on November 29, and there have been helpful comments
> since then from Barry Scott, Cameron Simpson, Peter Holzer and Chris
> Angelico. Thanks to all of you.
>
> I've found a solution that works for my purpose, and I said earlier that I
> would post the solution I found. If anyone has a better solution I would
> appreciate any feedback.
>
> To recap, I'm using a pair of named pipes for IPC between C and Python.
> Python runs as a child process after fork-execv. The Python program
> continues to run concurrently in a while True loop, and responds to requests
> from C at intervals, and continues to run until it receives a signal from C
> to exit. C sends signals to Python, then waits to receive data back from
> Python. My problem was that C was blocked when Python started.
>
> The solution was twofold: (1) for Python to run concurrently it must be a
> multiprocessing loop (from the multiprocessing module), and (2) Python must
> terminate its write strings with \n, or read will block in C waiting for
> something that never comes. The multiprocessing module sidesteps the GIL;
> without multiprocessing the GIL will block all other threads once Python
> starts.
>
> Originally I used epoll() on the pipes. Cameron Smith and Barry Scott
> advised against epoll, and for this case they are right. Blocking pipes work
> here, and epoll is too much overhead for watching on a single file
> descriptor.
>
> This is the Python code now:
>
> #!/usr/bin/python3
> from multiprocessing import Process
You already have feedback that multiprocessing is not required.
> import os
>
> print("Python is running")
>
> child_pid = os.getpid()
> print('child process id:', child_pid)
>
> def f(a, b):
>
> print("Python now in function f")
>
> pr = os.open('/tmp/Pipe_01', os.O_RDONLY)
> print("File Descriptor1 Opened " + str(pr))
> pw = os.open('/tmp/Pipe_02', os.O_WRONLY)
> print("File Descriptor2 Opened " + str(pw))
>
> while True:
>
> v = os.read(pr,64)
> print("Python read from pipe pr")
> print(v)
>
> if v == b'99':
> os.close(pr)
> os.close(pw)
> print("Python is terminating")
> os._exit(os.EX_OK)
>
> if v != "Send child PID":
> os.write(pw, b"OK message received\n")
The \n should not be required as UDS (unix domain sockets) are message based
not a stream of bytes.
> print("Python wrote back")
>
> if __name__ == '__main__':
> a = 0
> b = 0
> p = Process(target=f, args=(a, b,))
> p.start()
> p.join()
>
> The variables a and b are not currently used in the body, but they will be
> later.
>
> This is the part of the C code that communicates with Python:
>
> fifo_fd1 = open(fifo_path1, O_WRONLY);
> fifo_fd2 = open(fifo_path2, O_RDONLY);
>
> status_write = write(fifo_fd1, py_msg_01, sizeof(py_msg_01));
> if (status_write < 0) perror("write");
You continue on after the error, exit would be better.
>
> status_read = read(fifo_fd2, fifo_readbuf, sizeof(py_msg_01));
> if (status_read < 0) perror("read");
> printf("C received message 1 from Python\n");
> printf("%.*s",(int)buf_len, fifo_readbuf);
The length of the data read is in status_read not buf_len.
>
> status_write = write(fifo_fd1, py_msg_02, sizeof(py_msg_02));
How much data did you put into py_msg_01 buffer?
Is it a C string? If so you want to write sizeof(py_msg_02)-1 to avoid sending
the trailing NUL (0)
of the C string.
> if (status_write < 0) perror("write");
If it failed exit until you have figured out the error handling.
>
> status_read = read(fifo_fd2, fifo_readbuf, sizeof(py_msg_02));
> if (status_read < 0) perror("read");
> printf("C received message 2 from Python\n");
> printf("%.*s",(int)buf_len, fifo_readbuf);
The length of the data read is in status_read not buf_len.
At no point do I see in this C code the need for a \n in the data sent between
python and C.
>
> // Terminate Python multiprocessing
> printf("C is sending exit message to Python\n");
> status_write = write(fifo_fd1, py_msg_03, 2);
Would be better to not be using "2" for the length to write here.
If py_msg_03 only has the exit msg in it the use sizeof(py_msg_03).
If its a C string the subtract 1 for the trailing NUL (0).
>
> printf("C is closing\n");
> close(fifo_fd1);
> close(fifo_fd2);
>
> Screen output:
>
> Python is running
> child process id: 5353
> Python now in function f
> File Descriptor1 Opened 6
> Thread created 0
> File Descriptor2 Opened 7
> Process ID: 5351
> Parent Process ID: 5351
> I am the parent
> Core joined 0
> I am the child
> Python read from pipe pr
> b'Hello to Python from C\x00\x00'
The \x00 is because the length you used is wrong.
> Python wrote back
> C received message 1 from Python
> OK message received
