Re: Python script not letting go of files

[Mike Dewhirst]

Stefan

Thank you. I should have said this has been working fine for years and 
years until Ubuntu 2022.04 on a new droplet running Apache/2.4.52


I will refactor it one day - especially if the script is implicated. But 
I think I have to learn how to use lsof first!


Cheers

Mike

On 30/11/2022 1:44 am, Stefan Ram wrote:

"Weatherby,Gerard"  writes:

Do any of you Python folks see any blunders in the above code along the
lines of not letting go of py files or static assets?

   No, sorry. But I had to refactor it.

   *Warning* The following code is not intended to be executed,
   it is just intended to be read by humans. It was not tested
   and, therefore, it probably contains errors. If executed, it
   may cause destruction such as deletions of files etc. Stefan
   Ram (I) does not claim any copyright for his modifications.

def attempt( task, command ):
 if trycmd( command, log ):
 print( f'task "{task}" failed.'  )
 return False # failure
 else:
 return True # success

def main():
 with open( fetchlog, 'a' )as log_:
 global log
 log = log_
 remove_site()
 del log

def remove_site():
 log.write( f"\n\nFetching {tag}" )
 if attempt( "remove site", f"sudo rm -Rf {site_root}", log ):
 get_source()

def get_source():
 if attempt( "get source", f"sudo svn export --force --username {usr} --password 
{pw} {svn_repo} {site_root}" ):
 install_requirements()

def install_requirements():
 if attempt( "install requirements", f"sudo -H pip install -r {reqfile}" ):
 repo_migrations()

def repo_migrations():
 if attempt( "run any migrations shipped from the repo", f"sudo python3 
{site_root}/manage.py migrate --noinput --settings={settings}" ):
 makemigrations()

def makemigrations():
 # shouldn't find anything
 if attempt( "makemigrations", f"sudo python3 {site_root}/manage.py 
makemigrations --noinput --settings={settings}" ):
 migrate()

def migrate():
 # should already be done
 if attempt( "migrate 2", f"sudo python3 {site_root}/manage.py migrate --noinput 
--settings={settings}" ):
 remove_static()

def remove_static():
 if attempt( "remove all static files from their Apache dir", f"sudo rm -Rf 
{static_root}/*" ):
 copy_static()

def copy_static():
 if attempt( "copy all static files to the Apache location", f"sudo python3 
{site_root}/manage.py collectstatic --noinput --settings={settings}" ):
 set_permissions()

def set_permissions():
 if attempt( "set all permissions", f"sudo {scripts}/perms_{host}.sh" ):
 service_restart()

def service_restart():
 if attempt( "restart of Apache service", f"sudo service apache2 restart" ):
 log_success()

def log_success():
 log.write( f"\nFinish {tag}\n\n" )

# main()
# sys.exit()





--
Signed email is an absolute defence against phishing. This email has
been signed with my private key. If you import my public key you can
automatically decrypt my signature and be sure it came from me. Just
ask and I'll send it to you. Your email software can handle signing.



OpenPGP_signature
Description: OpenPGP digital signature
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Python script not letting go of files

[Weatherby,Gerard]

"Weatherby,Gerard"  writes:
>Do any of you Python folks see any blunders in the above code along the
>lines of not letting go of py files or static assets?

Er, no, I just replied to the original poster.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Python script not letting go of files

[Weatherby,Gerard]

Does the script exit when complete?

If you’re running on a Linux based system and have root access you can use lsof 
to see what processes have with files open. (Or use the psutil Python package).

From: Python-list  on 
behalf of Mike Dewhirst 
Date: Tuesday, November 29, 2022 at 2:20 AM
To: python-list@python.org 
Subject: Python script not letting go of files
I have a script which fetches a production site directly from a
Subversion repo using svn export

It runs a bunch of commands by calling this little method ...

def trycmd(cmd, log):
 retcode = -1
 ret = f"Trying {cmd}"
 try:
 retcode = os.system(cmd)
 ret = f"\n{cmd} -ok-> {retcode}"
 except Exception as err:
 ret = f"\n{cmd} -fail-> {err}"
 log.write(remove_password(ret))
 return retcode

This is the fetching script (omitting variables at the top) which
appears to be keeping a finger on files which Apache wants.

with open(fetchlog, 'a') as log:
 ret = f"\n\nFetching {tag}"
 log.write(ret)
 cmd = f"sudo rm -Rf {site_root}"
 if trycmd(cmd, log) == 0:
 cmd = f"sudo svn export --force --username {usr} --password
{pw} {svn_repo} {site_root}"
 if trycmd(cmd, log) == 0:
 # get any new dependencies
 cmd = f"sudo -H pip install -r {reqfile}"
 if trycmd(cmd, log) == 0:
 # run any migrations shipped from the repo
 cmd = f"sudo python3 {site_root}/manage.py migrate
--noinput --settings={settings}"
 if trycmd(cmd, log) == 0:
 # shouldn't find anything
 cmd = f"sudo python3 {site_root}/manage.py
makemigrations --noinput --settings={settings}"
 if trycmd(cmd, log) == 0:
 # should have been done already
 cmd = f"sudo python3 {site_root}/manage.py
migrate --noinput --settings={settings}"
 if trycmd(cmd, log) == 0:
 # remove all static files from their Apache dir
 cmd = f"sudo rm -Rf {static_root}/*"
 if trycmd(cmd, log) == 0:
 # copy all static files to the Apache
location
 cmd = f"sudo python3
{site_root}/manage.py collectstatic --noinput --settings={settings}"
 if trycmd(cmd, log) == 0:
 # set all permissions
 cmd = f"sudo {scripts}/perms_{host}.sh"
 if trycmd(cmd, log) == 0:
 cmd = "sudo service apache2
restart"
 if trycmd(cmd, log) == 0:
 ret = f"\nFinish {tag}\n\n"
 log.write(ret)
 else:
 print("Apache didn't restart")
 else:
 print("Didn't set permissions")
 else:
 print("Didn't collectstaic")
 else:
 print("Didn't delete static files")
 else:
 print("Didn't migrate 2")
 else:
 print("Didn't makemigration")
 else:
 print("Didn't migrate 1")
 else:
 print("Didn't install requirements")
 else:
 print("Didn't get source")
 else:
 print("Didn't remove site")
exit()

The problem I'm trying to fix is that after an Apache reload Apache
seems hellbent on filling up its scoreboard and running out of
resources. The kind folk on the Apache mailing list say something is
hogging the workers.

You can see the last operation above is an Apache restart. It should be
an Apache reload. Reload however lets Apache run out of resources.

Do any of you Python folks see any blunders in the above code along the
lines of not letting go of py files or static assets?

mod_wsgi is configured with 100% defaults.
mod_mpm_event conf per advice from the Apache mailing list is ...


 ServerLimit 32
 StartServers16
 MinSpareThreads 400
 MaxSpareThreads 800
 ThreadLimit 64
 ThreadsPerChild 50
 Asyn

Python script not letting go of files

[Mike Dewhirst]

I have a script which fetches a production site directly from a 
Subversion repo using svn export


It runs a bunch of commands by calling this little method ...

def trycmd(cmd, log):
    retcode = -1
    ret = f"Trying {cmd}"
    try:
    retcode = os.system(cmd)
    ret = f"\n{cmd} -ok-> {retcode}"
    except Exception as err:
    ret = f"\n{cmd} -fail-> {err}"
    log.write(remove_password(ret))
    return retcode

This is the fetching script (omitting variables at the top) which 
appears to be keeping a finger on files which Apache wants.


with open(fetchlog, 'a') as log:
    ret = f"\n\nFetching {tag}"
    log.write(ret)
    cmd = f"sudo rm -Rf {site_root}"
    if trycmd(cmd, log) == 0:
    cmd = f"sudo svn export --force --username {usr} --password 
{pw} {svn_repo} {site_root}"

    if trycmd(cmd, log) == 0:
    # get any new dependencies
    cmd = f"sudo -H pip install -r {reqfile}"
    if trycmd(cmd, log) == 0:
    # run any migrations shipped from the repo
    cmd = f"sudo python3 {site_root}/manage.py migrate 
--noinput --settings={settings}"

    if trycmd(cmd, log) == 0:
    # shouldn't find anything
    cmd = f"sudo python3 {site_root}/manage.py 
makemigrations --noinput --settings={settings}"

    if trycmd(cmd, log) == 0:
    # should have been done already
    cmd = f"sudo python3 {site_root}/manage.py 
migrate --noinput --settings={settings}"

    if trycmd(cmd, log) == 0:
    # remove all static files from their Apache dir
    cmd = f"sudo rm -Rf {static_root}/*"
    if trycmd(cmd, log) == 0:
    # copy all static files to the Apache 
location
    cmd = f"sudo python3 
{site_root}/manage.py collectstatic --noinput --settings={settings}"

    if trycmd(cmd, log) == 0:
    # set all permissions
    cmd = f"sudo {scripts}/perms_{host}.sh"
    if trycmd(cmd, log) == 0:
    cmd = "sudo service apache2 
restart"

    if trycmd(cmd, log) == 0:
    ret = f"\nFinish {tag}\n\n"
    log.write(ret)
    else:
    print("Apache didn't restart")
    else:
    print("Didn't set permissions")
    else:
    print("Didn't collectstaic")
    else:
    print("Didn't delete static files")
    else:
    print("Didn't migrate 2")
    else:
    print("Didn't makemigration")
    else:
    print("Didn't migrate 1")
    else:
    print("Didn't install requirements")
    else:
    print("Didn't get source")
    else:
    print("Didn't remove site")
exit()

The problem I'm trying to fix is that after an Apache reload Apache 
seems hellbent on filling up its scoreboard and running out of 
resources. The kind folk on the Apache mailing list say something is 
hogging the workers.


You can see the last operation above is an Apache restart. It should be 
an Apache reload. Reload however lets Apache run out of resources.


Do any of you Python folks see any blunders in the above code along the 
lines of not letting go of py files or static assets?


mod_wsgi is configured with 100% defaults.
mod_mpm_event conf per advice from the Apache mailing list is ...


    ServerLimit 32
    StartServers    16
    MinSpareThreads 400
    MaxSpareThreads 800
    ThreadLimit 64
    ThreadsPerChild 50
    AsyncRequestWorkerFactor    2
    MaxRequestWorkers   800
    MaxConnectionsPerChild  0


Server Version: Apache/2.4.52 (Ubuntu 2022.04) OpenSSL/3.0.2 
mod_wsgi/4.9.0 Python/3.10

Server MPM: event
Server Built: 2022-09-30T04:09:50

DigitalOcean droplet 8BG RAM and lightly loaded.

Many thanks for any hints.

Cheers

Mike


--
Signed email is an absolute defence against phishing. This email has
been signed with my private key. If you import my public key you can
automatically decrypt my signature and be sure it came from me. Just
ask and I'll send it to you. Your email software can handle signing.



OpenPGP_signature
Description: OpenPGP digital signature
-- 
https://mail.python.org/mailman/listinfo

Re: Weird strace of #! python script

[Barry]

> On 14 Mar 2022, at 21:29, Dan Stromberg  wrote:
> 
> I expected to see an exec of /usr/bin/python2, but I don't.  I just see an
> exec of /tmp/t.

I understand that the kernel handles the #! Line itself, which is why you do 
not see it in strace.

Barry



-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Weird strace of #! python script

[Chris Angelico]

On Tue, 15 Mar 2022 at 08:28, Dan Stromberg  wrote:
>
> Hi folks.
>
> First off, I know, python2 is ancient history.  Porting to 3.x is on my
> list of things to do (though I'm afraid it's not yet at the top of the
> list), and the same thing happens with python3.
>
> So anyway, I'm strace'ing a #!/usr/bin/python2 script.
>
> I expected to see an exec of /usr/bin/python2, but I don't.  I just see an
> exec of /tmp/t.
>
> As follows:
> tact@celery_worker:/app$ strace -f -s 1024 -o /tmp/t.strace /tmp/t
> ^Z
> [1]+  Stopped strace -f -s 1024 -o /tmp/t.strace /tmp/t
> tact@celery_worker:/app$ bg
> [1]+ strace -f -s 1024 -o /tmp/t.strace /tmp/t &
> tact@celery_worker:/app$ ps axf
>   PID TTY  STAT   TIME COMMAND
>  1163 pts/0Ss 0:00 bash
>  1363 pts/0S  0:00  \_ strace -f -s 1024 -o /tmp/t.strace /tmp/t
>  1366 pts/0S  0:00  |   \_ /usr/bin/python2 /tmp/t
>  1367 pts/0R+ 0:00  \_ ps axf
> tact@celery_worker:/app$ fg
> bash: fg: job has terminated
> [1]+  Donestrace -f -s 1024 -o /tmp/t.strace /tmp/t
> tact@celery_worker:/app$ grep execve /tmp/t.strace
> 1366  execve("/tmp/t", ["/tmp/t"], 0x7ffd89f9c3b8 /* 49 vars */) = 0
> tact@celery_worker:/app$
>
> I've deleted some irrelevant processes from the 'ps axf'.
>
> /tmp/t is actually just:
> tact@celery_worker:/app$ cat /tmp/t
> #!/usr/bin/python2
>
> import time
>
> time.sleep(10)
>
>
> Was this some sort of security feature I never heard about?  I'm tracing a
> very simple time.sleep(10) here, but the same thing is (not) happening in a
> larger script that I need to track down a bug in.
>
> Is there a way I can coax Linux and/or strace to show all the exec's, like
> they used to?  Not having them makes me wonder what else is missing from
> the strace report.
>
> I'm on a Debian 11.2 system with strace 5.10 and Python 2.7.18.
>
> Thanks!

I'm not sure, but I suspect that strace is skipping the exec into the
process itself. When I try this myself, I can see the trace of the
sleep call itself (it's implemented on top of select()), so it does
seem to be doing the proper trace. My guess is that the shebang is
being handled inside the exec of /tmp/t itself, rather than being a
separate syscall.

In any case, it doesn't seem to be a Python thing; I can trigger the
same phenomenon with other interpreters.

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list

Weird strace of #! python script

[Dan Stromberg]

Hi folks.

First off, I know, python2 is ancient history.  Porting to 3.x is on my
list of things to do (though I'm afraid it's not yet at the top of the
list), and the same thing happens with python3.

So anyway, I'm strace'ing a #!/usr/bin/python2 script.

I expected to see an exec of /usr/bin/python2, but I don't.  I just see an
exec of /tmp/t.

As follows:
tact@celery_worker:/app$ strace -f -s 1024 -o /tmp/t.strace /tmp/t
^Z
[1]+  Stopped strace -f -s 1024 -o /tmp/t.strace /tmp/t
tact@celery_worker:/app$ bg
[1]+ strace -f -s 1024 -o /tmp/t.strace /tmp/t &
tact@celery_worker:/app$ ps axf
  PID TTY  STAT   TIME COMMAND
 1163 pts/0Ss 0:00 bash
 1363 pts/0S  0:00  \_ strace -f -s 1024 -o /tmp/t.strace /tmp/t
 1366 pts/0S  0:00  |   \_ /usr/bin/python2 /tmp/t
 1367 pts/0R+ 0:00  \_ ps axf
tact@celery_worker:/app$ fg
bash: fg: job has terminated
[1]+  Donestrace -f -s 1024 -o /tmp/t.strace /tmp/t
tact@celery_worker:/app$ grep execve /tmp/t.strace
1366  execve("/tmp/t", ["/tmp/t"], 0x7ffd89f9c3b8 /* 49 vars */) = 0
tact@celery_worker:/app$

I've deleted some irrelevant processes from the 'ps axf'.

/tmp/t is actually just:
tact@celery_worker:/app$ cat /tmp/t
#!/usr/bin/python2

import time

time.sleep(10)


Was this some sort of security feature I never heard about?  I'm tracing a
very simple time.sleep(10) here, but the same thing is (not) happening in a
larger script that I need to track down a bug in.

Is there a way I can coax Linux and/or strace to show all the exec's, like
they used to?  Not having them makes me wonder what else is missing from
the strace report.

I'm on a Debian 11.2 system with strace 5.10 and Python 2.7.18.

Thanks!
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: mac app from a python script?

[Dan Stromberg]

On Sun, Jan 23, 2022 at 9:59 AM Dan Stromberg  wrote:

>
> Hi folks.
>
> I have a Python 3 script (built on top of gi.respository.Gtk) that runs on
> Linux and macOS 11.5.  It's at
> https://stromberg.dnsalias.org/~strombrg/hcm/ if you're curious.
>
> It works the way I want on Linux, but on macOS I seem to have to start it
> from the command line, like:
> hcm --gui
> ...because I don't know how to create a macOS "app" that goes under
> /Applications.
>
> I don't really care about having a single executable on macOS, and I don't
> really care about creating a .dmg or .pkg file. I'd be perfectly happy just
> running "make install" and putting a #!'d script under /Applications with
> appropriate metadata - but if it's easier to do a single executable, .dmg
> or .pkg, I'd be fine with that.
>

It turns out that using something called "appify" does this well.

I found it on a gist I believe.

I liked it so much that I adopted it and put it at
https://stromberg.dnsalias.org/~strombrg/mactools/ , with a few style
changes and small bug fixes.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: mac app from a python script?

[Dan Stromberg]

On Wed, Jan 26, 2022 at 2:35 PM Barry  wrote:

>
>
> On 26 Jan 2022, at 05:17, Dan Stromberg  wrote:
>
>
> On Tue, Jan 25, 2022 at 6:41 PM Dan Stromberg  wrote:
>
>>
>> On Tue, Jan 25, 2022 at 2:23 PM Barry  wrote:
>>
>>>
>>> On 25 Jan 2022, at 02:56, Dan Stromberg  wrote:
>>>
>>> 
>>>
>>> On Sun, Jan 23, 2022 at 1:37 PM Barry  wrote:
>>>

 I do not have experience with great, but you might try pyinstaller.
 I use it to make a PyQt Mac app successfully.

 It’s command line plus setup script.

>>>
>>> I wound up doing:
>>> 1) pyinstaller, as normal, but this created a broken all-encompassing
>>> binary of my script.  At least it gave me the metadata I needed though.
>>>
>>>
>>> You mean it created a .app bundle?
>>>
>>> That is the way that macOS makes it trivia to install apps
>>> Just by drag and drop in /Applications.
>>>
>>
>> Yes, it created an hcm.app for me.  But the executable it created didn't
>> work.  Hence the hack.
>>
>
> $ file /Applications/hcm.app/Contents/MacOS/hcm
> cmd output started 2022 Tue Jan 25 09:00:54 PM PST
> /Applications/hcm.app/Contents/MacOS/hcm: Mach-O 64-bit executable x86_64
>
>
> It’s intended to be started as an app.
>
> What if you double click the app? Does it work?
>
No, it does not start that way either.


> Also you can use the open command to run use the app name you give it.
>
 Thanks for the tip.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: mac app from a python script?

[Barry]

> On 26 Jan 2022, at 05:17, Dan Stromberg  wrote:
> 
> 
> 
>> On Tue, Jan 25, 2022 at 6:41 PM Dan Stromberg  wrote:
>> 
>>> On Tue, Jan 25, 2022 at 2:23 PM Barry  wrote:
>>> 
> On 25 Jan 2022, at 02:56, Dan Stromberg  wrote:
> 
 
 
> On Sun, Jan 23, 2022 at 1:37 PM Barry  wrote:
> 
> I do not have experience with great, but you might try pyinstaller.
> I use it to make a PyQt Mac app successfully.
> 
> It’s command line plus setup script.
 
 I wound up doing:
 1) pyinstaller, as normal, but this created a broken all-encompassing 
 binary of my script.  At least it gave me the metadata I needed though.
>>> 
>>> You mean it created a .app bundle?
>>> 
>>> That is the way that macOS makes it trivia to install apps
>>> Just by drag and drop in /Applications.
>> 
>> Yes, it created an hcm.app for me.  But the executable it created didn't 
>> work.  Hence the hack.
> 
> More specifically:
> $ /Applications/hcm.app/Contents/MacOS/hcm --gui
> cmd output started 2022 Tue Jan 25 09:00:33 PM PST
> Traceback (most recent call last):
>   File "/Applications/hcm.app/Contents/Resources/__boot__.py", line 146, in 
> 
> _run()
>   File "/Applications/hcm.app/Contents/Resources/__boot__.py", line 129, in 
> _run
> exec(compile(source, path, "exec"), globals(), globals())
>   File "/Applications/hcm.app/Contents/Resources/hcm.py", line 1950, in 
> 
> import gi
>   File "", line 1007, in _find_and_load
>   File "", line 986, in _find_and_load_unlocked
>   File "", line 664, in _load_unlocked
>   File "", line 627, in _load_backward_compatible
>   File "", line 259, in load_module
>   File "gi/__init__.pyc", line 40, in 
>   File "", line 1007, in _find_and_load
>   File "", line 986, in _find_and_load_unlocked
>   File "", line 664, in _load_unlocked
>   File "", line 627, in _load_backward_compatible
>   File "", line 259, in load_module
>   File "gi/_gi.pyc", line 14, in 
>   File "gi/_gi.pyc", line 10, in __load
>   File "imp.pyc", line 342, in load_dynamic
>   File "", line 1007, in _find_and_load
>   File "", line 986, in _find_and_load_unlocked
>   File "", line 664, in _load_unlocked
>   File "", line 627, in _load_backward_compatible
>   File "/Applications/hcm.app/Contents/Resources/__boot__.py", line 36, in 
> load_module
> return imp.load_module(
>   File "imp.pyc", line 244, in load_module
>   File "imp.pyc", line 216, in load_package
>   File "", line 710, in _load
> AttributeError: 'NoneType' object has no attribute 'name'
> 2022-01-25 21:00:34.576 hcm[62695:1322031] hcm Error
> ^C^\Quit: 3
> above cmd output done2022 Tue Jan 25 09:00:42 PM PST
> dstromberg@Daniels-Mini:~/src/home-svn/hcm/trunk x86_64-apple-darwin20.6.0 
> 61933
> 
> $ file /Applications/hcm.app/Contents/MacOS/hcm
> cmd output started 2022 Tue Jan 25 09:00:54 PM PST
> /Applications/hcm.app/Contents/MacOS/hcm: Mach-O 64-bit executable x86_64

It’s intended to be started as an app.

What if you double click the app? Does it work?
Also you can use the open command to run use the app name you give it.

> 
> But if I replace /Applications/hcm.app/Contents/MacOS/hcm with a symlink to a 
> wrapper shell script, hcm runs fine from the Applications menu.
> 
> It seems that gi.repository.Gtk applications are not packaged correctly by 
> pyinstaller and py2app.  Some Python modules require a little assistance to 
> be packaged up into a Mach-O executable neatly by such tools.  But it's just 
> not that important to me to have a Mach-O of my app.

Understood, was just curious.

Barry
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: mac app from a python script?

[Dan Stromberg]

On Tue, Jan 25, 2022 at 6:41 PM Dan Stromberg  wrote:

>
> On Tue, Jan 25, 2022 at 2:23 PM Barry  wrote:
>
>>
>> On 25 Jan 2022, at 02:56, Dan Stromberg  wrote:
>>
>> 
>>
>> On Sun, Jan 23, 2022 at 1:37 PM Barry  wrote:
>>
>>>
>>> I do not have experience with great, but you might try pyinstaller.
>>> I use it to make a PyQt Mac app successfully.
>>>
>>> It’s command line plus setup script.
>>>
>>
>> I wound up doing:
>> 1) pyinstaller, as normal, but this created a broken all-encompassing
>> binary of my script.  At least it gave me the metadata I needed though.
>>
>>
>> You mean it created a .app bundle?
>>
>> That is the way that macOS makes it trivia to install apps
>> Just by drag and drop in /Applications.
>>
>
> Yes, it created an hcm.app for me.  But the executable it created didn't
> work.  Hence the hack.
>

More specifically:
$ /Applications/hcm.app/Contents/MacOS/hcm --gui
cmd output started 2022 Tue Jan 25 09:00:33 PM PST
Traceback (most recent call last):
  File "/Applications/hcm.app/Contents/Resources/__boot__.py", line 146, in

_run()
  File "/Applications/hcm.app/Contents/Resources/__boot__.py", line 129, in
_run
exec(compile(source, path, "exec"), globals(), globals())
  File "/Applications/hcm.app/Contents/Resources/hcm.py", line 1950, in

import gi
  File "", line 1007, in _find_and_load
  File "", line 986, in _find_and_load_unlocked
  File "", line 664, in _load_unlocked
  File "", line 627, in
_load_backward_compatible
  File "", line 259, in load_module
  File "gi/__init__.pyc", line 40, in 
  File "", line 1007, in _find_and_load
  File "", line 986, in _find_and_load_unlocked
  File "", line 664, in _load_unlocked
  File "", line 627, in
_load_backward_compatible
  File "", line 259, in load_module
  File "gi/_gi.pyc", line 14, in 
  File "gi/_gi.pyc", line 10, in __load
  File "imp.pyc", line 342, in load_dynamic
  File "", line 1007, in _find_and_load
  File "", line 986, in _find_and_load_unlocked
  File "", line 664, in _load_unlocked
  File "", line 627, in
_load_backward_compatible
  File "/Applications/hcm.app/Contents/Resources/__boot__.py", line 36, in
load_module
return imp.load_module(
  File "imp.pyc", line 244, in load_module
  File "imp.pyc", line 216, in load_package
  File "", line 710, in _load
AttributeError: 'NoneType' object has no attribute 'name'
2022-01-25 21:00:34.576 hcm[62695:1322031] hcm Error
^C^\Quit: 3
above cmd output done2022 Tue Jan 25 09:00:42 PM PST
dstromberg@Daniels-Mini:~/src/home-svn/hcm/trunk x86_64-apple-darwin20.6.0
61933

$ file /Applications/hcm.app/Contents/MacOS/hcm
cmd output started 2022 Tue Jan 25 09:00:54 PM PST
/Applications/hcm.app/Contents/MacOS/hcm: Mach-O 64-bit executable x86_64

But if I replace /Applications/hcm.app/Contents/MacOS/hcm with a symlink to
a wrapper shell script, hcm runs fine from the Applications menu.

It seems that gi.repository.Gtk applications are not packaged correctly by
pyinstaller and py2app.  Some Python modules require a little assistance to
be packaged up into a Mach-O executable neatly by such tools.  But it's
just not that important to me to have a Mach-O of my app.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: mac app from a python script?

[Dan Stromberg]

On Tue, Jan 25, 2022 at 2:23 PM Barry  wrote:

>
> On 25 Jan 2022, at 02:56, Dan Stromberg  wrote:
>
> 
>
> On Sun, Jan 23, 2022 at 1:37 PM Barry  wrote:
>
>>
>> I do not have experience with great, but you might try pyinstaller.
>> I use it to make a PyQt Mac app successfully.
>>
>> It’s command line plus setup script.
>>
>
> I wound up doing:
> 1) pyinstaller, as normal, but this created a broken all-encompassing
> binary of my script.  At least it gave me the metadata I needed though.
>
>
> You mean it created a .app bundle?
>
> That is the way that macOS makes it trivia to install apps
> Just by drag and drop in /Applications.
>

Yes, it created an hcm.app for me.  But the executable it created didn't
work.  Hence the hack.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: mac app from a python script?

[Barry]

> On 25 Jan 2022, at 02:56, Dan Stromberg  wrote:
> 
> 
> 
>> On Sun, Jan 23, 2022 at 1:37 PM Barry  wrote:
>> 
>> I do not have experience with great, but you might try pyinstaller.
>> I use it to make a PyQt Mac app successfully.
>> 
>> It’s command line plus setup script.
> 
> I wound up doing:
> 1) pyinstaller, as normal, but this created a broken all-encompassing binary 
> of my script.  At least it gave me the metadata I needed though.

You mean it created a .app bundle?

That is the way that macOS makes it trivia to install apps
Just by drag and drop in /Applications.

Barry

> 2) overwriting /Applications/hcm.app/Contents/MacOS/hcm with a proper 
> #!/usr/bin/env python3 script


> 
> This mostly works.  It's a kinda ugly hack, and it doesn't stay in the dock 
> after starting it.
> 
> There should be a way of installing a python GUI in the macOS Applications 
> list, without having to bundle everything up into a big binary.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: mac app from a python script?

[Dan Stromberg]

On Sun, Jan 23, 2022 at 1:37 PM Barry  wrote:

>
> I do not have experience with great, but you might try pyinstaller.
> I use it to make a PyQt Mac app successfully.
>
> It’s command line plus setup script.
>

I wound up doing:
1) pyinstaller, as normal, but this created a broken all-encompassing
binary of my script.  At least it gave me the metadata I needed though.
2) overwriting /Applications/hcm.app/Contents/MacOS/hcm with a proper
#!/usr/bin/env python3 script

This mostly works.  It's a kinda ugly hack, and it doesn't stay in the dock
after starting it.

There should be a way of installing a python GUI in the macOS Applications
list, without having to bundle everything up into a big binary.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: mac app from a python script?

[Cameron Simpson]

On 24Jan2022 17:48, Barry Scott  wrote:
>> On 24 Jan 2022, at 02:31, Cameron Simpson  wrote:
>> On 23Jan2022 21:37, Barry Scott  wrote:
>>> I do not have experience with great, but you might try pyinstaller.
>>> I use it to make a PyQt Mac app successfully.
>>
>> And I have used py2app/py2applet to create a PyQt Mac app.
>
>Current Py2app does not work anymore for PyQt5. I have an question on
>the Mac python list about this.

Ah. My last build was March 2021 (also PyQt5 - well, PySide2, which I 
believe uses Qt5 behind the scenes).  And I've certainly had troubles.

Thanks for the heads up.

Cheers,
Cameron Simpson 
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: mac app from a python script?

[Barry]

> On 24 Jan 2022, at 02:31, Cameron Simpson  wrote:
> 
> On 23Jan2022 21:37, Barry Scott  wrote:
>> I do not have experience with great, but you might try pyinstaller.
>> I use it to make a PyQt Mac app successfully.
> 
> And I have used py2app/py2applet to create a PyQt Mac app.

Current Py2app does not work anymore for PyQt5. I have an question on
the Mac python list about this.

Barry

> 
> AIUI, they both do the same basic task: bundle a Python interpreter and 
> all the implied libraries from your script (i.e. including PyQt) into a 
> Mac App tree, with an executable in the right place to invoke your 
> module using the shipped interpreter.
> 
> So they make a Thing.app directory. A .dmg is just a disc image people 
> ship these things in, often merely containing the .app and a symlink or 
> alias to /Applications for the user to drag the app to.
> 
> Cheers,
> Cameron Simpson 
> -- 
> https://mail.python.org/mailman/listinfo/python-list
> 

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: mac app from a python script?

[Mats Wichmann]

On 1/23/22 10:59, Dan Stromberg wrote:
> Hi folks.
> 
> I have a Python 3 script (built on top of gi.respository.Gtk) that runs on
> Linux and macOS 11.5.  It's at https://stromberg.dnsalias.org/~strombrg/hcm/
> if you're curious.
> 
> It works the way I want on Linux, but on macOS I seem to have to start it
> from the command line, like:
> hcm --gui
> ...because I don't know how to create a macOS "app" that goes under
> /Applications.
> 
> I don't really care about having a single executable on macOS, and I don't
> really care about creating a .dmg or .pkg file. I'd be perfectly happy just
> running "make install" and putting a #!'d script under /Applications with
> appropriate metadata - but if it's easier to do a single executable, .dmg
> or .pkg, I'd be fine with that.
> 
> I've experimented with a few different options for this (months ago),
> mostly py2app, but it doesn't appear to like gi.repository.Gtk much.
> 
> What's the most straightforward way of installing a Python script under
> /Applications on macOS?  I'd -love- to find a way of doing something
> analogous to Linux' desktop-file-install - that is, something that isn't
> click-happy, but GUI's are acceptable too if they'll get the job done.

There are notes on the topic here, see if they help:

https://docs.python.org/3/using/mac.html#how-to-run-a-python-script


-- 
https://mail.python.org/mailman/listinfo/python-list

Re: mac app from a python script?

[Cameron Simpson]

On 23Jan2022 21:37, Barry Scott  wrote:
>I do not have experience with great, but you might try pyinstaller.
>I use it to make a PyQt Mac app successfully.

And I have used py2app/py2applet to create a PyQt Mac app.

AIUI, they both do the same basic task: bundle a Python interpreter and 
all the implied libraries from your script (i.e. including PyQt) into a 
Mac App tree, with an executable in the right place to invoke your 
module using the shipped interpreter.

So they make a Thing.app directory. A .dmg is just a disc image people 
ship these things in, often merely containing the .app and a symlink or 
alias to /Applications for the user to drag the app to.

Cheers,
Cameron Simpson 
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: mac app from a python script?

[Barry]

> On 23 Jan 2022, at 21:40, Barry  wrote:
> 
> 
> 
>> On 23 Jan 2022, at 18:02, Dan Stromberg  wrote:
>> 
>> Hi folks.
>> 
>> I have a Python 3 script (built on top of gi.respository.Gtk) that runs on
>> Linux and macOS 11.5.  It's at https://stromberg.dnsalias.org/~strombrg/hcm/
>> if you're curious.
>> 
>> It works the way I want on Linux, but on macOS I seem to have to start it
>> from the command line, like:
>>   hcm --gui
>> ...because I don't know how to create a macOS "app" that goes under
>> /Applications.
>> 
>> I don't really care about having a single executable on macOS, and I don't
>> really care about creating a .dmg or .pkg file. I'd be perfectly happy just
>> running "make install" and putting a #!'d script under /Applications with
>> appropriate metadata - but if it's easier to do a single executable, .dmg
>> or .pkg, I'd be fine with that.
>> 
>> I've experimented with a few different options for this (months ago),
>> mostly py2app, but it doesn't appear to like gi.repository.Gtk much.
>> 
>> What's the most straightforward way of installing a Python script under
>> /Applications on macOS?  I'd -love- to find a way of doing something
>> analogous to Linux' desktop-file-install - that is, something that isn't
>> click-happy, but GUI's are acceptable too if they'll get the job done.
> 
> I do not have experience with great, but you might try pyinstaller.
> I use it to make a PyQt Mac app successfully.

Gtk spell correct to great…

> 
> It’s command line plus setup script.
> 
> Barry
>> 
>> Thanks!
>> -- 
>> https://mail.python.org/mailman/listinfo/python-list
>> 

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: mac app from a python script?

[Barry]

> On 23 Jan 2022, at 18:02, Dan Stromberg  wrote:
> 
> Hi folks.
> 
> I have a Python 3 script (built on top of gi.respository.Gtk) that runs on
> Linux and macOS 11.5.  It's at https://stromberg.dnsalias.org/~strombrg/hcm/
> if you're curious.
> 
> It works the way I want on Linux, but on macOS I seem to have to start it
> from the command line, like:
>hcm --gui
> ...because I don't know how to create a macOS "app" that goes under
> /Applications.
> 
> I don't really care about having a single executable on macOS, and I don't
> really care about creating a .dmg or .pkg file. I'd be perfectly happy just
> running "make install" and putting a #!'d script under /Applications with
> appropriate metadata - but if it's easier to do a single executable, .dmg
> or .pkg, I'd be fine with that.
> 
> I've experimented with a few different options for this (months ago),
> mostly py2app, but it doesn't appear to like gi.repository.Gtk much.
> 
> What's the most straightforward way of installing a Python script under
> /Applications on macOS?  I'd -love- to find a way of doing something
> analogous to Linux' desktop-file-install - that is, something that isn't
> click-happy, but GUI's are acceptable too if they'll get the job done.

I do not have experience with great, but you might try pyinstaller.
I use it to make a PyQt Mac app successfully.

It’s command line plus setup script.

Barry
> 
> Thanks!
> -- 
> https://mail.python.org/mailman/listinfo/python-list
> 

-- 
https://mail.python.org/mailman/listinfo/python-list

mac app from a python script?

[Dan Stromberg]

Hi folks.

I have a Python 3 script (built on top of gi.respository.Gtk) that runs on
Linux and macOS 11.5.  It's at https://stromberg.dnsalias.org/~strombrg/hcm/
if you're curious.

It works the way I want on Linux, but on macOS I seem to have to start it
from the command line, like:
hcm --gui
...because I don't know how to create a macOS "app" that goes under
/Applications.

I don't really care about having a single executable on macOS, and I don't
really care about creating a .dmg or .pkg file. I'd be perfectly happy just
running "make install" and putting a #!'d script under /Applications with
appropriate metadata - but if it's easier to do a single executable, .dmg
or .pkg, I'd be fine with that.

I've experimented with a few different options for this (months ago),
mostly py2app, but it doesn't appear to like gi.repository.Gtk much.

What's the most straightforward way of installing a Python script under
/Applications on macOS?  I'd -love- to find a way of doing something
analogous to Linux' desktop-file-install - that is, something that isn't
click-happy, but GUI's are acceptable too if they'll get the job done.

Thanks!
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Python script seems to stop running when handling very large dataset

[Shaozhong SHI]

On Saturday, 30 October 2021, Dieter Maurer  wrote:

> Shaozhong SHI wrote at 2021-10-29 23:42 +0100:
> >Python script works well, but seems to stop running at a certain point
> when
> >handling very large dataset.
> >
> >Can anyone shed light on this?
>
> Some algorithms have non linear runtime.
>
>
> For example, it is quite easy to write code with
> quadratic runtime in Python:
>   s = ""
>   for x in ...: s += f(x)
> You will see the problem only for large data sets.
>

Has anyone compared this with iterrow?  which looping option is faster?
Regards, David
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Python script seems to stop running when handling very large dataset

[Dieter Maurer]

Shaozhong SHI wrote at 2021-10-29 23:42 +0100:
>Python script works well, but seems to stop running at a certain point when
>handling very large dataset.
>
>Can anyone shed light on this?

Some algorithms have non linear runtime.


For example, it is quite easy to write code with
quadratic runtime in Python:
  s = ""
  for x in ...: s += f(x)
You will see the problem only for large data sets.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Python script seems to stop running when handling very large dataset

[Grant Edwards]

On 2021-10-29, Shaozhong SHI  wrote:
> Python script works well, but seems to stop running at a certain point when
> handling very large dataset.
>
> Can anyone shed light on this?

No.

Nobody can help you with the amount of information you have provided.

--
Grant



-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Python script seems to stop running when handling very large dataset

[Paul Bryan]

With so little information provided, not much light will be shed. When
it stops running, are there any errors? How is the dataset being
processed? How large is the dataset? How large a dataset can be
successfully processed? What libraries are being used? What version of
Python are you using? On what operating system? With how much memory?
With how much disk space is used? How much is free? Are you processing
files or using a database? If the latter, what database? Does it write
intermediate files during processing? Can you monitor memory usage
during processing (e.g. with a system monitor) to see how much memory
is consumed?


On Fri, 2021-10-29 at 23:42 +0100, Shaozhong SHI wrote:
> Python script works well, but seems to stop running at a certain
> point when
> handling very large dataset.
> 
> Can anyone shed light on this?
> 
> Regards, David

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Python script seems to stop running when handling very large dataset

[Dan Stromberg]

On Fri, Oct 29, 2021 at 4:04 PM dn via Python-list 
wrote:

> On 30/10/2021 11.42, Shaozhong SHI wrote:
> > Python script works well, but seems to stop running at a certain point
> when
> > handling very large dataset.
> >
> > Can anyone shed light on this?
>
> Storage space?
> Taking time to load/format/process data-set?
>

It could be many things.

What operating system are you on?

If you're on Linux, you can use strace to attach to a running process to
see what it's up to.  Check out the -p option.  See
https://stromberg.dnsalias.org/~strombrg/debugging-with-syscall-tracers.html

macOS has dtruss, but it's a little hard to enable.  dtruss is similar to
strace.

Both of these tools are better for processes doing system calls (kernel
interactions).  They do not help nearly as much with CPU-bound processes.

It could also be that you're running out of virtual memory, and the
system's virtual memory system is thrashing.

Does the load average on the system go up significantly when the process
seems to get stuck?

You could try attaching to the process with a debugger, too, EG with pudb:
https://github.com/inducer/pudb/issues/31

Barring those, you could sprinkle some print statements in your code, to
see where it's getting stuck. This tends to be an iterative process, where
you add some prints, run, observe the result, and repeat.

HTH.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Python script seems to stop running when handling very large dataset

[dn via Python-list]

On 30/10/2021 11.42, Shaozhong SHI wrote:
> Python script works well, but seems to stop running at a certain point when
> handling very large dataset.
> 
> Can anyone shed light on this?

Storage space?
Taking time to load/format/process data-set?

-- 
Regards,
=dn
-- 
https://mail.python.org/mailman/listinfo/python-list

Python script seems to stop running when handling very large dataset

[Shaozhong SHI]

Python script works well, but seems to stop running at a certain point when
handling very large dataset.

Can anyone shed light on this?

Regards, David
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Python script accessing own source code

[Chris Angelico]

On Thu, May 13, 2021 at 5:27 PM Robin Becker  wrote:
>
> On 12/05/2021 20:17, Mirko via Python-list wrote:
> > Am 12.05.2021 um 20:41 schrieb Robin Becker:
> >> ...
> >>>
> >...
> >> since GvR has been shown to have time traveling abilities such a
> >> script could paradoxically appear acausally.
> >> --
> >> yrs-not-too-seriously
> >> Robin Becker
> >
> >
> > Not sure, if that's what you mean, but writing a self-replicating
> > script is easy too:
>
> actually I was really joking about self creating scripts that do something 
> useful like finding future lotto numbers.
>
> the artificial programmer servant is some way off
>

Perhaps not so far off as you might think. With import hooks, it's
entirely possible to have a program that creates itself on demand -
for instance, you might transpile your script from some variant form
of the language (maybe you're trying out a proposed new syntax). It
won't help you with the lotto, but it certainly is an AI assistant for
a programmer that creates runnable code when called upon.

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Python script accessing own source code

[Robin Becker]

On 12/05/2021 20:17, Mirko via Python-list wrote:

Am 12.05.2021 um 20:41 schrieb Robin Becker:

...



...

since GvR has been shown to have time traveling abilities such a
script could paradoxically appear acausally.
--
yrs-not-too-seriously
Robin Becker



Not sure, if that's what you mean, but writing a self-replicating
script is easy too:


actually I was really joking about self creating scripts that do something 
useful like finding future lotto numbers.

the artificial programmer servant is some way off



import os
import sys

with open(os.path.abspath(__file__)) as myself:
 with open(sys.argv[1], "w") as yourself:
 yourself.write(myself.read())


Give it a filename as a command-line argument and it will write
itself to that file.


--
Robin Becker
--
https://mail.python.org/mailman/listinfo/python-list

Re: Python script accessing own source code

[Mirko via Python-list]

Am 12.05.2021 um 20:41 schrieb Robin Becker:
> ...
>>
>> with open(__file__) as myself:
>>  print(myself.read(), end='')
> 
> very nice, but accessing code that's already seems quite easy. I
> think the real problem is to get a python script name that creates
> and writes itself. So I would ask if any one has the solution to the
> self writing script
> 
> python find-tomorrows-lotto-numbers.py
> 
> since GvR has been shown to have time traveling abilities such a
> script could paradoxically appear acausally.
> -- 
> yrs-not-too-seriously
> Robin Becker


Not sure, if that's what you mean, but writing a self-replicating
script is easy too:

import os
import sys

with open(os.path.abspath(__file__)) as myself:
with open(sys.argv[1], "w") as yourself:
yourself.write(myself.read())


Give it a filename as a command-line argument and it will write
itself to that file.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Python script accessing own source code

[Robin Becker]

...


with open(__file__) as myself:
     print(myself.read(), end='')


very nice, but accessing code that's already seems quite easy. I think the real problem is to get a python script name 
that creates and writes itself. So I would ask if any one has the solution to the self writing script


python find-tomorrows-lotto-numbers.py

since GvR has been shown to have time traveling abilities such a script could 
paradoxically appear acausally.
--
yrs-not-too-seriously
Robin Becker
--
https://mail.python.org/mailman/listinfo/python-list

Re: Python script accessing own source code

[Stestagg]

On 2021-05-12 15:48, Michael F. Stemper wrote:

> > On 12/05/2021 08.26, Dino wrote:
> >
> >> Hi, here's my (probably unusual) problem. Can a Python (3.7+) script
> >> access its own source code?
> >
> > Here is a fairly simple python program that reads itself:
> >
> > 
> > #!/usr/bin/python
> >
> > import sys
> >
> > with open( sys.argv[0], "rt" ) as myself:
> > for line in myself:
> >   junk = sys.stdout.write( "%s" % (line) )
> >
> > sys.exit(0)
> > 
> >
> > It's not bullet-proof. If you put it in a directory in your $PATH and
> > run it from somewhere else, it won't work.
> >
>
>
Here's a fairly simple option:

==
import inspect
import sys

print(inspect.getsource(sys.modules[__name__]))

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Python script accessing own source code

[MRAB]

On 2021-05-12 15:48, Michael F. Stemper wrote:

On 12/05/2021 08.26, Dino wrote:

Hi, here's my (probably unusual) problem. Can a Python (3.7+) script 
access its own source code?


Here is a fairly simple python program that reads itself:


#!/usr/bin/python

import sys

with open( sys.argv[0], "rt" ) as myself:
for line in myself:
  junk = sys.stdout.write( "%s" % (line) )

sys.exit(0)


It's not bullet-proof. If you put it in a directory in your $PATH and
run it from somewhere else, it won't work.


How about this:

with open(__file__) as myself:
print(myself.read(), end='')
--
https://mail.python.org/mailman/listinfo/python-list

Re: Python script accessing own source code

[Michael F. Stemper]

On 12/05/2021 08.26, Dino wrote:

Hi, here's my (probably unusual) problem. Can a Python (3.7+) script 
access its own source code?


Here is a fairly simple python program that reads itself:


#!/usr/bin/python

import sys

with open( sys.argv[0], "rt" ) as myself:
  for line in myself:
junk = sys.stdout.write( "%s" % (line) )

sys.exit(0)


It's not bullet-proof. If you put it in a directory in your $PATH and
run it from somewhere else, it won't work.

--
Michael F. Stemper
I feel more like I do now than I did when I came in.
--
https://mail.python.org/mailman/listinfo/python-list

Re: Ram memory not freed after executing python script on ubuntu system (rmlibre)

[Peter J. Holzer]

On 2020-05-29 14:28:59 +0900, Inada Naoki wrote:
> pymalloc manages only small blocks of memory.
> Large (more than 512 byte) memory blocks are managed by malloc/free.
> 
> glibc malloc doesn't return much freed memory to OS.

That depends on what "much" means.

Glibc does return blocks to the OS which it allocated via mmap, By
default, these are allocations larger than 128 kB.

So that means that

* Blocks smaller than 512 bytes are returned to the OS if the arena they
  are in is completely empty.

* Blocks between 512 bytes and 128 kB are not returned to the OS (unless
  they happen to be at the end of the heap).

* Blocks larger than 128 kB are returned to the OS:

Most Python objects are probably smaller than 512 bytes, so whether
"much" is returned to the OS mostly depends on whether arenas ever get
completely empty.

This stupid little test program returns memory to the OS quite nicely,
because it allocates and frees lots of objects together:

---8<--8<--8<--8<--8<--8<--8<--8<---
#!/usr/bin/python3
import time

a = []
for i in range(10):
print('a', i)
x = []
for j in range(100):
x.append(j)
a.append(x)
time.sleep(1)
if i >= 5:
print('d', i - 5)
a[i - 5] = None
time.sleep(1)
print('f')
---8<--8<--8<--8<--8<--8<--8<--8<---

(run it with strace to watch the mmap/munmap system calls)

A program with more random allocation patterns may suffer severe
internal fragmentation and end up with many mostly empty arenas.

> You can try jemalloc instead of glibc.

I think that would only make a difference if you have lots of objects
between 512 B and 128 kB. And only if those of similar size are
allocated and freed together.

hp

-- 
   _  | Peter J. Holzer| Story must make more sense than reality.
|_|_) ||
| |   | h...@hjp.at |-- Charles Stross, "Creative writing
__/   | http://www.hjp.at/ |   challenge!"


signature.asc
Description: PGP signature
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Ram memory not freed after executing python script on ubuntu system (rmlibre)

[Inada Naoki]

pymalloc manages only small blocks of memory.
Large (more than 512 byte) memory blocks are managed by malloc/free.

glibc malloc doesn't return much freed memory to OS.
You can try jemalloc instead of glibc.

On Ubuntu 20.04, you can try it by:

  LD_PRELOAD=/usr/lib/x86_64-linux-gnu/libjemalloc.so python your-script.py

On Fri, May 29, 2020 at 11:07 AM  wrote:
>
>
> We just ran into this problem when running our aiootp package's memory
> hard password hashing function (https://github.com/rmlibre/aiootp/). The
> memory was not being cleared after the function finished running but the
> script was still live. We tried making sure everything went out of scope
> and deleting things explicitly inside the function, but that didn't
> help. We even tried forcing the garbage collector to free up
> unreferenced memory with import gc; gc.collect(). But that only had a
> small dent in the memory that was being built up.
>
> The most useful answer online had to do with Python's free lists being
> created automatically when very large datasets were being processed
> (https://stackoverflow.com/questions/23937189/how-do-i-use-subprocesses-to-force-python-to-release-memory/24126616#24126616).
>
>
> After putting the memory intensive work into a separate process, as the
> answer suggested, the memory was finally freed after execution.
>
> In our case, we wound up passing the result back in a
> ``multiprocessing.Manager().List()``. Though, passing your whole numpy
> dataset back doesn't seem feasible. I'd recommend doing the necessary
> memory intensive work in a separate process and passing only the
> necessary conclusions back to the main process.
>
> Or you could set up a multiprocessing queue to pass control messages to
> the spawned (daemon) process so it returns desired results on demand.
>
>
> Here's an excerpt from our code after the fix in case it's helpful.
>
> >@classmethod
> >def passcrypt(cls, password, salt, kb=1024, cpu=1024, hardness=256):
> >"""
> >The ``passcrypt`` function can be highly memory intensive.
> >These resources may not be freed up, & often are not, because of
> >python quirks around memory management. This is a huge problem.
> >So to force the release of those resources, we run the function
> >in another process which is guaranteed to release them.
> >"""
> >cls._validate_passcrypt_args(kb, cpu, hardness)
> >state = Manager().list()
> >process = Process(
> >target=cls._passcrypt,
> >args=(password, salt),
> >kwargs=dict(kb=kb, cpu=cpu, hardness=hardness, state=state),
> >)
> >process.start()
> >process.join()
> >return state.pop()
>
>
>
>
> On 2020-05-28 16:00, python-list-requ...@python.org wrote:
> > Send Python-list mailing list submissions to
> >   python-list@python.org
> >
> > To subscribe or unsubscribe via the World Wide Web, visit
> >   https://mail.python.org/mailman/listinfo/python-list
> > or, via email, send a message with subject or body 'help' to
> >   python-list-requ...@python.org
> >
> > You can reach the person managing the list at
> >   python-list-ow...@python.org
> >
> > When replying, please edit your Subject line so it is more specific
> > than "Re: Contents of Python-list digest..."
> >
> > Today's Topics:
> >
> >1. Re: Custom logging function (zljubi...@gmail.com)
> >2. Ram memory not freed after executing python script on ubuntu
> >   system (Rahul Gupta)
> >3. Re: Ram memory not freed after executing python script on
> >   ubuntu system (Chris Angelico)
> >4. Re: Custom logging function (Peter Otten)
> >5. Re: Elegant hack or gross hack? TextWrapper and escape codes
> >   (Peter Otten)
> >6. Re: Elegant hack or gross hack? TextWrapper and escape codes
> >   (Chris Angelico)
> >7. Re: Elegant hack or gross hack? TextWrapper and escape codes
> >   (Peter Otten)
> >8. Re: Ram memory not freed after executing python script on
> >   ubuntu system (Rahul Gupta)
> >9. Re: Ram memory not freed after executing python script on
> >   ubuntu system (Chris Angelico)
> >   10. Re: Behaviour of os.path.join (BlindAnagram)
> >   11. Constructing mime image attachment (Joseph L. Casale)
> >   12. Re: Behaviour of os.path.join (Eryk Sun)
> >   13. Re: Behaviour of os.path.join (Eryk Sun)
> >   14. Re: Behaviour of os.path.join (BlindAnagram)
> >   15. Re: Behaviour of os.path.join (Eryk Sun)
> --
> https://mail.python.org/mailman/listinfo/python-list



-- 
Inada Naoki  
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Ram memory not freed after executing python script on ubuntu system (rmlibre)

[Dieter Maurer]

rmli...@riseup.net wrote at 2020-5-28 18:56 -0700:
>We just ran into this problem when running our aiootp package's memory
>hard password hashing function (https://github.com/rmlibre/aiootp/). The
>memory was not being cleared after the function finished running but the
>script was still live.

I hope you are aware that Python can do nothing to change this:
the elementary memory management feature provided by operating systems
is a way to increase or decrease the "heap" (there is a different
possibility (memory mapping), but it is efficient only for large chunks
of memory). Python cannot truncate the heap below the last used
memory block. Python would need a "compacting" garbage collection
to ensure that used memory is as tight as possible; then it could
free more aggressively. But, a "compacting" garbage collection would
make life **much** more difficult for extensions (implemented in C);
especially, thousands existing extensions would stop working.

Following the example of the C runtime library, Python manages free
memory internally. This implies that operating system means do not
show exact information about the amount of really used memory
(for the operation system, free memory in the internal memory
management is used).
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Ram memory not freed after executing python script on ubuntu system (rmlibre)

[Chris Angelico]

On Fri, May 29, 2020 at 12:08 PM  wrote:
>
>
> We just ran into this problem when running our aiootp package's memory
> hard password hashing function (https://github.com/rmlibre/aiootp/).

Have you considered implementing that module in something else? Try
Cythonizing it and see if suddenly your memory usage drops - not
because of garbage collection, but because you're no longer using
integer objects at all.

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Ram memory not freed after executing python script on ubuntu system (rmlibre)

[rmlibre]

We just ran into this problem when running our aiootp package's memory
hard password hashing function (https://github.com/rmlibre/aiootp/). The
memory was not being cleared after the function finished running but the
script was still live. We tried making sure everything went out of scope
and deleting things explicitly inside the function, but that didn't
help. We even tried forcing the garbage collector to free up
unreferenced memory with import gc; gc.collect(). But that only had a
small dent in the memory that was being built up.

The most useful answer online had to do with Python's free lists being
created automatically when very large datasets were being processed
(https://stackoverflow.com/questions/23937189/how-do-i-use-subprocesses-to-force-python-to-release-memory/24126616#24126616).


After putting the memory intensive work into a separate process, as the
answer suggested, the memory was finally freed after execution.

In our case, we wound up passing the result back in a
``multiprocessing.Manager().List()``. Though, passing your whole numpy
dataset back doesn't seem feasible. I'd recommend doing the necessary
memory intensive work in a separate process and passing only the
necessary conclusions back to the main process. 

Or you could set up a multiprocessing queue to pass control messages to
the spawned (daemon) process so it returns desired results on demand. 


Here's an excerpt from our code after the fix in case it's helpful.

>@classmethod
>def passcrypt(cls, password, salt, kb=1024, cpu=1024, hardness=256):
>"""
>The ``passcrypt`` function can be highly memory intensive.
>These resources may not be freed up, & often are not, because of
>python quirks around memory management. This is a huge problem.
>So to force the release of those resources, we run the function
>in another process which is guaranteed to release them.
>"""
>cls._validate_passcrypt_args(kb, cpu, hardness)
>state = Manager().list()
>process = Process(
>target=cls._passcrypt,
>args=(password, salt),
>kwargs=dict(kb=kb, cpu=cpu, hardness=hardness, state=state),
>)
>process.start()
>process.join()
>return state.pop()




On 2020-05-28 16:00, python-list-requ...@python.org wrote:
> Send Python-list mailing list submissions to
>   python-list@python.org
> 
> To subscribe or unsubscribe via the World Wide Web, visit
>   https://mail.python.org/mailman/listinfo/python-list
> or, via email, send a message with subject or body 'help' to
>   python-list-requ...@python.org
> 
> You can reach the person managing the list at
>   python-list-ow...@python.org
> 
> When replying, please edit your Subject line so it is more specific
> than "Re: Contents of Python-list digest..."
> 
> Today's Topics:
> 
>    1. Re: Custom logging function (zljubi...@gmail.com)
>2. Ram memory not freed after executing python script on ubuntu
>   system (Rahul Gupta)
>3. Re: Ram memory not freed after executing python script on
>   ubuntu system (Chris Angelico)
>4. Re: Custom logging function (Peter Otten)
>5. Re: Elegant hack or gross hack? TextWrapper and escape codes
>   (Peter Otten)
>6. Re: Elegant hack or gross hack? TextWrapper and escape codes
>   (Chris Angelico)
>7. Re: Elegant hack or gross hack? TextWrapper and escape codes
>   (Peter Otten)
>8. Re: Ram memory not freed after executing python script on
>   ubuntu system (Rahul Gupta)
>9. Re: Ram memory not freed after executing python script on
>   ubuntu system (Chris Angelico)
>   10. Re: Behaviour of os.path.join (BlindAnagram)
>   11. Constructing mime image attachment (Joseph L. Casale)
>   12. Re: Behaviour of os.path.join (Eryk Sun)
>   13. Re: Behaviour of os.path.join (Eryk Sun)
>   14. Re: Behaviour of os.path.join (BlindAnagram)
>   15. Re: Behaviour of os.path.join (Eryk Sun)
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Ram memory not freed after executing python script on ubuntu system

[Peter J. Holzer]

On 2020-05-27 22:49:53 -0700, Rahul Gupta wrote:
> I am having a Ubuntu system which has 125 Gb of RAM. I executed few
> python scripts on that system. Those scripts uses numpy arrays and
> pandas. Now execution was over but still 50 gb of RAM and 2 Gb cache
> and 8.4 Gb of swap is occupied. At this moment nothing is running on
> the system. I have googled it. Most of th result shows that python
> garbage collector is poor in performance. I want this memory to be
> cleaned and re claim.

As others have already pointed out, a process which doesn't exist can't
use memory (on Linux or any other sane OS). So either python is still
running or something else is using the memory.

To find out what is using the memory, run the command
top
in a terminal. Then press Shift-M to sort processes by memory
consumption. The offenders should now be at the top.

hp

-- 
   _  | Peter J. Holzer| Story must make more sense than reality.
|_|_) ||
| |   | h...@hjp.at |-- Charles Stross, "Creative writing
__/   | http://www.hjp.at/ |   challenge!"


signature.asc
Description: PGP signature
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Ram memory not freed after executing python script on ubuntu system

[Sam]

On 5/28/20 12:49 AM, Rahul Gupta wrote:


I am having a Ubuntu system which has 125 Gb of RAM. I executed few python 
scripts on that system. Those scripts uses numpy arrays and pandas. Now 
execution was over but still 50 gb of RAM and 2 Gb cache and 8.4 Gb of swap is 
occupied. At this moment nothing is running on the system. I have googled it. 
Most of th result shows that python garbage collector is poor in performance. I 
want this memory to be cleaned and re claim. One of the easiest way is to 
restart the system but i dont want to restart i want a way to do this when the 
system is up and running. Kindly tell me how to do this. Thanks




Give us the output of:
cat /proc/meminfo
and
ps aux

Regards,
--
https://mail.python.org/mailman/listinfo/python-list

Re: Ram memory not freed after executing python script on ubuntu system

[Chris Angelico]

On Thu, May 28, 2020 at 6:26 PM Rahul Gupta  wrote:
>
> On Thursday, May 28, 2020 at 11:20:05 AM UTC+5:30, Rahul Gupta wrote:
> > I am having a Ubuntu system which has 125 Gb of RAM. I executed few python 
> > scripts on that system. Those scripts uses numpy arrays and pandas. Now 
> > execution was over but still 50 gb of RAM and 2 Gb cache and 8.4 Gb of swap 
> > is occupied. At this moment nothing is running on the system. I have 
> > googled it. Most of th result shows that python garbage collector is poor 
> > in performance. I want this memory to be cleaned and re claim. One of the 
> > easiest way is to restart the system but i dont want to restart i want a 
> > way to do this when the system is up and running. Kindly tell me how to do 
> > this. Thanks
> Yes i am sure 125 gb of ram is there.
> And you talked about refrences
> see these links
> https://stackoverflow.com/questions/39100971/how-do-i-release-memory-used-by-a-pandas-dataframe
> http://effbot.org/pyfaq/why-doesnt-python-release-the-memory-when-i-delete-a-large-object.htm
>

Both of those are extremely old. Also, you said that you had
terminated the Python process, so neither is relevant.

You need to look elsewhere for your memory usage. If Python isn't
running, it's not Python that's using up your memory.

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Ram memory not freed after executing python script on ubuntu system

[Rahul Gupta]

On Thursday, May 28, 2020 at 11:20:05 AM UTC+5:30, Rahul Gupta wrote:
> I am having a Ubuntu system which has 125 Gb of RAM. I executed few python 
> scripts on that system. Those scripts uses numpy arrays and pandas. Now 
> execution was over but still 50 gb of RAM and 2 Gb cache and 8.4 Gb of swap 
> is occupied. At this moment nothing is running on the system. I have googled 
> it. Most of th result shows that python garbage collector is poor in 
> performance. I want this memory to be cleaned and re claim. One of the 
> easiest way is to restart the system but i dont want to restart i want a way 
> to do this when the system is up and running. Kindly tell me how to do this. 
> Thanks
Yes i am sure 125 gb of ram is there.
And you talked about refrences 
see these links
https://stackoverflow.com/questions/39100971/how-do-i-release-memory-used-by-a-pandas-dataframe
http://effbot.org/pyfaq/why-doesnt-python-release-the-memory-when-i-delete-a-large-object.htm
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Ram memory not freed after executing python script on ubuntu system

[Chris Angelico]

On Thu, May 28, 2020 at 3:51 PM Rahul Gupta  wrote:
>
>
> I am having a Ubuntu system which has 125 Gb of RAM. I executed few python 
> scripts on that system. Those scripts uses numpy arrays and pandas. Now 
> execution was over but still 50 gb of RAM and 2 Gb cache and 8.4 Gb of swap 
> is occupied. At this moment nothing is running on the system. I have googled 
> it. Most of th result shows that python garbage collector is poor in 
> performance. I want this memory to be cleaned and re claim. One of the 
> easiest way is to restart the system but i dont want to restart i want a way 
> to do this when the system is up and running. Kindly tell me how to do this. 
> Thanks
>

The Python garbage collector is irrelevant once you have terminated
Python. I suggest you look elsewhere in your system - if 50GB of RAM
is still in use after the Python process has ended, it's not being
used by Python.

Also, that's an awful lot of RAM, but a surprising amount. Are you
sure it's not 128GB?

I don't know where you got your information from regarding the "poor
performance" of the Python GC, but without citations, it's nothing but
FUD.

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list

Ram memory not freed after executing python script on ubuntu system

[Rahul Gupta]

I am having a Ubuntu system which has 125 Gb of RAM. I executed few python 
scripts on that system. Those scripts uses numpy arrays and pandas. Now 
execution was over but still 50 gb of RAM and 2 Gb cache and 8.4 Gb of swap is 
occupied. At this moment nothing is running on the system. I have googled it. 
Most of th result shows that python garbage collector is poor in performance. I 
want this memory to be cleaned and re claim. One of the easiest way is to 
restart the system but i dont want to restart i want a way to do this when the 
system is up and running. Kindly tell me how to do this. Thanks
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: python script to give a list of prime no.

[Dan Sommers]

On Sun, 05 Apr 2020 19:46:00 +0200
Pieter van Oostrum  wrote:

> Sathvik Babu Veligatla  writes:
> 
> > hi,
> > I am new to python, and i am trying to output the prime numbers
> > beginning from 3 and i cannot get the required output.  It stops
> > after giving the output "7" and that's it.
> > CODE:
> > a = 3
> > l = []
> > while True:
> > for i in range(2,a):
> > if a%i == 0:
> > l.append(1)
> > else:
> > l.append(0)
> >
> > if l.count(1) == 0:
> > print(a)
> > a = a + 2
> > elif l.count(1) >> 0:
> > a = a + 2

[intervening optimizations omitted to conserve electrons]

> And you effectively use the counter as a boolean, so replace is by a boolean.
> 
> a = 3
> while True:
> is_prime = True
> for i in range(2,a):
> if a%i == 0:
> is_prime = False
> # once we found a divisor, it is no use to continue
> break
> if is_prime:
> print(a)
> a = a + 2
> 
> Further optimizations are possible, for example use range(2,a/2) or
> even range (2, sqrt(a)).

Now if only there were a way to get rid of that extraneous flag and
build some sort of "if the loop ran to completion" construct

a = 3
while True:
for i in range(2,a):
if a%i == 0:
# once we found a divisor, it is no use to continue
break
else:   # magic here!
print(a)
a = a + 2

The Time Machine Strikes Again,
Dan
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: python script to give a list of prime no.

[Pieter van Oostrum]

Sathvik Babu Veligatla  writes:

> hi,
> I am new to python, and i am trying to output the prime numbers beginning 
> from 3 and i cannot get the required output.
> It stops after giving the output "7" and that's it.
>
> CODE:
> a = 3
> l = []
> while True:
> for i in range(2,a):
> if a%i == 0:
> l.append(1)
> else:
> l.append(0)
>
> if l.count(1) == 0:
> print(a)
> a = a + 2
> elif l.count(1) >> 0:
> a = a + 2
> 
>
>
> Any help is appreciated,
> Thank you.

As others have already noticed, the l.count(1) >> 0: is faulty. You probably 
meant
l.count(1) > 0:, but else would be simpler.

But then, once there is a 1 in l that never disappears, so after that it will 
never print any more numbers. You should put the l = [] AFTER the while True, 
so that you start with a new empty list every cycle.

And then, putting 0 an 1's in a list and then counting how many 1's there are 
is quite inefficient. Just counting from the beginning would be more efficient. 
Like:

a = 3
while True:
count = 0
for i in range(2,a):
if a%i == 0:
count += 1
if count == 0:
print(a)
a = a + 2
else:
a = a + 2

Which can be simplified to:

a = 3
while True:
count = 0
for i in range(2,a):
if a%i == 0:
count += 1
if count == 0:
print(a)
a = a + 2

And you effectively use the counter as a boolean, so replace is by a boolean.

a = 3
while True:
is_prime = True
for i in range(2,a):
if a%i == 0:
is_prime = False
# once we found a divisor, it is no use to continue
break
if is_prime:
print(a)
a = a + 2

Further optimizations are possible, for example use range(2,a/2) or even range 
(2, sqrt(a)).
-- 
Pieter van Oostrum
www: http://pieter.vanoostrum.org/
PGP key: [8DAE142BE17999C4]
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: python script to give a list of prime no.

[Sathvik Babu Veligatla]

On Sunday, April 5, 2020 at 8:03:19 PM UTC+5:30, inhahe wrote:
> On Sun, Apr 5, 2020 at 8:26 AM Sathvik Babu Veligatla <
> sathvikveliga...@gmail.com> wrote:
> 
> > hi,
> > I am new to python, and i am trying to output the prime numbers beginning
> > from 3 and i cannot get the required output.
> > It stops after giving the output "7" and that's it.
> >
> > CODE:
> > a = 3
> > l = []
> > while True:
> > for i in range(2,a):
> > if a%i == 0:
> > l.append(1)
> > else:
> > l.append(0)
> >
> > if l.count(1) == 0:
> > print(a)
> > a = a + 2
> > elif l.count(1) >> 0:
> > a = a + 2
> >
> >
> >
> > Any help is appreciated,
> > Thank you.
> > --
> > https://mail.python.org/mailman/listinfo/python-list
> 
> 
> 
> "l = []" needs to go inside the "while True" loop. as it is, it keeps
> adding 1's and 0's to the list with each new "a" value, and then counts the
> 1's and 0's, without ever resetting the list back to empty, so once it gets
> to 9 the list has a 1 in it and l.count(1) will never be false again. 3, 5
> and 7 are prime so a 1 doesn't get added yet with those values, which is
> why 7 is the highest it gets to. so with "l = []" inside the while loop,
> "l" would be reset to empty with each new "a" value.
> 
> btw, this is a very inefficient way to compute prime numbers. for example,
> why add the 0's to the list? you never count the number of 0's so it's
> unnecessary. another example.. why use a list at all? all that matters is
> that it find *one* result where a%i==0, it doesn't matter how many results
> it found after that, so all you need is a boolean. and you might as well
> stop testing a%i for *all* i values up to i, because once it finds the
> first result that's 0, you know it's prime so you can just go to the next
> number. and you don't need i to go all the way to a, because a%i will
> always be > 0 for any i over a/2. and since you're not testing evens, it's
> actually any i over a/3. in actuality, though, computing any a%i where i >
> the square root of a is redundant because for each factor of a above the
> square root of a, you've already tested its cofactor, say a is 100. the
> square root of 100 is 10. if you're testing 100%20, that's unnecessary
> because 100/20 is 5 and you've already tested 100%5.

thank you buddy,
it worked...
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: python script to give a list of prime no.

[Sathvik Babu Veligatla]

On Sunday, April 5, 2020 at 8:03:19 PM UTC+5:30, inhahe wrote:
> On Sun, Apr 5, 2020 at 8:26 AM Sathvik Babu Veligatla <
> sathvikveliga...@gmail.com> wrote:
> 
> > hi,
> > I am new to python, and i am trying to output the prime numbers beginning
> > from 3 and i cannot get the required output.
> > It stops after giving the output "7" and that's it.
> >
> > CODE:
> > a = 3
> > l = []
> > while True:
> > for i in range(2,a):
> > if a%i == 0:
> > l.append(1)
> > else:
> > l.append(0)
> >
> > if l.count(1) == 0:
> > print(a)
> > a = a + 2
> > elif l.count(1) >> 0:
> > a = a + 2
> >
> >
> >
> > Any help is appreciated,
> > Thank you.
> > --
> > https://mail.python.org/mailman/listinfo/python-list
> 
> 
> 
> "l = []" needs to go inside the "while True" loop. as it is, it keeps
> adding 1's and 0's to the list with each new "a" value, and then counts the
> 1's and 0's, without ever resetting the list back to empty, so once it gets
> to 9 the list has a 1 in it and l.count(1) will never be false again. 3, 5
> and 7 are prime so a 1 doesn't get added yet with those values, which is
> why 7 is the highest it gets to. so with "l = []" inside the while loop,
> "l" would be reset to empty with each new "a" value.
> 
> btw, this is a very inefficient way to compute prime numbers. for example,
> why add the 0's to the list? you never count the number of 0's so it's
> unnecessary. another example.. why use a list at all? all that matters is
> that it find *one* result where a%i==0, it doesn't matter how many results
> it found after that, so all you need is a boolean. and you might as well
> stop testing a%i for *all* i values up to i, because once it finds the
> first result that's 0, you know it's prime so you can just go to the next
> number. and you don't need i to go all the way to a, because a%i will
> always be > 0 for any i over a/2. and since you're not testing evens, it's
> actually any i over a/3. in actuality, though, computing any a%i where i >
> the square root of a is redundant because for each factor of a above the
> square root of a, you've already tested its cofactor, say a is 100. the
> square root of 100 is 10. if you're testing 100%20, that's unnecessary
> because 100/20 is 5 and you've already tested 100%5.

yeah, I get that.
Initially, i put the else columns empty and had a somewhat different code. I 
thought maybe that's messing up with the code and added the 'l.append(0)' line 
and some other things.
Thank you for the feedback :-)
-- 
https://mail.python.org/mailman/listinfo/python-list

Fwd: python script to give a list of prime no.

[inhahe]

On Sun, Apr 5, 2020 at 8:26 AM Sathvik Babu Veligatla <
sathvikveliga...@gmail.com> wrote:

> hi,
> I am new to python, and i am trying to output the prime numbers beginning
> from 3 and i cannot get the required output.
> It stops after giving the output "7" and that's it.
>
> CODE:
> a = 3
> l = []
> while True:
> for i in range(2,a):
> if a%i == 0:
> l.append(1)
> else:
> l.append(0)
>
> if l.count(1) == 0:
> print(a)
> a = a + 2
> elif l.count(1) >> 0:
> a = a + 2
>
>
>
> Any help is appreciated,
> Thank you.
> --
> https://mail.python.org/mailman/listinfo/python-list



"l = []" needs to go inside the "while True" loop. as it is, it keeps
adding 1's and 0's to the list with each new "a" value, and then counts the
1's and 0's, without ever resetting the list back to empty, so once it gets
to 9 the list has a 1 in it and l.count(1) will never be false again. 3, 5
and 7 are prime so a 1 doesn't get added yet with those values, which is
why 7 is the highest it gets to. so with "l = []" inside the while loop,
"l" would be reset to empty with each new "a" value.

btw, this is a very inefficient way to compute prime numbers. for example,
why add the 0's to the list? you never count the number of 0's so it's
unnecessary. another example.. why use a list at all? all that matters is
that it find *one* result where a%i==0, it doesn't matter how many results
it found after that, so all you need is a boolean. and you might as well
stop testing a%i for *all* i values up to i, because once it finds the
first result that's 0, you know it's prime so you can just go to the next
number. and you don't need i to go all the way to a, because a%i will
always be > 0 for any i over a/2. and since you're not testing evens, it's
actually any i over a/3. in actuality, though, computing any a%i where i >
the square root of a is redundant because for each factor of a above the
square root of a, you've already tested its cofactor, say a is 100. the
square root of 100 is 10. if you're testing 100%20, that's unnecessary
because 100/20 is 5 and you've already tested 100%5.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: python script to give a list of prime no.

[Peter J. Holzer]

On 2020-04-05 05:22:45 -0700, Sathvik Babu Veligatla wrote:
> I am new to python, and i am trying to output the prime numbers
> beginning from 3 and i cannot get the required output. It stops after
> giving the output "7" and that's it.

A technique I learned when I started programming (back in the days when
computers were made from bones and rocks), is to "play computer":

Grab a piece of paper and a pencil. Make a column for each variable (a,
l, and i in your case). Then walk through the code line by line, think
about what each line does, and every time a new value is assigned to a
variable, note the new value on your paper.

Do the values of the variables change the way you expect them to? If
not, why not?

hp

PS: You can also single-step through the program in a debugger, but I
think doing this by hand on a piece of paper is very useful, especially
for a beginner, since it forces you to think what happens (The downside
is of course that you may make mistakes - but then you can learn from
them).

-- 
   _  | Peter J. Holzer| Story must make more sense than reality.
|_|_) ||
| |   | h...@hjp.at |-- Charles Stross, "Creative writing
__/   | http://www.hjp.at/ |   challenge!"


signature.asc
Description: PGP signature
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: python script to give a list of prime no.

[Barry Scott]

> On 5 Apr 2020, at 14:08, Sathvik Babu Veligatla  
> wrote:
> 
> On Sunday, April 5, 2020 at 6:09:04 PM UTC+5:30, Chris Angelico wrote:
>> On Sun, Apr 5, 2020 at 10:26 PM Sathvik Babu Veligatla
>>  wrote:
>>> 
>>> hi,
>>> I am new to python, and i am trying to output the prime numbers beginning 
>>> from 3 and i cannot get the required output.
>>> It stops after giving the output "7" and that's it.
>>> 
>>> CODE:
>>> a = 3
>>> l = []
>>> while True:
>>>for i in range(2,a):
>>>if a%i == 0:
>>>l.append(1)
>>>else:
>>>l.append(0)
>>> 
>>>if l.count(1) == 0:
>>>print(a)
>>>a = a + 2
>>>elif l.count(1) >> 0:
>>>a = a + 2
>>> 
>>> 
>> 
>> Firstly, good job on several points: you posted your code, you said
>> what the purpose is, and you said how the output differs from that.
>> Unfortunately not everyone does that :)
>> 
>> Secondly, though: What does "some_number >> 0" mean? Play around with
>> that in the interactive interpreter and see what results you get.
>> 
>> You may want to consider using a simple 'else'. When you have exactly
>> two possibilities, you don't need to say "else, if". For instance,
>> suppose you say "If today is a weekday, go to work; otherwise, play
>> games", you don't need to word it as "otherwise, if today is a
>> weekend, play games" - you just say "otherwise". You can simplify this
>> code the same way.
>> 
>> Something worth remembering is that all code can be buggy, so writing
>> less code usually means you have less bugs. Try to write your program
>> with less code rather than more. :)
>> 
>> All the best!
>> 
>> ChrisA
> 
> yeah bro, tried as said. But still no luck... :-(
> i cannot understand the issue in the code.

You need to figure out what the code is doing.

One way to do this is to add some more print statements to
show the values of your variables as the code runs.

What is the value of i for example in the for loop?
What does the list l look like?
Also try printing the values that you use the if statments.
print( l.count(1) == 0) etc.

Barry


> -- 
> https://mail.python.org/mailman/listinfo/python-list 
> 
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: python script to give a list of prime no.

[Sathvik Babu Veligatla]

On Sunday, April 5, 2020 at 6:09:04 PM UTC+5:30, Chris Angelico wrote:
> On Sun, Apr 5, 2020 at 10:26 PM Sathvik Babu Veligatla
>  wrote:
> >
> > hi,
> > I am new to python, and i am trying to output the prime numbers beginning 
> > from 3 and i cannot get the required output.
> > It stops after giving the output "7" and that's it.
> >
> > CODE:
> > a = 3
> > l = []
> > while True:
> > for i in range(2,a):
> > if a%i == 0:
> > l.append(1)
> > else:
> > l.append(0)
> >
> > if l.count(1) == 0:
> > print(a)
> > a = a + 2
> > elif l.count(1) >> 0:
> > a = a + 2
> >
> >
> 
> Firstly, good job on several points: you posted your code, you said
> what the purpose is, and you said how the output differs from that.
> Unfortunately not everyone does that :)
> 
> Secondly, though: What does "some_number >> 0" mean? Play around with
> that in the interactive interpreter and see what results you get.
> 
> You may want to consider using a simple 'else'. When you have exactly
> two possibilities, you don't need to say "else, if". For instance,
> suppose you say "If today is a weekday, go to work; otherwise, play
> games", you don't need to word it as "otherwise, if today is a
> weekend, play games" - you just say "otherwise". You can simplify this
> code the same way.
> 
> Something worth remembering is that all code can be buggy, so writing
> less code usually means you have less bugs. Try to write your program
> with less code rather than more. :)
> 
> All the best!
> 
> ChrisA

yeah bro, tried as said. But still no luck... :-(
i cannot understand the issue in the code.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: python script to give a list of prime no.

[Sathvik Babu Veligatla]

On Sunday, April 5, 2020 at 6:04:20 PM UTC+5:30, Orges Leka wrote:
> You can try the following:
> It is based on trial division and very slow, compared to the state of the
> art:
> 
> import math
> def is_prime(n):
> if int(math.sqrt(n))**2 == n:
> return(False)
> for i in range(2,int(math.ceil(math.sqrt(n:
> if n%i==0:
> return(False)
> return(True)
> 
> then:
> 
> primes = [x for x in range(1,1000) if is_prime(x)]
> 
> print(primes)
> 
> Kind regards,
> Orges
> 
> Am So., 5. Apr. 2020 um 14:27 Uhr schrieb Sathvik Babu Veligatla <
> sathvikveliga...@gmail.com>:
> 
> > hi,
> > I am new to python, and i am trying to output the prime numbers beginning
> > from 3 and i cannot get the required output.
> > It stops after giving the output "7" and that's it.
> >
> > CODE:
> > a = 3
> > l = []
> > while True:
> > for i in range(2,a):
> > if a%i == 0:
> > l.append(1)
> > else:
> > l.append(0)
> >
> > if l.count(1) == 0:
> > print(a)
> > a = a + 2
> > elif l.count(1) >> 0:
> > a = a + 2
> >
> >
> >
> > Any help is appreciated,
> > Thank you.
> > --
> > https://mail.python.org/mailman/listinfo/python-list
> >
> 
> 
> -- 
> Mit freundlichen Grüßen
> Herr Dipl. Math. Orges Leka
> 
> Mobil: 015751078391
> Email: orges.l...@googlemail.com
> Holzheimerstraße 25
> 65549 Limburg

umm, first of all, thank you for the quick response.
But, I am basically trying to get the output using the simplest means 
possible(beginner's trial :-)). So can you try to run it tell me, why it isn't 
working?
thank you.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: python script to give a list of prime no.

[Chris Angelico]

On Sun, Apr 5, 2020 at 10:35 PM Orges Leka  wrote:
>
> You can try the following:
> It is based on trial division and very slow, compared to the state of the
> art:
>

I think it's more helpful to assist the OP in learning coding, rather
than provide a completely different function to do a similar purpose
:) I strongly suspect that the OP is doing this prime number finder as
part of learning Python, not as a means of actually finding prime
numbers.

BTW, you may want to be careful of mixing floats and ints. It's
generally safest to stick to just integers - fewer things to have to
think about.

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: python script to give a list of prime no.

[Chris Angelico]

On Sun, Apr 5, 2020 at 10:26 PM Sathvik Babu Veligatla
 wrote:
>
> hi,
> I am new to python, and i am trying to output the prime numbers beginning 
> from 3 and i cannot get the required output.
> It stops after giving the output "7" and that's it.
>
> CODE:
> a = 3
> l = []
> while True:
> for i in range(2,a):
> if a%i == 0:
> l.append(1)
> else:
> l.append(0)
>
> if l.count(1) == 0:
> print(a)
> a = a + 2
> elif l.count(1) >> 0:
> a = a + 2
>
>

Firstly, good job on several points: you posted your code, you said
what the purpose is, and you said how the output differs from that.
Unfortunately not everyone does that :)

Secondly, though: What does "some_number >> 0" mean? Play around with
that in the interactive interpreter and see what results you get.

You may want to consider using a simple 'else'. When you have exactly
two possibilities, you don't need to say "else, if". For instance,
suppose you say "If today is a weekday, go to work; otherwise, play
games", you don't need to word it as "otherwise, if today is a
weekend, play games" - you just say "otherwise". You can simplify this
code the same way.

Something worth remembering is that all code can be buggy, so writing
less code usually means you have less bugs. Try to write your program
with less code rather than more. :)

All the best!

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: python script to give a list of prime no.

[Orges Leka]

You can try the following:
It is based on trial division and very slow, compared to the state of the
art:

import math
def is_prime(n):
if int(math.sqrt(n))**2 == n:
return(False)
for i in range(2,int(math.ceil(math.sqrt(n:
if n%i==0:
return(False)
return(True)

then:

primes = [x for x in range(1,1000) if is_prime(x)]

print(primes)

Kind regards,
Orges

Am So., 5. Apr. 2020 um 14:27 Uhr schrieb Sathvik Babu Veligatla <
sathvikveliga...@gmail.com>:

> hi,
> I am new to python, and i am trying to output the prime numbers beginning
> from 3 and i cannot get the required output.
> It stops after giving the output "7" and that's it.
>
> CODE:
> a = 3
> l = []
> while True:
> for i in range(2,a):
> if a%i == 0:
> l.append(1)
> else:
> l.append(0)
>
> if l.count(1) == 0:
> print(a)
> a = a + 2
> elif l.count(1) >> 0:
> a = a + 2
>
>
>
> Any help is appreciated,
> Thank you.
> --
> https://mail.python.org/mailman/listinfo/python-list
>


-- 
Mit freundlichen Grüßen
Herr Dipl. Math. Orges Leka

Mobil: 015751078391
Email: orges.l...@googlemail.com
Holzheimerstraße 25
65549 Limburg
-- 
https://mail.python.org/mailman/listinfo/python-list

python script to give a list of prime no.

[Sathvik Babu Veligatla]

hi,
I am new to python, and i am trying to output the prime numbers beginning from 
3 and i cannot get the required output.
It stops after giving the output "7" and that's it.

CODE:
a = 3
l = []
while True:
for i in range(2,a):
if a%i == 0:
l.append(1)
else:
l.append(0)

if l.count(1) == 0:
print(a)
a = a + 2
elif l.count(1) >> 0:
a = a + 2



Any help is appreciated,
Thank you.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: 【Regarding Performance of a Python Script....】

[Cameron Simpson]

On 28Feb2020 19:24, Kenzi  wrote:

I have a question regarding a simple code snippet in Python:

from subprocess import check_output
for i in range(1024):
   check_output(['/bin/bash', '-c', 'echo 42'], close_fds=True)

*I wonder why running it in Python 3.7 is much faster than Python 2.7? *
(Python 3.7 is still faster, after I used *xrange * in Python 2.7)


On a lot of UNIX systems you can use the strace command to inspect what 
is happening. I am personally surprised there's a significant difference 
between the Pythons because I'd expect the expense of invoking bash to 
dominate the cost here (consider using /bin/sh, which on many systems 
isn't bash but something smaller/faster for noninteractive shell 
scripting).


Write a script running just the check_output() call one, and strace it:

   strace python -c "from subprocess import check_output; check_output(['/bin/bash', 
'-c', 'echo 42'], close_fds=True)"

See if your 2 Pythons are doing something difference at the system call 
level.


Cheers,
Cameron Simpson 
--
https://mail.python.org/mailman/listinfo/python-list

Re: 【Regarding Performance of a Python Script....】

[Peter J. Holzer]

On 2020-02-28 19:24:18 -0500, Kenzi wrote:
>  I have a question regarding a simple code snippet in Python:
> 
> from subprocess import check_output
> for i in range(1024):
> check_output(['/bin/bash', '-c', 'echo 42'], close_fds=True)
> 
> *I wonder why running it in Python 3.7 is much faster than Python 2.7? *
> (Python 3.7 is still faster, after I used *xrange * in Python 2.7)

I think almost all of the time is spent in the child processes, so it
doesn't matter whether you use range or xrange.

On my laptop, the program takes about 2.1 seconds with python 2.7 and
1.6 seconds with python 3.6. So the difference is 0.5 seconds overall or
about 500 µs per execution.

strace shows that python 2.7 explicitely closes all unneeded file
descriptors below 1024 (even though most of them aren't actually open)
in the child before execing bash, python 3 doesn't do that.

I can see no other obvious difference. 

So that's ~ 1020 extra system calls. If this is indeed the only
difference, that's about 500 ns per system call. That sounds plausible.

hp

-- 
   _  | Peter J. Holzer| Story must make more sense than reality.
|_|_) ||
| |   | h...@hjp.at |-- Charles Stross, "Creative writing
__/   | http://www.hjp.at/ |   challenge!"


signature.asc
Description: PGP signature
-- 
https://mail.python.org/mailman/listinfo/python-list

【Regarding Performance of a Python Script....】

[Kenzi]

Hello there,

 I have a question regarding a simple code snippet in Python:

from subprocess import check_output
for i in range(1024):
check_output(['/bin/bash', '-c', 'echo 42'], close_fds=True)

*I wonder why running it in Python 3.7 is much faster than Python 2.7? *
(Python 3.7 is still faster, after I used *xrange * in Python 2.7)

Thanks all!
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: insert data in python script

[DL Neil via Python-list]

On 19/02/20 4:52 AM, alberto wrote:


Hi,
I solve it with external file as follows

import preos
# pass name, Tc, Pc, omega
methane = preos.Molecule("methane", -82.59 + 273.15, 45.99, 0.011)
methane.print_params()

thanks to everyone



It is difficult to learn a new programming language, even for 
professional programmers - and more-so if English is not one's 
native-language. Well done!


Now, if you could tell us how to capture and control methane, perhaps 
the world would become a better place...



Ciao!
--
Regards =dn
--
https://mail.python.org/mailman/listinfo/python-list

Re: insert data in python script

[alberto]

Il giorno martedì 18 febbraio 2020 11:01:11 UTC+1, David ha scritto:
> On Tue, 18 Feb 2020 at 20:45, alberto  wrote:
> > Il giorno martedì 18 febbraio 2020 09:34:51 UTC+1, DL Neil ha scritto:
> 
> > > > my code preos in one file preos.py
> > > > my commands are
> > > >
> > > > alberto@HENDRIX ~/PREOS $ python3.5
> > > > Python 3.5.2 (default, Oct  8 2019, 13:06:37)
> > > > [GCC 5.4.0 20160609] on linux
> > > > Type "help", "copyright", "credits" or "license" for more information.
> > >  import preos
> > >  methane = Molecule("methane", -82.59 + 273.15, 45.99, 0.011)
> > > > Traceback (most recent call last):
> > > >File "", line 1, in 
> > > > NameError: name 'Molecule' is not defined
> 
> > honestly i don't understand what i have to do.
> 
> $ python3
> Python 3.5.3 (default, Sep 27 2018, 17:25:39)
> [GCC 6.3.0 20170516] on linux
> Type "help", "copyright", "credits" or "license" for more information.
> >>> print(digits)
> Traceback (most recent call last):
>   File "", line 1, in 
> NameError: name 'digits' is not defined
> >>> print(string.digits)
> Traceback (most recent call last):
>   File "", line 1, in 
> NameError: name 'string' is not defined
> >>> import string
> >>> print(digits)
> Traceback (most recent call last):
>   File "", line 1, in 
> NameError: name 'digits' is not defined
> >>> print(string.digits)
> 0123456789
> >>>

Hi, 
I solve it with external file as follows

import preos
# pass name, Tc, Pc, omega
methane = preos.Molecule("methane", -82.59 + 273.15, 45.99, 0.011)
methane.print_params()

thanks to everyone

regards

Alberto
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: insert data in python script

[Michael Torrie]

On 2/18/20 2:41 AM, alberto wrote:
> Il giorno martedì 18 febbraio 2020 09:34:51 UTC+1, DL Neil ha scritto:
>> The first instruction (immediately above) imports the module preos.py. 
>> That works (no error message!).
>>
>> The second instruction refers to a Python class called Molecule. That 
>> fails. The error message says that 'Molecule' is not defined.
>>
>> Yet we can 'see' it. It *has* been defined! What is going on???
>>
>> In this case, we need to tell Python that Molecule is part of the preos 
>> module. So back to your original code (top):
>>
>> methane = preos.Molecule("methane", -82.59 + 273.15, 45.99, 0.011)
^^
He explicitly told you what to do.

The problem was "Molecule" is part of the "preos" module, so you have to
refer to it by the module name first.

>> Please refer to earlier message. If Module were called from code in the 
>> preos.py file, then the "preos." prefix would not be necessary.
>>
>> The formal term for this situation is "namespaces". Because Molecule is 
>> defined within the preos.py module's namespace we need to tell Python 
>> exactly where Molecule can be found. In the same way that we might say: 
>> if someone in Antarctica wants to see Alberto, (s)he will have to go to 
>> Italy to find him...
>>
>>
>> Don't hesitate to say if you think my reply is too complicated/advanced. 
>> People here are happy to help...
>> -- 
>> Regards =dn
> 
> hi
> honestly i don't understand what i have to do.
> I have been using python for too little time.
> could you help me understand

Did you read everything DL Neil said?  He told you what to do, including
giving you the line of code you need to replace, but also explained why.
 I don't know how one could make it more clear than what he said.

Spend some time to really try to understand what DL Neil is saying.
this is essential Python knowledge he's trying to impart.  Learning to
work with modules and namespaces in Python is pretty much required when
learning and using Python.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: insert data in python script

[David]

On Tue, 18 Feb 2020 at 20:45, alberto  wrote:
> Il giorno martedì 18 febbraio 2020 09:34:51 UTC+1, DL Neil ha scritto:

> > > my code preos in one file preos.py
> > > my commands are
> > >
> > > alberto@HENDRIX ~/PREOS $ python3.5
> > > Python 3.5.2 (default, Oct  8 2019, 13:06:37)
> > > [GCC 5.4.0 20160609] on linux
> > > Type "help", "copyright", "credits" or "license" for more information.
> >  import preos
> >  methane = Molecule("methane", -82.59 + 273.15, 45.99, 0.011)
> > > Traceback (most recent call last):
> > >File "", line 1, in 
> > > NameError: name 'Molecule' is not defined

> honestly i don't understand what i have to do.

$ python3
Python 3.5.3 (default, Sep 27 2018, 17:25:39)
[GCC 6.3.0 20170516] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> print(digits)
Traceback (most recent call last):
  File "", line 1, in 
NameError: name 'digits' is not defined
>>> print(string.digits)
Traceback (most recent call last):
  File "", line 1, in 
NameError: name 'string' is not defined
>>> import string
>>> print(digits)
Traceback (most recent call last):
  File "", line 1, in 
NameError: name 'digits' is not defined
>>> print(string.digits)
0123456789
>>>
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: insert data in python script

[alberto]

Il giorno martedì 18 febbraio 2020 09:34:51 UTC+1, DL Neil ha scritto:
> ...
> 
> >> import preos
> >> # pass name, Tc, Pc, omega
> >> methane = preos.Molecule("methane", -82.59 + 273.15, 45.99, 0.011)
> >> methane.print_params()
> 
> ...
> 
> > my code preos in one file preos.py
> > my commands are
> > 
> > alberto@HENDRIX ~/PREOS $ python3.5
> > Python 3.5.2 (default, Oct  8 2019, 13:06:37)
> > [GCC 5.4.0 20160609] on linux
> > Type "help", "copyright", "credits" or "license" for more information.
>  import preos
>  methane = Molecule("methane", -82.59 + 273.15, 45.99, 0.011)
> > Traceback (most recent call last):
> >File "", line 1, in 
> > NameError: name 'Molecule' is not defined
> 
> 
> The first instruction (immediately above) imports the module preos.py. 
> That works (no error message!).
> 
> The second instruction refers to a Python class called Molecule. That 
> fails. The error message says that 'Molecule' is not defined.
> 
> Yet we can 'see' it. It *has* been defined! What is going on???
> 
> In this case, we need to tell Python that Molecule is part of the preos 
> module. So back to your original code (top):
> 
> methane = preos.Molecule("methane", -82.59 + 273.15, 45.99, 0.011)
> 
> 
> Please refer to earlier message. If Module were called from code in the 
> preos.py file, then the "preos." prefix would not be necessary.
> 
> The formal term for this situation is "namespaces". Because Molecule is 
> defined within the preos.py module's namespace we need to tell Python 
> exactly where Molecule can be found. In the same way that we might say: 
> if someone in Antarctica wants to see Alberto, (s)he will have to go to 
> Italy to find him...
> 
> 
> Don't hesitate to say if you think my reply is too complicated/advanced. 
> People here are happy to help...
> -- 
> Regards =dn

hi
honestly i don't understand what i have to do.
I have been using python for too little time.
could you help me understand

regards 

Alberto

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: insert data in python script

[DL Neil via Python-list]

...


import preos
# pass name, Tc, Pc, omega
methane = preos.Molecule("methane", -82.59 + 273.15, 45.99, 0.011)
methane.print_params()


...


my code preos in one file preos.py
my commands are

alberto@HENDRIX ~/PREOS $ python3.5
Python 3.5.2 (default, Oct  8 2019, 13:06:37)
[GCC 5.4.0 20160609] on linux
Type "help", "copyright", "credits" or "license" for more information.

import preos
methane = Molecule("methane", -82.59 + 273.15, 45.99, 0.011)

Traceback (most recent call last):
   File "", line 1, in 
NameError: name 'Molecule' is not defined



The first instruction (immediately above) imports the module preos.py. 
That works (no error message!).


The second instruction refers to a Python class called Molecule. That 
fails. The error message says that 'Molecule' is not defined.


Yet we can 'see' it. It *has* been defined! What is going on???

In this case, we need to tell Python that Molecule is part of the preos 
module. So back to your original code (top):


methane = preos.Molecule("methane", -82.59 + 273.15, 45.99, 0.011)


Please refer to earlier message. If Module were called from code in the 
preos.py file, then the "preos." prefix would not be necessary.


The formal term for this situation is "namespaces". Because Molecule is 
defined within the preos.py module's namespace we need to tell Python 
exactly where Molecule can be found. In the same way that we might say: 
if someone in Antarctica wants to see Alberto, (s)he will have to go to 
Italy to find him...



Don't hesitate to say if you think my reply is too complicated/advanced. 
People here are happy to help...

--
Regards =dn
--
https://mail.python.org/mailman/listinfo/python-list

Re: insert data in python script

[alberto]

Il giorno lunedì 17 febbraio 2020 17:48:07 UTC+1, alberto ha scritto:
> Hi, 
> I would use this script to evaluate fugacity coefficient with PENG-ROBINSON 
> equation, but I don't understand the correct mode to insert data
> 
> import numpy as np
> import matplotlib.pyplot as plt
> from scipy.optimize import newton
> 
> R = 8.314e-5  # universal gas constant, m3-bar/K-mol
> class Molecule:
> """
> Store molecule info here
> """
> def __init__(self, name, Tc, Pc, omega):
> """
> Pass parameters desribing molecules
> """
> #! name
> self.name = name
> #! Critical temperature (K)
> self.Tc = Tc
> #! Critical pressure (bar)
> self.Pc = Pc
> #! Accentric factor
> self.omega = omega
> 
> def print_params(self):
> """
> Print molecule parameters.
> """
> print("""Molecule: %s.
> \tCritical Temperature = %.1f K
> \tCritical Pressure = %.1f bar.
> \tAccentric factor = %f""" % (self.name, self.Tc, self.Pc, 
> self.omega))
> 
> def preos(molecule, T, P, plotcubic=True, printresults=True):
> """
> Peng-Robinson equation of state (PREOS)
> 
> http://en.wikipedia.org/wiki/Equation_of_state#Peng.E2.80.93Robinson_equation_of_state
> :param molecule: Molecule molecule of interest
> :param T: float temperature in Kelvin
> :param P: float pressure in bar
> :param plotcubic: bool plot cubic polynomial in compressibility factor
> :param printresults: bool print off properties
> Returns a Dict() of molecule properties at this T and P.
> """
> # build params in PREOS
> Tr = T / molecule.Tc  # reduced temperature
> a = 0.457235 * R**2 * molecule.Tc**2 / molecule.Pc
> b = 0.0777961 * R * molecule.Tc / molecule.Pc
> kappa = 0.37464 + 1.54226 * molecule.omega - 0.26992 * molecule.omega**2
> alpha = (1 + kappa * (1 - np.sqrt(Tr)))**2
> 
> A = a * alpha * P / R**2 / T**2
> B = b * P / R / T
> 
> # build cubic polynomial
> def g(z):
> """
> Cubic polynomial in z from EOS. This should be zero.
> :param z: float compressibility factor
> """
> return z**3 - (1 - B) * z**2 + (A - 2*B - 3*B**2) * z - (
> A * B - B**2 - B**3)
> 
> # Solve cubic polynomial for the compressibility factor
> z = newton(g, 1.0)  # compressibility factor
> rho = P / (R * T * z)  # density
> 
> # fugacity coefficient comes from an integration
> fugacity_coeff = np.exp(z - 1 - np.log(z - B) - A / np.sqrt(8) / B * 
> np.log(
> (z + (1 + np.sqrt(2)) * B) / (z + (1 - np.sqrt(2)) * B)))
> 
> if printresults:
> print("""PREOS calculation at
> \t T = %.2f K
> \t P = %.2f bar""" % (T, P))
> print("\tCompressibility factor : ", z)
> print("\tFugacity coefficient: ", fugacity_coeff)
> print("\tFugacity at pressure %.3f bar = %.3f bar" % (
> P, fugacity_coeff * P))
> print("\tDensity: %f mol/m3" % rho)
> print("\tMolar volume: %f L/mol" % (1.0 / rho * 1000))
> print("\tDensity: %f v STP/v" % (rho * 22.4 / 1000))
> print("\tDensity of ideal gas at same conditions: %f v STP/v" % (
> rho * 22.4/ 1000 * z))
> 
> if plotcubic:
> # Plot the cubic equation to visualize the roots
> zz = np.linspace(0, 1.5)  # array for plotting
> 
> plt.figure()
> plt.plot(zz, g(zz), color='k')
> plt.xlabel('Compressibility, $z$')
> plt.ylabel('Cubic $g(z)$')
> plt.axvline(x=z)
> plt.axhline(y=0)
> plt.title('Root found @ z = %.2f' % z)
> plt.show()
> return {"density(mol/m3)": rho, "fugacity_coefficient": fugacity_coeff,
> "compressibility_factor": z, "fugacity(bar)": fugacity_coeff * P,
> "molar_volume(L/mol)": 1.0 / rho * 1000.0}
> 
> def preos_reverse(molecule, T, f, plotcubic=False, printresults=True):
> """
> Reverse Peng-Robinson equation of state (PREOS) to obtain pressure for a 
> particular fugacity
> :param molecule: Molecule molecule of interest
> :param T: float temperature in Kelvin
> :param f: float fugacity in bar
> :param plotcubic: bool plot cubic polynomial in compressibility factor
> :param printresults: bool print off properties
> Returns a Dict() of molecule properties at this T and f.
> """
> # build function to minimize: difference between desired fugacity and 
> that obtained from preos
> def g(P):
> """
> :param P: pressure
> """
> return (f - preos(molecule, T, P, plotcubic=False, 
> printresults=False)["fugacity(bar)"])
> 
> # Solve preos for the pressure
> P = newton(g, f)  # pressure
> 
> # Obtain remaining parameters
> pars = preos(molecule, T, P, plotcubic=plotcubic, 
> printresults=printresults)
> rho = pars["density(mol

Re: insert data in python script

[DL Neil via Python-list]

Please help us to help you!
1 is all of this code in a single file or spread across (at least) two 
modules? What are their names? What is the directory structure?

2 copy-paste the actual error message received.


It works for me!
1 not knowing your circumstances, I put all the code in one file
2 updated one line (possibly due to above)

# methane = preos.Molecule("methane", -82.59 + 273.15, 45.99, 0.011)
methane = Molecule("methane", -82.59 + 273.15, 45.99, 0.011)

dn $ python3 Projects/molecule.py
Molecule: methane.
Critical Temperature = 190.6 K
Critical Pressure = 46.0 bar.
Accentric factor = 0.011000


NB if the bulk of the code is stored in a module named preos.py then it 
is necessary to first:


import peos

and restore the above change.

NBB under such conditions peos.py must be located in the file-system 
where the 'mainline' can find (and import) it!

--
Regards =dn
--
https://mail.python.org/mailman/listinfo/python-list

insert data in python script

[alberto]

Hi, 
I would use this script to evaluate fugacity coefficient with PENG-ROBINSON 
equation, but I don't understand the correct mode to insert data

import numpy as np
import matplotlib.pyplot as plt
from scipy.optimize import newton

R = 8.314e-5  # universal gas constant, m3-bar/K-mol
class Molecule:
"""
Store molecule info here
"""
def __init__(self, name, Tc, Pc, omega):
"""
Pass parameters desribing molecules
"""
#! name
self.name = name
#! Critical temperature (K)
self.Tc = Tc
#! Critical pressure (bar)
self.Pc = Pc
#! Accentric factor
self.omega = omega

def print_params(self):
"""
Print molecule parameters.
"""
print("""Molecule: %s.
\tCritical Temperature = %.1f K
\tCritical Pressure = %.1f bar.
\tAccentric factor = %f""" % (self.name, self.Tc, self.Pc, self.omega))

def preos(molecule, T, P, plotcubic=True, printresults=True):
"""
Peng-Robinson equation of state (PREOS)

http://en.wikipedia.org/wiki/Equation_of_state#Peng.E2.80.93Robinson_equation_of_state
:param molecule: Molecule molecule of interest
:param T: float temperature in Kelvin
:param P: float pressure in bar
:param plotcubic: bool plot cubic polynomial in compressibility factor
:param printresults: bool print off properties
Returns a Dict() of molecule properties at this T and P.
"""
# build params in PREOS
Tr = T / molecule.Tc  # reduced temperature
a = 0.457235 * R**2 * molecule.Tc**2 / molecule.Pc
b = 0.0777961 * R * molecule.Tc / molecule.Pc
kappa = 0.37464 + 1.54226 * molecule.omega - 0.26992 * molecule.omega**2
alpha = (1 + kappa * (1 - np.sqrt(Tr)))**2

A = a * alpha * P / R**2 / T**2
B = b * P / R / T

# build cubic polynomial
def g(z):
"""
Cubic polynomial in z from EOS. This should be zero.
:param z: float compressibility factor
"""
return z**3 - (1 - B) * z**2 + (A - 2*B - 3*B**2) * z - (
A * B - B**2 - B**3)

# Solve cubic polynomial for the compressibility factor
z = newton(g, 1.0)  # compressibility factor
rho = P / (R * T * z)  # density

# fugacity coefficient comes from an integration
fugacity_coeff = np.exp(z - 1 - np.log(z - B) - A / np.sqrt(8) / B * np.log(
(z + (1 + np.sqrt(2)) * B) / (z + (1 - np.sqrt(2)) * B)))

if printresults:
print("""PREOS calculation at
\t T = %.2f K
\t P = %.2f bar""" % (T, P))
print("\tCompressibility factor : ", z)
print("\tFugacity coefficient: ", fugacity_coeff)
print("\tFugacity at pressure %.3f bar = %.3f bar" % (
P, fugacity_coeff * P))
print("\tDensity: %f mol/m3" % rho)
print("\tMolar volume: %f L/mol" % (1.0 / rho * 1000))
print("\tDensity: %f v STP/v" % (rho * 22.4 / 1000))
print("\tDensity of ideal gas at same conditions: %f v STP/v" % (
rho * 22.4/ 1000 * z))

if plotcubic:
# Plot the cubic equation to visualize the roots
zz = np.linspace(0, 1.5)  # array for plotting

plt.figure()
plt.plot(zz, g(zz), color='k')
plt.xlabel('Compressibility, $z$')
plt.ylabel('Cubic $g(z)$')
plt.axvline(x=z)
plt.axhline(y=0)
plt.title('Root found @ z = %.2f' % z)
plt.show()
return {"density(mol/m3)": rho, "fugacity_coefficient": fugacity_coeff,
"compressibility_factor": z, "fugacity(bar)": fugacity_coeff * P,
"molar_volume(L/mol)": 1.0 / rho * 1000.0}

def preos_reverse(molecule, T, f, plotcubic=False, printresults=True):
"""
Reverse Peng-Robinson equation of state (PREOS) to obtain pressure for a 
particular fugacity
:param molecule: Molecule molecule of interest
:param T: float temperature in Kelvin
:param f: float fugacity in bar
:param plotcubic: bool plot cubic polynomial in compressibility factor
:param printresults: bool print off properties
Returns a Dict() of molecule properties at this T and f.
"""
# build function to minimize: difference between desired fugacity and that 
obtained from preos
def g(P):
"""
:param P: pressure
"""
return (f - preos(molecule, T, P, plotcubic=False, 
printresults=False)["fugacity(bar)"])

# Solve preos for the pressure
P = newton(g, f)  # pressure

# Obtain remaining parameters
pars = preos(molecule, T, P, plotcubic=plotcubic, printresults=printresults)
rho = pars["density(mol/m3)"]
fugacity_coeff = pars["fugacity_coefficient"]
z = pars["compressibility_factor"]

return {"density(mol/m3)": rho, "fugacity_coefficient": fugacity_coeff,
"compressibility_factor": z, "pressure(bar)": P,
"molar_volume(L/mol)": 1.0 / rho * 1000.0}

# TODO: Implement mixture in object-oriented way 

Execute python script from VB6 to display Outlook Window

[wojtek . dominicana]

I have to prepare e-mail in python (e-mail address, body html, attachments 
etc.) and show Outlook window for user who will check everything and correct if 
necessary and will be able to send it.
My script work well :


o = win32com.client.Dispatch("Outlook.Application")
Msg = o.CreateItem(0)
Msg.To = "a...@gmail.com"
Msg.CC = ""
Msg.Subject = "The random subject"
Msg.HtmlBody = "content of e-mail"
Msg.Attachments.Add(attachment1)
Msg.Display()

This script work well but I have some problem with my Outlook 2019 or Command 
window (Win 10, 64). When Outlook is closed i can call my script using Shell() 
or ShellExecute() and everything is ok but when MS Outlook is open my script 
doesnt work. I use the same  Shell() or ShellExecute() function and still does 
not work although windows executing this script. It stops in the loine 
"win32com.client.Dispatch("Outlook.Application")". I tried many tricks and I 
gave up. The most important is that when I use windows command line and run my 
script manyally (python sendoutlookmail.py) everything works well regardless of 
state of Outlook (open or closed).
I think there is a different between calling script via Shell() function and 
run it directly from command line.
I tried of course solution as follows:

Set obj = CreateObject("WScript.Shell")
obj.Run sPythonFile 

but its the same situation. I createt bat file and I execute this bat file 
which has inside commant to execute my python script.

I have no idea how to run python script on my system that works the same as I 
call it directly from windows command line.

regards
Wojtek
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: how can i run python script in spamassassin

[Skip Montanaro]

Check out SpamBayes. It runs on Linux and is written in Python. Like
Spamassasin, it has no plugin architecture, but perhaps you'll discover you
don't need it, or can more easily tweak SpamBayes to call your external
script.

Skip


On Thu, Oct 3, 2019, 2:15 PM Doris Marca Guaraca 
wrote:

>
> Hello, I'm sorry to bother you, I just reviewed this post, the Python
> beginner, the Linux beginner, needs to run spamassassin, and now I'm trying
> to do something very similar with a Python script is for a project, maybe
> you can help me thanks.
>
> I appreciate it a lot.
>
> Regards
> Doris
>
> --
> https://mail.python.org/mailman/listinfo/python-list
>
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: how can i run python script in spamassassin

[Michael Torrie]

On 10/3/19 1:10 PM, Doris Marca Guaraca wrote:
> 
> Hello, I'm sorry to bother you, I just reviewed this post, the Python 
> beginner, the Linux beginner, needs to run spamassassin, and now I'm trying 
> to do something very similar with a Python script is for a project, maybe you 
> can help me thanks.

I haven't looked deeply at Spamassassin for a long time (nearly 10
years), but last time I worked with it, it was written entirely in Perl
and at that time there was no built-in facility to call out to external
programs or scripts.  The only way to do that would be to write an
extension module for Spamassassin in Perl that would run the external
script and communicate with it somehow.

Actually I'm not even sure what you're asking.  Are you trying to run a
python script from Spamassassin? If so, what are you trying to accomplish?
-- 
https://mail.python.org/mailman/listinfo/python-list

how can i run python script in spamassassin

[Doris Marca Guaraca]

Hello, I'm sorry to bother you, I just reviewed this post, the Python beginner, 
the Linux beginner, needs to run spamassassin, and now I'm trying to do 
something very similar with a Python script is for a project, maybe you can 
help me thanks.

I appreciate it a lot.

Regards
Doris

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: How control a GUI for an unrelated application from a Python script?

[Christian Seberino]

On Friday, June 14, 2019 at 1:42:17 PM UTC-5, Rob Gaddi wrote:
> Condolences.  TI is a world-leader in giving every eval board its own 
> complicated, proprietary digital interface, then not documenting it 
> because "You can just use the provided software" that hasn't been 
> updated since 2001 and doesn't actually let you test the thing you need to.
> 
> The underlying FPGA board that it's built on has its own page at 
> https://opalkelly.com/products/xem3010/ with an SDK.  That may turn out 
> to be your best way in.

You don't know how right you are.  It gets even betterSince they don't 
update their Windows GUI apps...10 years later it won't run on the latest 
Windows.  I had to set up a Windows 7 machine to run their @#$# software!!! ;)

This is the dreaded GUI app ...

http://www.ti.com/tool/ADCPRO

(That Opal Kelly code is very nice but isn't designed to support all the TI 
boards the FPGA plugs into.  Believe me I tried that route. ;)

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: How control a GUI for an unrelated application from a Python script?

[Rob Gaddi]

On 6/14/19 11:14 AM, Christian Seberino wrote:



Out of curiosity, what hardware?


Texas Instruments ADS1675REF card



Condolences.  TI is a world-leader in giving every eval board its own 
complicated, proprietary digital interface, then not documenting it 
because "You can just use the provided software" that hasn't been 
updated since 2001 and doesn't actually let you test the thing you need to.


The underlying FPGA board that it's built on has its own page at 
https://opalkelly.com/products/xem3010/ with an SDK.  That may turn out 
to be your best way in.


--
Rob Gaddi, Highland Technology -- www.highlandtechnology.com
Email address domain is currently out of order.  See above to fix.
--
https://mail.python.org/mailman/listinfo/python-list

Re: How control a GUI for an unrelated application from a Python script?

[Christian Seberino]

> Out of curiosity, what hardware?

Texas Instruments ADS1675REF card
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: How control a GUI for an unrelated application from a Python script?

[Rob Gaddi]

On 6/14/19 8:49 AM, Christian Seberino wrote:

Thanks for all the help.  I'll definitely try to bypass the GUI first if 
possible.  This is on Windows 7 so maybe AutoIt will do the trick if can't 
avoid the GUI.  Thanks again everyone.



Out of curiosity, what hardware?

--
Rob Gaddi, Highland Technology -- www.highlandtechnology.com
Email address domain is currently out of order.  See above to fix.
--
https://mail.python.org/mailman/listinfo/python-list

Re: How control a GUI for an unrelated application from a Python script?

[Christian Seberino]

Thanks for all the help.  I'll definitely try to bypass the GUI first if 
possible.  This is on Windows 7 so maybe AutoIt will do the trick if can't 
avoid the GUI.  Thanks again everyone.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: How control a GUI for an unrelated application from a Python script?

[Chris Hills]

On 6/14/2019 12:49 AM, Christian Seberino wrote:

I have a third party GUI that manages some hardware.

I want to control the hardware from a Python script.

This seems to mean I need to somehow have Python code
   that imitates a human doing the necessary
 actions on the GUI (selecting menu options, pressing buttons, etc.)

Is this possible / easy / doable?

Thanks!

Chris



Hi Chris

THere are two approaches for this. If the appication is inheritenty 
controllable, e.g. through COM, then you can use win32com.


Alternatively, you can go down the route of "Robotic Process Automation" 
using something like https://github.com/OakwoodAI/Automagica


Hope this helps,

Chris Hills
--
https://mail.python.org/mailman/listinfo/python-list

Re: How control a GUI for an unrelated application from a Python script?

[Thomas Jollans]

On 14/06/2019 01.49, Christian Seberino wrote:
> I have a third party GUI that manages some hardware.
>
> I want to control the hardware from a Python script.

Forget about the GUI, see if you can control your device without it.

See how well the device is documented. Maybe there's an API? If not for
Python, maybe for C? If not, maybe it's easy to write your own driver in
Python?

Writing a driver for a simple message-based wire protocol is not very
hard, and may be easier than reliably controlling somebody else's GUI
with a script.


>
> This seems to mean I need to somehow have Python code
>   that imitates a human doing the necessary
> actions on the GUI (selecting menu options, pressing buttons, etc.)
>
> Is this possible / easy / doable?
>
> Thanks!
>
> Chris

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: How control a GUI for an unrelated application from a Python script?

[Julio Oña]

try https://pypi.org/project/PyAutoIt/

Regards
Julio

El jue., 13 de jun. de 2019 a la(s) 21:37, Michael Torrie
(torr...@gmail.com) escribió:
>
> On 06/13/2019 05:49 PM, Christian Seberino wrote:
> > I have a third party GUI that manages some hardware.
> >
> > I want to control the hardware from a Python script.
> >
> > This seems to mean I need to somehow have Python code
> >   that imitates a human doing the necessary
> > actions on the GUI (selecting menu options, pressing buttons, etc.)
> >
> > Is this possible
>
> Maybe.
>
> > / easy
> No.
>
> > doable?
>
> Maybe.
>
> It's kind of the old "if you have to ask" sort of question.
>
> There are ways of programatically driving other applications' user
> interfaces.  You haven't said what OS you are using.  We used to use an
> application called AutoIt to drive GUI programs.  You can send clicks,
> keystrokes, and work with certain controls (read values, set values,
> etc), at least if they are standard win32 widgets.  More and more
> applications draw their own controls these days rather than use win32
> widgets, which wouldn't be usable for that kind of modification.
>
> As far as modifying a running GUI to add functionality, the answer to
> that is probably "very difficult" to "impossible."  If the GUI itself is
> just a frontend for command-line tools or even libraries that interact
> with the hardware, then you probably could develop your own GUI from
> scratch.
>
> This is one reason why free and open source software wins. It's just
> that much more flexible to manipulate and make to do cool new things.
> --
> https://mail.python.org/mailman/listinfo/python-list
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: How control a GUI for an unrelated application from a Python script?

[Michael Torrie]

On 06/13/2019 05:49 PM, Christian Seberino wrote:
> I have a third party GUI that manages some hardware.
> 
> I want to control the hardware from a Python script.
> 
> This seems to mean I need to somehow have Python code
>   that imitates a human doing the necessary
> actions on the GUI (selecting menu options, pressing buttons, etc.)
> 
> Is this possible

Maybe.

> / easy 
No.

> doable?

Maybe.

It's kind of the old "if you have to ask" sort of question.

There are ways of programatically driving other applications' user
interfaces.  You haven't said what OS you are using.  We used to use an
application called AutoIt to drive GUI programs.  You can send clicks,
keystrokes, and work with certain controls (read values, set values,
etc), at least if they are standard win32 widgets.  More and more
applications draw their own controls these days rather than use win32
widgets, which wouldn't be usable for that kind of modification.

As far as modifying a running GUI to add functionality, the answer to
that is probably "very difficult" to "impossible."  If the GUI itself is
just a frontend for command-line tools or even libraries that interact
with the hardware, then you probably could develop your own GUI from
scratch.

This is one reason why free and open source software wins. It's just
that much more flexible to manipulate and make to do cool new things.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: How control a GUI for an unrelated application from a Python script?

[Chris Angelico]

On Fri, Jun 14, 2019 at 9:51 AM Christian Seberino  wrote:
>
> I have a third party GUI that manages some hardware.
>
> I want to control the hardware from a Python script.
>
> This seems to mean I need to somehow have Python code
>   that imitates a human doing the necessary
> actions on the GUI (selecting menu options, pressing buttons, etc.)
>
> Is this possible / easy / doable?
>

Possible? Yes. Easy? Probably not. So I would recommend first seeing
if there's any sort of command-line tool, or command-line invocation
for the GUI tool (some programs will open a GUI if given no arguments,
but you can provide args to make them do stuff straight away).

Once you've exhausted all options of easy automation, yes, you CAN
have a program manipulate a GUI. It's fiddly, though.

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list

How control a GUI for an unrelated application from a Python script?

[Christian Seberino]

I have a third party GUI that manages some hardware.

I want to control the hardware from a Python script.

This seems to mean I need to somehow have Python code
  that imitates a human doing the necessary
actions on the GUI (selecting menu options, pressing buttons, etc.)

Is this possible / easy / doable?

Thanks!

Chris
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Block Ctrl+S while running Python script at Windows console?

[Peter J. Holzer]

On 2019-03-19 14:22:10 -, Grant Edwards wrote:
> On 2019-03-18, Malcolm Greene  wrote:
> > Wondering if there's a way to have my Python scripts ignore these
> > Ctrl+S signals or if this behavior is outside of my Python script's
> > control.
> 
> This has nothing to do with Python does it?
> 
> Isn't Python is just writing to stdout and those write calls are
> blocking due because the terminal emulator has stopped reading the
> other end of the pipe/pty/queue/buffer/whatever-it's-called-in-windows?

Yes, but there might be a way to control this behaviour from Python. On
Unix systems you can use the termios functions to turn flow control on
and off. This doesn't help you on Windows, but curses also has raw() and
noraw() functions which do that (among other things) and might work on
Windows, too. However, curses does a lot more than just control flow
control, so it may not be appropriate for the OP's problem.

hp

-- 
   _  | Peter J. Holzer| we build much bigger, better disasters now
|_|_) || because we have much more sophisticated
| |   | h...@hjp.at | management tools.
__/   | http://www.hjp.at/ | -- Ross Anderson 


signature.asc
Description: PGP signature
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Block Ctrl+S while running Python script at Windows console?

[eryk sun]

On 3/19/19, Grant Edwards  wrote:
>
> This has nothing to do with Python does it?
>
> Isn't Python is just writing to stdout and those write calls are
> blocking due because the terminal emulator has stopped reading

It turns out the original poster had quick-edit mode enabled and the
pause was due to accidentally selecting text. A Python script can
disable this mode via GetConsoleMode / SetConsoleMode [1], called via
PyWin32 or ctypes. The original mode should be restored using an
atexit function. With quick-edit mode disabled, selecting text
requires enabling mark mode via the edit menu or Ctrl+M.

If it had been a Ctrl+S pause, the only way to programmatically
disable it for the current console is to disable line-input mode,
which is what msvcrt.getwch() does temporarily. However, disabling
line-input mode in general would break the REPL and input().

[1]: https://docs.microsoft.com/en-us/windows/console/getconsolemode

> the other end of the pipe/pty/queue/buffer/whatever-it's-called-
> in-windows?

In Windows 8+, a console client has handles for files on the ConDrv
device, which by default includes "Reference" (inherited reference to
a console), "Connect" (generic functions, such as setting the title),
"Input" (stdin), and "Output" (stdout, stderr). The host process
(conhost.exe) has a handle for ConDrv as well, from which it receives
IOCTL requests from its registered clients.

Windows 10 (release 1809) also supports pseudoconsoles [2], which are
roughly equivalent to Unix PTYs. The pseudoconsole interface is
restricted to virtual-terminal streams over a pair of pipes, so the
console's function-based API still has to be implemented for clients
by an instance of conhost.exe. It basically looks like the following:
Pseudoconsole Host <-- Pipe I/O --> conhost.exe <-- ConDrv I/O -->
Clients.

[2]: https://docs.microsoft.com/en-us/windows/console/pseudoconsoles
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Block Ctrl+S while running Python script at Windows console?

[Malcolm Greene]

> This has nothing to do with Python does it? Isn't Python is just writing to 
> stdout and those write calls are blocking due because the terminal emulator 
> has stopped reading the other end of the 
> pipe/pty/queue/buffer/whatever-it's-called-in-windows?

You're right. But I wasn't sure. I know Python can trap Ctrl+C break signals, 
so I wondered if there was a similar tie-in for Ctrl+S.

Eryk did a great job explaining the tech details of what's happening behind the 
scenes in the Windows world. In my case it turned out that my clicking between 
windows was inadvertently selecting text pausing the running process. 
Unchecking the Windows console's Quick Edit mode stops this behavior.

Malcolm
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Block Ctrl+S while running Python script at Windows console?

[Grant Edwards]

On 2019-03-18, Malcolm Greene  wrote:

> Wondering if there's a way to have my Python scripts ignore these
> Ctrl+S signals or if this behavior is outside of my Python script's
> control.

This has nothing to do with Python does it?

Isn't Python is just writing to stdout and those write calls are
blocking due because the terminal emulator has stopped reading the
other end of the pipe/pty/queue/buffer/whatever-it's-called-in-windows?

-- 
Grant Edwards   grant.b.edwardsYow! It's some people
  at   inside the wall!  This is
  gmail.combetter than mopping!

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Block Ctrl+S while running Python script at Windows console?

[eryk sun]

On 3/18/19, Dennis Lee Bieber  wrote:
> On Mon, 18 Mar 2019 14:38:40 -0400, "Malcolm Greene" 
> declaimed the following:
>
>>
>>Wondering if there's a way to have my Python scripts ignore these Ctrl+S
>> signals or if this behavior is outside of my Python script's control. If
>> there's a way to disable this behavior under Windows 10/Windows Server
>> 2016, I'm open to that as well.
>>
>
>   / (XOFF/XON) are traditional terminal control codes to
> stop/start transmission. The behavior is baked into the console being used.

The Windows console does not really implement XOFF (DC3, Ctrl+S) and
XON (DC1, Ctrl+Q) terminal behavior. In line-input mode, it implements
Ctrl+S as equivalent to pressing the pause key (i.e. VK_PAUSE, which
has the same numeric value as DC3), so it's meant to be familiar to
people who are used to working in terminals. But, in contrast to a
terminal emulator, there is no requirement to press Ctrl+Q (i.e. DC1)
to resume. Almost any key suffices to resume the console, except for
the pause key and modifiers (Ctrl, Alt, Shift). In particular,
pressing Ctrl+S again, as the original poster did, will resume the
Windows console, but it would not resume a terminal emulator.

If enabled, the console's extended editing keys feature overrides the
default pause behavior of Ctrl+S. It can be enabled via the console's
ExtendedEditKey registry value, or in the defaults and properties
dialogs by enabling "extended text selection keys". Disabling it also
disables immediate access to the console's new text selection keys
such as Shift+Arrows, in which case we have to first enter mark mode
via Ctrl+M.

The default extended-editing keymap doesn't define anything for
Ctrl+S, so it's just a DC3 (0x13) character. In principle the extended
editing keys can be customized to map Ctrl+S to a virtual key such as
VK_PAUSE (i.e. retain the default behavior), but, as far as I know,
the binary format of the console's ExtendedEditKeyCustom registry
value is not documented.

Regardless of the extended editing keys setting, we can always read
Ctrl+S via msvcrt.getwch(), since it disables line-input mode.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Block Ctrl+S while running Python script at Windows console? (solved)

[Malcolm Greene]

Eryk,

> Another common culprit is quick-edit mode, in which case a stray mouse click 
> can select text, even just a single character. The console pauses while text 
> is selected.

MYSTERY SOLVED !! THANK YOU !!

Apparently, while mouse clicking between windows, I was inadvertently selecting 
a char on my console, thereby pausing the process that was running. Disabling 
Quick-Edit mode via the Properties dialog fixes this behavior.

I tried other console property setting combinations to block Ctrl+S keyboard 
behavior, but without success. Not a problem since I'm pretty sure I'm not 
typing random Ctrl+S keystrokes while working. 

Thanks again Eryk!
Malcolm
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Block Ctrl+S while running Python script at Windows console?

[eryk sun]

On 3/18/19, eryk sun  wrote:
>
> Ctrl+S functions as pause in line-edit mode if extended text selection
> is enabled in the console defaults or properties dialog

Correction, it pauses if extended text selection is *disabled*.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Block Ctrl+S while running Python script at Windows console?

[eryk sun]

On 3/18/19, Malcolm Greene  wrote:
>
> I'm running some Python 3.6 scripts at the Windows 10/Windows Server 2016
> console. In my every day workflow, I seem to be accidentally sending Ctrl+S
> keystrokes to some of console sessions, pausing my running scripts until I
> send another corresponding Ctrl+S to un-pause the affected scripts. My
> challenge is I don't know when I'm sending these keystrokes other than
> seeing scripts that seem to have stopped, clicking on their console window,
> and typing Ctrl+S to unblock them.
>
> Wondering if there's a way to have my Python scripts ignore these Ctrl+S
> signals or if this behavior is outside of my Python script's control. If
> there's a way to disable this behavior under Windows 10/Windows Server 2016,
> I'm open to that as well.

Ctrl+S functions as pause in line-edit mode if extended text selection
is enabled in the console defaults or properties dialog, It's
implemented by blocking console I/O operations in the host process,
conhost.exe.  Pretty much any key will resume. It doesn't have to be
Ctrl+Q or Ctrl+S. Another common culprit is quick-edit mode, in which
case a stray mouse click can select text, even just a single
character. The console pauses while text is selected.
-- 
https://mail.python.org/mailman/listinfo/python-list

Block Ctrl+S while running Python script at Windows console?

[Malcolm Greene]

I'm running some Python 3.6 scripts at the Windows 10/Windows Server 2016 
console. In my every day workflow, I seem to be accidentally sending Ctrl+S 
keystrokes to some of console sessions, pausing my running scripts until I send 
another corresponding Ctrl+S to un-pause the affected scripts. My challenge is 
I don't know when I'm sending these keystrokes other than seeing scripts that 
seem to have stopped, clicking on their console window, and typing Ctrl+S to 
unblock them.

Wondering if there's a way to have my Python scripts ignore these Ctrl+S 
signals or if this behavior is outside of my Python script's control. If 
there's a way to disable this behavior under Windows 10/Windows Server 2016, 
I'm open to that as well.

Thank you,
Malcolm
-- 
https://mail.python.org/mailman/listinfo/python-list

matplotlib.plot.show always blocks the execution of python script

[lampahome]

I want to plot a graph and *still run following code without closing the
graph automatically like Matlab does*.

I try plt.show(block=False) , it failed and appear in a small moment then
close itself.

I also try plt.draw() or interactive mode , it failed, too.
plt.draw() will block until I close it.
interactive mode and plot.show() will appear nothing

My version is below:
user@ya:~/$ sudo pip freeze | grep matplotlib
matplotlib==2.2.3
user@ya:~/$ sudo pip -V
pip 18.1 from /usr/local/lib/python2.7/dist-packages/pip (python 2.7)

Environments:
I only execute script in Ubuntu ex: user@ya: python xxx.py Distributor ID:
Ubuntu
Description: Ubuntu 16.04.5 LTS
Release: 16.04
Codename: xenial

Can anyone help me? I just want to do like Matlab which won't close the
plotted graph even if the script finishes.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Python script

[breamoreboy]

On Thursday, December 7, 2017 at 2:06:46 PM UTC, prvn...@gmail.com wrote:
> Hi All,
> I am new to python need help to write a script in python
> my requirement is :-
> write a python script to print sentence from a txt file to another txt file
> 
> Regards,
> Praveen

Read this https://docs.python.org/3/tutorial/inputoutput.html.  Run your 
favourite editor, type your code and save it.  Try running the script and when 
you hit problems show us your code, the input, the expected output and state 
exactly what happened.  Then we'll happily help you.

--
Kindest regards.

Mark Lawrence.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Python script

[Joel Goldstick]

On Thu, Dec 7, 2017 at 9:06 AM,  wrote:

> Hi All,
> I am new to python need help to write a script in python
> my requirement is :-
> write a python script to print sentence from a txt file to another txt file
>
> Regards,
> Praveen
> --
> https://mail.python.org/mailman/listinfo/python-list
>

So, the way it works here is that you should write some code as best you
can.  Paste it into your question using plaintext so that the formatting
remains as coded, and tell us what errors you get (copy the traceback).
People won't write code for you here, but they will help you figure out
what you don't understand

-- 
Joel Goldstick
http://joelgoldstick.com/blog
http://cc-baseballstats.info/stats/birthdays
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Python script

[Christian Gollwitzer]

Am 07.12.17 um 15:06 schrieb prvn.m...@gmail.com:

Hi All,
I am new to python need help to write a script in python
my requirement is :-
write a python script to print sentence from a txt file to another txt file


txt = open("another.txt", "w")
print("sentence from txt file", file = txt)


Christian
--
https://mail.python.org/mailman/listinfo/python-list

Python script

[prvn . mora]

Hi All,
I am new to python need help to write a script in python
my requirement is :-
write a python script to print sentence from a txt file to another txt file

Regards,
Praveen
-- 
https://mail.python.org/mailman/listinfo/python-list