Re: Find the path of a shell command

[Michael Torrie]

On 10/11/22 22:00, Paulo da Silva wrote:
> Hi!
> 
> The simple question: How do I find the full path of a shell command 
> (linux), i.e. how do I obtain the corresponding of, for example,
> "type rm" in command line?
> 
> The reason:
> I have python program that launches a detached rm. It works pretty well 
> until it is invoked by cron! I suspect that for cron we need to specify 
> the full path.
> Of course I can hardcode /usr/bin/rm. But, is rm always in /usr/bin? 
> What about other commands?

There are certain standards that suggest where to look.  For example,
there's the Linux Filesystem Hiearchy Standard 3.0:
https://refspecs.linuxfoundation.org/FHS_3.0/fhs/ch03s04.html

In short, you want to hard code /bin for a command like rm.  And yes it
will always be in /bin on any standard Linux OS.

Despite modern distros making /bin and /usr/bin the same directory, if
the target OS is anywhere close to the standard, you can always find the
basic commands in /bin.  I would not hard code any script to use
/usr/bin for any basic commands and I would not use anything other than
/bin/sh or /bin/bash as the shell script shebang if you want any sort of
portability.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: What might suddenly provoke this poplib error?

[Chris Green]

Peter J. Holzer  wrote:
> [-- text/plain, encoding quoted-printable, charset: us-ascii, 28 lines --]
> 
> On 2022-10-13 13:47:07 +0100, Chris Green wrote:
> > I have a short python3 program that collects E-Mails from a 'catchall'
> > mailbox, sends the few that might be interesting to me and dumps the
> > rest.
> > 
> > It has suddenly (after working for some years) started throwing the
> > following:-
> [...]
> > poplib.error_proto: b'-ERR internal server error'
> [...]
> > It seems to be saying that the POP3 server has a problem,
> 
> Yes. "-ERR" is the normal start of a POP error reply, so the message
> "-ERR internal server error" was sent by the POP server.
> 
> > if so there's not much I can do about it as it's my hosting provider's
> > mail server.
> 
> You can call you hosting provider and ask them to fix the problem.
> 
After a couple of hours someone obviously fixed something and the
error has stopped now.

Thanks everyone.

-- 
Chris Green
·
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: What might suddenly provoke this poplib error?

[Chris Green]

MRAB  wrote:
[snip boring code]
> > 
> > It seems to be saying that the POP3 server has a problem, if so there's not
> > much I can do about it as it's my hosting provider's mail server.  Is it
> > really saying the server has a problem?
> > 
> As you've already ascertained that it's a server error, I'd just like to 
> suggest that you add a sleep before retrying. If it has timed out after 
> 5 minutes, I doubt there's much point in retrying immediately.

I guess I could do that but this has only happened once in quite a few
years so I'm not sure if I;ll bother! :-)

The program is only run half-hourly by cron.

-- 
Chris Green
·
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: What might suddenly provoke this poplib error?

[dn]

On 14/10/2022 01.47, Chris Green wrote:

I have a short python3 program that collects E-Mails from a 'catchall'
mailbox, sends the few that might be interesting to me and dumps the
rest.

It has suddenly (after working for some years) started throwing the
following:-

 Traceback (most recent call last):
   File "/home/chris/.mutt/bin/getCatchall.py", line 83, in 
 pop3.pass_('brzmilla')
   File "/usr/lib/python3.10/poplib.py", line 218, in pass_
 return self._shortcmd('PASS %s' % pswd)
   File "/usr/lib/python3.10/poplib.py", line 181, in _shortcmd
 return self._getresp()
   File "/usr/lib/python3.10/poplib.py", line 157, in _getresp
 raise error_proto(resp)
 poplib.error_proto: b'-ERR internal server error'


The section of code throwing the error is as follows:-

 #
 #
 # Connect to the POP3 server, get message count, exit if no messages
 #
 for t in range(10): # retry 10 times
 try:
 pop3 = poplib.POP3_SSL('mail3.gridhost.co.uk',timeout=300)
 break
 except TimeoutError:
 if t == 9:
 log.err("Timed out 10 times, giving up")
 exit(1)
 else:
 log.warn("Timed out, try " + str(t))

 pop3.user('catch...@isbd.net')
 pop3.pass_('brzmilla')
 numMessages = len(pop3.list()[1])
 if (numMessages == 0):
 break


It seems to be saying that the POP3 server has a problem, if so there's not
much I can do about it as it's my hosting provider's mail server.  Is it
really saying the server has a problem?



There's a range of possibilities here. The first question to ask (as you 
probably have) is "what has changed?" - has the Python/PSL on this 
computer been updated*, has the computer's SSL/TLS library been updated, 
etc. Then there are possible changes at 'the other end' - per previous 
responses.


* IIRC the last changes to email lib were circa 3.5, but poplib's doc 
page shows more recent updates - including to timeouts in v3.9 (YMMV).



There are changes afoot in the TLS arena. Older algorithms being dumped, 
and newer, stronger, harder, faster, better mechanisms (with racing 
stripes) being added. Not using POP3, I can't say how much impact such 
may have...


Try using an independent checker to investigate continuity of access. 
From your own machine (I use "s_client -connect..." but for IMAP), thus 
should be able to replicate the 'error' conversation at the level of the 
POP3 'conversation'. Alternately, use an on-line service such as: 
https://www.wormly.com/test-pop3-mail-server
Disclaimer: again, not using POP3, I have never used this service, it 
'popped-up' whilst searching on DuckDuckGo.



Many general-purpose service/server-providers seem rather heavy-handed 
when it comes to email security, often as a result of some knee-jerk 
response to a 'live-issue' - ultimately caused by them not having 
competent email-specialists on-staff. Accordingly, they may have 
'changed policy' and 'forgotten' to let mere-clients know. (although, if 
your standard email access continues unabated, this seems less-likely) 
That said, it never hurts to ask/be in friendly-contact...



PS is that really your password? If so, ...
--
Regards,
=dn
--
https://mail.python.org/mailman/listinfo/python-list

[Python-announce] ANN: pvlib-0.9.3 released

[Dr. Mark Alexander Mikofski PhD]

Dear Pythonistas and solar power enthusiasts,

The maintainers are happy to announce a new release of pvlib python:
software for simulating performance of photovoltaic solar energy systems.

*See what's new for v0.9.3:*
** *https://pvlib-python.readthedocs.io/en/stable/whatsnew.html

*Releases are available from PyPI and the conda-forge channel:*
* https://pypi.org/project/pvlib/
* https://anaconda.org/conda-forge/pvlib
* https://anaconda.org/conda-forge/pvlib-python

*Note: you can now install from conda-forge using either "pvlib" or
"pvlib-python"*

*Read the Documentation:*
* https://pvlib-python.readthedocs.io/en/stable/index.html

*Report issues & contribute:*
* https://github.com/pvlib/pvlib-python

*Highlights:*
* Anton Driesse has joined the maintainers. Anton has been a consistent
contributor, a frequent poster on Google Groups, and a fixture in the PV
modeling community for many years. We gladly welcome Anton and are grateful
for his support.
* Several new functions to calculate spectral mismatch

of
photovoltaic devices between incident and reference spectra. Compare this
to the existing FSLR spectral mismatch function.
* The Townsend snow loss model

is
now available. Compare this with the existing snow loss model from Marion.
* A new generic linear temperature model

function
and class that can convert between PVsyst, Faiman, and SAPM thermal model
coefficients.
* A new altitude look-up function

using
public data from Mapzen, a project for open map data hosted by AWS.
* There were four new first time contributors! Thank you!
* There was a training using JupyterHub at PVPMC 2022
. Check it out!

*Thank you for using pvlib python!*

-- 
Mark Mikofski, PhD (2005)
*Fiat Lux*
___
Python-announce-list mailing list -- python-announce-list@python.org
To unsubscribe send an email to python-announce-list-le...@python.org
https://mail.python.org/mailman3/lists/python-announce-list.python.org/
Member address: arch...@mail-archive.com

Re: Fwd: Can you help me with this Python question?

[Axy via Python-list]

Well, although I never used pandas and never will, if that's about 
artworks, that's mine.


Obviously, you need to iterate columns and sum values returned by the 
snippet you provided. A quick search tells us to use colums property. 
So, it might look like this:


na_sum = sum(df[name].isnull().sum() for name in df.columns)

Axy

On 13/10/2022 13:44, Sarah Wallace wrote:

For a python class I am taking..

In this challenge, you'll be working with a DataFrame that contains data
about artworks, and it contains many missing values.

Your task is to create a variable called na_sum that contains the total
number of missing values in the DataFrame. When that's completed, print out
your answer!

Hint: The code given below will give you the number of missing (NaN) values
for the *Name* column in the DataFrame. How would you edit the code to get
the missing values for every column in the DataFrame?
Extra hint: You'll be returning a single number which is the final sum() of
everything.

df['Name'].isnull().sum()


--
https://mail.python.org/mailman/listinfo/python-list

Re: What might suddenly provoke this poplib error?

[Peter J. Holzer]

On 2022-10-13 13:47:07 +0100, Chris Green wrote:
> I have a short python3 program that collects E-Mails from a 'catchall'
> mailbox, sends the few that might be interesting to me and dumps the
> rest.
> 
> It has suddenly (after working for some years) started throwing the
> following:-
[...]
> poplib.error_proto: b'-ERR internal server error'
[...]
> It seems to be saying that the POP3 server has a problem,

Yes. "-ERR" is the normal start of a POP error reply, so the message
"-ERR internal server error" was sent by the POP server.

> if so there's not much I can do about it as it's my hosting provider's
> mail server.

You can call you hosting provider and ask them to fix the problem.

hp

-- 
   _  | Peter J. Holzer| Story must make more sense than reality.
|_|_) ||
| |   | h...@hjp.at |-- Charles Stross, "Creative writing
__/   | http://www.hjp.at/ |   challenge!"


signature.asc
Description: PGP signature
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: What might suddenly provoke this poplib error?

[MRAB]

On 2022-10-13 13:47, Chris Green wrote:

I have a short python3 program that collects E-Mails from a 'catchall'
mailbox, sends the few that might be interesting to me and dumps the
rest.

It has suddenly (after working for some years) started throwing the
following:-

 Traceback (most recent call last):
   File "/home/chris/.mutt/bin/getCatchall.py", line 83, in 
 pop3.pass_('brzmilla')
   File "/usr/lib/python3.10/poplib.py", line 218, in pass_
 return self._shortcmd('PASS %s' % pswd)
   File "/usr/lib/python3.10/poplib.py", line 181, in _shortcmd
 return self._getresp()
   File "/usr/lib/python3.10/poplib.py", line 157, in _getresp
 raise error_proto(resp)
 poplib.error_proto: b'-ERR internal server error'


The section of code throwing the error is as follows:-

 #
 #
 # Connect to the POP3 server, get message count, exit if no messages
 #
 for t in range(10): # retry 10 times
 try:
 pop3 = poplib.POP3_SSL('mail3.gridhost.co.uk',timeout=300)
 break
 except TimeoutError:
 if t == 9:
 log.err("Timed out 10 times, giving up")
 exit(1)
 else:
 log.warn("Timed out, try " + str(t))

 pop3.user('catch...@isbd.net')
 pop3.pass_('brzmilla')
 numMessages = len(pop3.list()[1])
 if (numMessages == 0):
 break


It seems to be saying that the POP3 server has a problem, if so there's not
much I can do about it as it's my hosting provider's mail server.  Is it
really saying the server has a problem?

As you've already ascertained that it's a server error, I'd just like to 
suggest that you add a sleep before retrying. If it has timed out after 
5 minutes, I doubt there's much point in retrying immediately.

--
https://mail.python.org/mailman/listinfo/python-list

What might suddenly provoke this poplib error?

[Chris Green]

I have a short python3 program that collects E-Mails from a 'catchall'
mailbox, sends the few that might be interesting to me and dumps the
rest.

It has suddenly (after working for some years) started throwing the
following:-

Traceback (most recent call last):
  File "/home/chris/.mutt/bin/getCatchall.py", line 83, in 
pop3.pass_('brzmilla')
  File "/usr/lib/python3.10/poplib.py", line 218, in pass_
return self._shortcmd('PASS %s' % pswd)
  File "/usr/lib/python3.10/poplib.py", line 181, in _shortcmd
return self._getresp()
  File "/usr/lib/python3.10/poplib.py", line 157, in _getresp
raise error_proto(resp)
poplib.error_proto: b'-ERR internal server error'


The section of code throwing the error is as follows:-

#
#
# Connect to the POP3 server, get message count, exit if no messages   
#
for t in range(10): # retry 10 times
try:
pop3 = poplib.POP3_SSL('mail3.gridhost.co.uk',timeout=300)
break
except TimeoutError: 
if t == 9:
log.err("Timed out 10 times, giving up")
exit(1)
else:
log.warn("Timed out, try " + str(t))

pop3.user('catch...@isbd.net')
pop3.pass_('brzmilla')
numMessages = len(pop3.list()[1])
if (numMessages == 0):
break


It seems to be saying that the POP3 server has a problem, if so there's not
much I can do about it as it's my hosting provider's mail server.  Is it
really saying the server has a problem?



-- 
Chris Green
·
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Find the path of a shell command

[Jon Ribbens via Python-list]

On 2022-10-12, Paulo da Silva  wrote:
> Às 22:38 de 12/10/22, Jon Ribbens escreveu:
>> On 2022-10-12, Jon Ribbens  wrote:
>>> On 2022-10-12, Paulo da Silva  wrote:
 Às 19:14 de 12/10/22, Jon Ribbens escreveu:
> On 2022-10-12, Paulo da Silva  
> wrote:
>> Às 05:00 de 12/10/22, Paulo da Silva escreveu:
>>> Hi!
>>>
>>> The simple question: How do I find the full path of a shell command
>>> (linux), i.e. how do I obtain the corresponding of, for example,
>>> "type rm" in command line?
>>>
>>> The reason:
>>> I have python program that launches a detached rm. It works pretty well
>>> until it is invoked by cron! I suspect that for cron we need to specify
>>> the full path.
>>> Of course I can hardcode /usr/bin/rm. But, is rm always in /usr/bin?
>>> What about other commands?
>>>
>> Thank you all who have responded so far.
>> I think that the the suggestion of searching the PATH env seems the best.
>> Another thing that I thought of is that of the 'which', but, to avoid
>> the mentioned recurrent problem of not knowing where 'which' is I would
>> use 'type' instead. 'type' is a bash (sh?) command.
>
> If you're using subprocess.run / subprocess.Popen then the computer is
> *already* searching PATH for you.
 Yes, and it works out of cron.
> Your problem must be that your cron
> job is being run without PATH being set, perhaps you just need to edit
> your crontab to set PATH to something sensible.
 I could do that, but I am using /etc/cron.* for convenience.

> Or just hard-code your
> program to run '/bin/rm' explicitly, which should always work (unless
> you're on Windows, of course!)
 It can also be in /bin, at least.
>>>
>>> I assume you mean /usr/bin. But it doesn't matter. As already
>>> discussed, even if 'rm' is in /usr/bin, it will be in /bin as well
>>> (or /usr/bin and /bin will be symlinks to the same place).
>>>
 A short idea is to just check /bin/rm and /usr/bin/rm, but I prefer
 searching thru PATH env. It only needs to do that once.
>>>
>>> I cannot think of any situation in which that will help you. But if for
>>> some reason you really want to do that, you can use the shutil.which()
>>> function from the standard library:
>>>
>>>  >>> import shutil
>>>  >>> shutil.which('rm')
>>>  '/usr/bin/rm'
>> 
>> Actually if I'm mentioning shutil I should probably mention
>> shutil.rmtree() as well, which does the same as 'rm -r', without
>> needing to find or run any other executables.
> Except that you can't have parallel tasks, at least in an easy way.
> Using Popen I just launch rm's and end the script.

[threading.Thread(target=shutil.rmtree, args=(item,)).start()
for item in items_to_delete]

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: What might suddenly provoke this poplib error?

[Chris Green]

A further little bit of information, I tried running getCatchall.py
from the command prompt and there was a long wait before it output the
same error message.  I.e. it looks rather as if the server is not
responding to requests. (A 'long wait' is a minute or two)

-- 
Chris Green
·
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Find the path of a shell command

[Eli the Bearded]

In comp.lang.python, jkn   wrote:
> On Wednesday, October 12, 2022 at 6:12:23 AM UTC+1, jak wrote:
>> I'm afraid you will have to look for the command in every path listed in 
>> the PATH environment variable.
> erm, or try 'which rm' ?

It is so hilarious seeing the responses to this thread. Hint: what do
you think the `which` program does?

$ touch /var/tmp/rm
$ chmod 755 /var/tmp/rm
$ env -i PATH=/etc:/usr:/lib:/var/tmp:/tmp /usr/bin/which rm
/var/tmp/rm
$ mv /var/tmp/rm /tmp/ 
$ env -i PATH=/etc:/usr:/lib:/var/tmp:/tmp /usr/bin/which rm
/tmp/rm
$ 

Elijah
--
/usr/bin/busybox rm /tmp/rm
-- 
https://mail.python.org/mailman/listinfo/python-list

Fwd: Can you help me with this Python question?

[Sarah Wallace]

For a python class I am taking..

In this challenge, you'll be working with a DataFrame that contains data
about artworks, and it contains many missing values.

Your task is to create a variable called na_sum that contains the total
number of missing values in the DataFrame. When that's completed, print out
your answer!

Hint: The code given below will give you the number of missing (NaN) values
for the *Name* column in the DataFrame. How would you edit the code to get
the missing values for every column in the DataFrame?
Extra hint: You'll be returning a single number which is the final sum() of
everything.

df['Name'].isnull().sum()

-- 
Thanks!

*Sarah Wallace*
sarah.wallac...@gmail.com




-- 
Thanks!

*Sarah Wallace*
sarah.wallac...@gmail.com
214.300.1064
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Find the path of a shell command

[Mats Wichmann]

On 10/12/22 14:51, Paulo da Silva wrote:

Às 19:14 de 12/10/22, Jon Ribbens escreveu:
On 2022-10-12, Paulo da Silva  
wrote:

Às 05:00 de 12/10/22, Paulo da Silva escreveu:

Hi!

The simple question: How do I find the full path of a shell command
(linux), i.e. how do I obtain the corresponding of, for example,
"type rm" in command line?

The reason:
I have python program that launches a detached rm. It works pretty well
until it is invoked by cron! I suspect that for cron we need to specify
the full path.
Of course I can hardcode /usr/bin/rm. But, is rm always in /usr/bin?
What about other commands?


Thank you all who have responded so far.
I think that the the suggestion of searching the PATH env seems the 
best.

Another thing that I thought of is that of the 'which', but, to avoid
the mentioned recurrent problem of not knowing where 'which' is I would
use 'type' instead. 'type' is a bash (sh?) command.


If you're using subprocess.run / subprocess.Popen then the computer is
*already* searching PATH for you.

Yes, and it works out of cron.

Your problem must be that your cron
job is being run without PATH being set, perhaps you just need to edit
your crontab to set PATH to something sensible.

I could do that, but I am using /etc/cron.* for convenience.


Or just hard-code your
program to run '/bin/rm' explicitly, which should always work (unless
you're on Windows, of course!)

It can also be in /bin, at least.
A short idea is to just check /bin/rm and /usr/bin/rm, but I prefer 
searching thru PATH env. It only needs to do that once.


I've read quite a bit of this thread without completely understanding 
the actual problem.


Crontabs have always been good at exposing the problem of "is my program 
expecting something to be set up which isn't guaranteed?" - usually 
environment variables, of which PATH is one. "But it worked from my 
login shell".


Python's standard library has a tool for you:

shutil.which = which(cmd, mode=1, path=None)
Given a command, mode, and a PATH string, return the path which
conforms to the given mode on the PATH, or None if there is no such
file.

`mode` defaults to os.F_OK | os.X_OK. `path` defaults to the result
of os.environ.get("PATH"), or can be overridden with a custom search
path.



Since cron usually runs in a minimal environment, you should be able to 
use this to supply a decent value for PATH, look up the command you 
need, and then use that path to launch the command - or fail in some 
sort of graceful way if not found, so you can hear about it and make 
corrections.





--
https://mail.python.org/mailman/listinfo/python-list