Re: Namespaces: memory vs 'pollution'

[DL Neil]

On 22/07/19 5:30 AM, Roel Schroeven wrote:

DL Neil schreef op 21/07/2019 om 2:02:

How do you remember to from-import- 'everything' that is needed?
... > Upon closer inspection, I realised it didn't just fail; it 
failed badly!

Some silly, little, boy had imported the PythonEnvironment class but
failed to ALSO import PythonVersionError. So, the reported error was not
the expected exception!
...
Is there some 'easy way' to make sure that one doesn't just import the
desired class, but also remembers to import 'everything else' that might
be necessary. In this case, it'd be rather nice to:

from environment_module import Python*

NB this is a syntax error, and won't import both the PythonEnvironment
and PythonVersionError classes.

 > ...

What do you do to (respecting purism) ensure 'everything' (necessary) is
imported (and nothing more), preferably without relying upon (faulty, in
my case) human-memory or reading through volumes of code/documentation?


This is one of the advantages of using import instead of from-import.

import environment_module

...
try:
     
     # Just an example, since I don't know PythonEnvironment
     env = environment_module.PythonEnvironment()
     ...
except environment_module.PythonVersionError:
     # Handle the exception

In this case you have a pretty long module name (by the way, you could 
probably shorten it by removing _module; there's normally no need to 
repeat it in the name of a module that it's a module), making repeating 
it everywhere somewhat awkward. You can import the module using another 
name to work around that:


import environment_module as envmod

...
try:
     
     # Just an example, since I don't know PythonEnvironment
     env = envmod.PythonEnvironment()
     ...
except envmod.PythonVersionError:
     # Handle the exception



Greetings to Belgians, and thanks!

Yes, I like this (interestingly, I recall posing a question some months 
back, asking how many of us bother to check for import exceptions).



Long names: agreed, although I don't worry about it too much because 
competent editors 'pop-up' suggestions as one types. (incidentally, you 
are correct - I wouldn't really use that naming system, but inserted the 
word "module" in a bid to make the example illustrative)


The opposite is even worse (and sometimes working with statisticians I'm 
often given 'algebra' with one-letter 'variable names') - in the general 
case, I criticise non-obvious abbreviations in code-review. Yet...



Yesterday, I went 'full-circle' around the options for import statements 
(and some importlib facilities), looking at the effects on 'namespace 
pollution' and trying to find some sort of "wild-card" method. In the 
end, my conclusions were close-to, but not as well-developed as the above.



Current thoughts:

import environment_module as em

- so, even more of an abbreviation than suggested!?
- I rarely need to write a long list of import statements, so there 
won't be many.

- not normally using such abbreviations in my code, they will stand-out.
- considered using upper-case, eg "EM" - it is a form of constant after-all
- considered adding a single under(-line) suffix, eg "em_" (on the basis 
of "made you think"). No, don't make me think (too much)!
- so, perhaps a two-letter abbreviation with a single under(-line), eg 
"e_m", won't be confused with other naming conventions and yet 
stands-out. (sadly, that is "stands-out" to me, but 'everyone else' 
won't know its significance...)


The try...except construct is a brilliant idea because it does exactly 
what I asked - requires both the class (what I wanted to include) AND 
the custom-exception class (what it needs included).


If the class does not have any related error classes, then the 
try...except can simply check for 'exists?'...



It's a habit I'm about to adopt. Many thanks!
--
Regards =dn
--
https://mail.python.org/mailman/listinfo/python-list

Re: Proper shebang for python3

[Cameron Simpson]

On 21Jul2019 15:47, Peter J. Holzer  wrote:

On 2019-07-21 09:04:43 +1000, Cameron Simpson wrote:
I'm with Tim Daneliuk. The environment matters and should be honoured 
except

in extremely weird cases.


I don't think that all the scripts in /usr/bin are extremely weird
cases.


I think I'd better add some nuance to my stance.

I've no problem with all the scripts shipped from an OS vendor having 
#!/usr/bin/python (or whatever fixed path) in them. They have been 
released tested against the system python and should _expect_ to run 
against it. My position here is that the entire OS distribution 
constitutes a working (and via the #! controlled) environment.


But consider third party tools. Including personal tools, but basicly 
anything from _outside_ the local system in terms of authorship.


Particularly with a language like Python which is strongly backwards 
compatible, they should generally use "#!/usr/bin/env python" (or 
python2 or python3 as appropriate, when that matters) so that they can 
run in the environment they find themselves in.


1: You can can always execute a script via a specific interpreter 
explicitly.


2: If you want to test a script it is easier to provide an environment 
that exercises the script in a particular way than to hand patch the 
shebang lines on every run/reconfig.


3: If the script (per one of Chris' examples) requires a specific python 
such as 3.6, you can always go "#!usr/bin/env python3.6" in the script 
to express the target version and provide a executable "python3.6" name 
in you environment. I keep a personal ~/bin-local directory for just 
this kind of per-host stuff myself, and of course one can do the same 
thing in places like venvs or /usr/local/bin etc. And thus _still_ leave 
the script itself without a hardwired path.


[...]
If you require a specific outcoming, set a specific environment. It 
is under

your control. Control it.

For your specific example, "man youtube-dl" _is_ affected by the
environment. It honours the $MANPATH variable.


MANPATH is explicitely intended to control man.

But man doesn't fail if you set your PATH to something weird. It will
still invoke /usr/bin/groff even if that isn't in the PATH.
(I expected that there is also an environment variable to control that
but the manpage doesn't mention one).


Heh.

I wrote my own "man" yonks ago for various reasons. Guess what? I expect 
to type "man" and get mine most of the time, but type "man" when not me 
and get /usr/bin/man (absent weirdness). That applies interactively and 
also in scripts.


Same philosophy. Use the command name to express intent and the 
environment to choose the implementation of the intent. And so also in 
the shebang lines.


For Peter J. Holzer, if we must play the "I've been doing this 
forever" game: I've been sysadmining etc longer than your 25 years and disagree with

you.


That's fine. Unlike Tim I don't claim that anybody who disagrees with me
must be a newbie.


Aye; sorry for the snarkiness. Which is why I'm disagreeing on some 
things instead of asserting that you're wrong, because you're not 
"wrong".


Cheers,
Cameron Simpson 
--
https://mail.python.org/mailman/listinfo/python-list

Re: How to print out html tags excluding the attributes

[Michael F. Stemper]

On 20/07/2019 20.04, sum abiut wrote:
> I want to use regular expression to print out the HTML tags excluding the
> attributes.
> 
> for example:
> 
> import re
> html = 'Hitest test'
> tags = re.findall(r'<[^>]+>', html)
> for a in tags:
> print(a)
> 
> 
> the output is :
> 
> 
> 
> 
> 
> 
> 
> 
> But I just want the tag, not the attributes

Try this:

for a in tags:
a = re.sub( " .*>", ">", a )
print(a)

(The two statements could be combined.)

-- 
Michael F. Stemper
Galatians 3:28
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Counting Python threads vs C/C++ threads

[Peter J. Holzer]

On 2019-07-16 12:48:33 -0700, Dan Stromberg wrote:
> On Tue, Jul 16, 2019 at 11:13 AM Barry Scott  wrote:
> > Does top show the process using 100% CPU?
> >
> Nope.  CPU utilization and disk use are both low.
> 
> We've been going into top, and then hitting '1' to see things broken down
> by CPU core (there are 32 of them, probably counting hyperthreads as
> different cores), but the CPU use is in the teens or so.

If you had many CPU-bound Python threads, then with 32 cores each core
might show as 3 % busy (the sum of the threads can't use more then 100 %
because of the GIL and 100 / 32 = 3.125).

hp

-- 
   _  | Peter J. Holzer| we build much bigger, better disasters now
|_|_) || because we have much more sophisticated
| |   | h...@hjp.at | management tools.
__/   | http://www.hjp.at/ | -- Ross Anderson 


signature.asc
Description: PGP signature
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Namespaces: memory vs 'pollution'

[Roel Schroeven]

DL Neil schreef op 21/07/2019 om 2:02:

How do you remember to from-import- 'everything' that is needed?
... > Upon closer inspection, I realised it didn't just fail; it failed badly!
Some silly, little, boy had imported the PythonEnvironment class but
failed to ALSO import PythonVersionError. So, the reported error was not
the expected exception!
...
Is there some 'easy way' to make sure that one doesn't just import the
desired class, but also remembers to import 'everything else' that might
be necessary. In this case, it'd be rather nice to:

from environment_module import Python*

NB this is a syntax error, and won't import both the PythonEnvironment
and PythonVersionError classes.

> ...

What do you do to (respecting purism) ensure 'everything' (necessary) is
imported (and nothing more), preferably without relying upon (faulty, in
my case) human-memory or reading through volumes of code/documentation?


This is one of the advantages of using import instead of from-import.

import environment_module

...
try:

# Just an example, since I don't know PythonEnvironment
env = environment_module.PythonEnvironment()
...
except environment_module.PythonVersionError:
# Handle the exception

In this case you have a pretty long module name (by the way, you could 
probably shorten it by removing _module; there's normally no need to 
repeat it in the name of a module that it's a module), making repeating 
it everywhere somewhat awkward. You can import the module using another 
name to work around that:


import environment_module as envmod

...
try:

# Just an example, since I don't know PythonEnvironment
env = envmod.PythonEnvironment()
...
except envmod.PythonVersionError:
# Handle the exception


--
"Honest criticism is hard to take, particularly from a relative, a
friend, an acquaintance, or a stranger."
-- Franklin P. Jones

Roel Schroeven

--
https://mail.python.org/mailman/listinfo/python-list

Re: Proper shebang for python3

[Peter J. Holzer]

On 2019-07-21 10:26:17 -0500, Tim Daneliuk wrote:
> On 7/21/19 8:47 AM, Peter J. Holzer wrote:
> > That's fine. Unlike Tim I don't claim that anybody who disagrees with me
> > must be a newbie.
> 
> Peter, that's ad hominem and unfair.

No, it isn't. Please read the first paragraph of
https://en.wikipedia.org/wiki/Ad_hominem

> I never said anything close to that.

Well, let's see what you wrote:

| We don't need to bikeshed this.

Translation: This is not a matter of opinion or taste. There is one
objectively true answer.

| All we need is people who disagree with this view to spend a year in
| software packaging, operations, deployment and DevOps ... then get back
| to us...

Translation: People who disagree with your opinion obviously haven't
worked even a single year in this field because otherwise they would
have discovered the one objectively true answer.

| Grr..

I don't have to translate that :-)


> What I said is that if someone were to spend an extended period of time
> in devops and systems engineering at large scale, they'd quickly come to
> hate hardwired paths.

You are just stating the same point again: Everybody who has experience
in the field agrees with you, therefore anybody who disagrees can't have
experience.


> The truth is that there is no single answer to this problem until you
> hermetically seal an environment.

If you can hermetically seal the environment, the argument is moot. I
thought (and still think) we are talking about the case where you can't.
Therefore I took exception to your insistence that there is in fact one
single answer and that this single answer is one which I know from
bitter experience to be quite fragile.


> But short of that, the ability to decide to use something other than
> system-provided tools absolutely IS a requirement in systems of any
> scale.

I completely agree with that. I just don't think that using
#!/usr/bin/env is a good way to achieve that.

hp

-- 
   _  | Peter J. Holzer| we build much bigger, better disasters now
|_|_) || because we have much more sophisticated
| |   | h...@hjp.at | management tools.
__/   | http://www.hjp.at/ | -- Ross Anderson 


signature.asc
Description: PGP signature
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Proper shebang for python3

[Chris Angelico]

On Mon, Jul 22, 2019 at 1:36 AM Tim Daneliuk  wrote:
>
> On 7/21/19 8:47 AM, Peter J. Holzer wrote:
> > That's fine. Unlike Tim I don't claim that anybody who disagrees with me
> > must be a newbie.
>
> Peter, that's ad hominem and unfair.  I never said anything close to that.
> What I said is that if someone were to spend an extended period of time
> in devops and systems engineering at large scale, they'd quickly come to
> hate hardwired paths.
>
> The truth is that there is no single answer to this problem until you
> hermetically seal an environment.  Docker comes close, a freestanding
> VM does this most completely.   But short of that, the ability to decide
> to use something other than system-provided tools absolutely IS a requirement
> in systems of any scale.

Your final paragraph does not justify your preceding. Yes, you need
the ability to decide to use something other than the system-provided
tool; that's why "/usr/bin/env python3" is a valid shebang. No, that
does NOT mean that people with any decent experience will "hate
hardwired paths". The hardwired path ("#!/usr/bin/python3") has its
own value. Thanks to an absolute path in its shebang, I can run "sudo
iotop" regardless of whether I have a venv active. I cannot run "sudo
python3 `which iotop`" reliably, because that command uses the
environment to look things up.

You're absolutely right that there's no single answer to the problem.
That's why script authors have the option to use any path they like,
including /usr/bin/env.

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Proper shebang for python3

[Tim Daneliuk]

On 7/21/19 8:47 AM, Peter J. Holzer wrote:
> That's fine. Unlike Tim I don't claim that anybody who disagrees with me
> must be a newbie.

Peter, that's ad hominem and unfair.  I never said anything close to that.
What I said is that if someone were to spend an extended period of time
in devops and systems engineering at large scale, they'd quickly come to
hate hardwired paths.

The truth is that there is no single answer to this problem until you
hermetically seal an environment.  Docker comes close, a freestanding
VM does this most completely.   But short of that, the ability to decide
to use something other than system-provided tools absolutely IS a requirement
in systems of any scale.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Namespaces: memory vs 'pollution'

[Peter J. Holzer]

On 2019-07-21 12:02:27 +1200, DL Neil wrote:
> What do you do to (respecting purism) ensure 'everything' (necessary) is
> imported (and nothing more), preferably without relying upon (faulty, in my
> case) human-memory or reading through volumes of code/documentation?

I write tests (not as consistently as I would like).

I don't think there is much more you can do. The problem is that Python
doesn't have variable declarations, so there is no reliable way to
discover an undefined symbol except by executing the line where it is
used.

An IDE might help. Even though the problem is in general not solvable,
most uses of symbols fall into a few common categories which can be
discovered by static analysis. So an IDE (or even a syntax-highlighting
editor) could flag all symbols where it can't find the definition.

hp

-- 
   _  | Peter J. Holzer| we build much bigger, better disasters now
|_|_) || because we have much more sophisticated
| |   | h...@hjp.at | management tools.
__/   | http://www.hjp.at/ | -- Ross Anderson 


signature.asc
Description: PGP signature
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Proper shebang for python3

[Peter J. Holzer]

On 2019-07-21 09:04:43 +1000, Cameron Simpson wrote:
> On 21Jul2019 08:14, Chris Angelico  wrote:
> > On Sun, Jul 21, 2019 at 5:26 AM Tim Daneliuk  wrote:
> > > So, no, do NOT encode the hard location - ever.  Always use env to
> > > discover the one that the user has specified.  The only exception
> > > is /bin/sh which - for a variety of reasons - can reliably counted
> > > upon.
> > 
> > A quick grep through my $PATH shows that there are a number of
> > executable Python scripts there, including add-apt-repository,
> > calibre, some lilypond stuff, trash-can management, samba-tool, iotop,
> > and youtube-dl. If I have a venv active with, say, Python 3.9, then
> > `/usr/bin/env python` is going to point to Python 3.9. What are the
> > odds that all those scripts will work with Python 3.9 with no
> > libraries installed? Why should typing "youtube-dl B7xai5u_tnk" be
> > affected by a virtual environment, when typing "man youtube-dl"
> > wouldn't be?? Using env for everything is a terrible idea and one that
> > will basically make virtual environments useless.
> 
> I'm with Tim Daneliuk. The environment matters and should be honoured except
> in extremely weird cases.

I don't think that all the scripts in /usr/bin are extremely weird
cases.


> If you require a specific outcoming, set a specific environment. It is under
> your control. Control it.
> 
> For your specific example, "man youtube-dl" _is_ affected by the
> environment. It honours the $MANPATH variable.

MANPATH is explicitely intended to control man.

But man doesn't fail if you set your PATH to something weird. It will
still invoke /usr/bin/groff even if that isn't in the PATH.
(I expected that there is also an environment variable to control that
but the manpage doesn't mention one).


> For Peter J. Holzer, if we must play the "I've been doing this forever"
> game: I've been sysadmining etc longer than your 25 years and disagree with
> you.

That's fine. Unlike Tim I don't claim that anybody who disagrees with me
must be a newbie.

hp

-- 
   _  | Peter J. Holzer| we build much bigger, better disasters now
|_|_) || because we have much more sophisticated
| |   | h...@hjp.at | management tools.
__/   | http://www.hjp.at/ | -- Ross Anderson 


signature.asc
Description: PGP signature
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Proper shebang for python3

[Peter J. Holzer]

On 2019-07-20 15:26:46 -0500, Tim Daneliuk wrote:
> On 7/20/19 2:56 PM, Peter J. Holzer wrote:
> > On 2019-07-20 14:11:44 -0500, Tim Daneliuk wrote:
> >> So, no, do NOT encode the hard location - ever.  Always use env to
> >> discover the one that the user has specified.  The only exception is
> >> /bin/sh which - for a variety of reasons - can reliably counted upon.
> >>
> >> We don't need to bikeshed this.  All we need is people who disagree
> >> with this view to spend a year in software packaging, operations,
> >> deployment and DevOps ... then get back to us...
> >
> > After 25 years in software packaging, operations, deployment and DevOps
> > I disagree: A program should not behave differently because of a
> > different path, #!/usr/bin/env is a total no-no.
> >
> > There is a nice way to achieve this: Just use the interpreter of the
> > virtual environment in the shebang.
> > (That requires rewriting the shebang during installation, but that's a
> > minor inconvenience)
> 
> 
> And what happens with programs that have no virtenv equivalent?
> perl, go, ruby, awk, sed, grep, etc. have no simple way to
> get installed virtual short of insisting the everything live in a
> docker container or VM?

Perl is an excellent example: When you install a Perl script which has
been packaged with one of the usual tools (e.g., Makemaker or
Module::Build), the shebang is set to the interpreter used for
installation. The user of the script doesn't have to know what
environment to use. (Perl also has perlbrew which is similar to virtual
environments, but I haven't used that much).

Go is a compiled language: The executables are binaries (no shebang) and
even statically linked. Again, the user doesn't have to care about a
language environment.

awk, grep, sed, etc. are often used in shell scripts, and they were
probably the reason why I first developed a "a program's behaviour must
not depend on the environment /except as documented/" policy: Back then
in the 1990s one usually wanted to have GNU awk, sed, grep etc. in
addition to the system utilities. Having scripts randomly fail or
(worse) produce wrong results depending on the order of /usr/bin and
/usr/local/bin in the user's PATH and the exact contents of
/usr/local/bin wasn't fun (oh, and don't forget cron).


> The fact is that most large compute environments are slow to upgrade
> the OS.  That means core tools also lag considerably behind as well.
> Being able to install newer versions along side the OS' own and then
> use them by default is manifestly necessary.

I agree. But (for me at least) it is important to do that robustly, and
#!/usr/bin/env is not robust. 

> That's why users have the ability to modify $PATH to suit their own
> needs.  All /usr/bin/env does is to tell the interpreter, "honor the
> intent of the spawning shell".

You probably just phrased that badly, but as written this is completely
wrong: /usr/bin/env doesn't tell the interpreter anything. It *chooses*
the interpreter to invoke.


> If you want really big fun, try going into an older CentOS or RedHat 
> instances and, say,
> upgrading system python to python3.

Don't do that. Install python3 somewhere else, e.g. into /usr/local.

> It's super fun.  Yes, in that case, you COULD use a venv.  But there
> are tons of other tools for which this is not an option - gcc,
> autoconf, perl, go awk, sed, bash, ad infinitum, ad nauseum are
> invariably backleveled on production OS instances.  My way fixes that
> problem.

Your way only fixes your problem only if environments where your script
may be called. You may know that /usr/local/bin/foo is a python script
that requires /usr/local/python3.8/bin/python to be in the path before
any other python interpreter and you may rememmber that in every
instance where that script is called directly or indirectly. But do your
colleagues? Especially if they aren't programmers?

And what happens if /usr/local/bin/bar requires
/usr/local/python3.4/bin/python? Do you explicitely set the PATH before
invoking each script?

By putting #!/usr/local/python3.8/bin/python at the top of
/usr/local/bin/foo and #!/usr/local/python3.4/bin/python at the top of
/usr/local/bin/bar I can just invoke both scripts. Even better, my users
can invoke both scripts and they don't even have to know they are
written in Python.


> You may have 25 years at this but I have 40 - does that make me nearly twice
> as right?  Arguments from authority are silly.

No, but it invalidates your notion that anybody with a bit of experience
would obviously use #!/usr/bin/env. I do have a bit more than one year
of experience and I think #!/usr/bin/env is a terrible idea. (I use it
for throwaway scripts which need a venv, but not for anything in
production.)

hp

-- 
   _  | Peter J. Holzer| we build much bigger, better disasters now
|_|_) || because we have much more sophisticated
| |   | h...@hjp.at | management tools.
__/   | http://www.hjp.at/ | -- Ross An

Re: Proper shebang for python3

[eryk sun]

On 7/20/19, Michael Speer  wrote:
>
> You may want to use `#!/usr/bin/env python3` instead.

This is partially supported in Windows, but only if .py files are
associated with the py.exe launcher and the shebang runs "python"
instead of "python3".

py.exe supports four builtin virtual commands: "/usr/bin/env python",
"/usr/bin/python", "/usr/local/bin/python", and "python". The "python"
command can be qualified with a version specification of the form
"X[.Y][-32|-64]", such as "python3". The "/usr/bin/env python" virtual
command searches PATH, but only if the command matches "python"
exactly. If there's a version specified, or if it's not an "env"
virtual command, the launcher looks for a registered Python
installation instead of searching PATH.

For all other commands, the launcher skips a virtual Unix prefix (i.e.
"/usr/bin/env", "/usr/bin", and "/usr/local/bin"), if present, and
searches PATH. For example, given "/usr/bin/pypy", it searches PATH
for "pypy" plus the file extensions in PATHEXT (i.e. .COM, .EXE, etc).
If the command isn't found in PATH, it checks in the "[commands]"
section of "%LocalAppData%\py.ini". For example, py.ini could define
"pypy=C:\PyPy71\pypy.exe".

A virtual environment should have a "python" executable, so shebangs
that use "/usr/bin/env python" will prefer an active virtual
environment. But a lot of scripts use "python3" instead of "python".
This is a stumbling block in Windows since we don't install versioned
"pythonX[.Y].exe" binaries. Thus the "env" virtual command is special
cased to find a qualified Python version in the list of registered
Python installations instead of searching PATH. (The new store app
distribution does install versioned app links, but, even in this case,
virtual environments lack versioned binaries in the "Scripts"
directory.)

That said, some developers manually create versioned binaries as
symlinks, hardlinks, or copies. Also, maybe a future version of the
full (non-app) Python installation will do the same. Thus I think the
launcher should search PATH for all "/usr/bin/env python*" shebangs,
but without the binary-type specification (i.e. "-32" or "-64") if
present. In this case, instead of a regular search based on WINAPI
SearchPath, we'll need a custom search that tokenizes and walks PATH
and checks the binary type via GetBinaryTypeW. If no matching version
is found in PATH, the "env" search should fall back on the list of
registered versions.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Proper shebang for python3

[Brian Oney via Python-list]

On July 21, 2019 10:04:47 AM GMT+02:00, Manfred Lotz  wrote:
>On Sun, 21 Jul 2019 10:21:55 +1000
>Cameron Simpson  wrote:
>
>> On 21Jul2019 09:31, Chris Angelico  wrote:
>> >On Sun, Jul 21, 2019 at 9:15 AM Cameron Simpson 
>> >wrote: So you mean that a tool that depends on running on a
>> >consistent environment, it should use a shebang of
>> >"/usr/bin/python3.6" instead of "/usr/bin/env python3"?  
>> 
>> Jeez. No. That is the _opposite_ of what I'm saying.
>> 
>> >Because, wow, that would be exactly what is
>> >already happening on my system. Why use /usr/bin/env and then wrap
>> >something around it to force the environment, when you could just
>set
>> >a correct shebang so it properly defines its execution environment? 
>
>> 
>> Because the shebang is hardwired and inflexible.
>> 
>> Because it means hand patching (even scripted) a bazillion scripts to
>
>> that they know their physical install.
>> 
>> Because it presumes detailed hardwired knowledge of the target system
>> in a script which should work anywhere.
>> 
>> Instead a tiny _common_ shell script resembling this:
>> 
>>   #!/bin/sh
>>   # Run command in the official environment.
>>   exec env - PATH=/path/to/3.6venv/bin:/usr/sbin:/bin exec ${1+"$@"}
>> 
>> arranges things. The "env -" is aimed at "clean" daemon or install 
>> environments.  You can do subtler or less intrusive things in other 
>> settings.
>> 
>
>I took a look and found that Fedora 30 and Debian Jessie both use
>hard-wired paths for python in the rpm resp. deb packages.
>
>I'm being new to Python and I am not acquainted in any way with
>virtualenv resp. venv so cannot currently judge its pro and cons.
>
>So I will stick to:
>   #!/usr/bin/env python3 
>
>as shebang for my scripts.

I think that's a good decision. Most of the conversation applies to sysadmins 
concerned with answering your question in absolute terms. When you start 
writing scripts which are restricted to a specific environment and are intended 
to be distributed, you may revisit this thread.

Be blissful until then :). Chris et al have "fixed" things for you.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Proper shebang for python3

[Manfred Lotz]

On Sun, 21 Jul 2019 10:21:55 +1000
Cameron Simpson  wrote:

> On 21Jul2019 09:31, Chris Angelico  wrote:
> >On Sun, Jul 21, 2019 at 9:15 AM Cameron Simpson 
> >wrote: So you mean that a tool that depends on running on a
> >consistent environment, it should use a shebang of
> >"/usr/bin/python3.6" instead of "/usr/bin/env python3"?  
> 
> Jeez. No. That is the _opposite_ of what I'm saying.
> 
> >Because, wow, that would be exactly what is
> >already happening on my system. Why use /usr/bin/env and then wrap
> >something around it to force the environment, when you could just set
> >a correct shebang so it properly defines its execution environment?  
> 
> Because the shebang is hardwired and inflexible.
> 
> Because it means hand patching (even scripted) a bazillion scripts to 
> that they know their physical install.
> 
> Because it presumes detailed hardwired knowledge of the target system
> in a script which should work anywhere.
> 
> Instead a tiny _common_ shell script resembling this:
> 
>   #!/bin/sh
>   # Run command in the official environment.
>   exec env - PATH=/path/to/3.6venv/bin:/usr/sbin:/bin exec ${1+"$@"}
> 
> arranges things. The "env -" is aimed at "clean" daemon or install 
> environments.  You can do subtler or less intrusive things in other 
> settings.
> 

I took a look and found that Fedora 30 and Debian Jessie both use
hard-wired paths for python in the rpm resp. deb packages.

I'm being new to Python and I am not acquainted in any way with
virtualenv resp. venv so cannot currently judge its pro and cons.

So I will stick to:
   #!/usr/bin/env python3 

as shebang for my scripts.


-- 
Manfred


-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Proper shebang for python3

[Barry Scott]

> On 21 Jul 2019, at 08:29, Christian Gollwitzer  wrote:
> 
> Am 21.07.19 um 07:31 schrieb Tim Daneliuk:
>> On 7/20/19 6:04 PM, Chris Angelico wrote:
>>> Are you aware of every systemwide command that happens to be
>>> implemented in Python, such that you won't execute any of them while
>>> you have the venv active?
>> No, but this has never been a problem because the newer versions of
>> python tend to be pretty good - within a major release tree - of being
>> backward compatible.  Certainly, if I were running, say, RedHat 4 with
>> a very new version of python in my path first, this could theoretically
>> be an issue.  I've just not run into it.
> 
> It's not about the core language, but these system tools depend on packages, 
> sometimes specialist packages, which might not be available in your venv.

Well said.

The fedora packaging system replaces all shebang lines with the full path to 
either python2 or python3 on the system
to make sure that scripts installed in the system run as intended and do not 
break because of the users PATH.

How you think about the shebang lines depends on where the scripts will be used.

If its as an installed system component the exact path is usually the right 
thing to do.

If it you personal scripts then you may well find /usr/bin/env python3 makes 
you life
simpler.

For code I'm testing I usual do not depend no shebang lines at all. Often I'm 
testing
on many python versions. And end up with something like this to test on all 
interesting python
versions.

for PTYHON in python3.6 python3.7 python3.8
do
$PYTHON my_script.py
done

Barry
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Proper shebang for python3

[Christian Gollwitzer]

Am 21.07.19 um 07:31 schrieb Tim Daneliuk:

On 7/20/19 6:04 PM, Chris Angelico wrote:

Are you aware of every systemwide command that happens to be
implemented in Python, such that you won't execute any of them while
you have the venv active?


No, but this has never been a problem because the newer versions of
python tend to be pretty good - within a major release tree - of being
backward compatible.  Certainly, if I were running, say, RedHat 4 with
a very new version of python in my path first, this could theoretically
be an issue.  I've just not run into it.



It's not about the core language, but these system tools depend on 
packages, sometimes specialist packages, which might not be available in 
your venv.


Christian

--
https://mail.python.org/mailman/listinfo/python-list