[issue20658] os.environ.clear() fails with empty keys (posix.unsetenv)

[STINNER Victor]

STINNER Victor  added the comment:

POSIX says:

(1) "The setenv() function shall fail if: [EINVAL] The name argument is a null 
pointer, points to an empty string, or points to a string containing an '=' 
character."

https://pubs.opengroup.org/onlinepubs/009604499/functions/setenv.html

(2) "The unsetenv() function shall fail if: [EINVAL] The name argument is a 
null pointer, points to an empty string, or points to a string containing an 
'=' character."

https://pubs.opengroup.org/onlinepubs/009695399/functions/unsetenv.html

But POSIX doesn't specify which keys are invalid for putenv. It only specifies 
a single error:

(3) "The putenv() function may fail if: [ENOMEM] Insufficient memory was 
available."

https://pubs.opengroup.org/onlinepubs/009696899/functions/putenv.html

The GNU libc respects POSIX.

IMO setenv() and unsetenv() are fine. The problem comes from putenv() which is 
under specified and allows keys which are causing a lot of subtle issues, not 
only in Python.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue20658] os.environ.clear() fails with empty keys (posix.unsetenv)

[Serhiy Storchaka]

Serhiy Storchaka  added the comment:

> For the very specific case of os.environ.clear(), the C function clearenv() 
> could be used if available.

It is bad in any way. The supposed example of using clear():

old_environ = dict(os.environ)
os.environ.clear()
os.environ.update(new_environ)
...
os.environ.clear()
os.environ.update(old_environ)

Even if clear() passed, it will fail on attempt to restore the old environment, 
with the empty key.

You can have the same issue in a corresponding C code. I think the only way is 
to report this as a bug in glibc and wait on their reaction.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue20658] os.environ.clear() fails with empty keys (posix.unsetenv)

[Terry J. Reedy]

Change by Terry J. Reedy :


--
versions: +Python 3.11 -Python 2.7, Python 3.3, Python 3.4

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue20658] os.environ.clear() fails with empty keys (posix.unsetenv)

[STINNER Victor]

STINNER Victor  added the comment:

For the very specific case of os.environ.clear(), the C function clearenv() 
could be used if available.

While clearenv() is available in the glibc, it's not a POSIX function.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue20658] os.environ.clear() fails with empty keys (posix.unsetenv)

[STINNER Victor]

STINNER Victor  added the comment:

Attached set_unset_env.c program calls putenv("=hello world") and then 
unsetenv("").

On my Fedora 34 with glibc-2.33-20.fc34.x86_64, putenv() succeed, but 
unsetenv() fails.
---
$ gcc set_unset_env.c -g -o set_unset_env && ./set_unset_env
putenv("=hello world") -> hello world
ERROR: unsetenv("") failed: [error 22] Invalid argument
---

By the way, getenv() fails to find an environment variable if its name is 
empty: I reimplemented getenv() using the 'environ' variable for my test.

--
Added file: https://bugs.python.org/file50245/set_unset_env.c

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue20658] os.environ.clear() fails with empty keys (posix.unsetenv)

[Irit Katriel]

Irit Katriel  added the comment:

I see, so intercepting the assignment is not enough. Reopening.

--
resolution: out of date -> 
status: closed -> open

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue20658] os.environ.clear() fails with empty keys (posix.unsetenv)

[STINNER Victor]

STINNER Victor  added the comment:

By the way:
---
The env command from GNU coreutils supports setting the environment variable 
with an empty name but not unsetting it. That's a bug.

$ env '=wibble' env |grep wibble 
=wibble
$ env '=wibble' env -u '' env
env: cannot unset `': Invalid argument
---

https://unix.stackexchange.com/questions/178522/unsetting-environment-variable-with-an-empty-name

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue20658] os.environ.clear() fails with empty keys (posix.unsetenv)

[STINNER Victor]

STINNER Victor  added the comment:

The following command still fails on the Python main branch on Linux:
---
$ env -i =value ./python -c 'import pprint, os; pprint.pprint(os.environ); del 
os.environ[""]'
environ({'': 'value', 'LC_CTYPE': 'C.UTF-8'})
Traceback (most recent call last):
  File "", line 1, in 
  File "/home/vstinner/python/main/Lib/os.py", line 689, in __delitem__
unsetenv(encodedkey)

OSError: [Errno 22] Invalid argument
---

'del os.environ[""]' calls unsetenv("") which fails with EINVAL.

Python is a thin wrapper to C functions setenv() and unsetenv(), and raises an 
exception when a C function fails. It works as expected.

Python exposes the variable with an empty name which is found in the 
environment variables: again, it works as expected.

I don't see how Python could do better, since the glibc unsetenv() fails with 
EINVAL if the string is empty. It is even a documented behaviour, see the 
unsetenv() manual page:
---
ERRORS

   EINVAL name is NULL, points to a string of length 0, or contains an '=' 
character.
---

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue20658] os.environ.clear() fails with empty keys (posix.unsetenv)

[Irit Katriel]

Change by Irit Katriel :


--
stage: test needed -> resolved
status: pending -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue20658] os.environ.clear() fails with empty keys (posix.unsetenv)

[Irit Katriel]

Irit Katriel  added the comment:

They keys are checked now, so I think this is resolved.  (Tested on windows and 
Mac).

>>> os.environ[''] = 'X'
Traceback (most recent call last):
  File "", line 1, in 
  File "/Users/iritkatriel/src/cpython/Lib/os.py", line 684, in __setitem__
putenv(key, value)
^^
OSError: [Errno 22] Invalid argument

>>> os.environ['a=b'] = 'c'
Traceback (most recent call last):
  File "", line 1, in 
  File "/Users/iritkatriel/src/cpython/Lib/os.py", line 684, in __setitem__
putenv(key, value)
^^
ValueError: illegal environment variable name

--
nosy: +iritkatriel
resolution:  -> out of date
status: open -> pending

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue20658] os.environ.clear() fails with empty keys (posix.unsetenv)

[Peter Funk]

Changes by Peter Funk :


--
nosy: +pefu

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue20658] os.environ.clear() fails with empty keys (posix.unsetenv)

[akira]

akira added the comment:

Related: issue4926 putenv() accepts names containing '=', return value of 
unsetenv() not checked 

`os.environ.clear()` also fails if the environment contains names with `=`:

   import os
   os.environ['a=b'] = 'c'
   os.environ.clear()
  Traceback (most recent call last):
File stdin, line 1, in module
File /.../lib/python3.4/_collections_abc.py, line 558, in clear
  self.popitem()
File /.../lib/python3.4/_collections_abc.py, line 551, in popitem
  del self[key]
File /.../lib/python3.4/os.py, line 662, in __delitem__
  self.unsetenv(encodedkey)
  OSError: [Errno 22] Invalid argument

--
nosy: +akira

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue20658
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue20658] os.environ.clear() fails with empty keys (posix.unsetenv)

[daniel hahler]

daniel hahler added the comment:

Please note that I have noticed this not because of setting it via 
`os.environ`, but because a program using `os.environ.clear()` crashed:
https://bugs.launchpad.net/ubuntu/+source/apport/+bug/1281086

I cannot say how this unusual entry was added to the environment, but it showed 
up in `env` and therefore it is possible somehow.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue20658
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue20658] os.environ.clear() fails with empty keys (posix.unsetenv)

[STINNER Victor]

STINNER Victor added the comment:

Please note that I have noticed this not because of setting it via 
`os.environ`, but because a program using `os.environ.clear()` crashed:
https://bugs.launchpad.net/ubuntu/+source/apport/+bug/1281086;

The bug is not os.environ.clear(), but that os.environ contains an empty string.

It would help to know if the key was set manually by apport, or if it comes 
from the real environment. The environment looks correct:
https://launchpadlibrarian.net/166517334/ProcEnviron.txt

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue20658
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue20658] os.environ.clear() fails with empty keys (posix.unsetenv)

[daniel hahler]

daniel hahler added the comment:

 It would help to know if the key was set manually by apport, or if it comes 
 from the real environment. The environment looks correct:

It comes from the real environment.

I wanted to use apport, but could not from the current shell, because of this 
bug.

I had then looked at the output of `env` in this shell, and it contained just 
= (IIRC, but an empty name/key for sure). This was in a tmux shell/pane, 
which I have closed already.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue20658
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue20658] os.environ.clear() fails with empty keys (posix.unsetenv)

[STINNER Victor]

STINNER Victor added the comment:

 It comes from the real environment.

Oh. It looks possible to define an environment variable with an empty key, but 
not to remove it:

$ env -i =value python -c 'import pprint, os; pprint.pprint(os.environ); del 
os.environ[]'
environ({'': 'value'})
Traceback (most recent call last):
  File string, line 1, in module
  File Lib/os.py, line 662, in __delitem__
self.unsetenv(encodedkey)
OSError: [Errno 22] Invalid argument

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue20658
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue20658] os.environ.clear() fails with empty keys (posix.unsetenv)

[STINNER Victor]

STINNER Victor added the comment:

By the way, Python allows also to set an environment with a name containing 
=, whereas getenv/setenv/unsetenv doesn't.

$ env -i python -c 'import pprint, posix, os; os.environ[a=]=1; 
print(os.environ); posix.unsetenv(a=)'

environ({'a=': '1'})
Traceback (most recent call last):
  File string, line 1, in module
OSError: [Errno 22] Invalid argument

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue20658
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue20658] os.environ.clear() fails with empty keys (posix.unsetenv)

[Serhiy Storchaka]

Changes by Serhiy Storchaka storch...@gmail.com:


--
Removed message: http://bugs.python.org/msg211621

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue20658
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue20658] os.environ.clear() fails with empty keys (posix.unsetenv)

[Serhiy Storchaka]

Serhiy Storchaka added the comment:

Are there supported platforms on which setenv() is not supported? When we will 
migrate to setenv(), an argument check will be not needed.

In any case I think we should wrap unsetenv() in os.environ.clear() so that it 
should try to remove all environment variables even if some calls of unsetenv() 
fails.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue20658
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue20658] os.environ.clear() fails with empty keys (posix.unsetenv)

[Serhiy Storchaka]

Serhiy Storchaka added the comment:

Are there supported platforms on which setenv() is not supported.

--
nosy: +serhiy.storchaka

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue20658
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue20658] os.environ.clear() fails with empty keys (posix.unsetenv)

[STINNER Victor]

STINNER Victor added the comment:

In fact, there is also a clearenv() function which could be used by 
os.environ.clear().

The clearenv() function clears the environment of all name-value pairs and 
sets the value of the external variable environ to NULL.

It looks like supported names depends a lot on the platform and platform 
version. Extract of Linux manual pages:

setenv:
---
BUGS:

POSIX.1-2001  specifies  that  if  name  contains an '=' character, then 
setenv() should fail with the error EINVAL; however, versions of glibc before 
2.3.4 allowed an '=' sign in name.
---

clearenv:
---
CONFORMING TO

Various  UNIX variants (DG/UX, HP-UX, QNX, ...).  POSIX.9 (bindings for 
FORTRAN77).  POSIX.1-1996 did not accept clearenv() and putenv(3), but changed 
its mind and scheduled these functions for some later issue of this standard 
(cf. B.4.6.1).  However, POSIX.1-2001 adds only putenv(3),  and  rejected 
clearenv().
---

 In any case I think we should wrap unsetenv() in os.environ.clear() so that 
 it should try to remove all environment variables even if some calls of 
 unsetenv() fails.

os.environ.clear() may tries to remove as much keys as possible, but keep keys 
for which unsetenv raised an error and raise a global error in this case.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue20658
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue20658] os.environ.clear() fails with empty keys (posix.unsetenv)

[Arfrever Frehtes Taifersar Arahesis]

Changes by Arfrever Frehtes Taifersar Arahesis arfrever@gmail.com:


--
nosy: +Arfrever

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue20658
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue20658] os.environ.clear() fails with empty keys (posix.unsetenv)

[STINNER Victor]

STINNER Victor added the comment:

putenv(=value) does nothing: it doesn't create a variable with an empty name. 
You can test with the attach test_empty_env_var.py script (written for Linux).

Attached reject_empty_env_var.patch patch modifies posix.putenv() to raise a 
ValueError if the name is empty.

The patch is incomplete, the Windows part should also be modified. But I didn't 
test yet if Windows supports variables with empty name.

--
nosy: +haypo
Added file: http://bugs.python.org/file34132/test_empty_env_var.py

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue20658
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue20658] os.environ.clear() fails with empty keys (posix.unsetenv)

[STINNER Victor]

Changes by STINNER Victor victor.stin...@gmail.com:


--
keywords: +patch
Added file: http://bugs.python.org/file34133/reject_empty_env_var.patch

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue20658
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue20658] os.environ.clear() fails with empty keys (posix.unsetenv)

[STINNER Victor]

STINNER Victor added the comment:

The workaround of this bug is to avoid os.environ['']=value.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue20658
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue20658] os.environ.clear() fails with empty keys (posix.unsetenv)

[daniel hahler]

New submission from daniel hahler:

posix.unsetenv fails to clear the environment if there's an entry with an empty 
key.

TEST CASE:

Python 2.7.6 (default, Jan  6 2014, 17:05:19) 
[GCC 4.8.1] on linux2
Type help, copyright, credits or license for more information.
 import os
 os.environ['']='foo'
 os.environ.clear()
Traceback (most recent call last):
  File stdin, line 1, in module
  File /path/to/python/2.7.6/lib/python2.7/os.py, line 499, in clear
unsetenv(key)
OSError: [Errno 22] Invalid argument

I believe that os.environ.clear() via unsetenv should handle empty keys.

--
components: Interpreter Core
messages: 211415
nosy: blueyed
priority: normal
severity: normal
status: open
title: os.environ.clear() fails with empty keys (posix.unsetenv)
type: crash
versions: Python 2.7, Python 3.3

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue20658
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue20658] os.environ.clear() fails with empty keys (posix.unsetenv)

[Ned Deily]

Ned Deily added the comment:

According to the Open Group Base Specification (Issue 7 2013 Issue): 

The setenv() function shall fail if:

[EINVAL]
The envname argument points to an empty string or points to a string containing 
an '=' character.

So it seems to me that the issue here is that os.environ[''] attempts to allow 
you to create a environment variable with a null key.

--
nosy: +ned.deily
stage:  - test needed
type: crash - 
versions: +Python 3.4

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue20658
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue20658] os.environ.clear() fails with empty keys (posix.unsetenv)

[Ned Deily]

Ned Deily added the comment:

OTOH, the specification for putenv, which is what is actually used by 
posixmodule.c, does not contain that requirement.

http://pubs.opengroup.org/onlinepubs/9699919799/functions/putenv.html
http://pubs.opengroup.org/onlinepubs/9699919799/functions/setenv.html

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue20658
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com