[issue1159] os.getenv() not updated after external module uses C putenv()
Changes by Martin Panter vadmium...@gmail.com: -- nosy: +vadmium ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue1159 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1159] os.getenv() not updated after external module uses C putenv()
Changes by Mark Lawrence breamore...@yahoo.co.uk: -- nosy: -BreamoreBoy ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue1159 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1159] os.getenv() not updated after external module uses C putenv()
Changes by Arfrever Frehtes Taifersar Arahesis arfrever@gmail.com: -- nosy: +Arfrever ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue1159 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1159] os.getenv() not updated after external module uses C putenv()
Amaury Forgeot d'Arc added the comment: FYI, PyPy recently got bitten by this: https://bugs.pypy.org/issue1518 A posix.libc_getenv() function could be a solution. -- nosy: +amaury.forgeotdarc ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue1159 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1159] os.getenv() not updated after external module uses C putenv()
Terry J. Reedy added the comment: The problem is the 'putenv' and 'getenv' appear to be parallel, and seem to be documented as being parallel Set the environment variable named key to the string value. Return the value of the environment variable key if it exists, but they are not. Getenv actually looks up the key in the internal os.environ copy while putenv puts to the actual external environment. This tripped-up someone today on python-list, who did putenv(key, val); getenv(key) and wondered why putenv seemed to have no effect. I think the solution for this should be to document the asymmetry by adding ' in os.environ' after 'key' in the getenv doc. -- putenv should not also update os.environ because one can already do that with os.environ[key] = value (as recommended) and because the latter uses putenv *if available* to also update the external environment *if possible*. Note that at the time this system was designed, not all systems supported putenv (and perhaps not 'true' getenv either). getenv(key) is not the same as os.environ[key] because the former has an optional 'default' parameter that defaults to None. So aside from back-compatibility, I do not think the behavior of existing code should change. A new parameter might be possible. To implement it, one would have to augment the underlying, undocumented, C-coded (for CPython) os-specific module -- posix, nt, os2, ce -- to define a new os-specific getenv function that parallels the os-specific putenv function. Adding os.environ.update (or .synchronize) to resynchronize os.environ with the external environment would also require a new os-specific function. Currently, the original os.environ is imported as a *data* attribute of the os-specific module. However, neither of these changes are needed for python code that used os.environ as intended. I don't think we should necessarily cater to badly written C libraries that modify the enviroment in a way that cannot be easily intercepted or controlled. So after making a doc change, I would be inclined to close this pending a python-ideas discussion that supported a new feature. -- assignee: - docs@python components: +Documentation nosy: +docs@python, terry.reedy versions: +Python 2.7, Python 3.3, Python 3.4 -Python 2.6 ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue1159 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1159] os.getenv() not updated after external module uses C putenv()
Mark Lawrence breamore...@yahoo.co.uk added the comment: Can someone please comment on whether or not this issue is still valid. -- nosy: +BreamoreBoy resolution: postponed - ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue1159 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1159] os.getenv() not updated after external module uses C putenv()
Jason R. Coombs jar...@jaraco.com added the comment: As a workaround, could you use ctypes to pull the environment back into the python context? For example: http://paste.turbogears.org/paste/34734 -- nosy: +jaraco ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue1159 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1159] os.getenv() not updated after external module uses C putenv()
Changes by Sean Reifschneider: -- priority: - normal resolution: - postponed __ Tracker [EMAIL PROTECTED] http://bugs.python.org/issue1159 __ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1159] os.getenv() not updated after external module uses C putenv()
Raghuram Devarakonda added the comment: Robert Ancell added the comment: draghuram, unfortunately while os.putenv() can be fixed to be symmetrical any putenv call from a C module cannot, for example: Hi Robert, I understood the problem from your very first report. I brought up putenv() not updating os.environ only because you mentioned in the original report that it does. All you need is a way to get the current value of an environment variable. The situation is a bit complicated since a cache is in the picture (os.environ). I would suggest that you bring up the topic for discussion on python-dev and once a consensus is reached, some one can come up with a patch (I can try). __ Tracker [EMAIL PROTECTED] http://bugs.python.org/issue1159 __ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1159] os.getenv() not updated after external module uses C putenv()
New submission from Robert Ancell: The Python os.getenv() function accesses an Python dictionary which is mirroring the process environment. This dictionary is populated when the interpreter starts and updated when os.environ.__setitem__() or os.putenv() are called. However if the python program imports an extension module that uses the system putenv() then the changes cannot be accessed using the Python standard library. This has been a problem for us as we have created Python bindings to an existing C based library that modifies the environment dynamically (not the best design decision...). The workaround we are using is to create our own wrapper to the system (Solaris/Linux) getenv(). A potential solution could be to make environ a class where os.environ.__setitem__() calls putenv(), os.environ.__getitem__() calls getenv() and os.environ.keys()/items()/iter*() uses **environ (or other appropriate system call). This does however have undefined issues on how the environment behaves on various systems (memory leaks etc). -- components: Library (Lib) messages: 55881 nosy: robert.ancell severity: normal status: open title: os.getenv() not updated after external module uses C putenv() type: behavior versions: Python 2.6 __ Tracker [EMAIL PROTECTED] http://bugs.python.org/issue1159 __ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1159] os.getenv() not updated after external module uses C putenv()
Martin v. Löwis added the comment: I can't see a bug here. If you want the current C library value of the environment variable, just use os.getenv, not os.environ. -- nosy: +loewis __ Tracker [EMAIL PROTECTED] http://bugs.python.org/issue1159 __ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1159] os.getenv() not updated after external module uses C putenv()
Martin v. Löwis added the comment: Ignore my comment - I see now that you are talking about os.getenv. __ Tracker [EMAIL PROTECTED] http://bugs.python.org/issue1159 __ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1159] os.getenv() not updated after external module uses C putenv()
Raghuram Devarakonda added the comment: On 9/13/07, Robert Ancell [EMAIL PROTECTED] wrote: The Python os.getenv() function accesses an Python dictionary which is mirroring the process environment. This dictionary is populated when the interpreter starts and updated when os.environ.__setitem__() or os.putenv() are called. However if the python program imports an As per the document and my simple test (on Linux), os.putenv() does not update os.environ. I think, it should update it. __ Tracker [EMAIL PROTECTED] http://bugs.python.org/issue1159 __ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1159] os.getenv() not updated after external module uses C putenv()
Stefan Sonnenberg-Carstens added the comment: As per the document and my simple test (on Linux), os.putenv() does not update os.environ. I think, it should update it. What would be the benefit ? -- nosy: +pythonmeister __ Tracker [EMAIL PROTECTED] http://bugs.python.org/issue1159 __ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1159] os.getenv() not updated after external module uses C putenv()
Raghuram Devarakonda added the comment: Stefan Sonnenberg-Carstens added the comment: As per the document and my simple test (on Linux), os.putenv() does not update os.environ. I think, it should update it. What would be the benefit ? Symmetrical behaviour. When os.getenv() returns the value from os.environ, one would expect, os.putenv to store the value there (At least, I did). On the other hand, it is also ok for both os.getenv and os.putenv get/set the environment directly instead of going through os.environ. I am sure there was some reason for the current behaviour of os.putenv. Perhaps, because putenv is supposedly not available on all platforms? Any way, I think the OP was asking to always get the value dynamically when ever os.environ['VAR'] or os.getenv['VAR'] is done. I don't see any problem with that approach. How ever, if it is considered backwards incompatible, I guess an optional parameter can be added to os.getenv. __ Tracker [EMAIL PROTECTED] http://bugs.python.org/issue1159 __ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1159] os.getenv() not updated after external module uses C putenv()
Stefan Sonnenberg-Carstens added the comment: I'd like to see perl/ruby behaviour: an dict (os.environ), nothing more (perl %ENV,ruby $ENV). Get rid of setenv/putenv at all. 3.0a1 has even more: There is os.environ (a dict), os.[put|get]env() and os.environ.putenv() __ Tracker [EMAIL PROTECTED] http://bugs.python.org/issue1159 __ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1159] os.getenv() not updated after external module uses C putenv()
Robert Ancell added the comment:
draghuram, unfortunately while os.putenv() can be fixed to be
symmetrical any putenv call from a C module cannot, for example:
If you make an extension:
#include stdlib.h
PyObject *putenvC(PyObject *module, PyObject *args)
{
int result;
if (!PyArg_ParseTuple(args, ))
return 0;
result = putenv(FOO=BAR);
return Py_BuildValue(i, result);
}
The following behaviour will occur:
$ python
import putenv
putenv.putenvC()
assert(os.getenv('FOO') == None)
assert(os.environ.get('FOO') == None)
This is because the os.environ dictionary will never be updated:
From Lib/os.py:
def getenv(key, default=None):
Get an environment variable, return None if it doesn't exist.
The optional second argument can specify an alternate default.
return environ.get(key, default)
__
Tracker [EMAIL PROTECTED]
http://bugs.python.org/issue1159
__
___
Python-bugs-list mailing list
Unsubscribe:
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1159] os.getenv() not updated after external module uses C putenv()
Robert Ancell added the comment:
I've attached proof-of-concept showing how os.environ would ideally
work. It'll only work in Posix, etc etc.
Reading into it more there are a lot of general issues with environments
and memory allocation which is why I suspect Python doesn't use
putenv... See putenv(3) for details.
Compile with:
gcc -shared -o environmodule.so -g -Wall -I /usr/include/python2.5
environmodule.c
__
Tracker [EMAIL PROTECTED]
http://bugs.python.org/issue1159
__#include Python.h
#include stdlib.h
static PyObject *environ_getenv(PyObject *self, PyObject *args)
{
char *name, *result;
if(!PyArg_ParseTuple(args, s, name))
return NULL;
result = getenv(name);
if(result == NULL)
Py_RETURN_NONE;
else
return PyString_FromString(result);
}
static PyObject *environ_putenv(PyObject *self, PyObject *args)
{
int result;
char *name, *value, *buffer;
if (!PyArg_ParseTuple(args, ss, name, value))
return NULL;
buffer = malloc(sizeof(char) * (strlen(name) + 1 + strlen(value)));
sprintf(buffer, %s=%s, name, value);
result = putenv(buffer);
return PyInt_FromLong((long)result);
}
/* Environment iterator */
typedef struct {
PyObject_HEAD
int offset;
} environ_EnvironIterObject;
static PyTypeObject environ_EnvironIterType = {
PyObject_HEAD_INIT(NULL)
0, /*ob_size*/
environ.EnvironIter, /*tp_name*/
sizeof(environ_EnvironIterObject) /*tp_basicsize*/
};
static PyObject *Environ_iternext(PyObject *object)
{
environ_EnvironIterObject *o = (environ_EnvironIterObject *)object;
char *value;
int i;
/* Must count from the start in case the environment has shrunk since the last iteration */
for(i = 0; i o-offset environ[i]; i++)
value = environ[i];
o-offset++;
if(value == NULL)
return NULL;
else
return PyString_FromString(value);
}
/* Environment object */
typedef struct {
PyObject_HEAD
} environ_EnvironObject;
static PyTypeObject environ_EnvironType = {
PyObject_HEAD_INIT(NULL)
0, /*ob_size*/
environ.Environ, /*tp_name*/
sizeof(environ_EnvironObject) /*tp_basicsize*/
};
static PyObject *Environ_getitem(PyObject *self, PyObject *arg)
{
char *result, *name;
name = PyString_AsString(arg);
result = getenv(name);
if(result == NULL)
Py_RETURN_NONE;
else
return PyString_FromString(result); /* FIXME: Should split into a 2-tuple */
}
static int Environ_setitem(PyObject *self, PyObject *key, PyObject *v)
{
int result;
char *name, *value, *buffer;
name = PyString_AsString(key);
value = PyString_AsString(v);
/* FIXME: free() the current value if we set it (i.e. getenv() == malloced_value + len(name) + len('=')) */
/* FIXME: We need to remember this malloc so we can free() it later */
buffer = malloc(sizeof(char) * (strlen(name) + 1 + strlen(value)));
sprintf(buffer, %s=%s, name, value);
result = putenv(buffer);
return 0;
}
static PyMethodDef Environ_methods[] = {
{NULL}
};
static Py_ssize_t Environ_length(PyObject *mp)
{
int count;
for(count = 0; environ[count] != NULL; count++);
return count;
}
static PyMappingMethods environ_as_mapping = {
(lenfunc)Environ_length, /*mp_length*/
(binaryfunc)Environ_getitem, /*mp_subscript*/
(objobjargproc)Environ_setitem, /*mp_ass_subscript*/
};
static PyObject *Environ_iter(PyObject *object)
{
/* NOTE: If the environment is changed during iteration it will be safe but unpredictable */
return environ_EnvironIterType.tp_alloc(environ_EnvironIterType, 0);
}
/* Module initialisation */
static PyMethodDef environ_methods[] = {
{getenv, environ_getenv, METH_VARARGS, },
{putenv, environ_putenv, METH_VARARGS, },
{NULL}
};
PyMODINIT_FUNC initenviron(void)
{
PyObject *m, *environ;
environ_EnvironType.tp_flags = Py_TPFLAGS_DEFAULT;
environ_EnvironType.tp_doc = Environment object;
environ_EnvironType.tp_new = PyType_GenericNew;
environ_EnvironType.tp_iter = Environ_iter;
environ_EnvironType.tp_methods = Environ_methods;
if(PyType_Ready(environ_EnvironType) 0)
return;
environ_EnvironIterType.tp_flags = Py_TPFLAGS_DEFAULT;
environ_EnvironIterType.tp_doc = Environment iterator;
environ_EnvironIterType.tp_new = PyType_GenericNew;
environ_EnvironIterType.tp_iternext = Environ_iternext;
environ_EnvironIterType.tp_as_mapping = environ_as_mapping;
if(PyType_Ready(environ_EnvironIterType) 0)
return;
m = Py_InitModule3(environ,
environ_methods,
Module to test potential os.environ replacement);
environ =
