Re: PyEval_GetLocals and unreferenced variables
On Wed, Dec 3, 2014 at 5:28 AM, Gregory Ewing greg.ew...@canterbury.ac.nz wrote: Kasper Peeters wrote: That may have been the design plan, but in Python 2.7.6, I definitely am able to inject locals via PyEval_GetLocals() and have them be visible both from the C and Python side; What seems to be happening is that the dict created by PyEval_GetLocals() is kept around, so you can change it and have the changes be visible through locals() in Python. That the dict is cached is an implementation detail. It's not advisable to rely upon this, and as stated in the Python docs, the locals dict should not be modified. -- https://mail.python.org/mailman/listinfo/python-list
Re: PyEval_GetLocals and unreferenced variables
I'm not sure how you think you're adding a local from C code. If you're using PyEval_GetLocals(), that only gives you a dict containing a *copy* of the locals; modifying that dict doesn't change the locals in the function's frame. That may have been the design plan, but in Python 2.7.6, I definitely am able to inject locals via PyEval_GetLocals() and have them be visible both from the C and Python side; see also http://stackoverflow.com/questions/22276502/create-python-object-in-local-scope-from-within-c If the bytecode of the nested function doesn't reference a given variable in the outer function, it doesn't get passed in. Ok, that's good to know because it rules out doing this without having an explicit reference to the variable in the inner scope. Thanks. Cheers, Kasper -- https://mail.python.org/mailman/listinfo/python-list
Re: PyEval_GetLocals and unreferenced variables
Kasper Peeters wrote: That may have been the design plan, but in Python 2.7.6, I definitely am able to inject locals via PyEval_GetLocals() and have them be visible both from the C and Python side; What seems to be happening is that the dict created by PyEval_GetLocals() is kept around, so you can change it and have the changes be visible through locals() in Python. However, that doesn't create a local *name* that's visible from Python. Or if the local name exists, changes made through locals() aren't reflected in the value of the local name. Existing local name is not changed: def f(): ... a = 42 ... locals()['a'] = 17 ... print a ... f() 42 Can't create a local name: def g(): ... locals()['a'] = 17 ... print a ... g() Traceback (most recent call last): File stdin, line 1, in module File stdin, line 3, in g NameError: global name 'a' is not defined Changes to locals() persist: def h(): ... locals()['a'] = 17 ... print locals()['a'] ... h() 17 -- Greg -- https://mail.python.org/mailman/listinfo/python-list
Re: PyEval_GetLocals and unreferenced variables
def fun(): cfun_that_creates_q_in_local_scope() def fun2(): cfun_that_wants_to_see_if_q_is_available() So the Python side actually doesn't see 'q' directly at all. I think you will need to elaborate. Ok, here goes (and thanks for listening). The behaviour of the C side is determined by certain settings/preferences. I want these settings to respect the Python scope. I could in principle decide to make these settings a proper Python object, and ask the user to create one and pass it along at every C-function call. Something like def fun(): settings = Cpp_Options() settings.set_some_flag() cfun(settings, ...) def fun2(): settings = Cpp_Options(settings) settings.set_some_other_flag() cfun(settings, ...) Then Python would automatically take care of the scope of 'settings'. However, this is difficult for the user to keep track of. So my idea was to allow for def fun(): set_some_flag() cfun(...) def fun2(): set_some_other_flag() cfun(...) I let the C side create a Cpp_Options object on the locals stack behind the scenes, and the 'cfun()' function takes it from there directly, without requiring the user to pass it. Much easier for the user. This works, but the problem is that the C side does not see the settings that were created in fun() when it gets called from fun2(). In fun2(), the locals do not contain objects constructed earlier in fun(), unless the Python side explicitly refers to them. So adding a line settings.do_something() inside fun2() would work and forces Python to pull the settings object created in fun() into scope, but that sort of defeats the purpose. Hence my question: how can I ask, purely on the C side, for Python to pull objects into the local frame? Final note: I am actually trying to make this look as close as possible to an older custom-built language, which didn't require passing the settings object either, so it's kinda important to make the user feel 'at home'. Hope this makes it more clear, thanks for your patience. Cheers, Kasper -- https://mail.python.org/mailman/listinfo/python-list
Re: PyEval_GetLocals and unreferenced variables
On 12/2/14 4:35 AM, Kasper Peeters wrote: def fun(): cfun_that_creates_q_in_local_scope() def fun2(): cfun_that_wants_to_see_if_q_is_available() So the Python side actually doesn't see 'q' directly at all. I think you will need to elaborate. Ok, here goes (and thanks for listening). The behaviour of the C side is determined by certain settings/preferences. I want these settings to respect the Python scope. I could in principle decide to make these settings a proper Python object, and ask the user to create one and pass it along at every C-function call. Something like def fun(): settings = Cpp_Options() settings.set_some_flag() cfun(settings, ...) def fun2(): settings = Cpp_Options(settings) settings.set_some_other_flag() cfun(settings, ...) Then Python would automatically take care of the scope of 'settings'. However, this is difficult for the user to keep track of. So my idea was to allow for def fun(): set_some_flag() cfun(...) def fun2(): set_some_other_flag() cfun(...) I let the C side create a Cpp_Options object on the locals stack behind the scenes, and the 'cfun()' function takes it from there directly, without requiring the user to pass it. Much easier for the user. This works, but the problem is that the C side does not see the settings that were created in fun() when it gets called from fun2(). In fun2(), the locals do not contain objects constructed earlier in fun(), unless the Python side explicitly refers to them. So adding a line settings.do_something() inside fun2() would work and forces Python to pull the settings object created in fun() into scope, but that sort of defeats the purpose. I would use thread locals for this: https://docs.python.org/2/library/threading.html#threading.local They act like global variables, in that they are available implicitly without being passed around, but like locals in that two separate threads will have different values. -- Ned Batchelder, http://nedbatchelder.com -- https://mail.python.org/mailman/listinfo/python-list
Re: PyEval_GetLocals and unreferenced variables
Kasper Peeters wrote: I could in principle decide to make these settings a proper Python object, and ask the user to create one and pass it along at every C-function call. I would make the C functions methods of the object holding the settings. Your nested function example would then look something like this: def fun(): cpp = Cpp() cpp.set_some_flag() cpp.cfun(...) def fun2(): cpp2 = Cpp(cpp) cpp2.set_some_other_flag() cpp2.fun(...) Hence my question: how can I ask, purely on the C side, for Python to pull objects into the local frame? You can't. Firstly, it's not possible to add locals to a frame that didn't exist when the bytecode was compiled. The compiler figures out how many locals there are, allocates them in an array, and generates bytecodes that reference them by their array index. I'm not sure how you think you're adding a local from C code. If you're using PyEval_GetLocals(), that only gives you a dict containing a *copy* of the locals; modifying that dict doesn't change the locals in the function's frame. Secondly, the way references to outer scopes is implemented is by effectively passing the referenced variables as implicit parameters. If the bytecode of the nested function doesn't reference a given variable in the outer function, it doesn't get passed in. Again, this is determined when the bytecode is compiled and can't be changed at run time. Final note: I am actually trying to make this look as close as possible to an older custom-built language, which didn't require passing the settings object either, so it's kinda important to make the user feel 'at home'. I doubt it's worth the extreme level of hackery that would be required to make it work, though. I think it would be better to provide a clean, pythonic API that makes the scope of the options explicit, rather than try to mimic another language's dubious implicit scoping features. -- Greg -- https://mail.python.org/mailman/listinfo/python-list
Re: PyEval_GetLocals and unreferenced variables
Ned Batchelder wrote: I would use thread locals for this: https://docs.python.org/2/library/threading.html#threading.local You could get dynamic scoping that way, but the OP seems to want lexical scoping. -- Greg -- https://mail.python.org/mailman/listinfo/python-list
Re: PyEval_GetLocals and unreferenced variables
Kasper Peeters wrote: I have something like def fun(): cfun_that_creates_q_in_local_scope() def fun2(): cfun_that_wants_to_see_if_q_is_available() So the Python side actually doesn't see 'q' directly at all. I am willing to elaborate on this if you want I think you will need to elaborate. There are several things about the way local scopes and references to outer scopes are implemented in Python that makes what you are asking for extremely difficult. If you explain the problem you're actually trying to solve, we'll probably be able to suggest a better solution. -- Greg -- https://mail.python.org/mailman/listinfo/python-list
PyEval_GetLocals and unreferenced variables
I have a question about PyEval_GetLocals(). The normal behaviour of
PyEval_GetLocals(), and in fact of the locals() function in Python
itself, is to return a list which includes only those variables which
are actually referenced in the local scope. Example:
def fun():
q=3
def fun2():
print(locals())
fun2()
fun()
will print {} because q has not been referenced in fun2(). On the
other hand, if you do
def fun():
q=3
def fun2():
print(q)
print(locals())
fun2()
fun()
you will instead get {'q': 3} as output. All fine and understood.
My question: I want to call a C function inside fun2(), which I want to
give access to the 'q' variable in the fun() scope, _without_ there
being any reference to 'q' in the python code itself. That is, I want
to do:
def fun():
q=3
def fun2():
cfun()
fun2()
fun()
and access 'q' inside the C-function cfun(). If I simply let it call
PyEval_GetLocals, then the result will again not contain q. Is there
any way in which I can convince python to pull 'q' into the local scope
from within my C code?
Thanks!
Cheers,
Kasper
--
https://mail.python.org/mailman/listinfo/python-list
Re: PyEval_GetLocals and unreferenced variables
On Wed, Nov 26, 2014 at 9:22 PM, Kasper Peeters kas...@phi-sci.com wrote: That is, I want to do: def fun(): q=3 def fun2(): cfun() fun2() fun() and access 'q' inside the C-function cfun(). If I simply let it call PyEval_GetLocals, then the result will again not contain q. Is there any way in which I can convince python to pull 'q' into the local scope from within my C code? Wouldn't this be a little surprising? Why not simply pass q as a parameter? ChrisA -- https://mail.python.org/mailman/listinfo/python-list
Re: PyEval_GetLocals and unreferenced variables
def fun(): q=3 def fun2(): cfun() fun2() fun() and access 'q' inside the C-function cfun(). If I simply let it call PyEval_GetLocals, then the result will again not contain q. Is there any way in which I can convince python to pull 'q' into the local scope from within my C code? Wouldn't this be a little surprising? Why not simply pass q as a parameter? I agree that in this example that would be the natural thing to do. My case is more tricky though: I have something like def fun(): cfun_that_creates_q_in_local_scope() def fun2(): cfun_that_wants_to_see_if_q_is_available() So the Python side actually doesn't see 'q' directly at all. I am willing to elaborate on this if you want (I have fairly good reasons to do things this way, mostly having to do with historical constraints of an older C library that I cannot avoid), but it requires more space. Cheers, Kasper -- https://mail.python.org/mailman/listinfo/python-list
Re: PyEval_GetLocals and unreferenced variables
On Wed, Nov 26, 2014 at 9:46 PM, Kasper Peeters kas...@phi-sci.com wrote: I agree that in this example that would be the natural thing to do. My case is more tricky though: I have something like def fun(): cfun_that_creates_q_in_local_scope() def fun2(): cfun_that_wants_to_see_if_q_is_available() So the Python side actually doesn't see 'q' directly at all. I am willing to elaborate on this if you want (I have fairly good reasons to do things this way, mostly having to do with historical constraints of an older C library that I cannot avoid), but it requires more space. To be honest, that's just made it even more weird :) You're creating something in a local namespace that the Python compiler isn't aware of. I wonder, would a 'nonlocal q' declaration inside fun2 affect things any? Otherwise, maybe there's some completely different way to transfer information around. Using locals in this way seems fraught with peril. ChrisA -- https://mail.python.org/mailman/listinfo/python-list
Re: PyEval_GetLocals and unreferenced variables
To be honest, that's just made it even more weird :) You're creating something in a local namespace that the Python compiler isn't aware of. Yes, I agree that retrieving the locals with PyEval_GetLocals and then sticking something in there on the C side is weird. I wouldn't say that the Python compiler is not aware of it though (I could ask my question purely in Python: how do I check in the scope of fun2 whether 'q' is available in the outer scope without actually using it). I wonder, would a 'nonlocal q' declaration inside fun2 affect things any? Yes, that pulls in 'q', but I would like to do that 'nonlocal q' on the C side. If I have to write 'nonlocal q' inside fun2 then I can also pull in 'q' using any other reference to 'q'. Otherwise, maybe there's some completely different way to transfer information around. Using locals in this way seems fraught with peril. There might very well be... Cheers, Kasper -- https://mail.python.org/mailman/listinfo/python-list
