TL;DR for those that may have been scared off by the length of my previous email: I propose a method of improving ob-python.el by using a progn-like eval() Python function which can wrap and execute source blocks.
If this list is not appropriate for discussion of improvements to ob-python.el, could you please give me pointers how to reach people that might be interested in helping and discussing? Kind regards, Ondřej Grover On Sat, Dec 5, 2015 at 6:17 PM, Ondřej Grover <ondrej.gro...@gmail.com> wrote: > Hello, > > I've been playing around with the Org-mode Babel framework and I am > grateful to all the contributors for making this wonderful library. After > some time I noticed that Python support seems a little hacky and > inconsistent and after reading through ob-python.el and consulting Python > documentation I came up with a proposal for improving it. > > The ob-ipython project tries to solve this hackiness in a different way by > using the client-server infrastructure of IPython/Jupyter. That works quite > well too, but my hope is that improving ob-python.el would also make it > simpler to use IPython as the python REPL, relying only on the core of the > Python language. > > It essentially boils down to implementing progn-like eval() function in > Python which would return the result of the last statement if it is an > expression. I have come up with a prototype of such a function by diving > into Python scope internals and its AST capabilities. It was written using > Org-mode and tangling so it is thoroughly documented and explained, a test > suite is included. This interesting exercise made me appreciate Lisp even > more. Here it is > https://github.com/smartass101/python_block_eval > I haven't licensed it yet, because I'm not sure what license would be > appropriate if it was used by org-mode. Any suggestions? > > My proposal is to implement an equivalent of the following bash pseudo > code for non session mode > > python -i << HEREDOC_END > ret = block_eval(""" > <CODE BLOCK BODY> > """) > open(<TMP FILE or PIPE>).write(str(ret)) > HEREDOC_END > > For session mode it would be even simpler, lines containing HEREDOC above > would be dropped and the rest piped directly into the Python REPL. > > This also means that the 'org_babel_python_eoe' string indicator may not > be necessary anymore because end of evaluation would be simply shown by a > new line with the primary prompt appearing. > > The reason why `python -i` (force interactive mode) is used is the IMHO > poor design choice in the CPython implementation (and other implementations > have a similar issue AFAIK) to have fast but RO access to local variable > scope in non-top-level (i.e. functions calling functions) frames/scopes. I > tried to hack my way around it in the update_locals_after_eval branch of my > repo to no avail, perhaps some Pythonista among you may know a solution. > > I also favor piping input into `python -i` because it means that a > temporary file does not have to be created. > > As explained in the README.org in my repo, this inconsistency can be > worked around by explicitly first evaluating the side-effect-only part of > the block and than the last expression with a direct eval() call for each. > This makes it longer by 2 lines, but has the advantage of properly handling > variable scope and separating side-effects, which could be used to e.g. > suppress output. Nevertheless, I think that for Org-mode Babel usage the > `python -i` and block_eval() approach suffices, unless someone finds a way > to use the advantages of the alternative approach to improve ob-python.el > even further. > > I'm not a skilled Elisp programmer, so I wanted to ask around as to the > feasibility of this endeavor or possibly availability of helping hands > before I devote more time to this. > > Kind regards, > Ondřej Grover >
