Documentation Results of Evaluation (was Re: Documentation of hline symbol in source blocks results)
Thanks for your message, Tim! You make many good points and I can
only respond to a few of them.
I'm also pleased to see you've got FSF Assignment:
https://orgmode.org/worg/contributors.html
Do you have time to work together on this?
I have four goals:
1. Understand your perspective
2. Remove confusion
3. Detail specific problems to solve
4. Suggest, or commit, specific changes to the manual which resolve
the specific problems
This email focuses on the first three. I'll first respond to you, try
to understand your perspective, remove confusion, and highlight
specific problems to solve. I'll then give some comments on the
specific problems. My hope is that after several exchanges, we'll
have a list of specific problems that the mailing list can then
discuss for solutions.
On Mon, 25 Dec 2023 17:37:41 +0100 Tim Landscheidt wrote ---
> the documentation reads like users having had an issue...and then
> someone feeling the need to document the solution right then and
> there.
What you say likely happened.
> IMNSHO this leads to documentation that is not very usable
> for the general audience.
Agreed!
> Just for further inspiration how the documentation could be
> rewritten, consider the current wording of "Results of Eval-
> uation/Collection/value":
>
> | Default for most Babel libraries(1). Functional mode. Org gets
> | the value by wrapping the code in a function definition in the
> | language of the source block. That is why when using ‘:results
> | value’, code should execute like a function and return a value.
> | For languages like Python, an explicit ‘return’ statement is
> | mandatory when using ‘:results value’. Result is the value
> | returned by the last statement in the code block.
>
> | When evaluating the code block in a session (see *note Environment
> | of a Code Block::), Org passes the code to an interpreter running
> | as an interactive Emacs inferior process. Org gets the value from
> | the source code interpreter’s last statement output. Org has to
> | use language-specific methods to obtain the value. For example,
> | from the variable ‘_’ in Ruby, and the value of ‘.Last.value’ in R.
>
> Wrapping the code?
When you write
#+begin_src C
printf("hello, world!\n");
#+end_src
The code that's executed is actually:
int main() {
printf("hello, world!\n");
return 0;
}
When you write
#+begin_src python
print("hello, world!")
#+end_src
The code that's executed is actually (brace for it...):
def __org_babel_python_format_value(result, result_file, result_params):
with open(result_file, 'w') as f:
if 'graphics' in result_params:
result.savefig(result_file)
elif 'pp' in result_params:
import pprint
f.write(pprint.pformat(result))
elif 'list' in result_params and isinstance(result, dict):
f.write(str(['{} :: {}'.format(k, v) for k, v in result.items()]))
else:
if not set(result_params).intersection(['scalar', 'verbatim',
'raw']):
def dict2table(res):
if isinstance(res, dict):
return [(k, dict2table(v)) for k, v in res.items()]
elif isinstance(res, list) or isinstance(res, tuple):
return [dict2table(x) for x in res]
else:
return res
if 'table' in result_params:
result = dict2table(result)
try:
import pandas
except ImportError:
pass
else:
if isinstance(result, pandas.DataFrame) and 'table' in
result_params:
result = [[result.index.name or ''] +
list(result.columns)] + [None] + [[i] + list(row) for i, row in
result.iterrows()]
elif isinstance(result, pandas.Series) and 'table' in
result_params:
result = list(result.items())
try:
import numpy
except ImportError:
pass
else:
if isinstance(result, numpy.ndarray):
if 'table' in result_params:
result = result.tolist()
else:
result = repr(result)
f.write(str(result))
def main():
print("hello, world!")
__org_babel_python_format_value(main(), '/tmp/babel-7Zq1c7/python-e7IyhX',
["replace"])
However, not every language is wrapped.
For bash, when you write
#+begin_src bash
echo "hello, world!"
#+end_src
Something like this is passed to a bash process:
echo "hello, world!"
PROBLEM: "wrapping" is inaccurate
> "Code /should/"?
I agree, this can improve. "Should" in this context reads as
"probably" and is non-committal. Documentation has a duty to be
author
Re: Documentation of hline symbol in source blocks results
Tim Landscheidt writes:
> this is just a data point for anyone looking at restructur-
> ing the Org documentation.
>
> I wanted an Emacs Lisp source block to produce a table with
> a column name and a horizontal line separating the column
> name and the data cells. The data-only source block:
> ...
> I wanted to get the result:
>
> | #+RESULTS:
> | | Number |
> | ||
> | | 1 |
> | | 2 |
> | | 3 |
>
> It took me quite a while to figure out that the first row
> returned from the source block can be viewed as the column
> name, and a horizontal line can be achieved by returning the
> symbol 'hline:
AFAIK, producing tables with column names in
emacs-lisp code blocks is not officially supported.
The `hline' is not documented.
> The only reference to this behaviour I could find, is liter-
> ally the last (!, :-)) paragraph in the info file:
>
> |For complicated translations the generic translator function could be
> | replaced by a custom translator function. Such a custom function must
> | take two arguments and return a single string containing the formatted
> | table. The first argument is the table whose lines are a list of fields
> ^^
> | or the symbol ‘hline’.
This happens to be the machinery Org babel uses to convert tables from
Elisp format to Org mode format. We do not give you any guarantee that
we are going to keep using `orgtbl-to-generic' in future the same way.
> Just for further inspiration how the documentation could be
> rewritten, consider the current wording of "Results of Eval-
> uation/Collection/value":
>
> | Default for most Babel libraries(1). Functional mode. Org gets
> | the value by wrapping the code in a function definition in the
> | language of the source block. That is why when using ‘:results
> | value’, code should execute like a function and return a value.
> | For languages like Python, an explicit ‘return’ statement is
> | mandatory when using ‘:results value’. Result is the value
> | returned by the last statement in the code block.
>
> | When evaluating the code block in a session (see *note Environment
> | of a Code Block::), Org passes the code to an interpreter running
> | as an interactive Emacs inferior process. Org gets the value from
> | the source code interpreter’s last statement output. Org has to
> | use language-specific methods to obtain the value. For example,
> | from the variable ‘_’ in Ruby, and the value of ‘.Last.value’ in R.
>
> Wrapping the code? "Code /should/"? "Like a function"?
> Why is the Python requirement stated here?
I am sorry, but I do not fully understand the problem.
What we are saying is that the code block should return a value.
And give an example that python blocks, in order to return a value,
should have `return' statement.
> ... Why "using
> ‘:results value’" when the paragraph should (only) document
> this?
Emm.. But we are documenting :results value is this paragraph.
Mentioning it again is a bit redundant, but not the end of the world.
Improvements welcome, of course.
> ... "Result is the value"? What kind of value? Why are
> there references to Ruby and R here? All this confuses me
> and does not provide the information I searched for (empha-
> sis on me).
Ruby and R have their own syntax to return value, just like "return" in
python.
I hope it is clear now for you.
> I would probably prefer a clean-slate approach that starts
> with something along the lines of: "Source blocks produce
> results that can be integrated into an Org document and used
> as input for other source blocks. The abstract specifica-
> tion for source blocks looks like this ("structured table
> data" vs. "text dump", header arguments, export, etc.).
> Emacs Lisp source blocks produce these values in this way.
> Python that way. Shell scripts return tables in this way."
>
> Org is extremely powerful, but (I think I wrote it somewhere
> before) the documentation reads like users having had an is-
> sue ("my Python code does not produce (correct) results")
> and then someone feeling the need to document the solution
> right then and there. IMNSHO this leads to documentation
> that is not very usable for the general audience.
AFAIK, the particular paragraph you pointed out does not mention Python
just to address someone issue. Python/R/Ruby are just examples used.
Though I do agree that this (and many other) sections of the manual can
be improved. Patches welcome!
--
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at ,
or support my work at
Documentation of hline symbol in source blocks results
Hi,
this is just a data point for anyone looking at restructur-
ing the Org documentation.
I wanted an Emacs Lisp source block to produce a table with
a column name and a horizontal line separating the column
name and the data cells. The data-only source block:
| #+BEGIN_SRC emacs-lisp
| '((1)
| (2)
| (3))
| #+END_SRC
produces the result:
| #+RESULTS:
| | 1 |
| | 2 |
| | 3 |
I wanted to get the result:
| #+RESULTS:
| | Number |
| ||
| | 1 |
| | 2 |
| | 3 |
It took me quite a while to figure out that the first row
returned from the source block can be viewed as the column
name, and a horizontal line can be achieved by returning the
symbol 'hline:
| #+BEGIN_SRC emacs-lisp
| '(("Number")
| hline
| (1)
| (2)
| (3))
| #+END_SRC
The only reference to this behaviour I could find, is liter-
ally the last (!, :-)) paragraph in the info file:
|For complicated translations the generic translator function could be
| replaced by a custom translator function. Such a custom function must
| take two arguments and return a single string containing the formatted
| table. The first argument is the table whose lines are a list of fields
^^
| or the symbol ‘hline’. The second argument is the property list
^
| consisting of parameters specified in the ‘#+ORGTBL: SEND’ line. Please
| share your translator functions by posting them to the Org users mailing
| list, at .
Just for further inspiration how the documentation could be
rewritten, consider the current wording of "Results of Eval-
uation/Collection/value":
| Default for most Babel libraries(1). Functional mode. Org gets
| the value by wrapping the code in a function definition in the
| language of the source block. That is why when using ‘:results
| value’, code should execute like a function and return a value.
| For languages like Python, an explicit ‘return’ statement is
| mandatory when using ‘:results value’. Result is the value
| returned by the last statement in the code block.
| When evaluating the code block in a session (see *note Environment
| of a Code Block::), Org passes the code to an interpreter running
| as an interactive Emacs inferior process. Org gets the value from
| the source code interpreter’s last statement output. Org has to
| use language-specific methods to obtain the value. For example,
| from the variable ‘_’ in Ruby, and the value of ‘.Last.value’ in R.
Wrapping the code? "Code /should/"? "Like a function"?
Why is the Python requirement stated here? Why "using
‘:results value’" when the paragraph should (only) document
this? "Result is the value"? What kind of value? Why are
there references to Ruby and R here? All this confuses me
and does not provide the information I searched for (empha-
sis on me).
I would probably prefer a clean-slate approach that starts
with something along the lines of: "Source blocks produce
results that can be integrated into an Org document and used
as input for other source blocks. The abstract specifica-
tion for source blocks looks like this ("structured table
data" vs. "text dump", header arguments, export, etc.).
Emacs Lisp source blocks produce these values in this way.
Python that way. Shell scripts return tables in this way."
Org is extremely powerful, but (I think I wrote it somewhere
before) the documentation reads like users having had an is-
sue ("my Python code does not produce (correct) results")
and then someone feeling the need to document the solution
right then and there. IMNSHO this leads to documentation
that is not very usable for the general audience.
Tim
