Re: [O] bug in expansion of variables in babel Perl
Achim Eric Schulte writes: Are you familiar with `org-babel-expand-src-block' bound to C-c C-v v? Achim I wasn't, obviously, and neither was the OP. If I understand the desire correctly, it should be what you're after. Perhaps an option to raise the expanded source code buffer along with the error buffer when an error is raised would be a useful addition. Achim Yes, so there's no three ways for Perl to get the same result for Achim comparison. :-P Just keep in mind, org-babel-expand-src-block is not exactly what emacs passes to perl. It does not contain the wrapping code. --dmg -- Daniel M. German http://turingmachine.org/ http://silvernegative.com/ dmg (at) uvic (dot) ca replace (at) with @ and (dot) with .
Re: [O] bug in expansion of variables in babel Perl
Mm, I also noticed that when :results output is used, there is no way to insert perl code before or after the executed code. org-babel-perl-wrapper-method only works for all the methods but output. It would be nice to have a variable that does this for any output type. --dmg On Sun, Feb 24, 2013 at 1:16 AM, D M German d...@uvic.ca wrote: Hi Everybody, I found a bug in the Babel perl code. When a table is used as input, the values of the table are not escaped. In fact, they are surrounded by double quotes instead of single ones '. This means that special characters are interpreted: $string, and @variable are considered variables. See below. --dmg --- Daniel M. German http://turingmachine.org
Re: [O] bug in expansion of variables in babel Perl
dmg Mm, I also noticed that when :results output is used, there is no way
dmg to insert perl code before or after the executed code.
dmg org-babel-perl-wrapper-method only works for all the methods but
dmg output. It would be nice to have a variable that
dmg does this for any output type.
I implemented a proof-of-concept. The idea is to have a variable called
org-babel-perl-preface that is inserted before the code. I also like to
be able to use my in all variables, so I can use strict, if I choose
to. See the patch at the bottom.
here is a test example:
==
Input table
#+RESULTS: patito
| id | title | year | index | notes | attr |
|+-+--+---+---+--|
| Taxi Driver (1944) | Taxi Driver | 1944 | | | |
| Taxi Driver (1954) | Taxi Driver | 1954 | | | |
| Taxi Driver (1973) | Taxi Driver | 1973 | | | |
| Taxi Driver (1976) | Taxi Driver | 1976 | | | |
| Taxi Driver (1978) | Taxi Driver | 1978 | | | |
| Taxi Driver (1981) | Taxi Driver | 1981 | | | |
| Taxi Driver (1990) | Taxi Driver | 1990 | | | |
| Taxi Driver (2004) | Taxi Driver | 2004 | | | |
Simple row output: the last statement is returned as a list, each in a line.
#+name: output2(data=patito)
#+begin_src perl :results raw
org_rows($data), org_columns($data);
#+end_src
#+RESULTS: output2
8
6
More complex example. By defining org_rows and org_columns in the
preface, it makes it easier to manipulate them. org-babel implements
tables as a reference to an array of references to arrays.
#+name: rip(data=patito)
#+begin_src perl :results output
my $rows = org_rows($data);
my $columns = org_columns($data);
for (my $j=0;$j$rows; $j++) {
for (my $i=0;$i$columns; $i++) {
print $i:$j ;
print $$data[$j][$i];
print;
}
print |\n;
}
print Row $rows\n;
print Columns $columns\n;
#+end_src
#+RESULTS: rip
#+begin_example
0:0 Taxi Driver (1944) ;1:0 Taxi Driver ;2:0 1944 ;3:0;4:0;5:0
;|
0:1 Taxi Driver (1954) ;1:1 Taxi Driver ;2:1 1954 ;3:1;4:1;5:1
;|
0:2 Taxi Driver (1973) ;1:2 Taxi Driver ;2:2 1973 ;3:2;4:2;5:2
;|
0:3 Taxi Driver (1976) ;1:3 Taxi Driver ;2:3 1976 ;3:3;4:3;5:3
;|
0:4 Taxi Driver (1978) ;1:4 Taxi Driver ;2:4 1978 ;3:4;4:4;5:4
;|
0:5 Taxi Driver (1981) ;1:5 Taxi Driver ;2:5 1981 ;3:5;4:5;5:5
;|
0:6 Taxi Driver (1990) ;1:6 Taxi Driver ;2:6 1990 ;3:6;4:6;5:6
;|
0:7 Taxi Driver (2004) ;1:7 Taxi Driver ;2:7 2004 ;3:7;4:7;5:7
;|
Row 8
Columns 6
#+end_example
==
diff --git a/lisp/ob-perl.el b/lisp/ob-perl.el
index ccd3826..65e6b88 100644
--- a/lisp/ob-perl.el
+++ b/lisp/ob-perl.el
@@ -62,7 +62,7 @@ This function is called by `org-babel-execute-src-block'.
Return list of perl statements assigning the block's variables.
(mapcar
(lambda (pair)
- (format $%s=%s;
+ (format my $%s=%s;
(car pair)
(org-babel-perl-var-to-perl (cdr pair
(mapcar #'cdr (org-babel-get-header params :var
@@ -85,13 +85,34 @@ specifying a var of the same value.
(defvar org-babel-perl-wrapper-method
+%s
sub main {
%s
}
-@r = main;
+my @r = main;
open(o, \%s\);
print o join(\\\n\, @r), \\\n\)
+(defvar org-babel-perl-preface
+
+use strict;
+
+sub org_columns
+{
+my ($table) = @_;
+my $y = $$table[0];
+return scalar(@$y);
+}
+
+sub org_rows
+{
+my ($table) = @_;
+return scalar(@$table);
+}
+
+)
+
+
(defvar org-babel-perl-pp-wrapper-method
nil)
@@ -102,11 +123,11 @@ of the statements in BODY, if RESULT-TYPE equals 'value
then
return the value of the last statement in BODY, as elisp.
(when session (error Sessions are not supported for Perl))
(case result-type
-(output (org-babel-eval org-babel-perl-command body))
+(output (org-babel-eval org-babel-perl-command (format %s\n%s
org-babel-perl-preface body)))
(value (let ((tmp-file (org-babel-temp-file perl-)))
(org-babel-eval
org-babel-perl-command
- (format org-babel-perl-wrapper-method body
+ (format org-babel-perl-wrapper-method org-babel-perl-preface body
(org-babel-process-file-name tmp-file 'noquote)))
(org-babel-eval-read-file tmp-file)
--
Daniel M. German In questions of science the authority of
a thousand is not worth the humble
Galileo - reasoning of a single individual.
http://turingmachine.org/
http://silvernegative.com/
dmg (at) uvic (dot) ca
replace (at) with @ and (dot) with .
Re: [O] bug in expansion of variables in babel Perl
D M German writes:
I see two ways to solve this. The first is simply to replace the output
format of the variable from %S to '%s' (use quotes ').
I think that's the right thing to do. There shouldn't be anything in
the table that needs to be interpolated by Perl while the variable is
defined.
+(format '%s' var)))
A slightly more perlish way would be to use
(format q(%s) var)))
Debugging perl is very cumbersome in org-mode. It would be nice to have
a feature to export the source to a file. This is because the variable
expansion needs to be done before the code can be used (hence simply cut
and paste does not work, nor shell-command-on-region)
The other languages have the same problem, maybe there should be a
general option to mirror the commands into a source block in the org
file or to make the buffer with the program to be eval'ed stick around.
Eric, WDYT? Also, I'm not really sure why we need all the complexity of
shell-command-on-region when it looks like we should be able to
call-process-region ourselves. Modifying Babel to run (non-session and
perhaps optionally) from files instead of buffers seems to be a more
wide-reaching operation.
As we are into it, I found this declaration to be very useful.
[…]
I think this is better done by altering org-babel-perl-command to
include -Mstrict. If you put the helper functions into a module in
@INC or tell Perl where to find them, then you can add -Mmyhelper as
well. Here's a wrapper to match:
(defvar org-babel-perl-wrapper-method
{
my @r = eval( q(%s) );
open my $BO, qq(%s));
print $BO join($\\, @r), $\\ ;
})
For your problem with :results output blocks, I think it would be
possible to wrap them (a bit differently) also, but the helper module
above would also solve this problem, so let's see what Eric says since I
don't know if another language has already set a precedent of wrapping
these commands too.
Finally, if interested, i can write a couple of examples for Perl that
could help people who want to use it.
Also a few tests would be highly welcome.
Regards,
Achim.
--
+[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]+
SD adaptations for KORG EX-800 and Poly-800MkII V0.9:
http://Synth.Stromeko.net/Downloads.html#KorgSDada
Re: [O] bug in expansion of variables in babel Perl
D M German writes: […] Please leave the formats alone, if you change the number of parameters there folks that use their own definitions won't know what hit them. What you want is to prepend something to the body that Babel gives you, so let-bind that result and use it. You could even advise the function and have it submit to your will without changing Org. --8---cut here---start-8--- (defun org-babel-perl-evaluate (session ibody optional result-type) Pass BODY to the Perl process in SESSION. If RESULT-TYPE equals 'output then return a list of the outputs of the statements in BODY, if RESULT-TYPE equals 'value then return the value of the last statement in BODY, as elisp. (when session (error Sessions are not supported for Perl)) (let ((body (concat org-babel-perl-preface ibody))) (case result-type (output (org-babel-eval org-babel-perl-command body)) (value (let ((tmp-file (org-babel-temp-file perl-))) (org-babel-eval org-babel-perl-command (format org-babel-perl-wrapper-method body (org-babel-process-file-name tmp-file 'noquote))) (org-babel-eval-read-file tmp-file)) --8---cut here---end---8--- BTW, now that I think some more about it: debugging Perl is much easier than you seem to let on: (setq org-babel-perl-command perl -Mstrict -ne print). This will echo the program sent to Perl in full glory into the output block. Regards, Achim. -- +[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]+ Waldorf MIDI Implementation additional documentation: http://Synth.Stromeko.net/Downloads.html#WaldorfDocs
Re: [O] bug in expansion of variables in babel Perl
Achim Gratz strom...@nexgo.de writes:
D M German writes:
I see two ways to solve this. The first is simply to replace the output
format of the variable from %S to '%s' (use quotes ').
I think that's the right thing to do. There shouldn't be anything in
the table that needs to be interpolated by Perl while the variable is
defined.
+(format '%s' var)))
A slightly more perlish way would be to use
(format q(%s) var)))
I just added the variable `org-babel-perl-var-wrap', into ob-perl.el
;; emacs-lisp
(defvar org-babel-perl-var-wrap q(%s)
Wrapper for variables inserted into Perl code.)
This way we will get what sounds like improved wrapping by default, but
users who really do want to insert interpolated values can customize
this variable.
Debugging perl is very cumbersome in org-mode. It would be nice to have
a feature to export the source to a file. This is because the variable
expansion needs to be done before the code can be used (hence simply cut
and paste does not work, nor shell-command-on-region)
The other languages have the same problem, maybe there should be a
general option to mirror the commands into a source block in the org
file or to make the buffer with the program to be eval'ed stick around.
Eric, WDYT?
Are you familiar with `org-babel-expand-src-block' bound to C-c C-v v?
If I understand the desire correctly, it should be what you're after.
Perhaps an option to raise the expanded source code buffer along with
the error buffer when an error is raised would be a useful addition.
Also, I'm not really sure why we need all the complexity of
shell-command-on-region when it looks like we should be able to
call-process-region ourselves. Modifying Babel to run (non-session
and perhaps optionally) from files instead of buffers seems to be a
more wide-reaching operation.
This complexity is related to the need to occasionally run in remote
directories or on remote machines. If there are ways to reduce this
complexity without losing functionality I'm game.
As we are into it, I found this declaration to be very useful.
[…]
I think this is better done by altering org-babel-perl-command to
include -Mstrict. If you put the helper functions into a module in
@INC or tell Perl where to find them, then you can add -Mmyhelper as
well. Here's a wrapper to match:
(defvar org-babel-perl-wrapper-method
{
my @r = eval( q(%s) );
open my $BO, qq(%s));
print $BO join($\\, @r), $\\ ;
})
For your problem with :results output blocks, I think it would be
possible to wrap them (a bit differently) also, but the helper module
above would also solve this problem, so let's see what Eric says since I
don't know if another language has already set a precedent of wrapping
these commands too.
I'm not currently aware of any language-wide support for printing the
expanded code block along with the results. I don't think there has
been any desire for this previously. It shouldn't be hard to write an
emacs lisp block to give the desired result...
Here's an example of such a function. Please forgive my complete lack
of Perl knowledge (at one time I knew it well). Note that I had to make
a small change to the return value of `org-babel-expand-src-block'.
#+name: table
| 1 | 2 | 3 |
| 4 | 5 | 6 |
#+name: perl-block
#+begin_src perl :var table=table
print $table
#+end_src
#+RESULTS: perl-block
: 1
#+begin_src emacs-lisp :var block=perl-block :results raw
(save-excursion
(message block is %S block)
(org-babel-goto-named-src-block block)
(format #+begin_example\n%S\n↓\n%s\n#+end_example\n
(org-babel-expand-src-block)
(org-babel-execute-src-block)))
#+end_src
#+RESULTS:
#+begin_example
$table=[[q(1), q(2), q(3)], [q(4), q(5), q(6)]];
print $table
↓
1
#+end_example
Finally, if interested, i can write a couple of examples for Perl that
could help people who want to use it.
Also a few tests would be highly welcome.
+1
Cheers,
Regards,
Achim.
--
Eric Schulte
http://cs.unm.edu/~eschulte
Re: [O] bug in expansion of variables in babel Perl
Eric Schulte writes: I just added the variable `org-babel-perl-var-wrap', into ob-perl.el ;; emacs-lisp (defvar org-babel-perl-var-wrap q(%s) Wrapper for variables inserted into Perl code.) This way we will get what sounds like improved wrapping by default, but users who really do want to insert interpolated values can customize this variable. It would be impossible to use any user variables for interpolation since we don't have Perl sessions, so only pre-defined variables from Perl would ever deliver a value (perhaps). So all interpolation would do most of the the time is to replace something that happens to look like a perl variable by nothing. Having the variable doesn't hurt, but I'm not sure we should advertize its existence widely since with a more devious definition you can do arbitrary code execution. This complexity is related to the need to occasionally run in remote directories or on remote machines. If there are ways to reduce this complexity without losing functionality I'm game. Already done by throwing away those parts of the code we never used anyway. It looks much more manageable now. I'm not currently aware of any language-wide support for printing the expanded code block along with the results. I don't think there has been any desire for this previously. It shouldn't be hard to write an emacs lisp block to give the desired result... Also done. The file the OP wanted to look at gets written out anyway, we just need to prevent its deletion. This is a manual affair for now, if this is really a big issue we can add an option for this. Regards, Achim. -- +[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]+ SD adaptation for Waldorf rackAttack V1.04R1: http://Synth.Stromeko.net/Downloads.html#WaldorfSDada
Re: [O] bug in expansion of variables in babel Perl
Eric Schulte writes: Are you familiar with `org-babel-expand-src-block' bound to C-c C-v v? I wasn't, obviously, and neither was the OP. If I understand the desire correctly, it should be what you're after. Perhaps an option to raise the expanded source code buffer along with the error buffer when an error is raised would be a useful addition. Yes, so there's no three ways for Perl to get the same result for comparison. :-P Still, the debugging option might be useful when one needs to to see all sources created during a publish or something like that; when you don't know if or where the error occured. Regards, Achim. -- +[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]+ SD adaptations for KORG EX-800 and Poly-800MkII V0.9: http://Synth.Stromeko.net/Downloads.html#KorgSDada
Re: [O] bug in expansion of variables in babel Perl
Achim D M German writes: Achim […] Achim Please leave the formats alone, if you change the number of parameters Achim there folks that use their own definitions won't know what hit them. Achim What you want is to prepend something to the body that Babel gives you, Achim so let-bind that result and use it. You could even advise the function Achim and have it submit to your will without changing Org. Hi Achim, thanks for the recommendation. As I said before, see my previous patch as a proof-of-concept and not as something that I think should be applied. I am fully aware of the potential consequences. Achim BTW, now that I think some more about it: debugging Perl is much easier Achim than you seem to let on: Achim (setq org-babel-perl-command perl -Mstrict -ne print). Using strict is complicated by use of variables from org tables and the way that the code in R is executed. It would require the variable @r and each of the created variables from tables to be defined as my. I know I can change the behaviour of org-babel-variable-assignment, but perhaps simply adding my to any variable created in org would be a good thing to do. I don't think it would harm, anyways. Achim This will echo the program sent to Perl in full glory into the output Achim block. thanks, this i a great suggestion that I didn't know about. --dmg Achim Regards, Achim Achim. Achim -- Achim +[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]+ Achim Waldorf MIDI Implementation additional documentation: Achim http://Synth.Stromeko.net/Downloads.html#WaldorfDocs -- Daniel M. German Beware of bugs in the above code; I have only proved it Donald Knuth -correct, not tried it. http://turingmachine.org/ http://silvernegative.com/ dmg (at) uvic (dot) ca replace (at) with @ and (dot) with .
