Re: [O] bug in expansion of variables in babel Perl
Achim> Eric Schulte writes: >> Are you familiar with `org-babel-expand-src-block' bound to C-c C-v v? Achim> I wasn't, obviously, and neither was the OP. >> If I understand the desire correctly, it should be what you're after. >> Perhaps an option to raise the expanded source code buffer along with >> the error buffer when an error is raised would be a useful addition. Achim> Yes, so there's no three ways for Perl to get the same result for Achim> comparison. :-P Just keep in mind, org-babel-expand-src-block is not exactly what emacs passes to perl. It does not contain the wrapping code. --dmg -- Daniel M. German http://turingmachine.org/ http://silvernegative.com/ dmg (at) uvic (dot) ca replace (at) with @ and (dot) with .
Re: [O] bug in expansion of variables in babel Perl
Achim> D M German writes: Achim> […] Achim> Please leave the formats alone, if you change the number of parameters Achim> there folks that use their own definitions won't know what hit them. Achim> What you want is to prepend something to the body that Babel gives you, Achim> so let-bind that result and use it. You could even advise the function Achim> and have it submit to your will without changing Org. Hi Achim, thanks for the recommendation. As I said before, see my previous patch as a proof-of-concept and not as something that I think should be applied. I am fully aware of the potential consequences. Achim> BTW, now that I think some more about it: debugging Perl is much easier Achim> than you seem to let on: Achim> (setq org-babel-perl-command "perl -Mstrict -ne print"). Using strict is complicated by use of variables from org tables and the way that the code in R is executed. It would require the variable @r and each of the created variables from tables to be defined as my. I know I can change the behaviour of org-babel-variable-assignment, but perhaps simply adding "my" to any variable created in org would be a good thing to do. I don't think it would harm, anyways. Achim> This will echo the program sent to Perl in full glory into the output Achim> block. thanks, this i a great suggestion that I didn't know about. --dmg Achim> Regards, Achim> Achim. Achim> -- Achim> +<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+ Achim> Waldorf MIDI Implementation & additional documentation: Achim> http://Synth.Stromeko.net/Downloads.html#WaldorfDocs -- Daniel M. German "Beware of bugs in the above code; I have only proved it Donald Knuth ->correct, not tried it." http://turingmachine.org/ http://silvernegative.com/ dmg (at) uvic (dot) ca replace (at) with @ and (dot) with .
Re: [O] bug in expansion of variables in babel Perl
Eric Schulte writes: > Are you familiar with `org-babel-expand-src-block' bound to C-c C-v v? I wasn't, obviously, and neither was the OP. > If I understand the desire correctly, it should be what you're after. > Perhaps an option to raise the expanded source code buffer along with > the error buffer when an error is raised would be a useful addition. Yes, so there's no three ways for Perl to get the same result for comparison. :-P Still, the debugging option might be useful when one needs to to see all sources created during a publish or something like that; when you don't know if or where the error occured. Regards, Achim. -- +<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+ SD adaptations for KORG EX-800 and Poly-800MkII V0.9: http://Synth.Stromeko.net/Downloads.html#KorgSDada
Re: [O] bug in expansion of variables in babel Perl
Eric Schulte writes: > I just added the variable `org-babel-perl-var-wrap', into ob-perl.el > > ;; emacs-lisp > (defvar org-babel-perl-var-wrap "q(%s)" > "Wrapper for variables inserted into Perl code.") > > This way we will get what sounds like improved wrapping by default, but > users who really do want to insert interpolated values can customize > this variable. It would be impossible to use any user variables for interpolation since we don't have Perl sessions, so only pre-defined variables from Perl would ever deliver a value (perhaps). So all interpolation would do most of the the time is to replace something that happens to look like a perl variable by nothing. Having the variable doesn't hurt, but I'm not sure we should advertize its existence widely since with a more devious definition you can do arbitrary code execution. > This complexity is related to the need to occasionally run in remote > directories or on remote machines. If there are ways to reduce this > complexity without losing functionality I'm game. Already done by throwing away those parts of the code we never used anyway. It looks much more manageable now. > I'm not currently aware of any language-wide support for printing the > expanded code block along with the results. I don't think there has > been any desire for this previously. It shouldn't be hard to write an > emacs lisp block to give the desired result... Also done. The file the OP wanted to look at gets written out anyway, we just need to prevent its deletion. This is a manual affair for now, if this is really a big issue we can add an option for this. Regards, Achim. -- +<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+ SD adaptation for Waldorf rackAttack V1.04R1: http://Synth.Stromeko.net/Downloads.html#WaldorfSDada
Re: [O] bug in expansion of variables in babel Perl
Achim Gratz writes:
> D M German writes:
>> I see two ways to solve this. The first is simply to replace the output
>> format of the variable from "%S" to "'%s'" (use quotes ').
>
> I think that's the right thing to do. There shouldn't be anything in
> the table that needs to be interpolated by Perl while the variable is
> defined.
>
>> +(format "'%s'" var)))
>
> A slightly more perlish way would be to use
>(format "q(%s)" var)))
>
I just added the variable `org-babel-perl-var-wrap', into ob-perl.el
;; emacs-lisp
(defvar org-babel-perl-var-wrap "q(%s)"
"Wrapper for variables inserted into Perl code.")
This way we will get what sounds like improved wrapping by default, but
users who really do want to insert interpolated values can customize
this variable.
>
>> Debugging perl is very cumbersome in org-mode. It would be nice to have
>> a feature to export the source to a file. This is because the variable
>> expansion needs to be done before the code can be used (hence simply cut
>> and paste does not work, nor shell-command-on-region)
>
> The other languages have the same problem, maybe there should be a
> general option to mirror the commands into a source block in the org
> file or to make the buffer with the program to be eval'ed stick around.
>
> Eric, WDYT?
Are you familiar with `org-babel-expand-src-block' bound to C-c C-v v?
If I understand the desire correctly, it should be what you're after.
Perhaps an option to raise the expanded source code buffer along with
the error buffer when an error is raised would be a useful addition.
> Also, I'm not really sure why we need all the complexity of
> shell-command-on-region when it looks like we should be able to
> call-process-region ourselves. Modifying Babel to run (non-session
> and perhaps optionally) from files instead of buffers seems to be a
> more wide-reaching operation.
>
This complexity is related to the need to occasionally run in remote
directories or on remote machines. If there are ways to reduce this
complexity without losing functionality I'm game.
>
>> As we are into it, I found this declaration to be very useful.
> […]
>
> I think this is better done by altering org-babel-perl-command to
> include "-Mstrict". If you put the helper functions into a module in
> @INC or tell Perl where to find them, then you can add "-Mmyhelper" as
> well. Here's a wrapper to match:
>
> (defvar org-babel-perl-wrapper-method
> "{
> my @r = eval( q(%s) );
> open my $BO, qq(>%s));
> print $BO join($\\, @r), $\\ ;
> }")
>
> For your problem with :results output blocks, I think it would be
> possible to wrap them (a bit differently) also, but the "helper module"
> above would also solve this problem, so let's see what Eric says since I
> don't know if another language has already set a precedent of wrapping
> these commands too.
>
I'm not currently aware of any language-wide support for printing the
expanded code block along with the results. I don't think there has
been any desire for this previously. It shouldn't be hard to write an
emacs lisp block to give the desired result...
Here's an example of such a function. Please forgive my complete lack
of Perl knowledge (at one time I knew it well). Note that I had to make
a small change to the return value of `org-babel-expand-src-block'.
#+name: table
| 1 | 2 | 3 |
| 4 | 5 | 6 |
#+name: perl-block
#+begin_src perl :var table=table
print $table
#+end_src
#+RESULTS: perl-block
: 1
#+begin_src emacs-lisp :var block="perl-block" :results raw
(save-excursion
(message "block is %S" block)
(org-babel-goto-named-src-block block)
(format "#+begin_example\n%S\n↓\n%s\n#+end_example\n"
(org-babel-expand-src-block)
(org-babel-execute-src-block)))
#+end_src
#+RESULTS:
#+begin_example
"$table=[[q(1), q(2), q(3)], [q(4), q(5), q(6)]];
print $table"
↓
1
#+end_example
>
>>
>> Finally, if interested, i can write a couple of examples for Perl that
>> could help people who want to use it.
>
> Also a few tests would be highly welcome.
>
+1
Cheers,
>
>
> Regards,
> Achim.
--
Eric Schulte
http://cs.unm.edu/~eschulte
Re: [O] bug in expansion of variables in babel Perl
D M German writes: […] Please leave the formats alone, if you change the number of parameters there folks that use their own definitions won't know what hit them. What you want is to prepend something to the body that Babel gives you, so let-bind that result and use it. You could even advise the function and have it submit to your will without changing Org. --8<---cut here---start->8--- (defun org-babel-perl-evaluate (session ibody &optional result-type) "Pass BODY to the Perl process in SESSION. If RESULT-TYPE equals 'output then return a list of the outputs of the statements in BODY, if RESULT-TYPE equals 'value then return the value of the last statement in BODY, as elisp." (when session (error "Sessions are not supported for Perl")) (let ((body (concat org-babel-perl-preface ibody))) (case result-type (output (org-babel-eval org-babel-perl-command body)) (value (let ((tmp-file (org-babel-temp-file "perl-"))) (org-babel-eval org-babel-perl-command (format org-babel-perl-wrapper-method body (org-babel-process-file-name tmp-file 'noquote))) (org-babel-eval-read-file tmp-file)) --8<---cut here---end--->8--- BTW, now that I think some more about it: debugging Perl is much easier than you seem to let on: (setq org-babel-perl-command "perl -Mstrict -ne print"). This will echo the program sent to Perl in full glory into the output block. Regards, Achim. -- +<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+ Waldorf MIDI Implementation & additional documentation: http://Synth.Stromeko.net/Downloads.html#WaldorfDocs
Re: [O] bug in expansion of variables in babel Perl
D M German writes:
> I see two ways to solve this. The first is simply to replace the output
> format of the variable from "%S" to "'%s'" (use quotes ').
I think that's the right thing to do. There shouldn't be anything in
the table that needs to be interpolated by Perl while the variable is
defined.
> +(format "'%s'" var)))
A slightly more perlish way would be to use
(format "q(%s)" var)))
> Debugging perl is very cumbersome in org-mode. It would be nice to have
> a feature to export the source to a file. This is because the variable
> expansion needs to be done before the code can be used (hence simply cut
> and paste does not work, nor shell-command-on-region)
The other languages have the same problem, maybe there should be a
general option to mirror the commands into a source block in the org
file or to make the buffer with the program to be eval'ed stick around.
Eric, WDYT? Also, I'm not really sure why we need all the complexity of
shell-command-on-region when it looks like we should be able to
call-process-region ourselves. Modifying Babel to run (non-session and
perhaps optionally) from files instead of buffers seems to be a more
wide-reaching operation.
> As we are into it, I found this declaration to be very useful.
[…]
I think this is better done by altering org-babel-perl-command to
include "-Mstrict". If you put the helper functions into a module in
@INC or tell Perl where to find them, then you can add "-Mmyhelper" as
well. Here's a wrapper to match:
(defvar org-babel-perl-wrapper-method
"{
my @r = eval( q(%s) );
open my $BO, qq(>%s));
print $BO join($\\, @r), $\\ ;
}")
For your problem with :results output blocks, I think it would be
possible to wrap them (a bit differently) also, but the "helper module"
above would also solve this problem, so let's see what Eric says since I
don't know if another language has already set a precedent of wrapping
these commands too.
>
> Finally, if interested, i can write a couple of examples for Perl that
> could help people who want to use it.
Also a few tests would be highly welcome.
Regards,
Achim.
--
+<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+
SD adaptations for KORG EX-800 and Poly-800MkII V0.9:
http://Synth.Stromeko.net/Downloads.html#KorgSDada
Re: [O] bug in expansion of variables in babel Perl
dmg> Mm, I also noticed that when :results output is used, there is no way
dmg> to insert perl code before or after the executed code.
dmg> org-babel-perl-wrapper-method only works for all the methods but
dmg> output. It would be nice to have a variable that
dmg> does this for any output type.
I implemented a proof-of-concept. The idea is to have a variable called
org-babel-perl-preface that is inserted before the code. I also like to
be able to use my in all variables, so I can use strict, if I choose
to. See the patch at the bottom.
here is a test example:
==
Input table
#+RESULTS: patito
| id | title | year | index | notes | attr |
|+-+--+---+---+--|
| Taxi Driver (1944) | Taxi Driver | 1944 | | | |
| Taxi Driver (1954) | Taxi Driver | 1954 | | | |
| Taxi Driver (1973) | Taxi Driver | 1973 | | | |
| Taxi Driver (1976) | Taxi Driver | 1976 | | | |
| Taxi Driver (1978) | Taxi Driver | 1978 | | | |
| Taxi Driver (1981) | Taxi Driver | 1981 | | | |
| Taxi Driver (1990) | Taxi Driver | 1990 | | | |
| Taxi Driver (2004) | Taxi Driver | 2004 | | | |
Simple row output: the last statement is returned as a list, each in a line.
#+name: output2(data=patito)
#+begin_src perl :results raw
org_rows($data), org_columns($data);
#+end_src
#+RESULTS: output2
8
6
More complex example. By defining org_rows and org_columns in the
preface, it makes it easier to manipulate them. org-babel implements
tables as a reference to an array of references to arrays.
#+name: rip(data=patito)
#+begin_src perl :results output
my $rows = org_rows($data);
my $columns = org_columns($data);
for (my $j=0;$j<$rows; $j++) {
for (my $i=0;$i<$columns; $i++) {
print "$i:$j ";
print $$data[$j][$i];
print " ;"
}
print "|\n";
}
print "Row $rows\n";
print "Columns $columns\n";
#+end_src
#+RESULTS: rip
#+begin_example
0:0 Taxi Driver (1944) ;1:0 Taxi Driver ;2:0 1944 ;3:0;4:0;5:0
;|
0:1 Taxi Driver (1954) ;1:1 Taxi Driver ;2:1 1954 ;3:1;4:1;5:1
;|
0:2 Taxi Driver (1973) ;1:2 Taxi Driver ;2:2 1973 ;3:2;4:2;5:2
;|
0:3 Taxi Driver (1976) ;1:3 Taxi Driver ;2:3 1976 ;3:3;4:3;5:3
;|
0:4 Taxi Driver (1978) ;1:4 Taxi Driver ;2:4 1978 ;3:4;4:4;5:4
;|
0:5 Taxi Driver (1981) ;1:5 Taxi Driver ;2:5 1981 ;3:5;4:5;5:5
;|
0:6 Taxi Driver (1990) ;1:6 Taxi Driver ;2:6 1990 ;3:6;4:6;5:6
;|
0:7 Taxi Driver (2004) ;1:7 Taxi Driver ;2:7 2004 ;3:7;4:7;5:7
;|
Row 8
Columns 6
#+end_example
==
diff --git a/lisp/ob-perl.el b/lisp/ob-perl.el
index ccd3826..65e6b88 100644
--- a/lisp/ob-perl.el
+++ b/lisp/ob-perl.el
@@ -62,7 +62,7 @@ This function is called by `org-babel-execute-src-block'."
"Return list of perl statements assigning the block's variables."
(mapcar
(lambda (pair)
- (format "$%s=%s;"
+ (format "my $%s=%s;"
(car pair)
(org-babel-perl-var-to-perl (cdr pair
(mapcar #'cdr (org-babel-get-header params :var
@@ -85,13 +85,34 @@ specifying a var of the same value."
(defvar org-babel-perl-wrapper-method
"
+%s
sub main {
%s
}
-@r = main;
+my @r = main;
open(o, \">%s\");
print o join(\"\\n\", @r), \"\\n\"")
+(defvar org-babel-perl-preface
+ "
+use strict;
+
+sub org_columns
+{
+my ($table) = @_;
+my $y = $$table[0];
+return scalar(@$y);
+}
+
+sub org_rows
+{
+my ($table) = @_;
+return scalar(@$table);
+}
+
+")
+
+
(defvar org-babel-perl-pp-wrapper-method
nil)
@@ -102,11 +123,11 @@ of the statements in BODY, if RESULT-TYPE equals 'value
then
return the value of the last statement in BODY, as elisp."
(when session (error "Sessions are not supported for Perl"))
(case result-type
-(output (org-babel-eval org-babel-perl-command body))
+(output (org-babel-eval org-babel-perl-command (format "%s\n%s"
org-babel-perl-preface body)))
(value (let ((tmp-file (org-babel-temp-file "perl-")))
(org-babel-eval
org-babel-perl-command
- (format org-babel-perl-wrapper-method body
+ (format org-babel-perl-wrapper-method org-babel-perl-preface body
(org-babel-process-file-name tmp-file 'noquote)))
(org-babel-eval-read-file tmp-file)
--
Daniel M. German "In questions of science the authority of
a thousand is not worth the humble
Galileo -> reasoning of a single individual."
http://turingmachine.org/
http://silvernegative.com/
dmg (at) uvic (dot) ca
replace (at) with @ and (dot) wi
Re: [O] bug in expansion of variables in babel Perl
Mm, I also noticed that when :results output is used, there is no way to insert perl code before or after the executed code. org-babel-perl-wrapper-method only works for all the methods but output. It would be nice to have a variable that does this for any output type. --dmg On Sun, Feb 24, 2013 at 1:16 AM, D M German wrote: > > Hi Everybody, > > I found a bug in the Babel perl code. When a table is used as input, the > values of the table are not escaped. In fact, they are surrounded by > double quotes " instead of single ones '. This means that special > characters are interpreted: $, and @ variables. See below. > > --dmg --- Daniel M. German http://turingmachine.org
[O] bug in expansion of variables in babel Perl
Hi Everybody,
I found a bug in the Babel perl code. When a table is used as input, the
values of the table are not escaped. In fact, they are surrounded by
double quotes " instead of single ones '. This means that special
characters are interpreted: $, and @ | jon
#+name: output(data=patito)
#+begin_src perl :results output
print "Begin\n";
print $$data[0][0], "\n";
print "End\n";
#+end_src
#+RESULTS: output
: Begin
: Jon
: End
--
I see two ways to solve this. The first is simply to replace the output
format of the variable from "%S" to "'%s'" (use quotes '). The other one
is to optionally escape the fields of the table (which is more
complicated, and would require replacing each). The third one is a
combination of both: replace them only if desired, via some header
configuration variable.
diff --git a/lisp/ob-perl.el b/lisp/ob-perl.el
index ccd3826..2f795aa 100644
--- a/lisp/ob-perl.el
+++ b/lisp/ob-perl.el
@@ -75,7 +75,7 @@ The elisp value, VAR, is converted to a string of perl source
code
specifying a var of the same value."
(if (listp var)
(concat "[" (mapconcat #'org-babel-perl-var-to-perl var ", ") "]")
-(format "%S" var)))
+(format "'%s'" var)))
Debugging perl is very cumbersome in org-mode. It would be nice to have
a feature to export the source to a file. This is because the variable
expansion needs to be done before the code can be used (hence simply cut
and paste does not work, nor shell-command-on-region)
I used the org-babel-perl-command variable to replace perl with a script
that simply wrote to a file.
It would be nice to be able to write the script created by org a file,
so this can be debugged (it would have the variable definitions). Maybe
this is already a feature and I don't know about it.
As we are into it, I found this declaration to be very useful.
--
(setq org-babel-perl-wrapper-method
"
use strict;
sub org_columns
{
my ($table) = @_;
my $y = $$table[0];
return scalar(@$y);
}
sub org_rows
{
my ($table) = @_;
return scalar(@$table);
}
sub main {
%s
}
my @r = main;
open(o, \">%s\");
print o join(\"\\n\", @r), \"\\n\"")
--
It does two things: it uses strict, so undeclared variables create
errors, and it also creates two functions: org_columns and org_rows
that, when used on the variable declared as input, return its number of
columns and rows:
my $rows = org_rows($data);
my $columns = org_columns($data);
the only problem with using strict is that variables would have to be
defined with "my" too: so that would require this patch:
diff --git a/lisp/ob-perl.el b/lisp/ob-perl.el
index ccd3826..82f8086 100644
--- a/lisp/ob-perl.el
+++ b/lisp/ob-perl.el
@@ -62,7 +62,7 @@ This function is called by `org-babel-execute-src-block'."
"Return list of perl statements assigning the block's variables."
(mapcar
(lambda (pair)
- (format "$%s=%s;"
+ (format "my $%s=%s;"
(car pair)
(org-babel-perl-var-to-perl (cdr pair
(mapcar #'cdr (org-babel-get-header params :var
Finally, if interested, i can write a couple of examples for Perl that
could help people who want to use it.
thanks again,
--
Daniel M. German "Great algorithms are
Francis Sullivan -> the poetry of computation"
http://turingmachine.org/
http://silvernegative.com/
dmg (at) uvic (dot) ca
replace (at) with @ and (dot) with .
