Re: [PATCH] Add support for tables in Calc src block :var

[Fraga, Eric]

On Saturday, 30 Mar 2024 at 18:34, Visuwesh wrote:
>> Specifically, I would love to make reference to calc variables,
>> especially those defined using embedded calc, in org tables.
>
> Can you please provide a minimal example for me to play around with?  I
> realise I would like something like this too [*] but I don't know
> concretely what this would/should look like.

So, embedded calc processes expressions in any buffer, including org
mode, which might look like this:

x := 3

y := 5

z := 3 x - y => 4

where, in this case, the value of z has been determined by calc and the
answer given after the => in the line.  The beauty of embedded calc is
you can change the value of x and the subsequent expressions will be
updated automatically (well, with C-x * u).

I would then love to be able to have a table that would allow me to
include the value of any variable, e.g. z above, something like

| var | value |
|-+---|
| x   | 3 |
| z   | 4 |

where the values in the second column are obtained by querying Calc.

-- 
: Eric S Fraga, with org release_9.6.23-1314-g945046 in Emacs 30.0.50

Re: [PATCH] Add support for tables in Calc src block :var

[Visuwesh]

[திங்கள் மார்ச் 18, 2024] Fraga, Eric wrote:

Good day, Eric!

> Thank you for this.  Potentially very useful.
>
> As you have managed to understand calc internals (to a much greater
> degrees than I have ever managed), do you know if there is any way to go
> the other way?  Specifically, I would love to make reference to calc
> variables, especially those defined using embedded calc, in org tables.

Can you please provide a minimal example for me to play around with?  I
realise I would like something like this too [*] but I don't know
concretely what this would/should look like.



[*] More specifically, I want to pass around src block evaluation
results in a table formula.

Re: [PATCH] Add support for tables in Calc src block :var

[Fraga, Eric]

On Monday, 18 Mar 2024 at 20:59, Visuwesh wrote:
> You give me far too credit: I merely placed an edebug trigger for
> calc-push-list and used a simple-minded ' [1,2,3;4,5,6] RET to figure
> out the vector format (then later I found the commentary).

Cute!

> If I get the time, I will try to look into your request.  Unfortunately,
> I cannot promise anything since I am near the end of my semester making
> me annoyingly busy.

That's perfectly fine!  I'm in no rush; it's just an itch that needs
scratching. :-)

Thank you,
eric

-- 
Eric S Fraga, https://profiles.ucl.ac.uk/5958-eric-fraga

Re: [PATCH] Add support for tables in Calc src block :var

[Visuwesh]

[திங்கள் மார்ச் 18, 2024] Fraga, Eric wrote:

> Thank you for this.  Potentially very useful.
>
> As you have managed to understand calc internals (to a much greater
> degrees than I have ever managed), do you know if there is any way to go
> the other way?  Specifically, I would love to make reference to calc
> variables, especially those defined using embedded calc, in org tables.

You give me far too credit: I merely placed an edebug trigger for
calc-push-list and used a simple-minded ' [1,2,3;4,5,6] RET to figure
out the vector format (then later I found the commentary).

If I get the time, I will try to look into your request.  Unfortunately,
I cannot promise anything since I am near the end of my semester making
me annoyingly busy.
[ I would like to make ob-calc turn matrices into tables in the result:
  like what ob-fortran and others already do.  ]

> Thanks again,
> eric

Re: [PATCH] Add support for tables in Calc src block :var

[Fraga, Eric]

Thank you for this.  Potentially very useful.

As you have managed to understand calc internals (to a much greater
degrees than I have ever managed), do you know if there is any way to go
the other way?  Specifically, I would love to make reference to calc
variables, especially those defined using embedded calc, in org tables.

Thanks again,
eric

-- 
Eric S Fraga, https://profiles.ucl.ac.uk/5958-eric-fraga

Re: [PATCH] Add support for tables in Calc src block :var

[Visuwesh]

[சனி மார்ச் 16, 2024] Ihor Radchenko wrote:

> Visuwesh  writes:
>
>> Subject: [PATCH] ob-calc.el: Add support for tables in Calc source block :var
>
> Thanks!
> Applied, onto main.
> I modified the patch to enable ob-calc tests by default.
> https://git.savannah.gnu.org/cgit/emacs/org-mode.git/commit/?id=89b0773c3

Thanks!

>>> Would you mind using `org-test-with-temp-text' instead of
>>> `org-test-at-id' as much as possible? Otherwise, looking at tests like
>>
>> OK, I wasn't sure what test style to use since the README in testing/
>> does not talk about writing new tests.  Now I have adapted the tests to
>> use org-test-with-temp-text.
>
> Yeah. We do not have a dedicated style document for Org mode tests.

Re: [PATCH] Add support for tables in Calc src block :var

[Ihor Radchenko]

Visuwesh  writes:

> Subject: [PATCH] ob-calc.el: Add support for tables in Calc source block :var

Thanks!
Applied, onto main.
I modified the patch to enable ob-calc tests by default.
https://git.savannah.gnu.org/cgit/emacs/org-mode.git/commit/?id=89b0773c3

>> Would you mind using `org-test-with-temp-text' instead of
>> `org-test-at-id' as much as possible? Otherwise, looking at tests like
>
> OK, I wasn't sure what test style to use since the README in testing/
> does not talk about writing new tests.  Now I have adapted the tests to
> use org-test-with-temp-text.

Yeah. We do not have a dedicated style document for Org mode tests.

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at .
Support Org development at ,
or support my work at 

Re: [PATCH] Add support for tables in Calc src block :var

[Visuwesh]

[சனி மார்ச் 16, 2024] Ihor Radchenko wrote:

> Visuwesh  writes:
>
>> Attached patch adds support for passing tables as matrices (or vector)
>> to a Calc source block through the :var parameter.  There might be a
>> better way to do it than manually construct the data structure expected
>> by Calc for a matrix but given that it rarely sees changes in this area,
>> it shouldn't be a bother.
>> I also added tests for this feature, and other simple tests to ensure
>> everything works.  I checked that all the tests passed on my side.
>
> Thanks!
> Since you are adding a new feature, may you also add an entry to etc/ORG-NEWS?

Now done.

>> -   (calc-push-list (list (cdr pair)))
>> +   (let ((val (cdr pair)))
>> + (calc-push-list
>> +  (list (if (listp val)
>> +(cons 'vec
>> +  (if (null (cdr val))
>> +  (car val)
>> +(mapcar (lambda (x) (if (listp x) (cons 'vec x) 
>> x))
>> +val)))
>> +  val
>
> It would be nice to add code comments explaining the Calc's internal
> format. Ideally, with references to Calc's manual or source code.
> Otherwise, this code looks like black magic :)

I hope the comment I added in the attached patch is clear enough.

>> +++ b/testing/examples/ob-calc-test.org
>> @@ -0,0 +1,57 @@
>> +#+TITLE: Tests for ob-calc
>> +#+OPTIONS: ^:nil
>
> Thanks a lot for adding many tests!
> Would you mind using `org-test-with-temp-text' instead of
> `org-test-at-id' as much as possible? Otherwise, looking at tests like

OK, I wasn't sure what test style to use since the README in testing/
does not talk about writing new tests.  Now I have adapted the tests to
use org-test-with-temp-text.

>From 29962bc3ec33e1e25f83f153b681d49182368592 Mon Sep 17 00:00:00 2001
From: Visuwesh 
Date: Sat, 16 Mar 2024 17:04:14 +0530
Subject: [PATCH] ob-calc.el: Add support for tables in Calc source block :var

A table with MxN dimensions is converted to a MxN matrix when given in
:var to a Calc source block.  A table with a single row is converted
to a vector (i.e., row vector).

* lisp/ob-calc.el (org-babel-execute-src-block:calc): Construct the
right data structure to pass tables as matrices to Calc.
* testing/lisp/test-ob-calc.el: Add tests for ob-calc, and this new
feature.
* etc/ORG-NEWS: Announce the feature.
---
 etc/ORG-NEWS |   7 +++
 lisp/ob-calc.el  |  14 -
 testing/lisp/test-ob-calc.el | 115 +++
 3 files changed, 135 insertions(+), 1 deletion(-)
 create mode 100644 testing/lisp/test-ob-calc.el

diff --git a/etc/ORG-NEWS b/etc/ORG-NEWS
index ca73f06e7..197d7503d 100644
--- a/etc/ORG-NEWS
+++ b/etc/ORG-NEWS
@@ -1121,6 +1121,13 @@ Maxima's graphics packages (~draw~ or ~plot~); the default remains
 ~plot~.  The graphics terminal is now determined from the file-ending
 of the file-name set in the ~:file~ header argument.
 
+*** =ob-calc.el=: Support for tables in ~:var~
+
+=ob-calc= now supports tables in ~:var~.  They are converted to a
+matrix or a vector depending on the dimensionality of the table.  A
+table with a single row is converted to a vector, the rest are
+converted to a matrix.
+
 *** Images and files in clipboard can be pasted
 
 Org asks the user what must be done when pasting images and files
diff --git a/lisp/ob-calc.el b/lisp/ob-calc.el
index f36df77ff..810ed1735 100644
--- a/lisp/ob-calc.el
+++ b/lisp/ob-calc.el
@@ -64,7 +64,19 @@
 	 (var-names (mapcar #'symbol-name org--var-syms)))
 (mapc
  (lambda (pair)
-   (calc-push-list (list (cdr pair)))
+   (let ((val (cdr pair)))
+ (calc-push-list
+  ;; For a vector, Calc follows the format (vec 1 2 3 ...)  so
+  ;; a matrix becomes (vec (vec 1 2 3) (vec 4 5 6) ...).  See
+  ;; the comments in "Arithmetic routines." section of
+  ;; calc.el.
+  (list (if (listp val)
+(cons 'vec
+  (if (null (cdr val))
+  (car val)
+(mapcar (lambda (x) (if (listp x) (cons 'vec x) x))
+val)))
+  val
(calc-store-into (car pair)))
  vars)
 (mapc
diff --git a/testing/lisp/test-ob-calc.el b/testing/lisp/test-ob-calc.el
new file mode 100644
index 0..6d6ca104d
--- /dev/null
+++ b/testing/lisp/test-ob-calc.el
@@ -0,0 +1,115 @@
+;;; test-ob-calc.el --- tests for ob-calc.el -*- lexical-binding: t; -*-
+
+;; Copyright (C) 2024  Visuwesh
+
+;; Author: Visuwesh 
+
+;; This program is free software; you can redistribute it and/or modify
+;; it under the terms of the GNU General Public License as published by
+;; the Free Software Foundation, either version 3 of the License, or
+;; (at your option) any later version.
+
+;; This program is distributed in the hope that it will be useful,
+;; but WITHO

Re: [PATCH] Add support for tables in Calc src block :var

[Ihor Radchenko]

Visuwesh  writes:

> Attached patch adds support for passing tables as matrices (or vector)
> to a Calc source block through the :var parameter.  There might be a
> better way to do it than manually construct the data structure expected
> by Calc for a matrix but given that it rarely sees changes in this area,
> it shouldn't be a bother.
> I also added tests for this feature, and other simple tests to ensure
> everything works.  I checked that all the tests passed on my side.

Thanks!
Since you are adding a new feature, may you also add an entry to etc/ORG-NEWS?

> -   (calc-push-list (list (cdr pair)))
> +   (let ((val (cdr pair)))
> + (calc-push-list
> +  (list (if (listp val)
> +(cons 'vec
> +  (if (null (cdr val))
> +  (car val)
> +(mapcar (lambda (x) (if (listp x) (cons 'vec x) 
> x))
> +val)))
> +  val

It would be nice to add code comments explaining the Calc's internal
format. Ideally, with references to Calc's manual or source code.
Otherwise, this code looks like black magic :)

> +++ b/testing/examples/ob-calc-test.org
> @@ -0,0 +1,57 @@
> +#+TITLE: Tests for ob-calc
> +#+OPTIONS: ^:nil

Thanks a lot for adding many tests!
Would you mind using `org-test-with-temp-text' instead of
`org-test-at-id' as much as possible? Otherwise, looking at tests like

> +(ert-deftest ob-calc/simple-program-mult ()
> +  "Test of simple multiplication."
> +  (org-test-at-id "40e4cd26-fe15-45c0-938b-111e021a5a99"
> +(org-babel-next-src-block)
> +(should (equal "2" (org-babel-execute-src-block)

it is very difficult to understand what exactly the tests is checking.
`org-test-with-temp-text' would make things a lot more readable as we
would see the calc code being tested right inside the test code.

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at .
Support Org development at ,
or support my work at