Re: help: howto aggregate several properties in one column
Two quick-and-dirty workarounds: 1. Eval that prior to using propview: (setq TEST_A "<>" TEST_B "<>" TEST_C "<>" TEST_D "<>") 2. Define this helper macro: (defmacro valof (symbol) `(if (boundp ',symbol) ,symbol "()")) Then specify columns like that: (concat (valof TEST_A) (valof TEST_B)) Have fun Thierry On 24-08-11 10:49, Pedro wrote: Hi, I am a heavy user of propview [1] and I am very happy with it, but looks like now I reached a limitation, or maybe someone founds a magical workaround. I am open to use another solution outside of propview. Find attached in file aggregate-props.org that serves as a playground environment and as an example of what I am struggling with, it is posed as a generic example: I want to "merge" two properties into one, no matter if the property exists or no. I am trying to use concat, concat works only when the properties on an item exist, if one of them does not exist, fails, which is unfortunate to what I am trying to achieve, because on one side, I would like to report (with a propview table) the frequency of appearance of each test_a, test_b, test_c, etc. on all nodes with tag :test:, and with another propview table, a summary of all test properties. There is a workaround based on a precalculation strategy that I kind of hate: do an org-map-entries of all targeted items with tag "test", and update SUMMARY property through an adhoc elisp function all the properties related to test (I am happy with just enumerating all of the involved subproperties, but I recognize, having a regex on property such as test_* or that "starts with" test, would be amazing), then, it is just as easy as adding the new column SUMMARY, but I would like to avoid this solution Extra note: related to propview, it is not a problem to concat 100 elements, because you can do it in an extra shortnamed function, so the column name can be short So this is what happens, concat of TEST_A and TEST_B is 0 #+BEGIN: propview :scope tree :match "+test" :noquote all :cols (ITEM CREATED TEST_A TEST_B (concat TEST_A TEST_B) (concat TEST_C TEST_D)) | ITEM | CREATED | TEST_A | TEST_B | (concat TEST_A TEST_B) | (concat TEST_C TEST_D) | |-+++++| | mytest | [2024-08-09 Fri 23:53] | a | 0 | 0 | cd | | mytest2 | [2024-08-09 Fri 23:53] | 0 | b | 0 | cd | | mytest3 | [2024-08-09 Fri 23:53] | 0 | 0 | 0 | 0 | |-+++++| | | | | | | | #+END: What I would expect to happen is that in the concat result of TEST_A and TEST_B would appear a or b, maybe that could be done with another function? #+BEGIN: propview :scope tree :match "+test" :noquote all :cols (ITEM CREATED TEST_A TEST_B (concat TEST_A TEST_B) (concat TEST_C TEST_D)) | ITEM | CREATED | TEST_A | TEST_B | (concat TEST_A TEST_B) | (concat TEST_C TEST_D) | |-+++++| | mytest | [2024-08-09 Fri 23:53] | a | 0 | a | cd | | mytest2 | [2024-08-09 Fri 23:53] | 0 | b | b | cd | | mytest3 | [2024-08-09 Fri 23:53] | 0 | 0 | 0 | 0 | |-+++++| | | | | | | | #+END: Cheers, pedeb [1] https://orgmode.org/worg/org-contrib/org-collector.html
[PATCH] org-table-to-lisp enhanced
Here is a new version of `org-table-to-lisp'. Why a new version? - Because it is more than 3x faster than the current one. Bench-marked on a 3822 rows by 16 columns Org table - current: 0.405 seconds - new one: 0.122 seconds on Emacs 30.0.50 with native compilation The speedup is achieved with or without native compilation. - Because it does not use regexps anymore. It uses `following-char' instead of `re-search-forward' Therefore the global regexp state is no longer clobbered. - Because after a 120x speedup in May 2020 (by Nicolas Goaziou and myself) it is funny to discover yet another way to a 3x speedup. - Because org-table-to-lisp is a basic building block of Org Mode, used in - table alignment - table export - Babel - plotting - table transposition - Because this version is a drop-in replacement. The overall algorithm does not changed. I ran all the standard unit tests of Org Mode as of [2024-02-24]. All went as expected. Have fun Thierry From 21b3e325e73d4a7332631eeed93cb2a025e024fd Mon Sep 17 00:00:00 2001 From: Thierry Banel Date: Sat, 24 Feb 2024 17:31:46 +0100 Subject: [PATCH] org-table.el: Enhanced table parsing * lisp/org-table.el (org-table-to-lisp): Refactored. * etc/ORG-NEWS: Document changes. `org-table-to-lisp' is significantly faster. It no longer uses regexps, nor clobbers the global regexp state. --- etc/ORG-NEWS | 6 ++ lisp/org-table.el | 37 - 2 files changed, 30 insertions(+), 13 deletions(-) diff --git a/etc/ORG-NEWS b/etc/ORG-NEWS index b79f275c4..a40a0f226 100644 --- a/etc/ORG-NEWS +++ b/etc/ORG-NEWS @@ -1275,6 +1275,12 @@ For symmetry with =\S= and =\sect= for the section symbol, =\P= has been added as an another form for the pilcrow symbol currently available as =\para=. +*** ~org-table-to-lisp~ no longer clobbers the regexp global state + +It does no longer use regexps. + +It is also faster. Large tables can be read quickly. + * Version 9.6 ** Important announcements and breaking changes diff --git a/lisp/org-table.el b/lisp/org-table.el index c6e16c1fd..67fffc23e 100644 --- a/lisp/org-table.el +++ b/lisp/org-table.el @@ -5494,25 +5494,36 @@ for a horizontal separator line, or a list of field values as strings. The table is taken from the parameter TXT, or from the buffer at point." (if txt (with-temp-buffer + (buffer-disable-undo) (insert txt) (goto-char (point-min)) (org-table-to-lisp)) (save-excursion (goto-char (org-table-begin)) - (let ((table nil)) -(while (re-search-forward "\\=[ \t]*|" nil t) - (let ((row nil)) - (if (looking-at "-") - (push 'hline table) - (while (not (progn (skip-chars-forward " \t") (eolp))) - (push (buffer-substring - (point) - (progn (re-search-forward "[ \t]*\\(|\\|$\\)") - (match-beginning 0))) - row)) - (push (nreverse row) table))) + (let (table) +(while (progn (skip-chars-forward " \t") + (eq (following-char) ?|)) + (forward-char) + (push + (if (eq (following-char) ?-) + 'hline + (let (row) + (while (progn +(skip-chars-forward " \t") +(not (eolp))) + (let ((q (point))) + (skip-chars-forward "^|\n") + (goto-char +(prog1 +(let ((p (point))) + (unless (eolp) (setq p (1+ p))) + p) + (skip-chars-backward " \t" q) + (push (buffer-substring-no-properties q (point)) row) + (nreverse row))) + table) (forward-line)) -(nreverse table) + (nreverse table) (defun org-table-collapse-header (table &optional separator max-header-lines) "Collapse the lines before `hline' into a single header. -- 2.34.1
Re: How to use mpirun with C or C++ Org-babel?
On 23-12-08 00:10, Edgar Lux wrote: Hello, I found [1][2] that it is possible to change the compiler for C and C++ source blocks. I would like to know if there is a way to add =mpirun -np 2= (where 2 can be any other number) to run the resulting executable. Thanks! [1] https://orgmode.org/worg//org-contrib/babel/languages/ob-doc-C.html [2] https://github.com/gheber/literate-hdf5 I guess you want to run the executable after it has been compiled, with a command like this one: mpirun -np 2 /tmp/babel-ad2pdk/C-bin-JTvjS4 Currently it is not possible. There is the :cmdline specifier which adds parameters to the executable, like this: #+begin_src C++ :cmdline AAA BBB CCC int main (int nargs, char** argv) { // argv will be { // "/tmp/babel-ad2pdk/C-bin-JTvjS4", // "AAA", // "BBB", // "CCC" // } } #+end_src Those parameters come AFTER the executable, whereas mpirun should come BEFORE the executable. An extension may be written in ob-C.el, in the org-babel-C-execute function. The relevant line is: (concat tmp-bin-file cmdline) If you want to contribute...
Re: [ANN] orgtbl-fit
On 3/3/23 16:13, Ihor Radchenko wrote: BTW, the dollar replacement is something org-table can benefit from---a number of people are confused because "$" is treated specially by Calc. I'm not sure what you mean. The dollar in spreadsheet formulas? Like: #+TBLFM: $6=$5+1 Which means that I misread the sources. I was referring to | 2$ | 3$| #ERROR | #+tblfm: $3=$2+$1 Error in the above is because Calc handles "$" specially. Org or Calc was waiting for something after the $, as in the last row of this table: | | | sum | error message | |-+++-| | 2$ | 3$ | #ERROR | $'s not allowed in this context | | 2_ | 3_ | #ERROR | Expected a number | | 2€ | 3€ | #ERROR | Expected `)' | | 2㍐ | 3¥ | #ERROR | Expected `)' | | 2£ | 3£ | #ERROR | Expected `)' | | 2* | 3+ | #ERROR | Expected a number | | 2 | 3 | 5 | | | $2 | 3 | 6 | | | 2$2 | 3 | 9 | | #+tblfm: $3=$2+$1 We can, but it should be first and foremost added to GNU Calc. On Org side, we just need appropriate integration. Maintaining generic data analysis code in Org is out of Org's scope. Absolutely Although the latest Calc release seams to be 2.02f, dating back in January 1992. Has it reached perfection 31 years ago? No. It became a part of Emacs, AFAIU. New (small) things are being added to Calc as a part of Emacs development. Good! So, what you said makes sense: adding features to Calc, and then giving access to them from Org.
Re: [ANN] orgtbl-fit
On 2/20/23 11:50, Ihor Radchenko wrote: tbanelwebmin writes: Examples & documentation can be read here: https://github.com/tbanel/orgtblfit/blob/main/README.org Interesting. Could it be somehow integrated with TBLFM formulas? I imagine something like ? +?*year +?*passengers +?*(year-2016)*passengers , when set as a column value in table formula, to be auto-updated with actual coefficients upon re-calculating the table. ... We need to specify the target column ("consumption" in this example). Therefore, the formula could be something like that: $4 = fit (consumption = ? +?*year +?*passengers +?*(year-2016)*passengers) It would benefit from other spreadsheet features, like constants and remote references. Makes sense. On the development side, the TBLFM handling is already quite a big chunk of code. We must take care that such an additional feature do not add complexity and maintenance burden. >From point of view of org-table.el, adding fitting functionality is mostly delegating the work to GNU Calc. Org side is just parsing the formula and transforming it to appropriate Calc function call. Absolutely. From Org table to Calc and back to Org table is what the orgtbl-fit package does. Currently it is around 400 lines of Elisp and 700 lines of unit tests. So, given that the parsing is extended cleanly, I do not think that maintenance burden will increase all that much. It may even benefit from someone taking a fresh look and possibly refactoring org-table TBLFM parser. Most likely. Orgtbl-fit as-is It is also possible to include orgtbl-fit as-is into Org Mode core. It would sit side-by-side with the core without changing anything in its code and its unit-tests. IMHO, it is not sufficiently integrated with org-table.el facilities in its current state. I'm afraid that we will have code duplication if include orgtbl-fit as is. Yes. One of the benefits from a fresh look you were talking about, would be avoiding code duplication. BTW, the dollar replacement is something org-table can benefit from---a number of people are confused because "$" is treated specially by Calc. I'm not sure what you mean. The dollar in spreadsheet formulas? Like: #+TBLFM: $6=$5+1 Data-analysis toolkit - From a higher perspective, we could give a consistent data-analysis toolkit to Org Mode (and call it org-data-analysis.el). It would start with fitting, clustering & aggregation. Then, new algorithms would be added upon user requests. Of course, there should be an interest among Org Mode users for such a toolkit. We can, but it should be first and foremost added to GNU Calc. On Org side, we just need appropriate integration. Maintaining generic data analysis code in Org is out of Org's scope. Absolutely Although the latest Calc release seams to be 2.02f, dating back in January 1992. Has it reached perfection 31 years ago? Contributing to GNU Calc will also benefit GNU Calc users who don't use Org mode.
Re: [ANN] orgtbl-fit
On 1/26/23 11:35, Ihor Radchenko wrote: tbanelwebmin writes: Actually, orgtbl-fit is a bridge between Org Mode tables and Calc. By the way, Org Mode table spreadsheet capabilities are also a bridge with Calc. Examples & documentation can be read here: https://github.com/tbanel/orgtblfit/blob/main/README.org Interesting. Could it be somehow integrated with TBLFM formulas? I imagine something like ? +?*year +?*passengers +?*(year-2016)*passengers , when set as a column value in table formula, to be auto-updated with actual coefficients upon re-calculating the table. Hey! That's an awesome idea. Expanding on the idea - We need to specify the target column ("consumption" in this example). Therefore, the formula could be something like that: $4 = fit (consumption = ? +?*year +?*passengers +?*(year-2016)*passengers) It would benefit from other spreadsheet features, like constants and remote references. On the development side, the TBLFM handling is already quite a big chunk of code. We must take care that such an additional feature do not add complexity and maintenance burden. Orgtbl-fit as-is It is also possible to include orgtbl-fit as-is into Org Mode core. It would sit side-by-side with the core without changing anything in its code and its unit-tests. Data-analysis toolkit - From a higher perspective, we could give a consistent data-analysis toolkit to Org Mode (and call it org-data-analysis.el). It would start with fitting, clustering & aggregation. Then, new algorithms would be added upon user requests. Of course, there should be an interest among Org Mode users for such a toolkit.
Re: [ANN] orgtbl-fit
Hi Ihor & the List GnuPlot is a great software. If you feel confortable with it, continue using it! If others are used to R or Python, that is fine too. Orgtbl-fit may be useful if: * - You want a pure Emacs process, without external dependencies. * - You know that Emacs-Calc can fit your data, but you are not familiar with it. * - Your data is already available as an Org Mode table. * - You don't want to write a script (just point at the target column and type M-x orgtbl-fit, that's all). Actually, orgtbl-fit is a bridge between Org Mode tables and Calc. By the way, Org Mode table spreadsheet capabilities are also a bridge with Calc. Examples & documentation can be read here: https://github.com/tbanel/orgtblfit/blob/main/README.org Have fun! On 1/24/23 20:55, Ihor Radchenko wrote: tbanelwebmin writes: The new orgtbl-fit package has just been released on Melpa. It does regression fitting on Org Mode tables. Example. We suspect that `obs' depends on `x' and `y'. ... Let us put the cursor on the `obs' column, and type M-x orgtbl-fit Two columns are added - predicted obs column - difference between obs and predicted | x | y | obs | Best Fit | Fit Diff | |+---+--+--+--| | 32 | 7 | 38.3 | 38.2 | -0.1 | | 18 | 3 | 11.4 | 11.6 | 0.2 | | 43 | 9 | 47.3 | 47.2 | -0.1 | | 11 | 2 | 8.9 | 8.7 | -0.2 | | 35 | 8 | 45.1 | 45.3 | 0.2 | #+TBLFM: $4=-0.289267886829 - 1.06613976706*$1 + 10.3668885192*$2; %.1f::$5=$4-$3; %.1f Are there situations when this package is actually useful for data analysis? (I am usually using gnuplot for fitting)
[ANN] orgtbl-fit
Hi the List! The new orgtbl-fit package has just been released on Melpa. It does regression fitting on Org Mode tables. Example. We suspect that `obs' depends on `x' and `y'. | x | y | obs | |+---+--| | 32 | 7 | 38.3 | | 18 | 3 | 11.4 | | 43 | 9 | 47.3 | | 11 | 2 | 8.9 | | 35 | 8 | 45.1 | Let us put the cursor on the `obs' column, and type M-x orgtbl-fit Two columns are added - predicted obs column - difference between obs and predicted | x | y | obs | Best Fit | Fit Diff | |+---+--+--+--| | 32 | 7 | 38.3 | 38.2 | -0.1 | | 18 | 3 | 11.4 | 11.6 | 0.2 | | 43 | 9 | 47.3 | 47.2 | -0.1 | | 11 | 2 | 8.9 | 8.7 | -0.2 | | 35 | 8 | 45.1 | 45.3 | 0.2 | #+TBLFM: $4=-0.289267886829 - 1.06613976706*$1 + 10.3668885192*$2; %.1f::$5=$4-$3; %.1f So we discovered that obs = -0.29 -1.07*x +10.37*y Behind the scene, the calcFunc-fit function from Emacs-Calc is called. Install through Melpa: M-x package-install orgtbl-fit Source and documentation here: https://github.com/tbanel/orgtblfit/blob/master/README.org Have fun Thierry
Re: [PATCH] Unit-test for Please add support for dlangs packagemanager to ob-C.el
For unit-testing Dlang, we could mock the «dub» command (compiler+packager). For instance, we could replace it by the «cat» Linux command, just while running the unit-test. But the «cat» command introduces a new dependency on Linux. Is there a portable equivalent for «cat»? On 10/2/22 09:29, Ihor Radchenko wrote: Max Nikulin writes: I am unsure concerning general policy related to org-babel test, so the following is rather discussion than a direct request to change the test. Is there a way to avoid dependency on remote resources? My concern is test results affected by network issues. In addition, more required 3rd party tools means less probability that a developer after modification of generic babel code noticed that the changes break D backend because missing tool on the development machine. I have no idea how much efforts is required to create a mock for isolated environment. Tests must not rely on remote resources. However, it does not mean that we cannot test remote resources in Org tests. One simply needs to mock the url-retrieve or whatever function is used to retrieve remove resource. Another point is that missed dub binary is not reflected in test report. If `skip-unless' feature of ert is available in Emacs-26 then it should be used. The current approach to optional binaries is throwing 'missing-test-dependency error when the binary is not available.
Re: Compiling a C++ source block
Good point Ihor! I will change that Le 29/09/2022 à 04:25, Ihor Radchenko a écrit : tbanelwebmin writes: You may try: #+header: :includes '("" "") I notice that https://orgmode.org/worg/org-contrib/babel/languages/ob-doc-C.html only has examples of a single library being using in :includes parameter. Although the text description does mention list of strings, it would be nice if there was an example using :includes with multiple libraries and/or local libraries.
Re: Please add support for dlangs packagemanager to ob-C.el
Thanks Christian. Your TINYCHANGE fits into the 15 lines limit. I will review it. As Bastien said, we follow as much as we can a predefined format for commit messages: https://orgmode.org/worg/org-contribute.html#commit-messages You may also look at past commit messages, for instance this one: https://git.savannah.gnu.org/cgit/emacs/org-mode.git/commit/?id=0a6a56c804099e358ca558119b7aede0b2b9b90f https://git.savannah.gnu.org/cgit/emacs/org-mode.git/log/ If you want to further contribute, maybe in another TINYCHANGE, automatic unit tests are very valuable. There are already examples of Dlang tests in: testing/examples/ob-C-test.org testing/lisp/test-ob-C.el The best Le 27/09/2022 à 22:41, Bastien a écrit : Hi Christian, thanks for the patch (you forgot to advertize it by adding [PATCH] in the subject.) Christian Köstlin writes: Please see the patch comment. I reworked my original patch to fit into the TINYPATCH category. I'm CC'ing Thierry as the maintainer of ob-C.el. The commit message should be reworked - see https://orgmode.org/worg/org-contribute.html#commit-messages I would recommend not using source blocks in the message. I hope Thierry will have time to review it. Thanks!
Re: Compiling a C++ source block
You may try: #+header: :includes '("" "") Also, you have this commented line: # #+header: :var table= It seems to break the #+header: chain Just remove it. By the way, to figure out future compilation problems, you may click on the error: /tmp/babel-eqG2i0/C-src-3zG2ec.cpp:16:1: error: unknown type name 'ex' This will open the source file actually being compiled. Have fun! Le 26/09/2022 à 21:13, Roger Mason a écrit : Hello, I wish to compile & run this source block in org-mode: #+name: gnc_cav #+header: :includes #+header: :libs -L/usr/local/lib -lginac -lcln #+header: :flags -std=c++17 #+header: :namespaces std GiNaC # #+header: :var table= #+header: :var job="vectors" :var vol=113.13115406160385926 :var r=1.0995524816924328312 #+header: :var a=4.916 :var c=5.4054 #+header: :main no #+begin_src C++ ex vol(symbol a, symbol c) { ex v = a*a * c * sin(60*Pi/180); return v; } ex coa(symbol a, symbol c) { ex covera = c/a; return covera; } ex scl(symbol a, symbol c, double ad, double cd) { // symbol v("v"); // symbol r("r"); ex v = vol(a,c); ex r = coa(a,c); numeric three(3); // ex s = power(v.subs(lst{a==ad,c==cd})),1/three)/(power(sin(60*Pi/180),1/three) * power(r.subs(lst{a==ad,c==cd}),1/three)); ex s = power(v.subs(lst{a==ad,c==cd}),1/three)/(power(sin(60*Pi/180),1/three) * power(r.subs(lst{a==ad,c==cd}),1/three)); return s; } ex scl(double vd, double rd) { numeric three(3); ex s = power(vd,1/three)/(power(sin(60*Pi/180),1/three) * power(rd,1/three)); return s; } int main () { if ( strncmp(job, "volume", 6) == 0) { // ./v volume a c, calculate volume symbol as("as"); symbol cs("cs"); ex v = vol(as, cs); cout << "Volume = " << v.subs(lst{as == a, cs == c}).evalf() << "\n"; ex r = coa(as, cs); cout << "c/a = " << r.subs(lst{as == a, cs == c}).evalf() << "\n"; ex a_calc = scl(as, cs, a, c); cout << "a_calc = " << a_calc.evalf() << "\n"; cout << "c_calc = " << a_calc.evalf() * r.subs(lst{as == a, cs == c}).evalf() << "\n"; } else if ( job,"vectors",5) == 0 ) { // ./v vectors vol c/a //double vd = stod(argv[2]); //double rd = stod(argv[3]); ex a_calc = scl(v,r); cout << "a_calc = " << a_calc.evalf() << "\n"; cout << "c_calc = " << a_calc.evalf() * rd << "\n"; //double a_scale = a; } // vectors } // main #+end_src When I attempt compilation using 'C-c C-c' I get: /tmp/babel-eqG2i0/C-src-3zG2ec.cpp:16:1: error: unknown type name 'ex' ex vol(symbol a, symbol c) and lots of similar errors. I infer this is because ginac.h is not found. I have tried specifying in ':includes' to no avail. I have tried specifying the full path to ginac.h: and specifying -I/usr/local/lib/ginac/ as part of ':includes', but the error persists. Maybe the C++ support is not set up to handle this use case, but perhaps I am missing something. Any help will be much appreciated. Thanks, Roger
Re: Babel C-mode corrupts double-quoted strings in output
This fix in ob-C.el passes all unit tests, as well as Martin's example, and some other examples. If someone get a regression, please tell me! In a few days, I will commit. The fix: in ob-C.el line 185, change: (org-babel-read results t) to: results Have fun Le 02/09/2022 à 13:02, tbanelwebmin a écrit : This looks like a bug in ob-C.el Around line 196 we should replace (org-babel-read results t) with results In this way, ob-C.el will look more like ob-shell.el Let me see what are the consequences with such a fix. Thanks Martin for investigating deep in the sources! Regards Le 31/08/2022 à 18:35, Martin Jerabek a écrit : Hi! I recently started to use Org Babel for C++ programs. One of the programs outputs several lines with double-quoted strings, similar to this: #+NAME: doublequotes_cpp #+begin_src cpp :includes :results output verbatim raw std::cout << "\"line 1\"\n"; std::cout << "\"line 2\"\n"; std::cout << "\"line 3\"\n"; #+end_src #+RESULTS: doublequotes_cpp line 1 As you can see, only the first line is copied to the RESULTS block, and it is stripped of the double quotes. I tracked down the problem to org-babel-read (in ob-core.el). org-babel-C-execute (in ob-C.el) calls this function with the output of the C++ program. The problem is the following line: ((eq (string-to-char cell) ?\") (read cell)) i.e. if the output of the program starts with a double quote, it is passed to read which reads only the first string and also removes the double quotes, resulting in the observed output. The original version of this piece of code was added with the following commit: commit 60a8ba556d682849eafb0f84e689967cd2965549 Author: Eric Schulte Date: Wed Mar 2 07:55:39 2011 -0700 ob: read string variable values wrapped in double quotes, removing the quotes * lisp/ob.el (org-babel-read): Read string variable values wrapped in double quotes, removing the quotes. AFAICT this modification was done in response to the email thread "[Orgmode] org-babel-read should have option NOT to interpret as elisp" started on 2011-02-27, more specifically the email on "Wed, 02 Mar 2011 07:56:45 -0700" from Eric Schulte. This was obviously done for parsing variables in the header line, not for the program output, but the Babel C mode uses org-babel-read also for the output. I assumed that ":results output verbatim raw" would prevent any postprocessing of the output but this is not the case for C mode. I am not sure how to fix this without breaking backward compatibility. I assume it should be fixed directly in org-babel-C-execute, not in a central function like org-babel-read to minimize the impact. Surprisingly (for me) the equivalent shell script works as expected: #+NAME: doublequotes #+begin_src shell :results output verbatim raw echo '"line 1"' echo '"line 2"' echo '"line 3"' #+end_src #+RESULTS: doublequotes "line 1" "line 2" "line 3" because org-babel-execute:shell does not process the output with org-babel-read. I do not know if languages other than the C family (C, C++, D) are affected. At the very least, the documentation of org-babel-read should be expanded to document the fact that if the CELL parameter starts with a double quote, it is processed by the read function. Best regards Martin Jerabek
Re: Babel C-mode corrupts double-quoted strings in output
This looks like a bug in ob-C.el Around line 196 we should replace (org-babel-read results t) with results In this way, ob-C.el will look more like ob-shell.el Let me see what are the consequences with such a fix. Thanks Martin for investigating deep in the sources! Regards Le 31/08/2022 à 18:35, Martin Jerabek a écrit : Hi! I recently started to use Org Babel for C++ programs. One of the programs outputs several lines with double-quoted strings, similar to this: #+NAME: doublequotes_cpp #+begin_src cpp :includes :results output verbatim raw std::cout << "\"line 1\"\n"; std::cout << "\"line 2\"\n"; std::cout << "\"line 3\"\n"; #+end_src #+RESULTS: doublequotes_cpp line 1 As you can see, only the first line is copied to the RESULTS block, and it is stripped of the double quotes. I tracked down the problem to org-babel-read (in ob-core.el). org-babel-C-execute (in ob-C.el) calls this function with the output of the C++ program. The problem is the following line: ((eq (string-to-char cell) ?\") (read cell)) i.e. if the output of the program starts with a double quote, it is passed to read which reads only the first string and also removes the double quotes, resulting in the observed output. The original version of this piece of code was added with the following commit: commit 60a8ba556d682849eafb0f84e689967cd2965549 Author: Eric Schulte Date: Wed Mar 2 07:55:39 2011 -0700 ob: read string variable values wrapped in double quotes, removing the quotes * lisp/ob.el (org-babel-read): Read string variable values wrapped in double quotes, removing the quotes. AFAICT this modification was done in response to the email thread "[Orgmode] org-babel-read should have option NOT to interpret as elisp" started on 2011-02-27, more specifically the email on "Wed, 02 Mar 2011 07:56:45 -0700" from Eric Schulte. This was obviously done for parsing variables in the header line, not for the program output, but the Babel C mode uses org-babel-read also for the output. I assumed that ":results output verbatim raw" would prevent any postprocessing of the output but this is not the case for C mode. I am not sure how to fix this without breaking backward compatibility. I assume it should be fixed directly in org-babel-C-execute, not in a central function like org-babel-read to minimize the impact. Surprisingly (for me) the equivalent shell script works as expected: #+NAME: doublequotes #+begin_src shell :results output verbatim raw echo '"line 1"' echo '"line 2"' echo '"line 3"' #+end_src #+RESULTS: doublequotes "line 1" "line 2" "line 3" because org-babel-execute:shell does not process the output with org-babel-read. I do not know if languages other than the C family (C, C++, D) are affected. At the very least, the documentation of org-babel-read should be expanded to document the fact that if the CELL parameter starts with a double quote, it is processed by the read function. Best regards Martin Jerabek
Re: how to transpose a table but not deleting the horizontal lines
The orgtbl-arggregate package (on Melpa and GitHub) does that. Horizontal lines are converted to empty columns. Transposing again recreates the original horizontal lines. Beware, as opposed to org-table-transpose-table-at-point, it does not modify the source table, it creates a new transposed table. Type C-c C-x x in an empty space in the buffer Then answer "transpose", and follow the wizard. It is a bonus, undocumented feature of the package. Le 10/05/2022 à 08:49, Uwe Brauer a écrit : Hi >From the docstring of org-table-transpose-table-at-point is Transpose Org table at point and eliminate hlines. Does anybody know about a, maybe, 3rd party packages that transpose the table but leaves the horizontal lines intact? Regards Uwe Brauer
Re: Please add support for dlangs packagemanager to ob-C.el
Thanks Christian for this contribution. Easily adding DLang dependencies while keeping a single code block seems quite useful. I will take a deeper look at your proposal. Also, you marked your patch as TINYCHANGE, which imply it is less than 15 lines long. However your patch is 24 lines long. This requires that you assign the copyright for your contributions to the FSF. Have you signed the assignment? More on this process here: https://orgmode.org/contribute.html The best Thierry ob-C.el maintainer Le 08/03/2022 à 23:47, Christian Köstlin a écrit : > I Hope the patch already contains enough background information why > its a nice feature to have in ob-C.el. > > Kind regards, > Christian >
Re: [PATCH] bad table formula recorded in some cases
Ok, Timothy, fair enough Le 21/07/2021 à 17:07, Timothy a écrit : > Hi Thierry, > > tbanelwebmin writes: >> I don't know the intention. But the answer may lie in the comment 4 >> lines above: >>;; Don't overwrite TBLFM, we might use text properties to >>;; store stuff. >> >> In this case, the intention would be to keep the original "#+TBLFM:" >> instead of inserting a fresh new one. >> >> But we are in the else branch of (if (looking-at ...)), which means >> there was no "#+TBLFM:". And no text properties to save. Therefore we >> may safely remove this (match-string 2). > Thank you for looking into this, I'm reassured by your inference that > this change is safe to make. I'm not really one of the main > contribution-acceptors/pushers though, so I'd rather leave this for > someone like Nicolas to sign off on. > > Would you mind bumping this thread in a few weeks if nothing happens? > > Hope that's not too much of an inconvenience, > > Timothy. > >> Le 21/07/2021 à 12:50, Timothy a écrit : >>> Hi Thierry, >>> >>> Thanks for this! Looking at the change you suggest, do you know why the >>> (match-string 2) bit might have been added in the first place? I'm just >>> wondering if there might be some edge-case adversely affected by this --- >>> hence trading one bug for another :P >>> >>> -- >>> Timothy >>> >>> tbanelwebmin writes: >>> >>>> Small bug, small fix. >>>> >>>> Suppose we have a table embedded in a begin-end block. >>>> >>>> #+begin: aaa :param value >>>> | a | b | >>>> | a | b | >>>> #+end: >>>> >>>> Suppose we want to add a formula, with C-c = >>>> We end up with an incorrect result: >>>> >>>> #+begin: aaa :param value >>>> | a | 33 | >>>> | a | b | >>>> :param value $2=33 >>>> #+end: >>>> >>>> The fix: in org-table.el, line 2177, change >>>> (insert (or (match-string 2) "#+TBLFM:"))) >>>> to >>>> (insert "#+TBLFM:")) >>>> >>>> Then we get the correct result: >>>> >>>> #+begin: aaa :param value >>>> | a | 33 | >>>> | a | b | >>>> #+TBLFM: $2=33 >>>> #+end: >>>> >>>> Why? Because (match-string 2) is supposed to refer to the (looking-at) >>>> instruction 7 lines above. But (match-string 2) is in the else branch, >>>> which means that (looking-at) failed. Therefore (match-string 2) returns >>>> garbage. >>>> >>>> Thanks to Uwe Brauer for pointing to this bug.
Re: [PATCH] bad table formula recorded in some cases
Hi Timothy I don't know the intention. But the answer may lie in the comment 4 lines above: ;; Don't overwrite TBLFM, we might use text properties to ;; store stuff. In this case, the intention would be to keep the original "#+TBLFM:" instead of inserting a fresh new one. But we are in the else branch of (if (looking-at ...)), which means there was no "#+TBLFM:". And no text properties to save. Therefore we may safely remove this (match-string 2). Thanks Timothy for taking this into account so carefully! Best Thierry Le 21/07/2021 à 12:50, Timothy a écrit : > Hi Thierry, > > Thanks for this! Looking at the change you suggest, do you know why the > (match-string 2) bit might have been added in the first place? I'm just > wondering if there might be some edge-case adversely affected by this --- > hence trading one bug for another :P > > -- > Timothy > > tbanelwebmin writes: > >> Small bug, small fix. >> >> Suppose we have a table embedded in a begin-end block. >> >> #+begin: aaa :param value >> | a | b | >> | a | b | >> #+end: >> >> Suppose we want to add a formula, with C-c = >> We end up with an incorrect result: >> >> #+begin: aaa :param value >> | a | 33 | >> | a | b | >> :param value $2=33 >> #+end: >> >> The fix: in org-table.el, line 2177, change >> (insert (or (match-string 2) "#+TBLFM:"))) >> to >> (insert "#+TBLFM:")) >> >> Then we get the correct result: >> >> #+begin: aaa :param value >> | a | 33 | >> | a | b | >> #+TBLFM: $2=33 >> #+end: >> >> Why? Because (match-string 2) is supposed to refer to the (looking-at) >> instruction 7 lines above. But (match-string 2) is in the else branch, >> which means that (looking-at) failed. Therefore (match-string 2) returns >> garbage. >> >> Thanks to Uwe Brauer for pointing to this bug.
[PATCH] bad table formula recorded in some cases
Small bug, small fix. Suppose we have a table embedded in a begin-end block. #+begin: aaa :param value | a | b | | a | b | #+end: Suppose we want to add a formula, with C-c = We end up with an incorrect result: #+begin: aaa :param value | a | 33 | | a | b | :param value $2=33 #+end: The fix: in org-table.el, line 2177, change (insert (or (match-string 2) "#+TBLFM:"))) to (insert "#+TBLFM:")) Then we get the correct result: #+begin: aaa :param value | a | 33 | | a | b | #+TBLFM: $2=33 #+end: Why? Because (match-string 2) is supposed to refer to the (looking-at) instruction 7 lines above. But (match-string 2) is in the else branch, which means that (looking-at) failed. Therefore (match-string 2) returns garbage. Thanks to Uwe Brauer for pointing to this bug.
Re: convert subtree or nested list to table
When I run you example I get: #+RESULTS: | Category | (unordered (A) (B) (C) (D) (F)) | | Writing | (unordered (great) (good) (ok) (lousy) (awful)) | To get your result I need to modify :results table to :results table code You may try :results table raw Or try your example without any hidden customisation emacs -q This info page may be useful: [[info:org#Results of Evaluation]] Le 07/07/2021 à 23:13, Matt Price a écrit : > I think this is exactly what I want (with just a little > moreprocessing). Thank you so much for the idea! > > I'm having a little bit of trouble getting the same output as you > though, and I'm wondering if there might be a setting that I need to > change. > > Here is what I tried, and the result. Do you have an idea of what is > going wrong here? > > Thank you! > > > > #+NAME:essay-rubric > - Category > - A > - B > - C > - D > - F > - Writing > - great > - good > - ok > - lousy > - awful > > #+begin_src emacs-lisp :var contents=essay-rubric :results table > contents > #+end_src > > #+RESULTS: > #+begin_src emacs-lisp > | (("Category" | > #+end_src > - > On Wed, Jul 7, 2021 at 6:29 AM tbanelwebmin <mailto:tbanelweb...@free.fr>> wrote: > > Hi Matt > > Le 05/07/2021 à 21:44, Matt Price a écrit : > > I have to write a number of text-heavy documents which need to be > > delivered as tables with wrapped paragraphs in most cells. Working > > directly in table format is pretty arduous and uncomfortable. Has > > anyone ever written a function to accept a list or subtree as input > > and process it into a table? > > > > If anyone has done something similar, I'd love some tips! > > Maybe you could use builtin Babel > Hereafter you have a starting point > - Give a name to your input Org list > - Process it with Emacs-Lisp (or whatever language you are comfortable > with) to output it as a table > > > self contained Org Mode example _ > > Example of a named list > #+NAME: BBB > - abc > + 123 > + 456 > - def > + red > + blue > - ghi > + big > + small > > Example of converting the named list into a table with Emacs-Lisp > #+begin_src elisp :var bbb=BBB :results table > bbb > #+end_src > > #+RESULTS: > | abc | (unordered (123) (456)) | > | def | (unordered (red) (blue)) | > | ghi | (unordered (big) (small)) | > ___ > >
Re: convert subtree or nested list to table
Hi Matt Le 05/07/2021 à 21:44, Matt Price a écrit : > I have to write a number of text-heavy documents which need to be > delivered as tables with wrapped paragraphs in most cells. Working > directly in table format is pretty arduous and uncomfortable. Has > anyone ever written a function to accept a list or subtree as input > and process it into a table? > > If anyone has done something similar, I'd love some tips! Maybe you could use builtin Babel Hereafter you have a starting point - Give a name to your input Org list - Process it with Emacs-Lisp (or whatever language you are comfortable with) to output it as a table self contained Org Mode example _ Example of a named list #+NAME: BBB - abc + 123 + 456 - def + red + blue - ghi + big + small Example of converting the named list into a table with Emacs-Lisp #+begin_src elisp :var bbb=BBB :results table bbb #+end_src #+RESULTS: | abc | (unordered (123) (456)) | | def | (unordered (red) (blue)) | | ghi | (unordered (big) (small)) | ___
Re: table: problem with nan and if
About the E format setting, you may look at documentation here: [[info:org#Formula syntax for Calc]] Le 22/06/2021 à 09:44, Eric S Fraga a écrit : > Uwe, > > what if you change the E in each equation with N? You'll get 0 entries > when the calculations involve all empty cells which might not be what > you want, of course.
Re: On using to-do lists efficiently
Todo lists... Of course it resonates. You describe the errors I do. I keep moving from to-do-list, to agenda, to notes, and back again. You say it is not about a tool or a process, but rather about discipline. Right! Discipline may be connected to habits. From 8:00 to 8:30 I (should) review my to-do-list Unfortunately, our lives are full of perturbations which invalidate our habits. Even though it is not about tools, I spent years looking for the right tool: - pen & paper - postit - electronic postit - note-pads, paper or electronic - countless PC & phone applications Eventually I settled on Emacs Org Mode. It was a refreshment when I discovered it. Thanks for sharing your thoughts. Note to my future self: enhance my to-do-list Le 26/04/2021 à 18:18, Bastien a écrit : > Slightly offtopic but I sat down this week-end trying to grasp with > very few words what I learned on how to use to-do lists efficiently > over the years, and here it is: > > https://bzg.fr/en/on-using-to-do-lists-efficiently/ > > Posted it on HN: https://news.ycombinator.com/item?id=26944239 > > I'm curious if this resonates with your experience. > > It took me quite long to appease this love-hate relationship I have > with my to-do lists. > > As usual: enjoy! > > >
Re: Bug: Incompatible return type in org-babel C when using a table of doubles with a header as a variable [9.4.4 (release_9.4.4-272-ga9f38b @ /home/richard/.emacs.d/straight/build/org/)]
I just pushed a patch. Your example works now. (Don't forget to change stdio.h into ).While waiting for the patch to be released, you can load this Lisp file: https://code.orgmode.org/bzg/org-mode/src/maint/lisp/ob-C.el Cheers Le 02/04/2021 à 19:40, Richard Sent via General discussions about Org-mode. a écrit : > When a table of floating pointer numbers that contains a header row is > set as a variable to a C source block, the generated header file > contains an invalid return type, causing the program to not compile. > > For example, if I have a table that looks like > > #+NAME: tbl-doubles-org-bug-report > #+begin_src C :includes :results table :colnames '("i" "val") > for (int i = 0; i < 5; i++) { > printf("%f %f\n", (double)i, i * 2.0); > } > #+end_src > > #+RESULTS: tbl-doubles-org-bug-report > | i | val | > |-+-| > | 0.0 | 0.0 | > | 1.0 | 2.0 | > | 2.0 | 4.0 | > | 3.0 | 6.0 | > | 4.0 | 8.0 | > > And then attempt to include that table as a variable called data > > #+begin_src C :includes stdio.h :var data=tbl-doubles-org-bug-report > printf("No errors!"); > #+end_src > > Nothing is printed, and *Org-Babel Error Output* will display > > /tmp/babel-NQHi51/C-src-lsA892.c: In function ‘data_h’: > /tmp/babel-NQHi51/C-src-lsA892.c:24:65: error: incompatible types when > returning type ‘double’ but ‘const char *’ was expected > 24 | const char* data_h (int row, const char* col) { return > data[row][get_column_num(2,data_header,col)]; } > | > ~^~~ > /bin/bash: line 1: /tmp/babel-NQHi51/C-bin-90Zk27: Permission denied > > If we look in the /tmp file mentioned, it's easy to see the > inconsistency. > > double data[5][2] = { > {0.00,0.00}, > {1.00,2.00}, > {2.00,4.00}, > {3.00,6.00}, > {4.00,8.00} > }; > const int data_rows = 5; > const int data_cols = 2; > int get_column_num (int nbcols, const char** header, const char* column) > { > int c; > for (c=0; c if (strcmp(header[c],column)==0) > return c; > return -1; > } > const char* data_header[2] = {"i","val"}; > const char* data_h (int row, const char* col) { return > data[row][get_column_num(2,data_header,col)]; } > int main() { > printf("No errors!"); > return 0; > } > > data_h() should not return a const char*, but a double. > > I've also tested this with $ emacs -Q and the issue is still present. It > does not occur if the table contains integer values instead of floats > or if a header row is omitted. Seeing as how the header row is processed > differently from the data, I don't think the presence of a header row is > meant to cause this error. > > > > Emacs : GNU Emacs 27.1 (build 1, x86_64-pc-linux-gnu, GTK+ Version > 3.24.22, cairo version 1.17.3) > of 2020-08-28 > Package: Org mode version 9.4.4 (release_9.4.4-272-ga9f38b @ > /home/richard/.emacs.d/straight/build/org/) > > current state: > == > (setq > org-src-mode-hook '(org-src-babel-configure-edit-buffer > org-src-mode-configure-edit-buffer doom-modeline-set-org-src-modeline) > org-link-shell-confirm-function 'yes-or-no-p > org-blank-before-new-entry '((heading . t) (plain-list-item . t)) > org-metadown-hook '(org-babel-pop-to-session-maybe) > org-clock-out-hook '(org-clock-remove-empty-clock-drawer) > org-html-format-inlinetask-function > 'org-html-format-inlinetask-default-function > org-odt-format-headline-function 'org-odt-format-headline-default-function > org-agenda-files '("~/org") > org-ascii-format-inlinetask-function 'org-ascii-format-inlinetask-default > org-startup-folded t > org-mode-hook '(#[0 "\301\211\207" [imenu-create-index-function > org-imenu-get-tree] 2] > #[0 "\300\301\302\303\304$\207" [add-hook change-major-mode-hook > org-show-all append local] 5] > #[0 "\300\301\302\303\304$\207" [add-hook change-major-mode-hook > org-babel-show-result-all append local] 5] > org-babel-result-hide-spec org-babel-hide-all-hashes turn-on-auto-fill > mixed-pitch-mode org-superstar-mode > highlight-parentheses-mode) > org-odt-format-drawer-function #[514 "\207" [] 3 "\n\n(fn NAME CONTENTS)"] > org-archive-hook '(org-attach-archive-delete-maybe) > org-confirm-elisp-link-function 'yes-or-no-p > org-agenda-before-write-hook '(org-agenda-add-entry-text) > org-metaup-hook '(org-babel-load-in-session-maybe) > org-bibtex-headline-format-function #[257 "\300\236A\207" [:title] 3 > "\n\n(fn ENTRY)"] > org-adapt-indentation nil > org-latex-format-drawer-function #[514 "\207" [] 3 "\n\n(fn _ CONTENTS)"] > org-babel-pre-tangle-hook '(save-buffer) > org-file-apps '(("\\.pdf\\'" . emacs) (auto-mode . emacs) (directory . > emacs) ("\\.mm\\'" . default) ("\\.x?html?\\'" . default) > ("\\.pdf\\'" . default)) > org-tab-first-hook '(org-babel-hide-result-toggle-maybe > org-babel-header-arg-expand) > org-hide-leading-stars t > org-babel-load-languages '((C . t) (emacs-lisp . t)) > org-log-done 'time > org-superstar-leading
Re: Bug: Incompatible return type in org-babel C when using a table of doubles with a header as a variable [9.4.4 (release_9.4.4-272-ga9f38b @ /home/richard/.emacs.d/straight/build/org/)]
Richard, yourbug submission uncovered 3 problems. From minor to major: 1. Missing delimitersin your example change :includes stdio.h to :includes 2. The function strcmp() is undefined A patch has already been committed to fix that, in early January. 3. A mess between char* and double in automatically generated code. This is a regression. I am investigating it. Stay connected for an update. Thanks for reporting! Cheers Thierry Le 02/04/2021 à 19:40, Richard Sent via General discussions about Org-mode. a écrit : > When a table of floating pointer numbers that contains a header row is > set as a variable to a C source block, the generated header file > contains an invalid return type, causing the program to not compile. > > For example, if I have a table that looks like > > #+NAME: tbl-doubles-org-bug-report > #+begin_src C :includes :results table :colnames '("i" "val") > for (int i = 0; i < 5; i++) { > printf("%f %f\n", (double)i, i * 2.0); > } > #+end_src > > #+RESULTS: tbl-doubles-org-bug-report > | i | val | > |-+-| > | 0.0 | 0.0 | > | 1.0 | 2.0 | > | 2.0 | 4.0 | > | 3.0 | 6.0 | > | 4.0 | 8.0 | > > And then attempt to include that table as a variable called data > > #+begin_src C :includes stdio.h :var data=tbl-doubles-org-bug-report > printf("No errors!"); > #+end_src > > Nothing is printed, and *Org-Babel Error Output* will display > > /tmp/babel-NQHi51/C-src-lsA892.c: In function ‘data_h’: > /tmp/babel-NQHi51/C-src-lsA892.c:24:65: error: incompatible types when > returning type ‘double’ but ‘const char *’ was expected > 24 | const char* data_h (int row, const char* col) { return > data[row][get_column_num(2,data_header,col)]; } > | > ~^~~ > /bin/bash: line 1: /tmp/babel-NQHi51/C-bin-90Zk27: Permission denied > > If we look in the /tmp file mentioned, it's easy to see the > inconsistency. > > double data[5][2] = { > {0.00,0.00}, > {1.00,2.00}, > {2.00,4.00}, > {3.00,6.00}, > {4.00,8.00} > }; > const int data_rows = 5; > const int data_cols = 2; > int get_column_num (int nbcols, const char** header, const char* column) > { > int c; > for (c=0; c if (strcmp(header[c],column)==0) > return c; > return -1; > } > const char* data_header[2] = {"i","val"}; > const char* data_h (int row, const char* col) { return > data[row][get_column_num(2,data_header,col)]; } > int main() { > printf("No errors!"); > return 0; > } > > data_h() should not return a const char*, but a double. > > I've also tested this with $ emacs -Q and the issue is still present. It > does not occur if the table contains integer values instead of floats > or if a header row is omitted. Seeing as how the header row is processed > differently from the data, I don't think the presence of a header row is > meant to cause this error. > > > > Emacs : GNU Emacs 27.1 (build 1, x86_64-pc-linux-gnu, GTK+ Version > 3.24.22, cairo version 1.17.3) > of 2020-08-28 > Package: Org mode version 9.4.4 (release_9.4.4-272-ga9f38b @ > /home/richard/.emacs.d/straight/build/org/) > > current state: > == > (setq > org-src-mode-hook '(org-src-babel-configure-edit-buffer > org-src-mode-configure-edit-buffer doom-modeline-set-org-src-modeline) > org-link-shell-confirm-function 'yes-or-no-p > org-blank-before-new-entry '((heading . t) (plain-list-item . t)) > org-metadown-hook '(org-babel-pop-to-session-maybe) > org-clock-out-hook '(org-clock-remove-empty-clock-drawer) > org-html-format-inlinetask-function > 'org-html-format-inlinetask-default-function > org-odt-format-headline-function 'org-odt-format-headline-default-function > org-agenda-files '("~/org") > org-ascii-format-inlinetask-function 'org-ascii-format-inlinetask-default > org-startup-folded t > org-mode-hook '(#[0 "\301\211\207" [imenu-create-index-function > org-imenu-get-tree] 2] > #[0 "\300\301\302\303\304$\207" [add-hook change-major-mode-hook > org-show-all append local] 5] > #[0 "\300\301\302\303\304$\207" [add-hook change-major-mode-hook > org-babel-show-result-all append local] 5] > org-babel-result-hide-spec org-babel-hide-all-hashes turn-on-auto-fill > mixed-pitch-mode org-superstar-mode > highlight-parentheses-mode) > org-odt-format-drawer-function #[514 "\207" [] 3 "\n\n(fn NAME CONTENTS)"] > org-archive-hook '(org-attach-archive-delete-maybe) > org-confirm-elisp-link-function 'yes-or-no-p > org-agenda-before-write-hook '(org-agenda-add-entry-text) > org-metaup-hook '(org-babel-load-in-session-maybe) > org-bibtex-headline-format-function #[257 "\300\236A\207" [:title] 3 > "\n\n(fn ENTRY)"] > org-adapt-indentation nil > org-latex-format-drawer-function #[514 "\207" [] 3 "\n\n(fn _ CONTENTS)"] > org-babel-pre-tangle-hook '(save-buffer) > org-file-apps '(("\\.pdf\\'" . emacs) (auto-mode . emacs) (directory . > emacs) ("\\.mm\\'" . default) ("\\.x?html?\\'" . default) > ("\\.pdf\\'" . default)
Re: c-y inserts after entry
I get the expected behavior in Emacs 27.0.90 * x asd fnjkasn dkfjan ksdjfn kajsdfn * kill this whole header line and insert before caret ^ajsdf kasdn kfjasnd jkfan ksdfn what i expect is that it is inserted above caret what occurs is it gets inserted below * z Le 18/07/2020 à 01:18, Samuel Wales a écrit : > in recent maint in emacs 25. > > * x > asd fnjkasn dkfjan ksdjfn kajsdfn > ^ajsdf kasdn kfjasnd jkfan ksdfn > * kill this whole header line and insert before caret > what i expect is that it is inserted above caret > what occurs is it gets inserted below > * z > > this is surprising because i expected plain text behavior for yanking. > > note that i might or might not want plain text behavior for inserting > a new header. > > i have a nagging feeling i am missing a NEWS item here. >
Re: [join rows] (was: join tables from different files)
Le 07/06/2020 à 17:09, Uwe Brauer a écrit : > "t" == tbanelwebmin writes: > > Yes you can. > Use an org-id > > Thanks! Nice! > > > I have a another question in this context: > can I join, say 4 tables, but row wise? > Say I have > > #+TBLNAME: RK > #+ATTR_HTML: :border 2 :rules all :frame border > | met | Q1 | Q2 | Q3 | total | > |-+++-+---| > | RK | 1 | 1 | 0.5 | 2.5 | > #+TBLFM: $5=$2+$3+$4 > > #+TBLNAME: SVD > #+ATTR_HTML: :border 2 :rules all :frame border > | met | Q1: | Q2 | Q3 | total | > |-+-+++---| > | SVD | 0.5 | 1 | 1 | 2.5 | > #+TBLFM: $5=$2+$3+$4;f1 > > #+TBLNAME: MIN > #+ATTR_HTML: :border 2 :rules all :frame border > | met | Q1 | Q2 | total | > |-++-+---| > | MIN | 2 | 0.5 | 2.5 | > #+TBLFM: $4=$2+$3 > > #+TBLNAME: BDF > #+ATTR_HTML: :border 2 :rules all :frame border > | met | Q1 | Q2 | total | > |-+--+--+---| > | MIN | 0.75 | 1.75 | 2.5 | > #+TBLFM: $4=$2+$3 > > Since they don't have the same amount of columns, I'd like to join the 4 > tables like this > #+TBLNAME: Total1 > #+ATTR_HTML: :border 2 :rules all :frame border > | met | Q1 | Q2 | Q3 | total | > |-+-++-+---| > | RK | 1 | 1 | 0.5 | 2.5 | > |-+-++-+---| > | SVD | 0.5 | 1 | 1 | 2.5 | > #+TBLFM: $5=$2+$3+$4;f1 > > #+TBLNAME: Total2 > #+ATTR_HTML: :border 2 :rules all :frame border > | met | Q1 | Q2 | total | > |-+--+--+---| > | MIN |2 | 0.5 | 2.5 | > |-+--+--+---| > | MIN | 0.75 | 1.75 | 2.5 | > #+TBLFM: $4=$2+$3 > > Is this possible? > > regards > Uwe Brauer I'm not aware of anything specific for this purpose. A starting point could be the `append' lisp function: #+begin_src elisp :var table1=RK :var table2=SVD :colnames t (append table1 table2) #+end_src #+RESULTS: | met | Q1: | Q2 | Q3 | total | |-+-++-+---| | RK | 1 | 1 | 0.5 | 2.5 | | SVD | 0.5 | 1 | 1 | 2.5 |
Re: join tables from different files
Yes you can. Use an org-id Suppose you have a table in file x.org, under some title: - x.org -- * title | a | b | |+| | aa | 5 | | bb | 7 | | aa | 11 | -- Add an ID by calling: M-x org-id-get-create. You end up with: - x.org -- * title :PROPERTIES: :ID: c8b8bb22-e42e-426f-afb0-4cb19aed27ad :END: | a | b | |+| | aa | 5 | | bb | 7 | | aa | 11 | -- Globally save the new ID by calling M-: (org-id-locations-save). You will see c8b8bb22-e42e-426f-afb0-4cb19aed27ad in the ~/.emacs.d/.org-id-locations file. In another file, say y.org, you may now reference this remote table with the ID in an aggregated, transposed, or joined block: - y.org -- #+BEGIN: aggregate :table "c8b8bb22-e42e-426f-afb0-4cb19aed27ad" :cols "a vsum(b)" | a | vsum(b) | |+-| | aa | 16 | | bb | 7 | #+END: -- The wizard for creating the aggregated (or joined, or transposed) block does not (yet) complete remote tables names, only buffer-local ones. The wizard may be called with C-c C-x x (or C-x C-x i on older versions of Org Mode). Note that those global IDs are also used by the spreadsheet remote() function. Example: - z.org -- | b | | 5 | | 7 | | 11 | #+TBLFM: $1=remote(c8b8bb22-e42e-426f-afb0-4cb19aed27ad,@@#$2) -- Have fun Thierry Le 06/06/2020 à 22:40, Uwe Brauer a écrit : > Hi > > I know I can either user > (org-insert-dblock:aggregate) > (org-insert-dblock:join) > > To join or aggregate tables with in the same files. > > But can I join tables from different files? > > Thanks > > Uwe Brauer > > >
Re: org table: one column of random numbers (but natural ones)
Le 27/05/2020 à 22:40, Uwe Brauer a écrit : > Hi > > > I have a org table and want to add a new colum, which natural numbers > which are randomly ordered > > So I tried > > #+TBLFM: $1=@#-1::$4=random($1@4);f1 > > But the column contains real numbers > > What can I do > > Uwe Brauer > > What about: #+TBLFM: $4=random($1);f0 In row 67 you would have a random integer in the range [0..67) f0 format removes any fractional part leaving only an integer number
Re: (Feature to Potentially Upstream) extending org-plot.el
+1 Le 18/05/2020 à 09:23, Timothy a écrit : Good news! This is the last of my "things I want to contact the mailing list about" backlog 😛 So, I recently wanted to be able to create a radar chart in org, using #+PLOT without a 500 character #+PLOT line. I started with advice-override, but quickly realised it would be better just to switch out org-plot.el with a modified version, which can be found here: https://github.com/tecosaur/emacs-config/blob/master/lisp/org-plot.el As this currently stands, with the below table and plot line I can produce this plot https://media.githubusercontent.com/media/tecosaur/emacs-config/master/misc/document-format-comparison.png. #+PLOT: transpose:yes type:radar min:0 max:4 file:"misc/document-format-comparison.png" | Format | Fine-grained-control | Initial Effort | Syntax simplicity | Editor Support | Integrations | Ease-of-referencing | Versatility | |---+--++---++--+-+-| | Word | 2 | 4 | 4 | 2 | 3 | 2 | 2 | | LaTeX | 4 | 1 | 1 | 3 | 2 | 4 | 3 | | Org Mode | 4 | 2 | 3.5 | 1 | 4 | 4 | 4 | | Markdown | 1 | 3 | 3 | 4 | 3 | 3 | 1 | | Markdown + Pandoc | 2.5 | 2.5 | 2.5 | 3 | 3 | 3 | 2 | The colours are a doom-specific bit, because I draw them from the current doom theme. I figure this could just be replaced with a general purpose entry point into the plot generation script where the user can add a 'custom preamble' function. I'm also interested in refactoring the type:XXX bit so that it's more general, less hardcoded. I don't know what may be involved, but I'm thinking something similar to org export when you can define a new export function could be nice (hough I imagine that example is far more complicated than this would be). I'm hoping that someone may be interested enough to provide feedback, and idealy help me extend org-plot in this manner, with the goal of having this functionality upstreamed (assuming interest in doing so). Please let me know! All the best, tecosaur (p.s. as mentioned in my first email, I am going to try getting dragged down this rabbit-hole again till late june)
Re: [PATCH] faster org-table-align
Le 08/05/2020 à 15:44, Nicolas Goaziou a écrit : > Hello, > > tbanelwebmin writes: > >> `org-table-align' may benefit from recent `org-table-to-lisp' >> speed-up. >> >> The current code contains the following lines in org-table.el. They >> parse the table into a Lisp structure, which is precisely what >> `org-table-to-lisp' is supposed to do. >> >> (fields (mapcar >>(lambda (l) >> (and (not (string-match-p org-table-hline-regexp l)) >> (org-split-string l "[ \t]*|[ \t]*"))) >>(split-string (buffer-substring beg end) "\n" t))) >> >> Just replace them by a call to `org-table-to-lisp': >> >> (fields (cl-loop for row in (org-table-to-lisp) >> collect (and (listp row) row))) > Good catch! I applied a similar change in master. Let me know how it > goes. > > Thank you! > > Regards, > I ran again my benchmark with your patch. The Emacs profiler is not accurate enough to tell which one among `mapcar' and `cl-loop' is best. Anyway, your patch is perfect! Thanks!
[PATCH] faster org-table-align
`org-table-align' may benefit from recent `org-table-to-lisp' speed-up. The current code contains the following lines in org-table.el. They parse the table into a Lisp structure, which is precisely what `org-table-to-lisp' is supposed to do. (fields (mapcar (lambda (l) (and (not (string-match-p org-table-hline-regexp l)) (org-split-string l "[ \t]*|[ \t]*"))) (split-string (buffer-substring beg end) "\n" t))) Just replace them by a call to `org-table-to-lisp': (fields (cl-loop for row in (org-table-to-lisp) collect (and (listp row) row))) I made a benchmark on a large table (1 rows, 6 columns). Adding a column, then removing it, M-S-right M-S-left. I measured time and memory with profiler-*, taking care to call `garbage-collect' beforehand. Conclusions: - `org-table-align' is faster, overall operation is faster - there is much less garbage collection - memory allocation is significantly lower | | current | new | % | |--+---+---+-| | org-table-align CPU samples | 1313 | 1102 | 16. | | overall CPU samples | 2040 | 1628 | 20. | | GC CPU samples | 4283 | 2609 | 39. | | overall allocated bytes | 622087730 | 272436087 | 56. | | org-table-align allocated bytes | 445910318 | 146646065 | 67. | #+TBLFM: $4=100*($-2-$-1)/$-2;f0 Note: running twice the same bench yields slightly different results. Have fun Thierry
Re: [ANN] faster org-table-to-lisp
Le 01/05/2020 à 15:11, Nicolas Goaziou a écrit : > Indeed. I also realized this, and fixed it a couple of hours ago. You > will notice I shamelessly used your "\\="© trick! :) This kind of shamelessness builds great communities :) > Great. I removed the check and added an entry in ORG-NEWS. It shouldn't > make any speed difference in your monster table, because it is located > right after a headline. It will, however, help tremendously in tables > buried deep within a section. Well done. There is a last cleanup that can be achieved (or not, doesn't really matter): simplifying two calls located in org-table.el: (org-table-to-lisp (buffer-substring-no-properties (org-table-begin) (org-table-end))) to: (org-table-to-lisp)
Re: [ANN] faster org-table-to-lisp
Le 01/05/2020 à 12:15, Nicolas Goaziou a écrit : > Hello, > > tbanelwebmin writes: > >> Nicolas, how did you do that? Your version is 25% faster than mine, >> and the code is 33% shorter! Very elegant. > Thank you. There's nothing fancy, really. > > The main difference is that it does not call `org-table-end'. Minor > tweaks are: > > - use simpler regexps, > - call `skip-chars-forward' whenever possible. I realized that not calling `org-table-end' may cause a corner case: | 2 | b | | c | d | #+TBLFM: @1$1=2||3 is read as: (("2" "b") ("c" "d") ("" "3")) because of the "|" just below the table. This can be fixed by changing the ending condition from (while (search-forward "|" (line-end-position) t) to (while (re-search-forward "^[ \t]*|" (line-end-position) t) Not a big deal. >> Sorry, I may not understood what you said: >> = Since you're changing the signature, I suggest to provide the table >> = element instead of ORG-AT-TABLE-P. AFAICT, `org-babel-read-element', >> = through `org-babel-read-table', would greatly benefit from this. >> >> Could you elaborate (if still relevant)? > If you know the table ELEMENT, you don't need to check if you're at > a table, nor do you need to compute table boundaries. We could have made > use of this information to avoid a call to `org-at-table-p', much like > your initial intent. > > Thinking about it, we don't even need to call `org-at-table-p' at all. > Indeed, this is a low-level, non-interactive, function. We can > reasonably expect the callers to check if they are really at a table in > the first place. > > It would increase speed for this function noticeably, and the ELEMENT > argument would not be relevant anymore. > > WDYT? I do agree. We can expect callers to be on a table. If they are not, they will get `nil', and instantly notice it. >> The side effect of `re-search-forward' was to advance point, while >> `looking-at' don't move. > Ah true. I overlooked that. > > Regards, > -- > Nicolas Goaziou > Have fun, Thierry Banel
Re: [ANN] faster org-table-to-lisp
Nicolas, how did you do that? Your version is 25% faster than mine, and the code is 33% shorter! Very elegant. Le 01/05/2020 à 00:35, Nicolas Goaziou a écrit : > tbanelwebmin writes: > >> I found a way to ensure full backward compatibility. I keep the same >> signature. When a table is given as a string parameter, it is inserted >> into a temporary buffer, which is then parsed. Overall, the resulting >> speed is quite satisfactory. > A, you didn't like my ELEMENT suggestion. Sorry, I may not understood what you said: = Since you're changing the signature, I suggest to provide the table = element instead of ORG-AT-TABLE-P. AFAICT, `org-babel-read-element', = through `org-babel-read-table', would greatly benefit from this. Could you elaborate (if still relevant)? > >> I also made the function more tolerant to ill-formed tables: missing >> "|" or excess of spaces at the end of a row are now gracefully >> accepted. > Great! > >> (while (not (re-search-forward "\\=\\s-*\n" end t)) > (re-search-forward "\\=..." ...) -> (looking-at "..." ...) The side effect of `re-search-forward' was to advance point, while `looking-at' don't move. This does not matter anymore. > > Note that Org does not use \\s- but the simpler [ \t]. > > Also, the regexp assumes the table end with a newline character, which > may not be the case. > >> (unless (re-search-forward "\\=\\s-*\\([^|\n]*\\)\\(|?\\)" end >> t) >> (user-error "Malformed table at char %s" (point))) > This cannot happen. The regexp above matches anything, i.e., the empty > string. Right! I didn't noticed... > >> (goto-char (match-end 1)) >> (skip-chars-backward " \t" (match-beginning 1)) >> (push >>(buffer-substring-no-properties (match-beginning 1) (point)) >>row) >> (goto-char (match-end 2))) >> (push (nreverse row) table))) >> (nreverse table) > I applied your suggestion, with a few simplifications. Hopefully, it > squeezed a bit more the execution time. Let me know! Yes! 25% My two unit tests give correct results. > >> The new implementation can be more than 100 times faster. This enhances >> responsiveness of Babel or Gnuplot blocks handling thousands long >> tables. > Looks good. However, we didn't change the signature, so I didn't add > this to ORG-NEWS. It is in the commit message, tho. > > Thank you! > Nice team work, thank you too! Thierry Banel
Re: [ANN] faster org-table-to-lisp
Better, thanks Daniele Let's go for "handling very large tables" Regards Thierry Le 30/04/2020 à 22:47, Daniele Nicolodi a écrit : > Hello, > > On 30-04-2020 14:28, tbanelwebmin wrote: >> * Version 9.4 (not yet released) >> ** Miscellaneous >> *** Faster org-table-to-lisp >> >> The new implementation can be more than 100 times faster. This enhances >> responsiveness of Babel or Gnuplot blocks handling thousands long tables. > Nitpicking: I think "handling thousands long tables" should be "handling > tables with thousands of (rows|columns|cells)" (pick the appropriate > one), but probably "handling very large tables" is just good enough. > > Cheers, > Dan > > >
Re: [ANN] faster org-table-to-lisp
Le 30/04/2020 à 10:09, Nicolas Goaziou a écrit : > Hello, > > tbanelwebmin writes: > >> Here is an alternative, faster version of org-table-to-lisp. It can be >> more than 100 times faster. > Great! Thank you! > >> #+BEGIN_SRC elisp >> (defun org-table-to-lisp-faster (&optional org-table-at-p-done) >> "Convert the table at point to a Lisp structure. >> The structure will be a list. Each item is either the symbol `hline' >> for a horizontal separator line, or a list of field values as strings. >> The table is taken from the buffer at point. >> When the optional ORG-TABLE-AT-P-DONE parameter is not nil, it is >> assumed that (org-at-table-p) was already called." > Since you're changing the signature, I suggest to provide the table > element instead of ORG-AT-TABLE-P. AFAICT, `org-babel-read-element', > through `org-babel-read-table', would greatly benefit from this. > > Or, to be backward compatible, I suggest > > &optional TEXT TABLE > >> (or org-table-at-p-done (org-at-table-p) (user-error "No table at point")) >> (save-excursion >> (goto-char (org-table-begin)) >> (let ((end (org-table-end)) >> (row) >> (table)) > Nitpick: > > (row nil) > (table nil) > >> (while (< (point) end) >> (setq row nil) >> (search-forward "|" end) >> (if (looking-at "-") >> (progn >> (search-forward "\n" end) > (forward-line) > >> (push 'hline table)) >> (while (not (search-forward-regexp "\\=\n" end t)) > (unless (eolp) > ...) > >> (unless (search-forward-regexp "\\=\\s-*\\([^|]*\\)" end t) >> (user-error "Malformed table at char %s" (point))) > A row may not be properly ended. It doesn't warrant an error. Could you > make it more tolerant? > > Also `search-forward-regexp' -> `re-search-forward', i.e., use the > original. > >> (let ((b (match-beginning 1)) >> (e (match-end 1))) > Nitpick: spurious spaces. > >> (and (search-backward-regexp "[^ \t]" b t) >> (forward-char 1)) > (skip-chars-backward " \t") > >> It is faster because it operates directly on the buffer with >> (search-forward-regexp). Whereas the standard function splits a string >> extracted from the buffer. > You are right. I guess the initial implementation didn't have these > monster tables in mind. > >> This function is a drop-in replacement for the standard one. It can >> benefit to Babel and Gnuplot. >> >> Would it make sense to upgrade Org Mode code base? > Certainly. Could you add an entry in ORG-NEWS, in "Miscellaneous"? > > Regards, > Thanks Nicolas for your nice suggestions. I've taken them into account. Particularly, the use of (skip-chars-backward " \t") gave a small additional speedup, and simplified the code. I found a way to ensure full backward compatibility. I keep the same signature. When a table is given as a string parameter, it is inserted into a temporary buffer, which is then parsed. Overall, the resulting speed is quite satisfactory. I also made the function more tolerant to ill-formed tables: missing "|" or excess of spaces at the end of a row are now gracefully accepted. Regards Thierry #+BEGIN_SRC elisp (defun org-table-to-lisp (&optional txt) "Convert the table at point to a Lisp structure. The structure will be a list. Each item is either the symbol `hline' for a horizontal separator line, or a list of field values as strings. The table is taken from the parameter TXT, or from the buffer at point." (if txt (with-temp-buffer (insert txt) (goto-char (point-min)) (org-table-to-lisp)) (unless (org-at-table-p) (user-error "No table at point")) (save-excursion (goto-char (org-table-begin)) (let ((end (org-table-end)) (row nil) (table nil)) (while (< (point) end) (setq row nil) (search-forward "|" end) (if (looking-at "-") (progn (forward-line) (push 'hline table)) (while (not (re-search-forward "\\=\\s-*\n" end t)) (unless (re-search-forward "\\=\\s-*\\([^|\n]*\\)\\(|?\\)" end t) (user-error "Malformed table at char %s" (point))) (goto-char (match-end 1)) (skip-chars-backward " \t" (match-beginning 1)) (push (buffer-substring-no-properties (match-beginning 1) (point)) row) (goto-char (match-end 2))) (push (nreverse row) table))) (nreverse table) #+END_SRC * Version 9.4 (not yet released) ** Miscellaneous *** Faster org-table-to-lisp The new implementation can be more than 100 times faster. This enhances responsiveness of Babel or Gnuplot blocks handling thousands long tables.
[ANN] faster org-table-to-lisp
Hi The List. Here is an alternative, faster version of org-table-to-lisp. It can be more than 100 times faster. #+BEGIN_SRC elisp (defun org-table-to-lisp-faster (&optional org-table-at-p-done) "Convert the table at point to a Lisp structure. The structure will be a list. Each item is either the symbol `hline' for a horizontal separator line, or a list of field values as strings. The table is taken from the buffer at point. When the optional ORG-TABLE-AT-P-DONE parameter is not nil, it is assumed that (org-at-table-p) was already called." (or org-table-at-p-done (org-at-table-p) (user-error "No table at point")) (save-excursion (goto-char (org-table-begin)) (let ((end (org-table-end)) (row) (table)) (while (< (point) end) (setq row nil) (search-forward "|" end) (if (looking-at "-") (progn (search-forward "\n" end) (push 'hline table)) (while (not (search-forward-regexp "\\=\n" end t)) (unless (search-forward-regexp "\\=\\s-*\\([^|]*\\)" end t) (user-error "Malformed table at char %s" (point))) (let ((b (match-beginning 1)) (e (match-end 1))) (and (search-backward-regexp "[^ \t]" b t) (forward-char 1)) (push (buffer-substring-no-properties b (point)) row) (goto-char (1+ e (push (nreverse row) table))) (nreverse table #+END_SRC Bellow is an example of a large table borrowed from the Datamash software. On my PC, the reproducible benches show: - Traditional org-table-to-lisp: 130 seconds - Alternative org-table-to-lisp: 0.8 seconds (not compiled) It is faster because it operates directly on the buffer with (search-forward-regexp). Whereas the standard function splits a string extracted from the buffer. This function is a drop-in replacement for the standard one. It can benefit to Babel and Gnuplot. Would it make sense to upgrade Org Mode code base? Beware! The optional parameter has a slightly different meaning for both functions: - for the traditional function, it is a string representing an Org table - for the alternative function, it is a Boolean telling whether (org-table-at-p) has been called or not This difference makes no difference for the use cases in the code base. The function is always called without a parameter, or as: #+BEGIN_SRC elisp (org-table-to-lisp (buffer-substring-no-properties (org-table-begin) (org-table-end))) #+END_SRC Here is the reproducible bench. It is a self-contained, Org Mode file to be opened in Emacs. wget http://tbanelwebmin.free.fr/OrgMode/bench-org-table-to-lisp.org.gz
Re: [Idea] Org Collections
Interesting idea! Is everyone aware of Emacs Projectile? https://github.com/bbatsov/projectile Not exactly the Org Collections you talks about, Gustav, but somehow related. Projectile manages collections of files that belong together. They may be anything (Org Mode, Python, C++, HTML, LibreOffice, anything). It is most often zero-configuration: if a directory contains a .git or .bzr sub-directory, it is a Projectile project. A Maven project is a Projectile project, and so on. If the directory is not of a known type, just adding and empty .projectile sub-directory makes it a Projectile project. Quite handy. Could your Org Collections idea be a Projectile add-on? As an example, there is such an add-on: org-projectile https://github.com/IvanMalison/org-projectilehttps://github.com/IvanMalison/org-projectile "org-projectile provides functions for the creation of org-mode TODOs that are associated with projectile projects." Regards Le 14/12/2019 à 18:32, Gustav Wikström a écrit : Hi list and all honored readers! I have an idea. One that I've mentioned before in side notes. And I want to emphasize that this still only is an idea. But I want to present this idea to you. As a way to gather feedback and get input. Maybe the idea is stupid!? Maybe you think it's already solved? Or maybe it's not, and lots of you resonate with it as well. In any case, please let me know what you think on the piece below! ORG MODE: COLLECTIONS/PROJECTS Gustav Wikström
Re: [O] html-email in org-mode
* Awesome This is awesome! * Simple I followed your instructions, and in 2 minutes I was able to send a /1st/ mail from Org Mode * Thanks Thanks John for showing that.