Re: [O] remote execution in heterogeneous environment
George Jones elu...@gmail.com writes: Success. No setting of temporary-file-directory needed. Great! Thanks for your inisistent testing. Thanks, ---George Best regards, Micvhael.
Re: [O] remote execution in heterogeneous environment
George Jones elu...@gmail.com writes: Still need to (setq temporary-file-directory /tmp/) for things to work, else I get: Couldn't write region to `/scpc:george@localhost#:/var/folders/j6/j6w17khs3vl9s2_yg4rb5zsmgn/T/input- 11249kNp', decode using `base64 -d -i %s\ ' failed Test script and output, without explicitly setting temporary-file-directory, attached: Indeed. I've published a patch to ob.el (as of Emacs trunk). Meanwhile, ob.el has been renamed to ob-core.el; likely that patch was lost :-( Bastien, could you please apply this patch (for `org-babel-temp-file')? Thanks, ---George Jones Best regards, Michael.
Re: [O] remote execution in heterogeneous environment
Hi Michael, Michael Albinus michael.albi...@gmx.de writes: Indeed. I've published a patch to ob.el (as of Emacs trunk). Meanwhile, ob.el has been renamed to ob-core.el; likely that patch was lost :-( Bastien, could you please apply this patch (for `org-babel-temp-file')? Mhh... not sure what patch you refer to -- can you send me the patch itself and/or a link to it? Thanks! -- Bastien
Re: [O] remote execution in heterogeneous environment
Bastien b...@altern.org writes: Hi Michael, Hi Bastien, Indeed. I've published a patch to ob.el (as of Emacs trunk). Meanwhile, ob.el has been renamed to ob-core.el; likely that patch was lost :-( Bastien, could you please apply this patch (for `org-babel-temp-file')? Mhh... not sure what patch you refer to -- can you send me the patch itself and/or a link to it? http://thread.gmane.org/gmane.emacs.orgmode/63586/focus=63709 The last patch in this message. Again, the function seems to be moved to ob-core.el now. Thanks!
Re: [O] remote execution in heterogeneous environment
Hi Michael, Michael Albinus michael.albi...@gmx.de writes: http://thread.gmane.org/gmane.emacs.orgmode/63586/focus=63709 The last patch in this message. Again, the function seems to be moved to ob-core.el now. Ah okay, applied, thanks! -- Bastien
Re: [O] remote execution in heterogeneous environment
Should a git pull of development version of org pull the right patch? Thanks, ---george jones On Dec 23, 2012 11:47 AM, Bastien b...@altern.org wrote: Hi Michael, Michael Albinus michael.albi...@gmx.de writes: http://thread.gmane.org/gmane.emacs.orgmode/63586/focus=63709 The last patch in this message. Again, the function seems to be moved to ob-core.el now. Ah okay, applied, thanks! -- Bastien
Re: [O] remote execution in heterogeneous environment
Success. No setting of temporary-file-directory needed. Thanks, ---George On Sun, Dec 23, 2012 at 2:54 PM, Bastien b...@altern.org wrote: George Jones elu...@gmail.com writes: Should a git pull of development version of org pull the right patch? Yes. Simply ~$ git clone git://orgmode.org/org-mode.git (our master branch is the development branch.) HTH, -- Bastien
Re: [O] remote execution in heterogeneous environment
George elu...@gmail.com writes: Hi George, Make everything works with: (setq temporary-file-directory /tmp/) That's a misunderstanding. I meant the workaround I have proposed yesterday on the org-mode ML, and which has been committed by Bastien. Meanwhile, everything shall work out-of-the box. Does it for you? Thanks, ---George Jones Best regards, Michael.
Re: [O] remote execution in heterogeneous environment
Michael Albinus michael.albi...@gmx.de writes: Similar procedure on a mac still yeilds: Couldn't write region to `/scpc:g...@example.com:/var/folders/j6/j6w17khs3vl9s2_yg4rb5zsmgn/T/input- 61683xyS', decode using `mimencode -u -b %s' f\ ailed Could you, please, show the exact test case? And please apply the workaround first (see my other email). Everything works if I do, as you suggested: (setq temporary-file-directory ~/tmp/) ---cut here--- #+NAME: hobbits | bilbo | frodo | #+name: executeOnLocalVM #+BEGIN_SRC sh :var foo=hobbits :dir /george@localhost#: :results output echo executed on `hostname` at `date` echo two $foo #+END_SRC #+RESULTS: executeOnLocalVM : executed on garlic at Sat Dec 22 13:25:35 UTC 2012 : two bilbo frodo #+name: relevantVars #+BEGIN_SRC emacs-lisp (concat temporary-file-directory temporary-file-directory ) #+END_SRC #+RESULTS: relevantVars : temporary-file-directory /tmp/ #+name: versions #+BEGIN_SRC emacs-lisp (concat (emacs-version) tramp version: tramp-version org version org-version) #+END_SRC #+RESULTS: versions : GNU Emacs 24.2.1 (x86_64-apple-darwin11.4.2) : of 2012-11-19 on tenseven-slave.macports.org : tramp version: 2.2.6 : org version 7.9.2 ---cut here--- Thanks, ---George Jones
Re: [O] remote execution in heterogeneous environment
Michael Albinus michael.albi...@gmx.de writes: Similar procedure on a mac still yeilds: Couldn't write region to `/scpc:g...@example.com:/var/folders/j6/j6w17khs3vl9s2_yg4rb5zsmgn/T/input- 61683xyS', decode using `mimencode -u -b %s' f\ ailed Could you, please, show the exact test case? And please apply the workaround first (see my other email). Make everything works with: (setq temporary-file-directory /tmp/) Thanks, ---George Jones
Re: [O] remote execution in heterogeneous environment
Bastien b...@altern.org writes: Hi Michael, Hi everybody, From the org-mode repo, simply run make test: it will compile the files and run the test suite. I could reproduce the problem. Finally, it is an error in `process-file', which wasn't detected until now. Thanks to triggering this! I've fixed it already in Emacs' trunk. For org-mode, I have used the following workaround: --8---cut here---start-8--- ~/src/org-mode git diff diff --git a/lisp/ob-eval.el b/lisp/ob-eval.el index 23e7143..df95d0a 100644 --- a/lisp/ob-eval.el +++ b/lisp/ob-eval.el @@ -143,6 +143,11 @@ specifies the value of ERROR-BUFFER. shell-file-name /bin/sh)) exit-status) +;; There is an error in `process-file', when `error-file' exists. +;; Fixed already in Emacs trunk; for the time being we apply a +;; workaround. +(unless (file-remote-p default-directory) + (delete-file error-file)) (if (or replace (and output-buffer (not (or (bufferp output-buffer) (stringp output-buffer) --8---cut here---end---8--- With that workaround, make test results in --8---cut here---start-8--- Ran 320 tests, 319 results as expected, 1 unexpected (2012-12-21 09:13:44+0100) 7 expected failures 1 unexpected results: FAILED ob-exp/exports-inline --8---cut here---end---8--- Best, Best regards, Michael.
Re: [O] remote execution in heterogeneous environment
George Jones elu...@gmail.com writes: Hi. Hi George, Similar procedure on a mac still yeilds: Couldn't write region to `/scpc:g...@example.com:/var/folders/j6/j6w17khs3vl9s2_yg4rb5zsmgn/T/input- 61683xyS', decode using `mimencode -u -b %s' f\ ailed Could you, please, show the exact test case? And please apply the workaround first (see my other email). FYI/Thanks, ---George Jones Best regards, Michael.
Re: [O] remote execution in heterogeneous environment
Hi Michael, Michael Albinus michael.albi...@gmx.de writes: Finally, it is an error in `process-file', which wasn't detected until now. Thanks to triggering this! I've fixed it already in Emacs' trunk. For org-mode, I have used the following workaround: great, thanks for fixing this so fast, I applied your change in Org. All tests in org's git repo now run fine, both for maint and for master. Best, -- Bastien
Re: [O] remote execution in heterogeneous environment
Michael Albinus michael.albi...@gmx.de wrote: Bastien b...@altern.org writes: Hi Michael, Hi everybody, From the org-mode repo, simply run make test: it will compile the files and run the test suite. I could reproduce the problem. Finally, it is an error in `process-file', which wasn't detected until now. Thanks to triggering this! I've fixed it already in Emacs' trunk. For org-mode, I have used the following workaround: ~/src/org-mode git diff diff --git a/lisp/ob-eval.el b/lisp/ob-eval.el index 23e7143..df95d0a 100644 --- a/lisp/ob-eval.el +++ b/lisp/ob-eval.el @@ -143,6 +143,11 @@ specifies the value of ERROR-BUFFER. shell-file-name /bin/sh)) exit-status) +;; There is an error in `process-file', when `error-file' exists. +;; Fixed already in Emacs trunk; for the time being we apply a +;; workaround. +(unless (file-remote-p default-directory) + (delete-file error-file)) (if (or replace (and output-buffer (not (or (bufferp output-buffer) (stringp output-buffer) I can confirm that this fixes the local dir case in the simple test I posted previously (and the remote dir case was fixed by the process-file fix previously). With that workaround, make test results in Ran 320 tests, 319 results as expected, 1 unexpected (2012-12-21 09:13:44+0100) 7 expected failures 1 unexpected results: FAILED ob-exp/exports-inline I don't get that failure with the current code, so maybe it has been fixed already (?): , | Ran 334 tests, 334 results as expected (2012-12-21 12:27:17-0500) | 5 expected failures ` Not sure why I have a different number of tests/expected failures either. But in any case, the main problem is resolved. Thanks for the fixes! Nick
Re: [O] remote execution in heterogeneous environment
Michael Albinus michael.albi...@gmx.de writes: Hi, Looks, like nobody did take the ball :-( Maybe I'll check it again over XMas days, and maybe I can propose a concrete patch then. The following patch I've committed to Emacs' trunk, it fixes the problem as far as I could test. Maybe somebody from the org maintainers could merge it into the org repository. --8---cut here---start-8--- ~/src/emacs/lisp/org bzr diff 123 === modified file 'lisp/org/ChangeLog' --- lisp/org/ChangeLog 2012-12-13 05:29:15 + +++ lisp/org/ChangeLog 2012-12-20 13:09:40 + @@ -1,3 +1,12 @@ +2012-12-20 Michael Albinus michael.albi...@gmx.de + + * ob.el (org-babel-temp-file): Fix setting of + `temporary-file-directory' on remote hosts. + + * ob-eval.el (org-babel-shell-command-on-region): Use + `process-file' instead of `call-process-region'. The latter one + does not work on remote hosts. + 2012-12-13 Bastien Guerry b...@gnu.org * org-latex.el (org-export-latex-links): Escape raw path when === modified file 'lisp/org/ob-eval.el' --- lisp/org/ob-eval.el 2012-09-30 15:14:59 + +++ lisp/org/ob-eval.el 2012-12-20 13:06:27 + @@ -134,14 +134,13 @@ current-prefix-arg shell-command-default-error-buffer t))) - (let ((error-file -(if error-buffer -(make-temp-file - (expand-file-name scor -(if (featurep 'xemacs) -(temp-directory) - temporary-file-directory))) - nil)) + (let ((input-file (org-babel-temp-file input-)) + (error-file (if error-buffer (org-babel-temp-file scor-) nil)) + (shell-file-name +(if (file-executable-p + (concat (file-remote-p default-directory) shell-file-name)) +shell-file-name + /bin/sh)) exit-status) (if (or replace (and output-buffer @@ -151,12 +150,14 @@ ;; Don't muck with mark unless REPLACE says we should. (goto-char start) (and replace (push-mark (point) 'nomsg)) + (write-region start end input-file) + (delete-region start end) (setq exit-status - (call-process-region start end shell-file-name t -(if error-file -(list output-buffer error-file) - t) -nil shell-command-switch command)) + (process-file shell-file-name input-file + (if error-file + (list output-buffer error-file) + t) + nil shell-command-switch command)) ;; It is rude to delete a buffer which the command is not using. ;; (let ((shell-buffer (get-buffer *Shell Command Output*))) ;; (and shell-buffer (not (eq shell-buffer (current-buffer))) @@ -175,14 +176,14 @@ (progn (setq buffer-read-only nil) (delete-region (max start end) (point-max)) (delete-region (point-min) (min start end)) + (write-region (point-min) (point-max) input-file) + (delete-region (point-min) (point-max)) (setq exit-status -(call-process-region (point-min) (point-max) - shell-file-name t - (if error-file - (list t error-file) - t) - nil shell-command-switch - command))) +(process-file shell-file-name input-file + (if error-file + (list t error-file) +t) + nil shell-command-switch command))) ;; Clear the output buffer, then run the command with ;; output there. (let ((directory default-directory)) @@ -192,11 +193,11 @@ (setq default-directory directory)) (erase-buffer))) (setq exit-status - (call-process-region start end shell-file-name nil -(if error-file -(list buffer error-file) - buffer) -nil shell-command-switch command))) + (process-file shell-file-name nil +
Re: [O] remote execution in heterogeneous environment
Hi Michael, Michael Albinus michael.albi...@gmx.de writes: The following patch I've committed to Emacs' trunk, it fixes the problem as far as I could test. Maybe somebody from the org maintainers could merge it into the org repository. I've already merged it in maint and master. Thanks a lot! -- Bastien
Re: [O] remote execution in heterogeneous environment
Bastien writes: Michael Albinus michael.albi...@gmx.de writes: The following patch I've committed to Emacs' trunk, it fixes the problem as far as I could test. Maybe somebody from the org maintainers could merge it into the org repository. I've already merged it in maint and master. Unfortunately it breaks testing, at least in batch mode. See my other posting about it. Regards, Achim. -- +[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]+ SD adaptation for Waldorf microQ V2.22R2: http://Synth.Stromeko.net/Downloads.html#WaldorfSDada
Re: [O] remote execution in heterogeneous environment
Achim Gratz strom...@nexgo.de wrote: Bastien writes: Michael Albinus michael.albi...@gmx.de writes: The following patch I've committed to Emacs' trunk, it fixes the problem as far as I could test. Maybe somebody from the org maintainers could merge it into the org repository. I've already merged it in maint and master. Unfortunately it breaks testing, at least in batch mode. See my other posting about it. Isn't ``make test'' standard procedure before a commit? If not, it should be. Otherwise, why have a test suite? Nick
Re: [O] remote execution in heterogeneous environment
Achim Gratz strom...@nexgo.de writes: Unfortunately it breaks testing, at least in batch mode. See my other posting about it. Not sure what other posting I should read -- sorry, I can't read and process on the threads at once. If you know how to fix the tests, please go ahead. Thanks! -- Bastien
Re: [O] remote execution in heterogeneous environment
Nick Dokos nicholas.do...@hp.com writes: Isn't ``make test'' standard procedure before a commit? If not, it should be. Otherwise, why have a test suite? Errare humanum est... snip Yes, I should have tested. But note that this is already committed in Emacs trunk. My assumption by looking at the code was that if tests breaks, then we need to fix the tests, not the patch. Again, if someone can help fixing the tests, that'd help me a lot! Thanks, -- Bastien
Re: [O] remote execution in heterogeneous environment
Achim Gratz strom...@nexgo.de writes: The following patch I've committed to Emacs' trunk, it fixes the problem as far as I could test. Maybe somebody from the org maintainers could merge it into the org repository. I've already merged it in maint and master. Unfortunately it breaks testing, at least in batch mode. See my other posting about it. Could you, please, give me a short pointer how to run the test suite? I will check then what happened. Note, that I don't use the org repository (yet). Maybe you must explain from the beginning ... Or you give me a short test program I could run for this dedicated test. Regards, Achim. Best regards, Michael.
Re: [O] remote execution in heterogeneous environment
Hi Michael, thanks for offering your help! The first step is to clone the repository: ~$ git clone git://orgmode.org/org-mode.git You are in the master branch, the one we use for latest developments. The other branch is maint, the one we use for bug fixing. The next version of Org, 7.9.3 will be released from maint, the next major version from master. From the org-mode repo, simply run make test: it will compile the files and run the test suite. To debug the tests, go to the testing/ directory, load org-*.el files, then find the failing tests in testing/lisp/. Since there are many tests affected, my guess is that some top-level testing function (either in testing/ or testing/lisp) is not up to date. If you have any other question, let us know. Also, if you can send patches here first next time, that'd of *great* help! Best, -- Bastien
Re: [O] remote execution in heterogeneous environment
Bastien writes: Unfortunately it breaks testing, at least in batch mode. See my other posting about it. Not sure what other posting I should read -- sorry, I can't read and process on the threads at once. The other posting is 27bcf9a70b Backport revno 111277 from Emacs trunk. If you know how to fix the tests, please go ahead. I don't. What I found out is that at least for the first four of these it only fils when the test is run in batch mode. The interactive test works. The error message indicates that it seems to attempt to create error-file twice, but I don't see how this might happen — the file is created with a unique name, so something must later use the bound variable with that name to again attempt to create the file, perhaps when it creates a buffer for it (the test used to be in a buffer that then gets written to file IIRC). Regards, Achim. -- +[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]+ Factory and User Sound Singles for Waldorf rackAttack: http://Synth.Stromeko.net/Downloads.html#WaldorfSounds
Re: [O] remote execution in heterogeneous environment
Hi. Patch seems to work. Tested on fresh emacs-snapshot (24.3.50.1) install on ubuntu with org-mode freshly pulled from git (7.9.2). #+NAME: hobbits | bilbo | frodo | #+name: execOnExample.com #+BEGIN_SRC sh :var foo=hobbits :dir /g...@example.com: :results output hostname echo executed on `hostname` at `date` echo two $foo #+END_SRC Similar procedure on a mac still yeilds: FYI/Thanks, ---George Jones Couldn't write region to `/scpc:g...@example.com:/var/folders/j6/j6w17khs3vl9s2_yg4rb5zsmgn/T/input-61683xyS', decode using `mimencode -u -b %s' f\ ailed On Thu, Dec 20, 2012 at 11:39 AM, Bastien b...@altern.org wrote: Hi Michael, Michael Albinus michael.albi...@gmx.de writes: The following patch I've committed to Emacs' trunk, it fixes the problem as far as I could test. Maybe somebody from the org maintainers could merge it into the org repository. I've already merged it in maint and master. Thanks a lot!
Re: [O] remote execution in heterogeneous environment
[Cc emacs-orgmode@gnu.org] George Jones elu...@gmail.com writes: Hi George, Did you ever get resolution on these? I think I'm hitting the same exact problems. In my last message, I gave some hints what to do in org's code. Since I don't use org myself (yet), I haven't followed that. Looks, like nobody did take the ball :-( Maybe I'll check it again over XMas days, and maybe I can propose a concrete patch then. Thanks, ---George Jones Best regards, Michael.
Re: [O] remote execution in heterogeneous environment
Neil Best nb...@ci.uchicago.edu writes: I had just recently upgraded Emacs through Macports but had not since restarted it, that's all. No such luck on the Tramp upgrade. I will just wait for the next release to propagate through Macports unless some intolerable behavior crops up. I do have a shiny new version of Org to play with thanks to ELPA. :) There are still several problems, even with Tramp 2.2.6. I use Org 20121126 from ELPA. 1. test === My local and remote hosts use /tmp, both run Linux. So I have applied prior to the test --8---cut here---start-8--- (setq temporary-file-directory ~/tmp/) --8---cut here---end---8--- This directory is known to exist locally, but not remotely. When I eval the source code block, I don't run into this infloop as you have seen (this is fixed with Tramp 2.2.6, indeed). But there is another error --8---cut here---start-8--- Tramp: Decoding region into remote file /scpc:ford:/home/albinus/tmp/sh-205735cU...done cond: Couldn't write region to `/scpc:ford:/home/albinus/tmp/sh-205735cU', decode using `base64 -d -i %s' failed --8---cut here---end---8--- Somewhere in the code, the remote temporary directory is created based on `temporary-file-directory', which is wrong. I believe this is in Org's code, must be debugged further. 2. test === I keep `temporary-file-directory' as /tmp. The source code block evaluates fine, but the result is: --8---cut here---start-8--- #+BEGIN_SRC sh :dir /ford:~ echo Executed by `whoami` on `hostname` in `pwd` #+END_SRC #+RESULTS: : Executed by albinus on detlef in /home/albinus --8---cut here---end---8--- Obviously, the code is executed on the local host. As far as I could debug it, this is because `org-babel-shell-command-on-region' calls `call-process-region', which is not aware of remote execution. PS: it's on my wishlist for a long time to learn org-mode. Maybe I have a chance now. You won't regret it. Have fun. What's your use case? Accessing lists on different devices, under Linux, Android etc. How did you happen to get on this thread? Was there a cross-post that I missed? Nothing special. I have looked into the ML archive for some other reason, and I have seen this thread by accident. Thanks for pitching in, everyone. Best regards, Michael.
Re: [O] remote execution in heterogeneous environment
Michael Albinus michael.albi...@gmx.de wrote: Neil Best nb...@ci.uchicago.edu writes: Hi, I run Macport's Emacs and ELPA's Org on my Mac but my data lives in a Red Hat environment. I thought I could edit locally and compute remotely but there seems to be a problem with temp files for executing source blocks. #+BEGIN_SRC sh :dir /midway:~ echo Executed by `whoami` on `hostname` in `pwd` #+END_SRC Results are empty and in the message buffers appears this: executing Sh code block... Tramp: Opening connection for midway using ssh... Opening connection for midway using ssh... \ Tramp: Opening connection for midway using ssh...done Tramp: Encoding region using function `base64-encode-region'...done Tramp: Decoding region into remote file /ssh:midway:/var/folders/2y/qrw3hhkx0dlb9sbl51mjy5wmgn/T/sh-510999vN...done byte-code: Couldn't write region to `/ssh:midway:/var/folders/2y/qrw3hhkx0dlb9sbl51mjy5wmgn/T/sh-510999vN', decode using `base64 -d -i %s' failed Which version of Tramp are you using? You find it by C-h v tramp-version. I remember a problem with remote temporary directories, fixed in Tramp 2.2.6. Btw, I don't use org. For local tests, what shall I apply in order to execute the code in #+BEGIN_SRC ... #+END_SRC? See http://thread.gmane.org/gmane.emacs.orgmode/62963 It was a weird setting of the variable ``temporary-file-directory'' in Neil's Macport setup, plus the assumption by tramp that that setting was valid for the remote - that turned out to be a bad assumption. Not sure it's worth it for tramp to provide a way to customize that, but that's your decision :-) AFAIAC, /tmp should be a universal standard. Nick
Re: [O] remote execution in heterogeneous environment
Nick Dokos nicholas.do...@hp.com writes: See http://thread.gmane.org/gmane.emacs.orgmode/62963 I've seen that. It was a weird setting of the variable ``temporary-file-directory'' in Neil's Macport setup, plus the assumption by tramp that that setting was valid for the remote - that turned out to be a bad assumption. Not sure it's worth it for tramp to provide a way to customize that, but that's your decision :-) AFAIAC, /tmp should be a universal standard. I don't believe it is a weird setting of `temporary-file-directory'. It is a valid value for OS X, AFAIK. And /tmp isn't a universal standard. For example, it doesn't work, when your local machine runs MS Windows. The patch in Tramp I have mentioned tries to distinguish between local and remote temporary directories. I would like to know, whether this works correctly for org-mode, or whether there is still an issue. Nick Best regards, Michael.
Re: [O] remote execution in heterogeneous environment
Thanks for taking a look, Michael. On Fri, Nov 30, 2012 at 1:37 AM, Michael Albinus michael.albi...@gmx.de wrote: Neil Best nb...@ci.uchicago.edu writes: #+BEGIN_SRC sh :dir /midway:~ echo Executed by `whoami` on `hostname` in `pwd` #+END_SRC Which version of Tramp are you using? You find it by C-h v tramp-version. I remember a problem with remote temporary directories, fixed in Tramp 2.2.6. tramp-version 2.2.3-24.1 emacs-version 24.2.1 Looks like I should upgrade tramp. I will look into that. I think if I restart emacs it will be upgraded to some minor, minor release, so maybe that took care of it already. Fat chance? Btw, I don't use org. For local tests, what shall I apply in order to execute the code in #+BEGIN_SRC ... #+END_SRC? I should mention that my source code block got mangled when I tried to post through Gmane. It should look like this: #+BEGIN_SRC sh :dir /midway:~ echo Executed by `whoami` on `hostname` in `pwd` #+END_SRC Just go into Org mode, move the point within the block, and hit C-c C-c.
Re: [O] remote execution in heterogeneous environment
Michael Albinus michael.albi...@gmx.de wrote: Nick Dokos nicholas.do...@hp.com writes: See http://thread.gmane.org/gmane.emacs.orgmode/62963 I've seen that. It was a weird setting of the variable ``temporary-file-directory'' in Neil's Macport setup, plus the assumption by tramp that that setting was valid for the remote - that turned out to be a bad assumption. Not sure it's worth it for tramp to provide a way to customize that, but that's your decision :-) AFAIAC, /tmp should be a universal standard. I don't believe it is a weird setting of `temporary-file-directory'. It is a valid value for OS X, AFAIK. And /tmp isn't a universal standard. For example, it doesn't work, when your local machine runs MS Windows. The patch in Tramp I have mentioned tries to distinguish between local and remote temporary directories. I would like to know, whether this works correctly for org-mode, or whether there is still an issue. OK - thanks. Sorry for the noise. Trying-to-get-foot-out-of-mouth-ly yours, Nick
Re: [O] remote execution in heterogeneous environment
Nick Dokos nicholas.do...@hp.com writes: OK - thanks. Sorry for the noise. Trying-to-get-foot-out-of-mouth-ly yours, Come on, there's nothing you have to sorry. You don't know Tramp, that's all. And this is not exclusive by you :-) Nick Best regards, Michael.
Re: [O] remote execution in heterogeneous environment
Neil Best nb...@ci.uchicago.edu writes: tramp-version 2.2.3-24.1 emacs-version 24.2.1 Looks like I should upgrade tramp. I will look into that. Emacs 24.3 is in pretest, it contains Tramp 2.2.6. This might be another option. See http://thread.gmane.org/gmane.emacs.devel/155058. I think if I restart emacs it will be upgraded to some minor, minor release, so maybe that took care of it already. Fat chance? Sorry, this I don't understand ... Btw, I don't use org. For local tests, what shall I apply in order to execute the code in #+BEGIN_SRC ... #+END_SRC? I should mention that my source code block got mangled when I tried to post through Gmane. It should look like this: #+BEGIN_SRC sh :dir /midway:~ echo Executed by `whoami` on `hostname` in `pwd` #+END_SRC Just go into Org mode, move the point within the block, and hit C-c C-c. Thanks, I will try to reproduce your problem locally. Best regards, Michael. PS: it's on my wishlist for a long time to learn org-mode. Maybe I have a chance now.
Re: [O] remote execution in heterogeneous environment
On Fri, Nov 30, 2012 at 1:22 PM, Michael Albinus michael.albi...@gmx.de wrote: Neil Best nb...@ci.uchicago.edu writes: tramp-version 2.2.3-24.1 emacs-version 24.2.1 Looks like I should upgrade tramp. I will look into that. Emacs 24.3 is in pretest, it contains Tramp 2.2.6. This might be another option. See http://thread.gmane.org/gmane.emacs.devel/155058. I think if I restart emacs it will be upgraded to some minor, minor release, so maybe that took care of it already. Fat chance? Sorry, this I don't understand ... I had just recently upgraded Emacs through Macports but had not since restarted it, that's all. No such luck on the Tramp upgrade. I will just wait for the next release to propagate through Macports unless some intolerable behavior crops up. I do have a shiny new version of Org to play with thanks to ELPA. :) PS: it's on my wishlist for a long time to learn org-mode. Maybe I have a chance now. You won't regret it. Have fun. What's your use case? How did you happen to get on this thread? Was there a cross-post that I missed? Thanks for pitching in, everyone.
[O] remote execution in heterogeneous environment
I run Macport's Emacs and ELPA's Org on my Mac but my data lives in a Red Hat environment. I thought I could edit locally and compute remotely but there seems to be a problem with temp files for executing source blocks. #+BEGIN_SRC sh :dir /midway:~ echo Executed by `whoami` on `hostname` in `pwd` #+END_SRC Results are empty and in the message buffers appears this: executing Sh code block... Tramp: Opening connection for midway using ssh... Opening connection for midway using ssh... \ Tramp: Opening connection for midway using ssh...done Tramp: Encoding region using function `base64-encode-region'...done Tramp: Decoding region into remote file /ssh:midway:/var/folders/2y/qrw3hhkx0dlb9sbl51mjy5wmgn/T/sh-510999vN...done byte-code: Couldn't write region to `/ssh:midway:/var/folders/2y/qrw3hhkx0dlb9sbl51mjy5wmgn/T/sh-510999vN', decode using `base64 -d -i %s' failed I suspect this has something to do with how Emacs asks OSX for temp file names. Can anyone confirm this? Is there a work-around? Ultimately I want to inspect the environment where the remote execution is happening. We use a system called Modules [1] to customize $PATH, etc. When I try this: #+BEGIN_SRC sh :dir /midway:~ module list #+END_SRC I get sh: line 1: module: command not found which makes me think that the command is running on the local system where Modules is not installed. How does this square with the discussion at [2]? [1] http://modules.sourceforge.net/ [2] http://lists.gnu.org/archive/html/emacs-orgmode/2010-03/msg00071.html
Re: [O] remote execution in heterogeneous environment
Neil Best nb...@ci.uchicago.edu wrote: I run Macport's Emacs and ELPA's Org on my Mac but my data lives in a Red Hat environment. I thought I could edit locally and compute remotely but there seems to be a problem with temp files for executing source blocks. #+BEGIN_SRC sh :dir /midway:~ echo Executed by `whoami` on `hostname` in `pwd` #+END_SRC Results are empty and in the message buffers appears this: executing Sh code block... Tramp: Opening connection for midway using ssh... Opening connection for midway using ssh... \ Tramp: Opening connection for midway using ssh...done Tramp: Encoding region using function `base64-encode-region'...done Tramp: Decoding region into remote file /ssh:midway:/var/folders/2y/qrw3hhkx0dlb9sbl51mjy5wmgn/T/sh-510999vN...done byte-code: Couldn't write region to `/ssh:midway:/var/folders/2y/qrw3hhkx0dlb9sbl51mjy5wmgn/T/sh-510999vN', decode using `base64 -d -i %s' failed I suspect this has something to do with how Emacs asks OSX for temp file names. Can anyone confirm this? Is there a work-around? Not sure about this. Ultimately I want to inspect the environment where the remote execution is happening. We use a system called Modules [1] to customize $PATH, etc. When I try this: #+BEGIN_SRC sh :dir /midway:~ module list #+END_SRC I get sh: line 1: module: command not found which makes me think that the command is running on the local system where Modules is not installed. How does this square with the discussion at [2]? But here you are probably right: see the thread at http://thread.gmane.org/gmane.emacs.orgmode/60102 and particularly this post: http://thread.gmane.org/gmane.emacs.orgmode/60102/focus=62587 It might be worth trying the fix in that mail (the funcall part), although it is known to be incomplete (at the very least: there are a few more things that seem to go wrong). Nick [1] http://modules.sourceforge.net/ [2] http://lists.gnu.org/archive/html/emacs-orgmode/2010-03/msg00071.html
Re: [O] remote execution in heterogeneous environment
Neil Best nb...@ci.uchicago.edu wrote: I run Macport's Emacs and ELPA's Org on my Mac but my data lives in a Red Hat environment. I thought I could edit locally and compute remotely but there seems to be a problem with temp files for executing source blocks. #+BEGIN_SRC sh :dir /midway:~ echo Executed by `whoami` on `hostname` in `pwd` #+END_SRC Results are empty and in the message buffers appears this: executing Sh code block... Tramp: Opening connection for midway using ssh... Opening connection for midway using ssh... \ Tramp: Opening connection for midway using ssh...done Tramp: Encoding region using function `base64-encode-region'...done Tramp: Decoding region into remote file /ssh:midway:/var/folders/2y/qrw3hhkx0dlb9sbl51mjy5wmgn/T/sh-510999vN...done byte-code: Couldn't write region to `/ssh:midway:/var/folders/2y/qrw3hhkx0dlb9sbl51mjy5wmgn/T/sh-510999vN', decode using `base64 -d -i %s' failed I suspect this has something to do with how Emacs asks OSX for temp file names. Can anyone confirm this? Is there a work-around? It may be that the fix I pointed to in the previous email will change this (who knows? it may even fix it :-) ) But there is one additional point that you might want to check. It looks as if you are creating temp files under /var/folders/2y on the remote: are you sure that a) the structure exists and b) permissions are set correctly? Do you have the /var/folders/2y structure on the local host or only on the remote? If only on the remote and the source block is running locally (because of the funcall bug), that would sort-of explain this behavior. Nick
Re: [O] remote execution in heterogeneous environment
On Thu, Nov 29, 2012 at 11:16 AM, Nick Dokos nicholas.do...@hp.com wrote: It looks as if you are creating temp files under /var/folders/2y on the remote: are you sure that a) the structure exists and b) permissions are set correctly? Do you have the /var/folders/2y structure on the local host or only on the remote? If only on the remote and the source block is running locally (because of the funcall bug), that would sort-of explain this behavior. /var/folders/2y is peculiar to the local host, my Mac. I am pretty sure that this is a Mac (BSD?) idiom for temp files. /var on the remote is for root only and I have no influence over its permissions. By the way I have also tried this with :session *ssh-midway* in the header pointing to a session started with M-x ssh -- same results. In case more precision about my (org-version) is helpful: Org-mode version 7.9.2 (7.9.2-82-g2aeb28-elpa @ /Users/nbest/.emacs.d/elpa/org-20121112/)
Re: [O] remote execution in heterogeneous environment
On Thu, Nov 29, 2012 at 1:25 PM, Nick Dokos nicholas.do...@hp.com wrote: But your earlier mail says that it is trying to create the temp file on the remote , | Tramp: Decoding region into remote file | /ssh:midway:/var/folders/2y/qrw3hhkx0dlb9sbl51mjy5wmgn/T/sh-510999vN...done | byte-code: Couldn't write region to | `/ssh:midway:/var/folders/2y/qrw3hhkx0dlb9sbl51mjy5wmgn/T/sh-510999vN' ` Not sure what causes tramp to use /var/folders/2y for remote temp files but that seems to be the main problem here. If you evaluate (tramp-compat-temporary-file-directory) on your local system, what do you get? tramp may be assuming that whatever temp directory you are using on your local system should work on the remote as well, an assumption that's not true in your case. You may have to provide your own function to get around this - or use /tmp just like the rest of the world. It's exactly as you suspected, Nick. That crazy /var/folders/... value was apparently hard-coded in the value of temporary-file-directory out of the box. That must be something that happens in Macports. When I changed it to /tmp things appear to work better: #+BEGIN_SRC sh :dir /midway:~ echo Executed by `whoami` on `hostname` in `pwd` #+END_SRC #+RESULTS: : Executed by nbest on lep in /Users/nbest The temp file problem was preventing any results from appearing, but now we see that the code is being executed locally, as suspected. There is a work-around for this, however: M-x ssh. #+BEGIN_SRC sh :session *ssh-midway* echo Executed by `whoami` on `hostname` in `pwd` #+END_SRC #+RESULTS: : Executed by nbest on midway-login1 in /pwd/from/that/session This opens up some possibilities but probably does not address the shortcomings detailed in the thread that Nick points to. The elisp hacking going on there is over my head so I will play with this some more and watch for updates.
Re: [O] remote execution in heterogeneous environment
Neil Best nb...@ci.uchicago.edu writes: Hi, I run Macport's Emacs and ELPA's Org on my Mac but my data lives in a Red Hat environment. I thought I could edit locally and compute remotely but there seems to be a problem with temp files for executing source blocks. #+BEGIN_SRC sh :dir /midway:~ echo Executed by `whoami` on `hostname` in `pwd` #+END_SRC Results are empty and in the message buffers appears this: executing Sh code block... Tramp: Opening connection for midway using ssh... Opening connection for midway using ssh... \ Tramp: Opening connection for midway using ssh...done Tramp: Encoding region using function `base64-encode-region'...done Tramp: Decoding region into remote file /ssh:midway:/var/folders/2y/qrw3hhkx0dlb9sbl51mjy5wmgn/T/sh-510999vN...done byte-code: Couldn't write region to `/ssh:midway:/var/folders/2y/qrw3hhkx0dlb9sbl51mjy5wmgn/T/sh-510999vN', decode using `base64 -d -i %s' failed Which version of Tramp are you using? You find it by C-h v tramp-version. I remember a problem with remote temporary directories, fixed in Tramp 2.2.6. Btw, I don't use org. For local tests, what shall I apply in order to execute the code in #+BEGIN_SRC ... #+END_SRC? Best regards, Michael (Tramp maintainer).
