Re: [O] Erroneous "No such file or directory" with babel and remote dir
Nick Dokos writes: > The first set of code transformations (implemented as commit > 63b5f8f2e85b3059a2d30041db6939347a7a2d7d) dealt with the situation by > doing a mass substitution: flet --> org-flet and labels --> org-labels > (and in at least one case, flet --> org-labels to deal with a > recursive definition - I presume that was a preexisting bug that was > fixed by this substitution) and adding compatibility aliases in > org-compat.el to use the cl-flet/cl-labels macros from cl.el in emacs > versions >= 24.1.50. As has slowly transpired in the meantime, there is no substitution for the deprecated flet. The new cl-flet does lexical instead of dynamical binding of the function slot and the other alternatives have other subtle differences that don't really seem to be documented in a single place. > So the moral of the story is that the code transformations have *not* > left functionality unchanged. Something went awry but to be honest, I > don't know what. I didn't spend much time on it because of what I > found out next. [...] > And is this the only problem? Probably not: every flet->let > transformation would have to be scrutinized. Thanks for this extensive explanation. I don't know if that might convince Stefan Monnier to un-deprecate letf, you'd not be the only one to be rattling his cage on this issue. Regards, Achim. -- +<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+ Waldorf MIDI Implementation & additional documentation: http://Synth.Stromeko.net/Downloads.html#WaldorfDocs
Re: [O] Erroneous "No such file or directory" with babel and remote dir
In a previous mail, I wrote: , | Which tells me that the tmp file error is a red herring and the real | breakage occurred after 5cb80c7, probably through a commit that touched | ob-sh.el (although that's far from guaranteed). Here's that list: | | $ git log --oneline -- lisp/ob-sh.el | 86e515d fix remote execution w/empty shebang header arg | 70dd119 Massive code clean-up. | 966447c Don't use `org-flet' in ob-awk.el and ob-sh.el | 8eb5843 Add punctuation at the end of the first line of docstrings. Code cleanup. | 63b5f8f replace flet/labels with org-flet/org-labels | ecd0562 Fix the master branch. | 6e306f6 Fix copyright years in maint. | de42649 Manually revert maint to e85080. | 73bb18b Manually revert to the Release 7.8.04 tag. | 38c5045 Fix copyright years. | 6e534f9 Manually revert back to commit e85080. | 5cb80c7 apply :shebang and :padline to shell script execution | ... ` Continuing along these lines, I think I've figured it out and it's not pretty. The executive summary is that the last commit in the list introduced a small bug that was fixed by the first commit in the list. But neither of those has much to do with the remote-dir breakage (except that the bug made things harder to bisect). The principal culprits are two sets of commits for code transformations that were supposed to do nothing functionally: they were just supposed to get away from using flet/labels (which are deprecated and will be obsolete by emacs version 24.3.) The end result is that one change fixes the remote dir problem in Loris's example. I'm not sure that it solves every such problem though: I haven't audited all the code. The change is a one-liner: in org-babel-shell-command-on-region, replace the line (call-process-region start end shell-file-name t by (funcall call-process-region start end shell-file-name t BTW, I'm using Org-mode version 7.9.2 (release_7.9.2-582-g6d099e.dirty @ /home/nick/elisp/org-mode/lisp/). The rest of the email explains why the change is needed (in excruciating detail: grab a beer or maybe a cup of coffee before starting on it.) I hope this is (mostly) correct but corrections would be more than welcome. The problem seems multi-faceted and therefore needs as many eyes on it as possible. Nick P.S. I'm not sure whether to thank Loris or to curse him for pushing me on this path, but there is no question that he is responsible for finding the bug, providing the reproducer and then beating on the gates with pitchfork and torch :-) === * Root cause The root cause of the problem was a set of code transformations that were supposed to leave the functionality intact. The code transformations were driven by the need to replace the flet/labels constructs which were declared obsolete (as of 24.3 - they are still available, but cause warnings to be issued). The problem was hard to find because there were four commits (at widely varying times) that contributed to various manifestations of the problem and it was difficult to bisect. In chronological order, they were - commit 5cb80c7 apply :shebang and :padline to shell script execution This did not cause the problem, but it introduced a bug that causes errors when executing a source block with a remote :dir spec. That confused the issue (at least it confused me: I fingered this as the culprit in a bisect, but it was only guilty of the bug fixed by commit 86e515d - see below -, not of the remote-dir problem). - commit 63b5f8f replace flet/labels with org-flet/org-labels Section [[flet --> org-flet]] describes this. - commit e85479a Don't use `org-flet' in some functions and several others that slowly got rid of org-flet in favor of let, and org-labels (somehow - I didn't check this carefully), the latter reverted and reapplied, presumably because problems were found and fixed in several iterations. I've only skimmed the surface here: I think this has the potential to be a minefield of problems waiting to explode - see the [[org-flet --> let]] section below. - commit 86e515d fix remote execution w/empty shebang header arg This finally fixed the little bug that was introduced by 5cb80c7. The first and the last of these commits are irrelevant to the remote-dir problem, except that the bug gets in the way of testing: any version later than 5cb80c7 exhibits the bug and that bug hides whether the remote-dir problem is present or not. The general procedure I followed was to make a branch with some commit as its tip and then manually apply the patch of commit 86e515d. Only then could I test for the remote-dir problem. In the following, when I say commit X, I mean commit X *plus* the manually applied patch from 86e515d. ** flet --> org-flet # < org-flet>> The first set of code transformations (implemented as commit 63b5f8f2e85b3059a2d30041db6939347a7a2d7d) dealt with the situation by doing a mass substitu
Re: [O] Erroneous "No such file or directory" with babel and remote dir
Nick Dokos wrote: > I checked successive versions from 7.01h onwards and found breakage > between release_7.8 and release_7.9. So I bisected and came up > with this as the bad commit: > > , > | commit 5cb80c7e5b9bcae180b799d2a49c78d529e029f0 > | Author: Eric Schulte > | Date: Mon Mar 12 13:23:53 2012 -0400 > | > | apply :shebang and :padline to shell script execution > | > | * lisp/ob-sh.el (org-babel-execute:sh): Pass all params to subroutine. > | (org-babel-sh-evaluate): Apply :shebang and :padline to shell script > | execution. > ` > > Reverting it caused a merge conflict that I didn't have the patience to > resolve. But I made branches, one with this commit as its tip and one > with its predecessor: > > git checkout -b foo 5cb80c7 > git checkout -b bar de09874 > > Testing on foo gives me an error, testing on bar gives the correct > result. So I'm pretty sure this commit introduced the problem. Eric has > applied a partial fix since then which gets rid of the error but still > gives the wrong answer (not Eric's fault: he followed my suggestion, but > I was treating the symptom, not the disease). > > But I think this commit is the place that deserves more scrutiny. > The plot thickens. If I create a branch with 5cb80c7 as the tip and then immediately apply the later patch that fixes the error, then things are working as they should. Here is the procedure in case somebody wants to try to replicate: o First, save the later commit as a patch to apply: git show 86e515d > patch.to.apply o Then go back in time: git checkout -b foo 5cb80c7 That gets you back to 7.8.03 and when I try the code block with emacs -q -l /path/to/minimal.emacs.that.requires.ob-sh /path/to/loris.org I get an error: /bin/bash: /scpc:n...@lefou.usa.hp.com:/tmp/sh-script-6207TKx: No such file or directory o I then apply the patch: git apply patch.to.apply and retry: I get the remote host name as I should. Which tells me that the tmp file error is a red herring and the real breakage occurred after 5cb80c7, probably through a commit that touched ob-sh.el (although that's far from guaranteed). Here's that list: $ git log --oneline -- lisp/ob-sh.el 86e515d fix remote execution w/empty shebang header arg 70dd119 Massive code clean-up. 966447c Don't use `org-flet' in ob-awk.el and ob-sh.el 8eb5843 Add punctuation at the end of the first line of docstrings. Code cleanup. 63b5f8f replace flet/labels with org-flet/org-labels ecd0562 Fix the master branch. 6e306f6 Fix copyright years in maint. de42649 Manually revert maint to e85080. 73bb18b Manually revert to the Release 7.8.04 tag. 38c5045 Fix copyright years. 6e534f9 Manually revert back to commit e85080. 5cb80c7 apply :shebang and :padline to shell script execution ... So unless somebody sees something wrong with the above procedure, we are a bit closer to finding the culprit, but it's certainly not who I thought it was. Nick
Re: [O] Erroneous "No such file or directory" with babel and remote dir
Nick Dokos writes: > Nick Dokos wrote: > >> > Back in the days of release_7.01h, the following used to work: >> > >> > ,-- >> > | #+begin_src sh :dir /loris@othercomputer: >> > | hostname >> > | #+end_src >> > `-- >> > >> > Currently it produces the error: >> > >> > ,--- >> > | apply: Wrong type argument: number-or-marker-p, "" >> > `--- >> > > > BTW, I get no error with the (more or less) current version of > org I'm using: > > Org-mode version 7.9.2 (release_7.9.2-582-g6d099e @ > /home/nick/elisp/org-mode/lisp/) > > I just get the wrong answer: instead of evaluating on the remote, babel > evaluates it locallly. I also get this using 7.9.2-78-gf48a8b-elpa and Emacs 24.2.50.1. However, I then tested this on Emacs 23.1.1 with the following minimal .emacs: ,- | (add-to-list 'load-path "~/elisp/org-mode/lisp") | (require 'org-install) | | (org-babel-do-load-languages | 'org-babel-load-languages | '((emacs-lisp . nil) |(R . t) |(sh . t))) `- and got: ,-- | OVERVIEW | release_7.02 | Org-mode version 7.02 (release_7.02) | Quit | byte-code: Beginning of buffer [2 times] | Loading tramp...done | executing Sh code block... | Wrote /tmp/tramp.6477nIt | Tramp: Opening connection for loris@othercomputer using scp... | Tramp: Waiting 60s for local shell to come up... | Tramp: Sending command `ssh node003 -l loris -q -e none && exit || exit' | Tramp: Waiting for prompts from remote shell | Tramp: Found remote shell prompt on `node003' | Wrote /scp:loris@othercomputer:/tmp/tramp.64770Sz | Wrote /tmp/tramp.6477N7U.6477AxO | Wrote /tmp/scor6477a-m | apply: Wrong type argument: number-or-marker-p, "" `-- So there seem to be two problems, namely 1. Wrong type argument: number-or-marker-p, "", which only occurs in my minimal set-up post release_7.01h 2. No error, but local execution in the current version of Org. So before I can get started looking at 2, I need to sort out 1 The way I am testing is ,-- | git checkout release_7.02 | cd $HOME/elisp/org-mode && make clean | cd $HOME/elisp/org-mode && make | emacs -q -l /home/loris/.emacs_minimal /home/loris/tmp/test.org & `-- Is there something wrong with this? Cheers, Loris >> > I have had another go at bisecting this problem and came up with this: >> > >> > ,- >> > | 9c878a8290c071fbe5e97bc33c300ef2f07d6153 is the first bad commit >> > | commit 9c878a8290c071fbe5e97bc33c300ef2f07d6153 >> > | Author: Dan Davison >> > | Date: Mon Aug 30 09:34:05 2010 -0700 >> > | >> > | babel: Fix temporary file processing in the remote execution case. >> > | >> > | * ob.el (org-babel-temp-file): Don't use babel temporary >> > | directory in remote case; use make-temp-file with remote file >> > | name so that temp file is guaranteed not to exist previously >> > | on remote machine. >> > | (org-babel-tramp-localname): New function to return local name >> > | portion of possibly remote file specification >> > | >> > | * ob-R.el (org-babel-R-evaluate-external-process): Respond to >> > | changes in `org-babel-temp-file'; pass local file name to >> > | remote R process. >> > | (org-babel-R-evaluate-session) Respond to >> > | changes in `org-babel-temp-file'; pass local file name to >> > | remote R process. >> > | >> > | :04 04 b31e072cf6b2951e95b7956d907303e7a04a8cfd >> > 5f794ada52cceb0614fe7962a399f7e549759003 M lisp >> > `- >> > >> > Does that help anyone any further? >> > >> >> I don't think so: afaict, both 7.01h (which did not contain this patch) >> and 7.02 (which did) do the right thing (at least with your simple >> script above). Something else broke it. >> > > I checked successive versions from 7.01h onwards and found breakage > between release_7.8 and release_7.9. So I bisected and came up > with this as the bad commit: > > , > | commit 5cb80c7e5b9bcae180b799d2a49c78d529e029f0 > | Author: Eric Schulte > | Date: Mon Mar 12 13:23:53 2012 -0400 > | > | apply :shebang and :padline to shell script execution > | > | * lisp/ob-sh.el (org-babel-execute:sh): Pass all params to subroutine. > | (org-babel-sh-evaluate): Apply :shebang and :padline to shell sc
Re: [O] Erroneous "No such file or directory" with babel and remote dir
Nick Dokos writes: > Loris Bennett wrote: > >> Achim Gratz writes: >> >> > Loris Bennett writes: >> >> OK, I must have goofed. What I did after starting the bisection was >> >> >> >> - run 'make autoload' >> >> - open test file in emacs with minimal .emacs >> >> - test >> >> - end emacs >> >> - mark bisection good or bad >> >> >> >> I then repeated this for the next commit. >> > >> > This would be the correct thing to do, yes. >> > >> >> Is something missing here, or did I just incorrectly mark a commit? >> > >> > I really don't know, I'm just guessing… :-) >> > >> > It may be more informative if you traced the invocation of the remote >> > execution in both a failing and a working version in the debugger and >> > look for any differences as the arguments are cobbled together. Once >> > you find where an how these diverge it would be much easier to find the >> > code that is responsible for it (if it wasn't glaringly obvious from the >> > trace already). >> > >> > >> > Regards, >> > Achim. >> >> So just as a reminder: >> >> Back in the days of release_7.01h, the following used to work: >> >> ,-- >> | #+begin_src sh :dir /loris@othercomputer: >> | hostname >> | #+end_src >> `-- >> >> Currently it produces the error: >> >> ,--- >> | apply: Wrong type argument: number-or-marker-p, "" >> `--- >> >> I have had another go at bisecting this problem and came up with this: >> >> ,- >> | 9c878a8290c071fbe5e97bc33c300ef2f07d6153 is the first bad commit >> | commit 9c878a8290c071fbe5e97bc33c300ef2f07d6153 >> | Author: Dan Davison >> | Date: Mon Aug 30 09:34:05 2010 -0700 >> | >> | babel: Fix temporary file processing in the remote execution case. >> | >> | * ob.el (org-babel-temp-file): Don't use babel temporary >> | directory in remote case; use make-temp-file with remote file >> | name so that temp file is guaranteed not to exist previously >> | on remote machine. >> | (org-babel-tramp-localname): New function to return local name >> | portion of possibly remote file specification >> | >> | * ob-R.el (org-babel-R-evaluate-external-process): Respond to >> | changes in `org-babel-temp-file'; pass local file name to >> | remote R process. >> | (org-babel-R-evaluate-session) Respond to >> | changes in `org-babel-temp-file'; pass local file name to >> | remote R process. >> | >> | :04 04 b31e072cf6b2951e95b7956d907303e7a04a8cfd >> 5f794ada52cceb0614fe7962a399f7e549759003 M lisp >> `- >> >> Does that help anyone any further? >> > > I don't think so: afaict, both 7.01h (which did not contain this patch) > and 7.02 (which did) do the right thing (at least with your simple > script above). Something else broke it. > > [I've been wrong about these things before so before Achim asks, I > did > > git tag --contains 9c878a8290c071fbe5e97bc33c300ef2f07d6153 > > and used its output as a basis of the above statement. I think (hope?) > it's correct.] > >> Loris >> >> PS: I am amazed there aren't any more peasants like myself with torches >> and pitchforks beating on the gates of Castle Org about this - remote >> execution is/was such great feature! >> > > Agreed, but Dan Davison (the principal contributor of the remoting code) > is gone from Castle Org (as you put it), Eric S. is busy, and nobody > else has stepped up to the plate. I've made fitful attempts to debug it, > but have had no success: I've never looked at the code in detail and I > don't have much time either. > > So how about becoming a resident of Castle Org rather than beating on > the gates? There's glory galore if you fix it - and you get to scratch > your itch too :-) OK, I don't have that much time either but, as my Lisp skills are pretty rudimentary, I've often thought it might be a good idea to take on some sort of lisp project, so I'll try to give it a go. Cheers, Loris -- no sig is good sig
Re: [O] Erroneous "No such file or directory" with babel and remote dir
Nick Dokos wrote: > > Back in the days of release_7.01h, the following used to work: > > > > ,-- > > | #+begin_src sh :dir /loris@othercomputer: > > | hostname > > | #+end_src > > `-- > > > > Currently it produces the error: > > > > ,--- > > | apply: Wrong type argument: number-or-marker-p, "" > > `--- > > BTW, I get no error with the (more or less) current version of org I'm using: Org-mode version 7.9.2 (release_7.9.2-582-g6d099e @ /home/nick/elisp/org-mode/lisp/) I just get the wrong answer: instead of evaluating on the remote, babel evaluates it locallly. > > I have had another go at bisecting this problem and came up with this: > > > > ,- > > | 9c878a8290c071fbe5e97bc33c300ef2f07d6153 is the first bad commit > > | commit 9c878a8290c071fbe5e97bc33c300ef2f07d6153 > > | Author: Dan Davison > > | Date: Mon Aug 30 09:34:05 2010 -0700 > > | > > | babel: Fix temporary file processing in the remote execution case. > > | > > | * ob.el (org-babel-temp-file): Don't use babel temporary > > | directory in remote case; use make-temp-file with remote file > > | name so that temp file is guaranteed not to exist previously > > | on remote machine. > > | (org-babel-tramp-localname): New function to return local name > > | portion of possibly remote file specification > > | > > | * ob-R.el (org-babel-R-evaluate-external-process): Respond to > > | changes in `org-babel-temp-file'; pass local file name to > > | remote R process. > > | (org-babel-R-evaluate-session) Respond to > > | changes in `org-babel-temp-file'; pass local file name to > > | remote R process. > > | > > | :04 04 b31e072cf6b2951e95b7956d907303e7a04a8cfd > > 5f794ada52cceb0614fe7962a399f7e549759003 M lisp > > `- > > > > Does that help anyone any further? > > > > I don't think so: afaict, both 7.01h (which did not contain this patch) > and 7.02 (which did) do the right thing (at least with your simple > script above). Something else broke it. > I checked successive versions from 7.01h onwards and found breakage between release_7.8 and release_7.9. So I bisected and came up with this as the bad commit: , | commit 5cb80c7e5b9bcae180b799d2a49c78d529e029f0 | Author: Eric Schulte | Date: Mon Mar 12 13:23:53 2012 -0400 | | apply :shebang and :padline to shell script execution | | * lisp/ob-sh.el (org-babel-execute:sh): Pass all params to subroutine. | (org-babel-sh-evaluate): Apply :shebang and :padline to shell script | execution. ` Reverting it caused a merge conflict that I didn't have the patience to resolve. But I made branches, one with this commit as its tip and one with its predecessor: git checkout -b foo 5cb80c7 git checkout -b bar de09874 Testing on foo gives me an error, testing on bar gives the correct result. So I'm pretty sure this commit introduced the problem. Eric has applied a partial fix since then which gets rid of the error but still gives the wrong answer (not Eric's fault: he followed my suggestion, but I was treating the symptom, not the disease). But I think this commit is the place that deserves more scrutiny. HTH, Nick
Re: [O] Erroneous "No such file or directory" with babel and remote dir
Loris Bennett wrote: > Achim Gratz writes: > > > Loris Bennett writes: > >> OK, I must have goofed. What I did after starting the bisection was > >> > >> - run 'make autoload' > >> - open test file in emacs with minimal .emacs > >> - test > >> - end emacs > >> - mark bisection good or bad > >> > >> I then repeated this for the next commit. > > > > This would be the correct thing to do, yes. > > > >> Is something missing here, or did I just incorrectly mark a commit? > > > > I really don't know, I'm just guessing… :-) > > > > It may be more informative if you traced the invocation of the remote > > execution in both a failing and a working version in the debugger and > > look for any differences as the arguments are cobbled together. Once > > you find where an how these diverge it would be much easier to find the > > code that is responsible for it (if it wasn't glaringly obvious from the > > trace already). > > > > > > Regards, > > Achim. > > So just as a reminder: > > Back in the days of release_7.01h, the following used to work: > > ,-- > | #+begin_src sh :dir /loris@othercomputer: > | hostname > | #+end_src > `-- > > Currently it produces the error: > > ,--- > | apply: Wrong type argument: number-or-marker-p, "" > `--- > > I have had another go at bisecting this problem and came up with this: > > ,- > | 9c878a8290c071fbe5e97bc33c300ef2f07d6153 is the first bad commit > | commit 9c878a8290c071fbe5e97bc33c300ef2f07d6153 > | Author: Dan Davison > | Date: Mon Aug 30 09:34:05 2010 -0700 > | > | babel: Fix temporary file processing in the remote execution case. > | > | * ob.el (org-babel-temp-file): Don't use babel temporary > | directory in remote case; use make-temp-file with remote file > | name so that temp file is guaranteed not to exist previously > | on remote machine. > | (org-babel-tramp-localname): New function to return local name > | portion of possibly remote file specification > | > | * ob-R.el (org-babel-R-evaluate-external-process): Respond to > | changes in `org-babel-temp-file'; pass local file name to > | remote R process. > | (org-babel-R-evaluate-session) Respond to > | changes in `org-babel-temp-file'; pass local file name to > | remote R process. > | > | :04 04 b31e072cf6b2951e95b7956d907303e7a04a8cfd > 5f794ada52cceb0614fe7962a399f7e549759003 M lisp > `- > > Does that help anyone any further? > I don't think so: afaict, both 7.01h (which did not contain this patch) and 7.02 (which did) do the right thing (at least with your simple script above). Something else broke it. [I've been wrong about these things before so before Achim asks, I did git tag --contains 9c878a8290c071fbe5e97bc33c300ef2f07d6153 and used its output as a basis of the above statement. I think (hope?) it's correct.] > Loris > > PS: I am amazed there aren't any more peasants like myself with torches > and pitchforks beating on the gates of Castle Org about this - remote > execution is/was such great feature! > Agreed, but Dan Davison (the principal contributor of the remoting code) is gone from Castle Org (as you put it), Eric S. is busy, and nobody else has stepped up to the plate. I've made fitful attempts to debug it, but have had no success: I've never looked at the code in detail and I don't have much time either. So how about becoming a resident of Castle Org rather than beating on the gates? There's glory galore if you fix it - and you get to scratch your itch too :-) Nick
Re: [O] Erroneous "No such file or directory" with babel and remote dir
Achim Gratz writes: > Loris Bennett writes: >> OK, I must have goofed. What I did after starting the bisection was >> >> - run 'make autoload' >> - open test file in emacs with minimal .emacs >> - test >> - end emacs >> - mark bisection good or bad >> >> I then repeated this for the next commit. > > This would be the correct thing to do, yes. > >> Is something missing here, or did I just incorrectly mark a commit? > > I really don't know, I'm just guessing… :-) > > It may be more informative if you traced the invocation of the remote > execution in both a failing and a working version in the debugger and > look for any differences as the arguments are cobbled together. Once > you find where an how these diverge it would be much easier to find the > code that is responsible for it (if it wasn't glaringly obvious from the > trace already). > > > Regards, > Achim. So just as a reminder: Back in the days of release_7.01h, the following used to work: ,-- | #+begin_src sh :dir /loris@othercomputer: | hostname | #+end_src `-- Currently it produces the error: ,--- | apply: Wrong type argument: number-or-marker-p, "" `--- I have had another go at bisecting this problem and came up with this: ,- | 9c878a8290c071fbe5e97bc33c300ef2f07d6153 is the first bad commit | commit 9c878a8290c071fbe5e97bc33c300ef2f07d6153 | Author: Dan Davison | Date: Mon Aug 30 09:34:05 2010 -0700 | | babel: Fix temporary file processing in the remote execution case. | | * ob.el (org-babel-temp-file): Don't use babel temporary | directory in remote case; use make-temp-file with remote file | name so that temp file is guaranteed not to exist previously | on remote machine. | (org-babel-tramp-localname): New function to return local name | portion of possibly remote file specification | | * ob-R.el (org-babel-R-evaluate-external-process): Respond to | changes in `org-babel-temp-file'; pass local file name to | remote R process. | (org-babel-R-evaluate-session) Respond to | changes in `org-babel-temp-file'; pass local file name to | remote R process. | | :04 04 b31e072cf6b2951e95b7956d907303e7a04a8cfd 5f794ada52cceb0614fe7962a399f7e549759003 M lisp `- Does that help anyone any further? Cheers, Loris PS: I am amazed there aren't any more peasants like myself with torches and pitchforks beating on the gates of Castle Org about this - remote execution is/was such great feature! -- no sig is good sig
Re: [O] Erroneous "No such file or directory" with babel and remote dir
Achim Gratz writes: > Loris Bennett writes: >> OK, I must have goofed. What I did after starting the bisection was >> >> - run 'make autoload' >> - open test file in emacs with minimal .emacs >> - test >> - end emacs >> - mark bisection good or bad >> >> I then repeated this for the next commit. > > This would be the correct thing to do, yes. > >> Is something missing here, or did I just incorrectly mark a commit? > > I really don't know, I'm just guessing… :-) > > It may be more informative if you traced the invocation of the remote > execution in both a failing and a working version in the debugger and > look for any differences as the arguments are cobbled together. Once > you find where an how these diverge it would be much easier to find the > code that is responsible for it (if it wasn't glaringly obvious from the > trace already). > > > Regards, > Achim. [Sorry if this appears twice - Gnus got the better of me] So just as a reminder: Back in the days of release_7.01h, the following used to work: ,-- | #+begin_src sh :dir /loris@othercomputer: | hostname | #+end_src `-- Currently it produces the error: ,--- | apply: Wrong type argument: number-or-marker-p, "" `--- I have had another go at bisecting this problem and came up with this: ,- | 9c878a8290c071fbe5e97bc33c300ef2f07d6153 is the first bad commit | commit 9c878a8290c071fbe5e97bc33c300ef2f07d6153 | Author: Dan Davison | Date: Mon Aug 30 09:34:05 2010 -0700 | | babel: Fix temporary file processing in the remote execution case. | | * ob.el (org-babel-temp-file): Don't use babel temporary | directory in remote case; use make-temp-file with remote file | name so that temp file is guaranteed not to exist previously | on remote machine. | (org-babel-tramp-localname): New function to return local name | portion of possibly remote file specification | | * ob-R.el (org-babel-R-evaluate-external-process): Respond to | changes in `org-babel-temp-file'; pass local file name to | remote R process. | (org-babel-R-evaluate-session) Respond to | changes in `org-babel-temp-file'; pass local file name to | remote R process. | | :04 04 b31e072cf6b2951e95b7956d907303e7a04a8cfd 5f794ada52cceb0614fe7962a399f7e549759003 M lisp `- Does that help anyone any further? Cheers, Loris PS: I am amazed there aren't any more peasants like myself with torches and pitchforks beating on the gates of Castle Org about this - remote execution is/was such great feature! Dr. Loris Bennett (Mr.) ZEDAT, Freie Universität Berlin Email loris.benn...@fu-berlin.de
Re: [O] Erroneous "No such file or directory" with babel and remote dir
Loris Bennett writes: > OK, I must have goofed. What I did after starting the bisection was > > - run 'make autoload' > - open test file in emacs with minimal .emacs > - test > - end emacs > - mark bisection good or bad > > I then repeated this for the next commit. This would be the correct thing to do, yes. > Is something missing here, or did I just incorrectly mark a commit? I really don't know, I'm just guessing… :-) It may be more informative if you traced the invocation of the remote execution in both a failing and a working version in the debugger and look for any differences as the arguments are cobbled together. Once you find where an how these diverge it would be much easier to find the code that is responsible for it (if it wasn't glaringly obvious from the trace already). Regards, Achim. -- +<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+ Factory and User Sound Singles for Waldorf rackAttack: http://Synth.Stromeko.net/Downloads.html#WaldorfSounds
Re: [O] Erroneous "No such file or directory" with babel and remote dir
Achim Gratz writes: > Loris Bennett writes: >> OK. I don't really get what is going on here, as this is my first >> attempt at bisecting. The bad commit does seem to be among those >> skipped: > > I've done a diff around those commits and there seems nothing related to > your problem. So I'm afraid you've marked another commit as "bad" that > rather should have been "skip"ped. The commits you "skip" have > problems, but unrelated to the bug you are hunting — it's important to > mark them correctly. > > > Regards, > Achim. OK, I must have goofed. What I did after starting the bisection was - run 'make autoload' - open test file in emacs with minimal .emacs - test - end emacs - mark bisection good or bad I then repeated this for the next commit. Is something missing here, or did I just incorrectly mark a commit? Cheers, Loris -- no sig is good sig
Re: [O] Erroneous "No such file or directory" with babel and remote dir
Loris Bennett writes: > OK. I don't really get what is going on here, as this is my first > attempt at bisecting. The bad commit does seem to be among those > skipped: I've done a diff around those commits and there seems nothing related to your problem. So I'm afraid you've marked another commit as "bad" that rather should have been "skip"ped. The commits you "skip" have problems, but unrelated to the bug you are hunting — it's important to mark them correctly. Regards, Achim. -- +<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+ Wavetables for the Waldorf Blofeld: http://Synth.Stromeko.net/Downloads.html#BlofeldUserWavetables
Re: [O] Erroneous "No such file or directory" with babel and remote dir
Achim Gratz writes: > Loris Bennett writes: >>> Can you try to bisect and find the bad commit (or a set of >>> suspicious ones)? >> >> 552b0edb254a104e441e28f3a942dc6005e97f87 is the first bad commit >> commit 552b0edb254a104e441e28f3a942dc6005e97f87 >> Author: Bastien Guerry >> Date: Sat Mar 17 15:44:41 2012 +0100 > > This is not the commit you're looking for... > > Please "skip" these commits after starting the bisect by doing > > git bisect skip 7e903acccd..df82832fb7 > > If the offending change is inside that range we can still deal with it > later. > > > Regards, > Achim. OK. I don't really get what is going on here, as this is my first attempt at bisecting. The bad commit does seem to be among those skipped: $ git bisect bad There are only 'skip'ped commits left to test. The first bad commit could be any of: 095e5786761d4edd51dfdfc9a69cd22b942096fa b0e24a2b5e3ce872e9eb43462f03edf616438f57 cdd9e2b91a7b9257e4f94e0aacc8e470ac1c951a 73cbeabe1b457da1364547d7f2d89db889bec926 9382faf5927f84a2c496ae677d99b64517abeeb2 05847efd4ace13fda174c2f1d8dc354eb217ea35 c63275ca8ea46e0e96ab7ed70efe6bb937c39b29 82019bc7c7a786303e82cfa5891193acb3754606 b2913c9227504ffd31d616482d258f5b10f821fb 49dd6de08384f913f014ac06c24da4ff08ccdb6a dab84afa7592e842ae866881c532cd60944bb0da 552b0edb254a104e441e28f3a942dc6005e97f87 b32d2f737d567222bdbdf742329b9a34d0dd069a a1a01224d4c1f76761e3562c54020c04bfd3ac9b ecd0562c5f6e69ce0e4546d9f64d8e89eda8eef1 5ed0384e51449e6462eed3a6ff4ecba6a6f52790 6bcba149195c2c281eefc3065dabe2ec1fdea687 4e5c24ea81c6758241a828e28ea44b75d36115e3 856f5c27fb11a1f034c80fc29c39f211434fc650 06b94ca884733eaefb0ec868d641da78ed536ceb df82832fb7525e5d57f1e968334e1057329e95f2 6c911234ac66d62d99cb0c996dc10c938b2b3679 We cannot bisect more! -- Dr. Loris Bennett (Mr.) ZEDAT, Freie Universität Berlin Email loris.benn...@fu-berlin.de
Re: [O] Erroneous "No such file or directory" with babel and remote dir
Achim Gratz wrote:
> Loris Bennett writes:
> >> Can you try to bisect and find the bad commit (or a set of
> >> suspicious ones)?
> >
> > 552b0edb254a104e441e28f3a942dc6005e97f87 is the first bad commit
> > commit 552b0edb254a104e441e28f3a942dc6005e97f87
> > Author: Bastien Guerry
> > Date: Sat Mar 17 15:44:41 2012 +0100
>
> This is not the commit you're looking for...
>
> Please "skip" these commits after starting the bisect by doing
>
> git bisect skip 7e903acccd..df82832fb7
>
> If the offending change is inside that range we can still deal with it
> later.
>
>
I sent mail to Eric with some partial findings about this, but I guess I
should have sent it to the list as well. Here is what I sent to Eric
(slightly edited by adding a few commas):
--8<---cut here---start->8---
Eric,
I went back to the above problem (see
http://thread.gmane.org/gmane.emacs.orgmode/57152/focus=60469 and the
associated thread) and I think I have taken it a bit further.
5cb80c7e5b9bca introduced the following bit of code at the end of
org-babel-sh-evaluate:
,
| ('otherwise ; external shell script
| - (org-babel-eval org-babel-sh-command (org-babel-trim body))
| + (if (cdr (assoc :shebang params))
| + (let ((script-file (org-babel-temp-file "sh-script-"))
| +(shebang (cdr (assoc :shebang params)))
| +(padline (not (string= "no" (cdr (assoc :padline params))
| +(with-temp-file script-file
| + (when shebang (insert (concat shebang "\n")))
| + (when padline (insert "\n"))
| + (insert body))
| +(set-file-modes script-file #o755)
| +(org-babel-eval script-file ""))
| + (org-babel-eval org-babel-sh-command (org-babel-trim body)))
`
So before, there was no if and we went straight into the org-babel-eval.
Now, the ``if'' seems to succeed all the time: the (cdr ..) returns "" with
Loris's example and (org-babel-eval script-file "") returns nil.
If I modify the condition so that it takes the false path and executes
the (org-babel-eval org-babel-sh-command (org-babel-trim body)) form,
it works.
I have no idea if this is correct - but it does seem to me that in this
particular case, it's taking the wrong branch: in particular, if no
shebang is specified, it should go the way it used to before this
change. Of course, there is also the problem of specifying a
shebang *and* a remote dir which is not handled correctly by this
modification.
For the record, the change I made was:
- (if (cdr (assoc :shebang params))
+ (if (not (string= (cdr (assoc :shebang params)) ""))
It's at the very least incomplete and it may be wrong, but I thought
I'd let you know.
--8<---cut here---end--->8---
HTH,
Nick
Re: [O] Erroneous "No such file or directory" with babel and remote dir
Loris Bennett writes: >> Can you try to bisect and find the bad commit (or a set of >> suspicious ones)? > > 552b0edb254a104e441e28f3a942dc6005e97f87 is the first bad commit > commit 552b0edb254a104e441e28f3a942dc6005e97f87 > Author: Bastien Guerry > Date: Sat Mar 17 15:44:41 2012 +0100 This is not the commit you're looking for... Please "skip" these commits after starting the bisect by doing git bisect skip 7e903acccd..df82832fb7 If the offending change is inside that range we can still deal with it later. Regards, Achim. -- +<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+ SD adaptation for Waldorf Blofeld V1.15B11: http://Synth.Stromeko.net/Downloads.html#WaldorfSDada
Re: [O] Erroneous "No such file or directory" with babel and remote dir
Bastien writes: > Hi Loris, > > "Loris Bennett" writes: > >> I have updated from 7.8.something to 7.9.1 and evaluating the following >> source block: >> >> ,-- >> | #+header: :cache no :eval query >> | #+name: sacct-output >> | #+begin_src sh :exports both :dir /root@xx: >> | module load shared slurm >> | sacct -d -S 2011-11-01 | grep JOB_TERMINATED | cut -d ' ' -f 4,23 | grep >> -v ' 0$' >> | #+end_src >> `-- >> >> used to work, but now produces the error: > > Can you try to bisect and find the bad commit (or a set of > suspicious ones)? > > Thanks, 552b0edb254a104e441e28f3a942dc6005e97f87 is the first bad commit commit 552b0edb254a104e441e28f3a942dc6005e97f87 Author: Bastien Guerry Date: Sat Mar 17 15:44:41 2012 +0100 Manually revert to 78ec8e21 (Release 7.8.04) Let's take a fresh start. Sorry Mama. :04 04 163e6eefeae7ce8287a9b95376a182725463b6e4 00f1fc024be6f61d8f7b520be509d46555418adb M EXPERIMENTAL :100644 100644 1753dd8d44eab0fc512284172e8902302959bbab 1022cdc9ff14c9d7f3004c4181507acf9f7414a2 M Makefile :100644 100644 ad822658a8954b7d02b79f2cccd9c9b721b3e889 cf67c9933ccbc13676b6b9ded82bb62c3ab67220 M README_DIST :04 04 35c93a0ab0b246483543b14ba3cb218dae9172e1 46f1a162838f9234bf0f34ee202ad88690af8e5e M contrib :04 04 e680a29de46430d26ee79a24abd7ec4f6cec28ba c34f8f62ceced35f6519935bbe0404b95a6d5b4d M doc :04 04 5989cfb368b5bee95e1fcceb42a88c381d72bdee 012e82d1831a339b52148716cd9a1cad7d3159f9 M etc :04 04 7ef0c8f2ecdd18ef48acfc0e13815ac6e8b01d64 d2895eead691b78d01435a0c9ba45c161e462f39 M lisp :04 04 99d7331b0f659fe1b1e6323cb5a33c66152e7d67 e2e86614d689cef98523b2d6f6c8f3b1dea5ddff M testing Loris -- no sig is good sig
Re: [O] Erroneous "No such file or directory" with babel and remote dir
Hi Loris, "Loris Bennett" writes: > Thanks, Achim, that worked. But what exactly does it do? let me try -- I'm confident Achim will correct me if I'm wrong or incomplete. `make autoloads' creates two files in your lisp/ directory: - org-install.el - org-version.el The first one declares the functions and macros that should be autoloaded with this version of Org. Those functions/macros are always accessible, even when the library that contain them is not explicitely loaded. Calling one of those functions/macros will load the library. See the Emacs manual for further details. The second one defines the correct version for your Org distrib and it is further checked by (org-version). If it does not exist, systems that have Git will try to infer the version number from Git (checking against the latest tag, which is always of the form "release_X.X[.X]". If org-version.el has not been created and Git is not available, you will end up with a "N/A" version number. Note that org-version.el and org-install.el are both included in the .tar.gz/.zip distribution files. HTH, -- Bastien
Re: [O] Erroneous "No such file or directory" with babel and remote dir
Achim Gratz writes: > Loris Bennett writes: >> How do I avoid the mixed installation problem when testing with a clone >> of the org repository? My version is >> >> , >> | Org-mode version 7.9.1 (release_7.9.1-git @ mixed installation! >> | /usr/local/share/emacs/site-lisp/ and >> | /home/loris/git/org-mode.git/lisp/) >> ` > > Assuming that you've set up the load-path correctly, this should correct > itself when you do a > > make autoloads > > in the Git working tree. > > > Regards, > Achim. Thanks, Achim, that worked. But what exactly does it do? Cheers, Loris -- no sig is good sig
Re: [O] Erroneous "No such file or directory" with babel and remote dir
Loris Bennett writes: > How do I avoid the mixed installation problem when testing with a clone > of the org repository? My version is > > , > | Org-mode version 7.9.1 (release_7.9.1-git @ mixed installation! > | /usr/local/share/emacs/site-lisp/ and > | /home/loris/git/org-mode.git/lisp/) > ` Assuming that you've set up the load-path correctly, this should correct itself when you do a make autoloads in the Git working tree. Regards, Achim. -- +<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+ Waldorf MIDI Implementation & additional documentation: http://Synth.Stromeko.net/Downloads.html#WaldorfDocs
Re: [O] Erroneous "No such file or directory" with babel and remote dir
"Loris Bennett" writes: > How do I avoid the mixed installation problem when testing with a clone > of the org repository? ,[ http://orgmode.org/worg/org-faq.html#mixed-install ] | Among the most common reasons is Orgmode gets loaded before the | load-path variable is updated to include the installation directory of | the latest Orgmode. To avoid issues like this, it is recommended that | the load path is updated very early on in your init file. ` hth Memnon
Re: [O] Erroneous "No such file or directory" with babel and remote dir
Bastien writes: > Hi Loris, > > "Loris Bennett" writes: > >> I have updated from 7.8.something to 7.9.1 and evaluating the following >> source block: >> >> ,-- >> | #+header: :cache no :eval query >> | #+name: sacct-output >> | #+begin_src sh :exports both :dir /root@xx: >> | module load shared slurm >> | sacct -d -S 2011-11-01 | grep JOB_TERMINATED | cut -d ' ' -f 4,23 | grep >> -v ' 0$' >> | #+end_src >> `-- >> >> used to work, but now produces the error: > > Can you try to bisect and find the bad commit (or a set of > suspicious ones)? > > Thanks, How do I avoid the mixed installation problem when testing with a clone of the org repository? My version is , | Org-mode version 7.9.1 (release_7.9.1-git @ mixed installation! | /usr/local/share/emacs/site-lisp/ and | /home/loris/git/org-mode.git/lisp/) ` Loris -- no sig is good sig
Re: [O] Erroneous "No such file or directory" with babel and remote dir
Hi Loris, "Loris Bennett" writes: > I have updated from 7.8.something to 7.9.1 and evaluating the following > source block: > > ,-- > | #+header: :cache no :eval query > | #+name: sacct-output > | #+begin_src sh :exports both :dir /root@xx: > | module load shared slurm > | sacct -d -S 2011-11-01 | grep JOB_TERMINATED | cut -d ' ' -f 4,23 | grep -v > ' 0$' > | #+end_src > `-- > > used to work, but now produces the error: Can you try to bisect and find the bad commit (or a set of suspicious ones)? Thanks, -- Bastien
Re: [O] Erroneous "No such file or directory" with babel and remote dir
"Loris Bennett" writes: > Dear List, > > I have updated from 7.8.something to 7.9.1 and evaluating the following > source block: > > ,-- > | #+header: :cache no :eval query > | #+name: sacct-output > | #+begin_src sh :exports both :dir /root@xx: > | module load shared slurm > | sacct -d -S 2011-11-01 | grep JOB_TERMINATED | cut -d ' ' -f 4,23 | grep -v > ' 0$' > | #+end_src > `-- > > used to work, but now produces the error: > > ,- > | bin/bash: /scp:root@xx:/tmp/sh-script-2829ZfG: No such file or > directory > `- I should mention here that the file does in fact exist and does the right thing when executed on the remote host. > Have I overlooked some change? > > Cheers, > > Loris > > > > > -- Dr. Loris Bennett (Mr.) ZEDAT, Freie Universität Berlin Email loris.benn...@fu-berlin.de
