Re: [O] [BUG] Infinite loop in org-agenda-show-new-time
Matt Lundin writes: > When the agenda buffer is filtered by tag, I find that rescheduling an > item (via org-agenda-do-date-later, org-agenda-schedule, etc.) often > results in an infinite loop. > > I believe this loop occurs in the function org-agenda-show-new-time. > > Here are the steps I took to debug this: > > 1. I add a counter to the function: > > > 2. I narrow an agenda diary buffer by tag (e.g., "home"). > > 3. I reschedule an item. > > 4. Emacs hangs. > > 5. I hit C-g to stop the process. > > 6. I check the value of my-counter. The longer I let the process run, > the higher the value of my-counter. E.g., letting it run for > approximately ten seconds results in the following: > > my-counter's value is 32193 > My one feeble attempt to reproduce this failed. Looking at the code --8<---cut here---start->8--- (while (not (bobp)) (when (equal marker (org-get-at-bol 'org-marker)) (org-move-to-column (- (window-width) (length stamp)) t) (org-agenda-fix-tags-filter-overlays-at (point)) ... (beginning-of-line 1)) (beginning-of-line 0) --8<---cut here---end--->8--- let's assume we are not at the beginning of the buffer, so we don't exit the loop that way. If the when succeeds, we do a couple of things and then do (beginning-of-line 1). This just takes us to the beginning of the current line. But after the when is done, we do (beginning-of-line 0) which should take us to the previous line. So we should be making steady progress towards the beginning of the buffer and the loop should terminate. Since you can reproduce it (and you've already done the hard work of figuring out where the inf loop is), maybe you can edebug the function and step through it a couple of times to see what's happening. -- Nick
Re: [O] [BUG] Infinite loop in org-agenda-show-new-time
Nick Dokos writes: > Matt Lundin writes: > > My one feeble attempt to reproduce this failed. Looking at the code Here are the steps to reproduce the problem: 1. Create file test.org with the following content: --8<---cut here---start->8--- * TODO A :home: SCHEDULED: <2013-08-05 Mon> * TODO B :work: SCHEDULED: <2013-08-05 Mon> * TODO C :play: SCHEDULED: <2013-08-05 Mon> * TODO D :home: SCHEDULED: <2013-08-05 Mon> * TODO E :work: SCHEDULED: <2013-08-05 Mon> * TODO F :play: SCHEDULED: <2013-08-05 Mon> * TODO G :home: SCHEDULED: <2013-08-05 Mon> * TODO H :work: SCHEDULED: <2013-08-05 Mon> * TODO I :play: SCHEDULED: <2013-08-05 Mon> --8<---cut here---end--->8--- 2. /usr/bin/emacs -Q 3. find file test.org 4. M-x org-agenda -> hit "<" to restrict to buffer and then "a" for diary 5. / home 6. Attempt to reschedule one of the visible items. > (while (not (bobp)) > (when (equal marker (org-get-at-bol 'org-marker)) > (org-move-to-column (- (window-width) (length stamp)) t) > (org-agenda-fix-tags-filter-overlays-at (point)) > ... > (beginning-of-line 1)) > (beginning-of-line 0) > > let's assume we are not at the beginning of the buffer, so we don't exit > the loop that way. If the when succeeds, we do a couple of things and > then do (beginning-of-line 1). This just takes us to the beginning of > the current line. But after the when is done, we do (beginning-of-line > 0) which should take us to the previous line. So we should be making > steady progress towards the beginning of the buffer and the loop should > terminate. > > Since you can reproduce it (and you've already done the hard work of > figuring out where the inf loop is), maybe you can edebug the function > and step through it a couple of times to see what's happening. Thanks for the pointers. Running edebug with the file above reveals that org-move-to-column is not working with the invisible sections of the buffer. With the sample file above, I filter the agenda to display only items tagged :home:. --8<---cut here---start->8--- Day-agenda (W32): Monday 5 August 2013 W32 test: Scheduled: TODO A :home: test: Scheduled: TODO D :home: test: Scheduled: TODO G :home: --8<---cut here---end--->8--- When stepping through org-agenda-do-date-later, edebug reveals that the point goes to the end of the buffer, as expected and then works its way backward. When it arrives at the beginning of the line with task "G", it finds and match and executes the following functions: --8<---cut here---start->8--- (org-move-to-column (- (window-width) (length stamp)) t) (org-agenda-fix-tags-filter-overlays-at (point)) --8<---cut here---end--->8--- The problem is that org-move-to-column shifts the point several lines forward. In fact, if I make all contents of the agenda buffer visible after edebug executes org-move-to-column, I find that the point is now all the way at the end of line "I," which, of course, will trigger an endless loop. In other words, org-move-to-column moves the point to the end of the entire invisible section. --8<---cut here---start->8--- Day-agenda (W32): Monday 5 August 2013 W32 test: Scheduled: TODO A :home: test: Scheduled: TODO B :work: test: Scheduled: TODO C :play: test: Scheduled: TODO D :home: test: Scheduled: TODO E :work: test: Scheduled: TODO F :play: test: Scheduled: TODO G :home: test: Scheduled: TODO H :work: test: Scheduled: TODO I :play: --8<---cut here---end--->8---
Re: [O] [BUG] Infinite loop in org-agenda-show-new-time
Matt Lundin writes: > Nick Dokos writes: > >> Matt Lundin writes: >> >> My one feeble attempt to reproduce this failed. Looking at the code > > Here are the steps to reproduce the problem: > Thanks for the recipe: I still can't reproduce it - it just works for me. Org-mode version 8.0.7 (release_8.0.7-367-gd1d918 @ /home/nick/elisp/org-mode/lisp/) GNU Emacs 24.3.50.2 (x86_64-unknown-linux-gnu, GTK+ Version 2.24.10) of 2013-07-14 on pierrot > > Thanks for the pointers. Running edebug with the file above reveals that > org-move-to-column is not working with the invisible sections of the > buffer. > > With the sample file above, I filter the agenda to display only items > tagged :home:. > > Day-agenda (W32): > Monday 5 August 2013 W32 > test: Scheduled: TODO A > :home: > test: Scheduled: TODO D > :home: > test: Scheduled: TODO G > :home: > > When stepping through org-agenda-do-date-later, edebug reveals that the > point goes to the end of the buffer, as expected and then works its way > backward. When it arrives at the beginning of the line with task "G", it > finds and match and executes the following functions: > > (org-move-to-column (- (window-width) (length stamp)) t) > (org-agenda-fix-tags-filter-overlays-at (point)) > > The problem is that org-move-to-column shifts the point several lines > forward. In fact, if I make all contents of the agenda buffer visible > after edebug executes org-move-to-column, I find that the point is now > all the way at the end of line "I," which, of course, will trigger an > endless loop. In other words, org-move-to-column moves the point to the > end of the entire invisible section. > org-move-to-column is just a compatibility function that calls move-to-column whose doc says: Move point to column COLUMN in the current line. So I'm not sure why it ends up on a different line. > This bug was introduced with the following commit: > > commit fafb5f3429c41cba1eddb9fc78d9f9e0980acbe2 > Author: Bastien Guerry > Date: Mon Feb 11 14:56:38 2013 +0100 > > org-agenda.el: Fix bug when displaying a temporary overlay > > * org-agenda.el (org-agenda-schedule, org-agenda-deadline): > Cosmetic changes. > (org-agenda-show-new-time): Fix bug when displaying a > temporary overlay with the scheduled/deadline information. > > Thanks to Thomas Morgan for reporting this bug and testing the patch. > > This commit removed the local binding of buffer-invisibility-spec to > nil in org-agenda-show-new-time. > > Here is the bug this change was meant to fix: > > http://permalink.gmane.org/gmane.emacs.orgmode/52667 > > Might we revert this change? The original bug was cosmetic. This bug, > however, interferes in an essential way with the functioning of agenda > buffers. > So you revert that commit and the infloop goes away? I looked at the functions briefly and I don't understand how this commit could affect what move-to-column does. Can you provide emacs and org versions? Maybe there is something weird in what you use. -- Nick
Re: [O] [BUG] Infinite loop in org-agenda-show-new-time
Nick Dokos writes: > So you revert that commit and the infloop goes away? I looked at the > functions briefly and I don't understand how this commit could affect > what move-to-column does. Just to confirm: Did you try rescheduling with the agenda buffer filtered only to the home tag using the sample file I provided? To trigger the bug one must attempt to reschedule an item when 1) the agenda is filtered by tags and 2) the item being changed contains invisible lines immediately beneath it (b/c of the filtering). Both these conditions are necessary to cause the bug on my end. When the agenda buffer is filtered to a tag (i.e., when it contains invisible lines), move-to-column treats the entire invisible region as a single line. Moving to the end of the line moves the point to the end of the entire invisible region. An easy way to see this behavior in action is to do the following: Take the following sample file (test.org): --8<---cut here---start->8--- * TODO A :home: SCHEDULED: <2013-08-06 Tue> * TODO B :work: SCHEDULED: <2013-08-06 Tue> * TODO C :play: SCHEDULED: <2013-08-06 Tue> --8<---cut here---end--->8--- 1. Call the agenda restricted to that file. --8<---cut here---start->8--- Day-agenda (W32): Tuesday 6 August 2013 test: Scheduled: TODO A :home: test: Scheduled: TODO B :work: test: Scheduled: TODO C :play: --8<---cut here---end--->8--- 2. Narrow an agenda buffer to the tag home. --8<---cut here---start->8--- Day-agenda (W32): Tuesday 6 August 2013 test: Scheduled: TODO A :home: --8<---cut here---end--->8--- 3. Move to the beginning of the task "A" line. 4. Call either M-x move-end-of-line or M-x move-to-column 100 (or another large number). 5. M-: (point) On my machine the point is at 287, which is at the at the end of the task "C" line (i.e., the end of the entire invisible region). If I call move-end-of-line on the task "A" line when the buffer is not filtered, the point moves to 125 - i.e., the end of the task A line. In other words, within the agenda buffer, move-to-column and move-end-of-line will move to the point to the end of the entire invisible region. That is why removing the local binding of buffer-invisibility-spec to nil triggers this bug, because when that variable is nil, the function org-agenda-show-new-time temporarily treats the agenda buffer as if it were visible (i.e., it ignores the invisible overlay). I reproduced the problem with emacs -Q, so I can confirm that it is not a result of my configuration. > org-move-to-column is just a compatibility function that calls > move-to-column whose doc says: > > Move point to column COLUMN in the current line. > > So I'm not sure why it ends up on a different line. I think the following explains why this is happening: (info "(elisp) Invisible Text") , |Ordinarily, functions that operate on text or move point do not care | whether the text is invisible. The user-level line motion commands | ignore invisible newlines if `line-move-ignore-invisible' is non-`nil' | (the default), but only because they are explicitly programmed to do so. | |However, if a command ends with point inside or at the boundary of | invisible text, the main editing loop relocates point to one of the two | ends of the invisible text. Emacs chooses the direction of relocation | so that it is the same as the overall movement direction of the | command; if in doubt, it prefers a position where an inserted char | would not inherit the `invisible' property. Additionally, if the text | is not replaced by an ellipsis and the command only moved within the | invisible text, then point is moved one extra character so as to try | and reflect the command's movement by a visible movement of the cursor. | |Thus, if the command moved point back to an invisible range (with | the usual stickiness), Emacs moves point back to the beginning of that | range. If the command moved point forward into an invisible range, | Emacs moves point forward to the first visible character that follows | the invisible text and then forward one more character. ` > Can you provide emacs and org versions? Maybe there is something weird > in what you use. (org-version) "Org-mode version 8.0.7 (release_8.0.7-367-gd1d918 @ /home/matt/org-mode/lisp/)" (emacs-version) GNU Emacs 24.3.1 (x86_64-unknown-linux-gnu, GTK+ Version 3.8.2) of 2013-07-30 on -var-lib
Re: [O] [BUG] Infinite loop in org-agenda-show-new-time
Oh, oh: I just tried following the steps in your last email and I *did* get the infinite loop, with the same backtrace that you did. Then I tried it again just now and it did not happen again. Is this thing not 100% reproducible? In any case, I'm working through the rest of your email. I should have more later on tonight. -- Nick
Re: [O] [BUG] Infinite loop in org-agenda-show-new-time
Matt Lundin writes: > ... > In other words, within the agenda buffer, move-to-column and > move-end-of-line will move to the point to the end of the entire > invisible region. That is why removing the local binding of > buffer-invisibility-spec to nil triggers this bug, because when that > variable is nil, the function org-agenda-show-new-time temporarily > treats the agenda buffer as if it were visible (i.e., it ignores the > invisible overlay). > I haven't been able to work on the problem, but assuming that your diagnosis above is correct, perhaps the thing to do is to bind buffeer-invisibility-spec to nil inside org-move-to-column: --8<---cut here---start->8--- (defun org-move-to-column (column &optional force buffer) (let ((buffer-invisibility-spec nil)) (if (featurep 'xemacs) (org-xemacs-without-invisibility (move-to-column column force buffer)) (move-to-column column force --8<---cut here---end--->8--- What do you think? -- Nick
Re: [O] [BUG] Infinite loop in org-agenda-show-new-time
Nick Dokos writes: > Matt Lundin writes: > >> ... >> In other words, within the agenda buffer, move-to-column and >> move-end-of-line will move to the point to the end of the entire >> invisible region. That is why removing the local binding of >> buffer-invisibility-spec to nil triggers this bug, because when that >> variable is nil, the function org-agenda-show-new-time temporarily >> treats the agenda buffer as if it were visible (i.e., it ignores the >> invisible overlay). >> > > I haven't been able to work on the problem, but assuming that your > diagnosis above is correct, perhaps the thing to do is to bind > buffeer-invisibility-spec to nil inside org-move-to-column: > > (defun org-move-to-column (column &optional force buffer) > (let ((buffer-invisibility-spec nil)) > (if (featurep 'xemacs) > (org-xemacs-without-invisibility (move-to-column column force > buffer)) > (move-to-column column force > > What do you think? That solves the problem beautifully. Would it have any negative side-effects for other org functions, especially those that invoke org-move-to-column in normal org buffers? Here's a list of such invocations: --8<---cut here---start->8--- -*- mode: grep; default-directory: "~/org-mode/lisp/" -*- Grep started at Wed Aug 14 10:14:17 grep -nH -e org-move-to-column *.el org-agenda.el:8248:(org-move-to-column col)) org-agenda.el:8260: (org-move-to-column col))) org-agenda.el:8766: (org-move-to-column col org-agenda.el:9130: (org-move-to-column (- (window-width) (length stamp)) t) org-agenda.el:9232: (org-move-to-column col org-agenda.el:9252:(org-move-to-column col) org-clock.el:1851:(org-move-to-column c) org-colview.el:503: (org-move-to-column col) org-colview.el:632: (org-move-to-column col) org-compat.el:338:(defun org-move-to-column (column &optional force buffer) org.el:9440: (if (< (current-column) gc) (org-move-to-column gc t) (insert " ")) org.el:14377: (org-move-to-column (min ncol col) t)) org.el:14532: (org-move-to-column col) org.el:14616: (org-move-to-column (- (window-width) 19) t) org.el:22037: (org-move-to-column column org.el:22465: (org-move-to-column min-indent t)) org.el:22588: (org-move-to-column cc t) org.el:22593:((not off) (org-move-to-column cc t) (insert ": "))) org.el:23360: (org-move-to-column c org-list.el:2170:(org-move-to-column col))) org-list.el:2189:(org-move-to-column col))) org-macs.el:110: (org-move-to-column ,col) org-src.el:358: (org-move-to-column org-src.el:364: (org-move-to-column org-src.el:537: (org-move-to-column (max 0 (- col block-nindent 2))) org-src.el:771: (org-move-to-column (if preserve-indentation col (+ col total-nindent delta) org-table.el:1121:(org-move-to-column col) org-table.el:1126:(org-move-to-column col)) org-table.el:1353: (org-move-to-column col) org-table.el:1358: (org-move-to-column col)) org-table.el:1507:(org-move-to-column col) org-table.el:1559:(org-move-to-column col) org-table.el:1599:(org-move-to-column col) Grep finished (matches found) at Wed Aug 14 10:14:17 --8<---cut here---end--->8--- Matt
Re: [O] [BUG] Infinite loop in org-agenda-show-new-time
Nick Dokos writes: > Oh, oh: I just tried following the steps in your last email and I *did* > get the infinite loop, with the same backtrace that you did. Then I > tried it again just now and it did not happen again. Is this thing not > 100% reproducible? FWIW, it is 100% reproducible here when there are invisible lines immediate following the item I am trying to reschedule. Matt
Re: [O] [BUG] Infinite loop in org-agenda-show-new-time
Matt Lundin writes: > Nick Dokos writes: > >> Oh, oh: I just tried following the steps in your last email and I *did* >> get the infinite loop, with the same backtrace that you did. Then I >> tried it again just now and it did not happen again. Is this thing not >> 100% reproducible? > > FWIW, it is 100% reproducible here when there are invisible lines > immediate following the item I am trying to reschedule. > OK - I've tried a few time since then and I have not been able to hit it again. I'll keep trying. -- Nick
Re: [O] [BUG] Infinite loop in org-agenda-show-new-time
Matt Lundin writes: > Nick Dokos writes: > >> >> I haven't been able to work on the problem, but assuming that your >> diagnosis above is correct, perhaps the thing to do is to bind >> buffeer-invisibility-spec to nil inside org-move-to-column: >> >> (defun org-move-to-column (column &optional force buffer) >> (let ((buffer-invisibility-spec nil)) >> (if (featurep 'xemacs) >> (org-xemacs-without-invisibility (move-to-column column force >> buffer)) >>(move-to-column column force >> >> What do you think? > > That solves the problem beautifully. Would it have any negative > side-effects for other org functions, especially those that invoke > org-move-to-column in normal org buffers? > I hope not, but I don't know for sure. OTOH, we can try it and, if there are complaints, we can revert it and apply a more localized version of the same fix: just wrap the relevant *call* of org-move-to-column in a (let ((buffer-invisibility-spec nil)) (org-move-to-column )) But (without any solid evidence) it seems to me that having the behavior of move-to-column change with the buffer-invisibility-spec setting is a bug: how can anybody expect reproducible behavior from it if you don't know where point is going to end up? So I would be surprised if the fix does have negative side effects on anything: on the contrary, it might resolve some mysteries. OTOH, putting the let in the compat function would be a workaround for org, but the real fix should probably be in move-to-column itself. Perhaps a question to emacs devel is warranted. -- Nick
Re: [O] [BUG] Infinite loop in org-agenda-show-new-time
On 14.8.2013, at 17:45, Nick Dokos wrote: > Matt Lundin writes: > >> Nick Dokos writes: >> >>> >>> I haven't been able to work on the problem, but assuming that your >>> diagnosis above is correct, perhaps the thing to do is to bind >>> buffeer-invisibility-spec to nil inside org-move-to-column: >>> >>> (defun org-move-to-column (column &optional force buffer) >>> (let ((buffer-invisibility-spec nil)) >>> (if (featurep 'xemacs) >>> (org-xemacs-without-invisibility (move-to-column column force >>> buffer)) >>> (move-to-column column force >>> >>> What do you think? >> >> That solves the problem beautifully. Would it have any negative >> side-effects for other org functions, especially those that invoke >> org-move-to-column in normal org buffers? >> > > I hope not, but I don't know for sure. OTOH, we can try it and, if there > are complaints, we can revert it and apply a more localized version of > the same fix: just wrap the relevant *call* of org-move-to-column in a > > (let ((buffer-invisibility-spec nil)) >(org-move-to-column )) > > > But (without any solid evidence) it seems to me that having the behavior > of move-to-column change with the buffer-invisibility-spec setting is a > bug: how can anybody expect reproducible behavior from it if you don't > know where point is going to end up? So I would be surprised if the fix > does have negative side effects on anything: on the contrary, it might > resolve some mysteries. OTOH, putting the let in the compat function > would be a workaround for org, but the real fix should probably be in > move-to-column itself. Perhaps a question to emacs devel is warranted. Hi Nick, I also do not expect negative consequences. Please apply the patch, will you? Thanks! - Carsten > > -- > Nick > >
Re: [O] [BUG] Infinite loop in org-agenda-show-new-time
Carsten Dominik writes: > I also do not expect negative consequences. Please apply the patch, will you? > OK - patch attached. NB: there is at least one place where the "wrap the call to org-move-to-column" has been applied, in org.el:`org-comment-or-uncomment-region', presumably for exactly this reason. If this patch does not cause unexpected problem, then the wrapped setting above can be cleaned up. >From b0aebe182a72df8e7d5428a02e0eca7a6922fefe Mon Sep 17 00:00:00 2001 From: Nick Dokos Date: Fri, 23 Aug 2013 07:42:44 -0400 Subject: [PATCH] Fix infinite loop in org-agenda-show-new-time * lisp/org-agenda.el (org-move-to-column): Make sure that `buffer-invisibility-spec' is nil when calling move-to-column. This was caused by a previous commit (fafb5f34) which eliminated the setting of `buffer-invisibility-spec' in `org-agenda-show-new-time'. Matt Lundin ran into this and there is a discussion on the ML: http://thread.gmane.org/gmane.emacs.orgmode/75288 --- lisp/org-compat.el | 9 ++--- 1 file changed, 6 insertions(+), 3 deletions(-) diff --git a/lisp/org-compat.el b/lisp/org-compat.el index d10aeea..c4d15d8 100644 --- a/lisp/org-compat.el +++ b/lisp/org-compat.el @@ -336,9 +336,12 @@ Works on both Emacs and XEmacs." (indent-line-to column))) (defun org-move-to-column (column &optional force buffer) - (if (featurep 'xemacs) - (org-xemacs-without-invisibility (move-to-column column force buffer)) -(move-to-column column force))) + ;; set buffer-invisibility-spec to nil so that move-to-column + ;; does the right thing despite the presence of invisible text. + (let ((buffer-invisibility-spec nil)) +(if (featurep 'xemacs) + (org-xemacs-without-invisibility (move-to-column column force buffer)) + (move-to-column column force (defun org-get-x-clipboard-compat (value) "Get the clipboard value on XEmacs or Emacs 21." -- 1.8.3.101.g727a46b I hope it conforms to conventions but let me know of any problems. -- Nick
Re: [O] [BUG] Infinite loop in org-agenda-show-new-time
I've been having this same issue - the issue is quite reproducible for me, and it has been for years. I just finally grew tired of the issue and decided to investigate it, and yes, the issue is org-agenda-show-new-time. I also have invisible entries in the org buffer, and the call to org-move-to-column apparently will move several lines forward, which causes us to process the same lines over and over. Wrapping the call to org-move-to-column with a let setting the buffer-invisibility-spec to nil does fix the issue. On Fri, Aug 23, 2013 at 8:08 AM Nick Dokos wrote: > Carsten Dominik writes: > > > I also do not expect negative consequences. Please apply the patch, > will you? > > > > OK - patch attached. NB: there is at least one place where the "wrap the > call to org-move-to-column" has been applied, in > org.el:`org-comment-or-uncomment-region', presumably for exactly this > reason. If this patch does not cause unexpected problem, then the > wrapped setting above can be cleaned up. > > > I hope it conforms to conventions but let me know of any problems. > -- > Nick >