[O] bug in org-habits
org-habit.el is sensitive to the ordering PROPERTIES and SCHEDULED and expects them /in only/ the following order: * TODO habit name SCHEDULED: <2015-11-03 Tue 07:00 .+1d> :PROPERTIES: :LAST_REPEAT: [2015-11-02 Mon 07:54] :STYLE: habit :END: Any other ordering, or doubling of PROPERTIES (e.g. STYLE in one, LAST REPEAT in another) and the habit will show up as a simple scheduled item and not a habit in the agenda view. org-version: 8.3.2 emacs-version: 24.4.1 Let me know if I need to provide any other information. Thanks! Mark. -- http://hexmode.com/ Love every man in spite of his falling into sin. Never mind the sins, but remember that the foundation of the man is the same - the image of God. -- St. John of Kronstadt
Re: [O] bug in org-habits
"Mark A. Hershberger" writes: > org-habit.el is sensitive to the ordering PROPERTIES and SCHEDULED > and expects them /in only/ the following order: > > * TODO habit name > SCHEDULED: <2015-11-03 Tue 07:00 .+1d> > :PROPERTIES: > :LAST_REPEAT: [2015-11-02 Mon 07:54] > :STYLE: habit > :END: > > Any other ordering, or doubling of PROPERTIES (e.g. STYLE in one, LAST > REPEAT in another) and the habit will show up as a simple scheduled item > and not a habit in the agenda view. > > org-version: 8.3.2 Actually there has been introduced a constraint on the ordering planning lines and property drawers in 8.3. See http://orgmode.org/Changes.html. This at least invalidates to use PROPERTIES before SCHEDULED afaics. HTH -- Marco Wahl GPG: 0x49010A040A3AE6F2
Re: [O] bug in org-habits
On Tue, Nov 3, 2015 at 3:26 PM, Marco Wahl wrote: > > > Actually there has been introduced a constraint on the ordering planning > lines and property drawers in 8.3. See http://orgmode.org/Changes.html. > > This at least invalidates to use PROPERTIES before SCHEDULED afaics. Yes, that is correct and you can use the `org-repair-property-drawers` utility function provided to fix your org trees. -- Puneeth
Re: [O] bug in org-habits
> Puneeth Chaganti writes: >> Actually there has been introduced a constraint on the ordering planning >> lines and property drawers in 8.3. See http://orgmode.org/Changes.html.> >> This at least invalidates to use PROPERTIES before SCHEDULED afaics. > Yes, that is correct and you can use the `org-repair-property-drawers` > utility function provided to fix your org trees. I would very much like to see this constraint removed from 8.3. I have always preferred having SCHEDULED before PROPERTIES, as this is how all my Org files are arranged. In fact, this one aspect of 8.3 has kept me at 8.2. I do not understand the need for such an abitrary restriction. John
Re: [O] bug in org-habits
John Wiegley writes: >> Puneeth Chaganti writes: > >>> Actually there has been introduced a constraint on the ordering planning >>> lines and property drawers in 8.3. See http://orgmode.org/Changes.html.> >>> This at least invalidates to use PROPERTIES before SCHEDULED afaics. > >> Yes, that is correct and you can use the `org-repair-property-drawers` >> utility function provided to fix your org trees. > > I would very much like to see this constraint removed from 8.3. I have always > preferred having SCHEDULED before PROPERTIES, as this is how all my Org files > are arranged. Actually what you describe _is_ the expected order for the 8.3 files. > In fact, this one aspect of 8.3 has kept me at 8.2. I do not understand the > need for such an abitrary restriction. So this constraint on the order should not hold you back to move on to 8.3. I can report that in my personal org practice this constraint never got in my way. Further I can't imagine a use case when the user can benefit of a SCHEDULE line below a property drawer, but this might be a bit personal. I can say nothing about the technical view of having that constraint. Best regards, -- Marco Wahl GPG: 0x49010A040A3AE6F2
Re: [O] bug in org-habits
On 03/11/15 14:46, Marco Wahl wrote: John Wiegley writes: Puneeth Chaganti writes: Actually there has been introduced a constraint on the ordering planning lines and property drawers in 8.3. See http://orgmode.org/Changes.html.> This at least invalidates to use PROPERTIES before SCHEDULED afaics. Yes, that is correct and you can use the `org-repair-property-drawers` utility function provided to fix your org trees. I would very much like to see this constraint removed from 8.3. I have always preferred having SCHEDULED before PROPERTIES, as this is how all my Org files are arranged. Actually what you describe _is_ the expected order for the 8.3 files. My issue is a bit different. I have an org file with calendar appointments. I also have attachments to the appointments. The attachment appears in a PROPERTIES drawer. Now if I have the timestamp (plain one) before the drawer, I cannot open the attachment. Pressing C-c C-a o just inserts another PROPERTIES drawer (with another ID) above the timestamp. If, however, I move the timestamp after the PROPERTIES drawer, I can successfully open the attachment and the second drawer doesn't appear. Also, when the second drawer appears, a new empty folder is created in the data directory of my org directory. IMHO this is a bug, as at the moment I have to have the timestamp after the drawer. Org version is Org-mode version 8.3.2 (release_8.3.2-253-g9b5757 @ /Users/si/elisp/org-mode/lisp/)
Re: [O] bug in org-habits
Hello, Stelian Iancu writes: > I have an org file with calendar appointments. I also have attachments > to the appointments. The attachment appears in a PROPERTIES drawer. > > Now if I have the timestamp (plain one) before the drawer, I cannot > open the attachment. Pressing C-c C-a o just inserts another > PROPERTIES drawer (with another ID) above the timestamp. > > If, however, I move the timestamp after the PROPERTIES drawer, I can > successfully open the attachment and the second drawer doesn't appear. > > Also, when the second drawer appears, a new empty folder is created in > the data directory of my org directory. > > IMHO this is a bug, as at the moment I have to have the timestamp > after the drawer. This is not a bug. The order is HEADLINE (SCHEDULED PROPERTIES). If you insert a timestamp between SCHEDULED and PROPERTIES, the latter is no longer recognized as a PROPERTIES DRAWER (it becomes, in fact, a regular drawer which happens to be named "PROPERTIES") and you lose any information attached to it. You can get this information back by putting the PROPERTIES drawer (the special one) at its expected location. Regards, -- Nicolas Goaziou
Re: [O] bug in org-habits
> Nicolas Goaziou writes: > This is not a bug. The order is HEADLINE (SCHEDULED PROPERTIES). Ah, I had misspoken. It's not SCHEDULE-before-PROPERTIES that has broken for me, it is the inability to have PROPERTIES at the very end of the entry. Why must the properties drawer appear before the entry's body? John
Re: [O] bug in org-habits
Hello, John Wiegley writes: >> Nicolas Goaziou writes: > >> This is not a bug. The order is HEADLINE (SCHEDULED PROPERTIES). > > Ah, I had misspoken. It's not SCHEDULE-before-PROPERTIES that has broken for > me, it is the inability to have PROPERTIES at the very end of the entry. > > Why must the properties drawer appear before the entry's body? IIRC, you already asked the question. It is for efficiency reasons. Properties are an important feature in Org, letting them anywhere in a potentially long entry doesn't scale well. Regards, -- Nicolas Goaziou
Re: [O] bug in org-habits
> Nicolas Goaziou writes: > It is for efficiency reasons. Properties are an important feature in Org, > letting them anywhere in a potentially long entry doesn't scale well. Since it scales for my use case, can I please have a customization variable to relax this restriction? I prefer PROPERTIES drawers at the end, and would prefer not to trade something I want for something that doesn't affect me. Thanks, John
Re: [O] bug in org-habits
John Wiegley writes: > Since it scales for my use case, can I please have a customization variable to > relax this restriction? I'd rather not have syntax too much customizable, for portability, ease of maintenance, too. There are already too many mistakes in that area. Also, please note that previous behaviour didn't handle some pathological cases (e.g., property drawer in a comment block, or past an inlinetask). Fixing that properly, i.e. parsing the section all the way down to the drawer, would probably have had an impact in your use case. The current behaviour is technically much more reliable. > I prefer PROPERTIES drawers at the end, and would prefer not to trade > something I want for something that doesn't affect me. If you need to end your entry with a drawer, couldn't you put any of them there? You can even have one there named "PROPERTIES". Regards,
Re: [O] bug in org-habits
> Nicolas Goaziou writes: > I'd rather not have syntax too much customizable, for portability, ease of > maintenance, too. There are already too many mistakes in that area. Thanks for discussing this with me, Nicolas. I appreciate there may be technical complexities involved. Could we special-case allow PROPERTIES to be the *very last thing* in an entry? I don't need it to float anywhere else. I just like it to be at the end. > If you need to end your entry with a drawer, couldn't you put any of them > there? You can even have one there named "PROPERTIES". But it wouldn't be the true PROPERTIES, would it? Most of my entries look like this (and I have many thousands of them): ** TODO Update auto insurance cards :Home:ATTACH: SCHEDULED: <2016-03-11 Fri +6m> - State "DONE" from "TODO" [2015-09-11 Fri 11:28] - State "WAITING"from "TODO" [2015-07-22 Wed 19:48] - State "DONE" from "STARTED"[2014-12-13 Sat 15:56] - State "DONE" from "TODO" [2014-03-31 Mon 03:11] - State "CANCELED" from "TODO" [2013-12-11 Wed 19:23] - State "CANCELED" from "TODO" [2013-05-21 Tue 12:30] - State "DONE" from "TODO" [2012-05-17 Thu 22:24] - State "CANCELED" from "TODO" [2011-04-23 Sat 22:04] - State "TODO" from "DONE" [2010-10-23 Sat 21:54] - State "DONE" from "TODO" [2010-10-08 Fri 14:26] :PROPERTIES: :ID: B1F3D3F6-9F39-4899-9AF8-93E019E7C6BB :CREATED: [2010-05-08 Sat 17:34] :LAST_REPEAT: [2015-09-11 Fri 11:28] :Attachments: AutoInsuranceIdCards.pdf :END: I suppose I've just become familiar with seeing :PROPERTIES: as the "period" at the end of the entry, and I'll like to keep it there if we can devise a technical answer that does not make life more difficult for you. John
Re: [O] bug in org-habits
John Wiegley writes: > Thanks for discussing this with me, Nicolas. I appreciate there may be > technical complexities involved. Could we special-case allow PROPERTIES to be > the *very last thing* in an entry? I don't need it to float anywhere else. I > just like it to be at the end. Well, that's precisely the thing that doesn't scale and that Nicolas wanted to avoid. Putting the properties at the beginning of an entry makes the search pretty much constant time and if you find something else at the start of the entry then you know there aren't any and can go on (this is pretty important for making sure property inheritance works correctly, among other things). If you could put them _anywhere_ else, you'd have to keep searching until you either find them or you've exhausted the span of the entry. Regards, Achim. -- +<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+ SD adaptations for KORG EX-800 and Poly-800MkII V0.9: http://Synth.Stromeko.net/Downloads.html#KorgSDada
Re: [O] bug in org-habits
> Achim Gratz writes: > Well, that's precisely the thing that doesn't scale and that Nicolas wanted > to avoid. Putting the properties at the beginning of an entry makes the > search pretty much constant time and if you find something else at the start > of the entry then you know there aren't any and can go on (this is pretty > important for making sure property inheritance works correctly, among other > things). If you could put them _anywhere_ else, you'd have to keep searching > until you either find them or you've exhausted the span of the entry. As a user, I'm willing to pay that cost. Also, I never have other property drawers. If it were just doing (re-search-forward "^:PROPERTIES:$" (end-of-entry)), then it wouldn't matter where the properties drawer was, so long as it's understood that the search may be wrong if the user has such a string appearing elsewhere. John
Re: [O] bug in org-habits
On November 3, 2015 4:31:11 PM EST, Achim Gratz wrote: >John Wiegley writes: >> Thanks for discussing this with me, Nicolas. I appreciate there may >be >> technical complexities involved. Could we special-case allow >PROPERTIES to be >> the *very last thing* in an entry? I don't need it to float anywhere >else. I >> just like it to be at the end. > >Well, that's precisely the thing that doesn't scale and that Nicolas >wanted to avoid. Putting the properties at the beginning of an entry >makes the search pretty much constant time and if you find something >else at the start of the entry then you know there aren't any and can >go >on (this is pretty important for making sure property inheritance works >correctly, among other things). If you could put them _anywhere_ else, >you'd have to keep searching until you either find them or you've >exhausted the span of the entry. Wouldn't last item in entry scale without issues? Find end of headline (start of next or end of buffer) and search backwards. If first element from end is a property drawer you have it, otherwise you still know there is none. Jon > >Regards, >Achim. -- Sent from my Android device with K-9 Mail. Please excuse my brevity.
Re: [O] bug in org-habits
> Jonathan Leech-Pepin writes: > Wouldn't last item in entry scale without issues? Find end of headline > (start of next or end of buffer) and search backwards. If first element from > end is a property drawer you have it, otherwise you still know there is > none. That sounds even better than what I mentioned about re-search-forward, and O(1) time complexity as well. John
Re: [O] bug in org-habits
John Wiegley writes: >> Jonathan Leech-Pepin writes: >> Wouldn't last item in entry scale without issues? Find end of headline >> (start of next or end of buffer) and search backwards. If first element from >> end is a property drawer you have it, otherwise you still know there is >> none. > > That sounds even better than what I mentioned about re-search-forward, and > O(1) time complexity as well. I don't think so. Search for end of entry can be complex in itself and you would never know if the properties you find by looking back aren't belonging to an entry one level down unless you scanned the whole span again. Also, properties can be any size and you have to search for the beginning of the drawer. Regards, Achim. -- +<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+ Factory and User Sound Singles for Waldorf Q+, Q and microQ: http://Synth.Stromeko.net/Downloads.html#WaldorfSounds
Re: [O] bug in org-habits
> Achim Gratz writes: > I don't think so. Search for end of entry can be complex in itself and you > would never know if the properties you find by looking back aren't belonging > to an entry one level down unless you scanned the whole span again. Also, > properties can be any size and you have to search for the beginning of the > drawer. Good point. Still, the option I'd still choose would be: org-allow-floating-properties If set to `non-nil', the :PROPERTIES: drawer may begin anywhere within an entry. Note that this will slow down many operations due to the additional scan needed, and may lead to incorrect result if similar text is found elsewhere within an entry, which is the recommended and default value for this variable is nil. John
Re: [O] bug in org-habits
John Wiegley writes: > Thanks for discussing this with me, Nicolas. I appreciate there may be > technical complexities involved. Could we special-case allow PROPERTIES to be > the *very last thing* in an entry? I don't need it to float anywhere else. I > just like it to be at the end. I think it's a false good idea. As a matter of fact, going to the end of an entry is not negligible, because of inlinetasks. Also, it is not really O(1) since it depends on the size of the entry. To get an idea, on my computer, moving past a 500 lines entry takes around 0.001s. I can imagine slower computers, or larger entries, neither being unheard of, having a slightly noticeable delay. Also, supporting both locations means that users will pay the full price in entries without a property drawer, even if they chose the fast path, i.e., drawer at the beginning of the entry, in the first place. This kind of defeats some of the advantages of the current state. There is also a meaner problem. Unlike to beginning of entry, end of entry is a moving target. Org sometimes automatically adds stuff there (e.g., footnotes when `org-footnote-section' is nil). So do other parts of Emacs (file local variables come to mind). Preventing insertions in this area can be tedious, if possible at all. OTOH, the mechanism protecting the beginning of the entry, i.e, `org-end-of-meta-data', is already implemented since planning lines require it anyway. Again, speed _was_ an issue, as testified by, e.g., org-use-property-inheritance's docstring. Current solution is both robust and fast, and is well worth the high price of an incompatible change, considering its central place in Org. >> If you need to end your entry with a drawer, couldn't you put any of them >> there? You can even have one there named "PROPERTIES". > > But it wouldn't be the true PROPERTIES, would it? Most of my entries look like > this (and I have many thousands of them): It wouldn't, of course. But at least you would get your "period" at the end of the entry. It doesn't matter much if it is empty since its contents are hidden anyway. > ** TODO Update auto insurance cards :Home:ATTACH: > SCHEDULED: <2016-03-11 Fri +6m> > - State "DONE" from "TODO" [2015-09-11 Fri 11:28] > - State "WAITING"from "TODO" [2015-07-22 Wed 19:48] > - State "DONE" from "STARTED"[2014-12-13 Sat 15:56] > - State "DONE" from "TODO" [2014-03-31 Mon 03:11] > - State "CANCELED" from "TODO" [2013-12-11 Wed 19:23] > - State "CANCELED" from "TODO" [2013-05-21 Tue 12:30] > - State "DONE" from "TODO" [2012-05-17 Thu 22:24] > - State "CANCELED" from "TODO" [2011-04-23 Sat 22:04] > - State "TODO" from "DONE" [2010-10-23 Sat 21:54] > - State "DONE" from "TODO" [2010-10-08 Fri 14:26] > :PROPERTIES: > :ID: B1F3D3F6-9F39-4899-9AF8-93E019E7C6BB > :CREATED: [2010-05-08 Sat 17:34] > :LAST_REPEAT: [2015-09-11 Fri 11:28] > :Attachments: AutoInsuranceIdCards.pdf > :END: The following is not ugly either ** TODO Update auto insurance cards :Home:ATTACH: SCHEDULED: <2016-03-11 Fri +6m> :PROPERTIES: :ID: B1F3D3F6-9F39-4899-9AF8-93E019E7C6BB :CREATED: [2010-05-08 Sat 17:34] :LAST_REPEAT: [2015-09-11 Fri 11:28] :Attachments: AutoInsuranceIdCards.pdf :END: - State "DONE" from "TODO" [2015-09-11 Fri 11:28] - State "WAITING"from "TODO" [2015-07-22 Wed 19:48] - State "DONE" from "STARTED"[2014-12-13 Sat 15:56] - State "DONE" from "TODO" [2014-03-31 Mon 03:11] - State "CANCELED" from "TODO" [2013-12-11 Wed 19:23] - State "CANCELED" from "TODO" [2013-05-21 Tue 12:30] - State "DONE" from "TODO" [2012-05-17 Thu 22:24] - State "CANCELED" from "TODO" [2011-04-23 Sat 22:04] - State "TODO" from "DONE" [2010-10-23 Sat 21:54] - State "DONE" from "TODO" [2010-10-08 Fri 14:26] even more so considering it really appears as ** TODO Update auto insurance cards :Home:ATTACH: SCHEDULED: <2016-03-11 Fri +6m> :PROPERTIES: - State "DONE" from "TODO" [2015-09-11 Fri 11:28] - State "WAITING"from "TODO" [2015-07-22 Wed 19:48] - State "DONE" from "STARTED"[2014-12-13 Sat 15:56] - State "DONE" from "TODO" [2014-03-31 Mon 03:11] - State "CANCELED" from "TODO" [2013-12-11 Wed 19:23] - State "CANCELED" from "TODO" [2013-05-21 Tue 12:30] - State "DONE" from "TODO" [2012-05-17 Thu 22:24] - State "CANCELED" from "TODO" [2011-04-23 Sat 22:04] - State "TODO" from "DONE" [2010-10-23 Sat 21:54] - State "DONE" from "TODO" [2010-10-08 Fri 14:26] and with a fake "THE-END" drawer ** TODO Update auto insurance cards :Home:ATTACH: SCHEDULED: <2016-03-11 Fri +6m> :P
Re: [O] bug in org-habits
> Nicolas Goaziou writes: > As a matter of fact, going to the end of an entry is not negligible, because > of inlinetasks. Also, it is not really O(1) since it depends on the size of > the entry. To get an idea, on my computer, moving past a 500 lines entry > takes around 0.001s. I can imagine slower computers, or larger entries, > neither being unheard of, having a slightly noticeable delay. I'm having a hard time buying the performance argument, since I've been using Org-mode for many years, and never has this been a problem. You're making me pay a cost (enforced structure), for a value only one of us perceives. > Also, supporting both locations means that users will pay the full price in > entries without a property drawer, even if they chose the fast path, i.e., > drawer at the beginning of the entry, in the first place. This kind of > defeats some of the advantages of the current state. It wouldn't be "users": it would be people intentionally opting to allow floating properties. _I_ would be paying the price, and I will pay it happily to keep 8.2 behavior. John
Re: [O] bug in org-habits
On 04/11/15 02:01, John Wiegley wrote: Nicolas Goaziou writes: Also, supporting both locations means that users will pay the full price in entries without a property drawer, even if they chose the fast path, i.e., drawer at the beginning of the entry, in the first place. This kind of defeats some of the advantages of the current state. It wouldn't be "users": it would be people intentionally opting to allow floating properties. _I_ would be paying the price, and I will pay it happily to keep 8.2 behavior. John I would also like to have the 8.2 behavior. If it's easier, it would be fine to always have the drawer either in the beginning or in the end and nowhere else (and have an option to chose with the default in the beginning). S.
Re: [O] bug in org-habits
On Tuesday, 3 Nov 2015 at 20:01, John Wiegley wrote: [...] > I'm having a hard time buying the performance argument, since I've been using > Org-mode for many years, and never has this been a problem. You're making me > pay a cost (enforced structure), for a value only one of us perceives. John, others *have* found issues with performance. In 8.2, org struggled with many of my documents, some of which are very large. Most, if not all, of the performance problems I was having have disappeared since then (thank you Nicolas!). I cannot attribute the improvements directly to the move to a fixed structure for headline meta-data but I can easily believe that this move has contributed due to anecdotal experience with complex sections within my org files. This does not mean I am against your suggestion of a variable controlling this behaviour. Maybe implement it and post a patch for the community to try out? I am perfectly happy with the current format, however, so would be unlikely to try it. (sorry) -- : Eric S Fraga (0xFFFCF67D), Emacs 25.0.50.2, Org release_8.3.2-209-gba4d33
Re: [O] bug in org-habits
> Bastien Guerry writes: > I think Achim's points above are very valid, and the flexibility offered by > the option above should be very carefully examined. I spoke to Nicolas directly and he mentioned that a goal for syntax regularity is to make it possible to reliably read and manipulate Org files outside of Emacs. For this I *am* willing to give up order independence of PROPERTIES. Having a customization option would needlessly increases the number of possibilities external processors must consider, and so I retract my request. John
Re: [O] bug in org-habits
Hi John and Achim, John Wiegley writes: >> Achim Gratz writes: > >> I don't think so. Search for end of entry can be complex in itself and you >> would never know if the properties you find by looking back aren't belonging >> to an entry one level down unless you scanned the whole span again. Also, >> properties can be any size and you have to search for the beginning of the >> drawer. > > Good point. Still, the option I'd still choose would be: > > org-allow-floating-properties > > If set to `non-nil', the :PROPERTIES: drawer may begin anywhere within > an entry. Note that this will slow down many operations due to the > additional scan needed, and may lead to incorrect result if similar text > is found elsewhere within an entry, which is the recommended and default > value for this variable is nil. I think Achim's points above are very valid, and the flexibility offered by the option above should be very carefully examined. I'm trying hard to put aside one day or two for org-mode next week, I'll come back on this issue then. Thanks, -- Bastien