Re: [O] Enriched/Org is a colorful Org
I want to add to the thanks to everybody for making speed improvements. -- The Kafka Pandemic: http://thekafkapandemic.blogspot.com The disease DOES progress. MANY people have died from it. ANYBODY can get it. There is NO hope without action. This means YOU.
Re: [O] Enriched/Org is a colorful Org
Hi Eli, On Fri, Apr 12, 2013 at 09:38:47PM +0300, Eli Zaretskii wrote: Date: Fri, 12 Apr 2013 20:00:56 +0200 From: Suvayu Ali fatkasuvayu+li...@gmail.com Cc: Torsten Wagner torsten.wag...@gmail.com, emacs-orgmode@gnu.org, carsten.domi...@gmail.com If some package wants to keep something updated (line number displays in this case), is using the post-command-hook the only option? No. The other one is piggy-back jit-lock. See nlinum in ELPA for an example, which actually is a drop-in replacement for linum, but without many of its problems. Thanks a lot :) -- Suvayu Open source is the future. It sets us free.
Re: [O] Enriched/Org is a colorful Org
In your opinion, would it be possible to reproduce the functionality of outline-mode using text properties rather than overlays? And in the case of org-mode, would this really make that much of a difference in terms of performance? Regards, Sean On Fri, Apr 12, 2013 at 1:36 PM, Eli Zaretskii e...@gnu.org wrote: From: Carsten Dominik carsten.domi...@gmail.com Date: Fri, 12 Apr 2013 12:56:11 +0200 Cc: emacs-orgmode@gnu.org I guess outline mode does have the exact same problem in this case, in fact any mode with large amount of hidden text. Of course. The only difference is that outline is not as popular as Org, and usually is used with relatively short chunks of text. But the problems are exactly the same.
Re: [O] Enriched/Org is a colorful Org
Sean O'Halpin sean.ohal...@gmail.com writes: In your opinion, would it be possible to reproduce the functionality of outline-mode using text properties rather than overlays? And in the case of org-mode, would this really make that much of a difference in terms of performance? Broadly speaking, answer seems to be a Yes. It is possible that some details have to be hammered out wrt indirect buffers. See the last item in this post http://lists.gnu.org/archive/html/emacs-devel/2010-10/msg00177.html Regards, Sean On Fri, Apr 12, 2013 at 1:36 PM, Eli Zaretskii e...@gnu.org wrote: From: Carsten Dominik carsten.domi...@gmail.com Date: Fri, 12 Apr 2013 12:56:11 +0200 Cc: emacs-orgmode@gnu.org I guess outline mode does have the exact same problem in this case, in fact any mode with large amount of hidden text. Of course. The only difference is that outline is not as popular as Org, and usually is used with relatively short chunks of text. But the problems are exactly the same.
Re: [O] Enriched/Org is a colorful Org
Date: Sat, 13 Apr 2013 13:24:13 +0100 From: Sean O'Halpin sean.ohal...@gmail.com Cc: Carsten Dominik carsten.domi...@gmail.com, Org Mode emacs-orgmode@gnu.org In your opinion, would it be possible to reproduce the functionality of outline-mode using text properties rather than overlays? This needs to be analyzed. Outline mode uses several features that are specific to overlays, but not to text properties: isearch-open-invisible and front-advance. (It also uses 'evaporate', but that happens automatically with text properties.) I think front-advance will happen automatically, but searching inside invisible text is currently supported only for overlays (for reasons that I consider irrelevant, but that's me). And in the case of org-mode, would this really make that much of a difference in terms of performance? I don't know, because I didn't measure that, and neither did I see any measurements published. If there are many overlays in a buffer, redisplay performance will certainly be better with text properties, but I don't know whether this improvement will be significant: there could be other factors at work that affect performance to a much greater effect. If we want to know for sure, there's no way around profiling this. Luckily, we now have profiler.el to help us.
Re: [O] Enriched/Org is a colorful Org
From: Carsten Dominik carsten.domi...@gmail.com Date: Fri, 12 Apr 2013 00:49:32 +0200 Cc: emacs-orgmode@gnu.org List emacs-orgmode@gnu.org Overlays should be OK as long as they aren't too many, and as long as you don't move them around too much, particularly in post-command-hook or some such. This explanation sounds to me as if the display engine is building some kind of tree of overlays to find properties changes quickly. Too bad I don't have time to understand this stuff in more detail, it sounds interesting. Just search xdisp.c for overlay, you will see the story quite clearly, I think. There are actually 2 aspects that make display engine's job harder with overlays. One is indeed that finding overlays that cover a given buffer position is not a very efficient operation. The display engine tries to overcome this to some extent by centering the overlay data base around the place in the buffer it is rendering. This is done for every screen line that is being examined by the display code. The other problem is that there's no easy way of finding out which parts of the buffer are being affected by changes in overlays. The consequence of this is that redisplay cannot easily determine whether a given portion of what's on the glass needs to be redrawn when overlays change. The analysis of which part(s) of a window need to be redrawn is at the heart of redisplay optimizations, whereby Emacs tries very hard to reuse the portions of display that are already up to date. Changes in overlays defeat that. Therefore, changing _any_ overlays in a buffer disables most, if not all, redisplay optimizations, meaning that the entire portion of the buffer that is displayed will be completely redrawn each time overlays _anywhere_ in the buffer are changed.
Re: [O] Enriched/Org is a colorful Org
On 12 apr. 2013, at 08:41, Eli Zaretskii e...@gnu.org wrote: From: Carsten Dominik carsten.domi...@gmail.com Date: Fri, 12 Apr 2013 00:49:32 +0200 Cc: emacs-orgmode@gnu.org List emacs-orgmode@gnu.org Overlays should be OK as long as they aren't too many, and as long as you don't move them around too much, particularly in post-command-hook or some such. This explanation sounds to me as if the display engine is building some kind of tree of overlays to find properties changes quickly. Too bad I don't have time to understand this stuff in more detail, it sounds interesting. Just search xdisp.c for overlay, you will see the story quite clearly, I think. My Sunday pleasure reading project. There are actually 2 aspects that make display engine's job harder with overlays. One is indeed that finding overlays that cover a given buffer position is not a very efficient operation. The display engine tries to overcome this to some extent by centering the overlay data base around the place in the buffer it is rendering. This is done for every screen line that is being examined by the display code. The other problem is that there's no easy way of finding out which parts of the buffer are being affected by changes in overlays. The consequence of this is that redisplay cannot easily determine whether a given portion of what's on the glass needs to be redrawn when overlays change. The analysis of which part(s) of a window need to be redrawn is at the heart of redisplay optimizations, whereby Emacs tries very hard to reuse the portions of display that are already up to date. Changes in overlays defeat that. Therefore, changing _any_ overlays in a buffer disables most, if not all, redisplay optimizations, meaning that the entire portion of the buffer that is displayed will be completely redrawn each time overlays _anywhere_ in the buffer are changed. Wow. OK. I need to think to that, and I will try to take a fresh look at overlay use in Org. So the reason that the combination with hi-line is slow is because hl-line is using post-command-hook to move its overlay, and redisplay of a full window with org-mode is slow because so much stuff is hidden and Emacs makes a full re-evaluation of what needs to be displayed? This makes sense. Thanks for taking the time to get me this far. - Carsten
Re: [O] Enriched/Org is a colorful Org
From: Carsten Dominik carsten.domi...@gmail.com Date: Fri, 12 Apr 2013 09:13:47 +0200 Cc: emacs-orgmode@gnu.org Just search xdisp.c for overlay, you will see the story quite clearly, I think. My Sunday pleasure reading project. Good luck, and let me know if you need something explained. The commentary at the beginning of the file might serve as an introduction, although it doesn't really touch the issue at hand. So the reason that the combination with hi-line is slow is because hl-line is using post-command-hook to move its overlay, and redisplay of a full window with org-mode is slow because so much stuff is hidden and Emacs makes a full re-evaluation of what needs to be displayed? Right. If hi-line (or any similar mode) is off, then at least horizontal cursor motion should be fast, because then Emacs knows that nothing changed, and finding the place where to put the cursor on the same line it was before is relatively easy. But even C-n and C-p is quite another story in an Org buffer: Emacs needs to determine where that puts point, and doing so generally means traversing all of the hidden parts of the buffer between the line which was current and the new current line. In a complex Org buffer, that could easily be many thousands of buffer positions. Also, recall that, under line-move-visual, which is nowadays on by default, Emacs moves by _screen_ lines, not by physical lines. So a simple C-n must internally emulate display to find the next line visible on the screen by traversing the buffer one character at a time and taking note of each and every text property and overlay in between, until it finds the buffer position whose screen coordinates are [X,Y+N], where [X,Y] are the coordinates of the previous cursor position and N is the line height in pixels. And this is just to find where point will be; then the screen must actually be redisplayed, which might mean more work, if the new position of point requires scrolling, e.g. if cursor went off the scroll margins or whatever. We only get reasonably fast performance with all this complexity because our machines are incredibly fast. But we are many times on the edge, as the bug I cited and similar ones show.
Re: [O] Enriched/Org is a colorful Org
Carsten Dominik carsten.domi...@gmail.com writes: Thanks for taking the time to get me this far. +1! Thanks Eli, great to learn about the internals of Emacs display engine. The Emacs Lisp manual already contains some directions and warnings, but not so detailed. -- Bastien
Re: [O] Enriched/Org is a colorful Org
On 12 apr. 2013, at 10:31, Eli Zaretskii e...@gnu.org wrote: From: Carsten Dominik carsten.domi...@gmail.com Date: Fri, 12 Apr 2013 09:13:47 +0200 Cc: emacs-orgmode@gnu.org Just search xdisp.c for overlay, you will see the story quite clearly, I think. My Sunday pleasure reading project. Good luck, and let me know if you need something explained. The commentary at the beginning of the file might serve as an introduction, although it doesn't really touch the issue at hand. So the reason that the combination with hi-line is slow is because hl-line is using post-command-hook to move its overlay, and redisplay of a full window with org-mode is slow because so much stuff is hidden and Emacs makes a full re-evaluation of what needs to be displayed? Right. If hi-line (or any similar mode) is off, then at least horizontal cursor motion should be fast, because then Emacs knows that nothing changed, and finding the place where to put the cursor on the same line it was before is relatively easy. But even C-n and C-p is quite another story in an Org buffer: Emacs needs to determine where that puts point, and doing so generally means traversing all of the hidden parts of the buffer between the line which was current and the new current line. In a complex Org buffer, that could easily be many thousands of buffer positions. I guess outline mode does have the exact same problem in this case, in fact any mode with large amount of hidden text. Also, recall that, under line-move-visual, which is nowadays on by default, Not in my setup, but since it the default, yes, this causes more issues. Another important point to be aware of. Emacs moves by _screen_ lines, not by physical lines. So a simple C-n must internally emulate display to find the next line visible on the screen by traversing the buffer one character at a time and taking note of each and every text property and overlay in between, until it finds the buffer position whose screen coordinates are [X,Y+N], where [X,Y] are the coordinates of the previous cursor position and N is the line height in pixels. And this is just to find where point will be; then the screen must actually be redisplayed, which might mean more work, if the new position of point requires scrolling, e.g. if cursor went off the scroll margins or whatever. We only get reasonably fast performance with all this complexity because our machines are incredibly fast. But we are many times on the edge, as the bug I cited and similar ones show. Thanks again. - Carsten
Re: [O] Enriched/Org is a colorful Org
Hi, just want to add some observation. I guess it has nothing to do with the display engine but it might be somehow related. I used to use line-mode to display line-numbers as a left column on all my buffers. I noticed a very painful slowdown up to a totally unusable state during working on very large org-files. It consisted of coursework for a programming class and contained single headers with the student-id numbers and a babel-code block in the headers body (hence, easily goes into 1000th of lines). I was happy with it since I could execute and proof each submitted coursework within a single org-file and folding helped me to move quickly from one to the other coursework. However, as longer as the list get as more it slowed down. After some fiddling and searching, I noticed that the line-mode was kind of struggling with the org-mode text-collapse feature. Whenever, I closed a header, it took large amount of times to recalculate the line-numbers. Not sure where exactly line-mode did consume the time. But it might as well be related to the redisplaying of the numbers. Switching off the line-mode made the time delay disappear completely. Just an observation which might or might not related to the later discussion. Torsten On 12 April 2013 12:56, Carsten Dominik carsten.domi...@gmail.com wrote: On 12 apr. 2013, at 10:31, Eli Zaretskii e...@gnu.org wrote: From: Carsten Dominik carsten.domi...@gmail.com Date: Fri, 12 Apr 2013 09:13:47 +0200 Cc: emacs-orgmode@gnu.org Just search xdisp.c for overlay, you will see the story quite clearly, I think. My Sunday pleasure reading project. Good luck, and let me know if you need something explained. The commentary at the beginning of the file might serve as an introduction, although it doesn't really touch the issue at hand. So the reason that the combination with hi-line is slow is because hl-line is using post-command-hook to move its overlay, and redisplay of a full window with org-mode is slow because so much stuff is hidden and Emacs makes a full re-evaluation of what needs to be displayed? Right. If hi-line (or any similar mode) is off, then at least horizontal cursor motion should be fast, because then Emacs knows that nothing changed, and finding the place where to put the cursor on the same line it was before is relatively easy. But even C-n and C-p is quite another story in an Org buffer: Emacs needs to determine where that puts point, and doing so generally means traversing all of the hidden parts of the buffer between the line which was current and the new current line. In a complex Org buffer, that could easily be many thousands of buffer positions. I guess outline mode does have the exact same problem in this case, in fact any mode with large amount of hidden text. Also, recall that, under line-move-visual, which is nowadays on by default, Not in my setup, but since it the default, yes, this causes more issues. Another important point to be aware of. Emacs moves by _screen_ lines, not by physical lines. So a simple C-n must internally emulate display to find the next line visible on the screen by traversing the buffer one character at a time and taking note of each and every text property and overlay in between, until it finds the buffer position whose screen coordinates are [X,Y+N], where [X,Y] are the coordinates of the previous cursor position and N is the line height in pixels. And this is just to find where point will be; then the screen must actually be redisplayed, which might mean more work, if the new position of point requires scrolling, e.g. if cursor went off the scroll margins or whatever. We only get reasonably fast performance with all this complexity because our machines are incredibly fast. But we are many times on the edge, as the bug I cited and similar ones show. Thanks again. - Carsten
Re: [O] Enriched/Org is a colorful Org
From: Carsten Dominik carsten.domi...@gmail.com Date: Fri, 12 Apr 2013 12:56:11 +0200 Cc: emacs-orgmode@gnu.org I guess outline mode does have the exact same problem in this case, in fact any mode with large amount of hidden text. Of course. The only difference is that outline is not as popular as Org, and usually is used with relatively short chunks of text. But the problems are exactly the same.
Re: [O] Enriched/Org is a colorful Org
Date: Fri, 12 Apr 2013 13:49:47 +0200 From: Torsten Wagner torsten.wag...@gmail.com Cc: Eli Zaretskii e...@gnu.org, Org Mode Mailing List emacs-orgmode@gnu.org just want to add some observation. I guess it has nothing to do with the display engine but it might be somehow related. I used to use line-mode to display line-numbers as a left column on all my buffers. I noticed a very painful slowdown up to a totally unusable state during working on very large org-files. It consisted of coursework for a programming class and contained single headers with the student-id numbers and a babel-code block in the headers body (hence, easily goes into 1000th of lines). I was happy with it since I could execute and proof each submitted coursework within a single org-file and folding helped me to move quickly from one to the other coursework. However, as longer as the list get as more it slowed down. After some fiddling and searching, I noticed that the line-mode was kind of struggling with the org-mode text-collapse feature. Whenever, I closed a header, it took large amount of times to recalculate the line-numbers. Not sure where exactly line-mode did consume the time. But it might as well be related to the redisplaying of the numbers. Switching off the line-mode made the time delay disappear completely. So this was an Org file with only 1 level of headers, but with large blocks of text between the headers, is that right? Can you show an example of such babel-code block? I'd like to look into the slowdown and find its reasons. In general, linum does exactly what defeats redisplay optimizations: it modifies overlays in a post-command-hook. But that doesn't mean this was the reason for the slow operation you saw, it could be something else. If you can easily produce a recipe for recreating the problem, it might be worthwhile to file an Emacs bug report. Thanks.
Re: [O] Enriched/Org is a colorful Org
Bastien b...@gnu.org writes: Thanks Eli, great to learn about the internals of Emacs display engine. Eli is, and always has been, quite a resourceful man. And along the years, I got the pleasure of discovering him as a good friend too! :-) François
Re: [O] Enriched/Org is a colorful Org
Hi Eli, I hope you don't mind me taking this opportunity to ask a tangential question. On Fri, Apr 12, 2013 at 04:03:10PM +0300, Eli Zaretskii wrote: In general, linum does exactly what defeats redisplay optimizations: it modifies overlays in a post-command-hook. But that doesn't mean If some package wants to keep something updated (line number displays in this case), is using the post-command-hook the only option? -- Suvayu Open source is the future. It sets us free.
Re: [O] Enriched/Org is a colorful Org
Date: Fri, 12 Apr 2013 20:00:56 +0200 From: Suvayu Ali fatkasuvayu+li...@gmail.com Cc: Torsten Wagner torsten.wag...@gmail.com, emacs-orgmode@gnu.org, carsten.domi...@gmail.com If some package wants to keep something updated (line number displays in this case), is using the post-command-hook the only option? No. The other one is piggy-back jit-lock. See nlinum in ELPA for an example, which actually is a drop-in replacement for linum, but without many of its problems.
Re: [O] Enriched/Org is a colorful Org
[Please CC me on responses, as I'm not subscribed to this list.] From: Carsten Dominik carsten.domi...@gmail.com Date: Wed, 10 Apr 2013 21:58:06 +0200 Cc: emacs-orgmode@gnu.org I beg the Org developers to please be very careful when introducing expensive display features such as overlays into Org. Org already puts the Emacs display engine to its limits in many of its popular features; this is interesting input, I was not aware of this. Has this been discussed before, can you point me to relevant threads, and what are the symptoms of the display engine being at its limits? You won't find explicit discussions of this, except maybe a random comment from me here and there. There aren't too many discussions about the display engine in general; maybe it's my fault. But you can find indirect evidence to what I say in quite a few reports about slow redisplay. Here's one example (it's just the first one that popped up on Google): http://lists.gnu.org/archive/html/emacs-devel/2011-09/msg00276.html Note how two display features: bidi and hl-line -- each one of them cause significant slow-down in Org buffers, and almost nowhere else. This is just an example. I keep bumping into similar issues frequently enough to lead me to the conclusion you see above. In general, hiding from display large parts of a buffer, and using a lot of display strings and overlays that add to buffer text or replace buffer text with something else -- these all make redisplay much more expensive. In particular, moving overlays disables many redisplay optimizations, so e.g. any mode that moves overlays as result of post-command-hook will considerably slow down display and degrade user experience. After hacking the display code for a few years, it is painfully clear to me that its basic design assumed that such use cases are rare. Org mode makes these assumptions more and more false, and it does that faster than the CPU speed improves ;-) For these reasons, and as long as we don't have any development going on that aims at a complete redesign of the display engine, I think every feature, especially one expected to be popular, that adversely impacts redisplay efficiency, should be considered very carefully, and the various alternatives for its implementation assessed also from this aspect. HTH
Re: [O] Enriched/Org is a colorful Org
From: Carsten Dominik carsten.domi...@gmail.com Date: Thu, 11 Apr 2013 04:58:15 +0200 Cc: emacs-orgmode@gnu.org List emacs-orgmode@gnu.org, Eli Zaretskii e...@gnu.org I guess Eli simply means, in a general way, that overlays do negatively impact display performance, as you said as well a couple of times: Yes, but Eli says that Org already severely tests the display engine, and he uses the word mess, even though we mostly use text properties for faces and other display-related things. Well, don't interpret mess too literally ;-) Of course, Org already uses overlays, for example for folding (as does outline.el), and for temporary marking of text like during src block editing. But as your digging shows, I ave avoided them in the past, and we are also not using them for org-indent.el, for example. The reason why I said overlays would be better is simply that they would allow to add display properties in a persistent way that would not interfere that our font-lock-unfontify-region function removes face and invisibility text properties. So they are better for implementing hand-made faces selection that should overrule font-lock. Overlays should be OK as long as they aren't too many, and as long as you don't move them around too much, particularly in post-command-hook or some such.
Re: [O] Enriched/Org is a colorful Org
On 11.4.2013, at 19:27, Eli Zaretskii e...@gnu.org wrote: [Please CC me on responses, as I'm not subscribed to this list.] From: Carsten Dominik carsten.domi...@gmail.com Date: Wed, 10 Apr 2013 21:58:06 +0200 Cc: emacs-orgmode@gnu.org I beg the Org developers to please be very careful when introducing expensive display features such as overlays into Org. Org already puts the Emacs display engine to its limits in many of its popular features; this is interesting input, I was not aware of this. Has this been discussed before, can you point me to relevant threads, and what are the symptoms of the display engine being at its limits? You won't find explicit discussions of this, except maybe a random comment from me here and there. There aren't too many discussions about the display engine in general; maybe it's my fault. But you can find indirect evidence to what I say in quite a few reports about slow redisplay. Here's one example (it's just the first one that popped up on Google): http://lists.gnu.org/archive/html/emacs-devel/2011-09/msg00276.html Note how two display features: bidi and hl-line -- each one of them cause significant slow-down in Org buffers, and almost nowhere else. This is just an example. I keep bumping into similar issues frequently enough to lead me to the conclusion you see above. Yes, OK, I also remember reports like this. Funny, often it is not Org by itself, but in combination with something else that affects the display engine. In general, hiding from display large parts of a buffer, and using a lot of display strings and overlays that add to buffer text or replace buffer text with something else -- these all make redisplay much more expensive. In particular, moving overlays disables many redisplay optimizations, so e.g. any mode that moves overlays as result of post-command-hook will considerably slow down display and degrade user experience. OK, this is a concrete thing we can be on the lookout for. I don't think we do that, but I will take a look. After hacking the display code for a few years, it is painfully clear to me that its basic design assumed that such use cases are rare. Org mode makes these assumptions more and more false, and it does that faster than the CPU speed improves ;-) For these reasons, and as long as we don't have any development going on that aims at a complete redesign of the display engine, I think every feature, especially one expected to be popular, that adversely impacts redisplay efficiency, should be considered very carefully, and the various alternatives for its implementation assessed also from this aspect. This is clear enough. I will try to keep this in mind and evaluate changes in Org in this way. If you have other concrete things where you think Org could be improved in this direction, let us know. HTH Certainly, thank you. - Carsten
Re: [O] Enriched/Org is a colorful Org
On 11.4.2013, at 19:30, Eli Zaretskii e...@gnu.org wrote: From: Carsten Dominik carsten.domi...@gmail.com Date: Thu, 11 Apr 2013 04:58:15 +0200 Cc: emacs-orgmode@gnu.org List emacs-orgmode@gnu.org, Eli Zaretskii e...@gnu.org I guess Eli simply means, in a general way, that overlays do negatively impact display performance, as you said as well a couple of times: Yes, but Eli says that Org already severely tests the display engine, and he uses the word mess, even though we mostly use text properties for faces and other display-related things. Well, don't interpret mess too literally ;-) I will add an overlay that displays mess as mix :D Of course, Org already uses overlays, for example for folding (as does outline.el), and for temporary marking of text like during src block editing. But as your digging shows, I ave avoided them in the past, and we are also not using them for org-indent.el, for example. The reason why I said overlays would be better is simply that they would allow to add display properties in a persistent way that would not interfere that our font-lock-unfontify-region function removes face and invisibility text properties. So they are better for implementing hand-made faces selection that should overrule font-lock. Overlays should be OK as long as they aren't too many, and as long as you don't move them around too much, particularly in post-command-hook or some such. This explanation sounds to me as if the display engine is building some kind of tree of overlays to find properties changes quickly. Too bad I don't have time to understand this stuff in more detail, it sounds interesting. - Carsten
Re: [O] Enriched/Org is a colorful Org
On Wed, Apr 10, 2013 at 09:32:44AM +0530, Jambunathan K wrote: See Side note towards the end of this message http://debbugs.gnu.org/cgi/bugreport.cgi?bug=14157#8 This request is common enough; every time it comes up overlays are proposed as a solution. It would be good if this is available even as a library outside of Org. -- Suvayu Open source is the future. It sets us free.
Re: [O] Enriched/Org is a colorful Org
On 10 apr. 2013, at 11:54, Suvayu Ali fatkasuvayu+li...@gmail.com wrote: On Wed, Apr 10, 2013 at 09:32:44AM +0530, Jambunathan K wrote: See Side note towards the end of this message http://debbugs.gnu.org/cgi/bugreport.cgi?bug=14157#8 This request is common enough; every time it comes up overlays are proposed as a solution. It would be good if this is available even as a library outside of Org. Yes, overlays are better. However, maybe I am just no getting it, but what is even the purpose of facemenu? AFAICS, the faces are non-permanent, so when I save the file and reopen it, all the faces are gone. I really cannot see a useful application for this. - Carsten
Re: [O] Enriched/Org is a colorful Org
Hmmm as application an ambient-light org-mode coloring just joking Torsten On 10 April 2013 12:16, Carsten Dominik carsten.domi...@gmail.com wrote: On 10 apr. 2013, at 11:54, Suvayu Ali fatkasuvayu+li...@gmail.com wrote: On Wed, Apr 10, 2013 at 09:32:44AM +0530, Jambunathan K wrote: See Side note towards the end of this message http://debbugs.gnu.org/cgi/bugreport.cgi?bug=14157#8 This request is common enough; every time it comes up overlays are proposed as a solution. It would be good if this is available even as a library outside of Org. Yes, overlays are better. However, maybe I am just no getting it, but what is even the purpose of facemenu? AFAICS, the faces are non-permanent, so when I save the file and reopen it, all the faces are gone. I really cannot see a useful application for this. - Carsten
Re: [O] Enriched/Org is a colorful Org
Hi Carsten, On Wed, Apr 10, 2013 at 12:16:28PM +0200, Carsten Dominik wrote: On 10 apr. 2013, at 11:54, Suvayu Ali fatkasuvayu+li...@gmail.com wrote: On Wed, Apr 10, 2013 at 09:32:44AM +0530, Jambunathan K wrote: See Side note towards the end of this message http://debbugs.gnu.org/cgi/bugreport.cgi?bug=14157#8 This request is common enough; every time it comes up overlays are proposed as a solution. It would be good if this is available even as a library outside of Org. Yes, overlays are better. However, maybe I am just no getting it, but what is even the purpose of facemenu? AFAICS, the faces are non-permanent, so when I save the file and reopen it, all the faces are gone. I really cannot see a useful application for this. AFAIR, the use cases presented so far involved adding highlighting-like information in Org files. If the overlays are generated consistently based on the user's setup, it doesn't matter if they are non-permanent (as in not part of the Org file, but dependent on the Emacs setup instead). Of course this means the highlighting information will not be portable; but I don't think people mind that. I personally do not find any use for the feature as such; although it might be interesting to be able to insert plain text cookies in Org files and have them highlighted in some fashion. I could then use a list of ideas like this: Some topic ... 1. Idea 1 2. Idea 2 (?) where I'm doubtful about idea (2); having (?) highlighted would remind me of that. Just an idea. :) -- Suvayu Open source is the future. It sets us free.
Re: [O] Enriched/Org is a colorful Org
On 10 apr. 2013, at 12:48, Suvayu Ali fatkasuvayu+li...@gmail.com wrote: Hi Carsten, On Wed, Apr 10, 2013 at 12:16:28PM +0200, Carsten Dominik wrote: On 10 apr. 2013, at 11:54, Suvayu Ali fatkasuvayu+li...@gmail.com wrote: On Wed, Apr 10, 2013 at 09:32:44AM +0530, Jambunathan K wrote: See Side note towards the end of this message http://debbugs.gnu.org/cgi/bugreport.cgi?bug=14157#8 This request is common enough; every time it comes up overlays are proposed as a solution. It would be good if this is available even as a library outside of Org. Yes, overlays are better. However, maybe I am just no getting it, but what is even the purpose of facemenu? AFAICS, the faces are non-permanent, so when I save the file and reopen it, all the faces are gone. I really cannot see a useful application for this. AFAIR, the use cases presented so far involved adding highlighting-like information in Org files. If the overlays are generated consistently based on the user's setup, it doesn't matter if they are non-permanent (as in not part of the Org file, but dependent on the Emacs setup instead). Of course this means the highlighting information will not be portable; but I don't think people mind that. I personally do not find any use for the feature as such; although it might be interesting to be able to insert plain text cookies in Org files and have them highlighted in some fashion. I could then use a list of ideas like this: Some topic ... 1. Idea 1 2. Idea 2 (?) where I'm doubtful about idea (2); having (?) highlighted would remind me of that. Just an idea. Yes, this would make sense if the highlighting would be reestablished automatically next time you visit the file. If not, it would be pretty useless in my eyes. - Carsten
Re: [O] Enriched/Org is a colorful Org
Carsten Dominik wrote: On 10 apr. 2013, at 12:48, Suvayu Ali fatkasuvayu+li...@gmail.com wrote: On Wed, Apr 10, 2013 at 12:16:28PM +0200, Carsten Dominik wrote: On 10 apr. 2013, at 11:54, Suvayu Ali fatkasuvayu+li...@gmail.com wrote: On Wed, Apr 10, 2013 at 09:32:44AM +0530, Jambunathan K wrote: See Side note towards the end of this message http://debbugs.gnu.org/cgi/bugreport.cgi?bug=14157#8 This request is common enough; every time it comes up overlays are proposed as a solution. It would be good if this is available even as a library outside of Org. Yes, overlays are better. However, maybe I am just no getting it, but what is even the purpose of facemenu? AFAICS, the faces are non-permanent, so when I save the file and reopen it, all the faces are gone. I really cannot see a useful application for this. AFAIR, the use cases presented so far involved adding highlighting-like information in Org files. If the overlays are generated consistently based on the user's setup, it doesn't matter if they are non-permanent (as in not part of the Org file, but dependent on the Emacs setup instead). Of course this means the highlighting information will not be portable; but I don't think people mind that. I personally do not find any use for the feature as such; although it might be interesting to be able to insert plain text cookies in Org files and have them highlighted in some fashion. I could then use a list of ideas like this: Some topic ... 1. Idea 1 2. Idea 2 (?) where I'm doubtful about idea (2); having (?) highlighted would remind me of that. Just an idea. Yes, this would make sense if the highlighting would be reestablished automatically next time you visit the file. If not, it would be pretty useless in my eyes. IIUC, I do use something similar: automatic highlighting of some words, hooked on the mode (so, permanent... for me). --8---cut here---start-8--- (defface lvn/highlight-face '((t (:weight normal :slant normal :box '(:line-width 1 :color #CC) :foreground #CC :background #88))) Face for making FIXME and other warnings stand out.) (defvar lvn/highlight-org-regexps \\(FIXME\\|BUG\\|XXX\\|[Ee]rror\\|[Ww]arning\\|WARNING\\) Patterns to highlight (for Org mode only).) ;; set up highlighting of special patterns for Org mode only (dolist (mode '(org-mode)) (font-lock-add-keywords mode `((,lvn/highlight-org-regexps 1 'lvn/highlight-face prepend --8---cut here---end---8--- Best regards, Seb -- Sebastien Vauban
Re: [O] Enriched/Org is a colorful Org
On 10 apr. 2013, at 13:20, Sebastien Vauban wxhgmqzgw...@spammotel.com wrote: Carsten Dominik wrote: On 10 apr. 2013, at 12:48, Suvayu Ali fatkasuvayu+li...@gmail.com wrote: On Wed, Apr 10, 2013 at 12:16:28PM +0200, Carsten Dominik wrote: On 10 apr. 2013, at 11:54, Suvayu Ali fatkasuvayu+li...@gmail.com wrote: On Wed, Apr 10, 2013 at 09:32:44AM +0530, Jambunathan K wrote: See Side note towards the end of this message http://debbugs.gnu.org/cgi/bugreport.cgi?bug=14157#8 This request is common enough; every time it comes up overlays are proposed as a solution. It would be good if this is available even as a library outside of Org. Yes, overlays are better. However, maybe I am just no getting it, but what is even the purpose of facemenu? AFAICS, the faces are non-permanent, so when I save the file and reopen it, all the faces are gone. I really cannot see a useful application for this. AFAIR, the use cases presented so far involved adding highlighting-like information in Org files. If the overlays are generated consistently based on the user's setup, it doesn't matter if they are non-permanent (as in not part of the Org file, but dependent on the Emacs setup instead). Of course this means the highlighting information will not be portable; but I don't think people mind that. I personally do not find any use for the feature as such; although it might be interesting to be able to insert plain text cookies in Org files and have them highlighted in some fashion. I could then use a list of ideas like this: Some topic ... 1. Idea 1 2. Idea 2 (?) where I'm doubtful about idea (2); having (?) highlighted would remind me of that. Just an idea. Yes, this would make sense if the highlighting would be reestablished automatically next time you visit the file. If not, it would be pretty useless in my eyes. IIUC, I do use something similar: automatic highlighting of some words, hooked on the mode (so, permanent... for me). --8---cut here---start-8--- (defface lvn/highlight-face '((t (:weight normal :slant normal :box '(:line-width 1 :color #CC) :foreground #CC :background #88))) Face for making FIXME and other warnings stand out.) (defvar lvn/highlight-org-regexps \\(FIXME\\|BUG\\|XXX\\|[Ee]rror\\|[Ww]arning\\|WARNING\\) Patterns to highlight (for Org mode only).) ;; set up highlighting of special patterns for Org mode only (dolist (mode '(org-mode)) (font-lock-add-keywords mode `((,lvn/highlight-org-regexps 1 'lvn/highlight-face prepend --8---cut here---end---8--- Yes, this is definitely very useful, I do similar stuff for FIXME etc. - Carsten Best regards, Seb -- Sebastien Vauban
Re: [O] Enriched/Org is a colorful Org
Carsten Dominik carsten.domi...@gmail.com writes: On 10 apr. 2013, at 11:54, Suvayu Ali fatkasuvayu+li...@gmail.com wrote: On Wed, Apr 10, 2013 at 09:32:44AM +0530, Jambunathan K wrote: See Side note towards the end of this message http://debbugs.gnu.org/cgi/bugreport.cgi?bug=14157#8 This request is common enough; every time it comes up overlays are proposed as a solution. It would be good if this is available even as a library outside of Org. Yes, overlays are better. However, maybe I am just no getting it, but what is even the purpose of facemenu? AFAICS, the faces are non-permanent, so when I save the file and reopen it, all the faces are gone. I really cannot see a useful application for this. Look up `enriched-mode' or just visit enriched.doc in `data-directory' (/etc). facemenu.el is internally used by `enriched-mode'. Let the color markup - encoding and decoding - be persisted by enriched mode. - Carsten
Re: [O] Enriched/Org is a colorful Org
Suvayu Ali fatkasuvayu+li...@gmail.com writes: On Wed, Apr 10, 2013 at 09:32:44AM +0530, Jambunathan K wrote: See Side note towards the end of this message http://debbugs.gnu.org/cgi/bugreport.cgi?bug=14157#8 This request is common enough; every time it comes up overlays are proposed as a solution. It would be good if this is available even as a library outside of Org. It is possible with little work of existing libraries.
Re: [O] Enriched/Org is a colorful Org
;; set up highlighting of special patterns for Org mode only (dolist (mode '(org-mode)) (font-lock-add-keywords mode `((,lvn/highlight-org-regexps 1 'lvn/highlight-face prepend hi-lock.el is for regexp based highlighting. facemenu.el is for one-off, case-by-case highlighting. If facemenu.el can be enhanced to use overlays and have it co-exist with font lock based modes, one should be able to color code one's notes in a text/org file and have it saved in a disk. Best regards, Seb
Re: [O] Enriched/Org is a colorful Org
From: Carsten Dominik carsten.domi...@gmail.com Date: Wed, 10 Apr 2013 12:16:28 +0200 Cc: emacs-orgmode@gnu.org On 10 apr. 2013, at 11:54, Suvayu Ali fatkasuvayu+li...@gmail.com wrote: On Wed, Apr 10, 2013 at 09:32:44AM +0530, Jambunathan K wrote: See Side note towards the end of this message http://debbugs.gnu.org/cgi/bugreport.cgi?bug=14157#8 This request is common enough; every time it comes up overlays are proposed as a solution. It would be good if this is available even as a library outside of Org. Yes, overlays are better. I beg the Org developers to please be very careful when introducing expensive display features such as overlays into Org. Org already puts the Emacs display engine to its limits in many of its popular features; adding overlays to this mess might be too much. I don't know enough about Org to understand why overlays are being considered instead of text properties, but feel free to describe the issues (preferably on emacs-devel) and start a discussion about the possible alternatives.
Re: [O] Enriched/Org is a colorful Org
Eli Zaretskii e...@gnu.org writes: I beg the Org developers to please be very careful when introducing expensive display features such as overlays into Org. Org already puts the Emacs display engine to its limits in many of its popular features; adding overlays to this mess might be too much. We shouldn't be afraid of building a prototype or release the feature with a warning you are on your own. I don't know enough about Org to understand why overlays are being considered instead of text properties, but feel free to describe the issues (preferably on emacs-devel) and start a discussion about the possible alternatives. From Org side of things, it is the rigidity of Org syntax wrt emphasis etc. From user side of things, an ability to color code his notes (and possibly create an exported document that retains those highlights.) The intention is to really keep the discussion moving and see what comes out of it.
Re: [O] Enriched/Org is a colorful Org
From: Jambunathan K kjambunat...@gmail.com Cc: Carsten Dominik carsten.domi...@gmail.com, emacs-orgmode@gnu.org Date: Wed, 10 Apr 2013 22:13:46 +0530 Eli Zaretskii e...@gnu.org writes: I beg the Org developers to please be very careful when introducing expensive display features such as overlays into Org. Org already puts the Emacs display engine to its limits in many of its popular features; adding overlays to this mess might be too much. We shouldn't be afraid of building a prototype or release the feature with a warning you are on your own. Fear has nothing to do with this. Good software design takes into consideration the particulars and the limitation of the infrastructure, and tries to avoid limitations and weak spots in that infrastructure. Building a house on shaky ground is silly; at best it is waste of effort. From Org side of things, it is the rigidity of Org syntax wrt emphasis etc. From user side of things, an ability to color code his notes (and possibly create an exported document that retains those highlights.) I meant technical details. The above is too high-level to be useful. The intention is to really keep the discussion moving and see what comes out of it. My intent was to advise people not to go in directions that can hardly be fruitful. Of course, if they want to...
Re: [O] Enriched/Org is a colorful Org
On 10.4.2013, at 18:21, Eli Zaretskii e...@gnu.org wrote: From: Carsten Dominik carsten.domi...@gmail.com Date: Wed, 10 Apr 2013 12:16:28 +0200 Cc: emacs-orgmode@gnu.org On 10 apr. 2013, at 11:54, Suvayu Ali fatkasuvayu+li...@gmail.com wrote: On Wed, Apr 10, 2013 at 09:32:44AM +0530, Jambunathan K wrote: See Side note towards the end of this message http://debbugs.gnu.org/cgi/bugreport.cgi?bug=14157#8 This request is common enough; every time it comes up overlays are proposed as a solution. It would be good if this is available even as a library outside of Org. Yes, overlays are better. I beg the Org developers to please be very careful when introducing expensive display features such as overlays into Org. Org already puts the Emacs display engine to its limits in many of its popular features; Hi Eli, this is interesting input, I was not aware of this. Has this been discussed before, can you point me to relevant threads, and what are the symptoms of the display engine being at its limits? Regards - Carsten adding overlays to this mess might be too much. I don't know enough about Org to understand why overlays are being considered instead of text properties, but feel free to describe the issues (preferably on emacs-devel) and start a discussion about the possible alternatives.
Re: [O] Enriched/Org is a colorful Org
Hi Carsten, Carsten Dominik wrote: On 10.4.2013, at 18:21, Eli Zaretskii e...@gnu.org wrote: From: Carsten Dominik carsten.domi...@gmail.com On 10 apr. 2013, at 11:54, Suvayu Ali fatkasuvayu+li...@gmail.com wrote: This request is common enough; every time it comes up overlays are proposed as a solution. It would be good if this is available even as a library outside of Org. Yes, overlays are better. I beg the Org developers to please be very careful when introducing expensive display features such as overlays into Org. Org already puts the Emacs display engine to its limits in many of its popular features; this is interesting input, I was not aware of this. Has this been discussed before, can you point me to relevant threads, and what are the symptoms of the display engine being at its limits? adding overlays to this mess might be too much. I guess Eli simply means, in a general way, that overlays do negatively impact display performance, as you said as well a couple of times: ╭ │ From: Carsten Dominik carsten.domi...@gmail.com │ Subject: Re: performance problems with drawers │ Newsgroups: gmane.emacs.orgmode │ To: Al gman...@wilec.net │ Cc: emacs-orgmode@gnu.org │ Date: Wed, 8 Jul 2009 07:05:53 +0200 (3 years, 39 weeks, 3 days ago) │ │ Hi Al, │ │ first of all, I cannot reproduce the fact that drawers have such │ a major influence on time, wit a test file that I created to │ be similar to what you describe. │ │ There is a way to speed up drawer handling, by using text properties │ instead of overlays. How have some vague plans to do this, but nothing │ concrete or soon. │ │ - Carsten ╰ and ╭ │ From: Carsten Dominik carsten.domi...@gmail.com │ Subject: Re: fontification and icon issues │ Newsgroups: gmane.emacs.orgmode │ To: David O'Toole dto1...@gmail.com │ Cc: emacs-orgmode@gnu.org │ Date: Thu, 24 Sep 2009 10:46:24 +0100 (3 years, 28 weeks, 2 days ago) │ │ On Sep 22, 2009, at 4:11 PM, David O'Toole wrote: │ [...] │ │ 2. using add-text-properties to specify a display property (or even just │ a face) for any part of an org headline text kills the fontification of │ the rest of the text (TODO keyword and leading stars unaffected.) I'm │ trying to use font-lock-add-keywords to display the images. │ │ Can you make an example file, and maybe a small function that does set these │ properties, so that I can see what you mean? │ │ - Carsten │ │ Maybe I should use overlays instead? │ │ This can be done, but if every line in a very large file gets │ an overlay, performance is severely degraded. │ │ - Carsten ╰ I don't know enough about Org to understand why overlays are being considered instead of text properties, but feel free to describe the issues (preferably on emacs-devel) and start a discussion about the possible alternatives. Best regards, Seb -- Sebastien Vauban
Re: [O] Enriched/Org is a colorful Org
On 10.4.2013, at 22:16, Sebastien Vauban wxhgmqzgw...@spammotel.com wrote: Hi Carsten, Carsten Dominik wrote: On 10.4.2013, at 18:21, Eli Zaretskii e...@gnu.org wrote: From: Carsten Dominik carsten.domi...@gmail.com On 10 apr. 2013, at 11:54, Suvayu Ali fatkasuvayu+li...@gmail.com wrote: This request is common enough; every time it comes up overlays are proposed as a solution. It would be good if this is available even as a library outside of Org. Yes, overlays are better. I beg the Org developers to please be very careful when introducing expensive display features such as overlays into Org. Org already puts the Emacs display engine to its limits in many of its popular features; this is interesting input, I was not aware of this. Has this been discussed before, can you point me to relevant threads, and what are the symptoms of the display engine being at its limits? adding overlays to this mess might be too much. I guess Eli simply means, in a general way, that overlays do negatively impact display performance, as you said as well a couple of times: Yes, but Eli says that Org already severely tests the display engine, and he uses the word mess, even though we mostly use text properties for faces and other display-related things. So I was wondering if there is something we should put onto our todo list. Of course, Org already uses overlays, for example for folding (as does outline.el), and for temporary marking of text like during src block editing. But as your digging shows, I ave avoided them in the past, and we are also not using them for org-indent.el, for example. The reason why I said overlays would be better is simply that they would allow to add display properties in a persistent way that would not interfere that our font-lock-unfontify-region function removes face and invisibility text properties. So they are better for implementing hand-made faces selection that should overrule font-lock. - Carsten ╭ │ From: Carsten Dominik carsten.domi...@gmail.com │ Subject: Re: performance problems with drawers │ Newsgroups: gmane.emacs.orgmode │ To: Al gman...@wilec.net │ Cc: emacs-orgmode@gnu.org │ Date: Wed, 8 Jul 2009 07:05:53 +0200 (3 years, 39 weeks, 3 days ago) │ │ Hi Al, │ │ first of all, I cannot reproduce the fact that drawers have such │ a major influence on time, wit a test file that I created to │ be similar to what you describe. │ │ There is a way to speed up drawer handling, by using text properties │ instead of overlays. How have some vague plans to do this, but nothing │ concrete or soon. │ │ - Carsten ╰ and ╭ │ From: Carsten Dominik carsten.domi...@gmail.com │ Subject: Re: fontification and icon issues │ Newsgroups: gmane.emacs.orgmode │ To: David O'Toole dto1...@gmail.com │ Cc: emacs-orgmode@gnu.org │ Date: Thu, 24 Sep 2009 10:46:24 +0100 (3 years, 28 weeks, 2 days ago) │ │ On Sep 22, 2009, at 4:11 PM, David O'Toole wrote: │ [...] │ │ 2. using add-text-properties to specify a display property (or even just │ a face) for any part of an org headline text kills the fontification of │ the rest of the text (TODO keyword and leading stars unaffected.) I'm │ trying to use font-lock-add-keywords to display the images. │ │ Can you make an example file, and maybe a small function that does set these │ properties, so that I can see what you mean? │ │ - Carsten │ │ Maybe I should use overlays instead? │ │ This can be done, but if every line in a very large file gets │ an overlay, performance is severely degraded. │ │ - Carsten ╰ I don't know enough about Org to understand why overlays are being considered instead of text properties, but feel free to describe the issues (preferably on emacs-devel) and start a discussion about the possible alternatives. Best regards, Seb -- Sebastien Vauban
[O] Enriched/Org is a colorful Org
See Side note towards the end of this message http://debbugs.gnu.org/cgi/bugreport.cgi?bug=14157#8
