Re: [O] Converting from OmniFocus to org-mode
Hi Jay, Jay McCarthy jay.mccar...@gmail.com writes: I'm an avid OmniFocus user and I've been thinking about trying org-mode. Unless I misread this thread, we don't know what version of Emacs you are using -- this is crucial information for usability problems such as the one you are reporting. https://github.com/jeapostrophe/exp/blob/master/of2org.rkt Thanks for this! Can you write a small paragraph presenting the tool so that I add it to http://orgmode.org/worg/org-tools/index.html? However, once I converted my 1.6M OmniFocus database, I got a 252K org-mode file. There should be no problem with such file -- my main org file is currently 900k and I'm not experiencing major slowliness. I am using Emacs from bzr, compiled a few days ago 2011-08-13. Best, -- Bastien
Re: [O] Converting from OmniFocus to org-mode
Hi Jay, Jay McCarthy jay.mccar...@gmail.com writes: It is worth disabling font-lock in Org buffers. I am not sure whether that would prevent you from making use of any of the Org features. I don't want to use it that way, but I can check if it speeds it up and it doesn't make any appreciable difference on the speed and the crash still happens. Slowliness often comes from using many text overlays, which are used fondamentally by outline-mode, the mode org-mode is derived from. Turning off font-lock will not do anything about the problem you're reporting. Please bear with us and Emacs developers while we are improving things is this area, this is quite a common request and should be a priority. Best, -- Bastien
Re: [O] Converting from OmniFocus to org-mode
Hi Sébastien, Sebastien Vauban wxhgmqzgw...@spammotel.com writes: As far as I know, Bastien began talking of caches with the author of Emacs-calfw, but nothing has been done yet in that direction. Confirmed. Once I'm done doing plumbering work and catching up emails, I'll put a stab at this. Best, -- Bastien
Re: [O] Converting from OmniFocus to org-mode
On Mon, Aug 15, 2011 at 2:37 AM, Bastien b...@altern.org wrote: Hi Jay, Jay McCarthy jay.mccar...@gmail.com writes: I'm an avid OmniFocus user and I've been thinking about trying org-mode. Unless I misread this thread, we don't know what version of Emacs you are using -- this is crucial information for usability problems such as the one you are reporting. I thought that this information in the stack report would be enough: Identifier: org.gnu.Emacs Version: Version 23.3 (9.0) OS Version: Mac OS X 10.7 (11A511) It's the version from http://emacsformacosx.com/ What else should I send? https://github.com/jeapostrophe/exp/blob/master/of2org.rkt Thanks for this! Can you write a small paragraph presenting the tool so that I add it to http://orgmode.org/worg/org-tools/index.html? A script to convert the internal database of OmniFocus to org-mode with minimal metadata loss. In particular, it maintains repeating events and multi-line notes fields. However, once I converted my 1.6M OmniFocus database, I got a 252K org-mode file. There should be no problem with such file -- my main org file is currently 900k and I'm not experiencing major slowliness. I am using Emacs from bzr, compiled a few days ago 2011-08-13. Any idea what's likely to be my problem? Jay -- Jay McCarthy j...@cs.byu.edu Assistant Professor / Brigham Young University http://faculty.cs.byu.edu/~jay The glory of God is Intelligence - DC 93
Re: [O] Converting from OmniFocus to org-mode
On Mon, Aug 15, 2011 at 1:32 PM, Jay McCarthy jay.mccar...@gmail.com wrote: What else should I send? M-x org-version -- Suvayu Open source is the future. It sets us free.
Re: [O] Converting from OmniFocus to org-mode
It was the latest git when I sent the email (August 5th) Jay On Mon, Aug 15, 2011 at 6:02 AM, suvayu ali fatkasuvayu+li...@gmail.com wrote: On Mon, Aug 15, 2011 at 1:32 PM, Jay McCarthy jay.mccar...@gmail.com wrote: What else should I send? M-x org-version -- Suvayu Open source is the future. It sets us free. -- Jay McCarthy j...@cs.byu.edu Assistant Professor / Brigham Young University http://faculty.cs.byu.edu/~jay The glory of God is Intelligence - DC 93
Re: [O] Converting from OmniFocus to org-mode
On Mon, 15 Aug 2011 06:09:53 -0600 Jay McCarthy jay.mccar...@gmail.com wrote: It was the latest git when I sent the email (August 5th) Knowingly precisely which commit always helps. Hence it is customary to post the output of the command I mentioned. M-x org-version or the output of 'git describe' from the shell in the org directory. Jay Hope this is clearer. -- Suvayu Open source is the future. It sets us free.
Re: [O] Converting from OmniFocus to org-mode
On Mon, Aug 15, 2011 at 6:16 AM, Suvayu Ali fatkasuvayu+li...@gmail.com wrote: On Mon, 15 Aug 2011 06:09:53 -0600 Jay McCarthy jay.mccar...@gmail.com wrote: It was the latest git when I sent the email (August 5th) Knowingly precisely which commit always helps. Hence it is customary to post the output of the command I mentioned. M-x org-version 7.7.20.g4447 or the output of 'git describe' from the shell in the org directory. release_7.7-20-g44472fa -- Jay McCarthy j...@cs.byu.edu Assistant Professor / Brigham Young University http://faculty.cs.byu.edu/~jay The glory of God is Intelligence - DC 93
Re: [O] Converting from OmniFocus to org-mode
On Mon, 15 Aug 2011 06:19:28 -0600 Jay McCarthy jay.mccar...@gmail.com wrote: or the output of 'git describe' from the shell in the org directory. release_7.7-20-g44472fa Does this happen with emacs -Q -l /path/to/minimal/org-setup.el. If it is repeatable with the above, can you try to isolate the problematic part in your org file and post a minimal example for the devs to work with? There is a small FAQ entry about what is expected of a minimal example http://orgmode.org/worg/org-faq.html#bug-reporting As for a minimal org setup after 'emacs -Q', this is what I use: (add-to-list 'load-path (expand-file-name ~/build/org-mode/lisp)) (add-to-list 'load-path (expand-file-name ~/build/org-mode/contrib/lisp)) ;; activate org (require 'org-install) I hope this helps you isolating your problem and reporting back with more details. GL -- Suvayu Open source is the future. It sets us free.
Re: [O] Converting from OmniFocus to org-mode
This was very useful. I didn't have the problem with the minimal setup. I then went through my global modes, turning them off until I found that by turning off global-linum-mode it drastically improved the performance of org-mode and removed the crash. Great! Now, I'll try org-mode for realsies. Jay On Mon, Aug 15, 2011 at 6:50 AM, Suvayu Ali fatkasuvayu+li...@gmail.com wrote: On Mon, 15 Aug 2011 06:19:28 -0600 Jay McCarthy jay.mccar...@gmail.com wrote: or the output of 'git describe' from the shell in the org directory. release_7.7-20-g44472fa Does this happen with emacs -Q -l /path/to/minimal/org-setup.el. If it is repeatable with the above, can you try to isolate the problematic part in your org file and post a minimal example for the devs to work with? There is a small FAQ entry about what is expected of a minimal example http://orgmode.org/worg/org-faq.html#bug-reporting As for a minimal org setup after 'emacs -Q', this is what I use: (add-to-list 'load-path (expand-file-name ~/build/org-mode/lisp)) (add-to-list 'load-path (expand-file-name ~/build/org-mode/contrib/lisp)) ;; activate org (require 'org-install) I hope this helps you isolating your problem and reporting back with more details. GL -- Suvayu Open source is the future. It sets us free. -- Jay McCarthy j...@cs.byu.edu Assistant Professor / Brigham Young University http://faculty.cs.byu.edu/~jay The glory of God is Intelligence - DC 93
Re: [O] Converting from OmniFocus to org-mode
Hi Jay, On Mon, Aug 15, 2011 at 6:12 PM, Jay McCarthy jay.mccar...@gmail.com wrote: This was very useful. I didn't have the problem with the minimal setup. I then went through my global modes, turning them off until I found that by turning off global-linum-mode it drastically improved the performance of org-mode and removed the crash. Great! Now, I'll try org-mode for realsies. Thanks for reporting back. This has been reported at least once before[1]. Maybe its time to document this on Worg. I hope you enjoy orgmode (minus line numbers ;)). To get started I would suggest going through the wonderful tutorials on Worg[2]. Depending on what you expect from orgmode pick _one_ of the tutorials and get started. As and when you come up with circumstances where you need new features, you will usually find its already available within orgmode and there is a corresponding discussion on Worg. In the off chance the required feature is missing, asking on the list will probably get you that feature within a few weeks! Jay GL [1] http://thread.gmane.org/gmane.emacs.orgmode/37860/focus=37864 [2] http://orgmode.org/worg/ -- Suvayu Open source is the future. It sets us free.
Re: [O] Converting from OmniFocus to org-mode
Jay McCarthy jay.mccar...@gmail.com wrote: This was very useful. I didn't have the problem with the minimal setup. I then went through my global modes, turning them off until I found that by turning off global-linum-mode it drastically improved the performance of org-mode and removed the crash. Great! Now, I'll try org-mode for realsies. It used to be the case (and probably still is, given your experience above) that linum-mode is expensive: if you can live without it, you'd be much better off. emacs tries to make it possible to go places without having to know line numbers (e.g M-x compile will print out error messages with line numbers but all you have to do is C-x ` to find the next error message *and* get to the source code that caused it). That's generally a better way to go. It's not always possible to do without line numbers (although you can come pretty close), so there is a mode to print out the current line number in the mode line which is not nearly as expensive: line-number-mode. I have that on all the time and I do not notice any performance impact. Nick
Re: [O] Converting from OmniFocus to org-mode
Jay McCarthy jay.mccar...@gmail.com writes: I thought that this information in the stack report would be enough: Identifier: org.gnu.Emacs Version: Version 23.3 (9.0) OS Version: Mac OS X 10.7 (11A511) It's the version from http://emacsformacosx.com/ So yes, I correctly misread the thread ;) Thanks for the additional info, glad you made some progress! -- Bastien
[O] Converting from OmniFocus to org-mode
I'm an avid OmniFocus user and I've been thinking about trying org-mode. The conversion tool on Worg [1] was inadequate because it didn't capture whether the repetition in tasks and it didn't handle the notes field properly for me. I've written a new tool that works directly on the XML database behind OmniFocus. I specifically ignored completed tasks, whether something was flagged, and durations, because I don't use those so I couldn't test them. It is written in Racket [2] and available here: https://github.com/jeapostrophe/exp/blob/master/of2org.rkt However, once I converted my 1.6M OmniFocus database, I got a 252K org-mode file. Opening this file in org-mode is unbearable. For example, pressing up after opening the document (and having the cursor at the end of the buffer) takes 5 seconds to select the last item. Pressing up again takes 6 seconds. Almost every trivial browsing operation like this takes 5 seconds. (Except expanding the notes fields once a sub-tree has been revealed. That's instant.) Finally, the killer. If I select the last top-level heading and press M-up, Emacs crashes. A section of the stack trace is at the end of this email. I'm happy to send my org-mode file to a developer to try out, but I don't want to post it online. FWIW, the file is 6738 lines long and the final heading starts on line 2632, so M-up is moving a significant number of lines. I updated to the latest git version of org-mode to see if anything was faster, but it wasn't. I find org-mode theoretically appealing, but as it stands, I can't use it because it isn't scalable enough. Are there any techniques to getting around this lack of scalability? I'd appreciated any advice. Best, Jay 1. https://bitbucket.org/legoscia/of2org/src 2. http://racket-lang.org/ -- The stack trace -- Process: Emacs [5146] Path:/Users/USER/*/Emacs.app/Contents/MacOS/Emacs Identifier: org.gnu.Emacs Version: Version 23.3 (9.0) Code Type: X86-64 (Native) Parent Process: launchd [176] Date/Time: 2011-08-05 09:36:46.341 -0600 OS Version: Mac OS X 10.7 (11A511) Report Version: 9 Interval Since Last Report: 201400 sec Crashes Since Last Report: 5 Per-App Interval Since Last Report: 231997 sec Per-App Crashes Since Last Report: 2 Anonymous UUID: 91A4D920-5BEC-4689-B878-21946C6D436B Crashed Thread: 0 Dispatch queue: com.apple.main-thread Exception Type: EXC_BAD_ACCESS (SIGABRT) Exception Codes: KERN_INVALID_ADDRESS at 0x7fff5fc0 VM Regions Near 0x7fff5fc0: Stack 7fff5f40-7fff5fc0 [ 8192K] rw-/rwx SM=COW thread 0 -- __TEXT 7fff61e0e000-7fff61e43000 [ 212K] r-x/rwx SM=COW /usr/lib/dyld Application Specific Information: objc[5146]: garbage collection is OFF abort() called Thread 0 Crashed:: Dispatch queue: com.apple.main-thread 0 libsystem_kernel.dylib 0x7fff867cd82a __kill + 10 1 org.gnu.Emacs 0x0001000901ae fatal_error_signal + 286 2 libsystem_c.dylib 0x7fff84905cfa _sigtramp + 26 3 libsystem_kernel.dylib 0x7fff867cdce2 __pthread_kill + 10 4 libsystem_c.dylib 0x7fff848b37d2 pthread_kill + 95 5 libsystem_c.dylib 0x7fff848a4a7a abort + 143 6 org.gnu.Emacs 0x00010016f31c ns_term_shutdown + 44 7 org.gnu.Emacs 0x0001000901e4 fatal_error_signal + 340 8 libsystem_c.dylib 0x7fff84905cfa _sigtramp + 26 9 org.gnu.Emacs 0x0001000281a0 handle_face_prop + 256 10 org.gnu.Emacs 0x000100028b0f handle_stop + 127 11 org.gnu.Emacs 0x00010002e3b1 next_element_from_string + 273 12 org.gnu.Emacs 0x00010002b2a1 get_next_display_element + 81 13 org.gnu.Emacs 0x00010002c6e5 move_it_in_display_line_to + 245 14 org.gnu.Emacs 0x00010002db9e move_it_to + 302 15 org.gnu.Emacs 0x0001000393c1 move_it_vertically + 65 16 org.gnu.Emacs 0x00010004ef95 Fwindow_end + 325 17 org.gnu.Emacs 0x0001001093f1 Ffuncall + 1201 18 org.gnu.Emacs 0x0001001463be Fbyte_code + 6814 19 org.gnu.Emacs 0x000100108e9c funcall_lambda + 588 20 org.gnu.Emacs 0x000100109192 Ffuncall + 594 21 org.gnu.Emacs 0x00010010aade call1 + 30 -- Jay McCarthy j...@cs.byu.edu Assistant Professor / Brigham Young University http://faculty.cs.byu.edu/~jay The glory of God is Intelligence - DC 93
Re: [O] Converting from OmniFocus to org-mode
A section of the stack trace is at the end of this email. Crash is really in core Emacs (re-display engine may be?) triggered by Org related things. It is worth disabling font-lock in Org buffers. I am not sure whether that would prevent you from making use of any of the Org features. If you could bisect the Org file and narrow down the crash to a smaller example file that would be wonderful. Can you give us an idea of number of scheduled entries per day and how many months they typically span. I know Org uses cache so may be the performance hit is one time only. May be there could be a script that populates the cache as part of first step in the migration process. Btw, I find the use of theoretically a bit inconsistent in your email. As a developer myself, I wouldn't go to the extent of a writing a migration assistant just to reap purely theoretical benefits. Surely I am nitpicking here. Jambunathan K.
Re: [O] Converting from OmniFocus to org-mode
On Sat, Aug 6, 2011 at 4:35 AM, Jambunathan K kjambunat...@gmail.com wrote: A section of the stack trace is at the end of this email. Crash is really in core Emacs (re-display engine may be?) triggered by Org related things. It is worth disabling font-lock in Org buffers. I am not sure whether that would prevent you from making use of any of the Org features. I don't want to use it that way, but I can check if it speeds it up and it doesn't make any appreciable difference on the speed and the crash still happens. If you could bisect the Org file and narrow down the crash to a smaller example file that would be wonderful. Can you give us an idea of number of scheduled entries per day and how many months they typically span. % grep -e DEADLINE -e SCHEDULED brain.org | wc -l 462 % grep -e DEADLINE brain.org| sort | head -1 DEADLINE: 2011-08-01 17:00 +1m % grep -e DEADLINE brain.org| sort | tail -1 DEADLINE: 2015-09-05 00:00 % grep -e DEADLINE brain.org| sort | uniq -c | sort | tail -10 3 DEADLINE: 2012-04-01 17:00 +1y 4 DEADLINE: 2011-08-04 23:00 +1d 4 DEADLINE: 2011-08-05 17:00 +1d 4 DEADLINE: 2012-05-01 17:00 +1y 4 DEADLINE: 2012-09-01 17:00 +1y 5 DEADLINE: 2011-08-05 17:00 +1w 5 DEADLINE: 2011-08-06 17:00 +1w 7 DEADLINE: 2011-09-01 17:00 8 DEADLINE: 2011-08-05 21:00 +1d 14 DEADLINE: 2011-09-01 17:00 +1m Is this useful information? I know Org uses cache so may be the performance hit is one time only. May be there could be a script that populates the cache as part of first step in the migration process. That sounds like a very good idea. http://www.google.com/search?q=org-mode%20cache Only tells me about the caching on the evaluation of code blocks, so I'm not in a position to implement it, as someone who's never actually used org-mode. Btw, I find the use of theoretically a bit inconsistent in your email. As a developer myself, I wouldn't go to the extent of a writing a migration assistant just to reap purely theoretical benefits. Surely I am nitpicking here. I mean that it seems appealing, but I have no idea because I've never used it. I need to write a migration assistant before I could even figure out if there was a benefit. Jay -- Jay McCarthy j...@cs.byu.edu Assistant Professor / Brigham Young University http://faculty.cs.byu.edu/~jay The glory of God is Intelligence - DC 93
Re: [O] Converting from OmniFocus to org-mode
Hi Jambunathan and Jay Jambunathan K wrote: If you could bisect the Org file and narrow down the crash to a smaller example file that would be wonderful. Can you give us an idea of number of scheduled entries per day and how many months they typically span. I know Org uses cache so may be the performance hit is one time only. As far as I know, Bastien began talking of caches with the author of Emacs-calfw, but nothing has been done yet in that direction. So, Jay's performance problem must be identical be it the first time or not. May be there could be a script that populates the cache as part of first step in the migration process. Best regards, Seb -- Sebastien Vauban