[O] Bug?/performance issue in org-icalendar-export-current-agenda
Hi all, I pulled from master recently (for the first time since the spring), and since the update, I have noticed a performance/lockup issue. The problem occurs when exporting an agenda view to iCalendar format. My Emacs version is 23.4.1 and my Org version is 8.3beta (I am currently on commit 767895...) I have traced the problem down to a call to `org-icalendar-export-current-agenda', though I don't know if the problem is in this function or in another function called by it. Somewhere during the execution of this function, Emacs goes to 100% CPU usage (on one core) and memory usage starts growing until I kill the process. Here's what my setup looks like. I have just one entry in `org-agenda-custom-commands' that defines the files field. It looks like: (Z Export to iCalendar file tags +event|+appointment nil (~/Documents/website/public_html/lib/attachments/calendar.ics) I normally only call this command via a cron job that exports my agenda files to .ics, which until recently was working fine. But I have noticed I can reproduce the lockup interactively by building an agenda view with this command, and then calling `org-icalendar-export-current-agenda' on it (which is also what happens, a few steps down the call chain, during the cron job). I have run some tests by manually setting org-agenda-files, then building this agenda and eval'ing (org-icalendar-export-current-agenda /tmp/agenda.ics) in the agenda buffer. The results: 1) Using just a simple three-entry test file, the export completes quickly and I don't see the lockup. 2) Using an agenda file which contains plenty of real data (about 1000 lines) but no headlines tagged event or appointment (i.e., a file for which this agenda view is empty), the call to `org-icalendar-export-current-agenda' takes about 5 seconds to complete, even though there are no entries to export. During those 5 seconds, the Emacs process goes to 100% CPU usage. 3) Using a different agenda file (of about 4000 lines) which contains about 75 matching headlines, the call to `org-icalendar-export-current-agenda' does not complete within several minutes, during which time I see 100% CPU usage and rapid memory usage growth. So it looks to me like maybe some part of the export process uses exponentially more time and memory, depending on the input, since a minimal test case does not produce the problem. What's the next step for debugging this? Please let me know if I can provide more information. Thanks! -- Best, Richard
Re: [O] Bug?/performance issue in org-icalendar-export-current-agenda
El mar, 26 ago 2014, Richard Lawrence decĂa: Hi all, I pulled from master recently (for the first time since the spring), and since the update, I have noticed a performance/lockup issue. The problem occurs when exporting an agenda view to iCalendar format. My Emacs version is 23.4.1 and my Org version is 8.3beta (I am currently on commit 767895...) I have traced the problem down to a call to `org-icalendar-export-current-agenda', though I don't know if the problem is in this function or in another function called by it. Somewhere during the execution of this function, Emacs goes to 100% CPU usage (on one core) and memory usage starts growing until I kill the process. Here's what my setup looks like. I have just one entry in `org-agenda-custom-commands' that defines the files field. It looks like: (Z Export to iCalendar file tags +event|+appointment nil (~/Documents/website/public_html/lib/attachments/calendar.ics) I normally only call this command via a cron job that exports my agenda files to .ics, which until recently was working fine. But I have noticed I can reproduce the lockup interactively by building an agenda view with this command, and then calling `org-icalendar-export-current-agenda' on it (which is also what happens, a few steps down the call chain, during the cron job). I have run some tests by manually setting org-agenda-files, then building this agenda and eval'ing (org-icalendar-export-current-agenda /tmp/agenda.ics) in the agenda buffer. The results: 1) Using just a simple three-entry test file, the export completes quickly and I don't see the lockup. 2) Using an agenda file which contains plenty of real data (about 1000 lines) but no headlines tagged event or appointment (i.e., a file for which this agenda view is empty), the call to `org-icalendar-export-current-agenda' takes about 5 seconds to complete, even though there are no entries to export. During those 5 seconds, the Emacs process goes to 100% CPU usage. 3) Using a different agenda file (of about 4000 lines) which contains about 75 matching headlines, the call to `org-icalendar-export-current-agenda' does not complete within several minutes, during which time I see 100% CPU usage and rapid memory usage growth. So it looks to me like maybe some part of the export process uses exponentially more time and memory, depending on the input, since a minimal test case does not produce the problem. What's the next step for debugging this? Please let me know if I can provide more information. Thanks! -- Best, Richard I use a bash script: =agenda-ascii= [0] to reduce the list of file to export, first =grep= current =year-month= pattern in the *.org files: #+BEGIN_EXAMPLE ORG_FILES=$($GREP -l $YEAR-$MONTH *.org) echo $ORG_FILES if [ $EXPORT_ICS = TRUE ] then for AGENDA in $(echo $ORG_FILES) do echo EXPORT $AGENDA ... ARCHIVO=$DIR/$AGENDA $EMACS --batch -q --directory=${DIR} --visit=${ARCHIVO} --eval '(org-export-icalendar-this-file)' /dev/null 21 done fi #+END_EXAMPLE Actually the process, cosume 42%-97% CPU, for 526 appoinments, takes 0m 30.396s Org-mode version 8.3beta (release_8.3beta-264-g7a5f3f GNU Emacs 24.3.1 (x86_64-pc-linux-gnu) of 2014-06-06 on barber, modified by Debian [0] https://github.com/osiris/org-bash-utils/blob/master/agenda-ascii -- :: Osiris Alejandro Gomez (OSiUX) os...@osiux.com.ar DC44 95D2 0D5D D544 FC1A F00F B308 A671 9237 D36C http://www.osiux.com.ar http://www.altermundi.net signature.asc Description: Digital signature
Re: [O] Bug?/performance issue in org-icalendar-export-current-agenda
Hello, Richard Lawrence richard.lawre...@berkeley.edu writes: I pulled from master recently (for the first time since the spring), and since the update, I have noticed a performance/lockup issue. The problem occurs when exporting an agenda view to iCalendar format. My Emacs version is 23.4.1 and my Org version is 8.3beta (I am currently on commit 767895...) I have traced the problem down to a call to `org-icalendar-export-current-agenda', though I don't know if the problem is in this function or in another function called by it. Somewhere during the execution of this function, Emacs goes to 100% CPU usage (on one core) and memory usage starts growing until I kill the process. Interestingly, I introduced a patch that should speed up this function yesterday. You may want to update Org (the patch landed after 767895) and try again. If it is still suspiciously slow, please use profiler (`profiler-start') and report the results here. Regards, -- Nicolas Goaziou
Re: [O] Bug?/performance issue in org-icalendar-export-current-agenda
Nicolas Goaziou m...@nicolasgoaziou.fr writes: Interestingly, I introduced a patch that should speed up this function yesterday. You may want to update Org (the patch landed after 767895) and try again. Indeed! I just pulled again and the problem is gone. The export runs much more reasonably now, completing in a few seconds even with all my agenda files in the mix. Thanks! Best, Richard