Re: [O] Sync up the org in emacs master to org maint branch?

[David Engster]

Edward John Steere writes:
>  - Suppose that Emacs 22.0 is the current release and Emacs 22.1 is then
>released; CEDET is at 
>  - we update a registry somewhere indicating that Emacs 22.0 works with
> and 22.1 works with
>
>  - When we make updates to CEDET we mark 22.1 as working with
> but we don't change that reference
>for 22.0 (or any older versions)
>  - When someone complains that there's a bug in CEDET for 22.0 we
>indicate that it's no longer supported and that they should update
>Emacs to receive updates
>
> This process would almost be the same as what you get just by bundling
> CEDET with Emacs except that:
>
>  - You can get the latest CEDET *if* you have the latest Emacs

No. We have two branches: emacs-25 and master. The CEDET from master
will usually not work on any 25.x version.

>  - The version of CEDET for any particular version of Emacs is as far as
>CEDET got before the next release of Emacs came out
>
> If this is what you were thinking of then please could you elaborate on
> what ended up being the problem which added more work.

First off, CEDET is currently *not* a package, although that notion gets
thrown around a lot. It is scattered across the Emacs code base:
lisp/cedet, admin/grammars, etc/srecode, documentation, and test
suite. All this needs to be packaged in a way so that it can be
integrated into Emacs during a normal checkout. It needs to build and
test in such a normal checkout, but also separately when installed from
ELPA, including grammar compilation. And you need this twice: one for
emacs-25, one for master, with the possiblity to merge between the two.

Then there's this "registry". No one has said how that should
work. "Submodules/Subtrees" are *not* a sufficient answer, they are just
tools. People will need to say how the *workflow* should be, including
such things like merges from stable, ChangeLog generation, AUTHORS,
NEWS, creating release tarballs, and so on. Once someone has written
this down *in detail*, we can discuss again if this indeed will make
things simpler and reduce our workload.

> I feel like there are aspects of CEDET which are still under
> development.

I hope so.

>> Well, we cannot really discuss this since there's no real plan how this
>> all should work. I can only speak from experience.
>
> We can still put ideas forward though.  Haven't come up with anything
> myself yet though.

Yes, you can, but it has a cost. Once again, the CEDET merge is stalled,
and we spend our time writing mails. I find this situation incredibly
frustrating.

>> How does CEDET, Gnus and Org affect the rest of Emacs? They strongly
>> depend on Emacs "core" (whatever exactly that is), not the other way
>> round.
>
> I believe that one of the intentions of the move is to enforce this so
> we can't build bad dependencies -- am I wrong?

I think other modes should use Semantic.

> Perhaps I'm wrong, but to my mind the package approach would afford us
> with more testing before we get to the point of another release of
> Emacs.

If you develop on Emacs 'master', you can use all the new features (like
threading and FFI), but you loose testers on 25.x. A package won't
change that.

-David

Re: [O] Sync up the org in emacs master to org maint branch?

[David Engster]

Edward John Steere writes:
>> It's not like packages communicate with Emacs over a well
>> defined RESTful interface. In other words: CEDET and Gnus are not
>> loosely coupled, quite the opposite: they are extremely dependend on
>> many obscure Emacs internals (not sure about Org).
>
> Shouldn't we be trying to avoid this situation?  It's sure to come back
> and bite us in the future.  If we continue to develop a package
> (wherever it ends up being developed) which is so tightly coupled that
> any kind of re factoring in core becomes a nightmare, because we have to
> consider the umpteen ways in which it'll break the package, then surely
> that's a bad methodology to continue?

I don't know what you have in mind. All I can say is: CEDET couldn't do
what it does if we'd restrict ourselves to some subset of Emacs.

>> As a consequence, we and also the Gnus guys decided to not do
>> separate releases anymore.  We used to provide CEDET for different
>> Emacs versions, and it was a *huge* amount of work. I had a buildbot
>> running with 7 or 8 Emacs versions to run the test suite, and things
>> broke pretty regularly.  Now you might say: fine, only release a
>> package for current master. But let's say we put CEDET into
>> ELPA. Emacs 26 gets released, and work on Emacs 27 starts. Now there
>> are changes happening in Emacs 27 that require changes in CEDET,
>> which make it incompatible with Emacs 26. So you have to create two
>> packages: one for 26, one for 27? Not only is this confusing, but it
>> most definitely increases my workload.
>
> I feel like this problem isn't intractable.

I didn't say it's intractable. I just said it means more work for me.

> We can mark some state of CEDET as being stable under the various
> versions of Emacs (because it was at the time) and then only support
> the current release for the latest package.  This would most likely
> require changes to package to ensure that you get an appropriate
> version of the package when you download it.

As I said: we did that. It was a huge amount of work. Please understand
where I'm coming from: if you look through the CEDET history, you will
see that in the past few years I almost exclusively did infrastructure
work and no real coding. All I can say is: *I* won't do this anymore,
and I don't want to be part in something which will increase grunt
work. We did not make this decision lightly. But with the amount of
developers we have, I'm convinced it's the right thing to do.

> Consider the problem which our users currently face.  The built in CEDET
> is miles behind and missing very important bug fixes and features.  The
> upstream CEDET can be a real pain to setup, but it has the latest
> features.  This is not a good place to be.  If we merge CEDET in and
> only release it with the realeses of Emacs then we are heading for a
> state where you only get the new features and bug fixes every time Emacs
> is realesed, i.e. our users might have to wait up to two years to have
> something fixed.  This is also a bad place to be.

Bug fixes can go with point release, which we should have every
year. But yes, if you want the latest, greatest and buggiest, you'll
have to use Emacs master. But that goes for a lot of Emacs features, so
I don't see why it's particularly bad for CEDET.

>>> We can arrange things so that a Git clone of Emacs includes pulling the
>>> submodules (or trees, or ELPA.git, or what not) that are considered part of
>>> "main Emacs development", including some of those batteries. I see this all 
>>> as
>>> a process issue, and that "living in one Git repository" has just been an
>>> implementation strategy to satisfy that process.
>>
>> Obviously, I'm very skeptical towards such an approach. Our tooling
>> around Emacs development is already very intricate and only works
>> because a few people work quietly behind the scenes to keep this all
>> running. Introducing even more complexity through
>> submodules/subtrees/whatever will do the opposite of what you want to
>> achieve: it creates more work for those few people who manage the Emacs
>> infrastructure. But I'd love to hear what for instance Glenn and Paul
>> think about this.
>
> I'm interested in exploring more with regards to how the subtree
> approach would work.  In particular how it would impact the Makefiles
> and build process.  I don't think that introducing features like this
> necessarily increases the burden of maintaining our tooling.  If we get
> it right it could reduce it.

Well, we cannot really discuss this since there's no real plan how this
all should work. I can only speak from experience.

> I think that it's a worthwhile goal to make core smaller.  It may not be
> a gigantic enterprise system with tens of thousands of source files,
> like some of us are accustomed to working on, but I think that it
> becomes easier to reason about the features and behaviour of core when
> it's smaller.

How does CEDET, Gnus and Org affect the rest of Emacs? They 

Re: [O] Sync up the org in emacs master to org maint branch?

[David Engster]

John Wiegley writes:
> Many software projects have package ecosystems surrounding them that deal with
> similar issues, and I can't, off-hand, think of cases where the answer was
> "let's move some of those packages into core development to ease
> ___".

Just from the software I worked with over the years, I remember
KDevelop, Typo3, Drupal and MediaWiki moving plugins to core. It's
really not uncommon. I still use Drupal, and moving important plugins to
core has made updates *much* less brittle. I just wish Jenkins would do
the same already...

-David

Re: [O] Sync up the org in emacs master to org maint branch?

[David Engster]

John Wiegley <jwieg...@gmail.com> writes:
>>>>>> "DE" == David Engster <d...@randomsample.de> writes:
>
> DE> So if you don't get convinced, we'll just move again, right? No big deal.
>
> I suppose I'm asking that of you, yes.

Sorry, but I rather wait.

> DE> You are insinuating that my motivation is to delegate CEDET development to
> DE> the core Emacs developers. This is simply not true, and I don't see how my
> DE> original mail could be interpreted like that.
>
> I didn't mean to insinuate anything; it seems we may have gotten off on the
> wrong foot, my intention is to make your life easier, not harder. If all this
> would do is make more work for people, it's not worth it.

This will most definitely make things harder for me. 

> DE> So let me try again: What I find completely misguided is to move packages
> DE> out of core *but still putting them into the release*. In other words, in
> DE> my opinion there are really just two options that make sense: you either
[+]
> DE> keep a package in core, or you kick it out and don't ship it with the
> DE> release.
>
> Why is this so? Right now I see the Emacs release as more than just releasing
> Emacs core; it's more of a "batteries included" release, combining the editor
> with lots of other default packages. It makes sense to me to move these
> batteries outside the core repository, than to put them all together in the
> same Git repository.

For package developers, keeping up with Emacs has become much harder in
recent years, as its development has accelerated (which is a good
thing). It's not like packages communicate with Emacs over a well
defined RESTful interface. In other words: CEDET and Gnus are not
loosely coupled, quite the opposite: they are extremely dependend on
many obscure Emacs internals (not sure about Org). As a consequence, we
and also the Gnus guys decided to not do separate releases anymore.  We
used to provide CEDET for different Emacs versions, and it was a *huge*
amount of work. I had a buildbot running with 7 or 8 Emacs versions to
run the test suite, and things broke pretty regularly.

Now you might say: fine, only release a package for current master. But
let's say we put CEDET into ELPA. Emacs 26 gets released, and work on
Emacs 27 starts. Now there are changes happening in Emacs 27 that
require changes in CEDET, which make it incompatible with Emacs 26. So
you have to create two packages: one for 26, one for 27? Not only is
this confusing, but it most definitely increases my workload.

> We can arrange things so that a Git clone of Emacs includes pulling the
> submodules (or trees, or ELPA.git, or what not) that are considered part of
> "main Emacs development", including some of those batteries. I see this all as
> a process issue, and that "living in one Git repository" has just been an
> implementation strategy to satisfy that process.

Obviously, I'm very skeptical towards such an approach. Our tooling
around Emacs development is already very intricate and only works
because a few people work quietly behind the scenes to keep this all
running. Introducing even more complexity through
submodules/subtrees/whatever will do the opposite of what you want to
achieve: it creates more work for those few people who manage the Emacs
infrastructure. But I'd love to hear what for instance Glenn and Paul
think about this.

> Why do the split at all? Core becomes smaller,

First off, the Emacs repo isn't that big w.r.t. the number of
files. Secondly, while there surely is an inverse correlation between
repo size and maintainability, I would argue that as long as the Big
Three are well maintained, they are not the problem. When did CEDET,
Gnus or Org ever significantly delay an Emacs release?  When was an
Emacs core developer ever forced to fix a critical thing in those
packages he did not break?  These are the questions we should be
asking. From watching this list over the past years, I don't get the
feeling that the inclusion of these packages has been a significant
burden, but I may be wrong.

> its future history less cluttered,

That's actually a bit funny, since we gave up an uncluttered history
when we switched to git's spaghetti DAG.

> updating packages within it is no longer a major issue, and (I hope)
> it will be clearer when something is a core issue vs. a package issue.

I find this whole core vs package argument strange. If you ship Emacs
with Org, Gnus and CEDET, they are part of Emacs, so it's in the
interest of all Emacs developers that they work well, whether they use
them or not. The users won't care if they originate from a separate repo
and are considered a "package". So if Paul is determined to fix all
occurences of "compatilibity" in the doc-strings, why would he only do
that for core?  People won't care

Re: [O] Sync up the org in emacs master to org maint branch?

[David Engster]

Eli Zaretskii writes:
>> From: David Engster <d...@randomsample.de>
>> Cc: Lars Ingebrigtsen <la...@gnus.org>, b...@gnu.org,
>> emacs-de...@gnu.org, phillip.l...@russet.org.uk,
>
>> which implies that they are not supposed to be present at a "normal"
>> checkout.
>
> I don't see how it implies that.  Release tarballs are prepared
> specially for a release, so their build procedures don't have to be
> identical to building from Git.
>
>> Otherwise, what difference would it make?
>
> Ask the package maintainers, they see significant advantages in being
> able to release interim versions independent of Emacs releases.

But we are discussing moving CEDET, Gnus and Org out of Emacs core, and
at least the former two do not plan to provide updates between Emacs
releases. We already had this discussion in 2015, where John was pretty
determined to remove CEDET from core:

https://lists.gnu.org/archive/html/emacs-devel/2015-11/msg00877.html

-David

Re: [O] Sync up the org in emacs master to org maint branch?

[David Engster]

Eli Zaretskii writes:
>> From: Lars Ingebrigtsen 
>> Date: Thu, 02 Feb 2017 13:10:07 +0100
>> Cc: Bastien Guerry , Emacs developers ,
>
>>  Phillip Lord ,
>>  emacs-org list ,
>>  Kaushal Modi 
>> 
>> If Django had traditionally always been distributed along with Python,
>> and maintained in the Python repo, and the suggestion now would be to
>> move Django to a part of the Python repo that very few developers look
>> at, but Django would continue to be distributed with Python, and all
>> Django bug reports would continue to go to the Python bug repository,
>> and Python developers would continue to be responsible for QA and bug
>> fixing of Django.
>
> I believe the intent is to make it so that checking out and building
> Emacs also checks out and builds all the packages that are intended to
> be part of a release tarball.  If we indeed do that this way, there
> will be no difference, QA-wise, between core packages and ELPA
> packages that are logically part of an Emacs release.

That's not how I understood it. It was always said that Emacs must not
depend on those ELPA packages that are shipped with the release, which
implies that they are not supposed to be present at a "normal"
checkout. Otherwise, what difference would it make? (Besides requiring
more cumbersome tooling to make this all work.)

-David

Re: [O] Sync up the org in emacs master to org maint branch?

[David Engster]

John Wiegley writes:
>>>>>> "DE" == David Engster <d...@randomsample.de> writes:
>
> DE> Also, I currently have no idea how to continue with CEDET, as the future
> DE> where development should happen is unclear, and I get the feeling we're
> DE> just waisting our time with the ongoing merge.
>
> Until the dust has settled, please proceed, assuming nothing has
> changed. Move your primary development into Emacs.git.
>
> The changes I'm proposing don't have to happen tomorrow, and I can still be
> convinced they're unnecessary.

So if you don't get convinced, we'll just move again, right? No big
deal.

> My gut tells me, however, that we're supporting an unnecessarily
> monolithic development model for no better reason than "we're used to
> it".
>
> In fact, what we're doing feels like if Python included Django in its main
> repository, just to solve Django's problems of compatibility, testing, and
> making its bugs known to the main Python developers.

You are insinuating that my motivation is to delegate CEDET development
to the core Emacs developers. This is simply not true, and I don't see
how my original mail could be interpreted like that.

So let me try again: What I find completely misguided is to move
packages out of core *but still putting them into the release*. In other
words, in my opinion there are really just two options that make sense:
you either keep a package in core, or you kick it out and don't ship it
with the release.

Say the Python developers would decide: Hey, many people like Django, so
let's just put their latest git master into our release and ship
it. Would you think that is a good approach?

-David

Re: [O] Sync up the org in emacs master to org maint branch?

[David Engster]

John Wiegley writes:
>>>>>> "DE" == David Engster <d...@randomsample.de> writes:
>
> DE> It is a mistake because you are creating more moving targets and bring
> DE> them together very late in the release process. This reduces the amount of
> DE> testing that is done for those packages, so bugs will be noticed later and
> DE> the quality of the relases suffer. It moves even more work into the
> DE> RC-phase, which is already crowded and where people who can fix those bugs
> DE> might not be readily available. It removes those packages from Emacs CI,
> DE> so that breakages due to changes in core are not immediately noticed, and
> DE> often times they have to be fixed not by those who created the breakage,
> DE> but by those who notice them.
>
> These are issues to be fixed in the way ELPA integrates with our development
> process. I recognize today's ELPA may have these drawbacks, but I believe they
> can be fixed.

This really has not much to do with how ELPA works. My points above are
about the underlying concept you are proposing: moving packages out of
Emacs core and hence removing them from current Emacs development, but
still bundling them with the release. It's a have-and-eat-cake concept.

> We're moving toward a future where Emacs.git will represent "core Emacs", and
> only contain what core needs (plus a few historical bits, I'm sure). There
> should be no argument for keeping a project in core just to gain auxiliary
> benefits.

Of the points I raise above, which fall under "just to gain auxiliary
benefits"? I'm honestly confused. I'm specifically talking about the
quality of the Emacs relases.

Also, I currently have no idea how to continue with CEDET, as the future
where development should happen is unclear, and I get the feeling we're
just waisting our time with the ongoing merge.

-David

Re: [O] Sync up the org in emacs master to org maint branch?

[David Engster]

Lars Ingebrigtsen writes:
> The question is: What is the most effective way for Emacs developers to
> spend their time?  I can't really see that anybody has made the case
> that shifting stuff from Emacs core to ELPA will mean less work for...
> well, anybody.
>
> (Except perhaps CEDET.  There seems to be a lot of merging problems
> there.)

This is precisely why we're currently moving all development to Emacs
and abandon the external repo. At least we were planning to...

-David

Re: [O] Sync up the org in emacs master to org maint branch?

[David Engster]

John Wiegley writes:
>>>>>> "DE" == David Engster <d...@randomsample.de> writes:
>
> DE> This is a misunderstanding. I said I wanted to move support for certain
> DE> languages and project types into ELPA, not CEDET core. I'm still of the
> DE> opinion that moving it completely to ELPA is a mistake.
>
> It would only be a mistake if other parts of core need to use it. If only
> users make use of it, then having it ELPA will be invisible to them.

It is a mistake because you are creating more moving targets and bring
them together very late in the release process. This reduces the amount
of testing that is done for those packages, so bugs will be noticed
later and the quality of the relases suffer. It moves even more work
into the RC-phase, which is already crowded and where people who can fix
those bugs might not be readily available. It removes those packages
from Emacs CI, so that breakages due to changes in core are not
immediately noticed, and often times they have to be fixed not by those
who created the breakage, but by those who notice them.

-David

Re: [O] Sync up the org in emacs master to org maint branch?

[David Engster]

John Wiegley writes:
>> "KM" == Kaushal Modi  writes:
>
> KM> If we are able the release the new packaging method in emacs 26.x, then we
> KM> can remove org from emacs master completely, but if not, then at least as
> KM> backup we have a newer org version to go out with that release.
>
> For Emacs 26, I intend the new ELPA process to be in place, whereby "default"
> packages can be developed separately, and declare a way to get slip-streamed
> into the release tarball so users are unaware of the separate nature of their
> development.
>
> The CEDET developers have agreed to support this, and it sounds like you are
> willing to as well.

This is a misunderstanding. I said I wanted to move support for certain
languages and project types into ELPA, not CEDET core. I'm still of the
opinion that moving it completely to ELPA is a mistake.

> If Lars is game, I'd like for Gnus to be third major
> package we do this for initially. That will reduce considerably the number of
> external files we track in Emacs.git.

CEDET and Gnus are not external anymore. Both have abandonded their
upstream repos and moved to Emacs, because the faster development of
Emacs has made that necessary.

-David

[O] Failing test with current emacs-git on the buildbot

[David Engster]

FYI:

I've updated the emacs-git master on the Buildbot and now
'test-ob/process-params-no-duplicates' is failing:

http://randomsample.de/org-buildbot

I've also added a new builder 'emacs-25.1' which is the current emacs-25
branch.

-David

Re: [O] [DEV] Bump Emacs requirement to 24.4?

[David Engster]

Bastien writes:
 Emacs 23 and XEmacs support will be officially dropped as of Org 9.0.

Support for XEmacs should be dropped right away; it would just state a
fact, as Org didn't even compile with XEmacs for quite some time now
(and nobody complained).

 I recognize having the manpower to watch after those branches might be
 an issue, but we can overcome it by calling for careful testing before
 major releases.

At least the Buildbot can test the 8.3 branch with older Emacsen. I
already changed that master is only checked with 24.3+.

-David

Re: [O] Test failure with current head

[David Engster]

Nick Dokos writes:
 Rasmus ras...@gmx.us writes:

 Nick Dokos ndo...@gmail.com writes:

 I just updated to release_8.3beta-1286-g20795fd.

 ``make test'' shows a failure on test 226:
FAILED  226/592  test-org-clock/clocktable-until-now


 The backtrace was edited to get rid of a NUL that gnus complained

 I know.  I don't really understand what untilnow does and the org
 clocktable tests are really obscure to me.  Fell free to do a bisect
 master or/and submit a fix.


 Seems like a heisenbug - first attempt to bisect fingered
 20795fd1c4e48096226f3072de8d93bd99d761bf but only because it is the HEAD
 and I had declared it as bad: it turns out that `make test' does not
 get any failures now. And nothing changed - weird.

Yes, it's not always failing, but it's been happening for a few days now
on the buildbot:

  http://randomsample.de/org-buildbot

First time it happened was after this commit

  
http://orgmode.org/cgit.cgi/org-mode.git/commit/?id=22c652599c8bfedcd27e78d7ad9544826eae7dd0

but it looks harmless. Could very well be something else that triggers
this.

-David

Re: [O] Javascript syntax highlighting?

[David Engster]

Peter Davis writes:
 On 5/25/15 10:44 AM, David Engster wrote:
 Peter Davis writes:
 #+BEGIN_SRC Javascript
 Don't capitalize, use 'javascript'.
 Thanks, David.

 That doesn't appear to make any difference. Neither does js.

Do you mean you don't see any highlighting in Emacs, or only in the
export?

What does 'C-h v major-mode RET' say when you are editing a Javascript
file?

-David

Re: [O] Javascript syntax highlighting?

[David Engster]

Peter Davis writes:
 #+BEGIN_SRC Javascript

Don't capitalize, use 'javascript'.

-David

Re: [O] org-caldav problem; used to work

[David Engster]

Detlef Steuer writes:
 org-check-agenda-file: Wrong type argument: stringp, nil

Do M-x toggle-debug-on-error before the sync and you should get a
backtrace for the error. Just post that here and we might be able to see
what's going on.

-David

Re: [O] org-caldav problem; used to work

[David Engster]

Detlef Steuer writes:
 Am Sun, 16 Nov 2014 22:53:11 +0100
 schrieb David Engster d...@randomsample.de:

 Detlef Steuer writes:
  org-check-agenda-file: Wrong type argument: stringp, nil
 
 Do M-x toggle-debug-on-error before the sync and you should get a
 backtrace for the error. Just post that here and we might be able to
 see what's going on.

 Here it is:

[...]

The signature of org-icalendar--combine-files changed in
67ae102b4b. I've pushed a workaround which should fix this.

-David

Re: [O] Pushing and pulling to google calendar

[David Engster]

Marvin M. Doyley writes:
 Is there a way to sync org-agenda with google calendar so that I can
 exploit googles calendar reminder.

org-caldav[1] works with Google Calendar on the old, deprecated CalDAV
endpoint (https://www.google.com/calendar/dav). I hear that the endpoint
is still working for some people, so maybe you are in luck.

Moving to the new endpoint would require OAuth authentication, which
shouldn't be hard to add using the oauth-library in ELPA, but since I
don't use Google, I'm lacking motivation...

-David

https://github.com/dengste/org-caldav

Re: [O] problem with org-caldav and ox-icalendar: UID property wrapping

[David Engster]

David Engster writes:
 Nicolas Goaziou writes:
 Eric S Fraga e.fr...@ucl.ac.uk writes:
 I have tracked this down to org-icalendar outputing wrapped lines even
 for UID entries:

 ,
 | BEGIN:VEVENT
 | DTSTAMP:20140507T114443Z
 | UID:0400[...]00
 |  00[...]6
 | DTSTART;TZID=Europe/London:20120403T06
 `

 AFAIU RFC 5545, all lines longer than 75 octets, including UID lines,
 are expected to be folded. Therefore I think ox-icalendar is right.

 Yes, it is.

 I've already rewritten that part in org-caldav, but it needs more
 testing. I'll push a fix in the coming days.

Well, that took a bit longer, but I pushed it now...

-David

Re: [O] problem with org-caldav and ox-icalendar: UID property wrapping

[David Engster]

Nicolas Goaziou writes:
 Hello,

 Eric S Fraga e.fr...@ucl.ac.uk writes:

 I have tracked this down to org-icalendar outputing wrapped lines even
 for UID entries:

 ,
 | BEGIN:VEVENT
 | DTSTAMP:20140507T114443Z
 | UID:0400[...]00
 |  00[...]6
 | DTSTART;TZID=Europe/London:20120403T06
 `

 AFAIU RFC 5545, all lines longer than 75 octets, including UID lines,
 are expected to be folded. Therefore I think ox-icalendar is right.

Yes, it is.

I've already rewritten that part in org-caldav, but it needs more
testing. I'll push a fix in the coming days.

-David

Re: [O] Timestamps in properties not exported by icalendar exporter

[David Engster]

Nicolas Goaziou writes:
 David Engster d...@randomsample.de writes:
 These entries show up in the agenda just fine, but the icalendar
 exporter does not export it because the timestamp is in the properties
 (the gnus-icalendar package puts it there so that you can easily change
 it if the organizer decides to change the event and sends a change
 request).

 The exporter could of course simply take the first timestamp it finds in
 the properties, but it could be the wrong one; for instance, there could
 be a LOGBOOK timestamp before the one in DT. Hence I think it would be
 nice if you could tell the exporter which property to use as timestamp
 during export.

 Can't you just walk the buffer and turn such properties into plain
 time-stamps (or scheduled, deadline...), in a hook?

Yes, I could do that for my specific setup. But it would be nice if this
stuff could just work, so that things like Outlook calendar invites
can be directly exported to .ics.

I mean, those entries show up in the agenda, so I found it rather
surprising that they are completely ignored by the exporter. I think it
would make sense if the exporter also looked for time-stamps in the
properties. Maybe it could just take the first one it finds (which I
guess is what the agenda does?), and also give the user the ability to
configure which properties to look at (or maybe an exclude option).

-David

Re: [O] Timestamps in properties not exported by icalendar exporter

[David Engster]

Nicolas Goaziou writes:
 David Engster d...@randomsample.de writes:
 I mean, those entries show up in the agenda, so I found it rather
 surprising that they are completely ignored by the exporter.

 This is an agenda bug, which probably use a regexp to find timestamps.
 But timestamps in properties are not valid Org timestamps, per Org
 syntax.

OK.

 I think it would make sense if the exporter also looked for
 time-stamps in the properties.

 There are already plenty of locations to use timestamps. We have
 scheduled, deadline, plain timestamps... I don't think we need more of
 them.

It's these multitude of timestamp locations which makes changing the
timestamp of an existing entry through Elisp so tedious (I had to deal
with that in org-caldav), so using a property for it seemed like a good
idea. Anyway, I understand your reasoning, and I guess we should change
how gnus-icalendar generates its Org entries.

-David

Re: [O] Timestamps in properties not exported by icalendar exporter

[David Engster]

Nicolas Goaziou writes:
 David Engster d...@randomsample.de writes:

 It's these multitude of timestamp locations which makes changing the
 timestamp of an existing entry through Elisp so tedious (I had to deal
 with that in org-caldav)

 The number of locations may be daunting but all of them make sense
 actually. Also, I don't think it is really difficult to change
 a timestamp through Elisp, due to specialized functions (e.g.
 `org-schedule').

Not difficult, but tedious. The problem is that an entry might have
several timestamps, and you have to find the correct one to change.

 Anyway, if you think this area can be improved, feel free to make
 suggestions.

I'm actually not sure what org-element is capable of nowadays. What I'd
like to have is a way to parse entries into a structure which lets me
access certain elements of the entry, like headline, timestamps,
properties, body text, etc., and also a way to convert this structure
*back* to a plain Org entry. This way, I could change certain elements
of the entry in a robust way. It might be that the *appearance* of the
entry changes a bit during this procedure, but its elements should be
preserved.

(BTW: This is similar to what we have in CEDET, where the Semantic
parser generates 'tags' which describes things like variable or function
declarations, and you can define SRecode templates which can generate
textual output from these tags in pretty much any way you like.)

For example, in org-caldav I had to write a function which changes the
headline of an entry, but preserves any timestamps which might be in
it. Again: not difficult to do, especially since there is
org-complex-heading-regexp, but it's still tedious since Org entries are
so flexible in their appearance.

 so using a property for it seemed like a good idea.

 This has been discussed a few times on this ML already.

 Carsten's argument is that new users shouldn't have to deal with hidden
 data, like property drawers, for such basic features.

Oh no, the user shouldn't have to do this, I agree
completely. Properties are nice for *code*, since they can be easily
parsed.

-David

Re: [O] Timestamps in properties not exported by icalendar exporter

[David Engster]

Nicolas Goaziou writes:
 David Engster d...@randomsample.de writes:

 I'm actually not sure what org-element is capable of nowadays. What I'd
 like to have is a way to parse entries into a structure

 See `org-element-parse-buffer'.

 which lets me access certain elements of the entry, like headline,
 timestamps, properties, body text, etc.,

 See `org-element-map'.

 and also a way to convert this structure *back* to a plain Org entry.

 See `org-element-interpret-data'.

I had a hunch you already implemented all this. :-)

I'll look into it.

Thanks,
-David

[O] Timestamps in properties not exported by icalendar exporter

[David Engster]

I have the problem that a certain kind Org entries is not exported by
the icalendar exporter, namely those created by the gnus-icalendar
package.

This package creates Org entries from calendar invites in the following
way (I've omitted some of the properties, but you get the idea);

** Some appointment (location)
   :PROPERTIES:
   :ICAL_EVENT: t
   :ID:   04008200E00074C5B7101A82E00800
   :DT:   2013-12-09 Mon 13:00-13:30
   :END:

   Body text


These entries show up in the agenda just fine, but the icalendar
exporter does not export it because the timestamp is in the properties
(the gnus-icalendar package puts it there so that you can easily change
it if the organizer decides to change the event and sends a change
request).

The exporter could of course simply take the first timestamp it finds in
the properties, but it could be the wrong one; for instance, there could
be a LOGBOOK timestamp before the one in DT. Hence I think it would be
nice if you could tell the exporter which property to use as timestamp
during export.

-David

[O] bug#15896: 24.3.50; Org-8.0: M-x customize-changed 24.3 doesn't show new export options

[David Engster]

Jambunathan K. writes:
 It was rude of you to close the bug of which you nothing about.

It was rude of you to close all your existing bugs. Did it cross your
mind that maybe people already started working on those? This is not
a playground for your ego trips.

-David

[O] bug#15896: 24.3.50; Org-8.0: M-x customize-changed 24.3 doesn't show new export options

[David Engster]

Jambunathan K. writes:
 David Engster d...@randomsample.de writes:

 This is not a playground for your ego trips.

 Or Eli Zaretskii's.  That's precisely my point.

Nobody cares for your points if you're pulling stunts like closing all
bugs you've opened. I for one am done talking to you.

-David

Re: [O] org-caldav for Google or Zimbra?

[David Engster]

Jason Riedy writes:
 Google has deprecated the URL in org-caldav, and that old URL
 does not work for me.  The new one is documented at
   
 https://developers.google.com/google-apps/calendar/caldav/v2/guide#new_endpoint

 Fiddling with the regexp in org-caldav-events-url triggers an
 error apparently in the authentication code:

Google has changed authentication on the new endpoint to OAuth. Julien
has written an OAuth2 client implementation, which is in GNU ELPA, but I
didn't have time to look at it yet. Not sure when I get to it; I hope
someone beats me to it.

-David

Re: [O] advice needed: how do you guys sync org files between devices?

[David Engster]

Suvayu Ali writes:
 I have noticed that git *always* conflicts with TODO state changes.  It
 seems changes to a headline is not easy to resolve.

You mean you change state in a file and git fails to merge this on the
other side, although you didn't change the same line there? I've never
seen such a spurious merge conflict with my Org files. Could you give an
example? It might be that simply switching to another diff.algorithm and
maybe also increasing diff.context would get rid of this.

-David

Re: [O] advice needed: how do you guys sync org files between devices?

[David Engster]

Suvayu Ali writes:
 That said, I think you might be right.  Possibly it could be resolved by
 just changing the algorithm, I don't know why I never tried that though.

You might want to try setting diff.algorithm to 'patience' and maybe
also raise diff.context to something like 6 or so if that doesn't solve
your problem. It very much depends on how your Org files look.

-David

Re: [O] [PATCH 1/2] org-notify: Don't use obsolete flet and macrolet

[David Engster]

Achim Gratz writes:
 Peter Münster writes:
 cl-lib was just introduced in Emacs 24.3.

 Does that mean, that the trunk should be modified later?

 It means you should implement a solution that doesn't make Org
 non-functional for Emacs 23 users.

The easiest solution is to simply do nothing, until Org drops support
for Emacs 24.2 and earlier. The 'cl' library will be shipped with Emacs
for a long time.

BTW, which Emacs versions does Org support? Is this documented anywhere?

-David

[O] org-caldav will continue to work with Google Calendar (was: org-caldav will cease to work with Google Calendar)

[David Engster]

David Engster writes:
 Google has announced today that they will shut down their CalDAV API in
 September, since hey, everybody's using their own protocol anyway.

Well, Google has suddenly realized that not only is CalDAV an open
standard, but it's actually used outside the Googleverse. Who could've
known? Anyway, they will continue to support it [1], so org-caldav
should work with Google Calendar for the time being (read: until they
change their mind again).

-David

PS: I know that there are problems with org-caldav and the new
exporter. I'll switch to Org 8 soon and will then merge the pending pull
requests.


[1]
http://googledevelopers.blogspot.ca/2013/06/making-googles-caldav-and-carddav-apis.html

[O] Org as a static site generator

[David Engster]

I'd like to use Org as a static site generator. I know quite a few
people use Org to manage their sites, so I'd like to know what's already
available and what I'd need to add to make this working properly.

I know of course how to export a bunch of Org files to HTML through the
publishing features. However, that's not really cutting it, I'm afraid.

Thing of a typical HTML5 template having a header, nav, footer,
and article. I'd like Org to include the different exported files into
the article section, and the rest to remain the same. The nav would
contain a global navigation menu, also highlighting the current active
section (though CSS, no JS please).

Has anybode done something like this?

-David

Re: [O] Org as a static site generator

[David Engster]

Vincent Beffara writes:
 I am using o-blog for that, it is pretty great. 

Thanks, that looks pretty nice. I'll take a look.

 Thing of a typical HTML5 template having a header, nav, footer,
 and article. I'd like Org to include the different exported files into
 the article section, and the rest to remain the same. The nav would
 contain a global navigation menu, also highlighting the current active
 section (though CSS, no JS please).

 I never quite managed to do that ... but it should definitely be doable.

It's really simple to do. You just have to include some 'id' in the
body which indicates to which section it belongs to.

I've also just found this, which uses Org only as a markup tool and
Jekyll to generate the site:

http://orgmode.org/worg/org-tutorials/org-jekyll.html

But doing everything in Org is tempting, of course. And then just serve
that stuff with ElNode. :-)

-David

Re: [O] two-way sync org agenda/ical

[David Engster]

Eric S. Fraga writes:
 Bastien b...@altern.org writes:

 Eric S Fraga e.fr...@ucl.ac.uk writes:

 Given that I no longer use my old syncing approach, described on that
 page, and instead use org-caldav, I would be happy to have the reference
 to org-caldav moved to the top!

 It's a wiki, be *bold* :)

 Actually, given the recent announcement regarding the removal of caldav
 support from Google's calendar service, I think it's best to leave the
 page as it is until things settle down.

FWIW, I agree.

But I think it would be good to have a page listing free alternatives
for Google Calendar (free as in speech, not necessarily free of charge
when it comes to hosting), and how to use those with Org and Android. On
such a page, org-caldav would fit very well. I currently do not have
time to write something up, though.

-David

Re: [O] org-caldav will cease to work with Google Calendar

[David Engster]

'rasmus' writes:
 For now I'm using fluux, but I haven't managed to get org-CalDAV to
 sync yet.

What is that? I'm afraid I cannot find it.

  I'm also looking to sync bbdb via CardDAV, although I would be
 willing to switch to org-contacts if a solution emerged here before.

CardDAV shouldn't be very hard to add. There are two reasons however why
at least I won't work on this in the near future: I don't need it, and
it's boring.

 Could Orgmode.org provide (and earn revenues from) a /standard-based/
 (CardDAV/CalDAV) synchronization framework without bells and whistles
 and a reasonable price per year?  Say €5-10 per year?  (I have no idea
 how much upload/download goes to such a system).

Frankly, I'd leave such services to professionals. Calendaring data is
very critical to most people. There are many CalDAV hosting services out
there (but I'm afraid you won't get a good one for 5-10 Euros/year).

 Baïkal and Radicale are nice, small CalDAV servers.  The latter also
 provides CardDAV.

I just discovered Radicale. I think it is very nice for people who don't
want/need a full-blown Workgroup or Cloud solution like SOGo/Owncloud.
org-caldav does not currently work with it, but this should be fixable.

-David

Re: [O] org-caldav will cease to work with Google Calendar

[David Engster]

Vincent Beffara writes:
 But it _can_ list my calendars, meaning that it can authenticate and
 have some sort of interaction with google's servers. Maybe interfacing
 with that would not be too hard?

Supporting the Google Calendar API in org-caldav wouldn't be hard. It's
actually a very clean, RESTful service; much better than CalDAV, in
fact. Just what you would expect from Google.

However, this is not a technical issue. This is also why I said that
anyone who wants to implement support for the Google Calendar API in
org-caldav should fork it; I won't accept pull requests which implement
that.

If Google decides to discontinue a well established, IETF-standardized
API in favor of a proprietary one for which there exist no free server
implementations, I will not support that. I think the best solution for
anyone using Google Calendar is to migrate away from that service.

-David

[O] org-caldav will cease to work with Google Calendar

[David Engster]

Google has announced today that they will shut down their CalDAV API in
September, since hey, everybody's using their own protocol anyway.
org-caldav will then cease to work with Google calendar. I won't work on
supporting the Google calendaring API until there's a free server
implementation for it, which can be self-hosted. If someone else would
like to work on that, please create a fork under a different name.

-David

Re: [O] org-caldav will cease to work with Google Calendar

[David Engster]

Bastien writes:
 David Engster d...@randomsample.de writes:
 I won't work on supporting the Google calendaring API until there's a
 free server implementation for it, which can be self-hosted. If
 someone else would like to work on that, please create a fork under a
 different name.

 Are there already people interested in working on a free server
 implementation of the Google Calendaring API?  Maybe a pointer for
 those potentially interested on the list might be useful -- just
 in case.

Just to be clear, with on that I meant support for the Google API in
org-caldav. I think we should not play Google's game and start
supporting their proprietary API because they shut down their CalDAV
interface (which was crappy, but usable).

I don't know of any free calendar servers which are planning to
implement Google's API. Maybe it's not even allowed because of patents
or whatever.

-David

Re: [O] [RFC] Org syntax (draft)

[David Engster]

Jambunathan K. writes:
 Still you haven't answered my Fudging the mail reply headers question
 to my satisfaction.

http://www.gnu.org/software/emacs/manual/html_node/message/Mailing-Lists.html

A mailing list poster can use MFT to express that responses should be
sent to just the list, and not the poster as well. This will happen if
the poster is already subscribed to the list.

-David

Re: [O] [Out-of-Thread] Re: [RFC] Org syntax (draft)

[David Engster]

Jambunathan K. writes:
 I know that.  

 But that doesn't answer the question why Carsten will appear in the To
 header of a mail that I reply to a mail I receive from Eric S Fraga.

Because Carsten started the thread and did not set MFT.

-David

Re: [O] [Out-of-Thread] Re: [RFC] Org syntax (draft)

[David Engster]

Jambunathan K. writes:
 David Engster d...@randomsample.de writes:

 Jambunathan K. writes:
 I know that.  

 But that doesn't answer the question why Carsten will appear in the To
 header of a mail that I reply to a mail I receive from Eric S Fraga.

 Because Carsten started the thread and did not set MFT.

 In this very specific thread - the one I am typing right now, Nicolas
 Goaziou doesn't appear in To - why?  Normally, when I wide reply to a
 Ngz's post it gets sent to him as well as the mailing list.

 If your argument is right, Ngz should appear in To or CC headers of the
 post.  I see only ML address.  So what am I missing?

You didn't do a wide reply in 871ubntg32@gmail.com.

-David

Re: [O] org-caldav can't find org-prepare-agenda-buffers

[David Engster]

Vincent Beffara writes:
 In the meantime I also tweaked the way org-caldav manages UIDs to
 exploit the fact that multiply occurring events are now exported
 multiple times in ox-icalendar, everything works out well if only
 org-ical is used. One thing remaining, on ical-org sync, there is an
 issue that makes the next sync erase the event on ical and upload it
 again, under another id (and with the info not generated by org-caldav
 missing, of course). I think I know how to fix it, just need to find
 some time ...

 I will send what works to the list. (In the meantime, David, don't
 pull my second pull request, it is not right.)

I appreciate your work, of course, but let me add a word of warning. I
think if you give up the one-to-one correspondence that one Org event
has exactly *one* event in the remote calendar, you are entering a world
of pain.

Remember that there are three things: new items, changed items and
deleted items. Since these can happen on both sides, you have to figure
out six different cases which must be handled properly. As you've
already experienced, the changes on the calendar side are more
difficult. What happens if you delete one of the events that stems from
one Org entry?  You would first have to figure out which timestamp
created it, but how? Timestamps don't have UIDs, only the full entry has
one. So you would have to somehow add this information to the event
database which gets stored to disk. You might get this to work, but I
have a gut feeling that you'll loose robustness. For example, the Google
calendar CalDAV interface is pretty slow and not very reliable, so a
robust resume functionality is essential.

Also, I deliberately shoved conflict handling at the side for the
moment, but this is a feature which will have to be added one day, and
will complicate things much more.

-David

Re: [O] org-caldav can't find org-prepare-agenda-buffers

[David Engster]

Julien Cubizolles writes:
 Bastien b...@altern.org writes:

 Achim Gratz strom...@nexgo.de writes:

 I hope to put this together in the next week or
 so, then it will be possible to nuke all traces fr4om an old Org and
 then start from a clean slate with a new Org installation.

 Okay, thanks.  Let us know how it goes,

 What temporary (as in every user fixing it locally) solution would you
 recommend to get org-caldav running in the mean time ?

I just pushed a change which should fix this (if the new exporter is
compatible; I don't use Org from git).

-David

Re: [O] org-caldav can't find org-prepare-agenda-buffers

[David Engster]

Nicolas Goaziou writes:
 Hello,

 David Engster d...@randomsample.de writes:

 Julien Cubizolles writes:
 Bastien b...@altern.org writes:

 Achim Gratz strom...@nexgo.de writes:

 I hope to put this together in the next week or
 so, then it will be possible to nuke all traces fr4om an old Org and
 then start from a clean slate with a new Org installation.

 Okay, thanks.  Let us know how it goes,

 What temporary (as in every user fixing it locally) solution would you
 recommend to get org-caldav running in the mean time ?

 I just pushed a change which should fix this (if the new exporter is
 compatible; I don't use Org from git).

 Interactive functions are usually not compatible between old export
 back-ends and new ones.

 What do you need from org-icalendar.el?

Only org-export-icalendar. As long as that one is producing the same
output, everything should be fine.

-David

Re: [O] org-caldav can't find org-prepare-agenda-buffers

[David Engster]

Julien Cubizolles writes:
 David Engster d...@randomsample.de writes:
 I just pushed a change which should fix this (if the new exporter is
 compatible; 

 It doesn't seem to be, I get 

 apply: Symbol's function definition is void: org-export-icalendar

 when running org-caldav-sync, which is weird since org-export-icalendar
 is present in both org-icalendar and ox-icalendar.

No, it's not defined in ox-icalendar. I guess it must be generated by
the new exporter first, probably through some additional require.

 I don't use Org from git).

 I'm thinking about reverting to an older Org since org-caldav is more
 important to me than the new exporter. Does it work with the latest
 stable release (7.9.3f) ?

I'm still on 7.9.2, but I wouldn't know why it shouldn't work with that.

-David

Re: [O] org-caldav can't find org-prepare-agenda-buffers

[David Engster]

Nicolas Goaziou writes:
 David Engster d...@randomsample.de writes:

 What do you need from org-icalendar.el?

 Only org-export-icalendar. As long as that one is producing the same
 output, everything should be fine.

 Unfortunately, there is no `org-export-icalendar' anymore.

 There is:

   (org-icalendar--combine-files nil FILES)

 which is equivalent to:

   (org-export-icalendar t FILES)

Thanks, that's what I need.

After skimming through the new exporter, it also seems that
org-combined-agenda-icalendar-file was replaced with
org-icalendar-combined-agenda-file. Is that correct?

I'm also using org-icalendar-timezone, org-icalendar-store-UID, and
org-icalendar-date-time-format, but it seems those are still there.

-David

Re: [O] org-caldav can't find org-prepare-agenda-buffers

[David Engster]

Julien Cubizolles writes:
 David Engster d...@randomsample.de writes:
 No, it's not defined in ox-icalendar. I guess it must be generated by
 the new exporter first, probably through some additional require.

 I have the following definition in ox-icalendar.el. 

 (defgroup org-export-icalendar nil
   Options specific for iCalendar export back-end.
   :tag Org Export iCalendar
   :group 'org-export)

That's just defining a group for the customization system. It's not a
function.

-David

Re: [O] org-caldav can't find org-prepare-agenda-buffers

[David Engster]

Nicolas Goaziou writes:
 David Engster d...@randomsample.de writes:

 After skimming through the new exporter, it also seems that
 org-combined-agenda-icalendar-file was replaced with
 org-icalendar-combined-agenda-file. Is that correct?

 I'm also using org-icalendar-timezone, org-icalendar-store-UID, and
 org-icalendar-date-time-format, but it seems those are still there.

 Correct.

Thanks. I've now pushed a further fix which should hopefully make things
work with the new exporter. Julien, please let me know if it works.

-David

Re: [O] org-caldav can't find org-prepare-agenda-buffers

[David Engster]

Bastien writes:
 Hi David,

 David Engster d...@randomsample.de writes:

 The most serious issue is that things will often seem to work because
 old exporters are pulled in from Emacs, possibly *very* old
 exporters.

 I've added (provide 'org-icalendar) to ox-icalendar.el so that
 a user will load the correct file instead of the old file when
 ox-icalendar.el takes precedence over org-icalendar.el in the
 load-path.  Not the most elegant solution, I agree.

Did you actually try that? How should Emacs possibly know that the file
ox-icalendar provides the feature org-icalendar? This will only work if
ox-icalendar is already loaded.

-David

Re: [O] org-caldav feedback

[David Engster]

Torsten Wagner writes:
 Now I make a tiny change e.g. change the length of the appointment from within
 SOGo and sync back I get

 TODO TODO Neuer Termin mit Foo und Bar r2013-03-06 Wed 10:00-11:00
     :PROPERTIES:
     :ID:       8a9651c0-faee-4416-afa6-979e328a3d15
     :END:

 As you can see the TODO doubled and the last character of the title is
 repeated.

 I guess its simply some regexp, which needs some finetuning.

Thank you for this bug report. I'll look into it.

 CC. Did you had a chance to look into calfw and think about how to make use of
 it for org-caldav?

Not sure what you have in mind here.

-David

Re: [O] org-caldav can't find org-prepare-agenda-buffers

[David Engster]

Bastien writes:
 Hi David,

 David Engster d...@randomsample.de writes:

 (eval-after-load org-icalendar
   '(error The old org-icalendar exporter is deprecated; use
 ox-icalendar instead.))

 I'm not sure about this one: where are you suggesting to add this?
 In org.el?

Your call. Anywhere where it's guaranteed to be loaded upon Org startup.

  Will the users get the warning if org-icalendar.el has been loaded
 from a previous install?

The user will get the error as soon as org-icalendar is loaded (from
wherever). org-icalendar *will* be loaded then, and the user can ignore
the error and continue to use it if he likes, but at least he got a
clear error that this is not a supported.

-David

Re: [O] org-caldav can't find org-prepare-agenda-buffers

[David Engster]

Bastien writes:
 David Engster d...@randomsample.de writes:

 Of course I can fix this. But I hope you realize that any third-party
 code out there that requires an exporter will load the old one from
 Emacs proper. 

 Yes, I'm well aware of this.  The change now lives in the master
 branch, and will happen when we release 8.0, hopefully in a not so
 distant future.

 We will document all incompatible changes in the release notes, as we
 usually do.  I expect third-party maintainers to read these notes.

Well, I don't expect it.

 I'm wondering why you felt the need to rename them. If the
 new exporters are compatible with the old ones, why not keep the names?
 This would also avoid that the provided feature differs from the used
 name prefix (ox-icalendar != org-icalendar).

 There are several good reasons for this:

 1) conflicting library names: we now have org-man.el (for links to man
pages) and ox-man.el (for exporting);

 2) using the dedicated prefix ox- makes it clear that the library is
an export backend, the same way that the ob- prefix makes it clear
it is to support a language for Org Babel.

 In general, the change is incompatible for third-part libraries by is 
 clearly useful for future maintainance, so the trade-off was in favor
 of making it, and 8.0 is a good time for it.

I'm afraid I remain unconvinced. 1) is just one name clash, so just
rename one of it. Also, like all the other ox-* packages now, ox-man
uses 'org-man' as a prefix for its names; what will 'org-man' use, then?

2) is nice, but IMO not a good enough reason to break compatibility in
such a major way.

Anyway, you've made a decision and did the rename. Let's just agree to
disagree here.

The most serious issue is that things will often seem to work because
old exporters are pulled in from Emacs, possibly *very* old
exporters. They might work, or they might fail for various reasons,
making it difficult for users and developers to see what went wrong.
Just look at the other bug report by Karl Voit from yesterday; I guess
it's the same issue.

So if you change your API in an incompatible way (and packages names are
part of that), you should at least throw a clear error when the old API
is used, and tell the user what happened and how it can be fixed. Hence
I would suggest to use something like

(eval-after-load org-icalendar
  '(error The old org-icalendar exporter is deprecated; use ox-icalendar 
instead.))

An alternative would be to remove the bundled Org from load-path when a
newer version is loaded. We do that with CEDET, but it is difficult to
do right (because of autoloading, for instance), so I think the
eval-after-load hack is better.

-David

Re: [O] org-caldav can't find org-prepare-agenda-buffers

[David Engster]

Achim Gratz writes:
 David Engster writes:
 An alternative would be to remove the bundled Org from load-path when a
 newer version is loaded. We do that with CEDET, but it is difficult to
 do right (because of autoloading, for instance), so I think the
 eval-after-load hack is better.

 That part is actually relatively easy, I have posted code to do that
 here.  The part that gives me headaches is the defcustom definitions,
 which are nowhere near as easy to defeat and provide yet another way of
 autoloading stuff in Emacs.

You mean this cus-load thingie? CEDET is actually excluded from that, so
we don't have to deal with it. But wouldn't it be enough to remove all
properties beginning with 'org-' from custom-loads?

-David

Re: [O] org-caldav can't find org-prepare-agenda-buffers

[David Engster]

Bastien writes:
 Hi Julien,

 Julien Cubizolles j.cubizol...@free.fr writes:

 As of today, org-cadav-syn fails with

 org-export-icalendar: Symbol's function definition is void:
 org-prepare-agenda-buffers

 Actually, I can't find this function but org-agenda-prepare-buffers
 exist. Are there two different functions or is there a confusion
 somewhere ?

 The function has been renamed a while ago.

 `org-agenda-prepare-buffers' is the correct name.

org-caldav does not call this function. It however requires
org-icalendar, and that was renamed to ox-icalendar in org git. So I
guess it pulls org-icalendar from the Org that is included with Emacs,
which calls the obsoleted function.

I don't follow org-development very closely. I realize there's a new
exporter, but renaming the exporters in this way is asking for
trouble. In this case, we've actually been lucky that an error like the
above is thrown; much more subtle things can happen when new and old Org
functions interact.

-David

Re: [O] org-caldav can't find org-prepare-agenda-buffers

[David Engster]

Bastien writes:
 Hi David,

 David Engster d...@randomsample.de writes:

 org-caldav does not call this function. It however requires
 org-icalendar, and that was renamed to ox-icalendar in org git. So I
 guess it pulls org-icalendar from the Org that is included with Emacs,
 which calls the obsoleted function.

 Can you fix this by requiring ox-icalendar.el when it's available,
 org-icalendar.el otherwise?

Of course I can fix this. But I hope you realize that any third-party
code out there that requires an exporter will load the old one from
Emacs proper. I'm wondering why you felt the need to rename them. If the
new exporters are compatible with the old ones, why not keep the names?
This would also avoid that the provided feature differs from the used
name prefix (ox-icalendar != org-icalendar).

-David

Re: [O] bug-tracker

[David Engster]

Michael Albinus writes:
 Andreas Röhler andreas.roeh...@easy-emacs.de writes:
 What about using some of the tools around?

 Mirroring the stuff at github would provide a tracker just by the way...

 Or you use debbugs.gnu.org, which hosts the Emacs bugtracker. Some of
 the org bugs will arrive there anyway, from people using M-x
 report-emacs-bug.

Is it OK to report bugs for upstream/development versions through
`report-emacs-bugs', though? I always thought this should only be used
for stuff that's actually bundled with Emacs (what about bugs in
org/contrib, for instance?).

-David

Re: [O] bug-tracker

[David Engster]

Michael Albinus writes:
 David Engster d...@randomsample.de writes:

 Is it OK to report bugs for upstream/development versions through
 `report-emacs-bugs', though? I always thought this should only be used
 for stuff that's actually bundled with Emacs (what about bugs in
 org/contrib, for instance?).

 debbugs.gnu.org is not only for Emacs. See
 http://debbugs.gnu.org/db/ix/packages.html which other projects are
 hosted there. gnus is a prominent example for a subpackage of Emacs
 with an own bug package using debbugs.gnu.org.

 An alternative approach is tagging bugs of a project with your package
 name. See existing user tags of project emacs at
 http://debbugs.gnu.org/cgi/pkgindex.cgi?indexon=usertag;user=emacs. cedet
 uses this mechanism already.

I know. I'm the guy who started using those tags for CEDET. :-)

I was always under the impression that those tags are merely a way for,
well, tagging bugs, so that you can search for them.  Same for
packages. I didn't know that this also implied that you're encouraged to
report bugs against upstream packages, possibly for features which are
not yet in Emacs (or may even never land there, like some stuff in
org/contrib, cedet/contrib, etc.).

-David

Re: [O] org-caldav problem: void-function url-http-options

[David Engster]

Julien Cubizolles writes:
 Since a few days (maybe an emacs update) I get this error message
 whenever I run org-caldav-sync. 

 Debugger entered--Lisp error: (void-function url-http-options)

 I've been digging around a bit and url-http-options is defined in
 url-http.el which is present on my system
 (/usr/share/emacs/24.3.50/lisp/url/url-http.el.gz) so I don't understand
 why it isn't found (assuming that void-function message actually means
 it can't find the definition of the function).

Does doing a (require 'url-http) before calling org-caldav help?

-David

Re: [O] org-caldav problem: void-function url-http-options

[David Engster]

Julien Cubizolles writes:
 David Engster d...@randomsample.de writes:

 Julien Cubizolles writes:
 Since a few days (maybe an emacs update) I get this error message
 whenever I run org-caldav-sync. 

 Debugger entered--Lisp error: (void-function url-http-options)

 I've been digging around a bit and url-http-options is defined in
 url-http.el which is present on my system
 (/usr/share/emacs/24.3.50/lisp/url/url-http.el.gz) so I don't understand
 why it isn't found (assuming that void-function message actually means
 it can't find the definition of the function).

 Does doing a (require 'url-http) before calling org-caldav help?

 Yes it does, thanks, I should have thought of it. Why has it become
 necessary ? 

This change here is responsible:

revno: 110337
committer: Stefan Monnier monn...@iro.umontreal.ca
branch nick: trunk
timestamp: Mon 2012-10-01 23:48:01 -0400
message:
  * lisp/url/url-http.el (url-http-user-agent-string): Leak less info.
  (url-http, url-http-file-exists-p, url-http-file-readable-p)
  (url-http-file-attributes, url-http-options, url-https-default-port)
  (url-https-asynchronous-p): Don't autoload.

 It seems that url-http-options is called by
 url-dav-supported-p, defined in url-dav.el. Shouldn't url-dav.el require
 url-http.el ?

Yes, it should. Could you file a bug report about this?

-David

Re: [O] Problems with org-caldav (wrong-type-argument stringp 47)

[David Engster]

Eric S. Fraga writes:
 David Engster d...@randomsample.de writes:
 OK, I took a shot at dealing with sexp entries. It's a complicated
 issue, since s-expressions can be in Org entries or by themselves. I
 updated the Readme with a section on how they are handled now. Please
 let me know how it works out for you. If you don't want/need sexp-based
 entries in your calendar, just set org-icalendar-include-sexps to nil.

 this fails.  See below for debug trace.

Thanks for the trace. This should be fixed now.

 As an aside, this new version of org-caldav changes point in my diary
 file (leaving it at the last entry in the file).  Maybe a save-excursion
 somewhere is needed?

Org-caldav does not use the diary file directly. This would rather be a
bug in the icalendar exporter, or maybe in the icalendar package. I'm
not sure how and where exactly these sexp entries are handled.

-David

Re: [O] Problems with org-caldav (wrong-type-argument stringp 47)

[David Engster]

Eric S. Fraga writes:
 David Engster d...@randomsample.de writes:

 Sven Bretfeld writes:
 Hi David and all


 I've got it. By try and error I found out that entries like these in an
 org-file cause the problem:

 %%(diary-anniversary  6 8 1969) Christian is %d years old

 Ah OK. Thanks for looking into this. Those entries apparently produce
 events during export but don't seem to get an ID. I'll look into this
 issue during the next week, when my sinuses have calmed down a bit...

 Yes, this is the same problem I was having when I started using
 org-caldav-sync.  I currently have all my sexp based diary entries
 commented out so would definitely appreciate getting this working!

OK, I took a shot at dealing with sexp entries. It's a complicated
issue, since s-expressions can be in Org entries or by themselves. I
updated the Readme with a section on how they are handled now. Please
let me know how it works out for you. If you don't want/need sexp-based
entries in your calendar, just set org-icalendar-include-sexps to nil.

-David

Re: [O] org-caldav issue: Search failed: ;\\([A-Za-z0-9-]+\\)=

[David Engster]

David Engster writes:
 'giles' writes:
 David Engster d...@randomsample.de writes:
 Could you please do M-x toggle-debug-on-error before running the sync
 and post the resulting backtrace here?


 Slightly different behaviour this time: first seven events synced fine,
 number 8 blew up the same way.

 Debugger entered--Lisp error: (search-failed ;\\([A-Za-z0-9-]+\\)=)
   re-search-forward(;\\([A-Za-z0-9-]+\\)= nil nil)
   icalendar--read-element(VEVENT nil)
   icalendar--read-element(VCALENDAR nil)
   icalendar--read-element(nil nil)

 There must be something weird with the events you have. Could be that
 it's the special characters you mentioned, but I'm afraid I'll need to
 see the event which triggers this.

Just FYI: Giles send me the offending calendar entry off-list and this
issue is fixed.

-David

Re: [O] org-caldav feedback

[David Engster]

Torsten Wagner writes:
 Tested and you are right. Adding a timestamp in the body doesn't get
 lost during sync.  Actually, I guess the problem is a combination of
 export and import to org-mode.  During the export, the timestamp gets
 read in correctly, however, it get stripped from the Summary line
 (which is good).  During the import, org-caldav does not find a
 timestamp in the body to update and does nothing (wild speculation).

 A possible solution would be to teach org-caldav to update the timestamp 
 within
 the node header if available.

I pushed a change which should correctly deal with timestamps inside the
header line. Please let me know if this works for you.

-David

Re: [O] org-caldav issue: Search failed: ;\\([A-Za-z0-9-]+\\)=

[David Engster]

'giles' writes:
 Contacting host: www.google.com:443
 Getting event 1 of 28
 icalendar--read-element: Search failed: ;\\([A-Za-z0-9-]+\\)=

Could you please do M-x toggle-debug-on-error before running the sync
and post the resulting backtrace here?

-David

Re: [O] org-caldav issue: Search failed: ;\\([A-Za-z0-9-]+\\)=

[David Engster]

'giles' writes:
 David Engster d...@randomsample.de writes:
 Could you please do M-x toggle-debug-on-error before running the sync
 and post the resulting backtrace here?

 Slightly different behaviour this time: first seven events synced fine,
 number 8 blew up the same way.

 Debugger entered--Lisp error: (search-failed ;\\([A-Za-z0-9-]+\\)=)
   re-search-forward(;\\([A-Za-z0-9-]+\\)= nil nil)
   icalendar--read-element(VEVENT nil)
   icalendar--read-element(VCALENDAR nil)
   icalendar--read-element(nil nil)

There must be something weird with the events you have. Could be that
it's the special characters you mentioned, but I'm afraid I'll need to
see the event which triggers this.

I just pushed a change to org-caldav which introduces the option for
excessive debug output. Please pull the new version and do

(setq org-caldav-debug-level 2)

before running org-caldav-sync. The iCalendar events will then be put
fully into the *org-caldav-debug-buffer*. Please send me the event which
triggers this bug; it should simply be the last one you see in there.
You can also send me the whole buffer, of course, but *please* remember
to edit/delete any private data from these events you don't want me to
see. I'd also recommend to send it directly to me and not to the list.

-David

Re: [O] Problems with org-caldav (wrong-type-argument stringp 47)

[David Engster]

Sven Bretfeld writes:
 - progn: Could not find UID emacs207403667799062360.

I'm currently struck with a nasty cold, so I have trouble thinking. But
this means that it tries to find this ID in your Org files, and it does
not seem to be there. You can try to go there by calling

M-x org-id-goto

and yank the above ID. Does this get you anywhere?

If not, could you grep through your Org files and see if there's maybe
an ID which at least is similar? Maybe some special character was
stripped while putting the event.

Also, the *org-caldav-debug* buffer might contain more information.

 - void-function pop-to-buffer-same-window

 Could the last one be a function not implemented in my 23 version of
 Emacs?

Yes. I will have to add some compatibility code for older Emacsen.

-David

Re: [O] Problems with org-caldav (wrong-type-argument stringp 47)

[David Engster]

Sven Bretfeld writes:
 Hi David and all

 I've got it. By try and error I found out that entries like these in an
 org-file cause the problem:

 %%(diary-anniversary  6 8 1969) Christian is %d years old

Ah OK. Thanks for looking into this. Those entries apparently produce
events during export but don't seem to get an ID. I'll look into this
issue during the next week, when my sinuses have calmed down a bit...

-David

Re: [O] Problems with org-caldav (wrong-type-argument stringp 47)

[David Engster]

Sven Bretfeld writes:
 (setq org-caldav-files /home/sven/Dropbox/myconf/testlocalcal.org)

No, that's wrong. It has to be a list of files:

(setq org-caldav-files '(/home/sven/Dropbox/myconf/testlocalcal.org))

-David

Re: [O] org-caldav feedback

[David Engster]

Torsten Wagner writes:
 If I understood David right, the UTC option is just an addition to the
 already present options.
 Thus, if you used e.g. Europe/Berlin before, you do not need to change
 anything and in fact, you shouldn't see a difference.

Yes, exactly.

-David

Re: [O] org-caldav feedback

[David Engster]

Torsten Wagner writes:
 I see the problem that you might changed the text in the summary field in the
 caldav calendar, which potentially mess up the header (where to place the old
  timestamp within the context of the new text?!) but for now, I would suggest
 to simply search for a timestamp within the node-header and update it by 
 adding
 a new timestamp at the very end (but before tags ;) ).

Yes, something like this needs to be done. As I've written in my last
mail, I'm hoping for a more general solution. I'm guessing that
org-elements can help me with this, but I haven't yet looked at it in
detail.

 In addition a new variable

 org-caldav-timestamp-pos which can be either header or body

 could indicate where to place the timestamp for a new entry coming from 
 caldav.

Hmm. What are the advantages of putting it in the header?

-David

Re: [O] org-caldav feedback

[David Engster]

Eric S. Fraga writes:
 I've had to clear out the org-caldav-xxx.el file in .emacs.d a couple
 of times but that's typically due to my doing things on the same entry
 in both calendar systems (org and Google).

If you change an item in Org as well as in the Calendar, the calendar
entry should simply get overwritten with the change you did in Org. So
while this Org always wins strategy doesn't qualify as proper conflict
handling, I still wonder why you had to restart from scratch?

Anyway, this is a temporary issue; I will add proper conflict handling
in the coming weeks.

-David

Re: [O] org-caldav feedback

[David Engster]

Torsten Wagner writes:
 I did not had time to play with the different parameters. For now I simply
 added all of them.
 I guess it has to do either with the timezone or with the daylight settings.
 Maybe you want to add this to a How-to install for SOGo as a workaround.

I think I found a better solution. I pushed a change to org-caldav which
allows to set org-icalendar-timezone to the string UTC, which will put
events using universal time. The server should then transpose it to the
timezone you have set in your SOGo preferences. It works for me (for
SOGo, mind you; other calendar servers don't work well with that).

 One problem remain. If I change something in the caldav calendar, the time
 information in org get lost completely.
 E.g.
 * Meeting 2013-01-16 Wed 14:00
 becomes
 * Meeting
 It subsitutes the right entry and hence I believe it gets the ID stuff right.
 However, it seems to have trouble to interpret the time information right (and
 ignore them?).
 If there is a way to help you debugging this please let me know.

My test suite runs fine with the SOGo server, so I'm guessing it has to
do with how you format your entries. Does this also happen when you put
the timestamp underneath the heading?

On a general note, I find manipulating Org entries rather delicate and
wonder why there are no helper functions to change things like headings,
timestamps, etc., which take care of the multitude of possibilities how
entries can be formatted. My guess is that org-elements might be the
solution for this, but I haven't looked at it yet...

-David

Re: [O] org-caldav feedback

[David Engster]

Torsten Wagner writes:
 Hey David,
 could you please help me and steer me in the right direction to find the 
 cuprit
 which makes the caldav calendar lagging an hour compared to the timestamps in
 org-mode.

It's difficult. I will need to add some (optional) excessive debugging
output for seeing what's happening. Or maybe you could provide an
account on some server running SUGo so that I can debug this myself.

I guess the easy way out would be to add a variable which allows
shifting the time by X hours before sending it to the server. But I'd
rather avoid that kludge.

-David

Re: [O] org-caldav feedback

[David Engster]

Torsten Wagner writes:
 I also noticed the files org-caldav-2094e16.el and org-caldav-backup.org.
 However, they are stored in my .emacs.d folder. Would it make sens to have an
 option to save them relatively to the org-file? E.g. relative to the path set
 in org-caldav-files? That would help to keep infos together and might be even 
 a
 security concern (e.g., you might forget to move or delete backup infos in
 .emacs.d)!?

Just adapt the variables org-caldav-backup-file and org-caldav-save-directory. 

 SOGo calendars allow to set events and tasks (not sure whether tasks are part
 of the caldav specs). From what I can say they differ only in the fact that a
 task has a certain deadline and can be marked done. Thus, this would
 be equivalent to a TODO DEADLINE entry in org-mode

I'm not very familiar with the VTODO stuff. It doesn't make much of a
difference as far as CalDAV is concerned, but the import/export will
surely be tricky. For now, I'll concentrate on making the existing
features more stable.

-David

Re: [O] org-caldav feedback

[David Engster]

Torsten Wagner writes:
 You might can try

    http://sogo-demo.inverse.ca/SOGo/dav/sogo1/Calendar/personal/

 which is the demo account of the Sogo. 

Thanks. That'll work.

After a bit of fiddling it seems that SOGo really really wants a
timezone definition. I have no idea how those can be generated
on-the-fly. I have a hunch you have to hard-code them.

Anyway, you can put the definition you need into
org-caldav-calendar-preamble. But first you need the correct one.

For getting it, just create an event in your calendar. Then run
org-caldav-sync and it should be put into your org-caldav-inbox. Then,
evaluate

(pop-to-buffer (org-caldav-get-event ID))

where you have replaced ID with the ID of the event. You should see a
buffer with the iCalendar entry in it. Then copypaste everything from
BEGIN:VCALENDAR to END:VTIMEZONE into org-caldav-calendar-preamble. For
example, for Europe/Berlin it now seems to work with

(setq org-caldav-calendar-preamble
  BEGIN:VCALENDAR
PRODID:-//Inverse inc./SOGo 2.0.3a//EN
VERSION:2.0
BEGIN:VTIMEZONE
TZID:Europe/Berlin
X-LIC-LOCATION:Europe/Berlin
BEGIN:DAYLIGHT
TZOFFSETFROM:+0100
TZOFFSETTO:+0200
TZNAME:CEST
DTSTART:19700329T02
RRULE:FREQ=YEARLY;BYMONTH=3;BYDAY=-1SU
END:DAYLIGHT
BEGIN:STANDARD
TZOFFSETFROM:+0200
TZOFFSETTO:+0100
TZNAME:CET
DTSTART:19701025T03
RRULE:FREQ=YEARLY;BYMONTH=10;BYDAY=-1SU
END:STANDARD
END:VTIMEZONE
)

If it works for you with such a timezone definition, it'd be interesting
to know if SOGo needs all of that or if you could drop most of this
stuff.

-David

Re: [O] org-caldav feedback

[David Engster]

Torsten Wagner writes:
 we just tried org-caldav and it seems to work very nice.
 We use Sogo http://www.sogo.nu/ and hence David might like to ad Sogo on the
 list of possible caldav servers.

Thanks, that's good news. I'm actually pretty surprised that it works
right out of the box.

  Any plans to sync tasks too?

Could you elaborate? What exactly to you mean by 'task'? Everything with
an active timestamp should get synced.

-David

Re: [O] org-caldav: New version with proper two-way sync

[David Engster]

Detlef Steuer writes:
 So it seems all events get deleted immediately after loading them up.
 Indeed my calendars show up empty again in owncloud :-)

 Any hints?

Could you post the contents of the *org-caldav-debug* buffer when this
happens?

-David

Re: [O] org-caldav: New version with proper two-way sync

[David Engster]

Eric S. Fraga writes:
 I've tracked down the root of the various problems I have encountered
 with the synchronisation.  It all comes down to character sets.  I
 have some entries that have UTF-8 characters that are not present in
 ASCII, specifically accented characters and similar (latin1,
 basically, but not exclusively).  Any such entries cause the
 synchronisation to fail. 

Could you post an example entry where this happens?

 Resuming a failed synchronisation seems to forget to try to bring in
 the external calendar entries into org? 

That happens last, so I wouldn't know why this would be skipped on
resume. It's kinda hard to debug, though.

 The fact that descriptions are not synchronised for changed entries is
 something I understand but probably need to think about some more (in
 terms of default behaviour).  Is there a practical limitation on the
 size of the description entry that could be synchronised?  The default
 is 100 but would there be any harm in having this 1, 2 or even 3
 orders of magnitude larger?  Or even unlimited?  Just wondering.

I was wondering about that, too, but couldn't find any definite
statements on the maximum length of the DESCRIPTION field. I only
skimmed RFC 2445, though. Even if there is such a definite limit, I
wouldn't count on servers to correctly implement that. I guess you just
have to try what works.

 Anyway, with respect to my problems, would you please have a look to
 see if you are assuming ASCII or similar and whether there is anything
 you can do about this?  I (and many others, I assume) really do need
 to be able to work in UTF-8 at the very least.

I will look into that. I'm pretty sure this is fixable.

-David

Re: [O] org-caldav: New version with proper two-way sync

[David Engster]

Vincent Beffara writes:
 Works great for me, thanks! One thing that changed (but it was kind of
 a glitch anyway, even if a nice one to have): events with several
 timestamps now appear only once rather than one per timestamp. Not
 sure if this is due to org-caldav or to recent evolutions in the
 org-ics export. Anyway, not a big deal.

That's a consequence of org-caldav now doing proper syncing. Every Org
entry is linked through the UID to exactly one event in the calendar. If
I would allow an entry to produce several events in the calendar, we
would get duplicate UIDs there, which is not allowed. One could code
around that (in fact, the iCalendar export already does that by putting
stuff like 'DL' or 'TS' in front of the UIDs), but it would make syncing
from the Calendar to Org much more difficult.

 One thing hinted at in the docs, but I couldn't figure out how to do
 it: how do you use an authinfo file? Mine seems to be ignored totally
 ...

The example given in the Readme

machine www.google.com:443 port https login username password secret

should work. If it doesn't, set `auth-source-debug' to 't' and look into
the Messages buffer. Maybe it doesn't work properly on older Emacs
versions; a lot has changed in auth-source recently.

-David

[O] org-caldav: New version with proper two-way sync

[David Engster]

I just pushed a pretty big update to org-caldav. Get it at

https://github.com/dengste/org-caldav

The short story: org-caldav now does proper two-way syncing. It's pretty
much a rewrite, actually. If you're already using org-caldav, you will
have to start from scratch after updating.

Please read the README before using this version. Most notably,
org-caldav will set org-icalendar-store-UID when doing the iCalendar
export, so every entry with an activate timestamp will get an ID
property like this:

  :PROPERTIES:
  :ID:   6cdf8805-8d1a-46ac-94fc-d225cac5f098
  :END:

If you don't want this, do not use this package.

Please report bugs here or in the github tracker. And please have
backups.

-David

Re: [O] Emacs hangs in org file

[David Engster]

Stelian Iancu writes:
 Well it seems that it didn't hang at all. What happens is that the
 buffer which contains the org file doesn't seem to accept any keyboard
 input anymore.

This bug is currently being discussed on emacs-devel:

http://thread.gmane.org/gmane.emacs.devel/156120

Jan has already posted a possible fix; maybe you can help testing it?

-David

[O] Bug: org-icalendar converts org-icalendar-timezone to uppercase during export

[David Engster]

When exporting an Org file as iCalendar and org-icalendar-timezone is
set to e.g. Europe/Berlin, it gets inserted as EUROPE/BERLIN during
export (and some CalDAV servers don't like that).

This is because %Z gets replaced with org-icalendar-timezone, and the
FIXEDCASE parameter isn't set in the call to
replace-regexp-in-string. The attached patch fixes that.

-David

diff --git a/lisp/org-icalendar.el b/lisp/org-icalendar.el
index 389dc5d..12cd058 100644
--- a/lisp/org-icalendar.el
+++ b/lisp/org-icalendar.el
@@ -677,7 +677,7 @@ a time), or the day by one (if it does not contain a time).
   (setq fmt (if have-time
(replace-regexp-in-string %Z
  org-icalendar-timezone
- org-icalendar-date-time-format)
+ org-icalendar-date-time-format t)
  ;VALUE=DATE:%Y%m%d))
   (concat keyword (format-time-string fmt time
  (and 
(org-icalendar-use-UTC-date-timep)

Re: [O] org-caldav: scripting for multiple calnedars?

[David Engster]

Detlef Steuer writes:
 What I'm trying to figure out at the moment, is how to setup a batch so, that
 say work.org goes into calendar work, family.org into calendar family 
 etc., 
 without asking for  user/password combinations.

That's not implemented. I guess you can workaround the problem by
calling org-caldav-sync several times, setting org-caldav-calendar-id
and org-caldav-files accordingly each time.

-David

Re: [O] Org Writer's room

[David Engster]

Rainer M. Krug writes:
 On 06/12/12 09:36, Jambunathan K wrote:
 On the left is the navbar. - You can quickly navigate to any
 heading, a table or a captioned
 figure.

 Couldn't the navbar from emacs be used for that? I haven't used it in
 a long time, but in ecb
 (Emacs Code Browswer) it is used for this - see Screenshots on
 http://ecb.sourceforge.net/ for how
 it looks there.

Speedbar can use 'imenu' to get a list of tags, and org supports
'imenu', so it pretty much works right away, also without ECB. Just do

(require 'speedbar)
(speedbar-add-supported-extension .org)

and fire up speedbar with

M-x speedbar

You can now be able to click on org files and you should see the section
headings. It should also be possible to generate a speedbar frame or
buffer which only shows the tags of the current file, like ECB does, but
I would have to look that up if that's important.

-David

Re: [O] Org Writer's room

[David Engster]

Matt Price writes:
 (1) do you know if it's possible to get the speedbar buffer in a
 window instead of a frame?

I initially thought that would be easy. Turns out it isn't. ECB uses all
kinds of 'defadvice' to achieve that.

There's a package sr-speedbar at

http://www.emacswiki.org/SrSpeedbar

but I don't know if that one's still working.

 (2) org headings are not showing up for me with those two lines of
 code.  Evaluating
 (speedbar-add-supported-extension .org)
 gives
  
 \\(\\(\\.\\(org\\|[ch]\\(\\+\\+\\|pp\\|c\\|h\\|xx\\)?\\|tex\\(i\\(nfo\\)?\\)?\\|el\\|emacs\\|l\\|lsp\\|p\\|java\\|js\\|f\\(90\\|77\\|or\\)?\\|ad[abs]\\|p[lm]\\|tcl\\|m\\|scm\\|pm\\|py\\|g\\|s?html\\|ma?k\\)\\)\\|\\([Mm]akefile\\(\\.in\\)?\\)\\)$

 but trying to click on the + symbol in the speedbar frame next to an
 org file gives only:

 Sorry, no support for a file of that extension

 Is it possible I need something else to make the extension work?

I tried with 'emacs -Q' and it works for me. Make sure you do the
above *before* firing up speedbar.

 A speedbar buffer in the same frame that shows only headings of the
 current file would be fantastic...

Instead of bending Speedbar to your will, maybe it's just easier if
you'd look at other solutions. As I've written, Speedbar simply resorts
to 'imenu' to get the tags. Calling 'imenu-add-to-menubar' will let you
generate a menu entry for the headings; that should also work with every
org file. Maybe there's a little package out there putting the imenu
headings in a buffer; it can't be very hard to do. Just take a look

http://emacswiki.org/emacs/ImenuMode

as a starting point.

-David

Re: [O] org-caldav: Sync Org with external calendars through CalDAV (Owncloud, Google, ...)

[David Engster]

Stephen Eglen writes:
 I think it is a problem somewhere with my local setup; if I add the
 following to .authinfo (chmod 600) then it uses the right username:

 Machine www.google.com login my-gmail-login

 I'm now about to see how to setup .authinfo.gpg so that I can also
 include my password in that file.

 Hmmm. not having any luck here; it works with .authinfo, but not with
 .authinfo.gpg in Emacs 24.1.

It's bug:

http://lists.gnu.org/archive/html/bug-gnu-emacs/2012-07/msg00674.html

Not sure if that fix is in 24.2, but surely the latest pretest for 24.3
should work (which is pretty stable).

I'm still wondering why it didn't work at the beginning. I mean, even if
you had your Google username set in some .netrc or .authinfo, the
password should still work?

-David

Re: [O] org-caldav: Sync Org with external calendars through CalDAV (Owncloud, Google, ...)

[David Engster]

Suvayu Ali writes:
 On Mon, Dec 03, 2012 at 08:50:35PM +0100, David Engster wrote:
 I'm at a loss why this happens, and I could not reproduce it with my
 account. This problem really has nothing to do with org-caldav, since it
 is delegating this part to the url package. As I've already written to
 Bastien, please try
 
  (url-retrieve-synchronously

 https://www.google.com/calendar/dav/your-calendar...@group.calendar.google.com/events/;)
 
 If this doesn't ask for your username, then there's your
 problem. Depending on the Emacs version you're using, you might want to
 set
 
 (setq auth-source-debug t)
 
 to see whether some .netrc file or similar is setting the username for
 you (look into the *Messages* buffer).
 

 I see the same issue, I get only a password prompt.  This is what I get
 on trying your suggestion above.

 Contacting host: www.google.com:443
 auth-source-search: found 1 results (max 1) matching (:max 1 :host
 www.google.com:443 :port https)
 auth-source-search: found 1 CACHED results matching (:max 1 :host
 www.google.com:443 :port https)
 #buffer  *http www.google.com:443*-771021

 I'm not sure from this if my netrc is being used.

Yes. The mesage found 1 results means it found an entry in your .netrc
or .authinfo which matches.

  My netrc also has the password for my Google account, if it is being
 used should I even get a password prompt?

The auth-source stuff can be a bit tricky. For https connections with
the URL package, you need to use the following

  machine www.google.com:443 port https login USERNAME password PASS

That's no typo: You have to use :443 as well as port https. See also

 (info (auth)Help for users)

-David

Re: [O] org-caldav: Sync Org with external calendars through CalDAV (Owncloud, Google, ...)

[David Engster]

Stephen Eglen writes:
 David Engster deng at randomsample.de writes:
 Bastien writes:
  David Engster deng at randomsample.de writes:
  That is very strange. It should first
  ask: Username [for Google CalDAV]:. If it does not do that, maybe you
  have that information in your .authinfo?
 
  I have this:
 
  machine smtp.gmail.com login bastienguerry at gmail.com password xx
 
 Since we're connecting to google.com this shouldn't matter, obviously,
 thus I'm at a loss why it doesn't ask. I currently cannot test anything
 reliably because I'm on vacation and there's no 3G around here, so I'm
 afraid this has to wait a bit. Unless someone beats me to it. 

 did either of you resolve this problem about org-caldav just asking for the
 password, and not the username?  I am now trying org-caldav, and am seeing the
 same problem.  Happy to debug further if you can tell me which functions to 
 look
 into.

I'm at a loss why this happens, and I could not reproduce it with my
account. This problem really has nothing to do with org-caldav, since it
is delegating this part to the url package. As I've already written to
Bastien, please try

 (url-retrieve-synchronously
   
https://www.google.com/calendar/dav/your-calendar...@group.calendar.google.com/events/;)

If this doesn't ask for your username, then there's your
problem. Depending on the Emacs version you're using, you might want to
set

(setq auth-source-debug t)

to see whether some .netrc file or similar is setting the username for
you (look into the *Messages* buffer).

-David

Re: [O] org-caldav: Sync Org with external calendars through CalDAV (Owncloud, Google, ...)

[David Engster]

Bastien writes:
 David Engster d...@randomsample.de writes:
 That is very strange. It should first
 ask: Username [for Google CalDAV]:. If it does not do that, maybe you
 have that information in your .authinfo?

 I have this:

 machine smtp.gmail.com login bastiengue...@gmail.com password xx

Since we're connecting to google.com this shouldn't matter, obviously,
thus I'm at a loss why it doesn't ask. I currently cannot test anything
reliably because I'm on vacation and there's no 3G around here, so I'm
afraid this has to wait a bit. Unless someone beats me to it. ;-)

-David

Re: [O] org-caldav: Sync Org with external calendars through CalDAV (Owncloud, Google, ...)

[David Engster]

Bastien writes:
 I use Emacs from after your patch to url-dav (7/26/2012)

Then I guess you patched org-caldav-sync to not test for
`url-dav-patched-version'? Anyway, I've now added support for the
Emacs-bzr version of url-dav.

 and I use this simple configuration:

 (setq org-caldav-calendar-id
 4ttssrunbsh9km06csbjkb2...@group.calendar.google.com
   org-caldav-url https://www.google.com/calendar/dav;
   org-caldav-files '(~/org/rdv.org)
   org-caldav-inbox ~/org/inbox.org)

 (The id is a true one for a public test calendar.)

 Then I'm asked for a password.

It does not ask for a username? That is very strange. It should first
ask: Username [for Google CalDAV]:. If it does not do that, maybe you
have that information in your .authinfo?

Please restart Emacs and evaluate

(url-retrieve-synchronously
  
https://www.google.com/calendar/dav/4ttssrunbsh9km06csbjkb2...@group.calendar.google.com/events/;)

Does that one ask you for a username?

-David

[O] org-caldav: Sync Org with external calendars through CalDAV (Owncloud, Google, ...)

[David Engster]

I have written a package 'org-caldav' which can sync items to a remote
calendar server using the CalDAV protocol. The main purpose of this
package is to make better use of Org in combination with Android-based
mobile devices (yes, there is mobileOrg, but I have several problems
with it; that's a topic for another day, though).

I think org-caldav is now good enough to allow some testing by
adventurous people. I put the code up on github here

https://github.com/dengste/org-caldav

I've tested the code with my Owncloud server and also with Google
Calendar. Please see the Readme on how to set things up. Most
importantly, please use a new, dedicated calendar for testing this
package; do *not* use your precious main calendar. Also, while
org-caldav should only access the calendar you provide through
`org-caldav-calendar-id', you should have a backup of all your calendar
data, just in case.

The initial sync can take some time, depending on the number of
events. Especially the CalDAV interface to Google can be quite slow at
times. If the initial sync aborts for any reason, just run
`org-caldav-sync' again. Note that after the initial sync, only new
events will be transferred, so things should become much snappier.

A few further notes regarding the code:

I am relying mainly on three other packages to do the real heavy
lifting, which is also why org-caldav is actually pretty small:

- url-dav.el for talking to WebDAV servers (CalDAV is essentially WebDAV
  with a few extensions). Unfortunately, the current version of url-dav
  that is shipping with Emacs is broken for practically all CalDAV
  servers (see bug #11916), which is why I provide a patched version of
  url-dav.el in the above repository. You have to make sure that this
  patched version is loaded instead of the one in Emacs proper. I am
  hoping that this will be fixed in Emacs 24.2.

- org-icalendar.el is doing all the conversion from org to icalendar
  events which are needed for CalDAV. This has the advantage that you
  can use all the variables from that package to control how things
  should be exported. The only exception is the actual org files being
  searched for time stamps, which is determined through
  `org-caldav-files'.

- icalendar.el for parsing events from CalDAV. I took some code from
  that package to generate org items from the calendar events.

The syncing between org and the external calendar is not yet really
finished. I would like to have proper two-way-syncing so that you can
freely change events on any side, but this is rather difficult to do and
I'd first like to get some feedback on the current code. Also, I have
a hunch that some Org experts around here might have some good ideas on
how to properly implement this.

Here's what the code can currently do:

- All items in `org-caldav-files' with active time stamps are synced to
  the calendar.

- If you change an item in one of those files in orgmode, this change
  will also get synced to the external calendar (to be exact: this will
  generate a new event and the old one will be deleted).

- If you create a *new* event in the calendar through Android or any
  other client (like browser), this event will land in the
  `org-caldav-inbox' file as an org item.

However:

- If you *change* an item directly on the CalDAV server, this will *not*
  get synced back to Org. What happens exactly depends on the server
  (whether it renames the event or not). Just try it out. The item might
  even get deleted after the next sync - you have been warned.

- Likewise, if you *change* and item in `org-caldav-inbox', this will
  not get synced back to the calendar.

OK, this has gotten way too long already. Please try it out and let me
know how it goes.

Cheers,
David

[Orgmode] Re: Org-mode to blog with Drupal/Wordpress?

[David Engster]

David Maus maus.da...@gmail.com writes:
 At Wed, 11 Nov 2009 11:19:27 -0500,
 U Avalos wrote:
 Hi all. I know that there are various packages out there to use org-mode to 
 publish *websites* but I don't want to reinvent the wheel. Is there a way to 
 use org-mode to add posts to an existing Drupal or Wordpress site? (I 
 believe 
 they both use the same API, that's why I'm mentioning both.) I also know I 
 can 
 just cut-and-paste text into the web browser, but that method fails for 
 images.
 
 As a plus, it would be great if I could also edit *existing* posts (even if 
 using nxtml-mode or html-mode)... but I'm happy if you could just add new 
 posts...

 As far as I am aware there's no such extension to orgmode right now. A
 fast glance on drupals and wordpress' remote editing capabilities
 shows that they use different APIs but both use xmlrpc. So in theory
 it should be possible as there already is a xml-rpc library for Emacs
 that works according to my experiments with xml-rpc.el quite well.

I'm a bit late on this, but I just tried editing Drupal blogs with
Weblogger-Mode (see http://www.emacswiki.org/emacs/WebloggerMode), and
it works right out of the box. I wrote about the details here:

http://www.randomsample.de/dru5/node/76

You can also edit existing entries - it's really a great package. I
guess one could hook weblogger directly into orgmode, but manually
exporting to HTML and then doing a quick copypaste isn't really a big
deal.

-David



___
Emacs-orgmode mailing list
Remember: use `Reply All' to send replies to the list.
Emacs-orgmode@gnu.org
http://lists.gnu.org/mailman/listinfo/emacs-orgmode

[Orgmode] Re: Sorting TODOs with time-up

[David Engster]

Carsten Dominik carsten.domi...@gmail.com writes:
 On Mar 30, 2009, at 11:03 PM, David Engster wrote:
 While browsing through chapter 10, I got the impression that the
 different views are just different filters, but that I'm always
 dealing with the same agenda mode, where each entry carries the same
 information.

 Actually, I think this is so, entries always do carry the same
 information in all these views.  remember that, a Matt said,
 time-up is time-of-day only.

Yes. What I wanted to say is that when it do C-u C-x = on an entry in
the daily/weekly agenda, I see properties like

  date (4 1 2009)
  day  733498
  dotime   [Show]
  extraScheduled: 
  org-day-cnt  3
  time 12:00..
  time-of-day  1200

which are not existent or 'nil' in the TODO list (but can be parsed
through the org-hd-marker information).

The reason I'm dealing with this, aside from sorting, is that I'd like
to extract date information from (custom) agenda views, so that I can
display them in other programs, e.g. in special Gnus groups. I don't
know yet if that really makes sense, but since I'm mostly living in
Gnus, it would probably have advantages for me.

Regards,
David



___
Emacs-orgmode mailing list
Remember: use `Reply All' to send replies to the list.
Emacs-orgmode@gnu.org
http://lists.gnu.org/mailman/listinfo/emacs-orgmode

[Orgmode] Sorting TODOs with time-up

[David Engster]

Hi all,

I guess this should be simple but somehow it's not working for me. I
want to generate a todo list which is sorted according to the scheduled
date. Consider the following test.org file:

** TODO: first SCHEDULED: 2009-04-01 Wed
** TODO: second SCHEDULED: 2009-04-23 Thu
** TODO: third SCHEDULED: 2009-04-16 Thu

This minimal setup

(setq org-agenda-files '(~/orgtest/test.org))
(setq org-agenda-sorting-strategy 
  '((agenda time-up priority-down category-keep)
(todo time-up priority-down category-keep)
(tags priority-down category-keep)
(search category-keep)))

should sort TODOs according to time, right? But when I do

M-x org-agenda RET t

I get:

Global list of TODO items of type: ALL
Available with `N r': (0)ALL (1)TODO (2)DONE
  test:   TODO: first SCHEDULED: 2009-04-01 Wed
  test:   TODO: second SCHEDULED: 2009-04-23 Thu
  test:   TODO: third SCHEDULED: 2009-04-16 Thu

Same with org-todo-list. Also, doing 'C-u C-x =' on one of the lines shows
that the time-of-day property is nil, so it seems that there's just no
information for sorting according to date. What am I missing? (I'm using
org-mode snapshot from today with latest Emacs 23 pretest).

Thanks for your help.

-David



___
Emacs-orgmode mailing list
Remember: use `Reply All' to send replies to the list.
Emacs-orgmode@gnu.org
http://lists.gnu.org/mailman/listinfo/emacs-orgmode

[Orgmode] Re: Sorting TODOs with time-up

[David Engster]

Matthew Lundin m...@imapmail.org writes:
 David Engster d...@randomsample.de writes:
 I guess this should be simple but somehow it's not working for me. I
 want to generate a todo list which is sorted according to the scheduled
 date. Consider the following test.org file:

[...]

 This minimal setup

 (setq org-agenda-files '(~/orgtest/test.org))
 (setq org-agenda-sorting-strategy 
   '((agenda time-up priority-down category-keep)
  (todo time-up priority-down category-keep)
  (tags priority-down category-keep)
  (search category-keep)))

 should sort TODOs according to time, right? But when I do

 As far as I understand it, time-up sorts only by time of day in the
 agenda view. I.e., it will place items that have been scheduled for a
 particular hour at the top of the agenda.

 Here's some info from the docstring:

 ,
 | time-upPut entries with time-of-day indications first, early 
 first
 | time-down  Put entries with time-of-day indications first, late 
 first
 | 
 | Pull out all entries having a specified time of day and sort them,
 | in order to make a time schedule for the current day the first thing in the
 | agenda listing for the day.
 `

Thanks for the clarification. This is a bit confusing, since sorting by
priority works in the todo list, but sorting with date does not. Also,
since I call the function 'org-agenda', it is a bit unexpected that when
I press 't' afterwards I don't really get an agenda but a TODO list,
which seems to behave differently in some aspects from the normal agenda
view I get when pressing 'a'.

 There is the possibility of using a user-defined function. Perhaps you
 could use that to sort by date.

 ,
 | user-defined-upSort according to `org-agenda-cmp-user-defined', high 
 last.
 | user-defined-down  Sort according to `org-agenda-cmp-user-defined', high 
 first.
 `

Unfortunately, this does not help, since there is no time information in
the entries 'a' and 'b' to sort by; it seems the timestamps are simply
not extracted when building TODO lists.

Thanks for your help,
-David



___
Emacs-orgmode mailing list
Remember: use `Reply All' to send replies to the list.
Emacs-orgmode@gnu.org
http://lists.gnu.org/mailman/listinfo/emacs-orgmode

[Orgmode] Re: Sorting TODOs with time-up

[David Engster]

Carsten Dominik carsten.domi...@gmail.com writes:
 On Mar 30, 2009, at 4:46 PM, David Engster wrote:
 Thanks for the clarification. This is a bit confusing, since sorting
 by priority works in the todo list, but sorting with date does
 not. Also, since I call the function 'org-agenda', it is a bit
 unexpected that when I press 't' afterwards I don't really get an
 agenda but a TODO list, which seems to behave differently in some
 aspects from the normal agenda view I get when pressing 'a'.

 Yes, this is bad terminology, which evolved historically.

 org-agenda is a dispatcher for the different views that we call
 agenda views.

 One of the agenda views is the global todo list (on `t'), another
 one is the daily/weekly agenda (on `'a), which is a list of what is
 due today or this week.
  I always try to call it explicitly
 daily/weekly agenda to avoid some of that confusion, but I agree, if
 I'd get a chance to start again I would use better terms.  Now these
 are so engrained in our mailing list culture and into tutorials etc
 that I do not dare to change them anymore.

Thank you for the explanation. Maybe this situation could be cleared up
a bit in the manual. While browsing through chapter 10, I got the
impression that the different views are just different filters, but that
I'm always dealing with the same agenda mode, where each entry carries
the same information.

Maybe one could explicitly mention which commands and sorting strategies
only apply to the daily/weekly agenda? I'd also say that the doc-string
of org-agenda-sorting-strategy does not make this point clear enough.

 Yes can use the text property on a, `org-hd-marker', which is a marker
 pointing to the original location of the item.  You can make your
 function go back to get this information.

 For example with
  (org-entry-get (get-text-property 0 'org-hd-marker a) SCHEDULED)

I will try that. Thank you for your help!

-David



___
Emacs-orgmode mailing list
Remember: use `Reply All' to send replies to the list.
Emacs-orgmode@gnu.org
http://lists.gnu.org/mailman/listinfo/emacs-orgmode