Re: Cyrus IMAP / CalDAV

[Ken Murchison]

I can't make a formal announcement of beta9 until tomorrow when I get my 
coworker to upload the distro to cyrusimap.org, but here it is if you 
want to give it a try:

http://www.contrib.andrew.cmu.edu/~murch/cyrus-imapd-2.4.17-caldav-beta9.tar.gz 




On 12/16/2013 04:19 PM, Marty Lee wrote:
> Thanks for all the hard work to get the actual answer Ken; I’ll apply the 
> patch to my
> local server for me to test (only 2 of us using the calendar stuff at the 
> moment) and
> wait with baited breath for an apple update :-)
>
> If you get wind of apple fixing things, let me know - if I spot it at this 
> end, I’ll send
> something out too.
>
> Cheers
>
> marty
>
>
>
> On 16 Dec 2013, at 19:09, Ken Murchison  wrote:
>
>> I confirmed that the DELETE problem is indeed a bug in the Apple client, and 
>> that Apple is aware of it. I'm somewhat reluctant to to include a fix in 
>> Cyrus for a bug in a client that will hopefully get fixed sooner rather than 
>> later. The patch below will work around the problem by making the faulty 
>> conditional DELETE a non-conditional one.  But, by doing so we may delete a 
>> resource that has been changed by another user/client/session.  Given that 
>> we really don't support shared calendars at the moment, this probably isn't 
>> a big deal but I don't really want to create potentially bigger problems 
>> moving forward.
>>
>> The real fix is Apple correcting their client to use an If-Match header 
>> rather than If-Schedule-Tag-Match header if the resource doesn't have a 
>> Schedule-Tag and/or isn't a scheduling object.
>>
>>
>> On 12/14/2013 01:02 PM, Ken Murchison wrote:
>>> I just committed a fix to git for the 406 response to GET.  I will make
>>> a beta9 release with this fix, and hopefully with a fix for the DELETE
>>> issue by early next week.
>>>
>>> I have an email into one of the CalDAV experts that I know at Apple to
>>> see what CalendarServer does with the empty If-Schedule-Tag-Match
>>> header.  I think its a bug in the Apple client, but I will have to come
>>> up with a sane workaround for it. In the meantime, this uncommitted
>>> patch should fix your problem with DELETE:
>>>
>>>
>>> diff --git a/imap/http_caldav.c b/imap/http_caldav.c
>>> index c00223f..641feb8 100644
>>> --- a/imap/http_caldav.c
>>> +++ b/imap/http_caldav.c
>>> @@ -695,6 +695,7 @@ static int caldav_check_precond(struct transaction_t
>>> *txn, const void *data,
>>>
>>>/* Per RFC 6638, check Schedule-Tag */
>>>if ((hdr = spool_getheader(txn->req_hdrs, "If-Schedule-Tag-Match"))) 
>>> {
>>> +if (!*hdr[0]) return precond;  /* XXX  Hack for bug in Apple client */
>>>if (etagcmp(hdr[0], stag)) return HTTP_PRECOND_FAILED;
>>>}
>>>
>>>
>>>
>>>
>>> On 12/14/2013 09:39 AM, Marty Lee wrote:
 No worries.. I'm about to get back onto another train so will back out 
 b8.. Only me using it in earnest, so if you need anything else tested 
 before pushing out, just send me a link.

 Marty Lee
 v: 07827 950 918

> On 14 Dec 2013, at 14:26, Ken Murchison  wrote:
>
> Hi Marty,
>
> Thanks for the info.  The 406 is in response to the GET, caused by a bug 
> I introduced when I added support for jCal and xCal data.  I can't 
> believe that this didn't present itself in my testing.  I will need to 
> fix this immediately.  You probably want to downgrade to beta7 in the 
> meantime.
>
> I *think* the problem with DELETE is that iCal is sending an empty 
> If-Schedule-Tag-Match header.  I will need to test this here and possibly 
> talk to the Apple guys to find out why they are sending an empty header, 
> and what they expect the behavior to be.
>
>
>> On 12/14/2013 03:09 AM, Marty Lee wrote:
>> Ken,
>>
>> I haven’t but have just taken the opportunity to update to Beta 8 and 
>> also to refresh Sqlite, which
>> seems to be the source of the error message…
>>
>> Using cyrus beta 7, the iCal client would delete the event, but when it 
>> updated with the server, the
>> event would magically just re-appear. With b8, this has changed; now I 
>> get a dialog box:
>>
>> --
>> The request for “Marty” in account “Maui” failed.
>>
>> The server responded with
>> “406” to operation CalDAVDeleteEntityQueueableOperation.
>> -
>>
>> Telemetry log:
>>
>> <1387007669> /dav/calendars/user/marty/Default/0C48ECD9-44A7-4F1F-9C87-9A2EF647C574.ics
>>  HTTP/1.1
>> Accept-encoding: gzip, deflate
>> Max-forwards: 10
>> Accept-language: en-gb
>> User-agent: Mac_OS_X/10.9 (13A603) CalendarAgent/174
>> Host: 192.168.253.16:1443
>> Accept: */*
>> Content-length: 0
>> X-forwarded-server: dav.maui.co.uk
>> If-schedule-tag-match:
>> X-forwarded-for: 176.12.107.140
>> Authorization: Basic ...
>

Re: Cyrus IMAP / CalDAV

[Ken Murchison]

I have decided to commit my workaround patch to git, but it will only 
work for requests from the Apple client in question.  I will be making a 
beta9 release shortly.

The Apple guys aren't at liberty to disclose if/when there will be a 
release, but I would expect them to fix this soonish.


On 12/16/2013 04:19 PM, Marty Lee wrote:
> Thanks for all the hard work to get the actual answer Ken; I’ll apply the 
> patch to my
> local server for me to test (only 2 of us using the calendar stuff at the 
> moment) and
> wait with baited breath for an apple update :-)
>
> If you get wind of apple fixing things, let me know - if I spot it at this 
> end, I’ll send
> something out too.
>
> Cheers
>
> marty
>
>
>
> On 16 Dec 2013, at 19:09, Ken Murchison  wrote:
>
>> I confirmed that the DELETE problem is indeed a bug in the Apple client, and 
>> that Apple is aware of it. I'm somewhat reluctant to to include a fix in 
>> Cyrus for a bug in a client that will hopefully get fixed sooner rather than 
>> later. The patch below will work around the problem by making the faulty 
>> conditional DELETE a non-conditional one.  But, by doing so we may delete a 
>> resource that has been changed by another user/client/session.  Given that 
>> we really don't support shared calendars at the moment, this probably isn't 
>> a big deal but I don't really want to create potentially bigger problems 
>> moving forward.
>>
>> The real fix is Apple correcting their client to use an If-Match header 
>> rather than If-Schedule-Tag-Match header if the resource doesn't have a 
>> Schedule-Tag and/or isn't a scheduling object.
>>
>>
>> On 12/14/2013 01:02 PM, Ken Murchison wrote:
>>> I just committed a fix to git for the 406 response to GET.  I will make
>>> a beta9 release with this fix, and hopefully with a fix for the DELETE
>>> issue by early next week.
>>>
>>> I have an email into one of the CalDAV experts that I know at Apple to
>>> see what CalendarServer does with the empty If-Schedule-Tag-Match
>>> header.  I think its a bug in the Apple client, but I will have to come
>>> up with a sane workaround for it. In the meantime, this uncommitted
>>> patch should fix your problem with DELETE:
>>>
>>>
>>> diff --git a/imap/http_caldav.c b/imap/http_caldav.c
>>> index c00223f..641feb8 100644
>>> --- a/imap/http_caldav.c
>>> +++ b/imap/http_caldav.c
>>> @@ -695,6 +695,7 @@ static int caldav_check_precond(struct transaction_t
>>> *txn, const void *data,
>>>
>>>/* Per RFC 6638, check Schedule-Tag */
>>>if ((hdr = spool_getheader(txn->req_hdrs, "If-Schedule-Tag-Match"))) 
>>> {
>>> +if (!*hdr[0]) return precond;  /* XXX  Hack for bug in Apple client */
>>>if (etagcmp(hdr[0], stag)) return HTTP_PRECOND_FAILED;
>>>}
>>>
>>>
>>>
>>>
>>> On 12/14/2013 09:39 AM, Marty Lee wrote:
 No worries.. I'm about to get back onto another train so will back out 
 b8.. Only me using it in earnest, so if you need anything else tested 
 before pushing out, just send me a link.

 Marty Lee
 v: 07827 950 918

> On 14 Dec 2013, at 14:26, Ken Murchison  wrote:
>
> Hi Marty,
>
> Thanks for the info.  The 406 is in response to the GET, caused by a bug 
> I introduced when I added support for jCal and xCal data.  I can't 
> believe that this didn't present itself in my testing.  I will need to 
> fix this immediately.  You probably want to downgrade to beta7 in the 
> meantime.
>
> I *think* the problem with DELETE is that iCal is sending an empty 
> If-Schedule-Tag-Match header.  I will need to test this here and possibly 
> talk to the Apple guys to find out why they are sending an empty header, 
> and what they expect the behavior to be.
>
>
>> On 12/14/2013 03:09 AM, Marty Lee wrote:
>> Ken,
>>
>> I haven’t but have just taken the opportunity to update to Beta 8 and 
>> also to refresh Sqlite, which
>> seems to be the source of the error message…
>>
>> Using cyrus beta 7, the iCal client would delete the event, but when it 
>> updated with the server, the
>> event would magically just re-appear. With b8, this has changed; now I 
>> get a dialog box:
>>
>> --
>> The request for “Marty” in account “Maui” failed.
>>
>> The server responded with
>> “406” to operation CalDAVDeleteEntityQueueableOperation.
>> -
>>
>> Telemetry log:
>>
>> <1387007669> /dav/calendars/user/marty/Default/0C48ECD9-44A7-4F1F-9C87-9A2EF647C574.ics
>>  HTTP/1.1
>> Accept-encoding: gzip, deflate
>> Max-forwards: 10
>> Accept-language: en-gb
>> User-agent: Mac_OS_X/10.9 (13A603) CalendarAgent/174
>> Host: 192.168.253.16:1443
>> Accept: */*
>> Content-length: 0
>> X-forwarded-server: dav.maui.co.uk
>> If-schedule-tag-match:
>> X-forwarded-for: 176.12.107.140
>> Authorization: Basic ...
>> X-forwarded-host: cal.maui.co.uk
>>>

Re: Cyrus IMAP / CalDAV

[Marty Lee]

Thanks for all the hard work to get the actual answer Ken; I’ll apply the patch 
to my
local server for me to test (only 2 of us using the calendar stuff at the 
moment) and
wait with baited breath for an apple update :-)

If you get wind of apple fixing things, let me know - if I spot it at this end, 
I’ll send
something out too.

Cheers

marty



On 16 Dec 2013, at 19:09, Ken Murchison  wrote:

> I confirmed that the DELETE problem is indeed a bug in the Apple client, and 
> that Apple is aware of it. I'm somewhat reluctant to to include a fix in 
> Cyrus for a bug in a client that will hopefully get fixed sooner rather than 
> later. The patch below will work around the problem by making the faulty 
> conditional DELETE a non-conditional one.  But, by doing so we may delete a 
> resource that has been changed by another user/client/session.  Given that we 
> really don't support shared calendars at the moment, this probably isn't a 
> big deal but I don't really want to create potentially bigger problems moving 
> forward.
> 
> The real fix is Apple correcting their client to use an If-Match header 
> rather than If-Schedule-Tag-Match header if the resource doesn't have a 
> Schedule-Tag and/or isn't a scheduling object.
> 
> 
> On 12/14/2013 01:02 PM, Ken Murchison wrote:
>> I just committed a fix to git for the 406 response to GET.  I will make
>> a beta9 release with this fix, and hopefully with a fix for the DELETE
>> issue by early next week.
>> 
>> I have an email into one of the CalDAV experts that I know at Apple to
>> see what CalendarServer does with the empty If-Schedule-Tag-Match
>> header.  I think its a bug in the Apple client, but I will have to come
>> up with a sane workaround for it. In the meantime, this uncommitted
>> patch should fix your problem with DELETE:
>> 
>> 
>> diff --git a/imap/http_caldav.c b/imap/http_caldav.c
>> index c00223f..641feb8 100644
>> --- a/imap/http_caldav.c
>> +++ b/imap/http_caldav.c
>> @@ -695,6 +695,7 @@ static int caldav_check_precond(struct transaction_t
>> *txn, const void *data,
>> 
>>   /* Per RFC 6638, check Schedule-Tag */
>>   if ((hdr = spool_getheader(txn->req_hdrs, "If-Schedule-Tag-Match"))) {
>> +if (!*hdr[0]) return precond;  /* XXX  Hack for bug in Apple client */
>>   if (etagcmp(hdr[0], stag)) return HTTP_PRECOND_FAILED;
>>   }
>> 
>> 
>> 
>> 
>> On 12/14/2013 09:39 AM, Marty Lee wrote:
>>> No worries.. I'm about to get back onto another train so will back out b8.. 
>>> Only me using it in earnest, so if you need anything else tested before 
>>> pushing out, just send me a link.
>>> 
>>> Marty Lee
>>> v: 07827 950 918
>>> 
 On 14 Dec 2013, at 14:26, Ken Murchison  wrote:
 
 Hi Marty,
 
 Thanks for the info.  The 406 is in response to the GET, caused by a bug I 
 introduced when I added support for jCal and xCal data.  I can't believe 
 that this didn't present itself in my testing.  I will need to fix this 
 immediately.  You probably want to downgrade to beta7 in the meantime.
 
 I *think* the problem with DELETE is that iCal is sending an empty 
 If-Schedule-Tag-Match header.  I will need to test this here and possibly 
 talk to the Apple guys to find out why they are sending an empty header, 
 and what they expect the behavior to be.
 
 
> On 12/14/2013 03:09 AM, Marty Lee wrote:
> Ken,
> 
> I haven’t but have just taken the opportunity to update to Beta 8 and 
> also to refresh Sqlite, which
> seems to be the source of the error message…
> 
> Using cyrus beta 7, the iCal client would delete the event, but when it 
> updated with the server, the
> event would magically just re-appear. With b8, this has changed; now I 
> get a dialog box:
> 
> --
> The request for “Marty” in account “Maui” failed.
> 
> The server responded with
> “406” to operation CalDAVDeleteEntityQueueableOperation.
> -
> 
> Telemetry log:
> 
> <1387007669 /dav/calendars/user/marty/Default/0C48ECD9-44A7-4F1F-9C87-9A2EF647C574.ics
>  HTTP/1.1
> Accept-encoding: gzip, deflate
> Max-forwards: 10
> Accept-language: en-gb
> User-agent: Mac_OS_X/10.9 (13A603) CalendarAgent/174
> Host: 192.168.253.16:1443
> Accept: */*
> Content-length: 0
> X-forwarded-server: dav.maui.co.uk
> If-schedule-tag-match:
> X-forwarded-for: 176.12.107.140
> Authorization: Basic ...
> X-forwarded-host: cal.maui.co.uk
> 
> BEGIN:VCALENDAR
> VERSION:2.0
> PRODID:-//Apple Inc.//Mac OS X 10.9//EN
> CALSCALE:GREGORIAN
> BEGIN:VTIMEZONE
> TZID:Europe/London
> BEGIN:DAYLIGHT
> TZOFFSETFROM:+
> RRULE:FREQ=YEARLY;BYMONTH=3;BYDAY=-1SU
> DTSTART:19810329T01
> TZNAME:BST
> TZOFFSETTO:+0100
> END:DAYLIGHT
> BEGIN:STANDARD
> TZOFFSETFROM:+0100
> RRULE:FREQ=YEARLY;BYMONTH=10;BYDAY=-1SU
> DTSTART:19961027T02

Re: Cyrus IMAP / CalDAV

[Ken Murchison]

I confirmed that the DELETE problem is indeed a bug in the Apple client, 
and that Apple is aware of it. I'm somewhat reluctant to to include a 
fix in Cyrus for a bug in a client that will hopefully get fixed sooner 
rather than later. The patch below will work around the problem by 
making the faulty conditional DELETE a non-conditional one.  But, by 
doing so we may delete a resource that has been changed by another 
user/client/session.  Given that we really don't support shared 
calendars at the moment, this probably isn't a big deal but I don't 
really want to create potentially bigger problems moving forward.

The real fix is Apple correcting their client to use an If-Match header 
rather than If-Schedule-Tag-Match header if the resource doesn't have a 
Schedule-Tag and/or isn't a scheduling object.


On 12/14/2013 01:02 PM, Ken Murchison wrote:
> I just committed a fix to git for the 406 response to GET.  I will make
> a beta9 release with this fix, and hopefully with a fix for the DELETE
> issue by early next week.
>
> I have an email into one of the CalDAV experts that I know at Apple to
> see what CalendarServer does with the empty If-Schedule-Tag-Match
> header.  I think its a bug in the Apple client, but I will have to come
> up with a sane workaround for it. In the meantime, this uncommitted
> patch should fix your problem with DELETE:
>
>
> diff --git a/imap/http_caldav.c b/imap/http_caldav.c
> index c00223f..641feb8 100644
> --- a/imap/http_caldav.c
> +++ b/imap/http_caldav.c
> @@ -695,6 +695,7 @@ static int caldav_check_precond(struct transaction_t
> *txn, const void *data,
>
>/* Per RFC 6638, check Schedule-Tag */
>if ((hdr = spool_getheader(txn->req_hdrs, "If-Schedule-Tag-Match"))) {
> +if (!*hdr[0]) return precond;  /* XXX  Hack for bug in Apple client */
>if (etagcmp(hdr[0], stag)) return HTTP_PRECOND_FAILED;
>}
>
>
>
>
> On 12/14/2013 09:39 AM, Marty Lee wrote:
>> No worries.. I'm about to get back onto another train so will back out b8.. 
>> Only me using it in earnest, so if you need anything else tested before 
>> pushing out, just send me a link.
>>
>> Marty Lee
>> v: 07827 950 918
>>
>>> On 14 Dec 2013, at 14:26, Ken Murchison  wrote:
>>>
>>> Hi Marty,
>>>
>>> Thanks for the info.  The 406 is in response to the GET, caused by a bug I 
>>> introduced when I added support for jCal and xCal data.  I can't believe 
>>> that this didn't present itself in my testing.  I will need to fix this 
>>> immediately.  You probably want to downgrade to beta7 in the meantime.
>>>
>>> I *think* the problem with DELETE is that iCal is sending an empty 
>>> If-Schedule-Tag-Match header.  I will need to test this here and possibly 
>>> talk to the Apple guys to find out why they are sending an empty header, 
>>> and what they expect the behavior to be.
>>>
>>>
 On 12/14/2013 03:09 AM, Marty Lee wrote:
 Ken,

 I haven’t but have just taken the opportunity to update to Beta 8 and also 
 to refresh Sqlite, which
 seems to be the source of the error message…

 Using cyrus beta 7, the iCal client would delete the event, but when it 
 updated with the server, the
 event would magically just re-appear. With b8, this has changed; now I get 
 a dialog box:

 --
 The request for “Marty” in account “Maui” failed.

 The server responded with
 “406” to operation CalDAVDeleteEntityQueueableOperation.
 -

 Telemetry log:

 <1387007669>>> /dav/calendars/user/marty/Default/0C48ECD9-44A7-4F1F-9C87-9A2EF647C574.ics 
 HTTP/1.1
 Accept-encoding: gzip, deflate
 Max-forwards: 10
 Accept-language: en-gb
 User-agent: Mac_OS_X/10.9 (13A603) CalendarAgent/174
 Host: 192.168.253.16:1443
 Accept: */*
 Content-length: 0
 X-forwarded-server: dav.maui.co.uk
 If-schedule-tag-match:
 X-forwarded-for: 176.12.107.140
 Authorization: Basic ...
 X-forwarded-host: cal.maui.co.uk

 BEGIN:VCALENDAR
 VERSION:2.0
 PRODID:-//Apple Inc.//Mac OS X 10.9//EN
 CALSCALE:GREGORIAN
 BEGIN:VTIMEZONE
 TZID:Europe/London
 BEGIN:DAYLIGHT
 TZOFFSETFROM:+
 RRULE:FREQ=YEARLY;BYMONTH=3;BYDAY=-1SU
 DTSTART:19810329T01
 TZNAME:BST
 TZOFFSETTO:+0100
 END:DAYLIGHT
 BEGIN:STANDARD
 TZOFFSETFROM:+0100
 RRULE:FREQ=YEARLY;BYMONTH=10;BYDAY=-1SU
 DTSTART:19961027T02
 TZNAME:GMT
 TZOFFSETTO:+
 END:STANDARD
 END:VTIMEZONE
 BEGIN:VEVENT
 CREATED:<1387007670>>> /dav/calendars/user/marty/Default/0C48ECD9-44A7-4F1F-9C87-9A2EF647C574.ics 
 HTTP/1.1
 Accept-encoding: gzip, deflate
 Max-forwards: 10
 Accept-language: en-gb
 User-agent: Mac_OS_X/10.9 (13A603) CalendarAgent/174
 Host: 192.168.253.16:1443
 Accept: */*
 Content-length: 0
 X-forwarded-server: dav.maui.co.uk
 X-forwarded-for: 176.12.107.140
 Authorization: Basic ...
 X-forwarded