Re: [MeeGo-dev] migration (back) to EDS - contacts and calendar
Hello again! (I didn't press send on Friday) here it goes now... That wasn't the intention. Instead the goal was to point out areas that'll need improvements, because that is relevant for staffing/resource planning discussions. My apologies if this necessarily subjective summary turned out to be offending. No apologies need. But anyway lets skip that part, as we won't go anywhere. If you have the time to work on better change reporting, then supporting the KCalCore CalendarObserver [1] would be useful for the QML calendar app [2]. Right now that app does not react to changes made in the background by some other app or process (like sync), which is an obvious missing feature. Adding it based on the database changed signal was under discussion, but then there never was time to actually implement it. The app doesn't react to that because the Observer that gets those modifications is the StorageObserver not the CalendarObserver. And as you mentioned before, right now there is no diff changes. So you need to reload all. To the community: if anyone is interested in improving that part of the app, with CalendarObserver or the already existing mKCal signal, please shout. I'm sure such help would be welcome. I'm copying Sirisha, the main developer of that app. [1] http://api.kde.org/4.x-api/kdepimlibs-apidocs/kcalcore/html/classKCalCore_ 1_1Calendar.html#acf6d752ffc403159f1ac8633dd07e680 [2] http://meego.gitorious.org/meego-ux/meego-app-calendar About the partial loading of mKCal. Is it faulty? Please provide the bug number. It was faulty: https://bugs.meego.com/show_bug.cgi?id=6061#c5 I still find it hard to use correctly. For example, which events need to be loaded to get a correct response for events falling into the time range 2011-03-31 00:00 Europe/Berlin until 2011-03-31 23:59 Europe/Berlin? Which API calls, which parameters? You need to create a KDateTime with those start and end times. Make sure that the storage has that range loaded storage-load(start, end); and the recurringIncidences also of course. Thanks for clarifying that. The of course part for recurring incidences is not that obvious for users of the library :-/ I have seen the call and suspected that it is needed for this purpose, but never found it spelled out explicitly. As with the previous problem, I think there are two drawbacks of mKCal. First one, Documentation. The methods are documented but there are no clear examples on how to use the apis. And the APis can be strange at the beggining due to the inheritance of the Storage and Calendar concepts from the originall KCal and extending them. (this most likely can be overcome fixing the first problem). It would also have been possible that storage-load(start, end) loads relevant recurring events. This could be implemented by calculating a bounding box (start time, end time, with end time = infinity also allowed) for recurring events and use that to determine the relevant part quickly. I think it is not that simple. Because you need to expand the recurrences to see if something from the past happens in that range. And if you move to the future, calculating those windows can be too complicated to make right. I don't know what the official position is for binary-only extensions and plugin mechanisms in MeeGo middleware. My expectation is that vendors will be allowed to modify middleware if they have specific needs not addressed by it. Even if it is too wide topic, some mechanisms should be available (at least something basic as in mKCal) to extend functionality to extend with services. And with that I am not including only close as Exchange, but if I want to use CalDav to send invitations? That should be taken into account when creating an architecture. IMHO if there are valid reasons for fixing something that is broken in KCalCore, then these fixes should be used in KDE and MeeGo. I agree with that. That can be taken into the other mailing list. Enjoy the vacation! :) It's not much of a vacation, just some days off because I didn't take enough time off last year. And here I am again enjoying my vacation by posting on meego-dev. Darn! ;-) Take it easy and read you next week :) ___ MeeGo-dev mailing list MeeGo-dev@meego.com http://lists.meego.com/listinfo/meego-dev http://wiki.meego.com/Mailing_list_guidelines
Re: [MeeGo-dev] migration (back) to EDS - contacts and calendar
On Fr, 2011-04-01 at 12:26 +0100, Mathias Hasselmann wrote: Am Dienstag, den 29.03.2011, 16:35 +0200 schrieb Patrick Ohly: http://wiki.meego.com/Architecture/planning/evolution-data-server/QtContacts_storage_plugin change notifications via ebook view: created when opened, initial data dump ignored (performance problem!), report all following changes This problem was successfully worked around for Fremantle by adding lazy vCard parsing to its EDS version: EDS transfers changed contacts as vCard string and then libebook spends a lot of time in parsing it, just to deliver a EContact from which maybe only the UID is read. I was wondering about the UID part and came to the same conclusion as you: passing the UID inline doesn't look right. Another conceptual problem with it is that EDS cannot store a true, creator-assigned UID (think iCalendar 2.0 UID property). EDS will always wipe out that real (U)UID and replace it with a local ID. vCard doesn't mandate such a globally unique ID, but for MeeGo-MeeGo sync we could take advantage of some extra knowledge about our peer - if we had a real UID, which currently we don't (in EDS - not sure about QtContacts-Tracker). Another trick would be, to introduce a new signal that only reports UIDs and let clients later fetch the contacts they are interested in, like QtContacts does. I had considered that, but I guess the actual benefit would depend a lot on the access patterns by the apps: if reporting the UID is always followed by an immediate data access by the app, then it might be better to leave the EDS [D-Bus] API as it is, cache the transmitted data in QtContacts-EDS and then report it directly when the app asks. http://wiki.meego.com/Architecture/planning/evolution-data-server/eds-improvements extend EDS query language with a flag that suppresses existing data Can you explain this idea a bit? To me it seems quite hard to do, since EDS only stores vCard strings. It has now idea about their meaning. This is for QtContacts change notifications. The way EDS works now, I have to set up a view that matches all contacts. When the view is created, EDS will send all data followed by a view-complete signal. For QtContacts, only changes *after* the view-complete are relevant. All the data would just be dropped (or cached as suggested above, but caching the whole address book won't work well). The line above is misleading: it should say suppress sending initial data dump. All following changes should be handled normally. optional parsing of data in client Nice to see lazy vCard parsing considered already, but let me comment on that section a bit. 1. If delayed parsing fails, how can error be reported. 1. EDS only stores and reports valid vCard strings. This is guaranteed by the contact saving API only accepting EContact instances, which then are converted to valid vCard via e_vcard_to_string(). 2. Upstream EDS also doesn't do any error reporting right now. It only spits warnings to the console and e_vcard_get_attributes() will return a quite terse list. Same API works with lazy parsing. So ebook is covered, or at least doesn't regress. With libical it might be a bit harder. icalcomponent_new_from_string() currently returns NULL if the item cannot be parsed; silently returning an icalcomponent with unparsed data would disable the existing error checking in apps. more efficient access to meta data for change tracking + atomic updates Sounds familiar: e_book_get_changes() SyncEvolution used that initially and moved away from it because it was too limited. For example, it is impossible to distinguish or even detect changes made by some other app while making changes yourself. No, what I would prefer are improvements for reading meta data (think HTTP PROPFIND and eTags), and then let clients of EDS implement the necessary logic that fits their use cases. contacts: store PHOTO data as plain files For apps which want to create the photo file directly: * Apps get a file descriptor and the path from a new libebook API. * They own the file until the EDS server acknowledges the successful storing of a contact with a matching PHOTO;VALUE=uri:file property. If that fails, the app must remove the file. Why such complicated API? In sync, the photo is already detached. I wanted to have some way of storing it in the right location without having to encode it inline first. Perhaps it isn't important enough to make the API as complex as it (obviously) becomes. We definitely can focus on detach inline data automatically first. EContactPhoto permits both detached and inline photos. libebook only would have to take care, of generally detaching inline photos. Minor caveat here: some EDS backends need the photo inline, like the various Groupware backends. So libebook shouldn't do it unconditionally. Trust TYPE=JPEG/PNG/... or look into file content to determine type? First
Re: [MeeGo-dev] migration (back) to EDS - contacts and calendar
On Fr, 2011-04-01 at 10:57 +0100, Mathias Hasselmann wrote: Am Dienstag, den 29.03.2011, 16:35 +0200 schrieb Patrick Ohly: Hello! Let's be more specific about identifying work which needs to be done. I've put together some thoughts here: http://wiki.meego.com/Architecture/planning/evolution-data-server http://wiki.meego.com/Architecture/planning/evolution-data-server/QtContacts_storage_plugin http://wiki.meego.com/Architecture/planning/evolution-data-server/eds-improvements Hi Patrick, Thanks alot for those wiki pages. Good approach to collect rationales in the public. Will you take care of updating the pages, or shall we do this together somehow? I'll do another pass over the pages today (after catching up with email), but then they could also be maintained together. Contact Storage Maemo solution: QtContacts (API) + QtContacts-Tracker (glue code) + Tracker (storage) EDS: QtContacts (API) + libebook (client side) + EDS (server side, storage of vCards in Berkley DB) Let's start with some nitpicking: Maybe a more neutral term than glue code can be used for QtContacts-Tracker. ;-) Yes, see my other email about that - that's an obvious (and easy to fix) bug in the Wiki page ;-} [...] Thanks for sharing these insights. Good points, and not much that I can add to that. If you look at the future Tracker can provider significantly better data protection than EDS. Tracker supports RDF graphs per resource property (EDS speak: vCard attribute). Code can be written to restrict access for each single graph. Virtual SQLite tables should do the job. If this should be implemented, you could aggregate information from different data sources into one single contact, and still could make sure, that for instance only the Addressbook itself can see information retrieved from e.g. Facebook. I'm still not sure how that'll be combined with execution of QSparql queries inside the application's own process. Once that process gets read access to the sqlite databases containing the data, it has access to all of it. Does virtual SQLite tables and RDF graphs here mean that the data would be split into different files? slow write performance in QtContacts-Tracker (?) Please give concrete test data. I don't have any. That line in the Wiki repeats hearsay, some of it directly from people who have worked on QtContacts-Tracker, and the question mark is there because I don't know how relevant that still is. I included these rumors in the Wiki because they are TODO items: we need data and use cases from the app developers that can be turned into benchmarks. Then we can remove the question marks. -- Best Regards, Patrick Ohly The content of this message is my personal opinion only and although I am an employee of Intel, the statements I make here in no way represent Intel's position on the issue, nor am I authorized to speak on behalf of Intel on this matter. ___ MeeGo-dev mailing list MeeGo-dev@meego.com http://lists.meego.com/listinfo/meego-dev http://wiki.meego.com/Mailing_list_guidelines
Re: [MeeGo-dev] migration (back) to EDS - contacts and calendar
On Mon, 04 Apr 2011 10:10:34 +0200, Patrick Ohly wrote: On Fr, 2011-04-01 at 10:57 +0100, Mathias Hasselmann wrote: Am Dienstag, den 29.03.2011, 16:35 +0200 schrieb Patrick Ohly: Hello! Let's be more specific about identifying work which needs to be done. I've put together some thoughts here: http://wiki.meego.com/Architecture/planning/evolution-data-server http://wiki.meego.com/Architecture/planning/evolution-data-server/QtContacts_storage_plugin http://wiki.meego.com/Architecture/planning/evolution-data-server/eds-improvements Hi Patrick, Thanks alot for those wiki pages. Good approach to collect rationales in the public. Will you take care of updating the pages, or shall we do this together somehow? I'll do another pass over the pages today (after catching up with email), but then they could also be maintained together. Contact Storage Maemo solution: QtContacts (API) + QtContacts-Tracker (glue code) + Tracker (storage) EDS: QtContacts (API) + libebook (client side) + EDS (server side, storage of vCards in Berkley DB) Let's start with some nitpicking: Maybe a more neutral term than glue code can be used for QtContacts-Tracker. ;-) Yes, see my other email about that - that's an obvious (and easy to fix) bug in the Wiki page ;-} [...] Thanks for sharing these insights. Good points, and not much that I can add to that. If you look at the future Tracker can provider significantly better data protection than EDS. Tracker supports RDF graphs per resource property (EDS speak: vCard attribute). Code can be written to restrict access for each single graph. Virtual SQLite tables should do the job. If this should be implemented, you could aggregate information from different data sources into one single contact, and still could make sure, that for instance only the Addressbook itself can see information retrieved from e.g. Facebook. I'm still not sure how that'll be combined with execution of QSparql queries inside the application's own process. Once that process gets read access to the sqlite databases containing the data, it has access to all of it. Does virtual SQLite tables and RDF graphs here mean that the data would be split into different files? If the process uses the DBus interface to access Tracker, then the store has control on who can query what. Direct access would be reserved for processes with access to protected data. Note that DBus access is not that slow, since actual data transfer is done over a UNIX socket. slow write performance in QtContacts-Tracker (?) Please give concrete test data. I don't have any. That line in the Wiki repeats hearsay, some of it directly from people who have worked on QtContacts-Tracker, and the question mark is there because I don't know how relevant that still is. I included these rumors in the Wiki because they are TODO items: we need data and use cases from the app developers that can be turned into benchmarks. Then we can remove the question marks. ___ MeeGo-dev mailing list MeeGo-dev@meego.com http://lists.meego.com/listinfo/meego-dev http://wiki.meego.com/Mailing_list_guidelines
Re: [MeeGo-dev] migration (back) to EDS - contacts and calendar
Am Montag, den 04.04.2011, 09:48 +0200 schrieb Patrick Ohly: On Fr, 2011-04-01 at 12:26 +0100, Mathias Hasselmann wrote: Am Dienstag, den 29.03.2011, 16:35 +0200 schrieb Patrick Ohly: http://wiki.meego.com/Architecture/planning/evolution-data-server/QtContacts_storage_plugin change notifications via ebook view: created when opened, initial data dump ignored (performance problem!), report all following changes This problem was successfully worked around for Fremantle by adding lazy vCard parsing to its EDS version: EDS transfers changed contacts as vCard string and then libebook spends a lot of time in parsing it, just to deliver a EContact from which maybe only the UID is read. I was wondering about the UID part and came to the same conclusion as you: passing the UID inline doesn't look right. Another conceptual problem with it is that EDS cannot store a true, creator-assigned UID (think iCalendar 2.0 UID property). EDS will always wipe out that real (U)UID and replace it with a local ID. vCard doesn't mandate such a globally unique ID, but for MeeGo-MeeGo sync we could take advantage of some extra knowledge about our peer - if we had a real UID, which currently we don't (in EDS - not sure about QtContacts-Tracker). Another trick would be, to introduce a new signal that only reports UIDs and let clients later fetch the contacts they are interested in, like QtContacts does. I had considered that, but I guess the actual benefit would depend a lot on the access patterns by the apps: if reporting the UID is always followed by an immediate data access by the app, then it might be better to leave the EDS [D-Bus] API as it is, cache the transmitted data in QtContacts-EDS and then report it directly when the app asks. http://wiki.meego.com/Architecture/planning/evolution-data-server/eds-improvements extend EDS query language with a flag that suppresses existing data Can you explain this idea a bit? To me it seems quite hard to do, since EDS only stores vCard strings. It has now idea about their meaning. This is for QtContacts change notifications. The way EDS works now, I have to set up a view that matches all contacts. When the view is created, EDS will send all data followed by a view-complete signal. For QtContacts, only changes *after* the view-complete are relevant. All the data would just be dropped (or cached as suggested above, but caching the whole address book won't work well). The line above is misleading: it should say suppress sending initial data dump. All following changes should be handled normally. optional parsing of data in client Nice to see lazy vCard parsing considered already, but let me comment on that section a bit. 1. If delayed parsing fails, how can error be reported. 1. EDS only stores and reports valid vCard strings. This is guaranteed by the contact saving API only accepting EContact instances, which then are converted to valid vCard via e_vcard_to_string(). 2. Upstream EDS also doesn't do any error reporting right now. It only spits warnings to the console and e_vcard_get_attributes() will return a quite terse list. Same API works with lazy parsing. So ebook is covered, or at least doesn't regress. With libical it might be a bit harder. icalcomponent_new_from_string() currently returns NULL if the item cannot be parsed; silently returning an icalcomponent with unparsed data would disable the existing error checking in apps. more efficient access to meta data for change tracking + atomic updates Sounds familiar: e_book_get_changes() SyncEvolution used that initially and moved away from it because it was too limited. For example, it is impossible to distinguish or even detect changes made by some other app while making changes yourself. AFAIR we had an solution for that in F-EDS, but have some trouble to remember right now. Guess I'd have to compare upstream and Fremantle source code. Some one mates from F-EDS times follow this thread and remember the details? contacts: store PHOTO data as plain files For apps which want to create the photo file directly: * Apps get a file descriptor and the path from a new libebook API. * They own the file until the EDS server acknowledges the successful storing of a contact with a matching PHOTO;VALUE=uri:file property. If that fails, the app must remove the file. Why such complicated API? In sync, the photo is already detached. I wanted to have some way of storing it in the right location without having to encode it inline first. Perhaps it isn't important enough to make the API as complex as it (obviously) becomes. We definitely can focus on detach inline data automatically first. EContactPhoto permits both detached and inline photos. libebook only would have to take care, of generally detaching inline photos. Minor caveat
Re: [MeeGo-dev] migration (back) to EDS - contacts and calendar
On Mon Apr 4 2011 10:48:42 AM CEST, Mathias Hasselmann math...@openismus.com wrote: Is there someone at Intel who could convince Nokia to contribute libosso-abook to Meego? So much useful code which would should not have to be written again. https://bugs.maemo.org/show_bug.cgi?id=5549 Given the new architecture direction, I think it might be valid to ask them to reconsider that open sourcing proposal. Br, Carsten Munk ___ MeeGo-dev mailing list MeeGo-dev@meego.com http://lists.meego.com/listinfo/meego-dev http://wiki.meego.com/Mailing_list_guidelines
Re: [MeeGo-dev] migration (back) to EDS - contacts and calendar
On Mo, 2011-04-04 at 09:10 +0100, Patrick Ohly wrote: On Fr, 2011-04-01 at 10:57 +0100, Mathias Hasselmann wrote: Am Dienstag, den 29.03.2011, 16:35 +0200 schrieb Patrick Ohly: Let's be more specific about identifying work which needs to be done. I've put together some thoughts here: http://wiki.meego.com/Architecture/planning/evolution-data-server http://wiki.meego.com/Architecture/planning/evolution-data-server/QtContacts_storage_plugin http://wiki.meego.com/Architecture/planning/evolution-data-server/eds-improvements Hi Patrick, Thanks alot for those wiki pages. Good approach to collect rationales in the public. Will you take care of updating the pages, or shall we do this together somehow? I'll do another pass over the pages today (after catching up with email), but then they could also be maintained together. Changes: Calendar - removed part about past experience with it, added KCalCore-EDS (empty place holder at the moment) Contact - removed the offending (and unintentional) clue code comment on QtContacts-Tracker ;-), clarified libfolks and moved it from opens to plan Email - clarified that QtMobility/QtMessaging remains the main app API, added rewrite QtMobility Messaging to use Camel, rewrite mail app to use QtMobility as another option EDS improvements - clarified the change tracking/view part I would have loved to link to the public information about each component mentioned in the Wiki, but that would take too much time right now. If anyone wants to make the pages nicer, feel free to add links. -- Best Regards, Patrick Ohly The content of this message is my personal opinion only and although I am an employee of Intel, the statements I make here in no way represent Intel's position on the issue, nor am I authorized to speak on behalf of Intel on this matter. ___ MeeGo-dev mailing list MeeGo-dev@meego.com http://lists.meego.com/listinfo/meego-dev http://wiki.meego.com/Mailing_list_guidelines
Re: [MeeGo-dev] migration (back) to EDS - contacts and calendar
On Fr, 2011-04-01 at 11:29 +0200, Patrick Ohly wrote: I beg to disagree here. The MeeGo version of KCalCore is compiled differently than the upstream version: #if !defined(KCALCORE_FOR_MEEGO) endDate = kdt.date().addDays( -1 ); #else endDate = kdt.date(); #endif Go to the kde git, and you will see those there. So the code, as I said, is from upstream. My point is that the MeeGo version of KCalCore is functionally different from the one in KDE. That the code is upstream is irrelevant as long as ifdefs are used for more fundamental changes than merely getting the code to compile on a different platform. My suspicion all along was that these defines around end date change the semantic of KCalCore. To verify that, I modified the test program from BMC #6050 such that it imports an iCalendar 2.0 all-day event and prints its start/end date after importing into KCalCore. Code attached, compile/run with: rm -rf ~/.calendar \ g++ -g test-allday-semantic.cpp `pkg-config --cflags --libs libmkcal ` \ ./a.out In MeeGo with KCalCore 4.1.18 I get start QDate(Wed Apr 5 2006) end QDate(Thu Apr 6 2006) After recompiling that version without KCALCORE_FOR_MEEGO I get: start QDate(Wed Apr 5 2006) end QDate(Wed Apr 5 2006) In other words, the patch changes the semantic of dtEnd() for applications. Upstream KDE uses dtEnd() is inclusive, the MeeGo version uses dtEnd() is exclusive. The latter is closer to iCalendar 2.0, but is that a good enough justification to break compatibility with apps ported from KDE? I don't think so. Another possible justification would be that the original semantic cannot express an empty all-day event (start == end). Again I would consider not modifying the API as more important than this cleanup. -- Best Regards, Patrick Ohly The content of this message is my personal opinion only and although I am an employee of Intel, the statements I make here in no way represent Intel's position on the issue, nor am I authorized to speak on behalf of Intel on this matter. /* * Calendar - For testing adding alarm to an event * * Copyright (c) 2010, Intel Corporation. * Author: Sirisha Muppavarapu (sirisha.muppavar...@intel.com) * * This program is licensed under the terms and conditions of the * Apache License, version 2.0. The full text of the Apache License is at * http://www.apache.org/licenses/LICENSE-2.0 * */ #include QtCore/QCoreApplication #include extendedcalendar.h #include event.h #include notebook.h #include sqlitestorage.h #include kdatetime.h #include duration.h #include icalformat.h #include QDebug const char *item = BEGIN:VCALENDAR\n PRODID:-//Ximian//NONSGML Evolution Calendar//EN\n VERSION:2.0\n BEGIN:VEVENT\n SUMMARY:all day\n DTSTART;VALUE=DATE:20060405\n DTEND;VALUE=DATE:20060406\n UID:foobar\n END:VEVENT\n END:VCALENDAR\n; int main(int argc, char *argv[]) { QCoreApplication a(argc, argv); mKCal::ExtendedCalendar::Ptr calendar(new mKCal::ExtendedCalendar(KDateTime::Spec::LocalZone())); mKCal::ExtendedStorage::Ptr storage(mKCal::ExtendedCalendar::defaultStorage(calendar)); storage-open(); storage-load(); // import event KCalCore::ICalFormat parser; if (!parser.fromString(calendar, QString::fromUtf8(item))) { qFatal(error parsing iCalendar 2.0 item); } KCalCore::ICalFormat formatter; qDebug() formatter.toString(KCalCore::Calendar::Ptr(calendar)); KCalCore::Incidence::List incidences = calendar-incidences(); qDebug() incidences; KCalCore::Event::Ptr event = incidences[0].staticCastKCalCore::Event(); qDebug() start event-dtStart().date() end event-dtEnd().date(); // MeeGo with FOR_MEEGO define: // start QDate(Wed Apr 5 2006) end QDate(Thu Apr 6 2006) // without define: // start QDate(Wed Apr 5 2006) end QDate(Wed Apr 5 2006) return 0; } ___ MeeGo-dev mailing list MeeGo-dev@meego.com http://lists.meego.com/listinfo/meego-dev http://wiki.meego.com/Mailing_list_guidelines
Re: [MeeGo-dev] migration (back) to EDS - contacts and calendar
On Fr, 2011-04-01 at 19:21 +0100, Jens Georg wrote: Let's be more specific about identifying work which needs to be done. I've put together some thoughts here: http://wiki.meego.com/Architecture/planning/evolution-data-server I'd like to clarify something from the page: Maemo solution: QtContacts (API) + QtContacts-Tracker (glue code) + Could we stop calling it glue code? It's about as much glue code as the folks backends. That's my bad. I know that QtContacts-Tracker does a significant amount of work that goes way beyond mere glue code. What I must have had in mind is QtContacts-EDS: that really is just some glue code that binds together the QtContacts, QtVersit and libebook APIs. -- Best Regards, Patrick Ohly The content of this message is my personal opinion only and although I am an employee of Intel, the statements I make here in no way represent Intel's position on the issue, nor am I authorized to speak on behalf of Intel on this matter. ___ MeeGo-dev mailing list MeeGo-dev@meego.com http://lists.meego.com/listinfo/meego-dev http://wiki.meego.com/Mailing_list_guidelines
Re: [MeeGo-dev] migration (back) to EDS - contacts and calendar
On Sat, 2011-04-02 at 12:34 +0200, Patrick Ohly wrote: Could we stop calling it glue code? It's about as much glue code as the folks backends. That's my bad. I know that QtContacts-Tracker does a significant amount of work that goes way beyond mere glue code. Hm, and my sentence probably sounds way ruder than it was intended (not rude at all, but with a smile), sorry for that. ___ MeeGo-dev mailing list MeeGo-dev@meego.com http://lists.meego.com/listinfo/meego-dev http://wiki.meego.com/Mailing_list_guidelines
Re: [MeeGo-dev] migration (back) to EDS - contacts and calendar
Hello Alvaro! Let me reply on the technical aspects below. I've skipped over the MeeGo direction and decision making questions, because those belong elsewhere and I don't want to get drawn into that. On Do, 2011-03-31 at 13:34 +0100, Alvaro Manera wrote: * Only one coarse change method. There is not fine diff notifications. I agree, this can be improved, and I think it would be quite easy to implement. But was it requested anywhere? No. I was merely comparing the current status. Then it is quite unfair to do so unless you want to bias the readers. That wasn't the intention. Instead the goal was to point out areas that'll need improvements, because that is relevant for staffing/resource planning discussions. My apologies if this necessarily subjective summary turned out to be offending. If you have the time to work on better change reporting, then supporting the KCalCore CalendarObserver [1] would be useful for the QML calendar app [2]. Right now that app does not react to changes made in the background by some other app or process (like sync), which is an obvious missing feature. Adding it based on the database changed signal was under discussion, but then there never was time to actually implement it. To the community: if anyone is interested in improving that part of the app, with CalendarObserver or the already existing mKCal signal, please shout. I'm sure such help would be welcome. I'm copying Sirisha, the main developer of that app. [1] http://api.kde.org/4.x-api/kdepimlibs-apidocs/kcalcore/html/classKCalCore_1_1Calendar.html#acf6d752ffc403159f1ac8633dd07e680 [2] http://meego.gitorious.org/meego-ux/meego-app-calendar About the partial loading of mKCal. Is it faulty? Please provide the bug number. It was faulty: https://bugs.meego.com/show_bug.cgi?id=6061#c5 I still find it hard to use correctly. For example, which events need to be loaded to get a correct response for events falling into the time range 2011-03-31 00:00 Europe/Berlin until 2011-03-31 23:59 Europe/Berlin? Which API calls, which parameters? You need to create a KDateTime with those start and end times. Make sure that the storage has that range loaded storage-load(start, end); and the recurringIncidences also of course. Thanks for clarifying that. The of course part for recurring incidences is not that obvious for users of the library :-/ I have seen the call and suspected that it is needed for this purpose, but never found it spelled out explicitly. It would also have been possible that storage-load(start, end) loads relevant recurring events. This could be implemented by calculating a bounding box (start time, end time, with end time = infinity also allowed) for recurring events and use that to determine the relevant part quickly. And the same goes to the next question. How is it planned to enable vendors to extend the current solution? I think it is a very valid point that should be addressed. I don't know what the official position is for binary-only extensions and plugin mechanisms in MeeGo middleware. My expectation is that vendors will be allowed to modify middleware if they have specific needs not addressed by it. I beg to disagree here. The MeeGo version of KCalCore is compiled differently than the upstream version: #if !defined(KCALCORE_FOR_MEEGO) endDate = kdt.date().addDays( -1 ); #else endDate = kdt.date(); #endif Go to the kde git, and you will see those there. So the code, as I said, is from upstream. My point is that the MeeGo version of KCalCore is functionally different from the one in KDE. That the code is upstream is irrelevant as long as ifdefs are used for more fundamental changes than merely getting the code to compile on a different platform. And most of them are IOP issues that we have found that are not 100% standard but we believe that need to be supported anyway. There are only two as the one you have pointed, and most likely can be fixed somewhere else. Are you saying that the MeeGo version supports non-standard behavior that KDE upstream does not want to support? Can you give an example? Attached is a patch which removes all KCALCORE_FOR_MEEGO ifdefs in favor of using the code in KDE upstream. What would break if this patch was used in MeeGo (app API, storage API, exported/imported vCalendar/iCalendar)? I looked briefly at commit messages to find an answer for that, but didn't get far. One of these ifdefs (the N900 and s60-phones need enddate) was already in the initial commit, so the justification for this change isn't public. Another line in that code was modified later, but the commit message isn't helpful either: commit b7d1d6032b5599320523d597ae2c4e84efc37bc4 Date: Sat Dec 4 16:33:55 2010 +0200 Fixes: NB#206351 - [Dali][PCCS] Special characters are not transferred properly from device (Bugzilla) IMHO if there are valid reasons for fixing something that is broken in KCalCore,
Re: [MeeGo-dev] migration (back) to EDS - contacts and calendar
On Do, 2011-03-31 at 14:38 +0100, mikhail.zabal...@nokia.com wrote: Hi, -Original Message- From: meego-dev-boun...@meego.com [mailto:meego-dev-boun...@meego.com] On Behalf Of ext Patrick Ohly Sent: Wednesday, March 30, 2011 9:57 AM To: virot...@viroteck.net Cc: meego-dev@meego.com Subject: Re: [MeeGo-dev] migration (back) to EDS - contacts and calendar * In the past, meego.com was a dumping ground for technology developed and used elsewhere. Often it wasn't properly tested there. I have heard arguments that this testing can't be done without the apps in MeeGo using the middleware. IMHO this is just a lame excuse for not doing proper QA on middleware compoments, and leads to situations as with Dialer/libcommhistory were an app developer would have to stop working on his main task to investigate problems in someone else's code. Not cool. If no bugs were filed and no questions asked by mail, any efforts made to take that technology into use are indistinguishable from not even trying. Trust me, I have tried. I filed bugs and exchanged emails. When that didn't help because my peers didn't have time to reply, I escalated to managers and participated in meetings, just to get some attention to these problems. A lot of work for very little gain, and I don't think any app developers deserves to be blamed for not investing a similar effort when running into issues. I hope collaborative processes in MeeGo will be more transparent in the future. I hope so too. -- Best Regards, Patrick Ohly The content of this message is my personal opinion only and although I am an employee of Intel, the statements I make here in no way represent Intel's position on the issue, nor am I authorized to speak on behalf of Intel on this matter. ___ MeeGo-dev mailing list MeeGo-dev@meego.com http://lists.meego.com/listinfo/meego-dev http://wiki.meego.com/Mailing_list_guidelines
Re: [MeeGo-dev] migration (back) to EDS - contacts and calendar
Am Dienstag, den 29.03.2011, 16:35 +0200 schrieb Patrick Ohly: Hello! Let's be more specific about identifying work which needs to be done. I've put together some thoughts here: http://wiki.meego.com/Architecture/planning/evolution-data-server http://wiki.meego.com/Architecture/planning/evolution-data-server/QtContacts_storage_plugin http://wiki.meego.com/Architecture/planning/evolution-data-server/eds-improvements Hi Patrick, Thanks alot for those wiki pages. Good approach to collect rationales in the public. Will you take care of updating the pages, or shall we do this together somehow? Contact Storage Maemo solution: QtContacts (API) + QtContacts-Tracker (glue code) + Tracker (storage) EDS: QtContacts (API) + libebook (client side) + EDS (server side, storage of vCards in Berkley DB) Let's start with some nitpicking: Maybe a more neutral term than glue code can be used for QtContacts-Tracker. ;-) Advantages of Tracker: supports potential future use cases (combining data from different categories, partial reading/writing, fine-grained annotations on data origin) This will become very useful when implementation communication history. For Maemo5 we used a detached, custom sqlite database for storing communication history. Bringing pieces together, keeping them in sync, avoiding problems from concurrent access, getting reasonable response times and memory consumptions where a permanent problem with that solution. By using tracker as data store, we get the data aggregation for free, and we have to solve most concurency and performance problems at justone single place. Similar things apply for presence information. It might feel strange to put such information into a data store, but Tracker permits marking RDF properties as volatile. So presence information should not hit the disk if desired. Reminds me I have to push the Tracker guys harder to finally repair transient property support. better searching scalability (?) When aggregating information from multiple, independent, incompatible databases you'll face several scalability problems: 1. You get the I/O overhead for multiple databases 2. You get concurency management overhead for multiple databases 3. Most importantly you get quite some memory overhead, since most likely eachapplication interested in that data aggregation will have to keep the aggregated data in memory. Since aggregation takes time, and dialogs shall be shown within milliseconds, they most probably will aggregate the data ahead and keep it around during lifetime. One might believe only few applications need that aggregated data, but both in Fremantle and now also in Harmattan we have dozens if not more applications using a contact picker and therefore using aggregated data. It was quite a memory usage hazard in Fremantle. read performance (?) Reading performance of both solutions should be compareable for fetch-all. At this moment tracker should show better reading performance when using (not so complex) filters, as it stores contacts decomposed. One can add indexes for vCard attributes to EDS. Together with lazy vCard parsing in libebook this should give compareable reading performance. Disadvantages of Tracker: data protection missing in both EDS and Tracker, but less obvious how to implement it in Tracker The solution proposed for EDS is protecting its database files. Same solution can be applied to tracker. Since you can deal without data aggregation at this moment, you should not loose anything if we create a separate Tracker database for contacts and apply same file level protection as for EDS. If you look at the future Tracker can provider significantly better data protection than EDS. Tracker supports RDF graphs per resource property (EDS speak: vCard attribute). Code can be written to restrict access for each single graph. Virtual SQLite tables should do the job. If this should be implemented, you could aggregate information from different data sources into one single contact, and still could make sure, that for instance only the Addressbook itself can see information retrieved from e.g. Facebook. Considering privace concerns this is a very significant feature, which cannot be reasonable implemented with EDS - AFAIK. slow write performance in QtContacts-Tracker (?) Please give concrete test data. Like Philip Van Hoof demonstrated, it rather seems writing performance of EDS and Tracker seems to be comparable - which is not surprising that I/O is the limiting factor. With Tracker's recent addition of sparql-update (INSERT OR REPLACE statement) we have seen performance improvments of up to 25%. This would set QtContactsTracker ahead of EDS. Also note that the filebackend of EDS doesn't seem to implement vcard attribute indexes yet. Adding them to EDS to get search performance comparable to Tracker would add another performance penalty to EDS. Plan: write a QtContacts storage plugin (details on QtContacts-EDS),
Re: [MeeGo-dev] migration (back) to EDS - contacts and calendar
Now after criticizing the comparison page something constructive information for the EDS plan. First of all it should be noted, that the QtContacts API was designed after careful review of the libebook API and considering experience gained during Fremantle. Therefore QtContacts can be seen as some kind of descendant of EDS, trying to avoid some of its faults. Am Dienstag, den 29.03.2011, 16:35 +0200 schrieb Patrick Ohly: http://wiki.meego.com/Architecture/planning/evolution-data-server/QtContacts_storage_plugin change notifications via ebook view: created when opened, initial data dump ignored (performance problem!), report all following changes This problem was successfully worked around for Fremantle by adding lazy vCard parsing to its EDS version: EDS transfers changed contacts as vCard string and then libebook spends a lot of time in parsing it, just to deliver a EContact from which maybe only the UID is read. We changed the signal to deliver both the UID and the vCard string for each contact and patched EVCard (base class of EContact) to only store a copy of both. The vCard string itself only would be parsed when vCard attributes are read for the first time. After parsing the now obsolete vCard string is discarded. This gave us a really nice performance boost. From something likes 14 seconds to process all notifications, down to some hundred milliseconds. This trick works because usually only very few contacts are shown at a time and therefore only very few vCards must be parsed initially. Another trick would be, to introduce a new signal that only reports UIDs and let clients later fetch the contacts they are interested in, like QtContacts does. http://wiki.meego.com/Architecture/planning/evolution-data-server/eds-improvements extend EDS query language with a flag that suppresses existing data Can you explain this idea a bit? To me it seems quite hard to do, since EDS only stores vCard strings. It has now idea about their meaning. optional parsing of data in client Nice to see lazy vCard parsing considered already, but let me comment on that section a bit. 1. If delayed parsing fails, how can error be reported. 1. EDS only stores and reports valid vCard strings. This is guaranteed by the contact saving API only accepting EContact instances, which then are converted to valid vCard via e_vcard_to_string(). 2. Upstream EDS also doesn't do any error reporting right now. It only spits warnings to the console and e_vcard_get_attributes() will return a quite terse list. Same API works with lazy parsing. more efficient access to meta data for change tracking + atomic updates Sounds familiar: e_book_get_changes() contacts: store PHOTO data as plain files For apps which want to create the photo file directly: * Apps get a file descriptor and the path from a new libebook API. * They own the file until the EDS server acknowledges the successful storing of a contact with a matching PHOTO;VALUE=uri:file property. If that fails, the app must remove the file. Why such complicated API? EContactPhoto permits both detached and inline photos. libebook only would have to take care, of generally detaching inline photos. This writing should be done using async API to avoid UI blocking. Another approach might be something like e_vcard_detach_photo(). This TBD function would explicitly detaches inline photos, any maybe directly return the generated filename. Trust TYPE=JPEG/PNG/... or look into file content to determine type? First rule of vCard survival: Never trust any vCard. Those you get are malformed crap, lying about their charsets and image file formats. Content inspection is needed to figure out the image type. While scratching that topic: You also want to have a smart charset detected for vCards not passing UTF-8 validation. They will be in any random encoding ever invented. For Fremantle we have integrated Mozilla's charset detector [1][2]. As I learned just yesterday it seems MeegoTouch also features MCharsetDetector. Maybe that code is even more useful - have not tested it yet. Since libosso-abook already has most of the code it might be worth to take this last chance and ask Nokia for source code of libosso-abook. It's of little value for Nokia now, but would be a great contribution for future development of Meego on its way back to EDS. We could reopen the related Maemo.org bug[3] or maybe someone has ideas for even more efficient channels. Ciao, Mathias [1] https://maemo.gitorious.org/eds-fremantle/eds-fremantle/blobs/master/addressbook/libebook-dbus/e-uchardet.cpp [2] https://maemo.gitorious.org/eds-fremantle/eds-fremantle/trees/master/libuchardet [3] https://bugs.maemo.org/show_bug.cgi?id=5549 -- Mathias Hasselmann math...@openismus.com http://openismus.com/ ___ MeeGo-dev mailing list MeeGo-dev@meego.com http://lists.meego.com/listinfo/meego-dev
Re: [MeeGo-dev] migration (back) to EDS - contacts and calendar
On Fri, 2011-04-01 at 11:57 +0200, Mathias Hasselmann wrote: Am Dienstag, den 29.03.2011, 16:35 +0200 schrieb Patrick Ohly: [CUT] This will become very useful when implementation communication history. For Maemo5 we used a detached, custom sqlite database for storing communication history. Bringing pieces together, keeping them in sync, avoiding problems from concurrent access, getting reasonable response times and memory consumptions where a permanent problem with that solution. By using tracker as data store, we get the data aggregation for free, and we have to solve most concurency and performance problems at justone single place. Similar things apply for presence information. It might feel strange to put such information into a data store, but Tracker permits marking RDF properties as volatile. So presence information should not hit the disk if desired. Reminds me I have to push the Tracker guys harder to finally repair transient property support. We have transient property support. The reason why it's not using memory but instead is using the normal meta.db is because with direct-access you have multiple processes connecting to the same .db file. When you do this, you can't use SQLite's TEMPORARY and still share the data between different processes. TEMPORARY in SQLite is per process. We tried using a transient.db in /dev/shm and then ATTACH-ing that .db file in SQLite for each process that connects, but this was several times slower for a yet unknown reason. We still have this code somewhere in a branch, but because it was slower we decided not to use it. We do delete the last-session values of transient properties at startup of tracker-store and we don't journal values of transient properties. Given that meta.db is in .cache (and is considered cache), nothing is written in the persistent data (in .local) when writing transient props. Backup also doesn't back up values of persistent properties nor will consequentially restore restore them. We don't have many properties set as tracker:transient, though. Only a few IM related ones. We don't believe that I/O introduced by these is going to be problematic in any way. Disadvantages of Tracker: data protection missing in both EDS and Tracker, but less obvious how to implement it in Tracker The solution proposed for EDS is protecting its database files. Same solution can be applied to tracker. Since you can deal without data aggregation at this moment, you should not loose anything if we create a separate Tracker database for contacts and apply same file level protection as for EDS. It's by the way very easy to start from libtracker-data and implement your own tracker-store using its own SQLite database and etc. This should be doable in a few lines of code, and in Vala too (like tracker-store itself is). You can let that one use its own D-Bus name or even let it use its own special IPC mechanism. And protect all this as much as you want. The effort to do this is by no means larger than the effort for adding some sort of protection to EDS, the results are the same. In Harmattan the database of Tracker is already protected using UNIX file permissions and Aegis (if processes aren't in the group metadata-users, which only Aegis can grant to them, then they can't access meta.db - then they must go over D-Bus to do queries). Note that with Aegis is also D-Bus protectable. If you look at the future Tracker can provider significantly better data protection than EDS. Tracker supports RDF graphs per resource property (EDS speak: vCard attribute). Code can be written to restrict access for each single graph. Virtual SQLite tables should do the job. If this should be implemented, you could aggregate information from different data sources into one single contact, and still could make sure, that for instance only the Addressbook itself can see information retrieved from e.g. Facebook. Considering privace concerns this is a very significant feature, which cannot be reasonable implemented with EDS - AFAIK. Not trivial to implement, this idea, but mostly correct. Yes. slow write performance in QtContacts-Tracker (?) Please give concrete test data. Like Philip Van Hoof demonstrated, it rather seems writing performance of EDS and Tracker seems to be comparable - which is not surprising that I/O is the limiting factor. With Tracker's recent addition of sparql-update (INSERT OR REPLACE statement) we have seen performance improvments of up to 25%. This would set QtContactsTracker ahead of EDS. My tests are indeed showing that Tracker is beating EDS in every field: Storage speed, a lot more flexible query capability and query speed. In every sense is Tracker scaling better than EDS. These are verifiable and reproduceable measurements and the code of the tests for Tracker is available in Tracker's repository (the EDS tests are available in the mailing list archives). On a FS that has a slow fsync, EDS is
Re: [MeeGo-dev] migration (back) to EDS - contacts and calendar
On Fr, 2011-04-01 at 14:04 +0100, Philip Van Hoof wrote: We don't have many properties set as tracker:transient, though. Only a few IM related ones. We don't believe that I/O introduced by these is going to be problematic in any way. What about wear on flash storage? One of the BKM for flash-based file systems is to mount with relatime, to avoid one write on each file access. Just wondering. I suspect that the order of magnitude (in terms of writes per time) of transient property changes is less than number of files read, but I'm really just guessing. -- Best Regards, Patrick Ohly The content of this message is my personal opinion only and although I am an employee of Intel, the statements I make here in no way represent Intel's position on the issue, nor am I authorized to speak on behalf of Intel on this matter. ___ MeeGo-dev mailing list MeeGo-dev@meego.com http://lists.meego.com/listinfo/meego-dev http://wiki.meego.com/Mailing_list_guidelines
Re: [MeeGo-dev] migration (back) to EDS - contacts and calendar
On Tue, 2011-03-29 at 20:47 +0100, Robin Burchell wrote: As far as my understanding of the technologies is, libfolks isn't related to libcommhistory anyway. It's more similar to contactsd / qtcontacts, in that it fetches contacts from multiple sources (like Telepathy) and stuff. Right - Folks is meant for meta-contacts and their immediate attributes. Associated non-trivial data (eg, pictures containing the person, communication history) should be exposed by a library that correlates FolksIndividuals with that particular data. The main reason for this is to keep Folks lean and focused on doing one thing well. More core functionality and sources are on their way. Regards, -Travis ___ MeeGo-dev mailing list MeeGo-dev@meego.com http://lists.meego.com/listinfo/meego-dev http://wiki.meego.com/Mailing_list_guidelines
Re: [MeeGo-dev] migration (back) to EDS - contacts and calendar
Hi Patrick, Thanks for putting these wiki pages together. Let's be more specific about identifying work which needs to be done. I've put together some thoughts here: http://wiki.meego.com/Architecture/planning/evolution-data-server I'd like to clarify something from the page: Maemo solution: QtContacts (API) + QtContacts-Tracker (glue code) + Tracker (storage) EDS: QtContacts (API) + libebook (client side) + EDS (server side, storage of vCards in Berkley DB) Used by: ... libfolks QtFolks is an implementation of a QContactManagerEngine that sources its contacts from libfolks. Libfolks doesn't sit on top of QtContacts. So, an application using QtContacts has this layout: App - QtContacts - QtFolks - libfolks - {Folks backends} {Folks backends}: folks-telepathy - Telepathy folks-tracker - Tracker folks-libsocialweb - Libsocialweb ... Regards, -Travis ___ MeeGo-dev mailing list MeeGo-dev@meego.com http://lists.meego.com/listinfo/meego-dev http://wiki.meego.com/Mailing_list_guidelines
Re: [MeeGo-dev] migration (back) to EDS - contacts and calendar
Let's be more specific about identifying work which needs to be done. I've put together some thoughts here: http://wiki.meego.com/Architecture/planning/evolution-data-server I'd like to clarify something from the page: Maemo solution: QtContacts (API) + QtContacts-Tracker (glue code) + Could we stop calling it glue code? It's about as much glue code as the folks backends. folks-telepathy - Telepathy folks-tracker - Tracker folks-libsocialweb - Libsocialweb ... Out of interest - since I've not really actively following the folks development - does the aggregation happen completely in memory? So each application who wants to do aggregating has to do it on its own all over again? I'm thinking of the memory implications here ... ___ MeeGo-dev mailing list MeeGo-dev@meego.com http://lists.meego.com/listinfo/meego-dev http://wiki.meego.com/Mailing_list_guidelines
Re: [MeeGo-dev] migration (back) to EDS - contacts and calendar
On Fri, 2011-04-01 at 21:21 +0300, Jens Georg wrote: Let's be more specific about identifying work which needs to be done. I've put together some thoughts here: http://wiki.meego.com/Architecture/planning/evolution-data-server I'd like to clarify something from the page: Maemo solution: QtContacts (API) + QtContacts-Tracker (glue code) + Could we stop calling it glue code? It's about as much glue code as the folks backends. Sorry - I was quoting the wiki page. I didn't intend to diminish qtcontacts-tracker. folks-telepathy - Telepathy folks-tracker - Tracker folks-libsocialweb - Libsocialweb ... Out of interest - since I've not really actively following the folks development - does the aggregation happen completely in memory? So each application who wants to do aggregating has to do it on its own all over again? I'm thinking of the memory implications here ... Right. We've got plans to add lazy-loading for contact attributes and search-based retrieval to significantly reduce the overhead here. It hadn't been necessary until this point. Regards, -Travis ___ MeeGo-dev mailing list MeeGo-dev@meego.com http://lists.meego.com/listinfo/meego-dev http://wiki.meego.com/Mailing_list_guidelines
Re: [MeeGo-dev] migration (back) to EDS - contacts and calendar
On Tuesday, March 29, 2011 05:35:04 pm ext Patrick Ohly wrote: Hello! Let's be more specific about identifying work which needs to be done. I've put together some thoughts here: http://wiki.meego.com/Architecture/planning/evolution-data-server http://wiki.meego.com/Architecture/planning/evolution-data-server/QtContact s_storage_plugin http://wiki.meego.com/Architecture/planning/evolution-data-server/eds-impr ovements The first page does include a very simplistic comparison of the different options. I thought about leaving out that part altogether, but decided otherwise because I believe that listing some of the pros and cons in public is useful. No flame wars, please. For contacts and calendar it is fairly obvious where the pain points are, so let's focus on that for now. I'm interested in getting feedback on the proposed EDS improvements. At this point, I have the QtContacts-EDS contact manager working for reading and writing contacts, as outlined in the page linked to above. Change notifications are missing. It's just proof of concept (non-)quality. I suspect that I need to get open source approval first before publishing it somewhere. I have not done any prototyping yet with KCalCore, but I expect that similar improvements as for contacts will be useful for that work, too. Hi, So far the calendar area hasn't been discussed, but I would like to comment on a few things... In the wiki it says: Disavantages of mKCal. * Only one coarse change method. There is not fine diff notifications. I agree, this can be improved, and I think it would be quite easy to implement. But was it requested anywhere? You are the promoter of FeatureZilla, where is the link? Nobody has ever heard of this feature request. * iCal support was faulty. Well yes it had some bugs, but most of them are closed only two are open and one assigned to you. Is the bug history a reason to dich a project? Has EDS a better history? This is FUD and I think having bugs and fixing them is a healthy symptom of a project. Where are the EDS advantages in the wiki? Because the problems of the current solution are stated. But why is eds better? Where are the reasons? About the partial loading of mKCal. Is it faulty? Please provide the bug number. Or is it FUD again? And as it is said in the wiki, eds holds everything into memory. I find this a problem. What happens if a manager type person with an average 4-6 events per day, and a two year calendar? (Roughly 54 weeks * 5days/week * 6 events = 1620 events. Do you need to keep all this in memory? Nowadays memory is cheap, so we could argue that. But, on every startup you have to read one iCalendar text file (as said on the wiki) with all that content?? So how long do you think the startup will be? mKcal also has a quite flexible plugin invitations handling. How do you plan to integrate the new backend with exchange or any other service? And with Buteo also, there was a way to add sync services. How is it planned now? Continuing with the statements on the wiki. As I said before KCalCore in meego is from upstream. The modifications are the patches we have implemented that haven't been sent to kde's git yet. (or the ones which hasn't been merged yet from their git). So again FUD. Your proposal says that you would use KCalCore for the iCal parsing and use it from EDS. Wasn't it said above that the iCal parsing was faulty? Concluding this email... As it has said before, there are no technical reasons to back up the decision of changing backeds. From all the stated ones, only one is valid and it could be implemented (isn't it that easier that creating something new?). Everything else is pure FUD to hide a political decision and change a project not invented here for another one that it is. Please provide real facts why the change is made. Álvaro ___ MeeGo-dev mailing list MeeGo-dev@meego.com http://lists.meego.com/listinfo/meego-dev http://wiki.meego.com/Mailing_list_guidelines
Re: [MeeGo-dev] migration (back) to EDS - contacts and calendar
On Do, 2011-03-31 at 07:56 +0100, Alvaro Manera wrote: On Tuesday, March 29, 2011 05:35:04 pm ext Patrick Ohly wrote: So far the calendar area hasn't been discussed, but I would like to comment on a few things... In the wiki it says: Disavantages of mKCal. Okay, I knew it was a mistake to include such a comparison. Now that it is public, let's discuss it. * Only one coarse change method. There is not fine diff notifications. I agree, this can be improved, and I think it would be quite easy to implement. But was it requested anywhere? No. I was merely comparing the current status. * iCal support was faulty. Well yes it had some bugs, but most of them are closed only two are open and one assigned to you. Is the bug history a reason to ditch a project? It certainly leaves a bad impression that fixing bugs like https://bugs.meego.com/show_bug.cgi?id=6050 took 8 months and several false bug closed status changes (https://bugs.meego.com/show_activity.cgi?id=6050) before the fix eventually arrived in MeeGo. Now we all know that the situation for fixing bugs in MeeGo was difficult in the past. But can we be sure that it will be better in the future? Where are the EDS advantages in the wiki? Because the problems of the current solution are stated. But why is eds better? Where are the reasons? I'm not saying that EDS is necessarily better. It's better understood, we know how to fix it if we run into problems. That's all, but that matters. About the partial loading of mKCal. Is it faulty? Please provide the bug number. It was faulty: https://bugs.meego.com/show_bug.cgi?id=6061#c5 I still find it hard to use correctly. For example, which events need to be loaded to get a correct response for events falling into the time range 2011-03-31 00:00 Europe/Berlin until 2011-03-31 23:59 Europe/Berlin? Which API calls, which parameters? mKcal also has a quite flexible plugin invitations handling. How do you plan to integrate the new backend with exchange or any other service? Valid point, but not relevant yet. There is no Exchange support in MeeGo. The UIs on meego.com are barely able to handle basic calendar and email tasks. Handling meeting invitations is out of scope right now. Yes, it is a pity that the more capable UIs taking full advantage of mKCal and QMF capabilities are not in meego.com or the situation would be a lot different. And with Buteo also, there was a way to add sync services. How is it planned now? Who needs that? Let me know and I'll add it to SyncEvolution. I've said it before: MeeGo now needs to focus on the things for which there is a real need. Everything else just binds resources which are desperately needed elsewhere. Continuing with the statements on the wiki. As I said before KCalCore in meego is from upstream. The modifications are the patches we have implemented that haven't been sent to kde's git yet. (or the ones which hasn't been merged yet from their git). So again FUD. I beg to disagree here. The MeeGo version of KCalCore is compiled differently than the upstream version: #if !defined(KCALCORE_FOR_MEEGO) endDate = kdt.date().addDays( -1 ); #else endDate = kdt.date(); #endif It's minor, but I suspect that if the upstream KCalCore was used in MeeGo, it would break mKCal. I also don't see how upstream would accept such a patch or if they did, why MeeGo should have a different semantic than the upstream KDE version. Your proposal says that you would use KCalCore for the iCal parsing and use it from EDS. Wasn't it said above that the iCal parsing was faulty? Yes, and the ifdef above is part of the fix for one of these bugs. My hope is that once upstream KCalCore is used, it'll have no iCalendar handling issues. Everything else is pure FUD to hide a political decision and change a project not invented here for another one that it is. Minor correction: EDS was not invented at Intel or for Moblin/MeeGo. I'm also not trying to hide a political decision. I was told to investigate EDS for MeeGo, and that's what I am doing. If you disagree with that direction, then please talk to those who decided to go that way. That decision was made way above my head, so talking with me about it will gain you nothing except some clarifications about technical aspects. rantIn case folks on the list still haven't noticed, I'm sticking my neck out here because I believe that discussions should be open. So far the result is definitely not going to encourage others to do the same. Oh, and I'm on vacation today, I shouldn't even be reading your emails, much less responding to them./rant -- Best Regards, Patrick Ohly The content of this message is my personal opinion only and although I am an employee of Intel, the statements I make here in no way represent Intel's position on the issue, nor am I authorized to speak on behalf of Intel on this matter. ___ MeeGo-dev mailing list
Re: [MeeGo-dev] migration (back) to EDS - contacts and calendar
* Only one coarse change method. There is not fine diff notifications. I agree, this can be improved, and I think it would be quite easy to implement. But was it requested anywhere? No. I was merely comparing the current status. Then it is quite unfair to do so unless you want to bias the readers. * iCal support was faulty. Well yes it had some bugs, but most of them are closed only two are open and one assigned to you. Is the bug history a reason to ditch a project? It certainly leaves a bad impression that fixing bugs like https://bugs.meego.com/show_bug.cgi?id=6050 took 8 months and several false bug closed status changes (https://bugs.meego.com/show_activity.cgi?id=6050) before the fix eventually arrived in MeeGo. Now we all know that the situation for fixing bugs in MeeGo was difficult in the past. But can we be sure that it will be better in the future? I also believe that many things can be done to improve that. I'm not saying that EDS is necessarily better. It's better understood, we know how to fix it if we run into problems. That's all, but that matters. Quite understandable, but isn't it that we are trying to create a platform? Why then are we sticking to a solution that is not the best one? Wouldn't it be better trying to get a nice solution? And with this I am not saying that mKcal is the best one. If there is a better option, taking into account functionality and time, lets go for it. About the partial loading of mKCal. Is it faulty? Please provide the bug number. It was faulty: https://bugs.meego.com/show_bug.cgi?id=6061#c5 I still find it hard to use correctly. For example, which events need to be loaded to get a correct response for events falling into the time range 2011-03-31 00:00 Europe/Berlin until 2011-03-31 23:59 Europe/Berlin? Which API calls, which parameters? You need to create a KDateTime with those start and end times. Make sure that the storage has that range loaded storage-load(start, end); and the recurringIncidences also of course. You can query the calendar, for those dates also. calendar-incidences(start, end); And then you ExtendedCalendar::ExpandedIncidenceList expanded = calendar-expandRecurrences(newNotes, KDateTime(start), KDateTime(end), numberDays, hitLimit); To get the full expanded set. mKcal also has a quite flexible plugin invitations handling. How do you plan to integrate the new backend with exchange or any other service? Valid point, but not relevant yet. There is no Exchange support in MeeGo. The UIs on meego.com are barely able to handle basic calendar and email tasks. Handling meeting invitations is out of scope right now. Yes, it is a pity that the more capable UIs taking full advantage of mKCal and QMF capabilities are not in meego.com or the situation would be a lot different. Well I think it is quite relevant, that even if there is no Exchange in MeeGo we want to have the enablers so vendors can extend the core features. And the same goes to the next question. How is it planned to enable vendors to extend the current solution? I think it is a very valid point that should be addressed. I beg to disagree here. The MeeGo version of KCalCore is compiled differently than the upstream version: #if !defined(KCALCORE_FOR_MEEGO) endDate = kdt.date().addDays( -1 ); #else endDate = kdt.date(); #endif Go to the kde git, and you will see those there. So the code, as I said, is from upstream. And most of them are IOP issues that we have found that are not 100% standard but we believe that need to be supported anyway. There are only two as the one you have pointed, and most likely can be fixed somewhere else. Yes, and the ifdef above is part of the fix for one of these bugs. My hope is that once upstream KCalCore is used, it'll have no iCalendar handling issues. Wishful thinking. And you can easily try building your own copy of KcalCore without that flag and see for yourself. But at least the two open bugs, won't be affected at all. Minor correction: EDS was not invented at Intel or for Moblin/MeeGo. I'm also not trying to hide a political decision. I was told to investigate EDS for MeeGo, and that's what I am doing. If you disagree with that direction, then please talk to those who decided to go that way. That decision was made way above my head, so talking with me about it will gain you nothing except some clarifications about technical aspects. I know that wasn't you, and I think it is a waste of time to try to talk to the one who made the change. But I believe that in an Open Project, doing the changes how they have been done it is not the way to go. And the technical aspects is what I want to hear. But so far, except very few based complaints about mKCal, everything else was far from
Re: [MeeGo-dev] migration (back) to EDS - contacts and calendar
On Do, 2011-03-31 at 10:59 +0200, Patrick Ohly wrote: On Do, 2011-03-31 at 07:56 +0100, Alvaro Manera wrote: On Tuesday, March 29, 2011 05:35:04 pm ext Patrick Ohly wrote: * iCal support was faulty. Well yes it had some bugs, but most of them are closed only two are open and one assigned to you. Forgot to comment on that: https://bugs.meego.com/show_bug.cgi?id=6049 is the meta bug where I described how to test iCalendar 2.0 import/export in MeeGo, before filing individual bug reports about every problem found that way. I'll close it once #6054 is resolved. In #6054 I asked about unexpected behavior in mKCal, which hasn't been answered yet. About the partial loading of mKCal. Is it faulty? Please provide the bug number. It was faulty: https://bugs.meego.com/show_bug.cgi?id=6061#c5 I still find it hard to use correctly. For example, which events need to be loaded to get a correct response for events falling into the time range 2011-03-31 00:00 Europe/Berlin until 2011-03-31 23:59 Europe/Berlin? Which API calls, which parameters? Incidentally, that's the same question that I asked before in 6061#c5, without getting a reply. I'm still interested in the answer, so I hope we can discuss it now here on the list. -- Best Regards, Patrick Ohly The content of this message is my personal opinion only and although I am an employee of Intel, the statements I make here in no way represent Intel's position on the issue, nor am I authorized to speak on behalf of Intel on this matter. ___ MeeGo-dev mailing list MeeGo-dev@meego.com http://lists.meego.com/listinfo/meego-dev http://wiki.meego.com/Mailing_list_guidelines
Re: [MeeGo-dev] migration (back) to EDS - contacts and calendar
Hi, -Original Message- From: meego-dev-boun...@meego.com [mailto:meego-dev-boun...@meego.com] On Behalf Of ext Patrick Ohly Sent: Wednesday, March 30, 2011 9:57 AM To: virot...@viroteck.net Cc: meego-dev@meego.com Subject: Re: [MeeGo-dev] migration (back) to EDS - contacts and calendar * In the past, meego.com was a dumping ground for technology developed and used elsewhere. Often it wasn't properly tested there. I have heard arguments that this testing can't be done without the apps in MeeGo using the middleware. IMHO this is just a lame excuse for not doing proper QA on middleware compoments, and leads to situations as with Dialer/libcommhistory were an app developer would have to stop working on his main task to investigate problems in someone else's code. Not cool. If no bugs were filed and no questions asked by mail, any efforts made to take that technology into use are indistinguishable from not even trying. Sure, Nokia bumbled on getting the actual component developers participate directly in MeeGo, but at least some QA feedback could help prioritizing efforts, especially as we were not expected to contribute client applications. I hope collaborative processes in MeeGo will be more transparent in the future. Best regards, Mikhail ___ MeeGo-dev mailing list MeeGo-dev@meego.com http://lists.meego.com/listinfo/meego-dev http://wiki.meego.com/Mailing_list_guidelines
Re: [MeeGo-dev] migration (back) to EDS - contacts and calendar
On Di, 2011-03-29 at 20:47 +0100, Robin Burchell wrote: On Tue, Mar 29, 2011 at 7:59 PM, zoltan@nokia.com wrote: If we agreed to handle libcommhistory by indexing contacts to tracker, then eventually IM/VoIP contacts could continue staying in tracker as well and everything would continue working via libcommhistory. I don't mean to throw fuel on the fire, but I have to re-ask something which was hinted at by Arjan earlier (I think): does anything actually use libcommhistory yet? The dialer in Handset would be the obvious candidate. But when Shane tried to use libcommhistory for dialer, it did not work for him and therefore he kept the current approach (QSettings as storage). I don't know whether bugs were filed; if not, then probably because it looked like work in progress which would be fixed eventually anyway. I also asked about UI features and designs which explicitly depend on features offered by libcommhistory. Shane wasn't aware of any. So in a nutshell, libcommhistory is not used in MeeGo as far as I know, and therefore not having it working for a while until it is really needed is acceptable. offtopic Obviously I'm not succeeding with keeping this thread on topic ;-} So here are some of my own thoughts on the meta aspect: * In the past, meego.com was a dumping ground for technology developed and used elsewhere. Often it wasn't properly tested there. I have heard arguments that this testing can't be done without the apps in MeeGo using the middleware. IMHO this is just a lame excuse for not doing proper QA on middleware compoments, and leads to situations as with Dialer/libcommhistory were an app developer would have to stop working on his main task to investigate problems in someone else's code. Not cool. * Now the situation is hopefully different, but still, the focus has to be on the code which really is needed. You can do anything you set your mind to when you have vision, determination, and an endless supply of expendable labor. - http://despair.com/achievement.html. We don't have that endless supply of expendable labor, so work needs to be prioritized. Of course that doesn't stop anyone from doing whatever he wants in his own time. The current planning for EDS focuses on core features, PIM storage. There are ideas how the rest of the features could be added again in a MeeGo based on EDS, and for me that is good enough to go ahead with improving EDS first. One has to start somewhere. Boiling the ocean hardly ever succeeds. /offtopic -- Best Regards, Patrick Ohly The content of this message is my personal opinion only and although I am an employee of Intel, the statements I make here in no way represent Intel's position on the issue, nor am I authorized to speak on behalf of Intel on this matter. ___ MeeGo-dev mailing list MeeGo-dev@meego.com http://lists.meego.com/listinfo/meego-dev http://wiki.meego.com/Mailing_list_guidelines
[MeeGo-dev] migration (back) to EDS - contacts and calendar
Hello! Let's be more specific about identifying work which needs to be done. I've put together some thoughts here: http://wiki.meego.com/Architecture/planning/evolution-data-server http://wiki.meego.com/Architecture/planning/evolution-data-server/QtContacts_storage_plugin http://wiki.meego.com/Architecture/planning/evolution-data-server/eds-improvements The first page does include a very simplistic comparison of the different options. I thought about leaving out that part altogether, but decided otherwise because I believe that listing some of the pros and cons in public is useful. No flame wars, please. For contacts and calendar it is fairly obvious where the pain points are, so let's focus on that for now. I'm interested in getting feedback on the proposed EDS improvements. At this point, I have the QtContacts-EDS contact manager working for reading and writing contacts, as outlined in the page linked to above. Change notifications are missing. It's just proof of concept (non-)quality. I suspect that I need to get open source approval first before publishing it somewhere. I have not done any prototyping yet with KCalCore, but I expect that similar improvements as for contacts will be useful for that work, too. -- Best Regards, Patrick Ohly The content of this message is my personal opinion only and although I am an employee of Intel, the statements I make here in no way represent Intel's position on the issue, nor am I authorized to speak on behalf of Intel on this matter. ___ MeeGo-dev mailing list MeeGo-dev@meego.com http://lists.meego.com/listinfo/meego-dev http://wiki.meego.com/Mailing_list_guidelines
Re: [MeeGo-dev] migration (back) to EDS - contacts and calendar
Hello, It is important to address IM/VoIP contacts case. Now, contactsd is storing them to tracker (https://gitorious.org/qtcontacts-tracker/contactsd) where they are mapped to communication history. QtContacts clients fetch/search/sort them along with local contacts (in the same fetch request). Best Regards, Aleksandar Stojiljkovic From: meego-dev-boun...@meego.com [meego-dev-boun...@meego.com] on behalf of ext Patrick Ohly [patrick.o...@intel.com] Sent: Tuesday, March 29, 2011 5:35 PM To: MeeGo-dev Subject: [MeeGo-dev] migration (back) to EDS - contacts and calendar Hello! Let's be more specific about identifying work which needs to be done. I've put together some thoughts here: http://wiki.meego.com/Architecture/planning/evolution-data-server http://wiki.meego.com/Architecture/planning/evolution-data-server/QtContacts_storage_plugin http://wiki.meego.com/Architecture/planning/evolution-data-server/eds-improvements The first page does include a very simplistic comparison of the different options. I thought about leaving out that part altogether, but decided otherwise because I believe that listing some of the pros and cons in public is useful. No flame wars, please. For contacts and calendar it is fairly obvious where the pain points are, so let's focus on that for now. I'm interested in getting feedback on the proposed EDS improvements. At this point, I have the QtContacts-EDS contact manager working for reading and writing contacts, as outlined in the page linked to above. Change notifications are missing. It's just proof of concept (non-)quality. I suspect that I need to get open source approval first before publishing it somewhere. I have not done any prototyping yet with KCalCore, but I expect that similar improvements as for contacts will be useful for that work, too. -- Best Regards, Patrick Ohly The content of this message is my personal opinion only and although I am an employee of Intel, the statements I make here in no way represent Intel's position on the issue, nor am I authorized to speak on behalf of Intel on this matter. ___ MeeGo-dev mailing list MeeGo-dev@meego.com http://lists.meego.com/listinfo/meego-dev http://wiki.meego.com/Mailing_list_guidelines ___ MeeGo-dev mailing list MeeGo-dev@meego.com http://lists.meego.com/listinfo/meego-dev http://wiki.meego.com/Mailing_list_guidelines
Re: [MeeGo-dev] migration (back) to EDS - contacts and calendar
On Tue, 2011-03-29 at 20:37 +0200, Patrick Ohly wrote: On Di, 2011-03-29 at 18:48 +0100, aleksandar.stojiljko...@nokia.com wrote: It is important to address IM/VoIP contacts case. Now, contactsd is storing them to tracker (https://gitorious.org/qtcontacts-tracker/contactsd) where they are mapped to communication history. This has never been used by the Tablet UX. The focus there is to use libfolks instead. Admittedly libfolks isn't ready yet (neither with nor without Tracker). When did Tablet UX become the focus of MeeGo? regards, Michael ___ MeeGo-dev mailing list MeeGo-dev@meego.com http://lists.meego.com/listinfo/meego-dev http://wiki.meego.com/Mailing_list_guidelines
Re: [MeeGo-dev] migration (back) to EDS - contacts and calendar
From: Patrick Ohly Sent: 29 March, 2011 21:37 On Di, 2011-03-29 at 18:48 +0100, aleksandar.stojiljko...@nokia.com wrote: It is important to address IM/VoIP contacts case. Now, contactsd is storing them to tracker (https://gitorious.org/qtcontacts-tracker/contactsd) where they are mapped to communication history. This has never been used by the Tablet UX. The focus there is to use libfolks instead. Admittedly libfolks isn't ready yet (neither with nor without Tracker). The architecture should support the use case of IM/VoIP contacts, whenever it will come. You can't exclude the use case from neither Tablet, nor Handset UX, since manufacturers might want the functionality. Moreover, we have it and it works. Now is it worth investing a lot to ruin it? If we agreed to handle libcommhistory by indexing contacts to tracker, then eventually IM/VoIP contacts could continue staying in tracker as well and everything would continue working via libcommhistory. Of course this would not prevent using libfolks either, if someone wanted that one. Regards, Zoltan smime.p7s Description: S/MIME cryptographic signature ___ MeeGo-dev mailing list MeeGo-dev@meego.com http://lists.meego.com/listinfo/meego-dev http://wiki.meego.com/Mailing_list_guidelines
Re: [MeeGo-dev] migration (back) to EDS - contacts and calendar
On Di, 2011-03-29 at 19:41 +0100, Michael Hasselmann wrote: On Tue, 2011-03-29 at 20:37 +0200, Patrick Ohly wrote: On Di, 2011-03-29 at 18:48 +0100, aleksandar.stojiljko...@nokia.com wrote: It is important to address IM/VoIP contacts case. Now, contactsd is storing them to tracker (https://gitorious.org/qtcontacts-tracker/contactsd) where they are mapped to communication history. This has never been used by the Tablet UX. The focus there is to use libfolks instead. Admittedly libfolks isn't ready yet (neither with nor without Tracker). When did Tablet UX become the focus of MeeGo? I'm not going to debate what the focus of MeeGo was, is, isn't, or should be - in particular not on meego-dev. Everyone, please take that discussion to the steering committee and/or to program managers (via feature requests in bugs.meego.com). I know that this is an unsatisfactory answer, but it is the only one that I have. Now can we move on with discussing EDS improvements, please? -- Best Regards, Patrick Ohly The content of this message is my personal opinion only and although I am an employee of Intel, the statements I make here in no way represent Intel's position on the issue, nor am I authorized to speak on behalf of Intel on this matter. ___ MeeGo-dev mailing list MeeGo-dev@meego.com http://lists.meego.com/listinfo/meego-dev http://wiki.meego.com/Mailing_list_guidelines
Re: [MeeGo-dev] migration (back) to EDS - contacts and calendar
Hi, On Tue, Mar 29, 2011 at 7:59 PM, zoltan@nokia.com wrote: If we agreed to handle libcommhistory by indexing contacts to tracker, then eventually IM/VoIP contacts could continue staying in tracker as well and everything would continue working via libcommhistory. I don't mean to throw fuel on the fire, but I have to re-ask something which was hinted at by Arjan earlier (I think): does anything actually use libcommhistory yet? If not, it's a bit of a moot point, isn't it? If libcommhistory is going to be relevant in the open source stack, it either needs to be used. To be used, someone needs to write the code for it. That's probably going to be you [you meaning the team working on it], unless you get very lucky and convince someone else to get interested in it, and write code using it. Until this happens, it working, or not working, isn't really a concern for MeeGo, because MeeGo doesn't use it. If Nokia does, then that would be Nokia's concern. As far as my understanding of the technologies is, libfolks isn't related to libcommhistory anyway. It's more similar to contactsd / qtcontacts, in that it fetches contacts from multiple sources (like Telepathy) and stuff. /offtopic Best regards, Robin ___ MeeGo-dev mailing list MeeGo-dev@meego.com http://lists.meego.com/listinfo/meego-dev http://wiki.meego.com/Mailing_list_guidelines
Re: [MeeGo-dev] migration (back) to EDS - contacts and calendar
Hello, From: Robin Burchell [mailto:surreal.w...@gmail.com] Sent: 29 March, 2011 22:48 On Tue, Mar 29, 2011 at 7:59 PM, zoltan@nokia.com wrote: If we agreed to handle libcommhistory by indexing contacts to tracker, then eventually IM/VoIP contacts could continue staying in tracker as well and everything would continue working via libcommhistory. I don't mean to throw fuel on the fire, but I have to re-ask something which was hinted at by Arjan earlier (I think): does anything actually use libcommhistory yet? If not, it's a bit of a moot point, isn't it? I don't know who is using it. Commhistory provides Qt models for messaging and call events (the lib), plus a daemon for Telepathy observer for call and messaging channels, notification handling, etc. The code has been part of MeeGo. If not used, then it's work done, work tested, work lost: business as usual. If libcommhistory is going to be relevant in the open source stack, it either needs to be used. To be used, someone needs to write the code for it. That's probably going to be you [you meaning the team working on it], unless you get very lucky and convince someone else to get interested in it, and write code using it. Until this happens, it working, or not working, isn't really a concern for MeeGo, because MeeGo doesn't use it. If Nokia does, then that would be Nokia's concern. True. Actually we are trying to extend a bit the scope of libcommhistory, by contributing the associated call and messaging Qt middleware to MeeGo during the coming weeks. Although we cannot contribute any UI related code, after this it will be much easier to fit a thin Qt or QML UI on top of this for call and messaging. This is work in progress, so at this point I would just like to avoid premature killing of libcommhistory as a byproduct of the latest MeeGo architecture changes and it corollaries. When this contribution completes, might put QtContacts and tracker into slightly different light as well, because they all work together nicely and tested (OK, maybe not for the coming release, but hopefully the next). Of course this is only great news if someone is interested to write Qt/QML call UI and messaging UI very easily. It will be of no help for people who want to do the same from scratch in a more complex way. As far as my understanding of the technologies is, libfolks isn't related to libcommhistory anyway. It's more similar to contactsd / qtcontacts, in that it fetches contacts from multiple sources (like Telepathy) and stuff. The talk was about the need of gathering IM and VoIP contacts (used by libcomhistory) also by the new solution. There is a working solution for it now, which is going to be gracefully ditched, so I expressed my hope that the workaround kind-of agreed for libcommhistory would be applicable here as well. Libfolks is not really an alternative at least for commhistory. Indeed, if libcommhistory is not used, then the point is moot, but at least until now it has been part of MeeGo. Of course, we never know for how long we have road under the weels, but keep running until meeting the brick wall. Regards, Zoltan smime.p7s Description: S/MIME cryptographic signature ___ MeeGo-dev mailing list MeeGo-dev@meego.com http://lists.meego.com/listinfo/meego-dev http://wiki.meego.com/Mailing_list_guidelines
Re: [MeeGo-dev] migration (back) to EDS - contacts and calendar
On Tue, 2011-03-29 at 21:08 +0200, Patrick Ohly wrote: On Di, 2011-03-29 at 19:41 +0100, Michael Hasselmann wrote: On Tue, 2011-03-29 at 20:37 +0200, Patrick Ohly wrote: On Di, 2011-03-29 at 18:48 +0100, aleksandar.stojiljko...@nokia.com wrote: It is important to address IM/VoIP contacts case. Now, contactsd is storing them to tracker (https://gitorious.org/qtcontacts-tracker/contactsd) where they are mapped to communication history. This has never been used by the Tablet UX. The focus there is to use libfolks instead. Admittedly libfolks isn't ready yet (neither with nor without Tracker). When did Tablet UX become the focus of MeeGo? I'm not going to debate what the focus of MeeGo was, is, isn't, or should be - in particular not on meego-dev. Everyone, please take that discussion to the steering committee and/or to program managers (via feature requests in bugs.meego.com). OK https://bugs.meego.com/show_bug.cgi?id=15014 Cheers, Philip -- Philip Van Hoof freelance software developer Codeminded BVBA - http://codeminded.be ___ MeeGo-dev mailing list MeeGo-dev@meego.com http://lists.meego.com/listinfo/meego-dev http://wiki.meego.com/Mailing_list_guidelines
Re: [MeeGo-dev] migration (back) to EDS - contacts and calendar
Hi, On Tue, Mar 29, 2011 at 10:01 PM, zoltan@nokia.com wrote: I don't know who is using it. Commhistory provides Qt models for messaging and call events (the lib), plus a daemon for Telepathy observer for call and messaging channels, notification handling, etc. The code has been part of MeeGo. If not used, then it's work done, work tested, work lost: business as usual. This is great, but you're kind of illustrating the point I'm trying to make. You generally can't just create great technology, and hope for people to batter down your doors for it. You need to publicize it, make it visible, get it integrated into software that's out there to make cool things with it. As a result, your stuff will get used, people will love it, and fight for it. The fact that you don't *know* what uses it kind of tells me that this hasn't really been happening. Food for thought? Regards, Zoltan -- Robin Burchell http://rburchell.com ___ MeeGo-dev mailing list MeeGo-dev@meego.com http://lists.meego.com/listinfo/meego-dev http://wiki.meego.com/Mailing_list_guidelines
Re: [MeeGo-dev] migration (back) to EDS - contacts and calendar
Hello, Related to IM/VoIP contacts and social networking contacts: mapping of QtContacts filters to EDS queries, with unsupported filters implemented by QtContacts after retrieving contacts (slower than filtering in EDS daemon) Tablet UX choice (according to your previous mail in this thread) sounds flexible: 1) fetch time contacts aggregation using 1a)QtMobility/QtContacts or 1b) libfolks 2) write time aggregation - if you decide to store contacts to EDS (like we do now to tracker) and use EDS search/filter capabilities. It depends on your use cases, performance targets and it is also technology choice if you'll support fetch time pluggable aggregation - 1a or 1b or both at the same time,... or focus on 2). I just wanted to point that it could be useful to put something about this to http://wiki.meego.com/Architecture/planning/evolution-data-server/QtContacts_storage_plugin page. Best Regards, Aleksandar Stojiljkovic From: meego-dev-boun...@meego.com [meego-dev-boun...@meego.com] on behalf of ext aleksandar.stojiljko...@nokia.com [aleksandar.stojiljko...@nokia.com] Sent: Tuesday, March 29, 2011 8:48 PM To: patrick.o...@intel.com; meego-dev@meego.com Subject: Re: [MeeGo-dev] migration (back) to EDS - contacts and calendar Hello, It is important to address IM/VoIP contacts case. Now, contactsd is storing them to tracker (https://gitorious.org/qtcontacts-tracker/contactsd) where they are mapped to communication history. QtContacts clients fetch/search/sort them along with local contacts (in the same fetch request). Best Regards, Aleksandar Stojiljkovic From: meego-dev-boun...@meego.com [meego-dev-boun...@meego.com] on behalf of ext Patrick Ohly [patrick.o...@intel.com] Sent: Tuesday, March 29, 2011 5:35 PM To: MeeGo-dev Subject: [MeeGo-dev] migration (back) to EDS - contacts and calendar Hello! Let's be more specific about identifying work which needs to be done. I've put together some thoughts here: http://wiki.meego.com/Architecture/planning/evolution-data-server http://wiki.meego.com/Architecture/planning/evolution-data-server/QtContacts_storage_plugin http://wiki.meego.com/Architecture/planning/evolution-data-server/eds-improvements The first page does include a very simplistic comparison of the different options. I thought about leaving out that part altogether, but decided otherwise because I believe that listing some of the pros and cons in public is useful. No flame wars, please. For contacts and calendar it is fairly obvious where the pain points are, so let's focus on that for now. I'm interested in getting feedback on the proposed EDS improvements. At this point, I have the QtContacts-EDS contact manager working for reading and writing contacts, as outlined in the page linked to above. Change notifications are missing. It's just proof of concept (non-)quality. I suspect that I need to get open source approval first before publishing it somewhere. I have not done any prototyping yet with KCalCore, but I expect that similar improvements as for contacts will be useful for that work, too. -- Best Regards, Patrick Ohly The content of this message is my personal opinion only and although I am an employee of Intel, the statements I make here in no way represent Intel's position on the issue, nor am I authorized to speak on behalf of Intel on this matter. ___ MeeGo-dev mailing list MeeGo-dev@meego.com http://lists.meego.com/listinfo/meego-dev http://wiki.meego.com/Mailing_list_guidelines ___ MeeGo-dev mailing list MeeGo-dev@meego.com http://lists.meego.com/listinfo/meego-dev http://wiki.meego.com/Mailing_list_guidelines ___ MeeGo-dev mailing list MeeGo-dev@meego.com http://lists.meego.com/listinfo/meego-dev http://wiki.meego.com/Mailing_list_guidelines
Re: [MeeGo-dev] migration (back) to EDS
On Tue, 2011-03-22 at 10:15 +0100, Patrick Ohly wrote: On Di, 2011-03-22 at 09:04 +, zoltan@nokia.com wrote: From: Patrick Ohly [mailto:patrick.o...@intel.com], sent on 22 March, 2011 10:35 Option 2 seems like a reasonable approach to me, if we can make it so that the call history database is self-contained and merely mirrors the subset of the contact data in read-only mode that is relevant for a quick list of results, with more details fetched on demand from the main contact database. Yes, and same for messaging (IM+SMS). This assumes the storage for IM + SMS + call history is still tracker. Strictly speaking, the requirement is just that contacts and the above should be stored in the same DB, whatever it is. Just out of curiosity, what would prevent support in EDS for IM+SMS+callhistory? It should be just a matter of time. It's designed around single daemons for contacts and calendar. IM+SMS +callhistory could be mapped into the calendar part, but that wouldn't solve the problem of correlating the two data sources. It should be more about exending EContact to incorporate additional fields. Call history could be extended into EContact. Just for example, the birthday calendar of EDS relates contacts into a calendar, some to look for correlating data sources, may be extending along those lines could help to co-relate sources. I also suspect that the EDS APIs simply wouldn't support the additional use cases well enough. The goal is to keep EDS in MeeGo aligned with upstream EDS, to avoid maintaining a fork of it for Handsets in MeeGo. I'm sure, we can extend EDS with the upstream support. I could help in where ever required. -Srini. About shared models and caching (and then supporting heterogenous data sources) we are doing some prototyping now, I hope to share more on the MeeGo conference, but this won't work very soon, so it's out of question for now. That sounds interesting, please share as soon as you can. In case it wasn't obvious, I'm trying to keep the discussion about this architecture change public instead of working on a fully-formed plan in private exactly because I hope to get such additional feedback and pointers. Unfortunately private reasons prevent me from coming to the next MeeGo conference. ___ MeeGo-dev mailing list MeeGo-dev@meego.com http://lists.meego.com/listinfo/meego-dev http://wiki.meego.com/Mailing_list_guidelines
Re: [MeeGo-dev] migration (back) to EDS
On Tue, 22 Mar 2011 17:14:39 +0100, Patrick Ohly wrote: On Di, 2011-03-22 at 15:40 +, Adrien Bustany wrote: On Tue, 22 Mar 2011 13:15:48 +0100, Patrick Ohly wrote: On Di, 2011-03-22 at 09:34 +0100, Patrick Ohly wrote: One more thought on this. Consider that a call was recorded at a time when there was a corresponding contact. Later the address book gets changed such that there is no match anymore - perhaps the contact was removed to clean up the address book, perhaps because his or her phone number changed. I would find it convenient if the call history gave me more than just the old phone number in such a case. Ultimately this is a question to the UX designers. My point here is that the caching scheme proposed above would support such a use case, whereas creating the match at the time of the query based on the then current data (as it is presumably done in Tracker/libcommhistory) doesn't. You just degrade the contact from a nco:PersonContact to a nco:Contact in Tracker at delete time if it's linked to an existing call event. Interesting approach... but how would that tie into deleting a contact via QtContacts? That's the API that will be used, not low-level Tracker manipulations. It's all a matter of conventions. QtMobility makes no difference between a real delete and a delete but keep for call history. The behavior to choose could be based or either a parameter to the QContactRemoveRequest, a system wide setting, you name it... qtcontacts-tracker is basically a sparql query generator, so it generates what you want. The problem would be more how to fit that in the current mobility API, we actually already have some extensions to it since it has no official support for implementing custom requests. Note that the code change to add this special delete would be really small, something you can do in half a day... What amuses me is that we started from a global architecture discussion (are we going to make a house in wood or concrete), and we're right now discussing the color of the handle of the window in the basement ;) But details do matter too... Of course they do. It's part of verifying that architecture decisions do not run afoul of some constraints. We know that a house can be build from wood and concrete, because both has been done before. But can a plane be built from concrete? And no, I am not saying that either EDS or Tracker are the concrete in this example ;-) :-) ___ MeeGo-dev mailing list MeeGo-dev@meego.com http://lists.meego.com/listinfo/meego-dev http://wiki.meego.com/Mailing_list_guidelines
Re: [MeeGo-dev] migration (back) to EDS
Hello Arjan, From: Arjan van de Ven [mailto:ar...@linux.intel.com] Sent: 22 March, 2011 23:44 the libcommhistory package is not even in any of the images... how can you claim that it's actually this critical? Critical in the handset case at least when you need to build Qt models for conversation history and call history. Historically sqlite and tracker storage backends have been supported. I assume it would be usable for tablets and even netbooks (for messaging). Whether it is in the images, it's up to you, the code is there. You can also reinvent it, if needed. Does the MeeGo dialer even use it? When I spoke with the owner of the dialer app (which is supposedly the key user of all this), this really did not come up as something that is currently in use and working... quite the opposite. I guess this is again up to the dialer owner. Unfortunately we could not open any UI related stuff, and there was no time doing a nice separation of strictly UI related and middleware components, so there are a few internal interfaces which have not been opened but might have made your life much easier. This was mainly because program pressure on this side, we were too busy doing phones. I guess it was frustrating to see code just thrown over the fence, without clear indications how to use it. (it might work in your maemo OS, but that is of no consideration) :). I can understand why you are moving back to Moblin in these dark times ;). But there are a number of things Maemo got right in the handset case, and it would be pity dumping them and reinventing the pain, since that's going to delay things. (I am not talking about tracker here :). It is important to realize how different the handset case is from the rest. So just not considering it is a mistake. The MeeGo architecture team made these decisions in consultation with the various handset and tablet architects. It looks like it was an internal Intel decision (or at least without Nokia). I can't blame you on that, but if the governance model changed in the background, would you state that on meego.com, just to avoid fights coming from false assumptions? I know it's not popular with you and some other @nokia.com folks... but so be it. And the reason for this mainly was the way the decision was made, communicated, and how questions, facts and measurements were ignored. Don't you think if it was correctly made, it wouldn't have created such a mail-storm? It was not according to http://wiki.meego.com/Architecture (there was no discussion, just announcement, with all feedback ignored). It was confusing and frustrating to see this secrecy and dictatorial decision making, with no criteria to measure it. I think it would take just a little bit of positive/cooperative attitude to do it much better. Maybe you didn't have much time and for some reason you had to make a quick big decision, I don't know. But please always state what is the goal and how will it be measured. In this tracker-EDS case: data sets, performance tests, measurements. (I have struggled enough with tracker vs sqlite to learn this lesson well. If you think I am on tracker's side, it's wrong: I am on the side of correct technical approach). Then you can choose a solution to the problem, and not vice versa. Regards, Zoltan smime.p7s Description: S/MIME cryptographic signature ___ MeeGo-dev mailing list MeeGo-dev@meego.com http://lists.meego.com/listinfo/meego-dev http://wiki.meego.com/Mailing_list_guidelines
Re: [MeeGo-dev] migration (back) to EDS
Hi, sakari.pou...@nokia.com wrote: On 22/03/11 11:44 PM, ext Arjan van de Ven ar...@linux.intel.com wrote: The MeeGo architecture team made these decisions in consultation with the various handset and tablet architects. Just to be clear, Nokia members of the MeeGo architecture team where not involved in these latest decisions of the MeeGo architecture direction (MSSF/Buteo/PIM). This message requires some clarification, I think. Arjan, what is the make-up of the architecture team now? Are any Nokia people still members as far as you are concerned? For all the talk of meritocracy, it seems like people are being appointed by their employers to fill their seats, and those seats are subject to changing corporate winds. Cheers, Dave. -- Email: dne...@maemo.org Jabber: bo...@jabber.org ___ MeeGo-dev mailing list MeeGo-dev@meego.com http://lists.meego.com/listinfo/meego-dev http://wiki.meego.com/Mailing_list_guidelines
Re: [MeeGo-dev] migration (back) to EDS
On Mo, 2011-03-21 at 21:31 +, Philip Van Hoof wrote: On Mon, 2011-03-21 at 20:54 +0100, Patrick Ohly wrote: On Mo, 2011-03-21 at 17:45 +, zoltan@nokia.com wrote: From: Patrick Ohly, on 21 March, 2011 16:37 In a nutshell, the goal is to keep QtContacts, QMF and KCalCore as the main APIs used by applications and the higher-level QtMobility APIs. If we can achieve that, very little will have to be done in applications once we change the core components. This will break libcommhistory. I'm aware of that, and I don't claim that there is an answer. There is a task open for those who need libcommhistory (or something like it) to look into this problem and comment on the current proposals. Ok, so.. I'll need something like libcommhistory. Check. What are the current proposals for things like libcommhistory? (this is a real question. You're the architect: don't sidestep it again) No, I'm not officially a MeeGo architect. I might be doing some of the work at the moment, but others also need to comment, in particular for communication history. This part is opinion I'll refrain from commenting on opinions. Remember, let's keep it technical. -- Best Regards, Patrick Ohly The content of this message is my personal opinion only and although I am an employee of Intel, the statements I make here in no way represent Intel's position on the issue, nor am I authorized to speak on behalf of Intel on this matter. ___ MeeGo-dev mailing list MeeGo-dev@meego.com http://lists.meego.com/listinfo/meego-dev http://wiki.meego.com/Mailing_list_guidelines
Re: [MeeGo-dev] migration (back) to EDS
On Mo, 2011-03-21 at 20:40 +, Ville M. Vainio wrote: On Mon, Mar 21, 2011 at 3:36 PM, Patrick Ohly patrick.o...@intel.com wrote: Mail Storage and Transports * write simplistic server which runs Camel * replace QMF client library with one which accesses that server; source code compatibility of just the functionality needed by the Tablet mail app would be a good first step Why do you need to use Camel instead of QMF? Does Camel have some virtues over QMF messageserver? QMF has been a rough experience for us at Intel. There were issues with the QMF+Accounts/SSO integration in the MeeGo fork of it, which we couldn't get help for because this functionality wasn't used by Nokia the same we did. These problems went away when using the upstream QMF directly. We still saw issues with the QMF daemon hanging under unreliable network conditions. It turned out later that a potential fix for that hadn't been merged into MeeGo yet. All of that could have been handled better and probably isn't representative of QMF itself, but it has tainted our experience with it. I personally don't think that there are some specific technical merits. Others who have worked more with it might be able to list some, not sure. The key differences are of a less technical nature, for example: 1. unifying the mail infrastructure across all verticals 2. aligning with an upstream project which has maintainers who have a business interest in it (arguably such a comparison is very subjective and speculative) 3. in-house expertise I understand that this must be a disappointment for those who have worked hard on QMF. I've been in the same boat when Nokia's Buteo replaced SyncEvolution in MeeGo. My apologies if anyone feels slighted by these decisions. -- Best Regards, Patrick Ohly The content of this message is my personal opinion only and although I am an employee of Intel, the statements I make here in no way represent Intel's position on the issue, nor am I authorized to speak on behalf of Intel on this matter. ___ MeeGo-dev mailing list MeeGo-dev@meego.com http://lists.meego.com/listinfo/meego-dev http://wiki.meego.com/Mailing_list_guidelines
Re: [MeeGo-dev] migration (back) to EDS
From: Patrick Ohly [mailto:patrick.o...@intel.com] Sent on 21 March, 2011 21:55 This will break libcommhistory. I'm aware of that, and I don't claim that there is an answer. There is a task open for those who need libcommhistory Those who need it are messaging-ui, call-history, commhistory-daemon (handling the storage), qt-mobility, qt webruntime, at-phonebook, msgsync. I assumed the dependencies have been checked before announcing the change. Where is this task open and to whom? Is there a bug? (or something like it) What are the other options instead of libcommhistory? to look into this problem and comment on the current proposals. I expect there is an answer already now, because this was called an architecture decision, and as such, it must have addressed all the basic use cases and consequences _before_ it was made, right? I cannot believe these important dependencies were not considered when the decision was made to move back to EDS... BTW who made the decision (meeting minutes please?), and where is it documented? I have asked it earlier too: what is the governance model of MeeGo now? So far no change was announced, and since moving Contacts back to EDS was an authoritative and arbitrary, rather than technically justified architecture decision, it should be considered *void* until it's done properly. I am not against it per se, just against the non-professional way it was made, and against ignoring all facts, measurements and dependencies listed on this mailing list, that make clear this was a bad (incomplete) decision, at least concerning Contacts. Making Sync somewhat simpler is not enough reason to break half the system (see the list below). CC'ing Arjan, Sakari, and Mikko. Option 2 from below would be the cheapest, fastest and least pain path but it's duplicating contacts to tracker. Option 3 would be optimal but difficult, option 1 could also work. Choose one, or suggest something else. Please finish the job you started :). Regards, Zoltan -Original Message- From: Zoltan Kis, on 21 March, 2011 19:45 To: patrick.o...@intel.com; meego-dev@meego.com Subject: Re: [MeeGo-dev] migration (back) to EDS From: Patrick Ohly, on 21 March, 2011 16:37 In a nutshell, the goal is to keep QtContacts, QMF and KCalCore as the main APIs used by applications and the higher-level QtMobility APIs. If we can achieve that, very little will have to be done in applications once we change the core components. This will break libcommhistory. What do you suggest? As explained earlier, one technical requirement is that contacts and IM/callhistory data should be in the same database. Options: 1. Support for IM and call history in EDS 2. Duplicate/sync to tracker of contact fields needed for building conversation history and call history 3. Shared memory contacts model with own indexing across domains. 4. Catastrophic messaging performance drop (10-100 times if I remember right from early tracker bugs for missing index). See the queries here: http://gitorious.net/commhistory/libcommhistory/trees/master/sparql Where else do you want to see performance improvements by moving Contacts to EDS, if not in the most performance critical areas (e.g. IM conversational view)? This use case shall be solved by the ones who made the architecture decision ruining it. Which one should it be, and who will do the work? Regards, Zoltan Calendar Storage * use upstream KDE version of KCalCore * write storage which uses libecal/EDS * add content protection to EDS * remove unnecessary parsing of iCalendar in libecal (not needed by storage and sync) * improve reading of meta data (can speed up sync, see concurrent modifications of items in GUI and EDS database on the evo-hackers list from back in 2009) Contact Storage * write a QtContacts manager which uses libebook/EDS, similar to QtMobility manager for Harmattan, but with some crucial differences: * avoid ID mapping because it requires reading all contacts at startup; must change EDS to use 32 bit integers as local IDs for that (patch ready) * convert between QtContacts and vCard using QtVersit, with EDS specific properties and including custom properties for QtContactDetails which have no other mapping to vCard (same approach as in SyncEvolution QtContacts backend); this avoids lots of hand-written mapping code which depends on the (necessarily incomplete) EContact API * no support for relationships * no support for change tracking, that logic needs to be in higher layers * support for groups open - special contact as done by Evolution itself? * self contact also under debate
Re: [MeeGo-dev] migration (back) to EDS
On Tue, 22 Mar 2011 09:34:40 +0100, Patrick Ohly wrote: Hello Zoltan, I can't comment on the decision making process (that's above my grade level) and don't want to (this is a technical mailing). On Di, 2011-03-22 at 08:02 +, zoltan@nokia.com wrote: Option 2 from below would be the cheapest, fastest and least pain path but it's duplicating contacts to tracker. Option 3 would be optimal but difficult, option 1 could also work. Choose one, or suggest something else. Please finish the job you started :). Option 2 seems like a reasonable approach to me, if we can make it so that the call history database is self-contained and merely mirrors the subset of the contact data in read-only mode that is relevant for a quick list of results, with more details fetched on demand from the main contact database. Whether that database is Tracker remains to be seen. In my understanding, matching phone numbers in Tracker is based on a fixed number of digits at the end of the phone number. The Wiki already documents that this is only configurable on a system-wide basis, so if my contacts are from China, the US and Europe (which in fact they are), then the system settings might be wrong for some contacts (from http://wiki.meego.com/Architecture/Documentation/PIM/Contacts_Engine#User_Data.2C_Configurability). I also heard that some US numbers have additional digits at the end which also break the current matching. Sorry, I never saw a specific example, otherwise I would provide it. To me that suggests that a matching between an address book and call events is a fairly complex process which needs to be: 1. done by custom software written specifically for this problem domain (as in http://code.google.com/p/libphonenumber/), 2. calculated as the data changes, 3. cached to speed up the actual displaying. You're right, currently that matching length is set system wide. That would be however quite easy to fix (using libphonenumber for matching wouldn't be hard either), it's actually not the jobs of Tracker but of the apps to fill that cached lookup value. Patching commhistory and qtcontacts-tracker to do the lookup not using a fixed length but whatever value would be very easy. We never had a bug report about it though, as far as I remember, and it does not look like a fundamental architecture issue. Cheers Adrien ___ MeeGo-dev mailing list MeeGo-dev@meego.com http://lists.meego.com/listinfo/meego-dev http://wiki.meego.com/Mailing_list_guidelines
Re: [MeeGo-dev] migration (back) to EDS
Hi Patrick, From: Patrick Ohly [mailto:patrick.o...@intel.com], sent on 22 March, 2011 10:35 On Di, 2011-03-22 at 08:02 +, zoltan@nokia.com wrote: Option 2 from below would be the cheapest, fastest and least pain path but it's duplicating contacts to tracker. Option 3 would be optimal but difficult, option 1 could also work. Choose one, or suggest something else. Please finish the job you started :). Option 2 seems like a reasonable approach to me, if we can make it so that the call history database is self-contained and merely mirrors the subset of the contact data in read-only mode that is relevant for a quick list of results, with more details fetched on demand from the main contact database. Yes, and same for messaging (IM+SMS). This assumes the storage for IM + SMS + call history is still tracker. Strictly speaking, the requirement is just that contacts and the above should be stored in the same DB, whatever it is. Just out of curiosity, what would prevent support in EDS for IM+SMS+callhistory? Not that I would suggest it :). About shared models and caching (and then supporting heterogenous data sources) we are doing some prototyping now, I hope to share more on the MeeGo conference, but this won't work very soon, so it's out of question for now. In my understanding, matching phone numbers in Tracker is based on a fixed number of digits at the end of the phone number. The Wiki already documents that this is only configurable on a system-wide basis, so if my contacts are from China, the US and Europe (which in fact they are), then the system settings might be wrong for some contacts (from http://wiki.meego.com/Architecture/Documentation/PIM/Contacts_Engine#User_Data.2C_Configurability ) . I also heard that some US numbers have additional digits at the end which also break the current matching. Sorry, I never saw a specific example, otherwise I would provide it. To me that suggests that a matching between an address book and call events is a fairly complex process which needs to be: 1. done by custom software written specifically for this problem domain (as in http://code.google.com/p/libphonenumber/), 2. calculated as the data changes, 3. cached to speed up the actual displaying. Valid points, we should improve on this with finer grained rules. Adding Andrea. Regards, Zoltan smime.p7s Description: S/MIME cryptographic signature ___ MeeGo-dev mailing list MeeGo-dev@meego.com http://lists.meego.com/listinfo/meego-dev http://wiki.meego.com/Mailing_list_guidelines
Re: [MeeGo-dev] migration (back) to EDS
On Di, 2011-03-22 at 09:04 +, zoltan@nokia.com wrote: From: Patrick Ohly [mailto:patrick.o...@intel.com], sent on 22 March, 2011 10:35 Option 2 seems like a reasonable approach to me, if we can make it so that the call history database is self-contained and merely mirrors the subset of the contact data in read-only mode that is relevant for a quick list of results, with more details fetched on demand from the main contact database. Yes, and same for messaging (IM+SMS). This assumes the storage for IM + SMS + call history is still tracker. Strictly speaking, the requirement is just that contacts and the above should be stored in the same DB, whatever it is. Just out of curiosity, what would prevent support in EDS for IM+SMS+callhistory? It's designed around single daemons for contacts and calendar. IM+SMS +callhistory could be mapped into the calendar part, but that wouldn't solve the problem of correlating the two data sources. I also suspect that the EDS APIs simply wouldn't support the additional use cases well enough. The goal is to keep EDS in MeeGo aligned with upstream EDS, to avoid maintaining a fork of it for Handsets in MeeGo. About shared models and caching (and then supporting heterogenous data sources) we are doing some prototyping now, I hope to share more on the MeeGo conference, but this won't work very soon, so it's out of question for now. That sounds interesting, please share as soon as you can. In case it wasn't obvious, I'm trying to keep the discussion about this architecture change public instead of working on a fully-formed plan in private exactly because I hope to get such additional feedback and pointers. Unfortunately private reasons prevent me from coming to the next MeeGo conference. -- Best Regards, Patrick Ohly The content of this message is my personal opinion only and although I am an employee of Intel, the statements I make here in no way represent Intel's position on the issue, nor am I authorized to speak on behalf of Intel on this matter. ___ MeeGo-dev mailing list MeeGo-dev@meego.com http://lists.meego.com/listinfo/meego-dev http://wiki.meego.com/Mailing_list_guidelines
Re: [MeeGo-dev] migration (back) to EDS
On Tue, 2011-03-22 at 09:34 +0100, Patrick Ohly wrote: On Di, 2011-03-22 at 08:02 +, zoltan@nokia.com wrote: Option 2 from below would be the cheapest, fastest and least pain path but it's duplicating contacts to tracker. Option 3 would be optimal but difficult, option 1 could also work. Choose one, or suggest something else. Please finish the job you started :). Option 2 seems like a reasonable approach to me, if we can make it so that the call history database is self-contained and merely mirrors the subset of the contact data in read-only mode that is relevant for a quick list of results, with more details fetched on demand from the main contact database. Keeping Tracker's RDF store in sync with EDS's data sounds like a viable option that would likely be accepted in the Tracker upstream project. Tracker already has a Evolution plugin that keeps E-mail metadata in Evolution in sync. You can find the code for that EPlugin here: http://git.gnome.org/browse/tracker/tree/src/plugins/evolution Adapting this plugin to also work in the Evolution UI as shipped on MeeGo should as far as I know not be very difficult (if even any adaptation is needed). The EDS one would have to be done by implementing a TrackerMiner under for example src/miners/eds, the existing RSS one is a good example to get started: http://git.gnome.org/browse/tracker/tree/src/miners/rss/tracker-miner-rss.c And for a TrackerMiner implemented in Vala, look at the Flickr one: http://git.gnome.org/browse/tracker/tree/src/miners/flickr/tracker-miner-flickr.vala I can imagine this miner using EBookView to give live updates whenever a contact changes in E-D-S, to Tracker's RDF store. In the other direction can, by listening for GraphUpdated and/or the Writeback signal, the miner also writeback to the E-D-S store. For an example on how this would or could work: the Flickr one also writes back to the Flickr website whenever metadata changes in Tracker first. Cheers, Philip -- Philip Van Hoof freelance software developer Codeminded BVBA - http://codeminded.be ___ MeeGo-dev mailing list MeeGo-dev@meego.com http://lists.meego.com/listinfo/meego-dev http://wiki.meego.com/Mailing_list_guidelines
Re: [MeeGo-dev] migration (back) to EDS
On Di, 2011-03-22 at 09:34 +0100, Patrick Ohly wrote: Hello Zoltan, I can't comment on the decision making process (that's above my grade level) and don't want to (this is a technical mailing). On Di, 2011-03-22 at 08:02 +, zoltan@nokia.com wrote: Option 2 from below would be the cheapest, fastest and least pain path but it's duplicating contacts to tracker. Option 3 would be optimal but difficult, option 1 could also work. Choose one, or suggest something else. Please finish the job you started :). Option 2 seems like a reasonable approach to me, if we can make it so that the call history database is self-contained and merely mirrors the subset of the contact data in read-only mode that is relevant for a quick list of results, with more details fetched on demand from the main contact database. One more thought on this. Consider that a call was recorded at a time when there was a corresponding contact. Later the address book gets changed such that there is no match anymore - perhaps the contact was removed to clean up the address book, perhaps because his or her phone number changed. I would find it convenient if the call history gave me more than just the old phone number in such a case. Ultimately this is a question to the UX designers. My point here is that the caching scheme proposed above would support such a use case, whereas creating the match at the time of the query based on the then current data (as it is presumably done in Tracker/libcommhistory) doesn't. -- Best Regards, Patrick Ohly The content of this message is my personal opinion only and although I am an employee of Intel, the statements I make here in no way represent Intel's position on the issue, nor am I authorized to speak on behalf of Intel on this matter. ___ MeeGo-dev mailing list MeeGo-dev@meego.com http://lists.meego.com/listinfo/meego-dev http://wiki.meego.com/Mailing_list_guidelines
Re: [MeeGo-dev] migration (back) to EDS
On 3/2 Those who need it are messaging-ui, call-history, commhistory-daemon (handling the storage), qt-mobility, qt webruntime, at-phonebook, msgsync. I assumed the dependencies have been checked before announcing the change. Where is this task open and to whom? Is there a bug? What are the other options instead of libcommhistory? the libcommhistory package is not even in any of the images... how can you claim that it's actually this critical? Does the MeeGo dialer even use it? When I spoke with the owner of the dialer app (which is supposedly the key user of all this), this really did not come up as something that is currently in use and working... quite the opposite. (it might work in your maemo OS, but that is of no consideration) architecture decision, and as such, it must have addressed all the basic use cases and consequences _before_ it was made, right? all the cases that we cared about were looked into, risk assessment was made and some detail investigation was scheduled for after the decision for some corner cases that were not deemed critical and where no good current solution was in place. I cannot believe these important dependencies were not considered when the decision was made to move back to EDS... BTW who made the decision (meeting minutes please?), and where is it documented? The MeeGo architecture team made these decisions in consultation with the various handset and tablet architects. I know it's not popular with you and some other @nokia.com folks... but so be it. ___ MeeGo-dev mailing list MeeGo-dev@meego.com http://lists.meego.com/listinfo/meego-dev http://wiki.meego.com/Mailing_list_guidelines
Re: [MeeGo-dev] migration (back) to EDS
To all, On 22/03/11 11:44 PM, ext Arjan van de Ven ar...@linux.intel.com wrote: The MeeGo architecture team made these decisions in consultation with the various handset and tablet architects. Just to be clear, Nokia members of the MeeGo architecture team where not involved in these latest decisions of the MeeGo architecture direction (MSSF/Buteo/PIM). -sakari ___ MeeGo-dev mailing list MeeGo-dev@meego.com http://lists.meego.com/listinfo/meego-dev http://wiki.meego.com/Mailing_list_guidelines
[MeeGo-dev] migration (back) to EDS
Hello! I promised to share some more information about what we intend to do about using EDS in MeeGo. Please let's focus on the technical challenges. There are some exploratory steps at this time, but no specific schedule. Components which are currently in MeeGo are still needed, so please continue to support them with bug fixes. In a nutshell, the goal is to keep QtContacts, QMF and KCalCore as the main APIs used by applications and the higher-level QtMobility APIs. If we can achieve that, very little will have to be done in applications once we change the core components. Calendar Storage * use upstream KDE version of KCalCore * write storage which uses libecal/EDS * add content protection to EDS * remove unnecessary parsing of iCalendar in libecal (not needed by storage and sync) * improve reading of meta data (can speed up sync, see concurrent modifications of items in GUI and EDS database on the evo-hackers list from back in 2009) Contact Storage * write a QtContacts manager which uses libebook/EDS, similar to QtMobility manager for Harmattan, but with some crucial differences: * avoid ID mapping because it requires reading all contacts at startup; must change EDS to use 32 bit integers as local IDs for that (patch ready) * convert between QtContacts and vCard using QtVersit, with EDS specific properties and including custom properties for QtContactDetails which have no other mapping to vCard (same approach as in SyncEvolution QtContacts backend); this avoids lots of hand-written mapping code which depends on the (necessarily incomplete) EContact API * no support for relationships * no support for change tracking, that logic needs to be in higher layers * support for groups open - special contact as done by Evolution itself? * self contact also under debate * change notifications via ebook view * add content protection to EDS * store PHOTO data outside of EDS * remove unnecessary parsing of vCard in libebook (not needed by plugin and sync) * other performance and feature improvements as needed (for example, reading meta data as for calendar above) * correlation with other data sources (online status, communication history) handled by systems above EDS (libfolks?) Obviously we are going down a similar route as Nokia did with Maemo Fremantle. One difference is that all work should be done and reviewed upstream first, and that the (admittedly rather ugly) EDS APIs for manipulating contacts and events in apps would be replaced with more convenient C++ APIs. C code still has the option of using the traditional EDS APIs. Mail Storage and Transports * write simplistic server which runs Camel * replace QMF client library with one which accesses that server; source code compatibility of just the functionality needed by the Tablet mail app would be a good first step -- Best Regards, Patrick Ohly The content of this message is my personal opinion only and although I am an employee of Intel, the statements I make here in no way represent Intel's position on the issue, nor am I authorized to speak on behalf of Intel on this matter. ___ MeeGo-dev mailing list MeeGo-dev@meego.com http://lists.meego.com/listinfo/meego-dev http://wiki.meego.com/Mailing_list_guidelines
Re: [MeeGo-dev] migration (back) to EDS
Hello, On Mon, 21 Mar 2011, ext Patrick Ohly wrote: Mail Storage and Transports * write simplistic server which runs Camel * replace QMF client library with one which accesses that server; source code compatibility of just the functionality needed by the Tablet mail app would be a good first step So, you are talking about QMailServiceActions here. What about mail storage per se? Database etc? -- WBR WBW, Vitaly Repin, Productivity, Nokia-D ___ MeeGo-dev mailing list MeeGo-dev@meego.com http://lists.meego.com/listinfo/meego-dev http://wiki.meego.com/Mailing_list_guidelines
Re: [MeeGo-dev] migration (back) to EDS
On Mo, 2011-03-21 at 15:26 +, Vitaly Repin wrote: On Mon, 21 Mar 2011, ext Patrick Ohly wrote: Mail Storage and Transports * write simplistic server which runs Camel * replace QMF client library with one which accesses that server; source code compatibility of just the functionality needed by the Tablet mail app would be a good first step So, you are talking about QMailServiceActions here. What about mail storage per se? Database etc? I suppose those could be done with the Camel library, but it is currently out of the scope of the initial exploration. Several app developers have already expressed an interest in keeping QMF as storage around. Srini (on CC) is looking into the specific APIs, perhaps he can say more. -- Best Regards, Patrick Ohly The content of this message is my personal opinion only and although I am an employee of Intel, the statements I make here in no way represent Intel's position on the issue, nor am I authorized to speak on behalf of Intel on this matter. ___ MeeGo-dev mailing list MeeGo-dev@meego.com http://lists.meego.com/listinfo/meego-dev http://wiki.meego.com/Mailing_list_guidelines
Re: [MeeGo-dev] migration (back) to EDS
On Mon, 2011-03-21 at 16:44 +0100, Patrick Ohly wrote: On Mo, 2011-03-21 at 15:26 +, Vitaly Repin wrote: On Mon, 21 Mar 2011, ext Patrick Ohly wrote: Mail Storage and Transports * write simplistic server which runs Camel * replace QMF client library with one which accesses that server; source code compatibility of just the functionality needed by the Tablet mail app would be a good first step So, you are talking about QMailServiceActions here. What about mail storage per se? Database etc? I suppose those could be done with the Camel library, but it is currently out of the scope of the initial exploration. Several app developers have already expressed an interest in keeping QMF as storage around. The big plan would be to write a mail data server using Camel and let Tablet mail app without much modifications use a modified client library to connect to the mail data server. If you have some specific questions I can answer those. -Srini. Srini (on CC) is looking into the specific APIs, perhaps he can say more. ___ MeeGo-dev mailing list MeeGo-dev@meego.com http://lists.meego.com/listinfo/meego-dev http://wiki.meego.com/Mailing_list_guidelines
Re: [MeeGo-dev] migration (back) to EDS
From: Patrick Ohly, on 21 March, 2011 16:37 In a nutshell, the goal is to keep QtContacts, QMF and KCalCore as the main APIs used by applications and the higher-level QtMobility APIs. If we can achieve that, very little will have to be done in applications once we change the core components. This will break libcommhistory. What do you suggest? As explained earlier, one technical requirement is that contacts and IM/callhistory data should be in the same database. Options: 1. Support for IM and call history in EDS 2. Duplicate/sync to tracker of contact fields needed for building conversation history and call history 3. Shared memory contacts model with own indexing across domains. 4. Catastrophic messaging performance drop (10-100 times if I remember right from early tracker bugs for missing index). See the queries here: http://gitorious.net/commhistory/libcommhistory/trees/master/sparql Where else do you want to see performance improvements by moving Contacts to EDS, if not in the most performance critical areas (e.g. IM conversational view)? This use case shall be solved by the ones who made the architecture decision ruining it. Which one should it be, and who will do the work? Regards, Zoltan Calendar Storage * use upstream KDE version of KCalCore * write storage which uses libecal/EDS * add content protection to EDS * remove unnecessary parsing of iCalendar in libecal (not needed by storage and sync) * improve reading of meta data (can speed up sync, see concurrent modifications of items in GUI and EDS database on the evo-hackers list from back in 2009) Contact Storage * write a QtContacts manager which uses libebook/EDS, similar to QtMobility manager for Harmattan, but with some crucial differences: * avoid ID mapping because it requires reading all contacts at startup; must change EDS to use 32 bit integers as local IDs for that (patch ready) * convert between QtContacts and vCard using QtVersit, with EDS specific properties and including custom properties for QtContactDetails which have no other mapping to vCard (same approach as in SyncEvolution QtContacts backend); this avoids lots of hand-written mapping code which depends on the (necessarily incomplete) EContact API * no support for relationships * no support for change tracking, that logic needs to be in higher layers * support for groups open - special contact as done by Evolution itself? * self contact also under debate * change notifications via ebook view * add content protection to EDS * store PHOTO data outside of EDS * remove unnecessary parsing of vCard in libebook (not needed by plugin and sync) * other performance and feature improvements as needed (for example, reading meta data as for calendar above) * correlation with other data sources (online status, communication history) handled by systems above EDS (libfolks?) Obviously we are going down a similar route as Nokia did with Maemo Fremantle. One difference is that all work should be done and reviewed upstream first, and that the (admittedly rather ugly) EDS APIs for manipulating contacts and events in apps would be replaced with more convenient C++ APIs. C code still has the option of using the traditional EDS APIs. Mail Storage and Transports * write simplistic server which runs Camel * replace QMF client library with one which accesses that server; source code compatibility of just the functionality needed by the Tablet mail app would be a good first step -- Best Regards, Patrick Ohly The content of this message is my personal opinion only and although I am an employee of Intel, the statements I make here in no way represent Intel's position on the issue, nor am I authorized to speak on behalf of Intel on this matter. ___ MeeGo-dev mailing list MeeGo-dev@meego.com http://lists.meego.com/listinfo/meego-dev http://wiki.meego.com/Mailing_list_guidelines smime.p7s Description: S/MIME cryptographic signature ___ MeeGo-dev mailing list MeeGo-dev@meego.com http://lists.meego.com/listinfo/meego-dev http://wiki.meego.com/Mailing_list_guidelines
Re: [MeeGo-dev] migration (back) to EDS
On Mo, 2011-03-21 at 17:45 +, zoltan@nokia.com wrote: From: Patrick Ohly, on 21 March, 2011 16:37 In a nutshell, the goal is to keep QtContacts, QMF and KCalCore as the main APIs used by applications and the higher-level QtMobility APIs. If we can achieve that, very little will have to be done in applications once we change the core components. This will break libcommhistory. I'm aware of that, and I don't claim that there is an answer. There is a task open for those who need libcommhistory (or something like it) to look into this problem and comment on the current proposals. -- Best Regards, Patrick Ohly The content of this message is my personal opinion only and although I am an employee of Intel, the statements I make here in no way represent Intel's position on the issue, nor am I authorized to speak on behalf of Intel on this matter. ___ MeeGo-dev mailing list MeeGo-dev@meego.com http://lists.meego.com/listinfo/meego-dev http://wiki.meego.com/Mailing_list_guidelines
Re: [MeeGo-dev] migration (back) to EDS
On Mon, Mar 21, 2011 at 3:36 PM, Patrick Ohly patrick.o...@intel.com wrote: Mail Storage and Transports * write simplistic server which runs Camel * replace QMF client library with one which accesses that server; source code compatibility of just the functionality needed by the Tablet mail app would be a good first step Why do you need to use Camel instead of QMF? Does Camel have some virtues over QMF messageserver? ___ MeeGo-dev mailing list MeeGo-dev@meego.com http://lists.meego.com/listinfo/meego-dev http://wiki.meego.com/Mailing_list_guidelines
Re: [MeeGo-dev] migration (back) to EDS
On Mon, 2011-03-21 at 20:54 +0100, Patrick Ohly wrote: On Mo, 2011-03-21 at 17:45 +, zoltan@nokia.com wrote: From: Patrick Ohly, on 21 March, 2011 16:37 In a nutshell, the goal is to keep QtContacts, QMF and KCalCore as the main APIs used by applications and the higher-level QtMobility APIs. If we can achieve that, very little will have to be done in applications once we change the core components. This will break libcommhistory. I'm aware of that, and I don't claim that there is an answer. There is a task open for those who need libcommhistory (or something like it) to look into this problem and comment on the current proposals. Ok, so.. I'll need something like libcommhistory. Check. What are the current proposals for things like libcommhistory? (this is a real question. You're the architect: don't sidestep it again) 1. Support things like IM and call history in EDS? 2. Synchronize contact data in EDS to Tracker's RDF store? 3. Solutions that involve shared memory with EDS? 4. A big drop in messaging performance? 5. A big drop in capabilities? Applications can no longer make queries like these: http://gitorious.net/commhistory/libcommhistory/trees/master/sparql 6. ... 7. Profit? If I know what the proposal is, then I can comment on it. This part is opinion: I find it rather strange. A working solution that does scale well, that performs as good and soon probably better than E-D-S, that supports more use-cases and application domains than even necessary, that is already integrated and very well tested, that has been or is being optimized to meet the performance requirements of each and every use-case and that does support the query flexibility required by applications .. Is being replaced by: o. A solution that needs 20 bullet point task items before even the solution itself actually works. Most of those task items are vague and it's all but clear how they are to be implemented o. that has less support for all the use-cases by its design (being a PIM data -only storage) o. that can not support all application domains unless rather large U-turns are made in the very root of the design o. that has not been optimized for btrfs, as apparently even something as simple as sensible fsync() use is done wrong in E-D-S (see Adrien's measurements on btrfs with and without libeatmydata). https://mail.gnome.org/archives/tracker-list/2011-March/msg00035.html o. A solution that makes all the IPC mistakes possible with D-Bus: o. Passing data, not messages over D-Bus: congestion on the bus o. Passing data marshalled as a DBusMessage instead of FD passing o. Not using D-Bus's signaling mechanism where appropriate o. A solution that has not been tested whatsoever against neither the use-cases, nor the APIs o. Basically, a solution that needs more time to be shaped into something workable (workable doesn't mean works well) than any release time line for a mobile product permits since the last ten years or more. It's strange mathematics. But, ok .. Cheers, Philip -- Philip Van Hoof freelance software developer Codeminded BVBA - http://codeminded.be ___ MeeGo-dev mailing list MeeGo-dev@meego.com http://lists.meego.com/listinfo/meego-dev http://wiki.meego.com/Mailing_list_guidelines