Le Mon, 07 Mar 2011 22:40:10 +0100, Philip Van Hoof <phi...@codeminded.be> a écrit :
> On Mon, 2011-03-07 at 08:09 -0800, Arjan van de Ven wrote: > > Hi Arjan, > > > PIM storage > > =========== > > The Address book, Calendar data and Email are currently stored in a > > tracker database, and accessed (officially) via a QtMobility API > > set. There are a range of issues with this implementation, starting > > with the complexity of adding privacy controls, the performance and > > scalability as well as the completeness for doing a proper syncml > > sync. > > That's great! > > Can you explain us, how does EDS improve privacy control compared to > Tracker's RDF store? Does EDS have a way to represent different graphs > (part of Tracker's future plans to do per-resource privacy control)? > > How in EDS do you say that data (about) x isn't to be made > available to queries made by application y? It's the first > time that I hear that EDS has such a privacy related capabilities. > > Can you also go a bit deeper into how EDS's performance compares to > Tracker's RDF store for comparable data entry operations? Have you > guys done measurements on this? Can the results of these measurements > and the method to reproduce be publicized? Are the queries you used > available somewhere? Where? > > What do you mean with scalability? Scalable over multiple application > domains other than PIM? Or scalable to more data? And what kind of > data? I wonder how EDS is or can be more scalable when the underlying > database technology (SQLite, apparently) is identical. > > In Tracker is each class stored in a decomposed table. This means that > contact and E-mail data are each in separate tables. Although > decomposed typically makes INSERT a bit slower, SELECT is a lot > faster. How will or does EDS make SELECT equally 'scalable' with > equally complex queries? > > Also note that the Tracker team optimizes both INSERT and SELECT > use-cases on a use-case per use-case basis, using its domainindexes > and indexes. Have you contacted the Tracker team about the scalability > problems (if any)? > > Or is EDS only more scalable during data entry? > > > Because of all these items and the available expertise, we have > > decided to start replacing PIM storage with the Evolution Data > > Server. > > Ah, I see. > > > This change will land together with the SyncEvolution change (due > > to the intimate relationship between the storage and sync of PIM > > data). This change should largely be invisible to applications > > since applications are supposed to access this data via the > > appropriate QtMobility APIs. > > That's not entirely true. Many applications are nowadays accessing > said data through the QSparql API of Harmattan. QtMobility isn't > always the ideal access-layer. Especially not for more complex > use-cases. > > Are you planning to support or implement a QSparql backend for EDS? > > I'd be thrilled to have a query language like or as powerful as SPARQL > in EDS! In what timeframe can we expect this? Are you planning to use > existing SPARQL endpoint technology for this? Which? > > > But to avoid setting precedents, the lower level components will > > be removed from the architecture diagrams effective immediately. > > > > To be clear, this does not mean that "tracker" is completely > > removed; tracker is still being used (together with tumbler) for > > indexing media on the device. At this point we are seeing serious > > issues (performance/stability) with this solution, but the first > > attempt will be to fix the > > deficiencies rather than a replacement. > > At this moment is Tracker (via QSparql and earlier but deprecated > libqttracker) used on Harmattan within quite a wide variety of > (horizontal) applications. > > Are the plans to only use Tracker for indexing? That's a fairly small > use-case of Tracker on Harmattan nowadays. Fremantle's metadata > solution of course had an almost exclusive focus on file metadata > (indexing). But that's not the case on Harmattan. What's the plan for > those apps? > > > > Cheers! > > Philip > Just to complement Philip's message, here's my take on the subject: Comparing Tracker and EDS is a bit like comparing a hitch-hiker's backpack and a purse. The backpack can definitely store your credit card, and the purse too. It is probably easier to put the card in your purse than to take off the backpack, open the good pocket etc. Now the question is, do you need a purse or a backpack? If we go on the technical field, Tracker as a solution for contacts is still a solution that is evolving a lot (you might argue that then it's not stable/mature, I won't contradict you if you say it's still under development). With our current versions, we fetch 5000 contacts (when I say contact, I mean a full blown contact, many phone numbers, mail addresses etc.) in around 15 seconds, and store them in around 80 seconds. That is not lightning fast. You can do better with a SQLite database. But then we're getting back to the backpack and the purse. Now, here are various cases that we resolved with Tracker: 1. Storing any detail any symbian phone can invent. You might laugh at that one, look at the complexity of our contact schema and you'll understand :) If we reduce the scope of our schema, we obviously get faster. 2. Storing contacts from many sources and differentiating them: we store contacts from local address book, from various services (mail for exchange etc.). Sync services are able to sync back only meaningful contacts. 3. Lossless merging and unmerging of contacts: we can track the origin of fields in a very fine grained way (useful when merging local/IM/service contacts) 4. Schema flexibility: Tracker handles ontology changes at no cost 4. Robustness: If the DB gets corrupted, Tracker has a journal, like a filesystem, and can replay it. Here are some things we half solved: 1. IM contacts handling: we can save, merge, edit etc. IM contacts. IM contacts are handled the same way as local contacts, in a unified way. That makes showing a roster super easy. On the not so good side, the way we handle transient data (presence etc.) is maybe currently not optimal (stored on disk in the Tracker DB). Some people have been presenting folks as an alternative: while I think folks in an interesting project, it separates the information in two sources, so you can't query information in a unified way anymore. 2. Restricted access to data: so far we need a separate DB, that sucks. That has not been investigated seriously yet in Tracker. Synchronization is a solved problem, mind you, our mail for exchange plugin does a pretty good job at saving contacts in a slow way. Using APIs the right way, it could sync 500 contacts in 80 seconds, as mentioned above. Now for the last part, supposing you had the courage to read the above blurb: here are several valid hypothetical reasons I see to **pick EDS over Tracker** 1. Tracker is not a mainstream technology, and all its expertise is concentrated in Nokia. Given Nokia's "commitment" to the future of MeeGo, choosing Tracker is a risk unless you recruit skilled people to work on it 2. Horizontality is not in your requirements: you don't care about crossing data from different domains. That is an architecture decision, and is perfectly valid. 3. MeeGo has never received the love it should have received from Nokia, so the contacts backend state in MeeGo has always been poor. For having been asked to play the firefighters in that domain, I know how poor it was. Rare updates, low reactivity on bugs, etc. Only recently (two or three months ago) did Nokia deem worth putting a real team on MeeGo work. I chatted with Patrick Oulry (not sure of your last name Patrick, my apologies if I butchered it) at FOSDEM, and he told me how frustrated he was with the state of things. I understand him. I am ashamed of the behaviour of Nokia there. After the last part logically comes the conclusion: this mail is not a plea for Tracker in MeeGo. It is a sad assessment that Nokia, in spite of the good will of some of its people, failed at properly collaborating on MeeGo. Collaborating did not mean drawing powerpoints, and dropping code and letting it rot. It meant maintaining MeeGo in a proactive way, which I have not seen (in contacts). Now I'm also a bit worried of the affirmative nature of the original message of this thread. To me it sounds like "the Intel architects decided that MeeGo will be that way". I don't exactly how this kind of decision is supposed to be taken, and while I understand you can't ask everybody's opinion for all decisions, I see little transparency there. Hopefully somebody will enlighten me! I also hope in the future MeeGo architects will ask the right people when they have questions on a software piece. Regards Adrien _______________________________________________ MeeGo-dev mailing list MeeGo-dev@meego.com http://lists.meego.com/listinfo/meego-dev http://wiki.meego.com/Mailing_list_guidelines
