On Fri, 2011-04-01 at 11:57 +0200, Mathias Hasselmann wrote: > Am Dienstag, den 29.03.2011, 16:35 +0200 schrieb Patrick Ohly:
[CUT] > This will become very useful when implementation communication history. > For Maemo5 we used a detached, custom sqlite database for storing > communication history. Bringing pieces together, keeping them in sync, > avoiding problems from concurrent access, getting reasonable response > times and memory consumptions where a permanent problem with that > solution. > > By using tracker as data store, we get the data aggregation for free, > and we have to solve most concurency and performance problems at justone > single place. > > Similar things apply for presence information. It might feel strange to > put such information into a data store, but Tracker permits marking RDF > properties as volatile. So presence information should not hit the disk > if desired. Reminds me I have to push the Tracker guys harder to finally > repair transient property support. We have transient property support. The reason why it's not using memory but instead is using the normal meta.db is because with direct-access you have multiple processes connecting to the same .db file. When you do this, you can't use SQLite's TEMPORARY and still share the data between different processes. TEMPORARY in SQLite is per process. We tried using a transient.db in /dev/shm and then ATTACH-ing that .db file in SQLite for each process that connects, but this was several times slower for a yet unknown reason. We still have this code somewhere in a branch, but because it was slower we decided not to use it. We do delete the last-session values of transient properties at startup of tracker-store and we don't journal values of transient properties. Given that meta.db is in .cache (and is considered cache), nothing is written in the persistent data (in .local) when writing transient props. Backup also doesn't back up values of persistent properties nor will consequentially restore restore them. We don't have many properties set as tracker:transient, though. Only a few IM related ones. We don't believe that I/O introduced by these is going to be problematic in any way. > > Disadvantages of Tracker: > > > data protection missing in both EDS and Tracker, but less obvious how > > to implement it in Tracker > The solution proposed for EDS is protecting its database files. Same > solution can be applied to tracker. Since you can deal without data > aggregation at this moment, you should not loose anything if we create a > separate Tracker database for contacts and apply same file level > protection as for EDS. It's by the way very easy to start from libtracker-data and implement your own tracker-store using its own SQLite database and etc. This should be doable in a few lines of code, and in Vala too (like tracker-store itself is). You can let that one use its own D-Bus name or even let it use its own special IPC mechanism. And protect all this as much as you want. The effort to do this is by no means larger than the effort for adding some sort of protection to EDS, the results are the same. In Harmattan the database of Tracker is already protected using UNIX file permissions and Aegis (if processes aren't in the group metadata-users, which only Aegis can grant to them, then they can't access meta.db - then they must go over D-Bus to do queries). Note that with Aegis is also D-Bus protectable. > If you look at the future Tracker can provider significantly better data > protection than EDS. Tracker supports RDF graphs per resource property > (EDS speak: vCard attribute). Code can be written to restrict access for > each single graph. Virtual SQLite tables should do the job. If this > should be implemented, you could aggregate information from different > data sources into one single contact, and still could make sure, that > for instance only the Addressbook itself can see information retrieved > from e.g. Facebook. Considering privace concerns this is a very > significant feature, which cannot be reasonable implemented with EDS - > AFAIK. Not trivial to implement, this idea, but mostly correct. Yes. > > slow write performance in QtContacts-Tracker (?) > Please give concrete test data. Like Philip Van Hoof demonstrated, it > rather seems writing performance of EDS and Tracker seems to be > comparable - which is not surprising that I/O is the limiting factor. > With Tracker's recent addition of sparql-update (INSERT OR REPLACE > statement) we have seen performance improvments of up to 25%. This would > set QtContactsTracker ahead of EDS. My tests are indeed showing that Tracker is beating EDS in every field: Storage speed, a lot more flexible query capability and query speed. In every sense is Tracker scaling better than EDS. These are verifiable and reproduceable measurements and the code of the tests for Tracker is available in Tracker's repository (the EDS tests are available in the mailing list archives). On a FS that has a slow fsync, EDS is actually problematically slow. > Also note that the filebackend of > EDS doesn't seem to implement vcard attribute indexes yet. Adding them > to EDS to get search performance comparable to Tracker would add another > performance penalty to EDS. Indeed. All tests that I did with Tracker last few weeks and days where "with" indexes on various properties related to contacts use-cases for Tracker, and "without" any such indexes for EDS. And yet was Tracker faster in storage too. Cheers, Philip -- Philip Van Hoof freelance software developer Codeminded BVBA - http://codeminded.be _______________________________________________ MeeGo-dev mailing list MeeGo-dev@meego.com http://lists.meego.com/listinfo/meego-dev http://wiki.meego.com/Mailing_list_guidelines
