Hi there, I've done a performance analysis and comparison of Tracker's RDF store versus E-D-S (Evolution Data Server).
I'll first make a summary of what I think is necessary for readers to know and understand because both Tracker's RDF store and E-D-S are in fact different products service a similar but different purpose. o. The VCard that I'm testing with for E-D-S isn't as complex as the Nepomuk-ontology based contact that the tests are saving in the RDF example for Tracker. This means that this performance test isn't perfectly compared. But I'm not a VCard expert myself. And results were rather clear (for me) early on. I have included the source code of both test softwares in attachment so that you can match the complexity to produce better numbers. o. Given that E-D-S is a single-purpose store for PIM data we expected E-D-S to perform much better than Tracker's RDF store. Tracker is of course not a single-purpose store. It can store all within the realm of Nepomuk's ontology (which has a lot more application and use-case domains than PIM) It is not the case that E-D-S performs much better, as the report below will illustrate. o. I personally expected E-D-S to scale to large amounts of contacts. But a simple loop of adding 2x 1000 contacts makes Evolution's UI go flat on its face. The UI doesn't respond anymore at all. The only thing that helps is a evolution --force-shutdown. The Evolution UI also makes the entire Desktop unresponsive. This is important for for example a platform like MeeGo: a version of the Evolution UI _does_ run on MeeGo devices (making it part of E-D-S as solution on aforementioned platform -- the UI can and often will be running, my poor batteries). o. For a very very long time after the test.c 1000 contacts loop has finished is Evolution running at 95% CPU. Doing who knows what (draining massive amounts of power, in any case) This makes me conclude that the API is cheating and returning earlier than allowed. Maybe not everything is finished, and I wonder what would happen if the system would crash. Would all data be guaranteed to be persistently stored on the storage hardware? Tracker's RDF store guarantees that at return of the API the data will be stored (it has a journal for this). o. After+while the data has been/is being entered into the store can Tracker's RDF store perform more complex queries and at the same time it can allow queries that cross multiple use-case and application domains (deeply link contacts to IM, E-mail, Photos, Videos, Events, GEO locations, and many more classes _and_ query using the links). This is something that E-D-S, given that it's a single-purpose store, can't offer at this moment. Nor does E-D-S provide a rich query lang. like SPARQL for this purpose. E-D-S is more a get and set system with a rather flat query language. We didn't test query performance this time. Given the huge differences in capabilities it's probably not an interesting comparison the make between Tracker's RDF store and E-D-S. With Tracker's GraphUpdated you can do a auto-update-model comparable to EBookView's API. QSparql have such a model for Qt implemented. o. Both E-D-S and Tracker's RDF store have at this moment the same or comparable security capabilities. With E-D-S you can have multiple addressbooks, with Tracker's RDF store you can have either GRAPHs or grouping with a nie:DataSource; or any other typical Nepomuk technique for this purpose (like adding a addressbook class, if necessary -- there are plenty of ways for this). A few somewhat larger tests: ---------------------------- o. Tracker's RDF store with its newest INSERT OR REPLACE support: INSERT OR REPLACE is not yet available in master). For reference is the original that doesn't use INSERT OR REPLACE also being ran. The support for INSERT OR REPLACE is in branch sparql-update. More info on INSERT OR REPLACE here: http://pvanhoof.be/blog/index.php/2011/03/09/a-replace-extension-for-trackers-sparqls-update pvanhoof@lors:~/repos/gnome/tracker/master/tests/functional-tests/ipc$ ./test-insert-or-replace REPLACE: 1000 contacts: 12.997943 ORIGINAL: 1000 contacts: 13.850410 pvanhoof@lors:~/repos/gnome/tracker/master/tests/functional-tests/ipc$ ./test-insert-or-replace REPLACE: 1000 contacts: 15.442745 ORIGINAL: 1000 contacts: 27.525495 pvanhoof@lors:~/repos/gnome/tracker/master/tests/functional-tests/ipc$ ./test-insert-or-replace REPLACE: 1000 contacts: 17.257888 ORIGINAL: 1000 contacts: 27.712315 pvanhoof@lors:~/repos/gnome/tracker/master/tests/functional-tests/ipc$ The GraphUpdated stuff was cleanly being emitted, dbus-daemon was not being flooded as GraphUpdated is engineered for these volumes of data deltas. More info on how GraphUpdated works here: http://pvanhoof.be/blog/index.php/2010/08/24/trackers-new-class-signal-system-being-developed o. And now E-D-S: I had to kill Evolution's UI because after the second run was my entire Desktop computer completely unresponsive and was Evolution's shell using 95% CPU. I do mention this because for just 2x 1000 contacts is this behaviour in my opinion unacceptable. I know I will receive a lot of typical hate for saying this, but the Evolution team shouldn't feel too proud of this. In my opinion should NO amount of input make it possible to let Evolution's UI hang. pvanhoof@lors:~$ gcc test.c `pkg-config libebook-1.2 --cflags --libs` pvanhoof@lors:~$ ./a.out EDS 1000 contacts: 10.604454 pvanhoof@lors:~$ ./a.out EDS 1000 contacts: 11.362855 pvanhoof@lors:~$ evolution --force-shutdown No response from Evolution -- killing the process That "No response from Evolution" should illustrate how bad the situation actually became; the process is not even responding to a private IPC asking it to cleanly shutdown. Forcing the tool to use the kernel's KILL instead. Yikes. Some smaller tests (yields smaller differences too): ---------------------------------------------------- Luckily didn't Evolution die and crash on these smaller tests ... pvanhoof@lors:~/repos/gnome/tracker/master/tests/functional-tests/ipc$ ./test-insert-or-replace REPLACE: 100 contacts: 1.348496 ORIGINAL: 100 contacts: 2.839032 pvanhoof@lors:~/repos/gnome/tracker/master/tests/functional-tests/ipc$ cd /home/pvanhoof/ pvanhoof@lors:~$ gcc test.c `pkg-config libebook-1.2 --cflags --libs` pvanhoof@lors:~$ ./a.out EDS 100 contacts: 1.000067 pvanhoof@lors:~$ ./a.out EDS 100 contacts: 0.886793 pvanhoof@lors:~$ ./a.out EDS 100 contacts: 0.902376 pvanhoof@lors:~$ cd ~/repos/gnome/tracker/master/tests/functional-tests/ipc pvanhoof@lors:~/repos/gnome/tracker/master/tests/functional-tests/ipc$ ./test-insert-or-replace REPLACE: 100 contacts: 1.375554 ORIGINAL: 100 contacts: 2.631252 pvanhoof@lors:~/repos/gnome/tracker/master/tests/functional-tests/ipc$ ./test-insert-or-replace REPLACE: 100 contacts: 1.448647 ORIGINAL: 100 contacts: 2.700024 pvanhoof@lors:~/repos/gnome/tracker/master/tests/functional-tests/ipc$ ./test-insert-or-replace REPLACE: 100 contacts: 1.400238 ORIGINAL: 100 contacts: 2.787517 pvanhoof@lors:~/repos/gnome/tracker/master/tests/functional-tests/ipc$ I'd love to receive performance numbers from different people, using different circumstances. Please feel free to use Tracker's mailing list for publicizing your results. Cheers, Philip -- Philip Van Hoof freelance software developer Codeminded BVBA - http://codeminded.be
#include <libebook/e-book.h> const gchar *vcard = "BEGIN:VCARD\n" "VERSION:3.0\n" "CLASS:PUBLIC\n" "REV:2011-03-14T13:47:25Z\n" "FN:First %d Last %d\n" "N:Last %d;First %d;;;\n" "EMAIL;WORK:rho...@example.com%d\n" "TEL;TYPE=cell,voice:02141730585%d\n" "NOTE;ENCODING=QUOTED-PRINTABLE:c1f1b12d-bc75-4d45-9a1f-b1efe934409f\n" "END:VCARD\n"; static EContact * new_test_contact (guint i) { EContact *contact; gchar *str = g_strdup_printf (vcard, i,i,i,i,i,i); contact = e_contact_new_from_vcard (str); g_free (str); return contact; } EBook *book; int main () { GError *error = NULL; guint i, y = 1000; GTimer *timer; g_type_init (); timer = g_timer_new (); book = e_book_new_default_addressbook (&error); if (error) { g_critical ("%s\n", error->message); return 0; } e_book_open (book, FALSE, &error); if (error) { g_critical ("%s\n", error->message); return 0; } g_timer_start (timer); for (i = 0; i < y; i++) { EContact *contact = new_test_contact (i); e_book_add_contact (book, contact, &error); if (error) { g_critical ("%s\n", error->message); g_object_unref (contact); return 0; } e_book_commit_contact (book, contact, &error); if (error) { g_critical ("%s\n", error->message); g_object_unref (contact); return 0; } g_object_unref (contact); } g_timer_stop (timer); g_print ("EDS %d contacts: %f\n", y, g_timer_elapsed (timer, NULL)); g_object_unref (book); }
using GLib; using Tracker; using Tracker.Sparql; const string insert_query_replace = " DELETE { ?r nao:hasProperty ?property . } WHERE { ?r a nco:PhoneNumber; nco:phoneNumber \"02141730585%d\"; nao:hasProperty ?property . } DELETE { ?r a nco:CarPhoneNumber, nco:BbsNumber, nco:PagerNumber, nco:VideoTelephoneNumber, nco:MessagingNumber, nco:VoicePhoneNumber, nco:CellPhoneNumber, nco:FaxNumber, nco:ModemNumber . } WHERE { ?r a nco:PhoneNumber; nco:phoneNumber \"02141730585%d\" . } INSERT { _:tel a nco:PhoneNumber ; nco:phoneNumber \"02141730585%d\" . } WHERE { OPTIONAL { ?r a nco:PhoneNumber; nco:phoneNumber \"02141730585%d\" . } FILTER(!bound(?r)) . } DELETE { <mailto:rho...@example.com%d> ?predicate ?object . } WHERE { <mailto:rho...@example.com%d> ?predicate ?object . FILTER(?predicate IN (nco:emailAddress)) . } INSERT { <mailto:rho...@example.com%d> a nco:EmailAddress ; nco:emailAddress \"rho...@example.com%d\" . } DELETE { <contact:r:%d> nco:hasAffiliation ?e } WHERE { <contact:r:%d> nco:hasAffiliation ?e } INSERT OR REPLACE { _:af1 a nco:Affiliation ; rdfs:label \"Work\" ; nco:hasEmailAddress <mailto:rho...@example.com%d> . _:af2 a nco:Affiliation ; rdfs:label \"Other\" ; nco:hasPhoneNumber ?tel . <contact:r:%d> a nco:PersonContact ; nco:nameHonorificPrefix \"Mrs.\" ; nco:nameGiven \"First %d\" ; nco:nameFamily \"Last %d\" ; nco:hasAffiliation _:af1 ; nco:hasAffiliation _:af2 ; nie:generator \"addressbook\" ; nie:contentLastModified \"2011-03-14T13:47:25\" ; nie:contentCreated \"2011-03-14T13:47:25\" ; nco:contactUID \"c1f1b12d-bc75-4d45-9a1f-b1efe934409f\" . } WHERE { ?tel nco:phoneNumber \"02141730585%d\" } "; const string insert_query_orig = " DELETE { <mailto:ho...@example.com%d> ?predicate ?object . } WHERE { <mailto:ho...@example.com%d> ?predicate ?object . FILTER(?predicate IN (nco:emailAddress)) . } DELETE { ?resource nao:hasProperty ?property . } WHERE { ?resource nao:hasProperty ?property ; nco:phoneNumber ?key . FILTER(?key IN (\"2141730585%d\")) . } DELETE { ?resource a nco:CarPhoneNumber, nco:BbsNumber, nco:PagerNumber, nco:VideoTelephoneNumber, nco:MessagingNumber, nco:VoicePhoneNumber, nco:CellPhoneNumber, nco:FaxNumber, nco:ModemNumber . } WHERE { ?resource a nco:PhoneNumber ; nco:phoneNumber ?key . FILTER(?key IN (\"2141730585%d\")) . } INSERT { _:_ a nco:PhoneNumber ; nco:phoneNumber \"2141730585%d\" . } WHERE { OPTIONAL { ?resource a nco:PhoneNumber ; nco:phoneNumber \"2141730585%d\" . } FILTER(!bound(?resource)) . } DELETE { GRAPH <urn:uuid:08070f5c-a334-4d19-a8b0-12a3071bfab9> { <contact:o:%d> ?predicate ?object . } } WHERE { GRAPH <urn:uuid:08070f5c-a334-4d19-a8b0-12a3071bfab9> { <contact:o:%d> ?predicate ?object . FILTER(?predicate NOT IN (nco:contactLocalUID,nco:contactUID,rdf:type)) . } } INSERT { GRAPH <urn:uuid:08070f5c-a334-4d19-a8b0-12a3071bfab9> { <mailto:ho...@example.com%d> a nco:EmailAddress ; nco:emailAddress \"ho...@example.com%d\" . _:_Affiliation_Work1 a nco:Affiliation ; rdfs:label \"Work\" ; nco:hasEmailAddress <mailto:ho...@example.com%d> . ?_PhoneNumber_Resource3 a nco:VoicePhoneNumber, nco:PhoneNumber ; nco:phoneNumber \"2141730585%d\" . _:_Affiliation_Other2 a nco:Affiliation ; rdfs:label \"Other\" ; nco:hasPhoneNumber ?_PhoneNumber_Resource3 . <contact:o:%d> a nco:PersonContact ; nco:nameHonorificPrefix \"Mrs.\" ; nco:nameGiven \"First %d\" ; nco:nameFamily \"Last %d\" ; nco:hasAffiliation _:_Affiliation_Work1 ; nco:hasAffiliation _:_Affiliation_Other2 ; nie:generator \"addressbook\" ; nie:contentLastModified \"2011-03-14T13:47:25\" ; nie:contentCreated \"2011-03-14T13:47:25\" ; nco:contactUID \"c1f1b12d-bc75-4d45-9a1f-b1efe934409f\" . } } WHERE { ?_PhoneNumber_Resource3 nco:phoneNumber \"2141730585%d\" } "; int main( string[] args ) { uint i, y = 1000; Timer t1 = new Timer (); Timer t2 = new Timer (); Connection c; c = Connection.get (); t1.start (); for (i = 0; i < y; i++) { c.update (insert_query_replace.printf (i,i,i,i,i,i,i,i,i,i,i,i,i,i,i)); } t1.stop (); print ("REPLACE: %u contacts: %f\n", y, t1.elapsed()); t2.start (); for (i = 0; i < y; i++) { c.update (insert_query_orig.printf (i,i,i,i,i,i,i,i,i,i,i,i,i,i,i,i)); } t2.stop (); print ("ORIGINAL: %u contacts: %f\n", y, t2.elapsed()); return 0; }
_______________________________________________ tracker-list mailing list tracker-list@gnome.org http://mail.gnome.org/mailman/listinfo/tracker-list