On Thu, 2013-04-11 at 15:53 +0530, Vishesh Handa wrote:
>
> I'm curious as to how you handle -
>
>
> 1. Type Inference - Say something like this 'select ?r where { ?r a
> nco:Contact . }. Lets say one has some 10 nco:Contacts and some 15
> nco:PersonContacts. In this case one would have to iterate over both
> the tables. Does tracker do that?
This gets translated to something like
SELECT Uri FROM "nco:Contact", to translate from nco:Contact to the SQL
table "nco:Contact" is something Tracker does internally. I think there
is a environment variable that you can turn on to print the SQLite
statements the first time they are prepared (there is a LRU cache of
SQLite statements). These statements include the query.
Type interference itself (select ?p ?o { nco:Contact ?p ?o }) works
because Tracker stores its own ontology in itself, and an ontology are
just a bunch of rdf statements.
> 2. Property Inference - Do you handle cases such as 'select ?r ?l
> where { ?r nao:prefLabel ?l . }', where the 'nao:prefLabel' has not
> been explicitly defined. Lets assume that the 'nie:title' has been
> set.
> The nie:title is a rdfs:subPropertyOf nao:prefLabel
I don't remember.
> Does tracker handle cases like this? Cause this is a rather common
> usecase in Nepomuk where we want to fetch a good label for the
> resource and we do not want to query specific properties.
Afaik yes.
> 3. I read that you have some support for graphs - How is that
> implemented? From what I understand from your db schema each property
> has their own column, so I'm not sure where you would store the graph
> related info.
The graph support is limited. More explanation on its limitations here:
https://live.gnome.org/Tracker/Documentation/SparqlFeatures#Named_Graphs
> Also, does tracker use graphs for any purpose?
Only for storing the origin of a statement (which was the only required
use-case for the N9, which we were mainly targeting while developing
Tracker's SPARQL endpoint and Nepomuk ontology support).
> In the Nepomuk KDE world we generally use graphs to group triples
> based on which application has stored the information. This is
> especially useful in the case of indexing. When a file has been
> modified and needs to be re-indexed, we need to throw away the
> previous data and re-index it. The file could in this case have both
> indexed and non-indexed data such as tags and ratings. So, we only
> remove the statements that were added by the indexer and then reindex
> the file.
This isn't supported by Tracker.
> This seems like a very common use case. I'm curious as to how tracker
> solves this problem.
It doesn't. Full graph support was not a design consideration, only
limited support for it was.
Kind regards,
Philip
> Then the libtrackersparql makes a WAL SQLite connection,
> parses your
> SPARQL and generates on the fly SQL for that. This happens
> often using
> subqueries, and without building an AST first - making the
> parse-translate phase relatively fast and resource friendly,
> which is a
> design-choice as Tracker is indented to run on devices with
> few
> resources.
>
> This code does that:
>
> https://git.gnome.org/browse/tracker/tree/src/libtracker-data/tracker-sparql-query.vala
>
> That design-choice of course has a draw back in that the
> queries have to
> manually optimized very often and/or to get things fast enough
> we often
> had to store data in a rendundant way. We did this with domain
> specific
> indexes which I explained in this blog item:
>
>
> http://pvanhoof.be/blog/index.php/2010/07/07/domain-indexes-finished-technical-conclusions
>
> http://pvanhoof.be/blog/index.php/2010/07/03/sqlites-wal-deleting-a-domain-specific-index
>
> http://pvanhoof.be/blog/index.php/2010/07/01/working-on-domain-specific-indexes
>
>
> This function is the code that translates a statement (from
> the SPARQL
> UPDATE we make RDF statements and then we process those) to
> SQL inserts:
>
>
> https://git.gnome.org/browse/tracker/tree/src/libtracker-data/tracker-data-update.c#n732
>
> A lot of the SPARQL UPDATE part is here:
>
> https://git.gnome.org/browse/tracker/tree/src/libtracker-data/tracker-sparql-pattern.vala
>
> Note that we have a buffer where we eliminate duplicates like:
>
> <a> a Class.
> <a> Prop1 Value1.
> <a> a Class.
> <a> Prop2 Value2.
> <a> Prop1 Value3.
>
> We translate that to:
>
> <a> a Class; Prop1 Value; Prop2 Value3.
>
> Except when we utilize the null support:
>
>
> http://pvanhoof.be/blog/index.php/2011/08/09/support-for-null-with-trackers-insert-or-replace-feature
>
> http://pvanhoof.be/blog/index.php/2011/08/15/null-support-for-insert-or-replace-available-in-master
>
> The second blog item and my own comments in the first
> illustrate the
> problem with optimizing statement-sets like above just like
> that.
>
> Then we also added a few SQLite functions which are used by
> the
> SPARQL->SQL translation and for most of our own SPARQL
> extensions (we
> ship with a bunch of SPARQL extensions that allow query
> writers to make
> queries faster):
>
>
> https://git.gnome.org/browse/tracker/tree/src/libtracker-data/tracker-db-interface-sqlite.c#n826
>
> The data-manager is mostly about creating the SQL tables and
> preparation. It can also handle a limited amount of ontology
> changes
> (adding and removing of classes and properties and their
> extensions like
> domain specific indexes):
>
> https://git.gnome.org/browse/tracker/tree/src/libtracker-data/tracker-data-manager.c
>
>
>
> Thanks for all this information
>
>
>
> Kind regards,
>
> Philip
>
>
> On Thu, 2013-04-11 at 05:18 +0530, Vishesh Handa wrote:
> > Hey Philip
> >
> > I'm one of the KDE Nepomuk developers. I've been looking
> into the
> > tracker project for some time now, since it's good to know
> how other
> > people internally implement things. However, it has been
> very hard for
> > me to find any documentation on the inner working of
> tracker.
> >
> >
> > I'm specifically interested in how the database schema is
> designed. I
> > did find this "Semantic Social Desktop and Mobile Devices"
> > presentation [1] which gives a very rough overview of how
> each class
> > has its own table.
> >
> >
> > Could you perhaps point me to some internal documentation?
> It would be
> > most helpful. Otherwise, could I ask your some detailed
> questions
> > about the tracker internals?
> >
> > I have looked at the source code, but it's a little hard to
> understand
> > for a new comer.
> >
> > [1]
> https://live.gnome.org/Tracker/Documentation/Presentations
> >
> > --
> > Vishesh Handa
> >
>
>
>
>
>
>
> --
> Vishesh Handa
>
--
Philip Van Hoof
Software developer
Codeminded BVBA - http://codeminded.be
_______________________________________________
tracker-list mailing list
tracker-list@gnome.org
https://mail.gnome.org/mailman/listinfo/tracker-list
