Re: [Tracker] How to use tracker:modified right

2014-09-15 Thread Philip Van Hoof
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

On 15/09/2014 12:43, Martyn Russell wrote:
> On 12/09/14 15:47, Philip Van Hoof wrote:
>> -BEGIN PGP SIGNED MESSAGE- Hash: SHA1
>> 
>> Having been asked to review this bug, it came to my attention
>> that the tracker:modified feature is not well understood by all.
> 
> First, thanks for writing this.

Np Martyn.

> It's now on the WIKI: 
> https://wiki.gnome.org/Projects/Tracker/Documentation/SparqlInternals

Awesome.
> 
Thanks for putting it there.

> and this is linked to on the main Documentation page.

> Thanks Philip!

np :)

Thanks for pointing me to the bug as I was not aware people at GNOME
were actually using this.

Kind regards,

Philip

-BEGIN PGP SIGNATURE-
Version: GnuPG v2.0.20 (MingW32)

iQEcBAEBAgAGBQJUFxZdAAoJEEP2NSGEz4aD7R4IAJvbs48/Bvsf2rOLLugm4diw
ADWuUFSMfdLD/LQ1svR+K2Itv52oWpaYHWWCY++BxzNuyMa0EXu3Ey7cXJHhZnKh
vMsxSQalIlBOxZQ+lQ6pzlL/wkhCx/EEVEM2NZBd4r4qDMqhlJ2tVaSna6YDqLEM
9DRw/tkZaMT8ej4HmcBVjIKBsXb0CaHHBeg/YzB0l95n7zkTfBVnCRBLv3p4Q2Xu
XMBmumS0Pg70+XVHugMXGPBppfhbfr1PyWa73BY7FGic278UIFV3oQeMoL5DthKJ
YWR8xL9yTvzBWbCHXYGl3TtjR/+Slk/yR3jrs40ynRR7PzwhzB97wLWKKxgM9T4=
=XiZh
-END PGP SIGNATURE-
___
tracker-list mailing list
tracker-list@gnome.org
https://mail.gnome.org/mailman/listinfo/tracker-list


Re: [Tracker] How to use tracker:modified right

2014-09-15 Thread Martyn Russell

On 12/09/14 15:47, Philip Van Hoof wrote:

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

Having been asked to review this bug, it came to my attention that the
tracker:modified feature is not well understood by all.


First, thanks for writing this.

It's now on the WIKI:
  https://wiki.gnome.org/Projects/Tracker/Documentation/SparqlInternals

and this is linked to on the main Documentation page.

Thanks Philip!

--
Regards,
Martyn

Founder & Director @ Lanedo GmbH.
http://www.linkedin.com/in/martynrussell
___
tracker-list mailing list
tracker-list@gnome.org
https://mail.gnome.org/mailman/listinfo/tracker-list


[Tracker] How to use tracker:modified right

2014-09-12 Thread Philip Van Hoof
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

Having been asked to review this bug, it came to my attention that the
tracker:modified feature is not well understood by all.

https://bugzilla.gnome.org/show_bug.cgi?id=727759

I did write it down, though ;-). Perhaps certain people ignored that.

http://pvanhoof.be/blog/index.php/2011/01/31/synchronizing-your-applications-data-with-trackers-rdf-store

How to use it would be like in this example:

Say you wanted to create a stored hashmap to your
filesystem as title-cache.txt. You'd do this:

select ?s ?title ?modseq { ?s nie:title ?title ;
tracker:modified ?modseq }

You write title-cache.txt with ?s and ?title and you write a file
title-cache-modseq.txt that contains the max value of all ?modseq you saw.


Time passes and titles gets changed in tracker's RDF store. Your
application wants a new title-cache.txt. But it doesn't want to fetch
all titles and subjects, it just wants to know the ones that got changed.

So it reads title-cache-modseq.txt and takes that max value of last
time, let's call it $last_modseq. Now it does this:

select ?s ?title ?modseq { ?s nie:title ?title ; tracker:modified
?modseq . FILTER { ?modseq > $last_modseq } }

It updates title-cache.txt with the ?s and ?title it received (it
received a delta, not the complete list, due to the FILTER). And it
again writes the max value of all ?modseq it received in
title-cache-modseq.txt

This can go on to keep title-cache.txt up to date. It's an unsigned
64bit number and we reset it to 0 after overflow.

The brilliant minds now understand that if ever their $last_modseq is
larger than the ?modseq of the resource you most recently changed, we
had the 64bit-u-number overflow in tracker-store. We didn't design to
make detecting that overflow easy or something, because while hoping
to reach the history books too we thought that unsigned 64bit ought to
be enough for everybody.

If you guys after having done unsigned-64bit amount of transactions on
tracker, must have your title-cache.txt updated with least amount of
delta, I'm sure you can also get the time to develop detecting the
overflow.

I don't think tracker:modified should be used for any other purpose than
synchronization like explained above.


-BEGIN PGP SIGNATURE-
Version: GnuPG v2.0.20 (MingW32)

iQEcBAEBAgAGBQJUEwd9AAoJEEP2NSGEz4aDu00H/0rEJhmFXAqz1Zh/xZVkh5d/
npdQahXLzZ2JU84K9TO0/TaKzJoj38lU2wuQ2gNV/JWyp3llLL0kLjug3LxwOrDU
3xJy5C2xLolrm+aIeeaSRTqJIxdg2Hp3acaHGX+79R+fV7d/ky0w0ExvX7YM1Up3
Si2bQPbW8MLLT+VJAnkRbYMqv0EkI8AToBWbW1ppQtQJ9L3HOsGh1DZQKDfHrfvI
/LRT/6fXHmtLkmKD2TBv2PVD0VIs+2qUDZ1Mn3aWimpr0p/zVEY6jeu6DFexfLUH
AgX8U7lMVaCtfWc/x6YO1JWiWYpirYttyW+yLHN3i0a15b9Wgny/FMxdwI1zK0Y=
=T3jz
-END PGP SIGNATURE-
___
tracker-list mailing list
tracker-list@gnome.org
https://mail.gnome.org/mailman/listinfo/tracker-list