Pre note: This is about the Xesam support being done (since this week) in the indexer-split.
About:
Xesam requires notifying live searches about changes that affect them.
We plan to implement this with a "events" table that journals all
creates, deletes and updates that the indexer causes.
Periodically we will handle and then flush the items in that events
table.
I made a cracktasty diagram that contains the from-a-high-distance
abstract proposal that we have in mind for this.
This is pseudo code that illustrates the periodic handler:
bool periodic_handler (...)
{
lock indexer
update eventstable set beinghandled=1 where 1=1 (all items)
unlock indexer
foreach query in all livequeries
added, modified, removed = query.execute-on (eventstable)
query.emit_added (added)
query.emit_removed (removed)
query.emit_modified (modified)
done
lock indexer
delete from eventstable where beinghandled = 1
unlock indexer
return (!stopping)
}
Here's a piece of IRC log between me and jamiecc about the proposal:
pvanhoof ping jamiemcc
pvanhoof same thing
pvanhoof I'll make a pdf
jamiemcc oh ok
pvanhoof Sending
pvanhoof ok
pvanhoof so
pvanhoof it's about the hitsadded, hitsremoved and hitsmodified signals for
xesam
pvanhoof What we have in mind is using a "events" table that is a journal for
all creates, deletes and updates
pvanhoof Periodically we will flush that table, each create (insert), update
and each delete we add a record in that table
pvanhoof We'll make sure the table is queryable in a similar fashion as how the
Xesam query will execute
pvanhoof In the periodical handler we'll for each live search check whether it
got affected by the items in the events table
pvanhoof In pseudo, the handler:
jamiemcc sounds feasible
pvanhoof gboolean periodic_handler (void data) {
pvanhoof lock indexer
pvanhoof update eventstable set beinghandled=1 where 1=1 (all items)
pvanhoof unlock indexer
pvanhoof foreach query in all live queries
pvanhoof added, modified, removed = query.execute-on (eventstable)
pvanhoof query.emit_added (added)
pvanhoof query.emit_removed (removed)
pvanhoof query.emit_modified (modified)
pvanhoof done
pvanhoof lock indexer
pvanhoof delete from eventstable where beinghandled = 1
pvanhoof unlock indexer
pvanhoof }
pvanhoof I've send you a diagram that you can look at as if it's a
state-activity one, a ERD and a class diagram :) now how cool is that?? :)
pvanhoof it's just three columns, although the ERD is quite simplistic of course
jamiemcc yeah just go tit
* fritschy ([EMAIL PROTECTED]) has left #tracker
pvanhoof so, the current idea is to adapt those stored procedures into
transactions that will also add this record to the "events" table
* fritschy ([EMAIL PROTECTED]) has joined #tracker
pvanhoof Which might not be sufficient, and we kinda lack the in-depth know-how
of all the db handling of tracker
pvanhoof So that's a first issue we want to discuss with you
pvanhoof The other is stopping the indexing, restarting it (locking it, in the
pseudo code): what you think about that
jamiemcc ok I will need to think about it - I iwll probably reply later tonight
and we can discuss tomorrow
pvanhoof I adapted my initial proposal to have two short critical sections
rather than letting the entire periodic handler be one critical section
pvanhoof that way the lock is smaller
jamiemcc the indexer will be seaparte process so will need to be locked via
dbus signals
pvanhoof by just adding a column to the events table
pvanhoof yes but I guess we want any such locking to be short
jamiemcc well yes
pvanhoof then once the items that are to be handled are identified, we for each
live-search check whether the live-search is affected
pvanhoof and we perform the necessary hitsadded, hitsremoved and hitsmodified
signals if needed
pvanhoof if all is done, we simply purge the handled items from the events table
jamiemcc the query results will be store din temp tables
pvanhoof which is the second location where we want the indexer to be locked-out
jamiemcc remember a query may be a cursor so wont include entire result set
pvanhoof No okay, but that's something the check needs to worry about
pvanhoof so ottela is working on a query for the live-search
jamiemcc ok cool
pvanhoof and if we only want to update if the client has the affected item
visible, due to cursor-usage
pvanhoof then i guess we'll somehow need to get that info into trackerd
jamiemcc any reason we dont store whats change din memory rather than sqlite
table?
pvanhoof oh, that's abstract right now
jamiemcc o
jamiemcc ok
pvanhoof "tracker's event table" can also be a hashtable for me ..
jamiemcc yeah fine
pvanhoof implementation detail
pvanhoof since it doesn't need to be persistent ...
pvanhoof difference is that either we use a memory table and still a
transaction for the three stored procedures
pvanhoof or we adapt code
jamiemcc prefer hashtable as amount of data will be small
jamiemcc can even be a list
pvanhoof ok, your comments/ideas on this would of course be very useful btw
jamiemcc yeah I will think about it more tonight and get back to you
pvanhoof sounds great
pvanhoof I'll make a mail about this to the mailing list? or I await your ideas
tomorrow?
pvanhoof I'll just wait for now
jamiemcc you cna mail if you like
jamiemcc I will reply to it
--
Philip Van Hoof, freelance software developer
home: me at pvanhoof dot be
gnome: pvanhoof at gnome dot org
http://pvanhoof.be/blog
http://codeminded.be
tracker-xesam-hitssignals.dia
Description: application/dia-diagram
_______________________________________________ tracker-list mailing list tracker-list@gnome.org http://mail.gnome.org/mailman/listinfo/tracker-list
