Hi Martyn, just back from SambaXP and rereading your great reply. Thanks again for taking time!
Am 12.05.2014 um 16:18 schrieb Martyn Russell <mar...@lanedo.com>: >> Hm. But then, how made tracker-extract made aware of filesystem >> event, ie new files? I though tracker-miner-fs sets up the watches > > So: > > 1. tracker-miner-fs uses inotify (and other backends), it is signalled on a > new file. > > 2. tracker-miner-fs processes that file adding basic information (file size, > etc). > > 3. tracker-extract is started on login and listens for GraphUpdated, a signal > from tracker-store. The tracker-store process is the only one that can insert > data. tracker-extract also queries for a list of IDs of resources on start up > of the ones that it won't be notified of and which have no nie:data-source > set. These are the files it knows it needs to process. > > 4. tracker-extract then runs with that list of files and also queues any new > files from the GraphUpdated signal. > >> and notifies other processes (tracker-extract) via DBUS IPC ? > > We used to use DBus and that would instantiate processes for us, but now we > have this "passive" approach. It has advantages and disadvantages. The > advantage is, tracker-miner-fs is simpler and doesn't have to deal with the > mess of broken files crashing tracker-extract and timeouts, etc. Also it > means the file basic information is added to Tracker quicker than it was > previously because we have no delay on tracker-extract. Another advantage > half realised (bits can be improved still), is that we can group process > files of the same time and prioritise files of particular types. We have mime > type priorities currently I think. Disadvantages include having to wait for > the second pass indexing to complete :) > > tracker-extract has _ALWAYS_ crashed because of the nature of the files and > libraries we're using being the weakest point. Yeah, I know, been there. :) > With this in mind, it's quite bad that we don't have some sort of watchdog or > way to keep it up and running. We could easily do this in tracker-miner-fs. > Part of me thinks we should actually have a tracker-control --daemon which > watches all process and keeps some sort of journal / log going and also > maintains processes like tracker-extract keep running. I should add, this is > something that nearly EVERY embedded solution does itself with some script > which keeps miner-fs running (not even tracker-extract), because there is no > other way to guarantee Tracker is there to catch all events. Ok, thanks. So taking up on this approach I wrote two Solaris SMF service manifests (resembles systemd) for tracker-miner-fs and tracker-extract. This works in that it restarts crashing tracker-extract and then tracker-extract continues indexing. Unfortunately this doesn't work for files that crash it reproducibly as you mention below: > The only downsides I see are repeated attempts causing a circular problem > with tracker-extract. Definitely. For uses with possibly many files causing crashes this renders Tracker useless. > But that's easily remedied, we had to do something like this in > tracker-miner-fs before already. So I'm taking you on this here, that you can easily fix this please! :) If you don't feel an apparent need to get this fixed for your own usecases, please let me know, so I can try to bring affected parties and their pockets together next week. :) Thanks again! -Ralph _______________________________________________ tracker-list mailing list tracker-list@gnome.org https://mail.gnome.org/mailman/listinfo/tracker-list
