On 08/01/13 11:56, Philip Van Hoof wrote:
On Tue, 2013-01-08 at 11:40 +0000, Martyn Russell wrote:
I guess putting INSERT queries upfront the metadata one needs to be
possible as well as getting a separate query for in case the MTP daemon
wants to do multiple passes. ie. First protocol based metadata insert,
which needs the parent URNs at that point and which doesn't yet need the
file content to be available, then extraction based metadata insert; for
which the file content must be available.
Not really. If that insert fails, then you're not guaranteeing the
parent is inserted, which we do at the moment.
Why would the insert fail other than something being a bug that must be
fixed?
In the past, inserts have failed because the extracted metadata is not
in line with the ontology of the day OR because the database is fscked.
So it's usually a corner case anyway. Sometimes it's a limitation or
restraint we have in the DB, like unique nie:url cases - but that's more
about the code being wrong than something we can't control.
Separate queries for the parent are really the way to go here. Once
that's done you have to handle failures and retries before the child can
be added, etc. This is why the miner-fs has a lot of code in this area
to handle this stuff. Ideally, it would be good to make use of that code
here so we don't have to fix issues we've already fixed in the miner-fs.
And what this creates must also be compatible with what miner-fs will
otherwise create.
Yea I agree.
The idea here is that while a (large) file is being transferred,
protocol based metadata is already in the Nepomuk store available for
applications that are interested in tracker:available-false 'to be
deployed' data. ie. a DivX or DVD being transferred and a movie
application saying: "DVD with title XYZ with actors ABC and written by
DEF is being transferred to your phone (20% completed)". And After that
an extraction to get extra metadata that isn't included in the protocol
based metadata (and all that without needing miner-fs).
We did actually have support for this in the past with inotify and half
downloaded files, but Nokia decided not to "notify" about half
downloaded files after a while. I think this was an area which changed a
few times due to requirements. In the end performance was one of the
reasons (due to spamming of updates with everything else going on at the
same time).
Right. Nokia's requirements might not be everybody else's requirements
though.
Yea, indeed.
And with this miner-fs isn't involved. The MTP daemon has better control
than inotify over the file-transfer to know when to update and when not
to update.
Sure. Though Nokia IIRC wanted updates to things like nfo:fileSize and
done so we didn't spam too much but had updates every n seconds, etc.
BTW. the '20% completed' does not belong in Nepomuk and needs to be
communicated differently (it's application state information).
Yea, I thought it might.
--
Regards,
Martyn
Founder and CEO of Lanedo GmbH.
_______________________________________________
tracker-list mailing list
tracker-list@gnome.org
https://mail.gnome.org/mailman/listinfo/tracker-list
