Re: [Tracker] [Evolution-hackers] [Evolution] Beagle and Tracker, letting Evolution feed those beasts RDF triples instead
2008/12/10 Michael Meeks > Hi Philip, > > On Tue, 2008-12-09 at 19:59 +0100, Philip Van Hoof wrote: > > > http://live.gnome.org/Evolution/Metadata > > > > For early visitors of that page, refresh because I have added/changed > > quite a lot of it already. > > Looks really good. > But I have some comments to attach and some bike sheds to paint :-) Comments on the proposal below. >The only thing that I don't quite understand (the perennial problem > with asynchronous interfaces), is the memory issue: it seems we need to > store all Unset information on deleted mails somewhere [ unless you are > a womble like me that keeps ~all mail forever ;-]. > Is it that big a problem? I mean if you store 100,000 uris of avg. length 50 chars you will have a file about 5mb... One needs only keep an absolute minimum amount of metadata around. That aside here are my comments: * I had personally expected something more like a harvesting API, but what you present is more like a metadata writing API. Much like the set of writing methods in http://xesam.org/main/XesamMetadataAPI. This may just be a matter of taste though. * If we are talking a "harvesting" metaphor (which we may not be) then it seems wrong to imply in the methods that Evo is writing the data and not just sending it * I thought Tracker needed a service type to select the right table. Of course we know we are dealing with emails here, but the API fails to generalize. * Timestamps are not mandatory in the protocol. The receiver will have to parse the payload to extract a timestamp (which is not guaranteed to be there in the first place). Without a timestamp a payload is meaningless * Is there any form of sorting mandated by the SetMany method? Fx sorting subjects by mtime? * About the Set method. What happens to predicates set on the subject that are not in the 'predicates' arg? Are they kept or deleted? Or to rephrase the question do you overwrite the entire set of metadata for the subject? I have cooked together something that should work as a generic harvesting API here: http://xesam.org/main/Drafts/XesamPMH. It should address all of the points I raise here too. -- Cheers, Mikkel ___ tracker-list mailing list tracker-list@gnome.org http://mail.gnome.org/mailman/listinfo/tracker-list
Re: [Tracker] Support in Tracker for ultra-new Evolution installs that use SQLite for the summary format
I just posted a new version of the patch which now opens the DB in read-only mode. Thanks for the suggestion On Fri, 2008-12-12 at 15:12 +0800, Jerry Tan wrote: > Hi, philip > > How about to open folders.db in readonly mode? > > I just check sqlite's API. > sqlite3_open_v2 support a para with readonly flag > > > > On 2008ǯ12·î12Æü 00:34, Philip Van Hoof wrote: > > This patch makes ultra-new Evolution installs work again with Tracker. > > > > There's one problem and that is that the query will only find E-mails in > > the INBOX folder. You can easily find the Query and figure out what the > > problem is: > > > > The design that Carlos made assumes that for each folder there's a > > "summary" file. In the new Evolution cache format there's just one > > "folders.db" for each account. > > > > I could do a generated UNION select after first doing "select * from > > folders" on folders.db and then generating a query that includes all > > folders. I just have not done this for now and instead I'm just using > > INBOX and I'm neglecting the other folders. > > > > This is NOT the same as the proposal that I am doing at (a). This is > > instead a ad-hoc solution for the new situation (Evolution using SQLite > > for the summaries). I find this solution rather nasty, to be honest. > > > > (a) http://live.gnome.org/Evolution/Metadata > > > > For Carlos: I have also fixed a serious problem in evolution-pop.c, > > which is by the way unaffected by Evolution's changes (and works, if you > > just apply the patch that I included in this larger patch). The POP > > support's get_message_metadata was not returning metadata. > > > > This was crashing my tracker-indexer (as seemingly my compiler was > > putting "return 0x2" where the return was omitted, and the memory I have > > at 0x2 didn't dereference TrackerModuleMetadata's members very well). > > > > Please review and/or rework the patch. > > > > > > > > > > > > > > ___ > > tracker-list mailing list > > tracker-list@gnome.org > > http://mail.gnome.org/mailman/listinfo/tracker-list > -- Philip Van Hoof, freelance software developer home: me at pvanhoof dot be gnome: pvanhoof at gnome dot org http://pvanhoof.be/blog http://codeminded.be ___ tracker-list mailing list tracker-list@gnome.org http://mail.gnome.org/mailman/listinfo/tracker-list
Re: [Tracker] Support in Tracker for ultra-new Evolution installs that use SQLite for the summary format
On Fri, 2008-12-12 at 01:59 +0100, Philip Van Hoof wrote:
> On Thu, 2008-12-11 at 23:58 +0200, Ivan Frade wrote:
> > hi philip,
> >
> > short comment (the tablet is not the best device to review patches):
> > can we call this module "evolution224" and keep also the old module?
> > we could choose between one or other with a flag in compilation time
> > (the distributions will choose the correct for them)
>
> With this patch the code will detect the format and automatically choose
> the right way to parse both.
>
This version of the patch adds support for IMAP subfolders and fixes the
fact that in the last patch the "email://" URI was generated wrong (it
still contained the "folders.db" string-piece).
> > regards,
> >
> > Ivan
> >
> > On 12/11/08, Philip Van Hoof wrote:
> > > This patch makes ultra-new Evolution installs work again with Tracker.
> > >
> > > There's one problem and that is that the query will only find E-mails in
> > > the INBOX folder. You can easily find the Query and figure out what the
> > > problem is:
> > >
> > > The design that Carlos made assumes that for each folder there's a
> > > "summary" file. In the new Evolution cache format there's just one
> > > "folders.db" for each account.
> > >
> > > I could do a generated UNION select after first doing "select * from
> > > folders" on folders.db and then generating a query that includes all
> > > folders. I just have not done this for now and instead I'm just using
> > > INBOX and I'm neglecting the other folders.
> > >
> > > This is NOT the same as the proposal that I am doing at (a). This is
> > > instead a ad-hoc solution for the new situation (Evolution using SQLite
> > > for the summaries). I find this solution rather nasty, to be honest.
> > >
> > > (a) http://live.gnome.org/Evolution/Metadata
> > >
> > > For Carlos: I have also fixed a serious problem in evolution-pop.c,
> > > which is by the way unaffected by Evolution's changes (and works, if you
> > > just apply the patch that I included in this larger patch). The POP
> > > support's get_message_metadata was not returning metadata.
> > >
> > > This was crashing my tracker-indexer (as seemingly my compiler was
> > > putting "return 0x2" where the return was omitted, and the memory I have
> > > at 0x2 didn't dereference TrackerModuleMetadata's members very well).
> > >
> > > Please review and/or rework the patch.
> > >
> > > --
> > > Philip Van Hoof, freelance software developer
> > > home: me at pvanhoof dot be
> > > gnome: pvanhoof at gnome dot org
> > > http://pvanhoof.be/blog
> > > http://codeminded.be
> > >
> >
--
Philip Van Hoof, freelance software developer
home: me at pvanhoof dot be
gnome: pvanhoof at gnome dot org
http://pvanhoof.be/blog
http://codeminded.be
Index: src/tracker-indexer/modules/evolution-pop.c
===
--- src/tracker-indexer/modules/evolution-pop.c (revision 2701)
+++ src/tracker-indexer/modules/evolution-pop.c (working copy)
@@ -360,6 +360,8 @@
}
g_list_free (list);
+
+ return metadata;
}
static TrackerModuleMetadata *
Index: src/tracker-indexer/modules/evolution.c
===
--- src/tracker-indexer/modules/evolution.c (revision 2701)
+++ src/tracker-indexer/modules/evolution.c (working copy)
@@ -70,7 +70,9 @@
strchr (basename, '.') == NULL) {
type = MAIL_STORAGE_LOCAL;
} else if (g_str_has_prefix (path, imap_dir) &&
- strcmp (basename, "summary") == 0) {
+ (strcmp (basename, "summary") == 0 ||
+ strcmp (basename, "folders.db") == 0) ||
+ strcmp (basename, "cmeta") == 0) {
type = MAIL_STORAGE_IMAP;
}
Index: src/tracker-indexer/modules/Makefile.am
===
--- src/tracker-indexer/modules/Makefile.am (revision 2701)
+++ src/tracker-indexer/modules/Makefile.am (working copy)
@@ -16,7 +16,8 @@
$(GIO_CFLAGS) \
$(GLIB2_CFLAGS) \
$(GCONF_CFLAGS) \
- $(GMIME_CFLAGS)
+ $(GMIME_CFLAGS) \
+ $(SQLITE3_CFLAGS)
indexer_modules_LTLIBRARIES = \
libtracker-module-applications.la\
@@ -67,6 +68,7 @@
$(GMODULE_LIBS) \
$(GLIB2_LIBS) \
$(GCONF_LIBS) \
- $(GMIME_LIBS)
+ $(GMIME_LIBS) \
+ $(SQLITE3_LIBS)
endif
Index: src/tracker-indexer/modules/evolution-imap.c
===
--- src/tracker-indexer/modules/evolution-imap.c (revision 2701)
+++ src/tracker-indexer/modules/evolution-imap.c (working copy)
@@ -128,7 +128,18 @@
g_free (file->imap_dir);
g_free (file->cur_message_uid);
-fclose (file->summary);
+ if (file->db) {
+ sqlite3_close (file->db);
+ }
+
+ if (file->stmt) {
+ sqlite3_finalize (file->stmt);
+ }
+
+ if (file->summary) {
+ fclose (file->summary);
+ }
+
close (file->fd);
G_OBJECT_CLASS (tracker_evolution_imap_file_parent_class)->finalize (object);
@@ -482,10 +
