[Tracker] Support in Tracker for ultra-new Evolution installs that use SQLite for the summary format
This patch makes ultra-new Evolution installs work again with Tracker.
There's one problem and that is that the query will only find E-mails in
the INBOX folder. You can easily find the Query and figure out what the
problem is:
The design that Carlos made assumes that for each folder there's a
"summary" file. In the new Evolution cache format there's just one
"folders.db" for each account.
I could do a generated UNION select after first doing "select * from
folders" on folders.db and then generating a query that includes all
folders. I just have not done this for now and instead I'm just using
INBOX and I'm neglecting the other folders.
This is NOT the same as the proposal that I am doing at (a). This is
instead a ad-hoc solution for the new situation (Evolution using SQLite
for the summaries). I find this solution rather nasty, to be honest.
(a) http://live.gnome.org/Evolution/Metadata
For Carlos: I have also fixed a serious problem in evolution-pop.c,
which is by the way unaffected by Evolution's changes (and works, if you
just apply the patch that I included in this larger patch). The POP
support's get_message_metadata was not returning metadata.
This was crashing my tracker-indexer (as seemingly my compiler was
putting "return 0x2" where the return was omitted, and the memory I have
at 0x2 didn't dereference TrackerModuleMetadata's members very well).
Please review and/or rework the patch.
--
Philip Van Hoof, freelance software developer
home: me at pvanhoof dot be
gnome: pvanhoof at gnome dot org
http://pvanhoof.be/blog
http://codeminded.be
Index: src/tracker-indexer/modules/evolution-pop.c
===
--- src/tracker-indexer/modules/evolution-pop.c (revision 2701)
+++ src/tracker-indexer/modules/evolution-pop.c (working copy)
@@ -360,6 +360,8 @@
}
g_list_free (list);
+
+ return metadata;
}
static TrackerModuleMetadata *
Index: src/tracker-indexer/modules/evolution.c
===
--- src/tracker-indexer/modules/evolution.c (revision 2701)
+++ src/tracker-indexer/modules/evolution.c (working copy)
@@ -70,7 +70,8 @@
strchr (basename, '.') == NULL) {
type = MAIL_STORAGE_LOCAL;
} else if (g_str_has_prefix (path, imap_dir) &&
- strcmp (basename, "summary") == 0) {
+ (strcmp (basename, "summary") == 0 ||
+ strcmp (basename, "folders.db") == 0)) {
type = MAIL_STORAGE_IMAP;
}
Index: src/tracker-indexer/modules/Makefile.am
===
--- src/tracker-indexer/modules/Makefile.am (revision 2701)
+++ src/tracker-indexer/modules/Makefile.am (working copy)
@@ -16,7 +16,8 @@
$(GIO_CFLAGS) \
$(GLIB2_CFLAGS) \
$(GCONF_CFLAGS) \
- $(GMIME_CFLAGS)
+ $(GMIME_CFLAGS) \
+ $(SQLITE3_CFLAGS)
indexer_modules_LTLIBRARIES = \
libtracker-module-applications.la\
@@ -67,6 +68,7 @@
$(GMODULE_LIBS) \
$(GLIB2_LIBS) \
$(GCONF_LIBS) \
- $(GMIME_LIBS)
+ $(GMIME_LIBS) \
+ $(SQLITE3_LIBS)
endif
Index: src/tracker-indexer/modules/evolution-imap.c
===
--- src/tracker-indexer/modules/evolution-imap.c (revision 2701)
+++ src/tracker-indexer/modules/evolution-imap.c (working copy)
@@ -128,7 +128,18 @@
g_free (file->imap_dir);
g_free (file->cur_message_uid);
-fclose (file->summary);
+ if (file->db) {
+ sqlite3_close (file->db);
+ }
+
+ if (file->stmt) {
+ sqlite3_finalize (file->stmt);
+ }
+
+ if (file->summary) {
+ fclose (file->summary);
+ }
+
close (file->fd);
G_OBJECT_CLASS (tracker_evolution_imap_file_parent_class)->finalize (object);
@@ -494,25 +505,62 @@
NULL);
path = g_file_get_path (tracker_module_file_get_file (file));
-self->fd = tracker_file_open (path, TRUE);
-g_free (path);
-if (self->fd == -1) {
-return;
-}
+ self->db = NULL;
+ self->summary = NULL;
+ self->stmt = NULL;
-self->summary = fdopen (self->fd, "r");
-self->n_messages = read_summary_header (self->summary);
-self->cur_message = 1;
+ if (g_str_has_suffix (path, ".db")) {
+ /* New SQLite based format */
+ sqlite3_stmt *stmt;
+ gint result = SQLITE_OK;
-if (self->n_messages > 0) {
-/* save current message uid */
-read_summary (self->summary,
- SUMMARY_TYPE_STRING, &self->cur_message_uid, /* message uid */
- -1);
-}
+ sqlite3_open (path, &self->db);
-ensure_imap_accounts ();
+ sqlite3_prepare_v2 (self->db, "select saved_count from folders where folder_name = 'INBOX'",
+-1, &stmt, NULL);
+
+ result = sqlite3_step (stmt);
+ self->n_messages = sqlite3_column_int (stmt, 0);
+ self->cur_message = 1;
+
+ if (self->n_mes
Re: [Tracker] Support in Tracker for ultra-new Evolution installs that use SQLite for the summary format
hi philip, short comment (the tablet is not the best device to review patches): can we call this module "evolution224" and keep also the old module? we could choose between one or other with a flag in compilation time (the distributions will choose the correct for them) regards, Ivan On 12/11/08, Philip Van Hoof wrote: > This patch makes ultra-new Evolution installs work again with Tracker. > > There's one problem and that is that the query will only find E-mails in > the INBOX folder. You can easily find the Query and figure out what the > problem is: > > The design that Carlos made assumes that for each folder there's a > "summary" file. In the new Evolution cache format there's just one > "folders.db" for each account. > > I could do a generated UNION select after first doing "select * from > folders" on folders.db and then generating a query that includes all > folders. I just have not done this for now and instead I'm just using > INBOX and I'm neglecting the other folders. > > This is NOT the same as the proposal that I am doing at (a). This is > instead a ad-hoc solution for the new situation (Evolution using SQLite > for the summaries). I find this solution rather nasty, to be honest. > > (a) http://live.gnome.org/Evolution/Metadata > > For Carlos: I have also fixed a serious problem in evolution-pop.c, > which is by the way unaffected by Evolution's changes (and works, if you > just apply the patch that I included in this larger patch). The POP > support's get_message_metadata was not returning metadata. > > This was crashing my tracker-indexer (as seemingly my compiler was > putting "return 0x2" where the return was omitted, and the memory I have > at 0x2 didn't dereference TrackerModuleMetadata's members very well). > > Please review and/or rework the patch. > > -- > Philip Van Hoof, freelance software developer > home: me at pvanhoof dot be > gnome: pvanhoof at gnome dot org > http://pvanhoof.be/blog > http://codeminded.be > ___ tracker-list mailing list tracker-list@gnome.org http://mail.gnome.org/mailman/listinfo/tracker-list
Re: [Tracker] Support in Tracker for ultra-new Evolution installs that use SQLite for the summary format
On Thu, 2008-12-11 at 23:58 +0200, Ivan Frade wrote: > hi philip, > > short comment (the tablet is not the best device to review patches): > can we call this module "evolution224" and keep also the old module? > we could choose between one or other with a flag in compilation time > (the distributions will choose the correct for them) With this patch the code will detect the format and automatically choose the right way to parse both. > regards, > > Ivan > > On 12/11/08, Philip Van Hoof wrote: > > This patch makes ultra-new Evolution installs work again with Tracker. > > > > There's one problem and that is that the query will only find E-mails in > > the INBOX folder. You can easily find the Query and figure out what the > > problem is: > > > > The design that Carlos made assumes that for each folder there's a > > "summary" file. In the new Evolution cache format there's just one > > "folders.db" for each account. > > > > I could do a generated UNION select after first doing "select * from > > folders" on folders.db and then generating a query that includes all > > folders. I just have not done this for now and instead I'm just using > > INBOX and I'm neglecting the other folders. > > > > This is NOT the same as the proposal that I am doing at (a). This is > > instead a ad-hoc solution for the new situation (Evolution using SQLite > > for the summaries). I find this solution rather nasty, to be honest. > > > > (a) http://live.gnome.org/Evolution/Metadata > > > > For Carlos: I have also fixed a serious problem in evolution-pop.c, > > which is by the way unaffected by Evolution's changes (and works, if you > > just apply the patch that I included in this larger patch). The POP > > support's get_message_metadata was not returning metadata. > > > > This was crashing my tracker-indexer (as seemingly my compiler was > > putting "return 0x2" where the return was omitted, and the memory I have > > at 0x2 didn't dereference TrackerModuleMetadata's members very well). > > > > Please review and/or rework the patch. > > > > -- > > Philip Van Hoof, freelance software developer > > home: me at pvanhoof dot be > > gnome: pvanhoof at gnome dot org > > http://pvanhoof.be/blog > > http://codeminded.be > > > -- Philip Van Hoof, freelance software developer home: me at pvanhoof dot be gnome: pvanhoof at gnome dot org http://pvanhoof.be/blog http://codeminded.be ___ tracker-list mailing list tracker-list@gnome.org http://mail.gnome.org/mailman/listinfo/tracker-list
Re: [Tracker] Support in Tracker for ultra-new Evolution installs that use SQLite for the summary format
Hi, philip How about to open folders.db in readonly mode? I just check sqlite's API. sqlite3_open_v2 support a para with readonly flag On 2008ǯ12·î12Æü 00:34, Philip Van Hoof wrote: This patch makes ultra-new Evolution installs work again with Tracker. There's one problem and that is that the query will only find E-mails in the INBOX folder. You can easily find the Query and figure out what the problem is: The design that Carlos made assumes that for each folder there's a "summary" file. In the new Evolution cache format there's just one "folders.db" for each account. I could do a generated UNION select after first doing "select * from folders" on folders.db and then generating a query that includes all folders. I just have not done this for now and instead I'm just using INBOX and I'm neglecting the other folders. This is NOT the same as the proposal that I am doing at (a). This is instead a ad-hoc solution for the new situation (Evolution using SQLite for the summaries). I find this solution rather nasty, to be honest. (a) http://live.gnome.org/Evolution/Metadata For Carlos: I have also fixed a serious problem in evolution-pop.c, which is by the way unaffected by Evolution's changes (and works, if you just apply the patch that I included in this larger patch). The POP support's get_message_metadata was not returning metadata. This was crashing my tracker-indexer (as seemingly my compiler was putting "return 0x2" where the return was omitted, and the memory I have at 0x2 didn't dereference TrackerModuleMetadata's members very well). Please review and/or rework the patch. ___ tracker-list mailing list tracker-list@gnome.org http://mail.gnome.org/mailman/listinfo/tracker-list ___ tracker-list mailing list tracker-list@gnome.org http://mail.gnome.org/mailman/listinfo/tracker-list
Re: [Tracker] Support in Tracker for ultra-new Evolution installs that use SQLite for the summary format
On Fri, 2008-12-12 at 01:59 +0100, Philip Van Hoof wrote:
> On Thu, 2008-12-11 at 23:58 +0200, Ivan Frade wrote:
> > hi philip,
> >
> > short comment (the tablet is not the best device to review patches):
> > can we call this module "evolution224" and keep also the old module?
> > we could choose between one or other with a flag in compilation time
> > (the distributions will choose the correct for them)
>
> With this patch the code will detect the format and automatically choose
> the right way to parse both.
>
This version of the patch adds support for IMAP subfolders and fixes the
fact that in the last patch the "email://" URI was generated wrong (it
still contained the "folders.db" string-piece).
> > regards,
> >
> > Ivan
> >
> > On 12/11/08, Philip Van Hoof wrote:
> > > This patch makes ultra-new Evolution installs work again with Tracker.
> > >
> > > There's one problem and that is that the query will only find E-mails in
> > > the INBOX folder. You can easily find the Query and figure out what the
> > > problem is:
> > >
> > > The design that Carlos made assumes that for each folder there's a
> > > "summary" file. In the new Evolution cache format there's just one
> > > "folders.db" for each account.
> > >
> > > I could do a generated UNION select after first doing "select * from
> > > folders" on folders.db and then generating a query that includes all
> > > folders. I just have not done this for now and instead I'm just using
> > > INBOX and I'm neglecting the other folders.
> > >
> > > This is NOT the same as the proposal that I am doing at (a). This is
> > > instead a ad-hoc solution for the new situation (Evolution using SQLite
> > > for the summaries). I find this solution rather nasty, to be honest.
> > >
> > > (a) http://live.gnome.org/Evolution/Metadata
> > >
> > > For Carlos: I have also fixed a serious problem in evolution-pop.c,
> > > which is by the way unaffected by Evolution's changes (and works, if you
> > > just apply the patch that I included in this larger patch). The POP
> > > support's get_message_metadata was not returning metadata.
> > >
> > > This was crashing my tracker-indexer (as seemingly my compiler was
> > > putting "return 0x2" where the return was omitted, and the memory I have
> > > at 0x2 didn't dereference TrackerModuleMetadata's members very well).
> > >
> > > Please review and/or rework the patch.
> > >
> > > --
> > > Philip Van Hoof, freelance software developer
> > > home: me at pvanhoof dot be
> > > gnome: pvanhoof at gnome dot org
> > > http://pvanhoof.be/blog
> > > http://codeminded.be
> > >
> >
--
Philip Van Hoof, freelance software developer
home: me at pvanhoof dot be
gnome: pvanhoof at gnome dot org
http://pvanhoof.be/blog
http://codeminded.be
Index: src/tracker-indexer/modules/evolution-pop.c
===
--- src/tracker-indexer/modules/evolution-pop.c (revision 2701)
+++ src/tracker-indexer/modules/evolution-pop.c (working copy)
@@ -360,6 +360,8 @@
}
g_list_free (list);
+
+ return metadata;
}
static TrackerModuleMetadata *
Index: src/tracker-indexer/modules/evolution.c
===
--- src/tracker-indexer/modules/evolution.c (revision 2701)
+++ src/tracker-indexer/modules/evolution.c (working copy)
@@ -70,7 +70,9 @@
strchr (basename, '.') == NULL) {
type = MAIL_STORAGE_LOCAL;
} else if (g_str_has_prefix (path, imap_dir) &&
- strcmp (basename, "summary") == 0) {
+ (strcmp (basename, "summary") == 0 ||
+ strcmp (basename, "folders.db") == 0) ||
+ strcmp (basename, "cmeta") == 0) {
type = MAIL_STORAGE_IMAP;
}
Index: src/tracker-indexer/modules/Makefile.am
===
--- src/tracker-indexer/modules/Makefile.am (revision 2701)
+++ src/tracker-indexer/modules/Makefile.am (working copy)
@@ -16,7 +16,8 @@
$(GIO_CFLAGS) \
$(GLIB2_CFLAGS) \
$(GCONF_CFLAGS) \
- $(GMIME_CFLAGS)
+ $(GMIME_CFLAGS) \
+ $(SQLITE3_CFLAGS)
indexer_modules_LTLIBRARIES = \
libtracker-module-applications.la\
@@ -67,6 +68,7 @@
$(GMODULE_LIBS) \
$(GLIB2_LIBS) \
$(GCONF_LIBS) \
- $(GMIME_LIBS)
+ $(GMIME_LIBS) \
+ $(SQLITE3_LIBS)
endif
Index: src/tracker-indexer/modules/evolution-imap.c
===
--- src/tracker-indexer/modules/evolution-imap.c (revision 2701)
+++ src/tracker-indexer/modules/evolution-imap.c (working copy)
@@ -128,7 +128,18 @@
g_free (file->imap_dir);
g_free (file->cur_message_uid);
-fclose (file->summary);
+ if (file->db) {
+ sqlite3_close (file->db);
+ }
+
+ if (file->stmt) {
+ sqlite3_finalize (file->stmt);
+ }
+
+ if (file->summary) {
+ fclose (file->summary);
+ }
+
close (file->fd);
G_OBJECT_CLASS (tracker_evolution_imap_file_parent_class)->finalize (object);
@@ -482,10 +
Re: [Tracker] Support in Tracker for ultra-new Evolution installs that use SQLite for the summary format
I just posted a new version of the patch which now opens the DB in read-only mode. Thanks for the suggestion On Fri, 2008-12-12 at 15:12 +0800, Jerry Tan wrote: > Hi, philip > > How about to open folders.db in readonly mode? > > I just check sqlite's API. > sqlite3_open_v2 support a para with readonly flag > > > > On 2008ǯ12·î12Æü 00:34, Philip Van Hoof wrote: > > This patch makes ultra-new Evolution installs work again with Tracker. > > > > There's one problem and that is that the query will only find E-mails in > > the INBOX folder. You can easily find the Query and figure out what the > > problem is: > > > > The design that Carlos made assumes that for each folder there's a > > "summary" file. In the new Evolution cache format there's just one > > "folders.db" for each account. > > > > I could do a generated UNION select after first doing "select * from > > folders" on folders.db and then generating a query that includes all > > folders. I just have not done this for now and instead I'm just using > > INBOX and I'm neglecting the other folders. > > > > This is NOT the same as the proposal that I am doing at (a). This is > > instead a ad-hoc solution for the new situation (Evolution using SQLite > > for the summaries). I find this solution rather nasty, to be honest. > > > > (a) http://live.gnome.org/Evolution/Metadata > > > > For Carlos: I have also fixed a serious problem in evolution-pop.c, > > which is by the way unaffected by Evolution's changes (and works, if you > > just apply the patch that I included in this larger patch). The POP > > support's get_message_metadata was not returning metadata. > > > > This was crashing my tracker-indexer (as seemingly my compiler was > > putting "return 0x2" where the return was omitted, and the memory I have > > at 0x2 didn't dereference TrackerModuleMetadata's members very well). > > > > Please review and/or rework the patch. > > > > > > > > > > > > > > ___ > > tracker-list mailing list > > tracker-list@gnome.org > > http://mail.gnome.org/mailman/listinfo/tracker-list > -- Philip Van Hoof, freelance software developer home: me at pvanhoof dot be gnome: pvanhoof at gnome dot org http://pvanhoof.be/blog http://codeminded.be ___ tracker-list mailing list tracker-list@gnome.org http://mail.gnome.org/mailman/listinfo/tracker-list
Re: [Tracker] Support in Tracker for ultra-new Evolution installs that use SQLite for the summary format
On Thu, 2008-12-11 at 17:34 +0100, Philip Van Hoof wrote: > This patch makes ultra-new Evolution installs work again with Tracker. > > There's one problem and that is that the query will only find E-mails in > the INBOX folder. You can easily find the Query and figure out what the > problem is: > > The design that Carlos made assumes that for each folder there's a > "summary" file. In the new Evolution cache format there's just one > "folders.db" for each account. > > I could do a generated UNION select after first doing "select * from > folders" on folders.db and then generating a query that includes all > folders. I just have not done this for now and instead I'm just using > INBOX and I'm neglecting the other folders. > > This is NOT the same as the proposal that I am doing at (a). This is > instead a ad-hoc solution for the new situation (Evolution using SQLite > for the summaries). I find this solution rather nasty, to be honest. > > (a) http://live.gnome.org/Evolution/Metadata > > For Carlos: I have also fixed a serious problem in evolution-pop.c, > which is by the way unaffected by Evolution's changes (and works, if you > just apply the patch that I included in this larger patch). The POP > support's get_message_metadata was not returning metadata. Ugh, right. Tracker now compiles with a variety of compiler flags, so this and other errors have been spotted and fixed in trunk > > This was crashing my tracker-indexer (as seemingly my compiler was > putting "return 0x2" where the return was omitted, and the memory I have > at 0x2 didn't dereference TrackerModuleMetadata's members very well). > > Please review and/or rework the patch. > I think I agree with Ivan here, it could be better done as a new TrackerModuleFile implementation, the code to deal with DBs and summaries look quite unrelated (perhaps some functions should be moved to evolution-common.[ch]), and it would help maintenance. In src/tracker-indexer/modules/evolution.c you can see the code that returns the appropriate TrackerModuleFile for the currently scanned mail store, so you could hook there the implementation for newer evolution versions. Regards, Carlos ___ tracker-list mailing list tracker-list@gnome.org http://mail.gnome.org/mailman/listinfo/tracker-list
