Re: [Tracker] Random SIGPIPEs with tracker-indexer
Are these troubles fixed?
On Fri, Oct 24, 2008 at 12:21 PM, laurent.aguerre...@free.fr wrote:
Selon Carlos Garnacho car...@imendio.com:
Hi Laurent!,
Hi Carlos!
On miÃ(c), 2008-10-22 at 23:33 +0200, Laurent Aguerreche wrote:
hello,
my daily bug report is today about random SIGPIPEs with tracker-indexer.
Until now, I didn't see a crash with a text file, only with binaries.
Hrm, looks like the sigaction() call to ignore SIGPIPE so we can handle
it ourselves was commented accidentally, thanks for noticing, I've just
uncommented it in trunk :)
Now, gdb likes to manage such signals itself, so to be sure you won't
get random (and harmless) SIGPIPE when debugging you should type:
(gdb) handle SIGPIPE nostop
in the gdb prompt.
So I did that and now I get a SIGSEGV :
Tracker-Message: Extracting text
for:'/local/laguerre/Blah/Soutenance/soutenance.pdf' using
filter:'/local/laguerre/Tracker/Installs/Tracker/lib/tracker/filters/application/pdf_filter'
Detaching after fork from child process 7344.
24 oct 2008, 11:08:34: Tracker: Process '7344' spawned for
command:'/local/laguerre/Tracker/Installs/Tracker/lib/tracker/filters/application/pdf_filter'
(tracker-indexer:5649): Tracker-DEBUG: Process '7344' spawned for
command:'/local/laguerre/Tracker/Installs/Tracker/lib/tracker/filters/application/pdf_filter'
24 oct 2008, 11:08:34: Tracker: Process '7344' exited with code 0
(tracker-indexer:5649): Tracker-DEBUG: Process '7344' exited with code 0
Program received signal SIGPIPE, Broken pipe.
Detaching after fork from child process 7346.
24 oct 2008, 11:08:34: Tracker: Process '7346' spawned for
command:'/local/laguerre/Tracker/Installs/Tracker/libexec/tracker-extract'
(tracker-indexer:5649): Tracker-DEBUG: Process '7346' spawned for
command:'/local/laguerre/Tracker/Installs/Tracker/libexec/tracker-extract'
24 oct 2008, 11:08:34: Tracker: Process '7259' exited with code 14
(tracker-indexer:5649): Tracker-DEBUG: Process '7259' exited with code 14
Program received signal SIGSEGV, Segmentation fault.
0x00304522ef19 in g_io_channel_shutdown () from /lib64/libglib-2.0.so.0
Missing separate debuginfos, use: debuginfo-install GConf2.x86_64
ORBit2.x86_64
dbus-glib.x86_64 dbus.x86_64 gamin.x86_64 glib2.x86_64 glibc.x86_64
gmime.x86_64
gvfs.x86_64 hal.x86_64 libcap.x86_64 libselinux.x86_64 pango.x86_64
zlib.x86_64
(gdb) bt
#0 0x00304522ef19 in g_io_channel_shutdown () from
/lib64/libglib-2.0.so.0
#1 0x0011312a in process_context_destroy (context=0x2578270) at
tracker-metadata-utils.c:75
#2 0x00113209 in process_context_child_watch_cb (pid=7259, status=14,
user_data=0x2578270) at tracker-metadata-utils.c:110
#3 0x003045235484 in ?? () from /lib64/libglib-2.0.so.0
#4 0x00304523742b in g_main_context_dispatch () from
/lib64/libglib-2.0.so.0
#5 0x00304523ac0d in ?? () from /lib64/libglib-2.0.so.0
#6 0x00304523b13d in g_main_loop_run () from /lib64/libglib-2.0.so.0
#7 0x00113708 in metadata_query_file (path=0x2ae3280
/local/laguerre/Blah/Soutenance/soutenance.log, mimetype=0x34a8ac0
text/x-log) at tracker-metadata-utils.c:292
#8 0x001137af in metadata_utils_get_embedded (path=0x2ae3280
/local/laguerre/Blah/Soutenance/soutenance.log, mime_type=0x34a8ac0
text/x-log, metadata=0x2ae0d80)
at tracker-metadata-utils.c:326
#9 0x001146c5 in tracker_metadata_utils_get_data (path=0x2ae3280
/local/laguerre/Blah/Soutenance/soutenance.log) at
tracker-metadata-utils.c:960
#10 0x04e148e1 in tracker_module_file_get_metadata (file=0x2ae3410) at
files.c:151
#11 0x0040dcc1 in tracker_indexer_module_file_get_metadata
(module=0x210fdc0, file=0x2ae3410) at tracker-indexer-module.c:185
#12 0x0040a83c in process_file (indexer=0x2135820, info=0x2ae2fa0) at
tracker-indexer.c:1989
#13 0x0040abf9 in process_func (data=0x2135820) at
tracker-indexer.c:2116
#14 0x00304523742b in g_main_context_dispatch () from
/lib64/libglib-2.0.so.0
#15 0x00304523ac0d in ?? () from /lib64/libglib-2.0.so.0
#16 0x00304523b13d in g_main_loop_run () from /lib64/libglib-2.0.so.0
#17 0x0040efce in main (argc=1, argv=0x7fffcb4bdb18) at
tracker-main.c:371
(gdb) frame 7
#7 0x00113708 in metadata_query_file (path=0x2ae3280
/local/laguerre/Blah/Soutenance/soutenance.log, mimetype=0x34a8ac0
text/x-log) at tracker-metadata-utils.c:292
292 g_main_loop_run (metadata_context-data_incoming_loop);
(gdb) info locals
utf_path = (gchar *) 0x2ae3220 #65533;#65533;#65533;\002
str = (gchar *) 0x2ae0e60
/local/laguerre/Blah/Soutenance/soutenance.log\ntext/x-log\n
array = (GPtrArray *) 0x2ae0b70
status = G_IO_STATUS_NORMAL
error = (GError *) 0x2addbc0
(gdb) print *error
$1 = {domain = 44961056, code = 0, message = 0x0}
(gdb)
and this log file is a text only file... :-/
Rgds,
Laurent.
This is what I get:
Program received signal SIGPIPE,
Re: [Tracker] Random SIGPIPEs with tracker-indexer
On 17/12/08 08:10, Tshepang Lekhonkhobe wrote: Are these troubles fixed? I suspect this is related to the escaping issues we have. If we receive '\n' then memory is cleaned up the indexer's side. So if you still have information to send on the extractor's side, it might do something unexpected or not handled there. I am guessing at this point. We should wait until Carlos has had a chance to improve the extractor--indexer communication here I think. -- Regards, Martyn ___ tracker-list mailing list tracker-list@gnome.org http://mail.gnome.org/mailman/listinfo/tracker-list
Re: [Tracker] Update gmime to 2.4
El mié, 17-12-2008 a las 12:23 +0100, ext Michael Biebl escribió: 2008/12/17 Ivan Frade ivan.fr...@nokia.com: Hi, Thanks to Bastien Nocera, we have a patch to compile with gmime2.4 in the bugzilla: http://bugzilla.gnome.org/show_bug.cgi?id=564640 The question is: in the next release of tracker should we use gmime-2.0 or gmime-2.4? Some points to take into account: * To do it optional looks like a lot of ugly #ifdefs in the code. * What version of gmime is shipping GNOME 2.26? It looks like jhbuild is using gmime-2.4 * Current Ubuntu (intrepid) has only gmime-2.0 Even if intrepid had 2.4, there are also other distros out there ;-) Yes, of course. Sorry for the lapsus :) Ivan ___ tracker-list mailing list tracker-list@gnome.org http://mail.gnome.org/mailman/listinfo/tracker-list
Re: [Tracker] Update gmime to 2.4
On 17/12/08 10:48, Ivan Frade wrote: Hi, Thanks to Bastien Nocera, we have a patch to compile with gmime2.4 in the bugzilla: http://bugzilla.gnome.org/show_bug.cgi?id=564640 The question is: in the next release of tracker should we use gmime-2.0 or gmime-2.4? Some points to take into account: * To do it optional looks like a lot of ugly #ifdefs in the code. * What version of gmime is shipping GNOME 2.26? It looks like jhbuild is using gmime-2.4 * Current Ubuntu (intrepid) has only gmime-2.0 I think the safe option is to use gmime-2.0 and try to keep the patch to gmime-2.4 updated for distributions. What do you think? I agree. -- Regards, Martyn ___ tracker-list mailing list tracker-list@gnome.org http://mail.gnome.org/mailman/listinfo/tracker-list
Re: [Tracker] Indexer modules API afterthoughts
Hi!, As a heads up, all the suggestions that could affect tracker module developers are already implemented in trunk, if you compile tracker with --enable-gtk-doc, you'll see the docs in: tracker/docs/reference/libtracker-module/html/index.html I've also developed a new version for a sample dummy tracker module which could work as the base implementation for anyone who wants to make tracker index their application specific data: http://people.imendio.com/carlos/tracker/dummy-tracker-module-0.0.2.tar.gz Regards, Carlos On Mon, 2008-11-10 at 17:52 +0100, Carlos Garnacho wrote: Hi!, Since a while ago tracker allows compiling external modules, so tracker-indexer is able to collect data from external resources, here's a list of things that could be considered to be improved: * Improve services/metadata definitions i18n. DisplayName, Description, etc... should be translated. These also should be available to apps as first class objects through the libtracker API, so we don't end up doing strcmp() tricks like in tracker-search-tool so we get proper i18n, etc... * Do not split URIs into dirname/basename from the API. this is a problem if modules split differently URIs than what would tracker do, so I'd just make this function return a full URI string so all uri splitting in order to be stored is done the same way. * Way too much undeeded API is exported, we should devise what could or couldn't be useful to the module API users, so the whole experience is less confusing * Since we depend on recent enough glib, I'd encourage using GFiles rather than path/URI strings generally, at least for the current TrackerFile uses. * TrackerMetadata could offer API to store ints/dates/doubles/... and transform internally these to the storage type (strings), instead of forcing modules to do that by themselves. Also whether to use API for inserting single elements or lists is unclear, since users can just resort to either using common sense (which doesn't always work) or read the .metadata files. As a possible option, we could have a helper app that displays services/metadata types, descriptive names, and whether they take single elements or lists, that could be helpful for developers. * A suggestion I got is to make the API fully GObject oriented, so it's easier to wrap in other languages like vala [1] or python. I've already got an idea of the best design if we were using GTypeModule+GInterface, but could just be too effort and too late now, and also depends on whether we want non-gnomies using the API or not, so I'm hesitant about this Comments on any of the points, or further issues with the modules API are welcome :), I really think we should get this nailed before this is in a release. Regards, [1] Note there's already a VAPI file for the tracker API, but it had to be mostly handmade, http://svn.gnome.org/viewvc/vala/trunk/vapi/tracker-indexer-module-1.0.vapi?revision=1839view=markup ___ tracker-list mailing list tracker-list@gnome.org http://mail.gnome.org/mailman/listinfo/tracker-list
Re: [Tracker] Indexer modules API afterthoughts
On 17/12/08 16:41, Carlos Garnacho wrote: Hi!, As a heads up, all the suggestions that could affect tracker module developers are already implemented in trunk, if you compile tracker with --enable-gtk-doc, you'll see the docs in: tracker/docs/reference/libtracker-module/html/index.html I've also developed a new version for a sample dummy tracker module which could work as the base implementation for anyone who wants to make tracker index their application specific data: http://people.imendio.com/carlos/tracker/dummy-tracker-module-0.0.2.tar.gz Superb! Thanks Carlos! -- Regards, Martyn ___ tracker-list mailing list tracker-list@gnome.org http://mail.gnome.org/mailman/listinfo/tracker-list
Re: [Tracker] Update gmime to 2.4
2008/12/17 Ivan Frade ivan.fr...@nokia.com: Hi, Thanks to Bastien Nocera, we have a patch to compile with gmime2.4 in the bugzilla: http://bugzilla.gnome.org/show_bug.cgi?id=564640 The question is: in the next release of tracker should we use gmime-2.0 or gmime-2.4? Some points to take into account: * To do it optional looks like a lot of ugly #ifdefs in the code. * What version of gmime is shipping GNOME 2.26? It looks like jhbuild is using gmime-2.4 * Current Ubuntu (intrepid) has only gmime-2.0 Even if intrepid had 2.4, there are also other distros out there ;-) I think the safe option is to use gmime-2.0 and try to keep the patch to gmime-2.4 updated for distributions. What do you think? I would definitely keep the gmime-2.0 compatibility, atleast until all major distros ship 2.4 in their stable distributions. Michael -- Why is it that all of the instruments seeking intelligent life in the universe are pointed away from Earth? ___ tracker-list mailing list tracker-list@gnome.org http://mail.gnome.org/mailman/listinfo/tracker-list
Re: [Tracker] Update gmime to 2.4
Hi, Thanks to Bastien Nocera, we have a patch to compile with gmime2.4 in the bugzilla: http://bugzilla.gnome.org/show_bug.cgi?id=564640 The question is: in the next release of tracker should we use gmime-2.0 or gmime-2.4? Some points to take into account: * To do it optional looks like a lot of ugly #ifdefs in the code. * What version of gmime is shipping GNOME 2.26? It looks like jhbuild is using gmime-2.4 * Current Ubuntu (intrepid) has only gmime-2.0 I think the safe option is to use gmime-2.0 and try to keep the patch to gmime-2.4 updated for distributions. What do you think? Regards, Ivan El lun, 10-11-2008 a las 20:58 +0100, ext Luca Ferretti escribió: This was yet discussed on July (or June), gmime API was updated and the provided PC file is now gmime-2.4, not gmime-2.0. This breaks the build on jhbuild :( Any plan to update to this version, maybe optionally? It's OK to open a bug un bugzilla to track this? ___ tracker-list mailing list tracker-list@gnome.org http://mail.gnome.org/mailman/listinfo/tracker-list ___ tracker-list mailing list tracker-list@gnome.org http://mail.gnome.org/mailman/listinfo/tracker-list
Re: [Tracker] Support in Tracker for ultra-new Evolution installs that use SQLite for the summary format
On Thu, 2008-12-11 at 17:34 +0100, Philip Van Hoof wrote: This patch makes ultra-new Evolution installs work again with Tracker. There's one problem and that is that the query will only find E-mails in the INBOX folder. You can easily find the Query and figure out what the problem is: The design that Carlos made assumes that for each folder there's a summary file. In the new Evolution cache format there's just one folders.db for each account. I could do a generated UNION select after first doing select * from folders on folders.db and then generating a query that includes all folders. I just have not done this for now and instead I'm just using INBOX and I'm neglecting the other folders. This is NOT the same as the proposal that I am doing at (a). This is instead a ad-hoc solution for the new situation (Evolution using SQLite for the summaries). I find this solution rather nasty, to be honest. (a) http://live.gnome.org/Evolution/Metadata For Carlos: I have also fixed a serious problem in evolution-pop.c, which is by the way unaffected by Evolution's changes (and works, if you just apply the patch that I included in this larger patch). The POP support's get_message_metadata was not returning metadata. Ugh, right. Tracker now compiles with a variety of compiler flags, so this and other errors have been spotted and fixed in trunk This was crashing my tracker-indexer (as seemingly my compiler was putting return 0x2 where the return was omitted, and the memory I have at 0x2 didn't dereference TrackerModuleMetadata's members very well). Please review and/or rework the patch. I think I agree with Ivan here, it could be better done as a new TrackerModuleFile implementation, the code to deal with DBs and summaries look quite unrelated (perhaps some functions should be moved to evolution-common.[ch]), and it would help maintenance. In src/tracker-indexer/modules/evolution.c you can see the code that returns the appropriate TrackerModuleFile for the currently scanned mail store, so you could hook there the implementation for newer evolution versions. Regards, Carlos ___ tracker-list mailing list tracker-list@gnome.org http://mail.gnome.org/mailman/listinfo/tracker-list
