Re: indexing Maildir mails
Hi, On Wed, 2005-07-20 at 22:51 +0300, Aviram Jenik wrote: > The problem I'm experiencing is quite different. The mail *is* rfc822, but it > does not show up in the results. In fact, when I add a first line to screw > the mime detection (make the mail file a 'text/plain') it *does* appear in > the results - so the problem is definitely in the FilterMail object. Are you having a problem with body searches, or names of people? I noticed a problem where names were being indexed as keywords and not regular text. Which meant in a mail from me, "Joe" would match, but "Shaw" wouldn't. I just fixed this in CVS, but it will require you to reindex the mail. In my tests, I haven't had any problem searching for text in bodies of emails. Joe ___ Dashboard-hackers mailing list Dashboard-hackers@gnome.org http://mail.gnome.org/mailman/listinfo/dashboard-hackers
Re: indexing Maildir mails
Hi, On Thu, 2005-07-21 at 10:30 +, Raghu GS wrote: > i fixed a header and search in best for it, best showed that fixed header mail > in same Files but this time it won't open it via a text editor but instead > shows > this long error > > Open URI: action:_tile_568!Open > Cmd: kmail --view Beagle is actually trying to run a program named "kmail --view" rather than recognizing that "--view" is an argument. I just checked a fix into CVS. Joe ___ Dashboard-hackers mailing list Dashboard-hackers@gnome.org http://mail.gnome.org/mailman/listinfo/dashboard-hackers
Re: indexing Maildir mails
Hello Everybody in my turn while trying to make Beagle index Maildir mail files produced by Kmail with the great help from a person nicnamed dsd in #dashboard irc channal i had bit success, first-of all for indexing Mails produced by KMail you must add /.kde/shere/apps/kmail/mails/inbox/cur /.kde/shere/apps/kmail/mails/sent-mails/cur to Beagle list of roots, then it would be indexed, but if you use KMail it would be indexed by text filter not by mail filter, this is mishandling is not caused by beagle but by gnome-vfs's inability to correct indentify Maildir Mail, i heard this is caused by manipulation of headers by application after receiving it or by Mail server, the result would be is in best when click a result it would call a text editor to proccess it, Note: even if after headers fixed, best shows results in Files section not in mails section. i fixed a header and search in best for it, best showed that fixed header mail in same Files but this time it won't open it via a text editor but instead shows this long error Open URI: action:_tile_568!Open Cmd: kmail --view Arg: 'file:///home/raghugs/.kde/share/apps/kmail /mail/inbox/cur/1121582219.7423.q55ap' Error in OpenFromMime: System.ComponentModel.Win32Exception: Cannot find the specified file in [0x00289] (at /tmp/scratch/BUILD/mono-1.1.8.1/mcs/class/System /System.Diagnostics/Process.cs:823) System.Diagnostics.Process:Start_common (System.Diagnostics.ProcessStartInfo startInfo, System.Diagnostics.Process process) in [0x7] (at /tmp/scratch/BUILD/mono-1.1.8.1/mcs/class/System /System.Diagnostics/Process.cs:856) System.Diagnostics.Process:Start () in (wrapper remoting-invoke-with-check) System.Diagnostics.Process:Start () in <0x0021c> Beagle.Tile.Tile:OpenFromMime (Beagle.Hit hit, System.String command_fallback, System.String args_fallback, Boolean expects_uris_fallback) ___ Dashboard-hackers mailing list Dashboard-hackers@gnome.org http://mail.gnome.org/mailman/listinfo/dashboard-hackers
Re: indexing Maildir mails
On Wednesday 20 July 2005 19:59, Ian Soboroff wrote: > I have a similar problem... [...] > $ gnomevfs-info ~/Mail/mail/misc/8225 > MIME type : text/plain The problem I'm experiencing is quite different. The mail *is* rfc822, but it does not show up in the results. In fact, when I add a first line to screw the mime detection (make the mail file a 'text/plain') it *does* appear in the results - so the problem is definitely in the FilterMail object. - Aviram ___ Dashboard-hackers mailing list Dashboard-hackers@gnome.org http://mail.gnome.org/mailman/listinfo/dashboard-hackers
Re: indexing Maildir mails
Hi, On Wed, 2005-07-20 at 13:15 -0400, Ian Soboroff wrote: > $ head -4 ~/Mail/mail/misc/8225 > X-From-Line: imap Fri Jul 5 16:33:27 2002 > Received: from nist-i.nist.gov (nist1.nist.gov [129.6.94.1]) > by email.nist.gov (8.12.2/8.12.2) with SMTP id g65KV3FN011792 > for <[EMAIL PROTECTED]>; Fri, 5 Jul 2002 16:31:03 -0400 (EDT) The "X-From-Line" is throwing off the MIME sniffer. Most sniffers (including gnome-vfs and Apache) expect one of "From", "Received", "Relay-Version", or "Return-Path". They're not technically required to start the file, but it's generally a good, easy to detect convention. Some KMail users are seeing something similar. You could argue that the bug is in freedesktop.org's mime magic database, but realistically it's impossible to detect every possible first line that would be a compliant message. It's generally good behavior for mailers to append their headers rather than prepend them. For this reason you may want to file an interoperability bug against Gnus. See http://bugzilla.gnome.org/show_bug.cgi?id=310462 Joe ___ Dashboard-hackers mailing list Dashboard-hackers@gnome.org http://mail.gnome.org/mailman/listinfo/dashboard-hackers
Re: indexing Maildir mails
Joe Shaw <[EMAIL PROTECTED]> writes: > Hi, > > On Wed, 2005-07-20 at 12:59 -0400, Ian Soboroff wrote: >> I have a similar problem... I use Emacs Gnus for mail, which stores a >> single email per file, but since gnomevfs-info doesn't correctly >> identify the file type, it doesn't come up as mail: > > What are the first 4 lines of the file? $ head -4 ~/Mail/mail/misc/8225 X-From-Line: imap Fri Jul 5 16:33:27 2002 Received: from nist-i.nist.gov (nist1.nist.gov [129.6.94.1]) by email.nist.gov (8.12.2/8.12.2) with SMTP id g65KV3FN011792 for <[EMAIL PROTECTED]>; Fri, 5 Jul 2002 16:31:03 -0400 (EDT) Ian ___ Dashboard-hackers mailing list Dashboard-hackers@gnome.org http://mail.gnome.org/mailman/listinfo/dashboard-hackers
Re: indexing Maildir mails
Hi, On Wed, 2005-07-20 at 12:59 -0400, Ian Soboroff wrote: > I have a similar problem... I use Emacs Gnus for mail, which stores a > single email per file, but since gnomevfs-info doesn't correctly > identify the file type, it doesn't come up as mail: What are the first 4 lines of the file? Joe ___ Dashboard-hackers mailing list Dashboard-hackers@gnome.org http://mail.gnome.org/mailman/listinfo/dashboard-hackers
Re: indexing Maildir mails
I have a similar problem... I use Emacs Gnus for mail, which stores a single email per file, but since gnomevfs-info doesn't correctly identify the file type, it doesn't come up as mail: $ gnomevfs-info ~/Mail/mail/misc/8225 Name : 8225 Type : Regular MIME type : text/plain Size : 1383 Blocks: 8 I/O block size: 32768 Local : YES SUID : NO SGID : NO Sticky: NO Permissions : 600600 Link count: 1 UID : 34544 GID : 4254 Access time : Wed Jul 20 09:09:02 2005 Modification time : Fri Jul 5 16:33:27 2002 Change time : Mon Mar 14 14:49:36 2005 Device # : 31 Inode # : 1023412966 Readable : YES Writable : YES Executable: NO Ian ___ Dashboard-hackers mailing list Dashboard-hackers@gnome.org http://mail.gnome.org/mailman/listinfo/dashboard-hackers
Re: indexing Maildir mails
Hi, The only issue in Maildir case (Kmail at least) is the fact that most if not all the directories start with a "." (the name of the folder), which by default, as far as I remember are regarded by Beagle has hidden and shouldn't be indexed. I am not sure whether they have addressed this or not. On 7/20/05, Aviram Jenik <[EMAIL PROTECTED]> wrote: > On Wednesday 20 July 2005 05:23, D Bera wrote: > > > I think it might still be possible to recusrively scan for cur/ new/ > > tmp/ directories in the maildir directory and find rfc822 files in > > them. At least, that is what I think > > http://www.rpcurnow.force9.co.uk/mairix/ does. > > > I'm pretty sure this is what the FilterMail filter in beagle *currently* does. > > > I will see if I can recursively scan directories and pick up mail > > files for indexing. > > > > From what I've seen, it seems beagle uses the standard File Indexing backend > to recursively scan directories. If the file is an rfc822 file, it filters it > through the mail filter which indexes it - all that works fine for me, except > for the fact the mail doesn't show up later in the search results. > > > - Bera > > > - Aviram > ___ > Dashboard-hackers mailing list > Dashboard-hackers@gnome.org > http://mail.gnome.org/mailman/listinfo/dashboard-hackers > -- Thanks Noam Rathaus CTO Beyond Security Ltd. Join the SecuriTeam community on Orkut: http://www.orkut.com/Community.aspx?cmm=1 ___ Dashboard-hackers mailing list Dashboard-hackers@gnome.org http://mail.gnome.org/mailman/listinfo/dashboard-hackers
Re: indexing Maildir mails
On Wednesday 20 July 2005 05:23, D Bera wrote: > > I think it might still be possible to recusrively scan for cur/ new/ > tmp/ directories in the maildir directory and find rfc822 files in > them. At least, that is what I think > http://www.rpcurnow.force9.co.uk/mairix/ does. > I'm pretty sure this is what the FilterMail filter in beagle *currently* does. > I will see if I can recursively scan directories and pick up mail > files for indexing. > From what I've seen, it seems beagle uses the standard File Indexing backend to recursively scan directories. If the file is an rfc822 file, it filters it through the mail filter which indexes it - all that works fine for me, except for the fact the mail doesn't show up later in the search results. > - Bera > - Aviram ___ Dashboard-hackers mailing list Dashboard-hackers@gnome.org http://mail.gnome.org/mailman/listinfo/dashboard-hackers
Re: indexing Maildir mails
It seems that the directory structure for maildir is not standardised. This page talks about two possible formats: http://wiki.mutt.org/index.cgi?action=browse&diff=2&id=MuttFaq/Maildir Ubuntu seems to be coming up with yet another naming standard (more at https://wiki.ubuntu.com//MailStorageStandardisation ). And to talk about kmail, their structure is given in http://wiki.mutt.org/index.cgi?ExtendedMaildirFormat and it doesnt seem they have any plans of changing it soon (http://mail.kde.org/pipermail/kmail-devel/2005-January/014614.html ). However, all these are different conventions for directory structures. I think it might still be possible to recusrively scan for cur/ new/ tmp/ directories in the maildir directory and find rfc822 files in them. At least, that is what I think http://www.rpcurnow.force9.co.uk/mairix/ does. I will see if I can recursively scan directories and pick up mail files for indexing. - Bera > > The changelog of beagle-0.12 says it can index maildir mails. Does > > that mean mails in maildir format from any mail application ? I tried > > to change local_path, imap_path etc. in EvolutionMailQueryable to > > point to a maildir directory I have but there was no response in the > > debug output. > > Filtering maildir mail is done via a filter on regular file indexing. If the > file is identified to be message/rfc822 (you can check this with > gnomevfs-info) it passed through the FilterMail filter and indexed. ___ Dashboard-hackers mailing list Dashboard-hackers@gnome.org http://mail.gnome.org/mailman/listinfo/dashboard-hackers
Re: indexing Maildir mails
Hi, On Wednesday 20 July 2005 01:07, D Bera wrote: > Hi, > The changelog of beagle-0.12 says it can index maildir mails. Does > that mean mails in maildir format from any mail application ? I tried > to change local_path, imap_path etc. in EvolutionMailQueryable to > point to a maildir directory I have but there was no response in the > debug output. Filtering maildir mail is done via a filter on regular file indexing. If the file is identified to be message/rfc822 (you can check this with gnomevfs-info) it passed through the FilterMail filter and indexed. My experience has been as follows: so far I haven't been able to see the indexed files in the results (either best or beagle-query) but I haven't investigated enough to see where the problem is - according to the logs the file is indeed indexed, and according to beagle-index-info the indexed file count grows (notice Maildir is marked by the backend as "file" and not "mail). If you can check this out and let me know what you find that would be great - Maildir support is extremely important to me and I would like to give the beagle developers as much accurate information as I can regarding bugs in this feature. - Aviram ___ Dashboard-hackers mailing list Dashboard-hackers@gnome.org http://mail.gnome.org/mailman/listinfo/dashboard-hackers
indexing Maildir mails
Hi, The changelog of beagle-0.12 says it can index maildir mails. Does that mean mails in maildir format from any mail application ? I tried to change local_path, imap_path etc. in EvolutionMailQueryable to point to a maildir directory I have but there was no response in the debug output. From the code it seems the crawler tries to find files named "summary" or *.ev-summary and quits otherwise. Are these evolution specific files ? If indexing general maildir mails is not yet supported, is anybody working on it ? Thanks, - Bera ___ Dashboard-hackers mailing list Dashboard-hackers@gnome.org http://mail.gnome.org/mailman/listinfo/dashboard-hackers
