Re: [Gossip] search library battle royale
But anyway, utf-8 support for Xapian is pretty much written now. I've just sent JeffB a patch to try (but it'll require a reindex so it'll probably not be live for a while). Major, major kudos to Olly for getting this feature ready. I've patched it in and it seems to work great! Let's revist queries in Danish and Japanese. http://www.mail-archive.com/cgi-bin/omega/omega?P=%E6%A1%90%E5%B1%B1&DB=announce-jp%40jp.freebsd.org http://www.mail-archive.com/cgi-bin/omega/omega?P=%C3%B8l+eftersmag&DB=brygforum%40lists.haandbryg.dk I've just asked the computer to start re-indexing all the lists with this new improved version of Xapian. That process will take about 5 full days, and will proceed in alphabetical order. As of this second, it is crunching away on a list called "activedir" > In the meantime, I'd love to hear some pointed comments > on the user interface. Do people prefer Omega's "search engine > standard" layout, or PyLucene's "blend into service" appearance? FWIW, I think I prefer the "blend in" look. But the look is essentially orthogonal to the choice of engine - it's just a matter of slotting the appropriate HTML into the templates. Unless I hear otherwise, I'm going to assume everyone is in complete agreement that "blend in" is significantly better. Any dissenters? ___ Discussion list for The Mail Archive [email protected] http://jab.org/cgi-bin/mailman/listinfo/gossip
Re: [Gossip] search library battle royale
On Tue, Sep 05, 2006 at 09:39:46PM -0700, Jeff Breidenbach wrote: > By private email, one person noted that Xapian is not doing > so hot on Czech. As in not returning search results. I'm a little suprised that anything gets found that isn't in ASCII since Omega's indexer is generating words from utf-8 text but treating it as iso-8859-1 (unless JeffB has patched it). Since the second and subsequent characters of any multibyte utf-8 sequence are symbols in iso-8859-1, that means you'll get a word break after any non-ASCII character, even if it's an accented letter. But anyway, utf-8 support for Xapian is pretty much written now. I've just sent JeffB a patch to try (but it'll require a reindex so it'll probably not be live for a while). > Another managed to craft a query that causes PyLucene to throw an > exception. Any query which it can't parse seems to do that. Try searching for * or ", for example. > In the meantime, I'd love to hear some pointed comments > on the user interface. Do people prefer Omega's "search engine > standard" layout, or PyLucene's "blend into service" appearance? FWIW, I think I prefer the "blend in" look. But the look is essentially orthogonal to the choice of engine - it's just a matter of slotting the appropriate HTML into the templates. Cheers, Olly ___ Discussion list for The Mail Archive [email protected] http://jab.org/cgi-bin/mailman/listinfo/gossip
Re: [Gossip] search library battle royale
Hi Jeff, Once we've settled on which search software to use, I promise to make sure there's a "help" link leading to some examples. In the meantime, here's some documentation from the two vendors. To my eye, they are both offer ridiculously overpowered. Kind of like monster trucks. :) Thanks for your time and the examples you developed. And thanks for your promise to post a help link. Please keep in mind that when those of you who already know a process try to explain it to those of us who don't, the result, far too often, is about as clear as mud. Xapian even has a super fancy algorithm that tries to figure out subtopics within search results. It is way too confusing for my taste, but you can see it action below. Apparently the software thinks you are into dipleidoscopes and ebay. Are you? http://www.mail-archive.com/cgi-bin/omega/omega?P=%22Mac+Oglesby%22&DEFAULTOP=or&DB=sundial%40rrz.uni-koeln.de&FMT=query.orig&xP=Rmac.Roglesby.machin.&xDB=sundial%40rrz.uni-koeln.de&xFILTERS=--O Well, just about everyone is into eBay. And if you aren't into dipleidoscopes yet, what's holding you back?;-) It's almost frightening how much information we voluntarily put out there for anyone to see! It's beyond frightening to see what that Xapian search string looks like! On the other hand, it seems as though you intend to improve our archive search tools and help us understand how to use them. Thank you. Mac ___ Discussion list for The Mail Archive [email protected] http://jab.org/cgi-bin/mailman/listinfo/gossip
Re: [Gossip] search library battle royale
I don't understand a lot of what's been said about evaluating these search engines, but I'll comment anyway. Thanks, appreciated. Much of what's been said is just me having fun. Since, for me at least, easy to use but full-featured search software is a vital part of any archive, the ideal search engine should be powerful, yet intuitive to operate, or come with good instructions. A tutorial would be helpful. That's a great point. Once we've settled on which search software to use, I promise to make sure there's a "help" link leading to some examples. In the meantime, here's some documentation from the two vendors. To my eye, they are both offer ridiculously overpowered. Kind of like monster trucks. :) PyLucene: http://lucene.apache.org/java/docs/queryparsersyntax.html. Xapian: http://www.xapian.org/docs/queryparser.html I'd like to be able to search the results of an earlier search. Here's an example of two searches. The second will return a subset of the first. "Mac Oglesby" "Mac Oglesby" AND machines Xapian even has a super fancy algorithm that tries to figure out subtopics within search results. It is way too confusing for my taste, but you can see it action below. Apparently the software thinks you are into dipleidoscopes and ebay. Are you? http://www.mail-archive.com/cgi-bin/omega/omega?P=%22Mac+Oglesby%22&DEFAULTOP=or&DB=sundial%40rrz.uni-koeln.de&FMT=query.orig&xP=Rmac.Roglesby.machin.&xDB=sundial%40rrz.uni-koeln.de&xFILTERS=--O ___ Discussion list for The Mail Archive [email protected] http://jab.org/cgi-bin/mailman/listinfo/gossip
Re: [Gossip] search library battle royale
Hi Jeff, I don't understand a lot of what's been said about evaluating these search engines, but I'll comment anyway. Since, for me at least, easy to use but full-featured search software is a vital part of any archive, the ideal search engine should be powerful, yet intuitive to operate, or come with good instructions. A tutorial would be helpful. I'd like to be able to search the results of an earlier search. Mac Oglesby It's starting to get bloody. All right! By private email, one person noted that Xapian is not doing so hot on Czech. As in not returning search results. Another managed to craft a query that causes PyLucene to throw an exception. Jeff and I are patching up the staggering contenders as best we can, and then sending them right back out for more punishment. In the meantime, I'd love to hear some pointed comments on the user interface. Do people prefer Omega's "search engine standard" layout, or PyLucene's "blend into service" appearance? -Jeff ___ Discussion list for The Mail Archive [email protected] http://jab.org/cgi-bin/mailman/listinfo/gossip ___ Discussion list for The Mail Archive [email protected] http://jab.org/cgi-bin/mailman/listinfo/gossip
Re: [Gossip] search library battle royale
It's starting to get bloody. All right! By private email, one person noted that Xapian is not doing so hot on Czech. As in not returning search results. Another managed to craft a query that causes PyLucene to throw an exception. Jeff and I are patching up the staggering contenders as best we can, and then sending them right back out for more punishment. In the meantime, I'd love to hear some pointed comments on the user interface. Do people prefer Omega's "search engine standard" layout, or PyLucene's "blend into service" appearance? -Jeff ___ Discussion list for The Mail Archive [email protected] http://jab.org/cgi-bin/mailman/listinfo/gossip
Re: [Gossip] search library battle royale
Jeff Breidenbach wrote: Ok, ok, you score some serious points here, especially on Asian languages, but Xapian holds its own on European languages. I think. Check out some Brazilian Portuguese action below. And I hear the Xapian team is working hard on full UTF-8 support. Hope they aren't planning any holidays soon ;-) http://www.mail-archive.com/cgi-bin/omega/omega?P=%C3%B8l+eftersmag&DB=brygforum%40lists.haandbryg.dk&FMT=query&xP=beer.&xDB=brygforum%40lists.haandbryg.dk&xFILTERS=--O Check out the Danish characters - they are displayed as: Firefox: reverse-field questionmarks Explorer: Japanese/Chinese! The mails look fine in the mail-archive view, its just the search engine results that look ugly. /Dan ___ Discussion list for The Mail Archive [email protected] http://jab.org/cgi-bin/mailman/listinfo/gossip
