Re: Sites with Innovative Presentation of Tags and Facets

2010-05-29 Thread Dennis Gearon
I'm interested in this stuff, but what is a 'sparkline', and can I get a URL of 
an example?

Dennis Gearon

Signature Warning

EARTH has a Right To Life,
  otherwise we all die.

Read 'Hot, Flat, and Crowded'
Laugh at http://www.yert.com/film.php


--- On Fri, 5/28/10, Chris Hostetter hossman_luc...@fucit.org wrote:

 From: Chris Hostetter hossman_luc...@fucit.org
 Subject: Re: Sites with Innovative Presentation of Tags and Facets
 To: solr-user@lucene.apache.org
 Date: Friday, May 28, 2010, 3:34 PM
 
 :  you mean something like the following?
 :  http://hledani.rozhlas.cz/?query=jazzback=defaultNavigation=;
 
 : Also http://markmail.org has some nice chart
 
 Yeah ... those are close to what i mean -- but in both
 cases there is 
 really one big visual graph of a single numeric value
 (ironicly it's a 
 timeline in both cases) ... i was thinking more along the
 lines of when a 
 facet UI has *multiple* numeric facets.
 
 Imagine if a site like kayak.com for example, that has a
 search UI with 7 
 numeric sliders (departure take off time, departure landing
 time, return 
 take off time, return landing time, layover duration, trip
 duration, and 
 price) showed you a small sparkline above each slider that
 showed you 
 where the various options tended to cluster based on the
 other filters you 
 had applied -- so you can see that most flights have
 layovers in the ~30 
 minute range, and the key price point is around $99 ... but
 when you move 
 the take off time slider to early in the morning the
 sparkline above 
 layover duration shifts up to longer layovers, and the
 prices start 
 tnreding up.
 
 
 -Hoss
 
 


Re: Solr trunk and Jetty threadpool implementation problem

2010-05-29 Thread Mark Miller

On 5/29/10 3:13 AM, Chris Hostetter wrote:


: Wow, thanks for the heads-up David!
: This probably got inadvertently changed when Jetty was upgraded...
: sounds like we should prob change back to BoundedThreadPool as a
: default!

it seems to have been a deliberate choice miler made, it was a distinct
commit 20 minutes after the jetty upgrade was committed...

http://svn.apache.org/viewvc?view=revisionrevision=926575

http://svn.apache.org/viewvc?view=revisionrevision=926579

Mark: any recollection why you thought that change was needed?

-Hoss




I believe it was because BoundedThreadPool was deprecated and the new 
sample xml uses QueuedThreadPool. Plus, BoundedThreadPool says:


 * @deprecated Use {...@link QueuedThreadPool}


--
- Mark

http://www.lucidimagination.com


Re: Sites with Innovative Presentation of Tags and Facets

2010-05-29 Thread Gora Mohanty
On Sat, 29 May 2010 00:00:57 -0700 (PDT)
Dennis Gearon gear...@sbcglobal.net wrote:

 I'm interested in this stuff, but what is a 'sparkline', and can
 I get a URL of an example?
[...]

Here is one that I recently came across, and liked (look at the
last example): http://moritz.stefaner.eu/projects/elastic-lists/
The code has apparently also been recently open-sourced.

Regards,
Gora


Re: Prefix-Search with Stopwords - no results?

2010-05-29 Thread Gert Brinkmann


Thank you, Chris and Erick, for the answers,

it was new to me that the* is expanded to all known the* words in the 
index. Good to know.


And yes, the AND operation between the query terms are certainly the 
problem. (I would like to switch to OR instead. The result set will grow 
the more words you are searching for, but as the results are ordered for 
the hit quality this would be ok. But the customer does not like this 
behaviour, because he thinks that the more words you are searching for, 
the smaller the result set should become. So this is not an option.).


On 28.05.2010 22:06, Chris Hostetter wrote:

word2*) ... in the client, that you instead consider using multiple
fields -- one text defined as you have it now, and one text_prefix
defined similarly but with an additional EdgeNGramTokenFilter used when
indexing to generate prefix tokens. then search those fields using
dismax...

q=word1 word2 word3  qf=text text_prefix  mm=100%  tie=0


Ok, I will think about this. But I wonder if this will be more efficient 
than just not filtering stopwords? (But I have to study the EdgeNGram 
thing first. AFAIK it indexes all WORDS as WORDS, WORD, WOR, WO. So the 
index will be blown up, too?)


What I do not understand in your idea, why I should use a second 
text_prefix field. Wouldn't it work with just this text_prefix without 
the normal text field, too, as I always let search for word and 
word* and never without the prefix?


Thanks,
Gert


Re: Prefix-Search with Stopwords - no results?

2010-05-29 Thread Erick Erickson
Well, the index does, indeed, get bigger. But the searches
get much faster because there's no term expansion going
on. It's another time/space tradeoff.  I'm afraid you'll have
to just experiment a bit to see if this is an acceptable tradeoff.
in your particular situation

The real memory hit in Lucene comes from *sorting* a field
with many unique terms. And you won't sort on the NGram
field I don't think and disk space is cheap.

Best
Erick

On Sat, May 29, 2010 at 3:44 AM, Gert Brinkmann g...@netcologne.de wrote:


 Thank you, Chris and Erick, for the answers,

 it was new to me that the* is expanded to all known the* words in the
 index. Good to know.

 And yes, the AND operation between the query terms are certainly the
 problem. (I would like to switch to OR instead. The result set will grow the
 more words you are searching for, but as the results are ordered for the hit
 quality this would be ok. But the customer does not like this behaviour,
 because he thinks that the more words you are searching for, the smaller the
 result set should become. So this is not an option.).


 On 28.05.2010 22:06, Chris Hostetter wrote:

 word2*) ... in the client, that you instead consider using multiple
 fields -- one text defined as you have it now, and one text_prefix
 defined similarly but with an additional EdgeNGramTokenFilter used when
 indexing to generate prefix tokens. then search those fields using
 dismax...

 q=word1 word2 word3  qf=text text_prefix  mm=100%  tie=0


 Ok, I will think about this. But I wonder if this will be more efficient
 than just not filtering stopwords? (But I have to study the EdgeNGram thing
 first. AFAIK it indexes all WORDS as WORDS, WORD, WOR, WO. So the index will
 be blown up, too?)

 What I do not understand in your idea, why I should use a second
 text_prefix field. Wouldn't it work with just this text_prefix without the
 normal text field, too, as I always let search for word and word* and
 never without the prefix?

 Thanks,
 Gert



facet order

2010-05-29 Thread search
Hi,

how can i configuratively order facets according to total count of facet
fields?

for example - facets with the highest count be on top.

facet1 [0]
abc (20)
def (18)
ghi (16)

facet2 [1]
jkl (10)
mno (9)
pqr (2)

thanks

dev.



Re: facet order

2010-05-29 Thread Furkan Kuru
use: facet.sort=true


http://wiki.apache.org/solr/SimpleFacetParameters#facet.sort


On Sat, May 29, 2010 at 3:53 PM, sea...@butterflycluster.com wrote:

 Hi,

 how can i configuratively order facets according to total count of facet
 fields?

 for example - facets with the highest count be on top.

 facet1 [0]
 abc (20)
 def (18)
 ghi (16)

 facet2 [1]
 jkl (10)
 mno (9)
 pqr (2)

 thanks

 dev.




-- 
Furkan Kuru


Re: Solr trunk and Jetty threadpool implementation problem

2010-05-29 Thread Yonik Seeley
David, if it's fast for you to reproduce, would it be possible for you
to try the latest Jetty 6.1.24 and see if the issue still exists?
http://dist.codehaus.org/jetty/

Seems like we should upgrade to 6.1.24 anyway (there were quite a few
fixes in 6.1.23)
http://jira.codehaus.org/secure/ReleaseNote.jspa?projectId=11160version=16044

-Yonik
http://www.lucidimagination.com

On Thu, May 27, 2010 at 5:43 PM, Smiley, David W. dsmi...@mitre.org wrote:
 I'd like to warn people about the default configuration of Jetty in the Solr 
 trunk release (not present in Solr 1.4 and prior).  There is a difference in 
 the jetty configuration which is for the latest Solr to use the 
 QueuedThreadPool (as seen in jetty.xml).  Previously, it had used a 
 BoundedThreadPool implementation that I've heard is considered deprecated 
 presently.  I have a multi-core setup where Jetty is serving up lots of Solr 
 cores 9+ and when our client does a distributed search (3 of them at a time 
 actually), it triggers a condition in which the query takes 50 plus seconds 
 to respond.  During this time, the machine is effectively idle, seemingly 
 waiting for something.  To fix this, go back to the former BoundedThreadPool 
 implementation or don't use Jetty.  FWIW this has triggered us to swtich to 
 Tomcat.

 Sorry but I have sunk so much resources into tracking down this nasty problem 
 that I can't spend much more on further figuring out why QueuedThreadPool is 
 failing us.

 ~ David Smiley
 Author: http://www.packtpub.com/solr-1-4-enterprise-search-server/







matching only empty fields

2010-05-29 Thread Lukas Kahwe Smith
Hi,

I have implemented a facet search, where users essentially select what should 
not be included. I do this by constructing an fq filter where I match for the 
deselected items that I then negate:
$fq = {!tag=dt}!($fq);
$criteria-addParam('fq', $fq);

Now in some cases the field the user is deselecting is actually an empty string.

For example go to the below address and open the legal value section. there 
is an option none there, which is actually an empty string.
http://search.un-informed.org/search?q=malaria=tm=anys=Search

The field itself is just an untokenized string. Of course I could just turn an 
empty string into none at index time, but I am wondering how to do it in 
general :)

I tried using just  or [ TO ] to match for empty strings, but this 
matches everything and due to the negation removes everything.

regards,
Lukas Kahwe Smith
m...@pooteeweet.org





Re: matching only empty fields

2010-05-29 Thread Yonik Seeley
On Sat, May 29, 2010 at 10:01 AM, Lukas Kahwe Smith m...@pooteeweet.org wrote:
 Hi,

 I have implemented a facet search, where users essentially select what should 
 not be included. I do this by constructing an fq filter where I match for the 
 deselected items that I then negate:
            $fq = {!tag=dt}!($fq);
            $criteria-addParam('fq', $fq);

 Now in some cases the field the user is deselecting is actually an empty 
 string.

 For example go to the below address and open the legal value section. there 
 is an option none there, which is actually an empty string.
 http://search.un-informed.org/search?q=malaria=tm=anys=Search

 The field itself is just an untokenized string. Of course I could just turn 
 an empty string into none at index time, but I am wondering how to do it in 
 general :)

 I tried using just  or [ TO ] to match for empty strings, but this 
 matches everything and due to the negation removes everything.

Hmmm, if this is on a String field, it seemed to work for me.
http://localhost:8983/solr/select?debugQuery=onq=foo_s:;

The raw query parser would also work (it skips analysis):
http://localhost:8983/solr/select?debugQuery=onq={!raw f=foo_s}

But yes, indexing differently may be easier for you... either by
indexing a special token for _NONE_
or by leaving out the value completely (not indexing it), and using
facet.missing=true and filtering for none via fq=-myfield:[* TO *]

-Yonik
http://www.lucidimagination.com


Re: Sites with Innovative Presentation of Tags and Facets

2010-05-29 Thread Chris Hostetter

: I'm interested in this stuff, but what is a 'sparkline', and can I get a URL 
of an example?

The email in this thread where i first suggested that sparklines on 
numeric facets would be cool had two links, one to the definitive 
Sparklines essay by Tufte 

http://en.wikipedia.org/wiki/Sparkline
http://www.edwardtufte.com/bboard/q-and-a-fetch-msg?msg_id=0001OR





-Hoss



Re: Sites with Innovative Presentation of Tags and Facets

2010-05-29 Thread Chris Hostetter

: Here is one that I recently came across, and liked (look at the
: last example): http://moritz.stefaner.eu/projects/elastic-lists/
: The code has apparently also been recently open-sourced.

Ah... that is a pretty awesome visual UI for facets -- and they do use 
sparklines but not in the way i was suggesting.  If you show sparklines 
in that UI, then each facet *constraint* includes a sparkline showing it's 
distribution over time ... so in the nobel price demo, if you turn 
sparklines on and look at the prize facet, each type of prize has a 
sparkline showing how many were given out over the years (so it's easy to 
see that economics prizes were added relatively late) but there isn't a 
sparkline showing the statistical distribution of values across numeric 
fields -- the only numeric field is year (well, they also have decade but 
that's the same thing) and by having hte sparkline on the constraints 
instead of on the facet itself, you can't tell at quick glance wether the 
number of total prizes given out is trending up or down.

The sparklines also aren't updated as constraincts from other facets are 
applied -- if i click on the female constraint in the gender facet, i 
would like to see the sparklines on all of the other facets updated to 
provide a visual cue of how the results have changed for that 
facet/constraint (instead, this ui shrinks the bounding boxes arround each 
constraint in a collaping model -- which makes perfect sense given that 
the entire point of hte UI is elastic lists ... but it doesn't convey 
distribution information)



-Hoss



Re: Sites with Innovative Presentation of Tags and Facets

2010-05-29 Thread Chris Hostetter
: see that economics prizes were added relatively late) but there isn't a 
: sparkline showing the statistical distribution of values across numeric 
: fields -- the only numeric field is year (well, they also have decade but 
: that's the same thing) and by having hte sparkline on the constraints 
: instead of on the facet itself, you can't tell at quick glance wether the 
: number of total prizes given out is trending up or down.

FWIW: I found an article that really hits the nail on the head with what i 
was trying to suggest about using sparklines on numeric sliders...

http://www.uxmatters.com/mt/archives/2010/02/numeric-filters-issues-and-best-practices.php


-Hoss



Solr on Tomcat, how to use an external data directory?

2010-05-29 Thread jlist9
My solr index works fine with the embedded Jetty. I'm trying to move the
index to Tomcat. Following the wiki page http://wiki.apache.org/solr/SolrTomcat,
I put this line in setenv.bat:

set JAVA_OPTS=%JAVA_OPTS% -Dfile.encoding=UTF-8
-Dsolr.solr.home=D:\opt\solr\example

Tomcat seems to be picking it up. If I point it to a non-existent
directory or an empty
directory, Tomcat complains about it in console log.

However, the /solr/admin/stats.jsp page still shows that solr is using a default
directory, which is unsurprisingly located in tomcat bin dir:

readerDir : 
org.apache.lucene.store.simplefsdirect...@d:\Java\apache-tomcat-6.0.20\bin\solr\data\index

Any idea what else I need to do to use an external solr index
directory in Tomcat?

Thanks,
Jack


Re: Solr on Tomcat, how to use an external data directory?

2010-05-29 Thread K Wong
There are directions on this page under Tomcat on Windows  Multiple Solr apps:

http://wiki.apache.org/solr/SolrTomcat

I'm running a multicore install on RHEL5/Tomcat5.5 and I just followed
the docs and it went fine. I'm not sure that I can be of much help
other than pointing you to the relevant docs, since I am not running
this on Windows.

K



On Sat, May 29, 2010 at 1:48 PM, jlist9 jli...@gmail.com wrote:
 My solr index works fine with the embedded Jetty. I'm trying to move the
 index to Tomcat. Following the wiki page 
 http://wiki.apache.org/solr/SolrTomcat,
 I put this line in setenv.bat:

 set JAVA_OPTS=%JAVA_OPTS% -Dfile.encoding=UTF-8
 -Dsolr.solr.home=D:\opt\solr\example

 Tomcat seems to be picking it up. If I point it to a non-existent
 directory or an empty
 directory, Tomcat complains about it in console log.

 However, the /solr/admin/stats.jsp page still shows that solr is using a 
 default
 directory, which is unsurprisingly located in tomcat bin dir:

 readerDir : 
 org.apache.lucene.store.simplefsdirect...@d:\Java\apache-tomcat-6.0.20\bin\solr\data\index

 Any idea what else I need to do to use an external solr index
 directory in Tomcat?

 Thanks,
 Jack



Re: Solr on Tomcat, how to use an external data directory?

2010-05-29 Thread Abdelhamid ABID
Most likely you have missed to point data directory in solrconf.xml,
this should help :
http://wiki.apache.org/solr/SolrConfigXml#dataDir_parameter

On Sat, May 29, 2010 at 9:48 PM, jlist9 jli...@gmail.com wrote:

 My solr index works fine with the embedded Jetty. I'm trying to move the
 index to Tomcat. Following the wiki page
 http://wiki.apache.org/solr/SolrTomcat,
 I put this line in setenv.bat:

 set JAVA_OPTS=%JAVA_OPTS% -Dfile.encoding=UTF-8
 -Dsolr.solr.home=D:\opt\solr\example

 Tomcat seems to be picking it up. If I point it to a non-existent
 directory or an empty
 directory, Tomcat complains about it in console log.

 However, the /solr/admin/stats.jsp page still shows that solr is using a
 default
 directory, which is unsurprisingly located in tomcat bin dir:

 readerDir : org.apache.lucene.store.simplefsdirect...@d
 :\Java\apache-tomcat-6.0.20\bin\solr\data\index

 Any idea what else I need to do to use an external solr index
 directory in Tomcat?

 Thanks,
 Jack




-- 
Abdelhamid ABID
Software Engineer- J2EE / WEB


Re: Solr on Tomcat, how to use an external data directory?

2010-05-29 Thread jlist9
I only have one solr instance so I followed the Single Solr Instance,
which basically tells me to set -Dsolr.solr.home to the solr directory,
which I did ...

On Sat, May 29, 2010 at 2:32 PM, K Wong wongo...@gmail.com wrote:
 There are directions on this page under Tomcat on Windows  Multiple Solr 
 apps:

 http://wiki.apache.org/solr/SolrTomcat

 I'm running a multicore install on RHEL5/Tomcat5.5 and I just followed
 the docs and it went fine. I'm not sure that I can be of much help
 other than pointing you to the relevant docs, since I am not running
 this on Windows.

 K



 On Sat, May 29, 2010 at 1:48 PM, jlist9 jli...@gmail.com wrote:
 My solr index works fine with the embedded Jetty. I'm trying to move the
 index to Tomcat. Following the wiki page 
 http://wiki.apache.org/solr/SolrTomcat,
 I put this line in setenv.bat:

 set JAVA_OPTS=%JAVA_OPTS% -Dfile.encoding=UTF-8
 -Dsolr.solr.home=D:\opt\solr\example

 Tomcat seems to be picking it up. If I point it to a non-existent
 directory or an empty
 directory, Tomcat complains about it in console log.

 However, the /solr/admin/stats.jsp page still shows that solr is using a 
 default
 directory, which is unsurprisingly located in tomcat bin dir:

 readerDir : 
 org.apache.lucene.store.simplefsdirect...@d:\Java\apache-tomcat-6.0.20\bin\solr\data\index

 Any idea what else I need to do to use an external solr index
 directory in Tomcat?

 Thanks,
 Jack




Re: Solr on Tomcat, how to use an external data directory?

2010-05-29 Thread Chris Hostetter

: Most likely you have missed to point data directory in solrconf.xml,
: this should help :
: http://wiki.apache.org/solr/SolrConfigXml#dataDir_parameter

right .. double check what the dataDir setting looks like ... if it's 
unset it uses data in your solr instance directory, but if it is set, 
it's (unfortunately) evaluated relative to the current working directory 
of your servlet container and some versions of solr had ./data listed in 
the example solrconfig.xml



-Hoss



Re: NoSuchFieldError: submap

2010-05-29 Thread Mauricio Scheffer
Thanks Hoss, yeah I had an outdated jar, I ran ant clean then ant dist and
all is well now.

--
Mauricio

On Fri, May 28, 2010 at 3:39 PM, Chris Hostetter
hossman_luc...@fucit.orgwrote:


 : Hi, I'm trying to build from source to apply the field collapsing patch.
 : 'Ant dist' runs just fine, no errors, but at startup I get a
 : NoSuchFieldError: submap exception (stack trace:
 : http://pastebin.com/NXsf0KJS ). This is before sending any requests. I
 don't
 : have any 'submap' field defined anywhere.
 : Has anyone seen this? Any ideas?

 the field in question isn't refering to a field in your index -- it's a
 java error refering to a field of a java class.

 in a nutshell: some class file you are using at runtime is inconsistent
 with a class file thta you used at compile time.  the submap field of
 some object is manipulated on line 89 of your SynonymFilter.java file, but
 that object doesn't have a submap field.

 typically this type of problem happens when you doing have a clean
 classpath: older versions of jars are included as well, or the jars  you
 compiled against aren't included but other differnet jars with the same
 classes in them are.



 -Hoss




Re: facet order

2010-05-29 Thread search
oh yes thats correct, but that orders the items inside a facet.

i am using multiple facet fields. so i want to have the field with the
most items first.

thanks.

 use: facet.sort=true


 http://wiki.apache.org/solr/SimpleFacetParameters#facet.sort


 On Sat, May 29, 2010 at 3:53 PM, sea...@butterflycluster.com wrote:

 Hi,

 how can i configuratively order facets according to total count of facet
 fields?

 for example - facets with the highest count be on top.

 facet1 [0]
 abc (20)
 def (18)
 ghi (16)

 facet2 [1]
 jkl (10)
 mno (9)
 pqr (2)

 thanks

 dev.




 --
 Furkan Kuru





RIA sample and minimal JARs required to embed Solr

2010-05-29 Thread Thomas J. Buhr
Solr,

The Solr 1.4 EES book arrived yesterday and I'm very much enjoying it. I was 
glad to see that rich clients are one case for embedding Solr as this is the 
case for my application. Multi Cores will also be important for my RIA.

The book covers a lot and makes it clear that Solr has extensive abilities. 
There is however no clean and simple sample of embedding Solr in a RIA in the 
book, only a few alternate language usage samples. Is there a link to a Java 
sample that simply embeds Solr for local indexing and searching using Multi 
Cores?

Also, what kind of memory footprint am I looking at for embedding Solr? What 
are the minimal dependancies?

Thom

Re: Solr on Tomcat, how to use an external data directory?

2010-05-29 Thread jlist9
Thanks! I tried that and it worked. It turned out that I also need to set
-Dsolr.data.dir=/opt/solr/example/data

On Sat, May 29, 2010 at 3:07 PM, Abdelhamid  ABID aeh.a...@gmail.com wrote:
 Most likely you have missed to point data directory in solrconf.xml,
 this should help :
 http://wiki.apache.org/solr/SolrConfigXml#dataDir_parameter

 On Sat, May 29, 2010 at 9:48 PM, jlist9 jli...@gmail.com wrote:

 My solr index works fine with the embedded Jetty. I'm trying to move the
 index to Tomcat. Following the wiki page
 http://wiki.apache.org/solr/SolrTomcat,
 I put this line in setenv.bat:

 set JAVA_OPTS=%JAVA_OPTS% -Dfile.encoding=UTF-8
 -Dsolr.solr.home=D:\opt\solr\example

 Tomcat seems to be picking it up. If I point it to a non-existent
 directory or an empty
 directory, Tomcat complains about it in console log.

 However, the /solr/admin/stats.jsp page still shows that solr is using a
 default
 directory, which is unsurprisingly located in tomcat bin dir:

 readerDir : org.apache.lucene.store.simplefsdirect...@d
 :\Java\apache-tomcat-6.0.20\bin\solr\data\index

 Any idea what else I need to do to use an external solr index
 directory in Tomcat?

 Thanks,
 Jack




 --
 Abdelhamid ABID
 Software Engineer- J2EE / WEB



Re: Solr on Tomcat, how to use an external data directory?

2010-05-29 Thread jlist9
You are right. Thanks Chris!

On Sat, May 29, 2010 at 4:15 PM, Chris Hostetter
hossman_luc...@fucit.org wrote:

 : Most likely you have missed to point data directory in solrconf.xml,
 : this should help :
 : http://wiki.apache.org/solr/SolrConfigXml#dataDir_parameter

 right .. double check what the dataDir setting looks like ... if it's
 unset it uses data in your solr instance directory, but if it is set,
 it's (unfortunately) evaluated relative to the current working directory
 of your servlet container and some versions of solr had ./data listed in
 the example solrconfig.xml



 -Hoss




Luke browser does not show non-String Solr fields?

2010-05-29 Thread jlist9
I tried the stand-alone Luke tool (not Luke request handler) to browse
a solr index and find a few strange things:

1. Queries like id:123 which work fine in /solr/admin web interface
returns nothing in Luke. *:* returns everything fine in Luke.

2. When Luke displays records with query *:*, it shows the string
values fine but the numeric fields and date fields shows blank. It shows
DocID OK, though.

Anyone else has tried Luke on a solr index?


Re: facet order

2010-05-29 Thread Chris Hostetter

: i am using multiple facet fields. so i want to have the field with the
: most items first.

the facet fields are returned i nthe order they were requested in the 
params -- reordering them in the client is trivial, so setting up special 
commands to re-order them on the server side isn't something that has ever 
been (or is likely to ever be) added.

FWIW...

:  how can i configuratively order facets according to total count of facet
:  fields?

...if you truely want them ordered by the *total* count (and not just the 
sum of returend counts -- ie: if you want to include the long tail beyond 
what's left out because of facet.limit and facet.mincount) then just add a 
facet.query=field1:[*+TO+*] to get the total count of documents with a 
value for that field.


-Hoss



Re: Build query programmatically with lucene, but issue to solr?

2010-05-29 Thread Chris Hostetter

: I am building up a query with quite a bit of logic such as parentheses, plus
: signs, etc... and it's a little tedious dealing with it all at a string
: level.  I was wondering if anyone has any thoughts on constructing the query
: in lucene and using the string representation of the query to send to solr.

There was another thread where this idea came up recently; although your 
motivation is very differnet from that persons, all of the general 
concerns still apply...

http://search.lucidimagination.com/search/document/514adb88f7288ac1/can_i_use_per_field_analyzers_and_dynamic_fields

...that's not to say that it couldn't be *made* to work out, assuming a 
limited set of Query types and a specific analyzer were used, but it's not 
really waht the Query class (or it's toString method) were designed for -- 
it would probably be easier to write a custom query string formatter 
class, or to implement a QParser to apply your logic on the server side 
given simple key-val(s) input params.

-Hoss



Re: Rebuild an index

2010-05-29 Thread Chris Hostetter

: We use Drupal as the CMS and Solr for our search engine needs and are
: planning to have Solr Master-Slave replication setup across the data
: centers. I am in the process of testing my replication - what is the
: best means to delete the index on the Solr slave and then replicate a
: fresh copy from Master?  We use Solr 1.3.

I don't really understand your question -- if you wnat to test 
replication, just update some docs on the master and monitor the slave 
(via logs and test queries) to see when those changes show up.

asking about deleting the index seems  oddd ... XY problem?

http://people.apache.org/~hossman/#xyproblem
XY Problem

Your question appears to be an XY Problem ... that is: you are dealing
with X, you are assuming Y will help you, and you are asking about Y
without giving more details about the X so that we can understand the
full issue.  Perhaps the best solution doesn't involve Y at all?
See Also: http://www.perlmonks.org/index.pl?node_id=542341



-Hoss



Re: nested querries, and LocalParams syntax

2010-05-29 Thread Chris Hostetter

In addition to yonik's point about the LocalParams wiki page (and please 
let us know if you aren't sure of the answers to any of your questions 
after reading it) I wanted to clear up one thing...

: Let's start with that not-nested query example.   Can you in fact use it as
: above, to force dismax handling of the 'q' even if the qt or request handler

Quick side note: qt determines the ReequestHandler -- if it's dismax 
then you get the DisMaxRequestHandler which in recent versions of solr is 
just a thin subclass of the SearchHandler subclass where the 
default value of defType (which is used to pick a QParser) is dismax 
instead of lucene ... i tried to explain this in a recent blog...

http://www.lucidimagination.com/blog/2010/05/23/whats-a-dismax/

... the key thing to note is that defType is a param that is specific to 
SearchHandler -- if you use qt to pick some other third party 
RequestHandler, it's not neccessarily going to do *anything* and the 
nested params syntax may not work at all.

: default is something else?  The documentation is confusing: In standard Solr
: search handlers, the default type of the main query only may be specified via
: the defType parameter. The default type of all other query parameters will
: remain lucene http://wiki.apache.org/solr/SolrQuerySyntax#lucene.
: 
: I _think_ it's trying to say that I _can_, even in a standard search handler,
: force dismax with {!dismax}, I just can't change the type of _other_ query
: parameters -- rather than saying that I _can't_ use {!dismax} to force dismax
: type of 'q' in a standard search handler.  Yes?

You're right, it is confusing -- the point is tha defType changes the 
default QParser type for the q param -- but it doesn't change it for 
any other param.  I've improved the wording, but the key to keep in mind 
is that that is completley orthoginal to using the local params syntax 
that you asked about.

What that documentation is trying to illustrate is that in this request...

   defType=XXXq=AAAfq=BBB

...the XXX QParser will be used to parse the value AAA -- but the 
stock lucene QParser will be used to parse the fq param

Regardless of the value of defType, if you put the local params 
syntax ({!foo}) at the begining of a query param, you can force that param 
to be parsed the way you wish...

   defType=XXXq={!foo}AAAfq={!bar}BBB

...in that example, neither the XXX or lucene QParsers are ever used.



-Hoss