Re: [Update] Solr can be started from jetty but not tomcat
Hi, I am using java 1.6, the tomcat 5.5 also contains this file. But actually this is the problems solved by adding 4 jar...I think the problem may be come will xalan.jar ( resolver.jar I am not so sure) but anyone in trouble can try this sequence xalan.jar->resolver.jar->xercesImpl.jar->xml-apis.jar (I tried the resolver.jar come first, but problem still no solvedmay be someone can try this sequence...) hossman wrote: > > > : By your hint, I found there is XML library missing in tomcat, not > ... > : ** Also please someone kindly put those library in the system > requirement/ > : wiki FAQ. That save a lot of people and forum resources if all library > used > : is listed properly (with links) - newly build tomcat server have > nothing... > > While there are certainly XML libraries out there that can provide > alternate implementaions of XPathFactory, there is normally no *need* for > any extra XML libraries... > > : >> at > org.apache.catalina.startup.Bootstrap.main(Bootstrap.java:433) > : >> Caused by: java.lang.RuntimeException: XPathFactory#newInstance() > failed > : >> to > : >> create an XPathFactory for the default object model: > : >> http://java.sun.com/jaxp/xpath/dom with the > : >> XPathFactoryConfigurationException: > : >> javax.xml.xpath.XPathFactoryConfiguration > > ... Java 1.5 provides that object model out of the box. > > a quick google search for the error you encountered turns up several > promising links, most notably this post... > > http://www.psuedoguru.com/roller/page/psuedoblog?entry=failed_to_create_an_xpathfactory > > ...which indicates... > Finally found out that the problem was due to the Tomcat Java 1.4 compatibility libraries. I had recently upgraded to Java 1.5, but forgot to remove them. $TOMCAT_HOME/common/endorsed/xercesImpl.jar $TOMCAT_HOME/common/endorsed/xml-apis.jar > > With Java 1.5 and a completley clean tomcat install, you shouldn't get > this error. > > > > > -Hoss > > > -- View this message in context: http://www.nabble.com/-Update--Solr-can-be-started-from-jetty-but-not-tomcat-tp15998642p16047002.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: field collapsing
The latest one won't apply to the trunk because it's too old. It hasn't been updated to match changes made to Solr since mid-February. One of the things I know has to change is that in CollapseComponent->prepare/process, the parameters need to change to just accept a ResponseBuilder. Other than that, I'm not sure what will have to be changed. I'm not planning on updating our Solr build until 1.3 is released. Doug muddassir hasan wrote: Hi, I have unsuccessfully tried to apply solr field collapsing patches available at https://issues.apache.org/jira/browse/SOLR-236?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel One of the patch could be applied to trunk but it could not be compiled. Please let me know which of the available field collapsing patch could be applied to solr trunk or release 1.2.0. Thanks. M.Hasan - Now you can chat without downloading messenger. Click here to know how.
Re: Question about dismax query parsing
On Mar 13, 2008, at 3:06 AM, Brendan Grainger wrote:
Just started using the Dismax handler and it looks very promising.
However I'm a little confused about this query. Could somebody
please explain why I'm getting a phrase query here?
You're not actually getting a PhraseQuery. The
DisjunctionMaxQuery#toString() puts a "~" in for the tie breaker. In
general, a Query#toString is lossy/confusing - but better than nothing.
public String toString(String field) {
StringBuffer buffer = new StringBuffer();
buffer.append("(");
for (int i = 0 ; i < disjuncts.size(); i++) {
Query subquery = (Query) disjuncts.get(i);
if (subquery instanceof BooleanQuery) { // wrap sub-bools in
parens
buffer.append("(");
buffer.append(subquery.toString(field));
buffer.append(")");
}
else buffer.append(subquery.toString(field));
if (i != disjuncts.size()-1) buffer.append(" | ");
}
buffer.append(")");
if (tieBreakerMultiplier != 0.0f) {
buffer.append("~");
buffer.append(tieBreakerMultiplier);
}
if (getBoost() != 1.0) {
buffer.append("^");
buffer.append(getBoost());
}
return buffer.toString();
}
+(((title_t:mass) (title_t:air) (title_t:flow))~3) ()',
And is that extra () indicative of something? I have some stuff
going on with synonyms and I'm wondering if the position of the
tokens is off and causing this.
That extra clause is Solr's dismax code putting in an empty clause -
I think that is the boosting query(?).
Erik
The relevant output from debugQuery is below:
'rawquerystring'=>'mass air flow',
'querystring'=>'mass air flow',
'parsedquery'=>'+((DisjunctionMaxQuery((title_t:mass))
DisjunctionMaxQuery((title_t:air)) DisjunctionMaxQuery
((title_t:flow)))~3) ()',
'parsedquery_toString'=>'+(((title_t:mass) (title_t:air)
(title_t:flow))~3) ()',
Thanks!
Re: Question about dismax query parsing
Got it.
Thanks so much.
Brendan
On Mar 14, 2008, at 8:11 AM, Erik Hatcher wrote:
On Mar 13, 2008, at 3:06 AM, Brendan Grainger wrote:
Just started using the Dismax handler and it looks very promising.
However I'm a little confused about this query. Could somebody
please explain why I'm getting a phrase query here?
You're not actually getting a PhraseQuery. The
DisjunctionMaxQuery#toString() puts a "~" in for the tie breaker.
In general, a Query#toString is lossy/confusing - but better than
nothing.
public String toString(String field) {
StringBuffer buffer = new StringBuffer();
buffer.append("(");
for (int i = 0 ; i < disjuncts.size(); i++) {
Query subquery = (Query) disjuncts.get(i);
if (subquery instanceof BooleanQuery) { // wrap sub-bools in
parens
buffer.append("(");
buffer.append(subquery.toString(field));
buffer.append(")");
}
else buffer.append(subquery.toString(field));
if (i != disjuncts.size()-1) buffer.append(" | ");
}
buffer.append(")");
if (tieBreakerMultiplier != 0.0f) {
buffer.append("~");
buffer.append(tieBreakerMultiplier);
}
if (getBoost() != 1.0) {
buffer.append("^");
buffer.append(getBoost());
}
return buffer.toString();
}
+(((title_t:mass) (title_t:air) (title_t:flow))~3) ()',
And is that extra () indicative of something? I have some stuff
going on with synonyms and I'm wondering if the position of the
tokens is off and causing this.
That extra clause is Solr's dismax code putting in an empty clause -
I think that is the boosting query(?).
Erik
The relevant output from debugQuery is below:
'rawquerystring'=>'mass air flow',
'querystring'=>'mass air flow',
'parsedquery'=>'+((DisjunctionMaxQuery((title_t:mass))
DisjunctionMaxQuery((title_t:air))
DisjunctionMaxQuery((title_t:flow)))~3) ()',
'parsedquery_toString'=>'+(((title_t:mass) (title_t:air)
(title_t:flow))~3) ()',
Thanks!
Solrj jar?
I am sorry for the newbie question but I have searched every where i can not seem to find a build of the solr java client. It seems as though perhaps it does not exist. Is that true? thanks, tashfeen - Looking for last minute shopping deals? Find them fast with Yahoo! Search.
Re: What can get past document uniqueness?
Doesn't look like it. We do rsyncing but only as a backup for this index-- these queries are hitting the live index. Also, the results we get back are not exact duplicates, even though the ID is the same. For example, if we update a document (replace an existing document) with new information, the index will only sometimes store two copies -- one with the old data and one with the new content. If I update again (with the same content) the duplicate goes away. I have optimized & committed the index with no change to the pre- existing duplicates. Where within Solr is uniqueness enforced? I'd like to at least put some debug checking in there. On Mar 13, 2008, at 4:47 PM, Ryan McKinley wrote: Check this thread: http://www.nabble.com/duplicate-entries-being-returned%2C-possible-caching-issue--td15237016.html perhaps it is related? Brian Whitman wrote: On a solr instance with id This is happening: http://solr/select?q=id:abc123&fl=id abc123 abc123 Lots of weird stuff is writing to this index: solrj code, python solr.py, curl, etc. -- many things at the same time. Autocommit is at 30m. 4500 of the ca. 1.5m docs in this index are doubled like this. What can get past the doc uniqueness constraint? Has anyone seen this before?
Re: Solrj jar?
Hi Tashfeen, SolrJ (the java client) is going to be released with the upcoming Solr 1.3 You can download a nightly build and run "ant dist" to build SolrJ jar. It's very stable and well tested so you shouldn't have any problems. On Fri, Mar 14, 2008 at 8:25 PM, Tashfeen Ekram <[EMAIL PROTECTED]> wrote: > I am sorry for the newbie question but I have searched every where i can not > seem to find a build of the solr java client. > It seems as though perhaps it does not exist. Is that true? > > thanks, > tashfeen > > > > - > Looking for last minute shopping deals? Find them fast with Yahoo! Search. -- Regards, Shalin Shekhar Mangar.
Re: What can get past document uniqueness?
On Mar 14, 2008, at 11:37 AM, Yonik Seeley wrote: During the add (in DirectUpdateHandler2) docs are kept track of, and during a commit they are checked for dups. That code has been very well tested though, and I've only seen duplicates on a JVM crash/restart. That's because docs are added to the index, but Solr never gets a chance to remove the duplicates (and that info is only kept in RAM). This is certainly possible. The dupes are all within the same indexed time period, and yep, just checked the logs, Resin crashed right around then (Lock obtain timed out: SingleInstanceLock: write.lock). So, if we sent 4000 "updates" (replacements of docs already existing) to the index that has a 30m autocommit, and it crashed before the commit could happen, when it started back up I'd see both copies like that?
Re: What can get past document uniqueness?
On Fri, Mar 14, 2008 at 11:49 AM, Brian Whitman <[EMAIL PROTECTED]> wrote: > So, if we sent 4000 "updates" (replacements of docs already existing) > to the index that has a 30m autocommit, and it crashed before the > commit could happen, when it started back up I'd see both copies like > that? Yep. We could remove dups after the fact, but there is currently no code in Solr to do that. -Yonik
Re: What can get past document uniqueness?
On Fri, Mar 14, 2008 at 11:31 AM, Brian Whitman <[EMAIL PROTECTED]> wrote: > Where within Solr is uniqueness enforced? I'd like to at least put > some debug checking in there. During the add (in DirectUpdateHandler2) docs are kept track of, and during a commit they are checked for dups. That code has been very well tested though, and I've only seen duplicates on a JVM crash/restart. That's because docs are added to the index, but Solr never gets a chance to remove the duplicates (and that info is only kept in RAM). Lucene can now handle this (updating) in an atomic fashion, so we should probably start using that. -Yonik
Re: What can get past document uniqueness?
: > Where within Solr is uniqueness enforced? I'd like to at least put : > some debug checking in there. : : During the add (in DirectUpdateHandler2) docs are kept track of, and : during a commit they are checked for dups. That code has been very Just for the record: there is one other easy way docs with identical duplicate keys could get added to the index: if the client sending the (duplicate) doc explicitly uses the 'allowDups="true"' option on the command. -Hoss
Re: Question about dismax query parsing
: > +(((title_t:mass) (title_t:air) (title_t:flow))~3) ()', : > And is that extra () indicative of something? I have some stuff going on : > with synonyms and I'm wondering if the position of the tokens is off and : > causing this. : : That extra clause is Solr's dismax code putting in an empty clause - I think : that is the boosting query(?). boost queries and functions are left out if they aren't specified ... but it looks like an empty "pf" (phrase fields) param causes an empty clause to be added ... i guess i assumed pf would always be used. the fix should be minor, i'll open a bug to track it, but the impact is pretty minor too ... the empty clause might through off scores because of hte coordFactory but honestly you should almost always be better off setting the pf to something so you can get higher scores for docs where all of the query terms appear in close proximity. -Hoss
Re: searching from command line?
Great! I'll take a look at Luke. Thanks. - peter jonbaer wrote: > > You can do this via Luke > > http://www.getopt.org/luke/ > > -snip- > Command-line argument parsing. Now you can open an index on startup, > and optionally execute a script > Scripting plugin, which allows you to interactively experiment with > Luke and Lucene indexes. This plugin also can run scripts from Luke > command-line. > -snip- > > - Jon > > On Mar 13, 2008, at 7:12 PM, peter360 wrote: > >> >> Hi, >> >> Is anyone aware of a command line tool that builds and searches a >> solr index >> without running solr as a servlet? >> >> My plan is to do the following: build and validate an index on a >> single >> indexer machine, then push the index to a few search machines. It >> seems to >> me that there is no need to run a servlet container on the indexer >> box if a >> tool as mentioned above exists. >> >> Can someone point me to the right direction? >> Thanks! >> Peter >> -- >> View this message in context: >> http://www.nabble.com/searching-from-command-line--tp16040889p16040889.html >> Sent from the Solr - User mailing list archive at Nabble.com. >> > > > -- View this message in context: http://www.nabble.com/searching-from-command-line--tp16040889p16048942.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: searching from command line?
Thanks for your suggestion and I pretty much agree. Part of the reason, which I didn't mention in my original question, that I am looking for a command line tool is to use it for quick diagnosis. I could point it to a different index just by changing one of the command line parameters, without having to modify the solrconfig.xml file and restart a servlet container, then go to a browser to issue query. Thanks for all who replied. The solr community is impressive. - peter Golly, let me think. I can use the out-of-the-box, tested Solr stuff for syncing indexes or I can invent some command line kludge that does the same thing, except I will need to write it and test it myself. Which one is easier? Seriously, the existing Solr index distribution is great stuff. I strongly recommend that you try it and ask the list about any problems you run into. I'm quite impressed with Solr. It is working very well here at Netflix. We have 2M queries/day on the front end and 6M qpd at the search farm (five servers). wunder -- View this message in context: http://www.nabble.com/searching-from-command-line--tp16040889p16049036.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: Finding an empty field
: Somehow the index has acquired one record out of millions in which an : integer value has been populated by an empty string. I would like to isolate : this record and remove it. This field exists solely to make sorting faster, : and since it has an empty record, sorting blows up. : : Is it possible to find this record? Is there any way to differentiate : between this record and all of the other records which have real numbers : populated? have you tried searching for... dateorigin_sort:"" ? : This query will isolate records which do not have the field populated. (It : works on all field types.) : -dateorigin_sort:[* TO *] : But, since this record is an integer (not an sint) no other range query : works. well, technically range queries "work" they just don't "work" on numeric ranges ... they'd be lexigraphical ranges on the string value, so... dateorigin_sort:[* TO " "] ...could probably help you find anything that is lexigraphically lower then a string representation of an integer (assuming dateorigin_sort:"" doesn't work) disclaimer: i haven't actaully tested either of these on an index with a bogus integer like you describe ... but i'm pretty sure they should work given what i'm remembering about the code) -Hoss
RE: Finding an empty field
dateorigin_sort:"" gives a syntax error. I'm using Solr 1.2. Should this work in Solr 1.3? Is it legal in a newer Lucene parser? message Query parsing error: Cannot parse 'dateorigin_sort:""': Lexical error at line 1, column 19. Encountered: after : "\"\"" description The request sent by the client was syntactically incorrect (Query parsing error: Cannot parse 'dateorigin_sort:""': Lexical error at line 1, column 19. Encountered: after : "\"\""). Thanks, Lance -Original Message- From: Chris Hostetter [mailto:[EMAIL PROTECTED] Sent: Friday, March 14, 2008 11:38 AM To: solr-user@lucene.apache.org Subject: Re: Finding an empty field : Somehow the index has acquired one record out of millions in which an : integer value has been populated by an empty string. I would like to isolate : this record and remove it. This field exists solely to make sorting faster, : and since it has an empty record, sorting blows up. : : Is it possible to find this record? Is there any way to differentiate : between this record and all of the other records which have real numbers : populated? have you tried searching for... dateorigin_sort:"" ? : This query will isolate records which do not have the field populated. (It : works on all field types.) : -dateorigin_sort:[* TO *] : But, since this record is an integer (not an sint) no other range query : works. well, technically range queries "work" they just don't "work" on numeric ranges ... they'd be lexigraphical ranges on the string value, so... dateorigin_sort:[* TO " "] ...could probably help you find anything that is lexigraphically lower then a string representation of an integer (assuming dateorigin_sort:"" doesn't work) disclaimer: i haven't actaully tested either of these on an index with a bogus integer like you describe ... but i'm pretty sure they should work given what i'm remembering about the code) -Hoss
Re: searching from command line?
On Mar 14, 2008, at 2:23 PM, peter360 wrote:
Thanks for your suggestion and I pretty much agree. Part of the
reason,
which I didn't mention in my original question, that I am looking
for a
command line tool is to use it for quick diagnosis. I could point
it to a
different index just by changing one of the command line
parameters, without
having to modify the solrconfig.xml file and restart a servlet
container,
then go to a browser to issue query.
For the record, you can point to a different index with Solr with a
command-line startup parameter (provided you have the directory in
solrconfig parameterized this way): ${solr.data.dir:./solr/
data}
java -Dsolr.data.dir=some/other/directory -jar start.jar
Startup time for Solr is a few seconds max for me.
And no browser needed:
curl "http://localhost:8983/solr/select?q=whatever&wt=ruby&indent=on";
(ruby and indentation enabled for readability in a console).
Erik
RE: Finding an empty field
: dateorigin_sort:"" gives a syntax error. I'm using Solr 1.2. Should : this work in Solr 1.3? Is it legal in a newer Lucene parser? Hmm.. not sure. did you try the range query suggestion? ... : well, technically range queries "work" they just don't "work" on numeric : ranges ... they'd be lexigraphical ranges on the string value, so... : : dateorigin_sort:[* TO " "] : : ...could probably help you find anything that is lexigraphically lower : then a string representation of an integer (assuming dateorigin_sort:"" : doesn't work) -Hoss
AlphaNumeric search in Solr
Hi, I am trying to do alphanumeric search for terms like "axd110", but I don't get any results back from Solr. When I open my index using Luke, I am able to search for these terms and get results back, so I definitely know that the terms are present in the index. Below is the query section from my solr schema, Is there anything that I am missing here. -- View this message in context: http://www.nabble.com/AlphaNumeric-search-in-Solr-tp16063101p16063101.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: AlphaNumeric search in Solr
Was this index generated with Solr? If so, what does the index analyzer look like for that field? WordDelimiterFilter would split axd110 into two tokens, so it must have been indexed in a compatible way (see the example schema.xml). -Yonik On Fri, Mar 14, 2008 at 8:51 PM, solr_user <[EMAIL PROTECTED]> wrote: > > Hi, > > I am trying to do alphanumeric search for terms like "axd110", but I don't > get any results back from Solr. When I open my index using Luke, I am able > to search for these terms and get results back, so I definitely know that > the terms are present in the index. Below is the query section from my solr > schema, > > > > > generateWordParts="1" generateNumberParts="1" catenateWords="0" > catenateNumbers="0" catenateAll="0" splitOnCaseChange="1"/> > > > > > > > Is there anything that I am missing here. > > -- > View this message in context: > http://www.nabble.com/AlphaNumeric-search-in-Solr-tp16063101p16063101.html > Sent from the Solr - User mailing list archive at Nabble.com. > >
RE: Finding an empty field
It was a surprise to discover that dateorigin_sort:"" is a syntax error, but dateorigin_sort:["" TO *] is legit. This says that there's a bug in the Lucene syntax parser? Anyway, with a little more research I discover that this query: http://64.71.164.205:8080/solr/select/?q=*:*&version=2.2&start=0&rows=0&; indent=on&facet=true&facet.field=dateorigin_sort&facet.mincount=0&facet. sort=false .../solr/select/?q=*:*&version=2.2&start=0&rows=0&indent=on&facet=true&f acet.field=dateorigin_sort&facet.mincount=0&facet.sort=false This query says, "Select all records in the index. For each indexed value of dateorigin_sort, count the number of records with that value." It yields the following output snippet: 0 1 1 Umm... it has an indexed empty value that does not correspond to a record? Is it an unanchored data item in the index? Would optimizing make this index data go away? Thanks, Lance -Original Message- From: Chris Hostetter [mailto:[EMAIL PROTECTED] Sent: Friday, March 14, 2008 4:46 PM To: solr-user@lucene.apache.org Subject: RE: Finding an empty field : dateorigin_sort:"" gives a syntax error. I'm using Solr 1.2. Should : this work in Solr 1.3? Is it legal in a newer Lucene parser? Hmm.. not sure. did you try the range query suggestion? ... : well, technically range queries "work" they just don't "work" on numeric : ranges ... they'd be lexigraphical ranges on the string value, so... : : dateorigin_sort:[* TO " "] : : ...could probably help you find anything that is lexigraphically lower : then a string representation of an integer (assuming dateorigin_sort:"" : doesn't work) -Hoss
Re: AlphaNumeric search in Solr
Hi Yonik, No this index was not generated using Solr. I just have the index files without access to the source that generated those files. Is there a way that I can change my Solr schema so that it wont split axd110 into two tokens. Yonik Seeley wrote: > > Was this index generated with Solr? If so, what does the index > analyzer look like for that field? > WordDelimiterFilter would split axd110 into two tokens, so it must > have been indexed in a compatible way (see the example schema.xml). > > -Yonik > > On Fri, Mar 14, 2008 at 8:51 PM, solr_user <[EMAIL PROTECTED]> wrote: >> >> Hi, >> >> I am trying to do alphanumeric search for terms like "axd110", but I >> don't >> get any results back from Solr. When I open my index using Luke, I am >> able >> to search for these terms and get results back, so I definitely know >> that >> the terms are present in the index. Below is the query section from my >> solr >> schema, >> >> >> >> >> > generateWordParts="1" generateNumberParts="1" catenateWords="0" >> catenateNumbers="0" catenateAll="0" splitOnCaseChange="1"/> >> >> >> >> >> >> >> Is there anything that I am missing here. >> >> -- >> View this message in context: >> http://www.nabble.com/AlphaNumeric-search-in-Solr-tp16063101p16063101.html >> Sent from the Solr - User mailing list archive at Nabble.com. >> >> > > -- View this message in context: http://www.nabble.com/AlphaNumeric-search-in-Solr-tp16063101p16063747.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: AlphaNumeric search in Solr
On Fri, Mar 14, 2008 at 10:15 PM, solr_user <[EMAIL PROTECTED]> wrote: > No this index was not generated using Solr. I just have the index files > without access to the source that generated those files. Is there a way > that I can change my Solr schema so that it wont split axd110 into two > tokens. The ideal answer would be to use the analyzer that generated the index. If you don't exactly know, remove the WordDelimiterFilter for starts. -Yonik
question on xsl sytlesheet and update
Hi all, I have 2 problem: 1. how to attach my own stylesheet to the output? 2. how the Solr do the update? What kind of update I can ask Solr to do? Thank you for any answer. -- View this message in context: http://www.nabble.com/question-on-xsl-sytlesheet-and-update-tp16064251p16064251.html Sent from the Solr - User mailing list archive at Nabble.com.
