Re: Doc's FunctionQuery result field in my custom SearchComponent class ?
Eric ,
In freq:termfreq(product,'spider') , freq is alias for 'termfreq' function
query so I could have that field with name 'freq' in document response.
this is my code which I am using to get document object and there is no
termfreq field in its fields collection.
DocList docs = rb.getResults().docList;
DocIterator iterator = docs.iterator();
int sumFreq = 0;
String id = null;
for (int i = 0; i docs.size(); i++) {
try {
int docId = iterator.nextDoc();
// Document doc = searcher.doc(docId, fieldSet);
Document doc = searcher.doc(docId);
Thanks,
Tony
On Wed, Jul 17, 2013 at 5:30 PM, Erick Erickson erickerick...@gmail.comwrote:
Where are you getting the syntax
freq:termfreq(product,'spider')
? Try just
termfreq(product,'spider')
you'll get an element in the doc labeled 'termfreq', at least
I do.
Best
Erick
On Tue, Jul 16, 2013 at 1:03 PM, Tony Mullins tonymullins...@gmail.com
wrote:
OK, So thats why I cannot see the FunctionQuery fields in my
SearchComponent class.
So then question would be how can I apply my custom processing/logic to
these FunctionQuery ? Whats the ExtensionPoint in Solr for such
scenarios ?
Basically I want to call termfreq() for each document and then apply the
sum to all doc's termfreq() results and show in one aggregated TermFreq
field in my query response.
Thanks.
Tony
On Tue, Jul 16, 2013 at 6:01 PM, Jack Krupansky j...@basetechnology.com
wrote:
Basically, the evaluation of function queries in the fl parameter
occurs
when the response writer is composing the document results. That's AFTER
all of the search components are done.
SolrReturnFields.**getTransformer() gets the DocTransformer, which is
really a DocTransformers, and then a call to
DocTransformers.transform() in
each response writer will evaluate the embedded function queries and
insert
their values in the results as they are being written.
-- Jack Krupansky
-Original Message- From: Tony Mullins
Sent: Tuesday, July 16, 2013 1:37 AM
To: solr-user@lucene.apache.org
Subject: Re: Doc's FunctionQuery result field in my custom
SearchComponent
class ?
No sorry, I am still not getting the termfreq() field in my 'doc'
object.
I do get the _version_ field in my 'doc' object which I think is
realValue=StoredField.
At which point termfreq() or any other FunctionQuery field becomes the
part
of doc object in Solr ? And at that point can I perform some custom
logic
and append the response ?
Thanks.
Tony
On Tue, Jul 16, 2013 at 1:34 AM, Patanachai Tangchaisin
patanachai.tangchaisin@**wizecommerce.com
patanachai.tangchai...@wizecommerce.com
wrote:
Hi,
I think the process of retrieving a stored field (through fl) is
happens
after SearchComponent.
One solution: If you wrap a q params with function your score will be a
result of the function.
For example,
http://localhost:8080/solr/collection2/demoendpoint?q=**
http://localhost:8080/solr/**collection2/demoendpoint?q=**
termfreq%28product,%27spider%27%29wt=xmlindent=truefl=***,**score
http://localhost:**8080/solr/collection2/**demoendpoint?q=termfreq%**
28product,%27spider%27%29wt=**xmlindent=truefl=*,score
http://localhost:8080/solr/collection2/demoendpoint?q=termfreq%28product,%27spider%27%29wt=xmlindent=truefl=*,score
Now your score is going to be a result of termfreq(product,'spider')
--
Patanachai Tangchaisin
On 07/15/2013 12:01 PM, Tony Mullins wrote:
any help plz !!!
On Mon, Jul 15, 2013 at 4:13 PM, Tony Mullins
tonymullins...@gmail.com
*
*wrote:
Please any help on how to get the value of 'freq' field in my custom
SearchComponent ?
http://localhost:8080/solr/collection2/demoendpoint?q=**
http://localhost:8080/solr/**collection2/demoendpoint?q=**
spiderwt=xmlindent=truefl=*,freq:termfreq%28product,%**
27spider%27%29http://**localhost:8080/solr/**
collection2/demoendpoint?q=**spiderwt=xmlindent=truefl=***
,freq:termfreq%28product,%**27spider%27%29
http://localhost:8080/solr/collection2/demoendpoint?q=spiderwt=xmlindent=truefl=*,freq:termfreq%28product,%27spider%27%29
docstr name=id11/strstr name=typeVideo Games/strstr
name=formatxbox 360/strstr name=productThe Amazing
Spider-Man/strint name=popularity11/intlong
name=_version_1439994081345273856/longint
name=freq1/int/doc
Here is my code
DocList docs = rb.getResults().docList;
DocIterator iterator = docs.iterator();
int sumFreq = 0;
String id = null;
for (int i = 0; i docs.size(); i++) {
try {
int docId = iterator.nextDoc();
// Document doc = searcher.doc(docId, fieldSet);
Document doc = searcher.doc(docId);
In doc object I can
Re: Doc's FunctionQuery result field in my custom SearchComponent class ?
OK, So thats why I cannot see the FunctionQuery fields in my
SearchComponent class.
So then question would be how can I apply my custom processing/logic to
these FunctionQuery ? Whats the ExtensionPoint in Solr for such scenarios ?
Basically I want to call termfreq() for each document and then apply the
sum to all doc's termfreq() results and show in one aggregated TermFreq
field in my query response.
Thanks.
Tony
On Tue, Jul 16, 2013 at 6:01 PM, Jack Krupansky j...@basetechnology.comwrote:
Basically, the evaluation of function queries in the fl parameter occurs
when the response writer is composing the document results. That's AFTER
all of the search components are done.
SolrReturnFields.**getTransformer() gets the DocTransformer, which is
really a DocTransformers, and then a call to DocTransformers.transform() in
each response writer will evaluate the embedded function queries and insert
their values in the results as they are being written.
-- Jack Krupansky
-Original Message- From: Tony Mullins
Sent: Tuesday, July 16, 2013 1:37 AM
To: solr-user@lucene.apache.org
Subject: Re: Doc's FunctionQuery result field in my custom SearchComponent
class ?
No sorry, I am still not getting the termfreq() field in my 'doc' object.
I do get the _version_ field in my 'doc' object which I think is
realValue=StoredField.
At which point termfreq() or any other FunctionQuery field becomes the part
of doc object in Solr ? And at that point can I perform some custom logic
and append the response ?
Thanks.
Tony
On Tue, Jul 16, 2013 at 1:34 AM, Patanachai Tangchaisin
patanachai.tangchaisin@**wizecommerce.compatanachai.tangchai...@wizecommerce.com
wrote:
Hi,
I think the process of retrieving a stored field (through fl) is happens
after SearchComponent.
One solution: If you wrap a q params with function your score will be a
result of the function.
For example,
http://localhost:8080/solr/collection2/demoendpoint?q=**http://localhost:8080/solr/**collection2/demoendpoint?q=**
termfreq%28product,%27spider%27%29wt=xmlindent=truefl=***,**score
http://localhost:**8080/solr/collection2/**demoendpoint?q=termfreq%**
28product,%27spider%27%29wt=**xmlindent=truefl=*,scorehttp://localhost:8080/solr/collection2/demoendpoint?q=termfreq%28product,%27spider%27%29wt=xmlindent=truefl=*,score
Now your score is going to be a result of termfreq(product,'spider')
--
Patanachai Tangchaisin
On 07/15/2013 12:01 PM, Tony Mullins wrote:
any help plz !!!
On Mon, Jul 15, 2013 at 4:13 PM, Tony Mullins tonymullins...@gmail.com
*
*wrote:
Please any help on how to get the value of 'freq' field in my custom
SearchComponent ?
http://localhost:8080/solr/collection2/demoendpoint?q=**http://localhost:8080/solr/**collection2/demoendpoint?q=**
spiderwt=xmlindent=truefl=*,freq:termfreq%28product,%**
27spider%27%29http://**localhost:8080/solr/**
collection2/demoendpoint?q=**spiderwt=xmlindent=truefl=***
,freq:termfreq%28product,%**27spider%27%29http://localhost:8080/solr/collection2/demoendpoint?q=spiderwt=xmlindent=truefl=*,freq:termfreq%28product,%27spider%27%29
docstr name=id11/strstr name=typeVideo Games/strstr
name=formatxbox 360/strstr name=productThe Amazing
Spider-Man/strint name=popularity11/intlong
name=_version_1439994081345273856/longint
name=freq1/int/doc
Here is my code
DocList docs = rb.getResults().docList;
DocIterator iterator = docs.iterator();
int sumFreq = 0;
String id = null;
for (int i = 0; i docs.size(); i++) {
try {
int docId = iterator.nextDoc();
// Document doc = searcher.doc(docId, fieldSet);
Document doc = searcher.doc(docId);
In doc object I can see the schema fields like 'id', 'type','format'
etc.
but I cannot find the field 'freq' which I needed. Is there any way to
get
the FunctionQuery fields in doc object ?
Thanks,
Tony
On Mon, Jul 15, 2013 at 1:16 PM, Tony Mullins tonymullins...@gmail.com
**wrote:
Hi,
I have extended Solr's SearchComonent class and I am iterating through
all the docs in ResponseBuilder in @overrider Process() method.
Here I want to get the value of FucntionQuery result but in Document
object I am only seeing the standard field of document not the
FucntionQuery result.
This is my query
http://localhost:8080/solr/collection2/demoendpoint?q=**http://localhost:8080/solr/**collection2/demoendpoint?q=**
spiderwt=xmlindent=truefl=*,freq:termfreq%28product,%**
27spider%27%29http://**localhost:8080/solr/**
collection2/demoendpoint?q=**spiderwt=xmlindent=truefl=***
,freq:termfreq%28product,%**27spider%27%29http://localhost:8080/solr/collection2/demoendpoint?q=spiderwt=xmlindent=truefl=*,freq:termfreq%28product,%27spider%27%29
Result of above query in browser shows me that 'freq' is part of doc
but its not there in Document object
Re: Custom processing in Solr Request Handler plugin and its debugging ?
Ok Thanks Erick, for your help. Tony. On Sun, Jul 14, 2013 at 5:12 PM, Erick Erickson erickerick...@gmail.comwrote: Not sure how to do the pass to another request handler thing, but the debugging part is pretty straightforward. I use IntelliJ, but as far as I know Eclipse has very similar capabilities. First, I cheat and path to the jar that's the output from my IDE, that saves copying the jar around. So my solrconfig.xml file has a lib directive like ../../../../../eoe/project/out/artifact/jardir where this is wherever your IDE wants to put it. It can sometimes be tricky to get enough ../../../ in there. Second, edit config, select remote and a form comes up. Fill in host and port, something like localhost and 5900 (this latter is whatever you want. In IntelliJ that'll give you the specific command to use to start Solr so you can attach. This looks like the following for my setup: java -Xdebug -Xrunjdwp:transport=dt_socket,server=y,suspend=y,address=5900 -jar start.jar Now just fire up Solr as above. Fire up your remote debugging session in IntelliJ. Set breakpoints as you wish. NOT: the suspend=y bit above means that Solr will do _nothing_ until you attach the debugger and hit go HTH Erick On Sat, Jul 13, 2013 at 6:57 AM, Tony Mullins tonymullins...@gmail.com wrote: Please any help on how to pass the search request to different RequestHandler from within the custom RequestHandler and how to debug the custom RequestHandler plugin ? Thanks, Tony On Fri, Jul 12, 2013 at 4:41 PM, Tony Mullins tonymullins...@gmail.com wrote: Hi, I have defined my new Solr RequestHandler plugin like this in SolrConfig.xml requestHandler name=/myendpoint class=com.abc.MyRequestPlugin /requestHandler And its working fine. Now I want to do some custom processing from my this plugin by making a search query to regular '/select' handler. requestHandler name=/select class=solr.SearchHandler /requestHandler And then receive the results back from '/select' handler and perform some custom processing on those results and send the response back to my custom /myendpoint handler. And for this I need help on how to make a call to '/select' handler from within the .MyRequestPlugin class and perform some calculation on the results. I also need some help on how to debug my plugin ? As its .jar is been deployed to solr_hom/lib ... how can I attach my plugin's code in eclipse to Solr process so I could debug it when user will send request to my plugin. Thanks, Tony
Doc's FunctionQuery result field in my custom SearchComponent class ?
Hi, I have extended Solr's SearchComonent class and I am iterating through all the docs in ResponseBuilder in @overrider Process() method. Here I want to get the value of FucntionQuery result but in Document object I am only seeing the standard field of document not the FucntionQuery result. This is my query http://localhost:8080/solr/collection2/demoendpoint?q=spiderwt=xmlindent=truefl=*,freq:termfreq%28product,%27spider%27%29 Result of above query in browser shows me that 'freq' is part of doc but its not there in Document object in my @overrider Process() method. How can I get the value of FunctionQuery result in my custom SearchComponent ? Thanks, Tony
Re: Doc's FunctionQuery result field in my custom SearchComponent class ?
Please any help on how to get the value of 'freq' field in my custom
SearchComponent ?
http://localhost:8080/solr/collection2/demoendpoint?q=spiderwt=xmlindent=truefl=*,freq:termfreq%28product,%27spider%27%29
docstr name=id11/strstr name=typeVideo Games/strstr
name=formatxbox 360/strstr name=productThe Amazing
Spider-Man/strint name=popularity11/intlong
name=_version_1439994081345273856/longint name=freq1/int/doc
Here is my code
DocList docs = rb.getResults().docList;
DocIterator iterator = docs.iterator();
int sumFreq = 0;
String id = null;
for (int i = 0; i docs.size(); i++) {
try {
int docId = iterator.nextDoc();
// Document doc = searcher.doc(docId, fieldSet);
Document doc = searcher.doc(docId);
In doc object I can see the schema fields like 'id', 'type','format' etc.
but I cannot find the field 'freq' which I needed. Is there any way to get
the FunctionQuery fields in doc object ?
Thanks,
Tony
On Mon, Jul 15, 2013 at 1:16 PM, Tony Mullins tonymullins...@gmail.comwrote:
Hi,
I have extended Solr's SearchComonent class and I am iterating through all
the docs in ResponseBuilder in @overrider Process() method.
Here I want to get the value of FucntionQuery result but in Document
object I am only seeing the standard field of document not the
FucntionQuery result.
This is my query
http://localhost:8080/solr/collection2/demoendpoint?q=spiderwt=xmlindent=truefl=*,freq:termfreq%28product,%27spider%27%29
Result of above query in browser shows me that 'freq' is part of doc but
its not there in Document object in my @overrider Process() method.
How can I get the value of FunctionQuery result in my custom
SearchComponent ?
Thanks,
Tony
Re: Doc's FunctionQuery result field in my custom SearchComponent class ?
any help plz !!!
On Mon, Jul 15, 2013 at 4:13 PM, Tony Mullins tonymullins...@gmail.comwrote:
Please any help on how to get the value of 'freq' field in my custom
SearchComponent ?
http://localhost:8080/solr/collection2/demoendpoint?q=spiderwt=xmlindent=truefl=*,freq:termfreq%28product,%27spider%27%29
docstr name=id11/strstr name=typeVideo Games/strstr
name=formatxbox 360/strstr name=productThe Amazing
Spider-Man/strint name=popularity11/intlong
name=_version_1439994081345273856/longint name=freq1/int/doc
Here is my code
DocList docs = rb.getResults().docList;
DocIterator iterator = docs.iterator();
int sumFreq = 0;
String id = null;
for (int i = 0; i docs.size(); i++) {
try {
int docId = iterator.nextDoc();
// Document doc = searcher.doc(docId, fieldSet);
Document doc = searcher.doc(docId);
In doc object I can see the schema fields like 'id', 'type','format' etc.
but I cannot find the field 'freq' which I needed. Is there any way to get
the FunctionQuery fields in doc object ?
Thanks,
Tony
On Mon, Jul 15, 2013 at 1:16 PM, Tony Mullins tonymullins...@gmail.comwrote:
Hi,
I have extended Solr's SearchComonent class and I am iterating through
all the docs in ResponseBuilder in @overrider Process() method.
Here I want to get the value of FucntionQuery result but in Document
object I am only seeing the standard field of document not the
FucntionQuery result.
This is my query
http://localhost:8080/solr/collection2/demoendpoint?q=spiderwt=xmlindent=truefl=*,freq:termfreq%28product,%27spider%27%29
Result of above query in browser shows me that 'freq' is part of doc
but its not there in Document object in my @overrider Process() method.
How can I get the value of FunctionQuery result in my custom
SearchComponent ?
Thanks,
Tony
Re: Doc's FunctionQuery result field in my custom SearchComponent class ?
No sorry, I am still not getting the termfreq() field in my 'doc' object.
I do get the _version_ field in my 'doc' object which I think is
realValue=StoredField.
At which point termfreq() or any other FunctionQuery field becomes the part
of doc object in Solr ? And at that point can I perform some custom logic
and append the response ?
Thanks.
Tony
On Tue, Jul 16, 2013 at 1:34 AM, Patanachai Tangchaisin
patanachai.tangchai...@wizecommerce.com wrote:
Hi,
I think the process of retrieving a stored field (through fl) is happens
after SearchComponent.
One solution: If you wrap a q params with function your score will be a
result of the function.
For example,
http://localhost:8080/solr/**collection2/demoendpoint?q=**
termfreq%28product,%27spider%**27%29wt=xmlindent=truefl=*,**scorehttp://localhost:8080/solr/collection2/demoendpoint?q=termfreq%28product,%27spider%27%29wt=xmlindent=truefl=*,score
Now your score is going to be a result of termfreq(product,'spider')
--
Patanachai Tangchaisin
On 07/15/2013 12:01 PM, Tony Mullins wrote:
any help plz !!!
On Mon, Jul 15, 2013 at 4:13 PM, Tony Mullins tonymullins...@gmail.com*
*wrote:
Please any help on how to get the value of 'freq' field in my custom
SearchComponent ?
http://localhost:8080/solr/**collection2/demoendpoint?q=**
spiderwt=xmlindent=truefl=***,freq:termfreq%28product,%**
27spider%27%29http://localhost:8080/solr/collection2/demoendpoint?q=spiderwt=xmlindent=truefl=*,freq:termfreq%28product,%27spider%27%29
docstr name=id11/strstr name=typeVideo Games/strstr
name=formatxbox 360/strstr name=productThe Amazing
Spider-Man/strint name=popularity11/int**long
name=_version_**1439994081345273856/longint
name=freq1/int/doc
Here is my code
DocList docs = rb.getResults().docList;
DocIterator iterator = docs.iterator();
int sumFreq = 0;
String id = null;
for (int i = 0; i docs.size(); i++) {
try {
int docId = iterator.nextDoc();
// Document doc = searcher.doc(docId, fieldSet);
Document doc = searcher.doc(docId);
In doc object I can see the schema fields like 'id', 'type','format' etc.
but I cannot find the field 'freq' which I needed. Is there any way to
get
the FunctionQuery fields in doc object ?
Thanks,
Tony
On Mon, Jul 15, 2013 at 1:16 PM, Tony Mullins tonymullins...@gmail.com
**wrote:
Hi,
I have extended Solr's SearchComonent class and I am iterating through
all the docs in ResponseBuilder in @overrider Process() method.
Here I want to get the value of FucntionQuery result but in Document
object I am only seeing the standard field of document not the
FucntionQuery result.
This is my query
http://localhost:8080/solr/**collection2/demoendpoint?q=**
spiderwt=xmlindent=truefl=***,freq:termfreq%28product,%**
27spider%27%29http://localhost:8080/solr/collection2/demoendpoint?q=spiderwt=xmlindent=truefl=*,freq:termfreq%28product,%27spider%27%29
Result of above query in browser shows me that 'freq' is part of doc
but its not there in Document object in my @overrider Process() method.
How can I get the value of FunctionQuery result in my custom
SearchComponent ?
Thanks,
Tony
CONFIDENTIALITY NOTICE
==
This email message and any attachments are for the exclusive use of the
intended recipient(s) and may contain confidential and privileged
information. Any unauthorized review, use, disclosure or distribution is
prohibited. If you are not the intended recipient, please contact the
sender by reply email and destroy all copies of the original message along
with any attachments, from your computer system. If you are the intended
recipient, please be advised that the content of this message is subject to
access, review and disclosure by the sender's Email System Administrator.
Re: Custom processing in Solr Request Handler plugin and its debugging ?
Please any help on how to pass the search request to different RequestHandler from within the custom RequestHandler and how to debug the custom RequestHandler plugin ? Thanks, Tony On Fri, Jul 12, 2013 at 4:41 PM, Tony Mullins tonymullins...@gmail.comwrote: Hi, I have defined my new Solr RequestHandler plugin like this in SolrConfig.xml requestHandler name=/myendpoint class=com.abc.MyRequestPlugin /requestHandler And its working fine. Now I want to do some custom processing from my this plugin by making a search query to regular '/select' handler. requestHandler name=/select class=solr.SearchHandler /requestHandler And then receive the results back from '/select' handler and perform some custom processing on those results and send the response back to my custom /myendpoint handler. And for this I need help on how to make a call to '/select' handler from within the .MyRequestPlugin class and perform some calculation on the results. I also need some help on how to debug my plugin ? As its .jar is been deployed to solr_hom/lib ... how can I attach my plugin's code in eclipse to Solr process so I could debug it when user will send request to my plugin. Thanks, Tony
Custom processing in Solr Request Handler plugin and its debugging ?
Hi, I have defined my new Solr RequestHandler plugin like this in SolrConfig.xml requestHandler name=/myendpoint class=com.abc.MyRequestPlugin /requestHandler And its working fine. Now I want to do some custom processing from my this plugin by making a search query to regular '/select' handler. requestHandler name=/select class=solr.SearchHandler /requestHandler And then receive the results back from '/select' handler and perform some custom processing on those results and send the response back to my custom /myendpoint handler. And for this I need help on how to make a call to '/select' handler from within the .MyRequestPlugin class and perform some calculation on the results. I also need some help on how to debug my plugin ? As its .jar is been deployed to solr_hom/lib ... how can I attach my plugin's code in eclipse to Solr process so I could debug it when user will send request to my plugin. Thanks, Tony
Re: Calculating Solr document score by ignoring the boost field.
Ok thanks, I just wanted the know is it possible to ignore boost value or not during score calculation and as you said its not. Now I would have to focus on nutch to fix the issue and not to send boost=0 to Solr. Regards, Khan On Wed, Jul 10, 2013 at 12:14 PM, Daniel Collins danwcoll...@gmail.comwrote: Sorry to repeat Jacks' previous answer but x times zero is always zero :) A index boost is just what the name suggests, a factor by which the document score is boosted (multiplied). Since it is in an index time value, it is stored alongside the document, so any future scoring of the document by any query will take this value into account. If you take Solr's internal document score and then multiply it by zero, the result is by definition zero... What you seem to be saying is you are passing in an index time boost (which is incorrect but that's an issue with Nutch), but you want Solr to ignore it, surely the correct approach then is *not* to pass it in? Once the data is indexed, it is fixed, unless you re-index the document, so if that data is wrong, there is nothing Solr can do about it, you have to re-index the documents that have incorrect data. If you want to just use TF-IDF for scoring and not use boosting, don't supply any boosting, it's that simple. Sorry if this sounds repetitive, but can't think of any other way to say it. On 10 July 2013 06:33, Tony Mullins tonymullins...@gmail.com wrote: Jack due to 'some' reason my nutch is returning me index time boost =0.0 and just for a moment suppose that nutch is and will always return boost =0. Now my simple question was why Solr is showing me document's score = 0 ? Why is it depending upon index time boost value ? Why or how to make Solr to only calculate the score value on TF-IDF ? Regards, Khan On Tue, Jul 9, 2013 at 6:31 PM, Jack Krupansky j...@basetechnology.com wrote: Simple math: x times zero equals zero. That's why the default document boost is 1.0 - score times 1.0 equals score. Any particular reason you wanted to zero out the document score from the document level? -- Jack Krupansky -Original Message- From: Tony Mullins Sent: Tuesday, July 09, 2013 9:23 AM To: solr-user@lucene.apache.org Subject: Re: Calculating Solr document score by ignoring the field. I am passing boost value (via nutch) and i.e boost =0.0. But my question is why Solr is showing me score = 0.0 when my boost (index time boost) = 0.0 ? Should not Solr calculate its documents score on the basis of TF-IDF ? And if not how can I make Solr to only consider TF-IDF while calculating document's score ? Regards, Khan On Tue, Jul 9, 2013 at 4:46 PM, Erick Erickson erickerick...@gmail.com ** wrote: My guess is that you're not really passing on the boost field's value and getting the default. Don't quite know how I'd track that down though Best Erick On Tue, Jul 9, 2013 at 4:09 AM, imran khan imrankhan.x...@gmail.com wrote: Greetings, I am using nutch 2.x as my datasource for Solr 4.3.0. And nutch passes on its own boost field to my Solr schema field name=boost type=float stored=true indexed=false/ Now due to some reason I always get boost = 0.0 and due to this my Solr's document score is also always 0.0. Is there any way in Solr that it ignores the boost field's value for its document's score calculation ? Regards, Khan
Re: Calculating Solr document score by ignoring the boost field.
I am passing boost value (via nutch) and i.e boost =0.0. But my question is why Solr is showing me score = 0.0 when my boost (index time boost) = 0.0 ? Should not Solr calculate its documents score on the basis of TF-IDF ? And if not how can I make Solr to only consider TF-IDF while calculating document's score ? Regards, Khan On Tue, Jul 9, 2013 at 4:46 PM, Erick Erickson erickerick...@gmail.comwrote: My guess is that you're not really passing on the boost field's value and getting the default. Don't quite know how I'd track that down though Best Erick On Tue, Jul 9, 2013 at 4:09 AM, imran khan imrankhan.x...@gmail.com wrote: Greetings, I am using nutch 2.x as my datasource for Solr 4.3.0. And nutch passes on its own boost field to my Solr schema field name=boost type=float stored=true indexed=false/ Now due to some reason I always get boost = 0.0 and due to this my Solr's document score is also always 0.0. Is there any way in Solr that it ignores the boost field's value for its document's score calculation ? Regards, Khan
Re: Calculating Solr document score by ignoring the boost field.
Jack due to 'some' reason my nutch is returning me index time boost =0.0 and just for a moment suppose that nutch is and will always return boost =0. Now my simple question was why Solr is showing me document's score = 0 ? Why is it depending upon index time boost value ? Why or how to make Solr to only calculate the score value on TF-IDF ? Regards, Khan On Tue, Jul 9, 2013 at 6:31 PM, Jack Krupansky j...@basetechnology.comwrote: Simple math: x times zero equals zero. That's why the default document boost is 1.0 - score times 1.0 equals score. Any particular reason you wanted to zero out the document score from the document level? -- Jack Krupansky -Original Message- From: Tony Mullins Sent: Tuesday, July 09, 2013 9:23 AM To: solr-user@lucene.apache.org Subject: Re: Calculating Solr document score by ignoring the field. I am passing boost value (via nutch) and i.e boost =0.0. But my question is why Solr is showing me score = 0.0 when my boost (index time boost) = 0.0 ? Should not Solr calculate its documents score on the basis of TF-IDF ? And if not how can I make Solr to only consider TF-IDF while calculating document's score ? Regards, Khan On Tue, Jul 9, 2013 at 4:46 PM, Erick Erickson erickerick...@gmail.com** wrote: My guess is that you're not really passing on the boost field's value and getting the default. Don't quite know how I'd track that down though Best Erick On Tue, Jul 9, 2013 at 4:09 AM, imran khan imrankhan.x...@gmail.com wrote: Greetings, I am using nutch 2.x as my datasource for Solr 4.3.0. And nutch passes on its own boost field to my Solr schema field name=boost type=float stored=true indexed=false/ Now due to some reason I always get boost = 0.0 and due to this my Solr's document score is also always 0.0. Is there any way in Solr that it ignores the boost field's value for its document's score calculation ? Regards, Khan
Total Term Frequency per ResultSet in Solr 4.3 ?
Hi , I have lots of crawled data, indexed in my Solr (4.3.0) and lets say user creates a search criteria 'X1' and he/she wants to know the occurrence of a specific term in the result set of that 'X1' search criteria. And then again he/she creates another search criteria 'X2' and he/she wants to know the occurrence of that same term in the result set of that 'X2' search criteria. At the moment if I give termfreq(field,term) then it gives me the term frequency per document and if I use totaltermfreq(field,term), it gives me the total term frequency in entire index not in the result set of my search criteria. So what I need is your help to find how to how to get total occurrence of a term in query's result set. If this is my result set doc str name=typeMovies/str str name=formatdvd/str str name=productThe Hunger Games/str/doc doc str name=typeBooks/str str name=formatpaperback/str str name=productThe Hunger Book/str/doc And I am looking for term 'hunger' in product field then I want to get value = '2' , and if I am searching for term 'games' in product field I want to get value = '1' . Thanks, Tony
Re: Total Term Frequency per ResultSet in Solr 4.3 ?
Hi Yonik, With facet it didn't work. Please see the result set doc below http://localhost:8080/solr/collection2/select?fl=*,amazing_freq:termfreq%28product,%27amazing%27%29,spider_freq:termfreq%28product,%27spider%27%29fq=id%3A27q=spiderfl=*df=productwt=xmlindent=truefacet=truefacet.query=product:spiderfacet.query=product:amazingrows=20 doc str name=id27/str str name=typeMovies/str str name=formatdvd/str str name=productThe amazing spider man is amazing spider the spider/str int name=popularity1/int long name=_version_1439641369145507840/long int name=amazing_freq2/int int name=spider_freq3/int /doc /resultlst name=facet_countslst name=facet_queries int name=product:spider1/int int name=product:amazing1/int /lst As you can see facet is actually just returning the no. of docs found against those keywrods not the actual frequency. Actual frequency is returned by the field 'amazing_freq' 'spider_freq' ! So is there any workaround for this to get the total of term-frequency in resultset without any modification to Solr source code ? Thanks, Tony On Thu, Jul 4, 2013 at 7:05 PM, Yonik Seeley yo...@lucidworks.com wrote: If you just want to retrieve those counts, this seems like simple faceting. q=something facet=true facet.query=product:hunger facet.query=product:games -Yonik http://lucidworks.com On Thu, Jul 4, 2013 at 9:45 AM, Tony Mullins tonymullins...@gmail.com wrote: Hi , I have lots of crawled data, indexed in my Solr (4.3.0) and lets say user creates a search criteria 'X1' and he/she wants to know the occurrence of a specific term in the result set of that 'X1' search criteria. And then again he/she creates another search criteria 'X2' and he/she wants to know the occurrence of that same term in the result set of that 'X2' search criteria. At the moment if I give termfreq(field,term) then it gives me the term frequency per document and if I use totaltermfreq(field,term), it gives me the total term frequency in entire index not in the result set of my search criteria. So what I need is your help to find how to how to get total occurrence of a term in query's result set. If this is my result set doc str name=typeMovies/str str name=formatdvd/str str name=productThe Hunger Games/str/doc doc str name=typeBooks/str str name=formatpaperback/str str name=productThe Hunger Book/str/doc And I am looking for term 'hunger' in product field then I want to get value = '2' , and if I am searching for term 'games' in product field I want to get value = '1' . Thanks, Tony
Re: Total Term Frequency per ResultSet in Solr 4.3 ?
So what is the workaround for this problem ? Can it be done without changing any source code ? Thanks, Tony On Thu, Jul 4, 2013 at 8:01 PM, Yonik Seeley yo...@lucidworks.com wrote: Ah, sorry - I thought you were after docfreq, not termfreq. -Yonik http://lucidworks.com On Thu, Jul 4, 2013 at 10:57 AM, Tony Mullins tonymullins...@gmail.com wrote: Hi Yonik, With facet it didn't work. Please see the result set doc below http://localhost:8080/solr/collection2/select?fl=*,amazing_freq:termfreq%28product,%27amazing%27%29,spider_freq:termfreq%28product,%27spider%27%29fq=id%3A27q=spiderfl=*df=productwt=xmlindent=truefacet=truefacet.query=product:spiderfacet.query=product:amazingrows=20 doc str name=id27/str str name=typeMovies/str str name=formatdvd/str str name=productThe amazing spider man is amazing spider the spider/str int name=popularity1/int long name=_version_1439641369145507840/long int name=amazing_freq2/int int name=spider_freq3/int /doc /resultlst name=facet_countslst name=facet_queries int name=product:spider1/int int name=product:amazing1/int /lst As you can see facet is actually just returning the no. of docs found against those keywrods not the actual frequency. Actual frequency is returned by the field 'amazing_freq' 'spider_freq' ! So is there any workaround for this to get the total of term-frequency in resultset without any modification to Solr source code ? Thanks, Tony On Thu, Jul 4, 2013 at 7:05 PM, Yonik Seeley yo...@lucidworks.com wrote: If you just want to retrieve those counts, this seems like simple faceting. q=something facet=true facet.query=product:hunger facet.query=product:games -Yonik http://lucidworks.com On Thu, Jul 4, 2013 at 9:45 AM, Tony Mullins tonymullins...@gmail.com wrote: Hi , I have lots of crawled data, indexed in my Solr (4.3.0) and lets say user creates a search criteria 'X1' and he/she wants to know the occurrence of a specific term in the result set of that 'X1' search criteria. And then again he/she creates another search criteria 'X2' and he/she wants to know the occurrence of that same term in the result set of that 'X2' search criteria. At the moment if I give termfreq(field,term) then it gives me the term frequency per document and if I use totaltermfreq(field,term), it gives me the total term frequency in entire index not in the result set of my search criteria. So what I need is your help to find how to how to get total occurrence of a term in query's result set. If this is my result set doc str name=typeMovies/str str name=formatdvd/str str name=productThe Hunger Games/str/doc doc str name=typeBooks/str str name=formatpaperback/str str name=productThe Hunger Book/str/doc And I am looking for term 'hunger' in product field then I want to get value = '2' , and if I am searching for term 'games' in product field I want to get value = '1' . Thanks, Tony
Re: Total Term Frequency per ResultSet in Solr 4.3 ?
OK. Thanks Tricia , Jack Yonik for your suggestions and time. Regards, Tony. On Fri, Jul 5, 2013 at 1:20 AM, P Williams williams.tricia.l...@gmail.comwrote: Hi Tony, Have you seen the TermVectorComponenthttp://wiki.apache.org/solr/TermVectorComponent? It will return the TermVectors for the documents in your result set (note that the rows parameter matters if you want results for the whole set, the default is 10). TermVectors also must be stored for each field that you want term frequency returned for. Suppose you have the query http://localhost:8983/solr/collection1/tvrh?q=cablefl=includestv.tf=trueon the example that comes packaged with Solr. Then part of the response is: lst name=termVectors str name=uniqueKeyFieldNameid/str lst name=IW-02 str name=uniqueKeyIW-02/str /lst lst name=9885A004 str name=uniqueKey9885A004/str lst name=includes lst name=32mb int name=tf1/int /lst lst name=av int name=tf1/int /lst lst name=battery int name=tf1/int /lst lst name=cable int name=tf2/int /lst lst name=card int name=tf1/int /lst lst name=sd int name=tf1/int /lst lst name=usb int name=tf1/int /lst /lst /lst lst name=3007WFP str name=uniqueKey3007WFP/str lst name=includes lst name=cable int name=tf1/int /lst lst name=usb int name=tf1/int /lst /lst /lst lst name=MA147LL/A str name=uniqueKeyMA147LL/A/str lst name=includes lst name=cable int name=tf1/int /lst lst name=earbud int name=tf1/int /lst lst name=headphones int name=tf1/int /lst lst name=usb int name=tf1/int /lst /lst /lst /lst Then you can use an XPath query like sum(//lst[@name='cable']/int[@name='tf']) where 'cable' was the term, to calculate the term frequency in the 'includes' field for the whole result set. You could extend this to get the term frequency across all fields for your result set with some alterations to the query and schema.xml configuration. Alternately you could get the response as json (wt=json) and use javascript to sum. I know this is not terribly efficient but, if I'm understanding your request correctly, it's possible. Cheers, Tricia On Thu, Jul 4, 2013 at 10:24 AM, Tony Mullins tonymullins...@gmail.com wrote: So what is the workaround for this problem ? Can it be done without changing any source code ? Thanks, Tony On Thu, Jul 4, 2013 at 8:01 PM, Yonik Seeley yo...@lucidworks.com wrote: Ah, sorry - I thought you were after docfreq, not termfreq. -Yonik http://lucidworks.com On Thu, Jul 4, 2013 at 10:57 AM, Tony Mullins tonymullins...@gmail.com wrote: Hi Yonik, With facet it didn't work. Please see the result set doc below http://localhost:8080/solr/collection2/select?fl=*,amazing_freq:termfreq%28product,%27amazing%27%29,spider_freq:termfreq%28product,%27spider%27%29fq=id%3A27q=spiderfl=*df=productwt=xmlindent=truefacet=truefacet.query=product:spiderfacet.query=product:amazingrows=20 doc str name=id27/str str name=typeMovies/str str name=formatdvd/str str name=productThe amazing spider man is amazing spider the spider/str int name=popularity1/int long name=_version_1439641369145507840/long int name=amazing_freq2/int int name=spider_freq3/int /doc /resultlst name=facet_countslst name=facet_queries int name=product:spider1/int int name=product:amazing1/int /lst As you can see facet is actually just returning the no. of docs found against those keywrods not the actual frequency. Actual frequency is returned by the field 'amazing_freq' 'spider_freq' ! So is there any workaround for this to get the total of term-frequency in resultset without any modification to Solr source code ? Thanks, Tony On Thu, Jul 4, 2013 at 7:05 PM, Yonik Seeley yo...@lucidworks.com wrote: If you just want to retrieve those counts, this seems like simple faceting. q=something facet=true facet.query=product:hunger facet.query=product:games -Yonik http://lucidworks.com On Thu, Jul 4, 2013 at 9:45 AM, Tony Mullins tonymullins...@gmail.com wrote: Hi , I have lots of crawled data, indexed in my Solr (4.3.0) and lets say user creates a search criteria 'X1' and he/she wants to know the occurrence of a specific term in the result set of that 'X1' search criteria. And then again he/she creates another search criteria 'X2' and he/she wants to know the occurrence of that same term in the result set of that 'X2' search criteria. At the moment if I give termfreq(field,term) then it gives me the term frequency per document and if I use totaltermfreq(field,term), it gives me the total term frequency in entire index not in the result set of my search criteria. So what I need is your help
Re: Aggregate TermFrequency on Result Grouping / Field Collapsing
Any suggestions please ! On Tue, Jul 2, 2013 at 3:24 PM, Tony Mullins tonymullins...@gmail.comwrote: Hi, Is it possible to perform aggregated termfreq(field,term) on Result Grouping ? I am trying to get total count of term's appearance in a document and then want to aggregate that count by grouping the document on one of my field. Like this http://localhost:8080/solr/collection1/select?q=iphonewt=jsonindent=truegroup=truegroup.field=urlfl=freq%3Atermfreq%28CommentX%2C%27iphone%27%29 Problem is it returning only top level result (doc) in each group and thus the term frequency of that result (doc). How can I make it to sum the termfred() of all the documents per group ? Thanks, Tony
Re: How to re-index Solr get term frequency within documents
Hi Otis, I am quite new to Solr. And have looked at this link http://search-lucene.com/jd/solr/solr-dataimporthandler/org/apache/solr/handler/dataimport/SolrEntityProcessor.html; but could not figure out how to use it to re-index my all data in solr. Could you please explain in little detail that how to use it ? Can it be done using the Solr Admin Panel (http://localhost:8080/solr) ? Thanks, Tony On Tue, Jul 2, 2013 at 9:24 PM, Otis Gospodnetic otis.gospodne...@gmail.com wrote: Hi Tony, There is, you can do it with that SolrEntityProcessor I pointed out, if you have all your fields stored in Solr. Otis -- Solr ElasticSearch Support -- http://sematext.com/ Performance Monitoring -- http://sematext.com/spm
Re: Aggregate TermFrequency on Result Grouping / Field Collapsing
Ok let me explain the problem. I have lots of crawled data, indexed in my Solr (4.3.0) and lets say user creates a search criteria 'X1' and he/she wants to know the occurrence of a specific term in the result set of that 'X1' search criteria. And then again he/she creates another search criteria 'X2' and he/she wants to know the occurrence of that same term in the result set of that 'X2' search criteria. At the moment if I give termfreq(field,term) then it gives me the term frequency per document and if I use totaltermfreq(field,term), it gives me the total term frequency in entire index not in the result set of my search criteria. So what I need is your help to find how to how to get total occurrence of a term in query's result set. If this is my result set doc str name=typeMovies/str str name=formatdvd/str str name=productThe Hunger Games/str/doc doc str name=typeBooks/str str name=formatpaperback/str str name=productThe Hunger Book/str/doc And I am looking for term 'hunger' in product field then I want to get value = '2' , and if I am searching for term 'games' in product field I want to get value = '1' . Thanks, Tony ** On Wed, Jul 3, 2013 at 4:41 PM, Erick Erickson erickerick...@gmail.comwrote: I don't think you can, thus the silence. But why do you want to do this thing? Smells like an XY problem, you've asked how to do a specific thing without stating the problem. Perhaps there's a better approach that _is_ do-able. Best Erick On Wed, Jul 3, 2013 at 2:14 AM, Tony Mullins tonymullins...@gmail.com wrote: Any suggestions please ! On Tue, Jul 2, 2013 at 3:24 PM, Tony Mullins tonymullins...@gmail.com wrote: Hi, Is it possible to perform aggregated termfreq(field,term) on Result Grouping ? I am trying to get total count of term's appearance in a document and then want to aggregate that count by grouping the document on one of my field. Like this http://localhost:8080/solr/collection1/select?q=iphonewt=jsonindent=truegroup=truegroup.field=urlfl=freq%3Atermfreq%28CommentX%2C%27iphone%27%29 Problem is it returning only top level result (doc) in each group and thus the term frequency of that result (doc). How can I make it to sum the termfred() of all the documents per group ? Thanks, Tony
Re: How to re-index Solr get term frequency within documents
I use Nutch as input datasource for my Solr. So I cannot re-run all the Nutch jobs to generate data again for Solr as it will take very long to generate that much data. I was hoping there would be an easier way inside Solr to just re-index all the existing data. Thanks, Tony On Tue, Jul 2, 2013 at 1:37 AM, Jack Krupansky j...@basetechnology.comwrote: Or, go with a commercial product that has a single-click Solr re-index capability, such as: 1. DataStax Enterprise - data is stored in Cassandra and reindexed into Solr from there. 2. LucidWorks Search - data sources are declared so that the package can automatically re-crawl the data sources. But, yeah, as Otis says, re-index is really just a euphemism for deleting your Solr data directory and indexing from scratch from the original data sources. -- Jack Krupansky -Original Message- From: Otis Gospodnetic Sent: Monday, July 01, 2013 2:26 PM To: solr-user@lucene.apache.org Subject: Re: How to re-index Solr get term frequency within documents If all your fields are stored, you can do it with http://search-lucene.com/?q=**solrentityprocessorhttp://search-lucene.com/?q=solrentityprocessor Otherwise, just reindex the same way you indexed in the first place. *Always* be ready to reindex from scratch. Otis -- Solr ElasticSearch Support -- http://sematext.com/ Performance Monitoring -- http://sematext.com/spm On Mon, Jul 1, 2013 at 1:29 PM, Tony Mullins tonymullins...@gmail.com wrote: Thanks Jack , it worked. Could you please provide some info on how to re-index existing data in Solr, after changing the schema.xml ? Thanks, Tony On Mon, Jul 1, 2013 at 8:21 PM, Jack Krupansky j...@basetechnology.com* *wrote: You can write any function query in the field list of the fl parameter. Sounds like you want termfreq: termfreq(field_arg,term) fl=id,a,b,c,termfreq(a,xyz) -- Jack Krupansky -Original Message- From: Tony Mullins Sent: Monday, July 01, 2013 10:47 AM To: solr-user@lucene.apache.org Subject: How to re-index Solr get term frequency within documents Hi, I am using Solr 4.3.0. If I change my solr's schema.xml then do I need to re-index my solr ? And if yes , how to ? My 2nd question is I need to find the frequency of term per document in all documents of search result. My field is field name=CommentX type=text_general stored=true indexed=true multiValued=true termVectors=true termPositions=true termOffsets=true/ And I am trying this query http://localhost:8080/solr/select/?q=iphonefl=AuthorX%** 2CTitleX%2CCommentXdf=CommentXwt=xmlindent=true** qt=tvrhtv=truetv.tf=truetv.df=truetv.positionstv. offsets=truehttp://localhost:**8080/solr/select/?q=iphonefl=** AuthorX%2CTitleX%2CCommentX**df=CommentXwt=xmlindent=** trueqt=tvrhtv=truetv.tf=**truetv.df=truetv.positions** tv.offsets=truehttp://localhost:8080/solr/select/?q=iphonefl=AuthorX%2CTitleX%2CCommentXdf=CommentXwt=xmlindent=trueqt=tvrhtv=truetv.tf=truetv.df=truetv.positionstv.offsets=true Its just returning me the result set, no info on my searched term's (iphone) frequency in each document. How can I make Solr to return the frequency of searched term per document in result set ? Thanks, Tony.
Aggregate TermFrequency on Result Grouping / Field Collapsing
Hi, Is it possible to perform aggregated termfreq(field,term) on Result Grouping ? I am trying to get total count of term's appearance in a document and then want to aggregate that count by grouping the document on one of my field. Like this http://localhost:8080/solr/collection1/select?q=iphonewt=jsonindent=truegroup=truegroup.field=urlfl=freq%3Atermfreq%28CommentX%2C%27iphone%27%29 Problem is it returning only top level result (doc) in each group and thus the term frequency of that result (doc). How can I make it to sum the termfred() of all the documents per group ? Thanks, Tony
How to re-index Solr get term frequency within documents
Hi, I am using Solr 4.3.0. If I change my solr's schema.xml then do I need to re-index my solr ? And if yes , how to ? My 2nd question is I need to find the frequency of term per document in all documents of search result. My field is field name=CommentX type=text_general stored=true indexed=true multiValued=true termVectors=true termPositions=true termOffsets=true/ And I am trying this query http://localhost:8080/solr/select/?q=iphonefl=AuthorX%2CTitleX%2CCommentXdf=CommentXwt=xmlindent=trueqt=tvrhtv=truetv.tf=truetv.df=truetv.positionstv.offsets=true Its just returning me the result set, no info on my searched term's (iphone) frequency in each document. How can I make Solr to return the frequency of searched term per document in result set ? Thanks, Tony.
Re: How to re-index Solr get term frequency within documents
Thanks Jack , it worked. Could you please provide some info on how to re-index existing data in Solr, after changing the schema.xml ? Thanks, Tony On Mon, Jul 1, 2013 at 8:21 PM, Jack Krupansky j...@basetechnology.comwrote: You can write any function query in the field list of the fl parameter. Sounds like you want termfreq: termfreq(field_arg,term) fl=id,a,b,c,termfreq(a,xyz) -- Jack Krupansky -Original Message- From: Tony Mullins Sent: Monday, July 01, 2013 10:47 AM To: solr-user@lucene.apache.org Subject: How to re-index Solr get term frequency within documents Hi, I am using Solr 4.3.0. If I change my solr's schema.xml then do I need to re-index my solr ? And if yes , how to ? My 2nd question is I need to find the frequency of term per document in all documents of search result. My field is field name=CommentX type=text_general stored=true indexed=true multiValued=true termVectors=true termPositions=true termOffsets=true/ And I am trying this query http://localhost:8080/solr/**select/?q=iphonefl=AuthorX%** 2CTitleX%2CCommentXdf=**CommentXwt=xmlindent=true** qt=tvrhtv=truetv.tf=truetv.**df=truetv.positionstv.**offsets=truehttp://localhost:8080/solr/select/?q=iphonefl=AuthorX%2CTitleX%2CCommentXdf=CommentXwt=xmlindent=trueqt=tvrhtv=truetv.tf=truetv.df=truetv.positionstv.offsets=true Its just returning me the result set, no info on my searched term's (iphone) frequency in each document. How can I make Solr to return the frequency of searched term per document in result set ? Thanks, Tony.
Apache Nutch data to Solr 4.3 schema issues ?
Hi ,
I am trying to index my Solr 4.3 from Apache Nutch 2.2 data. And for that I
have copied the schema-solr4.xml from Nutch2.2 runtime/local/conf and
pasted it to my SolrHome solr/collection1/conf.
My Solr4.3 is hosted in Tomcat. And initially when I tried
http://localhost:8080/solr/#/collection1
it wasn't working and on further investigation I found _version_ field was
missing so I added this field as
field name=_version_ type=long indexed=true stored=true/ and it
started working ok.
And now when I try http://localhost:8080/solr/collection1/browse ... it
shows me errors like
HTTP Status 500 - {msg=lazy loading
error,trace=org.apache.solr.common.SolrException: lazy loading error at
org.apache.solr.core.SolrCore$LazyQueryResponseWriterWrapper.getWrappedWriter(SolrCore.java:2260)
at
org.apache.solr.core.SolrCore$LazyQueryResponseWriterWrapper.getContentType(SolrCore.java:2279)
at
org.apache.solr.servlet.SolrDispatchFilter.writeResponse(SolrDispatchFilter.java:623)
at
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:372)
at
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:155)
at
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:243)
at
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210)
at
So could you please guide me that whats missing now ? is this again due to
any schema issue or something else ?
Thanks,
Tony
Re: Apache Nutch data to Solr 4.3 schema issues ?
Hi.
I was hoping by replacing Nutch provided schema to my Solr schema ( as the
described by Nutch documentation) would solve all my problems.
So you are suggesting I edit my existing Solr schema and just add the
additional information found in Nutch-Solr schema line by line .
Thanks,
Tony.
On Thu, Jun 13, 2013 at 5:06 PM, Jack Krupansky j...@basetechnology.comwrote:
Look further down in the stack trace in the Solr log for the final Caused
By:.
And better to start with the Solr 4.3 schema and config files and then
merge in your Nutch changes one line at a time.
-- Jack Krupansky
-Original Message- From: Tony Mullins
Sent: Thursday, June 13, 2013 3:56 AM
To: solr-user@lucene.apache.org
Subject: Apache Nutch data to Solr 4.3 schema issues ?
Hi ,
I am trying to index my Solr 4.3 from Apache Nutch 2.2 data. And for that I
have copied the schema-solr4.xml from Nutch2.2 runtime/local/conf and
pasted it to my SolrHome solr/collection1/conf.
My Solr4.3 is hosted in Tomcat. And initially when I tried
http://localhost:8080/solr/#/**collection1http://localhost:8080/solr/#/collection1
it wasn't working and on further investigation I found _version_ field was
missing so I added this field as
field name=_version_ type=long indexed=true stored=true/ and it
started working ok.
And now when I try
http://localhost:8080/solr/**collection1/browsehttp://localhost:8080/solr/collection1/browse...
it
shows me errors like
HTTP Status 500 - {msg=lazy loading
error,trace=org.apache.solr.**common.SolrException: lazy loading error at
org.apache.solr.core.SolrCore$**LazyQueryResponseWriterWrapper**
.getWrappedWriter(SolrCore.**java:2260)
at
org.apache.solr.core.SolrCore$**LazyQueryResponseWriterWrapper**
.getContentType(SolrCore.java:**2279)
at
org.apache.solr.servlet.**SolrDispatchFilter.**writeResponse(**
SolrDispatchFilter.java:623)
at
org.apache.solr.servlet.**SolrDispatchFilter.doFilter(**
SolrDispatchFilter.java:372)
at
org.apache.solr.servlet.**SolrDispatchFilter.doFilter(**
SolrDispatchFilter.java:155)
at
org.apache.catalina.core.**ApplicationFilterChain.**internalDoFilter(**
ApplicationFilterChain.java:**243)
at
org.apache.catalina.core.**ApplicationFilterChain.**doFilter(**
ApplicationFilterChain.java:**210)
at
So could you please guide me that whats missing now ? is this again due to
any schema issue or something else ?
Thanks,
Tony
Re: Apache Nutch data to Solr 4.3 schema issues ?
OK. Thanks. Tony. On Thu, Jun 13, 2013 at 7:02 PM, Shawn Heisey s...@elyograg.org wrote: Hi. I was hoping by replacing Nutch provided schema to my Solr schema ( as the described by Nutch documentation) would solve all my problems. So you are suggesting I edit my existing Solr schema and just add the additional information found in Nutch-Solr schema line by line I hate to tell you to do such a labor intensive process, but Jack is right. The fact that you had to add the _version_ field means that Nutch 2.2 Had a schema designed for a Solr release prior to 4.0, which was released last October. There have been four Solr releases since then and another should be out in The next few days. Someone on the nutch mailing list might have a schema designed to work with Solr 4.x, and you might also want to look in the source code repository for Nutch. I'm on my phone so it's difficult to interrupt this email in progress to locate resources for you. Thanks, Shawn
