Re: Is it possible to searh Solr with a longer query string?
On 6/25/2013 6:15 PM, Jack Krupansky wrote: Are you using Tomcat? See: http://wiki.apache.org/solr/SolrTomcat#Enabling_Longer_Query_Requests Enabling Longer Query Requests If you try to submit too long a GET query to Solr, then Tomcat will reject your HTTP request on the grounds that the HTTP header is too large; symptoms may include an HTTP 400 Bad Request error or (if you execute the query in a web browser) a blank browser window. If you need to enable longer queries, you can set the maxHttpHeaderSize attribute on the HTTP Connector element in your server.xml file. The default value is 4K. (See http://tomcat.apache.org/tomcat-5.5-doc/config/http.html) Even better would be to force SolrJ to use a POST request. In newer versions (4.1 and later) Solr sets the servlet container's POST buffer size and defaults it to 2MB. In older versions, you'd have to adjust this in your servlet container config, but the default should be considerably larger than the header buffer used for GET requests. I thought that SolrJ used POST by default, but after looking at the code, it seems that I was wrong. Here's how to send a POST query: response = server.query(query, METHOD.POST); The import required for this is: import org.apache.solr.client.solrj.SolrRequest.METHOD; Gary, if you can avoid it, you should not be creating a new HttpSolrServer object every time you make a query. It is completely thread-safe, so create a singleton and use it for all queries against the medline core. Thanks, Shawn
Re: Is it possible to searh Solr with a longer query string?
Oh this is good! On Wed, Jun 26, 2013 at 12:05 PM, Shawn Heisey s...@elyograg.org wrote: On 6/25/2013 6:15 PM, Jack Krupansky wrote: Are you using Tomcat? See: http://wiki.apache.org/solr/SolrTomcat#Enabling_Longer_Query_Requests Enabling Longer Query Requests If you try to submit too long a GET query to Solr, then Tomcat will reject your HTTP request on the grounds that the HTTP header is too large; symptoms may include an HTTP 400 Bad Request error or (if you execute the query in a web browser) a blank browser window. If you need to enable longer queries, you can set the maxHttpHeaderSize attribute on the HTTP Connector element in your server.xml file. The default value is 4K. (See http://tomcat.apache.org/tomcat-5.5-doc/config/http.html) Even better would be to force SolrJ to use a POST request. In newer versions (4.1 and later) Solr sets the servlet container's POST buffer size and defaults it to 2MB. In older versions, you'd have to adjust this in your servlet container config, but the default should be considerably larger than the header buffer used for GET requests. I thought that SolrJ used POST by default, but after looking at the code, it seems that I was wrong. Here's how to send a POST query: response = server.query(query, METHOD.POST); The import required for this is: import org.apache.solr.client.solrj.SolrRequest.METHOD; Gary, if you can avoid it, you should not be creating a new HttpSolrServer object every time you make a query. It is completely thread-safe, so create a singleton and use it for all queries against the medline core. Thanks, Shawn
RE: Is it possible to searh Solr with a longer query string?
Hi,
I'm using Solr server to develop a search service, and I encounter a problem
when trying to input a longer query string:
Here is the code:
StringBuffer stringBuffer = new StringBuffer();
... ...
try{
//search Pubmed server( a NCBI server ), it returns a list of IDs.
EFetchPubmedServiceStub service = new EFetchPubmedServiceStub();
EFetchPubmedServiceStub.EFetchRequest req = new
EFetchPubmedServiceStub.EFetchRequest();
req.setWebEnv( WebEnv );
req.setQuery_key( query_key );
req.setRetstart( 1110 );
// return 295 IDs
req.setRetmax( 295 );
EFetchPubmedServiceStub.EFetchResult res = service.run_eFetch( req
);
//connect returned IDs with OR and query my local Solr server
for( int i = 0; i
res.getPubmedArticleSet().getPubmedArticleSetChoice().length; i++ ){
EFetchPubmedServiceStub.PubmedArticleType art =
res.getPubmedArticleSet().getPubmedArticleSetChoice()
[ i ].getPubmedArticle();
if( i 0 ){
stringBuffer.append( OR );
}
stringBuffer.append( ( pmid: +
art.getMedlineCitation().getPMID().getString() + ) );
}
HttpSolrServer solrServer = new HttpSolrServer(
http://127.0.0.1:8087/solr430/medline; );
String q = stringBuffer.toString();
//when input query has more 300 IDs, query will throw
org.apache.solr.client.solrj.SolrServerException: Server at
http://127.0.0.1:8087/solr430/medline returned non ok status:400, message:Bad
Request
QueryResponse solrRes = solrServer.query( new SolrQuery( q ) );
long found = solrRes.getResults().getNumFound();
System.out.println( found );
}
catch( Exception e ){
e.printStackTrace();
}
... ...
Do you think it's possible to change the query string length limit so that Solr
can accept more IDs?
Thanks.
-Gary
Re: Is it possible to searh Solr with a longer query string?
Are you using Tomcat?
See:
http://wiki.apache.org/solr/SolrTomcat#Enabling_Longer_Query_Requests
Enabling Longer Query Requests
If you try to submit too long a GET query to Solr, then Tomcat will reject
your HTTP request on the grounds that the HTTP header is too large; symptoms
may include an HTTP 400 Bad Request error or (if you execute the query in a
web browser) a blank browser window.
If you need to enable longer queries, you can set the maxHttpHeaderSize
attribute on the HTTP Connector element in your server.xml file. The default
value is 4K. (See http://tomcat.apache.org/tomcat-5.5-doc/config/http.html)
---
If you're not using Tomcat, your container may have a similar limit.
-- Jack Krupansky
-Original Message-
From: yang, gang
Sent: Tuesday, June 25, 2013 5:47 PM
To: solr-user@lucene.apache.org
Cc: Meng, Fan
Subject: RE: Is it possible to searh Solr with a longer query string?
Hi,
I'm using Solr server to develop a search service, and I encounter a problem
when trying to input a longer query string:
Here is the code:
StringBuffer stringBuffer = new StringBuffer();
... ...
try{
//search Pubmed server( a NCBI server ), it returns a list of
IDs.
EFetchPubmedServiceStub service = new EFetchPubmedServiceStub();
EFetchPubmedServiceStub.EFetchRequest req = new
EFetchPubmedServiceStub.EFetchRequest();
req.setWebEnv( WebEnv );
req.setQuery_key( query_key );
req.setRetstart( 1110 );
// return 295 IDs
req.setRetmax( 295 );
EFetchPubmedServiceStub.EFetchResult res = service.run_eFetch(
req );
//connect returned IDs with OR and query my local Solr
server
for( int i = 0; i
res.getPubmedArticleSet().getPubmedArticleSetChoice().length; i++ ){
EFetchPubmedServiceStub.PubmedArticleType art =
res.getPubmedArticleSet().getPubmedArticleSetChoice()
[ i ].getPubmedArticle();
if( i 0 ){
stringBuffer.append( OR );
}
stringBuffer.append( ( pmid: +
art.getMedlineCitation().getPMID().getString() + ) );
}
HttpSolrServer solrServer = new HttpSolrServer(
http://127.0.0.1:8087/solr430/medline; );
String q = stringBuffer.toString();
//when input query has more 300 IDs, query will throw
org.apache.solr.client.solrj.SolrServerException: Server at
http://127.0.0.1:8087/solr430/medline returned non ok status:400,
message:Bad Request
QueryResponse solrRes = solrServer.query( new SolrQuery( q ) );
long found = solrRes.getResults().getNumFound();
System.out.println( found );
}
catch( Exception e ){
e.printStackTrace();
}
... ...
Do you think it's possible to change the query string length limit so that
Solr can accept more IDs?
Thanks.
-Gary
Re: Is it possible to searh Solr with a longer query string?
If your query is arriving on the server correctly, but throwing an
exception, adjust maxBooleanClauses in your solrconfig.xml. I'm not sure
what the consequences are of making it too large, but we had to adjust it
from the default of 1024 to 5000 in one implementation.
Basically, each ID in your query is a separate clause. So, you may have
exceeded maxBooleanClauses.
-Kevin
On Tue, Jun 25, 2013 at 5:15 PM, Jack Krupansky j...@basetechnology.comwrote:
Are you using Tomcat?
See:
http://wiki.apache.org/solr/**SolrTomcat#Enabling_Longer_**Query_Requestshttp://wiki.apache.org/solr/SolrTomcat#Enabling_Longer_Query_Requests
Enabling Longer Query Requests
If you try to submit too long a GET query to Solr, then Tomcat will reject
your HTTP request on the grounds that the HTTP header is too large;
symptoms may include an HTTP 400 Bad Request error or (if you execute the
query in a web browser) a blank browser window.
If you need to enable longer queries, you can set the maxHttpHeaderSize
attribute on the HTTP Connector element in your server.xml file. The
default value is 4K. (See http://tomcat.apache.org/**
tomcat-5.5-doc/config/http.**htmlhttp://tomcat.apache.org/tomcat-5.5-doc/config/http.html
)
---
If you're not using Tomcat, your container may have a similar limit.
-- Jack Krupansky
-Original Message- From: yang, gang
Sent: Tuesday, June 25, 2013 5:47 PM
To: solr-user@lucene.apache.org
Cc: Meng, Fan
Subject: RE: Is it possible to searh Solr with a longer query string?
Hi,
I'm using Solr server to develop a search service, and I encounter a
problem when trying to input a longer query string:
Here is the code:
StringBuffer stringBuffer = new StringBuffer();
... ...
try{
//search Pubmed server( a NCBI server ), it returns a list of
IDs.
EFetchPubmedServiceStub service = new EFetchPubmedServiceStub();
EFetchPubmedServiceStub.**EFetchRequest req = new
EFetchPubmedServiceStub.**EFetchRequest();
req.setWebEnv( WebEnv );
req.setQuery_key( query_key );
req.setRetstart( 1110 );
// return 295 IDs
req.setRetmax( 295 );
EFetchPubmedServiceStub.**EFetchResult res =
service.run_eFetch( req );
//connect returned IDs with OR and query my local Solr
server
for( int i = 0; i res.getPubmedArticleSet().**
getPubmedArticleSetChoice().**length; i++ ){
EFetchPubmedServiceStub.**PubmedArticleType art =
res.getPubmedArticleSet().**getPubmedArticleSetChoice()
[ i ].getPubmedArticle();
if( i 0 ){
stringBuffer.append( OR );
}
stringBuffer.append( ( pmid: + art.getMedlineCitation().*
*getPMID().getString() + ) );
}
HttpSolrServer solrServer = new HttpSolrServer(
http://127.0.0.1:8087/**solr430/medlinehttp://127.0.0.1:8087/solr430/medline
);
String q = stringBuffer.toString();
//when input query has more 300 IDs, query will throw
org.apache.solr.client.solrj.**SolrServerException: Server at
http://127.0.0.1:8087/solr430/**medlinehttp://127.0.0.1:8087/solr430/medlinereturned
non ok status:400, message:Bad Request
QueryResponse solrRes = solrServer.query( new SolrQuery( q ) );
long found = solrRes.getResults().**getNumFound();
System.out.println( found );
}
catch( Exception e ){
e.printStackTrace();
}
... ...
Do you think it's possible to change the query string length limit so that
Solr can accept more IDs?
Thanks.
-Gary
--
*KEVIN OSBORN*
LEAD SOFTWARE ENGINEER
CNET Content Solutions
OFFICE 949.399.8714
CELL 949.310.4677 SKYPE osbornk
5 Park Plaza, Suite 600, Irvine, CA 92614
[image: CNET Content Solutions]
