subject:"How to reduce the search speed of solrcloud"

Re: How to reduce the search speed of solrcloud

2014-04-08 Thread Sathya

Hi Anshum,

I am using Solr 4.7. And i follow this
tutorialhttp://myjeeva.com/solrcloud-cluster-single-collection-deployment.htmlto
setup a solrcloud. I have only one collection in my solr. Kindly let
me,
if u need more details.

On Fri, Apr 4, 2014 at 11:19 PM, Anshum Gupta [via Lucene]
ml-node+s472066n4129190...@n3.nabble.com wrote:

I am not sure if you setup your SolrCloud right. Can you also provide me
with the version of Solr that you're running.
Also, if you could tell me about how did you setup your SolrCloud cluster.
Are the times consistent? Is this the only collection on the cluster?

Also, if I am getting it right, you have 15 ZKs running. Correct me if I'm
wrong, but if I'm not, you don't need that kind of a zk setup.

On Fri, Apr 4, 2014 at 9:39 AM, Sathya [hidden
email]http://user/SendEmail.jtp?type=nodenode=4129190i=0
wrote:

Hi shawn,

I have indexed 50 million data in 5 servers. 3 servers have 8gb ram. One
have 24gb and another one have 64gb ram. I was allocate 4 gb ram to solr
in
each machine. I am using solrcloud. My total index size is 50gb
including 5
servers. Each server have 3 zookeepers. Still I didnt check about OS
disk
cache and heap memory. I will check and let u know shawn. If anything,
pls
let me know.

Thank u shawn.

On Friday, April 4, 2014, Shawn Heisey-4 [via Lucene]
[hidden email] http://user/SendEmail.jtp?type=nodenode=4129190i=1
wrote:
On 4/4/2014 1:31 AM, Sathya wrote:
Hi All,

Hi All, I am new to Solr. And i dont know how to increase the search
speed
of solrcloud. I have indexed nearly 4 GB of data. When i am searching
a
document using java with solrj, solr takes more 6 seconds to return a
query
result. Any one please help me to reduce the search query time to
less
than
500 ms. i have allocate the 4 GB ram for solr. Please let me know for
further details about solrcloud config.

How much total RAM do you have on the system, and how much total index
data is on that system (adding up all the Solr cores)? You've already
said that you have allocated 4GB of RAM for Solr.

Later you said you had 50 million documents, and then you showed us a
URL that looks like SolrCloud.

I suspect that you don't have enough RAM left over to cache your index
effectively -- the OS Disk Cache is too small.

http://wiki.apache.org/solr/SolrPerformanceProblems

Another possible problem, also discussed on that page, is that your
Java
heap is too small.

Thanks,
Shawn

If you reply to this email, your message will be added to the
discussion
below:

http://lucene.472066.n3.nabble.com/How-to-reduce-the-search-speed-of-solrcloud-tp4129067p4129150.html
To unsubscribe from How to reduce the search speed of solrcloud, click
here.
NAML

--
View this message in context:

http://lucene.472066.n3.nabble.com/How-to-reduce-the-search-speed-of-solrcloud-tp4129067p4129173.html

Sent from the Solr - User mailing list archive at Nabble.com.

Anshum Gupta
http://www.anshumgupta.net

--
If you reply to this email, your message will be added to the discussion
below:

http://lucene.472066.n3.nabble.com/How-to-reduce-the-search-speed-of-solrcloud-tp4129067p4129190.html
To unsubscribe from How to reduce the search speed of solrcloud, click
herehttp://lucene.472066.n3.nabble.com/template/NamlServlet.jtp?macro=unsubscribe_by_codenode=4129067code=c2F0aGlhLmJsYWNrc3RhckBnbWFpbC5jb218NDEyOTA2N3wtMjEyNDcwMTI5OA==
.
NAMLhttp://lucene.472066.n3.nabble.com/template/NamlServlet.jtp?macro=macro_viewerid=instant_html%21nabble%3Aemail.namlbase=nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.view.web.template.NodeNamespacebreadcrumbs=notify_subscribers%21nabble%3Aemail.naml-instant_emails%21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml

--
View this message in context:
http://lucene.472066.n3.nabble.com/How-to-reduce-the-search-speed-of-solrcloud-tp4129067p4129910.html
Sent from the Solr - User mailing list archive at Nabble.com.

Re: How to reduce the search speed of solrcloud

2014-04-08 Thread Sathya

Hi All,

I found that which is taking more time. It is *server.query*

 SolrDataDAO dataDao = new SolrDataDAO();
QueryResponse resp = dataDao.queryData(0, 1, subject);
SolrDocumentList data = resp.getResults();
System.out.println(len  + data.size());
System.out.println();

Subject is passed to here:

public QueryResponse queryData(int start, int rows, String query)
throws Exception {
ModifiableSolrParams solrParams = new ModifiableSolrParams();
try {

query = Clean(query);
query = escaping(query);

// String q = \;
// query = q + query + q;
// System.out.println(\t\t--);

System.out.println(Checking : + query);
solrParams.set(q, escaping(query));
solrParams.set(start, start);
solrParams.set(rows, rows);

// solrParams.set(facet, facet); // Facets if required

} catch (Exception e) {
e.printStackTrace();
// System.err.println(Error subject :+query);
}
//return is taking more time...
return server.query(solrParams);
}


Here i am sending the subject to Solr for searching and it is searched
within 200 to 300 ms. But while returning the search result
to QueryResponse, Its getting delay. I dont know why and how to solve this.
Please help me friends. My project is nearly completing stage. Please help
me to complete. Also let me know more details.



On Tue, Apr 8, 2014 at 10:48 PM, Sathya Narayanan 
sathia.blacks...@gmail.com wrote:

 Hi Anshum,

 I am using Solr 4.7. And i follow this 
 tutorialhttp://myjeeva.com/solrcloud-cluster-single-collection-deployment.htmlto
  setup a solrcloud. I have only one collection in my solr. Kindly let me,
 if u need more details.


 On Fri, Apr 4, 2014 at 11:19 PM, Anshum Gupta [via Lucene] 
 ml-node+s472066n4129190...@n3.nabble.com wrote:

 I am not sure if you setup your SolrCloud right. Can you also provide me
 with the version of Solr that you're running.
 Also, if you could tell me about how did you setup your SolrCloud
 cluster.
 Are the times consistent? Is this the only collection on the cluster?

 Also, if I am getting it right, you have 15 ZKs running. Correct me if
 I'm
 wrong, but if I'm not, you don't need that kind of a zk setup.


 On Fri, Apr 4, 2014 at 9:39 AM, Sathya [hidden 
 email]http://user/SendEmail.jtp?type=nodenode=4129190i=0
 wrote:

  Hi shawn,
 
  I have indexed 50 million data in 5 servers. 3 servers have 8gb ram.
 One
  have 24gb and another one have 64gb ram. I was allocate 4 gb ram to
 solr in
  each machine. I am using solrcloud. My total index size is 50gb
 including 5
  servers. Each server have 3 zookeepers. Still I didnt check about OS
 disk
  cache and heap memory. I will check and let u know shawn. If anything,
 pls
  let me know.
 
  Thank u shawn.
 
  On Friday, April 4, 2014, Shawn Heisey-4 [via Lucene] 
  [hidden email] http://user/SendEmail.jtp?type=nodenode=4129190i=1
 wrote:
   On 4/4/2014 1:31 AM, Sathya wrote:
   Hi All,
  
   Hi All, I am new to Solr. And i dont know how to increase the search
  speed
   of solrcloud. I have indexed nearly 4 GB of data. When i am
 searching a
   document using java with solrj, solr takes more 6 seconds to return
 a
  query
   result. Any one please help me to reduce the search query time to
 less
  than
   500 ms. i have allocate the 4 GB ram for solr. Please let me know
 for
   further details about solrcloud config.
  
   How much total RAM do you have on the system, and how much total
 index
   data is on that system (adding up all the Solr cores)?  You've
 already
   said that you have allocated 4GB of RAM for Solr.
  
   Later you said you had 50 million documents, and then you showed us a
   URL that looks like SolrCloud.
  
   I suspect that you don't have enough RAM left over to cache your
 index
   effectively -- the OS Disk Cache is too small.
  
   http://wiki.apache.org/solr/SolrPerformanceProblems
  
   Another possible problem, also discussed on that page, is that your
 Java
   heap is too small.
  
   Thanks,
   Shawn
  
  
  
   
   If you reply to this email, your message will be added to the
 discussion
  below:
  
 
 
 http://lucene.472066.n3.nabble.com/How-to-reduce-the-search-speed-of-solrcloud-tp4129067p4129150.html
   To unsubscribe from How to reduce the search speed of solrcloud,
 click
  here.
   NAML
 
 
 
 
  --
  View this message in context:
 
 http://lucene.472066.n3.nabble.com/How-to-reduce-the-search-speed-of-solrcloud-tp4129067p4129173.html

  Sent from the Solr - User mailing list archive at Nabble.com.
 



 --

 Anshum Gupta
 http://www.anshumgupta.net


 --
  If you reply to this email, your message will be added to the
 discussion below:

 http://lucene.472066.n3.nabble.com/How-to-reduce-the-search-speed-of-solrcloud-tp4129067p4129190.html
  To unsubscribe from How to reduce the search speed of solrcloud, click

Re: How to reduce the search speed of solrcloud

2014-04-06 Thread Sathya

Hi,

I use this link to setup a solrcloud
http://myjeeva.com/solrcloud-cluster-single-collection-deployment.html And
i use 5 different machine to setup this cloud. I use Unique id.

On Sat, Apr 5, 2014 at 6:30 AM, Alexandre Rafalovitch [via Lucene]
ml-node+s472066n4129333...@n3.nabble.com wrote:

And 50 million records of 3 fields each should not become 50Gb of
data. Something smells wrong there. Do you have unique IDs setup?

Regards,
Alex.
Personal website: http://www.outerthoughts.com/
Current project: http://www.solr-start.com/ - Accelerating your Solr
proficiency

On Sat, Apr 5, 2014 at 12:48 AM, Anshum Gupta [hidden
email]http://user/SendEmail.jtp?type=nodenode=4129333i=0
wrote:

I am not sure if you setup your SolrCloud right. Can you also provide me
with the version of Solr that you're running.
Also, if you could tell me about how did you setup your SolrCloud
cluster.
Are the times consistent? Is this the only collection on the cluster?

Also, if I am getting it right, you have 15 ZKs running. Correct me if
I'm
wrong, but if I'm not, you don't need that kind of a zk setup.

On Fri, Apr 4, 2014 at 9:39 AM, Sathya [hidden
email]http://user/SendEmail.jtp?type=nodenode=4129333i=1
wrote:

Hi shawn,

I have indexed 50 million data in 5 servers. 3 servers have 8gb ram.
One
have 24gb and another one have 64gb ram. I was allocate 4 gb ram to
solr in
each machine. I am using solrcloud. My total index size is 50gb
including 5
servers. Each server have 3 zookeepers. Still I didnt check about OS
disk
cache and heap memory. I will check and let u know shawn. If anything,
pls
let me know.

Thank u shawn.

On Friday, April 4, 2014, Shawn Heisey-4 [via Lucene]
[hidden email] http://user/SendEmail.jtp?type=nodenode=4129333i=2
wrote:
On 4/4/2014 1:31 AM, Sathya wrote:
Hi All,

Hi All, I am new to Solr. And i dont know how to increase the search
speed
of solrcloud. I have indexed nearly 4 GB of data. When i am
searching a
document using java with solrj, solr takes more 6 seconds to return
a
query
result. Any one please help me to reduce the search query time to
less
than
500 ms. i have allocate the 4 GB ram for solr. Please let me know
for
further details about solrcloud config.

How much total RAM do you have on the system, and how much total
index
data is on that system (adding up all the Solr cores)? You've
already
said that you have allocated 4GB of RAM for Solr.

Later you said you had 50 million documents, and then you showed us a
URL that looks like SolrCloud.

I suspect that you don't have enough RAM left over to cache your
index
effectively -- the OS Disk Cache is too small.

http://wiki.apache.org/solr/SolrPerformanceProblems

Another possible problem, also discussed on that page, is that your
Java
heap is too small.

Thanks,
Shawn

If you reply to this email, your message will be added to the
discussion
below:

http://lucene.472066.n3.nabble.com/How-to-reduce-the-search-speed-of-solrcloud-tp4129067p4129150.html
To unsubscribe from How to reduce the search speed of solrcloud,
click
here.
NAML

--
View this message in context:

http://lucene.472066.n3.nabble.com/How-to-reduce-the-search-speed-of-solrcloud-tp4129067p4129173.html
Sent from the Solr - User mailing list archive at Nabble.com.

Anshum Gupta
http://www.anshumgupta.net

--
If you reply to this email, your message will be added to the discussion
below:

http://lucene.472066.n3.nabble.com/How-to-reduce-the-search-speed-of-solrcloud-tp4129067p4129333.html
To unsubscribe from How to reduce the search speed of solrcloud, click
herehttp://lucene.472066.n3.nabble.com/template/NamlServlet.jtp?macro=unsubscribe_by_codenode=4129067code=c2F0aGlhLmJsYWNrc3RhckBnbWFpbC5jb218NDEyOTA2N3wtMjEyNDcwMTI5OA==
.
NAMLhttp://lucene.472066.n3.nabble.com/template/NamlServlet.jtp?macro=macro_viewerid=instant_html%21nabble%3Aemail.namlbase=nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.view.web.template.NodeNamespacebreadcrumbs=notify_subscribers%21nabble%3Aemail.naml-instant_emails%21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml

--
View this message in context:
http://lucene.472066.n3.nabble.com/How-to-reduce-the-search-speed-of-solrcloud-tp4129067p4129564.html
Sent from the Solr - User mailing list archive at Nabble.com.

How to reduce the search speed of solrcloud

2014-04-04 Thread Sathya

Hi All,

Hi All, I am new to Solr. And i dont know how to increase the search speed
of solrcloud. I have indexed nearly 4 GB of data. When i am searching a
document using java with solrj, solr takes more 6 seconds to return a query
result. Any one please help me to reduce the search query time to less than
500 ms. i have allocate the 4 GB ram for solr. Please let me know for
further details about solrcloud config.



--
View this message in context: 
http://lucene.472066.n3.nabble.com/How-to-reduce-the-search-speed-of-solrcloud-tp4129067.html
Sent from the Solr - User mailing list archive at Nabble.com.

Re: How to reduce the search speed of solrcloud

2014-04-04 Thread Alexandre Rafalovitch

Show a sample query string that does that (takes 6 seconds to return).
Including all defaults you may have put in solrconfig.xml (if any).
That might give us a hint which features you are using and what
possible direction you could go in next. For the bonus points, enable
debug flag and rows=1 parameter to see how big your documents
themselves are.

You may have issues with a particular non-cloud-friendly feature, with
caches, with not reusing parts of your queries as 'fq', returning too
many fields or a bunch of other things.

Regards,
   Alex.
Personal website: http://www.outerthoughts.com/
Current project: http://www.solr-start.com/ - Accelerating your Solr proficiency


On Fri, Apr 4, 2014 at 2:31 PM, Sathya sathia.blacks...@gmail.com wrote:
 Hi All,

 Hi All, I am new to Solr. And i dont know how to increase the search speed
 of solrcloud. I have indexed nearly 4 GB of data. When i am searching a
 document using java with solrj, solr takes more 6 seconds to return a query
 result. Any one please help me to reduce the search query time to less than
 500 ms. i have allocate the 4 GB ram for solr. Please let me know for
 further details about solrcloud config.



 --
 View this message in context: 
 http://lucene.472066.n3.nabble.com/How-to-reduce-the-search-speed-of-solrcloud-tp4129067.html
 Sent from the Solr - User mailing list archive at Nabble.com.

Re: How to reduce the search speed of solrcloud

2014-04-04 Thread Sathya

Hi Alex,

str name=id33026985/str str name=subjectComponent Audio\:A
Shopping List/str str name=download_date2012-01-11
09:02:42.96/str

This is what i am  indexed in solr. I have only 3 fields in index. And
i am just indexing id, subject and date of the news articles. Nearly 5
crore documents. Also i have attached my solrconfig and solr.xml file.
If u need more information, pls let me know.

On Fri, Apr 4, 2014 at 1:15 PM, Alexandre Rafalovitch [via Lucene]
ml-node+s472066n4129068...@n3.nabble.com wrote:

 Show a sample query string that does that (takes 6 seconds to return).
 Including all defaults you may have put in solrconfig.xml (if any).
 That might give us a hint which features you are using and what
 possible direction you could go in next. For the bonus points, enable
 debug flag and rows=1 parameter to see how big your documents
 themselves are.

 You may have issues with a particular non-cloud-friendly feature, with
 caches, with not reusing parts of your queries as 'fq', returning too
 many fields or a bunch of other things.

 Regards,
Alex.
 Personal website: http://www.outerthoughts.com/
 Current project: http://www.solr-start.com/ - Accelerating your Solr 
 proficiency


 On Fri, Apr 4, 2014 at 2:31 PM, Sathya [hidden email] wrote:

  Hi All,
 
  Hi All, I am new to Solr. And i dont know how to increase the search speed
  of solrcloud. I have indexed nearly 4 GB of data. When i am searching a
  document using java with solrj, solr takes more 6 seconds to return a query
  result. Any one please help me to reduce the search query time to less than
  500 ms. i have allocate the 4 GB ram for solr. Please let me know for
  further details about solrcloud config.
 
 
 
  --
  View this message in context: 
  http://lucene.472066.n3.nabble.com/How-to-reduce-the-search-speed-of-solrcloud-tp4129067.html
  Sent from the Solr - User mailing list archive at Nabble.com.


 
 If you reply to this email, your message will be added to the discussion 
 below:
 http://lucene.472066.n3.nabble.com/How-to-reduce-the-search-speed-of-solrcloud-tp4129067p4129068.html
 To unsubscribe from How to reduce the search speed of solrcloud, click here.
 NAML


solrconfig.xml (101K) 
http://lucene.472066.n3.nabble.com/attachment/4129073/0/solrconfig.xml
solr.xml (1K) http://lucene.472066.n3.nabble.com/attachment/4129073/1/solr.xml




--
View this message in context: 
http://lucene.472066.n3.nabble.com/How-to-reduce-the-search-speed-of-solrcloud-tp4129067p4129073.html
Sent from the Solr - User mailing list archive at Nabble.com.

Re: How to reduce the search speed of solrcloud

2014-04-04 Thread Alexandre Rafalovitch

What does your Solr query looks like (check the Solr backend log if
you don't know)?

And how many document is that? 50 million? Does not sound like much
for 3 fields. And what's the definitions (schema.xml rather than
solr.xml).

And what happens if you issue the query directly to Solr rather than
through the client? Is the speed much different?

Regards,
Alex.

Personal website: http://www.outerthoughts.com/
Current project: http://www.solr-start.com/ - Accelerating your Solr proficiency

On Fri, Apr 4, 2014 at 3:12 PM, Sathya sathia.blacks...@gmail.com wrote:
Hi Alex,

str name=id33026985/str str name=subjectComponent Audio\:A
Shopping List/str str name=download_date2012-01-11
09:02:42.96/str

This is what i am indexed in solr. I have only 3 fields in index. And
i am just indexing id, subject and date of the news articles. Nearly 5
crore documents. Also i have attached my solrconfig and solr.xml file.
If u need more information, pls let me know.

On Fri, Apr 4, 2014 at 1:15 PM, Alexandre Rafalovitch [via Lucene]
ml-node+s472066n4129068...@n3.nabble.com wrote:

Show a sample query string that does that (takes 6 seconds to return).
Including all defaults you may have put in solrconfig.xml (if any).
That might give us a hint which features you are using and what
possible direction you could go in next. For the bonus points, enable
debug flag and rows=1 parameter to see how big your documents
themselves are.

You may have issues with a particular non-cloud-friendly feature, with
caches, with not reusing parts of your queries as 'fq', returning too
many fields or a bunch of other things.

Regards,
Alex.
Personal website: http://www.outerthoughts.com/
Current project: http://www.solr-start.com/ - Accelerating your Solr
proficiency

On Fri, Apr 4, 2014 at 2:31 PM, Sathya [hidden email] wrote:

Hi All,

Hi All, I am new to Solr. And i dont know how to increase the search speed
of solrcloud. I have indexed nearly 4 GB of data. When i am searching a
document using java with solrj, solr takes more 6 seconds to return a query
result. Any one please help me to reduce the search query time to less than
500 ms. i have allocate the 4 GB ram for solr. Please let me know for
further details about solrcloud config.

--
View this message in context:
http://lucene.472066.n3.nabble.com/How-to-reduce-the-search-speed-of-solrcloud-tp4129067.html
Sent from the Solr - User mailing list archive at Nabble.com.

If you reply to this email, your message will be added to the discussion
below:
http://lucene.472066.n3.nabble.com/How-to-reduce-the-search-speed-of-solrcloud-tp4129067p4129068.html
To unsubscribe from How to reduce the search speed of solrcloud, click here.
NAML

solrconfig.xml (101K)
http://lucene.472066.n3.nabble.com/attachment/4129073/0/solrconfig.xml
solr.xml (1K)
http://lucene.472066.n3.nabble.com/attachment/4129073/1/solr.xml

--
View this message in context:
http://lucene.472066.n3.nabble.com/How-to-reduce-the-search-speed-of-solrcloud-tp4129067p4129073.html
Sent from the Solr - User mailing list archive at Nabble.com.

Re: How to reduce the search speed of solrcloud

2014-04-04 Thread Sathya

Hi,

I have attached my schema.xml file too.

And you are right. I have 50 million documents. When i use solr
browser to search a document, it will return within 1000 to 2000 ms.

My query looks like this:
http://10.10.1.14:5050/solr/set_recent_shard1_replica5/select?q=subjectindent=true

On 4/4/14, Alexandre Rafalovitch [via Lucene]
ml-node+s472066n4129074...@n3.nabble.com wrote:

What does your Solr query looks like (check the Solr backend log if
you don't know)?

And how many document is that? 50 million? Does not sound like much
for 3 fields. And what's the definitions (schema.xml rather than
solr.xml).

And what happens if you issue the query directly to Solr rather than
through the client? Is the speed much different?

Regards,
Alex.

Personal website: http://www.outerthoughts.com/
Current project: http://www.solr-start.com/ - Accelerating your Solr
proficiency

On Fri, Apr 4, 2014 at 3:12 PM, Sathya sathia.blacks...@gmail.com wrote:
Hi Alex,

str name=id33026985/str str name=subjectComponent Audio\:A
Shopping List/str str name=download_date2012-01-11
09:02:42.96/str

On Fri, Apr 4, 2014 at 1:15 PM, Alexandre Rafalovitch [via Lucene]
ml-node+s472066n4129068...@n3.nabble.com wrote:

You may have issues with a particular non-cloud-friendly feature, with
caches, with not reusing parts of your queries as 'fq', returning too
many fields or a bunch of other things.

Regards,
Alex.
Personal website: http://www.outerthoughts.com/
Current project: http://www.solr-start.com/ - Accelerating your Solr
proficiency

On Fri, Apr 4, 2014 at 2:31 PM, Sathya [hidden email] wrote:

Hi All,

Hi All, I am new to Solr. And i dont know how to increase the search
speed
of solrcloud. I have indexed nearly 4 GB of data. When i am searching
a
document using java with solrj, solr takes more 6 seconds to return a
query
result. Any one please help me to reduce the search query time to less
than
500 ms. i have allocate the 4 GB ram for solr. Please let me know for
further details about solrcloud config.

--
View this message in context:
http://lucene.472066.n3.nabble.com/How-to-reduce-the-search-speed-of-solrcloud-tp4129067.html
Sent from the Solr - User mailing list archive at Nabble.com.

solrconfig.xml (101K)
http://lucene.472066.n3.nabble.com/attachment/4129073/0/solrconfig.xml
solr.xml (1K)
http://lucene.472066.n3.nabble.com/attachment/4129073/1/solr.xml

--
View this message in context:
http://lucene.472066.n3.nabble.com/How-to-reduce-the-search-speed-of-solrcloud-tp4129067p4129073.html
Sent from the Solr - User mailing list archive at Nabble.com.

___
If you reply to this email, your message will be added to the discussion
below:
http://lucene.472066.n3.nabble.com/How-to-reduce-the-search-speed-of-solrcloud-tp4129067p4129074.html

To unsubscribe from How to reduce the search speed of solrcloud, visit
http://lucene.472066.n3.nabble.com/template/NamlServlet.jtp?macro=unsubscribe_by_codenode=4129067code=c2F0aGlhLmJsYWNrc3RhckBnbWFpbC5jb218NDEyOTA2N3wtMjEyNDcwMTI5OA==

schema.xml (81K)
http://lucene.472066.n3.nabble.com/attachment/4129075/0/schema.xml

--
View this message in context:
http://lucene.472066.n3.nabble.com/How-to-reduce-the-search-speed-of-solrcloud-tp4129067p4129075.html
Sent from the Solr - User mailing list archive at Nabble.com.

Re: How to reduce the search speed of solrcloud

2014-04-04 Thread Alexandre Rafalovitch

Well, if the direct browser query is 1000ms and your client query is
6seconds, then it is not Solr itself you need to worry about first.
Something must be wrong at the client. Trying timing that bit. Maybe
it is writing from the client to your ultimate consumer that's the
problem.

Regards,
Alex.
P.s. You should probably trim your schema to get rid of all the
example fields. Keep _version_ and _root_ but delete all the rest you
don't actually use. Same with dynamic fields and all fieldType
definitions you do not actually use. You can always reintroduce them
later from the example schemas if something is missing.

Personal website: http://www.outerthoughts.com/
Current project: http://www.solr-start.com/ - Accelerating your Solr proficiency

On Fri, Apr 4, 2014 at 3:41 PM, Sathya sathia.blacks...@gmail.com wrote:
Hi,

I have attached my schema.xml file too.

And you are right. I have 50 million documents. When i use solr
browser to search a document, it will return within 1000 to 2000 ms.

My query looks like this:
http://10.10.1.14:5050/solr/set_recent_shard1_replica5/select?q=subjectindent=true

On 4/4/14, Alexandre Rafalovitch [via Lucene]
ml-node+s472066n4129074...@n3.nabble.com wrote:

What does your Solr query looks like (check the Solr backend log if
you don't know)?

And how many document is that? 50 million? Does not sound like much
for 3 fields. And what's the definitions (schema.xml rather than
solr.xml).

And what happens if you issue the query directly to Solr rather than
through the client? Is the speed much different?

Regards,
Alex.

Personal website: http://www.outerthoughts.com/
Current project: http://www.solr-start.com/ - Accelerating your Solr
proficiency

On Fri, Apr 4, 2014 at 3:12 PM, Sathya sathia.blacks...@gmail.com wrote:
Hi Alex,

str name=id33026985/str str name=subjectComponent Audio\:A
Shopping List/str str name=download_date2012-01-11
09:02:42.96/str

On Fri, Apr 4, 2014 at 1:15 PM, Alexandre Rafalovitch [via Lucene]
ml-node+s472066n4129068...@n3.nabble.com wrote:

You may have issues with a particular non-cloud-friendly feature, with
caches, with not reusing parts of your queries as 'fq', returning too
many fields or a bunch of other things.

Regards,
Alex.
Personal website: http://www.outerthoughts.com/
Current project: http://www.solr-start.com/ - Accelerating your Solr
proficiency

On Fri, Apr 4, 2014 at 2:31 PM, Sathya [hidden email] wrote:

Hi All,

Hi All, I am new to Solr. And i dont know how to increase the search
speed
of solrcloud. I have indexed nearly 4 GB of data. When i am searching
a
document using java with solrj, solr takes more 6 seconds to return a
query
result. Any one please help me to reduce the search query time to less
than
500 ms. i have allocate the 4 GB ram for solr. Please let me know for
further details about solrcloud config.

--
View this message in context:
http://lucene.472066.n3.nabble.com/How-to-reduce-the-search-speed-of-solrcloud-tp4129067.html
Sent from the Solr - User mailing list archive at Nabble.com.

solrconfig.xml (101K)
http://lucene.472066.n3.nabble.com/attachment/4129073/0/solrconfig.xml
solr.xml (1K)
http://lucene.472066.n3.nabble.com/attachment/4129073/1/solr.xml

--
View this message in context:
http://lucene.472066.n3.nabble.com/How-to-reduce-the-search-speed-of-solrcloud-tp4129067p4129073.html
Sent from the Solr - User mailing list archive at Nabble.com.

___
If you reply to this email, your message will be added to the discussion
below:
http://lucene.472066.n3.nabble.com/How-to-reduce-the-search-speed-of-solrcloud-tp4129067p4129074.html

schema.xml (81K)
http://lucene.472066.n3.nabble.com/attachment/4129075/0/schema.xml

Re: How to reduce the search speed of solrcloud

2014-04-04 Thread Sathya

Hi,

Sorry, i cant get u alex. Can you please explain me(if you can). Because
now only i entered into solr.

On Fri, Apr 4, 2014 at 2:20 PM, Alexandre Rafalovitch [via Lucene]
ml-node+s472066n4129077...@n3.nabble.com wrote:

Personal website: http://www.outerthoughts.com/
Current project: http://www.solr-start.com/ - Accelerating your Solr
proficiency

On Fri, Apr 4, 2014 at 3:41 PM, Sathya [hidden
email]http://user/SendEmail.jtp?type=nodenode=4129077i=0
wrote:

Hi,

I have attached my schema.xml file too.

And you are right. I have 50 million documents. When i use solr
browser to search a document, it will return within 1000 to 2000 ms.

My query looks like this:

http://10.10.1.14:5050/solr/set_recent_shard1_replica5/select?q=subjectindent=true

On 4/4/14, Alexandre Rafalovitch [via Lucene]
[hidden email] http://user/SendEmail.jtp?type=nodenode=4129077i=1
wrote:

What does your Solr query looks like (check the Solr backend log if
you don't know)?

And how many document is that? 50 million? Does not sound like much
for 3 fields. And what's the definitions (schema.xml rather than
solr.xml).

And what happens if you issue the query directly to Solr rather than
through the client? Is the speed much different?

Regards,
Alex.

Personal website: http://www.outerthoughts.com/
Current project: http://www.solr-start.com/ - Accelerating your Solr
proficiency

On Fri, Apr 4, 2014 at 3:12 PM, Sathya [hidden
email]http://user/SendEmail.jtp?type=nodenode=4129077i=2
wrote:
Hi Alex,

str name=id33026985/str str name=subjectComponent Audio\:A
Shopping List/str str name=download_date2012-01-11
09:02:42.96/str

On Fri, Apr 4, 2014 at 1:15 PM, Alexandre Rafalovitch [via Lucene]
[hidden email] http://user/SendEmail.jtp?type=nodenode=4129077i=3
wrote:

Show a sample query string that does that (takes 6 seconds to
return).
Including all defaults you may have put in solrconfig.xml (if any).
That might give us a hint which features you are using and what
possible direction you could go in next. For the bonus points, enable
debug flag and rows=1 parameter to see how big your documents
themselves are.

You may have issues with a particular non-cloud-friendly feature,
with
caches, with not reusing parts of your queries as 'fq', returning too
many fields or a bunch of other things.

Regards,
Alex.
Personal website: http://www.outerthoughts.com/
Current project: http://www.solr-start.com/ - Accelerating your Solr
proficiency

On Fri, Apr 4, 2014 at 2:31 PM, Sathya [hidden email] wrote:

Hi All,

Hi All, I am new to Solr. And i dont know how to increase the
search
speed
of solrcloud. I have indexed nearly 4 GB of data. When i am
searching
a
document using java with solrj, solr takes more 6 seconds to return
a
query
result. Any one please help me to reduce the search query time to
less
than
500 ms. i have allocate the 4 GB ram for solr. Please let me know
for
further details about solrcloud config.

--
View this message in context:

http://lucene.472066.n3.nabble.com/How-to-reduce-the-search-speed-of-solrcloud-tp4129067.html
Sent from the Solr - User mailing list archive at Nabble.com.

If you reply to this email, your message will be added to the
discussion
below:

http://lucene.472066.n3.nabble.com/How-to-reduce-the-search-speed-of-solrcloud-tp4129067p4129068.html
To unsubscribe from How to reduce the search speed of solrcloud,
click
here.
NAML

solrconfig.xml (101K)

http://lucene.472066.n3.nabble.com/attachment/4129073/0/solrconfig.xml
solr.xml (1K)
http://lucene.472066.n3.nabble.com/attachment/4129073/1/solr.xml

--
View this message in context:

http://lucene.472066.n3.nabble.com/How-to-reduce-the-search-speed-of-solrcloud-tp4129067p4129073.html
Sent from the Solr - User mailing list archive at Nabble.com.

___
If you

Re: How to reduce the search speed of solrcloud

2014-04-04 Thread Alexandre Rafalovitch

You said your request is 6 seconds when going through the SolrJ
client. But it is 1 second (1000 ms) when going directly to Solr
bypassing the SolrJ. So, the other 5 seconds must be added outside of
Solr. Concentrate on that.

Regarding your schema, you used example schema that has a lot of stuff
you do not need. Here is what a very small schema looks like:
https://github.com/arafalov/solr-indexing-book/blob/master/published/collection1/conf/schema.xml
, so you can compare. That's an example from my book. You may find the
book a fast way to get from your current state to early intermediate
(no cloud examples, though).

Contact me directly if you need a discount.

Regards,
Alex.
Personal website: http://www.outerthoughts.com/
Current project: http://www.solr-start.com/ - Accelerating your Solr proficiency

On Fri, Apr 4, 2014 at 4:11 PM, Sathya sathia.blacks...@gmail.com wrote:
Hi,