Re: Solr multicore join performance tuning

2016-01-25 Thread Mikhail Khludnev
What a solr version, query parameters and debug output? 26.01.2016 6:38 пользователь "Bhawna Asnani" написал: > Hi, > I am using solr multicore join queries for some admin filters. The queries > are really slow taking up to 40-60 seconds ins some cases. > > I recently read that the schema field u

Re: solr multicore vs sharding vs 1 big collection

2015-08-04 Thread Shawn Heisey
On 8/4/2015 3:30 PM, Jay Potharaju wrote: > For the last few days I have been trying to correlate the timeouts with GC. > I noticed in the GC logs that full GC takes long time once in a while. Does > this mean that the jvm memory is to high or is it set to low? > 1973953.560: [GC 4474277K->33004

Re: solr multicore vs sharding vs 1 big collection

2015-08-04 Thread Jay Potharaju
For the last few days I have been trying to correlate the timeouts with GC. I noticed in the GC logs that full GC takes long time once in a while. Does this mean that the jvm memory is to high or is it set to low? [GC 4730643K->3552794K(4890112K), 0.0433146 secs] 1973853.751: [Full GC 3552794K->

Re: solr multicore vs sharding vs 1 big collection

2015-08-03 Thread Bill Bell
Yeah a separate by month or year is good and can really help in this case. Bill Bell Sent from mobile > On Aug 2, 2015, at 5:29 PM, Jay Potharaju wrote: > > Shawn, > Thanks for the feedback. I agree that increasing timeout might alleviate > the timeout issue. The main problem with increasing t

Re: solr multicore vs sharding vs 1 big collection

2015-08-03 Thread Upayavira
There are two things that are likely to cause the timeouts you are seeing, I'd say. Firstly, your server is overloaded - that can be handled by adding additional replicas. However, it doesn't seem like this is the case, because the second query works fine. Secondly, you are hitting garbage colle

Re: solr multicore vs sharding vs 1 big collection

2015-08-02 Thread Jay Potharaju
Shawn, Thanks for the feedback. I agree that increasing timeout might alleviate the timeout issue. The main problem with increasing timeout is the detrimental effect it will have on the user experience, therefore can't increase it. I have looked at the queries that threw errors, next time I try it

Re: solr multicore vs sharding vs 1 big collection

2015-08-02 Thread Shawn Heisey
On 8/2/2015 8:29 AM, Jay Potharaju wrote: > The document contains around 30 fields and have stored set to true for > almost 15 of them. And these stored fields are queried and updated all the > time. You will notice that the deleted documents is almost 30% of the > docs. And it has stayed around t

Re: solr multicore vs sharding vs 1 big collection

2015-08-02 Thread Jay Potharaju
The document contains around 30 fields and have stored set to true for almost 15 of them. And these stored fields are queried and updated all the time. You will notice that the deleted documents is almost 30% of the docs. And it has stayed around that percent and has not come down. I did try optim

Re: solr multicore vs sharding vs 1 big collection

2015-08-01 Thread Shawn Heisey
On 8/1/2015 6:49 PM, Jay Potharaju wrote: > I currently have a single collection with 40 million documents and index > size of 25 GB. The collections gets updated every n minutes and as a result > the number of deleted documents is constantly growing. The data in the > collection is an amalgamation

Re: solr multicore vs sharding vs 1 big collection

2015-08-01 Thread Erick Erickson
40 million docs isn't really very many by modern standards, although if they're huge documents then that might be an issue. So is this a single shard or multiple shards? If you're really facing performance issues, simply making a new collection with more than one shard (independent of how many rep

Re: solr multicore problem on SLES 11

2012-09-17 Thread Jochen Lienhard
Great. Thanks. That solves my problem. Greetings Jochen André Widhani schrieb: The first thing I would check is the virtual memory limit (ulimit -v, check this for the operating system user that runs Tomcat /Solr). It should be set to "unlimited", but this is as far as i remember not the de

Re: Solr Multicore - cant seem to get it working?

2011-02-28 Thread Ahmet Arslan
> I'm waiting on my > host to add my IP to the > firewall for me, so I can test that part of it (they setup > the bog standard > Solr for me, but I'm now trying to get it working with the > multicore, as I > think we are gonna need several Solr instances for > different parts of the If you have m

Re: Solr Multicore - cant seem to get it working?

2011-02-28 Thread ultranerds
Hi, I already did answer that one =) I'm waiting on my host to add my IP to the firewall for me, so I can test that part of it (they setup the bog standard Solr for me, but I'm now trying to get it working with the multicore, as I think we are gonna need several Solr instances for different parts

Re: Solr Multicore - cant seem to get it working?

2011-02-28 Thread Stefan Matheis
Andy, try to answer the question other guys asked you .. and not just write something. On Sun, Feb 27, 2011 at 4:16 PM, Ahmet Arslan wrote: > Can you see 'Admin graphics' listed in http://localhost:210/solr/ page? Regards Stefan On Mon, Feb 28, 2011 at 8:57 AM, ultranerds wrote: > Hi, > > Tha

Re: Solr Multicore - cant seem to get it working?

2011-02-28 Thread ultranerds
Hi, Thanks for the reply. I sure did. I did a bit more research, and found out that java was still running, so I run: sudo /etc/init.d/solr stop ...then: killall java (just to make sure it was all closed) ...and finally: sudo /etc/init.d/solr start Yet I'm still getting the error about the

Re: Solr Multicore - cant seem to get it working?

2011-02-27 Thread Chris Wilkes
Did you copy the files into the solr/graphics/ directory? That subdirectory should be the same name as the instanceDir. That is you should have this directory structure: solr/graphics/conf/ solr/graphics/data/index/ On Feb 27, 2011, at 5:20 AM, ultranerds wrote: > Hi, > > I'm trying to get

Re: Solr Multicore - cant seem to get it working?

2011-02-27 Thread Ahmet Arslan
> I'm trying to get a Solr install to work with multicores, > as I wanna use it > on several sites (each totally different, and I don't > obviously wanna have > loads of seperate Solr installs) > > Here is what I've done so far (Solr was already uploaded > and working > normally, without multicore

Re: solr multicore and nfs

2010-02-08 Thread Lance Norskog
Solr generally does not work well over NFS. This looks like a transient NFS error; apps have to assume that NFS will randomly fail and that they have to try again. This may be due to a locking problem. There is a LockFactory class in Lucene that controls how indexes are shared between programs. So

RE: Solr MultiCore query

2009-07-17 Thread Manepalli, Kalyan
ject: Re: Solr MultiCore query Both schema.xml ( in example/multicore/core0/conf and example/multicore/core1/conf ) already have * name* Here are the following query responses: 1) http://localhost:8983/solr/core0/select?shards=localhost:8983/solr/core0,localhost:8983/solr/core1&q=*:* 025

Re: Solr MultiCore query

2009-07-17 Thread Code Tester
Both schema.xml ( in example/multicore/core0/conf and example/multicore/core1/conf ) already have * name* Here are the following query responses: 1) http://localhost:8983/solr/core0/select?shards=localhost:8983/solr/core0,localhost:8983/solr/core1&q=*:* 0254MA147LL/AApple 60 GB iPod with Video

Re: Solr MultiCore query

2009-07-17 Thread ahammad
Hello joe_coder, Are you using the default example docs in your queries? If so, then I see that the word "ipod" appears in a field called "name". By default, the default search field (defined in solrconfig.xml) is the field called "text". This means that when you submit a query without specifyin

Re: Solr MultiCore query

2009-07-17 Thread Code Tester
Thanks ahammad for the quick reply. As suggested, I am trying out multi core way of implementing the search. I am trying out the multicore example and getting stuck at an issue. Here is what I did and the issue I am facing 1) Downloaded 1.4 and started the multicore example using java -Dsolr.solr

Re: Solr MultiCore query

2009-07-17 Thread ahammad
Hello, I'm not sure what the best way is to do this, but I have done something identical. I have the same requirements, ie several datasources. I also used SolrJ and jsp for this. The way I ended up doing it was to create a multi core environment, one core per datasource. When I do a query acros

Re: Solr MultiCore query

2009-07-17 Thread joe_coder
I missed adding some size related information in the query above. D1 and D2 would have close to 1 million records each D3 would have ~10 million records. Thanks! -- View this message in context: http://www.nabble.com/Solr-MultiCore-query-tp24534383p24534421.html Sent from the Solr - User maili

Re: Solr Multicore ...

2008-11-07 Thread Shalin Shekhar Mangar
>From what I can understand, you have little full-text search involved here. You should probably look at Hadoop and its contrib and sub-projects such as Pig, Hive and Chukwa. http://wiki.apache.org/hadoop/ http://wiki.apache.org/hadoop/Hive http://wiki.apache.org/hadoop/Chukwa http://incubator.apa

RE: Solr Multicore ...

2008-11-07 Thread souravm
Hi Guys, Here I'm struggling with to decide whether Solr would be a fitting solution for me. Highly appreciate you The key requirements can be summarized as below - 1. Need to process very high volume of data online from log files of various applications - around 100s of Millions of total size

RE: Solr Multicore ...

2008-11-07 Thread souravm
Thanks Noble for your answer. Regards, Sourav -Original Message- From: Noble Paul നോബിള്‍ नोब्ळ् [mailto:[EMAIL PROTECTED] Sent: Thursday, November 06, 2008 7:41 PM To: solr-user@lucene.apache.org Subject: Re: Solr Multicore ... On Fri, Nov 7, 2008 at 3:28 AM, souravm <[EMAIL PROTEC

Re: Solr Multicore ...

2008-11-06 Thread Noble Paul നോബിള്‍ नोब्ळ्
On Fri, Nov 7, 2008 at 3:28 AM, souravm <[EMAIL PROTECTED]> wrote: > > Hi, > > Can I use multi core feature to have multiple indexes (That is each core > would take care of one type of index) within a single Solar instance ? Yes .And this is why it is conceived > > Will there be any performance im

Re: Solr Multicore, are there any way to retrieve all the cores registered?

2008-05-09 Thread Walter Ferrara
Ryan McKinley wrote: check the "status" action also, check the index.jsp page index.jsp do: org.apache.solr.core.MultiCore multicore = (org.apache.solr.core.MultiCore)request.getAttribute("org.apache.solr.MultiCore"); which is ok in a servlet, but how should I do the same inside an handler,

Re: Solr Multicore, are there any way to retrieve all the cores registered?

2008-05-09 Thread Ryan McKinley
check the "status" action also, check the index.jsp page (i don't have the code in front of me) On May 9, 2008, at 10:16 AM, Walter Ferrara wrote: In solr, last trunk version in svn, is it possible to access the "core registry", or what used to be the static MultiCore object? My goal is to

Re: Solr multicore admin JSP problem on tomcat

2008-04-28 Thread Chris Hostetter
: 1. I use the following URL to successfully browse to the Admin interface of : one of the cores: : : http://devbox:8080/solr/solrtest/admin/ : : 2. On the resulting page, I click on the link [SCHEMA] : : 3. This results in a 404 error. The link to this page is : http://devbox:8080/solr/solrtest

Re: Solr Multicore

2008-01-08 Thread Ryan McKinley
The multicore stuff has changed around a bit since its debut (and may change some more before the final release) check: http://wiki.apache.org/solr/MultiCore There is no longer a 'SETASDEFAULT' action and all requests require the core name, so you will need: http://localhost:8983/solr/core0/