Hi, it's definitely a problem to store 5TB in Solr without using sharding. I try to split data over solr instances, so that the index will fit in my memory on the server.
I ran into trouble with a Solr using 50G index. Daniel On Jan 13, 2012, at 1:08 PM, mustafozbek wrote: > I am an apache solr user about a year. I used solr for simple search tools > but now I want to use solr with 5TB of data. I assume that 5TB data will be > 7TB when solr index it according to filter that I use. And then I will add > nearly 50MB of data per hour to the same index. > 1- Are there any problem using single solr server with 5TB data. (without > shards) > a- Can solr server answers the queries in an acceptable time > b- what is the expected time for commiting of 50MB data on 7TB index. > c- Is there an upper limit for index size. > 2- what are the suggestions that you offer > a- How many shards should I use > b- Should I use solr cores > c- What is the committing frequency you offered. (is 1 hour OK) > 3- are there any test results for this kind of large data > > There is no available 5TB data, I just want to estimate what will be the > result. > Note: You can assume that hardware resourses are not a problem. > > > -- > View this message in context: > http://lucene.472066.n3.nabble.com/Can-Apache-Solr-Handle-TeraByte-Large-Data-tp3656484p3656484.html > Sent from the Solr - User mailing list archive at Nabble.com.
