Hi, I have a huge amount of log files which are stored in HDFS need to be indexed.
And I did some tests,the result follows: <http://lucene.472066.n3.nabble.com/file/n4203155/tests1.png> Then several questions: 1. Why HDFS showed such disappointed performance? 2. If each doc size is about 200~300 bytes,how fast is expected with hdfs or local using mapreduce? 3. If my indexing rate is lower than the expected value,did I do something wrong?Or,how can I optimize it? 4. Is there some better way to deal with my log files? 5. It is said that hdfs is weak in handling small files,how solr deal with this? Any help is sincerely appreciated. Thanks. -- View this message in context: http://lucene.472066.n3.nabble.com/SolrCloud-HDFS-disappointed-indexing-performance-tp4203155.html Sent from the Solr - User mailing list archive at Nabble.com.
