Dear All, Thanks for your replies. Conclusion is, we can not store more than 2147483647 records per shard as of now. The only option is we need to increase the shard count.
Thanks Prasath Rajan On Tuesday, October 14, 2014 9:34:33 PM UTC+5:30, Jörg Prante wrote: > > You can not store more than 2G docs per shard in "Lucene 4.x" codecs. This > is a documented Lucene limit: > > "Similarly, Lucene uses a Java int to refer to document numbers, and the > index file format uses an Int32 on-disk to store document numbers. This is > a limitation of both the index file format and the current implementation. > Eventually these should be replaced with either UInt64 values, or better > yet, VInt values which have no limit." > > > https://lucene.apache.org/core/4_9_1/core/org/apache/lucene/codecs/lucene49/package-summary.html#Limitations > > Jörg > > > On Tue, Oct 14, 2014 at 5:37 PM, Prasanth R <prasanth...@gmail.com > <javascript:>> wrote: > >> Thanks for the reply. >> >> My scenario here is, >> 1) No nested docs. >> 2) I don't have any limit per shard.. >> >> I didn't know about internal limit of ES. >> On Oct 14, 2014 8:23 PM, "Alexandre Rafalovitch" <araf...@gmail.com >> <javascript:>> wrote: >> >>> On 14 October 2014 10:33, Prasanth R <prasanth...@gmail.com >>> <javascript:>> wrote: >>> > There is no upper limit... >>> >>> Well, then you must have an infinitely scalable architecture and a >>> decision when the content starts getting shared. So, then the question >>> is what is your individual shard allowed to grow to. Which is "how >>> many documents - including nested - you are expecting to have in a >>> single shard". >>> >>> Because, ElasticSearch has an internal limit and you just hit it. So, >>> the question is whether it is intentional, unintentional or a result >>> of a bug. >>> >>> Regards, >>> Alex. >>> >>> Personal: http://www.outerthoughts.com/ and @arafalov >>> Solr resources and newsletter: http://www.solr-start.com/ and @solrstart >>> Solr popularizers community: https://www.linkedin.com/groups?gid=6713853 >>> >>> -- >>> You received this message because you are subscribed to a topic in the >>> Google Groups "elasticsearch" group. >>> To unsubscribe from this topic, visit >>> https://groups.google.com/d/topic/elasticsearch/H74KAYmGtoc/unsubscribe. >>> To unsubscribe from this group and all its topics, send an email to >>> elasticsearc...@googlegroups.com <javascript:>. >>> To view this discussion on the web visit >>> https://groups.google.com/d/msgid/elasticsearch/CAEFAe-G8GsLZpeO9-b5_RGs%3D-tGBNp39QgeY2rywjRnOQfcfnw%40mail.gmail.com >>> . >>> For more options, visit https://groups.google.com/d/optout. >>> >> -- >> You received this message because you are subscribed to the Google Groups >> "elasticsearch" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to elasticsearc...@googlegroups.com <javascript:>. >> To view this discussion on the web visit >> https://groups.google.com/d/msgid/elasticsearch/CAJLGCR1LRDnv9Nw71py%3DPFksSqLnXOG6x6-K0kUW1j26%3D6_pYA%40mail.gmail.com >> >> <https://groups.google.com/d/msgid/elasticsearch/CAJLGCR1LRDnv9Nw71py%3DPFksSqLnXOG6x6-K0kUW1j26%3D6_pYA%40mail.gmail.com?utm_medium=email&utm_source=footer> >> . >> >> For more options, visit https://groups.google.com/d/optout. >> > > -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/d85eb58f-2c9d-4fba-88ca-55b64a74be3a%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
