Regarding a 12TB index: Yago Riveiro <yago.rive...@gmail.com> wrote:
> Our cluster is small for the data we hold (12 machines with SSD and 32G of > RAM), but we don't need sub-second queries, we need facet with high > cardinality (in worst case scenarios we aggregate 5M unique string values) > In a peak of inserts we can handle around 25K docs per second without any > issue with 2 replicas and without compromise reads or put a node in stress. > Nodes in stress can eject him selfs from the Zookepeer cluster due a GC or a > lack of CPU to communicate. I am surprised that you manage to have this working on that hardware. As you have replicas, it seems to me that you handle 2*12TB of index with 12*32GB of RAM? This is very close to our setup (22TB of index with 320GB of RAM (updated last week from 256GB) per machine), but we benefit hugely from having a static index. I assume the SSDs are local? How much memory do you use for heap on each machine? - Toke Eskildsen
