Your main concern with greater numbers of partitions is contention for IO (and CPU time for the vnodes to process requests). Consider that with 6 nodes, you will have 170(+/- 1) partitions running on each physical node. That's *at least* 170 files open, all scribbling to and reading from disk. You might get away with 512, but my experience tells me to be wary of that for small clusters. I think unless you're doing big-web-property-level traffic you won't need to grow your cluster as large as you think, especially with machines that beefy.
On Thu, Oct 3, 2013 at 12:46 PM, John Kavanagh <[email protected]>wrote: > Hi Sean , > > What would the perform issues be with setting a ring size to 1024 (instead > of 128 or 256) on a 6 node cluster? > > I can see the possibility of memory overflow (which is not a high concern > on a 128G machine ) , but other than that , are there any additional > concerns (with Bitcask) ? > > I'm trying to understand if there is a significant performance degradation > with a value of 1024 versus future proofing growth. > > Cheers, > > John > > On 3 Oct 2013, at 16:33, Sean Cribbs <[email protected]> wrote: > > Hi John, > > Our rule-of-thumb is 8-64 partitions per physical node, so a good starting > point for a 6-node cluster would be 128 or 256. 256 will let you expand up > to about 30 nodes without needing a ring resize. > > > On Thu, Oct 3, 2013 at 10:14 AM, John Kavanagh <[email protected]> wrote: > >> Hi all, >> >> I'm trying to understand the optimal ring size / vnode config for a 6 >> node Riak cluster, which will most likely expand to many more nodes over >> time . Does everyone bump from the default of 64 to 1024 to facilitate >> future growth ? I can provide more detailed capacity metrics off list if >> necessary. >> >> Thanks, >> >> John >> >> _______________________________________________ >> riak-users mailing list >> [email protected] >> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com >> >> > > > -- > Sean Cribbs <[email protected]> > Software Engineer > Basho Technologies, Inc. > http://basho.com/ > _______________________________________________ > riak-users mailing list > [email protected] > http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com > > > -- Sean Cribbs <[email protected]> Software Engineer Basho Technologies, Inc. http://basho.com/
_______________________________________________ riak-users mailing list [email protected] http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
