Currently, in our production cluster, almost all of the traffic for a
day ends up assigned to a single RS and that causes the load on that
machine to be too high.
With our last release, we salted our rowkeys so that rather than
starting with the date:
100617<guid>
they now start with the first letter of the guid followed by the date:
e100617<guid_that_starts_with_e>
When I look at the region assignments though, I see a single server
assigned the following regions:
0100617...
1100617...
2100617...
3100617...
4100617...
...
d100617...
e100617...
f100617...
Is there anything we can do to try to get the cluster to shuffle this up
some more?
We are getting compaction times in the minutes (one I saw was over 12
minutes) and this causes our clients to time out and shut down which
causes production outages.
-Daniel