So it sounds like what's going to have the biggest effect is making sure the problematic mapreduce tasks run at a lower priority. I'd definitely be interested in your results of you experiment with this.
On Tue, Aug 18, 2015, 17:33 Jeff Kubina <[email protected]> wrote: > > I haven't heard of anybody setting the niceness of the Accumulo processes >> > before. Are you experiencing a lot of CPU contention on your nodes, such >> > that you need to prioritize processes? >> > > Yes, we have had mapreduce jobs "lock out" the tserver so long that the > master removes them from its pool for not responding. But also when doing a > query from another service while the mapreduce jobs are maxed out I want to > make sure the tserver takes priority. > > >> > However, if you do this, I think you actually want HDFS to -2, Accumulo >> to -1, >> > and Mapreduce tasks 0 (you'll have to edit the system security limits >> to allow >> > niceness below 0 for non-root users). >> > Otherwise, you're lowering the priority, not increasing it (and >> lowering it >> > below other, normal user processes, since the default niceness is 0). >> > > Agreed. That is what I wanted to do originally but it requires much more > reconfiguration. Ideally all OS root processes currently at 0 get reniced > to -2, HDFS to -2, Accumulo to -2, and mapreduce to -1 and all other user > processes stay at 0. Given the amount of threading a datanode does by > default it could lockout a tserver if its priority was lower, so I think > the HDFS and Accumulo should have equal priority. > >
