Damien Nozay edited a comment on Improvement JENKINS-14923

I've run into this issue where we got 10 nodes with 10 executors each and the same one gets full before it gets executed elsewhere. somehow the contents of the job can kill the agent (OOM), which means it is very bothersome to greedily fill the same node.

my coworker looked up the code:
https://github.com/jenkinsci/jenkins/blob/master/core/src/main/java/hudson/model/LoadBalancer.java

LoadBalancer.java
@Override
        public Mapping map(Task task, MappingWorksheet ws) {
            // build consistent hash for each work chunk
            List<ConsistentHash<ExecutorChunk>> hashes = new ArrayList<ConsistentHash<ExecutorChunk>>(ws.works.size());
            for (int i=0; i<ws.works.size(); i++) {
                ConsistentHash<ExecutorChunk> hash = new ConsistentHash<ExecutorChunk>(new Hash<ExecutorChunk>() {
                    public String hash(ExecutorChunk node) {
                        return node.getName();
                    }
                });
                for (ExecutorChunk ec : ws.works(i).applicableExecutorChunks())
                    hash.add(ec,ec.size()*100);

                hashes.add(hash);
            }

            // do a greedy assignment
            Mapping m = ws.new Mapping();
            assert m.size()==ws.works.size();   // just so that you the reader of the source code don't get confused with the for loop index

            if (assignGreedily(m,task,hashes,0)) {
                assert m.isCompletelyValid();
                return m;
            } else
                return null;
        }

        private boolean assignGreedily(Mapping m, Task task, List<ConsistentHash<ExecutorChunk>> hashes, int i) {
            if (i==hashes.size())   return true;    // fully assigned

            String key = task.getFullDisplayName() + (i>0 ? String.valueOf(i) : "");

            for (ExecutorChunk ec : hashes.get(i).list(key)) {
                // let's attempt this assignment
                m.assign(i,ec);

                if (m.isPartiallyValid() && assignGreedily(m,task,hashes,i+1))
                    return true;    // successful greedily allocation

                // otherwise 'ec' wasn't a good fit for us. try next.
            }

            // every attempt failed
            m.assign(i,null);
            return false;
        }
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators.
For more information on JIRA, see: http://www.atlassian.com/software/jira

Reply via email to