[JIRA] [core] (JENKINS-16301) Fingerprint performance

Tue, 28 May 2013 13:29:05 -0700

Jesse Glick commented on Bug JENKINS-16301

New baseline times are a bit shorter: 0.942msec to save. Using a hand-written serializer, I can get that down to 0.7601msec, a 19% savings. Not exactly dramatic but this is based on a single thread doing serial operations; the hand-written version does not contend locks.

So to check that, changed the test program to run under concurrency stress (each thread saving its own Fingerprint over and over), and also using a tmpfs so that actual I/O time can be ignored:

package hudson.model;
import hudson.Util;
import java.io.File;
import java.io.IOException;
import java.util.ArrayList;
import java.util.List;
import java.util.concurrent.atomic.AtomicInteger;
import java.util.concurrent.atomic.AtomicLong;
public class Fingerprints {
    private static byte[] toByteArray(String md5sum) {
        byte[] data = "" class="code-keyword">new byte[16];
        for( int i=0; i<md5sum.length(); i+=2 )
            data[i/2] = (byte)Integer.parseInt(md5sum.substring(i,i+2),16);
        return data;
    }
    private static final byte[] SOME_MD5 = toByteArray(Util.getDigestOf("whatever"));
    public static void main(String[] args) throws InterruptedException {
        for (int concurrency = 1; concurrency <= 10; concurrency++) {
            List<Thread> threads = new ArrayList<Thread>();
            final AtomicLong totalTime = new AtomicLong();
            final AtomicInteger totalCount = new AtomicInteger();
            for (int i = 0; i < concurrency; i++) {
                final File x = new File("/run/user/" + System.getenv("USER") + "/fp" + i + ".xml");
                Thread t = new Thread("saver-" + i) {
                    @Override public void run() {
                        Fingerprint f = new Fingerprint(new Fingerprint.BuildPtr("foo", 13), "stuff.jar", SOME_MD5);
                        f.addWithoutSaving("some", 1);
                        f.addWithoutSaving("some", 2);
                        f.addWithoutSaving("some", 3);
                        f.addWithoutSaving("some", 10);
                        f.addWithoutSaving("other", 6);
                        for (int i = 0; i < 100; i++) {
                            for (int b = 0; b < 100; b += 2) {
                                f.addWithoutSaving("job" + i, b);
                            }
                        }
                        int count = 10000;
                        long start = System.currentTimeMillis();
                        for (int i = 0; i < count; i++) {
                            try {
                                f.save(x);
                            } catch (IOException e) {
                                assert false : e;
                            }
                        }
                        long end = System.currentTimeMillis();
                        totalTime.addAndGet(end - start);
                        totalCount.addAndGet(count);
                    }
                };
                t.start();
                threads.add(t);
            }
            for (Thread t : threads) {
                t.join();
            }
            System.out.println(concurrency + "×: " + (totalTime.floatValue() / totalCount.floatValue()) + "msec to save");
        }
    }
}


Baseline times (on a quad-core):





1×: 0.7479msec to save
2×: 0.6742msec to save
3×: 0.8395333msec to save
4×: 1.042125msec to save
5×: 1.2944msec to save
6×: 1.5228167msec to save
7×: 1.7666msec to save
8×: 2.0307374msec to save
9×: 2.297811msec to save
10×: 2.55688msec to save





With patch:





1×: 0.2807msec to save
2×: 0.234msec to save
3×: 0.26313335msec to save
4×: 0.329525msec to save
5×: 0.41194msec to save
6×: 0.49443334msec to save
7×: 0.56754285msec to save
8×: 0.650325msec to save
9×: 0.7313111msec to save
10×: 0.81052msec to save





This shows a 2–3× speedup in CPU time, with the biggest gains under concurrent load. I am going to stop there for now; not bothering to optimize load which is I think called rather less often.
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators.
For more information on JIRA, see: http://www.atlassian.com/software/jira

--
You received this message because you are subscribed to the Google Groups "Jenkins Issues" group.
To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-issues+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
 
 

Reply via email to