Is it reasonable to compare outputs between JMH and hprof?

[Wang Weijun]

Hi All

I am comparing the difference of SHA-1 and SHA-256. First I wrote a JMH 
benchmark:

@Benchmark
public void sig1(Blackhole bh) throws Exception {
    bh.consume(sig("SHA-1"));
}

@Benchmark
public void sig2(Blackhole bh) throws Exception {
    bh.consume(sig("SHA-256"));
}

byte[] sig(String alg) throws Exception {
    MessageDigest md = MessageDigest.getInstance(alg);
    md.update(new byte[10000]);
    return md.digest();
}

The output is

Benchmark            Mode  Samples      Score      Error  Units
o.o.b.Weird.sig1    thrpt        5  20984.435 ± 3356.455  ops/s
o.o.b.Weird.sig2    thrpt        5  13130.330 ±  976.824  ops/s

so the difference is there but not huge.

Then I wrote a simple app with

public static void main(String args[]) throws Exception {
    int i = Arrays.hashCode(sig("SHA-1"));
    i += Arrays.hashCode(sig("SHA-256"));
    System.out.println(i);
}

static byte[] sig(String alg) throws Exception {
    MessageDigest md = MessageDigest.getInstance(alg);
    md.update(new byte[10000]);
    return md.digest();
}

and then profile it with -agentlib:hprof=cpu=times, and get

SHA2    1 10.16% 10.16%     156 303276 sun.security.provider.SHA2.implCompress
SHA2    2  6.91% 17.07%    9984 303274 sun.security.provider.SHA2.lf_sigma0
SHA2    3  5.28% 22.36%    9984 303271 sun.security.provider.SHA2.lf_sigma1
SHA2    4  4.61% 26.96%    7488 303269 sun.security.provider.SHA2.lf_delta0
SHA2    5  4.20% 31.17%   29952 303273 sun.security.provider.SHA2.lf_S
SHA2    7  3.79% 39.16%    7488 303266 sun.security.provider.SHA2.lf_delta1
SHA2    9  2.85% 44.99%   29952 303270 sun.security.provider.SHA2.lf_S
SHA2   13  1.90% 54.47%   14976 303267 sun.security.provider.SHA2.lf_S
SHA2   17  1.49% 61.25%   14976 303264 sun.security.provider.SHA2.lf_S
SHA2   22  0.81% 66.12%    7488 303265 sun.security.provider.SHA2.lf_R
SHA2   23  0.81% 66.94%    9984 303275 sun.security.provider.SHA2.lf_maj
SHA2   25  0.81% 68.56%     156 303263 
sun.security.provider.ByteArrayAccess.b2iBig64
SHA2   27  0.68% 70.05%    9984 303272 sun.security.provider.SHA2.lf_ch
SHA2   31  0.54% 72.63%    7488 303268 sun.security.provider.SHA2.lf_R
SHA1   34  0.54% 74.25%     156 303224 sun.security.provider.SHA.implCompress
SHA1   43  0.41% 78.05%     156 303223 
sun.security.provider.ByteArrayAccess.b2iBig64
SHA2   60  0.27% 82.66%    2496 303262 java.lang.Integer.reverseBytes
SHA2   61  0.27% 82.93%      64 303290 sun.security.provider.SHA2.lf_sigma1
SHA1  116  0.14% 91.06%    2496 303222 java.lang.Integer.reverseBytes

These are calls with SHA (i.e. SHA1) or SHA2 in the stack (depth=4), and time 
for SHA2 vs SHA1 is 45.38% vs 1.09%. With such a small app I don't think SHA or 
SHA2 is called anywhere else. This is jdk9 b40.

Why is the output so different from JMH? Is it reasonable comparing them?

Thanks
Max