[ https://issues.apache.org/jira/browse/HADOOP-16601?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16955101#comment-16955101 ]
MacChen01 edited comment on HADOOP-16601 at 10/19/19 7:42 AM: -------------------------------------------------------------- [~ste...@apache.org] Thanks~ * Modify the patch according the Yetus's suggestion. * Run the native tests on arm64 , the execute file is :hadoop-rel-release-3.1.1/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-nativetask/target/native/test/nttest, attaching the results file: [^aarch64_nativetask_test.log] * The issue is optimized for performance, adding new tests in test_bulk_crc32.c if need it. * Testing the hadoop test case (include: terasort/wordcount/kmeans..) with the test tool HiBench-HiBench-7.0 , all testcases had passed. was (Author: macchen01): [~ste...@apache.org] Thanks~ * Modify the patch according the Yetus's suggestion. * Run the native tests on arm64 , the execute file is :hadoop-rel-release-3.1.1/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-nativetask/target/native/test/nttest, attaching the results [file:[|file://[/]^aarch64_nativetask_test.log] * The issue is optimized for performance, adding new tests in test_bulk_crc32.c if need it. * Testing the hadoop test case (include: terasort/wordcount/kmeans..) with the test tool HiBench-HiBench-7.0 , all testcases had passed. > Add support for hardware crc32 of nativetask checksums on aarch64 arch > ---------------------------------------------------------------------- > > Key: HADOOP-16601 > URL: https://issues.apache.org/jira/browse/HADOOP-16601 > Project: Hadoop Common > Issue Type: Improvement > Components: common > Affects Versions: 3.2.1 > Reporter: MacChen01 > Priority: Major > Labels: performance > Fix For: site > > Attachments: HADOOP-16601.patch, aarch64_nativetask_test.log, > aarch64_nativetask_test.log > > > Add support for aarch64 CRC instructions in nativetask module, optimize the > CRC32 and CRC32C. > Use the benchmark tools : nttest , the improvement is quite substantial: > *CRC32 Zlib polynomial 0x04C11DB7* > |KeyValueType-IO|Before(MB/s)|After(MB/s)|Improvement| > |TextType-Write|425.98|602.92|+42%| > |TextType-Read|796.06|1716.59|+116%| > |BytesType-Write|474.25|686.84|+45%| > |BytesType-Read|844.96|1955.03|+131%| > |UnknownType-Write|434.84|608.81|+40%| > |UnknownType-Read|805.76|1733.82|+115%| > > > *CRC32C Castagnoli polynomial 0x1EDC6F41* > > |KeyValueType-IO|Before(MB/s)|After(MB/s)|Improvement| > |TextType-Write|423.39|606.55|+43%| > |TextType-Read|799.20|1783.28|+123%| > |BytesType-Write|473.95|696.47|+47%| > |BytesType-Read|846.30|2018.06|+138%| > |UnknownType-Write|434.07|612.31|+41%| > |UnknownType-Read|807.16|1783.95|+121%| -- This message was sent by Atlassian Jira (v8.3.4#803005) --------------------------------------------------------------------- To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org