[
https://issues.apache.org/jira/browse/HADOOP-8648?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13442668#comment-13442668
]
Colin Patrick McCabe commented on HADOOP-8648:
----------------------------------------------
The bug in the inline assembly is that there are extra parameters. You can see
this clearly in the patch:
{code}
@@ -433,7 +456,7 @@ static void pipelined_crc32c(uint32_t *crc1, uint32_t
*crc2, uint32_t *crc3, con
"crc32b (%5), %0;\n\t"
"crc32b (%5,%4,1), %1;\n\t"
: "=r"(c1), "=r"(c2)
- : "r"(c1), "r"(c2), "r"(c3), "r"(block_size), "r"(bdata)
+ : "r"(c1), "r"(c2), "r"(block_size), "r"(bdata)
);
bdata++;
remainder--;
{code}
You can see that it doesn't make sense for the assembly to have 7 parameters,
because only 6 are actually used. And indeed, the fact that 'c3' is inserted
in the parameter list is the bug. Another thing to keep in mind is that c3 is
actually uninitialized at this point-- another clue that we should not be using
it.
Incidentally, this patch unconditionally initializes c3 to 0xffffffff just to
avoid heisenbugs in the future.
> libhadoop: native CRC32 validation crashes when io.bytes.per.checksum=1
> ------------------------------------------------------------------------
>
> Key: HADOOP-8648
> URL: https://issues.apache.org/jira/browse/HADOOP-8648
> Project: Hadoop Common
> Issue Type: Bug
> Affects Versions: 2.0.0-alpha
> Reporter: Colin Patrick McCabe
> Assignee: Colin Patrick McCabe
> Attachments: HADOOP-8648.001.patch, HADOOP-8648.002.patch,
> HADOOP-8648.003.patch
>
>
> The native CRC32 code, found in {{pipelined_crc32c}}, crashes when blocksize
> is set to 1.
> {code}
> 12:27:14,886 INFO NativeCodeLoader:50 - Loaded the native-hadoop library
> #
> # A fatal error has been detected by the Java Runtime Environment:
> #
> # SIGSEGV (0xb) at pc=0x00007fa00ee5a340, pid=24100, tid=140326058854144
> #
> # JRE version: 6.0_29-b11
> # Java VM: Java HotSpot(TM) 64-Bit Server VM (20.4-b02 mixed mode linux-amd64
> compressed oops)
> # Problematic frame:
> # C [libhadoop.so.1.0.0+0x8340] pipelined_crc32c+0xa0
> #
> # An error report file with more information is saved as:
> # /h/hs_err_pid24100.log
> #
> # If you would like to submit a bug report, please visit:
> # http://java.sun.com/webapps/bugreport/crash.jsp
> #
> Aborted
> {code}
> The Java CRC code works fine in this case.
> Choosing blocksize=1 is a __very__ odd choice. It means that we're storing a
> 4-byte checksum for every byte.
> {code}
> -rw-r--r-- 1 cmccabe users 49398 Aug 3 11:33 blk_4702510289566780538
> -rw-r--r-- 1 cmccabe users 197599 Aug 3 11:33
> blk_4702510289566780538_1199.meta
> {code}
> However, obviously crashing is never the right thing to do.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
