[ https://issues.apache.org/jira/browse/HDFS-16544?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
qinyuren updated HDFS-16544: ---------------------------- Description: In [HDFS-16538|http://https//issues.apache.org/jira/browse/HDFS-16538] , we found a ec decoding bug if more than one data block read failed. Currently, we found another bug trigger by #StatefulStripeReader.decode. If we read an EC file which {*}length more than one stripe{*}, and this file have *one data block* and *the first parity block* corrupted, this error will happen. {code:java} org.apache.hadoop.HadoopIllegalArgumentException: Invalid buffer found, not allowing null at org.apache.hadoop.io.erasurecode.rawcoder.ByteBufferDecodingState.checkOutputBuffers(ByteBufferDecodingState.java:132) at org.apache.hadoop.io.erasurecode.rawcoder.ByteBufferDecodingState.<init>(ByteBufferDecodingState.java:48) at org.apache.hadoop.io.erasurecode.rawcoder.RawErasureDecoder.decode(RawErasureDecoder.java:86) at org.apache.hadoop.io.erasurecode.rawcoder.RawErasureDecoder.decode(RawErasureDecoder.java:170) at org.apache.hadoop.hdfs.StripeReader.decodeAndFillBuffer(StripeReader.java:435) at org.apache.hadoop.hdfs.StatefulStripeReader.decode(StatefulStripeReader.java:94) at org.apache.hadoop.hdfs.StripeReader.readStripe(StripeReader.java:392) at org.apache.hadoop.hdfs.DFSStripedInputStream.readOneStripe(DFSStripedInputStream.java:315) at org.apache.hadoop.hdfs.DFSStripedInputStream.readWithStrategy(DFSStripedInputStream.java:408) at org.apache.hadoop.hdfs.DFSInputStream.read(DFSInputStream.java:918) {code} was: In [HDFS-16538|http://https://issues.apache.org/jira/browse/HDFS-16538] , we found a ec decoding bug if more than one data block read failed. Currently, we found another bug trigger by #StatefulStripeReader.decode. If we read an EC file which length more than one stripe, and this file have one data block and the first parity block corrupted, this error will happen. {code:java} org.apache.hadoop.HadoopIllegalArgumentException: Invalid buffer found, not allowing null at org.apache.hadoop.io.erasurecode.rawcoder.ByteBufferDecodingState.checkOutputBuffers(ByteBufferDecodingState.java:132) at org.apache.hadoop.io.erasurecode.rawcoder.ByteBufferDecodingState.<init>(ByteBufferDecodingState.java:48) at org.apache.hadoop.io.erasurecode.rawcoder.RawErasureDecoder.decode(RawErasureDecoder.java:86) at org.apache.hadoop.io.erasurecode.rawcoder.RawErasureDecoder.decode(RawErasureDecoder.java:170) at org.apache.hadoop.hdfs.StripeReader.decodeAndFillBuffer(StripeReader.java:435) at org.apache.hadoop.hdfs.StatefulStripeReader.decode(StatefulStripeReader.java:94) at org.apache.hadoop.hdfs.StripeReader.readStripe(StripeReader.java:392) at org.apache.hadoop.hdfs.DFSStripedInputStream.readOneStripe(DFSStripedInputStream.java:315) at org.apache.hadoop.hdfs.DFSStripedInputStream.readWithStrategy(DFSStripedInputStream.java:408) at org.apache.hadoop.hdfs.DFSInputStream.read(DFSInputStream.java:918) {code} > EC decoding failed due to invalid buffer > ---------------------------------------- > > Key: HDFS-16544 > URL: https://issues.apache.org/jira/browse/HDFS-16544 > Project: Hadoop HDFS > Issue Type: Bug > Reporter: qinyuren > Priority: Major > > In [HDFS-16538|http://https//issues.apache.org/jira/browse/HDFS-16538] , we > found a ec decoding bug if more than one data block read failed. > Currently, we found another bug trigger by #StatefulStripeReader.decode. > If we read an EC file which {*}length more than one stripe{*}, and this file > have *one data block* and *the first parity block* corrupted, this error will > happen. > {code:java} > org.apache.hadoop.HadoopIllegalArgumentException: Invalid buffer found, not > allowing null at > org.apache.hadoop.io.erasurecode.rawcoder.ByteBufferDecodingState.checkOutputBuffers(ByteBufferDecodingState.java:132) > at > org.apache.hadoop.io.erasurecode.rawcoder.ByteBufferDecodingState.<init>(ByteBufferDecodingState.java:48) > at > org.apache.hadoop.io.erasurecode.rawcoder.RawErasureDecoder.decode(RawErasureDecoder.java:86) > at > org.apache.hadoop.io.erasurecode.rawcoder.RawErasureDecoder.decode(RawErasureDecoder.java:170) > at > org.apache.hadoop.hdfs.StripeReader.decodeAndFillBuffer(StripeReader.java:435) > at > org.apache.hadoop.hdfs.StatefulStripeReader.decode(StatefulStripeReader.java:94) > at org.apache.hadoop.hdfs.StripeReader.readStripe(StripeReader.java:392) > at > org.apache.hadoop.hdfs.DFSStripedInputStream.readOneStripe(DFSStripedInputStream.java:315) > at > org.apache.hadoop.hdfs.DFSStripedInputStream.readWithStrategy(DFSStripedInputStream.java:408) > at org.apache.hadoop.hdfs.DFSInputStream.read(DFSInputStream.java:918) > {code} -- This message was sent by Atlassian Jira (v8.20.1#820001) --------------------------------------------------------------------- To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org