[jira] [Updated] (HDFS-17298) Fix NPE in DataNode.handleBadBlock and BlockSender
[
https://issues.apache.org/jira/browse/HDFS-17298?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Shilun Fan updated HDFS-17298:
--
Hadoop Flags: Reviewed
Target Version/s: 3.4.0, 3.3.9
Affects Version/s: 3.4.0
3.3.9
> Fix NPE in DataNode.handleBadBlock and BlockSender
> --
>
> Key: HDFS-17298
> URL: https://issues.apache.org/jira/browse/HDFS-17298
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: datanode
>Affects Versions: 3.4.0, 3.3.9
>Reporter: Haiyang Hu
>Assignee: Haiyang Hu
>Priority: Major
> Labels: pull-request-available
> Fix For: 3.4.0, 3.3.9
>
>
> There are some NPE issues on the DataNode side of our online environment.
> The detailed exception information is
> {code:java}
> 2023-12-20 13:58:25,449 ERROR datanode.DataNode (DataXceiver.java:run(330))
> [DataXceiver for client DFSClient_NONMAPREDUCE_xxx at /xxx:41452 [Sending
> block BP-xxx:blk_xxx]] - xxx:50010:DataXceiver error processing READ_BLOCK
> operation src: /xxx:41452 dst: /xxx:50010
> java.lang.NullPointerException
> at
> org.apache.hadoop.hdfs.server.datanode.BlockSender.(BlockSender.java:301)
> at
> org.apache.hadoop.hdfs.server.datanode.DataXceiver.readBlock(DataXceiver.java:607)
> at
> org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.opReadBlock(Receiver.java:152)
> at
> org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.processOp(Receiver.java:104)
> at
> org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:298)
> at java.lang.Thread.run(Thread.java:748)
> {code}
> NPE Code logic:
> {code:java}
> if (!fromScanner && blockScanner.isEnabled()) {
> // data.getVolume(block) is null
> blockScanner.markSuspectBlock(data.getVolume(block).getStorageID(),
> block);
> }
> {code}
> {code:java}
> 2023-12-20 13:52:18,844 ERROR datanode.DataNode (DataXceiver.java:run(330))
> [DataXceiver for client /xxx:61052 [Copying block BP-xxx:blk_xxx]] -
> xxx:50010:DataXceiver error processing COPY_BLOCK operation src: /xxx:61052
> dst: /xxx:50010
> java.lang.NullPointerException
> at
> org.apache.hadoop.hdfs.server.datanode.DataNode.handleBadBlock(DataNode.java:4045)
> at
> org.apache.hadoop.hdfs.server.datanode.DataXceiver.copyBlock(DataXceiver.java:1163)
> at
> org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.opCopyBlock(Receiver.java:291)
> at
> org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.processOp(Receiver.java:113)
> at
> org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:298)
> at java.lang.Thread.run(Thread.java:748)
> {code}
> NPE Code logic:
> {code:java}
> // Obtain a reference before reading data
> volumeRef = datanode.data.getVolume(block).obtainReference();
> //datanode.data.getVolume(block) is null
> {code}
> We need to fix it.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-17298) Fix NPE in DataNode.handleBadBlock and BlockSender
[
https://issues.apache.org/jira/browse/HDFS-17298?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Takanobu Asanuma updated HDFS-17298:
Fix Version/s: 3.3.9
> Fix NPE in DataNode.handleBadBlock and BlockSender
> --
>
> Key: HDFS-17298
> URL: https://issues.apache.org/jira/browse/HDFS-17298
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: datanode
>Reporter: Haiyang Hu
>Assignee: Haiyang Hu
>Priority: Major
> Labels: pull-request-available
> Fix For: 3.4.0, 3.3.9
>
>
> There are some NPE issues on the DataNode side of our online environment.
> The detailed exception information is
> {code:java}
> 2023-12-20 13:58:25,449 ERROR datanode.DataNode (DataXceiver.java:run(330))
> [DataXceiver for client DFSClient_NONMAPREDUCE_xxx at /xxx:41452 [Sending
> block BP-xxx:blk_xxx]] - xxx:50010:DataXceiver error processing READ_BLOCK
> operation src: /xxx:41452 dst: /xxx:50010
> java.lang.NullPointerException
> at
> org.apache.hadoop.hdfs.server.datanode.BlockSender.(BlockSender.java:301)
> at
> org.apache.hadoop.hdfs.server.datanode.DataXceiver.readBlock(DataXceiver.java:607)
> at
> org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.opReadBlock(Receiver.java:152)
> at
> org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.processOp(Receiver.java:104)
> at
> org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:298)
> at java.lang.Thread.run(Thread.java:748)
> {code}
> NPE Code logic:
> {code:java}
> if (!fromScanner && blockScanner.isEnabled()) {
> // data.getVolume(block) is null
> blockScanner.markSuspectBlock(data.getVolume(block).getStorageID(),
> block);
> }
> {code}
> {code:java}
> 2023-12-20 13:52:18,844 ERROR datanode.DataNode (DataXceiver.java:run(330))
> [DataXceiver for client /xxx:61052 [Copying block BP-xxx:blk_xxx]] -
> xxx:50010:DataXceiver error processing COPY_BLOCK operation src: /xxx:61052
> dst: /xxx:50010
> java.lang.NullPointerException
> at
> org.apache.hadoop.hdfs.server.datanode.DataNode.handleBadBlock(DataNode.java:4045)
> at
> org.apache.hadoop.hdfs.server.datanode.DataXceiver.copyBlock(DataXceiver.java:1163)
> at
> org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.opCopyBlock(Receiver.java:291)
> at
> org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.processOp(Receiver.java:113)
> at
> org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:298)
> at java.lang.Thread.run(Thread.java:748)
> {code}
> NPE Code logic:
> {code:java}
> // Obtain a reference before reading data
> volumeRef = datanode.data.getVolume(block).obtainReference();
> //datanode.data.getVolume(block) is null
> {code}
> We need to fix it.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-17298) Fix NPE in DataNode.handleBadBlock and BlockSender
[
https://issues.apache.org/jira/browse/HDFS-17298?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Haiyang Hu updated HDFS-17298:
--
Component/s: datanode
> Fix NPE in DataNode.handleBadBlock and BlockSender
> --
>
> Key: HDFS-17298
> URL: https://issues.apache.org/jira/browse/HDFS-17298
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: datanode
>Reporter: Haiyang Hu
>Assignee: Haiyang Hu
>Priority: Major
> Labels: pull-request-available
>
> There are some NPE issues on the DataNode side of our online environment.
> The detailed exception information is
> {code:java}
> 2023-12-20 13:58:25,449 ERROR datanode.DataNode (DataXceiver.java:run(330))
> [DataXceiver for client DFSClient_NONMAPREDUCE_xxx at /xxx:41452 [Sending
> block BP-xxx:blk_xxx]] - xxx:50010:DataXceiver error processing READ_BLOCK
> operation src: /xxx:41452 dst: /xxx:50010
> java.lang.NullPointerException
> at
> org.apache.hadoop.hdfs.server.datanode.BlockSender.(BlockSender.java:301)
> at
> org.apache.hadoop.hdfs.server.datanode.DataXceiver.readBlock(DataXceiver.java:607)
> at
> org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.opReadBlock(Receiver.java:152)
> at
> org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.processOp(Receiver.java:104)
> at
> org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:298)
> at java.lang.Thread.run(Thread.java:748)
> {code}
> NPE Code logic:
> {code:java}
> if (!fromScanner && blockScanner.isEnabled()) {
> // data.getVolume(block) is null
> blockScanner.markSuspectBlock(data.getVolume(block).getStorageID(),
> block);
> }
> {code}
> {code:java}
> 2023-12-20 13:52:18,844 ERROR datanode.DataNode (DataXceiver.java:run(330))
> [DataXceiver for client /xxx:61052 [Copying block BP-xxx:blk_xxx]] -
> xxx:50010:DataXceiver error processing COPY_BLOCK operation src: /xxx:61052
> dst: /xxx:50010
> java.lang.NullPointerException
> at
> org.apache.hadoop.hdfs.server.datanode.DataNode.handleBadBlock(DataNode.java:4045)
> at
> org.apache.hadoop.hdfs.server.datanode.DataXceiver.copyBlock(DataXceiver.java:1163)
> at
> org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.opCopyBlock(Receiver.java:291)
> at
> org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.processOp(Receiver.java:113)
> at
> org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:298)
> at java.lang.Thread.run(Thread.java:748)
> {code}
> NPE Code logic:
> {code:java}
> // Obtain a reference before reading data
> volumeRef = datanode.data.getVolume(block).obtainReference();
> //datanode.data.getVolume(block) is null
> {code}
> We need to fix it.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-17298) Fix NPE in DataNode.handleBadBlock and BlockSender
[
https://issues.apache.org/jira/browse/HDFS-17298?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ASF GitHub Bot updated HDFS-17298:
--
Labels: pull-request-available (was: )
> Fix NPE in DataNode.handleBadBlock and BlockSender
> --
>
> Key: HDFS-17298
> URL: https://issues.apache.org/jira/browse/HDFS-17298
> Project: Hadoop HDFS
> Issue Type: Bug
>Reporter: Haiyang Hu
>Assignee: Haiyang Hu
>Priority: Major
> Labels: pull-request-available
>
> There are some NPE issues on the DataNode side of our online environment.
> The detailed exception information is
> {code:java}
> 2023-12-20 13:58:25,449 ERROR datanode.DataNode (DataXceiver.java:run(330))
> [DataXceiver for client DFSClient_NONMAPREDUCE_xxx at /xxx:41452 [Sending
> block BP-xxx:blk_xxx]] - xxx:50010:DataXceiver error processing READ_BLOCK
> operation src: /xxx:41452 dst: /xxx:50010
> java.lang.NullPointerException
> at
> org.apache.hadoop.hdfs.server.datanode.BlockSender.(BlockSender.java:301)
> at
> org.apache.hadoop.hdfs.server.datanode.DataXceiver.readBlock(DataXceiver.java:607)
> at
> org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.opReadBlock(Receiver.java:152)
> at
> org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.processOp(Receiver.java:104)
> at
> org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:298)
> at java.lang.Thread.run(Thread.java:748)
> {code}
> NPE Code logic:
> {code:java}
> if (!fromScanner && blockScanner.isEnabled()) {
> // data.getVolume(block) is null
> blockScanner.markSuspectBlock(data.getVolume(block).getStorageID(),
> block);
> }
> {code}
> {code:java}
> 2023-12-20 13:52:18,844 ERROR datanode.DataNode (DataXceiver.java:run(330))
> [DataXceiver for client /xxx:61052 [Copying block BP-xxx:blk_xxx]] -
> xxx:50010:DataXceiver error processing COPY_BLOCK operation src: /xxx:61052
> dst: /xxx:50010
> java.lang.NullPointerException
> at
> org.apache.hadoop.hdfs.server.datanode.DataNode.handleBadBlock(DataNode.java:4045)
> at
> org.apache.hadoop.hdfs.server.datanode.DataXceiver.copyBlock(DataXceiver.java:1163)
> at
> org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.opCopyBlock(Receiver.java:291)
> at
> org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.processOp(Receiver.java:113)
> at
> org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:298)
> at java.lang.Thread.run(Thread.java:748)
> {code}
> NPE Code logic:
> {code:java}
> // Obtain a reference before reading data
> volumeRef = datanode.data.getVolume(block).obtainReference();
> //datanode.data.getVolume(block) is null
> {code}
> We need to fix it.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-17298) Fix NPE in DataNode.handleBadBlock and BlockSender
[
https://issues.apache.org/jira/browse/HDFS-17298?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Haiyang Hu updated HDFS-17298:
--
Description:
There are some NPE issues on the DataNode side of our online environment.
The detailed exception information is
{code:java}
2023-12-20 13:58:25,449 ERROR datanode.DataNode (DataXceiver.java:run(330))
[DataXceiver for client DFSClient_NONMAPREDUCE_xxx at /xxx:41452 [Sending block
BP-xxx:blk_xxx]] - xxx:50010:DataXceiver error processing READ_BLOCK operation
src: /xxx:41452 dst: /xxx:50010
java.lang.NullPointerException
at
org.apache.hadoop.hdfs.server.datanode.BlockSender.(BlockSender.java:301)
at
org.apache.hadoop.hdfs.server.datanode.DataXceiver.readBlock(DataXceiver.java:607)
at
org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.opReadBlock(Receiver.java:152)
at
org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.processOp(Receiver.java:104)
at
org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:298)
at java.lang.Thread.run(Thread.java:748)
{code}
NPE Code logic:
{code:java}
if (!fromScanner && blockScanner.isEnabled()) {
// data.getVolume(block) is null
blockScanner.markSuspectBlock(data.getVolume(block).getStorageID(),
block);
}
{code}
{code:java}
2023-12-20 13:52:18,844 ERROR datanode.DataNode (DataXceiver.java:run(330))
[DataXceiver for client /xxx:61052 [Copying block BP-xxx:blk_xxx]] -
xxx:50010:DataXceiver error processing COPY_BLOCK operation src: /xxx:61052
dst: /xxx:50010
java.lang.NullPointerException
at
org.apache.hadoop.hdfs.server.datanode.DataNode.handleBadBlock(DataNode.java:4045)
at
org.apache.hadoop.hdfs.server.datanode.DataXceiver.copyBlock(DataXceiver.java:1163)
at
org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.opCopyBlock(Receiver.java:291)
at
org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.processOp(Receiver.java:113)
at
org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:298)
at java.lang.Thread.run(Thread.java:748)
{code}
NPE Code logic:
{code:java}
// Obtain a reference before reading data
volumeRef = datanode.data.getVolume(block).obtainReference();
//datanode.data.getVolume(block) is null
{code}
We need to fix it.
> Fix NPE in DataNode.handleBadBlock and BlockSender
> --
>
> Key: HDFS-17298
> URL: https://issues.apache.org/jira/browse/HDFS-17298
> Project: Hadoop HDFS
> Issue Type: Bug
>Reporter: Haiyang Hu
>Assignee: Haiyang Hu
>Priority: Major
>
> There are some NPE issues on the DataNode side of our online environment.
> The detailed exception information is
> {code:java}
> 2023-12-20 13:58:25,449 ERROR datanode.DataNode (DataXceiver.java:run(330))
> [DataXceiver for client DFSClient_NONMAPREDUCE_xxx at /xxx:41452 [Sending
> block BP-xxx:blk_xxx]] - xxx:50010:DataXceiver error processing READ_BLOCK
> operation src: /xxx:41452 dst: /xxx:50010
> java.lang.NullPointerException
> at
> org.apache.hadoop.hdfs.server.datanode.BlockSender.(BlockSender.java:301)
> at
> org.apache.hadoop.hdfs.server.datanode.DataXceiver.readBlock(DataXceiver.java:607)
> at
> org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.opReadBlock(Receiver.java:152)
> at
> org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.processOp(Receiver.java:104)
> at
> org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:298)
> at java.lang.Thread.run(Thread.java:748)
> {code}
> NPE Code logic:
> {code:java}
> if (!fromScanner && blockScanner.isEnabled()) {
> // data.getVolume(block) is null
> blockScanner.markSuspectBlock(data.getVolume(block).getStorageID(),
> block);
> }
> {code}
> {code:java}
> 2023-12-20 13:52:18,844 ERROR datanode.DataNode (DataXceiver.java:run(330))
> [DataXceiver for client /xxx:61052 [Copying block BP-xxx:blk_xxx]] -
> xxx:50010:DataXceiver error processing COPY_BLOCK operation src: /xxx:61052
> dst: /xxx:50010
> java.lang.NullPointerException
> at
> org.apache.hadoop.hdfs.server.datanode.DataNode.handleBadBlock(DataNode.java:4045)
> at
> org.apache.hadoop.hdfs.server.datanode.DataXceiver.copyBlock(DataXceiver.java:1163)
> at
> org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.opCopyBlock(Receiver.java:291)
> at
> org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.processOp(Receiver.java:113)
> at
> org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:298)
> at java.lang.Thread.run(Thread.java:748)
> {code}
> NPE Code logic:
> {code:java}
> // Obtain a reference before reading data
> volumeRef = datanode.data.getVolume(block).obtainReference();
> //datanode.data.getVolume(block) is null
>
