[jira] [Commented] (HDFS-8450) Erasure Coding: Consolidate erasure coding zone related implementation into a single class
[
https://issues.apache.org/jira/browse/HDFS-8450?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14938031#comment-14938031
]
Hudson commented on HDFS-8450:
--
FAILURE: Integrated in Hadoop-Hdfs-trunk-Java8 #439 (See
[https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/439/])
HDFS-8450. Erasure Coding: Consolidate erasure coding zone related
(vinayakumarb: rev 98d340745be682fb251677bb4830aca76119868f)
*
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSDirWriteFileOp.java
* hadoop-hdfs-project/hadoop-hdfs/CHANGES-HDFS-EC-7285.txt
*
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NamenodeFsck.java
*
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSDirectory.java
*
hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/NameNodeAdapter.java
*
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NameNodeRpcServer.java
*
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSDirStatAndListingOp.java
*
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSEditLogLoader.java
*
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/ErasureCodingZoneManager.java
*
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java
*
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSDirErasureCodingOp.java
> Erasure Coding: Consolidate erasure coding zone related implementation into a
> single class
> --
>
> Key: HDFS-8450
> URL: https://issues.apache.org/jira/browse/HDFS-8450
> Project: Hadoop HDFS
> Issue Type: Sub-task
>Reporter: Rakesh R
>Assignee: Rakesh R
> Fix For: HDFS-7285
>
> Attachments: HDFS-8450-FYI.patch, HDFS-8450-HDFS-7285-00.patch,
> HDFS-8450-HDFS-7285-01.patch, HDFS-8450-HDFS-7285-02.patch,
> HDFS-8450-HDFS-7285-03.patch, HDFS-8450-HDFS-7285-04.patch,
> HDFS-8450-HDFS-7285-05.patch, HDFS-8450-HDFS-7285-07.patch,
> HDFS-8450-HDFS-7285-08.patch, HDFS-8450-HDFS-7285-09.patch,
> HDFS-8450-HDFS-7285-10.patch, HDFS-8450-HDFS-7285-11.patch,
> HDFS-8450-HDFS-7285-12.patch, HDFS-8450-HDFS-7285-13.patch
>
>
> The idea is to follow the same pattern suggested by HDFS-7416. It is good to
> consolidate all the erasure coding zone related implementations of
> {{FSNamesystem}}. Here, proposing {{FSDirErasureCodingZoneOp}} class to have
> functions to perform related erasure coding zone operations.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HDFS-8450) Erasure Coding: Consolidate erasure coding zone related implementation into a single class
[
https://issues.apache.org/jira/browse/HDFS-8450?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14937812#comment-14937812
]
Hudson commented on HDFS-8450:
--
FAILURE: Integrated in Hadoop-Yarn-trunk-Java8 #473 (See
[https://builds.apache.org/job/Hadoop-Yarn-trunk-Java8/473/])
HDFS-8450. Erasure Coding: Consolidate erasure coding zone related
(vinayakumarb: rev 98d340745be682fb251677bb4830aca76119868f)
*
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java
*
hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/NameNodeAdapter.java
*
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/ErasureCodingZoneManager.java
*
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NameNodeRpcServer.java
*
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSDirectory.java
*
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSDirErasureCodingOp.java
*
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSDirWriteFileOp.java
* hadoop-hdfs-project/hadoop-hdfs/CHANGES-HDFS-EC-7285.txt
*
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSDirStatAndListingOp.java
*
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NamenodeFsck.java
*
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSEditLogLoader.java
> Erasure Coding: Consolidate erasure coding zone related implementation into a
> single class
> --
>
> Key: HDFS-8450
> URL: https://issues.apache.org/jira/browse/HDFS-8450
> Project: Hadoop HDFS
> Issue Type: Sub-task
>Reporter: Rakesh R
>Assignee: Rakesh R
> Fix For: HDFS-7285
>
> Attachments: HDFS-8450-FYI.patch, HDFS-8450-HDFS-7285-00.patch,
> HDFS-8450-HDFS-7285-01.patch, HDFS-8450-HDFS-7285-02.patch,
> HDFS-8450-HDFS-7285-03.patch, HDFS-8450-HDFS-7285-04.patch,
> HDFS-8450-HDFS-7285-05.patch, HDFS-8450-HDFS-7285-07.patch,
> HDFS-8450-HDFS-7285-08.patch, HDFS-8450-HDFS-7285-09.patch,
> HDFS-8450-HDFS-7285-10.patch, HDFS-8450-HDFS-7285-11.patch,
> HDFS-8450-HDFS-7285-12.patch, HDFS-8450-HDFS-7285-13.patch
>
>
> The idea is to follow the same pattern suggested by HDFS-7416. It is good to
> consolidate all the erasure coding zone related implementations of
> {{FSNamesystem}}. Here, proposing {{FSDirErasureCodingZoneOp}} class to have
> functions to perform related erasure coding zone operations.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HDFS-8450) Erasure Coding: Consolidate erasure coding zone related implementation into a single class
[
https://issues.apache.org/jira/browse/HDFS-8450?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14938601#comment-14938601
]
Hudson commented on HDFS-8450:
--
FAILURE: Integrated in Hadoop-Mapreduce-trunk-Java8 #465 (See
[https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Java8/465/])
HDFS-8450. Erasure Coding: Consolidate erasure coding zone related
(vinayakumarb: rev 98d340745be682fb251677bb4830aca76119868f)
*
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/ErasureCodingZoneManager.java
*
hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/NameNodeAdapter.java
*
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSEditLogLoader.java
*
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSDirectory.java
*
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSDirStatAndListingOp.java
*
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java
*
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NamenodeFsck.java
*
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NameNodeRpcServer.java
*
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSDirErasureCodingOp.java
*
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSDirWriteFileOp.java
* hadoop-hdfs-project/hadoop-hdfs/CHANGES-HDFS-EC-7285.txt
> Erasure Coding: Consolidate erasure coding zone related implementation into a
> single class
> --
>
> Key: HDFS-8450
> URL: https://issues.apache.org/jira/browse/HDFS-8450
> Project: Hadoop HDFS
> Issue Type: Sub-task
>Reporter: Rakesh R
>Assignee: Rakesh R
> Fix For: HDFS-7285
>
> Attachments: HDFS-8450-FYI.patch, HDFS-8450-HDFS-7285-00.patch,
> HDFS-8450-HDFS-7285-01.patch, HDFS-8450-HDFS-7285-02.patch,
> HDFS-8450-HDFS-7285-03.patch, HDFS-8450-HDFS-7285-04.patch,
> HDFS-8450-HDFS-7285-05.patch, HDFS-8450-HDFS-7285-07.patch,
> HDFS-8450-HDFS-7285-08.patch, HDFS-8450-HDFS-7285-09.patch,
> HDFS-8450-HDFS-7285-10.patch, HDFS-8450-HDFS-7285-11.patch,
> HDFS-8450-HDFS-7285-12.patch, HDFS-8450-HDFS-7285-13.patch
>
>
> The idea is to follow the same pattern suggested by HDFS-7416. It is good to
> consolidate all the erasure coding zone related implementations of
> {{FSNamesystem}}. Here, proposing {{FSDirErasureCodingZoneOp}} class to have
> functions to perform related erasure coding zone operations.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HDFS-8450) Erasure Coding: Consolidate erasure coding zone related implementation into a single class
[
https://issues.apache.org/jira/browse/HDFS-8450?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14938408#comment-14938408
]
Hudson commented on HDFS-8450:
--
FAILURE: Integrated in Hadoop-Mapreduce-trunk #2408 (See
[https://builds.apache.org/job/Hadoop-Mapreduce-trunk/2408/])
HDFS-8450. Erasure Coding: Consolidate erasure coding zone related
(vinayakumarb: rev 98d340745be682fb251677bb4830aca76119868f)
*
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSDirWriteFileOp.java
*
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSDirErasureCodingOp.java
*
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NamenodeFsck.java
*
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSDirStatAndListingOp.java
*
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSEditLogLoader.java
*
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java
*
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/ErasureCodingZoneManager.java
*
hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/NameNodeAdapter.java
*
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSDirectory.java
* hadoop-hdfs-project/hadoop-hdfs/CHANGES-HDFS-EC-7285.txt
*
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NameNodeRpcServer.java
> Erasure Coding: Consolidate erasure coding zone related implementation into a
> single class
> --
>
> Key: HDFS-8450
> URL: https://issues.apache.org/jira/browse/HDFS-8450
> Project: Hadoop HDFS
> Issue Type: Sub-task
>Reporter: Rakesh R
>Assignee: Rakesh R
> Fix For: HDFS-7285
>
> Attachments: HDFS-8450-FYI.patch, HDFS-8450-HDFS-7285-00.patch,
> HDFS-8450-HDFS-7285-01.patch, HDFS-8450-HDFS-7285-02.patch,
> HDFS-8450-HDFS-7285-03.patch, HDFS-8450-HDFS-7285-04.patch,
> HDFS-8450-HDFS-7285-05.patch, HDFS-8450-HDFS-7285-07.patch,
> HDFS-8450-HDFS-7285-08.patch, HDFS-8450-HDFS-7285-09.patch,
> HDFS-8450-HDFS-7285-10.patch, HDFS-8450-HDFS-7285-11.patch,
> HDFS-8450-HDFS-7285-12.patch, HDFS-8450-HDFS-7285-13.patch
>
>
> The idea is to follow the same pattern suggested by HDFS-7416. It is good to
> consolidate all the erasure coding zone related implementations of
> {{FSNamesystem}}. Here, proposing {{FSDirErasureCodingZoneOp}} class to have
> functions to perform related erasure coding zone operations.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HDFS-8450) Erasure Coding: Consolidate erasure coding zone related implementation into a single class
[
https://issues.apache.org/jira/browse/HDFS-8450?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14937555#comment-14937555
]
Hudson commented on HDFS-8450:
--
FAILURE: Integrated in Hadoop-Yarn-trunk #1203 (See
[https://builds.apache.org/job/Hadoop-Yarn-trunk/1203/])
HDFS-8450. Erasure Coding: Consolidate erasure coding zone related
(vinayakumarb: rev 98d340745be682fb251677bb4830aca76119868f)
*
hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/NameNodeAdapter.java
*
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSDirectory.java
*
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java
*
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSEditLogLoader.java
*
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSDirErasureCodingOp.java
*
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSDirWriteFileOp.java
*
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/ErasureCodingZoneManager.java
*
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NameNodeRpcServer.java
*
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSDirStatAndListingOp.java
*
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NamenodeFsck.java
* hadoop-hdfs-project/hadoop-hdfs/CHANGES-HDFS-EC-7285.txt
> Erasure Coding: Consolidate erasure coding zone related implementation into a
> single class
> --
>
> Key: HDFS-8450
> URL: https://issues.apache.org/jira/browse/HDFS-8450
> Project: Hadoop HDFS
> Issue Type: Sub-task
>Reporter: Rakesh R
>Assignee: Rakesh R
> Fix For: HDFS-7285
>
> Attachments: HDFS-8450-FYI.patch, HDFS-8450-HDFS-7285-00.patch,
> HDFS-8450-HDFS-7285-01.patch, HDFS-8450-HDFS-7285-02.patch,
> HDFS-8450-HDFS-7285-03.patch, HDFS-8450-HDFS-7285-04.patch,
> HDFS-8450-HDFS-7285-05.patch, HDFS-8450-HDFS-7285-07.patch,
> HDFS-8450-HDFS-7285-08.patch, HDFS-8450-HDFS-7285-09.patch,
> HDFS-8450-HDFS-7285-10.patch, HDFS-8450-HDFS-7285-11.patch,
> HDFS-8450-HDFS-7285-12.patch, HDFS-8450-HDFS-7285-13.patch
>
>
> The idea is to follow the same pattern suggested by HDFS-7416. It is good to
> consolidate all the erasure coding zone related implementations of
> {{FSNamesystem}}. Here, proposing {{FSDirErasureCodingZoneOp}} class to have
> functions to perform related erasure coding zone operations.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HDFS-8450) Erasure Coding: Consolidate erasure coding zone related implementation into a single class
[
https://issues.apache.org/jira/browse/HDFS-8450?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14937307#comment-14937307
]
Hudson commented on HDFS-8450:
--
FAILURE: Integrated in Hadoop-trunk-Commit #8548 (See
[https://builds.apache.org/job/Hadoop-trunk-Commit/8548/])
HDFS-8450. Erasure Coding: Consolidate erasure coding zone related
(vinayakumarb: rev 98d340745be682fb251677bb4830aca76119868f)
*
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSDirectory.java
*
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java
*
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSDirStatAndListingOp.java
*
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/ErasureCodingZoneManager.java
*
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NamenodeFsck.java
*
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSDirErasureCodingOp.java
* hadoop-hdfs-project/hadoop-hdfs/CHANGES-HDFS-EC-7285.txt
*
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSDirWriteFileOp.java
*
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NameNodeRpcServer.java
*
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSEditLogLoader.java
*
hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/NameNodeAdapter.java
> Erasure Coding: Consolidate erasure coding zone related implementation into a
> single class
> --
>
> Key: HDFS-8450
> URL: https://issues.apache.org/jira/browse/HDFS-8450
> Project: Hadoop HDFS
> Issue Type: Sub-task
>Reporter: Rakesh R
>Assignee: Rakesh R
> Fix For: HDFS-7285
>
> Attachments: HDFS-8450-FYI.patch, HDFS-8450-HDFS-7285-00.patch,
> HDFS-8450-HDFS-7285-01.patch, HDFS-8450-HDFS-7285-02.patch,
> HDFS-8450-HDFS-7285-03.patch, HDFS-8450-HDFS-7285-04.patch,
> HDFS-8450-HDFS-7285-05.patch, HDFS-8450-HDFS-7285-07.patch,
> HDFS-8450-HDFS-7285-08.patch, HDFS-8450-HDFS-7285-09.patch,
> HDFS-8450-HDFS-7285-10.patch, HDFS-8450-HDFS-7285-11.patch,
> HDFS-8450-HDFS-7285-12.patch, HDFS-8450-HDFS-7285-13.patch
>
>
> The idea is to follow the same pattern suggested by HDFS-7416. It is good to
> consolidate all the erasure coding zone related implementations of
> {{FSNamesystem}}. Here, proposing {{FSDirErasureCodingZoneOp}} class to have
> functions to perform related erasure coding zone operations.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HDFS-8450) Erasure Coding: Consolidate erasure coding zone related implementation into a single class
[
https://issues.apache.org/jira/browse/HDFS-8450?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14937110#comment-14937110
]
Hudson commented on HDFS-8450:
--
FAILURE: Integrated in Hadoop-Hdfs-trunk #2379 (See
[https://builds.apache.org/job/Hadoop-Hdfs-trunk/2379/])
HDFS-8450. Erasure Coding: Consolidate erasure coding zone related
(vinayakumarb: rev 98d340745be682fb251677bb4830aca76119868f)
*
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java
*
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSDirWriteFileOp.java
*
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NamenodeFsck.java
*
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NameNodeRpcServer.java
*
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/ErasureCodingZoneManager.java
*
hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/NameNodeAdapter.java
*
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSEditLogLoader.java
*
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSDirectory.java
*
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSDirStatAndListingOp.java
*
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSDirErasureCodingOp.java
* hadoop-hdfs-project/hadoop-hdfs/CHANGES-HDFS-EC-7285.txt
> Erasure Coding: Consolidate erasure coding zone related implementation into a
> single class
> --
>
> Key: HDFS-8450
> URL: https://issues.apache.org/jira/browse/HDFS-8450
> Project: Hadoop HDFS
> Issue Type: Sub-task
>Reporter: Rakesh R
>Assignee: Rakesh R
> Fix For: HDFS-7285
>
> Attachments: HDFS-8450-FYI.patch, HDFS-8450-HDFS-7285-00.patch,
> HDFS-8450-HDFS-7285-01.patch, HDFS-8450-HDFS-7285-02.patch,
> HDFS-8450-HDFS-7285-03.patch, HDFS-8450-HDFS-7285-04.patch,
> HDFS-8450-HDFS-7285-05.patch, HDFS-8450-HDFS-7285-07.patch,
> HDFS-8450-HDFS-7285-08.patch, HDFS-8450-HDFS-7285-09.patch,
> HDFS-8450-HDFS-7285-10.patch, HDFS-8450-HDFS-7285-11.patch,
> HDFS-8450-HDFS-7285-12.patch, HDFS-8450-HDFS-7285-13.patch
>
>
> The idea is to follow the same pattern suggested by HDFS-7416. It is good to
> consolidate all the erasure coding zone related implementations of
> {{FSNamesystem}}. Here, proposing {{FSDirErasureCodingZoneOp}} class to have
> functions to perform related erasure coding zone operations.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HDFS-8450) Erasure Coding: Consolidate erasure coding zone related implementation into a single class
[
https://issues.apache.org/jira/browse/HDFS-8450?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14585145#comment-14585145
]
Rakesh R commented on HDFS-8450:
Thank you [~vinayrpet] for the great support in reviewing and committing the
patch.
Thank you [~drankye] for the great support in reviewing the patch.
Erasure Coding: Consolidate erasure coding zone related implementation into a
single class
--
Key: HDFS-8450
URL: https://issues.apache.org/jira/browse/HDFS-8450
Project: Hadoop HDFS
Issue Type: Sub-task
Reporter: Rakesh R
Assignee: Rakesh R
Fix For: HDFS-7285
Attachments: HDFS-8450-FYI.patch, HDFS-8450-HDFS-7285-00.patch,
HDFS-8450-HDFS-7285-01.patch, HDFS-8450-HDFS-7285-02.patch,
HDFS-8450-HDFS-7285-03.patch, HDFS-8450-HDFS-7285-04.patch,
HDFS-8450-HDFS-7285-05.patch, HDFS-8450-HDFS-7285-07.patch,
HDFS-8450-HDFS-7285-08.patch, HDFS-8450-HDFS-7285-09.patch,
HDFS-8450-HDFS-7285-10.patch, HDFS-8450-HDFS-7285-11.patch,
HDFS-8450-HDFS-7285-12.patch, HDFS-8450-HDFS-7285-13.patch
The idea is to follow the same pattern suggested by HDFS-7416. It is good to
consolidate all the erasure coding zone related implementations of
{{FSNamesystem}}. Here, proposing {{FSDirErasureCodingZoneOp}} class to have
functions to perform related erasure coding zone operations.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HDFS-8450) Erasure Coding: Consolidate erasure coding zone related implementation into a single class
[
https://issues.apache.org/jira/browse/HDFS-8450?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14580420#comment-14580420
]
Rakesh R commented on HDFS-8450:
Thanks again [~vinayrpet] for the thorough reviews. Attached another patch
addressing the comments.
Erasure Coding: Consolidate erasure coding zone related implementation into a
single class
--
Key: HDFS-8450
URL: https://issues.apache.org/jira/browse/HDFS-8450
Project: Hadoop HDFS
Issue Type: Sub-task
Reporter: Rakesh R
Assignee: Rakesh R
Attachments: HDFS-8450-FYI.patch, HDFS-8450-HDFS-7285-00.patch,
HDFS-8450-HDFS-7285-01.patch, HDFS-8450-HDFS-7285-02.patch,
HDFS-8450-HDFS-7285-03.patch, HDFS-8450-HDFS-7285-04.patch,
HDFS-8450-HDFS-7285-05.patch, HDFS-8450-HDFS-7285-07.patch,
HDFS-8450-HDFS-7285-08.patch, HDFS-8450-HDFS-7285-09.patch,
HDFS-8450-HDFS-7285-10.patch, HDFS-8450-HDFS-7285-11.patch,
HDFS-8450-HDFS-7285-12.patch, HDFS-8450-HDFS-7285-13.patch
The idea is to follow the same pattern suggested by HDFS-7416. It is good to
consolidate all the erasure coding zone related implementations of
{{FSNamesystem}}. Here, proposing {{FSDirErasureCodingZoneOp}} class to have
functions to perform related erasure coding zone operations.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HDFS-8450) Erasure Coding: Consolidate erasure coding zone related implementation into a single class
[
https://issues.apache.org/jira/browse/HDFS-8450?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14580511#comment-14580511
]
Hadoop QA commented on HDFS-8450:
-
\\
\\
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:red}-1{color} | pre-patch | 15m 10s | Findbugs (version ) appears to
be broken on HDFS-7285. |
| {color:green}+1{color} | @author | 0m 0s | The patch does not contain any
@author tags. |
| {color:green}+1{color} | tests included | 0m 0s | The patch appears to
include 1 new or modified test files. |
| {color:green}+1{color} | javac | 7m 33s | There were no new javac warning
messages. |
| {color:green}+1{color} | javadoc | 9m 42s | There were no new javadoc
warning messages. |
| {color:red}-1{color} | release audit | 0m 15s | The applied patch generated
1 release audit warnings. |
| {color:green}+1{color} | checkstyle | 0m 36s | There were no new checkstyle
issues. |
| {color:green}+1{color} | whitespace | 0m 1s | The patch has no lines that
end in whitespace. |
| {color:green}+1{color} | install | 1m 37s | mvn install still works. |
| {color:green}+1{color} | eclipse:eclipse | 0m 33s | The patch built with
eclipse:eclipse. |
| {color:red}-1{color} | findbugs | 3m 27s | The patch appears to introduce 4
new Findbugs (version 3.0.0) warnings. |
| {color:green}+1{color} | native | 3m 20s | Pre-build of native portion |
| {color:red}-1{color} | hdfs tests | 51m 32s | Tests failed in hadoop-hdfs. |
| | | 93m 52s | |
\\
\\
|| Reason || Tests ||
| FindBugs | module:hadoop-hdfs |
| Failed unit tests | hadoop.hdfs.TestFileAppend4 |
| | hadoop.hdfs.TestRead |
| | hadoop.hdfs.server.namenode.TestFSPermissionChecker |
| | hadoop.hdfs.server.namenode.TestNameNodeRetryCacheMetrics |
| | hadoop.hdfs.server.namenode.snapshot.TestSnapshot |
| | hadoop.hdfs.server.namenode.TestSecondaryWebUi |
| | hadoop.hdfs.server.namenode.TestSaveNamespace |
| | hadoop.hdfs.server.namenode.TestNNStorageRetentionFunctional |
| | hadoop.hdfs.tools.TestGetGroups |
| | hadoop.hdfs.server.namenode.TestFavoredNodesEndToEnd |
| | hadoop.hdfs.server.namenode.ha.TestHAStateTransitions |
| | hadoop.hdfs.server.datanode.TestRefreshNamenodes |
| | hadoop.hdfs.TestHdfsAdmin |
| | hadoop.hdfs.server.datanode.TestDataNodeVolumeFailureReporting |
| | hadoop.hdfs.server.datanode.TestBlockHasMultipleReplicasOnSameDN |
| | hadoop.hdfs.server.namenode.snapshot.TestSnapshotFileLength |
| | hadoop.hdfs.server.namenode.ha.TestLossyRetryInvocationHandler |
| | hadoop.hdfs.TestClientReportBadBlock |
| | hadoop.hdfs.server.namenode.TestNamenodeCapacityReport |
| | hadoop.hdfs.server.namenode.TestNamenodeRetryCache |
| | hadoop.hdfs.server.namenode.TestFSEditLogLoader |
| | hadoop.hdfs.server.namenode.TestAuditLogger |
| | hadoop.hdfs.server.blockmanagement.TestUnderReplicatedBlocks |
| | hadoop.hdfs.tools.TestDebugAdmin |
| | hadoop.hdfs.server.datanode.fsdataset.impl.TestLazyPersistPolicy |
| | hadoop.hdfs.server.blockmanagement.TestDatanodeManager |
| | hadoop.hdfs.server.datanode.TestDataNodeMetrics |
| | hadoop.hdfs.TestAppendSnapshotTruncate |
| | hadoop.hdfs.server.datanode.fsdataset.impl.TestLazyPersistFiles |
| | hadoop.hdfs.server.balancer.TestBalancerWithNodeGroup |
| |
hadoop.hdfs.server.datanode.fsdataset.impl.TestLazyPersistReplicaPlacement |
| | hadoop.hdfs.server.namenode.TestNameNodeRpcServer |
| | hadoop.hdfs.TestFileAppendRestart |
| | hadoop.hdfs.server.namenode.TestSecondaryNameNodeUpgrade |
| | hadoop.cli.TestErasureCodingCLI |
| | hadoop.hdfs.TestMultiThreadedHflush |
| | hadoop.hdfs.TestParallelRead |
| | hadoop.hdfs.server.namenode.snapshot.TestSetQuotaWithSnapshot |
| | hadoop.hdfs.server.namenode.ha.TestHAFsck |
| | hadoop.hdfs.server.namenode.snapshot.TestNestedSnapshots |
| | hadoop.hdfs.tools.TestStoragePolicyCommands |
| |
hadoop.hdfs.tools.offlineImageViewer.TestOfflineImageViewerWithStripedBlocks |
| | hadoop.hdfs.protocol.TestBlockListAsLongs |
| | hadoop.hdfs.server.namenode.snapshot.TestOpenFilesWithSnapshot |
| | hadoop.hdfs.server.namenode.TestHDFSConcat |
| | hadoop.hdfs.server.namenode.metrics.TestNameNodeMetrics |
| | hadoop.TestRefreshCallQueue |
| | hadoop.hdfs.TestSetTimes |
| | hadoop.hdfs.TestListFilesInDFS |
| | hadoop.hdfs.server.namenode.TestAddBlock |
| | hadoop.hdfs.server.namenode.TestMalformedURLs |
| | hadoop.hdfs.server.datanode.TestDnRespectsBlockReportSplitThreshold |
| | hadoop.hdfs.TestEncryptedTransfer |
| | hadoop.hdfs.server.namenode.TestNameEditsConfigs |
| | hadoop.hdfs.TestMiniDFSCluster |
| | hadoop.hdfs.tools.TestDFSZKFailoverController |
| | hadoop.hdfs.server.mover.TestMover |
| | hadoop.security.TestPermissionSymlinks |
| | hadoop.hdfs.TestDFSRollback |
| |
[jira] [Commented] (HDFS-8450) Erasure Coding: Consolidate erasure coding zone related implementation into a single class
[
https://issues.apache.org/jira/browse/HDFS-8450?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14580296#comment-14580296
]
Hadoop QA commented on HDFS-8450:
-
\\
\\
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:red}-1{color} | pre-patch | 15m 32s | Findbugs (version ) appears to
be broken on HDFS-7285. |
| {color:green}+1{color} | @author | 0m 0s | The patch does not contain any
@author tags. |
| {color:green}+1{color} | tests included | 0m 0s | The patch appears to
include 1 new or modified test files. |
| {color:green}+1{color} | javac | 7m 36s | There were no new javac warning
messages. |
| {color:green}+1{color} | javadoc | 9m 42s | There were no new javadoc
warning messages. |
| {color:red}-1{color} | release audit | 0m 14s | The applied patch generated
1 release audit warnings. |
| {color:green}+1{color} | checkstyle | 0m 38s | There were no new checkstyle
issues. |
| {color:green}+1{color} | whitespace | 0m 1s | The patch has no lines that
end in whitespace. |
| {color:green}+1{color} | install | 1m 38s | mvn install still works. |
| {color:green}+1{color} | eclipse:eclipse | 0m 32s | The patch built with
eclipse:eclipse. |
| {color:red}-1{color} | findbugs | 3m 26s | The patch appears to introduce 4
new Findbugs (version 3.0.0) warnings. |
| {color:green}+1{color} | native | 3m 16s | Pre-build of native portion |
| {color:red}-1{color} | hdfs tests | 174m 24s | Tests failed in hadoop-hdfs. |
| | | 217m 5s | |
\\
\\
|| Reason || Tests ||
| FindBugs | module:hadoop-hdfs |
| Failed unit tests | hadoop.hdfs.server.namenode.TestFileTruncate |
| | hadoop.hdfs.server.blockmanagement.TestBlockTokenWithDFS |
| | hadoop.hdfs.TestEncryptedTransfer |
| | hadoop.cli.TestErasureCodingCLI |
| | hadoop.hdfs.server.balancer.TestBalancer |
| | hadoop.hdfs.server.datanode.TestDataNodeMetrics |
| | hadoop.hdfs.server.namenode.TestAuditLogs |
| | hadoop.hdfs.server.blockmanagement.TestBlockInfo |
\\
\\
|| Subsystem || Report/Notes ||
| Patch URL |
http://issues.apache.org/jira/secure/attachment/12738743/HDFS-8450-HDFS-7285-12.patch
|
| Optional Tests | javadoc javac unit findbugs checkstyle |
| git revision | HDFS-7285 / c5d4652 |
| Release Audit |
https://builds.apache.org/job/PreCommit-HDFS-Build/11297/artifact/patchprocess/patchReleaseAuditProblems.txt
|
| Findbugs warnings |
https://builds.apache.org/job/PreCommit-HDFS-Build/11297/artifact/patchprocess/newPatchFindbugsWarningshadoop-hdfs.html
|
| hadoop-hdfs test log |
https://builds.apache.org/job/PreCommit-HDFS-Build/11297/artifact/patchprocess/testrun_hadoop-hdfs.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-HDFS-Build/11297/testReport/ |
| Java | 1.7.0_55 |
| uname | Linux asf904.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP
PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |
| Console output |
https://builds.apache.org/job/PreCommit-HDFS-Build/11297/console |
This message was automatically generated.
Erasure Coding: Consolidate erasure coding zone related implementation into a
single class
--
Key: HDFS-8450
URL: https://issues.apache.org/jira/browse/HDFS-8450
Project: Hadoop HDFS
Issue Type: Sub-task
Reporter: Rakesh R
Assignee: Rakesh R
Attachments: HDFS-8450-FYI.patch, HDFS-8450-HDFS-7285-00.patch,
HDFS-8450-HDFS-7285-01.patch, HDFS-8450-HDFS-7285-02.patch,
HDFS-8450-HDFS-7285-03.patch, HDFS-8450-HDFS-7285-04.patch,
HDFS-8450-HDFS-7285-05.patch, HDFS-8450-HDFS-7285-07.patch,
HDFS-8450-HDFS-7285-08.patch, HDFS-8450-HDFS-7285-09.patch,
HDFS-8450-HDFS-7285-10.patch, HDFS-8450-HDFS-7285-11.patch,
HDFS-8450-HDFS-7285-12.patch
The idea is to follow the same pattern suggested by HDFS-7416. It is good to
consolidate all the erasure coding zone related implementations of
{{FSNamesystem}}. Here, proposing {{FSDirErasureCodingZoneOp}} class to have
functions to perform related erasure coding zone operations.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HDFS-8450) Erasure Coding: Consolidate erasure coding zone related implementation into a single class
[
https://issues.apache.org/jira/browse/HDFS-8450?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14580315#comment-14580315
]
Vinayakumar B commented on HDFS-8450:
-
{{TestErasureCodingCLI}} related to HDFS-8571.
Oters are unrelated
Erasure Coding: Consolidate erasure coding zone related implementation into a
single class
--
Key: HDFS-8450
URL: https://issues.apache.org/jira/browse/HDFS-8450
Project: Hadoop HDFS
Issue Type: Sub-task
Reporter: Rakesh R
Assignee: Rakesh R
Attachments: HDFS-8450-FYI.patch, HDFS-8450-HDFS-7285-00.patch,
HDFS-8450-HDFS-7285-01.patch, HDFS-8450-HDFS-7285-02.patch,
HDFS-8450-HDFS-7285-03.patch, HDFS-8450-HDFS-7285-04.patch,
HDFS-8450-HDFS-7285-05.patch, HDFS-8450-HDFS-7285-07.patch,
HDFS-8450-HDFS-7285-08.patch, HDFS-8450-HDFS-7285-09.patch,
HDFS-8450-HDFS-7285-10.patch, HDFS-8450-HDFS-7285-11.patch,
HDFS-8450-HDFS-7285-12.patch
The idea is to follow the same pattern suggested by HDFS-7416. It is good to
consolidate all the erasure coding zone related implementations of
{{FSNamesystem}}. Here, proposing {{FSDirErasureCodingZoneOp}} class to have
functions to perform related erasure coding zone operations.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HDFS-8450) Erasure Coding: Consolidate erasure coding zone related implementation into a single class
[
https://issues.apache.org/jira/browse/HDFS-8450?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14580323#comment-14580323
]
Vinayakumar B commented on HDFS-8450:
-
Latest patch looks almost good. We are near to make it final.
Just one more change,
In NamenodeFsck.java, include below code also inside readLock, because there
could be chances of file getting deleted at same time. if not under readlock,
later problems will come.
{code} INode inode =
namenode.getNamesystem().getFSDirectory().getINode(path);
INodesInPath iip = INodesInPath.fromINode(inode);{code}
+1 once this is addressed.
Erasure Coding: Consolidate erasure coding zone related implementation into a
single class
--
Key: HDFS-8450
URL: https://issues.apache.org/jira/browse/HDFS-8450
Project: Hadoop HDFS
Issue Type: Sub-task
Reporter: Rakesh R
Assignee: Rakesh R
Attachments: HDFS-8450-FYI.patch, HDFS-8450-HDFS-7285-00.patch,
HDFS-8450-HDFS-7285-01.patch, HDFS-8450-HDFS-7285-02.patch,
HDFS-8450-HDFS-7285-03.patch, HDFS-8450-HDFS-7285-04.patch,
HDFS-8450-HDFS-7285-05.patch, HDFS-8450-HDFS-7285-07.patch,
HDFS-8450-HDFS-7285-08.patch, HDFS-8450-HDFS-7285-09.patch,
HDFS-8450-HDFS-7285-10.patch, HDFS-8450-HDFS-7285-11.patch,
HDFS-8450-HDFS-7285-12.patch
The idea is to follow the same pattern suggested by HDFS-7416. It is good to
consolidate all the erasure coding zone related implementations of
{{FSNamesystem}}. Here, proposing {{FSDirErasureCodingZoneOp}} class to have
functions to perform related erasure coding zone operations.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HDFS-8450) Erasure Coding: Consolidate erasure coding zone related implementation into a single class
[
https://issues.apache.org/jira/browse/HDFS-8450?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14580942#comment-14580942
]
Hadoop QA commented on HDFS-8450:
-
\\
\\
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:red}-1{color} | pre-patch | 15m 8s | Findbugs (version ) appears to
be broken on HDFS-7285. |
| {color:green}+1{color} | @author | 0m 0s | The patch does not contain any
@author tags. |
| {color:green}+1{color} | tests included | 0m 0s | The patch appears to
include 1 new or modified test files. |
| {color:green}+1{color} | javac | 7m 30s | There were no new javac warning
messages. |
| {color:green}+1{color} | javadoc | 9m 37s | There were no new javadoc
warning messages. |
| {color:red}-1{color} | release audit | 0m 15s | The applied patch generated
1 release audit warnings. |
| {color:green}+1{color} | checkstyle | 0m 37s | There were no new checkstyle
issues. |
| {color:green}+1{color} | whitespace | 0m 1s | The patch has no lines that
end in whitespace. |
| {color:green}+1{color} | install | 1m 36s | mvn install still works. |
| {color:green}+1{color} | eclipse:eclipse | 0m 33s | The patch built with
eclipse:eclipse. |
| {color:red}-1{color} | findbugs | 3m 24s | The patch appears to introduce 4
new Findbugs (version 3.0.0) warnings. |
| {color:green}+1{color} | native | 3m 12s | Pre-build of native portion |
| {color:red}-1{color} | hdfs tests | 177m 10s | Tests failed in hadoop-hdfs. |
| | | 219m 9s | |
\\
\\
|| Reason || Tests ||
| FindBugs | module:hadoop-hdfs |
| Failed unit tests | hadoop.hdfs.TestEncryptedTransfer |
| | hadoop.hdfs.server.mover.TestMover |
| | hadoop.hdfs.server.blockmanagement.TestBlockTokenWithDFS |
| | hadoop.cli.TestErasureCodingCLI |
| | hadoop.hdfs.server.namenode.TestAuditLogs |
| | hadoop.hdfs.server.blockmanagement.TestBlockInfo |
| | hadoop.hdfs.server.namenode.TestFileTruncate |
| | hadoop.hdfs.TestDFSClientRetries |
\\
\\
|| Subsystem || Report/Notes ||
| Patch URL |
http://issues.apache.org/jira/secure/attachment/12738797/HDFS-8450-HDFS-7285-13.patch
|
| Optional Tests | javadoc javac unit findbugs checkstyle |
| git revision | HDFS-7285 / c5d4652 |
| Release Audit |
https://builds.apache.org/job/PreCommit-HDFS-Build/11302/artifact/patchprocess/patchReleaseAuditProblems.txt
|
| Findbugs warnings |
https://builds.apache.org/job/PreCommit-HDFS-Build/11302/artifact/patchprocess/newPatchFindbugsWarningshadoop-hdfs.html
|
| hadoop-hdfs test log |
https://builds.apache.org/job/PreCommit-HDFS-Build/11302/artifact/patchprocess/testrun_hadoop-hdfs.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-HDFS-Build/11302/testReport/ |
| Java | 1.7.0_55 |
| uname | Linux asf909.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP
PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |
| Console output |
https://builds.apache.org/job/PreCommit-HDFS-Build/11302/console |
This message was automatically generated.
Erasure Coding: Consolidate erasure coding zone related implementation into a
single class
--
Key: HDFS-8450
URL: https://issues.apache.org/jira/browse/HDFS-8450
Project: Hadoop HDFS
Issue Type: Sub-task
Reporter: Rakesh R
Assignee: Rakesh R
Attachments: HDFS-8450-FYI.patch, HDFS-8450-HDFS-7285-00.patch,
HDFS-8450-HDFS-7285-01.patch, HDFS-8450-HDFS-7285-02.patch,
HDFS-8450-HDFS-7285-03.patch, HDFS-8450-HDFS-7285-04.patch,
HDFS-8450-HDFS-7285-05.patch, HDFS-8450-HDFS-7285-07.patch,
HDFS-8450-HDFS-7285-08.patch, HDFS-8450-HDFS-7285-09.patch,
HDFS-8450-HDFS-7285-10.patch, HDFS-8450-HDFS-7285-11.patch,
HDFS-8450-HDFS-7285-12.patch, HDFS-8450-HDFS-7285-13.patch
The idea is to follow the same pattern suggested by HDFS-7416. It is good to
consolidate all the erasure coding zone related implementations of
{{FSNamesystem}}. Here, proposing {{FSDirErasureCodingZoneOp}} class to have
functions to perform related erasure coding zone operations.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HDFS-8450) Erasure Coding: Consolidate erasure coding zone related implementation into a single class
[
https://issues.apache.org/jira/browse/HDFS-8450?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14581438#comment-14581438
]
Vinayakumar B commented on HDFS-8450:
-
All test failures are unrelated to this patch.
I am committing this soon
Erasure Coding: Consolidate erasure coding zone related implementation into a
single class
--
Key: HDFS-8450
URL: https://issues.apache.org/jira/browse/HDFS-8450
Project: Hadoop HDFS
Issue Type: Sub-task
Reporter: Rakesh R
Assignee: Rakesh R
Attachments: HDFS-8450-FYI.patch, HDFS-8450-HDFS-7285-00.patch,
HDFS-8450-HDFS-7285-01.patch, HDFS-8450-HDFS-7285-02.patch,
HDFS-8450-HDFS-7285-03.patch, HDFS-8450-HDFS-7285-04.patch,
HDFS-8450-HDFS-7285-05.patch, HDFS-8450-HDFS-7285-07.patch,
HDFS-8450-HDFS-7285-08.patch, HDFS-8450-HDFS-7285-09.patch,
HDFS-8450-HDFS-7285-10.patch, HDFS-8450-HDFS-7285-11.patch,
HDFS-8450-HDFS-7285-12.patch, HDFS-8450-HDFS-7285-13.patch
The idea is to follow the same pattern suggested by HDFS-7416. It is good to
consolidate all the erasure coding zone related implementations of
{{FSNamesystem}}. Here, proposing {{FSDirErasureCodingZoneOp}} class to have
functions to perform related erasure coding zone operations.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HDFS-8450) Erasure Coding: Consolidate erasure coding zone related implementation into a single class
[
https://issues.apache.org/jira/browse/HDFS-8450?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14580628#comment-14580628
]
Vinayakumar B commented on HDFS-8450:
-
+1 for patch.
Triggered jenkins again to get a clean build in HDFS-7285 branch.
Erasure Coding: Consolidate erasure coding zone related implementation into a
single class
--
Key: HDFS-8450
URL: https://issues.apache.org/jira/browse/HDFS-8450
Project: Hadoop HDFS
Issue Type: Sub-task
Reporter: Rakesh R
Assignee: Rakesh R
Attachments: HDFS-8450-FYI.patch, HDFS-8450-HDFS-7285-00.patch,
HDFS-8450-HDFS-7285-01.patch, HDFS-8450-HDFS-7285-02.patch,
HDFS-8450-HDFS-7285-03.patch, HDFS-8450-HDFS-7285-04.patch,
HDFS-8450-HDFS-7285-05.patch, HDFS-8450-HDFS-7285-07.patch,
HDFS-8450-HDFS-7285-08.patch, HDFS-8450-HDFS-7285-09.patch,
HDFS-8450-HDFS-7285-10.patch, HDFS-8450-HDFS-7285-11.patch,
HDFS-8450-HDFS-7285-12.patch, HDFS-8450-HDFS-7285-13.patch
The idea is to follow the same pattern suggested by HDFS-7416. It is good to
consolidate all the erasure coding zone related implementations of
{{FSNamesystem}}. Here, proposing {{FSDirErasureCodingZoneOp}} class to have
functions to perform related erasure coding zone operations.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HDFS-8450) Erasure Coding: Consolidate erasure coding zone related implementation into a single class
[
https://issues.apache.org/jira/browse/HDFS-8450?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14578718#comment-14578718
]
Vinayakumar B commented on HDFS-8450:
-
1. {{FSNamesystem#getErasureCodingZoneForPath()}} should not call
{{FSNamesystem#getErasureCodingZone(src)}}. Here in the call hierarchy again
{{checkOperation()}} will happen.
Instead it should be reverse. {{FSNamesystem#getErasureCodingZoneForPath(src)}}
should return {{FSDirErasureCodingOp.getErasureCodingZone(this, src);}} and
keep {{FSNamesystem#getErasureCodingZone(src)}} unchanged, as it already
calling {{FSNamesystem#getErasureCodingZoneForPath(src)}}.
2. in {{NameNodeFsck#collectBlocksSummary}} get readLock within if block.
{code} if(file.getReplication() == 0) {
INode inode = namenode.getNamesystem().getFSDirectory().getINode(path);
INodesInPath iip = INodesInPath.fromINode(inode);
ECSchema ecSchema = FSDirErasureCodingOp.getErasureCodingSchema(
namenode.getNamesystem(), iip);
targetFileReplication = (short) (ecSchema.getNumDataUnits() +
ecSchema.getNumParityUnits());
} else {{code}
3. {{NameNodeAdapter.java}}, acquire fsn#readLock() within getFileInfo().
Erasure Coding: Consolidate erasure coding zone related implementation into a
single class
--
Key: HDFS-8450
URL: https://issues.apache.org/jira/browse/HDFS-8450
Project: Hadoop HDFS
Issue Type: Sub-task
Reporter: Rakesh R
Assignee: Rakesh R
Attachments: HDFS-8450-FYI.patch, HDFS-8450-HDFS-7285-00.patch,
HDFS-8450-HDFS-7285-01.patch, HDFS-8450-HDFS-7285-02.patch,
HDFS-8450-HDFS-7285-03.patch, HDFS-8450-HDFS-7285-04.patch,
HDFS-8450-HDFS-7285-05.patch, HDFS-8450-HDFS-7285-07.patch,
HDFS-8450-HDFS-7285-08.patch, HDFS-8450-HDFS-7285-09.patch,
HDFS-8450-HDFS-7285-10.patch, HDFS-8450-HDFS-7285-11.patch
The idea is to follow the same pattern suggested by HDFS-7416. It is good to
consolidate all the erasure coding zone related implementations of
{{FSNamesystem}}. Here, proposing {{FSDirErasureCodingZoneOp}} class to have
functions to perform related erasure coding zone operations.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HDFS-8450) Erasure Coding: Consolidate erasure coding zone related implementation into a single class
[
https://issues.apache.org/jira/browse/HDFS-8450?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14578781#comment-14578781
]
Rakesh R commented on HDFS-8450:
Thanks again [~vinayrpet] for reviews. Could you please give few details about
below comments:
bq. NameNodeAdapter.java, acquire fsn#readLock() within getFileInfo().
I'm referring to this class
[NameNodeAdapter.java|https://github.com/apache/hadoop/blob/HDFS-7285/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/NameNodeAdapter.java#L71].
I couldn't see any erasure coding related function calls in this class, it
would be helpful if you give some details about the necessity of
{{sn#readLock()}} in {{#getFileInfo}} function. Thanks!
Erasure Coding: Consolidate erasure coding zone related implementation into a
single class
--
Key: HDFS-8450
URL: https://issues.apache.org/jira/browse/HDFS-8450
Project: Hadoop HDFS
Issue Type: Sub-task
Reporter: Rakesh R
Assignee: Rakesh R
Attachments: HDFS-8450-FYI.patch, HDFS-8450-HDFS-7285-00.patch,
HDFS-8450-HDFS-7285-01.patch, HDFS-8450-HDFS-7285-02.patch,
HDFS-8450-HDFS-7285-03.patch, HDFS-8450-HDFS-7285-04.patch,
HDFS-8450-HDFS-7285-05.patch, HDFS-8450-HDFS-7285-07.patch,
HDFS-8450-HDFS-7285-08.patch, HDFS-8450-HDFS-7285-09.patch,
HDFS-8450-HDFS-7285-10.patch, HDFS-8450-HDFS-7285-11.patch
The idea is to follow the same pattern suggested by HDFS-7416. It is good to
consolidate all the erasure coding zone related implementations of
{{FSNamesystem}}. Here, proposing {{FSDirErasureCodingZoneOp}} class to have
functions to perform related erasure coding zone operations.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HDFS-8450) Erasure Coding: Consolidate erasure coding zone related implementation into a single class
[
https://issues.apache.org/jira/browse/HDFS-8450?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=1457#comment-1457
]
Vinayakumar B commented on HDFS-8450:
-
bq. I'm referring to this class NameNodeAdapter.java. I couldn't see any
erasure coding related function calls in this class, it would be helpful if you
give some details about the necessity of sn#readLock() in #getFileInfo
function. Thanks!
See the below stacktrace, so many test failures are due to this.
{{NameNodeAdapter.java}} is for only test.
FSDirErasureCodingOp.getErasureCodingZone is called internally through this.
This change will fix many test failures.
{noformat}java.lang.AssertionError: null
at
org.apache.hadoop.hdfs.server.namenode.FSDirErasureCodingOp.getErasureCodingZone(FSDirErasureCodingOp.java:105)
at
org.apache.hadoop.hdfs.server.namenode.FSDirStatAndListingOp.createFileStatus(FSDirStatAndListingOp.java:389)
at
org.apache.hadoop.hdfs.server.namenode.FSDirStatAndListingOp.getFileInfo(FSDirStatAndListingOp.java:302)
at
org.apache.hadoop.hdfs.server.namenode.FSDirStatAndListingOp.getFileInfo(FSDirStatAndListingOp.java:329)
at
org.apache.hadoop.hdfs.server.namenode.FSDirStatAndListingOp.getFileInfo(FSDirStatAndListingOp.java:112)
at
org.apache.hadoop.hdfs.server.namenode.NameNodeAdapter.getFileInfo(NameNodeAdapter.java:74)
at
org.apache.hadoop.hdfs.server.namenode.ha.TestEditLogTailer.testTailer(TestEditLogTailer.java:86){noformat}
Erasure Coding: Consolidate erasure coding zone related implementation into a
single class
--
Key: HDFS-8450
URL: https://issues.apache.org/jira/browse/HDFS-8450
Project: Hadoop HDFS
Issue Type: Sub-task
Reporter: Rakesh R
Assignee: Rakesh R
Attachments: HDFS-8450-FYI.patch, HDFS-8450-HDFS-7285-00.patch,
HDFS-8450-HDFS-7285-01.patch, HDFS-8450-HDFS-7285-02.patch,
HDFS-8450-HDFS-7285-03.patch, HDFS-8450-HDFS-7285-04.patch,
HDFS-8450-HDFS-7285-05.patch, HDFS-8450-HDFS-7285-07.patch,
HDFS-8450-HDFS-7285-08.patch, HDFS-8450-HDFS-7285-09.patch,
HDFS-8450-HDFS-7285-10.patch, HDFS-8450-HDFS-7285-11.patch
The idea is to follow the same pattern suggested by HDFS-7416. It is good to
consolidate all the erasure coding zone related implementations of
{{FSNamesystem}}. Here, proposing {{FSDirErasureCodingZoneOp}} class to have
functions to perform related erasure coding zone operations.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HDFS-8450) Erasure Coding: Consolidate erasure coding zone related implementation into a single class
[
https://issues.apache.org/jira/browse/HDFS-8450?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14580006#comment-14580006
]
Rakesh R commented on HDFS-8450:
Good catch, I didn't see the {{testReport}}. Thanks a lot for pointing out this.
Erasure Coding: Consolidate erasure coding zone related implementation into a
single class
--
Key: HDFS-8450
URL: https://issues.apache.org/jira/browse/HDFS-8450
Project: Hadoop HDFS
Issue Type: Sub-task
Reporter: Rakesh R
Assignee: Rakesh R
Attachments: HDFS-8450-FYI.patch, HDFS-8450-HDFS-7285-00.patch,
HDFS-8450-HDFS-7285-01.patch, HDFS-8450-HDFS-7285-02.patch,
HDFS-8450-HDFS-7285-03.patch, HDFS-8450-HDFS-7285-04.patch,
HDFS-8450-HDFS-7285-05.patch, HDFS-8450-HDFS-7285-07.patch,
HDFS-8450-HDFS-7285-08.patch, HDFS-8450-HDFS-7285-09.patch,
HDFS-8450-HDFS-7285-10.patch, HDFS-8450-HDFS-7285-11.patch
The idea is to follow the same pattern suggested by HDFS-7416. It is good to
consolidate all the erasure coding zone related implementations of
{{FSNamesystem}}. Here, proposing {{FSDirErasureCodingZoneOp}} class to have
functions to perform related erasure coding zone operations.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HDFS-8450) Erasure Coding: Consolidate erasure coding zone related implementation into a single class
[
https://issues.apache.org/jira/browse/HDFS-8450?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14576713#comment-14576713
]
Kai Zheng commented on HDFS-8450:
-
The latest patch looks good to me. +1 pending on Jenkins build. Better if
[~vinayrpet] could also confirm.
Erasure Coding: Consolidate erasure coding zone related implementation into a
single class
--
Key: HDFS-8450
URL: https://issues.apache.org/jira/browse/HDFS-8450
Project: Hadoop HDFS
Issue Type: Sub-task
Reporter: Rakesh R
Assignee: Rakesh R
Attachments: HDFS-8450-FYI.patch, HDFS-8450-HDFS-7285-00.patch,
HDFS-8450-HDFS-7285-01.patch, HDFS-8450-HDFS-7285-02.patch,
HDFS-8450-HDFS-7285-03.patch, HDFS-8450-HDFS-7285-04.patch,
HDFS-8450-HDFS-7285-05.patch, HDFS-8450-HDFS-7285-07.patch,
HDFS-8450-HDFS-7285-08.patch, HDFS-8450-HDFS-7285-09.patch,
HDFS-8450-HDFS-7285-10.patch
The idea is to follow the same pattern suggested by HDFS-7416. It is good to
consolidate all the erasure coding zone related implementations of
{{FSNamesystem}}. Here, proposing {{FSDirErasureCodingZoneOp}} class to have
functions to perform related erasure coding zone operations.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HDFS-8450) Erasure Coding: Consolidate erasure coding zone related implementation into a single class
[
https://issues.apache.org/jira/browse/HDFS-8450?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14576660#comment-14576660
]
Vinayakumar B commented on HDFS-8450:
-
One more nit. Not exactly for this patch. But could go with this. In
ErasureCodingZoneManager
It would be better if all exceptions includes the {{src}} in messages.
Ex:
{code} throw new IOException(Attempt to create an erasure coding zone +
for a file.);{code}
Erasure Coding: Consolidate erasure coding zone related implementation into a
single class
--
Key: HDFS-8450
URL: https://issues.apache.org/jira/browse/HDFS-8450
Project: Hadoop HDFS
Issue Type: Sub-task
Reporter: Rakesh R
Assignee: Rakesh R
Attachments: HDFS-8450-FYI.patch, HDFS-8450-HDFS-7285-00.patch,
HDFS-8450-HDFS-7285-01.patch, HDFS-8450-HDFS-7285-02.patch,
HDFS-8450-HDFS-7285-03.patch, HDFS-8450-HDFS-7285-04.patch,
HDFS-8450-HDFS-7285-05.patch, HDFS-8450-HDFS-7285-07.patch,
HDFS-8450-HDFS-7285-08.patch, HDFS-8450-HDFS-7285-09.patch
The idea is to follow the same pattern suggested by HDFS-7416. It is good to
consolidate all the erasure coding zone related implementations of
{{FSNamesystem}}. Here, proposing {{FSDirErasureCodingZoneOp}} class to have
functions to perform related erasure coding zone operations.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HDFS-8450) Erasure Coding: Consolidate erasure coding zone related implementation into a single class
[
https://issues.apache.org/jira/browse/HDFS-8450?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14576733#comment-14576733
]
Rakesh R commented on HDFS-8450:
Thanks a lot [~drankye] and [~vinayrpet] for the reviews. Vinay, the latest
patch addresses your comments too. Please review when you get a chance!
Erasure Coding: Consolidate erasure coding zone related implementation into a
single class
--
Key: HDFS-8450
URL: https://issues.apache.org/jira/browse/HDFS-8450
Project: Hadoop HDFS
Issue Type: Sub-task
Reporter: Rakesh R
Assignee: Rakesh R
Attachments: HDFS-8450-FYI.patch, HDFS-8450-HDFS-7285-00.patch,
HDFS-8450-HDFS-7285-01.patch, HDFS-8450-HDFS-7285-02.patch,
HDFS-8450-HDFS-7285-03.patch, HDFS-8450-HDFS-7285-04.patch,
HDFS-8450-HDFS-7285-05.patch, HDFS-8450-HDFS-7285-07.patch,
HDFS-8450-HDFS-7285-08.patch, HDFS-8450-HDFS-7285-09.patch,
HDFS-8450-HDFS-7285-10.patch
The idea is to follow the same pattern suggested by HDFS-7416. It is good to
consolidate all the erasure coding zone related implementations of
{{FSNamesystem}}. Here, proposing {{FSDirErasureCodingZoneOp}} class to have
functions to perform related erasure coding zone operations.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HDFS-8450) Erasure Coding: Consolidate erasure coding zone related implementation into a single class
[
https://issues.apache.org/jira/browse/HDFS-8450?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14576835#comment-14576835
]
Vinayakumar B commented on HDFS-8450:
-
{{fsd.getFSNamesystem().checkOperation(OperationCategory.READ);}}
This check is not mandatorily required in all places wherever
{{FSDirErasureCodingOp}} methods are called.
{{checkOperation}} is applicable only for the RPC calls. And if its already
done after acquiring the lock, then no need to do again.
All other cases which can happen only on ActiveNamenode after aquiring lock,
{{checkOperation}} call not required.
Following are the places where this check is not required in the patch.
1. {{FSDirStatAndListingOp.java}}
2. {{FSDirWriteFileOp.java}}
3. {{FSEditLogLoader}}
4. {{FSNamesystem#commitOrCompleteLastBlock(..)}} as this comes somewhere
inside in the call heirarchy, before reaching here check already done.
5. {{NamenodeFsck.java}} as in StandbyNameNode call will not reach this point,
if READ is disabled.
As though IMO, all these except #3, wont create any problem, but it would be
unnecessary. #3 might create problem in SNN while loading edits.
+1 once these are addressed.
Erasure Coding: Consolidate erasure coding zone related implementation into a
single class
--
Key: HDFS-8450
URL: https://issues.apache.org/jira/browse/HDFS-8450
Project: Hadoop HDFS
Issue Type: Sub-task
Reporter: Rakesh R
Assignee: Rakesh R
Attachments: HDFS-8450-FYI.patch, HDFS-8450-HDFS-7285-00.patch,
HDFS-8450-HDFS-7285-01.patch, HDFS-8450-HDFS-7285-02.patch,
HDFS-8450-HDFS-7285-03.patch, HDFS-8450-HDFS-7285-04.patch,
HDFS-8450-HDFS-7285-05.patch, HDFS-8450-HDFS-7285-07.patch,
HDFS-8450-HDFS-7285-08.patch, HDFS-8450-HDFS-7285-09.patch,
HDFS-8450-HDFS-7285-10.patch
The idea is to follow the same pattern suggested by HDFS-7416. It is good to
consolidate all the erasure coding zone related implementations of
{{FSNamesystem}}. Here, proposing {{FSDirErasureCodingZoneOp}} class to have
functions to perform related erasure coding zone operations.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HDFS-8450) Erasure Coding: Consolidate erasure coding zone related implementation into a single class
[
https://issues.apache.org/jira/browse/HDFS-8450?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14577128#comment-14577128
]
Rakesh R commented on HDFS-8450:
Thanks [~vinayrpet]. Attached another patch addressing {{checkOperation}}
related comments.
Erasure Coding: Consolidate erasure coding zone related implementation into a
single class
--
Key: HDFS-8450
URL: https://issues.apache.org/jira/browse/HDFS-8450
Project: Hadoop HDFS
Issue Type: Sub-task
Reporter: Rakesh R
Assignee: Rakesh R
Attachments: HDFS-8450-FYI.patch, HDFS-8450-HDFS-7285-00.patch,
HDFS-8450-HDFS-7285-01.patch, HDFS-8450-HDFS-7285-02.patch,
HDFS-8450-HDFS-7285-03.patch, HDFS-8450-HDFS-7285-04.patch,
HDFS-8450-HDFS-7285-05.patch, HDFS-8450-HDFS-7285-07.patch,
HDFS-8450-HDFS-7285-08.patch, HDFS-8450-HDFS-7285-09.patch,
HDFS-8450-HDFS-7285-10.patch, HDFS-8450-HDFS-7285-11.patch
The idea is to follow the same pattern suggested by HDFS-7416. It is good to
consolidate all the erasure coding zone related implementations of
{{FSNamesystem}}. Here, proposing {{FSDirErasureCodingZoneOp}} class to have
functions to perform related erasure coding zone operations.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HDFS-8450) Erasure Coding: Consolidate erasure coding zone related implementation into a single class
[
https://issues.apache.org/jira/browse/HDFS-8450?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14577056#comment-14577056
]
Hadoop QA commented on HDFS-8450:
-
\\
\\
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:red}-1{color} | pre-patch | 15m 35s | Findbugs (version ) appears to
be broken on HDFS-7285. |
| {color:green}+1{color} | @author | 0m 0s | The patch does not contain any
@author tags. |
| {color:red}-1{color} | tests included | 0m 0s | The patch doesn't appear
to include any new or modified tests. Please justify why no new tests are
needed for this patch. Also please list what manual steps were performed to
verify this patch. |
| {color:green}+1{color} | javac | 7m 47s | There were no new javac warning
messages. |
| {color:green}+1{color} | javadoc | 9m 57s | There were no new javadoc
warning messages. |
| {color:red}-1{color} | release audit | 0m 15s | The applied patch generated
1 release audit warnings. |
| {color:green}+1{color} | checkstyle | 0m 40s | There were no new checkstyle
issues. |
| {color:green}+1{color} | whitespace | 0m 0s | The patch has no lines that
end in whitespace. |
| {color:green}+1{color} | install | 1m 43s | mvn install still works. |
| {color:green}+1{color} | eclipse:eclipse | 0m 33s | The patch built with
eclipse:eclipse. |
| {color:red}-1{color} | findbugs | 3m 30s | The patch appears to introduce 4
new Findbugs (version 3.0.0) warnings. |
| {color:green}+1{color} | native | 3m 20s | Pre-build of native portion |
| {color:red}-1{color} | hdfs tests | 233m 51s | Tests failed in hadoop-hdfs. |
| | | 277m 18s | |
\\
\\
|| Reason || Tests ||
| FindBugs | module:hadoop-hdfs |
| Failed unit tests | hadoop.hdfs.server.namenode.TestFileTruncate |
| | hadoop.hdfs.server.blockmanagement.TestBlockTokenWithDFS |
| | hadoop.hdfs.server.namenode.ha.TestHAMetrics |
| | hadoop.hdfs.TestEncryptedTransfer |
| | hadoop.hdfs.server.namenode.ha.TestEditLogTailer |
| | hadoop.hdfs.TestDecommission |
| | hadoop.hdfs.server.mover.TestMover |
| | hadoop.hdfs.server.namenode.ha.TestInitializeSharedEdits |
| | hadoop.hdfs.server.namenode.TestFsck |
| | hadoop.hdfs.server.namenode.TestAuditLogs |
| | hadoop.hdfs.server.namenode.ha.TestPendingCorruptDnMessages |
| | hadoop.hdfs.server.namenode.ha.TestEditLogsDuringFailover |
| | hadoop.hdfs.server.namenode.ha.TestHAAppend |
| | hadoop.hdfs.server.namenode.ha.TestHAStateTransitions |
| | hadoop.hdfs.server.namenode.ha.TestFailureToReadEdits |
| | hadoop.hdfs.server.blockmanagement.TestBlockInfo |
| Timed out tests | org.apache.hadoop.hdfs.server.namenode.ha.TestHASafeMode |
| | org.apache.hadoop.hdfs.server.namenode.ha.TestStandbyCheckpoints |
| | org.apache.hadoop.hdfs.server.namenode.ha.TestRetryCacheWithHA |
| | org.apache.hadoop.hdfs.server.namenode.ha.TestDNFencing |
\\
\\
|| Subsystem || Report/Notes ||
| Patch URL |
http://issues.apache.org/jira/secure/attachment/12738306/HDFS-8450-HDFS-7285-10.patch
|
| Optional Tests | javadoc javac unit findbugs checkstyle |
| git revision | HDFS-7285 / c41b02c |
| Release Audit |
https://builds.apache.org/job/PreCommit-HDFS-Build/11266/artifact/patchprocess/patchReleaseAuditProblems.txt
|
| Findbugs warnings |
https://builds.apache.org/job/PreCommit-HDFS-Build/11266/artifact/patchprocess/newPatchFindbugsWarningshadoop-hdfs.html
|
| hadoop-hdfs test log |
https://builds.apache.org/job/PreCommit-HDFS-Build/11266/artifact/patchprocess/testrun_hadoop-hdfs.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-HDFS-Build/11266/testReport/ |
| Java | 1.7.0_55 |
| uname | Linux asf904.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP
PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |
| Console output |
https://builds.apache.org/job/PreCommit-HDFS-Build/11266/console |
This message was automatically generated.
Erasure Coding: Consolidate erasure coding zone related implementation into a
single class
--
Key: HDFS-8450
URL: https://issues.apache.org/jira/browse/HDFS-8450
Project: Hadoop HDFS
Issue Type: Sub-task
Reporter: Rakesh R
Assignee: Rakesh R
Attachments: HDFS-8450-FYI.patch, HDFS-8450-HDFS-7285-00.patch,
HDFS-8450-HDFS-7285-01.patch, HDFS-8450-HDFS-7285-02.patch,
HDFS-8450-HDFS-7285-03.patch, HDFS-8450-HDFS-7285-04.patch,
HDFS-8450-HDFS-7285-05.patch, HDFS-8450-HDFS-7285-07.patch,
HDFS-8450-HDFS-7285-08.patch, HDFS-8450-HDFS-7285-09.patch,
HDFS-8450-HDFS-7285-10.patch
The idea is to follow the same pattern suggested by HDFS-7416. It is good to
consolidate all the erasure coding zone related implementations of
{{FSNamesystem}}. Here, proposing
[jira] [Commented] (HDFS-8450) Erasure Coding: Consolidate erasure coding zone related implementation into a single class
[
https://issues.apache.org/jira/browse/HDFS-8450?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14577089#comment-14577089
]
Hadoop QA commented on HDFS-8450:
-
\\
\\
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:red}-1{color} | pre-patch | 14m 59s | Findbugs (version ) appears to
be broken on HDFS-7285. |
| {color:green}+1{color} | @author | 0m 0s | The patch does not contain any
@author tags. |
| {color:red}-1{color} | tests included | 0m 0s | The patch doesn't appear
to include any new or modified tests. Please justify why no new tests are
needed for this patch. Also please list what manual steps were performed to
verify this patch. |
| {color:green}+1{color} | javac | 7m 28s | There were no new javac warning
messages. |
| {color:green}+1{color} | javadoc | 9m 37s | There were no new javadoc
warning messages. |
| {color:red}-1{color} | release audit | 0m 15s | The applied patch generated
1 release audit warnings. |
| {color:green}+1{color} | checkstyle | 0m 38s | There were no new checkstyle
issues. |
| {color:green}+1{color} | whitespace | 0m 1s | The patch has no lines that
end in whitespace. |
| {color:green}+1{color} | install | 1m 35s | mvn install still works. |
| {color:green}+1{color} | eclipse:eclipse | 0m 33s | The patch built with
eclipse:eclipse. |
| {color:red}-1{color} | findbugs | 3m 24s | The patch appears to introduce 4
new Findbugs (version 3.0.0) warnings. |
| {color:green}+1{color} | native | 3m 14s | Pre-build of native portion |
| {color:red}-1{color} | hdfs tests | 232m 56s | Tests failed in hadoop-hdfs. |
| | | 274m 45s | |
\\
\\
|| Reason || Tests ||
| FindBugs | module:hadoop-hdfs |
| Failed unit tests | hadoop.hdfs.TestEncryptedTransfer |
| | hadoop.hdfs.server.namenode.ha.TestInitializeSharedEdits |
| | hadoop.hdfs.TestDecommission |
| | hadoop.hdfs.server.mover.TestMover |
| | hadoop.hdfs.server.namenode.TestFileTruncate |
| | hadoop.hdfs.server.namenode.ha.TestPendingCorruptDnMessages |
| | hadoop.hdfs.server.namenode.ha.TestEditLogTailer |
| | hadoop.hdfs.server.namenode.ha.TestHAStateTransitions |
| | hadoop.hdfs.server.namenode.TestFsck |
| | hadoop.hdfs.server.namenode.ha.TestHAAppend |
| | hadoop.hdfs.server.namenode.ha.TestHAMetrics |
| | hadoop.hdfs.server.namenode.TestAuditLogs |
| | hadoop.hdfs.server.blockmanagement.TestBlockInfo |
| | hadoop.hdfs.server.namenode.ha.TestEditLogsDuringFailover |
| | hadoop.hdfs.server.namenode.ha.TestFailureToReadEdits |
| | hadoop.hdfs.server.blockmanagement.TestBlockTokenWithDFS |
| Timed out tests | org.apache.hadoop.hdfs.server.namenode.ha.TestDNFencing |
| | org.apache.hadoop.hdfs.server.namenode.ha.TestHASafeMode |
| | org.apache.hadoop.hdfs.server.namenode.ha.TestRetryCacheWithHA |
| | org.apache.hadoop.hdfs.server.namenode.ha.TestStandbyCheckpoints |
\\
\\
|| Subsystem || Report/Notes ||
| Patch URL |
http://issues.apache.org/jira/secure/attachment/12738308/HDFS-8450-HDFS-7285-10.patch
|
| Optional Tests | javadoc javac unit findbugs checkstyle |
| git revision | HDFS-7285 / c41b02c |
| Release Audit |
https://builds.apache.org/job/PreCommit-HDFS-Build/11267/artifact/patchprocess/patchReleaseAuditProblems.txt
|
| Findbugs warnings |
https://builds.apache.org/job/PreCommit-HDFS-Build/11267/artifact/patchprocess/newPatchFindbugsWarningshadoop-hdfs.html
|
| hadoop-hdfs test log |
https://builds.apache.org/job/PreCommit-HDFS-Build/11267/artifact/patchprocess/testrun_hadoop-hdfs.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-HDFS-Build/11267/testReport/ |
| Java | 1.7.0_55 |
| uname | Linux asf906.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP
PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |
| Console output |
https://builds.apache.org/job/PreCommit-HDFS-Build/11267/console |
This message was automatically generated.
Erasure Coding: Consolidate erasure coding zone related implementation into a
single class
--
Key: HDFS-8450
URL: https://issues.apache.org/jira/browse/HDFS-8450
Project: Hadoop HDFS
Issue Type: Sub-task
Reporter: Rakesh R
Assignee: Rakesh R
Attachments: HDFS-8450-FYI.patch, HDFS-8450-HDFS-7285-00.patch,
HDFS-8450-HDFS-7285-01.patch, HDFS-8450-HDFS-7285-02.patch,
HDFS-8450-HDFS-7285-03.patch, HDFS-8450-HDFS-7285-04.patch,
HDFS-8450-HDFS-7285-05.patch, HDFS-8450-HDFS-7285-07.patch,
HDFS-8450-HDFS-7285-08.patch, HDFS-8450-HDFS-7285-09.patch,
HDFS-8450-HDFS-7285-10.patch, HDFS-8450-HDFS-7285-11.patch
The idea is to follow the same pattern suggested by HDFS-7416. It is good to
consolidate all the erasure coding zone related implementations of
{{FSNamesystem}}.
[jira] [Commented] (HDFS-8450) Erasure Coding: Consolidate erasure coding zone related implementation into a single class
[
https://issues.apache.org/jira/browse/HDFS-8450?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14577384#comment-14577384
]
Hadoop QA commented on HDFS-8450:
-
\\
\\
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:red}-1{color} | pre-patch | 15m 19s | Findbugs (version ) appears to
be broken on HDFS-7285. |
| {color:green}+1{color} | @author | 0m 0s | The patch does not contain any
@author tags. |
| {color:red}-1{color} | tests included | 0m 0s | The patch doesn't appear
to include any new or modified tests. Please justify why no new tests are
needed for this patch. Also please list what manual steps were performed to
verify this patch. |
| {color:green}+1{color} | javac | 7m 33s | There were no new javac warning
messages. |
| {color:green}+1{color} | javadoc | 9m 41s | There were no new javadoc
warning messages. |
| {color:red}-1{color} | release audit | 0m 15s | The applied patch generated
1 release audit warnings. |
| {color:green}+1{color} | checkstyle | 0m 37s | There were no new checkstyle
issues. |
| {color:green}+1{color} | whitespace | 0m 1s | The patch has no lines that
end in whitespace. |
| {color:green}+1{color} | install | 1m 39s | mvn install still works. |
| {color:green}+1{color} | eclipse:eclipse | 0m 33s | The patch built with
eclipse:eclipse. |
| {color:red}-1{color} | findbugs | 3m 28s | The patch appears to introduce 4
new Findbugs (version 3.0.0) warnings. |
| {color:green}+1{color} | native | 3m 16s | Pre-build of native portion |
| {color:red}-1{color} | hdfs tests | 177m 19s | Tests failed in hadoop-hdfs. |
| | | 219m 47s | |
\\
\\
|| Reason || Tests ||
| FindBugs | module:hadoop-hdfs |
| Failed unit tests | hadoop.hdfs.server.blockmanagement.TestBlockInfo |
| | hadoop.hdfs.server.namenode.TestFsck |
| | hadoop.hdfs.server.mover.TestMover |
| | hadoop.hdfs.server.blockmanagement.TestBlockTokenWithDFS |
| | hadoop.hdfs.server.namenode.ha.TestInitializeSharedEdits |
| | hadoop.hdfs.server.namenode.ha.TestEditLogsDuringFailover |
| | hadoop.hdfs.server.namenode.ha.TestEditLogTailer |
| | hadoop.hdfs.server.namenode.TestAuditLogs |
| | hadoop.hdfs.server.namenode.TestFileTruncate |
| | hadoop.hdfs.server.namenode.ha.TestFailureToReadEdits |
| | hadoop.hdfs.TestEncryptedTransfer |
\\
\\
|| Subsystem || Report/Notes ||
| Patch URL |
http://issues.apache.org/jira/secure/attachment/12738346/HDFS-8450-HDFS-7285-11.patch
|
| Optional Tests | javadoc javac unit findbugs checkstyle |
| git revision | HDFS-7285 / c41b02c |
| Release Audit |
https://builds.apache.org/job/PreCommit-HDFS-Build/11270/artifact/patchprocess/patchReleaseAuditProblems.txt
|
| Findbugs warnings |
https://builds.apache.org/job/PreCommit-HDFS-Build/11270/artifact/patchprocess/newPatchFindbugsWarningshadoop-hdfs.html
|
| hadoop-hdfs test log |
https://builds.apache.org/job/PreCommit-HDFS-Build/11270/artifact/patchprocess/testrun_hadoop-hdfs.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-HDFS-Build/11270/testReport/ |
| Java | 1.7.0_55 |
| uname | Linux asf900.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP
PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |
| Console output |
https://builds.apache.org/job/PreCommit-HDFS-Build/11270/console |
This message was automatically generated.
Erasure Coding: Consolidate erasure coding zone related implementation into a
single class
--
Key: HDFS-8450
URL: https://issues.apache.org/jira/browse/HDFS-8450
Project: Hadoop HDFS
Issue Type: Sub-task
Reporter: Rakesh R
Assignee: Rakesh R
Attachments: HDFS-8450-FYI.patch, HDFS-8450-HDFS-7285-00.patch,
HDFS-8450-HDFS-7285-01.patch, HDFS-8450-HDFS-7285-02.patch,
HDFS-8450-HDFS-7285-03.patch, HDFS-8450-HDFS-7285-04.patch,
HDFS-8450-HDFS-7285-05.patch, HDFS-8450-HDFS-7285-07.patch,
HDFS-8450-HDFS-7285-08.patch, HDFS-8450-HDFS-7285-09.patch,
HDFS-8450-HDFS-7285-10.patch, HDFS-8450-HDFS-7285-11.patch
The idea is to follow the same pattern suggested by HDFS-7416. It is good to
consolidate all the erasure coding zone related implementations of
{{FSNamesystem}}. Here, proposing {{FSDirErasureCodingZoneOp}} class to have
functions to perform related erasure coding zone operations.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HDFS-8450) Erasure Coding: Consolidate erasure coding zone related implementation into a single class
[
https://issues.apache.org/jira/browse/HDFS-8450?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14576226#comment-14576226
]
Hadoop QA commented on HDFS-8450:
-
\\
\\
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:red}-1{color} | pre-patch | 15m 0s | Findbugs (version ) appears to
be broken on HDFS-7285. |
| {color:green}+1{color} | @author | 0m 0s | The patch does not contain any
@author tags. |
| {color:red}-1{color} | tests included | 0m 0s | The patch doesn't appear
to include any new or modified tests. Please justify why no new tests are
needed for this patch. Also please list what manual steps were performed to
verify this patch. |
| {color:green}+1{color} | javac | 7m 32s | There were no new javac warning
messages. |
| {color:green}+1{color} | javadoc | 9m 39s | There were no new javadoc
warning messages. |
| {color:red}-1{color} | release audit | 0m 15s | The applied patch generated
1 release audit warnings. |
| {color:green}+1{color} | checkstyle | 0m 37s | There were no new checkstyle
issues. |
| {color:green}+1{color} | whitespace | 0m 0s | The patch has no lines that
end in whitespace. |
| {color:green}+1{color} | install | 1m 37s | mvn install still works. |
| {color:green}+1{color} | eclipse:eclipse | 0m 33s | The patch built with
eclipse:eclipse. |
| {color:red}-1{color} | findbugs | 3m 25s | The patch appears to introduce 4
new Findbugs (version 3.0.0) warnings. |
| {color:green}+1{color} | native | 3m 14s | Pre-build of native portion |
| {color:red}-1{color} | hdfs tests | 230m 17s | Tests failed in hadoop-hdfs. |
| | | 272m 14s | |
\\
\\
|| Reason || Tests ||
| FindBugs | module:hadoop-hdfs |
| Failed unit tests | hadoop.hdfs.TestEncryptedTransfer |
| | hadoop.hdfs.server.namenode.ha.TestInitializeSharedEdits |
| | hadoop.hdfs.server.blockmanagement.TestBlockTokenWithDFS |
| | hadoop.hdfs.server.namenode.ha.TestPipelinesFailover |
| | hadoop.hdfs.server.namenode.ha.TestHAMetrics |
| | hadoop.hdfs.server.namenode.ha.TestEditLogTailer |
| | hadoop.hdfs.server.namenode.ha.TestEditLogsDuringFailover |
| | hadoop.hdfs.server.namenode.TestAuditLogs |
| | hadoop.hdfs.server.namenode.TestFsck |
| | hadoop.hdfs.TestDecommission |
| | hadoop.hdfs.server.namenode.ha.TestFailureToReadEdits |
| | hadoop.hdfs.server.namenode.ha.TestPendingCorruptDnMessages |
| | hadoop.hdfs.server.blockmanagement.TestBlockInfo |
| | hadoop.hdfs.server.namenode.ha.TestHAAppend |
| | hadoop.hdfs.server.namenode.TestFileTruncate |
| | hadoop.hdfs.server.namenode.ha.TestHAStateTransitions |
| Timed out tests | org.apache.hadoop.hdfs.server.namenode.ha.TestDNFencing |
| | org.apache.hadoop.hdfs.server.namenode.ha.TestHASafeMode |
| | org.apache.hadoop.hdfs.server.namenode.ha.TestRetryCacheWithHA |
| | org.apache.hadoop.hdfs.server.namenode.ha.TestStandbyCheckpoints |
\\
\\
|| Subsystem || Report/Notes ||
| Patch URL |
http://issues.apache.org/jira/secure/attachment/12738228/HDFS-8450-HDFS-7285-09.patch
|
| Optional Tests | javadoc javac unit findbugs checkstyle |
| git revision | HDFS-7285 / 2eee19c |
| Release Audit |
https://builds.apache.org/job/PreCommit-HDFS-Build/11261/artifact/patchprocess/patchReleaseAuditProblems.txt
|
| Findbugs warnings |
https://builds.apache.org/job/PreCommit-HDFS-Build/11261/artifact/patchprocess/newPatchFindbugsWarningshadoop-hdfs.html
|
| hadoop-hdfs test log |
https://builds.apache.org/job/PreCommit-HDFS-Build/11261/artifact/patchprocess/testrun_hadoop-hdfs.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-HDFS-Build/11261/testReport/ |
| Java | 1.7.0_55 |
| uname | Linux asf909.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP
PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |
| Console output |
https://builds.apache.org/job/PreCommit-HDFS-Build/11261/console |
This message was automatically generated.
Erasure Coding: Consolidate erasure coding zone related implementation into a
single class
--
Key: HDFS-8450
URL: https://issues.apache.org/jira/browse/HDFS-8450
Project: Hadoop HDFS
Issue Type: Sub-task
Reporter: Rakesh R
Assignee: Rakesh R
Attachments: HDFS-8450-FYI.patch, HDFS-8450-HDFS-7285-00.patch,
HDFS-8450-HDFS-7285-01.patch, HDFS-8450-HDFS-7285-02.patch,
HDFS-8450-HDFS-7285-03.patch, HDFS-8450-HDFS-7285-04.patch,
HDFS-8450-HDFS-7285-05.patch, HDFS-8450-HDFS-7285-07.patch,
HDFS-8450-HDFS-7285-08.patch, HDFS-8450-HDFS-7285-09.patch
The idea is to follow the same pattern suggested by HDFS-7416. It is good to
consolidate all the erasure coding zone related implementations of
{{FSNamesystem}}. Here, proposing
[jira] [Commented] (HDFS-8450) Erasure Coding: Consolidate erasure coding zone related implementation into a single class
[
https://issues.apache.org/jira/browse/HDFS-8450?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14576594#comment-14576594
]
Rakesh R commented on HDFS-8450:
I've attached patch addressing [~drankye]'s comments. Please review!
Note: It seems the FindBug warning and test case failures are unrelated to the
patch.
Erasure Coding: Consolidate erasure coding zone related implementation into a
single class
--
Key: HDFS-8450
URL: https://issues.apache.org/jira/browse/HDFS-8450
Project: Hadoop HDFS
Issue Type: Sub-task
Reporter: Rakesh R
Assignee: Rakesh R
Attachments: HDFS-8450-FYI.patch, HDFS-8450-HDFS-7285-00.patch,
HDFS-8450-HDFS-7285-01.patch, HDFS-8450-HDFS-7285-02.patch,
HDFS-8450-HDFS-7285-03.patch, HDFS-8450-HDFS-7285-04.patch,
HDFS-8450-HDFS-7285-05.patch, HDFS-8450-HDFS-7285-07.patch,
HDFS-8450-HDFS-7285-08.patch, HDFS-8450-HDFS-7285-09.patch
The idea is to follow the same pattern suggested by HDFS-7416. It is good to
consolidate all the erasure coding zone related implementations of
{{FSNamesystem}}. Here, proposing {{FSDirErasureCodingZoneOp}} class to have
functions to perform related erasure coding zone operations.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HDFS-8450) Erasure Coding: Consolidate erasure coding zone related implementation into a single class
[
https://issues.apache.org/jira/browse/HDFS-8450?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14576596#comment-14576596
]
Kai Zheng commented on HDFS-8450:
-
Rakesh,
1. Please rename {{FSDirErasureCodingZoneOp.java}} to
{{FSDirErasureCodingOp.java}};
2. Please correct comment in below:
{code}
+ /**
+ * Get erasure coding zone.
+ *
+ * @param fsn namespace
+ * @param iip inodes in the path containing the file
+ * @return {@link ErasureCodingZoneInfo}
+ * @throws IOException
+ */
+ static ErasureCodingZone getErasureCodingZone(final FSNamesystem fsn,
+ final INodesInPath iip) throws IOException {
{code}
3. Maybe we could move all the {{checkOperation}} calls to {{FSNamesystem}}
like it does in other {{*Op}} classes?
Erasure Coding: Consolidate erasure coding zone related implementation into a
single class
--
Key: HDFS-8450
URL: https://issues.apache.org/jira/browse/HDFS-8450
Project: Hadoop HDFS
Issue Type: Sub-task
Reporter: Rakesh R
Assignee: Rakesh R
Attachments: HDFS-8450-FYI.patch, HDFS-8450-HDFS-7285-00.patch,
HDFS-8450-HDFS-7285-01.patch, HDFS-8450-HDFS-7285-02.patch,
HDFS-8450-HDFS-7285-03.patch, HDFS-8450-HDFS-7285-04.patch,
HDFS-8450-HDFS-7285-05.patch, HDFS-8450-HDFS-7285-07.patch,
HDFS-8450-HDFS-7285-08.patch, HDFS-8450-HDFS-7285-09.patch
The idea is to follow the same pattern suggested by HDFS-7416. It is good to
consolidate all the erasure coding zone related implementations of
{{FSNamesystem}}. Here, proposing {{FSDirErasureCodingZoneOp}} class to have
functions to perform related erasure coding zone operations.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HDFS-8450) Erasure Coding: Consolidate erasure coding zone related implementation into a single class
[
https://issues.apache.org/jira/browse/HDFS-8450?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14576632#comment-14576632
]
Rakesh R commented on HDFS-8450:
Thanks [~drankye] for the quick reviews. My env had a local copy of
{{FSDirErasureCodingZoneOp}} that created the mistake:( Yes, I agree to move
{{checkOperation}} calls outside. In java comment, I think
{{ErasureCodingZoneInfo}} is the mistake. I will correct all the three comments
and prepare a patch. Thanks again!
Erasure Coding: Consolidate erasure coding zone related implementation into a
single class
--
Key: HDFS-8450
URL: https://issues.apache.org/jira/browse/HDFS-8450
Project: Hadoop HDFS
Issue Type: Sub-task
Reporter: Rakesh R
Assignee: Rakesh R
Attachments: HDFS-8450-FYI.patch, HDFS-8450-HDFS-7285-00.patch,
HDFS-8450-HDFS-7285-01.patch, HDFS-8450-HDFS-7285-02.patch,
HDFS-8450-HDFS-7285-03.patch, HDFS-8450-HDFS-7285-04.patch,
HDFS-8450-HDFS-7285-05.patch, HDFS-8450-HDFS-7285-07.patch,
HDFS-8450-HDFS-7285-08.patch, HDFS-8450-HDFS-7285-09.patch
The idea is to follow the same pattern suggested by HDFS-7416. It is good to
consolidate all the erasure coding zone related implementations of
{{FSNamesystem}}. Here, proposing {{FSDirErasureCodingZoneOp}} class to have
functions to perform related erasure coding zone operations.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HDFS-8450) Erasure Coding: Consolidate erasure coding zone related implementation into a single class
[
https://issues.apache.org/jira/browse/HDFS-8450?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14574197#comment-14574197
]
Vinayakumar B commented on HDFS-8450:
-
bq. I referred other FSNamesystem implementations except encryption related
implementaion all other cases checkSuperuserPrivilege function is called only
once. It looks like not consistently followed. It would be good to find best
practise and follow the same. Any suggestions?
Yes, only once {{checkSuperuserPrivilege}} is sufficient. Authorization check
doesnot depend on the FSNamesystem's state, all no need to check again after
acquiring lock.
bq. Vinayakumar B, could you please correct me if am missing anything. Thanks!
Yes, you are right. Anyway, everything in the patch is just moved from existing
code. So nothing induced. Still its a good time to correct :)
Erasure Coding: Consolidate erasure coding zone related implementation into a
single class
--
Key: HDFS-8450
URL: https://issues.apache.org/jira/browse/HDFS-8450
Project: Hadoop HDFS
Issue Type: Sub-task
Reporter: Rakesh R
Assignee: Rakesh R
Attachments: HDFS-8450-FYI.patch, HDFS-8450-HDFS-7285-00.patch,
HDFS-8450-HDFS-7285-01.patch, HDFS-8450-HDFS-7285-02.patch,
HDFS-8450-HDFS-7285-03.patch, HDFS-8450-HDFS-7285-04.patch,
HDFS-8450-HDFS-7285-05.patch, HDFS-8450-HDFS-7285-07.patch,
HDFS-8450-HDFS-7285-08.patch
The idea is to follow the same pattern suggested by HDFS-7416. It is good to
consolidate all the erasure coding zone related implementations of
{{FSNamesystem}}. Here, proposing {{FSDirErasureCodingZoneOp}} class to have
functions to perform related erasure coding zone operations.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HDFS-8450) Erasure Coding: Consolidate erasure coding zone related implementation into a single class
[
https://issues.apache.org/jira/browse/HDFS-8450?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14574249#comment-14574249
]
Rakesh R commented on HDFS-8450:
Thanks [~vinayrpet] and [~drankye] for the reply. Attached another patch. I
hope I've addressed the comments, could you please review it again.
Erasure Coding: Consolidate erasure coding zone related implementation into a
single class
--
Key: HDFS-8450
URL: https://issues.apache.org/jira/browse/HDFS-8450
Project: Hadoop HDFS
Issue Type: Sub-task
Reporter: Rakesh R
Assignee: Rakesh R
Attachments: HDFS-8450-FYI.patch, HDFS-8450-HDFS-7285-00.patch,
HDFS-8450-HDFS-7285-01.patch, HDFS-8450-HDFS-7285-02.patch,
HDFS-8450-HDFS-7285-03.patch, HDFS-8450-HDFS-7285-04.patch,
HDFS-8450-HDFS-7285-05.patch, HDFS-8450-HDFS-7285-07.patch,
HDFS-8450-HDFS-7285-08.patch
The idea is to follow the same pattern suggested by HDFS-7416. It is good to
consolidate all the erasure coding zone related implementations of
{{FSNamesystem}}. Here, proposing {{FSDirErasureCodingZoneOp}} class to have
functions to perform related erasure coding zone operations.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HDFS-8450) Erasure Coding: Consolidate erasure coding zone related implementation into a single class
[
https://issues.apache.org/jira/browse/HDFS-8450?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14574411#comment-14574411
]
Kai Zheng commented on HDFS-8450:
-
Thanks Rakesh again for the update. More comments, would you help check one
more time? Thanks!
1. {{ErasureCodingZoneManager}} mostly uses {{iip}} instead of {{src}} except
{{createErasureCodingZone}}, so better to refactor it? Thus
{{FSDirErasureCodingOp#createErasureCodingZone}} can also be simplified, using
the utility {{getINodesInPath}}.
2. I don't quite like passing {{fsd}} and {{ecManager}}, instead just passing
{{fsn}} is good enough. Please check other places.
{code}
+ private static ErasureCodingZone getErasureCodingZone(final FSDirectory fsd,
+ final INodesInPath iip, final ErasureCodingZoneManager ecManager)
+ throws IOException {
{code}
3. How about changing {{fsn.getECSchemaManager}} this time together?
Thanks again!
Erasure Coding: Consolidate erasure coding zone related implementation into a
single class
--
Key: HDFS-8450
URL: https://issues.apache.org/jira/browse/HDFS-8450
Project: Hadoop HDFS
Issue Type: Sub-task
Reporter: Rakesh R
Assignee: Rakesh R
Attachments: HDFS-8450-FYI.patch, HDFS-8450-HDFS-7285-00.patch,
HDFS-8450-HDFS-7285-01.patch, HDFS-8450-HDFS-7285-02.patch,
HDFS-8450-HDFS-7285-03.patch, HDFS-8450-HDFS-7285-04.patch,
HDFS-8450-HDFS-7285-05.patch, HDFS-8450-HDFS-7285-07.patch,
HDFS-8450-HDFS-7285-08.patch
The idea is to follow the same pattern suggested by HDFS-7416. It is good to
consolidate all the erasure coding zone related implementations of
{{FSNamesystem}}. Here, proposing {{FSDirErasureCodingZoneOp}} class to have
functions to perform related erasure coding zone operations.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HDFS-8450) Erasure Coding: Consolidate erasure coding zone related implementation into a single class
[
https://issues.apache.org/jira/browse/HDFS-8450?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14574463#comment-14574463
]
Hadoop QA commented on HDFS-8450:
-
\\
\\
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:red}-1{color} | pre-patch | 15m 34s | Findbugs (version ) appears to
be broken on HDFS-7285. |
| {color:green}+1{color} | @author | 0m 0s | The patch does not contain any
@author tags. |
| {color:red}-1{color} | tests included | 0m 0s | The patch doesn't appear
to include any new or modified tests. Please justify why no new tests are
needed for this patch. Also please list what manual steps were performed to
verify this patch. |
| {color:green}+1{color} | javac | 7m 40s | There were no new javac warning
messages. |
| {color:green}+1{color} | javadoc | 9m 57s | There were no new javadoc
warning messages. |
| {color:red}-1{color} | release audit | 0m 15s | The applied patch generated
1 release audit warnings. |
| {color:green}+1{color} | checkstyle | 0m 39s | There were no new checkstyle
issues. |
| {color:green}+1{color} | whitespace | 0m 1s | The patch has no lines that
end in whitespace. |
| {color:green}+1{color} | install | 1m 37s | mvn install still works. |
| {color:green}+1{color} | eclipse:eclipse | 0m 33s | The patch built with
eclipse:eclipse. |
| {color:red}-1{color} | findbugs | 3m 30s | The patch appears to introduce 4
new Findbugs (version 3.0.0) warnings. |
| {color:green}+1{color} | native | 3m 18s | Pre-build of native portion |
| {color:red}-1{color} | hdfs tests | 193m 5s | Tests failed in hadoop-hdfs. |
| | | 236m 14s | |
\\
\\
|| Reason || Tests ||
| FindBugs | module:hadoop-hdfs |
| Failed unit tests | hadoop.fs.TestUrlStreamHandler |
| | hadoop.hdfs.server.namenode.TestFileLimit |
| | hadoop.TestRefreshCallQueue |
| | hadoop.hdfs.protocolPB.TestPBHelper |
| | hadoop.cli.TestCryptoAdminCLI |
| | hadoop.fs.viewfs.TestViewFsWithAcls |
| | hadoop.hdfs.server.namenode.TestAddStripedBlocks |
| | hadoop.hdfs.server.namenode.TestFSEditLogLoader |
| | hadoop.fs.contract.hdfs.TestHDFSContractDelete |
| | hadoop.fs.TestFcHdfsSetUMask |
| | hadoop.fs.TestUnbuffer |
| | hadoop.hdfs.server.namenode.TestFSDirectory |
| | hadoop.fs.contract.hdfs.TestHDFSContractOpen |
| | hadoop.hdfs.server.namenode.TestFileTruncate |
| | hadoop.fs.contract.hdfs.TestHDFSContractMkdir |
| | hadoop.fs.contract.hdfs.TestHDFSContractAppend |
| | hadoop.hdfs.server.namenode.TestSecondaryWebUi |
| | hadoop.hdfs.server.blockmanagement.TestBlockTokenWithDFS |
| | hadoop.hdfs.server.namenode.TestHDFSConcat |
| | hadoop.hdfs.server.namenode.TestAddBlockRetry |
| | hadoop.fs.TestSymlinkHdfsFileSystem |
| | hadoop.fs.viewfs.TestViewFsDefaultValue |
| | hadoop.fs.TestSymlinkHdfsFileContext |
| | hadoop.hdfs.TestClientProtocolForPipelineRecovery |
| | hadoop.hdfs.TestFSInputChecker |
| | hadoop.hdfs.server.mover.TestStorageMover |
| | hadoop.cli.TestAclCLI |
| | hadoop.hdfs.server.namenode.ha.TestHAMetrics |
| | hadoop.hdfs.TestEncryptedTransfer |
| | hadoop.hdfs.server.namenode.TestNNStorageRetentionFunctional |
| | hadoop.fs.contract.hdfs.TestHDFSContractSeek |
| | hadoop.hdfs.server.namenode.TestFileContextXAttr |
| | hadoop.hdfs.server.namenode.TestAclConfigFlag |
| | hadoop.hdfs.server.namenode.TestFSImageWithXAttr |
| | hadoop.hdfs.server.namenode.metrics.TestNameNodeMetrics |
| | hadoop.tracing.TestTracing |
| | hadoop.hdfs.server.namenode.TestGenericJournalConf |
| | hadoop.fs.viewfs.TestViewFsWithXAttrs |
| | hadoop.cli.TestErasureCodingCLI |
| | hadoop.hdfs.server.namenode.TestEditLogJournalFailures |
| | hadoop.hdfs.TestPersistBlocks |
| | hadoop.hdfs.TestDatanodeReport |
| | hadoop.tools.TestJMXGet |
| | hadoop.fs.contract.hdfs.TestHDFSContractCreate |
| | hadoop.hdfs.server.namenode.TestCreateEditsLog |
| | hadoop.hdfs.TestWriteRead |
| | hadoop.hdfs.server.balancer.TestBalancerWithMultipleNameNodes |
| | hadoop.fs.TestEnhancedByteBufferAccess |
| | hadoop.fs.TestFcHdfsPermission |
| | hadoop.security.TestRefreshUserMappings |
| | hadoop.fs.viewfs.TestViewFsHdfs |
| | hadoop.fs.TestResolveHdfsSymlink |
| | hadoop.hdfs.server.namenode.metrics.TestNNMetricFilesInGetListingOps |
| | hadoop.tracing.TestTracingShortCircuitLocalRead |
| | hadoop.hdfs.server.namenode.TestDecommissioningStatus |
| | hadoop.hdfs.server.namenode.ha.TestEditLogTailer |
| | hadoop.tracing.TestTraceAdmin |
| | hadoop.hdfs.server.namenode.TestNNThroughputBenchmark |
| | hadoop.hdfs.TestDecommission |
| | hadoop.hdfs.server.namenode.ha.TestPipelinesFailover |
| | hadoop.hdfs.TestDatanodeStartupFixesLegacyStorageIDs |
| | hadoop.hdfs.server.balancer.TestBalancerWithEncryptedTransfer |
| | hadoop.hdfs.server.namenode.TestNamenodeRetryCache |
| |
[jira] [Commented] (HDFS-8450) Erasure Coding: Consolidate erasure coding zone related implementation into a single class
[
https://issues.apache.org/jira/browse/HDFS-8450?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14574429#comment-14574429
]
Rakesh R commented on HDFS-8450:
Thanks [~drankye], Point-1. {{FSDirErasureCodingOp#createErasureCodingZone}}
function is using {{#getINodesInPath4Write}}, so I think we cannot reuse the
{{FSDirErasureCodingOp#getINodesInPath}} utility because he checks only READ
permission. So I will retain the same logic in #createErasureCodingZone and
refactor {{ErasureCodingZoneManager}} part also. Whats your opinion?
Point-2. and Point-3. will do as per your suggestion.
Erasure Coding: Consolidate erasure coding zone related implementation into a
single class
--
Key: HDFS-8450
URL: https://issues.apache.org/jira/browse/HDFS-8450
Project: Hadoop HDFS
Issue Type: Sub-task
Reporter: Rakesh R
Assignee: Rakesh R
Attachments: HDFS-8450-FYI.patch, HDFS-8450-HDFS-7285-00.patch,
HDFS-8450-HDFS-7285-01.patch, HDFS-8450-HDFS-7285-02.patch,
HDFS-8450-HDFS-7285-03.patch, HDFS-8450-HDFS-7285-04.patch,
HDFS-8450-HDFS-7285-05.patch, HDFS-8450-HDFS-7285-07.patch,
HDFS-8450-HDFS-7285-08.patch
The idea is to follow the same pattern suggested by HDFS-7416. It is good to
consolidate all the erasure coding zone related implementations of
{{FSNamesystem}}. Here, proposing {{FSDirErasureCodingZoneOp}} class to have
functions to perform related erasure coding zone operations.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HDFS-8450) Erasure Coding: Consolidate erasure coding zone related implementation into a single class
[
https://issues.apache.org/jira/browse/HDFS-8450?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14575334#comment-14575334
]
Kai Zheng commented on HDFS-8450:
-
Sounds great. Thanks!
Erasure Coding: Consolidate erasure coding zone related implementation into a
single class
--
Key: HDFS-8450
URL: https://issues.apache.org/jira/browse/HDFS-8450
Project: Hadoop HDFS
Issue Type: Sub-task
Reporter: Rakesh R
Assignee: Rakesh R
Attachments: HDFS-8450-FYI.patch, HDFS-8450-HDFS-7285-00.patch,
HDFS-8450-HDFS-7285-01.patch, HDFS-8450-HDFS-7285-02.patch,
HDFS-8450-HDFS-7285-03.patch, HDFS-8450-HDFS-7285-04.patch,
HDFS-8450-HDFS-7285-05.patch, HDFS-8450-HDFS-7285-07.patch,
HDFS-8450-HDFS-7285-08.patch
The idea is to follow the same pattern suggested by HDFS-7416. It is good to
consolidate all the erasure coding zone related implementations of
{{FSNamesystem}}. Here, proposing {{FSDirErasureCodingZoneOp}} class to have
functions to perform related erasure coding zone operations.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HDFS-8450) Erasure Coding: Consolidate erasure coding zone related implementation into a single class
[
https://issues.apache.org/jira/browse/HDFS-8450?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14572731#comment-14572731
]
Rakesh R commented on HDFS-8450:
Thanks again [~drankye] for the reviews.
bq. 1. Minor, please update the header comment Helper class to perform erasure
coding zone related operations.
OK. I will modify.
bq. 2. In createErasureCodingZone, checkSuperuserPrivilege, checkOperation and
writeLock were called two times. I'm not very sure about this, maybe
Vinayakumar B could give a comment?
+checkSuperuserPrivilege+
I referred other FSNamesystem implementations except encryption related
implementaion all other cases {{checkSuperuserPrivilege}} function is called
only once. It looks like not consistently followed. It would be good to find
best practise and follow the same. Any suggestions?
+checkOperation+
All the other FSN implementations are following the same pattern, so I feel two
times call is OK.
+writeLock+
First {{writeLock}} is by FSNamesystem and second is by FsDirectory. All others
are following the same pattern, so I feel two times call is OK.
[~vinayrpet], could you please correct me if am missing anything. Thanks!
bq. 3. getErasureCodingZoneForPath could be private as internally used? Note no
lock here so it shouldn't be used by outside.
Yes, Agreed.
bq. 4. In isInErasureCodingZone, no lock and operation check as did in
createErasureCodingZone.
I think there is one improvement I have to do is to move the
{{fsn.writeLock()}} outside the helper class. After referring other
{{FSDir*Op}} implementations, it seems that the caller should acquire the
{{fsn.writeLock/fsn.readLock}} and then inside the helper class it is taking
{{fsd.writeLock()/fsd.readLock()}} depends on the operation. I will refactor
{{#createErasureCodingZone}} and also will acquire necessary lock for
{{isInErasureCodingZone}} function.
bq. 5. Maybe we could have a private function to transform src to iip?
OK, I will do it.
Erasure Coding: Consolidate erasure coding zone related implementation into a
single class
--
Key: HDFS-8450
URL: https://issues.apache.org/jira/browse/HDFS-8450
Project: Hadoop HDFS
Issue Type: Sub-task
Reporter: Rakesh R
Assignee: Rakesh R
Attachments: HDFS-8450-FYI.patch, HDFS-8450-HDFS-7285-00.patch,
HDFS-8450-HDFS-7285-01.patch, HDFS-8450-HDFS-7285-02.patch,
HDFS-8450-HDFS-7285-03.patch, HDFS-8450-HDFS-7285-04.patch,
HDFS-8450-HDFS-7285-05.patch, HDFS-8450-HDFS-7285-07.patch
The idea is to follow the same pattern suggested by HDFS-7416. It is good to
consolidate all the erasure coding zone related implementations of
{{FSNamesystem}}. Here, proposing {{FSDirErasureCodingZoneOp}} class to have
functions to perform related erasure coding zone operations.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HDFS-8450) Erasure Coding: Consolidate erasure coding zone related implementation into a single class
[
https://issues.apache.org/jira/browse/HDFS-8450?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14569514#comment-14569514
]
Hadoop QA commented on HDFS-8450:
-
\\
\\
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:red}-1{color} | pre-patch | 15m 20s | Findbugs (version ) appears to
be broken on HDFS-7285. |
| {color:green}+1{color} | @author | 0m 0s | The patch does not contain any
@author tags. |
| {color:red}-1{color} | tests included | 0m 0s | The patch doesn't appear
to include any new or modified tests. Please justify why no new tests are
needed for this patch. Also please list what manual steps were performed to
verify this patch. |
| {color:green}+1{color} | javac | 7m 30s | There were no new javac warning
messages. |
| {color:green}+1{color} | javadoc | 9m 42s | There were no new javadoc
warning messages. |
| {color:red}-1{color} | release audit | 0m 14s | The applied patch generated
1 release audit warnings. |
| {color:green}+1{color} | checkstyle | 0m 37s | There were no new checkstyle
issues. |
| {color:green}+1{color} | whitespace | 0m 0s | The patch has no lines that
end in whitespace. |
| {color:green}+1{color} | install | 1m 36s | mvn install still works. |
| {color:green}+1{color} | eclipse:eclipse | 0m 33s | The patch built with
eclipse:eclipse. |
| {color:red}-1{color} | findbugs | 3m 23s | The patch appears to introduce 1
new Findbugs (version 3.0.0) warnings. |
| {color:green}+1{color} | native | 3m 15s | Pre-build of native portion |
| {color:red}-1{color} | hdfs tests | 174m 53s | Tests failed in hadoop-hdfs. |
| | | 217m 20s | |
\\
\\
|| Reason || Tests ||
| FindBugs | module:hadoop-hdfs |
| Failed unit tests | hadoop.hdfs.server.namenode.TestAuditLogs |
| | hadoop.hdfs.TestRecoverStripedFile |
| | hadoop.hdfs.server.blockmanagement.TestBlockTokenWithDFS |
| | hadoop.hdfs.server.namenode.TestFileTruncate |
| | hadoop.hdfs.TestEncryptedTransfer |
| | hadoop.hdfs.server.blockmanagement.TestBlockInfo |
\\
\\
|| Subsystem || Report/Notes ||
| Patch URL |
http://issues.apache.org/jira/secure/attachment/12736846/HDFS-8450-HDFS-7285-07.patch
|
| Optional Tests | javadoc javac unit findbugs checkstyle |
| git revision | HDFS-7285 / 2d847e7 |
| Release Audit |
https://builds.apache.org/job/PreCommit-HDFS-Build/11207/artifact/patchprocess/patchReleaseAuditProblems.txt
|
| Findbugs warnings |
https://builds.apache.org/job/PreCommit-HDFS-Build/11207/artifact/patchprocess/newPatchFindbugsWarningshadoop-hdfs.html
|
| hadoop-hdfs test log |
https://builds.apache.org/job/PreCommit-HDFS-Build/11207/artifact/patchprocess/testrun_hadoop-hdfs.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-HDFS-Build/11207/testReport/ |
| Java | 1.7.0_55 |
| uname | Linux asf905.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP
PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |
| Console output |
https://builds.apache.org/job/PreCommit-HDFS-Build/11207/console |
This message was automatically generated.
Erasure Coding: Consolidate erasure coding zone related implementation into a
single class
--
Key: HDFS-8450
URL: https://issues.apache.org/jira/browse/HDFS-8450
Project: Hadoop HDFS
Issue Type: Sub-task
Reporter: Rakesh R
Assignee: Rakesh R
Attachments: HDFS-8450-FYI.patch, HDFS-8450-HDFS-7285-00.patch,
HDFS-8450-HDFS-7285-01.patch, HDFS-8450-HDFS-7285-02.patch,
HDFS-8450-HDFS-7285-03.patch, HDFS-8450-HDFS-7285-04.patch,
HDFS-8450-HDFS-7285-05.patch, HDFS-8450-HDFS-7285-07.patch
The idea is to follow the same pattern suggested by HDFS-7416. It is good to
consolidate all the erasure coding zone related implementations of
{{FSNamesystem}}. Here, proposing {{FSDirErasureCodingZoneOp}} class to have
functions to perform related erasure coding zone operations.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HDFS-8450) Erasure Coding: Consolidate erasure coding zone related implementation into a single class
[
https://issues.apache.org/jira/browse/HDFS-8450?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14569998#comment-14569998
]
Kai Zheng commented on HDFS-8450:
-
Thanks Rakesh again for the update! Some minor comments now:
1. Minor, please update the header comment Helper class to perform erasure
coding zone related operations.
2. In {{createErasureCodingZone}}, checkSuperuserPrivilege, checkOperation and
writeLock were called two times. I'm not very sure about this, maybe
[~vinayrpet] could give a comment?
3. getErasureCodingZoneForPath could be private as internally used? Note no
lock here so it shouldn't be used by outside.
4. In isInErasureCodingZone, no lock and operation check as did in
createErasureCodingZone.
5. Maybe we could have a private function to transform {{src}} to {{iip}}?
Erasure Coding: Consolidate erasure coding zone related implementation into a
single class
--
Key: HDFS-8450
URL: https://issues.apache.org/jira/browse/HDFS-8450
Project: Hadoop HDFS
Issue Type: Sub-task
Reporter: Rakesh R
Assignee: Rakesh R
Attachments: HDFS-8450-FYI.patch, HDFS-8450-HDFS-7285-00.patch,
HDFS-8450-HDFS-7285-01.patch, HDFS-8450-HDFS-7285-02.patch,
HDFS-8450-HDFS-7285-03.patch, HDFS-8450-HDFS-7285-04.patch,
HDFS-8450-HDFS-7285-05.patch, HDFS-8450-HDFS-7285-07.patch
The idea is to follow the same pattern suggested by HDFS-7416. It is good to
consolidate all the erasure coding zone related implementations of
{{FSNamesystem}}. Here, proposing {{FSDirErasureCodingZoneOp}} class to have
functions to perform related erasure coding zone operations.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HDFS-8450) Erasure Coding: Consolidate erasure coding zone related implementation into a single class
[
https://issues.apache.org/jira/browse/HDFS-8450?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14568906#comment-14568906
]
Rakesh R commented on HDFS-8450:
Thanks again [~drankye] for the help. I've few clarifications about the
{{FSDirErasureCodingOp#createErasureCodingZone}} function, could you please see
below points:
# {{fsn.checkOperation(OperationCategory.WRITE);}} should be checked after
acquiring the {{fsn.writeLock();}} again, isn't it ? Please refer
[~vinayrpet]'s
[comment|https://issues.apache.org/jira/browse/HDFS-8450?focusedCommentId=14567220page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14567220]
# IMHO below function to be called after acquiring {{fsd.writeLock();}} lock.
Otherwise it will throw exception.
{code}
+ ListXAttr xAttrs = fsd.ecZoneManager.createErasureCodingZone(
+ src, schema, cellSize);
{code}
Erasure Coding: Consolidate erasure coding zone related implementation into a
single class
--
Key: HDFS-8450
URL: https://issues.apache.org/jira/browse/HDFS-8450
Project: Hadoop HDFS
Issue Type: Sub-task
Reporter: Rakesh R
Assignee: Rakesh R
Attachments: HDFS-8450-FYI.patch, HDFS-8450-HDFS-7285-00.patch,
HDFS-8450-HDFS-7285-01.patch, HDFS-8450-HDFS-7285-02.patch,
HDFS-8450-HDFS-7285-03.patch, HDFS-8450-HDFS-7285-04.patch,
HDFS-8450-HDFS-7285-05.patch
The idea is to follow the same pattern suggested by HDFS-7416. It is good to
consolidate all the erasure coding zone related implementations of
{{FSNamesystem}}. Here, proposing {{FSDirErasureCodingZoneOp}} class to have
functions to perform related erasure coding zone operations.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HDFS-8450) Erasure Coding: Consolidate erasure coding zone related implementation into a single class
[
https://issues.apache.org/jira/browse/HDFS-8450?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14569027#comment-14569027
]
Kai Zheng commented on HDFS-8450:
-
Hi Rakesh my FYI patch is just for your reference to illustrate some of rough
ideas. I thought you're right at above points, sorry if any confusing in that.
Erasure Coding: Consolidate erasure coding zone related implementation into a
single class
--
Key: HDFS-8450
URL: https://issues.apache.org/jira/browse/HDFS-8450
Project: Hadoop HDFS
Issue Type: Sub-task
Reporter: Rakesh R
Assignee: Rakesh R
Attachments: HDFS-8450-FYI.patch, HDFS-8450-HDFS-7285-00.patch,
HDFS-8450-HDFS-7285-01.patch, HDFS-8450-HDFS-7285-02.patch,
HDFS-8450-HDFS-7285-03.patch, HDFS-8450-HDFS-7285-04.patch,
HDFS-8450-HDFS-7285-05.patch
The idea is to follow the same pattern suggested by HDFS-7416. It is good to
consolidate all the erasure coding zone related implementations of
{{FSNamesystem}}. Here, proposing {{FSDirErasureCodingZoneOp}} class to have
functions to perform related erasure coding zone operations.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HDFS-8450) Erasure Coding: Consolidate erasure coding zone related implementation into a single class
[
https://issues.apache.org/jira/browse/HDFS-8450?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14569034#comment-14569034
]
Rakesh R commented on HDFS-8450:
Thanks for the reply. I will prepare another patch including {{function
naming}}, {{fsn}} arg and {{FSDirErasureCodingOp}}.
Erasure Coding: Consolidate erasure coding zone related implementation into a
single class
--
Key: HDFS-8450
URL: https://issues.apache.org/jira/browse/HDFS-8450
Project: Hadoop HDFS
Issue Type: Sub-task
Reporter: Rakesh R
Assignee: Rakesh R
Attachments: HDFS-8450-FYI.patch, HDFS-8450-HDFS-7285-00.patch,
HDFS-8450-HDFS-7285-01.patch, HDFS-8450-HDFS-7285-02.patch,
HDFS-8450-HDFS-7285-03.patch, HDFS-8450-HDFS-7285-04.patch,
HDFS-8450-HDFS-7285-05.patch
The idea is to follow the same pattern suggested by HDFS-7416. It is good to
consolidate all the erasure coding zone related implementations of
{{FSNamesystem}}. Here, proposing {{FSDirErasureCodingZoneOp}} class to have
functions to perform related erasure coding zone operations.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HDFS-8450) Erasure Coding: Consolidate erasure coding zone related implementation into a single class
[
https://issues.apache.org/jira/browse/HDFS-8450?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14568610#comment-14568610
]
Hadoop QA commented on HDFS-8450:
-
\\
\\
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:red}-1{color} | patch | 0m 0s | The patch command could not apply
the patch during dryrun. |
\\
\\
|| Subsystem || Report/Notes ||
| Patch URL |
http://issues.apache.org/jira/secure/attachment/12736754/HDFS-8450-FYI.patch |
| Optional Tests | javadoc javac unit findbugs checkstyle |
| git revision | trunk / 990078b |
| Console output |
https://builds.apache.org/job/PreCommit-HDFS-Build/11197/console |
This message was automatically generated.
Erasure Coding: Consolidate erasure coding zone related implementation into a
single class
--
Key: HDFS-8450
URL: https://issues.apache.org/jira/browse/HDFS-8450
Project: Hadoop HDFS
Issue Type: Sub-task
Reporter: Rakesh R
Assignee: Rakesh R
Attachments: HDFS-8450-FYI.patch, HDFS-8450-HDFS-7285-00.patch,
HDFS-8450-HDFS-7285-01.patch, HDFS-8450-HDFS-7285-02.patch,
HDFS-8450-HDFS-7285-03.patch, HDFS-8450-HDFS-7285-04.patch,
HDFS-8450-HDFS-7285-05.patch
The idea is to follow the same pattern suggested by HDFS-7416. It is good to
consolidate all the erasure coding zone related implementations of
{{FSNamesystem}}. Here, proposing {{FSDirErasureCodingZoneOp}} class to have
functions to perform related erasure coding zone operations.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HDFS-8450) Erasure Coding: Consolidate erasure coding zone related implementation into a single class
[
https://issues.apache.org/jira/browse/HDFS-8450?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14566960#comment-14566960
]
Kai Zheng commented on HDFS-8450:
-
Or in other words I meant to get rid of the 6th parameter of the function.
Similar to other functions.
Erasure Coding: Consolidate erasure coding zone related implementation into a
single class
--
Key: HDFS-8450
URL: https://issues.apache.org/jira/browse/HDFS-8450
Project: Hadoop HDFS
Issue Type: Sub-task
Reporter: Rakesh R
Assignee: Rakesh R
Attachments: HDFS-8450-HDFS-7285-00.patch,
HDFS-8450-HDFS-7285-01.patch, HDFS-8450-HDFS-7285-02.patch,
HDFS-8450-HDFS-7285-03.patch
The idea is to follow the same pattern suggested by HDFS-7416. It is good to
consolidate all the erasure coding zone related implementations of
{{FSNamesystem}}. Here, proposing {{FSDirErasureCodingZoneOp}} class to have
functions to perform related erasure coding zone operations.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HDFS-8450) Erasure Coding: Consolidate erasure coding zone related implementation into a single class
[
https://issues.apache.org/jira/browse/HDFS-8450?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14566970#comment-14566970
]
Rakesh R commented on HDFS-8450:
Thanks [~drankye], IIUC {{updateBlocks}} function takes {{ecSchema}} as one of
the argument, so he can determine {{ecSchema != null}} inside the function
implementation.
Erasure Coding: Consolidate erasure coding zone related implementation into a
single class
--
Key: HDFS-8450
URL: https://issues.apache.org/jira/browse/HDFS-8450
Project: Hadoop HDFS
Issue Type: Sub-task
Reporter: Rakesh R
Assignee: Rakesh R
Attachments: HDFS-8450-HDFS-7285-00.patch,
HDFS-8450-HDFS-7285-01.patch, HDFS-8450-HDFS-7285-02.patch,
HDFS-8450-HDFS-7285-03.patch
The idea is to follow the same pattern suggested by HDFS-7416. It is good to
consolidate all the erasure coding zone related implementations of
{{FSNamesystem}}. Here, proposing {{FSDirErasureCodingZoneOp}} class to have
functions to perform related erasure coding zone operations.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HDFS-8450) Erasure Coding: Consolidate erasure coding zone related implementation into a single class
[
https://issues.apache.org/jira/browse/HDFS-8450?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14566972#comment-14566972
]
Rakesh R commented on HDFS-8450:
I think this is good idea by avoiding unnecessary argument.
Erasure Coding: Consolidate erasure coding zone related implementation into a
single class
--
Key: HDFS-8450
URL: https://issues.apache.org/jira/browse/HDFS-8450
Project: Hadoop HDFS
Issue Type: Sub-task
Reporter: Rakesh R
Assignee: Rakesh R
Attachments: HDFS-8450-HDFS-7285-00.patch,
HDFS-8450-HDFS-7285-01.patch, HDFS-8450-HDFS-7285-02.patch,
HDFS-8450-HDFS-7285-03.patch
The idea is to follow the same pattern suggested by HDFS-7416. It is good to
consolidate all the erasure coding zone related implementations of
{{FSNamesystem}}. Here, proposing {{FSDirErasureCodingZoneOp}} class to have
functions to perform related erasure coding zone operations.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HDFS-8450) Erasure Coding: Consolidate erasure coding zone related implementation into a single class
[
https://issues.apache.org/jira/browse/HDFS-8450?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14566973#comment-14566973
]
Kai Zheng commented on HDFS-8450:
-
Yeah, thanks for making sure about this.
Erasure Coding: Consolidate erasure coding zone related implementation into a
single class
--
Key: HDFS-8450
URL: https://issues.apache.org/jira/browse/HDFS-8450
Project: Hadoop HDFS
Issue Type: Sub-task
Reporter: Rakesh R
Assignee: Rakesh R
Attachments: HDFS-8450-HDFS-7285-00.patch,
HDFS-8450-HDFS-7285-01.patch, HDFS-8450-HDFS-7285-02.patch,
HDFS-8450-HDFS-7285-03.patch
The idea is to follow the same pattern suggested by HDFS-7416. It is good to
consolidate all the erasure coding zone related implementations of
{{FSNamesystem}}. Here, proposing {{FSDirErasureCodingZoneOp}} class to have
functions to perform related erasure coding zone operations.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HDFS-8450) Erasure Coding: Consolidate erasure coding zone related implementation into a single class
[
https://issues.apache.org/jira/browse/HDFS-8450?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14567220#comment-14567220
]
Vinayakumar B commented on HDFS-8450:
-
Hi [~rakeshr], few comments.
1. In {{FSNamesystem#createErasureCodingZone(..)}} Need to remove duplicate
calls after finally block.
{code}
+ if (success) {
+getEditLog().logSync();
+ }
+ logAuditEvent(success, createErasureCodingZone, srcArg, null,
+ resultingStat);
}
getEditLog().logSync();
logAuditEvent(success, createErasureCodingZone, srcArg, null,
resultingStat);
{code}
2. In {{FSDirErasureCodingZoneOp.getErasureCodingZone(..)}},
{{fsn.checkOperation(OperationCategory.READ);}} missing after aquiring the
readlock. Similar check for other methods moved from FSNamesystem.
+1 once addressed.
Erasure Coding: Consolidate erasure coding zone related implementation into a
single class
--
Key: HDFS-8450
URL: https://issues.apache.org/jira/browse/HDFS-8450
Project: Hadoop HDFS
Issue Type: Sub-task
Reporter: Rakesh R
Assignee: Rakesh R
Attachments: HDFS-8450-HDFS-7285-00.patch,
HDFS-8450-HDFS-7285-01.patch, HDFS-8450-HDFS-7285-02.patch,
HDFS-8450-HDFS-7285-03.patch, HDFS-8450-HDFS-7285-04.patch
The idea is to follow the same pattern suggested by HDFS-7416. It is good to
consolidate all the erasure coding zone related implementations of
{{FSNamesystem}}. Here, proposing {{FSDirErasureCodingZoneOp}} class to have
functions to perform related erasure coding zone operations.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HDFS-8450) Erasure Coding: Consolidate erasure coding zone related implementation into a single class
[
https://issues.apache.org/jira/browse/HDFS-8450?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14567185#comment-14567185
]
Hadoop QA commented on HDFS-8450:
-
\\
\\
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:red}-1{color} | pre-patch | 15m 5s | Findbugs (version ) appears to
be broken on HDFS-7285. |
| {color:green}+1{color} | @author | 0m 0s | The patch does not contain any
@author tags. |
| {color:red}-1{color} | tests included | 0m 0s | The patch doesn't appear
to include any new or modified tests. Please justify why no new tests are
needed for this patch. Also please list what manual steps were performed to
verify this patch. |
| {color:green}+1{color} | javac | 7m 31s | There were no new javac warning
messages. |
| {color:green}+1{color} | javadoc | 9m 43s | There were no new javadoc
warning messages. |
| {color:red}-1{color} | release audit | 0m 15s | The applied patch generated
1 release audit warnings. |
| {color:green}+1{color} | checkstyle | 0m 37s | There were no new checkstyle
issues. |
| {color:green}+1{color} | whitespace | 0m 1s | The patch has no lines that
end in whitespace. |
| {color:green}+1{color} | install | 1m 37s | mvn install still works. |
| {color:green}+1{color} | eclipse:eclipse | 0m 34s | The patch built with
eclipse:eclipse. |
| {color:red}-1{color} | findbugs | 3m 22s | The patch appears to introduce 1
new Findbugs (version 3.0.0) warnings. |
| {color:green}+1{color} | native | 3m 14s | Pre-build of native portion |
| {color:red}-1{color} | hdfs tests | 172m 53s | Tests failed in hadoop-hdfs. |
| | | 214m 57s | |
\\
\\
|| Reason || Tests ||
| FindBugs | module:hadoop-hdfs |
| Failed unit tests | hadoop.hdfs.TestEncryptedTransfer |
| | hadoop.hdfs.TestDFSStripedInputStream |
| | hadoop.hdfs.server.blockmanagement.TestBlockTokenWithDFS |
| | hadoop.hdfs.TestRecoverStripedFile |
| | hadoop.hdfs.server.namenode.TestAuditLogs |
| | hadoop.hdfs.server.blockmanagement.TestBlockInfo |
| | hadoop.hdfs.server.namenode.TestFileTruncate |
\\
\\
|| Subsystem || Report/Notes ||
| Patch URL |
http://issues.apache.org/jira/secure/attachment/12736508/HDFS-8450-HDFS-7285-04.patch
|
| Optional Tests | javadoc javac unit findbugs checkstyle |
| git revision | HDFS-7285 / 1299357 |
| Release Audit |
https://builds.apache.org/job/PreCommit-HDFS-Build/11180/artifact/patchprocess/patchReleaseAuditProblems.txt
|
| Findbugs warnings |
https://builds.apache.org/job/PreCommit-HDFS-Build/11180/artifact/patchprocess/newPatchFindbugsWarningshadoop-hdfs.html
|
| hadoop-hdfs test log |
https://builds.apache.org/job/PreCommit-HDFS-Build/11180/artifact/patchprocess/testrun_hadoop-hdfs.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-HDFS-Build/11180/testReport/ |
| Java | 1.7.0_55 |
| uname | Linux asf909.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP
PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |
| Console output |
https://builds.apache.org/job/PreCommit-HDFS-Build/11180/console |
This message was automatically generated.
Erasure Coding: Consolidate erasure coding zone related implementation into a
single class
--
Key: HDFS-8450
URL: https://issues.apache.org/jira/browse/HDFS-8450
Project: Hadoop HDFS
Issue Type: Sub-task
Reporter: Rakesh R
Assignee: Rakesh R
Attachments: HDFS-8450-HDFS-7285-00.patch,
HDFS-8450-HDFS-7285-01.patch, HDFS-8450-HDFS-7285-02.patch,
HDFS-8450-HDFS-7285-03.patch, HDFS-8450-HDFS-7285-04.patch
The idea is to follow the same pattern suggested by HDFS-7416. It is good to
consolidate all the erasure coding zone related implementations of
{{FSNamesystem}}. Here, proposing {{FSDirErasureCodingZoneOp}} class to have
functions to perform related erasure coding zone operations.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HDFS-8450) Erasure Coding: Consolidate erasure coding zone related implementation into a single class
[
https://issues.apache.org/jira/browse/HDFS-8450?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14567374#comment-14567374
]
Rakesh R commented on HDFS-8450:
Thanks a lot [~vinayrpet] for the review comments. Attached another patch
addressing these comments.
Erasure Coding: Consolidate erasure coding zone related implementation into a
single class
--
Key: HDFS-8450
URL: https://issues.apache.org/jira/browse/HDFS-8450
Project: Hadoop HDFS
Issue Type: Sub-task
Reporter: Rakesh R
Assignee: Rakesh R
Attachments: HDFS-8450-HDFS-7285-00.patch,
HDFS-8450-HDFS-7285-01.patch, HDFS-8450-HDFS-7285-02.patch,
HDFS-8450-HDFS-7285-03.patch, HDFS-8450-HDFS-7285-04.patch,
HDFS-8450-HDFS-7285-05.patch
The idea is to follow the same pattern suggested by HDFS-7416. It is good to
consolidate all the erasure coding zone related implementations of
{{FSNamesystem}}. Here, proposing {{FSDirErasureCodingZoneOp}} class to have
functions to perform related erasure coding zone operations.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HDFS-8450) Erasure Coding: Consolidate erasure coding zone related implementation into a single class
[
https://issues.apache.org/jira/browse/HDFS-8450?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14567745#comment-14567745
]
Rakesh R commented on HDFS-8450:
+Note:+ It looks like the test case failures are not related to my patch. Many
are failing due to some strange exception like below:
{code}
java.lang.NoClassDefFoundError: org/apache/hadoop/io/erasurecode/ECSchema
{code}
Erasure Coding: Consolidate erasure coding zone related implementation into a
single class
--
Key: HDFS-8450
URL: https://issues.apache.org/jira/browse/HDFS-8450
Project: Hadoop HDFS
Issue Type: Sub-task
Reporter: Rakesh R
Assignee: Rakesh R
Attachments: HDFS-8450-HDFS-7285-00.patch,
HDFS-8450-HDFS-7285-01.patch, HDFS-8450-HDFS-7285-02.patch,
HDFS-8450-HDFS-7285-03.patch, HDFS-8450-HDFS-7285-04.patch,
HDFS-8450-HDFS-7285-05.patch
The idea is to follow the same pattern suggested by HDFS-7416. It is good to
consolidate all the erasure coding zone related implementations of
{{FSNamesystem}}. Here, proposing {{FSDirErasureCodingZoneOp}} class to have
functions to perform related erasure coding zone operations.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HDFS-8450) Erasure Coding: Consolidate erasure coding zone related implementation into a single class
[
https://issues.apache.org/jira/browse/HDFS-8450?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14567680#comment-14567680
]
Hadoop QA commented on HDFS-8450:
-
\\
\\
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:red}-1{color} | pre-patch | 15m 19s | Findbugs (version ) appears to
be broken on HDFS-7285. |
| {color:green}+1{color} | @author | 0m 0s | The patch does not contain any
@author tags. |
| {color:red}-1{color} | tests included | 0m 0s | The patch doesn't appear
to include any new or modified tests. Please justify why no new tests are
needed for this patch. Also please list what manual steps were performed to
verify this patch. |
| {color:green}+1{color} | javac | 7m 49s | There were no new javac warning
messages. |
| {color:green}+1{color} | javadoc | 9m 51s | There were no new javadoc
warning messages. |
| {color:red}-1{color} | release audit | 0m 15s | The applied patch generated
1 release audit warnings. |
| {color:green}+1{color} | checkstyle | 0m 38s | There were no new checkstyle
issues. |
| {color:green}+1{color} | whitespace | 0m 1s | The patch has no lines that
end in whitespace. |
| {color:green}+1{color} | install | 1m 36s | mvn install still works. |
| {color:green}+1{color} | eclipse:eclipse | 0m 34s | The patch built with
eclipse:eclipse. |
| {color:red}-1{color} | findbugs | 3m 25s | The patch appears to introduce 1
new Findbugs (version 3.0.0) warnings. |
| {color:green}+1{color} | native | 3m 14s | Pre-build of native portion |
| {color:red}-1{color} | hdfs tests | 170m 45s | Tests failed in hadoop-hdfs. |
| | | 213m 32s | |
\\
\\
|| Reason || Tests ||
| FindBugs | module:hadoop-hdfs |
| Failed unit tests | hadoop.hdfs.TestAppendSnapshotTruncate |
| | hadoop.cli.TestErasureCodingCLI |
| | hadoop.TestRefreshCallQueue |
| | hadoop.hdfs.TestListFilesInDFS |
| | hadoop.hdfs.TestEncryptedTransfer |
| | hadoop.security.TestPermissionSymlinks |
| | hadoop.hdfs.TestFileAppend2 |
| | hadoop.security.TestRefreshUserMappings |
| | hadoop.hdfs.TestRecoverStripedFile |
| | hadoop.hdfs.server.namenode.TestAuditLogs |
| | hadoop.hdfs.TestBlockStoragePolicy |
| | hadoop.hdfs.TestCrcCorruption |
| | hadoop.hdfs.server.blockmanagement.TestBlockInfo |
| | hadoop.hdfs.TestDFSUpgrade |
| | hadoop.cli.TestXAttrCLI |
| | hadoop.hdfs.TestDFSShell |
| | hadoop.hdfs.server.namenode.TestFileTruncate |
| | hadoop.security.TestPermission |
| | hadoop.hdfs.TestFileCreationDelete |
| | hadoop.hdfs.server.blockmanagement.TestBlockTokenWithDFS |
| | hadoop.hdfs.TestRestartDFS |
| | hadoop.cli.TestHDFSCLI |
| | hadoop.hdfs.TestHDFSFileSystemContract |
| | hadoop.cli.TestAclCLI |
| | hadoop.hdfs.TestDFSPermission |
| | hadoop.hdfs.TestHDFSServerPorts |
| | hadoop.cli.TestCryptoAdminCLI |
| | hadoop.hdfs.TestFetchImage |
| | hadoop.cli.TestCacheAdminCLI |
| | hadoop.hdfs.TestDatanodeDeath |
| | hadoop.hdfs.TestDFSClientExcludedNodes |
| | hadoop.hdfs.TestDFSOutputStream |
| | hadoop.hdfs.TestDFSStripedInputStream |
| Timed out tests | org.apache.hadoop.hdfs.TestFileAppendRestart |
\\
\\
|| Subsystem || Report/Notes ||
| Patch URL |
http://issues.apache.org/jira/secure/attachment/12736574/HDFS-8450-HDFS-7285-05.patch
|
| Optional Tests | javadoc javac unit findbugs checkstyle |
| git revision | HDFS-7285 / 1299357 |
| Release Audit |
https://builds.apache.org/job/PreCommit-HDFS-Build/11188/artifact/patchprocess/patchReleaseAuditProblems.txt
|
| Findbugs warnings |
https://builds.apache.org/job/PreCommit-HDFS-Build/11188/artifact/patchprocess/newPatchFindbugsWarningshadoop-hdfs.html
|
| hadoop-hdfs test log |
https://builds.apache.org/job/PreCommit-HDFS-Build/11188/artifact/patchprocess/testrun_hadoop-hdfs.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-HDFS-Build/11188/testReport/ |
| Java | 1.7.0_55 |
| uname | Linux asf903.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP
PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |
| Console output |
https://builds.apache.org/job/PreCommit-HDFS-Build/11188/console |
This message was automatically generated.
Erasure Coding: Consolidate erasure coding zone related implementation into a
single class
--
Key: HDFS-8450
URL: https://issues.apache.org/jira/browse/HDFS-8450
Project: Hadoop HDFS
Issue Type: Sub-task
Reporter: Rakesh R
Assignee: Rakesh R
Attachments: HDFS-8450-HDFS-7285-00.patch,
HDFS-8450-HDFS-7285-01.patch, HDFS-8450-HDFS-7285-02.patch,
HDFS-8450-HDFS-7285-03.patch, HDFS-8450-HDFS-7285-04.patch,
HDFS-8450-HDFS-7285-05.patch
The idea is to follow the same pattern suggested by HDFS-7416. It is
[jira] [Commented] (HDFS-8450) Erasure Coding: Consolidate erasure coding zone related implementation into a single class
[
https://issues.apache.org/jira/browse/HDFS-8450?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14568369#comment-14568369
]
Kai Zheng commented on HDFS-8450:
-
Sorry I'm late to follow the update and will provide more comments today.
Thanks.
Erasure Coding: Consolidate erasure coding zone related implementation into a
single class
--
Key: HDFS-8450
URL: https://issues.apache.org/jira/browse/HDFS-8450
Project: Hadoop HDFS
Issue Type: Sub-task
Reporter: Rakesh R
Assignee: Rakesh R
Attachments: HDFS-8450-HDFS-7285-00.patch,
HDFS-8450-HDFS-7285-01.patch, HDFS-8450-HDFS-7285-02.patch,
HDFS-8450-HDFS-7285-03.patch, HDFS-8450-HDFS-7285-04.patch,
HDFS-8450-HDFS-7285-05.patch
The idea is to follow the same pattern suggested by HDFS-7416. It is good to
consolidate all the erasure coding zone related implementations of
{{FSNamesystem}}. Here, proposing {{FSDirErasureCodingZoneOp}} class to have
functions to perform related erasure coding zone operations.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HDFS-8450) Erasure Coding: Consolidate erasure coding zone related implementation into a single class
[
https://issues.apache.org/jira/browse/HDFS-8450?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14566922#comment-14566922
]
Rakesh R commented on HDFS-8450:
{code}
+ updateBlocks(fsDir, updateOp, iip, oldFile, ecSchema,
+ FSDirErasureCodingZoneOp.getECSchema(fsDir, iip) !=null);
{code}
Erasure Coding: Consolidate erasure coding zone related implementation into a
single class
--
Key: HDFS-8450
URL: https://issues.apache.org/jira/browse/HDFS-8450
Project: Hadoop HDFS
Issue Type: Sub-task
Reporter: Rakesh R
Assignee: Rakesh R
Attachments: HDFS-8450-HDFS-7285-00.patch,
HDFS-8450-HDFS-7285-01.patch, HDFS-8450-HDFS-7285-02.patch,
HDFS-8450-HDFS-7285-03.patch
The idea is to follow the same pattern suggested by HDFS-7416. It is good to
consolidate all the erasure coding zone related implementations of
{{FSNamesystem}}. Here, proposing {{FSDirErasureCodingZoneOp}} class to have
functions to perform related erasure coding zone operations.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HDFS-8450) Erasure Coding: Consolidate erasure coding zone related implementation into a single class
[
https://issues.apache.org/jira/browse/HDFS-8450?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14566803#comment-14566803
]
Kai Zheng commented on HDFS-8450:
-
Thanks Rakesh for the update. A few more comments:
1. Do we have to need the function now?
{code}
+ private static ListXAttr createErasureCodingZone(final ECSchema schema,
+ final int cellSize, final String src, final FSDirectory fsd)
+ throws IOException {
+fsd.writeLock();
+try {
+ return fsd.ecZoneManager.createErasureCodingZone(src, schema, cellSize);
+} finally {
+ fsd.writeUnlock();
+}
+ }
{code}
2. Maybe getECSchema = getErasureCodingSchema as we did for others? Similar to
getECSchemas.
{code}
+ static ECSchema getECSchema(final FSDirectory fsd, final INodesInPath iip)
{code}
3. Thanks for the following update. Maybe we could do further, getting rid of
the {{pathIsInErasureCodingZone}} parameter at all? I mean we could change the
signature of functions like updateBlocks and etc.
{code}
+ ECSchema ecSchema = FSDirErasureCodingZoneOp.getECSchema(fsDir, iip);
+ boolean pathIsInErasureCodingZone = ecSchema != null;
+ updateBlocks(fsDir, updateOp, iip, oldFile, ecSchema,
+ pathIsInErasureCodingZone);
{code}
Erasure Coding: Consolidate erasure coding zone related implementation into a
single class
--
Key: HDFS-8450
URL: https://issues.apache.org/jira/browse/HDFS-8450
Project: Hadoop HDFS
Issue Type: Sub-task
Reporter: Rakesh R
Assignee: Rakesh R
Attachments: HDFS-8450-HDFS-7285-00.patch,
HDFS-8450-HDFS-7285-01.patch, HDFS-8450-HDFS-7285-02.patch,
HDFS-8450-HDFS-7285-03.patch
The idea is to follow the same pattern suggested by HDFS-7416. It is good to
consolidate all the erasure coding zone related implementations of
{{FSNamesystem}}. Here, proposing {{FSDirErasureCodingZoneOp}} class to have
functions to perform related erasure coding zone operations.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HDFS-8450) Erasure Coding: Consolidate erasure coding zone related implementation into a single class
[
https://issues.apache.org/jira/browse/HDFS-8450?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14566921#comment-14566921
]
Rakesh R commented on HDFS-8450:
Thanks [~drankye] for the reviews. Do you meant to refactor the function call
like below to avoid {{pathIsInErasureCodingZone}} parameter creation.
+ updateBlocks(fsDir, updateOp, iip, oldFile, ecSchema,
+ FSDirErasureCodingZoneOp.getECSchema(fsDir, iip) !=null);
Erasure Coding: Consolidate erasure coding zone related implementation into a
single class
--
Key: HDFS-8450
URL: https://issues.apache.org/jira/browse/HDFS-8450
Project: Hadoop HDFS
Issue Type: Sub-task
Reporter: Rakesh R
Assignee: Rakesh R
Attachments: HDFS-8450-HDFS-7285-00.patch,
HDFS-8450-HDFS-7285-01.patch, HDFS-8450-HDFS-7285-02.patch,
HDFS-8450-HDFS-7285-03.patch
The idea is to follow the same pattern suggested by HDFS-7416. It is good to
consolidate all the erasure coding zone related implementations of
{{FSNamesystem}}. Here, proposing {{FSDirErasureCodingZoneOp}} class to have
functions to perform related erasure coding zone operations.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HDFS-8450) Erasure Coding: Consolidate erasure coding zone related implementation into a single class
[
https://issues.apache.org/jira/browse/HDFS-8450?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14564425#comment-14564425
]
Rakesh R commented on HDFS-8450:
Attached another patch addressing [~drankye] comments.
Erasure Coding: Consolidate erasure coding zone related implementation into a
single class
--
Key: HDFS-8450
URL: https://issues.apache.org/jira/browse/HDFS-8450
Project: Hadoop HDFS
Issue Type: Sub-task
Reporter: Rakesh R
Assignee: Rakesh R
Attachments: HDFS-8450-HDFS-7285-00.patch,
HDFS-8450-HDFS-7285-01.patch, HDFS-8450-HDFS-7285-02.patch,
HDFS-8450-HDFS-7285-03.patch
The idea is to follow the same pattern suggested by HDFS-7416. It is good to
consolidate all the erasure coding zone related implementations of
{{FSNamesystem}}. Here, proposing {{FSDirErasureCodingZoneOp}} class to have
functions to perform related erasure coding zone operations.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HDFS-8450) Erasure Coding: Consolidate erasure coding zone related implementation into a single class
[
https://issues.apache.org/jira/browse/HDFS-8450?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14564597#comment-14564597
]
Hadoop QA commented on HDFS-8450:
-
\\
\\
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:red}-1{color} | pre-patch | 15m 28s | Findbugs (version ) appears to
be broken on HDFS-7285. |
| {color:green}+1{color} | @author | 0m 0s | The patch does not contain any
@author tags. |
| {color:red}-1{color} | tests included | 0m 0s | The patch doesn't appear
to include any new or modified tests. Please justify why no new tests are
needed for this patch. Also please list what manual steps were performed to
verify this patch. |
| {color:green}+1{color} | javac | 8m 38s | There were no new javac warning
messages. |
| {color:green}+1{color} | javadoc | 11m 37s | There were no new javadoc
warning messages. |
| {color:red}-1{color} | release audit | 0m 16s | The applied patch generated
1 release audit warnings. |
| {color:green}+1{color} | checkstyle | 0m 41s | There were no new checkstyle
issues. |
| {color:green}+1{color} | whitespace | 0m 1s | The patch has no lines that
end in whitespace. |
| {color:green}+1{color} | install | 1m 55s | mvn install still works. |
| {color:green}+1{color} | eclipse:eclipse | 0m 36s | The patch built with
eclipse:eclipse. |
| {color:red}-1{color} | findbugs | 4m 3s | The patch appears to introduce 1
new Findbugs (version 3.0.0) warnings. |
| {color:green}+1{color} | native | 3m 57s | Pre-build of native portion |
| {color:red}-1{color} | hdfs tests | 105m 47s | Tests failed in hadoop-hdfs. |
| | | 153m 6s | |
\\
\\
|| Reason || Tests ||
| FindBugs | module:hadoop-hdfs |
| Failed unit tests | hadoop.hdfs.TestEncryptedTransfer |
| | hadoop.hdfs.server.namenode.TestAuditLogs |
| | hadoop.hdfs.server.namenode.TestFileTruncate |
| | hadoop.hdfs.TestDFSStripedInputStream |
| Timed out tests | org.apache.hadoop.hdfs.TestInjectionForSimulatedStorage |
\\
\\
|| Subsystem || Report/Notes ||
| Patch URL |
http://issues.apache.org/jira/secure/attachment/12736087/HDFS-8450-HDFS-7285-03.patch
|
| Optional Tests | javadoc javac unit findbugs checkstyle |
| git revision | HDFS-7285 / 1299357 |
| Release Audit |
https://builds.apache.org/job/PreCommit-HDFS-Build/11163/artifact/patchprocess/patchReleaseAuditProblems.txt
|
| Findbugs warnings |
https://builds.apache.org/job/PreCommit-HDFS-Build/11163/artifact/patchprocess/newPatchFindbugsWarningshadoop-hdfs.html
|
| hadoop-hdfs test log |
https://builds.apache.org/job/PreCommit-HDFS-Build/11163/artifact/patchprocess/testrun_hadoop-hdfs.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-HDFS-Build/11163/testReport/ |
| Java | 1.7.0_55 |
| uname | Linux asf903.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP
PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |
| Console output |
https://builds.apache.org/job/PreCommit-HDFS-Build/11163/console |
This message was automatically generated.
Erasure Coding: Consolidate erasure coding zone related implementation into a
single class
--
Key: HDFS-8450
URL: https://issues.apache.org/jira/browse/HDFS-8450
Project: Hadoop HDFS
Issue Type: Sub-task
Reporter: Rakesh R
Assignee: Rakesh R
Attachments: HDFS-8450-HDFS-7285-00.patch,
HDFS-8450-HDFS-7285-01.patch, HDFS-8450-HDFS-7285-02.patch,
HDFS-8450-HDFS-7285-03.patch
The idea is to follow the same pattern suggested by HDFS-7416. It is good to
consolidate all the erasure coding zone related implementations of
{{FSNamesystem}}. Here, proposing {{FSDirErasureCodingZoneOp}} class to have
functions to perform related erasure coding zone operations.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HDFS-8450) Erasure Coding: Consolidate erasure coding zone related implementation into a single class
[
https://issues.apache.org/jira/browse/HDFS-8450?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14565813#comment-14565813
]
Rakesh R commented on HDFS-8450:
[~drankye] I hope I've addressed your comments. Could you please review the
patch again when you get a chance. Thanks!
Erasure Coding: Consolidate erasure coding zone related implementation into a
single class
--
Key: HDFS-8450
URL: https://issues.apache.org/jira/browse/HDFS-8450
Project: Hadoop HDFS
Issue Type: Sub-task
Reporter: Rakesh R
Assignee: Rakesh R
Attachments: HDFS-8450-HDFS-7285-00.patch,
HDFS-8450-HDFS-7285-01.patch, HDFS-8450-HDFS-7285-02.patch,
HDFS-8450-HDFS-7285-03.patch
The idea is to follow the same pattern suggested by HDFS-7416. It is good to
consolidate all the erasure coding zone related implementations of
{{FSNamesystem}}. Here, proposing {{FSDirErasureCodingZoneOp}} class to have
functions to perform related erasure coding zone operations.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HDFS-8450) Erasure Coding: Consolidate erasure coding zone related implementation into a single class
[
https://issues.apache.org/jira/browse/HDFS-8450?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14564151#comment-14564151
]
Rakesh R commented on HDFS-8450:
Thanks a lot [~drankye] for the reviews. I will address these comments.
Erasure Coding: Consolidate erasure coding zone related implementation into a
single class
--
Key: HDFS-8450
URL: https://issues.apache.org/jira/browse/HDFS-8450
Project: Hadoop HDFS
Issue Type: Sub-task
Reporter: Rakesh R
Assignee: Rakesh R
Attachments: HDFS-8450-HDFS-7285-00.patch,
HDFS-8450-HDFS-7285-01.patch, HDFS-8450-HDFS-7285-02.patch
The idea is to follow the same pattern suggested by HDFS-7416. It is good to
consolidate all the erasure coding zone related implementations of
{{FSNamesystem}}. Here, proposing {{FSDirErasureCodingZoneOp}} class to have
functions to perform related erasure coding zone operations.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HDFS-8450) Erasure Coding: Consolidate erasure coding zone related implementation into a single class
[
https://issues.apache.org/jira/browse/HDFS-8450?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14563164#comment-14563164
]
Kai Zheng commented on HDFS-8450:
-
Nice work Rakesh! The patch looks good. Some comments and suggestions:
1. In {{createErasureCodingZone}} and related helper functions, would you check
to ensure least required calls are made, for checkSuperuserPrivilege,
checkOperation, writeLock and etc.?
2. Maybe {{ErasureCodingZoneManager#createErasureCodingZone}} could return a
list directly?
{code}
+ private static ListXAttr createErasureCodingZone(final ECSchema schema,
+ int cellSize, String src, FSDirectory fsd) throws IOException {
+fsd.writeLock();
+ListXAttr xAttrs = Lists.newArrayListWithCapacity(1);
+try {
+ final XAttr ecXAttr = fsd.ecZoneManager.createErasureCodingZone(src,
+ schema, cellSize);
+ xAttrs.add(ecXAttr);
+} finally {
+ fsd.writeUnlock();
+}
+return xAttrs;
+ }
{code}
3. Gets the ECZone info for path. Gets = Get
4. I guess it would be a good chance to clean up the messy function names here,
either following {{createErasureCodingZone}} or {{getECZoneInfo}}.
5. In similar places like below, maybe we could avoid the call {{isInECZone}}
because if {{getECSchema}} returns non-null then it's true otherwise it's false.
{code}
+ FSDirErasureCodingZoneOp.getECSchema(fsDir, iip),
+ FSDirErasureCodingZoneOp.isInECZone(fsDir, iip));
{code}
Erasure Coding: Consolidate erasure coding zone related implementation into a
single class
--
Key: HDFS-8450
URL: https://issues.apache.org/jira/browse/HDFS-8450
Project: Hadoop HDFS
Issue Type: Sub-task
Reporter: Rakesh R
Assignee: Rakesh R
Attachments: HDFS-8450-HDFS-7285-00.patch,
HDFS-8450-HDFS-7285-01.patch, HDFS-8450-HDFS-7285-02.patch
The idea is to follow the same pattern suggested by HDFS-7416. It is good to
consolidate all the erasure coding zone related implementations of
{{FSNamesystem}}. Here, proposing {{FSDirErasureCodingZoneOp}} class to have
functions to perform related erasure coding zone operations.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HDFS-8450) Erasure Coding: Consolidate erasure coding zone related implementation into a single class
[
https://issues.apache.org/jira/browse/HDFS-8450?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14555706#comment-14555706
]
Rakesh R commented on HDFS-8450:
Test case failures, findbug, release audit warnings are unrelated to this
patch. Appreciate reviews. Thanks!
Erasure Coding: Consolidate erasure coding zone related implementation into a
single class
--
Key: HDFS-8450
URL: https://issues.apache.org/jira/browse/HDFS-8450
Project: Hadoop HDFS
Issue Type: Sub-task
Reporter: Rakesh R
Assignee: Rakesh R
Attachments: HDFS-8450-HDFS-7285-00.patch,
HDFS-8450-HDFS-7285-01.patch, HDFS-8450-HDFS-7285-02.patch
The idea is to follow the same pattern suggested by HDFS-7416. It is good to
consolidate all the erasure coding zone related implementations of
{{FSNamesystem}}. Here, proposing {{FSDirErasureCodingZoneOp}} class to have
functions to perform related erasure coding zone operations.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HDFS-8450) Erasure Coding: Consolidate erasure coding zone related implementation into a single class
[
https://issues.apache.org/jira/browse/HDFS-8450?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14554161#comment-14554161
]
Rakesh R commented on HDFS-8450:
Attached patch contains {{FSDirErasureCodingZoneOp}} utility class. I've tried
moving all the related erasure coding functions to this utility. Kindly review!
Erasure Coding: Consolidate erasure coding zone related implementation into a
single class
--
Key: HDFS-8450
URL: https://issues.apache.org/jira/browse/HDFS-8450
Project: Hadoop HDFS
Issue Type: Sub-task
Reporter: Rakesh R
Assignee: Rakesh R
Attachments: HDFS-8450-HDFS-7285-00.patch
The idea is to follow the same pattern suggested by HDFS-7416. It is good to
consolidate all the erasure coding zone related implementations of
{{FSNamesystem}}. Here, proposing {{FSDirErasureCodingZoneOp}} class to have
functions to perform related erasure coding zone operations.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HDFS-8450) Erasure Coding: Consolidate erasure coding zone related implementation into a single class
[
https://issues.apache.org/jira/browse/HDFS-8450?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14554199#comment-14554199
]
Rakesh R commented on HDFS-8450:
I haven't added any new test cases, as this is refactoring of existing logic to
new utility class. I think the existing unit test cases will verify the
functionality.
Erasure Coding: Consolidate erasure coding zone related implementation into a
single class
--
Key: HDFS-8450
URL: https://issues.apache.org/jira/browse/HDFS-8450
Project: Hadoop HDFS
Issue Type: Sub-task
Reporter: Rakesh R
Assignee: Rakesh R
Attachments: HDFS-8450-HDFS-7285-00.patch,
HDFS-8450-HDFS-7285-01.patch
The idea is to follow the same pattern suggested by HDFS-7416. It is good to
consolidate all the erasure coding zone related implementations of
{{FSNamesystem}}. Here, proposing {{FSDirErasureCodingZoneOp}} class to have
functions to perform related erasure coding zone operations.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HDFS-8450) Erasure Coding: Consolidate erasure coding zone related implementation into a single class
[
https://issues.apache.org/jira/browse/HDFS-8450?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14554517#comment-14554517
]
Hadoop QA commented on HDFS-8450:
-
\\
\\
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | pre-patch | 14m 36s | Pre-patch HDFS-7285 compilation
is healthy. |
| {color:green}+1{color} | @author | 0m 0s | The patch does not contain any
@author tags. |
| {color:red}-1{color} | tests included | 0m 0s | The patch doesn't appear
to include any new or modified tests. Please justify why no new tests are
needed for this patch. Also please list what manual steps were performed to
verify this patch. |
| {color:green}+1{color} | javac | 7m 30s | There were no new javac warning
messages. |
| {color:green}+1{color} | javadoc | 9m 38s | There were no new javadoc
warning messages. |
| {color:red}-1{color} | release audit | 0m 14s | The applied patch generated
1 release audit warnings. |
| {color:red}-1{color} | checkstyle | 2m 20s | The applied patch generated 6
new checkstyle issues (total was 451, now 451). |
| {color:red}-1{color} | whitespace | 0m 0s | The patch has 1 line(s) that
end in whitespace. Use git apply --whitespace=fix. |
| {color:green}+1{color} | install | 1m 37s | mvn install still works. |
| {color:green}+1{color} | eclipse:eclipse | 0m 32s | The patch built with
eclipse:eclipse. |
| {color:red}-1{color} | findbugs | 3m 12s | The patch appears to introduce 6
new Findbugs (version 3.0.0) warnings. |
| {color:green}+1{color} | native | 3m 16s | Pre-build of native portion |
| {color:red}-1{color} | hdfs tests | 167m 47s | Tests failed in hadoop-hdfs. |
| | | 210m 46s | |
\\
\\
|| Reason || Tests ||
| FindBugs | module:hadoop-hdfs |
| | Inconsistent synchronization of
org.apache.hadoop.hdfs.DFSOutputStream.streamer; locked 89% of time
Unsynchronized access at DFSOutputStream.java:89% of time Unsynchronized
access at DFSOutputStream.java:[line 146] |
| | Possible null pointer dereference of arr$ in
org.apache.hadoop.hdfs.server.blockmanagement.BlockInfoStripedUnderConstruction.initializeBlockRecovery(long)
Dereferenced at BlockInfoStripedUnderConstruction.java:arr$ in
org.apache.hadoop.hdfs.server.blockmanagement.BlockInfoStripedUnderConstruction.initializeBlockRecovery(long)
Dereferenced at BlockInfoStripedUnderConstruction.java:[line 194] |
| | Unread field:field be static? At ErasureCodingWorker.java:[line 254] |
| | Should
org.apache.hadoop.hdfs.server.datanode.erasurecode.ErasureCodingWorker$StripedReader
be a _static_ inner class? At ErasureCodingWorker.java:inner class? At
ErasureCodingWorker.java:[lines 907-914] |
| | Result of integer multiplication cast to long in
org.apache.hadoop.hdfs.util.StripedBlockUtil.constructInternalBlock(LocatedStripedBlock,
int, int, int, int) At StripedBlockUtil.java:to long in
org.apache.hadoop.hdfs.util.StripedBlockUtil.constructInternalBlock(LocatedStripedBlock,
int, int, int, int) At StripedBlockUtil.java:[line 108] |
| | Result of integer multiplication cast to long in
org.apache.hadoop.hdfs.util.StripedBlockUtil.getStartOffsetsForInternalBlocks(ECSchema,
int, LocatedStripedBlock, long) At StripedBlockUtil.java:to long in
org.apache.hadoop.hdfs.util.StripedBlockUtil.getStartOffsetsForInternalBlocks(ECSchema,
int, LocatedStripedBlock, long) At StripedBlockUtil.java:[line 409] |
| Failed unit tests | hadoop.hdfs.TestEncryptedTransfer |
| | hadoop.hdfs.server.namenode.TestQuotaWithStripedBlocks |
| | hadoop.hdfs.server.namenode.TestFSEditLogLoader |
| |
hadoop.hdfs.tools.offlineImageViewer.TestOfflineImageViewerWithStripedBlocks |
| | hadoop.hdfs.TestWriteReadStripedFile |
| | hadoop.hdfs.TestDFSStripedOutputStreamWithFailure |
| | hadoop.hdfs.TestDFSStripedOutputStream |
| | hadoop.hdfs.TestFileStatusWithECschema |
| | hadoop.hdfs.server.namenode.TestAddStripedBlocks |
| | hadoop.hdfs.server.namenode.TestFileTruncate |
| | hadoop.hdfs.server.namenode.TestFSImage |
| | hadoop.cli.TestErasureCodingCLI |
| | hadoop.hdfs.TestRecoverStripedFile |
| | hadoop.hdfs.server.namenode.TestRecoverStripedBlocks |
| | hadoop.hdfs.TestDFSStripedInputStream |
| | hadoop.hdfs.server.namenode.TestAuditLogs |
| | hadoop.hdfs.server.blockmanagement.TestBlockInfo |
| | hadoop.hdfs.server.blockmanagement.TestBlockTokenWithDFS |
| | hadoop.hdfs.TestErasureCodingZones |
\\
\\
|| Subsystem || Report/Notes ||
| Patch URL |
http://issues.apache.org/jira/secure/attachment/12734429/HDFS-8450-HDFS-7285-01.patch
|
| Optional Tests | javadoc javac unit findbugs checkstyle |
| git revision | HDFS-7285 / 9fdb5be |
| Release Audit |
https://builds.apache.org/job/PreCommit-HDFS-Build/11085/artifact/patchprocess/patchReleaseAuditProblems.txt
|
| checkstyle |
[jira] [Commented] (HDFS-8450) Erasure Coding: Consolidate erasure coding zone related implementation into a single class
[
https://issues.apache.org/jira/browse/HDFS-8450?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14555317#comment-14555317
]
Hadoop QA commented on HDFS-8450:
-
\\
\\
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | pre-patch | 14m 52s | Pre-patch HDFS-7285 compilation
is healthy. |
| {color:green}+1{color} | @author | 0m 0s | The patch does not contain any
@author tags. |
| {color:red}-1{color} | tests included | 0m 1s | The patch doesn't appear
to include any new or modified tests. Please justify why no new tests are
needed for this patch. Also please list what manual steps were performed to
verify this patch. |
| {color:green}+1{color} | javac | 7m 38s | There were no new javac warning
messages. |
| {color:green}+1{color} | javadoc | 9m 46s | There were no new javadoc
warning messages. |
| {color:red}-1{color} | release audit | 0m 16s | The applied patch generated
1 release audit warnings. |
| {color:green}+1{color} | checkstyle | 0m 40s | There were no new checkstyle
issues. |
| {color:green}+1{color} | whitespace | 0m 0s | The patch has no lines that
end in whitespace. |
| {color:green}+1{color} | install | 1m 40s | mvn install still works. |
| {color:green}+1{color} | eclipse:eclipse | 0m 36s | The patch built with
eclipse:eclipse. |
| {color:red}-1{color} | findbugs | 3m 14s | The patch appears to introduce 6
new Findbugs (version 3.0.0) warnings. |
| {color:green}+1{color} | native | 3m 18s | Pre-build of native portion |
| {color:red}-1{color} | hdfs tests | 173m 28s | Tests failed in hadoop-hdfs. |
| | | 215m 33s | |
\\
\\
|| Reason || Tests ||
| FindBugs | module:hadoop-hdfs |
| | Inconsistent synchronization of
org.apache.hadoop.hdfs.DFSOutputStream.streamer; locked 89% of time
Unsynchronized access at DFSOutputStream.java:89% of time Unsynchronized
access at DFSOutputStream.java:[line 146] |
| | Possible null pointer dereference of arr$ in
org.apache.hadoop.hdfs.server.blockmanagement.BlockInfoStripedUnderConstruction.initializeBlockRecovery(long)
Dereferenced at BlockInfoStripedUnderConstruction.java:arr$ in
org.apache.hadoop.hdfs.server.blockmanagement.BlockInfoStripedUnderConstruction.initializeBlockRecovery(long)
Dereferenced at BlockInfoStripedUnderConstruction.java:[line 194] |
| | Unread field:field be static? At ErasureCodingWorker.java:[line 254] |
| | Should
org.apache.hadoop.hdfs.server.datanode.erasurecode.ErasureCodingWorker$StripedReader
be a _static_ inner class? At ErasureCodingWorker.java:inner class? At
ErasureCodingWorker.java:[lines 907-914] |
| | Result of integer multiplication cast to long in
org.apache.hadoop.hdfs.util.StripedBlockUtil.constructInternalBlock(LocatedStripedBlock,
int, int, int, int) At StripedBlockUtil.java:to long in
org.apache.hadoop.hdfs.util.StripedBlockUtil.constructInternalBlock(LocatedStripedBlock,
int, int, int, int) At StripedBlockUtil.java:[line 108] |
| | Result of integer multiplication cast to long in
org.apache.hadoop.hdfs.util.StripedBlockUtil.getStartOffsetsForInternalBlocks(ECSchema,
int, LocatedStripedBlock, long) At StripedBlockUtil.java:to long in
org.apache.hadoop.hdfs.util.StripedBlockUtil.getStartOffsetsForInternalBlocks(ECSchema,
int, LocatedStripedBlock, long) At StripedBlockUtil.java:[line 409] |
| Failed unit tests | hadoop.hdfs.server.blockmanagement.TestBlockInfo |
| | hadoop.hdfs.server.blockmanagement.TestBlockTokenWithDFS |
| | hadoop.hdfs.server.namenode.TestAuditLogs |
| | hadoop.hdfs.server.namenode.TestFileTruncate |
| | hadoop.hdfs.TestEncryptedTransfer |
\\
\\
|| Subsystem || Report/Notes ||
| Patch URL |
http://issues.apache.org/jira/secure/attachment/12734630/HDFS-8450-HDFS-7285-02.patch
|
| Optional Tests | javadoc javac unit findbugs checkstyle |
| git revision | HDFS-7285 / 9fdb5be |
| Release Audit |
https://builds.apache.org/job/PreCommit-HDFS-Build/11088/artifact/patchprocess/patchReleaseAuditProblems.txt
|
| Findbugs warnings |
https://builds.apache.org/job/PreCommit-HDFS-Build/11088/artifact/patchprocess/newPatchFindbugsWarningshadoop-hdfs.html
|
| hadoop-hdfs test log |
https://builds.apache.org/job/PreCommit-HDFS-Build/11088/artifact/patchprocess/testrun_hadoop-hdfs.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-HDFS-Build/11088/testReport/ |
| Java | 1.7.0_55 |
| uname | Linux asf900.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP
PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |
| Console output |
https://builds.apache.org/job/PreCommit-HDFS-Build/11088/console |
This message was automatically generated.
Erasure Coding: Consolidate erasure coding zone related implementation into a
single class
--
Key:
