[jira] [Updated] (HDFS-11977) Ozone: cannot enable test debug/trace log
[
https://issues.apache.org/jira/browse/HDFS-11977?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Tsz Wo Nicholas Sze updated HDFS-11977:
---
Attachment: HDFS-11977-HDFS-7240.20170615.patch
> Ozone: cannot enable test debug/trace log
> -
>
> Key: HDFS-11977
> URL: https://issues.apache.org/jira/browse/HDFS-11977
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: ozone, test
>Reporter: Tsz Wo Nicholas Sze
>Assignee: Tsz Wo Nicholas Sze
>Priority: Minor
> Attachments: HDFS-11977-HDFS-7240.20170615.patch
>
>
> Interestingly, the test debug/trace logs are not printed for Ozone classes
> even if we invoke GenericTestUtils.setLogLevel(log, Level.ALL). Other
> classes such as Object do not have such problem. Here is a test:
> {code}
> @Test
> public void testLogLevel() throws Exception {
> runTestLogLevel(StorageContainerManager.class);
> runTestLogLevel(Object.class);
> }
> static void runTestLogLevel(Class clazz) throws Exception {
> final Logger log = LoggerFactory.getLogger(clazz);
> GenericTestUtils.setLogLevel(log, Level.ALL);
> log.trace(clazz.getSimpleName() + " trace log");
> log.debug(clazz.getSimpleName() + " debug log");
> log.info(clazz.getSimpleName() + " info log");
> log.warn(clazz.getSimpleName() + " warn log");
> log.error(clazz.getSimpleName() + " error log");
> }
> {code}
> Output:
> {code}
> 2017-06-15 00:19:07,133 [Thread-0] INFO - StorageContainerManager info
> log
> 2017-06-15 00:19:07,135 [Thread-0] WARN - StorageContainerManager warn
> log
> 2017-06-15 00:19:07,135 [Thread-0] ERROR - StorageContainerManager error
> log
> 2017-06-15 00:19:07,135 [Thread-0] TRACE
> lang.Object(TestOzoneContainer.java:runTestLogLevel(64)) - Object trace log
> 2017-06-15 00:19:07,135 [Thread-0] DEBUG
> lang.Object(TestOzoneContainer.java:runTestLogLevel(65)) - Object debug log
> 2017-06-15 00:19:07,135 [Thread-0] INFO
> lang.Object(TestOzoneContainer.java:runTestLogLevel(66)) - Object info log
> 2017-06-15 00:19:07,135 [Thread-0] WARN
> lang.Object(TestOzoneContainer.java:runTestLogLevel(67)) - Object warn log
> 2017-06-15 00:19:07,135 [Thread-0] ERROR
> lang.Object(TestOzoneContainer.java:runTestLogLevel(68)) - Object error log
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-8520) Patch for PPC64 block size
[ https://issues.apache.org/jira/browse/HDFS-8520?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16050080#comment-16050080 ] Ayappan commented on HDFS-8520: --- [~andrew.wang] What is the way going forward here ? > Patch for PPC64 block size > -- > > Key: HDFS-8520 > URL: https://issues.apache.org/jira/browse/HDFS-8520 > Project: Hadoop HDFS > Issue Type: Bug >Affects Versions: 2.7.1 > Environment: RHEL 7.1 /PPC64 >Reporter: Tony Reix >Assignee: Tony Reix > Labels: ppc64 > Attachments: HDFS-8520-2.patch, HDFS-8520.patch > > > The attached patch enables Hadoop to work on PPC64. > That deals with SystemPageSize and BloclSize , which are not 4096 on PPC64. > There are changes in 3 files: > - > hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/io/nativeio/NativeIO.java > - > hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/TestFsDatasetCache.java > - > hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestCacheDirectives.java > where 4096 is replaced by getOperatingSystemPageSize() or by using PAGE_SIZE > The patch has been built on branch-2.7 . -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Created] (HDFS-11978) Remove invalid '-usage' command of 'ec' and add missing commands 'addPolicies' 'listCodecs'
wenxin he created HDFS-11978: Summary: Remove invalid '-usage' command of 'ec' and add missing commands 'addPolicies' 'listCodecs' Key: HDFS-11978 URL: https://issues.apache.org/jira/browse/HDFS-11978 Project: Hadoop HDFS Issue Type: Bug Components: documentation Affects Versions: 3.0.0-alpha4 Reporter: wenxin he Priority: Minor Remove invalid '-usage' command of 'ec' in HDFSErasureCoding.md. Add missing commands 'addPolicies' 'listCodecs' in HDFSCommands.md. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-11978) Remove invalid '-usage' command of 'ec' and add missing commands 'addPolicies' 'listCodecs'
[ https://issues.apache.org/jira/browse/HDFS-11978?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16050092#comment-16050092 ] ASF GitHub Bot commented on HDFS-11978: --- GitHub user wenxinhe opened a pull request: https://github.com/apache/hadoop/pull/235 HDFS-11978. Remove invalid '-usage' command of 'ec' and add missing c… …ommands 'addPolicies' 'listCodecs' You can merge this pull request into a Git repository by running: $ git pull https://github.com/wenxinhe/hadoop HDFS-11978 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/hadoop/pull/235.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #235 commit 32b70d3b81a231c2373d85a3ff560d27c026061a Author: wenxin he Date: 2017-06-15T07:24:12Z HDFS-11978. Remove invalid '-usage' command of 'ec' and add missing commands 'addPolicies' 'listCodecs' > Remove invalid '-usage' command of 'ec' and add missing commands > 'addPolicies' 'listCodecs' > --- > > Key: HDFS-11978 > URL: https://issues.apache.org/jira/browse/HDFS-11978 > Project: Hadoop HDFS > Issue Type: Bug > Components: documentation >Affects Versions: 3.0.0-alpha4 >Reporter: wenxin he >Priority: Minor > > Remove invalid '-usage' command of 'ec' in HDFSErasureCoding.md. > Add missing commands 'addPolicies' 'listCodecs' in HDFSCommands.md. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-11978) Remove invalid '-usage' command of 'ec' and add missing commands 'addPolicies' 'listCodecs'
[ https://issues.apache.org/jira/browse/HDFS-11978?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] wenxin he updated HDFS-11978: - Status: Patch Available (was: Open) https://github.com/apache/hadoop/pull/235.patch > Remove invalid '-usage' command of 'ec' and add missing commands > 'addPolicies' 'listCodecs' > --- > > Key: HDFS-11978 > URL: https://issues.apache.org/jira/browse/HDFS-11978 > Project: Hadoop HDFS > Issue Type: Bug > Components: documentation >Affects Versions: 3.0.0-alpha4 >Reporter: wenxin he >Priority: Minor > > Remove invalid '-usage' command of 'ec' in HDFSErasureCoding.md. > Add missing commands 'addPolicies' 'listCodecs' in HDFSCommands.md. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Created] (HDFS-11979) Ozone: TestContainerPersistence never uses MiniOzoneCluster
Tsz Wo Nicholas Sze created HDFS-11979: -- Summary: Ozone: TestContainerPersistence never uses MiniOzoneCluster Key: HDFS-11979 URL: https://issues.apache.org/jira/browse/HDFS-11979 Project: Hadoop HDFS Issue Type: Sub-task Components: ozone, test Reporter: Tsz Wo Nicholas Sze Assignee: Tsz Wo Nicholas Sze Priority: Minor -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-11979) Ozone: TestContainerPersistence never uses MiniOzoneCluster
[ https://issues.apache.org/jira/browse/HDFS-11979?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tsz Wo Nicholas Sze updated HDFS-11979: --- Docs Text: (was: We should either remove MiniOzoneCluster from the test or somehow use it.) We should either remove MiniOzoneCluster from the test or somehow use it. > Ozone: TestContainerPersistence never uses MiniOzoneCluster > --- > > Key: HDFS-11979 > URL: https://issues.apache.org/jira/browse/HDFS-11979 > Project: Hadoop HDFS > Issue Type: Sub-task > Components: ozone, test >Reporter: Tsz Wo Nicholas Sze >Assignee: Tsz Wo Nicholas Sze >Priority: Minor > -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-11979) Ozone: TestContainerPersistence never uses MiniOzoneCluster
[ https://issues.apache.org/jira/browse/HDFS-11979?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tsz Wo Nicholas Sze updated HDFS-11979: --- Description: We should either remove MiniOzoneCluster from the test or somehow use it. > Ozone: TestContainerPersistence never uses MiniOzoneCluster > --- > > Key: HDFS-11979 > URL: https://issues.apache.org/jira/browse/HDFS-11979 > Project: Hadoop HDFS > Issue Type: Sub-task > Components: ozone, test >Reporter: Tsz Wo Nicholas Sze >Assignee: Tsz Wo Nicholas Sze >Priority: Minor > > We should either remove MiniOzoneCluster from the test or somehow use it. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-11979) Ozone: TestContainerPersistence never uses MiniOzoneCluster
[ https://issues.apache.org/jira/browse/HDFS-11979?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tsz Wo Nicholas Sze updated HDFS-11979: --- Attachment: HDFS-11979-HDFS-7240.20170615.patch > Ozone: TestContainerPersistence never uses MiniOzoneCluster > --- > > Key: HDFS-11979 > URL: https://issues.apache.org/jira/browse/HDFS-11979 > Project: Hadoop HDFS > Issue Type: Sub-task > Components: ozone, test >Reporter: Tsz Wo Nicholas Sze >Assignee: Tsz Wo Nicholas Sze >Priority: Minor > Attachments: HDFS-11979-HDFS-7240.20170615.patch > > > We should either remove MiniOzoneCluster from the test or somehow use it. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Comment Edited] (HDFS-11979) Ozone: TestContainerPersistence never uses MiniOzoneCluster
[ https://issues.apache.org/jira/browse/HDFS-11979?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16050103#comment-16050103 ] Tsz Wo Nicholas Sze edited comment on HDFS-11979 at 6/15/17 7:43 AM: - HDFS-11979-HDFS-7240.20170615.patch: remove MiniOzoneCluster from the test. was (Author: szetszwo): We should either remove MiniOzoneCluster from the test or somehow use it. > Ozone: TestContainerPersistence never uses MiniOzoneCluster > --- > > Key: HDFS-11979 > URL: https://issues.apache.org/jira/browse/HDFS-11979 > Project: Hadoop HDFS > Issue Type: Sub-task > Components: ozone, test >Reporter: Tsz Wo Nicholas Sze >Assignee: Tsz Wo Nicholas Sze >Priority: Minor > Attachments: HDFS-11979-HDFS-7240.20170615.patch > > > We should either remove MiniOzoneCluster from the test or somehow use it. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-11979) Ozone: TestContainerPersistence never uses MiniOzoneCluster
[ https://issues.apache.org/jira/browse/HDFS-11979?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tsz Wo Nicholas Sze updated HDFS-11979: --- Status: Patch Available (was: Open) > Ozone: TestContainerPersistence never uses MiniOzoneCluster > --- > > Key: HDFS-11979 > URL: https://issues.apache.org/jira/browse/HDFS-11979 > Project: Hadoop HDFS > Issue Type: Sub-task > Components: ozone, test >Reporter: Tsz Wo Nicholas Sze >Assignee: Tsz Wo Nicholas Sze >Priority: Minor > Attachments: HDFS-11979-HDFS-7240.20170615.patch > > > We should either remove MiniOzoneCluster from the test or somehow use it. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-11443) Ozone: Support SCM multiple instance for HA
[ https://issues.apache.org/jira/browse/HDFS-11443?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mukul Kumar Singh updated HDFS-11443: - Status: Patch Available (was: Open) > Ozone: Support SCM multiple instance for HA > --- > > Key: HDFS-11443 > URL: https://issues.apache.org/jira/browse/HDFS-11443 > Project: Hadoop HDFS > Issue Type: Sub-task > Components: ozone >Affects Versions: HDFS-7240 >Reporter: Anu Engineer >Assignee: Anu Engineer > Fix For: HDFS-7240 > > Attachments: HDFS-11972-HDFS-7240.001.patch > > > SCM clients are capable of talking to multiple SCMs. This is achieved by > reading the SCM config information or by one of the active SCM sending out > the current set of machines that make up the RAFT Quorum. > Currently we are not sending out the list of SCM machines dynamically, that > is clients read the SCM info only from the config files. This hurts cluster > level HA since if a node fails in the SCM RAFT ring, the operator should be > able to add a new node for high availability without having to do anything > else. > This JIRA is to track that membership change send updates to all datanodes, > so they all start talking to the newly added SCM node automatically. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-11443) Ozone: Support SCM multiple instance for HA
[ https://issues.apache.org/jira/browse/HDFS-11443?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mukul Kumar Singh updated HDFS-11443: - Attachment: HDFS-11972-HDFS-7240.001.patch > Ozone: Support SCM multiple instance for HA > --- > > Key: HDFS-11443 > URL: https://issues.apache.org/jira/browse/HDFS-11443 > Project: Hadoop HDFS > Issue Type: Sub-task > Components: ozone >Affects Versions: HDFS-7240 >Reporter: Anu Engineer >Assignee: Anu Engineer > Fix For: HDFS-7240 > > Attachments: HDFS-11972-HDFS-7240.001.patch > > > SCM clients are capable of talking to multiple SCMs. This is achieved by > reading the SCM config information or by one of the active SCM sending out > the current set of machines that make up the RAFT Quorum. > Currently we are not sending out the list of SCM machines dynamically, that > is clients read the SCM info only from the config files. This hurts cluster > level HA since if a node fails in the SCM RAFT ring, the operator should be > able to add a new node for high availability without having to do anything > else. > This JIRA is to track that membership change send updates to all datanodes, > so they all start talking to the newly added SCM node automatically. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-11443) Ozone: Support SCM multiple instance for HA
[ https://issues.apache.org/jira/browse/HDFS-11443?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mukul Kumar Singh updated HDFS-11443: - Status: Open (was: Patch Available) > Ozone: Support SCM multiple instance for HA > --- > > Key: HDFS-11443 > URL: https://issues.apache.org/jira/browse/HDFS-11443 > Project: Hadoop HDFS > Issue Type: Sub-task > Components: ozone >Affects Versions: HDFS-7240 >Reporter: Anu Engineer >Assignee: Anu Engineer > Fix For: HDFS-7240 > > Attachments: HDFS-11972-HDFS-7240.001.patch > > > SCM clients are capable of talking to multiple SCMs. This is achieved by > reading the SCM config information or by one of the active SCM sending out > the current set of machines that make up the RAFT Quorum. > Currently we are not sending out the list of SCM machines dynamically, that > is clients read the SCM info only from the config files. This hurts cluster > level HA since if a node fails in the SCM RAFT ring, the operator should be > able to add a new node for high availability without having to do anything > else. > This JIRA is to track that membership change send updates to all datanodes, > so they all start talking to the newly added SCM node automatically. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-11443) Ozone: Support SCM multiple instance for HA
[ https://issues.apache.org/jira/browse/HDFS-11443?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mukul Kumar Singh updated HDFS-11443: - Attachment: (was: HDFS-11972-HDFS-7240.001.patch) > Ozone: Support SCM multiple instance for HA > --- > > Key: HDFS-11443 > URL: https://issues.apache.org/jira/browse/HDFS-11443 > Project: Hadoop HDFS > Issue Type: Sub-task > Components: ozone >Affects Versions: HDFS-7240 >Reporter: Anu Engineer >Assignee: Anu Engineer > Fix For: HDFS-7240 > > > SCM clients are capable of talking to multiple SCMs. This is achieved by > reading the SCM config information or by one of the active SCM sending out > the current set of machines that make up the RAFT Quorum. > Currently we are not sending out the list of SCM machines dynamically, that > is clients read the SCM info only from the config files. This hurts cluster > level HA since if a node fails in the SCM RAFT ring, the operator should be > able to add a new node for high availability without having to do anything > else. > This JIRA is to track that membership change send updates to all datanodes, > so they all start talking to the newly added SCM node automatically. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-11972) CBlocks use wrong OPT env vars
[
https://issues.apache.org/jira/browse/HDFS-11972?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mukul Kumar Singh updated HDFS-11972:
-
Status: Patch Available (was: Open)
> CBlocks use wrong OPT env vars
> --
>
> Key: HDFS-11972
> URL: https://issues.apache.org/jira/browse/HDFS-11972
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: hdfs
>Reporter: Allen Wittenauer
>Assignee: Mukul Kumar Singh
> Attachments: HDFS-11972-HDFS-7240.001.patch
>
>
> Current codebase does:
> {code}
> hadoop_debug "Appending HADOOP_CBLOCK_OPTS onto HADOOP_OPTS"
> HADOOP_OPTS="${HADOOP_OPTS} ${HADOOP_CBLOCK_OPTS}"
> ...
> hadoop_debug "Appending HADOOP_JSCSI_OPTS onto HADOOP_OPTS"
> HADOOP_OPTS="${HADOOP_OPTS} ${HADOOP_JSCSI_OPTS}"
> {code}
> This code block breaks the consistency with the rest of the shell scripts:
> a) It should be HDFS_CBLOCKSERVER_OPTS, HDFS_JSCSI_OPTS
> b) _OPTS, unless they are a deprecated form or some other special case, are
> already/automatically appended; there is no need to do them specifically
> Also, a description of these env vars should be hadoop-env.sh for
> documentation purposes.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-11972) CBlocks use wrong OPT env vars
[
https://issues.apache.org/jira/browse/HDFS-11972?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mukul Kumar Singh updated HDFS-11972:
-
Attachment: HDFS-11972-HDFS-7240.001.patch
> CBlocks use wrong OPT env vars
> --
>
> Key: HDFS-11972
> URL: https://issues.apache.org/jira/browse/HDFS-11972
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: hdfs
>Reporter: Allen Wittenauer
>Assignee: Mukul Kumar Singh
> Attachments: HDFS-11972-HDFS-7240.001.patch
>
>
> Current codebase does:
> {code}
> hadoop_debug "Appending HADOOP_CBLOCK_OPTS onto HADOOP_OPTS"
> HADOOP_OPTS="${HADOOP_OPTS} ${HADOOP_CBLOCK_OPTS}"
> ...
> hadoop_debug "Appending HADOOP_JSCSI_OPTS onto HADOOP_OPTS"
> HADOOP_OPTS="${HADOOP_OPTS} ${HADOOP_JSCSI_OPTS}"
> {code}
> This code block breaks the consistency with the rest of the shell scripts:
> a) It should be HDFS_CBLOCKSERVER_OPTS, HDFS_JSCSI_OPTS
> b) _OPTS, unless they are a deprecated form or some other special case, are
> already/automatically appended; there is no need to do them specifically
> Also, a description of these env vars should be hadoop-env.sh for
> documentation purposes.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Created] (HDFS-11980) Add documentation for metrics in CBlockTargetMetrics to OzoneMetrics.md
Mukul Kumar Singh created HDFS-11980: Summary: Add documentation for metrics in CBlockTargetMetrics to OzoneMetrics.md Key: HDFS-11980 URL: https://issues.apache.org/jira/browse/HDFS-11980 Project: Hadoop HDFS Issue Type: Sub-task Components: ozone Affects Versions: HDFS-7240 Reporter: Mukul Kumar Singh Assignee: Mukul Kumar Singh Fix For: HDFS-7240 Metrics defined in CBlockTargetMetrics are not documented in OzoneMetrics.md. This bug will track documentation of metrics to the bug. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-11980) Add documentation for metrics in CBlockTargetMetrics to OzoneMetrics.md
[ https://issues.apache.org/jira/browse/HDFS-11980?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mukul Kumar Singh updated HDFS-11980: - Labels: documentation (was: ) > Add documentation for metrics in CBlockTargetMetrics to OzoneMetrics.md > --- > > Key: HDFS-11980 > URL: https://issues.apache.org/jira/browse/HDFS-11980 > Project: Hadoop HDFS > Issue Type: Sub-task > Components: ozone >Affects Versions: HDFS-7240 >Reporter: Mukul Kumar Singh >Assignee: Mukul Kumar Singh > Labels: documentation > Fix For: HDFS-7240 > > > Metrics defined in CBlockTargetMetrics are not documented in OzoneMetrics.md. > This bug will track documentation of metrics to the bug. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Created] (HDFS-11981) Add markdown documentation about Cblock
Mukul Kumar Singh created HDFS-11981: Summary: Add markdown documentation about Cblock Key: HDFS-11981 URL: https://issues.apache.org/jira/browse/HDFS-11981 Project: Hadoop HDFS Issue Type: Sub-task Components: ozone Affects Versions: HDFS-7240 Reporter: Mukul Kumar Singh Assignee: Mukul Kumar Singh Fix For: HDFS-7240 Cblock should be documented in detail using markdown. This documentation should include information about setting up various Cblock components, usage and configuration. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Created] (HDFS-11982) Add markdown documentation about Ozone
Mukul Kumar Singh created HDFS-11982: Summary: Add markdown documentation about Ozone Key: HDFS-11982 URL: https://issues.apache.org/jira/browse/HDFS-11982 Project: Hadoop HDFS Issue Type: Sub-task Components: ozone Affects Versions: HDFS-7240 Reporter: Mukul Kumar Singh Assignee: Mukul Kumar Singh Fix For: HDFS-7240 -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-11982) Add markdown documentation about Ozone
[ https://issues.apache.org/jira/browse/HDFS-11982?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mukul Kumar Singh updated HDFS-11982: - Description: Ozone should be documented in detail using markdown. This documentation should include information about setting up various Cblock components, usage and configuration. > Add markdown documentation about Ozone > -- > > Key: HDFS-11982 > URL: https://issues.apache.org/jira/browse/HDFS-11982 > Project: Hadoop HDFS > Issue Type: Sub-task > Components: ozone >Affects Versions: HDFS-7240 >Reporter: Mukul Kumar Singh >Assignee: Mukul Kumar Singh > Fix For: HDFS-7240 > > > Ozone should be documented in detail using markdown. > This documentation should include information about setting up various Cblock > components, usage and configuration. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-11982) Add markdown documentation about Ozone
[ https://issues.apache.org/jira/browse/HDFS-11982?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mukul Kumar Singh updated HDFS-11982: - Labels: docu (was: ) > Add markdown documentation about Ozone > -- > > Key: HDFS-11982 > URL: https://issues.apache.org/jira/browse/HDFS-11982 > Project: Hadoop HDFS > Issue Type: Sub-task > Components: ozone >Affects Versions: HDFS-7240 >Reporter: Mukul Kumar Singh >Assignee: Mukul Kumar Singh > Labels: documentation > Fix For: HDFS-7240 > > > Ozone should be documented in detail using markdown. > This documentation should include information about setting up various Cblock > components, usage and configuration. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-11982) Add markdown documentation about Ozone
[ https://issues.apache.org/jira/browse/HDFS-11982?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mukul Kumar Singh updated HDFS-11982: - Labels: documentation (was: docu) > Add markdown documentation about Ozone > -- > > Key: HDFS-11982 > URL: https://issues.apache.org/jira/browse/HDFS-11982 > Project: Hadoop HDFS > Issue Type: Sub-task > Components: ozone >Affects Versions: HDFS-7240 >Reporter: Mukul Kumar Singh >Assignee: Mukul Kumar Singh > Labels: documentation > Fix For: HDFS-7240 > > > Ozone should be documented in detail using markdown. > This documentation should include information about setting up various Ozone > components, usage and configuration. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-11982) Add markdown documentation about Ozone
[ https://issues.apache.org/jira/browse/HDFS-11982?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mukul Kumar Singh updated HDFS-11982: - Description: Ozone should be documented in detail using markdown. This documentation should include information about setting up various Ozone components, usage and configuration. was: Ozone should be documented in detail using markdown. This documentation should include information about setting up various Cblock components, usage and configuration. > Add markdown documentation about Ozone > -- > > Key: HDFS-11982 > URL: https://issues.apache.org/jira/browse/HDFS-11982 > Project: Hadoop HDFS > Issue Type: Sub-task > Components: ozone >Affects Versions: HDFS-7240 >Reporter: Mukul Kumar Singh >Assignee: Mukul Kumar Singh > Labels: documentation > Fix For: HDFS-7240 > > > Ozone should be documented in detail using markdown. > This documentation should include information about setting up various Ozone > components, usage and configuration. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-11736) OIV tests should not write outside 'target' directory.
[
https://issues.apache.org/jira/browse/HDFS-11736?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16050255#comment-16050255
]
Hudson commented on HDFS-11736:
---
SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #11866 (See
[https://builds.apache.org/job/Hadoop-trunk-Commit/11866/])
HDFS-11736. OIV tests should not write outside 'target' directory. (aajisaka:
rev 315f07700dcfe32513b87f63070db854ecb3cf48)
* (edit)
hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/tools/offlineImageViewer/TestOfflineImageViewer.java
* (edit)
hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestCheckpoint.java
* (edit)
hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/ha/TestStandbyCheckpoints.java
> OIV tests should not write outside 'target' directory.
> --
>
> Key: HDFS-11736
> URL: https://issues.apache.org/jira/browse/HDFS-11736
> Project: Hadoop HDFS
> Issue Type: Bug
>Affects Versions: 2.6.0
>Reporter: Konstantin Shvachko
>Assignee: Yiqun Lin
> Labels: newbie++, test
> Fix For: 2.9.0, 2.7.4, 3.0.0-alpha4, 2.8.2
>
> Attachments: HDFS-11736.001.patch, HDFS-11736.002.patch,
> HDFS-11736.003.patch, HDFS-11736-branch-2.7.001.patch,
> HDFS-11736-branch-2.7.002.patch
>
>
> A few tests use {{Files.createTempDir()}} from Guava package, but do not set
> {{java.io.tmpdir}} system property. Thus the temp directory is created in
> unpredictable places and is not being cleaned up by {{mvn clean}}.
> This was probably introduced in {{TestOfflineImageViewer}} and then
> replicated in {{TestCheckpoint}}, {{TestStandbyCheckpoints}}.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-11977) Ozone: cannot enable test debug/trace log
[
https://issues.apache.org/jira/browse/HDFS-11977?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16050279#comment-16050279
]
Hadoop QA commented on HDFS-11977:
--
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
13s{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 15m
38s{color} | {color:green} HDFS-7240 passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
51s{color} | {color:green} hadoop-hdfs in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
19s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 17m 57s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Image:yetus/hadoop:14b5c93 |
| JIRA Issue | HDFS-11977 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12873067/HDFS-11977-HDFS-7240.20170615.patch
|
| Optional Tests | asflicense unit |
| uname | Linux 7d3fb1f608e4 3.13.0-107-generic #154-Ubuntu SMP Tue Dec 20
09:57:27 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh
|
| git revision | HDFS-7240 / ba1afb2 |
| Test Results |
https://builds.apache.org/job/PreCommit-HDFS-Build/19912/testReport/ |
| modules | C: hadoop-hdfs-project/hadoop-hdfs U:
hadoop-hdfs-project/hadoop-hdfs |
| Console output |
https://builds.apache.org/job/PreCommit-HDFS-Build/19912/console |
| Powered by | Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org |
This message was automatically generated.
> Ozone: cannot enable test debug/trace log
> -
>
> Key: HDFS-11977
> URL: https://issues.apache.org/jira/browse/HDFS-11977
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: ozone, test
>Reporter: Tsz Wo Nicholas Sze
>Assignee: Tsz Wo Nicholas Sze
>Priority: Minor
> Attachments: HDFS-11977-HDFS-7240.20170615.patch
>
>
> Interestingly, the test debug/trace logs are not printed for Ozone classes
> even if we invoke GenericTestUtils.setLogLevel(log, Level.ALL). Other
> classes such as Object do not have such problem. Here is a test:
> {code}
> @Test
> public void testLogLevel() throws Exception {
> runTestLogLevel(StorageContainerManager.class);
> runTestLogLevel(Object.class);
> }
> static void runTestLogLevel(Class clazz) throws Exception {
> final Logger log = LoggerFactory.getLogger(clazz);
> GenericTestUtils.setLogLevel(log, Level.ALL);
> log.trace(clazz.getSimpleName() + " trace log");
> log.debug(clazz.getSimpleName() + " debug log");
> log.info(clazz.getSimpleName() + " info log");
> log.warn(clazz.getSimpleName() + " warn log");
> log.error(clazz.getSimpleName() + " error log");
> }
> {code}
> Output:
> {code}
> 2017-06-15 00:19:07,133 [Thread-0] INFO - StorageContainerManager info
> log
> 2017-06-15 00:19:07,135 [Thread-0] WARN - StorageContainerManager warn
> log
> 2017-06-15 00:19:07,135 [Thread-0] ERROR - StorageContainerManager error
> log
> 2017-06-15 00:19:07,135 [Thread-0] TRACE
> lang.Object(TestOzoneContainer.java:runTestLogLevel(64)) - Object trace log
> 2017-06-15 00:19:07,135 [Thread-0] DEBUG
> lang.Object(TestOzoneContainer.java:runTestLogLevel(65)) - Object debug log
> 2017-06-15 00:19:07,135 [Thread-0] INFO
> lang.Object(TestOzoneContainer.java:runTestLogLevel(66)) - Object info log
> 2017-06-15 00:19:07,135 [Thread-0] WARN
> lang.Object(TestOzoneContainer.java:runTestLogLevel(67)) - Object warn log
> 2017-06-15 00:19:07,135 [Thread-0] ERROR
> lang.Object(TestOzoneContainer.java:runTestLogLevel(68)) - Object error log
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-11972) CBlocks use wrong OPT env vars
[
https://issues.apache.org/jira/browse/HDFS-11972?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16050315#comment-16050315
]
Hadoop QA commented on HDFS-11972:
--
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
11s{color} | {color:blue} Docker mode activated. {color} |
| {color:blue}0{color} | {color:blue} shelldocs {color} | {color:blue} 0m
5s{color} | {color:blue} Shelldocs was not available. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m
0s{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 2m
6s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 14m
19s{color} | {color:green} HDFS-7240 passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
51s{color} | {color:green} HDFS-7240 passed {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
17s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
43s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} shellcheck {color} | {color:green} 0m
20s{color} | {color:green} The patch generated 0 new + 0 unchanged - 2 fixed =
0 total (was 2) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 1m
33s{color} | {color:green} hadoop-common in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
48s{color} | {color:green} hadoop-hdfs in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
17s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 23m 43s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Image:yetus/hadoop:14b5c93 |
| JIRA Issue | HDFS-11972 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12873082/HDFS-11972-HDFS-7240.001.patch
|
| Optional Tests | asflicense mvnsite unit shellcheck shelldocs |
| uname | Linux cc940fc16e73 4.4.0-43-generic #63-Ubuntu SMP Wed Oct 12
13:48:03 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh
|
| git revision | HDFS-7240 / ba1afb2 |
| shellcheck | v0.4.6 |
| Test Results |
https://builds.apache.org/job/PreCommit-HDFS-Build/19917/testReport/ |
| modules | C: hadoop-common-project/hadoop-common
hadoop-hdfs-project/hadoop-hdfs U: . |
| Console output |
https://builds.apache.org/job/PreCommit-HDFS-Build/19917/console |
| Powered by | Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org |
This message was automatically generated.
> CBlocks use wrong OPT env vars
> --
>
> Key: HDFS-11972
> URL: https://issues.apache.org/jira/browse/HDFS-11972
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: hdfs
>Reporter: Allen Wittenauer
>Assignee: Mukul Kumar Singh
> Attachments: HDFS-11972-HDFS-7240.001.patch
>
>
> Current codebase does:
> {code}
> hadoop_debug "Appending HADOOP_CBLOCK_OPTS onto HADOOP_OPTS"
> HADOOP_OPTS="${HADOOP_OPTS} ${HADOOP_CBLOCK_OPTS}"
> ...
> hadoop_debug "Appending HADOOP_JSCSI_OPTS onto HADOOP_OPTS"
> HADOOP_OPTS="${HADOOP_OPTS} ${HADOOP_JSCSI_OPTS}"
> {code}
> This code block breaks the consistency with the rest of the shell scripts:
> a) It should be HDFS_CBLOCKSERVER_OPTS, HDFS_JSCSI_OPTS
> b) _OPTS, unless they are a deprecated form or some other special case, are
> already/automatically appended; there is no need to do them specifically
> Also, a description of these env vars should be hadoop-env.sh for
> documentation purposes.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.ap
[jira] [Created] (HDFS-11983) Add documentation for metrics in KSMMetrics to OzoneMetrics.md
Yiqun Lin created HDFS-11983: Summary: Add documentation for metrics in KSMMetrics to OzoneMetrics.md Key: HDFS-11983 URL: https://issues.apache.org/jira/browse/HDFS-11983 Project: Hadoop HDFS Issue Type: Sub-task Components: documentation, ozone Affects Versions: HDFS-7240 Reporter: Yiqun Lin Assignee: Yiqun Lin Metrics defined in KSMMetrics are not documented in OzoneMetrics.md. This JIRA will track on this. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-11978) Remove invalid '-usage' command of 'ec' and add missing commands 'addPolicies' 'listCodecs'
[ https://issues.apache.org/jira/browse/HDFS-11978?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] wenxin he updated HDFS-11978: - Status: Open (was: Patch Available) > Remove invalid '-usage' command of 'ec' and add missing commands > 'addPolicies' 'listCodecs' > --- > > Key: HDFS-11978 > URL: https://issues.apache.org/jira/browse/HDFS-11978 > Project: Hadoop HDFS > Issue Type: Bug > Components: documentation >Affects Versions: 3.0.0-alpha4 >Reporter: wenxin he >Priority: Minor > > Remove invalid '-usage' command of 'ec' in HDFSErasureCoding.md. > Add missing commands 'addPolicies' 'listCodecs' in HDFSCommands.md. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-11978) Remove invalid '-usage' command of 'ec' and add missing commands 'addPolicies' 'listCodecs'
[ https://issues.apache.org/jira/browse/HDFS-11978?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] wenxin he updated HDFS-11978: - Status: Patch Available (was: Open) > Remove invalid '-usage' command of 'ec' and add missing commands > 'addPolicies' 'listCodecs' > --- > > Key: HDFS-11978 > URL: https://issues.apache.org/jira/browse/HDFS-11978 > Project: Hadoop HDFS > Issue Type: Bug > Components: documentation >Affects Versions: 3.0.0-alpha4 >Reporter: wenxin he >Priority: Minor > > Remove invalid '-usage' command of 'ec' in HDFSErasureCoding.md. > Add missing commands 'addPolicies' 'listCodecs' in HDFSCommands.md. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-11345) Document the configuration key for FSNamesystem lock fairness
[
https://issues.apache.org/jira/browse/HDFS-11345?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16050374#comment-16050374
]
Hadoop QA commented on HDFS-11345:
--
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
14s{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 13m
30s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
47s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
40s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
55s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
42s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
39s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
49s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
46s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
46s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
37s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
51s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m
1s{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
45s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
37s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 70m 2s{color}
| {color:red} hadoop-hdfs in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
20s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 95m 32s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests |
hadoop.hdfs.server.datanode.TestDataNodeVolumeFailureReporting |
| | hadoop.hdfs.TestDFSStripedOutputStreamWithFailure090 |
| | hadoop.hdfs.TestDFSStripedOutputStreamWithFailure080 |
| Timed out junit tests |
org.apache.hadoop.hdfs.server.blockmanagement.TestBlockStatsMXBean |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Image:yetus/hadoop:14b5c93 |
| JIRA Issue | HDFS-11345 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12849323/HADOOP-11345.001.patch
|
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite
unit findbugs checkstyle xml |
| uname | Linux c4e2bed2212d 3.13.0-116-generic #163-Ubuntu SMP Fri Mar 31
14:13:22 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh
|
| git revision | trunk / 315f077 |
| Default Java | 1.8.0_131 |
| findbugs | v3.1.0-RC1 |
| unit |
https://builds.apache.org/job/PreCommit-HDFS-Build/19916/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-HDFS-Build/19916/testReport/ |
| modules | C: hadoop-hdfs-project/hadoop-hdfs U:
hadoop-hdfs-project/hadoop-hdfs |
| Console output |
https://builds.apache.org/job/PreCommit-HDFS-Build/19916/console |
| Powered by | Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org |
This message was automatically generated.
> Document the configuration key for FSNamesystem lock fairness
> -
>
> Key: HDFS-11345
> URL: https://issues.apache.org/jira/browse/HDFS-11345
> Project: Hadoop HDFS
>
[jira] [Commented] (HDFS-11979) Ozone: TestContainerPersistence never uses MiniOzoneCluster
[
https://issues.apache.org/jira/browse/HDFS-11979?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16050396#comment-16050396
]
Hadoop QA commented on HDFS-11979:
--
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 1m
2s{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 15m
28s{color} | {color:green} HDFS-7240 passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
3s{color} | {color:green} HDFS-7240 passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
44s{color} | {color:green} HDFS-7240 passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
8s{color} | {color:green} HDFS-7240 passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
8s{color} | {color:green} HDFS-7240 passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
59s{color} | {color:green} HDFS-7240 passed {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
3s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
59s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
59s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
40s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
8s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
20s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
59s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 87m 7s{color}
| {color:red} hadoop-hdfs in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
23s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}118m 45s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.ozone.scm.TestXceiverClientManager |
| | hadoop.ozone.container.ozoneimpl.TestRatisManager |
| | hadoop.hdfs.server.datanode.TestDataNodeVolumeFailure |
| | hadoop.hdfs.server.datanode.checker.TestThrottledAsyncCheckerTimeout |
| | hadoop.hdfs.server.datanode.TestDataNodeVolumeFailureReporting |
| | hadoop.hdfs.web.TestWebHdfsTimeouts |
| | hadoop.hdfs.server.namenode.ha.TestHAAppend |
| Timed out junit tests |
org.apache.hadoop.hdfs.server.blockmanagement.TestBlockStatsMXBean |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Image:yetus/hadoop:14b5c93 |
| JIRA Issue | HDFS-11979 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12873074/HDFS-11979-HDFS-7240.20170615.patch
|
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite
unit findbugs checkstyle |
| uname | Linux 11ad4a9cc9f7 4.4.0-43-generic #63-Ubuntu SMP Wed Oct 12
13:48:03 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh
|
| git revision | HDFS-7240 / ba1afb2 |
| Default Java | 1.8.0_131 |
| findbugs | v3.1.0-RC1 |
| unit |
https://builds.apache.org/job/PreCommit-HDFS-Build/19913/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-HDFS-Build/19913/testReport/ |
| modules | C: hadoop-hdfs-project/hadoop-hdfs U:
hadoop-hdfs-project/hadoop-hdfs |
| Console output |
https://builds.apache.org/job/PreCommit-HDFS-Build/19913/console |
| Powered by | Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org |
This message was automatically generated.
> Ozone: TestContainerPersistence never uses MiniOzoneCluster
> ---
>
> Key: HDFS-11979
>
[jira] [Updated] (HDFS-11782) Ozone: KSM: Add listKey
[ https://issues.apache.org/jira/browse/HDFS-11782?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yiqun Lin updated HDFS-11782: - Attachment: HDFS-11782-HDFS-7240.003.patch > Ozone: KSM: Add listKey > --- > > Key: HDFS-11782 > URL: https://issues.apache.org/jira/browse/HDFS-11782 > Project: Hadoop HDFS > Issue Type: Sub-task > Components: HDFS-7240 >Affects Versions: ozone >Reporter: Anu Engineer >Assignee: Yiqun Lin > Attachments: HDFS-11782-HDFS-7240.001.patch, > HDFS-11782-HDFS-7240.002.patch, HDFS-11782-HDFS-7240.003.patch > > > Add support for listing keys in a bucket. Just like other 2 list operations, > this API supports paging via, prevKey, prefix and maxKeys. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-11782) Ozone: KSM: Add listKey
[ https://issues.apache.org/jira/browse/HDFS-11782?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yiqun Lin updated HDFS-11782: - Attachment: (was: HDFS-11782-HDFS-7240.003.patch) > Ozone: KSM: Add listKey > --- > > Key: HDFS-11782 > URL: https://issues.apache.org/jira/browse/HDFS-11782 > Project: Hadoop HDFS > Issue Type: Sub-task > Components: HDFS-7240 >Affects Versions: ozone >Reporter: Anu Engineer >Assignee: Yiqun Lin > Attachments: HDFS-11782-HDFS-7240.001.patch, > HDFS-11782-HDFS-7240.002.patch, HDFS-11782-HDFS-7240.003.patch > > > Add support for listing keys in a bucket. Just like other 2 list operations, > this API supports paging via, prevKey, prefix and maxKeys. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-8520) Patch for PPC64 block size
[
https://issues.apache.org/jira/browse/HDFS-8520?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16050425#comment-16050425
]
Hadoop QA commented on HDFS-8520:
-
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
11s{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 3 new or modified test
files. {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m
28s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 14m
51s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 15m
35s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 2m
0s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 2m
6s{color} | {color:green} trunk passed {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 1m
26s{color} | {color:red} hadoop-common-project/hadoop-common in trunk has 19
extant Findbugs warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
38s{color} | {color:green} trunk passed {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
15s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
44s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 12m
4s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 12m
4s{color} | {color:green} the patch passed {color} |
| {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange}
1m 56s{color} | {color:orange} root: The patch generated 3 new + 178 unchanged
- 0 fixed = 181 total (was 178) {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 2m
6s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
40s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
39s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 7m 16s{color}
| {color:red} hadoop-common in the patch failed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 76m 38s{color}
| {color:red} hadoop-hdfs in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
44s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}150m 34s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.fs.sftp.TestSFTPFileSystem |
| | hadoop.hdfs.TestDFSStripedOutputStreamWithFailure080 |
| | hadoop.hdfs.TestDFSStripedInputStreamWithRandomECPolicy |
| | hadoop.hdfs.TestBlockStoragePolicy |
| | hadoop.hdfs.TestDFSStripedOutputStreamWithFailure150 |
| | hadoop.hdfs.server.datanode.TestDataNodeVolumeFailureReporting |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Image:yetus/hadoop:14b5c93 |
| JIRA Issue | HDFS-8520 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12738583/HDFS-8520-2.patch |
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite
unit findbugs checkstyle |
| uname | Linux cea81f577b16 3.13.0-107-generic #154-Ubuntu SMP Tue Dec 20
09:57:27 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh
|
| git revision | trunk / 315f077 |
| Default Java | 1.8.0_131 |
| findbugs | v3.1.0-RC1 |
| findbugs |
https://builds.apache.org/job/PreCommit-HDFS-Build/19915/artifact/patchprocess/branch-findbugs-hadoop-common-project_hadoop-common-warnings.html
|
| checkstyle |
https://builds.apache.org/job/PreCommit-HDFS-Build/19915/artifact/patchprocess/diff-checkstyle-root.txt
|
| unit |
https://builds.apache.org/job/PreCommit-HDFS-Build/1
[jira] [Commented] (HDFS-11782) Ozone: KSM: Add listKey
[
https://issues.apache.org/jira/browse/HDFS-11782?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16050424#comment-16050424
]
Hadoop QA commented on HDFS-11782:
--
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
24s{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
31s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 17m
43s{color} | {color:green} HDFS-7240 passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
38s{color} | {color:green} HDFS-7240 passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
45s{color} | {color:green} HDFS-7240 passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
43s{color} | {color:green} HDFS-7240 passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
47s{color} | {color:green} HDFS-7240 passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
43s{color} | {color:green} HDFS-7240 passed {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
8s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
33s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
34s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} cc {color} | {color:green} 1m
34s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
34s{color} | {color:green} the patch passed {color} |
| {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange}
0m 42s{color} | {color:orange} hadoop-hdfs-project: The patch generated 1 new +
1 unchanged - 0 fixed = 2 total (was 1) {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
34s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
56s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
31s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 1m
19s{color} | {color:green} hadoop-hdfs-client in the patch passed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red}108m 19s{color}
| {color:red} hadoop-hdfs in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
35s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}151m 3s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.hdfs.server.balancer.TestBalancer |
| | hadoop.ozone.container.ozoneimpl.TestOzoneContainerRatis |
| | hadoop.hdfs.TestDFSStripedOutputStreamWithFailure120 |
| | hadoop.ozone.scm.node.TestNodeManager |
| | hadoop.cblock.TestCBlockReadWrite |
| | hadoop.ozone.container.ozoneimpl.TestRatisManager |
| Timed out junit tests | org.apache.hadoop.cblock.TestLocalBlockCache |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Image:yetus/hadoop:14b5c93 |
| JIRA Issue | HDFS-11782 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12873063/HDFS-11782-HDFS-7240.003.patch
|
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite
unit findbugs checkstyle cc |
| uname | Linux 3c61f38f3e81 3.13.0-108-generic #155-Ubuntu SMP Wed Jan 11
16:58:52 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh
|
| git revision | HDFS-7240 / ba1afb2 |
| Default Java | 1.8.0_131 |
| findbugs | v3.1.0-RC1 |
| checkstyle |
https://builds.apache.org/job/PreCommit-HDFS-Build/19914/artifact/patchprocess/diff-checkstyle-hadoop-hdfs-project.txt
|
| unit |
https:
[jira] [Updated] (HDFS-11782) Ozone: KSM: Add listKey
[ https://issues.apache.org/jira/browse/HDFS-11782?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yiqun Lin updated HDFS-11782: - Attachment: HDFS-11782-HDFS-7240.004.patch Attach the updated patch to fix checkstyle warning. > Ozone: KSM: Add listKey > --- > > Key: HDFS-11782 > URL: https://issues.apache.org/jira/browse/HDFS-11782 > Project: Hadoop HDFS > Issue Type: Sub-task > Components: HDFS-7240 >Affects Versions: ozone >Reporter: Anu Engineer >Assignee: Yiqun Lin > Attachments: HDFS-11782-HDFS-7240.001.patch, > HDFS-11782-HDFS-7240.002.patch, HDFS-11782-HDFS-7240.003.patch, > HDFS-11782-HDFS-7240.004.patch > > > Add support for listing keys in a bucket. Just like other 2 list operations, > this API supports paging via, prevKey, prefix and maxKeys. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Created] (HDFS-11984) Ozone: Ensures listKey lists all required key fields
Weiwei Yang created HDFS-11984: -- Summary: Ozone: Ensures listKey lists all required key fields Key: HDFS-11984 URL: https://issues.apache.org/jira/browse/HDFS-11984 Project: Hadoop HDFS Issue Type: Sub-task Components: ozone Reporter: Weiwei Yang HDFS-11782 implements the listKey operation which only lists the basic key fields, we need to make sure it return all required fields # version # md5hash # createdOn # size # keyName # dataFileName this task is depending on the work of HDFS-11886. See more discussion [here | https://issues.apache.org/jira/browse/HDFS-11782?focusedCommentId=16045562&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-16045562]. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-11782) Ozone: KSM: Add listKey
[ https://issues.apache.org/jira/browse/HDFS-11782?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16050549#comment-16050549 ] Weiwei Yang commented on HDFS-11782: +1 to latest patch, agree to get basic function committed and track the remaining work via HDFS-11984. Thanks [~linyiqun]. > Ozone: KSM: Add listKey > --- > > Key: HDFS-11782 > URL: https://issues.apache.org/jira/browse/HDFS-11782 > Project: Hadoop HDFS > Issue Type: Sub-task > Components: HDFS-7240 >Affects Versions: ozone >Reporter: Anu Engineer >Assignee: Yiqun Lin > Attachments: HDFS-11782-HDFS-7240.001.patch, > HDFS-11782-HDFS-7240.002.patch, HDFS-11782-HDFS-7240.003.patch, > HDFS-11782-HDFS-7240.004.patch > > > Add support for listing keys in a bucket. Just like other 2 list operations, > this API supports paging via, prevKey, prefix and maxKeys. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-11976) Examine code base for cases that exception is thrown from finally block and fix it
[ https://issues.apache.org/jira/browse/HDFS-11976?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16050561#comment-16050561 ] Pavel Avgustinov commented on HDFS-11976: - This concern is easy to capture with the query language used by [lgtm|https://lgtm.com]. I've just written a query for it, which you can see [here|https://lgtm.com/query/1995330256/project:5629499534213120/lang:java/] -- it finds 32 results in the [GitHub mirror of Hadoop|https://github.com/apache/hadoop], including a number within HDFS itself. (Disclaimer: I'm a co-founder of Semmle, the company that provides lgtm as a free service to open source.) > Examine code base for cases that exception is thrown from finally block and > fix it > -- > > Key: HDFS-11976 > URL: https://issues.apache.org/jira/browse/HDFS-11976 > Project: Hadoop HDFS > Issue Type: Bug >Reporter: Yongjun Zhang > > If exception X is thrown in try block, and exception Y is thrown is finally > block, X will be swallowed. > In addition, finally block is used to ensure resources are released properly > in general. If we throw exception from there, some resources may be leaked. > So it's not recommended to throw exception in the finally block > I caught one today and reported HDFS-11794, creating this jira as a master > one to catch other similar cases. > Hope there is some static analyzer to find all. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-11962) Ozone: Add stop-ozone.sh script
[ https://issues.apache.org/jira/browse/HDFS-11962?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Weiwei Yang updated HDFS-11962: --- Attachment: HDFS-11962-HDFS-7240.003.patch Attach v3 patch to fix shellcheck warning. > Ozone: Add stop-ozone.sh script > --- > > Key: HDFS-11962 > URL: https://issues.apache.org/jira/browse/HDFS-11962 > Project: Hadoop HDFS > Issue Type: Sub-task >Reporter: Anu Engineer >Assignee: Weiwei Yang > Attachments: HDFS-11962-HDFS-7240.001.patch, > HDFS-11962-HDFS-7240.002.patch, HDFS-11962-HDFS-7240.003.patch > > > This script should stop the cluster along with all ozone services. -- > # run 'stop-dfs.sh' > # run 'hdfs --daemon stop ksm' > # run 'hdfs --daemon stop scm`' -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-11961) Ozone: Add start-ozone.sh to quickly start ozone.
[ https://issues.apache.org/jira/browse/HDFS-11961?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Weiwei Yang updated HDFS-11961: --- Attachment: HDFS-11961-HDFS-7240.002.patch Attached v2 patch to fix checkstyle and shellcheck warnings. > Ozone: Add start-ozone.sh to quickly start ozone. > - > > Key: HDFS-11961 > URL: https://issues.apache.org/jira/browse/HDFS-11961 > Project: Hadoop HDFS > Issue Type: Sub-task >Reporter: Anu Engineer >Assignee: Weiwei Yang > Attachments: HDFS-11961-HDFS-7240.001.patch, > HDFS-11961-HDFS-7240.002.patch > > > Add start ozone script. Internally this script should call into > # start-dfs.sh > # run `hdfs --daemon start scm' > # run `hdfs --daemon start ksm` > This just makes it easy to start Ozone with a single command. This command > assumes that Namenode format has been run before, since it will bring up HDFS > also. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-11736) OIV tests should not write outside 'target' directory.
[
https://issues.apache.org/jira/browse/HDFS-11736?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16050605#comment-16050605
]
Jason Lowe commented on HDFS-11736:
---
The branch-2 and branch-2.8 builds broke after this commit due to a compilation
error:
{noformat}
[ERROR] COMPILATION ERROR :
[INFO] -
[ERROR]
/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/tools/offlineImageViewer/TestOfflineImageViewer.java:[117,35]
cannot find symbol
symbol: method getNameNodeDirectory(java.lang.String,int,int)
location: class org.apache.hadoop.hdfs.MiniDFSCluster
{noformat}
> OIV tests should not write outside 'target' directory.
> --
>
> Key: HDFS-11736
> URL: https://issues.apache.org/jira/browse/HDFS-11736
> Project: Hadoop HDFS
> Issue Type: Bug
>Affects Versions: 2.6.0
>Reporter: Konstantin Shvachko
>Assignee: Yiqun Lin
> Labels: newbie++, test
> Fix For: 2.9.0, 2.7.4, 3.0.0-alpha4, 2.8.2
>
> Attachments: HDFS-11736.001.patch, HDFS-11736.002.patch,
> HDFS-11736.003.patch, HDFS-11736-branch-2.7.001.patch,
> HDFS-11736-branch-2.7.002.patch
>
>
> A few tests use {{Files.createTempDir()}} from Guava package, but do not set
> {{java.io.tmpdir}} system property. Thus the temp directory is created in
> unpredictable places and is not being cleaned up by {{mvn clean}}.
> This was probably introduced in {{TestOfflineImageViewer}} and then
> replicated in {{TestCheckpoint}}, {{TestStandbyCheckpoints}}.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Reopened] (HDFS-11736) OIV tests should not write outside 'target' directory.
[
https://issues.apache.org/jira/browse/HDFS-11736?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Akira Ajisaka reopened HDFS-11736:
--
> OIV tests should not write outside 'target' directory.
> --
>
> Key: HDFS-11736
> URL: https://issues.apache.org/jira/browse/HDFS-11736
> Project: Hadoop HDFS
> Issue Type: Bug
>Affects Versions: 2.6.0
>Reporter: Konstantin Shvachko
>Assignee: Yiqun Lin
> Labels: newbie++, test
> Fix For: 2.9.0, 2.7.4, 3.0.0-alpha4, 2.8.2
>
> Attachments: HDFS-11736.001.patch, HDFS-11736.002.patch,
> HDFS-11736.003.patch, HDFS-11736-branch-2.7.001.patch,
> HDFS-11736-branch-2.7.002.patch
>
>
> A few tests use {{Files.createTempDir()}} from Guava package, but do not set
> {{java.io.tmpdir}} system property. Thus the temp directory is created in
> unpredictable places and is not being cleaned up by {{mvn clean}}.
> This was probably introduced in {{TestOfflineImageViewer}} and then
> replicated in {{TestCheckpoint}}, {{TestStandbyCheckpoints}}.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-11736) OIV tests should not write outside 'target' directory.
[
https://issues.apache.org/jira/browse/HDFS-11736?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16050615#comment-16050615
]
Akira Ajisaka commented on HDFS-11736:
--
Thanks [~jlowe] for the report. I've reverted this commit from branch-2 and
branch-2.8. Hi [~linyiqun], would you create patch for branch-2/2.8?
> OIV tests should not write outside 'target' directory.
> --
>
> Key: HDFS-11736
> URL: https://issues.apache.org/jira/browse/HDFS-11736
> Project: Hadoop HDFS
> Issue Type: Bug
>Affects Versions: 2.6.0
>Reporter: Konstantin Shvachko
>Assignee: Yiqun Lin
> Labels: newbie++, test
> Fix For: 2.7.4, 3.0.0-alpha4
>
> Attachments: HDFS-11736.001.patch, HDFS-11736.002.patch,
> HDFS-11736.003.patch, HDFS-11736-branch-2.7.001.patch,
> HDFS-11736-branch-2.7.002.patch
>
>
> A few tests use {{Files.createTempDir()}} from Guava package, but do not set
> {{java.io.tmpdir}} system property. Thus the temp directory is created in
> unpredictable places and is not being cleaned up by {{mvn clean}}.
> This was probably introduced in {{TestOfflineImageViewer}} and then
> replicated in {{TestCheckpoint}}, {{TestStandbyCheckpoints}}.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-11736) OIV tests should not write outside 'target' directory.
[
https://issues.apache.org/jira/browse/HDFS-11736?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Akira Ajisaka updated HDFS-11736:
-
Fix Version/s: (was: 2.8.2)
(was: 2.9.0)
> OIV tests should not write outside 'target' directory.
> --
>
> Key: HDFS-11736
> URL: https://issues.apache.org/jira/browse/HDFS-11736
> Project: Hadoop HDFS
> Issue Type: Bug
>Affects Versions: 2.6.0
>Reporter: Konstantin Shvachko
>Assignee: Yiqun Lin
> Labels: newbie++, test
> Fix For: 2.7.4, 3.0.0-alpha4
>
> Attachments: HDFS-11736.001.patch, HDFS-11736.002.patch,
> HDFS-11736.003.patch, HDFS-11736-branch-2.7.001.patch,
> HDFS-11736-branch-2.7.002.patch
>
>
> A few tests use {{Files.createTempDir()}} from Guava package, but do not set
> {{java.io.tmpdir}} system property. Thus the temp directory is created in
> unpredictable places and is not being cleaned up by {{mvn clean}}.
> This was probably introduced in {{TestOfflineImageViewer}} and then
> replicated in {{TestCheckpoint}}, {{TestStandbyCheckpoints}}.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-11977) Ozone: cannot enable test debug/trace log
[
https://issues.apache.org/jira/browse/HDFS-11977?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16050625#comment-16050625
]
Xiaoyu Yao commented on HDFS-11977:
---
Thanks [~szetszwo] for reporting and posting the patch to fix it. The patch
looks good to me. +1.
> Ozone: cannot enable test debug/trace log
> -
>
> Key: HDFS-11977
> URL: https://issues.apache.org/jira/browse/HDFS-11977
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: ozone, test
>Reporter: Tsz Wo Nicholas Sze
>Assignee: Tsz Wo Nicholas Sze
>Priority: Minor
> Attachments: HDFS-11977-HDFS-7240.20170615.patch
>
>
> Interestingly, the test debug/trace logs are not printed for Ozone classes
> even if we invoke GenericTestUtils.setLogLevel(log, Level.ALL). Other
> classes such as Object do not have such problem. Here is a test:
> {code}
> @Test
> public void testLogLevel() throws Exception {
> runTestLogLevel(StorageContainerManager.class);
> runTestLogLevel(Object.class);
> }
> static void runTestLogLevel(Class clazz) throws Exception {
> final Logger log = LoggerFactory.getLogger(clazz);
> GenericTestUtils.setLogLevel(log, Level.ALL);
> log.trace(clazz.getSimpleName() + " trace log");
> log.debug(clazz.getSimpleName() + " debug log");
> log.info(clazz.getSimpleName() + " info log");
> log.warn(clazz.getSimpleName() + " warn log");
> log.error(clazz.getSimpleName() + " error log");
> }
> {code}
> Output:
> {code}
> 2017-06-15 00:19:07,133 [Thread-0] INFO - StorageContainerManager info
> log
> 2017-06-15 00:19:07,135 [Thread-0] WARN - StorageContainerManager warn
> log
> 2017-06-15 00:19:07,135 [Thread-0] ERROR - StorageContainerManager error
> log
> 2017-06-15 00:19:07,135 [Thread-0] TRACE
> lang.Object(TestOzoneContainer.java:runTestLogLevel(64)) - Object trace log
> 2017-06-15 00:19:07,135 [Thread-0] DEBUG
> lang.Object(TestOzoneContainer.java:runTestLogLevel(65)) - Object debug log
> 2017-06-15 00:19:07,135 [Thread-0] INFO
> lang.Object(TestOzoneContainer.java:runTestLogLevel(66)) - Object info log
> 2017-06-15 00:19:07,135 [Thread-0] WARN
> lang.Object(TestOzoneContainer.java:runTestLogLevel(67)) - Object warn log
> 2017-06-15 00:19:07,135 [Thread-0] ERROR
> lang.Object(TestOzoneContainer.java:runTestLogLevel(68)) - Object error log
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-11977) Ozone: cannot enable test debug/trace log
[
https://issues.apache.org/jira/browse/HDFS-11977?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xiaoyu Yao updated HDFS-11977:
--
Resolution: Fixed
Hadoop Flags: Reviewed
Fix Version/s: HDFS-7240
Target Version/s: HDFS-7240
Status: Resolved (was: Patch Available)
Thanks [~szetszwo] for the contribution. I've commit the patch to the feature
branch.
> Ozone: cannot enable test debug/trace log
> -
>
> Key: HDFS-11977
> URL: https://issues.apache.org/jira/browse/HDFS-11977
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: ozone, test
>Reporter: Tsz Wo Nicholas Sze
>Assignee: Tsz Wo Nicholas Sze
>Priority: Minor
> Fix For: HDFS-7240
>
> Attachments: HDFS-11977-HDFS-7240.20170615.patch
>
>
> Interestingly, the test debug/trace logs are not printed for Ozone classes
> even if we invoke GenericTestUtils.setLogLevel(log, Level.ALL). Other
> classes such as Object do not have such problem. Here is a test:
> {code}
> @Test
> public void testLogLevel() throws Exception {
> runTestLogLevel(StorageContainerManager.class);
> runTestLogLevel(Object.class);
> }
> static void runTestLogLevel(Class clazz) throws Exception {
> final Logger log = LoggerFactory.getLogger(clazz);
> GenericTestUtils.setLogLevel(log, Level.ALL);
> log.trace(clazz.getSimpleName() + " trace log");
> log.debug(clazz.getSimpleName() + " debug log");
> log.info(clazz.getSimpleName() + " info log");
> log.warn(clazz.getSimpleName() + " warn log");
> log.error(clazz.getSimpleName() + " error log");
> }
> {code}
> Output:
> {code}
> 2017-06-15 00:19:07,133 [Thread-0] INFO - StorageContainerManager info
> log
> 2017-06-15 00:19:07,135 [Thread-0] WARN - StorageContainerManager warn
> log
> 2017-06-15 00:19:07,135 [Thread-0] ERROR - StorageContainerManager error
> log
> 2017-06-15 00:19:07,135 [Thread-0] TRACE
> lang.Object(TestOzoneContainer.java:runTestLogLevel(64)) - Object trace log
> 2017-06-15 00:19:07,135 [Thread-0] DEBUG
> lang.Object(TestOzoneContainer.java:runTestLogLevel(65)) - Object debug log
> 2017-06-15 00:19:07,135 [Thread-0] INFO
> lang.Object(TestOzoneContainer.java:runTestLogLevel(66)) - Object info log
> 2017-06-15 00:19:07,135 [Thread-0] WARN
> lang.Object(TestOzoneContainer.java:runTestLogLevel(67)) - Object warn log
> 2017-06-15 00:19:07,135 [Thread-0] ERROR
> lang.Object(TestOzoneContainer.java:runTestLogLevel(68)) - Object error log
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-11962) Ozone: Add stop-ozone.sh script
[
https://issues.apache.org/jira/browse/HDFS-11962?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16050629#comment-16050629
]
Hadoop QA commented on HDFS-11962:
--
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
19s{color} | {color:blue} Docker mode activated. {color} |
| {color:blue}0{color} | {color:blue} shelldocs {color} | {color:blue} 0m
5s{color} | {color:blue} Shelldocs was not available. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m
0s{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 15m
21s{color} | {color:green} HDFS-7240 passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
6s{color} | {color:green} HDFS-7240 passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
58s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} shellcheck {color} | {color:green} 0m
0s{color} | {color:green} There were no new shellcheck issues. {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
56s{color} | {color:green} hadoop-hdfs in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
21s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 20m 2s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Image:yetus/hadoop:14b5c93 |
| JIRA Issue | HDFS-11962 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12873133/HDFS-11962-HDFS-7240.003.patch
|
| Optional Tests | asflicense mvnsite unit shellcheck shelldocs |
| uname | Linux f6ea55b8c5e0 3.13.0-116-generic #163-Ubuntu SMP Fri Mar 31
14:13:22 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh
|
| git revision | HDFS-7240 / ba1afb2 |
| shellcheck | v0.4.6 |
| Test Results |
https://builds.apache.org/job/PreCommit-HDFS-Build/19919/testReport/ |
| modules | C: hadoop-hdfs-project/hadoop-hdfs U:
hadoop-hdfs-project/hadoop-hdfs |
| Console output |
https://builds.apache.org/job/PreCommit-HDFS-Build/19919/console |
| Powered by | Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org |
This message was automatically generated.
> Ozone: Add stop-ozone.sh script
> ---
>
> Key: HDFS-11962
> URL: https://issues.apache.org/jira/browse/HDFS-11962
> Project: Hadoop HDFS
> Issue Type: Sub-task
>Reporter: Anu Engineer
>Assignee: Weiwei Yang
> Attachments: HDFS-11962-HDFS-7240.001.patch,
> HDFS-11962-HDFS-7240.002.patch, HDFS-11962-HDFS-7240.003.patch
>
>
> This script should stop the cluster along with all ozone services. --
> # run 'stop-dfs.sh'
> # run 'hdfs --daemon stop ksm'
> # run 'hdfs --daemon stop scm`'
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-11972) CBlocks use wrong OPT env vars
[
https://issues.apache.org/jira/browse/HDFS-11972?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16050640#comment-16050640
]
Anu Engineer commented on HDFS-11972:
-
+1, Looks good to me. Thanks for taking care of this. Let us hold off on
committing this and give Allen a chance to look at it.
> CBlocks use wrong OPT env vars
> --
>
> Key: HDFS-11972
> URL: https://issues.apache.org/jira/browse/HDFS-11972
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: hdfs
>Reporter: Allen Wittenauer
>Assignee: Mukul Kumar Singh
> Attachments: HDFS-11972-HDFS-7240.001.patch
>
>
> Current codebase does:
> {code}
> hadoop_debug "Appending HADOOP_CBLOCK_OPTS onto HADOOP_OPTS"
> HADOOP_OPTS="${HADOOP_OPTS} ${HADOOP_CBLOCK_OPTS}"
> ...
> hadoop_debug "Appending HADOOP_JSCSI_OPTS onto HADOOP_OPTS"
> HADOOP_OPTS="${HADOOP_OPTS} ${HADOOP_JSCSI_OPTS}"
> {code}
> This code block breaks the consistency with the rest of the shell scripts:
> a) It should be HDFS_CBLOCKSERVER_OPTS, HDFS_JSCSI_OPTS
> b) _OPTS, unless they are a deprecated form or some other special case, are
> already/automatically appended; there is no need to do them specifically
> Also, a description of these env vars should be hadoop-env.sh for
> documentation purposes.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-11472) Fix inconsistent replica size after a data pipeline failure
[
https://issues.apache.org/jira/browse/HDFS-11472?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16050642#comment-16050642
]
Wei-Chiu Chuang commented on HDFS-11472:
Hi Konstantin thanks for the review,
1. If blockFileLength < bytesAcked = bytesOnDisk, that means the block actually
written into disk is less than what DataNode thinks it does, in which case
there is nothing I can do to salvage the replica, and I believe throwing an
exception and let recoverRbw() to fail is reasonable.
2. What you suggested is why existing recovery handling doesn't work (it would
skip truncate). The extra conditions needed is to handle this special case
where there is a mistmatch between in-memory and on-disk length.
3. That sounds a good suggestion.
> Fix inconsistent replica size after a data pipeline failure
> ---
>
> Key: HDFS-11472
> URL: https://issues.apache.org/jira/browse/HDFS-11472
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: datanode
>Reporter: Wei-Chiu Chuang
>Assignee: Wei-Chiu Chuang
>Priority: Critical
> Labels: release-blocker
> Attachments: HDFS-11472.001.patch, HDFS-11472.002.patch,
> HDFS-11472.003.patch, HDFS-11472.testcase.patch
>
>
> We observed a case where a replica's on disk length is less than acknowledged
> length, breaking the assumption in recovery code.
> {noformat}
> 2017-01-08 01:41:03,532 WARN
> org.apache.hadoop.hdfs.server.protocol.InterDatanodeProtocol: Failed to
> obtain replica info for block
> (=BP-947993742-10.204.0.136-1362248978912:blk_2526438952_1101394519586) from
> datanode (=DatanodeInfoWithStorage[10.204.138.17:1004,null,null])
> java.io.IOException: THIS IS NOT SUPPOSED TO HAPPEN: getBytesOnDisk() <
> getVisibleLength(), rip=ReplicaBeingWritten, blk_2526438952_1101394519586, RBW
> getNumBytes() = 27530
> getBytesOnDisk() = 27006
> getVisibleLength()= 27268
> getVolume() = /data/6/hdfs/datanode/current
> getBlockFile()=
> /data/6/hdfs/datanode/current/BP-947993742-10.204.0.136-1362248978912/current/rbw/blk_2526438952
> bytesAcked=27268
> bytesOnDisk=27006
> at
> org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl.initReplicaRecovery(FsDatasetImpl.java:2284)
> at
> org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl.initReplicaRecovery(FsDatasetImpl.java:2260)
> at
> org.apache.hadoop.hdfs.server.datanode.DataNode.initReplicaRecovery(DataNode.java:2566)
> at
> org.apache.hadoop.hdfs.server.datanode.DataNode.callInitReplicaRecovery(DataNode.java:2577)
> at
> org.apache.hadoop.hdfs.server.datanode.DataNode.recoverBlock(DataNode.java:2645)
> at
> org.apache.hadoop.hdfs.server.datanode.DataNode.access$400(DataNode.java:245)
> at
> org.apache.hadoop.hdfs.server.datanode.DataNode$5.run(DataNode.java:2551)
> at java.lang.Thread.run(Thread.java:745)
> {noformat}
> It turns out that if an exception is thrown within
> {{BlockReceiver#receivePacket}}, the in-memory replica on disk length may not
> be updated, but the data is written to disk anyway.
> For example, here's one exception we observed
> {noformat}
> 2017-01-08 01:40:59,512 INFO org.apache.hadoop.hdfs.server.datanode.DataNode:
> Exception for
> BP-947993742-10.204.0.136-1362248978912:blk_2526438952_1101394499067
> java.nio.channels.ClosedByInterruptException
> at
> java.nio.channels.spi.AbstractInterruptibleChannel.end(AbstractInterruptibleChannel.java:202)
> at sun.nio.ch.FileChannelImpl.position(FileChannelImpl.java:269)
> at
> org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl.adjustCrcChannelPosition(FsDatasetImpl.java:1484)
> at
> org.apache.hadoop.hdfs.server.datanode.BlockReceiver.adjustCrcFilePosition(BlockReceiver.java:994)
> at
> org.apache.hadoop.hdfs.server.datanode.BlockReceiver.receivePacket(BlockReceiver.java:670)
> at
> org.apache.hadoop.hdfs.server.datanode.BlockReceiver.receiveBlock(BlockReceiver.java:857)
> at
> org.apache.hadoop.hdfs.server.datanode.DataXceiver.writeBlock(DataXceiver.java:797)
> at
> org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.opWriteBlock(Receiver.java:169)
> at
> org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.processOp(Receiver.java:106)
> at
> org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:244)
> at java.lang.Thread.run(Thread.java:745)
> {noformat}
> There are potentially other places and causes where an exception is thrown
> within {{BlockReceiver#receivePacket}}, so it may not make much sense to
> alleviate it for this particular exception. Instead, we should improve
> replica recovery code to handle the case whe
[jira] [Issue Comment Deleted] (HDFS-11472) Fix inconsistent replica size after a data pipeline failure
[
https://issues.apache.org/jira/browse/HDFS-11472?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Wei-Chiu Chuang updated HDFS-11472:
---
Comment: was deleted
(was: Hi Konstantin thanks for the review,
1. If blockFileLength < bytesAcked = bytesOnDisk, that means the block actually
written into disk is less than what DataNode thinks it does, in which case
there is nothing I can do to salvage the replica, and I believe throwing an
exception and let recoverRbw() to fail is reasonable.
2. What you suggested is why existing recovery handling doesn't work (it would
skip truncate). The extra conditions needed is to handle this special case
where there is a mistmatch between in-memory and on-disk length.
3. That sounds a good suggestion.)
> Fix inconsistent replica size after a data pipeline failure
> ---
>
> Key: HDFS-11472
> URL: https://issues.apache.org/jira/browse/HDFS-11472
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: datanode
>Reporter: Wei-Chiu Chuang
>Assignee: Wei-Chiu Chuang
>Priority: Critical
> Labels: release-blocker
> Attachments: HDFS-11472.001.patch, HDFS-11472.002.patch,
> HDFS-11472.003.patch, HDFS-11472.testcase.patch
>
>
> We observed a case where a replica's on disk length is less than acknowledged
> length, breaking the assumption in recovery code.
> {noformat}
> 2017-01-08 01:41:03,532 WARN
> org.apache.hadoop.hdfs.server.protocol.InterDatanodeProtocol: Failed to
> obtain replica info for block
> (=BP-947993742-10.204.0.136-1362248978912:blk_2526438952_1101394519586) from
> datanode (=DatanodeInfoWithStorage[10.204.138.17:1004,null,null])
> java.io.IOException: THIS IS NOT SUPPOSED TO HAPPEN: getBytesOnDisk() <
> getVisibleLength(), rip=ReplicaBeingWritten, blk_2526438952_1101394519586, RBW
> getNumBytes() = 27530
> getBytesOnDisk() = 27006
> getVisibleLength()= 27268
> getVolume() = /data/6/hdfs/datanode/current
> getBlockFile()=
> /data/6/hdfs/datanode/current/BP-947993742-10.204.0.136-1362248978912/current/rbw/blk_2526438952
> bytesAcked=27268
> bytesOnDisk=27006
> at
> org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl.initReplicaRecovery(FsDatasetImpl.java:2284)
> at
> org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl.initReplicaRecovery(FsDatasetImpl.java:2260)
> at
> org.apache.hadoop.hdfs.server.datanode.DataNode.initReplicaRecovery(DataNode.java:2566)
> at
> org.apache.hadoop.hdfs.server.datanode.DataNode.callInitReplicaRecovery(DataNode.java:2577)
> at
> org.apache.hadoop.hdfs.server.datanode.DataNode.recoverBlock(DataNode.java:2645)
> at
> org.apache.hadoop.hdfs.server.datanode.DataNode.access$400(DataNode.java:245)
> at
> org.apache.hadoop.hdfs.server.datanode.DataNode$5.run(DataNode.java:2551)
> at java.lang.Thread.run(Thread.java:745)
> {noformat}
> It turns out that if an exception is thrown within
> {{BlockReceiver#receivePacket}}, the in-memory replica on disk length may not
> be updated, but the data is written to disk anyway.
> For example, here's one exception we observed
> {noformat}
> 2017-01-08 01:40:59,512 INFO org.apache.hadoop.hdfs.server.datanode.DataNode:
> Exception for
> BP-947993742-10.204.0.136-1362248978912:blk_2526438952_1101394499067
> java.nio.channels.ClosedByInterruptException
> at
> java.nio.channels.spi.AbstractInterruptibleChannel.end(AbstractInterruptibleChannel.java:202)
> at sun.nio.ch.FileChannelImpl.position(FileChannelImpl.java:269)
> at
> org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl.adjustCrcChannelPosition(FsDatasetImpl.java:1484)
> at
> org.apache.hadoop.hdfs.server.datanode.BlockReceiver.adjustCrcFilePosition(BlockReceiver.java:994)
> at
> org.apache.hadoop.hdfs.server.datanode.BlockReceiver.receivePacket(BlockReceiver.java:670)
> at
> org.apache.hadoop.hdfs.server.datanode.BlockReceiver.receiveBlock(BlockReceiver.java:857)
> at
> org.apache.hadoop.hdfs.server.datanode.DataXceiver.writeBlock(DataXceiver.java:797)
> at
> org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.opWriteBlock(Receiver.java:169)
> at
> org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.processOp(Receiver.java:106)
> at
> org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:244)
> at java.lang.Thread.run(Thread.java:745)
> {noformat}
> There are potentially other places and causes where an exception is thrown
> within {{BlockReceiver#receivePacket}}, so it may not make much sense to
> alleviate it for this particular exception. Instead, we should improve
> replica recovery code to handle the case where ondisk size is less th
[jira] [Commented] (HDFS-11782) Ozone: KSM: Add listKey
[
https://issues.apache.org/jira/browse/HDFS-11782?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16050667#comment-16050667
]
Hadoop QA commented on HDFS-11782:
--
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
19s{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
39s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 17m
5s{color} | {color:green} HDFS-7240 passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
50s{color} | {color:green} HDFS-7240 passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
49s{color} | {color:green} HDFS-7240 passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
48s{color} | {color:green} HDFS-7240 passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
56s{color} | {color:green} HDFS-7240 passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
44s{color} | {color:green} HDFS-7240 passed {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
8s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
28s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
31s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} cc {color} | {color:green} 1m
31s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
31s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
40s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
31s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
53s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
38s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 1m
18s{color} | {color:green} hadoop-hdfs-client in the patch passed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red}116m 27s{color}
| {color:red} hadoop-hdfs in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
34s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}158m 57s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.hdfs.TestDFSStripedOutputStreamWithFailure070 |
| | hadoop.hdfs.TestDFSStripedOutputStreamWithFailure080 |
| | hadoop.cblock.TestBufferManager |
| | hadoop.hdfs.server.namenode.ha.TestDFSUpgradeWithHA |
| | hadoop.cblock.TestCBlockReadWrite |
| Timed out junit tests | org.apache.hadoop.ozone.scm.node.TestNodeManager |
| | org.apache.hadoop.hdfs.server.blockmanagement.TestBlockStatsMXBean |
| | org.apache.hadoop.ozone.container.ozoneimpl.TestOzoneContainerRatis |
| | org.apache.hadoop.cblock.TestLocalBlockCache |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Image:yetus/hadoop:14b5c93 |
| JIRA Issue | HDFS-11782 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12873116/HDFS-11782-HDFS-7240.004.patch
|
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite
unit findbugs checkstyle cc |
| uname | Linux 85169d2c070c 3.13.0-108-generic #155-Ubuntu SMP Wed Jan 11
16:58:52 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh
|
| git revision | HDFS-7240 / ba1afb2 |
| Default Java | 1.8.0_131 |
| findbugs | v3.1.0-RC1 |
| unit |
https://builds.apache.org/job/PreCommit-HDFS-Build/19918/artifact/patc
[jira] [Commented] (HDFS-11345) Document the configuration key for FSNamesystem lock fairness
[ https://issues.apache.org/jira/browse/HDFS-11345?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16050674#comment-16050674 ] Erik Krogen commented on HDFS-11345: Yup, thanks [~ajisakaa]. I need to configure my IDE to stop doing that on me. Attached v002 patch. > Document the configuration key for FSNamesystem lock fairness > - > > Key: HDFS-11345 > URL: https://issues.apache.org/jira/browse/HDFS-11345 > Project: Hadoop HDFS > Issue Type: Improvement > Components: documentation, namenode >Reporter: Zhe Zhang >Assignee: Erik Krogen >Priority: Minor > Attachments: HADOOP-11345.000.patch, HADOOP-11345.001.patch > > > Per [earlier | > https://issues.apache.org/jira/browse/HDFS-5239?focusedCommentId=15536471&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-15536471] > discussion. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-11345) Document the configuration key for FSNamesystem lock fairness
[ https://issues.apache.org/jira/browse/HDFS-11345?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Erik Krogen updated HDFS-11345: --- Attachment: HDFS-11345.002.patch > Document the configuration key for FSNamesystem lock fairness > - > > Key: HDFS-11345 > URL: https://issues.apache.org/jira/browse/HDFS-11345 > Project: Hadoop HDFS > Issue Type: Improvement > Components: documentation, namenode >Reporter: Zhe Zhang >Assignee: Erik Krogen >Priority: Minor > Attachments: HADOOP-11345.000.patch, HADOOP-11345.001.patch, > HDFS-11345.002.patch > > > Per [earlier | > https://issues.apache.org/jira/browse/HDFS-5239?focusedCommentId=15536471&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-15536471] > discussion. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-11977) Ozone: cannot enable test debug/trace log
[
https://issues.apache.org/jira/browse/HDFS-11977?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16050742#comment-16050742
]
Anu Engineer commented on HDFS-11977:
-
+1 thanks for fixing this.
> Ozone: cannot enable test debug/trace log
> -
>
> Key: HDFS-11977
> URL: https://issues.apache.org/jira/browse/HDFS-11977
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: ozone, test
>Reporter: Tsz Wo Nicholas Sze
>Assignee: Tsz Wo Nicholas Sze
>Priority: Minor
> Fix For: HDFS-7240
>
> Attachments: HDFS-11977-HDFS-7240.20170615.patch
>
>
> Interestingly, the test debug/trace logs are not printed for Ozone classes
> even if we invoke GenericTestUtils.setLogLevel(log, Level.ALL). Other
> classes such as Object do not have such problem. Here is a test:
> {code}
> @Test
> public void testLogLevel() throws Exception {
> runTestLogLevel(StorageContainerManager.class);
> runTestLogLevel(Object.class);
> }
> static void runTestLogLevel(Class clazz) throws Exception {
> final Logger log = LoggerFactory.getLogger(clazz);
> GenericTestUtils.setLogLevel(log, Level.ALL);
> log.trace(clazz.getSimpleName() + " trace log");
> log.debug(clazz.getSimpleName() + " debug log");
> log.info(clazz.getSimpleName() + " info log");
> log.warn(clazz.getSimpleName() + " warn log");
> log.error(clazz.getSimpleName() + " error log");
> }
> {code}
> Output:
> {code}
> 2017-06-15 00:19:07,133 [Thread-0] INFO - StorageContainerManager info
> log
> 2017-06-15 00:19:07,135 [Thread-0] WARN - StorageContainerManager warn
> log
> 2017-06-15 00:19:07,135 [Thread-0] ERROR - StorageContainerManager error
> log
> 2017-06-15 00:19:07,135 [Thread-0] TRACE
> lang.Object(TestOzoneContainer.java:runTestLogLevel(64)) - Object trace log
> 2017-06-15 00:19:07,135 [Thread-0] DEBUG
> lang.Object(TestOzoneContainer.java:runTestLogLevel(65)) - Object debug log
> 2017-06-15 00:19:07,135 [Thread-0] INFO
> lang.Object(TestOzoneContainer.java:runTestLogLevel(66)) - Object info log
> 2017-06-15 00:19:07,135 [Thread-0] WARN
> lang.Object(TestOzoneContainer.java:runTestLogLevel(67)) - Object warn log
> 2017-06-15 00:19:07,135 [Thread-0] ERROR
> lang.Object(TestOzoneContainer.java:runTestLogLevel(68)) - Object error log
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-11961) Ozone: Add start-ozone.sh to quickly start ozone.
[
https://issues.apache.org/jira/browse/HDFS-11961?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16050758#comment-16050758
]
Hadoop QA commented on HDFS-11961:
--
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
12s{color} | {color:blue} Docker mode activated. {color} |
| {color:blue}0{color} | {color:blue} shelldocs {color} | {color:blue} 0m
4s{color} | {color:blue} Shelldocs was not available. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m
0s{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 14m
24s{color} | {color:green} HDFS-7240 passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
54s{color} | {color:green} HDFS-7240 passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
36s{color} | {color:green} HDFS-7240 passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
59s{color} | {color:green} HDFS-7240 passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
52s{color} | {color:green} HDFS-7240 passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
50s{color} | {color:green} HDFS-7240 passed {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
53s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
50s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
50s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
34s{color} | {color:green} hadoop-hdfs-project/hadoop-hdfs: The patch generated
0 new + 11 unchanged - 1 fixed = 11 total (was 12) {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
56s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} shellcheck {color} | {color:green} 0m
0s{color} | {color:green} There were no new shellcheck issues. {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
57s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
48s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 76m 49s{color}
| {color:red} hadoop-hdfs in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
20s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}104m 12s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.hdfs.TestDFSStripedOutputStreamWithFailure070 |
| | hadoop.ozone.container.ozoneimpl.TestRatisManager |
| | hadoop.hdfs.server.datanode.TestDataNodeVolumeFailure |
| | hadoop.hdfs.server.datanode.TestDataNodeVolumeFailureReporting |
| | hadoop.hdfs.web.TestWebHdfsTimeouts |
| | hadoop.hdfs.server.datanode.TestDirectoryScanner |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Image:yetus/hadoop:14b5c93 |
| JIRA Issue | HDFS-11961 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12873136/HDFS-11961-HDFS-7240.002.patch
|
| Optional Tests | asflicense mvnsite unit shellcheck shelldocs compile
javac javadoc mvninstall findbugs checkstyle |
| uname | Linux 8259240279dd 4.4.0-43-generic #63-Ubuntu SMP Wed Oct 12
13:48:03 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh
|
| git revision | HDFS-7240 / ba1afb2 |
| Default Java | 1.8.0_131 |
| shellcheck | v0.4.6 |
| findbugs | v3.1.0-RC1 |
| unit |
https://builds.apache.org/job/PreCommit-HDFS-Build/19920/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-HDFS-Build/19920/testReport/ |
| modules | C: hadoop-hdfs-project/hadoop-hdfs U:
hadoop-hdf
[jira] [Commented] (HDFS-11961) Ozone: Add start-ozone.sh to quickly start ozone.
[ https://issues.apache.org/jira/browse/HDFS-11961?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16050770#comment-16050770 ] Anu Engineer commented on HDFS-11961: - +1, I will commit this shortly. Thanks for taking care of this. > Ozone: Add start-ozone.sh to quickly start ozone. > - > > Key: HDFS-11961 > URL: https://issues.apache.org/jira/browse/HDFS-11961 > Project: Hadoop HDFS > Issue Type: Sub-task >Reporter: Anu Engineer >Assignee: Weiwei Yang > Attachments: HDFS-11961-HDFS-7240.001.patch, > HDFS-11961-HDFS-7240.002.patch > > > Add start ozone script. Internally this script should call into > # start-dfs.sh > # run `hdfs --daemon start scm' > # run `hdfs --daemon start ksm` > This just makes it easy to start Ozone with a single command. This command > assumes that Namenode format has been run before, since it will bring up HDFS > also. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-11961) Ozone: Add start-ozone.sh to quickly start ozone.
[ https://issues.apache.org/jira/browse/HDFS-11961?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anu Engineer updated HDFS-11961: Resolution: Fixed Hadoop Flags: Reviewed Fix Version/s: HDFS-7240 Status: Resolved (was: Patch Available) [~cheersyang] Thanks for the contribution. I have committed this to the feature branch. > Ozone: Add start-ozone.sh to quickly start ozone. > - > > Key: HDFS-11961 > URL: https://issues.apache.org/jira/browse/HDFS-11961 > Project: Hadoop HDFS > Issue Type: Sub-task >Reporter: Anu Engineer >Assignee: Weiwei Yang > Fix For: HDFS-7240 > > Attachments: HDFS-11961-HDFS-7240.001.patch, > HDFS-11961-HDFS-7240.002.patch > > > Add start ozone script. Internally this script should call into > # start-dfs.sh > # run `hdfs --daemon start scm' > # run `hdfs --daemon start ksm` > This just makes it easy to start Ozone with a single command. This command > assumes that Namenode format has been run before, since it will bring up HDFS > also. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-11962) Ozone: Add stop-ozone.sh script
[ https://issues.apache.org/jira/browse/HDFS-11962?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16050797#comment-16050797 ] Anu Engineer commented on HDFS-11962: - +1, I will commit this shortly. > Ozone: Add stop-ozone.sh script > --- > > Key: HDFS-11962 > URL: https://issues.apache.org/jira/browse/HDFS-11962 > Project: Hadoop HDFS > Issue Type: Sub-task >Reporter: Anu Engineer >Assignee: Weiwei Yang > Attachments: HDFS-11962-HDFS-7240.001.patch, > HDFS-11962-HDFS-7240.002.patch, HDFS-11962-HDFS-7240.003.patch > > > This script should stop the cluster along with all ozone services. -- > # run 'stop-dfs.sh' > # run 'hdfs --daemon stop ksm' > # run 'hdfs --daemon stop scm`' -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-11962) Ozone: Add stop-ozone.sh script
[ https://issues.apache.org/jira/browse/HDFS-11962?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anu Engineer updated HDFS-11962: Resolution: Fixed Hadoop Flags: Reviewed Status: Resolved (was: Patch Available) [~cheersyang] Thank you for the contribution. I have committed this to the feature branch. > Ozone: Add stop-ozone.sh script > --- > > Key: HDFS-11962 > URL: https://issues.apache.org/jira/browse/HDFS-11962 > Project: Hadoop HDFS > Issue Type: Sub-task >Reporter: Anu Engineer >Assignee: Weiwei Yang > Attachments: HDFS-11962-HDFS-7240.001.patch, > HDFS-11962-HDFS-7240.002.patch, HDFS-11962-HDFS-7240.003.patch > > > This script should stop the cluster along with all ozone services. -- > # run 'stop-dfs.sh' > # run 'hdfs --daemon stop ksm' > # run 'hdfs --daemon stop scm`' -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-11682) TestBalancer#testBalancerWithStripedFile is flaky
[ https://issues.apache.org/jira/browse/HDFS-11682?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lei (Eddy) Xu updated HDFS-11682: - Resolution: Fixed Fix Version/s: 3.0.0-alpha4 Status: Resolved (was: Patch Available) Thanks for the reviews, [~manojg] and [~andrew.wang]. Committed. > TestBalancer#testBalancerWithStripedFile is flaky > - > > Key: HDFS-11682 > URL: https://issues.apache.org/jira/browse/HDFS-11682 > Project: Hadoop HDFS > Issue Type: Bug > Components: test >Affects Versions: 3.0.0-alpha4 >Reporter: Andrew Wang >Assignee: Lei (Eddy) Xu > Fix For: 3.0.0-alpha4 > > Attachments: HDFS-11682.00.patch, HDFS-11682.01.patch, > IndexOutOfBoundsException.log, timeout.log > > > Saw this fail in two different ways on a precommit run, but pass locally. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-11345) Document the configuration key for FSNamesystem lock fairness
[
https://issues.apache.org/jira/browse/HDFS-11345?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16050893#comment-16050893
]
Hadoop QA commented on HDFS-11345:
--
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
17s{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 13m
50s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
54s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
46s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
58s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
48s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
47s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
58s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
56s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
56s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
44s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
58s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m
2s{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
5s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
44s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 96m 32s{color}
| {color:red} hadoop-hdfs in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
24s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}124m 15s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.hdfs.TestDFSStripedOutputStreamWithFailure070 |
| | hadoop.hdfs.TestClientProtocolForPipelineRecovery |
| | hadoop.hdfs.server.namenode.TestDecommissioningStatus |
| Timed out junit tests |
org.apache.hadoop.hdfs.server.blockmanagement.TestBlockStatsMXBean |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Image:yetus/hadoop:14b5c93 |
| JIRA Issue | HDFS-11345 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12873149/HDFS-11345.002.patch |
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite
unit findbugs checkstyle xml |
| uname | Linux e0e55daa7bcf 3.13.0-108-generic #155-Ubuntu SMP Wed Jan 11
16:58:52 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh
|
| git revision | trunk / d780a67 |
| Default Java | 1.8.0_131 |
| findbugs | v3.1.0-RC1 |
| unit |
https://builds.apache.org/job/PreCommit-HDFS-Build/19921/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-HDFS-Build/19921/testReport/ |
| modules | C: hadoop-hdfs-project/hadoop-hdfs U:
hadoop-hdfs-project/hadoop-hdfs |
| Console output |
https://builds.apache.org/job/PreCommit-HDFS-Build/19921/console |
| Powered by | Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org |
This message was automatically generated.
> Document the configuration key for FSNamesystem lock fairness
> -
>
> Key: HDFS-11345
> URL: https://issues.apache.org/jira/browse/HDFS-11345
> Project: Hadoop HDFS
> Issue
[jira] [Commented] (HDFS-11646) Add -E option in 'ls' to list erasure coding policy of each file and directory if applicable
[
https://issues.apache.org/jira/browse/HDFS-11646?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16050936#comment-16050936
]
Lei (Eddy) Xu commented on HDFS-11646:
--
Hi, [~luhuichun]
Sorry, I should make my comments more clear. In {{Ls#processPath}}, it prints
EC policy using {{String.format(lineFormat, )}}, while the placeholder of
ec policy name is not set from {{adjustColumnWidths()}}. So It does not to
print path if {{displayPolicy}} is enabled. The test
{{testErasureCodingConf.xml}} seemed not to catch this bug. Could you fix it?
Thanks!
> Add -E option in 'ls' to list erasure coding policy of each file and
> directory if applicable
>
>
> Key: HDFS-11646
> URL: https://issues.apache.org/jira/browse/HDFS-11646
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: erasure-coding
>Reporter: SammiChen
>Assignee: luhuichun
> Labels: hdfs-ec-3.0-nice-to-have
> Attachments: HDFS-11646-001.patch, HDFS-11646-002.patch,
> HDFS-11646-003.patch, HDFS-11646-004.patch
>
>
> Add -E option in "ls" to show erasure coding policy of file and directory,
> leverage the "number_of_replicas " column.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-11682) TestBalancer#testBalancerWithStripedFile is flaky
[ https://issues.apache.org/jira/browse/HDFS-11682?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16050938#comment-16050938 ] Hudson commented on HDFS-11682: --- SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #11870 (See [https://builds.apache.org/job/Hadoop-trunk-Commit/11870/]) HDFS-11682. TestBalancer.testBalancerWithStripedFile is flaky. (lei) (lei: rev 3f5108723c6272d2fded8d3563c4b793e1d88f8b) * (edit) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/balancer/TestBalancer.java > TestBalancer#testBalancerWithStripedFile is flaky > - > > Key: HDFS-11682 > URL: https://issues.apache.org/jira/browse/HDFS-11682 > Project: Hadoop HDFS > Issue Type: Bug > Components: test >Affects Versions: 3.0.0-alpha4 >Reporter: Andrew Wang >Assignee: Lei (Eddy) Xu > Fix For: 3.0.0-alpha4 > > Attachments: HDFS-11682.00.patch, HDFS-11682.01.patch, > IndexOutOfBoundsException.log, timeout.log > > > Saw this fail in two different ways on a precommit run, but pass locally. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-11943) Warn log frequently print to screen in doEncode function on AbstractNativeRawEncoder class
[ https://issues.apache.org/jira/browse/HDFS-11943?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16050939#comment-16050939 ] SammiChen commented on HDFS-11943: -- Hi [~liaoyuxiangqin], thanks for provide the detail information. Based on what we know currently, I would suggest that we add the bq. if (usingDirectBuffer) condition check to correctly popup the warn message at the right time. What do you think? > Warn log frequently print to screen in doEncode function on > AbstractNativeRawEncoder class > --- > > Key: HDFS-11943 > URL: https://issues.apache.org/jira/browse/HDFS-11943 > Project: Hadoop HDFS > Issue Type: Improvement > Components: erasure-coding, native >Affects Versions: 3.0.0-alpha4 > Environment: cluster: 3 nodes > os:(Red Hat 2.6.33.20, Red Hat 3.10.0-514.6.1.el7.x86_64, > Ubuntu4.4.0-31-generic) > hadoop version: hadoop-3.0.0-alpha4 > erasure coding: XOR-2-1-64k and enabled Intel ISA-L > hadoop fs -put file / >Reporter: liaoyuxiangqin >Assignee: liaoyuxiangqin >Priority: Minor > Attachments: HDFS-11943.002.patch, HDFS-11943.patch > > Original Estimate: 0.05h > Remaining Estimate: 0.05h > > when i write file to hdfs on above environment, the hdfs client frequently > print warn log of use direct ByteBuffer inputs/outputs in doEncode function > to screen, detail information as follows: > 2017-06-07 15:20:42,856 WARN rawcoder.AbstractNativeRawEncoder: > convertToByteBufferState is invoked, not efficiently. Please use direct > ByteBuffer inputs/outputs -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-11606) Add CLI cmd to remove an erasure code policy
[ https://issues.apache.org/jira/browse/HDFS-11606?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16050944#comment-16050944 ] SammiChen commented on HDFS-11606: -- Hi [~eddyxu], it has kind of dependency on HDFS-7859 regarding the fsImage update part. > Add CLI cmd to remove an erasure code policy > > > Key: HDFS-11606 > URL: https://issues.apache.org/jira/browse/HDFS-11606 > Project: Hadoop HDFS > Issue Type: Sub-task > Components: erasure-coding >Reporter: Kai Zheng >Assignee: Tim Yao > Fix For: 3.0.0-alpha4 > > Attachments: HDFS-11606.01.patch > > > This is to develop a CLI cmd allowing user to remove a user defined erasure > code policy by specifying its name. Note if the policy is referenced and used > by existing HDFS files, the removal should fail with a good message. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-11606) Add CLI cmd to remove an erasure code policy
[ https://issues.apache.org/jira/browse/HDFS-11606?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16051012#comment-16051012 ] Wei-Chiu Chuang commented on HDFS-11606: One quick suggestion: You should also add a removeErasureCodingPolicy API in HdfsAdmin, which is basically a wrapper around DistributedFileSystem#removeErasureCodingPolicy. See HdfsAdmin#setErasureCodingPolicy for reference. > Add CLI cmd to remove an erasure code policy > > > Key: HDFS-11606 > URL: https://issues.apache.org/jira/browse/HDFS-11606 > Project: Hadoop HDFS > Issue Type: Sub-task > Components: erasure-coding >Reporter: Kai Zheng >Assignee: Tim Yao > Fix For: 3.0.0-alpha4 > > Attachments: HDFS-11606.01.patch > > > This is to develop a CLI cmd allowing user to remove a user defined erasure > code policy by specifying its name. Note if the policy is referenced and used > by existing HDFS files, the removal should fail with a good message. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-8520) Patch for PPC64 block size
[ https://issues.apache.org/jira/browse/HDFS-8520?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16051034#comment-16051034 ] Andrew Wang commented on HDFS-8520: --- I think my earlier comments on this topic are pretty clear. We still don't have a PPC64 build host in our pool or official PPC64 support AFAIK. > Patch for PPC64 block size > -- > > Key: HDFS-8520 > URL: https://issues.apache.org/jira/browse/HDFS-8520 > Project: Hadoop HDFS > Issue Type: Bug >Affects Versions: 2.7.1 > Environment: RHEL 7.1 /PPC64 >Reporter: Tony Reix >Assignee: Tony Reix > Labels: ppc64 > Attachments: HDFS-8520-2.patch, HDFS-8520.patch > > > The attached patch enables Hadoop to work on PPC64. > That deals with SystemPageSize and BloclSize , which are not 4096 on PPC64. > There are changes in 3 files: > - > hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/io/nativeio/NativeIO.java > - > hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/TestFsDatasetCache.java > - > hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestCacheDirectives.java > where 4096 is replaced by getOperatingSystemPageSize() or by using PAGE_SIZE > The patch has been built on branch-2.7 . -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-8520) Patch for PPC64 block size
[ https://issues.apache.org/jira/browse/HDFS-8520?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Wang updated HDFS-8520: -- Status: Open (was: Patch Available) > Patch for PPC64 block size > -- > > Key: HDFS-8520 > URL: https://issues.apache.org/jira/browse/HDFS-8520 > Project: Hadoop HDFS > Issue Type: Bug >Affects Versions: 2.7.1 > Environment: RHEL 7.1 /PPC64 >Reporter: Tony Reix >Assignee: Tony Reix > Labels: ppc64 > Attachments: HDFS-8520-2.patch, HDFS-8520.patch > > > The attached patch enables Hadoop to work on PPC64. > That deals with SystemPageSize and BloclSize , which are not 4096 on PPC64. > There are changes in 3 files: > - > hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/io/nativeio/NativeIO.java > - > hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/TestFsDatasetCache.java > - > hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestCacheDirectives.java > where 4096 is replaced by getOperatingSystemPageSize() or by using PAGE_SIZE > The patch has been built on branch-2.7 . -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-11606) Add CLI cmd to remove an erasure code policy
[ https://issues.apache.org/jira/browse/HDFS-11606?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16051057#comment-16051057 ] SammiChen commented on HDFS-11606: -- Hi [~jojochuang], well reminder! I will add the API in HdfsAdmin later. > Add CLI cmd to remove an erasure code policy > > > Key: HDFS-11606 > URL: https://issues.apache.org/jira/browse/HDFS-11606 > Project: Hadoop HDFS > Issue Type: Sub-task > Components: erasure-coding >Reporter: Kai Zheng >Assignee: Tim Yao > Fix For: 3.0.0-alpha4 > > Attachments: HDFS-11606.01.patch > > > This is to develop a CLI cmd allowing user to remove a user defined erasure > code policy by specifying its name. Note if the policy is referenced and used > by existing HDFS files, the removal should fail with a good message. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-11956) Fix BlockToken compatibility with Hadoop 2.x clients
[
https://issues.apache.org/jira/browse/HDFS-11956?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16051084#comment-16051084
]
Andrew Wang commented on HDFS-11956:
LGTM, though we should also add an entry to hdfs-default.xml as documentation
for this new option, and some of the checkstyles look fixable. Would appreciate
if you could validate that the failed unit tests are unrelated (sadly, there
are a lot).
[~chris.douglas] do you want to review as well?
> Fix BlockToken compatibility with Hadoop 2.x clients
>
>
> Key: HDFS-11956
> URL: https://issues.apache.org/jira/browse/HDFS-11956
> Project: Hadoop HDFS
> Issue Type: Bug
>Affects Versions: 3.0.0-alpha4
>Reporter: Andrew Wang
>Assignee: Ewan Higgs
>Priority: Blocker
> Attachments: HDFS-11956.001.patch, HDFS-11956.002.patch
>
>
> Seems like HDFS-9807 broke backwards compatibility with Hadoop 2.x clients.
> When talking to a 3.0.0-alpha4 DN with security on:
> {noformat}
> 2017-06-06 23:27:22,568 WARN org.apache.hadoop.hdfs.server.datanode.DataNode:
> Block token verification failed: op=WRITE_BLOCK,
> remoteAddress=/172.28.208.200:53900, message=Block token with StorageIDs
> [DS-c0f24154-a39b-4941-93cd-5b8323067ba2] not valid for access with
> StorageIDs []
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-11583) Parent spans are not initialized to NullScope for every DFSPacket
[
https://issues.apache.org/jira/browse/HDFS-11583?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16051089#comment-16051089
]
Karan Mehta commented on HDFS-11583:
Thank you [~ajisakaa] and [~iwasakims]!
> Parent spans are not initialized to NullScope for every DFSPacket
> -
>
> Key: HDFS-11583
> URL: https://issues.apache.org/jira/browse/HDFS-11583
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: tracing
>Affects Versions: 2.7.1
>Reporter: Karan Mehta
>Assignee: Masatake Iwasaki
> Fix For: 2.7.4
>
> Attachments: HDFS-11583-branch-2.7.001.patch,
> HDFS-11583-branch-2.7.002.patch, HDFS-11583-branch-2.7.003.patch
>
>
> The issue was found while working with PHOENIX-3752.
> Each packet received by the {{run()}} method of {{DataStreamer}} class, uses
> the {{parents}} field of the {{DFSPacket}} to create a new {{dataStreamer}}
> span, which in turn creates a {{writeTo}} span as its child span. The parents
> field is initialized when the packet is added to the {{dataQueue}} and the
> value is initialized from the {{ThreadLocal}}. This is how HTrace handles
> spans.
> A {{TraceScope}} is created and initialized to {{NullScope}} before the loop
> which runs till the point when the stream is closed.
> Consider the following scenario, when the {{dataQueue}} contains multiple
> packets, only the first of which has a tracing enabled. The scope is
> initialized to the {{dataStreamer}} scope and a {{writeTo}} span is created
> as its child, which gets closed once the packet is sent out to a remote
> datanode. Before {{writeTo}} span is started, the {{dataStreamer}} scope is
> detached. So calling the close method on it doesn't do anything at the end of
> loop.
> The second iteration will be using the stale value of the {{scope}} variable
> with a DFSPacket on which tracing is not enabled. This results in generation
> of an orphan {{writeTo}} spans which are being delivered to the
> {{SpanReceiver}} as registered in the TraceFramework. This may result in
> unlimited number of spans being generated and sent out to receiver.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-10480) Add an admin command to list currently open files
[ https://issues.apache.org/jira/browse/HDFS-10480?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16051102#comment-16051102 ] Andrew Wang commented on HDFS-10480: LGTM thanks for working on this Manoj! Will commit this shortly. > Add an admin command to list currently open files > - > > Key: HDFS-10480 > URL: https://issues.apache.org/jira/browse/HDFS-10480 > Project: Hadoop HDFS > Issue Type: Improvement >Reporter: Kihwal Lee >Assignee: Manoj Govindassamy > Attachments: HDFS-10480.02.patch, HDFS-10480.03.patch, > HDFS-10480.04.patch, HDFS-10480.05.patch, HDFS-10480.06.patch, > HDFS-10480.07.patch, HDFS-10480-trunk-1.patch, HDFS-10480-trunk.patch > > > Currently there is no easy way to obtain the list of active leases or files > being written. It will be nice if we have an admin command to list open files > and their lease holders. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-10480) Add an admin command to list currently open files
[ https://issues.apache.org/jira/browse/HDFS-10480?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16051117#comment-16051117 ] Andrew Wang commented on HDFS-10480: I had to do a small imports conflict resolution to get this to apply to trunk, committed it there. However, I was unable to cherry-pick back to branch-2. Manoj, could you supply a branch-2 patch? Thanks! > Add an admin command to list currently open files > - > > Key: HDFS-10480 > URL: https://issues.apache.org/jira/browse/HDFS-10480 > Project: Hadoop HDFS > Issue Type: Improvement >Reporter: Kihwal Lee >Assignee: Manoj Govindassamy > Fix For: 3.0.0-alpha4 > > Attachments: HDFS-10480.02.patch, HDFS-10480.03.patch, > HDFS-10480.04.patch, HDFS-10480.05.patch, HDFS-10480.06.patch, > HDFS-10480.07.patch, HDFS-10480-trunk-1.patch, HDFS-10480-trunk.patch > > > Currently there is no easy way to obtain the list of active leases or files > being written. It will be nice if we have an admin command to list open files > and their lease holders. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-10480) Add an admin command to list currently open files
[ https://issues.apache.org/jira/browse/HDFS-10480?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Wang updated HDFS-10480: --- Fix Version/s: 3.0.0-alpha4 > Add an admin command to list currently open files > - > > Key: HDFS-10480 > URL: https://issues.apache.org/jira/browse/HDFS-10480 > Project: Hadoop HDFS > Issue Type: Improvement >Reporter: Kihwal Lee >Assignee: Manoj Govindassamy > Fix For: 3.0.0-alpha4 > > Attachments: HDFS-10480.02.patch, HDFS-10480.03.patch, > HDFS-10480.04.patch, HDFS-10480.05.patch, HDFS-10480.06.patch, > HDFS-10480.07.patch, HDFS-10480-trunk-1.patch, HDFS-10480-trunk.patch > > > Currently there is no easy way to obtain the list of active leases or files > being written. It will be nice if we have an admin command to list open files > and their lease holders. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-10480) Add an admin command to list currently open files
[ https://issues.apache.org/jira/browse/HDFS-10480?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16051119#comment-16051119 ] Andrew Wang commented on HDFS-10480: Also, if you want to start working on HDFS-11847 concurrently, happy to review that too. It'd be nice to have the functionality for the full admin workflow ready by the time this makes its way into a release. > Add an admin command to list currently open files > - > > Key: HDFS-10480 > URL: https://issues.apache.org/jira/browse/HDFS-10480 > Project: Hadoop HDFS > Issue Type: Improvement >Reporter: Kihwal Lee >Assignee: Manoj Govindassamy > Fix For: 3.0.0-alpha4 > > Attachments: HDFS-10480.02.patch, HDFS-10480.03.patch, > HDFS-10480.04.patch, HDFS-10480.05.patch, HDFS-10480.06.patch, > HDFS-10480.07.patch, HDFS-10480-trunk-1.patch, HDFS-10480-trunk.patch > > > Currently there is no easy way to obtain the list of active leases or files > being written. It will be nice if we have an admin command to list open files > and their lease holders. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-10480) Add an admin command to list currently open files
[ https://issues.apache.org/jira/browse/HDFS-10480?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16051155#comment-16051155 ] Hudson commented on HDFS-10480: --- SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #11872 (See [https://builds.apache.org/job/Hadoop-trunk-Commit/11872/]) HDFS-10480. Add an admin command to list currently open files. (wang: rev fb68980959f95f0d89e86f91909867724ad01791) * (edit) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/tools/TestDFSAdmin.java * (add) hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/protocol/OpenFileEntry.java * (edit) hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/client/HdfsAdmin.java * (edit) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/LeaseManager.java * (edit) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestHdfsAdmin.java * (edit) hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/DFSClient.java * (edit) hadoop-hdfs-project/hadoop-hdfs-client/src/main/proto/ClientNamenodeProtocol.proto * (edit) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/DFSTestUtil.java * (edit) hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/protocolPB/ClientNamenodeProtocolTranslatorPB.java * (edit) hadoop-hdfs-project/hadoop-hdfs/src/site/markdown/HDFSCommands.md * (edit) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NameNodeRpcServer.java * (edit) hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/protocol/ClientProtocol.java * (edit) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/protocolPB/ClientNamenodeProtocolServerSideTranslatorPB.java * (edit) hadoop-hdfs-project/hadoop-hdfs/src/main/resources/hdfs-default.xml * (add) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestListOpenFiles.java * (edit) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/tools/DFSAdmin.java * (edit) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestLeaseManager.java * (add) hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/protocol/OpenFilesIterator.java * (edit) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSConfigKeys.java * (edit) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java * (edit) hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/protocolPB/PBHelperClient.java * (edit) hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/DistributedFileSystem.java > Add an admin command to list currently open files > - > > Key: HDFS-10480 > URL: https://issues.apache.org/jira/browse/HDFS-10480 > Project: Hadoop HDFS > Issue Type: Improvement >Reporter: Kihwal Lee >Assignee: Manoj Govindassamy > Fix For: 3.0.0-alpha4 > > Attachments: HDFS-10480.02.patch, HDFS-10480.03.patch, > HDFS-10480.04.patch, HDFS-10480.05.patch, HDFS-10480.06.patch, > HDFS-10480.07.patch, HDFS-10480-trunk-1.patch, HDFS-10480-trunk.patch > > > Currently there is no easy way to obtain the list of active leases or files > being written. It will be nice if we have an admin command to list open files > and their lease holders. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-11956) Fix BlockToken compatibility with Hadoop 2.x clients
[
https://issues.apache.org/jira/browse/HDFS-11956?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ewan Higgs updated HDFS-11956:
--
Attachment: HDFS-11956.003.patch
Attaching patch with value for hdfs-default.xml and some checkstyle fixes.
> Fix BlockToken compatibility with Hadoop 2.x clients
>
>
> Key: HDFS-11956
> URL: https://issues.apache.org/jira/browse/HDFS-11956
> Project: Hadoop HDFS
> Issue Type: Bug
>Affects Versions: 3.0.0-alpha4
>Reporter: Andrew Wang
>Assignee: Ewan Higgs
>Priority: Blocker
> Attachments: HDFS-11956.001.patch, HDFS-11956.002.patch,
> HDFS-11956.003.patch
>
>
> Seems like HDFS-9807 broke backwards compatibility with Hadoop 2.x clients.
> When talking to a 3.0.0-alpha4 DN with security on:
> {noformat}
> 2017-06-06 23:27:22,568 WARN org.apache.hadoop.hdfs.server.datanode.DataNode:
> Block token verification failed: op=WRITE_BLOCK,
> remoteAddress=/172.28.208.200:53900, message=Block token with StorageIDs
> [DS-c0f24154-a39b-4941-93cd-5b8323067ba2] not valid for access with
> StorageIDs []
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Created] (HDFS-11985) Intermittent untit test failures on 2.7.4 branch.
Konstantin Shvachko created HDFS-11985: -- Summary: Intermittent untit test failures on 2.7.4 branch. Key: HDFS-11985 URL: https://issues.apache.org/jira/browse/HDFS-11985 Project: Hadoop HDFS Issue Type: Bug Components: test Affects Versions: 2.7.4 Reporter: Konstantin Shvachko Some unit tests are failing intermittently on Jenkins nightly builds for branch-2.7. Here is the list of test, which failed more than once within last week: * org.apache.hadoop.hdfs.server.datanode.TestDataNodeVolumeFailure.testUnderReplicationAfterVolFailure * org.apache.hadoop.hdfs.server.datanode.TestDataNodeHotSwapVolumes.testRemoveVolumeBeingWritten * org.apache.hadoop.hdfs.server.namenode.TestNamenodeCapacityReport.testXceiverCount * org.apache.hadoop.hdfs.server.namenode.snapshot.TestOpenFilesWithSnapshot -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-11985) Intermittent unit test failures on 2.7.4 branch.
[ https://issues.apache.org/jira/browse/HDFS-11985?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Konstantin Shvachko updated HDFS-11985: --- Summary: Intermittent unit test failures on 2.7.4 branch. (was: Intermittent untit test failures on 2.7.4 branch.) > Intermittent unit test failures on 2.7.4 branch. > > > Key: HDFS-11985 > URL: https://issues.apache.org/jira/browse/HDFS-11985 > Project: Hadoop HDFS > Issue Type: Bug > Components: test >Affects Versions: 2.7.4 >Reporter: Konstantin Shvachko > > Some unit tests are failing intermittently on Jenkins nightly builds for > branch-2.7. > Here is the list of test, which failed more than once within last week: > * > org.apache.hadoop.hdfs.server.datanode.TestDataNodeVolumeFailure.testUnderReplicationAfterVolFailure > * > org.apache.hadoop.hdfs.server.datanode.TestDataNodeHotSwapVolumes.testRemoveVolumeBeingWritten > * > org.apache.hadoop.hdfs.server.namenode.TestNamenodeCapacityReport.testXceiverCount > > * org.apache.hadoop.hdfs.server.namenode.snapshot.TestOpenFilesWithSnapshot -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-11978) Remove invalid '-usage' command of 'ec' and add missing commands 'addPolicies' 'listCodecs'
[ https://issues.apache.org/jira/browse/HDFS-11978?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16051248#comment-16051248 ] wenxin he commented on HDFS-11978: -- https://github.com/apache/hadoop/pull/235.patch > Remove invalid '-usage' command of 'ec' and add missing commands > 'addPolicies' 'listCodecs' > --- > > Key: HDFS-11978 > URL: https://issues.apache.org/jira/browse/HDFS-11978 > Project: Hadoop HDFS > Issue Type: Bug > Components: documentation >Affects Versions: 3.0.0-alpha4 >Reporter: wenxin he >Priority: Minor > > Remove invalid '-usage' command of 'ec' in HDFSErasureCoding.md. > Add missing commands 'addPolicies' 'listCodecs' in HDFSCommands.md. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-11956) Fix BlockToken compatibility with Hadoop 2.x clients
[
https://issues.apache.org/jira/browse/HDFS-11956?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16051255#comment-16051255
]
Hadoop QA commented on HDFS-11956:
--
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
19s{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 2 new or modified test
files. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 14m
39s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
57s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
54s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
9s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
57s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
47s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
56s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
44s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
44s{color} | {color:green} the patch passed {color} |
| {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange}
0m 43s{color} | {color:orange} hadoop-hdfs-project/hadoop-hdfs: The patch
generated 3 new + 711 unchanged - 0 fixed = 714 total (was 711) {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
55s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m
0s{color} | {color:red} The patch has 1 line(s) that end in whitespace. Use git
apply --whitespace=fix <>. Refer https://git-scm.com/docs/git-apply
{color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m
1s{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
49s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
38s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 93m 39s{color}
| {color:red} hadoop-hdfs in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
20s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}121m 54s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.hdfs.TestDFSStripedOutputStreamWithFailure080 |
| | hadoop.hdfs.TestDFSStripedOutputStreamWithFailure150 |
| | hadoop.hdfs.server.balancer.TestBalancerWithMultipleNameNodes |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Image:yetus/hadoop:14b5c93 |
| JIRA Issue | HDFS-11956 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12873187/HDFS-11956.003.patch |
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite
unit findbugs checkstyle xml |
| uname | Linux 60f266674f7c 3.13.0-108-generic #155-Ubuntu SMP Wed Jan 11
16:58:52 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh
|
| git revision | trunk / fb68980 |
| Default Java | 1.8.0_131 |
| findbugs | v3.1.0-RC1 |
| checkstyle |
https://builds.apache.org/job/PreCommit-HDFS-Build/19922/artifact/patchprocess/diff-checkstyle-hadoop-hdfs-project_hadoop-hdfs.txt
|
| whitespace |
https://builds.apache.org/job/PreCommit-HDFS-Build/19922/artifact/patchprocess/whitespace-eol.txt
|
| unit |
https://builds.apache.org/job/PreCommit-HDFS-Build/19922/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-HDFS-Build/19922/testReport/ |
| modules | C: hadoop-hdfs-project/hadoop-hdfs U:
hadoop-hdfs-project/hadoop-hdfs |
| Console output |
https://builds.apache.org/job/PreCommit-HDFS-Build/19922/console |
| Powered by | Apache Yetus 0.5.0-SNAPSHOT ht
[jira] [Commented] (HDFS-11943) Warn log frequently print to screen in doEncode function on AbstractNativeRawEncoder class
[
https://issues.apache.org/jira/browse/HDFS-11943?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16051272#comment-16051272
]
liaoyuxiangqin commented on HDFS-11943:
---
Hi [~Sammi], thanks for your review. I think the condition
{code:borderStyle=solid}
if (usingDirectBuffer)
{code}
in AbstractNativeRawEncoder.doEncode always false when hadfs use default heap
buffer , and the PerformanceAdvisory.LOG.debug will not be able to record info.
In addition, when use direct buffer the AbstractNativeRawEncoder.doEncode will
not be called, and we don't worry about frequent log, so i think the
condition not neccessary to add, thanks!
> Warn log frequently print to screen in doEncode function on
> AbstractNativeRawEncoder class
> ---
>
> Key: HDFS-11943
> URL: https://issues.apache.org/jira/browse/HDFS-11943
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: erasure-coding, native
>Affects Versions: 3.0.0-alpha4
> Environment: cluster: 3 nodes
> os:(Red Hat 2.6.33.20, Red Hat 3.10.0-514.6.1.el7.x86_64,
> Ubuntu4.4.0-31-generic)
> hadoop version: hadoop-3.0.0-alpha4
> erasure coding: XOR-2-1-64k and enabled Intel ISA-L
> hadoop fs -put file /
>Reporter: liaoyuxiangqin
>Assignee: liaoyuxiangqin
>Priority: Minor
> Attachments: HDFS-11943.002.patch, HDFS-11943.patch
>
> Original Estimate: 0.05h
> Remaining Estimate: 0.05h
>
> when i write file to hdfs on above environment, the hdfs client frequently
> print warn log of use direct ByteBuffer inputs/outputs in doEncode function
> to screen, detail information as follows:
> 2017-06-07 15:20:42,856 WARN rawcoder.AbstractNativeRawEncoder:
> convertToByteBufferState is invoked, not efficiently. Please use direct
> ByteBuffer inputs/outputs
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Comment Edited] (HDFS-11736) OIV tests should not write outside 'target' directory.
[
https://issues.apache.org/jira/browse/HDFS-11736?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16051278#comment-16051278
]
Yiqun Lin edited comment on HDFS-11736 at 6/16/17 1:44 AM:
---
Thanks [~jlowe] for reporting this. Attach the patch for branch-2/2.8.
was (Author: linyiqun):
Thanks [~jlowe] for reporting tthis. Attach the patch for branch-2/2.8.
> OIV tests should not write outside 'target' directory.
> --
>
> Key: HDFS-11736
> URL: https://issues.apache.org/jira/browse/HDFS-11736
> Project: Hadoop HDFS
> Issue Type: Bug
>Affects Versions: 2.6.0
>Reporter: Konstantin Shvachko
>Assignee: Yiqun Lin
> Labels: newbie++, test
> Fix For: 2.7.4, 3.0.0-alpha4
>
> Attachments: HDFS-11736.001.patch, HDFS-11736.002.patch,
> HDFS-11736.003.patch, HDFS-11736-branch-2.001.patch,
> HDFS-11736-branch-2.7.001.patch, HDFS-11736-branch-2.7.002.patch
>
>
> A few tests use {{Files.createTempDir()}} from Guava package, but do not set
> {{java.io.tmpdir}} system property. Thus the temp directory is created in
> unpredictable places and is not being cleaned up by {{mvn clean}}.
> This was probably introduced in {{TestOfflineImageViewer}} and then
> replicated in {{TestCheckpoint}}, {{TestStandbyCheckpoints}}.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-11736) OIV tests should not write outside 'target' directory.
[
https://issues.apache.org/jira/browse/HDFS-11736?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yiqun Lin updated HDFS-11736:
-
Attachment: HDFS-11736-branch-2.001.patch
Thanks [~jlowe] for reporting tthis. Attach the patch for branch-2/2.8.
> OIV tests should not write outside 'target' directory.
> --
>
> Key: HDFS-11736
> URL: https://issues.apache.org/jira/browse/HDFS-11736
> Project: Hadoop HDFS
> Issue Type: Bug
>Affects Versions: 2.6.0
>Reporter: Konstantin Shvachko
>Assignee: Yiqun Lin
> Labels: newbie++, test
> Fix For: 2.7.4, 3.0.0-alpha4
>
> Attachments: HDFS-11736.001.patch, HDFS-11736.002.patch,
> HDFS-11736.003.patch, HDFS-11736-branch-2.001.patch,
> HDFS-11736-branch-2.7.001.patch, HDFS-11736-branch-2.7.002.patch
>
>
> A few tests use {{Files.createTempDir()}} from Guava package, but do not set
> {{java.io.tmpdir}} system property. Thus the temp directory is created in
> unpredictable places and is not being cleaned up by {{mvn clean}}.
> This was probably introduced in {{TestOfflineImageViewer}} and then
> replicated in {{TestCheckpoint}}, {{TestStandbyCheckpoints}}.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-11646) Add -E option in 'ls' to list erasure coding policy of each file and directory if applicable
[ https://issues.apache.org/jira/browse/HDFS-11646?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16051276#comment-16051276 ] luhuichun commented on HDFS-11646: -- [~eddyxu] ok, will update today > Add -E option in 'ls' to list erasure coding policy of each file and > directory if applicable > > > Key: HDFS-11646 > URL: https://issues.apache.org/jira/browse/HDFS-11646 > Project: Hadoop HDFS > Issue Type: Improvement > Components: erasure-coding >Reporter: SammiChen >Assignee: luhuichun > Labels: hdfs-ec-3.0-nice-to-have > Attachments: HDFS-11646-001.patch, HDFS-11646-002.patch, > HDFS-11646-003.patch, HDFS-11646-004.patch > > > Add -E option in "ls" to show erasure coding policy of file and directory, > leverage the "number_of_replicas " column. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-11890) Handle NPE in BlockRecoveryWorker when DN is getting shoutdown.
[
https://issues.apache.org/jira/browse/HDFS-11890?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Surendra Singh Lilhore updated HDFS-11890:
--
Attachment: HDFS-11890-002.patch
> Handle NPE in BlockRecoveryWorker when DN is getting shoutdown.
> ---
>
> Key: HDFS-11890
> URL: https://issues.apache.org/jira/browse/HDFS-11890
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: datanode
>Affects Versions: 2.7.2
>Reporter: Surendra Singh Lilhore
>Assignee: Surendra Singh Lilhore
> Attachments: HDFS-11890-001.patch, HDFS-11890-002.patch
>
>
> {code}
> Exception in thread
> "org.apache.hadoop.hdfs.server.datanode.BlockRecoveryWorker$1@1c03e6ae"
> java.lang.NullPointerException
> at
> org.apache.hadoop.hdfs.server.datanode.BlockRecoveryWorker$RecoveryTaskContiguous.recover(BlockRecoveryWorker.java:131)
> at
> org.apache.hadoop.hdfs.server.datanode.BlockRecoveryWorker$1.run(BlockRecoveryWorker.java:596)
> at java.lang.Thread.run(Thread.java:748)
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-11890) Handle NPE in BlockRecoveryWorker when DN is getting shoutdown.
[
https://issues.apache.org/jira/browse/HDFS-11890?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16051399#comment-16051399
]
Surendra Singh Lilhore commented on HDFS-11890:
---
Thanks [~brahmareddy] for review..
Attached updated patch.. Please review...
> Handle NPE in BlockRecoveryWorker when DN is getting shoutdown.
> ---
>
> Key: HDFS-11890
> URL: https://issues.apache.org/jira/browse/HDFS-11890
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: datanode
>Affects Versions: 2.7.2
>Reporter: Surendra Singh Lilhore
>Assignee: Surendra Singh Lilhore
> Attachments: HDFS-11890-001.patch, HDFS-11890-002.patch
>
>
> {code}
> Exception in thread
> "org.apache.hadoop.hdfs.server.datanode.BlockRecoveryWorker$1@1c03e6ae"
> java.lang.NullPointerException
> at
> org.apache.hadoop.hdfs.server.datanode.BlockRecoveryWorker$RecoveryTaskContiguous.recover(BlockRecoveryWorker.java:131)
> at
> org.apache.hadoop.hdfs.server.datanode.BlockRecoveryWorker$1.run(BlockRecoveryWorker.java:596)
> at java.lang.Thread.run(Thread.java:748)
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-11978) Remove invalid '-usage' command of 'ec' and add missing commands 'addPolicies' 'listCodecs'
[ https://issues.apache.org/jira/browse/HDFS-11978?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16051398#comment-16051398 ] ASF GitHub Bot commented on HDFS-11978: --- Github user wenxinhe closed the pull request at: https://github.com/apache/hadoop/pull/235 > Remove invalid '-usage' command of 'ec' and add missing commands > 'addPolicies' 'listCodecs' > --- > > Key: HDFS-11978 > URL: https://issues.apache.org/jira/browse/HDFS-11978 > Project: Hadoop HDFS > Issue Type: Bug > Components: documentation >Affects Versions: 3.0.0-alpha4 >Reporter: wenxin he >Priority: Minor > > Remove invalid '-usage' command of 'ec' in HDFSErasureCoding.md. > Add missing commands 'addPolicies' 'listCodecs' in HDFSCommands.md. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-11978) Remove invalid '-usage' command of 'ec' and add missing commands 'addPolicies' 'listCodecs'
[ https://issues.apache.org/jira/browse/HDFS-11978?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16051400#comment-16051400 ] ASF GitHub Bot commented on HDFS-11978: --- GitHub user wenxinhe opened a pull request: https://github.com/apache/hadoop/pull/237 HDFS-11978. Remove invalid '-usage' command of 'ec' and add missing commands 'addPolicies' 'listCodecs' You can merge this pull request into a Git repository by running: $ git pull https://github.com/wenxinhe/hadoop HDFS-11978 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/hadoop/pull/237.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #237 commit 32b70d3b81a231c2373d85a3ff560d27c026061a Author: wenxin he Date: 2017-06-15T07:24:12Z HDFS-11978. Remove invalid '-usage' command of 'ec' and add missing commands 'addPolicies' 'listCodecs' > Remove invalid '-usage' command of 'ec' and add missing commands > 'addPolicies' 'listCodecs' > --- > > Key: HDFS-11978 > URL: https://issues.apache.org/jira/browse/HDFS-11978 > Project: Hadoop HDFS > Issue Type: Bug > Components: documentation >Affects Versions: 3.0.0-alpha4 >Reporter: wenxin he >Priority: Minor > > Remove invalid '-usage' command of 'ec' in HDFSErasureCoding.md. > Add missing commands 'addPolicies' 'listCodecs' in HDFSCommands.md. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-11978) Remove invalid '-usage' command of 'ec' and add missing commands 'addPolicies' 'listCodecs'
[ https://issues.apache.org/jira/browse/HDFS-11978?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] wenxin he updated HDFS-11978: - Status: Patch Available (was: Open) > Remove invalid '-usage' command of 'ec' and add missing commands > 'addPolicies' 'listCodecs' > --- > > Key: HDFS-11978 > URL: https://issues.apache.org/jira/browse/HDFS-11978 > Project: Hadoop HDFS > Issue Type: Bug > Components: documentation >Affects Versions: 3.0.0-alpha4 >Reporter: wenxin he >Priority: Minor > > Remove invalid '-usage' command of 'ec' in HDFSErasureCoding.md. > Add missing commands 'addPolicies' 'listCodecs' in HDFSCommands.md. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-11978) Remove invalid '-usage' command of 'ec' and add missing commands 'addPolicies' 'listCodecs'
[ https://issues.apache.org/jira/browse/HDFS-11978?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16051402#comment-16051402 ] wenxin he commented on HDFS-11978: -- https://github.com/apache/hadoop/pull/237.patch > Remove invalid '-usage' command of 'ec' and add missing commands > 'addPolicies' 'listCodecs' > --- > > Key: HDFS-11978 > URL: https://issues.apache.org/jira/browse/HDFS-11978 > Project: Hadoop HDFS > Issue Type: Bug > Components: documentation >Affects Versions: 3.0.0-alpha4 >Reporter: wenxin he >Priority: Minor > > Remove invalid '-usage' command of 'ec' in HDFSErasureCoding.md. > Add missing commands 'addPolicies' 'listCodecs' in HDFSCommands.md. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-11978) Remove invalid '-usage' command of 'ec' and add missing commands 'addPolicies' 'listCodecs'
[ https://issues.apache.org/jira/browse/HDFS-11978?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] wenxin he updated HDFS-11978: - Status: Open (was: Patch Available) > Remove invalid '-usage' command of 'ec' and add missing commands > 'addPolicies' 'listCodecs' > --- > > Key: HDFS-11978 > URL: https://issues.apache.org/jira/browse/HDFS-11978 > Project: Hadoop HDFS > Issue Type: Bug > Components: documentation >Affects Versions: 3.0.0-alpha4 >Reporter: wenxin he >Priority: Minor > > Remove invalid '-usage' command of 'ec' in HDFSErasureCoding.md. > Add missing commands 'addPolicies' 'listCodecs' in HDFSCommands.md. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-11736) OIV tests should not write outside 'target' directory.
[
https://issues.apache.org/jira/browse/HDFS-11736?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yiqun Lin updated HDFS-11736:
-
Attachment: HDFS-11736-branch-2.001.patch
> OIV tests should not write outside 'target' directory.
> --
>
> Key: HDFS-11736
> URL: https://issues.apache.org/jira/browse/HDFS-11736
> Project: Hadoop HDFS
> Issue Type: Bug
>Affects Versions: 2.6.0
>Reporter: Konstantin Shvachko
>Assignee: Yiqun Lin
> Labels: newbie++, test
> Fix For: 2.7.4, 3.0.0-alpha4
>
> Attachments: HDFS-11736.001.patch, HDFS-11736.002.patch,
> HDFS-11736.003.patch, HDFS-11736-branch-2.001.patch,
> HDFS-11736-branch-2.7.001.patch, HDFS-11736-branch-2.7.002.patch
>
>
> A few tests use {{Files.createTempDir()}} from Guava package, but do not set
> {{java.io.tmpdir}} system property. Thus the temp directory is created in
> unpredictable places and is not being cleaned up by {{mvn clean}}.
> This was probably introduced in {{TestOfflineImageViewer}} and then
> replicated in {{TestCheckpoint}}, {{TestStandbyCheckpoints}}.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-11736) OIV tests should not write outside 'target' directory.
[
https://issues.apache.org/jira/browse/HDFS-11736?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yiqun Lin updated HDFS-11736:
-
Attachment: (was: HDFS-11736-branch-2.001.patch)
> OIV tests should not write outside 'target' directory.
> --
>
> Key: HDFS-11736
> URL: https://issues.apache.org/jira/browse/HDFS-11736
> Project: Hadoop HDFS
> Issue Type: Bug
>Affects Versions: 2.6.0
>Reporter: Konstantin Shvachko
>Assignee: Yiqun Lin
> Labels: newbie++, test
> Fix For: 2.7.4, 3.0.0-alpha4
>
> Attachments: HDFS-11736.001.patch, HDFS-11736.002.patch,
> HDFS-11736.003.patch, HDFS-11736-branch-2.001.patch,
> HDFS-11736-branch-2.7.001.patch, HDFS-11736-branch-2.7.002.patch
>
>
> A few tests use {{Files.createTempDir()}} from Guava package, but do not set
> {{java.io.tmpdir}} system property. Thus the temp directory is created in
> unpredictable places and is not being cleaned up by {{mvn clean}}.
> This was probably introduced in {{TestOfflineImageViewer}} and then
> replicated in {{TestCheckpoint}}, {{TestStandbyCheckpoints}}.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-11976) Examine code base for cases that exception is thrown from finally block and fix it
[ https://issues.apache.org/jira/browse/HDFS-11976?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16051449#comment-16051449 ] Yongjun Zhang commented on HDFS-11976: -- That's great [~pavgustinov]! very nice tool and thank you so much! > Examine code base for cases that exception is thrown from finally block and > fix it > -- > > Key: HDFS-11976 > URL: https://issues.apache.org/jira/browse/HDFS-11976 > Project: Hadoop HDFS > Issue Type: Bug >Reporter: Yongjun Zhang > > If exception X is thrown in try block, and exception Y is thrown is finally > block, X will be swallowed. > In addition, finally block is used to ensure resources are released properly > in general. If we throw exception from there, some resources may be leaked. > So it's not recommended to throw exception in the finally block > I caught one today and reported HDFS-11794, creating this jira as a master > one to catch other similar cases. > Hope there is some static analyzer to find all. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-11736) OIV tests should not write outside 'target' directory.
[
https://issues.apache.org/jira/browse/HDFS-11736?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yiqun Lin updated HDFS-11736:
-
Status: Patch Available (was: Reopened)
> OIV tests should not write outside 'target' directory.
> --
>
> Key: HDFS-11736
> URL: https://issues.apache.org/jira/browse/HDFS-11736
> Project: Hadoop HDFS
> Issue Type: Bug
>Affects Versions: 2.6.0
>Reporter: Konstantin Shvachko
>Assignee: Yiqun Lin
> Labels: newbie++, test
> Fix For: 2.7.4, 3.0.0-alpha4
>
> Attachments: HDFS-11736.001.patch, HDFS-11736.002.patch,
> HDFS-11736.003.patch, HDFS-11736-branch-2.001.patch,
> HDFS-11736-branch-2.7.001.patch, HDFS-11736-branch-2.7.002.patch
>
>
> A few tests use {{Files.createTempDir()}} from Guava package, but do not set
> {{java.io.tmpdir}} system property. Thus the temp directory is created in
> unpredictable places and is not being cleaned up by {{mvn clean}}.
> This was probably introduced in {{TestOfflineImageViewer}} and then
> replicated in {{TestCheckpoint}}, {{TestStandbyCheckpoints}}.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
