subject:"\[jira\] \[Commented\] \(HDFS\-13283\) Percentage based Reserved Space Calculation for DataNode"

[jira] [Commented] (HDFS-13283) Percentage based Reserved Space Calculation for DataNode

2018-04-30 Thread JIRA


[ 
https://issues.apache.org/jira/browse/HDFS-13283?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16459180#comment-16459180
 ] 

Íñigo Goiri commented on HDFS-13283:


Thanks [~lukmajercak] for the patches and [~chris.douglas] and [~virajith] for 
the reviews.
Committed all the way to 2.9: turnk, branch-3.1, branch-3.0, branch-2, and 
branch-2.9.

> Percentage based Reserved Space Calculation for DataNode
> 
>
> Key: HDFS-13283
> URL: https://issues.apache.org/jira/browse/HDFS-13283
> Project: Hadoop HDFS
>  Issue Type: New Feature
>  Components: datanode, hdfs
>Reporter: Lukas Majercak
>Assignee: Lukas Majercak
>Priority: Major
> Fix For: 2.10.0, 3.2.0, 3.1.1, 2.9.2, 3.0.4
>
> Attachments: HDFS-13283.000.patch, HDFS-13283.001.patch, 
> HDFS-13283.002.patch, HDFS-13283.003.patch, HDFS-13283.004.patch, 
> HDFS-13283.005.patch, HDFS-13283.006.patch, HDFS-13283.007.patch, 
> HDFS-13283_branch-2.000.patch, HDFS-13283_branch-3.0.000.patch
>
>
> Currently, the only way to configure reserved disk space for non-HDFS data on 
> a DataNode is a constant value via {{dfs.datanode.du.reserved}}. This can be 
> an issue in non-heterogeneous clusters where size of DNs can differ. The 
> proposed solution is to allow percentage based configuration (and their 
> combination):
>  # ABSOLUTE
>  ** based on absolute number of reserved space
>  # PERCENTAGE
>  ** based on percentage of total capacity in the storage
>  # CONSERVATIVE
>  ** calculates both of the above and takes the one that will yield more 
> reserved space
>  # AGGRESSIVE
>  ** calculates 1. 2. and takes the one that will yield less reserved space
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org

[jira] [Commented] (HDFS-13283) Percentage based Reserved Space Calculation for DataNode

2018-04-30 Thread genericqa (JIRA)


[ 
https://issues.apache.org/jira/browse/HDFS-13283?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16459163#comment-16459163
 ] 

genericqa commented on HDFS-13283:
--

| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue}  0m  
0s{color} | {color:blue} Docker mode activated. {color} |
| {color:red}-1{color} | {color:red} patch {color} | {color:red}  0m  6s{color} 
| {color:red} HDFS-13283 does not apply to trunk. Rebase required? Wrong 
Branch? See https://wiki.apache.org/hadoop/HowToContribute for help. {color} |
\\
\\
|| Subsystem || Report/Notes ||
| JIRA Issue | HDFS-13283 |
| JIRA Patch URL | 
https://issues.apache.org/jira/secure/attachment/12921316/HDFS-13283_branch-2.000.patch
 |
| Console output | 
https://builds.apache.org/job/PreCommit-HDFS-Build/24113/console |
| Powered by | Apache Yetus 0.8.0-SNAPSHOT   http://yetus.apache.org |


This message was automatically generated.



> Percentage based Reserved Space Calculation for DataNode
> 
>
> Key: HDFS-13283
> URL: https://issues.apache.org/jira/browse/HDFS-13283
> Project: Hadoop HDFS
>  Issue Type: New Feature
>  Components: datanode, hdfs
>Reporter: Lukas Majercak
>Assignee: Lukas Majercak
>Priority: Major
> Fix For: 3.2.0, 3.1.1
>
> Attachments: HDFS-13283.000.patch, HDFS-13283.001.patch, 
> HDFS-13283.002.patch, HDFS-13283.003.patch, HDFS-13283.004.patch, 
> HDFS-13283.005.patch, HDFS-13283.006.patch, HDFS-13283.007.patch, 
> HDFS-13283_branch-2.000.patch, HDFS-13283_branch-3.0.000.patch
>
>
> Currently, the only way to configure reserved disk space for non-HDFS data on 
> a DataNode is a constant value via {{dfs.datanode.du.reserved}}. This can be 
> an issue in non-heterogeneous clusters where size of DNs can differ. The 
> proposed solution is to allow percentage based configuration (and their 
> combination):
>  # ABSOLUTE
>  ** based on absolute number of reserved space
>  # PERCENTAGE
>  ** based on percentage of total capacity in the storage
>  # CONSERVATIVE
>  ** calculates both of the above and takes the one that will yield more 
> reserved space
>  # AGGRESSIVE
>  ** calculates 1. 2. and takes the one that will yield less reserved space
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org

[jira] [Commented] (HDFS-13283) Percentage based Reserved Space Calculation for DataNode

2018-04-30 Thread Lukas Majercak (JIRA)


[ 
https://issues.apache.org/jira/browse/HDFS-13283?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16459150#comment-16459150
 ] 

Lukas Majercak commented on HDFS-13283:
---

Added HDFS-13283_branch-2.000.patch

> Percentage based Reserved Space Calculation for DataNode
> 
>
> Key: HDFS-13283
> URL: https://issues.apache.org/jira/browse/HDFS-13283
> Project: Hadoop HDFS
>  Issue Type: New Feature
>  Components: datanode, hdfs
>Reporter: Lukas Majercak
>Assignee: Lukas Majercak
>Priority: Major
> Fix For: 3.2.0, 3.1.1
>
> Attachments: HDFS-13283.000.patch, HDFS-13283.001.patch, 
> HDFS-13283.002.patch, HDFS-13283.003.patch, HDFS-13283.004.patch, 
> HDFS-13283.005.patch, HDFS-13283.006.patch, HDFS-13283.007.patch, 
> HDFS-13283_branch-2.000.patch, HDFS-13283_branch-3.0.000.patch
>
>
> Currently, the only way to configure reserved disk space for non-HDFS data on 
> a DataNode is a constant value via {{dfs.datanode.du.reserved}}. This can be 
> an issue in non-heterogeneous clusters where size of DNs can differ. The 
> proposed solution is to allow percentage based configuration (and their 
> combination):
>  # ABSOLUTE
>  ** based on absolute number of reserved space
>  # PERCENTAGE
>  ** based on percentage of total capacity in the storage
>  # CONSERVATIVE
>  ** calculates both of the above and takes the one that will yield more 
> reserved space
>  # AGGRESSIVE
>  ** calculates 1. 2. and takes the one that will yield less reserved space
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org

[jira] [Commented] (HDFS-13283) Percentage based Reserved Space Calculation for DataNode

2018-04-30 Thread genericqa (JIRA)


[ 
https://issues.apache.org/jira/browse/HDFS-13283?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16459120#comment-16459120
 ] 

genericqa commented on HDFS-13283:
--

| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue}  0m  
0s{color} | {color:blue} Docker mode activated. {color} |
| {color:red}-1{color} | {color:red} patch {color} | {color:red}  0m  6s{color} 
| {color:red} HDFS-13283 does not apply to trunk. Rebase required? Wrong 
Branch? See https://wiki.apache.org/hadoop/HowToContribute for help. {color} |
\\
\\
|| Subsystem || Report/Notes ||
| JIRA Issue | HDFS-13283 |
| JIRA Patch URL | 
https://issues.apache.org/jira/secure/attachment/12921310/HDFS-13283_branch-3.0.000.patch
 |
| Console output | 
https://builds.apache.org/job/PreCommit-HDFS-Build/24112/console |
| Powered by | Apache Yetus 0.8.0-SNAPSHOT   http://yetus.apache.org |


This message was automatically generated.



> Percentage based Reserved Space Calculation for DataNode
> 
>
> Key: HDFS-13283
> URL: https://issues.apache.org/jira/browse/HDFS-13283
> Project: Hadoop HDFS
>  Issue Type: New Feature
>  Components: datanode, hdfs
>Reporter: Lukas Majercak
>Assignee: Lukas Majercak
>Priority: Major
> Fix For: 3.2.0, 3.1.1
>
> Attachments: HDFS-13283.000.patch, HDFS-13283.001.patch, 
> HDFS-13283.002.patch, HDFS-13283.003.patch, HDFS-13283.004.patch, 
> HDFS-13283.005.patch, HDFS-13283.006.patch, HDFS-13283.007.patch, 
> HDFS-13283_branch-3.0.000.patch
>
>
> Currently, the only way to configure reserved disk space for non-HDFS data on 
> a DataNode is a constant value via {{dfs.datanode.du.reserved}}. This can be 
> an issue in non-heterogeneous clusters where size of DNs can differ. The 
> proposed solution is to allow percentage based configuration (and their 
> combination):
>  # ABSOLUTE
>  ** based on absolute number of reserved space
>  # PERCENTAGE
>  ** based on percentage of total capacity in the storage
>  # CONSERVATIVE
>  ** calculates both of the above and takes the one that will yield more 
> reserved space
>  # AGGRESSIVE
>  ** calculates 1. 2. and takes the one that will yield less reserved space
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org

[jira] [Commented] (HDFS-13283) Percentage based Reserved Space Calculation for DataNode

2018-04-30 Thread Lukas Majercak (JIRA)


[ 
https://issues.apache.org/jira/browse/HDFS-13283?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16459100#comment-16459100
 ] 

Lukas Majercak commented on HDFS-13283:
---

Added HDFS-13283_branch-3.0.000.patch [~elgoiri].

> Percentage based Reserved Space Calculation for DataNode
> 
>
> Key: HDFS-13283
> URL: https://issues.apache.org/jira/browse/HDFS-13283
> Project: Hadoop HDFS
>  Issue Type: New Feature
>  Components: datanode, hdfs
>Reporter: Lukas Majercak
>Assignee: Lukas Majercak
>Priority: Major
> Fix For: 3.2.0, 3.1.1
>
> Attachments: HDFS-13283.000.patch, HDFS-13283.001.patch, 
> HDFS-13283.002.patch, HDFS-13283.003.patch, HDFS-13283.004.patch, 
> HDFS-13283.005.patch, HDFS-13283.006.patch, HDFS-13283.007.patch, 
> HDFS-13283_branch-3.0.000.patch
>
>
> Currently, the only way to configure reserved disk space for non-HDFS data on 
> a DataNode is a constant value via {{dfs.datanode.du.reserved}}. This can be 
> an issue in non-heterogeneous clusters where size of DNs can differ. The 
> proposed solution is to allow percentage based configuration (and their 
> combination):
>  # ABSOLUTE
>  ** based on absolute number of reserved space
>  # PERCENTAGE
>  ** based on percentage of total capacity in the storage
>  # CONSERVATIVE
>  ** calculates both of the above and takes the one that will yield more 
> reserved space
>  # AGGRESSIVE
>  ** calculates 1. 2. and takes the one that will yield less reserved space
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org

[jira] [Commented] (HDFS-13283) Percentage based Reserved Space Calculation for DataNode

2018-04-30 Thread Hudson (JIRA)


[ 
https://issues.apache.org/jira/browse/HDFS-13283?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16459049#comment-16459049
 ] 

Hudson commented on HDFS-13283:
---

SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #14093 (See 
[https://builds.apache.org/job/Hadoop-trunk-Commit/14093/])
HDFS-13283. Percentage based Reserved Space Calculation for DataNode. 
(inigoiri: rev fc074a359c44e84dd9612be2bd772763f943eb04)
* (edit) 
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSConfigKeys.java
* (add) 
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/ReservedSpaceCalculator.java
* (edit) 
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/FsVolumeImpl.java
* (edit) 
hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/TestFsVolumeList.java
* (add) 
hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/TestReservedSpaceCalculator.java
* (edit) 
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/ProvidedVolumeImpl.java
* (edit) 
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/FsVolumeImplBuilder.java
* (edit) hadoop-hdfs-project/hadoop-hdfs/src/main/resources/hdfs-default.xml


> Percentage based Reserved Space Calculation for DataNode
> 
>
> Key: HDFS-13283
> URL: https://issues.apache.org/jira/browse/HDFS-13283
> Project: Hadoop HDFS
>  Issue Type: New Feature
>  Components: datanode, hdfs
>Reporter: Lukas Majercak
>Assignee: Lukas Majercak
>Priority: Major
> Fix For: 3.2.0, 3.1.1
>
> Attachments: HDFS-13283.000.patch, HDFS-13283.001.patch, 
> HDFS-13283.002.patch, HDFS-13283.003.patch, HDFS-13283.004.patch, 
> HDFS-13283.005.patch, HDFS-13283.006.patch, HDFS-13283.007.patch
>
>
> Currently, the only way to configure reserved disk space for non-HDFS data on 
> a DataNode is a constant value via {{dfs.datanode.du.reserved}}. This can be 
> an issue in non-heterogeneous clusters where size of DNs can differ. The 
> proposed solution is to allow percentage based configuration (and their 
> combination):
>  # ABSOLUTE
>  ** based on absolute number of reserved space
>  # PERCENTAGE
>  ** based on percentage of total capacity in the storage
>  # CONSERVATIVE
>  ** calculates both of the above and takes the one that will yield more 
> reserved space
>  # AGGRESSIVE
>  ** calculates 1. 2. and takes the one that will yield less reserved space
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org

[jira] [Commented] (HDFS-13283) Percentage based Reserved Space Calculation for DataNode

2018-04-30 Thread JIRA


[ 
https://issues.apache.org/jira/browse/HDFS-13283?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16459019#comment-16459019
 ] 

Íñigo Goiri commented on HDFS-13283:


Thanks [~lukmajercak], I committed to trunk and branch-3.1.
Do you mind providing patches for branch-3.0 and branch-2?

> Percentage based Reserved Space Calculation for DataNode
> 
>
> Key: HDFS-13283
> URL: https://issues.apache.org/jira/browse/HDFS-13283
> Project: Hadoop HDFS
>  Issue Type: New Feature
>  Components: datanode, hdfs
>Reporter: Lukas Majercak
>Assignee: Lukas Majercak
>Priority: Major
> Fix For: 3.2.0, 3.1.1
>
> Attachments: HDFS-13283.000.patch, HDFS-13283.001.patch, 
> HDFS-13283.002.patch, HDFS-13283.003.patch, HDFS-13283.004.patch, 
> HDFS-13283.005.patch, HDFS-13283.006.patch, HDFS-13283.007.patch
>
>
> Currently, the only way to configure reserved disk space for non-HDFS data on 
> a DataNode is a constant value via {{dfs.datanode.du.reserved}}. This can be 
> an issue in non-heterogeneous clusters where size of DNs can differ. The 
> proposed solution is to allow percentage based configuration (and their 
> combination):
>  # ABSOLUTE
>  ** based on absolute number of reserved space
>  # PERCENTAGE
>  ** based on percentage of total capacity in the storage
>  # CONSERVATIVE
>  ** calculates both of the above and takes the one that will yield more 
> reserved space
>  # AGGRESSIVE
>  ** calculates 1. 2. and takes the one that will yield less reserved space
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org

[jira] [Commented] (HDFS-13283) Percentage based Reserved Space Calculation for DataNode

2018-04-24 Thread Lukas Majercak (JIRA)


[ 
https://issues.apache.org/jira/browse/HDFS-13283?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16450405#comment-16450405
 ] 

Lukas Majercak commented on HDFS-13283:
---

LGTM too, the tests don't seem to be related. [~chris.douglas]?

> Percentage based Reserved Space Calculation for DataNode
> 
>
> Key: HDFS-13283
> URL: https://issues.apache.org/jira/browse/HDFS-13283
> Project: Hadoop HDFS
>  Issue Type: New Feature
>  Components: datanode, hdfs
>Reporter: Lukas Majercak
>Assignee: Lukas Majercak
>Priority: Major
> Attachments: HDFS-13283.000.patch, HDFS-13283.001.patch, 
> HDFS-13283.002.patch, HDFS-13283.003.patch, HDFS-13283.004.patch, 
> HDFS-13283.005.patch, HDFS-13283.006.patch, HDFS-13283.007.patch
>
>
> Currently, the only way to configure reserved disk space for non-HDFS data on 
> a DataNode is a constant value via {{dfs.datanode.du.reserved}}. This can be 
> an issue in non-heterogeneous clusters where size of DNs can differ. The 
> proposed solution is to allow percentage based configuration (and their 
> combination):
>  # ABSOLUTE
>  ** based on absolute number of reserved space
>  # PERCENTAGE
>  ** based on percentage of total capacity in the storage
>  # CONSERVATIVE
>  ** calculates both of the above and takes the one that will yield more 
> reserved space
>  # AGGRESSIVE
>  ** calculates 1. 2. and takes the one that will yield less reserved space
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org

[jira] [Commented] (HDFS-13283) Percentage based Reserved Space Calculation for DataNode

2018-04-23 Thread JIRA


[ 
https://issues.apache.org/jira/browse/HDFS-13283?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16449158#comment-16449158
 ] 

Íñigo Goiri commented on HDFS-13283:


[^HDFS-13283.007.patch] LGTM.
The failed unit tests seem like the usual suspects.
+1

> Percentage based Reserved Space Calculation for DataNode
> 
>
> Key: HDFS-13283
> URL: https://issues.apache.org/jira/browse/HDFS-13283
> Project: Hadoop HDFS
>  Issue Type: New Feature
>  Components: datanode, hdfs
>Reporter: Lukas Majercak
>Assignee: Lukas Majercak
>Priority: Major
> Attachments: HDFS-13283.000.patch, HDFS-13283.001.patch, 
> HDFS-13283.002.patch, HDFS-13283.003.patch, HDFS-13283.004.patch, 
> HDFS-13283.005.patch, HDFS-13283.006.patch, HDFS-13283.007.patch
>
>
> Currently, the only way to configure reserved disk space for non-HDFS data on 
> a DataNode is a constant value via {{dfs.datanode.du.reserved}}. This can be 
> an issue in non-heterogeneous clusters where size of DNs can differ. The 
> proposed solution is to allow percentage based configuration (and their 
> combination):
>  # ABSOLUTE
>  ** based on absolute number of reserved space
>  # PERCENTAGE
>  ** based on percentage of total capacity in the storage
>  # CONSERVATIVE
>  ** calculates both of the above and takes the one that will yield more 
> reserved space
>  # AGGRESSIVE
>  ** calculates 1. 2. and takes the one that will yield less reserved space
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org

[jira] [Commented] (HDFS-13283) Percentage based Reserved Space Calculation for DataNode

2018-04-23 Thread genericqa (JIRA)


[ 
https://issues.apache.org/jira/browse/HDFS-13283?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16449052#comment-16449052
 ] 

genericqa commented on HDFS-13283:
--

| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue}  0m 
26s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green}  0m  
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green}  0m 
 0s{color} | {color:green} The patch appears to include 2 new or modified test 
files. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 22m 
55s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  0m 
51s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  0m 
50s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green}  0m 
55s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green} 
10m 58s{color} | {color:green} branch has no errors when building and testing 
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green}  1m 
39s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  0m 
45s{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  0m 
55s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  0m 
48s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green}  0m 
48s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  0m 
46s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green}  0m 
55s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green}  0m 
 0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green}  0m  
1s{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}  
9m 16s{color} | {color:green} patch has no errors when building and testing our 
client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green}  1m 
46s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  0m 
40s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red}120m 33s{color} 
| {color:red} hadoop-hdfs in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green}  0m 
21s{color} | {color:green} The patch does not generate ASF License warnings. 
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}175m  1s{color} | 
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.hdfs.server.datanode.TestDataNodeVolumeFailure |
|   | hadoop.hdfs.server.namenode.ha.TestRetryCacheWithHA |
|   | hadoop.hdfs.web.TestWebHdfsTimeouts |
|   | hadoop.hdfs.server.blockmanagement.TestUnderReplicatedBlocks |
|   | hadoop.hdfs.server.datanode.TestDataNodeVolumeFailureReporting |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hadoop:8620d2b |
| JIRA Issue | HDFS-13283 |
| JIRA Patch URL | 
https://issues.apache.org/jira/secure/attachment/12920336/HDFS-13283.007.patch |
| Optional Tests |  asflicense  compile  javac  javadoc  mvninstall  mvnsite  
unit  shadedclient  findbugs  checkstyle  xml  |
| uname | Linux a9df7d936b43 4.4.0-64-generic #85-Ubuntu SMP Mon Feb 20 
11:50:30 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/patchprocess/precommit/personality/provided.sh |
| git revision | trunk / f411de6 |
| maven | version: Apache Maven 3.3.9 |
| Default Java | 1.8.0_162 |
| findbugs | v3.1.0-RC1 |
| unit | 
https://builds.apache.org/job/PreCommit-HDFS-Build/24046/artifact/out/patch-unit-hadoop-hdfs-project_

[jira] [Commented] (HDFS-13283) Percentage based Reserved Space Calculation for DataNode

2018-04-23 Thread Lukas Majercak (JIRA)


[ 
https://issues.apache.org/jira/browse/HDFS-13283?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16448846#comment-16448846
 ] 

Lukas Majercak commented on HDFS-13283:
---

Added patch007, hope this one works.

> Percentage based Reserved Space Calculation for DataNode
> 
>
> Key: HDFS-13283
> URL: https://issues.apache.org/jira/browse/HDFS-13283
> Project: Hadoop HDFS
>  Issue Type: New Feature
>  Components: datanode, hdfs
>Reporter: Lukas Majercak
>Assignee: Lukas Majercak
>Priority: Major
> Attachments: HDFS-13283.000.patch, HDFS-13283.001.patch, 
> HDFS-13283.002.patch, HDFS-13283.003.patch, HDFS-13283.004.patch, 
> HDFS-13283.005.patch, HDFS-13283.006.patch, HDFS-13283.007.patch
>
>
> Currently, the only way to configure reserved disk space for non-HDFS data on 
> a DataNode is a constant value via {{dfs.datanode.du.reserved}}. This can be 
> an issue in non-heterogeneous clusters where size of DNs can differ. The 
> proposed solution is to allow percentage based configuration (and their 
> combination):
>  # ABSOLUTE
>  ** based on absolute number of reserved space
>  # PERCENTAGE
>  ** based on percentage of total capacity in the storage
>  # CONSERVATIVE
>  ** calculates both of the above and takes the one that will yield more 
> reserved space
>  # AGGRESSIVE
>  ** calculates 1. 2. and takes the one that will yield less reserved space
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org

[jira] [Commented] (HDFS-13283) Percentage based Reserved Space Calculation for DataNode

2018-04-23 Thread Chris Douglas (JIRA)


[ 
https://issues.apache.org/jira/browse/HDFS-13283?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16448701#comment-16448701
 ] 

Chris Douglas commented on HDFS-13283:
--

[~lukmajercak], could you regenerate the patch? +1 on the changes

> Percentage based Reserved Space Calculation for DataNode
> 
>
> Key: HDFS-13283
> URL: https://issues.apache.org/jira/browse/HDFS-13283
> Project: Hadoop HDFS
>  Issue Type: New Feature
>  Components: datanode, hdfs
>Reporter: Lukas Majercak
>Assignee: Lukas Majercak
>Priority: Major
> Attachments: HDFS-13283.000.patch, HDFS-13283.001.patch, 
> HDFS-13283.002.patch, HDFS-13283.003.patch, HDFS-13283.004.patch, 
> HDFS-13283.005.patch, HDFS-13283.006.patch
>
>
> Currently, the only way to configure reserved disk space for non-HDFS data on 
> a DataNode is a constant value via {{dfs.datanode.du.reserved}}. This can be 
> an issue in non-heterogeneous clusters where size of DNs can differ. The 
> proposed solution is to allow percentage based configuration (and their 
> combination):
>  # ABSOLUTE
>  ** based on absolute number of reserved space
>  # PERCENTAGE
>  ** based on percentage of total capacity in the storage
>  # CONSERVATIVE
>  ** calculates both of the above and takes the one that will yield more 
> reserved space
>  # AGGRESSIVE
>  ** calculates 1. 2. and takes the one that will yield less reserved space
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org

[jira] [Commented] (HDFS-13283) Percentage based Reserved Space Calculation for DataNode

2018-04-20 Thread genericqa (JIRA)


[ 
https://issues.apache.org/jira/browse/HDFS-13283?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16446548#comment-16446548
 ] 

genericqa commented on HDFS-13283:
--

| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue}  0m  
0s{color} | {color:blue} Docker mode activated. {color} |
| {color:red}-1{color} | {color:red} patch {color} | {color:red}  0m  7s{color} 
| {color:red} HDFS-13283 does not apply to trunk. Rebase required? Wrong 
Branch? See https://wiki.apache.org/hadoop/HowToContribute for help. {color} |
\\
\\
|| Subsystem || Report/Notes ||
| JIRA Issue | HDFS-13283 |
| JIRA Patch URL | 
https://issues.apache.org/jira/secure/attachment/12920120/HDFS-13283.006.patch |
| Console output | 
https://builds.apache.org/job/PreCommit-HDFS-Build/24024/console |
| Powered by | Apache Yetus 0.8.0-SNAPSHOT   http://yetus.apache.org |


This message was automatically generated.



> Percentage based Reserved Space Calculation for DataNode
> 
>
> Key: HDFS-13283
> URL: https://issues.apache.org/jira/browse/HDFS-13283
> Project: Hadoop HDFS
>  Issue Type: New Feature
>  Components: datanode, hdfs
>Reporter: Lukas Majercak
>Assignee: Lukas Majercak
>Priority: Major
> Attachments: HDFS-13283.000.patch, HDFS-13283.001.patch, 
> HDFS-13283.002.patch, HDFS-13283.003.patch, HDFS-13283.004.patch, 
> HDFS-13283.005.patch, HDFS-13283.006.patch
>
>
> Currently, the only way to configure reserved disk space for non-HDFS data on 
> a DataNode is a constant value via {{dfs.datanode.du.reserved}}. This can be 
> an issue in non-heterogeneous clusters where size of DNs can differ. The 
> proposed solution is to allow percentage based configuration (and their 
> combination):
>  # ABSOLUTE
>  ** based on absolute number of reserved space
>  # PERCENTAGE
>  ** based on percentage of total capacity in the storage
>  # CONSERVATIVE
>  ** calculates both of the above and takes the one that will yield more 
> reserved space
>  # AGGRESSIVE
>  ** calculates 1. 2. and takes the one that will yield less reserved space
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org

[jira] [Commented] (HDFS-13283) Percentage based Reserved Space Calculation for DataNode

2018-04-20 Thread Lukas Majercak (JIRA)


[ 
https://issues.apache.org/jira/browse/HDFS-13283?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16446545#comment-16446545
 ] 

Lukas Majercak commented on HDFS-13283:
---

Sorry for being quiet on this. Thanks [~virajith] for the comments and 
[~chris.douglas] for making the changes. I added patch006 to address the 
comments for the documentation.

> Percentage based Reserved Space Calculation for DataNode
> 
>
> Key: HDFS-13283
> URL: https://issues.apache.org/jira/browse/HDFS-13283
> Project: Hadoop HDFS
>  Issue Type: New Feature
>  Components: datanode, hdfs
>Reporter: Lukas Majercak
>Assignee: Lukas Majercak
>Priority: Major
> Attachments: HDFS-13283.000.patch, HDFS-13283.001.patch, 
> HDFS-13283.002.patch, HDFS-13283.003.patch, HDFS-13283.004.patch, 
> HDFS-13283.005.patch, HDFS-13283.006.patch
>
>
> Currently, the only way to configure reserved disk space for non-HDFS data on 
> a DataNode is a constant value via {{dfs.datanode.du.reserved}}. This can be 
> an issue in non-heterogeneous clusters where size of DNs can differ. The 
> proposed solution is to allow percentage based configuration (and their 
> combination):
>  # ABSOLUTE
>  ** based on absolute number of reserved space
>  # PERCENTAGE
>  ** based on percentage of total capacity in the storage
>  # CONSERVATIVE
>  ** calculates both of the above and takes the one that will yield more 
> reserved space
>  # AGGRESSIVE
>  ** calculates 1. 2. and takes the one that will yield less reserved space
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org

[jira] [Commented] (HDFS-13283) Percentage based Reserved Space Calculation for DataNode

2018-04-20 Thread genericqa (JIRA)


[ 
https://issues.apache.org/jira/browse/HDFS-13283?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16445441#comment-16445441
 ] 

genericqa commented on HDFS-13283:
--

| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue}  0m 
29s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green}  0m  
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green}  0m 
 0s{color} | {color:green} The patch appears to include 2 new or modified test 
files. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 26m 
27s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  1m  
3s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  0m 
56s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green}  1m  
9s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green} 
13m 17s{color} | {color:green} branch has no errors when building and testing 
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green}  2m  
3s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  0m 
53s{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  1m 
13s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  1m  
0s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green}  1m  
0s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  0m 
55s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green}  1m  
5s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green}  0m 
 0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green}  0m  
2s{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green} 
12m 19s{color} | {color:green} patch has no errors when building and testing 
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green}  2m 
14s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  0m 
55s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red}108m 14s{color} 
| {color:red} hadoop-hdfs in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green}  0m 
26s{color} | {color:green} The patch does not generate ASF License warnings. 
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}174m 30s{color} | 
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | 
hadoop.hdfs.TestDFSStripedOutputStreamWithFailureWithRandomECPolicy |
|   | hadoop.hdfs.server.datanode.TestDataNodeVolumeFailureToleration |
|   | hadoop.hdfs.server.namenode.ha.TestRetryCacheWithHA |
|   | hadoop.hdfs.server.blockmanagement.TestUnderReplicatedBlocks |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hadoop:8620d2b |
| JIRA Issue | HDFS-13283 |
| JIRA Patch URL | 
https://issues.apache.org/jira/secure/attachment/12919954/HDFS-13283.005.patch |
| Optional Tests |  asflicense  compile  javac  javadoc  mvninstall  mvnsite  
unit  shadedclient  findbugs  checkstyle  xml  |
| uname | Linux fc44fe564b7b 3.13.0-143-generic #192-Ubuntu SMP Tue Feb 27 
10:45:36 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/patchprocess/precommit/personality/provided.sh |
| git revision | trunk / da5bcf5 |
| maven | version: Apache Maven 3.3.9 |
| Default Java | 1.8.0_162 |
| findbugs | v3.1.0-RC1 |
| unit | 
https://builds.apache.org/job/PreCommit-HDFS-Build/24014/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt
 |
|  Tes

[jira] [Commented] (HDFS-13283) Percentage based Reserved Space Calculation for DataNode

2018-04-19 Thread Chris Douglas (JIRA)


[ 
https://issues.apache.org/jira/browse/HDFS-13283?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16445289#comment-16445289
 ] 

Chris Douglas commented on HDFS-13283:
--

Wrong syntax for the inner class in {{hdfs-default.xml}}. Fixed, trying again.

> Percentage based Reserved Space Calculation for DataNode
> 
>
> Key: HDFS-13283
> URL: https://issues.apache.org/jira/browse/HDFS-13283
> Project: Hadoop HDFS
>  Issue Type: New Feature
>  Components: datanode, hdfs
>Reporter: Lukas Majercak
>Assignee: Lukas Majercak
>Priority: Major
> Attachments: HDFS-13283.000.patch, HDFS-13283.001.patch, 
> HDFS-13283.002.patch, HDFS-13283.003.patch, HDFS-13283.004.patch, 
> HDFS-13283.005.patch
>
>
> Currently, the only way to configure reserved disk space for non-HDFS data on 
> a DataNode is a constant value via {{dfs.datanode.du.reserved}}. This can be 
> an issue in non-heterogeneous clusters where size of DNs can differ. The 
> proposed solution is to allow percentage based configuration (and their 
> combination):
>  # ABSOLUTE
>  ** based on absolute number of reserved space
>  # PERCENTAGE
>  ** based on percentage of total capacity in the storage
>  # CONSERVATIVE
>  ** calculates both of the above and takes the one that will yield more 
> reserved space
>  # AGGRESSIVE
>  ** calculates 1. 2. and takes the one that will yield less reserved space
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org

[jira] [Commented] (HDFS-13283) Percentage based Reserved Space Calculation for DataNode

2018-04-19 Thread genericqa (JIRA)


[ 
https://issues.apache.org/jira/browse/HDFS-13283?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16445257#comment-16445257
 ] 

genericqa commented on HDFS-13283:
--

| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue}  0m 
32s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green}  0m  
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green}  0m 
 0s{color} | {color:green} The patch appears to include 2 new or modified test 
files. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 25m 
15s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  0m 
55s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  0m 
55s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green}  1m  
0s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green} 
12m 13s{color} | {color:green} branch has no errors when building and testing 
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green}  1m 
51s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  0m 
45s{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  0m 
58s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  0m 
51s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green}  0m 
51s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  0m 
52s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green}  0m 
55s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green}  0m 
 0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green}  0m  
1s{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:red}-1{color} | {color:red} shadedclient {color} | {color:red} 11m 
29s{color} | {color:red} patch has errors when building and testing our client 
artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green}  1m 
58s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  0m 
43s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red}145m 40s{color} 
| {color:red} hadoop-hdfs in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green}  0m 
24s{color} | {color:green} The patch does not generate ASF License warnings. 
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}206m 57s{color} | 
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.hdfs.TestParallelShortCircuitRead |
|   | hadoop.fs.contract.hdfs.TestHDFSContractMkdir |
|   | hadoop.hdfs.server.datanode.TestDataNodeUUID |
|   | hadoop.hdfs.server.namenode.TestAllowFormat |
|   | hadoop.hdfs.server.namenode.TestCheckPointForSecurityTokens |
|   | hadoop.hdfs.TestBlockStoragePolicy |
|   | hadoop.hdfs.server.datanode.TestRefreshNamenodes |
|   | hadoop.hdfs.TestEncryptedTransfer |
|   | hadoop.hdfs.server.namenode.snapshot.TestSnapshotMetrics |
|   | hadoop.hdfs.TestDFSInotifyEventInputStream |
|   | hadoop.hdfs.TestSnapshotCommands |
|   | hadoop.cli.TestDeleteCLI |
|   | hadoop.hdfs.TestFileLengthOnClusterRestart |
|   | hadoop.hdfs.TestAppendSnapshotTruncate |
|   | hadoop.hdfs.server.namenode.snapshot.TestSnapshottableDirListing |
|   | hadoop.hdfs.TestRead |
|   | hadoop.fs.contract.hdfs.TestHDFSContractRootDirectory |
|   | hadoop.hdfs.server.namenode.snapshot.TestUpdatePipelineWithSnapshots |
|   | hadoop.hdfs.TestBlocksScheduledCounter |
|   | hadoop.hdfs.server.namenode.TestDiskspaceQuotaUpdate |
|   | hadoop.cli.TestHDFSCLI |
|   | hadoop.hdfs.TestDFSPermission |
|   | hadoop.hdfs.server.da

[jira] [Commented] (HDFS-13283) Percentage based Reserved Space Calculation for DataNode

2018-04-19 Thread Chris Douglas (JIRA)


[ 
https://issues.apache.org/jira/browse/HDFS-13283?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16445046#comment-16445046
 ] 

Chris Douglas commented on HDFS-13283:
--

Rebased patch. Minor changes:
* Renamed static, inner class {{ReservedSpaceCalculatorBuilder}} to {{Builder}}
* Changed invalid spec to throw {{IllegalArgumentException}} instead of failing 
back to absolute
* Changed {{ReservedSpaceCalculatorAggressive}} to extend 
{{ReservedSpaceCalculator}} rather than another, similar option
* Moved created of {{DF}} into the {{Builder}}

Missed [~virajith]'s comments on {{hdfs-default}}, +1 after those are addressed.

> Percentage based Reserved Space Calculation for DataNode
> 
>
> Key: HDFS-13283
> URL: https://issues.apache.org/jira/browse/HDFS-13283
> Project: Hadoop HDFS
>  Issue Type: New Feature
>  Components: datanode, hdfs
>Reporter: Lukas Majercak
>Assignee: Lukas Majercak
>Priority: Major
> Attachments: HDFS-13283.000.patch, HDFS-13283.001.patch, 
> HDFS-13283.002.patch, HDFS-13283.003.patch, HDFS-13283.004.patch
>
>
> Currently, the only way to configure reserved disk space for non-HDFS data on 
> a DataNode is a constant value via {{dfs.datanode.du.reserved}}. This can be 
> an issue in non-heterogeneous clusters where size of DNs can differ. The 
> proposed solution is to allow percentage based configuration (and their 
> combination):
>  # ABSOLUTE
>  ** based on absolute number of reserved space
>  # PERCENTAGE
>  ** based on percentage of total capacity in the storage
>  # CONSERVATIVE
>  ** calculates both of the above and takes the one that will yield more 
> reserved space
>  # AGGRESSIVE
>  ** calculates 1. 2. and takes the one that will yield less reserved space
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org

[jira] [Commented] (HDFS-13283) Percentage based Reserved Space Calculation for DataNode

2018-03-19 Thread Virajith Jalaparti (JIRA)


[ 
https://issues.apache.org/jira/browse/HDFS-13283?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16405664#comment-16405664
 ] 

Virajith Jalaparti commented on HDFS-13283:
---

[~lukmajercak] --  I think we need better documentation for this. In 
hdfs-default.xml
bq. The basic calculators are Absolute, Percentage, Conservative and Aggressive.
I think this should be ReservedSpaceCalculatorAbsolute, 
ReservedSpaceCalculatorPercentage, etc. Including a reference to this JIRA will 
be useful (would be good to change the description from using the Enums to the 
classes that can be configured).
Also, adding a note that the "dfs.datanode.du.reserved.pct" parameter will be 
used by these calculators will be helpful.

Once this is fixed, I am +1 on the patch.

> Percentage based Reserved Space Calculation for DataNode
> 
>
> Key: HDFS-13283
> URL: https://issues.apache.org/jira/browse/HDFS-13283
> Project: Hadoop HDFS
>  Issue Type: New Feature
>  Components: datanode, hdfs
>Reporter: Lukas Majercak
>Assignee: Lukas Majercak
>Priority: Major
> Attachments: HDFS-13283.000.patch, HDFS-13283.001.patch, 
> HDFS-13283.002.patch, HDFS-13283.003.patch
>
>
> Currently, the only way to configure reserved disk space for non-HDFS data on 
> a DataNode is a constant value via {{dfs.datanode.du.reserved}}. This can be 
> an issue in non-heterogeneous clusters where size of DNs can differ. The 
> proposed solution is to allow percentage based configuration (and their 
> combination):
>  # ABSOLUTE
>  ** based on absolute number of reserved space
>  # PERCENTAGE
>  ** based on percentage of total capacity in the storage
>  # CONSERVATIVE
>  ** calculates both of the above and takes the one that will yield more 
> reserved space
>  # AGGRESSIVE
>  ** calculates 1. 2. and takes the one that will yield less reserved space
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org

[jira] [Commented] (HDFS-13283) Percentage based Reserved Space Calculation for DataNode

2018-03-19 Thread JIRA


[ 
https://issues.apache.org/jira/browse/HDFS-13283?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16405619#comment-16405619
 ] 

Íñigo Goiri commented on HDFS-13283:


[^HDFS-13283.003.patch] looks good.
+1

> Percentage based Reserved Space Calculation for DataNode
> 
>
> Key: HDFS-13283
> URL: https://issues.apache.org/jira/browse/HDFS-13283
> Project: Hadoop HDFS
>  Issue Type: New Feature
>  Components: datanode, hdfs
>Reporter: Lukas Majercak
>Assignee: Lukas Majercak
>Priority: Major
> Attachments: HDFS-13283.000.patch, HDFS-13283.001.patch, 
> HDFS-13283.002.patch, HDFS-13283.003.patch
>
>
> Currently, the only way to configure reserved disk space for non-HDFS data on 
> a DataNode is a constant value via {{dfs.datanode.du.reserved}}. This can be 
> an issue in non-heterogeneous clusters where size of DNs can differ. The 
> proposed solution is to allow percentage based configuration (and their 
> combination):
>  # ABSOLUTE
>  ** based on absolute number of reserved space
>  # PERCENTAGE
>  ** based on percentage of total capacity in the storage
>  # CONSERVATIVE
>  ** calculates both of the above and takes the one that will yield more 
> reserved space
>  # AGGRESSIVE
>  ** calculates 1. 2. and takes the one that will yield less reserved space
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org

[jira] [Commented] (HDFS-13283) Percentage based Reserved Space Calculation for DataNode

2018-03-19 Thread genericqa (JIRA)


[ 
https://issues.apache.org/jira/browse/HDFS-13283?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16405607#comment-16405607
 ] 

genericqa commented on HDFS-13283:
--

| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue}  0m 
16s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green}  0m  
1s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green}  0m 
 0s{color} | {color:green} The patch appears to include 2 new or modified test 
files. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 15m 
36s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  0m 
50s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  0m 
47s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green}  0m 
55s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green} 
10m 23s{color} | {color:green} branch has no errors when building and testing 
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green}  1m 
52s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  0m 
52s{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  0m 
55s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  0m 
50s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green}  0m 
50s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  0m 
43s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green}  0m 
51s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green}  0m 
 0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green}  0m  
1s{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}  
9m 43s{color} | {color:green} patch has no errors when building and testing our 
client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green}  1m 
53s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  0m 
53s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 76m 48s{color} 
| {color:red} hadoop-hdfs in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green}  0m 
24s{color} | {color:green} The patch does not generate ASF License warnings. 
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}124m 15s{color} | 
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.hdfs.web.TestWebHdfsTimeouts |
|   | hadoop.hdfs.server.datanode.TestDataNodeVolumeFailureReporting |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hadoop:d4cc50f |
| JIRA Issue | HDFS-13283 |
| JIRA Patch URL | 
https://issues.apache.org/jira/secure/attachment/12915193/HDFS-13283.003.patch |
| Optional Tests |  asflicense  compile  javac  javadoc  mvninstall  mvnsite  
unit  shadedclient  findbugs  checkstyle  xml  |
| uname | Linux 04eb988deeaf 4.4.0-43-generic #63-Ubuntu SMP Wed Oct 12 
13:48:03 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/patchprocess/precommit/personality/provided.sh |
| git revision | trunk / a08921c |
| maven | version: Apache Maven 3.3.9 |
| Default Java | 1.8.0_151 |
| findbugs | v3.1.0-RC1 |
| unit | 
https://builds.apache.org/job/PreCommit-HDFS-Build/23545/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt
 |
|  Test Results | 
https://builds.apache.org/job/PreCommit-HDFS-Build/23545/testReport/ |
| Max. process+thread count | 4847 (vs. ulimit of 1) |
| modules | C: hadoop-h

[jira] [Commented] (HDFS-13283) Percentage based Reserved Space Calculation for DataNode

2018-03-19 Thread Lukas Majercak (JIRA)


[ 
https://issues.apache.org/jira/browse/HDFS-13283?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16405379#comment-16405379
 ] 

Lukas Majercak commented on HDFS-13283:
---

Patch003 to fix the last checkstyle error. Unit test failures should be 
unrelated.

> Percentage based Reserved Space Calculation for DataNode
> 
>
> Key: HDFS-13283
> URL: https://issues.apache.org/jira/browse/HDFS-13283
> Project: Hadoop HDFS
>  Issue Type: New Feature
>  Components: datanode, hdfs
>Reporter: Lukas Majercak
>Assignee: Lukas Majercak
>Priority: Major
> Attachments: HDFS-13283.000.patch, HDFS-13283.001.patch, 
> HDFS-13283.002.patch, HDFS-13283.003.patch
>
>
> Currently, the only way to configure reserved disk space for non-HDFS data on 
> a DataNode is a constant value via {{dfs.datanode.du.reserved}}. This can be 
> an issue in non-heterogeneous clusters where size of DNs can differ. The 
> proposed solution is to allow percentage based configuration (and their 
> combination):
>  # ABSOLUTE
>  ** based on absolute number of reserved space
>  # PERCENTAGE
>  ** based on percentage of total capacity in the storage
>  # CONSERVATIVE
>  ** calculates both of the above and takes the one that will yield more 
> reserved space
>  # AGGRESSIVE
>  ** calculates 1. 2. and takes the one that will yield less reserved space
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org

[jira] [Commented] (HDFS-13283) Percentage based Reserved Space Calculation for DataNode

2018-03-19 Thread genericqa (JIRA)


[ 
https://issues.apache.org/jira/browse/HDFS-13283?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16405345#comment-16405345
 ] 

genericqa commented on HDFS-13283:
--

| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue}  0m 
32s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green}  0m  
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green}  0m 
 0s{color} | {color:green} The patch appears to include 2 new or modified test 
files. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 15m 
44s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  0m 
53s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  0m 
44s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green}  0m 
59s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}  
9m 44s{color} | {color:green} branch has no errors when building and testing 
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green}  1m 
44s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  0m 
50s{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  0m 
57s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  0m 
51s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green}  0m 
51s{color} | {color:green} the patch passed {color} |
| {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange}  
0m 42s{color} | {color:orange} hadoop-hdfs-project/hadoop-hdfs: The patch 
generated 1 new + 449 unchanged - 0 fixed = 450 total (was 449) {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green}  0m 
56s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green}  0m 
 0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green}  0m  
2s{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}  
9m  1s{color} | {color:green} patch has no errors when building and testing our 
client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green}  1m 
52s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  0m 
47s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red}107m 27s{color} 
| {color:red} hadoop-hdfs in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green}  0m 
19s{color} | {color:green} The patch does not generate ASF License warnings. 
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}153m 36s{color} | 
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.hdfs.server.datanode.TestDataNodeVolumeFailure |
|   | hadoop.hdfs.qjournal.server.TestJournalNodeSync |
|   | hadoop.hdfs.web.TestWebHdfsTimeouts |
|   | hadoop.hdfs.server.datanode.TestDataNodeVolumeFailureReporting |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hadoop:d4cc50f |
| JIRA Issue | HDFS-13283 |
| JIRA Patch URL | 
https://issues.apache.org/jira/secure/attachment/12915167/HDFS-13283.002.patch |
| Optional Tests |  asflicense  compile  javac  javadoc  mvninstall  mvnsite  
unit  shadedclient  findbugs  checkstyle  xml  |
| uname | Linux 272b133dcc3b 4.4.0-64-generic #85-Ubuntu SMP Mon Feb 20 
11:50:30 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/patchprocess/precommit/personality/provided.sh |
| git revision | trunk / d67a5e2 |
| maven | version: Apache Maven 3.3.9 |
| Default Java | 1.8.0_151 |
| findbugs | v3.1.0-RC1 |
| checkstyle | 
https://builds.apache.org/job/PreCommit-HDFS-Build/23540/artifact/out/dif

[jira] [Commented] (HDFS-13283) Percentage based Reserved Space Calculation for DataNode

2018-03-19 Thread Lukas Majercak (JIRA)


[ 
https://issues.apache.org/jira/browse/HDFS-13283?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16405147#comment-16405147
 ] 

Lukas Majercak commented on HDFS-13283:
---

Added patch002 to fix checkstyle.

> Percentage based Reserved Space Calculation for DataNode
> 
>
> Key: HDFS-13283
> URL: https://issues.apache.org/jira/browse/HDFS-13283
> Project: Hadoop HDFS
>  Issue Type: New Feature
>  Components: datanode, hdfs
>Reporter: Lukas Majercak
>Assignee: Lukas Majercak
>Priority: Major
> Attachments: HDFS-13283.000.patch, HDFS-13283.001.patch, 
> HDFS-13283.002.patch
>
>
> Currently, the only way to configure reserved disk space for non-HDFS data on 
> a DataNode is a constant value via {{dfs.datanode.du.reserved}}. This can be 
> an issue in non-heterogeneous clusters where size of DNs can differ. The 
> proposed solution is to allow percentage based configuration (and their 
> combination):
>  # ABSOLUTE
>  ** based on absolute number of reserved space
>  # PERCENTAGE
>  ** based on percentage of total capacity in the storage
>  # CONSERVATIVE
>  ** calculates both of the above and takes the one that will yield more 
> reserved space
>  # AGGRESSIVE
>  ** calculates 1. 2. and takes the one that will yield less reserved space
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org

[jira] [Commented] (HDFS-13283) Percentage based Reserved Space Calculation for DataNode

2018-03-16 Thread Virajith Jalaparti (JIRA)


[ 
https://issues.apache.org/jira/browse/HDFS-13283?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16403085#comment-16403085
 ] 

Virajith Jalaparti commented on HDFS-13283:
---

Thanks for changing the enums to class [~lukmajercak]. I am ok with having the 
various implementations in a single file, given they are small.

> Percentage based Reserved Space Calculation for DataNode
> 
>
> Key: HDFS-13283
> URL: https://issues.apache.org/jira/browse/HDFS-13283
> Project: Hadoop HDFS
>  Issue Type: New Feature
>  Components: datanode, hdfs
>Reporter: Lukas Majercak
>Assignee: Lukas Majercak
>Priority: Major
> Attachments: HDFS-13283.000.patch, HDFS-13283.001.patch
>
>
> Currently, the only way to configure reserved disk space for non-HDFS data on 
> a DataNode is a constant value via {{dfs.datanode.du.reserved}}. This can be 
> an issue in non-heterogeneous clusters where size of DNs can differ. The 
> proposed solution is to allow percentage based configuration (and their 
> combination):
>  # ABSOLUTE
>  ** based on absolute number of reserved space
>  # PERCENTAGE
>  ** based on percentage of total capacity in the storage
>  # CONSERVATIVE
>  ** calculates both of the above and takes the one that will yield more 
> reserved space
>  # AGGRESSIVE
>  ** calculates 1. 2. and takes the one that will yield less reserved space
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org

[jira] [Commented] (HDFS-13283) Percentage based Reserved Space Calculation for DataNode

2018-03-16 Thread genericqa (JIRA)


[ 
https://issues.apache.org/jira/browse/HDFS-13283?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16402974#comment-16402974
 ] 

genericqa commented on HDFS-13283:
--

| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue}  0m 
19s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green}  0m  
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green}  0m 
 0s{color} | {color:green} The patch appears to include 2 new or modified test 
files. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 19m 
48s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  1m  
7s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  1m 
 4s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green}  1m 
11s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green} 
13m 15s{color} | {color:green} branch has no errors when building and testing 
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green}  2m 
11s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  1m  
0s{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  1m 
 8s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  1m  
6s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green}  1m  
6s{color} | {color:green} the patch passed {color} |
| {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange}  
0m 59s{color} | {color:orange} hadoop-hdfs-project/hadoop-hdfs: The patch 
generated 15 new + 449 unchanged - 0 fixed = 464 total (was 449) {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green}  1m  
9s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green}  0m 
 0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green}  0m  
2s{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green} 
11m 48s{color} | {color:green} patch has no errors when building and testing 
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green}  2m 
23s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  0m 
59s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 80m 39s{color} 
| {color:red} hadoop-hdfs in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green}  0m 
23s{color} | {color:green} The patch does not generate ASF License warnings. 
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}140m  2s{color} | 
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | 
hadoop.hdfs.server.datanode.TestDataNodeVolumeFailureReporting |
|   | hadoop.hdfs.TestSafeModeWithStripedFile |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hadoop:d4cc50f |
| JIRA Issue | HDFS-13283 |
| JIRA Patch URL | 
https://issues.apache.org/jira/secure/attachment/12914924/HDFS-13283.001.patch |
| Optional Tests |  asflicense  compile  javac  javadoc  mvninstall  mvnsite  
unit  shadedclient  findbugs  checkstyle  xml  |
| uname | Linux d4c2f387deda 3.13.0-139-generic #188-Ubuntu SMP Tue Jan 9 
14:43:09 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/patchprocess/precommit/personality/provided.sh |
| git revision | trunk / 08ff158 |
| maven | version: Apache Maven 3.3.9 |
| Default Java | 1.8.0_151 |
| findbugs | v3.1.0-RC1 |
| checkstyle | 
https://builds.apache.org/job/PreCommit-HDFS-Build/23523/artifact/out/diff-checkstyle-hadoop-hdfs-project_hadoop-hdfs.txt
 |
| unit | 
https://builds.apache.org/job/PreCommit-HDFS-Bui

[jira] [Commented] (HDFS-13283) Percentage based Reserved Space Calculation for DataNode

2018-03-16 Thread Lukas Majercak (JIRA)


[ 
https://issues.apache.org/jira/browse/HDFS-13283?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16402646#comment-16402646
 ] 

Lukas Majercak commented on HDFS-13283:
---

Considering the implementation is minimal, I'd rather keep them all in the same 
file. What do you think [~virajith]? I'll wait for yetus to come with 
checkstyle before making changes

> Percentage based Reserved Space Calculation for DataNode
> 
>
> Key: HDFS-13283
> URL: https://issues.apache.org/jira/browse/HDFS-13283
> Project: Hadoop HDFS
>  Issue Type: New Feature
>  Components: datanode, hdfs
>Reporter: Lukas Majercak
>Assignee: Lukas Majercak
>Priority: Major
> Attachments: HDFS-13283.000.patch, HDFS-13283.001.patch
>
>
> Currently, the only way to configure reserved disk space for non-HDFS data on 
> a DataNode is a constant value via {{dfs.datanode.du.reserved}}. This can be 
> an issue in non-heterogeneous clusters where size of DNs can differ. The 
> proposed solution is to allow percentage based configuration (and their 
> combination):
>  # ABSOLUTE
>  ** based on absolute number of reserved space
>  # PERCENTAGE
>  ** based on percentage of total capacity in the storage
>  # CONSERVATIVE
>  ** calculates both of the above and takes the one that will yield more 
> reserved space
>  # AGGRESSIVE
>  ** calculates 1. 2. and takes the one that will yield less reserved space
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org

[jira] [Commented] (HDFS-13283) Percentage based Reserved Space Calculation for DataNode

2018-03-16 Thread JIRA


[ 
https://issues.apache.org/jira/browse/HDFS-13283?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16402476#comment-16402476
 ] 

Íñigo Goiri commented on HDFS-13283:


I would probably move the implementations of ReservedSpaceCalculator to a 
separate package and have each one in a file.
I also like this approach better than the previous one.
Nit: put the extends line split all together.

> Percentage based Reserved Space Calculation for DataNode
> 
>
> Key: HDFS-13283
> URL: https://issues.apache.org/jira/browse/HDFS-13283
> Project: Hadoop HDFS
>  Issue Type: New Feature
>  Components: datanode, hdfs
>Reporter: Lukas Majercak
>Assignee: Lukas Majercak
>Priority: Major
> Attachments: HDFS-13283.000.patch, HDFS-13283.001.patch
>
>
> Currently, the only way to configure reserved disk space for non-HDFS data on 
> a DataNode is a constant value via {{dfs.datanode.du.reserved}}. This can be 
> an issue in non-heterogeneous clusters where size of DNs can differ. The 
> proposed solution is to allow percentage based configuration (and their 
> combination):
>  # ABSOLUTE
>  ** based on absolute number of reserved space
>  # PERCENTAGE
>  ** based on percentage of total capacity in the storage
>  # CONSERVATIVE
>  ** calculates both of the above and takes the one that will yield more 
> reserved space
>  # AGGRESSIVE
>  ** calculates 1. 2. and takes the one that will yield less reserved space
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org

[jira] [Commented] (HDFS-13283) Percentage based Reserved Space Calculation for DataNode

2018-03-16 Thread Lukas Majercak (JIRA)


[ 
https://issues.apache.org/jira/browse/HDFS-13283?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16402448#comment-16402448
 ] 

Lukas Majercak commented on HDFS-13283:
---

Thanks for the review and suggestion [~virajith]. I removed the enum and 
changed the config to specify the class of the calculator to be used.

> Percentage based Reserved Space Calculation for DataNode
> 
>
> Key: HDFS-13283
> URL: https://issues.apache.org/jira/browse/HDFS-13283
> Project: Hadoop HDFS
>  Issue Type: New Feature
>  Components: datanode, hdfs
>Reporter: Lukas Majercak
>Assignee: Lukas Majercak
>Priority: Major
> Attachments: HDFS-13283.000.patch, HDFS-13283.001.patch
>
>
> Currently, the only way to configure reserved disk space for non-HDFS data on 
> a DataNode is a constant value via {{dfs.datanode.du.reserved}}. This can be 
> an issue in non-heterogeneous clusters where size of DNs can differ. The 
> proposed solution is to allow percentage based configuration (and their 
> combination):
>  # ABSOLUTE
>  ** based on absolute number of reserved space
>  # PERCENTAGE
>  ** based on percentage of total capacity in the storage
>  # CONSERVATIVE
>  ** calculates both of the above and takes the one that will yield more 
> reserved space
>  # AGGRESSIVE
>  ** calculates 1. 2. and takes the one that will yield less reserved space
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org

[jira] [Commented] (HDFS-13283) Percentage based Reserved Space Calculation for DataNode

2018-03-14 Thread Virajith Jalaparti (JIRA)


[ 
https://issues.apache.org/jira/browse/HDFS-13283?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16399682#comment-16399682
 ] 

Virajith Jalaparti commented on HDFS-13283:
---

Thanks for reporting this [~lukmajercak], and posting a patch.

One question -- why use an enum to define this and not directly specify the 
class to use in the configuration? I think the latter makes this more general. 
While {{PERCENTAGE}} seems like a simple alternative of the absolute value used 
before, {{CONSERVATIVE}} and {{AGGRESSIVE}} seem like something specific 
someone would want to configure. If someone wants to define a custom policy for 
defining the reserved space at a later time, then they would have to add a new 
enum. I would rather they just implement a new sub-class of 
{{ReservedSpaceCalculator}} and directly configure. This will allow defining 
and configuring the custom policies easier, and reduce divergence from trunk.






> Percentage based Reserved Space Calculation for DataNode
> 
>
> Key: HDFS-13283
> URL: https://issues.apache.org/jira/browse/HDFS-13283
> Project: Hadoop HDFS
>  Issue Type: New Feature
>  Components: datanode, hdfs
>Reporter: Lukas Majercak
>Assignee: Lukas Majercak
>Priority: Major
> Attachments: HDFS-13283.000.patch
>
>
> Currently, the only way to configure reserved disk space for non-HDFS data on 
> a DataNode is a constant value via {{dfs.datanode.du.reserved}}. This can be 
> an issue in non-heterogeneous clusters where size of DNs can differ. The 
> proposed solution is to allow percentage based configuration (and their 
> combination):
>  # ABSOLUTE
>  ** based on absolute number of reserved space
>  # PERCENTAGE
>  ** based on percentage of total capacity in the storage
>  # CONSERVATIVE
>  ** calculates both of the above and takes the one that will yield more 
> reserved space
>  # AGGRESSIVE
>  ** calculates 1. 2. and takes the one that will yield less reserved space
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org

[jira] [Commented] (HDFS-13283) Percentage based Reserved Space Calculation for DataNode

2018-03-14 Thread genericqa (JIRA)


[ 
https://issues.apache.org/jira/browse/HDFS-13283?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16399650#comment-16399650
 ] 

genericqa commented on HDFS-13283:
--

| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue}  0m 
33s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green}  0m  
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green}  0m 
 0s{color} | {color:green} The patch appears to include 2 new or modified test 
files. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 17m 
37s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  1m  
3s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  0m 
57s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green}  1m  
5s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green} 
12m 24s{color} | {color:green} branch has no errors when building and testing 
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green}  2m  
4s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  0m 
57s{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  1m 
 8s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  0m 
56s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green}  0m 
56s{color} | {color:green} the patch passed {color} |
| {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange}  
0m 53s{color} | {color:orange} hadoop-hdfs-project/hadoop-hdfs: The patch 
generated 22 new + 449 unchanged - 0 fixed = 471 total (was 449) {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green}  1m  
0s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green}  0m 
 0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green}  0m  
1s{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green} 
11m  5s{color} | {color:green} patch has no errors when building and testing 
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green}  2m  
6s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  0m 
55s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red}103m 12s{color} 
| {color:red} hadoop-hdfs in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green}  0m 
24s{color} | {color:green} The patch does not generate ASF License warnings. 
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}157m 58s{color} | 
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | 
hadoop.hdfs.server.datanode.TestDataNodeVolumeFailureReporting |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hadoop:d4cc50f |
| JIRA Issue | HDFS-13283 |
| JIRA Patch URL | 
https://issues.apache.org/jira/secure/attachment/12914586/HDFS-13283.000.patch |
| Optional Tests |  asflicense  compile  javac  javadoc  mvninstall  mvnsite  
unit  shadedclient  findbugs  checkstyle  xml  |
| uname | Linux c7bf7b0c82b1 3.13.0-135-generic #184-Ubuntu SMP Wed Oct 18 
11:55:51 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/patchprocess/precommit/personality/provided.sh |
| git revision | trunk / 252c2b4 |
| maven | version: Apache Maven 3.3.9 |
| Default Java | 1.8.0_151 |
| findbugs | v3.1.0-RC1 |
| checkstyle | 
https://builds.apache.org/job/PreCommit-HDFS-Build/23489/artifact/out/diff-checkstyle-hadoop-hdfs-project_hadoop-hdfs.txt
 |
| unit | 
https://builds.apache.org/job/PreCommit-HDFS-Build/23489/artifact/out/patch-unit-hadoop-hdfs-pr

[jira] [Commented] (HDFS-13283) Percentage based Reserved Space Calculation for DataNode

2018-03-14 Thread JIRA


[ 
https://issues.apache.org/jira/browse/HDFS-13283?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16399495#comment-16399495
 ] 

Íñigo Goiri commented on HDFS-13283:


I think the documentation in hdfs-default.xml should be good enough.
If anybody has a suggestion whether we can add documentation for this.

> Percentage based Reserved Space Calculation for DataNode
> 
>
> Key: HDFS-13283
> URL: https://issues.apache.org/jira/browse/HDFS-13283
> Project: Hadoop HDFS
>  Issue Type: New Feature
>  Components: datanode, hdfs
>Reporter: Lukas Majercak
>Assignee: Lukas Majercak
>Priority: Major
> Attachments: HDFS-13283.000.patch
>
>
> Currently, the only way to configure reserved disk space for non-HDFS data on 
> a DataNode is a constant value via {{dfs.datanode.du.reserved}}. This can be 
> an issue in non-heterogeneous clusters where size of DNs can differ. The 
> proposed solution is to allow percentage based configuration (and their 
> combination):
>  # ABSOLUTE
>  ** based on absolute number of reserved space
>  # PERCENTAGE
>  ** based on percentage of total capacity in the storage
>  # CONSERVATIVE
>  ** calculates both of the above and takes the one that will yield more 
> reserved space
>  # AGGRESSIVE
>  ** calculates 1. 2. and takes the one that will yield less reserved space
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org

[jira] [Commented] (HDFS-13283) Percentage based Reserved Space Calculation for DataNode

2018-03-14 Thread Lukas Majercak (JIRA)


[ 
https://issues.apache.org/jira/browse/HDFS-13283?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16399490#comment-16399490
 ] 

Lukas Majercak commented on HDFS-13283:
---

We've been running this in production for 4months~ with the following 
configurations :

dfs.datanode.du.reserved.type=CONSERVATIVE
dfs.datanode.du.reserved.pct=10
dfs.datanode.du.reserved=107374182400


This allows us to leave at least 100GB reserved space and at the same time 
adjust the reserved space to the DNs disk size so that we don't use more than 
90%.

 

 

> Percentage based Reserved Space Calculation for DataNode
> 
>
> Key: HDFS-13283
> URL: https://issues.apache.org/jira/browse/HDFS-13283
> Project: Hadoop HDFS
>  Issue Type: New Feature
>  Components: datanode, hdfs
>Reporter: Lukas Majercak
>Assignee: Lukas Majercak
>Priority: Major
> Attachments: HDFS-13283.000.patch
>
>
> Currently, the only way to configure reserved disk space for non-HDFS data on 
> a DataNode is a constant value via {{dfs.datanode.du.reserved}}. This can be 
> an issue in non-heterogeneous clusters where size of DNs can differ. The 
> proposed solution is to allow percentage based configuration (and their 
> combination):
>  # ABSOLUTE
>  ** based on absolute number of reserved space
>  # PERCENTAGE
>  ** based on percentage of total capacity in the storage
>  # CONSERVATIVE
>  ** calculates both of the above and takes the one that will yield more 
> reserved space
>  # AGGRESSIVE
>  ** calculates 1. 2. and takes the one that will yield less reserved space
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org

[jira] [Commented] (HDFS-13283) Percentage based Reserved Space Calculation for DataNode

2018-03-14 Thread JIRA


[ 
https://issues.apache.org/jira/browse/HDFS-13283?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16399444#comment-16399444
 ] 

Íñigo Goiri commented on HDFS-13283:


Thanks [~lukmajercak] for  [^HDFS-13283.000.patch].
The unit tests are a pretty good guide for the use of this feature.
Can you share what's the experience with this?

> Percentage based Reserved Space Calculation for DataNode
> 
>
> Key: HDFS-13283
> URL: https://issues.apache.org/jira/browse/HDFS-13283
> Project: Hadoop HDFS
>  Issue Type: New Feature
>  Components: datanode, hdfs
>Reporter: Lukas Majercak
>Assignee: Lukas Majercak
>Priority: Major
> Attachments: HDFS-13283.000.patch
>
>
> Currently, the only way to configure reserved disk space for non-HDFS data on 
> a DataNode is a constant value via {{dfs.datanode.du.reserved}}. This can be 
> an issue in non-heterogeneous clusters where size of DNs can differ. The 
> proposed solution is to allow percentage based configuration (and their 
> combination):
>  # ABSOLUTE
>  ** based on absolute number of reserved space
>  # PERCENTAGE
>  ** based on percentage of total capacity in the storage
>  # CONSERVATIVE
>  ** calculates both of the above and takes the one that will yield more 
> reserved space
>  # AGGRESSIVE
>  ** calculates 1. 2. and takes the one that will yield less reserved space
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org

[jira] [Commented] (HDFS-13283) Percentage based Reserved Space Calculation for DataNode

2018-03-14 Thread Lukas Majercak (JIRA)


[ 
https://issues.apache.org/jira/browse/HDFS-13283?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16399403#comment-16399403
 ] 

Lukas Majercak commented on HDFS-13283:
---

Will send a patch momentarily.

> Percentage based Reserved Space Calculation for DataNode
> 
>
> Key: HDFS-13283
> URL: https://issues.apache.org/jira/browse/HDFS-13283
> Project: Hadoop HDFS
>  Issue Type: New Feature
>  Components: datanode, hdfs
>Reporter: Lukas Majercak
>Assignee: Lukas Majercak
>Priority: Major
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org

[jira] [Commented] (HDFS-13283) Percentage based Reserved Space Calculation for DataNode

[jira] [Commented] (HDFS-13283) Percentage based Reserved Space Calculation for DataNode

[jira] [Commented] (HDFS-13283) Percentage based Reserved Space Calculation for DataNode

[jira] [Commented] (HDFS-13283) Percentage based Reserved Space Calculation for DataNode

[jira] [Commented] (HDFS-13283) Percentage based Reserved Space Calculation for DataNode

[jira] [Commented] (HDFS-13283) Percentage based Reserved Space Calculation for DataNode

[jira] [Commented] (HDFS-13283) Percentage based Reserved Space Calculation for DataNode

[jira] [Commented] (HDFS-13283) Percentage based Reserved Space Calculation for DataNode

[jira] [Commented] (HDFS-13283) Percentage based Reserved Space Calculation for DataNode

[jira] [Commented] (HDFS-13283) Percentage based Reserved Space Calculation for DataNode

[jira] [Commented] (HDFS-13283) Percentage based Reserved Space Calculation for DataNode

[jira] [Commented] (HDFS-13283) Percentage based Reserved Space Calculation for DataNode

[jira] [Commented] (HDFS-13283) Percentage based Reserved Space Calculation for DataNode

[jira] [Commented] (HDFS-13283) Percentage based Reserved Space Calculation for DataNode

[jira] [Commented] (HDFS-13283) Percentage based Reserved Space Calculation for DataNode

[jira] [Commented] (HDFS-13283) Percentage based Reserved Space Calculation for DataNode

[jira] [Commented] (HDFS-13283) Percentage based Reserved Space Calculation for DataNode

[jira] [Commented] (HDFS-13283) Percentage based Reserved Space Calculation for DataNode

[jira] [Commented] (HDFS-13283) Percentage based Reserved Space Calculation for DataNode

[jira] [Commented] (HDFS-13283) Percentage based Reserved Space Calculation for DataNode

[jira] [Commented] (HDFS-13283) Percentage based Reserved Space Calculation for DataNode

[jira] [Commented] (HDFS-13283) Percentage based Reserved Space Calculation for DataNode

[jira] [Commented] (HDFS-13283) Percentage based Reserved Space Calculation for DataNode

[jira] [Commented] (HDFS-13283) Percentage based Reserved Space Calculation for DataNode

[jira] [Commented] (HDFS-13283) Percentage based Reserved Space Calculation for DataNode

[jira] [Commented] (HDFS-13283) Percentage based Reserved Space Calculation for DataNode

[jira] [Commented] (HDFS-13283) Percentage based Reserved Space Calculation for DataNode

[jira] [Commented] (HDFS-13283) Percentage based Reserved Space Calculation for DataNode

[jira] [Commented] (HDFS-13283) Percentage based Reserved Space Calculation for DataNode

[jira] [Commented] (HDFS-13283) Percentage based Reserved Space Calculation for DataNode

[jira] [Commented] (HDFS-13283) Percentage based Reserved Space Calculation for DataNode

[jira] [Commented] (HDFS-13283) Percentage based Reserved Space Calculation for DataNode

[jira] [Commented] (HDFS-13283) Percentage based Reserved Space Calculation for DataNode

[jira] [Commented] (HDFS-13283) Percentage based Reserved Space Calculation for DataNode

[jira] [Commented] (HDFS-13283) Percentage based Reserved Space Calculation for DataNode

35 matches

Site Navigation

Mail list logo

Footer information