[jira] [Commented] (HDFS-9625) set replication for empty file failed when set storage policy
[
https://issues.apache.org/jira/browse/HDFS-9625?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15110557#comment-15110557
]
Hadoop QA commented on HDFS-9625:
-
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 0s
{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s {color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 8m
11s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 43s
{color} | {color:green} trunk passed with JDK v1.8.0_66 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 44s
{color} | {color:green} trunk passed with JDK v1.7.0_91 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
18s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m 54s
{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
15s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m 4s
{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 9s
{color} | {color:green} trunk passed with JDK v1.8.0_66 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 55s
{color} | {color:green} trunk passed with JDK v1.7.0_91 {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
49s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 41s
{color} | {color:green} the patch passed with JDK v1.8.0_66 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 41s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 41s
{color} | {color:green} the patch passed with JDK v1.7.0_91 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 41s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
16s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m 53s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
12s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} Patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
14s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 6s
{color} | {color:green} the patch passed with JDK v1.8.0_66 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 52s
{color} | {color:green} the patch passed with JDK v1.7.0_91 {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 57m 23s {color}
| {color:red} hadoop-hdfs in the patch failed with JDK v1.8.0_66. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 55m 59s {color}
| {color:red} hadoop-hdfs in the patch failed with JDK v1.7.0_91. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
22s {color} | {color:green} Patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 141m 10s {color}
| {color:black} {color} |
\\
\\
|| Reason || Tests ||
| JDK v1.8.0_66 Failed junit tests |
hadoop.hdfs.server.datanode.TestBlockScanner |
| | hadoop.hdfs.TestDFSClientRetries |
| | hadoop.hdfs.web.TestWebHdfsTimeouts |
| | hadoop.hdfs.server.namenode.TestNNThroughputBenchmark |
| JDK v1.7.0_91 Failed junit tests |
hadoop.hdfs.server.namenode.TestNNThroughputBenchmark |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Image:yetus/hadoop:0ca8df7 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12783558/HDFS-9625.004.patch |
| JIRA Issue | HDFS-9625 |
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite
unit findbugs checkstyle |
| uname | Linux 63b17c828298
[jira] [Commented] (HDFS-9625) set replication for empty file failed when set storage policy
[ https://issues.apache.org/jira/browse/HDFS-9625?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15110400#comment-15110400 ] DENG FEI commented on HDFS-9625: [~Vinayakumar B] Thanks for you advice,your test code is suitable. > set replication for empty file failed when set storage policy > -- > > Key: HDFS-9625 > URL: https://issues.apache.org/jira/browse/HDFS-9625 > Project: Hadoop HDFS > Issue Type: Bug > Components: namenode >Affects Versions: 2.7.1 >Reporter: DENG FEI >Assignee: DENG FEI > Attachments: HDFS-9625.003.patch, patch.HDFS-9625.002, > patch_HDFS-9625.20160107 > > > When setReplication, the FSDirectory#updateCount need calculate the > related storageTypes quota,but will check the file consume the ds quota is > positive. > Actually,it's may set replication after create file,like > JobSplitWriter#createSplitFiles. > It's also can reproduce on command shell: > 1. hdfs storagepolicies -setStoragePolicy -path /tmp -policy HOT > 2. hdfs dfs -touchz /tmp/test > 3. hdfs dfs -setrep 5 /tmp/test -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-9625) set replication for empty file failed when set storage policy
[ https://issues.apache.org/jira/browse/HDFS-9625?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15110641#comment-15110641 ] Hudson commented on HDFS-9625: -- FAILURE: Integrated in Hadoop-trunk-Commit #9147 (See [https://builds.apache.org/job/Hadoop-trunk-Commit/9147/]) HDFS-9625. set replication for empty file failed when set storage policy (vinayakumarb: rev b7372b7166a13111b98794602ca4c166dfd78d29) * hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt * hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSDirectory.java * hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestSetrepIncreasing.java > set replication for empty file failed when set storage policy > -- > > Key: HDFS-9625 > URL: https://issues.apache.org/jira/browse/HDFS-9625 > Project: Hadoop HDFS > Issue Type: Bug > Components: namenode >Affects Versions: 2.7.1 >Reporter: DENG FEI >Assignee: DENG FEI > Fix For: 2.7.3 > > Attachments: HDFS-9625.003.patch, HDFS-9625.004.patch, > patch.HDFS-9625.002, patch_HDFS-9625.20160107 > > > When setReplication, the FSDirectory#updateCount need calculate the > related storageTypes quota,but will check the file consume the ds quota is > positive. > Actually,it's may set replication after create file,like > JobSplitWriter#createSplitFiles. > It's also can reproduce on command shell: > 1. hdfs storagepolicies -setStoragePolicy -path /tmp -policy HOT > 2. hdfs dfs -touchz /tmp/test > 3. hdfs dfs -setrep 5 /tmp/test -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-9625) set replication for empty file failed when set storage policy
[
https://issues.apache.org/jira/browse/HDFS-9625?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15106488#comment-15106488
]
Vinayakumar B commented on HDFS-9625:
-
For the test, I think simple and direct test code would be enough, instead of
trying to change multiple files and multiple places. As below.
{code}
@Test
public void testSetRepWithStoragePolicyOnEmptyFile() throws Exception {
Configuration conf = new HdfsConfiguration();
MiniDFSCluster cluster =
new MiniDFSCluster.Builder(conf).numDataNodes(1).build();
DistributedFileSystem dfs = cluster.getFileSystem();
try {
Path d = new Path("/tmp");
dfs.mkdirs(d);
dfs.setStoragePolicy(d, "HOT");
Path f = new Path(d, "foo");
dfs.createNewFile(f);
dfs.setReplication(f, (short) 4);
} finally {
dfs.close();
cluster.shutdown();
}
}
{code}
> set replication for empty file failed when set storage policy
> --
>
> Key: HDFS-9625
> URL: https://issues.apache.org/jira/browse/HDFS-9625
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: namenode
>Affects Versions: 2.7.1
>Reporter: DENG FEI
>Assignee: DENG FEI
> Attachments: HDFS-9625.003.patch, patch.HDFS-9625.002,
> patch_HDFS-9625.20160107
>
>
> When setReplication, the FSDirectory#updateCount need calculate the
> related storageTypes quota,but will check the file consume the ds quota is
> positive.
> Actually,it's may set replication after create file,like
> JobSplitWriter#createSplitFiles.
> It's also can reproduce on command shell:
> 1. hdfs storagepolicies -setStoragePolicy -path /tmp -policy HOT
> 2. hdfs dfs -touchz /tmp/test
> 3. hdfs dfs -setrep 5 /tmp/test
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HDFS-9625) set replication for empty file failed when set storage policy
[ https://issues.apache.org/jira/browse/HDFS-9625?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15091443#comment-15091443 ] DENG FEI commented on HDFS-9625: [~vinayrpet] Thanks for your good advice. I have uploaded the new patch,please check it once. > set replication for empty file failed when set storage policy > -- > > Key: HDFS-9625 > URL: https://issues.apache.org/jira/browse/HDFS-9625 > Project: Hadoop HDFS > Issue Type: Bug > Components: namenode >Affects Versions: 2.7.1 >Reporter: DENG FEI >Assignee: DENG FEI > Attachments: HDFS-9625.003.patch, patch.HDFS-9625.002, > patch_HDFS-9625.20160107 > > > When setReplication, the FSDirectory#updateCount need calculate the > related storageTypes quota,but will check the file consume the ds quota is > positive. > Actually,it's may set replication after create file,like > JobSplitWriter#createSplitFiles. > It's also can reproduce on command shell: > 1. hdfs storagepolicies -setStoragePolicy -path /tmp -policy HOT > 2. hdfs dfs -touchz /tmp/test > 3. hdfs dfs -setrep 5 /tmp/test -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-9625) set replication for empty file failed when set storage policy
[
https://issues.apache.org/jira/browse/HDFS-9625?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15091548#comment-15091548
]
Hadoop QA commented on HDFS-9625:
-
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 0s
{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s {color} | {color:green} The patch appears to include 2 new or modified test
files. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
47s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 46s
{color} | {color:green} trunk passed with JDK v1.8.0_66 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 41s
{color} | {color:green} trunk passed with JDK v1.7.0_91 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
16s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m 52s
{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
12s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
52s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 9s
{color} | {color:green} trunk passed with JDK v1.8.0_66 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 47s
{color} | {color:green} trunk passed with JDK v1.7.0_91 {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
46s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 44s
{color} | {color:green} the patch passed with JDK v1.8.0_66 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 44s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 39s
{color} | {color:green} the patch passed with JDK v1.7.0_91 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 39s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
16s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m 49s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
10s {color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m 0s
{color} | {color:red} The patch has 1 line(s) with tabs. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
56s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 6s
{color} | {color:green} the patch passed with JDK v1.8.0_66 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 47s
{color} | {color:green} the patch passed with JDK v1.7.0_91 {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 72m 51s {color}
| {color:red} hadoop-hdfs in the patch failed with JDK v1.8.0_66. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 56m 45s {color}
| {color:red} hadoop-hdfs in the patch failed with JDK v1.7.0_91. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
23s {color} | {color:green} Patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 156m 10s {color}
| {color:black} {color} |
\\
\\
|| Reason || Tests ||
| JDK v1.8.0_66 Failed junit tests | hadoop.hdfs.server.namenode.TestDeleteRace
|
| | hadoop.hdfs.server.namenode.TestDecommissioningStatus |
| | hadoop.hdfs.server.namenode.snapshot.TestRenameWithSnapshots |
| | hadoop.hdfs.TestDFSStripedOutputStreamWithFailure170 |
| | hadoop.hdfs.TestFileCorruption |
| | hadoop.hdfs.TestErasureCodingPolicyWithSnapshot |
| | hadoop.hdfs.server.namenode.TestNamenodeCapacityReport |
| | hadoop.hdfs.TestDFSClientRetries |
| | hadoop.hdfs.server.datanode.TestDataNodeHotSwapVolumes |
| | hadoop.hdfs.server.datanode.TestBlockScanner |
| JDK v1.7.0_91 Failed junit tests |
hadoop.hdfs.shortcircuit.TestShortCircuitCache |
| |
[jira] [Commented] (HDFS-9625) set replication for empty file failed when set storage policy
[ https://issues.apache.org/jira/browse/HDFS-9625?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15088580#comment-15088580 ] DENG FEI commented on HDFS-9625: ok,i will upload a trunk patch for it > set replication for empty file failed when set storage policy > -- > > Key: HDFS-9625 > URL: https://issues.apache.org/jira/browse/HDFS-9625 > Project: Hadoop HDFS > Issue Type: Bug > Components: namenode >Affects Versions: 2.7.1 >Reporter: DENG FEI >Assignee: DENG FEI > Attachments: patch_HDFS-9625.20160107 > > > When setReplication, the FSDirectory#updateCount need calculate the > related storageTypes quota,but will check the file consume the ds quota is > positive. > Actually,it's may set replication after create file,like > JobSplitWriter#createSplitFiles. > It's also can reproduce on command shell: > 1. hdfs storagepolicies -setStoragePolicy -path /tmp -policy HOT > 2. hdfs dfs -touchz /tmp/test > 3. hdfs dfs -setrep 5 /tmp/test -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-9625) set replication for empty file failed when set storage policy
[
https://issues.apache.org/jira/browse/HDFS-9625?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15088722#comment-15088722
]
Hadoop QA commented on HDFS-9625:
-
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 0s
{color} | {color:blue} Docker mode activated. {color} |
| {color:blue}0{color} | {color:blue} patch {color} | {color:blue} 0m 8s
{color} | {color:blue} The patch file was not named according to hadoop's
naming conventions. Please see https://wiki.apache.org/hadoop/HowToContribute
for instructions. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m 0s
{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
25s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 37s
{color} | {color:green} trunk passed with JDK v1.8.0_66 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 40s
{color} | {color:green} trunk passed with JDK v1.7.0_91 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
16s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m 53s
{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
13s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
57s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 7s
{color} | {color:green} trunk passed with JDK v1.8.0_66 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 46s
{color} | {color:green} trunk passed with JDK v1.7.0_91 {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
46s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 35s
{color} | {color:green} the patch passed with JDK v1.8.0_66 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 35s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 39s
{color} | {color:green} the patch passed with JDK v1.7.0_91 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 39s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
16s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m 49s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
11s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} Patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m 3s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 2s
{color} | {color:green} the patch passed with JDK v1.8.0_66 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 43s
{color} | {color:green} the patch passed with JDK v1.7.0_91 {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 53m 33s {color}
| {color:red} hadoop-hdfs in the patch failed with JDK v1.8.0_66. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 50m 54s {color}
| {color:red} hadoop-hdfs in the patch failed with JDK v1.7.0_91. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
21s {color} | {color:green} Patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 130m 21s {color}
| {color:black} {color} |
\\
\\
|| Reason || Tests ||
| JDK v1.8.0_66 Failed junit tests |
hadoop.hdfs.server.namenode.TestCacheDirectives |
| | hadoop.hdfs.server.namenode.TestNNThroughputBenchmark |
| | hadoop.hdfs.server.datanode.TestDataNodeRollingUpgrade |
| JDK v1.7.0_91 Failed junit tests | hadoop.TestGenericRefresh |
\\
\\
|| Subsystem
[jira] [Commented] (HDFS-9625) set replication for empty file failed when set storage policy
[
https://issues.apache.org/jira/browse/HDFS-9625?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15088802#comment-15088802
]
Vinayakumar B commented on HDFS-9625:
-
[~Deng FEI],
Thanks for the good find and patch.
Fix looks good.
1. It would be better to add a simple MiniDfsCluster test. mostly in
{{TestSetrepIncreasing.java}}
Nits:
bq. The patch file was not named according to hadoop's naming conventions.
Please see https://wiki.apache.org/hadoop/HowToContribute for instructions
You can name the patch in this format,*HDFS-9625.003.patch*
+1 once addressed.
> set replication for empty file failed when set storage policy
> --
>
> Key: HDFS-9625
> URL: https://issues.apache.org/jira/browse/HDFS-9625
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: namenode
>Affects Versions: 2.7.1
>Reporter: DENG FEI
>Assignee: DENG FEI
> Attachments: patch.HDFS-9625.002, patch_HDFS-9625.20160107
>
>
> When setReplication, the FSDirectory#updateCount need calculate the
> related storageTypes quota,but will check the file consume the ds quota is
> positive.
> Actually,it's may set replication after create file,like
> JobSplitWriter#createSplitFiles.
> It's also can reproduce on command shell:
> 1. hdfs storagepolicies -setStoragePolicy -path /tmp -policy HOT
> 2. hdfs dfs -touchz /tmp/test
> 3. hdfs dfs -setrep 5 /tmp/test
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HDFS-9625) set replication for empty file failed when set storage policy
[ https://issues.apache.org/jira/browse/HDFS-9625?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15087946#comment-15087946 ] Xiaoyu Yao commented on HDFS-9625: -- [~Deng FEI], thanks for reporting the issue and attach the fix. Can you rebase the patch against the latest trunk as it won't apply? > set replication for empty file failed when set storage policy > -- > > Key: HDFS-9625 > URL: https://issues.apache.org/jira/browse/HDFS-9625 > Project: Hadoop HDFS > Issue Type: Bug > Components: namenode >Affects Versions: 2.7.1 >Reporter: DENG FEI >Assignee: DENG FEI > Attachments: patch_HDFS-9625.20160107 > > > When setReplication, the FSDirectory#updateCount need calculate the > related storageTypes quota,but will check the file consume the ds quota is > positive. > Actually,it's may set replication after create file,like > JobSplitWriter#createSplitFiles. > It's also can reproduce on command shell: > 1. hdfs storagepolicies -setStoragePolicy -path /tmp -policy HOT > 2. hdfs dfs -touchz /tmp/test > 3. hdfs dfs -setrep 5 /tmp/test -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-9625) set replication for empty file failed when set storage policy
[
https://issues.apache.org/jira/browse/HDFS-9625?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15088160#comment-15088160
]
Hadoop QA commented on HDFS-9625:
-
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 0s
{color} | {color:blue} Docker mode activated. {color} |
| {color:blue}0{color} | {color:blue} patch {color} | {color:blue} 0m 8s
{color} | {color:blue} The patch file was not named according to hadoop's
naming conventions. Please see https://wiki.apache.org/hadoop/HowToContribute
for instructions. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m 0s
{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
37s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 40s
{color} | {color:green} trunk passed with JDK v1.8.0_66 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 41s
{color} | {color:green} trunk passed with JDK v1.7.0_91 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
15s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m 51s
{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
14s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
52s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 8s
{color} | {color:green} trunk passed with JDK v1.8.0_66 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 48s
{color} | {color:green} trunk passed with JDK v1.7.0_91 {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
46s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 38s
{color} | {color:green} the patch passed with JDK v1.8.0_66 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 38s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 38s
{color} | {color:green} the patch passed with JDK v1.7.0_91 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 38s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
15s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m 48s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
11s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} Patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
58s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 5s
{color} | {color:green} the patch passed with JDK v1.8.0_66 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 48s
{color} | {color:green} the patch passed with JDK v1.7.0_91 {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 52m 47s {color}
| {color:red} hadoop-hdfs in the patch failed with JDK v1.8.0_66. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 49m 45s {color}
| {color:red} hadoop-hdfs in the patch failed with JDK v1.7.0_91. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
22s {color} | {color:green} Patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 128m 40s {color}
| {color:black} {color} |
\\
\\
|| Reason || Tests ||
| JDK v1.8.0_66 Failed junit tests |
hadoop.hdfs.server.namenode.TestNNThroughputBenchmark |
| JDK v1.7.0_91 Failed junit tests | hadoop.hdfs.TestWriteReadStripedFile |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Image:yetus/hadoop:0ca8df7 |
| JIRA Patch URL |
