[jira] [Commented] (HDFS-6099) HDFS file system limits not enforced on renames.
[
https://issues.apache.org/jira/browse/HDFS-6099?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13940540#comment-13940540
]
Hudson commented on HDFS-6099:
--
SUCCESS: Integrated in Hadoop-Mapreduce-trunk #1731 (See
[https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1731/])
HDFS-6099. HDFS file system limits not enforced on renames. Contributed by
Chris Nauroth. (cnauroth:
http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1579122)
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
*
/hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSDirectory.java
*
/hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NameNodeRpcServer.java
*
/hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestFsLimits.java
> HDFS file system limits not enforced on renames.
>
>
> Key: HDFS-6099
> URL: https://issues.apache.org/jira/browse/HDFS-6099
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: namenode
>Affects Versions: 2.3.0
>Reporter: Chris Nauroth
>Assignee: Chris Nauroth
> Fix For: 2.4.0
>
> Attachments: HDFS-6099.1.patch, HDFS-6099.2.patch, HDFS-6099.3.patch,
> HDFS-6099.4.patch, HDFS-6099.5.patch
>
>
> {{dfs.namenode.fs-limits.max-component-length}} and
> {{dfs.namenode.fs-limits.max-directory-items}} are not enforced on the
> destination path during rename operations. This means that it's still
> possible to create files that violate these limits.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HDFS-6099) HDFS file system limits not enforced on renames.
[
https://issues.apache.org/jira/browse/HDFS-6099?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13940499#comment-13940499
]
Hudson commented on HDFS-6099:
--
SUCCESS: Integrated in Hadoop-Hdfs-trunk #1706 (See
[https://builds.apache.org/job/Hadoop-Hdfs-trunk/1706/])
HDFS-6099. HDFS file system limits not enforced on renames. Contributed by
Chris Nauroth. (cnauroth:
http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1579122)
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
*
/hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSDirectory.java
*
/hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NameNodeRpcServer.java
*
/hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestFsLimits.java
> HDFS file system limits not enforced on renames.
>
>
> Key: HDFS-6099
> URL: https://issues.apache.org/jira/browse/HDFS-6099
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: namenode
>Affects Versions: 2.3.0
>Reporter: Chris Nauroth
>Assignee: Chris Nauroth
> Fix For: 2.4.0
>
> Attachments: HDFS-6099.1.patch, HDFS-6099.2.patch, HDFS-6099.3.patch,
> HDFS-6099.4.patch, HDFS-6099.5.patch
>
>
> {{dfs.namenode.fs-limits.max-component-length}} and
> {{dfs.namenode.fs-limits.max-directory-items}} are not enforced on the
> destination path during rename operations. This means that it's still
> possible to create files that violate these limits.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HDFS-6099) HDFS file system limits not enforced on renames.
[
https://issues.apache.org/jira/browse/HDFS-6099?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13940402#comment-13940402
]
Hudson commented on HDFS-6099:
--
FAILURE: Integrated in Hadoop-Yarn-trunk #514 (See
[https://builds.apache.org/job/Hadoop-Yarn-trunk/514/])
HDFS-6099. HDFS file system limits not enforced on renames. Contributed by
Chris Nauroth. (cnauroth:
http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1579122)
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
*
/hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSDirectory.java
*
/hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NameNodeRpcServer.java
*
/hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestFsLimits.java
> HDFS file system limits not enforced on renames.
>
>
> Key: HDFS-6099
> URL: https://issues.apache.org/jira/browse/HDFS-6099
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: namenode
>Affects Versions: 2.3.0
>Reporter: Chris Nauroth
>Assignee: Chris Nauroth
> Fix For: 2.4.0
>
> Attachments: HDFS-6099.1.patch, HDFS-6099.2.patch, HDFS-6099.3.patch,
> HDFS-6099.4.patch, HDFS-6099.5.patch
>
>
> {{dfs.namenode.fs-limits.max-component-length}} and
> {{dfs.namenode.fs-limits.max-directory-items}} are not enforced on the
> destination path during rename operations. This means that it's still
> possible to create files that violate these limits.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HDFS-6099) HDFS file system limits not enforced on renames.
[
https://issues.apache.org/jira/browse/HDFS-6099?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13940134#comment-13940134
]
Hudson commented on HDFS-6099:
--
SUCCESS: Integrated in Hadoop-trunk-Commit #5357 (See
[https://builds.apache.org/job/Hadoop-trunk-Commit/5357/])
HDFS-6099. HDFS file system limits not enforced on renames. Contributed by
Chris Nauroth. (cnauroth:
http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1579122)
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
*
/hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSDirectory.java
*
/hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NameNodeRpcServer.java
*
/hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestFsLimits.java
> HDFS file system limits not enforced on renames.
>
>
> Key: HDFS-6099
> URL: https://issues.apache.org/jira/browse/HDFS-6099
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: namenode
>Affects Versions: 2.3.0
>Reporter: Chris Nauroth
>Assignee: Chris Nauroth
> Fix For: 2.4.0
>
> Attachments: HDFS-6099.1.patch, HDFS-6099.2.patch, HDFS-6099.3.patch,
> HDFS-6099.4.patch, HDFS-6099.5.patch
>
>
> {{dfs.namenode.fs-limits.max-component-length}} and
> {{dfs.namenode.fs-limits.max-directory-items}} are not enforced on the
> destination path during rename operations. This means that it's still
> possible to create files that violate these limits.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HDFS-6099) HDFS file system limits not enforced on renames.
[
https://issues.apache.org/jira/browse/HDFS-6099?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13940092#comment-13940092
]
Hadoop QA commented on HDFS-6099:
-
{color:green}+1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12635429/HDFS-6099.5.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 1 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. There were no new javadoc warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in
hadoop-hdfs-project/hadoop-hdfs.
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results:
https://builds.apache.org/job/PreCommit-HDFS-Build/6431//testReport/
Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/6431//console
This message is automatically generated.
> HDFS file system limits not enforced on renames.
>
>
> Key: HDFS-6099
> URL: https://issues.apache.org/jira/browse/HDFS-6099
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: namenode
>Affects Versions: 2.3.0
>Reporter: Chris Nauroth
>Assignee: Chris Nauroth
> Fix For: 2.4.0
>
> Attachments: HDFS-6099.1.patch, HDFS-6099.2.patch, HDFS-6099.3.patch,
> HDFS-6099.4.patch, HDFS-6099.5.patch
>
>
> {{dfs.namenode.fs-limits.max-component-length}} and
> {{dfs.namenode.fs-limits.max-directory-items}} are not enforced on the
> destination path during rename operations. This means that it's still
> possible to create files that violate these limits.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HDFS-6099) HDFS file system limits not enforced on renames.
[
https://issues.apache.org/jira/browse/HDFS-6099?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13940043#comment-13940043
]
Hadoop QA commented on HDFS-6099:
-
{color:green}+1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12635410/HDFS-6099.4.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 1 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. There were no new javadoc warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in
hadoop-hdfs-project/hadoop-hdfs.
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results:
https://builds.apache.org/job/PreCommit-HDFS-Build/6430//testReport/
Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/6430//console
This message is automatically generated.
> HDFS file system limits not enforced on renames.
>
>
> Key: HDFS-6099
> URL: https://issues.apache.org/jira/browse/HDFS-6099
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: namenode
>Affects Versions: 2.3.0
>Reporter: Chris Nauroth
>Assignee: Chris Nauroth
> Fix For: 2.4.0
>
> Attachments: HDFS-6099.1.patch, HDFS-6099.2.patch, HDFS-6099.3.patch,
> HDFS-6099.4.patch, HDFS-6099.5.patch
>
>
> {{dfs.namenode.fs-limits.max-component-length}} and
> {{dfs.namenode.fs-limits.max-directory-items}} are not enforced on the
> destination path during rename operations. This means that it's still
> possible to create files that violate these limits.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HDFS-6099) HDFS file system limits not enforced on renames.
[
https://issues.apache.org/jira/browse/HDFS-6099?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13939964#comment-13939964
]
Hadoop QA commented on HDFS-6099:
-
{color:green}+1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12635396/HDFS-6099.3.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 1 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. There were no new javadoc warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in
hadoop-hdfs-project/hadoop-hdfs.
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results:
https://builds.apache.org/job/PreCommit-HDFS-Build/6427//testReport/
Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/6427//console
This message is automatically generated.
> HDFS file system limits not enforced on renames.
>
>
> Key: HDFS-6099
> URL: https://issues.apache.org/jira/browse/HDFS-6099
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: namenode
>Affects Versions: 2.3.0
>Reporter: Chris Nauroth
>Assignee: Chris Nauroth
> Fix For: 2.4.0
>
> Attachments: HDFS-6099.1.patch, HDFS-6099.2.patch, HDFS-6099.3.patch,
> HDFS-6099.4.patch, HDFS-6099.5.patch
>
>
> {{dfs.namenode.fs-limits.max-component-length}} and
> {{dfs.namenode.fs-limits.max-directory-items}} are not enforced on the
> destination path during rename operations. This means that it's still
> possible to create files that violate these limits.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HDFS-6099) HDFS file system limits not enforced on renames.
[
https://issues.apache.org/jira/browse/HDFS-6099?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13939907#comment-13939907
]
Tsz Wo Nicholas Sze commented on HDFS-6099:
---
One minor comment:
- if isRenameInSameDir, let's don't call verifyMaxDirItems in
verifyFsLimitsForRename. Then, we don't need to add the parameter
isRenameInSameDir to verifyMaxDirItems.
+1 other than that.
> HDFS file system limits not enforced on renames.
>
>
> Key: HDFS-6099
> URL: https://issues.apache.org/jira/browse/HDFS-6099
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: namenode
>Affects Versions: 2.3.0
>Reporter: Chris Nauroth
>Assignee: Chris Nauroth
> Fix For: 2.4.0
>
> Attachments: HDFS-6099.1.patch, HDFS-6099.2.patch, HDFS-6099.3.patch,
> HDFS-6099.4.patch
>
>
> {{dfs.namenode.fs-limits.max-component-length}} and
> {{dfs.namenode.fs-limits.max-directory-items}} are not enforced on the
> destination path during rename operations. This means that it's still
> possible to create files that violate these limits.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HDFS-6099) HDFS file system limits not enforced on renames.
[
https://issues.apache.org/jira/browse/HDFS-6099?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13939821#comment-13939821
]
Tsz Wo Nicholas Sze commented on HDFS-6099:
---
I still think that rename in the same dir should not throw exception since the
rename does not increase the usage. Just like that we should allow
setPermission if there is already some violation in the fs limits.
For quota, I believe it allows rename under the same directory even if the
quota is already violated.
> HDFS file system limits not enforced on renames.
>
>
> Key: HDFS-6099
> URL: https://issues.apache.org/jira/browse/HDFS-6099
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: namenode
>Affects Versions: 2.3.0
>Reporter: Chris Nauroth
>Assignee: Chris Nauroth
> Fix For: 2.4.0
>
> Attachments: HDFS-6099.1.patch, HDFS-6099.2.patch, HDFS-6099.3.patch
>
>
> {{dfs.namenode.fs-limits.max-component-length}} and
> {{dfs.namenode.fs-limits.max-directory-items}} are not enforced on the
> destination path during rename operations. This means that it's still
> possible to create files that violate these limits.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HDFS-6099) HDFS file system limits not enforced on renames.
[
https://issues.apache.org/jira/browse/HDFS-6099?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13939702#comment-13939702
]
Tsz Wo Nicholas Sze commented on HDFS-6099:
---
- For rename in the same dir, is it possible to get over limit? It particular,
I think the following is never true. Otherwise, it already exceeds the limit
before rename.
{code}
//FSDirectory.verifyMaxDirItems
(isRenameInSameDir && count > maxDirItems))
{code}
- It is better to verifyFsLimitsForRename before verifyQuotaForRename since it
is cheaper.
- The patch actually does not apply anymore. Need to update it.
> HDFS file system limits not enforced on renames.
>
>
> Key: HDFS-6099
> URL: https://issues.apache.org/jira/browse/HDFS-6099
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: namenode
>Affects Versions: 2.3.0
>Reporter: Chris Nauroth
>Assignee: Chris Nauroth
> Fix For: 2.4.0
>
> Attachments: HDFS-6099.1.patch, HDFS-6099.2.patch
>
>
> {{dfs.namenode.fs-limits.max-component-length}} and
> {{dfs.namenode.fs-limits.max-directory-items}} are not enforced on the
> destination path during rename operations. This means that it's still
> possible to create files that violate these limits.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HDFS-6099) HDFS file system limits not enforced on renames.
[
https://issues.apache.org/jira/browse/HDFS-6099?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13935144#comment-13935144
]
Chris Nauroth commented on HDFS-6099:
-
The failure in {{TestBalancer}} is unrelated.
> HDFS file system limits not enforced on renames.
>
>
> Key: HDFS-6099
> URL: https://issues.apache.org/jira/browse/HDFS-6099
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: namenode
>Affects Versions: 2.3.0
>Reporter: Chris Nauroth
>Assignee: Chris Nauroth
> Fix For: 2.4.0
>
> Attachments: HDFS-6099.1.patch, HDFS-6099.2.patch
>
>
> {{dfs.namenode.fs-limits.max-component-length}} and
> {{dfs.namenode.fs-limits.max-directory-items}} are not enforced on the
> destination path during rename operations. This means that it's still
> possible to create files that violate these limits.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HDFS-6099) HDFS file system limits not enforced on renames.
[
https://issues.apache.org/jira/browse/HDFS-6099?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13934497#comment-13934497
]
Hadoop QA commented on HDFS-6099:
-
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12634573/HDFS-6099.2.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 1 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. There were no new javadoc warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:red}-1 core tests{color}. The patch failed these unit tests in
hadoop-hdfs-project/hadoop-hdfs:
org.apache.hadoop.hdfs.server.balancer.TestBalancer
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results:
https://builds.apache.org/job/PreCommit-HDFS-Build/6400//testReport/
Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/6400//console
This message is automatically generated.
> HDFS file system limits not enforced on renames.
>
>
> Key: HDFS-6099
> URL: https://issues.apache.org/jira/browse/HDFS-6099
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: namenode
>Affects Versions: 2.3.0
>Reporter: Chris Nauroth
>Assignee: Chris Nauroth
> Fix For: 2.4.0
>
> Attachments: HDFS-6099.1.patch, HDFS-6099.2.patch
>
>
> {{dfs.namenode.fs-limits.max-component-length}} and
> {{dfs.namenode.fs-limits.max-directory-items}} are not enforced on the
> destination path during rename operations. This means that it's still
> possible to create files that violate these limits.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
