[jira] [Commented] (HDFS-14760) Log INFO mode if snapshot usage and actual usage differ
[
https://issues.apache.org/jira/browse/HDFS-14760?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16917432#comment-16917432
]
Hudson commented on HDFS-14760:
---
FAILURE: Integrated in Jenkins build Hadoop-trunk-Commit #17192 (See
[https://builds.apache.org/job/Hadoop-trunk-Commit/17192/])
HDFS-14760. Log INFO mode if snapshot usage and actual usage differ. (weichiu:
rev 6e37d65b03ff57cca25a46695ca3852da795d6f7)
* (edit)
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/DirectoryWithQuotaFeature.java
> Log INFO mode if snapshot usage and actual usage differ
> ---
>
> Key: HDFS-14760
> URL: https://issues.apache.org/jira/browse/HDFS-14760
> Project: Hadoop HDFS
> Issue Type: Improvement
>Reporter: CR Hota
>Assignee: CR Hota
>Priority: Major
> Fix For: 3.3.0
>
> Attachments: HDFS-14760.001.patch, HDFS-14760.002.patch
>
>
> In DirectoryWithQuotaFeature#checkStoragespace code logs in error mode
> without throwing any exceptions or action and pollutes logs. This should be
> in INFO mode.
> {code}
> private void checkStoragespace(final INodeDirectory dir, final long
> computed) {
> if (-1 != quota.getStorageSpace() && usage.getStorageSpace() != computed)
> {
> NameNode.LOG.error("BUG: Inconsistent storagespace for directory "
> + dir.getFullPathName() + ". Cached = " + usage.getStorageSpace()
> + " != Computed = " + computed);
> }
> }
> {code}
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-14760) Log INFO mode if snapshot usage and actual usage differ
[
https://issues.apache.org/jira/browse/HDFS-14760?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16917229#comment-16917229
]
Wei-Chiu Chuang commented on HDFS-14760:
+1 on the 002 patch. Will commit soon.
> Log INFO mode if snapshot usage and actual usage differ
> ---
>
> Key: HDFS-14760
> URL: https://issues.apache.org/jira/browse/HDFS-14760
> Project: Hadoop HDFS
> Issue Type: Improvement
>Reporter: CR Hota
>Assignee: CR Hota
>Priority: Major
> Attachments: HDFS-14760.001.patch, HDFS-14760.002.patch
>
>
> In DirectoryWithQuotaFeature#checkStoragespace code logs in error mode
> without throwing any exceptions or action and pollutes logs. This should be
> in INFO mode.
> {code}
> private void checkStoragespace(final INodeDirectory dir, final long
> computed) {
> if (-1 != quota.getStorageSpace() && usage.getStorageSpace() != computed)
> {
> NameNode.LOG.error("BUG: Inconsistent storagespace for directory "
> + dir.getFullPathName() + ". Cached = " + usage.getStorageSpace()
> + " != Computed = " + computed);
> }
> }
> {code}
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-14760) Log INFO mode if snapshot usage and actual usage differ
[
https://issues.apache.org/jira/browse/HDFS-14760?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16916836#comment-16916836
]
CR Hota commented on HDFS-14760:
[~jojochuang] Thanks for the review. Could you help commit 002.patch?
> Log INFO mode if snapshot usage and actual usage differ
> ---
>
> Key: HDFS-14760
> URL: https://issues.apache.org/jira/browse/HDFS-14760
> Project: Hadoop HDFS
> Issue Type: Improvement
>Reporter: CR Hota
>Assignee: CR Hota
>Priority: Major
> Attachments: HDFS-14760.001.patch, HDFS-14760.002.patch
>
>
> In DirectoryWithQuotaFeature#checkStoragespace code logs in error mode
> without throwing any exceptions or action and pollutes logs. This should be
> in INFO mode.
> {code}
> private void checkStoragespace(final INodeDirectory dir, final long
> computed) {
> if (-1 != quota.getStorageSpace() && usage.getStorageSpace() != computed)
> {
> NameNode.LOG.error("BUG: Inconsistent storagespace for directory "
> + dir.getFullPathName() + ". Cached = " + usage.getStorageSpace()
> + " != Computed = " + computed);
> }
> }
> {code}
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-14760) Log INFO mode if snapshot usage and actual usage differ
[
https://issues.apache.org/jira/browse/HDFS-14760?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16913826#comment-16913826
]
CR Hota commented on HDFS-14760:
[~jojochuang] Thanks!
Seems 002.patch is safe to commit.
> Log INFO mode if snapshot usage and actual usage differ
> ---
>
> Key: HDFS-14760
> URL: https://issues.apache.org/jira/browse/HDFS-14760
> Project: Hadoop HDFS
> Issue Type: Improvement
>Reporter: CR Hota
>Assignee: CR Hota
>Priority: Major
> Attachments: HDFS-14760.001.patch, HDFS-14760.002.patch
>
>
> In DirectoryWithQuotaFeature#checkStoragespace code logs in error mode
> without throwing any exceptions or action and pollutes logs. This should be
> in INFO mode.
> {code}
> private void checkStoragespace(final INodeDirectory dir, final long
> computed) {
> if (-1 != quota.getStorageSpace() && usage.getStorageSpace() != computed)
> {
> NameNode.LOG.error("BUG: Inconsistent storagespace for directory "
> + dir.getFullPathName() + ". Cached = " + usage.getStorageSpace()
> + " != Computed = " + computed);
> }
> }
> {code}
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-14760) Log INFO mode if snapshot usage and actual usage differ
[
https://issues.apache.org/jira/browse/HDFS-14760?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16913823#comment-16913823
]
Wei-Chiu Chuang commented on HDFS-14760:
+1
> Log INFO mode if snapshot usage and actual usage differ
> ---
>
> Key: HDFS-14760
> URL: https://issues.apache.org/jira/browse/HDFS-14760
> Project: Hadoop HDFS
> Issue Type: Improvement
>Reporter: CR Hota
>Assignee: CR Hota
>Priority: Major
> Attachments: HDFS-14760.001.patch, HDFS-14760.002.patch
>
>
> In DirectoryWithQuotaFeature#checkStoragespace code logs in error mode
> without throwing any exceptions or action and pollutes logs. This should be
> in INFO mode.
> {code}
> private void checkStoragespace(final INodeDirectory dir, final long
> computed) {
> if (-1 != quota.getStorageSpace() && usage.getStorageSpace() != computed)
> {
> NameNode.LOG.error("BUG: Inconsistent storagespace for directory "
> + dir.getFullPathName() + ". Cached = " + usage.getStorageSpace()
> + " != Computed = " + computed);
> }
> }
> {code}
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-14760) Log INFO mode if snapshot usage and actual usage differ
[
https://issues.apache.org/jira/browse/HDFS-14760?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16913814#comment-16913814
]
Hadoop QA commented on HDFS-14760:
--
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
52s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m
0s{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 20m
56s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
59s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
41s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
6s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
13m 23s{color} | {color:green} branch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
8s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
52s{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
2s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
54s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
54s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
37s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
6s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
12m 51s{color} | {color:green} patch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
25s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
57s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red}128m 26s{color}
| {color:red} hadoop-hdfs in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
46s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}190m 8s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests |
hadoop.hdfs.server.balancer.TestBalancerWithMultipleNameNodes |
| | hadoop.hdfs.server.blockmanagement.TestSlowDiskTracker |
| | hadoop.hdfs.server.blockmanagement.TestBlocksWithNotEnoughRacks |
| | hadoop.hdfs.server.mover.TestMover |
| | hadoop.hdfs.server.blockmanagement.TestBlockManager |
| | hadoop.hdfs.TestSafeModeWithStripedFileWithRandomECPolicy |
| | hadoop.hdfs.server.blockmanagement.TestBlockTokenWithDFSStriped |
| | hadoop.hdfs.server.sps.TestExternalStoragePolicySatisfier |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=19.03.1 Server=19.03.1 Image:yetus/hadoop:bdbca0e53b4 |
| JIRA Issue | HDFS-14760 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12978318/HDFS-14760.002.patch |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient findbugs checkstyle |
| uname | Linux 0ac300ca276a 4.15.0-48-generic #51-Ubuntu SMP Wed Apr 3
08:28:49 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/patchprocess/precommit/personality/provided.sh |
| git revision | trunk / 28fb4b5 |
| maven | version: Apache Maven
[jira] [Commented] (HDFS-14760) Log INFO mode if snapshot usage and actual usage differ
[
https://issues.apache.org/jira/browse/HDFS-14760?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16913578#comment-16913578
]
Ayush Saxena commented on HDFS-14760:
-
Thanx everyone for the discussion here, IMO too error is too harsh for it,
moving it warn seems fair enough
> Log INFO mode if snapshot usage and actual usage differ
> ---
>
> Key: HDFS-14760
> URL: https://issues.apache.org/jira/browse/HDFS-14760
> Project: Hadoop HDFS
> Issue Type: Improvement
>Reporter: CR Hota
>Assignee: CR Hota
>Priority: Major
> Attachments: HDFS-14760.001.patch, HDFS-14760.002.patch
>
>
> In DirectoryWithQuotaFeature#checkStoragespace code logs in error mode
> without throwing any exceptions or action and pollutes logs. This should be
> in INFO mode.
> {code}
> private void checkStoragespace(final INodeDirectory dir, final long
> computed) {
> if (-1 != quota.getStorageSpace() && usage.getStorageSpace() != computed)
> {
> NameNode.LOG.error("BUG: Inconsistent storagespace for directory "
> + dir.getFullPathName() + ". Cached = " + usage.getStorageSpace()
> + " != Computed = " + computed);
> }
> }
> {code}
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-14760) Log INFO mode if snapshot usage and actual usage differ
[
https://issues.apache.org/jira/browse/HDFS-14760?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16913524#comment-16913524
]
Erik Krogen commented on HDFS-14760:
Sure, makes sense to me CR!
> Log INFO mode if snapshot usage and actual usage differ
> ---
>
> Key: HDFS-14760
> URL: https://issues.apache.org/jira/browse/HDFS-14760
> Project: Hadoop HDFS
> Issue Type: Improvement
>Reporter: CR Hota
>Assignee: CR Hota
>Priority: Major
> Attachments: HDFS-14760.001.patch, HDFS-14760.002.patch
>
>
> In DirectoryWithQuotaFeature#checkStoragespace code logs in error mode
> without throwing any exceptions or action and pollutes logs. This should be
> in INFO mode.
> {code}
> private void checkStoragespace(final INodeDirectory dir, final long
> computed) {
> if (-1 != quota.getStorageSpace() && usage.getStorageSpace() != computed)
> {
> NameNode.LOG.error("BUG: Inconsistent storagespace for directory "
> + dir.getFullPathName() + ". Cached = " + usage.getStorageSpace()
> + " != Computed = " + computed);
> }
> }
> {code}
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-14760) Log INFO mode if snapshot usage and actual usage differ
[
https://issues.apache.org/jira/browse/HDFS-14760?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16913522#comment-16913522
]
CR Hota commented on HDFS-14760:
[~xkrogen] Thanks for the review. 'WARN' makes sense too. Honestly I haven't
been able to wrap around my head on the whole feature yet and how to handle
these cases. But at this point, our hdfs installation wants to make sure no
'ERROR' is logged if it's not really an error that should/can be actionized.
> Log INFO mode if snapshot usage and actual usage differ
> ---
>
> Key: HDFS-14760
> URL: https://issues.apache.org/jira/browse/HDFS-14760
> Project: Hadoop HDFS
> Issue Type: Improvement
>Reporter: CR Hota
>Assignee: CR Hota
>Priority: Major
> Attachments: HDFS-14760.001.patch
>
>
> In DirectoryWithQuotaFeature#checkStoragespace code logs in error mode
> without throwing any exceptions or action and pollutes logs. This should be
> in INFO mode.
> {code}
> private void checkStoragespace(final INodeDirectory dir, final long
> computed) {
> if (-1 != quota.getStorageSpace() && usage.getStorageSpace() != computed)
> {
> NameNode.LOG.error("BUG: Inconsistent storagespace for directory "
> + dir.getFullPathName() + ". Cached = " + usage.getStorageSpace()
> + " != Computed = " + computed);
> }
> }
> {code}
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-14760) Log INFO mode if snapshot usage and actual usage differ
[
https://issues.apache.org/jira/browse/HDFS-14760?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16912393#comment-16912393
]
Erik Krogen commented on HDFS-14760:
I think at least WARN level is warranted, given that this is an issue which
could potentially cause directories to exceed their quota or be blocked even
though their quota is not yet met. But I think your point about logging and not
doing anything is valid. Perhaps in cases such as this we should be fixing the
cached version after logging -- self-heal, but also warn that something is not
quite right.
> Log INFO mode if snapshot usage and actual usage differ
> ---
>
> Key: HDFS-14760
> URL: https://issues.apache.org/jira/browse/HDFS-14760
> Project: Hadoop HDFS
> Issue Type: Improvement
>Reporter: CR Hota
>Assignee: CR Hota
>Priority: Major
> Attachments: HDFS-14760.001.patch
>
>
> In DirectoryWithQuotaFeature#checkStoragespace code logs in error mode
> without throwing any exceptions or action and pollutes logs. This should be
> in INFO mode.
> {code}
> private void checkStoragespace(final INodeDirectory dir, final long
> computed) {
> if (-1 != quota.getStorageSpace() && usage.getStorageSpace() != computed)
> {
> NameNode.LOG.error("BUG: Inconsistent storagespace for directory "
> + dir.getFullPathName() + ". Cached = " + usage.getStorageSpace()
> + " != Computed = " + computed);
> }
> }
> {code}
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-14760) Log INFO mode if snapshot usage and actual usage differ
[
https://issues.apache.org/jira/browse/HDFS-14760?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16911882#comment-16911882
]
Hadoop QA commented on HDFS-14760:
--
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
26s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m
0s{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 20m
2s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
4s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
44s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
5s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
13m 8s{color} | {color:green} branch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
0s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
49s{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
56s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
52s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
52s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
36s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
57s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
12m 34s{color} | {color:green} patch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
6s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
50s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 87m 1s{color}
| {color:red} hadoop-hdfs in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
34s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}145m 33s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.hdfs.server.namenode.ha.TestEditLogTailer |
| | hadoop.hdfs.server.namenode.TestFsck |
| | hadoop.hdfs.server.namenode.TestNameNodeMetadataConsistency |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=19.03.1 Server=19.03.1 Image:yetus/hadoop:bdbca0e53b4 |
| JIRA Issue | HDFS-14760 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12978122/HDFS-14760.001.patch |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient findbugs checkstyle |
| uname | Linux 942d1501b0aa 4.15.0-54-generic #58-Ubuntu SMP Mon Jun 24
10:55:24 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/patchprocess/precommit/personality/provided.sh |
| git revision | trunk / 3a145e2 |
| maven | version: Apache Maven 3.3.9 |
| Default Java | 1.8.0_222 |
| findbugs | v3.1.0-RC1 |
| unit |
https://builds.apache.org/job/PreCommit-HDFS-Build/27595/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-HDFS-Build/27595/testReport/ |
| Max. process+thread count | 3348 (vs. ulimit of 5500) |
|
[jira] [Commented] (HDFS-14760) Log INFO mode if snapshot usage and actual usage differ
[
https://issues.apache.org/jira/browse/HDFS-14760?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16911837#comment-16911837
]
CR Hota commented on HDFS-14760:
Thanks [~jojochuang]
Adding some more folks for context. [~ayushtkn] [~xkrogen] [~RANith]
[~brahmareddy] .
> Log INFO mode if snapshot usage and actual usage differ
> ---
>
> Key: HDFS-14760
> URL: https://issues.apache.org/jira/browse/HDFS-14760
> Project: Hadoop HDFS
> Issue Type: Improvement
>Reporter: CR Hota
>Assignee: CR Hota
>Priority: Major
> Attachments: HDFS-14760.001.patch
>
>
> In DirectoryWithQuotaFeature#checkStoragespace code logs in error mode
> without throwing any exceptions or action and pollutes logs. This should be
> in INFO mode.
> {code}
> private void checkStoragespace(final INodeDirectory dir, final long
> computed) {
> if (-1 != quota.getStorageSpace() && usage.getStorageSpace() != computed)
> {
> NameNode.LOG.error("BUG: Inconsistent storagespace for directory "
> + dir.getFullPathName() + ". Cached = " + usage.getStorageSpace()
> + " != Computed = " + computed);
> }
> }
> {code}
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-14760) Log INFO mode if snapshot usage and actual usage differ
[
https://issues.apache.org/jira/browse/HDFS-14760?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16911833#comment-16911833
]
Wei-Chiu Chuang commented on HDFS-14760:
I'm pretty sure i've seen this before.
There are a few recent bug reports around quota calculation. Might be related:
HDFS-14566 HDFS-14633 HDFS-14519
> Log INFO mode if snapshot usage and actual usage differ
> ---
>
> Key: HDFS-14760
> URL: https://issues.apache.org/jira/browse/HDFS-14760
> Project: Hadoop HDFS
> Issue Type: Improvement
>Reporter: CR Hota
>Assignee: CR Hota
>Priority: Major
> Attachments: HDFS-14760.001.patch
>
>
> In DirectoryWithQuotaFeature#checkStoragespace code logs in error mode
> without throwing any exceptions or action and pollutes logs. This should be
> in INFO mode.
> {code}
> private void checkStoragespace(final INodeDirectory dir, final long
> computed) {
> if (-1 != quota.getStorageSpace() && usage.getStorageSpace() != computed)
> {
> NameNode.LOG.error("BUG: Inconsistent storagespace for directory "
> + dir.getFullPathName() + ". Cached = " + usage.getStorageSpace()
> + " != Computed = " + computed);
> }
> }
> {code}
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-14760) Log INFO mode if snapshot usage and actual usage differ
[
https://issues.apache.org/jira/browse/HDFS-14760?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16911816#comment-16911816
]
CR Hota commented on HDFS-14760:
[~jojochuang]
Could you help take a look at this? Am not very familiar why historically this
check was added and also that it doesn't take any action but logged in error
mode.
> Log INFO mode if snapshot usage and actual usage differ
> ---
>
> Key: HDFS-14760
> URL: https://issues.apache.org/jira/browse/HDFS-14760
> Project: Hadoop HDFS
> Issue Type: Improvement
>Reporter: CR Hota
>Assignee: CR Hota
>Priority: Major
> Attachments: HDFS-14760.001.patch
>
>
> In DirectoryWithQuotaFeature#checkStoragespace code logs in error mode
> without throwing any exceptions or action and pollutes logs. This should be
> in INFO mode.
> {code}
> private void checkStoragespace(final INodeDirectory dir, final long
> computed) {
> if (-1 != quota.getStorageSpace() && usage.getStorageSpace() != computed)
> {
> NameNode.LOG.error("BUG: Inconsistent storagespace for directory "
> + dir.getFullPathName() + ". Cached = " + usage.getStorageSpace()
> + " != Computed = " + computed);
> }
> }
> {code}
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
