[jira] [Commented] (HDFS-9187) Check if tracer is null before using it
[
https://issues.apache.org/jira/browse/HDFS-9187?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14952657#comment-14952657
]
Sangjin Lee commented on HDFS-9187:
---
I also ran into this issue with the HBase mini cluster.
In the latest patch (v.2), the latter change in Globber.java is causing the
NullPointerException in the test failures, and doesn't seem necessary:
{code}
58 this.tracer = FsTracer.get(fs.getConf());
{code}
> Check if tracer is null before using it
> ---
>
> Key: HDFS-9187
> URL: https://issues.apache.org/jira/browse/HDFS-9187
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: tracing
>Affects Versions: 2.8.0
>Reporter: stack
>Assignee: Colin Patrick McCabe
> Attachments: HDFS-9187.001.patch, HDFS-9187.002.patch
>
>
> Saw this where an hbase that has not been updated to htrace-4.0.1 was trying
> to start:
> {code}
> Oct 1, 5:12:11.861 AM FATAL org.apache.hadoop.hbase.master.HMaster
> Failed to become active master
> java.lang.NullPointerException
> at org.apache.hadoop.fs.Globber.glob(Globber.java:145)
> at org.apache.hadoop.fs.FileSystem.globStatus(FileSystem.java:1634)
> at org.apache.hadoop.hbase.util.FSUtils.getTableDirs(FSUtils.java:1372)
> at
> org.apache.hadoop.hbase.util.FSTableDescriptors.getAll(FSTableDescriptors.java:206)
> at
> org.apache.hadoop.hbase.master.HMaster.finishActiveMasterInitialization(HMaster.java:619)
> at org.apache.hadoop.hbase.master.HMaster.access$500(HMaster.java:169)
> at org.apache.hadoop.hbase.master.HMaster$1.run(HMaster.java:1481)
> at java.lang.Thread.run(Thread.java:745)
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HDFS-9187) Check if tracer is null before using it
[
https://issues.apache.org/jira/browse/HDFS-9187?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14940238#comment-14940238
]
Masatake Iwasaki commented on HDFS-9187:
{{tracer}} seems to be null if the filssystem instance is not created by
{{FileSystem#createFileSystem}}. Should we add NullTracer (similar to
NullScope) to HTrace and use it's singleton as default tracer rather than
adding around null check everywhere?
> Check if tracer is null before using it
> ---
>
> Key: HDFS-9187
> URL: https://issues.apache.org/jira/browse/HDFS-9187
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: tracing
>Affects Versions: 2.8.0
>Reporter: stack
>
> Saw this where an hbase that has not been updated to htrace-4.0.1 was trying
> to start:
> {code}
> Oct 1, 5:12:11.861 AM FATAL org.apache.hadoop.hbase.master.HMaster
> Failed to become active master
> java.lang.NullPointerException
> at org.apache.hadoop.fs.Globber.glob(Globber.java:145)
> at org.apache.hadoop.fs.FileSystem.globStatus(FileSystem.java:1634)
> at org.apache.hadoop.hbase.util.FSUtils.getTableDirs(FSUtils.java:1372)
> at
> org.apache.hadoop.hbase.util.FSTableDescriptors.getAll(FSTableDescriptors.java:206)
> at
> org.apache.hadoop.hbase.master.HMaster.finishActiveMasterInitialization(HMaster.java:619)
> at org.apache.hadoop.hbase.master.HMaster.access$500(HMaster.java:169)
> at org.apache.hadoop.hbase.master.HMaster$1.run(HMaster.java:1481)
> at java.lang.Thread.run(Thread.java:745)
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HDFS-9187) Check if tracer is null before using it
[
https://issues.apache.org/jira/browse/HDFS-9187?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14940338#comment-14940338
]
stack commented on HDFS-9187:
-
[~iwasakims] Yeah, we should probably get a NullTracer in... Less null-checks
all around. Would be htrace-4.0.2?
> Check if tracer is null before using it
> ---
>
> Key: HDFS-9187
> URL: https://issues.apache.org/jira/browse/HDFS-9187
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: tracing
>Affects Versions: 2.8.0
>Reporter: stack
>
> Saw this where an hbase that has not been updated to htrace-4.0.1 was trying
> to start:
> {code}
> Oct 1, 5:12:11.861 AM FATAL org.apache.hadoop.hbase.master.HMaster
> Failed to become active master
> java.lang.NullPointerException
> at org.apache.hadoop.fs.Globber.glob(Globber.java:145)
> at org.apache.hadoop.fs.FileSystem.globStatus(FileSystem.java:1634)
> at org.apache.hadoop.hbase.util.FSUtils.getTableDirs(FSUtils.java:1372)
> at
> org.apache.hadoop.hbase.util.FSTableDescriptors.getAll(FSTableDescriptors.java:206)
> at
> org.apache.hadoop.hbase.master.HMaster.finishActiveMasterInitialization(HMaster.java:619)
> at org.apache.hadoop.hbase.master.HMaster.access$500(HMaster.java:169)
> at org.apache.hadoop.hbase.master.HMaster$1.run(HMaster.java:1481)
> at java.lang.Thread.run(Thread.java:745)
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HDFS-9187) Check if tracer is null before using it
[
https://issues.apache.org/jira/browse/HDFS-9187?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14940594#comment-14940594
]
Colin Patrick McCabe commented on HDFS-9187:
I think adding a NullTracer would make sense. I also think there is an easier
solution via using FsTracer.java... let me check it out
> Check if tracer is null before using it
> ---
>
> Key: HDFS-9187
> URL: https://issues.apache.org/jira/browse/HDFS-9187
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: tracing
>Affects Versions: 2.8.0
>Reporter: stack
> Attachments: HDFS-9187.001.patch
>
>
> Saw this where an hbase that has not been updated to htrace-4.0.1 was trying
> to start:
> {code}
> Oct 1, 5:12:11.861 AM FATAL org.apache.hadoop.hbase.master.HMaster
> Failed to become active master
> java.lang.NullPointerException
> at org.apache.hadoop.fs.Globber.glob(Globber.java:145)
> at org.apache.hadoop.fs.FileSystem.globStatus(FileSystem.java:1634)
> at org.apache.hadoop.hbase.util.FSUtils.getTableDirs(FSUtils.java:1372)
> at
> org.apache.hadoop.hbase.util.FSTableDescriptors.getAll(FSTableDescriptors.java:206)
> at
> org.apache.hadoop.hbase.master.HMaster.finishActiveMasterInitialization(HMaster.java:619)
> at org.apache.hadoop.hbase.master.HMaster.access$500(HMaster.java:169)
> at org.apache.hadoop.hbase.master.HMaster$1.run(HMaster.java:1481)
> at java.lang.Thread.run(Thread.java:745)
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HDFS-9187) Check if tracer is null before using it
[
https://issues.apache.org/jira/browse/HDFS-9187?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14940608#comment-14940608
]
Colin Patrick McCabe commented on HDFS-9187:
This is a different approach that just removes Tracer from FileSystem, given
the difficulty of ensuring it's initialized. (The other Tracers like DFSClient
don't have this problem since the constructor ALWAYS does it... unlike FS where
there is a no-argument constructor.) I also added a unit test which is a
regression test for this bug.
> Check if tracer is null before using it
> ---
>
> Key: HDFS-9187
> URL: https://issues.apache.org/jira/browse/HDFS-9187
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: tracing
>Affects Versions: 2.8.0
>Reporter: stack
>Assignee: Colin Patrick McCabe
> Attachments: HDFS-9187.001.patch, HDFS-9187.002.patch
>
>
> Saw this where an hbase that has not been updated to htrace-4.0.1 was trying
> to start:
> {code}
> Oct 1, 5:12:11.861 AM FATAL org.apache.hadoop.hbase.master.HMaster
> Failed to become active master
> java.lang.NullPointerException
> at org.apache.hadoop.fs.Globber.glob(Globber.java:145)
> at org.apache.hadoop.fs.FileSystem.globStatus(FileSystem.java:1634)
> at org.apache.hadoop.hbase.util.FSUtils.getTableDirs(FSUtils.java:1372)
> at
> org.apache.hadoop.hbase.util.FSTableDescriptors.getAll(FSTableDescriptors.java:206)
> at
> org.apache.hadoop.hbase.master.HMaster.finishActiveMasterInitialization(HMaster.java:619)
> at org.apache.hadoop.hbase.master.HMaster.access$500(HMaster.java:169)
> at org.apache.hadoop.hbase.master.HMaster$1.run(HMaster.java:1481)
> at java.lang.Thread.run(Thread.java:745)
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HDFS-9187) Check if tracer is null before using it
[
https://issues.apache.org/jira/browse/HDFS-9187?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14940612#comment-14940612
]
stack commented on HDFS-9187:
-
Globber is only place that uses the FS tracer?
This patch is better than what I was thinking of doing.
This patch seems good to me. We should work on the [~iwasakims] idea in the
meantime "for everyone else.." to save having to do null checks.
> Check if tracer is null before using it
> ---
>
> Key: HDFS-9187
> URL: https://issues.apache.org/jira/browse/HDFS-9187
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: tracing
>Affects Versions: 2.8.0
>Reporter: stack
>Assignee: Colin Patrick McCabe
> Attachments: HDFS-9187.001.patch, HDFS-9187.002.patch
>
>
> Saw this where an hbase that has not been updated to htrace-4.0.1 was trying
> to start:
> {code}
> Oct 1, 5:12:11.861 AM FATAL org.apache.hadoop.hbase.master.HMaster
> Failed to become active master
> java.lang.NullPointerException
> at org.apache.hadoop.fs.Globber.glob(Globber.java:145)
> at org.apache.hadoop.fs.FileSystem.globStatus(FileSystem.java:1634)
> at org.apache.hadoop.hbase.util.FSUtils.getTableDirs(FSUtils.java:1372)
> at
> org.apache.hadoop.hbase.util.FSTableDescriptors.getAll(FSTableDescriptors.java:206)
> at
> org.apache.hadoop.hbase.master.HMaster.finishActiveMasterInitialization(HMaster.java:619)
> at org.apache.hadoop.hbase.master.HMaster.access$500(HMaster.java:169)
> at org.apache.hadoop.hbase.master.HMaster$1.run(HMaster.java:1481)
> at java.lang.Thread.run(Thread.java:745)
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HDFS-9187) Check if tracer is null before using it
[
https://issues.apache.org/jira/browse/HDFS-9187?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14940651#comment-14940651
]
Hadoop QA commented on HDFS-9187:
-
\\
\\
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | pre-patch | 18m 1s | Pre-patch trunk compilation is
healthy. |
| {color:green}+1{color} | @author | 0m 0s | The patch does not contain any
@author tags. |
| {color:green}+1{color} | tests included | 0m 0s | The patch appears to
include 1 new or modified test files. |
| {color:green}+1{color} | javac | 8m 28s | There were no new javac warning
messages. |
| {color:green}+1{color} | javadoc | 10m 25s | There were no new javadoc
warning messages. |
| {color:red}-1{color} | release audit | 0m 16s | The applied patch generated
1 release audit warnings. |
| {color:red}-1{color} | checkstyle | 1m 14s | The applied patch generated 1
new checkstyle issues (total was 145, now 145). |
| {color:green}+1{color} | whitespace | 0m 0s | The patch has no lines that
end in whitespace. |
| {color:green}+1{color} | install | 1m 34s | mvn install still works. |
| {color:green}+1{color} | eclipse:eclipse | 0m 36s | The patch built with
eclipse:eclipse. |
| {color:red}-1{color} | findbugs | 2m 6s | The patch appears to introduce 1
new Findbugs (version 3.0.0) warnings. |
| {color:red}-1{color} | common tests | 6m 24s | Tests failed in
hadoop-common. |
| | | 49m 8s | |
\\
\\
|| Reason || Tests ||
| FindBugs | module:hadoop-common |
| Failed unit tests | hadoop.fs.TestFileUtil |
| | hadoop.fs.TestLocalFSFileContextMainOperations |
| | hadoop.fs.viewfs.TestFcMainOperationsLocalFs |
| | hadoop.metrics2.sink.TestFileSink |
\\
\\
|| Subsystem || Report/Notes ||
| Patch URL |
http://issues.apache.org/jira/secure/attachment/12764711/HDFS-9187.002.patch |
| Optional Tests | javadoc javac unit findbugs checkstyle |
| git revision | trunk / fd026f5 |
| Release Audit |
https://builds.apache.org/job/PreCommit-HDFS-Build/12767/artifact/patchprocess/patchReleaseAuditProblems.txt
|
| checkstyle |
https://builds.apache.org/job/PreCommit-HDFS-Build/12767/artifact/patchprocess/diffcheckstylehadoop-common.txt
|
| Findbugs warnings |
https://builds.apache.org/job/PreCommit-HDFS-Build/12767/artifact/patchprocess/newPatchFindbugsWarningshadoop-common.html
|
| hadoop-common test log |
https://builds.apache.org/job/PreCommit-HDFS-Build/12767/artifact/patchprocess/testrun_hadoop-common.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-HDFS-Build/12767/testReport/ |
| Java | 1.7.0_55 |
| uname | Linux asf904.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP
PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |
| Console output |
https://builds.apache.org/job/PreCommit-HDFS-Build/12767/console |
This message was automatically generated.
> Check if tracer is null before using it
> ---
>
> Key: HDFS-9187
> URL: https://issues.apache.org/jira/browse/HDFS-9187
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: tracing
>Affects Versions: 2.8.0
>Reporter: stack
>Assignee: Colin Patrick McCabe
> Attachments: HDFS-9187.001.patch, HDFS-9187.002.patch
>
>
> Saw this where an hbase that has not been updated to htrace-4.0.1 was trying
> to start:
> {code}
> Oct 1, 5:12:11.861 AM FATAL org.apache.hadoop.hbase.master.HMaster
> Failed to become active master
> java.lang.NullPointerException
> at org.apache.hadoop.fs.Globber.glob(Globber.java:145)
> at org.apache.hadoop.fs.FileSystem.globStatus(FileSystem.java:1634)
> at org.apache.hadoop.hbase.util.FSUtils.getTableDirs(FSUtils.java:1372)
> at
> org.apache.hadoop.hbase.util.FSTableDescriptors.getAll(FSTableDescriptors.java:206)
> at
> org.apache.hadoop.hbase.master.HMaster.finishActiveMasterInitialization(HMaster.java:619)
> at org.apache.hadoop.hbase.master.HMaster.access$500(HMaster.java:169)
> at org.apache.hadoop.hbase.master.HMaster$1.run(HMaster.java:1481)
> at java.lang.Thread.run(Thread.java:745)
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HDFS-9187) Check if tracer is null before using it
[
https://issues.apache.org/jira/browse/HDFS-9187?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14940616#comment-14940616
]
Colin Patrick McCabe commented on HDFS-9187:
bq. Globber is only place that uses the FS tracer?
Yeah
bq. This patch seems good to me. We should work on the Masatake Iwasaki idea in
the meantime "for everyone else.." to save having to do null checks.
Agree. I filed HTRACE-275 for this.
> Check if tracer is null before using it
> ---
>
> Key: HDFS-9187
> URL: https://issues.apache.org/jira/browse/HDFS-9187
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: tracing
>Affects Versions: 2.8.0
>Reporter: stack
>Assignee: Colin Patrick McCabe
> Attachments: HDFS-9187.001.patch, HDFS-9187.002.patch
>
>
> Saw this where an hbase that has not been updated to htrace-4.0.1 was trying
> to start:
> {code}
> Oct 1, 5:12:11.861 AM FATAL org.apache.hadoop.hbase.master.HMaster
> Failed to become active master
> java.lang.NullPointerException
> at org.apache.hadoop.fs.Globber.glob(Globber.java:145)
> at org.apache.hadoop.fs.FileSystem.globStatus(FileSystem.java:1634)
> at org.apache.hadoop.hbase.util.FSUtils.getTableDirs(FSUtils.java:1372)
> at
> org.apache.hadoop.hbase.util.FSTableDescriptors.getAll(FSTableDescriptors.java:206)
> at
> org.apache.hadoop.hbase.master.HMaster.finishActiveMasterInitialization(HMaster.java:619)
> at org.apache.hadoop.hbase.master.HMaster.access$500(HMaster.java:169)
> at org.apache.hadoop.hbase.master.HMaster$1.run(HMaster.java:1481)
> at java.lang.Thread.run(Thread.java:745)
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HDFS-9187) Check if tracer is null before using it
[
https://issues.apache.org/jira/browse/HDFS-9187?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14940622#comment-14940622
]
stack commented on HDFS-9187:
-
Let me test to see if it fixes the issue I saw above...
> Check if tracer is null before using it
> ---
>
> Key: HDFS-9187
> URL: https://issues.apache.org/jira/browse/HDFS-9187
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: tracing
>Affects Versions: 2.8.0
>Reporter: stack
>Assignee: Colin Patrick McCabe
> Attachments: HDFS-9187.001.patch, HDFS-9187.002.patch
>
>
> Saw this where an hbase that has not been updated to htrace-4.0.1 was trying
> to start:
> {code}
> Oct 1, 5:12:11.861 AM FATAL org.apache.hadoop.hbase.master.HMaster
> Failed to become active master
> java.lang.NullPointerException
> at org.apache.hadoop.fs.Globber.glob(Globber.java:145)
> at org.apache.hadoop.fs.FileSystem.globStatus(FileSystem.java:1634)
> at org.apache.hadoop.hbase.util.FSUtils.getTableDirs(FSUtils.java:1372)
> at
> org.apache.hadoop.hbase.util.FSTableDescriptors.getAll(FSTableDescriptors.java:206)
> at
> org.apache.hadoop.hbase.master.HMaster.finishActiveMasterInitialization(HMaster.java:619)
> at org.apache.hadoop.hbase.master.HMaster.access$500(HMaster.java:169)
> at org.apache.hadoop.hbase.master.HMaster$1.run(HMaster.java:1481)
> at java.lang.Thread.run(Thread.java:745)
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
