[jira] [Comment Edited] (HADOOP-9435) Minimal code change to support IBM jvm build dependency
[ https://issues.apache.org/jira/browse/HADOOP-9435?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13664943#comment-13664943 ] Suresh Srinivas edited comment on HADOOP-9435 at 5/24/13 6:23 AM: -- [~tianhonglouis] Want to make sure that this is ready to be committed and no tests are pending. was (Author: sureshms): [~tianhonglouis] Want to make sure that this is ready to be committed and not tests are pending. > Minimal code change to support IBM jvm build dependency > --- > > Key: HADOOP-9435 > URL: https://issues.apache.org/jira/browse/HADOOP-9435 > Project: Hadoop Common > Issue Type: Bug > Components: build >Reporter: Tian Hong Wang >Assignee: Tian Hong Wang > Labels: patch > Attachments: HADOOP-9435.patch, HADOOP-9435-v1.patch > > > When native build hadoop-common-project with IBM java using command like: > mvn package -Pnative > it will exist the following errors. > [exec] -- Configuring incomplete, errors occurred! > [exec] JAVA_HOME=, > JAVA_JVM_LIBRARY=/home/louis/ibm-java-i386-60/jre/lib/i386/classic/libjvm.so > [exec] JAVA_INCLUDE_PATH=/home/louis/ibm-java-i386-60/include, > JAVA_INCLUDE_PATH2=JAVA_INCLUDE_PATH2-NOTFOUND > [exec] CMake Error at JNIFlags.cmake:113 (MESSAGE): > [exec] Failed to find a viable JVM installation under JAVA_HOME. > [exec] Call Stack (most recent call first): > [exec] CMakeLists.txt:24 (include) > The reason is that IBM java uses $JAVA_HOME/include/jniport.h instead of > $JAVA_HOME/include/jni_md.h in non-IBM java. > [exec] > /usr/lib/jvm/java-1.6.0-ibm-1.6.0.12.0.x86_64/jre/lib/amd64/default/libjvm.so: > undefined reference to `dlsym' > [exec] > /usr/lib/jvm/java-1.6.0-ibm-1.6.0.12.0.x86_64/jre/lib/amd64/default/libjvm.so: > undefined reference to `dlerror' > [exec] > /usr/lib/jvm/java-1.6.0-ibm-1.6.0.12.0.x86_64/jre/lib/amd64/default/libjvm.so: > undefined reference to `dladdr' > [exec] > /usr/lib/jvm/java-1.6.0-ibm-1.6.0.12.0.x86_64/jre/lib/amd64/default/libjvm.so: > undefined reference to `dlopen' > [exec] > /usr/lib/jvm/java-1.6.0-ibm-1.6.0.12.0.x86_64/jre/lib/amd64/default/libjvm.so: > undefined reference to `dlclose' > [exec] collect2: ld returned 1 exit status > [exec] make[2]: *** [test_libhdfs_ops] Error 1 > [exec] make[1]: *** [CMakeFiles/test_libhdfs_ops.dir/all] Error 2 > [exec] make: *** [all] Error > The reason is libjvm.so need libdl when linking. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (HADOOP-9597) FileSystem open() API is not clear if FileNotFoundException is throw when the path does not exist
[ https://issues.apache.org/jira/browse/HADOOP-9597?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Suresh Srinivas resolved HADOOP-9597. - Resolution: Invalid > FileSystem open() API is not clear if FileNotFoundException is throw when the > path does not exist > - > > Key: HADOOP-9597 > URL: https://issues.apache.org/jira/browse/HADOOP-9597 > Project: Hadoop Common > Issue Type: Improvement > Components: documentation, fs >Affects Versions: 2.0.4-alpha >Reporter: Jerry He >Priority: Minor > Fix For: 2.0.4.1-alpha > > > The current FileSystem open() method throws a generic IOException in its API > specification. > Some FileSystem implementations (DFS, RawLocalFileSystem ...) throws more > specific FileNotFoundException if the path does not exist. Some throws > IOException only (FTPFileSystem, HftpFileSystem ...). > If we have a new FileSystem implementation, what should we follow exactly for > open()? > What should the application expect in this case. > -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9597) FileSystem open() API is not clear if FileNotFoundException is throw when the path does not exist
[ https://issues.apache.org/jira/browse/HADOOP-9597?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13666025#comment-13666025 ] Suresh Srinivas commented on HADOOP-9597: - Jerry, please do not open jiras for asking questions. Jiras are for reporting problem. FileSystem.java is not well documented and is implemented differently by different file systems. My recommendation would be to follow DistributedFileSystem implementation. You could also use FileContext javadoc to guide in what exception to throw. > FileSystem open() API is not clear if FileNotFoundException is throw when the > path does not exist > - > > Key: HADOOP-9597 > URL: https://issues.apache.org/jira/browse/HADOOP-9597 > Project: Hadoop Common > Issue Type: Improvement > Components: documentation, fs >Affects Versions: 2.0.4-alpha >Reporter: Jerry He >Priority: Minor > Fix For: 2.0.4.1-alpha > > > The current FileSystem open() method throws a generic IOException in its API > specification. > Some FileSystem implementations (DFS, RawLocalFileSystem ...) throws more > specific FileNotFoundException if the path does not exist. Some throws > IOException only (FTPFileSystem, HftpFileSystem ...). > If we have a new FileSystem implementation, what should we follow exactly for > open()? > What should the application expect in this case. > -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HADOOP-9597) FileSystem open() API is not clear if FileNotFoundException is throw when the path does not exist
Jerry He created HADOOP-9597: Summary: FileSystem open() API is not clear if FileNotFoundException is throw when the path does not exist Key: HADOOP-9597 URL: https://issues.apache.org/jira/browse/HADOOP-9597 Project: Hadoop Common Issue Type: Improvement Components: documentation, fs Affects Versions: 2.0.4-alpha Reporter: Jerry He Priority: Minor Fix For: 2.0.4.1-alpha The current FileSystem open() method throws a generic IOException in its API specification. Some FileSystem implementations (DFS, RawLocalFileSystem ...) throws more specific FileNotFoundException if the path does not exist. Some throws IOException only (FTPFileSystem, HftpFileSystem ...). If we have a new FileSystem implementation, what should we follow exactly for open()? What should the application expect in this case. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-9431) TestSecurityUtil#testLocalHostNameForNullOrWild on systems where hostname contains capital letters
[
https://issues.apache.org/jira/browse/HADOOP-9431?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Suresh Srinivas updated HADOOP-9431:
Fix Version/s: 3.0.0
> TestSecurityUtil#testLocalHostNameForNullOrWild on systems where hostname
> contains capital letters
> --
>
> Key: HADOOP-9431
> URL: https://issues.apache.org/jira/browse/HADOOP-9431
> Project: Hadoop Common
> Issue Type: Bug
> Components: security, test
>Affects Versions: 3.0.0
>Reporter: Chris Nauroth
>Assignee: Chris Nauroth
>Priority: Trivial
> Fix For: 3.0.0
>
> Attachments: HADOOP-9431.1.patch
>
>
> {{TestSecurityUtil#testLocalHostNameForNullOrWild}} contains assertions about
> the principal returned for a null or wildcard hostname. The logic in
> {{SecurityUtil#getServerPrincipal}} converts the local hostname to
> lower-case. The test method does not perform the same conversion though, so
> the test passes on systems with a hostname containing no upper-case letters,
> but it fails if the hostname does contain upper-case letters.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-9433) TestLocalFileSystem#testHasFileDescriptor leaks file handle
[ https://issues.apache.org/jira/browse/HADOOP-9433?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Suresh Srinivas updated HADOOP-9433: Fix Version/s: 3.0.0 > TestLocalFileSystem#testHasFileDescriptor leaks file handle > --- > > Key: HADOOP-9433 > URL: https://issues.apache.org/jira/browse/HADOOP-9433 > Project: Hadoop Common > Issue Type: Bug > Components: fs, test >Affects Versions: 3.0.0 >Reporter: Chris Nauroth >Assignee: Chris Nauroth > Fix For: 3.0.0 > > Attachments: HADOOP-9433.1.patch > > > The test does not close a file handle. This can cause subsequent tests in > the suite to fail due to failure to delete the test working directory. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9573) Fix test-patch script to work with the enhanced PreCommit-Admin script.
[ https://issues.apache.org/jira/browse/HADOOP-9573?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13665951#comment-13665951 ] Giridharan Kesavan commented on HADOOP-9573: reivew board: https://reviews.apache.org/r/11263/diff/2/ > Fix test-patch script to work with the enhanced PreCommit-Admin script. > --- > > Key: HADOOP-9573 > URL: https://issues.apache.org/jira/browse/HADOOP-9573 > Project: Hadoop Common > Issue Type: Sub-task > Components: build >Affects Versions: 1.0.3 >Reporter: Giridharan Kesavan >Assignee: Giridharan Kesavan > Attachments: > 0001-Fix-test-patch-scrit-to-work-with-the-enhanced-PreCo.patch, > hadoop-9573.patch > > > test-patch script currently take the latest available patch for a given jira > and performs the test. This jira is to enhance test-patch script to take > attachment-id of a patch as an input and perform the tests using that > attachment-id -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9435) Minimal code change to support IBM jvm build dependency
[ https://issues.apache.org/jira/browse/HADOOP-9435?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13665952#comment-13665952 ] Tian Hong Wang commented on HADOOP-9435: Suresh, please commit it, no pending test. Thanks. > Minimal code change to support IBM jvm build dependency > --- > > Key: HADOOP-9435 > URL: https://issues.apache.org/jira/browse/HADOOP-9435 > Project: Hadoop Common > Issue Type: Bug > Components: build >Reporter: Tian Hong Wang >Assignee: Tian Hong Wang > Labels: patch > Attachments: HADOOP-9435.patch, HADOOP-9435-v1.patch > > > When native build hadoop-common-project with IBM java using command like: > mvn package -Pnative > it will exist the following errors. > [exec] -- Configuring incomplete, errors occurred! > [exec] JAVA_HOME=, > JAVA_JVM_LIBRARY=/home/louis/ibm-java-i386-60/jre/lib/i386/classic/libjvm.so > [exec] JAVA_INCLUDE_PATH=/home/louis/ibm-java-i386-60/include, > JAVA_INCLUDE_PATH2=JAVA_INCLUDE_PATH2-NOTFOUND > [exec] CMake Error at JNIFlags.cmake:113 (MESSAGE): > [exec] Failed to find a viable JVM installation under JAVA_HOME. > [exec] Call Stack (most recent call first): > [exec] CMakeLists.txt:24 (include) > The reason is that IBM java uses $JAVA_HOME/include/jniport.h instead of > $JAVA_HOME/include/jni_md.h in non-IBM java. > [exec] > /usr/lib/jvm/java-1.6.0-ibm-1.6.0.12.0.x86_64/jre/lib/amd64/default/libjvm.so: > undefined reference to `dlsym' > [exec] > /usr/lib/jvm/java-1.6.0-ibm-1.6.0.12.0.x86_64/jre/lib/amd64/default/libjvm.so: > undefined reference to `dlerror' > [exec] > /usr/lib/jvm/java-1.6.0-ibm-1.6.0.12.0.x86_64/jre/lib/amd64/default/libjvm.so: > undefined reference to `dladdr' > [exec] > /usr/lib/jvm/java-1.6.0-ibm-1.6.0.12.0.x86_64/jre/lib/amd64/default/libjvm.so: > undefined reference to `dlopen' > [exec] > /usr/lib/jvm/java-1.6.0-ibm-1.6.0.12.0.x86_64/jre/lib/amd64/default/libjvm.so: > undefined reference to `dlclose' > [exec] collect2: ld returned 1 exit status > [exec] make[2]: *** [test_libhdfs_ops] Error 1 > [exec] make[1]: *** [CMakeFiles/test_libhdfs_ops.dir/all] Error 2 > [exec] make: *** [all] Error > The reason is libjvm.so need libdl when linking. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-8469) Make NetworkTopology class pluggable
[
https://issues.apache.org/jira/browse/HADOOP-8469?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Junping Du updated HADOOP-8469:
---
Attachment: HADOOP-8469-branch-2.patch
Attach patch against branch-2
> Make NetworkTopology class pluggable
>
>
> Key: HADOOP-8469
> URL: https://issues.apache.org/jira/browse/HADOOP-8469
> Project: Hadoop Common
> Issue Type: Sub-task
>Affects Versions: 1.0.0, 2.0.0-alpha
>Reporter: Junping Du
>Assignee: Junping Du
> Fix For: 3.0.0
>
> Attachments: HADOOP-8469-branch-2.patch,
> HADOOP-8469-NetworkTopology-pluggable.patch,
> HADOOP-8469-NetworkTopology-pluggable-v2.patch,
> HADOOP-8469-NetworkTopology-pluggable-v3.patch,
> HADOOP-8469-NetworkTopology-pluggable-v4.patch,
> HADOOP-8469-NetworkTopology-pluggable-v5.patch
>
>
> The class NetworkTopology is where the three-layer hierarchical topology is
> modeled in the current code base and is instantiated directly by the
> DatanodeManager and Balancer.
> To support alternative topologies, changes were make the topology class
> pluggable, that is to support using a user specified topology class specified
> in the Hadoop configuration file core-defaul.xml. The user specified topology
> class is instantiated using reflection in the same manner as other
> customizable classes in Hadoop. If no use specified topology class is found,
> the fallback is to use the NetworkTopology to preserve current behavior. To
> make it possible to reuse code in NetworkTopology several minor changes were
> made to make the class more extensible. The NetworkTopology class is
> currently annotated with @InterfaceAudience.LimitedPrivate({"HDFS",
> "MapReduce"}) and @InterfaceStability.Unstable.
> The proposed changes in NetworkTopology listed below
> 1. Some fields were changes from private to protected
> 2. Added some protected methods so that sub classes could override behavior
> 3. Added a new method,isNodeGroupAware,to NetworkTopology
> 4. The inner class InnerNode was made a package protected class to it would
> be easier to subclass
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Reopened] (HADOOP-8469) Make NetworkTopology class pluggable
[
https://issues.apache.org/jira/browse/HADOOP-8469?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Junping Du reopened HADOOP-8469:
Backport this patch to branch-2
> Make NetworkTopology class pluggable
>
>
> Key: HADOOP-8469
> URL: https://issues.apache.org/jira/browse/HADOOP-8469
> Project: Hadoop Common
> Issue Type: Sub-task
>Affects Versions: 1.0.0, 2.0.0-alpha
>Reporter: Junping Du
>Assignee: Junping Du
> Fix For: 3.0.0
>
> Attachments: HADOOP-8469-NetworkTopology-pluggable.patch,
> HADOOP-8469-NetworkTopology-pluggable-v2.patch,
> HADOOP-8469-NetworkTopology-pluggable-v3.patch,
> HADOOP-8469-NetworkTopology-pluggable-v4.patch,
> HADOOP-8469-NetworkTopology-pluggable-v5.patch
>
>
> The class NetworkTopology is where the three-layer hierarchical topology is
> modeled in the current code base and is instantiated directly by the
> DatanodeManager and Balancer.
> To support alternative topologies, changes were make the topology class
> pluggable, that is to support using a user specified topology class specified
> in the Hadoop configuration file core-defaul.xml. The user specified topology
> class is instantiated using reflection in the same manner as other
> customizable classes in Hadoop. If no use specified topology class is found,
> the fallback is to use the NetworkTopology to preserve current behavior. To
> make it possible to reuse code in NetworkTopology several minor changes were
> made to make the class more extensible. The NetworkTopology class is
> currently annotated with @InterfaceAudience.LimitedPrivate({"HDFS",
> "MapReduce"}) and @InterfaceStability.Unstable.
> The proposed changes in NetworkTopology listed below
> 1. Some fields were changes from private to protected
> 2. Added some protected methods so that sub classes could override behavior
> 3. Added a new method,isNodeGroupAware,to NetworkTopology
> 4. The inner class InnerNode was made a package protected class to it would
> be easier to subclass
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9593) stack trace printed at ERROR for all yarn clients without hadoop.home set
[ https://issues.apache.org/jira/browse/HADOOP-9593?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13665852#comment-13665852 ] Arpit Agarwal commented on HADOOP-9593: --- HADOOP-8562 has been merged into branch-2. > stack trace printed at ERROR for all yarn clients without hadoop.home set > - > > Key: HADOOP-9593 > URL: https://issues.apache.org/jira/browse/HADOOP-9593 > Project: Hadoop Common > Issue Type: Bug > Components: util >Affects Versions: 3.0.0 >Reporter: Steve Loughran > Attachments: HADOOP-9593-001.patch, HADOOP-9593-002.patch > > > This is the problem of HADOOP-9482 now showing up in a different application > -one whose log4j settings haven't turned off all Shell logging. > Unless you do that, all yarn clients will have a stack trace at error in > their logs, which is generating false alarms and is utterly pointless. Why > does this merit a stack trace? Why log it at error? It's not an error for a > client app to not have these values set as long as they have the relevant > JARs on their classpath. And if they don't, they'll get some classpath error > instead -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8562) Enhancements to support Hadoop on Windows Server and Windows Azure environments
[ https://issues.apache.org/jira/browse/HADOOP-8562?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13665849#comment-13665849 ] Arpit Agarwal commented on HADOOP-8562: --- The merge is looking good. Just two failures on Mac. # TestRBWBlockInvalidation (timeout) # TestBookKeeperHACheckpoints (JVM OOM) > Enhancements to support Hadoop on Windows Server and Windows Azure > environments > --- > > Key: HADOOP-8562 > URL: https://issues.apache.org/jira/browse/HADOOP-8562 > Project: Hadoop Common > Issue Type: New Feature >Affects Versions: 3.0.0 >Reporter: Bikas Saha >Assignee: Bikas Saha > Fix For: 2.0.5-beta > > Attachments: branch-2.merge.patch, branch-2.merge.patch, > branch-trunk-win.min-notest.patch, branch-trunk-win-min.patch, > branch-trunk-win.min.patch, branch-trunk-win.patch, branch-trunk-win.patch, > branch-trunk-win.patch, branch-trunk-win.patch, branch-trunk-win.patch, > branch-trunk-win.patch, branch-trunk-win.patch, branch-trunk-win.patch, > branch-trunk-win.patch, branch-trunk-win.patch, branch-trunk-win.patch, > test-untar.tar, test-untar.tgz > > > This JIRA tracks the work that needs to be done on trunk to enable Hadoop to > run on Windows Server and Azure environments. This incorporates porting > relevant work from the similar effort on branch 1 tracked via HADOOP-8079. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9596) Add additional attributes to JMX beans
[
https://issues.apache.org/jira/browse/HADOOP-9596?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13665841#comment-13665841
]
Hadoop QA commented on HADOOP-9596:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12584618/0001-Hadoop-namenode-JMX-metrics-update.patch
against trunk revision .
{color:red}-1 patch{color}. The patch command could not apply the patch.
Console output:
https://builds.apache.org/job/PreCommit-HADOOP-Build/2563//console
This message is automatically generated.
> Add additional attributes to JMX beans
> --
>
> Key: HADOOP-9596
> URL: https://issues.apache.org/jira/browse/HADOOP-9596
> Project: Hadoop Common
> Issue Type: Improvement
> Components: fs
>Affects Versions: 2.0.4-alpha
>Reporter: Trevor Lorimer
>Priority: Trivial
> Attachments: 0001-Hadoop-namenode-JMX-metrics-update.patch
>
>
> Currently the JMX bean returns much of the data contained on the HDFS Health
> webpage (dfsHealth.html). However there are several other attributes that are
> required to be added.
> I intend to add the following items to the appropriate bean in parenthesis :
> Started time (NameNodeInfo),
> Compiled info (NameNodeInfo),
> Jvm MaxHeap, MaxNonHeap (JvmMetrics)
> Node Usage stats (i.e. Min, Median, Max, stdev) (NameNodeInfo),
> Count of decommissioned Live and Dead nodes (FSNamesystemState),
> Journal Status (NodeNameInfo)
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-9596) Add additional attributes to JMX beans
[ https://issues.apache.org/jira/browse/HADOOP-9596?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Trevor Lorimer updated HADOOP-9596: --- Description: Currently the JMX bean returns much of the data contained on the HDFS Health webpage (dfsHealth.html). However there are several other attributes that are required to be added. I intend to add the following items to the appropriate bean in parenthesis : Started time (NameNodeInfo), Compiled info (NameNodeInfo), Jvm MaxHeap, MaxNonHeap (JvmMetrics) Node Usage stats (i.e. Min, Median, Max, stdev) (NameNodeInfo), Count of decommissioned Live and Dead nodes (FSNamesystemState), Journal Status (NodeNameInfo) was: Currently the JMX bean returns much of the data contained on the HDFS Health webpage (dfsHealth.html). However there are several other attributes that are required to be added. I intend to add the following items to the appropriate bean in parenthesis : Started time (NameNodeInfo), Compiled info (NameNodeInfo), Jvm MaxHeap, MaxNonHeap (JvmMetrics) Node Usage stats (i.e. Min, Median, Max, stdev) (NameNodeInfo), Count of decommissioned Live and Dead nodes (FSNamesystemState), Journal Status (NodeNameInfo) > Add additional attributes to JMX beans > -- > > Key: HADOOP-9596 > URL: https://issues.apache.org/jira/browse/HADOOP-9596 > Project: Hadoop Common > Issue Type: Improvement > Components: fs >Affects Versions: 2.0.4-alpha >Reporter: Trevor Lorimer >Priority: Trivial > Attachments: 0001-Hadoop-namenode-JMX-metrics-update.patch > > > Currently the JMX bean returns much of the data contained on the HDFS Health > webpage (dfsHealth.html). However there are several other attributes that are > required to be added. > I intend to add the following items to the appropriate bean in parenthesis : > Started time (NameNodeInfo), > Compiled info (NameNodeInfo), > Jvm MaxHeap, MaxNonHeap (JvmMetrics) > Node Usage stats (i.e. Min, Median, Max, stdev) (NameNodeInfo), > Count of decommissioned Live and Dead nodes (FSNamesystemState), > Journal Status (NodeNameInfo) -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-9596) Add additional attributes to JMX beans
[ https://issues.apache.org/jira/browse/HADOOP-9596?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Trevor Lorimer updated HADOOP-9596: --- Environment: (was: Currently the JMX bean returns much of the data contained on the HDFS Health webpage (dfsHealth.html). However there are several other attributes that are required to be added. I intend to add the following items to the appropriate bean in parenthesis : Started time (NameNodeInfo), Compiled info (NameNodeInfo), Jvm MaxHeap, MaxNonHeap (JvmMetrics) Node Usage stats (i.e. Min, Median, Max, stdev) (NameNodeInfo), Count of decommissioned Live and Dead nodes (FSNamesystemState), Journal Status (NodeNameInfo)) > Add additional attributes to JMX beans > -- > > Key: HADOOP-9596 > URL: https://issues.apache.org/jira/browse/HADOOP-9596 > Project: Hadoop Common > Issue Type: Improvement > Components: fs >Affects Versions: 2.0.4-alpha >Reporter: Trevor Lorimer >Priority: Trivial > Attachments: 0001-Hadoop-namenode-JMX-metrics-update.patch > > > Currently the JMX bean returns much of the data contained on the HDFS Health > webpage (dfsHealth.html). However there are several other attributes that are > required to be added. > I intend to add the following items to the appropriate bean in parenthesis : > Started time (NameNodeInfo), > Compiled info (NameNodeInfo), > Jvm MaxHeap, MaxNonHeap (JvmMetrics) > Node Usage stats (i.e. Min, Median, Max, stdev) (NameNodeInfo), > Count of decommissioned Live and Dead nodes (FSNamesystemState), > Journal Status (NodeNameInfo) -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-9596) Add additional attributes to JMX beans
[ https://issues.apache.org/jira/browse/HADOOP-9596?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Trevor Lorimer updated HADOOP-9596: --- Status: Patch Available (was: Open) > Add additional attributes to JMX beans > -- > > Key: HADOOP-9596 > URL: https://issues.apache.org/jira/browse/HADOOP-9596 > Project: Hadoop Common > Issue Type: Improvement > Components: fs >Affects Versions: 2.0.4-alpha > Environment: Currently the JMX bean returns much of the data > contained on the HDFS Health webpage (dfsHealth.html). However there are > several other attributes that are required to be added. > I intend to add the following items to the appropriate bean in parenthesis : > Started time (NameNodeInfo), > Compiled info (NameNodeInfo), > Jvm MaxHeap, MaxNonHeap (JvmMetrics) > Node Usage stats (i.e. Min, Median, Max, stdev) (NameNodeInfo), > Count of decommissioned Live and Dead nodes (FSNamesystemState), > Journal Status (NodeNameInfo) >Reporter: Trevor Lorimer >Priority: Trivial > Attachments: 0001-Hadoop-namenode-JMX-metrics-update.patch > > -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-9596) Add additional attributes to JMX beans
[ https://issues.apache.org/jira/browse/HADOOP-9596?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Trevor Lorimer updated HADOOP-9596: --- Description: Currently the JMX bean returns much of the data contained on the HDFS Health webpage (dfsHealth.html). However there are several other attributes that are required to be added. I intend to add the following items to the appropriate bean in parenthesis : Started time (NameNodeInfo), Compiled info (NameNodeInfo), Jvm MaxHeap, MaxNonHeap (JvmMetrics) Node Usage stats (i.e. Min, Median, Max, stdev) (NameNodeInfo), Count of decommissioned Live and Dead nodes (FSNamesystemState), Journal Status (NodeNameInfo) > Add additional attributes to JMX beans > -- > > Key: HADOOP-9596 > URL: https://issues.apache.org/jira/browse/HADOOP-9596 > Project: Hadoop Common > Issue Type: Improvement > Components: fs >Affects Versions: 2.0.4-alpha > Environment: Currently the JMX bean returns much of the data > contained on the HDFS Health webpage (dfsHealth.html). However there are > several other attributes that are required to be added. > I intend to add the following items to the appropriate bean in parenthesis : > Started time (NameNodeInfo), > Compiled info (NameNodeInfo), > Jvm MaxHeap, MaxNonHeap (JvmMetrics) > Node Usage stats (i.e. Min, Median, Max, stdev) (NameNodeInfo), > Count of decommissioned Live and Dead nodes (FSNamesystemState), > Journal Status (NodeNameInfo) >Reporter: Trevor Lorimer >Priority: Trivial > Attachments: 0001-Hadoop-namenode-JMX-metrics-update.patch > > > Currently the JMX bean returns much of the data contained on the HDFS Health > webpage (dfsHealth.html). However there are several other attributes that are > required to be added. > I intend to add the following items to the appropriate bean in parenthesis : > Started time (NameNodeInfo), > Compiled info (NameNodeInfo), > Jvm MaxHeap, MaxNonHeap (JvmMetrics) > Node Usage stats (i.e. Min, Median, Max, stdev) (NameNodeInfo), > Count of decommissioned Live and Dead nodes (FSNamesystemState), > Journal Status (NodeNameInfo) -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-9596) Add additional attributes to JMX beans
[ https://issues.apache.org/jira/browse/HADOOP-9596?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Trevor Lorimer updated HADOOP-9596: --- Attachment: 0001-Hadoop-namenode-JMX-metrics-update.patch Patch submitted. > Add additional attributes to JMX beans > -- > > Key: HADOOP-9596 > URL: https://issues.apache.org/jira/browse/HADOOP-9596 > Project: Hadoop Common > Issue Type: Improvement > Components: fs >Affects Versions: 2.0.4-alpha > Environment: Currently the JMX bean returns much of the data > contained on the HDFS Health webpage (dfsHealth.html). However there are > several other attributes that are required to be added. > I intend to add the following items to the appropriate bean in parenthesis : > Started time (NameNodeInfo), > Compiled info (NameNodeInfo), > Jvm MaxHeap, MaxNonHeap (JvmMetrics) > Node Usage stats (i.e. Min, Median, Max, stdev) (NameNodeInfo), > Count of decommissioned Live and Dead nodes (FSNamesystemState), > Journal Status (NodeNameInfo) >Reporter: Trevor Lorimer >Priority: Trivial > Attachments: 0001-Hadoop-namenode-JMX-metrics-update.patch > > -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HADOOP-9596) Add additional attributes to JMX beans
Trevor Lorimer created HADOOP-9596: -- Summary: Add additional attributes to JMX beans Key: HADOOP-9596 URL: https://issues.apache.org/jira/browse/HADOOP-9596 Project: Hadoop Common Issue Type: Improvement Components: fs Affects Versions: 2.0.4-alpha Environment: Currently the JMX bean returns much of the data contained on the HDFS Health webpage (dfsHealth.html). However there are several other attributes that are required to be added. I intend to add the following items to the appropriate bean in parenthesis : Started time (NameNodeInfo), Compiled info (NameNodeInfo), Jvm MaxHeap, MaxNonHeap (JvmMetrics) Node Usage stats (i.e. Min, Median, Max, stdev) (NameNodeInfo), Count of decommissioned Live and Dead nodes (FSNamesystemState), Journal Status (NodeNameInfo) Reporter: Trevor Lorimer Priority: Trivial -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (HADOOP-9546) "setsid exited with exit code" message on each hadoop command
[
https://issues.apache.org/jira/browse/HADOOP-9546?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jason Lowe resolved HADOOP-9546.
Resolution: Duplicate
Fixed by HADOOP-9593.
> "setsid exited with exit code" message on each hadoop command
> -
>
> Key: HADOOP-9546
> URL: https://issues.apache.org/jira/browse/HADOOP-9546
> Project: Hadoop Common
> Issue Type: Bug
>Affects Versions: 3.0.0
>Reporter: Jason Lowe
>
> Each hadoop command is printing an INFO message about a setsid exit code.
> This is at best annoying, and it prevents the start-dfs.sh startup script
> from working properly since the INFO message is interpreted as more namenode
> hosts to connect to. start-dfs.sh uses {{hdfs getconf -namenodes}} which
> like other hadoop commands prints the extra INFO message. For example:
> {noformat}
> $ hdfs getconf -namenodes
> 2013-05-06 20:11:22,096 INFO [main] util.Shell
> (Shell.java:isSetsidSupported(311)) - setsid exited with exit code 0
> localhost
> {noformat}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-9573) Fix test-patch script to work with the enhanced PreCommit-Admin script.
[ https://issues.apache.org/jira/browse/HADOOP-9573?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Giridharan Kesavan updated HADOOP-9573: --- Attachment: hadoop-9573.patch hadoop-9573.patch addresses comments from Matt > Fix test-patch script to work with the enhanced PreCommit-Admin script. > --- > > Key: HADOOP-9573 > URL: https://issues.apache.org/jira/browse/HADOOP-9573 > Project: Hadoop Common > Issue Type: Sub-task > Components: build >Affects Versions: 1.0.3 >Reporter: Giridharan Kesavan >Assignee: Giridharan Kesavan > Attachments: > 0001-Fix-test-patch-scrit-to-work-with-the-enhanced-PreCo.patch, > hadoop-9573.patch > > > test-patch script currently take the latest available patch for a given jira > and performs the test. This jira is to enhance test-patch script to take > attachment-id of a patch as an input and perform the tests using that > attachment-id -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HADOOP-9595) Error when error message should be printed to stderr
Alexander Veit created HADOOP-9595:
--
Summary: Error when error message should be printed to stderr
Key: HADOOP-9595
URL: https://issues.apache.org/jira/browse/HADOOP-9595
Project: Hadoop Common
Issue Type: Bug
Components: bin
Affects Versions: 1.2.0
Reporter: Alexander Veit
RunJar contains the lines
{code}
if (!file.exists() || !file.isFile()) {
System.err.println("Not a valid JAR: " + file.getCanonicalPath());
System.exit(-1);
}
{code}
This code fails with
{quote}
Exception in thread "main" java.io.IOException: Invalid argument
at java.io.WinNTFileSystem.canonicalize0(Native Method)
at java.io.Win32FileSystem.canonicalize(Win32FileSystem.java:414)
at java.io.File.getCanonicalPath(File.java:589)
at org.apache.hadoop.util.RunJar.main(RunJar.java:85)
{quote}
A possible fix is
{code}
if (!file.exists() || !file.isFile()) {
System.err.println("Not a valid JAR: " + file);
System.exit(-1);
}
{code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9579) Contrib ant test target not setting the java.library.path
[ https://issues.apache.org/jira/browse/HADOOP-9579?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13665798#comment-13665798 ] Chris Nauroth commented on HADOOP-9579: --- This looks good to me. I'll take it for a test run and comment again with the test results as soon as I get some resources for a VM. > Contrib ant test target not setting the java.library.path > - > > Key: HADOOP-9579 > URL: https://issues.apache.org/jira/browse/HADOOP-9579 > Project: Hadoop Common > Issue Type: Bug >Affects Versions: 1-win >Reporter: Ivan Mitic >Assignee: Ivan Mitic > Attachments: HADOOP-9579.branch-1-win.patch > > > build-contrib.xml does not set java.library.path causing tests to run without > the hadoop native library. This is a bigger problem on Windows as having the > native library is required. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9438) LocalFileContext does not throw an exception on mkdir for already existing directory
[
https://issues.apache.org/jira/browse/HADOOP-9438?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13665767#comment-13665767
]
Ivan Mitic commented on HADOOP-9438:
Bq. While looking at this, I am starting to suspect that
ParentNotDirectoryException doesn't get thrown for a lot of the FileSystem
implementations.
Hi Steve. This Jira seems pretty well defined at this point so if there is a
problem with ParentNotDirectoryException my preference would be to address that
separately.
Bq. Would we be able to get away with changing the exception that is thrown to
this specific error code? As long as things were expecting any IOException
this should still work.
Not sure I fully understand this part.
> LocalFileContext does not throw an exception on mkdir for already existing
> directory
>
>
> Key: HADOOP-9438
> URL: https://issues.apache.org/jira/browse/HADOOP-9438
> Project: Hadoop Common
> Issue Type: Bug
>Affects Versions: 2.0.3-alpha
>Reporter: Robert Joseph Evans
>Priority: Critical
> Attachments: HADOOP-9438.20130501.1.patch,
> HADOOP-9438.20130521.1.patch, HADOOP-9438.patch, HADOOP-9438.patch
>
>
> according to
> http://hadoop.apache.org/docs/current/api/org/apache/hadoop/fs/FileContext.html#mkdir%28org.apache.hadoop.fs.Path,%20org.apache.hadoop.fs.permission.FsPermission,%20boolean%29
> should throw a FileAlreadyExistsException if the directory already exists.
> I tested this and
> {code}
> FileContext lfc = FileContext.getLocalFSFileContext(new Configuration());
> Path p = new Path("/tmp/bobby.12345");
> FsPermission cachePerms = new FsPermission((short) 0755);
> lfc.mkdir(p, cachePerms, false);
> lfc.mkdir(p, cachePerms, false);
> {code}
> never throws an exception.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9590) Move to JDK7 improved APIs for file operations when available
[ https://issues.apache.org/jira/browse/HADOOP-9590?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13665740#comment-13665740 ] Ivan Mitic commented on HADOOP-9590: Thanks Steve for responding. You’re proposal "build and run on Java7 everywhere" first makes sense to me. The intent of this Jira was solely to start documenting the benefits from switching to JDK7 w.r.t. local file system operations (so that we don't forget). I was not planning on working on this Jira anytime soon. Bq. Maybe we should create two meta-JIRA's "build and run on Java7 everywhere" and only after that "adopt java-7 specific features". +1. First step for "build and run on Java7 everywhere" would be to resolve things like HADOOP-9350 and get all unittests to pass. Not sure where we are w.r.t. pass rates on JDK7 (I know the random test case execution order introduced a lot of churn). Thanks Chris for adding on to the list! > Move to JDK7 improved APIs for file operations when available > - > > Key: HADOOP-9590 > URL: https://issues.apache.org/jira/browse/HADOOP-9590 > Project: Hadoop Common > Issue Type: Improvement >Reporter: Ivan Mitic > > JDK6 does not have a complete support for local file system file operations. > Specifically: > - There is no symlink/hardlink APIs what forced Hadoop to defer to shell > based tooling > - No error information returned when File#mkdir/mkdirs or File#renameTo > fails, making it unnecessary hard to troubleshoot some issues > - File#canRead/canWrite/canExecute do not perform any access checks on > Windows making APIs inconsistent with the Unix behavior > - File#setReadable/setWritable/setExecutable do not change access rights on > Windows making APIs inconsistent with the Unix behavior > - File#length does not work as expected on symlinks on Windows > - File#renameTo does not work as expected on symlinks on Windows > All above resulted in Hadoop community having to fill in the gaps by > providing equivalent native implementations or applying workarounds. > JDK7 addressed (as far as I know) all (or most) of the above problems, either > thru the newly introduced > [Files|http://docs.oracle.com/javase/7/docs/api/java/nio/file/Files.html] > class or thru bug fixes. > This is a tracking Jira to revisit above mediations once JDK7 becomes the > supported platform by the Hadoop community. This work would allow significant > portion of the native platform-dependent code to be replaced with Java > equivalents what is goodness w.r.t. Hadoop cross-platform support. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9594) Update apache commons math dependency
[
https://issues.apache.org/jira/browse/HADOOP-9594?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13665707#comment-13665707
]
Hadoop QA commented on HADOOP-9594:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12584578/HADOOP-9594.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:red}-1 tests included{color}. The patch doesn't appear to include
any new or modified tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in
hadoop-common-project/hadoop-common.
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results:
https://builds.apache.org/job/PreCommit-HADOOP-Build/2562//testReport/
Console output:
https://builds.apache.org/job/PreCommit-HADOOP-Build/2562//console
This message is automatically generated.
> Update apache commons math dependency
> -
>
> Key: HADOOP-9594
> URL: https://issues.apache.org/jira/browse/HADOOP-9594
> Project: Hadoop Common
> Issue Type: Improvement
> Components: build
>Affects Versions: 3.0.0, 2.0.5-beta
>Reporter: Timothy St. Clair
> Labels: newbie
> Fix For: 2.0.5-beta
>
> Attachments: HADOOP-9594.patch
>
>
> Current dependency is against 2.1 which has been deprecated, and numerous
> bugz have been fixed since then. See
> http://commons.apache.org/proper/commons-math/changes-report.html for
> details.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-9594) Update apache commons math dependency
[ https://issues.apache.org/jira/browse/HADOOP-9594?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Timothy St. Clair updated HADOOP-9594: -- Fix Version/s: 2.0.5-beta Status: Patch Available (was: Open) > Update apache commons math dependency > - > > Key: HADOOP-9594 > URL: https://issues.apache.org/jira/browse/HADOOP-9594 > Project: Hadoop Common > Issue Type: Improvement > Components: build >Affects Versions: 3.0.0, 2.0.5-beta >Reporter: Timothy St. Clair > Labels: newbie > Fix For: 2.0.5-beta > > Attachments: HADOOP-9594.patch > > > Current dependency is against 2.1 which has been deprecated, and numerous > bugz have been fixed since then. See > http://commons.apache.org/proper/commons-math/changes-report.html for > details. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-9594) Update apache commons math dependency
[ https://issues.apache.org/jira/browse/HADOOP-9594?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Timothy St. Clair updated HADOOP-9594: -- Attachment: HADOOP-9594.patch Simple modification to support math commons 3.1.1 > Update apache commons math dependency > - > > Key: HADOOP-9594 > URL: https://issues.apache.org/jira/browse/HADOOP-9594 > Project: Hadoop Common > Issue Type: Improvement > Components: build >Affects Versions: 3.0.0, 2.0.5-beta >Reporter: Timothy St. Clair > Labels: newbie > Attachments: HADOOP-9594.patch > > > Current dependency is against 2.1 which has been deprecated, and numerous > bugz have been fixed since then. See > http://commons.apache.org/proper/commons-math/changes-report.html for > details. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (HADOOP-8562) Enhancements to support Hadoop on Windows Server and Windows Azure environments
[ https://issues.apache.org/jira/browse/HADOOP-8562?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Suresh Srinivas resolved HADOOP-8562. - Resolution: Fixed Fix Version/s: (was: 3.0.0) 2.0.5-beta I merge this patch into branch-2. Thank you every one for testing. Can you folks run another quick test to verify things are merged correctly? > Enhancements to support Hadoop on Windows Server and Windows Azure > environments > --- > > Key: HADOOP-8562 > URL: https://issues.apache.org/jira/browse/HADOOP-8562 > Project: Hadoop Common > Issue Type: New Feature >Affects Versions: 3.0.0 >Reporter: Bikas Saha >Assignee: Bikas Saha > Fix For: 2.0.5-beta > > Attachments: branch-2.merge.patch, branch-2.merge.patch, > branch-trunk-win.min-notest.patch, branch-trunk-win-min.patch, > branch-trunk-win.min.patch, branch-trunk-win.patch, branch-trunk-win.patch, > branch-trunk-win.patch, branch-trunk-win.patch, branch-trunk-win.patch, > branch-trunk-win.patch, branch-trunk-win.patch, branch-trunk-win.patch, > branch-trunk-win.patch, branch-trunk-win.patch, branch-trunk-win.patch, > test-untar.tar, test-untar.tgz > > > This JIRA tracks the work that needs to be done on trunk to enable Hadoop to > run on Windows Server and Azure environments. This incorporates porting > relevant work from the similar effort on branch 1 tracked via HADOOP-8079. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Reopened] (HADOOP-8562) Enhancements to support Hadoop on Windows Server and Windows Azure environments
[ https://issues.apache.org/jira/browse/HADOOP-8562?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Suresh Srinivas reopened HADOOP-8562: - Reopening the issue for merging. > Enhancements to support Hadoop on Windows Server and Windows Azure > environments > --- > > Key: HADOOP-8562 > URL: https://issues.apache.org/jira/browse/HADOOP-8562 > Project: Hadoop Common > Issue Type: New Feature >Affects Versions: 3.0.0 >Reporter: Bikas Saha >Assignee: Bikas Saha > Fix For: 3.0.0 > > Attachments: branch-2.merge.patch, branch-2.merge.patch, > branch-trunk-win.min-notest.patch, branch-trunk-win-min.patch, > branch-trunk-win.min.patch, branch-trunk-win.patch, branch-trunk-win.patch, > branch-trunk-win.patch, branch-trunk-win.patch, branch-trunk-win.patch, > branch-trunk-win.patch, branch-trunk-win.patch, branch-trunk-win.patch, > branch-trunk-win.patch, branch-trunk-win.patch, branch-trunk-win.patch, > test-untar.tar, test-untar.tgz > > > This JIRA tracks the work that needs to be done on trunk to enable Hadoop to > run on Windows Server and Azure environments. This incorporates porting > relevant work from the similar effort on branch 1 tracked via HADOOP-8079. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9562) Create REST interface for HDFS health data
[ https://issues.apache.org/jira/browse/HADOOP-9562?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13665631#comment-13665631 ] Trevor Lorimer commented on HADOOP-9562: Hi, Thanks Philip about 75% of the information can be obtained from JMX. I only need to make these additions to pre-existing JMX interfaces (I will link this to a new JIRA): Started time, Compiled info, Jvm MaxHeap, MaxNonHeap Node Usage stats (i.e. Min, Median, Max, stdev), Number of decommissioned Live and Dead nodes, Journal Status The only information I have had difficulty obtaining is the status header (e.g. NameNode 'localhost:8020' (active)). This information is contained within the NameNode object which does not seem to be exposed through JMX and cannot be referenced from FSNamesystem. > Create REST interface for HDFS health data > -- > > Key: HADOOP-9562 > URL: https://issues.apache.org/jira/browse/HADOOP-9562 > Project: Hadoop Common > Issue Type: Improvement > Components: fs >Affects Versions: 2.0.4-alpha >Reporter: Trevor Lorimer >Priority: Minor > Attachments: 0001-HAD-162-Final-revision-refactor.patch > > > The HDFS health screen (dfshealth.jsp) displays basic Version, Security and > Health information concerning the NameNode, currently this information is > accessible from classes in the org.apache.hadoop,hdfs.server.namenode package > and cannot be accessed outside the NameNode. This becomes prevalent if the > data is required to be displayed using a new user interface. > The proposal is to create a REST interface to expose all the information > displayed on dfshealth.jsp using GET methods. Wrapper classes will be created > to serve the data to the REST root resource within the hadoop-hdfs project. > This will enable the HDFS health screen information to be accessed remotely. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HADOOP-9594) Update apache commons math dependency
Timothy St. Clair created HADOOP-9594: - Summary: Update apache commons math dependency Key: HADOOP-9594 URL: https://issues.apache.org/jira/browse/HADOOP-9594 Project: Hadoop Common Issue Type: Improvement Components: build Affects Versions: 3.0.0, 2.0.5-beta Reporter: Timothy St. Clair Current dependency is against 2.1 which has been deprecated, and numerous bugz have been fixed since then. See http://commons.apache.org/proper/commons-math/changes-report.html for details. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9593) stack trace printed at ERROR for all yarn clients without hadoop.home set
[
https://issues.apache.org/jira/browse/HADOOP-9593?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13665609#comment-13665609
]
Chris Nauroth commented on HADOOP-9593:
---
{quote}
OK, applied to trunk; it wouldn't take on branch-2. Should I create a new JIRA
for that?
{quote}
I'd say it's not necessary. We're pretty close to merging HADOOP-8562 from
trunk to branch-2, and I expect your trunk patch will apply cleanly after that.
> stack trace printed at ERROR for all yarn clients without hadoop.home set
> -
>
> Key: HADOOP-9593
> URL: https://issues.apache.org/jira/browse/HADOOP-9593
> Project: Hadoop Common
> Issue Type: Bug
> Components: util
>Affects Versions: 3.0.0
>Reporter: Steve Loughran
> Attachments: HADOOP-9593-001.patch, HADOOP-9593-002.patch
>
>
> This is the problem of HADOOP-9482 now showing up in a different application
> -one whose log4j settings haven't turned off all Shell logging.
> Unless you do that, all yarn clients will have a stack trace at error in
> their logs, which is generating false alarms and is utterly pointless. Why
> does this merit a stack trace? Why log it at error? It's not an error for a
> client app to not have these values set as long as they have the relevant
> JARs on their classpath. And if they don't, they'll get some classpath error
> instead
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9593) stack trace printed at ERROR for all yarn clients without hadoop.home set
[ https://issues.apache.org/jira/browse/HADOOP-9593?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13665607#comment-13665607 ] Hudson commented on HADOOP-9593: Integrated in Hadoop-trunk-Commit #3783 (See [https://builds.apache.org/job/Hadoop-trunk-Commit/3783/]) HADOOP-9593 stack trace printed at ERROR for all yarn clients without hadoop.home set (Revision 1485842) Result = SUCCESS stevel : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1485842 Files : * /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt * /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/Shell.java > stack trace printed at ERROR for all yarn clients without hadoop.home set > - > > Key: HADOOP-9593 > URL: https://issues.apache.org/jira/browse/HADOOP-9593 > Project: Hadoop Common > Issue Type: Bug > Components: util >Affects Versions: 3.0.0 >Reporter: Steve Loughran > Attachments: HADOOP-9593-001.patch, HADOOP-9593-002.patch > > > This is the problem of HADOOP-9482 now showing up in a different application > -one whose log4j settings haven't turned off all Shell logging. > Unless you do that, all yarn clients will have a stack trace at error in > their logs, which is generating false alarms and is utterly pointless. Why > does this merit a stack trace? Why log it at error? It's not an error for a > client app to not have these values set as long as they have the relevant > JARs on their classpath. And if they don't, they'll get some classpath error > instead -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9593) stack trace printed at ERROR for all yarn clients without hadoop.home set
[ https://issues.apache.org/jira/browse/HADOOP-9593?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13665602#comment-13665602 ] Steve Loughran commented on HADOOP-9593: OK, applied to trunk; it wouldn't take on branch-2. Should I create a new JIRA for that? > stack trace printed at ERROR for all yarn clients without hadoop.home set > - > > Key: HADOOP-9593 > URL: https://issues.apache.org/jira/browse/HADOOP-9593 > Project: Hadoop Common > Issue Type: Bug > Components: util >Affects Versions: 3.0.0 >Reporter: Steve Loughran > Attachments: HADOOP-9593-001.patch, HADOOP-9593-002.patch > > > This is the problem of HADOOP-9482 now showing up in a different application > -one whose log4j settings haven't turned off all Shell logging. > Unless you do that, all yarn clients will have a stack trace at error in > their logs, which is generating false alarms and is utterly pointless. Why > does this merit a stack trace? Why log it at error? It's not an error for a > client app to not have these values set as long as they have the relevant > JARs on their classpath. And if they don't, they'll get some classpath error > instead -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8562) Enhancements to support Hadoop on Windows Server and Windows Azure environments
[ https://issues.apache.org/jira/browse/HADOOP-8562?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13665565#comment-13665565 ] Chris Nauroth commented on HADOOP-8562: --- Arpit just let me know that the goal here is to merge the HADOOP-8562 changes, and since MAPREDUCE-4987 was not part of HADOOP-8562 (it was committed later), it is not included in the merge patch. Remaining patches like this will get merged later. Considering that, I am +1 for this merge patch. Thanks, everyone. > Enhancements to support Hadoop on Windows Server and Windows Azure > environments > --- > > Key: HADOOP-8562 > URL: https://issues.apache.org/jira/browse/HADOOP-8562 > Project: Hadoop Common > Issue Type: New Feature >Affects Versions: 3.0.0 >Reporter: Bikas Saha >Assignee: Bikas Saha > Fix For: 3.0.0 > > Attachments: branch-2.merge.patch, branch-2.merge.patch, > branch-trunk-win.min-notest.patch, branch-trunk-win-min.patch, > branch-trunk-win.min.patch, branch-trunk-win.patch, branch-trunk-win.patch, > branch-trunk-win.patch, branch-trunk-win.patch, branch-trunk-win.patch, > branch-trunk-win.patch, branch-trunk-win.patch, branch-trunk-win.patch, > branch-trunk-win.patch, branch-trunk-win.patch, branch-trunk-win.patch, > test-untar.tar, test-untar.tgz > > > This JIRA tracks the work that needs to be done on trunk to enable Hadoop to > run on Windows Server and Azure environments. This incorporates porting > relevant work from the similar effort on branch 1 tracked via HADOOP-8079. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8562) Enhancements to support Hadoop on Windows Server and Windows Azure environments
[ https://issues.apache.org/jira/browse/HADOOP-8562?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13665558#comment-13665558 ] Chris Nauroth commented on HADOOP-8562: --- I think the merge patch is still missing MAPREDUCE-4987, which fixed handling of symlinks in the distributed cache on Windows. HADOOP-9372 includes some of the timeout tuning, but it doesn't include the logic changes. For example, here are links to some trunk code for ContainerLaunch and TestMRJobs. I don't see this code in the merge patch. https://github.com/apache/hadoop-common/blob/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/launcher/ContainerLaunch.java#L610 https://github.com/apache/hadoop-common/blob/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapreduce/v2/TestMRJobs.java#L482 > Enhancements to support Hadoop on Windows Server and Windows Azure > environments > --- > > Key: HADOOP-8562 > URL: https://issues.apache.org/jira/browse/HADOOP-8562 > Project: Hadoop Common > Issue Type: New Feature >Affects Versions: 3.0.0 >Reporter: Bikas Saha >Assignee: Bikas Saha > Fix For: 3.0.0 > > Attachments: branch-2.merge.patch, branch-2.merge.patch, > branch-trunk-win.min-notest.patch, branch-trunk-win-min.patch, > branch-trunk-win.min.patch, branch-trunk-win.patch, branch-trunk-win.patch, > branch-trunk-win.patch, branch-trunk-win.patch, branch-trunk-win.patch, > branch-trunk-win.patch, branch-trunk-win.patch, branch-trunk-win.patch, > branch-trunk-win.patch, branch-trunk-win.patch, branch-trunk-win.patch, > test-untar.tar, test-untar.tgz > > > This JIRA tracks the work that needs to be done on trunk to enable Hadoop to > run on Windows Server and Azure environments. This incorporates porting > relevant work from the similar effort on branch 1 tracked via HADOOP-8079. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8562) Enhancements to support Hadoop on Windows Server and Windows Azure environments
[ https://issues.apache.org/jira/browse/HADOOP-8562?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13665554#comment-13665554 ] Vinod Kumar Vavilapalli commented on HADOOP-8562: - I meant timeout issues with the tests, and randomly failing tests like TestEncryptedShuffle. > Enhancements to support Hadoop on Windows Server and Windows Azure > environments > --- > > Key: HADOOP-8562 > URL: https://issues.apache.org/jira/browse/HADOOP-8562 > Project: Hadoop Common > Issue Type: New Feature >Affects Versions: 3.0.0 >Reporter: Bikas Saha >Assignee: Bikas Saha > Fix For: 3.0.0 > > Attachments: branch-2.merge.patch, branch-2.merge.patch, > branch-trunk-win.min-notest.patch, branch-trunk-win-min.patch, > branch-trunk-win.min.patch, branch-trunk-win.patch, branch-trunk-win.patch, > branch-trunk-win.patch, branch-trunk-win.patch, branch-trunk-win.patch, > branch-trunk-win.patch, branch-trunk-win.patch, branch-trunk-win.patch, > branch-trunk-win.patch, branch-trunk-win.patch, branch-trunk-win.patch, > test-untar.tar, test-untar.tgz > > > This JIRA tracks the work that needs to be done on trunk to enable Hadoop to > run on Windows Server and Azure environments. This incorporates porting > relevant work from the similar effort on branch 1 tracked via HADOOP-8079. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8562) Enhancements to support Hadoop on Windows Server and Windows Azure environments
[ https://issues.apache.org/jira/browse/HADOOP-8562?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13665552#comment-13665552 ] Vinod Kumar Vavilapalli commented on HADOOP-8562: - Sure, I'm holding off commits on YARN/MR. The test-failures can be fixed post merge. > Enhancements to support Hadoop on Windows Server and Windows Azure > environments > --- > > Key: HADOOP-8562 > URL: https://issues.apache.org/jira/browse/HADOOP-8562 > Project: Hadoop Common > Issue Type: New Feature >Affects Versions: 3.0.0 >Reporter: Bikas Saha >Assignee: Bikas Saha > Fix For: 3.0.0 > > Attachments: branch-2.merge.patch, branch-2.merge.patch, > branch-trunk-win.min-notest.patch, branch-trunk-win-min.patch, > branch-trunk-win.min.patch, branch-trunk-win.patch, branch-trunk-win.patch, > branch-trunk-win.patch, branch-trunk-win.patch, branch-trunk-win.patch, > branch-trunk-win.patch, branch-trunk-win.patch, branch-trunk-win.patch, > branch-trunk-win.patch, branch-trunk-win.patch, branch-trunk-win.patch, > test-untar.tar, test-untar.tgz > > > This JIRA tracks the work that needs to be done on trunk to enable Hadoop to > run on Windows Server and Azure environments. This incorporates porting > relevant work from the similar effort on branch 1 tracked via HADOOP-8079. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9593) stack trace printed at ERROR for all yarn clients without hadoop.home set
[
https://issues.apache.org/jira/browse/HADOOP-9593?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13665548#comment-13665548
]
Hadoop QA commented on HADOOP-9593:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12584556/HADOOP-9593-002.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:red}-1 tests included{color}. The patch doesn't appear to include
any new or modified tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in
hadoop-common-project/hadoop-common.
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results:
https://builds.apache.org/job/PreCommit-HADOOP-Build/2561//testReport/
Console output:
https://builds.apache.org/job/PreCommit-HADOOP-Build/2561//console
This message is automatically generated.
> stack trace printed at ERROR for all yarn clients without hadoop.home set
> -
>
> Key: HADOOP-9593
> URL: https://issues.apache.org/jira/browse/HADOOP-9593
> Project: Hadoop Common
> Issue Type: Bug
> Components: util
>Affects Versions: 3.0.0
>Reporter: Steve Loughran
> Attachments: HADOOP-9593-001.patch, HADOOP-9593-002.patch
>
>
> This is the problem of HADOOP-9482 now showing up in a different application
> -one whose log4j settings haven't turned off all Shell logging.
> Unless you do that, all yarn clients will have a stack trace at error in
> their logs, which is generating false alarms and is utterly pointless. Why
> does this merit a stack trace? Why log it at error? It's not an error for a
> client app to not have these values set as long as they have the relevant
> JARs on their classpath. And if they don't, they'll get some classpath error
> instead
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8562) Enhancements to support Hadoop on Windows Server and Windows Azure environments
[ https://issues.apache.org/jira/browse/HADOOP-8562?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13665544#comment-13665544 ] Arpit Agarwal commented on HADOOP-8562: --- Sounds good to me. Thanks Suresh. > Enhancements to support Hadoop on Windows Server and Windows Azure > environments > --- > > Key: HADOOP-8562 > URL: https://issues.apache.org/jira/browse/HADOOP-8562 > Project: Hadoop Common > Issue Type: New Feature >Affects Versions: 3.0.0 >Reporter: Bikas Saha >Assignee: Bikas Saha > Fix For: 3.0.0 > > Attachments: branch-2.merge.patch, branch-2.merge.patch, > branch-trunk-win.min-notest.patch, branch-trunk-win-min.patch, > branch-trunk-win.min.patch, branch-trunk-win.patch, branch-trunk-win.patch, > branch-trunk-win.patch, branch-trunk-win.patch, branch-trunk-win.patch, > branch-trunk-win.patch, branch-trunk-win.patch, branch-trunk-win.patch, > branch-trunk-win.patch, branch-trunk-win.patch, branch-trunk-win.patch, > test-untar.tar, test-untar.tgz > > > This JIRA tracks the work that needs to be done on trunk to enable Hadoop to > run on Windows Server and Azure environments. This incorporates porting > relevant work from the similar effort on branch 1 tracked via HADOOP-8079. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8562) Enhancements to support Hadoop on Windows Server and Windows Azure environments
[ https://issues.apache.org/jira/browse/HADOOP-8562?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13665539#comment-13665539 ] Suresh Srinivas commented on HADOOP-8562: - Thanks guys for running the tests. I plan to merge this change with + hadoop-9372. The other related jiras I will merge one at a time. [~vinodkv], [~arpitagarwal] and [~cnauroth], let me know of you guys are okay. > Enhancements to support Hadoop on Windows Server and Windows Azure > environments > --- > > Key: HADOOP-8562 > URL: https://issues.apache.org/jira/browse/HADOOP-8562 > Project: Hadoop Common > Issue Type: New Feature >Affects Versions: 3.0.0 >Reporter: Bikas Saha >Assignee: Bikas Saha > Fix For: 3.0.0 > > Attachments: branch-2.merge.patch, branch-2.merge.patch, > branch-trunk-win.min-notest.patch, branch-trunk-win-min.patch, > branch-trunk-win.min.patch, branch-trunk-win.patch, branch-trunk-win.patch, > branch-trunk-win.patch, branch-trunk-win.patch, branch-trunk-win.patch, > branch-trunk-win.patch, branch-trunk-win.patch, branch-trunk-win.patch, > branch-trunk-win.patch, branch-trunk-win.patch, branch-trunk-win.patch, > test-untar.tar, test-untar.tgz > > > This JIRA tracks the work that needs to be done on trunk to enable Hadoop to > run on Windows Server and Azure environments. This incorporates porting > relevant work from the similar effort on branch 1 tracked via HADOOP-8079. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8562) Enhancements to support Hadoop on Windows Server and Windows Azure environments
[ https://issues.apache.org/jira/browse/HADOOP-8562?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13665527#comment-13665527 ] Arpit Agarwal commented on HADOOP-8562: --- Thanks for the verification Chris! > Enhancements to support Hadoop on Windows Server and Windows Azure > environments > --- > > Key: HADOOP-8562 > URL: https://issues.apache.org/jira/browse/HADOOP-8562 > Project: Hadoop Common > Issue Type: New Feature >Affects Versions: 3.0.0 >Reporter: Bikas Saha >Assignee: Bikas Saha > Fix For: 3.0.0 > > Attachments: branch-2.merge.patch, branch-2.merge.patch, > branch-trunk-win.min-notest.patch, branch-trunk-win-min.patch, > branch-trunk-win.min.patch, branch-trunk-win.patch, branch-trunk-win.patch, > branch-trunk-win.patch, branch-trunk-win.patch, branch-trunk-win.patch, > branch-trunk-win.patch, branch-trunk-win.patch, branch-trunk-win.patch, > branch-trunk-win.patch, branch-trunk-win.patch, branch-trunk-win.patch, > test-untar.tar, test-untar.tgz > > > This JIRA tracks the work that needs to be done on trunk to enable Hadoop to > run on Windows Server and Azure environments. This incorporates porting > relevant work from the similar effort on branch 1 tracked via HADOOP-8079. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8562) Enhancements to support Hadoop on Windows Server and Windows Azure environments
[ https://issues.apache.org/jira/browse/HADOOP-8562?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13665520#comment-13665520 ] Chris Nauroth commented on HADOOP-8562: --- Yes, there were only 2 failures for me. TestEncryptedShuffle appears to be flaky, approximately 1 failure in every 5 test runs for me. This can occur on current branch-2, so it's unrelated to the merge patch. TestMRJobsWithHistoryService has a timeout value that is too low for a slower dev environment, like a VM. The current timeout is 30s. I typically see it complete in just under 30s, and sometimes it creeps over that and times out. I think we need to increase the timeout to accommodate slow VMs. Again, this is unrelated to the merge patch, because I can repro on branch-2 without it. > Enhancements to support Hadoop on Windows Server and Windows Azure > environments > --- > > Key: HADOOP-8562 > URL: https://issues.apache.org/jira/browse/HADOOP-8562 > Project: Hadoop Common > Issue Type: New Feature >Affects Versions: 3.0.0 >Reporter: Bikas Saha >Assignee: Bikas Saha > Fix For: 3.0.0 > > Attachments: branch-2.merge.patch, branch-2.merge.patch, > branch-trunk-win.min-notest.patch, branch-trunk-win-min.patch, > branch-trunk-win.min.patch, branch-trunk-win.patch, branch-trunk-win.patch, > branch-trunk-win.patch, branch-trunk-win.patch, branch-trunk-win.patch, > branch-trunk-win.patch, branch-trunk-win.patch, branch-trunk-win.patch, > branch-trunk-win.patch, branch-trunk-win.patch, branch-trunk-win.patch, > test-untar.tar, test-untar.tgz > > > This JIRA tracks the work that needs to be done on trunk to enable Hadoop to > run on Windows Server and Azure environments. This incorporates porting > relevant work from the similar effort on branch 1 tracked via HADOOP-8079. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9593) stack trace printed at ERROR for all yarn clients without hadoop.home set
[
https://issues.apache.org/jira/browse/HADOOP-9593?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13665511#comment-13665511
]
Chris Nauroth commented on HADOOP-9593:
---
+1 for the new patch. Thanks for correcting that, Steve.
{quote}
-1 tests included. The patch doesn't appear to include any new or modified
tests.
Please justify why no new tests are needed for this patch.
Also please list what manual steps were performed to verify this patch.
{quote}
I believe no new tests is justified since this is just tuning log levels.
> stack trace printed at ERROR for all yarn clients without hadoop.home set
> -
>
> Key: HADOOP-9593
> URL: https://issues.apache.org/jira/browse/HADOOP-9593
> Project: Hadoop Common
> Issue Type: Bug
> Components: util
>Affects Versions: 3.0.0
>Reporter: Steve Loughran
> Attachments: HADOOP-9593-001.patch, HADOOP-9593-002.patch
>
>
> This is the problem of HADOOP-9482 now showing up in a different application
> -one whose log4j settings haven't turned off all Shell logging.
> Unless you do that, all yarn clients will have a stack trace at error in
> their logs, which is generating false alarms and is utterly pointless. Why
> does this merit a stack trace? Why log it at error? It's not an error for a
> client app to not have these values set as long as they have the relevant
> JARs on their classpath. And if they don't, they'll get some classpath error
> instead
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8562) Enhancements to support Hadoop on Windows Server and Windows Azure environments
[ https://issues.apache.org/jira/browse/HADOOP-8562?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13665509#comment-13665509 ] Arpit Agarwal commented on HADOOP-8562: --- List of failures with the patch: # TestBinaryTokenFile # TestDistributedShell # TestEncryptedShuffle # TestMRJobsWithHistoryService # TestNoDefaultsJobConf # TestResources # TestSymLink # TestWebHdfsTimeouts # TestYarnVersionInfo Some of these are likely to be timeouts since I am running an underpowered VM. Chris just mentioned he can only reproduce two of these. > Enhancements to support Hadoop on Windows Server and Windows Azure > environments > --- > > Key: HADOOP-8562 > URL: https://issues.apache.org/jira/browse/HADOOP-8562 > Project: Hadoop Common > Issue Type: New Feature >Affects Versions: 3.0.0 >Reporter: Bikas Saha >Assignee: Bikas Saha > Fix For: 3.0.0 > > Attachments: branch-2.merge.patch, branch-2.merge.patch, > branch-trunk-win.min-notest.patch, branch-trunk-win-min.patch, > branch-trunk-win.min.patch, branch-trunk-win.patch, branch-trunk-win.patch, > branch-trunk-win.patch, branch-trunk-win.patch, branch-trunk-win.patch, > branch-trunk-win.patch, branch-trunk-win.patch, branch-trunk-win.patch, > branch-trunk-win.patch, branch-trunk-win.patch, branch-trunk-win.patch, > test-untar.tar, test-untar.tgz > > > This JIRA tracks the work that needs to be done on trunk to enable Hadoop to > run on Windows Server and Azure environments. This incorporates porting > relevant work from the similar effort on branch 1 tracked via HADOOP-8079. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-9593) stack trace printed at ERROR for all yarn clients without hadoop.home set
[ https://issues.apache.org/jira/browse/HADOOP-9593?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated HADOOP-9593: --- Status: Patch Available (was: Open) > stack trace printed at ERROR for all yarn clients without hadoop.home set > - > > Key: HADOOP-9593 > URL: https://issues.apache.org/jira/browse/HADOOP-9593 > Project: Hadoop Common > Issue Type: Bug > Components: util >Affects Versions: 3.0.0 >Reporter: Steve Loughran > Attachments: HADOOP-9593-001.patch, HADOOP-9593-002.patch > > > This is the problem of HADOOP-9482 now showing up in a different application > -one whose log4j settings haven't turned off all Shell logging. > Unless you do that, all yarn clients will have a stack trace at error in > their logs, which is generating false alarms and is utterly pointless. Why > does this merit a stack trace? Why log it at error? It's not an error for a > client app to not have these values set as long as they have the relevant > JARs on their classpath. And if they don't, they'll get some classpath error > instead -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9593) stack trace printed at ERROR for all yarn clients without hadoop.home set
[ https://issues.apache.org/jira/browse/HADOOP-9593?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13665501#comment-13665501 ] Steve Loughran commented on HADOOP-9593: Chris -you're right. new patch to follow > stack trace printed at ERROR for all yarn clients without hadoop.home set > - > > Key: HADOOP-9593 > URL: https://issues.apache.org/jira/browse/HADOOP-9593 > Project: Hadoop Common > Issue Type: Bug > Components: util >Affects Versions: 3.0.0 >Reporter: Steve Loughran > Attachments: HADOOP-9593-001.patch, HADOOP-9593-002.patch > > > This is the problem of HADOOP-9482 now showing up in a different application > -one whose log4j settings haven't turned off all Shell logging. > Unless you do that, all yarn clients will have a stack trace at error in > their logs, which is generating false alarms and is utterly pointless. Why > does this merit a stack trace? Why log it at error? It's not an error for a > client app to not have these values set as long as they have the relevant > JARs on their classpath. And if they don't, they'll get some classpath error > instead -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-9593) stack trace printed at ERROR for all yarn clients without hadoop.home set
[ https://issues.apache.org/jira/browse/HADOOP-9593?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated HADOOP-9593: --- Attachment: HADOOP-9593-002.patch now goes (null executor) > stack trace printed at ERROR for all yarn clients without hadoop.home set > - > > Key: HADOOP-9593 > URL: https://issues.apache.org/jira/browse/HADOOP-9593 > Project: Hadoop Common > Issue Type: Bug > Components: util >Affects Versions: 3.0.0 >Reporter: Steve Loughran > Attachments: HADOOP-9593-001.patch, HADOOP-9593-002.patch > > > This is the problem of HADOOP-9482 now showing up in a different application > -one whose log4j settings haven't turned off all Shell logging. > Unless you do that, all yarn clients will have a stack trace at error in > their logs, which is generating false alarms and is utterly pointless. Why > does this merit a stack trace? Why log it at error? It's not an error for a > client app to not have these values set as long as they have the relevant > JARs on their classpath. And if they don't, they'll get some classpath error > instead -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9593) stack trace printed at ERROR for all yarn clients without hadoop.home set
[
https://issues.apache.org/jira/browse/HADOOP-9593?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13665503#comment-13665503
]
Hadoop QA commented on HADOOP-9593:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12584539/HADOOP-9593-001.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:red}-1 tests included{color}. The patch doesn't appear to include
any new or modified tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in
hadoop-common-project/hadoop-common.
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results:
https://builds.apache.org/job/PreCommit-HADOOP-Build/2560//testReport/
Console output:
https://builds.apache.org/job/PreCommit-HADOOP-Build/2560//console
This message is automatically generated.
> stack trace printed at ERROR for all yarn clients without hadoop.home set
> -
>
> Key: HADOOP-9593
> URL: https://issues.apache.org/jira/browse/HADOOP-9593
> Project: Hadoop Common
> Issue Type: Bug
> Components: util
>Affects Versions: 3.0.0
>Reporter: Steve Loughran
> Attachments: HADOOP-9593-001.patch, HADOOP-9593-002.patch
>
>
> This is the problem of HADOOP-9482 now showing up in a different application
> -one whose log4j settings haven't turned off all Shell logging.
> Unless you do that, all yarn clients will have a stack trace at error in
> their logs, which is generating false alarms and is utterly pointless. Why
> does this merit a stack trace? Why log it at error? It's not an error for a
> client app to not have these values set as long as they have the relevant
> JARs on their classpath. And if they don't, they'll get some classpath error
> instead
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-9593) stack trace printed at ERROR for all yarn clients without hadoop.home set
[ https://issues.apache.org/jira/browse/HADOOP-9593?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated HADOOP-9593: --- Status: Open (was: Patch Available) > stack trace printed at ERROR for all yarn clients without hadoop.home set > - > > Key: HADOOP-9593 > URL: https://issues.apache.org/jira/browse/HADOOP-9593 > Project: Hadoop Common > Issue Type: Bug > Components: util >Affects Versions: 3.0.0 >Reporter: Steve Loughran > Attachments: HADOOP-9593-001.patch, HADOOP-9593-002.patch > > > This is the problem of HADOOP-9482 now showing up in a different application > -one whose log4j settings haven't turned off all Shell logging. > Unless you do that, all yarn clients will have a stack trace at error in > their logs, which is generating false alarms and is utterly pointless. Why > does this merit a stack trace? Why log it at error? It's not an error for a > client app to not have these values set as long as they have the relevant > JARs on their classpath. And if they don't, they'll get some classpath error > instead -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9583) test-patch gives +1 despite build failure when running tests
[
https://issues.apache.org/jira/browse/HADOOP-9583?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13665466#comment-13665466
]
Hadoop QA commented on HADOOP-9583:
---
{color:green}+1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12584540/HADOOP-9583.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 1 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in .
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results:
https://builds.apache.org/job/PreCommit-HADOOP-Build/2559//testReport/
Console output:
https://builds.apache.org/job/PreCommit-HADOOP-Build/2559//console
This message is automatically generated.
> test-patch gives +1 despite build failure when running tests
>
>
> Key: HADOOP-9583
> URL: https://issues.apache.org/jira/browse/HADOOP-9583
> Project: Hadoop Common
> Issue Type: Bug
>Affects Versions: 3.0.0
>Reporter: Jason Lowe
>Assignee: Jason Lowe
>Priority: Critical
> Attachments: HADOOP-9583.patch
>
>
> I've seen a couple of checkins recently where tests have timed out resulting
> in a Maven build failure yet test-patch reports an overall +1 on the patch.
> This is encouraging commits of patches that subsequently break builds.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (HADOOP-7803) The first put to a non-existing current directory doesn't work correctly
[
https://issues.apache.org/jira/browse/HADOOP-7803?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Kihwal Lee resolved HADOOP-7803.
Resolution: Duplicate
> The first put to a non-existing current directory doesn't work correctly
>
>
> Key: HADOOP-7803
> URL: https://issues.apache.org/jira/browse/HADOOP-7803
> Project: Hadoop Common
> Issue Type: Bug
> Components: fs
>Affects Versions: 0.23.0
>Reporter: Kihwal Lee
>
> Steps to reproduce:
> - Format the dfs.
> - Start it.
> - Put a file without specifying the destination.
> {noformat}
> $ hadoop fs -ls
> ls: `.': No such file or directory
> $ hadoop fs -put /etc/passwd
> $ hadoop fs -ls
> Found 1 items
> -rw-r--r-- 1 kihwal supergroup 2076 2011-11-04 10:37 .._COPYING_
> {noformat}
> The namenode log:
> {noformat}
> - ugi=kihwalip=/127.0.0.1 cmd=create src=/user/kihwal/.._COPYING_
> dst=null
> perm=kihwal:supergroup:rw-r--r--
> - BLOCK* NameSystem.allocateBlock: /user/kihwal/.._COPYING_.
> BP-221429388-10.74.90.166-1320420960536
> blk_1038813851536531761_1001{blockUCState=UNDER_CONSTRUCTION,
> primaryNodeIndex=-1,
> replicas=[ReplicaUnderConstruction[127.0.0.1:50010|RBW]]}
> - BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:50010 is added to
> blk_1038813851536531761_1001{blockUCState=UNDER_CONSTRUCTION,
> primaryNodeIndex=-1,
> replicas=[ReplicaUnderConstruction[127.0.0.1:50010|RBW]]} size 0
> - DIR* NameSystem.completeFile: file /user/kihwal/.._COPYING_ is closed by
> DFSClient_NONMAPREDUCE_809910865_1
> - ugi=kihwalip=/127.0.0.1 cmd=rename src=/user/kihwal/.._COPYING_
> dst=/user/kihwal
> perm=kihwal:supergroup:rwxr-xr-x
> {noformat}
> It ends up creating a wrong file.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9593) stack trace printed at ERROR for all yarn clients without hadoop.home set
[
https://issues.apache.org/jira/browse/HADOOP-9593?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13665460#comment-13665460
]
Chris Nauroth commented on HADOOP-9593:
---
+1 for the patch. Thanks, Steve. Downgrading to debug level sounds like a
helpful first step.
{code}
+LOG.debug("setsid exited with exit code "
+ + (shexec != null ? shexec.getExitCode() : "(null
executor"));
{code}
Really minor nitpick: did you intend to have a closing paren inside the string
literal for "(null executor" ?
> stack trace printed at ERROR for all yarn clients without hadoop.home set
> -
>
> Key: HADOOP-9593
> URL: https://issues.apache.org/jira/browse/HADOOP-9593
> Project: Hadoop Common
> Issue Type: Bug
> Components: util
>Affects Versions: 3.0.0
>Reporter: Steve Loughran
> Attachments: HADOOP-9593-001.patch
>
>
> This is the problem of HADOOP-9482 now showing up in a different application
> -one whose log4j settings haven't turned off all Shell logging.
> Unless you do that, all yarn clients will have a stack trace at error in
> their logs, which is generating false alarms and is utterly pointless. Why
> does this merit a stack trace? Why log it at error? It's not an error for a
> client app to not have these values set as long as they have the relevant
> JARs on their classpath. And if they don't, they'll get some classpath error
> instead
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9593) stack trace printed at ERROR for all yarn clients without hadoop.home set
[ https://issues.apache.org/jira/browse/HADOOP-9593?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13665457#comment-13665457 ] Jason Lowe commented on HADOOP-9593: The annoying setsid message is also tracked by HADOOP-9546. > stack trace printed at ERROR for all yarn clients without hadoop.home set > - > > Key: HADOOP-9593 > URL: https://issues.apache.org/jira/browse/HADOOP-9593 > Project: Hadoop Common > Issue Type: Bug > Components: util >Affects Versions: 3.0.0 >Reporter: Steve Loughran > Attachments: HADOOP-9593-001.patch > > > This is the problem of HADOOP-9482 now showing up in a different application > -one whose log4j settings haven't turned off all Shell logging. > Unless you do that, all yarn clients will have a stack trace at error in > their logs, which is generating false alarms and is utterly pointless. Why > does this merit a stack trace? Why log it at error? It's not an error for a > client app to not have these values set as long as they have the relevant > JARs on their classpath. And if they don't, they'll get some classpath error > instead -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-9583) test-patch gives +1 despite build failure when running tests
[ https://issues.apache.org/jira/browse/HADOOP-9583?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Lowe updated HADOOP-9583: --- Target Version/s: 3.0.0 Status: Patch Available (was: Open) > test-patch gives +1 despite build failure when running tests > > > Key: HADOOP-9583 > URL: https://issues.apache.org/jira/browse/HADOOP-9583 > Project: Hadoop Common > Issue Type: Bug >Affects Versions: 3.0.0 >Reporter: Jason Lowe >Priority: Critical > Attachments: HADOOP-9583.patch > > > I've seen a couple of checkins recently where tests have timed out resulting > in a Maven build failure yet test-patch reports an overall +1 on the patch. > This is encouraging commits of patches that subsequently break builds. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (HADOOP-9583) test-patch gives +1 despite build failure when running tests
[ https://issues.apache.org/jira/browse/HADOOP-9583?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Lowe reassigned HADOOP-9583: -- Assignee: Jason Lowe > test-patch gives +1 despite build failure when running tests > > > Key: HADOOP-9583 > URL: https://issues.apache.org/jira/browse/HADOOP-9583 > Project: Hadoop Common > Issue Type: Bug >Affects Versions: 3.0.0 >Reporter: Jason Lowe >Assignee: Jason Lowe >Priority: Critical > Attachments: HADOOP-9583.patch > > > I've seen a couple of checkins recently where tests have timed out resulting > in a Maven build failure yet test-patch reports an overall +1 on the patch. > This is encouraging commits of patches that subsequently break builds. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-9593) stack trace printed at ERROR for all yarn clients without hadoop.home set
[ https://issues.apache.org/jira/browse/HADOOP-9593?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated HADOOP-9593: --- Status: Patch Available (was: Open) this is a small patch, but I think the UX will be better. I've seen this twice in programs, and I'm used to grabbing the first stack trace as the root cause -this is diverting me from doing that. We will be seeing a lot of JIRAs containing this stack if we don't fix it > stack trace printed at ERROR for all yarn clients without hadoop.home set > - > > Key: HADOOP-9593 > URL: https://issues.apache.org/jira/browse/HADOOP-9593 > Project: Hadoop Common > Issue Type: Bug > Components: util >Affects Versions: 3.0.0 >Reporter: Steve Loughran > Attachments: HADOOP-9593-001.patch > > > This is the problem of HADOOP-9482 now showing up in a different application > -one whose log4j settings haven't turned off all Shell logging. > Unless you do that, all yarn clients will have a stack trace at error in > their logs, which is generating false alarms and is utterly pointless. Why > does this merit a stack trace? Why log it at error? It's not an error for a > client app to not have these values set as long as they have the relevant > JARs on their classpath. And if they don't, they'll get some classpath error > instead -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-9583) test-patch gives +1 despite build failure when running tests
[
https://issues.apache.org/jira/browse/HADOOP-9583?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jason Lowe updated HADOOP-9583:
---
Attachment: HADOOP-9583.patch
Patch to try to implement the double-pronged approach. Maven is now executed
with {{-fae}} rather than {{-fn}} so we can get a proper exit code for test
builds. The test build output is also collected for post-processing to look
for tests that timed out.
> test-patch gives +1 despite build failure when running tests
>
>
> Key: HADOOP-9583
> URL: https://issues.apache.org/jira/browse/HADOOP-9583
> Project: Hadoop Common
> Issue Type: Bug
>Affects Versions: 3.0.0
>Reporter: Jason Lowe
>Priority: Critical
> Attachments: HADOOP-9583.patch
>
>
> I've seen a couple of checkins recently where tests have timed out resulting
> in a Maven build failure yet test-patch reports an overall +1 on the patch.
> This is encouraging commits of patches that subsequently break builds.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9593) stack trace printed at ERROR for all yarn clients without hadoop.home set
[ https://issues.apache.org/jira/browse/HADOOP-9593?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13665448#comment-13665448 ] Steve Loughran commented on HADOOP-9593: chris -this is related; this patch just downgrades the messages from ERROR, WARN and INFO to DEBUG. > stack trace printed at ERROR for all yarn clients without hadoop.home set > - > > Key: HADOOP-9593 > URL: https://issues.apache.org/jira/browse/HADOOP-9593 > Project: Hadoop Common > Issue Type: Bug > Components: util >Reporter: Steve Loughran > Attachments: HADOOP-9593-001.patch > > > This is the problem of HADOOP-9482 now showing up in a different application > -one whose log4j settings haven't turned off all Shell logging. > Unless you do that, all yarn clients will have a stack trace at error in > their logs, which is generating false alarms and is utterly pointless. Why > does this merit a stack trace? Why log it at error? It's not an error for a > client app to not have these values set as long as they have the relevant > JARs on their classpath. And if they don't, they'll get some classpath error > instead -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-9593) stack trace printed at ERROR for all yarn clients without hadoop.home set
[ https://issues.apache.org/jira/browse/HADOOP-9593?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated HADOOP-9593: --- Affects Version/s: 3.0.0 > stack trace printed at ERROR for all yarn clients without hadoop.home set > - > > Key: HADOOP-9593 > URL: https://issues.apache.org/jira/browse/HADOOP-9593 > Project: Hadoop Common > Issue Type: Bug > Components: util >Affects Versions: 3.0.0 >Reporter: Steve Loughran > Attachments: HADOOP-9593-001.patch > > > This is the problem of HADOOP-9482 now showing up in a different application > -one whose log4j settings haven't turned off all Shell logging. > Unless you do that, all yarn clients will have a stack trace at error in > their logs, which is generating false alarms and is utterly pointless. Why > does this merit a stack trace? Why log it at error? It's not an error for a > client app to not have these values set as long as they have the relevant > JARs on their classpath. And if they don't, they'll get some classpath error > instead -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-9593) stack trace printed at ERROR for all yarn clients without hadoop.home set
[ https://issues.apache.org/jira/browse/HADOOP-9593?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated HADOOP-9593: --- Attachment: HADOOP-9593-001.patch downgrade messages to debug; the setsid log() even checks for a null ref before use > stack trace printed at ERROR for all yarn clients without hadoop.home set > - > > Key: HADOOP-9593 > URL: https://issues.apache.org/jira/browse/HADOOP-9593 > Project: Hadoop Common > Issue Type: Bug > Components: util >Reporter: Steve Loughran > Attachments: HADOOP-9593-001.patch > > > This is the problem of HADOOP-9482 now showing up in a different application > -one whose log4j settings haven't turned off all Shell logging. > Unless you do that, all yarn clients will have a stack trace at error in > their logs, which is generating false alarms and is utterly pointless. Why > does this merit a stack trace? Why log it at error? It's not an error for a > client app to not have these values set as long as they have the relevant > JARs on their classpath. And if they don't, they'll get some classpath error > instead -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8562) Enhancements to support Hadoop on Windows Server and Windows Azure environments
[
https://issues.apache.org/jira/browse/HADOOP-8562?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13665435#comment-13665435
]
Chris Nauroth commented on HADOOP-8562:
---
{quote}
Chris, 1 and part of 2 should be fixed by the updated patch posted earlier.
{quote}
Yes, you're right. I had been running the older patch. Thanks, Arpit!
> Enhancements to support Hadoop on Windows Server and Windows Azure
> environments
> ---
>
> Key: HADOOP-8562
> URL: https://issues.apache.org/jira/browse/HADOOP-8562
> Project: Hadoop Common
> Issue Type: New Feature
>Affects Versions: 3.0.0
>Reporter: Bikas Saha
>Assignee: Bikas Saha
> Fix For: 3.0.0
>
> Attachments: branch-2.merge.patch, branch-2.merge.patch,
> branch-trunk-win.min-notest.patch, branch-trunk-win-min.patch,
> branch-trunk-win.min.patch, branch-trunk-win.patch, branch-trunk-win.patch,
> branch-trunk-win.patch, branch-trunk-win.patch, branch-trunk-win.patch,
> branch-trunk-win.patch, branch-trunk-win.patch, branch-trunk-win.patch,
> branch-trunk-win.patch, branch-trunk-win.patch, branch-trunk-win.patch,
> test-untar.tar, test-untar.tgz
>
>
> This JIRA tracks the work that needs to be done on trunk to enable Hadoop to
> run on Windows Server and Azure environments. This incorporates porting
> relevant work from the similar effort on branch 1 tracked via HADOOP-8079.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8562) Enhancements to support Hadoop on Windows Server and Windows Azure environments
[ https://issues.apache.org/jira/browse/HADOOP-8562?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13665428#comment-13665428 ] Arpit Agarwal commented on HADOOP-8562: --- Chris, 1 and part of 2 should be fixed by the updated patch posted earlier. More Windows fixes are still to be merged in. > Enhancements to support Hadoop on Windows Server and Windows Azure > environments > --- > > Key: HADOOP-8562 > URL: https://issues.apache.org/jira/browse/HADOOP-8562 > Project: Hadoop Common > Issue Type: New Feature >Affects Versions: 3.0.0 >Reporter: Bikas Saha >Assignee: Bikas Saha > Fix For: 3.0.0 > > Attachments: branch-2.merge.patch, branch-2.merge.patch, > branch-trunk-win.min-notest.patch, branch-trunk-win-min.patch, > branch-trunk-win.min.patch, branch-trunk-win.patch, branch-trunk-win.patch, > branch-trunk-win.patch, branch-trunk-win.patch, branch-trunk-win.patch, > branch-trunk-win.patch, branch-trunk-win.patch, branch-trunk-win.patch, > branch-trunk-win.patch, branch-trunk-win.patch, branch-trunk-win.patch, > test-untar.tar, test-untar.tgz > > > This JIRA tracks the work that needs to be done on trunk to enable Hadoop to > run on Windows Server and Azure environments. This incorporates porting > relevant work from the similar effort on branch 1 tracked via HADOOP-8079. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8562) Enhancements to support Hadoop on Windows Server and Windows Azure environments
[
https://issues.apache.org/jira/browse/HADOOP-8562?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13665414#comment-13665414
]
Chris Nauroth commented on HADOOP-8562:
---
So far, I see the following problems in the branch-2 merge patch:
#
hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/util/TestWinUtils.java
appears to be incorrect. In particular, there are {{@Test}} annotations on
private methods that aren't annotated in trunk. This causes JUnit to fail
while trying to run the private methods as tests. Probably the easiest way to
resolve this is to just copy the current trunk version.
#
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapreduce/v2/TestMRJobs.java
does not include the patches required for the test to pass on Windows. This
included timeout tuning and special handling for differences in symlink
behavior on Windows. Once again, I believe the easiest solution is just to
copy the trunk version.
> Enhancements to support Hadoop on Windows Server and Windows Azure
> environments
> ---
>
> Key: HADOOP-8562
> URL: https://issues.apache.org/jira/browse/HADOOP-8562
> Project: Hadoop Common
> Issue Type: New Feature
>Affects Versions: 3.0.0
>Reporter: Bikas Saha
>Assignee: Bikas Saha
> Fix For: 3.0.0
>
> Attachments: branch-2.merge.patch, branch-2.merge.patch,
> branch-trunk-win.min-notest.patch, branch-trunk-win-min.patch,
> branch-trunk-win.min.patch, branch-trunk-win.patch, branch-trunk-win.patch,
> branch-trunk-win.patch, branch-trunk-win.patch, branch-trunk-win.patch,
> branch-trunk-win.patch, branch-trunk-win.patch, branch-trunk-win.patch,
> branch-trunk-win.patch, branch-trunk-win.patch, branch-trunk-win.patch,
> test-untar.tar, test-untar.tgz
>
>
> This JIRA tracks the work that needs to be done on trunk to enable Hadoop to
> run on Windows Server and Azure environments. This incorporates porting
> relevant work from the similar effort on branch 1 tracked via HADOOP-8079.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8562) Enhancements to support Hadoop on Windows Server and Windows Azure environments
[ https://issues.apache.org/jira/browse/HADOOP-8562?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13665412#comment-13665412 ] Arpit Agarwal commented on HADOOP-8562: --- Updated merge patch to include fix for HADOOP-9372. > Enhancements to support Hadoop on Windows Server and Windows Azure > environments > --- > > Key: HADOOP-8562 > URL: https://issues.apache.org/jira/browse/HADOOP-8562 > Project: Hadoop Common > Issue Type: New Feature >Affects Versions: 3.0.0 >Reporter: Bikas Saha >Assignee: Bikas Saha > Fix For: 3.0.0 > > Attachments: branch-2.merge.patch, branch-2.merge.patch, > branch-trunk-win.min-notest.patch, branch-trunk-win-min.patch, > branch-trunk-win.min.patch, branch-trunk-win.patch, branch-trunk-win.patch, > branch-trunk-win.patch, branch-trunk-win.patch, branch-trunk-win.patch, > branch-trunk-win.patch, branch-trunk-win.patch, branch-trunk-win.patch, > branch-trunk-win.patch, branch-trunk-win.patch, branch-trunk-win.patch, > test-untar.tar, test-untar.tgz > > > This JIRA tracks the work that needs to be done on trunk to enable Hadoop to > run on Windows Server and Azure environments. This incorporates porting > relevant work from the similar effort on branch 1 tracked via HADOOP-8079. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-8562) Enhancements to support Hadoop on Windows Server and Windows Azure environments
[ https://issues.apache.org/jira/browse/HADOOP-8562?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arpit Agarwal updated HADOOP-8562: -- Attachment: branch-2.merge.patch > Enhancements to support Hadoop on Windows Server and Windows Azure > environments > --- > > Key: HADOOP-8562 > URL: https://issues.apache.org/jira/browse/HADOOP-8562 > Project: Hadoop Common > Issue Type: New Feature >Affects Versions: 3.0.0 >Reporter: Bikas Saha >Assignee: Bikas Saha > Fix For: 3.0.0 > > Attachments: branch-2.merge.patch, branch-2.merge.patch, > branch-trunk-win.min-notest.patch, branch-trunk-win-min.patch, > branch-trunk-win.min.patch, branch-trunk-win.patch, branch-trunk-win.patch, > branch-trunk-win.patch, branch-trunk-win.patch, branch-trunk-win.patch, > branch-trunk-win.patch, branch-trunk-win.patch, branch-trunk-win.patch, > branch-trunk-win.patch, branch-trunk-win.patch, branch-trunk-win.patch, > test-untar.tar, test-untar.tgz > > > This JIRA tracks the work that needs to be done on trunk to enable Hadoop to > run on Windows Server and Azure environments. This incorporates porting > relevant work from the similar effort on branch 1 tracked via HADOOP-8079. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-6171) "package" task in build.xml should copy source with preservelastmodified
[ https://issues.apache.org/jira/browse/HADOOP-6171?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13665398#comment-13665398 ] Ted Yu commented on HADOOP-6171: I think this JIRA is no longer needed after mavenization. > "package" task in build.xml should copy source with preservelastmodified > > > Key: HADOOP-6171 > URL: https://issues.apache.org/jira/browse/HADOOP-6171 > Project: Hadoop Common > Issue Type: Bug > Components: build >Reporter: Todd Lipcon >Assignee: Ted Yu > Labels: newbie > Attachments: HADOOP-6171.patch, HDFS-6171.patch, MAPRED-6171.patch > > > In the package task, it copies the source to dist.dir without using > preservelastmodified=true. This can cause issues with the autotools configure > process for the C++ builds. Namely, it will sometimes think the configure > script is out of date with respect to its source files and try to > re-bootstrap, which relies on particular versions of autotools on the > building computer. This isn't something that should be required for those > wanting to build from a distribution tarball. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9593) stack trace printed at ERROR for all yarn clients without hadoop.home set
[ https://issues.apache.org/jira/browse/HADOOP-9593?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13665389#comment-13665389 ] Chris Nauroth commented on HADOOP-9593: --- Hi, Steve. Is this the same as HADOOP-9422? (Thanks for adding that it impacts YARN clients too though.) > stack trace printed at ERROR for all yarn clients without hadoop.home set > - > > Key: HADOOP-9593 > URL: https://issues.apache.org/jira/browse/HADOOP-9593 > Project: Hadoop Common > Issue Type: Bug > Components: util >Reporter: Steve Loughran > > This is the problem of HADOOP-9482 now showing up in a different application > -one whose log4j settings haven't turned off all Shell logging. > Unless you do that, all yarn clients will have a stack trace at error in > their logs, which is generating false alarms and is utterly pointless. Why > does this merit a stack trace? Why log it at error? It's not an error for a > client app to not have these values set as long as they have the relevant > JARs on their classpath. And if they don't, they'll get some classpath error > instead -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-8562) Enhancements to support Hadoop on Windows Server and Windows Azure environments
[ https://issues.apache.org/jira/browse/HADOOP-8562?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arpit Agarwal updated HADOOP-8562: -- Attachment: branch-2.merge.patch Initial patch for merge to branch-2. > Enhancements to support Hadoop on Windows Server and Windows Azure > environments > --- > > Key: HADOOP-8562 > URL: https://issues.apache.org/jira/browse/HADOOP-8562 > Project: Hadoop Common > Issue Type: New Feature >Affects Versions: 3.0.0 >Reporter: Bikas Saha >Assignee: Bikas Saha > Fix For: 3.0.0 > > Attachments: branch-2.merge.patch, branch-trunk-win.min-notest.patch, > branch-trunk-win-min.patch, branch-trunk-win.min.patch, > branch-trunk-win.patch, branch-trunk-win.patch, branch-trunk-win.patch, > branch-trunk-win.patch, branch-trunk-win.patch, branch-trunk-win.patch, > branch-trunk-win.patch, branch-trunk-win.patch, branch-trunk-win.patch, > branch-trunk-win.patch, branch-trunk-win.patch, test-untar.tar, test-untar.tgz > > > This JIRA tracks the work that needs to be done on trunk to enable Hadoop to > run on Windows Server and Azure environments. This incorporates porting > relevant work from the similar effort on branch 1 tracked via HADOOP-8079. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9593) stack trace printed at ERROR for all yarn clients without hadoop.home set
[
https://issues.apache.org/jira/browse/HADOOP-9593?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13665372#comment-13665372
]
Steve Loughran commented on HADOOP-9593:
stack
{code}
013-05-23 17:58:09,766 [main] ERROR util.Shell
(Shell.java:checkHadoopHome(230)) - Failed to detect a valid hadoop home
directory
java.io.IOException: HADOOP_HOME or hadoop.home.dir are not set.
at org.apache.hadoop.util.Shell.checkHadoopHome(Shell.java:213)
at org.apache.hadoop.util.Shell.(Shell.java:236)
at org.apache.hadoop.util.PlatformName.(PlatformName.java:35)
at
org.apache.hadoop.security.UserGroupInformation.getOSLoginModuleName(UserGroupInformation.java:314)
at
org.apache.hadoop.security.UserGroupInformation.(UserGroupInformation.java:359)
at org.apache.hadoop.ipc.RPC.getProtocolProxy(RPC.java:450)
at org.apache.hadoop.ipc.RPC.getProtocolProxy(RPC.java:604)
at org.apache.hadoop.ipc.RPC.getProxy(RPC.java:561)
at
org.apache.hadoop.yarn.api.impl.pb.client.ClientRMProtocolPBClientImpl.(ClientRMProtocolPBClientImpl.java:104)
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at
sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:39)
at
sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:27)
at java.lang.reflect.Constructor.newInstance(Constructor.java:513)
at
org.apache.hadoop.yarn.factories.impl.pb.RpcClientFactoryPBImpl.getClient(RpcClientFactoryPBImpl.java:73)
at
org.apache.hadoop.yarn.ipc.HadoopYarnProtoRPC.getProxy(HadoopYarnProtoRPC.java:46)
{code}
Also I get some gratuitous complaints that setsuid didn't get used but still
failed
{code}
2013-05-23 17:58:09,778 [main] WARN util.Shell
(Shell.java:isSetsidSupported(308)) - setsid is not available on this machine.
So not using it.
2013-05-23 17:58:09,778 [main] INFO util.Shell
(Shell.java:isSetsidSupported(311)) - setsid exited with exit code
{code}
> stack trace printed at ERROR for all yarn clients without hadoop.home set
> -
>
> Key: HADOOP-9593
> URL: https://issues.apache.org/jira/browse/HADOOP-9593
> Project: Hadoop Common
> Issue Type: Bug
> Components: util
>Reporter: Steve Loughran
>
> This is the problem of HADOOP-9482 now showing up in a different application
> -one whose log4j settings haven't turned off all Shell logging.
> Unless you do that, all yarn clients will have a stack trace at error in
> their logs, which is generating false alarms and is utterly pointless. Why
> does this merit a stack trace? Why log it at error? It's not an error for a
> client app to not have these values set as long as they have the relevant
> JARs on their classpath. And if they don't, they'll get some classpath error
> instead
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HADOOP-9593) stack trace printed at ERROR for all yarn clients without hadoop.home set
Steve Loughran created HADOOP-9593: -- Summary: stack trace printed at ERROR for all yarn clients without hadoop.home set Key: HADOOP-9593 URL: https://issues.apache.org/jira/browse/HADOOP-9593 Project: Hadoop Common Issue Type: Bug Components: util Reporter: Steve Loughran This is the problem of HADOOP-9482 now showing up in a different application -one whose log4j settings haven't turned off all Shell logging. Unless you do that, all yarn clients will have a stack trace at error in their logs, which is generating false alarms and is utterly pointless. Why does this merit a stack trace? Why log it at error? It's not an error for a client app to not have these values set as long as they have the relevant JARs on their classpath. And if they don't, they'll get some classpath error instead -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9582) Non-existent file to "hadoop fs -conf" doesn't throw error
[
https://issues.apache.org/jira/browse/HADOOP-9582?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13665048#comment-13665048
]
Steve Loughran commented on HADOOP-9582:
Ashwin
# I don't know the history of that, let's ask.
# fs shell calls {{setQuietMode(true)}} in {{init()}} and doesn't provide a way
to turn it off. So even if we consider your current patch surplus, we do still
need a way to turn the flag on.
> Non-existent file to "hadoop fs -conf" doesn't throw error
> --
>
> Key: HADOOP-9582
> URL: https://issues.apache.org/jira/browse/HADOOP-9582
> Project: Hadoop Common
> Issue Type: Bug
> Components: conf
>Affects Versions: 3.0.0, 0.23.7, 2.0.4-alpha
>Reporter: Ashwin Shankar
> Attachments: HADOOP-9582.txt, HADOOP-9582.txt
>
>
> When we run :
> hadoop fs -conf BAD_FILE -ls /
> we expect hadoop to throw an error,but it doesn't.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9435) Minimal code change to support IBM jvm build dependency
[ https://issues.apache.org/jira/browse/HADOOP-9435?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13664943#comment-13664943 ] Suresh Srinivas commented on HADOOP-9435: - [~tianhonglouis] Want to make sure that this is ready to be committed and not tests are pending. > Minimal code change to support IBM jvm build dependency > --- > > Key: HADOOP-9435 > URL: https://issues.apache.org/jira/browse/HADOOP-9435 > Project: Hadoop Common > Issue Type: Bug > Components: build >Reporter: Tian Hong Wang >Assignee: Tian Hong Wang > Labels: patch > Attachments: HADOOP-9435.patch, HADOOP-9435-v1.patch > > > When native build hadoop-common-project with IBM java using command like: > mvn package -Pnative > it will exist the following errors. > [exec] -- Configuring incomplete, errors occurred! > [exec] JAVA_HOME=, > JAVA_JVM_LIBRARY=/home/louis/ibm-java-i386-60/jre/lib/i386/classic/libjvm.so > [exec] JAVA_INCLUDE_PATH=/home/louis/ibm-java-i386-60/include, > JAVA_INCLUDE_PATH2=JAVA_INCLUDE_PATH2-NOTFOUND > [exec] CMake Error at JNIFlags.cmake:113 (MESSAGE): > [exec] Failed to find a viable JVM installation under JAVA_HOME. > [exec] Call Stack (most recent call first): > [exec] CMakeLists.txt:24 (include) > The reason is that IBM java uses $JAVA_HOME/include/jniport.h instead of > $JAVA_HOME/include/jni_md.h in non-IBM java. > [exec] > /usr/lib/jvm/java-1.6.0-ibm-1.6.0.12.0.x86_64/jre/lib/amd64/default/libjvm.so: > undefined reference to `dlsym' > [exec] > /usr/lib/jvm/java-1.6.0-ibm-1.6.0.12.0.x86_64/jre/lib/amd64/default/libjvm.so: > undefined reference to `dlerror' > [exec] > /usr/lib/jvm/java-1.6.0-ibm-1.6.0.12.0.x86_64/jre/lib/amd64/default/libjvm.so: > undefined reference to `dladdr' > [exec] > /usr/lib/jvm/java-1.6.0-ibm-1.6.0.12.0.x86_64/jre/lib/amd64/default/libjvm.so: > undefined reference to `dlopen' > [exec] > /usr/lib/jvm/java-1.6.0-ibm-1.6.0.12.0.x86_64/jre/lib/amd64/default/libjvm.so: > undefined reference to `dlclose' > [exec] collect2: ld returned 1 exit status > [exec] make[2]: *** [test_libhdfs_ops] Error 1 > [exec] make[1]: *** [CMakeFiles/test_libhdfs_ops.dir/all] Error 2 > [exec] make: *** [all] Error > The reason is libjvm.so need libdl when linking. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
