[jira] [Commented] (HADOOP-9486) Promote Windows and Shell related utils from YARN to Hadoop Common
[ https://issues.apache.org/jira/browse/HADOOP-9486?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13635965#comment-13635965 ] Hudson commented on HADOOP-9486: Integrated in Hadoop-trunk-Commit #3634 (See [https://builds.apache.org/job/Hadoop-trunk-Commit/3634/]) YARN-493. Fixed some shell related flaws in YARN on Windows. Contributed by Chris Nauroth. HADOOP-9486. Promoted Windows and Shell related utils from YARN to Hadoop Common. Contributed by Chris Nauroth. (Revision 1469667) Result = SUCCESS vinodkv : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1469667 Files : * /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt * /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/Shell.java * /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/winutils/task.c * /hadoop/common/trunk/hadoop-yarn-project/CHANGES.txt * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/ContainerExecutor.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/DefaultContainerExecutor.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/launcher/ContainerLaunch.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/TestNodeManagerShutdown.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/BaseContainerManagerTest.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/TestContainerManager.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/launcher/TestContainerLaunch.java > Promote Windows and Shell related utils from YARN to Hadoop Common > -- > > Key: HADOOP-9486 > URL: https://issues.apache.org/jira/browse/HADOOP-9486 > Project: Hadoop Common > Issue Type: Bug >Reporter: Vinod Kumar Vavilapalli >Assignee: Chris Nauroth > Fix For: 3.0.0 > > Attachments: HADOOP-9486.patch > > > This is happening as part of YARN-493, this is a tracking ticket for common > changes. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (HADOOP-9486) Promote Windows and Shell related utils from YARN to Hadoop Common
[ https://issues.apache.org/jira/browse/HADOOP-9486?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinod Kumar Vavilapalli resolved HADOOP-9486. - Resolution: Fixed Fix Version/s: 3.0.0 Hadoop Flags: Reviewed I just committed this trunk together with YARN-493. Closing this as resolved. I've been looking at this at YARN-493 for a while, so went ahead with the check-in Apologies for the small time delta between issue-creation and closure. Let me know if anyone finds any issues. Thanks. > Promote Windows and Shell related utils from YARN to Hadoop Common > -- > > Key: HADOOP-9486 > URL: https://issues.apache.org/jira/browse/HADOOP-9486 > Project: Hadoop Common > Issue Type: Bug >Reporter: Vinod Kumar Vavilapalli >Assignee: Chris Nauroth > Fix For: 3.0.0 > > Attachments: HADOOP-9486.patch > > > This is happening as part of YARN-493, this is a tracking ticket for common > changes. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-9486) Promote Windows and Shell related utils from YARN to Hadoop Common
[ https://issues.apache.org/jira/browse/HADOOP-9486?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinod Kumar Vavilapalli updated HADOOP-9486: Attachment: HADOOP-9486.patch Attaching patch on behalf of Chris. Contains Common only changes from YARN-493. I already reviewed this, looks good. Going to commit this as part of YARN-493 to trunk. > Promote Windows and Shell related utils from YARN to Hadoop Common > -- > > Key: HADOOP-9486 > URL: https://issues.apache.org/jira/browse/HADOOP-9486 > Project: Hadoop Common > Issue Type: Bug >Reporter: Vinod Kumar Vavilapalli >Assignee: Chris Nauroth > Attachments: HADOOP-9486.patch > > > This is happening as part of YARN-493, this is a tracking ticket for common > changes. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HADOOP-9486) Promote Windows and Shell related utils from YARN to Hadoop Common
Vinod Kumar Vavilapalli created HADOOP-9486: --- Summary: Promote Windows and Shell related utils from YARN to Hadoop Common Key: HADOOP-9486 URL: https://issues.apache.org/jira/browse/HADOOP-9486 Project: Hadoop Common Issue Type: Bug Reporter: Vinod Kumar Vavilapalli Assignee: Chris Nauroth This is happening as part of YARN-493, this is a tracking ticket for common changes. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9485) inconsistent defaults for hadoop.rpc.socket.factory.class.default
[
https://issues.apache.org/jira/browse/HADOOP-9485?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13635932#comment-13635932
]
Hadoop QA commented on HADOOP-9485:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12579451/HADOOP-9485.002.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 1 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:red}-1 core tests{color}. The patch failed these unit tests in
hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs:
org.apache.hadoop.ipc.TestSaslRPC
org.apache.hadoop.hdfs.TestDistributedFileSystem
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results:
https://builds.apache.org/job/PreCommit-HADOOP-Build/2459//testReport/
Console output:
https://builds.apache.org/job/PreCommit-HADOOP-Build/2459//console
This message is automatically generated.
> inconsistent defaults for hadoop.rpc.socket.factory.class.default
> -
>
> Key: HADOOP-9485
> URL: https://issues.apache.org/jira/browse/HADOOP-9485
> Project: Hadoop Common
> Issue Type: Bug
> Components: net
>Affects Versions: 2.0.5-beta
>Reporter: Colin Patrick McCabe
>Assignee: Colin Patrick McCabe
>Priority: Minor
> Attachments: HADOOP-9485.001.patch, HADOOP-9485.002.patch
>
>
> In {{core-default.xml}}, {{hadoop.rpc.socket.factory.class.default}} defaults
> to {{org.apache.hadoop.net.StandardSocketFactory}}. However, in
> {{CommonConfigurationKeysPublic.java}}, there is no default for this key.
> This is inconsistent (defaults in the code versus defaults in the XML files
> should match.) It also leads to problems with {{RemoteBlockReader2}}, since
> the default {{SocketFactory}} creates a {{Socket}} without an associated
> channel. {{RemoteBlockReader2}} cannot use such a {{Socket}}.
> This bug only really becomes apparent when you create a {{Configuration}}
> using the {{Configuration(loadDefaults=true)}} constructor. Thanks to AB
> Srinivasan for his help in discovering this bug.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9483) winutils support for readlink command
[
https://issues.apache.org/jira/browse/HADOOP-9483?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13635923#comment-13635923
]
Chuan Liu commented on HADOOP-9483:
---
MSDN says:
{quote}
A final path is the path that is returned when a path is fully resolved. For
example, for a symbolic link named "C:\tmp\mydir" that points to "D:\yourdir",
the final path would be "D:\yourdir".
The string that is returned by this function uses the \\?\ syntax.
{quote}
So I think it will be full path prefixed with "\\?\".
> winutils support for readlink command
> -
>
> Key: HADOOP-9483
> URL: https://issues.apache.org/jira/browse/HADOOP-9483
> Project: Hadoop Common
> Issue Type: Improvement
> Components: util
>Affects Versions: 3.0.0
>Reporter: Chris Nauroth
>
> The current codebase relies on the Unix readlink command to determine the
> target of a symlink on the local file system. winutils currently does not
> support this functionality on Windows. Adding the command to winutils will
> prevent the need to use GnuWin32 or Cygwin for readlink support.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9421) Add full length to SASL response to allow non-blocking readers
[ https://issues.apache.org/jira/browse/HADOOP-9421?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13635874#comment-13635874 ] Junping Du commented on HADOOP-9421: So the only change is server reply status-challenge (with preferred mechanism) rather than a hacked fixed minus length (to switch to simple) before. Daryn, do you agree on this way? > Add full length to SASL response to allow non-blocking readers > -- > > Key: HADOOP-9421 > URL: https://issues.apache.org/jira/browse/HADOOP-9421 > Project: Hadoop Common > Issue Type: Sub-task >Affects Versions: 2.0.3-alpha >Reporter: Sanjay Radia >Assignee: Junping Du > Attachments: HADOOP-9421.patch > > -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9485) inconsistent defaults for hadoop.rpc.socket.factory.class.default
[
https://issues.apache.org/jira/browse/HADOOP-9485?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13635856#comment-13635856
]
Hadoop QA commented on HADOOP-9485:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12579448/HADOOP-9485.001.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:red}-1 tests included{color}. The patch doesn't appear to include
any new or modified tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:red}-1 core tests{color}. The patch failed these unit tests in
hadoop-common-project/hadoop-common:
org.apache.hadoop.ipc.TestSaslRPC
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results:
https://builds.apache.org/job/PreCommit-HADOOP-Build/2458//testReport/
Console output:
https://builds.apache.org/job/PreCommit-HADOOP-Build/2458//console
This message is automatically generated.
> inconsistent defaults for hadoop.rpc.socket.factory.class.default
> -
>
> Key: HADOOP-9485
> URL: https://issues.apache.org/jira/browse/HADOOP-9485
> Project: Hadoop Common
> Issue Type: Bug
> Components: net
>Affects Versions: 2.0.5-beta
>Reporter: Colin Patrick McCabe
>Assignee: Colin Patrick McCabe
>Priority: Minor
> Attachments: HADOOP-9485.001.patch, HADOOP-9485.002.patch
>
>
> In {{core-default.xml}}, {{hadoop.rpc.socket.factory.class.default}} defaults
> to {{org.apache.hadoop.net.StandardSocketFactory}}. However, in
> {{CommonConfigurationKeysPublic.java}}, there is no default for this key.
> This is inconsistent (defaults in the code versus defaults in the XML files
> should match.) It also leads to problems with {{RemoteBlockReader2}}, since
> the default {{SocketFactory}} creates a {{Socket}} without an associated
> channel. {{RemoteBlockReader2}} cannot use such a {{Socket}}.
> This bug only really becomes apparent when you create a {{Configuration}}
> using the {{Configuration(loadDefaults=true)}} constructor. Thanks to AB
> Srinivasan for his help in discovering this bug.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8711) provide an option for IPC server users to avoid printing stack information for certain exceptions
[ https://issues.apache.org/jira/browse/HADOOP-8711?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13635845#comment-13635845 ] Brandon Li commented on HADOOP-8711: Thanks Kihwal! This patch could be useful for banch-1 too. Just uploaded a patch for branch-1. > provide an option for IPC server users to avoid printing stack information > for certain exceptions > - > > Key: HADOOP-8711 > URL: https://issues.apache.org/jira/browse/HADOOP-8711 > Project: Hadoop Common > Issue Type: Improvement > Components: ipc >Affects Versions: 3.0.0 >Reporter: Brandon Li >Assignee: Brandon Li > Fix For: 3.0.0, 0.23.7 > > Attachments: HADOOP-8711.patch, HADOOP-8711.patch, HADOOP-8711.patch, > HADOOP-8711.patch, HADOOP-8711.patch, HADOOP-8871.branch-1.patch > > > Currently it's hard coded in the server that it doesn't print the exception > stack for StandbyException. > Similarly, other components may have their own exceptions which don't need to > save the stack trace in log. One example is HDFS-3817. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-8711) provide an option for IPC server users to avoid printing stack information for certain exceptions
[ https://issues.apache.org/jira/browse/HADOOP-8711?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brandon Li updated HADOOP-8711: --- Attachment: HADOOP-8871.branch-1.patch > provide an option for IPC server users to avoid printing stack information > for certain exceptions > - > > Key: HADOOP-8711 > URL: https://issues.apache.org/jira/browse/HADOOP-8711 > Project: Hadoop Common > Issue Type: Improvement > Components: ipc >Affects Versions: 3.0.0 >Reporter: Brandon Li >Assignee: Brandon Li > Fix For: 3.0.0, 0.23.7 > > Attachments: HADOOP-8711.patch, HADOOP-8711.patch, HADOOP-8711.patch, > HADOOP-8711.patch, HADOOP-8711.patch, HADOOP-8871.branch-1.patch > > > Currently it's hard coded in the server that it doesn't print the exception > stack for StandbyException. > Similarly, other components may have their own exceptions which don't need to > save the stack trace in log. One example is HDFS-3817. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-9485) inconsistent defaults for hadoop.rpc.socket.factory.class.default
[
https://issues.apache.org/jira/browse/HADOOP-9485?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Colin Patrick McCabe updated HADOOP-9485:
-
Attachment: HADOOP-9485.002.patch
Let's also add a unit test, to make sure this doesn't happen again.
> inconsistent defaults for hadoop.rpc.socket.factory.class.default
> -
>
> Key: HADOOP-9485
> URL: https://issues.apache.org/jira/browse/HADOOP-9485
> Project: Hadoop Common
> Issue Type: Bug
> Components: net
>Affects Versions: 2.0.5-beta
>Reporter: Colin Patrick McCabe
>Assignee: Colin Patrick McCabe
>Priority: Minor
> Attachments: HADOOP-9485.001.patch, HADOOP-9485.002.patch
>
>
> In {{core-default.xml}}, {{hadoop.rpc.socket.factory.class.default}} defaults
> to {{org.apache.hadoop.net.StandardSocketFactory}}. However, in
> {{CommonConfigurationKeysPublic.java}}, there is no default for this key.
> This is inconsistent (defaults in the code versus defaults in the XML files
> should match.) It also leads to problems with {{RemoteBlockReader2}}, since
> the default {{SocketFactory}} creates a {{Socket}} without an associated
> channel. {{RemoteBlockReader2}} cannot use such a {{Socket}}.
> This bug only really becomes apparent when you create a {{Configuration}}
> using the {{Configuration(loadDefaults=true)}} constructor. Thanks to AB
> Srinivasan for his help in discovering this bug.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-9485) inconsistent defaults for hadoop.rpc.socket.factory.class.default
[
https://issues.apache.org/jira/browse/HADOOP-9485?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Colin Patrick McCabe updated HADOOP-9485:
-
Status: Patch Available (was: Open)
> inconsistent defaults for hadoop.rpc.socket.factory.class.default
> -
>
> Key: HADOOP-9485
> URL: https://issues.apache.org/jira/browse/HADOOP-9485
> Project: Hadoop Common
> Issue Type: Bug
> Components: net
>Affects Versions: 2.0.5-beta
>Reporter: Colin Patrick McCabe
>Assignee: Colin Patrick McCabe
>Priority: Minor
> Attachments: HADOOP-9485.001.patch
>
>
> In {{core-default.xml}}, {{hadoop.rpc.socket.factory.class.default}} defaults
> to {{org.apache.hadoop.net.StandardSocketFactory}}. However, in
> {{CommonConfigurationKeysPublic.java}}, there is no default for this key.
> This is inconsistent (defaults in the code versus defaults in the XML files
> should match.) It also leads to problems with {{RemoteBlockReader2}}, since
> the default {{SocketFactory}} creates a {{Socket}} without an associated
> channel. {{RemoteBlockReader2}} cannot use such a {{Socket}}.
> This bug only really becomes apparent when you create a {{Configuration}}
> using the {{Configuration(loadDefaults=true)}} constructor. Thanks to AB
> Srinivasan for his help in discovering this bug.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-9485) inconsistent defaults for hadoop.rpc.socket.factory.class.default
[
https://issues.apache.org/jira/browse/HADOOP-9485?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Colin Patrick McCabe updated HADOOP-9485:
-
Attachment: HADOOP-9485.001.patch
> inconsistent defaults for hadoop.rpc.socket.factory.class.default
> -
>
> Key: HADOOP-9485
> URL: https://issues.apache.org/jira/browse/HADOOP-9485
> Project: Hadoop Common
> Issue Type: Bug
> Components: net
>Affects Versions: 2.0.5-beta
>Reporter: Colin Patrick McCabe
>Assignee: Colin Patrick McCabe
>Priority: Minor
> Attachments: HADOOP-9485.001.patch
>
>
> In {{core-default.xml}}, {{hadoop.rpc.socket.factory.class.default}} defaults
> to {{org.apache.hadoop.net.StandardSocketFactory}}. However, in
> {{CommonConfigurationKeysPublic.java}}, there is no default for this key.
> This is inconsistent (defaults in the code versus defaults in the XML files
> should match.) It also leads to problems with {{RemoteBlockReader2}}, since
> the default {{SocketFactory}} creates a {{Socket}} without an associated
> channel. {{RemoteBlockReader2}} cannot use such a {{Socket}}.
> This bug only really becomes apparent when you create a {{Configuration}}
> using the {{Configuration(loadDefaults=true)}} constructor. Thanks to AB
> Srinivasan for his help in discovering this bug.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HADOOP-9485) inconsistent defaults for hadoop.rpc.socket.factory.class.default
Colin Patrick McCabe created HADOOP-9485:
Summary: inconsistent defaults for
hadoop.rpc.socket.factory.class.default
Key: HADOOP-9485
URL: https://issues.apache.org/jira/browse/HADOOP-9485
Project: Hadoop Common
Issue Type: Bug
Components: net
Affects Versions: 2.0.5-beta
Reporter: Colin Patrick McCabe
Assignee: Colin Patrick McCabe
Priority: Minor
In {{core-default.xml}}, {{hadoop.rpc.socket.factory.class.default}} defaults
to {{org.apache.hadoop.net.StandardSocketFactory}}. However, in
{{CommonConfigurationKeysPublic.java}}, there is no default for this key. This
is inconsistent (defaults in the code versus defaults in the XML files should
match.) It also leads to problems with {{RemoteBlockReader2}}, since the
default {{SocketFactory}} creates a {{Socket}} without an associated channel.
{{RemoteBlockReader2}} cannot use such a {{Socket}}.
This bug only really becomes apparent when you create a {{Configuration}} using
the {{Configuration(loadDefaults=true)}} constructor. Thanks to AB Srinivasan
for his help in discovering this bug.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8711) provide an option for IPC server users to avoid printing stack information for certain exceptions
[ https://issues.apache.org/jira/browse/HADOOP-8711?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13635681#comment-13635681 ] Kihwal Lee commented on HADOOP-8711: Committed to branch-2 as HDFS-4714 depends on it. > provide an option for IPC server users to avoid printing stack information > for certain exceptions > - > > Key: HADOOP-8711 > URL: https://issues.apache.org/jira/browse/HADOOP-8711 > Project: Hadoop Common > Issue Type: Improvement > Components: ipc >Affects Versions: 3.0.0 >Reporter: Brandon Li >Assignee: Brandon Li > Fix For: 3.0.0, 0.23.7 > > Attachments: HADOOP-8711.patch, HADOOP-8711.patch, HADOOP-8711.patch, > HADOOP-8711.patch, HADOOP-8711.patch > > > Currently it's hard coded in the server that it doesn't print the exception > stack for StandbyException. > Similarly, other components may have their own exceptions which don't need to > save the stack trace in log. One example is HDFS-3817. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8545) Filesystem Implementation for OpenStack Swift
[
https://issues.apache.org/jira/browse/HADOOP-8545?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13635541#comment-13635541
]
Hadoop QA commented on HADOOP-8545:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12579382/HADOOP-8545-22.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 26 new
or modified test files.
{color:red}-1 javac{color}. The applied patch generated 1376 javac
compiler warnings (more than the trunk's current 1367 warnings).
{color:red}-1 javadoc{color}. The javadoc tool appears to have generated 6
warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:red}-1 findbugs{color}. The patch appears to introduce 2 new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in
hadoop-common-project/hadoop-common hadoop-tools/hadoop-openstack
hadoop-tools/hadoop-tools-dist.
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results:
https://builds.apache.org/job/PreCommit-HADOOP-Build/2457//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HADOOP-Build/2457//artifact/trunk/patchprocess/newPatchFindbugsWarningshadoop-openstack.html
Javac warnings:
https://builds.apache.org/job/PreCommit-HADOOP-Build/2457//artifact/trunk/patchprocess/diffJavacWarnings.txt
Console output:
https://builds.apache.org/job/PreCommit-HADOOP-Build/2457//console
This message is automatically generated.
> Filesystem Implementation for OpenStack Swift
> -
>
> Key: HADOOP-8545
> URL: https://issues.apache.org/jira/browse/HADOOP-8545
> Project: Hadoop Common
> Issue Type: New Feature
> Components: fs
>Affects Versions: 2.0.3-alpha, 1.1.2
>Reporter: Tim Miller
>Assignee: Dmitry Mezhensky
> Labels: hadoop, patch
> Attachments: HADOOP-8545-10.patch, HADOOP-8545-11.patch,
> HADOOP-8545-12.patch, HADOOP-8545-13.patch, HADOOP-8545-14.patch,
> HADOOP-8545-15.patch, HADOOP-8545-16.patch, HADOOP-8545-17.patch,
> HADOOP-8545-18.patch, HADOOP-8545-19.patch, HADOOP-8545-1.patch,
> HADOOP-8545-20.patch, HADOOP-8545-21.patch, HADOOP-8545-22.patch,
> HADOOP-8545-2.patch, HADOOP-8545-3.patch, HADOOP-8545-4.patch,
> HADOOP-8545-5.patch, HADOOP-8545-6.patch, HADOOP-8545-7.patch,
> HADOOP-8545-8.patch, HADOOP-8545-9.patch, HADOOP-8545-javaclouds-2.patch,
> HADOOP-8545.patch, HADOOP-8545.patch
>
>
> ,Add a filesystem implementation for OpenStack Swift object store, similar to
> the one which exists today for S3.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9481) Broken conditional logic with HADOOP_SNAPPY_LIBRARY
[ https://issues.apache.org/jira/browse/HADOOP-9481?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13635523#comment-13635523 ] Colin Patrick McCabe commented on HADOOP-9481: -- I don't understand what problem is being fixed here, even after looking at the patch. Can you add a description? Also, have you installed snappy locally? If you have not, compiling without snappy is the correct course of action, and not a problem. > Broken conditional logic with HADOOP_SNAPPY_LIBRARY > --- > > Key: HADOOP-9481 > URL: https://issues.apache.org/jira/browse/HADOOP-9481 > Project: Hadoop Common > Issue Type: Bug >Affects Versions: 3.0.0 >Reporter: Vadim Bondarev >Priority: Minor > Attachments: HADOOP-9481-trunk--N1.patch > > -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-8545) Filesystem Implementation for OpenStack Swift
[ https://issues.apache.org/jira/browse/HADOOP-8545?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated HADOOP-8545: --- Status: Patch Available (was: Open) For testing. For all the 33+ watchers, it's time to D/L and play with this code -it is designed to work against branch-1 too. Integration and scale tests are what we need, especially on different Swift infrastructures. > Filesystem Implementation for OpenStack Swift > - > > Key: HADOOP-8545 > URL: https://issues.apache.org/jira/browse/HADOOP-8545 > Project: Hadoop Common > Issue Type: New Feature > Components: fs >Affects Versions: 1.1.2, 2.0.3-alpha >Reporter: Tim Miller >Assignee: Dmitry Mezhensky > Labels: hadoop, patch > Attachments: HADOOP-8545-10.patch, HADOOP-8545-11.patch, > HADOOP-8545-12.patch, HADOOP-8545-13.patch, HADOOP-8545-14.patch, > HADOOP-8545-15.patch, HADOOP-8545-16.patch, HADOOP-8545-17.patch, > HADOOP-8545-18.patch, HADOOP-8545-19.patch, HADOOP-8545-1.patch, > HADOOP-8545-20.patch, HADOOP-8545-21.patch, HADOOP-8545-22.patch, > HADOOP-8545-2.patch, HADOOP-8545-3.patch, HADOOP-8545-4.patch, > HADOOP-8545-5.patch, HADOOP-8545-6.patch, HADOOP-8545-7.patch, > HADOOP-8545-8.patch, HADOOP-8545-9.patch, HADOOP-8545-javaclouds-2.patch, > HADOOP-8545.patch, HADOOP-8545.patch > > > ,Add a filesystem implementation for OpenStack Swift object store, similar to > the one which exists today for S3. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-8545) Filesystem Implementation for OpenStack Swift
[ https://issues.apache.org/jira/browse/HADOOP-8545?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated HADOOP-8545: --- Attachment: HADOOP-8545-22.patch Patch adds # ability to specify partition size of large files, so break up large files into many smaller ones -good for long-haul uploads. # tests that partition setting works as intended. # ability to specify range size for the input stream, so request bigger ranges than 64KB on faster networks; smaller ranges for reads-with-many-seeks on slower networks. # The tests for seek proposed in HADOOP-9371. # a fix for seek to work reliably when it crosses range boundaries. # updated documentation for the new options. > Filesystem Implementation for OpenStack Swift > - > > Key: HADOOP-8545 > URL: https://issues.apache.org/jira/browse/HADOOP-8545 > Project: Hadoop Common > Issue Type: New Feature > Components: fs >Affects Versions: 2.0.3-alpha, 1.1.2 >Reporter: Tim Miller >Assignee: Dmitry Mezhensky > Labels: hadoop, patch > Attachments: HADOOP-8545-10.patch, HADOOP-8545-11.patch, > HADOOP-8545-12.patch, HADOOP-8545-13.patch, HADOOP-8545-14.patch, > HADOOP-8545-15.patch, HADOOP-8545-16.patch, HADOOP-8545-17.patch, > HADOOP-8545-18.patch, HADOOP-8545-19.patch, HADOOP-8545-1.patch, > HADOOP-8545-20.patch, HADOOP-8545-21.patch, HADOOP-8545-22.patch, > HADOOP-8545-2.patch, HADOOP-8545-3.patch, HADOOP-8545-4.patch, > HADOOP-8545-5.patch, HADOOP-8545-6.patch, HADOOP-8545-7.patch, > HADOOP-8545-8.patch, HADOOP-8545-9.patch, HADOOP-8545-javaclouds-2.patch, > HADOOP-8545.patch, HADOOP-8545.patch > > > ,Add a filesystem implementation for OpenStack Swift object store, similar to > the one which exists today for S3. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-8545) Filesystem Implementation for OpenStack Swift
[ https://issues.apache.org/jira/browse/HADOOP-8545?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated HADOOP-8545: --- Status: Open (was: Patch Available) > Filesystem Implementation for OpenStack Swift > - > > Key: HADOOP-8545 > URL: https://issues.apache.org/jira/browse/HADOOP-8545 > Project: Hadoop Common > Issue Type: New Feature > Components: fs >Affects Versions: 1.1.2, 2.0.3-alpha >Reporter: Tim Miller >Assignee: Dmitry Mezhensky > Labels: hadoop, patch > Attachments: HADOOP-8545-10.patch, HADOOP-8545-11.patch, > HADOOP-8545-12.patch, HADOOP-8545-13.patch, HADOOP-8545-14.patch, > HADOOP-8545-15.patch, HADOOP-8545-16.patch, HADOOP-8545-17.patch, > HADOOP-8545-18.patch, HADOOP-8545-19.patch, HADOOP-8545-1.patch, > HADOOP-8545-20.patch, HADOOP-8545-21.patch, HADOOP-8545-2.patch, > HADOOP-8545-3.patch, HADOOP-8545-4.patch, HADOOP-8545-5.patch, > HADOOP-8545-6.patch, HADOOP-8545-7.patch, HADOOP-8545-8.patch, > HADOOP-8545-9.patch, HADOOP-8545-javaclouds-2.patch, HADOOP-8545.patch, > HADOOP-8545.patch > > > ,Add a filesystem implementation for OpenStack Swift object store, similar to > the one which exists today for S3. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9407) commons-daemon 1.0.3 dependency has bad group id causing build issues
[
https://issues.apache.org/jira/browse/HADOOP-9407?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13635457#comment-13635457
]
Konstantin Boudnik commented on HADOOP-9407:
I have raised it again on the voting thread with a neutral vote (-0). For a
variety of reasons I am not very optimistic that RC will be re-spun at this
point.
> commons-daemon 1.0.3 dependency has bad group id causing build issues
> -
>
> Key: HADOOP-9407
> URL: https://issues.apache.org/jira/browse/HADOOP-9407
> Project: Hadoop Common
> Issue Type: Bug
> Components: build
>Affects Versions: 2.0.3-alpha, 2.0.4-alpha
>Reporter: Sangjin Lee
>Assignee: Sangjin Lee
> Labels: 2.0.4
> Fix For: 2.0.5-beta
>
> Attachments: HDFS-4497.patch
>
>
> The commons-daemon dependency of the hadoop-hdfs module has been at version
> 1.0.3 for a while. However, 1.0.3 has a pretty well-known groupId error in
> its pom ("org.apache.commons" as opposed to "commons-daemon"). This problem
> has since been corrected on commons-daemon starting 1.0.4.
> This causes build problems for many who depend on hadoop-hdfs directly and
> indirectly, however. Maven can skip over this metadata inconsistency. But
> other less forgiving build systems such as ivy and gradle have much harder
> time working around this problem. For example, in gradle, pretty much the
> only obvious way to work around this is to override this dependency version.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9484) Genetic Algorithm Library for Hadoop
[ https://issues.apache.org/jira/browse/HADOOP-9484?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13635443#comment-13635443 ] Colin Patrick McCabe commented on HADOOP-9484: -- Hi Vaibhav, Have you checked out Mahout? http://mahout.apache.org/ bq. The Apache Mahout™ machine learning library's goal is to build scalable machine learning libraries. > Genetic Algorithm Library for Hadoop > > > Key: HADOOP-9484 > URL: https://issues.apache.org/jira/browse/HADOOP-9484 > Project: Hadoop Common > Issue Type: New Feature > Components: contrib/hod > Environment: Linux Operating System >Reporter: Vaibhav Singh Rajput > > Developed a Genetic Algorithm Library for Hadoop. Using this library, > problems using Genetic Algrorithm can be solved based on Hadoop MapReduce. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9450) HADOOP_USER_CLASSPATH_FIRST is not honored; CLASSPATH is PREpended instead of APpended
[
https://issues.apache.org/jira/browse/HADOOP-9450?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13635295#comment-13635295
]
Harsh J commented on HADOOP-9450:
-
Mitch,
One comment though:
bq. Our problem is in the HADOOP_CONF_DIR jars. There are some JARs in there
that our customers are trying to override.
A jar will not be picked if its in a directory thats on the classpath. Classes
under package folders (such as foo/org/apache/bar.class, if "foo" is on
classpath) and other file resources may get picked up but never jars. Jars are
only picked up if they are directly placed onto the classpath (either via a
wildcard or explicitly). Might be worth checking your environment again for
whats really causing an issue.
> HADOOP_USER_CLASSPATH_FIRST is not honored; CLASSPATH is PREpended instead of
> APpended
> --
>
> Key: HADOOP-9450
> URL: https://issues.apache.org/jira/browse/HADOOP-9450
> Project: Hadoop Common
> Issue Type: Improvement
> Components: scripts
>Reporter: Mitch Wyle
>Assignee: Harsh J
> Fix For: 3.0.0, 1-win, 2.0.5-beta, 1.3.0
>
> Attachments: HADOOP-9450-branch-1.patch,
> HADOOP-9450-branch-1-win.patch, HADOOP-9450-branch-2.patch,
> HADOOP-9450.patch, HADOOP-9450.patch
>
>
> On line 133 of the hadoop shell wrapper, CLASSPATH is set as:
> CLASSPATH=${CLASSPATH}:${HADOOP_CLASSPATH}
> Notice that the built-up CLASSPATH, along with all the libs and unwanted JARS
> are pre-pended BEFORE the user's HADOOP_CLASSPATH. Therefore there is no way
> to put your own JARs in front of those that the hadoop wrapper script sets.
> We propose a patch that reverses this order. Failing that, we would like to
> add a command line option to override this behavior and enable a user's JARs
> to be found before the wrong ones in the Hadoop library paths.
> We always welcome your opinions.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9371) Define Semantics of FileSystem and FileContext more rigorously
[
https://issues.apache.org/jira/browse/HADOOP-9371?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13635267#comment-13635267
]
Steve Loughran commented on HADOOP-9371:
note that {{BufferedFSInputStream}} doesn't meet this spec as it treats a
negative seek as a no-op:
{code}
public void seek(long pos) throws IOException {
if( pos<0 ) {
return;
}
{code}
> Define Semantics of FileSystem and FileContext more rigorously
> --
>
> Key: HADOOP-9371
> URL: https://issues.apache.org/jira/browse/HADOOP-9371
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs
>Affects Versions: 1.2.0, 3.0.0
>Reporter: Steve Loughran
>Assignee: Steve Loughran
> Attachments: HADOOP-9361.2.patch, HADOOP-9361.patch,
> HadoopFilesystemContract.pdf
>
> Original Estimate: 48h
> Remaining Estimate: 48h
>
> The semantics of {{FileSystem}} and {{FileContext}} are not completely
> defined in terms of
> # core expectations of a filesystem
> # consistency requirements.
> # concurrency requirements.
> # minimum scale limits
> Furthermore, methods are not defined strictly enough in terms of their
> outcomes and failure modes.
> The requirements and method semantics should be defined more strictly.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9450) HADOOP_USER_CLASSPATH_FIRST is not honored; CLASSPATH is PREpended instead of APpended
[
https://issues.apache.org/jira/browse/HADOOP-9450?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13635256#comment-13635256
]
Chris Nauroth commented on HADOOP-9450:
---
Great, thank you for the commit Harsh!
Mitch, thank you for providing the detailed bug report.
> HADOOP_USER_CLASSPATH_FIRST is not honored; CLASSPATH is PREpended instead of
> APpended
> --
>
> Key: HADOOP-9450
> URL: https://issues.apache.org/jira/browse/HADOOP-9450
> Project: Hadoop Common
> Issue Type: Improvement
> Components: scripts
>Reporter: Mitch Wyle
>Assignee: Harsh J
> Fix For: 3.0.0, 1-win, 2.0.5-beta, 1.3.0
>
> Attachments: HADOOP-9450-branch-1.patch,
> HADOOP-9450-branch-1-win.patch, HADOOP-9450-branch-2.patch,
> HADOOP-9450.patch, HADOOP-9450.patch
>
>
> On line 133 of the hadoop shell wrapper, CLASSPATH is set as:
> CLASSPATH=${CLASSPATH}:${HADOOP_CLASSPATH}
> Notice that the built-up CLASSPATH, along with all the libs and unwanted JARS
> are pre-pended BEFORE the user's HADOOP_CLASSPATH. Therefore there is no way
> to put your own JARs in front of those that the hadoop wrapper script sets.
> We propose a patch that reverses this order. Failing that, we would like to
> add a command line option to override this behavior and enable a user's JARs
> to be found before the wrong ones in the Hadoop library paths.
> We always welcome your opinions.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9469) mapreduce/yarn source jars not included in dist tarball
[
https://issues.apache.org/jira/browse/HADOOP-9469?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13635237#comment-13635237
]
Hadoop QA commented on HADOOP-9469:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12579336/HADOOP-9469-branch-2.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:red}-1 tests included{color}. The patch doesn't appear to include
any new or modified tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in
hadoop-assemblies.
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results:
https://builds.apache.org/job/PreCommit-HADOOP-Build/2456//testReport/
Console output:
https://builds.apache.org/job/PreCommit-HADOOP-Build/2456//console
This message is automatically generated.
> mapreduce/yarn source jars not included in dist tarball
> ---
>
> Key: HADOOP-9469
> URL: https://issues.apache.org/jira/browse/HADOOP-9469
> Project: Hadoop Common
> Issue Type: Bug
>Affects Versions: 0.23.0
>Reporter: Thomas Graves
>Assignee: Robert Parker
> Attachments: HADOOP-9469-branch-0.23.patch,
> HADOOP-9469-branch-2.patch, HADOOP-9469.patch, HADOOP-9469.patch
>
>
> the mapreduce and yarn sources jars don't get included into the distribution
> tarball. It seems they get built by default just aren't assembled.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9407) commons-daemon 1.0.3 dependency has bad group id causing build issues
[
https://issues.apache.org/jira/browse/HADOOP-9407?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13635239#comment-13635239
]
Sangjin Lee commented on HADOOP-9407:
-
[~cos] +1. What needs to be done for that to happen? I raised it during the
first voting for 2.0.4-alpha. Thanks!
> commons-daemon 1.0.3 dependency has bad group id causing build issues
> -
>
> Key: HADOOP-9407
> URL: https://issues.apache.org/jira/browse/HADOOP-9407
> Project: Hadoop Common
> Issue Type: Bug
> Components: build
>Affects Versions: 2.0.3-alpha, 2.0.4-alpha
>Reporter: Sangjin Lee
>Assignee: Sangjin Lee
> Labels: 2.0.4
> Fix For: 2.0.5-beta
>
> Attachments: HDFS-4497.patch
>
>
> The commons-daemon dependency of the hadoop-hdfs module has been at version
> 1.0.3 for a while. However, 1.0.3 has a pretty well-known groupId error in
> its pom ("org.apache.commons" as opposed to "commons-daemon"). This problem
> has since been corrected on commons-daemon starting 1.0.4.
> This causes build problems for many who depend on hadoop-hdfs directly and
> indirectly, however. Maven can skip over this metadata inconsistency. But
> other less forgiving build systems such as ivy and gradle have much harder
> time working around this problem. For example, in gradle, pretty much the
> only obvious way to work around this is to override this dependency version.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-9469) mapreduce/yarn source jars not included in dist tarball
[ https://issues.apache.org/jira/browse/HADOOP-9469?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Robert Parker updated HADOOP-9469: -- Attachment: HADOOP-9469-branch-2.patch HADOOP-9469-branch-0.23.patch > mapreduce/yarn source jars not included in dist tarball > --- > > Key: HADOOP-9469 > URL: https://issues.apache.org/jira/browse/HADOOP-9469 > Project: Hadoop Common > Issue Type: Bug >Affects Versions: 0.23.0 >Reporter: Thomas Graves >Assignee: Robert Parker > Attachments: HADOOP-9469-branch-0.23.patch, > HADOOP-9469-branch-2.patch, HADOOP-9469.patch, HADOOP-9469.patch > > > the mapreduce and yarn sources jars don't get included into the distribution > tarball. It seems they get built by default just aren't assembled. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9371) Define Semantics of FileSystem and FileContext more rigorously
[
https://issues.apache.org/jira/browse/HADOOP-9371?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13635220#comment-13635220
]
Steve Loughran commented on HADOOP-9371:
We also need to specify {{Seekable}}, as the {{FSDataInputStream}} which must
be returned from {{open()}} calls implement it, and the specifics of
{{seek(long pos)}} are not completely defined, consistently implemented, or
explicitly tested.
* some implementation classes validate the range of a seek in the call; it can
also be postponed until the next read() (which is how Posix expects it).
* Not everything rejects negative seek offsets
* While {{EOFException}} would be the appropriate exception to raise on going
past the end of the file, it is rarely to be seen in the source.
Delayed seeks can deliver tangible performance benefits and it would be unwise
to demand stricter validation than {{::lseek()}} or {{::SetFilePointerEx()}}.
We ought to say "you can if you want", and write tests that verify either the
seek fails, or the read straight afterwards fails.
== Seekable ==
* When a file is opened, {{getPos()}} MUST equal 0
* Implementations MAY NOT implement {{seek()}}, and instead MAY throw an
{{IOException}}
* A {{seek(L)}} on a closed input stream MUST fail with an {{IOException}}.
* After a successful {{seek(L)}}, {{getPos()==L}} for all L: {{0 =< L <
length(file)}}
* On a {{seek(L)}} with L<0 an MUST be thrown. It SHOULD be an {{IOException}}.
It MAY be {{IllegalArgumentException}} or other {{RuntimeException}}
* On a {{seek(L)}} with L>length(file), an {{IOException}} MAY be thrown. It
SHOULD be an {{EndOfFileException}}
* If an {{IOException}} is not thrown, then an {{IOException}} MUST be thrown
on the next {{read()}} operation. It SHOULD be an {{EndOfFileException}}
This is actually a relaxation of the {{Seekable.seek()}} definition, which
states "Can't seek past the end of the file.". The {{RawLocalFileSystem}} on
which everything ultimately depends does support seeking past the end of the
file -it is only on the read operation where an exception is raised.
* After a {{seek(L)}} with {{L 0}}, verify {{getPos()==0}}
# {{seek(file_len)}}, verify {{getPos()==file_len}}
If an exception is not raised, read() and expect an {{IOException}} exception
# {{seek(file_len+1)}}, expect an {{EOFException}}
If an exception is not raised, read() and expect the exception then
# seek(-1), expect an {{IOException}} immediately.
open a file of length {{file_len == 0}}
# verify {{getPos()==0}}
# Verify that {{seek(0)}} succeeds.
# verify that {{read()}} returns -1.
Test to verify {{seek()}} actually changes the location for future reads.
* verify that after a {{seek()}}, {{read()}} returns the data at the seek
location. This must work for forward and backwards seeks.
* verify that after a {{seek()}}, a {{read(byte[])}} returns the bytes of data
at the seek location. This must work for forward and backwards seeks.]
Repeat for very large offsets (e.g. 128KB file), to ensure that filesystems
with local caches/buffers handle longer range seeks correctly.
> Define Semantics of FileSystem and FileContext more rigorously
> --
>
> Key: HADOOP-9371
> URL: https://issues.apache.org/jira/browse/HADOOP-9371
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs
>Affects Versions: 1.2.0, 3.0.0
>Reporter: Steve Loughran
>Assignee: Steve Loughran
> Attachments: HADOOP-9361.2.patch, HADOOP-9361.patch,
> HadoopFilesystemContract.pdf
>
> Original Estimate: 48h
> Remaining Estimate: 48h
>
> The semantics of {{FileSystem}} and {{FileContext}} are not completely
> defined in terms of
> # core expectations of a filesystem
> # consistency requirements.
> # concurrency requirements.
> # minimum scale limits
> Furthermore, methods are not defined strictly enough in terms of their
> outcomes and failure modes.
> The requirements and method semantics should be defined more strictly.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9450) HADOOP_USER_CLASSPATH_FIRST is not honored; CLASSPATH is PREpended instead of APpended
[
https://issues.apache.org/jira/browse/HADOOP-9450?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13635175#comment-13635175
]
Hudson commented on HADOOP-9450:
Integrated in Hadoop-Mapreduce-trunk #1403 (See
[https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1403/])
HADOOP-9450. HADOOP_USER_CLASSPATH_FIRST is not honored; CLASSPATH is
PREpended instead of APpended. Contributed by Chris Nauroth and Harsh J.
(harsh) (Revision 1469214)
Result = SUCCESS
harsh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1469214
Files :
* /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
*
/hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/bin/hadoop-config.cmd
*
/hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/bin/hadoop-config.sh
> HADOOP_USER_CLASSPATH_FIRST is not honored; CLASSPATH is PREpended instead of
> APpended
> --
>
> Key: HADOOP-9450
> URL: https://issues.apache.org/jira/browse/HADOOP-9450
> Project: Hadoop Common
> Issue Type: Improvement
> Components: scripts
>Reporter: Mitch Wyle
>Assignee: Harsh J
> Fix For: 3.0.0, 1-win, 2.0.5-beta, 1.3.0
>
> Attachments: HADOOP-9450-branch-1.patch,
> HADOOP-9450-branch-1-win.patch, HADOOP-9450-branch-2.patch,
> HADOOP-9450.patch, HADOOP-9450.patch
>
>
> On line 133 of the hadoop shell wrapper, CLASSPATH is set as:
> CLASSPATH=${CLASSPATH}:${HADOOP_CLASSPATH}
> Notice that the built-up CLASSPATH, along with all the libs and unwanted JARS
> are pre-pended BEFORE the user's HADOOP_CLASSPATH. Therefore there is no way
> to put your own JARs in front of those that the hadoop wrapper script sets.
> We propose a patch that reverses this order. Failing that, we would like to
> add a command line option to override this behavior and enable a user's JARs
> to be found before the wrong ones in the Hadoop library paths.
> We always welcome your opinions.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9450) HADOOP_USER_CLASSPATH_FIRST is not honored; CLASSPATH is PREpended instead of APpended
[
https://issues.apache.org/jira/browse/HADOOP-9450?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13635132#comment-13635132
]
Hudson commented on HADOOP-9450:
Integrated in Hadoop-Hdfs-trunk #1376 (See
[https://builds.apache.org/job/Hadoop-Hdfs-trunk/1376/])
HADOOP-9450. HADOOP_USER_CLASSPATH_FIRST is not honored; CLASSPATH is
PREpended instead of APpended. Contributed by Chris Nauroth and Harsh J.
(harsh) (Revision 1469214)
Result = FAILURE
harsh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1469214
Files :
* /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
*
/hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/bin/hadoop-config.cmd
*
/hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/bin/hadoop-config.sh
> HADOOP_USER_CLASSPATH_FIRST is not honored; CLASSPATH is PREpended instead of
> APpended
> --
>
> Key: HADOOP-9450
> URL: https://issues.apache.org/jira/browse/HADOOP-9450
> Project: Hadoop Common
> Issue Type: Improvement
> Components: scripts
>Reporter: Mitch Wyle
>Assignee: Harsh J
> Fix For: 3.0.0, 1-win, 2.0.5-beta, 1.3.0
>
> Attachments: HADOOP-9450-branch-1.patch,
> HADOOP-9450-branch-1-win.patch, HADOOP-9450-branch-2.patch,
> HADOOP-9450.patch, HADOOP-9450.patch
>
>
> On line 133 of the hadoop shell wrapper, CLASSPATH is set as:
> CLASSPATH=${CLASSPATH}:${HADOOP_CLASSPATH}
> Notice that the built-up CLASSPATH, along with all the libs and unwanted JARS
> are pre-pended BEFORE the user's HADOOP_CLASSPATH. Therefore there is no way
> to put your own JARs in front of those that the hadoop wrapper script sets.
> We propose a patch that reverses this order. Failing that, we would like to
> add a command line option to override this behavior and enable a user's JARs
> to be found before the wrong ones in the Hadoop library paths.
> We always welcome your opinions.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9450) HADOOP_USER_CLASSPATH_FIRST is not honored; CLASSPATH is PREpended instead of APpended
[
https://issues.apache.org/jira/browse/HADOOP-9450?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13635076#comment-13635076
]
Hudson commented on HADOOP-9450:
Integrated in Hadoop-Yarn-trunk #187 (See
[https://builds.apache.org/job/Hadoop-Yarn-trunk/187/])
HADOOP-9450. HADOOP_USER_CLASSPATH_FIRST is not honored; CLASSPATH is
PREpended instead of APpended. Contributed by Chris Nauroth and Harsh J.
(harsh) (Revision 1469214)
Result = FAILURE
harsh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1469214
Files :
* /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
*
/hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/bin/hadoop-config.cmd
*
/hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/bin/hadoop-config.sh
> HADOOP_USER_CLASSPATH_FIRST is not honored; CLASSPATH is PREpended instead of
> APpended
> --
>
> Key: HADOOP-9450
> URL: https://issues.apache.org/jira/browse/HADOOP-9450
> Project: Hadoop Common
> Issue Type: Improvement
> Components: scripts
>Reporter: Mitch Wyle
>Assignee: Harsh J
> Fix For: 3.0.0, 1-win, 2.0.5-beta, 1.3.0
>
> Attachments: HADOOP-9450-branch-1.patch,
> HADOOP-9450-branch-1-win.patch, HADOOP-9450-branch-2.patch,
> HADOOP-9450.patch, HADOOP-9450.patch
>
>
> On line 133 of the hadoop shell wrapper, CLASSPATH is set as:
> CLASSPATH=${CLASSPATH}:${HADOOP_CLASSPATH}
> Notice that the built-up CLASSPATH, along with all the libs and unwanted JARS
> are pre-pended BEFORE the user's HADOOP_CLASSPATH. Therefore there is no way
> to put your own JARs in front of those that the hadoop wrapper script sets.
> We propose a patch that reverses this order. Failing that, we would like to
> add a command line option to override this behavior and enable a user's JARs
> to be found before the wrong ones in the Hadoop library paths.
> We always welcome your opinions.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9482) regression: unable to create a file in the file:/// filesystem unless HADOOP_HOME_DIR or hadoop.home.dir is set
[
https://issues.apache.org/jira/browse/HADOOP-9482?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13635068#comment-13635068
]
Steve Loughran commented on HADOOP-9482:
I just grabbed that JIRA reference from the commit logs -HADOOP-8562 looks like
the root cause.
This also looks like an instance of HADOOP-9422: file creations calls
{{chmod()}}, which uses shell, which failed.
Looking at the source (my local branch/fork for HADOOP-8545, not been rebased
for a while), the IOE is thrown and then caught; a clean mvn install stops this
problem re-occurring. I think my machine may have been contaminated by some
other snapshot of the hadoop-common JAR.
> regression: unable to create a file in the file:/// filesystem unless
> HADOOP_HOME_DIR or hadoop.home.dir is set
> ---
>
> Key: HADOOP-9482
> URL: https://issues.apache.org/jira/browse/HADOOP-9482
> Project: Hadoop Common
> Issue Type: Bug
> Components: util
>Affects Versions: 2.0.5-beta
>Reporter: Steve Loughran
>Priority: Critical
>
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9450) HADOOP_USER_CLASSPATH_FIRST is not honored; CLASSPATH is PREpended instead of APpended
[
https://issues.apache.org/jira/browse/HADOOP-9450?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13634984#comment-13634984
]
Harsh J commented on HADOOP-9450:
-
Thanks Chris for the backports; and Nicholas for the additional review!
> HADOOP_USER_CLASSPATH_FIRST is not honored; CLASSPATH is PREpended instead of
> APpended
> --
>
> Key: HADOOP-9450
> URL: https://issues.apache.org/jira/browse/HADOOP-9450
> Project: Hadoop Common
> Issue Type: Improvement
> Components: scripts
>Reporter: Mitch Wyle
>Assignee: Harsh J
> Fix For: 3.0.0, 1-win, 2.0.5-beta, 1.3.0
>
> Attachments: HADOOP-9450-branch-1.patch,
> HADOOP-9450-branch-1-win.patch, HADOOP-9450-branch-2.patch,
> HADOOP-9450.patch, HADOOP-9450.patch
>
>
> On line 133 of the hadoop shell wrapper, CLASSPATH is set as:
> CLASSPATH=${CLASSPATH}:${HADOOP_CLASSPATH}
> Notice that the built-up CLASSPATH, along with all the libs and unwanted JARS
> are pre-pended BEFORE the user's HADOOP_CLASSPATH. Therefore there is no way
> to put your own JARs in front of those that the hadoop wrapper script sets.
> We propose a patch that reverses this order. Failing that, we would like to
> add a command line option to override this behavior and enable a user's JARs
> to be found before the wrong ones in the Hadoop library paths.
> We always welcome your opinions.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-9450) HADOOP_USER_CLASSPATH_FIRST is not honored; CLASSPATH is PREpended instead of APpended
[
https://issues.apache.org/jira/browse/HADOOP-9450?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Harsh J updated HADOOP-9450:
Resolution: Fixed
Fix Version/s: 1.3.0
2.0.5-beta
1-win
3.0.0
Status: Resolved (was: Patch Available)
Committed to branch-1, branch-1-win, branch-2, and trunk.
Although usually not necessary, am noting 3.0.0 as a fix version here
specifically to indicate that the fix on trunk carried Windows changes but
branch-2 (2.0.5) does not, cause Windows support had not been merged to it.
> HADOOP_USER_CLASSPATH_FIRST is not honored; CLASSPATH is PREpended instead of
> APpended
> --
>
> Key: HADOOP-9450
> URL: https://issues.apache.org/jira/browse/HADOOP-9450
> Project: Hadoop Common
> Issue Type: Improvement
> Components: scripts
>Reporter: Mitch Wyle
>Assignee: Harsh J
> Fix For: 3.0.0, 1-win, 2.0.5-beta, 1.3.0
>
> Attachments: HADOOP-9450-branch-1.patch,
> HADOOP-9450-branch-1-win.patch, HADOOP-9450-branch-2.patch,
> HADOOP-9450.patch, HADOOP-9450.patch
>
>
> On line 133 of the hadoop shell wrapper, CLASSPATH is set as:
> CLASSPATH=${CLASSPATH}:${HADOOP_CLASSPATH}
> Notice that the built-up CLASSPATH, along with all the libs and unwanted JARS
> are pre-pended BEFORE the user's HADOOP_CLASSPATH. Therefore there is no way
> to put your own JARs in front of those that the hadoop wrapper script sets.
> We propose a patch that reverses this order. Failing that, we would like to
> add a command line option to override this behavior and enable a user's JARs
> to be found before the wrong ones in the Hadoop library paths.
> We always welcome your opinions.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9450) HADOOP_USER_CLASSPATH_FIRST is not honored; CLASSPATH is PREpended instead of APpended
[
https://issues.apache.org/jira/browse/HADOOP-9450?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13634977#comment-13634977
]
Hadoop QA commented on HADOOP-9450:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12579287/HADOOP-9450-branch-2.patch
against trunk revision .
{color:red}-1 patch{color}. The patch command could not apply the patch.
Console output:
https://builds.apache.org/job/PreCommit-HADOOP-Build/2455//console
This message is automatically generated.
> HADOOP_USER_CLASSPATH_FIRST is not honored; CLASSPATH is PREpended instead of
> APpended
> --
>
> Key: HADOOP-9450
> URL: https://issues.apache.org/jira/browse/HADOOP-9450
> Project: Hadoop Common
> Issue Type: Improvement
> Components: scripts
>Reporter: Mitch Wyle
>Assignee: Harsh J
> Attachments: HADOOP-9450-branch-1.patch,
> HADOOP-9450-branch-1-win.patch, HADOOP-9450-branch-2.patch,
> HADOOP-9450.patch, HADOOP-9450.patch
>
>
> On line 133 of the hadoop shell wrapper, CLASSPATH is set as:
> CLASSPATH=${CLASSPATH}:${HADOOP_CLASSPATH}
> Notice that the built-up CLASSPATH, along with all the libs and unwanted JARS
> are pre-pended BEFORE the user's HADOOP_CLASSPATH. Therefore there is no way
> to put your own JARs in front of those that the hadoop wrapper script sets.
> We propose a patch that reverses this order. Failing that, we would like to
> add a command line option to override this behavior and enable a user's JARs
> to be found before the wrong ones in the Hadoop library paths.
> We always welcome your opinions.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9450) HADOOP_USER_CLASSPATH_FIRST is not honored; CLASSPATH is PREpended instead of APpended
[
https://issues.apache.org/jira/browse/HADOOP-9450?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13634971#comment-13634971
]
Hudson commented on HADOOP-9450:
Integrated in Hadoop-trunk-Commit #3626 (See
[https://builds.apache.org/job/Hadoop-trunk-Commit/3626/])
HADOOP-9450. HADOOP_USER_CLASSPATH_FIRST is not honored; CLASSPATH is
PREpended instead of APpended. Contributed by Chris Nauroth and Harsh J.
(harsh) (Revision 1469214)
Result = SUCCESS
harsh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1469214
Files :
* /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
*
/hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/bin/hadoop-config.cmd
*
/hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/bin/hadoop-config.sh
> HADOOP_USER_CLASSPATH_FIRST is not honored; CLASSPATH is PREpended instead of
> APpended
> --
>
> Key: HADOOP-9450
> URL: https://issues.apache.org/jira/browse/HADOOP-9450
> Project: Hadoop Common
> Issue Type: Improvement
> Components: scripts
>Reporter: Mitch Wyle
>Assignee: Harsh J
> Attachments: HADOOP-9450-branch-1.patch,
> HADOOP-9450-branch-1-win.patch, HADOOP-9450-branch-2.patch,
> HADOOP-9450.patch, HADOOP-9450.patch
>
>
> On line 133 of the hadoop shell wrapper, CLASSPATH is set as:
> CLASSPATH=${CLASSPATH}:${HADOOP_CLASSPATH}
> Notice that the built-up CLASSPATH, along with all the libs and unwanted JARS
> are pre-pended BEFORE the user's HADOOP_CLASSPATH. Therefore there is no way
> to put your own JARs in front of those that the hadoop wrapper script sets.
> We propose a patch that reverses this order. Failing that, we would like to
> add a command line option to override this behavior and enable a user's JARs
> to be found before the wrong ones in the Hadoop library paths.
> We always welcome your opinions.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-9450) HADOOP_USER_CLASSPATH_FIRST is not honored; CLASSPATH is PREpended instead of APpended
[
https://issues.apache.org/jira/browse/HADOOP-9450?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Harsh J updated HADOOP-9450:
Attachment: HADOOP-9450-branch-2.patch
Attaching the committed branch-2 patch as no Windows changes are present there
yet.
> HADOOP_USER_CLASSPATH_FIRST is not honored; CLASSPATH is PREpended instead of
> APpended
> --
>
> Key: HADOOP-9450
> URL: https://issues.apache.org/jira/browse/HADOOP-9450
> Project: Hadoop Common
> Issue Type: Improvement
> Components: scripts
>Reporter: Mitch Wyle
>Assignee: Harsh J
> Attachments: HADOOP-9450-branch-1.patch,
> HADOOP-9450-branch-1-win.patch, HADOOP-9450-branch-2.patch,
> HADOOP-9450.patch, HADOOP-9450.patch
>
>
> On line 133 of the hadoop shell wrapper, CLASSPATH is set as:
> CLASSPATH=${CLASSPATH}:${HADOOP_CLASSPATH}
> Notice that the built-up CLASSPATH, along with all the libs and unwanted JARS
> are pre-pended BEFORE the user's HADOOP_CLASSPATH. Therefore there is no way
> to put your own JARs in front of those that the hadoop wrapper script sets.
> We propose a patch that reverses this order. Failing that, we would like to
> add a command line option to override this behavior and enable a user's JARs
> to be found before the wrong ones in the Hadoop library paths.
> We always welcome your opinions.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-9450) HADOOP_USER_CLASSPATH_FIRST is not honored; CLASSPATH is PREpended instead of APpended
[
https://issues.apache.org/jira/browse/HADOOP-9450?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Harsh J updated HADOOP-9450:
Issue Type: Improvement (was: Bug)
> HADOOP_USER_CLASSPATH_FIRST is not honored; CLASSPATH is PREpended instead of
> APpended
> --
>
> Key: HADOOP-9450
> URL: https://issues.apache.org/jira/browse/HADOOP-9450
> Project: Hadoop Common
> Issue Type: Improvement
> Components: scripts
>Reporter: Mitch Wyle
>Assignee: Harsh J
> Attachments: HADOOP-9450-branch-1.patch,
> HADOOP-9450-branch-1-win.patch, HADOOP-9450.patch, HADOOP-9450.patch
>
>
> On line 133 of the hadoop shell wrapper, CLASSPATH is set as:
> CLASSPATH=${CLASSPATH}:${HADOOP_CLASSPATH}
> Notice that the built-up CLASSPATH, along with all the libs and unwanted JARS
> are pre-pended BEFORE the user's HADOOP_CLASSPATH. Therefore there is no way
> to put your own JARs in front of those that the hadoop wrapper script sets.
> We propose a patch that reverses this order. Failing that, we would like to
> add a command line option to override this behavior and enable a user's JARs
> to be found before the wrong ones in the Hadoop library paths.
> We always welcome your opinions.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9043) winutils can create unusable symlinks
[ https://issues.apache.org/jira/browse/HADOOP-9043?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13634939#comment-13634939 ] Ivan Mitic commented on HADOOP-9043: Looks good, +1 on both trunk and branch-1-win patches. Thanks! > winutils can create unusable symlinks > - > > Key: HADOOP-9043 > URL: https://issues.apache.org/jira/browse/HADOOP-9043 > Project: Hadoop Common > Issue Type: Bug > Components: util >Affects Versions: 3.0.0, 1-win >Reporter: Chris Nauroth >Assignee: Arpit Agarwal > Fix For: 3.0.0, 1-win > > Attachments: HADOOP-9043.branch-1.2.patch, > HADOOP-9043.branch-1-win.5.patch, HADOOP-9043.branch-1-win.patch, > HADOOP-9043.trunk.2.patch, HADOOP-9043.trunk.3.patch, > HADOOP-9043.trunk.4.patch, HADOOP-9043.trunk.5.patch, HADOOP-9043.trunk.patch > > > In general, the winutils symlink command rejects attempts to create symlinks > targeting a destination file that does not exist. However, if given a > symlink destination with forward slashes pointing at a file that does exist, > then it creates the symlink with the forward slashes, and then attempts to > open the file through the symlink will fail. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
