[jira] [Commented] (HADOOP-8562) Enhancements to Hadoop for Windows Server and Windows Azure development and runtime environments
[
https://issues.apache.org/jira/browse/HADOOP-8562?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13533707#comment-13533707
]
Hadoop QA commented on HADOOP-8562:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12561257/test-untar.tgz
against trunk revision .
{color:red}-1 patch{color}. The patch command could not apply the patch.
Console output:
https://builds.apache.org/job/PreCommit-HADOOP-Build/1897//console
This message is automatically generated.
> Enhancements to Hadoop for Windows Server and Windows Azure development and
> runtime environments
>
>
> Key: HADOOP-8562
> URL: https://issues.apache.org/jira/browse/HADOOP-8562
> Project: Hadoop Common
> Issue Type: New Feature
>Affects Versions: 3.0.0
>Reporter: Bikas Saha
>Assignee: Bikas Saha
> Attachments: branch-trunk-win.patch, branch-trunk-win.patch,
> branch-trunk-win.patch, branch-trunk-win.patch, test-untar.tar, test-untar.tgz
>
>
> This JIRA tracks the work that needs to be done on trunk to enable Hadoop to
> run on Windows Server and Azure environments. This incorporates porting
> relevant work from the similar effort on branch 1 tracked via HADOOP-8079.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-8562) Enhancements to Hadoop for Windows Server and Windows Azure development and runtime environments
[
https://issues.apache.org/jira/browse/HADOOP-8562?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chris Nauroth updated HADOOP-8562:
--
Attachment: test-untar.tgz
test-untar.tar
The failure in {{TestFileUtil}} is caused by 2 missing tar files required for
testing untar functionality. Patch files don't seem to have a good way to
carry over binary files, so we'll need to add them manually with the merge. I
have attached test-untar.tar and test-untar.tgz. These must be committed to
hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs when
it's time.
All of the streaming failures are unrelated to the merge. The current trunk
code has the same problem. I submitted a patch on MAPREDUCE-4884 to fix it.
For the failures in {{TestContainerManager}}, {{TestContainerLaunch}}, and
{{TestContainersMonitor}}, I still don't have a repro or an explanation. It
seems to be only happening on Jenkins. These need further investigation.
> Enhancements to Hadoop for Windows Server and Windows Azure development and
> runtime environments
>
>
> Key: HADOOP-8562
> URL: https://issues.apache.org/jira/browse/HADOOP-8562
> Project: Hadoop Common
> Issue Type: New Feature
>Affects Versions: 3.0.0
>Reporter: Bikas Saha
>Assignee: Bikas Saha
> Attachments: branch-trunk-win.patch, branch-trunk-win.patch,
> branch-trunk-win.patch, branch-trunk-win.patch, test-untar.tar, test-untar.tgz
>
>
> This JIRA tracks the work that needs to be done on trunk to enable Hadoop to
> run on Windows Server and Azure environments. This incorporates porting
> relevant work from the similar effort on branch 1 tracked via HADOOP-8079.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9149) During RPC, client will call the setupConnection() in the Client.java. We want that the connection time out can be configured int the core-site.xml
[
https://issues.apache.org/jira/browse/HADOOP-9149?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13533677#comment-13533677
]
liuyang commented on HADOOP-9149:
-
In Client.java, the process is as follow:
// connection time out is 20s
NetUtils.connect(this.socket, server, 2);
if (rpcTimeout > 0) {
pingInterval = rpcTimeout; // rpcTimeout overwrites pingInterval
}
this.socket.setSoTimeout(pingInterval);
return;
> During RPC, client will call the setupConnection() in the Client.java. We
> want that the connection time out can be configured int the core-site.xml
> ---
>
> Key: HADOOP-9149
> URL: https://issues.apache.org/jira/browse/HADOOP-9149
> Project: Hadoop Common
> Issue Type: Improvement
> Components: ipc
>Affects Versions: 2.0.1-alpha
>Reporter: liuyang
>Priority: Minor
>
> When optimize the hadoop cluster, we want the connection time out can be
> configured in the core-site.xml.
> In different scale hadoop cluster, client will take different connection time
> out during create connection to remote machine!
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HADOOP-9149) During RPC, client will call the setupConnection() in the Client.java. We want that the connection time out can be configured int the core-site.xml
liuyang created HADOOP-9149: --- Summary: During RPC, client will call the setupConnection() in the Client.java. We want that the connection time out can be configured int the core-site.xml Key: HADOOP-9149 URL: https://issues.apache.org/jira/browse/HADOOP-9149 Project: Hadoop Common Issue Type: Improvement Components: ipc Affects Versions: 2.0.1-alpha Reporter: liuyang Priority: Minor When optimize the hadoop cluster, we want the connection time out can be configured in the core-site.xml. In different scale hadoop cluster, client will take different connection time out during create connection to remote machine! -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8562) Enhancements to Hadoop for Windows Server and Windows Azure development and runtime environments
[
https://issues.apache.org/jira/browse/HADOOP-8562?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13533601#comment-13533601
]
Hadoop QA commented on HADOOP-8562:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12561221/branch-trunk-win.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 26 new
or modified test files.
{color:red}-1 javac{color}. The applied patch generated 2016 javac
compiler warnings (more than the trunk's current 2013 warnings).
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:red}-1 core tests{color}. The patch failed these unit tests in
hadoop-assemblies hadoop-common-project/hadoop-common hadoop-dist
hadoop-hdfs-project/hadoop-hdfs hadoop-hdfs-project/hadoop-hdfs-httpfs
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-shuffle
hadoop-tools/hadoop-streaming hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager:
org.apache.hadoop.fs.TestFileUtil
org.apache.hadoop.mapreduce.v2.app.job.impl.TestMapReduceChildJVM
org.apache.hadoop.streaming.TestSymLink
org.apache.hadoop.streaming.TestMultipleCachefiles
org.apache.hadoop.streaming.TestStreamingBadRecords
org.apache.hadoop.streaming.TestFileArgs
org.apache.hadoop.streaming.TestMultipleArchiveFiles
org.apache.hadoop.streaming.TestStreamingTaskLog
org.apache.hadoop.yarn.server.nodemanager.containermanager.TestContainerManager
org.apache.hadoop.yarn.server.nodemanager.TestNodeManagerShutdown
org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.TestContainerLaunch
org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.TestContainersMonitor
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results:
https://builds.apache.org/job/PreCommit-HADOOP-Build/1894//testReport/
Javac warnings:
https://builds.apache.org/job/PreCommit-HADOOP-Build/1894//artifact/trunk/patchprocess/diffJavacWarnings.txt
Console output:
https://builds.apache.org/job/PreCommit-HADOOP-Build/1894//console
This message is automatically generated.
> Enhancements to Hadoop for Windows Server and Windows Azure development and
> runtime environments
>
>
> Key: HADOOP-8562
> URL: https://issues.apache.org/jira/browse/HADOOP-8562
> Project: Hadoop Common
> Issue Type: New Feature
>Affects Versions: 3.0.0
>Reporter: Bikas Saha
>Assignee: Bikas Saha
> Attachments: branch-trunk-win.patch, branch-trunk-win.patch,
> branch-trunk-win.patch, branch-trunk-win.patch
>
>
> This JIRA tracks the work that needs to be done on trunk to enable Hadoop to
> run on Windows Server and Azure environments. This incorporates porting
> relevant work from the similar effort on branch 1 tracked via HADOOP-8079.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9147) Add missing fields to FIleStatus.toString
[
https://issues.apache.org/jira/browse/HADOOP-9147?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13533574#comment-13533574
]
Hadoop QA commented on HADOOP-9147:
---
{color:green}+1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12561223/HADOOP-9147.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 1 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in
hadoop-common-project/hadoop-common.
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results:
https://builds.apache.org/job/PreCommit-HADOOP-Build/1895//testReport/
Console output:
https://builds.apache.org/job/PreCommit-HADOOP-Build/1895//console
This message is automatically generated.
> Add missing fields to FIleStatus.toString
> -
>
> Key: HADOOP-9147
> URL: https://issues.apache.org/jira/browse/HADOOP-9147
> Project: Hadoop Common
> Issue Type: Improvement
>Reporter: Jonathan Allen
>Assignee: Jonathan Allen
>Priority: Trivial
> Attachments: HADOOP-9147.patch, HADOOP-9147.patch, HADOOP-9147.patch
>
>
> The FileStatus.toString method is missing the following fields:
> - modification_time
> - access_time
> - symlink
> These should be added in to aid debugging.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9148) Update CLITestHelper to skip selected tests
[
https://issues.apache.org/jira/browse/HADOOP-9148?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13533569#comment-13533569
]
Hadoop QA commented on HADOOP-9148:
---
{color:green}+1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12561219/HADOOP-9148.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 2 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in
hadoop-common-project/hadoop-common.
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results:
https://builds.apache.org/job/PreCommit-HADOOP-Build/1893//testReport/
Console output:
https://builds.apache.org/job/PreCommit-HADOOP-Build/1893//console
This message is automatically generated.
> Update CLITestHelper to skip selected tests
> ---
>
> Key: HADOOP-9148
> URL: https://issues.apache.org/jira/browse/HADOOP-9148
> Project: Hadoop Common
> Issue Type: Improvement
>Reporter: Jonathan Allen
>Assignee: Jonathan Allen
>Priority: Trivial
> Attachments: HADOOP-9148.patch, HADOOP-9148.patch
>
>
> The CLI test classes (subclasses from CLITestHelper) can take a long time to
> run. This makes it difficult to add new tests as the turnaround time is
> excessive. The CLITestHelper class should be updated with options to allow
> only selected tests to be run. Propose the following options (defined as
> system properties):
> - test.cli.include (comma separated list of regex to match against test
> description field)
> - test.cli.terminate (number of failed tests to allow before terminating the
> run)
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-9147) Add missing fields to FIleStatus.toString
[ https://issues.apache.org/jira/browse/HADOOP-9147?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Allen updated HADOOP-9147: --- Attachment: HADOOP-9147.patch actioned review comments > Add missing fields to FIleStatus.toString > - > > Key: HADOOP-9147 > URL: https://issues.apache.org/jira/browse/HADOOP-9147 > Project: Hadoop Common > Issue Type: Improvement >Reporter: Jonathan Allen >Assignee: Jonathan Allen >Priority: Trivial > Attachments: HADOOP-9147.patch, HADOOP-9147.patch, HADOOP-9147.patch > > > The FileStatus.toString method is missing the following fields: > - modification_time > - access_time > - symlink > These should be added in to aid debugging. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9148) Update CLITestHelper to skip selected tests
[
https://issues.apache.org/jira/browse/HADOOP-9148?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13533561#comment-13533561
]
Hadoop QA commented on HADOOP-9148:
---
{color:green}+1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12561219/HADOOP-9148.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 2 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in
hadoop-common-project/hadoop-common.
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results:
https://builds.apache.org/job/PreCommit-HADOOP-Build/1892//testReport/
Console output:
https://builds.apache.org/job/PreCommit-HADOOP-Build/1892//console
This message is automatically generated.
> Update CLITestHelper to skip selected tests
> ---
>
> Key: HADOOP-9148
> URL: https://issues.apache.org/jira/browse/HADOOP-9148
> Project: Hadoop Common
> Issue Type: Improvement
>Reporter: Jonathan Allen
>Assignee: Jonathan Allen
>Priority: Trivial
> Attachments: HADOOP-9148.patch, HADOOP-9148.patch
>
>
> The CLI test classes (subclasses from CLITestHelper) can take a long time to
> run. This makes it difficult to add new tests as the turnaround time is
> excessive. The CLITestHelper class should be updated with options to allow
> only selected tests to be run. Propose the following options (defined as
> system properties):
> - test.cli.include (comma separated list of regex to match against test
> description field)
> - test.cli.terminate (number of failed tests to allow before terminating the
> run)
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-8562) Enhancements to Hadoop for Windows Server and Windows Azure development and runtime environments
[ https://issues.apache.org/jira/browse/HADOOP-8562?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Suresh Srinivas updated HADOOP-8562: Attachment: branch-trunk-win.patch Updated patch. > Enhancements to Hadoop for Windows Server and Windows Azure development and > runtime environments > > > Key: HADOOP-8562 > URL: https://issues.apache.org/jira/browse/HADOOP-8562 > Project: Hadoop Common > Issue Type: New Feature >Affects Versions: 3.0.0 >Reporter: Bikas Saha >Assignee: Bikas Saha > Attachments: branch-trunk-win.patch, branch-trunk-win.patch, > branch-trunk-win.patch, branch-trunk-win.patch > > > This JIRA tracks the work that needs to be done on trunk to enable Hadoop to > run on Windows Server and Azure environments. This incorporates porting > relevant work from the similar effort on branch 1 tracked via HADOOP-8079. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-9148) Update CLITestHelper to skip selected tests
[ https://issues.apache.org/jira/browse/HADOOP-9148?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Allen updated HADOOP-9148: --- Attachment: HADOOP-9148.patch fixed some formatting issues > Update CLITestHelper to skip selected tests > --- > > Key: HADOOP-9148 > URL: https://issues.apache.org/jira/browse/HADOOP-9148 > Project: Hadoop Common > Issue Type: Improvement >Reporter: Jonathan Allen >Assignee: Jonathan Allen >Priority: Trivial > Attachments: HADOOP-9148.patch, HADOOP-9148.patch > > > The CLI test classes (subclasses from CLITestHelper) can take a long time to > run. This makes it difficult to add new tests as the turnaround time is > excessive. The CLITestHelper class should be updated with options to allow > only selected tests to be run. Propose the following options (defined as > system properties): > - test.cli.include (comma separated list of regex to match against test > description field) > - test.cli.terminate (number of failed tests to allow before terminating the > run) -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-9148) Update CLITestHelper to skip selected tests
[ https://issues.apache.org/jira/browse/HADOOP-9148?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Allen updated HADOOP-9148: --- Attachment: HADOOP-9148.patch Updates to CLITestHelper to allow tests to be skipped during development. Includes the following options (set by system properties): - test.cli.include - comma separated list of regex to be matched against the test description, defaults to run all tests - test.cli.terminate - number of test failures to allow before terminating the run, default to run all tests - test.cli.report.progress - log each as its run or skipped - test.cli.report.detail - display the detail report - test.cli.report.skipped - display the tests skipped report - test.cli.report.failing - display the tests failing report - test.cli.report.passing - display the tests passing report (the report display options added to stop large numbers of tests causing the detail results to scroll out of the history) > Update CLITestHelper to skip selected tests > --- > > Key: HADOOP-9148 > URL: https://issues.apache.org/jira/browse/HADOOP-9148 > Project: Hadoop Common > Issue Type: Improvement >Reporter: Jonathan Allen >Assignee: Jonathan Allen >Priority: Trivial > Attachments: HADOOP-9148.patch > > > The CLI test classes (subclasses from CLITestHelper) can take a long time to > run. This makes it difficult to add new tests as the turnaround time is > excessive. The CLITestHelper class should be updated with options to allow > only selected tests to be run. Propose the following options (defined as > system properties): > - test.cli.include (comma separated list of regex to match against test > description field) > - test.cli.terminate (number of failed tests to allow before terminating the > run) -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-9148) Update CLITestHelper to skip selected tests
[ https://issues.apache.org/jira/browse/HADOOP-9148?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Allen updated HADOOP-9148: --- Release Note: Update CLITestHelper to allow tests to be skipped during development Status: Patch Available (was: Open) > Update CLITestHelper to skip selected tests > --- > > Key: HADOOP-9148 > URL: https://issues.apache.org/jira/browse/HADOOP-9148 > Project: Hadoop Common > Issue Type: Improvement >Reporter: Jonathan Allen >Assignee: Jonathan Allen >Priority: Trivial > Attachments: HADOOP-9148.patch > > > The CLI test classes (subclasses from CLITestHelper) can take a long time to > run. This makes it difficult to add new tests as the turnaround time is > excessive. The CLITestHelper class should be updated with options to allow > only selected tests to be run. Propose the following options (defined as > system properties): > - test.cli.include (comma separated list of regex to match against test > description field) > - test.cli.terminate (number of failed tests to allow before terminating the > run) -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9147) Add missing fields to FIleStatus.toString
[ https://issues.apache.org/jira/browse/HADOOP-9147?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13533543#comment-13533543 ] Suresh Srinivas commented on HADOOP-9147: - Thanks for the patch. Comments: # The toString tests should reuse code. You could have a method validateToString(FileStatus status) that could be used by all the variants of the methods that are testing toString() # Similarly you can have a method for validateAccessors(FileStatus status, len, isDir, replication...) etc. # Please keep your code consistent. If have a blank line before a method, do the same for other methods. # Please add javadoc to every method with a brief description of what the test does. > Add missing fields to FIleStatus.toString > - > > Key: HADOOP-9147 > URL: https://issues.apache.org/jira/browse/HADOOP-9147 > Project: Hadoop Common > Issue Type: Improvement >Reporter: Jonathan Allen >Assignee: Jonathan Allen >Priority: Trivial > Attachments: HADOOP-9147.patch, HADOOP-9147.patch > > > The FileStatus.toString method is missing the following fields: > - modification_time > - access_time > - symlink > These should be added in to aid debugging. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9140) Cleanup rpc PB protos
[ https://issues.apache.org/jira/browse/HADOOP-9140?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13533541#comment-13533541 ] Suresh Srinivas commented on HADOOP-9140: - Comments: # It might be good to call RpcEnginePb.proto to just RpcEngine.proto or ProtobufRpcEngine.proto. I prefer the first one. # RpcHeader.proto - enum RpcRequestOperationProto could be moved into RpcRequestHeaderProto as just OpertionTypeProto. # # RpcHeader.proto - enum RpcKindProto could also be moved into RpcRequestHeaderProto and can be renamed as RpcEngineKind. > Cleanup rpc PB protos > - > > Key: HADOOP-9140 > URL: https://issues.apache.org/jira/browse/HADOOP-9140 > Project: Hadoop Common > Issue Type: Sub-task >Reporter: Sanjay Radia >Assignee: Sanjay Radia > Attachments: Hadoop-9140-2.patch, Hadoop-9140.patch > > -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Comment Edited] (HADOOP-9140) Cleanup rpc PB protos
[ https://issues.apache.org/jira/browse/HADOOP-9140?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13533541#comment-13533541 ] Suresh Srinivas edited comment on HADOOP-9140 at 12/16/12 10:30 PM: Comments: # It might be good to call RpcEnginePb.proto to just RpcEngine.proto or ProtobufRpcEngine.proto. I prefer the first one. # RpcHeader.proto - enum RpcRequestOperationProto could be moved into RpcRequestHeaderProto as just OpertionTypeProto. # RpcHeader.proto - enum RpcKindProto could also be moved into RpcRequestHeaderProto and can be renamed as RpcEngineKind. was (Author: sureshms): Comments: # It might be good to call RpcEnginePb.proto to just RpcEngine.proto or ProtobufRpcEngine.proto. I prefer the first one. # RpcHeader.proto - enum RpcRequestOperationProto could be moved into RpcRequestHeaderProto as just OpertionTypeProto. # # RpcHeader.proto - enum RpcKindProto could also be moved into RpcRequestHeaderProto and can be renamed as RpcEngineKind. > Cleanup rpc PB protos > - > > Key: HADOOP-9140 > URL: https://issues.apache.org/jira/browse/HADOOP-9140 > Project: Hadoop Common > Issue Type: Sub-task >Reporter: Sanjay Radia >Assignee: Sanjay Radia > Attachments: Hadoop-9140-2.patch, Hadoop-9140.patch > > -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9147) Add missing fields to FIleStatus.toString
[
https://issues.apache.org/jira/browse/HADOOP-9147?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13533535#comment-13533535
]
Hadoop QA commented on HADOOP-9147:
---
{color:green}+1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12561208/HADOOP-9147.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 1 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in
hadoop-common-project/hadoop-common.
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results:
https://builds.apache.org/job/PreCommit-HADOOP-Build/1891//testReport/
Console output:
https://builds.apache.org/job/PreCommit-HADOOP-Build/1891//console
This message is automatically generated.
> Add missing fields to FIleStatus.toString
> -
>
> Key: HADOOP-9147
> URL: https://issues.apache.org/jira/browse/HADOOP-9147
> Project: Hadoop Common
> Issue Type: Improvement
>Reporter: Jonathan Allen
>Assignee: Jonathan Allen
>Priority: Trivial
> Attachments: HADOOP-9147.patch, HADOOP-9147.patch
>
>
> The FileStatus.toString method is missing the following fields:
> - modification_time
> - access_time
> - symlink
> These should be added in to aid debugging.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-9147) Add missing fields to FIleStatus.toString
[ https://issues.apache.org/jira/browse/HADOOP-9147?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Allen updated HADOOP-9147: --- Attachment: HADOOP-9147.patch failing test fixed > Add missing fields to FIleStatus.toString > - > > Key: HADOOP-9147 > URL: https://issues.apache.org/jira/browse/HADOOP-9147 > Project: Hadoop Common > Issue Type: Improvement >Reporter: Jonathan Allen >Assignee: Jonathan Allen >Priority: Trivial > Attachments: HADOOP-9147.patch, HADOOP-9147.patch > > > The FileStatus.toString method is missing the following fields: > - modification_time > - access_time > - symlink > These should be added in to aid debugging. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9147) Add missing fields to FIleStatus.toString
[
https://issues.apache.org/jira/browse/HADOOP-9147?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13533503#comment-13533503
]
Hadoop QA commented on HADOOP-9147:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12561205/HADOOP-9147.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 1 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:red}-1 core tests{color}. The patch failed these unit tests in
hadoop-common-project/hadoop-common:
org.apache.hadoop.fs.TestFileStatus
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results:
https://builds.apache.org/job/PreCommit-HADOOP-Build/1890//testReport/
Console output:
https://builds.apache.org/job/PreCommit-HADOOP-Build/1890//console
This message is automatically generated.
> Add missing fields to FIleStatus.toString
> -
>
> Key: HADOOP-9147
> URL: https://issues.apache.org/jira/browse/HADOOP-9147
> Project: Hadoop Common
> Issue Type: Improvement
>Reporter: Jonathan Allen
>Assignee: Jonathan Allen
>Priority: Trivial
> Attachments: HADOOP-9147.patch
>
>
> The FileStatus.toString method is missing the following fields:
> - modification_time
> - access_time
> - symlink
> These should be added in to aid debugging.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HADOOP-9148) Update CLITestHelper to skip selected tests
Jonathan Allen created HADOOP-9148: -- Summary: Update CLITestHelper to skip selected tests Key: HADOOP-9148 URL: https://issues.apache.org/jira/browse/HADOOP-9148 Project: Hadoop Common Issue Type: Improvement Reporter: Jonathan Allen Assignee: Jonathan Allen Priority: Trivial The CLI test classes (subclasses from CLITestHelper) can take a long time to run. This makes it difficult to add new tests as the turnaround time is excessive. The CLITestHelper class should be updated with options to allow only selected tests to be run. Propose the following options (defined as system properties): - test.cli.include (comma separated list of regex to match against test description field) - test.cli.terminate (number of failed tests to allow before terminating the run) -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-9147) Add missing fields to FIleStatus.toString
[ https://issues.apache.org/jira/browse/HADOOP-9147?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Allen updated HADOOP-9147: --- Release Note: Update FileStatus.toString to include missing fields Status: Patch Available (was: Open) > Add missing fields to FIleStatus.toString > - > > Key: HADOOP-9147 > URL: https://issues.apache.org/jira/browse/HADOOP-9147 > Project: Hadoop Common > Issue Type: Improvement >Reporter: Jonathan Allen >Assignee: Jonathan Allen >Priority: Trivial > Attachments: HADOOP-9147.patch > > > The FileStatus.toString method is missing the following fields: > - modification_time > - access_time > - symlink > These should be added in to aid debugging. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-9147) Add missing fields to FIleStatus.toString
[ https://issues.apache.org/jira/browse/HADOOP-9147?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Allen updated HADOOP-9147: --- Attachment: HADOOP-9147.patch ready for code review > Add missing fields to FIleStatus.toString > - > > Key: HADOOP-9147 > URL: https://issues.apache.org/jira/browse/HADOOP-9147 > Project: Hadoop Common > Issue Type: Improvement >Reporter: Jonathan Allen >Assignee: Jonathan Allen >Priority: Trivial > Attachments: HADOOP-9147.patch > > > The FileStatus.toString method is missing the following fields: > - modification_time > - access_time > - symlink > These should be added in to aid debugging. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8989) hadoop dfs -find feature
[
https://issues.apache.org/jira/browse/HADOOP-8989?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13533489#comment-13533489
]
Hadoop QA commented on HADOOP-8989:
---
{color:green}+1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12561197/HADOOP-8989.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 41 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in
hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs.
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results:
https://builds.apache.org/job/PreCommit-HADOOP-Build/1889//testReport/
Console output:
https://builds.apache.org/job/PreCommit-HADOOP-Build/1889//console
This message is automatically generated.
> hadoop dfs -find feature
>
>
> Key: HADOOP-8989
> URL: https://issues.apache.org/jira/browse/HADOOP-8989
> Project: Hadoop Common
> Issue Type: New Feature
>Reporter: Marco Nicosia
>Assignee: Jonathan Allen
> Attachments: HADOOP-8989.patch, HADOOP-8989.patch, HADOOP-8989.patch,
> HADOOP-8989.patch, HADOOP-8989.patch, HADOOP-8989.patch, HADOOP-8989.patch
>
>
> Both sysadmins and users make frequent use of the unix 'find' command, but
> Hadoop has no correlate. Without this, users are writing scripts which make
> heavy use of hadoop dfs -lsr, and implementing find one-offs. I think hdfs
> -lsr is somewhat taxing on the NameNode, and a really slow experience on the
> client side. Possibly an in-NameNode find operation would be only a bit more
> taxing on the NameNode, but significantly faster from the client's point of
> view?
> The minimum set of options I can think of which would make a Hadoop find
> command generally useful is (in priority order):
> * -type (file or directory, for now)
> * -atime/-ctime-mtime (... and -creationtime?) (both + and - arguments)
> * -print0 (for piping to xargs -0)
> * -depth
> * -owner/-group (and -nouser/-nogroup)
> * -name (allowing for shell pattern, or even regex?)
> * -perm
> * -size
> One possible special case, but could possibly be really cool if it ran from
> within the NameNode:
> * -delete
> The "hadoop dfs -lsr | hadoop dfs -rm" cycle is really, really slow.
> Lower priority, some people do use operators, mostly to execute -or searches
> such as:
> * find / \(-nouser -or -nogroup\)
> Finally, I thought I'd include a link to the [Posix spec for
> find|http://www.opengroup.org/onlinepubs/009695399/utilities/find.html]
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HADOOP-9147) Add missing fields to FIleStatus.toString
Jonathan Allen created HADOOP-9147: -- Summary: Add missing fields to FIleStatus.toString Key: HADOOP-9147 URL: https://issues.apache.org/jira/browse/HADOOP-9147 Project: Hadoop Common Issue Type: Improvement Reporter: Jonathan Allen Assignee: Jonathan Allen Priority: Trivial The FileStatus.toString method is missing the following fields: - modification_time - access_time - symlink These should be added in to aid debugging. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-8989) hadoop dfs -find feature
[ https://issues.apache.org/jira/browse/HADOOP-8989?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Allen updated HADOOP-8989: --- Attachment: HADOOP-8989.patch Luke, I assume you mean something like ' Path linkedFile = item.fs.resolvePath(fileStatus.getSymlink());' (or am I missing something?). Patch updated to include this and add some tests for the BaseExpression class. > hadoop dfs -find feature > > > Key: HADOOP-8989 > URL: https://issues.apache.org/jira/browse/HADOOP-8989 > Project: Hadoop Common > Issue Type: New Feature >Reporter: Marco Nicosia >Assignee: Jonathan Allen > Attachments: HADOOP-8989.patch, HADOOP-8989.patch, HADOOP-8989.patch, > HADOOP-8989.patch, HADOOP-8989.patch, HADOOP-8989.patch, HADOOP-8989.patch > > > Both sysadmins and users make frequent use of the unix 'find' command, but > Hadoop has no correlate. Without this, users are writing scripts which make > heavy use of hadoop dfs -lsr, and implementing find one-offs. I think hdfs > -lsr is somewhat taxing on the NameNode, and a really slow experience on the > client side. Possibly an in-NameNode find operation would be only a bit more > taxing on the NameNode, but significantly faster from the client's point of > view? > The minimum set of options I can think of which would make a Hadoop find > command generally useful is (in priority order): > * -type (file or directory, for now) > * -atime/-ctime-mtime (... and -creationtime?) (both + and - arguments) > * -print0 (for piping to xargs -0) > * -depth > * -owner/-group (and -nouser/-nogroup) > * -name (allowing for shell pattern, or even regex?) > * -perm > * -size > One possible special case, but could possibly be really cool if it ran from > within the NameNode: > * -delete > The "hadoop dfs -lsr | hadoop dfs -rm" cycle is really, really slow. > Lower priority, some people do use operators, mostly to execute -or searches > such as: > * find / \(-nouser -or -nogroup\) > Finally, I thought I'd include a link to the [Posix spec for > find|http://www.opengroup.org/onlinepubs/009695399/utilities/find.html] -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8561) Introduce HADOOP_PROXY_USER for secure impersonation in child hadoop client processes
[ https://issues.apache.org/jira/browse/HADOOP-8561?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13533397#comment-13533397 ] Hudson commented on HADOOP-8561: Integrated in Hadoop-Mapreduce-trunk #1287 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1287/]) HADOOP-8561. Introduce HADOOP_PROXY_USER for secure impersonation in child hadoop client processes. (Yu Gao via llu) (Revision 1422429) Result = SUCCESS llu : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1422429 Files : * /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/security/UserGroupInformation.java * /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/security/TestProxyUserFromEnv.java > Introduce HADOOP_PROXY_USER for secure impersonation in child hadoop client > processes > - > > Key: HADOOP-8561 > URL: https://issues.apache.org/jira/browse/HADOOP-8561 > Project: Hadoop Common > Issue Type: Improvement > Components: security >Reporter: Luke Lu >Assignee: Yu Gao > Fix For: 1.2.0, 3.0.0, 2.0.3-alpha, 1.1.2 > > Attachments: hadoop-8561-branch-1.patch, hadoop-8561-branch-2.patch, > hadoop-8561.patch, hadoop-8561-v2.patch > > > To solve the problem for an authenticated user to type hadoop shell commands > in a web console, we can introduce an HADOOP_PROXY_USER environment variable > to allow proper impersonation in the child hadoop client processes. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9041) FileSystem initialization can go into infinite loop
[ https://issues.apache.org/jira/browse/HADOOP-9041?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13533393#comment-13533393 ] Hudson commented on HADOOP-9041: Integrated in Hadoop-Mapreduce-trunk #1287 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1287/]) HADOOP-9041. FsUrlStreamHandlerFactory could cause an infinite loop in FileSystem initialization. (Yanbo Liang and Radim Kolar via llu) (Revision 1422430) Result = SUCCESS llu : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1422430 Files : * /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/FsUrlStreamHandlerFactory.java * /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/TestFileSystemInitialization.java > FileSystem initialization can go into infinite loop > --- > > Key: HADOOP-9041 > URL: https://issues.apache.org/jira/browse/HADOOP-9041 > Project: Hadoop Common > Issue Type: Bug > Components: fs >Affects Versions: 2.0.2-alpha >Reporter: Radim Kolar >Assignee: Radim Kolar >Priority: Critical > Fix For: 3.0.0, 2.0.3-alpha > > Attachments: fsinit2.txt, fsinit3.txt, fsinit4.txt, fsinit5.txt, > fsinit6.txt, fsinit-unit.txt, fstest.groovy, HADOOP-9041.patch, > HADOOP-9041.patch, TestFileSystem.java > > > More information is there: https://jira.springsource.org/browse/SHDP-111 > Referenced source code from example is: > https://github.com/SpringSource/spring-hadoop/blob/master/src/main/java/org/springframework/data/hadoop/configuration/ConfigurationFactoryBean.java > from isolating that cause it looks like if you register: > org.apache.hadoop.fs.FsUrlStreamHandlerFactory before calling > FileSystem.loadFileSystems() then it goes into infinite loop. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8561) Introduce HADOOP_PROXY_USER for secure impersonation in child hadoop client processes
[ https://issues.apache.org/jira/browse/HADOOP-8561?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13533387#comment-13533387 ] Hudson commented on HADOOP-8561: Integrated in Hadoop-Hdfs-trunk #1256 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk/1256/]) HADOOP-8561. Introduce HADOOP_PROXY_USER for secure impersonation in child hadoop client processes. (Yu Gao via llu) (Revision 1422429) Result = FAILURE llu : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1422429 Files : * /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/security/UserGroupInformation.java * /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/security/TestProxyUserFromEnv.java > Introduce HADOOP_PROXY_USER for secure impersonation in child hadoop client > processes > - > > Key: HADOOP-8561 > URL: https://issues.apache.org/jira/browse/HADOOP-8561 > Project: Hadoop Common > Issue Type: Improvement > Components: security >Reporter: Luke Lu >Assignee: Yu Gao > Fix For: 1.2.0, 3.0.0, 2.0.3-alpha, 1.1.2 > > Attachments: hadoop-8561-branch-1.patch, hadoop-8561-branch-2.patch, > hadoop-8561.patch, hadoop-8561-v2.patch > > > To solve the problem for an authenticated user to type hadoop shell commands > in a web console, we can introduce an HADOOP_PROXY_USER environment variable > to allow proper impersonation in the child hadoop client processes. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9041) FileSystem initialization can go into infinite loop
[ https://issues.apache.org/jira/browse/HADOOP-9041?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13533383#comment-13533383 ] Hudson commented on HADOOP-9041: Integrated in Hadoop-Hdfs-trunk #1256 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk/1256/]) HADOOP-9041. FsUrlStreamHandlerFactory could cause an infinite loop in FileSystem initialization. (Yanbo Liang and Radim Kolar via llu) (Revision 1422430) Result = FAILURE llu : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1422430 Files : * /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/FsUrlStreamHandlerFactory.java * /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/TestFileSystemInitialization.java > FileSystem initialization can go into infinite loop > --- > > Key: HADOOP-9041 > URL: https://issues.apache.org/jira/browse/HADOOP-9041 > Project: Hadoop Common > Issue Type: Bug > Components: fs >Affects Versions: 2.0.2-alpha >Reporter: Radim Kolar >Assignee: Radim Kolar >Priority: Critical > Fix For: 3.0.0, 2.0.3-alpha > > Attachments: fsinit2.txt, fsinit3.txt, fsinit4.txt, fsinit5.txt, > fsinit6.txt, fsinit-unit.txt, fstest.groovy, HADOOP-9041.patch, > HADOOP-9041.patch, TestFileSystem.java > > > More information is there: https://jira.springsource.org/browse/SHDP-111 > Referenced source code from example is: > https://github.com/SpringSource/spring-hadoop/blob/master/src/main/java/org/springframework/data/hadoop/configuration/ConfigurationFactoryBean.java > from isolating that cause it looks like if you register: > org.apache.hadoop.fs.FsUrlStreamHandlerFactory before calling > FileSystem.loadFileSystems() then it goes into infinite loop. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8561) Introduce HADOOP_PROXY_USER for secure impersonation in child hadoop client processes
[ https://issues.apache.org/jira/browse/HADOOP-8561?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13533349#comment-13533349 ] Hudson commented on HADOOP-8561: Integrated in Hadoop-Yarn-trunk #67 (See [https://builds.apache.org/job/Hadoop-Yarn-trunk/67/]) HADOOP-8561. Introduce HADOOP_PROXY_USER for secure impersonation in child hadoop client processes. (Yu Gao via llu) (Revision 1422429) Result = SUCCESS llu : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1422429 Files : * /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/security/UserGroupInformation.java * /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/security/TestProxyUserFromEnv.java > Introduce HADOOP_PROXY_USER for secure impersonation in child hadoop client > processes > - > > Key: HADOOP-8561 > URL: https://issues.apache.org/jira/browse/HADOOP-8561 > Project: Hadoop Common > Issue Type: Improvement > Components: security >Reporter: Luke Lu >Assignee: Yu Gao > Fix For: 1.2.0, 3.0.0, 2.0.3-alpha, 1.1.2 > > Attachments: hadoop-8561-branch-1.patch, hadoop-8561-branch-2.patch, > hadoop-8561.patch, hadoop-8561-v2.patch > > > To solve the problem for an authenticated user to type hadoop shell commands > in a web console, we can introduce an HADOOP_PROXY_USER environment variable > to allow proper impersonation in the child hadoop client processes. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9041) FileSystem initialization can go into infinite loop
[ https://issues.apache.org/jira/browse/HADOOP-9041?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13533345#comment-13533345 ] Hudson commented on HADOOP-9041: Integrated in Hadoop-Yarn-trunk #67 (See [https://builds.apache.org/job/Hadoop-Yarn-trunk/67/]) HADOOP-9041. FsUrlStreamHandlerFactory could cause an infinite loop in FileSystem initialization. (Yanbo Liang and Radim Kolar via llu) (Revision 1422430) Result = SUCCESS llu : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1422430 Files : * /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/FsUrlStreamHandlerFactory.java * /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/TestFileSystemInitialization.java > FileSystem initialization can go into infinite loop > --- > > Key: HADOOP-9041 > URL: https://issues.apache.org/jira/browse/HADOOP-9041 > Project: Hadoop Common > Issue Type: Bug > Components: fs >Affects Versions: 2.0.2-alpha >Reporter: Radim Kolar >Assignee: Radim Kolar >Priority: Critical > Fix For: 3.0.0, 2.0.3-alpha > > Attachments: fsinit2.txt, fsinit3.txt, fsinit4.txt, fsinit5.txt, > fsinit6.txt, fsinit-unit.txt, fstest.groovy, HADOOP-9041.patch, > HADOOP-9041.patch, TestFileSystem.java > > > More information is there: https://jira.springsource.org/browse/SHDP-111 > Referenced source code from example is: > https://github.com/SpringSource/spring-hadoop/blob/master/src/main/java/org/springframework/data/hadoop/configuration/ConfigurationFactoryBean.java > from isolating that cause it looks like if you register: > org.apache.hadoop.fs.FsUrlStreamHandlerFactory before calling > FileSystem.loadFileSystems() then it goes into infinite loop. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
