[jira] [Commented] (HADOOP-8449) hadoop fs -text fails with compressed sequence files with the codec file extension
[ https://issues.apache.org/jira/browse/HADOOP-8449?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13460239#comment-13460239 ] Harsh J commented on HADOOP-8449: - Thanks Muddy, silly mistake of mine. I filed https://issues.apache.org/jira/browse/HADOOP-8833 > hadoop fs -text fails with compressed sequence files with the codec file > extension > -- > > Key: HADOOP-8449 > URL: https://issues.apache.org/jira/browse/HADOOP-8449 > Project: Hadoop Common > Issue Type: Bug >Affects Versions: 1.0.3, 2.0.0-alpha >Reporter: Joey Echeverria >Assignee: Harsh J >Priority: Minor > Fix For: 2.0.2-alpha > > Attachments: HADOOP-8449.patch, HADOOP-8449.patch > > > When the -text command is run on a file and the file ends in the default > extension for a codec (e.g. snappy or gz), but is a compressed sequence file, > the command will fail. > The issue is that it assumes that if it matches the extension, then it's > plain compressed file. It might be more helpful to check if it's a sequence > file first, and then check the file extension second. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HADOOP-8833) fs -text should make sure to call inputstream.seek(0) before using input stream
Harsh J created HADOOP-8833:
---
Summary: fs -text should make sure to call inputstream.seek(0)
before using input stream
Key: HADOOP-8833
URL: https://issues.apache.org/jira/browse/HADOOP-8833
Project: Hadoop Common
Issue Type: Bug
Components: fs
Affects Versions: 2.0.2-alpha
Reporter: Harsh J
Assignee: Harsh J
>From Muddy Dixon on HADOOP-8449:
Hi
We found the changes in order of switch and guard block in
{code}
private InputStream forMagic(Path p, FileSystem srcFs) throws IOException
{code}
Because of this change, return value of
{code}
codec.createInputStream(i)
{code}
is changed if codec exists.
{code}
private InputStream forMagic(Path p, FileSystem srcFs) throws IOException {
FSDataInputStream i = srcFs.open(p);
// check codecs
CompressionCodecFactory cf = new CompressionCodecFactory(getConf());
CompressionCodec codec = cf.getCodec(p);
if (codec != null) {
return codec.createInputStream(i);
}
switch(i.readShort()) {
// cases
}
{code}
New:
{code}
private InputStream forMagic(Path p, FileSystem srcFs) throws IOException {
FSDataInputStream i = srcFs.open(p);
switch(i.readShort()) { // <=== index (or pointer) processes!!
// cases
default: {
// Check the type of compression instead, depending on Codec class's
// own detection methods, based on the provided path.
CompressionCodecFactory cf = new CompressionCodecFactory(getConf());
CompressionCodec codec = cf.getCodec(p);
if (codec != null) {
return codec.createInputStream(i);
}
break;
}
}
// File is non-compressed, or not a file container we know.
i.seek(0);
return i;
}
{code}
Fix is to use i.seek(0) before we use i anywhere. I missed that.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8832) backport serviceplugin to branch-1
[ https://issues.apache.org/jira/browse/HADOOP-8832?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13460204#comment-13460204 ] Brandon Li commented on HADOOP-8832: @Eli, a few of our customers want to use this to start some custom services specific to their own setup. > backport serviceplugin to branch-1 > -- > > Key: HADOOP-8832 > URL: https://issues.apache.org/jira/browse/HADOOP-8832 > Project: Hadoop Common > Issue Type: Bug >Affects Versions: 1.2.0 >Reporter: Brandon Li >Assignee: Brandon Li > Fix For: 1.2.0 > > Attachments: HADOOP-8832.branch-1.patch, HADOOP-8832.branch-1.patch, > HADOOP-8832.branch-1.patch.all > > > The original patch was only partially back ported to branch-1. This JIRA is > to back port the rest of it. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8832) backport serviceplugin to branch-1
[ https://issues.apache.org/jira/browse/HADOOP-8832?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13460175#comment-13460175 ] Eli Collins commented on HADOOP-8832: - bq. Eli this is a generic mechianism to start some services. How does it matter what custom services? Per above, I'm just curious. In general it's good to understand how something is going to be used. If Brandon doesn't want to explain the motivation for his work that's his prerogative of course. > backport serviceplugin to branch-1 > -- > > Key: HADOOP-8832 > URL: https://issues.apache.org/jira/browse/HADOOP-8832 > Project: Hadoop Common > Issue Type: Bug >Affects Versions: 1.2.0 >Reporter: Brandon Li >Assignee: Brandon Li > Fix For: 1.2.0 > > Attachments: HADOOP-8832.branch-1.patch, HADOOP-8832.branch-1.patch, > HADOOP-8832.branch-1.patch.all > > > The original patch was only partially back ported to branch-1. This JIRA is > to back port the rest of it. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8823) ant package target should not depend on cn-docs
[ https://issues.apache.org/jira/browse/HADOOP-8823?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13460170#comment-13460170 ] Tsz Wo (Nicholas), SZE commented on HADOOP-8823: Hi Eli, I think it is a good idea. Will try to do it. > ant package target should not depend on cn-docs > --- > > Key: HADOOP-8823 > URL: https://issues.apache.org/jira/browse/HADOOP-8823 > Project: Hadoop Common > Issue Type: Improvement > Components: build >Reporter: Tsz Wo (Nicholas), SZE >Assignee: Tsz Wo (Nicholas), SZE > Attachments: c8823_20120918.patch > > > In branch-1, the package target depends on cn-docs but the doc is already > outdated. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8733) TestStreamingTaskLog, TestJvmManager, TestLinuxTaskControllerLaunchArgs fail on Windows
[ https://issues.apache.org/jira/browse/HADOOP-8733?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13460144#comment-13460144 ] Hadoop QA commented on HADOOP-8733: --- -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12545993/HADOOP-8733-scripts.3.patch against trunk revision . -1 patch. The patch command could not apply the patch. Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/1489//console This message is automatically generated. > TestStreamingTaskLog, TestJvmManager, TestLinuxTaskControllerLaunchArgs fail > on Windows > --- > > Key: HADOOP-8733 > URL: https://issues.apache.org/jira/browse/HADOOP-8733 > Project: Hadoop Common > Issue Type: Bug > Components: test >Affects Versions: 1-win >Reporter: Ivan Mitic >Assignee: Ivan Mitic > Attachments: HADOOP-8733-scripts.2.patch, > HADOOP-8733-scripts.2.patch, HADOOP-8733-scripts.3.patch, > HADOOP-8733-scripts.patch > > > Jira tracking test failures related to test .sh script dependencies. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-8733) TestStreamingTaskLog, TestJvmManager, TestLinuxTaskControllerLaunchArgs fail on Windows
[ https://issues.apache.org/jira/browse/HADOOP-8733?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ivan Mitic updated HADOOP-8733: --- Attachment: HADOOP-8733-scripts.3.patch Attaching updated patch. I added the part where attempt id is passed as part of the child task environment and circulated back for taskkill. Vinod, let me know if you have any comments on the latest patch. Thanks! > TestStreamingTaskLog, TestJvmManager, TestLinuxTaskControllerLaunchArgs fail > on Windows > --- > > Key: HADOOP-8733 > URL: https://issues.apache.org/jira/browse/HADOOP-8733 > Project: Hadoop Common > Issue Type: Bug > Components: test >Affects Versions: 1-win >Reporter: Ivan Mitic >Assignee: Ivan Mitic > Attachments: HADOOP-8733-scripts.2.patch, > HADOOP-8733-scripts.2.patch, HADOOP-8733-scripts.3.patch, > HADOOP-8733-scripts.patch > > > Jira tracking test failures related to test .sh script dependencies. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (HADOOP-8832) backport serviceplugin to branch-1
[ https://issues.apache.org/jira/browse/HADOOP-8832?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Suresh Srinivas resolved HADOOP-8832. - Resolution: Fixed Fix Version/s: 1.2.0 Hadoop Flags: Reviewed I committed the patch. Thank you Brandon for back porting the patch. > backport serviceplugin to branch-1 > -- > > Key: HADOOP-8832 > URL: https://issues.apache.org/jira/browse/HADOOP-8832 > Project: Hadoop Common > Issue Type: Bug >Affects Versions: 1.2.0 >Reporter: Brandon Li >Assignee: Brandon Li > Fix For: 1.2.0 > > Attachments: HADOOP-8832.branch-1.patch, HADOOP-8832.branch-1.patch, > HADOOP-8832.branch-1.patch.all > > > The original patch was only partially back ported to branch-1. This JIRA is > to back port the rest of it. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8832) backport serviceplugin to branch-1
[ https://issues.apache.org/jira/browse/HADOOP-8832?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13460106#comment-13460106 ] Suresh Srinivas commented on HADOOP-8832: - +1 for the patch. > backport serviceplugin to branch-1 > -- > > Key: HADOOP-8832 > URL: https://issues.apache.org/jira/browse/HADOOP-8832 > Project: Hadoop Common > Issue Type: Bug >Affects Versions: 1.2.0 >Reporter: Brandon Li >Assignee: Brandon Li > Attachments: HADOOP-8832.branch-1.patch, HADOOP-8832.branch-1.patch, > HADOOP-8832.branch-1.patch.all > > > The original patch was only partially back ported to branch-1. This JIRA is > to back port the rest of it. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8832) backport serviceplugin to branch-1
[ https://issues.apache.org/jira/browse/HADOOP-8832?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13460105#comment-13460105 ] Suresh Srinivas commented on HADOOP-8832: - bq. out of curiosity, what custom services do you want to run? Eli this is a generic mechianism to start some services. How does it matter what custom services? > backport serviceplugin to branch-1 > -- > > Key: HADOOP-8832 > URL: https://issues.apache.org/jira/browse/HADOOP-8832 > Project: Hadoop Common > Issue Type: Bug >Affects Versions: 1.2.0 >Reporter: Brandon Li >Assignee: Brandon Li > Attachments: HADOOP-8832.branch-1.patch, HADOOP-8832.branch-1.patch, > HADOOP-8832.branch-1.patch.all > > > The original patch was only partially back ported to branch-1. This JIRA is > to back port the rest of it. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-8832) backport serviceplugin to branch-1
[ https://issues.apache.org/jira/browse/HADOOP-8832?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brandon Li updated HADOOP-8832: --- Summary: backport serviceplugin to branch-1 (was: backport TestGetInstances to branch-1) > backport serviceplugin to branch-1 > -- > > Key: HADOOP-8832 > URL: https://issues.apache.org/jira/browse/HADOOP-8832 > Project: Hadoop Common > Issue Type: Bug >Affects Versions: 1.2.0 >Reporter: Brandon Li >Assignee: Brandon Li > Attachments: HADOOP-8832.branch-1.patch, HADOOP-8832.branch-1.patch, > HADOOP-8832.branch-1.patch.all > > > The original patch was only partially back ported to branch-1. This JIRA is > to back port the rest of it. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8832) backport TestGetInstances to branch-1
[ https://issues.apache.org/jira/browse/HADOOP-8832?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13460071#comment-13460071 ] Brandon Li commented on HADOOP-8832: Uploaded a patch for common only. > backport TestGetInstances to branch-1 > - > > Key: HADOOP-8832 > URL: https://issues.apache.org/jira/browse/HADOOP-8832 > Project: Hadoop Common > Issue Type: Bug >Affects Versions: 1.2.0 >Reporter: Brandon Li >Assignee: Brandon Li > Attachments: HADOOP-8832.branch-1.patch, HADOOP-8832.branch-1.patch, > HADOOP-8832.branch-1.patch.all > > > The original patch was only partially back ported to branch-1. This JIRA is > to back port the rest of it. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8832) backport TestGetInstances to branch-1
[ https://issues.apache.org/jira/browse/HADOOP-8832?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13460070#comment-13460070 ] Suresh Srinivas commented on HADOOP-8832: - Brandon, can we have separate jiras/patches for hdfs and common. > backport TestGetInstances to branch-1 > - > > Key: HADOOP-8832 > URL: https://issues.apache.org/jira/browse/HADOOP-8832 > Project: Hadoop Common > Issue Type: Bug >Affects Versions: 1.2.0 >Reporter: Brandon Li >Assignee: Brandon Li > Attachments: HADOOP-8832.branch-1.patch, HADOOP-8832.branch-1.patch, > HADOOP-8832.branch-1.patch.all > > > The original patch was only partially back ported to branch-1. This JIRA is > to back port the rest of it. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-8832) backport TestGetInstances to branch-1
[ https://issues.apache.org/jira/browse/HADOOP-8832?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brandon Li updated HADOOP-8832: --- Attachment: HADOOP-8832.branch-1.patch > backport TestGetInstances to branch-1 > - > > Key: HADOOP-8832 > URL: https://issues.apache.org/jira/browse/HADOOP-8832 > Project: Hadoop Common > Issue Type: Bug >Affects Versions: 1.2.0 >Reporter: Brandon Li >Assignee: Brandon Li > Attachments: HADOOP-8832.branch-1.patch, HADOOP-8832.branch-1.patch, > HADOOP-8832.branch-1.patch.all > > > The original patch was only partially back ported to branch-1. This JIRA is > to back port the rest of it. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Comment Edited] (HADOOP-8832) backport TestGetInstances to branch-1
[ https://issues.apache.org/jira/browse/HADOOP-8832?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13460070#comment-13460070 ] Suresh Srinivas edited comment on HADOOP-8832 at 9/21/12 11:01 AM: --- Brandon, can you post separate jiras/patches for hdfs and common. was (Author: sureshms): Brandon, can we have separate jiras/patches for hdfs and common. > backport TestGetInstances to branch-1 > - > > Key: HADOOP-8832 > URL: https://issues.apache.org/jira/browse/HADOOP-8832 > Project: Hadoop Common > Issue Type: Bug >Affects Versions: 1.2.0 >Reporter: Brandon Li >Assignee: Brandon Li > Attachments: HADOOP-8832.branch-1.patch, HADOOP-8832.branch-1.patch, > HADOOP-8832.branch-1.patch.all > > > The original patch was only partially back ported to branch-1. This JIRA is > to back port the rest of it. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8832) backport TestGetInstances to branch-1
[ https://issues.apache.org/jira/browse/HADOOP-8832?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13460053#comment-13460053 ] Brandon Li commented on HADOOP-8832: No problem. I understand. :-) > backport TestGetInstances to branch-1 > - > > Key: HADOOP-8832 > URL: https://issues.apache.org/jira/browse/HADOOP-8832 > Project: Hadoop Common > Issue Type: Bug >Affects Versions: 1.2.0 >Reporter: Brandon Li >Assignee: Brandon Li > Attachments: HADOOP-8832.branch-1.patch, > HADOOP-8832.branch-1.patch.all > > > The original patch was only partially back ported to branch-1. This JIRA is > to back port the rest of it. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-8832) backport TestGetInstances to branch-1
[ https://issues.apache.org/jira/browse/HADOOP-8832?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brandon Li updated HADOOP-8832: --- Attachment: HADOOP-8832.branch-1.patch.all > backport TestGetInstances to branch-1 > - > > Key: HADOOP-8832 > URL: https://issues.apache.org/jira/browse/HADOOP-8832 > Project: Hadoop Common > Issue Type: Bug >Affects Versions: 1.2.0 >Reporter: Brandon Li >Assignee: Brandon Li > Attachments: HADOOP-8832.branch-1.patch, > HADOOP-8832.branch-1.patch.all > > > The original patch was only partially back ported to branch-1. This JIRA is > to back port the rest of it. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8832) backport TestGetInstances to branch-1
[ https://issues.apache.org/jira/browse/HADOOP-8832?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13460011#comment-13460011 ] Todd Lipcon commented on HADOOP-8832: - I don't have any particular concerns, I just like to understand motivation behind spending time backporting things into 1.0 rather than spending time on forward development. So, consider me curious, not hostile :) > backport TestGetInstances to branch-1 > - > > Key: HADOOP-8832 > URL: https://issues.apache.org/jira/browse/HADOOP-8832 > Project: Hadoop Common > Issue Type: Bug >Affects Versions: 1.2.0 >Reporter: Brandon Li >Assignee: Brandon Li > Attachments: HADOOP-8832.branch-1.patch > > > The original patch was only partially back ported to branch-1. This JIRA is > to back port the rest of it. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8832) backport TestGetInstances to branch-1
[ https://issues.apache.org/jira/browse/HADOOP-8832?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13460010#comment-13460010 ] Brandon Li commented on HADOOP-8832: my bad :-( The change from original patch already in branch-1 is the change in Configuration.java. ServicePlugin is not there yet. > backport TestGetInstances to branch-1 > - > > Key: HADOOP-8832 > URL: https://issues.apache.org/jira/browse/HADOOP-8832 > Project: Hadoop Common > Issue Type: Bug >Affects Versions: 1.2.0 >Reporter: Brandon Li >Assignee: Brandon Li > Attachments: HADOOP-8832.branch-1.patch > > > The original patch was only partially back ported to branch-1. This JIRA is > to back port the rest of it. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8832) backport TestGetInstances to branch-1
[ https://issues.apache.org/jira/browse/HADOOP-8832?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13460001#comment-13460001 ] Suresh Srinivas commented on HADOOP-8832: - Brandon, ServicePlugin is also required with the patch. May be your development area for branch-1 has it, but the apache branch-1 does not. > backport TestGetInstances to branch-1 > - > > Key: HADOOP-8832 > URL: https://issues.apache.org/jira/browse/HADOOP-8832 > Project: Hadoop Common > Issue Type: Bug >Affects Versions: 1.2.0 >Reporter: Brandon Li >Assignee: Brandon Li > Attachments: HADOOP-8832.branch-1.patch > > > The original patch was only partially back ported to branch-1. This JIRA is > to back port the rest of it. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8832) backport TestGetInstances to branch-1
[ https://issues.apache.org/jira/browse/HADOOP-8832?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13459992#comment-13459992 ] Brandon Li commented on HADOOP-8832: We want to use this to start some custom services. Since this is already in trunk and seems to be a good way to achieve that. Half of its functionality is already in branch-1. Do you have any concerns with it? The backport seems pretty straightforward to me. > backport TestGetInstances to branch-1 > - > > Key: HADOOP-8832 > URL: https://issues.apache.org/jira/browse/HADOOP-8832 > Project: Hadoop Common > Issue Type: Bug >Affects Versions: 1.2.0 >Reporter: Brandon Li >Assignee: Brandon Li > Attachments: HADOOP-8832.branch-1.patch > > > The original patch was only partially back ported to branch-1. This JIRA is > to back port the rest of it. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-8832) backport TestGetInstances to branch-1
[ https://issues.apache.org/jira/browse/HADOOP-8832?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brandon Li updated HADOOP-8832: --- Attachment: HADOOP-8832.branch-1.patch > backport TestGetInstances to branch-1 > - > > Key: HADOOP-8832 > URL: https://issues.apache.org/jira/browse/HADOOP-8832 > Project: Hadoop Common > Issue Type: Bug >Affects Versions: 1.2.0 >Reporter: Brandon Li >Assignee: Brandon Li > Attachments: HADOOP-8832.branch-1.patch > > > The original patch was only partially back ported to branch-1. This JIRA is > to back port the rest of it. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8832) backport TestGetInstances to branch-1
[ https://issues.apache.org/jira/browse/HADOOP-8832?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13459987#comment-13459987 ] Todd Lipcon commented on HADOOP-8832: - The original motivation of the issue was to expose DFS protocols via Thrift. But now that we have protobufs in place, it seems less useful. Do you have a particular service or protocol that you're looking to expose? > backport TestGetInstances to branch-1 > - > > Key: HADOOP-8832 > URL: https://issues.apache.org/jira/browse/HADOOP-8832 > Project: Hadoop Common > Issue Type: Bug >Affects Versions: 1.2.0 >Reporter: Brandon Li >Assignee: Brandon Li > > The original patch was only partially back ported to branch-1. This JIRA is > to back port the rest of it. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-8832) backport TestGetInstances to branch-1
[ https://issues.apache.org/jira/browse/HADOOP-8832?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brandon Li updated HADOOP-8832: --- Summary: backport TestGetInstances to branch-1 (was: backport HADOOP-5257 to branch-1) > backport TestGetInstances to branch-1 > - > > Key: HADOOP-8832 > URL: https://issues.apache.org/jira/browse/HADOOP-8832 > Project: Hadoop Common > Issue Type: Bug >Affects Versions: 1.2.0 >Reporter: Brandon Li >Assignee: Brandon Li > > The original patch was only partially back ported to branch-1. This JIRA is > to back port the rest of it. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8832) backport TestGetInstances to branch-1
[ https://issues.apache.org/jira/browse/HADOOP-8832?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13459986#comment-13459986 ] Brandon Li commented on HADOOP-8832: For Hadoop Common, we actually only need to back port the test TestGetInstances. Changed the title accordingly. > backport TestGetInstances to branch-1 > - > > Key: HADOOP-8832 > URL: https://issues.apache.org/jira/browse/HADOOP-8832 > Project: Hadoop Common > Issue Type: Bug >Affects Versions: 1.2.0 >Reporter: Brandon Li >Assignee: Brandon Li > > The original patch was only partially back ported to branch-1. This JIRA is > to back port the rest of it. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8832) backport HADOOP-5257 to branch-1
[ https://issues.apache.org/jira/browse/HADOOP-8832?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13459980#comment-13459980 ] Brandon Li commented on HADOOP-8832: As stated in HADOOP-5357: "Adding support for pluggable components would allow exporting DFS functionallity using arbitrary protocols", to start custom services. > backport HADOOP-5257 to branch-1 > > > Key: HADOOP-8832 > URL: https://issues.apache.org/jira/browse/HADOOP-8832 > Project: Hadoop Common > Issue Type: Bug >Affects Versions: 1.2.0 >Reporter: Brandon Li >Assignee: Brandon Li > > The original patch was only partially back ported to branch-1. This JIRA is > to back port the rest of it. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8832) backport HADOOP-5257 to branch-1
[ https://issues.apache.org/jira/browse/HADOOP-8832?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13459913#comment-13459913 ] Todd Lipcon commented on HADOOP-8832: - Hey Brandon. Can you please explain the use case you're targeting? > backport HADOOP-5257 to branch-1 > > > Key: HADOOP-8832 > URL: https://issues.apache.org/jira/browse/HADOOP-8832 > Project: Hadoop Common > Issue Type: Bug >Affects Versions: 1.2.0 >Reporter: Brandon Li >Assignee: Brandon Li > > The original patch was only partially back ported to branch-1. This JIRA is > to back port the rest of it. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HADOOP-8832) backport HADOOP-5257 to branch-1
Brandon Li created HADOOP-8832: -- Summary: backport HADOOP-5257 to branch-1 Key: HADOOP-8832 URL: https://issues.apache.org/jira/browse/HADOOP-8832 Project: Hadoop Common Issue Type: Bug Affects Versions: 1.2.0 Reporter: Brandon Li Assignee: Brandon Li The original patch was only partially back ported to branch-1. This JIRA is to back port the rest of it. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8831) FSEditLog preallocate() needs to reset the position of PREALLOCATE_BUFFER when more than 1MB size is needed
[ https://issues.apache.org/jira/browse/HADOOP-8831?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13459833#comment-13459833 ] Jing Zhao commented on HADOOP-8831: --- I should create this jira in HDFS... > FSEditLog preallocate() needs to reset the position of PREALLOCATE_BUFFER > when more than 1MB size is needed > --- > > Key: HADOOP-8831 > URL: https://issues.apache.org/jira/browse/HADOOP-8831 > Project: Hadoop Common > Issue Type: Bug >Affects Versions: 1.2.0 >Reporter: Jing Zhao >Priority: Critical > > In the new preallocate() function, when the required size is larger 1MB, we > need to reset the position for PREALLOCATION_BUFFER every time when we have > allocated 1MB. Otherwise seems only 1MB can be allocated even if need is > larger than 1MB. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HADOOP-8831) FSEditLog preallocate() needs to reset the position of PREALLOCATE_BUFFER when more than 1MB size is needed
Jing Zhao created HADOOP-8831: - Summary: FSEditLog preallocate() needs to reset the position of PREALLOCATE_BUFFER when more than 1MB size is needed Key: HADOOP-8831 URL: https://issues.apache.org/jira/browse/HADOOP-8831 Project: Hadoop Common Issue Type: Bug Affects Versions: 1.2.0 Reporter: Jing Zhao Priority: Critical In the new preallocate() function, when the required size is larger 1MB, we need to reset the position for PREALLOCATION_BUFFER every time when we have allocated 1MB. Otherwise seems only 1MB can be allocated even if need is larger than 1MB. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8803) Make Hadoop running more secure public cloud envrionment
[ https://issues.apache.org/jira/browse/HADOOP-8803?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13459825#comment-13459825 ] Luke Lu commented on HADOOP-8803: - bq. as long as the hacker can get root privilege, she or he always can get the new key from the namenode and generate arbitrary Block Token to access other datanodes, in the end, the whole HDFS is exposed to attacker. Since block id generation is currently random (see also HDFS-898), there is no practical way to generate correct block tokens to access "the whole" HDFS. Monitoring system will immediately notice a spike of failed block requests if brutal force is used and blacklist the node. You should be more concerned about the delegation tokens stored locally on that node. Even more so, if the hacker can root one DN/TT node, it can probably root the rest of the nodes given all the nodes are uniformly configured in typical clusters. bq. NameNode has to generate token for each replica and that indeed degrade the performance...I can test how much of this overhead... Make sure you test cases, where number of replicas is way over 3. 10 is common for distributed cache files. 100 is not uncommon on large clusters. I've been fortunate to have on and off in depth discussions with Kan (Zhang, the original Hadoop security architect) about an alternative block token design with unique DN key for over a year now. IMO, the original hadoop security team made the right trade off on "perceived" security vs performance. I'm not saying that the current Hadoop security design/implementation is perfect. But I think more elaborate authorization scheme a la. POSIX.1e ACLs is probably more useful than futzing with block tokens. > Make Hadoop running more secure public cloud envrionment > > > Key: HADOOP-8803 > URL: https://issues.apache.org/jira/browse/HADOOP-8803 > Project: Hadoop Common > Issue Type: New Feature > Components: fs, ipc, security >Affects Versions: 0.20.204.0 >Reporter: Xianqing Yu > Labels: hadoop > Original Estimate: 2m > Remaining Estimate: 2m > > I am a Ph.D student in North Carolina State University. I am modifying the > Hadoop's code (which including most parts of Hadoop, e.g. JobTracker, > TaskTracker, NameNode, DataNode) to achieve better security. > > My major goal is that make Hadoop running more secure in the Cloud > environment, especially for public Cloud environment. In order to achieve > that, I redesign the currently security mechanism and achieve following > proprieties: > 1. Bring byte-level access control to Hadoop HDFS. Based on 0.20.204, HDFS > access control is based on user or block granularity, e.g. HDFS Delegation > Token only check if the file can be accessed by certain user or not, Block > Token only proof which block or blocks can be accessed. I make Hadoop can do > byte-granularity access control, each access party, user or task process can > only access the bytes she or he least needed. > 2. I assume that in the public Cloud environment, only Namenode, secondary > Namenode, JobTracker can be trusted. A large number of Datanode and > TaskTracker may be compromised due to some of them may be running under less > secure environment. So I re-design the secure mechanism to make the damage > the hacker can do to be minimized. > > a. Re-design the Block Access Token to solve wildly shared-key problem of > HDFS. In original Block Access Token design, all HDFS (Namenode and Datanode) > share one master key to generate Block Access Token, if one DataNode is > compromised by hacker, the hacker can get the key and generate any Block > Access Token he or she want. > > b. Re-design the HDFS Delegation Token to do fine-grain access control for > TaskTracker and Map-Reduce Task process on HDFS. > > In the Hadoop 0.20.204, all TaskTrackers can use their kerberos credentials > to access any files for MapReduce on HDFS. So they have the same privilege as > JobTracker to do read or write tokens, copy job file, etc.. However, if one > of them is compromised, every critical thing in MapReduce directory (job > file, Delegation Token) is exposed to attacker. I solve the problem by making > JobTracker to decide which TaskTracker can access which file in MapReduce > Directory on HDFS. > > For Task process, once it get HDFS Delegation Token, it can access everything > belong to this job or user on HDFS. By my design, it can only access the > bytes it needed from HDFS. > > There are some other improvement in the security, such as TaskTracker can not > know some information like blockID from the Block Token (because it is > encrypted by my way), and HDFS can set up secure channel to send data as a > option. > > By those features, Hadoop
[jira] [Commented] (HADOOP-8814) Inefficient comparison with the empty string. Use isEmpty() instead
[
https://issues.apache.org/jira/browse/HADOOP-8814?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13459608#comment-13459608
]
Hudson commented on HADOOP-8814:
Integrated in Hadoop-Mapreduce-trunk #1202 (See
[https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1202/])
HADOOP-8814. Replace string equals by String#isEmpty(). Contributed by
Brandon Li. (Revision 1387853)
Result = SUCCESS
suresh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1387853
Files :
* /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
*
/hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/conf/Configuration.java
*
/hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/conf/ReconfigurationServlet.java
*
/hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/FileContext.java
*
/hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/FileSystem.java
*
/hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/Path.java
*
/hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/RawLocalFileSystem.java
*
/hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/local/RawLocalFs.java
*
/hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/s3native/NativeS3FileSystem.java
*
/hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/ha/ActiveStandbyElector.java
*
/hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/ha/SshFenceByTcpPort.java
*
/hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/io/DefaultStringifier.java
*
/hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/io/file/tfile/TFile.java
*
/hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/metrics/util/MetricsDynamicMBeanBase.java
*
/hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/record/CsvRecordInput.java
*
/hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/record/CsvRecordOutput.java
*
/hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/security/SecurityUtil.java
*
/hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/security/UserGroupInformation.java
*
/hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/security/authorize/ServiceAuthorizationManager.java
*
/hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/security/token/delegation/AbstractDelegationTokenIdentifier.java
*
/hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/security/token/delegation/AbstractDelegationTokenSecretManager.java
*
/hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/HostsFileReader.java
*
/hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/StringUtils.java
*
/hadoop/common/trunk/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/io/nativeio/TestNativeIO.java
*
/hadoop/common/trunk/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/ipc/TestSaslRPC.java
*
/hadoop/common/trunk/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/util/TestStringUtils.java
> Inefficient comparison with the empty string. Use isEmpty() instead
> ---
>
> Key: HADOOP-8814
> URL: https://issues.apache.org/jira/browse/HADOOP-8814
> Project: Hadoop Common
> Issue Type: Improvement
> Components: conf, fs, fs/s3, ha, io, metrics, performance, record,
> security, util
>Reporter: Brandon Li
>Assignee: Brandon Li
>Priority: Minor
> Attachments: HADOOP-8814.patch, HADOOP-8814.patch
>
>
> Prior to JDK 6, we can check if a string is empty by doing "".equals(s) or
> s.equals("").
> Starting from JDK 6, String class has a new convenience and efficient method
> isEmpty() to check string's length.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8805) Move protocol buffer implementation of GetUserMappingProtocol from HDFS to Common
[ https://issues.apache.org/jira/browse/HADOOP-8805?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13459607#comment-13459607 ] Hudson commented on HADOOP-8805: Integrated in Hadoop-Mapreduce-trunk #1202 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1202/]) Reverting HADOOP-8805 (Revision 1387697) Result = SUCCESS tucu : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1387697 Files : * /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt * /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/tools/GetUserMappingsProtocolPB.java * /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/tools/impl * /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/proto/GetUserMappingsProtocol.proto * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/NameNodeProxies.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/protocolPB/GetUserMappingsProtocolClientSideTranslatorPB.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/protocolPB/GetUserMappingsProtocolPB.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/protocolPB/GetUserMappingsProtocolServerSideTranslatorPB.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NameNodeRpcServer.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/proto/GetUserMappingsProtocol.proto * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestIsMethodSupported.java > Move protocol buffer implementation of GetUserMappingProtocol from HDFS to > Common > - > > Key: HADOOP-8805 > URL: https://issues.apache.org/jira/browse/HADOOP-8805 > Project: Hadoop Common > Issue Type: Improvement >Reporter: Bo Wang >Assignee: Bo Wang > Fix For: 2.0.3-alpha > > Attachments: HADOOP-8805.patch, HADOOP-8805-v2.patch, > HADOOP-8805-v3.patch > > > org.apache.hadoop.tools.GetUserMappingProtocol is used in both HDFS and YARN. > We should move the protocol buffer implementation from HDFS to Common so that > it can also be used by YARN. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HADOOP-8830) org.apache.hadoop.security.authentication.server.AuthenticationFilter might be called twice, causing kerberos replay errors
Moritz Moeller created HADOOP-8830: -- Summary: org.apache.hadoop.security.authentication.server.AuthenticationFilter might be called twice, causing kerberos replay errors Key: HADOOP-8830 URL: https://issues.apache.org/jira/browse/HADOOP-8830 Project: Hadoop Common Issue Type: Bug Affects Versions: 2.0.1-alpha Reporter: Moritz Moeller AuthenticationFilter.doFilter is called twice (not sure if that is intentional or not). The second time it is called the ServletRequest is already authenticated, i.e. httpRequest.getRemoteUser() returns non-null info. If the kerberos authentication is triggered a second time it'll return a replay attack exception. I solved this by adding a if (httpRequest.getRemoteUser() == null) at the very beginning of doFilter. Alternatively one can set an attribute on the request, or figure out why doFilter is called twice. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8805) Move protocol buffer implementation of GetUserMappingProtocol from HDFS to Common
[ https://issues.apache.org/jira/browse/HADOOP-8805?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13459573#comment-13459573 ] Hudson commented on HADOOP-8805: Integrated in Hadoop-Hdfs-trunk #1171 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk/1171/]) Reverting HADOOP-8805 (Revision 1387697) Result = SUCCESS tucu : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1387697 Files : * /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt * /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/tools/GetUserMappingsProtocolPB.java * /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/tools/impl * /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/proto/GetUserMappingsProtocol.proto * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/NameNodeProxies.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/protocolPB/GetUserMappingsProtocolClientSideTranslatorPB.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/protocolPB/GetUserMappingsProtocolPB.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/protocolPB/GetUserMappingsProtocolServerSideTranslatorPB.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NameNodeRpcServer.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/proto/GetUserMappingsProtocol.proto * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestIsMethodSupported.java > Move protocol buffer implementation of GetUserMappingProtocol from HDFS to > Common > - > > Key: HADOOP-8805 > URL: https://issues.apache.org/jira/browse/HADOOP-8805 > Project: Hadoop Common > Issue Type: Improvement >Reporter: Bo Wang >Assignee: Bo Wang > Fix For: 2.0.3-alpha > > Attachments: HADOOP-8805.patch, HADOOP-8805-v2.patch, > HADOOP-8805-v3.patch > > > org.apache.hadoop.tools.GetUserMappingProtocol is used in both HDFS and YARN. > We should move the protocol buffer implementation from HDFS to Common so that > it can also be used by YARN. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8814) Inefficient comparison with the empty string. Use isEmpty() instead
[
https://issues.apache.org/jira/browse/HADOOP-8814?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13459574#comment-13459574
]
Hudson commented on HADOOP-8814:
Integrated in Hadoop-Hdfs-trunk #1171 (See
[https://builds.apache.org/job/Hadoop-Hdfs-trunk/1171/])
HADOOP-8814. Replace string equals by String#isEmpty(). Contributed by
Brandon Li. (Revision 1387853)
Result = SUCCESS
suresh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1387853
Files :
* /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
*
/hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/conf/Configuration.java
*
/hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/conf/ReconfigurationServlet.java
*
/hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/FileContext.java
*
/hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/FileSystem.java
*
/hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/Path.java
*
/hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/RawLocalFileSystem.java
*
/hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/local/RawLocalFs.java
*
/hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/s3native/NativeS3FileSystem.java
*
/hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/ha/ActiveStandbyElector.java
*
/hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/ha/SshFenceByTcpPort.java
*
/hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/io/DefaultStringifier.java
*
/hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/io/file/tfile/TFile.java
*
/hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/metrics/util/MetricsDynamicMBeanBase.java
*
/hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/record/CsvRecordInput.java
*
/hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/record/CsvRecordOutput.java
*
/hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/security/SecurityUtil.java
*
/hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/security/UserGroupInformation.java
*
/hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/security/authorize/ServiceAuthorizationManager.java
*
/hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/security/token/delegation/AbstractDelegationTokenIdentifier.java
*
/hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/security/token/delegation/AbstractDelegationTokenSecretManager.java
*
/hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/HostsFileReader.java
*
/hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/StringUtils.java
*
/hadoop/common/trunk/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/io/nativeio/TestNativeIO.java
*
/hadoop/common/trunk/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/ipc/TestSaslRPC.java
*
/hadoop/common/trunk/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/util/TestStringUtils.java
> Inefficient comparison with the empty string. Use isEmpty() instead
> ---
>
> Key: HADOOP-8814
> URL: https://issues.apache.org/jira/browse/HADOOP-8814
> Project: Hadoop Common
> Issue Type: Improvement
> Components: conf, fs, fs/s3, ha, io, metrics, performance, record,
> security, util
>Reporter: Brandon Li
>Assignee: Brandon Li
>Priority: Minor
> Attachments: HADOOP-8814.patch, HADOOP-8814.patch
>
>
> Prior to JDK 6, we can check if a string is empty by doing "".equals(s) or
> s.equals("").
> Starting from JDK 6, String class has a new convenience and efficient method
> isEmpty() to check string's length.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-8825) Reinstate constructors in SequenceFile.BlockCompressWriter and SequenceFile.RecordCompressWriter for compatibility with Hadoop 1
[ https://issues.apache.org/jira/browse/HADOOP-8825?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tom White updated HADOOP-8825: -- Resolution: Invalid Target Version/s: (was: 2.0.3-alpha) Status: Resolved (was: Patch Available) I managed to change AvroSequenceFile so it doesn't need to use the BlockCompressWriter or RecordCompressWriter constructors. Thanks for the help Bobby! > Reinstate constructors in SequenceFile.BlockCompressWriter and > SequenceFile.RecordCompressWriter for compatibility with Hadoop 1 > > > Key: HADOOP-8825 > URL: https://issues.apache.org/jira/browse/HADOOP-8825 > Project: Hadoop Common > Issue Type: Bug > Components: io >Affects Versions: 2.0.1-alpha >Reporter: Tom White >Assignee: Tom White > Attachments: HADOOP-8825.patch > > > Two constructors were removed in Hadoop 2 which causes a problem for Avro > being able to support Hadoop 1 and Hadoop 2. See AVRO-1170. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
