[jira] [Commented] (HADOOP-11455) KMS and Credential CLI should request confirmation for deletion by default
[ https://issues.apache.org/jira/browse/HADOOP-11455?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14264305#comment-14264305 ] Hudson commented on HADOOP-11455: - FAILURE: Integrated in Hadoop-trunk-Commit #6804 (See [https://builds.apache.org/job/Hadoop-trunk-Commit/6804/]) HADOOP-11455. KMS and Credential CLI should request confirmation for deletion by default. (Charles Lamb via yliu) (yliu: rev 21c6f01831dd3a63c46c2cbb94289206a6239166) * hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/crypto/key/TestKeyShell.java * hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/security/alias/TestCredShell.java * hadoop-common-project/hadoop-common/CHANGES.txt * hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/security/alias/CredentialShell.java * hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/crypto/key/KeyShell.java > KMS and Credential CLI should request confirmation for deletion by default > -- > > Key: HADOOP-11455 > URL: https://issues.apache.org/jira/browse/HADOOP-11455 > Project: Hadoop Common > Issue Type: Improvement > Components: security >Affects Versions: 2.7.0 >Reporter: Charles Lamb >Assignee: Charles Lamb >Priority: Minor > Fix For: 2.7.0 > > Attachments: HADOOP-11455.000.patch, HADOOP-11455.001.patch > > > The hadoop key delete and hadoop credential delete currently only ask for > confirmation of the delete if -i is specified. Asking for confirmation should > be the default action for both. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HADOOP-11455) KMS and Credential CLI should request confirmation for deletion by default
[ https://issues.apache.org/jira/browse/HADOOP-11455?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yi Liu updated HADOOP-11455: Resolution: Fixed Fix Version/s: 2.7.0 Hadoop Flags: Reviewed Status: Resolved (was: Patch Available) Commit to trunk and branch-2 > KMS and Credential CLI should request confirmation for deletion by default > -- > > Key: HADOOP-11455 > URL: https://issues.apache.org/jira/browse/HADOOP-11455 > Project: Hadoop Common > Issue Type: Improvement > Components: security >Affects Versions: 2.7.0 >Reporter: Charles Lamb >Assignee: Charles Lamb >Priority: Minor > Fix For: 2.7.0 > > Attachments: HADOOP-11455.000.patch, HADOOP-11455.001.patch > > > The hadoop key delete and hadoop credential delete currently only ask for > confirmation of the delete if -i is specified. Asking for confirmation should > be the default action for both. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HADOOP-11455) KMS and Credential CLI should request confirmation for deletion by default
[ https://issues.apache.org/jira/browse/HADOOP-11455?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yi Liu updated HADOOP-11455: Issue Type: Improvement (was: Bug) > KMS and Credential CLI should request confirmation for deletion by default > -- > > Key: HADOOP-11455 > URL: https://issues.apache.org/jira/browse/HADOOP-11455 > Project: Hadoop Common > Issue Type: Improvement > Components: security >Affects Versions: 2.7.0 >Reporter: Charles Lamb >Assignee: Charles Lamb >Priority: Minor > Attachments: HADOOP-11455.000.patch, HADOOP-11455.001.patch > > > The hadoop key delete and hadoop credential delete currently only ask for > confirmation of the delete if -i is specified. Asking for confirmation should > be the default action for both. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HADOOP-11455) KMS and Credential CLI should request confirmation for deletion by default
[ https://issues.apache.org/jira/browse/HADOOP-11455?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14264221#comment-14264221 ] Yi Liu commented on HADOOP-11455: - +1, thanks Charles for the contribution, will commit it later. > KMS and Credential CLI should request confirmation for deletion by default > -- > > Key: HADOOP-11455 > URL: https://issues.apache.org/jira/browse/HADOOP-11455 > Project: Hadoop Common > Issue Type: Bug > Components: security >Affects Versions: 2.7.0 >Reporter: Charles Lamb >Assignee: Charles Lamb >Priority: Minor > Attachments: HADOOP-11455.000.patch, HADOOP-11455.001.patch > > > The hadoop key delete and hadoop credential delete currently only ask for > confirmation of the delete if -i is specified. Asking for confirmation should > be the default action for both. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HADOOP-11032) Replace use of Guava Stopwatch with Apache StopWatch
[ https://issues.apache.org/jira/browse/HADOOP-11032?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14264171#comment-14264171 ] Akira AJISAKA commented on HADOOP-11032: Looks good to me, +1. The findbug warnings and test failure are not related to the patch. We should edit the title because the patch does not use Apache StopWatch. > Replace use of Guava Stopwatch with Apache StopWatch > > > Key: HADOOP-11032 > URL: https://issues.apache.org/jira/browse/HADOOP-11032 > Project: Hadoop Common > Issue Type: Improvement >Reporter: Gary Steelman >Assignee: Tsuyoshi OZAWA > Attachments: HADOOP-11032.1.patch, HADOOP-11032.2.patch, > HADOOP-11032.3.patch, HADOOP-11032.3.patch, HADOOP-11032.3.patch, > HADOOP-11032.3.patch, HADOOP-11032.3.patch, HADOOP-11032.4.patch, > HADOOP-11032.5.patch, HADOOP-11032.6.patch, HADOOP-11032.7.patch, > HADOOP-11032.8.patch > > > This patch reduces Hadoop's dependency on an old version of guava. > Stopwatch.elapsedMillis() isn't part of guava past v16 and the tools I'm > working on use v17. > To remedy this and also reduce Hadoop's reliance on old versions of guava, we > can use the Apache StopWatch (org.apache.commons.lang.time.StopWatch) which > provides nearly equivalent functionality. apache.commons.lang is already a > dependency for Hadoop so this will not introduce new dependencies. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HADOOP-11318) Update the document for hadoop fs -stat
[
https://issues.apache.org/jira/browse/HADOOP-11318?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Akira AJISAKA updated HADOOP-11318:
---
Status: Patch Available (was: Open)
> Update the document for hadoop fs -stat
> ---
>
> Key: HADOOP-11318
> URL: https://issues.apache.org/jira/browse/HADOOP-11318
> Project: Hadoop Common
> Issue Type: Bug
> Components: documentation
>Affects Versions: 2.5.1
>Reporter: Akira AJISAKA
>Assignee: Akira AJISAKA
> Labels: newbie
> Attachments: HADOOP-11318-001.patch, HADOOP-11318-002.patch
>
>
> In FileSystemShell.apt.vm,
> {code}
> stat
>Usage: <<>>
>Returns the stat information on the path.
> {code}
> Now {{-stat}} accepts the below formats.
> * %b: Size of file in blocks
> * %g: Group name of owner
> * %n: Filename
> * %o: Block size
> * %r: replication
> * %u: User name of owner
> * %y: UTC date as "-MM-dd HH:mm:ss"
> * %Y: Milliseconds since January 1, 1970 UTC
> They should be documented.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HADOOP-11318) Update the document for hadoop fs -stat
[
https://issues.apache.org/jira/browse/HADOOP-11318?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Akira AJISAKA updated HADOOP-11318:
---
Status: Open (was: Patch Available)
> Update the document for hadoop fs -stat
> ---
>
> Key: HADOOP-11318
> URL: https://issues.apache.org/jira/browse/HADOOP-11318
> Project: Hadoop Common
> Issue Type: Bug
> Components: documentation
>Affects Versions: 2.5.1
>Reporter: Akira AJISAKA
>Assignee: Akira AJISAKA
> Labels: newbie
> Attachments: HADOOP-11318-001.patch, HADOOP-11318-002.patch
>
>
> In FileSystemShell.apt.vm,
> {code}
> stat
>Usage: <<>>
>Returns the stat information on the path.
> {code}
> Now {{-stat}} accepts the below formats.
> * %b: Size of file in blocks
> * %g: Group name of owner
> * %n: Filename
> * %o: Block size
> * %r: replication
> * %u: User name of owner
> * %y: UTC date as "-MM-dd HH:mm:ss"
> * %Y: Milliseconds since January 1, 1970 UTC
> They should be documented.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (HADOOP-11458) Dos Scripts fail when spaces are in Java Path
Berin Loritsch created HADOOP-11458: --- Summary: Dos Scripts fail when spaces are in Java Path Key: HADOOP-11458 URL: https://issues.apache.org/jira/browse/HADOOP-11458 Project: Hadoop Common Issue Type: Bug Affects Versions: 2.6.0 Reporter: Berin Loritsch The issue came to light when trying to work with PIG, however, I've had to either repair the DOS commands for hadoop-config.cmd or perform some work-arounds. #. The default install location for both JDK and JRE are in "Program Files" That space causes problems with the %JAVA% variable #. Adding quotes around the CLASSPATH and JAVA variables fix a great number of problems, but breaks the platform detection Due to my fix in the script breaking the platform detection, I can't recommend it outright. My work-around for the platform detection was to run it from the command line directly, and then embed the result in the JAVA_PLATFORM variable with those results. I.e. it will only work on my machine. This is an old problem that several projects had to fix when Java went to 1.3, not sure why it's back again. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HADOOP-11455) KMS and Credential CLI should request confirmation for deletion by default
[
https://issues.apache.org/jira/browse/HADOOP-11455?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14264018#comment-14264018
]
Hadoop QA commented on HADOOP-11455:
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12689985/HADOOP-11455.001.patch
against trunk revision ddc5be4.
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 2 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. There were no new javadoc warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:red}-1 findbugs{color}. The patch appears to introduce 2 new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in
hadoop-common-project/hadoop-common.
Test results:
https://builds.apache.org/job/PreCommit-HADOOP-Build/5360//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HADOOP-Build/5360//artifact/patchprocess/newPatchFindbugsWarningshadoop-common.html
Console output:
https://builds.apache.org/job/PreCommit-HADOOP-Build/5360//console
This message is automatically generated.
> KMS and Credential CLI should request confirmation for deletion by default
> --
>
> Key: HADOOP-11455
> URL: https://issues.apache.org/jira/browse/HADOOP-11455
> Project: Hadoop Common
> Issue Type: Bug
> Components: security
>Affects Versions: 2.7.0
>Reporter: Charles Lamb
>Assignee: Charles Lamb
>Priority: Minor
> Attachments: HADOOP-11455.000.patch, HADOOP-11455.001.patch
>
>
> The hadoop key delete and hadoop credential delete currently only ask for
> confirmation of the delete if -i is specified. Asking for confirmation should
> be the default action for both.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HADOOP-10610) Upgrade S3n fs.s3.buffer.dir to support multi directories
[ https://issues.apache.org/jira/browse/HADOOP-10610?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rohit Agarwal updated HADOOP-10610: --- Summary: Upgrade S3n fs.s3.buffer.dir to support multi directories (was: Upgrade S3n s3.fs.buffer.dir to support multi directories) > Upgrade S3n fs.s3.buffer.dir to support multi directories > - > > Key: HADOOP-10610 > URL: https://issues.apache.org/jira/browse/HADOOP-10610 > Project: Hadoop Common > Issue Type: Improvement > Components: fs/s3 >Affects Versions: 2.4.0 >Reporter: Ted Malaska >Assignee: Ted Malaska >Priority: Minor > Fix For: 2.6.0 > > Attachments: HADOOP-10610.patch, HADOOP_10610-2.patch, HDFS-6383.patch > > > fs.s3.buffer.dir defines the tmp folder where files will be written to before > getting sent to S3. Right now this is limited to a single folder which > causes to major issues. > 1. You need a drive with enough space to store all the tmp files at once > 2. You are limited to the IO speeds of a single drive > This solution will resolve both and has been tested to increase the S3 write > speed by 2.5x with 10 mappers on hs1. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HADOOP-10610) Upgrade S3n s3.fs.buffer.dir to support multi directories
[ https://issues.apache.org/jira/browse/HADOOP-10610?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rohit Agarwal updated HADOOP-10610: --- Description: fs.s3.buffer.dir defines the tmp folder where files will be written to before getting sent to S3. Right now this is limited to a single folder which causes to major issues. 1. You need a drive with enough space to store all the tmp files at once 2. You are limited to the IO speeds of a single drive This solution will resolve both and has been tested to increase the S3 write speed by 2.5x with 10 mappers on hs1. was: s3.fs.buffer.dir defines the tmp folder where files will be written to before getting sent to S3. Right now this is limited to a single folder which causes to major issues. 1. You need a drive with enough space to store all the tmp files at once 2. You are limited to the IO speeds of a single drive This solution will resolve both and has been tested to increase the S3 write speed by 2.5x with 10 mappers on hs1. > Upgrade S3n s3.fs.buffer.dir to support multi directories > - > > Key: HADOOP-10610 > URL: https://issues.apache.org/jira/browse/HADOOP-10610 > Project: Hadoop Common > Issue Type: Improvement > Components: fs/s3 >Affects Versions: 2.4.0 >Reporter: Ted Malaska >Assignee: Ted Malaska >Priority: Minor > Fix For: 2.6.0 > > Attachments: HADOOP-10610.patch, HADOOP_10610-2.patch, HDFS-6383.patch > > > fs.s3.buffer.dir defines the tmp folder where files will be written to before > getting sent to S3. Right now this is limited to a single folder which > causes to major issues. > 1. You need a drive with enough space to store all the tmp files at once > 2. You are limited to the IO speeds of a single drive > This solution will resolve both and has been tested to increase the S3 write > speed by 2.5x with 10 mappers on hs1. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HADOOP-11455) KMS and Credential CLI should request confirmation for deletion by default
[ https://issues.apache.org/jira/browse/HADOOP-11455?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14263994#comment-14263994 ] Charles Lamb commented on HADOOP-11455: --- Hi [~hitliuyi], Thanks for the review. Yes, that's a good idea. The attached .001 patch addresses this. Charles > KMS and Credential CLI should request confirmation for deletion by default > -- > > Key: HADOOP-11455 > URL: https://issues.apache.org/jira/browse/HADOOP-11455 > Project: Hadoop Common > Issue Type: Bug > Components: security >Affects Versions: 2.7.0 >Reporter: Charles Lamb >Assignee: Charles Lamb >Priority: Minor > Attachments: HADOOP-11455.000.patch, HADOOP-11455.001.patch > > > The hadoop key delete and hadoop credential delete currently only ask for > confirmation of the delete if -i is specified. Asking for confirmation should > be the default action for both. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HADOOP-11455) KMS and Credential CLI should request confirmation for deletion by default
[ https://issues.apache.org/jira/browse/HADOOP-11455?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Charles Lamb updated HADOOP-11455: -- Attachment: HADOOP-11455.001.patch > KMS and Credential CLI should request confirmation for deletion by default > -- > > Key: HADOOP-11455 > URL: https://issues.apache.org/jira/browse/HADOOP-11455 > Project: Hadoop Common > Issue Type: Bug > Components: security >Affects Versions: 2.7.0 >Reporter: Charles Lamb >Assignee: Charles Lamb >Priority: Minor > Attachments: HADOOP-11455.000.patch, HADOOP-11455.001.patch > > > The hadoop key delete and hadoop credential delete currently only ask for > confirmation of the delete if -i is specified. Asking for confirmation should > be the default action for both. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HADOOP-11450) Cleanup DistCpV1 not to use deprecated methods and fix javadocs
[
https://issues.apache.org/jira/browse/HADOOP-11450?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14263891#comment-14263891
]
Varun Saxena commented on HADOOP-11450:
---
[~ozawa], kindly review
> Cleanup DistCpV1 not to use deprecated methods and fix javadocs
> ---
>
> Key: HADOOP-11450
> URL: https://issues.apache.org/jira/browse/HADOOP-11450
> Project: Hadoop Common
> Issue Type: Bug
>Reporter: Tsuyoshi OZAWA
>Assignee: Varun Saxena
> Attachments: HADOOP-11450.001.patch
>
>
> Currently, DistCpV1 is using deprecated methods and having wrong javadocs. We
> should fix them.
> 1. DistCpV1.copy doesn't have dstpath, but javadoc has it.
> {code}
> /**
> * Copy a file to a destination.
> * @param srcstat src path and metadata
> * @param dstpath dst path
> {code}
> 2. Removing deprecated methods.
> {code}
> SequenceFile.Writer dir_writer = SequenceFile.createWriter(jobfs,
> jobConf, dstdirlist, Text.class, FilePair.class,
> SequenceFile.CompressionType.NONE);
> {code}
> {code}
> basedir = args.basedir.makeQualified(basefs);
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HADOOP-10584) ActiveStandbyElector goes down if ZK quorum become unavailable
[
https://issues.apache.org/jira/browse/HADOOP-10584?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14263809#comment-14263809
]
Peng Zhang commented on HADOOP-10584:
-
I met the similar error for YARN RM that enabled HA automatic-failover.
{noformat}
2015-01-04,12:42:30,682 FATAL org.apache.hadoop.ha.ActiveStandbyElector:
Received stat error from Zookeeper. code:CONNECTIONLOSS. Not retrying further
znode monitoring connection errors.
2015-01-04,12:42:30,886 INFO org.apache.zookeeper.ZooKeeper: Session:
0x2498936f2a8c448 closed
2015-01-04,12:42:30,888 INFO org.apache.zookeeper.ClientCnxn: EventThread shut
down
2015-01-04,12:42:30,888 FATAL
org.apache.hadoop.yarn.server.resourcemanager.ResourceManager: Received a
org.apache.hadoop.yarn.server.resourcemanager.RMFatalEvent of type
EMBEDDED_ELECTOR_FAILED. Cause:
Received stat error from Zookeeper. code:CONNECTIONLOSS. Not retrying further
znode monitoring connection errors.
2015-01-04,12:42:30,891 INFO org.apache.hadoop.util.ExitUtil: Exiting with
status 1
{noformat}
> ActiveStandbyElector goes down if ZK quorum become unavailable
> --
>
> Key: HADOOP-10584
> URL: https://issues.apache.org/jira/browse/HADOOP-10584
> Project: Hadoop Common
> Issue Type: Bug
> Components: ha
>Affects Versions: 2.4.0
>Reporter: Karthik Kambatla
>Assignee: Karthik Kambatla
>Priority: Critical
> Attachments: hadoop-10584-prelim.patch
>
>
> ActiveStandbyElector retries operations for a few times. If the ZK quorum
> itself is down, it goes down and the daemons will have to be brought up
> again.
> Instead, it should log the fact that it is unable to talk to ZK, call
> becomeStandby on its client, and continue to attempt connecting to ZK.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HADOOP-7611) SequenceFile.Sorter creates local temp files on HDFS
[
https://issues.apache.org/jira/browse/HADOOP-7611?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14263802#comment-14263802
]
Akshay Rai commented on HADOOP-7611:
[~ozawa], sorry for the late reply. I am not clear with the point you raised.
From what I can gather, the output directory is different from the directory
where the temporary/intermediate files are stored. The temporary files are
stored in a location specified by "mapred.local.dir" or "io.seqfile.local.dir"
and the output is stored in the location specified by the user(outFile,
parameter to the sort method).
However, the problem here is that the code creates the temporary files in hdfs
if it has the required permissions and the reason as explained in the
description.
> SequenceFile.Sorter creates local temp files on HDFS
>
>
> Key: HADOOP-7611
> URL: https://issues.apache.org/jira/browse/HADOOP-7611
> Project: Hadoop Common
> Issue Type: Bug
> Components: io
>Affects Versions: 0.20.2
> Environment: CentOS 5.6 64-bit, Oracle JDK 1.6.0_26 64-bit
>Reporter: Bryan Keller
>
> When using SequenceFile.Sorter to sort or merge sequence files that exist in
> HDFS, it attempts to create temp files in a directory structure specified by
> mapred.local.dir but on HDFS, not in the local file system. The problem code
> is in MergeQueue.merge(). Starting at line 2953:
> {code}
> Path outputFile = lDirAlloc.getLocalPathForWrite(
> tmpFilename.toString(),
> approxOutputSize, conf);
> LOG.debug("writing intermediate results to " + outputFile);
> Writer writer = cloneFileAttributes(
>
> fs.makeQualified(segmentsToMerge.get(0).segmentPathName),
> fs.makeQualified(outputFile),
> null);
> {code}
> The outputFile here is a local path without a scheme, e.g.
> "/mnt/mnt1/mapred/local", specified by the mapred.local.dir property. If we
> are sorting files on HDFS, the fs object is a DistributedFileSystem. The call
> to fs.makeQualified(outputFile) appends the fs object's scheme to the local
> temp path returned by lDirAlloc, e.g. hdfs://mnt/mnt1/mapred/local. This
> directory is then created (if the proper permissions are available) on HDFS.
> If the HDFS permissions are not available, the sort/merge fails even though
> the directories exist locally.
> The code should instead always use the local file system if retrieving a path
> from the mapred.local.dir property. The unit tests do not test this
> condition, they only test using the local file system for sort and merge.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
