[jira] [Commented] (HADOOP-8468) Umbrella of enhancements to support different failure and locality topologies
[ https://issues.apache.org/jira/browse/HADOOP-8468?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13503097#comment-13503097 ] Konstantin Shvachko commented on HADOOP-8468: - Junping, Checked your article with performance results. Got a question about it. How do you explain the performance gain with DFSIO? MapReduce-wise DFSIO is completely unaware of the locality of the data it reads, because input data is just the file with the file name that the mapper should read. So the input file with name of the file to read is local to task, but not the file that it then reads. Not saying there is anything wrong with your results, I just think it needs more explanation. > Umbrella of enhancements to support different failure and locality topologies > - > > Key: HADOOP-8468 > URL: https://issues.apache.org/jira/browse/HADOOP-8468 > Project: Hadoop Common > Issue Type: Improvement > Components: ha, io >Affects Versions: 1.0.0, 2.0.0-alpha >Reporter: Junping Du >Assignee: Junping Du > Attachments: HADOOP-8468-total.patch, HADOOP-8468-total-v3.patch, > HVE_Hadoop World Meetup 2012.pptx, HVE User Guide on branch-1(draft ).pdf, > Proposal for enchanced failure and locality topologies.pdf, Proposal for > enchanced failure and locality topologies (revised-1.0).pdf > > > The current hadoop network topology (described in some previous issues like: > Hadoop-692) works well in classic three-tiers network when it comes out. > However, it does not take into account other failure models or changes in the > infrastructure that can affect network bandwidth efficiency like: > virtualization. > Virtualized platform has following genes that shouldn't been ignored by > hadoop topology in scheduling tasks, placing replica, do balancing or > fetching block for reading: > 1. VMs on the same physical host are affected by the same hardware failure. > In order to match the reliability of a physical deployment, replication of > data across two virtual machines on the same host should be avoided. > 2. The network between VMs on the same physical host has higher throughput > and lower latency and does not consume any physical switch bandwidth. > Thus, we propose to make hadoop network topology extend-able and introduce a > new level in the hierarchical topology, a node group level, which maps well > onto an infrastructure that is based on a virtualized environment. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9066) Sorting for FileStatus[]
[ https://issues.apache.org/jira/browse/HADOOP-9066?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13503096#comment-13503096 ] david king commented on HADOOP-9066: Use the FileStatusTool to sort the array of FileStstus[], for examle: //Ascending the array of FileStatus[] FileStatusTool.sortFileStatus(inputFiles, new ConcreteFileStatusAscComparable()); //Descending the array of FileStatus[] FileStatusTool.sortFileStatus(inputFiles, new ConcreteFileStatusDescComparable()); > Sorting for FileStatus[] > > > Key: HADOOP-9066 > URL: https://issues.apache.org/jira/browse/HADOOP-9066 > Project: Hadoop Common > Issue Type: Improvement > Environment: java7 , RedHat9 , Hadoop 0.20.2 > ,eclipse-jee-juno-linux-gtk.tar.gz >Reporter: david king > Labels: patch > Attachments: ConcreteFileStatusAscComparable.java, > ConcreteFileStatusDescComparable.java, FileStatusComparable.java, > FileStatusTool.java, TestFileStatusTool.java > > > I will submit a batch of FileStatusTool that used to sort FileStatus by the > Comparator, the Comparator not only customer to realizate , but alse use the > example code. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9066) Sorting for FileStatus[]
[ https://issues.apache.org/jira/browse/HADOOP-9066?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13503095#comment-13503095 ] david king commented on HADOOP-9066: Hey Harsh J: what wrong with the batch? could you give me some suggestion? > Sorting for FileStatus[] > > > Key: HADOOP-9066 > URL: https://issues.apache.org/jira/browse/HADOOP-9066 > Project: Hadoop Common > Issue Type: Improvement > Environment: java7 , RedHat9 , Hadoop 0.20.2 > ,eclipse-jee-juno-linux-gtk.tar.gz >Reporter: david king > Labels: patch > Attachments: ConcreteFileStatusAscComparable.java, > ConcreteFileStatusDescComparable.java, FileStatusComparable.java, > FileStatusTool.java, TestFileStatusTool.java > > > I will submit a batch of FileStatusTool that used to sort FileStatus by the > Comparator, the Comparator not only customer to realizate , but alse use the > example code. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8928) Add ability to reset topologies on master nodes
[ https://issues.apache.org/jira/browse/HADOOP-8928?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13503032#comment-13503032 ] liang xie commented on HADOOP-8928: --- IMHO, add the clear method to CachedDNSToSwitchMapping is enough, since only *cache* class need to be clear, right? I'll give a quick patch per Shinichi's design > Add ability to reset topologies on master nodes > --- > > Key: HADOOP-8928 > URL: https://issues.apache.org/jira/browse/HADOOP-8928 > Project: Hadoop Common > Issue Type: Improvement > Components: net >Reporter: Shinichi Yamashita > > For a topology decision of DataNode and TaskTracker, ScriptBasedMapping > (probably TableMapping) confirms HashMap first. > To decide topology of DataNode and TaskTracker again, it is necessary to > restart NameNode and JobTracker. > Therefore, it is necessary to change (or clear) HashMap function without > restarting NameNode and JobTracker. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9085) start namenode failure,bacause pid of namenode pid file is other process pid or thread id before start namenode
[
https://issues.apache.org/jira/browse/HADOOP-9085?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13503010#comment-13503010
]
liaowenrui commented on HADOOP-9085:
1.when stop namenode is success,we should be delete the namenode pid file.
2.when we start namenode,we should be ps,not use kill -0.
code in hadoop-daemo.sh
if [ -f $pid ]; then
if kill -0 `cat $pid` > /dev/null 2>&1; then
echo $command running as process `cat $pid`. Stop it first.
exit 1
fi
fi
we will change it like this:
if [ -f $pid ]; then
tmppid=`cat $pid`
curpid=`ps -ww -eo pid,user,euid,cmd | grep "org.apache.hadoop.hdfs." |
grep "$command" | grep $tmppid | grep -v "grep" | awk '{print $1}'`
if [ -n "$curpid" ]; then
echo $command running as process `cat $pid`. Stop it first.
exit 1
fi
fi
> start namenode failure,bacause pid of namenode pid file is other process pid
> or thread id before start namenode
> ---
>
> Key: HADOOP-9085
> URL: https://issues.apache.org/jira/browse/HADOOP-9085
> Project: Hadoop Common
> Issue Type: Bug
> Components: bin
>Affects Versions: 2.0.1-alpha, 2.0.3-alpha
> Environment: NA
>Reporter: liaowenrui
> Fix For: 2.0.1-alpha, 2.0.2-alpha, 2.0.3-alpha
>
>
> pid of namenode pid file is other process pid or thread id before start
> namenode,start namenode will failure.because the pid of namenode pid file
> will be checked use kill -0 command before start namenode in hadoop-daemo.sh
> script.when pid of namenode pid file is other process pid or thread id,checkt
> is use kil -0 command,and the kill -0 will return success.it means the
> namenode is runing.in really,namenode is not runing.
> 2338 is dead namenode pid
> 2305 is datanode pid
> cqn2:/tmp # kill -0 2338
> cqn2:/tmp # ps -wweLo pid,ppid,tid | grep 2338
> 2305 1 2338
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HADOOP-9085) start namenode failure,bacause pid of namenode pid file is other process pid or thread id before start namenode
liaowenrui created HADOOP-9085: -- Summary: start namenode failure,bacause pid of namenode pid file is other process pid or thread id before start namenode Key: HADOOP-9085 URL: https://issues.apache.org/jira/browse/HADOOP-9085 Project: Hadoop Common Issue Type: Bug Components: bin Affects Versions: 2.0.1-alpha, 2.0.3-alpha Environment: NA Reporter: liaowenrui Fix For: 2.0.3-alpha, 2.0.2-alpha, 2.0.1-alpha pid of namenode pid file is other process pid or thread id before start namenode,start namenode will failure.because the pid of namenode pid file will be checked use kill -0 command before start namenode in hadoop-daemo.sh script.when pid of namenode pid file is other process pid or thread id,checkt is use kil -0 command,and the kill -0 will return success.it means the namenode is runing.in really,namenode is not runing. 2338 is dead namenode pid 2305 is datanode pid cqn2:/tmp # kill -0 2338 cqn2:/tmp # ps -wweLo pid,ppid,tid | grep 2338 2305 1 2338 -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8628) TableMapping init sets initialized flag prematurely
[ https://issues.apache.org/jira/browse/HADOOP-8628?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13503002#comment-13503002 ] liang xie commented on HADOOP-8628: --- IMHO, it's not a problem, cause the load() method has catch all exception... > TableMapping init sets initialized flag prematurely > --- > > Key: HADOOP-8628 > URL: https://issues.apache.org/jira/browse/HADOOP-8628 > Project: Hadoop Common > Issue Type: Bug >Affects Versions: 2.0.0-alpha, 3.0.0, 2.0.2-alpha >Reporter: Steve Loughran >Priority: Minor > > As reported in HADOOP-7030; the TableMapping class sets the initialized flag > to true before attempting to load the table. This means that it is set even > if the load failed, so preventing other attempts to load the file from > working without restarting the service. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8629) Add option for TableMapping to reload mapping file on match failure
[ https://issues.apache.org/jira/browse/HADOOP-8629?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13502997#comment-13502997 ] liang xie commented on HADOOP-8629: --- I prefer to provide an admin cmd to reload by administrator > Add option for TableMapping to reload mapping file on match failure > --- > > Key: HADOOP-8629 > URL: https://issues.apache.org/jira/browse/HADOOP-8629 > Project: Hadoop Common > Issue Type: Improvement > Components: util >Reporter: Steve Loughran >Priority: Minor > > As commented in HADOOP-7030, the table mapping topology mapper handles new > node addition worse than the script mapping, because the table mapping is > frozen for the life of the service. > I propose adding an option (true by default?) for the class to look at the > timestamp of the mapping file, and reload it on a change -if a hostname > lookup failed. > -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8231) Make topologies easier to set up and debug
[ https://issues.apache.org/jira/browse/HADOOP-8231?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13502996#comment-13502996 ] liang xie commented on HADOOP-8231: --- "hdfs dfsadmin -printTopology" is a good way to dump topology state > Make topologies easier to set up and debug > -- > > Key: HADOOP-8231 > URL: https://issues.apache.org/jira/browse/HADOOP-8231 > Project: Hadoop Common > Issue Type: Improvement > Components: util >Affects Versions: 0.23.1 >Reporter: Steve Loughran >Priority: Minor > Fix For: 3.0.0 > > > Topology scripts are a source of problems as they > # are site-specific. > # hard to get right. > # can have adverse consequences on cluster operation when they go wrong. > This issue is to group up the features needed to make it easier for ops > people to get their scripts up and running. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-9076) Fix QA bot in YARN project
[ https://issues.apache.org/jira/browse/HADOOP-9076?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Radim Kolar updated HADOOP-9076: Status: Open (was: Patch Available) > Fix QA bot in YARN project > -- > > Key: HADOOP-9076 > URL: https://issues.apache.org/jira/browse/HADOOP-9076 > Project: Hadoop Common > Issue Type: Bug > Components: build >Reporter: Radim Kolar >Priority: Trivial > > purpose of this ticket is to run automatic tests on patches to YARN project > which have not working QA bot. There seems to be zero interest in fixing QA > bot to make it work for YARN subproject. > If you need to get your patch for YARN tested, just subscribe and attach it > to this ticket. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9016) Provide unit tests for class org.apache.hadoop.fs.HarFileSystem.HarFSDataInputStream
[
https://issues.apache.org/jira/browse/HADOOP-9016?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13502884#comment-13502884
]
Hadoop QA commented on HADOOP-9016:
---
{color:green}+1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12554690/HADOOP-9016--c.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 1 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in
hadoop-common-project/hadoop-common hadoop-tools/hadoop-archives.
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results:
https://builds.apache.org/job/PreCommit-HADOOP-Build/1807//testReport/
Console output:
https://builds.apache.org/job/PreCommit-HADOOP-Build/1807//console
This message is automatically generated.
> Provide unit tests for class
> org.apache.hadoop.fs.HarFileSystem.HarFSDataInputStream
> -
>
> Key: HADOOP-9016
> URL: https://issues.apache.org/jira/browse/HADOOP-9016
> Project: Hadoop Common
> Issue Type: Improvement
>Reporter: Ivan A. Veselovsky
>Assignee: Ivan A. Veselovsky
>Priority: Minor
> Attachments: HADOOP-9016--b.patch, HADOOP-9016--c.patch,
> HADOOP-9016.patch
>
>
> unit-test coverage of classes
> org.apache.hadoop.fs.HarFileSystem.HarFSDataInputStream,
> org.apache.hadoop.fs.HarFileSystem.HarFSDataInputStream.HarFsInputStream is
> zero.
> Suggested to provide unit-tests covering these classes.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-9016) Provide unit tests for class org.apache.hadoop.fs.HarFileSystem.HarFSDataInputStream
[ https://issues.apache.org/jira/browse/HADOOP-9016?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ivan A. Veselovsky updated HADOOP-9016: --- Attachment: HADOOP-9016--c.patch Patch "HADOOP-9016--c.patch" enhances testing of HarInputStream methods and fixes several bugs in their implementations. > Provide unit tests for class > org.apache.hadoop.fs.HarFileSystem.HarFSDataInputStream > - > > Key: HADOOP-9016 > URL: https://issues.apache.org/jira/browse/HADOOP-9016 > Project: Hadoop Common > Issue Type: Improvement >Reporter: Ivan A. Veselovsky >Assignee: Ivan A. Veselovsky >Priority: Minor > Attachments: HADOOP-9016--b.patch, HADOOP-9016--c.patch, > HADOOP-9016.patch > > > unit-test coverage of classes > org.apache.hadoop.fs.HarFileSystem.HarFSDataInputStream, > org.apache.hadoop.fs.HarFileSystem.HarFSDataInputStream.HarFsInputStream is > zero. > Suggested to provide unit-tests covering these classes. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9076) Fix QA bot in YARN project
[ https://issues.apache.org/jira/browse/HADOOP-9076?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13502849#comment-13502849 ] Tom White commented on HADOOP-9076: --- You're right, it hasn't picked up the change. I replied to the thread. > Fix QA bot in YARN project > -- > > Key: HADOOP-9076 > URL: https://issues.apache.org/jira/browse/HADOOP-9076 > Project: Hadoop Common > Issue Type: Bug > Components: build >Reporter: Radim Kolar >Priority: Trivial > > purpose of this ticket is to run automatic tests on patches to YARN project > which have not working QA bot. There seems to be zero interest in fixing QA > bot to make it work for YARN subproject. > If you need to get your patch for YARN tested, just subscribe and attach it > to this ticket. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8698) Do not call unneceseary setConf(null) in Configured constructor
[ https://issues.apache.org/jira/browse/HADOOP-8698?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13502802#comment-13502802 ] Radim Kolar commented on HADOOP-8698: - any progress with review? Its really simple patch. > Do not call unneceseary setConf(null) in Configured constructor > --- > > Key: HADOOP-8698 > URL: https://issues.apache.org/jira/browse/HADOOP-8698 > Project: Hadoop Common > Issue Type: Bug > Components: conf >Affects Versions: 0.23.3, 3.0.0 >Reporter: Radim Kolar >Priority: Minor > Fix For: 0.24.0, 3.0.0 > > Attachments: setconf-null.txt > > > no-arg constructor of /org/apache/hadoop/conf/Configured calls setConf(null). > This is unnecessary and it increases complexity of setConf() code because you > have to check for not null object reference before using it. Under normal > conditions setConf() is never called with null reference, so not null check > is unnecessary. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-9076) Fix QA bot in YARN project
[ https://issues.apache.org/jira/browse/HADOOP-9076?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Radim Kolar updated HADOOP-9076: Attachment: (was: capacity-unit.txt) > Fix QA bot in YARN project > -- > > Key: HADOOP-9076 > URL: https://issues.apache.org/jira/browse/HADOOP-9076 > Project: Hadoop Common > Issue Type: Bug > Components: build >Reporter: Radim Kolar >Priority: Trivial > > purpose of this ticket is to run automatic tests on patches to YARN project > which have not working QA bot. There seems to be zero interest in fixing QA > bot to make it work for YARN subproject. > If you need to get your patch for YARN tested, just subscribe and attach it > to this ticket. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9076) Fix QA bot in YARN project
[
https://issues.apache.org/jira/browse/HADOOP-9076?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13502784#comment-13502784
]
Hadoop QA commented on HADOOP-9076:
---
{color:green}+1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12554680/capacity-unit.txt
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 1 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager.
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results:
https://builds.apache.org/job/PreCommit-HADOOP-Build/1802//testReport/
Console output:
https://builds.apache.org/job/PreCommit-HADOOP-Build/1802//console
This message is automatically generated.
> Fix QA bot in YARN project
> --
>
> Key: HADOOP-9076
> URL: https://issues.apache.org/jira/browse/HADOOP-9076
> Project: Hadoop Common
> Issue Type: Bug
> Components: build
>Reporter: Radim Kolar
>Priority: Trivial
> Attachments: capacity-unit.txt
>
>
> purpose of this ticket is to run automatic tests on patches to YARN project
> which have not working QA bot. There seems to be zero interest in fixing QA
> bot to make it work for YARN subproject.
> If you need to get your patch for YARN tested, just subscribe and attach it
> to this ticket.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-9076) Fix QA bot in YARN project
[ https://issues.apache.org/jira/browse/HADOOP-9076?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Radim Kolar updated HADOOP-9076: Attachment: capacity-unit.txt > Fix QA bot in YARN project > -- > > Key: HADOOP-9076 > URL: https://issues.apache.org/jira/browse/HADOOP-9076 > Project: Hadoop Common > Issue Type: Bug > Components: build >Reporter: Radim Kolar >Priority: Trivial > Attachments: capacity-unit.txt > > > purpose of this ticket is to run automatic tests on patches to YARN project > which have not working QA bot. There seems to be zero interest in fixing QA > bot to make it work for YARN subproject. > If you need to get your patch for YARN tested, just subscribe and attach it > to this ticket. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-9076) Fix QA bot in YARN project
[ https://issues.apache.org/jira/browse/HADOOP-9076?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Radim Kolar updated HADOOP-9076: Attachment: (was: max-running.txt) > Fix QA bot in YARN project > -- > > Key: HADOOP-9076 > URL: https://issues.apache.org/jira/browse/HADOOP-9076 > Project: Hadoop Common > Issue Type: Bug > Components: build >Reporter: Radim Kolar >Priority: Trivial > Attachments: capacity-unit.txt > > > purpose of this ticket is to run automatic tests on patches to YARN project > which have not working QA bot. There seems to be zero interest in fixing QA > bot to make it work for YARN subproject. > If you need to get your patch for YARN tested, just subscribe and attach it > to this ticket. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9076) Fix QA bot in YARN project
[
https://issues.apache.org/jira/browse/HADOOP-9076?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13502775#comment-13502775
]
Hadoop QA commented on HADOOP-9076:
---
{color:green}+1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12554679/max-running.txt
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 2 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager.
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results:
https://builds.apache.org/job/PreCommit-HADOOP-Build/1801//testReport/
Console output:
https://builds.apache.org/job/PreCommit-HADOOP-Build/1801//console
This message is automatically generated.
> Fix QA bot in YARN project
> --
>
> Key: HADOOP-9076
> URL: https://issues.apache.org/jira/browse/HADOOP-9076
> Project: Hadoop Common
> Issue Type: Bug
> Components: build
>Reporter: Radim Kolar
>Priority: Trivial
> Attachments: max-running.txt
>
>
> purpose of this ticket is to run automatic tests on patches to YARN project
> which have not working QA bot. There seems to be zero interest in fixing QA
> bot to make it work for YARN subproject.
> If you need to get your patch for YARN tested, just subscribe and attach it
> to this ticket.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-9076) Fix QA bot in YARN project
[ https://issues.apache.org/jira/browse/HADOOP-9076?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Radim Kolar updated HADOOP-9076: Attachment: max-running.txt > Fix QA bot in YARN project > -- > > Key: HADOOP-9076 > URL: https://issues.apache.org/jira/browse/HADOOP-9076 > Project: Hadoop Common > Issue Type: Bug > Components: build >Reporter: Radim Kolar >Priority: Trivial > Attachments: max-running.txt > > > purpose of this ticket is to run automatic tests on patches to YARN project > which have not working QA bot. There seems to be zero interest in fixing QA > bot to make it work for YARN subproject. > If you need to get your patch for YARN tested, just subscribe and attach it > to this ticket. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9049) DelegationTokenRenewer needs to be Singleton and FileSystems should register/deregister to/from.
[ https://issues.apache.org/jira/browse/HADOOP-9049?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13502758#comment-13502758 ] Hudson commented on HADOOP-9049: Integrated in Hadoop-Hdfs-trunk #1234 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk/1234/]) HADOOP-9049. DelegationTokenRenewer needs to be Singleton and FileSystems should register/deregister to/from. Contributed by Karthik Kambatla. (Revision 1412077) Result = SUCCESS tomwhite : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1412077 Files : * /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt * /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/DelegationTokenRenewer.java * /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/TestDelegationTokenRenewer.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/HftpFileSystem.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/web/WebHdfsFileSystem.java > DelegationTokenRenewer needs to be Singleton and FileSystems should > register/deregister to/from. > > > Key: HADOOP-9049 > URL: https://issues.apache.org/jira/browse/HADOOP-9049 > Project: Hadoop Common > Issue Type: Bug > Components: security >Affects Versions: 2.0.2-alpha >Reporter: Karthik Kambatla >Assignee: Karthik Kambatla > Fix For: 2.0.3-alpha > > Attachments: hadoop-9049.patch, hadoop-9049.patch > > > Currently, DelegationTokenRenewer is not singleton. > Each filesystem using it spawns its own DelegationTokenRenewer. Also, they > don't stop the Renewer leading to other problems. > A single DelegationTokenRenewer should be sufficient for all FileSystems. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8931) Add Java version to startup message
[ https://issues.apache.org/jira/browse/HADOOP-8931?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13502747#comment-13502747 ] Hudson commented on HADOOP-8931: Integrated in Hadoop-Hdfs-0.23-Build #443 (See [https://builds.apache.org/job/Hadoop-Hdfs-0.23-Build/443/]) HADOOP-8931. Add Java version to startup message. (eli) (Revision 1412294) Result = SUCCESS tgraves : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1412294 Files : * /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/CHANGES.txt * /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/StringUtils.java > Add Java version to startup message > > > Key: HADOOP-8931 > URL: https://issues.apache.org/jira/browse/HADOOP-8931 > Project: Hadoop Common > Issue Type: Improvement >Affects Versions: 1.0.0, 2.0.0-alpha >Reporter: Eli Collins >Assignee: Eli Collins >Priority: Trivial > Fix For: 1.2.0, 2.0.3-alpha, 0.23.6 > > Attachments: hadoop-8931-b1.txt, hadoop-8931.txt > > > I often look at logs and have to track down the java version they were run > with, it would be useful if we logged this as part of the startup message. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9049) DelegationTokenRenewer needs to be Singleton and FileSystems should register/deregister to/from.
[ https://issues.apache.org/jira/browse/HADOOP-9049?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13502711#comment-13502711 ] Hudson commented on HADOOP-9049: Integrated in Hadoop-Yarn-trunk #44 (See [https://builds.apache.org/job/Hadoop-Yarn-trunk/44/]) HADOOP-9049. DelegationTokenRenewer needs to be Singleton and FileSystems should register/deregister to/from. Contributed by Karthik Kambatla. (Revision 1412077) Result = SUCCESS tomwhite : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1412077 Files : * /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt * /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/DelegationTokenRenewer.java * /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/TestDelegationTokenRenewer.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/HftpFileSystem.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/web/WebHdfsFileSystem.java > DelegationTokenRenewer needs to be Singleton and FileSystems should > register/deregister to/from. > > > Key: HADOOP-9049 > URL: https://issues.apache.org/jira/browse/HADOOP-9049 > Project: Hadoop Common > Issue Type: Bug > Components: security >Affects Versions: 2.0.2-alpha >Reporter: Karthik Kambatla >Assignee: Karthik Kambatla > Fix For: 2.0.3-alpha > > Attachments: hadoop-9049.patch, hadoop-9049.patch > > > Currently, DelegationTokenRenewer is not singleton. > Each filesystem using it spawns its own DelegationTokenRenewer. Also, they > don't stop the Renewer leading to other problems. > A single DelegationTokenRenewer should be sufficient for all FileSystems. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
