It is good to have hadoop-tools module separately. But as I asked before we
need to answer some questions here. I'm trying to answer them myself. Comments
are welcome.
> > 1. Should the patches for tools be created against Hadoop Common?
Here, I meant should Hadoop common mailing list be used O
Does a separate hadoop-tools module imply that there will be a separate Jenkins
build as well?
Thanks,
Joep
From: Alejandro Abdelnur [t...@cloudera.com]
Sent: Wednesday, September 07, 2011 11:35 AM
To: mapreduce-dev@hadoop.apache.org
Subject: Re: Hadoop T
See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Commit/858/
###
## LAST 60 LINES OF THE CONSOLE
###
[...truncated 9902 lines...]
[exec] depbase=`echo impl/
See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Commit/857/
###
## LAST 60 LINES OF THE CONSOLE
###
[...truncated 9902 lines...]
[exec] depbase=`echo impl/
Is there a jira for that? Seems like most tests wouldn't be MR1 specific.
On Wed, Sep 7, 2011 at 2:59 PM, Robert Evans wrote:
> There is a MiniYarnCluster and a MiniMRYarnCluster, it is just that the tests
> have not been ported over to use them yet.
>
> --Bobby
>
> On 9/7/11 2:01 PM, "Eli Coll
See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Commit/856/
###
## LAST 60 LINES OF THE CONSOLE
###
[...truncated 9902 lines...]
[exec] depbase=`echo impl/
See https://builds.apache.org/job/Hadoop-Mapreduce-22-branch/67/
###
## LAST 60 LINES OF THE CONSOLE
###
[...truncated 2258 lines...]
A src/examples/org/apache/h
There is a MiniYarnCluster and a MiniMRYarnCluster, it is just that the tests
have not been ported over to use them yet.
--Bobby
On 9/7/11 2:01 PM, "Eli Collins" wrote:
My understanding is that the MR1 code is currently needed to run the tests
because there is no Mini MR cluster for MR2. So t
See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Commit/855/
###
## LAST 60 LINES OF THE CONSOLE
###
[...truncated 9903 lines...]
[exec] g++ -DHAVE_CONF
Hadoop streaming test failure, post MR-2767
---
Key: MAPREDUCE-2948
URL: https://issues.apache.org/jira/browse/MAPREDUCE-2948
Project: Hadoop Map/Reduce
Issue Type: Bug
Components: contrib/st
My understanding is that the MR1 code is currently needed to run the tests
because there is no Mini MR cluster for MR2. So the code is needed until
the tests can run against MR2 (not sure if there's an effort underway).
However, see MR-2736, if we remove the ability to run the daemons I don't
thin
Makes sense
On Wed, Sep 7, 2011 at 11:32 AM, wrote:
> +1 for separate hadoop-tools module. However, if a tool is broken at
> release time, and no one comes forward to fix it, it should be removed.
> (i.e. Unlike contrib modules, where build and test failures were
> tolerated.)
>
> - milind
>
> O
+1 for separate hadoop-tools module. However, if a tool is broken at
release time, and no one comes forward to fix it, it should be removed.
(i.e. Unlike contrib modules, where build and test failures were
tolerated.)
- milind
On 9/7/11 11:27 AM, "Mahadev Konar" wrote:
>I like the idea of havin
I like the idea of having tools as a seperate module and I dont think
that it will be a dumping ground unless we choose to make one of it.
+1 for hadoop tools module under trunk.
thanks
mahadev
On Wed, Sep 7, 2011 at 11:18 AM, Alejandro Abdelnur wrote:
> Agreed, we should not have a dumping gro
Folks,
Has the community decided how long MRv1 will remain part of the codebase,
after 0.23 ? The reason I am asking is, for those who are working on
forward porting LinuxTaskController fixes (from 0.20.2xx) to 0.22, will
they have to patch 0.23 and trunk as well ? Or should these branches be
left
Agreed, we should not have a dumping ground. IMO, what it would go into
hadoop-tools (i.e. distcp, streaming and someone could argue for FsShell as
well) are effectively hadoop CLI utilities. Having them in a separate module
rather in than in the core module (common, hdfs, mapreduce) does not mean
Mapreduce and HDFS are distinct function of Hadoop. They are loosely
coupled. If we have tools aggregator module, it will not have as
clear distinct function as other Hadoop modules. Hence, it is
possible for a tool to be depend on both HDFS and map reduce. If
something broke in tools module, i
Sounds good, before proceeding I need closure on
https://issues.apache.org/jira/browse/HADOOP-7590?focusedCommentId=13094770&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-13094770in
order to tweak v2.
Thanks.
Alejandro
On Wed, Sep 7, 2011 at 10:13 AM, Vinod Kumar
Sort fails on YARN+MR with lots of task failures
Key: MAPREDUCE-2947
URL: https://issues.apache.org/jira/browse/MAPREDUCE-2947
Project: Hadoop Map/Reduce
Issue Type: Bug
Components:
+1 for doing this. I am willing to help too. But I think we should be
discussing about tools and contrib separately from this ticket (which is how
you started that issue BTW).
OTOH, this fixes only one part of the remaining bits. I've been seeing
first-hand from people working on some of the tools
Yes,
I think we need this, both for 0.23 and 0.24. The current form of build is
causing a lot of issues. Folks are not aware of checking ant after running
maven and unknowingly break the builds.
thanks
mahadev
On Sep 7, 2011, at 7:16 AM, Alejandro Abdelnur wrote:
> Do we still want/need to
[
https://issues.apache.org/jira/browse/MAPREDUCE-2946?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Owen O'Malley resolved MAPREDUCE-2946.
--
Resolution: Invalid
I forgot to chmod the task-controller to setuid. Sorry for the
TaskTrackers fail at startup
Key: MAPREDUCE-2946
URL: https://issues.apache.org/jira/browse/MAPREDUCE-2946
Project: Hadoop Map/Reduce
Issue Type: Bug
Components: tasktracker
Affects Versions: 0.20.2
Do we still want/need to do this?
Thanks.
Alejandro
There are a bunch of so called tools in hadoop-mapreduce-project/src/tools -
DistCp, HadoopArchives, Rumen etc. And contrib projects are in src/contrib
in all of common, hdfs and mapred source trees. Not sure how the distinction
was ever made.
The last time we had a discussion about moving contrib
See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/809/
###
## LAST 60 LINES OF THE CONSOLE
###
[...truncated 9393 lines...]
[ivy:resolve] -- artifact
com.sun
[
https://issues.apache.org/jira/browse/MAPREDUCE-2945?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Harsh J resolved MAPREDUCE-2945.
Resolution: Invalid
Arsalan,
The Apache JIRA is for filing issues or opening discussion topics
Hbase Batch Import Insertion Method
---
Key: MAPREDUCE-2945
URL: https://issues.apache.org/jira/browse/MAPREDUCE-2945
Project: Hadoop Map/Reduce
Issue Type: Task
Components: examples
Affects Vers
Improve checking of input for Api displayTasks()
-
Key: MAPREDUCE-2944
URL: https://issues.apache.org/jira/browse/MAPREDUCE-2944
Project: Hadoop Map/Reduce
Issue Type: Improvement
Co
Test-failures when using local-job-runner.
--
Key: MAPREDUCE-2943
URL: https://issues.apache.org/jira/browse/MAPREDUCE-2943
Project: Hadoop Map/Reduce
Issue Type: Bug
Components: distcp, mrv2
TestNMAuditLogger.testNMAuditLoggerWithIP failing
-
Key: MAPREDUCE-2942
URL: https://issues.apache.org/jira/browse/MAPREDUCE-2942
Project: Hadoop Map/Reduce
Issue Type: Bug
Reporter
See https://builds.apache.org/job/Hadoop-Mapreduce-22-branch/66/
###
## LAST 60 LINES OF THE CONSOLE
###
[...truncated 2259 lines...]
A src/examples/org/apache/h
Thanks Arun, I'll rebase/reimpl the ones where its necessary.
On Wed, Sep 7, 2011 at 1:31 PM, Arun C Murthy wrote:
> Also, I'll start to punt patches which only make sense for 'classic'
> MapReduce with MRv2 committed. Makes sense?
I'm fine with this, and would like it if you can also provide v
[
https://issues.apache.org/jira/browse/MAPREDUCE-2068?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Harsh J resolved MAPREDUCE-2068.
Resolution: Not A Problem
Thanks for checking in Arun,
Marking as resolved.
Not a problem any
Gang,
I'm starting to go through the PA queue and review stuff. Unfortunately a lot
of patches have gone stale due to recent merge of MR-279 and code restructuring
for maven. If you guys can rebase I promise to review all I can soon-ish and
commit appropriately. I'll use a fair bar for 0.23 si
Hi Venu,
Please go ahead and provide patches for them - I'll review.
thanks,
Arun
On Sep 5, 2011, at 9:47 PM, Venu Gopala Rao wrote:
> Hi Matt,
>
> The following defects are important defects encountered by us in our
> regular Hadoop usage. We have submitted patches to these defects in th
36 matches
Mail list logo
