Components: contrib/fair-share
Reporter: Matei Zaharia
Assignee: Matei Zaharia
Fix For: 0.20.205.0
A lot of improvements have been made to the fair scheduler in 0.21, 0.22 and
trunk, but have not been ported back to the new 0.20.20X releases that are
e.avro.ipc.Server.start()V
>> at
>> org.apache.hadoop.yarn.server.resourcemanager.ResourceTrackerService.start(ResourceTrackerService.java:128)
>> at
>> org.apache.hadoop.yarn.service.CompositeService.start(CompositeService.java:68)
>> ... 2 more
>> 2011-08-11
I get a similar error on Mac OS X. I've built YARN and extracted the tarball to
a directory, but when I run bin/yarn-daemon.sh start nodemanager, it prints the
following to its log (apologies for the long trace):
log4j:WARN No appenders could be found for logger
(org.apache.hadoop.metrics2.impl
Hi Venu,
The paper compares against a naive implementation of fair scheduling, not
against the Hadoop Fair Scheduler. You can find out about the data locality
algorithms used in the Fair Scheduler in
http://www.cs.berkeley.edu/~matei/papers/2010/eurosys_delay_scheduling.pdf.
These are availabl
Components: contrib/fair-share, documentation
Affects Versions: 0.21.0, 0.20.2, 0.20.203.0
Reporter: Matei Zaharia
Assignee: Matei Zaharia
Priority: Minor
The fair scheduler already works with the ACL system introduced through the
mapred.queue.* parameters, but the
[
https://issues.apache.org/jira/browse/MAPREDUCE-831?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Matei Zaharia resolved MAPREDUCE-831.
-
Resolution: Fixed
Fix Version/s: 0.21.0
This was fixed for 0.21, but I forgot
[
https://issues.apache.org/jira/browse/MAPREDUCE-2140?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Matei Zaharia resolved MAPREDUCE-2140.
--
Resolution: Fixed
I've committed the regenerated design doc to trunk and 0
mprovement
Reporter: Matei Zaharia
Assignee: Matei Zaharia
Priority: Minor
In order to support running Hadoop on the Mesos cluster manager
(http://mesos.berkeley.edu/), I'd like to add an extra String field to the Task
class to allow extra data (a Mesos task ID) t
Zaharia
Assignee: Matei Zaharia
Priority: Trivial
Fix For: 0.21.1, 0.22.0
The Fair Scheduler contains a design document in
src/contrib/fairscheduler/designdoc that is included both as a Latex file and
as a PDF. However, the PDF that's currently there i
Congrats, Scott!
Matei
On Aug 27, 2010, at 2:30 PM, Scott Chen wrote:
> Thank you very much. It’s my honor.
>
> Scott
>
>
> On 8/27/10 5:20 PM, "Dhruba Borthakur" wrote:
>
> The Hadoop PMC has voted to make Scott Chen a committer for the Apache
> Hadoop Mapreduce project.
> Welcome Scott
Improve TaskTrackerInstrumentation
--
Key: MAPREDUCE-1881
URL: https://issues.apache.org/jira/browse/MAPREDUCE-1881
Project: Hadoop Map/Reduce
Issue Type: Improvement
Reporter: Matei Zaharia
: contrib/fair-share
Affects Versions: 0.21.0, 0.22.0
Reporter: Matei Zaharia
Assignee: Matei Zaharia
Priority: Blocker
In testing the fair scheduler with preemption, I found a deadlock between
updatePreemptionVariables and some code in the JobTracker. This was
: contrib/fair-share
Affects Versions: 0.21.0
Reporter: Matei Zaharia
Assignee: Matei Zaharia
Priority: Minor
The MAPREDUCE-706 patch seems to have left an unfinished TODO in the Fair
Scheduler - namely, in the dump() function for periodically dumping scheduler
[
https://issues.apache.org/jira/browse/MAPREDUCE-543?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Matei Zaharia resolved MAPREDUCE-543.
-
Resolution: Duplicate
Fix Version/s: 0.21.0
Yes indeed, this issue was fixed
/MAPREDUCE-878
Project: Hadoop Map/Reduce
Issue Type: Task
Components: contrib/fair-share
Reporter: Matei Zaharia
Priority: Trivial
As suggested by Tsz Wo Tze in MAPREDUCE-706.
--
This message is automatically generated by JIRA.
-
You can reply to
Sorry to be pestering but I'd still like an answer about where to
place the fair-scheduler.xml config file (in common/conf or in mapred/
conf).
Matei
On Jul 4, 2009, at 7:50 AM, Matei Zaharia wrote:
I've added a config file template for the fair scheduler recently
and placed
Put fair scheduler design doc in SVN
Key: MAPREDUCE-831
URL: https://issues.apache.org/jira/browse/MAPREDUCE-831
Project: Hadoop Map/Reduce
Issue Type: Task
Reporter: Matei Zaharia
-715
Project: Hadoop Map/Reduce
Issue Type: Improvement
Components: contrib/fair-share
Reporter: Matei Zaharia
Over time a number of elements have been added to the fair scheduler config
file for setting default running job limits, preemption timeouts
Issue Type: New Feature
Components: contrib/fair-share
Reporter: Matei Zaharia
Priority: Trivial
A common use case of the fair scheduler is to have one pool per user, but then
to define some special pools for various production jobs, import jobs, etc.
Therefore
: contrib/fair-share
Reporter: Matei Zaharia
The fair scheduler should support making the internal scheduling algorithm for
some pools be FIFO instead of fair sharing in order to work better for batch
workloads. FIFO pools will behave exactly like the current default scheduler,
sorting
: contrib/fair-share
Reporter: Matei Zaharia
Some users would like to be able to limit the number of tasks from a job that
they run on each node to better control cluster load. This JIRA will add this
feature to the fair scheduler. MAPREDUCE-698 adds per-pool limits on number of
running
: contrib/fair-share
Reporter: Matei Zaharia
Fix For: 0.21.0
The fair scheduler could use a way to cap the share of a given pool similar to
MAPREDUCE-532.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[
https://issues.apache.org/jira/browse/MAPREDUCE-540?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Matei Zaharia resolved MAPREDUCE-540.
-
Resolution: Fixed
Assignee: Matei Zaharia
Alright, closing the JIRA because it
23 matches
Mail list logo
