[
https://issues.apache.org/jira/browse/HADOOP-4413?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vivek Ratan updated HADOOP-4413:
--------------------------------
Attachment: 4413.1.patch
Attaching the first patch (4413.1.patch), to get feedback on the approach.
* Much like _JobTrackerInstrumentation_, I've defined a
_CapacitySchedulerInstrumentation_ class that defines the events and data we
want to capture for the Capacity Scheduler.
* There is a single 'implementation' of this class, _CapacitySchedulerLogInst_,
that writes stuff to a log file. We may have implementations in the future that
interact with Chukwa directly.
* The time series data is captured the same way the scheduler UI does. The
Capacity Scheduler provides an object whose toString() method generates all the
data that needs to be captured. This is the same object used by the UI. A
thread in _CapacitySchedulerLogInst_ periodically writes this data to a log
file. The default period is 5 seconds, but can be overwritten through the
capacity scheduler's configuration.
* Events are written to the log file right away. If this proves to be
expensive, we can buffer them up (in a simple linked list of strings, perhaps)
and write them periodically as well. These events capture the main scheduler
decisions.
To be done:
* update documentation on capacity scheduler configuration
* log4J settings to log to a separate file
* make sure we're capturing all relevant events
Feedback welcome.
> Capacity Scheduler to provide a scheduler history log to record actions taken
> and why
> -------------------------------------------------------------------------------------
>
> Key: HADOOP-4413
> URL: https://issues.apache.org/jira/browse/HADOOP-4413
> Project: Hadoop Core
> Issue Type: Improvement
> Components: contrib/capacity-sched
> Reporter: Mac Yang
> Attachments: 4413.1.patch
>
>
> It would be very useful if the capacity scheduler can provide a log that
> record the decisions made and actions taken by the scheduler.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
