[
https://issues.apache.org/jira/browse/YARN-5525?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15517594#comment-15517594
]
Botong Huang edited comment on YARN-5525 at 9/23/16 9:16 PM:
-------------------------------------------------------------
Thank [~subru] for the review and comments. Let me provide some context why we
are doing this. By changing {{AppLogAggregator}} we can only customize the per
app log aggregation behavior. One reason that we also want to customize
{{LogAggregationService}} is to allow a different root log directory to upload
logs to per app, whereas in the current implementation, logs in all apps are
uploaded to a same (configurable) location in
{{LogAggregationService.remoteRootLogDir}}. In general, if we want to make log
aggregation pluggable, I think {{LogAggregationService}} is the right place
because it is the entry service class.
Besides, I have a question regarding all the {{private}} to {{protected}}
changes, where I am sure what is the right thing to do. In general, making log
aggregation service class configurable will suffice and people can plug in any
other implementation they need, without needing to modify
{{LogAggregationService}} and {{AppLogAggregator}} at all. However, in our
case, we only need to customize some features out of the current
implementation. Rather copy all the code and modify on top, I implemented mine
extending the current ones, so that lots of nice feature are inherited. The
code is much less and easier to maintain, by simply override the methods that
we need to customize. This is where the {{private}} to {{protected}} changes
are needed so that the member variables and methods are visible to the
subclasses.
was (Author: botong):
Thank [~subru] for the review and comments. Let me provide some context why we
are doing this. By changing {{AppLogAggregator}} we can only customize the per
app log aggregation behavior. One reason that we also want to customize
{{LogAggregationService}} is to allow a different root log directory to upload
logs to per app, whereas in the current implementation, logs in every app are
uploaded to a same (configurable) location in
{{LogAggregationService.remoteRootLogDir}}. In general, if we want to make log
aggregation pluggable, I think {{LogAggregationService}} is the right place
because it is the entry service class.
Besides, I have a question regarding all the {{private}} to {{protected}}
changes, where I am sure what is the right thing to do. In general, making log
aggregation service class configurable will suffice and people can plug in any
other implementation they need, without needing to modify
{{LogAggregationService}} and {{AppLogAggregator}} at all. However, in our
case, we only need to customize some features out of the current
implementation. Rather copy all the code and modify on top, I implemented mine
extending the current ones, so that lots of nice feature are inherited. The
code is much less and easier to maintain, by simply override the methods that
we need to customize. This is where the {{private}} to {{protected}} changes
are needed so that the member variables and methods are visible to the
subclasses.
> Make log aggregation service class configurable
> -----------------------------------------------
>
> Key: YARN-5525
> URL: https://issues.apache.org/jira/browse/YARN-5525
> Project: Hadoop YARN
> Issue Type: Improvement
> Components: log-aggregation
> Reporter: Giovanni Matteo Fumarola
> Assignee: Botong Huang
> Priority: Minor
> Attachments: YARN-5525.v1.patch, YARN-5525.v2.patch,
> YARN-5525.v3.patch
>
>
> Make the log aggregation class configurable and extensible, so that
> alternative log aggregation behaviors like app specific log aggregation
> directory, log aggregation format can be implemented and plugged in.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
