[
https://issues.apache.org/jira/browse/GOBBLIN-1162?focusedWorklogId=439147&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-439147
]
ASF GitHub Bot logged work on GOBBLIN-1162:
-------------------------------------------
Author: ASF GitHub Bot
Created on: 30/May/20 20:19
Start Date: 30/May/20 20:19
Worklog Time Spent: 10m
Work Description: sv2000 opened a new pull request #3002:
URL: https://github.com/apache/incubator-gobblin/pull/3002
…icide when unhealthy
Dear Gobblin maintainers,
Please accept this PR. I understand that it will not be reviewed until I
have checked off all the steps below!
### JIRA
- [x] My PR addresses the following [Gobblin
JIRA](https://issues.apache.org/jira/browse/GOBBLIN/) issues and references
them in the PR title. For example, "[GOBBLIN-XXX] My Gobblin PR"
- https://issues.apache.org/jira/browse/GOBBLIN-1162
### Description
- [x] Here are some details about my PR, including screenshots (if
applicable):
In execution environments such as Gobblin-on-Yarn, where Gobblin workers can
be re-assigned when the worker dies or is killed, it is useful to add a mode
where each Gobblin task running inside a Gobblin worker can perform
application-level health checks and report results of the health checks back to
the worker hosting the tasks. The worker on receiving a health check failure
event can be configured to exit the JVM. In the case of Gobblin-on-Yarn mode,
this will result in the worker getting re-assigned to a different node. The
proposed behavior is similar in flavor to speculative execution modes provided
in other execution frameworks such as MapReduce.
This change also provides an example of such an application-level health
check that arises in the case of Kafka ingestion. In addition, this PR adds an
EventBusFactory as an implementation of SharedResourcesBrokerFactory that
allows communication between different components of GobblinTaskRunner via a
shared EventBus instance.
### Tests
- [x] My PR adds the following unit tests __OR__ does not need testing for
this extremely good reason:
Added unit tests:
1.EventBusFactoryTest
2. GobblinTaskRunnerTest
3. KafkaIngestionHealthCheckTest
### Commits
- [x] My commits all reference JIRA issues in their subject lines, and I
have squashed multiple commits if they address the same issue. In addition, my
commits follow the guidelines from "[How to write a good git commit
message](http://chris.beams.io/posts/git-commit/)":
1. Subject is separated from body by a blank line
2. Subject is limited to 50 characters
3. Subject does not end with a period
4. Subject uses the imperative mood ("add", not "adding")
5. Body wraps at 72 characters
6. Body explains "what" and "why", not "how"
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
-------------------
Worklog Id: (was: 439147)
Time Spent: 2h 10m (was: 2h)
> Provide an option to allow slow containers to commit suicide when unhealthy
> ---------------------------------------------------------------------------
>
> Key: GOBBLIN-1162
> URL: https://issues.apache.org/jira/browse/GOBBLIN-1162
> Project: Apache Gobblin
> Issue Type: Improvement
> Components: gobblin-cluster
> Affects Versions: 0.15.0
> Reporter: Sudarshan Vasudevan
> Assignee: Hung Tran
> Priority: Major
> Fix For: 0.15.0
>
> Time Spent: 2h 10m
> Remaining Estimate: 0h
>
> In execution environments such as Gobblin-on-Yarn, where Gobblin workers can
> be re-assigned when the worker dies or is killed, it is useful to add a mode
> where each Gobblin task running inside a Gobblin worker can perform
> application-level health checks and report results of the health checks back
> to the worker hosting the tasks. The worker on receiving a health check
> failure event can be configured to exit the JVM. In the case of
> Gobblin-on-Yarn mode, this will result in the worker getting re-assigned to a
> different node. The proposed behavior is similar in flavor to speculative
> execution modes provided in other execution frameworks such as MapReduce.
> This change also provides an example of such an application-level health
> check that arises in the case of Kafka ingestion.
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
