Kip Kohn created GOBBLIN-2170:
---------------------------------
Summary: Implement dynamic scaling for Gobblin-on-Temporal
Key: GOBBLIN-2170
URL: https://issues.apache.org/jira/browse/GOBBLIN-2170
Project: Apache Gobblin
Issue Type: Bug
Components: gobblin-core
Reporter: Kip Kohn
Assignee: Abhishek Tiwari
Enable Gobblin-on-Temporal to dynamically scale up/down the number of workers
and also to reconfigure worker containers (including resource resizing).
Support a heterogeneous workforce of multiple simultaneous worker container
configurations.
Develop abstractions for implementing with the current GoT-atop-YARN deployment
that are also general enough for reuse even with containerization other than
YARN, such as with k8s.
Create a simple, human-readable syntax for Dynamic Scaling Directives that is
"FS-safe", where each may serve as a filename w/o excessive encoding. Then
allow an HDFS directory to function as a source of Dynamic Scaling Directives,
ordered by file modtime.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
