[
https://issues.apache.org/jira/browse/GOBBLIN-707?focusedWorklogId=226917&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-226917
]
ASF GitHub Bot logged work on GOBBLIN-707:
------------------------------------------
Author: ASF GitHub Bot
Created on: 12/Apr/19 20:43
Start Date: 12/Apr/19 20:43
Worklog Time Spent: 10m
Work Description: jhsenjaliya commented on pull request #2578:
[GOBBLIN-707] rewrite gobblin script to combine all modes and command
URL: https://github.com/apache/incubator-gobblin/pull/2578#discussion_r275056186
##########
File path: gobblin-docs/user-guide/Gobblin-CLI.md
##########
@@ -28,29 +28,29 @@ Gobblin ingestion applications
Gobblin ingestion applications can be accessed through the command `run`:
```bash
-bin/gobblin run [listQuickApps] [<quick-app>] -jobName <jobName> [OPTIONS]
+bin/gobblin cli run [listQuickApps] [<quick-app>] -jobName <jobName> [OPTIONS]
Review comment:
Hi @ibuenros , the `cli` has lot of options other than `run`, like
`passwordManager`, `watermarks`, etc... so we can keep it under `cli` category
for now, and in future, those options can be added directly to the `gobblin.sh`
if we really want to have `run` command directly under gobblin without `cli`.
btw, bringing all possible command line options at the `gobblin.sh` level could
be too much aggregation.
here the problem I m trying to solve is mostly inconsistency and lack of
clarity among functionality of all gobblin scripts. I have also added more
detail GOBBLIN-707 jira ticket to explain why we should combine all these
scripts.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
-------------------
Worklog Id: (was: 226917)
Time Spent: 3.5h (was: 3h 20m)
> combine & standardize all gobblin scripts into one master script &
> restructure configs accordingly
> --------------------------------------------------------------------------------------------------
>
> Key: GOBBLIN-707
> URL: https://issues.apache.org/jira/browse/GOBBLIN-707
> Project: Apache Gobblin
> Issue Type: Improvement
> Reporter: Jay Sen
> Priority: Major
> Time Spent: 3.5h
> Remaining Estimate: 0h
>
> gobblin supports multiple modes of executions ( CLI, Standalone,
> cluster-master, cluster-worker, AWS, YARN, MR ) and there is a individual
> script for each of them.
> Having individual script introduces lot of issues
> # all scripts handles gobblin variables, user parameters differently, and
> its highly inconsistent among various different gobblin scripts
> # functionality around start, stop, status checking and handling PID's among
> lot of other things, varies vastly as per the implementation of the script.
> # features like GC & JVM params, log4j file selection, classpath
> calculation, etc... exists in some gobblin scripts but not all, adding to
> inconsistent user experience.
> # maintaining total 13 script would be too much effort.
>
> All the gobblin scripts share lot of common code to handle params, start,
> stop services, status checks, pid handling, etc... combining all the scripts
> into 1 not only makes maintenance easier but also brings clarity and
> consistency.
>
> 1. there can be one gobblin.sh script
> {{gobblin.sh <command> <params>}}
> {{gobblin.sh <service-name> <start|stop|status>}}
> {{commands values: admin, cli, statestore-check, statestore-clean,
> historystore-manager}}
> {{service values: standalone, cluster-master, cluster-worker, aws, yarn, mr,
> service}}
> 2. Also configs needs to be structured and deduped accordingly.
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
