Jacky Wang created SPARK-52810:
----------------------------------
Summary: [SDP] Pipelines CLI Selection Options
Key: SPARK-52810
URL: https://issues.apache.org/jira/browse/SPARK-52810
Project: Spark
Issue Type: Sub-task
Components: Declarative Pipelines
Affects Versions: 4.1.0
Reporter: Jacky Wang
We want to give user the ability to choose a subset of datasets (ex: tables,
materialized views) to include in a run.
And the ability to specify if they should ran as regular refresh or full
refresh.
Below arguments can be added to the spark-pipelines CLI to achieve this
{*}--full-refresh strings{*}: List of tables to reset and recompute.
{*}--full-refresh-all{*}: Perform a full graph reset and recompute.
{*}--refresh{*}: List of tables to update.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
