dstandish commented on code in PR #29058:
URL: https://github.com/apache/airflow/pull/29058#discussion_r1084803393
##########
airflow/cli/cli_parser.py:
##########
@@ -468,7 +468,18 @@ def string_lower_type(val):
help="Don't preserve purged records in an archive table.",
action="store_true",
)
-
+ARG_DB_EXPORT_TO_CSV = Arg(
Review Comment:
or what if we have output path and format. we could support only csv at the
moment but json down the road.
perhaps output path can optionally be `-` for stdout.
##########
airflow/utils/db_cleanup.py:
##########
@@ -159,6 +170,14 @@ def _do_delete(*, query, orm_model, skip_archive, session):
logger.debug("delete statement:\n%s", delete.compile())
session.execute(delete)
session.commit()
+ if export_to_csv:
+ if not output_path.startswith(AIRFLOW_HOME):
+ output_path = os.path.join(AIRFLOW_HOME, output_path)
Review Comment:
yeah, probably we should just accept whatever path the user has requested.
i guess depending on whether we want to create the parent dirs for the user if
they don't exist, we could either do so, or fail fast if they don't
##########
airflow/utils/db_cleanup.py:
##########
@@ -159,6 +170,14 @@ def _do_delete(*, query, orm_model, skip_archive, session):
logger.debug("delete statement:\n%s", delete.compile())
session.execute(delete)
session.commit()
+ if export_to_csv:
+ if not output_path.startswith(AIRFLOW_HOME):
+ output_path = os.path.join(AIRFLOW_HOME, output_path)
+ os.makedirs(output_path, exist_ok=True)
+ _to_csv(
+ target_table=target_table,
file_path=f"{output_path}/{target_table_name}.csv", session=session
+ )
+ skip_archive = True
Review Comment:
i reckon we should probably not force skip archive just because user wants
csv output. user can add skip archive also. wdyt?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
