jroachgolf84 commented on code in PR #74330:
URL: https://github.com/apache/airflow/pull/74330#discussion_r4309397847
##########
providers/databricks/src/airflow/providers/databricks/operators/databricks.py:
##########
@@ -1402,6 +1408,11 @@ def __init__(
self.databricks_retry_delay = databricks_retry_delay
self.databricks_retry_args = databricks_retry_args
self.wait_for_termination = wait_for_termination
+ if self.outlets and not self.wait_for_termination:
+ self.log.warning(
+ "outlets are set with wait_for_termination=False. The asset
event will mean the "
Review Comment:
I would rephrase this as:
```
`outlets` are set with `wait_for_termination=False`. This means that an
Asset event will be emitted when the Databricks run is submitted, not when the
table was refreshed.
```
##########
providers/databricks/src/airflow/providers/databricks/operators/databricks.py:
##########
@@ -1713,6 +1724,11 @@ def __init__(
self.databricks_retry_delay = databricks_retry_delay
self.databricks_retry_args = databricks_retry_args
self.wait_for_termination = wait_for_termination
+ if self.outlets and not self.wait_for_termination:
+ self.log.warning(
Review Comment:
Reword this in a similar way to what I did above.
##########
providers/databricks/tests/system/databricks/example_databricks.py:
##########
@@ -155,8 +168,9 @@
sql_statement = DatabricksSQLStatementsOperator(
task_id="sql_statement",
databricks_conn_id="databricks_default",
- statement="select * from default.my_airflow_table",
+ statement="insert into default.my_airflow_table values (3, 'test 3')",
Review Comment:
Just curious, why this changes here?
##########
providers/databricks/docs/operators/sql_statements.rst:
##########
@@ -109,3 +109,47 @@ An example usage of the ``DatabricksSQLStatementsSensor``
is as follows:
:language: python
:start-after: [START howto_sensor_databricks_sql_statement]
:end-before: [END howto_sensor_databricks_sql_statement]
+
+Explicit assets
Review Comment:
I don't know if this makes sense. Why does this line need to be included:
```
``DatabricksSQLStatementsOperator`` does not infer which Databricks tables a
SQL
statement writes to.
```
I don't think that there are other Operators that are inferring this in the
Databricks provider, is there?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
