pankajkoti commented on code in PR #49516:
URL: https://github.com/apache/airflow/pull/49516#discussion_r2071321959
##########
providers/databricks/tests/system/databricks/example_databricks_sensors.py:
##########
@@ -30,6 +31,8 @@
# [DAG name to be shown on Airflow UI]
DAG_ID = "example_databricks_sensor"
+WAREHOUSE_ID = os.environ.get("WAREHOUSE_ID", "") # Defaults to an empty
string to satisfy typing
Review Comment:
warehouse_id is a required param for the APIs to work. I guess we have
mentioned those as positional params but if it receives empty values I would
expect the APIs to fail and report an error.
But yes, like mentioned by @alexott might make sense to raise an error from
the operator/sensor if we get an empty warehouse_id
##########
providers/databricks/docs/operators/sql_statements.rst:
##########
@@ -55,3 +55,49 @@ An example usage of the ``DatabricksSQLStatementsOperator``
is as follows:
:language: python
:start-after: [START howto_operator_sql_statements]
:end-before: [END howto_operator_sql_statements]
+
+
+.. _howto/sensor:DatabricksSQLStatementsSensor:
+
+DatabricksSQLStatementsOperator
+===============================
+
+Use the
:class:`~airflow.providers.databricks.sensor.databricks.DatabricksSQLStatementsSensor`
to either submit a
+Databricks SQL Statement to Databricks using the
+`Databricks SQL Statement Execution API
<https://docs.databricks.com/api/workspace/statementexecution>`_, or pass
+a Statement ID to the Sensor and await for the query to terminate execution.
+
+
+Using the Operator
Review Comment:
```suggestion
Using the Sensor
```
##########
providers/databricks/docs/operators/sql_statements.rst:
##########
@@ -55,3 +55,49 @@ An example usage of the ``DatabricksSQLStatementsOperator``
is as follows:
:language: python
:start-after: [START howto_operator_sql_statements]
:end-before: [END howto_operator_sql_statements]
+
+
+.. _howto/sensor:DatabricksSQLStatementsSensor:
+
+DatabricksSQLStatementsOperator
Review Comment:
```suggestion
DatabricksSQLStatementsSensor
```
##########
providers/databricks/docs/operators/sql_statements.rst:
##########
@@ -55,3 +55,49 @@ An example usage of the ``DatabricksSQLStatementsOperator``
is as follows:
:language: python
:start-after: [START howto_operator_sql_statements]
:end-before: [END howto_operator_sql_statements]
+
+
+.. _howto/sensor:DatabricksSQLStatementsSensor:
Review Comment:
Since this is a sensor, might make sense to move the documentation to a
separate `sensors.rst` or something. Just an opinion.
##########
providers/databricks/docs/operators/sql_statements.rst:
##########
@@ -55,3 +55,49 @@ An example usage of the ``DatabricksSQLStatementsOperator``
is as follows:
:language: python
:start-after: [START howto_operator_sql_statements]
:end-before: [END howto_operator_sql_statements]
+
+
+.. _howto/sensor:DatabricksSQLStatementsSensor:
+
+DatabricksSQLStatementsOperator
+===============================
+
+Use the
:class:`~airflow.providers.databricks.sensor.databricks.DatabricksSQLStatementsSensor`
to either submit a
+Databricks SQL Statement to Databricks using the
+`Databricks SQL Statement Execution API
<https://docs.databricks.com/api/workspace/statementexecution>`_, or pass
+a Statement ID to the Sensor and await for the query to terminate execution.
+
+
+Using the Operator
+------------------
+
+The ``DatabricksSQLStatementsSensor`` does one of two things. The Sensor can
submit SQL statements to Databricks using
+the `/api/2.0/sql/statements/
<https://docs.databricks.com/api/workspace/statementexecution/executestatement>`_
+endpoint. However, the Sensor can also take the Statement ID of an
already-submitted SQL Statement and handle the
+response to that execution.
+
+It supports configurable execution parameters such as warehouse selection,
catalog, schema, and parameterized queries.
+The operator can either synchronously poll for query completion or run in a
deferrable mode for improved efficiency.
+
+The only required parameters for using the Sensor are:
+
+* One of ``statement`` or ``statement_id`` - The SQL statement to execute. The
statement can optionally be
+ parameterized, see parameters.
+* ``warehouse_id`` - Warehouse upon which to execute a statement.
+
+All other parameters are optional and described in the documentation for
``DatabricksSQLStatementsSensor`` including
+but not limited to:
+
+* ``catalog``
+* ``schema``
+* ``parameters``
+
+Examples
+--------
+
+An example usage of the ``DatabricksSQLStatementsOperator`` is as follows:
Review Comment:
```suggestion
An example usage of the ``DatabricksSQLStatementsSensor`` is as follows:
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
