raphaelauv opened a new pull request, #35844:
URL: https://github.com/apache/airflow/pull/35844
how should I manage typing , since mypy fail with
```
error: Argument "sql" to
"generate_openlineage_metadata_from_sql" of "SQLParser" has incompatible type
"Union[str, List[str],
Taragolis commented on PR #35844:
URL: https://github.com/apache/airflow/pull/35844#issuecomment-1825987456
I don't think this required. There is good builtin replacement for exactly
the same things already exists for the every operator parameter which supports
templated fields: PythonOpera
raphaelauv commented on PR #35844:
URL: https://github.com/apache/airflow/pull/35844#issuecomment-1826019503
the limitation with the current implementation is that the full templated
query is not available in airflow UI or logs
->
```python
import os
from airflow impo
raphaelauv commented on PR #35844:
URL: https://github.com/apache/airflow/pull/35844#issuecomment-1826044862
@Taragolis
thanks for the advise , it work but it mean 2 operators and it's "less
friendly"
```python
def build_sql(date):
return "select * f
eladkal commented on PR #35844:
URL: https://github.com/apache/airflow/pull/35844#issuecomment-1826405918
I am not sure the suggestion simplify things. I am worried that it might
cause more confusion.
I replied on a similar problem in
https://stackoverflow.com/a/72246305/14624409
raphaelauv commented on PR #35844:
URL: https://github.com/apache/airflow/pull/35844#issuecomment-1826453061
I would like to render values with airflow but since `params` can't be
templated and I don't want to put airflow templating directly in the .sql file
( for separation of concern and
raphaelauv commented on PR #35844:
URL: https://github.com/apache/airflow/pull/35844#issuecomment-1826453641
the "unload" I do
```
def unload_sql(sql_select_query: str, s3_export_path: str) -> str:
sql_select_query = sql_select_query.replace("'", "''")
return f"""
Taragolis commented on PR #35844:
URL: https://github.com/apache/airflow/pull/35844#issuecomment-1826743579
There is couple of solution already exists, note this not end of list, I
believe there is also exist couple solution, so this list not final
**Solution 1**:
one of the, is J
raphaelauv commented on PR #35844:
URL: https://github.com/apache/airflow/pull/35844#issuecomment-1826844693
I agree that a general solution for all operator is better than adding the
feature to that specific operator
So the main question is
Do we agree that a pythonOperator b
potiuk commented on PR #35844:
URL: https://github.com/apache/airflow/pull/35844#issuecomment-1850480565
> 2. we add to the baseoperator the possibility to add a callable to custom
the templating logic
I would very much prefer this
--
This is an automated message from the Apache Gi
github-actions[bot] commented on PR #35844:
URL: https://github.com/apache/airflow/pull/35844#issuecomment-1992784104
This pull request has been automatically marked as stale because it has not
had recent activity. It will be closed in 5 days if no further activity occurs.
Thank you for you
raphaelauv closed pull request #35844: feat: SQLExecuteQueryOperator
sql_callable
URL: https://github.com/apache/airflow/pull/35844
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comme
Taragolis commented on PR #35844:
URL: https://github.com/apache/airflow/pull/35844#issuecomment-2002131114
> I would very much prefer this (main reason is that PythonOperator -> Other
Operator is complex to write, maintain , introduces completely unnecessary task
in the DAG and is way slow
github-actions[bot] commented on PR #35844:
URL: https://github.com/apache/airflow/pull/35844#issuecomment-1911201949
This pull request has been automatically marked as stale because it has not
had recent activity. It will be closed in 5 days if no further activity occurs.
Thank you for you
14 matches
Mail list logo
