dabla commented on code in PR #76363:
URL: https://github.com/apache/airflow/pull/76363#discussion_r4453863781
##########
providers/apache/arrow/src/airflow/providers/apache/arrow/hooks/adbc.py:
##########
@@ -276,27 +274,30 @@ def insert_rows(
replace=False,
*,
executemany=False,
- fast_executemany=False,
+ fast_executemany=True,
Review Comment:
**[warning]** Turning ingest on by default changes behaviour for drivers or
servers that have no bulk ingest.
In ADBC, bulk ingest is optional for a driver. One that does not implement
it raises `NotSupportedError` when `adbc_ingest` is called, and some drivers
depend on the server for it. Before this PR, `insert_rows` worked on those
connections through `executemany`. After it, they fail unless every caller
passes `fast_executemany=False`, and `GenericTransfer` users have to find out
that `insert_args` is where to pass it. There are two ways out, and either is
fine:
- Fall back to binding when the first `adbc_ingest` raises
`NotSupportedError`, with one info line. The flag then stays a hint, which is
also how `PostgresHook` treats it under psycopg3.
- Keep the hard failure, and add a note under the `Changelog` header in
`providers/apache/arrow/docs/changelog.rst`. It should say that the default
load path is now bulk ingest (`COPY` on PostgreSQL), and that
`fast_executemany=False` restores the previous behaviour.
### [2]
[`providers/apache/arrow/src/airflow/providers/apache/arrow/hooks/adbc.py` line
262](https://github.com/apache/airflow/blob/2270d63e5ad605d201901aeaf8a21bd3a76a342e/providers/apache/arrow/src/airflow/providers/apache/arrow/hooks/adbc.py#L262)
<!-- side:RIGHT -->
**[nit]** Every `replace=True` call now logs this line, because
`fast_executemany` defaults to True.
The caller who upserts never asked for ingest, so the message reads like a
warning about something they did. Either `log.debug`, or drop it and let the
docstring of `replace` carry the explanation, as it already does.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
