Yanislava opened a new pull request, #75967: URL: https://github.com/apache/airflow/pull/75967
## Problem vertica-python streams results: the error of a non-first statement in a single `execute()` is only raised once the client reaches its result set with `cursor.nextset()` (vertica/vertica-python#197). #32993 added `vertica_fetch_all_handler` to drain result sets, but it only applies when the handler is exactly `fetch_all_handler` **and** the first statement returns rows. As a result, a failing statement is silently ignored, the transaction is committed and the task succeeds when: - `handler=None`: `SQLExecuteQueryOperator` with `do_xcom_push=False` - the first statement has no result description, e.g. `TRUNCATE TABLE t; INSERT INTO t SELECT ...` - `fetch_one_handler` or a custom handler is used - `VerticaHook.run()` is called without a handler ## Reproduction Airflow 3 against a real Vertica; in every case the second statement is `SELECT 1/0`: | Case | Before | After | |---|---|---| | default handler, `SELECT 1; SELECT 1/0;` (already covered by #32993) | failed | failed | | `do_xcom_push=False` | **success** | failed | | default handler, `CREATE TABLE ...; SELECT 1/0;` | **success** | failed | | `handler=fetch_one_handler` | **success** | failed | | `VerticaHook().run(...)` without handler | **success** | failed | | `INSERT 1; SELECT 1/0; INSERT 2;`, `do_xcom_push=False` | **success** | failed | All "after" failures are `QueryError: Division by zero (Sqlstate 22012)`. For the last case, the buggy run committed a partial load (`[(1,)]` instead of `[(1,), (2,)]`) while the task was green; with the fix the task fails and nothing is committed (`[]`). <screenshots> ## Changes `VerticaHook.run` now drains all result sets after every statement regardless of the handler, including `None`. `nextset()` skips remaining rows without loading them into memory. Handler results are unchanged; `vertica_fetch_all_handler` is kept for backward compatibility and now drains unconditionally. ## Notes for reviewers - **Behaviour change:** tasks whose non-first statement fails, which previously succeeded silently (and could commit a partial load), will now fail and roll back. This is the intended fix, but users relying on such errors being ignored (e.g. `SELECT 1; DROP TABLE missing_table;`) will see failures after upgrading. Single-statement SQL, `split_statements=True` and `sql` passed as a list are not affected. - `hook.descriptions` for multi-statement SQL executed without splitting now reflects the last result set for all handlers, as it already did with `fetch_all_handler`. - Two existing tests in `test_vertica_sql.py` mocked several result sets for `split_statements=True`, which can't happen since each statement is executed separately; their mocks were adjusted. related: #32993, #20892 --- ##### Was generative AI tooling used to co-author this PR? - [X] Yes (please specify the tool below) Generated-by: Claude Code following [the guidelines](https://github.com/apache/airflow/blob/main/contributing-docs/05_pull_requests.rst#gen-ai-assisted-contributions) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
