Yanislava commented on code in PR #75967:
URL: https://github.com/apache/airflow/pull/75967#discussion_r4505241652
##########
providers/vertica/src/airflow/providers/vertica/hooks/vertica.py:
##########
@@ -29,13 +30,47 @@
from sqlalchemy.engine import URL
+def _drain_result_sets(cursor) -> None:
+ """
+ Advance the cursor through all remaining result sets.
+
+ vertica-python streams results: an error raised by a statement other than
the first one in a
+ multi-statement query is only reported once the client reaches its result
set. ``nextset()``
+ discards the remaining rows of the current set (without loading them in
memory) and raises
+ the server error if the next set is an error.
+
+ ``nextset()`` also overwrites ``cursor.description`` with the columns of
each next result set,
+ while ``DbApiHook.run`` reads it after the handler returns. The
description of the result set
+ the handler consumed is restored, so that ``hook.descriptions`` matches
the returned rows.
+
+ See https://github.com/apache/airflow/issues/32993 and
+ https://github.com/vertica/vertica-python/issues/197.
+ """
+ description = cursor.description
+ while cursor.nextset():
Review Comment:
You're right, thanks for digging into vertica-python here. The drain now
reads each result set with fetchone() before calling nextset(), so the error is
raised wherever it arrives, and this also avoids the hang on 1.3.0–1.3.4.
Verified on a real Vertica with `SELECT 1; SELECT a/b FROM t` (the second
row divides by zero):
- installed provider, no handler: task succeeds (the original bug)
- drain with nextset() only: task succeeds, as you described
- new drain: task fails with "Division by zero", both with no handler and
with fetch_all_handler
I also checked that SQL with an empty statement or comments (`SELECT 1; ;
SELECT 2`, a trailing `-- comment`) still runs fine with the new drain. Added
unit tests where nextset() returns True and the next fetchone() raises, both
for a later result set and for the rest of the set the handler consumed.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
