Nataneljpwd commented on code in PR #61143:
URL: https://github.com/apache/airflow/pull/61143#discussion_r2751611141
##########
providers/common/sql/src/airflow/providers/common/sql/operators/generic_transfer.py:
##########
@@ -139,6 +142,14 @@ def render_template_fields(
if isinstance(commit_every, str):
self.insert_args["commit_every"] = int(commit_every)
+ def _insert_rows(self, rows: list[Any], context: Context):
+ if self._rows_processor:
+ rows = self._rows_processor(rows, **context) # type: ignore
Review Comment:
Is there a specific rule ignored here? If so, I think it would be better to
ignore only it, rather than everything, I'm guessing it is there for the rows
parameter, which I think can be fixed by changing the return of the rows
processor to be of type list[Any] rather than just Any.
In addition, I think it is a good idea to document the method a little
better, as I had to read the code of the implementation to understand that it
is applied on all rows in a batch rather than one by one, I think fixing the
type hinting to
`Callable[[list[Any], Mapping[str, Any]], list[Any]]`
(Or even better, instead of the mapping use unpack for the airflow context
type) should suffice, it would also remove the need for the type ignore.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
