potiuk commented on issue #27894: URL: https://github.com/apache/airflow/issues/27894#issuecomment-1326737975
> In Databricks SQL operator (and I believe in others as well), there was following strategy: always return only last result - previous results were always discarded. Primary reason for this was following: > > * When you have multiple SQL statements, first one usually create table, inserts, etc. And only when you have select as the last statement, then you get results. This matches the logic of the SQL's `BATCH` statement > * When you have multiple SQL statements their result may have different schema, but results will be processed only according to the latest schema, not schemas for corresponding result sets > > We may need to think a bit about it - should we return results for each of the statements, or not. If yes, then we need to return pairs of description + results for each SQL statement, instead of using only the latest statement Yes - I noticed that too now. With two caveats: * depends on the oprator what is the default (no problem) * it behaves differently when there is an "sql" passed and return_last is true -> then instead of one-element result array it returns the results It is surprisingly difficult to unwind teh original convoluted behaviour :) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
