potiuk commented on issue #36963: URL: https://github.com/apache/airflow/issues/36963#issuecomment-1985727697
Well. if you find a way to create GCS objects (not stream to GCS - that is impossible) after the job is complete, and you follow naming convention of Airflow (which you have to be wary - is configurable and each airflow instance can have it's own configuration) and you would replace the usual logging that Airflow creaes in this place (for example starting task, seeing result of it etc) then likely yes, you could see logs from Cloud Run logs in the UI (but only AFTER Cloud Run and task completes) - streaming is impossible as mentioned - so basically you would see different logs (locally generated) while the task is running and different logs (only logs from Cloud Run) after the task completes. Of course this is only only under the condition that airlfow is configured to use GCP as remote logging - Airflow can use (in community providers) - with 10 different remote backends or so (and you can have your own custom logging handler). So the case where Airflow uses GCS as remote storate backend. All the other cases will not be handled at all And it's higly unlikely we would accept an operator that only provides a feature that works only in a very specific case (and replacing logs by removing those that are locally generated when Task is starting is also likely no go). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
