ramonvermeulen commented on issue #36963: URL: https://github.com/apache/airflow/issues/36963#issuecomment-2669561945
> ok I stopped being a lazy #_&$^&_(% > > Check this out: > > [airflow/providers/google/src/airflow/providers/google/cloud/operators/cloud_run.py](https://github.com/apache/airflow/blob/0adb5ad1cbee03309ffeffa91ec61722723713f1/providers/google/src/airflow/providers/google/cloud/operators/cloud_run.py#L297) > > Line 297 in [0adb5ad](/apache/airflow/commit/0adb5ad1cbee03309ffeffa91ec61722723713f1) > > self.operation = hook.execute_job( > This object has an attribute `metadata`: > > The output when you print `metadata` looks something like this: > > ``` > name: "projects/{project}/locations/{region}/jobs/{job_name}/executions/{execution_name}" > uid: "{some_uuid} > generation: 1 > labels { > key: "{key}" > value: "{value}" > } > labels { > key: "nb-environment" > value: "dev" > } > create_time { > seconds: 1739988001 > nanos: 119240000 > } > start_time { > seconds: 1739988007 > nanos: 971451000 > } > completion_time { > seconds: 1739988070 > nanos: 580709000 > } > update_time { > seconds: 1739988001 > nanos: 119240000 > } > launch_stage: GA > job: "{job}" > parallelism: 1 > task_count: 1 > template { > containers { > image: "{container}" > args: "python3" > args: "src/entrypoints/sync_api/cache_refresh.py" > env { > name: "ENVIRONMENT" > value: "dev" > } > resources { > limits { > key: "memory" > value: "2Gi" > } > limits { > key: "cpu" > value: "1" > } > } > } > max_retries: 0 > timeout { > seconds: 600 > } > service_account: "{service account}" > execution_environment: EXECUTION_ENVIRONMENT_GEN2 > } > conditions { > type_: "Started" > state: CONDITION_SUCCEEDED > last_transition_time { > seconds: 1739988007 > nanos: 971451000 > } > } > conditions { > type_: "Completed" > state: CONDITION_SUCCEEDED > last_transition_time { > seconds: 1739988070 > nanos: 580709000 > } > } > conditions { > type_: "ContainerReady" > state: CONDITION_SUCCEEDED > last_transition_time { > seconds: 1739988002 > nanos: 344637000 > } > } > conditions { > type_: "ResourcesAvailable" > state: CONDITION_SUCCEEDED > last_transition_time { > seconds: 1739988002 > nanos: 598798000 > } > } > conditions { > type_: "Retry" > state: CONDITION_SUCCEEDED > message: "System will retry after 0:00:00 from lastTransitionTime for attempt 0." > last_transition_time { > seconds: 1739988072 > nanos: 489829000 > } > severity: INFO > reason: IMMEDIATE_RETRY > } > observed_generation: 1 > succeeded_count: 1 > log_uri: "{link to logs in log console}" > etag: "{something}" > ``` > > See the `log_uri` key? thatll take you direct to the logs in log console. > > I have found exposing this in the airflow logs to be the most efficient, and doesnt run afoul of the 'call the logs client to get the logs if you have more than 60x operation a min' thing Thanks, I am also interested in this! It is a nice to have in our current project where we use cloud run jobs to run a [**dbt**](https://github.com/dbt-labs/dbt-core) workload every 15 minutes. Ideally we would have the container logs directly available in the airflow logs, however, reading through this issue this is quite complex. I already investigated a bit, and it is indeed quite easy to get a direct link to the logs in the GCP console which is quite nice. Maybe adding a `bool` property to the operator `expose_logging_url`, that logs a direct link to the GCP logging could be a solution? What do you think @potiuk, if this is something desirable I am willing to put in some effort to make this happen. The idea is then that you just click the direct link from the airflow logs, which will open the cloud run job logs. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
