ramonvermeulen commented on issue #36963:
URL: https://github.com/apache/airflow/issues/36963#issuecomment-2669561945

   > ok I stopped being a lazy #_&$^&_(%
   > 
   > Check this out:
   > 
   > 
[airflow/providers/google/src/airflow/providers/google/cloud/operators/cloud_run.py](https://github.com/apache/airflow/blob/0adb5ad1cbee03309ffeffa91ec61722723713f1/providers/google/src/airflow/providers/google/cloud/operators/cloud_run.py#L297)
   > 
   > Line 297 in 
[0adb5ad](/apache/airflow/commit/0adb5ad1cbee03309ffeffa91ec61722723713f1)
   > 
   >  self.operation = hook.execute_job( 
   > This object has an attribute `metadata`:
   > 
   > The output when you print `metadata` looks something like this:
   > 
   > ```
   > name: 
"projects/{project}/locations/{region}/jobs/{job_name}/executions/{execution_name}"
   > uid: "{some_uuid}
   > generation: 1
   > labels {
   >   key: "{key}"
   >   value: "{value}"
   > }
   > labels {
   >   key: "nb-environment"
   >   value: "dev"
   > }
   > create_time {
   >   seconds: 1739988001
   >   nanos: 119240000
   > }
   > start_time {
   >   seconds: 1739988007
   >   nanos: 971451000
   > }
   > completion_time {
   >   seconds: 1739988070
   >   nanos: 580709000
   > }
   > update_time {
   >   seconds: 1739988001
   >   nanos: 119240000
   > }
   > launch_stage: GA
   > job: "{job}"
   > parallelism: 1
   > task_count: 1
   > template {
   >   containers {
   >     image: "{container}"
   >     args: "python3"
   >     args: "src/entrypoints/sync_api/cache_refresh.py"
   >     env {
   >       name: "ENVIRONMENT"
   >       value: "dev"
   >     }
   >     resources {
   >       limits {
   >         key: "memory"
   >         value: "2Gi"
   >       }
   >       limits {
   >         key: "cpu"
   >         value: "1"
   >       }
   >     }
   >   }
   >   max_retries: 0
   >   timeout {
   >     seconds: 600
   >   }
   >   service_account: "{service account}"
   >   execution_environment: EXECUTION_ENVIRONMENT_GEN2
   > }
   > conditions {
   >   type_: "Started"
   >   state: CONDITION_SUCCEEDED
   >   last_transition_time {
   >     seconds: 1739988007
   >     nanos: 971451000
   >   }
   > }
   > conditions {
   >   type_: "Completed"
   >   state: CONDITION_SUCCEEDED
   >   last_transition_time {
   >     seconds: 1739988070
   >     nanos: 580709000
   >   }
   > }
   > conditions {
   >   type_: "ContainerReady"
   >   state: CONDITION_SUCCEEDED
   >   last_transition_time {
   >     seconds: 1739988002
   >     nanos: 344637000
   >   }
   > }
   > conditions {
   >   type_: "ResourcesAvailable"
   >   state: CONDITION_SUCCEEDED
   >   last_transition_time {
   >     seconds: 1739988002
   >     nanos: 598798000
   >   }
   > }
   > conditions {
   >   type_: "Retry"
   >   state: CONDITION_SUCCEEDED
   >   message: "System will retry after 0:00:00 from lastTransitionTime for 
attempt 0."
   >   last_transition_time {
   >     seconds: 1739988072
   >     nanos: 489829000
   >   }
   >   severity: INFO
   >   reason: IMMEDIATE_RETRY
   > }
   > observed_generation: 1
   > succeeded_count: 1
   > log_uri: "{link to logs in log console}"
   > etag: "{something}"
   > ```
   > 
   > See the `log_uri` key? thatll take you direct to the logs in log console.
   > 
   > I have found exposing this in the airflow logs to be the most efficient, 
and doesnt run afoul of the 'call the logs client to get the logs if you have 
more than 60x operation a min' thing
   
   Thanks, I am also interested in this! It is a nice to have in our current 
project where we use cloud run jobs to run a 
[**dbt**](https://github.com/dbt-labs/dbt-core) workload every 15 minutes. 
Ideally we would have the container logs directly available in the airflow 
logs, however, reading through this issue this is quite complex. I already 
investigated a bit, and it is indeed quite easy to get a direct link to the 
logs in the GCP console which is quite nice. 
   
   Maybe adding a `bool` property to the operator `expose_logging_url`, that 
logs a direct link to the GCP logging could be a solution? What do you think 
@potiuk, if this is something desirable I am willing to put in some effort to 
make this happen.
   
   The idea is then that you just click the direct link from the airflow logs, 
which will open the cloud run job logs.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to