[airflow] branch main updated: Update Error details for Generic Error Code (#32847)

[potiuk]

This is an automated email from the ASF dual-hosted git repository.

potiuk pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/airflow.git


The following commit(s) were added to refs/heads/main by this push:
     new b4102ce0b5 Update Error details for Generic Error Code  (#32847)
b4102ce0b5 is described below

commit b4102ce0b55e76baadf3efdec0df54762001f38c
Author: Srabasti Banerjee <srabast...@ymail.com>
AuthorDate: Mon Aug 14 03:52:03 2023 -0700

    Update Error details for Generic Error Code  (#32847)
    
    Adding details of Error as discussed per Issue - Improved error logging for 
failed Dataflow jobs #32107
    
    Co-authored-by: Hussein Awala <huss...@awala.fr>
---
 airflow/providers/google/cloud/hooks/dataflow.py | 6 +++++-
 1 file changed, 5 insertions(+), 1 deletion(-)

diff --git a/airflow/providers/google/cloud/hooks/dataflow.py 
b/airflow/providers/google/cloud/hooks/dataflow.py
index 19342cb26b..559164a2d0 100644
--- a/airflow/providers/google/cloud/hooks/dataflow.py
+++ b/airflow/providers/google/cloud/hooks/dataflow.py
@@ -1014,8 +1014,12 @@ class DataflowHook(GoogleBaseHook):
         self.log.info("Output: %s", proc.stdout.decode())
         self.log.warning("Stderr: %s", proc.stderr.decode())
         self.log.info("Exit code %d", proc.returncode)
+        stderr_last_20_lines = 
"\n".join(proc.stderr.decode().strip().splitlines()[-20:])
         if proc.returncode != 0:
-            raise AirflowException(f"Process exit with non-zero exit code. 
Exit code: {proc.returncode}")
+            raise AirflowException(
+                f"Process exit with non-zero exit code. Exit code: 
{proc.returncode} Error Details : "
+                f"{stderr_last_20_lines}"
+            )
         job_id = proc.stdout.decode().strip()
 
         self.log.info("Created job ID: %s", job_id)