kacpermuda commented on code in PR #75303:
URL: https://github.com/apache/airflow/pull/75303#discussion_r4377695464
##########
providers/openlineage/src/airflow/providers/openlineage/utils/utils.py:
##########
@@ -441,7 +470,30 @@ def _get_openlineage_data_from_dagrun_conf(dr_conf: dict |
None) -> dict:
return ol_data if isinstance(ol_data, dict) else {}
-def get_root_information_from_dagrun_conf(dr_conf: dict | None) -> dict[str,
str]:
+def _get_job_type_from_ol_data(ol_data: dict, key: str) -> dict[str, str] |
None:
+ """Return a copy of the job type facet fields stored under ``key`` if they
are valid, otherwise None."""
+ job_type = ol_data.get(key)
+ if job_type is None:
+ return None
+ expected_fields = {"processingType", "integration", "jobType"}
+ # The spec allows partial job type, but only complete information is
supported for now.
+ if (
+ not isinstance(job_type, dict)
+ or job_type.keys() != expected_fields
Review Comment:
Agreed. It now requires all three fields as non-empty strings and ignores
the rest (e.g. _producer / _schemaURL), returning only the three. Partial types
are still ignored for now.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
