This is an automated email from the ASF dual-hosted git repository. mobuchowski pushed a change to branch openlineage-process-execution in repository https://gitbox.apache.org/repos/asf/airflow.git
discard 2a1c57a346 openlineage: execute extraction and message sending in separate process discard e2acd51121 local task job: add timeout, to not kill on_task_instance_success listener prematurely add 914bccc387 Improve trigger UI for string array format validation (#39993) add 1ec3b39f28 Resolve www views deprecations in tests (#40009) add 32cf8cf37d Update index.rst (#40040) add ea682382bc Adding Glue Data Quality Rule Recommendation Run (#40014) add 21dae6ee39 Resolve pagerduty deprecations in tests (#39945) add f6148e6e33 Update dataplex.py (#40041) add 5d227860ad Fix `DbtCloudRunJobOperator` to Use Correct Status Parameters for `reuse_existing_run` (#40048) add 99dd24b436 Doc-only: mention minimum boto3 1.34.52 for AWS provider when using Batch `ecs_properties_override` (#39983) add 83c118413c Add `retry_from_failure` parameter to DbtCloudRunJobOperator (#38868) add 06a200d5fa Bump boto min versions (#40052) add 0f6e31f4b9 Fix reattach_on_restart parameter for the sync mode (#39329) add 73d02e0f21 Add PID and return code to _execute_in_fork logging (#40058) add c6f85f08f8 Set PR #39638 for Airflow 2.10 (#40044) add c202c07f67 Introduce AirflowJobFacet and AirflowStateRunFacet (#39520) add 07982b0e83 Add @utkarsharma2 in commiter list. (#40065) add 9bcf665570 Fix typos in overview.rst (#40061) add ac2a1ef81d Expllicitly skip API tests for compatiblity tests on 2.7 and 2.8 (#40059) add ef1c9a2a2b Chart: Allow AWS Executors (#38524) add e7f11b58b6 Resolve databricks deprecations in tests (#40068) add 4dea367047 Add unit test to cover back compat case in celery (#40035) add fcb9f87c55 fix typo and path to licences folder (#40067) add 16e0182025 Docs: Simpler command to check local scheduler is alive (#40074) add f28ee5a694 Resolve core operators deprecations in tests (#39992) add 45bf7b9721 Resolve dagbag deprecations in tests (#39989) add ec1b19417c local task job: add timeout, to not kill on_task_instance_success listener prematurely new 23849615a9 openlineage: execute extraction and message sending in separate process This update added new revisions after undoing existing revisions. That is to say, some revisions that were in the old version of the branch are not in the new version. This situation occurs when a user --force pushes a change and generates a repository containing something like this: * -- * -- B -- O -- O -- O (2a1c57a346) \ N -- N -- N refs/heads/openlineage-process-execution (23849615a9) You should already have received notification emails for all of the O revisions, and so the following emails describe only the N revisions from the common base, B. Any revisions marked "omit" are not gone; other references still refer to them. Any revisions marked "discard" are gone forever. The 1 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: Dockerfile.ci | 6 + LICENSE | 8 +- airflow/_vendor/README.md | 7 +- ...exes_on_dag_id_column_in_referencing_tables.py} | 2 +- airflow/providers/amazon/aws/hooks/glue.py | 23 ++ airflow/providers/amazon/aws/operators/bedrock.py | 26 +- airflow/providers/amazon/aws/operators/glue.py | 146 +++++++ airflow/providers/amazon/aws/sensors/glue.py | 125 +++++- airflow/providers/amazon/aws/triggers/glue.py | 35 ++ airflow/providers/amazon/aws/waiters/glue.json | 49 +++ airflow/providers/amazon/provider.yaml | 16 +- .../celery/executors/celery_executor_utils.py | 2 +- airflow/providers/cncf/kubernetes/operators/pod.py | 57 ++- airflow/providers/dbt/cloud/hooks/dbt.py | 33 ++ airflow/providers/dbt/cloud/operators/dbt.py | 9 +- .../providers/google/cloud/operators/dataplex.py | 2 +- .../openlineage/facets/AirflowJobFacet.json | 40 ++ .../openlineage/facets/AirflowRunFacet.json | 254 ++++++++++++ .../facets/AirflowStateRunFacet.json} | 25 +- .../openlineage/facets}/__init__.py | 0 airflow/providers/openlineage/plugins/adapter.py | 20 +- airflow/providers/openlineage/plugins/facets.py | 47 ++- airflow/providers/openlineage/plugins/listener.py | 4 + airflow/providers/openlineage/utils/utils.py | 186 ++++++++- airflow/utils/db.py | 2 +- airflow/www/templates/airflow/trigger.html | 2 +- chart/README.md | 3 + chart/values.schema.json | 4 +- dev/breeze/doc/images/output_build-docs.svg | 28 +- dev/breeze/doc/images/output_build-docs.txt | 2 +- ...elease-management_prepare-provider-packages.svg | 26 +- ...elease-management_prepare-provider-packages.txt | 2 +- .../output_release-management_publish-docs.svg | 28 +- .../output_release-management_publish-docs.txt | 2 +- .../airflow_breeze/commands/developer_commands.py | 2 +- .../commands/release_management_commands.py | 4 +- .../operators/glue.rst | 28 ++ .../operators.rst | 4 + .../logging-monitoring/check-health.rst | 2 +- docs/apache-airflow/core-concepts/overview.rst | 4 +- docs/apache-airflow/core-concepts/params.rst | 6 +- docs/apache-airflow/img/airflow_erd.sha256 | 2 +- docs/apache-airflow/installation/index.rst | 2 +- docs/apache-airflow/migrations-ref.rst | 2 +- docs/apache-airflow/project.rst | 1 + docs/spelling_wordlist.txt | 4 + generated/provider_dependencies.json | 14 +- helm_tests/airflow_aux/test_basic_helm_chart.py | 24 +- scripts/docker/entrypoint_ci.sh | 6 + tests/deprecations_ignore.yml | 103 ----- .../integration/executors/test_celery_executor.py | 26 +- tests/models/test_dag.py | 267 +++++++++--- tests/models/test_dagbag.py | 8 +- tests/operators/test_bash.py | 4 +- tests/operators/test_branch_operator.py | 4 + tests/operators/test_datetime.py | 7 +- tests/operators/test_latest_only_operator.py | 6 +- tests/providers/amazon/aws/hooks/test_glue.py | 21 + tests/providers/amazon/aws/operators/test_glue.py | 108 +++++ .../amazon/aws/sensors/test_glue_data_quality.py | 138 ++++++- tests/providers/amazon/aws/triggers/test_glue.py | 28 ++ tests/providers/amazon/aws/waiters/test_glue.py | 38 +- .../cncf/kubernetes/operators/test_pod.py | 69 ++++ .../databricks/hooks/test_databricks_sql.py | 17 +- .../sensors/test_databricks_partition.py | 1 + .../databricks/sensors/test_databricks_sql.py | 1 + tests/providers/dbt/cloud/hooks/test_dbt.py | 56 +++ tests/providers/dbt/cloud/operators/test_dbt.py | 44 +- .../providers/openlineage/plugins/test_adapter.py | 129 ++++-- tests/providers/openlineage/utils/test_utils.py | 449 ++++++++++++++++++++- tests/providers/pagerduty/hooks/test_pagerduty.py | 31 +- .../pagerduty/hooks/test_pagerduty_events.py | 14 +- .../amazon/aws/example_glue_data_quality.py | 2 +- ...ample_glue_data_quality_with_recommendation.py} | 57 ++- tests/www/test_utils.py | 4 +- tests/www/views/test_views_cluster_activity.py | 6 +- tests/www/views/test_views_grid.py | 3 +- tests/www/views/test_views_tasks.py | 2 +- 78 files changed, 2542 insertions(+), 427 deletions(-) rename airflow/migrations/versions/{0143_2_9_2_add_indexes_on_dag_id_column_in_referencing_tables.py => 0143_2_10_0_add_indexes_on_dag_id_column_in_referencing_tables.py} (99%) create mode 100644 airflow/providers/openlineage/facets/AirflowJobFacet.json create mode 100644 airflow/providers/openlineage/facets/AirflowRunFacet.json copy airflow/providers/{google/cloud/openlineage/BigQueryJobRunFacet.json => openlineage/facets/AirflowStateRunFacet.json} (51%) copy airflow/{api_connexion => providers/openlineage/facets}/__init__.py (100%) copy tests/system/providers/amazon/aws/{example_glue_data_quality.py => example_glue_data_quality_with_recommendation.py} (81%)