This is an automated email from the ASF dual-hosted git repository.
potiuk pushed a change to branch
dependabot/npm_and_yarn/airflow-core/src/airflow/ui/eslint-5c0cd3bb85
in repository https://gitbox.apache.org/repos/asf/airflow.git
omit 0806fa8a932 Apply eslint-plugin-unicorn 76 prefer-ternary and
logical-operator fixes
omit 30eb01177e8 Bump the eslint group across 1 directory with 5 updates
add 6f5d5d06b17 Align standard provider tests with source paths (#73267)
add c2f600f6ec6 Add airflowctl tasks state command (#71206)
add 86b8a521837 Bind the AWS auth manager SAML response to the browser
that started the login (#73698)
add cefc0c3239d Bump the java-sdk-dependency-updates group across 1
directory with 3 updates (#74108)
add 2a274ab3c93 Refuse nested secret ids looked up with no team in
multi-team mode (#73789)
add 8055f6a8717 Skip creating an empty hook lineage collector for
OpenLineage events (#74190)
add 38fefc1ca95 Fix invalid cron expression in OpenLineage system test Dag
(#74187)
add 27f827fcdce Honor the polling interval for deferred Kubernetes jobs
(#74143)
add 3bd4df6b2da Fix Snowflake SQL API wait_for_query timing out on
finished queries (#73761)
add f74a685978c Use configured default queue for Celery executor callbacks
(#73552)
add ad3bb1f676c Replace coordinator artifact roots with
task_handler_bundle_name (#73970)
add 1c3f78e1d57 Validate a serialized Dag and fill its unset settings from
the Airflow config (#74041)
add a44fd6d1f31 Allow explicit runs to drain a paused Dag (#73763)
add 40832901c59 Improve LocalExecutor bookkeeping: correctly add tasks to
the `running` list (#74142)
add 5487c080bea Add Azure Blob Storage support to the DataFusion object
storage layer (#73374)
add b191cc4767b Fail instead of silently falling behind when a Lang-SDK
vendored Dag schema drifts from airflow-core (#74091)
add e0202f40c78 Let a Dag importer find the coordinator that parses its
files (#74042)
add 94a677e8f87 Keep callback triggers assigned to healthy triggerers
(#72848)
add 4e1f2aca67f Keep the Dag trigger button one-click when a run has
config (#74039)
add bfe84598a52 Fix destination path validation in GCS-to-SFTP transfers
(#74186)
add 8239997bf20 Scope UI next_run_assets endpoint to assets the caller may
read (#72862)
add 7df6f0e69cc Parse native Lang-SDK Dags in the Dag processor with their
runtime (#74035)
add cffc14d173b Parse native Lang-SDK Dags in a Dag bag (#74043)
add e08216d0081 Go SDK: delete the XComs of earlier tries before a task
runs (#74178)
add ae082ab651f Go SDK: add dag.TaskGroup with nesting and group-level
edges (#74214)
add fa65343163d Remove FAB-related exclusion in SQLA tests (#74180)
add dbee3c6c11b Fix concurrent SFTP directory transfers dropping parent
hook's connection overrides (#73647)
add b79acc8a8d6 Fix google provider BQ openlineage checks (#71777)
add d6ab1aad3c6 Register TypeScript, Java and Go grammars for the Dag Code
tab (#74231)
add 747f69b1a8c Warn when db clean skips only some rows of a batch (#73568)
add 9b2cc1352c3 Handle plain Step Functions execution errors (#73881)
add d339bb2767a Fix Dataproc triggers swallowing task cancellation (#74192)
new 5350862aea9 Bump the eslint group across 1 directory with 5 updates
new a9aa67f3dc4 Apply eslint-plugin-unicorn 76 prefer-ternary and
logical-operator fixes
This update added new revisions after undoing existing revisions.
That is to say, some revisions that were in the old version of the
branch are not in the new version. This situation occurs
when a user --force pushes a change and generates a repository
containing something like this:
* -- * -- B -- O -- O -- O (0806fa8a932)
\
N -- N -- N
refs/heads/dependabot/npm_and_yarn/airflow-core/src/airflow/ui/eslint-5c0cd3bb85
(a9aa67f3dc4)
You should already have received notification emails for all of the O
revisions, and so the following emails describe only the N revisions
from the common base, B.
Any revisions marked "omit" are not gone; other references still
refer to them. Any revisions marked "discard" are gone forever.
The 2 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails. The revisions
listed as "add" were already present in the repository and have only
been added to this reference.
Summary of changes:
.agents/skills/airflow-java-sdk/SKILL.md | 11 +-
.pre-commit-config.yaml | 42 +-
Dockerfile.ci | 7 +-
airflow-core/adr/lang-sdk/0004-dag-parsing.md | 12 +-
.../adr/lang-sdk/0010-native-dag-processing.md | 118 +-
.../0013-persisted-task-handler-bindings.md | 2 +-
.../authoring-and-scheduling/language-sdks/go.rst | 93 +-
.../language-sdks/index.rst | 6 +-
.../language-sdks/java.rst | 112 +-
.../language-sdks/typescript.rst | 85 +-
airflow-core/docs/core-concepts/backfill.rst | 7 +-
airflow-core/docs/core-concepts/dags.rst | 13 +
airflow-core/newsfragments/72407.significant.rst | 10 +-
airflow-core/newsfragments/73970.significant.rst | 5 +
.../api_fastapi/core_api/datamodels/backfills.py | 8 +
.../api_fastapi/core_api/datamodels/dag_run.py | 7 +
.../core_api/openapi/v2-rest-api-generated.yaml | 28 +
.../api_fastapi/core_api/routes/public/assets.py | 10 +
.../core_api/routes/public/backfills.py | 17 +-
.../api_fastapi/core_api/routes/public/dag_run.py | 11 +
.../api_fastapi/core_api/routes/ui/assets.py | 1 +
.../src/airflow/cli/commands/dag_command.py | 9 +-
.../src/airflow/cli/commands/task_command.py | 5 +-
.../src/airflow/config_templates/config.yml | 31 +-
airflow-core/src/airflow/dag_processing/dagbag.py | 157 ++-
.../src/airflow/dag_processing/importer_routing.py | 52 +
.../airflow/dag_processing/lang_sdk_processor.py | 579 +++++++++
airflow-core/src/airflow/dag_processing/manager.py | 48 +-
.../src/airflow/dag_processing/processor.py | 71 +-
.../src/airflow/executors/local_executor.py | 75 +-
airflow-core/src/airflow/models/backfill.py | 3 +
airflow-core/src/airflow/models/dag.py | 16 +
airflow-core/src/airflow/models/trigger.py | 1 +
.../src/airflow/serialization/definitions/dag.py | 2 +
airflow-core/src/airflow/serialization/schema.json | 13 +-
.../airflow/serialization/serialized_objects.py | 96 ++
.../airflow/ui/openapi-gen/requests/schemas.gen.ts | 18 +
.../airflow/ui/openapi-gen/requests/types.gen.ts | 12 +
.../ui/public/i18n/locales/ar/components.json | 4 +-
.../ui/public/i18n/locales/ca/components.json | 4 +-
.../ui/public/i18n/locales/de/components.json | 4 +-
.../ui/public/i18n/locales/el/components.json | 4 +-
.../ui/public/i18n/locales/en/components.json | 15 +-
.../ui/public/i18n/locales/es/components.json | 4 +-
.../ui/public/i18n/locales/fr/components.json | 4 +-
.../ui/public/i18n/locales/he/components.json | 4 +-
.../ui/public/i18n/locales/hi/components.json | 4 +-
.../ui/public/i18n/locales/hu/components.json | 4 +-
.../ui/public/i18n/locales/it/components.json | 4 +-
.../ui/public/i18n/locales/ja/components.json | 4 +-
.../ui/public/i18n/locales/ko/components.json | 4 +-
.../ui/public/i18n/locales/nl/components.json | 4 +-
.../ui/public/i18n/locales/pl/components.json | 4 +-
.../ui/public/i18n/locales/pt/components.json | 4 +-
.../ui/public/i18n/locales/ru/components.json | 4 +-
.../ui/public/i18n/locales/th/components.json | 4 +-
.../ui/public/i18n/locales/tr/components.json | 4 +-
.../ui/public/i18n/locales/zh-CN/components.json | 4 +-
.../ui/public/i18n/locales/zh-TW/components.json | 4 +-
.../components/DagActions/RunBackfillForm.test.tsx | 96 +-
.../src/components/DagActions/RunBackfillForm.tsx | 35 +-
.../src/components/MonacoEditor/configureMonaco.ts | 122 +-
.../TriggerDag/PausedDagOptions.test.tsx | 250 ++++
.../src/components/TriggerDag/PausedDagOptions.tsx | 123 ++
.../TriggerDag/TriggerDAGButton.test.tsx | 102 +-
.../src/components/TriggerDag/TriggerDAGButton.tsx | 171 +--
.../components/TriggerDag/TriggerDAGForm.test.tsx | 78 +-
.../src/components/TriggerDag/TriggerDAGForm.tsx | 32 +-
.../components/TriggerDag/TriggerDAGModal.test.tsx | 238 ++++
.../src/components/TriggerDag/TriggerDAGModal.tsx | 21 +-
.../airflow/ui/src/components/TriggerDag/types.ts | 3 +
.../ui/src/layouts/Details/DetailsLayout.tsx | 1 -
.../src/pages/Asset/CreateAssetEventModal.test.tsx | 161 ++-
.../ui/src/pages/Asset/CreateAssetEventModal.tsx | 20 +-
.../ui/src/pages/DagsList/DagCardActions.tsx | 1 -
.../src/airflow/ui/src/pages/DagsList/DagsList.tsx | 1 -
.../ui/src/queries/useCreateBackfill.test.tsx | 99 ++
.../airflow/ui/src/queries/useCreateBackfill.ts | 39 +-
.../src/airflow/ui/src/queries/useTrigger.test.tsx | 98 ++
.../src/airflow/ui/src/queries/useTrigger.ts | 31 +-
.../ui/src/system-components/Menu/MenuTrigger.tsx | 2 +-
airflow-core/src/airflow/ui/src/vite-env.d.ts | 23 +-
airflow-core/src/airflow/utils/cli.py | 48 +-
airflow-core/src/airflow/utils/db_cleanup.py | 30 +-
.../tests/unit/always/test_project_structure.py | 2 -
airflow-core/tests/unit/api_fastapi/conftest.py | 13 +
.../core_api/routes/public/test_assets.py | 48 +-
.../core_api/routes/public/test_backfills.py | 72 +-
.../core_api/routes/public/test_dag_run.py | 75 +-
.../api_fastapi/core_api/routes/ui/test_assets.py | 79 ++
.../tests/unit/cli/commands/test_dag_command.py | 3 +
.../tests/unit/cli/commands/test_task_command.py | 26 +
.../tests/unit/dag_processing/fake_lang_sdk.py | 144 +++
.../lang_sdk_fixtures/java_native.json | 151 +++
.../lang_sdk_fixtures/ts_conformance.json | 661 ++++++++++
.../tests/unit/dag_processing/test_dagbag.py | 304 +++++
.../unit/dag_processing/test_importer_routing.py | 63 +
.../unit/dag_processing/test_lang_sdk_processor.py | 839 +++++++++++++
.../tests/unit/dag_processing/test_manager.py | 241 +++-
.../tests/unit/dag_processing/test_processor.py | 69 +-
.../tests/unit/executors/test_local_executor.py | 455 ++++++-
.../executors/test_local_executor_check_workers.py | 5 +-
airflow-core/tests/unit/jobs/test_scheduler_job.py | 9 +-
airflow-core/tests/unit/models/test_backfill.py | 133 +-
airflow-core/tests/unit/models/test_dag.py | 40 +
airflow-core/tests/unit/models/test_dagrun.py | 6 +-
.../tests/unit/models/test_serialized_dag.py | 4 +-
airflow-core/tests/unit/models/test_trigger.py | 45 +
.../unit/serialization/test_dag_serialization.py | 215 ++++
.../unit/serialization/test_dag_version_diff.py | 1 -
airflow-core/tests/unit/utils/test_cli_util.py | 79 ++
airflow-core/tests/unit/utils/test_db_cleanup.py | 62 +-
.../airflowctl_tests/test_airflowctl_commands.py | 2 +
airflow-ctl/docs/images/command_hashes.txt | 2 +-
airflow-ctl/docs/images/output_tasks.svg | 78 +-
.../src/airflowctl/api/datamodels/generated.py | 21 +
airflow-ctl/src/airflowctl/api/operations.py | 7 +-
airflow-ctl/src/airflowctl/ctl/cli_config.py | 17 +
.../src/airflowctl/ctl/commands/task_command.py | 53 +-
.../tests/airflow_ctl/api/test_operations.py | 54 +-
.../airflow_ctl/ctl/commands/test_task_command.py | 275 ++++-
.../tests/airflow_e2e_tests/conftest.py | 56 +-
contributing-docs/30_new_language_sdk.rst | 8 +-
docker-tests/tests/docker_tests/test_prod_image.py | 3 -
docs/spelling_wordlist.txt | 1 +
generated/known_airflow_exceptions.txt | 2 +-
generated/known_sdk_imports_in_core.txt | 2 +
go-sdk/Justfile | 2 +-
go-sdk/README.md | 50 +-
go-sdk/adr/0008-native-dag-interface.md | 21 +-
go-sdk/airflow/bundle.go | 11 +-
go-sdk/airflow/dag.go | 179 ++-
go-sdk/airflow/if.go | 30 +-
go-sdk/airflow/if_test.go | 47 +-
go-sdk/airflow/inputs.go | 14 +-
go-sdk/airflow/inputs_test.go | 4 +-
go-sdk/airflow/node.go | 172 ++-
go-sdk/airflow/node_test.go | 11 +-
go-sdk/airflow/spec.gen.go | 31 +-
go-sdk/airflow/spec.go | 6 +-
go-sdk/airflow/task_group.go | 553 +++++++++
go-sdk/airflow/task_group_test.go | 1274 ++++++++++++++++++++
go-sdk/airflow/task_option.go | 10 +-
go-sdk/airflow/trigger_dag_run.go | 4 +-
go-sdk/example/bundle/Justfile | 5 +-
go-sdk/internal/bundle/task.go | 22 +-
go-sdk/internal/bundle/task_test.go | 35 +-
go-sdk/internal/genspec/authoring.go | 35 +-
go-sdk/internal/genspec/authoring_test.go | 22 +-
go-sdk/internal/genspec/main.go | 4 +-
go-sdk/internal/genspec/normalize.go | 14 +-
go-sdk/internal/genspec/normalize_test.go | 33 +-
go-sdk/pkg/execution/client.go | 32 +-
go-sdk/pkg/execution/client_test.go | 59 +
go-sdk/pkg/execution/integration_test.go | 326 ++++-
go-sdk/pkg/execution/task_runner.go | 64 +-
go-sdk/schema/dag-schema.json | 13 +-
java-sdk/README.md | 30 +-
java-sdk/buildSrc/build.gradle.kts | 2 +-
java-sdk/gradle/verification-metadata.xml | 410 +++----
.../apache/airflow/sdk/plugin/AirflowSdkPlugin.kt | 4 +-
java-sdk/scala_spark_example/README.md | 6 +-
java-sdk/sdk/build.gradle.kts | 19 +-
java-sdk/sdk/schema/dag-schema.json | 13 +-
kubernetes-tests/lang_sdk/README.md | 14 +-
kubernetes-tests/lang_sdk/config/values.yaml | 14 +-
.../lang_sdk/pod_templates/lang_sdk_golang.yaml | 3 +-
.../lang_sdk/pod_templates/lang_sdk_java.yaml | 3 +-
kubernetes-tests/lang_sdk/stage_artifacts.py | 8 +-
providers/akeyless/docs/changelog.rst | 7 +
.../airflow/providers/akeyless/secrets/akeyless.py | 28 +-
.../tests/unit/akeyless/secrets/test_akeyless.py | 27 +-
.../docs/auth-manager/setup/identity-center.rst | 36 +
providers/amazon/docs/operators/step_functions.rst | 11 +
providers/amazon/provider.yaml | 15 +
.../providers/amazon/aws/auth_manager/constants.py | 1 +
.../amazon/aws/auth_manager/routes/login.py | 200 ++-
.../amazon/aws/operators/step_function.py | 5 +-
.../airflow/providers/amazon/get_provider_info.py | 7 +
.../amazon/aws/auth_manager/routes/test_login.py | 359 +++++-
.../amazon/aws/operators/test_step_function.py | 51 +
.../providers/celery/executors/celery_executor.py | 4 +-
.../unit/celery/executors/test_celery_executor.py | 29 +-
.../providers/cncf/kubernetes/triggers/job.py | 4 +-
.../unit/cncf/kubernetes/triggers/test_job.py | 13 +-
.../ai/docs/operators/llm_schema_compare.rst | 2 +-
providers/common/ai/docs/operators/llm_sql.rst | 2 +-
providers/common/ai/docs/toolsets/datafusion.rst | 2 +-
providers/common/sql/docs/operators.rst | 60 +-
.../sql/src/airflow/providers/common/sql/config.py | 3 +
.../providers/common/sql/datafusion/engine.py | 185 +++
.../sql/datafusion/object_storage_provider.py | 36 +-
.../common/sql/example_dags/example_analytics.py | 13 +
.../unit/common/sql/datafusion/test_engine.py | 447 +++++++
.../sql/datafusion/test_object_storage_provider.py | 48 +
.../sql/tests/unit/common/sql/test_config.py | 1 +
.../google/cloud/transfers/gcs_to_sftp.py | 29 +-
.../providers/google/cloud/triggers/dataproc.py | 3 +
.../resources/openlineage/bigquery_dts.json | 18 +-
.../resources/openlineage/bigquery_operations.json | 8 +-
.../resources/openlineage/bigquery_tables.json | 12 +-
.../resources/openlineage/bigquery_transfer.json | 8 +-
.../google/cloud/transfers/test_gcs_to_sftp.py | 50 +-
.../unit/google/cloud/triggers/test_dataproc.py | 63 +-
providers/hashicorp/docs/changelog.rst | 7 +
.../airflow/providers/hashicorp/secrets/vault.py | 31 +-
.../tests/unit/hashicorp/secrets/test_vault.py | 55 +
.../providers/openlineage/extractors/manager.py | 31 +
...schedule_multiple_cron_trigger_timetable_dag.py | 2 +-
...hedule_multiple_cron_trigger_timetable_dag.json | 12 +-
.../unit/openlineage/extractors/test_manager.py | 71 +-
.../sftp/src/airflow/providers/sftp/hooks/sftp.py | 44 +-
providers/sftp/tests/unit/sftp/hooks/test_sftp.py | 92 ++
.../providers/snowflake/hooks/snowflake_sql_api.py | 12 +-
.../unit/snowflake/hooks/test_snowflake_sql_api.py | 35 +-
.../tests/unit/standard/operators/test_empty.py | 56 +
...ternal_task_sensor.py => test_external_task.py} | 34 +-
.../ci/lang_sdk_serialization/serialize_python.py | 23 +-
.../prek/check_contextmanager_class_decorators.py | 1 +
scripts/ci/prek/check_go_sdk_generated_drift.py | 5 +-
scripts/ci/prek/sync_go_sdk_schemas.py | 21 +-
scripts/docker/entrypoint_ci.sh | 7 +-
.../test_check_contextmanager_class_decorators.py | 80 ++
task-sdk/docs/executable-bundle-spec.rst | 31 +-
task-sdk/docs/ts-bundle-spec.rst | 26 +-
.../airflow/sdk/coordinators/_bundle_metadata.py | 45 +-
.../src/airflow/sdk/coordinators/_dag_importer.py | 162 +++
.../src/airflow/sdk/coordinators/_subprocess.py | 258 ++--
.../sdk/coordinators/executable/coordinator.py | 30 +-
.../airflow/sdk/coordinators/java/coordinator.py | 34 +-
.../airflow/sdk/coordinators/node/coordinator.py | 31 +-
.../src/airflow/sdk/execution_time/coordinator.py | 190 ++-
.../src/airflow/sdk/execution_time/task_runner.py | 38 +-
task-sdk/src/airflow/sdk/importers/base.py | 51 +-
.../coordinators/executable/test_coordinator.py | 73 +-
.../task_sdk/coordinators/java/test_coordinator.py | 96 +-
.../task_sdk/coordinators/node/test_coordinator.py | 45 +-
.../task_sdk/coordinators/test_dag_importer.py | 214 ++++
.../tests/task_sdk/coordinators/test_subprocess.py | 309 ++---
.../task_sdk/execution_time/test_coordinator.py | 412 ++++++-
.../task_sdk/execution_time/test_task_runner.py | 130 ++
task-sdk/tests/task_sdk/importers/test_registry.py | 138 +++
ts-sdk/README.md | 23 +-
ts-sdk/example/README.md | 14 +-
ts-sdk/schema/dag-schema.json | 13 +-
245 files changed, 14462 insertions(+), 2054 deletions(-)
create mode 100644 airflow-core/newsfragments/73970.significant.rst
create mode 100644 airflow-core/src/airflow/dag_processing/importer_routing.py
create mode 100644
airflow-core/src/airflow/dag_processing/lang_sdk_processor.py
create mode 100644
airflow-core/src/airflow/ui/src/components/TriggerDag/PausedDagOptions.test.tsx
create mode 100644
airflow-core/src/airflow/ui/src/components/TriggerDag/PausedDagOptions.tsx
create mode 100644
airflow-core/src/airflow/ui/src/components/TriggerDag/TriggerDAGModal.test.tsx
create mode 100644
airflow-core/src/airflow/ui/src/queries/useCreateBackfill.test.tsx
create mode 100644 airflow-core/src/airflow/ui/src/queries/useTrigger.test.tsx
create mode 100644 airflow-core/tests/unit/dag_processing/fake_lang_sdk.py
create mode 100644
airflow-core/tests/unit/dag_processing/lang_sdk_fixtures/java_native.json
create mode 100644
airflow-core/tests/unit/dag_processing/lang_sdk_fixtures/ts_conformance.json
create mode 100644
airflow-core/tests/unit/dag_processing/test_importer_routing.py
create mode 100644
airflow-core/tests/unit/dag_processing/test_lang_sdk_processor.py
create mode 100644 go-sdk/airflow/task_group.go
create mode 100644 go-sdk/airflow/task_group_test.go
create mode 100644
providers/standard/tests/unit/standard/operators/test_empty.py
rename
providers/standard/tests/unit/standard/sensors/{test_external_task_sensor.py =>
test_external_task.py} (98%)
create mode 100644
scripts/tests/ci/prek/test_check_contextmanager_class_decorators.py
create mode 100644 task-sdk/src/airflow/sdk/coordinators/_dag_importer.py
create mode 100644 task-sdk/tests/task_sdk/coordinators/test_dag_importer.py
