This is an automated email from the ASF dual-hosted git repository.
kaxil pushed a change to branch aip99-crewai
in repository https://gitbox.apache.org/repos/asf/airflow.git
omit fdd8e50b1cd Override crewai dep pins to unblock workspace resolution
omit 4e4de2b8f41 Use dedicated crewai conn_type and reject
non-OpenAI-compatible providers
omit 361f79a347c Add CrewAI stock analysis example DAG using CrewAIHook
omit 9129ef3e50c Add CrewAI framework hook to common.ai provider
add a1784e880f8 Replace fixed sleep with active IAM trust policy
validation in example_emr_eks (#66736)
add 2017b407666 AGENTS.md: require translating domain-layer exceptions to
`HTTPException` at FastAPI route boundaries (#67221)
add 085459e6f96 AIP-76: Add PartitionAtRuntime authoring API to Task SDK
(#65447)
add 5aeb726b1f9 Sync RuntimeTaskInstanceProtocol with RuntimeTaskInstance
(#67216)
add ec2d56a473d AIP-103: Worker side custom state backend support (#66859)
add f9e4e6efd10 AIP-103: Add Core API endpoints for task state and asset
state (#67041)
add 41ec7a09398 Improve query validation, including for streaming (#67212)
add b2d4d63c64b Bump eslint (via audit fix) (#67220)
add 5ae6bb07cd5 Update French (fr) UI translations to 100% coverage
(#67240)
add 58a60ed2f48 tests: Add new options to test operator (#67195)
add 153623856ef Prevent AlreadyRunningBackfill error caused by invalid
date range request (#66874)
add fe1618f9f32 Fix LangChain hook tests failing when langchain is not
installed (#67237)
add 678d1dd0ce9 Replace `allow_producer_teams` with `access_control` on
Asset (#66954)
add 2970152b190 Fix CallbackKey type for more accurate type checking
(#66973)
add 13e2125a2c3 Add rerun_with_latest_version config hierarchy for
clear/rerun behavior (#63884)
add 3047ad7fa3c Fix autorefresh for deadlines and paused dags (#67249)
add aa3ae537a0d Fix console errors in gantt view (#67255)
add 6b64a621be4 [main] CI: Upgrade important CI environment (#67232)
add eec2f75e556 Add `DocumentLoaderOperator` to `common.ai` provider
(#67120)
add c0bc3d07747 Add chart/v*-stable to protected branches (#67259)
add cb2aaf41ed2 docs(edge3): clarify WorkerQueuesBase.team_name is an
experimental hint, cross-ref workload.rst (#66718)
add 315d1591644 Fix deadlock in ti_update_state caused by FOR UPDATE
locking dag_run (#67246)
add 8f216be1f1a Bump picomatch from 4.0.3 to 4.0.4 in /registry (#67124)
add 8d1ef7d628e Bump brace-expansion from 5.0.5 to 5.0.6 in /registry
(#67125)
add 22c450fbc35 Bump liquidjs from 10.25.5 to 10.27.0 in /registry (#67126)
add 7936ed6ce61 Bump ws from 8.19.0 to 8.20.1 in /registry (#67136)
add b64c302b2e9 Add LlamaIndex operators to common.ai provider (#67121)
add 6436d0ade09 Add CrewAI framework hook to common.ai provider
add f4090420f14 Add CrewAI stock analysis example DAG using CrewAIHook
add 7b4cfdf8f56 Use dedicated crewai conn_type and reject
non-OpenAI-compatible providers
add 1b1b2e398b1 Override crewai dep pins to unblock workspace resolution
add 80ae4fae677 Refresh uv.lock for rebase + crewai-3.11-only gating
This update added new revisions after undoing existing revisions.
That is to say, some revisions that were in the old version of the
branch are not in the new version. This situation occurs
when a user --force pushes a change and generates a repository
containing something like this:
* -- * -- B -- O -- O -- O (fdd8e50b1cd)
\
N -- N -- N refs/heads/aip99-crewai (80ae4fae677)
You should already have received notification emails for all of the O
revisions, and so the following emails describe only the N revisions
from the common base, B.
Any revisions marked "omit" are not gone; other references still
refer to them. Any revisions marked "discard" are gone forever.
No new revisions were added by this update.
Summary of changes:
.asf.yaml | 1 +
.pre-commit-config.yaml | 6 +-
AGENTS.md | 1 +
Dockerfile.ci | 2 +-
.../administration-and-deployment/dag-bundles.rst | 105 +
.../docs/authoring-and-scheduling/assets.rst | 2 +
airflow-core/docs/best-practices.rst | 2 +-
airflow-core/newsfragments/63884.significant.rst | 34 +
.../datamodels/asset_state.py} | 25 +-
.../api_fastapi/core_api/datamodels/backfills.py | 8 +-
.../api_fastapi/core_api/datamodels/dag_run.py | 9 +-
.../api_fastapi/core_api/datamodels/dags.py | 1 +
.../core_api/datamodels/task_instances.py | 9 +-
.../datamodels/task_state.py | 24 +-
.../api_fastapi/core_api/datamodels/ui/config.py | 1 +
.../api_fastapi/core_api/openapi/_private_ui.yaml | 5 +
.../core_api/openapi/v2-rest-api-generated.yaml | 776 ++-
.../api_fastapi/core_api/routes/public/__init__.py | 4 +
.../core_api/routes/public/asset_state.py | 163 +
.../core_api/routes/public/backfills.py | 13 +-
.../api_fastapi/core_api/routes/public/dag_run.py | 5 +-
.../core_api/routes/public/task_instances.py | 9 +-
.../core_api/routes/public/task_state.py | 210 +
.../api_fastapi/core_api/routes/ui/config.py | 7 +
.../api_fastapi/core_api/services/public/common.py | 26 +
.../execution_api/routes/task_instances.py | 4 +-
airflow-core/src/airflow/cli/cli_config.py | 8 +-
.../src/airflow/cli/commands/backfill_command.py | 13 +-
.../src/airflow/config_templates/config.yml | 24 +
.../src/airflow/dag_processing/collection.py | 4 +-
.../example_dags/example_asset_allow_teams.py | 22 +-
.../src/airflow/executors/base_executor.py | 12 +-
.../src/airflow/executors/workloads/callback.py | 6 +-
.../src/airflow/executors/workloads/types.py | 8 +-
.../src/airflow/jobs/scheduler_job_runner.py | 11 +-
airflow-core/src/airflow/models/backfill.py | 21 +-
airflow-core/src/airflow/models/callback.py | 12 +-
airflow-core/src/airflow/serialization/decoders.py | 2 +-
.../airflow/serialization/definitions/assets.py | 2 +-
.../src/airflow/serialization/definitions/dag.py | 2 +
airflow-core/src/airflow/serialization/encoders.py | 21 +-
airflow-core/src/airflow/serialization/schema.json | 3 +-
.../airflow/serialization/serialized_objects.py | 1 +
airflow-core/src/airflow/timetables/base.py | 7 +
airflow-core/src/airflow/timetables/simple.py | 16 +
.../src/airflow/ui/openapi-gen/queries/common.ts | 44 +-
.../ui/openapi-gen/queries/ensureQueryData.ts | 70 +-
.../src/airflow/ui/openapi-gen/queries/prefetch.ts | 70 +-
.../src/airflow/ui/openapi-gen/queries/queries.ts | 201 +-
.../src/airflow/ui/openapi-gen/queries/suspense.ts | 70 +-
.../airflow/ui/openapi-gen/requests/schemas.gen.ts | 183 +-
.../ui/openapi-gen/requests/services.gen.ts | 321 +-
.../airflow/ui/openapi-gen/requests/types.gen.ts | 410 +-
.../airflow/ui/public/i18n/locales/fr/admin.json | 21 +-
.../airflow/ui/public/i18n/locales/fr/assets.json | 8 +-
.../airflow/ui/public/i18n/locales/fr/browse.json | 26 +-
.../airflow/ui/public/i18n/locales/fr/common.json | 90 +-
.../ui/public/i18n/locales/fr/components.json | 34 +-
.../src/airflow/ui/public/i18n/locales/fr/dag.json | 92 +-
.../airflow/ui/public/i18n/locales/fr/dags.json | 13 +-
.../ui/public/i18n/locales/fr/dashboard.json | 3 +-
.../airflow/ui/public/i18n/locales/fr/hitl.json | 4 +
.../ui/src/components/Clear/Run/ClearRunDialog.tsx | 7 +-
.../TaskInstance/ClearGroupTaskInstanceDialog.tsx | 42 +-
.../Clear/TaskInstance/ClearTaskInstanceDialog.tsx | 39 +-
.../Clear/useRerunWithLatestVersion.test.tsx | 154 +
.../components/Clear/useRerunWithLatestVersion.ts | 55 +
.../src/components/DagActions/RunBackfillForm.tsx | 20 +-
.../ui/src/layouts/Details/DetailsLayout.tsx | 9 +-
.../ui/src/layouts/Details/Gantt/GanttTimeline.tsx | 2 +-
airflow-core/src/airflow/ui/src/pages/Dag/Dag.tsx | 2 +-
.../ui/src/pages/Dag/Overview/DagDeadlines.tsx | 8 +-
.../airflow/ui/src/queries/gridViewQueryKeys.ts | 10 +
.../ui/src/queries/useBulkClearTaskInstances.ts | 17 +-
.../airflow/ui/src/queries/useBulkTaskInstances.ts | 3 +
.../src/airflow/ui/src/queries/useClearRun.ts | 3 +-
.../ui/src/queries/useClearTaskInstances.ts | 3 +-
.../airflow/ui/src/queries/useCreateBackfill.ts | 1 +
.../src/airflow/ui/src/queries/useDeadlines.ts | 2 +-
.../src/airflow/ui/src/queries/useDeleteDagRun.ts | 9 +-
.../ui/src/queries/useDeleteTaskInstance.ts | 12 +-
.../airflow/ui/src/queries/useGridTISummaries.ts | 42 +
.../src/airflow/ui/src/queries/usePatchDagRun.ts | 9 +-
.../airflow/ui/src/queries/usePatchTaskGroup.ts | 3 +-
.../airflow/ui/src/queries/usePatchTaskInstance.ts | 3 +-
.../ui/src/queries/useRefreshOnNewDagRuns.ts | 8 +-
.../airflow/ui/src/queries/useUpdateHITLDetail.ts | 10 +-
.../core_api/routes/public/test_asset_state.py | 255 +
.../core_api/routes/public/test_dag_run.py | 81 +
.../core_api/routes/public/test_dags.py | 2 +
.../core_api/routes/public/test_task_state.py | 293 ++
.../api_fastapi/core_api/routes/ui/test_config.py | 1 +
.../unit/cli/commands/test_backfill_command.py | 3 +
.../tests/unit/dag_processing/test_collection.py | 14 +-
.../tests/unit/executors/test_base_executor.py | 25 +-
.../tests/unit/executors/test_local_executor.py | 2 +-
.../tests/unit/executors/test_workloads.py | 52 +
airflow-core/tests/unit/models/test_backfill.py | 18 +
.../unit/serialization/test_dag_serialization.py | 23 +
.../unit/serialization/test_serialized_objects.py | 55 +
.../src/airflowctl/api/datamodels/generated.py | 78 +-
dev/breeze/doc/ci/02_images.md | 2 +-
dev/breeze/pyproject.toml | 2 +-
.../commands/release_management_commands.py | 2 +-
dev/breeze/uv.lock | 68 +-
devel-common/pyproject.toml | 2 +-
.../tests/system/amazon/aws/example_emr_eks.py | 73 +-
providers/common/ai/docs/hooks/index.rst | 7 +
providers/common/ai/docs/hooks/llamaindex.rst | 115 +
.../common/ai/docs/operators/document_loader.rst | 297 ++
providers/common/ai/docs/operators/index.rst | 15 +-
.../ai/docs/operators/llamaindex_embedding.rst | 119 +
.../ai/docs/operators/llamaindex_retrieval.rst | 109 +
providers/common/ai/provider.yaml | 47 +
providers/common/ai/pyproject.toml | 21 +-
.../ai/example_dags/example_document_loader.py | 145 +
.../ai/example_dags/example_llamaindex_hook.py | 147 +
.../ai/example_dags/example_llamaindex_rag.py | 236 +
.../providers/common/ai/get_provider_info.py | 42 +
.../providers/common/ai/hooks/llamaindex.py | 189 +
.../common/ai/operators/document_loader.py | 401 ++
.../common/ai/operators/llamaindex_embedding.py | 210 +
.../common/ai/operators/llamaindex_retrieval.py | 199 +
.../tests/unit/common/ai/hooks/test_langchain.py | 23 +
.../tests/unit/common/ai/hooks/test_llamaindex.py | 170 +
.../common/ai/operators/test_document_loader.py | 596 +++
.../ai/operators/test_llamaindex_embedding.py | 211 +
.../ai/operators/test_llamaindex_retrieval.py | 238 +
providers/edge3/docs/deployment.rst | 9 +-
providers/edge3/docs/edge_executor.rst | 11 +-
.../src/airflow/providers/edge3/cli/definition.py | 8 +-
.../providers/edge3/worker_api/datamodels.py | 9 +-
.../edge3/worker_api/v2-edge-generated.yaml | 21 +-
.../fab/src/airflow/providers/fab/www/package.json | 2 +-
.../src/airflow/providers/fab/www/pnpm-lock.yaml | 83 +-
.../example_openlineage_base_complex_dag.py | 25 +-
.../example_openlineage_defer_simple_dag.py | 21 +-
.../example_openlineage_mapped_simple_dag.py | 25 +-
.../example_openlineage_short_circuit_dag.py | 29 +-
.../example_openlineage_trigger_dag_deferrable.py | 2 +-
.../openlineage_base_complex_dag__af2.json | 5156 ++++++++++----------
.../openlineage_base_complex_dag__af3.json | 5030 +++++++++----------
.../openlineage_short_circuit_dag.json | 23 +-
.../tests/system/openlineage/operator.py | 239 +-
registry/pnpm-lock.yaml | 34 +-
scripts/ci/prek/check_imports_in_providers.py | 2 +-
scripts/ci/prek/ruff_format.py | 2 +-
shared/state/src/airflow_shared/state/__init__.py | 71 +-
shared/state/tests/state/test_state.py | 83 +-
task-sdk/docs/api.rst | 6 +-
task-sdk/pyproject.toml | 3 +
task-sdk/src/airflow/sdk/__init__.py | 14 +-
task-sdk/src/airflow/sdk/__init__.pyi | 4 +
task-sdk/src/airflow/sdk/_shared/state | 1 +
task-sdk/src/airflow/sdk/api/client.py | 3 +-
task-sdk/src/airflow/sdk/bases/timetable.py | 8 +
task-sdk/src/airflow/sdk/configuration.py | 3 +-
.../src/airflow/sdk/definitions/asset/__init__.py | 26 +-
.../sdk/definitions/asset/access_control.py | 31 +-
.../airflow/sdk/definitions/asset/decorators.py | 8 +-
task-sdk/src/airflow/sdk/definitions/dag.py | 7 +
.../airflow/sdk/definitions/timetables/assets.py | 7 +
task-sdk/src/airflow/sdk/execution_time/context.py | 104 +-
.../src/airflow/sdk/execution_time/task_runner.py | 31 +-
.../src/airflow/sdk/state.py | 9 +-
task-sdk/src/airflow/sdk/types.py | 30 +-
task-sdk/tests/task_sdk/definitions/test_asset.py | 56 +-
.../definitions/test_asset_access_control.py | 66 +
.../task_sdk/definitions/test_asset_decorators.py | 68 +-
task-sdk/tests/task_sdk/definitions/test_dag.py | 15 +-
task-sdk/tests/task_sdk/docs/test_public_api.py | 1 +
.../tests/task_sdk/execution_time/test_context.py | 236 +-
.../task_sdk/execution_time/test_task_runner.py | 136 +-
uv.lock | 2240 +++++----
174 files changed, 15801 insertions(+), 6565 deletions(-)
create mode 100644 airflow-core/newsfragments/63884.significant.rst
copy
airflow-core/src/airflow/api_fastapi/{auth/managers/simple/datamodels/login.py
=> core_api/datamodels/asset_state.py} (65%)
copy airflow-core/src/airflow/api_fastapi/{execution_api =>
core_api}/datamodels/task_state.py (64%)
create mode 100644
airflow-core/src/airflow/api_fastapi/core_api/routes/public/asset_state.py
create mode 100644
airflow-core/src/airflow/api_fastapi/core_api/routes/public/task_state.py
create mode 100644
airflow-core/src/airflow/ui/src/components/Clear/useRerunWithLatestVersion.test.tsx
create mode 100644
airflow-core/src/airflow/ui/src/components/Clear/useRerunWithLatestVersion.ts
create mode 100644
airflow-core/tests/unit/api_fastapi/core_api/routes/public/test_asset_state.py
create mode 100644
airflow-core/tests/unit/api_fastapi/core_api/routes/public/test_task_state.py
create mode 100644 providers/common/ai/docs/hooks/llamaindex.rst
create mode 100644 providers/common/ai/docs/operators/document_loader.rst
create mode 100644 providers/common/ai/docs/operators/llamaindex_embedding.rst
create mode 100644 providers/common/ai/docs/operators/llamaindex_retrieval.rst
create mode 100644
providers/common/ai/src/airflow/providers/common/ai/example_dags/example_document_loader.py
create mode 100644
providers/common/ai/src/airflow/providers/common/ai/example_dags/example_llamaindex_hook.py
create mode 100644
providers/common/ai/src/airflow/providers/common/ai/example_dags/example_llamaindex_rag.py
create mode 100644
providers/common/ai/src/airflow/providers/common/ai/hooks/llamaindex.py
create mode 100644
providers/common/ai/src/airflow/providers/common/ai/operators/document_loader.py
create mode 100644
providers/common/ai/src/airflow/providers/common/ai/operators/llamaindex_embedding.py
create mode 100644
providers/common/ai/src/airflow/providers/common/ai/operators/llamaindex_retrieval.py
create mode 100644
providers/common/ai/tests/unit/common/ai/hooks/test_llamaindex.py
create mode 100644
providers/common/ai/tests/unit/common/ai/operators/test_document_loader.py
create mode 100644
providers/common/ai/tests/unit/common/ai/operators/test_llamaindex_embedding.py
create mode 100644
providers/common/ai/tests/unit/common/ai/operators/test_llamaindex_retrieval.py
create mode 120000 task-sdk/src/airflow/sdk/_shared/state
copy airflow-core/src/airflow/listeners/types.py =>
task-sdk/src/airflow/sdk/definitions/asset/access_control.py (60%)
copy airflow-core/src/airflow/api_fastapi/auth/managers/exceptions.py =>
task-sdk/src/airflow/sdk/state.py (85%)
create mode 100644
task-sdk/tests/task_sdk/definitions/test_asset_access_control.py
