vatsrahul1001 commented on code in PR #67233: URL: https://github.com/apache/airflow/pull/67233#discussion_r3273570047
########## RELEASE_NOTES.rst: ########## @@ -24,6 +24,225 @@ .. towncrier release notes start +Airflow 3.2.2 (2026-05-26) +-------------------------- + +Significant Changes +^^^^^^^^^^^^^^^^^^^ + +- OTel timer and timing metrics now use Histogram instead of Gauge, preserving count, sum, and bucket distribution across recordings. (#64207) +- The SMTP STARTTLS upgrade performed by ``airflow.utils.email.send_email`` now validates the SMTP server's certificate against the system's trusted CA bundle by default. Previously the ``starttls()`` call was made without an SSL context, so any certificate was accepted. + Deployments that intentionally point Airflow at an SMTP server with a self-signed or otherwise non-validating certificate and need to preserve the previous behaviour must set ``email.ssl_context = "none"`` in ``airflow.cfg``. The ``"default"`` value (now also the default when the option is unset) uses :func:`ssl.create_default_context`. Previously this option applied only to the ``SMTP_SSL`` path; it now applies to the STARTTLS path as well. (#65346) +- In #64963, the Airflow UI switched from full-match ``*_pattern`` REST API query parameters to the new index-friendly ``*_prefix_pattern`` parameters on list endpoints. This is a behavioral change for search-as-you-type filters in the UI: matches are prefix-based (``LIKE 'term%'`` via a range scan) instead of substring-based (``ILIKE '%term%'``), which means the database can use B-tree indexes and search stays fast on large deployments. The REST API itself keeps both forms: existing ``*_pattern`` parameters still behave exactly as before. + In #66015, a per-search-bar "Match anywhere" toggle was added so users who relied on the previous substring behavior can opt back into it from the UI. Each search input and each text filter pill now has a small regex-icon toggle next to the value; flipping it on switches that input from ``*_prefix_pattern`` to ``*_pattern``. (#66015) +- Fix triggerer race condition and deadlock that caused deferred tasks to stall indefinitely + + Triggers that call synchronous SDK methods (e.g. ``get_task_states`` used by + ``safe_to_cancel`` in several Google provider operators) could crash the triggerer's + internal subprocess. The triggerer would then continue to heartbeat normally — + appearing healthy to the scheduler — while silently processing zero triggers, causing + every deferred task to time out. This was first reported in issue #64620; a + partial fix shipped in Airflow 3.2.1 (#64882) but introduced a new deadlock + with the same visible symptom under load. + + Both issues are fixed by replacing the lock-based serialization with response + multiplexing: each request now carries a unique ID and the response is routed back to + the correct caller, so concurrent requests from trigger threads no longer contend or + deadlock regardless of how many triggers are running or what SDK methods they call. + + **New: triggerer subprocess watchdog** + + Even with the race fixed, a trigger that blocks the event loop (e.g. by calling + ``time.sleep()`` or performing blocking I/O directly in ``async def run()``) would + previously leave the triggerer appearing healthy indefinitely. + + A new ``[triggerer] runner_health_check_threshold`` config option (default: 30 seconds) + adds a watchdog: if the triggerer subprocess goes silent for longer than the threshold, + the parent process stops updating the heartbeat so the scheduler can detect the hang and + reassign triggers rather than waiting for them to individually time out. Set the option + to ``0`` to disable the watchdog. (#66412) + + +- Tighten ``[core] allowed_deserialization_classes_regexp`` to require full-string matches + + Patterns in ``[core] allowed_deserialization_classes_regexp`` are now matched + against the entire classname using ``re.fullmatch()`` instead of ``re.match()``. + Previously a pattern such as ``airflow\.models\.Variable`` admitted not only + the intended class but also names that started with it + (e.g. ``airflow.models.Variable_Malicious``), because ``re.match`` only anchors + at the start of the string. + + The default value of this option is empty, so out-of-the-box deployments are + unaffected. Deployments that configured this option with patterns relying on + prefix-match semantics — for example ``airflow\.models\.`` to mean "any class + under ``airflow.models``" — must add ``.*`` to the pattern + (``airflow\.models\..*``) to retain the previous behaviour. (#66499) + +- Custom deadline reference classes must now be registered via the new ``deadline_references`` attribute on ``AirflowPlugin``, matching the existing pattern for custom timetables and custom partition mappers. To use a custom ``DeadlineReference`` subclass, register it in a plugin's ``deadline_references`` list. Custom references that are not registered will raise ``DeadlineReferenceNotRegistered`` at deserialization. (#66737) + +Bug Fixes +^^^^^^^^^ + +- Fail closed when supervisor IPC fails on a non-success terminal state (#66573) (#67183) +- Refuse secrets-backend fallback on Execution-API authorization deny (#66575) (#67173) +- Harden ``_collect_teams_to_check`` and ``requires_access_backfill`` against malformed request bodies (#66504) (#67182) +- Don't crash supervisor IPC loop on transient network errors (#66572) (#67177) +- Default-deny auth at the API and UI router level (#66505) (#67171) +- Apply per-Dag audit log permission to event log detail endpoint (#67112) (#67159) +- Fix ``ValueError`` when supervisor force-closes stuck sockets after timeout (#67115) (#67162) +- Redact rendered template fields while still structured to preserve nested-key masking on truncation (#65906) (#67117) +- Fix migration 0080 to migrate existing deadline rows on upgrade and downgrade (#66016) (#67129) +- Fix ``XCom`` PATCH/POST to store native values instead of ``json.dumps`` output (#64220) (#67116) +- Fix ``max_active_runs`` lost during Dag serialization when value equals schema default (#65310) (#67097) +- Fix N+1 query pattern in bulk pool delete endpoint (#66222) (#67108) +- Optimize DB performance of datetime range filters in API queries (#66696) (#67102) +- Fix ``serialize_template_field`` handling callable value in dict (#63871) (#67092) +- Fix scheduler to ignore stale executor success after defer reschedule (#66431) (#67089) +- Fix ``ArgNotSet`` ``repr`` to use stable string instead of memory address (#65222) (#66897) +- Fix scheduler MySQL task instance index hint (#66785) (#67087) +- UI: Preserve Grid limit and filters when redirecting after manual Dag trigger (#66717) (#66867) +- Apply reserved-key check to ``XCom`` update payload (#65915) (#66913) +- Fix log server path extraction to use ``removeprefix`` (#66749) (#66772) +- Fix macOS ``SIGSEGV`` in task execution by using ``fork`` + ``exec`` (#64874) (#66872) +- Fix Dag auto-pause ordering to use ``run_after`` (#65207) (#66863) +- Fix Dag version inflation caused by unmatched serialized result of task using re-serialized command (#61077) (#66861) +- Fix ``pod_override`` serialization in Dag details and executor path (#65407) (#66898) +- Fix async engine missing ``pool_recycle`` and ``pool_pre_ping`` configuration (#65276) (#66866) +- UI: Make Dag detail page ``scrollable`` on mobile ``viewports`` (#65899) (#66975) +- Fix ``DagVersion`` when clearing tasks with run on latest version (#65835) (#66901) +- Fix millisecond floating point duration bug (#66560) (#66915) +- UI: Fix "Mark state as..." buttons grayed out when task or ``DagRun`` already in target state (#66198) (#66919) +- Fix memory leak in ``LocalExecutor`` caused by unreleased file descriptor locks (#65121) (#66887) +- Fix external DB manager upgrades with existing tables (#66674) (#66882) +- UI: Improve ``DagCalendarTab`` background color retrieval and loading overlay handling (#64189) (#66860) +- UI: Handle Dags state filter overflow on mobile (#66812) (#66847) +- UI: Fix Edit Connection dialog missing ``lazyMount`` causing JSON editor infinite loading (#65969) (#66828) +- UI: Fix ``ConnectionForm`` crashing when connection has invalid extra JSON (#66593) (#66831) +- Handle ``PermissionError`` in ``init_log_folder`` for mounted filesystems (#63878) (#66733) +- Fix scheduler crash by catching ``StaleDataError`` in ``verify_integrity`` (#64503) (#66727) +- Fix triggerer file handle leak when remote log upload fails (#66675) (#66684) +- Fix ``/tmp`` file leak when API server streams large task logs (#66450) (#66667) +- Fix ``XCom`` prior-dates lookup for duplicate ``run_id`` across Dags (#65227) (#66646) +- Fix HITL (Human-In-The-Loop) ``/required_actions`` listing to show mapped task instances (#66433) (#66482) +- Fix scheduler callback ``bundle_version`` when versioning disabled (#66485) (#66518) +- UI: Hide ``Next Run`` timestamp for paused Dags (#66552) (#66568) +- Fix task run context crash when ``DagRun`` state is expired (#66339) (#66347) +- Fix incorrect type warning from OTel spans (#66559) (#66567) +- Fix backfill to populate ``partition_date`` on partitioned backfill runs (#65998) (#66409) +- Fix ``remote_task_handler_kwargs`` passing handler params to ``RemoteLogIO`` (#65957) (#66440) +- Fix i18n translation files served stale after Airflow upgrade due to browser cache (#65720) (#66422) +- UI: Fix manual copy from Rendered Templates tab adding extra blank lines (#66221) (#66366) +- Fix slow and incomplete trigger cleanup in scheduler (#66210) (#66381) +- UI: Distinguish ``upstream_failed`` from ``failed`` in normal vision (#66324) (#66365) +- UI: Fix ``SearchBar`` input rewind (#66284) (#66359) +- Don't re-emit ``logical_date`` when previous ``data_interval`` is zero-length (#66132) (#66263) +- Fix variable access in triggerer for deferrable operators (#63387) (#66239) +- Fix missing ``autoincrement`` sequence on ``callback_request`` downgrade (#65230) (#66189) +- Bring back pre/post execute log groups for task logs (#66037) (#66049) +- Preload ``source_aliases`` in ``process_executor_events`` (#65422) (#66191) +- Use ``START_DATE_GTE`` for correct filtering according to ``dagRuns`` API (#66045) (#66098) +- Fix selection of unassigned asset triggers from DB (#65792) (#66043) +- UI: Fix ``retrigger`` form config sync (#65749) (#66044) +- Fix scheduler ``UniqueViolation`` crash on downgrade from 3.2.0 to 3.1.x (#65688) (#66003) +- Run ``finalize`` when supervisor call fails (#65714) (#65946) +- Follow-up fixes for double-encode trigger PR (#64823) (#65584) +- UI: Fix log fetch crash when ``ti.hostname`` is empty (#64285) (#65583) +- Fix backfill marked complete before Dag runs are created (#62561) (#65889) +- UI: Fix date time input ``year`` field unmodifiable (#63885) (#65890) +- UI: Fix pools slot input behavior (#63900) (#65891) +- Fix ``TypeError`` crashes on ``/users/list`` and ``/roles/list`` in FAB UI caused by concurrent API schema requests (#63986) (#65892) +- UI: Fix toaster behavior (#64142) (#65893) +- Fix FAB DB manager discovery in migration-only contexts (#64145) (#65894) +- UI: Fix ``PoolBar`` links using wrong query params for task instances filtering (#64182) (#65896) +- Fix memory growth from ``pathlib`` ``sys.intern`` in long-running processes (#65706) (#65855) +- Pre-assign ``external_executor_id`` at queuing time to prevent duplicate execution on scheduler crash (#65594) (#65711) +- Handle supervisor remote log upload failures gracefully (#65308) (#65318) +- Fix ``ti.start_date`` showing deferral-resume time instead of original start time (#63247) (#65491) +- Fix task CLI ``map_index`` bounds validation (#64133) (#65479) +- UI: Fix mapped task ``XCom`` navigation from Grid (#65192) (#65322) +- Fix connection schema field not saved for providers without field behaviour (#65263) (#65267) +- Fix bulk task instance update for mapped TIs and auth error rendering (#65874) +- Fix bulk task instance RBAC bypass (#64288) (#65846) +- Update ``is_url_safe`` to reject URLs with ``///`` (#65557) (#65737) +- UI: Improve Graph view performance (#65031) (#65537) +- Fix backfill params not overriding existing Dag run conf (#64939) (#65599) +- Fix ``run_id_pattern`` pipe OR operator dropping single-term edge cases (#65190) (#65565) +- Filter external dependency nodes by readable Dags in ``structure_data`` endpoint (#65342) (#65534) +- Respect Dag processor config option to show parsing logs on stdout (#65528) (#65541) +- Add per-Dag authorization to ``partitioned_dag_runs`` endpoints (#65344) (#65538) +- UI: Register trigger and sensor graph node types (#65167) (#65321) +- Ensure DB migrations run in a single connection (#65231) (#65368) +- Fix ``PATCH /dags`` pagination bug and document wildcard ``dag_id_pattern`` (#65309) +- Set JWT refresh cookie ``Secure`` flag when request is HTTPS (#65348) (#65363) +- Refuse to follow log symlinks that resolve outside the base log folder (#65325) (#65345) +- Enforce per-file import-error authorization using ``relative_fileloc`` and bundle (#65329) (#65343) +- UI: Invalidate task instances list query after clearing task instance (#63923) (#65304) +- Fix: Recover stuck TIs when direct terminal-state API call fails (#66574) (#67204) + +Miscellaneous +^^^^^^^^^^^^^ + +- UI: Use local Monaco editor module instead of CDN (#66647) (#67199) +- Use a distinct redact message for import errors with no registered Dag (#66923) (#67176) +- Surface remote-log upload failures via structured warnings (#66571) (#67172) +- UI: Filter task instances by rendered map index (#66008) (#67163) +- Move Task Identity line into Pre Execution block in logs (#67036) (#67134) +- Apply ``requires_access_event_log`` to ``GET /eventLogs`` list endpoint (#67185) (#67211) +- UI: Preserve proxy URL on login redirect (#66690) (#67091) +- Keep ``Named*Logger.name`` working across structlog releases (#66875) (#67088) +- Two-token mechanism for task execution to prevent token expiration while tasks wait in executor queues (#60108) (#66989) +- Validate task identity token claims with a typed schema (#63604) (#66988) +- Mark Dags stale when their bundle is removed from config (#66948) (#66985) +- UI: Allow pasting full datetime strings into date picker inputs (#66251) (#66958) +- Validate Dag run conf in backfill dry-run (#66196) (#66935) +- Improve post-task logs to show exception in failure (#66735) (#66920) +- UI: Show Dag run duration in grid tooltip (#65787) (#66900) +- UI: Add Dag run ID to grid bar tooltip and task instance tooltip (#65626) (#66871) +- UI: Change queued Dag runs color to gray in Calendar (#66623) (#66870) +- Add configurable LRU+TTL caching for API server Dag retrieval (#60804) (#66862) +- UI: Use link styling for Dag tags (#66750) (#66855) +- UI: Add hover feedback to ``Checkbox`` (#66714) (#66826) +- Check sensitive key names before applying recursion-depth cutoff in secrets masker (#65912) (#66748) +- Adjust log message header for expandable sources (#66570) (#66653) +- Allow triggerer to support memray memory profiling (#65994) (#66643) +- Show task ID attributes (``ti_id``, ``task_id``, etc.) once, not on every log line (#66036) (#66421) +- Propagate triggering user to child Dag runs via ``TriggerDagRunOperator`` (#65747) (#66378) +- UI: Add ``isExpanded`` prop on JSON expand/collapse buttons (#66340) (#66364) +- Pass ``try_number`` to extra links API (#65661) (#66171) +- UI: Serve grid TI summaries from shared cached ``DagBag`` (#65775) (#65966) +- Add cursor-based pagination for ``get_dag_runs`` endpoint (#65604) (#65746) Review Comment: I do not any of above mentioned PR making ay changes to execution api -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
