This is an automated email from the ASF dual-hosted git repository.
jason810496 pushed a change to branch
jason/core-taskhandler-refactor/12-zip-dag-discovery
in repository https://gitbox.apache.org/repos/asf/airflow.git
discard e963a67a836 List only .zip archives as packaged Dags
discard 1c5b1cac056 Describe the per-candidate fast path in ADR-0013
discard 10c50f5358a Decide per candidate whether a parse must probe it
discard 171cd02efa5 Read a TS bundle's cache digest from its layout line
discard 36f18ee0a26 List Java task handler JARs by their cache digest
discard a2bff0a369e List Go task handler bundles by their trailer
discard 4d546cb4ad9 Let a coordinator list the artifacts it can probe
discard 391143ba0b4 Build the binding and probed-artifact test params without
required
discard ad74c381e46 Align the artifact sweep with the other cleanup checks
discard 7bd0af21458 Warn only about bindings of Dags missing from the parse
result
discard eaf07addc1b Give the task handler race tests their own sessions
discard 0ba8df02c88 Describe the binding reconcile and artifact sweep in
ADR-0013
discard 94cc7fe2a87 Delete task handler artifacts no stub task references
discard 3354fb55b9b Index task handler artifacts by last_probed_at
discard 1b034ca6ef7 Record probed artifacts and reconcile task handler bindings
discard f7c85274bef Regenerate Go SDK models for task handler bindings
discard 14c2da95f2d Sync the Java SDK supervisor schema for task handler
bindings
discard e3cdd04fa0c Regenerate TS SDK supervisor types for task handler
bindings
discard 3af40cf4ced Carry task handler bindings and probed answers on the
parse result
discard 4ca5e766202 Build the known-artifact test params without required
discard bbdb9458c12 Test that a Dag file gets only named bundles without a
fallback
discard 61f06988804 Describe how known artifacts are scoped per Dag-parsing
child
discard d85ebe5aea6 Send the Dag-parsing child its known task-handler artifacts
discard 88e84be0fac Regenerate Go SDK models for known artifacts
discard 54fb0b711c8 Sync the Java SDK supervisor schema for known artifacts
discard 851120f0e86 Regenerate TS SDK supervisor types for known artifacts
discard ff3f1544b3e Add known task-handler artifacts to the Dag parse request
discard ce6aa243869 Let the coordinator registry list its task-handler bundles
discard f508e20b312 Java SDK: declare task handler params without required
discard 27c63963bed Java SDK: answer a task handler parse with every handler
discard 12c6724a8b7 Test the cache digest of classpath directories and non-zip
files
discard 80244de40b8 Tidy names and mocks in the Java probe test
discard d4fbf19150b Test the Java SDK schema sync hook's snapshot copy
discard b39aebff2de Say why the Java coordinator rejects a JAR it cannot probe
discard 7f43db70cd9 Java SDK: check the task handler reply against the
supervisor schema
discard ed4010de99d Java SDK: list a value schema's enum constants in a stable
order
discard 9aeecfcb3d4 Let the Java coordinator probe a JAR's task handlers
discard 4136b8ece27 Java SDK: register stub Dags' Java tasks as task handlers
discard bd2f0d62c1c Java SDK: stamp a cache digest on the bundle JAR manifest
discard e2e35ff1c32 Java SDK: answer TaskHandlerParseRequest with registered
handlers
discard be4499328a4 Java SDK: record handler parameters in generated task
classes
discard 5bb20d9ef56 Java SDK: sync the vendored supervisor schema from the
monorepo
discard 0f344dc7d12 TS SDK: declare task handlers without listing their params
discard b522a37f35f TS SDK: answer a task handler parse with every handler
discard 9af9e9cf31f Patch the TS real probe test with decorators
discard a3a13d2888a Skip the TS real probe when node cannot report a version
discard 3733aa17b6b TS SDK: name the task handler lookup like its siblings
discard a1cc9491204 Let the Node coordinator probe a bundle's task handlers
discard 422eb411914 TS SDK: answer TaskHandlerParseRequest from bundle.serve
discard 0fb5c4c2fda Describe the two task handler bindings for Language SDK
authors
discard 9e1936b10ff Go SDK: declare every lone struct handler as named
discard ddb48ee3075 State that a task handler parse answer depends only on the
artifact
discard 8dc677e3331 Go SDK: answer a task handler parse with every handler
discard 405713a5cbb Tidy names, mocks and cases in the Go probe and digest
tests
discard 146ec8981c8 Move Go SDK comments back to the code they describe
discard 51248188736 Resolve the probed bundle path and tighten the digest docs
discard 3d796536894 Go SDK: take the bundle digests from the staged executable
discard 113c3818b44 Document the task handler parse for Language SDK authors
discard 641aa09e0d4 Read a Go bundle's stored cache digest without hashing it
discard cd5d45c1ef1 Let the Executable coordinator probe a bundle's task
handlers
discard 2acb75863aa Go SDK: record integrity and cache digests in bundle
metadata
discard cc8a102f52d Go SDK: clarify parse timeout and nested-null docs, test
anyOf
discard 51e0a222867 Go SDK: answer TaskHandlerParseRequest with registered
handlers
discard ad605d636bb Go SDK: describe task handler params with JSON Schema
discard 69e2b807dd8 Clean up a Lang-SDK runtime parse whose start fails after
the fork
discard b4025a25e30 Do not exec a Lang-SDK runtime whose parent has already
exited
discard d432d42e704 Kill what a Lang-SDK runtime leaves behind when it exits
discard 71ee14604ac Read a Lang-SDK runtime's frames without blocking the
parse loop
discard 2f07e848745 Probe a Lang-SDK artifact for its task handlers
discard d9f8c3aba41 Let a subprocess coordinator exec a task-handler parse
runtime
discard 80c989b2c05 Keep a nested supervised child's standard streams open
discard 017a4010644 Describe the two task handler binding modes in the ADRs
discard 37eb81323aa Regenerate Go SDK models for task-handler messages
discard 31154a6ffe9 Regenerate TS SDK supervisor types for the two binding
modes
discard 13853f06ade Bind task handlers only by position or by name
discard dd1174d4558 Describe the all-handlers probe in ADR-0011 and ADR-0012
discard 0fe6c7bd317 Regenerate TS SDK supervisor types for the probe without
dag_ids
discard c090d5f84b4 Ask a task handler runtime for every handler it registers
discard f202225ab80 Add the named_open binding to the Lang-SDK ADRs
discard e0d7a0a24bf Regenerate TS SDK supervisor types for the named_open
binding
discard b8de183cce0 Add a named_open binding for non-exhaustive task-handler
params
discard a60b181c39a Describe the binding-mode parse check in ADR-0011 and
ADR-0013
discard 65f47584d62 Describe the task-handler binding mode in ADR-0012
discard 6b256280a83 Regenerate TS SDK supervisor types for the binding mode
discard 259f707da38 Add a binding mode to task-handler declarations
discard ae2f0e35146 Document the version rules for a new supervisor schema body
discard 65e405f46e1 Regenerate TS SDK supervisor types for task-handler
messages
discard d148fa37175 Add task-handler parse messages and their parse-channel
union
discard c6ea332cc53 Test closing a Dag file processor that has no parse log
file
discard 438e073af11 Move the Dag file processor's shared plumbing into a base
class
discard 1db024e76b7 Allow a NULL cache digest in ADR-0013's artifact table SQL
discard b2a45718777 Let a task handler artifact store no cache digest
discard 65e8396ac6b Move the handler declarations to the artifact in
ADR-0013's SQL
discard 50274e47e05 Cache an artifact's task handlers on its row, not on each
binding
discard 6f8fa4d28a6 Make session keyword-only in the task handler model test
helpers
discard 08b3428d5fd Add handler_binding to ADR-0013's task handler table SQL
discard b8ea4dc87d9 Store the task handler binding mode beside its params
discard 0c9521838f0 Regenerate the task handler migration revision id
discard 78f7154fc41 Describe LangSDKTaskHandler as a stub task binding
discard 8b1b2554fd9 Test that a referenced task handler artifact cannot be
deleted
discard fbae76d020f Keep the artifact path hash in sync on ORM updates
discard 2db819349d4 Match ADR-0013 task handler table SQL to the migration
discard f538f1f2f56 Add lang-SDK task handler binding tables
add c7055408b06 Validate task_handler_bundle_name only for routed
coordinators
add 9d00ead095f Add newsfragment for the removed coordinator artifact roots
add 68f03d615b8 Mark ADR-0013 as accepted
add c174aa41637 Add lang-SDK task handler binding tables
add 6762729ecfc Match ADR-0013 task handler table SQL to the migration
add dc15082f135 Keep the artifact path hash in sync on ORM updates
add ab8f0257223 Test that a referenced task handler artifact cannot be
deleted
add 5da25f7206e Describe LangSDKTaskHandler as a stub task binding
add 396cac2600d Regenerate the task handler migration revision id
add 61b2d76b38e Store the task handler binding mode beside its params
add fce1160d9b8 Add handler_binding to ADR-0013's task handler table SQL
add d18d0afe556 Make session keyword-only in the task handler model test
helpers
add 7b572ab02d5 Cache an artifact's task handlers on its row, not on each
binding
add 03613039ea9 Move the handler declarations to the artifact in
ADR-0013's SQL
add 2c96b482aa6 Let a task handler artifact store no cache digest
add da3372e04cd Allow a NULL cache digest in ADR-0013's artifact table SQL
add db4f8470d08 Move the Dag file processor's shared plumbing into a base
class
add 7e26f0d11be Test closing a Dag file processor that has no parse log
file
add 4f4b3bdc559 Add task-handler parse messages and their parse-channel
union
add 7e0327ce1e5 Regenerate TS SDK supervisor types for task-handler
messages
add 2826ec45c7f Document the version rules for a new supervisor schema body
add a2b0c2936d4 Add a binding mode to task-handler declarations
add f3601d6e8a0 Regenerate TS SDK supervisor types for the binding mode
add 7160a104c3d Describe the task-handler binding mode in ADR-0012
add de96d42d536 Describe the binding-mode parse check in ADR-0011 and
ADR-0013
add 06386fb2291 Add a named_open binding for non-exhaustive task-handler
params
add 12b142d0ab9 Regenerate TS SDK supervisor types for the named_open
binding
add 270011518de Add the named_open binding to the Lang-SDK ADRs
add d33f9041f22 Ask a task handler runtime for every handler it registers
add d86cead24b2 Regenerate TS SDK supervisor types for the probe without
dag_ids
add c9eaf687a1e Describe the all-handlers probe in ADR-0011 and ADR-0012
add 1bd56509d09 Bind task handlers only by position or by name
add a4598c4102a Regenerate TS SDK supervisor types for the two binding
modes
add 1d793f9974d Regenerate Go SDK models for task-handler messages
add 07b5199cce6 Describe the two task handler binding modes in the ADRs
add 32db985ad1d Keep a nested supervised child's standard streams open
add 1ac623cb01a Let a subprocess coordinator exec a task-handler parse
runtime
add 4cb786b1d35 Probe a Lang-SDK artifact for its task handlers
add bf185776a42 Read a Lang-SDK runtime's frames without blocking the
parse loop
add 896cc6faf9b Kill what a Lang-SDK runtime leaves behind when it exits
add 1b9321728ac Do not exec a Lang-SDK runtime whose parent has already
exited
add b45c6ec8df6 Clean up a Lang-SDK runtime parse whose start fails after
the fork
add aeff5da3dc4 Go SDK: describe task handler params with JSON Schema
add 4619601b5de Go SDK: answer TaskHandlerParseRequest with registered
handlers
add fc06e0786b7 Go SDK: clarify parse timeout and nested-null docs, test
anyOf
add f3d803492c5 Go SDK: record integrity and cache digests in bundle
metadata
add c440bf74d68 Let the Executable coordinator probe a bundle's task
handlers
add eb196107f46 Read a Go bundle's stored cache digest without hashing it
add 6cf833f9f89 Document the task handler parse for Language SDK authors
add 9ecf47d459c Go SDK: take the bundle digests from the staged executable
add 262ca91a98a Resolve the probed bundle path and tighten the digest docs
add f731a54bab2 Move Go SDK comments back to the code they describe
add a9d24bac054 Tidy names, mocks and cases in the Go probe and digest
tests
add f479c8aef71 Go SDK: answer a task handler parse with every handler
add 5155067e2c7 State that a task handler parse answer depends only on the
artifact
add 3d9eaf255fe Go SDK: declare every lone struct handler as named
add ccecc8115d5 Describe the two task handler bindings for Language SDK
authors
add d210b17c293 TS SDK: answer TaskHandlerParseRequest from bundle.serve
add 8ea0048ce68 Let the Node coordinator probe a bundle's task handlers
add e12ad5c2bb2 TS SDK: name the task handler lookup like its siblings
add 57d85ff4515 Skip the TS real probe when node cannot report a version
add 6d0064278c2 Patch the TS real probe test with decorators
add f8acbd27def TS SDK: answer a task handler parse with every handler
add 6c3e3e36f83 TS SDK: declare task handlers without listing their params
add 0d37b8643a6 Java SDK: sync the vendored supervisor schema from the
monorepo
add d2539597b1b Java SDK: record handler parameters in generated task
classes
add 078264dd994 Java SDK: answer TaskHandlerParseRequest with registered
handlers
add 8d360b8b17c Java SDK: stamp a cache digest on the bundle JAR manifest
add e4adc58a92a Java SDK: register stub Dags' Java tasks as task handlers
add f72d37f25bf Let the Java coordinator probe a JAR's task handlers
add 11fb9964dd1 Java SDK: list a value schema's enum constants in a stable
order
add e4c081cb718 Java SDK: check the task handler reply against the
supervisor schema
add 238799ed5d8 Say why the Java coordinator rejects a JAR it cannot probe
add 5826da213a6 Test the Java SDK schema sync hook's snapshot copy
add 8f942ec422d Tidy names and mocks in the Java probe test
add 18958abe00f Test the cache digest of classpath directories and non-zip
files
add 628e134066d Java SDK: answer a task handler parse with every handler
add f599ee69604 Java SDK: declare task handler params without required
add dc039bc5c08 Let the coordinator registry list its task-handler bundles
add 0b543b3ebec Add known task-handler artifacts to the Dag parse request
add e2d6f912ceb Regenerate TS SDK supervisor types for known artifacts
add 6960f9ba158 Sync the Java SDK supervisor schema for known artifacts
add 75859356cf8 Regenerate Go SDK models for known artifacts
add a0b128ed0eb Send the Dag-parsing child its known task-handler artifacts
add 78ff94a5f51 Describe how known artifacts are scoped per Dag-parsing
child
add 9f11c5403b8 Test that a Dag file gets only named bundles without a
fallback
add 7e11dfa0caa Build the known-artifact test params without required
add f4e9bd911c2 Carry task handler bindings and probed answers on the
parse result
add 52cd2febb58 Regenerate TS SDK supervisor types for task handler
bindings
add 7ee2d09a4f7 Sync the Java SDK supervisor schema for task handler
bindings
add c61b6806908 Regenerate Go SDK models for task handler bindings
add 3ea58f3d0ac Record probed artifacts and reconcile task handler bindings
add 29001278e49 Index task handler artifacts by last_probed_at
add 58f696472ee Delete task handler artifacts no stub task references
add 5b2965d7ec9 Describe the binding reconcile and artifact sweep in
ADR-0013
add 93cd21b0dac Give the task handler race tests their own sessions
add 8bc7d3cf996 Warn only about bindings of Dags missing from the parse
result
add f1e12096a2c Align the artifact sweep with the other cleanup checks
add dc3fd6621a6 Build the binding and probed-artifact test params without
required
add 3737950c01b Let a coordinator list the artifacts it can probe
add 337ed169e73 List Go task handler bundles by their trailer
add 6968215796a List Java task handler JARs by their cache digest
add f972a8e66fc Read a TS bundle's cache digest from its layout line
add 49319b32f28 Decide per candidate whether a parse must probe it
add 19e73b5710c Describe the per-candidate fast path in ADR-0013
add 745d796daa2 List only .zip archives as packaged Dags
This update added new revisions after undoing existing revisions.
That is to say, some revisions that were in the old version of the
branch are not in the new version. This situation occurs
when a user --force pushes a change and generates a repository
containing something like this:
* -- * -- B -- O -- O -- O (e963a67a836)
\
N -- N -- N
refs/heads/jason/core-taskhandler-refactor/12-zip-dag-discovery (745d796daa2)
You should already have received notification emails for all of the O
revisions, and so the following emails describe only the N revisions
from the common base, B.
Any revisions marked "omit" are not gone; other references still
refer to them. Any revisions marked "discard" are gone forever.
No new revisions were added by this update.
Summary of changes:
.../0013-persisted-task-handler-bindings.md | 2 +-
airflow-core/newsfragments/73970.significant.rst | 5 ++
.../src/airflow/sdk/execution_time/coordinator.py | 17 ++++---
.../task_sdk/execution_time/test_coordinator.py | 56 +++++++++++++++++++++-
4 files changed, 69 insertions(+), 11 deletions(-)
create mode 100644 airflow-core/newsfragments/73970.significant.rst
