This is an automated email from the ASF dual-hosted git repository.
jason810496 pushed a change to branch
jason/core-taskhandler-refactor/08-java-sdk-task-handler-parse
in repository https://gitbox.apache.org/repos/asf/airflow.git
omit 782aeda48e7 Say why the Java coordinator rejects a JAR it cannot probe
omit bd4fb5db07b Java SDK: check the task handler reply against the
supervisor schema
omit 8b71e3a3132 Java SDK: list a value schema's enum constants in a stable
order
omit 09350673e05 Let the Java coordinator probe a JAR's task handlers
omit 7b09272f803 Java SDK: register stub Dags' Java tasks as task handlers
omit 4f41ac96234 Java SDK: stamp a cache digest on the bundle JAR manifest
omit e40e32a58a9 Java SDK: answer TaskHandlerParseRequest with registered
handlers
omit bfaba7c83e7 Java SDK: record handler parameters in generated task
classes
omit 3a06ce0b13f Java SDK: sync the vendored supervisor schema from the
monorepo
omit 892f414788b Skip the TS real probe when node cannot report a version
omit 815526353ee TS SDK: name the task handler lookup like its siblings
omit 07487c8e200 Let the Node coordinator probe a bundle's task handlers
omit 3366e00c985 TS SDK: answer TaskHandlerParseRequest from bundle.serve
omit 337387a178f Resolve the probed bundle path and tighten the digest docs
omit 032e2f0a532 Go SDK: take the bundle digests from the staged executable
omit d1da1df59d9 Document the task handler parse for Language SDK authors
omit 7adea3a5f72 Read a Go bundle's stored cache digest without hashing it
omit c061c850f6c Let the Executable coordinator probe a bundle's task
handlers
omit d88945d16d2 Go SDK: record integrity and cache digests in bundle
metadata
omit 84d038e3d5b Go SDK: regenerate genmodels for the named_open binding
omit e2992496cbc Go SDK: clarify parse timeout and nested-null docs, test
anyOf
omit 56ee9fd69e9 Go SDK: answer TaskHandlerParseRequest with registered
handlers
omit 4207fc3cc94 Go SDK: describe task handler params with JSON Schema
omit 38e5a70d27b Go SDK: regenerate genmodels from the supervisor schema
omit ff057376091 Pass the binding mode in the task-handler probe tests
omit 69068944577 Tidy the runtime transport and the coordinator lookup
omit bc43656f8ca Clean up a Lang-SDK runtime parse whose start fails after
the fork
omit 8cf04ea05d4 Do not exec a Lang-SDK runtime whose parent has already
exited
omit 96a771300b6 Kill what a Lang-SDK runtime leaves behind when it exits
omit ca3873d1503 Read a Lang-SDK runtime's frames without blocking the
parse loop
omit fb9bf649c9f Probe a Lang-SDK artifact for its task handlers
omit b8c6c8edaab Keep a nested supervised child's standard streams open
omit 47f930ab4f4 Make the Dag file processor base generic over its result
omit 5008ebb5bed Let a subprocess coordinator exec a task-handler parse
runtime
omit 2581bfe5c00 Add the named_open binding to the Lang-SDK ADRs
omit ee50aa3e5d1 Regenerate TS SDK supervisor types for the named_open
binding
omit 33d85cd7025 Add a named_open binding for non-exhaustive task-handler
params
omit 47cc9b0e24a Describe the binding-mode parse check in ADR-0011 and
ADR-0013
omit 1dc0346b0a5 Describe the task-handler binding mode in ADR-0012
omit 5772a57753f Regenerate TS SDK supervisor types for the binding mode
omit 779fc128fc6 Add a binding mode to task-handler declarations
omit f82c8b3fa72 Document the version rules for a new supervisor schema body
omit 729cbb9b666 Regenerate TS SDK supervisor types for task-handler
messages
omit 9e8ea9b255c Add task-handler parse messages and their parse-channel
union
omit 43bb8f64aa6 Move the Dag file processor's shared plumbing into a base
class
omit 5da7e9fda61 Add handler_binding to ADR-0013's task handler table SQL
omit b3a9b84da90 Store the task handler binding mode beside its params
omit f40d491c6d2 Regenerate the task handler migration revision id
omit cef69c36b71 Describe LangSDKTaskHandler as a stub task binding
omit 2a1ca56a8f4 Test that a referenced task handler artifact cannot be
deleted
omit a0c8b36e0b6 Keep the artifact path hash in sync on ORM updates
omit 5e43536ef5c Match ADR-0013 task handler table SQL to the migration
omit 25cde48b0bf Add lang-SDK task handler binding tables
omit 3a388582f6d Drop em-dashes from the rewritten doc sentences
omit 0c71cd2d982 Say artifact Dag bundles are registered on every component
omit 7f8c94f44ba Tidy names and mocks in the artifact bundle tests
omit ce3283c8f60 Document artifact Dag bundles for language SDK coordinators
omit 05b910a8b6e Serve lang-SDK harness artifacts from Dag bundles
add aa86cf515bc Serve lang-SDK harness artifacts from Dag bundles
add ab6a29a7c17 Document artifact Dag bundles for language SDK coordinators
add 40816099c5f Tidy names and mocks in the artifact bundle tests
add 16ca1156fdb Say artifact Dag bundles are registered on every component
add aa4eac77d74 Drop em-dashes from the rewritten doc sentences
add 70fa88cbcea Add lang-SDK task handler binding tables
add 5f687e243ea Match ADR-0013 task handler table SQL to the migration
add c51dc0150d1 Keep the artifact path hash in sync on ORM updates
add c2895cdca55 Test that a referenced task handler artifact cannot be
deleted
add fd0edc74f5a Describe LangSDKTaskHandler as a stub task binding
add 3c9a6d4031d Regenerate the task handler migration revision id
add bd2da07a4c4 Store the task handler binding mode beside its params
add f31dc1698bd Add handler_binding to ADR-0013's task handler table SQL
add 5e4c9c78067 Make session keyword-only in the task handler model test
helpers
add bfebdca3491 Move the Dag file processor's shared plumbing into a base
class
add c93fea0611b Test closing a Dag file processor that has no parse log
file
add 8b588e041e6 Add task-handler parse messages and their parse-channel
union
add 382be9bd27a Regenerate TS SDK supervisor types for task-handler
messages
add 91bca197ee5 Document the version rules for a new supervisor schema body
add 47e91566620 Add a binding mode to task-handler declarations
add 100b367d4ca Regenerate TS SDK supervisor types for the binding mode
add eea3bb67d8e Describe the task-handler binding mode in ADR-0012
add df060bc10fb Describe the binding-mode parse check in ADR-0011 and
ADR-0013
add 144d9afd4fd Add a named_open binding for non-exhaustive task-handler
params
add fb12a154e87 Regenerate TS SDK supervisor types for the named_open
binding
add 545f056e252 Add the named_open binding to the Lang-SDK ADRs
add ff2d5a0e595 Let a subprocess coordinator exec a task-handler parse
runtime
add c12e1d19d12 Make the Dag file processor base generic over its result
add ce9dfb1f392 Keep a nested supervised child's standard streams open
add 4a633473f4b Probe a Lang-SDK artifact for its task handlers
add ba9f0b2e316 Read a Lang-SDK runtime's frames without blocking the
parse loop
add b13a33f3459 Kill what a Lang-SDK runtime leaves behind when it exits
add 11cca12bb4b Do not exec a Lang-SDK runtime whose parent has already
exited
add 04a05146041 Clean up a Lang-SDK runtime parse whose start fails after
the fork
add 83e3773bc59 Tidy the runtime transport and the coordinator lookup
add 80e12aa5128 Pass the binding mode in the task-handler probe tests
add 6044ba991e6 Import ctypes at the top of the subprocess coordinator
module
add 8a920ea547b Spec the mocks in the runtime transport and probe tests
add e0953d4e101 Test that a start failing before the fork closes its
listeners
add abc6f951bed Hang the fake runtimes without sleeping
add c0d2a85e08f Patch with decorators in the runtime transport tests
add a524d53e10a Name the runtime transport test helpers with verbs
add 99e8fa53d7c Close the fake runtime's sockets when its reply is done
add 4351274bf6a Go SDK: regenerate genmodels from the supervisor schema
add 58bfefc169b Go SDK: describe task handler params with JSON Schema
add a82057450ae Go SDK: answer TaskHandlerParseRequest with registered
handlers
add 8021b2d8497 Go SDK: clarify parse timeout and nested-null docs, test
anyOf
add ecd74f38eb1 Go SDK: regenerate genmodels for the named_open binding
add 7baa5a66352 Go SDK: record integrity and cache digests in bundle
metadata
add 407410e15fa Let the Executable coordinator probe a bundle's task
handlers
add 9845448ce30 Read a Go bundle's stored cache digest without hashing it
add cc60918e773 Document the task handler parse for Language SDK authors
add b10637cd04e Go SDK: take the bundle digests from the staged executable
add 60c8cfdce43 Resolve the probed bundle path and tighten the digest docs
add 0745805bcc2 Move Go SDK comments back to the code they describe
add b410623c870 Tidy names, mocks and cases in the Go probe and digest
tests
add 3dd3062f521 TS SDK: answer TaskHandlerParseRequest from bundle.serve
add 352b64898c6 Let the Node coordinator probe a bundle's task handlers
add c5d1198bd7b TS SDK: name the task handler lookup like its siblings
add 3ece37cf52c Skip the TS real probe when node cannot report a version
add e48a8b6c441 Patch the TS real probe test with decorators
add cafed91aae7 Java SDK: sync the vendored supervisor schema from the
monorepo
add e2aa07c1784 Java SDK: record handler parameters in generated task
classes
add cb7682a2cba Java SDK: answer TaskHandlerParseRequest with registered
handlers
add 4c067570a81 Java SDK: stamp a cache digest on the bundle JAR manifest
add d7deb6b6530 Java SDK: register stub Dags' Java tasks as task handlers
add cfc75934b9a Let the Java coordinator probe a JAR's task handlers
add 08f6939b6c6 Java SDK: list a value schema's enum constants in a stable
order
add 2bce81e624c Java SDK: check the task handler reply against the
supervisor schema
add d9ec6f76dda Say why the Java coordinator rejects a JAR it cannot probe
add b5ec72dcbd1 Test the Java SDK schema sync hook's snapshot copy
add 832755c34fc Tidy names and mocks in the Java probe test
add d6ba5da422b Test the cache digest of classpath directories and non-zip
files
This update added new revisions after undoing existing revisions.
That is to say, some revisions that were in the old version of the
branch are not in the new version. This situation occurs
when a user --force pushes a change and generates a repository
containing something like this:
* -- * -- B -- O -- O -- O (782aeda48e7)
\
N -- N -- N
refs/heads/jason/core-taskhandler-refactor/08-java-sdk-task-handler-parse
(d6ba5da422b)
You should already have received notification emails for all of the O
revisions, and so the following emails describe only the N revisions
from the common base, B.
Any revisions marked "omit" are not gone; other references still
refer to them. Any revisions marked "discard" are gone forever.
No new revisions were added by this update.
Summary of changes:
.../tests/unit/dag_processing/fake_lang_sdk.py | 24 ++--
.../unit/dag_processing/test_lang_sdk_runtime.py | 147 ++++++++++++---------
.../tests/unit/dag_processing/test_processor.py | 22 ++-
.../dag_processing/test_task_handler_processor.py | 7 +-
.../test_task_handler_processor_go.py | 10 +-
.../test_task_handler_processor_java.py | 46 ++++---
.../test_task_handler_processor_ts.py | 31 ++---
.../unit/models/test_lang_sdk_task_handler.py | 16 +--
go-sdk/cmd/airflow-go-pack/pack_test.go | 6 +-
go-sdk/pkg/binding/declaration.go | 3 +-
go-sdk/pkg/binding/declaration_test.go | 1 -
go-sdk/pkg/execution/task_handler_parse_test.go | 1 -
.../apache/airflow/sdk/plugin/BundleDigestTest.kt | 23 ++++
scripts/ci/prek/sync_java_sdk_supervisor_schema.py | 6 +-
.../prek/test_sync_java_sdk_supervisor_schema.py | 90 +++++++++++++
.../src/airflow/sdk/coordinators/_subprocess.py | 3 +-
.../coordinators/executable/test_coordinator.py | 31 +++--
.../tests/task_sdk/coordinators/test_subprocess.py | 3 +-
18 files changed, 315 insertions(+), 155 deletions(-)
create mode 100644
scripts/tests/ci/prek/test_sync_java_sdk_supervisor_schema.py
