This is an automated email from the ASF dual-hosted git repository.
uranusjr pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/airflow.git
The following commit(s) were added to refs/heads/main by this push:
new adb4b56a80d Document task subprocess lifecycle and pre-connect log
buffer (#70076)
adb4b56a80d is described below
commit adb4b56a80d5c0e61d645d37ad16d48f28f20bc5
Author: Jason(Zhe-You) Liu <[email protected]>
AuthorDate: Sat Jul 18 22:05:17 2026 +0800
Document task subprocess lifecycle and pre-connect log buffer (#70076)
---
contributing-docs/30_new_language_sdk.rst | 29 +++++++++++++++++++++++++++++
1 file changed, 29 insertions(+)
diff --git a/contributing-docs/30_new_language_sdk.rst
b/contributing-docs/30_new_language_sdk.rst
index f1fc2245909..21af384bb09 100644
--- a/contributing-docs/30_new_language_sdk.rst
+++ b/contributing-docs/30_new_language_sdk.rst
@@ -237,6 +237,30 @@ match responses to requests. The SDK MUST correlate by
``id`` whenever multiple
requests can be outstanding simultaneously (e.g. when tasks are executed
concurrently, or when a single task issues multiple requests).
+Task subprocess lifecycle
+~~~~~~~~~~~~~~~~~~~~~~~~~
+
+Before implementing logging, it helps to see the full lifecycle of the
+subprocess relative to the ``--logs`` socket:
+
+.. code-block:: text
+
+ 1. Supervisor launches the subprocess with
+ --comm=<host>:<port> --logs=<host>:<port>
+ │
+ └─► 2. Subprocess starts. Its logger is already active, but the
+ --logs socket is not connected yet.
+ │
+ └─► 3. Subprocess connects to --comm and --logs.
+ │
+ └─► 4. Each subsequent record is sent over the
+ --logs socket.
+
+The gap is stage 2: the SDK's own startup code (argument parsing, the connect
+calls themselves) can already produce log records before the ``--logs`` socket
+in stage 3 exists to carry them. See `Logging`_ below for how to handle that
+gap.
+
Logging
~~~~~~~
@@ -246,6 +270,11 @@ during execution to Airflow's task log store, so they
appear in the UI with the
rest of the output. Records travel over the ``--logs`` socket introduced in
`Startup`_.
+Records produced during stage 2 of the `Task subprocess lifecycle`_ — before
+the ``--logs`` socket connects — MUST NOT be dropped. Buffer them in memory and
+flush the buffer, in order, as soon as the socket connects, before sending any
+later record. Both the Go SDK and Java SDK already implement this buffer.
+
Log messages should be **newline-delimited JSON**. Each log is a UTF-8 encoded
JSON object in one line, terminated by a newline character (``\n``). Each
object
is a structlog-style event:
