seanghaeli commented on code in PR #76527:
URL: https://github.com/apache/airflow/pull/76527#discussion_r4625389390


##########
airflow-core/src/airflow/jobs/triggerer_job_runner.py:
##########
@@ -564,6 +601,76 @@ def upload_to_remote(self):
         upload_to_remote(self.bound_logger, self.ti)
 
 
[email protected](kw_only=True)
+class WatcherLoggingFactory(TriggerLoggingFactory):
+    """
+    An asset watcher's own log file on this triggerer, as its owner has no log.
+
+    It is kept while the trigger is assigned here, across restarts of the 
watcher (SQS-style ones restart
+    after every event), so it is appended to, and capped on local disk. It is 
uploaded with the remote log
+    handler task logs use, and every line keeps its ``trigger_id``.
+    """
+
+    max_bytes: int = 0
+    to_stdout: bool = True
+    # Told about every line, to cap how many of these files are open at once.
+    on_write: Callable[[int], None] | None = attrs.field(default=None, 
repr=False)
+    may_rotate: Callable[[], bool] | None = attrs.field(default=None, 
repr=False)
+    on_rotate: Callable[[], None] | None = attrs.field(default=None, 
repr=False)
+
+    def _open(self, processors: Iterable[structlog.typing.Processor]) -> None:
+        if hasattr(self, "bound_logger"):
+            # Before the processors run: a handler streaming each line 
(CloudWatch) names its stream after the
+            # open file, and would drop the line of a file the open-files cap 
closed.
+            self.bound_logger._logger.ensure_open()

Review Comment:
   Both calls are needed, but only one needs the check for a removed file, so 
it now only runs in `msg`. One fstat per line now.
   



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to