xBis7 commented on code in PR #43941:
URL: https://github.com/apache/airflow/pull/43941#discussion_r2004230514
##########
airflow/traces/otel_tracer.py:
##########
@@ -57,29 +56,65 @@ class OtelTrace:
When OTEL is enabled, the Trace class will be replaced by this class.
"""
- def __init__(self, span_exporter: OTLPSpanExporter, tag_string: str | None
= None):
+ def __init__(
+ self,
+ span_exporter: OTLPSpanExporter,
+ use_simple_processor: bool,
+ tag_string: str | None = None,
+ ):
self.span_exporter = span_exporter
- self.span_processor = BatchSpanProcessor(self.span_exporter)
+ self.use_simple_processor = use_simple_processor
+ if self.use_simple_processor:
+ # With a BatchSpanProcessor, spans are exported at an interval.
+ # A task can run fast and finish before spans have enough time to
get exported to the collector.
+ # When creating spans from inside a task, a SimpleSpanProcessor
needs to be used because
+ # it exports the spans immediately after they are created.
+ log.info("(__init__) - [SimpleSpanProcessor] is being used")
Review Comment:
The way that the Trace class is setup, `otel_tracer` is initialized every
time `Trace.something()` is called. This ends up in the caller's logs. Most of
the time the caller is the scheduler but it could also appear under task logs.
I'll update the comment.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
