[
https://issues.apache.org/jira/browse/HDDS-17581?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
rajesh updated HDDS-17581:
--------------------------
Description:
h3. Problem
The OM, SCM and datanode tracing work (HDDS-17586, HDDS-17593, HDDS-17594) will
wrap internal service components in tracing proxies and add attributes and
errors to spans. The current \{{hdds-common}} tracing utilities don't support
this:
* Proxies can't wrap components created at service startup.
* Proxy spans are always CLIENT and are named after the interface, not the
implementation class.
* Failed spans get only an ERROR status, without the exception or message.
* There are no shared helpers for span attributes, events and exceptions, and
no test helper to assert on spans.
h3. Proposal
Extend \{{TracingUtil}} and \{{TraceAllMethod}} with:
* A tracing proxy that can be created at startup, takes a span kind, and names
spans \{{<implementation class>.<method>}}.
* Separate INTERNAL and CLIENT proxy helpers. Existing RPC client proxies keep
CLIENT spans.
* Standard exception recording on proxy spans, \{{executeInSpan}} and the
protocol message dispatcher.
* Null-safe helpers for span attributes, events and exceptions, plus a test
span collector.
Only \{{hdds-common}} changes; no new service spans. No proto, RPC,
configuration or storage changes.
This blocks HDDS-17586, HDDS-17593 and HDDS-17594.
was:
h3. Background
As part of HDDS-13679, OM (and later SCM and datanodes) will create their
internal components as tracing proxies through their interfaces: for OM, the
request handler, client requests, the key, bucket and volume managers, and the
authorizer. Code inside request handling will also add attributes, events and
errors to the current span.
The tracing code in \{{hdds-common}} (\{{TracingUtil}}, \{{TraceAllMethod}})
doesn't support this yet:
* \{{createProxy(delegate, itf, conf)}} only creates a proxy when there is a
current span, so it can't wrap components created at service startup.
* Every proxy span is CLIENT unless the method has \{{@InternalSpanKind}},
which is wrong for in-process components.
* Proxy spans are named after the interface, not after the class doing the work.
* Exceptions from the delegate aren't recorded on the proxy span, and
\{{executeInSpan}} and \{{OzoneProtocolMessageDispatcher}} set only the ERROR
status, without the exception or its message.
* There are no null-safe helpers for adding attributes, events or exceptions to
the current span, and no test helper for asserting on finished spans.
This sub-task adds these building blocks in \{{hdds-common}} only. It doesn't
add new spans to any service; that is done in the follow-up OM, SCM and
datanode sub-tasks.
h3. Proposed changes
*Tracing proxies*
* \{{TraceAllMethod}} takes a \{{SpanKind}}. \{{@InternalSpanKind}} can be put
on a whole delegate class as well as on a method.
* New \{{createProxy(delegate, itf, spanKind)}} names spans \{{<delegate class
simple name>.<method>}}, for example \{{KeyManagerImpl.listStatus}}. It doesn't
need a current span, so it can wrap objects created at startup when
application-aware tracing is on. It returns the delegate itself when tracing is
neither enabled nor application-aware.
* \{{createProxy(delegate, itf, conf)}} now creates INTERNAL spans. New
\{{createClientProxy(delegate, itf, conf)}} creates CLIENT spans. The existing
RPC client proxies switch to \{{createClientProxy}}, so their span kind stays
CLIENT:
** the SCM block, container location, security and secret key clients, in
\{{HAUtils}} and \{{HddsServerUtil}};
** the OM client, in \{{RpcClient}}.
* Exceptions thrown by the delegate are recorded on the proxy span. Methods
missing from the delegate, and \{{@SkipTracing}} methods, are called on the
delegate directly instead of throwing \{{NoSuchMethodException}}.
*Span helpers in \{{TracingUtil}}*
* \{{createSpan(name)}} and \{{createSpan(name, SpanKind)}} create a child of
the current span.
* \{{recordException(span, t)}} and \{{recordExceptionInCurrentSpan(t)}} record
the exception event and set the ERROR status, with the exception message as the
description.
* \{{addAttributesToCurrentSpan(key, String|long|boolean)}} and
\{{addEventToCurrentSpan(name, key, value)}} do nothing when the current span
isn't recording, and skip null values.
* New \{{SpanAttributeKeys}} holds the attribute keys shared by services,
starting with \{{ozone.status}}.
*Error recording on existing spans*
* \{{executeInSpan}} and \{{OzoneProtocolMessageDispatcher#processRequest}} use
\{{recordException}}, so failed spans carry the exception event and message.
*Tests*
* New \{{TestSpanCollector}} installs an in-memory span processor and provides
assertions on span name, kind, parent, error status and recorded exceptions.
* New tests in \{{TestTracingUtil}}, \{{TestTraceAllMethod}},
\{{TestTracingAnnotations}}, \{{TestTracingInitModes}} and
\{{TestOzoneProtocolMessageDispatcher}}.
* Test tracer providers get a span processor: with OpenTelemetry 1.67, an
\{{SdkTracerProvider}} without a processor returns no-op spans.
h3. Compatibility
No proto, RPC, configuration or RocksDB changes. When tracing is disabled and
application-aware mode is off, proxies aren't created, so behaviour is
unchanged.
> Add TracingUtil span helpers and span-kind aware tracing proxies
> ----------------------------------------------------------------
>
> Key: HDDS-17581
> URL: https://issues.apache.org/jira/browse/HDDS-17581
> Project: Apache Ozone
> Issue Type: Sub-task
> Reporter: Sravani Revuri
> Assignee: rajesh
> Priority: Major
>
> h3. Problem
> The OM, SCM and datanode tracing work (HDDS-17586, HDDS-17593, HDDS-17594)
> will wrap internal service components in tracing proxies and add attributes
> and errors to spans. The current \{{hdds-common}} tracing utilities don't
> support this:
> * Proxies can't wrap components created at service startup.
> * Proxy spans are always CLIENT and are named after the interface, not the
> implementation class.
> * Failed spans get only an ERROR status, without the exception or message.
> * There are no shared helpers for span attributes, events and exceptions, and
> no test helper to assert on spans.
> h3. Proposal
> Extend \{{TracingUtil}} and \{{TraceAllMethod}} with:
> * A tracing proxy that can be created at startup, takes a span kind, and
> names spans \{{<implementation class>.<method>}}.
> * Separate INTERNAL and CLIENT proxy helpers. Existing RPC client proxies
> keep CLIENT spans.
> * Standard exception recording on proxy spans, \{{executeInSpan}} and the
> protocol message dispatcher.
> * Null-safe helpers for span attributes, events and exceptions, plus a test
> span collector.
> Only \{{hdds-common}} changes; no new service spans. No proto, RPC,
> configuration or storage changes.
> This blocks HDDS-17586, HDDS-17593 and HDDS-17594.
--
This message was sent by Atlassian Jira
(v10.3.26#10030026)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
