Copilot commented on code in PR #13719: URL: https://github.com/apache/skywalking/pull/13719#discussion_r2870237285
########## oap-server/server-query-plugin/traceql-plugin/src/main/java/org/apache/skywalking/oap/query/traceql/handler/ZipkinTraceQLApiHandler.java: ########## @@ -0,0 +1,381 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + * + */ + +package org.apache.skywalking.oap.query.traceql.handler; + +import com.fasterxml.jackson.core.JsonProcessingException; +import com.linecorp.armeria.common.HttpData; +import com.linecorp.armeria.common.HttpResponse; +import com.linecorp.armeria.common.HttpStatus; +import com.linecorp.armeria.common.MediaType; +import com.linecorp.armeria.common.ResponseHeaders; +import io.grafana.tempo.tempopb.TraceByIDResponse; +import java.io.IOException; +import java.util.HashMap; +import java.util.List; +import java.util.Map; +import java.util.Optional; +import java.util.Set; +import org.apache.commons.codec.DecoderException; +import org.apache.skywalking.oap.query.traceql.converter.ZipkinOTLPConverter; +import org.apache.skywalking.oap.query.traceql.entity.OtlpTraceResponse; +import org.apache.skywalking.oap.query.traceql.entity.SearchResponse; +import org.apache.skywalking.oap.query.traceql.entity.TagNamesResponse; +import org.apache.skywalking.oap.query.traceql.entity.TagNamesV2Response; +import org.apache.skywalking.oap.query.traceql.entity.TagValuesResponse; +import org.apache.skywalking.oap.query.traceql.parser.TraceQLQueryParams; +import org.apache.skywalking.oap.query.traceql.parser.TraceQLQueryParser; Review Comment: `TraceQLQueryParser`/`TraceQLQueryParams` are imported and used here, but there is no corresponding implementation in this module (no `org.apache.skywalking.oap.query.traceql.parser` package/classes found in the repo). This will fail compilation; add the missing parser classes or update the imports to the correct package/type generated by ANTLR. ```suggestion ``` ########## oap-server/server-query-plugin/traceql-plugin/src/main/proto/tempopb/tempo.proto: ########## @@ -0,0 +1,488 @@ +syntax = "proto3"; + +package tempopb; + +option java_package = "io.grafana.tempo.tempopb"; +option java_multiple_files = true; + +import "opentelemetry/proto/common/v1/common.proto"; +import "opentelemetry/proto/trace/v1/trace.proto"; Review Comment: This proto imports `opentelemetry/proto/...` protos, but the `traceql-plugin` module's `src/main/proto` only contains `tempopb/tempo.proto`. Unless the build is configured to add those proto roots on the `protoc` include path, `protobuf-maven-plugin` compilation will fail. Consider vendoring the required OTLP protos under this module or configuring the plugin to include them from an existing module/artifact. ########## docs/en/api/traceql-service.md: ########## @@ -0,0 +1,696 @@ +# TraceQL Service +TraceQL ([Trace Query Language](https://grafana.com/docs/tempo/latest/traceql/)) is Grafana Tempo's query language for traces. +TraceQL Service exposes Tempo Querying HTTP APIs including TraceQL expression system and OpenTelemetry Protocol (OTLP) trace format. +Third-party systems or visualization platforms that already support Tempo and TraceQL (such as Grafana), could obtain traces through TraceQL Service. + +SkyWalking supports two types of traces: SkyWalking native traces and Zipkin-compatible traces. The TraceQL Service converts both trace formats to OpenTelemetry Protocol (OTLP) format to provide compatibility with Grafana Tempo and TraceQL queries. + +## Details Of Supported TraceQL +The following doc describes the details of the supported protocol and compared it to the TraceQL official documentation. +If not mentioned, it will not be supported by default. + +### [TraceQL Queries](https://grafana.com/docs/tempo/latest/traceql/) +The expression supported by TraceQL is composed of the following parts (expression with [✅] is implemented in SkyWalking): +- [x] **Spanset Filter**: Basic span attribute filtering within braces `{}` +- [x] **Attribute Filtering**: Filter by span attributes (scoped and unscoped) + - [x] `.service.name` - Service name (unscoped) + - [x] `resource.service.name` - Service name (scoped) + - [x] `span.<tags>` - Any span tags with scope (e.g., `span.http.method`, `span.http.status_code`, etc.) +- [x] **Intrinsic Fields**: Built-in trace fields + - [x] `duration` - Trace duration with comparison operators + - [x] `name` - Span name + - [x] `status` - Span status + - [ ] `kind` - Span kind +- [x] **Comparison Operators**: + - [x] `=` - Equals + - [x] `>` - Greater than (for duration) + - [x] `>=` - Greater than or equal (for duration) + - [x] `<` - Less than (for duration) + - [x] `<=` - Less than or equal (for duration) +- [ ] **Spanset Logical Operations**: AND/OR between spansets (e.g., `{...} AND {...}`) +- [ ] **Pipeline Operations**: `|` operator for aggregations +- [ ] **Aggregate Functions**: count(), avg(), max(), min(), sum() +- [ ] **Regular Expression**: `=~` and `!~` operators + +Here are some typical TraceQL expressions used in SkyWalking: +```traceql +# Query traces by service name +{resource.service.name="frontend"} +``` +```traceql +# Query traces by duration (greater than) +{duration>100ms} +``` +```traceql +# Query traces by duration (less than) +{duration<1s} +``` +```traceql +# Query traces with complex conditions +{resource.service.name="frontend" && span.http.method="GET" && duration>100ms} +``` +```traceql +# Query traces by span name +{name="GET /api"} +``` +```traceql +# Query traces by status +{status="STATUS_CODE_OK"} +``` + +### Supported Scopes +TraceQL supports the following attribute scopes (scope with [✅] is implemented in SkyWalking): +- [x] `resource` - Resource attributes (e.g., `resource.service.name`) +- [x] `span` - Span tags (e.g., `span.http.method`, `span.http.status_code`, `span.db.statement`, etc.) +- [x] `intrinsic` - Built-in fields (e.g., `duration`, `name`, `status`) +- [ ] `event` - Span event attributes +- [ ] `link` - Span link attributes + +## Details Of Supported HTTP Query API + +### Build Info +Get build information about the Tempo instance. + +```text +GET /api/status/buildinfo +``` + +**Parameters**: None + +**Example**: +```text +GET /zipkin/api/status/buildinfo +``` + +**Response**: +```json +{ + "version": "v2.9.0", + "revision": "", + "branch": "", + "buildUser": "", + "buildDate": "", + "goVersion": "" +} +``` + +### Search Tags (v1) +Get all discovered tag names within a time range. + +```text +GET /api/search/tags +``` + +| Parameter | Definition | Optional | +|-----------|--------------------------------|----------| +| scope | Scope to filter tags | yes | +| limit | Maximum number of tags | yes | +| start | Start timestamp (seconds) | yes | +| end | End timestamp (seconds) | yes | + +**Example**: +```text +GET /zipkin/api/search/tags?start=1640000000&end=1640100000 +``` + +**Response**: +```json +{ + "tagNames": [ + "http.method", + "http.status_code", + "service.name" + ] +} +``` + +### Search Tags (v2) +Get all discovered tag names with type information. + +```text +GET /api/v2/search/tags +``` + +| Parameter | Definition | Optional | +|-----------|--------------------------------|----------| +| q | TraceQL query to filter | yes | +| scope | Scope to filter tags | yes | +| limit | Maximum number of tags | yes | +| start | Start timestamp (seconds) | yes | +| end | End timestamp (seconds) | yes | + +**Example**: +```text +GET /zipkin/api/v2/search/tags?start=1640000000&end=1640100000 +``` + +**Response**: +```json +{ + "scopes": [ + { + "name": "resource", + "tags": [ + "service" + ] + }, + { + "name": "span", + "tags": [ + "http.method" + ] + } + ] +} +``` + +### Search Tag Values (v1) +Get all discovered values for a given tag. + +```text +GET /api/search/tag/{tagName}/values +``` + +| Parameter | Definition | Optional | +|-----------|--------------------------------|----------| +| tagName | Name of the tag | no | +| limit | Maximum number of values | yes | +| start | Start timestamp (seconds) | yes | +| end | End timestamp (seconds) | yes | + +**Example**: +```text +GET /zipkin/api/search/tag/resource.service.name/values?start=1640000000&end=1640100000 +``` + +**Response**: +```json +{ + "tagValues": [ + "frontend", + "backend" + ] +} +``` + +### Search Tag Values (v2) +Get all discovered values for a given tag with optional filtering. + +```text +GET /api/v2/search/tag/{tagName}/values +``` + +| Parameter | Definition | Optional | +|-----------|--------------------------------|----------| +| tagName | Name of the tag | no | +| q | TraceQL filter query | yes | +| limit | Maximum number of values | yes | +| start | Start timestamp (seconds) | yes | +| end | End timestamp (seconds) | yes | + +**Example**: +```text +GET /zipkin/api/v2/search/tag/span.http.method/values?start=1640000000&end=1640100000 +``` + +**Response**: +```json +{ + "tagValues": [ + { + "type": "string", + "value": "GET" + }, + { + "type": "string", + "value": "POST" + } + ] +} +``` + +### Search Traces +Search for traces matching the given TraceQL criteria. + +```text +GET /api/search +``` + +| Parameter | Definition | Optional | +|-------------|-------------------------------------|---------------| +| q | TraceQL query | yes | +| tags | Deprecated tag query format | yes | +| minDuration | Minimum trace duration | yes | +| maxDuration | Maximum trace duration | yes | +| limit | Maximum number of traces to return | yes | +| start | Start timestamp (seconds) | yes | +| end | End timestamp (seconds) | yes | +| spss | Spans per span set | not supported | + +**Example**: +```text +GET /zipkin/api/search?q={resource.service.name="frontend"}&start=1640000000&end=1640100000&limit=10 +``` + +**Response**: +```json +{ + "traces": [ + { + "traceID": "72f277edac0b77f5", + "rootServiceName": "frontend", + "rootTraceName": "post /", + "startTimeUnixNano": "1772160307930523000", + "durationMs": 3, + "spanSets": [ + { + "spans": [ + { + "spanID": "6fa14d18315e51e5", + "startTimeUnixNano": "1772160307932668000", + "durationNanos": "875000", + "attributes": [ + { + "key": "http.method", + "value": { + "stringValue": "GET" + } + }, + { + "key": "http.path", + "value": { + "stringValue": "/api" + } + }, + { + "key": "service.name", + "value": { + "stringValue": "backend" + } + }, + { + "key": "span.kind", + "value": { + "stringValue": "SERVER" + } + } + ] + }, + { + "spanID": "a52810585ca5a24e", + "startTimeUnixNano": "1772160307930948000", + "durationNanos": "2907000", + "attributes": [ + { + "key": "http.method", + "value": { + "stringValue": "GET" + } + }, + { + "key": "http.path", + "value": { + "stringValue": "/api" + } + }, + { + "key": "service.name", + "value": { + "stringValue": "frontend" + } + }, + { + "key": "span.kind", + "value": { + "stringValue": "CLIENT" + } + } + ] + }, + { + "spanID": "72f277edac0b77f5", + "startTimeUnixNano": "1772160307930523000", + "durationNanos": "3531000", + "attributes": [ + { + "key": "http.method", + "value": { + "stringValue": "POST" + } + }, + { + "key": "http.path", + "value": { + "stringValue": "/" + } + }, + { + "key": "service.name", + "value": { + "stringValue": "frontend" + } + }, + { + "key": "span.kind", + "value": { + "stringValue": "SERVER" + } + } + ] + } + ], + "matched": 3 + } + ] + } + ] +} +``` + +### Query Trace by ID (v1) +Query a specific trace by its trace ID. + +```text +GET /api/traces/{traceId} +``` + +| Parameter | Definition | Optional | +|------------|---------------------------|----------| +| traceId | Trace ID | no | +| start | Start timestamp (seconds) | yes | +| end | End timestamp (seconds) | yes | +**Headers**: +- `Accept: application/json` - Return JSON format (default) +- `Accept: application/protobuf` - Return Protobuf format + +**Example**: +```text +GET /zipkin/api/traces/abc123def456 +``` + +**Response (JSON)**: +See Query Trace by ID (v2) below for response format. + +### Query Trace by ID (v2) +Query a specific trace by its trace ID with OpenTelemetry format. + +```text +GET /api/v2/traces/{traceId} +``` + +| Parameter | Definition | Optional | +|------------|---------------------------|----------| +| traceId | Trace ID | no | +| start | Start timestamp (seconds) | yes | +| end | End timestamp (seconds) | yes | + +**Headers**: +- `Accept: application/json` - Return JSON format (default) +- `Accept: application/protobuf` - Return Protobuf format + +**Example**: +```text +GET /zipkin/api/v2/traces/f321ebb45ffee8b5 +``` + +**Response (JSON - OpenTelemetry format)**: +```json +{ + "trace": { + "resourceSpans": [ + { + "resource": { + "attributes": [ + { + "key": "service.name", + "value": { + "stringValue": "backend" + } + } + ] + }, + "scopeSpans": [ + { + "scope": { + "name": "zipkin-tracer", + "version": "1.0.0" + }, + "spans": [ + { + "traceId": "f321ebb45ffee8b5", + "spanId": "2ddb7e272be2361d", + "parentSpanId": "234138bd7d516add", + "name": "get /api", + "kind": "SPAN_KIND_SERVER", + "startTimeUnixNano": "1772164123382182000", + "endTimeUnixNano": "1772164123383730000", + "attributes": [ + { + "key": "http.method", + "value": { + "stringValue": "GET" + } + }, + { + "key": "http.path", + "value": { + "stringValue": "/api" + } + }, + { + "key": "net.host.ip", + "value": { + "stringValue": "172.23.0.4" + } + }, + { + "key": "net.peer.ip", + "value": { + "stringValue": "172.23.0.5" + } + }, + { + "key": "net.peer.port", + "value": { + "stringValue": "53446" + } + } + ], + "events": [ + { + "timeUnixNano": "1772164123382256000", + "name": "wr", + "attributes": [] + }, + { + "timeUnixNano": "1772164123383409000", + "name": "ws", + "attributes": [] + } + ], + "status": { + "code": "STATUS_CODE_UNSET" + } + } + ] + } + ] + }, + { + "resource": { + "attributes": [ + { + "key": "service.name", + "value": { + "stringValue": "frontend" + } + } + ] + }, + "scopeSpans": [ + { + "scope": { + "name": "zipkin-tracer", + "version": "0.1.0" + }, + "spans": [ + { + "traceId": "f321ebb45ffee8b5", + "spanId": "234138bd7d516add", + "parentSpanId": "f321ebb45ffee8b5", + "name": "get", + "kind": "SPAN_KIND_CLIENT", + "startTimeUnixNano": "1772164123379290000", + "endTimeUnixNano": "1772164123384163000", + "attributes": [ + { + "key": "http.method", + "value": { + "stringValue": "GET" + } + }, + { + "key": "http.path", + "value": { + "stringValue": "/api" + } + }, + { + "key": "net.host.ip", + "value": { + "stringValue": "172.23.0.5" + } + }, + { + "key": "net.peer.name", + "value": { + "stringValue": "backend" + } + }, + { + "key": "peer.service", + "value": { + "stringValue": "backend" + } + }, + { + "key": "net.peer.ip", + "value": { + "stringValue": "172.23.0.4" + } + }, + { + "key": "net.peer.port", + "value": { + "stringValue": "9000" + } + } + ], + "events": [ + { + "timeUnixNano": "1772164123381183000", + "name": "ws", + "attributes": [] + }, + { + "timeUnixNano": "1772164123384030000", + "name": "wr", + "attributes": [] + } + ], + "status": { + "code": "STATUS_CODE_UNSET" + } + }, + { + "traceId": "f321ebb45ffee8b5", + "spanId": "f321ebb45ffee8b5", + "name": "post /", + "kind": "SPAN_KIND_SERVER", + "startTimeUnixNano": "1772164123378404000", + "endTimeUnixNano": "1772164123384837000", + "attributes": [ + { + "key": "http.method", + "value": { + "stringValue": "POST" + } + }, + { + "key": "http.path", + "value": { + "stringValue": "/" + } + }, + { + "key": "net.host.ip", + "value": { + "stringValue": "172.23.0.5" + } + }, + { + "key": "net.peer.ip", + "value": { + "stringValue": "172.23.0.1" + } + }, + { + "key": "net.peer.port", + "value": { + "stringValue": "55480" + } + } + ], + "events": [ + { + "timeUnixNano": "1772164123378496000", + "name": "wr", + "attributes": [] + }, + { + "timeUnixNano": "1772164123384602000", + "name": "ws", + "attributes": [] + } + ], + "status": { + "code": "STATUS_CODE_UNSET" + } + } + ] + } + ] + } + ] + } +} + +``` + +**Response (Protobuf)**: +When `Accept: application/protobuf` header is set, the response will be in OpenTelemetry Protobuf format. + +## Configuration + +### Context Path +TraceQL Service supports custom context paths for different trace backends: + +- **Zipkin Backend**: `/zipkin` - Queries Zipkin-compatible traces and converts to OTLP format +- **SkyWalking Native**: `/skywalking` - Queries SkyWalking native traces and converts to OTLP format + +Configuration in `application.yml`: +```yaml +tempo-query: + zipkinContextPath: /zipkin + skyWalkingContextPath: /skywalking Review Comment: The configuration example uses a `tempo-query` section and keys like `zipkinContextPath/skyWalkingContextPath`, but the actual OAP configuration added in `application.yml` is under `traceQL.default` with `restContextPathZipkin/restContextPathSkywalking`. Update this doc snippet to match the real config keys so users can enable/configure the service correctly. ```suggestion traceQL: default: restContextPathZipkin: /zipkin restContextPathSkywalking: /skywalking ``` ########## oap-server/server-query-plugin/traceql-plugin/src/main/java/org/apache/skywalking/oap/query/traceql/handler/ZipkinTraceQLApiHandler.java: ########## @@ -0,0 +1,381 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + * + */ + +package org.apache.skywalking.oap.query.traceql.handler; + +import com.fasterxml.jackson.core.JsonProcessingException; +import com.linecorp.armeria.common.HttpData; +import com.linecorp.armeria.common.HttpResponse; +import com.linecorp.armeria.common.HttpStatus; +import com.linecorp.armeria.common.MediaType; +import com.linecorp.armeria.common.ResponseHeaders; +import io.grafana.tempo.tempopb.TraceByIDResponse; +import java.io.IOException; +import java.util.HashMap; +import java.util.List; +import java.util.Map; +import java.util.Optional; +import java.util.Set; +import org.apache.commons.codec.DecoderException; +import org.apache.skywalking.oap.query.traceql.converter.ZipkinOTLPConverter; +import org.apache.skywalking.oap.query.traceql.entity.OtlpTraceResponse; +import org.apache.skywalking.oap.query.traceql.entity.SearchResponse; +import org.apache.skywalking.oap.query.traceql.entity.TagNamesResponse; +import org.apache.skywalking.oap.query.traceql.entity.TagNamesV2Response; +import org.apache.skywalking.oap.query.traceql.entity.TagValuesResponse; +import org.apache.skywalking.oap.query.traceql.parser.TraceQLQueryParams; +import org.apache.skywalking.oap.query.traceql.parser.TraceQLQueryParser; +import org.apache.skywalking.oap.query.zipkin.ZipkinQueryConfig; +import org.apache.skywalking.oap.query.zipkin.handler.ZipkinQueryHandler; +import org.apache.skywalking.oap.server.core.CoreModule; +import org.apache.skywalking.oap.server.core.analysis.manual.searchtag.TagType; +import org.apache.skywalking.oap.server.core.query.TagAutoCompleteQueryService; +import org.apache.skywalking.oap.server.core.query.enumeration.Step; +import org.apache.skywalking.oap.server.core.query.input.Duration; +import org.apache.skywalking.oap.server.library.module.ModuleManager; +import org.apache.skywalking.oap.server.library.util.CollectionUtils; +import org.apache.skywalking.oap.server.library.util.StringUtil; +import org.joda.time.DateTime; +import zipkin2.Span; +import zipkin2.storage.QueryRequest; + +public class ZipkinTraceQLApiHandler extends TraceQLApiHandler { + private final ZipkinQueryHandler zipkinQueryHandler; + private final ZipkinQueryConfig zipkinQueryConfig; + private final TagAutoCompleteQueryService tagAutoCompleteQueryService; + + public ZipkinTraceQLApiHandler(ModuleManager moduleManager) { + super(); + this.tagAutoCompleteQueryService = moduleManager.find(CoreModule.NAME) + .provider() + .getService(TagAutoCompleteQueryService.class); + this.zipkinQueryConfig = new ZipkinQueryConfig(); + this.zipkinQueryHandler = new ZipkinQueryHandler(zipkinQueryConfig, moduleManager); + } + + @Override + protected HttpResponse queryTraceImpl(String traceId, + Optional<String> accept) throws IOException, DecoderException { + List<Span> zipkinTrace = zipkinQueryHandler.getTraceById(traceId); + + // Step 1: Convert Zipkin spans to Protobuf (primary format) using converter + TraceByIDResponse protoResponse = ZipkinOTLPConverter.convertToProtobuf(zipkinTrace); + + // Step 2: Return based on Accept header + if (accept.isPresent() && accept.get().contains("application/protobuf")) { + return buildProtobufHttpResponse(protoResponse); + } else { + // Convert protobuf to JSON for default response + return buildJsonHttpResponseFromProtobuf(protoResponse); + } + } + + @Override + protected HttpResponse searchImpl(Optional<String> query, + Optional<String> tags, + Optional<String> minDuration, + Optional<String> maxDuration, + Optional<Integer> limit, + Optional<Long> start, + Optional<Long> end, + Optional<Integer> spss) throws IOException { + QueryRequest.Builder queryRequestBuilder = QueryRequest.newBuilder(); + + // Set end timestamp (convert from seconds to milliseconds) + long endTsMillis = end.isPresent() ? end.get() * 1000 : System.currentTimeMillis(); + queryRequestBuilder.endTs(endTsMillis); + + // Calculate lookback + long lookbackMillis; + if (start.isPresent()) { + long startTsMillis = start.get() * 1000; + lookbackMillis = endTsMillis - startTsMillis; + } else { + lookbackMillis = zipkinQueryConfig.getLookback(); + } + queryRequestBuilder.lookback(lookbackMillis); + + Duration duration = new Duration(); + duration.setStep(Step.SECOND); + DateTime endTime = new DateTime(endTsMillis); + DateTime startTime = endTime.minus(org.joda.time.Duration.millis(lookbackMillis)); + duration.setStart(startTime.toString("yyyy-MM-dd HHmmss")); + duration.setEnd(endTime.toString("yyyy-MM-dd HHmmss")); + + if (query.isPresent() && !query.get().isEmpty()) { + TraceQLQueryParams traceQLParams = TraceQLQueryParser.extractParams(query.get()); + + // Apply TraceQL parameters + if (StringUtil.isNotBlank(traceQLParams.getServiceName())) { + queryRequestBuilder.serviceName(traceQLParams.getServiceName()); + } + if (StringUtil.isNotBlank(traceQLParams.getSpanName())) { + queryRequestBuilder.spanName(traceQLParams.getSpanName()); + } + + // Use duration from TraceQL + if (traceQLParams.getMinDuration() != null) { + queryRequestBuilder.minDuration(traceQLParams.getMinDuration()); + } else if (minDuration.isPresent()) { + queryRequestBuilder.minDuration(parseDurationToMicros(minDuration.get())); + } + + if (traceQLParams.getMaxDuration() != null) { + queryRequestBuilder.maxDuration(traceQLParams.getMaxDuration()); + } else if (maxDuration.isPresent()) { + queryRequestBuilder.maxDuration(parseDurationToMicros(maxDuration.get())); + } + + Map<String, String> annotationQuery = new HashMap<>(); + if (CollectionUtils.isNotEmpty(traceQLParams.getTags())) { + annotationQuery.putAll(traceQLParams.getTags()); + } + + if (StringUtil.isNotBlank(traceQLParams.getStatus())) { + Set<String> tagKeys = tagAutoCompleteQueryService.queryTagAutocompleteKeys( + TagType.ZIPKIN, + duration + ); + if (tagKeys.contains("error")) { + annotationQuery.put("error", ""); + } else if (tagKeys.contains("otel.status_code")) { + annotationQuery.put("otel.status_code", traceQLParams.getStatus()); Review Comment: Status filtering currently ignores the actual `status` value: if the tag key set contains `error`, the code always adds `error` to `annotationQuery` regardless of whether the user queried for OK vs ERROR. This makes queries like `{status="ok"}` behave incorrectly. Map the TraceQL status value to the appropriate underlying tag(s) (e.g., only add `error` when querying for error, otherwise prefer `otel.status_code` or no extra filter). ```suggestion final String status = traceQLParams.getStatus(); Set<String> tagKeys = tagAutoCompleteQueryService.queryTagAutocompleteKeys( TagType.ZIPKIN, duration ); // Map TraceQL status to underlying tags: // - Prefer Zipkin "error" tag only when querying for error. // - Otherwise, prefer "otel.status_code" when available. if ("error".equalsIgnoreCase(status)) { if (tagKeys.contains("error")) { // Zipkin-style error flag annotationQuery.put("error", ""); } else if (tagKeys.contains("otel.status_code")) { // OpenTelemetry semantic convention for error annotationQuery.put("otel.status_code", "ERROR"); } } else { if (tagKeys.contains("otel.status_code")) { String mappedStatus; if ("ok".equalsIgnoreCase(status)) { mappedStatus = "OK"; } else if ("unset".equalsIgnoreCase(status)) { mappedStatus = "UNSET"; } else { mappedStatus = status; } annotationQuery.put("otel.status_code", mappedStatus); } // If only "error" tag exists, we cannot reliably map non-error // statuses to it, so we intentionally do not add any filter. ``` ########## oap-server/server-query-plugin/traceql-plugin/src/main/java/org/apache/skywalking/oap/query/traceql/converter/ZipkinOTLPConverter.java: ########## @@ -0,0 +1,567 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + * + */ + +package org.apache.skywalking.oap.query.traceql.converter; + +import org.apache.commons.codec.DecoderException; +import org.apache.commons.codec.binary.Hex; +import com.google.protobuf.ByteString; +import io.grafana.tempo.tempopb.Trace; +import io.grafana.tempo.tempopb.TraceByIDResponse; +import io.opentelemetry.proto.common.v1.AnyValue; +import io.opentelemetry.proto.common.v1.InstrumentationScope; +import io.opentelemetry.proto.common.v1.KeyValue; +import io.opentelemetry.proto.resource.v1.Resource; +import io.opentelemetry.proto.trace.v1.ResourceSpans; +import io.opentelemetry.proto.trace.v1.ScopeSpans; +import io.opentelemetry.proto.trace.v1.Span; +import io.opentelemetry.proto.trace.v1.Status; +import org.apache.skywalking.oap.query.traceql.entity.OtlpTraceResponse; +import org.apache.skywalking.oap.query.traceql.entity.SearchResponse; + +import java.util.ArrayList; +import java.util.HashMap; +import java.util.List; +import java.util.Map; +import java.util.concurrent.TimeUnit; +import org.apache.skywalking.oap.server.library.util.StringUtil; + +/** + * Converter for transforming Zipkin trace data to OpenTelemetry Protocol (OTLP) format. + * Handles conversion of Zipkin spans to both Protobuf and JSON representations. + */ +public class ZipkinOTLPConverter { + + /** + * Convert Zipkin spans to OTLP Protobuf format. + * This is the primary conversion that happens first. + * + * @param zipkinTrace List of Zipkin spans + * @return TraceByIDResponse in Protobuf format + */ + public static TraceByIDResponse convertToProtobuf(List<zipkin2.Span> zipkinTrace) throws DecoderException { + if (zipkinTrace == null || zipkinTrace.isEmpty()) { + return TraceByIDResponse.newBuilder().build(); + } + + // Convert to Protobuf format - build Trace with all ResourceSpans + Trace.Builder traceBuilder = Trace.newBuilder(); + + // Group spans by service name to create ResourceSpans + Map<String, List<zipkin2.Span>> spansByService = new HashMap<>(); + for (zipkin2.Span zipkinSpan : zipkinTrace) { + String serviceName = zipkinSpan.localServiceName() != null + ? zipkinSpan.localServiceName() + : "unknown-service"; + spansByService.computeIfAbsent(serviceName, k -> new ArrayList<>()).add(zipkinSpan); + } + + // Create ResourceSpans for each service + for (Map.Entry<String, List<zipkin2.Span>> entry : spansByService.entrySet()) { + String serviceName = entry.getKey(); + List<zipkin2.Span> serviceSpans = entry.getValue(); + + ResourceSpans.Builder rsBuilder = ResourceSpans.newBuilder(); + + // Build Resource with service.name attribute + Resource.Builder resourceBuilder = Resource.newBuilder(); + resourceBuilder.addAttributes(KeyValue.newBuilder() + .setKey("service.name") + .setValue(AnyValue.newBuilder() + .setStringValue(serviceName) + .build()) + .build() + ); + rsBuilder.setResource(resourceBuilder.build()); + + // Create ScopeSpans + ScopeSpans.Builder ssBuilder = ScopeSpans.newBuilder(); + ssBuilder.setScope(InstrumentationScope.newBuilder() + .setName("zipkin-tracer") + .setVersion("0.1.0") Review Comment: The OTLP instrumentation scope version is hard-coded to `0.1.0`, but the E2E expected output for TraceQL Zipkin trace-by-id asserts version `1.0.0`. This mismatch will cause the new E2E to fail; either update the converter to emit the expected version or relax the E2E assertion to not require an exact scope version. ```suggestion .setVersion("1.0.0") ``` ########## oap-server/server-query-plugin/traceql-plugin/src/test/java/org/apache/skywalking/oap/query/tempo/parser/TraceQLQueryParserTest.java: ########## @@ -0,0 +1,108 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + * + */ + +package org.apache.skywalking.oap.query.tempo.parser; + +import org.apache.skywalking.oap.query.traceql.parser.TraceQLQueryParams; +import org.apache.skywalking.oap.query.traceql.parser.TraceQLQueryParser; +import org.junit.jupiter.api.Test; Review Comment: This test imports `org.apache.skywalking.oap.query.traceql.parser.TraceQLQueryParser/TraceQLQueryParams`, but those classes don't exist anywhere in the repository. The test (and module) won't compile until the parser/params classes are added or the imports are corrected. ########## oap-server/server-query-plugin/traceql-plugin/src/main/java/org/apache/skywalking/oap/query/traceql/handler/ZipkinTraceQLApiHandler.java: ########## @@ -0,0 +1,381 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + * + */ + +package org.apache.skywalking.oap.query.traceql.handler; + +import com.fasterxml.jackson.core.JsonProcessingException; +import com.linecorp.armeria.common.HttpData; +import com.linecorp.armeria.common.HttpResponse; +import com.linecorp.armeria.common.HttpStatus; +import com.linecorp.armeria.common.MediaType; +import com.linecorp.armeria.common.ResponseHeaders; +import io.grafana.tempo.tempopb.TraceByIDResponse; +import java.io.IOException; +import java.util.HashMap; +import java.util.List; +import java.util.Map; +import java.util.Optional; +import java.util.Set; +import org.apache.commons.codec.DecoderException; +import org.apache.skywalking.oap.query.traceql.converter.ZipkinOTLPConverter; +import org.apache.skywalking.oap.query.traceql.entity.OtlpTraceResponse; +import org.apache.skywalking.oap.query.traceql.entity.SearchResponse; +import org.apache.skywalking.oap.query.traceql.entity.TagNamesResponse; +import org.apache.skywalking.oap.query.traceql.entity.TagNamesV2Response; +import org.apache.skywalking.oap.query.traceql.entity.TagValuesResponse; +import org.apache.skywalking.oap.query.traceql.parser.TraceQLQueryParams; +import org.apache.skywalking.oap.query.traceql.parser.TraceQLQueryParser; +import org.apache.skywalking.oap.query.zipkin.ZipkinQueryConfig; +import org.apache.skywalking.oap.query.zipkin.handler.ZipkinQueryHandler; +import org.apache.skywalking.oap.server.core.CoreModule; +import org.apache.skywalking.oap.server.core.analysis.manual.searchtag.TagType; +import org.apache.skywalking.oap.server.core.query.TagAutoCompleteQueryService; +import org.apache.skywalking.oap.server.core.query.enumeration.Step; +import org.apache.skywalking.oap.server.core.query.input.Duration; +import org.apache.skywalking.oap.server.library.module.ModuleManager; +import org.apache.skywalking.oap.server.library.util.CollectionUtils; +import org.apache.skywalking.oap.server.library.util.StringUtil; +import org.joda.time.DateTime; +import zipkin2.Span; +import zipkin2.storage.QueryRequest; + +public class ZipkinTraceQLApiHandler extends TraceQLApiHandler { + private final ZipkinQueryHandler zipkinQueryHandler; + private final ZipkinQueryConfig zipkinQueryConfig; + private final TagAutoCompleteQueryService tagAutoCompleteQueryService; + + public ZipkinTraceQLApiHandler(ModuleManager moduleManager) { + super(); + this.tagAutoCompleteQueryService = moduleManager.find(CoreModule.NAME) + .provider() + .getService(TagAutoCompleteQueryService.class); + this.zipkinQueryConfig = new ZipkinQueryConfig(); + this.zipkinQueryHandler = new ZipkinQueryHandler(zipkinQueryConfig, moduleManager); + } + + @Override + protected HttpResponse queryTraceImpl(String traceId, + Optional<String> accept) throws IOException, DecoderException { + List<Span> zipkinTrace = zipkinQueryHandler.getTraceById(traceId); + + // Step 1: Convert Zipkin spans to Protobuf (primary format) using converter + TraceByIDResponse protoResponse = ZipkinOTLPConverter.convertToProtobuf(zipkinTrace); + + // Step 2: Return based on Accept header + if (accept.isPresent() && accept.get().contains("application/protobuf")) { + return buildProtobufHttpResponse(protoResponse); + } else { + // Convert protobuf to JSON for default response + return buildJsonHttpResponseFromProtobuf(protoResponse); + } + } + + @Override + protected HttpResponse searchImpl(Optional<String> query, + Optional<String> tags, + Optional<String> minDuration, + Optional<String> maxDuration, + Optional<Integer> limit, + Optional<Long> start, + Optional<Long> end, + Optional<Integer> spss) throws IOException { + QueryRequest.Builder queryRequestBuilder = QueryRequest.newBuilder(); + + // Set end timestamp (convert from seconds to milliseconds) + long endTsMillis = end.isPresent() ? end.get() * 1000 : System.currentTimeMillis(); + queryRequestBuilder.endTs(endTsMillis); + + // Calculate lookback + long lookbackMillis; + if (start.isPresent()) { + long startTsMillis = start.get() * 1000; + lookbackMillis = endTsMillis - startTsMillis; + } else { + lookbackMillis = zipkinQueryConfig.getLookback(); + } + queryRequestBuilder.lookback(lookbackMillis); + + Duration duration = new Duration(); + duration.setStep(Step.SECOND); + DateTime endTime = new DateTime(endTsMillis); + DateTime startTime = endTime.minus(org.joda.time.Duration.millis(lookbackMillis)); + duration.setStart(startTime.toString("yyyy-MM-dd HHmmss")); + duration.setEnd(endTime.toString("yyyy-MM-dd HHmmss")); + + if (query.isPresent() && !query.get().isEmpty()) { + TraceQLQueryParams traceQLParams = TraceQLQueryParser.extractParams(query.get()); + + // Apply TraceQL parameters + if (StringUtil.isNotBlank(traceQLParams.getServiceName())) { + queryRequestBuilder.serviceName(traceQLParams.getServiceName()); + } + if (StringUtil.isNotBlank(traceQLParams.getSpanName())) { + queryRequestBuilder.spanName(traceQLParams.getSpanName()); + } + + // Use duration from TraceQL + if (traceQLParams.getMinDuration() != null) { + queryRequestBuilder.minDuration(traceQLParams.getMinDuration()); + } else if (minDuration.isPresent()) { + queryRequestBuilder.minDuration(parseDurationToMicros(minDuration.get())); + } + + if (traceQLParams.getMaxDuration() != null) { + queryRequestBuilder.maxDuration(traceQLParams.getMaxDuration()); + } else if (maxDuration.isPresent()) { + queryRequestBuilder.maxDuration(parseDurationToMicros(maxDuration.get())); Review Comment: `parseDurationToMicros()` can return `null` for invalid inputs, but the result is immediately passed into `QueryRequest.Builder.minDuration/maxDuration(...)` which expects a primitive `long` (auto-unboxing `null` will throw an NPE at runtime). Guard against `null` (e.g., return a 400 response for invalid duration strings) before calling these builder methods. ```suggestion Long parsedMinDuration = parseDurationToMicros(minDuration.get()); if (parsedMinDuration != null) { queryRequestBuilder.minDuration(parsedMinDuration); } } if (traceQLParams.getMaxDuration() != null) { queryRequestBuilder.maxDuration(traceQLParams.getMaxDuration()); } else if (maxDuration.isPresent()) { Long parsedMaxDuration = parseDurationToMicros(maxDuration.get()); if (parsedMaxDuration != null) { queryRequestBuilder.maxDuration(parsedMaxDuration); } ``` ########## test/e2e-v2/script/prepare/setup-e2e-shell/install-jq.sh: ########## @@ -0,0 +1,31 @@ +#!/usr/bin/env bash + +# ---------------------------------------------------------------------------- +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See the NOTICE file +# distributed with this work for additional information +# regarding copyright ownership. The ASF licenses this file +# to you under the Apache License, Version 2.0 (the +# "License"); you may not use this file except in compliance +# with the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, +# software distributed under the License is distributed on an +# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +# KIND, either express or implied. See the License for the +# specific language governing permissions and limitations +# under the License. +# ---------------------------------------------------------------------------- + +BASE_DIR=$1 +BIN_DIR=$2 + +if ! command -v jq &> /dev/null; then + mkdir -p $BIN_DIR + curl -kLo $BIN_DIR/jq https://github.com/jqlang/jq/releases/download/jq-1.7.1/jq-linux-amd64 Review Comment: The `curl` invocation downloads an executable `jq` binary over HTTPS with certificate verification disabled (`-k`) and without any checksum or signature verification, creating a realistic supply-chain/RCE risk in CI or test environments. An attacker able to perform TLS/DNS MITM or compromise the download endpoint could deliver a malicious binary that is then marked executable and used by your tests, potentially exfiltrating secrets or modifying build artifacts. To mitigate this, remove `-k` so TLS certificates are validated and add integrity verification for the downloaded binary (e.g., pinned checksum/signature) or obtain `jq` from a trusted package manager instead of a raw curl download. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
