This is an automated email from the ASF dual-hosted git repository.
adriancole pushed a commit to branch master
in repository
https://gitbox.apache.org/repos/asf/incubator-zipkin-b3-propagation.git
The following commit(s) were added to refs/heads/master by this push:
new ed5e76c Inlines resources hosted on ASF wiki and reverts disclaimers
(#35)
ed5e76c is described below
commit ed5e76c312ca47e76252e5f8e755bbb6e0fa9058
Author: Adrian Cole <adrianc...@users.noreply.github.com>
AuthorDate: Sun Jun 16 14:11:25 2019 +0800
Inlines resources hosted on ASF wiki and reverts disclaimers (#35)
---
DISCLAIMER | 5 --
NOTICE | 5 --
RATIONALE.md | 170 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
README.md | 2 +-
STATUS.md | 44 ++++++++++++++++
5 files changed, 215 insertions(+), 11 deletions(-)
diff --git a/DISCLAIMER b/DISCLAIMER
deleted file mode 100644
index 728b3bf..0000000
--- a/DISCLAIMER
+++ /dev/null
@@ -1,5 +0,0 @@
-Apache Zipkin B3 Propagation (incubating) is an effort undergoing incubation
at The Apache Software Foundation (ASF), sponsored by the Apache Incubator PMC.
-Incubation is required of all newly accepted projects until a further review
indicates
-that the infrastructure, communications, and decision making process have
stabilized in a manner consistent with other successful ASF projects.
-While incubation status is not necessarily a reflection of the completeness or
stability of the code,
-it does indicate that the project has yet to be fully endorsed by the ASF.
diff --git a/NOTICE b/NOTICE
deleted file mode 100644
index 0041e35..0000000
--- a/NOTICE
+++ /dev/null
@@ -1,5 +0,0 @@
-Apache Zipkin B3 Propagation (incubating)
-Copyright 2019 The Apache Software Foundation
-
-This product includes software developed at
-The Apache Software Foundation (http://www.apache.org/).
diff --git a/RATIONALE.md b/RATIONALE.md
new file mode 100644
index 0000000..9e1163f
--- /dev/null
+++ b/RATIONALE.md
@@ -0,0 +1,170 @@
+# b3 single header format
+
+In designing the Trace Context format, we made a section called tracestate
which holds the authoritative propagation data.
+
+This section defines a value that could be used as a separate "b3" header, and
would be the same value used in the w3c tracestate field. Specific to the w3c
format, this holds data not in the "traceparent" format, such as parent ID and
the debug flag. It would be a completely non-lossy way to allocate our current
headers into one value.
+
+In simplest terms it is a mapping:
+
+```
+b3={x-b3-traceid}-{x-b3-spanid}-{if x-b3-flags 'd' else
x-b3-sampled}-{x-b3-parentspanid}, where the last two fields are optional.
+```
+
+For example, the following headers:
+```
+X-B3-TraceId: 80f198ee56343ba864fe8b2a57d3eff7
+X-B3-ParentSpanId: 05e3ac9a4f6e3b90
+X-B3-SpanId: e457b5a2e4d86bd1
+X-B3-Sampled: 1
+```
+
+Become one header or state field. For example, if a header:
+```
+b3: 80f198ee56343ba864fe8b2a57d3eff7-e457b5a2e4d86bd1-1-05e3ac9a4f6e3b90
+```
+
+Or if using w3c trace context format
+```
+tracestate:
b3=80f198ee56343ba864fe8b2a57d3eff7-e457b5a2e4d86bd1-1-05e3ac9a4f6e3b90
+```
+
+Here are some more examples:
+
+A sampled root span would look like:
+```
+b3: 80f198ee56343ba864fe8b2a57d3eff7-e457b5a2e4d86bd1-1
+```
+
+A not yet sampled root span would look like:
+```
+b3: 80f198ee56343ba864fe8b2a57d3eff7-e457b5a2e4d86bd1
+```
+
+A debug RPC child span would look like:
+```
+b3: 80f198ee56343ba864fe8b2a57d3eff7-e457b5a2e4d86bd1-d-05e3ac9a4f6e3b90
+```
+
+Like normal B3, it is valid to omit trace identifiers in order to only
propagate a sampling decision. For example, the following are valid downstream
hints:
+
+* don't sample - b3: 0
+* sampled - b3: 1
+* debug - b3: d
+
+*NOTE* this does not match the prefix of traceparent, so we must define ours
independently and consider that the w3c may change in different ways. This is
ok as the "tracestate" entries in w3c format are required to be treated opaque.
In other words we can be different on purpose or by accident of drift in their
spec.
+
+## On positional encoding vs nested key/values
+
+Positional encoding is more space efficient and less complicated to parse vs
key/value encoding. For example, the AWS format code in brave is complex due to
splitting, dealing with white space etc. Positional is simple to parse and
straight-forward to map. Rationale is same as w3c traceparent for the most part.
+
+Different than new specs, we expect no additional fields. B3 is a very stable
spec and we are not defining anything new except how to encode it. For this
reason, positional should be fine.
+
+## On putting mandatory fields up front
+
+The trace ID and span ID fields are the only mandatory fields. This would
allow fixed-length parsing for those just correlating on these values. Usually
parentid is not used for correlation, rather scraping. Moreover, this is easier
for existing proxies who only create trace identifiers.
+
+Ex: you can control trace identifiers without making a sampling decision like
so:
+```
+# root trace and span IDs
+b3: 80f198ee56343ba864fe8b2a57d3eff7-e457b5a2e4d86bd1
+```
+
+## On sampled before parent span ID
+
+When name-values aren't used, it could be confusing which of the equal length
fields are the parent. By placing the sampled flag in-between, we make this
more clear. Also, it matches the prefix of the current [traceparent
encoding](https://github.com/w3c/distributed-tracing/blob/master/trace_context/HTTP_HEADER_FORMAT.md#traceparent-field).
+
+## Encoding "not yet sampled"
+
+Leaving out the single-character sampled field is how we encoded the "no
decision" state. This matches the way we used to address this (by leaving out
`X-B3-Sampled`).
+
+## Encoding debug
+
+We encode the debug flag (previously `X-B3-Flags: 1`), as the letter 'd' in
the same place as sampled ('1'). This is because debug is a boosted sampled
signal. Most implementations record it out-of-band as `Span.debug=true` to
ensure it reaches the collector tier.
+
+```
+# force trace on a root span
+b3: 80f198ee56343ba864fe8b2a57d3eff7-e457b5a2e4d86bd1-d
+```
+
+One alternative considered was adding another field just to hold debug (ex a
trailing -1). Not only was this less intuitive, it made parsing harder
especially as parentId is also optional. This was reverted in
openzipkin/brave#773
+
+## W3C drift alert
+
+While we should watch out for changes in the [TraceContext
spec](https://github.com/w3c/distributed-tracing/issues/8). For example, if
they add a "priority flag", we should keep our impl independent. B3 fields
haven't changed in years and we can lock in something far safer knowing that.
+
+## Why also define as a separate header
+
+We have had continual problems with b3 with technology like JMS. In addition
to declaring this format for w3c Trace Context, we could use it right away as
the header "b3". This would solve all the problems we have like JMS hating
hyphens in names, and allow those who opt into it a consistent format for when
they transition to w3c.
+
+openzipkin/brave#584
+
+In other words, in messaging propagation and even normal http, some libraries
could choose to read the "b3" header for the exact same format instead of
"X-B3-X"
+
+## Should we use flags instead of two fields for sampled and debug?
+
+We could encode the three sampled states and debug as a single 8-bit field
encoded as hex. If we used flags, an example sampled span would be:
+
+```
+b3: 80f198ee56343ba864fe8b2a57d3eff7-e457b5a2e4d86bd1-3-05e3ac9a4f6e3b90
+```
+
+Notice this is 3, not 1. That's because if using bit field we need to tell the
difference between unsampled and no sampling decision. This could be confusing
to people.
+
+On flags, in java, we already encode sampled and debug state internally flags
like this internally
+```java
+ static final int FLAG_SAMPLED = 1 << 1;
+ static final int FLAG_SAMPLED_SET = 1 << 2;
+ static final int FLAG_DEBUG = 1 << 3;
+
+ static Boolean sampled(int flags) {
+ return (flags & FLAG_SAMPLED_SET) == FLAG_SAMPLED_SET
+ ? (flags & FLAG_SAMPLED) == FLAG_SAMPLED
+ : null;
+ }
+
+ static boolean debug(int flags) {
+ return (flags & FLAG_DEBUG) == FLAG_DEBUG;
+ }
+}
+```
+
+This might be better off than having two fields, although it is less simple as
people often [make mistakes coding bit
fields](https://github.com/w3c/distributed-tracing/pull/116), and X-B3-Flags
caused confusion many times here including
[#20](https://github.com/openzipkin/b3-propagation/issues/20).
+
+## What about finagle's flags?
+
+If we used flags, we could also do it the same way as [finagle
does](https://github.com/twitter/finagle/blob/develop/finagle-core/src/main/scala/com/twitter/finagle/tracing/Flags.scala),
except I think it would be confusing as the length they allocated (64 bits or
16 characters in hex) was never used in practice.
+
+In practice we could use a single hex character to encode all the flags in our
format (that supports 8 flags). Also, using Finagle's flag encoding could
further confusion about the "X-B3-Flags" header, which in http encoding never
has a value besides "1"
[#20](https://github.com/openzipkin/b3-propagation/issues/20). At any rate, we
can consider using the first 8 bits of their format as prior art regardless of
if we use it.
+
+```scala
+/*
+ * The debug flag is used to ensure this the current trace passes
+ * all of the sampling stages.
+ */
+ val Debug = 1L << 0 // 1
+
+/**
+ * Reserved for future use to encode sampling behavior, currently
+ * encoded explicitly in TraceId.sampled (Option[Boolean]).
+ */
+ val SamplingKnown = 1L << 1
+ val Sampled = 1L << 2
+```
+
+## Relationship to JMS (Java Message Service)
+
+The single header format, notably the name of "b3" solves propagation concerns
a prefixed format causes with JMS.
+
+JMS requires that message header names follow java naming conventions. Notably
this excludes hyphens and dots. For example, [in
Camel](http://people.apache.org/~dkulp/camel/jms.html), there's a naming policy
to map to and from these constraints.
+
+For example, many simply downcase B3 headers to `x-b3-traceid` for use in
non-http transports (which might not be case insensitive). In JMS, this
wouldn't work, even if rabbitmq is used underneath which has no such
constraint. Ex, if using camel JMS, this header would map into
`x_HYPHEN_b3_HYPHEN_traceid`.
+
+In some cases, JMS headers are replaced based on constants or other patterns,
like globally replacing hyphens with underscores. Interestingly opentracing
decided on a [different
pattern](https://github.com/opentracing-contrib/java-jms/pull/1) `_$dash$_`,
though they have no pattern for dots: For example, if you were using the OT
library and camel on the other, you'd get `x_HYPHEN_b3_HYPHEN_traceid` on one
side and `x_$dash$_b3_$dash$_traceid` on the other, mutually unreadable.
+
+The universal implication of using JMS is that it implies global coordination
of these naming patterns, or there will be propagation incompatibility (ex
traces will restart). This will also break anything else propagated that isn't
a trace ID. This type of problem has [already been
noticed](https://github.com/spring-cloud/spring-cloud-sleuth/issues/537) in
early users of message tracing.
+
+Importantly, even if we were to have static replacements for `X-B3-` headers,
we don't know if users will introduce propagated headers with dots or hyphens
in them. In other words, a constant-based solution helps, but won't fix it.
+
+To compound issues, any mapping or heuristic approach to prefixed headers
becomes virulent and not limited to only messaging systems. Some integration
services pass headers from http directly into messaging headers. Even if it
feels tempting to make a "messaging" header name, we have to keep in mind that
many are already using header names with hyphens. If we start propagating
underscores universally, we introduce complexity to existing non-JMS consumers.
+
+For all of these reasons, a policy of only using "b3" format when using JMS is
the safest policy. That implies using prefixed headers `X-B3-` is an
antipattern.
diff --git a/README.md b/README.md
index 5aa6ae3..ab0be52 100644
--- a/README.md
+++ b/README.md
@@ -119,7 +119,7 @@ Note: Before this specification was written, some tracers
propagated `X-B3-Sampl
Debug is encoded as `X-B3-Flags: 1`. Debug implies an accept decision, so
don't also send the `X-B3-Sampled` header.
## Single Header
-A single header named `b3` standardized in late 2018 for use in JMS and w3c
`tracestate`. Design and rationale are captured
[here](https://cwiki.apache.org/confluence/display/ZIPKIN/b3+single+header+format).
Check or update our [support
page](https://cwiki.apache.org/confluence/display/ZIPKIN/b3+single+header+support)
for adoption status.
+A single header named `b3` standardized in late 2018 for use in JMS and w3c
`tracestate`. Design and rationale are captured [here](RATIONALE.md). Check or
update our [status page](STATUS.md) for adoption status.
In simplest terms `b3` maps propagation fields into a hyphen delimited string.
diff --git a/STATUS.md b/STATUS.md
new file mode 100644
index 0000000..af8dc49
--- /dev/null
+++ b/STATUS.md
@@ -0,0 +1,44 @@
+# Status of instrumentation libraries and B3 support
+
+This is a list of zipkin libraries and their status in supporting b3 features
+
+| language | library | 128-bit trace ID | b3 single format | notes |
+| ---------- | ------- | ---------------- | ---------------- | ----- |
+| javascript | [zipkin-js](https://github.com/openzipkin/zipkin-js) | v0.5+
[supported](https://github.com/openzipkin/zipkin-js/blob/a8ab73d26157a3b25b207425e2808ee39105afa1/packages/zipkin/README.md#usage)
| [unsupported](https://github.com/openzipkin/zipkin-js/issues/259) | |
+| python | [py_zipkin](https://github.com/Yelp/py_zipkin) | 0.8+
[supported](https://github.com/Yelp/py_zipkin/blob/665428305ba3cadec5e6488c801bdaca12b3e311/py_zipkin/zipkin.py#L164)
| [unsupported](https://github.com/Yelp/py_zipkin/issues/98) | |
+| java | [brave](https://github.com/openzipkin/brave) | 3.15+/4.9+
[supported/epoch128](https://github.com/openzipkin/brave/tree/master/brave-core#128-bit-trace-ids)
| 5.3+
[supported](https://github.com/apache/incubator-zipkin-brave/blob/5a17fc018613958db00cc7b6951826e95f1b9d6c/brave/src/main/java/brave/propagation/B3SingleFormat.java)
| |
+| scala | [finagle](https://github.com/twitter/finagle) | 0.40+ downgrade
| [in progress](https://github.com/twitter/finagle/pull/749) | |
+| ruby | [zipkin-ruby](https://github.com/openzipkin/zipkin-ruby) |
0.28+
[supported](https://github.com/openzipkin/zipkin-ruby/blob/ec875f3640dc2ce8070969f3b0c889d7b7121063/README.md#configuration-options)
| [unsupported](https://github.com/openzipkin/zipkin-ruby/issues/126) | |
+| go | [zipkin-go](https://github.com/openzipkin/zipkin-go) | 0.1+
[supported](https://godoc.org/github.com/openzipkin/zipkin-go#WithTraceID128Bit)
| 0.1+
[supported](https://godoc.org/github.com/openzipkin/zipkin-go/propagation/b3#InjectOption)
| |
+| go |
[zipkin-go-opentracing](https://github.com/openzipkin/zipkin-go-opentracing) |
0.2+
[supported](https://godoc.org/github.com/openzipkin/zipkin-go-opentracing#TraceID128Bit)
| unsupported | |
+| php | [zipkin-php](https://github.com/openzipkin/zipkin-php) | 1+
[supported](https://github.com/openzipkin/zipkin-php/blob/b143bf577b5b328df656dede4f6b8aeec38b201a/src/Zipkin/TracingBuilder.php#L122)
| [unsupported](https://github.com/openzipkin/zipkin-php/issues/92) | |
+| java | [wingtips](https://github.com/Nike-Inc/wingtips) | 0.11.2+
transparent | [unsupported](https://github.com/Nike-Inc/wingtips/issues/80) | |
+| java | [jaeger](https://github.com/uber/jaeger-client-java) | 0.10+
downgrade | unsupported | |
+| csharp | [zipkin4net](https://github.com/openzipkin/zipkin4net) | 0.4+
supported |
[supported](https://github.com/openzipkin/zipkin4net/blob/c30d8244cad5c219f9b891a513b0032f0047a2fd/Src/zipkin4net/Src/Propagation/B3SingleFormat.cs)
| |
+
+## 128-bit trace ID
+
+A 128-bit trace ID is when `X-B3-TraceId` has 32 hex characters as opposed to
16. For example, `X-B3-TraceId: 463ac35c9f6413ad48485a3953bb6124`.
+
+Here are the status options for 128bit `X-B3-TraceId`:
+* unsupported: 32 character trace ids will break the library
+* downgrade: Read 32 character trace ids by throwing away the high bits (any
characters left of 16 characters). This effectively downgrades the ID to 64
bits.
+* transparent: Pass `X-B3-TraceId` through the system without interpreting it.
+* supported: Can start traces with 128-bit trace IDs, propagates and reports
them as 128-bits
+* epoch128: Can start traces with 128-bit trace IDs which are prefixed with
epoch seconds
+
+### epoch128
+When a trace ID is 128-bits and the first 32 bits are epoch seconds, the ID
can be translated into
+an Amazon Web Services ID. Tracers who do this can tunnel through ELBs, for
example.
+
+Here's an example implementation
+```java
+ static long nextTraceIdHigh(Random prng) {
+ long epochSeconds = System.currentTimeMillis() / 1000;
+ int random = prng.nextInt();
+ return (epochSeconds & 0xffffffffL) << 32
+ | (random & 0xffffffffL);
+ }
+```
+
+See openzipkin/zipkin#1754
