wuxingye commented on issue #3365: add logstash logback plugin URL: https://github.com/apache/skywalking/pull/3365#issuecomment-525988229 # Add logstash logback plugin ## 1.user guide - Dependency the toolkit, such as using maven or gradle ```xml <dependency> <groupId>org.apache.skywalking</groupId> <artifactId>apm-toolkit-logback-1.x</artifactId> <version>${skywalking.version}</version> </dependency> ``` - set `LogstashEncoder` of logback.xml ```xml <encoder charset="UTF-8" class="net.logstash.logback.encoder.LogstashEncoder"> <provider class="org.apache.skywalking.apm.toolkit.log.logback.v1.x.logstash.TraceIdJsonProvider"> </provider> <customFields>{"app_id":"${app_id}"}</customFields> </encoder> ``` ## 2.why it works - extension of logstash-logback-encoder provider, [official guide](https://github.com/logstash/logstash-logback-encoder#custom-json-provider) - Custom LogstashEncoder JSON Provider like this: ```xml <encoder class="net.logstash.logback.encoder.LogstashEncoder"> ... <provider class="your.provider.YourJsonProvider"> <!-- Any properties exposed by your provider can be set here --> </provider> ... </encoder> ``` - TraceIdJsonProvider analyze ```java public class TraceIdJsonProvider extends AbstractFieldJsonProvider<ILoggingEvent> implements FieldNamesAware<LogstashFieldNames> { public static final String TRACING_ID = "TID"; @Override public void writeTo(JsonGenerator generator, ILoggingEvent event) throws IOException { Map<String, String> map = event.getLoggerContextVO().getPropertyMap(); JsonWritingUtils.writeStringField(generator, getFieldName(), map.get(TRACING_ID)); } @Override public void setFieldNames(LogstashFieldNames fieldNames) { setFieldName(TRACING_ID); } } ``` 1. the setFieldNames method will add a json field named `TID` 2. the writeTo method will set the `TID` value, for `propertyMap` is always not null, it at least contains `destination` of logstash, so the code will not throw `NPE` - TcpSocketAppenderActivation analyze ```java public class TcpSocketAppenderActivation extends ClassInstanceMethodsEnhancePluginDefine { public static final String INTERCEPT_CLASS = "org.apache.skywalking.apm.toolkit.activation.log.logback.v1.x.logstash.TcpSocketAppenderInterceptor"; public static final String ENHANCE_CLASS = "net.logstash.logback.appender.LogstashTcpSocketAppender"; public static final String ENHANCE_METHOD = "prepareForDeferredProcessing"; /** * @return the target class, which needs active. */ @Override protected ClassMatch enhanceClass() { return byName(ENHANCE_CLASS); } /** * @return null, no need to intercept constructor of enhance class. */ @Override public ConstructorInterceptPoint[] getConstructorsInterceptPoints() { return null; } /** * @return the collection of {@link StaticMethodsInterceptPoint}, represent the intercepted methods and their * interceptors. */ @Override public InstanceMethodsInterceptPoint[] getInstanceMethodsInterceptPoints() { return new InstanceMethodsInterceptPoint[] { new InstanceMethodsInterceptPoint() { @Override public ElementMatcher<MethodDescription> getMethodsMatcher() { return named(ENHANCE_METHOD).and(takesArgumentWithType(0, "ch.qos.logback.classic.spi.ILoggingEvent")); } @Override public String getMethodsInterceptor() { return INTERCEPT_CLASS; } @Override public boolean isOverrideArgs() { return false; } } }; } } ``` 1. net.logstash.logback.appender.`LogstashTcpSocketAppender` is logstash appender, contains a method: ```java @Override protected void prepareForDeferredProcessing(final ILoggingEvent event) { super.prepareForDeferredProcessing(event); if (includeCallerData) { event.getCallerData(); } } ``` 2. so we enhance it by the `beforeMethod` of `TcpSocketAppenderInterceptor` : ```java @Override public void beforeMethod(EnhancedInstance objInst, Method method, Object[] allArguments, Class<?>[] argumentsTypes, MethodInterceptResult result) throws Throwable { ILoggingEvent event = (ILoggingEvent)allArguments[0]; if (event != null && event.getLoggerContextVO() != null && event.getLoggerContextVO().getPropertyMap() != null) { event.getLoggerContextVO().getPropertyMap().put("TID", ContextManager.getGlobalTraceId()); } } ``` 3. it put a key value of `(TID, ContextManager.getGlobalTraceId())` to the `ILoggingEvent`, so we can get it in the `writeTo` method of `TraceIdJsonProvider` 4. the method `prepareForDeferredProcessing` of `LogstashTcpSocketAppender` works on the worker thread, but others works on the logstash thread, because `logstash-logback-encoder` if a async plugin, so we enhance it by this way ## 3.related pictures - ConsoleAppender plugin  - logstash logback plugin  - skywalking ui  - before add `TID`  - after add `TID` 
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
