Has anyone managed to integrate Camel logs with Logstash/Elasticsearch? I've got tracer and MDC logging enabled in my CamelContext and am sending logs to Logstash via log4j TCP appender. This is working great (especially MDC fields), but I'm trying to further parse and filter the message to individual fields in Elasticsearch.
For example, my message is: ID-dx123abc001-abc-com-42264-1475782902034-0-193168 >>> (system-sql) setHeader[Content-Type, application/json] --> marshal[org.apache.camel.component.jackson.JacksonDataFormat@7e38f55] <<< Pattern:InOut, Headers:{Accept-Encoding=gzip, deflate, Authorization=Basic aaaaXRpeDpreG1pZGFz, breadcrumbId=ID-dx123abc001-abc-com-42264-1475782902034-0-193169, CamelHttpMethod=GET, CamelHttpPath=, CamelHttpQuery=name=new-strategies, CamelHttpServletRequest=Request(GET /esb/system/sql?name=new-strategies)@6aaa8b00, CamelHttpServletResponse=HTTP/1.1 200 , CamelHttpUri=/esb/system/sql, CamelHttpUrl=http://system.abc.com:51950/esb/system/sql, CamelJdbcColumnNames=[STRATEGY_NAME], CamelJdbcRowCount=2, CamelServletContextPath=/esb/system/sql, Connection=keep-alive, Content-Type=application/json, Host=system.abc.com:51950, name=new-strategies, User-Agent=Apache-HttpClient/4.2.1 (java 1.5)}, BodyType:java.util.ArrayList, Body:[{STRATEGY_NAME=S1 }, {STRATEGY_NAME=S2 }] And I would like logstash to parse and filter this so that it is stored in Elasticsearch as individual fields: ... from=setHeader[Content-Type, application/json] to=marshal[org.apache.camel.component.jackson.JacksonDataFormat@7e38f55] breadcrumbId=ID-dx123abc001-abc-com-42264-1475782902034-0-193169 CamelJdbcColumnNames=[STRATEGY_NAME] CamelJdbcRowCount=2 BodyType:java.util.ArrayList Body:[{STRATEGY_NAME=S1 }, {STRATEGY_NAME=S2 }] etc. Has anyone been able to do this? I'm struggling with getting this to work with grok or ruby in the logstash config. Any tips are appreciated! Thanks -- View this message in context: http://camel.465427.n5.nabble.com/Camel-Tracer-and-Logstash-tp5788502.html Sent from the Camel - Users mailing list archive at Nabble.com.