Re: spark on k8s daemonset collect log

The filebeat supports multiline matching, here is an example[1] BTW, I’m working on External Log Service integration[2], it may be useful in your case, feel free to review/left comments [1] https://www.elastic.co/guide/en/beats/filebeat/current/multiline-examples.html#multiline [2]

spark on k8s daemonset collect log

hi, all Spark runs on k8s, uses daemonset filebeat to collect logs, and writes them to elasticsearch. The docker logs are in json format, and each line is a json string. How to merge multi-line exceptions?