Hi Stefan Thanks for the quick response. I am using Metron version 0.6.0.1.7.1.0 (https://link.getmailspring.com/link/750f1435-c1ca-4f45-b67f-3a275105c...@getmailspring.com/0?redirect=0.6.0.1.7.1.0&recipient=dXNlckBtZXRyb24uYXBhY2hlLm9yZw%3D%3D) Here is my parser config { "parserClassName": "org.apache.metron.parsers.csv.CSVParser", "filterClassName": null, "sensorTopic": "test_syslog", "outputTopic": null, "errorTopic": null, "writerClassName": null, "errorWriterClassName": null, "readMetadata": false, "mergeMetadata": false, "numWorkers": null, "numAckers": null, "spoutParallelism": 1, "spoutNumTasks": 1, "parserParallelism": 1, "parserNumTasks": 1, "errorWriterParallelism": 1, "errorWriterNumTasks": 1, "spoutConfig": {}, "securityProtocol": null, "stormConfig": {}, "parserConfig": { "columns": { "Id": 0, "Name": 1, "Age": 2 } }, "fieldTransformations": [], "cacheConfig": {}, "rawMessageStrategy": "ENVELOPE", "rawMessageStrategyConfig": { "messageField": "data", "readMetadata": "true", "mergeMetadata": "true" } }
I have been trying to do this for a couple of days. Tried it on multiple different parsers after studying documentation. At first I was using a parser chaining usecase. But now I wrote a simple parser to test the problem. Set the readMetadata and mergeMetadata fields to true just in case. It just doesn't work. Regards, Mustafa Akmal On Feb 27 2019, at 6:49 pm, Stefan Kupstaitis-Dunkler <stefan....@gmail.com> wrote: > Hi Mustafa, > > can you verify if the "mergeMetaData" property in the parser json is set to > "true". > If this property is set to false, other fields won't be merged as meta data. > > If this property is not set, it should default to true for the message > strategy "ENVELOPE". > > Any other behaviour is probably a bug. > > Also verify if you set "rawMessageStrategy" to "true" for the same reasons. > > > Best, > > Stefan > > > On Wed, Feb 27, 2019 at 5:34 AM Mustafa Akmal <mustafa.ak...@abcdata.org > (mailto:mustafa.ak...@abcdata.org)> wrote: > > Hello > > I am using a CSV parser. I have the following log > > {"data": "1,john,23","AdditionField": "abcd","AdditionField2": "12345"} > > > > Now I have set the raw message strategy to 'ENVELOPE' and the messageField > > to 'data' > > However after the record is indexed in elasticsearch, the parser does parse > > the value inside data but it does not get the additional fields as shown in > > the original log that is 'AdditionField' and 'AdditionField2'. What am I > > doing wrong? Can anyone help? > > Thanks! > > > > > > Virus-free. www.avg.com > > (http://www.avg.com/email-signature?utm_medium=email&utm_source=link&utm_campaign=sig-email&utm_content=emailclient) > > -- > Stefan Kupstaitis-Dunkler > > https://datahovel.com/ > (https://link.getmailspring.com/link/750f1435-c1ca-4f45-b67f-3a275105c...@getmailspring.com/1?redirect=https%3A%2F%2Fdatahovel.com%2F&recipient=dXNlckBtZXRyb24uYXBhY2hlLm9yZw%3D%3D) > https://twitter.com/StefanDunkler > (https://link.getmailspring.com/link/750f1435-c1ca-4f45-b67f-3a275105c...@getmailspring.com/2?redirect=https%3A%2F%2Ftwitter.com%2FStefanDunkler&recipient=dXNlckBtZXRyb24uYXBhY2hlLm9yZw%3D%3D) > > > > --- This email has been checked for viruses by AVG. https://www.avg.com
