Hi Rayees,
which version of Graylog and Elasticsearch are you running?
Cheers,
Jochen
On Saturday, 18 February 2017 01:46:16 UTC+1, Rayees Namathponnan wrote:
>
> Hi All,
>
> I created “Copy Input” extractor to get key value pair, here is my
> message, trying to extract *level* and *status* from all the messages
>
>
>
> 2016-09-28 19:21:52,466 level=INFO tag="run_workflow.py" msg="Run complete
> for appname=cils, job_date=20160912, status=Passed starttime=Wed Sep 28
> 19:15:25 2016, endtime=Wed Sep 28 19:21:47 2016, duration=0:6:21,
> inputs=[{"path": "/esss/srg/20160912_1473688239855_4f29bbb6efdb3c39",
> "tag": "cilPurge", "stats": {"size": "13.47MB"}}, {"path":
> "/compressed/cil/20160912", "tag": "cil", "stats": {"size": "580.16MB"}}],
> outputs=[{"path": "/processed/test/parse//cil", "tag": "cil.output.folder",
> "stats": {"diffSize": "645.78MB", "newFiles": [], "endSize": "18.93GB",
> "startSize": "18.30GB"}}]”
>
>
>
> Its failing with below error
>
> 0]: index [graylog_8], type [message], id
> [c0e7ea80-f572-11e6-b21e-5254007b267d], message
> [java.lang.IllegalArgumentException: Document contains at least one immense
> term in field="level" (whose UTF8 encoding is longer than the max length
> 32766), all of which were skipped. Please correct the analyzer to not
> produce such terms. The prefix of the first immense term is: '[70, 101,
> 98, 32, 49, 55, 32, 49, 57, 58, 51, 57, 58, 51, 53, 32, 115, 106, 101, 108,
> 107, 51, 51, 32, 115, 121, 115, 116, 101, 109]...', original message: bytes
> can be at most 32766 in length; got 34944]
> 2017-02-17T19:39:56.795-05:00 ERROR [Extractor] Could not apply converter
> [tokenizer] of extractor [77e451d0-f3b9-11e6-b21e-5254007b267d].
> java.lang.IllegalArgumentException: Multiple entries with same key:
> id=4038, and id=4038,
> at
> com.google.common.collect.ImmutableMap.checkNoConflict(ImmutableMap.java:136)
> ~[graylog.jar:?]
> at
> com.google.common.collect.RegularImmutableMap.checkNoConflictInKeyBucket(RegularImmutableMap.java:98)
>
> ~[graylog.jar:?]
> at
> com.google.common.collect.RegularImmutableMap.fromEntryArray(RegularImmutableMap.java:84)
>
> ~[graylog.jar:?]
> at
> com.google.common.collect.ImmutableMap$Builder.build(ImmutableMap.java:295)
> ~[graylog.jar:?]
> at
> org.graylog2.inputs.converters.TokenizerConverter.convert(TokenizerConverter.java:55)
>
> ~[graylog.jar:?]
> at org.graylog2.plugin.inputs.Extractor.runConverters(Extractor.java:242)
> [graylog.jar:?]
> at org.graylog2.plugin.inputs.Extractor.runExtractor(Extractor.java:228)
> [graylog.jar:?]
> at org.graylog2.filters.ExtractorFilter.filter(ExtractorFilter.java:73)
> [graylog.jar:?]
> at
> org.graylog2.messageprocessors.MessageFilterChainProcessor.process(MessageFilterChainProcessor.java:100)
>
> [graylog.jar:?]
> at
> org.graylog2.buffers.processors.ServerProcessBufferProcessor.handleMessage(ServerProcessBufferProcessor.java:56)
>
> [graylog.jar:?]
> at
> org.graylog2.shared.buffers.processors.ProcessBufferProcessor.dispatchMessage(ProcessBufferProcessor.java:82)
>
> [graylog.jar:?]
> at
> org.graylog2.shared.buffers.processors.ProcessBufferProcessor.onEvent(ProcessBufferProcessor.java:61)
>
> [graylog.jar:?]
> at
> org.graylog2.shared.buffers.processors.ProcessBufferProcessor.onEvent(ProcessBufferProcessor.java:35)
>
> [graylog.jar:?]
> at com.lmax.disruptor.WorkProcessor.run(WorkProcessor.java:143)
> [graylog.jar:?]
> at
> com.codahale.metrics.InstrumentedThreadFactory$InstrumentedRunnable.run(InstrumentedThreadFactory.java:66)
>
> [graylog.jar:?]
> at java.lang.Thread.run(Thread.java:745) [?:1.8.0_111]
> 2017-02-17T19:39:57.677-05:00 ERROR [Messages] Failed to index [1]
> messages. Please check the index error log in your web interface for the
> reason. Error: failure in bulk execution:
> [0]: index [graylog_8], type [message], id
> [c560a160-f572-11e6-b21e-5254007b267d], message
> [java.lang.IllegalArgumentException: Document contains at least one immense
> term in field="level" (whose UTF8 encoding is longer than the max length
> 32766), all of which were skipped. Please correct the analyzer to not
> produce such terms. The prefix of the first immense term is: '[70, 101,
> 98, 32, 49, 55, 32, 49, 57, 58, 51, 57, 58, 52, 53, 32, 115, 106, 101, 108,
> 107, 51, 51, 32, 115, 121, 115, 116, 101, 109]...', original message: bytes
> can be at most 32766 in length; got 34944]
> ^Z
>
--
You received this message because you are subscribed to the Google Groups
"Graylog Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to graylog2+unsubscr...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/graylog2/013d028e-f053-4b5f-8da6-18ded9bdd073%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
