Nothing is written to error indexing topic on es. I could seethe same timeout error in storm.can you please help which property I need to change for this.
Error 2019-09-05 18:43:31.622 o.a.s.d.executor Thread-12-parserBolt-executor[5 5] [ERROR] java.util.concurrent.ExecutionException: org.apache.kafka.common.errors.TimeoutException: Failed to update metadata after 60000 ms. at org.apache.kafka.clients. producer.KafkaProducer$FutureFailure.<init>(KafkaProducer.java:730) ~[stormjar.jar:?] at org.apache.kafka.clients.producer.KafkaProducer.doSend(KafkaProducer.java:483) ~[stormjar.jar:?] at org.apache.kafka.clients.producer.KafkaProducer.send(KafkaProducer.java:430) ~[stormjar.jar:?] at org.apache.kafka.clients.producer.KafkaProducer.send(KafkaProducer.java:353) ~[stormjar.jar:?] at org.apache.metron.writer.kafka.KafkaWriter.write(KafkaWriter.java:257) ~[stormjar.jar:?] at org.apache.metron.writer.BulkWriterComponent.flush(BulkWriterComponent.java:123) [stormjar.jar:?] at org.apache.metron.writer.BulkWriterComponent. applyShouldFlush(BulkWriterComponent.java:179) [stormjar.jar:?] at org.apache.metron.writer.BulkWriterComponent.write(BulkWriterComponent.java:99) [stormjar.jar:?] at org.apache.metron.parsers.bolt.WriterHandler.write(WriterHandler.java:90) [stormjar.jar:?] at org.apache.metron.parsers.bolt.ParserBolt.execute(ParserBolt.java:269) [stormjar.jar:?] at org.apache.storm.daemon.executor$fn__10195$tuple_ action_fn__10197.invoke(executor.clj:735) [storm-core-1.1.0.2.6.5.1175-1.jar:1.1.0.2.6.5.1175-1] at org.apache.storm.daemon.executor$mk_task_receiver$fn__10114.invoke(executor.clj:466) [storm-core-1.1.0.2.6.5.1175-1.jar:1.1.0.2.6.5.1175-1] at org.apache.storm.disruptor$clojure_handler$reify__4137.onEvent(disruptor.clj:40) [storm-core-1.1.0.2.6.5.1175-1.jar:1.1.0.2.6.5.1175-1] at org.apache.storm.utils.DisruptorQueue.consumeBatchToCursor(DisruptorQueue.java:472) [storm-core-1.1.0.2.6.5.1175-1.jar:1.1.0.2.6.5.1175-1] at org.apache.storm.utils.DisruptorQueue.consumeBatchWhenAvailable(DisruptorQueue.java:451) [storm-core-1.1.0.2.6.5.1175-1.jar:1.1.0.2.6.5.1175-1] at org.apache.storm.disruptor$consume_batch_when_available.invoke(disruptor.clj:73) [storm-core-1.1.0.2.6.5.1175-1.jar:1.1.0.2.6.5.1175-1] at org.apache.storm.daemon.executor$fn__10195$fn__10208$ fn__10263.invoke(executor.clj:855) [storm-core-1.1.0.2.6.5.1175-1.jar:1.1.0.2.6.5.1175-1] at org.apache.storm.util$async_loop$fn__1221.invoke(util.clj:484) [storm-core-1.1.0.2.6.5.1175-1.jar:1.1.0.2.6.5.1175-1] at clojure.lang.AFn.run(AFn.java:22) [clojure-1.7.0.jar:?] at java.lang.Thread.run(Thread.java:745) [?:1.8.0_112] Caused by: org.apache.kafka.common. On Thu, 5 Sep, 2019, 9:24 PM Michael Miklavcic, <[email protected]> wrote: > Also look in the error indexing topic if Solr or ES. And there should also > be data written to HDFS, even in case of error. hdfs dfs -ls > /apps/metron/indexing > > On Wed, Sep 4, 2019 at 10:17 AM Hema malini <[email protected]> > wrote: > >> Hi, >> >> I am using Metron 0.7.2. I have setup of three node cluster in centos 7. >> I am sending syslog and bro logs to corresponding Kafka topic but it is not >> reaching the bro or syslog storm topology. I could see the messages in the >> consumer of kafka. But itis neither reaching the enrichment or bro nor >> syslog3164. I couldn't find any error logs in storm(worker -artifacts not >> updated with any logs) nor no error messages in Kafka. Can someone please >> help to debug this issue. I could see only syslog5674 topic created. I >> created manually the other syslog3164 Kafka topic. I was able to send logs >> to syslog3164 and I could seethe messages in Kafka consumer. But nothing >> has reached storm.can someone please help. >> >> Thanks and regards, >> Hema >> >
