If you had the all active logs set to true it should send everything. What is the latest commit of the version of plugin are you running? I see it's 0.3 but since that hasn't been "released" (tagged) I'm assuming you are installing from master?
Jon Zeolla On Wed, Jul 3, 2019, 5:57 PM Sanket Sharma <sanket.sha...@dukstra.com> wrote: > Seems like all I had to do was to specify the exact logs that I wanted to > export. All working now. > > > > Thanks for the help @Jon Zeolla > > > > > > Best regards, > > Sanket > > > > > > *From: *Sanket Sharma <sanket.sha...@dukstra.com> > *Reply-To: *"user@metron.apache.org" <user@metron.apache.org> > *Date: *Wednesday, 03 July 2019 at 19:47 > *To: *"user@metron.apache.org" <user@metron.apache.org> > *Subject: *Re: metron-bro-plugin-kafka error > > > > Okay, I figured it out. There was a mismatch in my install bro (yum > installed), the source (git cloned) and the plugin version. I removed > everything and them compiled both zeek and the plugin from source and the > issue seems to have gone. I can run the test command I get the following > output. > > > > # zeek -N Apache::Kafka > > Apache::Kafka - Writes logs to Kafka (dynamic, version 0.3.0) > > > > However, now I can't seem to get alerts/logs to Kafka. Here's the config > I'm using in /usr/local/zeek/share/zeek/site/local.zeek > > > > #This doesn't work in the new version anymore. > > #@load packages/metron-bro-plugin-kafka/Apache/Kafka > > > > #Tried added this line to ensure all packages are automatically loaded. > > #@load packages > > > > #Then tried loading the specific module > > #@load metron-bro-plugin-kafka > > #And then I eventually removed the three previous load lines > > > > redef Kafka::send_all_active_logs = T; > > redef Kafka::tag_json = T; > > redef Kafka::kafka_conf = table( > > ["metadata.broker.list"] = "mysecrethost:6667", > > ["client.id"] = "bro" > > ); > > > > Even when I have the `@loads` disabled, I still see the script being > loaded (see logs below). > > > > To start, I did the following: > > > > zeekctl> deploy > > zeekctl> restart --clean > > zeekctl> start > > > > I can see the following in startup logs: > > > > starting ... > > starting zeek ... > > [ZeekControl] > diag > > [zeek] > > > > No core file found. > > > > Zeek 2.6-558 > > Linux 3.10.0-957.21.3.el7.x86_64 > > > > Zeek plugins: > > Apache::Kafka - Writes logs to Kafka (dynamic, version 0.3.0) > > > > ==== No reporter.log > > > > ==== stderr.log > > listening on em1 > > > > > > ==== stdout.log > > max memory size (kbytes, -m) unlimited > > data seg size (kbytes, -d) unlimited > > virtual memory (kbytes, -v) unlimited > > core file size (blocks, -c) unlimited > > > > ==== .cmdline > > -i em1 -U .status -p zeekctl -p zeekctl-live -p standalone -p local -p > zeek local.zeek zeekctl zeekctl/standalone zeekctl/auto > > > > ==== .env_vars > > > PATH=/usr/local/zeek/bin:/usr/local/zeek/share/zeekctl/scripts:/usr/local/openssl/bin:/opt/apache-maven-3.3.9/bin:/sbin:/bin:/usr/sbin:/usr/bin:/usr/local/zeek/bin > > > ZEEKPATH=/usr/local/zeek/spool/installed-scripts-do-not-touch/site::/usr/local/zeek/spool/installed-scripts-do-not-touch/auto:/usr/local/zeek/share/zeek:/usr/local/zeek/share/zeek/policy:/usr/local/zeek/share/zeek/site > > CLUSTER_NODE= > > > > ==== .status > > RUNNING [net_run] > > > > ==== No prof.log > > > > ==== packet_filter.log > > #separator \x09 > > #set_separator , > > #empty_field (empty) > > #unset_field - > > #path packet_filter > > #open 2019-07-03-19-36-56 > > #fields ts node filter init success > > #types time string string bool bool > > 1562175416.590048 zeek ip or not ip T T > > > > ==== loaded_scripts.log > > #separator \x09 > > #set_separator , > > #empty_field (empty) > > #unset_field - > > #path loaded_scripts > > #open 2019-07-03-19-36-56 > > #fields name > > #types string > > /usr/local/zeek/lib/zeek/plugins/APACHE_KAFKA/lib/bif/__load__.zeek > > /usr/local/zeek/lib/zeek/plugins/APACHE_KAFKA/lib/bif/kafka.bif.zeek > > /usr/local/zeek/lib/zeek/plugins/APACHE_KAFKA/scripts/__load__.bro > > /usr/local/zeek/lib/zeek/plugins/APACHE_KAFKA/scripts/init.bro > > > > > > it starts up fine, no error messages. running "diag" in zeekctl just gives > a long list of plugins that were loaded. > > > > If I tail logs in I see new connection logs being added. However, I dont > see any messages in Kafka console consumer. What am I missing? How do I go > about debugging this? > > > > Thank you for your help and assistance. > > > > Best regards, > > Sanket > > > > > > > > > > > ------------------------------ > > *From:* zeo...@gmail.com <zeo...@gmail.com> > *Sent:* Tuesday, July 2, 2019 11:46 AM > *To:* user@metron.apache.org > *Subject:* Re: metron-bro-plugin-kafka error > > > > Did you install it manual or with bro-pkg/zkg? I believe bro-pkg was > renamed to zkg as of their 2.0 release but I haven't used it in a little > while. Any more details regarding the installation process, or versions of > software in use may be helpful > > Jon Zeolla > > > > On Tue, Jul 2, 2019, 12:26 AM Sanket Sharma <sanket.sha...@dukstra.com> > wrote: > > Hi, > > > > I’m trying to configure Metron bro plugin by following instructions here: > https://github.com/apache/metron-bro-plugin-kafka > > > > > > I’m able to build/install the plugin successfully but when I test it using > the command: > > > > $ bro -N Apache::Kafka > > > > > > I get the following error: > > > > fatal error in /opt/bro/share/bro/base/init-bare.bro, line 1: cannot load > plugin library /opt/bro/lib/bro/plugins/APACHE_KAFKA//lib/ > APACHE-KAFKA.linux-x86_64.so: /opt/bro/lib/bro/plugins/APACHE_KAFKA//lib/ > APACHE-KAFKA.linux-x86_64.so: undefined symbol: > bro_version_2_6_558_plugin_7 > > > > Not sure what am I missing? Any help would be greatly appreciated. > > > > > > Best regards, > > Sanket > > > > > >
