Thank you, Nick and Stephane. Greatly appreciated! I will post the test result.
Yu Zhang Security Engineer - Big Data Virtualization and Security GM | Global Infrastructure [email protected]<mailto:[email protected]> C (303) 503-5481 From: [email protected] <[email protected]> Sent: Friday, January 24, 2020 1:44 AM To: [email protected] Subject: [EXTERNAL] RE: How to configure Rsyslog omkafka to send log to kafka topic with Kerberos Hello, Here is a piece of configuration: action(type="omkafka" name="xxxx" broker=[list of kafka brokers] partitions.auto="on" topic="your topic" confParam=["security.protocol=SASL_PLAINTEXT", "sasl.mechanism=GSSAPI", "sasl.kerberos.service.name=kafka", "sasl.kerberos.principal=your principal", "sasl.kerberos.keytab=/etc/security/keytabs/your keytab", "sasl.kerberos.kinit.cmd=/usr/bin/kinit -S %{sasl.kerberos.service.name}/%{broker.name} -t %{sasl.kerberos.keytab} -k %{sasl.kerberos.principal}"] ) Unfortunately, it doesn’t work when I tested it a few month ago because of a bug in rsyslog. I’ve lost the error message but when I got it and googled on it, I found some discussion about a known bug on rsyslog side. May be it is fixed now. Stéphane From: Nick Allen [mailto:[email protected]] Sent: Thursday, January 23, 2020 21:25 To: [email protected]<mailto:[email protected]> Subject: Re: How to configure Rsyslog omkafka to send log to kafka topic with Kerberos After reading a bit, I can see that you will want to use ConfParam<https://www.rsyslog.com/doc/master/configuration/modules/omkafka.html#confparam>. Per those docs, it looks like omkafka uses librdkafka under the hood. Fortunately, I am familiar with librdkafka. Here are the available settings for librdkafka<https://github.com/apache/metron/tree/master/metron-sensors/fastcapa#kerberos>. You might also be interested in some documentation for Fastcapa (which is a packet capture mechanism in Metron.) Fastcapa also uses librdkafka under-the-hood and we have documentation which describes how to make that work with kerberos<https://github.com/apache/metron/tree/master/metron-sensors/fastcapa#kerberos>. The configuration that you need will most likely be very similar. Hope this helps. On Thu, Jan 23, 2020 at 3:13 PM Nick Allen <[email protected]<mailto:[email protected]>> wrote: I do not have familiarity with omkafka, but you need to pass some Kafka-specific configuration options when using Kerberos. These links might help you understand what Kafka configuration options are needed. Then you would just need to determine how to make those adjustments with omkafka. * https://metron.apache.org/current-book/metron-deployment/Kerberos-manual-setup.html#Push_Data * https://docs.cloudera.com/HDPDocuments/HDP3/HDP-3.1.5/authentication-with-kerberos/content/kerberos_kafka_producing_events_or_messages_to_kafka_on_a_secured_cluster.html * https://kafka.apache.org/documentation/#producerconfigs On Thu, Jan 23, 2020 at 2:56 PM Yu Zhang <[email protected]<mailto:[email protected]>> wrote: Hi, I am doing a Metron PoC. Now, I can send rsyslog records to Kafka topic without Kerberos. How to configure Rsyslog omkafka to send log to kafka topic with Kerberos? Thanks, Yu Zhang Security Engineer - Big Data Virtualization and Security GM | Global Infrastructure [email protected]<mailto:[email protected]> C (303) 503-5481 Nothing in this message is intended to constitute an electronic signature unless a specific statement to the contrary is included in this message. Confidentiality Note: This message is intended only for the person or entity to which it is addressed. It may contain confidential and/or privileged material. Any review, transmission, dissemination or other use, or taking of any action in reliance upon this message by persons or entities other than the intended recipient is prohibited and may be unlawful. If you received this message in error, please contact the sender and delete it from your computer. Nothing in this message is intended to constitute an electronic signature unless a specific statement to the contrary is included in this message. Confidentiality Note: This message is intended only for the person or entity to which it is addressed. It may contain confidential and/or privileged material. Any review, transmission, dissemination or other use, or taking of any action in reliance upon this message by persons or entities other than the intended recipient is prohibited and may be unlawful. If you received this message in error, please contact the sender and delete it from your computer.
