Hello,
Here is a piece of configuration:
action(type="omkafka" name="xxxx" broker=[list of kafka brokers]
partitions.auto="on" topic="your topic"
confParam=["security.protocol=SASL_PLAINTEXT",
"sasl.mechanism=GSSAPI",
"sasl.kerberos.service.name=kafka",
"sasl.kerberos.principal=your
principal",
"sasl.kerberos.keytab=/etc/security/keytabs/your keytab",
"sasl.kerberos.kinit.cmd=/usr/bin/kinit -S
%{sasl.kerberos.service.name}/%{broker.name} -t %{sasl.kerberos.keytab} -k
%{sasl.kerberos.principal}"]
)
Unfortunately, it doesn’t work when I tested it a few month ago because of a
bug in rsyslog. I’ve lost the error message but when I got it and googled on
it, I found some discussion about a known bug on rsyslog side. May be it is
fixed now.
Stéphane
From: Nick Allen [mailto:[email protected]]
Sent: Thursday, January 23, 2020 21:25
To: [email protected]
Subject: Re: How to configure Rsyslog omkafka to send log to kafka topic with
Kerberos
After reading a bit, I can see that you will want to use ConfParam
<https://www.rsyslog.com/doc/master/configuration/modules/omkafka.html#confparam>
. Per those docs, it looks like omkafka uses librdkafka under the hood.
Fortunately, I am familiar with librdkafka. Here are the available settings
for librdkafka
<https://github.com/apache/metron/tree/master/metron-sensors/fastcapa#kerberos>
.
You might also be interested in some documentation for Fastcapa (which is a
packet capture mechanism in Metron.) Fastcapa also uses librdkafka
under-the-hood and we have documentation which describes how to make that work
with kerberos
<https://github.com/apache/metron/tree/master/metron-sensors/fastcapa#kerberos>
. The configuration that you need will most likely be very similar.
Hope this helps.
On Thu, Jan 23, 2020 at 3:13 PM Nick Allen <[email protected]> wrote:
I do not have familiarity with omkafka, but you need to pass some
Kafka-specific configuration options when using Kerberos.
These links might help you understand what Kafka configuration options are
needed. Then you would just need to determine how to make those adjustments
with omkafka.
*
https://metron.apache.org/current-book/metron-deployment/Kerberos-manual-setup.html#Push_Data
*
https://docs.cloudera.com/HDPDocuments/HDP3/HDP-3.1.5/authentication-with-kerberos/content/kerberos_kafka_producing_events_or_messages_to_kafka_on_a_secured_cluster.html
* https://kafka.apache.org/documentation/#producerconfigs
On Thu, Jan 23, 2020 at 2:56 PM Yu Zhang <[email protected]> wrote:
Hi,
I am doing a Metron PoC. Now, I can send rsyslog records to Kafka topic without
Kerberos. How to configure Rsyslog omkafka to send log to kafka topic with
Kerberos?
Thanks,
Yu Zhang
Security Engineer - Big Data Virtualization and Security
GM | Global Infrastructure
<mailto:[email protected]> [email protected]
C (303) 503-5481
Nothing in this message is intended to constitute an electronic signature
unless a specific statement to the contrary is included in this message.
Confidentiality Note: This message is intended only for the person or entity to
which it is addressed. It may contain confidential and/or privileged material.
Any review, transmission, dissemination or other use, or taking of any action
in reliance upon this message by persons or entities other than the intended
recipient is prohibited and may be unlawful. If you received this message in
error, please contact the sender and delete it from your computer.
smime.p7s
Description: S/MIME cryptographic signature
