[jira] [Created] (KAFKA-7710) Poor Zookeeper ACL management with Kerberos
Mr Kafka created KAFKA-7710:
---
Summary: Poor Zookeeper ACL management with Kerberos
Key: KAFKA-7710
URL: https://issues.apache.org/jira/browse/KAFKA-7710
Project: Kafka
Issue Type: Bug
Reporter: Mr Kafka
I have seen many organizations run many Kafka clusters. The simplest scenario
is you may have a *kafka.dev.example.com* cluster and a
*kafka.prod.example.com* cluster. The more extreme examples is teams with in an
organization may run their own individual clusters.
When you enable Zookeeper ACLs in Kafka the ACL looks to be set to the
principal (SPN) that is used to authenticate against Zookeeper.
For example I have brokers:
* *01.kafka.dev.example.com*
* *02.kafka.dev.example.com***
* *03.kafka.dev.example.com***
On *01.kafka.dev.example.com* **I run the below the security-migration tool:
{code:java}
KAFKA_OPTS="-Djava.security.auth.login.config=/etc/kafka/kafka_server_jaas.conf
-Dzookeeper.sasl.clientconfig=ZkClient" zookeeper-security-migration
--zookeeper.acl=secure --zookeeper.connect=a01.zookeeper.dev.example.com:2181
{code}
I end up with ACL's in Zookeeper as below:
{code:java}
# [zk: localhost:2181(CONNECTED) 2] getAcl /cluster
# 'sasl,'kafka/01.kafka.dev.example.com@EXAMPLE
# : cdrwa
{code}
This ACL means no other broker in the cluster can access the znode in Zookeeper
except broker 01.
To resolve the issue you need to set the below properties in Zookeeper's config:
{code:java}
kerberos.removeHostFromPrincipal = true
kerberos.removeRealmFromPrincipal = true
{code}
Now when Kafka set ACL's they are stored as:
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Created] (KAFKA-7510) KStreams RecordCollectorImpl leaks data to logs on error
Mr Kafka created KAFKA-7510:
---
Summary: KStreams RecordCollectorImpl leaks data to logs on error
Key: KAFKA-7510
URL: https://issues.apache.org/jira/browse/KAFKA-7510
Project: Kafka
Issue Type: Bug
Components: streams
Reporter: Mr Kafka
org.apache.kafka.streams.processor.internals.RecordCollectorImpl leaks data on
error as it dumps the *value* / message payload to the logs.
This is problematic as it may contain personally identifiable information (pii)
or other secret information to plain text log files which can then be
propagated to other log systems i.e Splunk.
I suggest the *key*, and *value* fields be moved to debug level as it is useful
for some people while error level contains the *errorMessage, timestamp, topic*
and *stackTrace*.
{code:java}
private void recordSendError(
final K key,
final V value,
final Long timestamp,
final String topic,
final Exception exception
) {
String errorLogMessage = LOG_MESSAGE;
String errorMessage = EXCEPTION_MESSAGE;
if (exception instanceof RetriableException) {
errorLogMessage += PARAMETER_HINT;
errorMessage += PARAMETER_HINT;
}
log.error(errorLogMessage, key, value, timestamp, topic,
exception.toString());
sendException = new StreamsException(
String.format(
errorMessage,
logPrefix,
"an error caught",
key,
value,
timestamp,
topic,
exception.toString()
),
exception);
}{code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
