how to make a user as consumer to all topics using kafka-acls.sh
Hi community, I need to grant a user consumer permissions on all topics (super consumer ), including new topics that haven’t been created. I have tried this: kafka-acls.sh --add --allow-principal User:$1 --operation Read --topic "*" kafka-acls.sh --add --allow-principal User:$1 --operation Describe --topic “*" (notice that I omitted irrelevant arguments) The two commands work for existing topics but do not apply to new topics afterwards. Any suggestion would be appreciated. Thanks.
Restart without shutdown log
Hi, I am using Ambari to manage Kafka, info as listed below: Ambari version: 2.7.4.0 Kafka version: 2.0.0 The problem I ran into is that one broker restarts without shutdown log, which makes it difficult to track down the reason. The related logs are as follows, in which I cannot find "shut down" message and it seems that there is a log pause around 1.5 minutes before a sudden restart. [2020-04-14 08:01:44,214] INFO [TransactionCoordinator id=1008] Initialized transactionalId Source: Custom Source -> Sink: test-d06beb747c9d00565739f0dfcdd14614-12 with producerId 217004 and producer epoch 31222 on partition __transaction_state-41 (kafka.coordinator.transaction.TransactionCoordinator) *[2020-04-14 08:01:45,276]* INFO [TransactionCoordinator id=1008] Initialized transactionalId Source: Custom Source -> Filter -> Map -> Filter -> Sink: Unnamed-9ab9edcfebd7d79af65d59b6e0b37d6f-23 with producerId 217030 and producer epoch 19220 on partition __transaction_state-1 (kafka.coordinator.transaction.TransactionCoordinator) *[2020-04-14 08:03:23,026]* INFO Registered kafka:type=kafka.Log4jController MBean (kafka.utils.Log4jControllerRegistration$) *[2020-04-14 08:03:23,678] INFO starting (kafka.server.KafkaServer)* [2020-04-14 08:03:23,679] INFO Connecting to zookeeper on luoge-kafka-03:2181,luoge-kafka-02:2181,luoge-kafka-04:2181,luoge-kafka-05:2181,luoge-kafka-06:2181 (kafka.server.KafkaServer) [2020-04-14 08:03:23,774] INFO [ZooKeeperClient] Initializing a new session to luoge-kafka-03:2181,luoge-kafka-02:2181,luoge-kafka-04:2181,luoge-kafka-05:2181,luoge-kafka-06:2181. (kafka.zookeeper.ZooKeeperClient) [2020-04-14 08:03:23,819] INFO [ZooKeeperClient] Waiting until connected. (kafka.zookeeper.ZooKeeperClient) [2020-04-14 08:03:23,874] ERROR [ZooKeeperClient] Auth failed. (kafka.zookeeper.ZooKeeperClient) [2020-04-14 08:03:23,941] INFO [ZooKeeperClient] Connected. (kafka.zookeeper.ZooKeeperClient) [2020-04-14 08:03:24,382] INFO Cluster ID = J7raFmuqQ8mh_7DwZArH0A (kafka.server.KafkaServer) [2020-04-14 08:03:24,629] INFO KafkaConfig values: Any insight would be appreciated, thanks.
endless authorizer log
I am using Ambari to manage Kafka, info listed below: Ambari version: 2.7.4.0 Kafka version: 2.0.0 broker number: 10 On every broker, authorizer logger keeps outputting following logs: [2020-04-14 07:56:40,214] INFO Principal = User:xxx is Denied Operation = Describe from host = 10.90.1.213 on resource = Topic:LITERAL:some_topic (kafka.authorizer.logger) [2020-04-14 07:56:40,214] INFO Principal = User:xxx is Denied Operation = Describe from host = 10.90.1.213 on resource = Topic:LITERAL:some_topic (kafka.authorizer.logger) [2020-04-14 07:56:40,214] INFO Principal = User: is Denied Operation = Describe from host = 10.90.1.213 on resource = Topic:LITERAL:some_topic (kafka.authorizer.logger) I would like to know why and how to prevent this. Thanks.
Re: MessagesOutPerSec JMX metric
Thanks for your reply, Weichu. How about directly count the messages that are sent to users instead of their committed offsets ? Weichu Liu 于2020年4月14日周二 下午12:19写道： > This is because of Kafka consumer's fetch model -- a consumer can > completely control how the committed offset moves, even resetting them > anywhere. > > Under these circumstances, MessagesOutPerSec is meaningless. > > On Mon, Apr 13, 2020 at 11:38 AM 张祥 wrote: > > > Hi, > > > > I am wondering why there isn't a metric called MessagesOutPerSec in Kafka > > JMX metrics to describe how many messages are consumed by clients and > > fetched by followers per second since there are already metrics like > > MessagesInPerSec, BytesInPerSec and BytesOutPerSec. Thanks. > > >
MessagesOutPerSec JMX metric
Hi, I am wondering why there isn't a metric called MessagesOutPerSec in Kafka JMX metrics to describe how many messages are consumed by clients and fetched by followers per second since there are already metrics like MessagesInPerSec, BytesInPerSec and BytesOutPerSec. Thanks.
JMX expired topic metircs
Hi, I notice that there are jmx metrics for deleted topics when using java code and jmxterm. Has anyone also run into this ? If yes, what is the reason behind this and how can I filter expired metrics ? Thanks.
Kafka JMX monitoring
Hi, I want to know what the best practice to collect Kafka JMX metrics is. I haven't found a decent way to collect and parse JMX in Java (because it is too much) and I learn that there are tools like tools like jmxtrans to do this. I wonder if there is more. Thanks. Regards.
Regarding segment size config
Hi community, I understand that there are two configs regarding segment file size, log.segment.bytes for broker and segment.bytes for topic. The default values are both 1G and they are required to be an integer so they cannot be larger than 2G. My question is, assuming I am not making any mistakes, what is the reason that log segment size is limited below 2G ? Thanks.
Re: what happened in case of single disk failure
Thanks, it helps a lot, for a long time I am used to read documentation on Kafka official site, you made me realize that there are also a lot of resources on Confluent. M. Manna 于2020年3月12日周四 下午9:06写道： > Please see the following link from Confluent. Also, if you register with > Confluent Technical Talks, they are running quite a lot of nice and > simplified webinar this month on Fundamentals of Kafka. > > https://www.youtube.com/watch?v=ibozaujze9k > > I thought the 2 part presentation was quite good (but I don't work for > Confluent :), so a disclaimer in advance). > > There is also an upcoming webinar on how Kafka is integrated in your > application/architecture. > > I hope it helps. > > Regards, > M. MAnna > > On Thu, 12 Mar 2020 at 00:51, 张祥 wrote: > > > Thanks, very helpful ! > > > > Peter Bukowinski 于2020年3月12日周四 上午5:48写道： > > > > > Yes, that’s correct. While a broker is down: > > > > > > all topic partitions assigned to that broker will be under-replicated > > > topic partitions with an unmet minimum ISR count will be offline > > > leadership of partitions meeting the minimum ISR count will move to the > > > next in-sync replica in the replica list > > > if no in-sync replica exists for a topic-partitions, it will be offline > > > Setting unclean.leader.election.enable=true will allow an out-of-sync > > > replica to become a leader. > > > If topic partition availability is more important to you than data > > > integrity, you should allow unclean leader election. > > > > > > > > > > On Mar 11, 2020, at 6:11 AM, 张祥 wrote: > > > > > > > > Hi, Peter, following what we talked about before, I want to > understand > > > what > > > > will happen when one broker goes down, I would say it will be very > > > similar > > > > to what happens under disk failure, except that the rules apply to > all > > > the > > > > partitions on that broker instead of only one malfunctioned disk. Am > I > > > > right? Thanks. > > > > > > > > 张祥 于2020年3月5日周四 上午9:25写道： > > > > > > > >> Thanks Peter, really appreciate it. > > > >> > > > >> Peter Bukowinski 于2020年3月4日周三 下午11:50写道： > > > >> > > > >>> Yes, you should restart the broker. I don’t believe there’s any > code > > to > > > >>> check if a Log directory previously marked as failed has returned > to > > > >>> healthy. > > > >>> > > > >>> I always restart the broker after a hardware repair. I treat broker > > > >>> restarts as a normal, non-disruptive operation in my clusters. I > use > > a > > > >>> minimum of 3x replication. > > > >>> > > > >>> -- Peter (from phone) > > > >>> > > > >>>> On Mar 4, 2020, at 12:46 AM, 张祥 wrote: > > > >>>> > > > >>>> Another question, according to my memory, the broker needs to be > > > >>> restarted > > > >>>> after replacing disk to recover this. Is that correct? If so, I > take > > > >>> that > > > >>>> Kafka cannot know by itself that the disk has been replaced, > > manually > > > >>>> restart is necessary. > > > >>>> > > > >>>> 张祥 于2020年3月4日周三 下午2:48写道： > > > >>>> > > > >>>>> Thanks Peter, it makes a lot of sense. > > > >>>>> > > > >>>>> Peter Bukowinski 于2020年3月3日周二 上午11:56写道： > > > >>>>> > > > >>>>>> Whether your brokers have a single data directory or multiple > data > > > >>>>>> directories on separate disks, when a disk fails, the topic > > > partitions > > > >>>>>> located on that disk become unavailable. What happens next > depends > > > on > > > >>> how > > > >>>>>> your cluster and topics are configured. > > > >>>>>> > > > >>>>>> If the topics on the affected broker have replicas and the > minimum > > > ISR > > > >>>>>> (in-sync replicas) count is met, then all topic partitions will > > > remain > > > >>>>>> online and leaders will move to another broker. Producers and > > > >>> consumers > > > >>>>>> will continue to operate as usual. > > > >>>>>> > > > >>>>>> If the topics don’t have replicas or the minimum ISR count is > not > > > met, > > > >>>>>> then the topic partitions on the failed disk will be offline. > > > >>> Producers can > > > >>>>>> still send data to the affected topics — it will just go to the > > > online > > > >>>>>> partitions. Consumers can still consume data from the online > > > >>> partitions. > > > >>>>>> > > > >>>>>> -- Peter > > > >>>>>> > > > >>>>>>>> On Mar 2, 2020, at 7:00 PM, 张祥 > > wrote: > > > >>>>>>>> > > > >>>>>>>> Hi community， > > > >>>>>>>> > > > >>>>>>>> I ran into disk failure when using Kafka, and fortunately it > did > > > not > > > >>>>>> crash > > > >>>>>>> the entire cluster. So I am wondering how Kafka handles > multiple > > > >>> disks > > > >>>>>> and > > > >>>>>>> it manages to work in case of single disk failure. The more > > > detailed, > > > >>>>>> the > > > >>>>>>> better. Thanks ! > > > >>>>>> > > > >>>>> > > > >>> > > > >> > > > > > > > > >
Re: what happened in case of single disk failure
Thanks, very helpful ! Peter Bukowinski 于2020年3月12日周四 上午5:48写道： > Yes, that’s correct. While a broker is down: > > all topic partitions assigned to that broker will be under-replicated > topic partitions with an unmet minimum ISR count will be offline > leadership of partitions meeting the minimum ISR count will move to the > next in-sync replica in the replica list > if no in-sync replica exists for a topic-partitions, it will be offline > Setting unclean.leader.election.enable=true will allow an out-of-sync > replica to become a leader. > If topic partition availability is more important to you than data > integrity, you should allow unclean leader election. > > > > On Mar 11, 2020, at 6:11 AM, 张祥 wrote: > > > > Hi, Peter, following what we talked about before, I want to understand > what > > will happen when one broker goes down, I would say it will be very > similar > > to what happens under disk failure, except that the rules apply to all > the > > partitions on that broker instead of only one malfunctioned disk. Am I > > right? Thanks. > > > > 张祥 于2020年3月5日周四 上午9:25写道： > > > >> Thanks Peter, really appreciate it. > >> > >> Peter Bukowinski 于2020年3月4日周三 下午11:50写道： > >> > >>> Yes, you should restart the broker. I don’t believe there’s any code to > >>> check if a Log directory previously marked as failed has returned to > >>> healthy. > >>> > >>> I always restart the broker after a hardware repair. I treat broker > >>> restarts as a normal, non-disruptive operation in my clusters. I use a > >>> minimum of 3x replication. > >>> > >>> -- Peter (from phone) > >>> > >>>> On Mar 4, 2020, at 12:46 AM, 张祥 wrote: > >>>> > >>>> Another question, according to my memory, the broker needs to be > >>> restarted > >>>> after replacing disk to recover this. Is that correct? If so, I take > >>> that > >>>> Kafka cannot know by itself that the disk has been replaced, manually > >>>> restart is necessary. > >>>> > >>>> 张祥 于2020年3月4日周三 下午2:48写道： > >>>> > >>>>> Thanks Peter, it makes a lot of sense. > >>>>> > >>>>> Peter Bukowinski 于2020年3月3日周二 上午11:56写道： > >>>>> > >>>>>> Whether your brokers have a single data directory or multiple data > >>>>>> directories on separate disks, when a disk fails, the topic > partitions > >>>>>> located on that disk become unavailable. What happens next depends > on > >>> how > >>>>>> your cluster and topics are configured. > >>>>>> > >>>>>> If the topics on the affected broker have replicas and the minimum > ISR > >>>>>> (in-sync replicas) count is met, then all topic partitions will > remain > >>>>>> online and leaders will move to another broker. Producers and > >>> consumers > >>>>>> will continue to operate as usual. > >>>>>> > >>>>>> If the topics don’t have replicas or the minimum ISR count is not > met, > >>>>>> then the topic partitions on the failed disk will be offline. > >>> Producers can > >>>>>> still send data to the affected topics — it will just go to the > online > >>>>>> partitions. Consumers can still consume data from the online > >>> partitions. > >>>>>> > >>>>>> -- Peter > >>>>>> > >>>>>>>> On Mar 2, 2020, at 7:00 PM, 张祥 wrote: > >>>>>>>> > >>>>>>>> Hi community， > >>>>>>>> > >>>>>>>> I ran into disk failure when using Kafka, and fortunately it did > not > >>>>>> crash > >>>>>>> the entire cluster. So I am wondering how Kafka handles multiple > >>> disks > >>>>>> and > >>>>>>> it manages to work in case of single disk failure. The more > detailed, > >>>>>> the > >>>>>>> better. Thanks ! > >>>>>> > >>>>> > >>> > >> > >
Re: what happened in case of single disk failure
Hi, Peter, following what we talked about before, I want to understand what will happen when one broker goes down, I would say it will be very similar to what happens under disk failure, except that the rules apply to all the partitions on that broker instead of only one malfunctioned disk. Am I right? Thanks. 张祥 于2020年3月5日周四 上午9:25写道： > Thanks Peter, really appreciate it. > > Peter Bukowinski 于2020年3月4日周三 下午11:50写道： > >> Yes, you should restart the broker. I don’t believe there’s any code to >> check if a Log directory previously marked as failed has returned to >> healthy. >> >> I always restart the broker after a hardware repair. I treat broker >> restarts as a normal, non-disruptive operation in my clusters. I use a >> minimum of 3x replication. >> >> -- Peter (from phone) >> >> > On Mar 4, 2020, at 12:46 AM, 张祥 wrote: >> > >> > Another question, according to my memory, the broker needs to be >> restarted >> > after replacing disk to recover this. Is that correct? If so, I take >> that >> > Kafka cannot know by itself that the disk has been replaced, manually >> > restart is necessary. >> > >> > 张祥 于2020年3月4日周三 下午2:48写道： >> > >> >> Thanks Peter, it makes a lot of sense. >> >> >> >> Peter Bukowinski 于2020年3月3日周二 上午11:56写道： >> >> >> >>> Whether your brokers have a single data directory or multiple data >> >>> directories on separate disks, when a disk fails, the topic partitions >> >>> located on that disk become unavailable. What happens next depends on >> how >> >>> your cluster and topics are configured. >> >>> >> >>> If the topics on the affected broker have replicas and the minimum ISR >> >>> (in-sync replicas) count is met, then all topic partitions will remain >> >>> online and leaders will move to another broker. Producers and >> consumers >> >>> will continue to operate as usual. >> >>> >> >>> If the topics don’t have replicas or the minimum ISR count is not met, >> >>> then the topic partitions on the failed disk will be offline. >> Producers can >> >>> still send data to the affected topics — it will just go to the online >> >>> partitions. Consumers can still consume data from the online >> partitions. >> >>> >> >>> -- Peter >> >>> >> >>>>> On Mar 2, 2020, at 7:00 PM, 张祥 wrote: >> >>>>> >> >>>>> Hi community， >> >>>>> >> >>>>> I ran into disk failure when using Kafka, and fortunately it did not >> >>> crash >> >>>> the entire cluster. So I am wondering how Kafka handles multiple >> disks >> >>> and >> >>>> it manages to work in case of single disk failure. The more detailed, >> >>> the >> >>>> better. Thanks ! >> >>> >> >> >> >
Re: what happened in case of single disk failure
Thanks Peter, really appreciate it. Peter Bukowinski 于2020年3月4日周三 下午11:50写道： > Yes, you should restart the broker. I don’t believe there’s any code to > check if a Log directory previously marked as failed has returned to > healthy. > > I always restart the broker after a hardware repair. I treat broker > restarts as a normal, non-disruptive operation in my clusters. I use a > minimum of 3x replication. > > -- Peter (from phone) > > > On Mar 4, 2020, at 12:46 AM, 张祥 wrote: > > > > Another question, according to my memory, the broker needs to be > restarted > > after replacing disk to recover this. Is that correct? If so, I take that > > Kafka cannot know by itself that the disk has been replaced, manually > > restart is necessary. > > > > 张祥 于2020年3月4日周三 下午2:48写道： > > > >> Thanks Peter, it makes a lot of sense. > >> > >> Peter Bukowinski 于2020年3月3日周二 上午11:56写道： > >> > >>> Whether your brokers have a single data directory or multiple data > >>> directories on separate disks, when a disk fails, the topic partitions > >>> located on that disk become unavailable. What happens next depends on > how > >>> your cluster and topics are configured. > >>> > >>> If the topics on the affected broker have replicas and the minimum ISR > >>> (in-sync replicas) count is met, then all topic partitions will remain > >>> online and leaders will move to another broker. Producers and consumers > >>> will continue to operate as usual. > >>> > >>> If the topics don’t have replicas or the minimum ISR count is not met, > >>> then the topic partitions on the failed disk will be offline. > Producers can > >>> still send data to the affected topics — it will just go to the online > >>> partitions. Consumers can still consume data from the online > partitions. > >>> > >>> -- Peter > >>> > >>>>> On Mar 2, 2020, at 7:00 PM, 张祥 wrote: > >>>>> > >>>>> Hi community， > >>>>> > >>>>> I ran into disk failure when using Kafka, and fortunately it did not > >>> crash > >>>> the entire cluster. So I am wondering how Kafka handles multiple disks > >>> and > >>>> it manages to work in case of single disk failure. The more detailed, > >>> the > >>>> better. Thanks ! > >>> > >> >
Re: what happened in case of single disk failure
Another question, according to my memory, the broker needs to be restarted after replacing disk to recover this. Is that correct? If so, I take that Kafka cannot know by itself that the disk has been replaced, manually restart is necessary. 张祥 于2020年3月4日周三 下午2:48写道： > Thanks Peter, it makes a lot of sense. > > Peter Bukowinski 于2020年3月3日周二 上午11:56写道： > >> Whether your brokers have a single data directory or multiple data >> directories on separate disks, when a disk fails, the topic partitions >> located on that disk become unavailable. What happens next depends on how >> your cluster and topics are configured. >> >> If the topics on the affected broker have replicas and the minimum ISR >> (in-sync replicas) count is met, then all topic partitions will remain >> online and leaders will move to another broker. Producers and consumers >> will continue to operate as usual. >> >> If the topics don’t have replicas or the minimum ISR count is not met, >> then the topic partitions on the failed disk will be offline. Producers can >> still send data to the affected topics — it will just go to the online >> partitions. Consumers can still consume data from the online partitions. >> >> -- Peter >> >> > On Mar 2, 2020, at 7:00 PM, 张祥 wrote: >> > >> > Hi community， >> > >> > I ran into disk failure when using Kafka, and fortunately it did not >> crash >> > the entire cluster. So I am wondering how Kafka handles multiple disks >> and >> > it manages to work in case of single disk failure. The more detailed, >> the >> > better. Thanks ! >> >
Re: what happened in case of single disk failure
Thanks Peter, it makes a lot of sense. Peter Bukowinski 于2020年3月3日周二 上午11:56写道： > Whether your brokers have a single data directory or multiple data > directories on separate disks, when a disk fails, the topic partitions > located on that disk become unavailable. What happens next depends on how > your cluster and topics are configured. > > If the topics on the affected broker have replicas and the minimum ISR > (in-sync replicas) count is met, then all topic partitions will remain > online and leaders will move to another broker. Producers and consumers > will continue to operate as usual. > > If the topics don’t have replicas or the minimum ISR count is not met, > then the topic partitions on the failed disk will be offline. Producers can > still send data to the affected topics — it will just go to the online > partitions. Consumers can still consume data from the online partitions. > > -- Peter > > > On Mar 2, 2020, at 7:00 PM, 张祥 wrote: > > > > Hi community， > > > > I ran into disk failure when using Kafka, and fortunately it did not > crash > > the entire cluster. So I am wondering how Kafka handles multiple disks > and > > it manages to work in case of single disk failure. The more detailed, the > > better. Thanks ! >
what happened in case of single disk failure
Hi community， I ran into disk failure when using Kafka, and fortunately it did not crash the entire cluster. So I am wondering how Kafka handles multiple disks and it manages to work in case of single disk failure. The more detailed, the better. Thanks !
Re: when to expand cluster
Thanks, it helps a lot. Peter Bukowinski 于2020年2月28日周五 上午5:18写道： > No, it’s not bad. Kafka is designed to serve data to many consumers at the > same time, whether they are independent of each other or in the same > consumer group. > > I would encourage you to play with different partition counts and use > kafka’s performance testing tools (kafka-producer-perf-test.sh and > kafka-consumer-perf-test.sh) to test throughput in different scenarios and > see the results for yourself. > > — > Peter > > > On Feb 27, 2020, at 1:28 AM, 张祥 wrote: > > > > I believe no matter the partition count exceeds the broker count, we can > > always have the same number of consumer instances as the partition count. > > > > So what I want to know is when two partition exists on the same broker, > two > > consumer instances will be talking to same broker, is that bad ? > > > > 张祥 于2020年2月27日周四 下午2:20写道： > > > >> Thanks. What influence does it have for consumers and producers when > >> partition number is more than broker number, which means at least one > >> broker serves two partitions for one topic ? performance wise. > >> > >> Peter Bukowinski 于2020年2月26日周三 下午11:02写道： > >> > >>> Disk usage is one reason to expand. Another reason is if you need more > >>> ingest or output throughout for your topic data. If your producers > aren’t > >>> able to send data to kafka fast enough or your consumers are lagging, > you > >>> might benefit from more brokers and more partitions. > >>> > >>> -- Peter > >>> > >>>> On Feb 26, 2020, at 12:56 AM, 张祥 wrote: > >>>> > >>>> In documentation, it is described how to expand cluster: > >>>> > >>> > https://kafka.apache.org/20/documentation.html#basic_ops_cluster_expansion > >>> . > >>>> But I am wondering what the criteria for expand is. I can only think > of > >>>> disk usage threshold. For example, suppose several disk usage exceed > >>> 80%. > >>>> Is this correct and is there more ? > >>> > >> > >
Re: when to expand cluster
I believe no matter the partition count exceeds the broker count, we can always have the same number of consumer instances as the partition count. So what I want to know is when two partition exists on the same broker, two consumer instances will be talking to same broker, is that bad ? 张祥 于2020年2月27日周四 下午2:20写道： > Thanks. What influence does it have for consumers and producers when > partition number is more than broker number, which means at least one > broker serves two partitions for one topic ? performance wise. > > Peter Bukowinski 于2020年2月26日周三 下午11:02写道： > >> Disk usage is one reason to expand. Another reason is if you need more >> ingest or output throughout for your topic data. If your producers aren’t >> able to send data to kafka fast enough or your consumers are lagging, you >> might benefit from more brokers and more partitions. >> >> -- Peter >> >> > On Feb 26, 2020, at 12:56 AM, 张祥 wrote: >> > >> > In documentation, it is described how to expand cluster: >> > >> https://kafka.apache.org/20/documentation.html#basic_ops_cluster_expansion >> . >> > But I am wondering what the criteria for expand is. I can only think of >> > disk usage threshold. For example, suppose several disk usage exceed >> 80%. >> > Is this correct and is there more ? >> >
Re: when to expand cluster
Thanks. What influence does it have for consumers and producers when partition number is more than broker number, which means at least one broker serves two partitions for one topic ? performance wise. Peter Bukowinski 于2020年2月26日周三 下午11:02写道： > Disk usage is one reason to expand. Another reason is if you need more > ingest or output throughout for your topic data. If your producers aren’t > able to send data to kafka fast enough or your consumers are lagging, you > might benefit from more brokers and more partitions. > > -- Peter > > > On Feb 26, 2020, at 12:56 AM, 张祥 wrote: > > > > In documentation, it is described how to expand cluster: > > > https://kafka.apache.org/20/documentation.html#basic_ops_cluster_expansion > . > > But I am wondering what the criteria for expand is. I can only think of > > disk usage threshold. For example, suppose several disk usage exceed 80%. > > Is this correct and is there more ? >
when to expand cluster
In documentation, it is described how to expand cluster: https://kafka.apache.org/20/documentation.html#basic_ops_cluster_expansion. But I am wondering what the criteria for expand is. I can only think of disk usage threshold. For example, suppose several disk usage exceed 80%. Is this correct and is there more ?
Is restart necessary when adding users in SASL/PLAIN ?
Hi everyone ! Our Kafka (Ambari distribution) is under SASL/PLAIN authentication mechanism. I am adding new users and it seems a restart is required to enable the new users. So I want to know whether brokers must restart in this situation. If yes, is there a workaround to avoid this, otherwise, SASL/PALIN is less useful. | | 张祥 | | 18133622...@163.com | 签名由网易邮箱大师定制
Re: Only use SSL to encrypt the authentication messages
So, is this a no ? | | | | | 签名由网易邮箱大师定制 On 09/29/2019 17:38，Pere Urbón Bayes wrote： Hi, if you're worried about the performance impact i would suggest : * first benchmark the impacts, by experience i would see around 30 to 40% performance degradation. * use java 11, and you will see a lot less performance impact when using ssl. -- Pere On Sun, 29 Sep 2019, 08:28 张祥 <18133622...@163.com> wrote: Hi everyone ! I am enabling SASL/PLAIN authentication for our Kafka and I am aware it should be used with SSL encryption. But it may bring a performance impact. So I am wondering whether it is possible that we only use SSL to encrypt the authentication messages but leave the data unencrypted. Thanks. | | 张祥 | | 18133622...@163.com | 签名由网易邮箱大师定制
Only use SSL to encrypt the authentication messages
Hi everyone ! I am enabling SASL/PLAIN authentication for our Kafka and I am aware it should be used with SSL encryption. But it may bring a performance impact. So I am wondering whether it is possible that we only use SSL to encrypt the authentication messages but leave the data unencrypted. Thanks. | | 张祥 | | 18133622...@163.com | 签名由网易邮箱大师定制
Kerberos authentication fails when brokers IPs/hostnames are not in client's hosts file (/etc/hosts)
Hi everyone ! I am adding two adjustments to our Kafka. One is enabling Kerberos authentication and the other is changing listeners config to IP address instead of hostnames so that client machines are not required to modify hosts file (/etc/hosts). The problem is the two adjustments can only work separately but when the two are applied at the same time, cluster cannot be reached. Below is the error message when I use kafka-console-produce script to access Kafka: 09/08/28 10:32:01 ERROR clients.NetworkClient: [Producer clientId=console-producer] Connection to node -1 failed authentication due to: An error: (java.security.PrivilegedActionException: javax.security.sasl.SaslException: GSS initiate failed [Caused by GSSException: No valid credentials provided (Mechanism level: Server not found in Kerberos database (7) - LOOKING_UP_SERVER)]) occurred when evaluating SASL token received from the Kafka Broker. Kafka Client will go to AUTHENTICATION_FAILED state. 19/08/28 10:32:01 ERROR internals.ErrorLoggingCallback: Error when sending message to topic test1 with key: null, value: 3 bytes with error: org.apache.kafka.common.errors.SaslAuthenticationException: An error: (java.security.PrivilegedActionException: javax.security.sasl.SaslException: GSS initiate failed [Caused by GSSException: No valid credentials provided (Mechanism level: Server not found in Kerberos database (7) - LOOKING_UP_SERVER)]) occurred when evaluating SASL token received from the Kafka Broker. Kafka Client will go to AUTHENTICATION_FAILED state. Caused by: javax.security.sasl.SaslException: GSS initiate failed [Caused by GSSException: No valid credentials provided (Mechanism level: Server not found in Kerberos database (7) - LOOKING_UP_SERVER)] at com.sun.security.sasl.gsskerb.GssKrb5Client.evaluateChallenge(GssKrb5Client.java:211) at org.apache.kafka.common.security.authenticator.SaslClientAuthenticator$2.run(SaslClientAuthenticator.java:361) at org.apache.kafka.common.security.authenticator.SaslClientAuthenticator$2.run(SaslClientAuthenticator.java:359) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:422) at org.apache.kafka.common.security.authenticator.SaslClientAuthenticator.createSaslToken(SaslClientAuthenticator.java:359) at org.apache.kafka.common.security.authenticator.SaslClientAuthenticator.sendSaslClientToken(SaslClientAuthenticator.java:269) at org.apache.kafka.common.security.authenticator.SaslClientAuthenticator.authenticate(SaslClientAuthenticator.java:206) at org.apache.kafka.common.network.KafkaChannel.prepare(KafkaChannel.java:81) at org.apache.kafka.common.network.Selector.pollSelectionKeys(Selector.java:474) at org.apache.kafka.common.network.Selector.poll(Selector.java:412) at org.apache.kafka.clients.NetworkClient.poll(NetworkClient.java:481) at org.apache.kafka.clients.producer.internals.Sender.run(Sender.java:239) at org.apache.kafka.clients.producer.internals.Sender.run(Sender.java:163) at java.lang.Thread.run(Thread.java:748) Caused by: GSSException: No valid credentials provided (Mechanism level: Server not found in Kerberos database (7) - LOOKING_UP_SERVER) at sun.security.jgss.krb5.Krb5Context.initSecContext(Krb5Context.java:770) at sun.security.jgss.GSSContextImpl.initSecContext(GSSContextImpl.java:248) at sun.security.jgss.GSSContextImpl.initSecContext(GSSContextImpl.java:179) at com.sun.security.sasl.gsskerb.GssKrb5Client.evaluateChallenge(GssKrb5Client.java:192) ... 14 more Caused by: KrbException: Server not found in Kerberos database (7) - LOOKING_UP_SERVER at sun.security.krb5.KrbTgsRep.(KrbTgsRep.java:73) at sun.security.krb5.KrbTgsReq.getReply(KrbTgsReq.java:251) at sun.security.krb5.KrbTgsReq.sendAndGetCreds(KrbTgsReq.java:262) at sun.security.krb5.internal.CredentialsUtil.serviceCreds(CredentialsUtil.java:308) at sun.security.krb5.internal.CredentialsUtil.acquireServiceCreds(CredentialsUtil.java:126) at sun.security.krb5.Credentials.acquireServiceCreds(Credentials.java:458) at sun.security.jgss.krb5.Krb5Context.initSecContext(Krb5Context.java:693) ... 17 more My guess is that when using IP addresses for listeners config and not having hostname records of the brokers in /etc/hosts, Kafka client constructs a service principal name like ‘kafka@@REALM’ (the actual principal name should be like ‘kafka@@REALM’) for the broker and requests corresponding ticket from KDC who does not have this principal in its database so LOOKING_UP_SERVER error is raised. Am I right ? And could somebody point out what is the right way to do this ? Thanks. P.S., I am using CDK 4.1.0. | | 张祥 | | 18133622...@163.com | 签名由网易邮箱大师定制
