Preventing Kafka crashes on disk fill-up
Hello everyone. As of today, when a data disk fills up and Kafka is unable to write to it anymore, the Kafka process crashes. In this situation, users are unable to use Kafka APIs to recover their cluster, and have to rely on manually cleaning up disk space. There are multiple approaches towards addressing this issue. One of the behaviors that users commonly want is to degrade gracefully, being unable to produce any new data to the disk, while still retaining the ability to call admin APIs and delete the data on disk. I am wondering what the views of the community are on this as a default behavior. Please let me know. I am sure this is a discussion that has happened multiple times in the past in the community. I would love to not re-invent the wheel or jump to a solution without context, and look to learn from the learnings from those discussions. What are the reasons why this might be challenging to implement? What do you think are the primary obstacles / unknowns? -- Regards Tirtha Chatterjee
Known issues with time stepping
Hi team Are there any known issues with Kafka that can happen because of the system time getting stepped forward or backward in one shot? This can happen because of NTP time syncs. -- Regards Tirtha Chatterjee
[jira] [Created] (KAFKA-8670) kafka-topics.sh shows IllegalArgumentException when describing all topics if no topics exist
Tirtha Chatterjee created KAFKA-8670:
Summary: kafka-topics.sh shows IllegalArgumentException when
describing all topics if no topics exist
Key: KAFKA-8670
URL: https://issues.apache.org/jira/browse/KAFKA-8670
Project: Kafka
Issue Type: Bug
Components: admin, tools
Affects Versions: 2.2.1, 2.3.0, 2.2.0
Reporter: Tirtha Chatterjee
Assignee: Tirtha Chatterjee
When trying to describe all the kafka-topics.sh utility, a user would run
kafka-topics.sh --describe without passing a --topic option. If there are no
topics on the cluster, Kafka returns an error with IllegalArgumentException.
{code:java}
./kafka-topics.sh --zookeeper
172.16.7.230:2181,172.16.17.27:2181,172.16.10.89:2181 --describe
Error while executing topic command : Topics in [] does not exist
[2019-07-07 03:33:15,288] ERROR java.lang.IllegalArgumentException: Topics in
[] does not exist
at
kafka.admin.TopicCommand$.kafka$admin$TopicCommand$$ensureTopicExists(TopicCommand.scala:416)
at
kafka.admin.TopicCommand$ZookeeperTopicService.describeTopic(TopicCommand.scala:332)
at kafka.admin.TopicCommand$.main(TopicCommand.scala:66)
at kafka.admin.TopicCommand.main(TopicCommand.scala)
(kafka.admin.TopicCommand$)
{code}
If no --topic option is passed to the command, it should not fail, rather have
empty output.
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
JIRA and KIP contributor permissions
Hi developers, I would like to be a contributor to the Apache Kafka project. I would like to have permissions on the KIP wiki and JIRAs (to be able to assign tickets to myself). Please grant me these permissions. JIRA username: wyuka Confluence username: tirtha.p.chatterjee Github username: wyuka Committer email: ti...@amazon.com Thanks Tirtha
Re: New release branch 2.1.0
Hi When are we planning on releasing the bugfix version 2.0.1 ? On Thu, Oct 4, 2018 at 5:22 PM Dong Lin wrote: > Hello Kafka developers and users, > > As promised in the release plan > https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=91554044, > we now have a release branch for 2.1.0 release. Trunk will soon be bumped > to 2.2.0-SNAPSHOT. > > I'll be going over the JIRAs to move every non-blocker from this release to > the next release. > > From this point, most changes should go to trunk. Blockers (existing and > new that we discover while testing the release) will be double-committed. > Please discuss with your reviewer whether your PR should go to trunk or to > trunk+release so they can merge accordingly. > > Please help us test the release! > > Thanks! > Dong > -- Regards Tirtha Chatterjee
