Using Kafka AdminUtils

2020-02-15 Thread Victoria Zuberman
Hi,

I have an application based on Kafka Streams.
It reads from Kafka topic (I call this topic “input topic”).
That topic has many partitions and their number varies based on the env in 
which application is running.
I don’t want to create different input topics manually.
Configuration of auto.create.topics.enable and num.partitions is not enough for 
me.
The solution I am looking to implement is to check during application init 
whether the input topic exists and if not to create it with relevant partition 
number and replication factor.

I found the following example that uses kafka.admin.AdminUtils and it seems to 
be suitable:
https://www.codota.com/code/java/methods/kafka.admin.AdminUtils/createTopic

Please advise whether using AdminUtils is considered a good practice.
Is AdminUtils functionality considered stable and reliable?
If there are other solutions, I would appreciate to hear about them.

Thanks,
Victoria

---
NOTICE:
This email and all attachments are confidential, may be proprietary, and may be 
privileged or otherwise protected from disclosure. They are intended solely for 
the individual or entity to whom the email is addressed. However, mistakes 
sometimes happen in addressing emails. If you believe that you are not an 
intended recipient, please stop reading immediately. Do not copy, forward, or 
rely on the contents in any way. Notify the sender and/or Imperva, Inc. by 
telephone at +1 (650) 832-6006 and then delete or destroy any copy of this 
email and its attachments. The sender reserves and asserts all rights to 
confidentiality, as well as any privileges that may apply. Any disclosure, 
copying, distribution or action taken or omitted to be taken by an unintended 
recipient in reliance on this message is prohibited and may be unlawful.
Please consider the environment before printing this email.


Re: Message prioritization in Kafka

2020-02-15 Thread Ivan Yurchenko
Hi,

Kafka does not support message priorities.

RabbitMQ may help you. Please evaluate if its clustering capabilities suit
you.

Best,
Ivan


On Sat, 15 Feb 2020 at 18:23, Vairavanathan Emalayan <
vairavanathan.emala...@move.com> wrote:

> Hi There,
>
> I am looking for a distributed priority queue implementation. Does Kafka
> supports message prioritization within a partition?
>
> Please let me know if you are aware of any distributed queues that support
> prioritization.
>
> Thank you
> Emal
>


Message prioritization in Kafka

2020-02-15 Thread Vairavanathan Emalayan
Hi There,

I am looking for a distributed priority queue implementation. Does Kafka
supports message prioritization within a partition?

Please let me know if you are aware of any distributed queues that support
prioritization.

Thank you
Emal