Questions about Apache Kafka messages type/size and publish/subscribe

2017-10-11 Thread Heloise Chevalier
Hi, I'm not entirely certain this is the right place to ask, but I have questions about the functioning of Apache Kafka to implement a publish/subscribe messaging system. I am investigating Kafka to see if it fits the needs of the company I work for, and I have quite a few questions I can't find a

Re: Questions about Apache Kafka

2016-08-24 Thread Marko Bonaći
Hi Karin, regarding 5 (fsyncing to disk), take a look at the broker configuration parameters whose names start with log.flush. http://kafka.apache.org/documentation.html#brokerconfigs Marko Bonaći Monitoring | Alerting | Anomaly Detection | Centralized Log Management Solr & Elasticsearch Support

Re: Questions about Apache Kafka

2016-08-24 Thread Ian Wrigley
Actually, for 3 and 4 Kafka Streams is also a very good option — and comes as part of the standard Apache Kafka distribution. Ian. --- Ian Wrigley Director, Education Services Confluent, Inc > On Aug 24, 2016, at 2:37 PM, David Garcia wrote: > > Regarding 3 and 4: https://calcite.apache.org/d

Re: Questions about Apache Kafka

2016-08-24 Thread David Garcia
Regarding 3 and 4: https://calcite.apache.org/docs/stream.html (i.e. streaming SQL queries) On 8/24/16, 6:29 AM, "Herwerth Karin (CI/OSW3)" wrote: Dear Sir or Madam, I'm a beginner in Apache Kafka and have questions which I don't get it from the documentation. 1.

Re: Questions about Apache Kafka

2016-08-24 Thread Radoslaw Gruchalski
Hi Karin, 1. Kafka does not support TTL for individual messages, only the topic. From the documentation: The Kafka cluster retains all published messages—whether or not they have been consumed—for a configurable period of time. For example if the log retention is set to two days, then for the two

Questions about Apache Kafka

2016-08-24 Thread Herwerth Karin (CI/OSW3)
Dear Sir or Madam, I'm a beginner in Apache Kafka and have questions which I don't get it from the documentation. 1.If you set a Time To Live to a topic and to a message, which Time To Live is prioritized? 2.Supports Apache Kafka a mechanism to stop the publisher if the subscriber is