Kafka Streams vs Flink (or any other stream processing framework)

Hi All, I have read enough blogs from Confluent and others and also books that tried to talk about the differences between the two and while it is great to know those differences I hardly find them any useful when it comes to decision making process of which one to pick since I don't see the clear

Subscription to this list

-- Thanks nVidia

Re: Kafka on windows

I'm curious as well. That doc blurb doesn't give specifics. How is kafka run (or tested) on Windows? Natively via the command line shell, etc. or via cygwin, within a *nix VM on Windows, or via Windows 10's Ubuntu Linux Bash shell? Would be interesting to see how each method I listed performs, mayb

Re: Kafka on windows

One issue is that Kafka leverage some very specific features of the linux kernel that are probably far different from Windows, so I imagine the performance profile is likewise much different. On 4/11/17, 8:52 AM, "Tomasz Rojek" wrote: Hi All, We want to choose provider of messagin

Kafka on windows

Hi All, We want to choose provider of messaging system in our company, one of possible choices is Apache Kafka. One of operating system that will host brokers is windows, according to documentation: https://kafka.apache.org/documentation.html#os "*We have seen a few issues running on Windows and

Re: [VOTE] 0.10.2.1 RC0

Wrong link :) http://kafka.apache.org/documentation/#upgrade and http://kafka.apache.org/documentation/streams#streams_api_changes_0102 On Tue, Apr 11, 2017 at 5:57 PM, Gwen Shapira wrote: > FYI: I just updated the upgrade notes with Streams changes: > http://kafka.apache.org/documentation/#getti

Re: [VOTE] 0.10.2.1 RC0

FYI: I just updated the upgrade notes with Streams changes: http://kafka.apache.org/documentation/#gettingStarted On Fri, Apr 7, 2017 at 5:12 PM, Gwen Shapira wrote: > Hello Kafka users, developers and client-developers, > > This is the first candidate for the release of Apache Kafka 0.10.2.1. Th

Re: [VOTE] 0.10.2.1 RC0

Thanks for the feedback. I'm not super familiar with the inner workings of Apache's Maven repos, so I can't explain why we do things the way we do. I followed the same process on all Apache projects I was on (Kafka, Sqoop, Flume). Do you know projects that do things the way you suggested? Either

Re: [VOTE] 0.10.2.1 RC0

> On Apr 7, 2017, at 5:12 PM, Gwen Shapira wrote: > > Hello Kafka users, developers and client-developers, > > This is the first candidate for the release of Apache Kafka 0.10.2.1. This > is a bug fix release and it includes fixes and improvements from 24 JIRAs > (including a few critical bugs)

Kafka Streams Application does not start after 10.1 to 10.2 update if topics need to be auto-created

I updated from 10.1 and 10.2. I updated both the broker and maven dependency. I am using topic auto-create. With 10.1, starting the application with a broker would sometimes result in an error like: > Exception in thread "StreamThread-1" org.apache.kafka.streams.errors.TopologyBuilderException: I

Re: Kafka producer drops large messages

Hi Smirit, You will have to change some of broker configuration like message.max.bytes to a larger value. The default value is 1 MB guess. Please check below configs: Broker Configuration

Re: Kafka security

Don't hard code it. Martin's suggestion allows it to be read from a configuration file or injected from another source such as an environment variable at runtime. If you neither of these are acceptable for corporate policy I suggest asking how it has been handled before at your company. Christian

Kafka producer drops large messages

Hello all, Can somebody shed light on kafka producer's behavior when the total size of all messages in the buffer (bounded by queue.buffering.max.ms) exceeds the socket buffer size (send.buffer.bytes)? I'm using Kafka v0.8.2 with the old Producer API and have noticed that our systems are dropping

Re: Kafka security

Thanks for your response . We aren't allowed to hard code password in any of our program On Apr 11, 2017 23:39, "Mar Ian" wrote: > Since is a java property you could set the property (keystore password) > programmatically, > > before you connect to kafka (ie, before creating a consumer or prod

Re: Kafka security

Since is a java property you could set the property (keystore password) programmatically, before you connect to kafka (ie, before creating a consumer or producer) System.setProperty("zookeeper.ssl.keyStore.password", password); martin From: IT Consultant <0bina

Kafka security

Hi All How can I avoid using password for keystore creation ? Our corporate policies doesn't​allow us to hardcore password. We are currently passing keystore password while accessing TLS enabled Kafka instance . I would like to use either passwordless keystore or avoid password for cleint access

Re: auto.offset.reset for Kafka streams 0.10.2.0

Thanks for the clarification Matthais / Michael! +1 to clear documentation around this because as far as I remember, default for normal consumers is "latest" and since Streams internally use normal consumers, the first intuition is that it will be "latest" for Streams as well. Best, Mahendra

Re: auto.offset.reset for Kafka streams 0.10.2.0

It's also documented at http://docs.confluent.io/current/streams/developer-guide.html#non-streams-configuration-parameters . FYI: We have already begun syncing the Confluent docs for Streams into the Apache Kafka docs for Streams, but there's still quite some work left (volunteers are welcome :-P)