Re: Same partition number of different Kafka topcs

2016-08-03 Thread Jack Huang
nt brokers. The only "default Kafka partitioner" is the one that you > could consider "standardized" by the Java client implementation. Some > client libraries will make this pluggable like the Java client does so you > could use a compatible implementation. > > -Ewen

Re: Same partition number of different Kafka topcs

2016-07-29 Thread Jack Huang
partitioner (for example round robin) > > On Thu, Jul 28, 2016 at 9:11 PM Jack Huang wrote: > > > Hi all, > > > > I have an application where I need to join events from two different > > topics. Every event is identified by an id, which is used as the key fo

Same partition number of different Kafka topcs

2016-07-28 Thread Jack Huang
Hi all, I have an application where I need to join events from two different topics. Every event is identified by an id, which is used as the key for the topic partition. After doing some experiment, I observed that events will go into different partitions even if the number of partitions for both

Using Kafka without persisting message to disk

2016-07-14 Thread Jack Huang
Hi all, Is there a way to make a topic to be stored in memory only and not writing to disk? If not, what's the best way to minimize writing to disk? For this application we only need the notion of partitions and a short retention time (1hr or so) from Kafka. We want to use Kafka because we want to