Hi,
> 1. Will Kafka distribute the 100 serialized files randomly say 20 files go > to Partition 1, 25 to Partition 2 etc or do I have an option to configure > how many files go to which partition . > Assuming you are using new producer, All keyed messages will be distributed based on the hash of the key (or) All non-keyed message will be distributed in round-robin fashion. (or) You can specify partition number for each message using "ProducerRecord" object. > 2. How to configure each consumer to read from a particular partition > only. Say consumer 1 to read from partition 4 only likewise. > > You can use SimpleConsumer to consumer from a particular partition. https://kafka.apache.org/documentation.html#simpleconsumerapi > 2. If I have not set the max size what will happen to that 1MB file, I read > in the documentation that default size is 10KB. Default "max.request.size" on new producer and "message.max.bytes" on broker is 1MB. Anything above these limits, you will get exception. Kumar
