Hi Otis, The partition segment files are only created when the first message of this partition as arrived, so 1) if your producer send keyed messages which specifies their partitions or 2) your producer has not sent enough data (in 0.8.x it stick to one partition for a while before going to the next one in round robin) then later partitions may not be created. To verify that this topic does have 24 partitions you can use bin/kafka-topics.sh tool to check its partition distribution.
Guozhang On Sat, May 31, 2014 at 12:46 PM, Otis Gospodnetic < [email protected]> wrote: > Hi, > > I have Kafka 0.8.1.1 running and configured to have 24 partitions: > $ grep partition /usr/lib/kafka_2.8.0-0.8.1.1/config/server.properties > # The default number of log partitions per topic. More partitions allow > greater > num.partitions=24 > > But some of the topics don't seem to have 24 partitions, like this one: > > $ ls -la spm_af_alerts* > > drwxrwxr-x 2 ec2-user ec2-user 4096 May 30 07:43 spm_af_alerts_topic-0 > drwxrwxr-x 2 ec2-user ec2-user 4096 May 30 07:43 spm_af_alerts_topic-1 > drwxrwxr-x 2 ec2-user ec2-user 4096 May 30 07:43 spm_af_alerts_topic-10 > drwxrwxr-x 2 ec2-user ec2-user 4096 May 30 07:43 spm_af_alerts_topic-11 > drwxrwxr-x 2 ec2-user ec2-user 4096 May 30 07:43 spm_af_alerts_topic-2 > drwxrwxr-x 2 ec2-user ec2-user 4096 May 30 07:43 spm_af_alerts_topic-3 > drwxrwxr-x 2 ec2-user ec2-user 4096 May 30 07:43 spm_af_alerts_topic-4 > drwxrwxr-x 2 ec2-user ec2-user 4096 May 30 07:43 spm_af_alerts_topic-5 > drwxrwxr-x 2 ec2-user ec2-user 4096 May 30 07:43 spm_af_alerts_topic-6 > drwxrwxr-x 2 ec2-user ec2-user 4096 May 31 14:49 spm_af_alerts_topic-7 > drwxrwxr-x 2 ec2-user ec2-user 4096 May 30 07:43 spm_af_alerts_topic-8 > drwxrwxr-x 2 ec2-user ec2-user 4096 May 30 07:43 spm_af_alerts_topic-9 > > How is this possible? That -N at the end represents the partition number, > right? > > Thanks, > Otis > -- > Performance Monitoring * Log Analytics * Search Analytics > Solr & Elasticsearch Support * http://sematext.com/ > -- -- Guozhang
