Best way to erase consumer state

[Ben Drees]

Hi,

What is the best way to erase a stopped, multiprocess consumer's state so
that it can be reset (to smallest offsets, for example) when the consumer
is restarted?  Deleting /consumers/group-name from Zookeeper has had the
desired effect in the past, but I ran into a situation today where some of
the consumer processes jumped to the desired offsets (in this case
largest) on restart but others did not.  I'm not sure whether the issue
was replication lag across the Zookeeper cluster or cached state in Kafka.
I’m using kafka-python.

Thanks,
Ben

Re: log4j dir?

[Ben Drees]

Hi,

I had trouble with this as well.  The version of Kafka I'm running insists
on using 'kafka/logs', so I create a soft link from there to the desired
destination directory:

# kafka scripts hard-code the logs dir, so point that path to where we want
the logs to be.
ln -s $STREAM_BUFFER_LOGS_DIR kafka/logs

-Ben


On Fri, Nov 14, 2014 at 11:17 AM, hsy...@gmail.com hsy...@gmail.com wrote:

 Anyone has any idea how do I config the log4j file dir?

 On Thu, Nov 13, 2014 at 4:58 PM, hsy...@gmail.com hsy...@gmail.com
 wrote:

  Hi guys,
 
  Just notice kafka.logs.dir in log4j.properties doesn't take effect
 
  It's always set to *$base_dir/logs* in kafka-run-class.sh
 
  LOG_DIR=$base_dir/logs
  KAFKA_LOG4J_OPTS=-Dkafka.logs.dir=$LOG_DIR $KAFKA_LOG4J_OPTS
 
  Best,
  Siyuan