There is no way to merge those log files in 0.8. But why would you want to do that ? To reduce the number of open file handles ?
Thanks, Neha On Thursday, April 18, 2013, Jason Huang wrote: > Just curious - given the situation above, if you have two sets of .log and > .index files under the same topic and same partition, is there any way to > merge these two sets of log files and index files? > > > On Thu, Apr 18, 2013 at 5:42 AM, Swapnil Ghike <sgh...@linkedin.com> > wrote: > > > Otherwise if you are using the high level consumer > > (ZookeeperConsumerConnector) in code, you will have to start a new > > consumer with a different group id. > > > > Thanks, > > Swapnil > > > > On 4/18/13 2:09 AM, "Swapnil Ghike" <sgh...@linkedin.com> wrote: > > > > >Snehalata, > > > > > >Are you using the console consumer? You can pass a --from-beginning flag > > >to the console consumer to get all the messages posted to that topic. > > > > > >Thanks, > > >Swapnil > > > > > >On 4/18/13 1:32 AM, "Snehalata Nagaje" > > ><snehalata.nag...@harbingergroup.com> wrote: > > > > > >>Hi Neha, > > >> > > >>I was able to figure this out, this is happening due to time based > > >>rolling. > > >> > > >>But now problem, when I fetch the messages for particular topic, server > > >>is > > >>reading only one log file, returns those messages only. > > >> > > >>Is there any way I can combine all log files which are already produced > > >>into > > >>one, so that I can get all messages posted in that topic. > > >> > > >>Thanks, > > >>Snehalata > > >> > > >> > > >>-----Original Message----- > > >>From: Snehalata Nagaje [mailto:snehalata.nag...@harbingergroup.com] > > >>Sent: Thursday, April 18, 2013 11:41 AM > > >>To: users@kafka.apache.org > > >>Subject: RE: producer creating multiple log files after message sending > > >> > > >>Thanks Neha, > > >> > > >>-----Original Message----- > > >>From: Neha Narkhede [mailto:neha.narkh...@gmail.com] > > >>Sent: Wednesday, April 17, 2013 9:08 PM > > >>To: users@kafka.apache.org > > >>Subject: Re: producer creating multiple log files after message sending > > >> > > >>Hi Snehalata, > > >> > > >>Would you grep through the log4j files for the topic under > > >>consideration and look for "Rolling.*due to". This will tell us why > > >>the log is getting rolled over. > > >> > > >>Thanks, > > >>Neha > > >> > > >>On Wed, Apr 17, 2013 at 12:05 AM, Snehalata Nagaje > > >><snehalata.nag...@harbingergroup.com> wrote: > > >>> Hi, > > >>> > > >>> > > >>> > > >>> I am using kafak 0.8 version and sending messages to kafka server > using > > >>> below code, > > >>> > > >>> > > >>> > > >>> Producer<Integer, String> producer = new Producer<Integer, > String>(new > > >>> ProducerConfig(props)); > > >>> > > >>> producer.send(new KeyedMessage(topicName,partitionKey,jsonString)); > > >>> > > >>> producer.close(); > > >>> > > >>> > > >>> > > >>> I am able to post messages with, but after certain days, kafka > creating > > >>> multiple log files, messages are posted in new log file, although old > > >>>file > > >>> size is very less. > > >>> > > >>> > > >>> > > >>> What can be the reason for creating multiple log files? Does message > > >>format > > >>> matters while sending message to kafka? > > >>> > > >>> > > >>> > > >>> Thanks, > > >>> > > >>> Snehalata > > >>> > > >>> > > >>> Disclaimer: > > >>> This e-mail may contain Privileged/Confidential information and is > > >>intended on
