Just curious - given the situation above, if you have two sets of .log and .index files under the same topic and same partition, is there any way to merge these two sets of log files and index files?
On Thu, Apr 18, 2013 at 5:42 AM, Swapnil Ghike <sgh...@linkedin.com> wrote: > Otherwise if you are using the high level consumer > (ZookeeperConsumerConnector) in code, you will have to start a new > consumer with a different group id. > > Thanks, > Swapnil > > On 4/18/13 2:09 AM, "Swapnil Ghike" <sgh...@linkedin.com> wrote: > > >Snehalata, > > > >Are you using the console consumer? You can pass a --from-beginning flag > >to the console consumer to get all the messages posted to that topic. > > > >Thanks, > >Swapnil > > > >On 4/18/13 1:32 AM, "Snehalata Nagaje" > ><snehalata.nag...@harbingergroup.com> wrote: > > > >>Hi Neha, > >> > >>I was able to figure this out, this is happening due to time based > >>rolling. > >> > >>But now problem, when I fetch the messages for particular topic, server > >>is > >>reading only one log file, returns those messages only. > >> > >>Is there any way I can combine all log files which are already produced > >>into > >>one, so that I can get all messages posted in that topic. > >> > >>Thanks, > >>Snehalata > >> > >> > >>-----Original Message----- > >>From: Snehalata Nagaje [mailto:snehalata.nag...@harbingergroup.com] > >>Sent: Thursday, April 18, 2013 11:41 AM > >>To: users@kafka.apache.org > >>Subject: RE: producer creating multiple log files after message sending > >> > >>Thanks Neha, > >> > >>-----Original Message----- > >>From: Neha Narkhede [mailto:neha.narkh...@gmail.com] > >>Sent: Wednesday, April 17, 2013 9:08 PM > >>To: users@kafka.apache.org > >>Subject: Re: producer creating multiple log files after message sending > >> > >>Hi Snehalata, > >> > >>Would you grep through the log4j files for the topic under > >>consideration and look for "Rolling.*due to". This will tell us why > >>the log is getting rolled over. > >> > >>Thanks, > >>Neha > >> > >>On Wed, Apr 17, 2013 at 12:05 AM, Snehalata Nagaje > >><snehalata.nag...@harbingergroup.com> wrote: > >>> Hi, > >>> > >>> > >>> > >>> I am using kafak 0.8 version and sending messages to kafka server using > >>> below code, > >>> > >>> > >>> > >>> Producer<Integer, String> producer = new Producer<Integer, String>(new > >>> ProducerConfig(props)); > >>> > >>> producer.send(new KeyedMessage(topicName,partitionKey,jsonString)); > >>> > >>> producer.close(); > >>> > >>> > >>> > >>> I am able to post messages with, but after certain days, kafka creating > >>> multiple log files, messages are posted in new log file, although old > >>>file > >>> size is very less. > >>> > >>> > >>> > >>> What can be the reason for creating multiple log files? Does message > >>format > >>> matters while sending message to kafka? > >>> > >>> > >>> > >>> Thanks, > >>> > >>> Snehalata > >>> > >>> > >>> Disclaimer: > >>> This e-mail may contain Privileged/Confidential information and is > >>intended only for the individual(s) named. Any review, retransmission, > >>dissemination or other use of, or taking of any action in reliance upon > >>this > >>information by persons or entities other than the intended recipient is > >>prohibited. Please notify the sender, if you have received this e-mail by > >>mistake and delete it from your system. Information in this message that > >>does not relate to the official business of the company shall be > >>understood > >>as neither given nor endorsed by it. E-mail transmission cannot be > >>guaranteed to be secure or error-free. The sender does not accept > >>liability > >>for any errors or omissions in the contents of this message which arise > >>as a > >>result of e-mail transmission. If verification is required please request > >>a > >>hard-copy version. Visit us at http://www.harbingergroup.com/ > >> > >>Disclaimer: > >>This e-mail may contain Privileged/Confidential information and is > >>intended > >>only for the individual(s) named. Any review, retransmission, > >>dissemination > >>or other use of, or taking of any action in reliance upon this > >>information > >>by persons or entities other than the intended recipient is prohibited. > >>Please notify the sender, if you have received this e-mail by mistake and > >>delete it from your system. Information in this message that does not > >>relate > >>to the official business of the company shall be understood as neither > >>given > >>nor endorsed by it. E-mail transmission cannot be guaranteed to be secure > >>or > >>error-free. The sender does not accept liability for any errors or > >>omissions > >>in the contents of this message which arise as a result of e-mail > >>transmission. If verification is required please request a hard-copy > >>version. Visit us at http://www.harbingergroup.com/ > >> > >>Disclaimer: > >>This e-mail may contain Privileged/Confidential information and is > >>intended only for the individual(s) named. Any review, retransmission, > >>dissemination or other use of, or taking of any action in reliance upon > >>this information by persons or entities other than the intended recipient > >>is prohibited. Please notify the sender, if you have received this e-mail > >>by mistake and delete it from your system. Information in this message > >>that does not relate to the official business of the company shall be > >>understood as neither given nor endorsed by it. E-mail transmission > >>cannot be guaranteed to be secure or error-free. The sender does not > >>accept liability for any errors or omissions in the contents of this > >>message which arise as a result of e-mail transmission. If verification > >>is required please request a hard-copy version. Visit us at > >>http://www.harbingergroup.com/ > > > >
