The consumer starts from offset 0. Yes, in the log dir.
On Fri, Nov 1, 2013 at 4:06 PM, Jun Rao wrote:
> Which offset did you use for fetching? Is there data in the kafka log dir?
>
> Thanks,
>
> Jun
>
>
> On Fri, Nov 1, 2013 at 11:48 AM, Lu Xuechao wrote:
>
> &g
checked fetchResponse.hasError() but has no error.
On Fri, Nov 1, 2013 at 7:45 AM, Jun Rao wrote:
> Did you check the error code associated with each partition in the fetch
> response?
>
> Thanks,
>
> Jun
>
>
> On Thu, Oct 31, 2013 at 9:59 PM, Lu Xuechao wrote:
&g
I enabled gzip compression. Each topic has 10 partitions and each partition
is handled by 1 simple consumer thread. All consumers stop to iterate after
iterate first several responses. The responses still return with bytes, but
cannot iterate.
On Thu, Oct 31, 2013 at 9:59 PM, Lu Xuechao wrote
A/FAQ#FAQ-Whybrokersdonotreceiveproducersentmessages%3F
> ?
>
> Thanks,
>
> Jun
>
>
> On Thu, Oct 31, 2013 at 2:23 PM, Lu Xuechao wrote:
>
> > It seems the reason is I enabled gzip compression.
> >
> > what the code would like to consume compressed messages?
> >
> > thank
It seems the reason is I enabled gzip compression.
what the code would like to consume compressed messages?
thanks.
On Thu, Oct 31, 2013 at 11:26 AM, Lu Xuechao wrote:
> Hi,
>
> I am following the
> https://cwiki.apache.org/confluence/display/KAFKA/0.8.0+SimpleConsumer+Exampl
Hi,
I am following the
https://cwiki.apache.org/confluence/display/KAFKA/0.8.0+SimpleConsumer+Example
When I send KeyedMessage with StringEncoder, I can get the
messages sent:
for (MessageAndOffset messageAndOffset : fetchResponse.messageSet(m_topic,
m_partition)) {
//handle messages
}
But whe
?
>
> Thanks,
> Neha
>
>
> On Mon, Oct 21, 2013 at 12:01 PM, Lu Xuechao wrote:
>
> > Hi,
> >
> > We observed below correlation between kafka configuration and performance
> > data:
> >
> > 1. producer insertion rate drops as compression enabled,
Hi,
We observed below correlation between kafka configuration and performance
data:
1. producer insertion rate drops as compression enabled, especially gzip;
2. when the producer batch size is below 200, we got high CPU/network IO on
brokers, got high network IO on producers/consumers;
What's th
Hi,
I wonder if it is supported to enable compression for storage on brokers
but the producer sends messages uncompressed and consumer receives messages
uncompressed?
thanks.
t;
> On Wed, Sep 11, 2013 at 9:59 PM, Lu Xuechao wrote:
>
> > Hi Team,
> >
> > I have some questions regarding Kafka partitions:
> >
> > 1. Based on my understanding, the partitions of the same broker have
> > contention on disk IO. Say If I have 10 hard d
Hi Team,
I have some questions regarding Kafka partitions:
1. Based on my understanding, the partitions of the same broker have
contention on disk IO. Say If I have 10 hard drives, can I specify all the
partitions spread evenly on those drives?
2. If I configure default.replication.factor=2, the
Hi, Joe. wiki updated. Hope it helps.
On Fri, Aug 30, 2013 at 3:22 PM, Joe Stein wrote:
> I feel like this is maybe a usual case as we have heard it before now a few
> bits
>
> Lu Xuechao would you mind updating the FAQ
> https://cwiki.apache.org/confluence/display/KAFKA/FA
Hi Jun,
Thanks for you help. Finally, I found the reason by enabling producer side
DEBUG info output. The snappy jar is not included in the classpath. Added
it and it worked.
Thanks again.
On Fri, Aug 30, 2013 at 12:53 PM, Lu Xuechao wrote:
> No.
>
>
> On Fri, Aug 30, 2013 at 1
No.
On Fri, Aug 30, 2013 at 11:57 AM, Jun Rao wrote:
> These are the metadata requests. Do you see Producer requests from your
> client?
>
> Thanks,
>
> Jun
>
>
> On Thu, Aug 29, 2013 at 5:40 PM, Lu Xuechao wrote:
>
> > After I sent 1,000 compressed even
Update: Sending compressed events with console producer works:
kafka-console-producer.bat --broker-list localhost:9092 --sync --topic
topic1 --compress
I am working on Windows 7.
On Fri, Aug 30, 2013 at 8:40 AM, Lu Xuechao wrote:
> After I sent 1,000 compressed events, I saw these messa
; Jun
>
>
> On Thu, Aug 29, 2013 at 6:29 AM, Lu Xuechao wrote:
>
> > Let me post my test code here. I could see producer.send(data); returned
> > with no error.
> >
> > public class TestProducer extends Thread {
> > private final Producer producer;
_request_required_acks = "1";
final static String p_producer_type = "async";
final static String p_batch_num = "100";
final static String p_compression_codec = "1";
final static String p_message_send_retries = "3";
final static
ponding errors in
> the broker logs? With the configuration below, I don't think any errors
> will be reported back to the producer.
>
> You could also try setting erquest.required.acks=1 to see if errors are
> reported back to the client.
>
> On 8/29/13 4:40 AM, "Lu Xuecha
Hi ,
I am trying to enable gzip compression for my events. But after I switched
compression.codec to "1" I found the produced events were even not be
persisted to disk log file. Of course, the consumer could not receive any
compressed events. I sent 10,000 or more events but the broker's log file
19 matches
Mail list logo
