If that is the case, I guess that might still be some value to try to run
broker and clients locally and see if the issue still exist.
Thanks,
Jiangjie (Becket) Qin
On 7/15/15, 1:23 PM, "Vadim Bobrov" wrote:
>it is pretty random
>
>On Wed, Jul 15, 2015 at 4:22 PM, Jiangjie Qin
>wrote:
>
>> I’
it is pretty random
On Wed, Jul 15, 2015 at 4:22 PM, Jiangjie Qin
wrote:
> I’m not sure if it is related to running in cloud. Do you see this
> disconnection issue always happening on committing offsets or it happens
> randomly?
>
> Jiangjie (becket) qin
>
> On 7/15/15, 12:53 PM, "Vadim Bobrov"
I’m not sure if it is related to running in cloud. Do you see this
disconnection issue always happening on committing offsets or it happens
randomly?
Jiangjie (becket) qin
On 7/15/15, 12:53 PM, "Vadim Bobrov" wrote:
>there are lots of files under logs directory of the broker, just in case I
>ch
there are lots of files under logs directory of the broker, just in case I
checked all modified around the time of error and found nothing unusual
both client and broker are 0.8.2.1
could it have something to do with running it in the cloud? we are on
Linode and I remember having random disconnect
Is there anything on the broker log?
Is it possible that your client and broker are not running on the same
version?
Jiangjie (Becket) Qin
On 7/15/15, 11:40 AM, "Vadim Bobrov" wrote:
>caught it, thanks for help!
>any ideas what to do?
>
>TRACE 2015-07-15 18:37:58,070 [chaos-akka.actor.jms-dispa
caught it, thanks for help!
any ideas what to do?
TRACE 2015-07-15 18:37:58,070 [chaos-akka.actor.jms-dispatcher-1019 ]
kafka.network.BoundedByteBufferSend - 113 bytes written.
ERROR 2015-07-15 18:37:58,078 [chaos-akka.actor.jms-dispatcher-1019 ]
kafka.consumer.ZookeeperConsumerConnector -
[chaos-
thanks Joel and Jiangjie,
I have figured it out. In addition to my log4j 2 config file I also needed
a log4j 1 config file, then it works. Let me trace what happens when the
offsets are not committed and report back
On Wed, Jul 15, 2015 at 1:33 PM, Joel Koshy wrote:
> - You can also change the l
- You can also change the log4j level dynamically via the
kafka.Log4jController mbean.
- You can also look at offset commit request metrics (mbeans) on the
broker (just to check if _any_ offset commits are coming through
during the period you see no moving offsets).
- The alternative is to ju
I am not sure how your project was setup. But I think it depends on what
log4j property file you specified when you started your application. Can
you check if you have log4j appender defined and the loggers are directed
to the correct appender?
Thanks,
Jiangjie (Becket) Qin
On 7/15/15, 8:10 AM,
Thanks Jiangjie,
unfortunately turning trace level on does not seem to work (any log level
actually) I am using log4j2 (through slf4j) and despite including log4j1
bridge and these lines:
in my conf file I could not squeeze out any logging from kafka. Logging for
all other libs (like zookeeper
Hi Vadim,
Can you turn on trace level logging on your consumer and search for
"offset commit response² in the log?
Also maybe take a look at the log to see if there is any exception thrown.
Thanks,
Jiangjie (Becket) Qin
On 7/14/15, 11:06 AM, "Vadim Bobrov" wrote:
>just caught this error agai
just caught this error again. I issue commitOffsets - no error but no
committng offsets either. __consumer_offsets watching shows no new messages
either. Then in a few minutes I issue commitOffsets again - all committed.
Unless I am doing something terribly wrong this is very unreliable
On Tue, Ju
I am using ZookeeperConsumerConnector
actually I set up a consumer for __consumer_offsets the way you suggested
and now I cannot reproduce the situation any longer. Offsets are committed
every time.
On Tue, Jul 14, 2015 at 1:49 PM, Joel Koshy wrote:
> Actually, how are you committing offsets? A
Actually, how are you committing offsets? Are you using the old
(zookeeperconsumerconnector) or new KafkaConsumer?
It is true that the current APIs don't return any result, but it would
help to check if anything is getting into the offsets topic - unless
you are seeing errors in the logs, the offs
Thanks, Joel, I will but regardless of my findings the basic problem will
still be there: there is no guarantee that the offsets will be committed
after commitOffsets. Because commitOffsets does not return its exit status,
nor does it block as I understand until offsets are committed. In other
word
Can you take a look at the kafka commit rate mbean on your consumer?
Also, can you consume the offsets topic while you are committing
offsets and see if/what offsets are getting committed?
(http://www.slideshare.net/jjkoshy/offset-management-in-kafka/32)
Thanks,
Joel
On Tue, Jul 14, 2015 at 11:1
I am trying to replace ActiveMQ with Kafka in our environment however I
have encountered a strange problem that basically prevents from using Kafka
in production. The problem is that sometimes the offsets are not committed.
I am using Kafka 0.8.2.1, offset storage = kafka, high level consumer,
aut
I am trying to replace ActiveMQ with Kafka in our environment however I
have encountered a strange problem that basically prevents from using Kafka
in production. The problem is that sometimes the offsets are not committed.
I am using Kafka 0.8.2.1, offset storage = kafka, high level consumer,
aut
18 matches
Mail list logo
