Hadoop clusters running Hbase.
We also push all of our metrics from TCollector through Kafka and use
multiple consumers to push the metrics into a cluster of OpenTSDB server
sitting in multiple Hadoop clusters, using consumers in different consumer
groups.
- Eric Sites
On 8/15/13 12:41 AM, "
stuff around to place them in
storage, and run them through our meta data collection workers. 100s of
servers.
I am just trying to track down some memory issues on one class of workers
that store the whitelist files for testing our av engine definitions
before release.
Cheers,
Eric Sites
On 8/15
Correction: So if you have n consumers in the same group you need n
partitions per topic.
On 8/15/13 12:14 AM, "Eric Sites" wrote:
>Yes, that is the expected.
>
>
>You need more partitions per topic, at lest 1 partition per consumer in
>the group.
>
>So if you hav
Yes, that is the expected.
You need more partitions per topic, at lest 1 partition per consumer in
the group.
So if you have n consumers you need n topics.
All messages sent to the topic will be divided amongst the partitions.
Cheers,
Eric Sites
On 8/14/13 3:59 PM, "Wu, Jiang2 " w
.get(0);
ConsumerIterator it = stream.iterator();
while (it.hasNext() && !time_to_shutdown) {
Thanks,
Eric Sites
Responses inline
On 8/13/13 9:57 PM, "Philip O'Toole" wrote:
>My experience is solely with 0.72. More inline.
I am currently using 0.8.
>
>On Tue, Aug 13, 2013 at 6:47 PM, Eric Sites
>wrote:
>> Hello everyone,
>>
>> I have a very low volume topic
ble to see
them.
Thanks for the help...
Cheers,
Eric Sites
fka Log on Broker 4], Deleting log
segment 65478 from incoming-1 (kafka.log.Log)
[2013-08-12 17:23:59,637] INFO Deleting index
/mnt/data/kafka/incoming-1/00065478.index
(kafka.log.OffsetIndex)
Cheers,
Eric Sites
On 8/12/13 1:20 PM, "Joel Koshy" wrote:
>The consumer uses
-08-12 15:36:18,624] ERROR Producer connection to kafka002:9092
unsuccessful (kafka.producer.SyncProducer)
[2013-08-12 15:36:18,625] ERROR Producer connection to kafka006:9092
unsuccessful (kafka.producer.SyncProducer)
Thanks for the help...
Cheers,
Eric Sites
Fixed it.
An entire cluster restart was required.
Not the best solution, but all 6 servers, topics, partitions, and replicas
are in sync now.
And all topics have leaders.
Not I just need todo a little rebalancing.
Cheers,
Eric Sites
On 8/4/13 11:06 PM, "Eric Sites" wrote:
>A
servers, now it is showing no
leader.
Any help would be great.
Thanks,
Eric Sites
ordination mechanism on your
code to commitOffsets after every x number of messages.
Cheers,
Eric Sites
On 7/12/13 9:39 PM, "Florin Trofin" wrote:
>Thanks Jay, this is helpful information. Controlling when offsets are
>committed is possible only with the SimpleConsumer, co
Jason,
Did you update your config file with the new name of the zookeeper
settings:
It was renamed from zk.connect to zookeeper.connect.
You should check all of the settings because other setting names have
changed as well.
Cheers,
Eric Sites
On 6/16/13 5:14 PM, "Jason Rosenberg&qu
servers for your
firewall
The .1 after the server is the id of the zookeeper which should be stored
in a file
Named /tmp/zookeeper/myid
Echo "1" > /tmp/zookeeper/myid`
Cheers,
Eric Sites
On 4/23/13 9:50 AM, "Jason Huang" wrote:
>Thanks Chris and Neha.
>
>Chris
e information
>from zookeeper when given the from-beginning option. So effectively only
>the 2nd consumer instance is alive. You can run ConsumerShell instead.
>
>Thanks,
>Neha
>
>On Thursday, April 18, 2013, Eric Sites wrote:
>
>> Jun,
>>
>> I have been tryi
Jun,
I have been trying to test 0.8 using two consumers on the same group ID.
When I start the second consumer the first stops receiving messages.
Can you help?
Single Kafka server, single topic, 3 partitions, single Zookeeper.
1 console consumer running in one session.
1 console consumer run
16 matches
Mail list logo
