Sharmadha Sainath created ATLAS-1077:
----------------------------------------
Summary: Kakfa exception during load testing in Atlas
Key: ATLAS-1077
URL: https://issues.apache.org/jira/browse/ATLAS-1077
Project: Atlas
Issue Type: Bug
Environment: Cluster set up :
Machine 1 : Atlas , Solr,
Machine 2 : Kafka , HBase
Machine 3 : Hive, client
Reporter: Sharmadha Sainath
5,00,000 Hive tables are created via hiveserver2 and ingested to Atlas. Only
4,18,655 tables are created in Atlas. 5,00,000 tables are created in Hive and
no exception is found in Hive. Zookeeper throws the following exception :
ERROR -
[atlas_cn058-10.l42scl.hortonworks.com-1469728635962-8a80738e_watcher_executor:]
~ [atlas_cn058-10.l42scl.hortonworks.com-1469728635962-8a80738e], error during
syncedRebalance (Logging$class:103)
kafka.common.ConsumerRebalanceFailedException:
atlas_cn058-10.l42scl.hortonworks.com-1469728635962-8a80738e can't rebalance
after 4 retries
at
kafka.consumer.ZookeeperConsumerConnector$ZKRebalancerListener.syncedRebalance(ZookeeperConsumerConnector.scala:670)
at
kafka.consumer.ZookeeperConsumerConnector$ZKRebalancerListener$$anon$2.run(ZookeeperConsumerConnector.scala:589)
the test is run with default settings :
atlas.kafka.zookeeper.session.timeout.ms=400
atlas.kafka.zookeeper.connection.timeout.ms=200
atlas.kafka.zookeeper.sync.time.ms=20
atlas.audit.zookeeper.session.timeout.ms=1000
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
