Raja, Can you find the machine running YARN ? Look for log files on that machine -- usually, they are under */var/log/hadoop-yarn* or */var/log/hadoop* or similar locations.
The files themselves will have names that vary depending on your installation; some examples: *yarn-<user>-resourcemanager-<host>.log* *hadoop-cmf-yarn-RESOURCEMANAGER-<host>.log.out* Look for lines similar to the following that reference your application id; they will tell you what containers were allocated on which nodes on behalf of this application and will show if the containers were successfully allocated or not. Ram ----------------------------------- 2016-05-24 02:53:42,636 INFO org.apache.hadoop.yarn.server.resourcemanager.RMAuditLogger: USER=<user> OPERATION=AM Allocated Container TARGET=SchedulerApp RESULT=SUCCESS APPID=application_1462948052533_0036 CONTAINERID=container_1462948052533_0036_01_022468 2016-05-24 02:53:42,636 INFO org.apache.hadoop.yarn.server.resourcemanager.scheduler.SchedulerNode: Assigned container container_1462948052533_0036_01_022468 of capacity <memory:1536, vCores:1> on host <host:port>, which has 9 containers, <memory:24064, vCores:9> used and <memory:180736, vCores:15> available after allocation 2016-05-24 02:53:42,636 INFO org.apache.hadoop.yarn.server.resourcemanager.rmcontainer.RMContainerImpl: container_1462948052533_0183_01_036933 Container Transitioned from ALLOCATED to ACQUIRED On Mon, Jun 13, 2016 at 8:46 AM, Raja.Aravapalli <[email protected] > wrote: > > I wrote a custom Hbase offset manager, and tried to set this for Kafka > input operator as below!! When I trigger the application with this setting, > application is staying long time in ACCEPTED state, and even it moves to > RUNNING state… all the operators are staying INACTIVE without processing > any records. > > > HBaseOffsetManager hbaseOffMgr = new HBaseOffsetManager(); > kafkaInput.setOffsetManager(hbaseOffMgr); > > > Any help will immensely help me. Thanks. > > > Regards, > Raja. > > From: "Raja.Aravapalli" <[email protected]> > Reply-To: "[email protected]" <[email protected]> > Date: Friday, June 10, 2016 at 5:07 PM > To: "[email protected]" <[email protected]> > Subject: Custom Kafka Offset Manager > > > Hi, > > If I write my own implementation of OffsetManager for kafka input > operator, > > How should I configure kafka input operator to use my custom build offset > manager ? I know about the property “OffsetManager”, But can someone please > share a small example of how to set the offset manager ? > > > Thanks a lot. > > > Regards, > Raja. >
