Hi Santosh, It looks like hadoop failed to execute some shell command in the container; You need dive into hadoop to see what¹s the concrete error. You can use yarn logs command to fetch all logs:
yarn logs -applicationId <app_id> On 2/25/15, 7:39 PM, "Santosh Akhilesh" <[email protected]> wrote: >Hi Luke / Shaofeng , > Can you please help me to check this issue. >Regards, >Santosh Akhilesh > >On Tue, Feb 24, 2015 at 10:41 PM, Santosh Akhilesh < >[email protected]> wrote: > >> Hi All , >> is it because of following error in map reduce job ? what could >>be >> way to resolve this , a google search says that its issue of Yarn class >> path , but I am not sure what it is ? >> >> Kylin Hive Column Cardinality Job table=RETAIL.FACT_SALES >> output=/tmp/cardinality/RETAIL.FACT_SALES >> >> Application application_1424791969399_0008 failed 2 times due to AM >> Container for appattempt_1424791969399_0008_000002 exited with >>exitCode: 1 >> For more detailed output, check application tracking page: >> http://santosh:8088/proxy/application_1424791969399_0008/Then, click on >> links to logs of each attempt. >> Diagnostics: Exception from container-launch. >> Container id: container_1424791969399_0008_02_000001 >> Exit code: 1 >> Stack trace: ExitCodeException exitCode=1: >> at org.apache.hadoop.util.Shell.runCommand(Shell.java:538) >> at org.apache.hadoop.util.Shell.run(Shell.java:455) >> at >> >>org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:715) >> at >> >>org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor.launch >>Container(DefaultContainerExecutor.java:211) >> at >> >>org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.Conta >>inerLaunch.call(ContainerLaunch.java:302) >> at >> >>org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.Conta >>inerLaunch.call(ContainerLaunch.java:82) >> at java.util.concurrent.FutureTask.run(FutureTask.java:262) >> at >> >>java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java >>:1145) >> at >> >>java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.jav >>a:615) >> at java.lang.Thread.run(Thread.java:745) >> Container exited with a non-zero exit code 1 >> Failing this attempt. Failing the application. >> >> ---------- Forwarded message ---------- >> From: Santoshakhilesh <[email protected]> >> Date: Tue, Feb 24, 2015 at 7:41 PM >> Subject: FW: Error while making cube & Measure option is not responding >>on >> GUI >> To: "[email protected]" <[email protected]> >> >> >> hi , >> please someone give me a hand to resolve this issue , thanks. >> >> Regards, >> Santosh Akhilesh >> Bangalore R&D >> HUAWEI TECHNOLOGIES CO.,LTD. >> >> www.huawei.com >> >> >>------------------------------------------------------------------------- >>------------------------------------------------------------ >> This e-mail and its attachments contain confidential information from >> HUAWEI, which >> is intended only for the person or entity whose address is listed above. >> Any use of the >> information contained herein in any way (including, but not limited to, >> total or partial >> disclosure, reproduction, or dissemination) by persons other than the >> intended >> recipient(s) is prohibited. If you receive this e-mail in error, please >> notify the sender by >> phone or email immediately and delete it! >> >> ________________________________________ >> From: Santoshakhilesh [[email protected]] >> Sent: Tuesday, February 24, 2015 12:55 PM >> To: [email protected] >> Cc: Kulbhushan Rana >> Subject: FW: Error while making cube & Measure option is not responding >>on >> GUI >> >> 2. If I ignore and continue and try to save the cube I get an exception >> in Kylin.log , I have checked the path is set correctly and >>HCatInputFormat >> this file is present in hive-hcatalog-core-0.14.0.jar . Please let me >>know >> what can I do to resolve this ? >> >> -- This was path issue , now no more exception in kylin.log >> >> But saveing cube still fails with error. And still can't add measures. >> >> Error Message >> Failed to take action. >> >> In log I can find no exception. Following is the last log in kylin.log >> >> [pool-3-thread-1]:[2015-02-24 >> >>20:47:15,613][INFO][org.apache.kylin.job.impl.threadpool.DefaultScheduler >>$FetcherRunner.run(DefaultScheduler.java:117)] >> - Job Fetcher: 0 running, 0 actual running, 0 ready, 6 others >> [http-bio-7070-exec-2]:[2015-02-24 >> >>20:47:51,610][DEBUG][org.apache.kylin.rest.controller.CubeController.dese >>rializeDataModelDesc(CubeController.java:459)] >> - Saving cube { >> "name": "", >> "fact_table": "RETAIL.FACT_SALES", >> "lookups": [], >> "filter_condition": "", >> "capacity": "SMALL", >> "partition_desc": { >> "partition_date_column": "", >> "partition_date_start": 0, >> "partition_type": "APPEND" >> }, >> "last_modified": 0 >> } >> >> >> local access logs all with 200 , so seems ok. >> >> 10.18.146.105 - - [24/Feb/2015:20:46:56 +0800] "GET >> /kylin/api/user/authentication HTTP/1.1" 200 246 >> 10.18.146.105 - - [24/Feb/2015:20:47:07 +0800] "GET >> /kylin/api/user/authentication HTTP/1.1" 200 246 >> 10.18.146.105 - - [24/Feb/2015:20:47:27 +0800] "GET >> /kylin/api/user/authentication HTTP/1.1" 200 246 >> 10.18.146.105 - - [24/Feb/2015:20:47:28 +0800] "GET >> /kylin/api/user/authentication HTTP/1.1" 200 246 >> 10.18.146.105 - - [24/Feb/2015:20:47:34 +0800] "GET >> /kylin/api/user/authentication HTTP/1.1" 200 246 >> 10.18.146.105 - - [24/Feb/2015:20:47:48 +0800] "GET >> /kylin/api/user/authentication HTTP/1.1" 200 246 >> 10.18.146.105 - - [24/Feb/2015:20:47:51 +0800] "POST /kylin/api/cubes >> HTTP/1.1" 200 701 >> >> >> Regards, >> Santosh Akhilesh >> Bangalore R&D >> HUAWEI TECHNOLOGIES CO.,LTD. >> >> www.huawei.com >> >> >>------------------------------------------------------------------------- >>------------------------------------------------------------ >> This e-mail and its attachments contain confidential information from >> HUAWEI, which >> is intended only for the person or entity whose address is listed above. >> Any use of the >> information contained herein in any way (including, but not limited to, >> total or partial >> disclosure, reproduction, or dissemination) by persons other than the >> intended >> recipient(s) is prohibited. If you receive this e-mail in error, please >> notify the sender by >> phone or email immediately and delete it! >> >> ________________________________________ >> From: Santoshakhilesh [[email protected]] >> Sent: Tuesday, February 24, 2015 12:09 PM >> To: [email protected] >> Cc: Kulbhushan Rana >> Subject: Error while making cube & Measure option is not responding on >>GUI >> >> Hi All , >> >> I am building a simple cube for test and using the binary build >>0.7.1 >> . I have following hive tables with columns. >> >> >> >> fact_sales: >> >> storeid int >> itemid int >> custid int >> qty int >> price double >> >> dim_customer >> custid int >> name string >> >> dim_item >> >> itemid int >> category string >> brand string >> color string >> >> dim_store >> >> storeid int >> city string >> state string >> >> Please help me to answer following issues; >> >> >> >> 1. When I go to measure section and click on measure option , there is >>no >> response , I want add measure on qty and price with sum >> >> 2. If I ignore and continue and try to save the cube I get an exception >> in Kylin.log , I have checked the path is set correctly and >>HCatInputFormat >> this file is present in hive-hcatalog-core-0.14.0.jar . Please let me >>know >> what can I do to resolve this ? >> >> 3. Also I have another question since this is a test and data is small I >> have not partitioned the fact table , is it ok to skip partition stage >> while cube build ? >> >> >> >> Exception >> >> pool-4-thread-4]:[2015-02-24 >> >>19:26:32,577][ERROR][org.apache.kylin.job.impl.threadpool.DefaultSchedule >>r$JobRunner.run(DefaultScheduler.java:134)] >> - ExecuteException job:c3532a6f-97ea-474a-b36a-218dd517cedb >> org.apache.kylin.job.exception.ExecuteException: >> org.apache.kylin.job.exception.ExecuteException: >> java.lang.NoClassDefFoundError: >> org/apache/hive/hcatalog/mapreduce/HCatInputFormat >> at >> >>org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecuta >>ble.java:102) >> at >> >>org.apache.kylin.job.impl.threadpool.DefaultScheduler$JobRunner.run(Defau >>ltScheduler.java:132) >> at >> >>java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java >>:1145) >> at >> >>java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.jav >>a:615) >> at java.lang.Thread.run(Thread.java:745) >> Caused by: org.apache.kylin.job.exception.ExecuteException: >> java.lang.NoClassDefFoundError: >> org/apache/hive/hcatalog/mapreduce/HCatInputFormat >> >> >> >> The JSON is as below. >> >> >> >> { >> "name": "Retail_Cube", >> "description": "", >> "dimensions": [ >> { >> "name": "RETAIL.FACT_SALES.STOREID", >> "table": "RETAIL.FACT_SALES", >> "hierarchy": false, >> "derived": null, >> "column": [ >> "STOREID" >> ], >> "id": 1 >> }, >> { >> "name": "RETAIL.FACT_SALES.ITEMID", >> "table": "RETAIL.FACT_SALES", >> "hierarchy": false, >> "derived": null, >> "column": [ >> "ITEMID" >> ], >> "id": 2 >> }, >> { >> "name": "RETAIL.FACT_SALES.CUSTID", >> "table": "RETAIL.FACT_SALES", >> "hierarchy": false, >> "derived": null, >> "column": [ >> "CUSTID" >> ], >> "id": 3 >> } >> ], >> "measures": [ >> { >> "id": 1, >> "name": "_COUNT_", >> "function": { >> "expression": "COUNT", >> "returntype": "bigint", >> "parameter": { >> "type": "constant", >> "value": 1 >> } >> } >> } >> ], >> "rowkey": { >> "rowkey_columns": [ >> { >> "column": "STOREID", >> "length": 0, >> "dictionary": "true", >> "mandatory": false >> }, >> { >> "column": "ITEMID", >> "length": 0, >> "dictionary": "true", >> "mandatory": false >> }, >> { >> "column": "CUSTID", >> "length": 0, >> "dictionary": "true", >> "mandatory": false >> } >> ], >> "aggregation_groups": [ >> [ >> "STOREID", >> "ITEMID", >> "CUSTID" >> ] >> ] >> }, >> "notify_list": [], >> "capacity": "", >> "hbase_mapping": { >> "column_family": [ >> { >> "name": "f1", >> "columns": [ >> { >> "qualifier": "m", >> "measure_refs": [ >> "_COUNT_" >> ] >> } >> ] >> } >> ] >> }, >> "project": "RetailProject", >> "model_name": "Retail_Cube" >> } >> >> >> >> >> >> Regards, >> Santosh Akhilesh >> Bangalore R&D >> HUAWEI TECHNOLOGIES CO.,LTD. >> >> www.huawei.com >> >> >>------------------------------------------------------------------------- >>------------------------------------------------------------ >> This e-mail and its attachments contain confidential information from >> HUAWEI, which >> is intended only for the person or entity whose address is listed above. >> Any use of the >> information contained herein in any way (including, but not limited to, >> total or partial >> disclosure, reproduction, or dissemination) by persons other than the >> intended >> recipient(s) is prohibited. If you receive this e-mail in error, please >> notify the sender by >> phone or email immediately and delete it! >> >> >> >> -- >> Regards, >> Santosh Akhilesh >> +91-0-9845482201 >> > > > >-- >Regards, >Santosh Akhilesh >+91-0-9845482201
