Hi All ,
is it because of following error in map reduce job ? what could be
way to resolve this , a google search says that its issue of Yarn class
path , but I am not sure what it is ?
Kylin Hive Column Cardinality Job table=RETAIL.FACT_SALES
output=/tmp/cardinality/RETAIL.FACT_SALES
Application application_1424791969399_0008 failed 2 times due to AM
Container for appattempt_1424791969399_0008_000002 exited with exitCode: 1
For more detailed output, check application tracking page:
http://santosh:8088/proxy/application_1424791969399_0008/Then, click on
links to logs of each attempt.
Diagnostics: Exception from container-launch.
Container id: container_1424791969399_0008_02_000001
Exit code: 1
Stack trace: ExitCodeException exitCode=1:
at org.apache.hadoop.util.Shell.runCommand(Shell.java:538)
at org.apache.hadoop.util.Shell.run(Shell.java:455)
at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:715)
at
org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor.launchContainer(DefaultContainerExecutor.java:211)
at
org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:302)
at
org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:82)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Container exited with a non-zero exit code 1
Failing this attempt. Failing the application.
---------- Forwarded message ----------
From: Santoshakhilesh <[email protected]>
Date: Tue, Feb 24, 2015 at 7:41 PM
Subject: FW: Error while making cube & Measure option is not responding on
GUI
To: "[email protected]" <[email protected]>
hi ,
please someone give me a hand to resolve this issue , thanks.
Regards,
Santosh Akhilesh
Bangalore R&D
HUAWEI TECHNOLOGIES CO.,LTD.
www.huawei.com
-------------------------------------------------------------------------------------------------------------------------------------
This e-mail and its attachments contain confidential information from
HUAWEI, which
is intended only for the person or entity whose address is listed above.
Any use of the
information contained herein in any way (including, but not limited to,
total or partial
disclosure, reproduction, or dissemination) by persons other than the
intended
recipient(s) is prohibited. If you receive this e-mail in error, please
notify the sender by
phone or email immediately and delete it!
________________________________________
From: Santoshakhilesh [[email protected]]
Sent: Tuesday, February 24, 2015 12:55 PM
To: [email protected]
Cc: Kulbhushan Rana
Subject: FW: Error while making cube & Measure option is not responding on
GUI
2. If I ignore and continue and try to save the cube I get an exception in
Kylin.log , I have checked the path is set correctly and HCatInputFormat
this file is present in hive-hcatalog-core-0.14.0.jar . Please let me know
what can I do to resolve this ?
-- This was path issue , now no more exception in kylin.log
But saveing cube still fails with error. And still can't add measures.
Error Message
Failed to take action.
In log I can find no exception. Following is the last log in kylin.log
[pool-3-thread-1]:[2015-02-24
20:47:15,613][INFO][org.apache.kylin.job.impl.threadpool.DefaultScheduler$FetcherRunner.run(DefaultScheduler.java:117)]
- Job Fetcher: 0 running, 0 actual running, 0 ready, 6 others
[http-bio-7070-exec-2]:[2015-02-24
20:47:51,610][DEBUG][org.apache.kylin.rest.controller.CubeController.deserializeDataModelDesc(CubeController.java:459)]
- Saving cube {
"name": "",
"fact_table": "RETAIL.FACT_SALES",
"lookups": [],
"filter_condition": "",
"capacity": "SMALL",
"partition_desc": {
"partition_date_column": "",
"partition_date_start": 0,
"partition_type": "APPEND"
},
"last_modified": 0
}
local access logs all with 200 , so seems ok.
10.18.146.105 - - [24/Feb/2015:20:46:56 +0800] "GET
/kylin/api/user/authentication HTTP/1.1" 200 246
10.18.146.105 - - [24/Feb/2015:20:47:07 +0800] "GET
/kylin/api/user/authentication HTTP/1.1" 200 246
10.18.146.105 - - [24/Feb/2015:20:47:27 +0800] "GET
/kylin/api/user/authentication HTTP/1.1" 200 246
10.18.146.105 - - [24/Feb/2015:20:47:28 +0800] "GET
/kylin/api/user/authentication HTTP/1.1" 200 246
10.18.146.105 - - [24/Feb/2015:20:47:34 +0800] "GET
/kylin/api/user/authentication HTTP/1.1" 200 246
10.18.146.105 - - [24/Feb/2015:20:47:48 +0800] "GET
/kylin/api/user/authentication HTTP/1.1" 200 246
10.18.146.105 - - [24/Feb/2015:20:47:51 +0800] "POST /kylin/api/cubes
HTTP/1.1" 200 701
Regards,
Santosh Akhilesh
Bangalore R&D
HUAWEI TECHNOLOGIES CO.,LTD.
www.huawei.com
-------------------------------------------------------------------------------------------------------------------------------------
This e-mail and its attachments contain confidential information from
HUAWEI, which
is intended only for the person or entity whose address is listed above.
Any use of the
information contained herein in any way (including, but not limited to,
total or partial
disclosure, reproduction, or dissemination) by persons other than the
intended
recipient(s) is prohibited. If you receive this e-mail in error, please
notify the sender by
phone or email immediately and delete it!
________________________________________
From: Santoshakhilesh [[email protected]]
Sent: Tuesday, February 24, 2015 12:09 PM
To: [email protected]
Cc: Kulbhushan Rana
Subject: Error while making cube & Measure option is not responding on GUI
Hi All ,
I am building a simple cube for test and using the binary build 0.7.1 .
I have following hive tables with columns.
fact_sales:
storeid int
itemid int
custid int
qty int
price double
dim_customer
custid int
name string
dim_item
itemid int
category string
brand string
color string
dim_store
storeid int
city string
state string
Please help me to answer following issues;
1. When I go to measure section and click on measure option , there is no
response , I want add measure on qty and price with sum
2. If I ignore and continue and try to save the cube I get an exception in
Kylin.log , I have checked the path is set correctly and HCatInputFormat
this file is present in hive-hcatalog-core-0.14.0.jar . Please let me know
what can I do to resolve this ?
3. Also I have another question since this is a test and data is small I
have not partitioned the fact table , is it ok to skip partition stage
while cube build ?
Exception
pool-4-thread-4]:[2015-02-24
19:26:32,577][ERROR][org.apache.kylin.job.impl.threadpool.DefaultScheduler$JobRunner.run(DefaultScheduler.java:134)]
- ExecuteException job:c3532a6f-97ea-474a-b36a-218dd517cedb
org.apache.kylin.job.exception.ExecuteException:
org.apache.kylin.job.exception.ExecuteException:
java.lang.NoClassDefFoundError:
org/apache/hive/hcatalog/mapreduce/HCatInputFormat
at
org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:102)
at
org.apache.kylin.job.impl.threadpool.DefaultScheduler$JobRunner.run(DefaultScheduler.java:132)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: org.apache.kylin.job.exception.ExecuteException:
java.lang.NoClassDefFoundError:
org/apache/hive/hcatalog/mapreduce/HCatInputFormat
The JSON is as below.
{
"name": "Retail_Cube",
"description": "",
"dimensions": [
{
"name": "RETAIL.FACT_SALES.STOREID",
"table": "RETAIL.FACT_SALES",
"hierarchy": false,
"derived": null,
"column": [
"STOREID"
],
"id": 1
},
{
"name": "RETAIL.FACT_SALES.ITEMID",
"table": "RETAIL.FACT_SALES",
"hierarchy": false,
"derived": null,
"column": [
"ITEMID"
],
"id": 2
},
{
"name": "RETAIL.FACT_SALES.CUSTID",
"table": "RETAIL.FACT_SALES",
"hierarchy": false,
"derived": null,
"column": [
"CUSTID"
],
"id": 3
}
],
"measures": [
{
"id": 1,
"name": "_COUNT_",
"function": {
"expression": "COUNT",
"returntype": "bigint",
"parameter": {
"type": "constant",
"value": 1
}
}
}
],
"rowkey": {
"rowkey_columns": [
{
"column": "STOREID",
"length": 0,
"dictionary": "true",
"mandatory": false
},
{
"column": "ITEMID",
"length": 0,
"dictionary": "true",
"mandatory": false
},
{
"column": "CUSTID",
"length": 0,
"dictionary": "true",
"mandatory": false
}
],
"aggregation_groups": [
[
"STOREID",
"ITEMID",
"CUSTID"
]
]
},
"notify_list": [],
"capacity": "",
"hbase_mapping": {
"column_family": [
{
"name": "f1",
"columns": [
{
"qualifier": "m",
"measure_refs": [
"_COUNT_"
]
}
]
}
]
},
"project": "RetailProject",
"model_name": "Retail_Cube"
}
Regards,
Santosh Akhilesh
Bangalore R&D
HUAWEI TECHNOLOGIES CO.,LTD.
www.huawei.com
-------------------------------------------------------------------------------------------------------------------------------------
This e-mail and its attachments contain confidential information from
HUAWEI, which
is intended only for the person or entity whose address is listed above.
Any use of the
information contained herein in any way (including, but not limited to,
total or partial
disclosure, reproduction, or dissemination) by persons other than the
intended
recipient(s) is prohibited. If you receive this e-mail in error, please
notify the sender by
phone or email immediately and delete it!
--
Regards,
Santosh Akhilesh
+91-0-9845482201
