Hi Luke / Shaofeng ,
Can you please help me to check this issue.
Regards,
Santosh Akhilesh
On Tue, Feb 24, 2015 at 10:41 PM, Santosh Akhilesh <
[email protected]> wrote:
> Hi All ,
> is it because of following error in map reduce job ? what could be
> way to resolve this , a google search says that its issue of Yarn class
> path , but I am not sure what it is ?
>
> Kylin Hive Column Cardinality Job table=RETAIL.FACT_SALES
> output=/tmp/cardinality/RETAIL.FACT_SALES
>
> Application application_1424791969399_0008 failed 2 times due to AM
> Container for appattempt_1424791969399_0008_000002 exited with exitCode: 1
> For more detailed output, check application tracking page:
> http://santosh:8088/proxy/application_1424791969399_0008/Then, click on
> links to logs of each attempt.
> Diagnostics: Exception from container-launch.
> Container id: container_1424791969399_0008_02_000001
> Exit code: 1
> Stack trace: ExitCodeException exitCode=1:
> at org.apache.hadoop.util.Shell.runCommand(Shell.java:538)
> at org.apache.hadoop.util.Shell.run(Shell.java:455)
> at
> org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:715)
> at
> org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor.launchContainer(DefaultContainerExecutor.java:211)
> at
> org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:302)
> at
> org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:82)
> at java.util.concurrent.FutureTask.run(FutureTask.java:262)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:745)
> Container exited with a non-zero exit code 1
> Failing this attempt. Failing the application.
>
> ---------- Forwarded message ----------
> From: Santoshakhilesh <[email protected]>
> Date: Tue, Feb 24, 2015 at 7:41 PM
> Subject: FW: Error while making cube & Measure option is not responding on
> GUI
> To: "[email protected]" <[email protected]>
>
>
> hi ,
> please someone give me a hand to resolve this issue , thanks.
>
> Regards,
> Santosh Akhilesh
> Bangalore R&D
> HUAWEI TECHNOLOGIES CO.,LTD.
>
> www.huawei.com
>
> -------------------------------------------------------------------------------------------------------------------------------------
> This e-mail and its attachments contain confidential information from
> HUAWEI, which
> is intended only for the person or entity whose address is listed above.
> Any use of the
> information contained herein in any way (including, but not limited to,
> total or partial
> disclosure, reproduction, or dissemination) by persons other than the
> intended
> recipient(s) is prohibited. If you receive this e-mail in error, please
> notify the sender by
> phone or email immediately and delete it!
>
> ________________________________________
> From: Santoshakhilesh [[email protected]]
> Sent: Tuesday, February 24, 2015 12:55 PM
> To: [email protected]
> Cc: Kulbhushan Rana
> Subject: FW: Error while making cube & Measure option is not responding on
> GUI
>
> 2. If I ignore and continue and try to save the cube I get an exception
> in Kylin.log , I have checked the path is set correctly and HCatInputFormat
> this file is present in hive-hcatalog-core-0.14.0.jar . Please let me know
> what can I do to resolve this ?
>
> -- This was path issue , now no more exception in kylin.log
>
> But saveing cube still fails with error. And still can't add measures.
>
> Error Message
> Failed to take action.
>
> In log I can find no exception. Following is the last log in kylin.log
>
> [pool-3-thread-1]:[2015-02-24
> 20:47:15,613][INFO][org.apache.kylin.job.impl.threadpool.DefaultScheduler$FetcherRunner.run(DefaultScheduler.java:117)]
> - Job Fetcher: 0 running, 0 actual running, 0 ready, 6 others
> [http-bio-7070-exec-2]:[2015-02-24
> 20:47:51,610][DEBUG][org.apache.kylin.rest.controller.CubeController.deserializeDataModelDesc(CubeController.java:459)]
> - Saving cube {
> "name": "",
> "fact_table": "RETAIL.FACT_SALES",
> "lookups": [],
> "filter_condition": "",
> "capacity": "SMALL",
> "partition_desc": {
> "partition_date_column": "",
> "partition_date_start": 0,
> "partition_type": "APPEND"
> },
> "last_modified": 0
> }
>
>
> local access logs all with 200 , so seems ok.
>
> 10.18.146.105 - - [24/Feb/2015:20:46:56 +0800] "GET
> /kylin/api/user/authentication HTTP/1.1" 200 246
> 10.18.146.105 - - [24/Feb/2015:20:47:07 +0800] "GET
> /kylin/api/user/authentication HTTP/1.1" 200 246
> 10.18.146.105 - - [24/Feb/2015:20:47:27 +0800] "GET
> /kylin/api/user/authentication HTTP/1.1" 200 246
> 10.18.146.105 - - [24/Feb/2015:20:47:28 +0800] "GET
> /kylin/api/user/authentication HTTP/1.1" 200 246
> 10.18.146.105 - - [24/Feb/2015:20:47:34 +0800] "GET
> /kylin/api/user/authentication HTTP/1.1" 200 246
> 10.18.146.105 - - [24/Feb/2015:20:47:48 +0800] "GET
> /kylin/api/user/authentication HTTP/1.1" 200 246
> 10.18.146.105 - - [24/Feb/2015:20:47:51 +0800] "POST /kylin/api/cubes
> HTTP/1.1" 200 701
>
>
> Regards,
> Santosh Akhilesh
> Bangalore R&D
> HUAWEI TECHNOLOGIES CO.,LTD.
>
> www.huawei.com
>
> -------------------------------------------------------------------------------------------------------------------------------------
> This e-mail and its attachments contain confidential information from
> HUAWEI, which
> is intended only for the person or entity whose address is listed above.
> Any use of the
> information contained herein in any way (including, but not limited to,
> total or partial
> disclosure, reproduction, or dissemination) by persons other than the
> intended
> recipient(s) is prohibited. If you receive this e-mail in error, please
> notify the sender by
> phone or email immediately and delete it!
>
> ________________________________________
> From: Santoshakhilesh [[email protected]]
> Sent: Tuesday, February 24, 2015 12:09 PM
> To: [email protected]
> Cc: Kulbhushan Rana
> Subject: Error while making cube & Measure option is not responding on GUI
>
> Hi All ,
>
> I am building a simple cube for test and using the binary build 0.7.1
> . I have following hive tables with columns.
>
>
>
> fact_sales:
>
> storeid int
> itemid int
> custid int
> qty int
> price double
>
> dim_customer
> custid int
> name string
>
> dim_item
>
> itemid int
> category string
> brand string
> color string
>
> dim_store
>
> storeid int
> city string
> state string
>
> Please help me to answer following issues;
>
>
>
> 1. When I go to measure section and click on measure option , there is no
> response , I want add measure on qty and price with sum
>
> 2. If I ignore and continue and try to save the cube I get an exception
> in Kylin.log , I have checked the path is set correctly and HCatInputFormat
> this file is present in hive-hcatalog-core-0.14.0.jar . Please let me know
> what can I do to resolve this ?
>
> 3. Also I have another question since this is a test and data is small I
> have not partitioned the fact table , is it ok to skip partition stage
> while cube build ?
>
>
>
> Exception
>
> pool-4-thread-4]:[2015-02-24
> 19:26:32,577][ERROR][org.apache.kylin.job.impl.threadpool.DefaultScheduler$JobRunner.run(DefaultScheduler.java:134)]
> - ExecuteException job:c3532a6f-97ea-474a-b36a-218dd517cedb
> org.apache.kylin.job.exception.ExecuteException:
> org.apache.kylin.job.exception.ExecuteException:
> java.lang.NoClassDefFoundError:
> org/apache/hive/hcatalog/mapreduce/HCatInputFormat
> at
> org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:102)
> at
> org.apache.kylin.job.impl.threadpool.DefaultScheduler$JobRunner.run(DefaultScheduler.java:132)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:745)
> Caused by: org.apache.kylin.job.exception.ExecuteException:
> java.lang.NoClassDefFoundError:
> org/apache/hive/hcatalog/mapreduce/HCatInputFormat
>
>
>
> The JSON is as below.
>
>
>
> {
> "name": "Retail_Cube",
> "description": "",
> "dimensions": [
> {
> "name": "RETAIL.FACT_SALES.STOREID",
> "table": "RETAIL.FACT_SALES",
> "hierarchy": false,
> "derived": null,
> "column": [
> "STOREID"
> ],
> "id": 1
> },
> {
> "name": "RETAIL.FACT_SALES.ITEMID",
> "table": "RETAIL.FACT_SALES",
> "hierarchy": false,
> "derived": null,
> "column": [
> "ITEMID"
> ],
> "id": 2
> },
> {
> "name": "RETAIL.FACT_SALES.CUSTID",
> "table": "RETAIL.FACT_SALES",
> "hierarchy": false,
> "derived": null,
> "column": [
> "CUSTID"
> ],
> "id": 3
> }
> ],
> "measures": [
> {
> "id": 1,
> "name": "_COUNT_",
> "function": {
> "expression": "COUNT",
> "returntype": "bigint",
> "parameter": {
> "type": "constant",
> "value": 1
> }
> }
> }
> ],
> "rowkey": {
> "rowkey_columns": [
> {
> "column": "STOREID",
> "length": 0,
> "dictionary": "true",
> "mandatory": false
> },
> {
> "column": "ITEMID",
> "length": 0,
> "dictionary": "true",
> "mandatory": false
> },
> {
> "column": "CUSTID",
> "length": 0,
> "dictionary": "true",
> "mandatory": false
> }
> ],
> "aggregation_groups": [
> [
> "STOREID",
> "ITEMID",
> "CUSTID"
> ]
> ]
> },
> "notify_list": [],
> "capacity": "",
> "hbase_mapping": {
> "column_family": [
> {
> "name": "f1",
> "columns": [
> {
> "qualifier": "m",
> "measure_refs": [
> "_COUNT_"
> ]
> }
> ]
> }
> ]
> },
> "project": "RetailProject",
> "model_name": "Retail_Cube"
> }
>
>
>
>
>
> Regards,
> Santosh Akhilesh
> Bangalore R&D
> HUAWEI TECHNOLOGIES CO.,LTD.
>
> www.huawei.com
>
> -------------------------------------------------------------------------------------------------------------------------------------
> This e-mail and its attachments contain confidential information from
> HUAWEI, which
> is intended only for the person or entity whose address is listed above.
> Any use of the
> information contained herein in any way (including, but not limited to,
> total or partial
> disclosure, reproduction, or dissemination) by persons other than the
> intended
> recipient(s) is prohibited. If you receive this e-mail in error, please
> notify the sender by
> phone or email immediately and delete it!
>
>
>
> --
> Regards,
> Santosh Akhilesh
> +91-0-9845482201
>
--
Regards,
Santosh Akhilesh
+91-0-9845482201
