Re:Mapreduce Exception When cube builds at 3rd step
Hi Kanivel, Have you run a mapreduce job in your Hadoop env (like word count)? Does it have similar error? Best regards, Ni Chunen / George On 12/5/2019 19:16，Kanivel Jayapal wrote： Hi, I have setup hadoop multinode cluster through ambari .ambari running as master node and hive running in another aws instance.I have installed kylin in master node and configured hive instance in kylin.properties. I can create model,can create cube.But exception occur when build cube.Build fails at 3rd step.When I check logs it shows Mapreduce exception.2nd step also taking more time.I have 3 tables and each have 10 records.sales,customers,products.sales is a fact table.customers and products are dimesions table..Please look at the attached screenshots. The below error I got, org.apache.kylin.engine.mr.exception.MapReduceException: Counters: 13 Job Counters Failed map tasks=4 Killed reduce tasks=3 Launched map tasks=4 Other local map tasks=3 Data-local map tasks=1 Total time spent by all maps in occupied slots (ms)=194628 Total time spent by all reduces in occupied slots (ms)=0 Total time spent by all map tasks (ms)=27804 Total vcore-milliseconds taken by all map tasks=27804 Total megabyte-milliseconds taken by all map tasks=199299072 Map-Reduce Framework CPU time spent (ms)=0 Physical memory (bytes) snapshot=0 Virtual memory (bytes) snapshot=0 Job Diagnostics:Task failed task_1575385745059_0009_m_00 Job failed as tasks failed. failedMaps:1 failedReduces:0 killedMaps:0 killedReduces: 0 Failure task Diagnostics: [2019-12-04 07:44:23.184]Exception from container-launch. Container id: container_e03_1575385745059_0009_01_04 Exit code: 255 [2019-12-04 07:44:23.186]Container exited with a non-zero exit code 255. Error file: prelaunch.err. Last 4096 bytes of prelaunch.err : Last 4096 bytes of stderr : [2019-12-04 07:44:23.186]Container exited with a non-zero exit code 255. Error file: prelaunch.err. Last 4096 bytes of prelaunch.err : Last 4096 bytes of stderr : at org.apache.kylin.engine.mr.common.MapReduceExecutable.doWork(MapReduceExecutable.java:173) at org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:167) at org.apache.kylin.job.execution.DefaultChainedExecutable.doWork(DefaultChainedExecutable.java:71) at org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:167) at org.apache.kylin.job.impl.threadpool.DefaultScheduler$JobRunner.run(DefaultScheduler.java:114) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) Please help me.
Re: Releasing Apache Kylin v3.0-GA
Good news, I cannot wait to the next generation of Kylin. Best wishes, Xiaoxiang Yu 发件人: George Ni 答复: "u...@kylin.apache.org" 日期: 2019年12月6日 星期五 10:25 收件人: "u...@kylin.apache.org" , "dev@kylin.apache.org" 主题: Releasing Apache Kylin v3.0-GA Hi Community, As we have released v3.0-alpha, v3.0-alpha2, v3.0-beta, we have enough cofidence to release the GA version for v3.0 next week, and I’m planning to create a branch for its release. Detail features, improvements and bug fixes will come later, the main features are: 1. Realtime OLAP 2. Job scheduler with Apache Curator 3. User and user group management Please feel free to leave your comments here. - Best regards, Ni Chunen / George
Re: Releasing Apache Kylin v3.0-GA
+1 Long-expected version! > 在 2019年12月6日，11:01，Xiaoyuan Gu 写道： > > Great features! +1 > > > Bests, > Xiaoyuan Gu > > > > > At 2019-12-06 10:25:40, "George Ni" wrote: >> Hi Community, >> >> As we have released v3.0-alpha, v3.0-alpha2, v3.0-beta, we have enough >> cofidence to >> release the GA version for v3.0 next week, and I’m planning to create a >> branch for its release. >> >> Detail features, improvements and bug fixes will come later, the main >> features are: >> 1. Realtime OLAP >> 2. Job scheduler with Apache Curator >> 3. User and user group management >> >> Please feel free to leave your comments here. >> >> - >> >> Best regards, >> >> >> >> Ni Chunen / George
Re:Releasing Apache Kylin v3.0-GA
Great features! +1 Bests, Xiaoyuan Gu At 2019-12-06 10:25:40, "George Ni" wrote: >Hi Community, > >As we have released v3.0-alpha, v3.0-alpha2, v3.0-beta, we have enough >cofidence to >release the GA version for v3.0 next week, and I’m planning to create a >branch for its release. > >Detail features, improvements and bug fixes will come later, the main >features are: >1. Realtime OLAP >2. Job scheduler with Apache Curator >3. User and user group management > >Please feel free to leave your comments here. > >- > >Best regards, > > > >Ni Chunen / George
Releasing Apache Kylin v3.0-GA
Hi Community, As we have released v3.0-alpha, v3.0-alpha2, v3.0-beta, we have enough cofidence to release the GA version for v3.0 next week, and I’m planning to create a branch for its release. Detail features, improvements and bug fixes will come later, the main features are: 1. Realtime OLAP 2. Job scheduler with Apache Curator 3. User and user group management Please feel free to leave your comments here. - Best regards, Ni Chunen / George
Mapreduce Exception When cube builds at 3rd step
Hi, I have setup hadoop multinode cluster through ambari .ambari running as master node and hive running in another aws instance.I have installed kylin in master node and configured hive instance in kylin.properties. I can create model,can create cube.But exception occur when build cube.Build fails at 3rd step.When I check logs it shows Mapreduce exception.2nd step also taking more time.I have 3 tables and each have 10 records.sales,customers,products.sales is a fact table.customers and products are dimesions table..Please look at the attached screenshots. The below error I got, org.apache.kylin.engine.mr.exception.MapReduceException: Counters: 13 Job Counters Failed map tasks=4 Killed reduce tasks=3 Launched map tasks=4 Other local map tasks=3 Data-local map tasks=1 Total time spent by all maps in occupied slots (ms)=194628 Total time spent by all reduces in occupied slots (ms)=0 Total time spent by all map tasks (ms)=27804 Total vcore-milliseconds taken by all map tasks=27804 Total megabyte-milliseconds taken by all map tasks=199299072 Map-Reduce Framework CPU time spent (ms)=0 Physical memory (bytes) snapshot=0 Virtual memory (bytes) snapshot=0 Job Diagnostics:Task failed task_1575385745059_0009_m_00 Job failed as tasks failed. failedMaps:1 failedReduces:0 killedMaps:0 killedReduces: 0 Failure task Diagnostics: [2019-12-04 07:44:23.184]Exception from container-launch. Container id: container_e03_1575385745059_0009_01_04 Exit code: 255 [2019-12-04 07:44:23.186]Container exited with a non-zero exit code 255. Error file: prelaunch.err. Last 4096 bytes of prelaunch.err : Last 4096 bytes of stderr : [2019-12-04 07:44:23.186]Container exited with a non-zero exit code 255. Error file: prelaunch.err. Last 4096 bytes of prelaunch.err : Last 4096 bytes of stderr : at org.apache.kylin.engine.mr.common.MapReduceExecutable.doWork(MapReduceExecutable.java:173) at org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:167) at org.apache.kylin.job.execution.DefaultChainedExecutable.doWork(DefaultChainedExecutable.java:71) at org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:167) at org.apache.kylin.job.impl.threadpool.DefaultScheduler$JobRunner.run(DefaultScheduler.java:114) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) Please help me.
[jira] [Created] (KYLIN-4279) Mapreduce Exception when building cube at step 3
Kanivel created KYLIN-4279: -- Summary: Mapreduce Exception when building cube at step 3 Key: KYLIN-4279 URL: https://issues.apache.org/jira/browse/KYLIN-4279 Project: Kylin Issue Type: Bug Components: Tools, Build and Test Affects Versions: v2.6.4 Reporter: Kanivel Attachments: Screenshot from 2019-12-05 12-55-22.png, Screenshot from 2019-12-05 12-55-47.png I have setup hadoop multinode cluster through ambari .ambari running as master node and hive running in another aws instance.I have installed kylin in master node and configured hive instance in kylin.properties. I can create model,can create cube.But exception occur when build cube.Build fails at 3rd step.When I check logs it shows Mapreduce exception.2nd step also taking more time.I have 3 tables and each have 10 records.sales,customers,products.sales is a fact table.customers and products are dimesions table..Please look at the attached screenshots. The below error I got, org.apache.kylin.engine.mr.exception.MapReduceException: Counters: 13 Job Counters Failed map tasks=4 Killed reduce tasks=3 Launched map tasks=4 Other local map tasks=3 Data-local map tasks=1 Total time spent by all maps in occupied slots (ms)=194628 Total time spent by all reduces in occupied slots (ms)=0 Total time spent by all map tasks (ms)=27804 Total vcore-milliseconds taken by all map tasks=27804 Total megabyte-milliseconds taken by all map tasks=199299072 Map-Reduce Framework CPU time spent (ms)=0 Physical memory (bytes) snapshot=0 Virtual memory (bytes) snapshot=0 Job Diagnostics:Task failed task_1575385745059_0009_m_00 Job failed as tasks failed. failedMaps:1 failedReduces:0 killedMaps:0 killedReduces: 0 Failure task Diagnostics: [2019-12-04 07:44:23.184]Exception from container-launch. Container id: container_e03_1575385745059_0009_01_04 Exit code: 255 [2019-12-04 07:44:23.186]Container exited with a non-zero exit code 255. Error file: prelaunch.err. Last 4096 bytes of prelaunch.err : Last 4096 bytes of stderr : [2019-12-04 07:44:23.186]Container exited with a non-zero exit code 255. Error file: prelaunch.err. Last 4096 bytes of prelaunch.err : Last 4096 bytes of stderr : at org.apache.kylin.engine.mr.common.MapReduceExecutable.doWork(MapReduceExecutable.java:173) at org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:167) at org.apache.kylin.job.execution.DefaultChainedExecutable.doWork(DefaultChainedExecutable.java:71) at org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:167) at org.apache.kylin.job.impl.threadpool.DefaultScheduler$JobRunner.run(DefaultScheduler.java:114) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) -- This message was sent by Atlassian Jira (v8.3.4#803005)
