After some bit of digging realized that kylin by default was trying to do mapred local instead of on submitting to yarn clusters, fixed that with params in the kylin.properties and yarn-site.xml. These errors were not quite intuitive in the logs, prolly need more trace in debug mode.
Another issue is with default dfs port. Hadoop 2.5 onwards frequent usage is 9000. While older version defaults to 8020. In kylin 1.2 and 1.3 explicit override the dfs port dint work for me/ dint find the right property to set. Instead restart dfs to listen on 8020 instead of 9000. This solve the subsequent issue. FYI for those who are using this configuration Kylin 1.2 or 1.3 Hbase 1.1.X Hadoop 2.5.x or 2.6.x On Fri, Jan 8, 2016 at 1:32 PM, michael jones <[email protected]> wrote: > I am consistently getting NPE for any cube building on stage2 of the > process, for any cube. > The sessionId consistently comes as null. Most likely i am missing setting > somthing in the conf. Am using the default kylin.properties of prebuit > kylin 1.3 binary download. > > > +------------------------------------------------------------------------------------------------------+ > | Extract Fact Table Distinct Columns > | > > +------------------------------------------------------------------------------------------------------+ > 2016-01-08 13:26:24,895 INFO [pool-7-thread-1] Configuration.deprecation: > session.id is deprecated. Instead, use dfs.metrics.session-id > 2016-01-08 13:26:24,896 INFO [pool-7-thread-1] jvm.JvmMetrics: > Initializing JVM Metrics with processName=JobTracker, *sessionId=* > > ==> /opt/kylin/tomcat/logs/kylin_job.log <== > [pool-7-thread-1]:[2016-01-08 > 13:26:24,905][ERROR][org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:109)] > - error running Executable > java.lang.NullPointerException > at org.apache.kylin.job.common.MapReduceExecutable.onExecuteStart( > *MapReduceExecutable.java:79*) > at > org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:106) > at > org.apache.kylin.job.execution.DefaultChainedExecutable.doWork(DefaultChainedExecutable.java:51) > at > org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:107) > at > org.apache.kylin.job.impl.threadpool.DefaultScheduler$JobRunner.run(DefaultScheduler.java:130) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) > at java.lang.Thread.run(Thread.java:745) > > > > https://github.com/apache/kylin/blob/master/job/src/main/java/org/apache/kylin/job/common/MapReduceExecutable.java#L79 > > Configuration conf = HadoopUtil.getCurrentConfiguration(); > Job job = new Cluster(conf).getJob(JobID.forName(mrJobId)); > if (job.getJobState() == JobStatus.State.FAILED) { > //remove previous mr job info > super.onExecuteStart(executableContext); > } > > Either Kylin is not able to get the hadoop cluster or not able to submit > the job. > Anyone encountered this issue before ? I couldnt find in the docs on how > to yarn cluster endpoint properties in kylin.properties > > Thanks > > > > > > On Thu, Jan 7, 2016 at 4:59 PM, michael jones <[email protected]> > wrote: > >> Kyline - apache-kylin-1.3-HBase-1.1-SNAPSHOT-bin >> Hbase - 1.1.2 >> Hadoop - 2.7.1 >> Hive 1.2.1 - Updated hive metastore to use mysqldb (instead of derby) >> Hive/Hbase by itself work fine tested with created tables etc. >> >> Kylin Sandbox/Local setup. >> >> Kylin starts properly except for this error >> java.io.FileNotFoundException: >> /opt/hadoop-2.7.1/contrib/capacity-scheduler/*.jar (No such file or >> directory) >> (I figure this is ok to ignore since the default task scheduler in local >> mode should work fine ?) >> >> After running bin/sample.sh >> >> Tried to follow steps as mentioned in the tutorial, When building the >> cube, stage 1 successed. >> #2 Step Name: Extract Fact Table Distinct Columns >> Fails with NPE. >> >> -conf /opt/kylin/conf/kylin_job_conf.xml -cubename kylin_sales_cube >> -output >> /kylin/kylin_metadata/kylin-1575f8f6-98df-44da-8ac2-b480ccd0380a/kylin_sales_cube/fact_distinct_columns >> -jobname Kylin_Fact_Distinct_Columns_kylin_sales_cube_Step -tablename >> default.kylin_intermediate_kylin_sales_cube_desc_19700101000000_20160112000000_1575f8f6_98df_44da_8ac2_b480ccd0380a >> >> java.lang.NullPointerException >> at >> org.apache.kylin.job.common.MapReduceExecutable.onExecuteStart(MapReduceExecutable.java:79) >> at >> org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:106) >> at >> org.apache.kylin.job.execution.DefaultChainedExecutable.doWork(DefaultChainedExecutable.java:51) >> at >> org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:107) >> at >> org.apache.kylin.job.impl.threadpool.DefaultScheduler$JobRunner.run(DefaultScheduler.java:130) >> at >> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) >> at >> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) >> at java.lang.Thread.run(Thread.java:745) >> >> >> kylin.log >> >> [http-bio-7070-exec-4]:[2016-01-07 >> 15:31:37,797][DEBUG][org.apache.kylin.rest.filter.KylinApiFilter.logRequest(KylinApiFilter.java:120)] >> - REQUEST: REQUESTER=ADMIN;REQ_TIME=GMT-08:00 2016-01-07 >> 15:31:37;URI=/kylin/api/jobs/1575f8f6-98df-44da-8ac2-b480ccd0380a/resume;METHOD=PUT;QUERY_STRING=null;PAYLOAD=;RESP_STATUS=200; >> [pool-6-thread-1]:[2016-01-07 >> 15:32:17,880][INFO][org.apache.kylin.job.impl.threadpool.DefaultScheduler$FetcherRunner.run(DefaultScheduler.java:102)] >> - CubingJob{id=1575f8f6-98df-44da-8ac2-b480ccd0380a, name=kylin_sales_cube >> - 19700101000000_20160112000000 - BUILD - GMT-08:00 2016-01-07 14:39:59, >> state=READY} prepare to schedule >> [pool-6-thread-1]:[2016-01-07 >> 15:32:17,880][INFO][org.apache.kylin.job.impl.threadpool.DefaultScheduler$FetcherRunner.run(DefaultScheduler.java:106)] >> - CubingJob{id=1575f8f6-98df-44da-8ac2-b480ccd0380a, name=kylin_sales_cube >> - 19700101000000_20160112000000 - BUILD - GMT-08:00 2016-01-07 14:39:59, >> state=READY} scheduled >> +------------------------------------------------------------------------------------------------------+[pool-6-thread-1]:[2016-01-07 >> 15:32:17,880][INFO][org.apache.kylin.job.impl.threadpool.DefaultScheduler$FetcherRunner.run(DefaultScheduler.java:112)] >> - Job Fetcher: 0 running, 1 actual running, 1 ready, 0 others >> >> | kylin_sales_cube - 19700101000000_20160112000000 - BUILD - GMT-08:00 >> 2016-01-07 14:39:59 | >> >> +------------------------------------------------------------------------------------------------------+ >> [pool-7-thread-2]:[2016-01-07 >> 15:32:17,883][DEBUG][org.apache.kylin.common.persistence.ResourceStore.putResource(ResourceStore.java:200)] >> - Saving resource /execute_output/1575f8f6-98df-44da-8ac2-b480ccd0380a >> (Store kylin_metadata@hbase) >> [pool-7-thread-2]:[2016-01-07 >> 15:32:17,885][INFO][org.apache.kylin.job.manager.ExecutableManager.updateJobOutput(ExecutableManager.java:241)] >> - job id:1575f8f6-98df-44da-8ac2-b480ccd0380a from READY to RUNNING >> >> +------------------------------------------------------------------------------------------------------+ >> | Extract Fact Table Distinct Columns >> | >> >> +------------------------------------------------------------------------------------------------------+ >> 2016-01-07 15:32:17,917 INFO [pool-7-thread-2] jvm.JvmMetrics: Cannot >> initialize JVM Metrics with processName=JobTracker, sessionId= - already >> initialized >> [pool-7-thread-2]:[2016-01-07 >> 15:32:17,918][ERROR][org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:109)] >> - error running Executable >> java.lang.NullPointerException >> at >> org.apache.kylin.job.common.MapReduceExecutable.onExecuteStart(MapReduceExecutable.java:79) >> at >> org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:106) >> at >> org.apache.kylin.job.execution.DefaultChainedExecutable.doWork(DefaultChainedExecutable.java:51) >> at >> org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:107) >> at >> org.apache.kylin.job.impl.threadpool.DefaultScheduler$JobRunner.run(DefaultScheduler.java:130) >> at >> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) >> at >> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) >> at java.lang.Thread.run(Thread.java:745) >> [pool-7-thread-2]:[2016-01-07 >> 15:32:17,923][DEBUG][org.apache.kylin.common.persistence.ResourceStore.putResource(ResourceStore.java:200)] >> - Saving resource /execute_output/1575f8f6-98df-44da-8ac2-b480ccd0380a-01 >> (Store kylin_metadata@hbase) >> [pool-7-thread-2]:[2016-01-07 >> 15:32:17,927][DEBUG][org.apache.kylin.common.persistence.ResourceStore.putResource(ResourceStore.java:200)] >> - Saving resource /execute_output/1575f8f6-98df-44da-8ac2-b480ccd0380a-01 >> (Store kylin_metadata@hbase) >> [pool-7-thread-2]:[2016-01-07 >> 15:32:17,929][INFO][org.apache.kylin.job.manager.ExecutableManager.updateJobOutput(ExecutableManager.java:241)] >> - job id:1575f8f6-98df-44da-8ac2-b480ccd0380a-01 from READY to ERROR >> [pool-7-thread-2]:[2016-01-07 >> 15:32:17,929][ERROR][org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:109)] >> - error running Executable >> org.apache.kylin.job.exception.ExecuteException: >> java.lang.NullPointerException >> at >> org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:111) >> at >> org.apache.kylin.job.execution.DefaultChainedExecutable.doWork(DefaultChainedExecutable.java:51) >> at >> org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:107) >> at >> org.apache.kylin.job.impl.threadpool.DefaultScheduler$JobRunner.run(DefaultScheduler.java:130) >> at >> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) >> at >> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) >> at java.lang.Thread.run(Thread.java:745) >> Caused by: java.lang.NullPointerException >> at >> org.apache.kylin.job.common.MapReduceExecutable.onExecuteStart(MapReduceExecutable.java:79) >> at >> org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:106) >> ... 6 more >> [pool-7-thread-2]:[2016-01-07 >> 15:32:17,933][DEBUG][org.apache.kylin.common.persistence.ResourceStore.putResource(ResourceStore.java:200)] >> - Saving resource /execute_output/1575f8f6-98df-44da-8ac2-b480ccd0380a >> (Store kylin_metadata@hbase) >> [pool-7-thread-2]:[2016-01-07 >> 15:32:17,937][DEBUG][org.apache.kylin.common.persistence.ResourceStore.putResource(ResourceStore.java:200)] >> - Saving resource /execute_output/1575f8f6-98df-44da-8ac2-b480ccd0380a >> (Store kylin_metadata@hbase) >> [pool-7-thread-2]:[2016-01-07 >> 15:32:17,938][INFO][org.apache.kylin.job.manager.ExecutableManager.updateJobOutput(ExecutableManager.java:241)] >> - job id:1575f8f6-98df-44da-8ac2-b480ccd0380a from RUNNING to ERROR >> [pool-7-thread-2]:[2016-01-07 >> 15:32:17,938][ERROR][org.apache.kylin.job.impl.threadpool.DefaultScheduler$JobRunner.run(DefaultScheduler.java:134)] >> - ExecuteException job:1575f8f6-98df-44da-8ac2-b480ccd0380a >> org.apache.kylin.job.exception.ExecuteException: >> org.apache.kylin.job.exception.ExecuteException: >> java.lang.NullPointerException >> >> >> >> >> Am I missing something setup on kylin properties or missing jars ? >> >> >> >
