According to our experience: you can try those :
1、use newer hive to promote the first step.
2、startup more mapper and reducer for every MR job, you can reduce the
value of 'kylin.job.mapreduce.default.reduce.input.mb' in kylin.properties
which means input size for every reducer in NDCuboid calculation steps.
smaller value means more reducer.
3、 you can set the property
'mapreduce.input.fileinputformat.split.maxsize'('mapred.max.split.size' in
prior hadoop version) in kylin_job_conf.xml, which means the max split size
of a mapper, we set the value less than block size of hadoop cluster, such
as 64MB
4、 try to set cube size as SMALL while creating cube, which can increase
reducer number while generate Hfile.Hope it is helpful to you . 2016-01-08 13:00 GMT+08:00 [email protected] <[email protected]>: > I have five machines (8 core, 32g MEM), using HDP 2.3 building cluster > environment, version of the kyling Kyline > apache-kylin-1.3-HBase-1.1-SNAPSHOT-bin, HBase for Version 1.1.1, hive > table data is now 30000000 ，but now job running the one hour, job schedule > is about 10%, view the task of MR found that job is not running to MR Do > you have any way to improve the performance of the job： > this is my configure： > 1.kylin.properties > # > # Licensed to the Apache Software Foundation (ASF) under one or more > # contributor license agreements. See the NOTICE file distributed with > # this work for additional information regarding copyright ownership. > # The ASF licenses this file to You under the Apache License, Version 2.0 > # (the "License"); you may not use this file except in compliance with > # the License. You may obtain a copy of the License at > # > # http://www.apache.org/licenses/LICENSE-2.0 > # > # Unless required by applicable law or agreed to in writing, software > # distributed under the License is distributed on an "AS IS" BASIS, > # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. > # See the License for the specific language governing permissions and > # limitations under the License. > # > > ## Config for Kylin Engine ## > > # List of web servers in use, this enables one web server instance to sync > up with other servers. > kylin.rest.servers=192.168.1.40:7070 > > #set display timezone on UI,format like[GMT+N or GMT-N] > kylin.rest.timezone=GMT-8 > kylin.query.cache.enabled=true > # The metadata store in hbase > kylin.metadata.url=kylin_metadata@hbase > > # The storage for final cube file in hbase > kylin.storage.url=hbase > kylin.job.yarn.app.rest.check.status.url= > http://192.168.1.40:8088/ws/v1/cluster/apps/${job_id}? > kylin.job.yarn.app.rest.check.interval.seconds=20 > kylin.query.security.enabled=false > # Temp folder in hdfs, make sure user has the right access to the hdfs > directory > kylin.hdfs.working.dir=/kylin > > # HBase Cluster FileSystem, which serving hbase, format as > hdfs://hbase-cluster:8020 > # leave empty if hbase running on same cluster with hive and mapreduce > kylin.hbase.cluster.fs=hdfs://mycluster/apps/hbase/data > kylin.route.hive.enabled=true > kylin.route.hive.url=jdbc:hive2://192.168.1.50:10000 > > kylin.job.mapreduce.default.reduce.input.mb=500 > > kylin.server.mode=all > > # If true, job engine will not assume that hadoop CLI reside on the same > server as it self > # you will have to specify kylin.job.remote.cli.hostname, > kylin.job.remote.cli.username and kylin.job.remote.cli.password > # It should not be set to "true" unless you're NOT running Kylin.sh on a > hadoop client machine > # (Thus kylin instance has to ssh to another real hadoop client machine to > execute hbase,hive,hadoop commands) > kylin.job.run.as.remote.cmd=false > > # Only necessary when kylin.job.run.as.remote.cmd=true > kylin.job.remote.cli.hostname= > > # Only necessary when kylin.job.run.as.remote.cmd=true > kylin.job.remote.cli.username= > > # Only necessary when kylin.job.run.as.remote.cmd=true > kylin.job.remote.cli.password= > > # Used by test cases to prepare synthetic data for sample cube > kylin.job.remote.cli.working.dir=/tmp/kylin > > # Max count of concurrent jobs running > kylin.job.concurrent.max.limit=10 > > # Time interval to check hadoop job status > kylin.job.yarn.app.rest.check.interval.seconds=10 > > # Hive database name for putting the intermediate flat tables > #kylin.job.hive.database.for.intermediatetable=kylin > > #default compression codec for htable,snappy,lzo,gzip,lz4 > kylin.hbase.default.compression.codec=snappy > > # The cut size for hbase region, in GB. > # E.g, for cube whose capacity be marked as "SMALL", split region per 10GB > by default > kylin.hbase.region.cut.small=10 > kylin.hbase.region.cut.medium=20 > kylin.hbase.region.cut.large=100 > > # HBase min and max region count > kylin.hbase.region.count.min=1 > kylin.hbase.region.count.max=500 > > ## Config for Restful APP ## > # database connection settings: > ldap.server= > ldap.username= > ldap.password= > ldap.user.searchBase= > ldap.user.searchPattern= > ldap.user.groupSearchBase= > ldap.service.searchBase=OU= > ldap.service.searchPattern= > ldap.service.groupSearchBase= > acl.adminRole= > acl.defaultRole= > ganglia.group= > ganglia.port=8664 > > ## Config for mail service > > # If true, will send email notification; > mail.enabled=false > mail.host= > mail.username= > mail.password= > mail.sender= > > ###########################config info for web####################### > > #help info ,format{name|displayName|link} ,optional > kylin.web.help.length=4 > kylin.web.help.0=start|Getting Started| > kylin.web.help.1=odbc|ODBC Driver| > kylin.web.help.2=tableau|Tableau Guide| > kylin.web.help.3=onboard|Cube Design Tutorial| > #hadoop url link ,optional > kylin.web.hadoop= > #job diagnostic url link ,optional > kylin.web.diagnostic= > #contact mail on web page ,optional > kylin.web.contact_mail= > > ###########################config info for front####################### > > #env DEV|QA|PROD > deploy.env=PROD > > ###########################config info for sandbox####################### > kylin.sandbox=true > > > ###########################config info for kylin > monitor####################### > # hive jdbc url > kylin.monitor.hive.jdbc.connection.url=jdbc:hive2://192.168.1.12:10000 > > #config where to parse query log,split with comma ,will also read > $KYLIN_HOME/tomcat/logs/ by default > kylin.monitor.ext.log.base.dir = /tmp/kylin_log1,/tmp/kylin_log2 > > #will create external hive table to query result csv file > #will set to kylin_query_log by default if not config here > kylin.monitor.query.log.parse.result.table = kylin_query_log > > > 2.kylin_job_conf.xml > <?xml version="1.0"?> > <!-- > Licensed under the Apache License, Version 2.0 (the "License"); > you may not use this file except in compliance with the License. > You may obtain a copy of the License at > > http://www.apache.org/licenses/LICENSE-2.0 > > Unless required by applicable law or agreed to in writing, software > distributed under the License is distributed on an "AS IS" BASIS, > WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. > See the License for the specific language governing permissions and > limitations under the License. See accompanying LICENSE file. > --> > > <configuration> > > <property> > <name>mapreduce.job.split.metainfo.maxsize</name> > <value>-1</value> > <description>The maximum permissible size of the split metainfo > file. > The JobTracker won't attempt to read split metainfo files > bigger than > the configured value. No limits if set to -1. > </description> > </property> > > <property> > <name>mapred.compress.map.output</name> > <value>true</value> > <description>Compress map outputs</description> > </property> > > <property> > <name>mapred.map.output.compression.codec</name> > <value>org.apache.hadoop.io.compress.SnappyCodec</value> > <description>The compression codec to use for map outputs > </description> > </property> > > <property> > <name>mapred.output.compress</name> > <value>true</value> > <description>Compress the output of a MapReduce job</description> > </property> > > <property> > <name>mapred.output.compression.codec</name> > <value>org.apache.hadoop.io.compress.SnappyCodec</value> > <description>The compression codec to use for job outputs > </description> > </property> > > <property> > <name>mapred.output.compression.type</name> > <value>BLOCK</value> > <description>The compression type to use for job > outputs</description> > </property> > > <property> > <name>mapreduce.job.max.split.locations</name> > <value>2000</value> > <description>No description</description> > </property> > > <property> > <name>dfs.replication</name> > <value>1</value> > <description>Block replication</description> > </property> > > > <property> > <name>hive.merge.mapfiles</name> > <value>true</value> > <description>Enable hive file merge on mapper only > job</description> > </property> > <property> > <name>hive.merge.mapredfiles</name> > <value>true</value> > <description>Enable hive file merge on map-reduce job</description> > </property> > <property> > <name>hive.merge.size.per.task</name> > <value>268435456</value> > <description>Size for the merged file: 256M</description> > </property> > > <property> > <name>hive.support.concurrency</name> > <value>false</value> > <description>Hive concurrency lock</description> > </property> > </configuration> > > > 3.debug： > [pool-7-thread-8]:[2016-01-08 > 12:48:18,488][DEBUG][org.apache.kylin.common.persistence.ResourceStore.putResource(ResourceStore.java:200)] > - Saving resource /execute_output/12226fd3-4750-4a55-8fa0-bd24b039c834-01 > (Store kylin_metadata@hbase) > [pool-7-thread-8]:[2016-01-08 > 12:48:38,500][WARN][org.apache.commons.httpclient.HttpMethodBase.getResponseBody(HttpMethodBase.java:682)] > - Going to buffer response body of large or unknown size. Using > getResponseBodyAsStream instead is recommended. > [pool-7-thread-8]:[2016-01-08 > 12:48:38,501][DEBUG][org.apache.kylin.job.tools.HadoopStatusGetter.getHttpResponse(HadoopStatusGetter.java:110)] > - Job job_1452156670999_0116 get status check result. > > [pool-7-thread-8]:[2016-01-08 > 12:48:38,501][DEBUG][org.apache.kylin.job.tools.HadoopStatusChecker.checkStatus(HadoopStatusChecker.java:57)] > - State of Hadoop job: job_1452156670999_0116:RUNNING-UNDEFINED > [pool-7-thread-8]:[2016-01-08 > 12:48:38,504][DEBUG][org.apache.kylin.common.persistence.ResourceStore.putResource(ResourceStore.java:200)] > - Saving resource /execute_output/12226fd3-4750-4a55-8fa0-bd24b039c834-01 > (Store kylin_metadata@hbase) > [pool-7-thread-8]:[2016-01-08 > 12:48:58,523][WARN][org.apache.commons.httpclient.HttpMethodBase.getResponseBody(HttpMethodBase.java:682)] > - Going to buffer response body of large or unknown size. Using > getResponseBodyAsStream instead is recommended. > [pool-7-thread-8]:[2016-01-08 > 12:48:58,523][DEBUG][org.apache.kylin.job.tools.HadoopStatusGetter.getHttpResponse(HadoopStatusGetter.java:110)] > - Job job_1452156670999_0116 get status check result. > > [pool-7-thread-8]:[2016-01-08 > 12:48:58,523][DEBUG][org.apache.kylin.job.tools.HadoopStatusChecker.checkStatus(HadoopStatusChecker.java:57)] > - State of Hadoop job: job_1452156670999_0116:RUNNING-UNDEFINED > [pool-7-thread-8]:[2016-01-08 > 12:48:58,526][DEBUG][org.apache.kylin.common.persistence.ResourceStore.putResource(ResourceStore.java:200)] > - Saving resource /execute_output/12226fd3-4750-4a55-8fa0-bd24b039c834-01 > (Store kylin_metadata@hbase) > [pool-6-thread-1]:[2016-01-08 > 12:49:08,108][INFO][org.apache.kylin.job.impl.threadpool.DefaultScheduler$FetcherRunner.run(DefaultScheduler.java:112)] > - Job Fetcher: 3 running, 3 actual running, 0 ready, 28 others > [pool-7-thread-8]:[2016-01-08 > 12:49:18,538][WARN][org.apache.commons.httpclient.HttpMethodBase.getResponseBody(HttpMethodBase.java:682)] > - Going to buffer response body of large or unknown size. Using > getResponseBodyAsStream instead is recommended. > [pool-7-thread-8]:[2016-01-08 > 12:49:18,538][DEBUG][org.apache.kylin.job.tools.HadoopStatusGetter.getHttpResponse(HadoopStatusGetter.java:110)] > - Job job_1452156670999_0116 get status check result. > > [pool-7-thread-8]:[2016-01-08 > 12:49:18,539][DEBUG][org.apache.kylin.job.tools.HadoopStatusChecker.checkStatus(HadoopStatusChecker.java:57)] > - State of Hadoop job: job_1452156670999_0116:RUNNING-UNDEFINED > [pool-7-thread-8]:[2016-01-08 > 12:49:18,542][DEBUG][org.apache.kylin.common.persistence.ResourceStore.putResource(ResourceStore.java:200)] > - Saving resource /execute_output/12226fd3-4750-4a55-8fa0-bd24b039c834-01 > (Store kylin_metadata@hbase) > [pool-7-thread-8]:[2016-01-08 > 12:49:38,555][WARN][org.apache.commons.httpclient.HttpMethodBase.getResponseBody(HttpMethodBase.java:682)] > - Going to buffer response body of large or unknown size. Using > getResponseBodyAsStream instead is recommended. > [pool-7-thread-8]:[2016-01-08 > 12:49:38,555][DEBUG][org.apache.kylin.job.tools.HadoopStatusGetter.getHttpResponse(HadoopStatusGetter.java:110)] > - Job job_1452156670999_0116 get status check result. > > [pool-7-thread-8]:[2016-01-08 > 12:49:38,556][DEBUG][org.apache.kylin.job.tools.HadoopStatusChecker.checkStatus(HadoopStatusChecker.java:57)] > - State of Hadoop job: job_1452156670999_0116:RUNNING-UNDEFINED > [pool-7-thread-8]:[2016-01-08 > 12:49:38,558][DEBUG][org.apache.kylin.common.persistence.ResourceStore.putResource(ResourceStore.java:200)] > - Saving resource /execute_output/12226fd3-4750-4a55-8fa0-bd24b039c834-01 > (Store kylin_metadata@hbase) > [pool-7-thread-8]:[2016-01-08 > 12:49:58,571][WARN][org.apache.commons.httpclient.HttpMethodBase.getResponseBody(HttpMethodBase.java:682)] > - Going to buffer response body of large or unknown size. Using > getResponseBodyAsStream instead is recommended. > [pool-7-thread-8]:[2016-01-08 > 12:49:58,571][DEBUG][org.apache.kylin.job.tools.HadoopStatusGetter.getHttpResponse(HadoopStatusGetter.java:110)] > - Job job_1452156670999_0116 get status check result. > > [pool-7-thread-8]:[2016-01-08 > 12:49:58,572][DEBUG][org.apache.kylin.job.tools.HadoopStatusChecker.checkStatus(HadoopStatusChecker.java:57)] > - State of Hadoop job: job_1452156670999_0116:RUNNING-UNDEFINED > [pool-7-thread-8]:[2016-01-08 > 12:49:58,574][DEBUG][org.apache.kylin.common.persistence.ResourceStore.putResource(ResourceStore.java:200)] > - Saving resource /execute_output/12226fd3-4750-4a55-8fa0-bd24b039c834-01 > (Store kylin_metadata@hbase) > [pool-6-thread-1]:[2016-01-08 > 12:50:08,118][INFO][org.apache.kylin.job.impl.threadpool.DefaultScheduler$FetcherRunner.run(DefaultScheduler.java:112)] > - Job Fetcher: 3 running, 3 actual running, 0 ready, 28 others > [pool-7-thread-8]:[2016-01-08 > 12:50:18,594][WARN][org.apache.commons.httpclient.HttpMethodBase.getResponseBody(HttpMethodBase.java:682)] > - Going to buffer response body of large or unknown size. Using > getResponseBodyAsStream instead is recommended. > [pool-7-thread-8]:[2016-01-08 > 12:50:18,594][DEBUG][org.apache.kylin.job.tools.HadoopStatusGetter.getHttpResponse(HadoopStatusGetter.java:110)] > - Job job_1452156670999_0116 get status check result. > > [pool-7-thread-8]:[2016-01-08 > 12:50:18,595][DEBUG][org.apache.kylin.job.tools.HadoopStatusChecker.checkStatus(HadoopStatusChecker.java:57)] > - State of Hadoop job: job_1452156670999_0116:RUNNING-UNDEFINED > [pool-7-thread-8]:[2016-01-08 > 12:50:18,597][DEBUG][org.apache.kylin.common.persistence.ResourceStore.putResource(ResourceStore.java:200)] > - Saving resource /execute_output/12226fd3-4750-4a55-8fa0-bd24b039c834-01 > (Store kylin_metadata@hbase) > [pool-7-thread-8]:[2016-01-08 > 12:50:38,609][WARN][org.apache.commons.httpclient.HttpMethodBase.getResponseBody(HttpMethodBase.java:682)] > - Going to buffer response body of large or unknown size. Using > getResponseBodyAsStream instead is recommended. > [pool-7-thread-8]:[2016-01-08 > 12:50:38,610][DEBUG][org.apache.kylin.job.tools.HadoopStatusGetter.getHttpResponse(HadoopStatusGetter.java:110)] > - Job job_1452156670999_0116 get status check result. > > [pool-7-thread-8]:[2016-01-08 > 12:50:38,610][DEBUG][org.apache.kylin.job.tools.HadoopStatusChecker.checkStatus(HadoopStatusChecker.java:57)] > - State of Hadoop job: job_1452156670999_0116:RUNNING-UNDEFINED > [pool-7-thread-8]:[2016-01-08 > 12:50:38,613][DEBUG][org.apache.kylin.common.persistence.ResourceStore.putResource(ResourceStore.java:200)] > - Saving resource /execute_output/12226fd3-4750-4a55-8fa0-bd24b039c834-01 > (Store kylin_metadata@hbase) > [pool-7-thread-8]:[2016-01-08 > 12:50:58,627][WARN][org.apache.commons.httpclient.HttpMethodBase.getResponseBody(HttpMethodBase.java:682)] > - Going to buffer response body of large or unknown size. Using > getResponseBodyAsStream instead is recommended. > [pool-7-thread-8]:[2016-01-08 > 12:50:58,627][DEBUG][org.apache.kylin.job.tools.HadoopStatusGetter.getHttpResponse(HadoopStatusGetter.java:110)] > - Job job_1452156670999_0116 get status check result. > > [pool-7-thread-8]:[2016-01-08 > 12:50:58,627][DEBUG][org.apache.kylin.job.tools.HadoopStatusChecker.checkStatus(HadoopStatusChecker.java:57)] > - State of Hadoop job: job_1452156670999_0116:RUNNING-UNDEFINED > [pool-7-thread-8]:[2016-01-08 > 12:50:58,630][DEBUG][org.apache.kylin.common.persistence.ResourceStore.putResource(ResourceStore.java:200)] > - Saving resource /execute_output/12226fd3-4750-4a55-8fa0-bd24b039c834-01 > (Store kylin_metadata@hbase) > [pool-6-thread-1]:[2016-01-08 > 12:51:08,111][INFO][org.apache.kylin.job.impl.threadpool.DefaultScheduler$FetcherRunner.run(DefaultScheduler.java:112)] > - Job Fetcher: 3 running, 3 actual running, 0 ready, 28 others > [http-bio-7070-exec-2]:[2016-01-08 > 12:51:11,522][DEBUG][org.apache.kylin.rest.filter.KylinApiFilter.logRequest(KylinApiFilter.java:120)] > - REQUEST: REQUESTER=ADMIN;REQ_TIME=GMT-08:00 2016-01-07 > 20:51:11;URI=/kylin/api/jobs;METHOD=GET;QUERY_STRING=limit=15&offset=0&projectName=learn_kylin;PAYLOAD=;RESP_STATUS=200; > [http-bio-7070-exec-2]:[2016-01-08 > 12:51:13,793][DEBUG][org.apache.kylin.rest.filter.KylinApiFilter.logRequest(KylinApiFilter.java:120)] > - REQUEST: REQUESTER=ADMIN;REQ_TIME=GMT-08:00 2016-01-07 > 20:51:13;URI=/kylin/api/jobs;METHOD=GET;QUERY_STRING=limit=15&offset=0&projectName=learn_kylin;PAYLOAD=;RESP_STATUS=200; > [http-bio-7070-exec-2]:[2016-01-08 > 12:51:14,431][DEBUG][org.apache.kylin.rest.filter.KylinApiFilter.logRequest(KylinApiFilter.java:120)] > - REQUEST: REQUESTER=ADMIN;REQ_TIME=GMT-08:00 2016-01-07 > 20:51:14;URI=/kylin/api/jobs;METHOD=GET;QUERY_STRING=limit=15&offset=0&projectName=learn_kylin;PAYLOAD=;RESP_STATUS=200; > [pool-7-thread-8]:[2016-01-08 > 12:51:18,643][WARN][org.apache.commons.httpclient.HttpMethodBase.getResponseBody(HttpMethodBase.java:682)] > - Going to buffer response body of large or unknown size. Using > getResponseBodyAsStream instead is recommended. > [pool-7-thread-8]:[2016-01-08 > 12:51:18,643][DEBUG][org.apache.kylin.job.tools.HadoopStatusGetter.getHttpResponse(HadoopStatusGetter.java:110)] > - Job job_1452156670999_0116 get status check result. > > [pool-7-thread-8]:[2016-01-08 > 12:51:18,644][DEBUG][org.apache.kylin.job.tools.HadoopStatusChecker.checkStatus(HadoopStatusChecker.java:57)] > - State of Hadoop job: job_1452156670999_0116:RUNNING-UNDEFINED > [pool-7-thread-8]:[2016-01-08 > 12:51:18,647][DEBUG][org.apache.kylin.common.persistence.ResourceStore.putResource(ResourceStore.java:200)] > - Saving resource /execute_output/12226fd3-4750-4a55-8fa0-bd24b039c834-01 > (Store kylin_metadata@hbase) > [pool-7-thread-8]:[2016-01-08 > 12:51:38,658][WARN][org.apache.commons.httpclient.HttpMethodBase.getResponseBody(HttpMethodBase.java:682)] > - Going to buffer response body of large or unknown size. Using > getResponseBodyAsStream instead is recommended. > [pool-7-thread-8]:[2016-01-08 > 12:51:38,659][DEBUG][org.apache.kylin.job.tools.HadoopStatusGetter.getHttpResponse(HadoopStatusGetter.java:110)] > - Job job_1452156670999_0116 get status check result. > > [pool-7-thread-8]:[2016-01-08 > 12:51:38,659][DEBUG][org.apache.kylin.job.tools.HadoopStatusChecker.checkStatus(HadoopStatusChecker.java:57)] > - State of Hadoop job: job_1452156670999_0116:RUNNING-UNDEFINED > [pool-7-thread-8]:[2016-01-08 > 12:51:38,662][DEBUG][org.apache.kylin.common.persistence.ResourceStore.putResource(ResourceStore.java:200)] > - Saving resource /execute_output/12226fd3-4750-4a55-8fa0-bd24b039c834-01 > (Store kylin_metadata@hbase) > [pool-7-thread-8]:[2016-01-08 > 12:51:58,674][WARN][org.apache.commons.httpclient.HttpMethodBase.getResponseBody(HttpMethodBase.java:682)] > - Going to buffer response body of large or unknown size. Using > getResponseBodyAsStream instead is recommended. > [pool-7-thread-8]:[2016-01-08 > 12:51:58,674][DEBUG][org.apache.kylin.job.tools.HadoopStatusGetter.getHttpResponse(HadoopStatusGetter.java:110)] > - Job job_1452156670999_0116 get status check result. > > [pool-7-thread-8]:[2016-01-08 > 12:51:58,675][DEBUG][org.apache.kylin.job.tools.HadoopStatusChecker.checkStatus(HadoopStatusChecker.java:57)] > - State of Hadoop job: job_1452156670999_0116:RUNNING-UNDEFINED > [pool-7-thread-8]:[2016-01-08 > 12:51:58,677][DEBUG][org.apache.kylin.common.persistence.ResourceStore.putResource(ResourceStore.java:200)] > - Saving resource /execute_output/12226fd3-4750-4a55-8fa0-bd24b039c834-01 > (Store kylin_metadata@hbase) > [pool-6-thread-1]:[2016-01-08 > 12:52:08,118][INFO][org.apache.kylin.job.impl.threadpool.DefaultScheduler$FetcherRunner.run(DefaultScheduler.java:112)] > - Job Fetcher: 3 running, 3 actual running, 0 ready, 28 others > [pool-7-thread-8]:[2016-01-08 > 12:52:18,696][WARN][org.apache.commons.httpclient.HttpMethodBase.getResponseBody(HttpMethodBase.java:682)] > - Going to buffer response body of large or unknown size. Using > getResponseBodyAsStream instead is recommended. > [pool-7-thread-8]:[2016-01-08 > 12:52:18,697][DEBUG][org.apache.kylin.job.tools.HadoopStatusGetter.getHttpResponse(HadoopStatusGetter.java:110)] > - Job job_1452156670999_0116 get status check result. > > [pool-7-thread-8]:[2016-01-08 > 12:52:18,697][DEBUG][org.apache.kylin.job.tools.HadoopStatusChecker.checkStatus(HadoopStatusChecker.java:57)] > - State of Hadoop job: job_1452156670999_0116:RUNNING-UNDEFINED > [pool-7-thread-8]:[2016-01-08 > 12:52:18,700][DEBUG][org.apache.kylin.common.persistence.ResourceStore.putResource(ResourceStore.java:200)] > - Saving resource /execute_output/12226fd3-4750-4a55-8fa0-bd24b039c834-01 > (Store kylin_metadata@hbase) > [pool-7-thread-8]:[2016-01-08 > 12:52:38,712][WARN][org.apache.commons.httpclient.HttpMethodBase.getResponseBody(HttpMethodBase.java:682)] > - Going to buffer response body of large or unknown size. Using > getResponseBodyAsStream instead is recommended. > [pool-7-thread-8]:[2016-01-08 > 12:52:38,713][DEBUG][org.apache.kylin.job.tools.HadoopStatusGetter.getHttpResponse(HadoopStatusGetter.java:110)] > - Job job_1452156670999_0116 get status check result. > > [pool-7-thread-8]:[2016-01-08 > 12:52:38,713][DEBUG][org.apache.kylin.job.tools.HadoopStatusChecker.checkStatus(HadoopStatusChecker.java:57)] > - State of Hadoop job: job_1452156670999_0116:RUNNING-UNDEFINED > [pool-7-thread-8]:[2016-01-08 > 12:52:38,716][DEBUG][org.apache.kylin.common.persistence.ResourceStore.putResource(ResourceStore.java:200)] > - Saving resource /execute_output/12226fd3-4750-4a55-8fa0-bd24b039c834-01 > (Store kylin_metadata@hbase) > [pool-7-thread-8]:[2016-01-08 > 12:52:58,728][WARN][org.apache.commons.httpclient.HttpMethodBase.getResponseBody(HttpMethodBase.java:682)] > - Going to buffer response body of large or unknown size. Using > getResponseBodyAsStream instead is recommended. > [pool-7-thread-8]:[2016-01-08 > 12:52:58,728][DEBUG][org.apache.kylin.job.tools.HadoopStatusGetter.getHttpResponse(HadoopStatusGetter.java:110)] > - Job job_1452156670999_0116 get status check result. > > [pool-7-thread-8]:[2016-01-08 > 12:52:58,729][DEBUG][org.apache.kylin.job.tools.HadoopStatusChecker.checkStatus(HadoopStatusChecker.java:57)] > - State of Hadoop job: job_1452156670999_0116:RUNNING-UNDEFINED > [pool-7-thread-8]:[2016-01-08 > 12:52:58,731][DEBUG][org.apache.kylin.common.persistence.ResourceStore.putResource(ResourceStore.java:200)] > - Saving resource /execute_output/12226fd3-4750-4a55-8fa0-bd24b039c834-01 > (Store kylin_metadata@hbase) > [pool-6-thread-1]:[2016-01-08 > 12:53:08,104][INFO][org.apache.kylin.job.impl.threadpool.DefaultScheduler$FetcherRunner.run(DefaultScheduler.java:112)] > - Job Fetcher: 3 running, 3 actual running, 0 ready, 28 others > [pool-7-thread-8]:[2016-01-08 > 12:53:18,744][WARN][org.apache.commons.httpclient.HttpMethodBase.getResponseBody(HttpMethodBase.java:682)] > - Going to buffer response body of large or unknown size. Using > getResponseBodyAsStream instead is recommended. > [pool-7-thread-8]:[2016-01-08 > 12:53:18,744][DEBUG][org.apache.kylin.job.tools.HadoopStatusGetter.getHttpResponse(HadoopStatusGetter.java:110)] > - Job job_1452156670999_0116 get status check result. > > [pool-7-thread-8]:[2016-01-08 > 12:53:18,745][DEBUG][org.apache.kylin.job.tools.HadoopStatusChecker.checkStatus(HadoopStatusChecker.java:57)] > - State of Hadoop job: job_1452156670999_0116:RUNNING-UNDEFINED > [pool-7-thread-8]:[2016-01-08 > 12:53:18,747][DEBUG][org.apache.kylin.common.persistence.ResourceStore.putResource(ResourceStore.java:200)] > - Saving resource /execute_output/12226fd3-4750-4a55-8fa0-bd24b039c834-01 > (Store kylin_metadata@hbase) > [pool-7-thread-8]:[2016-01-08 > 12:53:38,760][WARN][org.apache.commons.httpclient.HttpMethodBase.getResponseBody(HttpMethodBase.java:682)] > - Going to buffer response body of large or unknown size. Using > getResponseBodyAsStream instead is recommended. > [pool-7-thread-8]:[2016-01-08 > 12:53:38,760][DEBUG][org.apache.kylin.job.tools.HadoopStatusGetter.getHttpResponse(HadoopStatusGetter.java:110)] > - Job job_1452156670999_0116 get status check result. > > [pool-7-thread-8]:[2016-01-08 > 12:53:38,761][DEBUG][org.apache.kylin.job.tools.HadoopStatusChecker.checkStatus(HadoopStatusChecker.java:57)] > - State of Hadoop job: job_1452156670999_0116:RUNNING-UNDEFINED > [pool-7-thread-8]:[2016-01-08 > 12:53:38,764][DEBUG][org.apache.kylin.common.persistence.ResourceStore.putResource(ResourceStore.java:200)] > - Saving resource /execute_output/12226fd3-4750-4a55-8fa0-bd24b039c834-01 > (Store kylin_metadata@hbase) > > > > [email protected] >
