[jira] [Created] (KYLIN-1802) CLONE - protobuf jar file conflict with other

2016-06-16 Thread zhaotianshuo (JIRA) 
zhaotianshuo created KYLIN-1802:
---

 Summary: CLONE - protobuf jar file conflict with other
 Key: KYLIN-1802
 URL: https://issues.apache.org/jira/browse/KYLIN-1802
 Project: Kylin
  Issue Type: Bug
Affects Versions: v1.4.0
Reporter: zhaotianshuo
Assignee: Shaofeng SHI


I query a sql in kylin-2.0, and error happen, here is the log :
[http-bio-17070-exec-1]:[2016-01-04 
13:30:13,488][ERROR][org.apache.kylin.rest.controller.BasicController.handleError(BasicController.java:44)]
 - 
org.apache.kylin.rest.exception.InternalErrorException: loader constraint 
violation: loader (instance of org/apache/catalina/loader/WebappClassLoader) 
previously initiated loading for a different type with name 
"com/google/protobuf/Message"
at 
org.apache.kylin.rest.controller.QueryController.doQueryWithCache(QueryController.java:212)
at 
org.apache.kylin.rest.controller.QueryController.query(QueryController.java:95)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at 
org.springframework.web.method.support.InvocableHandlerMethod.invoke(InvocableHandlerMethod.java:213)
at 
org.springframework.web.method.support.InvocableHandlerMethod.invokeForRequest(InvocableHandlerMethod.java:126)
at 
org.springframework.web.servlet.mvc.method.annotation.ServletInvocableHandlerMethod.invokeAndHandle(ServletInvocableHandlerMethod.java:96)
at 
org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.invokeHandlerMethod(RequestMappingHandlerAdapter.java:617)
at 
org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.handleInternal(RequestMappingHandlerAdapter.java:578)
at 
org.springframework.web.servlet.mvc.method.AbstractHandlerMethodAdapter.handle(AbstractHandlerMethodAdapter.java:80)
at 
org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:923)
at 
org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:852)
at 
org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:882)
at 
org.springframework.web.servlet.FrameworkServlet.doPost(FrameworkServlet.java:789)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:646)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:727)
at 
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:303)
at 
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
at 
org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:52)
at 
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)
at 
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)

I delete protobuf-java-2.5.0.jar file in WEB-INF/lib and restart, it works fine.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (KYLIN-1800) IllegalArgumentExceptio: Too many digits for NumberDictionary: -0.009999999999877218. Expect 19 digits before decimal point at max.

2016-06-16 Thread lx (JIRA) 
lx created KYLIN-1800:
-

 Summary: IllegalArgumentExceptio: Too many digits for 
NumberDictionary: -0.0099877218. Expect 19 digits before decimal point 
at max.
 Key: KYLIN-1800
 URL: https://issues.apache.org/jira/browse/KYLIN-1800
 Project: Kylin
  Issue Type: Bug
  Components: General
Affects Versions: v1.5.2
Reporter: lx






--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Re: properties in kylin configuration files

2016-06-16 Thread Jie Tao 

I found the reason, but do not know the solution:

in the 36000seconds, the mapper does only one thing:

016-06-16 16:39:08,622 DEBUG [IPC Client (1991820533) connection to 
/10.106.4.2:49222 from job_1466061903809_0024] org.apache.hadoop.ipc.Client: 
IPC Client (1991820533) connection to /10.106.4.2:49222 from 
job_1466061903809_0024 got value #211
2016-06-16 16:39:08,622 DEBUG [communication thread] org.apache.hadoop.ipc.RPC: 
Call: ping 1
2016-06-16 16:39:11,623 DEBUG [IPC Parameter Sending Thread #0] 
org.apache.hadoop.ipc.Client: IPC Client (1991820533) connection to 
/10.106.4.2:49222 from job_1466061903809_0024 sending #212
2016-06-16 16:39:11,624 DEBUG [IPC Client (1991820533) connection to 
/10.106.4.2:49222 from job_1466061903809_0024] org.apache.hadoop.ipc.Client: 
IPC Client (1991820533) connection to /10.106.4.2:49222 from 
job_1466061903809_0024 got value #212
2016-06-16 16:39:11,624 DEBUG [communication thread] org.apache.hadoop.ipc.RPC: 
Call: ping 1
2016-06-16 16:39:14,625 DEBUG [IPC Parameter Sending Thread #0] 
org.apache.hadoop.ipc.Client: IPC Client (1991820533) connection to 
/10.106.4.2:49222 from job_1466061903809_0024 sending #213
2016-06-16 16:39:14,626 DEBUG [IPC Client (1991820533) connection to 
/10.106.4.2:49222 from job_1466061903809_0024] org.apache.hadoop.ipc.Client: 
IPC Client (1991820533) connection to /10.106.4.2:49222 from 
job_1466061903809_0024 got value #213
2016-06-16 16:39:14,626 DEBUG [communication thread] org.apache.hadoop.ipc.RPC: 
Call: ping 1
2016-06-16 16:39:17,627 DEBUG [IPC Parameter Sending Thread #0] 
org.apache.hadoop.ipc.Client: IPC Client (1991820533) connection to 
/10.106.4.2:49222 from job_1466061903809_0024 sending #214
2016-06-16 16:39:17,628 DEBUG [IPC Client (1991820533) connection to 
/10.106.4.2:49222 from job_1466061903809_0024] org.apache.hadoop.ipc.Client: 
IPC Client (1991820533) connection to /10.106.4.2:49222 from 
job_1466061903809_0024 got value #214
2016-06-16 16:39:17,628 DEBUG [communication thread] org.apache.hadoop.ipc.RPC: 
Call: ping 1
2016-06-16 16:39:20,629 DEBUG [IPC Parameter Sending Thread #0] 
org.apache.hadoop.ipc.Client: IPC Client (1991820533) connection to 
/10.106.4.2:49222 from job_1466061903809_0024 sending #215
2016-06-16 16:39:20,630 DEBUG [IPC Client (1991820533) connection to 
/10.106.4.2:49222 from job_1466061903809_0024] org.apache.hadoop.ipc.Client: 
IPC Client (1991820533) connection to /10.106.4.2:49222 from 
job_1466061903809_0024 got value #215
2016-06-16 16:39:20,630 DEBUG [communication thread] org.apache.hadoop.ipc.RPC: 
Call: ping 1
2016-06-16 16:39:23,631 DEBUG [IPC Parameter Sending Thread #0] 
org.apache.hadoop.ipc.Client: IPC Client (1991820533) connection to 
/10.106.4.2:49222 from job_1466061903809_0024 sending #216
2016-06-16 16:39:23,632 DEBUG [IPC Client (1991820533) connection to 
/10.106.4.2:49222 from job_1466061903809_0024] org.apache.hadoop.ipc.Client: 
IPC Client (1991820533) connection to /10.106.4.2:49222 from 
job_1466061903809_0024 got value #216
2016-06-16 16:39:23,632 DEBUG [communication thread] org.apache.hadoop.ipc.RPC: 
Call: ping 1
2016-06-16 16:39:26,633 DEBUG [IPC Parameter Sending Thread #0] 
org.apache.hadoop.ipc.Client: IPC Client (1991820533) connection to 
/10.106.4.2:49222 from job_1466061903809_0024 sending #217
2016-06-16 16:39:26,634 DEBUG [IPC Client (1991820533) connection to 
/10.106.4.2:49222 from job_1466061903809_0024] org.apache.hadoop.ipc.Client: 
IPC Client (1991820533) connection to /10.106.4.2:49222 from 
job_1466061903809_0024 got value #217
2016-06-16 16:39:26,635 DEBUG [communication thread] org.apache.hadoop.ipc.RPC: 
Call: ping 2
2016-06-16 16:39:29,635 DEBUG [IPC Parameter Sending Thread #0] 
org.apache.hadoop.ipc.Client: IPC Client (1991820533) connection to 
/10.106.4.2:49222 from job_1466061903809_0024 sending #218
2016-06-16 16:39:29,636 DEBUG [IPC Client (1991820533) connection to 
/10.106.4.2:49222 from job_1466061903809_0024] org.apache.hadoop.ipc.Client: 
IPC Client (1991820533) connection to /10.106.4.2:49222 from 
job_1466061903809_0024 got value #218
2016-06-16 16:39:29,637 DEBUG [communication thread] org.apache.hadoop.ipc.RPC: 
Call: ping 2


The target machine is listening at the port, so the connection is there. 
I think the connection is built to write the map output to the directory 
specified by mapreduce.cluster.local.dir. But I do not see any data at 
this directory on the target machine. The directory is write-allowed for 
any users. Maybe the remote-disk-eriting is too long?


Jie




Am 16.06.2016 um 10:19 schrieb Jie Tao:
Both mappers hangs in spilling map output. May some Hadoop 
configuration be wrong with our cluster?




e: Handled 120 records!
2016-06-16 10:12:01,895 INFO [main] 
org.apache.kylin.engine.mr.steps.BaseCuboidMapperBase: Handled 130 
records!
2016-06-16 10:12:02,571 INFO [main] 
org.apache.kylin.engine.mr.steps.BaseCuboidMapperBase: Handled 140 
records!
2016-06-16 10:12:03,247 INFO [main]

Re: Extract Fact Table Distinct Columns problem

2016-06-16 Thread ShaoFeng Shi 
You can read this article:
https://kylin.apache.org/blog/2015/08/13/kylin-dictionary/

2016-06-16 17:23 GMT+08:00 仇同心 :

> Kylin crew:
>
>
>
>“Extract Fact Table Distinct Columns”, In this step will often happen
> OOM,I don't know what this step is doing, Can do a simple function
> description?
>
>
>
> In this FactDistinctColumnsJob.class , Line 80 ,List
> columnsNeedDict = cubeMgr.getAllDictColumnsOnFact(cubeDesc) : The method to
> realize the function of what.
>
>
>
>
>
>
>
>
>
> Thanks!
>
>
>
>
>



-- 
Best regards,

Shaofeng Shi

[jira] [Created] (KYLIN-1799) Add a document to setup kylin on spark engine?

2016-06-16 Thread zhengyaohong (JIRA) 
zhengyaohong created KYLIN-1799:
---

 Summary: Add a document to setup kylin on spark engine?
 Key: KYLIN-1799
 URL: https://issues.apache.org/jira/browse/KYLIN-1799
 Project: Kylin
  Issue Type: Wish
Reporter: zhengyaohong


Add a document to setup kylin on spark engine?



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (KYLIN-1798) Convert all date/time to millis format during the creation of snapshot table

2016-06-16 Thread liyang (JIRA) 
liyang created KYLIN-1798:
-

 Summary: Convert all date/time to millis format during the 
creation of snapshot table
 Key: KYLIN-1798
 URL: https://issues.apache.org/jira/browse/KYLIN-1798
 Project: Kylin
  Issue Type: Improvement
Reporter: liyang


After KYLIN-1766, the norm form of date/time at query level becomes time millis 
as a string. However the snapshot still saves date/time in -MM-dd format 
and requires transformation at query time.

Ideally we should convert all date/time to millis format during the creation of 
snapshot table.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Extract Fact Table Distinct Columns problem

2016-06-16 Thread 仇同心 
Kylin crew:

   “Extract Fact Table Distinct Columns”, In this step will often happen OOM,I 
don't know what this step is doing, Can do a simple function description?

In this FactDistinctColumnsJob.class , Line 80 ,List columnsNeedDict 
= cubeMgr.getAllDictColumnsOnFact(cubeDesc) : The method to realize the 
function of what.




Thanks!

RE: Timeout visiting cube

2016-06-16 Thread Jiaojiao Li 
Thank you for taking time to investigation.
Today I rerun the query and all the log are as follows.
Kylin_query.log:
root@MartiniTest000:/home/martini/kylin/logs# cat kylin_query.log
2016-06-16 09:04:10,795 DEBUG [http-bio-7070-exec-2] service.QueryService:291 : 
getting table metas
2016-06-16 09:04:10,796 DEBUG [http-bio-7070-exec-2] service.QueryService:309 : 
getting column metas
2016-06-16 09:04:10,804 DEBUG [http-bio-7070-exec-2] service.QueryService:323 : 
done column metas
2016-06-16 09:04:25,880 DEBUG [http-bio-7070-exec-10] service.QueryService:291 
: getting table metas
2016-06-16 09:04:25,882 DEBUG [http-bio-7070-exec-10] service.QueryService:309 
: getting column metas
2016-06-16 09:04:25,889 DEBUG [http-bio-7070-exec-10] service.QueryService:323 
: done column metas
2016-06-16 09:04:34,045 INFO  [http-bio-7070-exec-10] 
controller.QueryController:174 : Using project: CortanaTest
2016-06-16 09:04:34,046 INFO  [http-bio-7070-exec-10] 
controller.QueryController:175 : The original query:  select count(distinct 
userid) from cortanauu
2016-06-16 09:04:34,047 INFO  [http-bio-7070-exec-10] service.QueryService:266 
: The corrected query: select count(distinct userid) from cortanauu
LIMIT 5
2016-06-16 09:04:34,104 INFO  [http-bio-7070-exec-10] routing.QueryRouter:48 : 
The project manager's reference is 
org.apache.kylin.metadata.project.ProjectManager@419808ad
2016-06-16 09:04:34,105 INFO  [http-bio-7070-exec-10] routing.QueryRouter:60 : 
Find candidates by table CORTANAUU.CORTANAUU and project=CORTANATEST : 
org.apache.kylin.query.routing.Candidate@f3ac47f
2016-06-16 09:04:34,105 INFO  [http-bio-7070-exec-10] routing.QueryRouter:49 : 
Applying rule: class 
org.apache.kylin.query.routing.rules.RemoveUncapableRealizationsRule, 
realizations before: [CortanaUUTest_clone(CUBE)], realizations after: 
[CortanaUUTest_clone(CUBE)]
2016-06-16 09:04:34,105 INFO  [http-bio-7070-exec-10] routing.QueryRouter:49 : 
Applying rule: class org.apache.kylin.query.routing.rules.RealizationSortRule, 
realizations before: [CortanaUUTest_clone(CUBE)], realizations after: 
[CortanaUUTest_clone(CUBE)]
2016-06-16 09:04:34,106 INFO  [http-bio-7070-exec-10] routing.QueryRouter:85 : 
Adjust DimensionAsMeasure for FunctionDesc [expression=COUNT_DISTINCT, 
parameter=ParameterDesc [type=column, value=USERID, nextParam=null], 
returnType=null]
2016-06-16 09:04:34,106 INFO  [http-bio-7070-exec-10] routing.QueryRouter:72 : 
The realizations remaining: [CortanaUUTest_clone(CUBE)] And the final chosen 
one is the first one
2016-06-16 09:04:34,152 DEBUG [http-bio-7070-exec-10] 
enumerator.OLAPEnumerator:107 : query storage...
2016-06-16 09:04:34,405 DEBUG [http-bio-7070-exec-10] 
enumerator.OLAPEnumerator:127 : return TupleIterator...
2016-06-16 09:06:46,286 ERROR [http-bio-7070-exec-10] 
controller.QueryController:209 : Exception when execute sql
java.sql.SQLException: Error while executing SQL "select count(distinct userid) 
from cortanauu
LIMIT 5": Timeout visiting cube!
at org.apache.calcite.avatica.Helper.createException(Helper.java:56)
at org.apache.calcite.avatica.Helper.createException(Helper.java:41)
at 
org.apache.calcite.avatica.AvaticaStatement.executeInternal(AvaticaStatement.java:143)
at 
org.apache.calcite.avatica.AvaticaStatement.executeQuery(AvaticaStatement.java:186)
at 
org.apache.kylin.rest.service.QueryService.execute(QueryService.java:361)
at 
org.apache.kylin.rest.service.QueryService.queryWithSqlMassage(QueryService.java:273)
at 
org.apache.kylin.rest.service.QueryService.query(QueryService.java:121)
at 
org.apache.kylin.rest.service.QueryService$$FastClassByCGLIB$$4957273f.invoke()
at net.sf.cglib.proxy.MethodProxy.invoke(MethodProxy.java:204)
at 
org.springframework.aop.framework.Cglib2AopProxy$DynamicAdvisedInterceptor.intercept(Cglib2AopProxy.java:618)
at 
org.apache.kylin.rest.service.QueryService$$EnhancerByCGLIB$$72e265c1.query()
at 
org.apache.kylin.rest.controller.QueryController.doQueryWithCache(QueryController.java:192)
at 
org.apache.kylin.rest.controller.QueryController.query(QueryController.java:94)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at 
org.springframework.web.method.support.InvocableHandlerMethod.invoke(InvocableHandlerMethod.java:213)
at 
org.springframework.web.method.support.InvocableHandlerMethod.invokeForRequest(InvocableHandlerMethod.java:126)
at 
org.springframework.web.servlet.mvc.method.annotation.ServletInvocableHandlerMethod.invokeAndHandle(ServletInvocableHandlerMethod.java:96)
at

[jira] [Created] (KYLIN-1797) bin/find-hive-dependency.sh may not get correct local classpath when configured hive client is beeline

2016-06-16 Thread Shaofeng SHI (JIRA) 
Shaofeng SHI created KYLIN-1797:
---

 Summary: bin/find-hive-dependency.sh may not get correct local 
classpath when configured hive client is beeline
 Key: KYLIN-1797
 URL: https://issues.apache.org/jira/browse/KYLIN-1797
 Project: Kylin
  Issue Type: Bug
  Components: Client - CLI
Affects Versions: v1.3.0, v1.5.0
Reporter: Shaofeng SHI






--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Extract Fact Table Distinct Columns problem

2016-06-16 Thread 仇同心 
Kylin crew:

   “Extract Fact Table Distinct Columns”, In this step will often happen OOM,I 
don't know what this step is doing, Can do a simple function description?

In this FactDistinctColumnsJob.class , Line 80 ,List columnsNeedDict 
= cubeMgr.getAllDictColumnsOnFact(cubeDesc) : The method to realize the 
function of what.




Thanks!

Re: properties in kylin configuration files

2016-06-16 Thread Jie Tao 

Both mappers hangs in spilling map output. May some Hadoop configuration be 
wrong with our cluster?



e: Handled 120 records!
2016-06-16 10:12:01,895 INFO [main] 
org.apache.kylin.engine.mr.steps.BaseCuboidMapperBase: Handled 130 records!
2016-06-16 10:12:02,571 INFO [main] 
org.apache.kylin.engine.mr.steps.BaseCuboidMapperBase: Handled 140 records!
2016-06-16 10:12:03,247 INFO [main] 
org.apache.kylin.engine.mr.steps.BaseCuboidMapperBase: Handled 150 records!
2016-06-16 10:12:03,948 INFO [main] 
org.apache.kylin.engine.mr.steps.BaseCuboidMapperBase: Handled 160 records!
2016-06-16 10:12:04,647 INFO [main] 
org.apache.kylin.engine.mr.steps.BaseCuboidMapperBase: Handled 170 records!
2016-06-16 10:12:05,368 INFO [main] 
org.apache.kylin.engine.mr.steps.BaseCuboidMapperBase: Handled 180 records!
2016-06-16 10:12:06,056 INFO [main] 
org.apache.kylin.engine.mr.steps.BaseCuboidMapperBase: Handled 190 records!
2016-06-16 10:12:06,744 INFO [main] 
org.apache.kylin.engine.mr.steps.BaseCuboidMapperBase: Handled 200 records!
2016-06-16 10:12:07,445 INFO [main] 
org.apache.kylin.engine.mr.steps.BaseCuboidMapperBase: Handled 210 records!
2016-06-16 10:12:08,143 INFO [main] 
org.apache.kylin.engine.mr.steps.BaseCuboidMapperBase: Handled 220 records!
2016-06-16 10:12:08,842 INFO [main] 
org.apache.kylin.engine.mr.steps.BaseCuboidMapperBase: Handled 230 records!
2016-06-16 10:12:09,540 INFO [main] 
org.apache.kylin.engine.mr.steps.BaseCuboidMapperBase: Handled 240 records!
2016-06-16 10:12:10,238 INFO [main] 
org.apache.kylin.engine.mr.steps.BaseCuboidMapperBase: Handled 250 records!
2016-06-16 10:12:10,938 INFO [main] 
org.apache.kylin.engine.mr.steps.BaseCuboidMapperBase: Handled 260 records!
2016-06-16 10:12:11,624 INFO [main] 
org.apache.kylin.engine.mr.steps.BaseCuboidMapperBase: Handled 270 records!
2016-06-16 10:12:12,312 INFO [main] 
org.apache.kylin.engine.mr.steps.BaseCuboidMapperBase: Handled 280 records!
2016-06-16 10:12:12,999 INFO [main] 
org.apache.kylin.engine.mr.steps.BaseCuboidMapperBase: Handled 290 records!
2016-06-16 10:12:13,698 INFO [main] 
org.apache.kylin.engine.mr.steps.BaseCuboidMapperBase: Handled 300 records!
2016-06-16 10:12:14,387 INFO [main] 
org.apache.kylin.engine.mr.steps.BaseCuboidMapperBase: Handled 310 records!
2016-06-16 10:12:15,075 INFO [main] 
org.apache.kylin.engine.mr.steps.BaseCuboidMapperBase: Handled 320 records!
2016-06-16 10:12:15,764 INFO [main] 
org.apache.kylin.engine.mr.steps.BaseCuboidMapperBase: Handled 330 records!
2016-06-16 10:12:16,460 INFO [main] 
org.apache.kylin.engine.mr.steps.BaseCuboidMapperBase: Handled 340 records!
2016-06-16 10:12:17,147 INFO [main] 
org.apache.kylin.engine.mr.steps.BaseCuboidMapperBase: Handled 350 records!
2016-06-16 10:12:17,834 INFO [main] 
org.apache.kylin.engine.mr.steps.BaseCuboidMapperBase: Handled 360 records!
2016-06-16 10:12:18,520 INFO [main] 
org.apache.kylin.engine.mr.steps.BaseCuboidMapperBase: Handled 370 records!
2016-06-16 10:12:19,216 INFO [main] 
org.apache.kylin.engine.mr.steps.BaseCuboidMapperBase: Handled 380 records!
2016-06-16 10:12:19,901 INFO [main] 
org.apache.kylin.engine.mr.steps.BaseCuboidMapperBase: Handled 390 records!
2016-06-16 10:12:20,590 INFO [main] 
org.apache.kylin.engine.mr.steps.BaseCuboidMapperBase: Handled 400 records!
2016-06-16 10:12:21,286 INFO [main] 
org.apache.kylin.engine.mr.steps.BaseCuboidMapperBase: Handled 410 records!
2016-06-16 10:12:21,970 INFO [main] 
org.apache.kylin.engine.mr.steps.BaseCuboidMapperBase: Handled 420 records!
2016-06-16 10:12:22,657 INFO [main] 
org.apache.kylin.engine.mr.steps.BaseCuboidMapperBase: Handled 430 records!
2016-06-16 10:12:22,801 INFO [main] org.apache.hadoop.mapred.MapTask: Starting 
flush of map output
2016-06-16 10:12:22,802 INFO [main] org.apache.hadoop.mapred.MapTask: Spilling 
map output
2016-06-16 10:12:22,802 INFO [main] org.apache.hadoop.mapred.MapTask: bufstart 
= 0; bufend = 190884986; bufvoid = 536870912
2016-06-16 10:12:22,802 INFO [main] org.apache.hadoop.mapred.MapTask: kvstart = 
134217724(536870896); kvend = 116938212(467752848); length = 17279513/33554432



Cheers,

Jie


Am 16.06.2016 um 04:38 schrieb hongbin ma:

​Since you intermediate table is merely 200M, each mapper deals with 100M
fact table. This is not very heavy for a mapper. Have you checked the log
for the mapper to know what is it doing in the 3600s?

On Thu, Jun 16, 2016 at 10:35 AM, hongbin ma  wrote:


3.7 GB of 8.4 GB virtual memory used

​do you have any idea why the container is using lots of virtual memory?
is there any heavy contention ​on the memory resource? Not sure if it's the
root cause for slow mappers
​


--
Regards,

*Bin Mahone | 马洪宾*
Apache Kylin: http://kylin.io
Github: https://github.com/binmahone

[jira] [Created] (KYLIN-1795) bin/sample.sh doesn't work when configured hive client is beeline

2016-06-16 Thread Shaofeng SHI (JIRA) 
Shaofeng SHI created KYLIN-1795:
---

 Summary: bin/sample.sh doesn't work when configured hive client is 
beeline
 Key: KYLIN-1795
 URL: https://issues.apache.org/jira/browse/KYLIN-1795
 Project: Kylin
  Issue Type: Bug
  Components: Client - CLI
Affects Versions: v1.5.0
Reporter: Shaofeng SHI
Priority: Minor


If configure to use beeline as hive client in kylin.properties, then run 
bin/sample.sh, it will fail to load data into hive, like:

0: jdbc:hive2://hiveserver:10> LOAD DATA LOCAL INPATH 'DEFAULT.KYLIN_SALES.csv' 
OVERWRITE INTO TABLE DEFAULT.KYLIN_SALES;
Error: Error while compiling statement: FAILED: SemanticException Line 1:23 
Invalid path ''DEFAULT.KYLIN_SALES.csv'': No files matching path 
file:/var/run/cloudera-scm-agent/process/1358-hive-HIVESERVER2/DEFAULT.KYLIN_SALES.csv
 (state=42000,code=4)



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (KYLIN-1794) Enable job list even some job metadata parsing failed

2016-06-16 Thread Roger Shi (JIRA) 
Roger Shi created KYLIN-1794:


 Summary: Enable job list even some job metadata parsing failed
 Key: KYLIN-1794
 URL: https://issues.apache.org/jira/browse/KYLIN-1794
 Project: Kylin
  Issue Type: Bug
  Components: Metadata
Reporter: Roger Shi
Assignee: Shaofeng SHI
Priority: Minor






--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (KYLIN-1793) Job couldn't stop when hive commands got error with beeline

2016-06-16 Thread Shaofeng SHI (JIRA) 
Shaofeng SHI created KYLIN-1793:
---

 Summary: Job couldn't stop when hive commands got error with 
beeline
 Key: KYLIN-1793
 URL: https://issues.apache.org/jira/browse/KYLIN-1793
 Project: Kylin
  Issue Type: Bug
  Components: Job Engine
Affects Versions: v1.5.2, v1.5.1, v1.5.0
Reporter: Shaofeng SHI
Assignee: Dong Li


Configure Kylin to use beeline as the hive command line; submit a cube build 
job, the job moves to 100% with success, while I found there was error in the 
hive related steps, but the error wasn't captured by Kylin;



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

RE: kylin question

2016-06-16 Thread 倪成伟 
OK，Thanks! I will try it.

--
View this message in context: 
http://apache-kylin.74782.x6.nabble.com/java-lang-NoSuchMethodError-org-apache-hadoop-hbase-coprocessor-RegionCoprocessorEnvironment-getRegi-tp4698p5017.html
Sent from the Apache Kylin mailing list archive at Nabble.com.