Re: Select * not returning any rows
Hi, Thank you Hongbin Ma for suggestion to upgrade. I've just now changed the version of Kylin to 1.2 and I'm able to see some results when I perform 'select *' on the tables. But the number of results returned are not exactly as expected. In my case, I have 100 rows in a table, some of which might be repeated. But when I'm using select *, it's kind of performing group by over all the columns in the cube and returning very few rows. Is this behavior intended or a bug? If the second, please let me know if there is a way to correct it. Thank you guys for responding quickly on new year's eve. On Dec 31, 2015 4:04 PM, "Shi, Shaofeng" wrote: > This “how to_upgrade” is specifically for v0.6 to v0.7 upgrade, not for > other versions; > > Between v0.7 to v1.2, the metadata is compatible; What user need do is > just backup and restore the $KYLIN_HOME/conf folder after switch to a new > Kylin binary; All the cube metadata and cube data are in HBase so no need > to re-create or rebuild; > > On 12/31/15, 2:49 PM, "250635...@qq.com" <250635...@qq.com> wrote: > > >Have you checked out this one ? > >http://kylin.apache.org/docs/howto/howto_upgrade.html > > > > > > > > > >250635...@qq.com > > > >From: Kiriti Sai > >Date: 2015-12-31 14:36 > >To: dev > >Subject: Re: Select * not returning any rows > >Since I'm just working with binaries, can you please explain how to > >upgrade > >from v1.1 to v1.2? > >Should I just extract and replace the whole folder or should I backup and > >restore the data also in some way? > > > >Thank you for the immediate response. :) > >On Dec 31, 2015 3:25 PM, "hongbin ma" wrote: > > > >> i believe this issue has been fixed in v1.2, why not use the latest > >> version? > >> > >> > >> -- > >> Regards, > >> > >> *Bin Mahone | 马洪宾* > >> Apache Kylin: http://kylin.io > >> Github: https://github.com/binmahone > >> > >
Re: Re: Select * not returning any rows
Ah, I see. Thanks shaofeng for clarifying. 250635...@qq.com From: Shi, Shaofeng Date: 2015-12-31 15:04 To: dev@kylin.apache.org Subject: Re: Select * not returning any rows This “how to_upgrade” is specifically for v0.6 to v0.7 upgrade, not for other versions; Between v0.7 to v1.2, the metadata is compatible; What user need do is just backup and restore the $KYLIN_HOME/conf folder after switch to a new Kylin binary; All the cube metadata and cube data are in HBase so no need to re-create or rebuild; On 12/31/15, 2:49 PM, "250635...@qq.com" <250635...@qq.com> wrote: >Have you checked out this one ? >http://kylin.apache.org/docs/howto/howto_upgrade.html > > > > >250635...@qq.com > >From: Kiriti Sai >Date: 2015-12-31 14:36 >To: dev >Subject: Re: Select * not returning any rows >Since I'm just working with binaries, can you please explain how to >upgrade >from v1.1 to v1.2? >Should I just extract and replace the whole folder or should I backup and >restore the data also in some way? > >Thank you for the immediate response. :) >On Dec 31, 2015 3:25 PM, "hongbin ma" wrote: > >> i believe this issue has been fixed in v1.2, why not use the latest >> version? >> >> >> -- >> Regards, >> >> *Bin Mahone | 马洪宾* >> Apache Kylin: http://kylin.io >> Github: https://github.com/binmahone >>
Re: Select * not returning any rows
This “how to_upgrade” is specifically for v0.6 to v0.7 upgrade, not for other versions; Between v0.7 to v1.2, the metadata is compatible; What user need do is just backup and restore the $KYLIN_HOME/conf folder after switch to a new Kylin binary; All the cube metadata and cube data are in HBase so no need to re-create or rebuild; On 12/31/15, 2:49 PM, "250635...@qq.com" <250635...@qq.com> wrote: >Have you checked out this one ? >http://kylin.apache.org/docs/howto/howto_upgrade.html > > > > >250635...@qq.com > >From: Kiriti Sai >Date: 2015-12-31 14:36 >To: dev >Subject: Re: Select * not returning any rows >Since I'm just working with binaries, can you please explain how to >upgrade >from v1.1 to v1.2? >Should I just extract and replace the whole folder or should I backup and >restore the data also in some way? > >Thank you for the immediate response. :) >On Dec 31, 2015 3:25 PM, "hongbin ma" wrote: > >> i believe this issue has been fixed in v1.2, why not use the latest >> version? >> >> >> -- >> Regards, >> >> *Bin Mahone | 马洪宾* >> Apache Kylin: http://kylin.io >> Github: https://github.com/binmahone >>
Re: Re: Select * not returning any rows
i think 1.2 is compabible with 1.1, it's only bug fixes On Thu, Dec 31, 2015 at 2:49 PM, 250635...@qq.com <250635...@qq.com> wrote: > Have you checked out this one ? > http://kylin.apache.org/docs/howto/howto_upgrade.html > > > > > 250635...@qq.com > > From: Kiriti Sai > Date: 2015-12-31 14:36 > To: dev > Subject: Re: Select * not returning any rows > Since I'm just working with binaries, can you please explain how to upgrade > from v1.1 to v1.2? > Should I just extract and replace the whole folder or should I backup and > restore the data also in some way? > > Thank you for the immediate response. :) > On Dec 31, 2015 3:25 PM, "hongbin ma" wrote: > > > i believe this issue has been fixed in v1.2, why not use the latest > > version? > > > > > > -- > > Regards, > > > > *Bin Mahone | 马洪宾* > > Apache Kylin: http://kylin.io > > Github: https://github.com/binmahone > > > -- Regards, *Bin Mahone | 马洪宾* Apache Kylin: http://kylin.io Github: https://github.com/binmahone
Re: Re: Select * not returning any rows
Have you checked out this one ? http://kylin.apache.org/docs/howto/howto_upgrade.html 250635...@qq.com From: Kiriti Sai Date: 2015-12-31 14:36 To: dev Subject: Re: Select * not returning any rows Since I'm just working with binaries, can you please explain how to upgrade from v1.1 to v1.2? Should I just extract and replace the whole folder or should I backup and restore the data also in some way? Thank you for the immediate response. :) On Dec 31, 2015 3:25 PM, "hongbin ma" wrote: > i believe this issue has been fixed in v1.2, why not use the latest > version? > > > -- > Regards, > > *Bin Mahone | 马洪宾* > Apache Kylin: http://kylin.io > Github: https://github.com/binmahone >
Re: Select * not returning any rows
Since I'm just working with binaries, can you please explain how to upgrade from v1.1 to v1.2? Should I just extract and replace the whole folder or should I backup and restore the data also in some way? Thank you for the immediate response. :) On Dec 31, 2015 3:25 PM, "hongbin ma" wrote: > i believe this issue has been fixed in v1.2, why not use the latest > version? > > > -- > Regards, > > *Bin Mahone | 马洪宾* > Apache Kylin: http://kylin.io > Github: https://github.com/binmahone >
?????? kylin sync hive table failed
run find-hive-dependency.sh can find jars; env: hadoop2.7.1,hive 1.21,kylin 1.2 log: Logging initialized using configuration in jar:file:/usr/local/hive/lib/hive-common-1.2.1.jar!/hive-log4j.properties SLF4J: Class path contains multiple SLF4J bindings. SLF4J: Found binding in [jar:file:/usr/local/hadoop/share/hadoop/common/lib/slf4j-log4j12-1.7.10.jar!/org/slf4j/impl/StaticLoggerBinder.class] SLF4J: Found binding in [jar:file:/usr/local/hive/lib/slf4j-log4j12-1.7.5.jar!/org/slf4j/impl/StaticLoggerBinder.class] SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation. SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory] HCAT_HOME not found, try to find hcatalog path from hadoop home hive dependency: /usr/local/hive/conf:/usr/local/hive/lib/jsr305-1.3.9.jar:/usr/local/hive/lib/jetty-all-7.6.0.v20120127.jar:/usr/local/hive/lib/servlet-api-2.5.jar:/usr/local/hive/lib/jets3t-0.9.0.jar:/usr/local/hive/lib/accumulo-core-1.6.0.jar:/usr/local/hive/lib/libfb303-0.9.2.jar:/usr/local/hive/lib/json-serde-1.3.6-jar-with-dependencies.jar:/usr/local/hive/lib/commons-httpclient-3.0.1.jar:/usr/local/hive/lib/ivy-2.4.0.jar:/usr/local/hive/lib/hbase-examples-0.98.16.1-hadoop2.jar:/usr/local/hive/lib/jersey-client-1.9.jar:/usr/local/hive/lib/jersey-server-1.9.jar:/usr/local/hive/lib/slf4j-log4j12-1.7.5.jar:/usr/local/hive/lib/zookeeper-3.4.6.jar:/usr/local/hive/lib/bonecp-0.8.0.RELEASE.jar:/usr/local/hive/lib/activation-1.1.jar:/usr/local/hive/lib/snappy-java-1.0.5.jar:/usr/local/hive/lib/commons-cli-1.2.jar:/usr/local/hive/lib/ST4-4.0.4.jar:/usr/local/hive/lib/asm-3.1.jar:/usr/local/hive/lib/hive-common-1.2.1.jar:/usr/local/hive/lib/avro-1.7.5.jar:/usr/local/hive/lib/findbugs-annotations-1.3.9-1.jar:/usr/local/hive/lib/accumulo-trace-1.6.0.jar:/usr/local/hive/lib/jcommander-1.32.jar:/usr/local/hive/lib/commons-lang-2.6.jar:/usr/local/hive/lib/ -- -- -- ??: "nichunen";; : 2015??12??31??(??) 1:15 ??: "dev"; : : kylin sync hive table failed Hi, Maybe you can run find-hive-dependency.sh to check whether the hive jar packages can be found. Best Regards, George/?? Software Engineer/?? Mobile:+86-13501723787| Fax:+8610-56842040 ??www.mininglamp.com?? ??398??14?? F4,1#,Zhongmei Construction Group Plaza,398# Zhongdong Road,Changping District,Beijing,102218 ?? 2015-12-31 11:24 dev ?? kylin sync hive table failed hi: kylin sync hive table failed; "failed to take action",but not error log. (l use database.table); log: org.apache.kylin.rest.filter.KylinApiFilter.logRequest(KylinApiFilter.java:120)] - REQUEST: REQUESTER=ADMIN;REQ_TIME=GMT-08:00 2015-12-30 19:05:21;URI=/kylin/api/tables/froad_data.bi_refund_log/hive_hbase_kylin;METHOD=POST;QUERY_STRING=null; PAYLOAD=;RESP_STATUS=200; yehefeng
Re: Select * not returning any rows
i believe this issue has been fixed in v1.2, why not use the latest version? -- Regards, *Bin Mahone | 马洪宾* Apache Kylin: http://kylin.io Github: https://github.com/binmahone
Select * not returning any rows
Hi, I've been using Kylin v1.1 setup with binaries on an EC2 cluster of 8 nodes. I've completed the setup and able to build some cubes apart from the sample cube. But when I try to query the cubes using select * on any table, including the sample tables, it doesn't return any rows. It just shows the column names and the number of results is shown as 1.
回复: kylin sync hive table failed
Hi, Maybe you can run find-hive-dependency.sh to check whether the hive jar packages can be found. Best Regards, George/倪春恩 Software Engineer/软件工程师 Mobile:+86-13501723787| Fax:+8610-56842040 北京明略软件系统有限公司（www.mininglamp.com） 北京市昌平区东小口镇中东路398号中煤建设集团大厦1号楼4层 F4,1#,Zhongmei Construction Group Plaza,398# Zhongdong Road,Changping District,Beijing,102218 发件人： 和风 发送时间： 2015-12-31 11:24 收件人： dev 主题： kylin sync hive table failed hi: kylin sync hive table failed; "failed to take action",but not error log. (l use database.table); log: org.apache.kylin.rest.filter.KylinApiFilter.logRequest(KylinApiFilter.java:120)] - REQUEST: REQUESTER=ADMIN;REQ_TIME=GMT-08:00 2015-12-30 19:05:21;URI=/kylin/api/tables/froad_data.bi_refund_log/hive_hbase_kylin;METHOD=POST;QUERY_STRING=null; PAYLOAD=;RESP_STATUS=200; yehefeng
[jira] [Created] (KYLIN-1275) "mapreduce.job.queuename" does not work for HiveColumnCardinalityJob
dixingxing created KYLIN-1275: - Summary: "mapreduce.job.queuename" does not work for HiveColumnCardinalityJob Key: KYLIN-1275 URL: https://issues.apache.org/jira/browse/KYLIN-1275 Project: Kylin Issue Type: Bug Components: Job Engine Affects Versions: v1.3 Environment: apache hadoop2.4.1 , hive 1.0.0, hbase 1.1.2 Reporter: dixingxing Assignee: ZhouQianhao kylin_job_conf.xml : mapreduce.job.queuename xxx Job queue This works fine for other jobs but HiveColumnCardinalityJob (still in default queue) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
Re: Welcome new Apache Kylin committer: Luwei Chen
hi all! congratulations, lu wei! and now I know how it works now! best regards! wangxianbin1...@gmail.com From: Luke Han Date: 2015-12-30 21:47 To: dev Subject: Welcome new Apache Kylin committer: Luwei Chen I am very pleased to announce that the Project Management Committee (PMC) of Apache Kylin has asked Luwei Chen to become Apache Kylin committer , and she has already accepted. Luwei has already made many contribution to Kylin community, about website, documentation, UI and others as well. Welcome Luwei, our first female committer:) Please share with us a little about yourself, Luke On behalf of the Apache Kylin PPMC
kylin sync hive table failed
hi: kylin sync hive table failed; "failed to take action",but not error log. (l use database.table); log: org.apache.kylin.rest.filter.KylinApiFilter.logRequest(KylinApiFilter.java:120)] - REQUEST: REQUESTER=ADMIN;REQ_TIME=GMT-08:00 2015-12-30 19:05:21;URI=/kylin/api/tables/froad_data.bi_refund_log/hive_hbase_kylin;METHOD=POST;QUERY_STRING=null; PAYLOAD=;RESP_STATUS=200; yehefeng
Re: Welcome new Apache Kylin committer: Luwei Chen
welcome~ Henry Saputra 于2015年12月31日周四 上午5:02写道： > Welcome, Luwei > > - Henry > > On Wed, Dec 30, 2015 at 5:47 AM, Luke Han wrote: > > I am very pleased to announce that the Project Management Committee > > (PMC) of Apache Kylin has asked Luwei Chen to become Apache Kylin > committer > > , > > and she has already accepted. > > > > Luwei has already made many contribution to Kylin community, about > website, > > documentation, UI and others as well. > > > > Welcome Luwei, our first female committer:) > > Please share with us a little about yourself, > > > > Luke > > > > On behalf of the Apache Kylin PPMC >
Re: Welcome new Apache Kylin committer: Luwei Chen
Welcome, Luwei - Henry On Wed, Dec 30, 2015 at 5:47 AM, Luke Han wrote: > I am very pleased to announce that the Project Management Committee > (PMC) of Apache Kylin has asked Luwei Chen to become Apache Kylin committer > , > and she has already accepted. > > Luwei has already made many contribution to Kylin community, about website, > documentation, UI and others as well. > > Welcome Luwei, our first female committer:) > Please share with us a little about yourself, > > Luke > > On behalf of the Apache Kylin PPMC
Re: Welcome new Apache Kylin committer: Luwei Chen
Welcome!!! > 在 2015年12月30日，22:42，ShaoFeng Shi 写道： > > Welcome Luwei! > > 2015-12-30 21:52 GMT+08:00 hongbin ma : > >> welcome! >> >> On Wed, Dec 30, 2015 at 9:51 PM, 李栋 wrote: >> >>> Welcome Luwei! >>> >>> Dong Li >>> >>> >>> 发自 网易邮箱大师 >>> >>> >>> >>> On 2015-12-30 21:47 , Luke Han Wrote: >>> >>> I am very pleased to announce that the Project Management Committee >>> (PMC) of Apache Kylin has asked Luwei Chen to become Apache Kylin >> committer >>> , >>> and she has already accepted. >>> >>> Luwei has already made many contribution to Kylin community, about >> website, >>> documentation, UI and others as well. >>> >>> Welcome Luwei, our first female committer:) >>> Please share with us a little about yourself, >>> >>> Luke >>> >>> On behalf of the Apache Kylin PPMC >>> >> >> >> >> -- >> Regards, >> >> *Bin Mahone | 马洪宾* >> Apache Kylin: http://kylin.io >> Github: https://github.com/binmahone >> > > > > -- > Best regards, > > Shaofeng Shi
Re: Welcome new Apache Kylin committer: Luwei Chen
Welcome Luwei! 2015-12-30 21:52 GMT+08:00 hongbin ma : > welcome! > > On Wed, Dec 30, 2015 at 9:51 PM, 李栋 wrote: > > > Welcome Luwei! > > > > Dong Li > > > > > > 发自 网易邮箱大师 > > > > > > > > On 2015-12-30 21:47 , Luke Han Wrote: > > > > I am very pleased to announce that the Project Management Committee > > (PMC) of Apache Kylin has asked Luwei Chen to become Apache Kylin > committer > > , > > and she has already accepted. > > > > Luwei has already made many contribution to Kylin community, about > website, > > documentation, UI and others as well. > > > > Welcome Luwei, our first female committer:) > > Please share with us a little about yourself, > > > > Luke > > > > On behalf of the Apache Kylin PPMC > > > > > > -- > Regards, > > *Bin Mahone | 马洪宾* > Apache Kylin: http://kylin.io > Github: https://github.com/binmahone > -- Best regards, Shaofeng Shi
Re: Welcome new Apache Kylin committer: Luwei Chen
welcome! On Wed, Dec 30, 2015 at 9:51 PM, 李栋 wrote: > Welcome Luwei! > > Dong Li > > > 发自 网易邮箱大师 > > > > On 2015-12-30 21:47 , Luke Han Wrote: > > I am very pleased to announce that the Project Management Committee > (PMC) of Apache Kylin has asked Luwei Chen to become Apache Kylin committer > , > and she has already accepted. > > Luwei has already made many contribution to Kylin community, about website, > documentation, UI and others as well. > > Welcome Luwei, our first female committer:) > Please share with us a little about yourself, > > Luke > > On behalf of the Apache Kylin PPMC > -- Regards, *Bin Mahone | 马洪宾* Apache Kylin: http://kylin.io Github: https://github.com/binmahone
Re: Welcome new Apache Kylin committer: Luwei Chen
Welcome Luwei! Dong Li 发自 网易邮箱大师 On 2015-12-30 21:47 , Luke Han Wrote: I am very pleased to announce that the Project Management Committee (PMC) of Apache Kylin has asked Luwei Chen to become Apache Kylin committer , and she has already accepted. Luwei has already made many contribution to Kylin community, about website, documentation, UI and others as well. Welcome Luwei, our first female committer:) Please share with us a little about yourself, Luke On behalf of the Apache Kylin PPMC
Welcome new Apache Kylin committer: Luwei Chen
I am very pleased to announce that the Project Management Committee (PMC) of Apache Kylin has asked Luwei Chen to become Apache Kylin committer , and she has already accepted. Luwei has already made many contribution to Kylin community, about website, documentation, UI and others as well. Welcome Luwei, our first female committer:) Please share with us a little about yourself, Luke On behalf of the Apache Kylin PPMC
[jira] [Created] (KYLIN-1274) Query from JDBC is partial results by default
Yerui Sun created KYLIN-1274: Summary: Query from JDBC is partial results by default Key: KYLIN-1274 URL: https://issues.apache.org/jira/browse/KYLIN-1274 Project: Kylin Issue Type: Bug Components: Driver - JDBC Affects Versions: 1.2 Reporter: Yerui Sun Assignee: Yerui Sun Fix For: 1.3, 2.0 How to produce this problem: create a query with the result over 1 rows, and query with 'order by desc'. Check the first row, it's not the last row in all result, but a middle row, maybe the 1th row. Checked the query log in kylin server, found 'Accept Partial: false', indicating it's indeed a partial query. The reason is, JDBC driver sent the QueryRequest with json encoding, and parsed into SQLRequest in server side. By default, QueryRequest only has sql and project parameters, and leave all other attributes as default in SQLRequest, and in SQLRequest, acceptPartial is true. That's why the query was processed as a partial query. The solution to solve this is simple, add acceptPartial as false in QueryRequest, and update JDBC driver. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (KYLIN-1273) remove suffix "_ts" of timestamps
Lola Liu created KYLIN-1273: --- Summary: remove suffix "_ts" of timestamps Key: KYLIN-1273 URL: https://issues.apache.org/jira/browse/KYLIN-1273 Project: Kylin Issue Type: Bug Reporter: Lola Liu Assignee: Zhong,Jason -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (KYLIN-1272) Add explanation for auto appended field of streaming table
Lola Liu created KYLIN-1272: --- Summary: Add explanation for auto appended field of streaming table Key: KYLIN-1272 URL: https://issues.apache.org/jira/browse/KYLIN-1272 Project: Kylin Issue Type: Bug Reporter: Lola Liu Assignee: Zhong,Jason Priority: Minor -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (KYLIN-1271) Create streaming table layer will disappear if click on outside
Lola Liu created KYLIN-1271: --- Summary: Create streaming table layer will disappear if click on outside Key: KYLIN-1271 URL: https://issues.apache.org/jira/browse/KYLIN-1271 Project: Kylin Issue Type: Bug Components: Web Affects Versions: v2.0 Reporter: Lola Liu Assignee: Zhong,Jason STEPS: 1. Sign in, go to models 2. Click "adding streaming table" 3. "Create Streaming Table Schema" layer showed up 4. Edit on layer, create streaming table schema 5. Click outside of this layer RESULT: The layer disappeared without saving anything -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (KYLIN-1270) improve TimedJsonStreamParser to support month_start,quarter_start,year_start
hongbin ma created KYLIN-1270: - Summary: improve TimedJsonStreamParser to support month_start,quarter_start,year_start Key: KYLIN-1270 URL: https://issues.apache.org/jira/browse/KYLIN-1270 Project: Kylin Issue Type: Improvement Reporter: hongbin ma Assignee: hongbin ma -- This message was sent by Atlassian JIRA (v6.3.4#6332)
Re: ApplicationNotFoundException
Seems a hadoop problem... On Mon, Dec 28, 2015 at 3:49 PM, 胡志华(万里通科技及数据中心商务智能团队数据分析组) < huzhihua...@pingan.com.cn> wrote: > Hi,all > > I stopped at step 14 “ Convert cuboid data to HFile” with > exception ApplicationNotFoundException, who can help me ? > > > org.apache.hadoop.yarn.exceptions.ApplicationNotFoundException: > Application with id 'application_1451274628461_0003' doesn't exist in RM. > at > org.apache.hadoop.yarn.server.resourcemanager.ClientRMService.getApplicationReport(ClientRMService.java:327) > at > org.apache.hadoop.yarn.api.impl.pb.service.ApplicationClientProtocolPBServiceImpl.getApplicationReport(ApplicationClientProtocolPBServiceImpl.java:175) > at > org.apache.hadoop.yarn.proto.ApplicationClientProtocol$ApplicationClientProtocolService$2.callBlockingMethod(ApplicationClientProtocol.java:417) > at > org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:616) > at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:969) > at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2049) > at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2045) > at java.security.AccessController.doPrivileged(Native Method) > at javax.security.auth.Subject.doAs(Subject.java:415) > at > org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657) > at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2043) > 2015-12-28 14:33:52,444 INFO org.apache.hadoop.ipc.Server: IPC Server > handler 41 on 8032, call > org.apache.hadoop.yarn.api.ApplicationClientProtocolPB.getApplicationReport > from 172.16.20.30:54477 Call#2 Retry#0 > org.apache.hadoop.yarn.exceptions.ApplicationNotFoundException: > Application with id 'application_1451274628461_0003' doesn't exist in RM. > at > org.apache.hadoop.yarn.server.resourcemanager.ClientRMService.getApplicationReport(ClientRMService.java:327) > at > org.apache.hadoop.yarn.api.impl.pb.service.ApplicationClientProtocolPBServiceImpl.getApplicationReport(ApplicationClientProtocolPBServiceImpl.java:175) > at > org.apache.hadoop.yarn.proto.ApplicationClientProtocol$ApplicationClientProtocolService$2.callBlockingMethod(ApplicationClientProtocol.java:417) > at > org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:616) > at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:969) > at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2049) > at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2045) > at java.security.AccessController.doPrivileged(Native Method) > at javax.security.auth.Subject.doAs(Subject.java:415) > at > org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657) > at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2043) > 2015-12-28 14:34:02,555 INFO org.apache.hadoop.ipc.Server: IPC Server > handler 8 on 8032, call > org.apache.hadoop.yarn.api.ApplicationClientProtocolPB.getApplicationReport > from 172.16.20.30:54492 Call#4 Retry#0 > > > > > > > The information in this email is confidential and may be legally > privileged. If you have received this email in error or are not the > intended recipient, please immediately notify the sender and delete this > message from your computer. Any use, distribution, or copying of this email > other than by the intended recipient is strictly prohibited. All messages > sent to and from us may be monitored to ensure compliance with internal > policies and to protect our business. > Emails are not secure and cannot be guaranteed to be error free as they > can be intercepted, amended, lost or destroyed, or contain viruses. Anyone > who communicates with us by email is taken to accept these risks. > > 收发邮件者请注意： > 本邮件含保密信息，若误收本邮件，请务必通知发送人并直接删去，不得使用、传播或复制本邮件。 > 进出邮件均受到本公司合规监控。邮件可能发生被截留、被修改、丢失、被破坏或包含计算机病毒等不安全情况。 > > >
Re: Incremental builds assumptions and clarifications
> a) How does kylin cut the data (initial / incremental) into segments ? > Does one day become one segment ? b) When new data comes, does it > automatically figure out which segments to rebuild ? c) How to rebuild > only part of the data / segment via Rest API. User tells Kylin how to cut segment. Every build request must carry a start date and end date which defines the time boundary of segment. Segment never overlaps (normally) and should be continuous. Say you have 3 segments, [T1, T2), [T2, T3), [T3, T4). When your data changes, be it record addition, removal, or update, you need to refresh (or rebuild) the related segments. E.g. if 2 records on T2 were removed, you should refresh [T2, T3) segment. On Fri, Dec 25, 2015 at 4:36 PM, Abhilash L L wrote: > Thanks for the clarification Luke, Li Yang. > > Please find my comments / questions inline > > >Is there a document explaining the assumptions for incremental builds. > >> *Luke: I'm afraid there's no such doc yet. what's exactly "assumption" > you > >> are looking for, to know the code level implementation or how to > optimize?* > --> Not at code / implementation level. More at a feature level. Like the > one shared by Li Yang regarding a TS column for differentiation. Also, on > how it breaks it up into segments and how a user can rebuild part of the > segments > > > >Do we allow 'updates' on a facts ? > > 1) Because of some typo the quantity came in as 100 instead of 10. What > is > > the suggested approach to handle this. > >>So you want to refresh a built piece of data. And yes, that's doable. > Kylin > >>cut cube into segments by time period. You can refresh (or rebuild) a > >>segment without impacting the rests. > --> a) How does kylin cut the data (initial / incremental) into segments ? > Does one day become one segment ? b) When new data comes, does it > automatically figure out which segments to rebuild ? c) How to rebuild > only part of the data / segment via Rest API. > > > >> Luke: Do you mean data model changes? Then you have to disable that > >>cube, purge data and refine it, the rebuild it. > --> No only data changes, not model changes. For model as now I understand > we have to rebuild the full cube. > > > >How to support deletes in fact / dimension ? > > > >>* Luke: delete in fact table is fine, but in dimension should be > >>careful, properly it will require rebuild.* > Lets for a time period T1-T2 there were 100 records earlier, now due to > deletion it should be only 98 for the same time period. How to trigger > delete of the 2 records ? Is it to populate all 98 records in facttable > and then ask kylin to rebuild for T1-T2 ? > > > > > > > > > > > Regards, > Abhilash > > On Fri, Dec 25, 2015 at 7:35 AM, Li Yang wrote: > > > Em.. don't think Luke has all the questions fully answered. My additions. > > > > >Is there a document explaining the assumptions for incremental > builds. > > The only assumption (or requirement) is that there is date or timestamp > > column on the fact table that distinguishes the old from the new. > > > > >Do we allow 'updates' on a facts ? > > > 1) Because of some typo the quantity came in as 100 instead of 10. What > > is > > > the suggested approach to handle this. > > So you want to refresh a built piece of data. And yes, that's doable. > Kylin > > cut cube into segments by time period. You can refresh (or rebuild) a > > segment without impacting the rests. > > > > > 2) Lets say the the value for dimension 1 was d1 in the facttable. Now > it > > > got updated to d2 for the same dimension. How does it 'deduct' from the > > > aggregation for d1 for all cuboids and 'accumulate' for d2 in all > > cuboids. > > > > > >How to support Slowly Changing Dimensions (SCD). Support for type 2 > > and > > > type 3. > > The design is Kylin remembers data at the point it's built. So you may > > build a daily segment on T day with category set C in lookup table; then > on > > T+1 day, the category lookup table is updated into C~, and with that > build > > a T+1 daily segment. Now if you query the cube, it will report categories > > including both C and C~. More precisely Kylin will return C for T day > > transactions and C~ for T+1 transactions. > > > > If what you want is to reflect C~ in historic data, then earlier segments > > have to be rebuild. > > > > On Thu, Dec 24, 2015 at 10:59 PM, Luke Han wrote: > > > > > Hi Abhilash, > > > Please refer to below comments inline. > > > > > > Thanks. > > > > > > > > > Best Regards! > > > - > > > > > > Luke Han > > > > > > On Thu, Dec 10, 2015 at 2:28 PM, Abhilash L L > > > wrote: > > > > > > > Hello, > > > > > > > >Is there a document explaining the assumptions for incremental > > builds. > > > > *Luke: I'm afraid there's no such doc yet. what's exactly > "assumption" > > > you > > > > are looking for, to know the code level implementation or how to > > > optimize?* > > > > > > > > > > > > > > > > >Is it p
Re: Re: encouter Deserialization error when load hive table
Do you 'add jar' in hive prompt before accessing the table? If so, make sure that jar is in classpath.
[jira] [Created] (KYLIN-1269) Can not get last error message in email
fengYu created KYLIN-1269: - Summary: Can not get last error message in email Key: KYLIN-1269 URL: https://issues.apache.org/jira/browse/KYLIN-1269 Project: Kylin Issue Type: Bug Components: Job Engine Affects Versions: v1.0, 2.0 Reporter: fengYu Assignee: ZhouQianhao Fix For: v2.0 When I deploy kylin-2.0 in my env and job execute failed, When send email of job status changed, I get the exception : java.lang.IllegalArgumentException: Illegal group reference at java.util.regex.Matcher.appendReplacement(Matcher.java:808) at java.util.regex.Matcher.replaceAll(Matcher.java:906) at java.lang.String.replaceAll(String.java:2162) at org.apache.kylin.engine.mr.CubingJob.formatNotifications(CubingJob.java:140) at org.apache.kylin.job.execution.AbstractExecutable.notifyUserStatusChange(AbstractExecutable.java:213) at org.apache.kylin.job.execution.DefaultChainedExecutable.onExecuteFinished(DefaultChainedExecutable.java:100) at org.apache.kylin.engine.mr.CubingJob.onExecuteFinished(CubingJob.java:166) at org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:113) at org.apache.kylin.job.impl.threadpool.DefaultScheduler$JobRunner.run(DefaultScheduler.java:124) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:745) This is because error message contain special character like '$' or '\', replaceAll will throw exception while encounter this string. http://docs.oracle.com/javase/6/docs/api/java/lang/String.html#replaceAll(java.lang.String, java.lang.String) What is more, Kylin can not send error message of last executable job, I modify it too... -- This message was sent by Atlassian JIRA (v6.3.4#6332)
