Re: [Discuss] Apache Kylin Component Owner Plan

may I take JDBC components, and backup with @Dong Li , Thanks. 2018-02-05 9:11 GMT+08:00 Jianhua Peng : > +1. > > I would like to take the ownership of "Security" components. > I also could be a backup owner for "Job Engine" components. > > Best regards, > Jianhua Peng > > On 2018/02/02 02:39:38,

Re: kylin job block hbase

g Shi : > Did you configure "kylin.hbase.cluster.fs", pointing to your HBase HDFS? > > Check this blog for more: > https://kylin.apache.org/blog/2016/06/10/standalone-hbase-cluster/ > > 2017-10-16 9:51 GMT+08:00 yu feng : > > > yes, hbase is running on another HD

Re: kylin job block hbase

ck HBase normal tasks. > > The HBase BulkLoad on HDFS should be very fast (second level), as it is > just a move operation. > > For your case, is your HBase running with another HDFS other than the > default HDFS? > > > 2017-10-13 16:16 GMT+08:00 yu feng : > > > A

kylin job block hbase

A very big cube, such as cube size id bigger than 1TB will block hbase's normal operation when doing the BulkLoad job (because the job will write to much data to HDFS), such as kylin metadata operation/ query. especially when the cube's merge job maybe write to hbase N TB in a mr job. Has anyone m

Re: different mr config for different project or cube

How to do it in GUI? by cube level configuration ? show me an example please. 2017-09-06 10:18 GMT+08:00 Billy Liu : > You could override the default MR config on the project level or cube level > through the GUI, not the file. > > 2017-09-06 10:11 GMT+08:00 yu feng : > >

different mr config for different project or cube

I remember Kylin support use different mr config file for different project, like KYLIN-1706 and KYLIN-1706 However, I do not know how to use it in kylin-2.0.0. it would be appreciated if anyone

Re: About kylin ACL

t; Click Access, then you can grant access at project level > > > 何京珂 > Joanna He > > > On 01/09/2017, 9:29 AM, "yu feng" wrote: > > How to grant permission at project level, I just find the way that > grant to > cube. > > > 2017-08-31 10

Re: About kylin ACL

How to grant permission at project level, I just find the way that grant to cube. 2017-08-31 10:24 GMT+08:00 Li Yang : > The ADMIN can grant permission to the MODELER at project level. After > creating the new project. > > On Tue, Aug 22, 2017 at 4:26 PM, yu feng wrote: > > &

About kylin ACL

In kylin, we have three role: ADMIN/ MODELER and ANALYST I find only ADMIN have Permission to create new project, But as a new normal MODELER user, What can I do in kylin? If ADMIN create a project for me, How to grant it to a MODELER user?

Kylin query return empty after upgrade to 2.0

Hi, After I upgrade our env. from 1.5.2.1 to 2.0.0, building things go well, However, every query to older(generated before upgrading) return empty(0 result), and newly build segment return result success. I add some debug log and find in GTFilterScanner, I get 0 input from inputIterator and retur

Re: [VOTE] Release apache-kylin-2.1.0 (RC2)

+1(no binding) build success md5&sha1 verified Best regards, 2017-08-15 23:57 GMT+08:00 Luke Han : > +1 (binding) > > mvn test passed > gpg/md5/sha1 verified > > > Best Regards! > - > > Luke Han > > On Tue, Aug 15, 2017 at 9:13 PM, 康凯森 wrote: > > > +1. > > > > > > Thanks Shao

Re: [VOTE] Release apache-kylin-2.0.0 (RC3)

+1(no binding) mvn test passed build(mvn clean install -DskipTests) success md5&sha1 verified 2017-04-29 0:30 GMT+08:00 Dong Li : > +1 binding > > mvn test passed > > Thanks, > Dong Li > > 2017-04-28 22:12 GMT+08:00 ShaoFeng Shi : > > > +1 binding; > > > > Verified mvn test, md5, sha1 and gpg sig

Re: [VOTE] Release apache-kylin-2.0.0 (RC2)

+1(no binding) mvn test passed build(mvn clean install -DskipTests) success md5&sha1 verified 2017-04-26 6:32 GMT+08:00 Li Yang : > Hi all, > > I have created a build for Apache Kylin 2.0.0, release candidate 2. > > Changes highlights: > > Support snowflake data model (KYLIN-1875) > Suppo

Re: Unable to connect to Kylin Web UI

we always use kylin index page : http://hostname:7070/kylin/login 2016-10-30 16:36 GMT+08:00 BigdataGR : > Hi, > > Can anyone please help me to move forward ? I am stuck at accessing kylin > web UI even kylin is started successfully as shown in the logs enclosed. > > Appreciate your help!!! > > -

Re: [VOTE] Release apache-kylin-1.5.4.1 (release candidate 2)

+1(binding) mvn test passed md5 and sha1 verify passed. 2016-09-26 21:35 GMT+08:00 Xiaoyu Wang : > +1 (binding) > mvn test passed > signature&md5&sha1 verified > > > > 在 2016年09月26日 20:16, Li Yang 写道: > >> +1 (binding) >> >> checked release commit >> >> mvn test passed on >> java version "1.7.0_7

normal user can not load table

when load table in a project, the user must has ADMIN role to syncTableToProject, is it some error about this logical

Re: id-name

derived dimension is a better choice. 2016-06-14 23:00 GMT+08:00 耳东 <775620...@qq.com>: > hi allwhen I build the cube, there are always dimension like > (city_id,city_name) or (terminal_id,terminal_name) and so on, which is one > to one relation. I don't want to let both id and name be dimens

TOPN group by column can not defined in dimension

I create a cube with kylin sample data, cube defination is below : { "uuid": "e6cf2ccc-edca-41c6-b637-b3bc50894b5e", "version": "1.5.1", "name": "kylin_sales_cube_desc_2_clone", "description": null, "dimensions": [ { "name": "CAL_DT", "table": "DEFAULT.KYLIN_CAL_DT",

TOPN group by column can not defined in dimension

I create a cube with kylin sample data, cube defination is below : { "uuid": "e6cf2ccc-edca-41c6-b637-b3bc50894b5e", "version": "1.5.1", "name": "kylin_sales_cube_desc_2_clone", "description": null, "dimensions": [ { "name": "CAL_DT", "table": "DEFAULT.KYLIN_CAL_DT",

Re: How to Build Binary Package in 1.5.x

the script has moved to ${source_code}/build/script/package.sh 2016-05-05 11:12 GMT+08:00 zeLiu : > 1.5.x has not script/package.sh,How to Build Binary Package? thanks > > -- > View this message in context: > http://apache-kylin.74782.x6.nabble.com/How-to-Build-Binary-Package-in-1-5-x-tp4410.html

Re: [VOTE] Release apache-kylin-1.5.1 (release candidate 1)

+1 (binding) signature&md5&sha1 verified mvn test passed 2016-04-09 17:38 GMT+08:00 Luke Han : > +1 (binding) > > mvn test passed > signature&md5&sha1 verified > > > Best Regards! > - > > Luke Han > > On Sat, Apr 9, 2016 at 5:15 PM, 周千昊 wrote: > > > +1 (binding) > > md5 pas

Re: [VOTE] Release apache-kylin-1.3.0 (release candidate 2)

+1(no binding) md5 & sha1 signature is Verified build success. mvn test passed. 2016-03-10 11:27 GMT+08:00 Li Yang : > +1 binding > > `mvn test` passed on java version "1.7.0_79", OpenJDK Runtime Environment > (rhel-2.5.5.1.el6_6-x86_64 u79-b14) > > On Thu, Mar 10, 2016 at 11:19 AM, nichunen >

Re: cube build failed if the fact_table is not in default database

FAILED: SemanticException 1:567 AS clause has an invalid number of aliases. I don't think it is error of hive, you should post you hive command here, and let's talk about it. 2016-03-10 1:07 GMT+08:00 Sarnath : > Its hard to believe that select * from db.table does not work... Have used > it man

Re: building cube error in step 2 (StandbyException)

you can deploy kylin at any node, you need ensure in that node you can run 'hadoop/hive/hbase' command in shell and get corrent outoput. you can try to move kylin to another node and config you hadoop configuration with HA mode which makes you can access it with nameservice. 2016-02-26 14:16 GMT+0

Re: when i use curl with Restful Api , HTTP Status 500 - Cannot create a session after the response has been committed

Hi, I have encountered this problem, this is caused by "Authorization: Basic QURNSU46S1lMSU4K" is incorrent, you should use "echo -n 'ADMIN:KYLIN'| base64" to generate base64 encoding, which output is "QURNSU46S1lMSU4=", and you can try it again. wish it can halp you ~ 2016-02-26 20:47 GMT+08:00

How to use Hybrid model in Kylin

Hi All : I find kylin-1.0 can generate Hybrid model which is composed by multiple cubes, jira is here : https://issues.apache.org/jira/browse/KYLIN-867 My question is how to use this feature in web UI. BTW, if I release some segments, but the cuboid files in HDFS and htable in Hbase of those

Re: How to use kylin with high cardinality dimensions.

IN-1313 for more details. 3. > > Adopt > > scalable dictionary solutions to replace current in-memory dictionary > > building (TBD) > > > > I'm also answering your questions inline with this pen > > > > On Thu, Feb 18, 2016 at 11:03 AM, yu feng wrote:

How to use kylin with high cardinality dimensions.

Hi All: We are encounting some problems while supporting a demand that a cube with some high cardinality dimensions, those dimensions are URLs and user want to use those dimensions in where clause and filter with like function. besides, the cube has one distinct count measure. We has such prob

Re: only one reducer in job

Hi, as far as I know, KYLIN-1066 is not about what you are talking about, this job "Kylin Hive Column Cardinality Job" is submitted after you loading table, it's function is calculate Cardinality of every column using hyperLogLog, so it have to u

Re: [DISCUSS]Apache Kylin 2.0 Release Features & Criteria

We are looking forward the release of 2.0, fast cubing algorithm, spark supporting and streaming cube is very useful to us. I have test 2.0-rc in our environment and it works fine, wish the release comes soon. 2016-02-02 18:02 GMT+08:00 Yerui Sun : > We’ve been looking forward the release of 2.0

Re: kylin concurrency test

re for convenience, bottleneck of kylin query throughput rely on hbase scan performance, which will related to regionserver number and machine configuration, network etc. 2016-01-28 10:08 GMT+08:00 Luke Han : > It's QPS, please contact Yu Feng (kylin committer) from NetEase for more > d

Re: Convert Cuboid Data to HFile failed when hbase in different HDFS

this problem caused by deploy hbase apart from hadoop, However we hadoop cluster can not recognize name service of HDFS which hbase depend on. 2016-01-22 15:55 GMT+08:00 Li Yang : > Sounds the same as https://issues.apache.org/jira/browse/KYLIN-957 > > > On Tue, Jan 19, 2016 at 10:23

Re: internal hive table and build the cube backward

it supplement, Hive definitely involves in the step of > > Create > > Intermediate Flat Hive Table and Build Dimension Dictionary. The question > > is that does Hive involve in the following steps of building cuboids? > > > > Best regards, > > Zhong > > >

Convert Cuboid Data to HFile failed when hbase in different HDFS

In the step 'Convert Cuboid Data to HFile' execute failed, error log is : java.io.IOException: Failed to run job : Unable to map logical nameservice URI 'hdfs://A' to a NameNode. Local configuration does not have a failover proxy provide r configured. at org.apache.hadoop.mapred.YARNRunner.submit

Re: internal hive table and build the cube backward

Firstly, kylin do not distinguish which kind table in hive, if only you can query it in hive, so the table can be normal table, external table, view or table with some serdes. then I think it is hard to build cube backward along the time in kylin. maybe someone has some good ideas at this point.

Re: Kylin capabilities

gt; > Luke Han > > On Sat, Jan 16, 2016 at 8:58 AM, yu feng wrote: > > > Does Kylin offer nesting dimensions or hierarchies? > > > > > Kylin does not support Snowflake schema, It is currently restricted to > > >Star. > > > > Generally sp

Re: Kylin capabilities

Does Kylin offer nesting dimensions or hierarchies? > Kylin does not support Snowflake schema, It is currently restricted to >Star. Generally speaking, we always transform snowflake schema to star schema by creating some view. 2016-01-16 3:31 GMT+08:00 Adunuthula, Seshu : > Does Kylin provide s

Re: Exception on building cube at 2nd step (IncompatibleClassChangeError)

I find there are some hadoop jar of 2.2.0 in /home/hadoop/hbase/hbase-0.98.15-hadoop2/lib, and your hadoop is 2.5.2, this will bring collision， please check your env again. 2016-01-16 4:26 GMT+08:00 Eric Engstfeld : > Hi, > > I have this exception on the 2nd step of cube build. I think its a > co

Re: can we support adding mapping cube columns to hive table columns

My suggestion is adding this mapping by creating view, So, you can change the column name in hive table, and recreate the view, that will do not have any effect to cube. and need not sync hive table. this view only used at the first step, so can recreate it in any time except doing this step. 2016

Re: Relationship between rowkey column length and cube size

Let me have a try to explain it. Cube size determines how to split region for table in hbase after generate all cuboid files, for example, If all of your cuboid file size is 100GB, your cube size set to "SMALL", and the property for SMALL is 10GB, kylin will create hbase table with 10 regions. it

Re: build a cube with two ultra high cardinality columns

assume average size of this column is 32 bytes, 50 millions cardinality means 1.5GB, in the step of 'Fact Table Distinct Columns.' mapper need read from intermediate table and remove duplicate values(do it in Combiner), however, this job will startup more than one mapper and just one reducer, there

Re: Welcome new Apache Kylin committer: Yu Feng

unce that the Project Management Committee > > (PMC) of Apache Kylin has asked Yu Feng to becomeApache Kylin committer, > > and she has already accepted. > > > > Yu has already made many contributions to Kylin community, to answer > > others questions activity, submit patches

Re: restful interface

- http-outgoing-0 << Date: > Fri, 08 Jan 2016 10:41:50 GMT > 18:43:03 MainClientExec - Connection can be kept alive indefinitely > 18:43:03 ResponseProcessCookies - Cookie accepted > [JSESSIONID="1B48E4EE1F327464EA6B32449346BBB7", version:0, > domain:192.168.1.1

Re: restful interface

You can add authentication infomation like this : protected void addHttpHeaders(HttpMethodBase method) { method.addRequestHeader("Accept", "application/json, text/plain, */*"); method.addRequestHeader("Content-Type", "application/json"); String basicAuth = DatatypeConverte

Re: Re: How to improve the performance of job！

r.java:1145) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) > at java.lang.Thread.run(Thread.java:745) > > My profile kylin_job_conf.xml： > > mapreduce.input.fileinputformat.split.maxsize > 64MB >

Re: How to improve the performance of job！

According to our experience: you can try those : 1、use newer hive to promote the first step. 2、startup more mapper and reducer for every MR job, you can reduce the value of 'kylin.job.mapreduce.default.reduce.input.mb' in kylin.properties which means input size for every reducer in NDCuboid calcula

Re: java.io.IOException: NoSuchObjectException(message:default.kylin_intermediate_learn_kylin

"hive.metastore.uris" property in hive-site.xml and > start hive metastore. so HCatalog can connect to it. > > > > 在 2016年01月07日 12:52, yu feng 写道: > >> you can check whether the table "default.kylin_intermediate_ >> learn_kylin_four_2015020100_2015123

Re: java.io.IOException: NoSuchObjectException(message:default.kylin_intermediate_learn_kylin

.jar,file:///usr/local/hive/lib/data-hive-udf.jar > The location of the plugin jars that contain > implementations of user defined functions and serdes. > > > > > > > -- 原始邮件 -- > 发件人: "yu feng";; > 发送时间: 2016年1月7日(星期四) 中午11:25 >

Re: java.io.IOException: NoSuchObjectException(message:default.kylin_intermediate_learn_kylin

I have encountered this problem, it is more likely because your current hive metastore config is error, could you tell some more detailed infomation about your env. 2016-01-07 10:23 GMT+08:00 和风 <363938...@qq.com>: > hi: > i build cube have a error. > logs: > java.io.IOException: > NoSuchObject

Re: Error when scan from lower key

This means your one of hbase scanner execute failed, you can check logs in regionserver for more detailed infomation. 2016-01-07 3:46 GMT+08:00 sdangi : > Hi Kylin Team -- > > > > I'm hitting the below error when executing query with the time dimension in > the where clause against a 35GB cube

Re: aboutthe; parameter 'acceptPartial'

In my opinion, acceptPartial means whether return partial result if your query result has more rows than limit value , and it will always set to true no matter query from web UI or jdbc. However, if you query from web UI, you can set the limit (default is 5), if you query from jdbc, the default

Feature about taking more than one hive source as kylin input in kylin-1.x

Hi all: I have been submit a patch on kylin-1.0(KYLIN-1172), it can take more hive as kylin input. even those hive based on different hadoop cluster. However, After I consult with @Luke Han and @Shi, Shaofeng, they do not plan to add new big fearures to kylin-1.x which is in a stable state.

Re: FAILED: ClassCastException org.apache.hadoop.hive.ql.plan.ConditionalWork cannot be cast to org.apache.hadoop.hive.ql.plan.MapredWork

execute this sql with hive -e 'sql' and check the result, I think this is something error with you hive env, maybe hive version problem... 2015-12-28 18:51 GMT+08:00 taylor zhang : > hi all > > I got below error when ran example cude 'kylin_sales_cube', it seems that > hive cannot insert data int

Re: cube model will be overridden while creating a new cube with the same name

che.org/jira/browse/KYLIN-1254 > > > > On Fri, Dec 25, 2015 at 3:17 PM, yu feng wrote: > > > >> Hi, I find a Bug like this in kylin-1.0: > >> I build a cube named TEST successfully, and then build another cube > named > >> TEST too, those two cube

cube model will be overridden while creating a new cube with the same name

Hi, I find a Bug like this in kylin-1.0: I build a cube named TEST successfully, and then build another cube named TEST too, those two cube has different fact table, the second cube will be error becacuse "The cube named TEST already exists", But the model of first cube will be overridden. this is

Re: error in calculate cardinality for view.

> https://issues.apache.org/jira/browse/KYLIN-916 > The workaround is to load and then calculate cardinality for the > underlying table which is not a view; > > On 12/24/15, 3:11 PM, "yu feng" wrote: > > >I load a view in onew project, However, calcu

error in calculate cardinality for view.

I load a view in onew project, However, calculat cardinality always error like this: java.io.IOException: java.lang.NullPointerException at org.apache.hive.hcatalog.mapreduce.HCatInputFormat.setInput(HCatInputFormat.java:97) at org.apache.hive.hcatalog.mapreduce.HCatInputFormat.se

Re: ClassNotFoundException: org.apache.kylin.common.mr.KylinMapper

In my current version(1.0)， for every mr job, kylin will find jar based on 'kylin.job.jar' property in config, if it not exist, it check file located at $kylin_home/lib/kylin-job-(.+)\\.jar, So, is your config right, or have you renamed the kylin-job-xxx.jar ? 2015-12-24 10:03 GMT+08:00 杨海乐 : > H

Re: Questions about query: cache, offset and order by desc

In my opinion, maybe you can increase your ehcache which set in ehcache.xml or ehcache-test.xml based on you "profile" setting, but I find the memoryStoreEvictionPolicy is "LRU", I am not clear about how it works, hope someone can explain it... you can disable the auto adding if you query it with j

Re: build dictionary for timestamp column with DateStrDictionary

imestamp is being supported > > > > On Tue, Dec 15, 2015 at 4:00 PM, yu feng wrote: > > > >> Hi All : > >> I build a cube, fact table like this : > >> hive> describe testtimestamp; > >> OK > >> ts times

build dictionary for timestamp column with DateStrDictionary

Hi All : I build a cube, fact table like this : hive> describe testtimestamp; OK ts timestamp fname string lname string type int cost int I build a cube with dimensions 'ts', 'fname', 'lname' and type, However, after

Re: about assigning different mr job queue to different user groups inside one kylin instances

Maybe a quick solution is just create a config file for every project named ''kylin_job_conf_${projectname}.xml", it will just modify code that selecting config file for a MR job, and you can control every MR config property in project level. 2015-12-11 22:00 GMT+08:00 Shi, Shaofeng : > This is a

Re: The Apache Software Foundation Announces Apache™ Kylin™ as a Top-Level Project

Congratulations！ 2015-12-11 10:10 GMT+08:00 hongbin ma : > ​cheers!​ > > On Thu, Dec 10, 2015 at 5:21 PM, Hao Chen wrote: > > > Congrats, Apache™ Kylin™ > > > > -- > > > > Hao > > > > > > On Thu, Dec 10, 2015 at 4:09 PM, Li Yang wrote: > > > > > Cheers! > > > > > > On Wed, Dec 9, 2015 at 12:13

Re: How to choose appropriate Cube size

Cube size will affect how to split region when creating htable, smaller cube size will cause more region with regard to the same source dara. 2015-12-10 16:51 GMT+08:00 Li Yang : > Em.. do need more document. > > The "Cube Size" under the advanced setting affects the MR job > configuration. Unde

Function of Kylin Hive Column Cardinality Job

Hi all: every time I load a hive table, kylin will do a job named "Kylin Hive Column Cardinality Job" asynchronous, which will calculate the Cardinality of every column with HyperLogLog, I am doubt the actual effect. any way to close it ?

Re: Kylin ingnores startTime in cube build process?

Kylin will ignore startTime while building a segment, It will take 1) start time you set in building cube if you are build first segment, 2) end time of last segment if you append new segment as start time of new segment. 2015-12-07 20:22 GMT+08:00, Marek Wiewiorka : > Hi All, > I came across a we

Re: Can not send email caused by Build Base Cuboid Data step failed

which lead to error in replaceAll function. Just remind you if you have not met this situation. 2015-10-29 18:33 GMT+08:00 yu feng : > Good for you~ > > 2015-10-29 18:15 GMT+08:00 Li Yang : > >> Met this one before. It has been fixed on latest code. >> >> On Tue,