Re:Re: [Announce] Welcome new Apache Kylin committer :Allen Ma
Thanks Luke and everyone's help, it is my honor to be a committer of Kylin, wish Kylin community become better and better. Thanks Gang(Allen) Ma At 2018-10-17 09:50:51, "Zhong, Yanghong" wrote: Congrats, Gang Ma! -- Best regards, Yanghong Zhong From: ShaoFeng Shi Reply-To: "u...@kylin.apache.org" Date: Wednesday, October 17, 2018 at 9:45 AM To: user Cc: dev , Apache Kylin PMC Subject: Re: [Announce] Welcome new Apache Kylin committer :Allen Ma Congratulations, Gang! Billy Liu 于2018年10月16日周二下午11:36写道： Welcome Allen. With Warm regards Billy Liu Luke Han 于2018年10月16日周二下午11:28写道： > > I am very pleased to announce that the Project Management Committee (PMC) of > Apache Kylin has asked Allen Ma (Gang Ma) to become Apache Kylin committer, > and he has already accepted. > > Allen has already made many contributions to Kylin community, to answer > others questions actively, submit patches for bug fixes and contribute to > some features. We are so glad to have him to be our new committer. > > Please join me to welcome Allen. > > Luke Han > > On behalf of the Apache Kylin PPMC -- Best regards, Shaofeng Shi 史少锋
Re: [Announce] Welcome new Apache Kylin committer :Allen Ma
Congrats, Gang Ma! -- Best regards, Yanghong Zhong From: ShaoFeng Shi Reply-To: "u...@kylin.apache.org" Date: Wednesday, October 17, 2018 at 9:45 AM To: user Cc: dev , Apache Kylin PMC Subject: Re: [Announce] Welcome new Apache Kylin committer :Allen Ma Congratulations, Gang! Billy Liu mailto:billy...@apache.org>> 于2018年10月16日周二 下午11:36写道： Welcome Allen. With Warm regards Billy Liu Luke Han mailto:luke...@apache.org>> 于2018年10月16日周二 下午11:28写道： > > I am very pleased to announce that the Project Management Committee (PMC) of > Apache Kylin has asked Allen Ma (Gang Ma) to become Apache Kylin committer, > and he has already accepted. > > Allen has already made many contributions to Kylin community, to answer > others questions actively, submit patches for bug fixes and contribute to > some features. We are so glad to have him to be our new committer. > > Please join me to welcome Allen. > > Luke Han > > On behalf of the Apache Kylin PPMC -- Best regards, Shaofeng Shi 史少锋
Re: [Announce] Welcome new Apache Kylin committer :Allen Ma
Congratulations, Gang! Billy Liu 于2018年10月16日周二 下午11:36写道： > Welcome Allen. > > With Warm regards > > Billy Liu > > Luke Han 于2018年10月16日周二 下午11:28写道： > > > > I am very pleased to announce that the Project Management Committee > (PMC) of Apache Kylin has asked Allen Ma (Gang Ma) to become Apache Kylin > committer, and he has already accepted. > > > > Allen has already made many contributions to Kylin community, to answer > others questions actively, submit patches for bug fixes and contribute to > some features. We are so glad to have him to be our new committer. > > > > Please join me to welcome Allen. > > > > Luke Han > > > > On behalf of the Apache Kylin PPMC > -- Best regards, Shaofeng Shi 史少锋
Re: [Announce] Welcome new Apache Kylin committer :Allen Ma
Welcome Allen. With Warm regards Billy Liu Luke Han 于2018年10月16日周二 下午11:28写道： > > I am very pleased to announce that the Project Management Committee (PMC) of > Apache Kylin has asked Allen Ma (Gang Ma) to become Apache Kylin committer, > and he has already accepted. > > Allen has already made many contributions to Kylin community, to answer > others questions actively, submit patches for bug fixes and contribute to > some features. We are so glad to have him to be our new committer. > > Please join me to welcome Allen. > > Luke Han > > On behalf of the Apache Kylin PPMC
[Announce] Welcome new Apache Kylin committer :Allen Ma
I am very pleased to announce that the Project Management Committee (PMC) of Apache Kylin has asked Allen Ma (Gang Ma) to become Apache Kylin committer, and he has already accepted. Allen has already made many contributions to Kylin community, to answer others questions actively, submit patches for bug fixes and contribute to some features. We are so glad to have him to be our new committer. Please join me to welcome Allen. Luke Han On behalf of the Apache Kylin PPMC
回复：api to build cube
thanks, i found that when use api to rebuild cube(non partition related), i will get http 500. pls see attached. Could you pls advise? thanks Thanks, Sent from phone 原始邮件 主题：答复: api to build cube 发件人：Na Zhai 收件人：dev@kylin.apache.org 抄送： Hi, koulong You are right. It is related to the partition that you choose in model. If you do not choose the partition, the start and end time will not need. I hope this can help you. http://kylin.apache.org/docs/howto/howto_build_cube_with_restapi.html 发送自 Windows 10 版邮件应用 发件人: 叩 龙 发送时间: Monday, October 15, 2018 10:18:44 PM 收件人: kylin dev 主题: api to build cube hi team, i am trying to build cube. from the kylin web page, i can see some of the cubes reqiure start and end time but some not. what is the different between them? does it related to partition？ also according to the rest api kylin provided, which api shall i use if the cube doesnt require starttime and endtime at all? Thanks, Sent from phone
Re: kylin 作为Grafana 支持的一个数据源
If your column is by hours, Days, . . . this use case is good for Apache Kylin If your column is by TimeStamp, is not the best scenario for Apache Kylin this means what in the best scenario in Grafana, you will see values grouped by Hours On Tue, 16 Oct 2018 at 13:20, 潘博存 wrote: > > >- >1.Grafana is time-based and needs to wrap the time columns, but that > doesn't mean that grafana's data sources are all sequential databases, just > as grafana supports MySQL and SQL Server。 >- >2.In our business scenario, we put more emphasis on Grafan's external > presentation capabilities, and in terms of timelines we use our business > dates by day, hour, etc. > > > > So I think grafana + kylin is another form of presentation besides saiku, > tableup, and so on. In fact, we're trying to put saiku as a grafan's layout > plug-in into grafan for data presentation > > > > > > -- > 发件人：Alberto Ramón > 发送时间：2018年10月16日(星期二) 17:31 > 收件人：user > 抄 送：潘博存 ; dev > 主 题：Re: kylin 作为Grafana 支持的一个数据源 > > I checked this possibility time ago (2-3years) > Grafana is focus in time-line series (one column must be TimeStamp) > Work with TS doesn't sense in A Kylin, because you are not aggregating > > On Tue, 16 Oct 2018 at 06:04, ShaoFeng Shi wrote: > Good question, let me translate it to English: > > Grafana is one of our important data visualization tools; Kylin is a > powerful tool for big data query, we want to display Kylin data on Grafana, > is there anyone already running this solution? Is there a grafana-kylin > plugin that can be used directly? Currently, Grafana doesn't provide a > plugin for Kylin. > > 潘博存 于2018年10月16日周二 上午11:27写道： > > hi,all >大数据可视化这一块，Grafana 是我们的一个重要展现工具，kylin 的快速查询 是大数据查询的利器，我们想在grafana > 上展示kylin的数据，不知道大家有没有这样使用的？是否有可以直接使用的grafana -kylin 插件.目前grafana > 是没有kylin的插件的， > > > -- > Best regards, > > Shaofeng Shi 史少锋 > > >
Re: kylin 作为Grafana 支持的一个数据源
I checked this possibility time ago (2-3years) Grafana is focus in time-line series (one column must be TimeStamp) Work with TS doesn't sense in A Kylin, because you are not aggregating On Tue, 16 Oct 2018 at 06:04, ShaoFeng Shi wrote: > Good question, let me translate it to English: > > Grafana is one of our important data visualization tools; Kylin is a > powerful tool for big data query, we want to display Kylin data on Grafana, > is there anyone already running this solution? Is there a grafana-kylin > plugin that can be used directly? Currently, Grafana doesn't provide a > plugin for Kylin. > > 潘博存 于2018年10月16日周二 上午11:27写道： > >> >> hi,all >>大数据可视化这一块，Grafana 是我们的一个重要展现工具，kylin 的快速查询 是大数据查询的利器，我们想在grafana >> 上展示kylin的数据，不知道大家有没有这样使用的？是否有可以直接使用的grafana -kylin 插件.目前grafana >> 是没有kylin的插件的， >> > > > -- > Best regards, > > Shaofeng Shi 史少锋 > >
Re: [DISCUSS] Columnar storage engine for Apache Kylin
Hi guys, I uploaded the initial design document to JIRA, please feel free to comment: https://issues.apache.org/jira/browse/KYLIN-3621 ShaoFeng Shi 于2018年10月12日周五 上午9:44写道： > JIRA and sub-tasks are created for this. Welcome to comment there: > https://issues.apache.org/jira/browse/KYLIN-3621 > > ShaoFeng Shi 于2018年10月8日周一 下午2:45写道： > >> I agree; the new storage should be Hadoop/HDFS compliant, and also need >> be cloud storage (like S3, blob storage) friendly, as more and more users >> are running big data analytics in the cloud. >> >> Luke Han 于2018年10月7日周日 下午7:44写道： >> >>> It makes sense to bring a better storage option for Kylin. >>> >>> The option should be open and people could have different ways to create >>> an >>> adaptor for the underlying storage. >>> Considering huge adoptions of Kylin today are all run on Hadoop/HDFS, I >>> prefer for Parquet or ORC or other HDFS compatible option at this time. >>> It >>> will easy for people to upgrade to the next generation and keep >>> consistency. >>> >>> Looking forward to this feature to be rolled out soon. >>> >>> Thanks. >>> >>> >>> >>> Best Regards! >>> - >>> >>> Luke Han >>> >>> >>> On Wed, Oct 3, 2018 at 2:37 PM Li Yang wrote: >>> >>> > Love this discussion. Like to highlight 3 major roles HBase is playing >>> > currently, so we don't miss any of them when looking for a replacement. >>> > >>> > 1) Storage: A high speed big data storage >>> > 2) Cache: A distributed storage cache layer (was BlockCache) >>> > 3) MPP: A distributed computation framework (was Coprocessor) >>> > >>> > The "Storage" seems at the central of discussion. Be it Parquet, ORC, >>> or a >>> > new file format, to me the standard interface is most important. As >>> long as >>> > we have consensus on the access interface, like MapReduce / Spark >>> Dataset, >>> > then the rest of debate can be easily resolved by a fair benchmark. >>> Also it >>> > allows people with different preference to keep their own >>> implementation >>> > under the standard interface, and not impacting the rest of Kylin. >>> > >>> > The "Cache" and the "MPP" were more or less overlooked. I suggest we >>> pay >>> > more attentions to them. Apart from Spark and Alluxio, any other >>> > alternatives? Actually Druid is a well-rounded choice, as like HBase, >>> it >>> > covers all the 3 roles pretty well. >>> > >>> > In general, I prefer to choose from the state of the art instead of >>> > re-inventing. Indeed, Kylin is not a storage project. A new storage >>> format >>> > is not Kylin's mission. Any storage innovations we come across here >>> would >>> > be more beneficial if contribute to Parquet or ORC community. >>> > >>> > Regards >>> > Yang >>> > >>> > >>> > >>> > On Tue, Oct 2, 2018 at 11:20 AM ShaoFeng Shi >>> > wrote: >>> > >>> > > Hi Billy, >>> > > >>> > > Yes, the cloud storage should be considered. The traditional file >>> layouts >>> > > on HDFS may not work well on cloud storage. Kylin needs to allow >>> > extension >>> > > here. I will add this to the requirement. >>> > > >>> > > Billy Liu 于2018年9月29日周六 下午3:22写道： >>> > > >>> > > > Hi Shaofeng, >>> > > > >>> > > > I'd like to add one more character: cloud-native storage support. >>> > > > Quite a few users are using S3 on AWS, or Azure Data Lake Storage >>> on >>> > > > Azure. If new storage engine could be more cloud friendly, more >>> user >>> > > > could get benefits from it. >>> > > > >>> > > > With Warm regards >>> > > > >>> > > > Billy Liu >>> > > > ShaoFeng Shi 于2018年9月28日周五 下午2:15写道： >>> > > > > >>> > > > > Hi Kylin developers. >>> > > > > >>> > > > > HBase has been Kylin’s storage engine since the first day; Kylin >>> on >>> > > HBase >>> > > > > has been verified as a success which can support low latency & >>> high >>> > > > > concurrency queries on a very large data scale. Thanks to HBase, >>> most >>> > > > Kylin >>> > > > > users can get on average less than 1-second query response. >>> > > > > >>> > > > > But we also see some limitations when putting Cubes into HBase; I >>> > > shared >>> > > > > some of them in the HBaseConf Asia 2018[1] this August. The >>> typical >>> > > > > limitations include: >>> > > > > >>> > > > >- Rowkey is the primary index, no secondary index so far; >>> > > > > >>> > > > > Filtering by row key’s prefix and suffix can get very different >>> > > > performance >>> > > > > result. So the user needs to do a good design about the row key; >>> > > > otherwise, >>> > > > > the query would be slow. This is difficult sometimes because the >>> user >>> > > > might >>> > > > > not predict the filtering patterns ahead of cube design. >>> > > > > >>> > > > >- HBase is a key-value instead of a columnar storage >>> > > > > >>> > > > > Kylin combines multiple measures (columns) into fewer column >>> families >>> > > for >>> > > > > smaller data size (row key size is remarkable). This causes HBase >>> > often >>> > > > > needing to read more data than requested. >>> > > > > >>> > >
