hadoop的hdfs写入策略为可用空间，有人用过吗
Re: 生成的cube部分数据缺失
Hi DId you set "filter" when modeling? ( http://kylin.apache.org/docs/tutorial/create_cube.html) And check the time range when you built the cube. Make sure it is a full build. [image: image.png] 奥威软件 <3513797...@qq.com> 于2019年1月22日周二 上午7:25写道： > Hi > here are results: > > > kylin: > select count(*) from ICSTOCKBILL_1W > result:10366 > > > hive: > select count(*) from ICSTOCKBILL_1W > result:10411 > > > > > -- 原始邮件 -- > 发件人: "JiaTao Tao"; > 发送时间: 2019年1月22日(星期二) 下午3:04 > 收件人: "dev"; > > 主题: Re: 生成的cube部分数据缺失 > > > > Hi > Can you try "select count(*)" and compare the result with hive? > > FYI: http://kylin.apache.org/docs/gettingstarted/faq.html (Why I got an > error when running a “select * “ query?) > > > 奥威软件 <3513797...@qq.com> 于2019年1月22日周二 上午5:21写道： > > > 没有group by 也一样能查到数据的 > > 例如把 goodsid 改为1137， > > select * from ICSTOCKBILL_1W where goodsid = '1137' > > 结果： > > > > > > 现在的问题是cube的数据有部分缺失了，查不到数据， > > 造成汇总数据错误 > > > > > > hive表里的数据是完整的， > > 生成的cube缺丢失了部分数据 > > > > > > -- 原始邮件 -- > > *发件人:* "Chao Long"; > > *发送时间:* 2019年1月22日(星期二) 中午12:11 > > *收件人:* "dev"; > > *主题:* 回复：生成的cube部分数据缺失 > > > > Hi, > > The cube only has aggregated data, so your queries should include > "group > > by" clause. > > You can check the faq: > > http://kylin.apache.org/docs/gettingstarted/faq.html(# Why I got an > error > > when running a “select * “ query) > > > > > > -- > > Best Regards, > > Chao Long > > > > > > -- 原始邮件 -- > > 发件人: "奥威软件"<3513797...@qq.com>; > > 发送时间: 2019年1月22日(星期二) 中午11:06 > > 收件人: "dev"; > > > > 主题: 生成的cube部分数据缺失 > > > > > > > > Hi, > > > > > > 已确认，在hive表里，数据完整，但同样的查询语句查询事实表，hive有数据且完整，但查询cube 就没有数据， > > 查询语句如 select * from ICSTOCKBILL_1W where goodsid = '643'， > > ICSTOCKBILL_1W 为事实表， > > 通过查询语句 > > hive表有数据 > > 查询cube没有数据 > > > > > > 经常检查有个位数的goodsid 缺失了，且找不到规律和疑点。 > > > > > > env: > > ubuntu 16.04 hadoop集群 3台 > > apache-kylin-2.5.2-bin-hadoop3 or apache-kylin-2.6.0-bin-hadoop3 （kylin > > 单机） > > hive:2.3.4 > > hbase:1.3.3 > > zookeeper:3.4.13 > > > > > > > > 请帮忙看下怎么解决，谢谢！ > > > > > > Best regards > > > > > -- > > > Regards! > > Aron Tao -- Regards! Aron Tao
Could you help me with a survey?
Dear all, I am investigating how Apache development teams use static analysis tools (in particular SonarQube). To this end, I kindly ask you to answer a small survey on this topic. The survey is available at: https://canedo.typeform.com/to/JxPfG6 All the best. Professora Dra. Edna Dias Canedo Department of Computer Science University of Brasília (UnB), Campus Darcy Ribeiro
[jira] [Created] (KYLIN-3781) Can not start Kylin in Idea on Windows
Na Zhai created KYLIN-3781: -- Summary: Can not start Kylin in Idea on Windows Key: KYLIN-3781 URL: https://issues.apache.org/jira/browse/KYLIN-3781 Project: Kylin Issue Type: Bug Environment: Windows Reporter: Na Zhai Assignee: Chao Long After merge pr of issue https://issues.apache.org/jira/browse/KYLIN-3734, I get an error about ZookeeperDistributedLock when starting Kylin in Idea. After check code, I find that "return new File(path).getCanonicalPath();" in ZookeeperDistributedLock.java add the drive letter of my operating system. It will make zookeeper not get the lock. My environment is Windows. Error message: Caused by: java.lang.IllegalStateException: Error while 6260@NaZhai-PC trying to lock F:\kylin\kylin_metadata_idea\job_engine\global_job_engine_lock at org.apache.kylin.storage.hbase.util.ZookeeperDistributedLock.lock(ZookeeperDistributedLock.java:153) at org.apache.kylin.storage.hbase.util.ZookeeperDistributedLock.lock(ZookeeperDistributedLock.java:170) at org.apache.kylin.storage.hbase.util.ZookeeperDistributedLock.lockJobEngine(ZookeeperDistributedLock.java:316) at org.apache.kylin.storage.hbase.util.ZookeeperJobLock.lockJobEngine(ZookeeperJobLock.java:81) at org.apache.kylin.job.impl.threadpool.DefaultScheduler.init(DefaultScheduler.java:152) at org.apache.kylin.rest.service.JobService.afterPropertiesSet(JobService.java:126) at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.invokeInitMethods(AbstractAutowireCapableBeanFactory.java:1687) at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.initializeBean(AbstractAutowireCapableBeanFactory.java:1624) ... 61 more Caused by: java.lang.IllegalArgumentException: Path must start with / character at org.apache.curator.utils.PathUtils.validatePath(PathUtils.java:54) at org.apache.curator.utils.PathUtils.validatePath(PathUtils.java:37) at org.apache.curator.utils.ZKPaths.fixForNamespace(ZKPaths.java:63) at org.apache.curator.framework.imps.NamespaceImpl.fixForNamespace(NamespaceImpl.java:82) at org.apache.curator.framework.imps.CuratorFrameworkImpl.fixForNamespace(CuratorFrameworkImpl.java:579) at org.apache.curator.framework.imps.CreateBuilderImpl.forPath(CreateBuilderImpl.java:434) at org.apache.curator.framework.imps.CreateBuilderImpl.forPath(CreateBuilderImpl.java:44) at org.apache.kylin.storage.hbase.util.ZookeeperDistributedLock.lock(ZookeeperDistributedLock.java:149) ... 68 more -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (KYLIN-3782) ZookeeperDistributedLock can't acquir lock on windows because wrong Path
hailin.huang created KYLIN-3782: --- Summary: ZookeeperDistributedLock can't acquir lock on windows because wrong Path Key: KYLIN-3782 URL: https://issues.apache.org/jira/browse/KYLIN-3782 Project: Kylin Issue Type: Bug Components: Storage - HBase Affects Versions: v2.6.0 Reporter: hailin.huang Fix For: v2.6.1 In my windows env, when I run kylin, Kylin service can't start. In class ZookeeperDistributedLock, I found kylin use File(path).getCanonicalPath() to norm windows path, however this will get a path like ' C:\kylin\kylin_metadata1\job_engine\global_job_engine_lock', for zookeeper,Path must start with / character. Below is stackTrace: Caused by: java.lang.IllegalStateException: Error while 832820@UC-20180829ZLJB trying to lock C:\kylin\kylin_metadata1\job_engine\global_job_engine_lock at org.apache.kylin.storage.hbase.util.ZookeeperDistributedLock.lock(ZookeeperDistributedLock.java:153) at org.apache.kylin.storage.hbase.util.ZookeeperDistributedLock.lock(ZookeeperDistributedLock.java:170) at org.apache.kylin.storage.hbase.util.ZookeeperDistributedLock.lockJobEngine(ZookeeperDistributedLock.java:318) at org.apache.kylin.storage.hbase.util.ZookeeperJobLock.lockJobEngine(ZookeeperJobLock.java:81) at org.apache.kylin.job.impl.threadpool.DefaultScheduler.init(DefaultScheduler.java:152) at org.apache.kylin.rest.service.JobService.afterPropertiesSet(JobService.java:126) at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.invokeInitMethods(AbstractAutowireCapableBeanFactory.java:1687) at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.initializeBean(AbstractAutowireCapableBeanFactory.java:1624) ... 60 more caused by: java.lang.IllegalArgumentException: Path must start with / character at org.apache.curator.utils.PathUtils.validatePath(PathUtils.java:54) at org.apache.curator.utils.PathUtils.validatePath(PathUtils.java:37) at org.apache.curator.utils.ZKPaths.fixForNamespace(ZKPaths.java:63) at org.apache.curator.framework.imps.NamespaceImpl.fixForNamespace(NamespaceImpl.java:82) at org.apache.curator.framework.imps.CuratorFrameworkImpl.fixForNamespace(CuratorFrameworkImpl.java:579) at org.apache.curator.framework.imps.CreateBuilderImpl.forPath(CreateBuilderImpl.java:434) at org.apache.curator.framework.imps.CreateBuilderImpl.forPath(CreateBuilderImpl.java:44) at org.apache.kylin.storage.hbase.util.ZookeeperDistributedLock.lock(ZookeeperDistributedLock.java:149) -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[Discuss] Moving toward Apache Kylin 3.0
Hi Kylin developers, In last week, Kylin released v2.6.0, with the enhanced & distributed query cache and JDBC data source SDK. After this release, the next batch candidate features include real-time streaming, parquet storage, and druid storage. These features were developed in the past 1-2 years by different Kylin players and were open sourced in the past 6 months. They have already been staged in separate branches and are under evaluation by the community. We have received much feedback from the community. These candidate features are big supplements to as-is Kylin functions; For example, the real-time streaming feature will bring Kylin from batch & historical analytics into real-time analytics. The parquet storage will make the deployment more flexible and more cloud-friendly. Of course, stabilizing and improving these features need additional time and effort. So, when we merging and releasing them, we'd better give it a new version number so that user can clearly know the difference with current 2.x versions. I discussed this with several developers offline, we think it is time to move toward Kylin 3.0. So, if one of the above features is merged, the version will be 3.0. The current 2.6 will be maintained until 3.x is ready for production use. Your comments, ideas, and suggestions are welcomed! Best regards, Shaofeng Shi 史少锋 Apache Kylin PMC Work email: shaofeng@kyligence.io Kyligence Inc: https://kyligence.io/ Apache Kylin FAQ: https://kylin.apache.org/docs/gettingstarted/faq.html Join Kylin user mail group: user-subscr...@kylin.apache.org Join Kylin dev mail group: dev-subscr...@kylin.apache.org
