[jira] [Issue Comment Deleted] (KYLIN-1122) Kylin support detail data query from fact table
[ https://issues.apache.org/jira/browse/KYLIN-1122?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiaoyu Wang updated KYLIN-1122: --- Comment: was deleted (was: for the "3)" adjustSqlDigest may be contain the parameter "cubeDesc" will be better, In the raw custom function scenario, will use all columns or dimensions. ) > Kylin support detail data query from fact table > --- > > Key: KYLIN-1122 > URL: https://issues.apache.org/jira/browse/KYLIN-1122 > Project: Kylin > Issue Type: New Feature > Components: Query Engine >Affects Versions: v1.2 >Reporter: Xiaoyu Wang >Assignee: liyang > Fix For: v2.0, v1.3 > > Attachments: > 0001-KYLIN-1122-Kylin-support-detail-data-query-from-fact(2.x-staging).patch, > 0001-KYLIN-1122-Kylin-support-detail-data-query-from-fact(update-v2-1.x-staging).patch, > > 0001-KYLIN-1122-Kylin-support-detail-data-query-from-fact-new-impl-under-refactoring-2.x-staging.patch > > > Now Kylin does not support query correct detail rows from fact table like: > select column1,column2,column3 from fact_table > The jira KYLIN-1075 add the "SUM" function on the measure column if defined. > But only the column number type is support. > I change some code to support this issue: > Add a "VALUE" measure function : the same value and datatype in the input and > output of this function. > If you want to query detail data from fact table > *require*: > 1.Configure the column which not dimensions to "VALUE" or "SUM" measure.(If > not configure measure function in the column will get NULL value) > 2.The source table must has an unique value column and configure it as > dimension. > If you have the better solution please comment here. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KYLIN-1122) Kylin support detail data query from fact table
[ https://issues.apache.org/jira/browse/KYLIN-1122?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15095778#comment-15095778 ] Xiaoyu Wang commented on KYLIN-1122: for the "3)" adjustSqlDigest may be contain the parameter "cubeDesc" will be better, In the raw custom function scenario, will use all columns or dimensions. > Kylin support detail data query from fact table > --- > > Key: KYLIN-1122 > URL: https://issues.apache.org/jira/browse/KYLIN-1122 > Project: Kylin > Issue Type: New Feature > Components: Query Engine >Affects Versions: v1.2 >Reporter: Xiaoyu Wang >Assignee: liyang > Fix For: v2.0, v1.3 > > Attachments: > 0001-KYLIN-1122-Kylin-support-detail-data-query-from-fact(2.x-staging).patch, > 0001-KYLIN-1122-Kylin-support-detail-data-query-from-fact(update-v2-1.x-staging).patch, > > 0001-KYLIN-1122-Kylin-support-detail-data-query-from-fact-new-impl-under-refactoring-2.x-staging.patch > > > Now Kylin does not support query correct detail rows from fact table like: > select column1,column2,column3 from fact_table > The jira KYLIN-1075 add the "SUM" function on the measure column if defined. > But only the column number type is support. > I change some code to support this issue: > Add a "VALUE" measure function : the same value and datatype in the input and > output of this function. > If you want to query detail data from fact table > *require*: > 1.Configure the column which not dimensions to "VALUE" or "SUM" measure.(If > not configure measure function in the column will get NULL value) > 2.The source table must has an unique value column and configure it as > dimension. > If you have the better solution please comment here. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KYLIN-1122) Kylin support detail data query from fact table
[ https://issues.apache.org/jira/browse/KYLIN-1122?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15095779#comment-15095779 ] Xiaoyu Wang commented on KYLIN-1122: for the "3)" adjustSqlDigest may be contain the parameter "cubeDesc" will be better, In the raw custom function scenario, will use all columns or dimensions. > Kylin support detail data query from fact table > --- > > Key: KYLIN-1122 > URL: https://issues.apache.org/jira/browse/KYLIN-1122 > Project: Kylin > Issue Type: New Feature > Components: Query Engine >Affects Versions: v1.2 >Reporter: Xiaoyu Wang >Assignee: liyang > Fix For: v2.0, v1.3 > > Attachments: > 0001-KYLIN-1122-Kylin-support-detail-data-query-from-fact(2.x-staging).patch, > 0001-KYLIN-1122-Kylin-support-detail-data-query-from-fact(update-v2-1.x-staging).patch, > > 0001-KYLIN-1122-Kylin-support-detail-data-query-from-fact-new-impl-under-refactoring-2.x-staging.patch > > > Now Kylin does not support query correct detail rows from fact table like: > select column1,column2,column3 from fact_table > The jira KYLIN-1075 add the "SUM" function on the measure column if defined. > But only the column number type is support. > I change some code to support this issue: > Add a "VALUE" measure function : the same value and datatype in the input and > output of this function. > If you want to query detail data from fact table > *require*: > 1.Configure the column which not dimensions to "VALUE" or "SUM" measure.(If > not configure measure function in the column will get NULL value) > 2.The source table must has an unique value column and configure it as > dimension. > If you have the better solution please comment here. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (KYLIN-1313) Enable deriving dimensions on non PK/FK
hongbin ma created KYLIN-1313: - Summary: Enable deriving dimensions on non PK/FK Key: KYLIN-1313 URL: https://issues.apache.org/jira/browse/KYLIN-1313 Project: Kylin Issue Type: Improvement Reporter: hongbin ma Assignee: hongbin ma currently derived column has to be columns on look table, and the derived host column has to be PK/FK(It's also a problem when the lookup table grows every large). Sometimes columns on the fact exhibit deriving relationship too. Here's an example fact table: (dt date, seller_id bigint, seller_name varchar(100) , item_id bigint, item_url varchar(1000), count decimal, price decimal) seller_name is uniquely determined by each seller id, and item_url is uniquely determined by each item_id. The users does not expect to do filtering on columns like seller name or item_url, they just want to retrieve it when they do grouping/filtering on other dimensions like selller id, item id or even other dimensions like dt. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Closed] (KYLIN-1309) cordova-plugin-inappbrowser v1.1.1 android PDF rendering
[ https://issues.apache.org/jira/browse/KYLIN-1309?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] hongbin ma closed KYLIN-1309. - Resolution: Invalid > cordova-plugin-inappbrowser v1.1.1 android PDF rendering > > > Key: KYLIN-1309 > URL: https://issues.apache.org/jira/browse/KYLIN-1309 > Project: Kylin > Issue Type: Bug > Components: Web >Reporter: Wade Cantrell >Assignee: Zhong,Jason > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (KYLIN-1312) Enhance DeployCoprocessorCLI to support Cube level filter
Dong Li created KYLIN-1312: -- Summary: Enhance DeployCoprocessorCLI to support Cube level filter Key: KYLIN-1312 URL: https://issues.apache.org/jira/browse/KYLIN-1312 Project: Kylin Issue Type: Improvement Components: Tools, Build and Test Affects Versions: v2.0 Reporter: Dong Li Assignee: Dong Li Priority: Minor Currently DeployCoprocessorCLI can only filter on Tables, we need to enhance it to support Cube as filter. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (KYLIN-299) add service to differentiate the deploy env
[ https://issues.apache.org/jira/browse/KYLIN-299?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] hongbin ma updated KYLIN-299: - Assignee: Dong Li (was: ZhouQianhao) > add service to differentiate the deploy env > --- > > Key: KYLIN-299 > URL: https://issues.apache.org/jira/browse/KYLIN-299 > Project: Kylin > Issue Type: Wish > Components: Tools, Build and Test >Reporter: Luke Han >Assignee: Dong Li > Labels: github-import > Fix For: v0.6.5 > > > different deploy env (dev,qa,prod) may need different config, > need to add this feature > Imported from GitHub > Url: https://github.com/KylinOLAP/Kylin/issues/206 > Created by: [janzhongi|https://github.com/janzhongi] > Labels: newfeature, > Assignee: [janzhongi|https://github.com/janzhongi] > Created at: Wed Dec 10 10:46:23 CST 2014 > State: open -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (KYLIN-480) Add client information into Job
[ https://issues.apache.org/jira/browse/KYLIN-480?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] hongbin ma updated KYLIN-480: - Assignee: Dong Li (was: ZhouQianhao) > Add client information into Job > --- > > Key: KYLIN-480 > URL: https://issues.apache.org/jira/browse/KYLIN-480 > Project: Kylin > Issue Type: Wish > Components: Job Engine >Reporter: Luke Han >Assignee: Dong Li > Labels: github-import > Fix For: v2.0 > > > As an administer and cube owner, I would like to know one job be triggered by > which user, when, and triggered by which client, corn tab, web, 3rd party > client... > Imported from GitHub > Url: https://github.com/KylinOLAP/Kylin/issues/25 > Created by: [lukehan|https://github.com/lukehan] > Labels: enhancement, > Milestone: v0.6.5 Release > Assignee: [Kejia-Wang|https://github.com/Kejia-Wang] > Created at: Wed Oct 29 10:09:22 CST 2014 > State: open -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (KYLIN-557) HBase KV pair check before and after Bulkload
[ https://issues.apache.org/jira/browse/KYLIN-557?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] hongbin ma updated KYLIN-557: - Assignee: Dong Li (was: ZhouQianhao) > HBase KV pair check before and after Bulkload > - > > Key: KYLIN-557 > URL: https://issues.apache.org/jira/browse/KYLIN-557 > Project: Kylin > Issue Type: Wish > Components: Job Engine >Reporter: Luke Han >Assignee: Dong Li > Fix For: v2.0 > > > Data consistency is key of cube to serve query. > Verify KV pair number before and after bulkload to ensure all data be stored > well in HBase. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (KYLIN-227) Support "Pause" on Kylin Job
[ https://issues.apache.org/jira/browse/KYLIN-227?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] hongbin ma updated KYLIN-227: - Assignee: Dong Li (was: ZhouQianhao) > Support "Pause" on Kylin Job > > > Key: KYLIN-227 > URL: https://issues.apache.org/jira/browse/KYLIN-227 > Project: Kylin > Issue Type: Wish > Components: Job Engine >Reporter: Luke Han >Assignee: Dong Li > Labels: github-import > Fix For: Future > > > Add one action called "Pause" to stop current job, user could resume this job > later. > ![image|https://cloud.githubusercontent.com/assets/1104017/5556023/54ae27e2-8d07-11e4-8efb-a22c041243ba.png] > Imported from GitHub > Url: https://github.com/KylinOLAP/Kylin/issues/278 > Created by: [lukehan|https://github.com/lukehan] > Labels: newfeature, > Created at: Fri Dec 26 13:59:03 CST 2014 > State: open -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (KYLIN-196) Support Job Priority
[ https://issues.apache.org/jira/browse/KYLIN-196?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] hongbin ma updated KYLIN-196: - Assignee: Dong Li (was: ZhouQianhao) > Support Job Priority > > > Key: KYLIN-196 > URL: https://issues.apache.org/jira/browse/KYLIN-196 > Project: Kylin > Issue Type: New Feature > Components: Job Engine >Reporter: Luke Han >Assignee: Dong Li > Labels: github-import, newbie > Fix For: v2.0 > > > Support Job Priority in cube build engine. > Imported from GitHub > Url: https://github.com/KylinOLAP/Kylin/issues/309 > Created by: [lukehan|https://github.com/lukehan] > Labels: newfeature, > Milestone: Backlog > Created at: Fri Dec 26 15:04:45 CST 2014 > State: open -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (KYLIN-741) Read data from SparkSQL
[ https://issues.apache.org/jira/browse/KYLIN-741?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] hongbin ma updated KYLIN-741: - Assignee: Dong Li (was: ZhouQianhao) > Read data from SparkSQL > --- > > Key: KYLIN-741 > URL: https://issues.apache.org/jira/browse/KYLIN-741 > Project: Kylin > Issue Type: New Feature > Components: Job Engine, SparkSQL >Reporter: Luke Han >Assignee: Dong Li > Fix For: Backlog > > > Read data from SparkSQL directly. > There are some instances enabled SparkSQL interface for data consuming, it > will be great if Kylin could read data directly from SparkSQL. > This feature does not require Spark Cube Build Engine to be ready. It could > continue to leverage existing MR cube build engine and process data on Hadoop > cluster then persistent cube to HBase. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (KYLIN-1057) No email notification on job failure in some rare cases
[ https://issues.apache.org/jira/browse/KYLIN-1057?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] hongbin ma updated KYLIN-1057: -- Assignee: Dong Li (was: ZhouQianhao) > No email notification on job failure in some rare cases > --- > > Key: KYLIN-1057 > URL: https://issues.apache.org/jira/browse/KYLIN-1057 > Project: Kylin > Issue Type: Bug > Components: Job Engine >Reporter: Shaofeng SHI >Assignee: Dong Li > > Usually Kylin will send email notification (if configured) on job failure; > while today we found there is no email under two job failures; This downgrade > the admin's response efficiency, need check the root cause. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (KYLIN-559) put different build job's log into different files
[ https://issues.apache.org/jira/browse/KYLIN-559?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] hongbin ma updated KYLIN-559: - Assignee: Dong Li (was: ZhouQianhao) > put different build job's log into different files > -- > > Key: KYLIN-559 > URL: https://issues.apache.org/jira/browse/KYLIN-559 > Project: Kylin > Issue Type: Wish >Reporter: hongbin ma >Assignee: Dong Li >Priority: Minor > > currently all build job's log are messed up in a single file > (kylin_job.log?), which is very unfriendly to those who want to check logs -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (KYLIN-745) Generic Data Reader
[ https://issues.apache.org/jira/browse/KYLIN-745?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] hongbin ma updated KYLIN-745: - Assignee: Dong Li (was: ZhouQianhao) > Generic Data Reader > --- > > Key: KYLIN-745 > URL: https://issues.apache.org/jira/browse/KYLIN-745 > Project: Kylin > Issue Type: Wish > Components: Job Engine, Spark Engine >Reporter: Luke Han >Assignee: Dong Li > > When data be stored on existing DW like Oracle, it's not be able to read > directly through Kylin to build cube. > There are many requirements coming from different teams like Candor about > this. > There are two options: > #1, copy your data to Hive and then build cube through Kylin. There are some > cases are running this model to bring data into Hive from DW and leveraging > Kylin very well. > #2, rewrite data read module to pull data from Oracle directly. Actually, > the first step of cube build is generate Hive Query to read data and generate > one temp table in Hive, so it should be not too complicated to do this (but > it depends network and others, otherwise, #1 will be more efficient one). > Then process cube build as normal. Using generical reader to read data from > any SQL rdbms through JDBC or other protocol will be perfect solution since > cube could be built without ETL process > Scope: > Only ready data directly from existing RDBMs and store jointed result in Hive > (temp table) for further processing, no any other transfomation. > By design, Kylin is OLAP system not ETL one. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (KYLIN-580) Auto resume jobs after server restart
[ https://issues.apache.org/jira/browse/KYLIN-580?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] hongbin ma updated KYLIN-580: - Assignee: Dong Li (was: ZhouQianhao) > Auto resume jobs after server restart > - > > Key: KYLIN-580 > URL: https://issues.apache.org/jira/browse/KYLIN-580 > Project: Kylin > Issue Type: New Feature > Components: Job Engine >Reporter: Luke Han >Assignee: Dong Li >Priority: Minor > Fix For: Backlog > > > Once kylin service re-deployed or restarted, all running jobs will be set to > pending status. > There's requirement to auto resume such jobs after these actions > And please leave one setting in kylin.properties to let administrator enable > this feature or not. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (KYLIN-1094) improve performance of spark cubing
[ https://issues.apache.org/jira/browse/KYLIN-1094?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] hongbin ma updated KYLIN-1094: -- Assignee: Dong Li (was: ZhouQianhao) > improve performance of spark cubing > --- > > Key: KYLIN-1094 > URL: https://issues.apache.org/jira/browse/KYLIN-1094 > Project: Kylin > Issue Type: Sub-task > Components: Spark Engine >Affects Versions: v2.0 >Reporter: ZhouQianhao >Assignee: Dong Li > Fix For: v2.1 > > > POC result of spark cubing shows that, on a dataset of 150 million records, > MR is about 100% faster than Spark, however we believe that Spark could be at > least at same speed as MR, so optimization is needed here. > We are asking Spark community for help now. > the cluster info: > vm: 8 nodes * (128G mem + 64 core) > hadoop cluster: hdp 2.2.6 > spark running mode: yarn-client > spark version: 1.5.1 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (KYLIN-744) Spark Cube Build Engine
[ https://issues.apache.org/jira/browse/KYLIN-744?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] hongbin ma updated KYLIN-744: - Assignee: Dong Li (was: ZhouQianhao) > Spark Cube Build Engine > --- > > Key: KYLIN-744 > URL: https://issues.apache.org/jira/browse/KYLIN-744 > Project: Kylin > Issue Type: New Feature > Components: Spark Engine >Reporter: Luke Han >Assignee: Dong Li > > Efficiency cube generate engine with Spark instead of MapReduce job. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (KYLIN-1221) Allow user to kill Hadoop job in the job list page
[ https://issues.apache.org/jira/browse/KYLIN-1221?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] hongbin ma updated KYLIN-1221: -- Assignee: Dong Li (was: ZhouQianhao) > Allow user to kill Hadoop job in the job list page > -- > > Key: KYLIN-1221 > URL: https://issues.apache.org/jira/browse/KYLIN-1221 > Project: Kylin > Issue Type: Wish > Components: Job Engine >Affects Versions: v1.1, v1.0, v0.7.2, v0.7.1 >Reporter: Shaofeng SHI >Assignee: Dong Li > Labels: newbie > Fix For: Backlog > > > Sometimes user expects to abondon the current running hadoop job, he can > choose to kill the related job directly from the Kylin job page, instead of > manually run yarn or mapred CLI to kill it. > Besides, when user discards a Kylin job, the underlying running hadoop job > should also be killed automatically, to release the computing resources in > time. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (KYLIN-947) streaming fail at fetching timestamp from kafka
[
https://issues.apache.org/jira/browse/KYLIN-947?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
hongbin ma updated KYLIN-947:
-
Assignee: Dong Li (was: ZhouQianhao)
> streaming fail at fetching timestamp from kafka
> ---
>
> Key: KYLIN-947
> URL: https://issues.apache.org/jira/browse/KYLIN-947
> Project: Kylin
> Issue Type: Bug
>Affects Versions: v2.0
>Reporter: hongbin ma
>Assignee: Dong Li
>
> It seems kafka is failing to fetch a offset that is exactly between
> startoffset and endoffset, which should not happen at all
> 15/08/13 21:25:21 INFO streaming.StreamingUtil: topic:
> Trkng.druid-sessionEvent, partitionId: 1, try to find closest offset with
> timestamp: 143949810 between offset {4476901430, 4479189591}
> 15/08/13 21:26:25 ERROR streaming.StreamingBootstrap: fail to get start
> offset partitionId: 1, target timestamp: 143949810
> java.lang.IllegalStateException: try to get timestamp of topic:,
> partitionId: 1, offset: 4478045510, failed to get StreamMessage from kafka
> at
> org.apache.kylin.streaming.StreamingUtil.getKafkaMessage(StreamingUtil.java:72)
> at
> org.apache.kylin.streaming.StreamingUtil.getDataTimestamp(StreamingUtil.java:135)
> at
> org.apache.kylin.streaming.StreamingUtil.getDataTimestamp(StreamingUtil.java:127)
> at
> org.apache.kylin.streaming.StreamingUtil.binarySearch(StreamingUtil.java:100)
> at
> org.apache.kylin.streaming.StreamingUtil.findClosestOffsetWithDataTimestamp(StreamingUtil.java:80)
> at
> org.apache.kylin.job.streaming.StreamingBootstrap$4.run(StreamingBootstrap.java:265)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:745)
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (KYLIN-903) automate metadata cleanup job
[ https://issues.apache.org/jira/browse/KYLIN-903?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] hongbin ma updated KYLIN-903: - Assignee: Dong Li (was: ZhouQianhao) > automate metadata cleanup job > - > > Key: KYLIN-903 > URL: https://issues.apache.org/jira/browse/KYLIN-903 > Project: Kylin > Issue Type: Improvement >Reporter: ZhouQianhao >Assignee: Dong Li > Fix For: v2.0 > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (KYLIN-1241) support machine resource isolation in project level with different kylin_job_conf.xml file
[ https://issues.apache.org/jira/browse/KYLIN-1241?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] hongbin ma updated KYLIN-1241: -- Assignee: Dong Li (was: ZhouQianhao) > support machine resource isolation in project level with different > kylin_job_conf.xml file > -- > > Key: KYLIN-1241 > URL: https://issues.apache.org/jira/browse/KYLIN-1241 > Project: Kylin > Issue Type: New Feature > Components: Job Engine >Affects Versions: v1.1.1 >Reporter: Wang,Dong >Assignee: Dong Li >Priority: Minor > Fix For: v1.3 > > Attachments: KYLIN-1241.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (KYLIN-1302) java.lang.NullPointerException
[ https://issues.apache.org/jira/browse/KYLIN-1302?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] hongbin ma updated KYLIN-1302: -- Assignee: Dong Li (was: ZhouQianhao) > java.lang.NullPointerException > -- > > Key: KYLIN-1302 > URL: https://issues.apache.org/jira/browse/KYLIN-1302 > Project: Kylin > Issue Type: Bug > Components: Job Engine >Affects Versions: v1.2 > Environment: CDH-5.4.8-1,hadoop-2.6.0,hive-1.1.0,hbase-1.0.0 >Reporter: gewenhui >Assignee: Dong Li > Fix For: 1.2 > > > in the UI of 7070,job run here: > #2 Step Name: Extract Fact Table Distinct Columns > 2016-01-10 07:44:35.222 - State of Hadoop job: job_1449729554133_9107:RUNNING > - UNDEFINED > 2016-01-10 07:44:45.236 - State of Hadoop job: > job_1449729554133_9107:FINISHED - SUCCEEDED > org.apache.hadoop.ipc.RemoteException(java.lang.NullPointerException): > java.lang.NullPointerException > at > org.apache.hadoop.mapreduce.v2.hs.HistoryClientService$HSClientProtocolHandler.getCounters(HistoryClientService.java:228) > at > org.apache.hadoop.mapreduce.v2.api.impl.pb.service.MRClientProtocolPBServiceImpl.getCounters(MRClientProtocolPBServiceImpl.java:159) > at > org.apache.hadoop.yarn.proto.MRClientProtocol$MRClientProtocolService$2.callBlockingMethod(MRClientProtocol.java:281) > at > org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:619) > at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:1060) > at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2044) > at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2040) > at java.security.AccessController.doPrivileged(Native Method) > at javax.security.auth.Subject.doAs(Subject.java:415) > at > org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1671) > at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2038) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (KYLIN-1269) Can not get last error message in email
[ https://issues.apache.org/jira/browse/KYLIN-1269?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] hongbin ma updated KYLIN-1269: -- Assignee: Dong Li (was: ZhouQianhao) > Can not get last error message in email > --- > > Key: KYLIN-1269 > URL: https://issues.apache.org/jira/browse/KYLIN-1269 > Project: Kylin > Issue Type: Bug > Components: Job Engine >Affects Versions: v1.0, 2.0 >Reporter: fengYu >Assignee: Dong Li > Fix For: v2.0 > > Attachments: > 0002-KYLIN-1269-Can-not-get-last-error-message-in-email.patch > > > When I deploy kylin-2.0 in my env and job execute failed, When send email of > job status changed, I get the exception : > java.lang.IllegalArgumentException: Illegal group reference > at java.util.regex.Matcher.appendReplacement(Matcher.java:808) > at java.util.regex.Matcher.replaceAll(Matcher.java:906) > at java.lang.String.replaceAll(String.java:2162) > at > org.apache.kylin.engine.mr.CubingJob.formatNotifications(CubingJob.java:140) > at > org.apache.kylin.job.execution.AbstractExecutable.notifyUserStatusChange(AbstractExecutable.java:213) > at > org.apache.kylin.job.execution.DefaultChainedExecutable.onExecuteFinished(DefaultChainedExecutable.java:100) > at > org.apache.kylin.engine.mr.CubingJob.onExecuteFinished(CubingJob.java:166) > at > org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:113) > at > org.apache.kylin.job.impl.threadpool.DefaultScheduler$JobRunner.run(DefaultScheduler.java:124) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) > at java.lang.Thread.run(Thread.java:745) > This is because error message contain special character like '$' or '\', > replaceAll will throw exception while encounter this string. > http://docs.oracle.com/javase/6/docs/api/java/lang/String.html#replaceAll(java.lang.String, > java.lang.String) > What is more, Kylin can not send error message of last executable job, I > modify it too... -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (KYLIN-1311) Stream cubing auto assignment and load balance
Shaofeng SHI created KYLIN-1311: --- Summary: Stream cubing auto assignment and load balance Key: KYLIN-1311 URL: https://issues.apache.org/jira/browse/KYLIN-1311 Project: Kylin Issue Type: Sub-task Components: Job Engine Reporter: Shaofeng SHI Assignee: Shaofeng SHI Fix For: v2.1 This is a sub task of KYLIN-1117, need assign the stream cubing job to cluster for load balance and fail over. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Issue Comment Deleted] (KYLIN-1302) java.lang.NullPointerException
[ https://issues.apache.org/jira/browse/KYLIN-1302?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] gewenhui updated KYLIN-1302: Comment: was deleted (was: I use CDH-5.4.8-1,hadoop-2.6.0,hive-1.1.0,hbase-1.0.0,kylin-1.2) > java.lang.NullPointerException > -- > > Key: KYLIN-1302 > URL: https://issues.apache.org/jira/browse/KYLIN-1302 > Project: Kylin > Issue Type: Bug > Components: Job Engine >Affects Versions: v1.2 > Environment: CDH-5.4.8-1,hadoop-2.6.0,hive-1.1.0,hbase-1.0.0 >Reporter: gewenhui >Assignee: ZhouQianhao > Fix For: 1.2 > > > in the UI of 7070,job run here: > #2 Step Name: Extract Fact Table Distinct Columns > 2016-01-10 07:44:35.222 - State of Hadoop job: job_1449729554133_9107:RUNNING > - UNDEFINED > 2016-01-10 07:44:45.236 - State of Hadoop job: > job_1449729554133_9107:FINISHED - SUCCEEDED > org.apache.hadoop.ipc.RemoteException(java.lang.NullPointerException): > java.lang.NullPointerException > at > org.apache.hadoop.mapreduce.v2.hs.HistoryClientService$HSClientProtocolHandler.getCounters(HistoryClientService.java:228) > at > org.apache.hadoop.mapreduce.v2.api.impl.pb.service.MRClientProtocolPBServiceImpl.getCounters(MRClientProtocolPBServiceImpl.java:159) > at > org.apache.hadoop.yarn.proto.MRClientProtocol$MRClientProtocolService$2.callBlockingMethod(MRClientProtocol.java:281) > at > org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:619) > at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:1060) > at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2044) > at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2040) > at java.security.AccessController.doPrivileged(Native Method) > at javax.security.auth.Subject.doAs(Subject.java:415) > at > org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1671) > at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2038) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KYLIN-1302) java.lang.NullPointerException
[ https://issues.apache.org/jira/browse/KYLIN-1302?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15095559#comment-15095559 ] gewenhui commented on KYLIN-1302: - I use CDH-5.4.8-1,hadoop-2.6.0,hive-1.1.0,hbase-1.0.0,kylin-1.2 > java.lang.NullPointerException > -- > > Key: KYLIN-1302 > URL: https://issues.apache.org/jira/browse/KYLIN-1302 > Project: Kylin > Issue Type: Bug > Components: Job Engine >Affects Versions: v1.2 > Environment: CDH-5.4.8-1,hadoop-2.6.0,hive-1.1.0,hbase-1.0.0 >Reporter: gewenhui >Assignee: ZhouQianhao > Fix For: 1.2 > > > in the UI of 7070,job run here: > #2 Step Name: Extract Fact Table Distinct Columns > 2016-01-10 07:44:35.222 - State of Hadoop job: job_1449729554133_9107:RUNNING > - UNDEFINED > 2016-01-10 07:44:45.236 - State of Hadoop job: > job_1449729554133_9107:FINISHED - SUCCEEDED > org.apache.hadoop.ipc.RemoteException(java.lang.NullPointerException): > java.lang.NullPointerException > at > org.apache.hadoop.mapreduce.v2.hs.HistoryClientService$HSClientProtocolHandler.getCounters(HistoryClientService.java:228) > at > org.apache.hadoop.mapreduce.v2.api.impl.pb.service.MRClientProtocolPBServiceImpl.getCounters(MRClientProtocolPBServiceImpl.java:159) > at > org.apache.hadoop.yarn.proto.MRClientProtocol$MRClientProtocolService$2.callBlockingMethod(MRClientProtocol.java:281) > at > org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:619) > at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:1060) > at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2044) > at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2040) > at java.security.AccessController.doPrivileged(Native Method) > at javax.security.auth.Subject.doAs(Subject.java:415) > at > org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1671) > at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2038) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KYLIN-1302) java.lang.NullPointerException
[ https://issues.apache.org/jira/browse/KYLIN-1302?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15095560#comment-15095560 ] gewenhui commented on KYLIN-1302: - I use CDH-5.4.8-1,hadoop-2.6.0,hive-1.1.0,hbase-1.0.0,kylin-1.2 > java.lang.NullPointerException > -- > > Key: KYLIN-1302 > URL: https://issues.apache.org/jira/browse/KYLIN-1302 > Project: Kylin > Issue Type: Bug > Components: Job Engine >Affects Versions: v1.2 > Environment: CDH-5.4.8-1,hadoop-2.6.0,hive-1.1.0,hbase-1.0.0 >Reporter: gewenhui >Assignee: ZhouQianhao > Fix For: 1.2 > > > in the UI of 7070,job run here: > #2 Step Name: Extract Fact Table Distinct Columns > 2016-01-10 07:44:35.222 - State of Hadoop job: job_1449729554133_9107:RUNNING > - UNDEFINED > 2016-01-10 07:44:45.236 - State of Hadoop job: > job_1449729554133_9107:FINISHED - SUCCEEDED > org.apache.hadoop.ipc.RemoteException(java.lang.NullPointerException): > java.lang.NullPointerException > at > org.apache.hadoop.mapreduce.v2.hs.HistoryClientService$HSClientProtocolHandler.getCounters(HistoryClientService.java:228) > at > org.apache.hadoop.mapreduce.v2.api.impl.pb.service.MRClientProtocolPBServiceImpl.getCounters(MRClientProtocolPBServiceImpl.java:159) > at > org.apache.hadoop.yarn.proto.MRClientProtocol$MRClientProtocolService$2.callBlockingMethod(MRClientProtocol.java:281) > at > org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:619) > at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:1060) > at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2044) > at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2040) > at java.security.AccessController.doPrivileged(Native Method) > at javax.security.auth.Subject.doAs(Subject.java:415) > at > org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1671) > at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2038) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KYLIN-1122) Kylin support detail data query from fact table
[ https://issues.apache.org/jira/browse/KYLIN-1122?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15095517#comment-15095517 ] hongbin ma commented on KYLIN-1122: --- i think this is will be a critical function for kylin, gonna speed up merging it > Kylin support detail data query from fact table > --- > > Key: KYLIN-1122 > URL: https://issues.apache.org/jira/browse/KYLIN-1122 > Project: Kylin > Issue Type: New Feature > Components: Query Engine >Affects Versions: v1.2 >Reporter: Xiaoyu Wang >Assignee: liyang > Fix For: v2.0, v1.3 > > Attachments: > 0001-KYLIN-1122-Kylin-support-detail-data-query-from-fact(2.x-staging).patch, > 0001-KYLIN-1122-Kylin-support-detail-data-query-from-fact(update-v2-1.x-staging).patch, > > 0001-KYLIN-1122-Kylin-support-detail-data-query-from-fact-new-impl-under-refactoring-2.x-staging.patch > > > Now Kylin does not support query correct detail rows from fact table like: > select column1,column2,column3 from fact_table > The jira KYLIN-1075 add the "SUM" function on the measure column if defined. > But only the column number type is support. > I change some code to support this issue: > Add a "VALUE" measure function : the same value and datatype in the input and > output of this function. > If you want to query detail data from fact table > *require*: > 1.Configure the column which not dimensions to "VALUE" or "SUM" measure.(If > not configure measure function in the column will get NULL value) > 2.The source table must has an unique value column and configure it as > dimension. > If you have the better solution please comment here. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (KYLIN-1310) Auto Merge Time Ranges are filled with default options when edit a cube without auto merge
Dong Li created KYLIN-1310: -- Summary: Auto Merge Time Ranges are filled with default options when edit a cube without auto merge Key: KYLIN-1310 URL: https://issues.apache.org/jira/browse/KYLIN-1310 Project: Kylin Issue Type: Bug Components: Web Affects Versions: v2.0 Reporter: Dong Li Assignee: Zhong,Jason 1. Create a cube without auto merge 2. Edit the cube, switch to "Refresh Setting" tabpage 3. Found Auto merge time ranges are filled with default options. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KYLIN-1309) cordova-plugin-inappbrowser v1.1.1 android PDF rendering
[ https://issues.apache.org/jira/browse/KYLIN-1309?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15094322#comment-15094322 ] Wade Cantrell commented on KYLIN-1309: -- Build the app to ios and android using cordova and run it on ios it worked well showing me the pdf file, but on android it shows me only a white page. do you have an experience with that? > cordova-plugin-inappbrowser v1.1.1 android PDF rendering > > > Key: KYLIN-1309 > URL: https://issues.apache.org/jira/browse/KYLIN-1309 > Project: Kylin > Issue Type: Bug > Components: Web >Reporter: Wade Cantrell >Assignee: Zhong,Jason > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (KYLIN-1309) cordova-plugin-inappbrowser v1.1.1 android PDF rendering
Wade Cantrell created KYLIN-1309: Summary: cordova-plugin-inappbrowser v1.1.1 android PDF rendering Key: KYLIN-1309 URL: https://issues.apache.org/jira/browse/KYLIN-1309 Project: Kylin Issue Type: Bug Components: Web Reporter: Wade Cantrell Assignee: Zhong,Jason -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (KYLIN-1304) Incorrect count estimate upon HyperLogLogPlusCounter.merge operation.
[ https://issues.apache.org/jira/browse/KYLIN-1304?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] hongbin ma updated KYLIN-1304: -- Assignee: liyang (was: hongbin ma) > Incorrect count estimate upon HyperLogLogPlusCounter.merge operation. > - > > Key: KYLIN-1304 > URL: https://issues.apache.org/jira/browse/KYLIN-1304 > Project: Kylin > Issue Type: Bug >Affects Versions: v1.1.1 >Reporter: Prashant Prakash >Assignee: liyang > Attachments: registers.txt > > > We are using hllc12 for evaluating uniques for a measure in a table > partitioned over date. > The uniques estimate for individual dates 2016-01-07, 2016-01-08, 2016-01-09 > are 93,728,324, 90,982,364, 45,485,278 respectively. > But the uniques across days, which is calculated through > HyperLogLogPlusCounter.merge operation gives a incorrect value 67,980,576. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Assigned] (KYLIN-1304) Incorrect count estimate upon HyperLogLogPlusCounter.merge operation.
[ https://issues.apache.org/jira/browse/KYLIN-1304?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] hongbin ma reassigned KYLIN-1304: - Assignee: hongbin ma > Incorrect count estimate upon HyperLogLogPlusCounter.merge operation. > - > > Key: KYLIN-1304 > URL: https://issues.apache.org/jira/browse/KYLIN-1304 > Project: Kylin > Issue Type: Bug >Affects Versions: v1.1.1 >Reporter: Prashant Prakash >Assignee: hongbin ma > Attachments: registers.txt > > > We are using hllc12 for evaluating uniques for a measure in a table > partitioned over date. > The uniques estimate for individual dates 2016-01-07, 2016-01-08, 2016-01-09 > are 93,728,324, 90,982,364, 45,485,278 respectively. > But the uniques across days, which is calculated through > HyperLogLogPlusCounter.merge operation gives a incorrect value 67,980,576. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KYLIN-1054) Support Hive client Beeline
[ https://issues.apache.org/jira/browse/KYLIN-1054?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15094015#comment-15094015 ] huawang commented on KYLIN-1054: The only thing need to do is changing the hive -e in AbstractJobBuilder.java to beeline -u -e , hive client beeline will be supported. The GarbageCollectionStep.java needs also to be edited to beeline -u -e. > Support Hive client Beeline > --- > > Key: KYLIN-1054 > URL: https://issues.apache.org/jira/browse/KYLIN-1054 > Project: Kylin > Issue Type: Wish >Affects Versions: v1.0, v0.7.2, v0.7.1 >Reporter: Shaofeng SHI > > Some user ever mentioned that in their environment the "hive" shell client > isn't available, only Beeline is allowed; If Kylin can support using Beeline, > that would be a nice feature for him. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Reopened] (KYLIN-742) Route unsupported queries to Hive (on Spark)
[ https://issues.apache.org/jira/browse/KYLIN-742?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shaofeng SHI reopened KYLIN-742: Didn't see this in 2.x branch, reopen it. > Route unsupported queries to Hive (on Spark) > > > Key: KYLIN-742 > URL: https://issues.apache.org/jira/browse/KYLIN-742 > Project: Kylin > Issue Type: New Feature > Components: Query Engine, SparkSQL >Reporter: Luke Han >Assignee: liyang > Fix For: v1.0 > > Attachments: 1.x-HBase1.x-KYLIN-742-update.patch, > 1.x-HBase1.x-KYLIN-742-with-review-comments-#1-update-21_09_2015.patch, > Design Documentation.pdf > > > When Kylin can't serve coming SQL, it will be better to route it to enabled > SQL on Hadoop like SparkSQL and execute there. > Then get result and return to client through Kylin server. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (KYLIN-1308) query storage v2 enable parallel cube visiting
hongbin ma created KYLIN-1308: - Summary: query storage v2 enable parallel cube visiting Key: KYLIN-1308 URL: https://issues.apache.org/jira/browse/KYLIN-1308 Project: Kylin Issue Type: Improvement Reporter: hongbin ma Assignee: hongbin ma Currently if a cube has multiple segments, the endpoint coprocessor invocations for each segments are executed sequentially. We'll try to parallize it to see how it contributes to performance -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KYLIN-1122) Kylin support detail data query from fact table
[ https://issues.apache.org/jira/browse/KYLIN-1122?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15093740#comment-15093740 ] Xiaoyu Wang commented on KYLIN-1122: I created a new branch "KYLIN-1122" base on 2.x-staging. and submit my patch. > Kylin support detail data query from fact table > --- > > Key: KYLIN-1122 > URL: https://issues.apache.org/jira/browse/KYLIN-1122 > Project: Kylin > Issue Type: New Feature > Components: Query Engine >Affects Versions: v1.2 >Reporter: Xiaoyu Wang >Assignee: liyang > Fix For: v2.0, v1.3 > > Attachments: > 0001-KYLIN-1122-Kylin-support-detail-data-query-from-fact(2.x-staging).patch, > 0001-KYLIN-1122-Kylin-support-detail-data-query-from-fact(update-v2-1.x-staging).patch, > > 0001-KYLIN-1122-Kylin-support-detail-data-query-from-fact-new-impl-under-refactoring-2.x-staging.patch > > > Now Kylin does not support query correct detail rows from fact table like: > select column1,column2,column3 from fact_table > The jira KYLIN-1075 add the "SUM" function on the measure column if defined. > But only the column number type is support. > I change some code to support this issue: > Add a "VALUE" measure function : the same value and datatype in the input and > output of this function. > If you want to query detail data from fact table > *require*: > 1.Configure the column which not dimensions to "VALUE" or "SUM" measure.(If > not configure measure function in the column will get NULL value) > 2.The source table must has an unique value column and configure it as > dimension. > If you have the better solution please comment here. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (KYLIN-1220) JsonMappingException: Can not deserialize instance of java.lang.String out of START_ARRAY
[ https://issues.apache.org/jira/browse/KYLIN-1220?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhong,Jason resolved KYLIN-1220. Resolution: Fixed Fix Version/s: 2.0 > JsonMappingException: Can not deserialize instance of java.lang.String out of > START_ARRAY > - > > Key: KYLIN-1220 > URL: https://issues.apache.org/jira/browse/KYLIN-1220 > Project: Kylin > Issue Type: Bug > Components: Web >Affects Versions: v2.0 > Environment: HDP2.3.2 >Reporter: wangxianbin >Assignee: Zhong,Jason > Fix For: 2.0 > > > JsonMappingException when I create cube, I think it cause by class var > "column" in class "DimensionDesc" which is not String array any more, how > ever web didn't follow up this change. > L4J [2015-12-10 > 17:23:50,781][ERROR][org.apache.kylin.rest.controller.CubeController] - The > cube definition is not valid. > com.fasterxml.jackson.databind.JsonMappingException: Can not deserialize > instance of java.lang.String out of START_ARRAY token > at [Source: java.io.StringReader@407d4132; line: 10, column: 22] (through > reference chain: > org.apache.kylin.cube.model.CubeDesc["dimensions"]->org.apache.kylin.cube.model.DimensionDesc["column"]) > at > com.fasterxml.jackson.databind.JsonMappingException.from(JsonMappingException.java:164) > at > com.fasterxml.jackson.databind.DeserializationContext.mappingException(DeserializationContext.java:575) > at > com.fasterxml.jackson.databind.deser.std.StringDeserializer.deserialize(StringDeserializer.java:46) > at > com.fasterxml.jackson.databind.deser.std.StringDeserializer.deserialize(StringDeserializer.java:11) > at > com.fasterxml.jackson.databind.deser.SettableBeanProperty.deserialize(SettableBeanProperty.java:464) > at > com.fasterxml.jackson.databind.deser.impl.FieldProperty.deserializeAndSet(FieldProperty.java:107) > at > com.fasterxml.jackson.databind.deser.BeanDeserializer.deserializeFromObject(BeanDeserializer.java:295) > at > com.fasterxml.jackson.databind.deser.BeanDeserializer.deserialize(BeanDeserializer.java:121) > at > com.fasterxml.jackson.databind.deser.std.CollectionDeserializer.deserialize(CollectionDeserializer.java:230) > at > com.fasterxml.jackson.databind.deser.std.CollectionDeserializer.deserialize(CollectionDeserializer.java:207) > at > com.fasterxml.jackson.databind.deser.std.CollectionDeserializer.deserialize(CollectionDeserializer.java:23) > at > com.fasterxml.jackson.databind.deser.SettableBeanProperty.deserialize(SettableBeanProperty.java:464) > at > com.fasterxml.jackson.databind.deser.impl.FieldProperty.deserializeAndSet(FieldProperty.java:107) > at > com.fasterxml.jackson.databind.deser.BeanDeserializer.deserializeFromObject(BeanDeserializer.java:295) > at > com.fasterxml.jackson.databind.deser.BeanDeserializer.deserialize(BeanDeserializer.java:121) > at > com.fasterxml.jackson.databind.ObjectMapper._readMapAndClose(ObjectMapper.java:2888) > at > com.fasterxml.jackson.databind.ObjectMapper.readValue(ObjectMapper.java:2034) > at org.apache.kylin.common.util.JsonUtil.readValue(JsonUtil.java:54) > at > org.apache.kylin.rest.controller.CubeController.deserializeCubeDesc(CubeController.java:699) > at > org.apache.kylin.rest.controller.CubeController.saveCubeDesc(CubeController.java:327) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KYLIN-1220) JsonMappingException: Can not deserialize instance of java.lang.String out of START_ARRAY
[ https://issues.apache.org/jira/browse/KYLIN-1220?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15093716#comment-15093716 ] Zhong,Jason commented on KYLIN-1220: it's resolved now > JsonMappingException: Can not deserialize instance of java.lang.String out of > START_ARRAY > - > > Key: KYLIN-1220 > URL: https://issues.apache.org/jira/browse/KYLIN-1220 > Project: Kylin > Issue Type: Bug > Components: Web >Affects Versions: v2.0 > Environment: HDP2.3.2 >Reporter: wangxianbin >Assignee: Zhong,Jason > Fix For: 2.0 > > > JsonMappingException when I create cube, I think it cause by class var > "column" in class "DimensionDesc" which is not String array any more, how > ever web didn't follow up this change. > L4J [2015-12-10 > 17:23:50,781][ERROR][org.apache.kylin.rest.controller.CubeController] - The > cube definition is not valid. > com.fasterxml.jackson.databind.JsonMappingException: Can not deserialize > instance of java.lang.String out of START_ARRAY token > at [Source: java.io.StringReader@407d4132; line: 10, column: 22] (through > reference chain: > org.apache.kylin.cube.model.CubeDesc["dimensions"]->org.apache.kylin.cube.model.DimensionDesc["column"]) > at > com.fasterxml.jackson.databind.JsonMappingException.from(JsonMappingException.java:164) > at > com.fasterxml.jackson.databind.DeserializationContext.mappingException(DeserializationContext.java:575) > at > com.fasterxml.jackson.databind.deser.std.StringDeserializer.deserialize(StringDeserializer.java:46) > at > com.fasterxml.jackson.databind.deser.std.StringDeserializer.deserialize(StringDeserializer.java:11) > at > com.fasterxml.jackson.databind.deser.SettableBeanProperty.deserialize(SettableBeanProperty.java:464) > at > com.fasterxml.jackson.databind.deser.impl.FieldProperty.deserializeAndSet(FieldProperty.java:107) > at > com.fasterxml.jackson.databind.deser.BeanDeserializer.deserializeFromObject(BeanDeserializer.java:295) > at > com.fasterxml.jackson.databind.deser.BeanDeserializer.deserialize(BeanDeserializer.java:121) > at > com.fasterxml.jackson.databind.deser.std.CollectionDeserializer.deserialize(CollectionDeserializer.java:230) > at > com.fasterxml.jackson.databind.deser.std.CollectionDeserializer.deserialize(CollectionDeserializer.java:207) > at > com.fasterxml.jackson.databind.deser.std.CollectionDeserializer.deserialize(CollectionDeserializer.java:23) > at > com.fasterxml.jackson.databind.deser.SettableBeanProperty.deserialize(SettableBeanProperty.java:464) > at > com.fasterxml.jackson.databind.deser.impl.FieldProperty.deserializeAndSet(FieldProperty.java:107) > at > com.fasterxml.jackson.databind.deser.BeanDeserializer.deserializeFromObject(BeanDeserializer.java:295) > at > com.fasterxml.jackson.databind.deser.BeanDeserializer.deserialize(BeanDeserializer.java:121) > at > com.fasterxml.jackson.databind.ObjectMapper._readMapAndClose(ObjectMapper.java:2888) > at > com.fasterxml.jackson.databind.ObjectMapper.readValue(ObjectMapper.java:2034) > at org.apache.kylin.common.util.JsonUtil.readValue(JsonUtil.java:54) > at > org.apache.kylin.rest.controller.CubeController.deserializeCubeDesc(CubeController.java:699) > at > org.apache.kylin.rest.controller.CubeController.saveCubeDesc(CubeController.java:327) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (KYLIN-1251) add toggle event for tree label
[ https://issues.apache.org/jira/browse/KYLIN-1251?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhong,Jason resolved KYLIN-1251. Resolution: Fixed Fix Version/s: v1.3 v2.0 > add toggle event for tree label > --- > > Key: KYLIN-1251 > URL: https://issues.apache.org/jira/browse/KYLIN-1251 > Project: Kylin > Issue Type: Improvement > Components: Web >Affects Versions: v2.0, v1.2 >Reporter: Zhong,Jason >Assignee: Zhong,Jason > Fix For: v2.0, v1.3 > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KYLIN-1294) Filter of NOT IN (...) can cause incorrect result or NPE
[
https://issues.apache.org/jira/browse/KYLIN-1294?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15093637#comment-15093637
]
huawang commented on KYLIN-1294:
Do you mean that only if the bug of CALCITE-980 fixed could we continue to
study my problem?
I don't know whether the problem of result data is incorrect related to the bug
of CALCITE-980?
> Filter of NOT IN (...) can cause incorrect result or NPE
>
>
> Key: KYLIN-1294
> URL: https://issues.apache.org/jira/browse/KYLIN-1294
> Project: Kylin
> Issue Type: Bug
>Reporter: liyang
>Assignee: hongbin ma
>
> The version I use is 1.0.
> Recently I updated to 1.2. The problent is like following:
> Case1:***
> When I use “not in” in my sql, the resultset is less than my
> correct data. Like following:
> When the sql I use not including a “not in” is :
> SELECT "KYLIN_TV_DIM_CITY_V_KYLIN"."H_CODE" AS "H_CODE",
>COUNT(1) AS "sum_Number_of_Records_ok",
>SUM("KYLIN_VIEW_TVAD_SUMMARY"."OCCU") AS "sum_OCCU_ok"
> FROM "BD_WAREHOUSE"."KYLIN_VIEW_TVAD_SUMMARY" "KYLIN_VIEW_TVAD_SUMMARY"
> LEFT JOIN "BD_WAREHOUSE"."KYLIN_TV_DIM_CITY_V_KYLIN"
> "KYLIN_TV_DIM_CITY_V_KYLIN"
> ON ("KYLIN_VIEW_TVAD_SUMMARY"."CITY" =
>"KYLIN_TV_DIM_CITY_V_KYLIN"."CITY")
> LEFT JOIN "BD_WAREHOUSE"."KYLIN_TV_DIM_DATE_CUBE_V_KYLIN"
> "KYLIN_TV_DIM_DATE_CUBE_V_KYLIN"
> ON ("KYLIN_VIEW_TVAD_SUMMARY"."DT" =
>"KYLIN_TV_DIM_DATE_CUBE_V_KYLIN"."DT1")
> GROUP BY "KYLIN_TV_DIM_CITY_V_KYLIN"."H_CODE"
> The result is like following:
> (null) 206735 436180
> other169103517 480104320
> 无 25970813161884
> A 42464098 148884448
> B 31945514 121964156
> O 27878957265948
> K 37694048 127243245
> Whiel when I use a sql having “not in”, like following:
> SELECT "KYLIN_TV_DIM_CITY_V_KYLIN"."H_CODE" AS "H_CODE",
>COUNT(1) AS "sum_Number_of_Records_ok",
>SUM("KYLIN_VIEW_TVAD_SUMMARY"."OCCU") AS "sum_OCCU_ok"
> FROM "BD_WAREHOUSE"."KYLIN_VIEW_TVAD_SUMMARY" "KYLIN_VIEW_TVAD_SUMMARY"
> LEFT JOIN "BD_WAREHOUSE"."KYLIN_TV_DIM_CITY_V_KYLIN"
> "KYLIN_TV_DIM_CITY_V_KYLIN"
> ON ("KYLIN_VIEW_TVAD_SUMMARY"."CITY" =
>"KYLIN_TV_DIM_CITY_V_KYLIN"."CITY")
> LEFT JOIN "BD_WAREHOUSE"."KYLIN_TV_DIM_DATE_CUBE_V_KYLIN"
> "KYLIN_TV_DIM_DATE_CUBE_V_KYLIN"
> ON ("KYLIN_VIEW_TVAD_SUMMARY"."DT" =
>"KYLIN_TV_DIM_DATE_CUBE_V_KYLIN"."DT1")
> WHERE ("KYLIN_TV_DIM_CITY_V_KYLIN"."H_CODE" NOT IN ('other'))
> GROUP BY "KYLIN_TV_DIM_CITY_V_KYLIN"."H_CODE"
> The result I got is, there are only 3 lines, and the results are all less
> than the true value:
> 无 323911 646111
> A 7463670 21674399
> K 15248274 54972183
> Case2:
> Sql1: select channel_id, sum(occu) from kylin_view_tvad_summary
> where channel_id not in (2, 3) group by channel_id.
> Here, 2 is a member of channel_id in kylin_view_tvad_summary, while
> 3 is not a member of kylin_view_tvad_summary, there will be a null pointer
> exception like following:
> java.sql.SQLException: error while executing SQL "select channel_id,
> sum(occu) from kylin_view_tvad_summary where channel_id not in (2, 3) group
> by channel_id
> LIMIT 5": null
> at org.apache.calcite.avatica.Helper.createException(Helper.java:41)
> at
> org.apache.calcite.avatica.AvaticaStatement.executeInternal(AvaticaStatement.java:112)
> at
> org.apache.calcite.avatica.AvaticaStatement.executeQuery(AvaticaStatement.java:130)
> at
> org.apache.kylin.rest.service.QueryService.execute(QueryService.java:354)
> at
> org.apache.kylin.rest.service.QueryService.queryWithSqlMassage(QueryService.java:268)
> at
> org.apache.kylin.rest.service.QueryService.query(QueryService.java:114)
> at
> org.apache.kylin.rest.service.QueryService$$FastClassByCGLIB$$4957273f.invoke()
> at net.sf.cglib.proxy.MethodProxy.invoke(MethodProxy.java:204)
> at
> org.springframework.aop.framework.Cglib2AopProxy$DynamicAdvisedInterceptor.intercept(Cglib2AopProxy.java:618)
> at
> org.apache.kylin.rest.service.QueryService$$EnhancerByCGLIB$$3a29d57a.query()
> at
> org.apache.kylin.rest.controller.QueryController.doQueryWithCache(QueryController.java:178)
> at
> org.apache.kylin.rest.controller.QueryController.query(QueryController.java:85)
> at sun.reflect.GeneratedMethodAccessor153.invoke(Unknown Source)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at j
[jira] [Commented] (KYLIN-1294) Filter of NOT IN (...) can cause incorrect result or NPE
[
https://issues.apache.org/jira/browse/KYLIN-1294?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15093623#comment-15093623
]
hongbin ma commented on KYLIN-1294:
---
the issue blocks at CALCITE-980 before we could even reproduce your problem
> Filter of NOT IN (...) can cause incorrect result or NPE
>
>
> Key: KYLIN-1294
> URL: https://issues.apache.org/jira/browse/KYLIN-1294
> Project: Kylin
> Issue Type: Bug
>Reporter: liyang
>Assignee: hongbin ma
>
> The version I use is 1.0.
> Recently I updated to 1.2. The problent is like following:
> Case1:***
> When I use “not in” in my sql, the resultset is less than my
> correct data. Like following:
> When the sql I use not including a “not in” is :
> SELECT "KYLIN_TV_DIM_CITY_V_KYLIN"."H_CODE" AS "H_CODE",
>COUNT(1) AS "sum_Number_of_Records_ok",
>SUM("KYLIN_VIEW_TVAD_SUMMARY"."OCCU") AS "sum_OCCU_ok"
> FROM "BD_WAREHOUSE"."KYLIN_VIEW_TVAD_SUMMARY" "KYLIN_VIEW_TVAD_SUMMARY"
> LEFT JOIN "BD_WAREHOUSE"."KYLIN_TV_DIM_CITY_V_KYLIN"
> "KYLIN_TV_DIM_CITY_V_KYLIN"
> ON ("KYLIN_VIEW_TVAD_SUMMARY"."CITY" =
>"KYLIN_TV_DIM_CITY_V_KYLIN"."CITY")
> LEFT JOIN "BD_WAREHOUSE"."KYLIN_TV_DIM_DATE_CUBE_V_KYLIN"
> "KYLIN_TV_DIM_DATE_CUBE_V_KYLIN"
> ON ("KYLIN_VIEW_TVAD_SUMMARY"."DT" =
>"KYLIN_TV_DIM_DATE_CUBE_V_KYLIN"."DT1")
> GROUP BY "KYLIN_TV_DIM_CITY_V_KYLIN"."H_CODE"
> The result is like following:
> (null) 206735 436180
> other169103517 480104320
> 无 25970813161884
> A 42464098 148884448
> B 31945514 121964156
> O 27878957265948
> K 37694048 127243245
> Whiel when I use a sql having “not in”, like following:
> SELECT "KYLIN_TV_DIM_CITY_V_KYLIN"."H_CODE" AS "H_CODE",
>COUNT(1) AS "sum_Number_of_Records_ok",
>SUM("KYLIN_VIEW_TVAD_SUMMARY"."OCCU") AS "sum_OCCU_ok"
> FROM "BD_WAREHOUSE"."KYLIN_VIEW_TVAD_SUMMARY" "KYLIN_VIEW_TVAD_SUMMARY"
> LEFT JOIN "BD_WAREHOUSE"."KYLIN_TV_DIM_CITY_V_KYLIN"
> "KYLIN_TV_DIM_CITY_V_KYLIN"
> ON ("KYLIN_VIEW_TVAD_SUMMARY"."CITY" =
>"KYLIN_TV_DIM_CITY_V_KYLIN"."CITY")
> LEFT JOIN "BD_WAREHOUSE"."KYLIN_TV_DIM_DATE_CUBE_V_KYLIN"
> "KYLIN_TV_DIM_DATE_CUBE_V_KYLIN"
> ON ("KYLIN_VIEW_TVAD_SUMMARY"."DT" =
>"KYLIN_TV_DIM_DATE_CUBE_V_KYLIN"."DT1")
> WHERE ("KYLIN_TV_DIM_CITY_V_KYLIN"."H_CODE" NOT IN ('other'))
> GROUP BY "KYLIN_TV_DIM_CITY_V_KYLIN"."H_CODE"
> The result I got is, there are only 3 lines, and the results are all less
> than the true value:
> 无 323911 646111
> A 7463670 21674399
> K 15248274 54972183
> Case2:
> Sql1: select channel_id, sum(occu) from kylin_view_tvad_summary
> where channel_id not in (2, 3) group by channel_id.
> Here, 2 is a member of channel_id in kylin_view_tvad_summary, while
> 3 is not a member of kylin_view_tvad_summary, there will be a null pointer
> exception like following:
> java.sql.SQLException: error while executing SQL "select channel_id,
> sum(occu) from kylin_view_tvad_summary where channel_id not in (2, 3) group
> by channel_id
> LIMIT 5": null
> at org.apache.calcite.avatica.Helper.createException(Helper.java:41)
> at
> org.apache.calcite.avatica.AvaticaStatement.executeInternal(AvaticaStatement.java:112)
> at
> org.apache.calcite.avatica.AvaticaStatement.executeQuery(AvaticaStatement.java:130)
> at
> org.apache.kylin.rest.service.QueryService.execute(QueryService.java:354)
> at
> org.apache.kylin.rest.service.QueryService.queryWithSqlMassage(QueryService.java:268)
> at
> org.apache.kylin.rest.service.QueryService.query(QueryService.java:114)
> at
> org.apache.kylin.rest.service.QueryService$$FastClassByCGLIB$$4957273f.invoke()
> at net.sf.cglib.proxy.MethodProxy.invoke(MethodProxy.java:204)
> at
> org.springframework.aop.framework.Cglib2AopProxy$DynamicAdvisedInterceptor.intercept(Cglib2AopProxy.java:618)
> at
> org.apache.kylin.rest.service.QueryService$$EnhancerByCGLIB$$3a29d57a.query()
> at
> org.apache.kylin.rest.controller.QueryController.doQueryWithCache(QueryController.java:178)
> at
> org.apache.kylin.rest.controller.QueryController.query(QueryController.java:85)
> at sun.reflect.GeneratedMethodAccessor153.invoke(Unknown Source)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:606)
> at
> org.springframework.web.method.support.Invoca
[jira] [Commented] (KYLIN-1294) Filter of NOT IN (...) can cause incorrect result or NPE
[
https://issues.apache.org/jira/browse/KYLIN-1294?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15093619#comment-15093619
]
huawang commented on KYLIN-1294:
The question is even the sql doesn't throw a NPE, the result set of the sql is
not correct as shown in the but description.
> Filter of NOT IN (...) can cause incorrect result or NPE
>
>
> Key: KYLIN-1294
> URL: https://issues.apache.org/jira/browse/KYLIN-1294
> Project: Kylin
> Issue Type: Bug
>Reporter: liyang
>Assignee: hongbin ma
>
> The version I use is 1.0.
> Recently I updated to 1.2. The problent is like following:
> Case1:***
> When I use “not in” in my sql, the resultset is less than my
> correct data. Like following:
> When the sql I use not including a “not in” is :
> SELECT "KYLIN_TV_DIM_CITY_V_KYLIN"."H_CODE" AS "H_CODE",
>COUNT(1) AS "sum_Number_of_Records_ok",
>SUM("KYLIN_VIEW_TVAD_SUMMARY"."OCCU") AS "sum_OCCU_ok"
> FROM "BD_WAREHOUSE"."KYLIN_VIEW_TVAD_SUMMARY" "KYLIN_VIEW_TVAD_SUMMARY"
> LEFT JOIN "BD_WAREHOUSE"."KYLIN_TV_DIM_CITY_V_KYLIN"
> "KYLIN_TV_DIM_CITY_V_KYLIN"
> ON ("KYLIN_VIEW_TVAD_SUMMARY"."CITY" =
>"KYLIN_TV_DIM_CITY_V_KYLIN"."CITY")
> LEFT JOIN "BD_WAREHOUSE"."KYLIN_TV_DIM_DATE_CUBE_V_KYLIN"
> "KYLIN_TV_DIM_DATE_CUBE_V_KYLIN"
> ON ("KYLIN_VIEW_TVAD_SUMMARY"."DT" =
>"KYLIN_TV_DIM_DATE_CUBE_V_KYLIN"."DT1")
> GROUP BY "KYLIN_TV_DIM_CITY_V_KYLIN"."H_CODE"
> The result is like following:
> (null) 206735 436180
> other169103517 480104320
> 无 25970813161884
> A 42464098 148884448
> B 31945514 121964156
> O 27878957265948
> K 37694048 127243245
> Whiel when I use a sql having “not in”, like following:
> SELECT "KYLIN_TV_DIM_CITY_V_KYLIN"."H_CODE" AS "H_CODE",
>COUNT(1) AS "sum_Number_of_Records_ok",
>SUM("KYLIN_VIEW_TVAD_SUMMARY"."OCCU") AS "sum_OCCU_ok"
> FROM "BD_WAREHOUSE"."KYLIN_VIEW_TVAD_SUMMARY" "KYLIN_VIEW_TVAD_SUMMARY"
> LEFT JOIN "BD_WAREHOUSE"."KYLIN_TV_DIM_CITY_V_KYLIN"
> "KYLIN_TV_DIM_CITY_V_KYLIN"
> ON ("KYLIN_VIEW_TVAD_SUMMARY"."CITY" =
>"KYLIN_TV_DIM_CITY_V_KYLIN"."CITY")
> LEFT JOIN "BD_WAREHOUSE"."KYLIN_TV_DIM_DATE_CUBE_V_KYLIN"
> "KYLIN_TV_DIM_DATE_CUBE_V_KYLIN"
> ON ("KYLIN_VIEW_TVAD_SUMMARY"."DT" =
>"KYLIN_TV_DIM_DATE_CUBE_V_KYLIN"."DT1")
> WHERE ("KYLIN_TV_DIM_CITY_V_KYLIN"."H_CODE" NOT IN ('other'))
> GROUP BY "KYLIN_TV_DIM_CITY_V_KYLIN"."H_CODE"
> The result I got is, there are only 3 lines, and the results are all less
> than the true value:
> 无 323911 646111
> A 7463670 21674399
> K 15248274 54972183
> Case2:
> Sql1: select channel_id, sum(occu) from kylin_view_tvad_summary
> where channel_id not in (2, 3) group by channel_id.
> Here, 2 is a member of channel_id in kylin_view_tvad_summary, while
> 3 is not a member of kylin_view_tvad_summary, there will be a null pointer
> exception like following:
> java.sql.SQLException: error while executing SQL "select channel_id,
> sum(occu) from kylin_view_tvad_summary where channel_id not in (2, 3) group
> by channel_id
> LIMIT 5": null
> at org.apache.calcite.avatica.Helper.createException(Helper.java:41)
> at
> org.apache.calcite.avatica.AvaticaStatement.executeInternal(AvaticaStatement.java:112)
> at
> org.apache.calcite.avatica.AvaticaStatement.executeQuery(AvaticaStatement.java:130)
> at
> org.apache.kylin.rest.service.QueryService.execute(QueryService.java:354)
> at
> org.apache.kylin.rest.service.QueryService.queryWithSqlMassage(QueryService.java:268)
> at
> org.apache.kylin.rest.service.QueryService.query(QueryService.java:114)
> at
> org.apache.kylin.rest.service.QueryService$$FastClassByCGLIB$$4957273f.invoke()
> at net.sf.cglib.proxy.MethodProxy.invoke(MethodProxy.java:204)
> at
> org.springframework.aop.framework.Cglib2AopProxy$DynamicAdvisedInterceptor.intercept(Cglib2AopProxy.java:618)
> at
> org.apache.kylin.rest.service.QueryService$$EnhancerByCGLIB$$3a29d57a.query()
> at
> org.apache.kylin.rest.controller.QueryController.doQueryWithCache(QueryController.java:178)
> at
> org.apache.kylin.rest.controller.QueryController.query(QueryController.java:85)
> at sun.reflect.GeneratedMethodAccessor153.invoke(Unknown Source)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:606)
> at
> or
[jira] [Commented] (KYLIN-1294) Filter of NOT IN (...) can cause incorrect result or NPE
[
https://issues.apache.org/jira/browse/KYLIN-1294?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15093604#comment-15093604
]
hongbin ma commented on KYLIN-1294:
---
the issue is pending on https://issues.apache.org/jira/browse/CALCITE-980,
which is fixed in calcite 1.6. We'll upgrade to calcite 1.6 as soon as they
release it
> Filter of NOT IN (...) can cause incorrect result or NPE
>
>
> Key: KYLIN-1294
> URL: https://issues.apache.org/jira/browse/KYLIN-1294
> Project: Kylin
> Issue Type: Bug
>Reporter: liyang
>Assignee: hongbin ma
>
> The version I use is 1.0.
> Recently I updated to 1.2. The problent is like following:
> Case1:***
> When I use “not in” in my sql, the resultset is less than my
> correct data. Like following:
> When the sql I use not including a “not in” is :
> SELECT "KYLIN_TV_DIM_CITY_V_KYLIN"."H_CODE" AS "H_CODE",
>COUNT(1) AS "sum_Number_of_Records_ok",
>SUM("KYLIN_VIEW_TVAD_SUMMARY"."OCCU") AS "sum_OCCU_ok"
> FROM "BD_WAREHOUSE"."KYLIN_VIEW_TVAD_SUMMARY" "KYLIN_VIEW_TVAD_SUMMARY"
> LEFT JOIN "BD_WAREHOUSE"."KYLIN_TV_DIM_CITY_V_KYLIN"
> "KYLIN_TV_DIM_CITY_V_KYLIN"
> ON ("KYLIN_VIEW_TVAD_SUMMARY"."CITY" =
>"KYLIN_TV_DIM_CITY_V_KYLIN"."CITY")
> LEFT JOIN "BD_WAREHOUSE"."KYLIN_TV_DIM_DATE_CUBE_V_KYLIN"
> "KYLIN_TV_DIM_DATE_CUBE_V_KYLIN"
> ON ("KYLIN_VIEW_TVAD_SUMMARY"."DT" =
>"KYLIN_TV_DIM_DATE_CUBE_V_KYLIN"."DT1")
> GROUP BY "KYLIN_TV_DIM_CITY_V_KYLIN"."H_CODE"
> The result is like following:
> (null) 206735 436180
> other169103517 480104320
> 无 25970813161884
> A 42464098 148884448
> B 31945514 121964156
> O 27878957265948
> K 37694048 127243245
> Whiel when I use a sql having “not in”, like following:
> SELECT "KYLIN_TV_DIM_CITY_V_KYLIN"."H_CODE" AS "H_CODE",
>COUNT(1) AS "sum_Number_of_Records_ok",
>SUM("KYLIN_VIEW_TVAD_SUMMARY"."OCCU") AS "sum_OCCU_ok"
> FROM "BD_WAREHOUSE"."KYLIN_VIEW_TVAD_SUMMARY" "KYLIN_VIEW_TVAD_SUMMARY"
> LEFT JOIN "BD_WAREHOUSE"."KYLIN_TV_DIM_CITY_V_KYLIN"
> "KYLIN_TV_DIM_CITY_V_KYLIN"
> ON ("KYLIN_VIEW_TVAD_SUMMARY"."CITY" =
>"KYLIN_TV_DIM_CITY_V_KYLIN"."CITY")
> LEFT JOIN "BD_WAREHOUSE"."KYLIN_TV_DIM_DATE_CUBE_V_KYLIN"
> "KYLIN_TV_DIM_DATE_CUBE_V_KYLIN"
> ON ("KYLIN_VIEW_TVAD_SUMMARY"."DT" =
>"KYLIN_TV_DIM_DATE_CUBE_V_KYLIN"."DT1")
> WHERE ("KYLIN_TV_DIM_CITY_V_KYLIN"."H_CODE" NOT IN ('other'))
> GROUP BY "KYLIN_TV_DIM_CITY_V_KYLIN"."H_CODE"
> The result I got is, there are only 3 lines, and the results are all less
> than the true value:
> 无 323911 646111
> A 7463670 21674399
> K 15248274 54972183
> Case2:
> Sql1: select channel_id, sum(occu) from kylin_view_tvad_summary
> where channel_id not in (2, 3) group by channel_id.
> Here, 2 is a member of channel_id in kylin_view_tvad_summary, while
> 3 is not a member of kylin_view_tvad_summary, there will be a null pointer
> exception like following:
> java.sql.SQLException: error while executing SQL "select channel_id,
> sum(occu) from kylin_view_tvad_summary where channel_id not in (2, 3) group
> by channel_id
> LIMIT 5": null
> at org.apache.calcite.avatica.Helper.createException(Helper.java:41)
> at
> org.apache.calcite.avatica.AvaticaStatement.executeInternal(AvaticaStatement.java:112)
> at
> org.apache.calcite.avatica.AvaticaStatement.executeQuery(AvaticaStatement.java:130)
> at
> org.apache.kylin.rest.service.QueryService.execute(QueryService.java:354)
> at
> org.apache.kylin.rest.service.QueryService.queryWithSqlMassage(QueryService.java:268)
> at
> org.apache.kylin.rest.service.QueryService.query(QueryService.java:114)
> at
> org.apache.kylin.rest.service.QueryService$$FastClassByCGLIB$$4957273f.invoke()
> at net.sf.cglib.proxy.MethodProxy.invoke(MethodProxy.java:204)
> at
> org.springframework.aop.framework.Cglib2AopProxy$DynamicAdvisedInterceptor.intercept(Cglib2AopProxy.java:618)
> at
> org.apache.kylin.rest.service.QueryService$$EnhancerByCGLIB$$3a29d57a.query()
> at
> org.apache.kylin.rest.controller.QueryController.doQueryWithCache(QueryController.java:178)
> at
> org.apache.kylin.rest.controller.QueryController.query(QueryController.java:85)
> at sun.reflect.GeneratedMethodAccessor153.invoke(Unknown Source)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Metho
[jira] [Commented] (KYLIN-1302) java.lang.NullPointerException
[ https://issues.apache.org/jira/browse/KYLIN-1302?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15093597#comment-15093597 ] Shaofeng SHI commented on KYLIN-1302: - Hi gewen, please provide more info about the issue (hadoop environment, kylin version etc), and how to reproduce; only a stacktrace sometimes is poor for a developer to understand the issue. If you can solve the issue and contribute a patch, that will be welcomed. > java.lang.NullPointerException > -- > > Key: KYLIN-1302 > URL: https://issues.apache.org/jira/browse/KYLIN-1302 > Project: Kylin > Issue Type: Bug > Components: Job Engine >Affects Versions: v1.2 > Environment: CDH-5.4.8-1,hadoop-2.6.0,hive-1.1.0,hbase-1.0.0 >Reporter: gewenhui >Assignee: ZhouQianhao > Fix For: 1.2 > > > in the UI of 7070,job run here: > #2 Step Name: Extract Fact Table Distinct Columns > 2016-01-10 07:44:35.222 - State of Hadoop job: job_1449729554133_9107:RUNNING > - UNDEFINED > 2016-01-10 07:44:45.236 - State of Hadoop job: > job_1449729554133_9107:FINISHED - SUCCEEDED > org.apache.hadoop.ipc.RemoteException(java.lang.NullPointerException): > java.lang.NullPointerException > at > org.apache.hadoop.mapreduce.v2.hs.HistoryClientService$HSClientProtocolHandler.getCounters(HistoryClientService.java:228) > at > org.apache.hadoop.mapreduce.v2.api.impl.pb.service.MRClientProtocolPBServiceImpl.getCounters(MRClientProtocolPBServiceImpl.java:159) > at > org.apache.hadoop.yarn.proto.MRClientProtocol$MRClientProtocolService$2.callBlockingMethod(MRClientProtocol.java:281) > at > org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:619) > at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:1060) > at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2044) > at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2040) > at java.security.AccessController.doPrivileged(Native Method) > at javax.security.auth.Subject.doAs(Subject.java:415) > at > org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1671) > at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2038) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KYLIN-1082) Hive dependencies should be add to tmpjars
[ https://issues.apache.org/jira/browse/KYLIN-1082?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15093523#comment-15093523 ] wangxianbin commented on KYLIN-1082: OK, I think all we need to do is just merge new patch uploaded by fengyu, maybe a little modification. > Hive dependencies should be add to tmpjars > -- > > Key: KYLIN-1082 > URL: https://issues.apache.org/jira/browse/KYLIN-1082 > Project: Kylin > Issue Type: Bug >Reporter: liyang >Assignee: Zhong Yanghong > Labels: newbie > > Currently kylin assume all data nodes have hive deployment at exact same FS > location. However, a better position is to think hive as a client side app. > Then we need to ship hive jar with MR job every time. > This make deploy kylin a lot easier in cluster that does not have hive on all > data nodes. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KYLIN-1082) Hive dependencies should be add to tmpjars
[ https://issues.apache.org/jira/browse/KYLIN-1082?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15093507#comment-15093507 ] liyang commented on KYLIN-1082: --- Yes it's agreed that hive jar better get auto submitted. And that's why we have this JIRA. I've invited Zhong, Yanghong to work on this item. > Hive dependencies should be add to tmpjars > -- > > Key: KYLIN-1082 > URL: https://issues.apache.org/jira/browse/KYLIN-1082 > Project: Kylin > Issue Type: Bug >Reporter: liyang >Assignee: Zhong Yanghong > Labels: newbie > > Currently kylin assume all data nodes have hive deployment at exact same FS > location. However, a better position is to think hive as a client side app. > Then we need to ship hive jar with MR job every time. > This make deploy kylin a lot easier in cluster that does not have hive on all > data nodes. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (KYLIN-1082) Hive dependencies should be add to tmpjars
[ https://issues.apache.org/jira/browse/KYLIN-1082?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] liyang updated KYLIN-1082: -- Assignee: Zhong Yanghong > Hive dependencies should be add to tmpjars > -- > > Key: KYLIN-1082 > URL: https://issues.apache.org/jira/browse/KYLIN-1082 > Project: Kylin > Issue Type: Bug >Reporter: liyang >Assignee: Zhong Yanghong > Labels: newbie > > Currently kylin assume all data nodes have hive deployment at exact same FS > location. However, a better position is to think hive as a client side app. > Then we need to ship hive jar with MR job every time. > This make deploy kylin a lot easier in cluster that does not have hive on all > data nodes. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (KYLIN-1307) revisit growing dictionary
hongbin ma created KYLIN-1307: - Summary: revisit growing dictionary Key: KYLIN-1307 URL: https://issues.apache.org/jira/browse/KYLIN-1307 Project: Kylin Issue Type: Improvement Reporter: hongbin ma Assignee: hongbin ma the optimization in KYLIN-871 will affect segment pruning based on dictionary values, which in turn hurts query performance. Will temporally roll back it and await more thorough thinking -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (KYLIN-871) growing dictionary for streaming case
[ https://issues.apache.org/jira/browse/KYLIN-871?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] hongbin ma updated KYLIN-871: - Description: streaming cubes will generate a lot of dictionaries, and they may differ from each other only a little. growing dict swallows new entries and generate a bigger dict each time a new segment is being built. in this way the entries of dicts get controlled. > growing dictionary for streaming case > - > > Key: KYLIN-871 > URL: https://issues.apache.org/jira/browse/KYLIN-871 > Project: Kylin > Issue Type: Improvement >Reporter: hongbin ma >Assignee: hongbin ma > Fix For: v2.0 > > > streaming cubes will generate a lot of dictionaries, and they may differ from > each other only a little. growing dict swallows new entries and generate a > bigger dict each time a new segment is being built. in this way the entries > of dicts get controlled. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
