[jira] [Commented] (KYLIN-3660) subquery, join wiht two cube return No Result
[ https://issues.apache.org/jira/browse/KYLIN-3660?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16671176#comment-16671176 ] XiaoXiang Yu commented on KYLIN-3660: - Does t1 and t2 are both fact tables? The SQL queries need be matched with the pre-defined OLAP model. It should be one fact table in a sql query. > subquery, join wiht two cube return No Result > - > > Key: KYLIN-3660 > URL: https://issues.apache.org/jira/browse/KYLIN-3660 > Project: Kylin > Issue Type: Bug > Components: Query Engine >Affects Versions: v2.5.0 >Reporter: jiahua.hu >Priority: Major > > I have two cubes corresponding to two tables t1 and t2. When I pass the > user_Id in t2 to limit the query range of the t1 table, I return an empty > result. Then there is data in my t2 table. > 我有两个cube分别对应两张表t1和t2。当我通过t2中的user_Id去限制t1表的查询范围时,返回空结果。然后我的t2表里是有数据的。 > select * from t1 inner join (select user_id from t2 where EXID='50') t2 on > t1.user_id = t2.user_id; > select * from t1 where user_id in (select user_id from t2 where EXID='50'); > select * from t1 where user_id in (select '1002001'); > The above three statements execution results are returned No Result. > > But the following statement execution can return results > select * from t1 where user_id in ('1002001'); > > === > Query Id: 4a12730c-5222-b4a9-43fc-282e6bea7b51 > SQL: select * from t1 inner join (select user_id from t2 where EXID='50') t2 > on t1.user_id = t2.user_id; > User: ADMIN > Success: true > Duration: 0.001 > Project: test_project > Realization Names: [CUBE[name=t1_CUBE], CUBE[name=t2_CUBE]] > Cuboid Ids: [3, 63] > Total scan count: 50123 > Total scan bytes: 5412669 > Result row count: 0 > Accept Partial: true > Is Partial Result: false > Hit Exception Cache: false > Storage cache used: true > Is Query Push-Down: false > Is Prepare: false > Trace URL: null > Message: null > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (KYLIN-3660) subquery, join wiht two cube return No Result
[ https://issues.apache.org/jira/browse/KYLIN-3660?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16671182#comment-16671182 ] Shaofeng SHI commented on KYLIN-3660: - Righ, the join need be matched with the model definition. Kylin doesn't support flexible join. Besides, Kylin doesn't support non-aggregation query like "select *". Please check Kylin's FAQ > subquery, join wiht two cube return No Result > - > > Key: KYLIN-3660 > URL: https://issues.apache.org/jira/browse/KYLIN-3660 > Project: Kylin > Issue Type: Bug > Components: Query Engine >Affects Versions: v2.5.0 >Reporter: jiahua.hu >Priority: Major > > I have two cubes corresponding to two tables t1 and t2. When I pass the > user_Id in t2 to limit the query range of the t1 table, I return an empty > result. Then there is data in my t2 table. > 我有两个cube分别对应两张表t1和t2。当我通过t2中的user_Id去限制t1表的查询范围时,返回空结果。然后我的t2表里是有数据的。 > select * from t1 inner join (select user_id from t2 where EXID='50') t2 on > t1.user_id = t2.user_id; > select * from t1 where user_id in (select user_id from t2 where EXID='50'); > select * from t1 where user_id in (select '1002001'); > The above three statements execution results are returned No Result. > > But the following statement execution can return results > select * from t1 where user_id in ('1002001'); > > === > Query Id: 4a12730c-5222-b4a9-43fc-282e6bea7b51 > SQL: select * from t1 inner join (select user_id from t2 where EXID='50') t2 > on t1.user_id = t2.user_id; > User: ADMIN > Success: true > Duration: 0.001 > Project: test_project > Realization Names: [CUBE[name=t1_CUBE], CUBE[name=t2_CUBE]] > Cuboid Ids: [3, 63] > Total scan count: 50123 > Total scan bytes: 5412669 > Result row count: 0 > Accept Partial: true > Is Partial Result: false > Hit Exception Cache: false > Storage cache used: true > Is Query Push-Down: false > Is Prepare: false > Trace URL: null > Message: null > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (KYLIN-3660) subquery, join wiht two cube return No Result
[ https://issues.apache.org/jira/browse/KYLIN-3660?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16671200#comment-16671200 ] jiahua.hu commented on KYLIN-3660: -- However, there will be such a situation. T1 is my full data table, then I need to filter the data in t1 according to the conditions. As mentioned above, this is limited by a subset of the t2 table. In the current situation, I need to build a model for each situation. But the establishment process of each model is the same, the difference is only need to associate different sub-tables extracted from t2, or view. Therefore, I have a suggestion, whether kylin can support this feature in the follow-up. 但是,会有这样的情况。t1是我的全量数据表,然后我需要根据条件来过滤t1中的数据。如上面所述,此处是通过t2表的子集来限制。在当前情况下,我需要取针对每一个情况建立model。但是每一个model的建立过程都是一样的,区别只是需要关联不同的从t2中提取出来的子表,或者是view。因此,我有个建议,kylin在后续中能否支持这一特性。 some case like: mode1 select * from t1 join t2 on t1.user_id = t2.user_id; t2: create view t2 as select * from tt where exid='50'; mode2 select * from t1 join t3 on t1.user_id = t3.user_id; t3: create view t3 as select * from tt where exid='51'; moden select * from t1 join tn on t1.user_id = tn.user_id; tn: create view tn as select * from tt where exid='52'; > subquery, join wiht two cube return No Result > - > > Key: KYLIN-3660 > URL: https://issues.apache.org/jira/browse/KYLIN-3660 > Project: Kylin > Issue Type: Bug > Components: Query Engine >Affects Versions: v2.5.0 >Reporter: jiahua.hu >Priority: Major > > I have two cubes corresponding to two tables t1 and t2. When I pass the > user_Id in t2 to limit the query range of the t1 table, I return an empty > result. Then there is data in my t2 table. > 我有两个cube分别对应两张表t1和t2。当我通过t2中的user_Id去限制t1表的查询范围时,返回空结果。然后我的t2表里是有数据的。 > select * from t1 inner join (select user_id from t2 where EXID='50') t2 on > t1.user_id = t2.user_id; > select * from t1 where user_id in (select user_id from t2 where EXID='50'); > select * from t1 where user_id in (select '1002001'); > The above three statements execution results are returned No Result. > > But the following statement execution can return results > select * from t1 where user_id in ('1002001'); > > === > Query Id: 4a12730c-5222-b4a9-43fc-282e6bea7b51 > SQL: select * from t1 inner join (select user_id from t2 where EXID='50') t2 > on t1.user_id = t2.user_id; > User: ADMIN > Success: true > Duration: 0.001 > Project: test_project > Realization Names: [CUBE[name=t1_CUBE], CUBE[name=t2_CUBE]] > Cuboid Ids: [3, 63] > Total scan count: 50123 > Total scan bytes: 5412669 > Result row count: 0 > Accept Partial: true > Is Partial Result: false > Hit Exception Cache: false > Storage cache used: true > Is Query Push-Down: false > Is Prepare: false > Trace URL: null > Message: null > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (KYLIN-3611) Upgrade Tomcat to 7.0.91, 8.5.34 or later
[ https://issues.apache.org/jira/browse/KYLIN-3611?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16671206#comment-16671206 ] ASF subversion and git services commented on KYLIN-3611: Commit 995e0fde0b8f62aa94180db8ffc912b006cd5260 in kylin's branch refs/heads/2.5.x from shaofengshi [ https://gitbox.apache.org/repos/asf?p=kylin.git;h=995e0fd ] KYLIN-3611 upgrade to apache tomcat 7.0.91, update version in pom > Upgrade Tomcat to 7.0.91, 8.5.34 or later > - > > Key: KYLIN-3611 > URL: https://issues.apache.org/jira/browse/KYLIN-3611 > Project: Kylin > Issue Type: Improvement >Reporter: Shaofeng SHI >Assignee: zhoujie >Priority: Major > Fix For: v2.6.0, v2.5.1 > > > h2. [SECURITY] CVE-2018-11784 Apache Tomcat - Open Redirect > > > > CVE-2018-11784 Apache Tomcat - Open Redirect > Severity: Moderate > Vendor: The Apache Software Foundation > Versions Affected: > Apache Tomcat 9.0.0.M1 to 9.0.11 > Apache Tomcat 8.5.0 to 8.5.33 > Apache Tomcat 7.0.23 to 7.0.90 > The unsupported 8.0.x release line has not been analysed but is likely > to be affected. > Description: > When the default servlet returned a redirect to a directory (e.g. > redirecting to '/foo/' when the user requested '/foo') a specially > crafted URL could be used to cause the redirect to be generated to any > URI of the attackers choice. > Mitigation: > Users of the affected versions should apply one of the following > mitigations: > - Upgrade to Apache Tomcat 9.0.12 or later. > - Upgrade to Apache Tomcat 8.5.34 or later. > - Upgrade to Apache Tomcat 7.0.91 or later. > - Use mapperDirectoryRedirectEnabled="true" and > mapperContextRootRedirectEnabled="true" on the Context to ensure that > redirects are issued by the Mapper rather than the default Servlet. > See the Context configuration documentation for further important > details. > Credit: > This vulnerability was found by Sergey Bobrov and reported responsibly > to the Apache Tomcat Security Team. > History: > 2018-10-03 Original advisory > References: > [1] [http://tomcat.apache.org/security-9.html] > [2] [http://tomcat.apache.org/security-8.html] > [3] [http://tomcat.apache.org/security-7.html] -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (KYLIN-3611) Upgrade Tomcat to 7.0.91, 8.5.34 or later
[ https://issues.apache.org/jira/browse/KYLIN-3611?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16671208#comment-16671208 ] ASF subversion and git services commented on KYLIN-3611: Commit 3fabbe6d2b1199bedb7b74083feabc6586a6d644 in kylin's branch refs/heads/master from shaofengshi [ https://gitbox.apache.org/repos/asf?p=kylin.git;h=3fabbe6 ] KYLIN-3611 upgrade to apache tomcat 7.0.91, update version in pom > Upgrade Tomcat to 7.0.91, 8.5.34 or later > - > > Key: KYLIN-3611 > URL: https://issues.apache.org/jira/browse/KYLIN-3611 > Project: Kylin > Issue Type: Improvement >Reporter: Shaofeng SHI >Assignee: zhoujie >Priority: Major > Fix For: v2.6.0, v2.5.1 > > > h2. [SECURITY] CVE-2018-11784 Apache Tomcat - Open Redirect > > > > CVE-2018-11784 Apache Tomcat - Open Redirect > Severity: Moderate > Vendor: The Apache Software Foundation > Versions Affected: > Apache Tomcat 9.0.0.M1 to 9.0.11 > Apache Tomcat 8.5.0 to 8.5.33 > Apache Tomcat 7.0.23 to 7.0.90 > The unsupported 8.0.x release line has not been analysed but is likely > to be affected. > Description: > When the default servlet returned a redirect to a directory (e.g. > redirecting to '/foo/' when the user requested '/foo') a specially > crafted URL could be used to cause the redirect to be generated to any > URI of the attackers choice. > Mitigation: > Users of the affected versions should apply one of the following > mitigations: > - Upgrade to Apache Tomcat 9.0.12 or later. > - Upgrade to Apache Tomcat 8.5.34 or later. > - Upgrade to Apache Tomcat 7.0.91 or later. > - Use mapperDirectoryRedirectEnabled="true" and > mapperContextRootRedirectEnabled="true" on the Context to ensure that > redirects are issued by the Mapper rather than the default Servlet. > See the Context configuration documentation for further important > details. > Credit: > This vulnerability was found by Sergey Bobrov and reported responsibly > to the Apache Tomcat Security Team. > History: > 2018-10-03 Original advisory > References: > [1] [http://tomcat.apache.org/security-9.html] > [2] [http://tomcat.apache.org/security-8.html] > [3] [http://tomcat.apache.org/security-7.html] -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] asfgit commented on issue #322: KYLIN-3644 Fix SparkFactDistinct step NumberFormatException
asfgit commented on issue #322: KYLIN-3644 Fix SparkFactDistinct step NumberFormatException URL: https://github.com/apache/kylin/pull/322#issuecomment-434959245 Can one of the admins verify this patch? This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] Wayne1c opened a new pull request #322: KYLIN-3644 Fix SparkFactDistinct step NumberFormatException
Wayne1c opened a new pull request #322: KYLIN-3644 Fix SparkFactDistinct step NumberFormatException URL: https://github.com/apache/kylin/pull/322 This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Assigned] (KYLIN-3644) NumberFormatExcetion on null values when building cube with Spark
[ https://issues.apache.org/jira/browse/KYLIN-3644?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chao Long reassigned KYLIN-3644: Assignee: Chao Long > NumberFormatExcetion on null values when building cube with Spark > - > > Key: KYLIN-3644 > URL: https://issues.apache.org/jira/browse/KYLIN-3644 > Project: Kylin > Issue Type: Bug > Components: Spark Engine >Affects Versions: v2.5.0 >Reporter: Hubert STEFANI >Assignee: Chao Long >Priority: Major > Fix For: v2.5.1 > > Attachments: 00_zeppelin_notebook.jpg, 01_overview_table.jpg, > 02_dimension_cube.jpg, 03_measure_cube.jpg, sortieData.csv > > > We encounter an error any time we try to build a cube with the following > steps : > * upload a csv on AWS S3 with following characteristics : the column on > which the measure will be defined has some null values (Cf. attachment) > * create a hive table with spark > * create a model on top of this table, > * create a cube with a SUM measure > * chose Spark as Engine > * Launch build > Result : The build process fails at '{color:#4383b4}#7 Step Name: > {color}Build Cube with Spark' with the following error : > > "" > 18/10/23 09:25:39 INFO scheduler.DAGScheduler: Job 0 failed: > saveAsNewAPIHadoopDataset at SparkCubingByLayer.java:253, took 7,277136 s > Exception in thread "main" java.lang.RuntimeException: error execute > org.apache.kylin.engine.spark.SparkCubingByLayer. Root cause: Job aborted due > to stage failure: Task 0 in stage 0.0 failed 4 times, most recent failure: > Lost task 0.3 in stage 0.0 (TID 4, > ip-172-31-35-113.eu-west-1.compute.internal, executor 4): > java.lang.NumberFormatException: For input string: "\N" > at > sun.misc.FloatingDecimal.readJavaFormatString(FloatingDecimal.java:2043) > at sun.misc.FloatingDecimal.parseDouble(FloatingDecimal.java:110) > at java.lang.Double.parseDouble(Double.java:538) > at > org.apache.kylin.measure.basic.DoubleIngester.valueOf(DoubleIngester.java:38) > at > org.apache.kylin.measure.basic.DoubleIngester.valueOf(DoubleIngester.java:28) > at > org.apache.kylin.engine.mr.common.BaseCuboidBuilder.buildValueOf(BaseCuboidBuilder.java:162) > at > org.apache.kylin.engine.mr.common.BaseCuboidBuilder.buildValueObjects(BaseCuboidBuilder.java:127) > at > org.apache.kylin.engine.spark.SparkCubingByLayer$EncodeBaseCuboid.call(SparkCubingByLayer.java:297) > at > org.apache.kylin.engine.spark.SparkCubingByLayer$EncodeBaseCuboid.call(SparkCubingByLayer.java:257) > at > org.apache.spark.api.java.JavaPairRDD$$anonfun$pairFunToScalaFun$1.apply(JavaPairRDD.scala:1043) > at > org.apache.spark.api.java.JavaPairRDD$$anonfun$pairFunToScalaFun$1.apply(JavaPairRDD.scala:1043) > " > Note 1: the build process is OK when run with Map/Reduce Engine. > Note 2: the error doesn't seem to be related to AWS environment. > > Sample of csv : > ID;CATEGORIE;TEL;MONTANT;MAGASIN;MATRICULE;VILLE; > 970;161;6-98-6-6-42;838.47034;Magasin_19;Client_Matricule_28;MARSEILLE; > 971;89;62-15-2-64-86;;;Client_Matricule_1;LYON; > 972;87;17-64-97-74-42;;;Client_Matricule_105;ORBEC; > 973;174;79-33-90-0-55;;Magasin_7;Client_Matricule_55;AJACCIO; > 974;172;89-95-71-6-49;141.64174;Magasin_9;Client_Matricule_105;BASTIA; > 975;83;7-27-95-28-7;897.28204;;Client_Matricule_199;AJACCIO; > 976;170;67-72-18-29-34;164.07967;Magasin_3;Client_Matricule_137;LILLE; > 977;130;14-69-4-23-27;1928.9557;Magasin_1;Client_Matricule_17;NOMNOM; > 978;43;55-91-84-98-49;891.2691;Magasin_0;Client_Matricule_22;NOMNOM; > 979;117;98-96-0-54-39;1636.3994;Magasin_9;Client_Matricule_142;MARSEILLE; > 980;163;37-55-76-53-38;;;Client_Matricule_64;NEWYORK; > 981;106;32-40-6-46-15;;Magasin_2;Client_Matricule_158;NOMNOM; > 982;56;95-60-83-89-90;;;Client_Matricule_102;NOMNOM; > 983;168;21-56-62-0-58;;;Client_Matricule_160;NOMNOM; > 984;154;92-67-37-94-60;;;Client_Matricule_137;PARIS; > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (KYLIN-3644) NumberFormatExcetion on null values when building cube with Spark
[ https://issues.apache.org/jira/browse/KYLIN-3644?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16671212#comment-16671212 ] ASF GitHub Bot commented on KYLIN-3644: --- asfgit commented on issue #322: KYLIN-3644 Fix SparkFactDistinct step NumberFormatException URL: https://github.com/apache/kylin/pull/322#issuecomment-434959245 Can one of the admins verify this patch? This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > NumberFormatExcetion on null values when building cube with Spark > - > > Key: KYLIN-3644 > URL: https://issues.apache.org/jira/browse/KYLIN-3644 > Project: Kylin > Issue Type: Bug > Components: Spark Engine >Affects Versions: v2.5.0 >Reporter: Hubert STEFANI >Priority: Major > Fix For: v2.5.1 > > Attachments: 00_zeppelin_notebook.jpg, 01_overview_table.jpg, > 02_dimension_cube.jpg, 03_measure_cube.jpg, sortieData.csv > > > We encounter an error any time we try to build a cube with the following > steps : > * upload a csv on AWS S3 with following characteristics : the column on > which the measure will be defined has some null values (Cf. attachment) > * create a hive table with spark > * create a model on top of this table, > * create a cube with a SUM measure > * chose Spark as Engine > * Launch build > Result : The build process fails at '{color:#4383b4}#7 Step Name: > {color}Build Cube with Spark' with the following error : > > "" > 18/10/23 09:25:39 INFO scheduler.DAGScheduler: Job 0 failed: > saveAsNewAPIHadoopDataset at SparkCubingByLayer.java:253, took 7,277136 s > Exception in thread "main" java.lang.RuntimeException: error execute > org.apache.kylin.engine.spark.SparkCubingByLayer. Root cause: Job aborted due > to stage failure: Task 0 in stage 0.0 failed 4 times, most recent failure: > Lost task 0.3 in stage 0.0 (TID 4, > ip-172-31-35-113.eu-west-1.compute.internal, executor 4): > java.lang.NumberFormatException: For input string: "\N" > at > sun.misc.FloatingDecimal.readJavaFormatString(FloatingDecimal.java:2043) > at sun.misc.FloatingDecimal.parseDouble(FloatingDecimal.java:110) > at java.lang.Double.parseDouble(Double.java:538) > at > org.apache.kylin.measure.basic.DoubleIngester.valueOf(DoubleIngester.java:38) > at > org.apache.kylin.measure.basic.DoubleIngester.valueOf(DoubleIngester.java:28) > at > org.apache.kylin.engine.mr.common.BaseCuboidBuilder.buildValueOf(BaseCuboidBuilder.java:162) > at > org.apache.kylin.engine.mr.common.BaseCuboidBuilder.buildValueObjects(BaseCuboidBuilder.java:127) > at > org.apache.kylin.engine.spark.SparkCubingByLayer$EncodeBaseCuboid.call(SparkCubingByLayer.java:297) > at > org.apache.kylin.engine.spark.SparkCubingByLayer$EncodeBaseCuboid.call(SparkCubingByLayer.java:257) > at > org.apache.spark.api.java.JavaPairRDD$$anonfun$pairFunToScalaFun$1.apply(JavaPairRDD.scala:1043) > at > org.apache.spark.api.java.JavaPairRDD$$anonfun$pairFunToScalaFun$1.apply(JavaPairRDD.scala:1043) > " > Note 1: the build process is OK when run with Map/Reduce Engine. > Note 2: the error doesn't seem to be related to AWS environment. > > Sample of csv : > ID;CATEGORIE;TEL;MONTANT;MAGASIN;MATRICULE;VILLE; > 970;161;6-98-6-6-42;838.47034;Magasin_19;Client_Matricule_28;MARSEILLE; > 971;89;62-15-2-64-86;;;Client_Matricule_1;LYON; > 972;87;17-64-97-74-42;;;Client_Matricule_105;ORBEC; > 973;174;79-33-90-0-55;;Magasin_7;Client_Matricule_55;AJACCIO; > 974;172;89-95-71-6-49;141.64174;Magasin_9;Client_Matricule_105;BASTIA; > 975;83;7-27-95-28-7;897.28204;;Client_Matricule_199;AJACCIO; > 976;170;67-72-18-29-34;164.07967;Magasin_3;Client_Matricule_137;LILLE; > 977;130;14-69-4-23-27;1928.9557;Magasin_1;Client_Matricule_17;NOMNOM; > 978;43;55-91-84-98-49;891.2691;Magasin_0;Client_Matricule_22;NOMNOM; > 979;117;98-96-0-54-39;1636.3994;Magasin_9;Client_Matricule_142;MARSEILLE; > 980;163;37-55-76-53-38;;;Client_Matricule_64;NEWYORK; > 981;106;32-40-6-46-15;;Magasin_2;Client_Matricule_158;NOMNOM; > 982;56;95-60-83-89-90;;;Client_Matricule_102;NOMNOM; > 983;168;21-56-62-0-58;;;Client_Matricule_160;NOMNOM; > 984;154;92-67-37-94-60;;;Client_Matricule_137;PARIS; > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (KYLIN-3644) NumberFormatExcetion on null values when building cube with Spark
[ https://issues.apache.org/jira/browse/KYLIN-3644?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16671211#comment-16671211 ] ASF GitHub Bot commented on KYLIN-3644: --- Wayne1c opened a new pull request #322: KYLIN-3644 Fix SparkFactDistinct step NumberFormatException URL: https://github.com/apache/kylin/pull/322 This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > NumberFormatExcetion on null values when building cube with Spark > - > > Key: KYLIN-3644 > URL: https://issues.apache.org/jira/browse/KYLIN-3644 > Project: Kylin > Issue Type: Bug > Components: Spark Engine >Affects Versions: v2.5.0 >Reporter: Hubert STEFANI >Priority: Major > Fix For: v2.5.1 > > Attachments: 00_zeppelin_notebook.jpg, 01_overview_table.jpg, > 02_dimension_cube.jpg, 03_measure_cube.jpg, sortieData.csv > > > We encounter an error any time we try to build a cube with the following > steps : > * upload a csv on AWS S3 with following characteristics : the column on > which the measure will be defined has some null values (Cf. attachment) > * create a hive table with spark > * create a model on top of this table, > * create a cube with a SUM measure > * chose Spark as Engine > * Launch build > Result : The build process fails at '{color:#4383b4}#7 Step Name: > {color}Build Cube with Spark' with the following error : > > "" > 18/10/23 09:25:39 INFO scheduler.DAGScheduler: Job 0 failed: > saveAsNewAPIHadoopDataset at SparkCubingByLayer.java:253, took 7,277136 s > Exception in thread "main" java.lang.RuntimeException: error execute > org.apache.kylin.engine.spark.SparkCubingByLayer. Root cause: Job aborted due > to stage failure: Task 0 in stage 0.0 failed 4 times, most recent failure: > Lost task 0.3 in stage 0.0 (TID 4, > ip-172-31-35-113.eu-west-1.compute.internal, executor 4): > java.lang.NumberFormatException: For input string: "\N" > at > sun.misc.FloatingDecimal.readJavaFormatString(FloatingDecimal.java:2043) > at sun.misc.FloatingDecimal.parseDouble(FloatingDecimal.java:110) > at java.lang.Double.parseDouble(Double.java:538) > at > org.apache.kylin.measure.basic.DoubleIngester.valueOf(DoubleIngester.java:38) > at > org.apache.kylin.measure.basic.DoubleIngester.valueOf(DoubleIngester.java:28) > at > org.apache.kylin.engine.mr.common.BaseCuboidBuilder.buildValueOf(BaseCuboidBuilder.java:162) > at > org.apache.kylin.engine.mr.common.BaseCuboidBuilder.buildValueObjects(BaseCuboidBuilder.java:127) > at > org.apache.kylin.engine.spark.SparkCubingByLayer$EncodeBaseCuboid.call(SparkCubingByLayer.java:297) > at > org.apache.kylin.engine.spark.SparkCubingByLayer$EncodeBaseCuboid.call(SparkCubingByLayer.java:257) > at > org.apache.spark.api.java.JavaPairRDD$$anonfun$pairFunToScalaFun$1.apply(JavaPairRDD.scala:1043) > at > org.apache.spark.api.java.JavaPairRDD$$anonfun$pairFunToScalaFun$1.apply(JavaPairRDD.scala:1043) > " > Note 1: the build process is OK when run with Map/Reduce Engine. > Note 2: the error doesn't seem to be related to AWS environment. > > Sample of csv : > ID;CATEGORIE;TEL;MONTANT;MAGASIN;MATRICULE;VILLE; > 970;161;6-98-6-6-42;838.47034;Magasin_19;Client_Matricule_28;MARSEILLE; > 971;89;62-15-2-64-86;;;Client_Matricule_1;LYON; > 972;87;17-64-97-74-42;;;Client_Matricule_105;ORBEC; > 973;174;79-33-90-0-55;;Magasin_7;Client_Matricule_55;AJACCIO; > 974;172;89-95-71-6-49;141.64174;Magasin_9;Client_Matricule_105;BASTIA; > 975;83;7-27-95-28-7;897.28204;;Client_Matricule_199;AJACCIO; > 976;170;67-72-18-29-34;164.07967;Magasin_3;Client_Matricule_137;LILLE; > 977;130;14-69-4-23-27;1928.9557;Magasin_1;Client_Matricule_17;NOMNOM; > 978;43;55-91-84-98-49;891.2691;Magasin_0;Client_Matricule_22;NOMNOM; > 979;117;98-96-0-54-39;1636.3994;Magasin_9;Client_Matricule_142;MARSEILLE; > 980;163;37-55-76-53-38;;;Client_Matricule_64;NEWYORK; > 981;106;32-40-6-46-15;;Magasin_2;Client_Matricule_158;NOMNOM; > 982;56;95-60-83-89-90;;;Client_Matricule_102;NOMNOM; > 983;168;21-56-62-0-58;;;Client_Matricule_160;NOMNOM; > 984;154;92-67-37-94-60;;;Client_Matricule_137;PARIS; > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (KYLIN-3656) Improve HLLCounter performance
[ https://issues.apache.org/jira/browse/KYLIN-3656?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chang chen updated KYLIN-3656: -- Attachment: 0001-KYLIN-3656-Improve-HLLCounter-performance.patch > Improve HLLCounter performance > -- > > Key: KYLIN-3656 > URL: https://issues.apache.org/jira/browse/KYLIN-3656 > Project: Kylin > Issue Type: Improvement >Affects Versions: all >Reporter: Chang chen >Priority: Major > Attachments: 0001-KYLIN-3656-Improve-HLLCounter-performance.patch > > > The current HLLCounter implementation has some room to improve performance, > as we find in our product environment. > # Create HLLCounter from another HLLCounter, we can copy register instead > of merge > # To compute harmonic mean in the HLLCSnapshot, we could > ## using table to cache all 1/2^r without computing on the fly > ## remove floating addition by using integer addition in the bigger loop > ## remove branch, e.g. needn't checking whether registers[i] is zero or not, > although this is minor improvement. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (KYLIN-3662) exception message "Cannot find project '%s'." should be formated
Lingang Deng created KYLIN-3662: --- Summary: exception message "Cannot find project '%s'." should be formated Key: KYLIN-3662 URL: https://issues.apache.org/jira/browse/KYLIN-3662 Project: Kylin Issue Type: Bug Affects Versions: v2.5.0 Reporter: Lingang Deng Assignee: Lingang Deng When use kylin dashboard without system cube, exception is threw as follows, {code:java} org.apache.kylin.rest.exception.BadRequestException: Cannot find project '%s'. at org.apache.kylin.rest.service.QueryService.doQueryWithCache(QueryService.java:378) at org.apache.kylin.rest.service.QueryService.doQueryWithCache(QueryService.java:359) at org.apache.kylin.rest.controller.DashboardController.getQueryMetrics(DashboardController.java:74) {code} The log is unfriendly to users. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (KYLIN-3656) Improve HLLCounter performance
[ https://issues.apache.org/jira/browse/KYLIN-3656?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shaofeng SHI updated KYLIN-3656: Fix Version/s: v2.6.0 > Improve HLLCounter performance > -- > > Key: KYLIN-3656 > URL: https://issues.apache.org/jira/browse/KYLIN-3656 > Project: Kylin > Issue Type: Improvement >Affects Versions: all >Reporter: Chang chen >Priority: Major > Fix For: v2.6.0 > > Attachments: 0001-KYLIN-3656-Improve-HLLCounter-performance.patch > > > The current HLLCounter implementation has some room to improve performance, > as we find in our product environment. > # Create HLLCounter from another HLLCounter, we can copy register instead > of merge > # To compute harmonic mean in the HLLCSnapshot, we could > ## using table to cache all 1/2^r without computing on the fly > ## remove floating addition by using integer addition in the bigger loop > ## remove branch, e.g. needn't checking whether registers[i] is zero or not, > although this is minor improvement. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (KYLIN-3656) Improve HLLCounter performance
[ https://issues.apache.org/jira/browse/KYLIN-3656?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shaofeng SHI reassigned KYLIN-3656: --- Assignee: Chang chen > Improve HLLCounter performance > -- > > Key: KYLIN-3656 > URL: https://issues.apache.org/jira/browse/KYLIN-3656 > Project: Kylin > Issue Type: Improvement >Affects Versions: all >Reporter: Chang chen >Assignee: Chang chen >Priority: Major > Fix For: v2.6.0 > > Attachments: 0001-KYLIN-3656-Improve-HLLCounter-performance.patch > > > The current HLLCounter implementation has some room to improve performance, > as we find in our product environment. > # Create HLLCounter from another HLLCounter, we can copy register instead > of merge > # To compute harmonic mean in the HLLCSnapshot, we could > ## using table to cache all 1/2^r without computing on the fly > ## remove floating addition by using integer addition in the bigger loop > ## remove branch, e.g. needn't checking whether registers[i] is zero or not, > although this is minor improvement. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (KYLIN-3662) exception message "Cannot find project '%s'." should be formated
[ https://issues.apache.org/jira/browse/KYLIN-3662?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lingang Deng updated KYLIN-3662: Attachment: 0001-KYLIN-3662.patch > exception message "Cannot find project '%s'." should be formated > > > Key: KYLIN-3662 > URL: https://issues.apache.org/jira/browse/KYLIN-3662 > Project: Kylin > Issue Type: Bug >Affects Versions: v2.5.0 >Reporter: Lingang Deng >Assignee: Lingang Deng >Priority: Minor > Attachments: 0001-KYLIN-3662.patch > > > When use kylin dashboard without system cube, exception is threw as follows, > {code:java} > org.apache.kylin.rest.exception.BadRequestException: Cannot find project '%s'. > at > org.apache.kylin.rest.service.QueryService.doQueryWithCache(QueryService.java:378) > at > org.apache.kylin.rest.service.QueryService.doQueryWithCache(QueryService.java:359) > at > org.apache.kylin.rest.controller.DashboardController.getQueryMetrics(DashboardController.java:74) > {code} > The log is unfriendly to users. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (KYLIN-3660) subquery, join wiht two cube return No Result
[ https://issues.apache.org/jira/browse/KYLIN-3660?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16671294#comment-16671294 ] XiaoXiang Yu commented on KYLIN-3660: - Could you build ONE cube which use t1 as fact table and use t2 as a lookup table? It might meet your request. > subquery, join wiht two cube return No Result > - > > Key: KYLIN-3660 > URL: https://issues.apache.org/jira/browse/KYLIN-3660 > Project: Kylin > Issue Type: Bug > Components: Query Engine >Affects Versions: v2.5.0 >Reporter: jiahua.hu >Priority: Major > > I have two cubes corresponding to two tables t1 and t2. When I pass the > user_Id in t2 to limit the query range of the t1 table, I return an empty > result. Then there is data in my t2 table. > 我有两个cube分别对应两张表t1和t2。当我通过t2中的user_Id去限制t1表的查询范围时,返回空结果。然后我的t2表里是有数据的。 > select * from t1 inner join (select user_id from t2 where EXID='50') t2 on > t1.user_id = t2.user_id; > select * from t1 where user_id in (select user_id from t2 where EXID='50'); > select * from t1 where user_id in (select '1002001'); > The above three statements execution results are returned No Result. > > But the following statement execution can return results > select * from t1 where user_id in ('1002001'); > > === > Query Id: 4a12730c-5222-b4a9-43fc-282e6bea7b51 > SQL: select * from t1 inner join (select user_id from t2 where EXID='50') t2 > on t1.user_id = t2.user_id; > User: ADMIN > Success: true > Duration: 0.001 > Project: test_project > Realization Names: [CUBE[name=t1_CUBE], CUBE[name=t2_CUBE]] > Cuboid Ids: [3, 63] > Total scan count: 50123 > Total scan bytes: 5412669 > Result row count: 0 > Accept Partial: true > Is Partial Result: false > Hit Exception Cache: false > Storage cache used: true > Is Query Push-Down: false > Is Prepare: false > Trace URL: null > Message: null > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Issue Comment Deleted] (KYLIN-3660) subquery, join wiht two cube return No Result
[ https://issues.apache.org/jira/browse/KYLIN-3660?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] XiaoXiang Yu updated KYLIN-3660: Comment: was deleted (was: Could you build ONE cube which use t1 as fact table and use t2 as a lookup table? It might meet your request.) > subquery, join wiht two cube return No Result > - > > Key: KYLIN-3660 > URL: https://issues.apache.org/jira/browse/KYLIN-3660 > Project: Kylin > Issue Type: Bug > Components: Query Engine >Affects Versions: v2.5.0 >Reporter: jiahua.hu >Priority: Major > > I have two cubes corresponding to two tables t1 and t2. When I pass the > user_Id in t2 to limit the query range of the t1 table, I return an empty > result. Then there is data in my t2 table. > 我有两个cube分别对应两张表t1和t2。当我通过t2中的user_Id去限制t1表的查询范围时,返回空结果。然后我的t2表里是有数据的。 > select * from t1 inner join (select user_id from t2 where EXID='50') t2 on > t1.user_id = t2.user_id; > select * from t1 where user_id in (select user_id from t2 where EXID='50'); > select * from t1 where user_id in (select '1002001'); > The above three statements execution results are returned No Result. > > But the following statement execution can return results > select * from t1 where user_id in ('1002001'); > > === > Query Id: 4a12730c-5222-b4a9-43fc-282e6bea7b51 > SQL: select * from t1 inner join (select user_id from t2 where EXID='50') t2 > on t1.user_id = t2.user_id; > User: ADMIN > Success: true > Duration: 0.001 > Project: test_project > Realization Names: [CUBE[name=t1_CUBE], CUBE[name=t2_CUBE]] > Cuboid Ids: [3, 63] > Total scan count: 50123 > Total scan bytes: 5412669 > Result row count: 0 > Accept Partial: true > Is Partial Result: false > Hit Exception Cache: false > Storage cache used: true > Is Query Push-Down: false > Is Prepare: false > Trace URL: null > Message: null > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] shaofengshi closed pull request #322: KYLIN-3644 Fix SparkFactDistinct step NumberFormatException
shaofengshi closed pull request #322: KYLIN-3644 Fix SparkFactDistinct step NumberFormatException URL: https://github.com/apache/kylin/pull/322 This is a PR merged from a forked repository. As GitHub hides the original diff on merge, it is displayed below for the sake of provenance: As this is a foreign pull request (from a fork), the diff is supplied below (as it won't show otherwise due to GitHub magic): diff --git a/core-cube/src/main/java/org/apache/kylin/cube/util/KeyValueBuilder.java b/core-cube/src/main/java/org/apache/kylin/cube/util/KeyValueBuilder.java index 0ba4fd8ed1..0636a5c343 100644 --- a/core-cube/src/main/java/org/apache/kylin/cube/util/KeyValueBuilder.java +++ b/core-cube/src/main/java/org/apache/kylin/cube/util/KeyValueBuilder.java @@ -56,7 +56,7 @@ private void initNullStrings() { } } -protected boolean isNull(String v) { +public boolean isNull(String v) { return nullStrs.contains(v); } diff --git a/engine-spark/src/main/java/org/apache/kylin/engine/spark/SparkFactDistinct.java b/engine-spark/src/main/java/org/apache/kylin/engine/spark/SparkFactDistinct.java index 5cfd2d7ccb..cdd0ac22d7 100644 --- a/engine-spark/src/main/java/org/apache/kylin/engine/spark/SparkFactDistinct.java +++ b/engine-spark/src/main/java/org/apache/kylin/engine/spark/SparkFactDistinct.java @@ -63,6 +63,7 @@ import org.apache.kylin.cube.cuboid.CuboidUtil; import org.apache.kylin.cube.model.CubeDesc; import org.apache.kylin.cube.model.CubeJoinedFlatTableEnrich; +import org.apache.kylin.cube.util.KeyValueBuilder; import org.apache.kylin.dict.DictionaryGenerator; import org.apache.kylin.dict.IDictionaryBuilder; import org.apache.kylin.engine.EngineFactory; @@ -250,6 +251,7 @@ protected void execute(OptionsHelper optionsHelper) throws Exception { private Map dimensionRangeInfoMap; private transient ByteBuffer tmpbuf; private LongAccumulator bytesWritten; +private KeyValueBuilder keyValueBuilder; public FlatOutputFucntion(String cubeName, String segmentId, String metaurl, SerializableConfiguration conf, int samplingPercent, LongAccumulator bytesWritten) { @@ -272,6 +274,7 @@ private void init() { CubeJoinedFlatTableEnrich intermediateTableDesc = new CubeJoinedFlatTableEnrich( EngineFactory.getJoinedFlatTableDesc(cubeSegment), cubeDesc); +keyValueBuilder = new KeyValueBuilder(intermediateTableDesc); reducerMapping = new FactDistinctColumnsReducerMapping(cubeInstance); tmpbuf = ByteBuffer.allocate(4096); @@ -317,7 +320,7 @@ private void init() { for (int i = 0; i < allCols.size(); i++) { String fieldValue = row[columnIndex[i]]; -if (fieldValue == null) +if (fieldValue == null || keyValueBuilder.isNull(fieldValue)) continue; final DataType type = allCols.get(i).getType(); This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (KYLIN-3644) NumberFormatExcetion on null values when building cube with Spark
[ https://issues.apache.org/jira/browse/KYLIN-3644?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16671427#comment-16671427 ] ASF subversion and git services commented on KYLIN-3644: Commit 65441434e0aff410050009b4538ef3bf9e7af93f in kylin's branch refs/heads/master from chao long [ https://gitbox.apache.org/repos/asf?p=kylin.git;h=6544143 ] KYLIN-3644 Fix SparkFactDistinct step NumberFormatException > NumberFormatExcetion on null values when building cube with Spark > - > > Key: KYLIN-3644 > URL: https://issues.apache.org/jira/browse/KYLIN-3644 > Project: Kylin > Issue Type: Bug > Components: Spark Engine >Affects Versions: v2.5.0 >Reporter: Hubert STEFANI >Assignee: Chao Long >Priority: Major > Fix For: v2.5.1 > > Attachments: 00_zeppelin_notebook.jpg, 01_overview_table.jpg, > 02_dimension_cube.jpg, 03_measure_cube.jpg, sortieData.csv > > > We encounter an error any time we try to build a cube with the following > steps : > * upload a csv on AWS S3 with following characteristics : the column on > which the measure will be defined has some null values (Cf. attachment) > * create a hive table with spark > * create a model on top of this table, > * create a cube with a SUM measure > * chose Spark as Engine > * Launch build > Result : The build process fails at '{color:#4383b4}#7 Step Name: > {color}Build Cube with Spark' with the following error : > > "" > 18/10/23 09:25:39 INFO scheduler.DAGScheduler: Job 0 failed: > saveAsNewAPIHadoopDataset at SparkCubingByLayer.java:253, took 7,277136 s > Exception in thread "main" java.lang.RuntimeException: error execute > org.apache.kylin.engine.spark.SparkCubingByLayer. Root cause: Job aborted due > to stage failure: Task 0 in stage 0.0 failed 4 times, most recent failure: > Lost task 0.3 in stage 0.0 (TID 4, > ip-172-31-35-113.eu-west-1.compute.internal, executor 4): > java.lang.NumberFormatException: For input string: "\N" > at > sun.misc.FloatingDecimal.readJavaFormatString(FloatingDecimal.java:2043) > at sun.misc.FloatingDecimal.parseDouble(FloatingDecimal.java:110) > at java.lang.Double.parseDouble(Double.java:538) > at > org.apache.kylin.measure.basic.DoubleIngester.valueOf(DoubleIngester.java:38) > at > org.apache.kylin.measure.basic.DoubleIngester.valueOf(DoubleIngester.java:28) > at > org.apache.kylin.engine.mr.common.BaseCuboidBuilder.buildValueOf(BaseCuboidBuilder.java:162) > at > org.apache.kylin.engine.mr.common.BaseCuboidBuilder.buildValueObjects(BaseCuboidBuilder.java:127) > at > org.apache.kylin.engine.spark.SparkCubingByLayer$EncodeBaseCuboid.call(SparkCubingByLayer.java:297) > at > org.apache.kylin.engine.spark.SparkCubingByLayer$EncodeBaseCuboid.call(SparkCubingByLayer.java:257) > at > org.apache.spark.api.java.JavaPairRDD$$anonfun$pairFunToScalaFun$1.apply(JavaPairRDD.scala:1043) > at > org.apache.spark.api.java.JavaPairRDD$$anonfun$pairFunToScalaFun$1.apply(JavaPairRDD.scala:1043) > " > Note 1: the build process is OK when run with Map/Reduce Engine. > Note 2: the error doesn't seem to be related to AWS environment. > > Sample of csv : > ID;CATEGORIE;TEL;MONTANT;MAGASIN;MATRICULE;VILLE; > 970;161;6-98-6-6-42;838.47034;Magasin_19;Client_Matricule_28;MARSEILLE; > 971;89;62-15-2-64-86;;;Client_Matricule_1;LYON; > 972;87;17-64-97-74-42;;;Client_Matricule_105;ORBEC; > 973;174;79-33-90-0-55;;Magasin_7;Client_Matricule_55;AJACCIO; > 974;172;89-95-71-6-49;141.64174;Magasin_9;Client_Matricule_105;BASTIA; > 975;83;7-27-95-28-7;897.28204;;Client_Matricule_199;AJACCIO; > 976;170;67-72-18-29-34;164.07967;Magasin_3;Client_Matricule_137;LILLE; > 977;130;14-69-4-23-27;1928.9557;Magasin_1;Client_Matricule_17;NOMNOM; > 978;43;55-91-84-98-49;891.2691;Magasin_0;Client_Matricule_22;NOMNOM; > 979;117;98-96-0-54-39;1636.3994;Magasin_9;Client_Matricule_142;MARSEILLE; > 980;163;37-55-76-53-38;;;Client_Matricule_64;NEWYORK; > 981;106;32-40-6-46-15;;Magasin_2;Client_Matricule_158;NOMNOM; > 982;56;95-60-83-89-90;;;Client_Matricule_102;NOMNOM; > 983;168;21-56-62-0-58;;;Client_Matricule_160;NOMNOM; > 984;154;92-67-37-94-60;;;Client_Matricule_137;PARIS; > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (KYLIN-3644) NumberFormatExcetion on null values when building cube with Spark
[ https://issues.apache.org/jira/browse/KYLIN-3644?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16671426#comment-16671426 ] ASF GitHub Bot commented on KYLIN-3644: --- shaofengshi closed pull request #322: KYLIN-3644 Fix SparkFactDistinct step NumberFormatException URL: https://github.com/apache/kylin/pull/322 This is a PR merged from a forked repository. As GitHub hides the original diff on merge, it is displayed below for the sake of provenance: As this is a foreign pull request (from a fork), the diff is supplied below (as it won't show otherwise due to GitHub magic): diff --git a/core-cube/src/main/java/org/apache/kylin/cube/util/KeyValueBuilder.java b/core-cube/src/main/java/org/apache/kylin/cube/util/KeyValueBuilder.java index 0ba4fd8ed1..0636a5c343 100644 --- a/core-cube/src/main/java/org/apache/kylin/cube/util/KeyValueBuilder.java +++ b/core-cube/src/main/java/org/apache/kylin/cube/util/KeyValueBuilder.java @@ -56,7 +56,7 @@ private void initNullStrings() { } } -protected boolean isNull(String v) { +public boolean isNull(String v) { return nullStrs.contains(v); } diff --git a/engine-spark/src/main/java/org/apache/kylin/engine/spark/SparkFactDistinct.java b/engine-spark/src/main/java/org/apache/kylin/engine/spark/SparkFactDistinct.java index 5cfd2d7ccb..cdd0ac22d7 100644 --- a/engine-spark/src/main/java/org/apache/kylin/engine/spark/SparkFactDistinct.java +++ b/engine-spark/src/main/java/org/apache/kylin/engine/spark/SparkFactDistinct.java @@ -63,6 +63,7 @@ import org.apache.kylin.cube.cuboid.CuboidUtil; import org.apache.kylin.cube.model.CubeDesc; import org.apache.kylin.cube.model.CubeJoinedFlatTableEnrich; +import org.apache.kylin.cube.util.KeyValueBuilder; import org.apache.kylin.dict.DictionaryGenerator; import org.apache.kylin.dict.IDictionaryBuilder; import org.apache.kylin.engine.EngineFactory; @@ -250,6 +251,7 @@ protected void execute(OptionsHelper optionsHelper) throws Exception { private Map dimensionRangeInfoMap; private transient ByteBuffer tmpbuf; private LongAccumulator bytesWritten; +private KeyValueBuilder keyValueBuilder; public FlatOutputFucntion(String cubeName, String segmentId, String metaurl, SerializableConfiguration conf, int samplingPercent, LongAccumulator bytesWritten) { @@ -272,6 +274,7 @@ private void init() { CubeJoinedFlatTableEnrich intermediateTableDesc = new CubeJoinedFlatTableEnrich( EngineFactory.getJoinedFlatTableDesc(cubeSegment), cubeDesc); +keyValueBuilder = new KeyValueBuilder(intermediateTableDesc); reducerMapping = new FactDistinctColumnsReducerMapping(cubeInstance); tmpbuf = ByteBuffer.allocate(4096); @@ -317,7 +320,7 @@ private void init() { for (int i = 0; i < allCols.size(); i++) { String fieldValue = row[columnIndex[i]]; -if (fieldValue == null) +if (fieldValue == null || keyValueBuilder.isNull(fieldValue)) continue; final DataType type = allCols.get(i).getType(); This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > NumberFormatExcetion on null values when building cube with Spark > - > > Key: KYLIN-3644 > URL: https://issues.apache.org/jira/browse/KYLIN-3644 > Project: Kylin > Issue Type: Bug > Components: Spark Engine >Affects Versions: v2.5.0 >Reporter: Hubert STEFANI >Assignee: Chao Long >Priority: Major > Fix For: v2.5.1 > > Attachments: 00_zeppelin_notebook.jpg, 01_overview_table.jpg, > 02_dimension_cube.jpg, 03_measure_cube.jpg, sortieData.csv > > > We encounter an error any time we try to build a cube with the following > steps : > * upload a csv on AWS S3 with following characteristics : the column on > which the measure will be defined has some null values (Cf. attachment) > * create a hive table with spark > * create a model on top of this table, > * create a cube with a SUM measure > * chose Spark as Engine > * Launch build > Result : The build process fails at '{color:#4383b4}#7 Step Name: > {color}Build Cube with Spark' with the following error : > > "" > 18/10/23 09:25:39 INFO scheduler.DAGScheduler: Job 0 failed: > saveAsNewAPIHadoopDataset at SparkCubingByLayer.java:253, took 7,277136 s > Exception in thread "main" java.lang.Runt
[jira] [Commented] (KYLIN-3644) NumberFormatExcetion on null values when building cube with Spark
[ https://issues.apache.org/jira/browse/KYLIN-3644?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16671428#comment-16671428 ] ASF subversion and git services commented on KYLIN-3644: Commit 0411360557ec2355448fa67c3ae18a9473179afa in kylin's branch refs/heads/2.5.x from chao long [ https://gitbox.apache.org/repos/asf?p=kylin.git;h=0411360 ] KYLIN-3644 Fix SparkFactDistinct step NumberFormatException > NumberFormatExcetion on null values when building cube with Spark > - > > Key: KYLIN-3644 > URL: https://issues.apache.org/jira/browse/KYLIN-3644 > Project: Kylin > Issue Type: Bug > Components: Spark Engine >Affects Versions: v2.5.0 >Reporter: Hubert STEFANI >Assignee: Chao Long >Priority: Major > Fix For: v2.5.1 > > Attachments: 00_zeppelin_notebook.jpg, 01_overview_table.jpg, > 02_dimension_cube.jpg, 03_measure_cube.jpg, sortieData.csv > > > We encounter an error any time we try to build a cube with the following > steps : > * upload a csv on AWS S3 with following characteristics : the column on > which the measure will be defined has some null values (Cf. attachment) > * create a hive table with spark > * create a model on top of this table, > * create a cube with a SUM measure > * chose Spark as Engine > * Launch build > Result : The build process fails at '{color:#4383b4}#7 Step Name: > {color}Build Cube with Spark' with the following error : > > "" > 18/10/23 09:25:39 INFO scheduler.DAGScheduler: Job 0 failed: > saveAsNewAPIHadoopDataset at SparkCubingByLayer.java:253, took 7,277136 s > Exception in thread "main" java.lang.RuntimeException: error execute > org.apache.kylin.engine.spark.SparkCubingByLayer. Root cause: Job aborted due > to stage failure: Task 0 in stage 0.0 failed 4 times, most recent failure: > Lost task 0.3 in stage 0.0 (TID 4, > ip-172-31-35-113.eu-west-1.compute.internal, executor 4): > java.lang.NumberFormatException: For input string: "\N" > at > sun.misc.FloatingDecimal.readJavaFormatString(FloatingDecimal.java:2043) > at sun.misc.FloatingDecimal.parseDouble(FloatingDecimal.java:110) > at java.lang.Double.parseDouble(Double.java:538) > at > org.apache.kylin.measure.basic.DoubleIngester.valueOf(DoubleIngester.java:38) > at > org.apache.kylin.measure.basic.DoubleIngester.valueOf(DoubleIngester.java:28) > at > org.apache.kylin.engine.mr.common.BaseCuboidBuilder.buildValueOf(BaseCuboidBuilder.java:162) > at > org.apache.kylin.engine.mr.common.BaseCuboidBuilder.buildValueObjects(BaseCuboidBuilder.java:127) > at > org.apache.kylin.engine.spark.SparkCubingByLayer$EncodeBaseCuboid.call(SparkCubingByLayer.java:297) > at > org.apache.kylin.engine.spark.SparkCubingByLayer$EncodeBaseCuboid.call(SparkCubingByLayer.java:257) > at > org.apache.spark.api.java.JavaPairRDD$$anonfun$pairFunToScalaFun$1.apply(JavaPairRDD.scala:1043) > at > org.apache.spark.api.java.JavaPairRDD$$anonfun$pairFunToScalaFun$1.apply(JavaPairRDD.scala:1043) > " > Note 1: the build process is OK when run with Map/Reduce Engine. > Note 2: the error doesn't seem to be related to AWS environment. > > Sample of csv : > ID;CATEGORIE;TEL;MONTANT;MAGASIN;MATRICULE;VILLE; > 970;161;6-98-6-6-42;838.47034;Magasin_19;Client_Matricule_28;MARSEILLE; > 971;89;62-15-2-64-86;;;Client_Matricule_1;LYON; > 972;87;17-64-97-74-42;;;Client_Matricule_105;ORBEC; > 973;174;79-33-90-0-55;;Magasin_7;Client_Matricule_55;AJACCIO; > 974;172;89-95-71-6-49;141.64174;Magasin_9;Client_Matricule_105;BASTIA; > 975;83;7-27-95-28-7;897.28204;;Client_Matricule_199;AJACCIO; > 976;170;67-72-18-29-34;164.07967;Magasin_3;Client_Matricule_137;LILLE; > 977;130;14-69-4-23-27;1928.9557;Magasin_1;Client_Matricule_17;NOMNOM; > 978;43;55-91-84-98-49;891.2691;Magasin_0;Client_Matricule_22;NOMNOM; > 979;117;98-96-0-54-39;1636.3994;Magasin_9;Client_Matricule_142;MARSEILLE; > 980;163;37-55-76-53-38;;;Client_Matricule_64;NEWYORK; > 981;106;32-40-6-46-15;;Magasin_2;Client_Matricule_158;NOMNOM; > 982;56;95-60-83-89-90;;;Client_Matricule_102;NOMNOM; > 983;168;21-56-62-0-58;;;Client_Matricule_160;NOMNOM; > 984;154;92-67-37-94-60;;;Client_Matricule_137;PARIS; > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (KYLIN-3656) Improve HLLCounter performance
[ https://issues.apache.org/jira/browse/KYLIN-3656?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16671478#comment-16671478 ] Chang chen commented on KYLIN-3656: --- To get better result, we disable compress by following settings: kylin.storage.hbase.endpoint-compress-result=false > Improve HLLCounter performance > -- > > Key: KYLIN-3656 > URL: https://issues.apache.org/jira/browse/KYLIN-3656 > Project: Kylin > Issue Type: Improvement >Affects Versions: all >Reporter: Chang chen >Assignee: Chang chen >Priority: Major > Fix For: v2.6.0 > > Attachments: 0001-KYLIN-3656-Improve-HLLCounter-performance.patch > > > The current HLLCounter implementation has some room to improve performance, > as we find in our product environment. > # Create HLLCounter from another HLLCounter, we can copy register instead > of merge > # To compute harmonic mean in the HLLCSnapshot, we could > ## using table to cache all 1/2^r without computing on the fly > ## remove floating addition by using integer addition in the bigger loop > ## remove branch, e.g. needn't checking whether registers[i] is zero or not, > although this is minor improvement. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] asfgit commented on issue #323: Update FAQ, update storage and metadata cleanup doc, add the online v…
asfgit commented on issue #323: Update FAQ, update storage and metadata cleanup doc, add the online v… URL: https://github.com/apache/kylin/pull/323#issuecomment-435013136 Can one of the admins verify this patch? This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] GinaZhai opened a new pull request #323: Update FAQ, update storage and metadata cleanup doc, add the online v…
GinaZhai opened a new pull request #323: Update FAQ, update storage and metadata cleanup doc, add the online v… URL: https://github.com/apache/kylin/pull/323 …ideo section This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Closed] (KYLIN-3660) subquery, join wiht two cube return No Result
[ https://issues.apache.org/jira/browse/KYLIN-3660?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] jiahua.hu closed KYLIN-3660. Resolution: Not A Bug > subquery, join wiht two cube return No Result > - > > Key: KYLIN-3660 > URL: https://issues.apache.org/jira/browse/KYLIN-3660 > Project: Kylin > Issue Type: Bug > Components: Query Engine >Affects Versions: v2.5.0 >Reporter: jiahua.hu >Priority: Major > > I have two cubes corresponding to two tables t1 and t2. When I pass the > user_Id in t2 to limit the query range of the t1 table, I return an empty > result. Then there is data in my t2 table. > 我有两个cube分别对应两张表t1和t2。当我通过t2中的user_Id去限制t1表的查询范围时,返回空结果。然后我的t2表里是有数据的。 > select * from t1 inner join (select user_id from t2 where EXID='50') t2 on > t1.user_id = t2.user_id; > select * from t1 where user_id in (select user_id from t2 where EXID='50'); > select * from t1 where user_id in (select '1002001'); > The above three statements execution results are returned No Result. > > But the following statement execution can return results > select * from t1 where user_id in ('1002001'); > > === > Query Id: 4a12730c-5222-b4a9-43fc-282e6bea7b51 > SQL: select * from t1 inner join (select user_id from t2 where EXID='50') t2 > on t1.user_id = t2.user_id; > User: ADMIN > Success: true > Duration: 0.001 > Project: test_project > Realization Names: [CUBE[name=t1_CUBE], CUBE[name=t2_CUBE]] > Cuboid Ids: [3, 63] > Total scan count: 50123 > Total scan bytes: 5412669 > Result row count: 0 > Accept Partial: true > Is Partial Result: false > Hit Exception Cache: false > Storage cache used: true > Is Query Push-Down: false > Is Prepare: false > Trace URL: null > Message: null > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] shaofengshi closed pull request #323: Update FAQ, update storage and metadata cleanup doc, add the online v…
shaofengshi closed pull request #323: Update FAQ, update storage and metadata cleanup doc, add the online v… URL: https://github.com/apache/kylin/pull/323 This is a PR merged from a forked repository. As GitHub hides the original diff on merge, it is displayed below for the sake of provenance: As this is a foreign pull request (from a fork), the diff is supplied below (as it won't show otherwise due to GitHub magic): diff --git a/website/_docs/gettingstarted/best_practices.md b/website/_docs/gettingstarted/best_practices.md index 191530e355..118a3889b0 100644 --- a/website/_docs/gettingstarted/best_practices.md +++ b/website/_docs/gettingstarted/best_practices.md @@ -24,4 +24,19 @@ List of articles about Kylin best practices contributed by community. Some of th * [Kylin and Qlik Integration](https://github.com/albertoRamon/Kylin/tree/master/KylinWithQlik) (Thanks to [Ramón Portolés, Alberto](https://www.linkedin.com/in/alberto-ramon-portoles-a02b523b)) -* [How to use Hue with Kylin](https://github.com/albertoRamon/Kylin/tree/master/KylinWithHue) (Thanks to [Ramón Portolés, Alberto](https://www.linkedin.com/in/alberto-ramon-portoles-a02b523b)) \ No newline at end of file +* [How to use Hue with Kylin](https://github.com/albertoRamon/Kylin/tree/master/KylinWithHue) (Thanks to [Ramón Portolés, Alberto](https://www.linkedin.com/in/alberto-ramon-portoles-a02b523b)) + + +**Here are online tutorials for self-studying Kylin:** + +- Free Kylin tutorial (registration needed), from the core developers, in Chinese: +[https://www.chinahadoop.cn/search?q=kylin](https://www.chinahadoop.cn/search?q=kylin) + +- A paid Kylin tutorial on Udemy, in English: +[https://www.udemy.com/apache-kylin-implementing-olap-on-the-hadoop-platform](https://www.udemy.com/apache-kylin-implementing-olap-on-the-hadoop-platform) + +- Tutorial of Kylin on Tableau, in Spanish: +[https://www.youtube.com/watch?v=k6fBw8yA1NI](https://www.youtube.com/watch?v=k6fBw8yA1NI) + +Besides, there is also some video of Kylin on the conferences; You can search for them on Youtube: +[https://www.youtube.com/results?search_query=Apache+Kylin](https://www.youtube.com/results?search_query=Apache+Kylin) diff --git a/website/_docs/gettingstarted/faq.md b/website/_docs/gettingstarted/faq.md index def985146e..0540857863 100644 --- a/website/_docs/gettingstarted/faq.md +++ b/website/_docs/gettingstarted/faq.md @@ -135,6 +135,10 @@ When the new cube is built, please disable or drop the old one. If you can accept the absence of new dimensions for historical data, you can build the new cube since the end time of the old cube. And then create a hybrid model over the old and new cube. + How to solve the data security problem of Tableau connection client? + + * Kylin's ACL control can solve this problem. Different analysts have the authority to work on different projects for Kylin. When you create a Kylin ODBC DSN, you can map different links to different analyst accounts. + The query result is not exactly matched with that in Hive, what's the possible reason? * Possible reasons: diff --git a/website/_docs/howto/howto_backup_metadata.cn.md b/website/_docs/howto/howto_backup_metadata.cn.md index 66ab3ed391..56db4c8b08 100644 --- a/website/_docs/howto/howto_backup_metadata.cn.md +++ b/website/_docs/howto/howto_backup_metadata.cn.md @@ -73,14 +73,14 @@ cd $KYLIN_HOME ## 从 metadata store 清理无用的资源 随着运行时间增长,类似字典、表快照的资源变得没有用(cube segment被丢弃或者合并了),但是它们依旧占用空间,你可以运行命令来找到并清除它们: -首先,运行一个检查,这是安全的因为它不会改变任何东西: +首先,运行一个检查,这是安全的因为它不会改变任何东西,通过添加 "--jobThreshold 30(默认值,您可以改为任何数字)" 参数,您可以设置要保留的 metadata resource 天数: {% highlight Groff markup %} -./bin/metastore.sh clean +./bin/metastore.sh clean --jobThreshold 30 {% endhighlight %} 将要被删除的资源会被列出来: 接下来,增加 “--delete true” 参数来清理这些资源;在这之前,你应该确保已经备份 metadata store: {% highlight Groff markup %} -./bin/metastore.sh clean --delete true +./bin/metastore.sh clean --delete true --jobThreshold 30 {% endhighlight %} diff --git a/website/_docs/howto/howto_backup_metadata.md b/website/_docs/howto/howto_backup_metadata.md index bbc0d0f5ff..deaaa70147 100644 --- a/website/_docs/howto/howto_backup_metadata.md +++ b/website/_docs/howto/howto_backup_metadata.md @@ -74,14 +74,14 @@ When developing/debugging Kylin, typically you have a dev machine with an IDE, a ## Cleanup unused resources from metadata store As time goes on, some resources like dictionary, table snapshots became useless (as the cube segment be dropped or merged), but they still take space there; You can run command to find and cleanup them from metadata store: -Firstly, run a check, this is safe as it will not change anything: +Firstly, run a check, this is safe as it will not change anything, you can set the number of days to keep metadata resource by adding the "--jobThreshold 30(default, you can change to any number)" option: {% highlight Groff markup %} -./bin/metastore.sh clean +./bin/metastore.sh clean --job
[jira] [Commented] (KYLIN-3662) exception message "Cannot find project '%s'." should be formated
[ https://issues.apache.org/jira/browse/KYLIN-3662?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16671628#comment-16671628 ] ASF subversion and git services commented on KYLIN-3662: Commit 2793a94331d85a35ca7144a4584dd6ac0e4fb901 in kylin's branch refs/heads/master from chitin [ https://gitbox.apache.org/repos/asf?p=kylin.git;h=2793a94 ] KYLIN-3662 > exception message "Cannot find project '%s'." should be formated > > > Key: KYLIN-3662 > URL: https://issues.apache.org/jira/browse/KYLIN-3662 > Project: Kylin > Issue Type: Bug >Affects Versions: v2.5.0 >Reporter: Lingang Deng >Assignee: Lingang Deng >Priority: Minor > Attachments: 0001-KYLIN-3662.patch > > > When use kylin dashboard without system cube, exception is threw as follows, > {code:java} > org.apache.kylin.rest.exception.BadRequestException: Cannot find project '%s'. > at > org.apache.kylin.rest.service.QueryService.doQueryWithCache(QueryService.java:378) > at > org.apache.kylin.rest.service.QueryService.doQueryWithCache(QueryService.java:359) > at > org.apache.kylin.rest.controller.DashboardController.getQueryMetrics(DashboardController.java:74) > {code} > The log is unfriendly to users. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Resolved] (KYLIN-3644) NumberFormatExcetion on null values when building cube with Spark
[ https://issues.apache.org/jira/browse/KYLIN-3644?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shaofeng SHI resolved KYLIN-3644. - Resolution: Fixed > NumberFormatExcetion on null values when building cube with Spark > - > > Key: KYLIN-3644 > URL: https://issues.apache.org/jira/browse/KYLIN-3644 > Project: Kylin > Issue Type: Bug > Components: Spark Engine >Affects Versions: v2.5.0 >Reporter: Hubert STEFANI >Assignee: Chao Long >Priority: Major > Fix For: v2.5.1 > > Attachments: 00_zeppelin_notebook.jpg, 01_overview_table.jpg, > 02_dimension_cube.jpg, 03_measure_cube.jpg, sortieData.csv > > > We encounter an error any time we try to build a cube with the following > steps : > * upload a csv on AWS S3 with following characteristics : the column on > which the measure will be defined has some null values (Cf. attachment) > * create a hive table with spark > * create a model on top of this table, > * create a cube with a SUM measure > * chose Spark as Engine > * Launch build > Result : The build process fails at '{color:#4383b4}#7 Step Name: > {color}Build Cube with Spark' with the following error : > > "" > 18/10/23 09:25:39 INFO scheduler.DAGScheduler: Job 0 failed: > saveAsNewAPIHadoopDataset at SparkCubingByLayer.java:253, took 7,277136 s > Exception in thread "main" java.lang.RuntimeException: error execute > org.apache.kylin.engine.spark.SparkCubingByLayer. Root cause: Job aborted due > to stage failure: Task 0 in stage 0.0 failed 4 times, most recent failure: > Lost task 0.3 in stage 0.0 (TID 4, > ip-172-31-35-113.eu-west-1.compute.internal, executor 4): > java.lang.NumberFormatException: For input string: "\N" > at > sun.misc.FloatingDecimal.readJavaFormatString(FloatingDecimal.java:2043) > at sun.misc.FloatingDecimal.parseDouble(FloatingDecimal.java:110) > at java.lang.Double.parseDouble(Double.java:538) > at > org.apache.kylin.measure.basic.DoubleIngester.valueOf(DoubleIngester.java:38) > at > org.apache.kylin.measure.basic.DoubleIngester.valueOf(DoubleIngester.java:28) > at > org.apache.kylin.engine.mr.common.BaseCuboidBuilder.buildValueOf(BaseCuboidBuilder.java:162) > at > org.apache.kylin.engine.mr.common.BaseCuboidBuilder.buildValueObjects(BaseCuboidBuilder.java:127) > at > org.apache.kylin.engine.spark.SparkCubingByLayer$EncodeBaseCuboid.call(SparkCubingByLayer.java:297) > at > org.apache.kylin.engine.spark.SparkCubingByLayer$EncodeBaseCuboid.call(SparkCubingByLayer.java:257) > at > org.apache.spark.api.java.JavaPairRDD$$anonfun$pairFunToScalaFun$1.apply(JavaPairRDD.scala:1043) > at > org.apache.spark.api.java.JavaPairRDD$$anonfun$pairFunToScalaFun$1.apply(JavaPairRDD.scala:1043) > " > Note 1: the build process is OK when run with Map/Reduce Engine. > Note 2: the error doesn't seem to be related to AWS environment. > > Sample of csv : > ID;CATEGORIE;TEL;MONTANT;MAGASIN;MATRICULE;VILLE; > 970;161;6-98-6-6-42;838.47034;Magasin_19;Client_Matricule_28;MARSEILLE; > 971;89;62-15-2-64-86;;;Client_Matricule_1;LYON; > 972;87;17-64-97-74-42;;;Client_Matricule_105;ORBEC; > 973;174;79-33-90-0-55;;Magasin_7;Client_Matricule_55;AJACCIO; > 974;172;89-95-71-6-49;141.64174;Magasin_9;Client_Matricule_105;BASTIA; > 975;83;7-27-95-28-7;897.28204;;Client_Matricule_199;AJACCIO; > 976;170;67-72-18-29-34;164.07967;Magasin_3;Client_Matricule_137;LILLE; > 977;130;14-69-4-23-27;1928.9557;Magasin_1;Client_Matricule_17;NOMNOM; > 978;43;55-91-84-98-49;891.2691;Magasin_0;Client_Matricule_22;NOMNOM; > 979;117;98-96-0-54-39;1636.3994;Magasin_9;Client_Matricule_142;MARSEILLE; > 980;163;37-55-76-53-38;;;Client_Matricule_64;NEWYORK; > 981;106;32-40-6-46-15;;Magasin_2;Client_Matricule_158;NOMNOM; > 982;56;95-60-83-89-90;;;Client_Matricule_102;NOMNOM; > 983;168;21-56-62-0-58;;;Client_Matricule_160;NOMNOM; > 984;154;92-67-37-94-60;;;Client_Matricule_137;PARIS; > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (KYLIN-3662) exception message "Cannot find project '%s'." should be formated
[ https://issues.apache.org/jira/browse/KYLIN-3662?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shaofeng SHI updated KYLIN-3662: Fix Version/s: v2.6.0 LGTM, merged in master branch; Thank you Lingang! > exception message "Cannot find project '%s'." should be formated > > > Key: KYLIN-3662 > URL: https://issues.apache.org/jira/browse/KYLIN-3662 > Project: Kylin > Issue Type: Bug >Affects Versions: v2.5.0 >Reporter: Lingang Deng >Assignee: Lingang Deng >Priority: Minor > Fix For: v2.6.0 > > Attachments: 0001-KYLIN-3662.patch > > > When use kylin dashboard without system cube, exception is threw as follows, > {code:java} > org.apache.kylin.rest.exception.BadRequestException: Cannot find project '%s'. > at > org.apache.kylin.rest.service.QueryService.doQueryWithCache(QueryService.java:378) > at > org.apache.kylin.rest.service.QueryService.doQueryWithCache(QueryService.java:359) > at > org.apache.kylin.rest.controller.DashboardController.getQueryMetrics(DashboardController.java:74) > {code} > The log is unfriendly to users. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (KYLIN-3555) Garbage collection on HBase step fails with S3 selected as storage
[ https://issues.apache.org/jira/browse/KYLIN-3555?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16672507#comment-16672507 ] Gaurav Rawat commented on KYLIN-3555: - I also faced the same with similar settings trying to use "kylin.storage.hbase.cluster-fs" as empty for now and looks to be working don't see the error in the last step now . > Garbage collection on HBase step fails with S3 selected as storage > -- > > Key: KYLIN-3555 > URL: https://issues.apache.org/jira/browse/KYLIN-3555 > Project: Kylin > Issue Type: Bug > Components: Job Engine >Affects Versions: v2.4.1 >Reporter: Iñigo Martinez >Priority: Major > Labels: build > Attachments: Screenshot from 2018-09-11 12-31-25.png > > > When building a cube with S3 selected has storage, build process fails at > latest step. > Although s3 has been defined as storage, cleanup task tries to delete from > HDFS and, of course, there is no file at HDFS. > > {code:java} > 2018-09-11 12:27:56,311 DEBUG [Scheduler 1407846257 Job > f8416975-eea6-4500-9cb7-4374f28451dc-237] > steps.HDFSPathGarbageCollectionStep:78 : Drop HDFS path on FileSystem: > s3://XXX-emr-kylin > 2018-09-11 12:27:57,364 DEBUG [Scheduler 1407846257 Job > f8416975-eea6-4500-9cb7-4374f28451dc-237] > steps.HDFSPathGarbageCollectionStep:87 : HDFS path > /kylin/kylin_metadata/kylin-f8416975-eea6-4500-9cb7-4374f28451dc/plataforma_transacciones_cubo_v1/fact_distinct_columns > is dropped. > 2018-09-11 12:27:58,104 DEBUG [Scheduler 1407846257 Job > f8416975-eea6-4500-9cb7-4374f28451dc-237] > steps.HDFSPathGarbageCollectionStep:87 : HDFS path > /kylin/kylin_metadata/kylin-f8416975-eea6-4500-9cb7-4374f28451dc/plataforma_transacciones_cubo_v1/hfile > is dropped. > 2018-09-11 12:27:58,140 DEBUG [Scheduler 1407846257 Job > f8416975-eea6-4500-9cb7-4374f28451dc-237] > steps.HDFSPathGarbageCollectionStep:78 : Drop HDFS path on FileSystem: > hdfs://ip-10-0-1-63.eu-west-1.compute.internal:8020 > 2018-09-11 12:27:58,142 DEBUG [Scheduler 1407846257 Job > f8416975-eea6-4500-9cb7-4374f28451dc-237] > steps.HDFSPathGarbageCollectionStep:90 : HDFS path > /kylin/kylin_metadata/kylin-f8416975-eea6-4500-9cb7-4374f28451dc/plataforma_transacciones_cubo_v1/fact_distinct_columns > not exists. > 2018-09-11 12:27:58,147 ERROR [Scheduler 1407846257 Job > f8416975-eea6-4500-9cb7-4374f28451dc-237] > steps.HDFSPathGarbageCollectionStep:68 : > job:f8416975-eea6-4500-9cb7-4374f28451dc-15 execute finished with exception > java.io.FileNotFoundException: File > /kylin/kylin_metadata/kylin-f8416975-eea6-4500-9cb7-4374f28451dc/plataforma_transacciones_cubo_v1 > does not exist. > at > org.apache.hadoop.hdfs.DistributedFileSystem.listStatusInternal(DistributedFileSystem.java:904) > at > org.apache.hadoop.hdfs.DistributedFileSystem.access$600(DistributedFileSystem.java:114) > at > org.apache.hadoop.hdfs.DistributedFileSystem$22.doCall(DistributedFileSystem.java:964) > at > org.apache.hadoop.hdfs.DistributedFileSystem$22.doCall(DistributedFileSystem.java:961) > at > org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81) > at > org.apache.hadoop.hdfs.DistributedFileSystem.listStatus(DistributedFileSystem.java:971) > at > org.apache.kylin.storage.hbase.steps.HDFSPathGarbageCollectionStep.dropHdfsPathOnCluster(HDFSPathGarbageCollectionStep.java:95) > at > org.apache.kylin.storage.hbase.steps.HDFSPathGarbageCollectionStep.doWork(HDFSPathGarbageCollectionStep.java:65) > at > org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:162) > at > org.apache.kylin.job.execution.DefaultChainedExecutable.doWork(DefaultChainedExecutable.java:69) > at > org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:162) > at > org.apache.kylin.job.impl.threadpool.DefaultScheduler$JobRunner.run(DefaultScheduler.java:113) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) > at java.lang.Thread.run(Thread.java:748){code} > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (KYLIN-3649) segment region count and size are not correct when using mysql as Kylin metadata storage
[ https://issues.apache.org/jira/browse/KYLIN-3649?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16672536#comment-16672536 ] ASF subversion and git services commented on KYLIN-3649: Commit aa3efa39f0a7d8cd8c4e4a487ded2ca2ea86d577 in kylin's branch refs/heads/2.5.x from shaofengshi [ https://gitbox.apache.org/repos/asf?p=kylin.git;h=aa3efa3 ] KYLIN-3649 fix ci > segment region count and size are not correct when using mysql as Kylin > metadata storage > - > > Key: KYLIN-3649 > URL: https://issues.apache.org/jira/browse/KYLIN-3649 > Project: Kylin > Issue Type: Bug > Components: Metadata >Affects Versions: v2.5.0 >Reporter: Lingang Deng >Assignee: Lingang Deng >Priority: Major > Fix For: v2.5.1 > > Attachments: 0001-KYLIN-3649.patch > > > As titles, segment region count and size are not correct. > {code:java} > if ("hbase".equals(getConfig().getMetadataUrl().getScheme())) { > try { > logger.debug("Loading HTable info " + cubeName + ", " + tableName); > // use reflection to isolate NoClassDef errors when HBase is not > available > hr = (HBaseResponse) > Class.forName("org.apache.kylin.rest.service.HBaseInfoUtil")// > .getMethod("getHBaseInfo", new Class[] { String.class, > KylinConfig.class })// > .invoke(null, tableName, this.getConfig()); > } catch (Throwable e) { > throw new IOException(e); > } > } > {code} > Judgement is not valid when using mysql as Kylin metadata storage. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (KYLIN-3649) segment region count and size are not correct when using mysql as Kylin metadata storage
[ https://issues.apache.org/jira/browse/KYLIN-3649?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16672549#comment-16672549 ] ASF subversion and git services commented on KYLIN-3649: Commit 37fdf651e5d0d986a177e17e302871400beb0c97 in kylin's branch refs/heads/master from shaofengshi [ https://gitbox.apache.org/repos/asf?p=kylin.git;h=37fdf65 ] KYLIN-3649 fix ci > segment region count and size are not correct when using mysql as Kylin > metadata storage > - > > Key: KYLIN-3649 > URL: https://issues.apache.org/jira/browse/KYLIN-3649 > Project: Kylin > Issue Type: Bug > Components: Metadata >Affects Versions: v2.5.0 >Reporter: Lingang Deng >Assignee: Lingang Deng >Priority: Major > Fix For: v2.5.1 > > Attachments: 0001-KYLIN-3649.patch > > > As titles, segment region count and size are not correct. > {code:java} > if ("hbase".equals(getConfig().getMetadataUrl().getScheme())) { > try { > logger.debug("Loading HTable info " + cubeName + ", " + tableName); > // use reflection to isolate NoClassDef errors when HBase is not > available > hr = (HBaseResponse) > Class.forName("org.apache.kylin.rest.service.HBaseInfoUtil")// > .getMethod("getHBaseInfo", new Class[] { String.class, > KylinConfig.class })// > .invoke(null, tableName, this.getConfig()); > } catch (Throwable e) { > throw new IOException(e); > } > } > {code} > Judgement is not valid when using mysql as Kylin metadata storage. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (KYLIN-3653) After kylin configured with hive data source with beeline, build failed if two jobs for creating flat table submitted at same time
[ https://issues.apache.org/jira/browse/KYLIN-3653?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shaofeng SHI updated KYLIN-3653: Fix Version/s: (was: v2.5.1) v2.6.0 > After kylin configured with hive data source with beeline, build failed if > two jobs for creating flat table submitted at same time > -- > > Key: KYLIN-3653 > URL: https://issues.apache.org/jira/browse/KYLIN-3653 > Project: Kylin > Issue Type: Bug > Components: Job Engine >Affects Versions: v2.4.1 >Reporter: nichunen >Assignee: nichunen >Priority: Major > Fix For: v2.6.0 > > > The root cause is in org.apache.kylin.common.util.HiveCmdBuilder#build > {code:java} > case BEELINE: > String tmpHqlPath = null; > StringBuilder hql = new StringBuilder(); > try { > tmpHqlPath = "/tmp/" + System.currentTimeMillis() + ".hql"; > for (String statement : statements) { > hql.append(statement); > hql.append("\n"); > } > String createFileCmd = > String.format(CREATE_HQL_TMP_FILE_TEMPLATE, tmpHqlPath, hql); > buf.append(createFileCmd); > buf.append("\n"); > buf.append(beelineShell); > buf.append(" "); > buf.append(beelineParams); > buf.append(parseProps()); > buf.append(" -f "); > buf.append(tmpHqlPath); > buf.append(";ret_code=$?;rm -f "); > buf.append(tmpHqlPath); > buf.append(";exit $ret_code"); > } finally { > if (tmpHqlPath != null && logger.isDebugEnabled()) { > logger.debug("The SQL to execute in beeline: \n" + hql); > } > } > break; > {code} > A temp hql file will be created when creating flat table, the file name is > the current time stamp. But if two jobs triggered with same time, there will > be only one file, and jobs may be failed -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (KYLIN-3663) Failed to delete project when project has more than one table
rongchuan.jin created KYLIN-3663: Summary: Failed to delete project when project has more than one table Key: KYLIN-3663 URL: https://issues.apache.org/jira/browse/KYLIN-3663 Project: Kylin Issue Type: Bug Components: Metadata Affects Versions: v2.5.0 Environment: MacOSX,JDK1.8+ Reporter: rongchuan.jin Fix For: v2.6.0 When I drop a project with more than one table.(When there is only one table,it works well) It comes out an error: {code:java} org.apache.kylin.rest.exception.InternalErrorException: Failed to delete project. Caused by: null at org.apache.kylin.rest.controller.ProjectController.deleteProject(ProjectController.java:199) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.springframework.web.method.support.InvocableHandlerMethod.doInvoke(InvocableHandlerMethod.java:205) at org.springframework.web.method.support.InvocableHandlerMethod.invokeForRequest(InvocableHandlerMethod.java:133) at org.springframework.web.servlet.mvc.method.annotation.ServletInvocableHandlerMethod.invokeAndHandle(ServletInvocableHandlerMethod.java:97) at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.invokeHandlerMethod(RequestMappingHandlerAdapter.java:827) at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.handleInternal(RequestMappingHandlerAdapter.java:738) at org.springframework.web.servlet.mvc.method.AbstractHandlerMethodAdapter.handle(AbstractHandlerMethodAdapter.java:85) at org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:967) at org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:901) at org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:970) at org.springframework.web.servlet.FrameworkServlet.doDelete(FrameworkServlet.java:894) at javax.servlet.http.HttpServlet.service(HttpServlet.java:656) at org.springframework.web.servlet.FrameworkServlet.service(FrameworkServlet.java:846) at javax.servlet.http.HttpServlet.service(HttpServlet.java:731) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:303) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208) at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:317) at org.springframework.security.web.access.intercept.FilterSecurityInterceptor.invoke(FilterSecurityInterceptor.java:127) at org.springframework.security.web.access.intercept.FilterSecurityInterceptor.doFilter(FilterSecurityInterceptor.java:91) at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331) at org.springframework.security.web.access.ExceptionTranslationFilter.doFilter(ExceptionTranslationFilter.java:114) at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331) at org.springframework.security.web.session.SessionManagementFilter.doFilter(SessionManagementFilter.java:137) at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331) at org.springframework.security.web.authentication.AnonymousAuthenticationFilter.doFilter(AnonymousAuthenticationFilter.java:111) at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331) at org.springframework.security.web.servletapi.SecurityContextHolderAwareRequestFilter.doFilter(SecurityContextHolderAwareRequestFilter.java:170) at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331) at org.springframework.security.web.savedrequest.RequestCacheAwareFilter.doFilter(RequestCacheAwareFilter.java:63) at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331) at org.springframework.security.web.authentication.www.BasicAuthenticationFilter.doFilterInternal(BasicAuthenticationFilter.java:158) at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107) at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331) at org.springframework.security.web.authentication.AbstractAuthenticationProcessingFilter.doFilter(AbstractAuthenticationProcessingFilter.java:200) at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331) at org.springframework.security.web.authentication.logout.LogoutFilter.doFilter(
[jira] [Assigned] (KYLIN-3663) Failed to delete project when project has more than one table
[ https://issues.apache.org/jira/browse/KYLIN-3663?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] rongchuan.jin reassigned KYLIN-3663: Assignee: rongchuan.jin > Failed to delete project when project has more than one table > - > > Key: KYLIN-3663 > URL: https://issues.apache.org/jira/browse/KYLIN-3663 > Project: Kylin > Issue Type: Bug > Components: Metadata >Affects Versions: v2.5.0 > Environment: MacOSX,JDK1.8+ >Reporter: rongchuan.jin >Assignee: rongchuan.jin >Priority: Minor > Fix For: v2.6.0 > > > When I drop a project with more than one table.(When there is only one > table,it works well) > It comes out an error: > {code:java} > org.apache.kylin.rest.exception.InternalErrorException: Failed to delete > project. Caused by: null > at > org.apache.kylin.rest.controller.ProjectController.deleteProject(ProjectController.java:199) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:498) > at > org.springframework.web.method.support.InvocableHandlerMethod.doInvoke(InvocableHandlerMethod.java:205) > at > org.springframework.web.method.support.InvocableHandlerMethod.invokeForRequest(InvocableHandlerMethod.java:133) > at > org.springframework.web.servlet.mvc.method.annotation.ServletInvocableHandlerMethod.invokeAndHandle(ServletInvocableHandlerMethod.java:97) > at > org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.invokeHandlerMethod(RequestMappingHandlerAdapter.java:827) > at > org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.handleInternal(RequestMappingHandlerAdapter.java:738) > at > org.springframework.web.servlet.mvc.method.AbstractHandlerMethodAdapter.handle(AbstractHandlerMethodAdapter.java:85) > at > org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:967) > at > org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:901) > at > org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:970) > at > org.springframework.web.servlet.FrameworkServlet.doDelete(FrameworkServlet.java:894) > at javax.servlet.http.HttpServlet.service(HttpServlet.java:656) > at > org.springframework.web.servlet.FrameworkServlet.service(FrameworkServlet.java:846) > at javax.servlet.http.HttpServlet.service(HttpServlet.java:731) > at > org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:303) > at > org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208) > at > org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:317) > at > org.springframework.security.web.access.intercept.FilterSecurityInterceptor.invoke(FilterSecurityInterceptor.java:127) > at > org.springframework.security.web.access.intercept.FilterSecurityInterceptor.doFilter(FilterSecurityInterceptor.java:91) > at > org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331) > at > org.springframework.security.web.access.ExceptionTranslationFilter.doFilter(ExceptionTranslationFilter.java:114) > at > org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331) > at > org.springframework.security.web.session.SessionManagementFilter.doFilter(SessionManagementFilter.java:137) > at > org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331) > at > org.springframework.security.web.authentication.AnonymousAuthenticationFilter.doFilter(AnonymousAuthenticationFilter.java:111) > at > org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331) > at > org.springframework.security.web.servletapi.SecurityContextHolderAwareRequestFilter.doFilter(SecurityContextHolderAwareRequestFilter.java:170) > at > org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331) > at > org.springframework.security.web.savedrequest.RequestCacheAwareFilter.doFilter(RequestCacheAwareFilter.java:63) > at > org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331) > at > org.springframework.security.web.authentication.www.BasicAuthenticationFilter.doFilterInternal(BasicAuthenticationFilter.java:158) > at > org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107) > at > org.springfra
[GitHub] woyumen4597 opened a new pull request #324: KYLIN-3663 fix deleting project error when project has more than one …
woyumen4597 opened a new pull request #324: KYLIN-3663 fix deleting project error when project has more than one … URL: https://github.com/apache/kylin/pull/324 …table This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] asfgit commented on issue #324: KYLIN-3663 fix deleting project error when project has more than one …
asfgit commented on issue #324: KYLIN-3663 fix deleting project error when project has more than one … URL: https://github.com/apache/kylin/pull/324#issuecomment-435284912 Can one of the admins verify this patch? This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (KYLIN-3663) Failed to delete project when project has more than one table
[ https://issues.apache.org/jira/browse/KYLIN-3663?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16672620#comment-16672620 ] ASF GitHub Bot commented on KYLIN-3663: --- woyumen4597 opened a new pull request #324: KYLIN-3663 fix deleting project error when project has more than one … URL: https://github.com/apache/kylin/pull/324 …table This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > Failed to delete project when project has more than one table > - > > Key: KYLIN-3663 > URL: https://issues.apache.org/jira/browse/KYLIN-3663 > Project: Kylin > Issue Type: Bug > Components: Metadata >Affects Versions: v2.5.0 > Environment: MacOSX,JDK1.8+ >Reporter: rongchuan.jin >Assignee: rongchuan.jin >Priority: Minor > Fix For: v2.6.0 > > > When I drop a project with more than one table.(When there is only one > table,it works well) > It comes out an error: > {code:java} > org.apache.kylin.rest.exception.InternalErrorException: Failed to delete > project. Caused by: null > at > org.apache.kylin.rest.controller.ProjectController.deleteProject(ProjectController.java:199) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:498) > at > org.springframework.web.method.support.InvocableHandlerMethod.doInvoke(InvocableHandlerMethod.java:205) > at > org.springframework.web.method.support.InvocableHandlerMethod.invokeForRequest(InvocableHandlerMethod.java:133) > at > org.springframework.web.servlet.mvc.method.annotation.ServletInvocableHandlerMethod.invokeAndHandle(ServletInvocableHandlerMethod.java:97) > at > org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.invokeHandlerMethod(RequestMappingHandlerAdapter.java:827) > at > org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.handleInternal(RequestMappingHandlerAdapter.java:738) > at > org.springframework.web.servlet.mvc.method.AbstractHandlerMethodAdapter.handle(AbstractHandlerMethodAdapter.java:85) > at > org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:967) > at > org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:901) > at > org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:970) > at > org.springframework.web.servlet.FrameworkServlet.doDelete(FrameworkServlet.java:894) > at javax.servlet.http.HttpServlet.service(HttpServlet.java:656) > at > org.springframework.web.servlet.FrameworkServlet.service(FrameworkServlet.java:846) > at javax.servlet.http.HttpServlet.service(HttpServlet.java:731) > at > org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:303) > at > org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208) > at > org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:317) > at > org.springframework.security.web.access.intercept.FilterSecurityInterceptor.invoke(FilterSecurityInterceptor.java:127) > at > org.springframework.security.web.access.intercept.FilterSecurityInterceptor.doFilter(FilterSecurityInterceptor.java:91) > at > org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331) > at > org.springframework.security.web.access.ExceptionTranslationFilter.doFilter(ExceptionTranslationFilter.java:114) > at > org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331) > at > org.springframework.security.web.session.SessionManagementFilter.doFilter(SessionManagementFilter.java:137) > at > org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331) > at > org.springframework.security.web.authentication.AnonymousAuthenticationFilter.doFilter(AnonymousAuthenticationFilter.java:111) > at > org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331) > at > org.springframework.security.web.servletapi.SecurityContextHolderAwareRequestFilter.doFilter(SecurityContextHolderAwareRequestFilter.java:170) > at > org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterC
[jira] [Commented] (KYLIN-3663) Failed to delete project when project has more than one table
[ https://issues.apache.org/jira/browse/KYLIN-3663?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16672621#comment-16672621 ] ASF GitHub Bot commented on KYLIN-3663: --- asfgit commented on issue #324: KYLIN-3663 fix deleting project error when project has more than one … URL: https://github.com/apache/kylin/pull/324#issuecomment-435284912 Can one of the admins verify this patch? This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > Failed to delete project when project has more than one table > - > > Key: KYLIN-3663 > URL: https://issues.apache.org/jira/browse/KYLIN-3663 > Project: Kylin > Issue Type: Bug > Components: Metadata >Affects Versions: v2.5.0 > Environment: MacOSX,JDK1.8+ >Reporter: rongchuan.jin >Assignee: rongchuan.jin >Priority: Minor > Fix For: v2.6.0 > > > When I drop a project with more than one table.(When there is only one > table,it works well) > It comes out an error: > {code:java} > org.apache.kylin.rest.exception.InternalErrorException: Failed to delete > project. Caused by: null > at > org.apache.kylin.rest.controller.ProjectController.deleteProject(ProjectController.java:199) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:498) > at > org.springframework.web.method.support.InvocableHandlerMethod.doInvoke(InvocableHandlerMethod.java:205) > at > org.springframework.web.method.support.InvocableHandlerMethod.invokeForRequest(InvocableHandlerMethod.java:133) > at > org.springframework.web.servlet.mvc.method.annotation.ServletInvocableHandlerMethod.invokeAndHandle(ServletInvocableHandlerMethod.java:97) > at > org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.invokeHandlerMethod(RequestMappingHandlerAdapter.java:827) > at > org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.handleInternal(RequestMappingHandlerAdapter.java:738) > at > org.springframework.web.servlet.mvc.method.AbstractHandlerMethodAdapter.handle(AbstractHandlerMethodAdapter.java:85) > at > org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:967) > at > org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:901) > at > org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:970) > at > org.springframework.web.servlet.FrameworkServlet.doDelete(FrameworkServlet.java:894) > at javax.servlet.http.HttpServlet.service(HttpServlet.java:656) > at > org.springframework.web.servlet.FrameworkServlet.service(FrameworkServlet.java:846) > at javax.servlet.http.HttpServlet.service(HttpServlet.java:731) > at > org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:303) > at > org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208) > at > org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:317) > at > org.springframework.security.web.access.intercept.FilterSecurityInterceptor.invoke(FilterSecurityInterceptor.java:127) > at > org.springframework.security.web.access.intercept.FilterSecurityInterceptor.doFilter(FilterSecurityInterceptor.java:91) > at > org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331) > at > org.springframework.security.web.access.ExceptionTranslationFilter.doFilter(ExceptionTranslationFilter.java:114) > at > org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331) > at > org.springframework.security.web.session.SessionManagementFilter.doFilter(SessionManagementFilter.java:137) > at > org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331) > at > org.springframework.security.web.authentication.AnonymousAuthenticationFilter.doFilter(AnonymousAuthenticationFilter.java:111) > at > org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331) > at > org.springframework.security.web.servletapi.SecurityContextHolderAwareRequestFilter.doFilter(SecurityContextHolderAwareRequestFilter.java:170) > at > org.springframework.security.web.FilterC
[jira] [Commented] (KYLIN-3388) Data may become not correct if mappers fail during the redistribute step, "distribute by rand()"
[ https://issues.apache.org/jira/browse/KYLIN-3388?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16672628#comment-16672628 ] liuzhixin commented on KYLIN-3388: -- [~Shaofengshi] Step1: Create Intermediate Flat Hive Table Step2: Redistribute intermediate table # Perhaps, Kylin can insert one rand column in the intermediate hive table for the next shard, (as default). At the same time, Kylin should support the custom column for shard. (has provided) # > Data may become not correct if mappers fail during the redistribute step, > "distribute by rand()" > > > Key: KYLIN-3388 > URL: https://issues.apache.org/jira/browse/KYLIN-3388 > Project: Kylin > Issue Type: Bug > Components: Job Engine >Reporter: Zhong Yanghong >Assignee: Chao Long >Priority: Critical > Fix For: v2.4.0, v2.3.2 > > Attachments: Hive Issue - distribute by rand().png > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] woyumen4597 closed pull request #324: KYLIN-3663 fix deleting project error when project has more than one …
woyumen4597 closed pull request #324: KYLIN-3663 fix deleting project error when project has more than one … URL: https://github.com/apache/kylin/pull/324 This is a PR merged from a forked repository. As GitHub hides the original diff on merge, it is displayed below for the sake of provenance: As this is a foreign pull request (from a fork), the diff is supplied below (as it won't show otherwise due to GitHub magic): This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (KYLIN-3663) Failed to delete project when project has more than one table
[ https://issues.apache.org/jira/browse/KYLIN-3663?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16672634#comment-16672634 ] ASF GitHub Bot commented on KYLIN-3663: --- woyumen4597 closed pull request #324: KYLIN-3663 fix deleting project error when project has more than one … URL: https://github.com/apache/kylin/pull/324 This is a PR merged from a forked repository. As GitHub hides the original diff on merge, it is displayed below for the sake of provenance: As this is a foreign pull request (from a fork), the diff is supplied below (as it won't show otherwise due to GitHub magic): This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > Failed to delete project when project has more than one table > - > > Key: KYLIN-3663 > URL: https://issues.apache.org/jira/browse/KYLIN-3663 > Project: Kylin > Issue Type: Bug > Components: Metadata >Affects Versions: v2.5.0 > Environment: MacOSX,JDK1.8+ >Reporter: rongchuan.jin >Assignee: rongchuan.jin >Priority: Minor > Fix For: v2.6.0 > > > When I drop a project with more than one table.(When there is only one > table,it works well) > It comes out an error: > {code:java} > org.apache.kylin.rest.exception.InternalErrorException: Failed to delete > project. Caused by: null > at > org.apache.kylin.rest.controller.ProjectController.deleteProject(ProjectController.java:199) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:498) > at > org.springframework.web.method.support.InvocableHandlerMethod.doInvoke(InvocableHandlerMethod.java:205) > at > org.springframework.web.method.support.InvocableHandlerMethod.invokeForRequest(InvocableHandlerMethod.java:133) > at > org.springframework.web.servlet.mvc.method.annotation.ServletInvocableHandlerMethod.invokeAndHandle(ServletInvocableHandlerMethod.java:97) > at > org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.invokeHandlerMethod(RequestMappingHandlerAdapter.java:827) > at > org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.handleInternal(RequestMappingHandlerAdapter.java:738) > at > org.springframework.web.servlet.mvc.method.AbstractHandlerMethodAdapter.handle(AbstractHandlerMethodAdapter.java:85) > at > org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:967) > at > org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:901) > at > org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:970) > at > org.springframework.web.servlet.FrameworkServlet.doDelete(FrameworkServlet.java:894) > at javax.servlet.http.HttpServlet.service(HttpServlet.java:656) > at > org.springframework.web.servlet.FrameworkServlet.service(FrameworkServlet.java:846) > at javax.servlet.http.HttpServlet.service(HttpServlet.java:731) > at > org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:303) > at > org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208) > at > org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:317) > at > org.springframework.security.web.access.intercept.FilterSecurityInterceptor.invoke(FilterSecurityInterceptor.java:127) > at > org.springframework.security.web.access.intercept.FilterSecurityInterceptor.doFilter(FilterSecurityInterceptor.java:91) > at > org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331) > at > org.springframework.security.web.access.ExceptionTranslationFilter.doFilter(ExceptionTranslationFilter.java:114) > at > org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331) > at > org.springframework.security.web.session.SessionManagementFilter.doFilter(SessionManagementFilter.java:137) > at > org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331) > at > org.springframework.security.web.authentication.AnonymousAuthenticationFilter.doFilter(AnonymousAuthenticationFilter.java:111) > at > org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainP
[GitHub] woyumen4597 opened a new pull request #325: KYLIN-3663 fix deleting project error when project has more than one …
woyumen4597 opened a new pull request #325: KYLIN-3663 fix deleting project error when project has more than one … URL: https://github.com/apache/kylin/pull/325 …table This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] asfgit commented on issue #325: KYLIN-3663 fix deleting project error when project has more than one …
asfgit commented on issue #325: KYLIN-3663 fix deleting project error when project has more than one … URL: https://github.com/apache/kylin/pull/325#issuecomment-435286399 Can one of the admins verify this patch? This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (KYLIN-3663) Failed to delete project when project has more than one table
[ https://issues.apache.org/jira/browse/KYLIN-3663?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16672641#comment-16672641 ] ASF GitHub Bot commented on KYLIN-3663: --- woyumen4597 opened a new pull request #325: KYLIN-3663 fix deleting project error when project has more than one … URL: https://github.com/apache/kylin/pull/325 …table This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > Failed to delete project when project has more than one table > - > > Key: KYLIN-3663 > URL: https://issues.apache.org/jira/browse/KYLIN-3663 > Project: Kylin > Issue Type: Bug > Components: Metadata >Affects Versions: v2.5.0 > Environment: MacOSX,JDK1.8+ >Reporter: rongchuan.jin >Assignee: rongchuan.jin >Priority: Minor > Fix For: v2.6.0 > > > When I drop a project with more than one table.(When there is only one > table,it works well) > It comes out an error: > {code:java} > org.apache.kylin.rest.exception.InternalErrorException: Failed to delete > project. Caused by: null > at > org.apache.kylin.rest.controller.ProjectController.deleteProject(ProjectController.java:199) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:498) > at > org.springframework.web.method.support.InvocableHandlerMethod.doInvoke(InvocableHandlerMethod.java:205) > at > org.springframework.web.method.support.InvocableHandlerMethod.invokeForRequest(InvocableHandlerMethod.java:133) > at > org.springframework.web.servlet.mvc.method.annotation.ServletInvocableHandlerMethod.invokeAndHandle(ServletInvocableHandlerMethod.java:97) > at > org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.invokeHandlerMethod(RequestMappingHandlerAdapter.java:827) > at > org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.handleInternal(RequestMappingHandlerAdapter.java:738) > at > org.springframework.web.servlet.mvc.method.AbstractHandlerMethodAdapter.handle(AbstractHandlerMethodAdapter.java:85) > at > org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:967) > at > org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:901) > at > org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:970) > at > org.springframework.web.servlet.FrameworkServlet.doDelete(FrameworkServlet.java:894) > at javax.servlet.http.HttpServlet.service(HttpServlet.java:656) > at > org.springframework.web.servlet.FrameworkServlet.service(FrameworkServlet.java:846) > at javax.servlet.http.HttpServlet.service(HttpServlet.java:731) > at > org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:303) > at > org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208) > at > org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:317) > at > org.springframework.security.web.access.intercept.FilterSecurityInterceptor.invoke(FilterSecurityInterceptor.java:127) > at > org.springframework.security.web.access.intercept.FilterSecurityInterceptor.doFilter(FilterSecurityInterceptor.java:91) > at > org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331) > at > org.springframework.security.web.access.ExceptionTranslationFilter.doFilter(ExceptionTranslationFilter.java:114) > at > org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331) > at > org.springframework.security.web.session.SessionManagementFilter.doFilter(SessionManagementFilter.java:137) > at > org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331) > at > org.springframework.security.web.authentication.AnonymousAuthenticationFilter.doFilter(AnonymousAuthenticationFilter.java:111) > at > org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331) > at > org.springframework.security.web.servletapi.SecurityContextHolderAwareRequestFilter.doFilter(SecurityContextHolderAwareRequestFilter.java:170) > at > org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterC
[jira] [Commented] (KYLIN-3663) Failed to delete project when project has more than one table
[ https://issues.apache.org/jira/browse/KYLIN-3663?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16672643#comment-16672643 ] ASF GitHub Bot commented on KYLIN-3663: --- asfgit commented on issue #325: KYLIN-3663 fix deleting project error when project has more than one … URL: https://github.com/apache/kylin/pull/325#issuecomment-435286399 Can one of the admins verify this patch? This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > Failed to delete project when project has more than one table > - > > Key: KYLIN-3663 > URL: https://issues.apache.org/jira/browse/KYLIN-3663 > Project: Kylin > Issue Type: Bug > Components: Metadata >Affects Versions: v2.5.0 > Environment: MacOSX,JDK1.8+ >Reporter: rongchuan.jin >Assignee: rongchuan.jin >Priority: Minor > Fix For: v2.6.0 > > > When I drop a project with more than one table.(When there is only one > table,it works well) > It comes out an error: > {code:java} > org.apache.kylin.rest.exception.InternalErrorException: Failed to delete > project. Caused by: null > at > org.apache.kylin.rest.controller.ProjectController.deleteProject(ProjectController.java:199) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:498) > at > org.springframework.web.method.support.InvocableHandlerMethod.doInvoke(InvocableHandlerMethod.java:205) > at > org.springframework.web.method.support.InvocableHandlerMethod.invokeForRequest(InvocableHandlerMethod.java:133) > at > org.springframework.web.servlet.mvc.method.annotation.ServletInvocableHandlerMethod.invokeAndHandle(ServletInvocableHandlerMethod.java:97) > at > org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.invokeHandlerMethod(RequestMappingHandlerAdapter.java:827) > at > org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.handleInternal(RequestMappingHandlerAdapter.java:738) > at > org.springframework.web.servlet.mvc.method.AbstractHandlerMethodAdapter.handle(AbstractHandlerMethodAdapter.java:85) > at > org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:967) > at > org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:901) > at > org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:970) > at > org.springframework.web.servlet.FrameworkServlet.doDelete(FrameworkServlet.java:894) > at javax.servlet.http.HttpServlet.service(HttpServlet.java:656) > at > org.springframework.web.servlet.FrameworkServlet.service(FrameworkServlet.java:846) > at javax.servlet.http.HttpServlet.service(HttpServlet.java:731) > at > org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:303) > at > org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208) > at > org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:317) > at > org.springframework.security.web.access.intercept.FilterSecurityInterceptor.invoke(FilterSecurityInterceptor.java:127) > at > org.springframework.security.web.access.intercept.FilterSecurityInterceptor.doFilter(FilterSecurityInterceptor.java:91) > at > org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331) > at > org.springframework.security.web.access.ExceptionTranslationFilter.doFilter(ExceptionTranslationFilter.java:114) > at > org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331) > at > org.springframework.security.web.session.SessionManagementFilter.doFilter(SessionManagementFilter.java:137) > at > org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331) > at > org.springframework.security.web.authentication.AnonymousAuthenticationFilter.doFilter(AnonymousAuthenticationFilter.java:111) > at > org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331) > at > org.springframework.security.web.servletapi.SecurityContextHolderAwareRequestFilter.doFilter(SecurityContextHolderAwareRequestFilter.java:170) > at > org.springframework.security.web.FilterC