[GitHub] [carbondata] qiuchenjian commented on a change in pull request #3084: [CARBONDATA-3258] Add more test case for mv datamap
qiuchenjian commented on a change in pull request #3084: [CARBONDATA-3258] Add
more test case for mv datamap
URL: https://github.com/apache/carbondata/pull/3084#discussion_r289583768
##
File path:
datamap/mv/core/src/test/scala/org/apache/carbondata/mv/rewrite/MVValidFunctionTest.scala
##
@@ -0,0 +1,70 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ *http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.carbondata.mv.rewrite
+
+import org.apache.spark.sql.Row
+import org.apache.spark.sql.test.util.QueryTest
+import org.scalatest.BeforeAndAfterAll
+
+class MVValidFunctionTest extends QueryTest with BeforeAndAfterAll {
Review comment:
done
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [carbondata] qiuchenjian commented on a change in pull request #3084: [CARBONDATA-3258] Add more test case for mv datamap
qiuchenjian commented on a change in pull request #3084: [CARBONDATA-3258] Add
more test case for mv datamap
URL: https://github.com/apache/carbondata/pull/3084#discussion_r289583566
##
File path:
datamap/mv/core/src/test/scala/org/apache/carbondata/mv/rewrite/MVInvalidTestCase.scala
##
@@ -0,0 +1,47 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ *http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.carbondata.mv.rewrite
+
+import org.apache.spark.sql.test.util.QueryTest
+import org.scalatest.BeforeAndAfterAll
+
+class MVInvalidTestCase extends QueryTest with BeforeAndAfterAll {
Review comment:
I think we should put different scenes in different class, It's ok
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [carbondata] qiuchenjian commented on a change in pull request #3084: [CARBONDATA-3258] Add more test case for mv datamap
qiuchenjian commented on a change in pull request #3084: [CARBONDATA-3258] Add
more test case for mv datamap
URL: https://github.com/apache/carbondata/pull/3084#discussion_r289583498
##
File path:
datamap/mv/core/src/test/scala/org/apache/carbondata/mv/rewrite/MVValidFunctionTest.scala
##
@@ -0,0 +1,70 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ *http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.carbondata.mv.rewrite
+
+import org.apache.spark.sql.Row
+import org.apache.spark.sql.test.util.QueryTest
+import org.scalatest.BeforeAndAfterAll
+
+class MVValidFunctionTest extends QueryTest with BeforeAndAfterAll {
+
+ override def beforeAll(): Unit = {
+drop
+sql("create table main_table (name string,age int,height int) stored by
'carbondata'")
+sql("create table dim_table (name string,age int,height int) stored by
'carbondata'")
+sql("create table sdr_table (name varchar(20),score int) stored by
'carbondata'")
+ }
+
+ def drop() {
+sql("drop table if exists main_table")
+sql("drop datamap if exists main_table_mv")
Review comment:
same as above
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [carbondata] qiuchenjian commented on a change in pull request #3084: [CARBONDATA-3258] Add more test case for mv datamap
qiuchenjian commented on a change in pull request #3084: [CARBONDATA-3258] Add
more test case for mv datamap
URL: https://github.com/apache/carbondata/pull/3084#discussion_r289583399
##
File path:
datamap/mv/core/src/test/scala/org/apache/carbondata/mv/rewrite/MVExceptionTestCase.scala
##
@@ -0,0 +1,55 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ *http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.carbondata.mv.rewrite
+
+import
org.apache.carbondata.common.exceptions.sql.MalformedDataMapCommandException
+import org.apache.spark.sql.catalyst.analysis.NoSuchTableException
+import org.apache.spark.sql.test.util.QueryTest
+import org.scalatest.BeforeAndAfterAll
+
+class MVExceptionTestCase extends QueryTest with BeforeAndAfterAll {
+ override def beforeAll: Unit = {
+drop()
+sql("create table main_table (name string,age int,height int) stored by
'carbondata'")
+ }
+
+ test("test mv no base table") {
+val ex = intercept[NoSuchTableException] {
+ sql("create datamap main_table_mv on table main_table_error using 'mv'
as select sum(age),name from main_table group by name")
+}
+assertResult("Table or view 'main_table_error' not found in database
'default';")(ex.getMessage())
+ }
+
+ test("test mv reduplicate mv table") {
+val ex = intercept[MalformedDataMapCommandException] {
+ sql("create datamap main_table_mv1 on table main_table using 'mv' as
select sum(age),name from main_table group by name")
+ sql("create datamap main_table_mv1 on table main_table using 'mv' as
select sum(age),name from main_table group by name")
+}
+assertResult("DataMap with name main_table_mv1 already exists in
storage")(ex.getMessage)
+ }
+
+ def drop(): Unit = {
+sql("drop table IF EXISTS main_table")
+sql("drop table if exists main_table_error")
+sql("drop datamap if exists main_table_mv")
Review comment:
No, it's for this datamap affect other table, not this testcase's main
table,thus it is needed
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [carbondata] qiuchenjian commented on a change in pull request #3084: [CARBONDATA-3258] Add more test case for mv datamap
qiuchenjian commented on a change in pull request #3084: [CARBONDATA-3258] Add
more test case for mv datamap
URL: https://github.com/apache/carbondata/pull/3084#discussion_r289277090
##
File path:
datamap/mv/core/src/test/scala/org/apache/carbondata/mv/rewrite/MVValidFunctionTest.scala
##
@@ -0,0 +1,70 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ *http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.carbondata.mv.rewrite
+
+import org.apache.spark.sql.Row
+import org.apache.spark.sql.test.util.QueryTest
+import org.scalatest.BeforeAndAfterAll
+
+class MVValidFunctionTest extends QueryTest with BeforeAndAfterAll {
+
+ override def beforeAll(): Unit = {
+drop
+sql("create table main_table (name string,age int,height int) stored by
'carbondata'")
+sql("create table dim_table (name string,age int,height int) stored by
'carbondata'")
+sql("create table sdr_table (name varchar(20),score int) stored by
'carbondata'")
+ }
+
+ def drop() {
+sql("drop table if exists main_table")
+sql("drop datamap if exists main_table_mv")
+sql("drop table if exists dim_table")
+sql("drop table if exists sdr_table")
+sql("drop datamap if exists main_table_mv1")
+ }
+
+ test("test mv no filter and query with filter") {
+val querySQL = "select sum(age),name from main_table where name = 'tom'
group by name"
+sql("insert into main_table select 'tom',20,170")
+sql("insert into main_table select 'lily',30,160")
+sql("create datamap main_table_mv on table main_table using 'mv' as select
sum(age),name from main_table group by name")
+sql("rebuild datamap main_table_mv")
+assert(TestUtil.verifyMVDataMap(sql(querySQL).queryExecution.analyzed,
"main_table_mv"))
+checkAnswer(sql(querySQL), Seq(Row(20,"tom")))
+ }
+
+ test("test mv rebuild twice and varchar string") {
+val querySQL = "select A.sum_score,A.name from (select sum(score) as
sum_score,age,sdr.name as name from sdr_table sdr " +
+ "left join dim_table dim on sdr.name = dim.name where sdr.name in
('tom','lily') group by sdr.name,age) A where name = 'tom'"
+sql("insert into dim_table select 'tom',20,170")
+sql("insert into dim_table select 'lily',30,160")
+sql("create datamap main_table_mv1 using 'mv' " +
+ "as select count(score),sum(score),age,sdr.name from sdr_table sdr left
join dim_table dim on sdr.name = dim.name group by sdr.name,age")
Review comment:
done
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [carbondata] qiuchenjian commented on a change in pull request #3084: [CARBONDATA-3258] Add more test case for mv datamap
qiuchenjian commented on a change in pull request #3084: [CARBONDATA-3258] Add
more test case for mv datamap
URL: https://github.com/apache/carbondata/pull/3084#discussion_r288853507
##
File path:
datamap/mv/core/src/test/scala/org/apache/carbondata/mv/rewrite/MVUtils.scala
##
@@ -0,0 +1,29 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ *http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.carbondata.mv.rewrite
+
+import org.apache.spark.sql.catalyst.plans.logical.LogicalPlan
+import org.apache.spark.sql.execution.datasources.LogicalRelation
+
+object MVUtils {
+ def verifyMVDataMap(logicalPlan: LogicalPlan, dataMapName: String): Boolean
= {
Review comment:
@ravipesala The new method ```isValidSelect``` of ```MVHelper``` cause
failure test case ```mv rebuild twice and varchar string```
Can you check it?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [carbondata] qiuchenjian commented on a change in pull request #3084: [CARBONDATA-3258] Add more test case for mv datamap
qiuchenjian commented on a change in pull request #3084: [CARBONDATA-3258] Add
more test case for mv datamap
URL: https://github.com/apache/carbondata/pull/3084#discussion_r288853507
##
File path:
datamap/mv/core/src/test/scala/org/apache/carbondata/mv/rewrite/MVUtils.scala
##
@@ -0,0 +1,29 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ *http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.carbondata.mv.rewrite
+
+import org.apache.spark.sql.catalyst.plans.logical.LogicalPlan
+import org.apache.spark.sql.execution.datasources.LogicalRelation
+
+object MVUtils {
+ def verifyMVDataMap(logicalPlan: LogicalPlan, dataMapName: String): Boolean
= {
Review comment:
@ravipesala The new method ```isValidSelect``` of ```MVHelper``` cause
failure test case ```mv rebuild twice and varchar string```
Can you check it?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [carbondata] qiuchenjian commented on a change in pull request #3084: [CARBONDATA-3258] Add more test case for mv datamap
qiuchenjian commented on a change in pull request #3084: [CARBONDATA-3258] Add
more test case for mv datamap
URL: https://github.com/apache/carbondata/pull/3084#discussion_r288828425
##
File path:
datamap/mv/core/src/test/scala/org/apache/carbondata/mv/rewrite/MVUtils.scala
##
@@ -0,0 +1,29 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ *http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.carbondata.mv.rewrite
+
+import org.apache.spark.sql.catalyst.plans.logical.LogicalPlan
+import org.apache.spark.sql.execution.datasources.LogicalRelation
+
+object MVUtils {
+ def verifyMVDataMap(logicalPlan: LogicalPlan, dataMapName: String): Boolean
= {
Review comment:
done
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
