[GitHub] [carbondata] CarbonDataQA1 commented on issue #3509: [CARBONDATA-3618] Update query should throw exception if key has more than one value
CarbonDataQA1 commented on issue #3509: [CARBONDATA-3618] Update query should throw exception if key has more than one value URL: https://github.com/apache/carbondata/pull/3509#issuecomment-568860815 Build Success with Spark 2.2.1, Please check CI http://121.244.95.60:12545/job/ApacheCarbonPRBuilder2.2/1285/ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [carbondata] CarbonDataQA1 commented on issue #3509: [CARBONDATA-3618] Update query should throw exception if key has more than one value
CarbonDataQA1 commented on issue #3509: [CARBONDATA-3618] Update query should throw exception if key has more than one value URL: https://github.com/apache/carbondata/pull/3509#issuecomment-568852708 Build Success with Spark 2.1.0, Please check CI http://121.244.95.60:12545/job/ApacheCarbonPRBuilder2.1/1275/ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [carbondata] ajantha-bhat commented on a change in pull request #3509: [CARBONDATA-3618] Update query should throw exception if key has more than one value
ajantha-bhat commented on a change in pull request #3509: [CARBONDATA-3618] Update query should throw exception if key has more than one value URL: https://github.com/apache/carbondata/pull/3509#discussion_r361265766 ## File path: docs/configuration-parameters.md ## @@ -144,6 +144,7 @@ This section provides the details of all the configurations required for the Car | carbon.heap.memory.pooling.threshold.bytes | 1048576 | CarbonData supports unsafe operations of Java to avoid GC overhead for certain operations. Using unsafe, memory can be allocated on Java Heap or off heap. This configuration controls the allocation mechanism on Java HEAP. If the heap memory allocations of the given size is greater or equal than this value,it should go through the pooling mechanism. But if set this size to -1, it should not go through the pooling mechanism. Default value is 1048576(1MB, the same as Spark). Value to be specified in bytes. | | carbon.push.rowfilters.for.vector | false | When enabled complete row filters will be handled by carbon in case of vector. If it is disabled then only page level pruning will be done by carbon and row level filtering will be done by spark for vector. And also there are scan optimizations in carbon to avoid multiple data copies when this parameter is set to false. There is no change in flow for non-vector based queries. | | carbon.query.prefetch.enable | true | By default this property is true, so prefetch is used in query to read next blocklet asynchronously in other thread while processing current blocklet in main thread. This can help to reduce CPU idle time. Setting this property false will disable this prefetch feature in query. | +| carbon.update.validate.key.to.value.mapping | true | By default this property is true, so update will validate key value mapping. This validation might have slight degrade in performance of update query. If user knows that key value mapping is correct, can disable this validation for better update performance by setting this property to false. | Review comment: ok. moved This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [carbondata] jackylk commented on a change in pull request #3509: [CARBONDATA-3618] Update query should throw exception if key has more than one value
jackylk commented on a change in pull request #3509: [CARBONDATA-3618] Update query should throw exception if key has more than one value URL: https://github.com/apache/carbondata/pull/3509#discussion_r361263439 ## File path: docs/configuration-parameters.md ## @@ -144,6 +144,7 @@ This section provides the details of all the configurations required for the Car | carbon.heap.memory.pooling.threshold.bytes | 1048576 | CarbonData supports unsafe operations of Java to avoid GC overhead for certain operations. Using unsafe, memory can be allocated on Java Heap or off heap. This configuration controls the allocation mechanism on Java HEAP. If the heap memory allocations of the given size is greater or equal than this value,it should go through the pooling mechanism. But if set this size to -1, it should not go through the pooling mechanism. Default value is 1048576(1MB, the same as Spark). Value to be specified in bytes. | | carbon.push.rowfilters.for.vector | false | When enabled complete row filters will be handled by carbon in case of vector. If it is disabled then only page level pruning will be done by carbon and row level filtering will be done by spark for vector. And also there are scan optimizations in carbon to avoid multiple data copies when this parameter is set to false. There is no change in flow for non-vector based queries. | | carbon.query.prefetch.enable | true | By default this property is true, so prefetch is used in query to read next blocklet asynchronously in other thread while processing current blocklet in main thread. This can help to reduce CPU idle time. Setting this property false will disable this prefetch feature in query. | +| carbon.update.validate.key.to.value.mapping | true | By default this property is true, so update will validate key value mapping. This validation might have slight degrade in performance of update query. If user knows that key value mapping is correct, can disable this validation for better update performance by setting this property to false. | Review comment: This should be moved to mutation related configuration This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [carbondata] jackylk closed pull request #3531: [WIP]Remove unused declarations
jackylk closed pull request #3531: [WIP]Remove unused declarations URL: https://github.com/apache/carbondata/pull/3531 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [carbondata] CarbonDataQA1 commented on issue #3531: [WIP]Remove unused declarations
CarbonDataQA1 commented on issue #3531: [WIP]Remove unused declarations URL: https://github.com/apache/carbondata/pull/3531#issuecomment-568846630 Build Success with Spark 2.3.4, Please check CI http://121.244.95.60:12545/job/ApacheCarbonPRBuilder2.3/1295/ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [carbondata] CarbonDataQA1 commented on issue #3531: [WIP]Remove unused declarations
CarbonDataQA1 commented on issue #3531: [WIP]Remove unused declarations URL: https://github.com/apache/carbondata/pull/3531#issuecomment-568846352 Build Success with Spark 2.2.1, Please check CI http://121.244.95.60:12545/job/ApacheCarbonPRBuilder2.2/1284/ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [carbondata] CarbonDataQA1 commented on issue #3529: [CARBONDATA-3628] Support alter hive table add complex column type
CarbonDataQA1 commented on issue #3529: [CARBONDATA-3628] Support alter hive table add complex column type URL: https://github.com/apache/carbondata/pull/3529#issuecomment-568845085 Build Success with Spark 2.3.4, Please check CI http://121.244.95.60:12545/job/ApacheCarbonPRBuilder2.3/1294/ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [carbondata] CarbonDataQA1 commented on issue #3529: [CARBONDATA-3628] Support alter hive table add complex column type
CarbonDataQA1 commented on issue #3529: [CARBONDATA-3628] Support alter hive table add complex column type URL: https://github.com/apache/carbondata/pull/3529#issuecomment-568844398 Build Success with Spark 2.2.1, Please check CI http://121.244.95.60:12545/job/ApacheCarbonPRBuilder2.2/1283/ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [carbondata] CarbonDataQA1 commented on issue #3531: [WIP]Remove unused declarations
CarbonDataQA1 commented on issue #3531: [WIP]Remove unused declarations URL: https://github.com/apache/carbondata/pull/3531#issuecomment-568839533 Build Success with Spark 2.1.0, Please check CI http://121.244.95.60:12545/job/ApacheCarbonPRBuilder2.1/1274/ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [carbondata] CarbonDataQA1 commented on issue #3531: [WIP]Remove unused declarations
CarbonDataQA1 commented on issue #3531: [WIP]Remove unused declarations URL: https://github.com/apache/carbondata/pull/3531#issuecomment-568839136 Build Failed with Spark 2.3.4, Please check CI http://121.244.95.60:12545/job/ApacheCarbonPRBuilder2.3/1291/ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [carbondata] CarbonDataQA1 commented on issue #3531: [WIP]Remove unused declarations
CarbonDataQA1 commented on issue #3531: [WIP]Remove unused declarations URL: https://github.com/apache/carbondata/pull/3531#issuecomment-568838664 Build Failed with Spark 2.2.1, Please check CI http://121.244.95.60:12545/job/ApacheCarbonPRBuilder2.2/1280/ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [carbondata] CarbonDataQA1 commented on issue #3529: [CARBONDATA-3628] Support alter hive table add complex column type
CarbonDataQA1 commented on issue #3529: [CARBONDATA-3628] Support alter hive table add complex column type URL: https://github.com/apache/carbondata/pull/3529#issuecomment-568838287 Build Success with Spark 2.1.0, Please check CI http://121.244.95.60:12545/job/ApacheCarbonPRBuilder2.1/1273/ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [carbondata] IceMimosa commented on a change in pull request #3529: [CARBONDATA-3628] Support alter hive table add complex column type
IceMimosa commented on a change in pull request #3529: [CARBONDATA-3628]
Support alter hive table add complex column type
URL: https://github.com/apache/carbondata/pull/3529#discussion_r361253978
##
File path:
integration/spark-common-cluster-test/src/test/scala/org/apache/carbondata/cluster/sdv/generated/AlterTableTestCase.scala
##
@@ -1022,7 +1022,31 @@ class AlterTableTestCase extends QueryTest with
BeforeAndAfterAll {
assert(exception.getMessage.contains("Unsupported alter operation on
hive table"))
} else if (SparkUtil.isSparkVersionXandAbove("2.2")) {
sql("alter table alter_hive add columns(add string)")
- sql("insert into alter_hive select 'abc','banglore'")
+ sql("alter table alter_hive add columns (var map)")
+ sql("insert into alter_hive select
'abc','banglore',map('age','10','birth','2020')")
+ checkAnswer(
+sql("select * from alter_hive"),
+Seq(Row("abc", "banglore", Map("age" -> "10", "birth" -> "2020")))
+ )
+}
+ }
+
+ test("Alter table add column for hive partitioned table for spark version
above 2.1") {
+sql("drop table if exists alter_hive")
+sql("create table alter_hive(name string) stored as rcfile partitioned by
(dt string)")
+if (SparkUtil.isSparkVersionXandAbove("2.2")) {
+ sql("alter table alter_hive add columns(add string)")
+ sql("alter table alter_hive add columns (var map)")
Review comment:
It does not work with default STRUCT Datatype (struct<>)
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [carbondata] IceMimosa commented on a change in pull request #3529: [CARBONDATA-3628] Support alter hive table add complex column type
IceMimosa commented on a change in pull request #3529: [CARBONDATA-3628] Support alter hive table add complex column type URL: https://github.com/apache/carbondata/pull/3529#discussion_r361253996 ## File path: integration/spark-common/src/main/scala/org/apache/carbondata/spark/rdd/CarbonMergerRDD.scala ## @@ -680,7 +681,7 @@ class CarbonMergerRDD[K, V]( partitionNames = null, splits = allSplits) val objectOrdering: Ordering[Object] = createOrderingForColumn(rangeColumn) -val sparkDataType = Util.convertCarbonToSparkDataType(dataType) Review comment: Done, thanks for reviewing. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [carbondata] CarbonDataQA1 commented on issue #3502: [CARBONATA-3605] Remove global dictionary feature
CarbonDataQA1 commented on issue #3502: [CARBONATA-3605] Remove global dictionary feature URL: https://github.com/apache/carbondata/pull/3502#issuecomment-568836638 Build Failed with Spark 2.2.1, Please check CI http://121.244.95.60:12545/job/ApacheCarbonPRBuilder2.2/1281/ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [carbondata] CarbonDataQA1 commented on issue #3502: [CARBONATA-3605] Remove global dictionary feature
CarbonDataQA1 commented on issue #3502: [CARBONATA-3605] Remove global dictionary feature URL: https://github.com/apache/carbondata/pull/3502#issuecomment-568836639 Build Failed with Spark 2.3.4, Please check CI http://121.244.95.60:12545/job/ApacheCarbonPRBuilder2.3/1292/ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [carbondata] CarbonDataQA1 commented on issue #3532: [CARBONDATA-3557] Write flink streaming data to partition table
CarbonDataQA1 commented on issue #3532: [CARBONDATA-3557] Write flink streaming data to partition table URL: https://github.com/apache/carbondata/pull/3532#issuecomment-568833241 Build Failed with Spark 2.1.0, Please check CI http://121.244.95.60:12545/job/ApacheCarbonPRBuilder2.1/1272/ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [carbondata] CarbonDataQA1 commented on issue #3502: [CARBONATA-3605] Remove global dictionary feature
CarbonDataQA1 commented on issue #3502: [CARBONATA-3605] Remove global dictionary feature URL: https://github.com/apache/carbondata/pull/3502#issuecomment-568833142 Build Success with Spark 2.1.0, Please check CI http://121.244.95.60:12545/job/ApacheCarbonPRBuilder2.1/1271/ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [carbondata] CarbonDataQA1 commented on issue #3532: [CARBONDATA-3557] Write flink streaming data to partition table
CarbonDataQA1 commented on issue #3532: [CARBONDATA-3557] Write flink streaming data to partition table URL: https://github.com/apache/carbondata/pull/3532#issuecomment-568833126 Build Failed with Spark 2.2.1, Please check CI http://121.244.95.60:12545/job/ApacheCarbonPRBuilder2.2/1282/ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [carbondata] CarbonDataQA1 commented on issue #3532: [CARBONDATA-3557] Write flink streaming data to partition table
CarbonDataQA1 commented on issue #3532: [CARBONDATA-3557] Write flink streaming data to partition table URL: https://github.com/apache/carbondata/pull/3532#issuecomment-568833034 Build Failed with Spark 2.3.4, Please check CI http://121.244.95.60:12545/job/ApacheCarbonPRBuilder2.3/1293/ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [carbondata] CarbonDataQA1 commented on issue #3531: [WIP]Remove unused declarations
CarbonDataQA1 commented on issue #3531: [WIP]Remove unused declarations URL: https://github.com/apache/carbondata/pull/3531#issuecomment-568833027 Build Failed with Spark 2.1.0, Please check CI http://121.244.95.60:12545/job/ApacheCarbonPRBuilder2.1/1270/ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [carbondata] niuge01 opened a new pull request #3532: [CARBONDATA-3557] Write flink streaming data to partition table
niuge01 opened a new pull request #3532: [CARBONDATA-3557] Write flink streaming data to partition table URL: https://github.com/apache/carbondata/pull/3532 Support write flink streaming data to a partition carbon table with stage file format. - [ ] Any interfaces changed? Yes, add an property [COMMIT_THRESHOLD] of carbon writer. - [ ] Any backward compatibility impacted? No - [ ] Document update required? - [ ] Testing done Please provide details on - Whether new unit test cases have been added or why no new tests are required? - How it is tested? Please attach test report. - Is it a performance related change? Please attach the performance test report. - Any additional information to help reviewers in testing this change. - [ ] For large changes, please consider breaking it into sub-tasks under an umbrella JIRA. NA This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [carbondata] CarbonDataQA1 commented on issue #3528: [CARBONDATA-3630] update should support limit 1 sub query and empty result subquery
CarbonDataQA1 commented on issue #3528: [CARBONDATA-3630] update should support limit 1 sub query and empty result subquery URL: https://github.com/apache/carbondata/pull/3528#issuecomment-568831070 Build Success with Spark 2.3.4, Please check CI http://121.244.95.60:12545/job/ApacheCarbonPRBuilder2.3/1290/ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [carbondata] CarbonDataQA1 commented on issue #3528: [CARBONDATA-3630] update should support limit 1 sub query and empty result subquery
CarbonDataQA1 commented on issue #3528: [CARBONDATA-3630] update should support limit 1 sub query and empty result subquery URL: https://github.com/apache/carbondata/pull/3528#issuecomment-568830563 Build Success with Spark 2.2.1, Please check CI http://121.244.95.60:12545/job/ApacheCarbonPRBuilder2.2/1279/ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [carbondata] Indhumathi27 commented on a change in pull request #3529: [CARBONDATA-3628] Support alter hive table add complex column type
Indhumathi27 commented on a change in pull request #3529: [CARBONDATA-3628]
Support alter hive table add complex column type
URL: https://github.com/apache/carbondata/pull/3529#discussion_r361248397
##
File path:
integration/spark-common-cluster-test/src/test/scala/org/apache/carbondata/cluster/sdv/generated/AlterTableTestCase.scala
##
@@ -1022,7 +1022,31 @@ class AlterTableTestCase extends QueryTest with
BeforeAndAfterAll {
assert(exception.getMessage.contains("Unsupported alter operation on
hive table"))
} else if (SparkUtil.isSparkVersionXandAbove("2.2")) {
sql("alter table alter_hive add columns(add string)")
- sql("insert into alter_hive select 'abc','banglore'")
+ sql("alter table alter_hive add columns (var map)")
+ sql("insert into alter_hive select
'abc','banglore',map('age','10','birth','2020')")
+ checkAnswer(
+sql("select * from alter_hive"),
+Seq(Row("abc", "banglore", Map("age" -> "10", "birth" -> "2020")))
+ )
+}
+ }
+
+ test("Alter table add column for hive partitioned table for spark version
above 2.1") {
+sql("drop table if exists alter_hive")
+sql("create table alter_hive(name string) stored as rcfile partitioned by
(dt string)")
+if (SparkUtil.isSparkVersionXandAbove("2.2")) {
+ sql("alter table alter_hive add columns(add string)")
+ sql("alter table alter_hive add columns (var map)")
Review comment:
Can you please add a case with STRUCT Datatype also
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [carbondata] Indhumathi27 commented on a change in pull request #3529: [CARBONDATA-3628] Support alter hive table add complex column type
Indhumathi27 commented on a change in pull request #3529: [CARBONDATA-3628] Support alter hive table add complex column type URL: https://github.com/apache/carbondata/pull/3529#discussion_r361248845 ## File path: integration/spark-common/src/main/scala/org/apache/carbondata/spark/rdd/CarbonMergerRDD.scala ## @@ -680,7 +681,7 @@ class CarbonMergerRDD[K, V]( partitionNames = null, splits = allSplits) val objectOrdering: Ordering[Object] = createOrderingForColumn(rangeColumn) -val sparkDataType = Util.convertCarbonToSparkDataType(dataType) Review comment: `Util` can be removed from Import if not used This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [carbondata] CarbonDataQA1 commented on issue #3528: [CARBONDATA-3630] update should support limit 1 sub query and empty result subquery
CarbonDataQA1 commented on issue #3528: [CARBONDATA-3630] update should support limit 1 sub query and empty result subquery URL: https://github.com/apache/carbondata/pull/3528#issuecomment-568830426 Build Success with Spark 2.1.0, Please check CI http://121.244.95.60:12545/job/ApacheCarbonPRBuilder2.1/1269/ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Created] (CARBONDATA-3630) update should support limit 1 sub query and empty result subquery
Ajantha Bhat created CARBONDATA-3630: Summary: update should support limit 1 sub query and empty result subquery Key: CARBONDATA-3630 URL: https://issues.apache.org/jira/browse/CARBONDATA-3630 Project: CarbonData Issue Type: Bug Reporter: Ajantha Bhat Assignee: Ajantha Bhat Fix For: 2.0.0 h1. update should support limit 1 sub query and empty result subquery -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [carbondata] MarvinLitt commented on a change in pull request #3521: [doc_zh_cn] add a commercial inventory DB and carbon data query performance comparison doc chinese doc to carbondata
MarvinLitt commented on a change in pull request #3521: [doc_zh_cn] add a commercial inventory DB and carbon data query performance comparison doc chinese doc to carbondata URL: https://github.com/apache/carbondata/pull/3521#discussion_r361239075 ## File path: docs/zh_cn/某商业列存DB和CarbonData查询性能对比.md ## @@ -0,0 +1,109 @@ + + +## Carbondata 替换某商业列存DB查询性能对比 + +本文主要在于给用户呈现Carbondata在替换某商业列存DB过程中对于该DB的查询性能提升，Carbondata自身的优势和特点，本文的数据仅为基于某领域查询特点框架下SQL的查询结果，只代表该特定查询特点下的性能对比。 + + + + + +## 1.集群状态对比 + +| 集群 | 描述 | +| | - | +| 某商业列存DB集群 | 1个加载节点，1个协调节点，1个查询节点，SSD硬盘| Review comment: done This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [carbondata] CarbonDataQA1 commented on issue #3502: [CARBONATA-3605] Remove global dictionary feature
CarbonDataQA1 commented on issue #3502: [CARBONATA-3605] Remove global dictionary feature URL: https://github.com/apache/carbondata/pull/3502#issuecomment-568776544 Build Failed with Spark 2.3.4, Please check CI http://121.244.95.60:12545/job/ApacheCarbonPRBuilder2.3/1288/ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [carbondata] CarbonDataQA1 commented on issue #3502: [CARBONATA-3605] Remove global dictionary feature
CarbonDataQA1 commented on issue #3502: [CARBONATA-3605] Remove global dictionary feature URL: https://github.com/apache/carbondata/pull/3502#issuecomment-568776543 Build Failed with Spark 2.2.1, Please check CI http://121.244.95.60:12545/job/ApacheCarbonPRBuilder2.2/1277/ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [carbondata] CarbonDataQA1 commented on issue #3531: [WIP]Remove unused declarations
CarbonDataQA1 commented on issue #3531: [WIP]Remove unused declarations URL: https://github.com/apache/carbondata/pull/3531#issuecomment-568772771 Build Failed with Spark 2.3.4, Please check CI http://121.244.95.60:12545/job/ApacheCarbonPRBuilder2.3/1289/ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [carbondata] CarbonDataQA1 commented on issue #3531: [WIP]Remove unused declarations
CarbonDataQA1 commented on issue #3531: [WIP]Remove unused declarations URL: https://github.com/apache/carbondata/pull/3531#issuecomment-568772356 Build Failed with Spark 2.2.1, Please check CI http://121.244.95.60:12545/job/ApacheCarbonPRBuilder2.2/1278/ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [carbondata] CarbonDataQA1 commented on issue #3531: [WIP]Remove unused declarations
CarbonDataQA1 commented on issue #3531: [WIP]Remove unused declarations URL: https://github.com/apache/carbondata/pull/3531#issuecomment-568772204 Build Failed with Spark 2.1.0, Please check CI http://121.244.95.60:12545/job/ApacheCarbonPRBuilder2.1/1268/ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [carbondata] CarbonDataQA1 commented on issue #3502: [CARBONATA-3605] Remove global dictionary feature
CarbonDataQA1 commented on issue #3502: [CARBONATA-3605] Remove global dictionary feature URL: https://github.com/apache/carbondata/pull/3502#issuecomment-568771411 Build Success with Spark 2.1.0, Please check CI http://121.244.95.60:12545/job/ApacheCarbonPRBuilder2.1/1267/ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [carbondata] CarbonDataQA1 commented on issue #3531: [WIP]Remove unused declarations
CarbonDataQA1 commented on issue #3531: [WIP]Remove unused declarations URL: https://github.com/apache/carbondata/pull/3531#issuecomment-568769154 Build Failed with Spark 2.1.0, Please check CI http://121.244.95.60:12545/job/ApacheCarbonPRBuilder2.1/1266/ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [carbondata] CarbonDataQA1 commented on issue #3531: [WIP]Remove unused declarations
CarbonDataQA1 commented on issue #3531: [WIP]Remove unused declarations URL: https://github.com/apache/carbondata/pull/3531#issuecomment-568769003 Build Failed with Spark 2.2.1, Please check CI http://121.244.95.60:12545/job/ApacheCarbonPRBuilder2.2/1276/ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [carbondata] CarbonDataQA1 commented on issue #3531: [WIP]Remove unused declarations
CarbonDataQA1 commented on issue #3531: [WIP]Remove unused declarations URL: https://github.com/apache/carbondata/pull/3531#issuecomment-568768834 Build Failed with Spark 2.3.4, Please check CI http://121.244.95.60:12545/job/ApacheCarbonPRBuilder2.3/1287/ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [carbondata] jackylk opened a new pull request #3531: [WIP]Remove unused declarations
jackylk opened a new pull request #3531: [WIP]Remove unused declarations URL: https://github.com/apache/carbondata/pull/3531 - [ ] Any interfaces changed? - [ ] Any backward compatibility impacted? - [ ] Document update required? - [ ] Testing done Please provide details on - Whether new unit test cases have been added or why no new tests are required? - How it is tested? Please attach test report. - Is it a performance related change? Please attach the performance test report. - Any additional information to help reviewers in testing this change. - [ ] For large changes, please consider breaking it into sub-tasks under an umbrella JIRA. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [carbondata] CarbonDataQA1 commented on issue #3529: [CARBONDATA-3628] Support alter hive table add complex column type
CarbonDataQA1 commented on issue #3529: [CARBONDATA-3628] Support alter hive table add complex column type URL: https://github.com/apache/carbondata/pull/3529#issuecomment-568758859 Build Success with Spark 2.3.4, Please check CI http://121.244.95.60:12545/job/ApacheCarbonPRBuilder2.3/1282/ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [carbondata] CarbonDataQA1 commented on issue #3528: [WIP] update should support limit 1 sub query
CarbonDataQA1 commented on issue #3528: [WIP] update should support limit 1 sub query URL: https://github.com/apache/carbondata/pull/3528#issuecomment-568754333 Build Success with Spark 2.3.4, Please check CI http://121.244.95.60:12545/job/ApacheCarbonPRBuilder2.3/1286/ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [carbondata] CarbonDataQA1 commented on issue #3528: [WIP] update should support limit 1 sub query
CarbonDataQA1 commented on issue #3528: [WIP] update should support limit 1 sub query URL: https://github.com/apache/carbondata/pull/3528#issuecomment-568753541 Build Success with Spark 2.2.1, Please check CI http://121.244.95.60:12545/job/ApacheCarbonPRBuilder2.2/1275/ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [carbondata] CarbonDataQA1 commented on issue #3528: [WIP] update should support limit 1 sub query
CarbonDataQA1 commented on issue #3528: [WIP] update should support limit 1 sub query URL: https://github.com/apache/carbondata/pull/3528#issuecomment-568743761 Build Success with Spark 2.1.0, Please check CI http://121.244.95.60:12545/job/ApacheCarbonPRBuilder2.1/1265/ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Created] (CARBONDATA-3629) Fix Select query failure on aggregation of same column on MV
Indhumathi Muthumurugesh created CARBONDATA-3629: Summary: Fix Select query failure on aggregation of same column on MV Key: CARBONDATA-3629 URL: https://issues.apache.org/jira/browse/CARBONDATA-3629 Project: CarbonData Issue Type: Bug Reporter: Indhumathi Muthumurugesh -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [carbondata] CarbonDataQA1 commented on issue #3530: [WIP]Fix Select query failure on aggregation of same column on MV
CarbonDataQA1 commented on issue #3530: [WIP]Fix Select query failure on aggregation of same column on MV URL: https://github.com/apache/carbondata/pull/3530#issuecomment-568726615 Build Success with Spark 2.3.4, Please check CI http://121.244.95.60:12545/job/ApacheCarbonPRBuilder2.3/1285/ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [carbondata] CarbonDataQA1 commented on issue #3530: [WIP]Fix Select query failure on aggregation of same column on MV
CarbonDataQA1 commented on issue #3530: [WIP]Fix Select query failure on aggregation of same column on MV URL: https://github.com/apache/carbondata/pull/3530#issuecomment-568725557 Build Success with Spark 2.2.1, Please check CI http://121.244.95.60:12545/job/ApacheCarbonPRBuilder2.2/1274/ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [carbondata] CarbonDataQA1 commented on issue #3521: [doc_zh_cn] add a commercial inventory DB and carbon data query performance comparison doc chinese doc to carbondata
CarbonDataQA1 commented on issue #3521: [doc_zh_cn] add a commercial inventory DB and carbon data query performance comparison doc chinese doc to carbondata URL: https://github.com/apache/carbondata/pull/3521#issuecomment-568725395 Build Success with Spark 2.3.4, Please check CI http://121.244.95.60:12545/job/ApacheCarbonPRBuilder2.3/1284/ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [carbondata] CarbonDataQA1 commented on issue #3521: [doc_zh_cn] add a commercial inventory DB and carbon data query performance comparison doc chinese doc to carbondata
CarbonDataQA1 commented on issue #3521: [doc_zh_cn] add a commercial inventory DB and carbon data query performance comparison doc chinese doc to carbondata URL: https://github.com/apache/carbondata/pull/3521#issuecomment-568724810 Build Success with Spark 2.2.1, Please check CI http://121.244.95.60:12545/job/ApacheCarbonPRBuilder2.2/1273/ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [carbondata] CarbonDataQA1 commented on issue #3529: [CARBONDATA-3628] Support alter hive table add complex column type
CarbonDataQA1 commented on issue #3529: [CARBONDATA-3628] Support alter hive table add complex column type URL: https://github.com/apache/carbondata/pull/3529#issuecomment-568720537 Build Success with Spark 2.2.1, Please check CI http://121.244.95.60:12545/job/ApacheCarbonPRBuilder2.2/1271/ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [carbondata] CarbonDataQA1 commented on issue #3530: [WIP]Fix Select query failure on aggregation of same column on MV
CarbonDataQA1 commented on issue #3530: [WIP]Fix Select query failure on aggregation of same column on MV URL: https://github.com/apache/carbondata/pull/3530#issuecomment-568713653 Build Success with Spark 2.1.0, Please check CI http://121.244.95.60:12545/job/ApacheCarbonPRBuilder2.1/1264/ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [carbondata] CarbonDataQA1 commented on issue #3521: [doc_zh_cn] add a commercial inventory DB and carbon data query performance comparison doc chinese doc to carbondata
CarbonDataQA1 commented on issue #3521: [doc_zh_cn] add a commercial inventory DB and carbon data query performance comparison doc chinese doc to carbondata URL: https://github.com/apache/carbondata/pull/3521#issuecomment-568712325 Build Success with Spark 2.1.0, Please check CI http://121.244.95.60:12545/job/ApacheCarbonPRBuilder2.1/1263/ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [carbondata] CarbonDataQA1 commented on issue #3529: [CARBONDATA-3628] Support alter hive table add complex column type
CarbonDataQA1 commented on issue #3529: [CARBONDATA-3628] Support alter hive table add complex column type URL: https://github.com/apache/carbondata/pull/3529#issuecomment-568708092 Build Success with Spark 2.1.0, Please check CI http://121.244.95.60:12545/job/ApacheCarbonPRBuilder2.1/1261/ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [carbondata] Indhumathi27 commented on issue #3530: [WIP]Fix Select query failure on aggregation of same column on MV
Indhumathi27 commented on issue #3530: [WIP]Fix Select query failure on aggregation of same column on MV URL: https://github.com/apache/carbondata/pull/3530#issuecomment-568706143 retest this please This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [carbondata] CarbonDataQA1 commented on issue #3530: [WIP]Fix Select query failure on aggregation of same column on MV
CarbonDataQA1 commented on issue #3530: [WIP]Fix Select query failure on aggregation of same column on MV URL: https://github.com/apache/carbondata/pull/3530#issuecomment-568702022 Build Failed with Spark 2.3.4, Please check CI http://121.244.95.60:12545/job/ApacheCarbonPRBuilder2.3/1281/ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [carbondata] IceMimosa commented on a change in pull request #3529: [CARBONDATA-3628] Support alter hive table add complex column type
IceMimosa commented on a change in pull request #3529: [CARBONDATA-3628]
Support alter hive table add complex column type
URL: https://github.com/apache/carbondata/pull/3529#discussion_r36573
##
File path:
integration/spark-common-cluster-test/src/test/scala/org/apache/carbondata/cluster/sdv/generated/AlterTableTestCase.scala
##
@@ -1023,6 +1023,7 @@ class AlterTableTestCase extends QueryTest with
BeforeAndAfterAll {
} else if (SparkUtil.isSparkVersionXandAbove("2.2")) {
sql("alter table alter_hive add columns(add string)")
sql("insert into alter_hive select 'abc','banglore'")
+ sql("alter table alter_hive add columns (var map)")
Review comment:
Done, thanks for reviewing.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [carbondata] CarbonDataQA1 commented on issue #3530: [WIP]Fix Select query failure on aggregation of same column on MV
CarbonDataQA1 commented on issue #3530: [WIP]Fix Select query failure on aggregation of same column on MV URL: https://github.com/apache/carbondata/pull/3530#issuecomment-568700141 Build Success with Spark 2.2.1, Please check CI http://121.244.95.60:12545/job/ApacheCarbonPRBuilder2.2/1270/ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [carbondata] jackylk commented on a change in pull request #3521: [doc_zh_cn] add a commercial inventory DB and carbon data query performance comparison doc chinese doc to carbondata
jackylk commented on a change in pull request #3521: [doc_zh_cn] add a commercial inventory DB and carbon data query performance comparison doc chinese doc to carbondata URL: https://github.com/apache/carbondata/pull/3521#discussion_r361106581 ## File path: docs/zh_cn/某商业列存DB和CarbonData查询性能对比.md ## @@ -0,0 +1,109 @@ + + +## Carbondata 替换某商业列存DB查询性能对比 + +本文主要在于给用户呈现Carbondata在替换某商业列存DB过程中对于该DB的查询性能提升，Carbondata自身的优势和特点，本文的数据仅为基于某领域查询特点框架下SQL的查询结果，只代表该特定查询特点下的性能对比。 + + + + + +## 1.集群状态对比 + +| 集群 | 描述 | +| | - | +| 某商业列存DB集群 | 1个加载节点，1个协调节点，1个查询节点，SSD硬盘| Review comment: please discribe how many CPU cores in both environements This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [carbondata] shenh062326 commented on a change in pull request #3525: [CARBONDATA-3626] Improve performance when load data into carbon table with lots of columns
shenh062326 commented on a change in pull request #3525: [CARBONDATA-3626]
Improve performance when load data into carbon table with lots of columns
URL: https://github.com/apache/carbondata/pull/3525#discussion_r361100784
##
File path:
integration/spark2/src/main/scala/org/apache/spark/sql/execution/datasources/SparkCarbonTableFormat.scala
##
@@ -353,7 +353,8 @@ private class CarbonOutputWriter(path: String,
def writeCarbon(row: InternalRow): Unit = {
val data = new Array[AnyRef](fieldTypes.length + partitionData.length)
var i = 0
-while (i < fieldTypes.length) {
+val fieldTypesLen = fieldTypes.length
+while (i < fieldTypesLen) {
Review comment:
> Is it really has difference after the change for lots of columns?
It can significantly improve load data into carbon table with lots of
columns.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [carbondata] CarbonDataQA1 commented on issue #3530: [WIP]Fix Select query failure on aggregation of same column on MV
CarbonDataQA1 commented on issue #3530: [WIP]Fix Select query failure on aggregation of same column on MV URL: https://github.com/apache/carbondata/pull/3530#issuecomment-568687805 Build Success with Spark 2.1.0, Please check CI http://121.244.95.60:12545/job/ApacheCarbonPRBuilder2.1/1260/ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [carbondata] jackylk commented on a change in pull request #3518: [DOC] add performance-tuning with codegen parameters support
jackylk commented on a change in pull request #3518: [DOC] add performance-tuning with codegen parameters support URL: https://github.com/apache/carbondata/pull/3518#discussion_r361096657 ## File path: docs/performance-tuning.md ## @@ -173,6 +173,8 @@ | carbon.sort.temp.compressor | spark/carbonlib/carbon.properties | Data loading | Specify the name of compressor to compress the intermediate sort temporary files during sort procedure in data loading. | The optional values are 'SNAPPY','GZIP','BZIP2','LZ4','ZSTD', and empty. Specially, empty means that Carbondata will not compress the sort temp files. This parameter will be useful if you encounter disk bottleneck. | | carbon.load.skewedDataOptimization.enabled | spark/carbonlib/carbon.properties | Data loading | Whether to enable size based block allocation strategy for data loading. | When loading, carbondata will use file size based block allocation strategy for task distribution. It will make sure that all the executors process the same size of data -- It's useful if the size of your input data files varies widely, say 1MB to 1GB. | | carbon.load.min.size.enabled | spark/carbonlib/carbon.properties | Data loading | Whether to enable node minumun input data size allocation strategy for data loading.| When loading, carbondata will use node minumun input data size allocation strategy for task distribution. It will make sure the nodes load the minimum amount of data -- It's useful if the size of your input data files very small, say 1MB to 256MB,Avoid generating a large number of small files. | +| spark.sql.codegen.wholeStage | spark/conf/spark-defaults.conf | Querying | improves the execution performance of a query by collapsing a query tree into a single optimized function that eliminates virtual function calls and leverages CPU registers for intermediate data. | The whole stage CodeGen mechanism introduced by spark SQL in version 2. X causes. This configuration is recommended to be off at spark 2.1 and on at spark 2.3. Because under spark2.1 user can only use spark.sql.codegen.wholeStage to control whether to use codegen, but can not config the size of the method. In fact, this parameter should be configured to be the same as the local JDK. Under spark2.3 support spark.sql.codegen.hugeMethodLimit use can use that to config the method size. | Review comment: This is spark configuration, suggest not to add in carbon's document. Or maybe you can add a link in the bottom of this section to point to the performance tuning page of spark community This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [carbondata] jackylk commented on a change in pull request #3529: [CARBONDATA-3628] Support alter hive table add complex column type
jackylk commented on a change in pull request #3529: [CARBONDATA-3628] Support
alter hive table add complex column type
URL: https://github.com/apache/carbondata/pull/3529#discussion_r361096161
##
File path:
integration/spark-common-cluster-test/src/test/scala/org/apache/carbondata/cluster/sdv/generated/AlterTableTestCase.scala
##
@@ -1023,6 +1023,7 @@ class AlterTableTestCase extends QueryTest with
BeforeAndAfterAll {
} else if (SparkUtil.isSparkVersionXandAbove("2.2")) {
sql("alter table alter_hive add columns(add string)")
sql("insert into alter_hive select 'abc','banglore'")
+ sql("alter table alter_hive add columns (var map)")
Review comment:
Thanks for contribution.
Can you add a testcase to load data after adding the complex column?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
