[GitHub] [carbondata] akkio-97 commented on pull request #4115: [CARBONDATA-4163] Support adding of single-level complex columns(array/struct)

[GitBox]

akkio-97 commented on pull request #4115:
URL: https://github.com/apache/carbondata/pull/4115#issuecomment-820958582


   retest this please


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [carbondata] CarbonDataQA2 commented on pull request #4113: [CARBONDATA-4161] Describe complex columns

[GitBox]

CarbonDataQA2 commented on pull request #4113:
URL: https://github.com/apache/carbondata/pull/4113#issuecomment-820958011


   Build Success with Spark 2.4.5, Please check CI 
http://121.244.95.60:12602/job/ApacheCarbon_PR_Builder_2.4.5/3444/
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [carbondata] akashrn5 commented on a change in pull request #3988: [CARBONDATA-4037] Improve the table status and segment file writing

[GitBox]

akashrn5 commented on a change in pull request #3988:
URL: https://github.com/apache/carbondata/pull/3988#discussion_r614596639



##
File path: 
integration/spark/src/main/scala/org/apache/spark/sql/secondaryindex/util/SecondaryIndexUtil.scala
##
@@ -353,16 +342,38 @@ object SecondaryIndexUtil {
 }
   }
 
+  /**
+   * This method returns the list of index/merge index files for a segment in 
carbonTable.
+   */
+  @throws[IOException]
+  private def getIndexFilesListForSegment(segment: Segment, tablePath: 
String): util.Set[String] = {
+var indexFiles : util.Set[String] = new util.HashSet[String]
+val segmentFileStore = new SegmentFileStore(tablePath,
+  segment.getSegmentFileName)
+val segmentPath = CarbonTablePath.getSegmentPath(tablePath, 
segment.getSegmentNo)
+if (segmentFileStore.getSegmentFile == null)  {
+  indexFiles = new SegmentIndexFileStore()
+.getMergeOrIndexFilesFromSegment(segmentPath).keySet
+}
+else {
+  indexFiles = segmentFileStore.getIndexAndMergeFiles.keySet
+}
+indexFiles
+  }
+
   /**
* This method delete the carbondata files present in pertition of during 
small
* datafile merge after loading a segment to SI table. It should be deleted 
after
* data file merge operation, else, concurrency can cause file not found 
issues.
*/
-  private def deleteOldCarbonDataFiles(partition: CarbonSparkPartition): Unit 
= {
+  private def deleteOldCarbonDataFiles(partition: CarbonSparkPartition,
+  validSegmentsToUse: List[Segment]): Unit = {
 val splitList = partition.split.value.getAllSplits
 splitList.asScala.foreach { split =>
-  val carbonFile = FileFactory.getCarbonFile(split.getFilePath)
-  carbonFile.delete()
+  if (validSegmentsToUse.contains(split.getSegment)) {
+val carbonFile = FileFactory.getCarbonFile(split.getFilePath)

Review comment:
   ```suggestion
   val mergedCarbonDataFile = 
FileFactory.getCarbonFile(split.getFilePath)
   ```




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [carbondata] akashrn5 commented on a change in pull request #3988: [CARBONDATA-4037] Improve the table status and segment file writing

[GitBox]

akashrn5 commented on a change in pull request #3988:
URL: https://github.com/apache/carbondata/pull/3988#discussion_r614596183



##
File path: 
integration/spark/src/main/scala/org/apache/spark/sql/secondaryindex/util/SecondaryIndexUtil.scala
##
@@ -353,16 +342,38 @@ object SecondaryIndexUtil {
 }
   }
 
+  /**
+   * This method returns the list of index/merge index files for a segment in 
carbonTable.
+   */
+  @throws[IOException]
+  private def getIndexFilesListForSegment(segment: Segment, tablePath: 
String): util.Set[String] = {
+var indexFiles : util.Set[String] = new util.HashSet[String]
+val segmentFileStore = new SegmentFileStore(tablePath,
+  segment.getSegmentFileName)
+val segmentPath = CarbonTablePath.getSegmentPath(tablePath, 
segment.getSegmentNo)
+if (segmentFileStore.getSegmentFile == null)  {
+  indexFiles = new SegmentIndexFileStore()
+.getMergeOrIndexFilesFromSegment(segmentPath).keySet
+}
+else {

Review comment:
   move this to above line, correct the style




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [carbondata] akashrn5 commented on a change in pull request #3988: [CARBONDATA-4037] Improve the table status and segment file writing

[GitBox]

akashrn5 commented on a change in pull request #3988:
URL: https://github.com/apache/carbondata/pull/3988#discussion_r614594602



##
File path: 
integration/spark/src/main/scala/org/apache/spark/sql/secondaryindex/util/SecondaryIndexUtil.scala
##
@@ -231,73 +240,36 @@ object SecondaryIndexUtil {
   } else {
 siRebuildRDD.partitions.foreach { partition =>
   val carbonSparkPartition = 
partition.asInstanceOf[CarbonSparkPartition]
-  deleteOldCarbonDataFiles(carbonSparkPartition)
+  deleteOldCarbonDataFiles(carbonSparkPartition, 
validSegmentsToUse.toList)
 }
   }
+  val segmentToLoadStartTimeMap: scala.collection.mutable.Map[String, 
java.lang.Long] =
+scala.collection.mutable.Map()
+  // merge index files and write segment file for merged segments
   mergedSegments.asScala.map { seg =>
+val segmentPath = CarbonTablePath.getSegmentPath(tablePath, 
seg.getLoadName)
+try {
+  new CarbonIndexFileMergeWriter(indexCarbonTable)
+.writeMergeIndexFileBasedOnSegmentFolder(null, false, 
segmentPath,
+  seg.getLoadName, carbonLoadModel.getFactTimeStamp.toString,
+  true)
+} catch {
+  case e: IOException =>
+val message =
+  s"Failed to merge index files in path: $segmentPath. " + 
e.getMessage()

Review comment:
   ```suggestion
 s"Failed to merge index files in path: $segmentPath. 
${e.getMessage()} "
   ```




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [carbondata] CarbonDataQA2 commented on pull request #4115: [CARBONDATA-4163] Support adding of single-level complex columns(array/struct)

[GitBox]

CarbonDataQA2 commented on pull request #4115:
URL: https://github.com/apache/carbondata/pull/4115#issuecomment-820942569


   Build Failed  with Spark 2.3.4, Please check CI 
http://121.244.95.60:12602/job/ApacheCarbonPRBuilder2.3/5197/
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [carbondata] akkio-97 commented on a change in pull request #4115: [CARBONDATA-4163] Support adding of single-level complex columns(array/struct)

[GitBox]

akkio-97 commented on a change in pull request #4115:
URL: https://github.com/apache/carbondata/pull/4115#discussion_r614576721



##
File path: 
integration/spark/src/main/scala/org/apache/spark/sql/execution/command/carbonTableSchemaCommon.scala
##
@@ -319,10 +373,6 @@ class AlterTableColumnSchemaGenerator(
 sys.error(s"Duplicate column found with name: $name")
   })
 
-if (newCols.exists(_.getDataType.isComplexType)) {

Review comment:
   done

##
File path: 
integration/spark/src/main/scala/org/apache/spark/sql/execution/command/carbonTableSchemaCommon.scala
##
@@ -255,6 +276,10 @@ class AlterTableColumnSchemaGenerator(
 
 // add new dimension columns
 alterTableModel.dimCols.foreach(field => {
+  if (field.dataType.get.equals("Map")) {

Review comment:
   done




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [carbondata] Indhumathi27 commented on a change in pull request #4115: [CARBONDATA-4163] Support adding of single-level complex columns(array/struct)

[GitBox]

Indhumathi27 commented on a change in pull request #4115:
URL: https://github.com/apache/carbondata/pull/4115#discussion_r614571362



##
File path: 
integration/spark/src/main/scala/org/apache/spark/sql/execution/command/carbonTableSchemaCommon.scala
##
@@ -255,6 +276,10 @@ class AlterTableColumnSchemaGenerator(
 
 // add new dimension columns
 alterTableModel.dimCols.foreach(field => {
+  if (field.dataType.get.equals("Map")) {

Review comment:
   Can use CarbonCommonConstants.MAP constant

##
File path: 
integration/spark/src/main/scala/org/apache/spark/sql/execution/command/carbonTableSchemaCommon.scala
##
@@ -319,10 +373,6 @@ class AlterTableColumnSchemaGenerator(
 sys.error(s"Duplicate column found with name: $name")
   })
 
-if (newCols.exists(_.getDataType.isComplexType)) {

Review comment:
   Please update document 
[Alter-add](https://github.com/apache/carbondata/blob/master/docs/ddl-of-carbondata.md#add-columns)




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [carbondata] Indhumathi27 commented on pull request #4113: [CARBONDATA-4161] Describe complex columns

[GitBox]

Indhumathi27 commented on pull request #4113:
URL: https://github.com/apache/carbondata/pull/4113#issuecomment-820913867


   retest this please


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [carbondata] akkio-97 commented on pull request #4115: [CARBONDATA-4163] Support adding of single-level complex columns(array/struct)

[GitBox]

akkio-97 commented on pull request #4115:
URL: https://github.com/apache/carbondata/pull/4115#issuecomment-820908025


   retest this please


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [carbondata] CarbonDataQA2 commented on pull request #4115: [CARBONDATA-4163] Support adding of single-level complex columns(array/struct)

[GitBox]

CarbonDataQA2 commented on pull request #4115:
URL: https://github.com/apache/carbondata/pull/4115#issuecomment-820764494


   Build Failed  with Spark 2.4.5, Please check CI 
http://121.244.95.60:12602/job/ApacheCarbon_PR_Builder_2.4.5/3442/
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [carbondata] CarbonDataQA2 commented on pull request #4115: [CARBONDATA-4163] Support adding of single-level complex columns(array/struct)

[GitBox]

CarbonDataQA2 commented on pull request #4115:
URL: https://github.com/apache/carbondata/pull/4115#issuecomment-820763395


   Build Success with Spark 2.3.4, Please check CI 
http://121.244.95.60:12602/job/ApacheCarbonPRBuilder2.3/5194/
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

Re: [DISCUSSION] Support JOIN query with spatial index

[Indhumathi]

Hello all,

Please find the design document link attached in JIRA,  CARBONDATA-4166
  
Any inputs/suggestions from the community is most welcomed.

Regards,
Indhumathi



--
Sent from: 
http://apache-carbondata-dev-mailing-list-archive.1130556.n5.nabble.com/

[GitHub] [carbondata] CarbonDataQA2 commented on pull request #4115: [CARBONDATA-4163] Support adding of single-level complex columns(array/struct)

[GitBox]

CarbonDataQA2 commented on pull request #4115:
URL: https://github.com/apache/carbondata/pull/4115#issuecomment-820435272


   Build Success with Spark 2.4.5, Please check CI 
http://121.244.95.60:12602/job/ApacheCarbon_PR_Builder_2.4.5/3441/
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [carbondata] CarbonDataQA2 commented on pull request #4115: [CARBONDATA-4163] Support adding of single-level complex columns(array/struct)

[GitBox]

CarbonDataQA2 commented on pull request #4115:
URL: https://github.com/apache/carbondata/pull/4115#issuecomment-820429606


   Build Success with Spark 2.3.4, Please check CI 
http://121.244.95.60:12602/job/ApacheCarbonPRBuilder2.3/5193/
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [carbondata] CarbonDataQA2 commented on pull request #4115: [CARBONDATA-4163] Support adding of single-level complex columns(array/struct)

[GitBox]

CarbonDataQA2 commented on pull request #4115:
URL: https://github.com/apache/carbondata/pull/4115#issuecomment-820361276


   Build Success with Spark 2.4.5, Please check CI 
http://121.244.95.60:12602/job/ApacheCarbon_PR_Builder_2.4.5/3440/
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [carbondata] CarbonDataQA2 commented on pull request #4115: [CARBONDATA-4163] Support adding of single-level complex columns(array/struct)

[GitBox]

CarbonDataQA2 commented on pull request #4115:
URL: https://github.com/apache/carbondata/pull/4115#issuecomment-820360590


   Build Success with Spark 2.3.4, Please check CI 
http://121.244.95.60:12602/job/ApacheCarbonPRBuilder2.3/5192/
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [carbondata] CarbonDataQA2 commented on pull request #4119: Enable github's merge function

[GitBox]

CarbonDataQA2 commented on pull request #4119:
URL: https://github.com/apache/carbondata/pull/4119#issuecomment-820307002


   Build Success with Spark 2.3.4, Please check CI 
http://121.244.95.60:12602/job/ApacheCarbonPRBuilder2.3/5191/
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [carbondata] akkio-97 commented on a change in pull request #4115: [CARBONDATA-4163] Support adding of single-level complex columns(array/struct)

[GitBox]

akkio-97 commented on a change in pull request #4115:
URL: https://github.com/apache/carbondata/pull/4115#discussion_r613940772



##
File path: 
integration/spark/src/main/scala/org/apache/spark/sql/execution/command/carbonTableSchemaCommon.scala
##
@@ -275,6 +295,39 @@ class AlterTableColumnSchemaGenerator(
 allColumns ++= Seq(columnSchema)
   }
   newCols ++= Seq(columnSchema)
+  if (DataTypes.isArrayType(columnSchema.getDataType)) {
+columnSchema.setNumberOfChild(field.children.size)
+val childField = field.children.get(0)
+val childSchema: ColumnSchema = createChildSchema(childField, 
currentSchemaOrdinal)
+if (childSchema.getDataType == DataTypes.VARCHAR) {
+  // put the new long string columns in 'longStringCols'
+  // and add them after old long string columns
+  longStringCols ++= Seq(childSchema)
+} else {
+  allColumns ++= Seq(childSchema)
+}
+newCols ++= Seq(childSchema)
+  } else if (DataTypes.isStructType(columnSchema.getDataType)) {
+val noOfChildren = field.children.get.size
+columnSchema.setNumberOfChild(noOfChildren)
+for (i <- 0 to noOfChildren - 1) {
+  val childField = field.children.get(i)
+  // TODO: should support adding multi-level complex columns: 
CARBONDATA-4164
+  if (!childField.children.contains(null)) {

Review comment:
   done

##
File path: 
integration/spark/src/main/scala/org/apache/spark/sql/parser/CarbonSparkSqlParserUtil.scala
##
@@ -683,12 +683,6 @@ object CarbonSparkSqlParserUtil {
   fields: List[Field],
   tblProp: Option[List[(String, String)]]
   ): CarbonAlterTableAddColumnCommand = {
-fields.foreach { f =>
-  if (CarbonParserUtil.isComplexType(f.dataType.get)) {

Review comment:
   done




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [carbondata] Indhumathi27 commented on pull request #4113: [CARBONDATA-4161] Describe complex columns

[GitBox]

Indhumathi27 commented on pull request #4113:
URL: https://github.com/apache/carbondata/pull/4113#issuecomment-820301972


   LGTM


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [carbondata] CarbonDataQA2 commented on pull request #4119: Enable github's merge function

[GitBox]

CarbonDataQA2 commented on pull request #4119:
URL: https://github.com/apache/carbondata/pull/4119#issuecomment-820290387


   Build Success with Spark 2.4.5, Please check CI 
http://121.244.95.60:12602/job/ApacheCarbon_PR_Builder_2.4.5/3439/
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [carbondata] asfgit merged pull request #4119: Enable github's merge function

[GitBox]

asfgit merged pull request #4119:
URL: https://github.com/apache/carbondata/pull/4119


   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [carbondata] CarbonDataQA2 commented on pull request #4115: [CARBONDATA-4163] Support adding of single-level complex columns(array/struct)

[GitBox]

CarbonDataQA2 commented on pull request #4115:
URL: https://github.com/apache/carbondata/pull/4115#issuecomment-820277896


   Build Success with Spark 2.3.4, Please check CI 
http://121.244.95.60:12602/job/ApacheCarbonPRBuilder2.3/5190/
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [carbondata] CarbonDataQA2 commented on pull request #4115: [CARBONDATA-4163] Support adding of single-level complex columns(array/struct)

[GitBox]

CarbonDataQA2 commented on pull request #4115:
URL: https://github.com/apache/carbondata/pull/4115#issuecomment-820272574


   Build Success with Spark 2.4.5, Please check CI 
http://121.244.95.60:12602/job/ApacheCarbon_PR_Builder_2.4.5/3438/
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [carbondata] CarbonDataQA2 commented on pull request #4113: [CARBONDATA-4161] Describe complex columns

[GitBox]

CarbonDataQA2 commented on pull request #4113:
URL: https://github.com/apache/carbondata/pull/4113#issuecomment-820256028


   Build Success with Spark 2.4.5, Please check CI 
http://121.244.95.60:12602/job/ApacheCarbon_PR_Builder_2.4.5/3437/
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [carbondata] CarbonDataQA2 commented on pull request #4113: [CARBONDATA-4161] Describe complex columns

[GitBox]

CarbonDataQA2 commented on pull request #4113:
URL: https://github.com/apache/carbondata/pull/4113#issuecomment-820250537


   Build Success with Spark 2.3.4, Please check CI 
http://121.244.95.60:12602/job/ApacheCarbonPRBuilder2.3/5189/
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [carbondata] Indhumathi27 commented on a change in pull request #4115: [CARBONDATA-4163] Support adding of single-level complex columns(array/struct)

[GitBox]

Indhumathi27 commented on a change in pull request #4115:
URL: https://github.com/apache/carbondata/pull/4115#discussion_r613876286



##
File path: 
integration/spark/src/main/scala/org/apache/spark/sql/parser/CarbonSparkSqlParserUtil.scala
##
@@ -683,12 +683,6 @@ object CarbonSparkSqlParserUtil {
   fields: List[Field],
   tblProp: Option[List[(String, String)]]
   ): CarbonAlterTableAddColumnCommand = {
-fields.foreach { f =>
-  if (CarbonParserUtil.isComplexType(f.dataType.get)) {

Review comment:
   I think,  still Alter add Map type is not supported. Please handle




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [carbondata] CarbonDataQA2 commented on pull request #4110: [CARBONDATA-4158]Secondary Index as a coarse-grain datamap and use them for Presto queries

[GitBox]

CarbonDataQA2 commented on pull request #4110:
URL: https://github.com/apache/carbondata/pull/4110#issuecomment-820235081


   Build Success with Spark 2.3.4, Please check CI 
http://121.244.95.60:12602/job/ApacheCarbonPRBuilder2.3/5187/
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [carbondata] Indhumathi27 commented on a change in pull request #4115: [CARBONDATA-4163] Support adding of single-level complex columns(array/struct)

[GitBox]

Indhumathi27 commented on a change in pull request #4115:
URL: https://github.com/apache/carbondata/pull/4115#discussion_r613866130



##
File path: 
integration/spark/src/main/scala/org/apache/spark/sql/execution/command/carbonTableSchemaCommon.scala
##
@@ -275,6 +295,39 @@ class AlterTableColumnSchemaGenerator(
 allColumns ++= Seq(columnSchema)
   }
   newCols ++= Seq(columnSchema)
+  if (DataTypes.isArrayType(columnSchema.getDataType)) {
+columnSchema.setNumberOfChild(field.children.size)
+val childField = field.children.get(0)
+val childSchema: ColumnSchema = createChildSchema(childField, 
currentSchemaOrdinal)
+if (childSchema.getDataType == DataTypes.VARCHAR) {
+  // put the new long string columns in 'longStringCols'
+  // and add them after old long string columns
+  longStringCols ++= Seq(childSchema)
+} else {
+  allColumns ++= Seq(childSchema)
+}
+newCols ++= Seq(childSchema)
+  } else if (DataTypes.isStructType(columnSchema.getDataType)) {
+val noOfChildren = field.children.get.size
+columnSchema.setNumberOfChild(noOfChildren)
+for (i <- 0 to noOfChildren - 1) {
+  val childField = field.children.get(i)
+  // TODO: should support adding multi-level complex columns: 
CARBONDATA-4164
+  if (!childField.children.contains(null)) {

Review comment:
   Please remove this check, since it is already handled in 
createChildschema method




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [carbondata] CarbonDataQA2 commented on pull request #4110: [CARBONDATA-4158]Secondary Index as a coarse-grain datamap and use them for Presto queries

[GitBox]

CarbonDataQA2 commented on pull request #4110:
URL: https://github.com/apache/carbondata/pull/4110#issuecomment-820230509


   Build Failed  with Spark 2.4.5, Please check CI 
http://121.244.95.60:12602/job/ApacheCarbon_PR_Builder_2.4.5/3435/
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [carbondata] chenliang613 opened a new pull request #4119: Enable github's merge function

[GitBox]

chenliang613 opened a new pull request #4119:
URL: https://github.com/apache/carbondata/pull/4119


   Enable github's merge function
   
### Why is this PR needed?


### What changes were proposed in this PR?
   
   
### Does this PR introduce any user interface change?
- No
- Yes. (please explain the change and update document)
   
### Is any new testcase added?
- No
- Yes
   
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [carbondata] akkio-97 commented on a change in pull request #4115: [CARBONDATA-4163] Support adding of single-level complex columns(array/struct)

[GitBox]

akkio-97 commented on a change in pull request #4115:
URL: https://github.com/apache/carbondata/pull/4115#discussion_r613828144



##
File path: 
integration/spark/src/main/scala/org/apache/spark/sql/execution/command/carbonTableSchemaCommon.scala
##
@@ -275,6 +275,64 @@ class AlterTableColumnSchemaGenerator(
 allColumns ++= Seq(columnSchema)
   }
   newCols ++= Seq(columnSchema)
+  if (DataTypes.isArrayType(columnSchema.getDataType)) {
+columnSchema.setNumberOfChild(field.children.size)
+val childField = field.children.get(0)
+// TODO: should support adding multi-level complex columns: 
CARBONDATA-4164
+if (childField.children != Some(null)) {

Review comment:
   done




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [carbondata-site] chenliang613 opened a new pull request #80: fix issue

[GitBox]

chenliang613 opened a new pull request #80:
URL: https://github.com/apache/carbondata-site/pull/80


   remove incubator


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org