[GitHub] carbondata pull request #1767: [CARBONDATA-1993] Carbon properties default v...
Github user pawanmalwal closed the pull request at: https://github.com/apache/carbondata/pull/1767 ---
[GitHub] carbondata pull request #1767: [CARBONDATA-1993] Carbon properties default v...
Github user pawanmalwal commented on a diff in the pull request: https://github.com/apache/carbondata/pull/1767#discussion_r160114212 --- Diff: conf/carbon.properties.template --- @@ -20,7 +20,7 @@ #Mandatory. Carbon Store path carbon.storelocation=hdfs://hacluster/Opt/CarbonStore #Base directory for Data files -carbon.ddl.base.hdfs.url=hdfs://hacluster/opt/data +carbon.ddl.base.hdfs.url= --- End diff -- I have commented it for now. ---
[GitHub] carbondata pull request #1775: [CARBONDATA-1993] Removed unused carbon prope...
GitHub user pawanmalwal opened a pull request: https://github.com/apache/carbondata/pull/1775 [CARBONDATA-1993] Removed unused carbon properties high.cardinality.identify⦠Removed carbon properties high.cardinality.identify.enable and high.cardinality.threshold Be sure to do all of the following checklist to help us incorporate your contribution quickly and easily: - [X] Any interfaces changed? None - [X] Any backward compatibility impacted? None - [X] Document update required? Removed references from document file - [X] Testing done Please provide details on - Whether new unit test cases have been added or why no new tests are required? - How it is tested? Please attach test report. - Is it a performance related change? Please attach the performance test report. - Any additional information to help reviewers in testing this change. NA - [X] For large changes, please consider breaking it into sub-tasks under an umbrella JIRA. None You can merge this pull request into a Git repository by running: $ git pull https://github.com/pawanmalwal/carbondata remove_high_cardinality_properties Alternatively you can review and apply these changes as the patch at: https://github.com/apache/carbondata/pull/1775.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #1775 commit 3687f6fbacf85889239b2e0a3db95d34b09b043b Author: Pawan Malwal <pmalwal1981@...> Date: 2018-01-08T10:30:47Z [CARBONDATA-1993] Removed carbon properties high.cardinality.identify.enable and high.cardinality.threshold ---
[GitHub] carbondata pull request #1767: [CARBONDATA-1993] Carbon properties default v...
GitHub user pawanmalwal opened a pull request: https://github.com/apache/carbondata/pull/1767 [CARBONDATA-1993] Carbon properties default values fix and correspond⦠Carbon properties default values fix and corresponding template and document correction. - [X] Any interfaces changed? None - [X] Any backward compatibility impacted? None - [X] Document update required? Updated corresponding template and document - [X] Testing done Please provide details on - Whether new unit test cases have been added or why no new tests are required? - How it is tested? Please attach test report. - Is it a performance related change? Please attach the performance test report. - Any additional information to help reviewers in testing this change. None - [X] For large changes, please consider breaking it into sub-tasks under an umbrella JIRA. None You can merge this pull request into a Git repository by running: $ git pull https://github.com/pawanmalwal/carbondata carbon_properties_default_value_fix Alternatively you can review and apply these changes as the patch at: https://github.com/apache/carbondata/pull/1767.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #1767 commit 72f4ebbf6ae06b0472990bd4d60704af697febef Author: Pawan Malwal <pmalwal1981@...> Date: 2018-01-05T15:56:15Z [CARBONDATA-1993] Carbon properties default values fix and corresponding and template, document correction. ---
[GitHub] carbondata issue #1696: [CARBONDATA-1884] SDV test cases for CTAS support to...
Github user pawanmalwal commented on the issue: https://github.com/apache/carbondata/pull/1696 retest sdv please ---
[GitHub] carbondata pull request #1696: [CARBONDATA-1884] SDV test cases for CTAS sup...
Github user pawanmalwal closed the pull request at: https://github.com/apache/carbondata/pull/1696 ---
[GitHub] carbondata pull request #1696: [CARBONDATA-1884] SDV test cases for CTAS sup...
GitHub user pawanmalwal reopened a pull request: https://github.com/apache/carbondata/pull/1696 [CARBONDATA-1884] SDV test cases for CTAS support to carbondata SDV test cases for CTAS support to carbondata. Be sure to do all of the following checklist to help us incorporate your contribution quickly and easily: - [X] Any interfaces changed? None - [X] Any backward compatibility impacted? NA - [X] Document update required? NA - [X] Testing done Please provide details on - Whether new unit test cases have been added or why no new tests are required? - How it is tested? Please attach test report. - Is it a performance related change? Please attach the performance test report. - Any additional information to help reviewers in testing this change. Added SDV test cases for CTAS support to carbondata - [X] For large changes, please consider breaking it into sub-tasks under an umbrella JIRA. NA You can merge this pull request into a Git repository by running: $ git pull https://github.com/pawanmalwal/carbondata sdv_Test_Cases_CreateTableAsSelect Alternatively you can review and apply these changes as the patch at: https://github.com/apache/carbondata/pull/1696.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #1696 commit e8fb2455a42eba6feff76f24b2f0b03635390056 Author: Pawan Malwal <pmalwal1981@...> Date: 2017-12-20T10:00:58Z [CARBONDATA-1884]SDV test cases for CTAS support to carbondata ---
[GitHub] carbondata pull request #1696: [CARBONDATA-1884] SDV test cases for CTAS sup...
GitHub user pawanmalwal opened a pull request: https://github.com/apache/carbondata/pull/1696 [CARBONDATA-1884] SDV test cases for CTAS support to carbondata SDV test cases for CTAS support to carbondata. Be sure to do all of the following checklist to help us incorporate your contribution quickly and easily: - [X] Any interfaces changed? None - [X] Any backward compatibility impacted? NA - [X] Document update required? NA - [X] Testing done Please provide details on - Whether new unit test cases have been added or why no new tests are required? - How it is tested? Please attach test report. - Is it a performance related change? Please attach the performance test report. - Any additional information to help reviewers in testing this change. Added SDV test cases for CTAS support to carbondata - [X] For large changes, please consider breaking it into sub-tasks under an umbrella JIRA. NA You can merge this pull request into a Git repository by running: $ git pull https://github.com/pawanmalwal/carbondata sdv_Test_Cases_CreateTableAsSelect Alternatively you can review and apply these changes as the patch at: https://github.com/apache/carbondata/pull/1696.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #1696 commit e8fb2455a42eba6feff76f24b2f0b03635390056 Author: Pawan Malwal <pmalwal1981@...> Date: 2017-12-20T10:00:58Z [CARBONDATA-1884]SDV test cases for CTAS support to carbondata ---
[GitHub] carbondata pull request #1656: [CARBONDATA-1247] Block pruning not working f...
Github user pawanmalwal commented on a diff in the pull request:
https://github.com/apache/carbondata/pull/1656#discussion_r156907109
--- Diff:
integration/spark2/src/main/scala/org/apache/spark/sql/execution/CastExpressionOptimization.scala
---
@@ -82,6 +109,21 @@ object CastExpressionOptimization {
}
}
+ def typeCastStringToLongListForDateType(list: Seq[Expression]):
Seq[Expression] = {
--- End diff --
Handled comments. Please review.
---
[GitHub] carbondata pull request #1656: [CARBONDATA-1247] Block pruning not working f...
Github user pawanmalwal commented on a diff in the pull request:
https://github.com/apache/carbondata/pull/1656#discussion_r156907140
--- Diff:
integration/spark2/src/main/scala/org/apache/spark/sql/execution/CastExpressionOptimization.scala
---
@@ -67,6 +68,32 @@ object CastExpressionOptimization {
}
}
+ def typeCastStringToLongForDateType(v: Any): Any = {
--- End diff --
Handled comments. Please review.
---
[GitHub] carbondata pull request #1656: [CARBONDATA-1247] Block pruning not working f...
Github user pawanmalwal commented on a diff in the pull request:
https://github.com/apache/carbondata/pull/1656#discussion_r156907084
--- Diff:
integration/spark2/src/main/scala/org/apache/spark/sql/execution/CastExpressionOptimization.scala
---
@@ -122,6 +164,13 @@ object CastExpressionOptimization {
} else {
Some(CastExpr(c))
}
+ case d: DateType if t.sameType(StringType) =>
--- End diff --
Handled comment. Please review.
---
[GitHub] carbondata issue #1281: [CARBONDATA-1326] Fixed findbug issue and univocity-...
Github user pawanmalwal commented on the issue: https://github.com/apache/carbondata/pull/1281 Duplicate PR - univocity jar changes done in PR https://github.com/apache/carbondata/pull/1532 ---
[GitHub] carbondata pull request #1281: [CARBONDATA-1326] Fixed findbug issue and uni...
Github user pawanmalwal closed the pull request at: https://github.com/apache/carbondata/pull/1281 ---
[GitHub] carbondata pull request #1656: [CARBONDATA-1247] Block pruning not working f...
GitHub user pawanmalwal reopened a pull request: https://github.com/apache/carbondata/pull/1656 [CARBONDATA-1247] Block pruning not working for date type column Block pruning not working for date type column. Root Cause : Type casting of String for DateType is not handled Solution: CastExpressionOptimization should handle the casting of String for DateType Be sure to do all of the following checklist to help us incorporate your contribution quickly and easily: - [X] Any interfaces changed? None - [X] Any backward compatibility impacted? None - [X] Document update required? NA - [X] Testing done Please provide details on - Whether new unit test cases have been added or why no new tests are required? - How it is tested? Please attach test report. - Is it a performance related change? Please attach the performance test report. - Any additional information to help reviewers in testing this change. Done manual testing - [ ] For large changes, please consider breaking it into sub-tasks under an umbrella JIRA. NA You can merge this pull request into a Git repository by running: $ git pull https://github.com/pawanmalwal/carbondata date_datatype_block_pruning_issue_fix Alternatively you can review and apply these changes as the patch at: https://github.com/apache/carbondata/pull/1656.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #1656 commit de83c25a03606a42eac049433798b2cfdfc7b5af Author: Pawan Malwal <pmalwal1...@gmail.com> Date: 2017-12-13T07:31:20Z [CARBONDATA-1887] Block pruning not working for date type column ---
[GitHub] carbondata pull request #1656: [CARBONDATA-1247] Block pruning not working f...
Github user pawanmalwal closed the pull request at: https://github.com/apache/carbondata/pull/1656 ---
[GitHub] carbondata pull request #1656: [CARBONDATA-1887] Block pruning not working f...
GitHub user pawanmalwal opened a pull request: https://github.com/apache/carbondata/pull/1656 [CARBONDATA-1887] Block pruning not working for date type column Block pruning not working for date type column. Root Cause : Type casting of String for DateType is not handled Solution: CastExpressionOptimization should handle the casting of String for DateType Be sure to do all of the following checklist to help us incorporate your contribution quickly and easily: - [X] Any interfaces changed? None - [X] Any backward compatibility impacted? None - [X] Document update required? NA - [X] Testing done Please provide details on - Whether new unit test cases have been added or why no new tests are required? - How it is tested? Please attach test report. - Is it a performance related change? Please attach the performance test report. - Any additional information to help reviewers in testing this change. Done manual testing - [ ] For large changes, please consider breaking it into sub-tasks under an umbrella JIRA. NA You can merge this pull request into a Git repository by running: $ git pull https://github.com/pawanmalwal/carbondata date_datatype_block_pruning_issue_fix Alternatively you can review and apply these changes as the patch at: https://github.com/apache/carbondata/pull/1656.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #1656 commit de83c25a03606a42eac049433798b2cfdfc7b5af Author: Pawan Malwal <pmalwal1...@gmail.com> Date: 2017-12-13T07:31:20Z [CARBONDATA-1887] Block pruning not working for date type column ---
[GitHub] carbondata pull request #1612: [CARBONDATA-1618] SDV test cases for Table co...
Github user pawanmalwal commented on a diff in the pull request:
https://github.com/apache/carbondata/pull/1612#discussion_r154995762
--- Diff:
integration/spark-common-cluster-test/src/test/scala/org/apache/carbondata/cluster/sdv/generated/TableCommentAlterTableTestCase.scala
---
@@ -0,0 +1,145 @@
+
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ *http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.carbondata.cluster.sdv.generated
+
+import org.apache.spark.sql.{AnalysisException, Row}
+import org.apache.spark.sql.common.util._
+import org.scalatest.BeforeAndAfterAll
+
+/**
+ * Test Class for AlterTableTestCase to verify all scenerios
+ */
+
+class TableCommentAlterTableTestCase extends QueryTest with
BeforeAndAfterAll {
--- End diff --
Added. Please review..
---
[GitHub] carbondata issue #1612: [CARBONDATA-1618] SDV test cases for Table comment s...
Github user pawanmalwal commented on the issue: https://github.com/apache/carbondata/pull/1612 retest this please ---
[GitHub] carbondata pull request #1612: [CARBONDATA-1618] SDV test cases for Table co...
GitHub user pawanmalwal opened a pull request: https://github.com/apache/carbondata/pull/1612 [CARBONDATA-1618] SDV test cases for Table comment support for alter t⦠SDV test cases for Table comment support for alter table. Be sure to do all of the following checklist to help us incorporate your contribution quickly and easily: - [X] Any interfaces changed? None - [X] Any backward compatibility impacted? None - [X] Document update required? NA - [X] Testing done - Added SDV test cases to accommodate all the test scenarios - [X] For large changes, please consider breaking it into sub-tasks under an umbrella JIRA. NA You can merge this pull request into a Git repository by running: $ git pull https://github.com/pawanmalwal/carbondata sdv_testcases_table_comment_AlterTable Alternatively you can review and apply these changes as the patch at: https://github.com/apache/carbondata/pull/1612.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #1612 commit aa6e65611c673344b24e12701611d475a78eddbb Author: Pawan Malwal <pmalwal1...@gmail.com> Date: 2017-12-05T07:25:46Z [CARBONDATA-1618]SDV test cases for Table comment support for alter table feature ---
[GitHub] carbondata pull request #1371: [WIP][CARBONDATA-1440] Fixed fortify issues
Github user pawanmalwal closed the pull request at: https://github.com/apache/carbondata/pull/1371 ---
[GitHub] carbondata issue #1472: [CARBONDATA-1618]Table comment support for alter tab...
Github user pawanmalwal commented on the issue: https://github.com/apache/carbondata/pull/1472 @ravipesala I have resolved the conflict. Due to that SDV had failed I guess. ---
[GitHub] carbondata pull request #1472: [CARBONDATA-1618]Table comment support for al...
Github user pawanmalwal commented on a diff in the pull request: https://github.com/apache/carbondata/pull/1472#discussion_r14959 --- Diff: core/src/main/java/org/apache/carbondata/core/constants/CarbonCommonConstants.java --- @@ -1374,6 +1374,12 @@ @CarbonProperty public static final String BITSET_PIPE_LINE = "carbon.use.bitset.pipe.line"; + /** + * this will be used to provide comment for table + */ + @CarbonProperty --- End diff -- Done. ---
[GitHub] carbondata pull request #1472: [CARBONDATA-1618]Table comment support for al...
Github user pawanmalwal commented on a diff in the pull request:
https://github.com/apache/carbondata/pull/1472#discussion_r149593291
--- Diff:
integration/spark2/src/main/scala/org/apache/spark/util/AlterTableUtil.scala ---
@@ -317,4 +321,84 @@ object AlterTableUtil {
}
}
+ /**
+ * This method add/modify the table comments.
+ *
+ * @param tableIdentifier
+ * @param properties
+ * @param propKeys
+ * @param set
+ * @param sparkSession
+ * @param sessionState
+ */
+ def modifyTableComment(tableIdentifier: TableIdentifier, properties:
Map[String, String],
+ propKeys: Seq[String], set: Boolean)
+(sparkSession: SparkSession, sessionState:
CarbonSessionState): Unit = {
+val tableName = tableIdentifier.table
+val dbName =
tableIdentifier.database.getOrElse(sparkSession.catalog.currentDatabase)
+LOGGER.audit(s"Alter table comment request has been received for
$dbName.$tableName")
+val locksToBeAcquired = List(LockUsage.METADATA_LOCK,
LockUsage.COMPACTION_LOCK)
+var locks = List.empty[ICarbonLock]
+var timeStamp = 0L
+var newCols =
Seq[org.apache.carbondata.core.metadata.schema.table.column.ColumnSchema]()
+var carbonTable: CarbonTable = null
+try {
+ locks = AlterTableUtil
+.validateTableAndAcquireLock(dbName, tableName,
locksToBeAcquired)(sparkSession)
+ val metastore = CarbonEnv.getInstance(sparkSession).carbonMetastore
+ carbonTable = metastore
+.lookupRelation(Some(dbName),
tableName)(sparkSession).asInstanceOf[CarbonRelation]
+.tableMeta.carbonTable
+ // get the latest carbon table
+ // read the latest schema file
+ val carbonTablePath =
CarbonStorePath.getCarbonTablePath(carbonTable.getStorePath,
+carbonTable.getCarbonTableIdentifier)
+ val thriftTableInfo: TableInfo =
metastore.getThriftTableInfo(carbonTablePath)(sparkSession)
+ val schemaConverter = new ThriftWrapperSchemaConverterImpl()
+ val wrapperTableInfo = schemaConverter
+.fromExternalToWrapperTableInfo(thriftTableInfo,
+ dbName,
+ tableName,
+ carbonTable.getStorePath)
+ val schemaEvolutionEntry = new
org.apache.carbondata.core.metadata.schema.SchemaEvolutionEntry
+ schemaEvolutionEntry.setTimeStamp(timeStamp)
+ val thriftTable = schemaConverter
+.fromWrapperToExternalTableInfo(wrapperTableInfo, dbName,
tableName)
+ val tblPropertiesMap: mutable.Map[String, String] =
+thriftTable.fact_table.getTableProperties.asScala
+ if (set) {
+// This overrides old properties and update the comment
parameter of thriftTable
+// with the newly added/modified comment since thriftTable
also holds comment as its
+// direct property.
+
+properties.foreach {
+ x =>
--- End diff --
Done.
---
[GitHub] carbondata pull request #1472: [CARBONDATA-1618]Table comment support for al...
Github user pawanmalwal commented on a diff in the pull request:
https://github.com/apache/carbondata/pull/1472#discussion_r149593267
--- Diff:
integration/spark2/src/main/scala/org/apache/spark/util/AlterTableUtil.scala ---
@@ -317,4 +321,84 @@ object AlterTableUtil {
}
}
+ /**
+ * This method add/modify the table comments.
+ *
+ * @param tableIdentifier
+ * @param properties
+ * @param propKeys
+ * @param set
+ * @param sparkSession
+ * @param sessionState
+ */
+ def modifyTableComment(tableIdentifier: TableIdentifier, properties:
Map[String, String],
+ propKeys: Seq[String], set: Boolean)
+(sparkSession: SparkSession, sessionState:
CarbonSessionState): Unit = {
+val tableName = tableIdentifier.table
+val dbName =
tableIdentifier.database.getOrElse(sparkSession.catalog.currentDatabase)
+LOGGER.audit(s"Alter table comment request has been received for
$dbName.$tableName")
+val locksToBeAcquired = List(LockUsage.METADATA_LOCK,
LockUsage.COMPACTION_LOCK)
+var locks = List.empty[ICarbonLock]
+var timeStamp = 0L
+var newCols =
Seq[org.apache.carbondata.core.metadata.schema.table.column.ColumnSchema]()
+var carbonTable: CarbonTable = null
+try {
+ locks = AlterTableUtil
+.validateTableAndAcquireLock(dbName, tableName,
locksToBeAcquired)(sparkSession)
+ val metastore = CarbonEnv.getInstance(sparkSession).carbonMetastore
+ carbonTable = metastore
+.lookupRelation(Some(dbName),
tableName)(sparkSession).asInstanceOf[CarbonRelation]
+.tableMeta.carbonTable
+ // get the latest carbon table
+ // read the latest schema file
+ val carbonTablePath =
CarbonStorePath.getCarbonTablePath(carbonTable.getStorePath,
+carbonTable.getCarbonTableIdentifier)
+ val thriftTableInfo: TableInfo =
metastore.getThriftTableInfo(carbonTablePath)(sparkSession)
+ val schemaConverter = new ThriftWrapperSchemaConverterImpl()
+ val wrapperTableInfo = schemaConverter
+.fromExternalToWrapperTableInfo(thriftTableInfo,
+ dbName,
+ tableName,
+ carbonTable.getStorePath)
+ val schemaEvolutionEntry = new
org.apache.carbondata.core.metadata.schema.SchemaEvolutionEntry
+ schemaEvolutionEntry.setTimeStamp(timeStamp)
+ val thriftTable = schemaConverter
+.fromWrapperToExternalTableInfo(wrapperTableInfo, dbName,
tableName)
+ val tblPropertiesMap: mutable.Map[String, String] =
+thriftTable.fact_table.getTableProperties.asScala
+ if (set) {
+// This overrides old properties and update the comment
parameter of thriftTable
+// with the newly added/modified comment since thriftTable
also holds comment as its
+// direct property.
+
+properties.foreach {
+ x =>
+if
(x._1.equalsIgnoreCase(CarbonCommonConstants.TABLE_COMMENT)) {
+ tblPropertiesMap.put(x._1, x._2)
+}
+}
+ } else {
+// This removes the comment parameter from thriftTable
+// since thriftTable also holds comment as its property.
+propKeys.foreach {
+ x =>
--- End diff --
Done.
---
[GitHub] carbondata pull request #1437: [CARBONDATA-1618] Fix issue of not support ta...
Github user pawanmalwal commented on a diff in the pull request: https://github.com/apache/carbondata/pull/1437#discussion_r149586320 --- Diff: core/src/main/java/org/apache/carbondata/core/metadata/schema/table/TableInfo.java --- @@ -79,6 +79,9 @@ // this idenifier is a lazy field which will be created when it is used first time private AbsoluteTableIdentifier identifier; + // table comment + private String tableComment; --- End diff -- Handled it in https://github.com/apache/carbondata/pull/1472 ---
[GitHub] carbondata pull request #1472: [WIP][CARBONDATA-1618]Table comment support f...
Github user pawanmalwal commented on a diff in the pull request:
https://github.com/apache/carbondata/pull/1472#discussion_r149585944
--- Diff:
integration/spark2/src/main/scala/org/apache/spark/sql/execution/command/schema/AlterTableSetCommand.scala
---
@@ -0,0 +1,108 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ *http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.spark.sql.execution.command.schema
+
+import scala.collection.JavaConverters._
+import scala.collection.mutable
+
+import org.apache.spark.sql.{CarbonEnv, Row, SparkSession}
+import org.apache.spark.sql.catalyst.TableIdentifier
+import org.apache.spark.sql.execution.command._
+import org.apache.spark.sql.hive.{CarbonRelation, CarbonSessionState}
+import org.apache.spark.util.AlterTableUtil
+
+import org.apache.carbondata.common.logging.{LogService, LogServiceFactory}
+import org.apache.carbondata.core.locks.{ICarbonLock, LockUsage}
+import
org.apache.carbondata.core.metadata.converter.ThriftWrapperSchemaConverterImpl
+import org.apache.carbondata.core.metadata.schema.table.CarbonTable
+import org.apache.carbondata.core.util.path.CarbonStorePath
+import org.apache.carbondata.format.TableInfo
+import org.apache.carbondata.spark.rdd.{AlterTableAddColumnRDD,
AlterTableDropColumnRDD}
+
+private[sql] case class AlterTableSetCommand(val tableIdentifier:
TableIdentifier,
+ val properties: Map[String,
String],
+ val isView: Boolean)
+ extends RunnableCommand with SchemaProcessCommand {
+
+ override def run(sparkSession: SparkSession): Seq[Row] = {
+processSchema(sparkSession)
+ }
+
+ override def processSchema(sparkSession: SparkSession): Seq[Row] = {
+val LOGGER: LogService =
LogServiceFactory.getLogService(this.getClass.getCanonicalName)
--- End diff --
Handled the comment please review.
---
[GitHub] carbondata pull request #1472: [WIP][CARBONDATA-1618]Table comment support f...
Github user pawanmalwal commented on a diff in the pull request:
https://github.com/apache/carbondata/pull/1472#discussion_r149586000
--- Diff:
integration/spark2/src/main/scala/org/apache/spark/sql/execution/command/CarbonDescribeFormattedCommand.scala
---
@@ -105,6 +105,10 @@ private[sql] case class CarbonDescribeFormattedCommand(
results ++= Seq(("Table Name: ",
relation.tableMeta.carbonTableIdentifier.getTableName, ""))
results ++= Seq(("CARBON Store Path: ", relation.tableMeta.storePath,
""))
val carbonTable = relation.tableMeta.carbonTable
+// Carbon table support table comment
+val tableComment =
carbonTable.getTableInfo.getFactTable.getTableProperties
+ .getOrDefault("comment", "")
--- End diff --
Done.
---
[GitHub] carbondata pull request #1472: [WIP][CARBONDATA-1618]Table comment support f...
Github user pawanmalwal commented on a diff in the pull request:
https://github.com/apache/carbondata/pull/1472#discussion_r149585846
--- Diff:
integration/spark2/src/main/scala/org/apache/spark/sql/execution/command/schema/AlterTableSetCommand.scala
---
@@ -0,0 +1,108 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ *http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.spark.sql.execution.command.schema
+
+import scala.collection.JavaConverters._
+import scala.collection.mutable
+
+import org.apache.spark.sql.{CarbonEnv, Row, SparkSession}
+import org.apache.spark.sql.catalyst.TableIdentifier
+import org.apache.spark.sql.execution.command._
+import org.apache.spark.sql.hive.{CarbonRelation, CarbonSessionState}
+import org.apache.spark.util.AlterTableUtil
+
+import org.apache.carbondata.common.logging.{LogService, LogServiceFactory}
+import org.apache.carbondata.core.locks.{ICarbonLock, LockUsage}
+import
org.apache.carbondata.core.metadata.converter.ThriftWrapperSchemaConverterImpl
+import org.apache.carbondata.core.metadata.schema.table.CarbonTable
+import org.apache.carbondata.core.util.path.CarbonStorePath
+import org.apache.carbondata.format.TableInfo
+import org.apache.carbondata.spark.rdd.{AlterTableAddColumnRDD,
AlterTableDropColumnRDD}
+
+private[sql] case class AlterTableSetCommand(val tableIdentifier:
TableIdentifier,
+ val properties: Map[String,
String],
+ val isView: Boolean)
+ extends RunnableCommand with SchemaProcessCommand {
+
+ override def run(sparkSession: SparkSession): Seq[Row] = {
+processSchema(sparkSession)
+ }
+
+ override def processSchema(sparkSession: SparkSession): Seq[Row] = {
+val LOGGER: LogService =
LogServiceFactory.getLogService(this.getClass.getCanonicalName)
--- End diff --
Handled the comment. Please review.
---
[GitHub] carbondata pull request #1371: [WIP][CARBONDATA-1440] Fixed fortify issues
GitHub user pawanmalwal opened a pull request: https://github.com/apache/carbondata/pull/1371 [WIP][CARBONDATA-1440] Fixed fortify issues You can merge this pull request into a Git repository by running: $ git pull https://github.com/pawanmalwal/carbondata fortify_issue_branch Alternatively you can review and apply these changes as the patch at: https://github.com/apache/carbondata/pull/1371.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #1371 commit ab79e08b06a994446b3cc97d1f5f170cfc01842c Author: Pawan Malwal <pmalwal1...@gmail.com> Date: 2017-09-12T09:35:06Z [WIP][CARBONDATA-1440] Fixed fortify issues ---
[GitHub] carbondata issue #1281: [CARBONDATA-1326] Fixed findbug issue and univocity-...
Github user pawanmalwal commented on the issue: https://github.com/apache/carbondata/pull/1281 retest this please ---
[GitHub] carbondata issue #1281: [CARBONDATA-1326] Fixed findbug issue and univocity-...
Github user pawanmalwal commented on the issue: https://github.com/apache/carbondata/pull/1281 retest this please ---
[GitHub] carbondata pull request #1281: [CARBONDATA-1326] Fixed findbug issue and uni...
GitHub user pawanmalwal reopened a pull request: https://github.com/apache/carbondata/pull/1281 [CARBONDATA-1326] Fixed findbug issue and univocity-parsers jar version to 2.2.1 Updated univocity-parsers jar version from 1.5.6 to 2.2.1 in processing/pom.xml and fixed one findbug issue. You can merge this pull request into a Git repository by running: $ git pull https://github.com/pawanmalwal/carbondata findbug_univocity_pom_changes Alternatively you can review and apply these changes as the patch at: https://github.com/apache/carbondata/pull/1281.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #1281 commit 2a36440af46fc9ed361109990dc9a7c8f367e19e Author: Pawan Malwal <pmalwal1...@gmail.com> Date: 2017-08-22T10:19:00Z [CARBONDATA-1326] Fixed findbug issue and univocity-parsers jar version to 2.2.1 ---
[GitHub] carbondata pull request #1281: [CARBONDATA-1326] Fixed findbug issue and uni...
Github user pawanmalwal closed the pull request at: https://github.com/apache/carbondata/pull/1281 ---
[GitHub] carbondata issue #1281: [CARBONDATA-1326] Fixed findbug issue and univocity-...
Github user pawanmalwal commented on the issue: https://github.com/apache/carbondata/pull/1281 Yes thats right. This PR also has some code change to avoid 1 more string object creation in the same method and another chanage to update univocity-parsers jar version to 2.2.1 in processing/pom.xml. Please review. ---
[GitHub] carbondata pull request #1281: [CARBONDATA-1326] Fixed findbug issue and uni...
Github user pawanmalwal commented on a diff in the pull request:
https://github.com/apache/carbondata/pull/1281#discussion_r134990100
--- Diff:
core/src/main/java/org/apache/carbondata/core/scan/filter/FilterExpressionProcessor.java
---
@@ -163,17 +163,17 @@ public FilterResolverIntf
getFilterResolver(Expression expressionTree,
}
private String joinByteArray(byte[] bytes, char delimiter) {
-StringBuffer byteArrayAsString = new StringBuffer();
-byteArrayAsString.append("");
+String byteArrayAsString = "";
--- End diff --
Changed to StringBuffer per your suggestion.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] carbondata pull request #1281: [CARBONDATA-1326] Fixed findbug issue and uni...
GitHub user pawanmalwal opened a pull request: https://github.com/apache/carbondata/pull/1281 [CARBONDATA-1326] Fixed findbug issue and univocity-parsers jar version to 2.2.1 Updated univocity-parsers jar version from 1.5.6 to 2.2.1 in processing/pom.xml and fixed one findbug issue. You can merge this pull request into a Git repository by running: $ git pull https://github.com/pawanmalwal/carbondata findbug_univocity_pom_changes Alternatively you can review and apply these changes as the patch at: https://github.com/apache/carbondata/pull/1281.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #1281 commit 218a4d8b44ae5efcc3d591358d5bba7ab813a68f Author: Pawan Malwal <pmalwal1...@gmail.com> Date: 2017-08-22T10:19:00Z [CARBONDATA-1326] Fixed findbug issue and univocity-parsers jar version to 2.2.1 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
