[GitHub] carbondata pull request #3032: [CARBONDATA-3210] Merge common method into Ca...
Github user xiaohui0318 commented on a diff in the pull request:
https://github.com/apache/carbondata/pull/3032#discussion_r246029848
--- Diff:
integration/spark2/src/main/scala/org/apache/carbondata/spark/thriftserver/CarbonThriftServer.scala
---
@@ -28,12 +28,13 @@ import org.slf4j.{Logger, LoggerFactory}
import org.apache.carbondata.common.logging.LogServiceFactory
import org.apache.carbondata.core.constants.CarbonCommonConstants
import org.apache.carbondata.core.util.CarbonProperties
+import org.apache.carbondata.spark.util.CarbonSparkUtil
-/**
- * CarbonThriftServer support different modes:
- * 1. read/write data from/to HDFS or local,it only needs configurate
storePath
- * 2. read/write data from/to S3, it needs provide access-key, secret-key,
s3-endpoint
- */
+ /**
+ * CarbonThriftServer support different modes:
+ * 1. read/write data from/to HDFS or local,it only needs configurate
storePath
+ * 2. read/write data from/to S3, it needs provide access-key,
secret-key, s3-endpoint
+ */
object CarbonThriftServer {
--- End diff --
done
---
[GitHub] carbondata pull request #3032: [CARBONDATA-3210] Merge common method into Ca...
Github user xiaohui0318 commented on a diff in the pull request:
https://github.com/apache/carbondata/pull/3032#discussion_r246029862
--- Diff:
examples/spark2/src/main/scala/org/apache/carbondata/examples/S3UsingSDkExample.scala
---
@@ -16,28 +16,26 @@
*/
package org.apache.carbondata.examples
-import org.apache.hadoop.conf.Configuration
-import org.apache.hadoop.fs.s3a.Constants.{ACCESS_KEY, ENDPOINT,
SECRET_KEY}
import org.apache.spark.sql.SparkSession
import org.slf4j.{Logger, LoggerFactory}
-import org.apache.carbondata.core.constants.CarbonCommonConstants
import org.apache.carbondata.core.metadata.datatype.DataTypes
import org.apache.carbondata.sdk.file.{CarbonWriter, Field, Schema}
+import org.apache.carbondata.spark.util.CarbonSparkUtil
/**
* Generate data and write data to S3
* User can generate different numbers of data by specifying the
number-of-rows in parameters
*/
-object S3UsingSDKExample {
+object S3UsingSdkExample {
--- End diff --
done
---
[GitHub] carbondata pull request #3032: [CARBONDATA-3210] Merge common method into Ca...
Github user xiaohui0318 commented on a diff in the pull request:
https://github.com/apache/carbondata/pull/3032#discussion_r246029819
--- Diff:
examples/spark2/src/main/scala/org/apache/carbondata/examples/S3Example.scala
---
@@ -18,11 +18,10 @@ package org.apache.carbondata.examples
import java.io.File
-import org.apache.hadoop.fs.s3a.Constants.{ACCESS_KEY, ENDPOINT,
SECRET_KEY}
import org.apache.spark.sql.{Row, SparkSession}
import org.slf4j.{Logger, LoggerFactory}
-import org.apache.carbondata.core.constants.CarbonCommonConstants
+import org.apache.carbondata.spark.util.CarbonSparkUtil
object S3Example {
--- End diff --
done
---
[GitHub] carbondata pull request #3032: [CARBONDATA-3210] Merge common method into Ca...
Github user xiaohui0318 commented on a diff in the pull request:
https://github.com/apache/carbondata/pull/3032#discussion_r245852864
--- Diff:
examples/spark2/src/main/scala/org/apache/carbondata/examples/S3Example.scala
---
@@ -18,52 +18,50 @@ package org.apache.carbondata.examples
import java.io.File
-import org.apache.hadoop.fs.s3a.Constants.{ACCESS_KEY, ENDPOINT,
SECRET_KEY}
import org.apache.spark.sql.{Row, SparkSession}
import org.slf4j.{Logger, LoggerFactory}
-import org.apache.carbondata.core.constants.CarbonCommonConstants
+import org.apache.carbondata.spark.util.CarbonSparkUtil
object S3Example {
- /**
- * This example demonstrate usage of
- * 1. create carbon table with storage location on object based storage
- * like AWS S3, Huawei OBS, etc
- * 2. load data into carbon table, the generated file will be stored on
object based storage
- * query the table.
- *
- * @param args require three parameters "Access-key" "Secret-key"
- * "table-path on s3" "s3-endpoint" "spark-master"
- */
+ /**
+* This example demonstrate usage of
+* 1. create carbon table with storage location on object based storage
+* like AWS S3, Huawei OBS, etc
+* 2. load data into carbon table, the generated file will be stored on
object based storage
+* query the table.
+*
+* @param args require three parameters "Access-key" "Secret-key"
+* "table-path on s3" "s3-endpoint" "spark-master"
+*/
--- End diff --
checked and fix
---
[GitHub] carbondata issue #3032: [CARBONDATA-3210] Merge common method into CarbonSpa...
Github user xiaohui0318 commented on the issue: https://github.com/apache/carbondata/pull/3032 fix already.tks ---
[GitHub] carbondata issue #3032: [CARBONDATA-3210] Merge common method into CarbonSpa...
Github user xiaohui0318 commented on the issue: https://github.com/apache/carbondata/pull/3032 > Be sure to do all of the following checklist to help us incorporate > your contribution quickly and easily: > > * [x] Any interfaces changed? NO > * [x] Any backward compatibility impacted? NO > * [x] Document update required? NO > * [x] Testing done NO,only fix style error > * [x] For large changes, please consider breaking it into sub-tasks under an umbrella JIRA. NO ---
[GitHub] carbondata pull request #3032: [CARBONDATA-3210] merge getKeyOnPrefix into C...
GitHub user xiaohui0318 opened a pull request: https://github.com/apache/carbondata/pull/3032 [CARBONDATA-3210] merge getKeyOnPrefix into CarbonSparkUtil Be sure to do all of the following checklist to help us incorporate your contribution quickly and easily: - [ ] Any interfaces changed? - [ ] Any backward compatibility impacted? - [ ] Document update required? - [ ] Testing done Please provide details on - Whether new unit test cases have been added or why no new tests are required? - How it is tested? Please attach test report. - Is it a performance related change? Please attach the performance test report. - Any additional information to help reviewers in testing this change. - [ ] For large changes, please consider breaking it into sub-tasks under an umbrella JIRA. You can merge this pull request into a Git repository by running: $ git pull https://github.com/xiaohui0318/carbondata master Alternatively you can review and apply these changes as the patch at: https://github.com/apache/carbondata/pull/3032.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #3032 commit b93f75f8a150c7cc9971c68a219ae38c31f22a3f Author: xiaohui0318 <245300759@...> Date: 2018-12-28T06:08:54Z test commit commit c34e88a9ad24773ec35cdefbeb7156268176b27b Author: xiaohui0318 <245300759@...> Date: 2018-12-28T08:23:13Z org.apache.carbondata.examples.S3UsingSDKExample#getKeyOnPrefix org.apache.carbondata.examples.S3Example$#getKeyOnPrefix org.apache.carbondata.spark.thriftserver.CarbonThriftServer#getKeyOnPrefix è¿ä¸ä¸ªç±»ä¸çæ¹æ³getKeyOnPrefix å并å°spark2/src/main/scala/org/apache/carbondata/spark/util/CarbonSparkUtil.scala ---
