[GitHub] carbondata pull request #1736: [CARBONDATA-1904][CARBONDATA-1905] Support au...

2017-12-29 Thread watermen
Github user watermen commented on a diff in the pull request: https://github.com/apache/carbondata/pull/1736#discussion_r159039529 --- Diff: streaming/src/main/scala/org/apache/spark/sql/execution/streaming/CarbonAppendableStreamSink.scala --- @@ -80,6 +80,12 @@ class

[GitHub] carbondata pull request #1736: [CARBONDATA-1904][CARBONDATA-1905] Support au...

2017-12-28 Thread watermen
Github user watermen commented on a diff in the pull request: https://github.com/apache/carbondata/pull/1736#discussion_r159030981 --- Diff: core/src/main/java/org/apache/carbondata/core/constants/CarbonCommonConstants.java --- @@ -1459,6 +1459,13 @@ */ public

[GitHub] carbondata pull request #1665: [CARBONDATA-1884] Add CTAS support to carbond...

2017-12-17 Thread watermen
Github user watermen commented on a diff in the pull request: https://github.com/apache/carbondata/pull/1665#discussion_r157410277 --- Diff: integration/spark2/src/main/spark2.2/CarbonSessionState.scala --- @@ -280,25 +280,26 @@ class CarbonOptimizer

[GitHub] carbondata pull request #1665: [CARBONDATA-1884] Add CTAS support to carbond...

2017-12-17 Thread watermen
Github user watermen commented on a diff in the pull request: https://github.com/apache/carbondata/pull/1665#discussion_r157409856 --- Diff: integration/spark2/src/main/spark2.1/CarbonSessionState.scala --- @@ -259,25 +259,26 @@ object CarbonOptimizerUtil

[GitHub] carbondata pull request #1665: [CARBONDATA-1884] Add CTAS support to carbond...

2017-12-17 Thread watermen
Github user watermen commented on a diff in the pull request: https://github.com/apache/carbondata/pull/1665#discussion_r157408233 --- Diff: integration/spark2/src/main/scala/org/apache/spark/sql/execution/command/table/CarbonCreateTableAsSelectCommand.scala --- @@ -0,0 +1,86

[GitHub] carbondata pull request #863: [CARBONDATA-998] Don't request executors when ...

2017-12-14 Thread watermen
Github user watermen closed the pull request at: https://github.com/apache/carbondata/pull/863 ---

[GitHub] carbondata pull request #606: [CARBONDATA-713] Make the store path in right ...

2017-12-14 Thread watermen
Github user watermen closed the pull request at: https://github.com/apache/carbondata/pull/606 ---

[GitHub] carbondata issue #1245: [CARBONDATA-1366]Change rdd storage level to 'MEMORY...

2017-08-08 Thread watermen
Github user watermen commented on the issue: https://github.com/apache/carbondata/pull/1245 @jackylk Agreed with @xuchuanyin, Spark’s storage levels are meant to provide different trade-offs between memory usage and CPU efficiency. So different environment correspond to different

[GitHub] carbondata issue #1223: [WIP] Support cleaning garbage segment in all tables

2017-08-02 Thread watermen
Github user watermen commented on the issue: https://github.com/apache/carbondata/pull/1223 LGTM --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so

[GitHub] carbondata pull request #1037: [CARBONDATA-1163][Bug-Fix] Rollback the code ...

2017-06-15 Thread watermen
GitHub user watermen opened a pull request: https://github.com/apache/carbondata/pull/1037 [CARBONDATA-1163][Bug-Fix] Rollback the code because it can't reuse the convertRow You can merge this pull request into a Git repository by running: $ git pull https://github.com

[GitHub] carbondata issue #972: [CARBONDATA-1065] Added set command in carbon to upda...

2017-06-14 Thread watermen
Github user watermen commented on the issue: https://github.com/apache/carbondata/pull/972 @mohammadshahidkhan But we'd better to follow the habit of users who is familiar with spark/hive/hdfs. --- If your project is set up for it, you can reply to this email and have your reply

[GitHub] carbondata issue #972: [CARBONDATA-1065] Added set command in carbon to upda...

2017-06-14 Thread watermen
Github user watermen commented on the issue: https://github.com/apache/carbondata/pull/972 I think we'd better to use `SET carbon.load.sort.scope = LOCAL_SORT` insteads of `SET SORT_SCOPE = LOCAL_SORT`, @mohammadshahidkhan can you explain the reason? cc @jackylk --- If your

[GitHub] carbondata issue #1000: [CARBONDATA-1018] Add unsafe ColumnPage implementati...

2017-06-14 Thread watermen
Github user watermen commented on the issue: https://github.com/apache/carbondata/pull/1000 LGTM --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so

[GitHub] carbondata pull request #1000: [CARBONDATA-1018] Add unsafe ColumnPage imple...

2017-06-14 Thread watermen
Github user watermen commented on a diff in the pull request: https://github.com/apache/carbondata/pull/1000#discussion_r121876896 --- Diff: core/src/main/java/org/apache/carbondata/core/datastore/page/ColumnPage.java --- @@ -323,15 +340,8 @@ public double getDouble(int rowId

[GitHub] carbondata pull request #1000: [CARBONDATA-1018] Add unsafe ColumnPage imple...

2017-06-14 Thread watermen
Github user watermen commented on a diff in the pull request: https://github.com/apache/carbondata/pull/1000#discussion_r121875823 --- Diff: core/src/main/java/org/apache/carbondata/core/datastore/page/ColumnPage.java --- @@ -508,7 +567,7 @@ public static ColumnPage decompress

[GitHub] carbondata pull request #1000: [CARBONDATA-1018] Add unsafe ColumnPage imple...

2017-06-14 Thread watermen
Github user watermen commented on a diff in the pull request: https://github.com/apache/carbondata/pull/1000#discussion_r121874665 --- Diff: core/src/main/java/org/apache/carbondata/core/datastore/page/ColumnPage.java --- @@ -98,56 +117,54 @@ public static ColumnPage newPage

[GitHub] carbondata pull request #910: [WIP] Global sort by spark in load process

2017-06-12 Thread watermen
Github user watermen commented on a diff in the pull request: https://github.com/apache/carbondata/pull/910#discussion_r121315052 --- Diff: processing/src/main/java/org/apache/carbondata/processing/newflow/sort/SortHelper.java --- @@ -0,0 +1,85 @@ +/* + * Licensed

[GitHub] carbondata pull request #910: [WIP] Global sort by spark in load process

2017-06-11 Thread watermen
Github user watermen commented on a diff in the pull request: https://github.com/apache/carbondata/pull/910#discussion_r121306203 --- Diff: integration/spark-common/src/main/scala/org/apache/carbondata/spark/load/GlobalSort.scala --- @@ -0,0 +1,152 @@ +/* + * Licensed

[GitHub] carbondata pull request #910: [WIP] Global sort by spark in load process

2017-06-11 Thread watermen
Github user watermen commented on a diff in the pull request: https://github.com/apache/carbondata/pull/910#discussion_r121305832 --- Diff: integration/spark-common/src/main/scala/org/apache/carbondata/spark/load/GlobalSortOperates.scala --- @@ -0,0 +1,242

[GitHub] carbondata pull request #910: [WIP] Global sort by spark in load process

2017-06-11 Thread watermen
Github user watermen commented on a diff in the pull request: https://github.com/apache/carbondata/pull/910#discussion_r121305557 --- Diff: integration/spark-common/src/main/scala/org/apache/carbondata/spark/load/GlobalSortOperates.scala --- @@ -0,0 +1,242

[GitHub] carbondata pull request #910: [WIP] Global sort by spark in load process

2017-06-11 Thread watermen
Github user watermen commented on a diff in the pull request: https://github.com/apache/carbondata/pull/910#discussion_r121305190 --- Diff: integration/spark-common/src/main/scala/org/apache/carbondata/spark/load/GlobalSort.scala --- @@ -0,0 +1,152 @@ +/* + * Licensed

[GitHub] carbondata pull request #910: [WIP] Global sort by spark in load process

2017-06-11 Thread watermen
Github user watermen commented on a diff in the pull request: https://github.com/apache/carbondata/pull/910#discussion_r121304371 --- Diff: integration/spark-common/src/main/scala/org/apache/carbondata/spark/load/GlobalSortHelper.scala --- @@ -0,0 +1,53

[GitHub] carbondata issue #978: [CARBONDATA-1109] Acquire semaphore before submit a p...

2017-06-02 Thread watermen
Github user watermen commented on the issue: https://github.com/apache/carbondata/pull/978 retest this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes

[GitHub] carbondata issue #978: [CARBONDATA-1109] Cover the case when last page is no...

2017-06-01 Thread watermen
Github user watermen commented on the issue: https://github.com/apache/carbondata/pull/978 @ravipesala Thanks for your solution, PR updated. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] carbondata issue #978: [CARBONDATA-1109] Cover the case when last page is no...

2017-06-01 Thread watermen
Github user watermen commented on the issue: https://github.com/apache/carbondata/pull/978 @ravipesala You can reproduce this case by add some log and run loading sample.csv testcase(TestDataLoadWithFileName). It's hard to reproducer, so I add sleep in Producer to simulate

[GitHub] carbondata issue #978: [CARBONDATA-1109] Cover the case when last page is no...

2017-05-31 Thread watermen
Github user watermen commented on the issue: https://github.com/apache/carbondata/pull/978 @kumarvishal09 It is very hard to write the testcase because this case is happened occasionally. --- If your project is set up for it, you can reply to this email and have your reply appear

[GitHub] carbondata issue #978: [CARBONDATA-1109] Cover the case when last page is no...

2017-05-31 Thread watermen
Github user watermen commented on the issue: https://github.com/apache/carbondata/pull/978 retest this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes

[GitHub] carbondata issue #978: [CARBONDATA-1109] Cover the case when last page is no...

2017-05-31 Thread watermen
Github user watermen commented on the issue: https://github.com/apache/carbondata/pull/978 retest this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes

[GitHub] carbondata issue #978: [CARBONDATA-1109] Cover the case when last page is no...

2017-05-31 Thread watermen
Github user watermen commented on the issue: https://github.com/apache/carbondata/pull/978 test it please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so

[GitHub] carbondata pull request #719: [WIP][CARBONDATA-844] Avoid to get useless spl...

2017-05-31 Thread watermen
Github user watermen closed the pull request at: https://github.com/apache/carbondata/pull/719 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature

[GitHub] incubator-carbondata pull request #863: [CARBONDATA-998] Don't request execu...

2017-04-27 Thread watermen
GitHub user watermen opened a pull request: https://github.com/apache/incubator-carbondata/pull/863 [CARBONDATA-998] Don't request executors when we use carbon distribution In the current implementation, carbon request executors if it need more executors. This brings the following

[GitHub] incubator-carbondata issue #659: [CARBONDATA-781] Reuse SegmentProperties ob...

2017-04-24 Thread watermen
Github user watermen commented on the issue: https://github.com/apache/incubator-carbondata/pull/659 @jackylk @kumarvishal09 Can you review the code again? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project