[GitHub] carbondata issue #2830: [CARBONDATA-3025]Added CLI enhancements
Github user CarbonDataQA commented on the issue: https://github.com/apache/carbondata/pull/2830 Build Success with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder2.1/947/ ---
[GitHub] carbondata issue #2829: [CARBONDATA-3025]add more metadata in carbon file fo...
Github user CarbonDataQA commented on the issue: https://github.com/apache/carbondata/pull/2829 Build Success with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder2.1/946/ ---
[GitHub] carbondata issue #2843: [CARBONDATA-3034] Carding parameters,Organized by pa...
Github user CarbonDataQA commented on the issue: https://github.com/apache/carbondata/pull/2843 Build Success with Spark 2.3.1, Please check CI http://136.243.101.176:8080/job/carbondataprbuilder2.3/9209/ ---
[GitHub] carbondata issue #2839: [CARBONDATA-3030] Remove no use parameter in test ca...
Github user CarbonDataQA commented on the issue: https://github.com/apache/carbondata/pull/2839 Build Success with Spark 2.3.1, Please check CI http://136.243.101.176:8080/job/carbondataprbuilder2.3/9208/ ---
[GitHub] carbondata issue #2826: [CARBONDATA-3023] Alter add column issue with SORT_C...
Github user CarbonDataQA commented on the issue: https://github.com/apache/carbondata/pull/2826 Build Success with Spark 2.2.1, Please check CI http://95.216.28.178:8080/job/ApacheCarbonPRBuilder1/1149/ ---
[GitHub] carbondata issue #2843: [CARBONDATA-3034] Carding parameters,Organized by pa...
Github user CarbonDataQA commented on the issue: https://github.com/apache/carbondata/pull/2843 Build Success with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder2.1/945/ ---
[GitHub] carbondata issue #2843: [CARBONDATA-3034] Carding parameters,Organized by pa...
Github user CarbonDataQA commented on the issue: https://github.com/apache/carbondata/pull/2843 Build Success with Spark 2.2.1, Please check CI http://95.216.28.178:8080/job/ApacheCarbonPRBuilder1/1151/ ---
[GitHub] carbondata issue #2839: [CARBONDATA-3030] Remove no use parameter in test ca...
Github user CarbonDataQA commented on the issue: https://github.com/apache/carbondata/pull/2839 Build Success with Spark 2.2.1, Please check CI http://95.216.28.178:8080/job/ApacheCarbonPRBuilder1/1150/ ---
[GitHub] carbondata issue #2842: [CARBONDATA-3032] Remove carbon.blocklet.size from p...
Github user CarbonDataQA commented on the issue: https://github.com/apache/carbondata/pull/2842 Build Success with Spark 2.3.1, Please check CI http://136.243.101.176:8080/job/carbondataprbuilder2.3/9206/ ---
[GitHub] carbondata issue #2826: [CARBONDATA-3023] Alter add column issue with SORT_C...
Github user CarbonDataQA commented on the issue: https://github.com/apache/carbondata/pull/2826 Build Success with Spark 2.3.1, Please check CI http://136.243.101.176:8080/job/carbondataprbuilder2.3/9207/ ---
[GitHub] carbondata issue #2842: [CARBONDATA-3032] Remove carbon.blocklet.size from p...
Github user CarbonDataQA commented on the issue: https://github.com/apache/carbondata/pull/2842 Build Success with Spark 2.2.1, Please check CI http://95.216.28.178:8080/job/ApacheCarbonPRBuilder1/1148/ ---
[GitHub] carbondata issue #2839: [CARBONDATA-3030] Remove no use parameter in test ca...
Github user CarbonDataQA commented on the issue: https://github.com/apache/carbondata/pull/2839 Build Success with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder2.1/944/ ---
[GitHub] carbondata issue #2814: [WIP][CARBONDATA-3001] configurable page size in MB
Github user CarbonDataQA commented on the issue: https://github.com/apache/carbondata/pull/2814 Build Failed with Spark 2.2.1, Please check CI http://95.216.28.178:8080/job/ApacheCarbonPRBuilder1/1147/ ---
[GitHub] carbondata issue #2834: [CARBONDATA-3028][32k] Fix bugs in spark file format...
Github user CarbonDataQA commented on the issue: https://github.com/apache/carbondata/pull/2834 Build Success with Spark 2.2.1, Please check CI http://95.216.28.178:8080/job/ApacheCarbonPRBuilder1/1146/ ---
[GitHub] carbondata issue #2842: [CARBONDATA-3032] Remove carbon.blocklet.size from p...
Github user CarbonDataQA commented on the issue: https://github.com/apache/carbondata/pull/2842 Build Success with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder2.1/942/ ---
[GitHub] carbondata issue #2826: [CARBONDATA-3023] Alter add column issue with SORT_C...
Github user CarbonDataQA commented on the issue: https://github.com/apache/carbondata/pull/2826 Build Success with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder2.1/943/ ---
[GitHub] carbondata pull request #2843: [CARBONDATA-3034] Carding parameters,Organize...
GitHub user ndwangsen opened a pull request: https://github.com/apache/carbondata/pull/2843 [CARBONDATA-3034] Carding parameters,Organized by parameter category. This PR is mainly combing parameters, organized by parameter category. Be sure to do all of the following checklist to help us incorporate your contribution quickly and easily: - [ ] Any interfaces changed? NA - [ ] Any backward compatibility impacted? NA - [ ] Document update required? NA - [ ] Testing done Test in local env. - [ ] For large changes, please consider breaking it into sub-tasks under an umbrella JIRA. NA You can merge this pull request into a Git repository by running: $ git pull https://github.com/ndwangsen/incubator-carbondata parameter_comb Alternatively you can review and apply these changes as the patch at: https://github.com/apache/carbondata/pull/2843.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #2843 commit 21dc71ba986ab1c2cbdd2cfaa5418a2d629bc34a Author: ndwangsen Date: 2018-10-23T03:35:17Z [CARBONDATA-3034] Carding parameters,Organized by parameter category. ---
[GitHub] carbondata issue #2814: [WIP][CARBONDATA-3001] configurable page size in MB
Github user CarbonDataQA commented on the issue: https://github.com/apache/carbondata/pull/2814 Build Failed with Spark 2.3.1, Please check CI http://136.243.101.176:8080/job/carbondataprbuilder2.3/9205/ ---
[GitHub] carbondata issue #2839: [CARBONDATA-3030] Remove no use parameter in test ca...
Github user xubo245 commented on the issue: https://github.com/apache/carbondata/pull/2839 retest this please ---
[GitHub] carbondata issue #2824: [CARBONDATA-3008] Optimize default value for multipl...
Github user CarbonDataQA commented on the issue: https://github.com/apache/carbondata/pull/2824 Build Success with Spark 2.3.1, Please check CI http://136.243.101.176:8080/job/carbondataprbuilder2.3/9203/ ---
[GitHub] carbondata issue #2824: [CARBONDATA-3008] Optimize default value for multipl...
Github user CarbonDataQA commented on the issue: https://github.com/apache/carbondata/pull/2824 Build Success with Spark 2.2.1, Please check CI http://95.216.28.178:8080/job/ApacheCarbonPRBuilder1/1145/ ---
[GitHub] carbondata issue #2834: [CARBONDATA-3028][32k] Fix bugs in spark file format...
Github user CarbonDataQA commented on the issue: https://github.com/apache/carbondata/pull/2834 Build Success with Spark 2.3.1, Please check CI http://136.243.101.176:8080/job/carbondataprbuilder2.3/9204/ ---
[GitHub] carbondata pull request #2842: [CARBONDATA-3032] Remove carbon.blocklet.size...
GitHub user xubo245 opened a pull request: https://github.com/apache/carbondata/pull/2842 [CARBONDATA-3032] Remove carbon.blocklet.size from properties template 1.Remove carbon.blocklet.size from properties template, because carbon.blocklet.size is in V2, V3 use carbon.blockletgroup.size.in.mb or page size 2.change numberComressor to numberCompressor 3.optimize the annotation of method - [ ] Any interfaces changed? No - [ ] Any backward compatibility impacted? No - [ ] Document update required? No - [ ] Testing done No - [ ] For large changes, please consider breaking it into sub-tasks under an umbrella JIRA. No You can merge this pull request into a Git repository by running: $ git pull https://github.com/xubo245/carbondata CARBONDATA-3032_removeBlockletSize Alternatively you can review and apply these changes as the patch at: https://github.com/apache/carbondata/pull/2842.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #2842 commit 90dfad9a443ae1e5fd791fac85b73096598ab87e Author: xubo245 Date: 2018-10-23T03:14:03Z [CARBONDATA-3032] Remove carbon.blocklet.size from properties template 1.Remove carbon.blocklet.size from properties template, because carbon.blocklet.size is in V2, V3 use carbon.blockletgroup.size.in.mb or page size 2.change numberComressor to numberCompressor 3.optimize the annotation of method ---
[jira] [Created] (CARBONDATA-3034) Combing CarbonCommonConstants
wangsen created CARBONDATA-3034: --- Summary: Combing CarbonCommonConstants Key: CARBONDATA-3034 URL: https://issues.apache.org/jira/browse/CARBONDATA-3034 Project: CarbonData Issue Type: Improvement Components: core Affects Versions: 1.5.0 Reporter: wangsen Assignee: wangsen Fix For: 1.5.1 Carding parameters，Organized by parameter category. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] carbondata issue #2826: [CARBONDATA-3023] Alter add column issue with SORT_C...
Github user dhatchayani commented on the issue: https://github.com/apache/carbondata/pull/2826 Retest this please ---
[GitHub] carbondata issue #2834: [CARBONDATA-3028][32k] Fix bugs in spark file format...
Github user CarbonDataQA commented on the issue: https://github.com/apache/carbondata/pull/2834 Build Success with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder2.1/940/ ---
[GitHub] carbondata pull request #2840: [CARBONDATA-3033][Doc]Fix bugs in documents
Github user QiangCai commented on a diff in the pull request: https://github.com/apache/carbondata/pull/2840#discussion_r227199190 --- Diff: core/src/main/java/org/apache/carbondata/core/constants/CarbonCommonConstants.java --- @@ -361,7 +361,7 @@ /** * CSV_READ_BUFFER_SIZE */ - public static final String CSV_READ_BUFFER_SIZE_DEFAULT = "5"; + public static final String CSV_READ_BUFFER_SIZE_DEFAULT = "1048576"; --- End diff -- ```suggestion public static final String CSV_READ_BUFFER_SIZE_DEFAULT = "1048576"; // 1mb ``` ---
[GitHub] carbondata pull request #2834: [CARBONDATA-3028][32k] Fix bugs in spark file...
Github user xuchuanyin commented on a diff in the pull request:
https://github.com/apache/carbondata/pull/2834#discussion_r227195246
--- Diff:
store/sdk/src/main/java/org/apache/carbondata/sdk/file/CarbonWriterBuilder.java
---
@@ -412,10 +413,13 @@ private void setCsvHeader(CarbonLoadModel model) {
public CarbonLoadModel buildLoadModel(Schema carbonSchema)
throws IOException, InvalidLoadOptionException {
timestamp = System.nanoTime();
-Set longStringColumns = null;
-if (options != null && options.get("long_string_columns") != null) {
- longStringColumns =
- new
HashSet<>(Arrays.asList(options.get("long_string_columns").toLowerCase().split(",")));
+Set longStringColumns = new HashSet<>();
+if (options != null &&
options.get(CarbonCommonConstants.LONG_STRING_COLUMNS) != null) {
--- End diff --
OK, will fix it
---
[GitHub] carbondata issue #2824: [CARBONDATA-3008] Optimize default value for multipl...
Github user CarbonDataQA commented on the issue: https://github.com/apache/carbondata/pull/2824 Build Success with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder2.1/939/ ---
[GitHub] carbondata issue #2834: [CARBONDATA-3028][32k] Fix bugs in spark file format...
Github user CarbonDataQA commented on the issue: https://github.com/apache/carbondata/pull/2834 Build Success with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder2.1/938/ ---
[GitHub] carbondata pull request #2834: [CARBONDATA-3028][32k] Fix bugs in spark file...
Github user jackylk commented on a diff in the pull request:
https://github.com/apache/carbondata/pull/2834#discussion_r227194616
--- Diff:
store/sdk/src/main/java/org/apache/carbondata/sdk/file/CarbonWriterBuilder.java
---
@@ -412,10 +413,13 @@ private void setCsvHeader(CarbonLoadModel model) {
public CarbonLoadModel buildLoadModel(Schema carbonSchema)
throws IOException, InvalidLoadOptionException {
timestamp = System.nanoTime();
-Set longStringColumns = null;
-if (options != null && options.get("long_string_columns") != null) {
- longStringColumns =
- new
HashSet<>(Arrays.asList(options.get("long_string_columns").toLowerCase().split(",")));
+Set longStringColumns = new HashSet<>();
+if (options != null &&
options.get(CarbonCommonConstants.LONG_STRING_COLUMNS) != null) {
--- End diff --
please add a comment to describe it is for long string
---
[jira] [Resolved] (CARBONDATA-3029) Failed to run spark data source test cases in windows env
[ https://issues.apache.org/jira/browse/CARBONDATA-3029?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jacky Li resolved CARBONDATA-3029. -- Resolution: Fixed Fix Version/s: 1.5.1 > Failed to run spark data source test cases in windows env > - > > Key: CARBONDATA-3029 > URL: https://issues.apache.org/jira/browse/CARBONDATA-3029 > Project: CarbonData > Issue Type: Bug >Reporter: xuchuanyin >Assignee: xuchuanyin >Priority: Major > Fix For: 1.5.1 > > Time Spent: 1h > Remaining Estimate: 0h > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] carbondata pull request #2835: [CARBONDATA-3029][Test] Fix errors in spark d...
Github user asfgit closed the pull request at: https://github.com/apache/carbondata/pull/2835 ---
[GitHub] carbondata pull request #2824: [CARBONDATA-3008] Optimize default value for ...
Github user xuchuanyin commented on a diff in the pull request: https://github.com/apache/carbondata/pull/2824#discussion_r227192328 --- Diff: core/src/main/java/org/apache/carbondata/core/constants/CarbonCommonConstants.java --- @@ -1371,16 +1371,15 @@ public static final String CARBON_SECURE_DICTIONARY_SERVER_DEFAULT = "true"; /** - * whether to use multi directories when loading data, - * the main purpose is to avoid single-disk-hot-spot + * whether to use yarn's local dir the main purpose is to avoid single disk hot spot */ @CarbonProperty - public static final String CARBON_USE_MULTI_TEMP_DIR = "carbon.use.multiple.temp.dir"; + public static final String CARBON_USE_YARN_LOCAL_DIR = "carbon.use.local.dir"; --- End diff -- yeah, I'll fix that ---
[GitHub] carbondata issue #2835: [CARBONDATA-3029][Test] Fix errors in spark datasour...
Github user jackylk commented on the issue: https://github.com/apache/carbondata/pull/2835 LGTM ---
[GitHub] carbondata issue #2840: [CARBONDATA-3033][Doc]Fix bugs in documents
Github user jackylk commented on the issue: https://github.com/apache/carbondata/pull/2840 LGTM ---
[GitHub] carbondata issue #2840: [CARBONDATA-3033][Doc]Fix bugs in documents
Github user sraghunandan commented on the issue: https://github.com/apache/carbondata/pull/2840 LGTM ---
[GitHub] carbondata pull request #2824: [CARBONDATA-3008] Optimize default value for ...
Github user jackylk commented on a diff in the pull request: https://github.com/apache/carbondata/pull/2824#discussion_r227191521 --- Diff: core/src/main/java/org/apache/carbondata/core/constants/CarbonCommonConstants.java --- @@ -1371,16 +1371,15 @@ public static final String CARBON_SECURE_DICTIONARY_SERVER_DEFAULT = "true"; /** - * whether to use multi directories when loading data, - * the main purpose is to avoid single-disk-hot-spot + * whether to use yarn's local dir the main purpose is to avoid single disk hot spot */ @CarbonProperty - public static final String CARBON_USE_MULTI_TEMP_DIR = "carbon.use.multiple.temp.dir"; + public static final String CARBON_USE_YARN_LOCAL_DIR = "carbon.use.local.dir"; --- End diff -- ok. Then I think it is better change the variable name only, we can use the existing variable value ---
[jira] [Resolved] (CARBONDATA-3004) Fix bug in writing dataframe to carbon table while the field order is different
[ https://issues.apache.org/jira/browse/CARBONDATA-3004?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jacky Li resolved CARBONDATA-3004. -- Resolution: Fixed Fix Version/s: 1.5.1 > Fix bug in writing dataframe to carbon table while the field order is > different > --- > > Key: CARBONDATA-3004 > URL: https://issues.apache.org/jira/browse/CARBONDATA-3004 > Project: CarbonData > Issue Type: Sub-task >Reporter: xuchuanyin >Assignee: xuchuanyin >Priority: Major > Fix For: 1.5.1 > > Time Spent: 1h 20m > Remaining Estimate: 0h > > More information about this issue can be found in this link: > http://apache-carbondata-dev-mailing-list-archive.1130556.n5.nabble.com/Issue-Long-string-columns-config-for-big-strings-not-work-td64876.html -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] carbondata pull request #2812: [CARBONDATA-3004][32k] Fix bugs in writing da...
Github user asfgit closed the pull request at: https://github.com/apache/carbondata/pull/2812 ---
[GitHub] carbondata issue #2812: [CARBONDATA-3004][32k] Fix bugs in writing dataframe...
Github user jackylk commented on the issue: https://github.com/apache/carbondata/pull/2812 LGTM ---
[GitHub] carbondata issue #2834: [CARBONDATA-3028][32k] Fix bugs in spark file format...
Github user xuchuanyin commented on the issue: https://github.com/apache/carbondata/pull/2834 retest this please ---
[GitHub] carbondata issue #2814: [WIP][CARBONDATA-3001] configurable page size in MB
Github user CarbonDataQA commented on the issue: https://github.com/apache/carbondata/pull/2814 Build Failed with Spark 2.3.1, Please check CI http://136.243.101.176:8080/job/carbondataprbuilder2.3/9201/ ---
[GitHub] carbondata issue #2814: [WIP][CARBONDATA-3001] configurable page size in MB
Github user CarbonDataQA commented on the issue: https://github.com/apache/carbondata/pull/2814 Build Failed with Spark 2.2.1, Please check CI http://95.216.28.178:8080/job/ApacheCarbonPRBuilder1/1143/ ---
[GitHub] carbondata issue #2829: [CARBONDATA-3025]add more metadata in carbon file fo...
Github user CarbonDataQA commented on the issue: https://github.com/apache/carbondata/pull/2829 Build Failed with Spark 2.3.1, Please check CI http://136.243.101.176:8080/job/carbondataprbuilder2.3/9200/ ---
[GitHub] carbondata issue #2823: [CARBONDATA-3015] Support Lazy load in carbon vector
Github user CarbonDataQA commented on the issue: https://github.com/apache/carbondata/pull/2823 Build Failed with Spark 2.3.1, Please check CI http://136.243.101.176:8080/job/carbondataprbuilder2.3/9196/ ---
[GitHub] carbondata issue #2829: [CARBONDATA-3025]add more metadata in carbon file fo...
Github user CarbonDataQA commented on the issue: https://github.com/apache/carbondata/pull/2829 Build Failed with Spark 2.2.1, Please check CI http://95.216.28.178:8080/job/ApacheCarbonPRBuilder1/1142/ ---
[GitHub] carbondata issue #2829: [CARBONDATA-3025]add more metadata in carbon file fo...
Github user CarbonDataQA commented on the issue: https://github.com/apache/carbondata/pull/2829 Build Failed with Spark 2.2.1, Please check CI http://95.216.28.178:8080/job/ApacheCarbonPRBuilder1/1138/ ---
[GitHub] carbondata issue #2841: [WIP] Unsafe fallback to heap and unsafe query fix
Github user CarbonDataQA commented on the issue: https://github.com/apache/carbondata/pull/2841 Build Failed with Spark 2.3.1, Please check CI http://136.243.101.176:8080/job/carbondataprbuilder2.3/9199/ ---
[GitHub] carbondata issue #2826: [CARBONDATA-3023] Alter add column issue with SORT_C...
Github user CarbonDataQA commented on the issue: https://github.com/apache/carbondata/pull/2826 Build Failed with Spark 2.3.1, Please check CI http://136.243.101.176:8080/job/carbondataprbuilder2.3/9195/ ---
[GitHub] carbondata issue #2814: [WIP][CARBONDATA-3001] configurable page size in MB
Github user CarbonDataQA commented on the issue: https://github.com/apache/carbondata/pull/2814 Build Success with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder2.1/937/ ---
[GitHub] carbondata issue #2830: [CARBONDATA-3025]Added CLI enhancements
Github user CarbonDataQA commented on the issue: https://github.com/apache/carbondata/pull/2830 Build Failed with Spark 2.2.1, Please check CI http://95.216.28.178:8080/job/ApacheCarbonPRBuilder1/1137/ ---
[GitHub] carbondata issue #2829: [CARBONDATA-3025]add more metadata in carbon file fo...
Github user CarbonDataQA commented on the issue: https://github.com/apache/carbondata/pull/2829 Build Success with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder2.1/936/ ---
[GitHub] carbondata issue #2829: [CARBONDATA-3025]add more metadata in carbon file fo...
Github user CarbonDataQA commented on the issue: https://github.com/apache/carbondata/pull/2829 Build Failed with Spark 2.3.1, Please check CI http://136.243.101.176:8080/job/carbondataprbuilder2.3/9194/ ---
[GitHub] carbondata issue #2841: [WIP] Unsafe fallback to heap and unsafe query fix
Github user CarbonDataQA commented on the issue: https://github.com/apache/carbondata/pull/2841 Build Failed with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder2.1/935/ ---
[GitHub] carbondata issue #1933: [CARBONDATA-2132] [Partition] Fixed Error while load...
Github user CarbonDataQA commented on the issue: https://github.com/apache/carbondata/pull/1933 Build Success with Spark 2.2.1, Please check CI http://95.216.28.178:8080/job/ApacheCarbonPRBuilder1/1136/ ---
[GitHub] carbondata issue #2830: [CARBONDATA-3025]Added CLI enhancements
Github user CarbonDataQA commented on the issue: https://github.com/apache/carbondata/pull/2830 Build Failed with Spark 2.3.1, Please check CI http://136.243.101.176:8080/job/carbondataprbuilder2.3/9193/ ---
[GitHub] carbondata issue #1138: [CARBONDATA-1263]Single pass load does not take defa...
Github user CarbonDataQA commented on the issue: https://github.com/apache/carbondata/pull/1138 Build Failed with Spark 2.3.1, Please check CI http://136.243.101.176:8080/job/carbondataprbuilder2.3/9198/ ---
[GitHub] carbondata issue #2841: [WIP] Unsafe fallback to heap and unsafe query fix
Github user CarbonDataQA commented on the issue: https://github.com/apache/carbondata/pull/2841 Build Failed with Spark 2.2.1, Please check CI http://95.216.28.178:8080/job/ApacheCarbonPRBuilder1/1141/ ---
[GitHub] carbondata issue #1164: [CARBONDATA-1303] Update CarbonContext.scala
Github user CarbonDataQA commented on the issue: https://github.com/apache/carbondata/pull/1164 Build Failed with Spark 2.3.1, Please check CI http://136.243.101.176:8080/job/carbondataprbuilder2.3/9197/ ---
[GitHub] carbondata issue #2834: [CARBONDATA-3028][32k] Fix bugs in spark file format...
Github user CarbonDataQA commented on the issue: https://github.com/apache/carbondata/pull/2834 Build Success with Spark 2.3.1, Please check CI http://136.243.101.176:8080/job/carbondataprbuilder2.3/9192/ ---
[GitHub] carbondata issue #1164: [CARBONDATA-1303] Update CarbonContext.scala
Github user CarbonDataQA commented on the issue: https://github.com/apache/carbondata/pull/1164 Build Failed with Spark 2.2.1, Please check CI http://95.216.28.178:8080/job/ApacheCarbonPRBuilder1/1139/ ---
[GitHub] carbondata issue #1138: [CARBONDATA-1263]Single pass load does not take defa...
Github user CarbonDataQA commented on the issue: https://github.com/apache/carbondata/pull/1138 Build Failed with Spark 2.2.1, Please check CI http://95.216.28.178:8080/job/ApacheCarbonPRBuilder1/1140/ ---
[GitHub] carbondata issue #2823: [CARBONDATA-3015] Support Lazy load in carbon vector
Github user CarbonDataQA commented on the issue: https://github.com/apache/carbondata/pull/2823 Build Failed with Spark 2.2.1, Please check CI http://95.216.28.178:8080/job/ApacheCarbonPRBuilder1/1134/ ---
[GitHub] carbondata issue #2840: [CARBONDATA-3033][Doc]Fix bugs in documents
Github user CarbonDataQA commented on the issue: https://github.com/apache/carbondata/pull/2840 Build Success with Spark 2.3.1, Please check CI http://136.243.101.176:8080/job/carbondataprbuilder2.3/9191/ ---
[GitHub] carbondata issue #2039: [CARBONDATA-2233] [Test] Improve test cases of DBLoc...
Github user CarbonDataQA commented on the issue: https://github.com/apache/carbondata/pull/2039 Build Failed with Spark 2.2.1, Please check CI http://95.216.28.178:8080/job/ApacheCarbonPRBuilder1/1135/ ---
[GitHub] carbondata issue #2823: [CARBONDATA-3015] Support Lazy load in carbon vector
Github user CarbonDataQA commented on the issue: https://github.com/apache/carbondata/pull/2823 Build Failed with Spark 2.3.1, Please check CI http://136.243.101.176:8080/job/carbondataprbuilder2.3/9189/ ---
[GitHub] carbondata issue #2834: [CARBONDATA-3028][32k] Fix bugs in spark file format...
Github user CarbonDataQA commented on the issue: https://github.com/apache/carbondata/pull/2834 Build Failed with Spark 2.2.1, Please check CI http://95.216.28.178:8080/job/ApacheCarbonPRBuilder1/1131/ ---
[GitHub] carbondata pull request #2819: [CARBONDATA-3012] Added support for full scan...
Github user kunal642 commented on a diff in the pull request:
https://github.com/apache/carbondata/pull/2819#discussion_r227027472
--- Diff:
core/src/main/java/org/apache/carbondata/core/datastore/page/encoding/adaptive/AdaptiveIntegralCodec.java
---
@@ -248,6 +266,136 @@ public double decodeDouble(float value) {
public double decodeDouble(double value) {
throw new RuntimeException("internal error: " + debugInfo());
}
+
+@Override
+public void decodeAndFillVector(ColumnPage columnPage,
ColumnVectorInfo vectorInfo) {
+ CarbonColumnVector vector = vectorInfo.vector;
+ BitSet nullBits = columnPage.getNullBits();
+ DataType dataType = vector.getType();
+ DataType type = columnPage.getDataType();
+ int pageSize = columnPage.getPageSize();
+ BitSet deletedRows = vectorInfo.deletedRows;
+ fillVector(columnPage, vector, dataType, type, pageSize, vectorInfo);
+ if (deletedRows == null || deletedRows.isEmpty()) {
+for (int i = nullBits.nextSetBit(0); i >= 0; i =
nullBits.nextSetBit(i + 1)) {
+ vector.putNull(i);
+}
+ }
+}
+
+private void fillVector(ColumnPage columnPage, CarbonColumnVector
vector, DataType dataType,
--- End diff --
For Timestamp type
`vector.putLong(i, byteData[i] * 1000);` should be changed to
`vector.putLong(i, (long) byteData[i] * 1000L);` otherwise it would cross
integer range and give wrong results.
Please handle the same for AdaptiveDeltaIntegralCodec
---
[GitHub] carbondata issue #2829: [CARBONDATA-3025]add more metadata in carbon file fo...
Github user CarbonDataQA commented on the issue: https://github.com/apache/carbondata/pull/2829 Build Failed with Spark 2.2.1, Please check CI http://95.216.28.178:8080/job/ApacheCarbonPRBuilder1/1133/ ---
[GitHub] carbondata issue #2841: [WIP] Unsafe fallback to heap and unsafe query fix
Github user CarbonDataQA commented on the issue: https://github.com/apache/carbondata/pull/2841 Build Failed with Spark 2.3.1, Please check CI http://136.243.101.176:8080/job/carbondataprbuilder2.3/9190/ ---
[GitHub] carbondata issue #2830: [CARBONDATA-3025]Added CLI enhancements
Github user CarbonDataQA commented on the issue: https://github.com/apache/carbondata/pull/2830 Build Failed with Spark 2.2.1, Please check CI http://95.216.28.178:8080/job/ApacheCarbonPRBuilder1/1132/ ---
[GitHub] carbondata issue #2840: [CARBONDATA-3033][Doc]Fix bugs in documents
Github user CarbonDataQA commented on the issue: https://github.com/apache/carbondata/pull/2840 Build Success with Spark 2.2.1, Please check CI http://95.216.28.178:8080/job/ApacheCarbonPRBuilder1/1130/ ---
[GitHub] carbondata issue #2829: [CARBONDATA-3025]add more metadata in carbon file fo...
Github user akashrn5 commented on the issue: https://github.com/apache/carbondata/pull/2829 retest this please ---
[GitHub] carbondata issue #2772: [CARBONDATA-2977] Write uncompress_size to ChunkComp...
Github user CarbonDataQA commented on the issue: https://github.com/apache/carbondata/pull/2772 Build Success with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder2.1/934/ ---
[GitHub] carbondata issue #2791: [WIP][HOTFIX]correct the exception handling in looku...
Github user CarbonDataQA commented on the issue: https://github.com/apache/carbondata/pull/2791 Build Success with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder2.1/933/ ---
[GitHub] carbondata issue #2829: [CARBONDATA-3025]add more metadata in carbon file fo...
Github user CarbonDataQA commented on the issue: https://github.com/apache/carbondata/pull/2829 Build Success with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder2.1/932/ ---
[GitHub] carbondata issue #2830: [CARBONDATA-3025]Added CLI enhancements
Github user CarbonDataQA commented on the issue: https://github.com/apache/carbondata/pull/2830 Build Success with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder2.1/931/ ---
[GitHub] carbondata issue #2826: [CARBONDATA-3023] Alter add column issue with SORT_C...
Github user CarbonDataQA commented on the issue: https://github.com/apache/carbondata/pull/2826 Build Success with Spark 2.3.1, Please check CI http://136.243.101.176:8080/job/carbondataprbuilder2.3/9188/ ---
[GitHub] carbondata issue #2829: [CARBONDATA-3025]add more metadata in carbon file fo...
Github user CarbonDataQA commented on the issue: https://github.com/apache/carbondata/pull/2829 Build Failed with Spark 2.3.1, Please check CI http://136.243.101.176:8080/job/carbondataprbuilder2.3/9187/ ---
[GitHub] carbondata issue #2830: [CARBONDATA-3025]Added CLI enhancements
Github user CarbonDataQA commented on the issue: https://github.com/apache/carbondata/pull/2830 Build Failed with Spark 2.3.1, Please check CI http://136.243.101.176:8080/job/carbondataprbuilder2.3/9186/ ---
[GitHub] carbondata issue #2834: [CARBONDATA-3028][32k] Fix bugs in spark file format...
Github user CarbonDataQA commented on the issue: https://github.com/apache/carbondata/pull/2834 Build Success with Spark 2.3.1, Please check CI http://136.243.101.176:8080/job/carbondataprbuilder2.3/9185/ ---
[GitHub] carbondata issue #2819: [CARBONDATA-3012] Added support for full scan querie...
Github user CarbonDataQA commented on the issue: https://github.com/apache/carbondata/pull/2819 Build Success with Spark 2.2.1, Please check CI http://95.216.28.178:8080/job/ApacheCarbonPRBuilder1/1128/ ---
[GitHub] carbondata issue #2820: [CARBONDATA-3013] Added support for pruning pages fo...
Github user CarbonDataQA commented on the issue: https://github.com/apache/carbondata/pull/2820 Build Success with Spark 2.2.1, Please check CI http://95.216.28.178:8080/job/ApacheCarbonPRBuilder1/1127/ ---
[GitHub] carbondata pull request #2822: [CARBONDATA-3014] Added support for inverted ...
Github user manishgupta88 commented on a diff in the pull request:
https://github.com/apache/carbondata/pull/2822#discussion_r22787
--- Diff:
core/src/main/java/org/apache/carbondata/core/scan/result/vector/impl/directread/ColumnarVectorWrapperDirectWithDeleteDeltaAndInvertedIndex.java
---
@@ -0,0 +1,258 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ *http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.carbondata.core.scan.result.vector.impl.directread;
+
+import java.math.BigDecimal;
+import java.util.BitSet;
+
+import org.apache.carbondata.core.metadata.datatype.DataType;
+import org.apache.carbondata.core.metadata.datatype.DataTypes;
+import org.apache.carbondata.core.metadata.datatype.DecimalType;
+import org.apache.carbondata.core.scan.result.vector.CarbonColumnVector;
+import
org.apache.carbondata.core.scan.result.vector.impl.CarbonColumnVectorImpl;
+
+/**
+ * Column vector for column pages which has delete delta and inverted
index, so it uses delta biset
+ * to filter out data and use inverted index before filling to actual
vector
+ */
+class ColumnarVectorWrapperDirectWithDeleteDeltaAndInvertedIndex
+extends AbstractCarbonColumnarVector implements ConvertableVector {
+
+ private BitSet deletedRows;
+
+ private int[] invertedIndex;
+
+ private CarbonColumnVectorImpl carbonColumnVector;
+
+ private CarbonColumnVector columnVector;
+
+ private int precision;
+
+ private BitSet nullBits;
+
+ private boolean isnullBitsExists;
+
+ public ColumnarVectorWrapperDirectWithDeleteDeltaAndInvertedIndex(
+ CarbonColumnVector vectorWrapper, BitSet deletedRows, int[]
invertedIndex, BitSet nullBits,
+ boolean isnullBitsExists) {
+this.deletedRows = deletedRows;
+this.invertedIndex = invertedIndex;
+carbonColumnVector = new CarbonColumnVectorImpl(invertedIndex.length,
vectorWrapper.getType());
--- End diff --
If feasible try to merge this code with class
`ColumnarVectorWrapperDirectWithInvertedIndex`. Either one can be extended from
another or a flag can be used to distinguish. I think it is better to extend,
that will keep the code clean
---
[GitHub] carbondata issue #2823: [CARBONDATA-3015] Support Lazy load in carbon vector
Github user CarbonDataQA commented on the issue: https://github.com/apache/carbondata/pull/2823 Build Failed with Spark 2.2.1, Please check CI http://95.216.28.178:8080/job/ApacheCarbonPRBuilder1/1125/ ---
[GitHub] carbondata issue #2841: [WIP] Unsafe fallback to heap and unsafe query fix
Github user CarbonDataQA commented on the issue: https://github.com/apache/carbondata/pull/2841 Build Failed with Spark 2.2.1, Please check CI http://95.216.28.178:8080/job/ApacheCarbonPRBuilder1/1129/ ---
[GitHub] carbondata pull request #2822: [CARBONDATA-3014] Added support for inverted ...
Github user kumarvishal09 commented on a diff in the pull request: https://github.com/apache/carbondata/pull/2822#discussion_r226994945 --- Diff: core/src/main/java/org/apache/carbondata/core/constants/CarbonV3DataFormatConstants.java --- @@ -63,6 +63,6 @@ /** * number of rows per blocklet column page default value */ - short NUMBER_OF_ROWS_PER_BLOCKLET_COLUMN_PAGE_DEFAULT = 32000; + short NUMBER_OF_ROWS_PER_BLOCKLET_COLUMN_PAGE_DEFAULT = 10240; --- End diff -- are we going to fix page size to 10240?? ---
[GitHub] carbondata issue #2829: [CARBONDATA-3025]add more metadata in carbon file fo...
Github user kumarvishal09 commented on the issue: https://github.com/apache/carbondata/pull/2829 @akashrn5 Instead of passing writtenBy and appName to all the classes can't we set to CarbonProperties and in writer step we can get from the same and write to footer?? ---
[GitHub] carbondata pull request #2829: [CARBONDATA-3025]add more metadata in carbon ...
Github user kunal642 commented on a diff in the pull request:
https://github.com/apache/carbondata/pull/2829#discussion_r226979070
--- Diff:
store/sdk/src/main/java/org/apache/carbondata/sdk/file/CarbonSchemaReader.java
---
@@ -76,6 +81,19 @@ public static Schema readSchemaInDataFile(String
dataFilePath) throws IOExceptio
return new Schema(columnSchemaList);
}
+ public static String getVersionDetails(String dataFilePath) throws
IOException {
+long fileSize =
+FileFactory.getCarbonFile(dataFilePath,
FileFactory.getFileType(dataFilePath)).getSize();
+FileReader fileReader =
FileFactory.getFileHolder(FileFactory.getFileType(dataFilePath));
+ByteBuffer buffer =
+
fileReader.readByteBuffer(FileFactory.getUpdatedFilePath(dataFilePath),
fileSize - 8, 8);
+CarbonFooterReaderV3 footerReader = new
CarbonFooterReaderV3(dataFilePath, buffer.getLong());
+FileFooter3 footer = footerReader.readFooterVersion3();
+return
footer.getExtra_info().get(CarbonCommonConstants.CARBON_WRITTEN_BY_FOOTER_INFO)
--- End diff --
getExtra_info().get(CarbonCommonConstants.CARBON_WRITTEN_BY_FOOTER_INFO)
will give null for old store tables. Same goes for
.getExtra_info().get(CarbonCommonConstants.CARBON_VERSION_FOOTER_INFO).
Check for null and return empty string
---
[GitHub] carbondata pull request #2830: [CARBONDATA-3025]Added CLI enhancements
Github user kumarvishal09 commented on a diff in the pull request:
https://github.com/apache/carbondata/pull/2830#discussion_r226987983
--- Diff: tools/cli/src/main/java/org/apache/carbondata/tool/CarbonCli.java
---
@@ -40,6 +41,10 @@
@InterfaceStability.Unstable
public class CarbonCli {
+ private static ArrayList outPuts;
+
+ private static boolean isPrintInConsole = true;
--- End diff --
please add doc for this variable
---
[GitHub] carbondata pull request #2830: [CARBONDATA-3025]Added CLI enhancements
Github user kumarvishal09 commented on a diff in the pull request:
https://github.com/apache/carbondata/pull/2830#discussion_r226991227
--- Diff:
tools/cli/src/main/java/org/apache/carbondata/tool/DataSummary.java ---
@@ -226,25 +282,46 @@ private int getColumnIndex(String columnName) {
private boolean collected = false;
private void printColumnStats(String columnName) throws IOException,
MemoryException {
-out.println();
-out.println("## Column Statistics for '" + columnName + "'");
+outPuts.add("");
+outPuts.add("## Column Statistics for '" + columnName + "'");
collectStats(columnName);
int columnIndex = getColumnIndex(columnName);
String[] header = new String[]{"BLK", "BLKLT", "Meta Size", "Data
Size",
-"LocalDict", "DictEntries", "DictSize", "AvgPageSize", "Min%",
"Max%"};
+"LocalDict", "DictEntries", "DictSize", "AvgPageSize", "Min%",
"Max%", "Min", "Max"};
-ShardPrinter printer = new ShardPrinter(header);
+ShardPrinter printer = new ShardPrinter(header, outPuts);
for (Map.Entry entry : dataFiles.entrySet()) {
DataFile file = entry.getValue();
for (DataFile.Blocklet blocklet : file.getAllBlocklets()) {
-String min, max;
+String min, max, minPercent, maxPercent;
--- End diff --
Now min max will not be present if size is more for example varchar string
this case is not handled please check
---
[GitHub] carbondata pull request #2830: [CARBONDATA-3025]Added CLI enhancements
Github user kumarvishal09 commented on a diff in the pull request:
https://github.com/apache/carbondata/pull/2830#discussion_r226989395
--- Diff:
tools/cli/src/main/java/org/apache/carbondata/tool/TablePrinter.java ---
@@ -17,26 +17,28 @@
package org.apache.carbondata.tool;
-import java.io.PrintStream;
+import java.util.ArrayList;
import java.util.LinkedList;
import java.util.List;
class TablePrinter {
--- End diff --
Now TablePrinter is not printing any details, it is adding to list. Please
change the class name, Please check other places if applicable
---
[GitHub] carbondata pull request #2830: [CARBONDATA-3025]Added CLI enhancements
Github user kumarvishal09 commented on a diff in the pull request:
https://github.com/apache/carbondata/pull/2830#discussion_r226987643
--- Diff:
integration/spark2/src/main/scala/org/apache/spark/sql/execution/command/management/CarbonShowSummaryCommand.scala
---
@@ -0,0 +1,57 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ *http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.spark.sql.execution.command.management
+
+import java.util
+
+import scala.collection.JavaConverters._
+
+import org.apache.spark.sql.{CarbonEnv, Row, SparkSession}
+import org.apache.spark.sql.catalyst.expressions.{Attribute,
AttributeReference}
+import org.apache.spark.sql.execution.command.{Checker, DataCommand}
+import org.apache.spark.sql.types.StringType
+
+import
org.apache.carbondata.common.exceptions.sql.MalformedCarbonCommandException
+import org.apache.carbondata.tool.CarbonCli
+
+case class CarbonShowSummaryCommand(
--- End diff --
Please add class level doc
---
[GitHub] carbondata pull request #2830: [CARBONDATA-3025]Added CLI enhancements
Github user kumarvishal09 commented on a diff in the pull request:
https://github.com/apache/carbondata/pull/2830#discussion_r226987833
--- Diff: tools/cli/src/main/java/org/apache/carbondata/tool/CarbonCli.java
---
@@ -40,6 +41,10 @@
@InterfaceStability.Unstable
public class CarbonCli {
+ private static ArrayList outPuts;
--- End diff --
change private static ArrayList outPuts to List
---
[GitHub] carbondata pull request #2822: [CARBONDATA-3014] Added support for inverted ...
Github user manishgupta88 commented on a diff in the pull request:
https://github.com/apache/carbondata/pull/2822#discussion_r226973433
--- Diff:
core/src/main/java/org/apache/carbondata/core/scan/result/vector/impl/directread/ColumnarVectorWrapperDirectWithDeleteDeltaAndInvertedIndex.java
---
@@ -0,0 +1,258 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ *http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.carbondata.core.scan.result.vector.impl.directread;
+
+import java.math.BigDecimal;
+import java.util.BitSet;
+
+import org.apache.carbondata.core.metadata.datatype.DataType;
+import org.apache.carbondata.core.metadata.datatype.DataTypes;
+import org.apache.carbondata.core.metadata.datatype.DecimalType;
+import org.apache.carbondata.core.scan.result.vector.CarbonColumnVector;
+import
org.apache.carbondata.core.scan.result.vector.impl.CarbonColumnVectorImpl;
+
+/**
+ * Column vector for column pages which has delete delta and inverted
index, so it uses delta biset
+ * to filter out data and use inverted index before filling to actual
vector
+ */
+class ColumnarVectorWrapperDirectWithDeleteDeltaAndInvertedIndex
+extends AbstractCarbonColumnarVector implements ConvertableVector {
--- End diff --
`AbstractCarbonColumnarVector` implements the` ConvertableVector`..no need
to implement here
---
[GitHub] carbondata pull request #2822: [CARBONDATA-3014] Added support for inverted ...
Github user manishgupta88 commented on a diff in the pull request:
https://github.com/apache/carbondata/pull/2822#discussion_r226980856
--- Diff:
core/src/main/java/org/apache/carbondata/core/scan/result/vector/impl/directread/ColumnarVectorWrapperDirectWithDeleteDeltaAndInvertedIndex.java
---
@@ -0,0 +1,258 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ *http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.carbondata.core.scan.result.vector.impl.directread;
+
+import java.math.BigDecimal;
+import java.util.BitSet;
+
+import org.apache.carbondata.core.metadata.datatype.DataType;
+import org.apache.carbondata.core.metadata.datatype.DataTypes;
+import org.apache.carbondata.core.metadata.datatype.DecimalType;
+import org.apache.carbondata.core.scan.result.vector.CarbonColumnVector;
+import
org.apache.carbondata.core.scan.result.vector.impl.CarbonColumnVectorImpl;
+
+/**
+ * Column vector for column pages which has delete delta and inverted
index, so it uses delta biset
+ * to filter out data and use inverted index before filling to actual
vector
+ */
+class ColumnarVectorWrapperDirectWithDeleteDeltaAndInvertedIndex
+extends AbstractCarbonColumnarVector implements ConvertableVector {
+
+ private BitSet deletedRows;
+
+ private int[] invertedIndex;
+
+ private CarbonColumnVectorImpl carbonColumnVector;
+
+ private CarbonColumnVector columnVector;
+
+ private int precision;
+
+ private BitSet nullBits;
+
+ private boolean isnullBitsExists;
+
+ public ColumnarVectorWrapperDirectWithDeleteDeltaAndInvertedIndex(
+ CarbonColumnVector vectorWrapper, BitSet deletedRows, int[]
invertedIndex, BitSet nullBits,
+ boolean isnullBitsExists) {
+this.deletedRows = deletedRows;
+this.invertedIndex = invertedIndex;
+carbonColumnVector = new CarbonColumnVectorImpl(invertedIndex.length,
vectorWrapper.getType());
+this.columnVector = vectorWrapper;
+this.nullBits = nullBits;
+this.isnullBitsExists = isnullBitsExists;
+ }
+
+ @Override
+ public void putBoolean(int rowId, boolean value) {
+carbonColumnVector.putBoolean(invertedIndex[rowId], value);
+ }
+
+ @Override
+ public void putFloat(int rowId, float value) {
+carbonColumnVector.putFloat(invertedIndex[rowId], value);
+ }
+
+ @Override
+ public void putShort(int rowId, short value) {
+carbonColumnVector.putShort(invertedIndex[rowId], value);
+ }
+
+ @Override
+ public void putInt(int rowId, int value) {
+carbonColumnVector.putInt(invertedIndex[rowId], value);
+ }
+
+ @Override
+ public void putLong(int rowId, long value) {
+carbonColumnVector.putLong(invertedIndex[rowId], value);
+ }
+
+ @Override
+ public void putDecimal(int rowId, BigDecimal value, int precision) {
+this.precision = precision;
+carbonColumnVector.putDecimal(invertedIndex[rowId], value, precision);
+ }
+
+ @Override
+ public void putDouble(int rowId, double value) {
+carbonColumnVector.putDouble(invertedIndex[rowId], value);
+ }
+
+ @Override
+ public void putByteArray(int rowId, byte[] value) {
+carbonColumnVector.putByteArray(invertedIndex[rowId], value);
+ }
+
+ @Override
+ public void putByteArray(int rowId, int offset, int length, byte[]
value) {
+carbonColumnVector.putByteArray(invertedIndex[rowId], offset, length,
value);
+ }
+
+ @Override
+ public void putByte(int rowId, byte value) {
+carbonColumnVector.putByte(invertedIndex[rowId], value);
+ }
+
+ @Override
+ public void putNull(int rowId) {
+if (isnullBitsExists) {
+ nullBits.set(rowId);
+} else {
+ nullBits.set(invertedIndex[rowId]);
--- End diff --
please add a comment to explain this if-else logic...flag
`isnullBitsExists` is to distinguish between measure and dimension. When the
[GitHub] carbondata issue #2819: [CARBONDATA-3012] Added support for full scan querie...
Github user CarbonDataQA commented on the issue: https://github.com/apache/carbondata/pull/2819 Build Success with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder2.1/930/ ---
[GitHub] carbondata issue #2823: [CARBONDATA-3015] Support Lazy load in carbon vector
Github user CarbonDataQA commented on the issue: https://github.com/apache/carbondata/pull/2823 Build Success with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder2.1/927/ ---
[GitHub] carbondata issue #2826: [CARBONDATA-3023] Alter add column issue with SORT_C...
Github user CarbonDataQA commented on the issue: https://github.com/apache/carbondata/pull/2826 Build Success with Spark 2.2.1, Please check CI http://95.216.28.178:8080/job/ApacheCarbonPRBuilder1/1124/ ---
