Repository: carbondata Updated Branches: refs/heads/master fb2f9d33b -> e5c1568de
[CARBONDATA-2714] Support merge index files for the segment - Documentation update This closes #2509 Project: http://git-wip-us.apache.org/repos/asf/carbondata/repo Commit: http://git-wip-us.apache.org/repos/asf/carbondata/commit/e5c1568d Tree: http://git-wip-us.apache.org/repos/asf/carbondata/tree/e5c1568d Diff: http://git-wip-us.apache.org/repos/asf/carbondata/diff/e5c1568d Branch: refs/heads/master Commit: e5c1568de68f6c803e2a42a146ec773239917efa Parents: fb2f9d3 Author: dhatchayani <dhatcha.offic...@gmail.com> Authored: Mon Jul 16 12:09:30 2018 +0530 Committer: ravipesala <ravi.pes...@gmail.com> Committed: Tue Jul 24 12:01:26 2018 +0530 ---------------------------------------------------------------------- docs/configuration-parameters.md | 2 +- docs/data-management-on-carbondata.md | 13 +++++++++++++ 2 files changed, 14 insertions(+), 1 deletion(-) ---------------------------------------------------------------------- http://git-wip-us.apache.org/repos/asf/carbondata/blob/e5c1568d/docs/configuration-parameters.md ---------------------------------------------------------------------- diff --git a/docs/configuration-parameters.md b/docs/configuration-parameters.md index c46157c..b614918 100644 --- a/docs/configuration-parameters.md +++ b/docs/configuration-parameters.md @@ -80,7 +80,7 @@ This section provides the details of all the configurations required for CarbonD | carbon.horizontal.UPDATE.compaction.threshold | 1 | This property specifies the threshold limit on number of UPDATE delta files within a segment. In case the number of delta files goes beyond the threshold, the UPDATE delta files within the segment becomes eligible for horizontal compaction and compacted into single UPDATE delta file. | Values between 1 to 10000. | | carbon.horizontal.DELETE.compaction.threshold | 1 | This property specifies the threshold limit on number of DELETE delta files within a block of a segment. In case the number of delta files goes beyond the threshold, the DELETE delta files for the particular block of the segment becomes eligible for horizontal compaction and compacted into single DELETE delta file. | Values between 1 to 10000. | | carbon.update.segment.parallelism | 1 | This property specifies the parallelism for each segment during update. If there are segments that contain too many records to update and the spark job encounter data-spill related errors, it is better to increase this property value. It is recommended to set this value to a multiple of the number of executors for balance. | Values between 1 to 1000. | - +| carbon.merge.index.in.segment | true | This property is used to merge all carbon index files (.carbonindex) inside a segment to a single carbon index merge file (.carbonindexmerge).| Values true or false | * **Query Configuration** http://git-wip-us.apache.org/repos/asf/carbondata/blob/e5c1568d/docs/data-management-on-carbondata.md ---------------------------------------------------------------------- diff --git a/docs/data-management-on-carbondata.md b/docs/data-management-on-carbondata.md index 750dc20..4532b41 100644 --- a/docs/data-management-on-carbondata.md +++ b/docs/data-management-on-carbondata.md @@ -333,6 +333,19 @@ This tutorial is going to introduce all commands and data operations on CarbonDa ``` ALTER TABLE test_db.carbon CHANGE a1 a1 DECIMAL(18,2) ``` +- **MERGE INDEX** + + This command is used to merge all the CarbonData index files (.carbonindex) inside a segment to a single CarbonData index merge file (.carbonindexmerge). This enhances the first query performance. + ``` + ALTER TABLE [db_name.]table_name COMPACT 'SEGMENT_INDEX' + ``` + + Examples: + ``` + ALTER TABLE test_db.carbon COMPACT 'SEGMENT_INDEX' + ``` + **NOTE:** + * Merge index is not supported on streaming table. ### DROP TABLE
