[CARBONDATA-1880] Documentation for merging small files Documentation for merging small files
This closes #1903 Project: http://git-wip-us.apache.org/repos/asf/carbondata/repo Commit: http://git-wip-us.apache.org/repos/asf/carbondata/commit/b48a8c21 Tree: http://git-wip-us.apache.org/repos/asf/carbondata/tree/b48a8c21 Diff: http://git-wip-us.apache.org/repos/asf/carbondata/diff/b48a8c21 Branch: refs/heads/branch-1.3 Commit: b48a8c21f75d642c5729bdc3f147a50685447f65 Parents: 71f8828 Author: sgururajshetty <sgururajshe...@gmail.com> Authored: Wed Jan 31 19:25:16 2018 +0530 Committer: chenliang613 <chenliang...@huawei.com> Committed: Sat Feb 3 16:05:56 2018 +0800 ---------------------------------------------------------------------- docs/configuration-parameters.md | 1 + 1 file changed, 1 insertion(+) ---------------------------------------------------------------------- http://git-wip-us.apache.org/repos/asf/carbondata/blob/b48a8c21/docs/configuration-parameters.md ---------------------------------------------------------------------- diff --git a/docs/configuration-parameters.md b/docs/configuration-parameters.md index b68a2d1..621574d 100644 --- a/docs/configuration-parameters.md +++ b/docs/configuration-parameters.md @@ -61,6 +61,7 @@ This section provides the details of all the configurations required for CarbonD | carbon.options.bad.record.path | | Specifies the HDFS path where bad records are stored. By default the value is Null. This path must to be configured by the user if bad record logger is enabled or bad record action redirect. | | | carbon.enable.vector.reader | true | This parameter increases the performance of select queries as it fetch columnar batch of size 4*1024 rows instead of fetching data row by row. | | | carbon.blockletgroup.size.in.mb | 64 MB | The data are read as a group of blocklets which are called blocklet groups. This parameter specifies the size of the blocklet group. Higher value results in better sequential IO access.The minimum value is 16MB, any value lesser than 16MB will reset to the default value (64MB). | | +| carbon.task.distribution | block | **block**: Setting this value will launch one task per block. This setting is suggested in case of concurrent queries and queries having big shuffling scenarios. **custom**: Setting this value will group the blocks and distribute it uniformly to the available resources in the cluster. This enhances the query performance but not suggested in case of concurrent queries and queries having big shuffling scenarios. **blocklet**: Setting this value will launch one task per blocklet. This setting is suggested in case of concurrent queries and queries having big shuffling scenarios. **merge_small_files**: Setting this value will merge all the small partitions to a size of (128 MB) during querying. The small partitions are combined to a map task to reduce the number of read task. This enhances the performance. | | * **Compaction Configuration**
