pratyakshsharma commented on a change in pull request #4210:
URL: https://github.com/apache/carbondata/pull/4210#discussion_r736481157
##########
File path: docs/configuration-parameters.md
##########
@@ -70,6 +75,7 @@ This section provides the details of all the configurations
required for the Car
| carbon.load.global.sort.partitions | 0 | The number of partitions to use
when shuffling data for global sort. Default value 0 means to use same number
of map tasks as reduce tasks. **NOTE:** In general, it is recommended to have
2-3 tasks per CPU core in your cluster. |
| carbon.sort.size | 100000 | Number of records to hold in memory to sort and
write intermediate sort temp files. **NOTE:** Memory required for data loading
will increase if you turn this value bigger. Besides each thread will cache
this amout of records. The number of threads is configured by
*carbon.number.of.cores.while.loading*. |
| carbon.options.bad.records.logger.enable | false | CarbonData can identify
the records that are not conformant to schema and isolate them as bad records.
Enabling this configuration will make CarbonData to log such bad records.
**NOTE:** If the input data contains many bad records, logging them will slow
down the over all data loading throughput. The data load operation status would
depend on the configuration in ***carbon.bad.records.action***. |
+| carbon.options.bad.records.action | FAIL | This property has four types of
bad record actions: FORCE, REDIRECT, IGNORE and FAIL. If set to FORCE then it
auto-corrects the data by storing the bad records as NULL. If set to REDIRECT
then bad records are written to the raw CSV instead of being loaded. If set to
IGNORE then bad records are neither loaded nor written to the raw CSV. If set
to FAIL then data loading fails if any bad records are found. |
Review comment:
done.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
