caoxuewen created SPARK-21353:
---------------------------------
Summary: add checkValue in spark.internal.config about how to
correctly set configurations
Key: SPARK-21353
URL: https://issues.apache.org/jira/browse/SPARK-21353
Project: Spark
Issue Type: Improvement
Components: Spark Core
Affects Versions: 2.3.0
Reporter: caoxuewen
Priority: Minor
add checkValue for configurations:
spark.driver.memory
spark.executor.memory
spark.blockManager.port
spark.task.cpus
spark.dynamicAllocation.minExecutors
spark.task.maxFailures
spark.blacklist.task.maxTaskAttemptsPerExecutor
spark.blacklist.task.maxTaskAttemptsPerNode
spark.blacklist.application.maxFailedTasksPerExecutor
spark.blacklist.stage.maxFailedTasksPerExecutor
spark.blacklist.application.maxFailedExecutorsPerNode
spark.blacklist.stage.maxFailedExecutorsPerNode
spark.scheduler.listenerbus.metrics.maxListenerClassesTimed
spark.ui.retainedTasks
spark.blockManager.port
spark.driver.blockManager.port
spark.files.maxPartitionBytes
spark.files.openCostInBytes
spark.shuffle.accurateBlockThreshold
spark.shuffle.registration.timeout
spark.shuffle.registration.maxAttempts
spark.reducer.maxReqSizeShuffleToMem
and we copy the document from
http://spark.apache.org/docs/latest/configuration.html
spark.driver.userClassPathFirst
spark.driver.memory
spark.executor.userClassPathFirst
spark.executor.memory
spark.yarn.isPython
spark.task.cpus
spark.dynamicAllocation.minExecutors
spark.dynamicAllocation.initialExecutors
spark.dynamicAllocation.maxExecutors
spark.shuffle.service.enabled
spark.submit.pyFiles
spark.task.maxFailures
spark.blacklist.enabled
spark.blacklist.task.maxTaskAttemptsPerExecutor
spark.blacklist.task.maxTaskAttemptsPerNode
spark.blacklist.stage.maxFailedTasksPerExecutor
spark.blacklist.stage.maxFailedExecutorsPerNode
spark.pyspark.driver.python
spark.pyspark.python
spark.ui.retainedTasks
spark.io.encryption.enabled
spark.io.encryption.keygen.algorithm
spark.io.encryption.keySizeBits
spark.authenticate.enableSaslEncryption
spark.authenticate
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
