caoxuewen created SPARK-21353: --------------------------------- Summary: add checkValue in spark.internal.config about how to correctly set configurations Key: SPARK-21353 URL: https://issues.apache.org/jira/browse/SPARK-21353 Project: Spark Issue Type: Improvement Components: Spark Core Affects Versions: 2.3.0 Reporter: caoxuewen Priority: Minor
add checkValue for configurations: spark.driver.memory spark.executor.memory spark.blockManager.port spark.task.cpus spark.dynamicAllocation.minExecutors spark.task.maxFailures spark.blacklist.task.maxTaskAttemptsPerExecutor spark.blacklist.task.maxTaskAttemptsPerNode spark.blacklist.application.maxFailedTasksPerExecutor spark.blacklist.stage.maxFailedTasksPerExecutor spark.blacklist.application.maxFailedExecutorsPerNode spark.blacklist.stage.maxFailedExecutorsPerNode spark.scheduler.listenerbus.metrics.maxListenerClassesTimed spark.ui.retainedTasks spark.blockManager.port spark.driver.blockManager.port spark.files.maxPartitionBytes spark.files.openCostInBytes spark.shuffle.accurateBlockThreshold spark.shuffle.registration.timeout spark.shuffle.registration.maxAttempts spark.reducer.maxReqSizeShuffleToMem and we copy the document from http://spark.apache.org/docs/latest/configuration.html spark.driver.userClassPathFirst spark.driver.memory spark.executor.userClassPathFirst spark.executor.memory spark.yarn.isPython spark.task.cpus spark.dynamicAllocation.minExecutors spark.dynamicAllocation.initialExecutors spark.dynamicAllocation.maxExecutors spark.shuffle.service.enabled spark.submit.pyFiles spark.task.maxFailures spark.blacklist.enabled spark.blacklist.task.maxTaskAttemptsPerExecutor spark.blacklist.task.maxTaskAttemptsPerNode spark.blacklist.stage.maxFailedTasksPerExecutor spark.blacklist.stage.maxFailedExecutorsPerNode spark.pyspark.driver.python spark.pyspark.python spark.ui.retainedTasks spark.io.encryption.enabled spark.io.encryption.keygen.algorithm spark.io.encryption.keySizeBits spark.authenticate.enableSaslEncryption spark.authenticate -- This message was sent by Atlassian JIRA (v6.4.14#64029) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org