Github user vanzin commented on the issue: https://github.com/apache/spark/pull/19973 Well, the default for fallback configs is the current value of the parent conf - it needs context. The code you reference in the a previous comment has that context (the SQL conf map), so the value of the parent conf might not be the default. If you look at `ConfigReader.getOrDefault` it has some (kinda nasty) code to do this. If I understand correctly, the problem is really here (the other `getConfString` method): ``` def getConfString(key: String): String = { Option(settings.get(key)). orElse { // Try to use the default value Option(sqlConfEntries.get(key)).map(_.defaultValueString) }. getOrElse(throw new NoSuchElementException(key)) } ``` That will be broken for fallback configs with the existing code, but it will also return the wrong thing with your updated code, in the case where the parent conf is set.
--- --------------------------------------------------------------------- To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org