change spark.speculation to spark.speculation.switch? maybe we can restrict that all properties in Spark should be "three levels"
On Sat, Jan 18, 2014 at 2:10 PM, Mridul Muralidharan <mri...@gmail.com>wrote: > Hi, > > Unless I am mistaken, the change to using typesafe ConfigFactory has > broken some of the system properties we use in spark. > > For example: if we have both > -Dspark.speculation=true -Dspark.speculation.multiplier=0.95 > set, then the spark.speculation property is dropped. > > The rules of parseProperty actually document this clearly [1] > > > I am not sure what the right fix here would be (other than replacing > use of config that is). > > Any thoughts ? > I would vote -1 for 0.9 to be released before this is fixed. > > > Regards, > Mridul > > > [1] > http://typesafehub.github.io/config/latest/api/com/typesafe/config/ConfigFactory.html#parseProperties%28java.util.Properties,%20com.typesafe.config.ConfigParseOptions%29 >
