This is definitely an important issue to fix. Instead of renaming properties, one solution would be to replace Typesafe Config with just reading Java system properties, and disable config files for this release. I kind of like that over renaming.
Matei On Jan 18, 2014, at 11:30 AM, Mridul Muralidharan <mri...@gmail.com> wrote: > Hi, > > Speculation was an example, there are others in spark which are > affected by this ... > Some of them have been around for a while, so will break existing > code/scripts. > > Regards, > Mridul > > On Sun, Jan 19, 2014 at 12:51 AM, Nan Zhu <zhunanmcg...@gmail.com> wrote: >> change spark.speculation to spark.speculation.switch? >> >> maybe we can restrict that all properties in Spark should be "three levels" >> >> >> On Sat, Jan 18, 2014 at 2:10 PM, Mridul Muralidharan <mri...@gmail.com>wrote: >> >>> Hi, >>> >>> Unless I am mistaken, the change to using typesafe ConfigFactory has >>> broken some of the system properties we use in spark. >>> >>> For example: if we have both >>> -Dspark.speculation=true -Dspark.speculation.multiplier=0.95 >>> set, then the spark.speculation property is dropped. >>> >>> The rules of parseProperty actually document this clearly [1] >>> >>> >>> I am not sure what the right fix here would be (other than replacing >>> use of config that is). >>> >>> Any thoughts ? >>> I would vote -1 for 0.9 to be released before this is fixed. >>> >>> >>> Regards, >>> Mridul >>> >>> >>> [1] >>> http://typesafehub.github.io/config/latest/api/com/typesafe/config/ConfigFactory.html#parseProperties%28java.util.Properties,%20com.typesafe.config.ConfigParseOptions%29 >>>
