Should we try to deprecate these types of configs for 1.0.0? We can start by accepting both and giving a warning if you use the old one, and then actually remove them in the next minor release. I think "spark.speculation.enabled=true" is better than "spark.speculation=true", and if we decide to use typesafe configs again ourselves, this change is necessary.
We actually don't have to ever complete the deprecation - we can always accept both spark.speculation and spark.speculation.enabled, and people just have to use the latter if they want to use typesafe config. On Wed, Mar 12, 2014 at 9:24 AM, Mark Hamstra <m...@clearstorydata.com>wrote: > That's the whole reason why some of the intended configuration changes > were backed out just before the 0.9.0 release. It's a well-known issue, > even if a completely satisfactory solution isn't as well-known and is > probably something which should do another iteration on. > > > On Wed, Mar 12, 2014 at 9:10 AM, Koert Kuipers <ko...@tresata.com> wrote: > >> i am reading the spark configuration params from another configuration >> object (typesafe config) before setting them as system properties. >> >> i noticed typesafe config has trouble with settings like: >> spark.speculation=true >> spark.speculation.interval=0.5 >> >> the issue seems to be that if spark.speculation is a "container" that has >> more values inside then it cannot be also a value itself, i think. so this >> would work fine: >> spark.speculation.enabled=true >> spark.speculation.interval=0.5 >> >> just a heads up. i would probably suggest we avoid this situation. >> > >
