To close the loop. This should work: sc._jsc.hadoopConfiguration
See this method in JavaSparkContext : def hadoopConfiguration(): Configuration = { sc.hadoopConfiguration On Tue, Jun 30, 2015 at 5:52 PM, Ted Yu <yuzhih...@gmail.com> wrote: > Minor correction: > It should be sc._jsc > > Cheers > > On Tue, Jun 30, 2015 at 4:23 PM, ayan guha <guha.a...@gmail.com> wrote: > >> There is a sc._jsc_ which you can access to get/set hadoop conf. >> >> On Wed, Jul 1, 2015 at 7:41 AM, Richard Ding <pigu...@gmail.com> wrote: >> >>> Hi, >>> >>> I noticed that, in Scala API, one can call hadoopConfiguration on >>> SparkContext to retrieve the hadoop configuration object which is very >>> handy in modifying certain hadoop properties at runtime. But there is no >>> corresponding method in Python API. >>> >>> Will this method be added to Python API in a later release? In the mean >>> time, what will be the work around (of setting hadoop properties at >>> runtime) using Python API? >>> >>> Thanks, >>> Richard >>> >> >> >> >> -- >> Best Regards, >> Ayan Guha >> > >