Unfortunately /tmp mount is really small in our environment. I need to
provide a per-user setting as the default value.
I hacked bin/spark-class for the similar effect. And spark-defaults.conf
can override it. :)
Jianshi
On Wed, Mar 11, 2015 at 3:28 PM, Patrick Wendell <pwend...@gmail.com> wrote:
> We don't support expressions or wildcards in that configuration. For
> each application, the local directories need to be constant. If you
> have users submitting different Spark applications, those can each set
> spark.local.dirs.
>
> - Patrick
>
> On Wed, Mar 11, 2015 at 12:14 AM, Jianshi Huang <jianshi.hu...@gmail.com>
> wrote:
> > Hi,
> >
> > I need to set per-user spark.local.dir, how can I do that?
> >
> > I tried both
> >
> > /x/home/${user.name}/spark/tmp
> > and
> > /x/home/${USER}/spark/tmp
> >
> > And neither worked. Looks like it has to be a constant setting in
> > spark-defaults.conf. Right?
> >
> > Any ideas how to do that?
> >
> > Thanks,
> > --
> > Jianshi Huang
> >
> > LinkedIn: jianshi
> > Twitter: @jshuang
> > Github & Blog: http://huangjs.github.com/
>
--
Jianshi Huang
LinkedIn: jianshi
Twitter: @jshuang
Github & Blog: http://huangjs.github.com/
