Re: SPARK environment settings issue when deploying a custom distribution

2017-06-12 Thread Chanh Le
Just add more information how I build the custom distribution. I clone spark repo then switch to branch 2.2 then make distribution that following. λ ~/workspace/big_data/spark/ branch-2.2* λ ~/workspace/big_data/spark/ ./dev/make-distribution.sh --name custom --tgz -Phadoop-2.7

SPARK environment settings issue when deploying a custom distribution

2017-06-12 Thread Chanh Le
Hi everyone, Recently I discovered an issue when processing csv of spark. So I decided to fix it following this https://issues.apache.org/jira/browse/SPARK-21024 I built a custom distribution for internal uses. I built it in my local machine then upload the distribution to server. server's