Note I'm +1 with the doc changed to tell users to export SPARK_YARN_MODE=true before using spark-shell on yarn.
I tested it on both hadoop 0.23 and 2.3 clusters using secure hdfs on linux. Tom On Tuesday, April 1, 2014 1:44 PM, Tom Graves <tgraves...@yahoo.com> wrote: No one else has reported seeing the issue so I think documenting it is fine. Tom On Tuesday, April 1, 2014 1:40 PM, Patrick Wendell <pwend...@gmail.com> wrote: Tom, Given this is a pretty straightforward workaround, what do yo think about the following course of action: (a) We can put the workaround in the docs for 0.9.1. We don't need to do a new RC/vote for this since we can update the published docs independently. (b) We try to get a fix in for this into the 0.9 branch so it can end up in 0.9.2. But this takes the fix off the critical path for this release. - Patrick On Tue, Apr 1, 2014 at 7:28 AM, Tom Graves <tgraves...@yahoo.com> wrote: > Thanks for extending the voting. > > Unfortunately I've found an issue with the spark-shell in yarn-client > mode. It doesn't work with secure HDFS unless you > export SPARK_YARN_MODE=true before starting the shell, or if you happen to > do something immediately with HDFS. If you wait for the connection to the > namenode to timeout it will fail. > > I think it was actually this way in the 0.9 release also so I thought I > would send this and get peoples feedback to see if you want it fixed? > > Another option would be to document that you have to export > SPARK_YARN_MODE=true for the shell. The fix actually went in with the > authentication changes I made in master but I never realized that change > needed to apply to 0.9. > > > https://github.com/apache/spark/commit/7edbea41b43e0dc11a2de156be220db8b7952d01#diff-0ae5b834ce90ec37c19af35aa7a5e1a0 > > See the SparkILoop diff. > > Tom > > > On Monday, March 31, 2014 1:33 PM, Tathagata Das < > tathagata.das1...@gmail.com> wrote: > > Yes, lets extend the vote for two more days from now. So the vote is open > till Wednesday, April 02, at 20:00 UTC > > On that note, my +1 > > > TD > > > > > > > On Mon, Mar 31, 2014 at 9:57 AM, Patrick Wendell <pwend...@gmail.com> > wrote: > > Yeah good point. Let's just extend this vote another few days? > > > > > > > >On Mon, Mar 31, 2014 at 8:12 AM, Tom Graves <tgraves...@yahoo.com> wrote: > > > >> I should probably pull this off into another thread, but going forward > can > >> we try to not have the release votes end on a weekend? Since we only > seem > >> to give 3 days, it makes it really hard for anyone who is offline for > the > >> weekend to try it out. Either that or extend the voting for more then > 3 > >> days. > >> > >> Tom > >> On Monday, March 31, 2014 12:50 AM, Patrick Wendell <pwend...@gmail.com > > > >> wrote: > >> > >> TD - I downloaded and did some local testing. Looks good to me! > >> > >> +1 > >> > >> You should cast your own vote - at that point it's enough to pass. > >> > >> - Patrick > >> > >> > >> > >> On Sun, Mar 30, 2014 at 9:47 PM, prabeesh k <prabsma...@gmail.com> > wrote: > >> > >> > +1 > >> > tested on Ubuntu12.04 64bit > >> > > >> > > >> > On Mon, Mar 31, 2014 at 3:56 AM, Matei Zaharia < > matei.zaha...@gmail.com > >> > >wrote: > >> > > >> > > +1 tested on Mac OS X. > >> > > > >> > > Matei > >> > > > >> > > On Mar 27, 2014, at 1:32 AM, Tathagata Das < > >> tathagata.das1...@gmail.com> > >> > > wrote: > >> > > > >> > > > Please vote on releasing the following candidate as Apache Spark > >> > version > >> > > 0.9.1 > >> > > > > >> > > > A draft of the release notes along with the CHANGES.txt file is > >> > > > attached to this e-mail. > >> > > > > >> > > > The tag to be voted on is v0.9.1-rc3 (commit 4c43182b): > >> > > > > >> > > > >> > > >> > https://git-wip-us.apache.org/repos/asf?p=spark.git;a=commit;h=4c43182b6d1b0b7717423f386c0214fe93073208 > >> > > > > >> > > > The release files, including signatures, digests, etc. can be > found > >> at: > >> > > > http://people.apache.org/~tdas/spark-0.9.1-rc3/ > >> > > > > >> > > > Release artifacts are signed with the following key: > >> > > > https://people.apache.org/keys/committer/tdas.asc > >> > > > > >> > > > The staging repository for this release can be found at: > >> > > > > >> > > https://repository.apache.org/content/repositories/orgapachespark-1009/ > >> > > > > >> > > > The documentation corresponding to this release can be found at: > >> > > > http://people.apache.org/~tdas/spark-0.9.1-rc3-docs/ > >> > > > > >> > > > Please vote on releasing this package as Apache Spark 0.9.1! > >> > > > > >> > > > The vote is open until Sunday, March 30, at 10:00 UTC and passes > if > >> > > > a majority of at least 3 +1 PMC votes are cast. > >> > > > > >> > > > [ ] +1 Release this package as Apache Spark 0.9.1 > >> > > > [ ] -1 Do not release this package because ... > >> > > > > >> > > > To learn more about Apache Spark, please see > >> > > > http://spark.apache.org/ > >> > > > <CHANGES.txt><RELEASE_NOTES.txt> > >> > > > >> > > > >> > > >> > > >
