GitHub user wypoon opened a pull request:
https://github.com/apache/spark/pull/19703
[SPARK-22403][SS] Add optional checkpointLocation argument to
StructuredKafkaWordCount example
## What changes were proposed in this pull request?
When run in YARN cluster mode, the StructuredKafkaWordCount example fails
because Spark tries to create a temporary checkpoint location in a subdirectory
of the path given by java.io.tmpdir, and YARN sets java.io.tmpdir to a path in
the local filesystem that usually does not correspond to an existing path in
the distributed filesystem.
Add an optional checkpointLocation argument to the StructuredKafkaWordCount
example so that users can specify the checkpoint location and avoid this issue.
## How was this patch tested?
Built and ran the example manually on YARN client and cluster mode.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/wypoon/spark SPARK-22403
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/19703.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #19703
----
commit 171496a424ed23ebadafe29ff74de72f3db5a49f
Author: Wing Yew Poon <wyp...@cloudera.com>
Date: 2017-11-09T04:06:48Z
Add optional checkpointLocation argument to StructuredKafkaWordCount example
----
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
