[GitHub] spark pull request #20834: [SPARK-23695][PYTHON] Fix the error message for K...

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/spark/pull/20834


---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request #20834: [SPARK-23695][PYTHON] Fix the error message for K...

[HyukjinKwon]

GitHub user HyukjinKwon opened a pull request:

https://github.com/apache/spark/pull/20834

[SPARK-23695][PYTHON] Fix the error message for Kinesis streaming tests

## What changes were proposed in this pull request?

```bash
ENABLE_KINESIS_TESTS=1 SPARK_TESTING=1 bin/pyspark pyspark.streaming.tests
```

Before:

```
Skipped test_flume_stream (enable by setting environment variable 
ENABLE_FLUME_TESTS=1Skipped test_kafka_stream (enable by setting environment 
variable ENABLE_KAFKA_0_8_TESTS=1Traceback (most recent call last):
  File 
"/usr/local/Cellar/python/2.7.14_3/Frameworks/Python.framework/Versions/2.7/lib/python2.7/runpy.py",
 line 174, in _run_module_as_main
"__main__", fname, loader, pkg_name)
  File 
"/usr/local/Cellar/python/2.7.14_3/Frameworks/Python.framework/Versions/2.7/lib/python2.7/runpy.py",
 line 72, in _run_code
exec code in run_globals
  File "/.../spark/python/pyspark/streaming/tests.py", line 1572, in 

% kinesis_asl_assembly_dir) +
NameError: name 'kinesis_asl_assembly_dir' is not defined
```

After:

```
Skipped test_flume_stream (enable by setting environment variable 
ENABLE_FLUME_TESTS=1Skipped test_kafka_stream (enable by setting environment 
variable ENABLE_KAFKA_0_8_TESTS=1Traceback (most recent call last):
  File 
"/usr/local/Cellar/python/2.7.14_3/Frameworks/Python.framework/Versions/2.7/lib/python2.7/runpy.py",
 line 174, in _run_module_as_main
"__main__", fname, loader, pkg_name)
  File 
"/usr/local/Cellar/python/2.7.14_3/Frameworks/Python.framework/Versions/2.7/lib/python2.7/runpy.py",
 line 72, in _run_code
exec code in run_globals
  File "/.../spark/python/pyspark/streaming/tests.py", line 1576, in 

"You need to build Spark with 'build/sbt -Pkinesis-asl "
Exception: Failed to find Spark Streaming Kinesis assembly jar in 
/.../spark/external/kinesis-asl-assembly. You need to build Spark with 
'build/sbt -Pkinesis-asl assembly/package 
streaming-kinesis-asl-assembly/assembly'or 'build/mvn -Pkinesis-asl package' 
before running this test.
```

## How was this patch tested?

Manually tested.

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/HyukjinKwon/spark minor-variable

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/spark/pull/20834.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #20834


commit d2fe1e9a3dfdba9958b9ffa893fd903e902b4033
Author: hyukjinkwon 
Date:   2018-03-15T11:48:48Z

Fix the error message for Kinesis streaming tests




---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org