sekikn commented on issue #24651: [SPARK-27420][DSTREAMS][Kinesis] KinesisInputDStream should expose a way to configure CloudWatch metrics URL: https://github.com/apache/spark/pull/24651#issuecomment-523816081 Hmm, I'm not sure why the CI failed on PySpark, because it succeeds with the same options on my local environment. ``` $ git checkout master $ curl -sLO https://github.com/apache/spark/pull/24651.patch $ git apply 24651.patch $ build/mvn clean install -Pkinesis-asl -DskipTests (snip) [INFO] Spark Project Kinesis Assembly ..................... SUCCESS [ 5.044 s] [INFO] ------------------------------------------------------------------------ [INFO] BUILD SUCCESS [INFO] ------------------------------------------------------------------------ [INFO] Total time: 20:39 min [INFO] Finished at: 2019-08-22T15:20:42+09:00 [INFO] ------------------------------------------------------------------------ $ ENABLE_KINESIS_TESTS=1 python/run-tests --modules=pyspark-streaming,pyspark-mllib,pyspark-ml Running PySpark tests. Output is in /home/sekikn/repos/spark/python/unit-tests.log Will test against the following Python executables: ['python2.7', 'python3.6', 'pypy'] Will test the following Python modules: ['pyspark-streaming', 'pyspark-mllib', 'pyspark-ml'] Starting test(pypy): pyspark.streaming.tests.test_dstream Starting test(pypy): pyspark.streaming.tests.test_listener Starting test(pypy): pyspark.streaming.tests.test_context Starting test(pypy): pyspark.streaming.tests.test_kinesis (snip) Finished test(pypy): pyspark.streaming.tests.test_kinesis (178s) (snip) Tests passed in 1140 seconds Skipped tests in pyspark.ml.tests.test_image with python2.7: test_read_images_multiple_times (pyspark.ml.tests.test_image.ImageFileFormatOnHiveContextTest) ... skipped 'Hive is not available.' Skipped tests in pyspark.ml.tests.test_image with python3.6: test_read_images_multiple_times (pyspark.ml.tests.test_image.ImageFileFormatOnHiveContextTest) ... skipped 'Hive is not available.' $ echo $? 0 ```
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services --------------------------------------------------------------------- To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
