Github user steveloughran commented on the issue: https://github.com/apache/spark/pull/21146 FYI, I just did a kinesis test run with this PR on a JVM with the unlimited JCE installed (explicitly verified by shasum of the relevant JARs); failure with cert errors. ``` success on second attempt after a Kinesis throttling exception - retry success on second attempt after a Kinesis dependency exception - retry failed after a shutdown exception - retry failed after an invalid state exception - retry failed after unexpected exception - retry failed after exhausting all retries WithAggregationKinesisBackedBlockRDDSuite: [2018-08-13 22:01:12.872175] [0x000070000b6f9000] [info] [kinesis_producer.cc:79] Created pipeline for stream "KinesisTestUtils-3116048591482212471" [2018-08-13 22:01:12.872918] [0x000070000b6f9000] [info] [shard_map.cc:83] Updating shard map for stream "KinesisTestUtils-3116048591482212471" [2018-08-13 22:01:13.040026] [0x000070000b905000] [error] [http_client.cc:148] Failed to open connection to monitoring.us-west-2.amazonaws.com:443 : certificate verify failed [2018-08-13 22:01:13.054034] [0x000070000b905000] [error] [http_client.cc:148] Failed to open connection to kinesis.us-west-2.amazonaws.com:443 : certificate verify failed [2018-08-13 22:01:15.943229] [0x000070000b905000] [error] [http_client.cc:148] Failed to open connection to kinesis.us-west-2.amazonaws.com:443 : certificate verify failed [2018-08-13 22:01:18.924473] [0x000070000b905000] [error] [http_client.cc:148] Failed to open connection to kinesis.us-west-2.amazonaws.com:443 : certificate verify failed [2018-08-13 22:01:21.919673] [0x000070000b905000] [error] [http_client.cc:148] Failed to open connection to kinesis.us-west-2.amazonaws.com:443 : certificate verify failed [2018-08-13 22:01:25.121685] [0x000070000b905000] [error] [http_client.cc:148] Failed to open connection to kinesis.us-west-2.amazonaws.com:443 : certificate verify failed [2018-08-13 22:01:27.925785] [0x000070000b905000] [error] [http_client.cc:148] Failed to open connection to kinesis.us-west-2.amazonaws.com:443 : certificate verify failed [2018-08-13 22:01:30.917030] [0x000070000b905000] [error] [http_client.cc:148] Failed to open connection to kinesis.us-west-2.amazonaws.com:443 : certificate verify failed [2018-08-13 22:01:33.960962] [0x000070000b905000] [error] [http_client.cc:148] Failed to open connection to kinesis.us-west-2.amazonaws.com:443 : certificate verify failed [2018-08-13 22:01:36.926987] [0x000070000b905000] [error] [http_client.cc:148] Failed to open connection to kinesis.us-west-2.amazonaws.com:443 : certificate verify failed [2018-08-13 22:01:39.912528] [0x000070000b905000] [error] [http_client.cc:148] Failed to open connection to kinesis.us-west-2.amazonaws.com:443 : certificate verify failed [2018-08-13 22:01:42.911743] [0x000070000b905000] [error] [http_client.cc:148] Failed to open connection to kinesis.us-west-2.amazonaws.com:443 : certificate verify failed org.apache.spark.streaming.kinesis.WithAggregationKinesisBackedBlockRDDSuite *** ABORTED *** java.lang.IllegalArgumentException: requirement failed: Need data to be sent to multiple shards at scala.Predef$.require(Predef.scala:224) at org.apache.spark.streaming.kinesis.KinesisBackedBlockRDDTests$$anonfun$beforeAll$1.apply$mcV$sp(KinesisBackedBlockRDDSuite.scala:47) at org.apache.spark.streaming.kinesis.KinesisFunSuite$class.runIfTestsEnabled(KinesisFunSuite.scala:41) at org.apache.spark.streaming.kinesis.KinesisBackedBlockRDDTests.runIfTestsEnabled(KinesisBackedBlockRDDSuite.scala:25) at org.apache.spark.streaming.kinesis.KinesisBackedBlockRDDTests.beforeAll(KinesisBackedBlockRDDSuite.scala:42) at org.scalatest.BeforeAndAfterAll$class.liftedTree1$1(BeforeAndAfterAll.scala:212) at org.scalatest.BeforeAndAfterAll$class.run(BeforeAndAfterAll.scala:210) at org.apache.spark.SparkFunSuite.run(SparkFunSuite.scala:52) at org.scalatest.Suite$class.callExecuteOnSuite$1(Suite.scala:1210) at org.scalatest.Suite$$anonfun$runNestedSuites$1.apply(Suite.scala:1257) ... the assert failure is possibly a followon from the previous problem
--- --------------------------------------------------------------------- To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org