Github user mccheah commented on a diff in the pull request: https://github.com/apache/spark/pull/21366#discussion_r191502285 --- Diff: pom.xml --- @@ -760,6 +760,12 @@ <version>1.10.19</version> <scope>test</scope> </dependency> + <dependency> --- End diff -- > I think I'm a bit concerned adding rxjava to the top level pom and to dev/deps/spark-deps-hadoop-* can it be just a `<arrow.version>0.8.0</arrow.version>` thing and not a dependency? Unsure what you mean here - we're using rxjava itself specifically to do the event handling in this PR. See https://github.com/apache/spark/pull/21366/files#diff-ae4cd884779fb4c3db58958ab984db59R40. If we wanted an alternative approach we can build something from first principles (executor service / manual linked blocking queues) but I like the elegance that rx-java buys us here. The code we'd save building ourselves seems worthwhile.
--- --------------------------------------------------------------------- To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org