GitHub user HeartSaVioR opened a pull request:
https://github.com/apache/spark/pull/21497
[SPARK-24466][SS] Fix TextSocketMicroBatchReader to be compatible with
netcat again
## What changes were proposed in this pull request?
TextSocketMicroBatchReader was no longer be compatible with netcat due to
launching temporary reader for reading schema, and closing reader, and
re-opening reader. While reliable socket server should be able to handle this
without any issue, nc command normally can't handle multiple connections and
simply exits when closing temporary reader.
This patch fixes TextSocketMicroBatchReader to be compatible with netcat
again, via deferring opening socket to the first call of planInputPartitions()
instead of constructor.
## How was this patch tested?
Added unit test which fails on current and succeeds with the patch. And
also manually tested.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/HeartSaVioR/spark SPARK-24466
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/21497.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #21497
----
commit 7b875279742fa295ab513cf8a489830237953d0c
Author: Jungtaek Lim <kabhwan@...>
Date: 2018-06-05T07:57:42Z
SPARK-24466 Fix TextSocketMicroBatchReader to be compatible with netcat
again
----
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
