[
https://issues.apache.org/jira/browse/SPARK-3210?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15386002#comment-15386002
]
Ian Brooks commented on SPARK-3210:
-----------------------------------
Hi,
I have also noticed an issue with resiliency for the flume polling receiver.
This issue I have is as follows
1. Start Flume agent then Spark application
2. Spark application correctly connects to the flume agent and can receive data
what is sent to Flume
3. Restart Flume
4. Spark application doesn't detect that Flume has been restarted and as such
doesn't reconnect at any point preventing the Spark application from receiving
any more data until its restarted.
I've had a trawl through the documentation and source code for
FlumeUtils.createPollingStream but couldn't see anyway to test and reconnect if
needed.
-Ian
> Flume Polling Receiver must be more tolerant to connection failures.
> --------------------------------------------------------------------
>
> Key: SPARK-3210
> URL: https://issues.apache.org/jira/browse/SPARK-3210
> Project: Spark
> Issue Type: Bug
> Components: Streaming
> Reporter: Hari Shreedharan
>
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
