Steve Blackmon created STREAMS-183:
--------------------------------------
Summary: streams-local closes persist-writers before datums
in-flight are written
Key: STREAMS-183
URL: https://issues.apache.org/jira/browse/STREAMS-183
Project: Streams
Issue Type: Bug
Reporter: Steve Blackmon
I have observed this behavior when writing datums from a readCurrent provider
through to ElasticsearchPersistWriter and ElasticsearchPersistDeleter.
The log looks like:
421777 [pool-2-thread-7] INFO o.a.s.e.ElasticsearchPersistReader -
PersistReader done
469271 [main] INFO o.a.s.l.builders.LocalStreamBuilder - Attempting to
shutdown tasks
476319 [pool-2-thread-3] ERROR o.a.s.e.ElasticsearchPersistWriter - There was
an error sending the batch: No node available
485275 [main] WARN o.a.s.l.builders.LocalStreamBuilder - Task
org.apache.streams.local.tasks.StreamsPersistWriterTask@6add646d failed to
terminate in allotted timeframe
495276 [pool-2-thread-3] WARN o.a.s.e.ElasticsearchPersistWriter - Catchup was
interrupted. Data may be lost
followed by additional No Node Available exceptions.
When the same stream is executed as a perpetual stream, this problem does not
occur.
Possible solutions:
a) localstreambuilder waits a while between detecting providers have completed
and stopping downstream components
b) localstreambuilder does not stop downstream components until all datums have
been processed/written
c) writers refuse to close until all datums have been written
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
