Samuel Tatipamula created STORM-3336:
----------------------------------------
Summary: Create a spout for listening and emitting the change
streams in MongoDB
Key: STORM-3336
URL: https://issues.apache.org/jira/browse/STORM-3336
Project: Apache Storm
Issue Type: New Feature
Components: storm-mongodb
Reporter: Samuel Tatipamula
MongoDB provides real-time updates to documents at a collection or database
level using Change Streams. From MongoDB documentation
([https://docs.mongodb.com/manual/changeStreams/)]
{quote}Change streams allow applications to access real-time data changes
without the complexity and risk of tailing the oplog. Applications can use
change streams to subscribe to all data changes on a single collection, a
database, or an entire deployment, and immediately react to them. Because
change streams use the aggregation framework, applications can also filter for
specific changes or transform the notifications at will.
{quote}
There can be use cases for external systems to process these change events in
real time.
The idea is to create a spout, which can listen to change events and emit them
as tuples. There should also be a mechanism to save the state of change streams
for a spout's id using `resumeToken` provided by MongoDB to resume the topology
from that point.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
