Hi
I am working on designing and building a SourceConnector to run in a
distributed mode to transfer data from Mainframe data sources which are exposed
as RESTful services. So , planning to spin multiple workers handling and
processing subset of data coordinating with all workers in distributed mode.
Any recommendations or references around this implementation and how to
distribute the data and handle offset management in the distributed environment.
The volume of data inflow will be so huge and looking for a scalable ,
distributed and fault tolerant model.
Any small input would be of great help.
RegardsSaravanan
