Re: PutRecord sinks

2021-03-25 Thread Chris Sampson
Pretty sure I'd have use of the second one and think it'd be a good addition - currently I split a file by record and push each one individually to the DMC (e.g. as a way of reporting errors between Wait/Notify branches of a Flow), which is obviously not very efficient. --- *Chris Sampson* IT

Re: PutRecord sinks

2021-03-25 Thread Pierre Villard
Hi Philip, I'd definitely see a lot of use cases for the first one. I'd however provide an option to either have one request per record, or have a single HTTP call to send the complete set of records to the endpoint. I don't have a use case for the second one, but if you have one, that's great

PutRecord sinks

2021-03-25 Thread Phillip Grenier
Looking for feedback on if 2 new sinks would be accepted by the community. 1. InvokeHttpSink: that would make an http request per record, to insert or update a remote end point 2. DistributredMapCacheSink: to insert records into the cache map by a key given a record path