Hi Andrew, as far as I know there is nothing particularly special about the sink in terms of how it handles state or time. You can not leave the pipeline "unfinished", only sinks trigger the execution of the whole pipeline.
Cheers, Konstantin On Fri, Jan 24, 2020 at 5:59 PM Andrew Roberts <arobe...@fuze.com> wrote: > Hello, > > I’m trying to push some behavior that we’ve currently got in a large, > stateful SinkFunction implementation into Flink’s windowing system. The > task at hand is similar to what StreamingFileSink provides, but more > flexible. I don’t want to re-implement that sink, because it uses the > StreamingRuntimeContext.getProcessingTimeService() via a cast - that class > is marked as internal, and I’d like to avoid the exposure to an interface > that could change. Extending it similarly introduces complexity I would > rather not add to our codebase. > > WindowedStream.process() provides more or less the pieces I need, but the > stream continues on after a ProcessFunction - there’s no way to process() > directly into a sink. I could use a ProcessFunction[In, Unit, Key, Window], > and follow that immediately with a no-op sink that discards the Unit > values, or I could just leave the stream “unfinished," with no sink. > > Is there a downside to either of these approaches? Is there anything > special about doing sink-like work in a ProcessFunction or FlatMapFunction > instead of a SinkFunction? > > Thanks, > > Andrew > > > > -- > *Confidentiality Notice: The information contained in this e-mail and any > > attachments may be confidential. If you are not an intended recipient, you > > are hereby notified that any dissemination, distribution or copying of this > > e-mail is strictly prohibited. If you have received this e-mail in error, > > please notify the sender and permanently delete the e-mail and any > > attachments immediately. You should not retain, copy or use this e-mail or > > any attachment for any purpose, nor disclose all or any part of the > > contents to any other person. Thank you.* > -- Konstantin Knauf | Solutions Architect +49 160 91394525 Follow us @VervericaData Ververica <https://www.ververica.com/> -- Join Flink Forward <https://flink-forward.org/> - The Apache Flink Conference Stream Processing | Event Driven | Real Time -- Ververica GmbH | Invalidenstrasse 115, 10115 Berlin, Germany -- Ververica GmbH Registered at Amtsgericht Charlottenburg: HRB 158244 B Managing Directors: Timothy Alexander Steinert, Yip Park Tung Jason, Ji (Tony) Cheng
