They serve a similar purpose.
OutputFormats originate from the Batch API, whereas SinkFunctions are a
Streaming API concept.
You can however use OutputFormats in the Streaming API using the
DataStrea#writeUsingOutputFormat.
Regards,
Chesnay
On 05.07.2016 12:51, Harikrishnan S wrote:
Ah that makes send. Also what's the difference between a
RichOutputFormat and a RichSinkFunction ? Can I use JDBCOutputFormat
as a sink in a stream ?
On Tue, Jul 5, 2016 at 3:53 PM, Chesnay Schepler <ches...@apache.org
<mailto:ches...@apache.org>> wrote:
Hello,
an instance of the JDBCOutputFormat will use a single connection
to send all values.
Essentially
- open(...) is called at the very start to create the connection
- then all invoke/writeRecord calls are executed (using the same
connection)
- then close() is called to clean up.
The total number of connections made to the database depends on
the parallelism of the Sink, as every parallel instance creates
it's own connection.
Regards,
Chesnay
On 05.07.2016 12:04, Harikrishnan S wrote:
The basic idea was that I would create a pool of connections in
the open() method in a custom sink and each invoke() method gets
one connection from the pool and does the upserts needed. I might
have misunderstood how sinks work in flink though.
On Tue, Jul 5, 2016 at 2:22 PM, Flavio Pompermaier
<pomperma...@okkam.it <mailto:pomperma...@okkam.it>> wrote:
why do you need a connection pool?
On 5 Jul 2016 11:41, "Harikrishnan S" <hihari...@gmail.com
<mailto:hihari...@gmail.com>> wrote:
Hi,
Are there any examples of implementing a jdbc sink in
flink using a connection pool ?
Thanks
On Tue, Jul 5, 2016 at 2:00 PM, Harikrishnan S
<hihari...@gmail.com <mailto:hihari...@gmail.com>> wrote:
Hi,
Are there any examples of implementing a jdbc sink in
flink using a connection pool ?
Thanks
On Tue, Jul 5, 2016 at 1:57 PM, Harikrishnan S
<hihari...@gmail.com <mailto:hihari...@gmail.com>> wrote:
Hi,
Are there any examples of implementing a jdbc
sink in flink using a connection pool ?
Thanks
