gt;
> > > > >>
> > > > >> On Thu, Jul 22, 2021 at 4:40 PM Hausmann, Steffen
> > > > >> wrote:
> > > > >>
> > > > >>> Hey,
> > > > >>>
> > > &
support use cases where
> the
> > > >>> destination can only ingest x messages per second or a total
> > > throughput of
> > > >>> y per second. We are also planning to support time outs so that
> data
> > is
> &g
.
> > > > >>
> > > > >>
> > > > >> Best,
> > > > >> Guowei
> > > > >>
> > > > >>
> > > > >> On Thu, Jul 22, 2021 at 4:40 PM Hausmann, Steffen
> > > > >> wrote
the queue
> size.
> > It
> > > >>> also depends on the number of in-flight async requests that must
> not
> > > grow
> > > >>> too large either [1, 2]. We also need to support use cases where
> the
> > > >>> destination ca
ination at least every n seconds by means of
> the
> > >>> `ProcessingTimeService`. The batching configuration will be part of
> the
> > >>> constructor, which has only been indicated in the current prototype
> > but is
> > &g
e sink and pass in the batching configuration from above [4]. The
> >>> `MailboxExecutor` and `ProcessingTimeService` are only relevant for
> sink
> >>> authors who want to create support for a new destination. I would
> expect
> >>> that there are only fe
link-connector-base/src/main/java/org/apache/flink/connector/base/sink/writer/AsyncSinkWriter.java#L118
>>> [2]
>>> https://github.com/sthm/flink/blob/51614dc9371d6e352db768a404ba3cafddad08f0/flink-connectors/flink-connector-base/src/main/java/org/apache/flink/connector/bas
onnector-base/src/main/java/org/apache/flink/connector/base/sink/writer/AsyncSinkWriter.java#L43-L49
>> [4]
>> https://github.com/sthm/flink/blob/51614dc9371d6e352db768a404ba3cafddad08f0/flink-connectors/flink-connector-kinesis-171/src/main/java/software/amazon/flink/connectors/Test.java
ctor-kinesis-171/src/main/java/software/amazon/flink/connectors/Test.java#L45
>
>
>
> From: Arvid Heise
> Date: Tuesday, 20. July 2021 at 23:03
> To: dev , Steffen Hausmann
> Subject: RE: [EXTERNAL] [DISCUSS] FLIP-177: Extend Sink API
>
>
> CAUTION: This email originate
: [EXTERNAL] [DISCUSS] FLIP-177: Extend Sink API
CAUTION: This email originated from outside of the organization. Do not click
links or open attachments unless you can confirm the sender and know the
content is safe.
Hi Guowei,
1. your idea is quite similar to FLIP-171 [1]. The question is if we
Hi, Arvid
1. You are right that the core difference between the two options is
whether to expose `MailboxExecutor`. My personal preference
is that the less internal implementations are exposed to users, the better.
2. `AsyncIO` does not support Batch output, but we can implement one based
on `As
Hi Guowei,
1. your idea is quite similar to FLIP-171 [1]. The question is if we
implement FLIP-171 based on public interfaces (which would require exposing
MailboxExecutor as described here in FLIP-177) or if it's better to
implement it internally and hide it.
The first option is an abstract base
Hi, Avrid
Thank you Avrid for perfecting Sink through this FLIP. I have two little
questions
1. What do you think of us directly providing an interface as follows? In
this way, there may be no need to expose the Mailbox to the user. We can
implement an `AsyncSinkWriterOperator` to control the leng
Dear devs,
today I'd like to start the discussion on the Sink API. I have drafted a
FLIP [1] with an accompanying PR [2].
This FLIP is a bit special as it's actually a few smaller Amend-FLIPs in
one. In this discussion, we should decide on the scope and cut out too
invasive steps if we can't reac
14 matches
Mail list logo
