Hi,
I think WARN makes sense and the safest approach. It allows users to be notify
and eventually update or back on previous Beam IO version.
Regards
JB
> Le 6 mars 2020 à 18:49, Kenneth Knowles a écrit :
>
> Since the user provides backendVersion, here are some possible levels of
> things
Since the user provides backendVersion, here are some possible levels of
things to add in expand() based on that (these are extra niceties beyond
the agreed number of releases to remove)
- WARN for backendVersion < n
- reject for backendVersion < n with opt-in pipeline option to keep it
working
Hi
The slowness is consumption not matching the rate of production in kafka.
In case I just consume messages from kafka and do nothing (no group by key)
the consumption matches up.
My watermark is one minute behind the kafka message.
Best
Anand Singh Kunwar
On Fri, Mar 6, 2020, 22:47 Luke Cwik
Slowness how?
Is the pipeline getting backed up so that the pipeline is falling behind
compared to where the Kafka source is?
Is the watermark associated with Kafka advancing?
On Fri, Mar 6, 2020 at 5:39 AM Anand Singh Kunwar
wrote:
> Hi
> Context
>
> Hi all, I have been using apache beam
Hi
Context
Hi all, I have been using apache beam pipelines to generate columnar DB to
store in GCS, I have a datastream coming in from kafka and have windows of
1m.
I want to transform all data of that 1m window into a columnar DB file (ORC
in my case, can be parquet or anything else), I have
Hi all,
it's been 3 weeks since the survey on ES versions the users use.
The survey received very few responses: only 9 responses for now
(multiple versions possible of course). The responses are the following:
ES2: 0 clients, ES5: 1, ES6: 5, ES7: 8
It tends to go toward a drop of ES2
