Hi all,
it's been 3 weeks since the survey on ES versions the users use.
The survey received very few responses: only 9 responses for now
(multiple versions possible of course). The responses are the following:
ES2: 0 clients, ES5: 1, ES6: 5, ES7: 8
It tends to go toward a drop of ES2 suppor
Hi
Context
Hi all, I have been using apache beam pipelines to generate columnar DB to
store in GCS, I have a datastream coming in from kafka and have windows of
1m.
I want to transform all data of that 1m window into a columnar DB file (ORC
in my case, can be parquet or anything else), I have wri
Slowness how?
Is the pipeline getting backed up so that the pipeline is falling behind
compared to where the Kafka source is?
Is the watermark associated with Kafka advancing?
On Fri, Mar 6, 2020 at 5:39 AM Anand Singh Kunwar
wrote:
> Hi
> Context
>
> Hi all, I have been using apache beam pipeli
Hi
The slowness is consumption not matching the rate of production in kafka.
In case I just consume messages from kafka and do nothing (no group by key)
the consumption matches up.
My watermark is one minute behind the kafka message.
Best
Anand Singh Kunwar
On Fri, Mar 6, 2020, 22:47 Luke Cwik w
Since the user provides backendVersion, here are some possible levels of
things to add in expand() based on that (these are extra niceties beyond
the agreed number of releases to remove)
- WARN for backendVersion < n
- reject for backendVersion < n with opt-in pipeline option to keep it
working
Hi,
I think WARN makes sense and the safest approach. It allows users to be notify
and eventually update or back on previous Beam IO version.
Regards
JB
> Le 6 mars 2020 à 18:49, Kenneth Knowles a écrit :
>
> Since the user provides backendVersion, here are some possible levels of
> things t