Re: [External] : Re: outputReceiver.output() does not emit the result immediately

Hi Boyuan, Thanks for replying. We are using beam 2.25.0 and direct runner for testing. We are trying to develop an unbounded streaming service connector with splittable DoFn. In our connector.read(), we want to commit the message back to stream after output the record to downstream user

Re: Beam SQL UDF with variable arguments list?

Thanks, that explains what I'm seeing. Niels On Tue, 26 Jan 2021, 20:57 Rui Wang, wrote: > Yes I think Calcite does not support varargs in for scalar function (so in > UDF). Please check this JIRA: > https://issues.apache.org/jira/browse/CALCITE-2772 > > > -Rui > > On Tue, Jan 26, 2021 at 2:04

Re: Beam support Flink Async I/O operator

+dev On Tue, Jan 26, 2021 at 1:07 PM Eleanore Jin wrote: > Hi community, > > Does Beam support Flink Async I/O operator? if so, can you please share > the doc, and if not, is there any workaround to achieve the same in Beam > semantics? > > >

Beam support Flink Async I/O operator

Hi community, Does Beam support Flink Async I/O operator? if so, can you please share the doc, and if not, is there any workaround to achieve the same in Beam semantics? https://ci.apache.org/projects/flink/flink-docs-stable/dev/stream/operators/asyncio.html

Re: Beam SQL UDF with variable arguments list?

Yes I think Calcite does not support varargs in for scalar function (so in UDF). Please check this JIRA: https://issues.apache.org/jira/browse/CALCITE-2772 -Rui On Tue, Jan 26, 2021 at 2:04 AM Niels Basjes wrote: > Hi, > > I want to define a Beam SQL user defined function that accepts a

Re: outputReceiver.output() does not emit the result immediately

+dev Hi Yu, Which runner are you using for your pipeline? Also it would be helpful to share your pipeline code as well. On Mon, Jan 25, 2021 at 10:19 PM wrote: > Hi Beam Community, > > I have a splittable `DoFn` that reads message from some stream and output > the result to down stream. The

ElasticsearchIO metrics

Hi, I have built a pipeline performing reads using ElasticsearchIO from a large Elasticsearch index version 6.5.3. While it seems to run just fine using Flink runners there is absolutely no feedback in terms of progress. Am I correct in saying that this connector has no progress reporting

Beam SQL UDF with variable arguments list?

Hi, I want to define a Beam SQL user defined function that accepts a variable list of arguments (which may be empty). What I essentially would like to have is public class ParseUserAgentJson implements BeamSqlUdf { public static String eval( String input,