Re: Release 2.26 has been cut - Update 1

There are still extant issues [1] that need handling, so if you've seen a comment from me on one of those issues, please respond if a cherry pick is needed, or mark the issue as resolved. So no RC has been cut at this time. However, the nightly snapshot issue was resolved! [2] Thank you! This w

Re: Cross language pipeline example

Thank you everyone for the examples! Looking forward to see the updated documents as well! Best, Ke > On Nov 12, 2020, at 1:17 PM, Chamikara Jayalath wrote: > > Seems like a good place to promote this PR that adds documentation for > cross-language transforms :) > https://github.com/apache/be

Re: Requesting Beam JIRA Permissions

Welcome back Vachan! Good to have you contributing again! On Wed, 11 Nov 2020 at 11:21, Vachan Shetty wrote: > Thank you :D > > On Wed, Nov 11, 2020 at 1:10 PM Pablo Estrada wrote: > >> Heyyy Vachaaan! >> Thanks for contributing. I'll be happy to review your work on the Beam >> connector. Feel

Re: PTransform Annotations Proposal

(Disclaimer, Mirac and their team did approach me about this beforehand as their interest is in the Go SDK.) +1 I think it's a good idea. As you've pointed out, there are many opportunities for optional pipeline analysis here as well. A strawman counter point would be to re-used the static Displa

Re: Cross language pipeline example

Seems like a good place to promote this PR that adds documentation for cross-language transforms :) https://github.com/apache/beam/pull/13317 This covers the following for both Java and Python SDKs. * Creating new cross-language transforms - primary audience will be transform authors who wish to m

Re: Question about LOGICAL_AND

Or the question is, which BeamSQL dialect you are using? -Rui On Thu, Nov 12, 2020 at 12:41 PM Robin Qiu wrote: > Hi Sonam, AND operator is already defined by Calcite and the mapping is > here: > https://github.com/apache/beam/blob/816017e44e3209d334f4f3b2bc3fa829663c530e/sdks/java/extensions

Re: Question about LOGICAL_AND

Hi Sonam, AND operator is already defined by Calcite and the mapping is here: https://github.com/apache/beam/blob/816017e44e3209d334f4f3b2bc3fa829663c530e/sdks/java/extensions/sql/zetasql/src/main/java/org/apache/beam/sdk/extensions/sql/zetasql/translation/SqlOperatorMappingTable.java#L39. Is this

Re: Problem being encountered while running the Query with COUNTIF function

COUNTIF was not defined properly. See my response to the other thread (let's continue discussion there to avoid context fragmentation). On Thu, Nov 12, 2020 at 10:16 AM Ahmet Altay wrote: > +Kenneth Knowles +Andrew Pilloud > +Robin > Qiu > > On Thu, Nov 12, 2020 at 9:32 AM Sonam Ramchand < >

Re: Question about LOGICAL_AND

If you're defining a new built-in function in ZetaSQL, you can define an operator for it here: https://github.com/apache/beam/blob/master/sdks/java/extensions/sql/zetasql/src/main/java/org/apache/beam/sdk/extensions/sql/zetasql/translation/SqlOperators.java Then add the operator the table here: ht

Python precommit breakage

Hi all, the breakage on Python precommit is due to some dependency issues between boto3, requests and urllib3. This is not a flake, so it won't help to retry your tests. https://github.com/apache/beam/pull/13321 is out to fix it by adding an upper bound on urllib3. I've asked Valentyn to review it

Question about LOGICAL_AND

There is no LOGICAL_AND operator in SqlStdOperatorTable, is there any other way to implement LOGICAL_AND? -- Regards, *Sonam* Software Engineer Mobile: +92 3088337296

Re: Bigtable table for BeamSQL - request for a review

Thanks Piotr, I should be able to have an initial review by the end of this week. -Rui On Thu, Nov 12, 2020 at 10:20 AM Piotr Szuberski < piotr.szuber...@polidea.com> wrote: > I've finished the Bigtable for BeamSQL with read operation (for now > without filters) - would anyone like to review

Bigtable table for BeamSQL - request for a review

I've finished the Bigtable for BeamSQL with read operation (for now without filters) - would anyone like to review it? Link: https://github.com/apache/beam/pull/13319 Thanks in advance!

Re: Problem being encountered while running the Query with COUNTIF function

+Kenneth Knowles +Andrew Pilloud +Robin Qiu On Thu, Nov 12, 2020 at 9:32 AM Sonam Ramchand < sonam.ramch...@venturedive.com> wrote: > Hi Devs, > > I am trying to implement the COUNTIF function for Beam SQL ZetaSQL > dialect, as CombineFn. > > Where I try to run the Test query (SELECT COUNTIF

PTransform Annotations Proposal

Hi all, We would like to propose adding functionality to add annotations to Beam transforms. These annotations would be readable by the runner, and the runner could then act on this information; for example by doing some special resource allocation. There have been discussions around annotations (

Problem being encountered while running the Query with COUNTIF function

Hi Devs, I am trying to implement the COUNTIF function for Beam SQL ZetaSQL dialect, as CombineFn. Where I try to run the Test query (SELECT COUNTIF(f_long > 0) AS countif_no FROM PCOLLECTION GROUP BY f_int2), I am getting Exception Caused by: org.apache.beam.vendor.calcite.v1_20_0.org.apache.c

Re: Cross language pipeline example

I was not aware of these examples Brian, thanks for sharing. Maybe we should make these examples more discoverable on the website or as part of Beam's programming guide. It would be nice to have an example of the opposite too, calling a Python transform from Java. Additionally Java users who want

Re: [blog about Beam]

Hi Pablo, Thanks for reading these. Etienne On 10/11/2020 20:09, Pablo Estrada wrote: Thanks Etienne! I read your post on why we can't have multiple aggregations in Spark Streaming. It was informative. Thanks for writing these! Best -P. On Tue, Nov 10, 2020 at 3:39 AM Etienne Chauchot