Small DNS update for Beam Playground

2023-05-16 Thread Pablo Estrada via dev
Hi all, I would like to update everyone on a small change for the Beam Playground. We've worked to move it from App Engine to Kubernetes. This means that we have to 'flip the switch', and change the domain name to point to the k8s endpoint. This does not require any action from you, but rather it'

Re: Incorrect python bigquery storage read API docs

2023-02-06 Thread Pablo Estrada via dev
("This is correct" meaning that you're correct Parth) On Mon, Feb 6, 2023 at 12:39 PM Pablo Estrada wrote: > This is correct! The Bigquery Storage Read API is supported in Python. We > need to fix that doc. > -P. > > On Mon, Feb 6, 2023 at 11:58 AM Ahmet Altay wrote: > >> Adding @Pablo Estrada

Re: Incorrect python bigquery storage read API docs

2023-02-06 Thread Pablo Estrada via dev
This is correct! The Bigquery Storage Read API is supported in Python. We need to fix that doc. -P. On Mon, Feb 6, 2023 at 11:58 AM Ahmet Altay wrote: > Adding @Pablo Estrada who reviewed that issue, > hopefully he could help. > > On Sun, Feb 5, 2023 at 4:37 PM Parth Doshi wrote: > >> Hi, >> >

Re: [fyi][discuss] Making jackson-dataformat-yaml a provided/optional dependency

2023-02-06 Thread Pablo Estrada via dev
It's worth mentioning that neither of the libraries (jackson-dataformat-yaml + snakeyaml) have a newer version without the CVE. -P. On Mon, Feb 6, 2023 at 9:19 AM Pablo Estrada wrote: > Hi all, > I am proposing that we make the jackson-dataformat-yaml dependency > optional in our expansion servi

[fyi][discuss] Making jackson-dataformat-yaml a provided/optional dependency

2023-02-06 Thread Pablo Estrada via dev
Hi all, I am proposing that we make the jackson-dataformat-yaml dependency optional in our expansion service module[1]. This is because it depends on SnakeYAML, and there is a known CVE for it[2]. It seems that given the way we use SnakeYAML, the CVE is not feasible to exploit[2], but this will no

[fyi] Updating ip address for Playground staging

2023-02-01 Thread Pablo Estrada via dev
Hi all, this email is just to inform contributors that due to resource constraints in the apache-beam-testing project, we want to update the IP address for the staging environment of the Beam playground from a us-west one to a us-east one. If you have any concerns about this, please reach out to m

Re: Access to Confluence

2023-01-23 Thread Pablo Estrada via dev
+Kenneth Knowles I believe Beam is still built based on JDK 8. I suppose our development manuals could be moved to Java 11 as long as Beam continues to build with JDK 8 in our CI/CD. I don't know enough about Java to know what's the best way of dealing with this : ) -P. On Mon, Jan 23, 2023 at 4:

Re: Access to Confluence

2023-01-12 Thread Pablo Estrada via dev
Okay I've granted you edit privileges! Thanks again for offering to make improvements Ait Zeouay! Best -P. On Thu, Jan 12, 2023 at 7:59 AM AIT-ZEOUAY Amrane < amrane.ait-zeo...@renault.com> wrote: > Hello Pablo, > > > > Thank you for your reply. I tried to edit it by just signing up, but it > di

Re: Access to Confluence

2023-01-11 Thread Pablo Estrada via dev
Hello Ait-Zelouai! (did I get that right?:)) Thanks for the help. You can sign up for an account here: https://cwiki.apache.org/confluence/signup.action After signing up, you can share you username with me, and I'll grant you edit privileges. Best -P. On Wed, Jan 11, 2023 at 10:11 AM AIT-ZEOUAY

Re: Gradle

2023-01-11 Thread Pablo Estrada via dev
The build scan configuration is here: https://github.com/apache/beam/blob/master/settings.gradle.kts#L25-L44 - we already upload for all CI builds. IIUC, all we would need to do is change the server to the node provided by Infra ( https://docs.gradle.com/enterprise/gradle-plugin/#connecting_to_gra

Re: [VOTE] Release 2.43.0, release candidate #2

2022-11-15 Thread Pablo Estrada via dev
+1 (binding) Tested local tests for existing DF templates. On Tue, Nov 15, 2022 at 8:17 AM Alexey Romanenko wrote: > +1 (binding) > > — > Alexey > > On 15 Nov 2022, at 14:37, Ritesh Ghorse via dev > wrote: > > +1 (non-binding) > > Validated Go SDK quickstart on Direct and Dataflow runner. Also

Re: bhulette stepping back (for now)

2022-11-11 Thread Pablo Estrada via dev
I promised I wouldn't cry so I won't. Cya! On Fri, Nov 11, 2022 at 10:46 AM Robin Qiu via dev wrote: > Thanks for your contribution Brian! Hope you enjoy your new team! > > Best, > Robin > > On Fri, Nov 11, 2022 at 10:27 AM Kenneth Knowles wrote: > >> Your contributions have been huge. You will

Re: [ANNOUNCE] New committer: Yi Hu

2022-11-09 Thread Pablo Estrada via dev
+1 thanks Yi : D On Wed, Nov 9, 2022 at 10:47 AM Danny McCormick via dev wrote: > Congrats Yi! I've really appreciated the ways you've consistently taken > responsibility for improving our team's infra and working through sharp > edges in the codebase that others have ignored. This is definitely

Re: github reviewer help / tips

2022-11-08 Thread Pablo Estrada via dev
Nice pointer Valentyn. If you are willing to open the Github page sometimes, it's worth just using the PR tab: https://github.com/pulls - which should show all PRs that mention you/have review requests for you from all repos. -P. On Tue, Nov 8, 2022 at 10:56 AM Valentyn Tymofieiev via dev < dev@b

On wrestling with beam/issues/21104

2022-10-25 Thread Pablo Estrada via dev
Hi all, I've spent a few days trying to narrow down whatever is going on with https://github.com/apache/beam/issues/21104. The bug reproduces frequently on Jenkins and very rarely on my laptop (sometimes ~20 min, sometimes hours, sometimes I give up after it doesn't reproduce all night). Seeing a

Re: [VOTE] Release 2.42.0, release candidate #2

2022-10-13 Thread Pablo Estrada via dev
+1 (binding) I've validated local/unit tests for existing dataflow templates. They look good! Best -P. On Thu, Oct 13, 2022 at 10:41 AM Ning Kang via dev wrote: > +1 Thank you, Robert! > > On Thu, Oct 13, 2022 at 12:47 AM Robert Burke wrote: > >> Hi everyone, >> Please review and vote on the r

Java compatibility checking library

2022-08-29 Thread Pablo Estrada via dev
Hi folks! I learned about this library: https://lvc.github.io/japi-compliance-checker/ It looks like it's able to check API compatibility between versions of libraries. We could instate checks that use this library to be more aware of backwards incompatibilities that we may introduce accidentally,

Re: [ANNOUNCE] Apache Beam 2.41.0 Released

2022-08-25 Thread Pablo Estrada via dev
Thank you Kiley! On Thu, Aug 25, 2022 at 10:55 AM Kiley Sok wrote: > The Apache Beam team is pleased to announce the release of version 2.41.0. > > Apache Beam is an open source unified programming model to define and > execute data processing pipelines, including ETL, batch and stream > (contin

Re: [VOTE] Release 2.41.0, release candidate #1

2022-08-15 Thread Pablo Estrada via dev
+1 - I validated tests/build with existing Dataflow Templates Best -P. On Fri, Aug 12, 2022 at 9:20 PM Ahmet Altay via dev wrote: > +1 - I validated python quickstarts on direct runner. > > Thank you Kiley! > > > > On Thu, Aug 11, 2022 at 9:56 PM Kiley Sok via dev > wrote: > >> Hi everyone, >>

Re: Unable to load class 'org.apache.beam.gradle.BeamModulePlugin'

2022-08-08 Thread Pablo Estrada via dev
Hi there! Did you try following the instructions here: https://cwiki.apache.org/confluence/display/BEAM/Set+up+IntelliJ+from+scratch Or how did you try to load your project into IntelliJ? Best -P. On Mon, Aug 8, 2022 at 8:38 PM 张涛 wrote: > Hi, I download the source code beam-2.40.0.tar.gz, and

Re: [VOTE] Vendored Dependencies Release

2022-08-08 Thread Pablo Estrada via dev
+1 Thanks! -P. On Mon, Aug 8, 2022 at 9:24 AM Chamikara Jayalath via dev < dev@beam.apache.org> wrote: > +1 > > Thanks, > Cham > > On Fri, Aug 5, 2022 at 1:49 PM Luke Cwik via dev > wrote: > >> +1 >> >> I verified the signatures of the artifacts, that the jar doesn't contain >> classes outside o

Re: Join a meeting to help coordinate implementing a Dask Runner for Beam

2022-07-22 Thread Pablo Estrada via dev
+Brian Hulette and +Kenneth Knowles were curious about this effort. Alex can you elaborate on what happened with GH actions? We can help you figure out the CI story. On Tue, Jun 21, 2022 at 11:06 AM Alex Merose wrote: > We had a great meeting last week on this topic! Here is a proposal / > me