Re: [PROPOSAL] Preparing for 2.47.0 Release

2023-04-12 Thread Ahmet Altay via dev
Jack, how is the release coming along? On Tue, Apr 4, 2023 at 12:23 PM Jack McCluskey via dev wrote: > Hey everyone, > > I need a PMC member's help adding my pubkey to > https://dist.apache.org/repos/dist/release/beam/KEYS as well as adding > PyPI user jrmccluskey to the maintainers of the Apach

Re: [DISCUSS] @Experimental, @Internal, @Stable, etc annotations

2023-04-12 Thread Ahmet Altay via dev
I agree with Alexey and Byron. 1. We do not have any concrete evidence of our users paying attention to any of those annotations. Experimental API that were in that state for a long while are good examples. A possible exception is a deprecated annotation. My preference would be to simplify annotati

Re: Python 3.11 support in Apache Beam

2023-04-12 Thread Ahmet Altay via dev
Thank you, this is great! Python 3.11 announcement had a claim about performance [1]: "CPython 3.11 is an average of 25% faster than CPython 3.10 as measured with the pyperformance benchmark suite, when compiled with GCC on Ubuntu Linux. Depending on your workload, the overall speedup could be 10

Re: [Python SDK] Use pre-released dependencies for Beam python unit testing

2023-04-12 Thread Yi Hu via dev
Sounds good, thanks! Best, Yi On Wed, Apr 12, 2023 at 2:20 PM Anand Inguva wrote: > @Yi Hu I think adding them to Jenkins or github > actions is okay with me. With Github actions, since we don't use self > hosted runners yet, I worry that action workers might get queued up. > > Also, I plan to

Re: [Python SDK] Use pre-released dependencies for Beam python unit testing

2023-04-12 Thread Anand Inguva via dev
@Yi Hu I think adding them to Jenkins or github actions is okay with me. With Github actions, since we don't use self hosted runners yet, I worry that action workers might get queued up. Also, I plan to not run these on every commit but run it as a cron job(maybe once per day) and also as trigger

Re: [Python SDK] Use pre-released dependencies for Beam python unit testing

2023-04-12 Thread Valentyn Tymofieiev via dev
I think case in point dependency that would benefit from this testing is grpcio, which includes pre-releases, and broke us and multiple of it's released versions were yanked. https://pypi.org/project/grpcio/#history . We can look at how grpcio affected Beam previously. Couple of issues: - https:/

Re: [Python SDK] Use pre-released dependencies for Beam python unit testing

2023-04-12 Thread Yi Hu via dev
Thanks Anand, This would be very helpful to avoid experiencing multiple time ( https://s.apache.org/beam-python-dependencies-pm). One thing to note is that Beam Jenkins CI is experiencing many issues recently, mostly due to that multiple Jenkins plugins does not scale (draining GitHub API call lim

Re: [Python SDK] Use pre-released dependencies for Beam python unit testing

2023-04-12 Thread Valentyn Tymofieiev via dev
2. Make use of the current PreCommit and PostCommit test suite and modify it so that it installs pre-released dependencies. > Leads to noisy test signals if the pre-release candidate is unstable. I am favor of option 2 since it's a simple solution that is easy to implement and try out. The disadv

Re: [Python SDK] Use pre-released dependencies for Beam python unit testing

2023-04-12 Thread Danny McCormick via dev
Thanks for doing this Anand, I'm +1 on option 1 as well - I think having the clear signal of the normal suite succeeding and the prerelease one failing would be helpful and there shouldn't be too much additional code necessary. That makes it really easy to treat the prerelease suite as a (at least

Beam High Priority Issue Report (26)

2023-04-12 Thread beamactions
This is your daily summary of Beam's current high priority issues that may need attention. See https://beam.apache.org/contribute/issue-priorities for the meaning and expectations around issue priorities. Unassigned P1 Issues: https://github.com/apache/beam/issues/26126 [Failing Test]: be