Re: [VOTE] Release 2.50.0, release candidate #2
+1 Verified that the issue detected in RC0 has been resolved. Successfully ran a Python pipeline on ARM Dataflow workers. Noted that Dataflow runner logs became less verbose as the result of https://github.com/apache/beam/pull/27788. One line that I often pay attention to no longer appears at the default INFO log level: ``` INFO:apache_beam.runners.dataflow.dataflow_runner:2023-08-26T03:45:35.126Z: JOB_MESSAGE_DETAILED: All workers have finished the startup processes and began to receive work requests. ``` Dataflow service can be adjusted to compensate for this (internal change: http://cl/560265419 ). On Fri, Aug 25, 2023 at 3:05 PM Bruno Volpato via dev wrote: > +1 (non-binding). > > Tested with https://github.com/GoogleCloudPlatform/DataflowTemplates > (Java SDK 11, Dataflow runner). > > Thanks Robert! > > On Thu, Aug 24, 2023 at 7:12 PM Robert Burke wrote: > >> Two minor erata from the previous email: >> >> The validation spreadsheet link should be: >> >> https://docs.google.com/spreadsheets/d/1qk-N5vjXvbcEk68GjbkSZTR8AGqyNUM-oLFo_ZXBpJw/edit#gid=1014811464 >> >> And the source code tag is: "v2.50.0-RC2" >> >> On 2023/08/24 23:09:23 Robert Burke wrote: >> > Hi everyone, >> > Please review and vote on the release candidate #2 for the version >> 2.50.0, >> > as follows: >> > [ ] +1, Approve the release >> > [ ] -1, Do not approve the release (please provide specific comments) >> > >> > >> > Reviewers are encouraged to test their own use cases with the release >> > candidate, and vote +1 if >> > no issues are found. Only PMC member votes will count towards the final >> > vote, but votes from all >> > community members is encouraged and helpful for finding regressions; you >> > can either test your own >> > use cases or use cases from the validation sheet [10]. >> > >> > Issues noted in RC1 vote proposal [13] have now been resolved. >> > >> > The staging area is available for your review, which includes: >> > * GitHub Release notes [1], >> > * the official Apache source release to be deployed to dist.apache.org >> [2], >> > which is signed with the key with fingerprint 02677FF4371A3756 ( >> > lostl...@apache.org) or D20316F712213422 >> > (GitHub Action automated) [[3], >> > * all artifacts to be deployed to the Maven Central Repository [4], >> > * source code tag "v2.50.0-RC2" [5], >> > * website pull request listing the release [6], the blog post [6], and >> > publishing the API reference manual [7]. >> > * Java artifacts were built with Gradle 7.5.1 and OpenJDK >> (Temurin)(build >> > 1.8.0_382-b05). >> > * Python artifacts are deployed along with the source release to the >> > dist.apache.org [2] and PyPI[8]. >> > * Go artifacts and documentation are available at pkg.go.dev [9] >> > * Validation sheet with a tab for 2.50.0 release to help with validation >> > [10]. >> > * Docker images published to Docker Hub [11]. >> > * PR to run tests against release branch [12]. >> > >> > The vote will be open for at least 72 hours. It is adopted by majority >> > approval, with at least 3 PMC affirmative votes. >> > >> > For guidelines on how to try the release in your projects, check out our >> > blog post at https://beam.apache.org/blog/validate-beam-release/. >> > >> > Thanks, >> > Robert Burke >> > Apache Beam 2.50.0 Release Manager >> > >> > [1] https://github.com/apache/beam/milestone/14 >> > [2] https://dist.apache.org/repos/dist/dev/beam/2.50.0/ >> > [3] https://dist.apache.org/repos/dist/release/beam/KEYS >> > [4] >> https://repository.apache.org/content/repositories/orgapachebeam-1355/ >> > [5] https://github.com/apache/beam/tree/v2.50.0-RC2 >> > [6] https://github.com/apache/beam/pull/28055 >> > [7] https://github.com/apache/beam-site/pull/648 >> > [8] https://pypi.org/project/apache-beam/2.50.0rc2/ >> > [9] >> > >> https://pkg.go.dev/github.com/apache/beam/sdks/v2@v2.50.0-RC2/go/pkg/beam >> > [10] >> > >> https://docs.google.com/spreadsheets/d/1qk-N5vjXvbcEk68GjbkSZTR8AGqyNUM-oLFo_ZXBpJw/edit#gid=1014811464 >> > [11] https://hub.docker.com/search?q=apache%2Fbeam=image >> > [12] https://github.com/apache/beam/pull/27962 >> > [13] https://lists.apache.org/thread/xgx49zshms7253lfx6d6lsnvwf7tyyfp >> > >> >
Re: [VOTE] Release 2.50.0, release candidate #2
+1 (non-binding). Tested with https://github.com/GoogleCloudPlatform/DataflowTemplates (Java SDK 11, Dataflow runner). Thanks Robert! On Thu, Aug 24, 2023 at 7:12 PM Robert Burke wrote: > Two minor erata from the previous email: > > The validation spreadsheet link should be: > > https://docs.google.com/spreadsheets/d/1qk-N5vjXvbcEk68GjbkSZTR8AGqyNUM-oLFo_ZXBpJw/edit#gid=1014811464 > > And the source code tag is: "v2.50.0-RC2" > > On 2023/08/24 23:09:23 Robert Burke wrote: > > Hi everyone, > > Please review and vote on the release candidate #2 for the version > 2.50.0, > > as follows: > > [ ] +1, Approve the release > > [ ] -1, Do not approve the release (please provide specific comments) > > > > > > Reviewers are encouraged to test their own use cases with the release > > candidate, and vote +1 if > > no issues are found. Only PMC member votes will count towards the final > > vote, but votes from all > > community members is encouraged and helpful for finding regressions; you > > can either test your own > > use cases or use cases from the validation sheet [10]. > > > > Issues noted in RC1 vote proposal [13] have now been resolved. > > > > The staging area is available for your review, which includes: > > * GitHub Release notes [1], > > * the official Apache source release to be deployed to dist.apache.org > [2], > > which is signed with the key with fingerprint 02677FF4371A3756 ( > > lostl...@apache.org) or D20316F712213422 > > (GitHub Action automated) [[3], > > * all artifacts to be deployed to the Maven Central Repository [4], > > * source code tag "v2.50.0-RC2" [5], > > * website pull request listing the release [6], the blog post [6], and > > publishing the API reference manual [7]. > > * Java artifacts were built with Gradle 7.5.1 and OpenJDK (Temurin)(build > > 1.8.0_382-b05). > > * Python artifacts are deployed along with the source release to the > > dist.apache.org [2] and PyPI[8]. > > * Go artifacts and documentation are available at pkg.go.dev [9] > > * Validation sheet with a tab for 2.50.0 release to help with validation > > [10]. > > * Docker images published to Docker Hub [11]. > > * PR to run tests against release branch [12]. > > > > The vote will be open for at least 72 hours. It is adopted by majority > > approval, with at least 3 PMC affirmative votes. > > > > For guidelines on how to try the release in your projects, check out our > > blog post at https://beam.apache.org/blog/validate-beam-release/. > > > > Thanks, > > Robert Burke > > Apache Beam 2.50.0 Release Manager > > > > [1] https://github.com/apache/beam/milestone/14 > > [2] https://dist.apache.org/repos/dist/dev/beam/2.50.0/ > > [3] https://dist.apache.org/repos/dist/release/beam/KEYS > > [4] > https://repository.apache.org/content/repositories/orgapachebeam-1355/ > > [5] https://github.com/apache/beam/tree/v2.50.0-RC2 > > [6] https://github.com/apache/beam/pull/28055 > > [7] https://github.com/apache/beam-site/pull/648 > > [8] https://pypi.org/project/apache-beam/2.50.0rc2/ > > [9] > > > https://pkg.go.dev/github.com/apache/beam/sdks/v2@v2.50.0-RC2/go/pkg/beam > > [10] > > > https://docs.google.com/spreadsheets/d/1qk-N5vjXvbcEk68GjbkSZTR8AGqyNUM-oLFo_ZXBpJw/edit#gid=1014811464 > > [11] https://hub.docker.com/search?q=apache%2Fbeam=image > > [12] https://github.com/apache/beam/pull/27962 > > [13] https://lists.apache.org/thread/xgx49zshms7253lfx6d6lsnvwf7tyyfp > > >
Re: [Request for Feedback] Mechanism for "Beam Website Feedback"
Thank you XQ! For reference, responses are going to this spreadsheet [1] for now. We can set up an email trigger for that if people here think it would be useful. [1] https://docs.google.com/spreadsheets/d/1bvPFLgMamtYXiUULjx3JY-b1SB-voUqaj6o1UMqTgX4/edit?usp=sharing -- not shared by default, you need to ask for access. On Thu, Aug 24, 2023 at 6:38 AM XQ Hu via dev wrote: > Hi all, > > Following what Ahmet proposed here ( > https://lists.apache.org/thread/9l18gs7vgqz6g1cfc2m66rbkrcnr9qmj), we > changed the "SEED FEEDBACK" button to open the Google form ( > https://docs.google.com/forms/d/e/1FAIpQLSfID7abne3GE6k6RdJIyZhPz2Gef7UkpggUEhTIDjjplHuxSA/viewform?usp=sharing=0-Qq8aUWTrw1trYbivzEKZ0Q). > We would like to get any feedback to improve this form. > > Thanks a lot! > > Best, > XQ >
Re: [ANNOUNCE] New committer: Ahmed Abualsaud
Congrats Ahmed! On Fri, Aug 25, 2023 at 10:43 AM Bjorn Pedersen via dev wrote: > Congrats Ahmed! Well deserved! > > On Fri, Aug 25, 2023 at 10:36 AM Yi Hu via dev > wrote: > >> Congrats Ahmed! >> >> On Fri, Aug 25, 2023 at 10:11 AM Ritesh Ghorse via dev < >> dev@beam.apache.org> wrote: >> >>> Congrats Ahmed! >>> >>> On Fri, Aug 25, 2023 at 9:53 AM Kerry Donny-Clark via dev < >>> dev@beam.apache.org> wrote: >>> Well done Ahmed! On Fri, Aug 25, 2023 at 9:17 AM Danny McCormick via dev < dev@beam.apache.org> wrote: > Congrats Ahmed! > > On Fri, Aug 25, 2023 at 3:16 AM Jan Lukavský wrote: > >> Congrats Ahmed! >> On 8/25/23 07:56, Anand Inguva via dev wrote: >> >> Congratulations Ahmed :) >> >> On Fri, Aug 25, 2023 at 1:17 AM Damon Douglas < >> damondoug...@apache.org> wrote: >> >>> Well deserved! Congratulations, Ahmed! I'm so happy for you. >>> >>> On Thu, Aug 24, 2023, 5:46 PM Byron Ellis via dev < >>> dev@beam.apache.org> wrote: >>> Congratulations! On Thu, Aug 24, 2023 at 5:34 PM Robert Burke wrote: > Congratulations Ahmed!! > > On Thu, Aug 24, 2023, 4:08 PM Chamikara Jayalath via dev < > dev@beam.apache.org> wrote: > >> Congrats Ahmed!! >> >> On Thu, Aug 24, 2023 at 4:06 PM Bruno Volpato via dev < >> dev@beam.apache.org> wrote: >> >>> Congratulations, Ahmed! >>> >>> Very well deserved! >>> >>> >>> On Thu, Aug 24, 2023 at 6:09 PM XQ Hu via dev < >>> dev@beam.apache.org> wrote: >>> Congratulations, Ahmed! On Thu, Aug 24, 2023, 5:49 PM Ahmet Altay via dev < dev@beam.apache.org> wrote: > Hi all, > > Please join me and the rest of the Beam PMC in welcoming a new > committer: Ahmed Abualsaud (ahmedabuals...@apache.org). > > Ahmed has been part of the Beam community since January 2022, > working mostly on IO connectors, made a large amount of > contributions to > make Beam IOs more usable, performant, and reliable. And at the > same time > Ahmed was active in the user list and at the Beam summit helping > users by > sharing his knowledge. > > Considering their contributions to the project over this > timeframe, the Beam PMC trusts Ahmed with the responsibilities of > a Beam > committer. [1] > > Thank you Ahmed! And we are looking to see more of your > contributions! > > Ahmet, on behalf of the Apache Beam PMC > > [1] > > https://beam.apache.org/contribute/become-a-committer/#an-apache-beam-committer > >
Re: [Request for Feedback] Swift SDK Prototype
Okay, after a brief detour through "get this working in the Flink Portable Runner" I think I have something pretty workable. PInput and POutput can actually be structs rather than protocols, which simplifies things quite a bit. It also allows us to use them with property wrappers for a SwiftUI-like experience if we want when defining DoFns (which is what I was originally intending to use them for). That also means the function signature you use for closures would match full-fledged DoFn definitions for the most part which is satisfying. On Thu, Aug 24, 2023 at 5:55 PM Byron Ellis wrote: > Okay, I tried a couple of different things. > > Implicitly passing the timestamp and window during iteration did not go > well. While physically possible it introduces an invisible side effect into > loop iteration which confused me when I tried to use it and I implemented > it. Also, I'm pretty sure there'd end up being some sort of race condition > nightmare continuing down that path. > > What I decided to do instead was the following: > > 1. Rename the existing "pardo" functions to "pstream" and require that > they always emit a window and timestamp along with their value. This > eliminates the side effect but lets us keep iteration in a bundle where > that might be convenient. For example, in my cheesy GCS implementation it > means that I can keep an OAuth token around for the lifetime of the bundle > as a local variable, which is convenient. It's a bit more typing for users > of pstream, but the expectation here is that if you're using pstream > functions You Know What You Are Doing and most people won't be using it > directly. > > 2. Introduce a new set of pardo functions (I didn't do all of them yet, > but enough to test the functionality and decide I liked it) which take a > function signature of (any PInput,any POutput). > PInput takes the (InputType,Date,Window) tuple and converts it into a > struct with friendlier names. Not strictly necessary, but makes the code > nicer to read I think. POutput introduces emit functions that optionally > allow you to specify a timestamp and a window. If you don't for either one > it will take the timestamp and/or window of the input. > > Trying to use that was pretty pleasant to use so I think we should > continue down that path. If you'd like to see it in use, I reimplemented > map() and flatMap() in terms of this new pardo functionality. > > Code has been pushed to the branch/PR if you're interested in taking a > look. > > > > > > > > > > On Thu, Aug 24, 2023 at 2:15 PM Byron Ellis wrote: > >> Gotcha, I think there's a fairly easy solution to link input and output >> streams Let me try it out... might even be possible to have both >> element and stream-wise closure pardos. Definitely possible to have that at >> the DoFn level (called SerializableFn in the SDK because I want to >> use @DoFn as a macro) >> >> On Thu, Aug 24, 2023 at 1:09 PM Robert Bradshaw >> wrote: >> >>> On Thu, Aug 24, 2023 at 12:58 PM Chamikara Jayalath < >>> chamik...@google.com> wrote: >>> On Thu, Aug 24, 2023 at 12:27 PM Robert Bradshaw wrote: > I would like to figure out a way to get the stream-y interface to > work, as I think it's more natural overall. > > One hypothesis is that if any elements are carried over loop > iterations, there will likely be some that are carried over beyond the > loop > (after all the callee doesn't know when the loop is supposed to end). We > could reject "plain" elements that are emitted after this point, requiring > one to emit timestamp-windowed-values. > Are you assuming that the same stream (or overlapping sets of data) are pushed to multiple workers ? I thought that the set of data streamed here are the data that belong to the current bundle (hence already assigned to the current worker) so any output from the current bundle invocation would be a valid output of that bundle. > >>> Yes, the content of the stream is exactly the contents of the bundle. >>> The question is how to do the input_element:output_element correlation for >>> automatically propagating metadata. >>> >>> Related to this, we could enforce that the only (user-accessible) way > to get such a timestamped value is to start with one, e.g. a > WindowedValue.withValue(O) produces a WindowedValue with the same > metadata but a new value. Thus a user wanting to do anything "fancy" would > have to explicitly request iteration over these windowed values rather > than > over the raw elements. (This is also forward compatible with expanding the > metadata that can get attached, e.g. pane infos, and makes the right thing > the easiest/most natural.) > > On Thu, Aug 24, 2023 at 12:10 PM Byron Ellis > wrote: > >> Ah, that is a good point—being element-wise would make managing >> windows and time stamps easier for the user. Fortunately
Re: [ANNOUNCE] New committer: Ahmed Abualsaud
Congrats Ahmed! Well deserved! On Fri, Aug 25, 2023 at 10:36 AM Yi Hu via dev wrote: > Congrats Ahmed! > > On Fri, Aug 25, 2023 at 10:11 AM Ritesh Ghorse via dev < > dev@beam.apache.org> wrote: > >> Congrats Ahmed! >> >> On Fri, Aug 25, 2023 at 9:53 AM Kerry Donny-Clark via dev < >> dev@beam.apache.org> wrote: >> >>> Well done Ahmed! >>> >>> On Fri, Aug 25, 2023 at 9:17 AM Danny McCormick via dev < >>> dev@beam.apache.org> wrote: >>> Congrats Ahmed! On Fri, Aug 25, 2023 at 3:16 AM Jan Lukavský wrote: > Congrats Ahmed! > On 8/25/23 07:56, Anand Inguva via dev wrote: > > Congratulations Ahmed :) > > On Fri, Aug 25, 2023 at 1:17 AM Damon Douglas > wrote: > >> Well deserved! Congratulations, Ahmed! I'm so happy for you. >> >> On Thu, Aug 24, 2023, 5:46 PM Byron Ellis via dev < >> dev@beam.apache.org> wrote: >> >>> Congratulations! >>> >>> On Thu, Aug 24, 2023 at 5:34 PM Robert Burke >>> wrote: >>> Congratulations Ahmed!! On Thu, Aug 24, 2023, 4:08 PM Chamikara Jayalath via dev < dev@beam.apache.org> wrote: > Congrats Ahmed!! > > On Thu, Aug 24, 2023 at 4:06 PM Bruno Volpato via dev < > dev@beam.apache.org> wrote: > >> Congratulations, Ahmed! >> >> Very well deserved! >> >> >> On Thu, Aug 24, 2023 at 6:09 PM XQ Hu via dev < >> dev@beam.apache.org> wrote: >> >>> Congratulations, Ahmed! >>> >>> On Thu, Aug 24, 2023, 5:49 PM Ahmet Altay via dev < >>> dev@beam.apache.org> wrote: >>> Hi all, Please join me and the rest of the Beam PMC in welcoming a new committer: Ahmed Abualsaud (ahmedabuals...@apache.org). Ahmed has been part of the Beam community since January 2022, working mostly on IO connectors, made a large amount of contributions to make Beam IOs more usable, performant, and reliable. And at the same time Ahmed was active in the user list and at the Beam summit helping users by sharing his knowledge. Considering their contributions to the project over this timeframe, the Beam PMC trusts Ahmed with the responsibilities of a Beam committer. [1] Thank you Ahmed! And we are looking to see more of your contributions! Ahmet, on behalf of the Apache Beam PMC [1] https://beam.apache.org/contribute/become-a-committer/#an-apache-beam-committer
Re: [ANNOUNCE] New committer: Ahmed Abualsaud
Congrats Ahmed! On Fri, Aug 25, 2023 at 10:11 AM Ritesh Ghorse via dev wrote: > Congrats Ahmed! > > On Fri, Aug 25, 2023 at 9:53 AM Kerry Donny-Clark via dev < > dev@beam.apache.org> wrote: > >> Well done Ahmed! >> >> On Fri, Aug 25, 2023 at 9:17 AM Danny McCormick via dev < >> dev@beam.apache.org> wrote: >> >>> Congrats Ahmed! >>> >>> On Fri, Aug 25, 2023 at 3:16 AM Jan Lukavský wrote: >>> Congrats Ahmed! On 8/25/23 07:56, Anand Inguva via dev wrote: Congratulations Ahmed :) On Fri, Aug 25, 2023 at 1:17 AM Damon Douglas wrote: > Well deserved! Congratulations, Ahmed! I'm so happy for you. > > On Thu, Aug 24, 2023, 5:46 PM Byron Ellis via dev > wrote: > >> Congratulations! >> >> On Thu, Aug 24, 2023 at 5:34 PM Robert Burke >> wrote: >> >>> Congratulations Ahmed!! >>> >>> On Thu, Aug 24, 2023, 4:08 PM Chamikara Jayalath via dev < >>> dev@beam.apache.org> wrote: >>> Congrats Ahmed!! On Thu, Aug 24, 2023 at 4:06 PM Bruno Volpato via dev < dev@beam.apache.org> wrote: > Congratulations, Ahmed! > > Very well deserved! > > > On Thu, Aug 24, 2023 at 6:09 PM XQ Hu via dev > wrote: > >> Congratulations, Ahmed! >> >> On Thu, Aug 24, 2023, 5:49 PM Ahmet Altay via dev < >> dev@beam.apache.org> wrote: >> >>> Hi all, >>> >>> Please join me and the rest of the Beam PMC in welcoming a new >>> committer: Ahmed Abualsaud (ahmedabuals...@apache.org). >>> >>> Ahmed has been part of the Beam community since January 2022, >>> working mostly on IO connectors, made a large amount of >>> contributions to >>> make Beam IOs more usable, performant, and reliable. And at the >>> same time >>> Ahmed was active in the user list and at the Beam summit helping >>> users by >>> sharing his knowledge. >>> >>> Considering their contributions to the project over this >>> timeframe, the Beam PMC trusts Ahmed with the responsibilities of a >>> Beam >>> committer. [1] >>> >>> Thank you Ahmed! And we are looking to see more of your >>> contributions! >>> >>> Ahmet, on behalf of the Apache Beam PMC >>> >>> [1] >>> >>> https://beam.apache.org/contribute/become-a-committer/#an-apache-beam-committer >>> >>>
Re: [ANNOUNCE] New committer: Ahmed Abualsaud
Congrats Ahmed! On Fri, Aug 25, 2023 at 9:53 AM Kerry Donny-Clark via dev < dev@beam.apache.org> wrote: > Well done Ahmed! > > On Fri, Aug 25, 2023 at 9:17 AM Danny McCormick via dev < > dev@beam.apache.org> wrote: > >> Congrats Ahmed! >> >> On Fri, Aug 25, 2023 at 3:16 AM Jan Lukavský wrote: >> >>> Congrats Ahmed! >>> On 8/25/23 07:56, Anand Inguva via dev wrote: >>> >>> Congratulations Ahmed :) >>> >>> On Fri, Aug 25, 2023 at 1:17 AM Damon Douglas >>> wrote: >>> Well deserved! Congratulations, Ahmed! I'm so happy for you. On Thu, Aug 24, 2023, 5:46 PM Byron Ellis via dev wrote: > Congratulations! > > On Thu, Aug 24, 2023 at 5:34 PM Robert Burke > wrote: > >> Congratulations Ahmed!! >> >> On Thu, Aug 24, 2023, 4:08 PM Chamikara Jayalath via dev < >> dev@beam.apache.org> wrote: >> >>> Congrats Ahmed!! >>> >>> On Thu, Aug 24, 2023 at 4:06 PM Bruno Volpato via dev < >>> dev@beam.apache.org> wrote: >>> Congratulations, Ahmed! Very well deserved! On Thu, Aug 24, 2023 at 6:09 PM XQ Hu via dev wrote: > Congratulations, Ahmed! > > On Thu, Aug 24, 2023, 5:49 PM Ahmet Altay via dev < > dev@beam.apache.org> wrote: > >> Hi all, >> >> Please join me and the rest of the Beam PMC in welcoming a new >> committer: Ahmed Abualsaud (ahmedabuals...@apache.org). >> >> Ahmed has been part of the Beam community since January 2022, >> working mostly on IO connectors, made a large amount of >> contributions to >> make Beam IOs more usable, performant, and reliable. And at the same >> time >> Ahmed was active in the user list and at the Beam summit helping >> users by >> sharing his knowledge. >> >> Considering their contributions to the project over this >> timeframe, the Beam PMC trusts Ahmed with the responsibilities of a >> Beam >> committer. [1] >> >> Thank you Ahmed! And we are looking to see more of your >> contributions! >> >> Ahmet, on behalf of the Apache Beam PMC >> >> [1] >> >> https://beam.apache.org/contribute/become-a-committer/#an-apache-beam-committer >> >>
Re: [ANNOUNCE] New committer: Ahmed Abualsaud
Well done Ahmed! On Fri, Aug 25, 2023 at 9:17 AM Danny McCormick via dev wrote: > Congrats Ahmed! > > On Fri, Aug 25, 2023 at 3:16 AM Jan Lukavský wrote: > >> Congrats Ahmed! >> On 8/25/23 07:56, Anand Inguva via dev wrote: >> >> Congratulations Ahmed :) >> >> On Fri, Aug 25, 2023 at 1:17 AM Damon Douglas >> wrote: >> >>> Well deserved! Congratulations, Ahmed! I'm so happy for you. >>> >>> On Thu, Aug 24, 2023, 5:46 PM Byron Ellis via dev >>> wrote: >>> Congratulations! On Thu, Aug 24, 2023 at 5:34 PM Robert Burke wrote: > Congratulations Ahmed!! > > On Thu, Aug 24, 2023, 4:08 PM Chamikara Jayalath via dev < > dev@beam.apache.org> wrote: > >> Congrats Ahmed!! >> >> On Thu, Aug 24, 2023 at 4:06 PM Bruno Volpato via dev < >> dev@beam.apache.org> wrote: >> >>> Congratulations, Ahmed! >>> >>> Very well deserved! >>> >>> >>> On Thu, Aug 24, 2023 at 6:09 PM XQ Hu via dev >>> wrote: >>> Congratulations, Ahmed! On Thu, Aug 24, 2023, 5:49 PM Ahmet Altay via dev < dev@beam.apache.org> wrote: > Hi all, > > Please join me and the rest of the Beam PMC in welcoming a new > committer: Ahmed Abualsaud (ahmedabuals...@apache.org). > > Ahmed has been part of the Beam community since January 2022, > working mostly on IO connectors, made a large amount of contributions > to > make Beam IOs more usable, performant, and reliable. And at the same > time > Ahmed was active in the user list and at the Beam summit helping > users by > sharing his knowledge. > > Considering their contributions to the project over this > timeframe, the Beam PMC trusts Ahmed with the responsibilities of a > Beam > committer. [1] > > Thank you Ahmed! And we are looking to see more of your > contributions! > > Ahmet, on behalf of the Apache Beam PMC > > [1] > > https://beam.apache.org/contribute/become-a-committer/#an-apache-beam-committer > >
Re: [ANNOUNCE] New committer: Ahmed Abualsaud
Congrats Ahmed! On Fri, Aug 25, 2023 at 3:16 AM Jan Lukavský wrote: > Congrats Ahmed! > On 8/25/23 07:56, Anand Inguva via dev wrote: > > Congratulations Ahmed :) > > On Fri, Aug 25, 2023 at 1:17 AM Damon Douglas > wrote: > >> Well deserved! Congratulations, Ahmed! I'm so happy for you. >> >> On Thu, Aug 24, 2023, 5:46 PM Byron Ellis via dev >> wrote: >> >>> Congratulations! >>> >>> On Thu, Aug 24, 2023 at 5:34 PM Robert Burke wrote: >>> Congratulations Ahmed!! On Thu, Aug 24, 2023, 4:08 PM Chamikara Jayalath via dev < dev@beam.apache.org> wrote: > Congrats Ahmed!! > > On Thu, Aug 24, 2023 at 4:06 PM Bruno Volpato via dev < > dev@beam.apache.org> wrote: > >> Congratulations, Ahmed! >> >> Very well deserved! >> >> >> On Thu, Aug 24, 2023 at 6:09 PM XQ Hu via dev >> wrote: >> >>> Congratulations, Ahmed! >>> >>> On Thu, Aug 24, 2023, 5:49 PM Ahmet Altay via dev < >>> dev@beam.apache.org> wrote: >>> Hi all, Please join me and the rest of the Beam PMC in welcoming a new committer: Ahmed Abualsaud (ahmedabuals...@apache.org). Ahmed has been part of the Beam community since January 2022, working mostly on IO connectors, made a large amount of contributions to make Beam IOs more usable, performant, and reliable. And at the same time Ahmed was active in the user list and at the Beam summit helping users by sharing his knowledge. Considering their contributions to the project over this timeframe, the Beam PMC trusts Ahmed with the responsibilities of a Beam committer. [1] Thank you Ahmed! And we are looking to see more of your contributions! Ahmet, on behalf of the Apache Beam PMC [1] https://beam.apache.org/contribute/become-a-committer/#an-apache-beam-committer
Beam High Priority Issue Report (40)
This is your daily summary of Beam's current high priority issues that may need attention. See https://beam.apache.org/contribute/issue-priorities for the meaning and expectations around issue priorities. Unassigned P1 Issues: https://github.com/apache/beam/issues/28104 [Failing Test]: gradlew :sdks:python:test-suites:tox:pycommon:docs is failing https://github.com/apache/beam/issues/27892 [Bug]: ignoreUnknownValues not working when using CreateDisposition.CREATE_IF_NEEDED https://github.com/apache/beam/issues/27648 [Bug]: Python SDFs (e.g. PeriodicImpulse) running in Flink and polling using tracker.defer_remainder have checkpoint size growing indefinitely https://github.com/apache/beam/issues/27616 [Bug]: Unable to use applyRowMutations() in bigquery IO apache beam java https://github.com/apache/beam/issues/27486 [Bug]: Read from datastore with inequality filters https://github.com/apache/beam/issues/27314 [Failing Test]: bigquery.StorageApiSinkCreateIfNeededIT.testCreateManyTables[1] https://github.com/apache/beam/issues/27238 [Bug]: Window trigger has lag when using Kafka and GroupByKey on Dataflow Runner https://github.com/apache/beam/issues/26981 [Bug]: Getting an error related to SchemaCoder after upgrading to 2.48 https://github.com/apache/beam/issues/26969 [Failing Test]: Python PostCommit is failing due to exceeded rate limits https://github.com/apache/beam/issues/26911 [Bug]: UNNEST ARRAY with a nested ROW (described below) https://github.com/apache/beam/issues/26354 [Bug]: BigQueryIO direct read not reading all rows when set --setEnableBundling=true https://github.com/apache/beam/issues/26343 [Bug]: apache_beam.io.gcp.bigquery_read_it_test.ReadAllBQTests.test_read_queries is flaky https://github.com/apache/beam/issues/26329 [Bug]: BigQuerySourceBase does not propagate a Coder to AvroSource https://github.com/apache/beam/issues/26041 [Bug]: Unable to create exactly-once Flink pipeline with stream source and file sink https://github.com/apache/beam/issues/25975 [Bug]: Reducing parallelism in FlinkRunner leads to a data loss https://github.com/apache/beam/issues/24776 [Bug]: Race condition in Python SDK Harness ProcessBundleProgress https://github.com/apache/beam/issues/24389 [Failing Test]: HadoopFormatIOElasticTest.classMethod ExceptionInInitializerError ContainerFetchException https://github.com/apache/beam/issues/24313 [Flaky]: apache_beam/runners/portability/portable_runner_test.py::PortableRunnerTestWithSubprocesses::test_pardo_state_with_custom_key_coder https://github.com/apache/beam/issues/23944 beam_PreCommit_Python_Cron regularily failing - test_pardo_large_input flaky https://github.com/apache/beam/issues/23709 [Flake]: Spark batch flakes in ParDoLifecycleTest.testTeardownCalledAfterExceptionInProcessElement and ParDoLifecycleTest.testTeardownCalledAfterExceptionInStartBundle https://github.com/apache/beam/issues/23525 [Bug]: Default PubsubMessage coder will drop message id and orderingKey https://github.com/apache/beam/issues/22913 [Bug]: beam_PostCommit_Java_ValidatesRunner_Flink is flakes in org.apache.beam.sdk.transforms.GroupByKeyTest$BasicTests.testAfterProcessingTimeContinuationTriggerUsingState https://github.com/apache/beam/issues/22605 [Bug]: Beam Python failure for dataflow_exercise_metrics_pipeline_test.ExerciseMetricsPipelineTest.test_metrics_it https://github.com/apache/beam/issues/21714 PulsarIOTest.testReadFromSimpleTopic is very flaky https://github.com/apache/beam/issues/21708 beam_PostCommit_Java_DataflowV2, testBigQueryStorageWrite30MProto failing consistently https://github.com/apache/beam/issues/21706 Flaky timeout in github Python unit test action StatefulDoFnOnDirectRunnerTest.test_dynamic_timer_clear_then_set_timer https://github.com/apache/beam/issues/21643 FnRunnerTest with non-trivial (order 1000 elements) numpy input flakes in non-cython environment https://github.com/apache/beam/issues/21476 WriteToBigQuery Dynamic table destinations returns wrong tableId https://github.com/apache/beam/issues/21469 beam_PostCommit_XVR_Flink flaky: Connection refused https://github.com/apache/beam/issues/21424 Java VR (Dataflow, V2, Streaming) failing: ParDoTest$TimestampTests/OnWindowExpirationTests https://github.com/apache/beam/issues/21262 Python AfterAny, AfterAll do not follow spec https://github.com/apache/beam/issues/21260 Python DirectRunner does not emit data at GC time https://github.com/apache/beam/issues/21121 apache_beam.examples.streaming_wordcount_it_test.StreamingWordCountIT.test_streaming_wordcount_it flakey https://github.com/apache/beam/issues/21104 Flaky: apache_beam.runners.portability.fn_api_runner.fn_runner_test.FnApiRunnerTestWithGrpcAndMultiWorkers https://github.com/apache/beam/issues/20976 apache_beam.runners.portability.flink_runner_test.FlinkRunnerTestOptimized.test_flink_metrics is flaky https://github.com/apache/beam/issues/20108 Python direct runner doesn't emit empty pane when it
Re: [ANNOUNCE] New committer: Ahmed Abualsaud
Congrats Ahmed! On 8/25/23 07:56, Anand Inguva via dev wrote: Congratulations Ahmed :) On Fri, Aug 25, 2023 at 1:17 AM Damon Douglas wrote: Well deserved! Congratulations, Ahmed! I'm so happy for you. On Thu, Aug 24, 2023, 5:46 PM Byron Ellis via dev wrote: Congratulations! On Thu, Aug 24, 2023 at 5:34 PM Robert Burke wrote: Congratulations Ahmed!! On Thu, Aug 24, 2023, 4:08 PM Chamikara Jayalath via dev wrote: Congrats Ahmed!! On Thu, Aug 24, 2023 at 4:06 PM Bruno Volpato via dev wrote: Congratulations, Ahmed! Very well deserved! On Thu, Aug 24, 2023 at 6:09 PM XQ Hu via dev wrote: Congratulations, Ahmed! On Thu, Aug 24, 2023, 5:49 PM Ahmet Altay via dev wrote: Hi all, Please join me and the rest of the Beam PMC in welcoming a new committer: Ahmed Abualsaud (ahmedabuals...@apache.org). Ahmed has been part of the Beam community since January 2022, working mostly on IO connectors, made a large amount of contributions to make Beam IOs more usable, performant, and reliable. And at the same time Ahmed was active in the user list and at the Beam summit helping users by sharing his knowledge. Considering their contributions to the project over this timeframe, the Beam PMC trusts Ahmed with the responsibilities of a Beam committer. [1] Thank you Ahmed! And we are looking to see more of your contributions! Ahmet, on behalf of the Apache Beam PMC [1] https://beam.apache.org/contribute/become-a-committer/#an-apache-beam-committer
