Re: DRAFT - Apache Beam Board Report - June 2023

[Jean-Baptiste Onofré]

It looks good to me, thanks !

Regards
JB

On Fri, Jun 9, 2023 at 11:28 PM Kenneth Knowles  wrote:
>
> Hi all,
>
> The next Beam board report is due next Wednesday, June 14. Please help me to 
> draft it at https://s.apache.org/beam-draft-report-2023-06.
>
> Ideas:
>
>  - highlights from CHANGES.md
>  - interesting technical discussions
>  - integrations with other projects
>  - community events
>  - major user facing addition/deprecation
>  - stuff that will be presented at Beam Summit next week :-)
>
> Past reports are at https://whimsy.apache.org/board/minutes/Beam.html for 
> examples.
>
> I will edit the final version from everyone's suggestions.
>
> Thanks,
>
> Kenn
>
>

Re: DRAFT - Apache Beam Board Report - June 2023

[Ahmet Altay via dev]

Thank you Kenn! Added a few items.

On Fri, Jun 9, 2023 at 2:28 PM Kenneth Knowles  wrote:

> Hi all,
>
> The next Beam board report is due next Wednesday, June 14. Please help me
> to draft it at https://s.apache.org/beam-draft-report-2023-06.
>
> Ideas:
>
>  - highlights from CHANGES.md
>  - interesting technical discussions
>  - integrations with other projects
>  - community events
>  - major user facing addition/deprecation
>  - stuff that will be presented at Beam Summit next week :-)
>
> Past reports are at https://whimsy.apache.org/board/minutes/Beam.html for
> examples.
>
> I will edit the final version from everyone's suggestions.
>
> Thanks,
>
> Kenn
>
>
>

DRAFT - Apache Beam Board Report - June 2023

[Kenneth Knowles]

Hi all,

The next Beam board report is due next Wednesday, June 14. Please help me
to draft it at https://s.apache.org/beam-draft-report-2023-06.

Ideas:

 - highlights from CHANGES.md
 - interesting technical discussions
 - integrations with other projects
 - community events
 - major user facing addition/deprecation
 - stuff that will be presented at Beam Summit next week :-)

Past reports are at https://whimsy.apache.org/board/minutes/Beam.html for
examples.

I will edit the final version from everyone's suggestions.

Thanks,

Kenn

Re: [Launch Announcement] Beam Quest

[Ahmet Altay via dev]

(dropping the user list)

Please also help us with spreading the word by sharing on your social media:
- Twitter: https://twitter.com/ApacheBeam/status/1666915798823084034
- Linkedin post:
https://www.linkedin.com/feed/update/urn:li:activity:7072675442012377088

On Fri, Jun 9, 2023 at 10:11 AM Ahmet Altay  wrote:

> Thank you Svetak! I would encourage everyone to try out, and get the
> badges :)
>
> On Fri, Jun 9, 2023 at 7:03 AM Svetak Sundhar via user <
> u...@beam.apache.org> wrote:
>
>> Hi Beam Community,
>>
>> We're excited to launch the "Getting Started with Apache Beam" Quest
>> . This quest provides a
>> completion badge that can be shared on social media (such as Linkedin and
>> Twitter) upon completion of four qwiklabs.
>>
>> These labs venture into various concepts of Beam in the Java and Python
>> SDK (that many of you have developed), and should take less than 7 hours to
>> obtain. I've written about it in our Beam Blog
>> ; we are offering this free of
>> charge till July 8.
>>
>> Please share the information with whomever you think may be interested,
>> and please share on social media once you obtain your badge. Additionally,
>> if you have any feedback on the labs, please contact me directly at
>> svetaksund...@google.com-- we plan to have these labs evolve over time!
>>
>> I look forward to discussing this more at Beam Summit next week.
>>
>> As this was one of GCP's first OSS quests, there were many people
>> instrumental in making this possible.
>>
>> Thanks to:
>> -Danielle Syse
>> -Ajay Hemnani
>> -Joellen Saunders
>> -Grzegorz Wierzchows
>> -Ahmet Altay
>> -XQ Hu
>> -Jenny Palomino
>> -Svetak Sundhar
>> -Shunping Huang
>>
>> Thanks,
>>
>>
>>
>> Svetak Sundhar
>>
>>   Data Engineer
>> s vetaksund...@google.com
>>
>>

Re: [Launch Announcement] Beam Quest

[Ahmet Altay via dev]

Thank you Svetak! I would encourage everyone to try out, and get the badges
:)

On Fri, Jun 9, 2023 at 7:03 AM Svetak Sundhar via user 
wrote:

> Hi Beam Community,
>
> We're excited to launch the "Getting Started with Apache Beam" Quest
> . This quest provides a
> completion badge that can be shared on social media (such as Linkedin and
> Twitter) upon completion of four qwiklabs.
>
> These labs venture into various concepts of Beam in the Java and Python
> SDK (that many of you have developed), and should take less than 7 hours to
> obtain. I've written about it in our Beam Blog
> ; we are offering this free of
> charge till July 8.
>
> Please share the information with whomever you think may be interested,
> and please share on social media once you obtain your badge. Additionally,
> if you have any feedback on the labs, please contact me directly at
> svetaksund...@google.com-- we plan to have these labs evolve over time!
>
> I look forward to discussing this more at Beam Summit next week.
>
> As this was one of GCP's first OSS quests, there were many people
> instrumental in making this possible.
>
> Thanks to:
> -Danielle Syse
> -Ajay Hemnani
> -Joellen Saunders
> -Grzegorz Wierzchows
> -Ahmet Altay
> -XQ Hu
> -Jenny Palomino
> -Svetak Sundhar
> -Shunping Huang
>
> Thanks,
>
>
>
> Svetak Sundhar
>
>   Data Engineer
> s vetaksund...@google.com
>
>

[Launch Announcement] Beam Quest

[Svetak Sundhar via dev]

Hi Beam Community,

We're excited to launch the "Getting Started with Apache Beam" Quest
. This quest provides a
completion badge that can be shared on social media (such as Linkedin and
Twitter) upon completion of four qwiklabs.

These labs venture into various concepts of Beam in the Java and Python SDK
(that many of you have developed), and should take less than 7 hours to
obtain. I've written about it in our Beam Blog
; we are offering this free of
charge till July 8.

Please share the information with whomever you think may be interested, and
please share on social media once you obtain your badge. Additionally, if
you have any feedback on the labs, please contact me directly at
svetaksund...@google.com-- we plan to have these labs evolve over time!

I look forward to discussing this more at Beam Summit next week.

As this was one of GCP's first OSS quests, there were many people
instrumental in making this possible.

Thanks to:
-Danielle Syse
-Ajay Hemnani
-Joellen Saunders
-Grzegorz Wierzchows
-Ahmet Altay
-XQ Hu
-Jenny Palomino
-Svetak Sundhar
-Shunping Huang

Thanks,



Svetak Sundhar

  Data Engineer
s vetaksund...@google.com

Beam High Priority Issue Report (37)

[beamactions]

This is your daily summary of Beam's current high priority issues that may need 
attention.

See https://beam.apache.org/contribute/issue-priorities for the meaning and 
expectations around issue priorities.

Unassigned P1 Issues:

https://github.com/apache/beam/issues/27019 [Failing Test]: Azure Integration 
test is failing in python 3.7 PostCommit
https://github.com/apache/beam/issues/27012 [Bug]: Beam Website cannot run 
locally on Mac
https://github.com/apache/beam/issues/26981 [Bug]: Getting an error related to 
SchemaCoder after upgrading to 2.48
https://github.com/apache/beam/issues/26969 [Failing Test]: Python PostCommit 
is failing due to exceeded rate limits
https://github.com/apache/beam/issues/26911 [Bug]: UNNEST ARRAY with a nested 
ROW (described below)
https://github.com/apache/beam/issues/26547 [Failing Test]: 
beam_PostCommit_Java_DataflowV2
https://github.com/apache/beam/issues/26354 [Bug]: BigQueryIO direct read not 
reading all rows when set --setEnableBundling=true
https://github.com/apache/beam/issues/26343 [Bug]: 
apache_beam.io.gcp.bigquery_read_it_test.ReadAllBQTests.test_read_queries is 
flaky
https://github.com/apache/beam/issues/26329 [Bug]: BigQuerySourceBase does not 
propagate a Coder to AvroSource
https://github.com/apache/beam/issues/26272 [Failing Test]: Python 3.7 
postcommit is red
https://github.com/apache/beam/issues/26041 [Bug]: Unable to create 
exactly-once Flink pipeline with stream source and file sink
https://github.com/apache/beam/issues/25975 [Bug]: Reducing parallelism in 
FlinkRunner leads to a data loss
https://github.com/apache/beam/issues/24776 [Bug]: Race condition in Python SDK 
Harness ProcessBundleProgress
https://github.com/apache/beam/issues/24389 [Failing Test]: 
HadoopFormatIOElasticTest.classMethod ExceptionInInitializerError 
ContainerFetchException
https://github.com/apache/beam/issues/24313 [Flaky]: 
apache_beam/runners/portability/portable_runner_test.py::PortableRunnerTestWithSubprocesses::test_pardo_state_with_custom_key_coder
https://github.com/apache/beam/issues/23944  beam_PreCommit_Python_Cron 
regularily failing - test_pardo_large_input flaky
https://github.com/apache/beam/issues/23709 [Flake]: Spark batch flakes in 
ParDoLifecycleTest.testTeardownCalledAfterExceptionInProcessElement and 
ParDoLifecycleTest.testTeardownCalledAfterExceptionInStartBundle
https://github.com/apache/beam/issues/22913 [Bug]: 
beam_PostCommit_Java_ValidatesRunner_Flink is flakes in 
org.apache.beam.sdk.transforms.GroupByKeyTest$BasicTests.testAfterProcessingTimeContinuationTriggerUsingState
https://github.com/apache/beam/issues/22605 [Bug]: Beam Python failure for 
dataflow_exercise_metrics_pipeline_test.ExerciseMetricsPipelineTest.test_metrics_it
https://github.com/apache/beam/issues/21714 
PulsarIOTest.testReadFromSimpleTopic is very flaky
https://github.com/apache/beam/issues/21708 beam_PostCommit_Java_DataflowV2, 
testBigQueryStorageWrite30MProto failing consistently
https://github.com/apache/beam/issues/21706 Flaky timeout in github Python unit 
test action 
StatefulDoFnOnDirectRunnerTest.test_dynamic_timer_clear_then_set_timer
https://github.com/apache/beam/issues/21643 FnRunnerTest with non-trivial 
(order 1000 elements) numpy input flakes in non-cython environment
https://github.com/apache/beam/issues/21476 WriteToBigQuery Dynamic table 
destinations returns wrong tableId
https://github.com/apache/beam/issues/21469 beam_PostCommit_XVR_Flink flaky: 
Connection refused
https://github.com/apache/beam/issues/21424 Java VR (Dataflow, V2, Streaming) 
failing: ParDoTest$TimestampTests/OnWindowExpirationTests
https://github.com/apache/beam/issues/21262 Python AfterAny, AfterAll do not 
follow spec
https://github.com/apache/beam/issues/21260 Python DirectRunner does not emit 
data at GC time
https://github.com/apache/beam/issues/21121 
apache_beam.examples.streaming_wordcount_it_test.StreamingWordCountIT.test_streaming_wordcount_it
 flakey
https://github.com/apache/beam/issues/21104 Flaky: 
apache_beam.runners.portability.fn_api_runner.fn_runner_test.FnApiRunnerTestWithGrpcAndMultiWorkers
https://github.com/apache/beam/issues/20976 
apache_beam.runners.portability.flink_runner_test.FlinkRunnerTestOptimized.test_flink_metrics
 is flaky
https://github.com/apache/beam/issues/20108 Python direct runner doesn't emit 
empty pane when it should
https://github.com/apache/beam/issues/19814 Flink streaming flakes in 
ParDoLifecycleTest.testTeardownCalledAfterExceptionInStartBundleStateful and 
ParDoLifecycleTest.testTeardownCalledAfterExceptionInProcessElementStateful
https://github.com/apache/beam/issues/19465 Explore possibilities to lower 
in-use IP address quota footprint.


P1 Issues with no update in the last week:

https://github.com/apache/beam/issues/26723 [Failing Test]: Tour of Beam 
Frontend Test suite is perma-red on master
https://github.com/apache/beam/issues/23525 [Bug]: Default PubsubMessage coder 
will drop message id and orderingKey
https://github.