Re: [PROPOSAL] Preparing for 2.42.0 Release

[Robert Burke]

A release branch has been cut!

https://github.com/apache/beam/tree/release-2.42.0

However, I don't have time to start further verification procedures. The
plan is to get them going tomorrow morning PT, and proceed to a quick RC so
that community verification can begin promptly. It's not anticipated that
that will be the final RC.

Either an RC will be published tomorrow, with it's accompanying RC thread,
or this thread will be updated by EoD.

Thank you for your patience,
Your friendly neighbourhood release manager
Robert Burke

On Wed, Sep 7, 2022 at 10:55 AM Robert Burke  wrote:

> 'One issue remains open in the 2.42.0 milestone [1] blocking the cut.
>
> Since it's a BOM update I'm inclined to wait for it, rather than cause a
> cherry pick to invalidate all prior testing.
>
> The PR [2] is making good progress in ensuring no linkage issues, so I
> still anticipate that the 2.42.0 branch cut will happen by EoD today
> Pacific Time.
>
> If that doesn't occur, I'll send another email to this thread with an
> update.
>
> Thank you for your patience,
> Your friendly neighbourhood release manager
> Robert Burke
>
> [1] https://github.com/apache/beam/milestone/4
> [2] https://github.com/apache/beam/pull/22996
>
> On Wed, Aug 24, 2022 at 3:05 PM Robert Burke  wrote:
>
>> Hi everyone!
>>
>> The next (2.42.0) release branch cut is scheduled for Sept 7th,
>> according to
>> the release calendar [1].
>>
>> I would like to volunteer myself to do this release. My plan is to cut
>> the branch on that date, and cherrypick release-blocking fixes afterwards,
>> if any.
>>
>> Please help me make sure the release goes smoothly by:
>> - Making sure that any unresolved release blocking issues for 2.42.0 should
>> have their "Milestone" marked as "2.42.0 Release" as soon as possible.
>> - Reviewing the current release blockers [2] and remove the Milestone if
>> they don't meet the criteria at [3].
>>
>> Let me know if you have any comments/objections/questions.
>>
>> Due to travel, I will be unavailable to answer any emails from August
>> 26th until Sept 5th, but cut blocking concerns will be resolved before I
>> make the cut, and will delay it if necessary. This thread will be notified
>> in that instance.
>>
>> Thanks,
>> Robert Burke
>>
>> [1]
>> https://calendar.google.com/calendar/embed?src=0p73sl034k80oob7seouanigd0%40group.calendar.google.com
>> [2] https://github.com/apache/beam/milestone/4
>> [3] https://beam.apache.org/contribute/release-blocking/
>>
>

Re:Re: Cannot find beam in project list on jira when I create issue

[张涛]

Thank you very much for your reply!
















At 2022-09-07 18:05:57, "Moritz Mack"  wrote:

Sorry for the confusion. Beam migrated to using Github issues just recently and 
the confluence docs haven’t been updated yet.

 

Please create a new issue under https://github.com/apache/beam/issues and then 
reference it in your commit message using the issue id, e.g.

git commit -am “Description of change (closes #12345)”

 

Regards,

Moritz

 

On 07.09.22, 11:20, "张涛"  wrote:

 

Hi，I followed the step to create a pull request: https: //cwiki. apache. 
org/confluence/display/BEAM/Git+Tips in step 8, need a jira issue: but I cannot 
find beam in project list on jira when I create issue : I was confused about 
what I should

Hi，I followed the step to create a pull request:

https://cwiki.apache.org/confluence/display/BEAM/Git+Tips

in step 8, need a jira issue:

but  I cannot find beam in  project list on jira when I create issue :

 

I was confused about what I should do.  I'm looking forward to getting your 
help，thanks very much!

As a recipient of an email from Talend, your contact personal data will be on 
our systems. Please see our privacy notice.

Re: [PROPOSAL] Preparing for 2.42.0 Release

[Robert Burke]

'One issue remains open in the 2.42.0 milestone [1] blocking the cut.

Since it's a BOM update I'm inclined to wait for it, rather than cause a
cherry pick to invalidate all prior testing.

The PR [2] is making good progress in ensuring no linkage issues, so I
still anticipate that the 2.42.0 branch cut will happen by EoD today
Pacific Time.

If that doesn't occur, I'll send another email to this thread with an
update.

Thank you for your patience,
Your friendly neighbourhood release manager
Robert Burke

[1] https://github.com/apache/beam/milestone/4
[2] https://github.com/apache/beam/pull/22996

On Wed, Aug 24, 2022 at 3:05 PM Robert Burke  wrote:

> Hi everyone!
>
> The next (2.42.0) release branch cut is scheduled for Sept 7th, according
> to
> the release calendar [1].
>
> I would like to volunteer myself to do this release. My plan is to cut
> the branch on that date, and cherrypick release-blocking fixes afterwards,
> if any.
>
> Please help me make sure the release goes smoothly by:
> - Making sure that any unresolved release blocking issues for 2.42.0 should
> have their "Milestone" marked as "2.42.0 Release" as soon as possible.
> - Reviewing the current release blockers [2] and remove the Milestone if
> they don't meet the criteria at [3].
>
> Let me know if you have any comments/objections/questions.
>
> Due to travel, I will be unavailable to answer any emails from August 26th
> until Sept 5th, but cut blocking concerns will be resolved before I make
> the cut, and will delay it if necessary. This thread will be notified in
> that instance.
>
> Thanks,
> Robert Burke
>
> [1]
> https://calendar.google.com/calendar/embed?src=0p73sl034k80oob7seouanigd0%40group.calendar.google.com
> [2] https://github.com/apache/beam/milestone/4
> [3] https://beam.apache.org/contribute/release-blocking/
>

Re: SingleStore IO design doc

[Kenneth Knowles]

Thanks for sharing the doc! Two big picture thoughts I had were:

1. Portability first
2. SDF first

Both of these are just ideas to take advantage of Beam's unique features.
Happy for the contribution no matter what!

Kenn

On Wed, Sep 7, 2022 at 7:21 AM John Casey via dev 
wrote:

> Hi Adalbert,
>
> This looks good. I've added a few comments to consider.
>
> John
>
> On Wed, Sep 7, 2022 at 5:33 AM Adalbert Makarovych <
> amakarovych...@singlestore.com> wrote:
>
>> Hi,
>>
>> I'm working on the SingleStore IO connector.
>> Here  is a GitHub task for
>> it.
>> It is going to be a large change, so I created a design doc
>> 
>> for it.
>> I would appreciate any review/comments regarding it :)
>>
>> --
>> Adalbert Makarovych
>> Software Engineer at SingleStore
>>
>>
>> 
>>
>

Re: Cannot find beam in project list on jira when I create issue

[Brian Hulette via dev]

Thank you Moritz for updating the docs!

On Wed, Sep 7, 2022 at 3:06 AM Moritz Mack  wrote:

> Sorry for the confusion. Beam migrated to using Github issues just
> recently and the confluence docs haven’t been updated yet.
>
>
>
> Please create a new issue under https://github.com/apache/beam/issues and
> then reference it in your commit message using the issue id, e.g.
>
> git commit -am “Description of change (closes #12345)”
>
>
>
> Regards,
>
> Moritz
>
>
>
> On 07.09.22, 11:20, "张涛"  wrote:
>
>
>
> Hi，I followed the step to create a pull request: https: //cwiki. apache.
> org/confluence/display/BEAM/Git+Tips in step 8, need a jira issue: but I
> cannot find beam in project list on jira when I create issue : I was
> confused about what I should
>
> Hi，I followed the step to create a pull request:
>
> https://cwiki.apache.org/confluence/display/BEAM/Git+Tips
> 
>
> in step 8, need a jira issue:
>
> but  I cannot find beam in  project list on jira when I create issue :
>
>
>
> I was confused about what I should do.  I'm looking forward to getting
> your help，thanks very much!
>
> *As a recipient of an email from Talend, your contact personal data will
> be on our systems. Please see our privacy notice.
> *
>
>
>

Re: SingleStore IO design doc

[John Casey via dev]

Hi Adalbert,

This looks good. I've added a few comments to consider.

John

On Wed, Sep 7, 2022 at 5:33 AM Adalbert Makarovych <
amakarovych...@singlestore.com> wrote:

> Hi,
>
> I'm working on the SingleStore IO connector.
> Here  is a GitHub task for
> it.
> It is going to be a large change, so I created a design doc
> 
> for it.
> I would appreciate any review/comments regarding it :)
>
> --
> Adalbert Makarovych
> Software Engineer at SingleStore
>
>
> 
>

Beam High Priority Issue Report (70)

[beamactions]

This is your daily summary of Beam's current high priority issues that may need 
attention.

See https://beam.apache.org/contribute/issue-priorities for the meaning and 
expectations around issue priorities.

Unassigned P1 Issues:

https://github.com/apache/beam/issues/23056 [Bug]: 
apache_beam.ml.inference.sklearn_inference_it_test.SklearnInference.test_sklearn_mnist_classification
 fails because of the removal of headers from the input file
https://github.com/apache/beam/issues/23020 [Bug]: Kubernetes service not 
cleaned up and reaching quota causing multiple performance test failure
https://github.com/apache/beam/issues/22969 Discrepancy in behavior of 
`DoFn.process()` when `yield` is combined with `return` statement, or vice versa
https://github.com/apache/beam/issues/22913 [Bug]: 
beam_PostCommit_Java_ValidatesRunner_Flink is failing
https://github.com/apache/beam/issues/22749 [Bug]: Bytebuddy version update 
causes Invisible parameter type error
https://github.com/apache/beam/issues/22743 [Bug]: Test flake: 
org.apache.beam.sdk.io.gcp.bigquery.BigQueryServicesImplTest.testInsertWithinRowCountLimits
https://github.com/apache/beam/issues/22440 [Bug]: Python Batch Dataflow 
SideInput LoadTests failing
https://github.com/apache/beam/issues/22321 
PortableRunnerTestWithExternalEnv.test_pardo_large_input is regularly failing 
on jenkins
https://github.com/apache/beam/issues/22303 [Task]: Add tests to Kafka SDF and 
fix known and discovered issues
https://github.com/apache/beam/issues/22299 [Bug]: JDBCIO Write freeze at 
getConnection() in WriteFn
https://github.com/apache/beam/issues/22283 [Bug]: Python Lots of fn runner 
test items cost exactly 5 seconds to run
https://github.com/apache/beam/issues/21794 Dataflow runner creates a new timer 
whenever the output timestamp is change
https://github.com/apache/beam/issues/21713 404s in BigQueryIO don't get output 
to Failed Inserts PCollection
https://github.com/apache/beam/issues/21704 beam_PostCommit_Java_DataflowV2 
failures parent bug
https://github.com/apache/beam/issues/21703 pubsublite.ReadWriteIT failing in 
beam_PostCommit_Java_DataflowV1 and V2
https://github.com/apache/beam/issues/21702 SpannerWriteIT failing in beam 
PostCommit Java V1
https://github.com/apache/beam/issues/21701 beam_PostCommit_Java_DataflowV1 
failing with a variety of flakes and errors
https://github.com/apache/beam/issues/21700 
--dataflowServiceOptions=use_runner_v2 is broken
https://github.com/apache/beam/issues/21696 Flink Tests failure :  
java.lang.NoClassDefFoundError: Could not initialize class 
org.apache.beam.runners.core.construction.SerializablePipelineOptions 
https://github.com/apache/beam/issues/21695 DataflowPipelineResult does not 
raise exception for unsuccessful states.
https://github.com/apache/beam/issues/21694 BigQuery Storage API insert with 
writeResult retry and write to error table
https://github.com/apache/beam/issues/21480 flake: 
FlinkRunnerTest.testEnsureStdoutStdErrIsRestored
https://github.com/apache/beam/issues/21472 Dataflow streaming tests failing 
new AfterSynchronizedProcessingTime test
https://github.com/apache/beam/issues/21471 Flakes: Failed to load cache entry
https://github.com/apache/beam/issues/21470 Test flake: test_split_half_sdf
https://github.com/apache/beam/issues/21469 beam_PostCommit_XVR_Flink flaky: 
Connection refused
https://github.com/apache/beam/issues/21468 
beam_PostCommit_Python_Examples_Dataflow failing
https://github.com/apache/beam/issues/21467 GBK and CoGBK streaming Java load 
tests failing
https://github.com/apache/beam/issues/21465 Kafka commit offset drop data on 
failure for runners that have non-checkpointing shuffle
https://github.com/apache/beam/issues/21463 NPE in Flink Portable 
ValidatesRunner streaming suite
https://github.com/apache/beam/issues/21462 Flake in 
org.apache.beam.sdk.io.mqtt.MqttIOTest.testReadObject: Address already in use
https://github.com/apache/beam/issues/21271 pubsublite.ReadWriteIT flaky in 
beam_PostCommit_Java_DataflowV2  
https://github.com/apache/beam/issues/21270 
org.apache.beam.sdk.transforms.CombineTest$WindowingTests.testWindowedCombineGloballyAsSingletonView
 flaky on Dataflow Runner V2
https://github.com/apache/beam/issues/21268 Race between member variable being 
accessed due to leaking uninitialized state via OutboundObserverFactory
https://github.com/apache/beam/issues/21267 WriteToBigQuery submits a duplicate 
BQ load job if a 503 error code is returned from googleapi
https://github.com/apache/beam/issues/21266 
org.apache.beam.sdk.transforms.ParDoLifecycleTest.testTeardownCalledAfterExceptionInProcessElementStateful
 is flaky in Java ValidatesRunner Flink suite.
https://github.com/apache/beam/issues/21265 
apache_beam.runners.portability.fn_api_runner.translations_test.TranslationsTest.test_run_packable_combine_globally
 'apache_beam.coders.coder_impl._AbstractIterable' object is not reversible
https://github.com/apache/beam/issues/21263 (Broken Pipe induced) 

Re: Cannot find beam in project list on jira when I create issue

[Moritz Mack]

Sorry for the confusion. Beam migrated to using Github issues just recently and 
the confluence docs haven’t been updated yet.

Please create a new issue under https://github.com/apache/beam/issues and then 
reference it in your commit message using the issue id, e.g.
git commit -am “Description of change (closes #12345)”

Regards,
Moritz

On 07.09.22, 11:20, "张涛"  wrote:

Hi，I followed the step to create a pull request: https: //cwiki. apache. 
org/confluence/display/BEAM/Git+Tips in step 8, need a jira issue: but I cannot 
find beam in project list on jira when I create issue : I was confused about 
what I should

Hi，I followed the step to create a pull request:
https://cwiki.apache.org/confluence/display/BEAM/Git+Tips
in step 8, need a jira issue:
[cid:23eb7fe9$2$183173c9711$Coremail$zhangtao7094004$163.com]
but  I cannot find beam in  project list on jira when I create issue :
[cid:273c1de6$1$18317380383$Coremail$zhangtao7094004$163.com]

I was confused about what I should do.  I'm looking forward to getting your 
help，thanks very much!

As a recipient of an email from Talend, your contact personal data will be on 
our systems. Please see our privacy notice. 

SingleStore IO design doc

[Adalbert Makarovych]

Hi,

I'm working on the SingleStore IO connector.
Here  is a GitHub task for it.
It is going to be a large change, so I created a design doc

for it.
I would appreciate any review/comments regarding it :)

-- 
Adalbert Makarovych
Software Engineer at SingleStore

Cannot find beam in project list on jira when I create issue

[张涛]

Hi，I followed the step to create a pull request:
https://cwiki.apache.org/confluence/display/BEAM/Git+Tips
in step 8, need a jira issue:
but  I cannot find beam in  project list on jira when I create issue :


I was confused about what I should do.  I'm looking forward to getting your 
help，thanks very much!