Beam Summit announces workshops

2022-05-06 Thread Mara Ruvalcaba 

*
*

**

*


*Beam Summit announces confirmed Workshops!*

**

**Take the opportunity to explore in-depth Apache Beam by participating 
at the workshops!


Workshops will be held on-site only on Wednesday, July 20th. Organized 
by morning and afternoon blocks, you can attend up to 2 workshops.


For participating at the workshops you need to acquire the 3 day pass. 
When registering, you will choose what workshops to attend.Remember, you 
canapply for a scholarship .


*
**Take a look at the workshops confirmed:*




   Apache Beam on Amazon Kinesis Data Analytics (KDA)
   

This workshop explores an end to end example that combines batch and 
streaming aspects in one uniform Apache Beam pipeline.





   Beam Cross Language Transforms in Python, with Google Cloud Dataflow
   

For Beam practitioners who wish to advance their knowledge of Apache 
Beam and Google Cloud Dataflow.






   Splittable DoFns in Python: a hands-on workshop
   

This workshop reviews the concept of Splittable DoFns and we will write 
two I/O connectors using this kind of DoFns: one in batch, and one for 
streaming.


**


**


   Early Bird price ends next week!
   Get your tickets for the onsite event before May 13th and obtain the
   Early Bird price + get the chance to win a $100 gift card

 *

   First 50 onsite registrations will get the chance to win a $100
   amazon gift card.

 *

   Early bird pricing for in-person passes is $290 USD for 2-day pass
   and $350 USD for 3-day pass.


   Apply for a scholarship

If you would like to attend in person but cannot afford a ticket, 
pleaseapply for a scholarship. 
Special thanks to our Diversity and 
Inclusion sponsor Maven Codefor the scholarships.


** 


   *Register Now!* 


 Why in Austin?

In Texas, more than 17,600 technology firms employ over 203,700 workers 
with companies like Apple and Wipro continuing to grow every day. The 
long-established Texas tech sector has invented everything from the 
semiconductor and hand-held calculator to a billion-dollar dating 
app. Texas is brimming with skilled talent ready to solve their next 
challenge. Tech workers are migrating to Texas at a record rate as well.


Plus, Austin has the best live music =)

*

--
Mara Ruvalcaba
COO, SG Software Guru & Nearshore Link
USA: 512 296 2884
MX: 55 5239 5502

Flaky test issue report (56)

2022-05-06 Thread Beam Jira Bot 
This is your daily summary of Beam's current flaky tests 
(https://issues.apache.org/jira/issues/?jql=project%20%3D%20BEAM%20AND%20statusCategory%20!%3D%20Done%20AND%20labels%20%3D%20flake)

These are P1 issues because they have a major negative impact on the community 
and make it hard to determine the quality of the software.

https://issues.apache.org/jira/browse/BEAM-14410: FnRunnerTest with 
non-trivial (order 1000 elements) numpy input flakes in non-cython environment 
(created 2022-05-04)
https://issues.apache.org/jira/browse/BEAM-14407: Jenkins worker sometimes 
crashes while running Python Flink pipeline (created 2022-05-04)
https://issues.apache.org/jira/browse/BEAM-14367: Flaky timeout in github 
Python unit test action 
StatefulDoFnOnDirectRunnerTest.test_dynamic_timer_clear_then_set_timer (created 
2022-04-26)
https://issues.apache.org/jira/browse/BEAM-14349: GroupByKeyTest BasicTests 
testLargeKeys100MB flake (on ULR) (created 2022-04-21)
https://issues.apache.org/jira/browse/BEAM-14276: 
beam_PostCommit_Java_DataflowV2 failures parent bug (created 2022-04-07)
https://issues.apache.org/jira/browse/BEAM-14269: 
PulsarIOTest.testReadFromSimpleTopic is very flaky (created 2022-04-06)
https://issues.apache.org/jira/browse/BEAM-14263: 
beam_PostCommit_Java_DataflowV2, testBigQueryStorageWrite30MProto failing 
consistently (created 2022-04-05)
https://issues.apache.org/jira/browse/BEAM-14252: 
beam_PostCommit_Java_DataflowV1 failing with a variety of flakes and errors 
(created 2022-04-05)
https://issues.apache.org/jira/browse/BEAM-14216: Multiple XVR Suites 
having similar flakes simultaneously (created 2022-03-31)
https://issues.apache.org/jira/browse/BEAM-14174: Flink Tests failure :  
java.lang.NoClassDefFoundError: Could not initialize class 
org.apache.beam.runners.core.construction.SerializablePipelineOptions  (created 
2022-03-24)
https://issues.apache.org/jira/browse/BEAM-14172: beam_PreCommit_PythonDocs 
failing (jinja2) (created 2022-03-24)
https://issues.apache.org/jira/browse/BEAM-13952: Dataflow streaming tests 
failing new AfterSynchronizedProcessingTime test (created 2022-02-15)
https://issues.apache.org/jira/browse/BEAM-13859: Test flake: 
test_split_half_sdf (created 2022-02-09)
https://issues.apache.org/jira/browse/BEAM-13850: 
beam_PostCommit_Python_Examples_Dataflow failing (created 2022-02-08)
https://issues.apache.org/jira/browse/BEAM-13822: GBK and CoGBK streaming 
Java load tests failing (created 2022-02-03)
https://issues.apache.org/jira/browse/BEAM-13810: Flaky tests: Gradle build 
daemon disappeared unexpectedly (created 2022-02-03)
https://issues.apache.org/jira/browse/BEAM-13809: beam_PostCommit_XVR_Flink 
flaky: Connection refused (created 2022-02-03)
https://issues.apache.org/jira/browse/BEAM-13797: Flakes: Failed to load 
cache entry (created 2022-02-01)
https://issues.apache.org/jira/browse/BEAM-13708: flake: 
FlinkRunnerTest.testEnsureStdoutStdErrIsRestored (created 2022-01-20)
https://issues.apache.org/jira/browse/BEAM-13575: Flink 
testParDoRequiresStableInput flaky (created 2021-12-28)
https://issues.apache.org/jira/browse/BEAM-13500: NPE in Flink Portable 
ValidatesRunner streaming suite (created 2021-12-21)
https://issues.apache.org/jira/browse/BEAM-13453: Flake in 
org.apache.beam.sdk.io.mqtt.MqttIOTest.testReadObject: Address already in use 
(created 2021-12-13)
https://issues.apache.org/jira/browse/BEAM-13393: GroupIntoBatchesTest is 
failing (created 2021-12-07)
https://issues.apache.org/jira/browse/BEAM-13367: 
[beam_PostCommit_Python36] [ 
apache_beam.io.gcp.experimental.spannerio_read_it_test] Failure summary 
(created 2021-12-01)
https://issues.apache.org/jira/browse/BEAM-13312: 
org.apache.beam.sdk.transforms.ParDoLifecycleTest.testTeardownCalledAfterExceptionInStartBundle
 is flaky in Java Spark ValidatesRunner suite  (created 2021-11-23)
https://issues.apache.org/jira/browse/BEAM-13311: 
org.apache.beam.sdk.transforms.ParDoLifecycleTest.testTeardownCalledAfterExceptionInProcessElementStateful
 is flaky in Java ValidatesRunner Flink suite. (created 2021-11-23)
https://issues.apache.org/jira/browse/BEAM-13237: 
org.apache.beam.sdk.transforms.CombineTest$WindowingTests.testWindowedCombineGloballyAsSingletonView
 flaky on Dataflow Runner V2 (created 2021-11-12)
https://issues.apache.org/jira/browse/BEAM-13025: pubsublite.ReadWriteIT 
flaky in beam_PostCommit_Java_DataflowV2   (created 2021-10-08)
https://issues.apache.org/jira/browse/BEAM-12928: beam_PostCommit_Python36 
- CrossLanguageSpannerIOTest - flakey failing (created 2021-09-21)
https://issues.apache.org/jira/browse/BEAM-12859: 
org.apache.beam.runners.dataflow.worker.fn.logging.BeamFnLoggingServiceTest.testMultipleClientsFailingIsHandledGracefullyByServer
 is flaky (created 2021-09-08)
https://issues.apache.org/jira/browse/BEAM-12809:

P1 issues report (79)

2022-05-06 Thread Beam Jira Bot 
This is your daily summary of Beam's current P1 issues, not including flaky 
tests 
(https://issues.apache.org/jira/issues/?jql=project%20%3D%20BEAM%20AND%20statusCategory%20!%3D%20Done%20AND%20priority%20%3D%20P1%20AND%20(labels%20is%20EMPTY%20OR%20labels%20!%3D%20flake).

See https://beam.apache.org/contribute/jira-priorities/#p1-critical for the 
meaning and expectations around P1 issues.

https://issues.apache.org/jira/browse/BEAM-14421: 
--dataflowServiceOptions=use_runner_v2 is broken (created 2022-05-05)
https://issues.apache.org/jira/browse/BEAM-14416: ParDo LoadTest 
performance regression on java streaming dataflow runner v2 (created 2022-05-04)
https://issues.apache.org/jira/browse/BEAM-14412: Block release on 
impersonation FR (created 2022-05-04)
https://issues.apache.org/jira/browse/BEAM-14411: TypeCodersTest is never 
executed (created 2022-05-04)
https://issues.apache.org/jira/browse/BEAM-14403: Allow Prime to be used 
with Legacy workers (created 2022-05-03)
https://issues.apache.org/jira/browse/BEAM-14399: 
org.apache.beam.sdk.io.gcp.spanner.SpannerWriteIT.testReportFailures is failing 
in Dataflow Java Runner V1 postcommits. (created 2022-05-03)
https://issues.apache.org/jira/browse/BEAM-14390: Java license check is 
broken (created 2022-05-02)
https://issues.apache.org/jira/browse/BEAM-14364: 404s in BigQueryIO don't 
get output to Failed Inserts PCollection (created 2022-04-25)
https://issues.apache.org/jira/browse/BEAM-14356: Java PostCommits: 
BigQueryIO.Read needs a GCS temp location (created 2022-04-22)
https://issues.apache.org/jira/browse/BEAM-14298: Can't resolve 
org.pentaho:pentaho-aggdesigner-algorithm:5.1.5-jhyde (created 2022-04-12)
https://issues.apache.org/jira/browse/BEAM-14291: DataflowPipelineResult 
does not raise exception for unsuccessful states. (created 2022-04-11)
https://issues.apache.org/jira/browse/BEAM-14276: 
beam_PostCommit_Java_DataflowV2 failures parent bug (created 2022-04-07)
https://issues.apache.org/jira/browse/BEAM-14275: SpannerWriteIT failing in 
beam PostCommit Java V1 (created 2022-04-07)
https://issues.apache.org/jira/browse/BEAM-14265: Flink should hold the 
watermark at the output timestamp for processing time timers (created 
2022-04-06)
https://issues.apache.org/jira/browse/BEAM-14263: 
beam_PostCommit_Java_DataflowV2, testBigQueryStorageWrite30MProto failing 
consistently (created 2022-04-05)
https://issues.apache.org/jira/browse/BEAM-14253: pubsublite.ReadWriteIT 
failing in beam_PostCommit_Java_DataflowV1 and V2 (created 2022-04-05)
https://issues.apache.org/jira/browse/BEAM-14239: Changing the output 
timestamp of a timer does not clear the previously set timer (created 
2022-04-04)
https://issues.apache.org/jira/browse/BEAM-14174: Flink Tests failure :  
java.lang.NoClassDefFoundError: Could not initialize class 
org.apache.beam.runners.core.construction.SerializablePipelineOptions  (created 
2022-03-24)
https://issues.apache.org/jira/browse/BEAM-14146: Python Streaming job 
failing to drain with BigQueryIO write errors (created 2022-03-22)
https://issues.apache.org/jira/browse/BEAM-14135: BigQuery Storage API 
insert with writeResult retry and write to error table (created 2022-03-20)
https://issues.apache.org/jira/browse/BEAM-13952: Dataflow streaming tests 
failing new AfterSynchronizedProcessingTime test (created 2022-02-15)
https://issues.apache.org/jira/browse/BEAM-13950: PVR_Spark2_Streaming 
perma-red (created 2022-02-15)
https://issues.apache.org/jira/browse/BEAM-13920: Beam x-lang Dataflow 
tests failing due to _InactiveRpcError (created 2022-02-10)
https://issues.apache.org/jira/browse/BEAM-13852: 
KafkaIO.read.withDynamicRead() doesn't pick up new TopicPartitions (created 
2022-02-08)
https://issues.apache.org/jira/browse/BEAM-13850: 
beam_PostCommit_Python_Examples_Dataflow failing (created 2022-02-08)
https://issues.apache.org/jira/browse/BEAM-13822: GBK and CoGBK streaming 
Java load tests failing (created 2022-02-03)
https://issues.apache.org/jira/browse/BEAM-13805: Simplify version override 
for Dev versions of the Go SDK. (created 2022-02-02)
https://issues.apache.org/jira/browse/BEAM-13747: Add integration testing 
for BQ Storage API  write modes (created 2022-01-26)
https://issues.apache.org/jira/browse/BEAM-13715: Kafka commit offset drop 
data on failure for runners that have non-checkpointing shuffle (created 
2022-01-21)
https://issues.apache.org/jira/browse/BEAM-13487: WriteToBigQuery Dynamic 
table destinations returns wrong tableId (created 2021-12-17)
https://issues.apache.org/jira/browse/BEAM-13393: GroupIntoBatchesTest is 
failing (created 2021-12-07)
https://issues.apache.org/jira/browse/BEAM-13164: Race between member 
variable being accessed due to leaking uninitialized state via 
OutboundObserverFactory (created 2021-11-01)
https://issues.apache.org/jira/browse/BEAM-13132:

P0 (outage) report

2022-05-06 Thread Beam Jira Bot 
This is your daily summary of Beam's current outages. See 
https://beam.apache.org/contribute/jira-priorities/#p0-outage for the meaning 
and expectations around P0 issues.

BEAM-14420: Mongo DB Upgrade to 5.0 having compatibitlity issue with Apache 
Beam included mong0-java-driver. 
(https://issues.apache.org/jira/browse/BEAM-14420)
BEAM-14396: PyPI apache-beam depends on httplib2<0.20.0 
(https://issues.apache.org/jira/browse/BEAM-14396)

[CdapIO] CDAP update and code reviews

2022-05-06 Thread Elizaveta Lomteva 
Hi, community!


Our team is working on the new CdapIO connector implementation and we prepared 
PRs for components of the CdapIO package.


PRs ready for code review:

[1] CDAP context classes for CDAP plugins

[2] CDAP plugin wrapper classes


We would appreciate it very much if you could review ready PRs and leave the 
comments.


Thank you for your attention to it,

Elizaveta


[1] CDAP context classes for CDAP plugins 
https://github.com/apache/beam/pull/17104

[2] CDAP plugin wrapper classes https://github.com/apache/beam/pull/17150

Access required - Jira contributor permissions

2022-05-06 Thread Sergey Makarkin 
Hello
My name is Sergey Makarkin.
I'll be working on Apache Beam Playground project and  I need to have Jira 
contributor permissions for the project contribution.
Could you please grant this permission to me?
My Github account: SMakarkinAkvelon
My Jira account: Sergey.Makarkin


Regards

Sergey Makarkin

Akvelon