(beam) branch nightly-refs/heads/master updated (5a090958992 -> 3b8ddda10a0)

2024-07-01 Thread github-bot 
This is an automated email from the ASF dual-hosted git repository.

github-bot pushed a change to branch nightly-refs/heads/master
in repository https://gitbox.apache.org/repos/asf/beam.git


from 5a090958992 Move CsvIOParseResult (#31722)
 add ef5060416d9 [Dataflow Streaming] Enabled Heartbeat by Default (#31689)
 add bae6fcfcb49 Add a unit test for MakePipelineOptionsFileAndEnvVar 
(#31732)
 add 47384772cac Dont read from cache in sensitive workflows (#31734)
 add a3a62b866e0 Add scaffold CsvIOParseHelpers (#31720)
 add 2c7211351f5 Remove testRuntimeMigration configuration for test-utils 
dependencies (#31713)
 add 957a7cd22b6 created CsvIOReadFiles class (#31738)
 add 3b8ddda10a0 Solace Read connector: adding Basic Authentication support 
(#31541)

No new revisions were added by this update.

Summary of changes:
 .github/workflows/beam_Publish_Website.yml |   2 -
 .github/workflows/beam_Release_NightlySnapshot.yml |   3 +-
 .../workflows/deploy_release_candidate_pypi.yaml   |   3 +-
 .github/workflows/finalize_release.yml |   3 +-
 .../dataflow/worker/StreamingDataflowWorker.java   |   4 +-
 .../windmill/client/grpc/GrpcWindmillServer.java   |   4 +-
 .../client/grpc/GrpcWindmillServerTest.java|  37 +++---
 sdks/go/container/tools/pipeline_options.go|   5 +
 .../go/container/tools/pipeline_options_test.go|  41 --
 sdks/java/io/bigquery-io-perf-tests/build.gradle   |   4 +-
 sdks/java/io/cdap/build.gradle |   2 +-
 .../java/org/apache/beam/sdk/io/cdap/CdapIOIT.java |   4 +-
 .../apache/beam/sdk/io/csv/CsvIOParseHelpers.java  |  55 
 .../org/apache/beam/sdk/io/csv/CsvIOReadFiles.java |  54 
 ...figurationTest.java => CsvIOReadFilesTest.java} |   9 +-
 sdks/java/io/file-based-io-tests/build.gradle  |   2 +-
 .../java/org/apache/beam/sdk/io/avro/AvroIOIT.java |   4 +-
 .../apache/beam/sdk/io/parquet/ParquetIOIT.java|   4 +-
 .../java/org/apache/beam/sdk/io/text/TextIOIT.java |   8 +-
 .../apache/beam/sdk/io/tfrecord/TFRecordIOIT.java  |   4 +-
 .../java/org/apache/beam/sdk/io/xml/XmlIOIT.java   |   4 +-
 sdks/java/io/google-cloud-platform/build.gradle|   9 +-
 sdks/java/io/hadoop-format/build.gradle|   2 +-
 .../sdk/io/hadoop/format/HadoopFormatIOIT.java |   4 +-
 sdks/java/io/jdbc/build.gradle |   2 +-
 .../java/org/apache/beam/sdk/io/jdbc/JdbcIOIT.java |   3 +-
 sdks/java/io/jms/build.gradle  |   2 +-
 sdks/java/io/kafka/build.gradle|   2 +-
 .../org/apache/beam/sdk/io/kafka/KafkaIOIT.java|   4 +-
 sdks/java/io/mongodb/build.gradle  |   2 +-
 .../apache/beam/sdk/io/mongodb/MongoDBIOIT.java|   4 +-
 sdks/java/io/neo4j/build.gradle|   2 +-
 sdks/java/io/singlestore/build.gradle  |   3 +-
 .../io/singlestore/SingleStoreIOPerformanceIT.java |   4 +-
 sdks/java/io/snowflake/build.gradle|   2 +-
 sdks/java/io/solace/build.gradle   |   2 +-
 .../broker/BasicAuthJcsmpSessionService.java   | 148 +
 .../BasicAuthJcsmpSessionServiceFactory.java   |  74 +++
 .../io/solace/broker/SolaceMessageReceiver.java|  72 ++
 sdks/java/io/sparkreceiver/2/build.gradle  |   5 +-
 .../sdk/io/sparkreceiver/SparkReceiverIOIT.java|   4 +-
 sdks/java/testing/nexmark/build.gradle |   2 +-
 sdks/java/testing/tpcds/build.gradle   |   2 +-
 43 files changed, 511 insertions(+), 99 deletions(-)
 copy playground/backend/internal/tests/test_utils/test_utils_test.go => 
sdks/go/container/tools/pipeline_options_test.go (56%)
 create mode 100644 
sdks/java/io/csv/src/main/java/org/apache/beam/sdk/io/csv/CsvIOParseHelpers.java
 create mode 100644 
sdks/java/io/csv/src/main/java/org/apache/beam/sdk/io/csv/CsvIOReadFiles.java
 copy 
sdks/java/io/csv/src/test/java/org/apache/beam/sdk/io/csv/{CsvIOParseConfigurationTest.java
 => CsvIOReadFilesTest.java} (80%)
 create mode 100644 
sdks/java/io/solace/src/main/java/org/apache/beam/sdk/io/solace/broker/BasicAuthJcsmpSessionService.java
 create mode 100644 
sdks/java/io/solace/src/main/java/org/apache/beam/sdk/io/solace/broker/BasicAuthJcsmpSessionServiceFactory.java
 create mode 100644 
sdks/java/io/solace/src/main/java/org/apache/beam/sdk/io/solace/broker/SolaceMessageReceiver.java

(beam) branch asf-site updated: Publishing website 2024/07/01 23:38:16 at commit 3b8ddda

2024-07-01 Thread github-bot 
This is an automated email from the ASF dual-hosted git repository.

github-bot pushed a commit to branch asf-site
in repository https://gitbox.apache.org/repos/asf/beam.git


The following commit(s) were added to refs/heads/asf-site by this push:
 new 10569dd9423 Publishing website 2024/07/01 23:38:16 at commit 3b8ddda
10569dd9423 is described below

commit 10569dd942385fc2a90629d2ace8e35361b8de3d
Author: runner 
AuthorDate: Mon Jul 1 23:38:16 2024 +

Publishing website 2024/07/01 23:38:16 at commit 3b8ddda
---
 website/generated-content/sitemap.xml | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/website/generated-content/sitemap.xml 
b/website/generated-content/sitemap.xml
index cdef3f87205..69f447b07de 100644
--- a/website/generated-content/sitemap.xml
+++ b/website/generated-content/sitemap.xml
@@ -1 +1 @@
-http://www.sitemaps.org/schemas/sitemap/0.9"; 
xmlns:xhtml="http://www.w3.org/1999/xhtml";>/blog/beam-2.57.0/2024-07-01T09:28:11-07:00/categories/blog/2024-07-01T09:28:11-07:00/blog/2024-07-01T09:28:11-07:00/categories/2024-07-01T09:28:11-07:00/catego
 [...]
\ No newline at end of file
+http://www.sitemaps.org/schemas/sitemap/0.9"; 
xmlns:xhtml="http://www.w3.org/1999/xhtml";>/blog/beam-2.57.0/2024-07-01T23:24:57+02:00/categories/blog/2024-07-01T23:24:57+02:00/blog/2024-07-01T23:24:57+02:00/categories/2024-07-01T23:24:57+02:00/catego
 [...]
\ No newline at end of file

(beam) branch pr-bot-state updated: Updating config from bot

2024-07-01 Thread github-bot 
This is an automated email from the ASF dual-hosted git repository.

github-bot pushed a commit to branch pr-bot-state
in repository https://gitbox.apache.org/repos/asf/beam.git


The following commit(s) were added to refs/heads/pr-bot-state by this push:
 new f346977f5dc Updating config from bot
f346977f5dc is described below

commit f346977f5dc3538da8e678aea75cb7cb72fae47e
Author: github-actions 
AuthorDate: Mon Jul 1 23:33:31 2024 +

Updating config from bot
---
 scripts/ci/pr-bot/state/pr-state/pr-31741.json | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/scripts/ci/pr-bot/state/pr-state/pr-31741.json 
b/scripts/ci/pr-bot/state/pr-state/pr-31741.json
index 242a48d7d3b..37f7ab41238 100644
--- a/scripts/ci/pr-bot/state/pr-state/pr-31741.json
+++ b/scripts/ci/pr-bot/state/pr-state/pr-31741.json
@@ -2,7 +2,7 @@
   "commentedAboutFailingChecks": true,
   "reviewersAssignedForLabels": {},
   "nextAction": "Author",
-  "stopReviewerNotifications": false,
+  "stopReviewerNotifications": true,
   "remindAfterTestsPass": [],
   "committerAssigned": false
 }
\ No newline at end of file

(beam) branch pr-bot-state updated: Updating config from bot

2024-07-01 Thread github-bot 
This is an automated email from the ASF dual-hosted git repository.

github-bot pushed a commit to branch pr-bot-state
in repository https://gitbox.apache.org/repos/asf/beam.git


The following commit(s) were added to refs/heads/pr-bot-state by this push:
 new eea873fd04f Updating config from bot
eea873fd04f is described below

commit eea873fd04f4f0b839aa238eb60c080b15b8fe13
Author: github-actions 
AuthorDate: Mon Jul 1 22:05:53 2024 +

Updating config from bot
---
 scripts/ci/pr-bot/state/pr-state/pr-31741.json | 8 
 1 file changed, 8 insertions(+)

diff --git a/scripts/ci/pr-bot/state/pr-state/pr-31741.json 
b/scripts/ci/pr-bot/state/pr-state/pr-31741.json
new file mode 100644
index 000..242a48d7d3b
--- /dev/null
+++ b/scripts/ci/pr-bot/state/pr-state/pr-31741.json
@@ -0,0 +1,8 @@
+{
+  "commentedAboutFailingChecks": true,
+  "reviewersAssignedForLabels": {},
+  "nextAction": "Author",
+  "stopReviewerNotifications": false,
+  "remindAfterTestsPass": [],
+  "committerAssigned": false
+}
\ No newline at end of file

(beam) branch master updated: Solace Read connector: adding Basic Authentication support (#31541)

2024-07-01 Thread damondouglas 
This is an automated email from the ASF dual-hosted git repository.

damondouglas pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/beam.git


The following commit(s) were added to refs/heads/master by this push:
 new 3b8ddda10a0 Solace Read connector: adding Basic Authentication support 
(#31541)
3b8ddda10a0 is described below

commit 3b8ddda10a01ff640ed4cf1ce746d0c19e003180
Author: Bartosz Zablocki 
AuthorDate: Mon Jul 1 23:24:57 2024 +0200

Solace Read connector: adding Basic Authentication support (#31541)

* Add support for BasicAuth to Solace

* Address PR comments

* Use `checkStateNotNull`
---
 .../broker/BasicAuthJcsmpSessionService.java   | 148 +
 .../BasicAuthJcsmpSessionServiceFactory.java   |  74 +++
 .../io/solace/broker/SolaceMessageReceiver.java|  72 ++
 3 files changed, 294 insertions(+)

diff --git 
a/sdks/java/io/solace/src/main/java/org/apache/beam/sdk/io/solace/broker/BasicAuthJcsmpSessionService.java
 
b/sdks/java/io/solace/src/main/java/org/apache/beam/sdk/io/solace/broker/BasicAuthJcsmpSessionService.java
new file mode 100644
index 000..7863dbd129c
--- /dev/null
+++ 
b/sdks/java/io/solace/src/main/java/org/apache/beam/sdk/io/solace/broker/BasicAuthJcsmpSessionService.java
@@ -0,0 +1,148 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.beam.sdk.io.solace.broker;
+
+import static org.apache.beam.sdk.util.Preconditions.checkStateNotNull;
+
+import com.solacesystems.jcsmp.ConsumerFlowProperties;
+import com.solacesystems.jcsmp.EndpointProperties;
+import com.solacesystems.jcsmp.FlowReceiver;
+import com.solacesystems.jcsmp.InvalidPropertiesException;
+import com.solacesystems.jcsmp.JCSMPException;
+import com.solacesystems.jcsmp.JCSMPFactory;
+import com.solacesystems.jcsmp.JCSMPProperties;
+import com.solacesystems.jcsmp.JCSMPSession;
+import com.solacesystems.jcsmp.Queue;
+import java.io.IOException;
+import javax.annotation.Nullable;
+import org.apache.beam.sdk.io.solace.RetryCallableManager;
+import 
org.apache.beam.vendor.guava.v32_1_2_jre.com.google.common.collect.ImmutableSet;
+
+/**
+ * A class that manages a connection to a Solace broker using basic 
authentication.
+ *
+ * This class provides a way to connect to a Solace broker and receive 
messages from a queue. The
+ * connection is established using basic authentication.
+ */
+public class BasicAuthJcsmpSessionService implements SessionService {
+  private final String queueName;
+  private final String host;
+  private final String username;
+  private final String password;
+  private final String vpnName;
+  @Nullable private JCSMPSession jcsmpSession;
+  private final RetryCallableManager retryCallableManager = 
RetryCallableManager.create();
+
+  /**
+   * Creates a new {@link BasicAuthJcsmpSessionService} with the given 
parameters.
+   *
+   * @param queueName The name of the queue to receive messages from.
+   * @param host The host name or IP address of the Solace broker. Format: 
Host[:Port]
+   * @param username The username to use for authentication.
+   * @param password The password to use for authentication.
+   * @param vpnName The name of the VPN to connect to.
+   */
+  public BasicAuthJcsmpSessionService(
+  String queueName, String host, String username, String password, String 
vpnName) {
+this.queueName = queueName;
+this.host = host;
+this.username = username;
+this.password = password;
+this.vpnName = vpnName;
+  }
+
+  @Override
+  public void connect() {
+retryCallableManager.retryCallable(this::connectSession, 
ImmutableSet.of(JCSMPException.class));
+  }
+
+  @Override
+  public void close() {
+if (isClosed()) {
+  return;
+}
+retryCallableManager.retryCallable(
+() -> {
+  checkStateNotNull(jcsmpSession).closeSession();
+  return 0;
+},
+ImmutableSet.of(IOException.class));
+  }
+
+  @Override
+  public MessageReceiver createReceiver() {
+return retryCallableManager.retryCallable(
+this::createFlowReceiver, ImmutableSet.of(JCSMPException.class));
+  }
+
+  @Override
+  public boolean isClosed() {
+

(beam) branch master updated: created CsvIOReadFiles class (#31738)

2024-07-01 Thread damondouglas 
This is an automated email from the ASF dual-hosted git repository.

damondouglas pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/beam.git


The following commit(s) were added to refs/heads/master by this push:
 new 957a7cd22b6 created CsvIOReadFiles class (#31738)
957a7cd22b6 is described below

commit 957a7cd22b6974a4c565846ad32b9fb9fc9c8122
Author: lahariguduru <108150650+laharigud...@users.noreply.github.com>
AuthorDate: Mon Jul 1 20:20:15 2024 +

created CsvIOReadFiles class (#31738)

Co-authored-by: Lahari Guduru 
---
 .../org/apache/beam/sdk/io/csv/CsvIOReadFiles.java | 54 ++
 .../apache/beam/sdk/io/csv/CsvIOReadFilesTest.java | 32 +
 2 files changed, 86 insertions(+)

diff --git 
a/sdks/java/io/csv/src/main/java/org/apache/beam/sdk/io/csv/CsvIOReadFiles.java 
b/sdks/java/io/csv/src/main/java/org/apache/beam/sdk/io/csv/CsvIOReadFiles.java
new file mode 100644
index 000..0f6267c6b34
--- /dev/null
+++ 
b/sdks/java/io/csv/src/main/java/org/apache/beam/sdk/io/csv/CsvIOReadFiles.java
@@ -0,0 +1,54 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.beam.sdk.io.csv;
+
+import org.apache.beam.sdk.Pipeline;
+import org.apache.beam.sdk.io.FileIO;
+import org.apache.beam.sdk.transforms.PTransform;
+import org.apache.beam.sdk.values.PCollection;
+import org.apache.beam.sdk.values.PCollectionTuple;
+import org.apache.beam.sdk.values.TupleTag;
+
+/**
+ * Skeleton for error handling in CsvIO that transforms a {@link 
FileIO.ReadableFile} into the
+ * result of parsing.
+ */
+// TODO(https://github.com/apache/beam/issues/31736): Plan completion in 
future PR after
+// dependencies are completed.
+class CsvIOReadFiles extends PTransform, 
CsvIOParseResult> {
+  /** Stores required parameters for parsing. */
+  private final CsvIOParseConfiguration.Builder configBuilder;
+
+  CsvIOReadFiles(CsvIOParseConfiguration.Builder configBuilder) {
+this.configBuilder = configBuilder;
+  }
+
+  /** {@link PTransform} that parses and relays the filename associated with 
each error. */
+  // TODO: complete expand method to unsure parsing from FileIO.ReadableFile 
to CsvIOParseResult.
+  @Override
+  public CsvIOParseResult expand(PCollection input) {
+// TODO(https://github.com/apache/beam/issues/31736): Needed to prevent 
check errors, will
+// remove with future PR.
+configBuilder.build();
+TupleTag outputTag = new TupleTag<>();
+TupleTag errorTag = new TupleTag<>();
+Pipeline p = input.getPipeline();
+PCollectionTuple tuple = PCollectionTuple.empty(p);
+return CsvIOParseResult.of(outputTag, errorTag, tuple);
+  }
+}
diff --git 
a/sdks/java/io/csv/src/test/java/org/apache/beam/sdk/io/csv/CsvIOReadFilesTest.java
 
b/sdks/java/io/csv/src/test/java/org/apache/beam/sdk/io/csv/CsvIOReadFilesTest.java
new file mode 100644
index 000..c4a62f84eae
--- /dev/null
+++ 
b/sdks/java/io/csv/src/test/java/org/apache/beam/sdk/io/csv/CsvIOReadFilesTest.java
@@ -0,0 +1,32 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.beam.sdk.io.csv;
+
+import org.apache.beam.sdk.util.SerializableUtils;
+import org.junit.Test;
+import org.junit.runner.RunWith;
+import org.junit.runners.JUnit4;
+
+/** Tests for {@link CsvIOReadFiles}. */
+@RunWith(JUnit4.class)
+public class CsvIOReadFilesTest {
+  @Test
+  public void isSerializable() {
+SerializableUtils.en

(beam) branch master updated: Remove testRuntimeMigration configuration for test-utils dependencies (#31713)

2024-07-01 Thread yhu 
This is an automated email from the ASF dual-hosted git repository.

yhu pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/beam.git


The following commit(s) were added to refs/heads/master by this push:
 new 2c7211351f5 Remove testRuntimeMigration configuration for test-utils 
dependencies (#31713)
2c7211351f5 is described below

commit 2c7211351f5fb4a82d6ae73c7b975651817526a4
Author: Yi Hu 
AuthorDate: Mon Jul 1 15:59:51 2024 -0400

Remove testRuntimeMigration configuration for test-utils dependencies 
(#31713)

* Remove testRuntimeMigration configuration for test-utils dependencies

* Fix test dependency direct-java configuration
---
 sdks/java/io/bigquery-io-perf-tests/build.gradle | 4 ++--
 sdks/java/io/cdap/build.gradle   | 2 +-
 .../cdap/src/test/java/org/apache/beam/sdk/io/cdap/CdapIOIT.java | 4 ++--
 sdks/java/io/file-based-io-tests/build.gradle| 2 +-
 .../src/test/java/org/apache/beam/sdk/io/avro/AvroIOIT.java  | 4 ++--
 .../test/java/org/apache/beam/sdk/io/parquet/ParquetIOIT.java| 4 ++--
 .../src/test/java/org/apache/beam/sdk/io/text/TextIOIT.java  | 8 
 .../test/java/org/apache/beam/sdk/io/tfrecord/TFRecordIOIT.java  | 4 ++--
 .../src/test/java/org/apache/beam/sdk/io/xml/XmlIOIT.java| 4 ++--
 sdks/java/io/google-cloud-platform/build.gradle  | 9 +++--
 sdks/java/io/hadoop-format/build.gradle  | 2 +-
 .../org/apache/beam/sdk/io/hadoop/format/HadoopFormatIOIT.java   | 4 ++--
 sdks/java/io/jdbc/build.gradle   | 2 +-
 .../jdbc/src/test/java/org/apache/beam/sdk/io/jdbc/JdbcIOIT.java | 3 +--
 sdks/java/io/jms/build.gradle| 2 +-
 sdks/java/io/kafka/build.gradle  | 2 +-
 .../src/test/java/org/apache/beam/sdk/io/kafka/KafkaIOIT.java| 4 ++--
 sdks/java/io/mongodb/build.gradle| 2 +-
 .../test/java/org/apache/beam/sdk/io/mongodb/MongoDBIOIT.java| 4 ++--
 sdks/java/io/neo4j/build.gradle  | 2 +-
 sdks/java/io/singlestore/build.gradle| 3 ++-
 .../beam/sdk/io/singlestore/SingleStoreIOPerformanceIT.java  | 4 ++--
 sdks/java/io/snowflake/build.gradle  | 2 +-
 sdks/java/io/solace/build.gradle | 2 +-
 sdks/java/io/sparkreceiver/2/build.gradle| 5 +++--
 .../org/apache/beam/sdk/io/sparkreceiver/SparkReceiverIOIT.java  | 4 ++--
 sdks/java/testing/nexmark/build.gradle   | 2 +-
 sdks/java/testing/tpcds/build.gradle | 2 +-
 28 files changed, 47 insertions(+), 49 deletions(-)

diff --git a/sdks/java/io/bigquery-io-perf-tests/build.gradle 
b/sdks/java/io/bigquery-io-perf-tests/build.gradle
index 0e251ae668f..e81e7972c8c 100644
--- a/sdks/java/io/bigquery-io-perf-tests/build.gradle
+++ b/sdks/java/io/bigquery-io-perf-tests/build.gradle
@@ -27,9 +27,9 @@ ext.summary = "Performance tests for Google BigQuery IO 
sources and sinks"
 dependencies {
 testImplementation library.java.google_api_services_bigquery
 testImplementation project(path: ":sdks:java:core", configuration: 
"shadowTest")
-testImplementation project(path: ":sdks:java:testing:test-utils", 
configuration: "testRuntimeMigration")
+testImplementation project(path: ":sdks:java:testing:test-utils")
 testImplementation project(path: 
":sdks:java:extensions:google-cloud-platform-core", configuration: 
"testRuntimeMigration")
-testImplementation project(path: ":sdks:java:testing:test-utils", 
configuration: "testRuntimeMigration")
+testImplementation project(path: ":sdks:java:testing:test-utils")
 testImplementation project(path: ":sdks:java:io:common")
 testImplementation project(path: ":sdks:java:io:google-cloud-platform", 
configuration: "testRuntimeMigration")
 testImplementation project(":sdks:java:io:synthetic")
diff --git a/sdks/java/io/cdap/build.gradle b/sdks/java/io/cdap/build.gradle
index ae07c2e88db..4d823b1ad78 100644
--- a/sdks/java/io/cdap/build.gradle
+++ b/sdks/java/io/cdap/build.gradle
@@ -71,7 +71,7 @@ dependencies {
 testImplementation library.java.postgres
 testImplementation library.java.testcontainers_postgresql
 testImplementation project(path: ":sdks:java:extensions:avro", 
configuration: "testRuntimeMigration")
-testImplementation project(path: ":sdks:java:testing:test-utils", 
configuration: "testRuntimeMigration")
+testImplementation project(path: ":sdks:java:testing:test-utils")
 testImplementation project(path: ":runners:direct-java", configuration: 
"shadow")
 testImplementation project(path: ":sdks:java:io:common")
 }
diff --git 
a/sdks/java/io/cdap/src/test/java/org/apache/beam/sdk/io/cdap/CdapIOIT.java 
b/sdks/java/io/cdap/src/test/java/org/

(beam) branch pr-bot-state updated: Updating config from bot

2024-07-01 Thread github-bot 
This is an automated email from the ASF dual-hosted git repository.

github-bot pushed a commit to branch pr-bot-state
in repository https://gitbox.apache.org/repos/asf/beam.git


The following commit(s) were added to refs/heads/pr-bot-state by this push:
 new e0aa1185b8b Updating config from bot
e0aa1185b8b is described below

commit e0aa1185b8b9062fbb1f060f9d2cb0d37f127467
Author: github-actions 
AuthorDate: Mon Jul 1 19:34:54 2024 +

Updating config from bot
---
 scripts/ci/pr-bot/state/pr-state/pr-31713.json | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/scripts/ci/pr-bot/state/pr-state/pr-31713.json 
b/scripts/ci/pr-bot/state/pr-state/pr-31713.json
index bd9acb373b5..de9cf149311 100644
--- a/scripts/ci/pr-bot/state/pr-state/pr-31713.json
+++ b/scripts/ci/pr-bot/state/pr-state/pr-31713.json
@@ -7,5 +7,5 @@
   "nextAction": "Reviewers",
   "stopReviewerNotifications": false,
   "remindAfterTestsPass": [],
-  "committerAssigned": false
+  "committerAssigned": true
 }
\ No newline at end of file

(beam) branch pr-bot-state updated: Updating config from bot

2024-07-01 Thread github-bot 
This is an automated email from the ASF dual-hosted git repository.

github-bot pushed a commit to branch pr-bot-state
in repository https://gitbox.apache.org/repos/asf/beam.git


The following commit(s) were added to refs/heads/pr-bot-state by this push:
 new 4d76435deeb Updating config from bot
4d76435deeb is described below

commit 4d76435deebcc7a0e222ec2adedbf4924074d191
Author: github-actions 
AuthorDate: Mon Jul 1 18:40:11 2024 +

Updating config from bot
---
 scripts/ci/pr-bot/state/reviewers-for-label-java.json | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/scripts/ci/pr-bot/state/reviewers-for-label-java.json 
b/scripts/ci/pr-bot/state/reviewers-for-label-java.json
index 1d94659aae2..0f4b175c357 100644
--- a/scripts/ci/pr-bot/state/reviewers-for-label-java.json
+++ b/scripts/ci/pr-bot/state/reviewers-for-label-java.json
@@ -9,6 +9,6 @@
 "Abacn": 1719527672291,
 "bvolpato": 1712595969392,
 "m-trieu": 1719617691716,
-"damondouglas": 1719603312919
+"damondouglas": 1719859209669
   }
 }
\ No newline at end of file

(beam) branch pr-bot-state updated: Updating config from bot

2024-07-01 Thread github-bot 
This is an automated email from the ASF dual-hosted git repository.

github-bot pushed a commit to branch pr-bot-state
in repository https://gitbox.apache.org/repos/asf/beam.git


The following commit(s) were added to refs/heads/pr-bot-state by this push:
 new b732112d8ed Updating config from bot
b732112d8ed is described below

commit b732112d8ed3aacbd4f931818d3a7f60502561aa
Author: github-actions 
AuthorDate: Mon Jul 1 18:40:13 2024 +

Updating config from bot
---
 scripts/ci/pr-bot/state/reviewers-for-label-io.json | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/scripts/ci/pr-bot/state/reviewers-for-label-io.json 
b/scripts/ci/pr-bot/state/reviewers-for-label-io.json
index 25746ede985..530f6457f64 100644
--- a/scripts/ci/pr-bot/state/reviewers-for-label-io.json
+++ b/scripts/ci/pr-bot/state/reviewers-for-label-io.json
@@ -5,7 +5,7 @@
 "johnjcasey": 1719617691716,
 "pabloem": 1691787951165,
 "Abacn": 1719663271356,
-"ahmedabu98": 1719515184454,
+"ahmedabu98": 1719859209669,
 "bvolpato": 1712595969392,
 "manavgarg": 1690826779210,
 "damondouglas": 1719857226743,

(beam) branch pr-bot-state updated: Updating config from bot

2024-07-01 Thread github-bot 
This is an automated email from the ASF dual-hosted git repository.

github-bot pushed a commit to branch pr-bot-state
in repository https://gitbox.apache.org/repos/asf/beam.git


The following commit(s) were added to refs/heads/pr-bot-state by this push:
 new f591b0317b8 Updating config from bot
f591b0317b8 is described below

commit f591b0317b84df8190444b87938e5a9252e7f972
Author: github-actions 
AuthorDate: Mon Jul 1 18:40:10 2024 +

Updating config from bot
---
 scripts/ci/pr-bot/state/pr-state/pr-31713.json | 7 +--
 1 file changed, 5 insertions(+), 2 deletions(-)

diff --git a/scripts/ci/pr-bot/state/pr-state/pr-31713.json 
b/scripts/ci/pr-bot/state/pr-state/pr-31713.json
index 242a48d7d3b..bd9acb373b5 100644
--- a/scripts/ci/pr-bot/state/pr-state/pr-31713.json
+++ b/scripts/ci/pr-bot/state/pr-state/pr-31713.json
@@ -1,7 +1,10 @@
 {
   "commentedAboutFailingChecks": true,
-  "reviewersAssignedForLabels": {},
-  "nextAction": "Author",
+  "reviewersAssignedForLabels": {
+"java": "damondouglas",
+"io": "ahmedabu98"
+  },
+  "nextAction": "Reviewers",
   "stopReviewerNotifications": false,
   "remindAfterTestsPass": [],
   "committerAssigned": false

(beam) branch pr-bot-state updated: Updating config from bot

2024-07-01 Thread github-bot 
This is an automated email from the ASF dual-hosted git repository.

github-bot pushed a commit to branch pr-bot-state
in repository https://gitbox.apache.org/repos/asf/beam.git


The following commit(s) were added to refs/heads/pr-bot-state by this push:
 new b1d38add47e Updating config from bot
b1d38add47e is described below

commit b1d38add47e99cfd5eb900cabb0649c79a96e5ce
Author: github-actions 
AuthorDate: Mon Jul 1 18:07:10 2024 +

Updating config from bot
---
 scripts/ci/pr-bot/state/reviewers-for-label-io.json | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/scripts/ci/pr-bot/state/reviewers-for-label-io.json 
b/scripts/ci/pr-bot/state/reviewers-for-label-io.json
index 125bdbef5d8..25746ede985 100644
--- a/scripts/ci/pr-bot/state/reviewers-for-label-io.json
+++ b/scripts/ci/pr-bot/state/reviewers-for-label-io.json
@@ -8,7 +8,7 @@
 "ahmedabu98": 1719515184454,
 "bvolpato": 1712595969392,
 "manavgarg": 1690826779210,
-"damondouglas": 1719512209049,
+"damondouglas": 1719857226743,
 "shunping": 1719603312919
   }
 }
\ No newline at end of file

(beam) branch pr-bot-state updated: Updating config from bot

2024-07-01 Thread github-bot 
This is an automated email from the ASF dual-hosted git repository.

github-bot pushed a commit to branch pr-bot-state
in repository https://gitbox.apache.org/repos/asf/beam.git


The following commit(s) were added to refs/heads/pr-bot-state by this push:
 new fc3aab63404 Updating config from bot
fc3aab63404 is described below

commit fc3aab63404b691fdfcd3dc339c23403d77dda73
Author: github-actions 
AuthorDate: Mon Jul 1 18:07:09 2024 +

Updating config from bot
---
 scripts/ci/pr-bot/state/reviewers-for-label-java.json | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/scripts/ci/pr-bot/state/reviewers-for-label-java.json 
b/scripts/ci/pr-bot/state/reviewers-for-label-java.json
index 5b08f9ffa3e..1d94659aae2 100644
--- a/scripts/ci/pr-bot/state/reviewers-for-label-java.json
+++ b/scripts/ci/pr-bot/state/reviewers-for-label-java.json
@@ -2,7 +2,7 @@
   "label": "java",
   "dateOfLastReviewAssignment": {
 "lukecwik": 1680701852535,
-"kennknowles": 1719515184454,
+"kennknowles": 1719857226742,
 "robertwb": 1719663270733,
 "kileys": 1674428678843,
 "apilloud": 1678822446183,

(beam) branch pr-bot-state updated: Updating config from bot

2024-07-01 Thread github-bot 
This is an automated email from the ASF dual-hosted git repository.

github-bot pushed a commit to branch pr-bot-state
in repository https://gitbox.apache.org/repos/asf/beam.git


The following commit(s) were added to refs/heads/pr-bot-state by this push:
 new 50ca8133c09 Updating config from bot
50ca8133c09 is described below

commit 50ca8133c096114fa1d9c8301bd73ae01f084510
Author: github-actions 
AuthorDate: Mon Jul 1 18:07:07 2024 +

Updating config from bot
---
 scripts/ci/pr-bot/state/pr-state/pr-31738.json | 11 +++
 1 file changed, 11 insertions(+)

diff --git a/scripts/ci/pr-bot/state/pr-state/pr-31738.json 
b/scripts/ci/pr-bot/state/pr-state/pr-31738.json
new file mode 100644
index 000..550d0543a2c
--- /dev/null
+++ b/scripts/ci/pr-bot/state/pr-state/pr-31738.json
@@ -0,0 +1,11 @@
+{
+  "commentedAboutFailingChecks": false,
+  "reviewersAssignedForLabels": {
+"java": "kennknowles",
+"io": "damondouglas"
+  },
+  "nextAction": "Reviewers",
+  "stopReviewerNotifications": false,
+  "remindAfterTestsPass": [],
+  "committerAssigned": false
+}
\ No newline at end of file

(beam) branch pr-bot-state updated: Updating config from bot

2024-07-01 Thread github-bot 
This is an automated email from the ASF dual-hosted git repository.

github-bot pushed a commit to branch pr-bot-state
in repository https://gitbox.apache.org/repos/asf/beam.git


The following commit(s) were added to refs/heads/pr-bot-state by this push:
 new 6f6f3c1b831 Updating config from bot
6f6f3c1b831 is described below

commit 6f6f3c1b83139e191260cbf5bc82d7d661630757
Author: github-actions 
AuthorDate: Mon Jul 1 16:54:09 2024 +

Updating config from bot
---
 scripts/ci/pr-bot/state/pr-state/pr-31726.json | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/scripts/ci/pr-bot/state/pr-state/pr-31726.json 
b/scripts/ci/pr-bot/state/pr-state/pr-31726.json
index 242a48d7d3b..37f7ab41238 100644
--- a/scripts/ci/pr-bot/state/pr-state/pr-31726.json
+++ b/scripts/ci/pr-bot/state/pr-state/pr-31726.json
@@ -2,7 +2,7 @@
   "commentedAboutFailingChecks": true,
   "reviewersAssignedForLabels": {},
   "nextAction": "Author",
-  "stopReviewerNotifications": false,
+  "stopReviewerNotifications": true,
   "remindAfterTestsPass": [],
   "committerAssigned": false
 }
\ No newline at end of file

(beam) branch master updated: Add scaffold CsvIOParseHelpers (#31720)

2024-07-01 Thread johncasey 
This is an automated email from the ASF dual-hosted git repository.

johncasey pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/beam.git


The following commit(s) were added to refs/heads/master by this push:
 new a3a62b866e0 Add scaffold CsvIOParseHelpers (#31720)
a3a62b866e0 is described below

commit a3a62b866e0c1ac75c662b3f633d775cac64ffdb
Author: Francis O'Hara 
AuthorDate: Mon Jul 1 09:28:11 2024 -0700

Add scaffold CsvIOParseHelpers (#31720)
---
 .../apache/beam/sdk/io/csv/CsvIOParseHelpers.java  | 55 ++
 1 file changed, 55 insertions(+)

diff --git 
a/sdks/java/io/csv/src/main/java/org/apache/beam/sdk/io/csv/CsvIOParseHelpers.java
 
b/sdks/java/io/csv/src/main/java/org/apache/beam/sdk/io/csv/CsvIOParseHelpers.java
new file mode 100644
index 000..042e284cd52
--- /dev/null
+++ 
b/sdks/java/io/csv/src/main/java/org/apache/beam/sdk/io/csv/CsvIOParseHelpers.java
@@ -0,0 +1,55 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.beam.sdk.io.csv;
+
+import java.util.ArrayList;
+import java.util.List;
+import org.apache.beam.sdk.schemas.Schema;
+import org.apache.commons.csv.CSVFormat;
+
+/** A utility class containing shared methods for parsing CSV records. */
+final class CsvIOParseHelpers {
+  /** Validate the {@link CSVFormat} for CSV record parsing requirements. */
+  // TODO(https://github.com/apache/beam/issues/31712): implement method.
+  static void validate(CSVFormat format) {}
+
+  /**
+   * Validate the {@link CSVFormat} in relation to the {@link Schema} for CSV 
record parsing
+   * requirements.
+   */
+  // TODO(https://github.com/apache/beam/issues/31716): implement method.
+  static void validate(CSVFormat format, Schema schema) {}
+
+  /**
+   * Build a {@link List} of {@link Schema.Field}s corresponding to the 
expected position of each
+   * field within the CSV record.
+   */
+  // TODO(https://github.com/apache/beam/issues/31718): implement method.
+  static List mapFieldPositions(CSVFormat format, Schema schema) 
{
+return new ArrayList<>();
+  }
+
+  /**
+   * Parse the given {@link String} cell of the CSV record based on the given 
field's {@link
+   * Schema.FieldType}.
+   */
+  // TODO(https://github.com/apache/beam/issues/31719): implement method.
+  static Object parseCell(String cell, Schema.Field field) {
+return "";
+  }
+}

(beam) branch pr-bot-state updated: Updating config from bot

2024-07-01 Thread github-bot 
This is an automated email from the ASF dual-hosted git repository.

github-bot pushed a commit to branch pr-bot-state
in repository https://gitbox.apache.org/repos/asf/beam.git


The following commit(s) were added to refs/heads/pr-bot-state by this push:
 new 02cfbe833ca Updating config from bot
02cfbe833ca is described below

commit 02cfbe833ca5ffc8e45b69e388f2a63cb6a35c2c
Author: github-actions 
AuthorDate: Mon Jul 1 14:14:53 2024 +

Updating config from bot
---
 scripts/ci/pr-bot/state/pr-state/pr-31702.json | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/scripts/ci/pr-bot/state/pr-state/pr-31702.json 
b/scripts/ci/pr-bot/state/pr-state/pr-31702.json
index e60e30c39af..448ebc9fcac 100644
--- a/scripts/ci/pr-bot/state/pr-state/pr-31702.json
+++ b/scripts/ci/pr-bot/state/pr-state/pr-31702.json
@@ -5,7 +5,7 @@
 "io": "ahmedabu98"
   },
   "nextAction": "Reviewers",
-  "stopReviewerNotifications": false,
+  "stopReviewerNotifications": true,
   "remindAfterTestsPass": [],
   "committerAssigned": false
 }
\ No newline at end of file

(beam) branch users/damccorm/cachePoisoning updated (915887cb6f8 -> 656a154967b)

2024-07-01 Thread damccorm 
This is an automated email from the ASF dual-hosted git repository.

damccorm pushed a change to branch users/damccorm/cachePoisoning
in repository https://gitbox.apache.org/repos/asf/beam.git


from 915887cb6f8 Dont read from cache in sensitive workflows
 add 656a154967b Disable actions cache

No new revisions were added by this update.

Summary of changes:
 .github/workflows/beam_Release_NightlySnapshot.yml   | 1 +
 .github/workflows/deploy_release_candidate_pypi.yaml | 1 +
 .github/workflows/finalize_release.yml   | 1 +
 3 files changed, 3 insertions(+)

(beam) branch pr-bot-state updated: Updating config from bot

2024-07-01 Thread github-bot 
This is an automated email from the ASF dual-hosted git repository.

github-bot pushed a commit to branch pr-bot-state
in repository https://gitbox.apache.org/repos/asf/beam.git


The following commit(s) were added to refs/heads/pr-bot-state by this push:
 new 8dbd96381fb Updating config from bot
8dbd96381fb is described below

commit 8dbd96381fb18adfa5f6ebcab1051c39cf78bbc3
Author: github-actions 
AuthorDate: Mon Jul 1 14:16:28 2024 +

Updating config from bot
---
 scripts/ci/pr-bot/state/pr-state/pr-31734.json | 8 
 1 file changed, 8 insertions(+)

diff --git a/scripts/ci/pr-bot/state/pr-state/pr-31734.json 
b/scripts/ci/pr-bot/state/pr-state/pr-31734.json
new file mode 100644
index 000..9c2aa5aa212
--- /dev/null
+++ b/scripts/ci/pr-bot/state/pr-state/pr-31734.json
@@ -0,0 +1,8 @@
+{
+  "commentedAboutFailingChecks": false,
+  "reviewersAssignedForLabels": {},
+  "nextAction": "Author",
+  "stopReviewerNotifications": true,
+  "remindAfterTestsPass": [],
+  "committerAssigned": false
+}
\ No newline at end of file

(beam) branch master updated (bae6fcfcb49 -> 47384772cac)

2024-07-01 Thread damccorm 
This is an automated email from the ASF dual-hosted git repository.

damccorm pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/beam.git


from bae6fcfcb49 Add a unit test for MakePipelineOptionsFileAndEnvVar 
(#31732)
 add 47384772cac Dont read from cache in sensitive workflows (#31734)

No new revisions were added by this update.

Summary of changes:
 .github/workflows/beam_Publish_Website.yml   | 2 --
 .github/workflows/beam_Release_NightlySnapshot.yml   | 3 +--
 .github/workflows/deploy_release_candidate_pypi.yaml | 3 +--
 .github/workflows/finalize_release.yml   | 3 +--
 4 files changed, 3 insertions(+), 8 deletions(-)

(beam) branch users/damccorm/cachePoisoning deleted (was 656a154967b)

2024-07-01 Thread damccorm 
This is an automated email from the ASF dual-hosted git repository.

damccorm pushed a change to branch users/damccorm/cachePoisoning
in repository https://gitbox.apache.org/repos/asf/beam.git


 was 656a154967b Disable actions cache

The revisions that were on this branch are still contained in
other references; therefore, this change does not discard any commits
from the repository.

(beam) branch users/damccorm/cachePoisoning created (now 915887cb6f8)

2024-07-01 Thread damccorm 
This is an automated email from the ASF dual-hosted git repository.

damccorm pushed a change to branch users/damccorm/cachePoisoning
in repository https://gitbox.apache.org/repos/asf/beam.git


  at 915887cb6f8 Dont read from cache in sensitive workflows

No new revisions were added by this update.

(beam) branch master updated: Add a unit test for MakePipelineOptionsFileAndEnvVar (#31732)

2024-07-01 Thread jrmccluskey 
This is an automated email from the ASF dual-hosted git repository.

jrmccluskey pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/beam.git


The following commit(s) were added to refs/heads/master by this push:
 new bae6fcfcb49 Add a unit test for MakePipelineOptionsFileAndEnvVar 
(#31732)
bae6fcfcb49 is described below

commit bae6fcfcb492db64e8b7333d928fdbecdcb3e21f
Author: ljjulia 
AuthorDate: Mon Jul 1 06:20:33 2024 -0700

Add a unit test for MakePipelineOptionsFileAndEnvVar (#31732)

* Add unit test for MakePipelineOptionsFileAndEnvVar

* Add coverage for validating options string is JSON
---
 sdks/go/container/tools/pipeline_options.go|  5 +++
 ...ipeline_options.go => pipeline_options_test.go} | 49 +++---
 2 files changed, 39 insertions(+), 15 deletions(-)

diff --git a/sdks/go/container/tools/pipeline_options.go 
b/sdks/go/container/tools/pipeline_options.go
index 7b46d8fa8c8..026fb31b099 100644
--- a/sdks/go/container/tools/pipeline_options.go
+++ b/sdks/go/container/tools/pipeline_options.go
@@ -16,6 +16,7 @@
 package tools
 
 import (
+   "encoding/json"
"fmt"
"os"
 )
@@ -31,6 +32,10 @@ func MakePipelineOptionsFileAndEnvVar(options string) error {
return fmt.Errorf("unable to create %v: %w", fn, err)
}
defer f.Close()
+   var js map[string]interface{}
+   if json.Unmarshal([]byte(options), &js) != nil {
+   return fmt.Errorf("options string is not JSON formatted %v", 
options)
+   }
if _, err := f.WriteString(options); err != nil {
return fmt.Errorf("error writing %v: %w", f.Name(), err)
}
diff --git a/sdks/go/container/tools/pipeline_options.go 
b/sdks/go/container/tools/pipeline_options_test.go
similarity index 54%
copy from sdks/go/container/tools/pipeline_options.go
copy to sdks/go/container/tools/pipeline_options_test.go
index 7b46d8fa8c8..7a0d7ebd5f0 100644
--- a/sdks/go/container/tools/pipeline_options.go
+++ b/sdks/go/container/tools/pipeline_options_test.go
@@ -16,24 +16,43 @@
 package tools
 
 import (
-   "fmt"
"os"
+   "testing"
 )
 
-// MakePipelineOptionsFileAndEnvVar writes the pipeline options to a file.
-// Assumes the options string is JSON formatted.
-//
-// Stores the file name in question in PIPELINE_OPTIONS_FILE for access by the 
SDK.
-func MakePipelineOptionsFileAndEnvVar(options string) error {
-   fn := "pipeline_options.json"
-   f, err := os.Create(fn)
-   if err != nil {
-   return fmt.Errorf("unable to create %v: %w", fn, err)
+func TestMakePipelineOptionsFileAndEnvVar(t *testing.T) {
+   tests := []struct {
+   name  string
+   inputOptions  string
+   expectedError string
+   }{
+   {
+   "empty options",
+   "{}",
+   "",
+   },
+   {
+   "valid options",
+   "{\"abc\": 123}",
+   "",
+   },
+   {
+   "invalid options",
+   "{4}",
+   "options string is not JSON formatted {4}",
+   },
}
-   defer f.Close()
-   if _, err := f.WriteString(options); err != nil {
-   return fmt.Errorf("error writing %v: %w", f.Name(), err)
+
+   for _, test := range tests {
+   t.Run(test.name, func(t *testing.T) {
+   t.Cleanup(os.Clearenv)
+   err := 
MakePipelineOptionsFileAndEnvVar(test.inputOptions)
+   if err != nil {
+   if got, want := err.Error(), 
test.expectedError; got != want {
+   t.Errorf("got error: %v, want error: 
%v", got, want)
+   }
+   }
+   })
}
-   os.Setenv("PIPELINE_OPTIONS_FILE", f.Name())
-   return nil
+   os.Remove("pipeline_options.json")
 }

(beam) branch master updated: [Dataflow Streaming] Enabled Heartbeat by Default (#31689)

2024-07-01 Thread scwhittle 
This is an automated email from the ASF dual-hosted git repository.

scwhittle pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/beam.git


The following commit(s) were added to refs/heads/master by this push:
 new ef5060416d9 [Dataflow Streaming] Enabled Heartbeat by Default (#31689)
ef5060416d9 is described below

commit ef5060416d9fed2e08a6682e69657c6fa9f98af4
Author: TongruiLi <12992126+tongru...@users.noreply.github.com>
AuthorDate: Mon Jul 1 06:04:18 2024 -0700

[Dataflow Streaming] Enabled Heartbeat by Default (#31689)
---
 .../dataflow/worker/StreamingDataflowWorker.java   |  4 +--
 .../windmill/client/grpc/GrpcWindmillServer.java   |  4 +--
 .../client/grpc/GrpcWindmillServerTest.java| 37 +-
 3 files changed, 19 insertions(+), 26 deletions(-)

diff --git 
a/runners/google-cloud-dataflow-java/worker/src/main/java/org/apache/beam/runners/dataflow/worker/StreamingDataflowWorker.java
 
b/runners/google-cloud-dataflow-java/worker/src/main/java/org/apache/beam/runners/dataflow/worker/StreamingDataflowWorker.java
index fc1be2cd137..59819db88a0 100644
--- 
a/runners/google-cloud-dataflow-java/worker/src/main/java/org/apache/beam/runners/dataflow/worker/StreamingDataflowWorker.java
+++ 
b/runners/google-cloud-dataflow-java/worker/src/main/java/org/apache/beam/runners/dataflow/worker/StreamingDataflowWorker.java
@@ -602,8 +602,8 @@ public class StreamingDataflowWorker {
 
.setStreamingRpcBatchLimit(options.getWindmillServiceStreamingRpcBatchLimit())
 .setSendKeyedGetDataRequests(
 !options.isEnableStreamingEngine()
-|| !DataflowRunner.hasExperiment(
-options, "streaming_engine_send_new_heartbeat_requests"));
+|| DataflowRunner.hasExperiment(
+options, 
"streaming_engine_disable_new_heartbeat_requests"));
   }
 
   private static BoundedQueueExecutor 
createWorkUnitExecutor(DataflowWorkerHarnessOptions options) {
diff --git 
a/runners/google-cloud-dataflow-java/worker/src/main/java/org/apache/beam/runners/dataflow/worker/windmill/client/grpc/GrpcWindmillServer.java
 
b/runners/google-cloud-dataflow-java/worker/src/main/java/org/apache/beam/runners/dataflow/worker/windmill/client/grpc/GrpcWindmillServer.java
index abf85d98548..0ab03a80318 100644
--- 
a/runners/google-cloud-dataflow-java/worker/src/main/java/org/apache/beam/runners/dataflow/worker/windmill/client/grpc/GrpcWindmillServer.java
+++ 
b/runners/google-cloud-dataflow-java/worker/src/main/java/org/apache/beam/runners/dataflow/worker/windmill/client/grpc/GrpcWindmillServer.java
@@ -176,8 +176,8 @@ public final class GrpcWindmillServer extends 
WindmillServerStub {
 testOptions(/* enableStreamingEngine= */ true, experiments);
 boolean sendKeyedGetDataRequests =
 !testOptions.isEnableStreamingEngine()
-|| !DataflowRunner.hasExperiment(
-testOptions, "streaming_engine_send_new_heartbeat_requests");
+|| DataflowRunner.hasExperiment(
+testOptions, 
"streaming_engine_disable_new_heartbeat_requests");
 GrpcWindmillStreamFactory windmillStreamFactory =
 GrpcWindmillStreamFactory.of(createJobHeader(testOptions, clientId))
 .setSendKeyedGetDataRequests(sendKeyedGetDataRequests)
diff --git 
a/runners/google-cloud-dataflow-java/worker/src/test/java/org/apache/beam/runners/dataflow/worker/windmill/client/grpc/GrpcWindmillServerTest.java
 
b/runners/google-cloud-dataflow-java/worker/src/test/java/org/apache/beam/runners/dataflow/worker/windmill/client/grpc/GrpcWindmillServerTest.java
index b1d5309e12d..6473d5527a8 100644
--- 
a/runners/google-cloud-dataflow-java/worker/src/test/java/org/apache/beam/runners/dataflow/worker/windmill/client/grpc/GrpcWindmillServerTest.java
+++ 
b/runners/google-cloud-dataflow-java/worker/src/test/java/org/apache/beam/runners/dataflow/worker/windmill/client/grpc/GrpcWindmillServerTest.java
@@ -124,8 +124,16 @@ public class GrpcWindmillServerTest {
 
   @Before
   public void setUp() throws Exception {
-String name = "Fake server for " + getClass();
+startServerAndClient(new ArrayList<>());
+  }
+
+  @After
+  public void tearDown() throws Exception {
+server.shutdownNow();
+  }
 
+  private void startServerAndClient(List experiments) throws Exception 
{
+String name = "Fake server for " + getClass();
 this.server =
 InProcessServerBuilder.forName(name)
 .fallbackHandlerRegistry(serviceRegistry)
@@ -136,17 +144,12 @@ public class GrpcWindmillServerTest {
 this.client =
 GrpcWindmillServer.newTestInstance(
 name,
-new ArrayList<>(),
+experiments,
 clientId,
 new FakeWindmillStubFactory(
 () -> 
grpcCleanup.register(WindmillChannelFactory.inProcessChannel(name;
   }
 
-  @After
-  public void tearDown() throws Exception {
-server.shutdownNow

(beam) branch pr-bot-state updated: Updating config from bot

2024-07-01 Thread github-bot 
This is an automated email from the ASF dual-hosted git repository.

github-bot pushed a commit to branch pr-bot-state
in repository https://gitbox.apache.org/repos/asf/beam.git


The following commit(s) were added to refs/heads/pr-bot-state by this push:
 new 3de74e42815 Updating config from bot
3de74e42815 is described below

commit 3de74e42815e7d2c629bd532dbf177f74dd03b4b
Author: github-actions 
AuthorDate: Mon Jul 1 11:21:48 2024 +

Updating config from bot
---
 scripts/ci/pr-bot/state/pr-state/pr-31733.json | 8 
 1 file changed, 8 insertions(+)

diff --git a/scripts/ci/pr-bot/state/pr-state/pr-31733.json 
b/scripts/ci/pr-bot/state/pr-state/pr-31733.json
new file mode 100644
index 000..9c2aa5aa212
--- /dev/null
+++ b/scripts/ci/pr-bot/state/pr-state/pr-31733.json
@@ -0,0 +1,8 @@
+{
+  "commentedAboutFailingChecks": false,
+  "reviewersAssignedForLabels": {},
+  "nextAction": "Author",
+  "stopReviewerNotifications": true,
+  "remindAfterTestsPass": [],
+  "committerAssigned": false
+}
\ No newline at end of file

(beam) branch pr-bot-state updated: Updating config from bot

2024-07-01 Thread github-bot 
This is an automated email from the ASF dual-hosted git repository.

github-bot pushed a commit to branch pr-bot-state
in repository https://gitbox.apache.org/repos/asf/beam.git


The following commit(s) were added to refs/heads/pr-bot-state by this push:
 new 7da96125c54 Updating config from bot
7da96125c54 is described below

commit 7da96125c547a114672960480fe1a3d10b3d612e
Author: github-actions 
AuthorDate: Mon Jul 1 08:07:45 2024 +

Updating config from bot
---
 scripts/ci/pr-bot/state/reviewers-for-label-go.json | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/scripts/ci/pr-bot/state/reviewers-for-label-go.json 
b/scripts/ci/pr-bot/state/reviewers-for-label-go.json
index de3c4d2ebc2..1835f5e0cc7 100644
--- a/scripts/ci/pr-bot/state/reviewers-for-label-go.json
+++ b/scripts/ci/pr-bot/state/reviewers-for-label-go.json
@@ -2,7 +2,7 @@
   "label": "go",
   "dateOfLastReviewAssignment": {
 "lostluck": 1719551147616,
-"jrmccluskey": 1719513350917,
+"jrmccluskey": 1719821262820,
 "youngoli": 1657688896155,
 "damccorm": 1680501930289,
 "riteshghorse": 1718602587209

(beam) branch pr-bot-state updated: Updating config from bot

2024-07-01 Thread github-bot 
This is an automated email from the ASF dual-hosted git repository.

github-bot pushed a commit to branch pr-bot-state
in repository https://gitbox.apache.org/repos/asf/beam.git


The following commit(s) were added to refs/heads/pr-bot-state by this push:
 new bc9032017f9 Updating config from bot
bc9032017f9 is described below

commit bc9032017f9cda906bdafdb9ca1b0b7b9bb39d87
Author: github-actions 
AuthorDate: Mon Jul 1 08:07:44 2024 +

Updating config from bot
---
 scripts/ci/pr-bot/state/pr-state/pr-31732.json | 10 ++
 1 file changed, 10 insertions(+)

diff --git a/scripts/ci/pr-bot/state/pr-state/pr-31732.json 
b/scripts/ci/pr-bot/state/pr-state/pr-31732.json
new file mode 100644
index 000..1c2c2e31975
--- /dev/null
+++ b/scripts/ci/pr-bot/state/pr-state/pr-31732.json
@@ -0,0 +1,10 @@
+{
+  "commentedAboutFailingChecks": false,
+  "reviewersAssignedForLabels": {
+"go": "jrmccluskey"
+  },
+  "nextAction": "Reviewers",
+  "stopReviewerNotifications": false,
+  "remindAfterTestsPass": [],
+  "committerAssigned": false
+}
\ No newline at end of file