This is an automated email from the ASF dual-hosted git repository.
github-bot pushed a change to branch
dependabot/github_actions/actions/setup-java-3.8.0
in repository https://gitbox.apache.org/repos/asf/beam.git
discard 764041ce39c Bump actions/setup-java from 3.6.0 to 3.8.0
add da100f967a6 Migrate testing subpackages from interface{} to any
(#24570)
add 6c24637b451 fix go lints (#24566)
add 11ed0e1857f Samza runner support for non unique stateId across
multiple ParDos (#24276)
add e0c494d9b57 Suppress keyfor warnings
add a22e204fd32 Suppress checker warnings that are confusing and difficult
add efa8a08549a Add @Pure annotations to MongoDbIO autovalue fields
add 186c37fe121 Suppress checker in FnApiDoFnRunner due to crash
add 3ae4efe3b64 Suppress checker framework in Dataflow
FetchAndFilterStreamingSideInputOperation due to crash
add f32c50d35eb Fix some nullness errors in Spark runner
add eb39a1b41fe Upgrade checker framework to 3.27.0
add de16b05f361 Merge pull request #24410: Upgrade checker framework to
3.27.0
add fb06ef3abc8 Bump to Hadoop 3.3.4 for performance tests (#24550)
add 4c0253d8009 pubsub: fix typo in grpc client factory
add 46055564fcc Merge pull request #24398 from Illedran/patch-1
add e233ef4d2a6 regenerate python dependencies (#24582)
add 80980b8be48 Return empty splits if unable to split, not errors (#24508)
add a40e79b16f7 [Tour Of Beam] Playground Router GRPC API host (#24542)
add d1a29704529 Bump golang.org/x/net from 0.3.0 to 0.4.0 in /sdks (#24587)
add dd7a581188e Replaced finalize with DoFn Teardown in Neo4jIO (#24571)
add da3af55f1e3 Simplify bug report templates
add 73e99b664ec Merge pull request #24585: Simplify issue report templates
add 495e1e0a309 Fix bugs in issue template yml
add 0ddf1f7ea8a Merge pull request #24595: Fix bugs in issue template yml
add 7f5ea613012 Fix issue templates (#24597)
add c0de849c1ce [#24024] Stop wrapping light weight functions with
Contextful as they add a lot of overhead for functions that are meant to do
almost no work. (#24025)
add 27c355c534b Sample window size as well (#24388)
add 00bf2d42eac Implement Kafka Write Schema Transform (#24495)
add 25365bde72f Exclude IOs already split from Java Precommit job
add f7c9f85f365 Move expansion services into appropriate precommits
add cd06ee187c5 Split more IOs out of Java precommit
add b4c33ca2dcb Fix trigger paths for separated IOs
add e8814846f5b Merge pull request #24584: Split java precommit more
add 90d280b43e6 Eliminate null errors from JdbcIO
add 451378851ab Merge pull request #22862: Eliminate null errors from
JdbcIO
add b8a335be8ab docs(fix): Filter.whereFieldName(s?) ->
Filter.create().whereFieldName(s?) (#24576)
add d523a536a08 ElasticsearchIO: Lower log level in flushBatch to avoid
noisy log
add ba38e41c702 Merge pull request #24604 from andthezhang/master
add a9cba697e05 Turn rawtype checking back on for core Java SDK
add 58b4d46655d Merge pull request #24574: Turn rawtype checking back on
for core Java SDK
add 884673b4fb6 interface{} -> any for registration (#24600)
add f57c7b94a7a Restrict tox to be in 3.x version (#24601)
add 816f3515c4a [Playground] support for Kafka-enabled examples (#24459)
add 46868c6153a Bump actions/setup-java from 3.6.0 to 3.8.0
This update added new revisions after undoing existing revisions.
That is to say, some revisions that were in the old version of the
branch are not in the new version. This situation occurs
when a user --force pushes a change and generates a repository
containing something like this:
* -- * -- B -- O -- O -- O (764041ce39c)
\
N -- N -- N
refs/heads/dependabot/github_actions/actions/setup-java-3.8.0 (46868c6153a)
You should already have received notification emails for all of the O
revisions, and so the following emails describe only the N revisions
from the common base, B.
Any revisions marked "omit" are not gone; other references still
refer to them. Any revisions marked "discard" are gone forever.
No new revisions were added by this update.
Summary of changes:
.github/ISSUE_TEMPLATE/bug.yml | 137 +-
.github/ISSUE_TEMPLATE/failing_test.yml | 136 +-
.github/ISSUE_TEMPLATE/feature.yml | 134 +-
.github/ISSUE_TEMPLATE/task.yml | 135 +-
.github/issue-rules.yml | 345 +---
.github/workflows/playground_examples_ci.yml | 2 +-
.../workflows/playground_examples_ci_reusable.yml | 18 +-
.github/workflows/python_tests.yml | 2 +-
.../workflows/tour_of_beam_backend_integration.yml | 2 +-
.../job_PerformanceTests_FileBasedIO_IT.groovy | 4 +-
.test-infra/jenkins/job_PreCommit_Java.groovy | 37 +-
.../job_PreCommit_Java_AWS2_IO_Direct.groovy | 2 +
.../job_PreCommit_Java_AWS_IO_Direct.groovy | 3 +
...vy => job_PreCommit_Java_Amqp_IO_Direct.groovy} | 6 +-
...y => job_PreCommit_Java_Azure_IO_Direct.groovy} | 6 +-
.../job_PreCommit_Java_Cassandra_IO_Direct.groovy | 1 +
...vy => job_PreCommit_Java_Cdap_IO_Direct.groovy} | 9 +-
.../job_PreCommit_Java_Debezium_IO_Direct.groovy | 4 +
..._PreCommit_Java_ElasticSearch_IO_Direct.groovy} | 13 +-
.../job_PreCommit_Java_GCP_IO_Direct.groovy | 9 +
...> job_PreCommit_Java_HCatalog_IO_Direct.groovy} | 8 +-
... => job_PreCommit_Java_Hadoop_IO_Direct.groovy} | 13 +-
...y => job_PreCommit_Java_Hbase_IO_Direct.groovy} | 8 +-
...> job_PreCommit_Java_InfluxDb_IO_Direct.groovy} | 7 +-
.../job_PreCommit_Java_JDBC_IO_Direct.groovy | 2 +
...ovy => job_PreCommit_Java_Jms_IO_Direct.groovy} | 6 +-
.../job_PreCommit_Java_Kafka_IO_Direct.groovy | 6 +
.../job_PreCommit_Java_Kinesis_IO_Direct.groovy | 2 +
...vy => job_PreCommit_Java_Kudu_IO_Direct.groovy} | 7 +-
...=> job_PreCommit_Java_MongoDb_IO_Direct.groovy} | 8 +-
...vy => job_PreCommit_Java_Mqtt_IO_Direct.groovy} | 7 +-
.../job_PreCommit_Java_Neo4j_IO_Direct.groovy | 2 +
...=> job_PreCommit_Java_Parquet_IO_Direct.groovy} | 7 +-
...> job_PreCommit_Java_RabbitMq_IO_Direct.groovy} | 7 +-
...y => job_PreCommit_Java_Redis_IO_Direct.groovy} | 7 +-
...ob_PreCommit_Java_Singlestore_IO_Direct.groovy} | 8 +-
... job_PreCommit_Java_Snowflake_IO_Direct.groovy} | 10 +-
...vy => job_PreCommit_Java_Solr_IO_Direct.groovy} | 7 +-
... => job_PreCommit_Java_Splunk_IO_Direct.groovy} | 6 +-
... => job_PreCommit_Java_Thrift_IO_Direct.groovy} | 6 +-
...vy => job_PreCommit_Java_Tika_IO_Direct.groovy} | 6 +-
.../LargeITCluster/hdfs-multi-datanode-cluster.yml | 4 +-
build.gradle.kts | 35 +-
.../org/apache/beam/gradle/BeamModulePlugin.groovy | 4 +-
.../apache/beam/examples/KafkaWordCountAvro.java | 120 ++
.../apache/beam/examples/KafkaWordCountJson.java | 120 ++
learning/tour-of-beam/backend/README.md | 18 +-
playground/api/v1/api.proto | 13 +
playground/backend/CONTRIBUTE.md | 30 +-
playground/backend/cmd/server/controller.go | 30 +-
playground/backend/cmd/server/controller_test.go | 85 +-
playground/backend/containers/java/Dockerfile | 26 +-
playground/backend/containers/router/Dockerfile | 19 +-
playground/backend/datasets/CountWords.avro | Bin 0 -> 888 bytes
playground/backend/datasets/CountWords.json | 22 +
playground/backend/go.mod | 53 +-
playground/backend/go.sum | 350 +++-
playground/backend/internal/api/v1/api.pb.go | 1221 +++++++------
playground/backend/internal/api/v1/api_grpc.pb.go | 17 -
.../internal/code_processing/code_processing.go | 25 +-
.../code_processing/code_processing_test.go | 10 +-
.../internal/constants/datastore_constants.go | 1 +
.../snippet.go => constants/emulator_constants.go} | 10 +-
.../backend/internal/db/datastore/datastore_db.go | 74 +-
.../backend/internal/db/dto/precompiled_object.go | 27 +-
.../internal/db/dto/precompiled_object_test.go | 219 +++
playground/backend/internal/db/dto/snippet.go | 10 +
playground/backend/internal/db/entity/snippet.go | 35 +-
playground/backend/internal/db/mapper/mapper.go | 1 +
.../db/mapper/precompiled_object_mapper.go | 63 +-
.../db/mapper/precompiled_object_mapper_test.go | 82 +-
playground/backend/internal/emulators/emulator.go | 106 ++
playground/backend/internal/emulators/kafka.go | 181 ++
.../backend/internal/executors/executor_test.go | 10 +-
playground/backend/internal/fs_tool/fs.go | 6 +-
playground/backend/internal/fs_tool/fs_test.go | 8 +-
.../internal/preparers/go_preparers_test.go | 7 +-
.../backend/internal/preparers/java_preparers.go | 45 +-
.../internal/preparers/java_preparers_test.go | 23 +-
playground/backend/internal/preparers/preparer.go | 14 +-
.../backend/internal/preparers/preparer_test.go | 53 +-
.../internal/preparers/python_preparers_test.go | 10 +-
.../internal/preparers/scio_preparers_test.go | 7 +-
.../internal/setup_tools/builder/setup_builder.go | 13 +-
.../setup_tools/builder/setup_builder_test.go | 26 +-
.../setup_tools/life_cycle/life_cycle_setuper.go | 27 +-
.../life_cycle/life_cycle_setuper_test.go | 12 +-
.../backend/internal/tests/test_data/test_data.go | 29 +-
.../backend/internal/utils/datastore_utils.go | 5 +
playground/backend/internal/utils/file_utils.go | 3 +-
.../backend/internal/utils/file_utils_test.go | 5 +
.../lib/src/api/v1/api.pb.dart | 119 +-
.../lib/src/api/v1/api.pbenum.dart | 35 +-
.../lib/src/api/v1/api.pbgrpc.dart | 20 +-
.../lib/src/api/v1/api.pbjson.dart | 63 +-
playground/infrastructure/api/v1/api_pb2.py | 1889 +++-----------------
playground/infrastructure/cd_helper.py | 7 +-
playground/infrastructure/checker.py | 4 +-
playground/infrastructure/ci_helper.py | 5 +-
playground/infrastructure/config.py | 37 +-
playground/infrastructure/datastore_client.py | 267 +--
playground/infrastructure/grpc_client.py | 8 +-
.../templates/deployment-router-grpc.yml | 48 +
.../templates/service-router-grpc.yml} | 35 +-
playground/infrastructure/helper.py | 95 +-
playground/infrastructure/proxy/allow_list.py | 6 +-
.../{proxy/allow_list.py => repository.py} | 36 +-
playground/infrastructure/requirements.txt | 1 +
playground/infrastructure/test_datastore_client.py | 43 +-
playground/infrastructure/test_grpc_client.py | 2 +-
playground/infrastructure/test_helper.py | 381 +++-
playground/infrastructure/test_repository.py | 53 +
.../google-cloud-dataflow-java/worker/build.gradle | 1 +
.../worker/legacy-worker/build.gradle | 1 +
.../runners/fnexecution/wire/CommonCoderTest.java | 6 +-
runners/samza/build.gradle | 4 -
.../org/apache/beam/runners/samza/SamzaRunner.java | 8 +-
.../apache/beam/runners/samza/runtime/DoFnOp.java | 6 +-
.../runners/samza/runtime/SamzaDoFnRunners.java | 10 +-
.../samza/runtime/SamzaStoreStateInternals.java | 40 +-
.../runners/samza/translation/ConfigBuilder.java | 5 +-
.../runners/samza/translation/ConfigContext.java | 13 +-
.../translation/ParDoBoundMultiTranslator.java | 32 +-
.../translation/PortableTranslationContext.java | 2 +-
.../samza/translation/SamzaPipelineTranslator.java | 4 +-
.../runners/samza/translation/StateIdParser.java | 69 +
.../samza/translation/TranslationContext.java | 8 +
.../samza/util/SamzaPipelineTranslatorUtils.java | 4 +-
.../beam/runners/samza/util/StoreIdGenerator.java | 48 +
.../runtime/SamzaStoreStateInternalsTest.java | 102 ++
.../samza/translation/ConfigGeneratorTest.java | 146 +-
.../samza/translation/TranslationContextTest.java | 6 +-
.../translation/batch/Aggregators.java | 18 +-
.../batch/functions/CachedSideInputReader.java | 4 +-
.../runners/spark/SparkRunnerDebuggerTest.java | 11 +-
sdks/go.mod | 2 +-
sdks/go.sum | 4 +-
sdks/go/pkg/beam/core/graph/xlang.go | 2 +-
sdks/go/pkg/beam/core/runtime/exec/datasource.go | 6 +-
.../pkg/beam/core/runtime/exec/datasource_test.go | 32 +-
.../go/pkg/beam/core/runtime/exec/dynsplit_test.go | 20 +-
sdks/go/pkg/beam/core/runtime/exec/plan.go | 13 +-
sdks/go/pkg/beam/core/runtime/harness/harness.go | 13 +-
sdks/go/pkg/beam/core/typex/special.go | 12 +-
sdks/go/pkg/beam/register/emitter.go | 8 +-
sdks/go/pkg/beam/register/iter.go | 4 +-
sdks/go/pkg/beam/register/register.go | 1338 +++++++-------
sdks/go/pkg/beam/register/register.tmpl | 120 +-
sdks/go/pkg/beam/register/register_test.go | 70 +-
sdks/go/pkg/beam/testing/passert/equals.go | 4 +-
sdks/go/pkg/beam/testing/passert/floats.go | 2 +-
sdks/go/pkg/beam/testing/passert/passert.go | 4 +-
sdks/go/pkg/beam/testing/ptest/ptest.go | 8 +-
sdks/go/pkg/beam/testing/ptest/ptest_test.go | 6 +-
sdks/go/pkg/beam/testing/teststream/teststream.go | 6 +-
.../pkg/beam/testing/teststream/teststream_test.go | 20 +-
sdks/java/core/build.gradle | 5 +-
.../org/apache/beam/sdk/coders/DefaultCoder.java | 4 +-
.../org/apache/beam/sdk/coders/SortedMapCoder.java | 2 +
.../beam/sdk/runners/PTransformOverride.java | 4 +-
.../sdk/schemas/FieldValueTypeInformation.java | 22 +-
.../beam/sdk/schemas/FromRowUsingCreator.java | 4 +-
.../java/org/apache/beam/sdk/schemas/Schema.java | 8 +-
.../apache/beam/sdk/schemas/transforms/Filter.java | 4 +-
.../apache/beam/sdk/schemas/transforms/Group.java | 2 +-
.../sdk/schemas/transforms/SchemaAggregateFn.java | 4 +
.../beam/sdk/schemas/utils/ReflectUtils.java | 18 +-
.../beam/sdk/transforms/FlatMapElements.java | 257 ++-
.../apache/beam/sdk/transforms/MapElements.java | 250 ++-
.../beam/sdk/transforms/join/CoGbkResult.java | 9 +-
.../transforms/resourcehints/ResourceHints.java | 2 +-
.../resourcehints/ResourceHintsOptions.java | 5 +-
.../java/org/apache/beam/sdk/util/MoreFutures.java | 10 +-
.../apache/beam/sdk/values/PCollectionList.java | 2 +-
.../apache/beam/sdk/values/PCollectionTuple.java | 2 +-
.../main/java/org/apache/beam/sdk/values/Row.java | 12 +-
.../org/apache/beam/sdk/values/RowWithGetters.java | 1 +
.../apache/beam/sdk/values/TypeDescriptors.java | 2 +-
.../sql/impl/rel/BeamAggregationRel.java | 4 +-
sdks/java/harness/build.gradle | 1 +
.../beam/fn/harness/PrecombineGroupingTable.java | 82 +-
.../fn/harness/PrecombineGroupingTableTest.java | 68 +-
.../beam/sdk/io/elasticsearch/ElasticsearchIO.java | 2 +-
.../beam/sdk/io/gcp/pubsub/PubsubGrpcClient.java | 2 +-
.../java/org/apache/beam/sdk/io/jdbc/JdbcIO.java | 392 ++--
.../beam/sdk/io/jdbc/JdbcSchemaIOProvider.java | 51 +-
.../java/org/apache/beam/sdk/io/jdbc/JdbcUtil.java | 83 +-
.../org/apache/beam/sdk/io/jdbc/LogicalTypes.java | 7 +-
.../org/apache/beam/sdk/io/jdbc/SchemaUtil.java | 38 +-
.../io/kafka/KafkaReadSchemaTransformProvider.java | 38 +-
.../kafka/KafkaWriteSchemaTransformProvider.java | 166 ++
.../org/apache/beam/sdk/io/kafka/KafkaIOIT.java | 144 ++
.../KafkaReadSchemaTransformProviderTest.java | 2 +-
.../beam/sdk/io/mongodb/AggregationQuery.java | 5 +
.../java/org/apache/beam/sdk/io/neo4j/Neo4jIO.java | 4 +-
.../container/py310/base_image_requirements.txt | 77 +-
.../container/py37/base_image_requirements.txt | 79 +-
.../container/py38/base_image_requirements.txt | 84 +-
.../container/py39/base_image_requirements.txt | 83 +-
sdks/python/setup.py | 2 -
.../content/en/documentation/programming-guide.md | 2 +-
201 files changed, 6477 insertions(+), 5423 deletions(-)
copy .test-infra/jenkins/{job_PreCommit_Java_Cassandra_IO_Direct.groovy =>
job_PreCommit_Java_Amqp_IO_Direct.groovy} (91%)
copy .test-infra/jenkins/{job_PreCommit_Java_Cassandra_IO_Direct.groovy =>
job_PreCommit_Java_Azure_IO_Direct.groovy} (91%)
copy .test-infra/jenkins/{job_PreCommit_Java_Cassandra_IO_Direct.groovy =>
job_PreCommit_Java_Cdap_IO_Direct.groovy} (85%)
copy .test-infra/jenkins/{job_PreCommit_Java_Cassandra_IO_Direct.groovy =>
job_PreCommit_Java_ElasticSearch_IO_Direct.groovy} (68%)
copy .test-infra/jenkins/{job_PreCommit_Java_Cassandra_IO_Direct.groovy =>
job_PreCommit_Java_HCatalog_IO_Direct.groovy} (86%)
copy .test-infra/jenkins/{job_PreCommit_Java_JDBC_IO_Direct.groovy =>
job_PreCommit_Java_Hadoop_IO_Direct.groovy} (76%)
copy .test-infra/jenkins/{job_PreCommit_Java_Cassandra_IO_Direct.groovy =>
job_PreCommit_Java_Hbase_IO_Direct.groovy} (87%)
copy .test-infra/jenkins/{job_PreCommit_Java_Cassandra_IO_Direct.groovy =>
job_PreCommit_Java_InfluxDb_IO_Direct.groovy} (89%)
copy .test-infra/jenkins/{job_PreCommit_Java_Cassandra_IO_Direct.groovy =>
job_PreCommit_Java_Jms_IO_Direct.groovy} (91%)
copy .test-infra/jenkins/{job_PreCommit_Java_Cassandra_IO_Direct.groovy =>
job_PreCommit_Java_Kudu_IO_Direct.groovy} (90%)
copy .test-infra/jenkins/{job_PreCommit_Java_Cassandra_IO_Direct.groovy =>
job_PreCommit_Java_MongoDb_IO_Direct.groovy} (86%)
copy .test-infra/jenkins/{job_PreCommit_Java_Cassandra_IO_Direct.groovy =>
job_PreCommit_Java_Mqtt_IO_Direct.groovy} (90%)
copy .test-infra/jenkins/{job_PreCommit_Java_Cassandra_IO_Direct.groovy =>
job_PreCommit_Java_Parquet_IO_Direct.groovy} (89%)
copy .test-infra/jenkins/{job_PreCommit_Java_Cassandra_IO_Direct.groovy =>
job_PreCommit_Java_RabbitMq_IO_Direct.groovy} (89%)
copy .test-infra/jenkins/{job_PreCommit_Java_Cassandra_IO_Direct.groovy =>
job_PreCommit_Java_Redis_IO_Direct.groovy} (90%)
copy .test-infra/jenkins/{job_PreCommit_Java_Cassandra_IO_Direct.groovy =>
job_PreCommit_Java_Singlestore_IO_Direct.groovy} (86%)
copy .test-infra/jenkins/{job_PreCommit_Java_Cassandra_IO_Direct.groovy =>
job_PreCommit_Java_Snowflake_IO_Direct.groovy} (80%)
copy .test-infra/jenkins/{job_PreCommit_Java_Cassandra_IO_Direct.groovy =>
job_PreCommit_Java_Solr_IO_Direct.groovy} (90%)
copy .test-infra/jenkins/{job_PreCommit_Java_Cassandra_IO_Direct.groovy =>
job_PreCommit_Java_Splunk_IO_Direct.groovy} (91%)
copy .test-infra/jenkins/{job_PreCommit_Java_Cassandra_IO_Direct.groovy =>
job_PreCommit_Java_Thrift_IO_Direct.groovy} (91%)
copy .test-infra/jenkins/{job_PreCommit_Java_Cassandra_IO_Direct.groovy =>
job_PreCommit_Java_Tika_IO_Direct.groovy} (91%)
create mode 100644
examples/java/src/main/java/org/apache/beam/examples/KafkaWordCountAvro.java
create mode 100644
examples/java/src/main/java/org/apache/beam/examples/KafkaWordCountJson.java
create mode 100755 playground/backend/datasets/CountWords.avro
create mode 100644 playground/backend/datasets/CountWords.json
copy playground/backend/internal/{db/dto/snippet.go =>
constants/emulator_constants.go} (86%)
create mode 100644
playground/backend/internal/db/dto/precompiled_object_test.go
create mode 100644 playground/backend/internal/emulators/emulator.go
create mode 100644 playground/backend/internal/emulators/kafka.go
create mode 100644
playground/infrastructure/helm-playground/templates/deployment-router-grpc.yml
copy playground/infrastructure/{proxy/allow_list.py =>
helm-playground/templates/service-router-grpc.yml} (59%)
copy playground/infrastructure/{proxy/allow_list.py => repository.py} (54%)
create mode 100644 playground/infrastructure/test_repository.py
create mode 100644
runners/samza/src/main/java/org/apache/beam/runners/samza/translation/StateIdParser.java
create mode 100644
runners/samza/src/main/java/org/apache/beam/runners/samza/util/StoreIdGenerator.java
create mode 100644
sdks/java/io/kafka/src/main/java/org/apache/beam/sdk/io/kafka/KafkaWriteSchemaTransformProvider.java
