This is an automated email from the ASF dual-hosted git repository. github-bot pushed a change to branch nightly-refs/heads/master in repository https://gitbox.apache.org/repos/asf/beam.git.
from 6e2d39f [BEAM-13015] Plumb the cache through contexts and transform executors. (#16188) add 2547dbe [BEAM-13441] Use quiet delete for S3 batch deletes. In quiet mode only errors will be returned in the service response. add 3985d8d Merge pull request #16197: [BEAM-13441] Use S3 quiet mode for deletes add cc11936 Merge pull request #16140 from [BEAM-13377][Playground] Update CI/CD python scripts to work with specific SDK add 0257792 Merge pull request #16120 from [BEAM-13333][Playground] Save Python logs to the cache add e183c14 Merge pull request #16185 from [BEAM-13425][Playground][Bugfix] Support logs while storing to bucket add b225f60 change context type add 79d6418 Merge pull request #16211 from [BEAM-13207][Playground] Change context type add 97a2acc Merge pull request #16121 from [BEAM-13334][Playground] Save Go logs to the cache add 290fc87 [BEAM-13423]: fix frontend failure if no examples add ad28b00 Merge pull request #16193 from [BEAM-13423] [Bugiix] [Playground]: fix frontend failure if no examples add 9a68b80 Docs for validators tests add bf77781 Merge pull request #16206 from [BEAM-13449][Playground] Docs for validators tests add 22e761c change return type of 2 methods add ba35c86 Merge pull request #16196 from [BEAM-13408][Playground] Change return type of 2 methods add d7e1a50 Merge pull request #16179 from [BEAM-13344][Playground] support python unit tests add 3e5630f Merge pull request #16204 from [BEAM-13444][Playground] Add processing of an error add 626e7a9 Merge pull request #16208 from [BEAM-13442][Playground] Filepath to log error add f969a7b [BEAM-13445] Correctly set data limit when flushing S3 upload buffer and avoid temporary copy to array when writing source buffer to upload buffer add e2ac013 Merge pull request #16205: [BEAM-13445] Correctly set data limit when flushing S3 upload buffer add cf35f72 [BEAM-13276] bump jackson-core to 2.13.0 for .test-infra (#16062) add 512ce86 Change Pub/Sub Lite PollResult to set explicit watermark (#16216) add 3cfec5b Updating Grafana from v8.1.2 to v8.1.6 add 37d1d99 Merge pull request #16201 from apache/aaltay-patch-1 add e056d73 Include name of missing tag in error message. add 25e6008 Merge pull request #16175 from ibzib/tag-name add 98c5266 Better type hints for Count combiners. add 0111905 Merge pull request #16145 Better type hints for Count combiners. add da11b84 [BEAM-13454] Fix and test dataframe read_fwf. (#16064) add d41ae47 [BEAM-12976] Pipeline visitor to discover pushdown opportunities. (#16176) No new revisions were added by this update. Summary of changes: .test-infra/metrics/grafana/Dockerfile | 2 +- .test-infra/validate-runner/build.gradle | 4 +- playground/backend/cmd/server/controller.go | 7 +- playground/backend/configs/SDK_PYTHON.json | 4 +- .../internal/cloud_bucket/precompiled_objects.go | 14 +- .../internal/code_processing/code_processing.go | 113 ++++++--- .../internal/preparators/python_preparators.go | 77 +++++- .../preparators/python_preparators_test.go | 101 ++++++++ .../internal/setup_tools/builder/setup_builder.go | 2 +- .../setup_tools/builder/setup_builder_test.go | 3 + .../internal/streaming/run_output_writer.go | 3 + playground/backend/internal/utils/cache_utils.go | 2 +- .../backend/internal/utils/validators_utils.go | 2 +- .../internal/validators/python_validators.go | 34 ++- .../backend/internal/validators/validator_test.go | 3 + .../components/playground_page_providers.dart | 3 + playground/frontend/pubspec.lock | 10 +- playground/infrastructure/api/v1/api_pb2_grpc.py | 2 +- playground/infrastructure/cd_helper.py | 15 +- playground/infrastructure/ci_cd.py | 67 +++-- playground/infrastructure/ci_helper.py | 18 +- playground/infrastructure/config.py | 12 +- playground/infrastructure/grpc_client.py | 15 ++ playground/infrastructure/helper.py | 68 +++-- playground/infrastructure/requirements.txt | 1 + playground/infrastructure/test_cd_helper.py | 35 +-- playground/infrastructure/test_ci_cd.py | 42 +--- playground/infrastructure/test_helper.py | 129 +++++----- .../graph/ProjectionProducerVisitor.java | 83 +++++++ .../graph/ProjectionProducerVisitorTest.java | 274 +++++++++++++++++++++ .../apache/beam/sdk/values/PCollectionTuple.java | 3 +- .../apache/beam/sdk/io/aws/s3/S3FileSystem.java | 3 +- .../beam/sdk/io/aws/s3/S3WritableByteChannel.java | 25 +- .../sdk/io/aws/s3/S3WritableByteChannelTest.java | 39 +-- .../apache/beam/sdk/io/aws2/s3/S3FileSystem.java | 2 +- .../beam/sdk/io/aws2/s3/S3WritableByteChannel.java | 25 +- .../apache/beam/sdk/io/aws2/s3/S3FileSystemIT.java | 18 +- .../sdk/io/aws2/s3/S3WritableByteChannelTest.java | 17 +- .../internal/SubscriptionPartitionLoader.java | 3 +- sdks/python/apache_beam/dataframe/io.py | 47 +++- sdks/python/apache_beam/dataframe/io_test.py | 29 ++- sdks/python/apache_beam/transforms/combiners.py | 6 + .../apache_beam/transforms/ptransform_test.py | 10 +- 43 files changed, 1046 insertions(+), 326 deletions(-) create mode 100644 playground/backend/internal/preparators/python_preparators_test.go create mode 100644 runners/core-construction-java/src/main/java/org/apache/beam/runners/core/construction/graph/ProjectionProducerVisitor.java create mode 100644 runners/core-construction-java/src/test/java/org/apache/beam/runners/core/construction/graph/ProjectionProducerVisitorTest.java