This is an automated email from the ASF dual-hosted git repository.
iemejia pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/beam.git.
from 3aaf39a Merge pull request #8251: [BEAM-7029] Add KafkaIO.Read as an
external transform
new 6065ef5 [BEAM-5775] Implement a custom class to lazily encode values
for persistence.
new 2bef33c [BEAM-5775] Rename BeamSparkRunnerRegistrator to
SparkRunnerKryoRegistrator
new fdedb4c [BEAM-5775] Rename avoidRddSerialization to
canAvoidRddSerialization
new 1a74c2d [BEAM-5775] Update spark runner to use non-deprecated Coder
API methods
new 51d146a [BEAM-5775] Make TranslationUtils iterators Java 8 style
new 28547d1 Merge pull request #6714: [BEAM-5775] Spark: implement a
custom class to lazily encode values for persistence.
The 20964 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails. The revisions
listed as "add" were already present in the repository and have only
been added to this reference.
Summary of changes:
.../utils/ValueAndCoderKryoSerializable.java | 5 +-
.../beam/runners/spark/coders/CoderHelpers.java | 4 +-
...trator.java => SparkRunnerKryoRegistrator.java} | 26 ++--
.../runners/spark/translation/BoundedDataset.java | 6 +-
.../spark/translation/GroupCombineFunctions.java | 162 +++------------------
.../spark/translation/SparkContextFactory.java | 4 +-
.../spark/translation/TransformTranslator.java | 4 +-
.../spark/translation/TranslationUtils.java | 80 +++++-----
.../translation/ValueAndCoderKryoSerializer.java | 50 +++++++
.../translation/ValueAndCoderLazySerializable.java | 137 +++++++++++++++++
.../runners/spark/util/SideInputBroadcast.java | 2 +-
...st.java => SparkRunnerKryoRegistratorTest.java} | 10 +-
...java => ValueAndCoderLazySerializableTest.java} | 51 ++-----
13 files changed, 289 insertions(+), 252 deletions(-)
rename
runners/spark/src/main/java/org/apache/beam/runners/spark/coders/{BeamSparkRunnerRegistrator.java
=> SparkRunnerKryoRegistrator.java} (90%)
create mode 100644
runners/spark/src/main/java/org/apache/beam/runners/spark/translation/ValueAndCoderKryoSerializer.java
create mode 100644
runners/spark/src/main/java/org/apache/beam/runners/spark/translation/ValueAndCoderLazySerializable.java
rename
runners/spark/src/test/java/org/apache/beam/runners/spark/coders/{BeamSparkRunnerRegistratorTest.java
=> SparkRunnerKryoRegistratorTest.java} (91%)
rename
runners/spark/src/test/java/org/apache/beam/runners/spark/translation/{GroupCombineFunctionsTest.java
=> ValueAndCoderLazySerializableTest.java} (63%)
