This is an automated email from the ASF dual-hosted git repository.
echauchot pushed a change to branch spark-runner_structured-streaming
in repository https://gitbox.apache.org/repos/asf/beam.git.
from 3533c89 Add source streaming test
new 6b84c2f Specify checkpointLocation at the pipeline start
new dbe88d4 Clean unneeded 0 arg constructor in batch source
new 5e584ce Remove unneeded 0 arg constructor in batch source
new 980a922 Clean streaming source
new 4562a89 Fllow up on offsets for streaming source
new 573cf0c Deal with checkpoint and offset based read
The 19459 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails. The revisions
listed as "add" were already present in the repository and have only
been added to this reference.
Summary of changes:
.../structuredstreaming/SparkPipelineOptions.java | 12 ---
.../translation/TranslationContext.java | 14 ++-
.../translation/batch/DatasetSourceBatch.java | 2 -
.../streaming/DatasetSourceStreaming.java | 111 +++++++++++++++++----
.../streaming/ReadSourceTranslatorStreaming.java | 22 +---
5 files changed, 105 insertions(+), 56 deletions(-)
