GitHub user jose-torres opened a pull request:
https://github.com/apache/spark/pull/20097
[SPARK-22912] v2 data source support in MicroBatchExecution
## What changes were proposed in this pull request?
Support for v2 data sources in microbatch streaming.
## How was this patch tested?
A very basic new unit test on the toy v2 implementation of rate source.
Once we have a v1 source fully migrated to v2, we'll need to do more detailed
compatibility testing.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/jose-torres/spark v2-impl
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/20097.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #20097
----
commit 3299a3da556757d949aad1687e8ea9cabe790e66
Author: Jose Torres <jose@...>
Date: 2017-12-21T22:58:57Z
partial v2
commit 19919de4f3862fa041b9b3c01d4d2339fc4278b8
Author: Jose Torres <jose@...>
Date: 2017-12-22T00:06:54Z
v2 microbatch with basic test
----
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
