Thank you, Ted! I'll start from there! On Wed, May 3, 2017 at 5:52 PM, Ted Yu <yuzhih...@gmail.com> wrote:
> See this related thread: > > http://search-hadoop.com/m/Flink/VkLeQcgMO91NWi982?subj= > flink+1+2+and+unit+testing+flinkspector > > On Wed, May 3, 2017 at 3:20 AM, Aljoscha Krettek <aljos...@apache.org> > wrote: > > > Hi, > > Yes, unfortunately there are no good unit test examples for testing Flink > > jobs (with windowing or without). And yes, I would like to have something > > like flink-spector (or flink-spector) as part of Flink as the promoted > way > > of doing Job testing. In my opinion this is blocked on a good way of > > verifying that assertions succeed inside operators and on an API to > > programmatically monitor and cancel a running streaming Job. > > > > If you had those two components (asynchronous job management API and > > assertion verification) you could do something similar to PAssert ( > > https://beam.apache.org/documentation/sdks/javadoc/0. > > 6.0/index.html?org/apache/beam/sdk/testing/PAssert.html < > > https://beam.apache.org/documentation/sdks/javadoc/0. > > 6.0/index.html?org/apache/beam/sdk/testing/PAssert.html>) in Apache Beam > > that is used for a lot of tests. The idea is that assertion operators > > inside the pipeline wait for a condition to be true (or fail of some > > negative condition is violated) and signal success by updating a unique > > Accumulator value. The test would use the asynchronous job API to listen > on > > Accumulator updates and wait until all required conditions have been met. > > Then it would stop the job and the test would succeed. > > > > Best, > > Aljoscha > > > > > On 2. May 2017, at 22:01, Bowen Li <bowen...@offerupnow.com> wrote: > > > > > > Hi Flink folks! > > > Are there any concrete good examples of unit testing Flink > windowing? > > > I've been searching around, and all those available online are quite > > > ambiguous. > > > > > > Another thing is, I found the flink-spector > > > <https://github.com/ottogroup/flink-spector> library is pretty > awesome, > > > simple to understand and use. But it's kind of out of maintenance, and > > only > > > supports 1.4 now. Is it possible to bring flink-spector into Apache > > Flink, > > > so we can better upgrade and maintain it? > > > > > > Thank you! > > > Bowen > > > > >
