Re: [SS] Writing a test for a possible bug in StateStoreSaveExec with Append output mode?

2017-09-04 Thread Jacek Laskowski
Hi, I think I know where the issue surfaces. This is with groupBy aggregation with Append output mode. What should happen when a state expires for a event time (in groupBy) with the new rows for the expired key in a streaming batch exactly when watermark has been moved up and thus expired the sta

[SS] Writing a test for a possible bug in StateStoreSaveExec with Append output mode?

2017-09-03 Thread Jacek Laskowski
Hi, I may have found a bug in StateStoreSaveExec with Append output mode and would love proving myself I'm wrong or help squashing it by writing a test for the case. Is there a test for StateStoreSaveExec with Append output mode? If not, is there a streaming test template that could be very close