Leonidas Fegaras created MRQL-92:
------------------------------------
Summary: Use outer-joins for incremental queries in Spark
streaming mode
Key: MRQL-92
URL: https://issues.apache.org/jira/browse/MRQL-92
Project: MRQL
Issue Type: Improvement
Components: Run-Time/Spark, Streaming
Affects Versions: 0.9.8
Reporter: Leonidas Fegaras
Assignee: Leonidas Fegaras
Priority: Critical
Currently, incremental queries use Spark's coGroup to merge the current state
with the results of processing the new data in the stream. With this patch, the
merge is done with a special outer join that doesn't shuffle the state again
(it only shuffles the results from the new data).
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
