[
https://issues.apache.org/jira/browse/MRQL-92?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15380811#comment-15380811
]
Hudson commented on MRQL-92:
----------------------------
ABORTED: Integrated in mrql-master-snapshot #36 (See
[https://builds.apache.org/job/mrql-master-snapshot/36/])
[MRQL-92] Use outer-joins for incremental queries in Spark streaming (fegaras:
[https://git-wip-us.apache.org/repos/asf?p=incubator-mrql.git&a=commit&h=baf8301cc8f98bcf745cda5a0ef875b98ab00204])
* spark/src/main/java/org/apache/mrql/SparkStreaming.gen
* core/src/main/java/org/apache/mrql/Streaming.gen
* queries/factorization3.mrql
* spark/src/main/java/org/apache/mrql/SparkEvaluator.gen
* conf/mrql-env.sh
* core/src/main/java/org/apache/mrql/Interpreter.gen
* core/src/main/java/org/apache/mrql/Translator.gen
* core/src/main/java/org/apache/mrql/PlanGeneration.gen
* core/src/main/java/org/apache/mrql/TypeInference.gen
* core/src/main/java/org/apache/mrql/TopLevel.gen
* pom.xml
* core/src/main/java/org/apache/mrql/Printer.gen
* core/src/main/java/org/apache/mrql/Main.java
* core/src/main/java/org/apache/mrql/Evaluator.java
> Use outer-joins for incremental queries in Spark streaming mode
> ---------------------------------------------------------------
>
> Key: MRQL-92
> URL: https://issues.apache.org/jira/browse/MRQL-92
> Project: MRQL
> Issue Type: Improvement
> Components: Run-Time/Spark, Streaming
> Affects Versions: 0.9.8
> Reporter: Leonidas Fegaras
> Assignee: Leonidas Fegaras
> Priority: Critical
>
> Currently, incremental queries use Spark's coGroup to merge the current state
> with the results of processing the new data in the stream. With this patch,
> the merge is done with a special outer join that doesn't shuffle the state
> again (it only shuffles the results from the new data).
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
