[
https://issues.apache.org/jira/browse/FLINK-1710?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14366939#comment-14366939
]
Stephan Ewen commented on FLINK-1710:
-------------------------------------
This compilation overhead is pretty massive. Two minutes for the page rank test
that otherwise takes 5 seconds or so is huge. I think that this means we need
to invest in making this a bit better.
If I understand it correctly, the code is currently compiled in every
serializer and comparator? What about compiling it on the client (with reusing
the compiler instance) and then shipping the code?
> Expression API Tests take very long
> -----------------------------------
>
> Key: FLINK-1710
> URL: https://issues.apache.org/jira/browse/FLINK-1710
> Project: Flink
> Issue Type: Bug
> Components: Expression API
> Affects Versions: 0.9
> Reporter: Stephan Ewen
> Assignee: Aljoscha Krettek
> Fix For: 0.9
>
>
> The tests of the Expression API take an immense amount of time, compared to
> the other API tests.
> Is that because they execute on large (generated) data sets, because the
> program compilation overhead is high, or because there is an inefficiency in
> the execution still?
> Running org.apache.flink.api.scala.expressions.AggregationsITCase
> Running org.apache.flink.api.scala.expressions.SelectITCase
> Running org.apache.flink.api.scala.expressions.AsITCase
> Running org.apache.flink.api.scala.expressions.StringExpressionsITCase
> Running org.apache.flink.api.scala.expressions.CastingITCase
> Running org.apache.flink.api.scala.expressions.JoinITCase
> Tests run: 12, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 34.652 sec -
> in org.apache.flink.api.scala.expressions.AsITCase
> Running org.apache.flink.api.scala.expressions.GroupedAggreagationsITCase
> Tests run: 8, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 55.797 sec -
> in org.apache.flink.api.scala.expressions.StringExpressionsITCase
> Running org.apache.flink.api.scala.expressions.PageRankExpressionITCase
> Tests run: 12, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 63.072 sec -
> in org.apache.flink.api.scala.expressions.SelectITCase
> Running org.apache.flink.api.scala.expressions.ExpressionsITCase
> Tests run: 6, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 65.628 sec -
> in org.apache.flink.api.scala.expressions.CastingITCase
> Running org.apache.flink.api.scala.expressions.FilterITCase
> Tests run: 12, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 74.174 sec -
> in org.apache.flink.api.scala.expressions.AggregationsITCase
> Tests run: 12, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 93.878 sec -
> in org.apache.flink.api.scala.expressions.JoinITCase
> Tests run: 6, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 63.4 sec - in
> org.apache.flink.api.scala.expressions.GroupedAggreagationsITCase
> Tests run: 12, Failures: 0, Errors: 0, Skipped: 4, Time elapsed: 44.179 sec -
> in org.apache.flink.api.scala.expressions.FilterITCase
> Tests run: 12, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 53.801 sec -
> in org.apache.flink.api.scala.expressions.ExpressionsITCase
> Tests run: 6, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 124.365 sec -
> in org.apache.flink.api.scala.expressions.PageRankExpressionITCase
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
