[ https://issues.apache.org/jira/browse/FLINK-1710?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14366939#comment-14366939 ]
Stephan Ewen commented on FLINK-1710: ------------------------------------- This compilation overhead is pretty massive. Two minutes for the page rank test that otherwise takes 5 seconds or so is huge. I think that this means we need to invest in making this a bit better. If I understand it correctly, the code is currently compiled in every serializer and comparator? What about compiling it on the client (with reusing the compiler instance) and then shipping the code? > Expression API Tests take very long > ----------------------------------- > > Key: FLINK-1710 > URL: https://issues.apache.org/jira/browse/FLINK-1710 > Project: Flink > Issue Type: Bug > Components: Expression API > Affects Versions: 0.9 > Reporter: Stephan Ewen > Assignee: Aljoscha Krettek > Fix For: 0.9 > > > The tests of the Expression API take an immense amount of time, compared to > the other API tests. > Is that because they execute on large (generated) data sets, because the > program compilation overhead is high, or because there is an inefficiency in > the execution still? > Running org.apache.flink.api.scala.expressions.AggregationsITCase > Running org.apache.flink.api.scala.expressions.SelectITCase > Running org.apache.flink.api.scala.expressions.AsITCase > Running org.apache.flink.api.scala.expressions.StringExpressionsITCase > Running org.apache.flink.api.scala.expressions.CastingITCase > Running org.apache.flink.api.scala.expressions.JoinITCase > Tests run: 12, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 34.652 sec - > in org.apache.flink.api.scala.expressions.AsITCase > Running org.apache.flink.api.scala.expressions.GroupedAggreagationsITCase > Tests run: 8, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 55.797 sec - > in org.apache.flink.api.scala.expressions.StringExpressionsITCase > Running org.apache.flink.api.scala.expressions.PageRankExpressionITCase > Tests run: 12, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 63.072 sec - > in org.apache.flink.api.scala.expressions.SelectITCase > Running org.apache.flink.api.scala.expressions.ExpressionsITCase > Tests run: 6, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 65.628 sec - > in org.apache.flink.api.scala.expressions.CastingITCase > Running org.apache.flink.api.scala.expressions.FilterITCase > Tests run: 12, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 74.174 sec - > in org.apache.flink.api.scala.expressions.AggregationsITCase > Tests run: 12, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 93.878 sec - > in org.apache.flink.api.scala.expressions.JoinITCase > Tests run: 6, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 63.4 sec - in > org.apache.flink.api.scala.expressions.GroupedAggreagationsITCase > Tests run: 12, Failures: 0, Errors: 0, Skipped: 4, Time elapsed: 44.179 sec - > in org.apache.flink.api.scala.expressions.FilterITCase > Tests run: 12, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 53.801 sec - > in org.apache.flink.api.scala.expressions.ExpressionsITCase > Tests run: 6, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 124.365 sec - > in org.apache.flink.api.scala.expressions.PageRankExpressionITCase -- This message was sent by Atlassian JIRA (v6.3.4#6332)