[ https://issues.apache.org/jira/browse/SYSTEMML-2120?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Matthias Boehm closed SYSTEMML-2120. ------------------------------------ Resolution: Done Assignee: Matthias Boehm Fix Version/s: SystemML 1.1 > Improve codegen optimizer (pruning effectiveness) > ------------------------------------------------- > > Key: SYSTEMML-2120 > URL: https://issues.apache.org/jira/browse/SYSTEMML-2120 > Project: SystemML > Issue Type: Sub-task > Reporter: Matthias Boehm > Assignee: Matthias Boehm > Priority: Major > Fix For: SystemML 1.1 > > > The codegen optimizer applies various techniques to prune the search space, > including the partitioning into independent problems, the restriction to > interesting points, as well as cost-based and structural pruning. These > techniques work very well for most use case rendering the codegen overhead > negligible. However, there are still exceptions where the additional > compilation overhead leads to a slowdown. One of these examples is lenet, the > below codegen statistics are obtained from 1000 iterations over mnist60k. > {code} > HOP DAGs recompiled (PRED, SB): 0/1154. > HOP DAGs recompile time: 237.579 sec. > Codegen compile (DAG,CP,JC): 1176/17450/36. > Codegen enum (ALLt/p,EVALt/p): 1064354993445/16387318/11342971/11331811. > Codegen compile times (DAG,JC): 234.174/0.835 sec. > Codegen plan cache hits: 17408/17444. > {code} > Pruning reduced the number of plans from > 1 trillion to 11 million, which is > still too high causing an overhead of 234s for a total execution time of 425s. -- This message was sent by Atlassian JIRA (v7.6.3#76005)