[
https://issues.apache.org/jira/browse/SYSTEMML-1884?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Matthias Boehm updated SYSTEMML-1884:
-------------------------------------
Description:
The builtin function {{removeEmpty}} takes an optional select parameter. If
this parameter is provided, we directly use it instead of explicitly computing
it. However, the compilation of removeEmpty handled this incorrectly, setting
the forced execution type to the initially compiled execution type of
removeEmpty. This is problematic in scenarios with dynamic recompilation. For
example, consider the following scenario:
{code}
C = replace(target=A, pattern=2, replacement=0);
D = removeEmpty(B, margin="rows", select=C);
{code}
If the {{removeEmpty}} is initially compiled to SPARK, this also mistakenly set
the forced execution type of {{replace}} to SPARK. During recompilation, the
{{removeEmpty}} might be compiled to CP but because {{replace}} has a forced
execution type, it will remain in SPARK.
Workaround: The workaround for this issue is to put a {{while(FALSE){}}} just
before the {{removeEmpty}} call (more precisely, C in above example needs to be
computed in a separate HOP DAG).
was:
The builtin function {{removeEmpty}} takes an optional select parameter. If
this parameter is provided, we directly use it instead of explicitly computing
it. However, the compilation of removeEmpty handled this incorrectly, setting
the forced execution type to the initially compiled execution type of
removeEmpty. This is problematic in scenarios with dynamic recompilation. For
example, consider the following scenario:
{code}
C = replace(target=A, pattern=2, replacement=0);
D = removeEmpty(B, margin="rows", select=C);
{code}
If the {{removeEmpty}} is initially compiled to SPARK, this also mistakenly set
the forced execution type of {{replace}} to SPARK. During recompilation, the
{{removeEmpty}} might be compiled to CP but because {{replace}} has a forced
execution type, it will remain in SPARK.
Workaround: The workaround until this is fixed, is to put a {{while(FALSE){}}}
just before the {{removeEmpty}} call (more precisely, C in above example needs
to be computed in a separate HOP DAG).
> Unnecessary spark/mr instructions on removeEmpty select
> -------------------------------------------------------
>
> Key: SYSTEMML-1884
> URL: https://issues.apache.org/jira/browse/SYSTEMML-1884
> Project: SystemML
> Issue Type: Bug
> Affects Versions: SystemML 0.14
> Reporter: Matthias Boehm
> Assignee: Matthias Boehm
> Fix For: SystemML 1.0
>
>
> The builtin function {{removeEmpty}} takes an optional select parameter. If
> this parameter is provided, we directly use it instead of explicitly
> computing it. However, the compilation of removeEmpty handled this
> incorrectly, setting the forced execution type to the initially compiled
> execution type of removeEmpty. This is problematic in scenarios with dynamic
> recompilation. For example, consider the following scenario:
> {code}
> C = replace(target=A, pattern=2, replacement=0);
> D = removeEmpty(B, margin="rows", select=C);
> {code}
> If the {{removeEmpty}} is initially compiled to SPARK, this also mistakenly
> set the forced execution type of {{replace}} to SPARK. During recompilation,
> the {{removeEmpty}} might be compiled to CP but because {{replace}} has a
> forced execution type, it will remain in SPARK.
> Workaround: The workaround for this issue is to put a {{while(FALSE){}}} just
> before the {{removeEmpty}} call (more precisely, C in above example needs to
> be computed in a separate HOP DAG).
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
