The constrained optimizer doesn't seem to know about a REMOTE_SPARK
execution mode and either sets CP or REMOTE_MR. I can open a jira for
that and provide a fix.
Felix
Am 22.11.2016 02:07 schrieb Matthias Boehm:
yes, this came up several times - initially we only supported opt=NONE
where users had to specify all other parameters. Meanwhile, there is a
so-called "constrained optimizer" that does the same as the rule-based
optimizer but respects any given parameters. Please try something like
this:
parfor (i in 1:10, opt=CONSTRAINED, par=10, mode=REMOTE_SPARK) {
// some code here
}
Regards,
Matthias
On 11/22/2016 12:33 AM, [email protected] wrote:
While debugging some ParFor code it became clear that the parameters
for
parfor can be easily overwritten by the optimizer.
One example is when I write:
```
parfor (i in 1:10, par=10, mode=REMOTE_SPARK) {
// some code here
}
```
Depending on the data size and cluster resources, the optimizer
(OptimizerRuleBased.java, line 844) will recognize that the work can
be
done locally and overwrite it to local execution. This might be valid
and definitely works (in my case) but kind of contradicts what I want
SystemML to do.
I wonder if we should disable this optimization in case a concrete
execution mode is given and go with the mode that is provided.
Felix
