yes, this came up several times - initially we only supported opt=NONE
where users had to specify all other parameters. Meanwhile, there is a
so-called "constrained optimizer" that does the same as the rule-based
optimizer but respects any given parameters. Please try something like this:
parfor (i in 1:10, opt=CONSTRAINED, par=10, mode=REMOTE_SPARK) {
// some code here
}
Regards,
Matthias
On 11/22/2016 12:33 AM, [email protected] wrote:
While debugging some ParFor code it became clear that the parameters for
parfor can be easily overwritten by the optimizer.
One example is when I write:
```
parfor (i in 1:10, par=10, mode=REMOTE_SPARK) {
// some code here
}
```
Depending on the data size and cluster resources, the optimizer
(OptimizerRuleBased.java, line 844) will recognize that the work can be
done locally and overwrite it to local execution. This might be valid
and definitely works (in my case) but kind of contradicts what I want
SystemML to do.
I wonder if we should disable this optimization in case a concrete
execution mode is given and go with the mode that is provided.
Felix
