[
https://issues.apache.org/jira/browse/SYSTEMML-1467?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15958339#comment-15958339
]
Matthias Boehm commented on SYSTEMML-1467:
------------------------------------------
ok the fix will be in master tomorrow morning, but just for completeness a
workaround is to separate the seq via
{code}
seqS = seq(1, nrow(S));
if(1==1) {}
P = table(seqS, S, ncol(X), ncol(X));
{code}
> optimizer changes the output of table funciton
> ----------------------------------------------
>
> Key: SYSTEMML-1467
> URL: https://issues.apache.org/jira/browse/SYSTEMML-1467
> Project: SystemML
> Issue Type: Bug
> Components: Runtime
> Reporter: Imran Younus
>
> Following is small code snippet which is part of an r4ml dml script.
> {code}
> reorder_matrix = function(
> matrix[double] X, # the input matrix, we have to account for extra beta0
> for the intercept
> matrix[double] B, # beta
> matrix[double] S # Selected
> ) return (matrix[double] Y) {
> S = t(S);
> whatever = ncol(X)
> num_empty_B = ncol(X) - nrow(B);
> if (num_empty_B < 0) {
> stop("Error: unable to re-order the matrix. Reason: B more than matrix
> X");
> }
> P = table(seq(1, nrow(S)), S, ncol(X), ncol(X));
> #P = table(seq(1, nrow(S)), S, whatever, whatever);
> Y = t(P) %*% B;
> }
> X = rand(rows=5000, cols=11, min=0, max=10, pdf="uniform")
> Beta = matrix("-0.048 -1.918 0.000 0.024 0.000 0.005 -0.001 1.842 0.216 0.483
> 0.361", rows=11, cols=1)
> Selected = matrix("11.000 8.000 2.000 4.000 1.000 6.000 10.000 7.000 9.000",
> rows=1, cols=9)
> B = reorder_matrix(X, Beta, Selected)
> print(toString(B))
> {code}
> So this works without any problem. Matrix {{P}} is {{11x11}}, with last two
> rows filled with zeros. BTW, I'm running it in standalone mode.
> Now, if I change line 14 to this:
> {code}
> P = table(seq(1, nrow(S)), S, whatever, whatever);
> {code}
> then nothing should change because {{whatever = ncol(X)}}. But, this breaks
> because now the shape of {{P}} changes. The last two rows are removed because
> these are all zeros.
> Here is the explain for this case:
> {code}
> 17/04/05 17:38:40 INFO api.DMLScript: EXPLAIN (RUNTIME):
> # Memory Budget local/remote = 5496MB/140MB/140MB
> # Degree of Parallelism (vcores) local/remote = 8/1/1
> PROGRAM ( size CP/MR = 0/0 )
> --MAIN PROGRAM
> ----GENERIC (lines 20-24) [recompile=false]
> ------CP createvar _mVar0 scratch_space//_p26900_127.0.1.1//_t0/temp0 true
> MATRIX binaryblock 11 1 1000 1000 -1 copy
> ------CP sinit 11 1 1000 1000 -0.048 -1.918 0.000 0.024 0.000 0.005 -0.001
> 1.842 0.216 0.483 0.361 _mVar0.MATRIX.DOUBLE
> ------CP createvar _mVar1 scratch_space//_p26900_127.0.1.1//_t0/temp1 true
> MATRIX binaryblock 9 1 1000 1000 -1 copy
> ------CP sinit 9 1 1000 1000 11.000 8.000 2.000 4.000 1.000 6.000 10.000
> 7.000 9.000 _mVar1.MATRIX.DOUBLE
> ------CP createvar _mVar2 scratch_space//_p26900_127.0.1.1//_t0/temp2 true
> MATRIX binarycell 1 11 -1 -1 -1 copy
> ------CP r' _mVar0.MATRIX.DOUBLE _mVar2.MATRIX.DOUBLE 8
> ------CP rmvar _mVar0
> ------CP createvar _mVar3 scratch_space//_p26900_127.0.1.1//_t0/temp3 true
> MATRIX binarycell 9 11 -1 -1 -1 copy
> ------CP rexpand cast=true max=11 ignore=false dir=cols target=_mVar1
> _mVar3.MATRIX.DOUBLE
> ------CP rmvar _mVar1
> ------CP createvar _mVar4 scratch_space//_p26900_127.0.1.1//_t0/temp4 true
> MATRIX binarycell 1 11 -1 -1 -1 copy
> ------CP ba+* _mVar2.MATRIX.DOUBLE _mVar3.MATRIX.DOUBLE _mVar4.MATRIX.DOUBLE 8
> ------CP rmvar _mVar2
> ------CP rmvar _mVar3
> ------CP createvar _mVar5 scratch_space//_p26900_127.0.1.1//_t0/temp5 true
> MATRIX binarycell 11 1 -1 -1 -1 copy
> ------CP r' _mVar4.MATRIX.DOUBLE _mVar5.MATRIX.DOUBLE 8
> ------CP rmvar _mVar4
> ------CP toString target=_mVar5 _Var6.SCALAR.STRING
> ------CP rmvar _mVar5
> ------CP print _Var6.SCALAR.STRING.false _Var7.SCALAR.STRING
> ------CP rmvar _Var6
> ------CP rmvar _Var7
> {code}
> Now, lets revert back the definition of {{P}} to
> {code}
> P = table(seq(1, nrow(S)), S, ncol(X), ncol(X));
> {code}
> If I comment out the {{if}} statement, then it breaks again because of the
> same reason.
> Off course, this is the "optimization" at work here. If I set the
> {{opetlevel}} to 0 or 1 in {{SystemML-config.xml}}, then it doesn't break.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
