[
https://issues.apache.org/jira/browse/SYSTEMML-1156?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Deron Eriksson resolved SYSTEMML-1156.
--------------------------------------
Resolution: Fixed
Fix Version/s: SystemML 0.12
> problem with MLContext and QR
> -----------------------------
>
> Key: SYSTEMML-1156
> URL: https://issues.apache.org/jira/browse/SYSTEMML-1156
> Project: SystemML
> Issue Type: Bug
> Components: Runtime
> Environment: spark 1.6.2
> centOS7
> Reporter: Imran Younus
> Assignee: Matthias Boehm
> Fix For: SystemML 0.12
>
>
> I'm trying to run this simple code to get QR
> {code}
> X = rand(rows=4, cols=2)
> [H, R] = qr(X)
> print(toString(H))
> print ("X is of size : " + nrow(X) + "," + ncol(X))
> print ("H is of size : " + nrow(H) + "," + ncol(H))
> print ("R is of size : " + nrow(R) + "," + ncol(R))
> n = ncol(H)
> for( j in n:1 ) {
> print(j);
> V = H[,j];
> print ("V is of size : " + nrow(V) + "," + ncol(V))
> VTV = t(V) %*% V
> print(toString(VTV))
> }
> {code}
> I ran this in CP mode and in hybrid spark mode.
> In the CP mode this works perfectly fine.
> But, when I run this with spark then the behavior is strange.
> The problem is that inside the for loop, when I assign {{H\[,j\]}} to {{V}},
> it becomes {{H}} instead of just a column of {{H}}. So, {{VTV}} then becomes
> a matrix instead of just a number which I want. This only happens inside the
> for loop. If I do this without for loop then there is no problem. Also, this
> is occurs only for matrix {{H}}. If I replace {{H}} with {{X}} instead, then
> there is no problem. Here is the out of the code when I run it with spark:
> {code}
> 16/12/16 11:53:27 INFO api.DMLScript: BEGIN DML run 12/16/2016 11:53:27
> 16/12/16 11:53:27 WARN util.NativeCodeLoader: Unable to load native-hadoop
> library for your platform... using builtin-java classes where applicable
> X is of size : 4,2
> H is of size : 4,2
> R is of size : 4,2
> 1.526 0.000
> 0.459 1.905
> 0.280 -0.202
> 0.659 0.373
> 2
> V is of size : 4,1
> 3.051 1.064
> 1.064 3.811
> 1
> V is of size : 4,1
> 3.051 1.064
> 1.064 3.811
> 16/12/16 11:53:27 INFO api.DMLScript: SystemML Statistics:
> Total execution time: 0.624 sec.
> Number of executed Spark inst: 0.
> 16/12/16 11:53:27 INFO api.DMLScript: END DML run 12/16/2016 11:53:27
> {code}
> As you can see from the output, the size of {{V}} is correct. Its supposed to
> be a column vector. But, {{VTV}} is a 2x2 matrix instead of a number because
> {{V}} is just {{H}}. We print {{V}} and see that.
> Here is correct output form CP mode:
> {code}
> ================================================================================
> ================================================================================
> 16/12/16 11:54:56 INFO api.DMLScript: BEGIN DML run 12/16/2016 11:54:56
> 16/12/16 11:54:57 WARN util.NativeCodeLoader: Unable to load native-hadoop
> library for your platform... using builtin-java classes where applicable
> X is of size : 4,2
> H is of size : 4,2
> R is of size : 4,2
> 1.575 0.000
> 0.476 1.591
> 0.296 -0.772
> 0.596 0.233
> 2
> V is of size : 4,1
> 3.182
> 1
> V is of size : 4,1
> 3.151
> 16/12/16 11:54:57 INFO api.DMLScript: SystemML Statistics:
> Total execution time: 0.199 sec.
> Number of executed MR Jobs: 0.
> 16/12/16 11:54:57 INFO api.DMLScript: END DML run 12/16/2016 11:54:57
> {code}
> [~mboehm7] [~niketanpansare]
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
