[
https://issues.apache.org/jira/browse/SYSTEMML-995?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15539339#comment-15539339
]
Mike Dusenberry commented on SYSTEMML-995:
------------------------------------------
Yeah here was the explain output:
{code}
--MAIN PROGRAM
----GENERIC (lines 1-7) [recompile=true]
------CP createvar _fVar9 scratch_space//_p21642_9.30.110.134//_t0/temp7 true
FRAME binaryblock -1 -1 1000 1000 -1 copy
------SPARK rblk trainf.FRAME.DOUBLE.false _fVar9.FRAME.DOUBLE 1000 1000 true
------CP createvar _fVar10 scratch_space//_p21642_9.30.110.134//_t0/temp8 true
FRAME binaryblock -1 -1 1000 1000 -1 copy
------SPARK chkpoint _fVar9.FRAME.DOUBLE.false _fVar10.FRAME.DOUBLE
MEMORY_AND_DISK
------CP rmvar _fVar9
------CP nrow _fVar10.FRAME.DOUBLE.false _Var11.SCALAR.INT
------CP ncol _fVar10.FRAME.DOUBLE.false _Var12.SCALAR.INT
------CP rmvar _fVar10
------CP assignvar _Var11.SCALAR.INT.false n.SCALAR.INT
------CP assignvar _Var12.SCALAR.INT.false d.SCALAR.INT
------CP rmvar _Var11
------CP rmvar _Var12
{code}
> MLContext dataframe-frame conversion with index column & vector column
> ----------------------------------------------------------------------
>
> Key: SYSTEMML-995
> URL: https://issues.apache.org/jira/browse/SYSTEMML-995
> Project: SystemML
> Issue Type: Bug
> Components: APIs
> Affects Versions: SystemML 0.11
> Reporter: Matthias Boehm
> Priority: Blocker
>
> MLContext currently always assumes data frame to frame conversion without
> existing index column. Since the user cannot communicate the existence of
> this column, the data conversion leads to incorrect results as an additional
> column is included in the output frame. We need make the MLContext handling
> of frames consistent with the handling of matrices.
> Additionally, the conversion code in
> {{MLContextConversionUtil.dataFrameToFrameObject()}} does not yet take into
> account frames with vectors, although the recent addition adds this support
> in the underlying {{FrameRDDConverterUtils.java}} class. Therefore, the
> number of columns set when {{mc == null}} is incorrect.
> Thanks [~mwdus...@us.ibm.com] for catching this issue. cc [~acs_s] [~deron]
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
