[
https://issues.apache.org/jira/browse/SYSTEMML-1595?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16003809#comment-16003809
]
Matthias Boehm commented on SYSTEMML-1595:
------------------------------------------
As it turns out the "unknown" blocksizes originate from the default format
"text" which is not a blocked representation. By specifying {{format="binary"}}
in the write statements, the blocksizes are correctly set.
However, every transient write should have proper block sizes and we need to
make sure they are set correctly when modifying persistent writes to transient
writes.
> Missing Block Sizes For PersistentWrites & TransientWrites
> ----------------------------------------------------------
>
> Key: SYSTEMML-1595
> URL: https://issues.apache.org/jira/browse/SYSTEMML-1595
> Project: SystemML
> Issue Type: Bug
> Reporter: Mike Dusenberry
> Attachments: scenario1.dml
>
>
> In the attached script, the resulting PersisentWrites for {{doutc1_agg}} &
> {{dWc1_agg}} end up having unknown block sizes, despite the input DAGs for
> those variables having known block sizes. Due to this, when we use MLContext
> and mark those variables as outputs, the PersistentWrites will be rewritten
> to TransientWrites, and the block sizes will remain unknown.
> To run:
> {code}
> spark-submit $SYSTEMML_HOME/target/SystemML.jar -f scenario1.dml -explain
> recompile_hops
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
