[ https://issues.apache.org/jira/browse/SYSTEMML-2419?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16534950#comment-16534950 ]
LI Guobao commented on SYSTEMML-2419: ------------------------------------- [~mboehm7] , I have a problem when serializing the instructions. I got some Spark instruction which could not be serialized. Hence, my question is that should we recreate the instruction by forcing the HOPs with CP type. Here is the stack: {code:java} Caused by: org.apache.sysml.runtime.DMLRuntimeException: Not supported: Instructions of type other than CP instructions org.apache.sysml.runtime.instructions.spark.BinaryMatrixScalarSPInstruction SPARK°max°0·SCALAR·INT·true°_mVar1279·MATRIX·DOUBLE°_mVar1280·MATRIX·DOUBLE {code} > Setup and cleanup of remote workers > ----------------------------------- > > Key: SYSTEMML-2419 > URL: https://issues.apache.org/jira/browse/SYSTEMML-2419 > Project: SystemML > Issue Type: Sub-task > Reporter: LI Guobao > Assignee: LI Guobao > Priority: Major > > In the context of distributed spark env, we need to firstly ship the > necessary functions and variables to the remote workers and then to > initialize and register the cleanup of buffer pool for each remote worker. > All these are inspired by the parfor implementation. -- This message was sent by Atlassian JIRA (v7.6.3#76005)