Yes, I agree that we should NOT expose any internal objects at API level.Objects like FrameObject, MatrixObject should not be exposed as those are internal objects. Rule of thumb should be if object (Frame, Object or Scalar) is exposed at DML level it should be exposed at MlContext level.If there is need to add anything extra object besides being exposed in DML it should be justifiable with rationale. I have introduced FrameObject as oversight. It should have been private method instead of public method. I can fix it soon. But there are more changes you have proposed I will let Deron to respond. Thanks for catching these issues. -Arvind
From: Matthias Boehm <mbo...@us.ibm.com> To: dev <dev@systemml.incubator.apache.org> Sent: Sunday, September 11, 2016 9:43 AM Subject: Simplification of MLContext and related APIs It's great to see the ongoing progress on MLContext and related APIs. However, one aspect that really concerns me is the creation of many redundant data types and exposition of various internal data structures. For example, exposing MatrixObject and FrameObject at API level is dangerous because it makes external programs data-dependent on internal structures that might be subject to change (no API stability) and users might not be aware of the implications their interactions have on the buffer pool etc. Furthermore, having such a plethora of entry points makes it very hard to ensure consistency of the compilation chain with regard to configuration handling, environment setup and advanced compilation techniques. I would recommend to create a holistic design across the various APIs that aims to (1) reduce the number of exposed data types (for instance, I would like to remove MatrixObject/FrameObject from the external interface, as well as remove BinaryBlockMatrix, BinaryBlockFrame, Matrix, Frame, and related meta data objects), and (2) create a configurable compilation chain that is invoked from all external APIs. I understand that these data types were introduced to simplify, for example, imports in user programs but I'm sure we find an alternative realization with less redundancy. What do you think? Regards, Matthias