[jira] [Commented] (SYSTEMML-1232) Migrate stringDataFrameToVectorDataFrame if needed
[
https://issues.apache.org/jira/browse/SYSTEMML-1232?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15852566#comment-15852566
]
Deron Eriksson commented on SYSTEMML-1232:
--
cc [~niketanpansare] [~a1singh]
> Migrate stringDataFrameToVectorDataFrame if needed
> --
>
> Key: SYSTEMML-1232
> URL: https://issues.apache.org/jira/browse/SYSTEMML-1232
> Project: SystemML
> Issue Type: Task
> Components: APIs
>Reporter: Deron Eriksson
>
> Restore and migrate RDDConvererUtilsExt.stringDataFrameToVectorDataFrame
> method to Spark 2 (mllib->ml Vector) if needed.
> The RDDConvererUtilsExt.stringDataFrameToVectorDataFrame method was removed
> by commit
> https://github.com/apache/incubator-systemml/commit/578e595fdc506fb8a0c0b18c312fe420a406276d.
> If this method is needed, migrate it to Spark 2.
> Old method:
> {code}
> public static Dataset stringDataFrameToVectorDataFrame(SQLContext
> sqlContext, Dataset inputDF)
> throws DMLRuntimeException {
> StructField[] oldSchema = inputDF.schema().fields();
> //create the new schema
> StructField[] newSchema = new StructField[oldSchema.length];
> for(int i = 0; i < oldSchema.length; i++) {
> String colName = oldSchema[i].name();
> newSchema[i] = DataTypes.createStructField(colName, new
> VectorUDT(), true);
> }
> //converter
> class StringToVector implements Function,
> Row> {
> private static final long serialVersionUID =
> -4733816995375745659L;
> @Override
> public Row call(Tuple2 arg0) throws
> Exception {
> Row oldRow = arg0._1;
> int oldNumCols = oldRow.length();
> if (oldNumCols > 1) {
> throw new DMLRuntimeException("The row
> must have at most one column");
> }
> // parse the various strings. i.e
> // ((1.2,4.3, 3.4)) or (1.2, 3.4, 2.2) or (1.2
> 3.4)
> // [[1.2,34.3, 1.2, 1.2]] or [1.2, 3.4] or [1.3
> 1.2]
> Object [] fields = new Object[oldNumCols];
> ArrayList fieldsArr = new
> ArrayList();
> for (int i = 0; i < oldRow.length(); i++) {
> Object ci=oldRow.get(i);
> if (ci instanceof String) {
> String cis = (String)ci;
> StringBuffer sb = new
> StringBuffer(cis.trim());
> for (int nid=0; i < 2; i++) {
> //remove two level nesting
> if ((sb.charAt(0) ==
> '(' && sb.charAt(sb.length() - 1) == ')') ||
>
> (sb.charAt(0) == '[' && sb.charAt(sb.length() - 1) == ']')
> ) {
>
> sb.deleteCharAt(0);
>
> sb.setLength(sb.length() - 1);
> }
> }
> //have the replace code
> String ncis = "[" +
> sb.toString().replaceAll(" *, *", ",") + "]";
> Vector v = Vectors.parse(ncis);
> fieldsArr.add(v);
> } else {
> throw new
> DMLRuntimeException("Only String is supported");
> }
> }
> Row row =
> RowFactory.create(fieldsArr.toArray());
> return row;
> }
> }
> //output DF
> JavaRDD newRows =
> inputDF.rdd().toJavaRDD().zipWithIndex().map(new StringToVector());
> // DataFrame outDF = sqlContext.createDataFrame(newRows, new
> StructType(newSchema)); //TODO investigate why it doesn't work
> Dataset outDF = sqlContext.createDataFrame(newRows.rdd(),
> DataTypes.createStructType(newSchema));
> return outDF;
> }
> {code}
> Note: the org.apache.spark.ml
[jira] [Created] (SYSTEMML-1232) Migrate stringDataFrameToVectorDataFrame if needed
Deron Eriksson created SYSTEMML-1232:
Summary: Migrate stringDataFrameToVectorDataFrame if needed
Key: SYSTEMML-1232
URL: https://issues.apache.org/jira/browse/SYSTEMML-1232
Project: SystemML
Issue Type: Task
Components: APIs
Reporter: Deron Eriksson
Restore and migrate RDDConvererUtilsExt.stringDataFrameToVectorDataFrame method
to Spark 2 (mllib->ml Vector) if needed.
The RDDConvererUtilsExt.stringDataFrameToVectorDataFrame method was removed by
commit
https://github.com/apache/incubator-systemml/commit/578e595fdc506fb8a0c0b18c312fe420a406276d.
If this method is needed, migrate it to Spark 2.
Old method:
{code}
public static Dataset stringDataFrameToVectorDataFrame(SQLContext
sqlContext, Dataset inputDF)
throws DMLRuntimeException {
StructField[] oldSchema = inputDF.schema().fields();
//create the new schema
StructField[] newSchema = new StructField[oldSchema.length];
for(int i = 0; i < oldSchema.length; i++) {
String colName = oldSchema[i].name();
newSchema[i] = DataTypes.createStructField(colName, new
VectorUDT(), true);
}
//converter
class StringToVector implements Function,
Row> {
private static final long serialVersionUID =
-4733816995375745659L;
@Override
public Row call(Tuple2 arg0) throws
Exception {
Row oldRow = arg0._1;
int oldNumCols = oldRow.length();
if (oldNumCols > 1) {
throw new DMLRuntimeException("The row
must have at most one column");
}
// parse the various strings. i.e
// ((1.2,4.3, 3.4)) or (1.2, 3.4, 2.2) or (1.2
3.4)
// [[1.2,34.3, 1.2, 1.2]] or [1.2, 3.4] or [1.3
1.2]
Object [] fields = new Object[oldNumCols];
ArrayList fieldsArr = new
ArrayList();
for (int i = 0; i < oldRow.length(); i++) {
Object ci=oldRow.get(i);
if (ci instanceof String) {
String cis = (String)ci;
StringBuffer sb = new
StringBuffer(cis.trim());
for (int nid=0; i < 2; i++) {
//remove two level nesting
if ((sb.charAt(0) ==
'(' && sb.charAt(sb.length() - 1) == ')') ||
(sb.charAt(0) == '[' && sb.charAt(sb.length() - 1) == ']')
) {
sb.deleteCharAt(0);
sb.setLength(sb.length() - 1);
}
}
//have the replace code
String ncis = "[" +
sb.toString().replaceAll(" *, *", ",") + "]";
Vector v = Vectors.parse(ncis);
fieldsArr.add(v);
} else {
throw new
DMLRuntimeException("Only String is supported");
}
}
Row row =
RowFactory.create(fieldsArr.toArray());
return row;
}
}
//output DF
JavaRDD newRows =
inputDF.rdd().toJavaRDD().zipWithIndex().map(new StringToVector());
// DataFrame outDF = sqlContext.createDataFrame(newRows, new
StructType(newSchema)); //TODO investigate why it doesn't work
Dataset outDF = sqlContext.createDataFrame(newRows.rdd(),
DataTypes.createStructType(newSchema));
return outDF;
}
{code}
Note: the org.apache.spark.ml.linalg.Vectors.parse() method does not exist in
Spark 2.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (SYSTEMML-1231) Migrate pyspark.mllib.common to pyspark.ml.common
[ https://issues.apache.org/jira/browse/SYSTEMML-1231?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15852548#comment-15852548 ] Deron Eriksson commented on SYSTEMML-1231: -- cc [~iyounus] [~mwdus...@us.ibm.com] [~niketanpansare] > Migrate pyspark.mllib.common to pyspark.ml.common > - > > Key: SYSTEMML-1231 > URL: https://issues.apache.org/jira/browse/SYSTEMML-1231 > Project: SystemML > Issue Type: Task > Components: APIs >Affects Versions: SystemML 0.13 >Reporter: Deron Eriksson > > Investigate migrating pyspark.mllib.common in the Python MLContext to the > newer pyspark.ml.common, as mentioned in > https://github.com/apache/incubator-systemml/pull/369 for Spark 2. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Created] (SYSTEMML-1231) Migrate pyspark.mllib.common to pyspark.ml.common
Deron Eriksson created SYSTEMML-1231: Summary: Migrate pyspark.mllib.common to pyspark.ml.common Key: SYSTEMML-1231 URL: https://issues.apache.org/jira/browse/SYSTEMML-1231 Project: SystemML Issue Type: Task Components: APIs Affects Versions: SystemML 0.13 Reporter: Deron Eriksson Investigate migrating pyspark.mllib.common in the Python MLContext to the newer pyspark.ml.common, as mentioned in https://github.com/apache/incubator-systemml/pull/369 -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (SYSTEMML-1231) Migrate pyspark.mllib.common to pyspark.ml.common
[ https://issues.apache.org/jira/browse/SYSTEMML-1231?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Deron Eriksson updated SYSTEMML-1231: - Description: Investigate migrating pyspark.mllib.common in the Python MLContext to the newer pyspark.ml.common, as mentioned in https://github.com/apache/incubator-systemml/pull/369 for Spark 2. (was: Investigate migrating pyspark.mllib.common in the Python MLContext to the newer pyspark.ml.common, as mentioned in https://github.com/apache/incubator-systemml/pull/369) > Migrate pyspark.mllib.common to pyspark.ml.common > - > > Key: SYSTEMML-1231 > URL: https://issues.apache.org/jira/browse/SYSTEMML-1231 > Project: SystemML > Issue Type: Task > Components: APIs >Affects Versions: SystemML 0.13 >Reporter: Deron Eriksson > > Investigate migrating pyspark.mllib.common in the Python MLContext to the > newer pyspark.ml.common, as mentioned in > https://github.com/apache/incubator-systemml/pull/369 for Spark 2. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Closed] (SYSTEMML-1224) Migrate vector and labeledpoint classes from mllib to ml
[
https://issues.apache.org/jira/browse/SYSTEMML-1224?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Deron Eriksson closed SYSTEMML-1224.
> Migrate vector and labeledpoint classes from mllib to ml
>
>
> Key: SYSTEMML-1224
> URL: https://issues.apache.org/jira/browse/SYSTEMML-1224
> Project: SystemML
> Issue Type: Task
> Components: APIs, Runtime
>Affects Versions: SystemML 0.13
>Reporter: Deron Eriksson
>Assignee: Deron Eriksson
> Fix For: SystemML 0.13
>
>
> For Spark 2, execution of test_mllearn_df.py gives SparseVector to Vector
> error:
> {code}
> spark-submit --driver-class-path $SYSTEMML_HOME/SystemML.jar
> test_mllearn_df.py
> {code}
> generates:
> {code}
> Py4JJavaError: An error occurred while calling o206.fit.
> : org.apache.spark.SparkException: Job aborted due to stage failure: Task 1
> in stage 2.0 failed 1 times, most recent failure: Lost task 1.0 in stage 2.0
> (TID 17, localhost, executor driver): java.lang.ClassCastException:
> org.apache.spark.ml.linalg.SparseVector cannot be cast to
> org.apache.spark.mllib.linalg.Vector
> at
> org.apache.sysml.runtime.instructions.spark.utils.RDDConverterUtils.countNnz(RDDConverterUtils.java:314)
> at
> org.apache.sysml.runtime.instructions.spark.utils.RDDConverterUtils.access$400(RDDConverterUtils.java:71)
> at
> org.apache.sysml.runtime.instructions.spark.utils.RDDConverterUtils$DataFrameAnalysisFunction.call(RDDConverterUtils.java:940)
> at
> org.apache.sysml.runtime.instructions.spark.utils.RDDConverterUtils$DataFrameAnalysisFunction.call(RDDConverterUtils.java:921)
> at
> org.apache.spark.api.java.JavaPairRDD$$anonfun$toScalaFunction$1.apply(JavaPairRDD.scala:1040)
> at scala.collection.Iterator$$anon$11.next(Iterator.scala:409)
> at org.apache.spark.util.Utils$.getIteratorSize(Utils.scala:1762)
> {code}
> This can most likely be fixed by migrating relevant classes (typically going
> from mllib package to ml package).
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Resolved] (SYSTEMML-1224) Migrate vector and labeledpoint classes from mllib to ml
[
https://issues.apache.org/jira/browse/SYSTEMML-1224?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Deron Eriksson resolved SYSTEMML-1224.
--
Resolution: Fixed
Fix Version/s: SystemML 0.13
Fixed by [PR369|https://github.com/apache/incubator-systemml/pull/369].
> Migrate vector and labeledpoint classes from mllib to ml
>
>
> Key: SYSTEMML-1224
> URL: https://issues.apache.org/jira/browse/SYSTEMML-1224
> Project: SystemML
> Issue Type: Task
> Components: APIs, Runtime
>Affects Versions: SystemML 0.13
>Reporter: Deron Eriksson
>Assignee: Deron Eriksson
> Fix For: SystemML 0.13
>
>
> For Spark 2, execution of test_mllearn_df.py gives SparseVector to Vector
> error:
> {code}
> spark-submit --driver-class-path $SYSTEMML_HOME/SystemML.jar
> test_mllearn_df.py
> {code}
> generates:
> {code}
> Py4JJavaError: An error occurred while calling o206.fit.
> : org.apache.spark.SparkException: Job aborted due to stage failure: Task 1
> in stage 2.0 failed 1 times, most recent failure: Lost task 1.0 in stage 2.0
> (TID 17, localhost, executor driver): java.lang.ClassCastException:
> org.apache.spark.ml.linalg.SparseVector cannot be cast to
> org.apache.spark.mllib.linalg.Vector
> at
> org.apache.sysml.runtime.instructions.spark.utils.RDDConverterUtils.countNnz(RDDConverterUtils.java:314)
> at
> org.apache.sysml.runtime.instructions.spark.utils.RDDConverterUtils.access$400(RDDConverterUtils.java:71)
> at
> org.apache.sysml.runtime.instructions.spark.utils.RDDConverterUtils$DataFrameAnalysisFunction.call(RDDConverterUtils.java:940)
> at
> org.apache.sysml.runtime.instructions.spark.utils.RDDConverterUtils$DataFrameAnalysisFunction.call(RDDConverterUtils.java:921)
> at
> org.apache.spark.api.java.JavaPairRDD$$anonfun$toScalaFunction$1.apply(JavaPairRDD.scala:1040)
> at scala.collection.Iterator$$anon$11.next(Iterator.scala:409)
> at org.apache.spark.util.Utils$.getIteratorSize(Utils.scala:1762)
> {code}
> This can most likely be fixed by migrating relevant classes (typically going
> from mllib package to ml package).
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (SYSTEMML-1181) Update documentation with changes related to Spark 2.1.0
[ https://issues.apache.org/jira/browse/SYSTEMML-1181?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15852494#comment-15852494 ] Deron Eriksson commented on SYSTEMML-1181: -- [PR377|https://github.com/apache/incubator-systemml/pull/377] submitted by [~fschueler] removed the old MLContext API from the Spark MLContext Programming Guide. > Update documentation with changes related to Spark 2.1.0 > > > Key: SYSTEMML-1181 > URL: https://issues.apache.org/jira/browse/SYSTEMML-1181 > Project: SystemML > Issue Type: Documentation >Reporter: Arvind Surve >Assignee: Felix Schüler > > Update web page for any changes related to SystemML on Spark 2.1.0 -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Resolved] (SYSTEMML-1227) Cleanup pyc files
[ https://issues.apache.org/jira/browse/SYSTEMML-1227?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Deron Eriksson resolved SYSTEMML-1227. -- Resolution: Fixed Fix Version/s: SystemML 0.13 Fixed by [PR372|https://github.com/apache/incubator-systemml/pull/372]. > Cleanup pyc files > - > > Key: SYSTEMML-1227 > URL: https://issues.apache.org/jira/browse/SYSTEMML-1227 > Project: SystemML > Issue Type: Task > Components: Build >Reporter: Deron Eriksson >Assignee: Deron Eriksson >Priority: Minor > Fix For: SystemML 0.13 > > > Currently *.pyc files can be generated under src/main/python (for instance > when tests are run). > They should be cleaned up when 'mvn clean' is run. > Additionally the *.pyc files should be added to .gitignore. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (SYSTEMML-839) Create Jupyter docs for new MLContext
[ https://issues.apache.org/jira/browse/SYSTEMML-839?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15852361#comment-15852361 ] Felix Schüler commented on SYSTEMML-839: I wanted to write some documentation for using Toree a Spark Scala kernel and MLContext but currently it is not possible to follow the Toree installation instructions and use Spark built with Scala 2.11 (which is the default since Spark 2.0). We would either have to let users build Toree with Scala 2.11 or Spark with Scala 2.10. I currently don't like either of those options. > Create Jupyter docs for new MLContext > - > > Key: SYSTEMML-839 > URL: https://issues.apache.org/jira/browse/SYSTEMML-839 > Project: SystemML > Issue Type: Task > Components: APIs, Documentation >Reporter: Deron Eriksson > > Update MLContext programming guide Jupyter section to use new MLContext -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Assigned] (SYSTEMML-610) Add Databricks Notebook example to MLContext documentation
[ https://issues.apache.org/jira/browse/SYSTEMML-610?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Deron Eriksson reassigned SYSTEMML-610: --- Assignee: (was: Deron Eriksson) > Add Databricks Notebook example to MLContext documentation > -- > > Key: SYSTEMML-610 > URL: https://issues.apache.org/jira/browse/SYSTEMML-610 > Project: SystemML > Issue Type: Task >Reporter: Deron Eriksson > > Add an example of configuring a Databricks Notebook to use SystemML to the > Spark MLContext Programming Guide. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (SYSTEMML-610) Add Databricks Notebook example to MLContext documentation
[ https://issues.apache.org/jira/browse/SYSTEMML-610?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15852309#comment-15852309 ] Deron Eriksson commented on SYSTEMML-610: - [~fschueler] I believe this would be useful. cc [~freiss] [~reinwald] > Add Databricks Notebook example to MLContext documentation > -- > > Key: SYSTEMML-610 > URL: https://issues.apache.org/jira/browse/SYSTEMML-610 > Project: SystemML > Issue Type: Task >Reporter: Deron Eriksson >Assignee: Deron Eriksson > > Add an example of configuring a Databricks Notebook to use SystemML to the > Spark MLContext Programming Guide. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Resolved] (SYSTEMML-1230) Add MLContext info functionality to docs
[ https://issues.apache.org/jira/browse/SYSTEMML-1230?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Deron Eriksson resolved SYSTEMML-1230. -- Resolution: Fixed Fix Version/s: SystemML 0.13 Fixed by commit https://github.com/apache/incubator-systemml/commit/503307bce84d6fae92f29c01238fc916a7132fca > Add MLContext info functionality to docs > > > Key: SYSTEMML-1230 > URL: https://issues.apache.org/jira/browse/SYSTEMML-1230 > Project: SystemML > Issue Type: Task > Components: APIs, Documentation >Reporter: Deron Eriksson >Assignee: Deron Eriksson >Priority: Minor > Fix For: SystemML 0.13 > > -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Closed] (SYSTEMML-1230) Add MLContext info functionality to docs
[ https://issues.apache.org/jira/browse/SYSTEMML-1230?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Deron Eriksson closed SYSTEMML-1230. > Add MLContext info functionality to docs > > > Key: SYSTEMML-1230 > URL: https://issues.apache.org/jira/browse/SYSTEMML-1230 > Project: SystemML > Issue Type: Task > Components: APIs, Documentation >Reporter: Deron Eriksson >Assignee: Deron Eriksson >Priority: Minor > Fix For: SystemML 0.13 > > -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (SYSTEMML-610) Add Databricks Notebook example to MLContext documentation
[ https://issues.apache.org/jira/browse/SYSTEMML-610?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15852269#comment-15852269 ] Felix Schüler commented on SYSTEMML-610: Is this still needed [~deron]? > Add Databricks Notebook example to MLContext documentation > -- > > Key: SYSTEMML-610 > URL: https://issues.apache.org/jira/browse/SYSTEMML-610 > Project: SystemML > Issue Type: Task >Reporter: Deron Eriksson >Assignee: Deron Eriksson > > Add an example of configuring a Databricks Notebook to use SystemML to the > Spark MLContext Programming Guide. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (SYSTEMML-839) Create Jupyter docs for new MLContext
[ https://issues.apache.org/jira/browse/SYSTEMML-839?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15852266#comment-15852266 ] Felix Schüler commented on SYSTEMML-839: This should be updated to the new MLContext API > Create Jupyter docs for new MLContext > - > > Key: SYSTEMML-839 > URL: https://issues.apache.org/jira/browse/SYSTEMML-839 > Project: SystemML > Issue Type: Task > Components: APIs, Documentation >Reporter: Deron Eriksson > > Update MLContext programming guide Jupyter section to use new MLContext -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (SYSTEMML-840) Create Zeppelin docs for new MLContext
[ https://issues.apache.org/jira/browse/SYSTEMML-840?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15852263#comment-15852263 ] Felix Schüler commented on SYSTEMML-840: this should be updated to the new MLContext API > Create Zeppelin docs for new MLContext > -- > > Key: SYSTEMML-840 > URL: https://issues.apache.org/jira/browse/SYSTEMML-840 > Project: SystemML > Issue Type: Task > Components: APIs, Documentation >Reporter: Deron Eriksson > > Update MLContext programming guide Zeppelin section to use new MLContext -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Assigned] (SYSTEMML-1230) Add MLContext info functionality to docs
[ https://issues.apache.org/jira/browse/SYSTEMML-1230?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Deron Eriksson reassigned SYSTEMML-1230: Assignee: Deron Eriksson > Add MLContext info functionality to docs > > > Key: SYSTEMML-1230 > URL: https://issues.apache.org/jira/browse/SYSTEMML-1230 > Project: SystemML > Issue Type: Task > Components: APIs, Documentation >Reporter: Deron Eriksson >Assignee: Deron Eriksson >Priority: Minor > -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Created] (SYSTEMML-1230) Add MLContext info to docs
Deron Eriksson created SYSTEMML-1230: Summary: Add MLContext info to docs Key: SYSTEMML-1230 URL: https://issues.apache.org/jira/browse/SYSTEMML-1230 Project: SystemML Issue Type: Task Components: APIs, Documentation Reporter: Deron Eriksson Priority: Minor -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (SYSTEMML-1230) Add MLContext info functionality to docs
[ https://issues.apache.org/jira/browse/SYSTEMML-1230?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Deron Eriksson updated SYSTEMML-1230: - Summary: Add MLContext info functionality to docs (was: Add MLContext info to docs) > Add MLContext info functionality to docs > > > Key: SYSTEMML-1230 > URL: https://issues.apache.org/jira/browse/SYSTEMML-1230 > Project: SystemML > Issue Type: Task > Components: APIs, Documentation >Reporter: Deron Eriksson >Priority: Minor > -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Closed] (SYSTEMML-1229) Add python mlcontext example to engine dev guide
[ https://issues.apache.org/jira/browse/SYSTEMML-1229?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Deron Eriksson closed SYSTEMML-1229. > Add python mlcontext example to engine dev guide > > > Key: SYSTEMML-1229 > URL: https://issues.apache.org/jira/browse/SYSTEMML-1229 > Project: SystemML > Issue Type: Task > Components: APIs, Documentation >Reporter: Deron Eriksson >Assignee: Deron Eriksson >Priority: Minor > Fix For: SystemML 0.13 > > > Add a basic python mlcontext example to engine dev guide to help developers > get started with the python mlcontext api. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Resolved] (SYSTEMML-1229) Add python mlcontext example to engine dev guide
[ https://issues.apache.org/jira/browse/SYSTEMML-1229?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Deron Eriksson resolved SYSTEMML-1229. -- Resolution: Fixed Fix Version/s: SystemML 0.13 Fixed by commit https://github.com/apache/incubator-systemml/commit/a813e808ba7a60a9593d2eb6cb1210e9a1ff72c7 > Add python mlcontext example to engine dev guide > > > Key: SYSTEMML-1229 > URL: https://issues.apache.org/jira/browse/SYSTEMML-1229 > Project: SystemML > Issue Type: Task > Components: APIs, Documentation >Reporter: Deron Eriksson >Assignee: Deron Eriksson >Priority: Minor > Fix For: SystemML 0.13 > > > Add a basic python mlcontext example to engine dev guide to help developers > get started with the python mlcontext api. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Assigned] (SYSTEMML-1228) SparkListener signatures have changed from Spark 2.0 to 2.1 and fail project build
[ https://issues.apache.org/jira/browse/SYSTEMML-1228?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Felix Schüler reassigned SYSTEMML-1228: --- Assignee: Felix Schüler > SparkListener signatures have changed from Spark 2.0 to 2.1 and fail project > build > -- > > Key: SYSTEMML-1228 > URL: https://issues.apache.org/jira/browse/SYSTEMML-1228 > Project: SystemML > Issue Type: Bug >Reporter: Felix Schüler >Assignee: Felix Schüler > > The method signatures in SparkListener have changed and fail our compilation. > This uses methods from a Spark internal class and is therefore not backwards > compatible 2.1 --> 2.0. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (SYSTEMML-1228) SparkListener signatures have changed from Spark 2.0 to 2.1 and fail project build
[ https://issues.apache.org/jira/browse/SYSTEMML-1228?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15852159#comment-15852159 ] Imran Younus commented on SYSTEMML-1228: I can confirm this. I cannot compile systemml with spark 2.1.0 because of this issue. > SparkListener signatures have changed from Spark 2.0 to 2.1 and fail project > build > -- > > Key: SYSTEMML-1228 > URL: https://issues.apache.org/jira/browse/SYSTEMML-1228 > Project: SystemML > Issue Type: Bug >Reporter: Felix Schüler > > The method signatures in SparkListener have changed and fail our compilation. > This uses methods from a Spark internal class and is therefore not backwards > compatible 2.1 --> 2.0. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Created] (SYSTEMML-1229) Add python mlcontext example to engine dev guide
Deron Eriksson created SYSTEMML-1229: Summary: Add python mlcontext example to engine dev guide Key: SYSTEMML-1229 URL: https://issues.apache.org/jira/browse/SYSTEMML-1229 Project: SystemML Issue Type: Task Components: APIs, Documentation Reporter: Deron Eriksson Assignee: Deron Eriksson Priority: Minor Add a basic python mlcontext example to engine dev guide to help developers get started with the python mlcontext api. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Created] (SYSTEMML-1228) SparkListener signatures have changed from Spark 2.0 to 2.1 and fail project build
Felix Schüler created SYSTEMML-1228: --- Summary: SparkListener signatures have changed from Spark 2.0 to 2.1 and fail project build Key: SYSTEMML-1228 URL: https://issues.apache.org/jira/browse/SYSTEMML-1228 Project: SystemML Issue Type: Bug Reporter: Felix Schüler The method signatures in SparkListener have changed and fail our compilation. This uses methods from a Spark internal class and is therefore not backwards compatible 2.1 --> 2.0. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
