[GitHub] spark issue #23009: SPARK-26011: pyspark app with "spark.jars.packages" conf...

2018-11-12 Thread imatiach-msft
Github user imatiach-msft commented on the issue: https://github.com/apache/spark/pull/23009 @srowen , @drdarshan mentioned that it may be better to fix livy instead of spark. The problem is that when using the %%configure magic spark.jars.packages does not add the python files

[GitHub] spark issue #23009: SPARK-26011: pyspark app with "spark.jars.packages" conf...

2018-11-11 Thread imatiach-msft
Github user imatiach-msft commented on the issue: https://github.com/apache/spark/pull/23009 @shanyu can you update the name as [SPARK-26011][CORE][PYSPARK] according to the guidelines? --- - To unsubscribe, e-mail

[GitHub] spark issue #23009: SPARK-26011: pyspark app with "spark.jars.packages" conf...

2018-11-11 Thread imatiach-msft
Github user imatiach-msft commented on the issue: https://github.com/apache/spark/pull/23009 Jenkins test this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e

[GitHub] spark issue #23009: SPARK-26011: pyspark app with "spark.jars.packages" conf...

2018-11-11 Thread imatiach-msft
Github user imatiach-msft commented on the issue: https://github.com/apache/spark/pull/23009 LGTM, nice find --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail

[GitHub] spark issue #17086: [SPARK-24101][ML][MLLIB] ML Evaluators should use weight...

2018-11-09 Thread imatiach-msft
Github user imatiach-msft commented on the issue: https://github.com/apache/spark/pull/17086 thank you @srowen ! I will try to update the other two PRs as soon as possible. Really exciting to see this get

[GitHub] spark issue #17086: [SPARK-24101][ML][MLLIB] ML Evaluators should use weight...

2018-11-08 Thread imatiach-msft
Github user imatiach-msft commented on the issue: https://github.com/apache/spark/pull/17086 @srowen thanks, I've fixed the merge conflict and updated to latest --- - To unsubscribe, e-mail: reviews-unsubscr

[GitHub] spark pull request #17086: [SPARK-24101][ML][MLLIB] ML Evaluators should use...

2018-11-07 Thread imatiach-msft
Github user imatiach-msft commented on a diff in the pull request: https://github.com/apache/spark/pull/17086#discussion_r231760882 --- Diff: mllib/src/main/scala/org/apache/spark/mllib/evaluation/MulticlassMetrics.scala --- @@ -27,10 +27,17 @@ import

[GitHub] spark pull request #17086: [SPARK-24101][ML][MLLIB] ML Evaluators should use...

2018-11-07 Thread imatiach-msft
Github user imatiach-msft commented on a diff in the pull request: https://github.com/apache/spark/pull/17086#discussion_r231760666 --- Diff: mllib/src/main/scala/org/apache/spark/mllib/evaluation/MulticlassMetrics.scala --- @@ -27,10 +27,17 @@ import

[GitHub] spark issue #22087: [SPARK-25097][ML] Support prediction on single instance ...

2018-11-07 Thread imatiach-msft
Github user imatiach-msft commented on the issue: https://github.com/apache/spark/pull/22087 @srowen @mgaido91 @erikerlandson I don't think it makes sense to use PredictionModel as it is currently because it is for supervised learning and contains label column specific params (eg

[GitHub] spark issue #17086: [SPARK-24101][ML][MLLIB] ML Evaluators should use weight...

2018-11-07 Thread imatiach-msft
Github user imatiach-msft commented on the issue: https://github.com/apache/spark/pull/17086 @srowen would you be able to take another look at this PR? Also tagging @WeichenXu123 . Thank you! --- - To unsubscribe

[GitHub] spark pull request #17086: [SPARK-24101][ML][MLLIB] ML Evaluators should use...

2018-11-06 Thread imatiach-msft
Github user imatiach-msft commented on a diff in the pull request: https://github.com/apache/spark/pull/17086#discussion_r231296074 --- Diff: mllib/src/main/scala/org/apache/spark/mllib/evaluation/MulticlassMetrics.scala --- @@ -39,21 +46,28 @@ class MulticlassMetrics @Since

[GitHub] spark pull request #17086: [SPARK-24101][ML][MLLIB] ML Evaluators should use...

2018-11-06 Thread imatiach-msft
Github user imatiach-msft commented on a diff in the pull request: https://github.com/apache/spark/pull/17086#discussion_r231294624 --- Diff: mllib/src/main/scala/org/apache/spark/mllib/evaluation/MulticlassMetrics.scala --- @@ -39,21 +46,28 @@ class MulticlassMetrics @Since

[GitHub] spark pull request #17086: [SPARK-24101][ML][MLLIB] ML Evaluators should use...

2018-11-06 Thread imatiach-msft
Github user imatiach-msft commented on a diff in the pull request: https://github.com/apache/spark/pull/17086#discussion_r231273689 --- Diff: mllib/src/main/scala/org/apache/spark/mllib/evaluation/MulticlassMetrics.scala --- @@ -39,21 +46,28 @@ class MulticlassMetrics @Since

[GitHub] spark pull request #17086: [SPARK-24101][ML][MLLIB] ML Evaluators should use...

2018-11-06 Thread imatiach-msft
Github user imatiach-msft commented on a diff in the pull request: https://github.com/apache/spark/pull/17086#discussion_r231227854 --- Diff: mllib/src/main/scala/org/apache/spark/mllib/evaluation/MulticlassMetrics.scala --- @@ -27,10 +27,17 @@ import

[GitHub] spark pull request #17086: [SPARK-24101][ML][MLLIB] ML Evaluators should use...

2018-11-06 Thread imatiach-msft
Github user imatiach-msft commented on a diff in the pull request: https://github.com/apache/spark/pull/17086#discussion_r231227296 --- Diff: mllib/src/main/scala/org/apache/spark/mllib/evaluation/MulticlassMetrics.scala --- @@ -27,10 +27,17 @@ import

[GitHub] spark pull request #17086: [SPARK-24101][ML][MLLIB] ML Evaluators should use...

2018-11-06 Thread imatiach-msft
Github user imatiach-msft commented on a diff in the pull request: https://github.com/apache/spark/pull/17086#discussion_r231214136 --- Diff: mllib/src/main/scala/org/apache/spark/mllib/evaluation/MulticlassMetrics.scala --- @@ -39,21 +46,28 @@ class MulticlassMetrics @Since

[GitHub] spark pull request #17086: [SPARK-24101][ML][MLLIB] ML Evaluators should use...

2018-11-06 Thread imatiach-msft
Github user imatiach-msft commented on a diff in the pull request: https://github.com/apache/spark/pull/17086#discussion_r231214331 --- Diff: mllib/src/main/scala/org/apache/spark/mllib/evaluation/MulticlassMetrics.scala --- @@ -39,21 +46,28 @@ class MulticlassMetrics @Since

[GitHub] spark pull request #17086: [SPARK-24101][ML][MLLIB] ML Evaluators should use...

2018-11-06 Thread imatiach-msft
Github user imatiach-msft commented on a diff in the pull request: https://github.com/apache/spark/pull/17086#discussion_r231213369 --- Diff: mllib/src/main/scala/org/apache/spark/mllib/evaluation/MulticlassMetrics.scala --- @@ -27,10 +27,17 @@ import

[GitHub] spark pull request #17086: [SPARK-24101][ML][MLLIB] ML Evaluators should use...

2018-11-05 Thread imatiach-msft
Github user imatiach-msft commented on a diff in the pull request: https://github.com/apache/spark/pull/17086#discussion_r230992722 --- Diff: mllib/src/main/scala/org/apache/spark/ml/evaluation/MulticlassClassificationEvaluator.scala --- @@ -67,6 +68,10 @@ class

[GitHub] spark pull request #17086: [SPARK-24101][ML][MLLIB] ML Evaluators should use...

2018-11-05 Thread imatiach-msft
Github user imatiach-msft commented on a diff in the pull request: https://github.com/apache/spark/pull/17086#discussion_r230992384 --- Diff: mllib/src/main/scala/org/apache/spark/mllib/evaluation/MulticlassMetrics.scala --- @@ -27,10 +27,17 @@ import

[GitHub] spark pull request #17086: [SPARK-24101][ML][MLLIB] ML Evaluators should use...

2018-11-05 Thread imatiach-msft
Github user imatiach-msft commented on a diff in the pull request: https://github.com/apache/spark/pull/17086#discussion_r230992285 --- Diff: mllib/src/test/scala/org/apache/spark/mllib/evaluation/MulticlassMetricsSuite.scala --- @@ -18,10 +18,14 @@ package

[GitHub] spark issue #22087: [SPARK-25097][ML] Support prediction on single instance ...

2018-11-01 Thread imatiach-msft
Github user imatiach-msft commented on the issue: https://github.com/apache/spark/pull/22087 @zhengruifeng thanks, the PR looks good to me, maybe @felixcheung or @jkbradley can review and possibly merge

[GitHub] spark issue #21632: [SPARK-19591][ML][MLlib] Add sample weights to decision ...

2018-10-23 Thread imatiach-msft
Github user imatiach-msft commented on the issue: https://github.com/apache/spark/pull/21632 @holdenk @sethah @HyukjinKwon @jkbradley ping... could you please take another look at this PR? I've updated it to latest master again. Thank you

[GitHub] spark issue #21632: [SPARK-19591][ML][MLlib] Add sample weights to decision ...

2018-10-23 Thread imatiach-msft
Github user imatiach-msft commented on the issue: https://github.com/apache/spark/pull/21632 jenkins retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e

[GitHub] spark issue #21632: [SPARK-19591][ML][MLlib] Add sample weights to decision ...

2018-10-23 Thread imatiach-msft
Github user imatiach-msft commented on the issue: https://github.com/apache/spark/pull/21632 jenkins retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e

[GitHub] spark issue #22675: [SPARK-25347][ML][DOC] Spark datasource for image/libsvm...

2018-10-22 Thread imatiach-msft
Github user imatiach-msft commented on the issue: https://github.com/apache/spark/pull/22675 LGTM, this is great to have! --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e

[GitHub] spark pull request #22675: [SPARK-25347][ML][DOC] Spark datasource for image...

2018-10-22 Thread imatiach-msft
Github user imatiach-msft commented on a diff in the pull request: https://github.com/apache/spark/pull/22675#discussion_r227214199 --- Diff: docs/ml-datasource.md --- @@ -0,0 +1,113 @@ +--- +layout: global +title: Data sources +displayTitle: Data sources

[GitHub] spark issue #21632: [SPARK-19591][ML][MLlib] Add sample weights to decision ...

2018-10-22 Thread imatiach-msft
Github user imatiach-msft commented on the issue: https://github.com/apache/spark/pull/21632 jenkins retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e

[GitHub] spark issue #21632: [SPARK-19591][ML][MLlib] Add sample weights to decision ...

2018-10-22 Thread imatiach-msft
Github user imatiach-msft commented on the issue: https://github.com/apache/spark/pull/21632 jenkins retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e

[GitHub] spark issue #21632: [SPARK-19591][ML][MLlib] Add sample weights to decision ...

2018-10-22 Thread imatiach-msft
Github user imatiach-msft commented on the issue: https://github.com/apache/spark/pull/21632 jenkins retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e

[GitHub] spark pull request #22675: [SPARK-25347][ML][DOC] Spark datasource for image...

2018-10-22 Thread imatiach-msft
Github user imatiach-msft commented on a diff in the pull request: https://github.com/apache/spark/pull/22675#discussion_r227062660 --- Diff: docs/ml-datasource.md --- @@ -0,0 +1,113 @@ +--- +layout: global +title: Data sources +displayTitle: Data sources

[GitHub] spark pull request #22675: [SPARK-25347][ML][DOC] Spark datasource for image...

2018-10-22 Thread imatiach-msft
Github user imatiach-msft commented on a diff in the pull request: https://github.com/apache/spark/pull/22675#discussion_r227062385 --- Diff: docs/ml-datasource.md --- @@ -0,0 +1,113 @@ +--- +layout: global +title: Data sources +displayTitle: Data sources

[GitHub] spark issue #22675: [SPARK-25347][ML][DOC] Spark datasource for image/libsvm...

2018-10-22 Thread imatiach-msft
Github user imatiach-msft commented on the issue: https://github.com/apache/spark/pull/22675 @WeichenXu123 this looks good - one recommendation though, in the examples it might be good to pass the param that ignores bad images so that the not-image.txt is not included in the output

[GitHub] spark pull request #22675: [SPARK-25347][ML][DOC] Spark datasource for image...

2018-10-22 Thread imatiach-msft
Github user imatiach-msft commented on a diff in the pull request: https://github.com/apache/spark/pull/22675#discussion_r227062204 --- Diff: docs/ml-datasource.md --- @@ -0,0 +1,113 @@ +--- +layout: global +title: Data sources +displayTitle: Data sources

[GitHub] spark issue #21632: [SPARK-19591][ML][MLlib] Add sample weights to decision ...

2018-10-02 Thread imatiach-msft
Github user imatiach-msft commented on the issue: https://github.com/apache/spark/pull/21632 @holdenk @jkbradley ping... could you please take a look at this PR? Thank you! --- - To unsubscribe, e-mail: reviews

[GitHub] spark issue #20168: [SPARK-22730][ML] Add ImageSchema support for all OpenCv...

2018-09-27 Thread imatiach-msft
Github user imatiach-msft commented on the issue: https://github.com/apache/spark/pull/20168 @tomasatdatabricks @MrBago @WeichenXu123 sorry, any updates on this PR? It has been a while. --- - To unsubscribe, e

[GitHub] spark pull request #22087: [SPARK-25097][ML] Support prediction on single in...

2018-09-26 Thread imatiach-msft
Github user imatiach-msft commented on a diff in the pull request: https://github.com/apache/spark/pull/22087#discussion_r220779540 --- Diff: mllib/src/test/scala/org/apache/spark/ml/clustering/GaussianMixtureSuite.scala --- @@ -268,6 +268,13 @@ class GaussianMixtureSuite extends

[GitHub] spark issue #17086: [SPARK-24101][ML][MLLIB] ML Evaluators should use weight...

2018-09-26 Thread imatiach-msft
Github user imatiach-msft commented on the issue: https://github.com/apache/spark/pull/17086 I've updated the PR to latest master --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #17086: [SPARK-24101][ML][MLLIB] ML Evaluators should use weight...

2018-09-26 Thread imatiach-msft
Github user imatiach-msft commented on the issue: https://github.com/apache/spark/pull/17086 @jkbradley @mengxr @HyukjinKwon ping, would you have time to take a look? Thank you! --- - To unsubscribe, e-mail

[GitHub] spark issue #21632: [SPARK-19591][ML][MLlib] Add sample weights to decision ...

2018-09-26 Thread imatiach-msft
Github user imatiach-msft commented on the issue: https://github.com/apache/spark/pull/21632 @holdenk @sethah @HyukjinKwon @jkbradley I've fixed all known issues in the PR and did some small cleanup, could you please review this PR? Thank you

[GitHub] spark pull request #21632: [SPARK-19591][ML][MLlib] Add sample weights to de...

2018-09-25 Thread imatiach-msft
Github user imatiach-msft commented on a diff in the pull request: https://github.com/apache/spark/pull/21632#discussion_r220413827 --- Diff: mllib/src/test/scala/org/apache/spark/ml/tree/impl/RandomForestSuite.scala --- @@ -700,6 +722,82 @@ class RandomForestSuite extends

[GitHub] spark pull request #21632: [SPARK-19591][ML][MLlib] Add sample weights to de...

2018-09-25 Thread imatiach-msft
Github user imatiach-msft commented on a diff in the pull request: https://github.com/apache/spark/pull/21632#discussion_r220413857 --- Diff: mllib/src/test/scala/org/apache/spark/ml/regression/DecisionTreeRegressorSuite.scala --- @@ -159,6 +169,31 @@ class

[GitHub] spark pull request #21632: [SPARK-19591][ML][MLlib] Add sample weights to de...

2018-09-18 Thread imatiach-msft
Github user imatiach-msft commented on a diff in the pull request: https://github.com/apache/spark/pull/21632#discussion_r218657097 --- Diff: mllib/src/main/scala/org/apache/spark/ml/classification/DecisionTreeClassifier.scala --- @@ -66,6 +69,9 @@ class DecisionTreeClassifier

[GitHub] spark pull request #21632: [SPARK-19591][ML][MLlib] Add sample weights to de...

2018-09-18 Thread imatiach-msft
Github user imatiach-msft commented on a diff in the pull request: https://github.com/apache/spark/pull/21632#discussion_r218657039 --- Diff: mllib/src/main/scala/org/apache/spark/ml/regression/DecisionTreeRegressor.scala --- @@ -65,6 +68,9 @@ class DecisionTreeRegressor @Since

[GitHub] spark pull request #21632: [SPARK-19591][ML][MLlib] Add sample weights to de...

2018-09-18 Thread imatiach-msft
Github user imatiach-msft commented on a diff in the pull request: https://github.com/apache/spark/pull/21632#discussion_r218657065 --- Diff: mllib/src/main/scala/org/apache/spark/ml/classification/DecisionTreeClassifier.scala --- @@ -97,28 +103,48 @@ class DecisionTreeClassifier

[GitHub] spark pull request #22356: [SPARK-22357][CORE][FOLLOWUP] SparkContext.binary...

2018-09-06 Thread imatiach-msft
Github user imatiach-msft commented on a diff in the pull request: https://github.com/apache/spark/pull/22356#discussion_r215749840 --- Diff: core/src/test/scala/org/apache/spark/FileSuite.scala --- @@ -299,6 +301,25 @@ class FileSuite extends SparkFunSuite with LocalSparkContext

[GitHub] spark pull request #22356: [SPARK-22357][CORE][FOLLOWUP] SparkContext.binary...

2018-09-06 Thread imatiach-msft
Github user imatiach-msft commented on a diff in the pull request: https://github.com/apache/spark/pull/22356#discussion_r215730938 --- Diff: core/src/test/scala/org/apache/spark/FileSuite.scala --- @@ -299,6 +301,25 @@ class FileSuite extends SparkFunSuite with LocalSparkContext

[GitHub] spark pull request #22356: [SPARK-22357][CORE][FOLLOWUP] SparkContext.binary...

2018-09-06 Thread imatiach-msft
Github user imatiach-msft commented on a diff in the pull request: https://github.com/apache/spark/pull/22356#discussion_r215730668 --- Diff: core/src/test/scala/org/apache/spark/FileSuite.scala --- @@ -299,6 +301,25 @@ class FileSuite extends SparkFunSuite with LocalSparkContext

[GitHub] spark pull request #22328: [SPARK-22666][ML][SQL] Spark datasource for image...

2018-09-04 Thread imatiach-msft
Github user imatiach-msft commented on a diff in the pull request: https://github.com/apache/spark/pull/22328#discussion_r215000161 --- Diff: data/mllib/images/images/license.txt --- @@ -0,0 +1,13 @@ +The images in the folder "kittens" are under the creative c

[GitHub] spark issue #22328: [SPARK-22666][ML][SQL] Spark datasource for image format

2018-09-04 Thread imatiach-msft
Github user imatiach-msft commented on the issue: https://github.com/apache/spark/pull/22328 @mhamilton723 could you take a look at this PR? Mark added some performance improvements in MMLSpark that we wanted to merge in and he also added support for streaming (this was one

[GitHub] spark issue #22328: [SPARK-22666][ML][SQL] Spark datasource for image format

2018-09-04 Thread imatiach-msft
Github user imatiach-msft commented on the issue: https://github.com/apache/spark/pull/22328 I don't think you need duplicate images in the PR - in the old tests you can just specify the path up to the cls/date folder. There might be a few minor changes to the tests but I think

[GitHub] spark issue #22328: [SPARK-22666][ML][SQL] Spark datasource for image format

2018-09-04 Thread imatiach-msft
Github user imatiach-msft commented on the issue: https://github.com/apache/spark/pull/22328 @mengxr "is to mark ImageSchema deprecated in 2.4 and remove it in 3.0" confused, this is still using ImageSchema though, right? You mean to remove the top-level read

[GitHub] spark issue #22328: [SPARK-22666][ML][SQL] Spark datasource for image format

2018-09-04 Thread imatiach-msft
Github user imatiach-msft commented on the issue: https://github.com/apache/spark/pull/22328 "specify numPartitions (it will be determined by datasource automatically)" This feature was because of this bug: https://issues.apache.org/jira/browse/SPARK-22357

[GitHub] spark pull request #22328: [SPARK-22666][ML][SQL] Spark datasource for image...

2018-09-04 Thread imatiach-msft
Github user imatiach-msft commented on a diff in the pull request: https://github.com/apache/spark/pull/22328#discussion_r214975055 --- Diff: mllib/src/main/scala/org/apache/spark/ml/source/image/ImageFileFormat.scala --- @@ -0,0 +1,109 @@ +/* + * Licensed to the Apache

[GitHub] spark pull request #22328: [SPARK-22666][ML][SQL] Spark datasource for image...

2018-09-04 Thread imatiach-msft
Github user imatiach-msft commented on a diff in the pull request: https://github.com/apache/spark/pull/22328#discussion_r214973718 --- Diff: mllib/src/test/scala/org/apache/spark/ml/source/image/ImageFileFormatSuite.scala --- @@ -0,0 +1,119 @@ +/* + * Licensed

[GitHub] spark pull request #22328: [SPARK-22666][ML][SQL] Spark datasource for image...

2018-09-04 Thread imatiach-msft
Github user imatiach-msft commented on a diff in the pull request: https://github.com/apache/spark/pull/22328#discussion_r214973111 --- Diff: mllib/src/main/scala/org/apache/spark/ml/source/image/ImageFileFormat.scala --- @@ -0,0 +1,109 @@ +/* + * Licensed to the Apache

[GitHub] spark pull request #22328: [SPARK-22666][ML][SQL] Spark datasource for image...

2018-09-04 Thread imatiach-msft
Github user imatiach-msft commented on a diff in the pull request: https://github.com/apache/spark/pull/22328#discussion_r214970271 --- Diff: mllib/src/main/scala/org/apache/spark/ml/source/image/ImageDataSource.scala --- @@ -0,0 +1,51 @@ +/* + * Licensed to the Apache

[GitHub] spark pull request #22328: [SPARK-22666][ML][SQL] Spark datasource for image...

2018-09-04 Thread imatiach-msft
Github user imatiach-msft commented on a diff in the pull request: https://github.com/apache/spark/pull/22328#discussion_r214969396 --- Diff: data/mllib/images/images/license.txt --- @@ -0,0 +1,13 @@ +The images in the folder "kittens" are under the creative c

[GitHub] spark pull request #22328: [SPARK-22666][ML][SQL] Spark datasource for image...

2018-09-04 Thread imatiach-msft
Github user imatiach-msft commented on a diff in the pull request: https://github.com/apache/spark/pull/22328#discussion_r214971542 --- Diff: mllib/src/main/scala/org/apache/spark/ml/source/image/ImageDataSource.scala --- @@ -0,0 +1,51 @@ +/* + * Licensed to the Apache

[GitHub] spark issue #22328: [SPARK-22666][ML][SQL] Spark datasource for image format

2018-09-04 Thread imatiach-msft
Github user imatiach-msft commented on the issue: https://github.com/apache/spark/pull/22328 question: why were the images renamed to images/images folder from just images? Seems a bit strange to me to have the same folder name twice

[GitHub] spark issue #17086: [SPARK-24101][ML][MLLIB] ML Evaluators should use weight...

2018-08-14 Thread imatiach-msft
Github user imatiach-msft commented on the issue: https://github.com/apache/spark/pull/17086 @jkbradley @mengxr ping, would you have time to take a look? Thank you! --- - To unsubscribe, e-mail: reviews-unsubscr

[GitHub] spark pull request #21632: [SPARK-19591][ML][MLlib] Add sample weights to de...

2018-08-14 Thread imatiach-msft
Github user imatiach-msft commented on a diff in the pull request: https://github.com/apache/spark/pull/21632#discussion_r209839725 --- Diff: mllib/src/main/scala/org/apache/spark/ml/classification/DecisionTreeClassifier.scala --- @@ -66,6 +69,9 @@ class DecisionTreeClassifier

[GitHub] spark pull request #21632: [SPARK-19591][ML][MLlib] Add sample weights to de...

2018-08-14 Thread imatiach-msft
Github user imatiach-msft commented on a diff in the pull request: https://github.com/apache/spark/pull/21632#discussion_r209839825 --- Diff: mllib/src/main/scala/org/apache/spark/ml/regression/DecisionTreeRegressor.scala --- @@ -65,6 +68,9 @@ class DecisionTreeRegressor @Since

[GitHub] spark pull request #21632: [SPARK-19591][ML][MLlib] Add sample weights to de...

2018-08-14 Thread imatiach-msft
Github user imatiach-msft commented on a diff in the pull request: https://github.com/apache/spark/pull/21632#discussion_r209839752 --- Diff: mllib/src/main/scala/org/apache/spark/ml/classification/DecisionTreeClassifier.scala --- @@ -97,28 +103,48 @@ class DecisionTreeClassifier

[GitHub] spark pull request #21632: [SPARK-19591][ML][MLlib] Add sample weights to de...

2018-08-14 Thread imatiach-msft
Github user imatiach-msft commented on a diff in the pull request: https://github.com/apache/spark/pull/21632#discussion_r209840163 --- Diff: mllib/src/test/scala/org/apache/spark/ml/tree/impl/RandomForestSuite.scala --- @@ -700,6 +722,82 @@ class RandomForestSuite extends

[GitHub] spark pull request #21632: [SPARK-19591][ML][MLlib] Add sample weights to de...

2018-08-14 Thread imatiach-msft
Github user imatiach-msft commented on a diff in the pull request: https://github.com/apache/spark/pull/21632#discussion_r209840017 --- Diff: mllib/src/test/scala/org/apache/spark/ml/regression/DecisionTreeRegressorSuite.scala --- @@ -159,6 +169,31 @@ class

[GitHub] spark issue #21632: [SPARK-19591][ML][MLlib] Add sample weights to decision ...

2018-08-06 Thread imatiach-msft
Github user imatiach-msft commented on the issue: https://github.com/apache/spark/pull/21632 @holdenk @sethah @HyukjinKwon I have a successful build, I need to look into 2-3 wacky test results that changed since when @sethah opened his PR (see comments in my PR). In the mean time

[GitHub] spark issue #21632: [SPARK-19591][ML][MLlib] Add sample weights to decision ...

2018-08-06 Thread imatiach-msft
Github user imatiach-msft commented on the issue: https://github.com/apache/spark/pull/21632 looks like a random test failure with hive client suite (not related to the PR), I'll try updating to latest master and rebuild

[GitHub] spark issue #21632: [SPARK-19591][ML][MLlib] Add sample weights to decision ...

2018-08-03 Thread imatiach-msft
Github user imatiach-msft commented on the issue: https://github.com/apache/spark/pull/21632 looks like a random failure --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e

[GitHub] spark issue #21632: [SPARK-19591][ML][MLlib] Add sample weights to decision ...

2018-08-03 Thread imatiach-msft
Github user imatiach-msft commented on the issue: https://github.com/apache/spark/pull/21632 Jenkins retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e

[GitHub] spark issue #16722: [SPARK-19591][ML][MLlib] Add sample weights to decision ...

2018-08-01 Thread imatiach-msft
Github user imatiach-msft commented on the issue: https://github.com/apache/spark/pull/16722 @HyukjinKwon yes, I've updated this PR in #21632 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org

[GitHub] spark issue #17086: [SPARK-24101][ML][MLLIB] ML Evaluators should use weight...

2018-06-27 Thread imatiach-msft
Github user imatiach-msft commented on the issue: https://github.com/apache/spark/pull/17086 @jkbradley @mengxr ping... would you be able to take a look at this PR? I received an email recently from someone who asked when this feature (weight columns in evaluators) is expected

[GitHub] spark issue #21632: [SPARK-19591][ML][MLlib] Add sample weights to decision ...

2018-06-25 Thread imatiach-msft
Github user imatiach-msft commented on the issue: https://github.com/apache/spark/pull/21632 jenkins retest this pretty please :) --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21632: [SPARK-19591][ML][MLlib] Add sample weights to decision ...

2018-06-24 Thread imatiach-msft
Github user imatiach-msft commented on the issue: https://github.com/apache/spark/pull/21632 @holdenk @sethah I've updated the PR to latest master (hopefully all of the tests still pass :) ) --- - To unsubscribe, e

[GitHub] spark pull request #21632: [SPARK-19591][ML][MLlib] Add sample weights to de...

2018-06-24 Thread imatiach-msft
GitHub user imatiach-msft opened a pull request: https://github.com/apache/spark/pull/21632 [SPARK-19591][ML][MLlib] Add sample weights to decision trees This is updated PR https://github.com/apache/spark/pull/16722 What changes were proposed in this pull request

[GitHub] spark issue #16722: [SPARK-19591][ML][MLlib] Add sample weights to decision ...

2018-06-22 Thread imatiach-msft
Github user imatiach-msft commented on the issue: https://github.com/apache/spark/pull/16722 @holdenk @sethah I'd be happy to help out in my spare time and take over the PR - it looks like the PR just needs to be updated to latest code, are there any other changes required? Thanks

[GitHub] spark issue #20168: [SPARK-22730][ML] Add ImageSchema support for all OpenCv...

2018-06-10 Thread imatiach-msft
Github user imatiach-msft commented on the issue: https://github.com/apache/spark/pull/20168 @tomasatdatabricks it looks like there are some conflicts that need to be resolved, otherwise looks good to me, can you please update the PR

[GitHub] spark issue #17086: [SPARK-24101][ML][MLLIB] ML Evaluators should use weight...

2018-06-08 Thread imatiach-msft
Github user imatiach-msft commented on the issue: https://github.com/apache/spark/pull/17086 @jkbradley @mengxr ping... would you be able to take a look? Thank you! --- - To unsubscribe, e-mail: reviews-unsubscr

[GitHub] spark issue #17086: [SPARK-24101][ML][MLLIB] ML Evaluators should use weight...

2018-05-25 Thread imatiach-msft
Github user imatiach-msft commented on the issue: https://github.com/apache/spark/pull/17086 @jkbradley @mengxr ping, would you have time to take a look? Thank you! --- - To unsubscribe, e-mail: reviews-unsubscr

[GitHub] spark issue #17086: [SPARK-24101][ML][MLLIB] ML Evaluators should use weight...

2018-05-05 Thread imatiach-msft
Github user imatiach-msft commented on the issue: https://github.com/apache/spark/pull/17086 jenkins, retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e

[GitHub] spark issue #17086: [SPARK-24101][ML][MLLIB] ML Evaluators should use weight...

2018-05-02 Thread imatiach-msft
Github user imatiach-msft commented on the issue: https://github.com/apache/spark/pull/17086 jenkins, retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e

[GitHub] spark issue #17086: [SPARK-24101][ML][MLLIB] ML Evaluators should use weight...

2018-05-01 Thread imatiach-msft
Github user imatiach-msft commented on the issue: https://github.com/apache/spark/pull/17086 jenkins, retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e

[GitHub] spark issue #17086: [SPARK-24101][ML][MLLIB] ML Evaluators should use weight...

2018-05-01 Thread imatiach-msft
Github user imatiach-msft commented on the issue: https://github.com/apache/spark/pull/17086 looks like a random failure in spark R, unrelated --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org

[GitHub] spark issue #17086: [SPARK-24101][ML][MLLIB] ML Evaluators should use weight...

2018-05-01 Thread imatiach-msft
Github user imatiach-msft commented on the issue: https://github.com/apache/spark/pull/17086 jenkins, retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e

[GitHub] spark issue #17086: [SPARK-24101][ML][MLLIB] ML Evaluators should use weight...

2018-05-01 Thread imatiach-msft
Github user imatiach-msft commented on the issue: https://github.com/apache/spark/pull/17086 jenkins, retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e

[GitHub] spark pull request #17086: [SPARK-24101][ML][MLLIB] ML Evaluators should use...

2018-04-30 Thread imatiach-msft
Github user imatiach-msft commented on a diff in the pull request: https://github.com/apache/spark/pull/17086#discussion_r185163922 --- Diff: mllib/src/test/scala/org/apache/spark/mllib/evaluation/MulticlassMetricsSuite.scala --- @@ -95,4 +95,95 @@ class MulticlassMetricsSuite

[GitHub] spark pull request #17086: [SPARK-24101][ML][MLLIB] ML Evaluators should use...

2018-04-30 Thread imatiach-msft
Github user imatiach-msft commented on a diff in the pull request: https://github.com/apache/spark/pull/17086#discussion_r185163674 --- Diff: mllib/src/test/scala/org/apache/spark/mllib/evaluation/MulticlassMetricsSuite.scala --- @@ -55,44 +60,128 @@ class MulticlassMetricsSuite

[GitHub] spark pull request #17086: [SPARK-24101][ML][MLLIB] ML Evaluators should use...

2018-04-30 Thread imatiach-msft
Github user imatiach-msft commented on a diff in the pull request: https://github.com/apache/spark/pull/17086#discussion_r185163483 --- Diff: mllib/src/test/scala/org/apache/spark/mllib/evaluation/MulticlassMetricsSuite.scala --- @@ -95,4 +95,95 @@ class MulticlassMetricsSuite

[GitHub] spark pull request #17086: [SPARK-24101][ML][MLLIB] ML Evaluators should use...

2018-04-26 Thread imatiach-msft
Github user imatiach-msft commented on a diff in the pull request: https://github.com/apache/spark/pull/17086#discussion_r184454635 --- Diff: mllib/src/test/scala/org/apache/spark/mllib/evaluation/MulticlassMetricsSuite.scala --- @@ -95,4 +95,95 @@ class MulticlassMetricsSuite

[GitHub] spark pull request #17086: [SPARK-18693][ML][MLLIB] ML Evaluators should use...

2018-04-26 Thread imatiach-msft
Github user imatiach-msft commented on a diff in the pull request: https://github.com/apache/spark/pull/17086#discussion_r184431022 --- Diff: mllib/src/test/scala/org/apache/spark/mllib/evaluation/MulticlassMetricsSuite.scala --- @@ -95,4 +95,95 @@ class MulticlassMetricsSuite

[GitHub] spark pull request #17086: [SPARK-18693][ML][MLLIB] ML Evaluators should use...

2018-04-26 Thread imatiach-msft
Github user imatiach-msft commented on a diff in the pull request: https://github.com/apache/spark/pull/17086#discussion_r184430271 --- Diff: mllib/src/test/scala/org/apache/spark/mllib/evaluation/MulticlassMetricsSuite.scala --- @@ -95,4 +95,95 @@ class MulticlassMetricsSuite

[GitHub] spark pull request #17086: [SPARK-18693][ML][MLLIB] ML Evaluators should use...

2018-04-26 Thread imatiach-msft
Github user imatiach-msft commented on a diff in the pull request: https://github.com/apache/spark/pull/17086#discussion_r184430052 --- Diff: mllib/src/test/scala/org/apache/spark/mllib/evaluation/MulticlassMetricsSuite.scala --- @@ -95,4 +95,95 @@ class MulticlassMetricsSuite

[GitHub] spark pull request #17086: [SPARK-18693][ML][MLLIB] ML Evaluators should use...

2018-04-26 Thread imatiach-msft
Github user imatiach-msft commented on a diff in the pull request: https://github.com/apache/spark/pull/17086#discussion_r184425723 --- Diff: mllib/src/test/scala/org/apache/spark/mllib/evaluation/MulticlassMetricsSuite.scala --- @@ -95,4 +95,95 @@ class MulticlassMetricsSuite

[GitHub] spark pull request #17086: [SPARK-18693][ML][MLLIB] ML Evaluators should use...

2018-04-26 Thread imatiach-msft
Github user imatiach-msft commented on a diff in the pull request: https://github.com/apache/spark/pull/17086#discussion_r184420934 --- Diff: mllib/src/test/scala/org/apache/spark/mllib/evaluation/MulticlassMetricsSuite.scala --- @@ -95,4 +95,95 @@ class MulticlassMetricsSuite

[GitHub] spark pull request #17086: [SPARK-18693][ML][MLLIB] ML Evaluators should use...

2018-04-26 Thread imatiach-msft
Github user imatiach-msft commented on a diff in the pull request: https://github.com/apache/spark/pull/17086#discussion_r184420711 --- Diff: mllib/src/main/scala/org/apache/spark/mllib/evaluation/MulticlassMetrics.scala --- @@ -39,21 +46,28 @@ class MulticlassMetrics @Since

[GitHub] spark issue #17086: [SPARK-18693][ML][MLLIB] ML Evaluators should use weight...

2018-04-20 Thread imatiach-msft
Github user imatiach-msft commented on the issue: https://github.com/apache/spark/pull/17086 @WeichenXu123 I've updated the PR, resolved all comments and the build passes - would you be able to take another look when you have time? Thank you

[GitHub] spark pull request #17086: [SPARK-18693][ML][MLLIB] ML Evaluators should use...

2018-04-18 Thread imatiach-msft
Github user imatiach-msft commented on a diff in the pull request: https://github.com/apache/spark/pull/17086#discussion_r182632357 --- Diff: mllib/src/main/scala/org/apache/spark/mllib/evaluation/MulticlassMetrics.scala --- @@ -27,10 +27,11 @@ import

[GitHub] spark pull request #17086: [SPARK-18693][ML][MLLIB] ML Evaluators should use...

2018-04-17 Thread imatiach-msft
Github user imatiach-msft commented on a diff in the pull request: https://github.com/apache/spark/pull/17086#discussion_r182303815 --- Diff: mllib/src/main/scala/org/apache/spark/mllib/evaluation/MulticlassMetrics.scala --- @@ -27,10 +27,11 @@ import

[GitHub] spark pull request #17086: [SPARK-18693][ML][MLLIB] ML Evaluators should use...

2018-04-17 Thread imatiach-msft
Github user imatiach-msft commented on a diff in the pull request: https://github.com/apache/spark/pull/17086#discussion_r182302080 --- Diff: mllib/src/main/scala/org/apache/spark/mllib/evaluation/MulticlassMetrics.scala --- @@ -27,10 +27,11 @@ import

[GitHub] spark pull request #17086: [SPARK-18693][ML][MLLIB] ML Evaluators should use...

2018-04-17 Thread imatiach-msft
Github user imatiach-msft commented on a diff in the pull request: https://github.com/apache/spark/pull/17086#discussion_r182300738 --- Diff: mllib/src/main/scala/org/apache/spark/ml/evaluation/MulticlassClassificationEvaluator.scala --- @@ -75,11 +80,16 @@ class

  1   2   3   4   5   6   7   >