[GitHub] spark pull request #18887: [SPARK-20642][core] Store FsHistoryProvider listi...

2017-09-11 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/18887#discussion_r137942505 --- Diff: core/src/main/scala/org/apache/spark/deploy/history/FsHistoryProvider.scala --- @@ -742,53 +698,146 @@ private[history] object

[GitHub] spark pull request #18887: [SPARK-20642][core] Store FsHistoryProvider listi...

2017-09-11 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/18887#discussion_r137941077 --- Diff: core/src/main/scala/org/apache/spark/deploy/history/FsHistoryProvider.scala --- @@ -117,17 +122,37 @@ private[history] class

[GitHub] spark pull request #18887: [SPARK-20642][core] Store FsHistoryProvider listi...

2017-09-11 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/18887#discussion_r137940633 --- Diff: core/src/main/scala/org/apache/spark/deploy/history/FsHistoryProvider.scala --- @@ -117,17 +122,37 @@ private[history] class

[GitHub] spark pull request #18887: [SPARK-20642][core] Store FsHistoryProvider listi...

2017-09-11 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/18887#discussion_r137941159 --- Diff: core/src/main/scala/org/apache/spark/deploy/history/FsHistoryProvider.scala --- @@ -742,53 +698,146 @@ private[history] object

[GitHub] spark pull request #18887: [SPARK-20642][core] Store FsHistoryProvider listi...

2017-09-11 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/18887#discussion_r137942178 --- Diff: core/src/main/scala/org/apache/spark/deploy/history/FsHistoryProvider.scala --- @@ -742,53 +698,146 @@ private[history] object

[GitHub] spark pull request #18887: [SPARK-20642][core] Store FsHistoryProvider listi...

2017-09-11 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/18887#discussion_r137942487 --- Diff: core/src/main/scala/org/apache/spark/deploy/history/FsHistoryProvider.scala --- @@ -422,208 +455,101 @@ private[history] class

[GitHub] spark pull request #18887: [SPARK-20642][core] Store FsHistoryProvider listi...

2017-09-11 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/18887#discussion_r137942186 --- Diff: core/src/main/scala/org/apache/spark/deploy/history/FsHistoryProvider.scala --- @@ -742,53 +698,146 @@ private[history] object

[GitHub] spark pull request #18887: [SPARK-20642][core] Store FsHistoryProvider listi...

2017-09-11 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/18887#discussion_r137942548 --- Diff: core/src/main/scala/org/apache/spark/deploy/history/FsHistoryProvider.scala --- @@ -229,10 +254,22 @@ private[history] class

[GitHub] spark pull request #18887: [SPARK-20642][core] Store FsHistoryProvider listi...

2017-09-11 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/18887#discussion_r137942697 --- Diff: core/src/main/scala/org/apache/spark/deploy/history/FsHistoryProvider.scala --- @@ -301,9 +334,13 @@ private[history] class

[GitHub] spark issue #19118: [SPARK-21882][CORE] OutputMetrics doesn't count written ...

2017-09-11 Thread jiangxb1987
Github user jiangxb1987 commented on the issue: https://github.com/apache/spark/pull/19118 You don't have to use `saveAsHadoopFile`, just call `saveAsHadoopDataset` directly? --- - To unsubscribe, e-mail:

[GitHub] spark issue #15326: [SPARK-17759] [CORE] Avoid adding duplicate schedulables

2017-09-11 Thread erenavsarogullari
Github user erenavsarogullari commented on the issue: https://github.com/apache/spark/pull/15326 Hi @kayousterhout, Many thanks again for your review. Patch is ready to re-review. --- - To unsubscribe,

[GitHub] spark issue #19182: [SPARK-21970][Core] Fix Redundant Throws Declarations in...

2017-09-11 Thread original-brownbear
Github user original-brownbear commented on the issue: https://github.com/apache/spark/pull/19182 @srowen looks like we're all green :) --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For

[GitHub] spark issue #18982: [SPARK-21685][PYTHON][ML] PySpark Params isSet state sho...

2017-09-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18982 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/81647/ Test PASSed. ---

[GitHub] spark issue #18982: [SPARK-21685][PYTHON][ML] PySpark Params isSet state sho...

2017-09-11 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18982 **[Test build #81647 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81647/testReport)** for PR 18982 at commit

[GitHub] spark issue #18982: [SPARK-21685][PYTHON][ML] PySpark Params isSet state sho...

2017-09-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18982 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19110: [SPARK-21027][ML][PYTHON] Added tunable parallelism to o...

2017-09-11 Thread jkbradley
Github user jkbradley commented on the issue: https://github.com/apache/spark/pull/19110 Other than that 1 item, this looks ready --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #19110: [SPARK-21027][ML][PYTHON] Added tunable paralleli...

2017-09-11 Thread jkbradley
Github user jkbradley commented on a diff in the pull request: https://github.com/apache/spark/pull/19110#discussion_r138180599 --- Diff: python/pyspark/ml/param/shared.py --- @@ -608,6 +608,30 @@ def getAggregationDepth(self): return

[GitHub] spark issue #19194: [SPARK-20589] Allow limiting task concurrency per stage

2017-09-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19194 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/81645/ Test PASSed. ---

[GitHub] spark issue #19194: [SPARK-20589] Allow limiting task concurrency per stage

2017-09-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19194 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19194: [SPARK-20589] Allow limiting task concurrency per stage

2017-09-11 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19194 **[Test build #81645 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81645/testReport)** for PR 19194 at commit

[GitHub] spark issue #19193: [WIP][SPARK-21896][SQL] Fix Stack Overflow when window f...

2017-09-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19193 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19193: [WIP][SPARK-21896][SQL] Fix Stack Overflow when window f...

2017-09-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19193 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/81644/ Test FAILed. ---

[GitHub] spark issue #19193: [WIP][SPARK-21896][SQL] Fix Stack Overflow when window f...

2017-09-11 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19193 **[Test build #81644 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81644/testReport)** for PR 19193 at commit

[GitHub] spark pull request #19130: [SPARK-21917][CORE][YARN] Supporting adding http(...

2017-09-11 Thread tgravescs
Github user tgravescs commented on a diff in the pull request: https://github.com/apache/spark/pull/19130#discussion_r138154503 --- Diff: core/src/main/scala/org/apache/spark/deploy/SparkSubmit.scala --- @@ -367,6 +368,52 @@ object SparkSubmit extends CommandLineUtils with Logging

[GitHub] spark issue #18875: [SPARK-21513][SQL] Allow UDF to_json support converting ...

2017-09-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18875 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/81643/ Test PASSed. ---

[GitHub] spark issue #18875: [SPARK-21513][SQL] Allow UDF to_json support converting ...

2017-09-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18875 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #18875: [SPARK-21513][SQL] Allow UDF to_json support converting ...

2017-09-11 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18875 **[Test build #81643 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81643/testReport)** for PR 18875 at commit

[GitHub] spark pull request #19130: [SPARK-21917][CORE][YARN] Supporting adding http(...

2017-09-11 Thread tgravescs
Github user tgravescs commented on a diff in the pull request: https://github.com/apache/spark/pull/19130#discussion_r138151499 --- Diff: core/src/main/scala/org/apache/spark/deploy/SparkSubmit.scala --- @@ -367,6 +368,52 @@ object SparkSubmit extends CommandLineUtils with Logging

[GitHub] spark pull request #19122: [SPARK-21911][ML][PySpark] Parallel Model Evaluat...

2017-09-11 Thread BryanCutler
Github user BryanCutler commented on a diff in the pull request: https://github.com/apache/spark/pull/19122#discussion_r138151063 --- Diff: python/pyspark/ml/tuning.py --- @@ -193,7 +194,8 @@ class CrossValidator(Estimator, ValidatorParams, MLReadable, MLWritable): >>> lr

[GitHub] spark issue #18875: [SPARK-21513][SQL] Allow UDF to_json support converting ...

2017-09-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18875 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/81641/ Test PASSed. ---

[GitHub] spark issue #18875: [SPARK-21513][SQL] Allow UDF to_json support converting ...

2017-09-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18875 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #18875: [SPARK-21513][SQL] Allow UDF to_json support converting ...

2017-09-11 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18875 **[Test build #81641 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81641/testReport)** for PR 18875 at commit

[GitHub] spark pull request #19122: [SPARK-21911][ML][PySpark] Parallel Model Evaluat...

2017-09-11 Thread BryanCutler
Github user BryanCutler commented on a diff in the pull request: https://github.com/apache/spark/pull/19122#discussion_r138144992 --- Diff: python/pyspark/ml/param/_shared_params_code_gen.py --- @@ -152,6 +152,8 @@ def get$Name(self): ("varianceCol", "column name for

[GitHub] spark pull request #19122: [SPARK-21911][ML][PySpark] Parallel Model Evaluat...

2017-09-11 Thread BryanCutler
Github user BryanCutler commented on a diff in the pull request: https://github.com/apache/spark/pull/19122#discussion_r138144361 --- Diff: python/pyspark/ml/tuning.py --- @@ -255,18 +257,27 @@ def _fit(self, dataset): randCol = self.uid + "_rand" df =

[GitHub] spark pull request #19122: [SPARK-21911][ML][PySpark] Parallel Model Evaluat...

2017-09-11 Thread BryanCutler
Github user BryanCutler commented on a diff in the pull request: https://github.com/apache/spark/pull/19122#discussion_r138142834 --- Diff: python/pyspark/ml/tuning.py --- @@ -208,23 +210,23 @@ class CrossValidator(Estimator, ValidatorParams, MLReadable, MLWritable):

[GitHub] spark issue #19186: [SPARK-21972][ML] Add param handlePersistence

2017-09-11 Thread smurching
Github user smurching commented on the issue: https://github.com/apache/spark/pull/19186 Note: This PR follows up on the work/discussions in [https://github.com/apache/spark/pull/17014](https://github.com/apache/spark/pull/17014) ---

[GitHub] spark pull request #19186: [SPARK-21972][ML] Add param handlePersistence

2017-09-11 Thread smurching
Github user smurching commented on a diff in the pull request: https://github.com/apache/spark/pull/19186#discussion_r138139729 --- Diff: mllib/src/main/scala/org/apache/spark/ml/clustering/KMeans.scala --- @@ -300,20 +300,23 @@ class KMeans @Since("1.5.0") (

[GitHub] spark pull request #19186: [SPARK-21972][ML] Add param handlePersistence

2017-09-11 Thread smurching
Github user smurching commented on a diff in the pull request: https://github.com/apache/spark/pull/19186#discussion_r138136774 --- Diff: mllib/src/main/scala/org/apache/spark/ml/classification/LogisticRegression.scala --- @@ -483,24 +488,17 @@ class LogisticRegression

[GitHub] spark pull request #19186: [SPARK-21972][ML] Add param handlePersistence

2017-09-11 Thread smurching
Github user smurching commented on a diff in the pull request: https://github.com/apache/spark/pull/19186#discussion_r138137893 --- Diff: mllib/src/main/scala/org/apache/spark/ml/classification/OneVsRest.scala --- @@ -163,9 +165,7 @@ final class OneVsRestModel private[ml] (

[GitHub] spark pull request #19186: [SPARK-21972][ML] Add param handlePersistence

2017-09-11 Thread smurching
Github user smurching commented on a diff in the pull request: https://github.com/apache/spark/pull/19186#discussion_r138139091 --- Diff: mllib/src/main/scala/org/apache/spark/ml/param/shared/SharedParamsCodeGen.scala --- @@ -82,7 +82,8 @@ private[shared] object

[GitHub] spark pull request #19186: [SPARK-21972][ML] Add param handlePersistence

2017-09-11 Thread smurching
Github user smurching commented on a diff in the pull request: https://github.com/apache/spark/pull/19186#discussion_r138140113 --- Diff: mllib/src/main/scala/org/apache/spark/ml/regression/IsotonicRegression.scala --- @@ -165,8 +170,7 @@ class IsotonicRegression @Since("1.5.0")

[GitHub] spark pull request #19186: [SPARK-21972][ML] Add param handlePersistence

2017-09-11 Thread smurching
Github user smurching commented on a diff in the pull request: https://github.com/apache/spark/pull/19186#discussion_r138139539 --- Diff: mllib/src/main/scala/org/apache/spark/mllib/classification/LogisticRegression.scala --- @@ -444,13 +444,13 @@ class

[GitHub] spark issue #19134: [SPARK-21893][BUILD][STREAMING][WIP] Put Kafka 0.8 behin...

2017-09-11 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/19134 OK by me, if it's all on-purpose. So, am I right that all of the Pyspark Kafka integration is effectively deprecated as of this change, because it depends on now-deprecated 0.8 support? just want to

[GitHub] spark issue #19134: [SPARK-21893][BUILD][STREAMING][WIP] Put Kafka 0.8 behin...

2017-09-11 Thread koeninger
Github user koeninger commented on the issue: https://github.com/apache/spark/pull/19134 There's already a jira about why 0.10 doesn't have python support, https://issues-test.apache.org/jira/browse/SPARK-16534 ---

[GitHub] spark pull request #19106: [SPARK-21770][ML] ProbabilisticClassificationMode...

2017-09-11 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/19106#discussion_r138130375 --- Diff: mllib/src/main/scala/org/apache/spark/ml/classification/ProbabilisticClassifier.scala --- @@ -245,6 +245,10 @@ private[ml] object

[GitHub] spark pull request #19106: [SPARK-21770][ML] ProbabilisticClassificationMode...

2017-09-11 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/19106#discussion_r138135778 --- Diff: mllib/src/main/scala/org/apache/spark/ml/classification/ProbabilisticClassifier.scala --- @@ -245,6 +245,10 @@ private[ml] object

[GitHub] spark issue #19195: [DOCS] Fix unreachable links in the document

2017-09-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19195 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19195: [DOCS] Fix unreachable links in the document

2017-09-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19195 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/81646/ Test PASSed. ---

[GitHub] spark issue #19195: [DOCS] Fix unreachable links in the document

2017-09-11 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19195 **[Test build #81646 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81646/testReport)** for PR 19195 at commit

[GitHub] spark issue #18982: [SPARK-21685][PYTHON][ML] PySpark Params isSet state sho...

2017-09-11 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18982 **[Test build #81647 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81647/testReport)** for PR 18982 at commit

[GitHub] spark issue #18982: [SPARK-21685][PYTHON][ML] PySpark Params isSet state sho...

2017-09-11 Thread BryanCutler
Github user BryanCutler commented on the issue: https://github.com/apache/spark/pull/18982 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #16158: [SPARK-18724][ML] Add TuningSummary for TrainValidationS...

2017-09-11 Thread hhbyyh
Github user hhbyyh commented on the issue: https://github.com/apache/spark/pull/16158 Update: To support pipeline estimator, change the tuning summary column name to include full param reference:

[GitHub] spark pull request #16158: [SPARK-18724][ML] Add TuningSummary for TrainVali...

2017-09-11 Thread hhbyyh
Github user hhbyyh commented on a diff in the pull request: https://github.com/apache/spark/pull/16158#discussion_r138133273 --- Diff: mllib/src/main/scala/org/apache/spark/ml/tuning/ValidatorParams.scala --- @@ -85,6 +86,32 @@ private[ml] trait ValidatorParams extends HasSeed

[GitHub] spark pull request #16158: [SPARK-18724][ML] Add TuningSummary for TrainVali...

2017-09-11 Thread hhbyyh
Github user hhbyyh commented on a diff in the pull request: https://github.com/apache/spark/pull/16158#discussion_r138133238 --- Diff: mllib/src/main/scala/org/apache/spark/ml/tuning/ValidatorParams.scala --- @@ -85,6 +86,32 @@ private[ml] trait ValidatorParams extends HasSeed

[GitHub] spark issue #18887: [SPARK-20642][core] Store FsHistoryProvider listing data...

2017-09-11 Thread vanzin
Github user vanzin commented on the issue: https://github.com/apache/spark/pull/18887 > What is the migration proposal/guides? Not sure what you mean. There's no change in behavior by default, so there's no migration of anything needed. > What should users do when

[GitHub] spark issue #19195: [DOCS] Fix unreachable links in the document

2017-09-11 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19195 **[Test build #81646 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81646/testReport)** for PR 19195 at commit

[GitHub] spark issue #19107: [SPARK-21799][ML] Fix `KMeans` performance regression ca...

2017-09-11 Thread smurching
Github user smurching commented on the issue: https://github.com/apache/spark/pull/19107 @jkbradley would you be able to give this a look? Thanks! --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org

[GitHub] spark pull request #19195: [DOCS] Fix unreachable links in the document

2017-09-11 Thread sarutak
GitHub user sarutak opened a pull request: https://github.com/apache/spark/pull/19195 [DOCS] Fix unreachable links in the document ## What changes were proposed in this pull request? Recently, I found two unreachable links in the document and fixed them. Because of

[GitHub] spark issue #19106: [SPARK-21770][ML] ProbabilisticClassificationModel fix c...

2017-09-11 Thread smurching
Github user smurching commented on the issue: https://github.com/apache/spark/pull/19106 This looks good to me! @srowen would you be able to give it another look? --- - To unsubscribe, e-mail:

[GitHub] spark issue #19134: [SPARK-21893][BUILD][STREAMING][WIP] Put Kafka 0.8 behin...

2017-09-11 Thread vanzin
Github user vanzin commented on the issue: https://github.com/apache/spark/pull/19134 LGTM. The pyspark vs. 0.10 discussion seems like it belongs in a separate bug. --- - To unsubscribe, e-mail:

[GitHub] spark issue #16578: [SPARK-4502][SQL] Parquet nested column pruning

2017-09-11 Thread mallman
Github user mallman commented on the issue: https://github.com/apache/spark/pull/16578 > NPE is a problem though. any luck isolating that further? I can push a commit that prevents it, I just can't say for certain it's a proper fix versus a mere workaround. I'd like

[GitHub] spark issue #19194: [SPARK-20589] Allow limiting task concurrency per stage

2017-09-11 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19194 **[Test build #81645 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81645/testReport)** for PR 19194 at commit

[GitHub] spark issue #19194: [SPARK-20589] Allow limiting task concurrency per stage

2017-09-11 Thread dhruve
Github user dhruve commented on the issue: https://github.com/apache/spark/pull/19194 Rebased this PR with current master and have squashed the earlier commits. --- - To unsubscribe, e-mail:

[GitHub] spark pull request #19194: [SPARK-20589] Allow limiting task concurrency per...

2017-09-11 Thread dhruve
GitHub user dhruve opened a pull request: https://github.com/apache/spark/pull/19194 [SPARK-20589] Allow limiting task concurrency per stage ## What changes were proposed in this pull request? This change allows the user to specify the maximum no. of tasks running in a given

[GitHub] spark pull request #19157: [SPARK-20589][Core][Scheduler] Allow limiting tas...

2017-09-11 Thread dhruve
Github user dhruve closed the pull request at: https://github.com/apache/spark/pull/19157 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #19193: [WIP][SPARK-21896][SQL] Fix Stack Overflow when window f...

2017-09-11 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19193 **[Test build #81644 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81644/testReport)** for PR 19193 at commit

[GitHub] spark pull request #19193: [WIP][SPARK-21896][SQL] Fix Stack Overflow when w...

2017-09-11 Thread aokolnychyi
GitHub user aokolnychyi opened a pull request: https://github.com/apache/spark/pull/19193 [WIP][SPARK-21896][SQL] Fix Stack Overflow when window function is nested inside an aggregate function ## What changes were proposed in this pull request? This WIP PR contains a

[GitHub] spark pull request #19136: [DO NOT MERGE][SPARK-15689][SQL] data source v2

2017-09-11 Thread rdblue
Github user rdblue commented on a diff in the pull request: https://github.com/apache/spark/pull/19136#discussion_r138123207 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/v2/DataSourceV2Strategy.scala --- @@ -0,0 +1,95 @@ +/* + * Licensed to

[GitHub] spark issue #19192: Quote table names in JDBC queries

2017-09-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19192 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #19192: Quote table names in JDBC queries

2017-09-11 Thread alexmojaki
GitHub user alexmojaki opened a pull request: https://github.com/apache/spark/pull/19192 Quote table names in JDBC queries In MySQL table names sometimes need to be quoted. This should fix that, although I haven't tested (I really don't want to go through the build and

[GitHub] spark pull request #18973: [SPARK-21765] Set isStreaming on leaf nodes for s...

2017-09-11 Thread joseph-torres
Github user joseph-torres commented on a diff in the pull request: https://github.com/apache/spark/pull/18973#discussion_r138114144 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/logical/basicLogicalOperators.scala --- @@ -443,7 +444,8 @@ case class

[GitHub] spark issue #19191: [SPARK-21958][ML] Word2VecModel save: transform data in ...

2017-09-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19191 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19185: [Spark-21854] Added LogisticRegressionTrainingSummary fo...

2017-09-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19185 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19185: [Spark-21854] Added LogisticRegressionTrainingSummary fo...

2017-09-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19185 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/81642/ Test PASSed. ---

[GitHub] spark issue #19185: [Spark-21854] Added LogisticRegressionTrainingSummary fo...

2017-09-11 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19185 **[Test build #81642 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81642/testReport)** for PR 19185 at commit

[GitHub] spark pull request #13067: [SPARK-4131] [SQL] Support INSERT OVERWRITE [LOCA...

2017-09-11 Thread Parth-Brahmbhatt
Github user Parth-Brahmbhatt closed the pull request at: https://github.com/apache/spark/pull/13067 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark pull request #19191: [SPARK-21958][ML] Word2VecModel save: transform d...

2017-09-11 Thread travishegner
GitHub user travishegner opened a pull request: https://github.com/apache/spark/pull/19191 [SPARK-21958][ML] Word2VecModel save: transform data in the cluster ## What changes were proposed in this pull request? Change a data transformation while saving a Word2VecModel to

[GitHub] spark issue #18875: [SPARK-21513][SQL] Allow UDF to_json support converting ...

2017-09-11 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18875 **[Test build #81643 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81643/testReport)** for PR 18875 at commit

[GitHub] spark pull request #18875: [SPARK-21513][SQL] Allow UDF to_json support conv...

2017-09-11 Thread goldmedal
Github user goldmedal commented on a diff in the pull request: https://github.com/apache/spark/pull/18875#discussion_r138110523 --- Diff: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/json/JacksonGeneratorSuite.scala --- @@ -0,0 +1,133 @@ +/* + * Licensed to

[GitHub] spark issue #19182: [SPARK-21970][Core] Fix Redundant Throws Declarations in...

2017-09-11 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19182 **[Test build #3916 has finished](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/3916/testReport)** for PR 19182 at commit

[GitHub] spark issue #19106: [SPARK-21770][ML] ProbabilisticClassificationModel fix c...

2017-09-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19106 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19106: [SPARK-21770][ML] ProbabilisticClassificationModel fix c...

2017-09-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19106 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/81640/ Test PASSed. ---

[GitHub] spark issue #19106: [SPARK-21770][ML] ProbabilisticClassificationModel fix c...

2017-09-11 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19106 **[Test build #81640 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81640/testReport)** for PR 19106 at commit

[GitHub] spark issue #19185: [Spark-21854] Added LogisticRegressionTrainingSummary fo...

2017-09-11 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19185 **[Test build #81642 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81642/testReport)** for PR 19185 at commit

[GitHub] spark pull request #18875: [SPARK-21513][SQL] Allow UDF to_json support conv...

2017-09-11 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/18875#discussion_r138107641 --- Diff: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/json/JacksonGeneratorSuite.scala --- @@ -0,0 +1,133 @@ +/* + * Licensed to

[GitHub] spark pull request #18875: [SPARK-21513][SQL] Allow UDF to_json support conv...

2017-09-11 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/18875#discussion_r138104414 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/JsonFunctionsSuite.scala --- @@ -180,10 +180,30 @@ class JsonFunctionsSuite extends QueryTest with

[GitHub] spark pull request #18875: [SPARK-21513][SQL] Allow UDF to_json support conv...

2017-09-11 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/18875#discussion_r138103265 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/JsonFunctionsSuite.scala --- @@ -180,10 +180,30 @@ class JsonFunctionsSuite extends QueryTest

[GitHub] spark issue #18875: [SPARK-21513][SQL] Allow UDF to_json support converting ...

2017-09-11 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18875 **[Test build #81641 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81641/testReport)** for PR 18875 at commit

[GitHub] spark pull request #18875: [SPARK-21513][SQL] Allow UDF to_json support conv...

2017-09-11 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/18875#discussion_r138102302 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/JsonFunctionsSuite.scala --- @@ -180,10 +180,30 @@ class JsonFunctionsSuite extends QueryTest

[GitHub] spark issue #18853: [SPARK-21646][SQL] CommonType for binary comparison

2017-09-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18853 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/81638/ Test PASSed. ---

[GitHub] spark issue #18853: [SPARK-21646][SQL] CommonType for binary comparison

2017-09-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18853 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #18853: [SPARK-21646][SQL] CommonType for binary comparison

2017-09-11 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18853 **[Test build #81638 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81638/testReport)** for PR 18853 at commit

[GitHub] spark issue #19188: [SPARK-21973][SQL] Add an new option to filter queries i...

2017-09-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19188 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19188: [SPARK-21973][SQL] Add an new option to filter queries i...

2017-09-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19188 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/81637/ Test PASSed. ---

[GitHub] spark issue #19188: [SPARK-21973][SQL] Add an new option to filter queries i...

2017-09-11 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19188 **[Test build #81637 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81637/testReport)** for PR 19188 at commit

[GitHub] spark pull request #18875: [SPARK-21513][SQL] Allow UDF to_json support conv...

2017-09-11 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/18875#discussion_r138095618 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/JsonFunctionsSuite.scala --- @@ -180,10 +180,30 @@ class JsonFunctionsSuite extends QueryTest with

[GitHub] spark pull request #18973: [SPARK-21765] Set isStreaming on leaf nodes for s...

2017-09-11 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/18973#discussion_r138095165 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/logical/basicLogicalOperators.scala --- @@ -443,7 +444,8 @@ case class Range(

[GitHub] spark pull request #18875: [SPARK-21513][SQL] Allow UDF to_json support conv...

2017-09-11 Thread goldmedal
Github user goldmedal commented on a diff in the pull request: https://github.com/apache/spark/pull/18875#discussion_r138094337 --- Diff: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/json/JacksonGeneratorSuite.scala --- @@ -0,0 +1,133 @@ +/* + * Licensed to

[GitHub] spark pull request #19147: [WIP][SPARK-21190][SQL][PYTHON] Vectorized UDFs i...

2017-09-11 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/19147#discussion_r138093529 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/python/BatchEvalPythonExec.scala --- @@ -62,6 +62,7 @@ import

[GitHub] spark pull request #18538: [SPARK-14516][ML] Adding ClusteringEvaluator with...

2017-09-11 Thread mgaido91
Github user mgaido91 commented on a diff in the pull request: https://github.com/apache/spark/pull/18538#discussion_r138090203 --- Diff: mllib/src/main/scala/org/apache/spark/ml/evaluation/ClusteringEvaluator.scala --- @@ -0,0 +1,437 @@ +/* + * Licensed to the Apache

<    1   2   3   4   >