[GitHub] spark issue #21201: [SPARK-24128][SQL] Mention configuration option in impli...

2018-05-01 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/21201 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews

[GitHub] spark issue #21040: [SPARK-23930][SQL] Add slice function

2018-05-01 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/21040 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...

[GitHub] spark issue #21202: [SPARK-24129] [K8S] Add option to pass --build-arg's to ...

2018-05-01 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21202 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21204: [SPARK-24132][ML]Expand instrumentation for classificati...

2018-05-01 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21204 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21195: [Spark-23975][ML] Add support of array input for all clu...

2018-05-01 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21195 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21200: [SPARK-24039][SS] Do continuous processing writes with m...

2018-05-01 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21200 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21199: [SPARK-24127][SS] Continuous text socket source

2018-05-01 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21199 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21194: [SPARK-24046][SS] Fix rate source when rowsPerSecond <= ...

2018-05-01 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21194 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21194: [SPARK-24046][SS] Fix rate source when rowsPerSecond <= ...

2018-05-01 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21194 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21202: [SPARK-24129] [K8S] Add option to pass --build-arg's to ...

2018-05-01 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21202 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21199: [SPARK-24127][SS] Continuous text socket source

2018-05-01 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21199 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21204: [SPARK-24132][ML]Expand instrumentation for classificati...

2018-05-01 Thread felixcheung
Github user felixcheung commented on the issue: https://github.com/apache/spark/pull/21204 ok to test --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@s

[GitHub] spark pull request #21198: [SPARK-24126][pyspark] Use build-specific temp di...

2018-05-01 Thread felixcheung
Github user felixcheung commented on a diff in the pull request: https://github.com/apache/spark/pull/21198#discussion_r185182067 --- Diff: python/run-tests.py --- @@ -77,13 +79,33 @@ def run_individual_python_test(test_name, pyspark_python): 'PYSPARK_PYTHON': which(py

[GitHub] spark pull request #21198: [SPARK-24126][pyspark] Use build-specific temp di...

2018-05-01 Thread felixcheung
Github user felixcheung commented on a diff in the pull request: https://github.com/apache/spark/pull/21198#discussion_r185182113 --- Diff: python/run-tests.py --- @@ -77,13 +79,33 @@ def run_individual_python_test(test_name, pyspark_python): 'PYSPARK_PYTHON': which(py

[GitHub] spark issue #21202: [SPARK-24129] [K8S] Add option to pass --build-arg's to ...

2018-05-01 Thread felixcheung
Github user felixcheung commented on the issue: https://github.com/apache/spark/pull/21202 @foxish --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@sp

[GitHub] spark issue #21202: [SPARK-24129] [K8S] Add option to pass --build-arg's to ...

2018-05-01 Thread felixcheung
Github user felixcheung commented on the issue: https://github.com/apache/spark/pull/21202 ok to test --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@s

[GitHub] spark issue #21201: [SPARK-24128][SQL] Mention configuration option in impli...

2018-05-01 Thread felixcheung
Github user felixcheung commented on the issue: https://github.com/apache/spark/pull/21201 Jenkins, ok to test --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: revie

[GitHub] spark issue #21200: [SPARK-24039][SS] Do continuous processing writes with m...

2018-05-01 Thread felixcheung
Github user felixcheung commented on the issue: https://github.com/apache/spark/pull/21200 ok to test --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@s

[GitHub] spark issue #21200: [SPARK-24039][SS] Do continuous processing writes with m...

2018-05-01 Thread felixcheung
Github user felixcheung commented on the issue: https://github.com/apache/spark/pull/21200 Jenkins, add to whitelist --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #21203: [SPARK-24131][PySpark] Add majorMinorVersion API to PySp...

2018-05-01 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/21203 > perhaps better to match python's version_info ? This for now is more close to `VersionUtils.majorMinorVersion`. To match python's version_info, means we need to have 5 components? --- --

[GitHub] spark pull request #21136: [SPARK-24061][SS]Add TypedFilter support for cont...

2018-05-01 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/21136 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request #21205: [Minor] To add a missing full-stop.

2018-05-01 Thread XD-DENG
GitHub user XD-DENG opened a pull request: https://github.com/apache/spark/pull/21205 [Minor] To add a missing full-stop. ## What changes were proposed in this pull request? In the document [Tuning Spark -> Determining Memory Consumption](https://spark.apache.org/docs/lates

[GitHub] spark pull request #21054: [SPARK-23907][SQL] Add regr_* functions

2018-05-01 Thread ueshin
Github user ueshin commented on a diff in the pull request: https://github.com/apache/spark/pull/21054#discussion_r185171846 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/aggregate/regression.scala --- @@ -0,0 +1,193 @@ +/* + * Licensed to

[GitHub] spark issue #21205: To add a missing full-stop in doc "Tuning Spark".

2018-05-01 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21205 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #21054: [SPARK-23907][SQL] Add regr_* functions

2018-05-01 Thread ueshin
Github user ueshin commented on a diff in the pull request: https://github.com/apache/spark/pull/21054#discussion_r185170928 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/aggregate/Average.scala --- @@ -23,23 +23,13 @@ import org.apache.spark.sql.

[GitHub] spark pull request #21054: [SPARK-23907][SQL] Add regr_* functions

2018-05-01 Thread ueshin
Github user ueshin commented on a diff in the pull request: https://github.com/apache/spark/pull/21054#discussion_r185170552 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/aggregate/Average.scala --- @@ -62,14 +52,6 @@ case class Average(child: Exp

[GitHub] spark issue #21205: To add a missing full-stop in doc "Tuning Spark".

2018-05-01 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21205 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21206: [SPARK-24133][SQL] Check for integer overflows when resi...

2018-05-01 Thread ala
Github user ala commented on the issue: https://github.com/apache/spark/pull/21206 @hvanhovell --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.a

[GitHub] spark pull request #21206: [SPARK-24133][SQL] Check for integer overflows wh...

2018-05-01 Thread ala
GitHub user ala opened a pull request: https://github.com/apache/spark/pull/21206 [SPARK-24133][SQL] Check for integer overflows when resizing WritableColumnVectors ## What changes were proposed in this pull request? `ColumnVector`s store string data in one big byte array.

[GitHub] spark pull request #21021: [SPARK-23921][SQL] Add array_sort function

2018-05-01 Thread ueshin
Github user ueshin commented on a diff in the pull request: https://github.com/apache/spark/pull/21021#discussion_r185196808 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/collectionOperations.scala --- @@ -191,28 +161,202 @@ case class SortArray(b

[GitHub] spark pull request #21021: [SPARK-23921][SQL] Add array_sort function

2018-05-01 Thread ueshin
Github user ueshin commented on a diff in the pull request: https://github.com/apache/spark/pull/21021#discussion_r185197041 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/collectionOperations.scala --- @@ -191,28 +161,202 @@ case class SortArray(b

[GitHub] spark issue #21206: [SPARK-24133][SQL] Check for integer overflows when resi...

2018-05-01 Thread ala
Github user ala commented on the issue: https://github.com/apache/spark/pull/21206 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@s

[GitHub] spark pull request #21192: [SPARK-24118][SQL] Flexible format for the lineSe...

2018-05-01 Thread MaxGekk
Github user MaxGekk commented on a diff in the pull request: https://github.com/apache/spark/pull/21192#discussion_r185197844 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/json/JSONOptions.scala --- @@ -120,8 +120,26 @@ private[sql] class JSONOptions(

[GitHub] spark pull request #21200: [SPARK-24039][SS] Do continuous processing writes...

2018-05-01 Thread HeartSaVioR
Github user HeartSaVioR commented on a diff in the pull request: https://github.com/apache/spark/pull/21200#discussion_r185198458 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/continuous/ContinuousQueuedDataReader.scala --- @@ -0,0 +1,90 @@ +/*

[GitHub] spark pull request #21200: [SPARK-24039][SS] Do continuous processing writes...

2018-05-01 Thread HeartSaVioR
Github user HeartSaVioR commented on a diff in the pull request: https://github.com/apache/spark/pull/21200#discussion_r185187424 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/continuous/ContinuousDataSourceRDD.scala --- @@ -0,0 +1,153 @@ +/* +

[GitHub] spark pull request #21200: [SPARK-24039][SS] Do continuous processing writes...

2018-05-01 Thread HeartSaVioR
Github user HeartSaVioR commented on a diff in the pull request: https://github.com/apache/spark/pull/21200#discussion_r185194384 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/continuous/ContinuousDataSourceRDD.scala --- @@ -0,0 +1,153 @@ +/* +

[GitHub] spark pull request #21200: [SPARK-24039][SS] Do continuous processing writes...

2018-05-01 Thread HeartSaVioR
Github user HeartSaVioR commented on a diff in the pull request: https://github.com/apache/spark/pull/21200#discussion_r185201032 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/continuous/WriteToContinuousDataSourceExec.scala --- @@ -46,28 +46,34 @@ cas

[GitHub] spark pull request #21200: [SPARK-24039][SS] Do continuous processing writes...

2018-05-01 Thread HeartSaVioR
Github user HeartSaVioR commented on a diff in the pull request: https://github.com/apache/spark/pull/21200#discussion_r18524 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/continuous/EpochPollRunnable.scala --- @@ -0,0 +1,61 @@ +/* + * Lice

[GitHub] spark issue #21070: [SPARK-23972][BUILD][SQL] Update Parquet to 1.10.0.

2018-05-01 Thread mswit-databricks
Github user mswit-databricks commented on the issue: https://github.com/apache/spark/pull/21070 @rdblue Do you see any risk of additional overhead coming from the extra stats? For example, if the data contains very long strings, performing comparison on them to generate stats will be

[GitHub] spark pull request #21123: [SPARK-24045][SQL]Create base class for file data...

2018-05-01 Thread gengliangwang
Github user gengliangwang commented on a diff in the pull request: https://github.com/apache/spark/pull/21123#discussion_r185215070 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/DataSourceStrategy.scala --- @@ -213,6 +215,26 @@ case class DataSourceA

[GitHub] spark issue #21158: [SPARK-23850][sql] Add separate config for SQL options r...

2018-05-01 Thread tgravescs
Github user tgravescs commented on the issue: https://github.com/apache/spark/pull/21158 the original change was merged into 2.3.0, see https://issues.apache.org/jira/browse/SPARK-22479. We are keeping the behavior of that fix and that is to not show the credentials of the jdbc con

[GitHub] spark pull request #21165: [Spark-20087][CORE] Attach accumulators / metrics...

2018-05-01 Thread advancedxy
Github user advancedxy commented on a diff in the pull request: https://github.com/apache/spark/pull/21165#discussion_r185226136 --- Diff: core/src/main/scala/org/apache/spark/TaskEndReason.scala --- @@ -212,9 +212,15 @@ case object TaskResultLost extends TaskFailedReason { *

[GitHub] spark issue #21158: [SPARK-23850][sql] Add separate config for SQL options r...

2018-05-01 Thread vanzin
Github user vanzin commented on the issue: https://github.com/apache/spark/pull/21158 But the main thing is that the change modified the previous behavior which was *not* to redact these things in the previous places where redaction occurred. So, in a way, this change is fixing a regr

[GitHub] spark pull request #21198: [SPARK-24126][pyspark] Use build-specific temp di...

2018-05-01 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/21198#discussion_r185242337 --- Diff: python/run-tests.py --- @@ -77,13 +79,33 @@ def run_individual_python_test(test_name, pyspark_python): 'PYSPARK_PYTHON': which(pyspark

[GitHub] spark pull request #21198: [SPARK-24126][pyspark] Use build-specific temp di...

2018-05-01 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/21198#discussion_r185242826 --- Diff: python/run-tests.py --- @@ -77,13 +79,33 @@ def run_individual_python_test(test_name, pyspark_python): 'PYSPARK_PYTHON': which(pyspark

[GitHub] spark issue #21185: [SPARK-23894][CORE][SQL] Defensively clear ActiveSession...

2018-05-01 Thread jiangxb1987
Github user jiangxb1987 commented on the issue: https://github.com/apache/spark/pull/21185 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: review

[GitHub] spark issue #21193: [SPARK-24121][SQL][WIP] Add API for handling expression ...

2018-05-01 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/21193 @hvanhovell Thanks for your comment! I tried to add new JavaCode abstraction `Block` which holds the code to generate. I hope this approach can be less side effect. Please comment if you hav

[GitHub] spark issue #17086: [SPARK-24101][ML][MLLIB] ML Evaluators should use weight...

2018-05-01 Thread imatiach-msft
Github user imatiach-msft commented on the issue: https://github.com/apache/spark/pull/17086 jenkins, retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-m

[GitHub] spark issue #21014: [SPARK-23941][Mesos] Mesos task failed on specific spark...

2018-05-01 Thread vanzin
Github user vanzin commented on the issue: https://github.com/apache/spark/pull/21014 Merging to master / 2.3 / 2.2. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark pull request #21014: [SPARK-23941][Mesos] Mesos task failed on specifi...

2018-05-01 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/21014 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #18390: [SPARK-21178][ML] Add support for label specific metrics...

2018-05-01 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18390 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21185: [SPARK-23894][CORE][SQL] Defensively clear ActiveSession...

2018-05-01 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/21185 test this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h

[GitHub] spark issue #21185: [SPARK-23894][CORE][SQL] Defensively clear ActiveSession...

2018-05-01 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/21185 ok to test --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@sp

[GitHub] spark issue #21185: [SPARK-23894][CORE][SQL] Defensively clear ActiveSession...

2018-05-01 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21185 **[Test build #89978 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/89978/testReport)** for PR 21185 at commit [`49783c5`](https://github.com/apache/spark/commit/49

[GitHub] spark pull request #21194: [SPARK-24046][SS] Fix rate source when rowsPerSec...

2018-05-01 Thread xuanyuanking
Github user xuanyuanking commented on a diff in the pull request: https://github.com/apache/spark/pull/21194#discussion_r185252544 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/sources/RateStreamProvider.scala --- @@ -101,25 +101,10 @@ object RateStrea

[GitHub] spark pull request #21194: [SPARK-24046][SS] Fix rate source when rowsPerSec...

2018-05-01 Thread xuanyuanking
Github user xuanyuanking commented on a diff in the pull request: https://github.com/apache/spark/pull/21194#discussion_r185252360 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/sources/RateStreamProvider.scala --- @@ -101,25 +101,10 @@ object RateStrea

[GitHub] spark issue #21206: [SPARK-24133][SQL] Check for integer overflows when resi...

2018-05-01 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/21206 cc @cloud-fan --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h..

[GitHub] spark issue #21206: [SPARK-24133][SQL] Check for integer overflows when resi...

2018-05-01 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/21206 test this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h

[GitHub] spark issue #21206: [SPARK-24133][SQL] Check for integer overflows when resi...

2018-05-01 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/21206 ok to test --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@sp

[GitHub] spark issue #21206: [SPARK-24133][SQL] Check for integer overflows when resi...

2018-05-01 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21206 **[Test build #89979 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/89979/testReport)** for PR 21206 at commit [`17e2d02`](https://github.com/apache/spark/commit/17

[GitHub] spark issue #21206: [SPARK-24133][SQL] Check for integer overflows when resi...

2018-05-01 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21206 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #21206: [SPARK-24133][SQL] Check for integer overflows when resi...

2018-05-01 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21206 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/2751/ Tes

[GitHub] spark issue #21187: [SPARK-24035][SQL] SQL syntax for Pivot

2018-05-01 Thread Tagar
Github user Tagar commented on the issue: https://github.com/apache/spark/pull/21187 @maryannxue, yep, "IN ANY" is what I meant. I missed it was already in the description - thanks for clarifying this. --- - To unsu

[GitHub] spark issue #21185: [SPARK-23894][CORE][SQL] Defensively clear ActiveSession...

2018-05-01 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21185 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/2752/ Tes

[GitHub] spark issue #21185: [SPARK-23894][CORE][SQL] Defensively clear ActiveSession...

2018-05-01 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21185 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #21206: [SPARK-24133][SQL] Check for integer overflows when resi...

2018-05-01 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21206 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/2753/ Tes

[GitHub] spark issue #21206: [SPARK-24133][SQL] Check for integer overflows when resi...

2018-05-01 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21206 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #21187: [SPARK-24035][SQL] SQL syntax for Pivot

2018-05-01 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/21187 In-any-subquery in Pivot can be implemented like what we did in the other parts, but let us first leave it as the future item. @maryannxue Maybe you can create a JIRA. The Oracle-compat

[GitHub] spark issue #21158: [SPARK-23850][sql] Add separate config for SQL options r...

2018-05-01 Thread vanzin
Github user vanzin commented on the issue: https://github.com/apache/spark/pull/21158 Also note that Tom's comment above is not 100% accurate - this change *does* redact the JDBC URL on the SQL side, for reasons I described in my comments on the bug. That's the whole reason why the SQ

[GitHub] spark issue #21198: [SPARK-24126][pyspark] Use build-specific temp directory...

2018-05-01 Thread vanzin
Github user vanzin commented on the issue: https://github.com/apache/spark/pull/21198 ok to test --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.

[GitHub] spark pull request #21187: [SPARK-24035][SQL] SQL syntax for Pivot

2018-05-01 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/21187#discussion_r185258526 --- Diff: sql/catalyst/src/main/antlr4/org/apache/spark/sql/catalyst/parser/SqlBase.g4 --- @@ -805,6 +810,7 @@ RIGHT: 'RIGHT'; FULL: 'FULL'; NA

[GitHub] spark pull request #21200: [SPARK-24039][SS] Do continuous processing writes...

2018-05-01 Thread arunmahadevan
Github user arunmahadevan commented on a diff in the pull request: https://github.com/apache/spark/pull/21200#discussion_r185259515 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/continuous/EpochPollRunnable.scala --- @@ -0,0 +1,61 @@ +/* + * Li

[GitHub] spark issue #21198: [SPARK-24126][pyspark] Use build-specific temp directory...

2018-05-01 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21198 **[Test build #89980 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/89980/testReport)** for PR 21198 at commit [`77e8ff5`](https://github.com/apache/spark/commit/77

[GitHub] spark pull request #21187: [SPARK-24035][SQL] SQL syntax for Pivot

2018-05-01 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/21187#discussion_r185260075 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/logical/basicLogicalOperators.scala --- @@ -691,7 +691,9 @@ case class Pivot(

[GitHub] spark pull request #21187: [SPARK-24035][SQL] SQL syntax for Pivot

2018-05-01 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/21187#discussion_r185259275 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/logical/basicLogicalOperators.scala --- @@ -691,7 +691,9 @@ case class Pivot(

[GitHub] spark issue #21195: [Spark-23975][ML] Add support of array input for all clu...

2018-05-01 Thread jkbradley
Github user jkbradley commented on the issue: https://github.com/apache/spark/pull/21195 add to whitelist --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h.

[GitHub] spark issue #21185: [SPARK-23894][CORE][SQL] Defensively clear ActiveSession...

2018-05-01 Thread squito
Github user squito commented on the issue: https://github.com/apache/spark/pull/21185 I don't understand the comments about "leaked threads". the executor thread pool is allowed to create threads whenever it wants. You can play around with this example if you like: https://gist.git

[GitHub] spark pull request #21153: [SPARK-24058][ML][PySpark] Default Params in ML s...

2018-05-01 Thread jkbradley
Github user jkbradley commented on a diff in the pull request: https://github.com/apache/spark/pull/21153#discussion_r185262870 --- Diff: python/pyspark/util.py --- @@ -61,6 +62,26 @@ def _get_argspec(f): return argspec +def majorMinorVersion(version): -

[GitHub] spark issue #21195: [Spark-23975][ML] Add support of array input for all clu...

2018-05-01 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21195 **[Test build #89981 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/89981/testReport)** for PR 21195 at commit [`45e6e96`](https://github.com/apache/spark/commit/45

[GitHub] spark issue #20973: [SPARK-20114][ML] spark.ml parity for sequential pattern...

2018-05-01 Thread jkbradley
Github user jkbradley commented on the issue: https://github.com/apache/spark/pull/20973 LGTM pending jenkins tests --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #20973: [SPARK-20114][ML] spark.ml parity for sequential pattern...

2018-05-01 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20973 **[Test build #4162 has started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/4162/testReport)** for PR 20973 at commit [`76d4119`](https://github.com/apache/spark/commit/7

[GitHub] spark issue #21204: [SPARK-24132][ML]Expand instrumentation for classificati...

2018-05-01 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21204 **[Test build #4163 has started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/4163/testReport)** for PR 21204 at commit [`7b75ed6`](https://github.com/apache/spark/commit/7

[GitHub] spark pull request #21200: [SPARK-24039][SS] Do continuous processing writes...

2018-05-01 Thread jose-torres
Github user jose-torres commented on a diff in the pull request: https://github.com/apache/spark/pull/21200#discussion_r185264079 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/continuous/EpochPollRunnable.scala --- @@ -0,0 +1,61 @@ +/* + * Lice

[GitHub] spark issue #20319: [SPARK-22884][ML][TESTS] ML test for StructuredStreaming...

2018-05-01 Thread jkbradley
Github user jkbradley commented on the issue: https://github.com/apache/spark/pull/20319 @smurakozi Do you have time to update this? I did a full review, though it now has a small merge conflict. Thanks! --- - To

[GitHub] spark issue #21198: [SPARK-24126][pyspark] Use build-specific temp directory...

2018-05-01 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21198 **[Test build #89980 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/89980/testReport)** for PR 21198 at commit [`77e8ff5`](https://github.com/apache/spark/commit/7

[GitHub] spark issue #21198: [SPARK-24126][pyspark] Use build-specific temp directory...

2018-05-01 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21198 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #21198: [SPARK-24126][pyspark] Use build-specific temp directory...

2018-05-01 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21198 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/89980/ Test FAILed. ---

[GitHub] spark pull request #21187: [SPARK-24035][SQL] SQL syntax for Pivot

2018-05-01 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/21187#discussion_r185264657 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/logical/basicLogicalOperators.scala --- @@ -691,7 +691,9 @@ case class Pivot(

[GitHub] spark issue #21203: [SPARK-24131][PySpark] Add majorMinorVersion API to PySp...

2018-05-01 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/21203 Retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: revi

[GitHub] spark issue #21197: [SPARK-23971] [BACKPORT-2.3] Should not leak Spark sessi...

2018-05-01 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/21197 Retest this please. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: rev

[GitHub] spark issue #21196: [SPARK-24123][SQL] Fix precision issues in monthsBetween...

2018-05-01 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/21196 Retest this please. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: rev

[GitHub] spark pull request #21200: [SPARK-24039][SS] Do continuous processing writes...

2018-05-01 Thread jose-torres
Github user jose-torres commented on a diff in the pull request: https://github.com/apache/spark/pull/21200#discussion_r185265321 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/continuous/ContinuousDataSourceRDD.scala --- @@ -0,0 +1,153 @@ +/* +

[GitHub] spark issue #21197: [SPARK-23971] [BACKPORT-2.3] Should not leak Spark sessi...

2018-05-01 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21197 **[Test build #89983 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/89983/testReport)** for PR 21197 at commit [`c1c6377`](https://github.com/apache/spark/commit/c1

[GitHub] spark issue #21203: [SPARK-24131][PySpark] Add majorMinorVersion API to PySp...

2018-05-01 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21203 **[Test build #89982 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/89982/testReport)** for PR 21203 at commit [`9558ad7`](https://github.com/apache/spark/commit/95

[GitHub] spark issue #21196: [SPARK-24123][SQL] Fix precision issues in monthsBetween...

2018-05-01 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21196 **[Test build #89984 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/89984/testReport)** for PR 21196 at commit [`4d1775c`](https://github.com/apache/spark/commit/4d

[GitHub] spark issue #21186: [SPARK-22279][SPARK-24112] Enable `convertMetastoreOrc` ...

2018-05-01 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/21186 Hi, @gatorsmile . Do you think we need to split this into two separate PRs? If you want, I will split this. - [SPARK-22279](https://issues.apache.org/jira/browse/SPARK-24112) Enable co

[GitHub] spark issue #20261: [SPARK-22885][ML][TEST] ML test for StructuredStreaming:...

2018-05-01 Thread jkbradley
Github user jkbradley commented on the issue: https://github.com/apache/spark/pull/20261 LGTM Will merge after fresh tests Thanks @WeichenXu123 and @smurakozi ! --- - To unsubscribe, e-mail: reviews-unsubscr.

[GitHub] spark issue #20261: [SPARK-22885][ML][TEST] ML test for StructuredStreaming:...

2018-05-01 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20261 **[Test build #4164 has started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/4164/testReport)** for PR 20261 at commit [`d52b8ca`](https://github.com/apache/spark/commit/d

[GitHub] spark issue #21186: [SPARK-22279][SPARK-24112] Enable `convertMetastoreOrc` ...

2018-05-01 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/21186 Retest this please. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: rev

[GitHub] spark issue #21203: [SPARK-24131][PySpark] Add majorMinorVersion API to PySp...

2018-05-01 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21203 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

  1   2   3   4   5   6   >