[GitHub] spark pull request #21258: [SPARK-23933][SQL] Add map_from_arrays function

2018-06-07 Thread ueshin
Github user ueshin commented on a diff in the pull request: https://github.com/apache/spark/pull/21258#discussion_r193929635 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/complexTypeCreator.scala --- @@ -236,6 +236,76 @@ case class

[GitHub] spark pull request #21258: [SPARK-23933][SQL] Add map_from_arrays function

2018-06-07 Thread ueshin
Github user ueshin commented on a diff in the pull request: https://github.com/apache/spark/pull/21258#discussion_r193929951 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/complexTypeCreator.scala --- @@ -236,6 +236,76 @@ case class

[GitHub] spark issue #21258: [SPARK-23933][SQL] Add map_from_arrays function

2018-06-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21258 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21258: [SPARK-23933][SQL] Add map_from_arrays function

2018-06-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21258 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/3841/

[GitHub] spark pull request #21155: [SPARK-23927][SQL] Add "sequence" expression

2018-06-07 Thread ueshin
Github user ueshin commented on a diff in the pull request: https://github.com/apache/spark/pull/21155#discussion_r193928034 --- Diff: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/expressions/CollectionExpressionsSuite.scala --- @@ -308,6 +313,292 @@ class

[GitHub] spark pull request #21155: [SPARK-23927][SQL] Add "sequence" expression

2018-06-07 Thread ueshin
Github user ueshin commented on a diff in the pull request: https://github.com/apache/spark/pull/21155#discussion_r193928619 --- Diff: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/expressions/CollectionExpressionsSuite.scala --- @@ -308,6 +313,292 @@ class

[GitHub] spark pull request #21155: [SPARK-23927][SQL] Add "sequence" expression

2018-06-07 Thread ueshin
Github user ueshin commented on a diff in the pull request: https://github.com/apache/spark/pull/21155#discussion_r193926575 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/collectionOperations.scala --- @@ -1887,6 +1889,402 @@ case class

[GitHub] spark pull request #21155: [SPARK-23927][SQL] Add "sequence" expression

2018-06-07 Thread ueshin
Github user ueshin commented on a diff in the pull request: https://github.com/apache/spark/pull/21155#discussion_r193927174 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/collectionOperations.scala --- @@ -1887,6 +1889,402 @@ case class

[GitHub] spark pull request #21155: [SPARK-23927][SQL] Add "sequence" expression

2018-06-07 Thread ueshin
Github user ueshin commented on a diff in the pull request: https://github.com/apache/spark/pull/21155#discussion_r193926459 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/collectionOperations.scala --- @@ -1887,6 +1889,402 @@ case class

[GitHub] spark pull request #21155: [SPARK-23927][SQL] Add "sequence" expression

2018-06-07 Thread ueshin
Github user ueshin commented on a diff in the pull request: https://github.com/apache/spark/pull/21155#discussion_r193926035 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/collectionOperations.scala --- @@ -1887,6 +1889,402 @@ case class

[GitHub] spark issue #21495: [SPARK-24418][Build] Upgrade Scala to 2.11.12 and 2.12.6

2018-06-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21495 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #21155: [SPARK-23927][SQL] Add "sequence" expression

2018-06-07 Thread ueshin
Github user ueshin commented on a diff in the pull request: https://github.com/apache/spark/pull/21155#discussion_r193927263 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/collectionOperations.scala --- @@ -1887,6 +1889,402 @@ case class

[GitHub] spark pull request #21155: [SPARK-23927][SQL] Add "sequence" expression

2018-06-07 Thread ueshin
Github user ueshin commented on a diff in the pull request: https://github.com/apache/spark/pull/21155#discussion_r193899743 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/collectionOperations.scala --- @@ -1887,6 +1889,402 @@ case class

[GitHub] spark issue #21495: [SPARK-24418][Build] Upgrade Scala to 2.11.12 and 2.12.6

2018-06-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21495 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/3840/

[GitHub] spark issue #21258: [SPARK-23933][SQL] Add map_from_arrays function

2018-06-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21258 **[Test build #91542 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/91542/testReport)** for PR 21258 at commit

[GitHub] spark pull request #21258: [SPARK-23933][SQL] Add map_from_arrays function

2018-06-07 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/21258#discussion_r193928013 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/complexTypeCreator.scala --- @@ -235,6 +235,86 @@ case class

[GitHub] spark pull request #21258: [SPARK-23933][SQL] Add map_from_arrays function

2018-06-07 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/21258#discussion_r193927995 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/complexTypeCreator.scala --- @@ -235,6 +235,86 @@ case class

[GitHub] spark issue #21495: [SPARK-24418][Build] Upgrade Scala to 2.11.12 and 2.12.6

2018-06-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21495 **[Test build #91541 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/91541/testReport)** for PR 21495 at commit

[GitHub] spark issue #21495: [SPARK-24418][Build] Upgrade Scala to 2.11.12 and 2.12.6

2018-06-07 Thread dbtsai
Github user dbtsai commented on the issue: https://github.com/apache/spark/pull/21495 @som-snytt initialize it in `printWelcome` will not work since in order version of Scala, `printWelcome` is the last one to be executed. ---

[GitHub] spark pull request #21258: [SPARK-23933][SQL] Add map_from_arrays function

2018-06-07 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/21258#discussion_r193927508 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/complexTypeCreator.scala --- @@ -235,6 +235,86 @@ case class

[GitHub] spark issue #21469: [SPARK-24441][SS] Expose total estimated size of states ...

2018-06-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21469 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/91535/ Test PASSed. ---

[GitHub] spark issue #21469: [SPARK-24441][SS] Expose total estimated size of states ...

2018-06-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21469 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21469: [SPARK-24441][SS] Expose total estimated size of states ...

2018-06-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21469 **[Test build #91535 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/91535/testReport)** for PR 21469 at commit

[GitHub] spark pull request #21495: [SPARK-24418][Build] Upgrade Scala to 2.11.12 and...

2018-06-07 Thread dbtsai
Github user dbtsai commented on a diff in the pull request: https://github.com/apache/spark/pull/21495#discussion_r193927042 --- Diff: repl/scala-2.11/src/main/scala/org/apache/spark/repl/SparkILoopInterpreter.scala --- @@ -21,8 +21,22 @@ import scala.collection.mutable

[GitHub] spark issue #21504: [SPARK-24479][SS] Added config for registering streaming...

2018-06-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21504 **[Test build #91540 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/91540/testReport)** for PR 21504 at commit

[GitHub] spark pull request #21504: [SPARK-24479][SS] Added config for registering st...

2018-06-07 Thread arunmahadevan
Github user arunmahadevan commented on a diff in the pull request: https://github.com/apache/spark/pull/21504#discussion_r193923588 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/streaming/StreamingQueryManager.scala --- @@ -55,6 +56,11 @@ class StreamingQueryManager

[GitHub] spark issue #21477: [WIP] [SPARK-24396] [SS] [PYSPARK] Add Structured Stream...

2018-06-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21477 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21477: [WIP] [SPARK-24396] [SS] [PYSPARK] Add Structured Stream...

2018-06-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21477 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/3839/

[GitHub] spark issue #21477: [WIP] [SPARK-24396] [SS] [PYSPARK] Add Structured Stream...

2018-06-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21477 **[Test build #91539 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/91539/testReport)** for PR 21477 at commit

[GitHub] spark issue #21402: SPARK-24355 Spark external shuffle server improvement to...

2018-06-07 Thread vanzin
Github user vanzin commented on the issue: https://github.com/apache/spark/pull/21402 > StreamRequest will not block the server netty handler thread. Hmm, I'm not so sure that's accurate. I think the main difference is that I don't think there is currently any code path that

[GitHub] spark issue #21511: [SPARK-24491][Kubernetes] Configuration support for requ...

2018-06-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21511 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #21495: [SPARK-24418][Build] Upgrade Scala to 2.11.12 and...

2018-06-07 Thread dongjoon-hyun
Github user dongjoon-hyun commented on a diff in the pull request: https://github.com/apache/spark/pull/21495#discussion_r193920472 --- Diff: repl/scala-2.11/src/main/scala/org/apache/spark/repl/SparkILoopInterpreter.scala --- @@ -21,8 +21,22 @@ import scala.collection.mutable

[GitHub] spark issue #21511: [SPARK-24491][Kubernetes] Configuration support for requ...

2018-06-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21511 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #21511: [SPARK-24491][Kubernetes] Configuration support f...

2018-06-07 Thread alexmilowski
GitHub user alexmilowski opened a pull request: https://github.com/apache/spark/pull/21511 [SPARK-24491][Kubernetes] Configuration support for requesting GPUs on k8s ## What changes were proposed in this pull request? Configuration support for generating the GPU requests in

[GitHub] spark issue #21495: [SPARK-24418][Build] Upgrade Scala to 2.11.12 and 2.12.6

2018-06-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21495 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21495: [SPARK-24418][Build] Upgrade Scala to 2.11.12 and 2.12.6

2018-06-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21495 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/3838/

[GitHub] spark pull request #21495: [SPARK-24418][Build] Upgrade Scala to 2.11.12 and...

2018-06-07 Thread dongjoon-hyun
Github user dongjoon-hyun commented on a diff in the pull request: https://github.com/apache/spark/pull/21495#discussion_r193919141 --- Diff: repl/scala-2.11/src/main/scala/org/apache/spark/repl/SparkILoopInterpreter.scala --- @@ -21,8 +21,22 @@ import scala.collection.mutable

[GitHub] spark issue #21495: [SPARK-24418][Build] Upgrade Scala to 2.11.12 and 2.12.6

2018-06-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21495 **[Test build #91538 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/91538/testReport)** for PR 21495 at commit

[GitHub] spark issue #21495: [SPARK-24418][Build] Upgrade Scala to 2.11.12 and 2.12.6

2018-06-07 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/21495 Retest this please. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #21482: [SPARK-24393][SQL] SQL builtin: isinf

2018-06-07 Thread henryr
Github user henryr commented on the issue: https://github.com/apache/spark/pull/21482 I think consistency in Spark's naming convention (and therefore increased discoverability by users) outweighs the advantage of naming it exactly for the Impala equivalent. I do agree that

[GitHub] spark pull request #21092: [SPARK-23984][K8S] Initial Python Bindings for Py...

2018-06-07 Thread ifilonenko
Github user ifilonenko commented on a diff in the pull request: https://github.com/apache/spark/pull/21092#discussion_r193913624 --- Diff: resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/Config.scala --- @@ -154,6 +176,24 @@ private[spark] object

[GitHub] spark issue #21092: [SPARK-23984][K8S] Initial Python Bindings for PySpark o...

2018-06-07 Thread ifilonenko
Github user ifilonenko commented on the issue: https://github.com/apache/spark/pull/21092 KubernetesSuite: - Run SparkPi with no resources - Run SparkPi with a very long application name. - Run SparkPi with a master URL without a scheme. - Run SparkPi with an argument.

[GitHub] spark issue #21109: [SPARK-24020][SQL] Sort-merge join inner range optimizat...

2018-06-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21109 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/91534/ Test FAILed. ---

[GitHub] spark issue #21109: [SPARK-24020][SQL] Sort-merge join inner range optimizat...

2018-06-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21109 Build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands,

[GitHub] spark issue #21109: [SPARK-24020][SQL] Sort-merge join inner range optimizat...

2018-06-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21109 **[Test build #91534 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/91534/testReport)** for PR 21109 at commit

[GitHub] spark pull request #21504: [SPARK-24479][SS] Added config for registering st...

2018-06-07 Thread merlintang
Github user merlintang commented on a diff in the pull request: https://github.com/apache/spark/pull/21504#discussion_r193911087 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/streaming/StreamingQueryManager.scala --- @@ -55,6 +56,11 @@ class StreamingQueryManager

[GitHub] spark issue #21482: [SPARK-24393][SQL] SQL builtin: isinf

2018-06-07 Thread rxin
Github user rxin commented on the issue: https://github.com/apache/spark/pull/21482 Thanks, Henry. In general I'm not a huge fan of adding something because hypothetically somebody might want it. Also if you want this to be compatible with Impala, wouldn't you want to name this the

[GitHub] spark issue #21092: [SPARK-23984][K8S] Initial Python Bindings for PySpark o...

2018-06-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21092 Kubernetes integration test status success URL: https://amplab.cs.berkeley.edu/jenkins/job/testing-k8s-prb-spark-integration/3694/ ---

[GitHub] spark issue #21092: [SPARK-23984][K8S] Initial Python Bindings for PySpark o...

2018-06-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21092 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/testing-k8s-prb-spark-integration/3694/ ---

[GitHub] spark issue #21092: [SPARK-23984][K8S] Initial Python Bindings for PySpark o...

2018-06-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21092 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21092: [SPARK-23984][K8S] Initial Python Bindings for PySpark o...

2018-06-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21092 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/3837/

[GitHub] spark issue #21510: [SPARK-24490][WebUI] Use WebUI.addStaticHandler in web U...

2018-06-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21510 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/3836/

[GitHub] spark issue #21510: [SPARK-24490][WebUI] Use WebUI.addStaticHandler in web U...

2018-06-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21510 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #21482: [SPARK-24393][SQL] SQL builtin: isinf

2018-06-07 Thread NihalHarish
Github user NihalHarish commented on a diff in the pull request: https://github.com/apache/spark/pull/21482#discussion_r193902999 --- Diff: R/pkg/NAMESPACE --- @@ -281,6 +281,8 @@ exportMethods("%<=>%", "initcap", "input_file_name",

[GitHub] spark issue #21510: [SPARK-24490][WebUI] Use WebUI.addStaticHandler in web U...

2018-06-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21510 **[Test build #91536 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/91536/testReport)** for PR 21510 at commit

[GitHub] spark issue #21092: [SPARK-23984][K8S] Initial Python Bindings for PySpark o...

2018-06-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21092 **[Test build #91537 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/91537/testReport)** for PR 21092 at commit

[GitHub] spark pull request #21510: [SPARK-24490][WebUI] Use WebUI.addStaticHandler i...

2018-06-07 Thread jaceklaskowski
GitHub user jaceklaskowski opened a pull request: https://github.com/apache/spark/pull/21510 [SPARK-24490][WebUI] Use WebUI.addStaticHandler in web UIs `WebUI` defines `addStaticHandler` that web UIs don't use (and simply introduce duplication). Let's clean them up and remove

[GitHub] spark pull request #21482: [SPARK-24393][SQL] SQL builtin: isinf

2018-06-07 Thread henryr
Github user henryr commented on a diff in the pull request: https://github.com/apache/spark/pull/21482#discussion_r193898812 --- Diff: R/pkg/NAMESPACE --- @@ -281,6 +281,8 @@ exportMethods("%<=>%", "initcap", "input_file_name",

[GitHub] spark issue #21504: [SPARK-24479][SS] Added config for registering streaming...

2018-06-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21504 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/91532/ Test PASSed. ---

[GitHub] spark issue #21504: [SPARK-24479][SS] Added config for registering streaming...

2018-06-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21504 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21504: [SPARK-24479][SS] Added config for registering streaming...

2018-06-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21504 **[Test build #91532 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/91532/testReport)** for PR 21504 at commit

[GitHub] spark issue #21483: [SPARK-24477][SPARK-24454][ML][PYTHON] Imports submodule...

2018-06-07 Thread BryanCutler
Github user BryanCutler commented on the issue: https://github.com/apache/spark/pull/21483 Looks good to me, pending the above comments --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For

[GitHub] spark issue #21482: [SPARK-24393][SQL] SQL builtin: isinf

2018-06-07 Thread henryr
Github user henryr commented on the issue: https://github.com/apache/spark/pull/21482 @rxin, that in itself is a bit weird, but there are ways to express inf values in Scala and thus inf values can show up flowing through Spark plans. I'm not sure MySQL has any such facility. ---

[GitHub] spark issue #21507: Branch 1.6

2018-06-07 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/21507 @deepaksonu Would it be possible to close this PR? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For

[GitHub] spark issue #20823: [SPARK-23674] Add Spark ML Listener for Tracking ML Pipe...

2018-06-07 Thread merlintang
Github user merlintang commented on the issue: https://github.com/apache/spark/pull/20823 @jmwdpk can you update this pr, since there is conflict. I have update this pr. https://github.com/merlintang/spark/commits/SPARK-23674 ---

[GitHub] spark pull request #21155: [SPARK-23927][SQL] Add "sequence" expression

2018-06-07 Thread ueshin
Github user ueshin commented on a diff in the pull request: https://github.com/apache/spark/pull/21155#discussion_r193895158 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/collectionOperations.scala --- @@ -1468,3 +1472,388 @@ case class

[GitHub] spark issue #21500: Scalable Memory option for HDFSBackedStateStore

2018-06-07 Thread HeartSaVioR
Github user HeartSaVioR commented on the issue: https://github.com/apache/spark/pull/21500 @aalobaidi One thing you may want to be aware is that in point of executor's view, executor must load at least 1 version of state in memory regardless of caching versions. I guess you may

[GitHub] spark pull request #21477: [WIP] [SPARK-24396] [SS] [PYSPARK] Add Structured...

2018-06-07 Thread tdas
Github user tdas commented on a diff in the pull request: https://github.com/apache/spark/pull/21477#discussion_r193892652 --- Diff: python/pyspark/sql/tests.py --- @@ -1884,7 +1885,164 @@ def test_query_manager_await_termination(self): finally:

[GitHub] spark pull request #21477: [WIP] [SPARK-24396] [SS] [PYSPARK] Add Structured...

2018-06-07 Thread tdas
Github user tdas commented on a diff in the pull request: https://github.com/apache/spark/pull/21477#discussion_r193892565 --- Diff: python/pyspark/sql/streaming.py --- @@ -843,6 +844,169 @@ def trigger(self, processingTime=None, once=None, continuous=None):

[GitHub] spark pull request #21477: [WIP] [SPARK-24396] [SS] [PYSPARK] Add Structured...

2018-06-07 Thread tdas
Github user tdas commented on a diff in the pull request: https://github.com/apache/spark/pull/21477#discussion_r193892571 --- Diff: python/pyspark/sql/streaming.py --- @@ -843,6 +844,169 @@ def trigger(self, processingTime=None, once=None, continuous=None):

[GitHub] spark pull request #21477: [WIP] [SPARK-24396] [SS] [PYSPARK] Add Structured...

2018-06-07 Thread tdas
Github user tdas commented on a diff in the pull request: https://github.com/apache/spark/pull/21477#discussion_r193892514 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/python/PythonForeachWriter.scala --- @@ -0,0 +1,161 @@ +/* + * Licensed to the

[GitHub] spark pull request #21477: [WIP] [SPARK-24396] [SS] [PYSPARK] Add Structured...

2018-06-07 Thread tdas
Github user tdas commented on a diff in the pull request: https://github.com/apache/spark/pull/21477#discussion_r193892217 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/ForeachWriter.scala --- @@ -71,23 +110,17 @@ abstract class ForeachWriter[T] extends Serializable {

[GitHub] spark issue #21509: [SPARK-24489]Check for invalid input type of weight data...

2018-06-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21509 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21509: [24489]Check for invalid input type of weight data in ml...

2018-06-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21509 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #21509: Check for invalid input type of weight data in ml...

2018-06-07 Thread shahidki31
GitHub user shahidki31 opened a pull request: https://github.com/apache/spark/pull/21509 Check for invalid input type of weight data in ml.PowerIterationClustering ## What changes were proposed in this pull request? The test case will result the following failure. currently in

[GitHub] spark pull request #21477: [WIP] [SPARK-24396] [SS] [PYSPARK] Add Structured...

2018-06-07 Thread tdas
Github user tdas commented on a diff in the pull request: https://github.com/apache/spark/pull/21477#discussion_r193887258 --- Diff: python/pyspark/sql/streaming.py --- @@ -843,6 +844,169 @@ def trigger(self, processingTime=None, once=None, continuous=None):

[GitHub] spark pull request #21477: [WIP] [SPARK-24396] [SS] [PYSPARK] Add Structured...

2018-06-07 Thread tdas
Github user tdas commented on a diff in the pull request: https://github.com/apache/spark/pull/21477#discussion_r193887147 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/ForeachWriter.scala --- @@ -71,23 +110,17 @@ abstract class ForeachWriter[T] extends Serializable {

[GitHub] spark pull request #21477: [WIP] [SPARK-24396] [SS] [PYSPARK] Add Structured...

2018-06-07 Thread tdas
Github user tdas commented on a diff in the pull request: https://github.com/apache/spark/pull/21477#discussion_r193886200 --- Diff: python/pyspark/sql/streaming.py --- @@ -843,6 +844,169 @@ def trigger(self, processingTime=None, once=None, continuous=None):

[GitHub] spark issue #21500: Scalable Memory option for HDFSBackedStateStore

2018-06-07 Thread HeartSaVioR
Github user HeartSaVioR commented on the issue: https://github.com/apache/spark/pull/21500 @aalobaidi You can also merge #21506 (maybe with changing log level or modify the patch to set message to INFO level) and see latencies on loading state, snapshotting, cleaning up. ---

[GitHub] spark issue #21469: [SPARK-24441][SS] Expose total estimated size of states ...

2018-06-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21469 **[Test build #91535 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/91535/testReport)** for PR 21469 at commit

[GitHub] spark issue #21469: [SPARK-24441][SS] Expose total estimated size of states ...

2018-06-07 Thread HeartSaVioR
Github user HeartSaVioR commented on the issue: https://github.com/apache/spark/pull/21469 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark pull request #21477: [WIP] [SPARK-24396] [SS] [PYSPARK] Add Structured...

2018-06-07 Thread tdas
Github user tdas commented on a diff in the pull request: https://github.com/apache/spark/pull/21477#discussion_r193878552 --- Diff: python/pyspark/sql/streaming.py --- @@ -843,6 +844,169 @@ def trigger(self, processingTime=None, once=None, continuous=None):

[GitHub] spark issue #21283: [SPARK-24224][ML-Examples]Java example code for Power It...

2018-06-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21283 **[Test build #4195 has finished](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/4195/testReport)** for PR 21283 at commit

[GitHub] spark issue #21248: [SPARK-24191][ML]Scala Example code for Power Iteration ...

2018-06-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21248 **[Test build #4196 has finished](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/4196/testReport)** for PR 21248 at commit

[GitHub] spark issue #21508: [SPARK-24488] [SQL] Fix issue when generator is aliased ...

2018-06-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21508 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21508: [SPARK-24488] [SQL] Fix issue when generator is aliased ...

2018-06-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21508 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #21508: [SPARK-24488] [SQL] Fix issue when generator is a...

2018-06-07 Thread bkrieger
GitHub user bkrieger opened a pull request: https://github.com/apache/spark/pull/21508 [SPARK-24488] [SQL] Fix issue when generator is aliased multiple times ## What changes were proposed in this pull request? Currently, the Analyzer throws an exception if your try to nest

[GitHub] spark issue #21248: [SPARK-24191][ML]Scala Example code for Power Iteration ...

2018-06-07 Thread shahidki31
Github user shahidki31 commented on the issue: https://github.com/apache/spark/pull/21248 Hi @hhbyyh @srowen , PowerIterationClustering API has some modifications based on the discussion in the JIRA, https://issues.apache.org/jira/browse/SPARK-15784. The examples also have

[GitHub] spark issue #21248: [SPARK-24191][ML]Scala Example code for Power Iteration ...

2018-06-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21248 **[Test build #4196 has started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/4196/testReport)** for PR 21248 at commit

[GitHub] spark issue #21248: [SPARK-24191][ML]Scala Example code for Power Iteration ...

2018-06-07 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/21248 OK, I think this could have been combined with https://github.com/apache/spark/pull/21283, but whatever. --- - To unsubscribe,

[GitHub] spark issue #21283: [SPARK-24224][ML-Examples]Java example code for Power It...

2018-06-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21283 **[Test build #4195 has started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/4195/testReport)** for PR 21283 at commit

[GitHub] spark issue #21502: [SPARK-22575][SQL] Add destroy to Dataset

2018-06-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21502 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/91528/ Test PASSed. ---

[GitHub] spark issue #21502: [SPARK-22575][SQL] Add destroy to Dataset

2018-06-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21502 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21502: [SPARK-22575][SQL] Add destroy to Dataset

2018-06-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21502 **[Test build #91528 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/91528/testReport)** for PR 21502 at commit

[GitHub] spark issue #21283: [SPARK-24224][ML-Examples]Java example code for Power It...

2018-06-07 Thread shahidki31
Github user shahidki31 commented on the issue: https://github.com/apache/spark/pull/21283 Hi @srowen , Thanks for the comment. I have modified the code. --- - To unsubscribe, e-mail:

[GitHub] spark issue #21500: Scalable Memory option for HDFSBackedStateStore

2018-06-07 Thread aalobaidi
Github user aalobaidi commented on the issue: https://github.com/apache/spark/pull/21500 Sorry for the late reply. The option is useful for specific use case which is micro-batches with relatively large number partitions with each of the partitions is very big in size. When this

[GitHub] spark issue #21506: [SPARK-24485][SS] Measure and log elapsed time for files...

2018-06-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21506 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/91525/ Test PASSed. ---

[GitHub] spark issue #21506: [SPARK-24485][SS] Measure and log elapsed time for files...

2018-06-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21506 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21506: [SPARK-24485][SS] Measure and log elapsed time for files...

2018-06-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21506 **[Test build #91525 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/91525/testReport)** for PR 21506 at commit

[GitHub] spark pull request #21477: [WIP] [SPARK-24396] [SS] [PYSPARK] Add Structured...

2018-06-07 Thread zsxwing
Github user zsxwing commented on a diff in the pull request: https://github.com/apache/spark/pull/21477#discussion_r193839997 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/ForeachWriter.scala --- @@ -71,23 +110,17 @@ abstract class ForeachWriter[T] extends Serializable {

<    1   2   3   4   >