[GitHub] spark issue #20398: [SPARK-23221][SS][TEST] Fix KafkaContinuousSourceStressF...

2018-01-26 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/20398 Retest this please. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #20393: [SPARK-23207][SQL] Shuffle+Repartition on an RDD/DataFra...

2018-01-26 Thread mridulm
Github user mridulm commented on the issue: https://github.com/apache/spark/pull/20393 @jiangxb1987 Other than hash partitioning, I dont see how this can be handled reliably ... You are right, this is a basic correctness issue - unfortunately I never used this family of methods

[GitHub] spark pull request #20411: [SPARK-17139][ML][FOLLOW-UP] update LogisticRegre...

2018-01-26 Thread WeichenXu123
Github user WeichenXu123 closed the pull request at: https://github.com/apache/spark/pull/20411 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #20411: [SPARK-17139][ML][FOLLOW-UP] update LogisticRegressionSu...

2018-01-26 Thread WeichenXu123
Github user WeichenXu123 commented on the issue: https://github.com/apache/spark/pull/20411 @sethah ok thanks! --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #20393: [SPARK-23207][SQL] Shuffle+Repartition on an RDD/DataFra...

2018-01-26 Thread jiangxb1987
Github user jiangxb1987 commented on the issue: https://github.com/apache/spark/pull/20393 Another simple way to ensure correctness of RDD.repartition() is to do HashPartitioning instead of current RoundRobinPartitioning, but that will lead to regression when you have skew input

[GitHub] spark issue #19892: [SPARK-22797][PySpark] Bucketizer support multi-column

2018-01-26 Thread MLnick
Github user MLnick commented on the issue: https://github.com/apache/spark/pull/19892 I reverted this (see #20410 for details) - we can re-open it once that issue is solved. --- - To unsubscribe, e-mail:

[GitHub] spark issue #20410: [SPARK-23234][ML][PYSPARK] Remove setting defaults on Ja...

2018-01-26 Thread MLnick
Github user MLnick commented on the issue: https://github.com/apache/spark/pull/20410 I reverted #19892 in master (f5911d4894700eb48f794133cbd363bf3b7c8c8e) / branch-2.3 (a8a3e9b7cf7b9346c43cfbbf7b26fd2fd28dd521), so that other test runs can be unblocked. ---

[GitHub] spark issue #20393: [SPARK-23207][SQL] Shuffle+Repartition on an RDD/DataFra...

2018-01-26 Thread jiangxb1987
Github user jiangxb1987 commented on the issue: https://github.com/apache/spark/pull/20393 Actually the similar approach cannot apply to fix RDD.repartition(), as in RDD[T], the data type `T` can be non-comparable, so we are not able to perform a local sort before actually

[GitHub] spark issue #20347: [SPARK-20129][Core] JavaSparkContext should use SparkCon...

2018-01-26 Thread rekhajoshm
Github user rekhajoshm commented on the issue: https://github.com/apache/spark/pull/20347 Thank you @srowen I admire you for doing what you do over all the jira/PR's I have studied, and followed up. If its ok, will keep this PR open for few days, and close if jira is getting to

[GitHub] spark pull request #20399: [SPARK-23209][core] Allow credential manager to w...

2018-01-26 Thread squito
Github user squito commented on a diff in the pull request: https://github.com/apache/spark/pull/20399#discussion_r164223721 --- Diff: core/src/main/scala/org/apache/spark/deploy/security/HadoopDelegationTokenManager.scala --- @@ -75,6 +75,17 @@ private[spark] class

[GitHub] spark issue #20385: [SPARK-21396][SQL] Fixes MatchError when UDTs are passed...

2018-01-26 Thread atallahhezbor
Github user atallahhezbor commented on the issue: https://github.com/apache/spark/pull/20385 @liufengdb @gatorsmile I'm happy to write a unit test if you require it. Though as I mentioned before, I did not see a clear way of testing the functionality in

[GitHub] spark issue #20397: [SPARK-23219][SQL]Rename ReadTask to DataReaderFactory i...

2018-01-26 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20397 **[Test build #86721 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86721/testReport)** for PR 20397 at commit

[GitHub] spark issue #20397: [SPARK-23219][SQL]Rename ReadTask to DataReaderFactory i...

2018-01-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20397 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/298/

[GitHub] spark issue #20397: [SPARK-23219][SQL]Rename ReadTask to DataReaderFactory i...

2018-01-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20397 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20397: [SPARK-23219][SQL]Rename ReadTask to DataReaderFactory i...

2018-01-26 Thread jiangxb1987
Github user jiangxb1987 commented on the issue: https://github.com/apache/spark/pull/20397 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #19408: [SPARK-22180][CORE] Allow IPv6 address in org.apache.spa...

2018-01-26 Thread jiangxb1987
Github user jiangxb1987 commented on the issue: https://github.com/apache/spark/pull/19408 Fully agree with @mridulm 's big picture suggestion, and I also think supporting IPv6 should be designed as a integral feature, instead of just putting together some PRs. ---

[GitHub] spark issue #20403: [SPARK-23238][PYTHON] Externalize SQLConf spark.sql.exec...

2018-01-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20403 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20403: [SPARK-23238][PYTHON] Externalize SQLConf spark.sql.exec...

2018-01-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20403 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/86716/ Test FAILed. ---

[GitHub] spark issue #20403: [SPARK-23238][PYTHON] Externalize SQLConf spark.sql.exec...

2018-01-26 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20403 **[Test build #86716 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86716/testReport)** for PR 20403 at commit

[GitHub] spark issue #20408: [SPARK-23189][Core][Web UI] Reflect stage level blacklis...

2018-01-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20408 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/86715/ Test FAILed. ---

[GitHub] spark issue #20408: [SPARK-23189][Core][Web UI] Reflect stage level blacklis...

2018-01-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20408 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20408: [SPARK-23189][Core][Web UI] Reflect stage level blacklis...

2018-01-26 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20408 **[Test build #86715 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86715/testReport)** for PR 20408 at commit

[GitHub] spark pull request #20399: [SPARK-23209][core] Allow credential manager to w...

2018-01-26 Thread mridulm
Github user mridulm commented on a diff in the pull request: https://github.com/apache/spark/pull/20399#discussion_r164212154 --- Diff: core/src/main/scala/org/apache/spark/deploy/security/HadoopDelegationTokenManager.scala --- @@ -75,6 +75,17 @@ private[spark] class

[GitHub] spark issue #20410: [SPARK-23234][ML][PYSPARK] Remove setting defaults on Ja...

2018-01-26 Thread MLnick
Github user MLnick commented on the issue: https://github.com/apache/spark/pull/20410 I think this is somewhat related to #15113 cc @BryanCutler --- - To unsubscribe, e-mail:

[GitHub] spark pull request #20400: [SPARK-23084][PYTHON]Add unboundedPreceding(), un...

2018-01-26 Thread jiangxb1987
Github user jiangxb1987 commented on a diff in the pull request: https://github.com/apache/spark/pull/20400#discussion_r164207466 --- Diff: python/pyspark/sql/window.py --- @@ -124,16 +124,19 @@ def rangeBetween(start, end): values directly. :param

[GitHub] spark pull request #20400: [SPARK-23084][PYTHON]Add unboundedPreceding(), un...

2018-01-26 Thread jiangxb1987
Github user jiangxb1987 commented on a diff in the pull request: https://github.com/apache/spark/pull/20400#discussion_r164207282 --- Diff: python/pyspark/sql/window.py --- @@ -124,16 +124,19 @@ def rangeBetween(start, end): values directly. :param

[GitHub] spark issue #20410: [SPARK-23234][ML][PYSPARK] Remove setting defaults on Ja...

2018-01-26 Thread MLnick
Github user MLnick commented on the issue: https://github.com/apache/spark/pull/20410 We should just revert SPARK-22797 for now to unblock others. SPARK-22799 itself is not the cause per se (it passed tests) but after it was merged SPARK-22797 causes the failure. ---

[GitHub] spark pull request #20400: [SPARK-23084][PYTHON]Add unboundedPreceding(), un...

2018-01-26 Thread jiangxb1987
Github user jiangxb1987 commented on a diff in the pull request: https://github.com/apache/spark/pull/20400#discussion_r164205837 --- Diff: python/pyspark/sql/window.py --- @@ -124,16 +124,19 @@ def rangeBetween(start, end): values directly. :param

[GitHub] spark issue #19408: [SPARK-22180][CORE] Allow IPv6 address in org.apache.spa...

2018-01-26 Thread mridulm
Github user mridulm commented on the issue: https://github.com/apache/spark/pull/19408 To rephrase @jiangxb1987's question - supporting IPv6 is a much larger effort, which spark currently does not. We should be addressing that as the problem to solve, and fix this as part of IPv6

[GitHub] spark issue #20411: [SPARK-17139][ML][FOLLOW-UP] update LogisticRegressionSu...

2018-01-26 Thread sethah
Github user sethah commented on the issue: https://github.com/apache/spark/pull/20411 This is already fixed in #20332. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands,

[GitHub] spark issue #20369: [SPARK-23196] Unify continuous and microbatch V2 sinks

2018-01-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20369 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20369: [SPARK-23196] Unify continuous and microbatch V2 sinks

2018-01-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20369 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/86714/ Test FAILed. ---

[GitHub] spark issue #20369: [SPARK-23196] Unify continuous and microbatch V2 sinks

2018-01-26 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20369 **[Test build #86714 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86714/testReport)** for PR 20369 at commit

[GitHub] spark issue #19575: [SPARK-22221][DOCS] Adding User Documentation for Arrow

2018-01-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19575 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/86720/ Test PASSed. ---

[GitHub] spark issue #19575: [SPARK-22221][DOCS] Adding User Documentation for Arrow

2018-01-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19575 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19575: [SPARK-22221][DOCS] Adding User Documentation for Arrow

2018-01-26 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19575 **[Test build #86720 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86720/testReport)** for PR 19575 at commit

[GitHub] spark issue #19575: [SPARK-22221][DOCS] Adding User Documentation for Arrow

2018-01-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19575 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/86719/ Test PASSed. ---

[GitHub] spark issue #19575: [SPARK-22221][DOCS] Adding User Documentation for Arrow

2018-01-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19575 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19575: [SPARK-22221][DOCS] Adding User Documentation for Arrow

2018-01-26 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19575 **[Test build #86719 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86719/testReport)** for PR 19575 at commit

[GitHub] spark issue #20411: [SPARK-17139][ML][FOLLOW-UP] update LogisticRegressionSu...

2018-01-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20411 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20411: [SPARK-17139][ML][FOLLOW-UP] update LogisticRegressionSu...

2018-01-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20411 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/86718/ Test PASSed. ---

[GitHub] spark issue #20403: [SPARK-23238][PYTHON] Externalize SQLConf spark.sql.exec...

2018-01-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20403 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20411: [SPARK-17139][ML][FOLLOW-UP] update LogisticRegressionSu...

2018-01-26 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20411 **[Test build #86718 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86718/testReport)** for PR 20411 at commit

[GitHub] spark issue #20403: [SPARK-23238][PYTHON] Externalize SQLConf spark.sql.exec...

2018-01-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20403 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/86712/ Test FAILed. ---

[GitHub] spark issue #20403: [SPARK-23238][PYTHON] Externalize SQLConf spark.sql.exec...

2018-01-26 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20403 **[Test build #86712 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86712/testReport)** for PR 20403 at commit

[GitHub] spark issue #20410: [SPARK-23234][ML][PYSPARK] Remove setting defaults on Ja...

2018-01-26 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/20410 CC @zhengruifeng @MLnick --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #19575: [SPARK-22221][DOCS] Adding User Documentation for Arrow

2018-01-26 Thread BryanCutler
Github user BryanCutler commented on the issue: https://github.com/apache/spark/pull/19575 Updated screens: ![arrow_doc_1](https://user-images.githubusercontent.com/4534389/35456374-11f5a186-028a-11e8-88a0-892163eaccd1.png)

[GitHub] spark issue #19575: [SPARK-22221][DOCS] Adding User Documentation for Arrow

2018-01-26 Thread BryanCutler
Github user BryanCutler commented on the issue: https://github.com/apache/spark/pull/19575 Thanks everyone for reviewing! I think I addressed all the comments, so please take one more look. --- - To unsubscribe,

[GitHub] spark issue #19575: [SPARK-22221][DOCS] Adding User Documentation for Arrow

2018-01-26 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19575 **[Test build #86720 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86720/testReport)** for PR 19575 at commit

[GitHub] spark issue #19575: [SPARK-22221][DOCS] Adding User Documentation for Arrow

2018-01-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19575 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19575: [SPARK-22221][DOCS] Adding User Documentation for Arrow

2018-01-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19575 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/297/

[GitHub] spark issue #19575: [SPARK-22221][DOCS] Adding User Documentation for Arrow

2018-01-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19575 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/296/

[GitHub] spark issue #19575: [SPARK-22221][DOCS] Adding User Documentation for Arrow

2018-01-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19575 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #19575: [SPARK-22221][DOCS] Adding User Documentation for...

2018-01-26 Thread BryanCutler
Github user BryanCutler commented on a diff in the pull request: https://github.com/apache/spark/pull/19575#discussion_r164195790 --- Diff: docs/sql-programming-guide.md --- @@ -1640,6 +1640,133 @@ Configuration of Hive is done by placing your `hive-site.xml`, `core-site.xml` a

[GitHub] spark issue #19575: [SPARK-22221][DOCS] Adding User Documentation for Arrow

2018-01-26 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19575 **[Test build #86719 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86719/testReport)** for PR 19575 at commit

[GitHub] spark issue #20411: [SPARK-17139][ML][FOLLOW-UP] update LogisticRegressionSu...

2018-01-26 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20411 **[Test build #86718 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86718/testReport)** for PR 20411 at commit

[GitHub] spark issue #20411: [SPARK-17139][ML][FOLLOW-UP] update LogisticRegressionSu...

2018-01-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20411 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20411: [SPARK-17139][ML][FOLLOW-UP] update LogisticRegressionSu...

2018-01-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20411 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/295/

[GitHub] spark pull request #20411: [SPARK-17139][ML][FOLLOW-UP] update LogisticRegre...

2018-01-26 Thread WeichenXu123
GitHub user WeichenXu123 opened a pull request: https://github.com/apache/spark/pull/20411 [SPARK-17139][ML][FOLLOW-UP] update LogisticRegressionSummaryExample code ## What changes were proposed in this pull request? New method `trainingSummary.asBinary` added so in this

[GitHub] spark issue #20405: [SPARK-23229][SQL] Dataset.hint should use planWithBarri...

2018-01-26 Thread jaceklaskowski
Github user jaceklaskowski commented on the issue: https://github.com/apache/spark/pull/20405 Looks like the tests failed due to _"java.io.IOException: Failed to delete: /home/jenkins/workspace/SparkPullRequestBuilder/target/tmp/spark-5a9b5811-306d-4ba0-8bfb-9e263ddf47b8"_ Is this

[GitHub] spark pull request #20405: [SPARK-23229][SQL] Dataset.hint should use planWi...

2018-01-26 Thread jaceklaskowski
Github user jaceklaskowski commented on a diff in the pull request: https://github.com/apache/spark/pull/20405#discussion_r164191381 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/Dataset.scala --- @@ -1216,7 +1216,7 @@ class Dataset[T] private[sql]( */

[GitHub] spark issue #20397: [SPARK-23219][SQL]Rename ReadTask to DataReaderFactory i...

2018-01-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20397 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20397: [SPARK-23219][SQL]Rename ReadTask to DataReaderFactory i...

2018-01-26 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20397 **[Test build #86708 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86708/testReport)** for PR 20397 at commit

[GitHub] spark issue #20397: [SPARK-23219][SQL]Rename ReadTask to DataReaderFactory i...

2018-01-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20397 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/86708/ Test FAILed. ---

[GitHub] spark issue #20410: [SPARK-23234][ML][PYSPARK] Remove setting defaults on Ja...

2018-01-26 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20410 **[Test build #86717 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86717/testReport)** for PR 20410 at commit

[GitHub] spark issue #20410: [SPARK-23234][ML][PYSPARK] Remove setting defaults on Ja...

2018-01-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20410 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/86717/ Test PASSed. ---

[GitHub] spark issue #20410: [SPARK-23234][ML][PYSPARK] Remove setting defaults on Ja...

2018-01-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20410 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20373: [SPARK-23159][PYTHON] Update cloudpickle to match 0.4.2

2018-01-26 Thread BryanCutler
Github user BryanCutler commented on the issue: https://github.com/apache/spark/pull/20373 @holdenk and @HyukjinKwon , is there any further testing you guys can think that needs to be done to verify this is ok? ---

[GitHub] spark issue #20373: [SPARK-23159][PYTHON] Update cloudpickle to match 0.4.2

2018-01-26 Thread BryanCutler
Github user BryanCutler commented on the issue: https://github.com/apache/spark/pull/20373 > Wait, so we left out cloudpickle#113 even though its in 0.4.2? That patch is in here and this exactly matches 0.4.2. I also manually verified that cloudpickle will pickle named

[GitHub] spark pull request #20409: [SPARK-23233][PYTHON] Reset the cache in asNondet...

2018-01-26 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/20409#discussion_r164178516 --- Diff: python/pyspark/sql/udf.py --- @@ -188,6 +188,9 @@ def asNondeterministic(self): .. versionadded:: 2.3 """ +

[GitHub] spark pull request #20408: [SPARK-23189][Core][Web UI] Reflect stage level b...

2018-01-26 Thread squito
Github user squito commented on a diff in the pull request: https://github.com/apache/spark/pull/20408#discussion_r164177919 --- Diff: core/src/main/scala/org/apache/spark/status/AppStatusListener.scala --- @@ -594,12 +606,24 @@ private[spark] class AppStatusListener(

[GitHub] spark pull request #20408: [SPARK-23189][Core][Web UI] Reflect stage level b...

2018-01-26 Thread squito
Github user squito commented on a diff in the pull request: https://github.com/apache/spark/pull/20408#discussion_r164175040 --- Diff: core/src/main/resources/org/apache/spark/ui/static/executorspage.js --- @@ -25,9 +25,13 @@ function getThreadDumpEnabled() { return

[GitHub] spark issue #20403: [SPARK-23238][PYTHON] Externalize SQLConf spark.sql.exec...

2018-01-26 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/20403 LGTM --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark pull request #19575: [SPARK-22221][DOCS] Adding User Documentation for...

2018-01-26 Thread BryanCutler
Github user BryanCutler commented on a diff in the pull request: https://github.com/apache/spark/pull/19575#discussion_r164176830 --- Diff: docs/sql-programming-guide.md --- @@ -1640,6 +1640,129 @@ Configuration of Hive is done by placing your `hive-site.xml`, `core-site.xml` a

[GitHub] spark issue #20410: [SPARK-23234][ML][PYSPARK] Remove setting defaults on Ja...

2018-01-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20410 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/294/

[GitHub] spark issue #20410: [SPARK-23234][ML][PYSPARK] Remove setting defaults on Ja...

2018-01-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20410 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #19575: [SPARK-22221][DOCS] Adding User Documentation for...

2018-01-26 Thread BryanCutler
Github user BryanCutler commented on a diff in the pull request: https://github.com/apache/spark/pull/19575#discussion_r164175878 --- Diff: docs/sql-programming-guide.md --- @@ -1640,6 +1640,129 @@ Configuration of Hive is done by placing your `hive-site.xml`, `core-site.xml` a

[GitHub] spark pull request #19575: [SPARK-22221][DOCS] Adding User Documentation for...

2018-01-26 Thread BryanCutler
Github user BryanCutler commented on a diff in the pull request: https://github.com/apache/spark/pull/19575#discussion_r164175749 --- Diff: docs/sql-programming-guide.md --- @@ -1640,6 +1640,129 @@ Configuration of Hive is done by placing your `hive-site.xml`, `core-site.xml` a

[GitHub] spark pull request #19575: [SPARK-22221][DOCS] Adding User Documentation for...

2018-01-26 Thread BryanCutler
Github user BryanCutler commented on a diff in the pull request: https://github.com/apache/spark/pull/19575#discussion_r164175559 --- Diff: docs/sql-programming-guide.md --- @@ -1640,6 +1640,129 @@ Configuration of Hive is done by placing your `hive-site.xml`, `core-site.xml` a

[GitHub] spark issue #20410: [SPARK-23234][ML][PYSPARK] Remove setting defaults on Ja...

2018-01-26 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20410 **[Test build #86717 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86717/testReport)** for PR 20410 at commit

[GitHub] spark issue #20402: [SPARK-23223][SQL] Make stacking dataset transforms more...

2018-01-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20402 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/86710/ Test FAILed. ---

[GitHub] spark issue #20402: [SPARK-23223][SQL] Make stacking dataset transforms more...

2018-01-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20402 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20402: [SPARK-23223][SQL] Make stacking dataset transforms more...

2018-01-26 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20402 **[Test build #86710 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86710/testReport)** for PR 20402 at commit

[GitHub] spark pull request #19575: [SPARK-22221][DOCS] Adding User Documentation for...

2018-01-26 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/19575#discussion_r164172824 --- Diff: docs/sql-programming-guide.md --- @@ -1640,6 +1640,129 @@ Configuration of Hive is done by placing your `hive-site.xml`, `core-site.xml` a

[GitHub] spark issue #20409: [SPARK-23233][PYTHON] Reset the cache in asNondeterminis...

2018-01-26 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/20409 FYI @ptkool. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #20403: [SPARK-23238][PYTHON] Minor doc correction for 'spark.sq...

2018-01-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20403 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/293/

[GitHub] spark issue #20403: [SPARK-23238][PYTHON] Minor doc correction for 'spark.sq...

2018-01-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20403 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20403: [SPARK-23238][PYTHON] Minor doc correction for 'spark.sq...

2018-01-26 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20403 **[Test build #86716 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86716/testReport)** for PR 20403 at commit

[GitHub] spark pull request #20403: [MINOR][PYTHON] Minor doc correction for 'spark.s...

2018-01-26 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/20403#discussion_r164170188 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala --- @@ -1045,9 +1045,10 @@ object SQLConf {

[GitHub] spark pull request #20395: [SPARK-23218][SQL] simplify ColumnVector.getArray

2018-01-26 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/20395 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #20395: [SPARK-23218][SQL] simplify ColumnVector.getArray

2018-01-26 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/20395 Thanks! Merged to master/2.3 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands,

[GitHub] spark pull request #20387: [SPARK-23203][SPARK-23204][SQL]: DataSourceV2: Us...

2018-01-26 Thread rdblue
Github user rdblue commented on a diff in the pull request: https://github.com/apache/spark/pull/20387#discussion_r164169060 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/v2/DataSourceV2Relation.scala --- @@ -17,15 +17,149 @@ package

[GitHub] spark pull request #20403: [MINOR][PYTHON] Minor doc correction for 'spark.s...

2018-01-26 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/20403#discussion_r164163536 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala --- @@ -1045,9 +1045,10 @@ object SQLConf {

[GitHub] spark issue #20408: [SPARK-23189][Core][Web UI] Reflect stage level blacklis...

2018-01-26 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20408 **[Test build #86715 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86715/testReport)** for PR 20408 at commit

[GitHub] spark issue #20401: [MINOR][SS][DOC] Fix `Trigger` Scala/Java doc examples

2018-01-26 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/20401 Thank you for review, @jerryshao and @srowen . Yep. This is all I found about `Trigger`. --- - To unsubscribe, e-mail:

[GitHub] spark issue #20408: [SPARK-23189][Core][Web UI] Reflect stage level blacklis...

2018-01-26 Thread tgravescs
Github user tgravescs commented on the issue: https://github.com/apache/spark/pull/20408 Jenkins, okay to test --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark pull request #20403: [MINOR][PYTHON] Minor doc correction for 'spark.s...

2018-01-26 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/20403#discussion_r164161869 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala --- @@ -1045,9 +1045,10 @@ object SQLConf {

[GitHub] spark issue #20408: [SPARK-23189][Core][Web UI] Reflect stage level blacklis...

2018-01-26 Thread tgravescs
Github user tgravescs commented on the issue: https://github.com/apache/spark/pull/20408 Jenkins, ok to test --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #20408: [SPARK-23189][Core][Web UI] Reflect stage level blacklis...

2018-01-26 Thread attilapiros
Github user attilapiros commented on the issue: https://github.com/apache/spark/pull/20408 cc @squito @tgravescs --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark pull request #20395: [SPARK-23218][SQL] simplify ColumnVector.getArray

2018-01-26 Thread dongjoon-hyun
Github user dongjoon-hyun commented on a diff in the pull request: https://github.com/apache/spark/pull/20395#discussion_r164160869 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/vectorized/ColumnarBatchBenchmark.scala --- @@ -479,10 +475,10 @@ object

<    1   2   3   4   5   >