[GitHub] spark pull request #22320: [SPARK-25313][SQL]Fix regression in FileFormatWri...

2018-09-05 Thread jaceklaskowski
Github user jaceklaskowski commented on a diff in the pull request: https://github.com/apache/spark/pull/22320#discussion_r215376132 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/execution/CreateHiveTableAsSelectCommand.scala --- @@ -82,7 +83,7 @@ case class

[GitHub] spark issue #22112: [SPARK-23243][Core] Fix RDD.repartition() data correctne...

2018-09-05 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/22112 The current solution looks good to me for unblocking the Apache 2.4 release. We definitely should continue improving the fix, as what the other reviewers suggested above. ---

[GitHub] spark pull request #22192: [SPARK-24918][Core] Executor Plugin API

2018-09-05 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/22192#discussion_r215371723 --- Diff: core/src/main/java/org/apache/spark/ExecutorPlugin.java --- @@ -0,0 +1,50 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under

[GitHub] spark pull request #22192: [SPARK-24918][Core] Executor Plugin API

2018-09-05 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/22192#discussion_r215372559 --- Diff: core/src/main/java/org/apache/spark/ExecutorPlugin.java --- @@ -0,0 +1,50 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under

[GitHub] spark pull request #22192: [SPARK-24918][Core] Executor Plugin API

2018-09-05 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/22192#discussion_r215373277 --- Diff: core/src/main/java/org/apache/spark/ExecutorPlugin.java --- @@ -0,0 +1,50 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under

[GitHub] spark pull request #22325: [SPARK-25318]. Add exception handling when wrappi...

2018-09-05 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/22325#discussion_r215368719 --- Diff: core/src/main/scala/org/apache/spark/storage/ShuffleBlockFetcherIterator.scala --- @@ -444,10 +444,23 @@ final class ShuffleBlockFetcherIterator(

[GitHub] spark issue #21404: [SPARK-24360][SQL] Support Hive 3.0 metastore

2018-09-05 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/21404 Thank you for review, @tooptoop4 and @wangyum . I'm going to update this to the latest Hive 3.1.0. --- - To

[GitHub] spark issue #22333: [SPARK-25335][BUILD] Skip Zinc downloading if it's insta...

2018-09-05 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22333 **[Test build #95728 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95728/testReport)** for PR 22333 at commit

[GitHub] spark issue #22333: [SPARK-25335][BUILD] Skip Zinc downloading if it's insta...

2018-09-05 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22333 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #22333: [SPARK-25335][BUILD] Skip Zinc downloading if it's insta...

2018-09-05 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22333 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request #22333: [SPARK-25335][BUILD] Skip Zinc downloading if it'...

2018-09-05 Thread dongjoon-hyun
Github user dongjoon-hyun commented on a diff in the pull request: https://github.com/apache/spark/pull/22333#discussion_r215365678 --- Diff: build/mvn --- @@ -91,15 +92,23 @@ install_mvn() { # Install zinc under the build/ folder install_zinc() { - local

[GitHub] spark issue #22112: [SPARK-23243][Core] Fix RDD.repartition() data correctne...

2018-09-05 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/22112 @tgravescs thanks for testing it out! I've created https://issues.apache.org/jira/browse/SPARK-25341 and https://issues.apache.org/jira/browse/SPARK-25342 to track the followup. I think

[GitHub] spark issue #22340: [SPARK-25337][SQL][TEST] `runSparkSubmit` should provide...

2018-09-05 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/22340 - Since `runSparkSubmit` is a part of `SparkSubmitTestUtils`, some test case will fail if we always remove the env variables like `SPARK_DIST_CLASSPATH`. It gives the child the correct

[GitHub] spark issue #22209: [SPARK-24415][Core] Fixed the aggregated stage metrics b...

2018-09-05 Thread vanzin
Github user vanzin commented on the issue: https://github.com/apache/spark/pull/22209 (Feel free to do it if you think it should be there, btw.) --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For

[GitHub] spark issue #22209: [SPARK-24415][Core] Fixed the aggregated stage metrics b...

2018-09-05 Thread vanzin
Github user vanzin commented on the issue: https://github.com/apache/spark/pull/22209 I guess it should be ok. Just trying to be conservative and not inadvertently making the branch less stable in the middle of an rc cycle... ---

[GitHub] spark issue #22318: [SPARK-25150][SQL] Rewrite condition when deduplicate Jo...

2018-09-05 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22318 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/95720/ Test PASSed. ---

[GitHub] spark issue #22318: [SPARK-25150][SQL] Rewrite condition when deduplicate Jo...

2018-09-05 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22318 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #22335: [SPARK-25091][SQL] reduce the storage memory in E...

2018-09-05 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/22335#discussion_r215359589 --- Diff: core/src/main/scala/org/apache/spark/status/LiveEntity.scala --- @@ -509,6 +507,10 @@ private class LiveRDD(val info: RDDInfo) extends LiveEntity

[GitHub] spark pull request #22335: [SPARK-25091][SQL] reduce the storage memory in E...

2018-09-05 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/22335#discussion_r215360659 --- Diff: core/src/main/scala/org/apache/spark/status/AppStatusListener.scala --- @@ -710,9 +719,21 @@ private[spark] class AppStatusListener( val

[GitHub] spark pull request #22335: [SPARK-25091][SQL] reduce the storage memory in E...

2018-09-05 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/22335#discussion_r215360036 --- Diff: core/src/main/scala/org/apache/spark/storage/BlockManager.scala --- @@ -1549,7 +1549,7 @@ private[spark] class BlockManager( // TODO:

[GitHub] spark pull request #22335: [SPARK-25091][SQL] reduce the storage memory in E...

2018-09-05 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/22335#discussion_r215359111 --- Diff: core/src/main/scala/org/apache/spark/status/LiveEntity.scala --- @@ -120,7 +118,7 @@ private class LiveTask( stageAttemptId: Int,

[GitHub] spark issue #22318: [SPARK-25150][SQL] Rewrite condition when deduplicate Jo...

2018-09-05 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22318 **[Test build #95720 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95720/testReport)** for PR 22318 at commit

[GitHub] spark issue #22209: [SPARK-24415][Core] Fixed the aggregated stage metrics b...

2018-09-05 Thread squito
Github user squito commented on the issue: https://github.com/apache/spark/pull/22209 any reason not to merge to 2.3? its a bug in 2.3 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For

[GitHub] spark issue #22328: [SPARK-22666][ML][SQL] Spark datasource for image format

2018-09-05 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22328 **[Test build #95727 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95727/testReport)** for PR 22328 at commit

[GitHub] spark issue #22328: [SPARK-22666][ML][SQL] Spark datasource for image format

2018-09-05 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22328 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #22335: [SPARK-25091][SQL] reduce the storage memory in E...

2018-09-05 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/22335#discussion_r215358096 --- Diff: core/src/main/scala/org/apache/spark/status/AppStatusListener.scala --- @@ -646,8 +646,17 @@ private[spark] class AppStatusListener( }

[GitHub] spark issue #22328: [SPARK-22666][ML][SQL] Spark datasource for image format

2018-09-05 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22328 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark issue #22328: [SPARK-22666][ML][SQL] Spark datasource for image format

2018-09-05 Thread mengxr
Github user mengxr commented on the issue: https://github.com/apache/spark/pull/22328 The image data source tests passed but JVM crashed at the end. Triggered another test. --- - To unsubscribe, e-mail:

[GitHub] spark issue #22328: [SPARK-22666][ML][SQL] Spark datasource for image format

2018-09-05 Thread mengxr
Github user mengxr commented on the issue: https://github.com/apache/spark/pull/22328 test this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark pull request #22140: [SPARK-25072][PySpark] Forbid extra value for cus...

2018-09-05 Thread BryanCutler
Github user BryanCutler commented on a diff in the pull request: https://github.com/apache/spark/pull/22140#discussion_r215356501 --- Diff: python/pyspark/sql/types.py --- @@ -1397,6 +1397,8 @@ def _create_row_inbound_converter(dataType): def _create_row(fields,

[GitHub] spark pull request #22140: [SPARK-25072][PySpark] Forbid extra value for cus...

2018-09-05 Thread BryanCutler
Github user BryanCutler commented on a diff in the pull request: https://github.com/apache/spark/pull/22140#discussion_r215355538 --- Diff: python/pyspark/sql/tests.py --- @@ -269,6 +269,10 @@ def test_struct_field_type_name(self): struct_field = StructField("a",

[GitHub] spark issue #22326: [SPARK-25314][SQL] Fix Python UDF accessing attibutes fr...

2018-09-05 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22326 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #22326: [SPARK-25314][SQL] Fix Python UDF accessing attibutes fr...

2018-09-05 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22326 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/95718/ Test PASSed. ---

[GitHub] spark issue #22326: [SPARK-25314][SQL] Fix Python UDF accessing attibutes fr...

2018-09-05 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22326 **[Test build #95718 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95718/testReport)** for PR 22326 at commit

[GitHub] spark issue #22112: [SPARK-23243][Core] Fix RDD.repartition() data correctne...

2018-09-05 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22112 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/95713/ Test PASSed. ---

[GitHub] spark issue #22112: [SPARK-23243][Core] Fix RDD.repartition() data correctne...

2018-09-05 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22112 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #22112: [SPARK-23243][Core] Fix RDD.repartition() data correctne...

2018-09-05 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22112 **[Test build #95713 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95713/testReport)** for PR 22112 at commit

[GitHub] spark issue #22325: [SPARK-25318]. Add exception handling when wrapping the ...

2018-09-05 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22325 **[Test build #95726 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95726/testReport)** for PR 22325 at commit

[GitHub] spark pull request #22209: [SPARK-24415][Core] Fixed the aggregated stage me...

2018-09-05 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/22209 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #22325: [SPARK-25318]. Add exception handling when wrapping the ...

2018-09-05 Thread vanzin
Github user vanzin commented on the issue: https://github.com/apache/spark/pull/22325 ok to test --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark pull request #22329: [SPARK-25328][PYTHON] Add an example for having t...

2018-09-05 Thread BryanCutler
Github user BryanCutler commented on a diff in the pull request: https://github.com/apache/spark/pull/22329#discussion_r215345817 --- Diff: python/pyspark/sql/functions.py --- @@ -2804,6 +2804,22 @@ def pandas_udf(f=None, returnType=None, functionType=None): | 1|1.5|

[GitHub] spark pull request #22209: [SPARK-24415][Core] Fixed the aggregated stage me...

2018-09-05 Thread ankuriitg
Github user ankuriitg commented on a diff in the pull request: https://github.com/apache/spark/pull/22209#discussion_r215345106 --- Diff: streaming/src/test/scala/org/apache/spark/streaming/UISeleniumSuite.scala --- @@ -77,7 +77,14 @@ class UISeleniumSuite

[GitHub] spark issue #22209: [SPARK-24415][Core] Fixed the aggregated stage metrics b...

2018-09-05 Thread vanzin
Github user vanzin commented on the issue: https://github.com/apache/spark/pull/22209 Merging to master. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark pull request #22209: [SPARK-24415][Core] Fixed the aggregated stage me...

2018-09-05 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/22209#discussion_r215344056 --- Diff: streaming/src/test/scala/org/apache/spark/streaming/UISeleniumSuite.scala --- @@ -77,7 +77,14 @@ class UISeleniumSuite

[GitHub] spark pull request #22209: [SPARK-24415][Core] Fixed the aggregated stage me...

2018-09-05 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/22209#discussion_r215343938 --- Diff: core/src/test/scala/org/apache/spark/status/AppStatusListenerSuite.scala --- @@ -1190,6 +1190,61 @@ class AppStatusListenerSuite extends

[GitHub] spark issue #22327: [SPARK-25330][BUILD] Revert Hadoop 2.7 to 2.7.3

2018-09-05 Thread steveloughran
Github user steveloughran commented on the issue: https://github.com/apache/spark/pull/22327 The goal for the 2.7.x line should be "nothing breaks", which is precisely why it's only getting critical patches. Reverting might make the problem go away, but you can assume that everyone

[GitHub] spark issue #22340: [SPARK-25337][SQL][TEST] `runSparkSubmit` should provide...

2018-09-05 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/22340 I see. It does seem like we don't want to run with test env variables in this context. I was going to ask if we ever do? should this function always strip the env variables for testing? I can see

[GitHub] spark issue #22328: [SPARK-22666][ML][SQL] Spark datasource for image format

2018-09-05 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22328 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/95724/ Test FAILed. ---

[GitHub] spark issue #22328: [SPARK-22666][ML][SQL] Spark datasource for image format

2018-09-05 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22328 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #22328: [SPARK-22666][ML][SQL] Spark datasource for image format

2018-09-05 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22328 **[Test build #95724 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95724/testReport)** for PR 22328 at commit

[GitHub] spark issue #22327: [SPARK-25330][BUILD] Revert Hadoop 2.7 to 2.7.3

2018-09-05 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/22327 If it's an unintentional regression in Hadoop, and is impacting users, I'd agree with down-grading the version here (at least to the version that doesn't exhibit the issue). If it were

[GitHub] spark issue #22262: [SPARK-25175][SQL] Field resolution should fail if there...

2018-09-05 Thread seancxmao
Github user seancxmao commented on the issue: https://github.com/apache/spark/pull/22262 @dongjoon-hyun I have updated PR description to explain in more details. As you mentioned, this PR is specific to the case when reading from data source table persisted in metastore. ---

[GitHub] spark pull request #22323: [SPARK-25262][K8S] Allow SPARK_LOCAL_DIRS to be t...

2018-09-05 Thread rvesse
Github user rvesse commented on a diff in the pull request: https://github.com/apache/spark/pull/22323#discussion_r215338145 --- Diff: docs/running-on-kubernetes.md --- @@ -215,6 +215,19 @@ spark.kubernetes.driver.volumes.persistentVolumeClaim.checkpointpvc.options.clai

[GitHub] spark pull request #21669: [SPARK-23257][K8S] Kerberos Support for Spark on ...

2018-09-05 Thread suryag10
Github user suryag10 commented on a diff in the pull request: https://github.com/apache/spark/pull/21669#discussion_r215338063 --- Diff: resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/features/hadoopsteps/HadoopBootstrapUtil.scala --- @@ -0,0 +1,186

[GitHub] spark issue #20999: [WIP][SPARK-14922][SPARK-23866][SQL] Support partition f...

2018-09-05 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20999 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark issue #20999: [WIP][SPARK-14922][SPARK-23866][SQL] Support partition f...

2018-09-05 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20999 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20999: [WIP][SPARK-23866][SQL] Support partition filters in ALT...

2018-09-05 Thread mgaido91
Github user mgaido91 commented on the issue: https://github.com/apache/spark/pull/20999 I am reopening this according to the discussion in https://github.com/apache/spark/pull/19691. cc @maropu @DazhuangSu ---

[GitHub] spark issue #22327: [SPARK-25330][BUILD] Revert Hadoop 2.7 to 2.7.3

2018-09-05 Thread steveloughran
Github user steveloughran commented on the issue: https://github.com/apache/spark/pull/22327 The 2.7.x branches updates are generally all security plus some fixes for JVM/JDK regressions. without looking at the details, you can assume that the regression will be related to

[GitHub] spark issue #20999: [WIP][SPARK-23866][SQL] Support partition filters in ALT...

2018-09-05 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20999 **[Test build #95725 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95725/testReport)** for PR 20999 at commit

[GitHub] spark pull request #20999: [WIP][SPARK-23866][SQL] Support partition filters...

2018-09-05 Thread mgaido91
GitHub user mgaido91 reopened a pull request: https://github.com/apache/spark/pull/20999 [WIP][SPARK-23866][SQL] Support partition filters in ALTER TABLE DROP PARTITION ## What changes were proposed in this pull request? Hive has been supporting for a while the ability of

[GitHub] spark issue #22340: [SPARK-25337][SQL][TEST] `runSparkSubmit` should provide...

2018-09-05 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/22340 Also, cc @gatorsmile . --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark pull request #22141: [SPARK-25154][SQL] Support NOT IN sub-queries ins...

2018-09-05 Thread dilipbiswal
Github user dilipbiswal commented on a diff in the pull request: https://github.com/apache/spark/pull/22141#discussion_r215333831 --- Diff: sql/core/src/test/resources/sql-tests/inputs/subquery/in-subquery/nested-not-in.sql --- @@ -0,0 +1,198 @@ +-- Tests NOT-IN subqueries

[GitHub] spark issue #22340: [SPARK-25337][SQL][TEST] `runSparkSubmit` should provide...

2018-09-05 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/22340 `HiveExternalCatalogVersionsSuite` downloads and executes old Spark in their directory. However, it gave the child the followings. None of them is expected here. Especially,

[GitHub] spark issue #22320: [SPARK-25313][SQL]Fix regression in FileFormatWriter out...

2018-09-05 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22320 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/95711/ Test PASSed. ---

[GitHub] spark issue #22320: [SPARK-25313][SQL]Fix regression in FileFormatWriter out...

2018-09-05 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22320 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #22328: [SPARK-22666][ML][SQL] Spark datasource for image format

2018-09-05 Thread mengxr
Github user mengxr commented on the issue: https://github.com/apache/spark/pull/22328 LGTM pending tests. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #22320: [SPARK-25313][SQL]Fix regression in FileFormatWriter out...

2018-09-05 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22320 **[Test build #95711 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95711/testReport)** for PR 22320 at commit

[GitHub] spark issue #22328: [SPARK-22666][ML][SQL] Spark datasource for image format

2018-09-05 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22328 **[Test build #95724 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95724/testReport)** for PR 22328 at commit

[GitHub] spark issue #22328: [SPARK-22666][ML][SQL] Spark datasource for image format

2018-09-05 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22328 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark issue #22328: [SPARK-22666][ML][SQL] Spark datasource for image format

2018-09-05 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22328 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #22141: [SPARK-25154][SQL] Support NOT IN sub-queries inside nes...

2018-09-05 Thread dilipbiswal
Github user dilipbiswal commented on the issue: https://github.com/apache/spark/pull/22141 @maropu I think, this will be considered post 2.4. So we have some time. I am now exploring options to plan better at physical level. ---

[GitHub] spark issue #22165: [SPARK-25017][Core] Add test suite for BarrierCoordinato...

2018-09-05 Thread mengxr
Github user mengxr commented on the issue: https://github.com/apache/spark/pull/22165 I didn't make a full pass over the tests. @jiangxb1987 let me know if you need me to take a pass. --- - To unsubscribe, e-mail:

[GitHub] spark pull request #22165: [SPARK-25017][Core] Add test suite for BarrierCoo...

2018-09-05 Thread mengxr
Github user mengxr commented on a diff in the pull request: https://github.com/apache/spark/pull/22165#discussion_r215326727 --- Diff: core/src/test/scala/org/apache/spark/scheduler/BarrierCoordinatorSuite.scala --- @@ -0,0 +1,153 @@ +/* + * Licensed to the Apache Software

[GitHub] spark pull request #22165: [SPARK-25017][Core] Add test suite for BarrierCoo...

2018-09-05 Thread mengxr
Github user mengxr commented on a diff in the pull request: https://github.com/apache/spark/pull/22165#discussion_r215326394 --- Diff: core/src/test/scala/org/apache/spark/scheduler/BarrierCoordinatorSuite.scala --- @@ -0,0 +1,153 @@ +/* + * Licensed to the Apache Software

[GitHub] spark pull request #22165: [SPARK-25017][Core] Add test suite for BarrierCoo...

2018-09-05 Thread mengxr
Github user mengxr commented on a diff in the pull request: https://github.com/apache/spark/pull/22165#discussion_r215324595 --- Diff: core/src/main/scala/org/apache/spark/BarrierCoordinator.scala --- @@ -65,7 +65,7 @@ private[spark] class BarrierCoordinator( // Record

[GitHub] spark issue #18906: [SPARK-21692][PYSPARK][SQL] Add nullability support to P...

2018-09-05 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18906 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/95712/ Test FAILed. ---

[GitHub] spark issue #18906: [SPARK-21692][PYSPARK][SQL] Add nullability support to P...

2018-09-05 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18906 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #18906: [SPARK-21692][PYSPARK][SQL] Add nullability support to P...

2018-09-05 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18906 **[Test build #95712 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95712/testReport)** for PR 18906 at commit

[GitHub] spark pull request #22328: [SPARK-22666][ML][SQL] Spark datasource for image...

2018-09-05 Thread mengxr
Github user mengxr commented on a diff in the pull request: https://github.com/apache/spark/pull/22328#discussion_r215322021 --- Diff: mllib/src/main/scala/org/apache/spark/ml/source/image/ImageDataSource.scala --- @@ -0,0 +1,54 @@ +/* + * Licensed to the Apache Software

[GitHub] spark pull request #22328: [SPARK-22666][ML][SQL] Spark datasource for image...

2018-09-05 Thread mengxr
Github user mengxr commented on a diff in the pull request: https://github.com/apache/spark/pull/22328#discussion_r215322673 --- Diff: mllib/src/main/scala/org/apache/spark/ml/source/image/ImageOptions.scala --- @@ -0,0 +1,28 @@ +/* + * Licensed to the Apache Software

[GitHub] spark pull request #22328: [SPARK-22666][ML][SQL] Spark datasource for image...

2018-09-05 Thread mengxr
Github user mengxr commented on a diff in the pull request: https://github.com/apache/spark/pull/22328#discussion_r215320762 --- Diff: mllib/src/main/scala/org/apache/spark/ml/source/image/ImageDataSource.scala --- @@ -0,0 +1,54 @@ +/* + * Licensed to the Apache Software

[GitHub] spark pull request #22328: [SPARK-22666][ML][SQL] Spark datasource for image...

2018-09-05 Thread mengxr
Github user mengxr commented on a diff in the pull request: https://github.com/apache/spark/pull/22328#discussion_r215321353 --- Diff: mllib/src/main/scala/org/apache/spark/ml/source/image/ImageDataSource.scala --- @@ -0,0 +1,54 @@ +/* + * Licensed to the Apache Software

[GitHub] spark pull request #22328: [SPARK-22666][ML][SQL] Spark datasource for image...

2018-09-05 Thread mengxr
Github user mengxr commented on a diff in the pull request: https://github.com/apache/spark/pull/22328#discussion_r215320923 --- Diff: mllib/src/main/scala/org/apache/spark/ml/source/image/ImageDataSource.scala --- @@ -0,0 +1,54 @@ +/* + * Licensed to the Apache Software

[GitHub] spark pull request #22328: [SPARK-22666][ML][SQL] Spark datasource for image...

2018-09-05 Thread mengxr
Github user mengxr commented on a diff in the pull request: https://github.com/apache/spark/pull/22328#discussion_r215323149 --- Diff: mllib/src/test/scala/org/apache/spark/ml/source/image/ImageFileFormatSuite.scala --- @@ -0,0 +1,119 @@ +/* + * Licensed to the Apache

[GitHub] spark issue #22167: [SPARK-25170][DOC] Add list and short description of Spa...

2018-09-05 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/22167 I like to add description for metrics. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands,

[GitHub] spark pull request #22341: [SPARK-24889][Core] Update block info when unpers...

2018-09-05 Thread xuanyuanking
Github user xuanyuanking commented on a diff in the pull request: https://github.com/apache/spark/pull/22341#discussion_r215318105 --- Diff: core/src/main/scala/org/apache/spark/storage/RDDInfo.scala --- @@ -55,7 +55,7 @@ class RDDInfo( } private[spark] object

[GitHub] spark pull request #22324: [SPARK-25237][SQL] Remove updateBytesReadWithFile...

2018-09-05 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/22324#discussion_r215318249 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/FileBasedDataSourceSuite.scala --- @@ -473,6 +476,27 @@ class FileBasedDataSourceSuite extends

[GitHub] spark pull request #22167: [SPARK-25170][DOC] Add list and short description...

2018-09-05 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/22167#discussion_r215315965 --- Diff: docs/monitoring.md --- @@ -388,6 +388,163 @@ value triggering garbage collection on jobs, and `spark.ui.retainedStages` that Note that the

[GitHub] spark pull request #22324: [SPARK-25237][SQL] Remove updateBytesReadWithFile...

2018-09-05 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/22324#discussion_r215315904 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/FileBasedDataSourceSuite.scala --- @@ -473,6 +476,27 @@ class FileBasedDataSourceSuite extends

[GitHub] spark pull request #22167: [SPARK-25170][DOC] Add list and short description...

2018-09-05 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/22167#discussion_r215315657 --- Diff: docs/monitoring.md --- @@ -388,6 +388,163 @@ value triggering garbage collection on jobs, and `spark.ui.retainedStages` that Note that the

[GitHub] spark pull request #22167: [SPARK-25170][DOC] Add list and short description...

2018-09-05 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/22167#discussion_r215315711 --- Diff: docs/monitoring.md --- @@ -388,6 +388,163 @@ value triggering garbage collection on jobs, and `spark.ui.retainedStages` that Note that the

[GitHub] spark pull request #22167: [SPARK-25170][DOC] Add list and short description...

2018-09-05 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/22167#discussion_r215314710 --- Diff: docs/monitoring.md --- @@ -388,6 +388,163 @@ value triggering garbage collection on jobs, and `spark.ui.retainedStages` that Note that the

[GitHub] spark pull request #22167: [SPARK-25170][DOC] Add list and short description...

2018-09-05 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/22167#discussion_r215314527 --- Diff: docs/monitoring.md --- @@ -388,6 +388,163 @@ value triggering garbage collection on jobs, and `spark.ui.retainedStages` that Note that the

[GitHub] spark pull request #22138: [SPARK-25151][SS] Apply Apache Commons Pool to Ka...

2018-09-05 Thread HeartSaVioR
Github user HeartSaVioR commented on a diff in the pull request: https://github.com/apache/spark/pull/22138#discussion_r215313888 --- Diff: external/kafka-0-10-sql/src/main/scala/org/apache/spark/sql/kafka010/InternalKafkaConsumerPool.scala --- @@ -0,0 +1,241 @@ +/* + *

[GitHub] spark pull request #22138: [SPARK-25151][SS] Apply Apache Commons Pool to Ka...

2018-09-05 Thread HeartSaVioR
Github user HeartSaVioR commented on a diff in the pull request: https://github.com/apache/spark/pull/22138#discussion_r215313215 --- Diff: external/kafka-0-10-sql/src/test/scala/org/apache/spark/sql/kafka010/FetchedPoolSuite.scala --- @@ -0,0 +1,299 @@ +/* + * Licensed

[GitHub] spark issue #22184: [SPARK-25132][SQL][DOC] Add migration doc for case-insen...

2018-09-05 Thread seancxmao
Github user seancxmao commented on the issue: https://github.com/apache/spark/pull/22184 @cloud-fan I've just sent a PR (#22343) for this. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For

[GitHub] spark issue #22341: [SPARK-24889][Core] Update block info when unpersist rdd...

2018-09-05 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22341 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/95709/ Test PASSed. ---

[GitHub] spark issue #22341: [SPARK-24889][Core] Update block info when unpersist rdd...

2018-09-05 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22341 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #22341: [SPARK-24889][Core] Update block info when unpersist rdd...

2018-09-05 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22341 **[Test build #95709 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95709/testReport)** for PR 22341 at commit

[GitHub] spark issue #22343: [SPARK-25132][SQL][FOLLOW-UP] The behavior must be consi...

2018-09-05 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22343 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

<    1   2   3   4   5   6   >