[GitHub] spark issue #22411: [SPARK-25421][SQL] Abstract an output path field in trai...

2018-09-17 Thread LantaoJin
Github user LantaoJin commented on the issue: https://github.com/apache/spark/pull/22411 Using pattern matching will face a problem. ```InsertIntoHiveDirCommand```,```CreateHiveTableAsSelectCommand``` and ```InsertIntoHiveTable``` are all in spark-hive module. SparkPlanInfo could not

[GitHub] spark issue #22447: [SPARK-25450][SQL] PushProjectThroughUnion rule uses the...

2018-09-17 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22447 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #22447: [SPARK-25450][SQL] PushProjectThroughUnion rule uses the...

2018-09-17 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22447 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/96166/ Test FAILed. ---

[GitHub] spark issue #22447: [SPARK-25450][SQL] PushProjectThroughUnion rule uses the...

2018-09-17 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22447 **[Test build #96166 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/96166/testReport)** for PR 22447 at commit

[GitHub] spark pull request #22441: [SPARK-25445][BUILD] the release script should be...

2018-09-17 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/22441#discussion_r218304237 --- Diff: dev/create-release/release-build.sh --- @@ -111,13 +111,17 @@ fi # different versions of Scala are supported. BASE_PROFILES="-Pmesos

[GitHub] spark issue #22441: [SPARK-25445][BUILD] the release script should be able t...

2018-09-17 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22441 **[Test build #96168 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/96168/testReport)** for PR 22441 at commit

[GitHub] spark issue #22441: [SPARK-25445][BUILD] the release script should be able t...

2018-09-17 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22441 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #22441: [SPARK-25445][BUILD] the release script should be...

2018-09-17 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/22441#discussion_r218304060 --- Diff: dev/create-release/release-build.sh --- @@ -183,8 +188,17 @@ if [[ "$1" == "package" ]]; then # Updated for each binary build

[GitHub] spark issue #22441: [SPARK-25445][BUILD] the release script should be able t...

2018-09-17 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22441 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request #22429: [SPARK-25440][SQL] Dumping query execution info t...

2018-09-17 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/22429#discussion_r218303899 --- Diff: core/src/main/scala/org/apache/spark/util/Utils.scala --- @@ -99,10 +99,11 @@ private[spark] object Utils extends Logging { * by

[GitHub] spark pull request #22441: [SPARK-25445][BUILD] the release script should be...

2018-09-17 Thread gengliangwang
Github user gengliangwang commented on a diff in the pull request: https://github.com/apache/spark/pull/22441#discussion_r218303693 --- Diff: dev/create-release/release-build.sh --- @@ -111,13 +111,17 @@ fi # different versions of Scala are supported.

[GitHub] spark issue #22411: [SPARK-25421][SQL] Abstract an output path field in trai...

2018-09-17 Thread LantaoJin
Github user LantaoJin commented on the issue: https://github.com/apache/spark/pull/22411 Agree that. Since this field is important to us. Could I refactor it following your advice and file a discussion in another Jira? ---

[GitHub] spark pull request #22441: [SPARK-25445][BUILD] the release script should be...

2018-09-17 Thread gengliangwang
Github user gengliangwang commented on a diff in the pull request: https://github.com/apache/spark/pull/22441#discussion_r218301766 --- Diff: dev/create-release/release-build.sh --- @@ -111,13 +111,17 @@ fi # different versions of Scala are supported.

[GitHub] spark issue #22441: [SPARK-25445][BUILD] the release script should be able t...

2018-09-17 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22441 **[Test build #96167 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/96167/testReport)** for PR 22441 at commit

[GitHub] spark issue #22402: [SPARK-25414][SS][TEST] make it clear that the numRows m...

2018-09-17 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22402 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #22402: [SPARK-25414][SS][TEST] make it clear that the numRows m...

2018-09-17 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22402 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/96163/ Test FAILed. ---

[GitHub] spark issue #22441: [SPARK-25445][BUILD] the release script should be able t...

2018-09-17 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22441 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark issue #22441: [SPARK-25445][BUILD] the release script should be able t...

2018-09-17 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22441 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #22402: [SPARK-25414][SS][TEST] make it clear that the numRows m...

2018-09-17 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22402 **[Test build #96163 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/96163/testReport)** for PR 22402 at commit

[GitHub] spark issue #22402: [SPARK-25414][SS][TEST] make it clear that the numRows m...

2018-09-17 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22402 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #22402: [SPARK-25414][SS][TEST] make it clear that the numRows m...

2018-09-17 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22402 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/96165/ Test FAILed. ---

[GitHub] spark issue #22402: [SPARK-25414][SS][TEST] make it clear that the numRows m...

2018-09-17 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22402 **[Test build #96165 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/96165/testReport)** for PR 22402 at commit

[GitHub] spark issue #17400: [SPARK-19981][SQL] Respect aliases in output partitionin...

2018-09-17 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17400 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/96164/ Test FAILed. ---

[GitHub] spark issue #17400: [SPARK-19981][SQL] Respect aliases in output partitionin...

2018-09-17 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17400 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #17400: [SPARK-19981][SQL] Respect aliases in output partitionin...

2018-09-17 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17400 **[Test build #96164 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/96164/testReport)** for PR 17400 at commit

[GitHub] spark pull request #22408: [SPARK-25417][SQL] ArrayContains function may ret...

2018-09-17 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/22408#discussion_r218295350 --- Diff: docs/sql-programming-guide.md --- @@ -1879,6 +1879,80 @@ working with timestamps in `pandas_udf`s to get the best performance, see

[GitHub] spark issue #22411: [SPARK-25421][SQL] Abstract an output path field in trai...

2018-09-17 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/22411 Since this is a new feature, we can't just merge it like #22353 without a proper design. Making the event logs as a structured, unified and reliable source for Spark metrics looks like a

[GitHub] spark issue #22411: [SPARK-25421][SQL] Abstract an output path field in trai...

2018-09-17 Thread LantaoJin
Github user LantaoJin commented on the issue: https://github.com/apache/spark/pull/22411 Most of the information we wanted could be analyzed out from event log except some metrics in Executor side which doesn't heartbeat to Driver, e.g RPC count with NameNode. Another case is #21221,

[GitHub] spark pull request #22443: [SPARK-25339][TEST] Refactor FilterPushdownBenchm...

2018-09-17 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/22443#discussion_r218293821 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/benchmark/FilterPushdownBenchmark.scala --- @@ -17,29 +17,28 @@ package

[GitHub] spark pull request #22443: [SPARK-25339][TEST] Refactor FilterPushdownBenchm...

2018-09-17 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/22443#discussion_r218293752 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/benchmark/FilterPushdownBenchmark.scala --- @@ -17,29 +17,28 @@ package

[GitHub] spark pull request #21403: [SPARK-24341][SQL] Support only IN subqueries wit...

2018-09-17 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/21403#discussion_r218293687 --- Diff: sql/core/src/test/resources/sql-tests/results/subquery/in-subquery/in-basic.sql.out --- @@ -0,0 +1,70 @@ +-- Automatically generated by

[GitHub] spark pull request #22420: [SPARK-25429][SQL]Use Set improve SparkListenerBu...

2018-09-17 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/22420#discussion_r218293704 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/ui/SQLAppStatusListener.scala --- @@ -83,7 +83,7 @@ class SQLAppStatusListener(

[GitHub] spark pull request #21403: [SPARK-24341][SQL] Support only IN subqueries wit...

2018-09-17 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/21403#discussion_r218293670 --- Diff: sql/core/src/test/resources/sql-tests/results/subquery/in-subquery/in-basic.sql.out --- @@ -0,0 +1,70 @@ +-- Automatically generated by

[GitHub] spark pull request #22444: [SPARK-25409][Core]Speed up Spark History loading...

2018-09-17 Thread jianjianjiao
Github user jianjianjiao commented on a diff in the pull request: https://github.com/apache/spark/pull/22444#discussion_r218292773 --- Diff: core/src/main/scala/org/apache/spark/deploy/history/FsHistoryProvider.scala --- @@ -465,20 +475,31 @@ private[history] class

[GitHub] spark issue #22138: [SPARK-25151][SS] Apply Apache Commons Pool to KafkaData...

2018-09-17 Thread HeartSaVioR
Github user HeartSaVioR commented on the issue: https://github.com/apache/spark/pull/22138 Now vote for Spark 2.4 is in progress. If we are not in stand-by mode for any blocker issues for Spark 2.4 RC, I'd be really happy if someone could revisit this and continue reviewing. ---

[GitHub] spark pull request #22439: [SPARK-25444][SQL] Refactor GenArrayData.genCodeT...

2018-09-17 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/22439 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #22439: [SPARK-25444][SQL] Refactor GenArrayData.genCodeToCreate...

2018-09-17 Thread ueshin
Github user ueshin commented on the issue: https://github.com/apache/spark/pull/22439 Thanks! merging to master. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #22433: [SPARK-25442][SQL][K8S] Support STS to run in k8s deploy...

2018-09-17 Thread suryag10
Github user suryag10 commented on the issue: https://github.com/apache/spark/pull/22433 > Agreed with @mridulm that the naming restriction is specific to k8s and should be handled in a k8s specific way, e.g., somewhere around

[GitHub] spark issue #22447: [SPARK-25450][SQL] PushProjectThroughUnion rule uses the...

2018-09-17 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22447 **[Test build #96166 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/96166/testReport)** for PR 22447 at commit

[GitHub] spark issue #22447: [SPARK-25450][SQL] PushProjectThroughUnion rule uses the...

2018-09-17 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22447 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark issue #22447: [SPARK-25450][SQL] PushProjectThroughUnion rule uses the...

2018-09-17 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22447 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #22447: [SPARK-25450][SQL] PushProjectThroughUnion rule u...

2018-09-17 Thread maryannxue
GitHub user maryannxue opened a pull request: https://github.com/apache/spark/pull/22447 [SPARK-25450][SQL] PushProjectThroughUnion rule uses the same exprId for project expressions in each Union child, causing mistakes in constant propagation ## What changes were proposed in this

[GitHub] spark issue #22381: [SPARK-25394][CORE] Add an application status metrics so...

2018-09-17 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22381 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/96159/ Test FAILed. ---

[GitHub] spark issue #22381: [SPARK-25394][CORE] Add an application status metrics so...

2018-09-17 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22381 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #22381: [SPARK-25394][CORE] Add an application status metrics so...

2018-09-17 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22381 **[Test build #96159 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/96159/testReport)** for PR 22381 at commit

[GitHub] spark issue #22446: [SPARK-19550][DOC][FOLLOW-UP] Update tuning.md to use JD...

2018-09-17 Thread wangyum
Github user wangyum commented on the issue: https://github.com/apache/spark/pull/22446 Yes. I can't find more references to the old JDK docs also. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org

[GitHub] spark issue #22395: [SPARK-16323][SQL] Add IntegralDivide expression

2018-09-17 Thread rxin
Github user rxin commented on the issue: https://github.com/apache/spark/pull/22395 Looks like a use case for a legacy config. On Mon, Sep 17, 2018 at 6:41 PM Wenchen Fan wrote: > To clarify, it's not following hive, but following the behavior of > previous

[GitHub] spark issue #22446: [SPARK-19550][DOC][FOLLOW-UP] Update tuning.md to use JD...

2018-09-17 Thread wangyum
Github user wangyum commented on the issue: https://github.com/apache/spark/pull/22446 Some references to Java 7, Some references to Java 6. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For

[GitHub] spark issue #22305: [WIP][SPARK-24561][SQL][Python] User-defined window aggr...

2018-09-17 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22305 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/96155/ Test PASSed. ---

[GitHub] spark issue #22305: [WIP][SPARK-24561][SQL][Python] User-defined window aggr...

2018-09-17 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22305 Build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands,

[GitHub] spark issue #22305: [WIP][SPARK-24561][SQL][Python] User-defined window aggr...

2018-09-17 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22305 **[Test build #96155 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/96155/testReport)** for PR 22305 at commit

[GitHub] spark pull request #22418: [SPARK-25427][SQL][TEST] Add BloomFilter creation...

2018-09-17 Thread dongjoon-hyun
Github user dongjoon-hyun commented on a diff in the pull request: https://github.com/apache/spark/pull/22418#discussion_r218282733 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/orc/OrcSourceSuite.scala --- @@ -50,6 +55,66 @@ abstract class OrcSuite

[GitHub] spark issue #22402: [SPARK-25414][SS][TEST] make it clear that the numRows m...

2018-09-17 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22402 **[Test build #96165 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/96165/testReport)** for PR 22402 at commit

[GitHub] spark issue #22402: [SPARK-25414][SS][TEST] make it clear that the numRows m...

2018-09-17 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22402 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #22402: [SPARK-25414][SS][TEST] make it clear that the numRows m...

2018-09-17 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22402 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request #22418: [SPARK-25427][SQL][TEST] Add BloomFilter creation...

2018-09-17 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/22418#discussion_r218281814 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/orc/OrcSourceSuite.scala --- @@ -50,6 +55,66 @@ abstract class OrcSuite

[GitHub] spark pull request #22438: [SPARK-25443][BUILD] fix issues when building doc...

2018-09-17 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/22438 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request #22418: [SPARK-25427][SQL][TEST] Add BloomFilter creation...

2018-09-17 Thread dongjoon-hyun
Github user dongjoon-hyun commented on a diff in the pull request: https://github.com/apache/spark/pull/22418#discussion_r218281581 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/orc/OrcSourceSuite.scala --- @@ -50,6 +55,66 @@ abstract class OrcSuite

[GitHub] spark issue #22402: [SPARK-25414][SS][TEST] make it clear that the numRows m...

2018-09-17 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22402 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #17400: [SPARK-19981][SQL] Respect aliases in output partitionin...

2018-09-17 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17400 **[Test build #96164 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/96164/testReport)** for PR 17400 at commit

[GitHub] spark issue #17400: [SPARK-19981][SQL] Respect aliases in output partitionin...

2018-09-17 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17400 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark issue #22402: [SPARK-25414][SS][TEST] make it clear that the numRows m...

2018-09-17 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22402 **[Test build #96163 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/96163/testReport)** for PR 22402 at commit

[GitHub] spark issue #22402: [SPARK-25414][SS][TEST] make it clear that the numRows m...

2018-09-17 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/22402 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #22402: [SPARK-25414][SS][TEST] make it clear that the numRows m...

2018-09-17 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22402 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/96160/ Test FAILed. ---

[GitHub] spark issue #17400: [SPARK-19981][SQL] Respect aliases in output partitionin...

2018-09-17 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17400 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #22402: [SPARK-25414][SS][TEST] make it clear that the numRows m...

2018-09-17 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22402 **[Test build #96160 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/96160/testReport)** for PR 22402 at commit

[GitHub] spark issue #22446: [SPARK-19550][DOC][FOLLOW-UP] Update tuning.md to use JD...

2018-09-17 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/22446 Are these all the references to Java 7 docs? Also, does the advice need a bit of updating since we require Java 8? around GC? ---

[GitHub] spark issue #22438: [SPARK-25443][BUILD] fix issues when building docs with ...

2018-09-17 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/22438 thanks, merging to master/2.4! --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands,

[GitHub] spark issue #17400: [SPARK-19981][SQL] Respect aliases in output partitionin...

2018-09-17 Thread maropu
Github user maropu commented on the issue: https://github.com/apache/spark/pull/17400 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #17400: [SPARK-19981][SQL] Respect aliases in output partitionin...

2018-09-17 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17400 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/96161/ Test FAILed. ---

[GitHub] spark issue #17400: [SPARK-19981][SQL] Respect aliases in output partitionin...

2018-09-17 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17400 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #17400: [SPARK-19981][SQL] Respect aliases in output partitionin...

2018-09-17 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17400 **[Test build #96161 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/96161/testReport)** for PR 17400 at commit

[GitHub] spark issue #22414: [SPARK-25424][SQL] Window duration and slide duration wi...

2018-09-17 Thread maropu
Github user maropu commented on the issue: https://github.com/apache/spark/pull/22414 Plz do not change the format: `Problem` -> `What changes were proposed in this pull request?`. --- - To unsubscribe, e-mail:

[GitHub] spark pull request #22414: [SPARK-25424][SQL] Window duration and slide dura...

2018-09-17 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/22414#discussion_r218280823 --- Diff: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/expressions/TimeWindowSuite.scala --- @@ -122,6 +123,51 @@ class TimeWindowSuite

[GitHub] spark pull request #22414: [SPARK-25424][SQL] Window duration and slide dura...

2018-09-17 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/22414#discussion_r218280792 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/TimeWindow.scala --- @@ -35,6 +35,10 @@ case class TimeWindow( with

[GitHub] spark issue #22446: [SPARK-19550][DOC][FOLLOW-UP] Update tuning.md to use JD...

2018-09-17 Thread wangyum
Github user wangyum commented on the issue: https://github.com/apache/spark/pull/22446 cc @srowen --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark pull request #22444: [SPARK-25409][Core]Speed up Spark History loading...

2018-09-17 Thread squito
Github user squito commented on a diff in the pull request: https://github.com/apache/spark/pull/22444#discussion_r218279175 --- Diff: core/src/main/scala/org/apache/spark/deploy/history/FsHistoryProvider.scala --- @@ -465,20 +475,31 @@ private[history] class

[GitHub] spark issue #22446: [SPARK-19550][DOC][FOLLOW-UP] Update tuning.md to use JD...

2018-09-17 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22446 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/96162/ Test PASSed. ---

[GitHub] spark issue #22446: [SPARK-19550][DOC][FOLLOW-UP] Update tuning.md to use JD...

2018-09-17 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22446 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #22446: [SPARK-19550][DOC][FOLLOW-UP] Update tuning.md to use JD...

2018-09-17 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22446 **[Test build #96162 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/96162/testReport)** for PR 22446 at commit

[GitHub] spark issue #22305: [WIP][SPARK-24561][SQL][Python] User-defined window aggr...

2018-09-17 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22305 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #22305: [WIP][SPARK-24561][SQL][Python] User-defined window aggr...

2018-09-17 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22305 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/96158/ Test FAILed. ---

[GitHub] spark issue #22305: [WIP][SPARK-24561][SQL][Python] User-defined window aggr...

2018-09-17 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22305 **[Test build #96158 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/96158/testReport)** for PR 22305 at commit

[GitHub] spark issue #22395: [SPARK-16323][SQL] Add IntegralDivide expression

2018-09-17 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/22395 To clarify, it's not following hive, but following the behavior of previous Spark versions, which is same as hive. I also think returning left operand's type is more reasonable, but we

[GitHub] spark issue #22446: [SPARK-19550][DOC][FOLLOW-UP] Update tuning.md to use JD...

2018-09-17 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22446 **[Test build #96162 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/96162/testReport)** for PR 22446 at commit

[GitHub] spark issue #22355: [SPARK-25358][SQL] MutableProjection supports fallback t...

2018-09-17 Thread maropu
Github user maropu commented on the issue: https://github.com/apache/spark/pull/22355 cc: @gatorsmile @rednaxelafx Could you check again and merge? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org

[GitHub] spark issue #22446: [SPARK-19550][DOC][FOLLOW-UP] Update tuning.md to use JD...

2018-09-17 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22446 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #22446: [SPARK-19550][DOC][FOLLOW-UP] Update tuning.md to use JD...

2018-09-17 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22446 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request #17400: [SPARK-19981][SQL] Respect aliases in output part...

2018-09-17 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/17400#discussion_r218277145 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/AliasAwareOutputPartitioning.scala --- @@ -0,0 +1,89 @@ +/* + * Licensed to the

[GitHub] spark pull request #22446: [SPARK-19550][DOC][FOLLOW-UP] Update tuning.md to...

2018-09-17 Thread wangyum
GitHub user wangyum opened a pull request: https://github.com/apache/spark/pull/22446 [SPARK-19550][DOC][FOLLOW-UP] Update tuning.md to use JDK8 ## What changes were proposed in this pull request? Update `tuning.md` and `building-spark.md` to use JDK8. ## How was

[GitHub] spark issue #22305: [WIP][SPARK-24561][SQL][Python] User-defined window aggr...

2018-09-17 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22305 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #22305: [WIP][SPARK-24561][SQL][Python] User-defined window aggr...

2018-09-17 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22305 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/96156/ Test FAILed. ---

[GitHub] spark issue #22305: [WIP][SPARK-24561][SQL][Python] User-defined window aggr...

2018-09-17 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22305 **[Test build #96156 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/96156/testReport)** for PR 22305 at commit

[GitHub] spark pull request #22418: [SPARK-25427][SQL][TEST] Add BloomFilter creation...

2018-09-17 Thread dbtsai
Github user dbtsai commented on a diff in the pull request: https://github.com/apache/spark/pull/22418#discussion_r218272427 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/orc/OrcSourceSuite.scala --- @@ -50,6 +55,66 @@ abstract class OrcSuite

[GitHub] spark pull request #22418: [SPARK-25427][SQL][TEST] Add BloomFilter creation...

2018-09-17 Thread dongjoon-hyun
Github user dongjoon-hyun commented on a diff in the pull request: https://github.com/apache/spark/pull/22418#discussion_r218271729 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/orc/OrcSourceSuite.scala --- @@ -50,6 +55,66 @@ abstract class OrcSuite

[GitHub] spark pull request #22445: Branch 2.3 udf nullability

2018-09-17 Thread ptkool
Github user ptkool closed the pull request at: https://github.com/apache/spark/pull/22445 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request #22445: Branch 2.3 udf nullability

2018-09-17 Thread ptkool
GitHub user ptkool opened a pull request: https://github.com/apache/spark/pull/22445 Branch 2.3 udf nullability ## What changes were proposed in this pull request? (Please fill in changes proposed in this fix) ## How was this patch tested? (Please explain

[GitHub] spark issue #22429: [SPARK-25440][SQL] Dumping query execution info to a fil...

2018-09-17 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22429 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/96154/ Test FAILed. ---

[GitHub] spark issue #22429: [SPARK-25440][SQL] Dumping query execution info to a fil...

2018-09-17 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22429 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #22429: [SPARK-25440][SQL] Dumping query execution info to a fil...

2018-09-17 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22429 **[Test build #96154 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/96154/testReport)** for PR 22429 at commit

  1   2   3   4   5   6   >