[GitHub] spark issue #19336: [SPARK-21947][SS] Check and report error when monotonica...

2017-09-24 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/19336 retest this please. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h.

[GitHub] spark issue #19336: [SPARK-21947][SS] Check and report error when monotonica...

2017-09-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19336 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #19336: [SPARK-21947][SS] Check and report error when monotonica...

2017-09-24 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19336 **[Test build #82142 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82142/testReport)** for PR 19336 at commit [`1a0c400`](https://github.com/apache/spark/commit/1

[GitHub] spark issue #19336: [SPARK-21947][SS] Check and report error when monotonica...

2017-09-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19336 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/82142/ Test FAILed. ---

[GitHub] spark pull request #19301: [SPARK-22084][SQL] Fix performance regression in ...

2017-09-24 Thread stanzhai
Github user stanzhai commented on a diff in the pull request: https://github.com/apache/spark/pull/19301#discussion_r140699522 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/aggregate/interfaces.scala --- @@ -72,11 +74,19 @@ object AggregateExpress

[GitHub] spark pull request #18887: [SPARK-20642][core] Store FsHistoryProvider listi...

2017-09-24 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/18887#discussion_r140691818 --- Diff: core/src/main/scala/org/apache/spark/deploy/history/FsHistoryProvider.scala --- @@ -422,208 +457,101 @@ private[history] class FsHistoryProvide

[GitHub] spark pull request #18887: [SPARK-20642][core] Store FsHistoryProvider listi...

2017-09-24 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/18887#discussion_r140689640 --- Diff: core/src/main/scala/org/apache/spark/deploy/history/FsHistoryProvider.scala --- @@ -720,75 +634,218 @@ private[history] class FsHistoryProvider

[GitHub] spark pull request #18887: [SPARK-20642][core] Store FsHistoryProvider listi...

2017-09-24 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/18887#discussion_r140689420 --- Diff: core/src/main/scala/org/apache/spark/deploy/history/FsHistoryProvider.scala --- @@ -720,75 +634,218 @@ private[history] class FsHistoryProvider

[GitHub] spark pull request #18887: [SPARK-20642][core] Store FsHistoryProvider listi...

2017-09-24 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/18887#discussion_r140689773 --- Diff: core/src/main/scala/org/apache/spark/deploy/history/FsHistoryProvider.scala --- @@ -742,53 +703,150 @@ private[history] object FsHistoryProvide

[GitHub] spark pull request #18887: [SPARK-20642][core] Store FsHistoryProvider listi...

2017-09-24 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/18887#discussion_r140691501 --- Diff: core/src/main/scala/org/apache/spark/deploy/history/FsHistoryProvider.scala --- @@ -422,208 +457,101 @@ private[history] class FsHistoryProvide

[GitHub] spark pull request #18887: [SPARK-20642][core] Store FsHistoryProvider listi...

2017-09-24 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/18887#discussion_r140692278 --- Diff: core/src/main/scala/org/apache/spark/deploy/history/FsHistoryProvider.scala --- @@ -720,75 +634,218 @@ private[history] class FsHistoryProvider

[GitHub] spark pull request #18887: [SPARK-20642][core] Store FsHistoryProvider listi...

2017-09-24 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/18887#discussion_r140689830 --- Diff: core/src/main/scala/org/apache/spark/deploy/history/FsHistoryProvider.scala --- @@ -422,208 +457,101 @@ private[history] class FsHistoryProvide

[GitHub] spark pull request #18887: [SPARK-20642][core] Store FsHistoryProvider listi...

2017-09-24 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/18887#discussion_r140699378 --- Diff: core/src/test/scala/org/apache/spark/deploy/history/FsHistoryProviderSuite.scala --- @@ -624,7 +639,9 @@ class FsHistoryProviderSuite extends

[GitHub] spark issue #19229: [SPARK-22001][ML][SQL] ImputerModel can do withColumn fo...

2017-09-24 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/19229 @gatorsmile Added the check for case sensitivity. Please take a look again. Thanks. --- - To unsubscribe, e-mail: reviews-unsubsc

[GitHub] spark issue #19229: [SPARK-22001][ML][SQL] ImputerModel can do withColumn fo...

2017-09-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19229 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/82141/ Test PASSed. ---

[GitHub] spark issue #19229: [SPARK-22001][ML][SQL] ImputerModel can do withColumn fo...

2017-09-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19229 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #19229: [SPARK-22001][ML][SQL] ImputerModel can do withColumn fo...

2017-09-24 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19229 **[Test build #82141 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82141/testReport)** for PR 19229 at commit [`07dec0f`](https://github.com/apache/spark/commit/0

[GitHub] spark pull request #19229: [SPARK-22001][ML][SQL] ImputerModel can do withCo...

2017-09-24 Thread WeichenXu123
Github user WeichenXu123 commented on a diff in the pull request: https://github.com/apache/spark/pull/19229#discussion_r140690182 --- Diff: mllib/src/main/scala/org/apache/spark/ml/feature/Imputer.scala --- @@ -223,20 +223,18 @@ class ImputerModel private[ml] ( overrid

[GitHub] spark pull request #19229: [SPARK-22001][ML][SQL] ImputerModel can do withCo...

2017-09-24 Thread WeichenXu123
Github user WeichenXu123 commented on a diff in the pull request: https://github.com/apache/spark/pull/19229#discussion_r140689908 --- Diff: mllib/src/main/scala/org/apache/spark/ml/feature/Imputer.scala --- @@ -223,20 +223,18 @@ class ImputerModel private[ml] ( overrid

[GitHub] spark issue #19336: [SPARK-21947][SS] Check and report error when monotonica...

2017-09-24 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19336 **[Test build #82142 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82142/testReport)** for PR 19336 at commit [`1a0c400`](https://github.com/apache/spark/commit/1a

[GitHub] spark pull request #19336: [SPARK-21947][SS] Check and report error when mon...

2017-09-24 Thread viirya
GitHub user viirya opened a pull request: https://github.com/apache/spark/pull/19336 [SPARK-21947][SS] Check and report error when monotonically_increasing_id is used in streaming query ## What changes were proposed in this pull request? `monotonically_increasing_id` doesn'

[GitHub] spark issue #19321: [SPARK-22100] [SQL] Make percentile_approx support numer...

2017-09-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19321 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/82140/ Test PASSed. ---

[GitHub] spark issue #19321: [SPARK-22100] [SQL] Make percentile_approx support numer...

2017-09-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19321 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #19321: [SPARK-22100] [SQL] Make percentile_approx support numer...

2017-09-24 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19321 **[Test build #82140 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82140/testReport)** for PR 19321 at commit [`1d26f50`](https://github.com/apache/spark/commit/1

[GitHub] spark issue #18029: [SPARK-20168] [DStream] Add changes to use kinesis fetch...

2017-09-24 Thread yssharma
Github user yssharma commented on the issue: https://github.com/apache/spark/pull/18029 @brkyvz Could you please check this for the last suggestions ? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.or

[GitHub] spark issue #19335: mapPartitions Api

2017-09-24 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/19335 ping @listenLearning! --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: rev

[GitHub] spark issue #19321: [SPARK-22100] [SQL] Make percentile_approx support numer...

2017-09-24 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/19321 Please update the PR title and description. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #19321: [SPARK-22100] [SQL] Make percentile_approx suppor...

2017-09-24 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/19321#discussion_r140684015 --- Diff: docs/sql-programming-guide.md --- @@ -1553,6 +1553,7 @@ options. ## Upgrading From Spark SQL 2.2 to 2.3 - Since Spark 2.3, the

[GitHub] spark issue #19335: mapPartitions Api

2017-09-24 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/19335 @listenLearning, If you'd like to ask a question, please ask this to the mailing list (see https://spark.apache.org/community.html). --- --

[GitHub] spark issue #19335: mapPartitions Api

2017-09-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19335 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19335: mapPartitions Api

2017-09-24 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/19335 @listenLearning Close this please. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands

[GitHub] spark pull request #19335: mapPartitions Api

2017-09-24 Thread listenLearning
GitHub user listenLearning opened a pull request: https://github.com/apache/spark/pull/19335 mapPartitions Api 您好,最近我在开发的时候遇到一个问题,就是如果我用mappartitions这个api去存储数据到hbase,会出现一个找不到partition的错误,然å

[GitHub] spark issue #19334: Branch 1.6

2017-09-24 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/19334 @listenLearning, looks mistakenly open. Can you close this please? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.ap

[GitHub] spark pull request #19334: Branch 1.6

2017-09-24 Thread listenLearning
GitHub user listenLearning opened a pull request: https://github.com/apache/spark/pull/19334 Branch 1.6 ## What changes were proposed in this pull request? (Please fill in changes proposed in this fix) ## How was this patch tested? (Please explain how this

[GitHub] spark issue #19334: Branch 1.6

2017-09-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19334 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19229: [SPARK-22001][ML][SQL] ImputerModel can do withColumn fo...

2017-09-24 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19229 **[Test build #82141 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82141/testReport)** for PR 19229 at commit [`07dec0f`](https://github.com/apache/spark/commit/07

[GitHub] spark pull request #19229: [SPARK-22001][ML][SQL] ImputerModel can do withCo...

2017-09-24 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/19229#discussion_r140680794 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/Dataset.scala --- @@ -2102,6 +2102,55 @@ class Dataset[T] private[sql]( } /** +

[GitHub] spark issue #19317: [SPARK-22098][CORE] Add new method aggregateByKeyLocally...

2017-09-24 Thread ConeyLiu
Github user ConeyLiu commented on the issue: https://github.com/apache/spark/pull/19317 Test case: ```scala test("performance of aggregateByKeyLocally ") { val random = new Random(1) val pairs = sc.parallelize(0 until 1000, 20) .map(p => (rando

[GitHub] spark pull request #19229: [SPARK-22001][ML][SQL] ImputerModel can do withCo...

2017-09-24 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/19229#discussion_r140680654 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/Dataset.scala --- @@ -2102,6 +2102,55 @@ class Dataset[T] private[sql]( } /** +

[GitHub] spark pull request #19229: [SPARK-22001][ML][SQL] ImputerModel can do withCo...

2017-09-24 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/19229#discussion_r140680629 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/Dataset.scala --- @@ -2102,6 +2102,55 @@ class Dataset[T] private[sql]( } /**

[GitHub] spark pull request #19229: [SPARK-22001][ML][SQL] ImputerModel can do withCo...

2017-09-24 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/19229#discussion_r140680435 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/Dataset.scala --- @@ -2102,6 +2102,55 @@ class Dataset[T] private[sql]( } /**

[GitHub] spark issue #19290: [SPARK-22063][R] Upgrades lintr to latest commit sha1 ID

2017-09-24 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/19290 For more context, I believe `lintr` was initially installed in https://github.com/apache/spark/commit/004f57374b98c4df32d9f1e19221f68e92639a49. Upgrade to jimhester/lintr@a769c0b was proposed he

[GitHub] spark pull request #19229: [SPARK-22001][ML][SQL] ImputerModel can do withCo...

2017-09-24 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/19229#discussion_r140678630 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/Dataset.scala --- @@ -2102,6 +2102,55 @@ class Dataset[T] private[sql]( } /** +

[GitHub] spark pull request #19229: [SPARK-22001][ML][SQL] ImputerModel can do withCo...

2017-09-24 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/19229#discussion_r140678574 --- Diff: mllib/src/main/scala/org/apache/spark/ml/feature/Imputer.scala --- @@ -223,20 +223,18 @@ class ImputerModel private[ml] ( override def

[GitHub] spark issue #19321: [SPARK-22100] [SQL] Make percentile_approx support numer...

2017-09-24 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19321 **[Test build #82140 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82140/testReport)** for PR 19321 at commit [`1d26f50`](https://github.com/apache/spark/commit/1d

[GitHub] spark pull request #19229: [SPARK-22001][ML][SQL] ImputerModel can do withCo...

2017-09-24 Thread WeichenXu123
Github user WeichenXu123 commented on a diff in the pull request: https://github.com/apache/spark/pull/19229#discussion_r140675967 --- Diff: mllib/src/main/scala/org/apache/spark/ml/feature/Imputer.scala --- @@ -223,20 +223,18 @@ class ImputerModel private[ml] ( overrid

[GitHub] spark issue #19290: [SPARK-22063][R] Upgrades lintr to latest commit sha1 ID

2017-09-24 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/19290 Do you maybe have some worries about this? If that worry is quite crucial, I think we could also consider an option, not upgrading this, leaving `lint-r.R` script as was, and only fixing the ins

[GitHub] spark issue #19290: [SPARK-22063][R] Upgrades lintr to latest commit sha1 ID

2017-09-24 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/19290 @shaneknapp Sure, it was my bad. I will be careful next time. It is required to fix an actual issue in order to to detect R codes that do not follow project's R style. --- ---

[GitHub] spark issue #19290: [SPARK-22063][R] Upgrades lintr to latest commit sha1 ID

2017-09-24 Thread shaneknapp
Github user shaneknapp commented on the issue: https://github.com/apache/spark/pull/19290 @HyukjinKwon -- you will absolutely not have builds install packages on the build system. this is a really bad idea. is this absolutely required, or just to fix a warning in the build ou

[GitHub] spark pull request #19283: Update quickstart python dataset example

2017-09-24 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/19283 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #19326: [SPARK-22107] Change as to alias in python quickstart

2017-09-24 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/19326 Merged to master and branch-2.2. @jgoleary, I merged this considering the first contribution but let's do this in a batch if possible in the future. --- --

[GitHub] spark pull request #19326: [SPARK-22107] Change as to alias in python quicks...

2017-09-24 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/19326 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #16548: [SPARK-19158][SPARKR][EXAMPLES] Fix ml.R example fails d...

2017-09-24 Thread holdenk
Github user holdenk commented on the issue: https://github.com/apache/spark/pull/16548 So there is something similar in the fulltests for R `./R/pkg/tests/fulltests/test_mllib.R` (found while working on packaging). ---

[GitHub] spark issue #19222: [SPARK-10399][CORE][SQL] Introduce multiple MemoryBlocks...

2017-09-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19222 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #19222: [SPARK-10399][CORE][SQL] Introduce multiple MemoryBlocks...

2017-09-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19222 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/82138/ Test PASSed. ---

[GitHub] spark issue #19222: [SPARK-10399][CORE][SQL] Introduce multiple MemoryBlocks...

2017-09-24 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19222 **[Test build #82138 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82138/testReport)** for PR 19222 at commit [`0714ddc`](https://github.com/apache/spark/commit/0

[GitHub] spark issue #19326: [SPARK-22107] Change as to alias in python quickstart

2017-09-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19326 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/82139/ Test PASSed. ---

[GitHub] spark issue #19326: [SPARK-22107] Change as to alias in python quickstart

2017-09-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19326 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #19326: [SPARK-22107] Change as to alias in python quickstart

2017-09-24 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19326 **[Test build #82139 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82139/testReport)** for PR 19326 at commit [`c6bf156`](https://github.com/apache/spark/commit/c

[GitHub] spark issue #19326: [SPARK-22107] Change as to alias in python quickstart

2017-09-24 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19326 **[Test build #82139 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82139/testReport)** for PR 19326 at commit [`c6bf156`](https://github.com/apache/spark/commit/c6

[GitHub] spark issue #19326: [SPARK-22107] Change as to alias in python quickstart

2017-09-24 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/19326 ok to test --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@s

[GitHub] spark issue #19294: [SPARK-21549][CORE] Respect OutputFormats with no output...

2017-09-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19294 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #19294: [SPARK-21549][CORE] Respect OutputFormats with no output...

2017-09-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19294 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/82137/ Test PASSed. ---

[GitHub] spark issue #19294: [SPARK-21549][CORE] Respect OutputFormats with no output...

2017-09-24 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19294 **[Test build #82137 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82137/testReport)** for PR 19294 at commit [`11445b4`](https://github.com/apache/spark/commit/1

[GitHub] spark issue #19324: [SPARK-22103] Move HashAggregateExec parent consume to a...

2017-09-24 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/19324 LGTM --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.ap

[GitHub] spark pull request #19324: [SPARK-22103] Move HashAggregateExec parent consu...

2017-09-24 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/19324#discussion_r140664581 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/joins/BroadcastHashJoinExec.scala --- @@ -186,8 +186,7 @@ case class BroadcastHashJoinE

[GitHub] spark pull request #19324: [SPARK-22103] Move HashAggregateExec parent consu...

2017-09-24 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/19324#discussion_r140664550 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/joins/BroadcastHashJoinExec.scala --- @@ -328,10 +325,11 @@ case class BroadcastHashJoi

[GitHub] spark pull request #19324: [SPARK-22103] Move HashAggregateExec parent consu...

2017-09-24 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/19324#discussion_r140664499 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/joins/BroadcastHashJoinExec.scala --- @@ -328,10 +325,11 @@ case class BroadcastHashJoi

[GitHub] spark issue #19222: [SPARK-10399][CORE][SQL] Introduce multiple MemoryBlocks...

2017-09-24 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/19222 @hvanhovell @rednaxelafx After running a benchmark program, I took a polymorphic approach (i.e. each subclass has `getInt()`/`putInt()` methods. Then, I got better performance than monomorphic app

[GitHub] spark issue #19222: [SPARK-10399][CORE][SQL] Introduce multiple MemoryBlocks...

2017-09-24 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19222 **[Test build #82138 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82138/testReport)** for PR 19222 at commit [`0714ddc`](https://github.com/apache/spark/commit/07

[GitHub] spark issue #19326: [SPARK-22107] Change as to alias in python quickstart

2017-09-24 Thread jgoleary
Github user jgoleary commented on the issue: https://github.com/apache/spark/pull/19326 Updated description. The only other mentions of `as()` I can find in the docs are in Java examples, and the method appears to exist on the Java side. --- -

[GitHub] spark issue #19294: [SPARK-21549][CORE] Respect OutputFormats with no output...

2017-09-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19294 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #19294: [SPARK-21549][CORE] Respect OutputFormats with no output...

2017-09-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19294 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/82136/ Test FAILed. ---

[GitHub] spark issue #19294: [SPARK-21549][CORE] Respect OutputFormats with no output...

2017-09-24 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19294 **[Test build #82136 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82136/testReport)** for PR 19294 at commit [`7c58abb`](https://github.com/apache/spark/commit/7

[GitHub] spark issue #19294: [SPARK-21549][CORE] Respect OutputFormats with no output...

2017-09-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19294 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/82135/ Test FAILed. ---

[GitHub] spark issue #19222: [SPARK-10399][CORE][SQL] Introduce multiple MemoryBlocks...

2017-09-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19222 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/82132/ Test PASSed. ---

[GitHub] spark issue #19222: [SPARK-10399][CORE][SQL] Introduce multiple MemoryBlocks...

2017-09-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19222 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #19294: [SPARK-21549][CORE] Respect OutputFormats with no output...

2017-09-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19294 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #19222: [SPARK-10399][CORE][SQL] Introduce multiple MemoryBlocks...

2017-09-24 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19222 **[Test build #82132 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82132/testReport)** for PR 19222 at commit [`7ec26f6`](https://github.com/apache/spark/commit/7

[GitHub] spark issue #19294: [SPARK-21549][CORE] Respect OutputFormats with no output...

2017-09-24 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19294 **[Test build #82135 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82135/testReport)** for PR 19294 at commit [`0cf4724`](https://github.com/apache/spark/commit/0

[GitHub] spark issue #19294: [SPARK-21549][CORE] Respect OutputFormats with no output...

2017-09-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19294 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #19294: [SPARK-21549][CORE] Respect OutputFormats with no output...

2017-09-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19294 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/82134/ Test FAILed. ---

[GitHub] spark issue #19294: [SPARK-21549][CORE] Respect OutputFormats with no output...

2017-09-24 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19294 **[Test build #82134 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82134/testReport)** for PR 19294 at commit [`1e42e83`](https://github.com/apache/spark/commit/1

[GitHub] spark issue #19294: [SPARK-21549][CORE] Respect OutputFormats with no output...

2017-09-24 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19294 **[Test build #82137 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82137/testReport)** for PR 19294 at commit [`11445b4`](https://github.com/apache/spark/commit/11

[GitHub] spark issue #19294: [SPARK-21549][CORE] Respect OutputFormats with no output...

2017-09-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19294 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/82133/ Test FAILed. ---

[GitHub] spark issue #19294: [SPARK-21549][CORE] Respect OutputFormats with no output...

2017-09-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19294 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #19294: [SPARK-21549][CORE] Respect OutputFormats with no output...

2017-09-24 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19294 **[Test build #82133 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82133/testReport)** for PR 19294 at commit [`7963b58`](https://github.com/apache/spark/commit/7

[GitHub] spark issue #19294: [SPARK-21549][CORE] Respect OutputFormats with no output...

2017-09-24 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19294 **[Test build #82136 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82136/testReport)** for PR 19294 at commit [`7c58abb`](https://github.com/apache/spark/commit/7c

[GitHub] spark pull request #19294: [SPARK-21549][CORE] Respect OutputFormats with no...

2017-09-24 Thread steveloughran
Github user steveloughran commented on a diff in the pull request: https://github.com/apache/spark/pull/19294#discussion_r140658582 --- Diff: core/src/main/scala/org/apache/spark/internal/io/HadoopMapReduceCommitProtocol.scala --- @@ -130,17 +135,21 @@ class HadoopMapReduceCommitP

[GitHub] spark issue #19294: [SPARK-21549][CORE] Respect OutputFormats with no output...

2017-09-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19294 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #19294: [SPARK-21549][CORE] Respect OutputFormats with no output...

2017-09-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19294 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/82130/ Test PASSed. ---

[GitHub] spark issue #19294: [SPARK-21549][CORE] Respect OutputFormats with no output...

2017-09-24 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19294 **[Test build #82130 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82130/testReport)** for PR 19294 at commit [`ae0ba0a`](https://github.com/apache/spark/commit/a

[GitHub] spark issue #19294: [SPARK-21549][CORE] Respect OutputFormats with no output...

2017-09-24 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19294 **[Test build #82135 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82135/testReport)** for PR 19294 at commit [`0cf4724`](https://github.com/apache/spark/commit/0c

[GitHub] spark issue #19229: [SPARK-22001][ML][SQL] ImputerModel can do withColumn fo...

2017-09-24 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/19229 @WeichenXu123 Have any more comments on this? Thanks. I think the ML part is straightforward. --- - To unsubscribe, e-mail: revie

[GitHub] spark issue #19294: [SPARK-21549][CORE] Respect OutputFormats with no output...

2017-09-24 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19294 **[Test build #82134 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82134/testReport)** for PR 19294 at commit [`1e42e83`](https://github.com/apache/spark/commit/1e

[GitHub] spark issue #19294: [SPARK-21549][CORE] Respect OutputFormats with no output...

2017-09-24 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19294 **[Test build #82133 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82133/testReport)** for PR 19294 at commit [`7963b58`](https://github.com/apache/spark/commit/79

[GitHub] spark issue #19317: [SPARK-22098][CORE] Add new method aggregateByKeyLocally...

2017-09-24 Thread WeichenXu123
Github user WeichenXu123 commented on the issue: https://github.com/apache/spark/pull/19317 It is better adding more perf test for `OpenHashSet` replacement to avoid perf regression. And I found `reduceByKeyLocally` also use `JHashSet`, I am not sure whether there is some special reas

[GitHub] spark issue #19222: [SPARK-10399][CORE][SQL] Introduce multiple MemoryBlocks...

2017-09-24 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19222 **[Test build #82132 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82132/testReport)** for PR 19222 at commit [`7ec26f6`](https://github.com/apache/spark/commit/7e

[GitHub] spark issue #19317: [SPARK-22098][CORE] Add new method aggregateByKeyLocally...

2017-09-24 Thread ConeyLiu
Github user ConeyLiu commented on the issue: https://github.com/apache/spark/pull/19317 OK, just keep it. Does this need more test or more improvements ? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache

[GitHub] spark issue #19294: [SPARK-21549][CORE] Respect OutputFormats with no output...

2017-09-24 Thread szhem
Github user szhem commented on the issue: https://github.com/apache/spark/pull/19294 @mridulm Updated `FileFormatWriterSuite` [to cover](https://github.com/apache/spark/pull/19294/files#diff-bc98a3d91cf4f95f4f473146400044aa) both branches of the [committer calling](https://github.com

  1   2   >