[GitHub] spark pull request #20366: [SPARK-23166] [ML] Add maxDF Parameter to CountVe...

2018-01-23 Thread ymazari
GitHub user ymazari opened a pull request: https://github.com/apache/spark/pull/20366 [SPARK-23166] [ML] Add maxDF Parameter to CountVectorizer ## What changes were proposed in this pull request? (Please fill in changes proposed in this fix) ## How was this patch

[GitHub] spark pull request #19671: [SPARK-22297][CORE TESTS] Flaky test: BlockManage...

2018-01-23 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/19671#discussion_r163323152 --- Diff: core/src/test/scala/org/apache/spark/storage/BlockManagerSuite.scala --- @@ -1322,33 +1322,55 @@ class BlockManagerSuite extends SparkFunSuite

[GitHub] spark pull request #17185: [SPARK-19602][SQL] Support column resolution of f...

2018-01-23 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/17185#discussion_r163322810 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/unresolved.scala --- @@ -88,12 +88,12 @@ case class

[GitHub] spark issue #18931: [SPARK-21717][SQL] Decouple consume functions of physica...

2018-01-23 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/18931 Also, can this idea be applied to `produce` too? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For

[GitHub] spark issue #18931: [SPARK-21717][SQL] Decouple consume functions of physica...

2018-01-23 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/18931 This fixes the regression of q66 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands,

[GitHub] spark issue #20321: [SPARK-23152][ML] - Correctly guard against empty datase...

2018-01-23 Thread tovbinm
Github user tovbinm commented on the issue: https://github.com/apache/spark/pull/20321 anyone? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #18931: [SPARK-21717][SQL] Decouple consume functions of physica...

2018-01-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18931 **[Test build #86538 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86538/testReport)** for PR 18931 at commit

[GitHub] spark issue #18931: [SPARK-21717][SQL] Decouple consume functions of physica...

2018-01-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18931 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/147/

[GitHub] spark issue #18931: [SPARK-21717][SQL] Decouple consume functions of physica...

2018-01-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18931 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #18931: [SPARK-21717][SQL] Decouple consume functions of physica...

2018-01-23 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/18931 Does this fix the performance regression of q66? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For

[GitHub] spark issue #18931: [SPARK-21717][SQL] Decouple consume functions of physica...

2018-01-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18931 **[Test build #86537 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86537/testReport)** for PR 18931 at commit

[GitHub] spark pull request #18931: [SPARK-21717][SQL] Decouple consume functions of ...

2018-01-23 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/18931#discussion_r163317809 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/WholeStageCodegenExec.scala --- @@ -149,13 +149,100 @@ trait CodegenSupport extends

[GitHub] spark issue #18931: [SPARK-21717][SQL] Decouple consume functions of physica...

2018-01-23 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/18931 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark pull request #18931: [SPARK-21717][SQL] Decouple consume functions of ...

2018-01-23 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/18931#discussion_r163316730 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/WholeStageCodegenExec.scala --- @@ -149,13 +149,100 @@ trait CodegenSupport extends

[GitHub] spark pull request #18931: [SPARK-21717][SQL] Decouple consume functions of ...

2018-01-23 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/18931#discussion_r163316509 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/WholeStageCodegenExec.scala --- @@ -149,13 +149,100 @@ trait CodegenSupport extends

[GitHub] spark pull request #18931: [SPARK-21717][SQL] Decouple consume functions of ...

2018-01-23 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/18931#discussion_r163315858 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/WholeStageCodegenExec.scala --- @@ -149,13 +149,100 @@ trait CodegenSupport extends

[GitHub] spark pull request #18931: [SPARK-21717][SQL] Decouple consume functions of ...

2018-01-23 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/18931#discussion_r163315164 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/WholeStageCodegenExec.scala --- @@ -149,13 +149,100 @@ trait CodegenSupport extends

[GitHub] spark issue #19082: [SPARK-21870][SQL] Split aggregation code into small fun...

2018-01-23 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/19082 Sorry for missing the excellent discussions here. After more thoughts, I feel this patch still makes sense. Currently we have 3 cases that may cause large method with whole stage codegen:

[GitHub] spark issue #20365: [SPARK-23192] [SQL] Keep the Hint after Using Cached Dat...

2018-01-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20365 **[Test build #86536 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86536/testReport)** for PR 20365 at commit

[GitHub] spark issue #20365: [SPARK-23192] [SQL] Keep the Hint after Using Cached Dat...

2018-01-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20365 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/146/

[GitHub] spark issue #20365: [SPARK-23192] [SQL] Keep the Hint after Using Cached Dat...

2018-01-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20365 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #20365: [SPARK-23192] [SQL] Keep the Hint after Using Cac...

2018-01-23 Thread gatorsmile
GitHub user gatorsmile opened a pull request: https://github.com/apache/spark/pull/20365 [SPARK-23192] [SQL] Keep the Hint after Using Cached Data ## What changes were proposed in this pull request? The hint of the plan segment is lost, if the plan segment is replaced by

[GitHub] spark pull request #20359: [SPARK-23186][SQL] Initialize DriverManager first...

2018-01-23 Thread dongjoon-hyun
Github user dongjoon-hyun commented on a diff in the pull request: https://github.com/apache/spark/pull/20359#discussion_r163305903 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/jdbc/DriverRegistry.scala --- @@ -32,6 +32,9 @@ import

[GitHub] spark issue #18906: [SPARK-21692][PYSPARK][SQL] Add nullability support to P...

2018-01-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18906 **[Test build #86535 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86535/testReport)** for PR 18906 at commit

[GitHub] spark pull request #19082: [SPARK-21870][SQL] Split aggregation code into sm...

2018-01-23 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/19082#discussion_r163305111 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/aggregate/HashAggregateExec.scala --- @@ -825,52 +924,92 @@ case class

[GitHub] spark issue #20359: [SPARK-23186][SQL] Initialize DriverManager first before...

2018-01-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20359 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/86530/ Test PASSed. ---

[GitHub] spark issue #20359: [SPARK-23186][SQL] Initialize DriverManager first before...

2018-01-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20359 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20359: [SPARK-23186][SQL] Initialize DriverManager first before...

2018-01-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20359 **[Test build #86530 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86530/testReport)** for PR 20359 at commit

[GitHub] spark issue #20350: [SPARK-23179][SQL] Support option to throw exception if ...

2018-01-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20350 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/86528/ Test PASSed. ---

[GitHub] spark issue #20350: [SPARK-23179][SQL] Support option to throw exception if ...

2018-01-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20350 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20350: [SPARK-23179][SQL] Support option to throw exception if ...

2018-01-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20350 **[Test build #86528 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86528/testReport)** for PR 20350 at commit

[GitHub] spark pull request #19671: [SPARK-22297][CORE TESTS] Flaky test: BlockManage...

2018-01-23 Thread mpetruska
Github user mpetruska commented on a diff in the pull request: https://github.com/apache/spark/pull/19671#discussion_r163298390 --- Diff: core/src/test/scala/org/apache/spark/storage/BlockManagerSuite.scala --- @@ -1322,33 +1322,55 @@ class BlockManagerSuite extends SparkFunSuite

[GitHub] spark pull request #19671: [SPARK-22297][CORE TESTS] Flaky test: BlockManage...

2018-01-23 Thread mpetruska
Github user mpetruska commented on a diff in the pull request: https://github.com/apache/spark/pull/19671#discussion_r163297550 --- Diff: core/src/test/scala/org/apache/spark/storage/BlockManagerSuite.scala --- @@ -1322,33 +1322,55 @@ class BlockManagerSuite extends SparkFunSuite

[GitHub] spark pull request #20364: [SPARK-23166][ML] Add maxDF to CountVectorizer

2018-01-23 Thread mgaido91
Github user mgaido91 closed the pull request at: https://github.com/apache/spark/pull/20364 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #20364: [SPARK-23166][ML] Add maxDF to CountVectorizer

2018-01-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20364 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/86534/ Test FAILed. ---

[GitHub] spark issue #20364: [SPARK-23166][ML] Add maxDF to CountVectorizer

2018-01-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20364 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20364: [SPARK-23166][ML] Add maxDF to CountVectorizer

2018-01-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20364 **[Test build #86534 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86534/testReport)** for PR 20364 at commit

[GitHub] spark issue #20364: [SPARK-23166][ML] Add maxDF to CountVectorizer

2018-01-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20364 **[Test build #86534 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86534/testReport)** for PR 20364 at commit

[GitHub] spark issue #20364: [SPARK-23166][ML] Add maxDF to CountVectorizer

2018-01-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20364 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/145/

[GitHub] spark issue #20364: [SPARK-23166][ML] Add maxDF to CountVectorizer

2018-01-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20364 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #20364: [SPARK-23166][ML] Add maxDF to CountVectorizer

2018-01-23 Thread mgaido91
GitHub user mgaido91 opened a pull request: https://github.com/apache/spark/pull/20364 [SPARK-23166][ML] Add maxDF to CountVectorizer ## What changes were proposed in this pull request? It might be useful to also have a maxDF parameter in CountVectorizer. It might be

[GitHub] spark issue #18906: [SPARK-21692][PYSPARK][SQL] Add nullability support to P...

2018-01-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18906 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/86526/ Test PASSed. ---

[GitHub] spark issue #18906: [SPARK-21692][PYSPARK][SQL] Add nullability support to P...

2018-01-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18906 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #18906: [SPARK-21692][PYSPARK][SQL] Add nullability support to P...

2018-01-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18906 **[Test build #86526 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86526/testReport)** for PR 18906 at commit

[GitHub] spark issue #20350: [SPARK-23179][SQL] Support option to throw exception if ...

2018-01-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20350 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/144/

[GitHub] spark issue #20350: [SPARK-23179][SQL] Support option to throw exception if ...

2018-01-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20350 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20350: [SPARK-23179][SQL] Support option to throw exception if ...

2018-01-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20350 **[Test build #86533 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86533/testReport)** for PR 20350 at commit

[GitHub] spark issue #19872: [SPARK-22274][PYTHON][SQL] User-defined aggregation func...

2018-01-23 Thread icexelloss
Github user icexelloss commented on the issue: https://github.com/apache/spark/pull/19872 Thanks all for review! --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark pull request #20350: [SPARK-23179][SQL] Support option to throw except...

2018-01-23 Thread bersprockets
Github user bersprockets commented on a diff in the pull request: https://github.com/apache/spark/pull/20350#discussion_r163269198 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala --- @@ -1074,6 +1074,16 @@ object SQLConf { .booleanConf

[GitHub] spark issue #20352: [SPARK-21727][R] Allow multi-element atomic vector as co...

2018-01-23 Thread felixcheung
Github user felixcheung commented on the issue: https://github.com/apache/spark/pull/20352 That looks good. Thanks! We work together on this fix so I’m pretty confident about this change. On a side note assuming we push this to 2.3 should we add the behavior change to the

[GitHub] spark issue #13599: [SPARK-13587] [PYSPARK] Support virtualenv in pyspark

2018-01-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13599 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/86532/ Test FAILed. ---

[GitHub] spark issue #13599: [SPARK-13587] [PYSPARK] Support virtualenv in pyspark

2018-01-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13599 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #13599: [SPARK-13587] [PYSPARK] Support virtualenv in pyspark

2018-01-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13599 **[Test build #86532 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86532/testReport)** for PR 13599 at commit

[GitHub] spark issue #18931: [SPARK-21717][SQL] Decouple consume functions of physica...

2018-01-23 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/18931 retest this please. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #13599: [SPARK-13587] [PYSPARK] Support virtualenv in pyspark

2018-01-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13599 **[Test build #86532 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86532/testReport)** for PR 13599 at commit

[GitHub] spark issue #13599: [SPARK-13587] [PYSPARK] Support virtualenv in pyspark

2018-01-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13599 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #13599: [SPARK-13587] [PYSPARK] Support virtualenv in pyspark

2018-01-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13599 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/143/

[GitHub] spark issue #20363: [SPARK-23112][WIP][DOC] Add highlights and migration gui...

2018-01-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20363 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/86531/ Test PASSed. ---

[GitHub] spark issue #20363: [SPARK-23112][WIP][DOC] Add highlights and migration gui...

2018-01-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20363 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20363: [SPARK-23112][WIP][DOC] Add highlights and migration gui...

2018-01-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20363 **[Test build #86531 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86531/testReport)** for PR 20363 at commit

[GitHub] spark issue #20347: [SPARK-20129][Core] JavaSparkContext should use SparkCon...

2018-01-23 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/20347 Yes, you can already get the new semantics here with `new JavaSparkContext(SparkContext.getOrCreate())`. Yes, probably better to add a new method, or else, decide that it's not worth a new

[GitHub] spark issue #20338: [SPARK-23174][BUILD][PYTHON] python code style checker u...

2018-01-23 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/20338 (If this also resolves SPARK-11222 you can also include that in the title.) --- - To unsubscribe, e-mail:

[GitHub] spark issue #20363: [SPARK-23112][WIP][DOC] Add highlights and migration gui...

2018-01-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20363 **[Test build #86531 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86531/testReport)** for PR 20363 at commit

[GitHub] spark issue #20363: [SPARK-23112][WIP][DOC] Add highlights and migration gui...

2018-01-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20363 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20363: [SPARK-23112][WIP][DOC] Add highlights and migration gui...

2018-01-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20363 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/142/

[GitHub] spark pull request #20363: [SPARK-23112][WIP][DOC] Add highlights and migrat...

2018-01-23 Thread MLnick
GitHub user MLnick opened a pull request: https://github.com/apache/spark/pull/20363 [SPARK-23112][WIP][DOC] Add highlights and migration guide for 2.3 Update ML user guide with highlights and migration guide for `2.3`. ## How was this patch tested? Doc only. You

[GitHub] spark issue #18143: [SPARK-20919][SS] Simplificaiton of CachedKafkaConsumer ...

2018-01-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18143 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #18143: [SPARK-20919][SS] Simplificaiton of CachedKafkaConsumer ...

2018-01-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18143 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/86529/ Test PASSed. ---

[GitHub] spark issue #18143: [SPARK-20919][SS] Simplificaiton of CachedKafkaConsumer ...

2018-01-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18143 **[Test build #86529 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86529/testReport)** for PR 18143 at commit

[GitHub] spark pull request #20338: [SPARK-23174][BUILD][PYTHON] python code style ch...

2018-01-23 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/20338#discussion_r163243073 --- Diff: dev/lint-python --- @@ -35,11 +35,10 @@ compile_status="${PIPESTATUS[0]}" # Get pep8 at runtime so that we don't rely on it being

[GitHub] spark pull request #20235: [Spark-22887][ML][TESTS][WIP] ML test for Structu...

2018-01-23 Thread gaborgsomogyi
Github user gaborgsomogyi commented on a diff in the pull request: https://github.com/apache/spark/pull/20235#discussion_r163241569 --- Diff: mllib/src/test/scala/org/apache/spark/ml/fpm/FPGrowthSuite.scala --- @@ -34,86 +35,122 @@ class FPGrowthSuite extends SparkFunSuite with

[GitHub] spark issue #20359: [SPARK-23186][SQL] Initialize DriverManager first before...

2018-01-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20359 **[Test build #86530 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86530/testReport)** for PR 20359 at commit

[GitHub] spark issue #20359: [SPARK-23186][SQL] Initialize DriverManager first before...

2018-01-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20359 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/141/

[GitHub] spark issue #20359: [SPARK-23186][SQL] Initialize DriverManager first before...

2018-01-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20359 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20359: [SPARK-23186][SQL] Initialize DriverManager first before...

2018-01-23 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/20359 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark pull request #18906: [SPARK-21692][PYSPARK][SQL] Add nullability suppo...

2018-01-23 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/18906#discussion_r163238156 --- Diff: python/pyspark/sql/udf.py --- @@ -180,17 +182,29 @@ def wrapper(*args): wrapper.deterministic = self.deterministic

[GitHub] spark issue #20350: [SPARK-23179][SQL] Support option to throw exception if ...

2018-01-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20350 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20350: [SPARK-23179][SQL] Support option to throw exception if ...

2018-01-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20350 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/140/

[GitHub] spark issue #18143: [SPARK-20919][SS] Simplificaiton of CachedKafkaConsumer ...

2018-01-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18143 **[Test build #86529 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86529/testReport)** for PR 18143 at commit

[GitHub] spark issue #20350: [SPARK-23179][SQL] Support option to throw exception if ...

2018-01-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20350 **[Test build #86528 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86528/testReport)** for PR 20350 at commit

[GitHub] spark issue #18143: [SPARK-20919][SS] Simplificaiton of CachedKafkaConsumer ...

2018-01-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18143 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #18143: [SPARK-20919][SS] Simplificaiton of CachedKafkaConsumer ...

2018-01-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18143 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/139/

[GitHub] spark issue #20360: [SPARK-23177][SQL][PySpark] Extract zero-parameter UDFs ...

2018-01-23 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/20360 LGTM --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark pull request #20360: [SPARK-23177][SQL][PySpark] Extract zero-paramete...

2018-01-23 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/20360#discussion_r163235275 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/python/ExtractPythonUDFs.scala --- @@ -45,7 +45,8 @@ object

[GitHub] spark pull request #20360: [SPARK-23177][SQL][PySpark] Extract zero-paramete...

2018-01-23 Thread ueshin
Github user ueshin commented on a diff in the pull request: https://github.com/apache/spark/pull/20360#discussion_r163234660 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/python/ExtractPythonUDFs.scala --- @@ -45,7 +45,8 @@ object ExtractPythonUDFFromAggregate

[GitHub] spark issue #20350: [SPARK-23179][SQL] Support option to throw exception if ...

2018-01-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20350 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/86524/ Test PASSed. ---

[GitHub] spark issue #20350: [SPARK-23179][SQL] Support option to throw exception if ...

2018-01-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20350 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20350: [SPARK-23179][SQL] Support option to throw exception if ...

2018-01-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20350 **[Test build #86524 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86524/testReport)** for PR 20350 at commit

[GitHub] spark issue #13599: [SPARK-13587] [PYSPARK] Support virtualenv in pyspark

2018-01-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13599 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #13599: [SPARK-13587] [PYSPARK] Support virtualenv in pyspark

2018-01-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13599 **[Test build #86527 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86527/testReport)** for PR 13599 at commit

[GitHub] spark issue #13599: [SPARK-13587] [PYSPARK] Support virtualenv in pyspark

2018-01-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13599 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/86527/ Test FAILed. ---

[GitHub] spark issue #13599: [SPARK-13587] [PYSPARK] Support virtualenv in pyspark

2018-01-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13599 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/138/

[GitHub] spark issue #13599: [SPARK-13587] [PYSPARK] Support virtualenv in pyspark

2018-01-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13599 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #13599: [SPARK-13587] [PYSPARK] Support virtualenv in pyspark

2018-01-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13599 **[Test build #86527 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86527/testReport)** for PR 13599 at commit

[GitHub] spark pull request #20358: [SPARK-20749][SQL][FOLLOW-UP] Override prettyName...

2018-01-23 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/20358 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #20358: [SPARK-20749][SQL][FOLLOW-UP] Override prettyName for bi...

2018-01-23 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/20358 Merged to master and branch-2.3. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands,

[GitHub] spark pull request #20360: [SPARK-23177][SQL][PySpark] Extract zero-paramete...

2018-01-23 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/20360#discussion_r163229202 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/python/ExtractPythonUDFs.scala --- @@ -45,7 +45,8 @@ object

[GitHub] spark issue #20360: [SPARK-23177][SQL][PySpark] Extract zero-parameter UDFs ...

2018-01-23 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/20360 LGTM --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark pull request #20360: [SPARK-23177][SQL][PySpark] Extract zero-paramete...

2018-01-23 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/20360#discussion_r163228866 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/python/ExtractPythonUDFs.scala --- @@ -45,7 +45,8 @@ object

[GitHub] spark issue #20362: [Spark-22886][ML][TESTS] ML test for structured streamin...

2018-01-23 Thread gaborgsomogyi
Github user gaborgsomogyi commented on the issue: https://github.com/apache/spark/pull/20362 cc @jkbradley @WeichenXu123 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands,

<    1   2   3   4   5   6   >