[GitHub] spark issue #20126: [DO-NOT-MERGE] Investigate if changes in flume.py actual...

2017-12-31 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/20126 Hm, I see. Will open a followup PR. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20126: [DO-NOT-MERGE] Investigate if changes in flume.py actual...

2017-12-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20126 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/85560/ Test FAILed. ---

[GitHub] spark issue #20126: [DO-NOT-MERGE] Investigate if changes in flume.py actual...

2017-12-31 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20126 **[Test build #85560 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85560/testReport)** for PR 20126 at commit

[GitHub] spark issue #20126: [DO-NOT-MERGE] Investigate if changes in flume.py actual...

2017-12-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20126 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20126: [DO-NOT-MERGE] Investigate if changes in flume.py actual...

2017-12-31 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20126 **[Test build #85560 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85560/testReport)** for PR 20126 at commit

[GitHub] spark pull request #20069: [SPARK-22895] [SQL] Push down the deterministic p...

2017-12-31 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/20069#discussion_r159141412 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/Optimizer.scala --- @@ -851,7 +851,7 @@ object PushDownPredicate extends

[GitHub] spark issue #20126: [DO-NOT-MERGE] Investigate if changes in flume.py actual...

2017-12-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20126 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20126: [DO-NOT-MERGE] Investigate if changes in flume.py actual...

2017-12-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20126 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/85559/ Test FAILed. ---

[GitHub] spark issue #20126: [DO-NOT-MERGE] Investigate if changes in flume.py actual...

2017-12-31 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20126 **[Test build #85559 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85559/testReport)** for PR 20126 at commit

[GitHub] spark issue #20126: [DO-NOT-MERGE] Investigate if changes in flume.py actual...

2017-12-31 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20126 **[Test build #85559 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85559/testReport)** for PR 20126 at commit

[GitHub] spark issue #20125: [SPARK-17967][SQL] Support for array as an option in SQL...

2017-12-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20125 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20125: [SPARK-17967][SQL] Support for array as an option in SQL...

2017-12-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20125 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/8/ Test PASSed. ---

[GitHub] spark issue #20125: [SPARK-17967][SQL] Support for array as an option in SQL...

2017-12-31 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20125 **[Test build #8 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/8/testReport)** for PR 20125 at commit

[GitHub] spark issue #20125: [SPARK-17967][SQL] Support for array as an option in SQL...

2017-12-31 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/20125 I actually think https://github.com/apache/spark/pull/20125#issuecomment-354604768 are good points and I was hesitant about it. Although IMHO I think it's fine but let me cc @hvanhovell and

[GitHub] spark issue #20125: [SPARK-17967][SQL] Support for array as an option in SQL...

2017-12-31 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/20125 > Btw, is this any difference than using string? Like: Nope, they will be the same but I was thinking this is a simplest fix. ---

[GitHub] spark issue #20125: [SPARK-17967][SQL] Support for array as an option in SQL...

2017-12-31 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/20125 Yup, I was thinking of SparkSQL only feature. For more details, the original intention was to support multiple values for `nullValue` but I realised such option support can be

[GitHub] spark issue #20125: [SPARK-17967][SQL] Support for array as an option in SQL...

2017-12-31 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/20125 Is this a special feature for SparkSQL only? Seems Hive doesn't have such support. --- - To unsubscribe, e-mail:

[GitHub] spark issue #20126: [DO-NOT-MERGE] Investigate if changes in flume.py actual...

2017-12-31 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20126 **[Test build #85558 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85558/testReport)** for PR 20126 at commit

[GitHub] spark issue #20126: [DO-NOT-MERGE] Investigate if changes in flume.py actual...

2017-12-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20126 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20126: [DO-NOT-MERGE] Investigate if changes in flume.py actual...

2017-12-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20126 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/85556/ Test FAILed. ---

[GitHub] spark issue #20126: [DO-NOT-MERGE] Investigate if changes in flume.py actual...

2017-12-31 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20126 **[Test build #85556 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85556/testReport)** for PR 20126 at commit

[GitHub] spark issue #20127: [SPARK-22932] [SQL] Refactor AnalysisContext

2017-12-31 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20127 **[Test build #85557 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85557/testReport)** for PR 20127 at commit

[GitHub] spark issue #20127: [SPARK-22932] [SQL] Refactor AnalysisContext

2017-12-31 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/20127 cc @cloud-fan @jiangxb1987 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands,

[GitHub] spark pull request #20127: [SPARK-22932] [SQL] Refactor AnalysisContext

2017-12-31 Thread gatorsmile
GitHub user gatorsmile opened a pull request: https://github.com/apache/spark/pull/20127 [SPARK-22932] [SQL] Refactor AnalysisContext ## What changes were proposed in this pull request? Add a `reset` function to ensure the state in `AnalysisContext ` is per-query. ##

[GitHub] spark issue #20126: [DO-NOT-MERGE] Investigate if changes in flume.py actual...

2017-12-31 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20126 **[Test build #85556 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85556/testReport)** for PR 20126 at commit

[GitHub] spark pull request #20126: [DO-NOT-MERGE] Investigate if changes in flume.py...

2017-12-31 Thread HyukjinKwon
GitHub user HyukjinKwon opened a pull request: https://github.com/apache/spark/pull/20126 [DO-NOT-MERGE] Investigate if changes in flume.py actually triggeres related tests ## What changes were proposed in this pull request? Do not merge this. Seems the changes in

[GitHub] spark pull request #19991: [SPARK-22801][ML][PYSPARK] Allow FeatureHasher to...

2017-12-31 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/19991 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #19991: [SPARK-22801][ML][PYSPARK] Allow FeatureHasher to treat ...

2017-12-31 Thread MLnick
Github user MLnick commented on the issue: https://github.com/apache/spark/pull/19991 Merged to master. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark pull request #19715: [SPARK-22397][ML]add multiple columns support to ...

2017-12-31 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/19715 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #19715: [SPARK-22397][ML]add multiple columns support to Quantil...

2017-12-31 Thread MLnick
Github user MLnick commented on the issue: https://github.com/apache/spark/pull/19715 Merged to master. If there are any further small comments / clean ups we can do that during QA for 2.3 Thanks @huaxingao and all others for review! ---

[GitHub] spark issue #20114: [SPARK-22530][PYTHON][SQL] Adding Arrow support for Arra...

2017-12-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20114 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/85554/ Test PASSed. ---

[GitHub] spark issue #20114: [SPARK-22530][PYTHON][SQL] Adding Arrow support for Arra...

2017-12-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20114 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20114: [SPARK-22530][PYTHON][SQL] Adding Arrow support for Arra...

2017-12-31 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20114 **[Test build #85554 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85554/testReport)** for PR 20114 at commit

[GitHub] spark issue #20125: [SPARK-17967][SQL] Support for array as an option in SQL...

2017-12-31 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20125 **[Test build #8 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/8/testReport)** for PR 20125 at commit

[GitHub] spark issue #20125: [SPARK-17967][SQL] Support for array as an option in SQL...

2017-12-31 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/20125 cc @gatorsmile could you take a look please? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For

[GitHub] spark pull request #20125: [SPARK-17967][SQL] Support for array as an option...

2017-12-31 Thread HyukjinKwon
GitHub user HyukjinKwon opened a pull request: https://github.com/apache/spark/pull/20125 [SPARK-17967][SQL] Support for array as an option in SQL parser ## What changes were proposed in this pull request? This PR targets to add the ability for dealing with an array (JSON

[GitHub] spark pull request #20076: [SPARK-21786][SQL] When acquiring 'compressionCod...

2017-12-31 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/20076#discussion_r159136922 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/CompressionCodecPrecedenceSuite.scala --- @@ -0,0 +1,60 @@ +/* + * Licensed to the Apache

[GitHub] spark issue #20124: [WIP][SPARK-22126][ML] Fix model-specific optimization s...

2017-12-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20124 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20124: [WIP][SPARK-22126][ML] Fix model-specific optimization s...

2017-12-31 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20124 **[Test build #85553 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85553/testReport)** for PR 20124 at commit

[GitHub] spark issue #20124: [WIP][SPARK-22126][ML] Fix model-specific optimization s...

2017-12-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20124 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/85553/ Test FAILed. ---

[GitHub] spark issue #20114: [SPARK-22530][PYTHON][SQL] Adding Arrow support for Arra...

2017-12-31 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20114 **[Test build #85554 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85554/testReport)** for PR 20114 at commit

[GitHub] spark issue #20114: [SPARK-22530][PYTHON][SQL] Adding Arrow support for Arra...

2017-12-31 Thread BryanCutler
Github user BryanCutler commented on the issue: https://github.com/apache/spark/pull/20114 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #20124: [WIP][SPARK-22126][ML] Fix model-specific optimization s...

2017-12-31 Thread BryanCutler
Github user BryanCutler commented on the issue: https://github.com/apache/spark/pull/20124 This basically works by splitting the array of ParamMaps into two. One that has params that can be optimized by the estimator, and one that can be parallelized over. These are then grouped

[GitHub] spark issue #20124: [WIP][SPARK-22126][ML] Fix model-specific optimization s...

2017-12-31 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20124 **[Test build #85553 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85553/testReport)** for PR 20124 at commit

[GitHub] spark pull request #20124: [WIP][SPARK-22126][ML] Fix model-specific optimiz...

2017-12-31 Thread BryanCutler
GitHub user BryanCutler opened a pull request: https://github.com/apache/spark/pull/20124 [WIP][SPARK-22126][ML] Fix model-specific optimization support for ML tuning. ## What changes were proposed in this pull request? Support model-specific optimizations for

[GitHub] spark issue #20114: [SPARK-22530][PYTHON][SQL] Adding Arrow support for Arra...

2017-12-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20114 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/85552/ Test FAILed. ---

[GitHub] spark issue #20114: [SPARK-22530][PYTHON][SQL] Adding Arrow support for Arra...

2017-12-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20114 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20114: [SPARK-22530][PYTHON][SQL] Adding Arrow support for Arra...

2017-12-31 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20114 **[Test build #85552 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85552/testReport)** for PR 20114 at commit

[GitHub] spark pull request #20072: [SPARK-22790][SQL] add a configurable factor to d...

2017-12-31 Thread wzhfy
Github user wzhfy commented on a diff in the pull request: https://github.com/apache/spark/pull/20072#discussion_r159135028 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala --- @@ -261,6 +261,17 @@ object SQLConf { .booleanConf

[GitHub] spark pull request #20072: [SPARK-22790][SQL] add a configurable factor to d...

2017-12-31 Thread wzhfy
Github user wzhfy commented on a diff in the pull request: https://github.com/apache/spark/pull/20072#discussion_r159134987 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala --- @@ -261,6 +261,17 @@ object SQLConf { .booleanConf

[GitHub] spark pull request #20072: [SPARK-22790][SQL] add a configurable factor to d...

2017-12-31 Thread wzhfy
Github user wzhfy commented on a diff in the pull request: https://github.com/apache/spark/pull/20072#discussion_r159135036 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/HadoopFsRelation.scala --- @@ -60,6 +60,8 @@ case class HadoopFsRelation(

[GitHub] spark pull request #20072: [SPARK-22790][SQL] add a configurable factor to d...

2017-12-31 Thread wzhfy
Github user wzhfy commented on a diff in the pull request: https://github.com/apache/spark/pull/20072#discussion_r159135272 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/HadoopFsRelation.scala --- @@ -82,7 +84,15 @@ case class HadoopFsRelation(

<    1   2