[GitHub] spark issue #17308: [SPARK-19968][SS] Use a cached instance of `KafkaProduce...

2017-03-15 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17308 **[Test build #74644 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/74644/testReport)** for PR 17308 at commit

[GitHub] spark issue #17291: [SPARK-19949][SQL][WIP] unify bad record handling in CSV...

2017-03-15 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17291 **[Test build #74645 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/74645/testReport)** for PR 17291 at commit

[GitHub] spark pull request #17308: [SPARK-19968][SS] Use a cached instance of `Kafka...

2017-03-15 Thread ScrapCodes
GitHub user ScrapCodes opened a pull request: https://github.com/apache/spark/pull/17308 [SPARK-19968][SS] Use a cached instance of `KafkaProducer` instead of creating one every batch. ## What changes were proposed in this pull request? Changes include a new API for doing

[GitHub] spark issue #17270: [SPARK-19929] [SQL] Showing Hive Managed table's LOATION...

2017-03-15 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/17270 Yeah. you need to close the PR by yourself. We are unable to close it. thanks --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If

[GitHub] spark issue #16626: [SPARK-19261][SQL] Alter add columns for Hive serde and ...

2017-03-15 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/16626 Could you add a scenario when users add a column name that already exists in the table schema? --- If your project is set up for it, you can reply to this email and have your reply appear on

[GitHub] spark issue #17287: [SPARK-19945][SQL]add test suite for SessionCatalog with...

2017-03-15 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17287 **[Test build #74643 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/74643/testReport)** for PR 17287 at commit

[GitHub] spark pull request #16626: [SPARK-19261][SQL] Alter add columns for Hive ser...

2017-03-15 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/16626#discussion_r106342233 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/jdbc/JDBCSuite.scala --- @@ -71,7 +71,6 @@ class JDBCSuite extends SparkFunSuite

[GitHub] spark pull request #16626: [SPARK-19261][SQL] Alter add columns for Hive ser...

2017-03-15 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/16626#discussion_r106342123 --- Diff: sql/hive/src/test/scala/org/apache/spark/sql/hive/execution/HiveDDLSuite.scala --- @@ -1860,4 +1860,72 @@ class HiveDDLSuite }

[GitHub] spark pull request #17287: [SPARK-19945][SQL]add test suite for SessionCatal...

2017-03-15 Thread windpiger
Github user windpiger commented on a diff in the pull request: https://github.com/apache/spark/pull/17287#discussion_r106342082 --- Diff: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/catalog/SessionCatalogSuite.scala --- @@ -1270,6 +1376,7 @@ class

[GitHub] spark pull request #16626: [SPARK-19261][SQL] Alter add columns for Hive ser...

2017-03-15 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/16626#discussion_r106341966 --- Diff: sql/hive/src/test/scala/org/apache/spark/sql/hive/execution/HiveDDLSuite.scala --- @@ -1860,4 +1860,72 @@ class HiveDDLSuite }

[GitHub] spark issue #17287: [SPARK-19945][SQL]add test suite for SessionCatalog with...

2017-03-15 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17287 **[Test build #74642 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/74642/testReport)** for PR 17287 at commit

[GitHub] spark pull request #16626: [SPARK-19261][SQL] Alter add columns for Hive ser...

2017-03-15 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/16626#discussion_r106341499 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/command/tables.scala --- @@ -175,6 +178,78 @@ case class AlterTableRenameCommand(

[GitHub] spark issue #17242: [SPARK-19902][SQL] Support more expression canonicalizat...

2017-03-15 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/17242 anyway, I will move it to optimizer in next update. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark pull request #17286: [SPARK-19915][SQL] Exclude cartesian product cand...

2017-03-15 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/17286#discussion_r106341181 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala --- @@ -696,6 +696,13 @@ object SQLConf { .intConf

[GitHub] spark pull request #17287: [SPARK-19945][SQL]add test suite for SessionCatal...

2017-03-15 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/17287#discussion_r106340900 --- Diff: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/catalog/SessionCatalogSuite.scala --- @@ -1270,6 +1376,7 @@ class

[GitHub] spark issue #17084: [SPARK-18693][ML][MLLIB] ML Evaluators should use weight...

2017-03-15 Thread actuaryzhang
Github user actuaryzhang commented on the issue: https://github.com/apache/spark/pull/17084 Thanks for the PR. I think this is helpful. Will take a look next week. Quite swamped recently. --- If your project is set up for it, you can reply to this email and have your reply appear

[GitHub] spark issue #16867: [SPARK-16929] Improve performance when check speculatabl...

2017-03-15 Thread jinxing64
Github user jinxing64 commented on the issue: https://github.com/apache/spark/pull/16867 @kayousterhout Thanks a lot for the comments :) very helpful. I've refined, please take another look when you have time. --- If your project is set up for it, you can reply to this

[GitHub] spark issue #16722: [SPARK-19591][ML][MLlib] Add sample weights to decision ...

2017-03-15 Thread imatiach-msft
Github user imatiach-msft commented on the issue: https://github.com/apache/spark/pull/16722 @jkbradley might you be able to take a look at the changes from @sethah ? Thank you! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub

[GitHub] spark pull request #11780: [SPARK-8884][MLlib] 1-sample Anderson-Darling Goo...

2017-03-15 Thread hhbyyh
Github user hhbyyh closed the pull request at: https://github.com/apache/spark/pull/11780 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark issue #11780: [SPARK-8884][MLlib] 1-sample Anderson-Darling Goodness-o...

2017-03-15 Thread hhbyyh
Github user hhbyyh commented on the issue: https://github.com/apache/spark/pull/11780 Close this. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if

[GitHub] spark pull request #16867: [SPARK-16929] Improve performance when check spec...

2017-03-15 Thread jinxing64
Github user jinxing64 commented on a diff in the pull request: https://github.com/apache/spark/pull/16867#discussion_r106340513 --- Diff: core/src/test/scala/org/apache/spark/scheduler/TaskSetManagerSuite.scala --- @@ -893,6 +893,7 @@ class TaskSetManagerSuite extends

[GitHub] spark pull request #16867: [SPARK-16929] Improve performance when check spec...

2017-03-15 Thread jinxing64
Github user jinxing64 commented on a diff in the pull request: https://github.com/apache/spark/pull/16867#discussion_r106340321 --- Diff: core/src/main/scala/org/apache/spark/scheduler/TaskSchedulerImpl.scala --- @@ -172,7 +172,7 @@ private[spark] class TaskSchedulerImpl

[GitHub] spark pull request #17287: [SPARK-19945][SQL]add test suite for SessionCatal...

2017-03-15 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/17287#discussion_r106340219 --- Diff: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/catalog/SessionCatalogSuite.scala --- @@ -999,257 +1083,279 @@ class

[GitHub] spark issue #17123: [SPARK-19781][ML] Handle NULLs as well as NaNs in Bucket...

2017-03-15 Thread imatiach-msft
Github user imatiach-msft commented on the issue: https://github.com/apache/spark/pull/17123 @crackcell I'm not sure about changing the UDF to be on a row instead of a column, I've found that the serialization costs are much higher and the spark code performs much less. Maybe an

[GitHub] spark pull request #17123: [SPARK-19781][ML] Handle NULLs as well as NaNs in...

2017-03-15 Thread imatiach-msft
Github user imatiach-msft commented on a diff in the pull request: https://github.com/apache/spark/pull/17123#discussion_r106339981 --- Diff: mllib/src/main/scala/org/apache/spark/ml/feature/Bucketizer.scala --- @@ -171,34 +173,34 @@ object Bucketizer extends

[GitHub] spark issue #17251: [SPARK-19910][SQL] `stack` should not reject NULL values...

2017-03-15 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17251 **[Test build #74641 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/74641/testReport)** for PR 17251 at commit

[GitHub] spark pull request #17123: [SPARK-19781][ML] Handle NULLs as well as NaNs in...

2017-03-15 Thread imatiach-msft
Github user imatiach-msft commented on a diff in the pull request: https://github.com/apache/spark/pull/17123#discussion_r106339731 --- Diff: mllib/src/main/scala/org/apache/spark/ml/feature/Bucketizer.scala --- @@ -105,20 +106,21 @@ final class Bucketizer @Since("1.4.0")

[GitHub] spark issue #17251: [SPARK-19910][SQL] `stack` should not reject NULL values...

2017-03-15 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/17251 Thank you, @cloud-fan ! I updated the PR according to the review comments. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If

[GitHub] spark issue #16867: [SPARK-16929] Improve performance when check speculatabl...

2017-03-15 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16867 **[Test build #74640 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/74640/testReport)** for PR 16867 at commit

[GitHub] spark issue #17130: [SPARK-19791] [ML] Add doc and example for fpgrowth

2017-03-15 Thread hhbyyh
Github user hhbyyh commented on the issue: https://github.com/apache/spark/pull/17130 Refined some comments and minor things. This should be ready for review. Thanks. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If

[GitHub] spark issue #16209: [WIP][SPARK-10849][SQL] Adds option to the JDBC data sou...

2017-03-15 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/16209 @sureshthalamati https://github.com/apache/spark/pull/17171 has been resolved. Can you update your PR by allowing users to specify the schema in DDL format? --- If your project is set up for

[GitHub] spark issue #17085: [SPARK-18693][ML][MLLIB] ML Evaluators should use weight...

2017-03-15 Thread imatiach-msft
Github user imatiach-msft commented on the issue: https://github.com/apache/spark/pull/17085 ping @sethah @Lewuathe @thunterdb @WeichenXu123 @jkbradley @actuaryzhang @srowen could you please take a look? thank you! --- If your project is set up for it, you can reply to this email

[GitHub] spark pull request #17251: [SPARK-19910][SQL] `stack` should not reject NULL...

2017-03-15 Thread dongjoon-hyun
Github user dongjoon-hyun commented on a diff in the pull request: https://github.com/apache/spark/pull/17251#discussion_r106338768 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/TypeCoercion.scala --- @@ -590,6 +591,23 @@ object TypeCoercion {

[GitHub] spark issue #17086: [SPARK-18693][ML][MLLIB] ML Evaluators should use weight...

2017-03-15 Thread imatiach-msft
Github user imatiach-msft commented on the issue: https://github.com/apache/spark/pull/17086 ping @sethah @Lewuathe @thunterdb @WeichenXu123 @jkbradley @actuaryzhang @srowen could you please take a look? thank you! --- If your project is set up for it, you can reply to this email

[GitHub] spark issue #17084: [SPARK-18693][ML][MLLIB] ML Evaluators should use weight...

2017-03-15 Thread imatiach-msft
Github user imatiach-msft commented on the issue: https://github.com/apache/spark/pull/17084 ping @sethah @Lewuathe @thunterdb @WeichenXu123 @jkbradley @actuaryzhang @srowen could you please take a look? thank you! --- If your project is set up for it, you can reply to this email

[GitHub] spark issue #17289: [SPARK-19948] Document that saveAsTable uses catalog as ...

2017-03-15 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/17289 This is the design decision we need to make here. Spark SQL is kind of a federation system. Two write APIs behave differently. The `saveAsTable` API expects users to register it in the

[GitHub] spark pull request #17286: [SPARK-19915][SQL] Exclude cartesian product cand...

2017-03-15 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/17286#discussion_r106338345 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/CostBasedJoinReorder.scala --- @@ -128,38 +128,43 @@ case class

[GitHub] spark pull request #17307: [SPARK-13369] Make number of consecutive fetch fa...

2017-03-15 Thread kayousterhout
Github user kayousterhout commented on a diff in the pull request: https://github.com/apache/spark/pull/17307#discussion_r106337950 --- Diff: docs/configuration.md --- @@ -1506,6 +1506,11 @@ Apart from these, the following properties are also available, and may be useful

[GitHub] spark issue #17297: [SPARK-14649][CORE] DagScheduler should not run duplicat...

2017-03-15 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17297 **[Test build #74631 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/74631/testReport)** for PR 17297 at commit

[GitHub] spark issue #17297: [SPARK-14649][CORE] DagScheduler should not run duplicat...

2017-03-15 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17297 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/74631/ Test FAILed. ---

[GitHub] spark issue #17297: [SPARK-14649][CORE] DagScheduler should not run duplicat...

2017-03-15 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17297 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #16476: [SPARK-19084][SQL] Implement expression field

2017-03-15 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16476 **[Test build #74639 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/74639/testReport)** for PR 16476 at commit

[GitHub] spark pull request #17251: [SPARK-19910][SQL] `stack` should not reject NULL...

2017-03-15 Thread dongjoon-hyun
Github user dongjoon-hyun commented on a diff in the pull request: https://github.com/apache/spark/pull/17251#discussion_r106337249 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/generators.scala --- @@ -156,9 +156,21 @@ case class Stack(children:

[GitHub] spark pull request #17251: [SPARK-19910][SQL] `stack` should not reject NULL...

2017-03-15 Thread dongjoon-hyun
Github user dongjoon-hyun commented on a diff in the pull request: https://github.com/apache/spark/pull/17251#discussion_r106337141 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/TypeCoercion.scala --- @@ -590,6 +591,23 @@ object TypeCoercion {

[GitHub] spark pull request #17287: [SPARK-19945][SQL]add test suite for SessionCatal...

2017-03-15 Thread windpiger
Github user windpiger commented on a diff in the pull request: https://github.com/apache/spark/pull/17287#discussion_r106336735 --- Diff: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/catalog/SessionCatalogSuite.scala --- @@ -76,468 +102,500 @@ class

[GitHub] spark issue #17287: [SPARK-19945][SQL]add test suite for SessionCatalog with...

2017-03-15 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17287 **[Test build #74638 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/74638/testReport)** for PR 17287 at commit

[GitHub] spark issue #17287: [SPARK-19945][SQL]add test suite for SessionCatalog with...

2017-03-15 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17287 **[Test build #74637 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/74637/testReport)** for PR 17287 at commit

[GitHub] spark issue #16981: [SPARK-19637][SQL] Add to_json in FunctionRegistry

2017-03-15 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/16981 cc @maropu https://github.com/apache/spark/pull/17171 is merged. Are you interested in working on `from_json`? JIRA: https://issues.apache.org/jira/browse/SPARK-19967 --- If your

[GitHub] spark pull request #17287: [SPARK-19945][SQL]add test suite for SessionCatal...

2017-03-15 Thread windpiger
Github user windpiger commented on a diff in the pull request: https://github.com/apache/spark/pull/17287#discussion_r106336045 --- Diff: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/catalog/SessionCatalogSuite.scala --- @@ -999,257 +1083,279 @@ class

[GitHub] spark pull request #17287: [SPARK-19945][SQL]add test suite for SessionCatal...

2017-03-15 Thread windpiger
Github user windpiger commented on a diff in the pull request: https://github.com/apache/spark/pull/17287#discussion_r106335967 --- Diff: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/catalog/SessionCatalogSuite.scala --- @@ -1270,6 +1376,7 @@ class

[GitHub] spark pull request #17088: [SPARK-19753][CORE] Un-register all shuffle outpu...

2017-03-15 Thread squito
Github user squito commented on a diff in the pull request: https://github.com/apache/spark/pull/17088#discussion_r106335824 --- Diff: core/src/main/scala/org/apache/spark/scheduler/DAGScheduler.scala --- @@ -1365,19 +1369,27 @@ class DAGScheduler( */

[GitHub] spark pull request #17088: [SPARK-19753][CORE] Un-register all shuffle outpu...

2017-03-15 Thread squito
Github user squito commented on a diff in the pull request: https://github.com/apache/spark/pull/17088#discussion_r106330358 --- Diff: core/src/main/scala/org/apache/spark/scheduler/DAGScheduler.scala --- @@ -1331,7 +1328,14 @@ class DAGScheduler( // TODO:

[GitHub] spark pull request #17088: [SPARK-19753][CORE] Un-register all shuffle outpu...

2017-03-15 Thread squito
Github user squito commented on a diff in the pull request: https://github.com/apache/spark/pull/17088#discussion_r106335566 --- Diff: core/src/test/scala/org/apache/spark/scheduler/DAGSchedulerSuite.scala --- @@ -394,6 +394,32 @@ class DAGSchedulerSuite extends SparkFunSuite with

[GitHub] spark pull request #17287: [SPARK-19945][SQL]add test suite for SessionCatal...

2017-03-15 Thread windpiger
Github user windpiger commented on a diff in the pull request: https://github.com/apache/spark/pull/17287#discussion_r106335778 --- Diff: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/catalog/SessionCatalogSuite.scala --- @@ -999,257 +1083,279 @@ class

[GitHub] spark pull request #17171: [SPARK-19830] [SQL] Add parseTableSchema API to P...

2017-03-15 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/17171 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark issue #17171: [SPARK-19830] [SQL] Add parseTableSchema API to ParserIn...

2017-03-15 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/17171 thanks, merging to master! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and

[GitHub] spark issue #17289: [SPARK-19948] Document that saveAsTable uses catalog as ...

2017-03-15 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/17289 It's pretty weird that we need to check if table exists in spark catalog and then check if data exists in data source, by the same save mode specified by users. I think the new behavior is more

[GitHub] spark pull request #16971: [SPARK-19573][SQL] Make NaN/null handling consist...

2017-03-15 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/16971#discussion_r106335040 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/aggregate/ApproximatePercentile.scala --- @@ -245,7 +245,7 @@ object

[GitHub] spark issue #16971: [SPARK-19573][SQL] Make NaN/null handling consistent in ...

2017-03-15 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/16971 ping @zhengruifeng --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes

[GitHub] spark pull request #16476: [SPARK-19084][SQL] Implement expression field

2017-03-15 Thread gczsjdy
Github user gczsjdy commented on a diff in the pull request: https://github.com/apache/spark/pull/16476#discussion_r106334932 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/conditionalExpressions.scala --- @@ -340,3 +343,105 @@ object CaseKeyWhen

[GitHub] spark pull request #17287: [SPARK-19945][SQL]add test suite for SessionCatal...

2017-03-15 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/17287#discussion_r106334939 --- Diff: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/catalog/SessionCatalogSuite.scala --- @@ -999,257 +1083,279 @@ class

[GitHub] spark issue #17171: [SPARK-19830] [SQL] Add parseTableSchema API to ParserIn...

2017-03-15 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17171 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/74635/ Test PASSed. ---

[GitHub] spark issue #17171: [SPARK-19830] [SQL] Add parseTableSchema API to ParserIn...

2017-03-15 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17171 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #17171: [SPARK-19830] [SQL] Add parseTableSchema API to ParserIn...

2017-03-15 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17171 **[Test build #74635 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/74635/testReport)** for PR 17171 at commit

[GitHub] spark pull request #17287: [SPARK-19945][SQL]add test suite for SessionCatal...

2017-03-15 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/17287#discussion_r106334827 --- Diff: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/catalog/SessionCatalogSuite.scala --- @@ -999,257 +1094,279 @@ class

[GitHub] spark pull request #17287: [SPARK-19945][SQL]add test suite for SessionCatal...

2017-03-15 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/17287#discussion_r106334626 --- Diff: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/catalog/SessionCatalogSuite.scala --- @@ -999,257 +1083,279 @@ class

[GitHub] spark pull request #17287: [SPARK-19945][SQL]add test suite for SessionCatal...

2017-03-15 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/17287#discussion_r106334485 --- Diff: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/catalog/SessionCatalogSuite.scala --- @@ -1270,6 +1376,7 @@ class

[GitHub] spark pull request #16476: [SPARK-19084][SQL] Implement expression field

2017-03-15 Thread gczsjdy
Github user gczsjdy commented on a diff in the pull request: https://github.com/apache/spark/pull/16476#discussion_r106334083 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/conditionalExpressions.scala --- @@ -340,3 +343,105 @@ object CaseKeyWhen

[GitHub] spark pull request #17287: [SPARK-19945][SQL]add test suite for SessionCatal...

2017-03-15 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/17287#discussion_r106333835 --- Diff: sql/hive/src/test/scala/org/apache/spark/sql/hive/HiveExternalSessionCatalogSuite.scala --- @@ -0,0 +1,40 @@ +/* + * Licensed to the

[GitHub] spark issue #16626: [SPARK-19261][SQL] Alter add columns for Hive serde and ...

2017-03-15 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16626 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #16626: [SPARK-19261][SQL] Alter add columns for Hive serde and ...

2017-03-15 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16626 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/74634/ Test PASSed. ---

[GitHub] spark issue #16626: [SPARK-19261][SQL] Alter add columns for Hive serde and ...

2017-03-15 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16626 **[Test build #74634 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/74634/testReport)** for PR 16626 at commit

[GitHub] spark issue #17289: [SPARK-19948] Document that saveAsTable uses catalog as ...

2017-03-15 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/17289 ```Scala test("saveAsTable API with SaveMode.Overwrite") { val df = spark.createDataFrame(sparkContext.parallelize(arr1x2), schema2) spark.read.jdbc(url1, "test.people",

[GitHub] spark issue #17307: [SPARK-13369] Make number of consecutive fetch failures ...

2017-03-15 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17307 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #17307: [SPARK-13369] Make number of consecutive fetch failures ...

2017-03-15 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17307 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/74633/ Test PASSed. ---

[GitHub] spark issue #17307: [SPARK-13369] Make number of consecutive fetch failures ...

2017-03-15 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17307 **[Test build #74633 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/74633/testReport)** for PR 17307 at commit

[GitHub] spark issue #17242: [SPARK-19902][SQL] Support more expression canonicalizat...

2017-03-15 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/17242 hmm, so you don't think canonicalizer should use this? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark issue #17242: [SPARK-19902][SQL] Support more expression canonicalizat...

2017-03-15 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/17242 not "integration", but "move". I think this logic belongs to optimizer instead of canonicalizer --- If your project is set up for it, you can reply to this email and have your reply appear on

[GitHub] spark issue #16905: [SPARK-19567][CORE][SCHEDULER] Support some Schedulable ...

2017-03-15 Thread squito
Github user squito commented on the issue: https://github.com/apache/spark/pull/16905 reopened https://issues.apache.org/jira/browse/SPARK-7420 for the failure Jenkins, retest this please --- If your project is set up for it, you can reply to this email and have your reply

[GitHub] spark issue #17289: [SPARK-19948] Document that saveAsTable uses catalog as ...

2017-03-15 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/17289 it's probably ok as we turn an error case runnable. But we should document this in release notes --- If your project is set up for it, you can reply to this email and have your reply appear on

[GitHub] spark pull request #17251: [SPARK-19910][SQL] `stack` should not reject NULL...

2017-03-15 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/17251#discussion_r106329490 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/TypeCoercion.scala --- @@ -590,6 +591,23 @@ object TypeCoercion { }

[GitHub] spark issue #17289: [SPARK-19948] Document that saveAsTable uses catalog as ...

2017-03-15 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/17289 We introduced a behavior change in Spark 2.2. In Spark 2.1, we reported an error if the underlying JDBC table exists. We changed [the mode to

[GitHub] spark issue #17242: [SPARK-19902][SQL] Support more expression canonicalizat...

2017-03-15 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/17242 ping @cloud-fan Except for the optimization integration, do you have more comments on this change? Thanks. --- If your project is set up for it, you can reply to this email and have your reply

[GitHub] spark pull request #17307: [SPARK-13369] Make number of consecutive fetch fa...

2017-03-15 Thread squito
Github user squito commented on a diff in the pull request: https://github.com/apache/spark/pull/17307#discussion_r106328854 --- Diff: docs/configuration.md --- @@ -1506,6 +1506,11 @@ Apart from these, the following properties are also available, and may be useful of this

[GitHub] spark issue #17130: [SPARK-19791] [ML] Add doc and example for fpgrowth

2017-03-15 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17130 **[Test build #74636 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/74636/testReport)** for PR 17130 at commit

[GitHub] spark issue #17130: [SPARK-19791] [ML] Add doc and example for fpgrowth

2017-03-15 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17130 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/74636/ Test PASSed. ---

[GitHub] spark issue #17130: [SPARK-19791] [ML] Add doc and example for fpgrowth

2017-03-15 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17130 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark pull request #17251: [SPARK-19910][SQL] `stack` should not reject NULL...

2017-03-15 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/17251#discussion_r106328813 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/generators.scala --- @@ -156,9 +156,21 @@ case class Stack(children:

[GitHub] spark pull request #17251: [SPARK-19910][SQL] `stack` should not reject NULL...

2017-03-15 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/17251#discussion_r106328573 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/TypeCoercion.scala --- @@ -590,6 +591,23 @@ object TypeCoercion { }

[GitHub] spark issue #17171: [SPARK-19830] [SQL] Add parseTableSchema API to ParserIn...

2017-03-15 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/17171 LGTM --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the

[GitHub] spark issue #17295: [SPARK-19556][core] Do not encrypt block manager data in...

2017-03-15 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/17295 > The penalty comes when transferring that encrypted data from disk. If the data ends up in memory again, it is as efficient as before; but if the evicted block needs to be transferred

[GitHub] spark issue #15363: [SPARK-17791][SQL] Join reordering using star schema det...

2017-03-15 Thread ioana-delaney
Github user ioana-delaney commented on the issue: https://github.com/apache/spark/pull/15363 @gatorsmile @hvanhovell @wzhfy @ron8hu Please let me know if we can move forward with this review. @wzhfy I removed the star-join call from CostBasedJoinReorder until the two are integrated.

[GitHub] spark issue #16905: [SPARK-19567][CORE][SCHEDULER] Support some Schedulable ...

2017-03-15 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16905 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/74629/ Test FAILed. ---

[GitHub] spark issue #16905: [SPARK-19567][CORE][SCHEDULER] Support some Schedulable ...

2017-03-15 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16905 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #16905: [SPARK-19567][CORE][SCHEDULER] Support some Schedulable ...

2017-03-15 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16905 **[Test build #74629 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/74629/testReport)** for PR 16905 at commit

[GitHub] spark issue #17307: [SPARK-13369] Make number of consecutive fetch failures ...

2017-03-15 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17307 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #17307: [SPARK-13369] Make number of consecutive fetch failures ...

2017-03-15 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17307 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/74624/ Test PASSed. ---

[GitHub] spark issue #17307: [SPARK-13369] Make number of consecutive fetch failures ...

2017-03-15 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17307 **[Test build #74624 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/74624/testReport)** for PR 17307 at commit

[GitHub] spark issue #17307: [SPARK-13369] Make number of consecutive fetch failures ...

2017-03-15 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17307 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/74625/ Test PASSed. ---

[GitHub] spark issue #17307: [SPARK-13369] Make number of consecutive fetch failures ...

2017-03-15 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17307 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

  1   2   3   4   5   6   >