[GitHub] spark pull request #15539: [SPARK-17994] [SQL] Add back a file status cache ...

2017-01-07 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/15539#discussion_r95073558 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/FileStatusCache.scala --- @@ -0,0 +1,149 @@ +/* + * Licensed to the

[GitHub] spark pull request #15539: [SPARK-17994] [SQL] Add back a file status cache ...

2017-01-07 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/15539#discussion_r95073551 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/FileStatusCache.scala --- @@ -0,0 +1,149 @@ +/* + * Licensed to the

[GitHub] spark pull request #15539: [SPARK-17994] [SQL] Add back a file status cache ...

2017-01-07 Thread ericl
Github user ericl commented on a diff in the pull request: https://github.com/apache/spark/pull/15539#discussion_r95073488 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/FileStatusCache.scala --- @@ -0,0 +1,149 @@ +/* + * Licensed to the

[GitHub] spark pull request #15539: [SPARK-17994] [SQL] Add back a file status cache ...

2017-01-07 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/15539#discussion_r95073252 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/FileStatusCache.scala --- @@ -0,0 +1,149 @@ +/* + * Licensed to the

[GitHub] spark pull request #16331: [SPARK-18920][HISTORYSERVER]Update outdated date ...

2017-01-07 Thread WangTaoTheTonic
Github user WangTaoTheTonic closed the pull request at: https://github.com/apache/spark/pull/16331 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the

[GitHub] spark issue #16473: [SPARK-19069] [CORE] Expose task 'status' and 'duration'...

2017-01-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16473 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #16473: [SPARK-19069] [CORE] Expose task 'status' and 'duration'...

2017-01-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16473 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/71028/ Test PASSed. ---

[GitHub] spark issue #16473: [SPARK-19069] [CORE] Expose task 'status' and 'duration'...

2017-01-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16473 **[Test build #71028 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/71028/testReport)** for PR 16473 at commit

[GitHub] spark issue #16138: [WIP][SPARK-16609] Add to_date/to_timestamp with format ...

2017-01-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16138 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/71031/ Test FAILed. ---

[GitHub] spark issue #16138: [WIP][SPARK-16609] Add to_date/to_timestamp with format ...

2017-01-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16138 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #16138: [WIP][SPARK-16609] Add to_date/to_timestamp with format ...

2017-01-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16138 **[Test build #71031 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/71031/testReport)** for PR 16138 at commit

[GitHub] spark issue #16138: [WIP][SPARK-16609] Add to_date/to_timestamp with format ...

2017-01-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16138 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/71030/ Test FAILed. ---

[GitHub] spark issue #16138: [WIP][SPARK-16609] Add to_date/to_timestamp with format ...

2017-01-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16138 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #16138: [WIP][SPARK-16609] Add to_date/to_timestamp with format ...

2017-01-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16138 **[Test build #71030 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/71030/testReport)** for PR 16138 at commit

[GitHub] spark issue #16138: [WIP][SPARK-16609] Add to_date/to_timestamp with format ...

2017-01-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16138 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #16138: [WIP][SPARK-16609] Add to_date/to_timestamp with format ...

2017-01-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16138 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/71029/ Test FAILed. ---

[GitHub] spark issue #16138: [WIP][SPARK-16609] Add to_date/to_timestamp with format ...

2017-01-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16138 **[Test build #71029 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/71029/testReport)** for PR 16138 at commit

[GitHub] spark issue #16481: [SPARK-19092] [SQL] Save() API of DataFrameWriter should...

2017-01-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16481 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/71026/ Test PASSed. ---

[GitHub] spark issue #16481: [SPARK-19092] [SQL] Save() API of DataFrameWriter should...

2017-01-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16481 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #16481: [SPARK-19092] [SQL] Save() API of DataFrameWriter should...

2017-01-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16481 **[Test build #71026 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/71026/testReport)** for PR 16481 at commit

[GitHub] spark issue #16493: [SPARK-19093][SQL] Cached tables are not used in Subquer...

2017-01-07 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/16493 Also cc @rxin @cloud-fan --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and

[GitHub] spark issue #16493: [SPARK-19093][SQL] Cached tables are not used in Subquer...

2017-01-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16493 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/71027/ Test PASSed. ---

[GitHub] spark issue #16493: [SPARK-19093][SQL] Cached tables are not used in Subquer...

2017-01-07 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/16493 LGTM --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the

[GitHub] spark issue #16493: [SPARK-19093][SQL] Cached tables are not used in Subquer...

2017-01-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16493 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #16493: [SPARK-19093][SQL] Cached tables are not used in Subquer...

2017-01-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16493 **[Test build #71027 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/71027/testReport)** for PR 16493 at commit

[GitHub] spark issue #16138: [WIP][SPARK-16609] Add to_date/to_timestamp with format ...

2017-01-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16138 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #16138: [WIP][SPARK-16609] Add to_date/to_timestamp with format ...

2017-01-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16138 **[Test build #71032 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/71032/testReport)** for PR 16138 at commit

[GitHub] spark issue #16138: [WIP][SPARK-16609] Add to_date/to_timestamp with format ...

2017-01-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16138 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/71032/ Test FAILed. ---

[GitHub] spark issue #16496: [SPARK-16101][SQL] Refactoring CSV write path to be cons...

2017-01-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16496 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #16138: [WIP][SPARK-16609] Add to_date/to_timestamp with format ...

2017-01-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16138 **[Test build #71032 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/71032/testReport)** for PR 16138 at commit

[GitHub] spark issue #16496: [SPARK-16101][SQL] Refactoring CSV write path to be cons...

2017-01-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16496 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/71025/ Test FAILed. ---

[GitHub] spark issue #16138: [WIP][SPARK-16609] Add to_date/to_timestamp with format ...

2017-01-07 Thread anabranch
Github user anabranch commented on the issue: https://github.com/apache/spark/pull/16138 I believe now why my previous implementation did not work. My implementation originally looked like this: ```scala case class ParseToTimestamp(left: Expression, format:

[GitHub] spark issue #16496: [SPARK-16101][SQL] Refactoring CSV write path to be cons...

2017-01-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16496 **[Test build #71025 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/71025/testReport)** for PR 16496 at commit

[GitHub] spark pull request #16138: [WIP][SPARK-16609] Add to_date/to_timestamp with ...

2017-01-07 Thread anabranch
Github user anabranch commented on a diff in the pull request: https://github.com/apache/spark/pull/16138#discussion_r95071556 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/datetimeExpressions.scala --- @@ -1047,6 +1048,60 @@ case class

[GitHub] spark pull request #16138: [WIP][SPARK-16609] Add to_date/to_timestamp with ...

2017-01-07 Thread anabranch
Github user anabranch commented on a diff in the pull request: https://github.com/apache/spark/pull/16138#discussion_r95071450 --- Diff: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/expressions/DateExpressionsSuite.scala --- @@ -389,6 +389,20 @@ class

[GitHub] spark pull request #16138: [WIP][SPARK-16609] Add to_date/to_timestamp with ...

2017-01-07 Thread anabranch
Github user anabranch commented on a diff in the pull request: https://github.com/apache/spark/pull/16138#discussion_r95071452 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/FunctionRegistry.scala --- @@ -342,7 +342,8 @@ object FunctionRegistry {

[GitHub] spark pull request #16138: [WIP][SPARK-16609] Add to_date/to_timestamp with ...

2017-01-07 Thread anabranch
Github user anabranch commented on a diff in the pull request: https://github.com/apache/spark/pull/16138#discussion_r95071447 --- Diff: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/expressions/DateExpressionsSuite.scala --- @@ -389,6 +389,20 @@ class

[GitHub] spark pull request #16138: [WIP][SPARK-16609] Add to_date/to_timestamp with ...

2017-01-07 Thread anabranch
Github user anabranch commented on a diff in the pull request: https://github.com/apache/spark/pull/16138#discussion_r95071721 --- Diff: python/pyspark/sql/functions.py --- @@ -143,6 +143,12 @@ def _(): 'measured in radians.', } +_functions_2_2 =

[GitHub] spark pull request #16487: [SPARK-19107][SQL] support creating hive table wi...

2017-01-07 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/16487#discussion_r95071633 --- Diff: sql/hive/src/test/scala/org/apache/spark/sql/hive/execution/HiveDDLSuite.scala --- @@ -1289,4 +1290,66 @@ class HiveDDLSuite }

[GitHub] spark pull request #16487: [SPARK-19107][SQL] support creating hive table wi...

2017-01-07 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/16487#discussion_r95071626 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/DataFrameWriter.scala --- @@ -385,6 +380,8 @@ final class DataFrameWriter[T] private[sql](ds:

[GitHub] spark issue #16138: [WIP][SPARK-16609] Add to_date/to_timestamp with format ...

2017-01-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16138 **[Test build #71031 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/71031/testReport)** for PR 16138 at commit

[GitHub] spark issue #16138: [WIP][SPARK-16609] Add to_date/to_timestamp with format ...

2017-01-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16138 **[Test build #71030 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/71030/testReport)** for PR 16138 at commit

[GitHub] spark issue #16138: [WIP][SPARK-16609] Add to_date/to_timestamp with format ...

2017-01-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16138 **[Test build #71029 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/71029/testReport)** for PR 16138 at commit

[GitHub] spark issue #16473: [SPARK-19069] [CORE] Expose task 'status' and 'duration'...

2017-01-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16473 **[Test build #71028 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/71028/testReport)** for PR 16473 at commit

[GitHub] spark issue #16249: [SPARK-18828][SPARKR] Refactor scripts for R

2017-01-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16249 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/71024/ Test PASSed. ---

[GitHub] spark issue #16249: [SPARK-18828][SPARKR] Refactor scripts for R

2017-01-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16249 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #16249: [SPARK-18828][SPARKR] Refactor scripts for R

2017-01-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16249 **[Test build #71024 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/71024/testReport)** for PR 16249 at commit

[GitHub] spark issue #16493: [SPARK-19093][SQL] Cached tables are not used in Subquer...

2017-01-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16493 **[Test build #71027 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/71027/testReport)** for PR 16493 at commit

[GitHub] spark issue #16500: [SPARK-19120] [SPARK-19121] Refresh Metadata Cache After...

2017-01-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16500 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/71023/ Test PASSed. ---

[GitHub] spark issue #16500: [SPARK-19120] [SPARK-19121] Refresh Metadata Cache After...

2017-01-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16500 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #16493: [SPARK-19093][SQL] Cached tables are not used in Subquer...

2017-01-07 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/16493 test this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so,

[GitHub] spark issue #16500: [SPARK-19120] [SPARK-19121] Refresh Metadata Cache After...

2017-01-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16500 **[Test build #71023 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/71023/testReport)** for PR 16500 at commit

[GitHub] spark pull request #16400: [SPARK-18941][SQL][DOC] Add a new behavior docume...

2017-01-07 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/16400 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark issue #16400: [SPARK-18941][SQL][DOC] Add a new behavior document on `...

2017-01-07 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/16400 Thanks! Merging to master/2.1/2.0 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #16481: [SPARK-19092] [SQL] Save() API of DataFrameWriter should...

2017-01-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16481 **[Test build #71026 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/71026/testReport)** for PR 16481 at commit

[GitHub] spark issue #16493: [SPARK-19093][SQL] Cached tables are not used in Subquer...

2017-01-07 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/16493 retest this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes

[GitHub] spark pull request #16481: [SPARK-19092] [SQL] Save() API of DataFrameWriter...

2017-01-07 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/16481#discussion_r95070345 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/DataSource.scala --- @@ -494,8 +500,13 @@ case class DataSource(

[GitHub] spark issue #16496: [SPARK-16101][SQL] Refactoring CSV write path to be cons...

2017-01-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16496 **[Test build #71025 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/71025/testReport)** for PR 16496 at commit

[GitHub] spark issue #16496: [SPARK-16101][SQL] Refactoring CSV write path to be cons...

2017-01-07 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/16496 retest this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes

[GitHub] spark issue #16249: [SPARK-18828][SPARKR] Refactor scripts for R

2017-01-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16249 **[Test build #71024 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/71024/testReport)** for PR 16249 at commit

[GitHub] spark pull request #16500: [SPARK-19120] [SPARK-19121] Refresh Metadata Cach...

2017-01-07 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/16500#discussion_r95068993 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/execution/InsertIntoHiveTable.scala --- @@ -392,7 +392,9 @@ case class InsertIntoHiveTable(

[GitHub] spark issue #16500: [SPARK-19120] [SPARK-19121] Refresh Metadata Cache After...

2017-01-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16500 **[Test build #71023 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/71023/testReport)** for PR 16500 at commit

[GitHub] spark issue #16500: [SPARK-19120] [SPARK-19121] Refresh Metadata Cache After...

2017-01-07 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/16500 cc @ericl @cloud-fan @mallman The actual code changes are just two lines. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as

[GitHub] spark issue #16312: [SPARK-18862][SPARKR][ML] Split SparkR mllib.R into mult...

2017-01-07 Thread felixcheung
Github user felixcheung commented on the issue: https://github.com/apache/spark/pull/16312 LGTM. thanks! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so,

[GitHub] spark pull request #16500: [SPARK-19120] [SPARK-19121] Refresh Metadata Cach...

2017-01-07 Thread gatorsmile
GitHub user gatorsmile opened a pull request: https://github.com/apache/spark/pull/16500 [SPARK-19120] [SPARK-19121] Refresh Metadata Cache After Load Partitioned Hive Tables ### What changes were proposed in this pull request? ```Scala sql("CREATE TABLE tab (a

[GitHub] spark pull request #16312: [SPARK-18862][SPARKR][ML] Split SparkR mllib.R in...

2017-01-07 Thread felixcheung
Github user felixcheung commented on a diff in the pull request: https://github.com/apache/spark/pull/16312#discussion_r95068898 --- Diff: R/pkg/R/mllib_utils.R --- @@ -81,9 +80,11 @@ predict_internal <- function(object, newData) { #' model <- read.ml(path) #' } #'

[GitHub] spark issue #16499: [SPARK-17204][CORE] Fix replicated off heap storage

2017-01-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16499 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/71022/ Test PASSed. ---

[GitHub] spark issue #16499: [SPARK-17204][CORE] Fix replicated off heap storage

2017-01-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16499 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #16499: [SPARK-17204][CORE] Fix replicated off heap storage

2017-01-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16499 **[Test build #71022 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/71022/testReport)** for PR 16499 at commit

[GitHub] spark issue #16473: [SPARK-19069] [CORE] Expose task 'status' and 'duration'...

2017-01-07 Thread paragpc
Github user paragpc commented on the issue: https://github.com/apache/spark/pull/16473 ```[error] * method this(Long,Int,Int,java.util.Date,java.lang.String,java.lang.String,java.lang.String,Boolean,scala.collection.Seq,scala.Option,scala.Option)Unit in class

[GitHub] spark issue #16498: SPARK-9487][Tests] Begin the work of robustifying unit t...

2017-01-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16498 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #16498: SPARK-9487][Tests] Begin the work of robustifying unit t...

2017-01-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16498 **[Test build #71021 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/71021/testReport)** for PR 16498 at commit

[GitHub] spark issue #16498: SPARK-9487][Tests] Begin the work of robustifying unit t...

2017-01-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16498 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/71021/ Test PASSed. ---

[GitHub] spark pull request #16499: [SPARK-17204][CORE] Fix replicated off heap stora...

2017-01-07 Thread mallman
Github user mallman commented on a diff in the pull request: https://github.com/apache/spark/pull/16499#discussion_r95066452 --- Diff: core/src/test/scala/org/apache/spark/storage/BlockManagerReplicationSuite.scala --- @@ -375,7 +375,8 @@ class BlockManagerReplicationSuite

[GitHub] spark issue #16499: [SPARK-17204][CORE] Fix replicated off heap storage

2017-01-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16499 **[Test build #71022 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/71022/testReport)** for PR 16499 at commit

[GitHub] spark pull request #16499: [SPARK-17204][CORE] Fix replicated off heap stora...

2017-01-07 Thread mallman
Github user mallman commented on a diff in the pull request: https://github.com/apache/spark/pull/16499#discussion_r95066296 --- Diff: core/src/test/scala/org/apache/spark/storage/BlockManagerReplicationSuite.scala --- @@ -387,12 +388,23 @@ class BlockManagerReplicationSuite

[GitHub] spark pull request #16499: [SPARK-17204][CORE] Fix replicated off heap stora...

2017-01-07 Thread mallman
GitHub user mallman opened a pull request: https://github.com/apache/spark/pull/16499 [SPARK-17204][CORE] Fix replicated off heap storage (Jira: https://issues.apache.org/jira/browse/SPARK-17204) ## What changes were proposed in this pull request? There are a

[GitHub] spark issue #15496: [SPARK-17950] [Python] Match SparseVector behavior with ...

2017-01-07 Thread itg-abby
Github user itg-abby commented on the issue: https://github.com/apache/spark/pull/15496 ping @holdenk , please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes

[GitHub] spark pull request #16481: [SPARK-19092] [SQL] Save() API of DataFrameWriter...

2017-01-07 Thread jaceklaskowski
Github user jaceklaskowski commented on a diff in the pull request: https://github.com/apache/spark/pull/16481#discussion_r95065596 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/DataSource.scala --- @@ -494,8 +500,13 @@ case class DataSource(

[GitHub] spark pull request #16492: [SPARK-19113][SS][Tests]Set UncaughtExceptionHand...

2017-01-07 Thread jaceklaskowski
Github user jaceklaskowski commented on a diff in the pull request: https://github.com/apache/spark/pull/16492#discussion_r95065464 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/streaming/StreamTest.scala --- @@ -235,7 +235,10 @@ trait StreamTest extends QueryTest with

[GitHub] spark pull request #16492: [SPARK-19113][SS][Tests]Set UncaughtExceptionHand...

2017-01-07 Thread jaceklaskowski
Github user jaceklaskowski commented on a diff in the pull request: https://github.com/apache/spark/pull/16492#discussion_r95065415 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/streaming/StreamSuite.scala --- @@ -238,7 +238,7 @@ class StreamSuite extends StreamTest {

[GitHub] spark issue #16496: [SPARK-16101][SQL] Refactoring CSV write path to be cons...

2017-01-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16496 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #16496: [SPARK-16101][SQL] Refactoring CSV write path to be cons...

2017-01-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16496 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/71019/ Test FAILed. ---

[GitHub] spark issue #16496: [SPARK-16101][SQL] Refactoring CSV write path to be cons...

2017-01-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16496 **[Test build #71019 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/71019/testReport)** for PR 16496 at commit

[GitHub] spark pull request #13818: [SPARK-15968][SQL] Nonempty partitioned metastore...

2017-01-07 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/13818#discussion_r95064603 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/execution/InsertIntoHiveTable.scala --- @@ -298,6 +298,7 @@ case class InsertIntoHiveTable(

[GitHub] spark pull request #13818: [SPARK-15968][SQL] Nonempty partitioned metastore...

2017-01-07 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/13818#discussion_r95064489 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/execution/InsertIntoHiveTable.scala --- @@ -298,6 +298,7 @@ case class InsertIntoHiveTable(

[GitHub] spark pull request #15413: [SPARK-17847][ML] Reduce shuffled data size of Ga...

2017-01-07 Thread sethah
Github user sethah commented on a diff in the pull request: https://github.com/apache/spark/pull/15413#discussion_r95064061 --- Diff: mllib/src/test/scala/org/apache/spark/ml/clustering/GaussianMixtureSuite.scala --- @@ -126,9 +143,93 @@ class GaussianMixtureSuite extends

[GitHub] spark pull request #15413: [SPARK-17847][ML] Reduce shuffled data size of Ga...

2017-01-07 Thread sethah
Github user sethah commented on a diff in the pull request: https://github.com/apache/spark/pull/15413#discussion_r95064361 --- Diff: mllib/src/test/scala/org/apache/spark/ml/clustering/GaussianMixtureSuite.scala --- @@ -141,4 +242,37 @@ object GaussianMixtureSuite {

[GitHub] spark pull request #15413: [SPARK-17847][ML] Reduce shuffled data size of Ga...

2017-01-07 Thread sethah
Github user sethah commented on a diff in the pull request: https://github.com/apache/spark/pull/15413#discussion_r95064130 --- Diff: mllib/src/test/scala/org/apache/spark/ml/clustering/GaussianMixtureSuite.scala --- @@ -126,9 +143,93 @@ class GaussianMixtureSuite extends

[GitHub] spark issue #16498: SPARK-9487][Tests] Begin the work of robustifying unit t...

2017-01-07 Thread skanjila
Github user skanjila commented on the issue: https://github.com/apache/spark/pull/16498 Whats different is that this time around I am only making one change at a time (in this case ContextCleanerSuite), fixing the unit tests by backtracking from the Jenkins failures. In this

[GitHub] spark pull request #13818: [SPARK-15968][SQL] Nonempty partitioned metastore...

2017-01-07 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/13818#discussion_r95064132 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/execution/InsertIntoHiveTable.scala --- @@ -298,6 +298,7 @@ case class InsertIntoHiveTable(

[GitHub] spark issue #16490: [SPARK-19106][DOCS] Styling for the configuration docs i...

2017-01-07 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/16490 Merged to master/2.1 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so,

[GitHub] spark issue #15770: [SPARK-15784][ML]:Add Power Iteration Clustering to spar...

2017-01-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15770 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/71020/ Test PASSed. ---

[GitHub] spark issue #15770: [SPARK-15784][ML]:Add Power Iteration Clustering to spar...

2017-01-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15770 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #15770: [SPARK-15784][ML]:Add Power Iteration Clustering to spar...

2017-01-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15770 **[Test build #71020 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/71020/testReport)** for PR 15770 at commit

[GitHub] spark pull request #16490: [SPARK-19106][DOCS] Styling for the configuration...

2017-01-07 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/16490 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark pull request #16491: [SPARK-19110][ML][MLLIB]:DistributedLDAModel retu...

2017-01-07 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/16491 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark pull request #16233: [SPARK-18801][SQL] Support resolve a nested view

2017-01-07 Thread hvanhovell
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/16233#discussion_r95061617 --- Diff: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/catalog/SessionCatalogSuite.scala --- @@ -465,6 +466,48 @@ class SessionCatalogSuite

[GitHub] spark pull request #16233: [SPARK-18801][SQL] Support resolve a nested view

2017-01-07 Thread hvanhovell
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/16233#discussion_r95061594 --- Diff: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/catalog/ExternalCatalogSuite.scala --- @@ -860,6 +864,25 @@ abstract class

[GitHub] spark pull request #16233: [SPARK-18801][SQL] Support resolve a nested view

2017-01-07 Thread hvanhovell
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/16233#discussion_r95061253 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala --- @@ -767,7 +857,45 @@ class Analyzer( } }

  1   2   3   >