[GitHub] spark issue #16294: [SPARK-18669][SS][DOCS] Update Apache docs for Structure...

2016-12-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16294 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #16294: [SPARK-18669][SS][DOCS] Update Apache docs for Structure...

2016-12-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16294 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/70529/ Test PASSed. ---

[GitHub] spark pull request #15996: [SPARK-18567][SQL] Simplify CreateDataSourceTable...

2016-12-22 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/15996#discussion_r93722231 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/DataFrameWriter.scala --- @@ -364,48 +366,162 @@ final class DataFrameWriter[T] private[sql](ds: Da

[GitHub] spark issue #16294: [SPARK-18669][SS][DOCS] Update Apache docs for Structure...

2016-12-22 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16294 **[Test build #70529 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/70529/testReport)** for PR 16294 at commit [`0cc1dd6`](https://github.com/apache/spark/commit/0

[GitHub] spark pull request #15996: [SPARK-18567][SQL] Simplify CreateDataSourceTable...

2016-12-22 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/15996#discussion_r93722074 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/DataFrameWriter.scala --- @@ -364,48 +366,162 @@ final class DataFrameWriter[T] private[sql](ds: Da

[GitHub] spark pull request #16294: [SPARK-18669][SS][DOCS] Update Apache docs for St...

2016-12-22 Thread tdas
Github user tdas commented on a diff in the pull request: https://github.com/apache/spark/pull/16294#discussion_r93722076 --- Diff: docs/structured-streaming-programming-guide.md --- @@ -1493,13 +1493,13 @@ SparkSession spark = ... spark.streams.addListener(new StreamingQ

[GitHub] spark pull request #13909: [SPARK-16213][SQL] Reduce runtime overhead of a p...

2016-12-22 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/13909#discussion_r93721954 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/complexTypeCreator.scala --- @@ -56,33 +58,100 @@ case class CreateArray(c

[GitHub] spark pull request #16294: [SPARK-18669][SS][DOCS] Update Apache docs for St...

2016-12-22 Thread zsxwing
Github user zsxwing commented on a diff in the pull request: https://github.com/apache/spark/pull/16294#discussion_r93721337 --- Diff: docs/structured-streaming-programming-guide.md --- @@ -1493,13 +1493,13 @@ SparkSession spark = ... spark.streams.addListener(new Streami

[GitHub] spark issue #16294: [SPARK-18669][SS][DOCS] Update Apache docs for Structure...

2016-12-22 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16294 **[Test build #70529 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/70529/testReport)** for PR 16294 at commit [`0cc1dd6`](https://github.com/apache/spark/commit/0c

[GitHub] spark pull request #13909: [SPARK-16213][SQL] Reduce runtime overhead of a p...

2016-12-22 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/13909#discussion_r93720959 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/complexTypeCreator.scala --- @@ -56,33 +58,100 @@ case class CreateArray(child

[GitHub] spark pull request #15996: [SPARK-18567][SQL] Simplify CreateDataSourceTable...

2016-12-22 Thread yhuai
Github user yhuai commented on a diff in the pull request: https://github.com/apache/spark/pull/15996#discussion_r93720714 --- Diff: sql/hive/src/test/scala/org/apache/spark/sql/hive/PartitionProviderCompatibilitySuite.scala --- @@ -195,12 +195,25 @@ class PartitionProviderCompati

[GitHub] spark pull request #15996: [SPARK-18567][SQL] Simplify CreateDataSourceTable...

2016-12-22 Thread yhuai
Github user yhuai commented on a diff in the pull request: https://github.com/apache/spark/pull/15996#discussion_r93720687 --- Diff: sql/hive/src/test/scala/org/apache/spark/sql/hive/PartitionProviderCompatibilitySuite.scala --- @@ -195,12 +195,25 @@ class PartitionProviderCompati

[GitHub] spark pull request #15996: [SPARK-18567][SQL] Simplify CreateDataSourceTable...

2016-12-22 Thread yhuai
Github user yhuai commented on a diff in the pull request: https://github.com/apache/spark/pull/15996#discussion_r93720630 --- Diff: sql/hive/src/test/scala/org/apache/spark/sql/hive/PartitionProviderCompatibilitySuite.scala --- @@ -195,12 +195,25 @@ class PartitionProviderCompati

[GitHub] spark issue #16337: [SPARK-18871][SQL] New test cases for IN/NOT IN subquery

2016-12-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16337 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #16337: [SPARK-18871][SQL] New test cases for IN/NOT IN subquery

2016-12-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16337 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/70528/ Test FAILed. ---

[GitHub] spark pull request #15996: [SPARK-18567][SQL] Simplify CreateDataSourceTable...

2016-12-22 Thread yhuai
Github user yhuai commented on a diff in the pull request: https://github.com/apache/spark/pull/15996#discussion_r93720613 --- Diff: sql/hive/src/test/scala/org/apache/spark/sql/hive/PartitionProviderCompatibilitySuite.scala --- @@ -195,12 +195,25 @@ class PartitionProviderCompati

[GitHub] spark issue #16337: [SPARK-18871][SQL] New test cases for IN/NOT IN subquery

2016-12-22 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16337 **[Test build #70528 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/70528/testReport)** for PR 16337 at commit [`1c1900a`](https://github.com/apache/spark/commit/1

[GitHub] spark pull request #15996: [SPARK-18567][SQL] Simplify CreateDataSourceTable...

2016-12-22 Thread yhuai
Github user yhuai commented on a diff in the pull request: https://github.com/apache/spark/pull/15996#discussion_r93720521 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/command/createDataSourceTables.scala --- @@ -140,153 +140,55 @@ case class CreateDataSourceTa

[GitHub] spark pull request #15996: [SPARK-18567][SQL] Simplify CreateDataSourceTable...

2016-12-22 Thread yhuai
Github user yhuai commented on a diff in the pull request: https://github.com/apache/spark/pull/15996#discussion_r93720313 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/DataFrameWriter.scala --- @@ -363,48 +365,125 @@ final class DataFrameWriter[T] private[sql](ds: Datase

[GitHub] spark pull request #15996: [SPARK-18567][SQL] Simplify CreateDataSourceTable...

2016-12-22 Thread yhuai
Github user yhuai commented on a diff in the pull request: https://github.com/apache/spark/pull/15996#discussion_r93720195 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/DataFrameWriter.scala --- @@ -364,48 +366,162 @@ final class DataFrameWriter[T] private[sql](ds: Datase

[GitHub] spark pull request #15996: [SPARK-18567][SQL] Simplify CreateDataSourceTable...

2016-12-22 Thread yhuai
Github user yhuai commented on a diff in the pull request: https://github.com/apache/spark/pull/15996#discussion_r93719938 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/DataFrameWriter.scala --- @@ -364,48 +366,162 @@ final class DataFrameWriter[T] private[sql](ds: Datase

[GitHub] spark pull request #15996: [SPARK-18567][SQL] Simplify CreateDataSourceTable...

2016-12-22 Thread yhuai
Github user yhuai commented on a diff in the pull request: https://github.com/apache/spark/pull/15996#discussion_r93719624 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/test/DataFrameReaderWriterSuite.scala --- @@ -635,4 +638,13 @@ class DataFrameReaderWriterSuite extends

[GitHub] spark pull request #15996: [SPARK-18567][SQL] Simplify CreateDataSourceTable...

2016-12-22 Thread yhuai
Github user yhuai commented on a diff in the pull request: https://github.com/apache/spark/pull/15996#discussion_r93718921 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/test/DataFrameReaderWriterSuite.scala --- @@ -635,4 +638,13 @@ class DataFrameReaderWriterSuite extends

[GitHub] spark pull request #16294: [SPARK-18669][SS][DOCS] Update Apache docs for St...

2016-12-22 Thread tdas
Github user tdas commented on a diff in the pull request: https://github.com/apache/spark/pull/16294#discussion_r93718406 --- Diff: docs/structured-streaming-programming-guide.md --- @@ -817,7 +997,14 @@ Here is a table of all the sinks, and the corresponding settings. wri

[GitHub] spark pull request #16294: [SPARK-18669][SS][DOCS] Update Apache docs for St...

2016-12-22 Thread tdas
Github user tdas commented on a diff in the pull request: https://github.com/apache/spark/pull/16294#discussion_r93718420 --- Diff: docs/structured-streaming-programming-guide.md --- @@ -763,16 +881,78 @@ returned through `Dataset.writeStream()`. You will have to specify one or mor

[GitHub] spark pull request #16380: [SPARK-18972][Core]Fix the netty thread names for...

2016-12-22 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/16380 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is ena

[GitHub] spark issue #16311: [SPARK-17807][core] split test-tags into test-JAR

2016-12-22 Thread vanzin
Github user vanzin commented on the issue: https://github.com/apache/spark/pull/16311 Merged to 2.0 also (after a trivial conflict resolution). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request #16385: [SPARK-18985][SS]Add missing @InterfaceStability....

2016-12-22 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/16385 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is ena

[GitHub] spark issue #16380: [SPARK-18972][Core]Fix the netty thread names for RPC

2016-12-22 Thread zsxwing
Github user zsxwing commented on the issue: https://github.com/apache/spark/pull/16380 Thanks! Merged to master, 2.1 and 2.0. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enab

[GitHub] spark pull request #16294: [SPARK-18669][SS][DOCS] Update Apache docs for St...

2016-12-22 Thread zsxwing
Github user zsxwing commented on a diff in the pull request: https://github.com/apache/spark/pull/16294#discussion_r93717198 --- Diff: docs/structured-streaming-programming-guide.md --- @@ -1089,46 +1276,94 @@ spark.streams().awaitAnyTermination() # block until any one of them ter

[GitHub] spark pull request #16294: [SPARK-18669][SS][DOCS] Update Apache docs for St...

2016-12-22 Thread zsxwing
Github user zsxwing commented on a diff in the pull request: https://github.com/apache/spark/pull/16294#discussion_r93717171 --- Diff: docs/structured-streaming-programming-guide.md --- @@ -1138,78 +1373,91 @@ Status of query 'queryName' {% highlight java %} StreamingQuery

[GitHub] spark pull request #16294: [SPARK-18669][SS][DOCS] Update Apache docs for St...

2016-12-22 Thread zsxwing
Github user zsxwing commented on a diff in the pull request: https://github.com/apache/spark/pull/16294#discussion_r93716944 --- Diff: docs/structured-streaming-programming-guide.md --- @@ -1245,15 +1492,14 @@ spark.streams.addListener(new StreamingQueryListener() { SparkSessi

[GitHub] spark pull request #16294: [SPARK-18669][SS][DOCS] Update Apache docs for St...

2016-12-22 Thread zsxwing
Github user zsxwing commented on a diff in the pull request: https://github.com/apache/spark/pull/16294#discussion_r93717344 --- Diff: docs/structured-streaming-programming-guide.md --- @@ -763,16 +881,78 @@ returned through `Dataset.writeStream()`. You will have to specify one or

[GitHub] spark pull request #16294: [SPARK-18669][SS][DOCS] Update Apache docs for St...

2016-12-22 Thread zsxwing
Github user zsxwing commented on a diff in the pull request: https://github.com/apache/spark/pull/16294#discussion_r93717005 --- Diff: docs/structured-streaming-programming-guide.md --- @@ -1225,15 +1473,14 @@ stopped and when there is progress made in an active query. Here is an e

[GitHub] spark issue #16385: [SPARK-18985][SS]Add missing @InterfaceStability.Evolvin...

2016-12-22 Thread zsxwing
Github user zsxwing commented on the issue: https://github.com/apache/spark/pull/16385 Thanks. Merging to master and 2.1. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled

[GitHub] spark pull request #16294: [SPARK-18669][SS][DOCS] Update Apache docs for St...

2016-12-22 Thread zsxwing
Github user zsxwing commented on a diff in the pull request: https://github.com/apache/spark/pull/16294#discussion_r93716901 --- Diff: docs/structured-streaming-programming-guide.md --- @@ -1245,15 +1492,14 @@ spark.streams.addListener(new StreamingQueryListener() { SparkSessi

[GitHub] spark pull request #16294: [SPARK-18669][SS][DOCS] Update Apache docs for St...

2016-12-22 Thread zsxwing
Github user zsxwing commented on a diff in the pull request: https://github.com/apache/spark/pull/16294#discussion_r93717114 --- Diff: docs/structured-streaming-programming-guide.md --- @@ -1138,78 +1373,91 @@ Status of query 'queryName' {% highlight java %} StreamingQuery

[GitHub] spark pull request #16294: [SPARK-18669][SS][DOCS] Update Apache docs for St...

2016-12-22 Thread zsxwing
Github user zsxwing commented on a diff in the pull request: https://github.com/apache/spark/pull/16294#discussion_r93716922 --- Diff: docs/structured-streaming-programming-guide.md --- @@ -1245,15 +1492,14 @@ spark.streams.addListener(new StreamingQueryListener() { SparkSessi

[GitHub] spark pull request #16294: [SPARK-18669][SS][DOCS] Update Apache docs for St...

2016-12-22 Thread zsxwing
Github user zsxwing commented on a diff in the pull request: https://github.com/apache/spark/pull/16294#discussion_r93717360 --- Diff: docs/structured-streaming-programming-guide.md --- @@ -817,7 +997,14 @@ Here is a table of all the sinks, and the corresponding settings.

[GitHub] spark pull request #16294: [SPARK-18669][SS][DOCS] Update Apache docs for St...

2016-12-22 Thread zsxwing
Github user zsxwing commented on a diff in the pull request: https://github.com/apache/spark/pull/16294#discussion_r93717130 --- Diff: docs/structured-streaming-programming-guide.md --- @@ -1138,78 +1373,91 @@ Status of query 'queryName' {% highlight java %} StreamingQuery

[GitHub] spark issue #16380: [SPARK-18972][Core]Fix the netty thread names for RPC

2016-12-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16380 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #16380: [SPARK-18972][Core]Fix the netty thread names for RPC

2016-12-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16380 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/70527/ Test PASSed. ---

[GitHub] spark issue #16380: [SPARK-18972][Core]Fix the netty thread names for RPC

2016-12-22 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16380 **[Test build #70527 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/70527/testReport)** for PR 16380 at commit [`5b51040`](https://github.com/apache/spark/commit/5

[GitHub] spark issue #16385: [SPARK-18985][SS]Add missing @InterfaceStability.Evolvin...

2016-12-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16385 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/70526/ Test PASSed. ---

[GitHub] spark issue #16385: [SPARK-18985][SS]Add missing @InterfaceStability.Evolvin...

2016-12-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16385 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #16385: [SPARK-18985][SS]Add missing @InterfaceStability.Evolvin...

2016-12-22 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16385 **[Test build #70526 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/70526/testReport)** for PR 16385 at commit [`291127f`](https://github.com/apache/spark/commit/2

[GitHub] spark issue #16384: [BUILD] make-distribution support alternate python to bu...

2016-12-22 Thread holdenk
Github user holdenk commented on the issue: https://github.com/apache/spark/pull/16384 Sure, just make the virtualenv a Python3 virtualenv. We do this in the shell script for testing pip installability. On Thu, Dec 22, 2016 at 3:47 PM Felix Cheung wrote: > Pos

[GitHub] spark issue #16384: [BUILD] make-distribution support alternate python to bu...

2016-12-22 Thread felixcheung
Github user felixcheung commented on the issue: https://github.com/apache/spark/pull/16384 Nm. python3 -m venv does not support -p. I can see it works with virtualenv. I'll close this PR. Thanks --- If your project is set up for it, you can reply to this email and h

[GitHub] spark issue #16311: [SPARK-17807][core] split test-tags into test-JAR

2016-12-22 Thread vanzin
Github user vanzin commented on the issue: https://github.com/apache/spark/pull/16311 Builds look good as far as I can tell, so I'm also merging this into 2.1. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your proje

[GitHub] spark issue #16384: [BUILD] make-distribution support alternate python to bu...

2016-12-22 Thread felixcheung
Github user felixcheung commented on the issue: https://github.com/apache/spark/pull/16384 Possibly ;) Is there a way to get it to use python3? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project doe

[GitHub] spark pull request #16308: [SPARK-18936][SQL] Infrastructure for session loc...

2016-12-22 Thread hvanhovell
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/16308#discussion_r93714338 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/QueryExecution.scala --- @@ -146,18 +146,20 @@ class QueryExecution(val sparkSession: Spa

[GitHub] spark pull request #16308: [SPARK-18936][SQL] Infrastructure for session loc...

2016-12-22 Thread hvanhovell
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/16308#discussion_r93713654 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/datetimeExpressions.scala --- @@ -78,11 +108,21 @@ case class CurrentTime

[GitHub] spark pull request #16308: [SPARK-18936][SQL] Infrastructure for session loc...

2016-12-22 Thread hvanhovell
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/16308#discussion_r93712892 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/Cast.scala --- @@ -135,6 +167,15 @@ case class Cast(child: Expression, da

[GitHub] spark pull request #16308: [SPARK-18936][SQL] Infrastructure for session loc...

2016-12-22 Thread hvanhovell
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/16308#discussion_r93714063 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/datetimeExpressions.scala --- @@ -78,11 +108,21 @@ case class CurrentTime

[GitHub] spark pull request #16308: [SPARK-18936][SQL] Infrastructure for session loc...

2016-12-22 Thread hvanhovell
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/16308#discussion_r93541632 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala --- @@ -104,6 +104,7 @@ class Analyzer( ResolveAgg

[GitHub] spark pull request #16308: [SPARK-18936][SQL] Infrastructure for session loc...

2016-12-22 Thread hvanhovell
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/16308#discussion_r93714774 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/FileFormatWriter.scala --- @@ -288,7 +289,7 @@ object FileFormatWriter exten

[GitHub] spark issue #12420: [SPARK-14585][ML][WIP] Provide accessor methods for Pipe...

2016-12-22 Thread jkbradley
Github user jkbradley commented on the issue: https://github.com/apache/spark/pull/12420 @pravingadakh Will you be able to continue with this? Thanks! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does

[GitHub] spark issue #16294: [SPARK-18669][SS][DOCS] Update Apache docs for Structure...

2016-12-22 Thread tdas
Github user tdas commented on the issue: https://github.com/apache/spark/pull/16294 ^^ The last failure on build 70524 is for a older commit. I reverted the culprit change and build 70525 passed. --- If your project is set up for it, you can reply to this email and have your reply ap

[GitHub] spark issue #16337: [SPARK-18871][SQL] New test cases for IN/NOT IN subquery

2016-12-22 Thread kevinyu98
Github user kevinyu98 commented on the issue: https://github.com/apache/spark/pull/16337 Hello Nat, thanks for the result checking and detail explanation. Based on your assessment, I have comment out 4 test cases for now, pending on the investigation of those open JIRAs. --- If yo

[GitHub] spark issue #16337: [SPARK-18871][SQL] New test cases for IN/NOT IN subquery

2016-12-22 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16337 **[Test build #70528 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/70528/testReport)** for PR 16337 at commit [`1c1900a`](https://github.com/apache/spark/commit/1c

[GitHub] spark issue #16233: [SPARK-18801][SQL] Add `View` operator to help resolve a...

2016-12-22 Thread hvanhovell
Github user hvanhovell commented on the issue: https://github.com/apache/spark/pull/16233 @nsyca we actually support a 2-part name (look at TableIdentifier). You don't have to define the database. We have the concept of a `currentDatabase` in the `SessionCatalog`, and use that to reso

[GitHub] spark issue #16370: [SPARK-18960][SQL][SS] Avoid double reading file which i...

2016-12-22 Thread zsxwing
Github user zsxwing commented on the issue: https://github.com/apache/spark/pull/16370 A similar PR got refused as this is not the correct way to stream files: https://github.com/apache/spark/pull/3203 You should always move files instead of copying them. --- If your projec

[GitHub] spark issue #16294: [SPARK-18669][SS][DOCS] Update Apache docs for Structure...

2016-12-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16294 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/70524/ Test FAILed. ---

[GitHub] spark issue #16294: [SPARK-18669][SS][DOCS] Update Apache docs for Structure...

2016-12-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16294 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #16294: [SPARK-18669][SS][DOCS] Update Apache docs for Structure...

2016-12-22 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16294 **[Test build #70524 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/70524/testReport)** for PR 16294 at commit [`a4a93aa`](https://github.com/apache/spark/commit/a

[GitHub] spark issue #16385: [SPARK-18985][SS]Add missing @InterfaceStability.Evolvin...

2016-12-22 Thread tdas
Github user tdas commented on the issue: https://github.com/apache/spark/pull/16385 LGTM! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature

[GitHub] spark issue #16240: [SPARK-16792][SQL] Dataset containing a Case Class with ...

2016-12-22 Thread michalsenkyr
Github user michalsenkyr commented on the issue: https://github.com/apache/spark/pull/16240 I actually read that but IDEA complained when I tried to place the `Product` encoder into a separate trait. So I opted for specificity. However, I tried it again right now and even though ID

[GitHub] spark issue #16380: [SPARK-18972][Core]Fix the netty thread names for RPC

2016-12-22 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16380 **[Test build #70527 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/70527/testReport)** for PR 16380 at commit [`5b51040`](https://github.com/apache/spark/commit/5b

[GitHub] spark issue #16294: [SPARK-18669][SS][DOCS] Update Apache docs for Structure...

2016-12-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16294 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #16294: [SPARK-18669][SS][DOCS] Update Apache docs for Structure...

2016-12-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16294 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/70525/ Test PASSed. ---

[GitHub] spark issue #16294: [SPARK-18669][SS][DOCS] Update Apache docs for Structure...

2016-12-22 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16294 **[Test build #70525 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/70525/testReport)** for PR 16294 at commit [`b96351c`](https://github.com/apache/spark/commit/b

[GitHub] spark issue #16380: [SPARK-18972][Core]Fix the netty thread names for RPC

2016-12-22 Thread zsxwing
Github user zsxwing commented on the issue: https://github.com/apache/spark/pull/16380 retest this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, o

[GitHub] spark issue #16385: [SPARK-18985][SS]Add missing @InterfaceStability.Evolvin...

2016-12-22 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16385 **[Test build #70526 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/70526/testReport)** for PR 16385 at commit [`291127f`](https://github.com/apache/spark/commit/29

[GitHub] spark pull request #16385: [SPARK-18985][SS]Add missing @InterfaceStability....

2016-12-22 Thread zsxwing
GitHub user zsxwing opened a pull request: https://github.com/apache/spark/pull/16385 [SPARK-18985][SS]Add missing @InterfaceStability.Evolving for Structured Streaming APIs ## What changes were proposed in this pull request? Add missing @InterfaceStability.Evolving for Str

[GitHub] spark issue #16253: [SPARK-18537][Web UI] Add a REST api to serve spark stre...

2016-12-22 Thread saturday-shi
Github user saturday-shi commented on the issue: https://github.com/apache/spark/pull/16253 @vanzin My JIRA account is [Xing Shi (saturday_s)](https://issues.apache.org/jira/secure/ViewProfile.jspa?name=saturday_s). --- If your project is set up for it, you can reply to this email an

[GitHub] spark issue #16384: [BUILD] make-distribution support alternate python to bu...

2016-12-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16384 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #16384: [BUILD] make-distribution support alternate python to bu...

2016-12-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16384 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/70523/ Test FAILed. ---

[GitHub] spark issue #16294: [SPARK-18669][SS][DOCS] Update Apache docs for Structure...

2016-12-22 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16294 **[Test build #70525 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/70525/testReport)** for PR 16294 at commit [`b96351c`](https://github.com/apache/spark/commit/b9

[GitHub] spark issue #16384: [BUILD] make-distribution support alternate python to bu...

2016-12-22 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16384 **[Test build #70523 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/70523/testReport)** for PR 16384 at commit [`5b8ab75`](https://github.com/apache/spark/commit/5

[GitHub] spark issue #16291: [SPARK-18838][CORE] Use separate executor service for ea...

2016-12-22 Thread vanzin
Github user vanzin commented on the issue: https://github.com/apache/spark/pull/16291 > I think that will be an over-kill. Having a separate thread per listener simplifies the design. I don't think it's overkill, and it wouldn't really be that much of a change from your exis

[GitHub] spark issue #16294: [SPARK-18669][SS][DOCS] Update Apache docs for Structure...

2016-12-22 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16294 **[Test build #70524 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/70524/testReport)** for PR 16294 at commit [`a4a93aa`](https://github.com/apache/spark/commit/a4

[GitHub] spark pull request #16253: [SPARK-18537][Web UI] Add a REST api to serve spa...

2016-12-22 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/16253 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is ena

[GitHub] spark issue #16253: [SPARK-18537][Web UI] Add a REST api to serve spark stre...

2016-12-22 Thread vanzin
Github user vanzin commented on the issue: https://github.com/apache/spark/pull/16253 @saturday-shi I don't know your Jira login, if you let me know I'll assign the bug to you. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as w

[GitHub] spark issue #16253: [SPARK-18537][Web UI] Add a REST api to serve spark stre...

2016-12-22 Thread vanzin
Github user vanzin commented on the issue: https://github.com/apache/spark/pull/16253 Merging to master. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or

[GitHub] spark issue #16233: [SPARK-18801][SQL] Add `View` operator to help resolve a...

2016-12-22 Thread nsyca
Github user nsyca commented on the issue: https://github.com/apache/spark/pull/16233 @hvanhovell: Would the issue of set/reset currentDB be resolved if we support a table/view as a 2-part name? The 2-part name, or in general, n-part name, is used in other databases, SQLServer, Oracle

[GitHub] spark issue #16233: [SPARK-18801][SQL] Add `View` operator to help resolve a...

2016-12-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16233 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/70522/ Test PASSed. ---

[GitHub] spark issue #16233: [SPARK-18801][SQL] Add `View` operator to help resolve a...

2016-12-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16233 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #16233: [SPARK-18801][SQL] Add `View` operator to help resolve a...

2016-12-22 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16233 **[Test build #70522 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/70522/testReport)** for PR 16233 at commit [`f0142ea`](https://github.com/apache/spark/commit/f

[GitHub] spark pull request #15211: [SPARK-14709][ML] spark.ml API for linear SVM

2016-12-22 Thread jkbradley
Github user jkbradley commented on a diff in the pull request: https://github.com/apache/spark/pull/15211#discussion_r93686797 --- Diff: mllib/src/test/scala/org/apache/spark/ml/classification/LinearSVCSuite.scala --- @@ -0,0 +1,178 @@ +/* + * Licensed to the Apache Softwar

[GitHub] spark pull request #15211: [SPARK-14709][ML] spark.ml API for linear SVM

2016-12-22 Thread jkbradley
Github user jkbradley commented on a diff in the pull request: https://github.com/apache/spark/pull/15211#discussion_r93686612 --- Diff: mllib/src/test/scala/org/apache/spark/ml/classification/LinearSVCSuite.scala --- @@ -0,0 +1,178 @@ +/* + * Licensed to the Apache Softwar

[GitHub] spark pull request #15211: [SPARK-14709][ML] spark.ml API for linear SVM

2016-12-22 Thread jkbradley
Github user jkbradley commented on a diff in the pull request: https://github.com/apache/spark/pull/15211#discussion_r93686501 --- Diff: mllib/src/test/scala/org/apache/spark/ml/classification/LinearSVCSuite.scala --- @@ -0,0 +1,178 @@ +/* + * Licensed to the Apache Softwar

[GitHub] spark pull request #15211: [SPARK-14709][ML] spark.ml API for linear SVM

2016-12-22 Thread jkbradley
Github user jkbradley commented on a diff in the pull request: https://github.com/apache/spark/pull/15211#discussion_r93686377 --- Diff: mllib/src/test/scala/org/apache/spark/ml/classification/LinearSVCSuite.scala --- @@ -0,0 +1,178 @@ +/* + * Licensed to the Apache Softwar

[GitHub] spark pull request #15211: [SPARK-14709][ML] spark.ml API for linear SVM

2016-12-22 Thread jkbradley
Github user jkbradley commented on a diff in the pull request: https://github.com/apache/spark/pull/15211#discussion_r93686305 --- Diff: mllib/src/test/scala/org/apache/spark/ml/classification/LinearSVCSuite.scala --- @@ -0,0 +1,178 @@ +/* + * Licensed to the Apache Softwar

[GitHub] spark pull request #15211: [SPARK-14709][ML] spark.ml API for linear SVM

2016-12-22 Thread jkbradley
Github user jkbradley commented on a diff in the pull request: https://github.com/apache/spark/pull/15211#discussion_r93686143 --- Diff: mllib/src/test/scala/org/apache/spark/ml/classification/LinearSVCSuite.scala --- @@ -0,0 +1,178 @@ +/* + * Licensed to the Apache Softwar

[GitHub] spark pull request #15211: [SPARK-14709][ML] spark.ml API for linear SVM

2016-12-22 Thread jkbradley
Github user jkbradley commented on a diff in the pull request: https://github.com/apache/spark/pull/15211#discussion_r93685938 --- Diff: mllib/src/test/scala/org/apache/spark/ml/classification/LinearSVCSuite.scala --- @@ -0,0 +1,178 @@ +/* + * Licensed to the Apache Softwar

[GitHub] spark pull request #16346: [SPARK-16654][CORE] Add UI coverage for Applicati...

2016-12-22 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/16346#discussion_r93685099 --- Diff: core/src/main/scala/org/apache/spark/ui/exec/ExecutorsTab.scala --- @@ -157,4 +158,44 @@ class ExecutorsListener(storageStatusListener: StorageStat

[GitHub] spark pull request #16346: [SPARK-16654][CORE] Add UI coverage for Applicati...

2016-12-22 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/16346#discussion_r93682032 --- Diff: core/src/main/scala/org/apache/spark/scheduler/SparkListener.scala --- @@ -106,6 +106,28 @@ case class SparkListenerExecutorRemoved(time: Long, exe

[GitHub] spark pull request #16346: [SPARK-16654][CORE] Add UI coverage for Applicati...

2016-12-22 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/16346#discussion_r93680964 --- Diff: core/src/main/scala/org/apache/spark/scheduler/BlacklistTracker.scala --- @@ -115,6 +121,10 @@ private[scheduler] class BlacklistTracker (

[GitHub] spark pull request #16346: [SPARK-16654][CORE] Add UI coverage for Applicati...

2016-12-22 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/16346#discussion_r93681435 --- Diff: core/src/main/scala/org/apache/spark/scheduler/BlacklistTracker.scala --- @@ -48,9 +48,14 @@ import org.apache.spark.util.{Clock, SystemClock, Utils

[GitHub] spark pull request #16346: [SPARK-16654][CORE] Add UI coverage for Applicati...

2016-12-22 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/16346#discussion_r93681035 --- Diff: core/src/main/scala/org/apache/spark/scheduler/BlacklistTracker.scala --- @@ -161,6 +171,9 @@ private[scheduler] class BlacklistTracker (

<    1   2   3   4   >