[GitHub] spark pull request: [SPARK-3247][SQL] An API for adding data sourc...

2015-08-20 Thread marmbrus
Github user marmbrus commented on a diff in the pull request: https://github.com/apache/spark/pull/2475#discussion_r37562081 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/sources/filters.scala --- @@ -0,0 +1,26 @@ +/* + * Licensed to the Apache Software Foundation

[GitHub] spark pull request: [SPARK-3247][SQL] An API for adding data sourc...

2015-08-20 Thread liancheng
Github user liancheng commented on a diff in the pull request: https://github.com/apache/spark/pull/2475#discussion_r37514803 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/sources/filters.scala --- @@ -0,0 +1,26 @@ +/* + * Licensed to the Apache Software Foundation

[GitHub] spark pull request: [SPARK-3247][SQL] An API for adding data sourc...

2015-08-20 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/2475#discussion_r37506459 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/sources/filters.scala --- @@ -0,0 +1,26 @@ +/* + * Licensed to the Apache Software Foundation

[GitHub] spark pull request: [SPARK-3247][SQL] An API for adding data sourc...

2014-11-02 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/2475 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enab

[GitHub] spark pull request: [SPARK-3247][SQL] An API for adding data sourc...

2014-11-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/2475#issuecomment-61422214 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/22

[GitHub] spark pull request: [SPARK-3247][SQL] An API for adding data sourc...

2014-11-02 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2475#issuecomment-61422212 [Test build #22765 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/22765/consoleFull) for PR 2475 at commit [`1ed6010`](https://gith

[GitHub] spark pull request: [SPARK-3247][SQL] An API for adding data sourc...

2014-11-02 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2475#issuecomment-61420077 [Test build #22765 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/22765/consoleFull) for PR 2475 at commit [`1ed6010`](https://githu

[GitHub] spark pull request: [SPARK-3247][SQL] An API for adding data sourc...

2014-11-02 Thread rxin
Github user rxin commented on the pull request: https://github.com/apache/spark/pull/2475#issuecomment-61420018 Jenkins, retest this please. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have th

[GitHub] spark pull request: [SPARK-3247][SQL] An API for adding data sourc...

2014-11-01 Thread rxin
Github user rxin commented on the pull request: https://github.com/apache/spark/pull/2475#issuecomment-61394068 LGTM. You'd want to update the PR description before merging it. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as we

[GitHub] spark pull request: [SPARK-3247][SQL] An API for adding data sourc...

2014-11-01 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/2475#issuecomment-61393233 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/22

[GitHub] spark pull request: [SPARK-3247][SQL] An API for adding data sourc...

2014-11-01 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2475#issuecomment-61393229 [Test build #22731 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/22731/consoleFull) for PR 2475 at commit [`1ed6010`](https://gith

[GitHub] spark pull request: [SPARK-3247][SQL] An API for adding data sourc...

2014-11-01 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2475#issuecomment-61392395 [Test build #22731 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/22731/consoleFull) for PR 2475 at commit [`1ed6010`](https://githu

[GitHub] spark pull request: [SPARK-3247][SQL] An API for adding data sourc...

2014-11-01 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/2475#issuecomment-61389566 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/22

[GitHub] spark pull request: [SPARK-3247][SQL] An API for adding data sourc...

2014-11-01 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2475#issuecomment-61389564 [Test build #22713 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/22713/consoleFull) for PR 2475 at commit [`ab2c31f`](https://gith

[GitHub] spark pull request: [SPARK-3247][SQL] An API for adding data sourc...

2014-11-01 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2475#issuecomment-61388270 [Test build #22713 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/22713/consoleFull) for PR 2475 at commit [`ab2c31f`](https://githu

[GitHub] spark pull request: [SPARK-3247][SQL] An API for adding data sourc...

2014-11-01 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2475#issuecomment-61387677 [Test build #22711 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/22711/consoleFull) for PR 2475 at commit [`1d41bb5`](https://gith

[GitHub] spark pull request: [SPARK-3247][SQL] An API for adding data sourc...

2014-11-01 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/2475#issuecomment-61387678 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/22

[GitHub] spark pull request: [SPARK-3247][SQL] An API for adding data sourc...

2014-11-01 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/2475#issuecomment-61387652 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/22

[GitHub] spark pull request: [SPARK-3247][SQL] An API for adding data sourc...

2014-11-01 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2475#issuecomment-61387420 [Test build #22711 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/22711/consoleFull) for PR 2475 at commit [`1d41bb5`](https://githu

[GitHub] spark pull request: [SPARK-3247][SQL] An API for adding data sourc...

2014-11-01 Thread marmbrus
Github user marmbrus commented on a diff in the pull request: https://github.com/apache/spark/pull/2475#discussion_r19708526 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/sources/filters.scala --- @@ -0,0 +1,22 @@ +/* + * Licensed to the Apache Software Foundation

[GitHub] spark pull request: [SPARK-3247][SQL] An API for adding data sourc...

2014-11-01 Thread marmbrus
Github user marmbrus commented on a diff in the pull request: https://github.com/apache/spark/pull/2475#discussion_r19708531 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/sources/filters.scala --- @@ -0,0 +1,22 @@ +/* + * Licensed to the Apache Software Foundation

[GitHub] spark pull request: [SPARK-3247][SQL] An API for adding data sourc...

2014-11-01 Thread liancheng
Github user liancheng commented on a diff in the pull request: https://github.com/apache/spark/pull/2475#discussion_r19703203 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/sources/interfaces.scala --- @@ -0,0 +1,86 @@ +/* +* Licensed to the Apache Software Foundati

[GitHub] spark pull request: [SPARK-3247][SQL] An API for adding data sourc...

2014-11-01 Thread rxin
Github user rxin commented on the pull request: https://github.com/apache/spark/pull/2475#issuecomment-61361324 I like the new API a lot. Java support. Long term binary compatibility! --- If your project is set up for it, you can reply to this email and have your reply appear o

[GitHub] spark pull request: [SPARK-3247][SQL] An API for adding data sourc...

2014-11-01 Thread rxin
Github user rxin commented on a diff in the pull request: https://github.com/apache/spark/pull/2475#discussion_r19702317 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/sources/filters.scala --- @@ -0,0 +1,22 @@ +/* + * Licensed to the Apache Software Foundation (ASF

[GitHub] spark pull request: [SPARK-3247][SQL] An API for adding data sourc...

2014-10-31 Thread mateiz
Github user mateiz commented on the pull request: https://github.com/apache/spark/pull/2475#issuecomment-61358186 The new API for sources looks good to me, thanks for making the changes. It will be easy to plug in a lot of neat data sources here. --- If your project is set up for it,

[GitHub] spark pull request: [SPARK-3247][SQL] An API for adding data sourc...

2014-10-31 Thread mateiz
Github user mateiz commented on a diff in the pull request: https://github.com/apache/spark/pull/2475#discussion_r19701683 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/sources/filters.scala --- @@ -0,0 +1,22 @@ +/* + * Licensed to the Apache Software Foundation (A

[GitHub] spark pull request: [SPARK-3247][SQL] An API for adding data sourc...

2014-10-31 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2475#issuecomment-61343404 [Test build #22653 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/22653/consoleFull) for PR 2475 at commit [`e3e690e`](https://gith

[GitHub] spark pull request: [SPARK-3247][SQL] An API for adding data sourc...

2014-10-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/2475#issuecomment-61343409 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/22

[GitHub] spark pull request: [SPARK-3247][SQL] An API for adding data sourc...

2014-10-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/2475#issuecomment-61342072 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/22

[GitHub] spark pull request: [SPARK-3247][SQL] An API for adding data sourc...

2014-10-31 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2475#issuecomment-61342067 [Test build #22646 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/22646/consoleFull) for PR 2475 at commit [`a70d602`](https://gith

[GitHub] spark pull request: [SPARK-3247][SQL] An API for adding data sourc...

2014-10-31 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2475#issuecomment-61339108 [Test build #22653 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/22653/consoleFull) for PR 2475 at commit [`e3e690e`](https://githu

[GitHub] spark pull request: [SPARK-3247][SQL] An API for adding data sourc...

2014-10-31 Thread marmbrus
Github user marmbrus commented on the pull request: https://github.com/apache/spark/pull/2475#issuecomment-61338760 Okay, thanks for all the comments. I've done the following: - Changed the various base relations into `abstract class` instead of `trait`. - Removed `

[GitHub] spark pull request: [SPARK-3247][SQL] An API for adding data sourc...

2014-10-31 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2475#issuecomment-61337655 [Test build #22646 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/22646/consoleFull) for PR 2475 at commit [`a70d602`](https://githu

[GitHub] spark pull request: [SPARK-3247][SQL] An API for adding data sourc...

2014-10-31 Thread marmbrus
Github user marmbrus commented on a diff in the pull request: https://github.com/apache/spark/pull/2475#discussion_r19696134 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/sources/interfaces.scala --- @@ -0,0 +1,86 @@ +/* +* Licensed to the Apache Software Foundatio

[GitHub] spark pull request: [SPARK-3247][SQL] An API for adding data sourc...

2014-10-31 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2475#issuecomment-61334955 [Test build #22641 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/22641/consoleFull) for PR 2475 at commit [`70da6d9`](https://gith

[GitHub] spark pull request: [SPARK-3247][SQL] An API for adding data sourc...

2014-10-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/2475#issuecomment-61334958 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/22

[GitHub] spark pull request: [SPARK-3247][SQL] An API for adding data sourc...

2014-10-31 Thread marmbrus
Github user marmbrus commented on a diff in the pull request: https://github.com/apache/spark/pull/2475#discussion_r19695119 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/sources/ddl.scala --- @@ -0,0 +1,110 @@ +/* + * Licensed to the Apache Software Foundation (ASF

[GitHub] spark pull request: [SPARK-3247][SQL] An API for adding data sourc...

2014-10-31 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2475#issuecomment-61334847 [Test build #22641 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/22641/consoleFull) for PR 2475 at commit [`70da6d9`](https://githu

[GitHub] spark pull request: [SPARK-3247][SQL] An API for adding data sourc...

2014-10-31 Thread yhuai
Github user yhuai commented on a diff in the pull request: https://github.com/apache/spark/pull/2475#discussion_r19687744 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/sources/FilteredScanSuite.scala --- @@ -0,0 +1,117 @@ +/* +* Licensed to the Apache Software Foun

[GitHub] spark pull request: [SPARK-3247][SQL] An API for adding data sourc...

2014-10-31 Thread yhuai
Github user yhuai commented on a diff in the pull request: https://github.com/apache/spark/pull/2475#discussion_r19687528 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/sources/PrunedScanSuite.scala --- @@ -0,0 +1,102 @@ +/* +* Licensed to the Apache Software Founda

[GitHub] spark pull request: [SPARK-3247][SQL] An API for adding data sourc...

2014-10-31 Thread marmbrus
Github user marmbrus commented on a diff in the pull request: https://github.com/apache/spark/pull/2475#discussion_r19686971 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/sources/interfaces.scala --- @@ -0,0 +1,86 @@ +/* +* Licensed to the Apache Software Foundatio

[GitHub] spark pull request: [SPARK-3247][SQL] An API for adding data sourc...

2014-10-31 Thread yhuai
Github user yhuai commented on a diff in the pull request: https://github.com/apache/spark/pull/2475#discussion_r19686480 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/sources/ddl.scala --- @@ -0,0 +1,110 @@ +/* + * Licensed to the Apache Software Foundation (ASF) u

[GitHub] spark pull request: [SPARK-3247][SQL] An API for adding data sourc...

2014-10-31 Thread yhuai
Github user yhuai commented on a diff in the pull request: https://github.com/apache/spark/pull/2475#discussion_r19685462 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/sources/LogicalRelation.scala --- @@ -0,0 +1,56 @@ +/* + * Licensed to the Apache Software Founda

[GitHub] spark pull request: [SPARK-3247][SQL] An API for adding data sourc...

2014-10-31 Thread marmbrus
Github user marmbrus commented on a diff in the pull request: https://github.com/apache/spark/pull/2475#discussion_r19684904 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/sources/LogicalRelation.scala --- @@ -0,0 +1,56 @@ +/* + * Licensed to the Apache Software Fou

[GitHub] spark pull request: [SPARK-3247][SQL] An API for adding data sourc...

2014-10-31 Thread yhuai
Github user yhuai commented on a diff in the pull request: https://github.com/apache/spark/pull/2475#discussion_r19684775 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/sources/LogicalRelation.scala --- @@ -0,0 +1,56 @@ +/* + * Licensed to the Apache Software Founda

[GitHub] spark pull request: [SPARK-3247][SQL] An API for adding data sourc...

2014-10-31 Thread yhuai
Github user yhuai commented on a diff in the pull request: https://github.com/apache/spark/pull/2475#discussion_r19681071 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/SQLContext.scala --- @@ -295,6 +306,7 @@ class SQLContext(@transient val sparkContext: SparkContext)

[GitHub] spark pull request: [SPARK-3247][SQL] An API for adding data sourc...

2014-10-31 Thread liancheng
Github user liancheng commented on a diff in the pull request: https://github.com/apache/spark/pull/2475#discussion_r19662385 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/sources/ddl.scala --- @@ -0,0 +1,110 @@ +/* + * Licensed to the Apache Software Foundation (AS

[GitHub] spark pull request: [SPARK-3247][SQL] An API for adding data sourc...

2014-10-31 Thread liancheng
Github user liancheng commented on a diff in the pull request: https://github.com/apache/spark/pull/2475#discussion_r19660051 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/sources/interfaces.scala --- @@ -0,0 +1,86 @@ +/* +* Licensed to the Apache Software Foundati

[GitHub] spark pull request: [SPARK-3247][SQL] An API for adding data sourc...

2014-10-31 Thread liancheng
Github user liancheng commented on a diff in the pull request: https://github.com/apache/spark/pull/2475#discussion_r19659690 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/sources/interfaces.scala --- @@ -0,0 +1,86 @@ +/* +* Licensed to the Apache Software Foundati

[GitHub] spark pull request: [SPARK-3247][SQL] An API for adding data sourc...

2014-10-31 Thread liancheng
Github user liancheng commented on a diff in the pull request: https://github.com/apache/spark/pull/2475#discussion_r19659626 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/sources/interfaces.scala --- @@ -0,0 +1,86 @@ +/* +* Licensed to the Apache Software Foundati

[GitHub] spark pull request: [SPARK-3247][SQL] An API for adding data sourc...

2014-10-31 Thread liancheng
Github user liancheng commented on a diff in the pull request: https://github.com/apache/spark/pull/2475#discussion_r19659305 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/sources/interfaces.scala --- @@ -0,0 +1,86 @@ +/* +* Licensed to the Apache Software Foundati

[GitHub] spark pull request: [SPARK-3247][SQL] An API for adding data sourc...

2014-10-31 Thread liancheng
Github user liancheng commented on a diff in the pull request: https://github.com/apache/spark/pull/2475#discussion_r19658936 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/sources/interfaces.scala --- @@ -0,0 +1,86 @@ +/* +* Licensed to the Apache Software Foundati

[GitHub] spark pull request: [SPARK-3247][SQL] An API for adding data sourc...

2014-10-31 Thread liancheng
Github user liancheng commented on a diff in the pull request: https://github.com/apache/spark/pull/2475#discussion_r19657566 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/sources/ddl.scala --- @@ -0,0 +1,110 @@ +/* + * Licensed to the Apache Software Foundation (AS

[GitHub] spark pull request: [SPARK-3247][SQL] An API for adding data sourc...

2014-10-31 Thread liancheng
Github user liancheng commented on the pull request: https://github.com/apache/spark/pull/2475#issuecomment-61228429 Oh, I see, the last Jenkins build success was actually for the commit right before the last one. --- If your project is set up for it, you can reply to this email and

[GitHub] spark pull request: [SPARK-3247][SQL] An API for adding data sourc...

2014-10-31 Thread liancheng
Github user liancheng commented on the pull request: https://github.com/apache/spark/pull/2475#issuecomment-61228389 Encountered compilation error while compiling locally (with a clean checkout `./sbt/sbt -Phive,hadoop-2.3 compil`): ``` [error] /Users/lian/local/src/spark

[GitHub] spark pull request: [SPARK-3247][SQL] An API for adding data sourc...

2014-10-31 Thread scwf
Github user scwf commented on a diff in the pull request: https://github.com/apache/spark/pull/2475#discussion_r19654727 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/json/JSONRelation.scala --- @@ -0,0 +1,49 @@ +/* + * Licensed to the Apache Software Foundation (A

[GitHub] spark pull request: [SPARK-3247][SQL] An API for adding data sourc...

2014-10-31 Thread scwf
Github user scwf commented on a diff in the pull request: https://github.com/apache/spark/pull/2475#discussion_r19654586 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/sources/interfaces.scala --- @@ -0,0 +1,86 @@ +/* +* Licensed to the Apache Software Foundation (A

[GitHub] spark pull request: [SPARK-3247][SQL] An API for adding data sourc...

2014-10-31 Thread scwf
Github user scwf commented on a diff in the pull request: https://github.com/apache/spark/pull/2475#discussion_r19654557 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/sources/interfaces.scala --- @@ -0,0 +1,86 @@ +/* +* Licensed to the Apache Software Foundation (A

[GitHub] spark pull request: [SPARK-3247][SQL] An API for adding data sourc...

2014-10-30 Thread scwf
Github user scwf commented on a diff in the pull request: https://github.com/apache/spark/pull/2475#discussion_r19654020 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/sources/ddl.scala --- @@ -0,0 +1,110 @@ +/* + * Licensed to the Apache Software Foundation (ASF) un

[GitHub] spark pull request: [SPARK-3247][SQL] An API for adding data sourc...

2014-10-30 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/2475#issuecomment-61222348 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/22

[GitHub] spark pull request: [SPARK-3247][SQL] An API for adding data sourc...

2014-10-30 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2475#issuecomment-61222343 [Test build #22592 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/22592/consoleFull) for PR 2475 at commit [`b069146`](https://gith

[GitHub] spark pull request: [SPARK-3247][SQL] An API for adding data sourc...

2014-10-30 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/2475#issuecomment-61220321 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/22

[GitHub] spark pull request: [SPARK-3247][SQL] An API for adding data sourc...

2014-10-30 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2475#issuecomment-61220318 [Test build #22593 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/22593/consoleFull) for PR 2475 at commit [`22963ef`](https://gith

[GitHub] spark pull request: [SPARK-3247][SQL] An API for adding data sourc...

2014-10-30 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2475#issuecomment-61220252 [Test build #22593 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/22593/consoleFull) for PR 2475 at commit [`22963ef`](https://githu

[GitHub] spark pull request: [SPARK-3247][SQL] An API for adding data sourc...

2014-10-30 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2475#issuecomment-61219607 [Test build #22592 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/22592/consoleFull) for PR 2475 at commit [`b069146`](https://githu

[GitHub] spark pull request: [SPARK-3247][SQL] An API for adding data sourc...

2014-10-30 Thread rxin
Github user rxin commented on a diff in the pull request: https://github.com/apache/spark/pull/2475#discussion_r19652398 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/sources/package.scala --- @@ -0,0 +1,92 @@ +/* + * Licensed to the Apache Software Foundation (ASF

[GitHub] spark pull request: [SPARK-3247][SQL] An API for adding data sourc...

2014-10-30 Thread marmbrus
Github user marmbrus commented on a diff in the pull request: https://github.com/apache/spark/pull/2475#discussion_r19651082 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/sources/package.scala --- @@ -0,0 +1,95 @@ +/* + * Licensed to the Apache Software Foundation

[GitHub] spark pull request: [SPARK-3247][SQL] An API for adding data sourc...

2014-10-30 Thread mateiz
Github user mateiz commented on a diff in the pull request: https://github.com/apache/spark/pull/2475#discussion_r19650306 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/sources/package.scala --- @@ -0,0 +1,95 @@ +/* + * Licensed to the Apache Software Foundation (A

[GitHub] spark pull request: [SPARK-3247][SQL] An API for adding data sourc...

2014-10-30 Thread mateiz
Github user mateiz commented on a diff in the pull request: https://github.com/apache/spark/pull/2475#discussion_r19650167 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/sources/package.scala --- @@ -0,0 +1,95 @@ +/* + * Licensed to the Apache Software Foundation (A

[GitHub] spark pull request: [SPARK-3247][SQL] An API for adding data sourc...

2014-10-30 Thread marmbrus
Github user marmbrus commented on a diff in the pull request: https://github.com/apache/spark/pull/2475#discussion_r19648912 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/sources/package.scala --- @@ -0,0 +1,95 @@ +/* + * Licensed to the Apache Software Foundation

[GitHub] spark pull request: [SPARK-3247][SQL] An API for adding data sourc...

2014-10-30 Thread marmbrus
Github user marmbrus commented on a diff in the pull request: https://github.com/apache/spark/pull/2475#discussion_r19648832 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/sources/package.scala --- @@ -0,0 +1,95 @@ +/* + * Licensed to the Apache Software Foundation

[GitHub] spark pull request: [SPARK-3247][SQL] An API for adding data sourc...

2014-10-30 Thread rxin
Github user rxin commented on a diff in the pull request: https://github.com/apache/spark/pull/2475#discussion_r19648788 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/sources/package.scala --- @@ -0,0 +1,95 @@ +/* + * Licensed to the Apache Software Foundation (ASF

[GitHub] spark pull request: [SPARK-3247][SQL] An API for adding data sourc...

2014-10-30 Thread marmbrus
Github user marmbrus commented on a diff in the pull request: https://github.com/apache/spark/pull/2475#discussion_r19648604 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/sources/package.scala --- @@ -0,0 +1,95 @@ +/* + * Licensed to the Apache Software Foundation

[GitHub] spark pull request: [SPARK-3247][SQL] An API for adding data sourc...

2014-10-30 Thread marmbrus
Github user marmbrus commented on a diff in the pull request: https://github.com/apache/spark/pull/2475#discussion_r19648230 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/types/dataTypes.scala --- @@ -389,7 +389,7 @@ case class ArrayType(elementType: DataType

[GitHub] spark pull request: [SPARK-3247][SQL] An API for adding data sourc...

2014-10-30 Thread marmbrus
Github user marmbrus commented on a diff in the pull request: https://github.com/apache/spark/pull/2475#discussion_r19647038 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/commands.scala --- @@ -21,10 +21,12 @@ import org.apache.spark.Logging import org.apach

[GitHub] spark pull request: [SPARK-3247][SQL] An API for adding data sourc...

2014-10-30 Thread rxin
Github user rxin commented on a diff in the pull request: https://github.com/apache/spark/pull/2475#discussion_r19646994 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/sources/LogicalRelation.scala --- @@ -0,0 +1,56 @@ +/* + * Licensed to the Apache Software Foundat

[GitHub] spark pull request: [SPARK-3247][SQL] An API for adding data sourc...

2014-10-30 Thread rxin
Github user rxin commented on a diff in the pull request: https://github.com/apache/spark/pull/2475#discussion_r19647025 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/sources/package.scala --- @@ -0,0 +1,95 @@ +/* + * Licensed to the Apache Software Foundation (ASF

[GitHub] spark pull request: [SPARK-3247][SQL] An API for adding data sourc...

2014-10-30 Thread rxin
Github user rxin commented on a diff in the pull request: https://github.com/apache/spark/pull/2475#discussion_r19646962 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/commands.scala --- @@ -21,10 +21,12 @@ import org.apache.spark.Logging import org.apache.sp

[GitHub] spark pull request: [SPARK-3247][SQL] An API for adding data sourc...

2014-10-30 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/2475#issuecomment-61198661 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/22

[GitHub] spark pull request: [SPARK-3247][SQL] An API for adding data sourc...

2014-10-30 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2475#issuecomment-61198655 [Test build #22581 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/22581/consoleFull) for PR 2475 at commit [`34f836a`](https://gith

[GitHub] spark pull request: [SPARK-3247][SQL] An API for adding data sourc...

2014-10-30 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/2475#issuecomment-61198599 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/22

[GitHub] spark pull request: [SPARK-3247][SQL] An API for adding data sourc...

2014-10-30 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2475#issuecomment-61198593 [Test build #22577 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/22577/consoleFull) for PR 2475 at commit [`de3b68c`](https://gith

[GitHub] spark pull request: [SPARK-3247][SQL] An API for adding data sourc...

2014-10-30 Thread rxin
Github user rxin commented on a diff in the pull request: https://github.com/apache/spark/pull/2475#discussion_r19646942 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/types/dataTypes.scala --- @@ -389,7 +389,7 @@ case class ArrayType(elementType: DataType, c

[GitHub] spark pull request: [SPARK-3247][SQL] An API for adding data sourc...

2014-10-30 Thread mateiz
Github user mateiz commented on a diff in the pull request: https://github.com/apache/spark/pull/2475#discussion_r19646688 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/sources/package.scala --- @@ -0,0 +1,95 @@ +/* + * Licensed to the Apache Software Foundation (A

[GitHub] spark pull request: [SPARK-3247][SQL] An API for adding data sourc...

2014-10-30 Thread marmbrus
Github user marmbrus commented on a diff in the pull request: https://github.com/apache/spark/pull/2475#discussion_r19646670 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/sources/package.scala --- @@ -0,0 +1,95 @@ +/* + * Licensed to the Apache Software Foundation

[GitHub] spark pull request: [SPARK-3247][SQL] An API for adding data sourc...

2014-10-30 Thread mateiz
Github user mateiz commented on a diff in the pull request: https://github.com/apache/spark/pull/2475#discussion_r19646622 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/sources/package.scala --- @@ -0,0 +1,95 @@ +/* + * Licensed to the Apache Software Foundation (A

[GitHub] spark pull request: [SPARK-3247][SQL] An API for adding data sourc...

2014-10-30 Thread marmbrus
Github user marmbrus commented on the pull request: https://github.com/apache/spark/pull/2475#issuecomment-61196797 @yhuai not all datasets will be path based. Consider the case of a JDBC based data source. We should be consistent it naming though, I'll change avroFile to match.

[GitHub] spark pull request: [SPARK-3247][SQL] An API for adding data sourc...

2014-10-30 Thread marmbrus
Github user marmbrus commented on a diff in the pull request: https://github.com/apache/spark/pull/2475#discussion_r19646282 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/sources/package.scala --- @@ -0,0 +1,95 @@ +/* + * Licensed to the Apache Software Foundation

[GitHub] spark pull request: [SPARK-3247][SQL] An API for adding data sourc...

2014-10-30 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/2475#issuecomment-61196071 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/22

[GitHub] spark pull request: [SPARK-3247][SQL] An API for adding data sourc...

2014-10-30 Thread marmbrus
Github user marmbrus commented on a diff in the pull request: https://github.com/apache/spark/pull/2475#discussion_r19646027 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/sources/package.scala --- @@ -0,0 +1,95 @@ +/* + * Licensed to the Apache Software Foundation

[GitHub] spark pull request: [SPARK-3247][SQL] An API for adding data sourc...

2014-10-30 Thread yhuai
Github user yhuai commented on the pull request: https://github.com/apache/spark/pull/2475#issuecomment-61195407 Since we probably always need to specify the location of a data set, do we want to define a keyword instead of allowing `RelationProvider`s using different keys? Seems JSON

[GitHub] spark pull request: [SPARK-3247][SQL] An API for adding data sourc...

2014-10-30 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2475#issuecomment-61195325 [Test build #22581 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/22581/consoleFull) for PR 2475 at commit [`34f836a`](https://githu

[GitHub] spark pull request: [SPARK-3247][SQL] An API for adding data sourc...

2014-10-30 Thread mateiz
Github user mateiz commented on a diff in the pull request: https://github.com/apache/spark/pull/2475#discussion_r19645938 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/sources/package.scala --- @@ -0,0 +1,95 @@ +/* + * Licensed to the Apache Software Foundation (A

[GitHub] spark pull request: [SPARK-3247][SQL] An API for adding data sourc...

2014-10-30 Thread mateiz
Github user mateiz commented on a diff in the pull request: https://github.com/apache/spark/pull/2475#discussion_r19645907 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/sources/package.scala --- @@ -0,0 +1,95 @@ +/* + * Licensed to the Apache Software Foundation (A

[GitHub] spark pull request: [SPARK-3247][SQL] An API for adding data sourc...

2014-10-30 Thread mateiz
Github user mateiz commented on a diff in the pull request: https://github.com/apache/spark/pull/2475#discussion_r19645814 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/sources/package.scala --- @@ -0,0 +1,95 @@ +/* + * Licensed to the Apache Software Foundation (A

[GitHub] spark pull request: [SPARK-3247][SQL] An API for adding data sourc...

2014-10-30 Thread mateiz
Github user mateiz commented on a diff in the pull request: https://github.com/apache/spark/pull/2475#discussion_r19645761 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/sources/package.scala --- @@ -0,0 +1,95 @@ +/* + * Licensed to the Apache Software Foundation (A

[GitHub] spark pull request: [SPARK-3247][SQL] An API for adding data sourc...

2014-10-30 Thread mateiz
Github user mateiz commented on a diff in the pull request: https://github.com/apache/spark/pull/2475#discussion_r19645705 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/sources/package.scala --- @@ -0,0 +1,95 @@ +/* + * Licensed to the Apache Software Foundation (A

[GitHub] spark pull request: [SPARK-3247][SQL] An API for adding data sourc...

2014-10-30 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2475#issuecomment-61193556 [Test build #22577 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/22577/consoleFull) for PR 2475 at commit [`de3b68c`](https://githu

[GitHub] spark pull request: [SPARK-3247][SQL] An API for adding data sourc...

2014-10-30 Thread rxin
Github user rxin commented on a diff in the pull request: https://github.com/apache/spark/pull/2475#discussion_r19645425 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/json/JSONRelation.scala --- @@ -0,0 +1,52 @@ +/* + * Licensed to the Apache Software Foundation (A

[GitHub] spark pull request: [SPARK-3247][SQL] An API for adding data sourc...

2014-10-30 Thread rxin
Github user rxin commented on a diff in the pull request: https://github.com/apache/spark/pull/2475#discussion_r19645438 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/json/JSONRelation.scala --- @@ -0,0 +1,52 @@ +/* + * Licensed to the Apache Software Foundation (A

  1   2   >