Github user chutium commented on a diff in the pull request:
https://github.com/apache/spark/pull/195#discussion_r15936076
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/parquet/ParquetRelation.scala ---
@@ -72,16 +71,56 @@ case class ParquetRelation(val tableName: String,
Github user asfgit closed the pull request at:
https://github.com/apache/spark/pull/195
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabl
Github user rxin commented on the pull request:
https://github.com/apache/spark/pull/195#issuecomment-39513179
merged. thx!
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enable
Github user marmbrus commented on the pull request:
https://github.com/apache/spark/pull/195#issuecomment-39500398
LGTM
@pwendell this is ready to merge.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your pr
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/195#issuecomment-39457183
All automated tests passed.
Refer to this link for build results:
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/13724/
---
If your project
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/195#issuecomment-39457180
Merged build finished. All automated tests passed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If
Github user AndreSchumacher commented on the pull request:
https://github.com/apache/spark/pull/195#issuecomment-39457091
@marmbrus Thanks again for the comments and changes. I shuffled the imports
and removed the additions to SQLContext. Also moved the insert examples to the
tests ra
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/195#issuecomment-39450885
Merged build triggered.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not ha
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/195#issuecomment-39450898
Merged build started.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/195#issuecomment-39421942
All automated tests passed.
Refer to this link for build results:
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/13720/
---
If your project
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/195#issuecomment-39421940
Merged build finished. All automated tests passed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/195#issuecomment-39418187
Merged build started.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/195#issuecomment-39418179
Merged build triggered.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not ha
Github user AndreSchumacher commented on the pull request:
https://github.com/apache/spark/pull/195#issuecomment-39418052
@marmbrus Great, thanks a lot. I will go through those comments and your PR
and extend the documentation.
---
If your project is set up for it, you can reply to t
Github user marmbrus commented on the pull request:
https://github.com/apache/spark/pull/195#issuecomment-39402493
Also sorry for commenting in a weird way that doesn't show up on the PR
page... oops!
---
If your project is set up for it, you can reply to this email and have your
rep
Github user marmbrus commented on the pull request:
https://github.com/apache/spark/pull/195#issuecomment-39402465
Hey @AndreSchumacher,
This is a pretty cool feature!
I think this is close to merging once the comments are addressed. I also
made a PR against your PR
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/195#issuecomment-39314778
Refer to this link for build results:
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/13679/
---
If your project is set up for it, you can r
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/195#issuecomment-39314774
Merged build finished.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/195#issuecomment-39304956
Merged build triggered.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not ha
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/195#issuecomment-39304970
Merged build started.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/195#issuecomment-39221977
Refer to this link for build results:
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/13632/
---
If your project is set up for it, you can r
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/195#issuecomment-39221975
Merged build finished.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/195#issuecomment-39215180
Merged build triggered.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not ha
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/195#issuecomment-39215197
Merged build started.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/195#issuecomment-39186014
Refer to this link for build results:
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/13627/
---
If your project is set up for it, you can r
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/195#issuecomment-39186013
Merged build finished.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/195#issuecomment-39185768
Merged build triggered.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not ha
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/195#issuecomment-39185786
Merged build started.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have
Github user marmbrus commented on the pull request:
https://github.com/apache/spark/pull/195#issuecomment-39108623
Yes, that sounds like a good solution. `reset()` should clear all state
from a `HiveContext`. Note that this may require us to make the table
registration more explicit
Github user AndreSchumacher commented on the pull request:
https://github.com/apache/spark/pull/195#issuecomment-39102778
@marmbrus I finally found out why the one test fails. It seems that the
SchemaRDD's that are registered inside SimpleCatalog are not removed after the
tests (the H
Github user marmbrus commented on a diff in the pull request:
https://github.com/apache/spark/pull/195#discussion_r11094163
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/parquet/ParquetTableOperations.scala
---
@@ -40,7 +40,7 @@ import java.util.Date
* Parquet table
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/195#issuecomment-38989837
Build is starting -or- tests failed to complete.
Refer to this link for build results:
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/13570/
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/195#issuecomment-38989836
Merged build finished. Build is starting -or- tests failed to complete.
---
If your project is set up for it, you can reply to this email and have your
reply appear
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/195#issuecomment-38989505
Merged build started. Build is starting -or- tests failed to complete.
---
If your project is set up for it, you can reply to this email and have your
reply appear o
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/195#issuecomment-38989502
Merged build triggered. Build is starting -or- tests failed to complete.
---
If your project is set up for it, you can reply to this email and have your
reply appea
Github user AndreSchumacher commented on the pull request:
https://github.com/apache/spark/pull/195#issuecomment-38989484
retest this please
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have th
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/195#issuecomment-38989047
Merged build triggered. Build is starting -or- tests failed to complete.
---
If your project is set up for it, you can reply to this email and have your
reply appea
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/195#issuecomment-38989050
Merged build started. Build is starting -or- tests failed to complete.
---
If your project is set up for it, you can reply to this email and have your
reply appear o
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/195#issuecomment-38988905
Merged build finished. Build is starting -or- tests failed to complete.
---
If your project is set up for it, you can reply to this email and have your
reply appear
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/195#issuecomment-38988906
Build is starting -or- tests failed to complete.
Refer to this link for build results:
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/13569/
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/195#issuecomment-38988222
Merged build started. Build is starting -or- tests failed to complete.
---
If your project is set up for it, you can reply to this email and have your
reply appear o
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/195#issuecomment-38988220
Merged build triggered. Build is starting -or- tests failed to complete.
---
If your project is set up for it, you can reply to this email and have your
reply appea
Github user AndreSchumacher commented on a diff in the pull request:
https://github.com/apache/spark/pull/195#discussion_r11072105
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/parquet/ParquetTableOperations.scala
---
@@ -210,3 +224,48 @@ case class InsertIntoParquetTable
Github user AndreSchumacher commented on a diff in the pull request:
https://github.com/apache/spark/pull/195#discussion_r11072079
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/parquet/ParquetRelation.scala ---
@@ -72,16 +71,56 @@ case class ParquetRelation(val tableName:
Github user AndreSchumacher commented on a diff in the pull request:
https://github.com/apache/spark/pull/195#discussion_r11071279
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/parquet/ParquetTableOperations.scala
---
@@ -40,7 +40,7 @@ import java.util.Date
* Parque
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/195#issuecomment-38945563
Build is starting -or- tests failed to complete.
Refer to this link for build results:
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/13551/
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/195#issuecomment-38945561
Merged build finished. Build is starting -or- tests failed to complete.
---
If your project is set up for it, you can reply to this email and have your
reply appear
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/195#issuecomment-38936429
Merged build started. Build is starting -or- tests failed to complete.
---
If your project is set up for it, you can reply to this email and have your
reply appear o
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/195#issuecomment-38942145
Build is starting -or- tests failed to complete.
Refer to this link for build results:
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/13550/
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/195#issuecomment-38942144
Merged build finished. Build is starting -or- tests failed to complete.
---
If your project is set up for it, you can reply to this email and have your
reply appear
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/195#issuecomment-38936414
Merged build triggered. Build is starting -or- tests failed to complete.
---
If your project is set up for it, you can reply to this email and have your
reply appea
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/195#issuecomment-38935866
Merged build started. Build is starting -or- tests failed to complete.
---
If your project is set up for it, you can reply to this email and have your
reply appear o
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/195#issuecomment-38935838
Merged build triggered. Build is starting -or- tests failed to complete.
---
If your project is set up for it, you can reply to this email and have your
reply appea
Github user jerryshao commented on a diff in the pull request:
https://github.com/apache/spark/pull/195#discussion_r11053408
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/parquet/ParquetTableOperations.scala
---
@@ -40,7 +40,7 @@ import java.util.Date
* Parquet tabl
Github user marmbrus commented on a diff in the pull request:
https://github.com/apache/spark/pull/195#discussion_r11053247
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/parquet/ParquetTableOperations.scala
---
@@ -40,7 +40,7 @@ import java.util.Date
* Parquet table
Github user jerryshao commented on a diff in the pull request:
https://github.com/apache/spark/pull/195#discussion_r11053203
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/parquet/ParquetTableOperations.scala
---
@@ -40,7 +40,7 @@ import java.util.Date
* Parquet tabl
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/195#issuecomment-38836510
Build finished. One or more automated tests failed
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/195#issuecomment-38836511
One or more automated tests failed
Refer to this link for build results:
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/13521/
---
If your p
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/195#issuecomment-38830164
Build started. One or more automated tests failed
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/195#issuecomment-38830140
Build triggered. One or more automated tests failed
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well.
Github user marmbrus commented on a diff in the pull request:
https://github.com/apache/spark/pull/195#discussion_r11002178
--- Diff:
sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveStrategies.scala ---
@@ -132,7 +132,7 @@ trait HiveStrategies {
relatio
Github user marmbrus commented on a diff in the pull request:
https://github.com/apache/spark/pull/195#discussion_r11002153
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/parquet/ParquetTableOperations.scala
---
@@ -210,3 +224,48 @@ case class InsertIntoParquetTable(
Github user marmbrus commented on a diff in the pull request:
https://github.com/apache/spark/pull/195#discussion_r11002083
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/parquet/ParquetRelation.scala ---
@@ -72,16 +71,56 @@ case class ParquetRelation(val tableName: String,
Github user marmbrus commented on a diff in the pull request:
https://github.com/apache/spark/pull/195#discussion_r11002041
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/parquet/ParquetRelation.scala ---
@@ -72,16 +71,56 @@ case class ParquetRelation(val tableName: String,
Github user marmbrus commented on a diff in the pull request:
https://github.com/apache/spark/pull/195#discussion_r11002023
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/parquet/ParquetRelation.scala ---
@@ -72,16 +71,56 @@ case class ParquetRelation(val tableName: String,
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/195#issuecomment-38711583
Build finished.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this fe
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/195#issuecomment-38711584
One or more automated tests failed
Refer to this link for build results:
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/13471/
---
If your p
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/195#issuecomment-38704075
Build started.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this fea
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/195#issuecomment-38704074
Build triggered.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this
Github user AndreSchumacher commented on the pull request:
https://github.com/apache/spark/pull/195#issuecomment-38701638
retest this please
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have th
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/195#issuecomment-38696187
One or more automated tests failed
Refer to this link for build results:
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/13470/
---
If your p
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/195#issuecomment-38696186
Build finished.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this fe
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/195#issuecomment-38695929
Build started.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this fea
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/195#issuecomment-38695928
Build triggered.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this
Github user AndreSchumacher commented on the pull request:
https://github.com/apache/spark/pull/195#issuecomment-38695808
@marmbrus @pwendell Thanks a lot for the detailed comments. Here are some
thoughts regarding the points you raised above:
* Exposing ``DataType`` to users is p
Github user AndreSchumacher commented on a diff in the pull request:
https://github.com/apache/spark/pull/195#discussion_r10975266
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/parquet/ParquetRelation.scala ---
@@ -72,16 +73,43 @@ case class ParquetRelation(val tableName:
Github user marmbrus commented on a diff in the pull request:
https://github.com/apache/spark/pull/195#discussion_r10903544
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/parquet/ParquetRelation.scala ---
@@ -72,16 +73,43 @@ case class ParquetRelation(val tableName: String,
Github user AndreSchumacher commented on a diff in the pull request:
https://github.com/apache/spark/pull/195#discussion_r10892494
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/parquet/ParquetRelation.scala ---
@@ -72,16 +73,43 @@ case class ParquetRelation(val tableName:
Github user AndreSchumacher commented on a diff in the pull request:
https://github.com/apache/spark/pull/195#discussion_r10867923
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/parquet/ParquetTableOperations.scala
---
@@ -210,3 +220,47 @@ case class InsertIntoParquetTable
Github user AndreSchumacher commented on a diff in the pull request:
https://github.com/apache/spark/pull/195#discussion_r10867922
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/parquet/ParquetRelation.scala ---
@@ -72,16 +73,43 @@ case class ParquetRelation(val tableName:
Github user marmbrus commented on a diff in the pull request:
https://github.com/apache/spark/pull/195#discussion_r10859923
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/parquet/ParquetTableOperations.scala
---
@@ -210,3 +220,47 @@ case class InsertIntoParquetTable(
Github user pwendell commented on a diff in the pull request:
https://github.com/apache/spark/pull/195#discussion_r10859905
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/parquet/ParquetTableOperations.scala
---
@@ -210,3 +220,47 @@ case class InsertIntoParquetTable(
Github user pwendell commented on a diff in the pull request:
https://github.com/apache/spark/pull/195#discussion_r10859898
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/parquet/ParquetTableOperations.scala
---
@@ -210,3 +220,47 @@ case class InsertIntoParquetTable(
Github user pwendell commented on a diff in the pull request:
https://github.com/apache/spark/pull/195#discussion_r10857590
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/parquet/ParquetRelation.scala ---
@@ -72,16 +73,43 @@ case class ParquetRelation(val tableName: String,
Github user marmbrus commented on a diff in the pull request:
https://github.com/apache/spark/pull/195#discussion_r10856974
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/parquet/ParquetRelation.scala ---
@@ -72,16 +73,43 @@ case class ParquetRelation(val tableName: String,
Github user pwendell commented on a diff in the pull request:
https://github.com/apache/spark/pull/195#discussion_r10856726
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/parquet/ParquetRelation.scala ---
@@ -72,16 +73,43 @@ case class ParquetRelation(val tableName: String,
Github user marmbrus commented on the pull request:
https://github.com/apache/spark/pull/195#issuecomment-38324790
Hey Andre, I haven't had a chance to look at this closely, but I do really
like the idea of this functionality. Some thoughts:
- Right now it looks like this is expos
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/195#issuecomment-38298439
Merged build finished.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/195#issuecomment-38298441
One or more automated tests failed
Refer to this link for build results:
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/13321/
---
If your p
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/195#issuecomment-38291880
Merged build started.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have t
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/195#issuecomment-38291879
Merged build triggered.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not hav
Github user AndreSchumacher commented on the pull request:
https://github.com/apache/spark/pull/195#issuecomment-38288151
@marmbrus : would be great if you could have a look at the changes. Thanks!
---
If your project is set up for it, you can reply to this email and have your
reply a
GitHub user AndreSchumacher opened a pull request:
https://github.com/apache/spark/pull/195
Spark parquet improvements
A few improvements to the Parquet support for SQL queries:
- Instead of files a ParquetRelation is now backed by a directory, which
simplifies importing data fr
93 matches
Mail list logo
