GitHub user marmbrus opened a pull request:
https://github.com/apache/spark/pull/354
[SQL] SPARK-1424 Generalize insertIntoTable functions on SchemaRDDs
I don't think this is quite done yet, but want to hear what people think
about this API.
You can merge this pull request into a G
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/354#issuecomment-39806215
Merged build triggered.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not ha
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/354#issuecomment-39806224
Merged build started.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/354#issuecomment-39807995
All automated tests passed.
Refer to this link for build results:
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/13875/
---
If your project
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/354#issuecomment-39807993
Merged build finished. All automated tests passed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If
Github user rxin commented on a diff in the pull request:
https://github.com/apache/spark/pull/354#discussion_r11375848
--- Diff: sql/core/src/main/scala/org/apache/spark/sql/SchemaRDDLike.scala
---
@@ -62,4 +64,41 @@ trait SchemaRDDLike {
def registerAsTable(tableName: Stri
Github user rxin commented on a diff in the pull request:
https://github.com/apache/spark/pull/354#discussion_r11375843
--- Diff: sql/core/src/main/scala/org/apache/spark/sql/SchemaRDDLike.scala
---
@@ -62,4 +64,41 @@ trait SchemaRDDLike {
def registerAsTable(tableName: Stri
Github user rxin commented on a diff in the pull request:
https://github.com/apache/spark/pull/354#discussion_r11375853
--- Diff: sql/core/src/main/scala/org/apache/spark/sql/SchemaRDDLike.scala
---
@@ -62,4 +64,41 @@ trait SchemaRDDLike {
def registerAsTable(tableName: Stri
Github user marmbrus commented on a diff in the pull request:
https://github.com/apache/spark/pull/354#discussion_r11516290
--- Diff: sql/core/src/main/scala/org/apache/spark/sql/SchemaRDDLike.scala
---
@@ -62,4 +64,41 @@ trait SchemaRDDLike {
def registerAsTable(tableName:
Github user marmbrus commented on a diff in the pull request:
https://github.com/apache/spark/pull/354#discussion_r11516306
--- Diff: sql/core/src/main/scala/org/apache/spark/sql/SchemaRDDLike.scala
---
@@ -62,4 +64,41 @@ trait SchemaRDDLike {
def registerAsTable(tableName:
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/354#issuecomment-40166320
Merged build triggered.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not ha
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/354#issuecomment-40166326
Merged build started.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have
Github user marmbrus commented on the pull request:
https://github.com/apache/spark/pull/354#issuecomment-40166368
Okay, you can now create both hive tables and parquet files using a schema
defined by a case class with or without an error if the table exists. You can
insert (append o
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/354#issuecomment-40167725
All automated tests passed.
Refer to this link for build results:
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/14039/
---
If your project
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/354#issuecomment-40167722
Merged build finished. All automated tests passed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If
Github user marmbrus commented on the pull request:
https://github.com/apache/spark/pull/354#issuecomment-40241700
@rxin @mateiz PTAL
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feat
Github user aarondav commented on a diff in the pull request:
https://github.com/apache/spark/pull/354#discussion_r11547314
--- Diff: sql/core/src/main/scala/org/apache/spark/sql/SchemaRDDLike.scala
---
@@ -62,4 +65,43 @@ trait SchemaRDDLike {
def registerAsTable(tableName:
Github user marmbrus commented on a diff in the pull request:
https://github.com/apache/spark/pull/354#discussion_r11548229
--- Diff: sql/core/src/main/scala/org/apache/spark/sql/SchemaRDDLike.scala
---
@@ -62,4 +65,43 @@ trait SchemaRDDLike {
def registerAsTable(tableName:
Github user mateiz commented on a diff in the pull request:
https://github.com/apache/spark/pull/354#discussion_r11554683
--- Diff: sql/core/src/main/scala/org/apache/spark/sql/SchemaRDDLike.scala
---
@@ -62,4 +65,43 @@ trait SchemaRDDLike {
def registerAsTable(tableName: St
Github user mateiz commented on a diff in the pull request:
https://github.com/apache/spark/pull/354#discussion_r11555907
--- Diff: sql/core/src/main/scala/org/apache/spark/sql/SchemaRDDLike.scala
---
@@ -62,4 +65,43 @@ trait SchemaRDDLike {
def registerAsTable(tableName: St
Github user mateiz commented on the pull request:
https://github.com/apache/spark/pull/354#issuecomment-40262979
Once the Scala ones are finalized, Michael, are you going to add these to
the Java API in the same PR? We should make sure the API translates well into
Java (e.g. for defau
Github user mateiz commented on the pull request:
https://github.com/apache/spark/pull/354#issuecomment-40263207
So yeah BTW my high-level comment is if we can get rid of createPaquetFile
and just add a "table name" argument to saveAsParquetFile. Then we should also
rename createTable
Github user rxin commented on a diff in the pull request:
https://github.com/apache/spark/pull/354#discussion_r11597602
--- Diff: sql/core/src/main/scala/org/apache/spark/sql/SQLContext.scala ---
@@ -88,6 +96,26 @@ class SQLContext(@transient val sparkContext:
SparkContext)
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/354#issuecomment-40437968
Merged build triggered.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not ha
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/354#issuecomment-40437974
Merged build started.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/354#issuecomment-40439180
Merged build started.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/354#issuecomment-40439169
Merged build triggered.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not ha
Github user marmbrus commented on the pull request:
https://github.com/apache/spark/pull/354#issuecomment-40439248
Okay, I updated the API based on a conversation with @mateiz. I also added
the relevant function to the Java API. We can do python in a follow up PR once
that is merged
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/354#issuecomment-40439900
All automated tests passed.
Refer to this link for build results:
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/14127/
---
If your project
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/354#issuecomment-40439899
Merged build finished. All automated tests passed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/354#issuecomment-40440870
All automated tests passed.
Refer to this link for build results:
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/14129/
---
If your project
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/354#issuecomment-40440869
Merged build finished. All automated tests passed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/354#issuecomment-40547218
Merged build triggered.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not ha
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/354#issuecomment-40547232
Merged build started.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/354#issuecomment-40551904
Refer to this link for build results:
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/14158/
---
If your project is set up for it, you can r
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/354#issuecomment-40551902
Merged build finished.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/354#issuecomment-40554839
Merged build triggered.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not ha
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/354#issuecomment-40554849
Merged build started.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/354#issuecomment-40558276
All automated tests passed.
Refer to this link for build results:
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/14162/
---
If your project
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/354#issuecomment-40558275
Merged build finished. All automated tests passed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If
Github user rxin commented on the pull request:
https://github.com/apache/spark/pull/354#issuecomment-40559997
Thanks. Merged.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
ena
Github user asfgit closed the pull request at:
https://github.com/apache/spark/pull/354
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabl
Github user mateiz commented on the pull request:
https://github.com/apache/spark/pull/354#issuecomment-40782397
@marmbrus do we need to add these new methods to the Python API too now? It
sounds like you just need to forward through to the methods.
---
If your project is set up for
Github user marmbrus commented on the pull request:
https://github.com/apache/spark/pull/354#issuecomment-40831481
@mateiz, yes we do. @ahirreddy was looking into that (but if you are too
busy with other things, Ahir, let me know and I'll take care of it).
---
If your project is set
44 matches
Mail list logo
