[GitHub] spark pull request #13890: [SPARK-16189][SQL] Add ExternalRDD logical plan f...
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/13890 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request #13890: [SPARK-16189][SQL] Add ExternalRDD logical plan f...
Github user marmbrus commented on a diff in the pull request:
https://github.com/apache/spark/pull/13890#discussion_r70139445
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/ExistingRDD.scala ---
@@ -74,13 +74,71 @@ object RDDConversions {
}
}
+private[sql] object ExternalRDD {
+
+ def apply[T: Encoder](rdd: RDD[T])(session: SparkSession): LogicalPlan =
{
--- End diff --
There is probably no reason to use multiple parameter lists here. We
sometimes use it for case classes so that arguments that should not effect
equality are not include in the generated `equals` method.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request #13890: [SPARK-16189][SQL] Add ExternalRDD logical plan f...
Github user cloud-fan commented on a diff in the pull request:
https://github.com/apache/spark/pull/13890#discussion_r70025823
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/ExistingRDD.scala ---
@@ -74,13 +74,71 @@ object RDDConversions {
}
}
+private[sql] object ExternalRDD {
+
+ def apply[T: Encoder](rdd: RDD[T])(session: SparkSession): LogicalPlan =
{
--- End diff --
Actually I'm not sure why we curry the constructor either. Since `RDDScan`
does it, it's ok we follow it. But for this apply `method`, I don't see the
value of doing it. cc @yhuai
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request #13890: [SPARK-16189][SQL] Add ExternalRDD logical plan f...
Github user ueshin commented on a diff in the pull request:
https://github.com/apache/spark/pull/13890#discussion_r70025315
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/ExistingRDD.scala ---
@@ -74,13 +74,71 @@ object RDDConversions {
}
}
+private[sql] object ExternalRDD {
+
+ def apply[T: Encoder](rdd: RDD[T])(session: SparkSession): LogicalPlan =
{
--- End diff --
Because I wanted to make the signature similar to the case class
constructor.
Should I uncurry?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request #13890: [SPARK-16189][SQL] Add ExternalRDD logical plan f...
Github user cloud-fan commented on a diff in the pull request:
https://github.com/apache/spark/pull/13890#discussion_r70024013
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/ExistingRDD.scala ---
@@ -74,13 +74,71 @@ object RDDConversions {
}
}
+private[sql] object ExternalRDD {
+
+ def apply[T: Encoder](rdd: RDD[T])(session: SparkSession): LogicalPlan =
{
--- End diff --
why curry here?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
