[
https://issues.apache.org/jira/browse/PHOENIX-6783?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17877614#comment-17877614
]
ASF GitHub Bot commented on PHOENIX-6783:
-----------------------------------------
rejeb commented on code in PR #139:
URL:
https://github.com/apache/phoenix-connectors/pull/139#discussion_r1735694996
##########
phoenix5-spark/src/it/scala/org/apache/phoenix/spark/PhoenixSparkDatasourceV1IT.scala:
##########
@@ -0,0 +1,807 @@
+/*
+ Licensed under the Apache License, Version 2.0 (the "License");
+ you may not use this file except in compliance with the License.
+ You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+ */
+package org.apache.phoenix.spark
+
+import org.apache.phoenix.mapreduce.util.PhoenixConfigurationUtil
+import org.apache.phoenix.query.QueryServices
+import org.apache.phoenix.schema.types.{PSmallintArray, PVarchar}
+import
org.apache.phoenix.spark.datasource.v2.reader.PhoenixTestingInputPartitionReader
+import
org.apache.phoenix.spark.datasource.v2.writer.PhoenixTestingDataSourceWriter
+import org.apache.phoenix.spark.datasource.v2.{PhoenixDataSource,
PhoenixTestingDataSource, SparkSchemaUtil}
+import org.apache.phoenix.util.{ColumnInfo, SchemaUtil}
+import org.apache.spark.SparkException
+import org.apache.spark.sql.types._
+import org.apache.spark.sql.{Row, SaveMode}
+
+import java.sql.DriverManager
+import java.util.Date
+import scala.collection.mutable
+import scala.collection.mutable.ListBuffer
+
+/**
+ * Note: If running directly from an IDE, these are the recommended VM
parameters:
+ * -Xmx1536m -XX:MaxPermSize=512m -XX:ReservedCodeCacheSize=512m
+ */
+class PhoenixSparkDatasourceV1IT extends AbstractPhoenixSparkIT {
Review Comment:
It is used for other tests.
> Phoenix5-Spark3 Connector Spark SQL Support
> -------------------------------------------
>
> Key: PHOENIX-6783
> URL: https://issues.apache.org/jira/browse/PHOENIX-6783
> Project: Phoenix
> Issue Type: Bug
> Components: connectors, spark-connector
> Environment: # Pure Spark 3.0.3
> # phoenix5-spark3-shaded-6.0.0-SNAPSHOT.jar located in $SPARK_HOME/jars
> # SQL used to create Spark Phoenix data source table is:
> {{create table Table1 (ID long, COL1 string) using phoenix options (table
> 'Table1', zkUrl '192.168.0.103:2181', primary 'ID')}}
> Reporter: yj
> Assignee: rejeb ben rejeb
> Priority: Major
>
> While conducting the Spark 3.0 Integration test using *Phoenix5-spark3-shaded
> module of project
> [Phoenix-Connectors|https://github.com/apache/phoenix-connectors]* the
> following questions occurred.
>
> The test process is as follows:
> 1. Phoenix created a table called "TABLE1" and then loaded the data into that
> table.
> 2. After that, I tried to select the data of the table using Spark.
>
> As written in the README of Phoenix5-spark3 module, the part that loads
> Phoenix data into the Spark DataFrame using the Phoenix data source seems to
> work well.
> However, when I *create Phoenix Spark data source table through Spark SQL*
> and try to select that data source table, the following error occurs:
> {{java.util.concurrent.ExecutionException:
> org.apache.spark.sql.AnalysisException: phoenix is not a valid Spark SQL Data
> Source}}
>
> I wonder if the Phoenix-Connectors does not support creating Phoenix Spark
> data source tables, or if there is any other reason for this error.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
