[
https://issues.apache.org/jira/browse/PHOENIX-6783?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17877764#comment-17877764
]
ASF GitHub Bot commented on PHOENIX-6783:
-----------------------------------------
rejeb commented on code in PR #139:
URL:
https://github.com/apache/phoenix-connectors/pull/139#discussion_r1736634460
##########
phoenix5-spark/README.md:
##########
@@ -89,7 +147,7 @@ public class PhoenixSparkRead {
### Save DataFrames to Phoenix using DataSourceV2
The `save` is method on DataFrame allows passing in a data source type. You
can use
-`phoenix` for DataSourceV2 and must also pass in a `table` and `zkUrl`
parameter to
+`phoenix` for DataSourceV2 and must also pass in a `table` and `jdbcUrl`
parameter to
Review Comment:
Done.
##########
phoenix5-spark/README.md:
##########
@@ -141,32 +201,32 @@ public class PhoenixSparkWriteFromInputTable {
public static void main() throws Exception {
SparkConf sparkConf = new
SparkConf().setMaster("local").setAppName("phoenix-test")
- .set("spark.hadoopRDD.ignoreEmptySplits", "false");
- JavaSparkContext jsc = new JavaSparkContext(sparkConf);
- SQLContext sqlContext = new SQLContext(jsc);
+ .set("spark.hadoopRDD.ignoreEmptySplits", "false");
+ SparkSessinon spark =
SparkSession.builder().config(sparkConf).getOrCreate();
// Load INPUT_TABLE
- Dataset<Row> df = sqlContext
+ Dataset<Row> df = spark
.read()
.format("phoenix")
.option("table", "INPUT_TABLE")
+ .option("jdbcUrl", "jdbc:phoenix:zkHost:zkport")
.load();
// Save to OUTPUT_TABLE
df.write()
.format("phoenix")
.mode(SaveMode.Overwrite)
.option("table", "OUTPUT_TABLE")
+ .option("jdbcUrl", "jdbc:phoenix:zkHost:zkport")
.save();
- jsc.stop();
}
}
```
### Save from an external RDD with a schema to a Phoenix table
Just like the previous example, you can pass in the data source type as
`phoenix` and specify the `table` and
-`zkUrl` parameters indicating which table and server to persist the DataFrame
to.
+`jdbcUrl` parameters indicating which table and server to persist the
DataFrame to.
Review Comment:
Done
> Phoenix5-Spark3 Connector Spark SQL Support
> -------------------------------------------
>
> Key: PHOENIX-6783
> URL: https://issues.apache.org/jira/browse/PHOENIX-6783
> Project: Phoenix
> Issue Type: Bug
> Components: connectors, spark-connector
> Environment: # Pure Spark 3.0.3
> # phoenix5-spark3-shaded-6.0.0-SNAPSHOT.jar located in $SPARK_HOME/jars
> # SQL used to create Spark Phoenix data source table is:
> {{create table Table1 (ID long, COL1 string) using phoenix options (table
> 'Table1', zkUrl '192.168.0.103:2181', primary 'ID')}}
> Reporter: yj
> Assignee: rejeb ben rejeb
> Priority: Major
>
> While conducting the Spark 3.0 Integration test using *Phoenix5-spark3-shaded
> module of project
> [Phoenix-Connectors|https://github.com/apache/phoenix-connectors]* the
> following questions occurred.
>
> The test process is as follows:
> 1. Phoenix created a table called "TABLE1" and then loaded the data into that
> table.
> 2. After that, I tried to select the data of the table using Spark.
>
> As written in the README of Phoenix5-spark3 module, the part that loads
> Phoenix data into the Spark DataFrame using the Phoenix data source seems to
> work well.
> However, when I *create Phoenix Spark data source table through Spark SQL*
> and try to select that data source table, the following error occurs:
> {{java.util.concurrent.ExecutionException:
> org.apache.spark.sql.AnalysisException: phoenix is not a valid Spark SQL Data
> Source}}
>
> I wonder if the Phoenix-Connectors does not support creating Phoenix Spark
> data source tables, or if there is any other reason for this error.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
