[
https://issues.apache.org/jira/browse/BAHIR-138?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16292950#comment-16292950
]
ASF GitHub Bot commented on BAHIR-138:
--------------------------------------
GitHub user emlaver opened a pull request:
https://github.com/apache/bahir/pull/59
[BAHIR-138] fix deprecated warnings in sql-cloudant
_What_
Fix warnings in `DefaultSource` class, and in `CloudantStreaming` and
`CloudantStreamingSelector` examples.
_How_
- Imported `spark.implicits._` to convert Spark RDD to Dataset
- Replaced deprecated `json(RDD[String])` with `json(Dataset[String])`
Improved streaming examples:
- Replaced registerTempTable with preferred createOrReplaceTempView
- Replaced !isEmpty with nonEmpty
- Use an accessible 'sales' database so users can run the example without
any setup
- Fixed error message when stopping tests by adding logic to streaming
receiver to not store documents in Spark memory when stream has stopped
See [BAHIR-138](https://issues.apache.org/jira/browse/BAHIR-138)
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/emlaver/bahir
138-fix-deprecated-warnings-sql-cloudant
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/bahir/pull/59.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #59
----
commit 88e239c8ab1a24ef39ca969cd11c1834ccaa1931
Author: Esteban Laver <[email protected]>
Date: 2017-10-02T20:18:40Z
Fix deprecation warning messages
Streaming example fixes:
- Replaced registerTempTable with preferred createOrReplaceTempView
- Replaced !isEmpty with nonEmpty
commit 19a0e15328c6b3026320b4cf6ff9a9255cc1ac9c
Author: Esteban Laver <[email protected]>
Date: 2017-12-15T17:44:50Z
Improved streaming tests by using an accessible 'sales' database
- Added logic to streaming receiver to not store doc in Spark memory when
stream has stopped
----
> Fix sql-cloudant deprecation messages
> -------------------------------------
>
> Key: BAHIR-138
> URL: https://issues.apache.org/jira/browse/BAHIR-138
> Project: Bahir
> Issue Type: Task
> Components: Spark Structured Streaming Connectors
> Affects Versions: Spark-2.2.0
> Reporter: Esteban Laver
> Assignee: Esteban Laver
> Priority: Minor
> Labels: warnings
>
> Deprecation warnings in {{DefaultSource}}:
> {code}
> [INFO] --- scala-maven-plugin:3.2.2:compile (scala-compile-first) @
> spark-sql-cloudant_2.11 ---
> [INFO] Compiling 11 Scala sources to sql-cloudant/target/scala-2.11/classes...
> [WARNING]
> sql-cloudant/src/main/scala/org/apache/bahir/cloudant/DefaultSource.scala:59:
> method json in class DataFrameReader is deprecated: Use json(Dataset[String])
> instead.
> [WARNING] val df = sqlContext.read.json(cloudantRDD)
> [WARNING] ^
> [WARNING]
> sql-cloudant/src/main/scala/org/apache/bahir/cloudant/DefaultSource.scala:115:
> method json in class DataFrameReader is deprecated: Use
> json(Dataset[String]) instead.
> [WARNING] dataFrame = sqlContext.read.json(cloudantRDD)
> [WARNING] ^
> [WARNING]
> sql-cloudant/src/main/scala/org/apache/bahir/cloudant/DefaultSource.scala:121:
> method json in class DataFrameReader is deprecated: Use
> json(Dataset[String]) instead.
> [WARNING] sqlContext.read.json(aRDD)
> [WARNING] ^
> [WARNING]
> sql-cloudant/src/main/scala/org/apache/bahir/cloudant/DefaultSource.scala:152:
> method json in class DataFrameReader is deprecated: Use
> json(Dataset[String]) instead.
> [WARNING] dataFrame =
> sqlContext.sparkSession.read.json(globalRDD)
> [WARNING] ^
> [WARNING] four warnings found
> {code}
> Deprecation warnings in {{CloudantStreaming}} and
> {{CloudantStreamingSelector}} examples:
> {code}
> [INFO] --- scala-maven-plugin:3.2.2:testCompile (scala-test-compile-first) @
> spark-sql-cloudant_2.11 ---
> [INFO] Compiling 11 Scala sources to
> sql-cloudant/target/scala-2.11/test-classes...
> [WARNING]
> sql-cloudant/examples/src/main/scala/org/apache/spark/examples/sql/cloudant/CloudantStreaming.scala:46:
> method json in class DataFrameReader is deprecated: Use
> json(Dataset[String]) instead.
> [WARNING] val changesDataFrame = spark.read.json(rdd)
> [WARNING] ^
> [WARNING]
> sql-cloudant/examples/src/main/scala/org/apache/spark/examples/sql/cloudant/CloudantStreaming.scala:67:
> method registerTempTable in class Dataset is deprecated: Use
> createOrReplaceTempView(viewName) instead.
> [WARNING] changesDataFrame.registerTempTable("airportcodemapping")
> [WARNING] ^
> [WARNING]
> sql-cloudant/examples/src/main/scala/org/apache/spark/examples/sql/cloudant/CloudantStreamingSelector.scala:50:
> method json in class DataFrameReader is deprecated: Use
> json(Dataset[String]) instead.
> [WARNING] val changesDataFrame = spark.read.json(rdd)
> [WARNING] ^
> [WARNING] three warnings found
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
