[
https://issues.apache.org/jira/browse/SPARK-53656?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18021907#comment-18021907
]
Ganesha S commented on SPARK-53656:
-----------------------------------
[~anandexplore] Thanks for reviewing this. Please see the answers below.
??Why was {{MemoryStream}} originally built on {{SQLContext}} instead of
{{{}SparkSession{}}}???
Because SQLContext was the main API for Spark SQL and Structured Streaming when
MemoryStream was introduced, before SparkSession existed.
??What are the key benefits of switching to {{SparkSession}} now???
This change aligns {{MemoryStream}} with Spark’s current best practices,
improves code clarity, and ensures better compatibility with future Spark
features by moving away from the older, more fragmented {{SQLContext}} APIs.
> Refactor MemoryStream to use SparkSession instead of SQLContext
> ---------------------------------------------------------------
>
> Key: SPARK-53656
> URL: https://issues.apache.org/jira/browse/SPARK-53656
> Project: Spark
> Issue Type: Task
> Components: SQL
> Affects Versions: 4.0.1
> Reporter: Ganesha S
> Priority: Major
> Labels: pull-request-available
>
> Update the {{MemoryStream}} object to replace {{SQLContext}} with
> {{{}SparkSession{}}}, as {{SQLContext}} is deprecated in newer versions of
> Spark.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
