[jira] [Updated] (SPARK-44027) create *permanent* Spark View from DataFrame via PySpark & Scala DataFrame API

2023-06-12 Thread Martin Bode (Jira) 


 [ 
https://issues.apache.org/jira/browse/SPARK-44027?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Martin Bode updated SPARK-44027:

Description: 
currently only *_temporary_ Spark Views* can be created from a DataFrame:
 * 
[DataFrame.createTempView|https://spark.apache.org/docs/latest/api/python/reference/pyspark.sql/api/pyspark.sql.DataFrame.createTempView.html#pyspark.sql.DataFrame.createTempView]
 * 
[DataFrame.createOrReplaceTempView|https://spark.apache.org/docs/latest/api/python/reference/pyspark.sql/api/pyspark.sql.DataFrame.createOrReplaceTempView.html#pyspark.sql.DataFrame.createOrReplaceTempView]
 * 
[DataFrame.createGlobalTempView|https://spark.apache.org/docs/latest/api/python/reference/pyspark.sql/api/pyspark.sql.DataFrame.createGlobalTempView.html#pyspark.sql.DataFrame.createGlobalTempView]
 * 
[DataFrame.createOrReplaceGlobalTempView|https://spark.apache.org/docs/latest/api/python/reference/pyspark.sql/api/pyspark.sql.DataFrame.createOrReplaceGlobalTempView.html#pyspark.sql.DataFrame.createOrReplaceGlobalTempView]

When a user needs a _*permanent*_ *Spark View* he has to fall back to Spark SQL 
({{{}CREATE VIEW AS SELECT...{}}}).

Sometimes it is easier and more readable to specify the desired logic of the 
view through {_}Scala/PySpark DataFrame API{_}.
Therefore, I'd like to suggest to implement a new PySpark method that allows 
creating a _*permanent*_ *Spark View* from a DataFrame (e.g. 
{{{}DataFrame.createOrReplaceView{}}}).

see also:
 * 
[https://community.databricks.com/s/question/0D53f1PANVgCAP/is-there-a-way-to-create-a-nontemporary-spark-view-with-pyspark]
 * [https://lists.apache.org/thread/jzkznvt7cfjhmo77w1tlksxkwyvmvvfb]

  was:
currently only *_temporary_ Spark Views* can be created from a DataFrame:
 * 
[DataFrame.createGlobalTempView|https://spark.apache.org/docs/latest/api/python/reference/pyspark.sql/api/pyspark.sql.DataFrame.createGlobalTempView.html#pyspark.sql.DataFrame.createGlobalTempView]
 * 
[DataFrame.createOrReplaceGlobalTempView|https://spark.apache.org/docs/latest/api/python/reference/pyspark.sql/api/pyspark.sql.DataFrame.createOrReplaceGlobalTempView.html#pyspark.sql.DataFrame.createOrReplaceGlobalTempView]
 * 
[DataFrame.createOrReplaceTempView|https://spark.apache.org/docs/latest/api/python/reference/pyspark.sql/api/pyspark.sql.DataFrame.createOrReplaceTempView.html#pyspark.sql.DataFrame.createOrReplaceTempView]
 * 
[DataFrame.createTempView|https://spark.apache.org/docs/latest/api/python/reference/pyspark.sql/api/pyspark.sql.DataFrame.createTempView.html#pyspark.sql.DataFrame.createTempView]

When a user needs a _*permanent*_ *Spark View* he has to fall back to Spark SQL 
({{{}CREATE VIEW AS SELECT...{}}}).

Sometimes it is easier and more readable to specify the desired logic of the 
view through {_}Scala/PySpark DataFrame API{_}.
Therefore, I'd like to suggest to implement a new PySpark method that allows 
creating a _*permanent*_ *Spark View* from a DataFrame (e.g. 
{{{}DataFrame.createOrReplaceView{}}}).

see also:
* 
[https://community.databricks.com/s/question/0D53f1PANVgCAP/is-there-a-way-to-create-a-nontemporary-spark-view-with-pyspark]
* https://lists.apache.org/thread/jzkznvt7cfjhmo77w1tlksxkwyvmvvfb


> create *permanent* Spark View from DataFrame via PySpark & Scala DataFrame API
> --
>
> Key: SPARK-44027
> URL: https://issues.apache.org/jira/browse/SPARK-44027
> Project: Spark
>  Issue Type: New Feature
>  Components: PySpark
>Affects Versions: 3.5.0
>Reporter: Martin Bode
>Priority: Major
>  Labels: features, newbie
>
> currently only *_temporary_ Spark Views* can be created from a DataFrame:
>  * 
> [DataFrame.createTempView|https://spark.apache.org/docs/latest/api/python/reference/pyspark.sql/api/pyspark.sql.DataFrame.createTempView.html#pyspark.sql.DataFrame.createTempView]
>  * 
> [DataFrame.createOrReplaceTempView|https://spark.apache.org/docs/latest/api/python/reference/pyspark.sql/api/pyspark.sql.DataFrame.createOrReplaceTempView.html#pyspark.sql.DataFrame.createOrReplaceTempView]
>  * 
> [DataFrame.createGlobalTempView|https://spark.apache.org/docs/latest/api/python/reference/pyspark.sql/api/pyspark.sql.DataFrame.createGlobalTempView.html#pyspark.sql.DataFrame.createGlobalTempView]
>  * 
> [DataFrame.createOrReplaceGlobalTempView|https://spark.apache.org/docs/latest/api/python/reference/pyspark.sql/api/pyspark.sql.DataFrame.createOrReplaceGlobalTempView.html#pyspark.sql.DataFrame.createOrReplaceGlobalTempView]
> When a user needs a _*permanent*_ *Spark View* he has to fall back to Spark 
> SQL ({{{}CREATE VIEW AS SELECT...{}}}).
> Sometimes it is easier and more readable to specify the desired logic of the 
> view through {_}Scala/PySpark DataFrame API{_}.
> Therefore, I'd like to

[jira] [Updated] (SPARK-44027) create *permanent* Spark View from DataFrame via PySpark & Scala DataFrame API

2023-06-12 Thread Martin Bode (Jira) 


 [ 
https://issues.apache.org/jira/browse/SPARK-44027?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Martin Bode updated SPARK-44027:

Affects Version/s: 3.5.0
   (was: 3.4.0)

> create *permanent* Spark View from DataFrame via PySpark & Scala DataFrame API
> --
>
> Key: SPARK-44027
> URL: https://issues.apache.org/jira/browse/SPARK-44027
> Project: Spark
>  Issue Type: New Feature
>  Components: PySpark
>Affects Versions: 3.5.0
>Reporter: Martin Bode
>Priority: Major
>  Labels: features, newbie
>
> currently only *_temporary_ Spark Views* can be created from a DataFrame:
>  * 
> [DataFrame.createGlobalTempView|https://spark.apache.org/docs/latest/api/python/reference/pyspark.sql/api/pyspark.sql.DataFrame.createGlobalTempView.html#pyspark.sql.DataFrame.createGlobalTempView]
>  * 
> [DataFrame.createOrReplaceGlobalTempView|https://spark.apache.org/docs/latest/api/python/reference/pyspark.sql/api/pyspark.sql.DataFrame.createOrReplaceGlobalTempView.html#pyspark.sql.DataFrame.createOrReplaceGlobalTempView]
>  * 
> [DataFrame.createOrReplaceTempView|https://spark.apache.org/docs/latest/api/python/reference/pyspark.sql/api/pyspark.sql.DataFrame.createOrReplaceTempView.html#pyspark.sql.DataFrame.createOrReplaceTempView]
>  * 
> [DataFrame.createTempView|https://spark.apache.org/docs/latest/api/python/reference/pyspark.sql/api/pyspark.sql.DataFrame.createTempView.html#pyspark.sql.DataFrame.createTempView]
> When a user needs a _*permanent*_ *Spark View* he has to fall back to Spark 
> SQL ({{{}CREATE VIEW AS SELECT...{}}}).
> Sometimes it is easier and more readable to specify the desired logic of the 
> view through {_}Scala/PySpark DataFrame API{_}.
> Therefore, I'd like to suggest to implement a new PySpark method that allows 
> creating a _*permanent*_ *Spark View* from a DataFrame (e.g. 
> {{{}DataFrame.createOrReplaceView{}}}).
> see also:
> * 
> [https://community.databricks.com/s/question/0D53f1PANVgCAP/is-there-a-way-to-create-a-nontemporary-spark-view-with-pyspark]
> * https://lists.apache.org/thread/jzkznvt7cfjhmo77w1tlksxkwyvmvvfb



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org

[jira] [Updated] (SPARK-44027) create *permanent* Spark View from DataFrame via PySpark & Scala DataFrame API

2023-06-12 Thread Martin Bode (Jira) 


 [ 
https://issues.apache.org/jira/browse/SPARK-44027?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Martin Bode updated SPARK-44027:

Description: 
currently only *_temporary_ Spark Views* can be created from a DataFrame:
 * 
[DataFrame.createGlobalTempView|https://spark.apache.org/docs/latest/api/python/reference/pyspark.sql/api/pyspark.sql.DataFrame.createGlobalTempView.html#pyspark.sql.DataFrame.createGlobalTempView]
 * 
[DataFrame.createOrReplaceGlobalTempView|https://spark.apache.org/docs/latest/api/python/reference/pyspark.sql/api/pyspark.sql.DataFrame.createOrReplaceGlobalTempView.html#pyspark.sql.DataFrame.createOrReplaceGlobalTempView]
 * 
[DataFrame.createOrReplaceTempView|https://spark.apache.org/docs/latest/api/python/reference/pyspark.sql/api/pyspark.sql.DataFrame.createOrReplaceTempView.html#pyspark.sql.DataFrame.createOrReplaceTempView]
 * 
[DataFrame.createTempView|https://spark.apache.org/docs/latest/api/python/reference/pyspark.sql/api/pyspark.sql.DataFrame.createTempView.html#pyspark.sql.DataFrame.createTempView]

When a user needs a _*permanent*_ *Spark View* he has to fall back to Spark SQL 
({{{}CREATE VIEW AS SELECT...{}}}).

Sometimes it is easier and more readable to specify the desired logic of the 
view through {_}Scala/PySpark DataFrame API{_}.
Therefore, I'd like to suggest to implement a new PySpark method that allows 
creating a _*permanent*_ *Spark View* from a DataFrame (e.g. 
{{{}DataFrame.createOrReplaceView{}}}).

see also:
* 
[https://community.databricks.com/s/question/0D53f1PANVgCAP/is-there-a-way-to-create-a-nontemporary-spark-view-with-pyspark]
* https://lists.apache.org/thread/jzkznvt7cfjhmo77w1tlksxkwyvmvvfb

  was:
currently only *_temporary_ Spark Views* can be created from a DataFrame:
 * 
[DataFrame.createGlobalTempView|https://spark.apache.org/docs/latest/api/python/reference/pyspark.sql/api/pyspark.sql.DataFrame.createGlobalTempView.html#pyspark.sql.DataFrame.createGlobalTempView]
 * 
[DataFrame.createOrReplaceGlobalTempView|https://spark.apache.org/docs/latest/api/python/reference/pyspark.sql/api/pyspark.sql.DataFrame.createOrReplaceGlobalTempView.html#pyspark.sql.DataFrame.createOrReplaceGlobalTempView]
 * 
[DataFrame.createOrReplaceTempView|https://spark.apache.org/docs/latest/api/python/reference/pyspark.sql/api/pyspark.sql.DataFrame.createOrReplaceTempView.html#pyspark.sql.DataFrame.createOrReplaceTempView]
 * 
[DataFrame.createTempView|https://spark.apache.org/docs/latest/api/python/reference/pyspark.sql/api/pyspark.sql.DataFrame.createTempView.html#pyspark.sql.DataFrame.createTempView]

When a user needs a _*permanent*_ *Spark View* he has to fall back to Spark SQL 
({{{}CREATE VIEW AS SELECT...{}}}).

Sometimes it is easier and more readable to specify the desired logic of the 
view through {_}Scala/PySpark DataFrame API{_}.
Therefore, I'd like to suggest to implement a new PySpark method that allows 
creating a _*permanent*_ *Spark View* from a DataFrame (e.g. 
{{{}DataFrame.createOrReplaceView{}}}).

see also:
[https://community.databricks.com/s/question/0D53f1PANVgCAP/is-there-a-way-to-create-a-nontemporary-spark-view-with-pyspark]


> create *permanent* Spark View from DataFrame via PySpark & Scala DataFrame API
> --
>
> Key: SPARK-44027
> URL: https://issues.apache.org/jira/browse/SPARK-44027
> Project: Spark
>  Issue Type: New Feature
>  Components: PySpark
>Affects Versions: 3.4.0
>Reporter: Martin Bode
>Priority: Major
>  Labels: features, newbie
>
> currently only *_temporary_ Spark Views* can be created from a DataFrame:
>  * 
> [DataFrame.createGlobalTempView|https://spark.apache.org/docs/latest/api/python/reference/pyspark.sql/api/pyspark.sql.DataFrame.createGlobalTempView.html#pyspark.sql.DataFrame.createGlobalTempView]
>  * 
> [DataFrame.createOrReplaceGlobalTempView|https://spark.apache.org/docs/latest/api/python/reference/pyspark.sql/api/pyspark.sql.DataFrame.createOrReplaceGlobalTempView.html#pyspark.sql.DataFrame.createOrReplaceGlobalTempView]
>  * 
> [DataFrame.createOrReplaceTempView|https://spark.apache.org/docs/latest/api/python/reference/pyspark.sql/api/pyspark.sql.DataFrame.createOrReplaceTempView.html#pyspark.sql.DataFrame.createOrReplaceTempView]
>  * 
> [DataFrame.createTempView|https://spark.apache.org/docs/latest/api/python/reference/pyspark.sql/api/pyspark.sql.DataFrame.createTempView.html#pyspark.sql.DataFrame.createTempView]
> When a user needs a _*permanent*_ *Spark View* he has to fall back to Spark 
> SQL ({{{}CREATE VIEW AS SELECT...{}}}).
> Sometimes it is easier and more readable to specify the desired logic of the 
> view through {_}Scala/PySpark DataFrame API{_}.
> Therefore, I'd like to suggest to implement a new PySpark method that allows 
> creating a

[jira] [Created] (SPARK-44027) create *permanent* Spark View from DataFrame via PySpark & Scala DataFrame API

2023-06-12 Thread Martin Bode (Jira) 
Martin Bode created SPARK-44027:
---

 Summary: create *permanent* Spark View from DataFrame via PySpark 
& Scala DataFrame API
 Key: SPARK-44027
 URL: https://issues.apache.org/jira/browse/SPARK-44027
 Project: Spark
  Issue Type: New Feature
  Components: PySpark
Affects Versions: 3.4.0
Reporter: Martin Bode


currently only *_temporary_ Spark Views* can be created from a DataFrame:
 * 
[DataFrame.createGlobalTempView|https://spark.apache.org/docs/latest/api/python/reference/pyspark.sql/api/pyspark.sql.DataFrame.createGlobalTempView.html#pyspark.sql.DataFrame.createGlobalTempView]
 * 
[DataFrame.createOrReplaceGlobalTempView|https://spark.apache.org/docs/latest/api/python/reference/pyspark.sql/api/pyspark.sql.DataFrame.createOrReplaceGlobalTempView.html#pyspark.sql.DataFrame.createOrReplaceGlobalTempView]
 * 
[DataFrame.createOrReplaceTempView|https://spark.apache.org/docs/latest/api/python/reference/pyspark.sql/api/pyspark.sql.DataFrame.createOrReplaceTempView.html#pyspark.sql.DataFrame.createOrReplaceTempView]
 * 
[DataFrame.createTempView|https://spark.apache.org/docs/latest/api/python/reference/pyspark.sql/api/pyspark.sql.DataFrame.createTempView.html#pyspark.sql.DataFrame.createTempView]

When a user needs a _*permanent*_ *Spark View* he has to fall back to Spark SQL 
({{{}CREATE VIEW AS SELECT...{}}}).

Sometimes it is easier and more readable to specify the desired logic of the 
view through {_}Scala/PySpark DataFrame API{_}.
Therefore, I'd like to suggest to implement a new PySpark method that allows 
creating a _*permanent*_ *Spark View* from a DataFrame (e.g. 
{{{}DataFrame.createOrReplaceView{}}}).

see also:
[https://community.databricks.com/s/question/0D53f1PANVgCAP/is-there-a-way-to-create-a-nontemporary-spark-view-with-pyspark]



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org