Re: [Feature Request] create *permanent* Spark View from DataFrame via PySpark

2023-06-04 Thread Mich Talebzadeh
Try sending it to dev@spark.apache.org (and join that group) You need to raise a JIRA for this request plus related doc related Example JIRA https://issues.apache.org/jira/browse/SPARK-42485 and the related *Spark project improvement proposals (SPIP) *to be filled in https://spark.apache.org/

Re: Apache Spark 4.0 Timeframe?

2023-06-04 Thread Dongjoon Hyun
Thank you, Yangjie. Yes, instead of switching, it will depend on the Scala community's promised compatibility completely. However, many other difficulties still arise from the 3rd-party libraries like Ammonite. Scala 3 might be better in long term perspective. I'll send another email about the c