Hi all,
I am concerned that the API-breaking changes in SPARK-25908 (as well as
SPARK-16775, and potentially others) will make the migration process from
Spark 2 to Spark 3 unnecessarily painful. For example, the removal of
SQLContext.getOrCreate will break a large number of libraries currently
bu
+1 to have Spark docker images for Dongjoon's arguments, having a container
based distribution is definitely something in the benefit of users and the
project too. Having this in the Apache Spark repo matters because of
multiple
eyes to fix/ímprove the images for the benefit of everyone.
What stil
Hi,
I am trying to connect to DB2 using spark with following code
spark.sparkContext.addFile("xyz.jks")
spark.sparkContext.addFile("xyz.pfx")
val url =
s"""jdbc:db2://host:port/schema:securityMechanism=18;sslConnection=true;user=user;sslTrustStoreLocation=${SparkFiles.get("xyz.jks")};sslKeyStore
I don't know what's the best way to deprecate an SQL function. Runtime
warning can be annoying if it keeps coming out. Maybe we should only log
the warning once per Spark application.
On Tue, Feb 18, 2020 at 3:45 PM Dongjoon Hyun
wrote:
> Thank you for feedback, Wenchen, Maxim, and Takeshi.
>
>
