Re: [DISCUSS] Multiple columns adding/replacing support in PySpark DataFrame API

2021-04-29 Thread Saurabh Chawla
Hi All, I also had a scenario where at runtime, I needed to loop through a dataframe to use withColumn many times. For the safer side I used the reflection to access the withColumns to prevent any java.lang.StackOverflowError. val dataSetClass = Class.forName("org.apache.spark.sql.Dataset")

Re: Should we add built in support for bouncy castle EC w/Kube

2021-04-29 Thread Sean Owen
I recall that Bouncy Castle has some crypto export implications. If it's in the distro then I think we'd have to update https://www.apache.org/licenses/exports/ to reflect that Bouncy Castle is again included in the product. But that's doable. Just have to recall how one updates that. On Thu, Apr

Should we add built in support for bouncy castle EC w/Kube

2021-04-29 Thread Holden Karau
Hi Folks, I've deployed a new version of K3s locally and I ran into an issue with the key format not being supported out of the box. We delegate to fabric8 which has bouncy castle EC as an optional dependency. Adding it would add ~6mb to the Kube jars. What do folks think? Cheers, Holden P.S.

Re: Bintray replacement for spark-packages.org

2021-04-29 Thread Dongjoon Hyun
I agree with Wenchen. I can volunteer for Apache Spark 3.1.2 release manager at least. Bests, Dongjoon. On Wed, Apr 28, 2021 at 10:15 AM Wenchen Fan wrote: > Shall we make new releases for 3.0 and 3.1? So that people don't need to > change the sbt resolver/pom files to work around Bintray