Unsubscribe

Re: Pyspark UDF as a data source for streaming

Hi, Do you have more info on this Jira besides the github link as I don't seem to find it! Thanks Mich Talebzadeh, Dad | Technologist | Solutions Architect | Engineer London United Kingdom view my Linkedin profile

Re: the life cycle shuffle Dependency

Hello, why would you like to delete the shuffle data yourself in the first place? On Thu, Dec 28, 2023, 10:08 yang chen wrote: > > hi, I'm learning spark, and wonder when to delete shuffle data, I find the > ContextCleaner class which clean the shuffle data when shuffle dependency > is GC-ed.

Select Columns from Dataframe in Java

Team I am using Java and want to select columns from Dataframe , columns are stored in List equivalent of below scala code * array_df=array_df.select(fields: _*)* When I try array_df=array_df.select(fields) , I get error saying Cast to Column I am using Spark 3.4