Spark Image resizing

Hi I'm new to spark image data source. After creating a dataframe using Spark's image data source, I would like to resize the images in PySpark. df = spark.read.format("image").load(imageDir) Can you please help me with this? Nick

Kafka Integration libraries put in the fat jar

Hello! I would like to use the spark structured streaming integrated with Kafka the way is described here: https://spark.apache.org/docs/latest/structured-streaming-kafka-integration.html but I got the following issue: Caused by: org.apache.spark.sql.AnalysisException: Failed to find data