Hi,
also please ensure that you have read all the required documentation to
understand whether you need to do any metadata migration or not.
Regards,
Gourav Sengupta
On Sun, Dec 19, 2021 at 11:55 AM Alex Ott wrote:
> Make sure that you're using compatible version of Delta Lake library. For
>
Make sure that you're using compatible version of Delta Lake library. For
Spark 3.2 it's 1.1.0: https://github.com/delta-io/delta/releases/tag/v1.1.0
For other Spark versions, check this table:
https://docs.delta.io/latest/releases.html
On Fri, Dec 17, 2021 at 2:36 PM Stelios Philippou
wrote:
>
Hi Abhinav,
Using ReadStream or Read will not mind.
The following error
java.lang.NoSuchMethodError:
org.apache.spark.sql.execution.datasources.parquet.ParquetSchemaConverter$.checkFieldNames(
states that you are using different version of Spark at someplace of your
project or you are using an
May I ask why you don’t use spark.read and spark.write instead of
readStream and writeStream? Thanks.
On 2021-12-17 15:09, Abhinav Gundapaneni wrote:
Hello Spark community,
I’m using Apache spark(version 3.2) to read a CSV file to a
dataframe using ReadStream, process the dataframe and write
Hello Spark community,
I’m using Apache spark(version 3.2) to read a CSV file to a dataframe using
ReadStream, process the dataframe and write the dataframe to Delta file using
WriteStream. I’m getting a failure during the WriteStream process. I’m trying
to run the script locally in my windows
