# python:

import pandas as pd

a = pd.DataFrame([[1, [2.3, 1.2]]], columns=['a', 'b'])
a.to_parquet('a.parquet')

# pyspark:

d2 = spark.read.parquet('a.parquet')

will return error:

An error was encountered: An error occurred while calling o277.showString. :
org.apache.spark.SparkException: Job aborted due to stage failure: Task 14
in stage 9.0 failed 4 times, most recent failure: Lost task 14.2 in stage
9.0 (TID 63, 10.169.0.196, executor 15): java.lang.IllegalArgumentException:
Illegal Capacity: -221

how can I fix it?

Thanks.

 

Reply via email to