i have panda dataframe with column 'image' using numpy.ndarray. shape is (500,
333, 3) per image. my panda dataframe has 10 rows, thus, shape is (10, 500,
333, 3)
when using spark.createDataframe(panda_dataframe, schema), i need to specify
the schema,
schema = StructType([
StructField("image",
ArrayType(ArrayType(ArrayType(ArrayType(IntegerType())))), nullable=False)
])
i get error
raise TypeError(
, TypeError: field image:
ArrayType(ArrayType(ArrayType(ArrayType(IntegerType(), True), True), True),
True) can not accept object array([[[14, 14, 14],...
Can advise how to set schema for image with numpy.ndarray ?
