Hi,
I am trying to read the multiple parquet files in sparksql. In one dir there
are two files, of which one is corrupted. While trying to read these files,
sparksql throws Exception for the corrupted file.
val newDataDF =
sqlContext.read.parquet("/data/testdir/data1.parquet","/data/testdir/corruptblock.0")
newDataDF.show
throws Exception.
Is there any way to just skip the file having corrupted block/footer and
just read the file/files which are proper?
Thanks
--
View this message in context:
http://apache-spark-developers-list.1001551.n3.nabble.com/Skip-Corrupted-Parquet-blocks-footer-tp20418.html
Sent from the Apache Spark Developers List mailing list archive at Nabble.com.
---------------------------------------------------------------------
To unsubscribe e-mail: dev-unsubscr...@spark.apache.org
