First test would be if you can write parquet files fine on HDFS from your
Spark job fine...If that also get's stuck then there is something with the
logic...If parquet files are dumped fine and u can load them on HBase then
there is something going on with Spark-HBase interaction
On Wed, Sep 24, 2
spark SQL reads parquet file fine...did you follow one of these to
read/write parquet from spark ?
http://zenfractal.com/2013/08/21/a-powerful-big-data-trio/
On Wed, Sep 24, 2014 at 9:29 AM, Ted Yu wrote:
> Adding a subject.
>
> bq. at parquet.hadoop.ParquetFileReader$
> ConsecutiveChunkL
Adding a subject.
bq. at parquet.hadoop.ParquetFileReader$ConsecutiveChunkList.readAll(
ParquetFileReader.java:599)
Looks like there might be some issue reading the Parquet file.
Cheers
On Wed, Sep 24, 2014 at 9:10 AM, Jianshi Huang
wrote:
> Hi Ted,
>
> See my previous reply to Debasish
