My guess is that the committer should call super.
public class MapredParquetOutputCommitter extends FileOutputCommitter {
@Override
public void commitJob(JobContext jobContext) throws IOException {
super.commitJob(jobContext);
...
}
}
Tianshuo, do you prefer trying to fix it or reverting?
We should not keep a failing build in master
On Fri, Mar 6, 2015 at 11:08 AM, Ryan Blue <[email protected]> wrote:
> It looks like the hadoop-2 tests are breaking in master from this commit:
>
> https://github.com/apache/incubator-parquet-mr/commit/5851e6da
>
> The problem is that mergeFooters doesn't find any footers. In Hadoop-1,
> the data files are in the output directory, but on hadoop-2 the output file
> is here:
>
>
> target/test/ParquetTBaseScheme/names-parquet-out/_temporary/0/task_
> local567920263_0001_m_000000/part-00000-m-00000.parquet
>
> Anyone want to volunteer to fix it? I'm happy to revert the change, since
> I'm not sure what the right fix for the bug is.
>
> Also, this is a good time to remind everyone to please check Travis CI or
> run the hadoop-2 profile locally before merging. Thanks!
>
> rb
>
> --
> Ryan Blue
> Software Engineer
> Cloudera, Inc.
>
