Sent from my iPhone
> On Sep 15, 2016, at 1:37 PM, Chen, Kevin <kevin.c...@neustar.biz> wrote: > > Hi, > > Has any one encountered an issue of missing output partition file in S3 ? My > spark job writes output to a S3 location. Occasionally, I noticed one > partition file is missing. As a result, one chunk of data was lost. If I > rerun the same job, the problem usually goes away. This has been happening > pretty random. I observed once or twice a week on a daily run job. I am using > Spark 1.2.1. > > Very much appreciated on any input, suggestion of fix/workaround. > > >
