.com
>
>
> *Disclaimer:* Use it at your own risk. Any and all responsibility for any
> loss, damage or destruction of data or any other property which may arise
> from relying on this email's technical content is explicitly disclaimed.
> The author will in no case be liable for any mone
this email's technical content is explicitly disclaimed.
> The author will in no case be liable for any monetary damages arising from
> such loss, damage or destruction.
>
>
>
> On 17 September 2016 at 13:59, Qiang Li <q...@appannie.com> wrote:
>
>> Hi,
>>
&g
Hi,
I use spark to generate data , then we use hive/pig/presto/spark to analyze
data, but I found even I add used bucketBy and sortBy with bucket number in
Spark, the results files was generate by Spark is always far more than
bucket number under each partition, then Presto can not recognize the
; https://www.mail-archive.com/user@spark.apache.org/msg56791.html
>
> // maropu
>
>
> On Sat, Sep 17, 2016 at 11:34 AM, Qiang Li <q...@appannie.com> wrote:
>
>> Hi,
>>
>>
>> I ran some jobs with Spark 2.0 on Yarn, I found all tasks finished very
>> qui
Hi,
I ran some jobs with Spark 2.0 on Yarn, I found all tasks finished very
quickly, but the last step, spark spend lots of time to rename or move data
from s3 temporary directory to real directory, then I try to set
