Yes, the explain plan definitely only has Move Operators (no Copy
Operators). With that though, this definitely looks like a hive bug? Does
anyone know if there is corresponding HIVE ticket or a workaround for the
issue? Thanks!
Stage: Stage-3
Move Operator
files:
hdfs directory: true
destination:
s3n://<access_key>:<secret_key>@my_bucket/a/b/2015-07-30
/.hive-staging_hive_2015-08-04_20-38-31_599_7368878475721182549-1/-ext-10000
Stage: Stage-0
Move Operator
files:
hdfs directory: true
destination:
s3n://<access_key>:<secret_key>@my_bucket/a/b/2015-07-30
Stage: Stage-5
Move Operator
files:
hdfs directory: true
destination:
s3n://<access_key>:<secret_key>@my_bucket/a/b/2015-07-30
/.hive-staging_hive_2015-08-04_20-38-31_599_7368878475721182549-1/-ext-10000
On Tue, Aug 4, 2015 at 4:08 PM, Gopal Vijayaraghavan <gop...@apache.org>
wrote:
>
> > Moving data to:
> >s3n://<access_key>:<secret_key>@my_bucket/a/b/2015-07-30/.hive-staging_hiv
> >e_2015-08-04_18-38-47_649_1476668515119011800-1/-ext-10000
> > Failed with exception Wrong FS:
> >s3n://<access_key>:<secret_key>@my_bucket/a/b/2015-07-30/.hive-staging_hiv
> >e_2015-08-04_18-38-47_649_1476668515119011800-1/-ext-10002, expected:
> >hdfs://s
> ..
> > FAILED: Execution Error, return code 1 from
> >org.apache.hadoop.hive.ql.exec.MoveTask
>
> Looks like the insert overwrite isn¹t triggering the CopyTask - MoveTask
> is the wrong operation for cross-FS operations.
>
> You can try looking at the Explain plan and confirm that.
>
> Cheers,
> Gopal
>
>
>
