Re: hdfs output file operator

[Chinmay Kolhatkar]

Hi Raja,

I can see such a log message in AbstractFileOutputOperator at line 455.

As this code is called from setup of the operator, the operator is getting
deployed and then failing while restoring existing file because of mismatch
in length of the file and the offset the operator has stored previously.

>From the code it looks like it takes care of such cases and restores the
file.

>From what I understand either the file got changes by some other way or the
offset management has a problem.

Are you restarting the application from previous application Id?

To narrow down the problem, can you please try to change the destination
path and see if that works?

Thanks,
Chinmay.



On Sat, Jul 23, 2016 at 5:00 AM, Sandesh Hegde 
wrote:

> Please check,
>  1. AppMaster logs
>  2. Cluster resources
>
> On Fri, Jul 22, 2016 at 1:14 PM Raja.Aravapalli <
> raja.aravapa...@target.com> wrote:
>
>>
>> Hi,
>>
>> I have File output operator which writes to hdfs files!!
>>
>> Application is trying to deploy the operator which writes to hdfs files
>> in many different containers for a long time… but is not succeeding!!!
>> Status is showing as PENDING_DEPLOY
>>
>> In the logs of the container which the Application is trying to deploy
>> hdfs write operator, I can only see, path corrupted!!
>>
>>
>> Can someone please guide or suggest me on this ?
>>
>>
>>
>> Regards,
>> Raja.
>>
>

hdfs output file operator

[Raja . Aravapalli]

Hi,

I have File output operator which writes to hdfs files!!

Application is trying to deploy the operator which writes to hdfs files in many 
different containers for a long time… but is not succeeding!!! Status is 
showing as PENDING_DEPLOY

In the logs of the container which the Application is trying to deploy hdfs 
write operator, I can only see, path corrupted!!


Can someone please guide or suggest me on this ?



Regards,
Raja.

Re: delegation token not found in cache

[Pradeep A. Dalvi]

Could you run following commands?

klist

klist -kt 

--prad

On Thu, Jul 21, 2016 at 10:14 PM, Raja.Aravapalli <
raja.aravapa...@target.com> wrote:

>
> Hi,
>
> Can someone please help me why my DAG is failing with below exception
> after 8 days running succesfully!!
>
>
> Caused by: 
> org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.security.token.SecretManager$InvalidToken):
>  token (HDFS_DELEGATION_TOKEN token 00 for ) can't be found in cache
>   at org.apache.hadoop.ipc.Client.call(Client.java:1427)
>   at org.apache.hadoop.ipc.Client.call(Client.java:1358)
>
>
>
>
> Thanks a lot.
>
> Regards,
> Raja.
>
>