Re: BucketingSink - Could not invoke truncate while recovering from state
Thats a nice observation! :) I haven't caught that. We need to check that for sure. Gyula On Tue, Aug 27, 2019 at 5:00 PM Kostas Kloudas wrote: > Hi Guyla, > > Thanks for looking into it. > I did not dig into it but in the trace you posted there is the line: > > Failed to TRUNCATE_FILE ... for > **DFSClient_NONMAPREDUCE_-1189574442_56** on 172.31.114.177 because > **DFSClient_NONMAPREDUCE_-1189574442_56 is already the current lease > holder**. > > The client seems to be the same so can this be that we are trying to > get the lease twice from the same task and the lease is not > "reentrant"? > > Cheers, > Kostas > > On Tue, Aug 27, 2019 at 4:53 PM Gyula Fóra wrote: > > > > Hi all! > > > > I am gonna try to resurrect this thread as I think I have hit the same > issue with the StreamingFileSink: > https://issues.apache.org/jira/browse/FLINK-13874 > > > > I don't have a good answer but it seems that we try to truncate before > we get the lease (even though there is logic both in BucketingSink and > HadoopRecoverable... to wait before calling truncate). > > > > Does anyone have any idea? > > > > Cheers, > > Gyula > > > > On Sun, Feb 24, 2019 at 4:13 AM sohimankotia > wrote: > >> > >> Any help ? > >> > >> > >> > >> -- > >> Sent from: > http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/ >
Re: BucketingSink - Could not invoke truncate while recovering from state
Hi Guyla, Thanks for looking into it. I did not dig into it but in the trace you posted there is the line: Failed to TRUNCATE_FILE ... for **DFSClient_NONMAPREDUCE_-1189574442_56** on 172.31.114.177 because **DFSClient_NONMAPREDUCE_-1189574442_56 is already the current lease holder**. The client seems to be the same so can this be that we are trying to get the lease twice from the same task and the lease is not "reentrant"? Cheers, Kostas On Tue, Aug 27, 2019 at 4:53 PM Gyula Fóra wrote: > > Hi all! > > I am gonna try to resurrect this thread as I think I have hit the same issue > with the StreamingFileSink: https://issues.apache.org/jira/browse/FLINK-13874 > > I don't have a good answer but it seems that we try to truncate before we get > the lease (even though there is logic both in BucketingSink and > HadoopRecoverable... to wait before calling truncate). > > Does anyone have any idea? > > Cheers, > Gyula > > On Sun, Feb 24, 2019 at 4:13 AM sohimankotia wrote: >> >> Any help ? >> >> >> >> -- >> Sent from: >> http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/
Re: BucketingSink - Could not invoke truncate while recovering from state
Hi all! I am gonna try to resurrect this thread as I think I have hit the same issue with the StreamingFileSink: https://issues.apache.org/jira/browse/FLINK-13874 I don't have a good answer but it seems that we try to truncate before we get the lease (even though there is logic both in BucketingSink and HadoopRecoverable... to wait before calling truncate). Does anyone have any idea? Cheers, Gyula On Sun, Feb 24, 2019 at 4:13 AM sohimankotia wrote: > Any help ? > > > > -- > Sent from: > http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/ >
Re: BucketingSink - Could not invoke truncate while recovering from state
Any help ? -- Sent from: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/
