> successful writing of the new one.
> Thanks,
> Assaf.
>
>
> From: Steve Loughran [mailto:ste...@hortonworks.com]
> Sent: Tuesday, February 14, 2017 3:25 PM
> To: Mendelson, Assaf
> Cc: Jörn Franke; user
> Subject: Re: fault tolerant dataframe write with overwrite
Loughran [mailto:ste...@hortonworks.com]
Sent: Tuesday, February 14, 2017 3:25 PM
To: Mendelson, Assaf
Cc: Jörn Franke; user
Subject: Re: fault tolerant dataframe write with overwrite
On 14 Feb 2017, at 11:12, Mendelson, Assaf
<assaf.mendel...@rsa.com<mailto:assaf.mendel...@rsa.com>> wr
:jornfra...@gmail.com]
Sent: Tuesday, February 14, 2017 12:54 PM
To: Mendelson, Assaf
Cc: user
Subject: Re: fault tolerant dataframe write with overwrite
Normally you can fetch the filesystem interface from the configuration ( I
assume you mean URI).
Managing to get the last itera
: fault tolerant dataframe write with overwrite
Normally you can fetch the filesystem interface from the configuration ( I
assume you mean URI).
Managing to get the last iteration: I do not understand the issue. You can have
as the directory the current timestamp and at the end you simply select
Normally you can fetch the filesystem interface from the configuration ( I
assume you mean URI).
Managing to get the last iteration: I do not understand the issue. You can have
as the directory the current timestamp and at the end you simply select the
directory with the highest number.
Hi,
I have a case where I have an iterative process which overwrites the results of
a previous iteration.
Every iteration I need to write a dataframe with the results.
The problem is that when I write, if I simply overwrite the results of the
previous iteration, this is not fault tolerant. i.e.
