Hi Tariq You need to handle the transaction semantics yourself. You could for example save from the dataframe to a staging table and then write to the final table using a single atomic "INSERT INTO finalTable from stagingTable" call. Remember to clear the staging table first to recover from previous failures if any.
Deenar On 2 June 2015 at 16:01, Mohammad Tariq <donta...@gmail.com> wrote: > Hi list, > > With the help of Spark DataFrame API we can save a DataFrame into a > database table through insertIntoJDBC() call. However, I could not find any > info about how it handles the transactional guarantee. What if my program > gets killed during the processing? Would it end up in partial load? > > Is it somehow possible to handle these kind of scenarios? Rollback or > something of that sort? > > Many thanks. > > P.S : I am using spark-1.3.1-bin-hadoop2.4 with java 1.7 > > [image: http://] > Tariq, Mohammad > about.me/mti > [image: http://] > <http://about.me/mti> > >
