Hi Mich, Right now I have a similar usecase where I have to delete some rows from a hive table. My hive table is of type ORC, Bucketed and included transactional property. I can delete from hive shell but not from my spark-shell or spark app. Were you able to find any work around? Thank you.
Regards, Ajay On Thursday, June 2, 2016, Mich Talebzadeh <mich.talebza...@gmail.com> wrote: > thanks for that. > > I will have a look > > Dr Mich Talebzadeh > > > > LinkedIn * > https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw > <https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw>* > > > > http://talebzadehmich.wordpress.com > > > > On 2 June 2016 at 10:46, Elliot West <tea...@gmail.com > <javascript:_e(%7B%7D,'cvml','tea...@gmail.com');>> wrote: > >> Related to this, there exists an API in Hive to simplify the integrations >> of other frameworks with Hive's ACID feature: >> >> See: >> https://cwiki.apache.org/confluence/display/Hive/HCatalog+Streaming+Mutation+API >> >> It contains code for maintaining heartbeats, handling locks and >> transactions, and submitting mutations in a distributed environment. >> >> We have used it to write to transactional tables from Cascading based >> processes. >> >> Elliot. >> >> >> On 2 June 2016 at 09:54, Mich Talebzadeh <mich.talebza...@gmail.com >> <javascript:_e(%7B%7D,'cvml','mich.talebza...@gmail.com');>> wrote: >> >>> >>> Hi, >>> >>> Spark does not support transactions because as I understand there is a >>> piece in the execution side that needs to send heartbeats to Hive metastore >>> saying a transaction is still alive". That has not been implemented in >>> Spark yet to my knowledge." >>> >>> Any idea on the timelines when we are going to have support for >>> transactions in Spark for Hive ORC tables. This will really be useful. >>> >>> >>> Thanks, >>> >>> >>> Dr Mich Talebzadeh >>> >>> >>> >>> LinkedIn * >>> https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw >>> <https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw>* >>> >>> >>> >>> http://talebzadehmich.wordpress.com >>> >>> >>> >> >> >
