Re: New Spark Datasource for Hive ACID tables
I realised that the build instructions in the README.md were not very clear due to some recent changes. I have updated those now. Thanks, Abhishek Somani On Sun, Jul 28, 2019 at 7:53 AM naresh Goud wrote: > Thanks Abhishek. > I will check it out. > > Thank you, > Naresh > > On Sat, Jul 27, 2019 at 9:21 PM Abhishek Somani < > abhisheksoman...@gmail.com> wrote: > >> Hey Naresh, >> >> There is a `shaded-dependecies` project inside the root directory. You >> need to go into that and build and publish that to local first. >> >> cd shaded-dependencies >>> sbt clean publishLocal >>> >> >> After that, come back out to the root directory and build that project. >> The spark-acid-shaded-dependencies jar will now be found: >> >>> cd .. >>> sbt assembly >> >> >> This will create the jar which you can use. >> >> On another note, unless you are making changes in the code, you don't >> need to build yourself as the jar is published in >> https://spark-packages.org/package/qubole/spark-acid. So you can just >> use it as: >> >> spark-shell --packages qubole:spark-acid:0.4.0-s_2.11 >> >> >> ...and it will be automatically fetched and used. >> >> Thanks, >> Abhishek >> >> >> On Sun, Jul 28, 2019 at 4:42 AM naresh Goud >> wrote: >> >>> It looks there is some internal dependency missing. >>> >>> libraryDependencies ++= Seq( >>> "com.qubole" %% "spark-acid-shaded-dependencies" % "0.1" >>> ) >>> >>> How do we get it? >>> >>> >>> Thank you, >>> Naresh >>> >>> >>> >>> >>> Thanks, >>> Naresh >>> www.linkedin.com/in/naresh-dulam >>> http://hadoopandspark.blogspot.com/ >>> >>> >>> >>> On Sat, Jul 27, 2019 at 5:34 PM naresh Goud >>> wrote: >>> Hi Abhishek, We are not able to build jar using git hub code with below error? Any others able to build jars? Is there anything else missing? Note: Unresolved dependencies path: [warn] com.qubole:spark-acid-shaded-dependencies_2.11:0.1 (C:\Data\Hadoop\spark-acid-master\build.sbt#L51-54) [warn]+- com.qubole:spark-acid_2.11:0.4.0 sbt.ResolveException: unresolved dependency: com.qubole#spark-acid-shaded-dependencies_2.11;0.1: not found at sbt.IvyActions$.sbt$IvyActions$$resolve(IvyActions.scala:313) at sbt.IvyActions$$anonfun$updateEither$1.apply(IvyActions.scala:191) at sbt.IvyActions$$anonfun$updateEither$1.apply(IvyActions.scala:168) at sbt.IvySbt$Module$$anonfun$withModule$1.apply(Ivy.scala:156) at sbt.IvySbt$Module$$anonfun$withModule$1.apply(Ivy.scala:156) at sbt.IvySbt$$anonfun$withIvy$1.apply(Ivy.scala:133) at sbt.IvySbt.sbt$IvySbt$$action$1(Ivy.scala:57) at sbt.IvySbt$$anon$4.call(Ivy.scala:65) at xsbt.boot.Locks$GlobalLock.withChannel$1(Locks.scala:93) at xsbt.boot.Locks$GlobalLock.xsbt$boot$Locks$GlobalLock$$withChannelRetries$1(Locks.scala:78) at xsbt.boot.Locks$GlobalLock$$anonfun$withFileLock$1.apply(Locks.scala:97) at xsbt.boot.Using$.withResource(Using.scala:10) at xsbt.boot.Using$.apply(Using.scala:9) at xsbt.boot.Locks$GlobalLock.ignoringDeadlockAvoided(Locks.scala:58) at xsbt.boot.Locks$GlobalLock.withLock(Locks.scala:48) at xsbt.boot.Locks$.apply0(Locks.scala:31) at xsbt.boot.Locks$.apply(Locks.scala:28) at sbt.IvySbt.withDefaultLogger(Ivy.scala:65) at sbt.IvySbt.withIvy(Ivy.scala:128) at sbt.IvySbt.withIvy(Ivy.scala:125) at sbt.IvySbt$Module.withModule(Ivy.scala:156) at sbt.IvyActions$.updateEither(IvyActions.scala:168) at sbt.Classpaths$$anonfun$sbt$Classpaths$$work$1$1.apply(Defaults.scala:1541) Thanks, Naresh www.linkedin.com/in/naresh-dulam http://hadoopandspark.blogspot.com/ On Sat, Jul 27, 2019 at 3:25 PM Nicolas Paris wrote: > Congrats > > The read/write feature with hive3 is highly interesting > > On Fri, Jul 26, 2019 at 06:07:55PM +0530, Abhishek Somani wrote: > > Hi All, > > > > We at Qubole have open sourced a datasource that will enable users > to work on > > their Hive ACID Transactional Tables using Spark. > > > > Github: https://github.com/qubole/spark-acid > > > > Hive ACID tables allow users to work on their data transactionally, > and also > > gives them the ability to Delete, Update and Merge data efficiently > without > > having to rewrite all of their data in a table, partition or file. > We believe > > that being able to work on these tables from Spark is a much desired > value add, > > as is also apparent in > https://issues.apache.org/jira/browse/SPARK-15348 and > > https://issues.apache.org/jira/browse/SPARK-16996
Re: New Spark Datasource for Hive ACID tables
Thanks Abhishek. I will check it out. Thank you, Naresh On Sat, Jul 27, 2019 at 9:21 PM Abhishek Somani wrote: > Hey Naresh, > > There is a `shaded-dependecies` project inside the root directory. You > need to go into that and build and publish that to local first. > > cd shaded-dependencies >> sbt clean publishLocal >> > > After that, come back out to the root directory and build that project. > The spark-acid-shaded-dependencies jar will now be found: > >> cd .. >> sbt assembly > > > This will create the jar which you can use. > > On another note, unless you are making changes in the code, you don't need > to build yourself as the jar is published in > https://spark-packages.org/package/qubole/spark-acid. So you can just use > it as: > > spark-shell --packages qubole:spark-acid:0.4.0-s_2.11 > > > ...and it will be automatically fetched and used. > > Thanks, > Abhishek > > > On Sun, Jul 28, 2019 at 4:42 AM naresh Goud > wrote: > >> It looks there is some internal dependency missing. >> >> libraryDependencies ++= Seq( >> "com.qubole" %% "spark-acid-shaded-dependencies" % "0.1" >> ) >> >> How do we get it? >> >> >> Thank you, >> Naresh >> >> >> >> >> Thanks, >> Naresh >> www.linkedin.com/in/naresh-dulam >> http://hadoopandspark.blogspot.com/ >> >> >> >> On Sat, Jul 27, 2019 at 5:34 PM naresh Goud >> wrote: >> >>> Hi Abhishek, >>> >>> >>> We are not able to build jar using git hub code with below error? >>> >>> Any others able to build jars? Is there anything else missing? >>> >>> >>> >>> Note: Unresolved dependencies path: >>> [warn] com.qubole:spark-acid-shaded-dependencies_2.11:0.1 >>> (C:\Data\Hadoop\spark-acid-master\build.sbt#L51-54) >>> [warn]+- com.qubole:spark-acid_2.11:0.4.0 >>> sbt.ResolveException: unresolved dependency: >>> com.qubole#spark-acid-shaded-dependencies_2.11;0.1: not found >>> at sbt.IvyActions$.sbt$IvyActions$$resolve(IvyActions.scala:313) >>> at >>> sbt.IvyActions$$anonfun$updateEither$1.apply(IvyActions.scala:191) >>> at >>> sbt.IvyActions$$anonfun$updateEither$1.apply(IvyActions.scala:168) >>> at sbt.IvySbt$Module$$anonfun$withModule$1.apply(Ivy.scala:156) >>> at sbt.IvySbt$Module$$anonfun$withModule$1.apply(Ivy.scala:156) >>> at sbt.IvySbt$$anonfun$withIvy$1.apply(Ivy.scala:133) >>> at sbt.IvySbt.sbt$IvySbt$$action$1(Ivy.scala:57) >>> at sbt.IvySbt$$anon$4.call(Ivy.scala:65) >>> at xsbt.boot.Locks$GlobalLock.withChannel$1(Locks.scala:93) >>> at >>> xsbt.boot.Locks$GlobalLock.xsbt$boot$Locks$GlobalLock$$withChannelRetries$1(Locks.scala:78) >>> at >>> xsbt.boot.Locks$GlobalLock$$anonfun$withFileLock$1.apply(Locks.scala:97) >>> at xsbt.boot.Using$.withResource(Using.scala:10) >>> at xsbt.boot.Using$.apply(Using.scala:9) >>> at >>> xsbt.boot.Locks$GlobalLock.ignoringDeadlockAvoided(Locks.scala:58) >>> at xsbt.boot.Locks$GlobalLock.withLock(Locks.scala:48) >>> at xsbt.boot.Locks$.apply0(Locks.scala:31) >>> at xsbt.boot.Locks$.apply(Locks.scala:28) >>> at sbt.IvySbt.withDefaultLogger(Ivy.scala:65) >>> at sbt.IvySbt.withIvy(Ivy.scala:128) >>> at sbt.IvySbt.withIvy(Ivy.scala:125) >>> at sbt.IvySbt$Module.withModule(Ivy.scala:156) >>> at sbt.IvyActions$.updateEither(IvyActions.scala:168) >>> at >>> sbt.Classpaths$$anonfun$sbt$Classpaths$$work$1$1.apply(Defaults.scala:1541) >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> Thanks, >>> Naresh >>> www.linkedin.com/in/naresh-dulam >>> http://hadoopandspark.blogspot.com/ >>> >>> >>> >>> On Sat, Jul 27, 2019 at 3:25 PM Nicolas Paris >>> wrote: >>> Congrats The read/write feature with hive3 is highly interesting On Fri, Jul 26, 2019 at 06:07:55PM +0530, Abhishek Somani wrote: > Hi All, > > We at Qubole have open sourced a datasource that will enable users to work on > their Hive ACID Transactional Tables using Spark. > > Github: https://github.com/qubole/spark-acid > > Hive ACID tables allow users to work on their data transactionally, and also > gives them the ability to Delete, Update and Merge data efficiently without > having to rewrite all of their data in a table, partition or file. We believe > that being able to work on these tables from Spark is a much desired value add, > as is also apparent in https://issues.apache.org/jira/browse/SPARK-15348 and > https://issues.apache.org/jira/browse/SPARK-16996 with multiple people looking > for it. Currently the datasource supports reading from these ACID tables only, > and we are working on adding the ability to write into these tables via Spark > as well. > > The datasource is also available as a spark package, and instructions on how to > use it are available on the Github page. > > We welcome your
Re: New Spark Datasource for Hive ACID tables
Hey Naresh, There is a `shaded-dependecies` project inside the root directory. You need to go into that and build and publish that to local first. cd shaded-dependencies > sbt clean publishLocal > After that, come back out to the root directory and build that project. The spark-acid-shaded-dependencies jar will now be found: > cd .. > sbt assembly This will create the jar which you can use. On another note, unless you are making changes in the code, you don't need to build yourself as the jar is published in https://spark-packages.org/package/qubole/spark-acid. So you can just use it as: spark-shell --packages qubole:spark-acid:0.4.0-s_2.11 ...and it will be automatically fetched and used. Thanks, Abhishek On Sun, Jul 28, 2019 at 4:42 AM naresh Goud wrote: > It looks there is some internal dependency missing. > > libraryDependencies ++= Seq( > "com.qubole" %% "spark-acid-shaded-dependencies" % "0.1" > ) > > How do we get it? > > > Thank you, > Naresh > > > > > Thanks, > Naresh > www.linkedin.com/in/naresh-dulam > http://hadoopandspark.blogspot.com/ > > > > On Sat, Jul 27, 2019 at 5:34 PM naresh Goud > wrote: > >> Hi Abhishek, >> >> >> We are not able to build jar using git hub code with below error? >> >> Any others able to build jars? Is there anything else missing? >> >> >> >> Note: Unresolved dependencies path: >> [warn] com.qubole:spark-acid-shaded-dependencies_2.11:0.1 >> (C:\Data\Hadoop\spark-acid-master\build.sbt#L51-54) >> [warn]+- com.qubole:spark-acid_2.11:0.4.0 >> sbt.ResolveException: unresolved dependency: >> com.qubole#spark-acid-shaded-dependencies_2.11;0.1: not found >> at sbt.IvyActions$.sbt$IvyActions$$resolve(IvyActions.scala:313) >> at >> sbt.IvyActions$$anonfun$updateEither$1.apply(IvyActions.scala:191) >> at >> sbt.IvyActions$$anonfun$updateEither$1.apply(IvyActions.scala:168) >> at sbt.IvySbt$Module$$anonfun$withModule$1.apply(Ivy.scala:156) >> at sbt.IvySbt$Module$$anonfun$withModule$1.apply(Ivy.scala:156) >> at sbt.IvySbt$$anonfun$withIvy$1.apply(Ivy.scala:133) >> at sbt.IvySbt.sbt$IvySbt$$action$1(Ivy.scala:57) >> at sbt.IvySbt$$anon$4.call(Ivy.scala:65) >> at xsbt.boot.Locks$GlobalLock.withChannel$1(Locks.scala:93) >> at >> xsbt.boot.Locks$GlobalLock.xsbt$boot$Locks$GlobalLock$$withChannelRetries$1(Locks.scala:78) >> at >> xsbt.boot.Locks$GlobalLock$$anonfun$withFileLock$1.apply(Locks.scala:97) >> at xsbt.boot.Using$.withResource(Using.scala:10) >> at xsbt.boot.Using$.apply(Using.scala:9) >> at >> xsbt.boot.Locks$GlobalLock.ignoringDeadlockAvoided(Locks.scala:58) >> at xsbt.boot.Locks$GlobalLock.withLock(Locks.scala:48) >> at xsbt.boot.Locks$.apply0(Locks.scala:31) >> at xsbt.boot.Locks$.apply(Locks.scala:28) >> at sbt.IvySbt.withDefaultLogger(Ivy.scala:65) >> at sbt.IvySbt.withIvy(Ivy.scala:128) >> at sbt.IvySbt.withIvy(Ivy.scala:125) >> at sbt.IvySbt$Module.withModule(Ivy.scala:156) >> at sbt.IvyActions$.updateEither(IvyActions.scala:168) >> at >> sbt.Classpaths$$anonfun$sbt$Classpaths$$work$1$1.apply(Defaults.scala:1541) >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> Thanks, >> Naresh >> www.linkedin.com/in/naresh-dulam >> http://hadoopandspark.blogspot.com/ >> >> >> >> On Sat, Jul 27, 2019 at 3:25 PM Nicolas Paris >> wrote: >> >>> Congrats >>> >>> The read/write feature with hive3 is highly interesting >>> >>> On Fri, Jul 26, 2019 at 06:07:55PM +0530, Abhishek Somani wrote: >>> > Hi All, >>> > >>> > We at Qubole have open sourced a datasource that will enable users to >>> work on >>> > their Hive ACID Transactional Tables using Spark. >>> > >>> > Github: https://github.com/qubole/spark-acid >>> > >>> > Hive ACID tables allow users to work on their data transactionally, >>> and also >>> > gives them the ability to Delete, Update and Merge data efficiently >>> without >>> > having to rewrite all of their data in a table, partition or file. We >>> believe >>> > that being able to work on these tables from Spark is a much desired >>> value add, >>> > as is also apparent in >>> https://issues.apache.org/jira/browse/SPARK-15348 and >>> > https://issues.apache.org/jira/browse/SPARK-16996 with multiple >>> people looking >>> > for it. Currently the datasource supports reading from these ACID >>> tables only, >>> > and we are working on adding the ability to write into these tables >>> via Spark >>> > as well. >>> > >>> > The datasource is also available as a spark package, and instructions >>> on how to >>> > use it are available on the Github page. >>> > >>> > We welcome your feedback and suggestions. >>> > >>> > Thanks, >>> > Abhishek Somani >>> >>> -- >>> nicolas >>> >>> - >>> To unsubscribe e-mail: user-unsubscr...@spark.apache.org >>> >>>
Re: New Spark Datasource for Hive ACID tables
It looks there is some internal dependency missing. libraryDependencies ++= Seq( "com.qubole" %% "spark-acid-shaded-dependencies" % "0.1" ) How do we get it? Thank you, Naresh Thanks, Naresh www.linkedin.com/in/naresh-dulam http://hadoopandspark.blogspot.com/ On Sat, Jul 27, 2019 at 5:34 PM naresh Goud wrote: > Hi Abhishek, > > > We are not able to build jar using git hub code with below error? > > Any others able to build jars? Is there anything else missing? > > > > Note: Unresolved dependencies path: > [warn] com.qubole:spark-acid-shaded-dependencies_2.11:0.1 > (C:\Data\Hadoop\spark-acid-master\build.sbt#L51-54) > [warn]+- com.qubole:spark-acid_2.11:0.4.0 > sbt.ResolveException: unresolved dependency: > com.qubole#spark-acid-shaded-dependencies_2.11;0.1: not found > at sbt.IvyActions$.sbt$IvyActions$$resolve(IvyActions.scala:313) > at > sbt.IvyActions$$anonfun$updateEither$1.apply(IvyActions.scala:191) > at > sbt.IvyActions$$anonfun$updateEither$1.apply(IvyActions.scala:168) > at sbt.IvySbt$Module$$anonfun$withModule$1.apply(Ivy.scala:156) > at sbt.IvySbt$Module$$anonfun$withModule$1.apply(Ivy.scala:156) > at sbt.IvySbt$$anonfun$withIvy$1.apply(Ivy.scala:133) > at sbt.IvySbt.sbt$IvySbt$$action$1(Ivy.scala:57) > at sbt.IvySbt$$anon$4.call(Ivy.scala:65) > at xsbt.boot.Locks$GlobalLock.withChannel$1(Locks.scala:93) > at > xsbt.boot.Locks$GlobalLock.xsbt$boot$Locks$GlobalLock$$withChannelRetries$1(Locks.scala:78) > at > xsbt.boot.Locks$GlobalLock$$anonfun$withFileLock$1.apply(Locks.scala:97) > at xsbt.boot.Using$.withResource(Using.scala:10) > at xsbt.boot.Using$.apply(Using.scala:9) > at > xsbt.boot.Locks$GlobalLock.ignoringDeadlockAvoided(Locks.scala:58) > at xsbt.boot.Locks$GlobalLock.withLock(Locks.scala:48) > at xsbt.boot.Locks$.apply0(Locks.scala:31) > at xsbt.boot.Locks$.apply(Locks.scala:28) > at sbt.IvySbt.withDefaultLogger(Ivy.scala:65) > at sbt.IvySbt.withIvy(Ivy.scala:128) > at sbt.IvySbt.withIvy(Ivy.scala:125) > at sbt.IvySbt$Module.withModule(Ivy.scala:156) > at sbt.IvyActions$.updateEither(IvyActions.scala:168) > at > sbt.Classpaths$$anonfun$sbt$Classpaths$$work$1$1.apply(Defaults.scala:1541) > > > > > > > > > > > > > > > Thanks, > Naresh > www.linkedin.com/in/naresh-dulam > http://hadoopandspark.blogspot.com/ > > > > On Sat, Jul 27, 2019 at 3:25 PM Nicolas Paris > wrote: > >> Congrats >> >> The read/write feature with hive3 is highly interesting >> >> On Fri, Jul 26, 2019 at 06:07:55PM +0530, Abhishek Somani wrote: >> > Hi All, >> > >> > We at Qubole have open sourced a datasource that will enable users to >> work on >> > their Hive ACID Transactional Tables using Spark. >> > >> > Github: https://github.com/qubole/spark-acid >> > >> > Hive ACID tables allow users to work on their data transactionally, and >> also >> > gives them the ability to Delete, Update and Merge data efficiently >> without >> > having to rewrite all of their data in a table, partition or file. We >> believe >> > that being able to work on these tables from Spark is a much desired >> value add, >> > as is also apparent in >> https://issues.apache.org/jira/browse/SPARK-15348 and >> > https://issues.apache.org/jira/browse/SPARK-16996 with multiple people >> looking >> > for it. Currently the datasource supports reading from these ACID >> tables only, >> > and we are working on adding the ability to write into these tables via >> Spark >> > as well. >> > >> > The datasource is also available as a spark package, and instructions >> on how to >> > use it are available on the Github page. >> > >> > We welcome your feedback and suggestions. >> > >> > Thanks, >> > Abhishek Somani >> >> -- >> nicolas >> >> - >> To unsubscribe e-mail: user-unsubscr...@spark.apache.org >> >>
Re: New Spark Datasource for Hive ACID tables
Hi Abhishek, We are not able to build jar using git hub code with below error? Any others able to build jars? Is there anything else missing? Note: Unresolved dependencies path: [warn] com.qubole:spark-acid-shaded-dependencies_2.11:0.1 (C:\Data\Hadoop\spark-acid-master\build.sbt#L51-54) [warn]+- com.qubole:spark-acid_2.11:0.4.0 sbt.ResolveException: unresolved dependency: com.qubole#spark-acid-shaded-dependencies_2.11;0.1: not found at sbt.IvyActions$.sbt$IvyActions$$resolve(IvyActions.scala:313) at sbt.IvyActions$$anonfun$updateEither$1.apply(IvyActions.scala:191) at sbt.IvyActions$$anonfun$updateEither$1.apply(IvyActions.scala:168) at sbt.IvySbt$Module$$anonfun$withModule$1.apply(Ivy.scala:156) at sbt.IvySbt$Module$$anonfun$withModule$1.apply(Ivy.scala:156) at sbt.IvySbt$$anonfun$withIvy$1.apply(Ivy.scala:133) at sbt.IvySbt.sbt$IvySbt$$action$1(Ivy.scala:57) at sbt.IvySbt$$anon$4.call(Ivy.scala:65) at xsbt.boot.Locks$GlobalLock.withChannel$1(Locks.scala:93) at xsbt.boot.Locks$GlobalLock.xsbt$boot$Locks$GlobalLock$$withChannelRetries$1(Locks.scala:78) at xsbt.boot.Locks$GlobalLock$$anonfun$withFileLock$1.apply(Locks.scala:97) at xsbt.boot.Using$.withResource(Using.scala:10) at xsbt.boot.Using$.apply(Using.scala:9) at xsbt.boot.Locks$GlobalLock.ignoringDeadlockAvoided(Locks.scala:58) at xsbt.boot.Locks$GlobalLock.withLock(Locks.scala:48) at xsbt.boot.Locks$.apply0(Locks.scala:31) at xsbt.boot.Locks$.apply(Locks.scala:28) at sbt.IvySbt.withDefaultLogger(Ivy.scala:65) at sbt.IvySbt.withIvy(Ivy.scala:128) at sbt.IvySbt.withIvy(Ivy.scala:125) at sbt.IvySbt$Module.withModule(Ivy.scala:156) at sbt.IvyActions$.updateEither(IvyActions.scala:168) at sbt.Classpaths$$anonfun$sbt$Classpaths$$work$1$1.apply(Defaults.scala:1541) Thanks, Naresh www.linkedin.com/in/naresh-dulam http://hadoopandspark.blogspot.com/ On Sat, Jul 27, 2019 at 3:25 PM Nicolas Paris wrote: > Congrats > > The read/write feature with hive3 is highly interesting > > On Fri, Jul 26, 2019 at 06:07:55PM +0530, Abhishek Somani wrote: > > Hi All, > > > > We at Qubole have open sourced a datasource that will enable users to > work on > > their Hive ACID Transactional Tables using Spark. > > > > Github: https://github.com/qubole/spark-acid > > > > Hive ACID tables allow users to work on their data transactionally, and > also > > gives them the ability to Delete, Update and Merge data efficiently > without > > having to rewrite all of their data in a table, partition or file. We > believe > > that being able to work on these tables from Spark is a much desired > value add, > > as is also apparent in https://issues.apache.org/jira/browse/SPARK-15348 > and > > https://issues.apache.org/jira/browse/SPARK-16996 with multiple people > looking > > for it. Currently the datasource supports reading from these ACID tables > only, > > and we are working on adding the ability to write into these tables via > Spark > > as well. > > > > The datasource is also available as a spark package, and instructions on > how to > > use it are available on the Github page. > > > > We welcome your feedback and suggestions. > > > > Thanks, > > Abhishek Somani > > -- > nicolas > > - > To unsubscribe e-mail: user-unsubscr...@spark.apache.org > >
Re: New Spark Datasource for Hive ACID tables
Congrats The read/write feature with hive3 is highly interesting On Fri, Jul 26, 2019 at 06:07:55PM +0530, Abhishek Somani wrote: > Hi All, > > We at Qubole have open sourced a datasource that will enable users to work on > their Hive ACID Transactional Tables using Spark. > > Github: https://github.com/qubole/spark-acid > > Hive ACID tables allow users to work on their data transactionally, and also > gives them the ability to Delete, Update and Merge data efficiently without > having to rewrite all of their data in a table, partition or file. We believe > that being able to work on these tables from Spark is a much desired value > add, > as is also apparent in https://issues.apache.org/jira/browse/SPARK-15348 and > https://issues.apache.org/jira/browse/SPARK-16996 with multiple people looking > for it. Currently the datasource supports reading from these ACID tables only, > and we are working on adding the ability to write into these tables via Spark > as well. > > The datasource is also available as a spark package, and instructions on how > to > use it are available on the Github page. > > We welcome your feedback and suggestions. > > Thanks, > Abhishek Somani -- nicolas - To unsubscribe e-mail: user-unsubscr...@spark.apache.org