Re: Spark on Kudu Roadmap

2017-03-27 Thread Benjamin Kim
Hi Mike, I believe what we are looking for is this below. It is an often request use case. Anyone know if the Spark package will ever allow for creating tables in Spark SQL? Such as: CREATE EXTERNAL TABLE USING org.apache.kudu.spark.kudu OPTIONS (Map("kudu.master" -> “",

Re: How to calculate the optimal value of `maintenance_manager_num_threads`

2017-03-27 Thread Todd Lipcon
Hi Jason, On Fri, Mar 24, 2017 at 1:39 AM, Jason Heo wrote: > Hi, > > I'm using Apache Kudu 1.2 on CDH 5.10. > > Recently, after reading "Bulk write performance improvements for Kudu 1.4 >

Re: Kudu on top of Alluxio

2017-03-27 Thread Mike Percy
+1 thanks for adding that Todd. Mike On Mon, Mar 27, 2017 at 9:55 AM, Todd Lipcon wrote: > On Sat, Mar 25, 2017 at 2:54 PM, Mike Percy wrote: > >> Kudu currently relies on local storage on a POSIX file system. Right now >> there is no support for S3,

Re: Spark on Kudu Roadmap

2017-03-27 Thread Mike Percy
Hi Ben, Is there anything in particular you are looking for? Thanks, Mike On Mon, Mar 27, 2017 at 9:48 AM, Benjamin Kim wrote: > Hi, > > Are there any plans for deeper integration with Spark especially Spark > SQL? Is there a roadmap to look at, so I can know what to expect

Re: Kudu on top of Alluxio

2017-03-27 Thread Todd Lipcon
On Sat, Mar 25, 2017 at 2:54 PM, Mike Percy wrote: > Kudu currently relies on local storage on a POSIX file system. Right now > there is no support for S3, which would be interesting but is non-trivial > in certain ways (particularly if we wanted to rely on S3's replication

Spark on Kudu Roadmap

2017-03-27 Thread Benjamin Kim
Hi, Are there any plans for deeper integration with Spark especially Spark SQL? Is there a roadmap to look at, so I can know what to expect in the future? Cheers, Ben