Re: Question about Data Sources API

2015-03-24 Thread Ashish Mukherjee
Hello Michael, Thanks for your quick reply. My question wrt Java/Scala was related to extending the classes to support new custom data sources, so was wondering if those could be written in Java, since our company is a Java shop. The additional push downs I am looking for are aggregations with

Re: Question about Data Sources API

2015-03-24 Thread Michael Armbrust
My question wrt Java/Scala was related to extending the classes to support new custom data sources, so was wondering if those could be written in Java, since our company is a Java shop. Yes, you should be able to extend the required interfaces using Java. The additional push downs I am

Re: Question about Data Sources API

2015-03-24 Thread Michael Armbrust
On Tue, Mar 24, 2015 at 12:57 AM, Ashish Mukherjee ashish.mukher...@gmail.com wrote: 1. Is the Data Source API stable as of Spark 1.3.0? It is marked DeveloperApi, but in general we do not plan to change even these APIs unless there is a very compelling reason to. 2. The Data Source API

Question about Data Sources API

2015-03-24 Thread Ashish Mukherjee
Hello, I have some questions related to the Data Sources API - 1. Is the Data Source API stable as of Spark 1.3.0? 2. The Data Source API seems to be available only in Scala. Is there any plan to make it available for Java too? 3. Are only filters and projections pushed down to the data