Re: Use SQL Script to Write Spark SQL Jobs

Hi Nihed, Interesting to see envelope. The idea is same there! Thanks for the sharing :) Best, Bo On Wed, Jun 14, 2017 at 12:22 AM, nihed mbarek wrote: > Hi > > I already saw a project with the same idea. > https://github.com/cloudera-labs/envelope > > Regards, > > On Wed,

Re: Use SQL Script to Write Spark SQL Jobs

Hi I already saw a project with the same idea. https://github.com/cloudera-labs/envelope Regards, On Wed, 14 Jun 2017 at 04:32, bo yang wrote: > Thanks Benjamin and Ayan for the feedback! You kind of represent two group > of people who need such script tool or not.

Re: Use SQL Script to Write Spark SQL Jobs

Thanks Benjamin and Ayan for the feedback! You kind of represent two group of people who need such script tool or not. Personally I find the script is very useful for myself to write ETL pipelines and daily jobs. Let's see whether there are other people interested in such project. Best, Bo

Re: Use SQL Script to Write Spark SQL Jobs

Hi IMHO, this approach is not very useful. Firstly, 2 use cases mentioned in the project page: 1. Simplify spark development - I think the only thing can be done there is to come up with some boilerplate function, which essentially will take a sql and come back with a temp table name and a

Re: Use SQL Script to Write Spark SQL Jobs

Hi Bo, +1 for your project. I come from the world of data warehouses, ETL, and reporting analytics. There are many individuals who do not know or want to do any coding. They are content with ANSI SQL and stick to it. ETL workflows are also done without any coding using a drag-and-drop user

Re: Use SQL Script to Write Spark SQL Jobs

Hi Aakash, Thanks for your willing to help :) It will be great if I could get more feedback on my project. For example, is there any other people feeling the need of using a script to write Spark job easily? Also, I would explore whether it is possible that the Spark project takes some work to

Re: Use SQL Script to Write Spark SQL Jobs

Hey, I work on Spark SQL and would pretty much be able to help you in this. Let me know your requirement. Thanks, Aakash. On 12-Jun-2017 11:00 AM, "bo yang" wrote: > Hi Guys, > > I am writing a small open source project > to use