Re: Exporting Spark paragraphs as Spark Applications

Hi Luciano, maybe I am wrong, just my two cents for your consideration. Jeff Zhang 于2017年1月6日周五 上午8:32写道： > > Thanks Luciano. I am not saying the community don't feel this is a good > idea. It's just my personal opinion (maybe with some bias, I didn't talk > with many customers as you) I just

Re: Exporting Spark paragraphs as Spark Applications

Thanks Luciano. I am not saying the community don't feel this is a good idea. It's just my personal opinion (maybe with some bias, I didn't talk with many customers as you) I just feel maybe you can spend time on improving zeppelin to make zeppelin to do the job rather than exporting the jar and

Re: Exporting Spark paragraphs as Spark Applications

Hi Jeff, While I agree with you that what you mentioned is completely acceptable for some users, particularly regarding the data science personas. Having said that, while talking with multiple enterprise companies, that have their own scheduler infrastructure with different quality of service or j

Re: Exporting Spark paragraphs as Spark Applications

Thanks Luciano. IIRC, what user want is to run the whole spark app, but they don't care about whether it is in zeppelin or through a standard spark app jar. I know zeppelin currently doesn't do well in converting note to production spark app as Lee mentioned. But exporting note as jar seems a short

Re: Exporting Spark paragraphs as Spark Applications

Some use cases discussed earlier on this thread: https://www.mail-archive.com/dev@zeppelin.apache.org/msg06323.html https://www.mail-archive.com/dev@zeppelin.apache.org/msg06332.html On Wed, Jan 4, 2017 at 4:51 PM, Jianfeng (Jeff) Zhang < jzh...@hortonworks.com> wrote: > > I don¹t understand wh

Re: Exporting Spark paragraphs as Spark Applications

I don¹t understand why user want to export zeppelin note as spark application. If they want to trigger the running of spark app, why not use zeppelin¹s rest api for that. Even user export it as spark application, most of time in reality, they need to submit it through spark job server, so why no

Re: Exporting Spark paragraphs as Spark Applications

I have made some progress with a tool to handle the points discussed in this thread. It's currently a command line tool and given a Zeppelin notebook (note.json) it generates a Spark scala application, compiles it using the compiler embedded in the scala sdk and then package all these resources int

Re: Exporting Spark paragraphs as Spark Applications

To Moon's point, This is what my vision is around this feature - 1. Use should be able to package 1, more than one, all of the paragraphs in a Notebook to create a Jar file which can be used with Spark-Submit. 2. The tool should automatically remove the all the interactive statements like print,

Re: Exporting Spark paragraphs as Spark Applications

I am also pretty much for this. I have got the similar request from each and every people/group who I showcased Zeppelin.Regards, Sourav On Fri, Sep 16, 2016 at 8:06 PM, moon soo Lee wrote: > Hi Luciano, > > I've also got a lot of questions about "Productize the notebook" every time > i meet us

Re: Exporting Spark paragraphs as Spark Applications

Hi Luciano, I've also got a lot of questions about "Productize the notebook" every time i meet users use Zeppelin in their work. I think it's actually about two different problems that Zeppelin need to address. *1) Provide way that interactive notebook becomes part of production data pipeline.*

Re: Exporting Spark paragraphs as Spark Applications

Luciano, I think this would be a terrific feature. I've heard the exact same workflow you've describe in all of the research we've done. ... Jeremy Anderson Founder, Object Adjective 415.493.8489 jer...@objectadjective.com objectadjective.com

Exporting Spark paragraphs as Spark Applications

While talking with a few different users, I have been seeing the use case of using iterative development in Notebooks or Spark Shell and then copying and pasting the final solution to a formal application repeating itself very often. I was wondering if an "Export Spark Paragraphs as a Spark Applic