Hi, great question - as cloud platform that you mention is not open-source, it is not immediately expected to get it done as a part of GSoC.
That being said, if that is something that you want to look into, or think that should be not very hard to implement, or think that would be of the great benefit of users - it's totally up to you and you can make it part of your project proposal scope, if you want to (i.e as extra bonus, at the end, if time permits) -- Alex On Thu, Mar 10, 2016 at 6:39 PM, Minudika Malshan <minudika...@gmail.com> wrote: > Hi Alex, > > As we know the origin of Apache beam is the Google cloud platform. It has > been developed mainly focusing on deploying pipelines in the cloud. > But AFAIK mostly zeppelin is used as a standalone system. > Is it expected to implement the cloud support for the interpreter under > this GSoC project? > > Best regards. > > Minudika Malshan > Undergraduate > Department of Computer Science and Engineering > University of Moratuwa > Sri Lanka. > > > > > On Thu, Mar 10, 2016 at 6:44 AM, Alexander Bezzubov <b...@apache.org> > wrote: > > > Hi Minudika, > > > > you plan sounds awesome - please either keep this thread posted on the > > progress or even better, feel free to start a new one, with more > > descriptive subject, asking community for a feedback on your proposal. > > > > Google docs \w ability for people with the link to post comments there > is a > > great medium. As soon as you send one I'll be happy to add it to https:// > > cwiki.apache.org/confluence/display/ZEPPELIN/Google+Summer+Of+Code+2016 > > > > -- > > Alex > > > > On Wed, Mar 9, 2016 at 9:21 PM, Minudika Malshan <minudika...@gmail.com> > > wrote: > > > > > Hi Alex, > > > > > > I think Java-repl can be used for the implementation. Currently I am > > > working on a sample implementation and will share it in here soon. > > > And will it be ok if I share my sample proposal as a google doc? > > > > > > Thanks for your help. > > > > > > Regards. > > > Minudika > > > > > > Minudika Malshan > > > Undergraduate > > > Department of Computer Science and Engineering > > > University of Moratuwa > > > Sri Lanka > > > > > > > > > > > > > > > On Mon, Mar 7, 2016 at 6:39 AM, Alexander Bezzubov <b...@apache.org> > > wrote: > > > > > > > Hi Minudika, > > > > > > > > if you are talking about Java REPL [1], this was just an example of > the > > > > project we might want to use to archive the first step of our goal: > get > > > > Java code interpreted by Zeppelin. > > > > > > > > > Should we have to add JAVA-REPL source code as a module for the > beam > > > > > interpreter or can we get the necessary features of it as > > dependencies > > > to > > > > > our project? > > > > Well, it looks like this particular project is built with Ant and is > > not > > > > published on Maven [2]. So right now there is no easy way to get it > as > > > > build-time dependency. > > > > > > > > What I would suggest here: > > > > - try finding any other other alternative java REPL implementations > > > > - if there are none, you could try hacking first PoC for Java\Beam > > > > interpreter together just using the code from this project, inside a > > new > > > > interpreter module directly (it is Apache 2.0 licensed) > > > > > > > > If it suits our needs and PoC works (it might not, looking though > open > > > > issues on Github), then at the end of the GSoC you could just > allocate > > > some > > > > time to contribute Maven publishing to that project (which should not > > > take > > > > long and can be done as a last step). > > > > > > > > Hope this helps and please let me know if you have further questions! > > > > > > > > 1. https://github.com/albertlatacz/java-repl/issues/79 > > > > 2. https://github.com/albertlatacz/java-repl > > > > > > > > On Sun, Mar 6, 2016 at 4:17 AM, Minudika Malshan < > > minudika...@gmail.com> > > > > wrote: > > > > > > > > > Hi, > > > > > > > > > > I have gained understanding about zeppelin and functionalities of > > > > DataFlow > > > > > SDK. please help me to get familiar with JAVA-REPL. > > > > > It would be a great help if you can help me to understand the flow > of > > > > > JAVA-REPL and the way to use it to interpret DataFlow SDK methods. > > > > > > > > > > Regards > > > > > Minudika > > > > > > > > > > Minudika Malshan > > > > > Undergraduate > > > > > Department of Computer Science and Engineering > > > > > University of Moratuwa. > > > > > > > > > > > > > > > > > > > > > > > > > On Sat, Mar 5, 2016 at 1:03 AM, Minudika Malshan < > > > minudika...@gmail.com> > > > > > wrote: > > > > > > > > > > > Hi, > > > > > > > > > > > > I was going through JAVA REPL implementation. > > > > > > I'm bit confused about the way it should be used to implement a > > java > > > > > > interpreter which supports Apache beam commands. > > > > > > Should we have to add JAVA-REPL source code as a module for the > > beam > > > > > > interpreter or can we get the necessary features of it as > > > dependencies > > > > to > > > > > > our project? > > > > > > Could you please explain the procedure I should follow in order > to > > > > > achieve > > > > > > this task. > > > > > > > > > > > > Thanks and regards, > > > > > > Minudika > > > > > > > > > > > > > > > > > > Minudika Malshan > > > > > > Undergraduate > > > > > > Department of Computer Science and Engineering > > > > > > University of Moratuwa. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > On Thu, Mar 3, 2016 at 11:23 AM, Alexander Bezzubov < > > b...@apache.org> > > > > > > wrote: > > > > > > > > > > > >> Hi, > > > > > >> > > > > > >> I would suggest you to take some time and read through the links > > > from > > > > > >> email > > > > > >> carefully and then post back here in this thread, in case you > > manage > > > > to > > > > > >> find the answer to your question. > > > > > >> > > > > > >> That might helpful to other students participating in GSoC as > > well. > > > > > >> > > > > > >> Thanks! > > > > > >> > > > > > >> -- > > > > > >> Alex > > > > > >> > > > > > >> > > > > > >> On Thu, Mar 3, 2016 at 2:43 PM, Minudika Malshan < > > > > minudika...@gmail.com > > > > > > > > > > > >> wrote: > > > > > >> > > > > > >> > Hi Alex, > > > > > >> > > > > > > >> > Thanks a lot for your detailed guide through the project. > > > > > >> > As you have mentioned, I'm going to create a draft proposal. > > > > > >> > Is there any specific template for the proposal? Or will it be > > ok > > > to > > > > > >> create > > > > > >> > a proposal on my own including the facts and points you've > > > mentioned > > > > > in > > > > > >> > your reply? > > > > > >> > > > > > > >> > Thanks again and regards. > > > > > >> > Minudika > > > > > >> > > > > > > >> > Minudika Malshan > > > > > >> > Undergraduate > > > > > >> > Department of Computer Science and Engineering > > > > > >> > University of Moratuwa. > > > > > >> > > > > > > >> > > > > > > >> > > > > > > >> > > > > > > >> > On Thu, Mar 3, 2016 at 8:49 AM, Alexander Bezzubov < > > > > > >> abezzu...@nflabs.com> > > > > > >> > wrote: > > > > > >> > > > > > > >> > > Hi Minudika, > > > > > >> > > > > > > > >> > > > Currently I am writing an interpreter for SiddhiQL which > is > > a > > > > real > > > > > >> time > > > > > >> > > > complex event processing engine, as a part of research. > > > > > >> > > > If you are interested, you can find the code at here. > (It's > > > too > > > > > >> messy > > > > > >> > > > though. Have a lot to clean up the in the code) > > > > > >> > > > https://github.com/minudika/incubator-zeppelin > > > > > >> > > > > > > > >> > > Thank you for sharing! That would be a huge plus for your > > > > proposal, > > > > > >> but > > > > > >> > > please remember that code quality would be an important > factor > > > on > > > > > >> > > the proposal selection process, so please make sure that you > > > have > > > > > >> > finished > > > > > >> > > possible cleanups before attaching it to your application. > > > > > >> > > > > > > > >> > > On the side note - I can see your commits in master branch > of > > > your > > > > > >> fork, > > > > > >> > > why not to try the workflow Zeppelin uses > > > > > >> > > < > > > > > >> > > > > > > > >> > > > > > > >> > > > > > > > > > > > > > > > https://github.com/apache/incubator-zeppelin/blob/master/CONTRIBUTING.md#source-control-workflow > > > > > >> > > > > > > > > >> > > [0] > > > > > >> > > by tracking your work in different branches? From there it > > will > > > be > > > > > >> easy > > > > > >> > to > > > > > >> > > submit a PR and ask other contributors for a code review, > > which > > > > > could > > > > > >> > > improve chances of getting high quality code at the end. > > > > > >> > > > > > > > >> > > > > > > > >> > > > I would like to know the exact purpose of this project and > > > > > expected > > > > > >> > > outcome > > > > > >> > > from the student. > > > > > >> > > > > > > > >> > > The purpose (the objective or the outcome or the goal) of > this > > > > > >> project is > > > > > >> > > to have merged PR with a code + documentation + example of > > usage > > > > > for a > > > > > >> > new > > > > > >> > > Interpreter in Zeppelin, that works with Apache Beam SKD by > > the > > > > end > > > > > of > > > > > >> > the > > > > > >> > > summer. > > > > > >> > > > > > > > >> > > It might be bold goal, so in order to be reachable and get > > > > accepted > > > > > it > > > > > >> > > should be further divided on smaller goal aligned with GSoC > > > > > timeline, > > > > > >> and > > > > > >> > > then even on a smaller "steps" on "how to get there". > > > > > >> > > The expectations are that you will do this work, as part of > > your > > > > > >> > > application proposal (with the help of potential mentors). > In > > > > prev. > > > > > >> email > > > > > >> > > in this thread I posted some ideas on how those smaller > goals > > > can > > > > > look > > > > > >> > > like, but ultimately it's up to you to decide. > > > > > >> > > > > > > > >> > > > > > > > >> > > So I would suggest you start looking into creating a > proposal > > > > draft > > > > > as > > > > > >> > > early as possible. [5] Have some examples and docs from ASF > on > > > the > > > > > >> > proposal > > > > > >> > > as well, so please make sure that you read it. Please also > > feel > > > > free > > > > > >> to > > > > > >> > > post it here to get the feedback from the community, and I > > will > > > be > > > > > >> happy > > > > > >> > to > > > > > >> > > further assist you with it. > > > > > >> > > > > > > > >> > > Good proposal consists of: > > > > > >> > > > > > > > >> > > - list of high-level, measurable "results", or steps to > > archive > > > > the > > > > > >> > final > > > > > >> > > objective\goal > > > > > >> > > - laid out over the timeline (so expectations for > > "milestones" > > > > like > > > > > >> > > mid-term are set) > > > > > >> > > - each "result" with the further scope breakdown to a > smaller > > > > > action > > > > > >> > items > > > > > >> > > (ideally, each not longer than 1day) > > > > > >> > > - plus some background information, showing that you are > > > > qualified > > > > > >> to do > > > > > >> > > all that and put some thinking into details (links to your > > code > > > > here > > > > > >> will > > > > > >> > > help very much) > > > > > >> > > > > > > > >> > > > > > > > >> > > > I am getting familiar with Apache beam nowadays. Please > can > > > you > > > > > >> provide > > > > > >> > > me some useful resources for that. > > > > > >> > > > > > > > >> > > [1], [2], [3], [4] are some resource to learn about Beam > > before > > > it > > > > > has > > > > > >> > > finalized its move to Apache. > > > > > >> > > > > > > > >> > > Hope this helps! > > > > > >> > > > > > > > >> > > -- > > > > > >> > > Alex > > > > > >> > > > > > > > >> > > > > > > > >> > > 0. > > > > > >> > > > > > > > >> > > > > > > > >> > > > > > > >> > > > > > > > > > > > > > > > https://github.com/apache/incubator-zeppelin/blob/master/CONTRIBUTING.md#source-control-workflow > > > > > >> > > 1. https://cloud.google.com/dataflow/ > > > > > >> > > 2. > > > > https://cloud.google.com/dataflow/what-is-google-cloud-dataflow > > > > > >> > > 3. > > > > > >> > > > > > > > >> > > > > > > >> > > > > > > > > > > > > > > > https://cloud.google.com/dataflow/blog/dataflow-beam-and-spark-comparison > > > > > >> > > 4. > > > > https://github.com/GoogleCloudPlatform/DataflowJavaSDK-examples > > > > > >> > > 5. https://community.apache.org/gsoc.html > > > > > >> > > > > > > > >> > > On Wed, Mar 2, 2016 at 2:50 AM, Minudika Malshan < > > > > > >> minudika...@gmail.com> > > > > > >> > > wrote: > > > > > >> > > > > > > > >> > > > Hi, > > > > > >> > > > > > > > > >> > > > First of all, thanks Alex for your reply. > > > > > >> > > > Sorry for the delay. > > > > > >> > > > Some thing has gone wrong with my mail thread. I just saw > > > Alex's > > > > > >> reply. > > > > > >> > > > > > > > > >> > > > I am getting familiar with Apache beam nowadays. Please > can > > > you > > > > > >> provide > > > > > >> > > me > > > > > >> > > > some useful resources for that. > > > > > >> > > > Beam developers said that it would take like a week to get > > the > > > > > >> official > > > > > >> > > > Apache documentation for beam done. > > > > > >> > > > > > > > > >> > > > Cheers, > > > > > >> > > > Minudika > > > > > >> > > > > > > > > >> > > > Minudika Malshan > > > > > >> > > > Undergraduate > > > > > >> > > > Department of Computer Science and Engineering > > > > > >> > > > University of Moratuwa. > > > > > >> > > > > > > > > >> > > > > > > > > >> > > > > > > > > >> > > > > > > > > >> > > > On Sun, Feb 21, 2016 at 2:30 AM, Minudika Malshan < > > > > > >> > minudika...@gmail.com > > > > > >> > > > > > > > > >> > > > wrote: > > > > > >> > > > > > > > > >> > > > > Hi all, > > > > > >> > > > > > > > > > >> > > > > I am looking forward to contribute in this year GSOC. > > > > > >> > > > > I have a basic understand in writing interpreters for > > > zeppelin > > > > > and > > > > > >> > have > > > > > >> > > > > already implement an interpreter for WSO2 siddhi which > is > > a > > > > > >> complex > > > > > >> > > event > > > > > >> > > > > processing language. > > > > > >> > > > > Could you please give me more details about the > > requirement > > > > for > > > > > >> > > > > implementing a new interpreter for Apache beam? > > > > > >> > > > > It would be a great help for me to getting familiar with > > > this > > > > > >> > > > > implementation. > > > > > >> > > > > > > > > > >> > > > > Thank you! > > > > > >> > > > > > > > > > >> > > > > Minudika Malshan > > > > > >> > > > > Undergraduate > > > > > >> > > > > Department of Computer Science and Engineering > > > > > >> > > > > University of Moratuwa. > > > > > >> > > > > > > > > > >> > > > > > > > > > >> > > > > > > > > > >> > > > > > > > > >> > > > > > > > >> > > > > > > > >> > > > > > > > >> > > -- > > > > > >> > > -- > > > > > >> > > Kind regards, > > > > > >> > > Alexander. > > > > > >> > > > > > > > >> > > > > > > >> > > > > > > > > > > > > > > > > > > > > > > > > > > >
