Please create a jira if there is none and assign it to yourself. Thanks!
On Thu, Jul 17, 2014 at 9:55 AM, Ajay Yadav <[email protected]> wrote: > Hi Venkatesh, > > Thanks! I will start right away and update once I have a POC ready. > > > On Wed, Jul 16, 2014 at 11:50 PM, Seetharam Venkatesh < > [email protected]> wrote: > > > Hi Ajay, > > > > Thanks for following this up. Appreciate if you could start a POC > > (proof-of-concept) with your ideas and enhance that over time. PLMK if > you > > need any help in understanding the APIs in falcon. Feel free to create > > issues in Jira if you find an issue. > > > > Thanks and looking forward for your contributions. > > > > > > On Wed, Jul 16, 2014 at 12:13 PM, Ajay Yadav <[email protected]> wrote: > > > > > Hi , > > > > > > Thanks Venkatesh for such an encouraging response. It's absolutely > > > wonderful to see a community where new comers are provided such deep > > > support for contributing to the project. > > > > > > However, I am sorry that the discussion has digressed to python vs. > > java. I > > > should have been clearer with the way I phrased it. Let me try to > > rephrase > > > it > > > > > > *Dust.js alone will not be flexible enough(and insufficient in some > > cases) > > > to build future UI requirements for Admin/Falcon pipeline designer. We > > need > > > one more server side web framework (which will be delegating core > falcon > > > tasks to falcon through REST API)* > > > > > > Again, *I am not advocating/discouraging any client side framework like > > > dust.js/bootstrap, just emphasising need of a new component.* > > > > > > If I try to explain it with an analogy, as per my modest knowledge > Oozie > > > also exposes REST api but also has a server side webapp. > > > > > > I interpreted Venkatesh's original question as "we are already using > > > dust.js and bootstrap and why do we need more components in the tech > > > stack". I tried explaining the need/advantages for a new middle level > > > component - a server side framework sitting between clients and REST > APIs > > > vs. using a client side framework like dust.js on top of REST Api. If > you > > > read my original email with this context, it will help make things > > clearer. > > > > > > Again to help clear Haohui's question, core tasks will still be > delegated > > > to REST layer which already exists and will be the only place for such > > > tasks. However, tasks like user sessions etc. will be done and managed > by > > > the new component. This layer will manage tasks only specific to Web UI > > > like user sessions etc. > > > > > > With new and ever evolving scenarios like pipeline designer, It will be > > > lean since it will be delegating most of tasks to the web server itself > > but > > > at the same time it gives immense flexibility to cater any possible use > > > case/extension through plugins in future. > > > > > > > > > *P.S.* Using Django won't cause change in architecture of current > > > deployment. It will be separate deployment. Django comes with a built > in > > > server to aid development and though it is not recommended for > production > > > set up, it works quite well for 50-80 users. For production level set > up, > > > It is very flexible in terms of deployment and you can use nginx, > > gunicorn, > > > apache etc. > > > > > > > > > > > > > > > > > > On Tue, Jul 15, 2014 at 5:58 PM, Srikanth Sundarrajan < > > [email protected] > > > > > > > wrote: > > > > > > > Perhaps we can settle on JS + any standard MVC framework in the > backend > > > if > > > > deemed necessary. > > > > > > > > Regards > > > > Srikanth Sundarrajan > > > > > > > > ---------------------------------------- > > > > > Date: Tue, 15 Jul 2014 17:25:30 +0530 > > > > > Subject: Re: Admin Dashboard > > > > > From: [email protected] > > > > > To: [email protected] > > > > > > > > > > Hi Ajay, > > > > > > > > > > Thanks for taking the initiative and sorry for my delayed > response. I > > > > think > > > > > I largely concur with Haohui that keeping the stack lean and using > JS > > > for > > > > > the front end and Java for the backend is desirable. > > > > > > > > > > Let me know what you need from serverside perspective and one of us > > can > > > > > jump in and help. Since the requirements are fairly trivial and > > reflect > > > > the > > > > > state in falcon, we should be able to achieve a lean > implementation. > > > > > > > > > > Today Falcon does not need to run python and embeds a jetty > > container. > > > We > > > > > may need to change the deployment architecture to accommodate > python. > > > > > > > > > > Lets take a stab at using any front end tech that you are aware of > > and > > > > > proceed. I'm not hung up on dust nor bootstrap. > > > > > > > > > > Thanks! > > > > > > > > > > > > > > > On Tue, Jul 15, 2014 at 3:32 AM, Haohui Mai <[email protected]> > > > > wrote: > > > > > > > > > >> bq. Try implementing a > > > > >> very basic requirement: user sessions( a logged in user doesn't > need > > > to > > > > >> login again on every page) with existing stack and you will > > understand > > > > what > > > > >> I am trying to say. > > > > >> > > > > >> Note that even simple features like user sessions in Falcon will > be > > > > coupled > > > > >> with other components in the Hadoop ecosystems. In secure set ups, > > > > Falcon > > > > >> will need to get delegation tokens from the HDFS NameNode and > > perform > > > > >> UGI-related operations. I don't see how Django will help in this > > case. > > > > >> > > > > >> bq. P.S. On a side note, you can use python to interact with > Hadoop, > > > > Hive, > > > > >> Pig, > > > > >> Oozie. Again, I am not suggesting to use python instead of java in > > > > existing > > > > >> components. > > > > >> > > > > >> If this is the case, can you elaborate on how Python+Django can > > > benefit > > > > >> Falcon, and what exact requirements need to be solved? > > > > >> > > > > >> > > > > >> > > > > >> > > > > >> On Sun, Jul 13, 2014 at 9:26 PM, Ajay Yadav <[email protected]> > > > wrote: > > > > >> > > > > >>> Hi Haohui, > > > > >>> > > > > >>> Thanks for the reply. I am not suggesting to change existing java > > > code > > > > to > > > > >>> python or saying that we can live without javascript. All I am > > > > suggesting > > > > >>> is that we do need to use a server side web framework. Try > > > > implementing a > > > > >>> very basic requirement: user sessions( a logged in user doesn't > > need > > > to > > > > >>> login again on every page) with existing stack and you will > > > understand > > > > >> what > > > > >>> I am trying to say. > > > > >>> > > > > >>> > > > > >>> P.S. On a side note, you can use python to interact with Hadoop, > > > Hive, > > > > >> Pig, > > > > >>> Oozie. Again, I am not suggesting to use python instead of java > in > > > > >> existing > > > > >>> components. > > > > >>> > > > > >>> > > > > >>> On Sun, Jul 13, 2014 at 11:40 PM, Haohui Mai < > [email protected] > > > > > > > >> wrote: > > > > >>> > > > > >>>> Thanks for your interests in contributing in Falcon. I am one of > > the > > > > >>>> early contributors in Falcon UI, here are my two cents: > > > > >>>> > > > > >>>> Falcon has a browser-server architecture which needs to address > > two > > > > >>>> requirements: > > > > >>>> > > > > >>>> 1. Provide a feature-rich UI (e.g. FALCON-290) to the browser. > > > > >>>> > > > > >>>> 2. The server-side logics have to integrate with existing Hadoop > > > > >>>> ecosystems (e.g., MapReduce, Hive, Pig, and Oozie) which are > > heavily > > > > >>>> based on Java. > > > > >>>> > > > > >>>> While personally I'm pretty open to any other technologies, but > I > > am > > > > >>>> yet to be convinced that how Python+Django will add any values > to > > > > >>>> address the above requirements. More concretely: > > > > >>>> > > > > >>>> i. JavaScript is (and will continue to be) an essential part of > > the > > > UI > > > > >>>> to meet requirement (1). The python stack cannot replace the > > > > >>>> JavaScript part on the client side. > > > > >>>> > > > > >>>> ii. Falcon has to rely on Java to interact with other projects > in > > > the > > > > >>>> Hadoop ecosystems, as there are few bindings in other languages > > for > > > > >>>> these projects. > > > > >>>> > > > > >>>> > > > > >>>> Using Python+Django might help attract attentions from the > python > > > > >>>> community, but I think that from the Falcon points of view, it > is > > > much > > > > >>>> more important to keep align with the whole Hadoop ecosystems > > (which > > > > >> are > > > > >>>> the main audiences of the Falcon project). Moreover, I don't > think > > > > that > > > > >>> the > > > > >>>> merits that you mentioned for server-side technologies have to > be > > > > >> coupled > > > > >>>> with Python+Django -- in fact, there are many more > implementation > > of > > > > >>>> enterprise > > > > >>>> features like exporting to Excel in Java. > > > > >>>> > > > > >>>> In short, I think that the current choice of > Java+REST+JavaScript > > is > > > > >>> quite > > > > >>>> reasonable. Though I'm pretty open to any other technologies, > I'm > > > yet > > > > >> to > > > > >>> be > > > > >>>> convinced that Python+Django can be a superior choice. > > > > >>>> > > > > >>>> > > > > >>>> Thanks, > > > > >>>> Haohui > > > > >>>> > > > > >>>> > > > > >>>> On Sat, Jul 12, 2014 at 7:06 PM, Ajay Yadav <[email protected] > > > > > > >> wrote: > > > > >>>> > > > > >>>>> Any comments? I am awaiting your decision to start working on > > this. > > > > >>>>> > > > > >>>>> > > > > >>>>> On Fri, Jul 11, 2014 at 11:10 AM, Ajay Yadav < > [email protected] > > > > > > > >>> wrote: > > > > >>>>> > > > > >>>>>> Hi Venkatesh, > > > > >>>>>> > > > > >>>>>> Valid question and I am glad that you gave an opportunity to > > > > >> discuss > > > > >>>> it. > > > > >>>>> I > > > > >>>>>> have no particular bias towards using django. I have worked > with > > > > >>> client > > > > >>>>>> side templates(not dust.js but more popular alternatives like > > > > >>> mustache, > > > > >>>>>> handlebars etc.) and client side MV* frameworks like > > backbone.js, > > > > >>>>> angularjs > > > > >>>>>> etc. It is completely possible to use both together, most > common > > > > >>> case. > > > > >>>>>> > > > > >>>>>> *Note: *I looked at the existing code and saw that the current > > > work > > > > >>> in > > > > >>>>>> dust.js is very less - around15-25 lines(only dust templates > > part > > > > >> not > > > > >>>>>> html/css). So may I take the liberty of rephrasing your > question > > > > >>>> slightly > > > > >>>>>> as below > > > > >>>>>> > > > > >>>>>> *Question*: Assuming we can choose only one to keep stack mix > to > > > > >>>> minimum > > > > >>>>>> - shall we use pure client side framework(like dust.js) or > pure > > > > >>> server > > > > >>>>>> side framework(like django)? > > > > >>>>>> > > > > >>>>>> *Answer*: > > > > >>>>>> *TLDR Version:* > > > > >>>>>> Server side components offer more flexibility and > extensibility > > > > >> than > > > > >>> a > > > > >>>>>> pure client side framework. Using only client side stack will > > rule > > > > >>> out > > > > >>>>> use > > > > >>>>>> of database for dashboards, talking to other services,plugins > > etc. > > > > >> by > > > > >>>>> using > > > > >>>>>> just dust.js. Things like hardcoding of rest server end > points, > > > > >>>> graceful > > > > >>>>>> degradation, number of parallel connections available in > browser > > > > >> will > > > > >>>>> limit > > > > >>>>>> pure client side stacks. In other cases it will put a stress > on > > > > >>> falcon > > > > >>>>> REST > > > > >>>>>> API to be modified just for the dashboard. Using python stack > > > > >> instead > > > > >>>> of > > > > >>>>>> javascript, will help in getting more support from community. > > > > >>>>>> > > > > >>>>>> *Long Version* > > > > >>>>>> > > > > >>>>>> - *Convenience vs. Possibility* - Server side components give > us > > > > >>>>>> flexibility and extensibility to build for any use case in > > > > >> future > > > > >>> as > > > > >>>>> well. > > > > >>>>>> Any future feature is possible to be built without dust.js and > > > > >>>>> bootstrap > > > > >>>>>> but not other way around. Dust.js and bootstrap just make it > > > > >>> easier > > > > >>>>> to do > > > > >>>>>> lot of things to enhance user experience. > > > > >>>>>> - *Features* - Consider the login page of the dashboard. We > will > > > > >>>> need > > > > >>>>>> a database to store users/ integrate with ldap. We will need > to > > > > >>>>> generate > > > > >>>>>> new random passwords when users forget passwords. Same for > > > > >>> features > > > > >>>>> like > > > > >>>>>> bookmarks, custom dashboards etc. A lot of features like this > > > > >>> can't > > > > >>>>> be done > > > > >>>>>> if we are not using any server side component. > > > > >>>>>> - *Sanity of REST API* - REST services of falcon are not only > > > > >> for > > > > >>>> web > > > > >>>>>> browser consumption. They can be used by command line clients > > > > >> etc. > > > > >>>> as > > > > >>>>> well. > > > > >>>>>> As the use cases for dashboard will evolve, it will become > more > > > > >>>>> involved to > > > > >>>>>> do things using js alone on client side and it will put lot of > > > > >>>> stress > > > > >>>>> on > > > > >>>>>> modifying REST API to suit just the client. For example > consider > > > > >>> the > > > > >>>>> use > > > > >>>>>> case of allowing an administrator to edit a process from an > HTML > > > > >>>>> form. REST > > > > >>>>>> API will return plain text/xml and it will be tedious and slow > > > > >> to > > > > >>>>> parse > > > > >>>>>> everything in javascript side and then recreate it. Sending a > > > > >> json > > > > >>>> and > > > > >>>>>> accepting json just to cater to web client will be problem for > > > > >>>>>> - Another use case can be to be able to export the reports > like > > > > >>> SLAs > > > > >>>>>> missed in last quarter, metrics in pdf/excel format and it > will > > > > >>> not > > > > >>>> be > > > > >>>>>> possible to do it on client side and in lack of a server side > > > > >>>>> component it > > > > >>>>>> will again put stress on REST API to be modified just for > > > > >>> dashboard > > > > >>>>> UI. > > > > >>>>>> - *Best Practices* - With just dust.js and simple htmls being > > > > >>> served > > > > >>>>>> through a webserver, we will have to deal with naming urls > with > > > > >>>> .html > > > > >>>>>> extensions, hardcoding the REST end points in each HTML etc. > > > > >>>>>> - *Browser constraints* - We will have to deal with browser > > > > >>>>>> constraints like number of parallel connections, cross site > > > > >>>> requests, > > > > >>>>>> javascript not being enabled etc. This will also restrict the > > > > >>>>> development. > > > > >>>>>> > > > > >>>>>> > > > > >>>>>> - *Community Support* - This is purely my impression and I > might > > > > >>> be > > > > >>>>>> completely wrong here. I see that more python developers are > > > > >>>> involved > > > > >>>>> with > > > > >>>>>> big data/data science/machine learning eco system than more > > > > >>>> javascript > > > > >>>>>> developers. Hence it will be easier to find more support for > > > > >>>>> maintaining > > > > >>>>>> and extending these components in community if we develop them > > > > >> in > > > > >>>>> django > > > > >>>>>> instead of dust.js. > > > > >>>>>> > > > > >>>>>> > > > > >>>>>> > > > > >>>>>> > > > > >>>>>> On Fri, Jul 11, 2014 at 2:28 AM, Seetharam Venkatesh < > > > > >>>>>> [email protected]> wrote: > > > > >>>>>> > > > > >>>>>>> I think we are using bootstrap and dust templates. I'd want > to > > > see > > > > >>> if > > > > >>>> we > > > > >>>>>>> need to introduce more technologies into the mix. Do we need > > > > >> Python > > > > >>>> and > > > > >>>>>>> Django? > > > > >>>>>>> > > > > >>>>>>> > > > > >>>>>>> On Thu, Jul 10, 2014 at 6:08 PM, Ajay Yadav < > > [email protected]> > > > > >>>> wrote: > > > > >>>>>>> > > > > >>>>>>>> Thanks Srikanth and Venkatesh. > > > > >>>>>>>> > > > > >>>>>>>> I had looked at the UI and the graph database related code > > and I > > > > >>>> will > > > > >>>>>>> reuse > > > > >>>>>>>> existing code where ever available. I checked for > > compatibility > > > > >> of > > > > >>>>> each > > > > >>>>>>> of > > > > >>>>>>>> the software being used and they are all compatible with APL > > 2.0 > > > > >>>>>>> (Python, > > > > >>>>>>>> Django & Bootstrap). > > > > >>>>>>>> > > > > >>>>>>>> If you can assign the task to me on JIRA also, it will be > > > > >>>>>>>> great(user:ajayyadava) Also, I got the balsamiq account for > > > > >>>>>>> falcon(thanks > > > > >>>>>>>> Srikant) and I will be asking all my questions on mocks > there. > > > > >>>>>>>> > > > > >>>>>>>> > > > > >>>>>>>> > > > > >>>>>>>> > > > > >>>>>>>> On Thu, Jul 10, 2014 at 2:45 PM, Seetharam Venkatesh < > > > > >>>>>>>> [email protected]> wrote: > > > > >>>>>>>> > > > > >>>>>>>>> Welcome aboard Ajay. Please let us know how we can help you > > to > > > > >>> get > > > > >>>>> up > > > > >>>>>>> and > > > > >>>>>>>>> running on this. The REST APIs are documented at: > > > > >>>>>>>>> > > > > >>> > http://falcon.incubator.apache.org/docs/restapi/ResourceList.html > > > > >>>>>>>>> > > > > >>>>>>>>> Thanks! > > > > >>>>>>>>> > > > > >>>>>>>>> > > > > >>>>>>>>> On Thu, Jul 10, 2014 at 7:12 AM, Ajay Yadav < > > > > >> [email protected] > > > > >>>> > > > > >>>>>>> wrote: > > > > >>>>>>>>> > > > > >>>>>>>>>> Hi, > > > > >>>>>>>>>> > > > > >>>>>>>>>> I would like to volunteer for > > > > >>>>>>>>>> https://issues.apache.org/jira/browse/FALCON-37 > > > > >>>>>>>>>> > > > > >>>>>>>>>> I have more than 7 years of experience in building > > > > >> websites. I > > > > >>>>> have > > > > >>>>>>>>>> experience in both client side > > > > >>>>>>> technologies(javascript/HTML5/Bootstrap > > > > >>>>>>>>>> etc.) and server side technologies(Python/Django). > > > > >>>>>>>>>> > > > > >>>>>>>>>> I will need some help with iterating on the mocks as they > > > > >>> don't > > > > >>>>>>> seem to > > > > >>>>>>>>> be > > > > >>>>>>>>>> complete. I spent couple of hours on building a couple of > > > > >>> pages > > > > >>>>>>> using > > > > >>>>>>>>>> "dummy" data to get an idea of what all APIs are required. > > > > >>>> Please > > > > >>>>>>> find > > > > >>>>>>>>> them > > > > >>>>>>>>>> attached. They are just a quick and dirty prototype and UI > > > > >> is > > > > >>>>> pretty > > > > >>>>>>>> raw > > > > >>>>>>>>> as > > > > >>>>>>>>>> of now, to get quick feedback. I used Django,HTML5 & > > > > >>> Bootstrap. > > > > >>>>>>>>>> > > > > >>>>>>>>>> Please let me know if I can own this task. > > > > >>>>>>>>>> > > > > >>>>>>>>>> > > > > >>>>>>>>> > > > > >>>>>>>>> > > > > >>>>>>>>> -- > > > > >>>>>>>>> Regards, > > > > >>>>>>>>> Venkatesh > > > > >>>>>>>>> > > > > >>>>>>>>> “Perfection (in design) is achieved not when there is > nothing > > > > >>> more > > > > >>>>> to > > > > >>>>>>>> add, > > > > >>>>>>>>> but rather when there is nothing more to take away.” > > > > >>>>>>>>> - Antoine de Saint-Exupéry > > > > >>>>>>>>> > > > > >>>>>>>> > > > > >>>>>>> > > > > >>>>>>> > > > > >>>>>>> > > > > >>>>>>> -- > > > > >>>>>>> Regards, > > > > >>>>>>> Venkatesh > > > > >>>>>>> > > > > >>>>>>> “Perfection (in design) is achieved not when there is nothing > > > more > > > > >>> to > > > > >>>>> add, > > > > >>>>>>> but rather when there is nothing more to take away.” > > > > >>>>>>> - Antoine de Saint-Exupéry > > > > >>>>>>> > > > > >>>>>> > > > > >>>>>> > > > > >>>>> > > > > >>>> > > > > >>>> -- > > > > >>>> CONFIDENTIALITY NOTICE > > > > >>>> NOTICE: This message is intended for the use of the individual > or > > > > >> entity > > > > >>> to > > > > >>>> which it is addressed and may contain information that is > > > > confidential, > > > > >>>> privileged and exempt from disclosure under applicable law. If > the > > > > >> reader > > > > >>>> of this message is not the intended recipient, you are hereby > > > notified > > > > >>> that > > > > >>>> any printing, copying, dissemination, distribution, disclosure > or > > > > >>>> forwarding of this communication is strictly prohibited. If you > > have > > > > >>>> received this communication in error, please contact the sender > > > > >>> immediately > > > > >>>> and delete it from your system. Thank You. > > > > >>>> > > > > >>> > > > > >> > > > > >> -- > > > > >> CONFIDENTIALITY NOTICE > > > > >> NOTICE: This message is intended for the use of the individual or > > > > entity to > > > > >> which it is addressed and may contain information that is > > > confidential, > > > > >> privileged and exempt from disclosure under applicable law. If the > > > > reader > > > > >> of this message is not the intended recipient, you are hereby > > notified > > > > that > > > > >> any printing, copying, dissemination, distribution, disclosure or > > > > >> forwarding of this communication is strictly prohibited. If you > have > > > > >> received this communication in error, please contact the sender > > > > immediately > > > > >> and delete it from your system. Thank You. > > > > >> > > > > > > > > > > > > > > > > > > > > -- > > > > > Regards, > > > > > Venkatesh > > > > > > > > > > “Perfection (in design) is achieved not when there is nothing more > to > > > > add, > > > > > but rather when there is nothing more to take away.” > > > > > - Antoine de Saint-Exupéry > > > > > > > > > > > > > > > > > > > -- > > Regards, > > Venkatesh > > > > “Perfection (in design) is achieved not when there is nothing more to > add, > > but rather when there is nothing more to take away.” > > - Antoine de Saint-Exupéry > > > -- Regards, Venkatesh “Perfection (in design) is achieved not when there is nothing more to add, but rather when there is nothing more to take away.” - Antoine de Saint-Exupéry
