Alan, Yes! We are using the Docker SDK and it's possible to use the API to automate, but it's a little bit harder than automate CLI calls.
Atenciosamente, Rafael J. R. Novello Skype: rafael.novello Blog: http://rafanovello.blogspot.com.br/ Em qua, 24 de out de 2018 às 12:02, Alan Silva <ju...@apache.org> escreveu: > Hi, > > One question here, I understand that we start to use with this PoC the > Docker SDK API, right? > > Why not use the API to expose some endpoints to permit this kind of > automation by devops? > > I think it is possible and it solves the CLI problem, right? > > > On Tue, Oct 23, 2018 at 1:05 PM Rafael Novello < > rafa.reis.nove...@gmail.com> > wrote: > > > Lucas, > > > > Sorry, I didn't understood your question bellow. > > > > "Would it make sense to use the same solution that we will use for > having a > > single-language REPL to have a single-language CLI?" > > > > For DevOps purposes, maybe this new toolbox concept is not ideal. I think > > we can keep the CLI inside the docker container but it will not easy to > > automate > > jobs by this way. > > > > How to deal with this issue? Voting? > > Atenciosamente, > > Rafael J. R. Novello > > > > Skype: rafael.novello > > Blog: http://rafanovello.blogspot.com.br/ > > > > > > Em sex, 19 de out de 2018 às 19:00, Lucas Bonatto Miguel < > > lucasb...@apache.org> escreveu: > > > > > Got it! Thanks for clarifying. > > > > > > Would it make sense to use the same solution that we will use for > having > > a > > > single-language REPL to have a single-language CLI? My only concern > with > > > killing the CLI is that you remove an essential feature for DevOps. > > > > > > - Lucas > > > > > > On Fri, Oct 19, 2018 at 1:52 PM Rafael Novello < > > > rafa.reis.nove...@gmail.com> > > > wrote: > > > > > > > Lucas, > > > > > > > > The idea is that REPL will substitute the actual CLI. It's because > with > > > the > > > > actual concept (using language specific CLI) we will need to > > re-implement > > > > the same features many times and probably each CLI will have a > > different > > > > behavior because some language specific restrictions and/or > > limitations. > > > > > > > > With this new concept, all users will have the same experience > > > interacting > > > > with Marvin REPL and they will use the bests tools to develop their > > > engines > > > > (Jupyter Notebook and/or Lab with different languages support or even > > > > Apache Zeppelin). All the interact will occur through REPL and docker > > > > protocol. > > > > > > > > Alan, Yes! As Lucas said, the concept is the same but we can use > docker > > > to > > > > do the same job. > > > > > > > > Best! > > > > > > > > Em sex, 19 de out de 2018 às 00:39, Lucas Bonatto Miguel < > > > > lucasb...@apache.org> escreveu: > > > > > > > > > Thanks for the clarifications Rafael, so from what I understood, > > > Marvin's > > > > > developers would not use the REPL to explore data and test models, > > i.e. > > > > > develop the engine. Is the idea to build something more like an > > > > interactive > > > > > CLI? I think an interactive CLI would be useful for the developer > > > > > experience, however, it's important to keep the unattended > (current) > > > CLI > > > > in > > > > > place for scenarios where users want to automate Marvin tasks. > > > > > > > > > > Alan, I believe the idea is still the same as you started, but > using > > > > > docker SDK now. > > > > > > > > > > - Lucas > > > > > > > > > > On Thu, Oct 18, 2018 at 1:29 PM Rafael Novello < > > > > > rafa.reis.nove...@gmail.com> > > > > > wrote: > > > > > > > > > > > Hi Lucas! > > > > > > > > > > > > First of all +1 for REPL POCs to Apache! > > > > > > > > > > > > Let me help with some comments: > > > > > > > > > > > > 1 - We have tested NodeJS, Scala and Python and the easiest one > was > > > > > Python. > > > > > > We have found a small project [1] that have all features we > desired > > > for > > > > > > REPL: > > > > > > - Autocomplete > > > > > > - Python commands disabled (the user have only the commands we > > > > provide). > > > > > > The other languages REPL options don't have this feature. > > > > > > - Easy to show output > > > > > > - Etc > > > > > > So, I think the language chosen will not be important here > because > > > the > > > > > user > > > > > > will only interact with the commands that we create. > > > > > > > > > > > > 2 - The "engine-generate" command will download a docker image > that > > > we > > > > > > create for that language and start a container with the basic > > project > > > > > > structure for that language. > > > > > > > > > > > > 3 - The REPL client will use the "docker protocol" to run > command, > > > > > > start/stop services and etc inside the container and it will > > receive > > > > the > > > > > > log stream to show. No, the REPL will no pass code snippets for > > > docker > > > > > > container (I think it will not be necessary) > > > > > > > > > > > > 4 - Yep! Like I said on the first item. > > > > > > > > > > > > [1] - https://github.com/italorossi/ishell > > > > > > > > > > > > Let me know if there is any other question! > > > > > > Best! > > > > > > Rafael J. R. Novello > > > > > > > > > > > > Skype: rafael.novello > > > > > > Blog: http://rafanovello.blogspot.com.br/ > > > > > > > > > > > > > > > > > > Em qui, 18 de out de 2018 às 17:00, Lucas Bonatto Miguel < > > > > > > lucasb...@apache.org> escreveu: > > > > > > > > > > > > > + 1 for migrating the REPL repo to Apache > > > > > > > > > > > > > > I have a few questions about the previous explanation: > > > > > > > 1) The REPL itself, it would be an application in which > > language? > > > > > > Remember > > > > > > > that the main idea is to allow the user to program on his > > preferred > > > > > > > language in the REPL. > > > > > > > 2) Should the engine-generate command also generate a docker > > image > > > > > with > > > > > > > the user's application? > > > > > > > 3) What type of communication would happen between the REPL > and > > > the > > > > > > engine > > > > > > > via Docker SDK? Would the REPL pass snippets of code to be > > executed > > > > by > > > > > > the > > > > > > > docker container? > > > > > > > 4) Have you considered code completion in the REPL? > > > > > > > > > > > > > > > > > > > > > On Thu, Oct 18, 2018 at 7:53 AM Zhang Yifei < > > > yifei.z.l...@gmail.com> > > > > > > > wrote: > > > > > > > > > > > > > > > Ok guys, > > > > > > > > > > > > > > > > The basic idea is to provide only one Toolbox for multiple > > > > languages. > > > > > > > > We are looking for possibility to build a single Marvin Repl, > > > > > > > > instead of severals toolboxes with differentes interfaces or > > > > > commands. > > > > > > > > > > > > > > > > In this case, the engine-generate command will download and > > > start a > > > > > > > Docker > > > > > > > > container with basic engine structure corresponding the > choosed > > > > > > language. > > > > > > > > this means we don't need to build Toolboxes of differents > > > > languages, > > > > > we > > > > > > > > just > > > > > > > > build the engine template of all languages that we want to > > > support > > > > > and > > > > > > > > provide it > > > > > > > > as Docker containers > > > > > > > > > > > > > > > > We have started researches around our basic requirements > like: > > > > > > > > - Repl interface > > > > > > > > - System communication > > > > > > > > - Connection security > > > > > > > > - Tool popularity > > > > > > > > - Update complexity > > > > > > > > - Languages support > > > > > > > > - ...... > > > > > > > > > > > > > > > > And we did some POC with code here: > > > > > > > > https://github.com/marvin-ai/marvin-repl > > > > > > > > > > > > > > > > There is POC testing gRPC using Scala and Python, > > > > > > > > Repl inteface and Docker SDK with NodeJS, > > > > > > > > Repl interface and Docker SDK with Python. > > > > > > > > > > > > > > > > At this moment we prefer the Repl interface + Docker SDK way, > > > > because > > > > > > > good > > > > > > > > part of the requirements > > > > > > > > will be guaranteed by Docker. > > > > > > > > > > > > > > > > With this informations, what do you think? Should we submit > all > > > > this > > > > > > POCs > > > > > > > > to Apache Repo? > > > > > > > > Please feel free to opine. > > > > > > > > > > > > > > > > > > > > > > > > Thats all, thanks!!! > > > > > > > > > > > > > > > > > > > > > > > > Em ter, 16 de out de 2018 às 18:55, Daniel Takabayashi < > > > > > > > > daniel.takabaya...@gmail.com> escreveu: > > > > > > > > > > > > > > > > > Zhang, > > > > > > > > > > > > > > > > > > I think the best approach is give us a better explanation > > about > > > > > this > > > > > > > new > > > > > > > > > feature and how this can help us to archive the desired > > feature > > > > > > > (support > > > > > > > > > multiple languages). Than if the majority agree than I can > do > > > the > > > > > > > merge, > > > > > > > > > just like I did before, but in another branch. > > > > > > > > > > > > > > > > > > Let me know if makes sense, ok? > > > > > > > > > > > > > > > > > > Taka > > > > > > > > > > > > > > > > > > Em ter, 16 de out de 2018 às 14:00, Luciano Resende < > > > > > > > > luckbr1...@gmail.com> > > > > > > > > > escreveu: > > > > > > > > > > > > > > > > > > > So, what is the POC, is it a refactoring of the existing > > > repo? > > > > Or > > > > > > is > > > > > > > > > > it a new rewrite of the repo? > > > > > > > > > > > > > > > > > > > > Just asking as it might make sense to make it a branch > then > > > > > > actually > > > > > > > a > > > > > > > > > > thing in parallel, as this will have an effect for > > releases, > > > > etc. > > > > > > but > > > > > > > > > > you guys know more here than I do. > > > > > > > > > > > > > > > > > > > > Also, it's probably good to have a write up of the main > > > > direction > > > > > > of > > > > > > > > > > the design that can help people get familiar with the new > > > > > approach. > > > > > > > > > > On Tue, Oct 16, 2018 at 11:12 AM Zhang Yifei < > > > > > > yifei.z.l...@gmail.com > > > > > > > > > > > > > > > > > > wrote: > > > > > > > > > > > > > > > > > > > > > > Hey guys, > > > > > > > > > > > > > > > > > > > > > > We have reorganized the Poc repo, and want to merge it > to > > > our > > > > > > > Apache > > > > > > > > > > repo. > > > > > > > > > > > Just making sure here before we do the merge, because > im > > > not > > > > > sure > > > > > > > if > > > > > > > > > > doing > > > > > > > > > > > this will perserve the Git commit history. > > > > > > > > > > > What we are planning to do is: > > > > > > > > > > > > > > > > > > > > > > git filter-branch --subdirectory-filter <git > > > > origin_repository > > > > > > > > > directory> > > > > > > > > > > > -- --all > > > > > > > > > > > mkdir POCs/ > > > > > > > > > > > git mv * POCs/ > > > > > > > > > > > git commit -m "colleted the data to move" > > > > > > > > > > > > > > > > > > > > > > git clone <git destination_repository_url> cd <git > > > > > > > > > > > destination_repository_directory> git remote add > > > > > > > > > origin_repository_branch > > > > > > > > > > > <git origin_repository directory> git pull > > > > > > origin_repository_branch > > > > > > > > > > master > > > > > > > > > > > --allow-unrelated-histories git remote rm > > > > > > origin_repository_branch > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Thanks in advance ! > > > > > > > > > > > > > > > > > > > > > > > > > > -------------------------------------------------------------- > > > > > > > > > > > Zhang Yifei > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > > > > > > > Luciano Resende > > > > > > > > > > http://twitter.com/lresende1975 > > > > > > > > > > http://lresende.blogspot.com/ > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > > > > > > -------------------------------------------------------------- > > > > > > > > Zhang Yifei > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > >
