Take a look at the Docker API https://docs.docker.com/engine/api/v1.24/, I believe this will simplify the solution. Cause if you add another layer (Flask application) you going to need to control/manage this layer as well, creating a chicken and egg problem.
I believe the Docker API is a simple solution that solves the problem (communication between toolbox and containers). Take a look here https://docs.google.com/drawings/d/shajxIpLJHxxMbFgDXiPuhg/image?w=602&h=461&rev=1423&ac=1&parent=1ySERHGBXbHeyCMRookq5UfTuFkzzU0ugtjvR3rF3deY Em sex., 13 de mar. de 2020 às 11:19, Lucas Cardoso Silva < cardosolucas61....@gmail.com> escreveu: > I thought about using a flask application, which can become very fast and > scalable with a Python WSGI server for production like Gunicorn. We can > make the communication establish by HTTPS protocol and use a public and > private key generation system for each engine generated. Another, althougt > complex, alternative would be use OAuth 2.0. > > Regards, > Lucas Cardoso > > Em sex., 13 de mar. de 2020 às 13:58, Daniel Takabayashi < > daniel.takabaya...@gmail.com> escreveu: > > > @Daniel The biggest challenge is to run commands in a remote container > > without having access to the whole O.S. Basically the idea is to create > an > > interface to make possible, in a secure way the communication > > between Toolbox and remote engines. > > > > Creating this we could start to run our engines in clouds and services > like > > Google Run, Kubernetes, Lambda Functions and etc in the same way Marvin > > runs locally. > > > > @Lucas Lets talk about the details of these APIs (interfaces, technology) > > > > Thanks, > > Taka > > > > Em sex., 13 de mar. de 2020 às 07:21, Daniel Lucredio < > > daniel.lucre...@ufscar.br> escreveu: > > > > > Hi Lucas and everyone, > > > > > > Couldn't the developer just run the CLI from inside the container, > > opening > > > a shell inside it? > > > > > > []s > > > > > > Daniel > > > > > > Em qui., 5 de mar. de 2020 às 13:31, Lucas Cardoso Silva < > > > cardosolucas61....@gmail.com> escreveu: > > > > > > > I think we could define better how we will make the communication CLI > > > with > > > > the Docker development instance in the new architecture. The > container > > > > needs a running process to stay active. We could make a communication > > via > > > > API endpoints to keep a web service running on the container > receiving > > > > information from the CLI for the execution of tasks in Marvin. I > > created > > > a > > > > use-case scenario to facilitate understanding and further discussion. > > > > > > > > Scenario: A developer created an engine on marvin through the CLI, he > > > > configures his engine in order to describe all the dependencies of > the > > > > operating system that will be used during development. After the > engine > > > > configuration process, the developer uses the CLI to upload a > > development > > > > environment on a Docker container, that development environment will > > > > contain the refined Marvin toolbox. The CLI then changes its > interface > > in > > > > order to contain the standard commands for using engines. He uses the > > CLI > > > > to communicate with the toolbox endpoints and inform that he wants to > > run > > > > an instance of the notebook. After building the model, the developer > > can, > > > > through the CLI, perform a dryrun and upload the http-server for > > testing. > > > > All of these procedures will be done using the communication through > > API > > > > endpoints. > > > > > > > > Does that sound like a good strategy? Do you have any suggestions, or > > > > something that was already foreseen in the original project? > > > > > > > > Best regards, > > > > Lucas Cardoso > > > > > > > > > >
