Re: Communication between CLI and Docker container in the new architecture.
+1 Em sex., 17 de abr. de 2020 às 11:15, Bruno Sette escreveu: > Standardizing with gRPC seems better to me. It is a good agnostic and safe > language solution. > > Em sex., 17 de abr. de 2020 às 14:56, Lucas Cardoso Silva < > cardosolucas61@gmail.com> escreveu: > > > Standardize with gRPC seems like a better idea. It has all the security > > features we need and maintains the code's agnosticism. What do the rest > of > > you guys think? > > > > Lucas Cardoso > > > > Em sex., 3 de abr. de 2020 às 14:45, Daniel Lucredio < > > daniel.lucre...@ufscar.br> escreveu: > > > > > If we are going for a server to maintain the daemon, instead of a > simple > > > process just to keep the container alive, why not go for a gRPC > solution? > > > In this way, both the engine's endpoints and the toolbox commands would > > be > > > accessible in the same way. And it would also support other languages > for > > > the clients. What do you think? > > > > > > Daniel Lucrédio > > > > > > Em seg., 30 de mar. de 2020 às 14:38, Lucas Cardoso Silva < > > > cardosolucas61@gmail.com> escreveu: > > > > > > > What do you think about using the Pyro4 library to maintain the > daemon? > > > The > > > > license is compatible and it deals with security issues. We can use > the > > > > docker API to provision containers and interact at a high level while > > > > remote method calls are made through Pyro lib. > > > > > > > > Pyro4 github: https://github.com/irmen/Pyro4 > > > > > > > > Best regards, > > > > Lucas Cardoso > > > > > > > > Em qui., 19 de mar. de 2020 às 16:24, Lucas Cardoso Silva < > > > > cardosolucas61@gmail.com> escreveu: > > > > > > > > > Great! I will do some tests with the "daemon" approach and report > any > > > > > updates. Thanks! > > > > > > > > > > Em ter., 17 de mar. de 2020 às 17:40, Daniel Takabayashi < > > > > > daniel.takabaya...@gmail.com> escreveu: > > > > > > > > > >> No, I think everything is here. The "daemon" idea is the best way > to > > > > >> handle > > > > >> this problem, basically, we need to have a Marvin agent running > > inside > > > > the > > > > >> container responsible to receive and run the commands from > outside. > > > But > > > > I > > > > >> didn't get the part related to Jupyter. > > > > >> > > > > >> > > > > >> Em sex., 13 de mar. de 2020 às 15:15, Lucas Cardoso Silva < > > > > >> cardosolucas61@gmail.com> escreveu: > > > > >> > > > > >> > I did some testing with the Docker API. The only problem is > that a > > > > >> > container needs a process running to stay active, if it does not > > > exist > > > > >> the > > > > >> > instance stop before receiving the command through the API. > There > > is > > > > the > > > > >> > possibility to maintain an infinite and inexpensive process > like: > > > tail > > > > >> -f / > > > > >> > dev / null (not a good idea). Another possibility would be > > refactor > > > > the > > > > >> > part of the marvin that would remain in the container to become > a > > > > >> daemon or > > > > >> > leave an instance of Jupyter running (the easy way). Is there > any > > > > other > > > > >> > possibility that I'm not seeing? > > > > >> > > > > > >> > Em sex., 13 de mar. de 2020 às 15:46, Daniel Takabayashi < > > > > >> > daniel.takabaya...@gmail.com> escreveu: > > > > >> > > > > > >> > > Take a look at the Docker API > > > > >> https://docs.docker.com/engine/api/v1.24/, > > > > >> > I > > > > >> > > believe this will simplify the solution. Cause if you add > > another > > > > >> layer > > > > >> > > (Flask application) you going to need to control/manage this > > layer > > > > as > > > > >> > well, > > > > >> > > creating a chicken and egg problem. > > > > >> > > > > > > >> > > I believe the Docker API is a simple solution that solves the > > > > problem > > > > >> > > (communication between toolbox and containers). > > > > >> > > > > > > >> > > Take a look here > > > > >> > > > > > > >> > > > > > > >> > > > > > >> > > > > > > > > > > https://docs.google.com/drawings/d/shajxIpLJHxxMbFgDXiPuhg/image?w=602&h=461&rev=1423&ac=1&parent=1ySERHGBXbHeyCMRookq5UfTuFkzzU0ugtjvR3rF3deY > > > > >> > > > > > > >> > > > > > > >> > > Em sex., 13 de mar. de 2020 às 11:19, Lucas Cardoso Silva < > > > > >> > > cardosolucas61@gmail.com> escreveu: > > > > >> > > > > > > >> > > > I thought about using a flask application, which can become > > very > > > > >> fast > > > > >> > and > > > > >> > > > scalable with a Python WSGI server for production like > > Gunicorn. > > > > We > > > > >> can > > > > >> > > > make the communication establish by HTTPS protocol and use a > > > > public > > > > >> and > > > > >> > > > private key generation system for each engine generated. > > > Another, > > > > >> > > althougt > > > > >> > > > complex, alternative would be use OAuth 2.0. > > > > >> > > > > > > > >> > > > Regards, > > > > >> > > > Lucas Cardoso > > > > >> > > > > > > > >> > > > Em sex., 13 de mar. de 2020 às 13:58, Daniel Takabayashi < > > > > >> > > > daniel.takabaya...
Re: Communication between CLI and Docker container in the new architecture.
Standardizing with gRPC seems better to me. It is a good agnostic and safe language solution. Em sex., 17 de abr. de 2020 às 14:56, Lucas Cardoso Silva < cardosolucas61@gmail.com> escreveu: > Standardize with gRPC seems like a better idea. It has all the security > features we need and maintains the code's agnosticism. What do the rest of > you guys think? > > Lucas Cardoso > > Em sex., 3 de abr. de 2020 às 14:45, Daniel Lucredio < > daniel.lucre...@ufscar.br> escreveu: > > > If we are going for a server to maintain the daemon, instead of a simple > > process just to keep the container alive, why not go for a gRPC solution? > > In this way, both the engine's endpoints and the toolbox commands would > be > > accessible in the same way. And it would also support other languages for > > the clients. What do you think? > > > > Daniel Lucrédio > > > > Em seg., 30 de mar. de 2020 às 14:38, Lucas Cardoso Silva < > > cardosolucas61@gmail.com> escreveu: > > > > > What do you think about using the Pyro4 library to maintain the daemon? > > The > > > license is compatible and it deals with security issues. We can use the > > > docker API to provision containers and interact at a high level while > > > remote method calls are made through Pyro lib. > > > > > > Pyro4 github: https://github.com/irmen/Pyro4 > > > > > > Best regards, > > > Lucas Cardoso > > > > > > Em qui., 19 de mar. de 2020 às 16:24, Lucas Cardoso Silva < > > > cardosolucas61@gmail.com> escreveu: > > > > > > > Great! I will do some tests with the "daemon" approach and report any > > > > updates. Thanks! > > > > > > > > Em ter., 17 de mar. de 2020 às 17:40, Daniel Takabayashi < > > > > daniel.takabaya...@gmail.com> escreveu: > > > > > > > >> No, I think everything is here. The "daemon" idea is the best way to > > > >> handle > > > >> this problem, basically, we need to have a Marvin agent running > inside > > > the > > > >> container responsible to receive and run the commands from outside. > > But > > > I > > > >> didn't get the part related to Jupyter. > > > >> > > > >> > > > >> Em sex., 13 de mar. de 2020 às 15:15, Lucas Cardoso Silva < > > > >> cardosolucas61@gmail.com> escreveu: > > > >> > > > >> > I did some testing with the Docker API. The only problem is that a > > > >> > container needs a process running to stay active, if it does not > > exist > > > >> the > > > >> > instance stop before receiving the command through the API. There > is > > > the > > > >> > possibility to maintain an infinite and inexpensive process like: > > tail > > > >> -f / > > > >> > dev / null (not a good idea). Another possibility would be > refactor > > > the > > > >> > part of the marvin that would remain in the container to become a > > > >> daemon or > > > >> > leave an instance of Jupyter running (the easy way). Is there any > > > other > > > >> > possibility that I'm not seeing? > > > >> > > > > >> > Em sex., 13 de mar. de 2020 às 15:46, Daniel Takabayashi < > > > >> > daniel.takabaya...@gmail.com> escreveu: > > > >> > > > > >> > > Take a look at the Docker API > > > >> https://docs.docker.com/engine/api/v1.24/, > > > >> > I > > > >> > > believe this will simplify the solution. Cause if you add > another > > > >> layer > > > >> > > (Flask application) you going to need to control/manage this > layer > > > as > > > >> > well, > > > >> > > creating a chicken and egg problem. > > > >> > > > > > >> > > I believe the Docker API is a simple solution that solves the > > > problem > > > >> > > (communication between toolbox and containers). > > > >> > > > > > >> > > Take a look here > > > >> > > > > > >> > > > > > >> > > > > >> > > > > > > https://docs.google.com/drawings/d/shajxIpLJHxxMbFgDXiPuhg/image?w=602&h=461&rev=1423&ac=1&parent=1ySERHGBXbHeyCMRookq5UfTuFkzzU0ugtjvR3rF3deY > > > >> > > > > > >> > > > > > >> > > Em sex., 13 de mar. de 2020 às 11:19, Lucas Cardoso Silva < > > > >> > > cardosolucas61@gmail.com> escreveu: > > > >> > > > > > >> > > > I thought about using a flask application, which can become > very > > > >> fast > > > >> > and > > > >> > > > scalable with a Python WSGI server for production like > Gunicorn. > > > We > > > >> can > > > >> > > > make the communication establish by HTTPS protocol and use a > > > public > > > >> and > > > >> > > > private key generation system for each engine generated. > > Another, > > > >> > > althougt > > > >> > > > complex, alternative would be use OAuth 2.0. > > > >> > > > > > > >> > > > Regards, > > > >> > > > Lucas Cardoso > > > >> > > > > > > >> > > > Em sex., 13 de mar. de 2020 às 13:58, Daniel Takabayashi < > > > >> > > > daniel.takabaya...@gmail.com> escreveu: > > > >> > > > > > > >> > > > > @Daniel The biggest challenge is to run commands in a remote > > > >> > container > > > >> > > > > without having access to the whole O.S. Basically the idea > is > > to > > > >> > create > > > >> > > > an > > > >> > > > > interface to make possible, in a secure way the > communication > > > >> > > > > be
Re: Communication between CLI and Docker container in the new architecture.
Standardize with gRPC seems like a better idea. It has all the security features we need and maintains the code's agnosticism. What do the rest of you guys think? Lucas Cardoso Em sex., 3 de abr. de 2020 às 14:45, Daniel Lucredio < daniel.lucre...@ufscar.br> escreveu: > If we are going for a server to maintain the daemon, instead of a simple > process just to keep the container alive, why not go for a gRPC solution? > In this way, both the engine's endpoints and the toolbox commands would be > accessible in the same way. And it would also support other languages for > the clients. What do you think? > > Daniel Lucrédio > > Em seg., 30 de mar. de 2020 às 14:38, Lucas Cardoso Silva < > cardosolucas61@gmail.com> escreveu: > > > What do you think about using the Pyro4 library to maintain the daemon? > The > > license is compatible and it deals with security issues. We can use the > > docker API to provision containers and interact at a high level while > > remote method calls are made through Pyro lib. > > > > Pyro4 github: https://github.com/irmen/Pyro4 > > > > Best regards, > > Lucas Cardoso > > > > Em qui., 19 de mar. de 2020 às 16:24, Lucas Cardoso Silva < > > cardosolucas61@gmail.com> escreveu: > > > > > Great! I will do some tests with the "daemon" approach and report any > > > updates. Thanks! > > > > > > Em ter., 17 de mar. de 2020 às 17:40, Daniel Takabayashi < > > > daniel.takabaya...@gmail.com> escreveu: > > > > > >> No, I think everything is here. The "daemon" idea is the best way to > > >> handle > > >> this problem, basically, we need to have a Marvin agent running inside > > the > > >> container responsible to receive and run the commands from outside. > But > > I > > >> didn't get the part related to Jupyter. > > >> > > >> > > >> Em sex., 13 de mar. de 2020 às 15:15, Lucas Cardoso Silva < > > >> cardosolucas61@gmail.com> escreveu: > > >> > > >> > I did some testing with the Docker API. The only problem is that a > > >> > container needs a process running to stay active, if it does not > exist > > >> the > > >> > instance stop before receiving the command through the API. There is > > the > > >> > possibility to maintain an infinite and inexpensive process like: > tail > > >> -f / > > >> > dev / null (not a good idea). Another possibility would be refactor > > the > > >> > part of the marvin that would remain in the container to become a > > >> daemon or > > >> > leave an instance of Jupyter running (the easy way). Is there any > > other > > >> > possibility that I'm not seeing? > > >> > > > >> > Em sex., 13 de mar. de 2020 às 15:46, Daniel Takabayashi < > > >> > daniel.takabaya...@gmail.com> escreveu: > > >> > > > >> > > Take a look at the Docker API > > >> https://docs.docker.com/engine/api/v1.24/, > > >> > I > > >> > > believe this will simplify the solution. Cause if you add another > > >> layer > > >> > > (Flask application) you going to need to control/manage this layer > > as > > >> > well, > > >> > > creating a chicken and egg problem. > > >> > > > > >> > > I believe the Docker API is a simple solution that solves the > > problem > > >> > > (communication between toolbox and containers). > > >> > > > > >> > > Take a look here > > >> > > > > >> > > > > >> > > > >> > > > https://docs.google.com/drawings/d/shajxIpLJHxxMbFgDXiPuhg/image?w=602&h=461&rev=1423&ac=1&parent=1ySERHGBXbHeyCMRookq5UfTuFkzzU0ugtjvR3rF3deY > > >> > > > > >> > > > > >> > > Em sex., 13 de mar. de 2020 às 11:19, Lucas Cardoso Silva < > > >> > > cardosolucas61@gmail.com> escreveu: > > >> > > > > >> > > > I thought about using a flask application, which can become very > > >> fast > > >> > and > > >> > > > scalable with a Python WSGI server for production like Gunicorn. > > We > > >> can > > >> > > > make the communication establish by HTTPS protocol and use a > > public > > >> and > > >> > > > private key generation system for each engine generated. > Another, > > >> > > althougt > > >> > > > complex, alternative would be use OAuth 2.0. > > >> > > > > > >> > > > Regards, > > >> > > > Lucas Cardoso > > >> > > > > > >> > > > Em sex., 13 de mar. de 2020 às 13:58, Daniel Takabayashi < > > >> > > > daniel.takabaya...@gmail.com> escreveu: > > >> > > > > > >> > > > > @Daniel The biggest challenge is to run commands in a remote > > >> > container > > >> > > > > without having access to the whole O.S. Basically the idea is > to > > >> > create > > >> > > > an > > >> > > > > interface to make possible, in a secure way the communication > > >> > > > > between Toolbox and remote engines. > > >> > > > > > > >> > > > > Creating this we could start to run our engines in clouds and > > >> > services > > >> > > > like > > >> > > > > Google Run, Kubernetes, Lambda Functions and etc in the same > way > > >> > Marvin > > >> > > > > runs locally. > > >> > > > > > > >> > > > > @Lucas Lets talk about the details of these APIs (interfaces, > > >> > > technology) > > >> > > > > > > >> > > > > Thanks, > > >> > > > > Taka > > >>
Re: Communication between CLI and Docker container in the new architecture.
If we are going for a server to maintain the daemon, instead of a simple process just to keep the container alive, why not go for a gRPC solution? In this way, both the engine's endpoints and the toolbox commands would be accessible in the same way. And it would also support other languages for the clients. What do you think? Daniel Lucrédio Em seg., 30 de mar. de 2020 às 14:38, Lucas Cardoso Silva < cardosolucas61@gmail.com> escreveu: > What do you think about using the Pyro4 library to maintain the daemon? The > license is compatible and it deals with security issues. We can use the > docker API to provision containers and interact at a high level while > remote method calls are made through Pyro lib. > > Pyro4 github: https://github.com/irmen/Pyro4 > > Best regards, > Lucas Cardoso > > Em qui., 19 de mar. de 2020 às 16:24, Lucas Cardoso Silva < > cardosolucas61@gmail.com> escreveu: > > > Great! I will do some tests with the "daemon" approach and report any > > updates. Thanks! > > > > Em ter., 17 de mar. de 2020 às 17:40, Daniel Takabayashi < > > daniel.takabaya...@gmail.com> escreveu: > > > >> No, I think everything is here. The "daemon" idea is the best way to > >> handle > >> this problem, basically, we need to have a Marvin agent running inside > the > >> container responsible to receive and run the commands from outside. But > I > >> didn't get the part related to Jupyter. > >> > >> > >> Em sex., 13 de mar. de 2020 às 15:15, Lucas Cardoso Silva < > >> cardosolucas61@gmail.com> escreveu: > >> > >> > I did some testing with the Docker API. The only problem is that a > >> > container needs a process running to stay active, if it does not exist > >> the > >> > instance stop before receiving the command through the API. There is > the > >> > possibility to maintain an infinite and inexpensive process like: tail > >> -f / > >> > dev / null (not a good idea). Another possibility would be refactor > the > >> > part of the marvin that would remain in the container to become a > >> daemon or > >> > leave an instance of Jupyter running (the easy way). Is there any > other > >> > possibility that I'm not seeing? > >> > > >> > Em sex., 13 de mar. de 2020 às 15:46, Daniel Takabayashi < > >> > daniel.takabaya...@gmail.com> escreveu: > >> > > >> > > Take a look at the Docker API > >> https://docs.docker.com/engine/api/v1.24/, > >> > I > >> > > believe this will simplify the solution. Cause if you add another > >> layer > >> > > (Flask application) you going to need to control/manage this layer > as > >> > well, > >> > > creating a chicken and egg problem. > >> > > > >> > > I believe the Docker API is a simple solution that solves the > problem > >> > > (communication between toolbox and containers). > >> > > > >> > > Take a look here > >> > > > >> > > > >> > > >> > https://docs.google.com/drawings/d/shajxIpLJHxxMbFgDXiPuhg/image?w=602&h=461&rev=1423&ac=1&parent=1ySERHGBXbHeyCMRookq5UfTuFkzzU0ugtjvR3rF3deY > >> > > > >> > > > >> > > Em sex., 13 de mar. de 2020 às 11:19, Lucas Cardoso Silva < > >> > > cardosolucas61@gmail.com> escreveu: > >> > > > >> > > > I thought about using a flask application, which can become very > >> fast > >> > and > >> > > > scalable with a Python WSGI server for production like Gunicorn. > We > >> can > >> > > > make the communication establish by HTTPS protocol and use a > public > >> and > >> > > > private key generation system for each engine generated. Another, > >> > > althougt > >> > > > complex, alternative would be use OAuth 2.0. > >> > > > > >> > > > Regards, > >> > > > Lucas Cardoso > >> > > > > >> > > > Em sex., 13 de mar. de 2020 às 13:58, Daniel Takabayashi < > >> > > > daniel.takabaya...@gmail.com> escreveu: > >> > > > > >> > > > > @Daniel The biggest challenge is to run commands in a remote > >> > container > >> > > > > without having access to the whole O.S. Basically the idea is to > >> > create > >> > > > an > >> > > > > interface to make possible, in a secure way the communication > >> > > > > between Toolbox and remote engines. > >> > > > > > >> > > > > Creating this we could start to run our engines in clouds and > >> > services > >> > > > like > >> > > > > Google Run, Kubernetes, Lambda Functions and etc in the same way > >> > Marvin > >> > > > > runs locally. > >> > > > > > >> > > > > @Lucas Lets talk about the details of these APIs (interfaces, > >> > > technology) > >> > > > > > >> > > > > Thanks, > >> > > > > Taka > >> > > > > > >> > > > > Em sex., 13 de mar. de 2020 às 07:21, Daniel Lucredio < > >> > > > > daniel.lucre...@ufscar.br> escreveu: > >> > > > > > >> > > > > > Hi Lucas and everyone, > >> > > > > > > >> > > > > > Couldn't the developer just run the CLI from inside the > >> container, > >> > > > > opening > >> > > > > > a shell inside it? > >> > > > > > > >> > > > > > []s > >> > > > > > > >> > > > > > Daniel > >> > > > > > > >> > > > > > Em qui., 5 de mar. de 2020 às 13:31, Lucas Cardoso Silva < > >> > > > > > cardosolucas61@gmail.
Re: Communication between CLI and Docker container in the new architecture.
What do you think about using the Pyro4 library to maintain the daemon? The license is compatible and it deals with security issues. We can use the docker API to provision containers and interact at a high level while remote method calls are made through Pyro lib. Pyro4 github: https://github.com/irmen/Pyro4 Best regards, Lucas Cardoso Em qui., 19 de mar. de 2020 às 16:24, Lucas Cardoso Silva < cardosolucas61@gmail.com> escreveu: > Great! I will do some tests with the "daemon" approach and report any > updates. Thanks! > > Em ter., 17 de mar. de 2020 às 17:40, Daniel Takabayashi < > daniel.takabaya...@gmail.com> escreveu: > >> No, I think everything is here. The "daemon" idea is the best way to >> handle >> this problem, basically, we need to have a Marvin agent running inside the >> container responsible to receive and run the commands from outside. But I >> didn't get the part related to Jupyter. >> >> >> Em sex., 13 de mar. de 2020 às 15:15, Lucas Cardoso Silva < >> cardosolucas61@gmail.com> escreveu: >> >> > I did some testing with the Docker API. The only problem is that a >> > container needs a process running to stay active, if it does not exist >> the >> > instance stop before receiving the command through the API. There is the >> > possibility to maintain an infinite and inexpensive process like: tail >> -f / >> > dev / null (not a good idea). Another possibility would be refactor the >> > part of the marvin that would remain in the container to become a >> daemon or >> > leave an instance of Jupyter running (the easy way). Is there any other >> > possibility that I'm not seeing? >> > >> > Em sex., 13 de mar. de 2020 às 15:46, Daniel Takabayashi < >> > daniel.takabaya...@gmail.com> escreveu: >> > >> > > Take a look at the Docker API >> https://docs.docker.com/engine/api/v1.24/, >> > I >> > > believe this will simplify the solution. Cause if you add another >> layer >> > > (Flask application) you going to need to control/manage this layer as >> > well, >> > > creating a chicken and egg problem. >> > > >> > > I believe the Docker API is a simple solution that solves the problem >> > > (communication between toolbox and containers). >> > > >> > > Take a look here >> > > >> > > >> > >> https://docs.google.com/drawings/d/shajxIpLJHxxMbFgDXiPuhg/image?w=602&h=461&rev=1423&ac=1&parent=1ySERHGBXbHeyCMRookq5UfTuFkzzU0ugtjvR3rF3deY >> > > >> > > >> > > Em sex., 13 de mar. de 2020 às 11:19, Lucas Cardoso Silva < >> > > cardosolucas61@gmail.com> escreveu: >> > > >> > > > I thought about using a flask application, which can become very >> fast >> > and >> > > > scalable with a Python WSGI server for production like Gunicorn. We >> can >> > > > make the communication establish by HTTPS protocol and use a public >> and >> > > > private key generation system for each engine generated. Another, >> > > althougt >> > > > complex, alternative would be use OAuth 2.0. >> > > > >> > > > Regards, >> > > > Lucas Cardoso >> > > > >> > > > Em sex., 13 de mar. de 2020 às 13:58, Daniel Takabayashi < >> > > > daniel.takabaya...@gmail.com> escreveu: >> > > > >> > > > > @Daniel The biggest challenge is to run commands in a remote >> > container >> > > > > without having access to the whole O.S. Basically the idea is to >> > create >> > > > an >> > > > > interface to make possible, in a secure way the communication >> > > > > between Toolbox and remote engines. >> > > > > >> > > > > Creating this we could start to run our engines in clouds and >> > services >> > > > like >> > > > > Google Run, Kubernetes, Lambda Functions and etc in the same way >> > Marvin >> > > > > runs locally. >> > > > > >> > > > > @Lucas Lets talk about the details of these APIs (interfaces, >> > > technology) >> > > > > >> > > > > Thanks, >> > > > > Taka >> > > > > >> > > > > Em sex., 13 de mar. de 2020 às 07:21, Daniel Lucredio < >> > > > > daniel.lucre...@ufscar.br> escreveu: >> > > > > >> > > > > > Hi Lucas and everyone, >> > > > > > >> > > > > > Couldn't the developer just run the CLI from inside the >> container, >> > > > > opening >> > > > > > a shell inside it? >> > > > > > >> > > > > > []s >> > > > > > >> > > > > > Daniel >> > > > > > >> > > > > > Em qui., 5 de mar. de 2020 às 13:31, Lucas Cardoso Silva < >> > > > > > cardosolucas61@gmail.com> escreveu: >> > > > > > >> > > > > > > I think we could define better how we will make the >> communication >> > > CLI >> > > > > > with >> > > > > > > the Docker development instance in the new architecture. The >> > > > container >> > > > > > > needs a running process to stay active. We could make a >> > > communication >> > > > > via >> > > > > > > API endpoints to keep a web service running on the container >> > > > receiving >> > > > > > > information from the CLI for the execution of tasks in >> Marvin. I >> > > > > created >> > > > > > a >> > > > > > > use-case scenario to facilitate understanding and further >> > > discussion. >> > > > > > > >> > > > > > > Scenario: A developer crea
Re: Communication between CLI and Docker container in the new architecture.
Great! I will do some tests with the "daemon" approach and report any updates. Thanks! Em ter., 17 de mar. de 2020 às 17:40, Daniel Takabayashi < daniel.takabaya...@gmail.com> escreveu: > No, I think everything is here. The "daemon" idea is the best way to handle > this problem, basically, we need to have a Marvin agent running inside the > container responsible to receive and run the commands from outside. But I > didn't get the part related to Jupyter. > > > Em sex., 13 de mar. de 2020 às 15:15, Lucas Cardoso Silva < > cardosolucas61@gmail.com> escreveu: > > > I did some testing with the Docker API. The only problem is that a > > container needs a process running to stay active, if it does not exist > the > > instance stop before receiving the command through the API. There is the > > possibility to maintain an infinite and inexpensive process like: tail > -f / > > dev / null (not a good idea). Another possibility would be refactor the > > part of the marvin that would remain in the container to become a daemon > or > > leave an instance of Jupyter running (the easy way). Is there any other > > possibility that I'm not seeing? > > > > Em sex., 13 de mar. de 2020 às 15:46, Daniel Takabayashi < > > daniel.takabaya...@gmail.com> escreveu: > > > > > Take a look at the Docker API > https://docs.docker.com/engine/api/v1.24/, > > I > > > believe this will simplify the solution. Cause if you add another layer > > > (Flask application) you going to need to control/manage this layer as > > well, > > > creating a chicken and egg problem. > > > > > > I believe the Docker API is a simple solution that solves the problem > > > (communication between toolbox and containers). > > > > > > Take a look here > > > > > > > > > https://docs.google.com/drawings/d/shajxIpLJHxxMbFgDXiPuhg/image?w=602&h=461&rev=1423&ac=1&parent=1ySERHGBXbHeyCMRookq5UfTuFkzzU0ugtjvR3rF3deY > > > > > > > > > Em sex., 13 de mar. de 2020 às 11:19, Lucas Cardoso Silva < > > > cardosolucas61@gmail.com> escreveu: > > > > > > > I thought about using a flask application, which can become very fast > > and > > > > scalable with a Python WSGI server for production like Gunicorn. We > can > > > > make the communication establish by HTTPS protocol and use a public > and > > > > private key generation system for each engine generated. Another, > > > althougt > > > > complex, alternative would be use OAuth 2.0. > > > > > > > > Regards, > > > > Lucas Cardoso > > > > > > > > Em sex., 13 de mar. de 2020 às 13:58, Daniel Takabayashi < > > > > daniel.takabaya...@gmail.com> escreveu: > > > > > > > > > @Daniel The biggest challenge is to run commands in a remote > > container > > > > > without having access to the whole O.S. Basically the idea is to > > create > > > > an > > > > > interface to make possible, in a secure way the communication > > > > > between Toolbox and remote engines. > > > > > > > > > > Creating this we could start to run our engines in clouds and > > services > > > > like > > > > > Google Run, Kubernetes, Lambda Functions and etc in the same way > > Marvin > > > > > runs locally. > > > > > > > > > > @Lucas Lets talk about the details of these APIs (interfaces, > > > technology) > > > > > > > > > > Thanks, > > > > > Taka > > > > > > > > > > Em sex., 13 de mar. de 2020 às 07:21, Daniel Lucredio < > > > > > daniel.lucre...@ufscar.br> escreveu: > > > > > > > > > > > Hi Lucas and everyone, > > > > > > > > > > > > Couldn't the developer just run the CLI from inside the > container, > > > > > opening > > > > > > a shell inside it? > > > > > > > > > > > > []s > > > > > > > > > > > > Daniel > > > > > > > > > > > > Em qui., 5 de mar. de 2020 às 13:31, Lucas Cardoso Silva < > > > > > > cardosolucas61@gmail.com> escreveu: > > > > > > > > > > > > > I think we could define better how we will make the > communication > > > CLI > > > > > > with > > > > > > > the Docker development instance in the new architecture. The > > > > container > > > > > > > needs a running process to stay active. We could make a > > > communication > > > > > via > > > > > > > API endpoints to keep a web service running on the container > > > > receiving > > > > > > > information from the CLI for the execution of tasks in Marvin. > I > > > > > created > > > > > > a > > > > > > > use-case scenario to facilitate understanding and further > > > discussion. > > > > > > > > > > > > > > Scenario: A developer created an engine on marvin through the > > CLI, > > > he > > > > > > > configures his engine in order to describe all the dependencies > > of > > > > the > > > > > > > operating system that will be used during development. After > the > > > > engine > > > > > > > configuration process, the developer uses the CLI to upload a > > > > > development > > > > > > > environment on a Docker container, that development environment > > > will > > > > > > > contain the refined Marvin toolbox. The CLI then changes its > > > > interface > > > > > in > > > > > > > order to contain the stan
Re: Communication between CLI and Docker container in the new architecture.
No, I think everything is here. The "daemon" idea is the best way to handle this problem, basically, we need to have a Marvin agent running inside the container responsible to receive and run the commands from outside. But I didn't get the part related to Jupyter. Em sex., 13 de mar. de 2020 às 15:15, Lucas Cardoso Silva < cardosolucas61@gmail.com> escreveu: > I did some testing with the Docker API. The only problem is that a > container needs a process running to stay active, if it does not exist the > instance stop before receiving the command through the API. There is the > possibility to maintain an infinite and inexpensive process like: tail -f / > dev / null (not a good idea). Another possibility would be refactor the > part of the marvin that would remain in the container to become a daemon or > leave an instance of Jupyter running (the easy way). Is there any other > possibility that I'm not seeing? > > Em sex., 13 de mar. de 2020 às 15:46, Daniel Takabayashi < > daniel.takabaya...@gmail.com> escreveu: > > > Take a look at the Docker API https://docs.docker.com/engine/api/v1.24/, > I > > believe this will simplify the solution. Cause if you add another layer > > (Flask application) you going to need to control/manage this layer as > well, > > creating a chicken and egg problem. > > > > I believe the Docker API is a simple solution that solves the problem > > (communication between toolbox and containers). > > > > Take a look here > > > > > https://docs.google.com/drawings/d/shajxIpLJHxxMbFgDXiPuhg/image?w=602&h=461&rev=1423&ac=1&parent=1ySERHGBXbHeyCMRookq5UfTuFkzzU0ugtjvR3rF3deY > > > > > > Em sex., 13 de mar. de 2020 às 11:19, Lucas Cardoso Silva < > > cardosolucas61@gmail.com> escreveu: > > > > > I thought about using a flask application, which can become very fast > and > > > scalable with a Python WSGI server for production like Gunicorn. We can > > > make the communication establish by HTTPS protocol and use a public and > > > private key generation system for each engine generated. Another, > > althougt > > > complex, alternative would be use OAuth 2.0. > > > > > > Regards, > > > Lucas Cardoso > > > > > > Em sex., 13 de mar. de 2020 às 13:58, Daniel Takabayashi < > > > daniel.takabaya...@gmail.com> escreveu: > > > > > > > @Daniel The biggest challenge is to run commands in a remote > container > > > > without having access to the whole O.S. Basically the idea is to > create > > > an > > > > interface to make possible, in a secure way the communication > > > > between Toolbox and remote engines. > > > > > > > > Creating this we could start to run our engines in clouds and > services > > > like > > > > Google Run, Kubernetes, Lambda Functions and etc in the same way > Marvin > > > > runs locally. > > > > > > > > @Lucas Lets talk about the details of these APIs (interfaces, > > technology) > > > > > > > > Thanks, > > > > Taka > > > > > > > > Em sex., 13 de mar. de 2020 às 07:21, Daniel Lucredio < > > > > daniel.lucre...@ufscar.br> escreveu: > > > > > > > > > Hi Lucas and everyone, > > > > > > > > > > Couldn't the developer just run the CLI from inside the container, > > > > opening > > > > > a shell inside it? > > > > > > > > > > []s > > > > > > > > > > Daniel > > > > > > > > > > Em qui., 5 de mar. de 2020 às 13:31, Lucas Cardoso Silva < > > > > > cardosolucas61@gmail.com> escreveu: > > > > > > > > > > > I think we could define better how we will make the communication > > CLI > > > > > with > > > > > > the Docker development instance in the new architecture. The > > > container > > > > > > needs a running process to stay active. We could make a > > communication > > > > via > > > > > > API endpoints to keep a web service running on the container > > > receiving > > > > > > information from the CLI for the execution of tasks in Marvin. I > > > > created > > > > > a > > > > > > use-case scenario to facilitate understanding and further > > discussion. > > > > > > > > > > > > Scenario: A developer created an engine on marvin through the > CLI, > > he > > > > > > configures his engine in order to describe all the dependencies > of > > > the > > > > > > operating system that will be used during development. After the > > > engine > > > > > > configuration process, the developer uses the CLI to upload a > > > > development > > > > > > environment on a Docker container, that development environment > > will > > > > > > contain the refined Marvin toolbox. The CLI then changes its > > > interface > > > > in > > > > > > order to contain the standard commands for using engines. He uses > > the > > > > CLI > > > > > > to communicate with the toolbox endpoints and inform that he > wants > > to > > > > run > > > > > > an instance of the notebook. After building the model, the > > developer > > > > can, > > > > > > through the CLI, perform a dryrun and upload the http-server for > > > > testing. > > > > > > All of these procedures will be done using the communication > > through > > > > API > >
Re: Communication between CLI and Docker container in the new architecture.
I did some testing with the Docker API. The only problem is that a container needs a process running to stay active, if it does not exist the instance stop before receiving the command through the API. There is the possibility to maintain an infinite and inexpensive process like: tail -f / dev / null (not a good idea). Another possibility would be refactor the part of the marvin that would remain in the container to become a daemon or leave an instance of Jupyter running (the easy way). Is there any other possibility that I'm not seeing? Em sex., 13 de mar. de 2020 às 15:46, Daniel Takabayashi < daniel.takabaya...@gmail.com> escreveu: > Take a look at the Docker API https://docs.docker.com/engine/api/v1.24/, I > believe this will simplify the solution. Cause if you add another layer > (Flask application) you going to need to control/manage this layer as well, > creating a chicken and egg problem. > > I believe the Docker API is a simple solution that solves the problem > (communication between toolbox and containers). > > Take a look here > > https://docs.google.com/drawings/d/shajxIpLJHxxMbFgDXiPuhg/image?w=602&h=461&rev=1423&ac=1&parent=1ySERHGBXbHeyCMRookq5UfTuFkzzU0ugtjvR3rF3deY > > > Em sex., 13 de mar. de 2020 às 11:19, Lucas Cardoso Silva < > cardosolucas61@gmail.com> escreveu: > > > I thought about using a flask application, which can become very fast and > > scalable with a Python WSGI server for production like Gunicorn. We can > > make the communication establish by HTTPS protocol and use a public and > > private key generation system for each engine generated. Another, > althougt > > complex, alternative would be use OAuth 2.0. > > > > Regards, > > Lucas Cardoso > > > > Em sex., 13 de mar. de 2020 às 13:58, Daniel Takabayashi < > > daniel.takabaya...@gmail.com> escreveu: > > > > > @Daniel The biggest challenge is to run commands in a remote container > > > without having access to the whole O.S. Basically the idea is to create > > an > > > interface to make possible, in a secure way the communication > > > between Toolbox and remote engines. > > > > > > Creating this we could start to run our engines in clouds and services > > like > > > Google Run, Kubernetes, Lambda Functions and etc in the same way Marvin > > > runs locally. > > > > > > @Lucas Lets talk about the details of these APIs (interfaces, > technology) > > > > > > Thanks, > > > Taka > > > > > > Em sex., 13 de mar. de 2020 às 07:21, Daniel Lucredio < > > > daniel.lucre...@ufscar.br> escreveu: > > > > > > > Hi Lucas and everyone, > > > > > > > > Couldn't the developer just run the CLI from inside the container, > > > opening > > > > a shell inside it? > > > > > > > > []s > > > > > > > > Daniel > > > > > > > > Em qui., 5 de mar. de 2020 às 13:31, Lucas Cardoso Silva < > > > > cardosolucas61@gmail.com> escreveu: > > > > > > > > > I think we could define better how we will make the communication > CLI > > > > with > > > > > the Docker development instance in the new architecture. The > > container > > > > > needs a running process to stay active. We could make a > communication > > > via > > > > > API endpoints to keep a web service running on the container > > receiving > > > > > information from the CLI for the execution of tasks in Marvin. I > > > created > > > > a > > > > > use-case scenario to facilitate understanding and further > discussion. > > > > > > > > > > Scenario: A developer created an engine on marvin through the CLI, > he > > > > > configures his engine in order to describe all the dependencies of > > the > > > > > operating system that will be used during development. After the > > engine > > > > > configuration process, the developer uses the CLI to upload a > > > development > > > > > environment on a Docker container, that development environment > will > > > > > contain the refined Marvin toolbox. The CLI then changes its > > interface > > > in > > > > > order to contain the standard commands for using engines. He uses > the > > > CLI > > > > > to communicate with the toolbox endpoints and inform that he wants > to > > > run > > > > > an instance of the notebook. After building the model, the > developer > > > can, > > > > > through the CLI, perform a dryrun and upload the http-server for > > > testing. > > > > > All of these procedures will be done using the communication > through > > > API > > > > > endpoints. > > > > > > > > > > Does that sound like a good strategy? Do you have any suggestions, > or > > > > > something that was already foreseen in the original project? > > > > > > > > > > Best regards, > > > > > Lucas Cardoso > > > > > > > > > > > > > > >
Re: Communication between CLI and Docker container in the new architecture.
Take a look at the Docker API https://docs.docker.com/engine/api/v1.24/, I believe this will simplify the solution. Cause if you add another layer (Flask application) you going to need to control/manage this layer as well, creating a chicken and egg problem. I believe the Docker API is a simple solution that solves the problem (communication between toolbox and containers). Take a look here https://docs.google.com/drawings/d/shajxIpLJHxxMbFgDXiPuhg/image?w=602&h=461&rev=1423&ac=1&parent=1ySERHGBXbHeyCMRookq5UfTuFkzzU0ugtjvR3rF3deY Em sex., 13 de mar. de 2020 às 11:19, Lucas Cardoso Silva < cardosolucas61@gmail.com> escreveu: > I thought about using a flask application, which can become very fast and > scalable with a Python WSGI server for production like Gunicorn. We can > make the communication establish by HTTPS protocol and use a public and > private key generation system for each engine generated. Another, althougt > complex, alternative would be use OAuth 2.0. > > Regards, > Lucas Cardoso > > Em sex., 13 de mar. de 2020 às 13:58, Daniel Takabayashi < > daniel.takabaya...@gmail.com> escreveu: > > > @Daniel The biggest challenge is to run commands in a remote container > > without having access to the whole O.S. Basically the idea is to create > an > > interface to make possible, in a secure way the communication > > between Toolbox and remote engines. > > > > Creating this we could start to run our engines in clouds and services > like > > Google Run, Kubernetes, Lambda Functions and etc in the same way Marvin > > runs locally. > > > > @Lucas Lets talk about the details of these APIs (interfaces, technology) > > > > Thanks, > > Taka > > > > Em sex., 13 de mar. de 2020 às 07:21, Daniel Lucredio < > > daniel.lucre...@ufscar.br> escreveu: > > > > > Hi Lucas and everyone, > > > > > > Couldn't the developer just run the CLI from inside the container, > > opening > > > a shell inside it? > > > > > > []s > > > > > > Daniel > > > > > > Em qui., 5 de mar. de 2020 às 13:31, Lucas Cardoso Silva < > > > cardosolucas61@gmail.com> escreveu: > > > > > > > I think we could define better how we will make the communication CLI > > > with > > > > the Docker development instance in the new architecture. The > container > > > > needs a running process to stay active. We could make a communication > > via > > > > API endpoints to keep a web service running on the container > receiving > > > > information from the CLI for the execution of tasks in Marvin. I > > created > > > a > > > > use-case scenario to facilitate understanding and further discussion. > > > > > > > > Scenario: A developer created an engine on marvin through the CLI, he > > > > configures his engine in order to describe all the dependencies of > the > > > > operating system that will be used during development. After the > engine > > > > configuration process, the developer uses the CLI to upload a > > development > > > > environment on a Docker container, that development environment will > > > > contain the refined Marvin toolbox. The CLI then changes its > interface > > in > > > > order to contain the standard commands for using engines. He uses the > > CLI > > > > to communicate with the toolbox endpoints and inform that he wants to > > run > > > > an instance of the notebook. After building the model, the developer > > can, > > > > through the CLI, perform a dryrun and upload the http-server for > > testing. > > > > All of these procedures will be done using the communication through > > API > > > > endpoints. > > > > > > > > Does that sound like a good strategy? Do you have any suggestions, or > > > > something that was already foreseen in the original project? > > > > > > > > Best regards, > > > > Lucas Cardoso > > > > > > > > > >
Re: Communication between CLI and Docker container in the new architecture.
I thought about using a flask application, which can become very fast and scalable with a Python WSGI server for production like Gunicorn. We can make the communication establish by HTTPS protocol and use a public and private key generation system for each engine generated. Another, althougt complex, alternative would be use OAuth 2.0. Regards, Lucas Cardoso Em sex., 13 de mar. de 2020 às 13:58, Daniel Takabayashi < daniel.takabaya...@gmail.com> escreveu: > @Daniel The biggest challenge is to run commands in a remote container > without having access to the whole O.S. Basically the idea is to create an > interface to make possible, in a secure way the communication > between Toolbox and remote engines. > > Creating this we could start to run our engines in clouds and services like > Google Run, Kubernetes, Lambda Functions and etc in the same way Marvin > runs locally. > > @Lucas Lets talk about the details of these APIs (interfaces, technology) > > Thanks, > Taka > > Em sex., 13 de mar. de 2020 às 07:21, Daniel Lucredio < > daniel.lucre...@ufscar.br> escreveu: > > > Hi Lucas and everyone, > > > > Couldn't the developer just run the CLI from inside the container, > opening > > a shell inside it? > > > > []s > > > > Daniel > > > > Em qui., 5 de mar. de 2020 às 13:31, Lucas Cardoso Silva < > > cardosolucas61@gmail.com> escreveu: > > > > > I think we could define better how we will make the communication CLI > > with > > > the Docker development instance in the new architecture. The container > > > needs a running process to stay active. We could make a communication > via > > > API endpoints to keep a web service running on the container receiving > > > information from the CLI for the execution of tasks in Marvin. I > created > > a > > > use-case scenario to facilitate understanding and further discussion. > > > > > > Scenario: A developer created an engine on marvin through the CLI, he > > > configures his engine in order to describe all the dependencies of the > > > operating system that will be used during development. After the engine > > > configuration process, the developer uses the CLI to upload a > development > > > environment on a Docker container, that development environment will > > > contain the refined Marvin toolbox. The CLI then changes its interface > in > > > order to contain the standard commands for using engines. He uses the > CLI > > > to communicate with the toolbox endpoints and inform that he wants to > run > > > an instance of the notebook. After building the model, the developer > can, > > > through the CLI, perform a dryrun and upload the http-server for > testing. > > > All of these procedures will be done using the communication through > API > > > endpoints. > > > > > > Does that sound like a good strategy? Do you have any suggestions, or > > > something that was already foreseen in the original project? > > > > > > Best regards, > > > Lucas Cardoso > > > > > >
Re: Communication between CLI and Docker container in the new architecture.
@Daniel The biggest challenge is to run commands in a remote container without having access to the whole O.S. Basically the idea is to create an interface to make possible, in a secure way the communication between Toolbox and remote engines. Creating this we could start to run our engines in clouds and services like Google Run, Kubernetes, Lambda Functions and etc in the same way Marvin runs locally. @Lucas Lets talk about the details of these APIs (interfaces, technology) Thanks, Taka Em sex., 13 de mar. de 2020 às 07:21, Daniel Lucredio < daniel.lucre...@ufscar.br> escreveu: > Hi Lucas and everyone, > > Couldn't the developer just run the CLI from inside the container, opening > a shell inside it? > > []s > > Daniel > > Em qui., 5 de mar. de 2020 às 13:31, Lucas Cardoso Silva < > cardosolucas61@gmail.com> escreveu: > > > I think we could define better how we will make the communication CLI > with > > the Docker development instance in the new architecture. The container > > needs a running process to stay active. We could make a communication via > > API endpoints to keep a web service running on the container receiving > > information from the CLI for the execution of tasks in Marvin. I created > a > > use-case scenario to facilitate understanding and further discussion. > > > > Scenario: A developer created an engine on marvin through the CLI, he > > configures his engine in order to describe all the dependencies of the > > operating system that will be used during development. After the engine > > configuration process, the developer uses the CLI to upload a development > > environment on a Docker container, that development environment will > > contain the refined Marvin toolbox. The CLI then changes its interface in > > order to contain the standard commands for using engines. He uses the CLI > > to communicate with the toolbox endpoints and inform that he wants to run > > an instance of the notebook. After building the model, the developer can, > > through the CLI, perform a dryrun and upload the http-server for testing. > > All of these procedures will be done using the communication through API > > endpoints. > > > > Does that sound like a good strategy? Do you have any suggestions, or > > something that was already foreseen in the original project? > > > > Best regards, > > Lucas Cardoso > > >
Re: Communication between CLI and Docker container in the new architecture.
Hi Lucas and everyone, Couldn't the developer just run the CLI from inside the container, opening a shell inside it? []s Daniel Em qui., 5 de mar. de 2020 às 13:31, Lucas Cardoso Silva < cardosolucas61@gmail.com> escreveu: > I think we could define better how we will make the communication CLI with > the Docker development instance in the new architecture. The container > needs a running process to stay active. We could make a communication via > API endpoints to keep a web service running on the container receiving > information from the CLI for the execution of tasks in Marvin. I created a > use-case scenario to facilitate understanding and further discussion. > > Scenario: A developer created an engine on marvin through the CLI, he > configures his engine in order to describe all the dependencies of the > operating system that will be used during development. After the engine > configuration process, the developer uses the CLI to upload a development > environment on a Docker container, that development environment will > contain the refined Marvin toolbox. The CLI then changes its interface in > order to contain the standard commands for using engines. He uses the CLI > to communicate with the toolbox endpoints and inform that he wants to run > an instance of the notebook. After building the model, the developer can, > through the CLI, perform a dryrun and upload the http-server for testing. > All of these procedures will be done using the communication through API > endpoints. > > Does that sound like a good strategy? Do you have any suggestions, or > something that was already foreseen in the original project? > > Best regards, > Lucas Cardoso >
Re: Communication between CLI and Docker container in the new architecture.
+1 Em sex., 13 de mar. de 2020 às 11:06, Bruno Sette escreveu: > LGTM > > +1 > > Em qui., 5 de mar. de 2020 às 13:31, Lucas Cardoso Silva < > cardosolucas61@gmail.com> escreveu: > > > I think we could define better how we will make the communication CLI > with > > the Docker development instance in the new architecture. The container > > needs a running process to stay active. We could make a communication via > > API endpoints to keep a web service running on the container receiving > > information from the CLI for the execution of tasks in Marvin. I created > a > > use-case scenario to facilitate understanding and further discussion. > > > > Scenario: A developer created an engine on marvin through the CLI, he > > configures his engine in order to describe all the dependencies of the > > operating system that will be used during development. After the engine > > configuration process, the developer uses the CLI to upload a development > > environment on a Docker container, that development environment will > > contain the refined Marvin toolbox. The CLI then changes its interface in > > order to contain the standard commands for using engines. He uses the CLI > > to communicate with the toolbox endpoints and inform that he wants to run > > an instance of the notebook. After building the model, the developer can, > > through the CLI, perform a dryrun and upload the http-server for testing. > > All of these procedures will be done using the communication through API > > endpoints. > > > > Does that sound like a good strategy? Do you have any suggestions, or > > something that was already foreseen in the original project? > > > > Best regards, > > Lucas Cardoso > > > > > -- > Att., > > *Bruno Silva Sette* >
Re: Communication between CLI and Docker container in the new architecture.
LGTM +1 Em qui., 5 de mar. de 2020 às 13:31, Lucas Cardoso Silva < cardosolucas61@gmail.com> escreveu: > I think we could define better how we will make the communication CLI with > the Docker development instance in the new architecture. The container > needs a running process to stay active. We could make a communication via > API endpoints to keep a web service running on the container receiving > information from the CLI for the execution of tasks in Marvin. I created a > use-case scenario to facilitate understanding and further discussion. > > Scenario: A developer created an engine on marvin through the CLI, he > configures his engine in order to describe all the dependencies of the > operating system that will be used during development. After the engine > configuration process, the developer uses the CLI to upload a development > environment on a Docker container, that development environment will > contain the refined Marvin toolbox. The CLI then changes its interface in > order to contain the standard commands for using engines. He uses the CLI > to communicate with the toolbox endpoints and inform that he wants to run > an instance of the notebook. After building the model, the developer can, > through the CLI, perform a dryrun and upload the http-server for testing. > All of these procedures will be done using the communication through API > endpoints. > > Does that sound like a good strategy? Do you have any suggestions, or > something that was already foreseen in the original project? > > Best regards, > Lucas Cardoso > -- Att., *Bruno Silva Sette*
Communication between CLI and Docker container in the new architecture.
I think we could define better how we will make the communication CLI with the Docker development instance in the new architecture. The container needs a running process to stay active. We could make a communication via API endpoints to keep a web service running on the container receiving information from the CLI for the execution of tasks in Marvin. I created a use-case scenario to facilitate understanding and further discussion. Scenario: A developer created an engine on marvin through the CLI, he configures his engine in order to describe all the dependencies of the operating system that will be used during development. After the engine configuration process, the developer uses the CLI to upload a development environment on a Docker container, that development environment will contain the refined Marvin toolbox. The CLI then changes its interface in order to contain the standard commands for using engines. He uses the CLI to communicate with the toolbox endpoints and inform that he wants to run an instance of the notebook. After building the model, the developer can, through the CLI, perform a dryrun and upload the http-server for testing. All of these procedures will be done using the communication through API endpoints. Does that sound like a good strategy? Do you have any suggestions, or something that was already foreseen in the original project? Best regards, Lucas Cardoso
