Hello Leonardo, The current documentation only contains high level concept and there are not many samples. If you have anything that you can help to share with us please submit a PR to this Repo! https://github.com/apache/incubator-marvin-website
Best Regards Wei On Wed, May 27, 2020 at 6:22 AM Zhang Yifei <yifei.z.l...@gmail.com> wrote: > Hello Leonardo, > > Lots of good advice! > You can find Marvin's issues here: issues.apache.org/jira > > Best regards > > Em seg., 25 de mai. de 2020 às 23:10, Leonardo Tavares < > leonardot1...@gmail.com> escreveu: > > > Hello everyone, first let me introduce myself: > > My name is Leonardo Tavares and I'm an undergrad Computer Engineering > > student at UFSCar doing my undergraduate dissertation supervised by > > Professor Daniel Lucrédio. > > > > I'm using Marvin as a MLOps framework for implementing a proof of concept > > architecture based on a project from work. It consists mainly on having > > multiple models running at the same time and a "model selector" that > parses > > all requests and sends to the correct model. > > > > Along the development of the project I was writing down every issue I > faced > > and points where Marvin helped me compared to vanilla Python. I was going > > to open some issues in the GitHub repository, but I couldn't find the > tab, > > so I'm going to drop all feedback here and if needed I can send each > issue > > in a separate email. > > > > Regarding specs, I was using a Windows 10 laptop but the Marvin > > engines/notebooks were running in a Docker container using Python 3.7. To > > reduce the message size, I'm going to focus on the issues I had during > the > > project development: > > > > - Probably my biggest issue, Marvin's documentation is scarce, meaning > > that most of the problems I faced I had to open the source code and > try > > to > > understand what was happening. > > - The public engines available on GitHub have an older Marvin > template, > > resulting in broken dependencies. > > - The usage of gRPC depends on compiling the "actions.proto" file from > > source (or has some other easier way to use that is not documented). > > - Reloading artifacts using gRPC does not work. Only works when using > > rest. > > - The artifact names are static, not allowing the creation of other > > artifacts and demanding the user knowledge of each artifact for each > > DASFE > > action, despite having artifacts definition on "engine.metadata". > > - If you select a blank cell and tag it as a DASFE action, it will > cause > > the engine to break (maybe could have a "pass" when the cell is > blank?). > > - "marvin test" command always fails, even with thedryrun and > http/gRPC > > server working. > > - "marvin pkg-updatedeps" is broken, get_installed_distributions was > > deprecated in newer versions of pip. > > - "make docker-build" is broken, due to the deprecation of the > > "ppa:webupd8team/java" (along with possibly some other issues). > > > > I'd also like to suggest on a point that isn't a issue but that I think > > would be a great addition to the framework: > > > > - If I managed to correctly understand the artifact versioning, > > it currently works by dumping each "marvin_" variable into a pickle > and > > loading it. Many problems can occur if the source code of the project > > changes between artifact versions. > > - A possible solution is to tag each artifact with a git commit, > > saving the pickle of each artifact along with the commit that it > was > > generated. > > - When reloading, the engine executor would not only load the > pickle > > of the artifact but also change the source code being executed to > the > > version tagged with the artifact. > > > > What do you think? > > Overall, even with all the issues presented above, the framework have > great > > features that speedup the development and management of the engines, > having > > great potential to be used in several fields. > > To me, the most glaring issue is the lack of detailed documentation and > > tutorials, turning away possible adopters due to the steep learning > curve. > > To help with this issue I'd like to help on writing some tutorials on > gRPC, > > if possible :) > > > > Thanks! > > > > -- > > Leonardo Tavares Oliveira > > Graduando em Engenharia de Computação > > Universidade Federal de São Carlos > > > > <http://www.instructables.com/member/leonardots/> > > <https://github.com/leotavares> > > <https://www.linkedin.com/in/leonardot1802/> > > <http://buscatextual.cnpq.br/buscatextual/visualizacv.do?id=K8064478U3> > > > > > -- > -------------------------------------------------------------- > Zhang Yifei >
