Hi, hadoop submarine team I am the head of the development of the machine learning platform of the Dahua Research Institute of Dahua (https://www.dahuatech.com/). We use hadoop submarine as a machine learning engine. Through machine learning algorithms, Characterizing our image and video data,
Currently, hadoop submarine is not good enough for model serving. I hope that the version of hadoop submarine is updated faster. +1 ---------- Forwarded message --------- 发件人： Szilard Nemeth <[email protected]<mailto:[email protected]>> Date: 2019年7月17日周三 下午2:55 Subject: Re: Any thoughts making Submarine a separate Apache project? To: runlin zhang <[email protected]<mailto:[email protected]>> Cc: Xun Liu <[email protected]<mailto:[email protected]>>, Hadoop Common <[email protected]<mailto:[email protected]>>, <[email protected]<mailto:[email protected]>>, Hdfs-dev <[email protected]<mailto:[email protected]>>, <[email protected]<mailto:[email protected]>>, <[email protected]<mailto:[email protected]>> +1, this is a very great idea. As Hadoop repository has already grown huge and contains many projects, I think in general it's a good idea to separate projects in the early phase. On Wed, Jul 17, 2019, 08:50 runlin zhang <[email protected]<mailto:[email protected]>> wrote: +1 ，That will be great ！ > 在 2019年7月10日，下午3:34，Xun Liu <[email protected]<mailto:[email protected]>> 写道： > > Hi all, > > This is Xun Liu contributing to the Submarine project for deep learning > workloads running with big data workloads together on Hadoop clusters. > > There are a bunch of integrations of Submarine to other projects are > finished or going on, such as Apache Zeppelin, TonY, Azkaban. The next step > of Submarine is going to integrate with more projects like Apache Arrow, > Redis, MLflow, etc. & be able to handle end-to-end machine learning use > cases like model serving, notebook management, advanced training > optimizations (like auto parameter tuning, memory cache optimizations for > large datasets for training, etc.), and make it run on other platforms like > Kubernetes or natively on Cloud. LinkedIn also wants to donate TonY project > to Apache so we can put Submarine and TonY together to the same codebase > (Page #30. > https://www.slideshare.net/xkrogen/hadoop-meetup-jan-2019-tony-tensorflow-on-yarn-and-beyond#30 > ). > > This expands the scope of the original Submarine project in exciting new > ways. Toward that end, would it make sense to create a separate Submarine > project at Apache? This can make faster adoption of Submarine, and allow > Submarine to grow to a full-blown machine learning platform. > > There will be lots of technical details to work out, but any initial > thoughts on this? > > Best Regards, > Xun Liu --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected]<mailto:[email protected]> For additional commands, e-mail: [email protected]<mailto:[email protected]>
