A new repository https://github.com/apache/dolphinscheduler-mlflow had been created to hold some ml related resource, such as our example and build-in machine learning algorithm
On Tue, May 17, 2022 at 10:19 AM Jiajie Zhong <[email protected]> wrote: > > The related discussion it is from > https://github.com/apache/dolphinscheduler/pull/10036#discussion_r873164232, > and jieguang want to have some build-in mlfow example and currently > use his repository to hold all this code, I think the best way is we > contribute some example to mlflow repository but it take time. So it > is ok to create a new repository to hold this code as a subproject for > dolphinscheduler, and I will create it. > > On Mon, May 16, 2022 at 4:27 PM 周哈哈 <[email protected]> wrote: > > > > Hi, I am supporting the MLflow task plugin to run the MLflow project. I > > have submitted the relevant PR > > (https://github.com/apache/dolphinscheduler/pull/10036). > > > > But I used a personal repository to provide algorithmic power out of the > > box, and hopefully a public repository. > > > > In order for engineers of different backgrounds to better use this task > > plugin, I suggest that: > > > > Create a repository for out-of-the-box machine learning ability. > > Support for setting up directories to use users' own projects. > > > > So, can we create a repository for the first point? > > > > > > > > > > > 2022年5月6日 上午11:56，Jiajie Zhong <[email protected]> 写道： > > > > > > Sound good! and thanks Jieguang brings this up. DolphinScheduler > > > focuses on task scheduling and workflow orchestration, I think machine > > > learning is also another kind of orchestration, and most of the > > > machine learning source data or training samples are from data > > > warehouses or data lakes, which we already supported in the current > > > version. If we DolphinScheduler could support machine learning tasks > > > then users could finish their jobs in one single tool instead of > > > separately. > > > > > > On Mon, Apr 25, 2022 at 2:39 PM 周哈哈 <[email protected]> wrote: > > >> > > >> Hi, community > > >> > > >> I have seen a Machine Learning Platform post on Medium. The post talk > > >> about Lizhi Machine Learning Platform&Apache DolphinScheduler. > > >> https://medium.com/@DolphinScheduler/a-formidable-combination-of-lizhi-machine-learning-platform-dolphinscheduler-creates-new-paradigm-e445938f1af > > >> > > >> <https://medium.com/@DolphinScheduler/a-formidable-combination-of-lizhi-machine-learning-platform-dolphinscheduler-creates-new-paradigm-e445938f1af> > > >> > > >> > > >> I want to use the dolphinscheduler to orchestrate some popular > > >> open-source projects about MLops, such as MLflow, Kubeflow, BentoML, > > >> Seldon Core, ClearML, TonY, etc. I think that can help users to build > > >> their custom MLops systems easier. > > >> > > >> For example, users can choose what deployment tools they like, or what > > >> training tools they like. > > >> > > >> Is somebody interested in the AI task about DS? > > >> > > >> I created an issue about the above: > > >> https://github.com/apache/dolphinscheduler/issues/9725 > > >> <https://github.com/apache/dolphinscheduler/issues/9725> > > >> > > >> > > >> Anyone who is interested is welcome to join us. > > >> > > >> > > >> Best Wish > > >> — Jieguang > > > > > > > > > > > > -- > > > Best Wish > > > — Jiajie > > > > > -- > Best Wish > — Jiajie -- Best Wish — Jiajie
