Hi the community Do you mind introducing the task plugin of TonY, which is a framework to natively run deep learning jobs on Apache Hadoop. It currently supports TensorFlow, PyTorch, MXNet and Horovod.
TonY enables running either single node or distributed training as a Hadoop application. This native connector, together with other TonY features, aims to run machine learning jobs reliably and flexibly.I think if can be supported into dolphinscheduler, it will make people reuse the existing on-perm Hadoop cluster resources and dolphinscheduler will also be a new workflow engine that covers the machine learning lifecycle jobs. Feel free to discuss if having any ideas. Best Junfan Zhang
