Yibo Cai <yibo....@arm.com> writes: > HPC infrastructure normally leverages RDMA for fast data transfer among > storage nodes and compute nodes. Computation tasks are dispatched to > compute nodes with best fit resources. > > Concretely, we are investigating porting UCX as Flight transport layer. > UCX is a communication framework for modern networks. [1] > Besides HPC usage, many projects (spark, dask, blazingsql, etc) also > adopt UCX to accelerate network transmission. [2][3]
I'm interested in this topic and think it's important that even if the focus is direct to UCX, that there be some thought into MPI interoperability and support for scalable collectives. MPI considers UCX to be an implementation detail, but the two main implementations (MPICH and Open MPI) support it and vendor implementations are all derived from these two.
