GitHub user Xuanwo edited a discussion: New Users: Expert Kit, A Distributed, Expert-Centric Framework for MoE LLM Inference
https://github.com/expert-kit/expert-kit They are using opendal to save/load tensors from local fs or s3. --- `Expert Kit (EK)` is a high-performance framework for scalable MoE (Mixture of Experts) LLM inference. The vision of EK is to provide an efficient foundation of Expert Parallelism (EP) on heterogeneous hardware (e.g., CPU and GPU) over commodity networks (e.g. PCIe, TCP, RDMA), thereby enabling easy deployment and fine-grained expert-level scaling.  GitHub link: https://github.com/apache/opendal/discussions/6153 ---- This is an automatically sent email for [email protected]. To unsubscribe, please send an email to: [email protected]
