Not in a position to help. the active amazon and microsoft engineers are busy in their relevant codebases, leaving: cloudera engineers (4 of us), and the people working on hadoop-cos and hadoop-huaweicloud
In "HADOOP-19236 Integration of Volcano Engine TOS in Hadoop" jinglun is going to work on that. and as an HDFS committer I'm not too worried about the initial integration It'll still take up time and integration effort such as dependency view -even that I'm not going to be active in. Probably the best bet is we talk to them about what it takes to do an independent FS downstream, just as google gcs do On Sun, 4 Aug 2024 at 07:10, Wei-Chiu Chuang <weic...@apache.org> wrote: > Would the Hadoop cloud connector developers feel interested? > > ---------- Forwarded message --------- > From: Wayne Gao <wayne.g...@solidigm.com.invalid> > Date: Tue, Jul 30, 2024 at 7:24 PM > Subject: need help one champion who is interested to bring Solidigm CSAL > cloud storage acceleration Layer into Apache Open Source SW family > To: gene...@incubator.apache.org <gene...@incubator.apache.org>, Wayne > Gao < > wayne.g...@solidigm.com> > > > > > > > Hello dear Apache mentors, > > > > This is Wayne Gao from Solidigm as Principal storage solution architect, > previous I worked at DellEMC objectscale and Isilon as Sr-Principal storage > engineer. Recently Solidigm 60T-120T QLC gain big success in US AI infra > building as backend storage. > > > > I would like to proposal our storage open-source SW to be into Apache. > > CSAL is cloud storage acceleration Layer. We developed this back to 5 years > ago at Intel. Now we continue working on based on customer demands. > > 1. CSAL offer both log structure cache and set associate cache, it is > global commercial release inside Alibaba EBS local disk and Cloud EBS > disk. > 2. With CSAL shaping feature, you can easily build scale out S+Q storage > solution to service the user workloads and maintain density, storage > industry have VAST that using similar design too. > 3. you can export CSAL inside DPU, CIPU, NVMeOF to provide storage > service, it supports K8S CSI too. > 4. CSAL is also one user mode FTL that user can tuned and customized > well for the workload, underlying device support large density QLC, > FDP(Flexible Data Placement) > > > > > > *We released CSAL with in Alibaba Cloud big data high density disk service > and jointly publish paper in Eurosys2024 together attached please find. > *For > Alibaba it is basically same HDD cost but get NVMe SSD performance story. > > > > > > *Brief introduction link from Solidigm* > > ====CSAL solution brief==== > > https://www.solidigm.com/products/technology/cloud-storage-acceleration-layer-write-shaping-csal.html > ===CSAL RSP JBOF design link=== > > > https://www.solidigm.com/products/technology/csal-based-reference-storage-platform.html > ===CSAL RSP solution brief paper=== > > https://www.solidigm.com/content/dam/solidigm/en/site/products/technology/csal-based-reference-storage-platform/documents/csal-based-reference-storage-platform-p5336-solution-brief.pdf > ===RSP step by step==== > > https://www.solidigm.com/content/solidigm/us/en/support-page/drivers-downloads/ka-01786.html > > > > *current open-source repo is in upstreaming SPDK, we have internal closed > source repo too.* > > SPDK: Flash Translation Layer <https://spdk.io/doc/ftl.html> > > spdk/lib/ftl at master · spdk/spdk (github.com) > <https://github.com/spdk/spdk/tree/master/lib/ftl> > > > > > > Wayne Gao > Principle Storage Solution Architect > > wechat: 13636331364 > > solidigm.com > > > > > > > > CONFIDENTIALITY NOTICE: This email and any files attached may contain > confidential information and may be restricted from disclosure by corporate > confidentiality guidelines, or applicable state and federal law. It is > intended solely for the use of the person or entity to whom the email was > addressed. If you are not the intended recipient of this message, be > advised that any dissemination, distribution, or use of the contents of > this message is strictly prohibited. Please delete this email from your > system if you are not the intended recipient. >
