> As we know, the observability is very important for a service. I think > we might need to define and export some metric to let people know the > ovfs daemon service status > > For now, we have some layer out of box in the OpenDAL(like Prometheus, > OpenTelementry, Dtrace). I'm not sure we should add more metric like > cache hit rate and anything else or not.
Hi, I agree that observability is important. However, it seems we're going too far. The feature requests keep extending, making it a never-ending project. I suggest we leave them as points for future expansion. On Wed, Mar 20, 2024, at 23:24, Manjusaka wrote: > On 2024/3/20 22:33, Runjie Yu wrote: >> Got it, thanks for your suggestions, I'll keep this in mind. >> >> I've put the main content of the proposal in a Google Docs, here's the link >> https://docs.google.com/document/d/1huy8vHcoCTf-GausabR3PCwXIXfRJAEckeyTulp2gDU/edit?usp=sharing >> >> I've specified in the deliverables section a description of the target for >> each storage type, S3 for object storage and HDFS for file storage. >> >> ``` >> 1) A code repository that implements the functions described in the project >> details. The services implemented by OVFS in the code repository need to >> meet the following requirements: (1) VirtioFS implementation, well >> integrated with VMs and QEMU, able to correctly handle VMs read and write >> requests to the file system. (2) Supports the use of distributed object >> storage systems and distributed file systems as storage backends, and >> provides complete and correct support for at least one specific storage >> service type for each storage system type. S3 can be used as the target for >> object storage systems, and HDFS can be used as the target for distributed >> file systems. (3) Supports related configurations of various storage >> systems. Users can configure storage system access and use according to >> actual needs. When an error occurs, users can use the configuration file to >> restart services. >> ``` >> >> On Tue, Mar 19, 2024 at 10:41 PM Manjusaka <[email protected]> wrote: >> >>> On 2024/3/19 20:57, 余润杰 wrote: >>>> Thank you for your suggestion. >>>> >>>> For the first point, I will update this in the proposal with an exact >>> goal for each storage system type. For the second point, I assume this >>> cache is shared by all VMs running in the same host OS. >>>> >>>> Regarding cloud documents, I think this is a very good suggestion. Yes, >>> I need to create and maintain a cloud document. This is not only easy to >>> browse, but by updating and maintaining this document during the GSoC >>> cycle, it helps us focus on our goals and demonstrate the phased results of >>> development. >>>> >>>> For now, I will create a Google Docs tomorrow to display the content of >>> the existing proposal. >>>> >>>> Thanks again for your advice! >>>> >>>> Manjusaka <[email protected] <mailto:[email protected]>> >>> 于2024年3月19日周二 17:48写道： >>>> >>>> On 2024/3/19 16:58, 余润杰 wrote: >>>> > Hi, Xuanwo and Manjusaka. >>>> > >>>> > I hope this email didn’t bother you! >>>> > >>>> > Applications for GSoC 2024 contributors opened today, and I hope >>> to join the GSoC project in Apache OpenDAL as a candidate. I have added you >>> to the list of mentors for the ovfsproject proposal and hope to have the >>> opportunity to be mentored by you! >>>> > >>>> > /Project Mentors: Xuanwo ([email protected] <mailto: >>> [email protected]> <mailto:[email protected] <mailto:[email protected]>>), >>> Manjusaka ([email protected] <mailto:[email protected]> <mailto: >>> [email protected] <mailto:[email protected]>>)/ >>>> > >>>> > I have supplemented and modified some of the content based on >>> previous proposal, mainly including the following points: >>>> > >>>> > 1) Based on the discussion in the previous email, the name of the >>> project was changed from ovirtiofs to ovfs. >>>> > >>>> > 2) Added explanation of ovfs design philosophy. >>>> > >>>> > 3) Avoid ovfs persisting any metadata. >>>> > >>>> > 4) Added potential application scenarios. >>>> > >>>> > 5) Added project deliverables. >>>> > >>>> > 6) Added Why Me And Why Do I Wish To Take Part In GSoC 2024 >>> section. >>>> > >>>> > I hope to submit the proposal this week. I'd like to know if there >>> are still areas that need to be revised or discussed before the proposal is >>> formally submitted. >>>> > >>>> > Have a nice day! >>>> >>>> Hi Runjie >>>> >>>> Glad to hear from you! >>>> >>>> Nice proposal! BTW maybe you can upload the document to a website >>> like Google Docs, Gist, so other people can preview the docs online(LOL >>>> >>>> Most LGTM about this version proposal, I may have some >>> issues/suggestions >>>> >>>> 1. We can make our target to implement only one service backend for >>> each category of the service(like S3 for blob, HDFS for file like, KV >>> storage is not in the plan). This will help us to focus on the function, >>> not the corner behavior. >>>> It would also can help us to reach the full-fuction tested target(I >>> think it's important for us) >>>> >>>> 2. About the cache, I would like to ask: Is the cache shared by all >>> the VM? or each VM would have their own cache >>>> >>>> >>>> Thanks for your nice proposal, have a nice day >>>> >>>> Best >>>> >>>> Manjusaka >>>> >>> >>> Sorry, I forget something in the previous email >>> >>> About the cache, I have another suggestion. I think we should split it >>> into two parts: the read cache and the write cache. The people can choose >>> to enable the cache base on their circumstance >>> >>> For example, if the user mount a S3 bucket as backend which is modified in >>> high frequency(modified by other serivce), the people shouldn't enable the >>> read cache. >>> >>> I think this is would good for the production usage. >>> >>> Best >>> >>> Manjusaka >>> >> > > Thanks for your public docs, I would like to say this is the most > extraordinary proposal I have ever seen before. Great Job! > > BTW, I'm not sure the following should be included into the original > proposal, for now, this is just personal idea. > > As we know, the observability is very important for a service. I think > we might need to define and export some metric to let people know the > ovfs daemon service status > > For now, we have some layer out of box in the OpenDAL(like Prometheus, > OpenTelementry, Dtrace). I'm not sure we should add more metric like > cache hit rate and anything else or not. > > WDYT? > > Best > > Manjusaka -- Xuanwo
