RGW, future directions
RGW is maturing. Beside looking at performance, which highly ties into RADOS performance, we'd like to hear whether there are certain pain points or future directions that you (you as in the ceph community) would like to see us taking. There are a few directions that we were thinking about: 1. Extend Object Storage API Swift and S3 has some features that we don't currently support. We can certainly extend our functionality, however, is there any demand for more features? E.g., self destructing objects, web site, user logs, etc. 2. Better OpenStack interoperability Keystone support? Other? 3. New features Some examples: - multitenancy: api for domains and user management - snapshots - computation front end: upload object, then do some data transformation/calculation. - simple key-value api 4. CDMI Sage brought up the CDMI support question to ceph-devel, and I don't remember him getting any response. Is there any intereset in CDMI? 5. Native apache/nginx module or embedded web server We still need to prove that the web server is a bottleneck, or poses scaling issues. Writing a correct native nginx module will require turning rgw process model into event driven, which is not going to be easy. 6. Improve garbage collection Currently rgw generates intent logs for garabage removal that require running an external tool later, which is an administrative pain. We can implement other solutions (OSD side garbage collection, integrating cleanup process into the gateway, etc.) but we need to understand the priority. 7. libradosgw We have had this in mind for some time now. Creating a programming api for rgw, not too different from librados and librbd. It'll hopefully make code much cleaner. It will allow users to write different front ends for the rgw backend, and it will make it easier for users to write applications that interact with the backend, e.g., do processing on objects that users uploaded, FUSE for rgw without S3 as an intermediate, etc. 8. Administration tools improvement We can always do better there. 9. Other ideas? Any comments are welcome! Thanks, Yehuda -- To unsubscribe from this list: send the line "unsubscribe ceph-devel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: RGW, future directions
On Tue, May 22, 2012 at 8:07 PM, Yehuda Sadeh wrote: > RGW is maturing. Beside looking at performance, which highly ties into > RADOS performance, we'd like to hear whether there are certain pain > points or future directions that you (you as in the ceph community) > would like to see us taking. > > There are a few directions that we were thinking about: > > 1. Extend Object Storage API > > Swift and S3 has some features that we don't currently support. We can > certainly extend our functionality, however, is there any demand for > more features? E.g., self destructing objects, web site, user logs, > etc. More compatibility with S3 and swift is good. > > 2. Better OpenStack interoperability > > Keystone support? Other? > > 3. New features > > Some examples: > > - multitenancy: api for domains and user management > - snapshots > - computation front end: upload object, then do some data > transformation/calculation. > - simple key-value api > > 4. CDMI > > Sage brought up the CDMI support question to ceph-devel, and I don't > remember him getting any response. Is there any intereset in CDMI? > > > 5. Native apache/nginx module or embedded web server > > We still need to prove that the web server is a bottleneck, or poses > scaling issues. Writing a correct native nginx module will require > turning rgw process model into event driven, which is not going to be > easy. > nginx module is nice thing. > 6. Improve garbage collection > > Currently rgw generates intent logs for garabage removal that require > running an external tool later, which is an administrative pain. We > can implement other solutions (OSD side garbage collection, > integrating cleanup process into the gateway, etc.) but we need to > understand the priority. crontab can handle this task for now, but in big workload, better if it's integrated, like scrub, and tuned via conf > > 7. libradosgw > > We have had this in mind for some time now. Creating a programming api > for rgw, not too different from librados and librbd. It'll hopefully > make code much cleaner. It will allow users to write different front > ends for the rgw backend, and it will make it easier for users to > write applications that interact with the backend, e.g., do processing > on objects that users uploaded, FUSE for rgw without S3 as an > intermediate, etc. > > 8. Administration tools improvement > > We can always do better there. > > 9. Other ideas? - I would like to see a feature that can make replication between clusters. For start 2 clusters. It's a very good feature, when you have two datacenters, and repliacation goes via hight speed link, but aplications at top of clusters does not need to handle this tasks, and data are consistent. - gracefull upgrade - reload cluster config without restart daemons - or maybe this exist right now ?? > > > Any comments are welcome! > > Thanks, > Yehuda > -- > To unsubscribe from this list: send the line "unsubscribe ceph-devel" in > the body of a message to majord...@vger.kernel.org > More majordomo info at http://vger.kernel.org/majordomo-info.html -- - Pozdrawiam Sławek "sZiBis" Skowron -- To unsubscribe from this list: send the line "unsubscribe ceph-devel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: RGW, future directions
On Tue, May 22, 2012 at 11:25 AM, Sławomir Skowron wrote: > On Tue, May 22, 2012 at 8:07 PM, Yehuda Sadeh wrote: >> RGW is maturing. Beside looking at performance, which highly ties into >> RADOS performance, we'd like to hear whether there are certain pain >> points or future directions that you (you as in the ceph community) >> would like to see us taking. >> >> There are a few directions that we were thinking about: >> >> 1. Extend Object Storage API >> >> Swift and S3 has some features that we don't currently support. We can >> certainly extend our functionality, however, is there any demand for >> more features? E.g., self destructing objects, web site, user logs, >> etc. > > More compatibility with S3 and swift is good. Any specific functional interest? > >> >> 2. Better OpenStack interoperability >> >> Keystone support? Other? >> >> 3. New features >> >> Some examples: >> >> - multitenancy: api for domains and user management >> - snapshots >> - computation front end: upload object, then do some data >> transformation/calculation. >> - simple key-value api >> >> 4. CDMI >> >> Sage brought up the CDMI support question to ceph-devel, and I don't >> remember him getting any response. Is there any intereset in CDMI? >> >> >> 5. Native apache/nginx module or embedded web server >> >> We still need to prove that the web server is a bottleneck, or poses >> scaling issues. Writing a correct native nginx module will require >> turning rgw process model into event driven, which is not going to be >> easy. >> > > nginx module is nice thing. It would be nice to have some concrete numbers as to where apache or nginx with fastcgi holding us back, and how a dedicated module is going to improve that. As a rule of thumb it is a no brainer, but still we want to have a better understanding of the situation before we dive into such a project. > >> 6. Improve garbage collection >> >> Currently rgw generates intent logs for garabage removal that require >> running an external tool later, which is an administrative pain. We >> can implement other solutions (OSD side garbage collection, >> integrating cleanup process into the gateway, etc.) but we need to >> understand the priority. > > crontab can handle this task for now, but in big workload, better if > it's integrated, like scrub, and tuned via conf Yeah. One of the original ideas was to leverage scrubbing for objects expiration (issue #1994). The discussion never converged, as the devil is as always in the details. We can revive that discussion. > >> >> 7. libradosgw >> >> We have had this in mind for some time now. Creating a programming api >> for rgw, not too different from librados and librbd. It'll hopefully >> make code much cleaner. It will allow users to write different front >> ends for the rgw backend, and it will make it easier for users to >> write applications that interact with the backend, e.g., do processing >> on objects that users uploaded, FUSE for rgw without S3 as an >> intermediate, etc. >> >> 8. Administration tools improvement >> >> We can always do better there. >> >> 9. Other ideas? > > - I would like to see a feature that can make replication between > clusters. For start 2 clusters. It's a very good feature, when you > have two datacenters, and repliacation goes via hight speed link, but > aplications at top of clusters does not need to handle this tasks, and > data are consistent. This is a more generic ceph issue. We do need, however, to be able to support multiple clusters in rgw. Opened issue #2460. > > - gracefull upgrade Can you elaborate on that? This is always our intention, we've put some mechanisms in place to help with that, although it is not always possible. > > - reload cluster config without restart daemons - or maybe this exist > right now ?? > I opened issue #2459. Thanks, Yehuda -- To unsubscribe from this list: send the line "unsubscribe ceph-devel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: RGW, future directions
On Tue, May 22, 2012 at 9:09 PM, Yehuda Sadeh wrote: > On Tue, May 22, 2012 at 11:25 AM, Sławomir Skowron wrote: >> On Tue, May 22, 2012 at 8:07 PM, Yehuda Sadeh wrote: >>> RGW is maturing. Beside looking at performance, which highly ties into >>> RADOS performance, we'd like to hear whether there are certain pain >>> points or future directions that you (you as in the ceph community) >>> would like to see us taking. >>> >>> There are a few directions that we were thinking about: >>> >>> 1. Extend Object Storage API >>> >>> Swift and S3 has some features that we don't currently support. We can >>> certainly extend our functionality, however, is there any demand for >>> more features? E.g., self destructing objects, web site, user logs, >>> etc. >> >> More compatibility with S3 and swift is good. Now i have no issues about that, because we use only simples parts of S3, but there is always, a good if compatibility is better. > > Any specific functional interest? > >> >>> >>> 2. Better OpenStack interoperability >>> >>> Keystone support? Other? >>> >>> 3. New features >>> >>> Some examples: >>> >>> - multitenancy: api for domains and user management multitenancy - quotas, QoS, and user management it's nice feature for admin, especially in cloud systems, with many different type of control, even performance, and nice integration API for external accounting services in cloud system. >>> - snapshots >>> - computation front end: upload object, then do some data >>> transformation/calculation. >>> - simple key-value api >>> >>> 4. CDMI >>> >>> Sage brought up the CDMI support question to ceph-devel, and I don't >>> remember him getting any response. Is there any intereset in CDMI? >>> >>> >>> 5. Native apache/nginx module or embedded web server >>> >>> We still need to prove that the web server is a bottleneck, or poses >>> scaling issues. Writing a correct native nginx module will require >>> turning rgw process model into event driven, which is not going to be >>> easy. >>> >> >> nginx module is nice thing. > > It would be nice to have some concrete numbers as to where apache or > nginx with fastcgi holding us back, and how a dedicated module is > going to improve that. As a rule of thumb it is a no brainer, but > still we want to have a better understanding of the situation before > we dive into such a project. Now it's hard to say, beacause for now there is now intensive traffic @ this cluster. I have only synthetic test. Reads from one machine in cluster are about 500req/s (no nginx cache) via nginx <-> radosgw fcgi, but machine can handle more (cpu, ram, drives). I need to test that how it's realy holding this a whole service. With cache enabled in nginx, reads operation are no issue, but writes (PUTs, DELETEs) will always be problematic, but now radosgw have improved with this a lot. > >> >>> 6. Improve garbage collection >>> >>> Currently rgw generates intent logs for garabage removal that require >>> running an external tool later, which is an administrative pain. We >>> can implement other solutions (OSD side garbage collection, >>> integrating cleanup process into the gateway, etc.) but we need to >>> understand the priority. >> >> crontab can handle this task for now, but in big workload, better if >> it's integrated, like scrub, and tuned via conf > > Yeah. One of the original ideas was to leverage scrubbing for objects > expiration (issue #1994). The discussion never converged, as the devil > is as always in the details. We can revive that discussion. > >> >>> >>> 7. libradosgw >>> >>> We have had this in mind for some time now. Creating a programming api >>> for rgw, not too different from librados and librbd. It'll hopefully >>> make code much cleaner. It will allow users to write different front >>> ends for the rgw backend, and it will make it easier for users to >>> write applications that interact with the backend, e.g., do processing >>> on objects that users uploaded, FUSE for rgw without S3 as an >>> intermediate, etc. >>> >>> 8. Administration tools improvement >>> >>> We can always do better there. >>> >>> 9. Other ideas? >> >> - I would like to see a feature that can make replication between >> clusters. For start 2 clusters. It's a very good feature, when you >> have two datacenters, and repliacation goes via hight speed link, but >> aplications at top of clusters does not need to handle this tasks, and >> data are consistent. > > This is a more generic ceph issue. We do need, however, to be able to > support multiple clusters in rgw. Opened issue #2460. For rbd is nice feature too :) > >> >> - gracefull upgrade > > Can you elaborate on that? This is always our intention, we've put > some mechanisms in place to help with that, although it is not always > possible. How to upgrade cluster without any down ?? How to upgrade cluster with new features that change logic with 100% uptime ?? Yes i understand that sometimes, big changes need restart, a whole cluster at once, but i
Re: RGW, future directions
2012/5/23 Yehuda Sadeh : > RGW is maturing. Beside looking at performance, which highly ties into > RADOS performance, we'd like to hear whether there are certain pain > points or future directions that you (you as in the ceph community) > would like to see us taking. > > There are a few directions that we were thinking about: > > 1. Extend Object Storage API > > Swift and S3 has some features that we don't currently support. We can > certainly extend our functionality, however, is there any demand for > more features? E.g., self destructing objects, web site, user logs, > etc. > I would like to see radosgw supporting: - Object versioning - Bucket policy > 2. Better OpenStack interoperability > > Keystone support? Other? > > 3. New features > > Some examples: > > - multitenancy: api for domains and user management > - snapshots > - computation front end: upload object, then do some data > transformation/calculation. > - simple key-value api > > 4. CDMI > > Sage brought up the CDMI support question to ceph-devel, and I don't > remember him getting any response. Is there any intereset in CDMI? > > > 5. Native apache/nginx module or embedded web server > > We still need to prove that the web server is a bottleneck, or poses > scaling issues. Writing a correct native nginx module will require > turning rgw process model into event driven, which is not going to be > easy. > > 6. Improve garbage collection > > Currently rgw generates intent logs for garabage removal that require > running an external tool later, which is an administrative pain. We > can implement other solutions (OSD side garbage collection, > integrating cleanup process into the gateway, etc.) but we need to > understand the priority. > > 7. libradosgw > > We have had this in mind for some time now. Creating a programming api > for rgw, not too different from librados and librbd. It'll hopefully > make code much cleaner. It will allow users to write different front > ends for the rgw backend, and it will make it easier for users to > write applications that interact with the backend, e.g., do processing > on objects that users uploaded, FUSE for rgw without S3 as an > intermediate, etc. > I am also interested in libradosgw. With it, we can support other protocols easily. (e.g., webdav) > 8. Administration tools improvement > > We can always do better there. > > 9. Other ideas? > > > Any comments are welcome! > > Thanks, > Yehuda > -- > To unsubscribe from this list: send the line "unsubscribe ceph-devel" in > the body of a message to majord...@vger.kernel.org > More majordomo info at http://vger.kernel.org/majordomo-info.html -- To unsubscribe from this list: send the line "unsubscribe ceph-devel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: RGW, future directions
Hello, How about this CloudFS management system ? The CloudFS management system consists of two parts: a very simple web-based management daemon called cloudfsd, and scripts to perform various discrete functions. http://git.fedorahosted.org/git/?p=CloudFS.git;a=summary http://git.fedorahosted.org/git/?p=CloudFS.git;a=blob;f=scripts/README.ssl;h=3547f6b354d8b3455359ea019f6db9ce74862d2e;hb=cloudfsd I am not sure, whether it can be reused for ceph. Thanks, Kiran. On Tue, May 22, 2012 at 11:37 PM, Yehuda Sadeh wrote: > > RGW is maturing. Beside looking at performance, which highly ties into > RADOS performance, we'd like to hear whether there are certain pain > points or future directions that you (you as in the ceph community) > would like to see us taking. > > There are a few directions that we were thinking about: > > 1. Extend Object Storage API > > Swift and S3 has some features that we don't currently support. We can > certainly extend our functionality, however, is there any demand for > more features? E.g., self destructing objects, web site, user logs, > etc. > > 2. Better OpenStack interoperability > > Keystone support? Other? > > 3. New features > > Some examples: > > - multitenancy: api for domains and user management > - snapshots > - computation front end: upload object, then do some data > transformation/calculation. > - simple key-value api > > 4. CDMI > > Sage brought up the CDMI support question to ceph-devel, and I don't > remember him getting any response. Is there any intereset in CDMI? > > > 5. Native apache/nginx module or embedded web server > > We still need to prove that the web server is a bottleneck, or poses > scaling issues. Writing a correct native nginx module will require > turning rgw process model into event driven, which is not going to be > easy. > > 6. Improve garbage collection > > Currently rgw generates intent logs for garabage removal that require > running an external tool later, which is an administrative pain. We > can implement other solutions (OSD side garbage collection, > integrating cleanup process into the gateway, etc.) but we need to > understand the priority. > > 7. libradosgw > > We have had this in mind for some time now. Creating a programming api > for rgw, not too different from librados and librbd. It'll hopefully > make code much cleaner. It will allow users to write different front > ends for the rgw backend, and it will make it easier for users to > write applications that interact with the backend, e.g., do processing > on objects that users uploaded, FUSE for rgw without S3 as an > intermediate, etc. > > 8. Administration tools improvement > > We can always do better there. > > 9. Other ideas? > > > Any comments are welcome! > > Thanks, > Yehuda > -- > To unsubscribe from this list: send the line "unsubscribe ceph-devel" in > the body of a message to majord...@vger.kernel.org > More majordomo info at http://vger.kernel.org/majordomo-info.html -- To unsubscribe from this list: send the line "unsubscribe ceph-devel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: RGW, future directions
Sorry I missed to send this link. http://git.fedorahosted.org/git/?p=CloudFS.git;a=blob;f=doc/mgmt_manual.md;h=bfcbbe9769f8726ecd1aefcf19e1159074971110;hb=HEAD On Wed, May 23, 2012 at 3:47 PM, Kiran Patil wrote: > Hello, > > How about this CloudFS management system ? > > The CloudFS management system consists of two parts: a very simple > web-based management daemon called cloudfsd, and scripts to perform > various discrete functions. > > http://git.fedorahosted.org/git/?p=CloudFS.git;a=summary > > http://git.fedorahosted.org/git/?p=CloudFS.git;a=blob;f=scripts/README.ssl;h=3547f6b354d8b3455359ea019f6db9ce74862d2e;hb=cloudfsd > > I am not sure, whether it can be reused for ceph. > > Thanks, > Kiran. > > On Tue, May 22, 2012 at 11:37 PM, Yehuda Sadeh wrote: >> >> RGW is maturing. Beside looking at performance, which highly ties into >> RADOS performance, we'd like to hear whether there are certain pain >> points or future directions that you (you as in the ceph community) >> would like to see us taking. >> >> There are a few directions that we were thinking about: >> >> 1. Extend Object Storage API >> >> Swift and S3 has some features that we don't currently support. We can >> certainly extend our functionality, however, is there any demand for >> more features? E.g., self destructing objects, web site, user logs, >> etc. >> >> 2. Better OpenStack interoperability >> >> Keystone support? Other? >> >> 3. New features >> >> Some examples: >> >> - multitenancy: api for domains and user management >> - snapshots >> - computation front end: upload object, then do some data >> transformation/calculation. >> - simple key-value api >> >> 4. CDMI >> >> Sage brought up the CDMI support question to ceph-devel, and I don't >> remember him getting any response. Is there any intereset in CDMI? >> >> >> 5. Native apache/nginx module or embedded web server >> >> We still need to prove that the web server is a bottleneck, or poses >> scaling issues. Writing a correct native nginx module will require >> turning rgw process model into event driven, which is not going to be >> easy. >> >> 6. Improve garbage collection >> >> Currently rgw generates intent logs for garabage removal that require >> running an external tool later, which is an administrative pain. We >> can implement other solutions (OSD side garbage collection, >> integrating cleanup process into the gateway, etc.) but we need to >> understand the priority. >> >> 7. libradosgw >> >> We have had this in mind for some time now. Creating a programming api >> for rgw, not too different from librados and librbd. It'll hopefully >> make code much cleaner. It will allow users to write different front >> ends for the rgw backend, and it will make it easier for users to >> write applications that interact with the backend, e.g., do processing >> on objects that users uploaded, FUSE for rgw without S3 as an >> intermediate, etc. >> >> 8. Administration tools improvement >> >> We can always do better there. >> >> 9. Other ideas? >> >> >> Any comments are welcome! >> >> Thanks, >> Yehuda >> -- >> To unsubscribe from this list: send the line "unsubscribe ceph-devel" in >> the body of a message to majord...@vger.kernel.org >> More majordomo info at http://vger.kernel.org/majordomo-info.html -- To unsubscribe from this list: send the line "unsubscribe ceph-devel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: RGW, future directions
> >I would like to see radosgw supporting: >- Object versioning >- Bucket policy > +1 for these. >> >> 7. libradosgw >> Yes, please :) Cheers, Xiaopong -- To unsubscribe from this list: send the line "unsubscribe ceph-devel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: RGW, future directions
> >I would like to see radosgw supporting: >- Object versioning >- Bucket policy > +1 for these. >> >> 7. libradosgw >> Yes, please :) Cheers, Xiaopong -- To unsubscribe from this list: send the line "unsubscribe ceph-devel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: RGW, future directions
Hi, 2012/5/22 Yehuda Sadeh : > RGW is maturing. Beside looking at performance, which highly ties into > RADOS performance, we'd like to hear whether there are certain pain > points or future directions that you (you as in the ceph community) > would like to see us taking. > > There are a few directions that we were thinking about: > 3. New features > > Some examples: > > - multitenancy: api for domains and user management > - snapshots > - computation front end: upload object, then do some data > transformation/calculation. > - simple key-value api +1 any new feature can be a new solution for futur needs. > 5. Native apache/nginx module or embedded web server > > We still need to prove that the web server is a bottleneck, or poses > scaling issues. Writing a correct native nginx module will require > turning rgw process model into event driven, which is not going to be > easy. It is, for me the #1 feature : - I only use Nginx for everythings... instead radosgw - another process / another deamon / another problem - resolve once for all the "buffering" problem : only apache/fastcgi can send without buffering. And when you send BIG files, it's not an option. > 8. Administration tools improvement > > We can always do better there. Agree, bucket / pool management is quite an ordeal. It come (of course) with a better documentation. > Thanks, > Yehuda Thanks, Guilhem Lettron -- To unsubscribe from this list: send the line "unsubscribe ceph-devel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: RGW, future directions
On Wed, May 23, 2012 at 1:59 AM, Henry C Chang wrote: > 2012/5/23 Yehuda Sadeh : >> RGW is maturing. Beside looking at performance, which highly ties into >> RADOS performance, we'd like to hear whether there are certain pain >> points or future directions that you (you as in the ceph community) >> would like to see us taking. >> >> There are a few directions that we were thinking about: >> >> 1. Extend Object Storage API >> >> Swift and S3 has some features that we don't currently support. We can >> certainly extend our functionality, however, is there any demand for >> more features? E.g., self destructing objects, web site, user logs, >> etc. >> > > I would like to see radosgw supporting: > - Object versioning Yeah, we had that in mind for while now. Ceph issue #418. We have made some progress towards that, but we're not there yet. > - Bucket policy Anything specific that you'd like to see there? Bucket policy is very tied into amazon aws, and I don't think we should clone that. We can, however, look at what additional useful functionality we can have. >> 7. libradosgw >> >> We have had this in mind for some time now. Creating a programming api >> for rgw, not too different from librados and librbd. It'll hopefully >> make code much cleaner. It will allow users to write different front >> ends for the rgw backend, and it will make it easier for users to >> write applications that interact with the backend, e.g., do processing >> on objects that users uploaded, FUSE for rgw without S3 as an >> intermediate, etc. >> > > I am also interested in libradosgw. With it, we can support other > protocols easily. (e.g., webdav) > Thanks, Yehuda -- To unsubscribe from this list: send the line "unsubscribe ceph-devel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: RGW, future directions
On Wed, May 23, 2012 at 3:17 AM, Kiran Patil wrote: > Hello, > > How about this CloudFS management system ? > > The CloudFS management system consists of two parts: a very simple > web-based management daemon called cloudfsd, and scripts to perform > various discrete functions. > > http://git.fedorahosted.org/git/?p=CloudFS.git;a=summary > > http://git.fedorahosted.org/git/?p=CloudFS.git;a=blob;f=scripts/README.ssl;h=3547f6b354d8b3455359ea019f6db9ce74862d2e;hb=cloudfsd > > I am not sure, whether it can be reused for ceph. > Not sure either. However, the idea of having management API may ease any such integration. Thanks, Yehuda -- To unsubscribe from this list: send the line "unsubscribe ceph-devel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: RGW, future directions
On 22-05-12 20:07, Yehuda Sadeh wrote: RGW is maturing. Beside looking at performance, which highly ties into RADOS performance, we'd like to hear whether there are certain pain points or future directions that you (you as in the ceph community) would like to see us taking. There are a few directions that we were thinking about: 1. Extend Object Storage API Swift and S3 has some features that we don't currently support. We can certainly extend our functionality, however, is there any demand for more features? E.g., self destructing objects, web site, user logs, etc. 2. Better OpenStack interoperability Keystone support? Other? 3. New features Some examples: - multitenancy: api for domains and user management - snapshots - computation front end: upload object, then do some data transformation/calculation. - simple key-value api 4. CDMI Sage brought up the CDMI support question to ceph-devel, and I don't remember him getting any response. Is there any intereset in CDMI? 5. Native apache/nginx module or embedded web server We still need to prove that the web server is a bottleneck, or poses scaling issues. Writing a correct native nginx module will require turning rgw process model into event driven, which is not going to be easy. I'd not go for a native nginx or Apache module, that would bring extra C code into the story which would mean extra dependencies. My vote would still go to a embedded webserver written in something like Python. You could then use Apache/nginx/Varnish as a reverse proxy in front and do all kinds of cool stuff. You could even doing caching in nginx or Varnish and let the RGW notify those proxy's when an object has changed so they can purge their cache. This would dramatically improve the performance of the gateway. It would also simplify the code, why try to do caching on your own when some great HTTP caches are out there? 6. Improve garbage collection Currently rgw generates intent logs for garabage removal that require running an external tool later, which is an administrative pain. We can implement other solutions (OSD side garbage collection, integrating cleanup process into the gateway, etc.) but we need to understand the priority. 7. libradosgw We have had this in mind for some time now. Creating a programming api for rgw, not too different from librados and librbd. It'll hopefully make code much cleaner. It will allow users to write different front ends for the rgw backend, and it will make it easier for users to write applications that interact with the backend, e.g., do processing on objects that users uploaded, FUSE for rgw without S3 as an intermediate, etc. Yes, I would really like this. Combine this with the Python stand-alone/embedded webserver I proposed and you get a really nice RGW I think. 8. Administration tools improvement We can always do better there. When we have libradosgw it wouldn't be that hard to make a nice web front-end where you can manage the whole thing. 9. Other ideas? Any comments are welcome! Thanks, Yehuda -- To unsubscribe from this list: send the line "unsubscribe ceph-devel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html -- To unsubscribe from this list: send the line "unsubscribe ceph-devel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: RGW, future directions
2012/5/24 Yehuda Sadeh : > On Wed, May 23, 2012 at 1:59 AM, Henry C Chang > wrote: >> 2012/5/23 Yehuda Sadeh : >>> RGW is maturing. Beside looking at performance, which highly ties into >>> RADOS performance, we'd like to hear whether there are certain pain >>> points or future directions that you (you as in the ceph community) >>> would like to see us taking. >>> >>> There are a few directions that we were thinking about: >>> >>> 1. Extend Object Storage API >>> >>> Swift and S3 has some features that we don't currently support. We can >>> certainly extend our functionality, however, is there any demand for >>> more features? E.g., self destructing objects, web site, user logs, >>> etc. >>> >> >> I would like to see radosgw supporting: >> - Object versioning > > Yeah, we had that in mind for while now. Ceph issue #418. We have made > some progress towards that, but we're not there yet. > >> - Bucket policy > > Anything specific that you'd like to see there? Bucket policy is very > tied into amazon aws, and I don't think we should clone that. We can, > however, look at what additional useful functionality we can have. > To me, one of the most powerful thing of bucket policy is: it allows you to set permissions for a large number of objects with one statement. Below is a simple example. You can specify the "Resource" with a wildcard (or even /*). { "Version":"2008-10-17", "Statement":[{ "Sid":"AddPerm", "Effect":"Allow", "Principal": { "AWS": "*" }, "Action":["s3:GetObject"], "Resource":["arn:aws:s3:::bucket/*" ] } ] } I agree, we don't have to clone that. Similar functionality would be great!! >>> 7. libradosgw >>> >>> We have had this in mind for some time now. Creating a programming api >>> for rgw, not too different from librados and librbd. It'll hopefully >>> make code much cleaner. It will allow users to write different front >>> ends for the rgw backend, and it will make it easier for users to >>> write applications that interact with the backend, e.g., do processing >>> on objects that users uploaded, FUSE for rgw without S3 as an >>> intermediate, etc. >>> >> >> I am also interested in libradosgw. With it, we can support other >> protocols easily. (e.g., webdav) >> > > > Thanks, > Yehuda -- To unsubscribe from this list: send the line "unsubscribe ceph-devel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: RGW, future directions
On Thu, May 24, 2012 at 7:15 AM, Wido den Hollander wrote: > > > On 22-05-12 20:07, Yehuda Sadeh wrote: >> >> RGW is maturing. Beside looking at performance, which highly ties into >> RADOS performance, we'd like to hear whether there are certain pain >> points or future directions that you (you as in the ceph community) >> would like to see us taking. >> >> There are a few directions that we were thinking about: >> >> 1. Extend Object Storage API >> >> Swift and S3 has some features that we don't currently support. We can >> certainly extend our functionality, however, is there any demand for >> more features? E.g., self destructing objects, web site, user logs, >> etc. >> >> 2. Better OpenStack interoperability >> >> Keystone support? Other? >> >> 3. New features >> >> Some examples: >> >> - multitenancy: api for domains and user management >> - snapshots >> - computation front end: upload object, then do some data >> transformation/calculation. >> - simple key-value api >> >> 4. CDMI >> >> Sage brought up the CDMI support question to ceph-devel, and I don't >> remember him getting any response. Is there any intereset in CDMI? >> >> >> 5. Native apache/nginx module or embedded web server >> >> We still need to prove that the web server is a bottleneck, or poses >> scaling issues. Writing a correct native nginx module will require >> turning rgw process model into event driven, which is not going to be >> easy. >> > > I'd not go for a native nginx or Apache module, that would bring extra C > code into the story which would mean extra dependencies. > > My vote would still go to a embedded webserver written in something like > Python. You could then use Apache/nginx/Varnish as a reverse proxy in front > and do all kinds of cool stuff. > > You could even doing caching in nginx or Varnish and let the RGW notify > those proxy's when an object has changed so they can purge their cache. This > would dramatically improve the performance of the gateway. > > It would also simplify the code, why try to do caching on your own when some > great HTTP caches are out there? > 100% +1 > >> 6. Improve garbage collection >> >> Currently rgw generates intent logs for garabage removal that require >> running an external tool later, which is an administrative pain. We >> can implement other solutions (OSD side garbage collection, >> integrating cleanup process into the gateway, etc.) but we need to >> understand the priority. >> >> 7. libradosgw >> >> We have had this in mind for some time now. Creating a programming api >> for rgw, not too different from librados and librbd. It'll hopefully >> make code much cleaner. It will allow users to write different front >> ends for the rgw backend, and it will make it easier for users to >> write applications that interact with the backend, e.g., do processing >> on objects that users uploaded, FUSE for rgw without S3 as an >> intermediate, etc. >> > > Yes, I would really like this. Combine this with the Python > stand-alone/embedded webserver I proposed and you get a really nice RGW I > think. > > >> 8. Administration tools improvement >> >> We can always do better there. >> > > When we have libradosgw it wouldn't be that hard to make a nice web > front-end where you can manage the whole thing. > > >> 9. Other ideas? >> >> >> Any comments are welcome! >> >> Thanks, >> Yehuda >> -- >> To unsubscribe from this list: send the line "unsubscribe ceph-devel" in >> the body of a message to majord...@vger.kernel.org >> More majordomo info at http://vger.kernel.org/majordomo-info.html > > -- > To unsubscribe from this list: send the line "unsubscribe ceph-devel" in > the body of a message to majord...@vger.kernel.org > More majordomo info at http://vger.kernel.org/majordomo-info.html -- - Pozdrawiam Sławek "sZiBis" Skowron -- To unsubscribe from this list: send the line "unsubscribe ceph-devel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html