Re: [ceph-users] RGW/Civet: Reads too much data when client doesn't close the connection
Yup already working on fixing the client, but it seems like a potentially nasty issue for RGW, as a malicious client could potentially DOS an endpoint pretty easily this way. Aaron > On Jul 12, 2017, at 11:48 AM, Jens Rosenboom wrote: > > 2017-07-12 15:23 GMT+00:00 Aaron Bassett : >> I have a situation where a client is GET'ing a large key (100GB) from >> RadosGW and just reading the first few bytes to determine if it's a gzip >> file or not, and then just moving on without closing the connection. I'm >> RadosGW then goes on to read the rest of the object out of the cluster, >> while sending nothing to the client as it's no longer listening. When this >> client does this to many objects in quick succession, it essentially creates >> a DOS on my cluster as all my rgws are reading out of the cluster as fast as >> they can but not sending the data anywhere. This is on an up to date Jewel >> cluster, using civetweb for the web server. >> >> I just wanted to reach out and see if anyone else has seen this before I dig >> in more and try to find more details about where the problem may lay. > > I would say your client is broken, if it is only interested in a range > of the object, it should include a corresponding range header with the > GET request. > > Though I agree that the behaviour for closed connections could > probably improved, too. See > https://urldefense.proofpoint.com/v2/url?u=http-3A__tracker.ceph.com_issues_20166&d=DwIFaQ&c=Tpa2GKmmYSmpYS4baANxQwQYqA0vwGXwkJOPBegaiTs&r=5nKer5huNDFQXjYpOR4o_7t5CRI8wb5Vb_v1pBywbYw&m=6pdFEFo2m68_ouTlVrEa4GOrzh-WcOpK4K8hRD2n2ho&s=wtiIaAqUaoNJeBMwjyIDRQXs-So9Hj6xELikPSSRuV0&e= > for a > similar issue, something like the opposite of your case. CONFIDENTIALITY NOTICE This e-mail message and any attachments are only for the use of the intended recipient and may contain information that is privileged, confidential or exempt from disclosure under applicable law. If you are not the intended recipient, any disclosure, distribution or other use of this e-mail message or attachments is prohibited. If you have received this e-mail message in error, please delete and notify the sender immediately. Thank you. ___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
Re: [ceph-users] RGW/Civet: Reads too much data when client doesn't close the connection
2017-07-12 15:23 GMT+00:00 Aaron Bassett : > I have a situation where a client is GET'ing a large key (100GB) from RadosGW > and just reading the first few bytes to determine if it's a gzip file or not, > and then just moving on without closing the connection. I'm RadosGW then goes > on to read the rest of the object out of the cluster, while sending nothing > to the client as it's no longer listening. When this client does this to many > objects in quick succession, it essentially creates a DOS on my cluster as > all my rgws are reading out of the cluster as fast as they can but not > sending the data anywhere. This is on an up to date Jewel cluster, using > civetweb for the web server. > > I just wanted to reach out and see if anyone else has seen this before I dig > in more and try to find more details about where the problem may lay. I would say your client is broken, if it is only interested in a range of the object, it should include a corresponding range header with the GET request. Though I agree that the behaviour for closed connections could probably improved, too. See http://tracker.ceph.com/issues/20166 for a similar issue, something like the opposite of your case. ___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
[ceph-users] RGW/Civet: Reads too much data when client doesn't close the connection
I have a situation where a client is GET'ing a large key (100GB) from RadosGW and just reading the first few bytes to determine if it's a gzip file or not, and then just moving on without closing the connection. I'm RadosGW then goes on to read the rest of the object out of the cluster, while sending nothing to the client as it's no longer listening. When this client does this to many objects in quick succession, it essentially creates a DOS on my cluster as all my rgws are reading out of the cluster as fast as they can but not sending the data anywhere. This is on an up to date Jewel cluster, using civetweb for the web server. I just wanted to reach out and see if anyone else has seen this before I dig in more and try to find more details about where the problem may lay. Aaron CONFIDENTIALITY NOTICE This e-mail message and any attachments are only for the use of the intended recipient and may contain information that is privileged, confidential or exempt from disclosure under applicable law. If you are not the intended recipient, any disclosure, distribution or other use of this e-mail message or attachments is prohibited. If you have received this e-mail message in error, please delete and notify the sender immediately. Thank you. ___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
