Re: [ceph-users] Limit bandwidth on RadosGW?
Yes, but I have not yet found an open source reverse proxy can achieve it. haproxy blocks requests instead of limit bandwidth in a fixed Mbps; nginx can only limit the download speed (by the option proxy_limit_rate), and has a negative side effect that it buffers the response body, it cause huge performance downgrade. On Fri, May 5, 2017 at 6:04 AM, Robin H. Johnson wrote: > On Thu, May 04, 2017 at 04:35:21PM +0800, hrchu wrote: > > Thanks for reply. > > > > tc can only do limit on interfaces or given IPs, but what I am talking > > about is "per connection", e.g., each put object could be 5MB/s, get > > object could be 1MB/s. > To achieve your required level of control, you need haproxy, or other > HTTP-aware reverse proxy, as to have a different limit based on the > operation (and possibly the access key). > > -- > Robin Hugh Johnson > Gentoo Linux: Dev, Infra Lead, Foundation Trustee & Treasurer > E-Mail : robb...@gentoo.org > GnuPG FP : 11ACBA4F 4778E3F6 E4EDF38E B27B944E 34884E85 > GnuPG FP : 7D0B3CEB E9B85B1F 825BCECF EE05E6F6 A48F6136 > ___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
Re: [ceph-users] Limit bandwidth on RadosGW?
According to the link you provided, haproxy seems can only block requests instead of limit bandwidth. If people violate the rate limiting in this example they are redirected to the backend ease-up-y0 which gives them 503 error page that can be customized. On Fri, May 5, 2017 at 7:38 AM, George Mihaiescu wrote: > Terminate the connections on haproxy which is great for ssl as well, and > use these instructions to set qos per connection and data transferred: > http://blog.serverfault.com/2010/08/26/1016491873/ > > > > > On May 4, 2017, at 04:35, hrchu wrote: > > Thanks for reply. > > tc can only do limit on interfaces or given IPs, but what I am talking > about is "per connection", e.g., each put object could be 5MB/s, get > object could be 1MB/s. > > Correct me if anything wrong. > > > Regards, > > Chu, Hua-Rong (曲華榮), +886-3-4227151 #57968 <+886%203%20422%207151> > Networklab, Computer Science & Information Engineering, > National Central University, Jhongli, Taiwan R.O.C. > > On Thu, May 4, 2017 at 4:01 PM, Marc Roos > wrote: > >> >> >> >> No experience with it. But why not use linux for it? Maybe this solution >> on every RGW is sufficient, I cannot imagine you need 3rd party for >> this. >> >> https://unix.stackexchange.com/questions/28198/how-to-limit- >> network-bandwidth >> https://wiki.archlinux.org/index.php/Advanced_traffic_control >> >> >> >> -Original Message- >> From: hrchu [mailto:petertc@gmail.com] >> Sent: donderdag 4 mei 2017 9:24 >> To: Ceph Users >> Subject: [ceph-users] Limit bandwidth on RadosGW? >> >> Hi all, >> I want to limit RadosGW per connection upload/download speed for QoS. >> There is no build-in option for this, so maybe a 3rd party reverse proxy >> in front of Radosgw is needed. Does anyone have experience about this? >> >> >> > ___ > ceph-users mailing list > ceph-users@lists.ceph.com > http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com > > ___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
Re: [ceph-users] Limit bandwidth on RadosGW?
Terminate the connections on haproxy which is great for ssl as well, and use these instructions to set qos per connection and data transferred: http://blog.serverfault.com/2010/08/26/1016491873/ > On May 4, 2017, at 04:35, hrchu wrote: > > Thanks for reply. > > tc can only do limit on interfaces or given IPs, but what I am talking about > is "per connection", e.g., each put object could be 5MB/s, get object could > be 1MB/s. > > Correct me if anything wrong. > > > Regards, > > Chu, Hua-Rong (曲華榮), +886-3-4227151 #57968 > Networklab, Computer Science & Information Engineering, > National Central University, Jhongli, Taiwan R.O.C. > >> On Thu, May 4, 2017 at 4:01 PM, Marc Roos wrote: >> >> >> >> No experience with it. But why not use linux for it? Maybe this solution >> on every RGW is sufficient, I cannot imagine you need 3rd party for >> this. >> >> https://unix.stackexchange.com/questions/28198/how-to-limit-network-bandwidth >> https://wiki.archlinux.org/index.php/Advanced_traffic_control >> >> >> >> -----Original Message- >> From: hrchu [mailto:petertc@gmail.com] >> Sent: donderdag 4 mei 2017 9:24 >> To: Ceph Users >> Subject: [ceph-users] Limit bandwidth on RadosGW? >> >> Hi all, >> I want to limit RadosGW per connection upload/download speed for QoS. >> There is no build-in option for this, so maybe a 3rd party reverse proxy >> in front of Radosgw is needed. Does anyone have experience about this? >> >> > > ___ > ceph-users mailing list > ceph-users@lists.ceph.com > http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com ___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
Re: [ceph-users] Limit bandwidth on RadosGW?
On Thu, May 04, 2017 at 04:35:21PM +0800, hrchu wrote: > Thanks for reply. > > tc can only do limit on interfaces or given IPs, but what I am talking > about is "per connection", e.g., each put object could be 5MB/s, get > object could be 1MB/s. To achieve your required level of control, you need haproxy, or other HTTP-aware reverse proxy, as to have a different limit based on the operation (and possibly the access key). -- Robin Hugh Johnson Gentoo Linux: Dev, Infra Lead, Foundation Trustee & Treasurer E-Mail : robb...@gentoo.org GnuPG FP : 11ACBA4F 4778E3F6 E4EDF38E B27B944E 34884E85 GnuPG FP : 7D0B3CEB E9B85B1F 825BCECF EE05E6F6 A48F6136 ___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
Re: [ceph-users] Limit bandwidth on RadosGW?
Mandi! Marc Roos In chel di` si favelave... > Just a thought, what about marking connections with iptables and using > that mark with tc? Surely, but many things have to be taken into account: a) doing traffic control mean disabling ALL network hardware optimizations (queue, offline checksumming, ...), and i don't know the impact on ceph. b) doing simple control (eg, traffic clamping on a interface) could add little overhead, but if complex setup are needed (multiqueue, traffic shaping by IP/port/...) i think that more overhead get added. c) again, doing simple control can be done on ingress easily, but if complex setup ar needed the ingress traffic must be routed to another interface (mostly, IFB interfaces) and proper egress shaping get done here. In ifbX interfaces, also, there's no netfilter. I'm using that stuffs on firewall, where performance on modern/decent hardware is not a trouble at al. So, no, i've no benchmark at all. ;) -- dott. Marco Gaiarin GNUPG Key ID: 240A3D66 Associazione ``La Nostra Famiglia'' http://www.lanostrafamiglia.it/ Polo FVG - Via della Bontà, 7 - 33078 - San Vito al Tagliamento (PN) marco.gaiarin(at)lanostrafamiglia.it t +39-0434-842711 f +39-0434-842797 Dona il 5 PER MILLE a LA NOSTRA FAMIGLIA! http://www.lanostrafamiglia.it/index.php/it/sostienici/5x1000 (cf 00307430132, categoria ONLUS oppure RICERCA SANITARIA) ___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
Re: [ceph-users] Limit bandwidth on RadosGW?
Just a thought, what about marking connections with iptables and using
that mark with tc?
-Original Message-
From: hrchu [mailto:petertc@gmail.com]
Sent: donderdag 4 mei 2017 10:35
To: Marc Roos; ceph-users
Subject: Re: [ceph-users] Limit bandwidth on RadosGW?
Thanks for reply.
tc can only do limit on interfaces or given IPs, but what I am talking
about is "per connection", e.g., each put object could be 5MB/s, get
object could be 1MB/s.
Correct me if anything wrong.
Regards,
Chu, Hua-Rong (曲華榮), +886-3-4227151 #57968 Networklab, Computer
Science & Information Engineering, National Central University, Jhongli,
Taiwan R.O.C.
On Thu, May 4, 2017 at 4:01 PM, Marc Roos
wrote:
No experience with it. But why not use linux for it? Maybe this
solution
on every RGW is sufficient, I cannot imagine you need 3rd party for
this.
https://unix.stackexchange.com/questions/28198/how-to-limit-network
-bandwidth
<https://unix.stackexchange.com/questions/28198/how-to-limit-network-bandwidth>
https://wiki.archlinux.org/index.php/Advanced_traffic_control
<https://wiki.archlinux.org/index.php/Advanced_traffic_control>
-Original Message-
From: hrchu [mailto:petertc@gmail.com]
Sent: donderdag 4 mei 2017 9:24
To: Ceph Users
Subject: [ceph-users] Limit bandwidth on RadosGW?
Hi all,
I want to limit RadosGW per connection upload/download speed for
QoS.
There is no build-in option for this, so maybe a 3rd party reverse
proxy
in front of Radosgw is needed. Does anyone have experience about
this?
___
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
Re: [ceph-users] Limit bandwidth on RadosGW?
Thanks for reply. tc can only do limit on interfaces or given IPs, but what I am talking about is "per connection", e.g., each put object could be 5MB/s, get object could be 1MB/s. Correct me if anything wrong. Regards, Chu, Hua-Rong (曲華榮), +886-3-4227151 #57968 Networklab, Computer Science & Information Engineering, National Central University, Jhongli, Taiwan R.O.C. On Thu, May 4, 2017 at 4:01 PM, Marc Roos wrote: > > > > No experience with it. But why not use linux for it? Maybe this solution > on every RGW is sufficient, I cannot imagine you need 3rd party for > this. > > https://unix.stackexchange.com/questions/28198/how-to- > limit-network-bandwidth > https://wiki.archlinux.org/index.php/Advanced_traffic_control > > > > -Original Message- > From: hrchu [mailto:petertc@gmail.com] > Sent: donderdag 4 mei 2017 9:24 > To: Ceph Users > Subject: [ceph-users] Limit bandwidth on RadosGW? > > Hi all, > I want to limit RadosGW per connection upload/download speed for QoS. > There is no build-in option for this, so maybe a 3rd party reverse proxy > in front of Radosgw is needed. Does anyone have experience about this? > > > ___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
Re: [ceph-users] Limit bandwidth on RadosGW?
No experience with it. But why not use linux for it? Maybe this solution on every RGW is sufficient, I cannot imagine you need 3rd party for this. https://unix.stackexchange.com/questions/28198/how-to-limit-network-bandwidth https://wiki.archlinux.org/index.php/Advanced_traffic_control -Original Message- From: hrchu [mailto:petertc@gmail.com] Sent: donderdag 4 mei 2017 9:24 To: Ceph Users Subject: [ceph-users] Limit bandwidth on RadosGW? Hi all, I want to limit RadosGW per connection upload/download speed for QoS. There is no build-in option for this, so maybe a 3rd party reverse proxy in front of Radosgw is needed. Does anyone have experience about this? ___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
[ceph-users] Limit bandwidth on RadosGW?
Hi all, I want to limit RadosGW per connection upload/download speed for QoS. There is no build-in option for this, so maybe a 3rd party reverse proxy in front of Radosgw is needed. Does anyone have experience about this? ___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
