Re: Multiprocess and backends
On Wed, Dec 17, 2014 at 10:39 PM, Pavlos Parissis wrote: > Hi, > > I remember someone( maybe Baptiste) saying that in multi process mode > backends will be picked up by the process which frontend is bound to. > But, I found not to be the case in 1.5.9. > I also remember that this works only when you have 1to1 relationship > between frontend and backends, which is my case. > > In the following output of stat sockets I see both backends to be > monitored by both processes. If I bind graphite_example.com_SSL backend > to the some process as the graphite_example.com_SSL frontend, it works > as expected where graphite_example.com_SSL is monitored only by process 2. > > It isn't a problem to use bind-process in backend settings and I am just > asking out of curiosity. > > Cheers, > Pavlos > > > echo 'show stat'|nc -U /var/lib/haproxy/stats1 > haproxy,FRONTEND,,,0,0,5,0,0,0,0,0,0,OPEN,1,2,00,0,2000 > haproxy,BACKEND,0,0,0,0,5000,0,0,0,0,0,,0,0,0,0,UP,0,0,0,,0,19,0,,1,2,0,,0, > graphite_example.com,FRONTEND,,,0,1,5,6,1008,7290,0,0,0,OPEN,,, > graphite_example.com,server-101.example.com,0,0,0,1,,3,504,3645,,0,,0,0, > graphite_example.com,server-102.example.com,0,0,0,1,,3,504,3645,,0,,0, > graphite_example.com,BACKEND,0,0,0,1,5000,6,1008,7290,0,0,,0,0,0,0,UP,2,2,0 > graphite_example.com_SSL,server-103.example.com,0,0,0,0,,0,0,0,,0,,0,0,0 > graphite_example.com_SSL,server-104.example.com,0,0,0,0,,0,0,0,,0,,0,0, > graphite_example.com_SSL,BACKEND,0,0,0,0,5000,0,0,0,0,0,,0,0,0,0,UP,2,2, > > echo 'show stat'|nc -U /var/lib/haproxy/stats2 > haproxy,FRONTEND,,,0,1,5,1,122,1955,0,0,0,OPEN,2,2,00, > haproxy,BACKEND,0,0,0,0,5000,0,122,1955,0,0,,0,0,0,0,UP,0,0,0,,0,28,0,,2,2 > graphite_example.com,server-101.example.com,0,0,0,0,,0,0,0,,0,,0,0,0,0,U > graphite_example.com,server-102.example.com,0,0,0,0,,0,0,0,,0,,0,0,0,0,U > graphite_example.com,BACKEND,0,0,0,0,5000,0,0,0,0,0,,0,0,0,0,UP,2,2,0,,0,2 > graphite_example.com_SSL,FRONTEND,,,0,0,5,0,0,0,0,0,0,OPEN > graphite_example.com_SSL,server-103.example.com,0,0,0,0,,0,0,0,,0,,0,0,0 > graphite_example.com_SSL,server-104.example.com,0,0,0,0,,0,0,0,,0,,0,0,0 > graphite_example.com_SSL,BACKEND,0,0,0,0,5000,0,0,0,0,0,,0,0,0,0,UP,2,2,0, > > > global > log 127.0.0.1 local2 > chroot /var/lib/haproxy > pidfile /var/run/haproxy.pid > maxconn 10 > userhaproxy > group haproxy > daemon > > stats socket /var/lib/haproxy/stats uid 0 gid 0 mode 0440 level admin > > ssl-server-verify none > tune.ssl.default-dh-param 2048 > > stats socket /var/lib/haproxy/stats1 uid 0 gid 0 mode 0440 level > admin process 1 > stats socket /var/lib/haproxy/stats2 uid 0 gid 0 mode 0440 level > admin process 2 > nbproc 2 > cpu-map 1 0 > cpu-map 2 1 > > defaults > maxconn 5 > rate-limit sessions 2000 > modehttp > log global > option contstats > option tcplog > option httplog > no option dontlognull > option tcp-smart-accept > option tcp-smart-connect > option http-keep-alive > option redispatch > balance roundrobin > timeout http-request15s > timeout http-keep-alive 15s > retries 2 > timeout queue 1m > timeout connect 10s > timeout client 15s > timeout server 15s > timeout check 5s > option forwardfor header F5SourceIP > listen haproxy > bind :8080 > stats uri / > stats show-node > stats refresh 10s > stats show-legends > no log > > frontend graphite_example.com > bind 10.189.200.1:80 > bind-process 1 > default_backend graphite_example.com > > backend graphite_example.com > #bind-process 1 > default-server inter 10s > option httpchk GET / HTTP/1.1\r\nHost:\ > graphite.example.com\r\nUser-Agent:\ HAProxy > server server-101.example.com 10.96.70.65:80 check > server server-102.example.com 10.96.70.66:80 check > > > frontend graphite_example.com_SSL > bind 10.189.200.1:443 ssl crt /somepath/pem > bind-process 2 > default_backend graphite_example.com_SSL > > backend graphite_example.com_SSL > default-server inter 10s > #bind-process 2 > option httpchk GET / HTTP/1.1\r\nHost:\ > graphite.example.com\r\nUser-Agent:\ HAProxy > server server-103.example.com 10.96.70.109:443 ssl check check-ssl > server server-104.example.com 10.96.70.160:443 ssl check check-ssl > > Hi Pavlos, Your test is not relevant. Since you have no bind-process on your SSL backend, HAProxy starts it up on both proceses you started up. Please try adding a bind-process 1 in your SSL backend and report us the result. Baptiste
Re: [BUG Report] Maybe a bug for epoll
Hi Willy, On 2014/12/18 0:07, Willy Tarreau wrote: Hi Godbach, On Wed, Dec 17, 2014 at 04:41:33PM +0800, Godbach wrote: From 798d50b1922e9680ea69cbc39d54f20665c782da Mon Sep 17 00:00:00 2001 From: Godbach Date: Wed, 17 Dec 2014 16:14:26 +0800 Subject: [PATCH 1/2] OPTIM/MINOR: epoll: epoll_events should be allocated according to global.tune.maxpollevents (...) OK so I've applied this one and marked it CLEANUP. From 7bda83893cc0bede6e5eb0865e805003aed792f4 Mon Sep 17 00:00:00 2001 From: Godbach Date: Wed, 17 Dec 2014 16:32:05 +0800 Subject: [PATCH 2/2] DOC: fix typo: "401 Unauthorized" => "407 Unauthorized" And this one as well but retagged it BUG/MINOR, as it's not a DOC issue, it's clearly in the code and is reported that way, so it definitely impacts users who see this message if we return it. Thanks! Willy Get it. Thank you. -- Best Regards, Godbach
Multiprocess and backends
Hi, I remember someone( maybe Baptiste) saying that in multi process mode backends will be picked up by the process which frontend is bound to. But, I found not to be the case in 1.5.9. I also remember that this works only when you have 1to1 relationship between frontend and backends, which is my case. In the following output of stat sockets I see both backends to be monitored by both processes. If I bind graphite_example.com_SSL backend to the some process as the graphite_example.com_SSL frontend, it works as expected where graphite_example.com_SSL is monitored only by process 2. It isn't a problem to use bind-process in backend settings and I am just asking out of curiosity. Cheers, Pavlos echo 'show stat'|nc -U /var/lib/haproxy/stats1 haproxy,FRONTEND,,,0,0,5,0,0,0,0,0,0,OPEN,1,2,00,0,2000 haproxy,BACKEND,0,0,0,0,5000,0,0,0,0,0,,0,0,0,0,UP,0,0,0,,0,19,0,,1,2,0,,0, graphite_example.com,FRONTEND,,,0,1,5,6,1008,7290,0,0,0,OPEN,,, graphite_example.com,server-101.example.com,0,0,0,1,,3,504,3645,,0,,0,0, graphite_example.com,server-102.example.com,0,0,0,1,,3,504,3645,,0,,0, graphite_example.com,BACKEND,0,0,0,1,5000,6,1008,7290,0,0,,0,0,0,0,UP,2,2,0 graphite_example.com_SSL,server-103.example.com,0,0,0,0,,0,0,0,,0,,0,0,0 graphite_example.com_SSL,server-104.example.com,0,0,0,0,,0,0,0,,0,,0,0, graphite_example.com_SSL,BACKEND,0,0,0,0,5000,0,0,0,0,0,,0,0,0,0,UP,2,2, echo 'show stat'|nc -U /var/lib/haproxy/stats2 haproxy,FRONTEND,,,0,1,5,1,122,1955,0,0,0,OPEN,2,2,00, haproxy,BACKEND,0,0,0,0,5000,0,122,1955,0,0,,0,0,0,0,UP,0,0,0,,0,28,0,,2,2 graphite_example.com,server-101.example.com,0,0,0,0,,0,0,0,,0,,0,0,0,0,U graphite_example.com,server-102.example.com,0,0,0,0,,0,0,0,,0,,0,0,0,0,U graphite_example.com,BACKEND,0,0,0,0,5000,0,0,0,0,0,,0,0,0,0,UP,2,2,0,,0,2 graphite_example.com_SSL,FRONTEND,,,0,0,5,0,0,0,0,0,0,OPEN graphite_example.com_SSL,server-103.example.com,0,0,0,0,,0,0,0,,0,,0,0,0 graphite_example.com_SSL,server-104.example.com,0,0,0,0,,0,0,0,,0,,0,0,0 graphite_example.com_SSL,BACKEND,0,0,0,0,5000,0,0,0,0,0,,0,0,0,0,UP,2,2,0, global log 127.0.0.1 local2 chroot /var/lib/haproxy pidfile /var/run/haproxy.pid maxconn 10 userhaproxy group haproxy daemon stats socket /var/lib/haproxy/stats uid 0 gid 0 mode 0440 level admin ssl-server-verify none tune.ssl.default-dh-param 2048 stats socket /var/lib/haproxy/stats1 uid 0 gid 0 mode 0440 level admin process 1 stats socket /var/lib/haproxy/stats2 uid 0 gid 0 mode 0440 level admin process 2 nbproc 2 cpu-map 1 0 cpu-map 2 1 defaults maxconn 5 rate-limit sessions 2000 modehttp log global option contstats option tcplog option httplog no option dontlognull option tcp-smart-accept option tcp-smart-connect option http-keep-alive option redispatch balance roundrobin timeout http-request15s timeout http-keep-alive 15s retries 2 timeout queue 1m timeout connect 10s timeout client 15s timeout server 15s timeout check 5s option forwardfor header F5SourceIP listen haproxy bind :8080 stats uri / stats show-node stats refresh 10s stats show-legends no log frontend graphite_example.com bind 10.189.200.1:80 bind-process 1 default_backend graphite_example.com backend graphite_example.com #bind-process 1 default-server inter 10s option httpchk GET / HTTP/1.1\r\nHost:\ graphite.example.com\r\nUser-Agent:\ HAProxy server server-101.example.com 10.96.70.65:80 check server server-102.example.com 10.96.70.66:80 check frontend graphite_example.com_SSL bind 10.189.200.1:443 ssl crt /somepath/pem bind-process 2 default_backend graphite_example.com_SSL backend graphite_example.com_SSL default-server inter 10s #bind-process 2 option httpchk GET / HTTP/1.1\r\nHost:\ graphite.example.com\r\nUser-Agent:\ HAProxy server server-103.example.com 10.96.70.109:443 ssl check check-ssl server server-104.example.com 10.96.70.160:443 ssl check check-ssl signature.asc Description: OpenPGP digital signature
Re: [BUG Report] Maybe a bug for epoll
Hi Godbach, On Wed, Dec 17, 2014 at 04:41:33PM +0800, Godbach wrote: > From 798d50b1922e9680ea69cbc39d54f20665c782da Mon Sep 17 00:00:00 2001 > From: Godbach > Date: Wed, 17 Dec 2014 16:14:26 +0800 > Subject: [PATCH 1/2] OPTIM/MINOR: epoll: epoll_events should be allocated > according to global.tune.maxpollevents (...) OK so I've applied this one and marked it CLEANUP. > From 7bda83893cc0bede6e5eb0865e805003aed792f4 Mon Sep 17 00:00:00 2001 > From: Godbach > Date: Wed, 17 Dec 2014 16:32:05 +0800 > Subject: [PATCH 2/2] DOC: fix typo: "401 Unauthorized" => "407 Unauthorized" And this one as well but retagged it BUG/MINOR, as it's not a DOC issue, it's clearly in the code and is reported that way, so it definitely impacts users who see this message if we return it. Thanks! Willy
I am happy!
:-)
Tooling and Mould manufacturer/rapid prototyping/OEM/ODM/Solution provider/Volvo supplier
Hello friend, Interested in cutting cost up to 70%?We're a mould manufacturer in Shenzhen,China specializing in rapid prototyping,injection mould tooling,die casting,metal stamping,secondary processing,OEM and ODM service.Wide range of products: Automotive parts, Home Appliances, Healthcare and Medical equipment , industrial products, electronics and so on. 1.the tolerance of our technology is within +-0.01mm2.Through streamlining the production of molds,we can help you minimize your production cost and achieve profit maximization!3.After receiving your 3D drawing,we"ll give you a quot in 2 hours.Welcome to get a competitive quotation from us. Thank you in advance! Best regards,Roger HouManager -Sales Department Factory Address：Industrial Zone, Shangxing, Shajing Street,Bao'an District, Shenzhen ,Guangdong,China.Email:i...@sz-powstar.net；mailto:powsta...@gmail.com"; target=_blank>powsta...@gmail.com;Tel: 0086-755-36939432Fax: 0086-755-26439884Mob: 0086-13723716379Skype :powstar06QQ:493550547Wechat:suzhenlong3 Unsubscribe.asp?mail=haproxy@formilux.org
Bonus de fêtes chez ZEturf, 250 euros offerts pour votre meeting d'hiver
Title: Bonus 250 Si vous ne voyez pas correctement ce message, visualisez notre version en ligne. Pour être sûr de recevoir tous nos emails, ajoutez newslet...@email.zeturf.com à votre carnet d´adresses Pour ne plus recevoir de messages de notre part, rendez-vous sur cette page. Mot de passe oublié ? | Désinscription Newsletter | Jeu responsable | Contactez-nous *Offre réservée aux nouveaux clients ZEturf uniquement. Voir conditions sur le site. À tout moment, vous disposez d'un droit d'accès, de modification, de rectification et de suppression des données qui vous concernent. Jouer comporte des risques : endettement, isolement… Pour être aidé, appelez le 09-74-75-13-13 (appel non surtaxé). Vous devez avoir plus de 18 ans pour jouer sur ZEturf
maps and nbproc
Hi If you load a large map into memory, such as a ip-range —> country/region, and use nbproc 12. Does the map get loaded into shared memory, or is it allocated and read per process? Thanks Will
Re: [BUG Report] Maybe a bug for epoll
Hi Willy, On 2014/12/17 16:40, Willy Tarreau wrote: In my opinion it's a cleanup since we're removing a leftover from an ancient piece of code :-) Willy OK, I agree with you and please modify the patch directly. -- Best Regards, Godbach
BHM_Custom Mould
Dear Sir or Madam, Good Day! This is Sherry Zhang from Best Hope Mold & Plastic, an ISO certified injection mould maker from China. We have rich experience in injection mould design and tooling as well as custom injection moulding for various plastic parts We do well in: 1. tight tolerance --we design and tooling totally according to customer's drawing 2. smooth de-moulding and shortened cycle time 3. clean and smooth surface due to good polishing work for tool core & cavity 4. optimized parting lines, ejector marks and gate location 5. competitive cost and high-efficient communication and work Please send us RFQ for quoting of moulds if you have the need or interest. Thank you! Yours faithfully, Sherry Zhang = Best Hope Mold & Plastic Co., Ltd Add: No.219 Songhai Rd, Qingpu Industrial Park, 201703, Shanghai, China Tel: +86-(0)21-39290552 Skype: sherrybhm Mob: +86-18121109009 Fax: +86-(0)21-39290600 Email: sale...@bhmould.com Web: www.bh-mold.net
Re: [BUG Report] Maybe a bug for epoll
Hi Willy,
On 2014/12/17 16:03, Godbach wrote:
On 2014/12/17 15:15, Willy Tarreau wrote:
In my opinion, absmaxevents should be passed to epoll_wait()
instead of global.tune.maxpollevents.
No, that would defeat the principle of maxpollevents. In fact, I found
what happened : commit f2e8ee2b introduced an optimization in the old
speculative epoll code, which implemented its own event cache. It was
needed to store that many events (it was bound to maxsock/4 btw). Now
the event cache lives on its own and we don't need this anymore. And
since events are allocated on the kernel side, we only need to allocate
the events we want to return. Thus I'd completely remove absmaxevents,
the comment on top of it suggesting to read the removed comment for
the formula, and would replace it everywhere with maxpollevents and
that will be cleaner (and use slightly less memory for large amounts
of sockets).
Got it. I also confused by the comment of absmaxevents before. And now I
know how to fix this issue.
Do you want to work on such a patch ?
Yes, I can send the patch later as you suggested:
1. remove the definition of absmaxevents
2. replace absmaxevents with global.tune.maxpollevents everywhere
3. remove the comment of absmaxevents
In the end, it seems that marking the bug as BUG/MINOR or OPTIM/MINOR is
OK. Since some memory will be saved by the fix, I'd prefer to
OPTIM/MINOR if you don't mind.
Attached are two patches:
0001 is the patch for epoll to remove absmaxevents
0002 is just a typo fix. Since I think it is irrelevant to the first
patch, I didn't merge them into a single one.
Both the patches should be also backported to 1.5.
--
Best Regards,
Godbach
>From 798d50b1922e9680ea69cbc39d54f20665c782da Mon Sep 17 00:00:00 2001
From: Godbach
Date: Wed, 17 Dec 2014 16:14:26 +0800
Subject: [PATCH 1/2] OPTIM/MINOR: epoll: epoll_events should be allocated
according to global.tune.maxpollevents
Willy: commit f2e8ee2b introduced an optimization in the oldspeculative epoll
code, which implemented its own event cache. It was needed to store that many
events (it was bound to maxsock/4 btw). Now the event cache lives on its own
and we don't need this anymore. And since events are allocated on the kernel
side, we only need to allocate the events we want to return.
As a result, absmaxevents will be not used anymore. Just remove the definition
and the comment of it, replace it with global.tune.maxpollevents. It is also an
optimization of memory usage for large amounts of sockets.
Signed-off-by: Godbach
---
src/ev_epoll.c | 5 +
1 file changed, 1 insertion(+), 4 deletions(-)
diff --git a/src/ev_epoll.c b/src/ev_epoll.c
index 755c6fa..c52d075 100644
--- a/src/ev_epoll.c
+++ b/src/ev_epoll.c
@@ -29,7 +29,6 @@
#include
-static int absmaxevents = 0;// absolute maximum amounts of polled events
/* private data */
static struct epoll_event *epoll_events;
@@ -195,10 +194,8 @@ REGPRM1 static int _do_init(struct poller *p)
if (epoll_fd < 0)
goto fail_fd;
- /* See comments at the top of the file about this formula. */
- absmaxevents = MAX(global.tune.maxpollevents, global.maxsock);
epoll_events = (struct epoll_event*)
- calloc(1, sizeof(struct epoll_event) * absmaxevents);
+ calloc(1, sizeof(struct epoll_event) *
global.tune.maxpollevents);
if (epoll_events == NULL)
goto fail_ee;
--
1.7.11.7
>From 7bda83893cc0bede6e5eb0865e805003aed792f4 Mon Sep 17 00:00:00 2001
From: Godbach
Date: Wed, 17 Dec 2014 16:32:05 +0800
Subject: [PATCH 2/2] DOC: fix typo: "401 Unauthorized" => "407 Unauthorized"
401 Unauthorized => 407 Unauthorized
Signed-off-by: Godbach
---
src/proto_http.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/src/proto_http.c b/src/proto_http.c
index f19a69b..0a94785 100644
--- a/src/proto_http.c
+++ b/src/proto_http.c
@@ -124,7 +124,7 @@ const char *HTTP_407_fmt =
"Content-Type: text/html\r\n"
"Proxy-Authenticate: Basic realm=\"%s\"\r\n"
"\r\n"
- "401 Unauthorized\nYou need a valid user and
password to access this content.\n\n";
+ "407 Unauthorized\nYou need a valid user and
password to access this content.\n\n";
const int http_err_codes[HTTP_ERR_SIZE] = {
--
1.7.11.7
Re: [BUG Report] Maybe a bug for epoll
On Wed, Dec 17, 2014 at 04:03:19PM +0800, Godbach wrote: > Got it. I also confused by the comment of absmaxevents before. And now I > know how to fix this issue. > > > > >Do you want to work on such a patch ? > > > > Yes, I can send the patch later as you suggested: > 1. remove the definition of absmaxevents > 2. replace absmaxevents with global.tune.maxpollevents everywhere > 3. remove the comment of absmaxevents > > In the end, it seems that marking the bug as BUG/MINOR or OPTIM/MINOR is > OK. Since some memory will be saved by the fix, I'd prefer to > OPTIM/MINOR if you don't mind. In my opinion it's a cleanup since we're removing a leftover from an ancient piece of code :-) Willy
Re: [BUG Report] Maybe a bug for epoll
On 2014/12/17 15:15, Willy Tarreau wrote: In my opinion, absmaxevents should be passed to epoll_wait() instead of global.tune.maxpollevents. No, that would defeat the principle of maxpollevents. In fact, I found what happened : commit f2e8ee2b introduced an optimization in the old speculative epoll code, which implemented its own event cache. It was needed to store that many events (it was bound to maxsock/4 btw). Now the event cache lives on its own and we don't need this anymore. And since events are allocated on the kernel side, we only need to allocate the events we want to return. Thus I'd completely remove absmaxevents, the comment on top of it suggesting to read the removed comment for the formula, and would replace it everywhere with maxpollevents and that will be cleaner (and use slightly less memory for large amounts of sockets). Got it. I also confused by the comment of absmaxevents before. And now I know how to fix this issue. Do you want to work on such a patch ? Yes, I can send the patch later as you suggested: 1. remove the definition of absmaxevents 2. replace absmaxevents with global.tune.maxpollevents everywhere 3. remove the comment of absmaxevents In the end, it seems that marking the bug as BUG/MINOR or OPTIM/MINOR is OK. Since some memory will be saved by the fix, I'd prefer to OPTIM/MINOR if you don't mind. -- Best Regards, Godbach
