Re: Makefile:813: recipe for target 'haproxy' failed

[Aleksandar Lazic]

Hi.

-- Originalnachricht --
Von: "Milenko Markovic" 
An: haproxy@formilux.org
Gesendet: 07.01.2018 07:53:44
Betreff: Makefile:813: recipe for target 'haproxy' failed


Dear Sir or Madam

When I run
make TARGET=linux24 USE_OPENSSL=1 SSL_INC=$STATICLIBSSL/include 
SSL_LIB=$STATICLIBSSL/lib ADDLIB=-ldl

this appears on screen
Makefile:813: recipe for target 'haproxy' failed

I have attached the whole output as txt file. It would be nice if 
someone could help me.
Looks like that you have build the 'staticlibssl' with thread support 
but the gcc miss the '-lpthread' or similar library.



```
gcc -g -o haproxy src/haproxy.o src/base64.o src/protocol.o 
src/uri_auth.o src/standard.o src/buffer.o src/log.o src/task.o 
src/chunk.o src/channel.o src/listener.o src/lru.o src/xxhash.o 
src/time.o src/fd.o src/pipe.o src/regex.o src/cfgparse.o src/server.o 
src/checks.o src/queue.o src/frontend.o src/proxy.o src/peers.o 
src/arg.o src/stick_table.o src/proto_uxst.o src/connection.o 
src/proto_http.o src/raw_sock.o src/backend.o src/tcp_rules.o 
src/lb_chash.o src/lb_fwlc.o src/lb_fwrr.o src/lb_map.o src/lb_fas.o 
src/stream_interface.o src/stats.o src/proto_tcp.o src/applet.o 
src/session.o src/stream.o src/hdr_idx.o src/ev_select.o src/signal.o 
src/acl.o src/sample.o src/memory.o src/freq_ctr.o src/auth.o 
src/proto_udp.o src/compression.o src/payload.o src/hash.o src/pattern.o 
src/map.o src/namespace.o src/mailers.o src/dns.o src/vars.o 
src/filters.o src/flt_http_comp.o src/flt_trace.o src/flt_spoe.o 
src/cli.o src/ev_poll.o src/ssl_sock.o src/shctx.o ebtree/ebtree.o 
ebtree/eb32tree.o ebtree/eb64tree.o ebtree/ebmbtree.o ebtree/ebsttree.o 
ebtree/ebimtree.o ebtree/ebistree.o -lcrypt -ldl -L/tmp/staticlibssl/lib 
-lssl -lcrypto -ldl -ldl
/tmp/staticlibssl/lib/libcrypto.a(threads_pthread.o): In function 
`CRYPTO_THREAD_lock_new':
threads_pthread.c:(.text+0x25): undefined reference to 
`pthread_rwlock_init'
/tmp/staticlibssl/lib/libcrypto.a(threads_pthread.o): In function 
`CRYPTO_THREAD_read_lock':
threads_pthread.c:(.text+0x65): undefined reference to 
`pthread_rwlock_rdlock'
/tmp/staticlibssl/lib/libcrypto.a(threads_pthread.o): In function 
`CRYPTO_THREAD_write_lock':
threads_pthread.c:(.text+0x85): undefined reference to 
`pthread_rwlock_wrlock'
/tmp/staticlibssl/lib/libcrypto.a(threads_pthread.o): In function 
`CRYPTO_THREAD_unlock':
threads_pthread.c:(.text+0xa5): undefined reference to 
`pthread_rwlock_unlock'
/tmp/staticlibssl/lib/libcrypto.a(threads_pthread.o): In function 
`CRYPTO_THREAD_lock_free':
threads_pthread.c:(.text+0xca): undefined reference to 
`pthread_rwlock_destroy'
/tmp/staticlibssl/lib/libcrypto.a(threads_pthread.o): In function 
`CRYPTO_THREAD_run_once':

threads_pthread.c:(.text+0xf5): undefined reference to `pthread_once'
/tmp/staticlibssl/lib/libcrypto.a(threads_pthread.o): In function 
`CRYPTO_THREAD_init_local':
threads_pthread.c:(.text+0x115): undefined reference to 
`pthread_key_create'
/tmp/staticlibssl/lib/libcrypto.a(threads_pthread.o): In function 
`CRYPTO_THREAD_set_local':
threads_pthread.c:(.text+0x147): undefined reference to 
`pthread_setspecific'
/tmp/staticlibssl/lib/libcrypto.a(threads_pthread.o): In function 
`CRYPTO_THREAD_cleanup_local':
threads_pthread.c:(.text+0x167): undefined reference to 
`pthread_key_delete'
/tmp/staticlibssl/lib/libcrypto.a(threads_pthread.o): In function 
`CRYPTO_THREAD_get_local':
threads_pthread.c:(.text+0x133): undefined reference to 
`pthread_getspecific'

collect2: error: ld returned 1 exit status
Makefile:813: recipe for target 'haproxy' failed
```

As usual please can you tell us more about your system.

Which OS?
Which glibc?
Which dev libs?
Which gcc?
Which libssl?


All the best

Milenko


Regards
Aleks

Re: High throughput SFTP server load balancing with HAProxy

[Marco Corte]

Hello!

All traffic will flow through haproxy which will act as a TCP layer4 
switch.
To avoid bottlenecks, the haproxy node NICs need to provide at least as 
much bandwidth as the sum of the expected traffic on each SFTP server.


.marcoc

Makefile:813: recipe for target 'haproxy' failed

[Milenko Markovic]

Dear Sir or Madam

When I run
make TARGET=linux24 USE_OPENSSL=1 SSL_INC=$STATICLIBSSL/include
SSL_LIB=$STATICLIBSSL/lib ADDLIB=-ldl
this appears on screen
Makefile:813: recipe for target 'haproxy' failed

I have attached the whole output as txt file. It would be nice if someone
could help me.
All the best

Milenko
 make TARGET=linux24 USE_OPENSSL=1 SSL_INC=$STATICLIBSSL/include 
SSL_LIB=$STATICLIBSSL/lib ADDLIB=-ldl
gcc -Iinclude -Iebtree -Wall  -O2 -g -fno-strict-aliasing 
-Wdeclaration-after-statement -fwrapv   -DTPROXY -DCONFIG_HAP_CRYPT 
-DENABLE_POLL -DNETFILTER -DUSE_GETSOCKNAME -DUSE_OPENSSL 
-I/tmp/staticlibssl/include  -DCONFIG_HAPROXY_VERSION=\"1.7.10-a7dcc3b\" 
-DCONFIG_HAPROXY_DATE=\"2018/01/02\" \
  -DBUILD_TARGET='"linux24"' \
  -DBUILD_ARCH='""' \
  -DBUILD_CPU='"generic"' \
  -DBUILD_CC='"gcc"' \
  -DBUILD_CFLAGS='"-O2 -g -fno-strict-aliasing 
-Wdeclaration-after-statement -fwrapv"' \
  -DBUILD_OPTIONS='"USE_OPENSSL=1"' \
   -c -o src/haproxy.o src/haproxy.c
gcc -Iinclude -Iebtree -Wall  -O2 -g -fno-strict-aliasing 
-Wdeclaration-after-statement -fwrapv   -DTPROXY -DCONFIG_HAP_CRYPT 
-DENABLE_POLL -DNETFILTER -DUSE_GETSOCKNAME -DUSE_OPENSSL 
-I/tmp/staticlibssl/include  -DCONFIG_HAPROXY_VERSION=\"1.7.10-a7dcc3b\" 
-DCONFIG_HAPROXY_DATE=\"2018/01/02\" -c -o src/base64.o src/base64.c
gcc -Iinclude -Iebtree -Wall  -O2 -g -fno-strict-aliasing 
-Wdeclaration-after-statement -fwrapv   -DTPROXY -DCONFIG_HAP_CRYPT 
-DENABLE_POLL -DNETFILTER -DUSE_GETSOCKNAME -DUSE_OPENSSL 
-I/tmp/staticlibssl/include  -DCONFIG_HAPROXY_VERSION=\"1.7.10-a7dcc3b\" 
-DCONFIG_HAPROXY_DATE=\"2018/01/02\" -c -o src/protocol.o src/protocol.c
gcc -Iinclude -Iebtree -Wall  -O2 -g -fno-strict-aliasing 
-Wdeclaration-after-statement -fwrapv   -DTPROXY -DCONFIG_HAP_CRYPT 
-DENABLE_POLL -DNETFILTER -DUSE_GETSOCKNAME -DUSE_OPENSSL 
-I/tmp/staticlibssl/include  -DCONFIG_HAPROXY_VERSION=\"1.7.10-a7dcc3b\" 
-DCONFIG_HAPROXY_DATE=\"2018/01/02\" -c -o src/uri_auth.o src/uri_auth.c
gcc -Iinclude -Iebtree -Wall  -O2 -g -fno-strict-aliasing 
-Wdeclaration-after-statement -fwrapv   -DTPROXY -DCONFIG_HAP_CRYPT 
-DENABLE_POLL -DNETFILTER -DUSE_GETSOCKNAME -DUSE_OPENSSL 
-I/tmp/staticlibssl/include  -DCONFIG_HAPROXY_VERSION=\"1.7.10-a7dcc3b\" 
-DCONFIG_HAPROXY_DATE=\"2018/01/02\" -c -o src/standard.o src/standard.c
gcc -Iinclude -Iebtree -Wall  -O2 -g -fno-strict-aliasing 
-Wdeclaration-after-statement -fwrapv   -DTPROXY -DCONFIG_HAP_CRYPT 
-DENABLE_POLL -DNETFILTER -DUSE_GETSOCKNAME -DUSE_OPENSSL 
-I/tmp/staticlibssl/include  -DCONFIG_HAPROXY_VERSION=\"1.7.10-a7dcc3b\" 
-DCONFIG_HAPROXY_DATE=\"2018/01/02\" -c -o src/buffer.o src/buffer.c
gcc -Iinclude -Iebtree -Wall  -O2 -g -fno-strict-aliasing 
-Wdeclaration-after-statement -fwrapv   -DTPROXY -DCONFIG_HAP_CRYPT 
-DENABLE_POLL -DNETFILTER -DUSE_GETSOCKNAME -DUSE_OPENSSL 
-I/tmp/staticlibssl/include  -DCONFIG_HAPROXY_VERSION=\"1.7.10-a7dcc3b\" 
-DCONFIG_HAPROXY_DATE=\"2018/01/02\" -c -o src/log.o src/log.c
gcc -Iinclude -Iebtree -Wall  -O2 -g -fno-strict-aliasing 
-Wdeclaration-after-statement -fwrapv   -DTPROXY -DCONFIG_HAP_CRYPT 
-DENABLE_POLL -DNETFILTER -DUSE_GETSOCKNAME -DUSE_OPENSSL 
-I/tmp/staticlibssl/include  -DCONFIG_HAPROXY_VERSION=\"1.7.10-a7dcc3b\" 
-DCONFIG_HAPROXY_DATE=\"2018/01/02\" -c -o src/task.o src/task.c
gcc -Iinclude -Iebtree -Wall  -O2 -g -fno-strict-aliasing 
-Wdeclaration-after-statement -fwrapv   -DTPROXY -DCONFIG_HAP_CRYPT 
-DENABLE_POLL -DNETFILTER -DUSE_GETSOCKNAME -DUSE_OPENSSL 
-I/tmp/staticlibssl/include  -DCONFIG_HAPROXY_VERSION=\"1.7.10-a7dcc3b\" 
-DCONFIG_HAPROXY_DATE=\"2018/01/02\" -c -o src/chunk.o src/chunk.c
gcc -Iinclude -Iebtree -Wall  -O2 -g -fno-strict-aliasing 
-Wdeclaration-after-statement -fwrapv   -DTPROXY -DCONFIG_HAP_CRYPT 
-DENABLE_POLL -DNETFILTER -DUSE_GETSOCKNAME -DUSE_OPENSSL 
-I/tmp/staticlibssl/include  -DCONFIG_HAPROXY_VERSION=\"1.7.10-a7dcc3b\" 
-DCONFIG_HAPROXY_DATE=\"2018/01/02\" -c -o src/channel.o src/channel.c
gcc -Iinclude -Iebtree -Wall  -O2 -g -fno-strict-aliasing 
-Wdeclaration-after-statement -fwrapv   -DTPROXY -DCONFIG_HAP_CRYPT 
-DENABLE_POLL -DNETFILTER -DUSE_GETSOCKNAME -DUSE_OPENSSL 
-I/tmp/staticlibssl/include  -DCONFIG_HAPROXY_VERSION=\"1.7.10-a7dcc3b\" 
-DCONFIG_HAPROXY_DATE=\"2018/01/02\" -c -o src/listener.o src/listener.c
gcc -Iinclude -Iebtree -Wall  -O2 -g -fno-strict-aliasing 
-Wdeclaration-after-statement -fwrapv   -DTPROXY -DCONFIG_HAP_CRYPT 
-DENABLE_POLL -DNETFILTER -DUSE_GETSOCKNAME -DUSE_OPENSSL 
-I/tmp/staticlibssl/include  -DCONFIG_HAPROXY_VERSION=\"1.7.10-a7dcc3b\" 
-DCONFIG_HAPROXY_DATE=\"2018/01/02\" -c -o src/lru.o src/lru.c
gcc -Iinclude -Iebtree -Wall  -O2 -g -fno-strict-aliasing 
-Wdeclaration-after-statement -fwrapv   -DTPROXY -DCONFIG_HAP_CRYPT 
-DENABLE_POLL -DNETFILTER -DUSE_GETSOCKNAME -DUSE_OPENSSL 
-I/tmp/staticlibssl/include  -DCONFIG_HAPROXY_VERSION=\"1.7.10-a7dcc3b\" 
-DCONFIG_HAPROXY_DATE=\"2018/01/02\" 

High throughput SFTP server load balancing with HAProxy

[Imam Toufique]

Hi,

I just started reading about HAProxy, I am very much new to it.  I am
planning to deploy 4 SFTP servers and a HAProxy balancer.  In this setup, I
am anticipating quite an intense level of file transfer activities to/from
these SFTP servers.  I am not sure how HAProxy would handle the very high
level of I/O (file transfers) , as I am not if all the data will be going
through HAProxy network connections - or does HAProxy would create a direct
connection between the clients and SFTP server(s) directly.  in my case,
could HAProxy become a bottleneck itself if all the data transfer traffic
is passing through it.

thanks

Re[2]: haproxy without balancing

[Aleksandar Lazic]

Hi Angelo.

-- Originalnachricht --
Von: "Angelo Hongens" 
An: "Aleksandar Lazic" ; haproxy@formilux.org
Gesendet: 06.01.2018 18:20:47
Betreff: Re: haproxy without balancing


Hey Aleksandar,

On 05-01-2018 22:05, Aleksandar Lazic wrote:
We run a lot of balancers with varnish+hitch+haproxy+corosync for 
high-available loadbalancing. Perhaps high-availability is not a 
requirement, but it's also nice to be able to do maintenance during 
the day and have your standby node take over..
Just for my curiosity why hitch and not only haproxy for ssl 
termination?


I use varnish as a single point of entry for requests and for caching. 
I guess because it's a really good product, and we've been using it for 
a long time. It has some custom business logic built in our vcl as 
well, and allows for a lot of http magic. I got training on varnish 
tuning and monitoring, and all of our scripts revolve around varnish 
and its logs. And they have very cool real-time analysis tools like 
varnishlog, varnishhist, varnishstat, etc.


Varnish passes all requests to a local haproxy instance, which passes 
requests to the right backends based on hostname. So we use haproxy for 
balancing to backends.


When the time came we needed ssl termination, I wanted a simple 
solution that does that one thing well, and I still wanted varnish as 
entry point. We played around with different products (squid, nginx), 
but then the varnish team forked stud and called it hitch. And the nice 
thing is almost all varnish users use hitch for ssl termination, and 
the varnish team is willing to offer commercial support for both.


I've been thinking about different setups as well, such as running one 
haproxy instance for ssl termination, passing requests to varnish and 
then pass it to another instance of haproxy that sends requests to the 
backends, but I think my current setup serves us best and we use the 
best tool for the jobs at hand. I think hitch is a great ssl 
terminator, varnish is a great cache/spoonfeeder, and haproxy is the 
best balancer.


--
met vriendelijke groet,
Angelo Höngens

Thank you very much for your detailed answer.
I fully agree with you, a specially as you have a working and supported 
set-up.


It would be interesting if hitch can be replaced with haproxy without 
any issues.


I plan to use haproxy in front of varnish and I would be very 
appreciative for any hints, maybe off-list so that we don't upset the 
haproxy list members.


Best regards
Aleks

[PATCH] BUG/MINOR: lua: Fix return value of Socket.settimeout

[Tim Duesterhus]

The `socket.tcp.settimeout` method of Lua returns `1` in all cases,
while the `Socket.settimeout` method of haproxy returns `0` in all
cases. This breaks the `socket.http` module, because it validates
the return value of `settimeout`.

This bug was introduced in commit 7e7ac32dad1e15c19152d37aaf9ea6b3f00a7226
(which is the very first commit adding the Socket class to Lua). This
bugfix should be backported to every branch containing that commit:
- 1.6
- 1.7
- 1.8

A test case for this bug is as follows:

The 'Test' response header will contain an HTTP status code with the
patch applied and will be zero (nil) without the patch applied.

http.lua:
  http = require("socket.http")

  core.register_action("bug", { "http-req" }, function(txn)
local b, c, h = http.request {
url = "http://93.184.216.34;,
headers = {
Host = "example.com"
},
create = core.tcp,
redirect = false
}

txn:set_var("txn.foo", c)
  end)

haproxy.cfg:
  global
lua-load /scratch/haproxy/http.lua

  frontend fe
bind 127.0.0.1:8080
http-request lua.bug
http-response set-header Test %[var(txn.foo)]

default_backend be

  backend be
server s example.com:80
---
 src/hlua.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/src/hlua.c b/src/hlua.c
index 3d5a81cac..fa629ba94 100644
--- a/src/hlua.c
+++ b/src/hlua.c
@@ -2490,7 +2490,7 @@ __LJMP static int hlua_socket_settimeout(struct lua_State 
*L)
s->res.wto = tmout;
xref_unlock(>xref, peer);
 
-   return 0;
+   return 1;
 }
 
 __LJMP static int hlua_socket_new(lua_State *L)
-- 
2.15.1

[PATCH] BUG/MEDIUM: lua: Fix IPv6 with separate port support for Socket.connect

[Tim Duesterhus]

The `socket.tcp.connect` method of Lua requires at least two parameters:
The host and the port. The `Socket.connect` method of haproxy requires
only one when a host with a combined port is provided. This stems from
the fact that `str2sa_range` is used internally in `hlua_socket_connect`.
This very fact unfortunately causes a diversion in the behaviour of
Lua's socket class and haproxy's for IPv6 addresses:

  sock:connect("::1", "80")

works fine with Lua, but fails with:

  connect: cannot parse destination address '::1'

in haproxy, because `str2sa_range` parses the trailing `:1` as the port.

This patch forcefully adds a `:` to the end of the address iff a port
number greater than `0` is given as the second parameter.

Technically this breaks backwards compatibility, because the docs state:

> The syntax "127.0.0.1:1234" is valid. in this case, the
> parameter *port* is ignored.

But: The connect() call can only succeed if the second parameter is left
out (which causes no breakage) or if the second parameter is an integer
or a numeric string.

It seems unlikely that someone would provide an address with a port number
and would also provide a second parameter containing a number other than
zero. Thus I feel this breakage is warranted to fix the mismatch between
haproxy's socket class and Lua's one.

This commit should be backported to haproxy 1.8 only, because of the
possible breakage of existing Lua scripts.
---
 doc/lua-api/index.rst |  2 +-
 src/hlua.c| 15 ++-
 2 files changed, 15 insertions(+), 2 deletions(-)

diff --git a/doc/lua-api/index.rst b/doc/lua-api/index.rst
index dceab01d7..7fe609fc9 100644
--- a/doc/lua-api/index.rst
+++ b/doc/lua-api/index.rst
@@ -1769,7 +1769,7 @@ Socket class
   "abns@", and finaly a filedescriotr can be passed with the prefix "fd@".
   The prefix "ipv4@", "ipv6@" and "unix@" are also supported. The port can be
   passed int the string. The syntax "127.0.0.1:1234" is valid. in this case, 
the
-  parameter *port* is ignored.
+  parameter *port* must not be set.
 
 .. js:function:: Socket.connect_ssl(socket, address, port)
 
diff --git a/src/hlua.c b/src/hlua.c
index 3b4fc3b54..3d5a81cac 100644
--- a/src/hlua.c
+++ b/src/hlua.c
@@ -2333,9 +2333,22 @@ __LJMP static int hlua_socket_connect(struct lua_State 
*L)
WILL_LJMP(luaL_error(L, "connect: cannot use socket on other 
thread"));
 
ip  = MAY_LJMP(luaL_checkstring(L, 2));
-   if (lua_gettop(L) >= 3)
+   if (lua_gettop(L) >= 3) {
+   luaL_Buffer b;
port = MAY_LJMP(luaL_checkinteger(L, 3));
 
+   /* Force the ip to end with a colon, to support IPv6 addresses
+* that are not enclosed within square brackets.
+*/
+   if (port > 0) {
+   luaL_buffinit(L, );
+   luaL_addstring(, ip);
+   luaL_addchar(, ':');
+   luaL_pushresult();
+   ip = lua_tolstring(L, lua_gettop(L), NULL);
+   }
+   }
+
/* check for connection break. If some data where read, return it. */
peer = xref_get_peer_and_lock(>xref);
if (!peer) {
-- 
2.15.1

Re: haproxy without balancing

[Angelo Hongens]

Hey Aleksandar,

On 05-01-2018 22:05, Aleksandar Lazic wrote:
We run a lot of balancers with varnish+hitch+haproxy+corosync for 
high-available loadbalancing. Perhaps high-availability is not a 
requirement, but it's also nice to be able to do maintenance during 
the day and have your standby node take over..

Just for my curiosity why hitch and not only haproxy for ssl termination?


I use varnish as a single point of entry for requests and for caching. I 
guess because it's a really good product, and we've been using it for a 
long time. It has some custom business logic built in our vcl as well, 
and allows for a lot of http magic. I got training on varnish tuning and 
monitoring, and all of our scripts revolve around varnish and its logs. 
And they have very cool real-time analysis tools like varnishlog, 
varnishhist, varnishstat, etc.


Varnish passes all requests to a local haproxy instance, which passes 
requests to the right backends based on hostname. So we use haproxy for 
balancing to backends.


When the time came we needed ssl termination, I wanted a simple solution 
that does that one thing well, and I still wanted varnish as entry 
point. We played around with different products (squid, nginx), but then 
the varnish team forked stud and called it hitch. And the nice thing is 
almost all varnish users use hitch for ssl termination, and the varnish 
team is willing to offer commercial support for both.


I've been thinking about different setups as well, such as running one 
haproxy instance for ssl termination, passing requests to varnish and 
then pass it to another instance of haproxy that sends requests to the 
backends, but I think my current setup serves us best and we use the 
best tool for the jobs at hand. I think hitch is a great ssl terminator, 
varnish is a great cache/spoonfeeder, and haproxy is the best balancer.



--

met vriendelijke groet,

Angelo Höngens