2015-12-01 10:57 GMT+01:00 Thierry FOURNIER <thierry.fourn...@arpalert.org>:
> On Mon, 30 Nov 2015 18:03:00 +0100
> joris dedieu <joris.ded...@gmail.com> wrote:
>
>> Thanks Thierry, for your answers.
>>
>>
>> 2015-11-30 16:53 GMT+01:00 Thierry FOURNIER <thierry.fourn...@arpalert.org>:
>> > On Mon, 30 Nov 2015 08:37:00 +0100
>> > joris dedieu <joris.ded...@gmail.com> wrote:
>> >
>> >> Hi all,
>> >>
>> >> I started to drive into haproxy's lua interface. I produced a few code
>> >> that allows dnsbl lookup and it seems to work.
>> >>
>> >> First I have a C wrapper against the libc resolver..
>> >>
>> >> #include <netdb.h>
>> >> #include <sys/types.h>
>> >> #include <sys/socket.h>
>> >> #include <netinet/in.h>
>> >> #include <arpa/inet.h>
>> >>
>> >> #include <lua53/lua.h>
>> >> #include <lua53/lauxlib.h>
>> >>
>> >> static int gethostbyname_wrapper(lua_State *L)
>> >> {
>> >> const char* query = luaL_checkstring(L, 1);
>> >> struct hostent *he;
>> >> if ((he = gethostbyname(query)) != NULL) {
>> >> const char *first_addr =
>> >> inet_ntoa(*(struct in_addr*)he->h_addr_list[0]);
>> >> lua_pushstring(L, first_addr);
>> >> return 1;
>> >> }
>> >> return 0;
>> >> }
>> >>
>> >> static const luaL_Reg sysdb_methods[] = {
>> >> {"gethostbyname", gethostbyname_wrapper},
>> >> {NULL, NULL}
>> >> };
>> >>
>> >> LUALIB_API int luaopen_sysdb(lua_State *L) {
>> >> luaL_newlib(L, sysdb_methods);
>> >> return 1;
>> >> }
>> >>
>> >> I have some doubts on the asyncness of libc operations but in other
>> >> side I don't want to reinvent the wheel. Should I prefer a resolver
>> >> implementation that uses lua socket ? As far as I tested libc seems to
>> >> do the job.
>> >
>> > Hello,
>> >
>> > I confirm your doubts: gethostbyname is synchronous and it is a
>> > blocking call. If your hostname resolution is in the /etc/hosts file,
>> > it blocks while reading file. It it is from DNS server, it blocks
>> > waiting for the server response (or worse: wainting for the timeout).
>> >
>> > So, this code seems to run, but your HAProxy will be not efficient
>> > because the entire haproxy process will be blocked during each
>> > resolution. For example: if your DNS fails after 3s timeout, during 3s,
>> > HAProxy doesn't process any data.
>> >
>> > Otherwise, your code is the good way to perform fast Lua/C libraries.
>> >
>> > There are no way to simulate blocking access out of the HAProxy core,
>> > all the functions writed for Lua must be non block.
>>
>> Ok, I will check for a non blocking solution (maybe lua socket + pack
>> / unpack in C) .
>
>
> This is an potential solution because the Lua Sockets (implemented in
> HAProxy) are non blocking. You cannot exploit the Lua socket from the C
> part, because the non blocking system can't throught 2 level of Lua/C.
>
> Maybe you can use the DNS protocol with and TCP/53 DNS server. With
> this solution, the DNS serveur already exists, and the TCP socket are
> already non block compliant.
I started to port lua-resty-dns (pure lua). TCP is not a problem as
it's known by all major resolvers and as far as I know require by the
standard.
>
> Note that HAProxy supports now the native DNS, maybe in the version 1.7
> the DNS will be repported in le Lua core.
It would be great. IMHO unless your are too hurry, DNS is a nice way
to distribute informations across a network. I already see tow usages
: dnsbl and googlebot^w crawler validation. But there should be more
usage.
Joris
>
>
>> >
>> >
>> >> Then the lua code
>> >>
>> >> local sysdb = require("sysdb")
>> >>
>> >> core.register_fetches("rbl", function(txn, rbl, ip)
>> >> if (not ip) then
>> >> ip = txn.sf:src()
>> >> end
>> >> if (not rbl) then
>> >> rbl = "zen.spamhaus.org"
>> >> end
>> >> local query = rbl
>> >> for x in string.gmatch(ip, "[^%.]+") do
>> >> query = x .. '.' .. query
>> >> end
>> >> if(sysdb.gethostbyname(query)) then
>> >> return 1
>> >> else
>> >> return 0
>> >> end
>> >> end)
>> >>
>> >> I want to use a sticky table as a local cache so my second question :
>> >> is there a way to set a gpt0 value from lua ?
>> >
>> >
>> > You can use the samples fetches mapper and use the sc_set_gpt0. The
>> > syntax is like this:
>> >
>> > For the read access:
>> >
>> > txn.sf:sc_set_gpt0()
>> > txn.sc:table_gpc0()
>> >
>> > For the write access, I don't have direct solution. You must use an Lua
>> > sample fetch and the following configuration directive:
>> >
>> > http-request sc-set-gpt0 lua.my_sample_fetch
>>
>> Yes that's an option.
>>
>> >
>> > Maybe it will be a good idea to implement the stick table access in Lua.
>> >
>> > If you want a other maneer to store shared data inhaproxy, you can use
>> > maps. The maps are shared by all the HAProxy process including Lua with
>> > a special API (see Map class)
>>
>> I thought on Maps but I didn't find a write access in lua according
>> to http://www.arpalert.org/src/haproxy-lua-api/1.6/index.html#map-class
>> and some of my experiments
>
>
> You're absolutely right. I'm afraid that I forgot to implement the
> write acces. Maybe I had a good reason to not implement this, but I'm
> not remember.
>
> I will look for this.
>
>
> Thierry
>
>
>> Thanks
>> Joris
>> >
>> > Thierry
