Re: haproxy reverse proxy to https streaming backend

[Thomas Schmiedl]

Hello Pieter,

thanks for your help, it works well now. The regex solution was my only
idea, because I'm not a developer. I know, the haproxy workaround isn't
the best solution, but nobody would fix the xupnpd2 hls-handling.

Maybe you could help me again. I see, that the playlist has 2 "states"
("header" tags).

#EXTM3U
#EXT-X-VERSION:3
#EXT-X-MEDIA-SEQUENCE:0
#EXT-X-TARGETDURATION:50
#EXT-X-DISCONTINUITY

and

#EXTM3U
#EXT-X-VERSION:3
#EXT-X-MEDIA-SEQUENCE:<number>
#EXT-X-TARGETDURATION:2

The result from the lua-script (header tags) should always be:

#EXTM3U
#EXT-X-VERSION:3
#EXT-X-MEDIA-SEQUENCE:<number>
#EXT-X-TARGETDURATION:2

Thanks,
Thomas

Am 15.03.2019 um 00:27 schrieb PiBa-NL:

Hi Thomas,

Op 14-3-2019 om 20:28 schreef Thomas Schmiedl:

Hello,

I never got a reply from the original author of xupnpd2 to fix the
hls-handling, so I created a lua-script (thanks to Thierry Fournier),
but it's too slow for the router cpu. Could someone rewrite the script
to a lua-c-module?

I don't think making this exact code a lua-c-module would solve the
issue, lua is not a 'slow' language. But I do wonder if regex is the
right tool for data manipulation..

Regards,
Thomas

test.cfg:
global
        lua-load /var/media/ftp/playlist.lua

frontend main
        mode http
        bind *:8080
        acl is_index_m3u8 path -m end /index.m3u8
        http-request use-service lua.playlist if is_index_m3u8
        default_backend forward

backend forward
        mode http
        server gjirafa puma.gjirafa.com:443 ssl verify none

playlist.lua:
core.register_service("playlist", "http", function(applet)
        local tcp = core.tcp()
        tcp:connect_ssl("51.75.52.73", 443)
        tcp:send("GET ".. applet.path .." HTTP/1.1\r\nConnection:
Close\r\nHost: puma.gjirafa.com\r\n\r\n")
        local body = tcp:receive("*a")

        local result = string.match(body,"^.*(#EXTM3U.-)#EXTINF")
        result = result ..
string.match(body,"(...............%d+.ts................%d+.ts................%d+.ts)[\r\n|0]*$")

I think a 'easier' regex might already improve performance, can you try
this one for example ?:
     result = result ..
string.match(body,"(#EXTINF:%d+[/.]%d+,\n%d+[/.]ts.#EXTINF:%d[/.]%d%d%d,.%d+[/.]ts.#EXTINF:%d+[/.]%d+,\n%d+[/.]ts)[\r\n|0]*$")


With my test using 'https://rextester.com/l/lua_online_compiler' and a
little sample m3u8 it seemed to work faster anyhow.

        applet:set_status(200)
        applet:add_header("Content-Type", "application/x-mpegURL")
        applet:add_header("content-length", string.len(result))
        applet:add_header("Connection", "close")
        applet:start_response()
        applet:send(result)
end)

Am 19.02.2019 um 21:31 schrieb Thomas Schmiedl:

Am 19.02.2019 um 05:29 schrieb Willy Tarreau:

Hello Thomas,

On Sun, Feb 17, 2019 at 05:55:29PM +0100, Thomas Schmiedl wrote:

Hello Bruno,

I think the problem is the parsing of the .m3u8-playlist in
xupnpd2. The
first entry to the .ts-file is 4 hours behind the actual time. But I
have no c++ experience to change the code.

For me if it works but not correctly like this, it clearly indicates
there is a (possibly minor) incompatibility between the client and the
server. It just happens that if your client doesn't support https, it
was never tested against this server and very likely needs to be
adapted
to work correctly.

Is it possible in haproxy to manipulate the playlist file (server
response), that only the last .ts-entries will be available and
returned
to xupnpd2?

No, haproxy doesn't manipulate contents. Not only it's completely
out of
the scope of a load balancing proxy, but it would also encourage some
users to try to work around some of their deployment issues in the
ugliest
possible way, causing even more trouble (and frankly, on *every*
infrastructure where you find such horrible tricks deployed, the admins
implore you to help them because they're in big trouble and are stuck
with
no option left to fix the issues they've created).

If it's only a matter of modifying one file on the fly, you may manage
to do it using Lua : instead of forwarding the request to the server,
you send it to a Lua function, which itself makes the request to the
server, buffers the response, rewrites it, then sends it back to the
client. You must just make sure to only send there the requests for
the playlist file and nothing else.

Could someone send me such a lua-script example and how to include in
haproxy. Thanks

I personally think this is ugly compared to trying to fix the faulty
client. Maybe you can report your issue to the author(s) and share your
config to help them reproduce it ?

Regards,
Willy

Regards,
PiBa-NL (Pieter)