Hi ports@, did anybody succeed at serving HTTP/3 traffic with HAProxy? It should be supported since 2.8, but I can't make it work: `curl --http3-only` gets stuck and usually ends with
curl: (55) ngtcp2_conn_writev_stream returned error: ERR_DRAINING It does work against https://http3.is, https://cloudflare.com and others. I'm trying with the following config, which does work for HTTP/1.1 and HTTP/2: global log 127.0.0.1 local0 debug maxconn 1024 chroot /var/haproxy user _haproxy group _haproxy daemon pidfile /var/run/haproxy.pid ssl-default-bind-options ssl-min-ver TLSv1.2 ssl-load-extra-del-ext defaults log global mode http option httplog option dontlognull option redispatch retries 3 maxconn 2000 timeout connect 5s timeout client 65s timeout server 5s frontend haproxy bind ipv4@:80,ipv6@:80 bind ipv4@:443,ipv6@:443 ssl crt /etc/haproxy/certs/ bind quic4@:443,quic6@:443 ssl crt /etc/haproxy/certs/ option forwardfor acl acme-challenge path_beg /.well-known/acme-challenge/ acl ntfy req.hdr(host) -i ntfy.example.com acl grafana req.hdr(host) -i grafana.example.com http-request redirect scheme https unless { ssl_fc } || acme-challenge http-after-response add-header alt-svc 'h3=":443"; ma=900;' use_backend httpd if acme-challenge use_backend ntfy_ws if ntfy { path_end /ws } use_backend ntfy if ntfy use_backend grafana if grafana default_backend httpd backend httpd server s1 127.0.0.1:8080 check backend ntfy_ws option httpchk /v1/health option http-server-close timeout tunnel 10m server s1 127.0.0.1:3010 check backend ntfy option httpchk /v1/health server s1 127.0.0.1:3010 check backend grafana option httpchk /api/health server s1 127.0.0.1:3000 check Adding an alpn directive to bind lines makes no difference, and according to the docs, the "normal" binds get an `alpn h2,http1.1` while the quic binds get an `alpn h3` by default. tcpdump shows that there is some handshakes attempts between client and server, and so does the stats socket of HAProxy: > show quic full * 0xd10d64000[00]: scid=4f5f572ad85655a9........................ dcid=4559862ad37160765abf2b2082ad0e624fe59237 loc. TPs: odcid=5cf7472f2d706253c37792abe48c49eea466ffd1 iscid=4f5f572ad85655a9 midle_timeout=30000ms mudp_payload_sz=2048 ack_delay_exp=3 mack_delay=25ms act_cid_limit=8 md=1687140 msd_bidi_l=16380 msd_bidi_r=16380 msd_uni=16380 ms_bidi=100 ms_uni=3 (no_act_migr,stless_rst_tok) rem. TPs: iscid=4559862ad37160765abf2b2082ad0e624fe59237 midle_timeout=120000ms mudp_payload_sz=65527 ack_delay_exp=3 mack_delay=25ms act_cid_limit=2 md=1310720 msd_bidi_l=131072 msd_bidi_r=131072 msd_uni=131072 ms_bidi=262144 ms_uni=262144 versions:chosen=0x00000001,negotiated=0x00000001 st=handshake mux=null expire=24s fd=-1 local_addr=128.140.63.137:443 foreign_addr=5.161.47.47:56773 [initl] rx.ackrng=1 tx.inflight=0 [hndshk] rx.ackrng=0 tx.inflight=9877 [01rtt] rx.ackrng=0 tx.inflight=0 srtt=274 rttvar=137 rttmin=274 ptoc=3 cwnd=12707 mcwnd=12707 sentpkts=11 lostpkts=0 > show quic full * 0xd10d64000[00]: scid=4f5f572ad85655a9........................ dcid=4559862ad37160765abf2b2082ad0e624fe59237 loc. TPs: odcid=5cf7472f2d706253c37792abe48c49eea466ffd1 iscid=4f5f572ad85655a9 midle_timeout=30000ms mudp_payload_sz=2048 ack_delay_exp=3 mack_delay=25ms act_cid_limit=8 md=1687140 msd_bidi_l=16380 msd_bidi_r=16380 msd_uni=16380 ms_bidi=100 ms_uni=3 (no_act_migr,stless_rst_tok) rem. TPs: iscid=4559862ad37160765abf2b2082ad0e624fe59237 midle_timeout=120000ms mudp_payload_sz=65527 ack_delay_exp=3 mack_delay=25ms act_cid_limit=2 md=1310720 msd_bidi_l=131072 msd_bidi_r=131072 msd_uni=131072 ms_bidi=262144 ms_uni=262144 versions:chosen=0x00000001,negotiated=0x00000001 st=handshake mux=null expire=10s fd=-1 local_addr=128.140.63.137:443 foreign_addr=5.161.47.47:56773 [initl] rx.ackrng=1 tx.inflight=0 [hndshk] rx.ackrng=0 tx.inflight=14137 [01rtt] rx.ackrng=0 tx.inflight=0 srtt=274 rttvar=137 rttmin=274 ptoc=5 cwnd=12707 mcwnd=12707 sentpkts=15 lostpkts=0 > show quic full * 0xd10d64000[00]: scid=4f5f572ad85655a9........................ dcid=4559862ad37160765abf2b2082ad0e624fe59237 loc. TPs: odcid=5cf7472f2d706253c37792abe48c49eea466ffd1 iscid=4f5f572ad85655a9 midle_timeout=30000ms mudp_payload_sz=2048 ack_delay_exp=3 mack_delay=25ms act_cid_limit=8 md=1687140 msd_bidi_l=16380 msd_bidi_r=16380 msd_uni=16380 ms_bidi=100 ms_uni=3 (no_act_migr,stless_rst_tok) rem. TPs: iscid=4559862ad37160765abf2b2082ad0e624fe59237 midle_timeout=120000ms mudp_payload_sz=65527 ack_delay_exp=3 mack_delay=25ms act_cid_limit=2 md=1310720 msd_bidi_l=131072 msd_bidi_r=131072 msd_uni=131072 ms_bidi=262144 ms_uni=262144 versions:chosen=0x00000001,negotiated=0x00000001 st=handshake mux=null expire=03s fd=-1 local_addr=128.140.63.137:443 foreign_addr=5.161.47.47:56773 [initl] rx.ackrng=1 tx.inflight=0 [hndshk] rx.ackrng=0 tx.inflight=14137 [01rtt] rx.ackrng=0 tx.inflight=0 srtt=274 rttvar=137 rttmin=274 ptoc=5 cwnd=12707 mcwnd=12707 sentpkts=15 lostpkts=0 I wanted to attempt inspecting the contents of a pcap capture in Wireshark, but with LibreSSL it isn't possible to use SSLKEYLOGFILE in curl and hence I can't inspect some parts of the packets. Does anybody have any clue on what to try or look at? TIA, Lucas OpenBSD 7.4-current (GENERIC.MP) #40: Wed Jan 10 02:01:40 MST 2024 dera...@arm64.openbsd.org:/usr/src/sys/arch/arm64/compile/GENERIC.MP real mem = 4185792512 (3991MB) avail mem = 3972042752 (3788MB) random: good seed from bootblocks mainbus0 at root: ACPI psci0 at mainbus0: PSCI 1.0, SMCCC 1.1 efi0 at mainbus0: UEFI 2.7 efi0: EDK II rev 0x10000 smbios0 at efi0: SMBIOS 3.0.0 smbios0: vendor Hetzner version "20171111" date 11/11/2017 smbios0: Hetzner vServer cpu0 at mainbus0 mpidr 0: ARM Neoverse N1 r3p1 cpu0: 64KB 64b/line 4-way L1 PIPT I-cache, 64KB 64b/line 4-way L1 D-cache cpu0: 1024KB 64b/line 8-way L2 cache cpu0: DP,RDM,Atomic,CRC32,SHA2,SHA1,AES+PMULL,LRCPC,DPB,ASID16,PAN+ATS1E1,LO,HPDS,VH,HAFDBS,CSV3,CSV2,SBSS+MSR cpu1 at mainbus0 mpidr 1: ARM Neoverse N1 r3p1 cpu1: 64KB 64b/line 4-way L1 PIPT I-cache, 64KB 64b/line 4-way L1 D-cache cpu1: 1024KB 64b/line 8-way L2 cache cpu1: DP,RDM,Atomic,CRC32,SHA2,SHA1,AES+PMULL,LRCPC,DPB,ASID16,PAN+ATS1E1,LO,HPDS,VH,HAFDBS,CSV3,CSV2,SBSS+MSR apm0 at mainbus0 agintc0 at mainbus0 shift 4:4 nirq 288 nredist 2 ipi: 0, 1, 2: "interrupt-controller" agintcmsi0 at agintc0 agtimer0 at mainbus0: 25000 kHz acpi0 at mainbus0: ACPI 5.1 acpi0: sleep states acpi0: tables DSDT FACP APIC GTDT MCFG SPCR DBG2 IORT BGRT acpi0: wakeup devices acpimcfg0 at acpi0 acpimcfg0: addr 0x4010000000, bus 0-255 acpiiort0 at acpi0 "ACPI0007" at acpi0 not configured "ACPI0007" at acpi0 not configured pluart0 at acpi0 COM0 addr 0x9000000/0x1000 irq 33 pluart0: console "LNRO0015" at acpi0 not configured "LNRO0015" at acpi0 not configured "QEMU0002" at acpi0 not configured "LNRO0005" at acpi0 not configured "LNRO0005" at acpi0 not configured "LNRO0005" at acpi0 not configured "LNRO0005" at acpi0 not configured "LNRO0005" at acpi0 not configured "LNRO0005" at acpi0 not configured "LNRO0005" at acpi0 not configured "LNRO0005" at acpi0 not configured "LNRO0005" at acpi0 not configured "LNRO0005" at acpi0 not configured "LNRO0005" at acpi0 not configured "LNRO0005" at acpi0 not configured "LNRO0005" at acpi0 not configured "LNRO0005" at acpi0 not configured "LNRO0005" at acpi0 not configured "LNRO0005" at acpi0 not configured "LNRO0005" at acpi0 not configured "LNRO0005" at acpi0 not configured "LNRO0005" at acpi0 not configured "LNRO0005" at acpi0 not configured "LNRO0005" at acpi0 not configured "LNRO0005" at acpi0 not configured "LNRO0005" at acpi0 not configured "LNRO0005" at acpi0 not configured "LNRO0005" at acpi0 not configured "LNRO0005" at acpi0 not configured "LNRO0005" at acpi0 not configured "LNRO0005" at acpi0 not configured "LNRO0005" at acpi0 not configured "LNRO0005" at acpi0 not configured "LNRO0005" at acpi0 not configured "LNRO0005" at acpi0 not configured acpipci0 at acpi0 PCI0 pci0 at acpipci0 0:4:0: io address conflict 0x8200/0x8 "Red Hat Host" rev 0x00 at pci0 dev 0 function 0 not configured virtio0 at pci0 dev 1 function 0 "Qumranet Virtio 1.x GPU" rev 0x01 viogpu0 at virtio0: 1024x768, 32bpp wsdisplay0 at viogpu0 mux 1: console (std, vt100 emulation) wsdisplay0: screen 1-5 added (std, vt100 emulation) virtio0: msix per-VQ ppb0 at pci0 dev 2 function 0 vendor "Red Hat", unknown product 0x000c rev 0x00: irq 37 pci1 at ppb0 bus 1 1:0:0: rom address conflict 0xfff80000/0x80000 virtio1 at pci1 dev 0 function 0 "Qumranet Virtio 1.x Network" rev 0x01 vio0 at virtio1: address 96:00:02:40:c5:c9 virtio1: msix shared ppb1 at pci0 dev 2 function 1 vendor "Red Hat", unknown product 0x000c rev 0x00: irq 37 pci2 at ppb1 bus 2 xhci0 at pci2 dev 0 function 0 vendor "Red Hat", unknown product 0x000d rev 0x01: msix, xHCI 0.0 usb0 at xhci0: USB revision 3.0 uhub0 at usb0 configuration 1 interface 0 "Red Hat xHCI root hub" rev 3.00/1.00 addr 1 ppb2 at pci0 dev 2 function 2 vendor "Red Hat", unknown product 0x000c rev 0x00: irq 37 pci3 at ppb2 bus 3 virtio2 at pci3 dev 0 function 0 "Qumranet Virtio 1.x Console" rev 0x01 virtio2: no matching child driver; not configured ppb3 at pci0 dev 2 function 3 vendor "Red Hat", unknown product 0x000c rev 0x00: irq 37 pci4 at ppb3 bus 4 virtio3 at pci4 dev 0 function 0 vendor "Qumranet", unknown product 0x1045 rev 0x01 viomb0 at virtio3 virtio3: irq 37 ppb4 at pci0 dev 2 function 4 vendor "Red Hat", unknown product 0x000c rev 0x00: irq 37 pci5 at ppb4 bus 5 virtio4 at pci5 dev 0 function 0 "Qumranet Virtio 1.x RNG" rev 0x01 viornd0 at virtio4 virtio4: irq 37 ppb5 at pci0 dev 2 function 5 vendor "Red Hat", unknown product 0x000c rev 0x00: irq 37 pci6 at ppb5 bus 6 virtio5 at pci6 dev 0 function 0 "Qumranet Virtio 1.x SCSI" rev 0x01 vioscsi0 at virtio5: qsize 128 scsibus0 at vioscsi0: 255 targets cd0 at scsibus0 targ 0 lun 0: <QEMU, QEMU CD-ROM, 2.5+> removable sd0 at scsibus0 targ 0 lun 2: <HC, Volume, 2.5+> serial.HC_Volume_100225372 sd0: 20480MB, 512 bytes/sector, 41943040 sectors, thin sd1 at scsibus0 targ 0 lun 1: <QEMU, QEMU HARDDISK, 2.5+> sd1: 39064MB, 512 bytes/sector, 80003072 sectors, thin virtio5: msix per-VQ ppb6 at pci0 dev 2 function 6 vendor "Red Hat", unknown product 0x000c rev 0x00: irq 37 pci7 at ppb6 bus 7 ppb7 at pci0 dev 2 function 7 vendor "Red Hat", unknown product 0x000c rev 0x00: irq 37 pci8 at ppb7 bus 8 ppb8 at pci0 dev 3 function 0 vendor "Red Hat", unknown product 0x000c rev 0x00: irq 38 pci9 at ppb8 bus 9 "Red Hat Qemu Serial" rev 0x01 at pci0 dev 4 function 0 not configured acpige0 at acpi0 irq 41 acpibtn0 at acpi0: PWRB uhidev0 at uhub0 port 5 configuration 1 interface 0 "QEMU QEMU USB Tablet" rev 2.00/0.00 addr 2 uhidev0: iclass 3/0 ums0 at uhidev0: 3 buttons, Z dir wsmouse0 at ums0 mux 0 uhidev1 at uhub0 port 6 configuration 1 interface 0 "QEMU QEMU USB Keyboard" rev 2.00/0.00 addr 3 uhidev1: iclass 3/1 ukbd0 at uhidev1: 8 variable keys, 6 key codes usbd_free_xfer: xfer=0xffffff800c3b8d20 not free usbd_free_xfer: xfer=0xffffff800c3b8d20 not free wskbd0 at ukbd0 mux 1 wskbd0: connecting to wsdisplay0 vscsi0 at root scsibus1 at vscsi0: 256 targets softraid0 at root scsibus2 at softraid0: 256 targets root on sd1a (e7fd1690e0003739.a) swap on sd1b dump on sd1b
