Reviving this old thread, lately I found out that the issue is only present in arm64 and not amd64. I also filled a bug in HAProxy repo, which contains relevant information.
https://github.com/haproxy/haproxy/issues/2569 Lucas On Mon, Jan 15, 2024 at 09:22:15PM GMT, Lucas Gabriel Vuotto wrote: > Hi ports@, > > did anybody succeed at serving HTTP/3 traffic with HAProxy? It should be > supported since 2.8, but I can't make it work: `curl --http3-only` gets > stuck and usually ends with > > curl: (55) ngtcp2_conn_writev_stream returned error: ERR_DRAINING > > It does work against https://http3.is, https://cloudflare.com and > others. > > I'm trying with the following config, which does work for HTTP/1.1 and > HTTP/2: > > > global > log 127.0.0.1 local0 debug > maxconn 1024 > chroot /var/haproxy > user _haproxy > group _haproxy > daemon > pidfile /var/run/haproxy.pid > > ssl-default-bind-options ssl-min-ver TLSv1.2 > ssl-load-extra-del-ext > > defaults > log global > mode http > option httplog > option dontlognull > option redispatch > retries 3 > maxconn 2000 > timeout connect 5s > timeout client 65s > timeout server 5s > > frontend haproxy > bind ipv4@:80,ipv6@:80 > bind ipv4@:443,ipv6@:443 ssl crt /etc/haproxy/certs/ > bind quic4@:443,quic6@:443 ssl crt /etc/haproxy/certs/ > > option forwardfor > > acl acme-challenge path_beg /.well-known/acme-challenge/ > acl ntfy req.hdr(host) -i ntfy.example.com > acl grafana req.hdr(host) -i grafana.example.com > > http-request redirect scheme https unless { ssl_fc } || acme-challenge > http-after-response add-header alt-svc 'h3=":443"; ma=900;' > > use_backend httpd if acme-challenge > use_backend ntfy_ws if ntfy { path_end /ws } > use_backend ntfy if ntfy > use_backend grafana if grafana > default_backend httpd > > backend httpd > server s1 127.0.0.1:8080 check > > backend ntfy_ws > option httpchk /v1/health > option http-server-close > timeout tunnel 10m > server s1 127.0.0.1:3010 check > > backend ntfy > option httpchk /v1/health > server s1 127.0.0.1:3010 check > > backend grafana > option httpchk /api/health > server s1 127.0.0.1:3000 check > > > Adding an alpn directive to bind lines makes no difference, and > according to the docs, the "normal" binds get an `alpn h2,http1.1` while > the quic binds get an `alpn h3` by default. > > tcpdump shows that there is some handshakes attempts between client and > server, and so does the stats socket of HAProxy: > > > > show quic full > * 0xd10d64000[00]: scid=4f5f572ad85655a9........................ > dcid=4559862ad37160765abf2b2082ad0e624fe59237 > loc. TPs: odcid=5cf7472f2d706253c37792abe48c49eea466ffd1 > iscid=4f5f572ad85655a9 > midle_timeout=30000ms mudp_payload_sz=2048 ack_delay_exp=3 > mack_delay=25ms act_cid_limit=8 > md=1687140 msd_bidi_l=16380 msd_bidi_r=16380 msd_uni=16380 ms_bidi=100 > ms_uni=3 > (no_act_migr,stless_rst_tok) > rem. TPs: iscid=4559862ad37160765abf2b2082ad0e624fe59237 > midle_timeout=120000ms mudp_payload_sz=65527 ack_delay_exp=3 > mack_delay=25ms act_cid_limit=2 > md=1310720 msd_bidi_l=131072 msd_bidi_r=131072 msd_uni=131072 > ms_bidi=262144 ms_uni=262144 > versions:chosen=0x00000001,negotiated=0x00000001 > st=handshake mux=null expire=24s > fd=-1 local_addr=128.140.63.137:443 > foreign_addr=5.161.47.47:56773 > [initl] rx.ackrng=1 tx.inflight=0 [hndshk] > rx.ackrng=0 tx.inflight=9877 > [01rtt] rx.ackrng=0 tx.inflight=0 > srtt=274 rttvar=137 rttmin=274 ptoc=3 cwnd=12707 mcwnd=12707 > sentpkts=11 lostpkts=0 > > > > show quic full > * 0xd10d64000[00]: scid=4f5f572ad85655a9........................ > dcid=4559862ad37160765abf2b2082ad0e624fe59237 > loc. TPs: odcid=5cf7472f2d706253c37792abe48c49eea466ffd1 > iscid=4f5f572ad85655a9 > midle_timeout=30000ms mudp_payload_sz=2048 ack_delay_exp=3 > mack_delay=25ms act_cid_limit=8 > md=1687140 msd_bidi_l=16380 msd_bidi_r=16380 msd_uni=16380 ms_bidi=100 > ms_uni=3 > (no_act_migr,stless_rst_tok) > rem. TPs: iscid=4559862ad37160765abf2b2082ad0e624fe59237 > midle_timeout=120000ms mudp_payload_sz=65527 ack_delay_exp=3 > mack_delay=25ms act_cid_limit=2 > md=1310720 msd_bidi_l=131072 msd_bidi_r=131072 msd_uni=131072 > ms_bidi=262144 ms_uni=262144 > versions:chosen=0x00000001,negotiated=0x00000001 > st=handshake mux=null expire=10s > fd=-1 local_addr=128.140.63.137:443 > foreign_addr=5.161.47.47:56773 > [initl] rx.ackrng=1 tx.inflight=0 [hndshk] > rx.ackrng=0 tx.inflight=14137 > [01rtt] rx.ackrng=0 tx.inflight=0 > srtt=274 rttvar=137 rttmin=274 ptoc=5 cwnd=12707 mcwnd=12707 > sentpkts=15 lostpkts=0 > > > > show quic full > * 0xd10d64000[00]: scid=4f5f572ad85655a9........................ > dcid=4559862ad37160765abf2b2082ad0e624fe59237 > loc. TPs: odcid=5cf7472f2d706253c37792abe48c49eea466ffd1 > iscid=4f5f572ad85655a9 > midle_timeout=30000ms mudp_payload_sz=2048 ack_delay_exp=3 > mack_delay=25ms act_cid_limit=8 > md=1687140 msd_bidi_l=16380 msd_bidi_r=16380 msd_uni=16380 ms_bidi=100 > ms_uni=3 > (no_act_migr,stless_rst_tok) > rem. TPs: iscid=4559862ad37160765abf2b2082ad0e624fe59237 > midle_timeout=120000ms mudp_payload_sz=65527 ack_delay_exp=3 > mack_delay=25ms act_cid_limit=2 > md=1310720 msd_bidi_l=131072 msd_bidi_r=131072 msd_uni=131072 > ms_bidi=262144 ms_uni=262144 > versions:chosen=0x00000001,negotiated=0x00000001 > st=handshake mux=null expire=03s > fd=-1 local_addr=128.140.63.137:443 > foreign_addr=5.161.47.47:56773 > [initl] rx.ackrng=1 tx.inflight=0 [hndshk] > rx.ackrng=0 tx.inflight=14137 > [01rtt] rx.ackrng=0 tx.inflight=0 > srtt=274 rttvar=137 rttmin=274 ptoc=5 cwnd=12707 mcwnd=12707 > sentpkts=15 lostpkts=0 > > > I wanted to attempt inspecting the contents of a pcap capture in > Wireshark, but with LibreSSL it isn't possible to use SSLKEYLOGFILE in > curl and hence I can't inspect some parts of the packets. > > Does anybody have any clue on what to try or look at? TIA, > > Lucas > > > OpenBSD 7.4-current (GENERIC.MP) #40: Wed Jan 10 02:01:40 MST 2024 > dera...@arm64.openbsd.org:/usr/src/sys/arch/arm64/compile/GENERIC.MP > real mem = 4185792512 (3991MB) > avail mem = 3972042752 (3788MB) > random: good seed from bootblocks > mainbus0 at root: ACPI > psci0 at mainbus0: PSCI 1.0, SMCCC 1.1 > efi0 at mainbus0: UEFI 2.7 > efi0: EDK II rev 0x10000 > smbios0 at efi0: SMBIOS 3.0.0 > smbios0: vendor Hetzner version "20171111" date 11/11/2017 > smbios0: Hetzner vServer > cpu0 at mainbus0 mpidr 0: ARM Neoverse N1 r3p1 > cpu0: 64KB 64b/line 4-way L1 PIPT I-cache, 64KB 64b/line 4-way L1 D-cache > cpu0: 1024KB 64b/line 8-way L2 cache > cpu0: > DP,RDM,Atomic,CRC32,SHA2,SHA1,AES+PMULL,LRCPC,DPB,ASID16,PAN+ATS1E1,LO,HPDS,VH,HAFDBS,CSV3,CSV2,SBSS+MSR > cpu1 at mainbus0 mpidr 1: ARM Neoverse N1 r3p1 > cpu1: 64KB 64b/line 4-way L1 PIPT I-cache, 64KB 64b/line 4-way L1 D-cache > cpu1: 1024KB 64b/line 8-way L2 cache > cpu1: > DP,RDM,Atomic,CRC32,SHA2,SHA1,AES+PMULL,LRCPC,DPB,ASID16,PAN+ATS1E1,LO,HPDS,VH,HAFDBS,CSV3,CSV2,SBSS+MSR > apm0 at mainbus0 > agintc0 at mainbus0 shift 4:4 nirq 288 nredist 2 ipi: 0, 1, 2: > "interrupt-controller" > agintcmsi0 at agintc0 > agtimer0 at mainbus0: 25000 kHz > acpi0 at mainbus0: ACPI 5.1 > acpi0: sleep states > acpi0: tables DSDT FACP APIC GTDT MCFG SPCR DBG2 IORT BGRT > acpi0: wakeup devices > acpimcfg0 at acpi0 > acpimcfg0: addr 0x4010000000, bus 0-255 > acpiiort0 at acpi0 > "ACPI0007" at acpi0 not configured > "ACPI0007" at acpi0 not configured > pluart0 at acpi0 COM0 addr 0x9000000/0x1000 irq 33 > pluart0: console > "LNRO0015" at acpi0 not configured > "LNRO0015" at acpi0 not configured > "QEMU0002" at acpi0 not configured > "LNRO0005" at acpi0 not configured > "LNRO0005" at acpi0 not configured > "LNRO0005" at acpi0 not configured > "LNRO0005" at acpi0 not configured > "LNRO0005" at acpi0 not configured > "LNRO0005" at acpi0 not configured > "LNRO0005" at acpi0 not configured > "LNRO0005" at acpi0 not configured > "LNRO0005" at acpi0 not configured > "LNRO0005" at acpi0 not configured > "LNRO0005" at acpi0 not configured > "LNRO0005" at acpi0 not configured > "LNRO0005" at acpi0 not configured > "LNRO0005" at acpi0 not configured > "LNRO0005" at acpi0 not configured > "LNRO0005" at acpi0 not configured > "LNRO0005" at acpi0 not configured > "LNRO0005" at acpi0 not configured > "LNRO0005" at acpi0 not configured > "LNRO0005" at acpi0 not configured > "LNRO0005" at acpi0 not configured > "LNRO0005" at acpi0 not configured > "LNRO0005" at acpi0 not configured > "LNRO0005" at acpi0 not configured > "LNRO0005" at acpi0 not configured > "LNRO0005" at acpi0 not configured > "LNRO0005" at acpi0 not configured > "LNRO0005" at acpi0 not configured > "LNRO0005" at acpi0 not configured > "LNRO0005" at acpi0 not configured > "LNRO0005" at acpi0 not configured > "LNRO0005" at acpi0 not configured > acpipci0 at acpi0 PCI0 > pci0 at acpipci0 > 0:4:0: io address conflict 0x8200/0x8 > "Red Hat Host" rev 0x00 at pci0 dev 0 function 0 not configured > virtio0 at pci0 dev 1 function 0 "Qumranet Virtio 1.x GPU" rev 0x01 > viogpu0 at virtio0: 1024x768, 32bpp > wsdisplay0 at viogpu0 mux 1: console (std, vt100 emulation) > wsdisplay0: screen 1-5 added (std, vt100 emulation) > virtio0: msix per-VQ > ppb0 at pci0 dev 2 function 0 vendor "Red Hat", unknown product 0x000c rev > 0x00: irq 37 > pci1 at ppb0 bus 1 > 1:0:0: rom address conflict 0xfff80000/0x80000 > virtio1 at pci1 dev 0 function 0 "Qumranet Virtio 1.x Network" rev 0x01 > vio0 at virtio1: address 96:00:02:40:c5:c9 > virtio1: msix shared > ppb1 at pci0 dev 2 function 1 vendor "Red Hat", unknown product 0x000c rev > 0x00: irq 37 > pci2 at ppb1 bus 2 > xhci0 at pci2 dev 0 function 0 vendor "Red Hat", unknown product 0x000d rev > 0x01: msix, xHCI 0.0 > usb0 at xhci0: USB revision 3.0 > uhub0 at usb0 configuration 1 interface 0 "Red Hat xHCI root hub" rev > 3.00/1.00 addr 1 > ppb2 at pci0 dev 2 function 2 vendor "Red Hat", unknown product 0x000c rev > 0x00: irq 37 > pci3 at ppb2 bus 3 > virtio2 at pci3 dev 0 function 0 "Qumranet Virtio 1.x Console" rev 0x01 > virtio2: no matching child driver; not configured > ppb3 at pci0 dev 2 function 3 vendor "Red Hat", unknown product 0x000c rev > 0x00: irq 37 > pci4 at ppb3 bus 4 > virtio3 at pci4 dev 0 function 0 vendor "Qumranet", unknown product 0x1045 > rev 0x01 > viomb0 at virtio3 > virtio3: irq 37 > ppb4 at pci0 dev 2 function 4 vendor "Red Hat", unknown product 0x000c rev > 0x00: irq 37 > pci5 at ppb4 bus 5 > virtio4 at pci5 dev 0 function 0 "Qumranet Virtio 1.x RNG" rev 0x01 > viornd0 at virtio4 > virtio4: irq 37 > ppb5 at pci0 dev 2 function 5 vendor "Red Hat", unknown product 0x000c rev > 0x00: irq 37 > pci6 at ppb5 bus 6 > virtio5 at pci6 dev 0 function 0 "Qumranet Virtio 1.x SCSI" rev 0x01 > vioscsi0 at virtio5: qsize 128 > scsibus0 at vioscsi0: 255 targets > cd0 at scsibus0 targ 0 lun 0: <QEMU, QEMU CD-ROM, 2.5+> removable > sd0 at scsibus0 targ 0 lun 2: <HC, Volume, 2.5+> serial.HC_Volume_100225372 > sd0: 20480MB, 512 bytes/sector, 41943040 sectors, thin > sd1 at scsibus0 targ 0 lun 1: <QEMU, QEMU HARDDISK, 2.5+> > sd1: 39064MB, 512 bytes/sector, 80003072 sectors, thin > virtio5: msix per-VQ > ppb6 at pci0 dev 2 function 6 vendor "Red Hat", unknown product 0x000c rev > 0x00: irq 37 > pci7 at ppb6 bus 7 > ppb7 at pci0 dev 2 function 7 vendor "Red Hat", unknown product 0x000c rev > 0x00: irq 37 > pci8 at ppb7 bus 8 > ppb8 at pci0 dev 3 function 0 vendor "Red Hat", unknown product 0x000c rev > 0x00: irq 38 > pci9 at ppb8 bus 9 > "Red Hat Qemu Serial" rev 0x01 at pci0 dev 4 function 0 not configured > acpige0 at acpi0 irq 41 > acpibtn0 at acpi0: PWRB > uhidev0 at uhub0 port 5 configuration 1 interface 0 "QEMU QEMU USB Tablet" > rev 2.00/0.00 addr 2 > uhidev0: iclass 3/0 > ums0 at uhidev0: 3 buttons, Z dir > wsmouse0 at ums0 mux 0 > uhidev1 at uhub0 port 6 configuration 1 interface 0 "QEMU QEMU USB Keyboard" > rev 2.00/0.00 addr 3 > uhidev1: iclass 3/1 > ukbd0 at uhidev1: 8 variable keys, 6 key codes > usbd_free_xfer: xfer=0xffffff800c3b8d20 not free > usbd_free_xfer: xfer=0xffffff800c3b8d20 not free > wskbd0 at ukbd0 mux 1 > wskbd0: connecting to wsdisplay0 > vscsi0 at root > scsibus1 at vscsi0: 256 targets > softraid0 at root > scsibus2 at softraid0: 256 targets > root on sd1a (e7fd1690e0003739.a) swap on sd1b dump on sd1b >
