Re: Add Intel Wi-Fi 6 AX211 0x51f1 device

[Reyk Floeter]

On Tue, Jun 27, 2023 at 11:55:15AM +0200, Stefan Sperling wrote:
> On Tue, Jun 27, 2023 at 10:55:04AM +0200, Reyk Floeter wrote:
> > Hi,
> > 
> > the following diff adds the Intel Wi-Fi 6 0x51f1 device, found in a
> > new Thinkpad X1 Carbon Gen 11.
> > 
> > iwx0 at pci0 dev 20 function 3 "Intel Wi-Fi 6 AX211" rev 0x01, msix
> > iwx0: hw rev 0x370, fw 77.f92b5fed.0, pnvm e4a49534, address ...
> > 
> > I looked at the Linux code for the values for 0x51f1 and 0x7a70:
> > https://github.com/torvalds/linux/commit/1c4db7613f35b248ff05b8bfb3a4580a3d11d75c
> 
> Thanks, your diff looks correct to me. ok stsp@
> 

Cool, thanks.

> > This works fine with my 11ac setup and performance seems OK (around
> > 200Mbps - what can I expect on OpenBSD these days?).  I don't know if
> > the device supports UHB channels.
> 
> This speed matches what I see in my environment, more or less.
> Apparently it goes up to 300 Mbit/s for some people, but I have never
> seen it go that far up myself.
> 

I tested it with tcpbench again and I get 210Mbps max locally when I
stand right next to the AP.

> We are capped to 80 MHz channels at the moment. I do have a 160MHz AP now
> but would need to find spare time to make that work (as things look right
> now, me finding time for this soon is rather unlikely).
> 
> And the size we advertise for aggregates on Rx in VHT capabilities is
> the smallest possible value we can advertise in 11ac (3895 octets).
> Increasing this frame size limit should result in a performance boost.
> We might have to increase the size of our Rx buffers accordingly, unless
> the hardware handles de-aggregation into a cluster of our 4k-sized Rx
> buffers, which may well be the case but I don't know for sure.
> It would not be an entirely trivial diff, but likely less coding work
> and more widely applicable than 160 MHz channel support.

I recently switched to UniFi and I regret it a bit. I remember that my
older Aruba setup was faster, even on Mac.  But maybe UniFi is doing
something differently and it's primarily a compatibility issue.

Reyk

Add Intel Wi-Fi 6 AX211 0x51f1 device

[Reyk Floeter]

Hi,

the following diff adds the Intel Wi-Fi 6 0x51f1 device, found in a
new Thinkpad X1 Carbon Gen 11.

iwx0 at pci0 dev 20 function 3 "Intel Wi-Fi 6 AX211" rev 0x01, msix
iwx0: hw rev 0x370, fw 77.f92b5fed.0, pnvm e4a49534, address ...

I looked at the Linux code for the values for 0x51f1 and 0x7a70:
https://github.com/torvalds/linux/commit/1c4db7613f35b248ff05b8bfb3a4580a3d11d75c

This works fine with my 11ac setup and performance seems OK (around
200Mbps - what can I expect on OpenBSD these days?).  I don't know if
the device supports UHB channels.

Reyk

Index: sys/dev/pci/if_iwx.c
===
RCS file: /cvs/src/sys/dev/pci/if_iwx.c,v
retrieving revision 1.172
diff -u -p -u -p -r1.172 if_iwx.c
--- sys/dev/pci/if_iwx.c21 Jun 2023 23:24:10 -  1.172
+++ sys/dev/pci/if_iwx.c27 Jun 2023 08:32:09 -
@@ -10430,6 +10430,7 @@ static const struct pci_matchid iwx_devi
/* _14 is an MA device, not yet supported */
{ PCI_VENDOR_INTEL, PCI_PRODUCT_INTEL_WL_22500_15,},
{ PCI_VENDOR_INTEL, PCI_PRODUCT_INTEL_WL_22500_16,},
+   { PCI_VENDOR_INTEL, PCI_PRODUCT_INTEL_WL_22500_17,},
 };
 
 
@@ -10520,6 +10521,8 @@ static const struct iwx_dev_info iwx_dev
IWX_DEV_INFO(0x2725, 0x1674, iwx_2ax_cfg_ty_gf_a0), /* killer_1675x */
IWX_DEV_INFO(0x51f0, 0x1691, iwx_2ax_cfg_so_gf4_a0), /* killer_1690s */
IWX_DEV_INFO(0x51f0, 0x1692, iwx_2ax_cfg_so_gf4_a0), /* killer_1690i */
+   IWX_DEV_INFO(0x51f1, 0x1691, iwx_2ax_cfg_so_gf4_a0),
+   IWX_DEV_INFO(0x51f1, 0x1692, iwx_2ax_cfg_so_gf4_a0),
IWX_DEV_INFO(0x54f0, 0x1691, iwx_2ax_cfg_so_gf4_a0), /* killer_1690s */
IWX_DEV_INFO(0x54f0, 0x1692, iwx_2ax_cfg_so_gf4_a0), /* killer_1690i */
IWX_DEV_INFO(0x7a70, 0x0090, iwx_2ax_cfg_so_gf_a0_long),
@@ -11122,7 +11125,6 @@ iwx_attach(struct device *parent, struct
case PCI_PRODUCT_INTEL_WL_22500_9:
case PCI_PRODUCT_INTEL_WL_22500_10:
case PCI_PRODUCT_INTEL_WL_22500_11:
-   case PCI_PRODUCT_INTEL_WL_22500_12:
case PCI_PRODUCT_INTEL_WL_22500_13:
/* _14 is an MA device, not yet supported */
case PCI_PRODUCT_INTEL_WL_22500_15:
@@ -11136,6 +11138,19 @@ iwx_attach(struct device *parent, struct
sc->sc_xtal_latency = 0;
sc->sc_tx_with_siso_diversity = 0;
sc->sc_uhb_supported = 1;
+   break;
+   case PCI_PRODUCT_INTEL_WL_22500_12:
+   case PCI_PRODUCT_INTEL_WL_22500_17:
+   sc->sc_fwname = IWX_SO_A_GF_A_FW;
+   sc->sc_pnvm_name = IWX_SO_A_GF_A_PNVM;
+   sc->sc_device_family = IWX_DEVICE_FAMILY_AX210;
+   sc->sc_integrated = 1;
+   sc->sc_ltr_delay = IWX_SOC_FLAGS_LTR_APPLY_DELAY_2500;
+   sc->sc_low_latency_xtal = 1;
+   sc->sc_xtal_latency = 12000;
+   sc->sc_tx_with_siso_diversity = 0;
+   sc->sc_uhb_supported = 0;
+   sc->sc_imr_enabled = 1;
break;
default:
printf("%s: unknown adapter type\n", DEVNAME(sc));
Index: sys/dev/pci/pcidevs
===
RCS file: /cvs/src/sys/dev/pci/pcidevs,v
retrieving revision 1.2039
diff -u -p -u -p -r1.2039 pcidevs
--- sys/dev/pci/pcidevs 13 Jun 2023 02:43:39 -  1.2039
+++ sys/dev/pci/pcidevs 27 Jun 2023 08:32:09 -
@@ -5875,6 +5875,7 @@ product INTEL 600SERIES_LP_XHCI   0x51ed  6
 product INTEL 600SERIES_LP_XDCI0x51ee  600 Series xDCI
 product INTEL 600SERIES_LP_SRAM0x51ef  600 Series SRAM
 product INTEL WL_22500_11  0x51f0  Wi-Fi 6 AX211
+product INTEL WL_22500_17  0x51f1  Wi-Fi 6 AX211
 product INTEL 600SERIES_LP_GSPI_2  0x51fb  600 Series GSPI
 product INTEL 600SERIES_LP_ISH 0x51fc  600 Series ISH
 product INTEL 600SERIES_LP_UFS 0x51ff  600 Series UFS
Index: sys/dev/pci/pcidevs.h
===
RCS file: /cvs/src/sys/dev/pci/pcidevs.h,v
retrieving revision 1.2033
diff -u -p -u -p -r1.2033 pcidevs.h
--- sys/dev/pci/pcidevs.h   13 Jun 2023 02:44:08 -  1.2033
+++ sys/dev/pci/pcidevs.h   27 Jun 2023 08:32:09 -
@@ -5880,6 +5880,7 @@
 #definePCI_PRODUCT_INTEL_600SERIES_LP_XDCI 0x51ee  /* 600 
Series xDCI */
 #definePCI_PRODUCT_INTEL_600SERIES_LP_SRAM 0x51ef  /* 600 
Series SRAM */
 #definePCI_PRODUCT_INTEL_WL_22500_11   0x51f0  /* Wi-Fi 6 
AX211 */
+#definePCI_PRODUCT_INTEL_WL_22500_17   0x51f1  /* Wi-Fi 6 
AX211 */
 #definePCI_PRODUCT_INTEL_600SERIES_LP_GSPI_2   0x51fb  /* 600 
Series GSPI */
 #definePCI_PRODUCT_INTEL_600SERIES_LP_ISH  0x51fc  /* 600 
Series ISH */
 #definePCI_PRODUCT_INTEL_600SERIES_LP_UFS  0x51ff  /* 600 
Series UFS */
Index: sys/dev/pci/pcidevs_data.h
==

Re: more relayd cleanup

[Reyk Floeter]

> On 20 Jun 2023, at 18:16, Claudio Jeker  wrote:
> 
> On Tue, Jun 20, 2023 at 03:35:11PM +0200, Theo Buehler wrote:
>>> On Tue, Jun 20, 2023 at 02:17:06PM +0200, Claudio Jeker wrote:
>>> Ok, this went overboard. I just wanted to clean up a bit more in
>>> check_tcp.c but noticed check_send_expect and CHECK_BINSEND_EXPECT.
>>> 
>>> This code is not very consitent in the differnt ways the strings are
>>> encoded. Especially check_send_expect() is a bit of a mess because of
>>> that.
>>> 
>>> While there I noticed string2binary() and decided to write it in simpler
>>> way (copying code over from rpki-client).
>>> 
>>> All in all I think this diff is improving the situation a little bit.
>> 
>> Write "I will not modify Reyk's daemons more than I need to" 100 times.
>> 
>> Then it's ok.
>> 
> 
> I will not modify Reyk's daemons more than I need to
> I will not modify Reyk's daemons more than I need to
> I will not modify Reyk's daemons more than I need to

say my name three times …

> I will not modify Reyk's daemons more than I need to
> I will not modify Reyk's daemons more than I need to
> I will not modify Reyk's daemons more than I need to
> I will not modify Reyk's daemons more than I need to
> I will not modify Reyk's daemons more than I need to
> I will not modify Reyk's daemons more than I need to
> I will not modify Reyk's daemons more than I need to
> I will not modify Reyk's daemons more than I need to
> I will not modify Reyk's daemons more than I need to
> I will not modify Reyk's daemons more than I need to
> I will not modify Reyk's daemons more than I need to
> I will not modify Reyk's daemons more than I need to
> I will not modify Reyk's daemons more than I need to
> I will not modify Reyk's daemons more than I need to
> I will not modify Reyk's daemons more than I need to
> I will not modify Reyk's daemons more than I need to
> I will not modify Reyk's daemons more than I need to
> I will not modify Reyk's daemons more than I need to
> I will not modify Reyk's daemons more than I need to
> I will not modify Reyk's daemons more than I need to
> I will not modify Reyk's daemons more than I need to
> I will not modify Reyk's daemons more than I need to
> I will not modify Reyk's daemons more than I need to
> I will not modify Reyk's daemons more than I need to
> I will not modify Reyk's daemons more than I need to
> I will not modify Reyk's daemons more than I need to
> I will not modify Reyk's daemons more than I need to
> I will not modify Reyk's daemons more than I need to
> I will not modify Reyk's daemons more than I need to
> I will not modify Reyk's daemons more than I need to
> I will not modify Reyk's daemons more than I need to
> I will not modify Reyk's daemons more than I need to
> I will not modify Reyk's daemons more than I need to
> I will not modify Reyk's daemons more than I need to
> I will not modify Reyk's daemons more than I need to
> I will not modify Reyk's daemons more than I need to
> I will not modify Reyk's daemons more than I need to
> I will not modify Reyk's daemons more than I need to
> I will not modify Reyk's daemons more than I need to
> I will not modify Reyk's daemons more than I need to
> I will not modify Reyk's daemons more than I need to
> I will not modify Reyk's daemons more than I need to
> I will not modify Reyk's daemons more than I need to
> I will not modify Reyk's daemons more than I need to
> I will not modify Reyk's daemons more than I need to
> I will not modify Reyk's daemons more than I need to
> I will not modify Reyk's daemons more than I need to
> I will not modify Reyk's daemons more than I need to
> I will not modify Reyk's daemons more than I need to
> I will not modify Reyk's daemons more than I need to
> I will not modify Reyk's daemons more than I need to
> I will not modify Reyk's daemons more than I need to
> I will not modify Reyk's daemons more than I need to
> I will not modify Reyk's daemons more than I need to
> I will not modify Reyk's daemons more than I need to
> I will not modify Reyk's daemons more than I need to
> I will not modify Reyk's daemons more than I need to
> I will not modify Reyk's daemons more than I need to
> I will not modify Reyk's daemons more than I need to
> I will not modify Reyk's daemons more than I need to
> I will not modify Reyk's daemons more than I need to
> I will not modify Reyk's daemons more than I need to
> I will not modify Reyk's daemons more than I need to
> I will not modify Reyk's daemons more than I need to
> I will not modify Reyk's daemons more than I need to
> I will not modify Reyk's daemons more than I need to
> I will not modify Reyk's daemons more than I need to
> I will not modify Reyk's daemons more than I need to
> I will not modify Reyk's daemons more than I need to
> I will not modify Reyk's daemons more than I need to
> I will not modify Reyk's daemons more than I need to
> I will not modify Reyk's daemons more than I need to
> I will not modify Reyk's daemons more th

Re: Remove booting from kernels in raw/qcow2 images in vmd(8)

[Reyk Floeter]

goto out;
>> -
>> -twiddle();
>> -rc = (f->f_dev->dv_strategy)(f->f_devdata,
>> -F_READ, fsbtodb(fs, disk_block),
>> -fs->fs_bsize, buf, &buf_size);
>> -if (rc)
>> -goto out;
>> -
>> -bcopy(buf, namebuf, link_len);
>> -}
>> -
>> -/*
>> - * If relative pathname, restart at parent directory.
>> - * If absolute pathname, restart at root.
>> - */
>> -cp = namebuf;
>> -if (*cp != '/')
>> -inumber = parent_inumber;
>> -else
>> -inumber = ROOTINO;
>> -
>> -if ((rc = read_inode(inumber, f)) != 0)
>> -goto out;
>> -}
>> -}
>> -
>> -/*
>> - * Found terminal component.
>> - */
>> -rc = 0;
>> -out:
>> -if (buf)
>> -free(buf, fs->fs_bsize);
>> -if (rc)
>> -(void)ufs_close_internal(fp);
>> -
>> -return (rc);
>> -}
>> -
>> -int
>> -ufs_close(struct open_file *f)
>> -{
>> -struct file *fp = (struct file *)f->f_fsdata;
>> -
>> -f->f_fsdata = NULL;
>> -if (fp == NULL)
>> -return (0);
>> -
>> -return (ufs_close_internal(fp));
>> -}
>> -
>> -static int
>> -ufs_close_internal(struct file *fp)
>> -{
>> -int level;
>> -
>> -for (level = 0; level < NIADDR; level++) {
>> -if (fp->f_blk[level])
>> -free(fp->f_blk[level], fp->f_fs->fs_bsize);
>> -}
>> -if (fp->f_buf)
>> -free(fp->f_buf, fp->f_fs->fs_bsize);
>> -free(fp->f_fs, SBSIZE);
>> -free(fp, sizeof(struct file));
>> -return (0);
>> -}
>> -
>> -/*
>> - * Copy a portion of a file into kernel memory.
>> - * Cross block boundaries when necessary.
>> - */
>> -int
>> -ufs_read(struct open_file *f, void *start, size_t size, size_t *resid)
>> -{
>> -struct file *fp = (struct file *)f->f_fsdata;
>> -char *buf, *addr = start;
>> -size_t csize, buf_size;
>> -int rc = 0;
>> -
>> -while (size != 0) {
>> -if ((u_int64_t)fp->f_seekp >= fp->f_di.di_size)
>> -break;
>> -
>> -rc = buf_read_file(f, &buf, &buf_size);
>> -if (rc)
>> -break;
>> -
>> -csize = size;
>> -if (csize > buf_size)
>> -csize = buf_size;
>> -
>> -bcopy(buf, addr, csize);
>> -
>> -fp->f_seekp += csize;
>> -addr += csize;
>> -    size -= csize;
>> -}
>> -if (resid)
>> -*resid = size;
>> -return (rc);
>> -}
>> -
>> -/*
>> - * Not implemented.
>> - */
>> -int
>> -ufs_write(struct open_file *f, void *start, size_t size, size_t *resid)
>> -{
>> -
>> -return (EROFS);
>> -}
>> -
>> -off_t
>> -ufs_seek(struct open_file *f, off_t offset, int where)
>> -{
>> -struct file *fp = (struct file *)f->f_fsdata;
>> -
>> -switch (where) {
>> -case SEEK_SET:
>> -fp->f_seekp = offset;
>> -break;
>> -case SEEK_CUR:
>> -fp->f_seekp += offset;
>> -break;
>> -case SEEK_END:
>> -fp->f_seekp = fp->f_di.di_size - offset;
>> -break;
>> -default:
>> -return (-1);
>> -}
>> -return (fp->f_seekp);
>> -}
>> -
>> -int
>> -ufs_stat(struct open_file *f, struct stat *sb)
>> -{
>> -struct file *fp = (struct file *)f->f_fsdata;
>> -
>> -/* only important stuff */
>> -sb->st_mode = fp->f_di.di_mode;
>> -sb->st_uid = fp->f_di.di_uid;
>> -sb->st_gid = fp->f_di.di_gid;
>> -sb->st_size = fp->f_di.di_size;
>> -return (0);
>> -}
>> -
>> -#ifndefNO_READDIR
>> -int
>> -ufs_readdir(struct open_file *f, char *name)
>> -{
>> -struct file *fp = (struct file *)f->f_fsdata;
>> -struct direct *dp, *edp;
>> -size_t buf_size;
>> -int rc, namlen;
>> -char *buf;
>> -
>> -if (name == NULL)
>> -fp->f_seek

Re: snmpd(8) remove OID CONFIGURATION

[Reyk Floeter]

> Am 27.02.2020 um 08:56 schrieb Martijn van Duren 
> :
> 
> On 2/12/20 7:48 AM, Martijn van Duren wrote:
>> Hello tech@,
>> 
>> Working on something else, this bit of code is somewhat in my way and it
>> feels like an early testing feature instead of actually being useful.
>> 

It wasn’t an early testing feature.

>> Is anyone actually using this, or can it be removed?
>> 

I was using it to set device-specific information besides the system MIB.

Enterprise NMS query weird custom values and sometimes this was the only option 
to integrate there (I think I used it specifically for the ProCurve Manager, a 
long time ago).

Reyk

>> martijn@
>> 
> One reply so far saying that it's not used in their setup.
> 
> Unless someone complains I'll probably commit the diff below somewhere
> early next week. Diff below is previous with additional manpage and
> examples cleanup.
> 
> Explicit OKs also welcome.
> 
> martijn@
> 
> Index: etc/examples/snmpd.conf
> ===
> RCS file: /cvs/src/etc/examples/snmpd.conf,v
> retrieving revision 1.1
> diff -u -p -r1.1 snmpd.conf
> --- etc/examples/snmpd.conf11 Jul 2014 21:20:10 -1.1
> +++ etc/examples/snmpd.conf27 Feb 2020 07:54:28 -
> @@ -14,10 +14,6 @@ listen on $listen_addr
> #system location "Rack A1-24, Room 13"
> system services 74
> 
> -# Provide static user-defined SNMP OIDs
> -oid 1.3.6.1.4.1.30155.42.3.1 name testStringValue read-only string "Test"
> -oid 1.3.6.1.4.1.30155.42.3.4 name testIntValue read-write integer 1
> -
> # Enable SNMPv3 USM with authentication, encryption and two defined users
> #seclevel enc
> #user "user1" authkey "password123" enc aes enckey "321drowssap"
> Index: usr.sbin/snmpd/mps.c
> ===
> RCS file: /cvs/src/usr.sbin/snmpd/mps.c,v
> retrieving revision 1.28
> diff -u -p -r1.28 mps.c
> --- usr.sbin/snmpd/mps.c24 Oct 2019 12:39:27 -1.28
> +++ usr.sbin/snmpd/mps.c27 Feb 2020 07:54:28 -
> @@ -51,17 +51,6 @@ struct ber_oid *
> extern void control_event_add(struct ctl_conn *, int, int, struct timeval *); 
> /* XXX */
> 
> int
> -mps_getstr(struct oid *oid, struct ber_oid *o, struct ber_element **elm)
> -{
> -char*s = oid->o_data;
> -
> -if (s == NULL)
> -return (-1);
> -*elm = ober_add_string(*elm, s);
> -return (0);
> -}
> -
> -int
> mps_setstr(struct oid *oid, struct ber_oid *o, struct ber_element **elm)
> {
>struct ber_element*ber = *elm;
> @@ -88,18 +77,6 @@ int
> mps_getint(struct oid *oid, struct ber_oid *o, struct ber_element **elm)
> {
>*elm = ober_add_integer(*elm, oid->o_val);
> -return (0);
> -}
> -
> -int
> -mps_setint(struct oid *oid, struct ber_oid *o, struct ber_element **elm)
> -{
> -long long i;
> -
> -if (ober_get_integer(*elm, &i) == -1)
> -return (-1);
> -oid->o_val = i;
> -
>return (0);
> }
> 
> Index: usr.sbin/snmpd/parse.y
> ===
> RCS file: /cvs/src/usr.sbin/snmpd/parse.y,v
> retrieving revision 1.57
> diff -u -p -r1.57 parse.y
> --- usr.sbin/snmpd/parse.y2 Jan 2020 10:55:53 -1.57
> +++ usr.sbin/snmpd/parse.y27 Feb 2020 07:54:28 -
> @@ -138,8 +138,8 @@ typedef struct {
> %token  NUMBER
> %typehostcmn
> %typesrcaddr
> -%typeoptwrite yesno seclevel socktype proto
> -%typeobjtype cmd
> +%typeyesno seclevel socktype proto
> +%typecmd
> %typeoid hostoid trapoid
> %typeauth
> %typeenc
> @@ -152,7 +152,6 @@ grammar: /* empty */
>| grammar varset '\n'
>| grammar main '\n'
>| grammar system '\n'
> -| grammar mib '\n'
>| grammar error '\n'{ file->errors++; }
>;
> 
> @@ -349,61 +348,6 @@ sysmib: CONTACT STRING{
>struct ber_oid o = OID(MIB_sysServices);
>mps_set(&o, NULL, $2);
>}
> -;
> -
> -mib: OBJECTID oid NAME STRING optwrite objtype{
> -struct oid*oid;
> -if ((oid = (struct oid *)
> -calloc(1, sizeof(*oid))) == NULL) {
> -yyerror("calloc");
> -free($2);
> -free($6.data);
> -YYERROR;
> -}
> -
> -smi_oidlen($2);
> -bcopy($2, &oid->o_id, sizeof(struct ber_oid));
> -free($2);
> -oid->o_name = $4;
> -oid->o_data = $6.data;
> -oid->o_val = $6.value;
> -switch ($6.type) {
> -case 1:
> -oid->o_get = mps_getint;
> -oid->o_set = mps_setint;
> -break;
> -case 2:
> -oid->o_get = mps_getstr;
> -oid->o_set = mps_setstr;
> -break;
> -}
> -oid->o_flags = OID_RD|OID_

Re: usbhidaction(1) with a foot pedal/control

[Reyk Floeter]

Hi,

On Tue, Dec 17, 2019 at 06:57:54PM +, Raf Czlonka wrote:
> I use a Philips USB foot pedal[0] as an additional input device.
> With usbhidaction(1), I can "program" its four "buttons".
> 
> After recent changes to uhid(4) device nodes' permissions, my USB
> foot pedal "stopped working".
> 
> I understand that it is a *very recent* change but I rely on this
> input device for my day-to-day computer usage and have a couple of
> questions. I understand that the below permissions are final(?).
> 
>   crw---  1 root  wheel   62,   0 Dec 16 11:17 /dev/uhid0
>   crw---  1 root  wheel   62,   1 Dec 16 11:17 /dev/uhid1
>   crw---  1 root  wheel   62,   2 Dec 16 11:17 /dev/uhid2
>   crw---  1 root  wheel   62,   3 Dec 16 11:17 /dev/uhid3
>   crw---  1 root  wheel   62,   4 Dec 16 11:17 /dev/uhid4
>   crw---  1 root  wheel   62,   5 Dec 16 11:17 /dev/uhid5
>   crw---  1 root  wheel   62,   6 Dec 16 11:17 /dev/uhid6
>   crw---  1 root  wheel   62,   7 Dec 16 11:17 /dev/uhid7
> 
> I use(d) the below script (executed from .xsession) in order to
> configure and use the foot pedal:
> 
>   #!/bin/sh
>   test -r $HOME/.footpedal.conf && {
>   pgrep usbhidaction > /dev/null ||
>   usbhidaction -f $(dmesg | awk '/Footcontrol USB/ {
>   getline
>   getline
>   print $1
>   }' | tail -n 1) -c $HOME/.footpedal.conf -i
>   }
> 
>   $ cat $HOME/.footpedal.conf
>   Generic_Desktop:Joystick.Generic_Desktop:Pointer.Button:Button_1
> 1
>   action1
>   Generic_Desktop:Joystick.Generic_Desktop:Pointer.Button:Button_2
> 1
>   action2
>   Generic_Desktop:Joystick.Generic_Desktop:Pointer.Button:Button_3
> 1
>   action3
>   Generic_Desktop:Joystick.Generic_Desktop:Pointer.Button:Button_4
> 1
>   action4
> 
> 1. Shall I *simply* introduce chmod(1)/chown(8) step in the above script?
> 

Yes, this is what I recommend.  The good thing is that your script
already knows the exact uhid device of your footpedal, so doing a
chmod/chown for this will not automatically open all other HID
devices.

> 2. Is the situation still more of a "watch this space"?
> 

Maybe.  It is difficult to find a solution for such simple and generic
devices, but we generally recommend to write drivers instead of
relying on uhid(4) or ugen(4).

> 3. Is there any other way to get the uhid(4) device node (-f option) for
>a particular USB device, other than the above dmesg(8)|awk(1) hack?
> 

All uhid(4) devices support the USB_DEVICEINFO ioctl which returns the
vendor/product as strings and Ids.  This is what most HID code does at
the moment: scan all /dev/uhid* devices and query either
USB_DEVICEINFO or the HID class (which is even more complicated).

But you probably don't want to do ioctls from a script (doing this
from perl or python is just ugly), don't you?  So you could eventually
parse output from usbdevs or the report/output from usbhidctl.

> 4. Or perhaps I'm doing it all wrong?
> 

No, you're not doing it wrong.  There is no better way to use your pedal.

Reyk

> Relevant bits:
> 
>   $ dmesg
>   [...]
>   uhidev1 at uhub0 port 10 configuration 1 interface 0 "Philips Speech 
> Processing Footcontrol USB" rev 1.00/3.02 addr 3
>   uhidev1: iclass 3/1
>   uhid0 at uhidev1: input=3, output=0, feature=0
>   [...]
> 
>   $ usbdevs -v
>   Controller /dev/usb0:
>   addr 01: 8086: Intel, xHCI root hub
>super speed, self powered, config 1, rev 1.00
>driver: uhub0
>   addr 02: 047d:1002 Kensington, USB/PS2 Wheel Mouse
>low speed, power 100 mA, config 1, rev 4.00
>driver: uhidev0
>   addr 03: 0911:091a Philips Speech Processing, Footcontrol USB
>full speed, power 100 mA, config 1, rev 3.02
>driver: uhidev1
>   addr 04: 1209:2301 Keyboardio, Model 01
>full speed, power 500 mA, config 1, rev 1.00, iSerial Ckbio01
>driver: umodem0
>driver: uhidev2
>driver: uhidev3
> 
> [0] 
> https://www.dictation.philips.com/gb/products/transcription-accessories/foot-control-acc2300/
> 
> Thanks in advance,
> 
> Raf
> 

Re: vmctl: start: Require one interface at minimium with -i

[Reyk Floeter]

On Sat, Oct 26, 2019 at 12:57:56AM +0200, Klemens Nanni wrote:
> It makes no sense to allow zero interfaces;  either a positive count is
> given or -i is omitted entirely.  vm.conf(5) does not allow interface
> configuration that results in zero interfaces either.
> 
>   $ doas vmctl start -Li0 -b ~/bsd.rd foo  
>   vmctl: starting without disks
>   vmctl: started vm 6 successfully, tty /dev/ttyph
> 

I'm not sure if I agree with the context here.

- Do you want to prevent "-i 0" and "interfaces 0" as it is equal to
  not specifying these options at all or do you want to prevent vmd from
  even running a VM without interfaces?

- It is true, the optional "interfaces" keyword in vm.conf cannot be
  0.  The main reason is that it should prevent that the same keyword is
  configured for multiple times.  This could be the same for -i.

- VMs can be started without interfaces: vmd prints a warning but
  doesn't prevent it.  While the use case without any interfaces seems
  exotic, I also don't see a reason to prevent it.  I did have PCs
  without NICs in the past ;-)

- Regarding -n and -L: The -i X option never lowers the number of
  interfaces.  It is a "minimum number of unconfigured interfaces"
  option.  Have a look at vmctl/main.c L913-919.  So -Li0 will work just
  fine, even if the -i0 is totally pointless.q

Besides that, I have no strong oppinion against the diff itself, OK.

Reyk

> Diff below raises the minimum to one and tells more about invalid counts
> with the usual strtonum(3) idiom:
> 
>   $ doas vmctl start -Li-1 -b ~/bsd.rd foo
>   vmctl: invalid count "-1": too small
>   vmctl: invalid interface count: -1
>   $ doas ./obj/vmctl start -Li0 -b ~/bsd.rd foo 
>   vmctl: count is too small: 0
>   vmctl: invalid interface count: 0
> 
> Those duplicate errors aren't easily merged without breaking consistency
> with how all the command line flags are passed, so I did nothing about
> this.
> 
> OK?
> 
> 
> Index: main.c
> ===
> RCS file: /cvs/src/usr.sbin/vmctl/main.c,v
> retrieving revision 1.58
> diff -u -p -r1.58 main.c
> --- main.c23 Aug 2019 07:55:20 -  1.58
> +++ main.c25 Oct 2019 22:48:49 -
> @@ -373,9 +373,9 @@ parse_ifs(struct parse_result *res, char
>   const char  *error;
>  
>   if (word != NULL) {
> - val = strtonum(word, 0, INT_MAX, &error);
> + val = strtonum(word, 1, INT_MAX, &error);
>   if (error != NULL)  {
> - warnx("invalid count \"%s\": %s", word, error);
> + warnx("count is %s: %s", error, word);
>   return (-1);
>   }
>   }
> 

Re: vmd: static address for local interfaces, fix static tapX names

[Reyk Floeter]

On Fri, Oct 25, 2019 at 12:27:25PM -0700, Mike Larkin wrote:
> On Fri, Oct 25, 2019 at 06:15:59PM +0000, Reyk Floeter wrote:
> > Hi,
> > 
> > the attached diff is rather large and implements two things for vmd:
> > 
> > 1) Allow to configure static IP address/gateway pairs local interfaces.
> > 2) Skip statically configured interface names (eg. tap0) when
> >   allocating dynamic interfaces.
> > 
> > Example:
> > ---snip---
> > vm "foo" {
> > disable
> > local interface "tap0" {
> > address 192.168.0.10/24 192.168.0.1
> > }
> > local interface "tap1"
> > disk "/home/vm/foo.qcow2"
> > }
> > 
> > vm "bar" {
> > local interface
> > disk "/home/vm/bar.qcow2"
> > }
> > ---snap---
> > 
> > 
> > 1) The VM "foo" has two interfaces: The first interface has a fixed
> > IPv4 address with 192.168.0.1/24 on the gateway and 192.168.0.10/24 on
> > the VM.  192.168.0.10/24 is assigned to the VM's first NIC via the
> > built-in DHCP server.  The second VM gets a default 100.64.x.x/31 IP.
> 
> I'm not sure the above description matches what I'm seeing in the vm.conf
> snippet above.
> 
> What's "the gateway" here? Is this the host machine, or the actual
> gateway, perhaps on some other machine? Does this just allow me to specify
> the host-side tap(4) IP address for a corresponding given VM vio(4) interface?
> 

Ah, OK.  I used the terms without explaining them:

With local interfaces, vmd(8) uses two IPs per interface: one for the
tap(4) on the host, one for the vio(4) on the VM.  It configures the
first one on the host and provides the second one via DHCP.  The IP on
the host IP is the default "gateway" router for the VM.

The address syntax is currently reversed:
address "address/prefix" "gateway"
Maybe I should change it to
address "gateway" "address/prefix"
or
address "address/prefix" gateway "gateway"

I also wonder if we could technically use a non-local IP address for
the gateway.  I currently enforce that the prefix matches, but I don't
enforce that both addresses are in the same subnet.

When using the default auto-generated 100.64.0.0/31 method, it uses
the first IP in the subnet as the gateway and the second IP for the
VM.

> And did you mean "The second interface" there instead of the "The second VM"?
> (Although I think the description fits for "The second VM" also...)
> 

Yes, both, the second interface is correct as well.

> I think the idea is sound. As long as we don't end up adding extra command
> line args to vmctl to manually configure this, which it doesn't appear we are
> doing here. :)
> 

I don't want to add it to vmctl either.

> I didn't read the diff in great detail, I'll wait until you say you have a
> final version.
> 

OK, thanks.

Reyk

> -ml
> 
> > This idea came up when I talked with Mischa at EuroBSDCon about
> > OpenBSDAms: instead of using L2 and external static dhcpd for all VMs,
> > it could be a solution to use L3 and to avoid bridge(4) and dhcpd(8).
> > But it would need a way to serve static IPs via the internal dhcp
> > server.  Using L3 with vmd is better with performance, routing, PF,
> > etc., but has the drawback that it wastes a subnet and gateway IP per
> > VM (maybe rdomains or other tricks could help here, but this is a
> > problem for later).
> > 
> > 2) The VM "foo" uses two static interface names, tap0 and tap1, and
> > the VM "bar" uses a dynamic interface name (tapX).  Without this diff,
> > vmd would most certainly use tap0 for bar's interface because foo is
> > disabled and not started before bar.  With the diff, the first
> > interface of bar will be tap2 or higher.
> > The problem was just reported by kn@.  I mixed both things into
> > one diff because I was working on 1) when kn@ reported it.  There are
> > other ways to implement 2) but solving both issues in a similar way
> > made more sense.
> > 
> > This is not the final diff.  I still have to clean it up, get
> > feedback, think a little bit about it, and split it into smaller parts
> > for review.  I wanted to share the big picture.
> > 
> > As a side node, I implemented the lookup with sorted tables because it
> > is the most efficient way to do it, but maybe a simple linear lookup
> > (iterating over all the VMs and all the interfaces all the time) would
> &

vmd: static address for local interfaces, fix static tapX names

[Reyk Floeter]

Hi,

the attached diff is rather large and implements two things for vmd:

1) Allow to configure static IP address/gateway pairs local interfaces.
2) Skip statically configured interface names (eg. tap0) when
  allocating dynamic interfaces.

Example:
---snip---
vm "foo" {
disable
local interface "tap0" {
address 192.168.0.10/24 192.168.0.1
}
local interface "tap1"
disk "/home/vm/foo.qcow2"
}

vm "bar" {
local interface
disk "/home/vm/bar.qcow2"
}
---snap---


1) The VM "foo" has two interfaces: The first interface has a fixed
IPv4 address with 192.168.0.1/24 on the gateway and 192.168.0.10/24 on
the VM.  192.168.0.10/24 is assigned to the VM's first NIC via the
built-in DHCP server.  The second VM gets a default 100.64.x.x/31 IP.
This idea came up when I talked with Mischa at EuroBSDCon about
OpenBSDAms: instead of using L2 and external static dhcpd for all VMs,
it could be a solution to use L3 and to avoid bridge(4) and dhcpd(8).
But it would need a way to serve static IPs via the internal dhcp
server.  Using L3 with vmd is better with performance, routing, PF,
etc., but has the drawback that it wastes a subnet and gateway IP per
VM (maybe rdomains or other tricks could help here, but this is a
problem for later).

2) The VM "foo" uses two static interface names, tap0 and tap1, and
the VM "bar" uses a dynamic interface name (tapX).  Without this diff,
vmd would most certainly use tap0 for bar's interface because foo is
disabled and not started before bar.  With the diff, the first
interface of bar will be tap2 or higher.
The problem was just reported by kn@.  I mixed both things into
one diff because I was working on 1) when kn@ reported it.  There are
other ways to implement 2) but solving both issues in a similar way
made more sense.

This is not the final diff.  I still have to clean it up, get
feedback, think a little bit about it, and split it into smaller parts
for review.  I wanted to share the big picture.

As a side node, I implemented the lookup with sorted tables because it
is the most efficient way to do it, but maybe a simple linear lookup
(iterating over all the VMs and all the interfaces all the time) would
be good enough.  But the current approach has benefits - if I did it
right ;)

Thoughts?

Reyk

Index: usr.sbin/vmd/dhcp.c
===
RCS file: /cvs/src/usr.sbin/vmd/dhcp.c,v
retrieving revision 1.8
diff -u -p -u -p -r1.8 dhcp.c
--- usr.sbin/vmd/dhcp.c 27 Dec 2018 19:51:30 -  1.8
+++ usr.sbin/vmd/dhcp.c 25 Oct 2019 18:11:05 -
@@ -119,8 +119,7 @@ dhcp_request(struct vionet_dev *dev, cha
}
 
if ((client_addr.s_addr =
-   vm_priv_addr(&env->vmd_cfg,
-   dev->vm_vmid, dev->idx, 1)) == 0)
+   vm_priv_addr(&env->vmd_cfg, dev->vm_vmid, dev->idx, 1, &mask)) == 0)
return (-1);
memcpy(&resp.yiaddr, &client_addr,
sizeof(client_addr));
@@ -129,7 +128,7 @@ dhcp_request(struct vionet_dev *dev, cha
ss2sin(&pc.pc_dst)->sin_port = htons(CLIENT_PORT);
 
if ((server_addr.s_addr = vm_priv_addr(&env->vmd_cfg, dev->vm_vmid,
-   dev->idx, 0)) == 0)
+   dev->idx, 0, &mask)) == 0)
return (-1);
memcpy(&resp.siaddr, &server_addr, sizeof(server_addr));
memcpy(&ss2sin(&pc.pc_src)->sin_addr, &server_addr,
Index: usr.sbin/vmd/parse.y
===
RCS file: /cvs/src/usr.sbin/vmd/parse.y,v
retrieving revision 1.52
diff -u -p -u -p -r1.52 parse.y
--- usr.sbin/vmd/parse.y14 May 2019 06:05:45 -  1.52
+++ usr.sbin/vmd/parse.y25 Oct 2019 18:11:05 -
@@ -120,9 +120,9 @@ typedef struct {
 
 
 %token INCLUDE ERROR
-%token ADD ALLOW BOOT CDROM DEVICE DISABLE DISK DOWN ENABLE FORMAT GROUP
-%token INET6 INSTANCE INTERFACE LLADDR LOCAL LOCKED MEMORY NET NIFS OWNER
-%token PATH PREFIX RDOMAIN SIZE SOCKET SWITCH UP VM VMID
+%token ADD ADDRESS ALLOW BOOT CDROM DEVICE DISABLE DISK DOWN ENABLE FORMAT
+%token GROUP INET6 INSTANCE INTERFACE LLADDR LOCAL LOCKED MEMORY NET NIFS
+%token OWNER PATH PREFIX RDOMAIN SIZE SOCKET SWITCH UP VM VMID
 %token   NUMBER
 %token   STRING
 %typelladdr
@@ -413,6 +413,12 @@ vm_opts: disable   {
 
if ($1)
vmc.vmc_ifflags[i] |= VMIFF_LOCAL;
+   else if (vmc.vmc_ifflags[i] &
+   (VMIFF_ADDR4|VMIFF_ADDR6)) {
+   yyerror("address on non-local interface");
+   free($3);
+   YYERROR;
+   }
if ($3 != NULL) {
if (strcmp($3, "tap") != 0 &&
(priv_getiftype($3, type, NULL) == -1 ||
@@ -617,7 +623,53 @@ iface_

Re: vmd: opentap: ifname is not optional

[Reyk Floeter]

On Fri, Oct 25, 2019 at 10:16:14AM +0200, Klemens Nanni wrote:
> The function argument is not checked at all and strlcpy(3) will segfault
> if it NULL.
> 
> Slightly adjust the comment to reflect this and defer the error case's
> default value to the end:  If a node is successfully opened, ifname gets
> rewritten.
> 
> There's only one usage of this function in config.c which passes a
> non-NULL buffer and always expects ifname to be non-NULL and non-empty.
> 
> config.c:config_setwm()
> 225 char ifname[IF_NAMESIZE], *s;
> ...
> 425 if (s != NULL) {
> 426 snprintf(path, PATH_MAX, "/dev/%s", s);
> 427 tapfds[i] = open(path, O_RDWR | O_NONBLOCK);
> 428 } else {
> 429 tapfds[i] = opentap(ifname);
> 430 s = ifname;
> 431 }
> 432 if (tapfds[i] == -1) {
> 433 log_warn("%s: can't open tap %s", __func__, s);
> 434 goto fail;
> 435 }
> 
> OK?
> 

You can also remove the "if (ifname != NULL)" check in opentap().

Otherwise OK.

Reyk

> Index: vmm.c
> ===
> RCS file: /cvs/src/usr.sbin/vmd/vmm.c,v
> retrieving revision 1.93
> diff -u -p -r1.93 vmm.c
> --- vmm.c 28 Jun 2019 13:32:51 -  1.93
> +++ vmm.c 24 Oct 2019 23:34:18 -
> @@ -590,7 +590,7 @@ terminate_vm(struct vm_terminate_params 
>   * Opens the next available tap device, up to MAX_TAP.
>   *
>   * Parameters
> - *  ifname: an optional buffer of at least IF_NAMESIZE bytes.
> + *  ifname: a buffer of at least IF_NAMESIZE bytes.
>   *
>   * Returns a file descriptor to the tap node opened, or -1 if no tap
>   * devices were available.
> @@ -601,7 +601,6 @@ opentap(char *ifname)
>   int i, fd;
>   char path[PATH_MAX];
>  
> - strlcpy(ifname, "tap", IF_NAMESIZE);
>   for (i = 0; i < MAX_TAP; i++) {
>   snprintf(path, PATH_MAX, "/dev/tap%d", i);
>   fd = open(path, O_RDWR | O_NONBLOCK);
> @@ -611,6 +610,7 @@ opentap(char *ifname)
>   return (fd);
>   }
>   }
> + strlcpy(ifname, "tap", IF_NAMESIZE);
>  
>   return (-1);
>  }
> 

Re: iked(8): improve logging output

[Reyk Floeter]

Hi,

I agree that __func__ should be removed from anything except log_debug() 
messages.

I think you should prepend the term sa or spi to explain what the hex numbers 
mean.

otherwise OK reyk

> Am 09.08.2019 um 17:31 schrieb Tobias Heider :
> 
> The current log output is not as helpful as i would like it to be, so here's
> a diff to make it better. This unifies the syntax of the send and recv output,
> uses the same terms as the configuration (peer and local instead of to and 
> from)
> and prepends the SAs SPI to each line to make it easier to spot which message
> belongs to which flow.
> 
> Example old:
> ikev2_msg_send: IKE_SA_INIT request from 0.0.0.0:500 to 10.0.1.23:500 msgid 
> 0, 510 bytes
> ikev2_recv: IKE_SA_INIT response from responder 10.0.1.23:500 to 
> 10.0.1.24:500 policy 'test' id 0, 446 bytes
> ikev2_msg_send: IKE_AUTH request from 10.0.1.24:500 to 10.0.1.23:500 msgid 1, 
> 272 bytes
> ikev2_recv: IKE_AUTH response from responder 10.0.1.23:500 to 10.0.1.24:500 
> policy 'test' id 1, 240 bytes
> sa_state: VALID -> ESTABLISHED from 10.0.1.23:500 to 10.0.1.24:500 policy 
> 'test'
> 
> Example new:
> 0x5003cf5b45df23ae: recv IKE_SA_INIT req 0 peer 10.0.1.24:500 local 
> 10.0.1.23:500, 510 bytes, policy 'test'
> 0x5003cf5b45df23ae: send IKE_SA_INIT res 0 peer 10.0.1.24:500 local 
> 10.0.1.23:500, 446 bytes
> 0x5003cf5b45df23ae: recv IKE_AUTH req 1 peer 10.0.1.24:500 local 
> 10.0.1.23:500, 272 bytes, policy 'test'
> 0x5003cf5b45df23ae: send IKE_AUTH res 1 peer 10.0.1.24:500 local 
> 10.0.1.23:500, 240 bytes
> 0x5003cf5b45df23ae: sa_state: VALID -> ESTABLISHED from 10.0.1.24:500 to 
> 10.0.1.23:500 policy 'test'
> 
> Ok?
> 
> Index: iked.h
> ===
> RCS file: /mount/openbsd/cvs/src/sbin/iked/iked.h,v
> retrieving revision 1.121
> diff -u -p -u -r1.121 iked.h
> --- iked.h11 May 2019 16:30:23 -1.121
> +++ iked.h9 Aug 2019 14:43:33 -
> @@ -843,6 +843,11 @@ int ikev2_rekey_sa(struct iked *, struc
> int ikev2_drop_sa(struct iked *, struct iked_spi *);
> int ikev2_print_id(struct iked_id *, char *, size_t);
> 
> +const char*ikev2_ikesa_info(uint64_t, const char *msg);
> +#define SPI_IH(hdr)  ikev2_ikesa_info(betoh64((hdr)->ike_ispi), NULL)
> +#define SPI_SH(sh, f)ikev2_ikesa_info((sh)->sh_ispi, (f))
> +#define SPI_SA(sa, f)SPI_SH(&(sa)->sa_hdr, (f))
> +
> /* ikev2_msg.c */
> void ikev2_msg_cb(int, short, void *);
> struct ibuf *
> Index: ikev2.c
> ===
> RCS file: /mount/openbsd/cvs/src/sbin/iked/ikev2.c,v
> retrieving revision 1.171
> diff -u -p -u -r1.171 ikev2.c
> --- ikev2.c11 May 2019 16:30:23 -1.171
> +++ ikev2.c9 Aug 2019 15:05:20 -
> @@ -366,6 +366,20 @@ ikev2_dispatch_cert(int fd, struct privs
>return (0);
> }
> 
> +const char *
> +ikev2_ikesa_info(uint64_t spi, const char *msg)
> +{
> +static char buf[1024];
> +const char *spistr;
> +
> +spistr = print_spi(spi, 8);
> +if (msg)
> +snprintf(buf, sizeof(buf), "%s: %s", spistr, msg);
> +else
> +snprintf(buf, sizeof(buf), "%s: ", spistr);
> +return buf;
> +}
> +
> struct iked_sa *
> ikev2_getimsgdata(struct iked *env, struct imsg *imsg, struct iked_sahdr *sh,
> uint8_t *type, uint8_t **buf, size_t *size)
> @@ -423,14 +437,15 @@ ikev2_recv(struct iked *env, struct iked
>if (policy_lookup(env, msg) != 0)
>return;
> 
> -log_info("%s: %s %s from %s %s to %s policy '%s' id %u, %ld bytes",
> -__func__, print_map(hdr->ike_exchange, ikev2_exchange_map),
> -msg->msg_response ? "response" : "request",
> -initiator ? "responder" : "initiator",
> +log_info("%srecv %s %s %u peer %s local %s, %ld bytes, policy '%s'",
> +SPI_IH(hdr),
> +print_map(hdr->ike_exchange, ikev2_exchange_map),
> +msg->msg_response ? "res" : "req",
> +msg->msg_msgid,
>print_host((struct sockaddr *)&msg->msg_peer, NULL, 0),
>print_host((struct sockaddr *)&msg->msg_local, NULL, 0),
> -msg->msg_policy->pol_name, msg->msg_msgid,
> -ibuf_length(msg->msg_data));
> +ibuf_length(msg->msg_data),
> +msg->msg_policy->pol_name);
>log_debug("%s: ispi %s rspi %s", __func__,
>print_spi(betoh64(hdr->ike_ispi), 8),
>print_spi(betoh64(hdr->ike_rspi), 8));
> Index: ikev2_msg.c
> ===
> RCS file: /mount/openbsd/cvs/src/sbin/iked/ikev2_msg.c,v
> retrieving revision 1.55
> diff -u -p -u -r1.55 ikev2_msg.c
> --- ikev2_msg.c11 May 2019 16:30:23 -1.55
> +++ ikev2_msg.c9 Aug 2019 15:14:23 -
> @@ -319,12 +319,13 @@ ikev2_msg_send(struct iked *env, struct 
> 
>exchange = hdr->ike_exchange;
>flags = hdr->ike_flags;
> -log_info("%s: %s %s from %s to %s msgid %u, %ld bytes%s", __func__,
> +log_info("%ssend %s %s %u peer %s local %s, %l

Re: [patch] relayd OCSP stapling for TLS server

[Reyk Floeter]

On Thu, Jun 20, 2019 at 07:58:10PM +0200, Bruno Flueckiger wrote:
> Hi,
> 
> The patch below adds OCSP stapling to the TLS server in relayd(8). The
> OCSP response is read from a binary encoded DER file that can be created
> using ocspcheck(8).
> 
> If a file with the same name as the certificate and private key files is
> found, its content is loaded and OCSP stapling is active. If there is no
> file or loading its content fails, OCSP stapling remains disabled.
> 
> relayd(8) uses the same mechanism it uses to find the certificate file,
> only the file name extension is different: .der instead of .pem
> 

I had this diff finished more than a month ago, but it had to wait for
the SNI diff to go in.  It is suprisingly similar to your version
except some minor difference in relay_tls_ctx_create(), the man page,
and the fact that I've decided for using ".ocsp" instead of ".der" for
the ending (as .der could be anything).

OK?

Reyk

Index: usr.sbin/relayd/config.c
===
RCS file: /cvs/src/usr.sbin/relayd/config.c,v
retrieving revision 1.39
diff -u -p -u -p -r1.39 config.c
--- usr.sbin/relayd/config.c1 Jun 2019 09:54:19 -   1.39
+++ usr.sbin/relayd/config.c21 Jun 2019 11:11:05 -
@@ -903,6 +903,16 @@ config_setrelay(struct relayd *env, stru
rlay->rl_conf.name);
return (-1);
}
+   if (id == PROC_RELAY &&
+   cert->cert_ocsp_fd != -1 &&
+   config_setrelayfd(ps, id, n,
+   cert->cert_id, cert->cert_relayid,
+   RELAY_FD_OCSP, cert->cert_ocsp_fd) == -1) {
+   log_warn("%s: fd passing failed for "
+   "`%s'", __func__,
+   rlay->rl_conf.name);
+   return (-1);
+   }
if (id == PROC_CA &&
cert->cert_key_fd != -1 &&
config_setrelayfd(ps, id, n,
@@ -992,6 +1002,10 @@ config_setrelay(struct relayd *env, stru
close(cert->cert_key_fd);
cert->cert_key_fd = -1;
}
+   if (cert->cert_ocsp_fd != -1) {
+   close(cert->cert_ocsp_fd);
+   cert->cert_ocsp_fd = -1;
+   }
}
 
return (0);
@@ -1113,6 +1127,7 @@ config_getrelayfd(struct relayd *env, st
switch (crfd.type) {
case RELAY_FD_CERT:
case RELAY_FD_KEY:
+   case RELAY_FD_OCSP:
if ((cert = cert_find(env, crfd.id)) == NULL) {
if ((cert = cert_add(env, crfd.id)) == NULL)
return (-1);
@@ -1133,6 +1148,9 @@ config_getrelayfd(struct relayd *env, st
break;
case RELAY_FD_KEY:
cert->cert_key_fd = imsg->fd;
+   break;
+   case RELAY_FD_OCSP:
+   cert->cert_ocsp_fd = imsg->fd;
break;
case RELAY_FD_CACERT:
rlay->rl_tls_ca_fd = imsg->fd;
Index: usr.sbin/relayd/relay.c
===
RCS file: /cvs/src/usr.sbin/relayd/relay.c,v
retrieving revision 1.247
diff -u -p -u -p -r1.247 relay.c
--- usr.sbin/relayd/relay.c 31 May 2019 15:15:37 -  1.247
+++ usr.sbin/relayd/relay.c 21 Jun 2019 11:11:06 -
@@ -2130,8 +2130,8 @@ relay_tls_ctx_create(struct relay *rlay)
struct relay_cert   *cert;
const char  *fake_key;
int  fake_keylen, keyfound = 0;
-   char*buf = NULL, *cabuf = NULL;
-   off_tlen = 0, calen = 0;
+   char*buf = NULL, *cabuf = NULL, *ocspbuf = NULL;
+   off_tlen = 0, calen = 0, ocsplen = 0;
 
if ((tls_cfg = tls_config_new()) == NULL) {
log_warnx("unable to allocate TLS config");
@@ -2203,6 +2203,16 @@ relay_tls_ctx_create(struct relay *rlay)
}
cert->cert_fd = -1;
 
+   if (cert->cert_ocsp_fd != -1 &&
+   (ocspbuf = relay_load_fd(cert->cert_ocsp_fd,
+   &ocsplen)) == NULL) {
+   log_warn("failed to load OCSP staplefile");
+   goto err;
+   }
+   if (ocsplen == 0)
+   purge_key(&ocspbuf, ocsplen);
+   cert->cert_ocsp_fd = -1;
+
if ((fake_keylen = ssl_ctx_fake_private_key(buf, len,
  

Re: [patch] relayd OCSP stapling for TLS server

[Reyk Floeter]

Hi Bruno,

thanks for your efforts. I‘ve already written an OCSP patch which was being 
delayed in review.

I don’t have the patch at hand but you can see the branch at 
https://github.com/reyk/relayd/tree/ocsp

Reyk

> Am 20.06.2019 um 19:58 schrieb Bruno Flueckiger :
> 
> Hi,
> 
> The patch below adds OCSP stapling to the TLS server in relayd(8). The
> OCSP response is read from a binary encoded DER file that can be created
> using ocspcheck(8).
> 
> If a file with the same name as the certificate and private key files is
> found, its content is loaded and OCSP stapling is active. If there is no
> file or loading its content fails, OCSP stapling remains disabled.
> 
> relayd(8) uses the same mechanism it uses to find the certificate file,
> only the file name extension is different: .der instead of .pem
> 
> Cheers,
> Bruno
> 
> Index: usr.sbin/relayd/config.c
> ===
> RCS file: /cvs/src/usr.sbin/relayd/config.c,v
> retrieving revision 1.39
> diff -u -p -r1.39 config.c
> --- usr.sbin/relayd/config.c1 Jun 2019 09:54:19 -1.39
> +++ usr.sbin/relayd/config.c20 Jun 2019 17:37:09 -
> @@ -913,6 +913,14 @@ config_setrelay(struct relayd *env, stru
>rlay->rl_conf.name);
>return (-1);
>}
> +if (cert->cert_ocsp_fd != -1 &&
> +config_setrelayfd(ps, id, n,
> +cert->cert_id, cert->cert_relayid,
> +RELAY_FD_OCSP, cert->cert_ocsp_fd) == -1) {
> +log_warn("%s: fd passing failed for "
> +"`%s'", __func__,
> +rlay->rl_conf.name);
> +}
>}
>}
> 
> @@ -992,6 +1000,10 @@ config_setrelay(struct relayd *env, stru
>close(cert->cert_key_fd);
>cert->cert_key_fd = -1;
>}
> +if (cert->cert_ocsp_fd != -1) {
> +close(cert->cert_ocsp_fd);
> +cert->cert_ocsp_fd = -1;
> +}
>}
> 
>return (0);
> @@ -1113,6 +1125,7 @@ config_getrelayfd(struct relayd *env, st
>switch (crfd.type) {
>case RELAY_FD_CERT:
>case RELAY_FD_KEY:
> +case RELAY_FD_OCSP:
>if ((cert = cert_find(env, crfd.id)) == NULL) {
>if ((cert = cert_add(env, crfd.id)) == NULL)
>return (-1);
> @@ -1139,6 +1152,9 @@ config_getrelayfd(struct relayd *env, st
>break;
>case RELAY_FD_CAFILE:
>rlay->rl_tls_cacert_fd = imsg->fd;
> +break;
> +case RELAY_FD_OCSP:
> +cert->cert_ocsp_fd = imsg->fd;
>break;
>}
> 
> Index: usr.sbin/relayd/relay.c
> ===
> RCS file: /cvs/src/usr.sbin/relayd/relay.c,v
> retrieving revision 1.247
> diff -u -p -r1.247 relay.c
> --- usr.sbin/relayd/relay.c31 May 2019 15:15:37 -1.247
> +++ usr.sbin/relayd/relay.c20 Jun 2019 17:37:09 -
> @@ -2130,8 +2130,8 @@ relay_tls_ctx_create(struct relay *rlay)
>struct relay_cert*cert;
>const char*fake_key;
>int fake_keylen, keyfound = 0;
> -char*buf = NULL, *cabuf = NULL;
> -off_t len = 0, calen = 0;
> +char*buf = NULL, *cabuf = NULL, *ocspbuf = NULL;
> +off_t len = 0, calen = 0, ocsplen = 0;
> 
>if ((tls_cfg = tls_config_new()) == NULL) {
>log_warnx("unable to allocate TLS config");
> @@ -2209,9 +2209,19 @@ relay_tls_ctx_create(struct relay *rlay)
>goto err;
>}
> 
> +if (cert->cert_ocsp_fd == -1)
> +goto without;
> +
> +if ((ocspbuf = relay_load_fd(cert->cert_ocsp_fd,
> +&ocsplen)) == NULL) {
> +log_warn("failed to load ocsp staple");
> +ocsplen = 0;
> +}
> +
> + without:
>if (keyfound == 1 &&
>tls_config_set_keypair_ocsp_mem(tls_cfg, buf, len,
> -fake_key, fake_keylen, NULL, 0) != 0) {
> +fake_key, fake_keylen, ocspbuf, ocsplen) != 0) {
>log_warnx("failed to set tls certificate: %s",
>tls_config_error(tls_cfg));
>goto err;
> @@ -2223,7 +2233,7 @@ relay_tls_ctx_create(struct relay *rlay)
>goto err;
> 
>if (tls_config_add_keypair_ocsp_mem(tls_cfg, buf, len,
> -fake_key, fake_keylen, NULL, 0) != 0) {
> +fake_key, fake_keylen, ocspbuf, ocsplen) != 0) {
>log_warnx("failed to add tls certificate: %s",
>tls_config_error(tls_cfg));
>goto err;
> Index: usr.sbin/relayd/relayd.c
> ===
> RCS file: /cvs/src/usr.sbin/relayd/relayd.c,v
> retrieving revision 1.179
> diff -u -p -r1.179 relayd.c
> --- usr.sbin/relayd/relayd.c31 May 2019 15:25:57 -0

Re: trunk(4) shouldn't need to play with a port's if_type

[Reyk Floeter]

Hi,

the initial intention was to differentiate a trunk port from a regular Ethernet 
interface.

As long as an interface is a member of a trunk, it is not a fully featured 
Ethernet interface. The changed type prevented from using it elsewhere.

I‘m not so familiar with the current network stack anymore, so maybe there are 
other ways to do it these days, but you should test that a trunk port cannot be 
attached to a bridge or carp or anything like this.

You also forgot a comment that mentions the type as well.

Reyk

> Am 11.06.2019 um 08:33 schrieb David Gwynne :
> 
> i think trunk(4) is the only thing left in the kernel that modifies an
> interfaces if_type at runtime. this diff removes that fiddling, so
> hopefully we can say that if_type is immutable after this.
> 
> however, while this diff reads well to me, i don't actually know if it
> works. could someone kick the tyres for me?
> 
> cheers,
> dlg
> 
> Index: if_trunk.c
> ===
> RCS file: /cvs/src/sys/net/if_trunk.c,v
> retrieving revision 1.140
> diff -u -p -r1.140 if_trunk.c
> --- if_trunk.c11 May 2019 18:10:45 -1.140
> +++ if_trunk.c11 Jun 2019 06:31:29 -
> @@ -330,10 +330,7 @@ trunk_port_create(struct trunk_softc *tr
>}
>}
> 
> -/* Change the interface type */
> -tp->tp_iftype = ifp->if_type;
> -ifp->if_type = IFT_IEEE8023ADLAG;
> -
> +/* Change the interface methods */
>tp->tp_ioctl = ifp->if_ioctl;
>ifp->if_ioctl = trunk_port_ioctl;
> 
> @@ -422,9 +419,7 @@ trunk_port_destroy(struct trunk_port *tp
>if (tr->tr_port_destroy != NULL)
>(*tr->tr_port_destroy)(tp);
> 
> -/* Restore interface type. */
> -ifp->if_type = tp->tp_iftype;
> -
> +/* Restore interface methods. */
>ifp->if_ioctl = tp->tp_ioctl;
>ifp->if_output = tp->tp_output;
> 
> @@ -474,8 +469,7 @@ trunk_port_ioctl(struct ifnet *ifp, u_lo
>int error = 0;
> 
>/* Should be checked by the caller */
> -if (ifp->if_type != IFT_IEEE8023ADLAG ||
> -(tp = trunk_port_get(NULL, ifp)) == NULL ||
> +if ((tp = trunk_port_get(NULL, ifp)) == NULL ||
>(tr = (struct trunk_softc *)tp->tp_trunk) == NULL) {
>error = EINVAL;
>goto fallback;
> @@ -521,8 +515,7 @@ trunk_port_output(struct ifnet *ifp, str
> struct rtentry *rt)
> {
>/* restrict transmission on trunk members to bpf only */
> -if (ifp->if_type == IFT_IEEE8023ADLAG &&
> -(m_tag_find(m, PACKET_TAG_DLT, NULL) == NULL)) {
> +if ((m_tag_find(m, PACKET_TAG_DLT, NULL) == NULL)) {
>m_freem(m);
>return (EBUSY);
>}
> @@ -1123,10 +1116,6 @@ trunk_input(struct ifnet *ifp, struct mb
>eh = mtod(m, struct ether_header *);
>if (ETHER_IS_MULTICAST(eh->ether_dhost))
>ifp->if_imcasts++;
> -
> -/* Should be checked by the caller */
> -if (ifp->if_type != IFT_IEEE8023ADLAG)
> -goto bad;
> 
>tp = (struct trunk_port *)cookie;
>if ((tr = (struct trunk_softc *)tp->tp_trunk) == NULL)
> 

Re: enc(4) free sizes

[Reyk Floeter]

Hi,

didn’t you miss a free(sc, M_DEVBUF, 0) in L119 enc_clone_create()?

Otherwise OK reyk

> Am 10.06.2019 um 18:53 schrieb Martin Pieuchot :
> 
> ok?
> 
> Index: net/if_enc.c
> ===
> RCS file: /cvs/src/sys/net/if_enc.c,v
> retrieving revision 1.73
> diff -u -p -r1.73 if_enc.c
> --- net/if_enc.c8 Jul 2018 16:41:12 -1.73
> +++ net/if_enc.c10 Jun 2019 16:38:00 -
> @@ -77,8 +77,7 @@ enc_clone_create(struct if_clone *ifc, i
>if (unit > ENC_MAX_UNITS)
>return (EINVAL);
> 
> -if ((sc = malloc(sizeof(struct enc_softc),
> -M_DEVBUF, M_NOWAIT|M_ZERO)) == NULL)
> +if ((sc = malloc(sizeof(*sc), M_DEVBUF, M_NOWAIT|M_ZERO)) == NULL)
>return (ENOBUFS);
> 
>sc->sc_unit = unit;
> @@ -131,7 +130,8 @@ enc_clone_create(struct if_clone *ifc, i
>if (enc_allifps != NULL) {
>memcpy(new, enc_allifps,
>sizeof(struct ifnet *) * (enc_max_unit + 1));
> -free(enc_allifps, M_DEVBUF, 0);
> +free(enc_allifps, M_DEVBUF,
> +sizeof(struct ifnet *) * (enc_max_unit + 1));
>}
>enc_allifps = new;
>enc_max_unit = unit;
> @@ -157,7 +157,7 @@ enc_clone_destroy(struct ifnet *ifp)
>NET_UNLOCK();
> 
>if_detach(ifp);
> -free(sc, M_DEVBUF, 0);
> +free(sc, M_DEVBUF, sizeof(*sc));
> 
>return (0);
> }
> 

Re: ftp.html: adjust mirror minimum space

[Reyk Floeter]

On Sat, Jun 01, 2019 at 12:18:33PM +0200, Theo Buehler wrote:
> On Sat, Jun 01, 2019 at 12:05:09PM +0200, Reyk Floeter wrote:
> > Hi,
> > 
> > a fresh rsync over night revealed that the minimum space for mirrors
> > should be adjusted.
> > 
> > OK?
> 
> I don't know whether the size is correct or whether it should be bumped
> further, but please note that ftp.html is generated. Edit
> www/build/mirrors/ftp.html.end instead and run 'make ftp' from
> www/build/:
> 
> $ head -1 ftp.html
> 
> 

Forget about this diff, we were mirroring from the wrong rsync source
which included old releases that aren't on the LF1 and LF2 anymore.
We'll switch to an LF2.

Background: ungleich glaurus is sponsoring a mirror to once again have
a full and up-to-date mirror in Switzerland.  These are the people
behind many nice things including digital glarus, datacenterlight, and
hack4glarus.

Thanks to Nico and the ungleich team!

The mirror in its current state (old releases will be deleted or moved
to OpenBSD-unsupported):

https://mirror.ungleich.ch/pub/OpenBSD/

Reyk

ftp.html: adjust mirror minimum space

[Reyk Floeter]

Hi,

a fresh rsync over night revealed that the minimum space for mirrors
should be adjusted.

OK?

Reyk

Index: ftp.html
===
RCS file: /cvs/www/ftp.html,v
retrieving revision 1.794
diff -u -p -u -p -r1.794 ftp.html
--- ftp.html30 May 2019 21:05:37 -  1.794
+++ ftp.html1 Jun 2019 09:57:55 -
@@ -1155,7 +1155,7 @@ Mirrors must carry the following:
 In addition, mirrors must use a second-level mirror as their upstream.
 
 
-As of 6.1, the minimum space required is approximately 700GB.
+As of 6.5, the minimum space required is approximately 1.3TB.
 However, to reduce problems when snapshot packages are updated, it is
 recommended to use the rsync options --delete-delay --delay-updates
 which will use additional space.

Re: relayd: SNI

[Reyk Floeter]

On Thu, May 09, 2019 at 02:51:23PM +0200, Reyk Floeter wrote:
> Hi,
> 
> this diff adds SNI support to relayd.
> 

Below is the same diff again -current minus one debug line.

jsing@ has noted that calling tls_config_set_keypair_ocsp_mem() with
NULL ocsp options could be replaced with tls_config_set_keypair_mem(),
but I'd like to keep it for now because I have an OCSP diff on top of
it that I'll send once SNI is in.

OK?

Reyk

Index: usr.sbin/relayd/ca.c
===
RCS file: /cvs/src/usr.sbin/relayd/ca.c,v
retrieving revision 1.34
diff -u -p -u -p -r1.34 ca.c
--- usr.sbin/relayd/ca.c19 Sep 2018 11:28:02 -  1.34
+++ usr.sbin/relayd/ca.c13 May 2019 08:53:17 -
@@ -108,56 +108,60 @@ hash_x509(X509 *cert, char *hash, size_t
 void
 ca_launch(void)
 {
-   char hash[TLS_CERT_HASH_SIZE];
-   char*buf;
-   BIO *in = NULL;
-   EVP_PKEY*pkey = NULL;
-   struct relay*rlay;
-   X509*cert = NULL;
-   off_tlen;
+   char hash[TLS_CERT_HASH_SIZE];
+   char*buf;
+   BIO *in = NULL;
+   EVP_PKEY*pkey = NULL;
+   struct relay*rlay;
+   struct relay_cert   *cert;
+   X509*x509 = NULL;
+   off_tlen;
 
-   TAILQ_FOREACH(rlay, env->sc_relays, rl_entry) {
-   if ((rlay->rl_conf.flags & (F_TLS|F_TLSCLIENT)) == 0)
+   TAILQ_FOREACH(cert, env->sc_certs, cert_entry) {
+   if (cert->cert_fd == -1 || cert->cert_key_fd == -1)
continue;
 
-   if (rlay->rl_tls_cert_fd != -1) {
-   if ((buf = relay_load_fd(rlay->rl_tls_cert_fd,
-   &len)) == NULL)
-   fatal("ca_launch: cert relay_load_fd");
+   if ((buf = relay_load_fd(cert->cert_fd, &len)) == NULL)
+   fatal("ca_launch: cert relay_load_fd");
 
-   if ((in = BIO_new_mem_buf(buf, len)) == NULL)
-   fatalx("ca_launch: cert BIO_new_mem_buf");
+   if ((in = BIO_new_mem_buf(buf, len)) == NULL)
+   fatalx("ca_launch: cert BIO_new_mem_buf");
 
-   if ((cert = PEM_read_bio_X509(in, NULL,
-   NULL, NULL)) == NULL)
-   fatalx("ca_launch: cert PEM_read_bio_X509");
+   if ((x509 = PEM_read_bio_X509(in, NULL,
+   NULL, NULL)) == NULL)
+   fatalx("ca_launch: cert PEM_read_bio_X509");
 
-   hash_x509(cert, hash, sizeof(hash));
+   hash_x509(x509, hash, sizeof(hash));
 
-   BIO_free(in);
-   X509_free(cert);
-   purge_key(&buf, len);
-   }
-   if (rlay->rl_conf.tls_key_len) {
-   if ((in = BIO_new_mem_buf(rlay->rl_tls_key,
-   rlay->rl_conf.tls_key_len)) == NULL)
-   fatalx("%s: key", __func__);
-
-   if ((pkey = PEM_read_bio_PrivateKey(in,
-   NULL, NULL, NULL)) == NULL)
-   fatalx("%s: PEM", __func__);
-   BIO_free(in);
+   BIO_free(in);
+   X509_free(x509);
+   purge_key(&buf, len);
 
-   rlay->rl_tls_pkey = pkey;
+   if ((buf = relay_load_fd(cert->cert_key_fd, &len)) == NULL)
+   fatal("ca_launch: key relay_load_fd");
 
-   if (pkey_add(env, pkey, hash) == NULL)
-   fatalx("tls pkey");
+   if ((in = BIO_new_mem_buf(buf, len)) == NULL)
+   fatalx("%s: key", __func__);
 
-   purge_key(&rlay->rl_tls_key,
-   rlay->rl_conf.tls_key_len);
-   }
+   if ((pkey = PEM_read_bio_PrivateKey(in,
+   NULL, NULL, NULL)) == NULL)
+   fatalx("%s: PEM", __func__);
 
-   if (rlay->rl_tls_cacert_fd != -1) {
+   cert->cert_pkey = pkey;
+
+   if (pkey_add(env, pkey, hash) == NULL)
+   fatalx("tls pkey");
+
+   BIO_free(in);
+   purge_key(&buf, len);
+   }
+
+   TAILQ_FOREACH(rlay, env->sc_relays, rl_entry) {
+   if ((rlay->rl_conf.flags & (F_TLS|F_TLSCLIENT)) == 0)
+   continue;
+
+   if (rlay->rl

Re: ssl(8), fix text about web browsers and SAN

[Reyk Floeter]

I was just stumbling over this as well when I did the relayd: SNI diff.

OK reyk

On Fri, May 10, 2019 at 1:50 PM Stuart Henderson 
wrote:

> it's standard behaviour for web browsers to not use hostnames in
> Subject at all but require SAN. current ssl(8) text suggests "some new"
> and "deprecated" rather than "stopped supporting".
>
> comments/ok?
>
>
> Index: ssl.8
> ===
> RCS file: /cvs/src/share/man/man8/ssl.8,v
> retrieving revision 1.67
> diff -u -p -r1.67 ssl.8
> --- ssl.8   25 Mar 2019 18:36:58 -  1.67
> +++ ssl.8   10 May 2019 11:48:41 -
> @@ -94,9 +94,9 @@ You can also sign the key yourself, usin
>-out /etc/ssl/server.crt
>  .Ed
>  .Pp
> -Note that some new browsers have deprecated using the common name of a
> -certificate and require that subject alt names are provided.
> -This may require the use of
> +Note that standard web browsers do not use the common name of a subject,
> +but instead require that subject alt names are provided.
> +This requires the use of
>  .Ar -extfile Pa server.ext
>  when self-signing.
>  .Bd -literal -offset indent
>
>

relayd: add from/to filter options

[Reyk Floeter]

Hi,

the relayd code already had a few bits for from/to specifiers in
filter rules, but it wasn't finished.  I did get occasional requests
if it would be possible to filter based on IPs (much like Allow/Deny
rules elsewhere).  Simple blocking should better be done in pf but the
purpose of this is to match URLs/headers related to IPs (see below).

```relayd.conf
http protocol foo {
block from 10.1.1.1
block from 10.1.1.2 to 192.168.0.0/24
pass from 10.1.0.0/16 path "/hello/*" forward to 
pass from 10.2.0.0/16 path "/hello/*" forward to 
}
```

Ok?

Reyk

Index: usr.sbin/relayd/parse.y
===
RCS file: /cvs/src/usr.sbin/relayd/parse.y,v
retrieving revision 1.233
diff -u -p -u -p -r1.233 parse.y
--- usr.sbin/relayd/parse.y 13 Mar 2019 23:29:32 -  1.233
+++ usr.sbin/relayd/parse.y 9 May 2019 17:12:09 -
@@ -153,6 +153,7 @@ typedef struct {
enum direction   dir;
struct {
struct sockaddr_storage  ss;
+   int  prefixlen;
char name[HOST_NAME_MAX+1];
}addr;
struct {
@@ -187,7 +188,7 @@ typedef struct {
 %typeaction ruleaf key_option
 %type  port
 %type  host
-%type  address
+%type  address rulesrc ruledst addrprefix
 %typetimeout
 %typedigest optdigest
 %type tablespec
@@ -1293,6 +1294,20 @@ filterrule   : action dir quick ruleaf rul
rule->rule_dir = $2;
rule->rule_flags |= $3;
rule->rule_af = $4;
+   rule->rule_src.addr = $5.ss;
+   rule->rule_src.addr_mask = $5.prefixlen;
+   rule->rule_dst.addr = $6.ss;
+   rule->rule_dst.addr_mask = $6.prefixlen;
+
+   if (RELAY_AF_NEQ(rule->rule_af,
+   rule->rule_src.addr.ss_family) ||
+   RELAY_AF_NEQ(rule->rule_af,
+   rule->rule_dst.addr.ss_family) ||
+   RELAY_AF_NEQ(rule->rule_src.addr.ss_family,
+   rule->rule_dst.addr.ss_family)) {
+   yyerror("address family mismatch");
+   YYERROR;
+   }
 
rulefile = NULL;
} ruleopts_l {
@@ -1341,10 +1356,20 @@ ruleaf  : /* empty */   { $$ = 
AF_UNSPEC
| INET  { $$ = AF_INET; }
;
 
-rulesrc: /* XXX */
+rulesrc: /* empty */   {
+   memset(&$$, 0, sizeof($$));
+   }
+   | FROM addrprefix   {
+   $$ = $2;
+   }
;
 
-ruledst: /* XXX */
+ruledst: /* empty */   {
+   memset(&$$, 0, sizeof($$));
+   }
+   | TO addrprefix {
+   $$ = $2;
+   }
;
 
 ruleopts_l : /* empty */
@@ -1967,7 +1992,7 @@ routeopts_l   : routeopts_l routeoptsl nl
| routeoptsl optnl
;
 
-routeoptsl : ROUTE address '/' NUMBER {
+routeoptsl : ROUTE addrprefix {
struct netroute *nr;
 
if (router->rt_conf.af == AF_UNSPEC)
@@ -1978,14 +2003,6 @@ routeoptsl   : ROUTE address '/' NUMBER {
YYERROR;
}
 
-   if ((router->rt_conf.af == AF_INET &&
-   ($4 > 32 || $4 < 0)) ||
-   (router->rt_conf.af == AF_INET6 &&
-   ($4 > 128 || $4 < 0))) {
-   yyerror("invalid prefixlen %d", $4);
-   YYERROR;
-   }
-
if ((nr = calloc(1, sizeof(*nr))) == NULL)
fatal("out of memory");
 
@@ -1995,7 +2012,7 @@ routeoptsl: ROUTE address '/' NUMBER {
free(nr);
YYERROR;
}
-   nr->nr_conf.prefixlen = $4;
+   nr->nr_conf.prefixlen = $2.prefixlen;
nr->nr_conf.routerid = router->rt_conf.id;
nr->nr_router = router;
bcopy(&$2.ss, &nr->nr_conf.ss, sizeof($2.ss));
@@ -2166,6 +2183,26 @@ address  : STRING{
h = TAILQ_FIRST(&al);
memcpy(&$$.ss, &h->ss, sizeof($$.ss));
host_free(&al);
+  

Re: Reorder comic fonts used in error pages for httpd(8) and relayd(8)

[Reyk Floeter]

Thanks for your patch!

So we have to figure out if people prefer the new fonts or the classic Comic 
Sans MS font.

It is style vs. authenticity.

But I think your suggestion is OK.

Reyk

> Am 09.05.2019 um 15:34 schrieb Nathan Galt :
> 
> I happened upon , currently a 404 Not 
> Found page, and noticed that its font ordering was odd. While I appreciate 
> the httpd author's attempt to inject a bit of fun into error pages, the 
> ordering of the fonts in the font-family property could be improved. 
> Currently, all devices that have Chalkboard SE installed (iOS, macOS) also 
> have Comic Sans MS installed. This means that Apple-device users aren't 
> getting the best comic font available on their system. Windows users are in a 
> similar situation; even if some of them install Comic Neue, Comic Sans MS 
> will be chosen first as Comic Sans MS comes with Windows, too.
> 
> Since the font stack appears to be crafted to pick the best-available comic 
> font, I'm submitting this patch so viewers of httpd and relayd error pages 
> get the best-available preinstalled comic font.
> 
> Apologies if I've submitted the patch incorrectly; this is my first time 
> posting in tech@.
> 
> diff --git usr.sbin/httpd/server_http.c usr.sbin/httpd/server_http.c
> index 817588e2278..bc11700a92a 100644
> --- usr.sbin/httpd/server_http.c
> +++ usr.sbin/httpd/server_http.c
> @@ -923,7 +923,7 @@ server_abort_http(struct client *clt, unsigned int code, 
> const char *msg)
> 
>/* A CSS stylesheet allows minimal customization by the user */
>style = "body { background-color: white; color: black; font-family: "
> -"'Comic Sans MS', 'Chalkboard SE', 'Comic Neue', sans-serif; }\n"
> +"'Chalkboard SE', 'Comic Neue', 'Comic Sans MS', sans-serif; }\n"
>"hr { border: 0; border-bottom: 1px dashed; }\n";
> 
>/* Generate simple HTML error document */
> diff --git usr.sbin/relayd/relay_http.c usr.sbin/relayd/relay_http.c
> index f31c7e6e183..4153f53cca8 100644
> --- usr.sbin/relayd/relay_http.c
> +++ usr.sbin/relayd/relay_http.c
> @@ -1052,7 +1052,7 @@ relay_abort_http(struct rsession *con, u_int code, 
> const char *msg,
>/* A CSS stylesheet allows minimal customization by the user */
>style = (rlay->rl_proto->style != NULL) ? rlay->rl_proto->style :
>"body { background-color: #a0; color: white; font-family: "
> -"'Comic Sans MS', 'Chalkboard SE', 'Comic Neue', sans-serif; }\n"
> +"'Chalkboard SE', 'Comic Neue', 'Comic Sans MS', sans-serif; }\n"
>"hr { border: 0; border-bottom: 1px dashed; }\n";
> 
>/* Generate simple HTTP+HTML error document */
> 

relayd: SNI

[Reyk Floeter]

Hi,

this diff adds SNI support to relayd.

It is a bit big and I have to break it down, but I'm sending this
first version now to give people a chance to test.  The major
"infrastructure" change is that keypairs are not stored in relay
structs anymore but in a global list where each keypair ("cert") is
associated to a relay id.

relayd currently loads the keypair using the listen address as the
file name for the .key and .crt files.  I decided to keep this
behavior.  The new optional "tls keypair" argument allows to specify
one or more keypairs by name instead and they will be loaded as .key
and .crt files accordingly.

```relayd.conf
protocol foo {
tls keypair "localhost"
tls keypair "server"
# Or: tls { keypair "localhost", keypair "server" }
}

relay assl {
listen on 127.0.0.1 port 443 tls
protocol foo
forward to 199.233.217.205 port 80
}
```

Results in:
relay_load_certfiles: using certificate /etc/ssl/localhost.crt
relay_load_certfiles: using private key /etc/ssl/private/localhost.key
relay_load_certfiles: using certificate /etc/ssl/server.crt
relay_load_certfiles: using private key /etc/ssl/private/server.key

Connecting to relayd now gives you certificates based on the server
name, using the first one as the default.

Comments?

Reyk

Index: usr.sbin/relayd/ca.c
===
RCS file: /cvs/src/usr.sbin/relayd/ca.c,v
retrieving revision 1.34
diff -u -p -u -p -r1.34 ca.c
--- usr.sbin/relayd/ca.c19 Sep 2018 11:28:02 -  1.34
+++ usr.sbin/relayd/ca.c9 May 2019 12:35:40 -
@@ -108,56 +108,60 @@ hash_x509(X509 *cert, char *hash, size_t
 void
 ca_launch(void)
 {
-   char hash[TLS_CERT_HASH_SIZE];
-   char*buf;
-   BIO *in = NULL;
-   EVP_PKEY*pkey = NULL;
-   struct relay*rlay;
-   X509*cert = NULL;
-   off_tlen;
+   char hash[TLS_CERT_HASH_SIZE];
+   char*buf;
+   BIO *in = NULL;
+   EVP_PKEY*pkey = NULL;
+   struct relay*rlay;
+   struct relay_cert   *cert;
+   X509*x509 = NULL;
+   off_tlen;
 
-   TAILQ_FOREACH(rlay, env->sc_relays, rl_entry) {
-   if ((rlay->rl_conf.flags & (F_TLS|F_TLSCLIENT)) == 0)
+   TAILQ_FOREACH(cert, env->sc_certs, cert_entry) {
+   if (cert->cert_fd == -1 || cert->cert_key_fd == -1)
continue;
 
-   if (rlay->rl_tls_cert_fd != -1) {
-   if ((buf = relay_load_fd(rlay->rl_tls_cert_fd,
-   &len)) == NULL)
-   fatal("ca_launch: cert relay_load_fd");
+   if ((buf = relay_load_fd(cert->cert_fd, &len)) == NULL)
+   fatal("ca_launch: cert relay_load_fd");
 
-   if ((in = BIO_new_mem_buf(buf, len)) == NULL)
-   fatalx("ca_launch: cert BIO_new_mem_buf");
+   if ((in = BIO_new_mem_buf(buf, len)) == NULL)
+   fatalx("ca_launch: cert BIO_new_mem_buf");
 
-   if ((cert = PEM_read_bio_X509(in, NULL,
-   NULL, NULL)) == NULL)
-   fatalx("ca_launch: cert PEM_read_bio_X509");
+   if ((x509 = PEM_read_bio_X509(in, NULL,
+   NULL, NULL)) == NULL)
+   fatalx("ca_launch: cert PEM_read_bio_X509");
 
-   hash_x509(cert, hash, sizeof(hash));
+   hash_x509(x509, hash, sizeof(hash));
 
-   BIO_free(in);
-   X509_free(cert);
-   purge_key(&buf, len);
-   }
-   if (rlay->rl_conf.tls_key_len) {
-   if ((in = BIO_new_mem_buf(rlay->rl_tls_key,
-   rlay->rl_conf.tls_key_len)) == NULL)
-   fatalx("%s: key", __func__);
-
-   if ((pkey = PEM_read_bio_PrivateKey(in,
-   NULL, NULL, NULL)) == NULL)
-   fatalx("%s: PEM", __func__);
-   BIO_free(in);
+   BIO_free(in);
+   X509_free(x509);
+   purge_key(&buf, len);
 
-   rlay->rl_tls_pkey = pkey;
+   if ((buf = relay_load_fd(cert->cert_key_fd, &len)) == NULL)
+   fatal("ca_launch: key relay_load_fd");
 
-   if (pkey_add(env, pkey, hash) == NULL)
-   fatalx("tls pkey");
+   if ((in = BIO_new_mem_buf(buf, len)) == NULL)
+   fatalx("%s: key", __func__);
 
-   purge_key(&rlay->rl_tls_key,
-   rlay->rl_conf.tls_key_len);
-   }
+

Re: Avoid system(3) in ikectl

[Reyk Floeter]

On Wed, May 08, 2019 at 07:05:24PM -0400, Ted Unangst wrote:
> Reyk Floeter wrote:
> > On Wed, May 08, 2019 at 06:44:32PM -0400, Ted Unangst wrote:
> > > Ted Unangst wrote:
> > > > Matthew Martin wrote:
> > > > > I did that originally [1], but Reyk preferred the varargs approach 
> > > > > [2],
> > > > > so I changed the patch to match.
> > > > 
> > > > Sorry, only wading into the thread at this point. Seems not everybody 
> > > > has the
> > > > same taste in code... Well, we have the original. Let me bring that 
> > > > back.
> > > 
> > > OK, here's diff one, but with run() renamed to ca_exec(). I think picking 
> > > a
> > > better name was at least one improvement. :)
> > > 
> > > And the file: comment added.
> > > 
> > 
> > His first diff included other things that got cleaned up, please don't
> > revert to this one (eg. the // comment).
> 
> I looked at history to pick that up. Anything else?
> 

I meant: could you use /* */ instead of //?

> I think returning an error code is the right thing, even if not checked. A
> void function means can't fail, not fails silently.
> 
> ca_exec or ca_system I could go either way, but there's no longer sh involved,
> so that's why I went back to exec.
> 

OK, that's a point.

> > If you want to switch to an array instead of varargs, fine.  But the
> > definition of char *cmd[] somewhere in the function, especially in an
> > extra {} block, looks very ugly to me.  That's why I suggested the
> > varargs in the first place to avoid such a thing.
> 
> I do think it's less ugly without the extra {}. Here's a version with less of
> that.
> 

Yes, it looks slightly better.

grumble OK reyk

> 
> Index: ikeca.c
> ===
> RCS file: /home/cvs/src/usr.sbin/ikectl/ikeca.c,v
> retrieving revision 1.48
> diff -u -p -r1.48 ikeca.c
> --- ikeca.c   26 Feb 2019 14:21:30 -  1.48
> +++ ikeca.c   8 May 2019 22:59:09 -
> @@ -18,6 +18,7 @@
>  
>  #include 
>  #include 
> +#include 
>  #include 
>  #include 
>  #include 
> @@ -63,12 +64,12 @@
>  
>  struct ca {
>   char sslpath[PATH_MAX];
> - char passfile[PATH_MAX];
> + char passfile[PATH_MAX + 5]; // Includes the "file:" prefix
>   char index[PATH_MAX];
>   char serial[PATH_MAX];
>   char sslcnf[PATH_MAX];
>   char extcnf[PATH_MAX];
> - char batch[PATH_MAX];
> + char*batch;
>   char*caname;
>  };
>  
> @@ -116,6 +117,7 @@ void   ca_setenv(const char *, const cha
>  void  ca_clrenv(void);
>  void  ca_setcnf(struct ca *, const char *);
>  void  ca_create_index(struct ca *);
> +int staticca_exec(char *const []);
>  
>  /* util.c */
>  int   expand_string(char *, size_t, const char *, const char *);
> @@ -130,7 +132,6 @@ int
>  ca_key_create(struct ca *ca, char *keyname)
>  {
>   struct stat  st;
> - char cmd[PATH_MAX * 2];
>   char path[PATH_MAX];
>  
>   snprintf(path, sizeof(path), "%s/private/%s.key", ca->sslpath, keyname);
> @@ -140,10 +141,8 @@ ca_key_create(struct ca *ca, char *keyna
>   return (0);
>   }
>  
> - snprintf(cmd, sizeof(cmd),
> - "%s genrsa -out %s 2048",
> - PATH_OPENSSL, path);
> - system(cmd);
> + char *cmd[] = { PATH_OPENSSL, "genrsa", "-out", path, "2048", NULL };
> + ca_exec(cmd);
>   chmod(path, 0600);
>  
>   return (0);
> @@ -200,9 +199,9 @@ ca_delkey(struct ca *ca, char *keyname)
>  int
>  ca_request(struct ca *ca, char *keyname, int type)
>  {
> - charcmd[PATH_MAX * 2];
>   charhostname[HOST_NAME_MAX+1];
>   charname[128];
> + charkey[PATH_MAX];
>   charpath[PATH_MAX];
>  
>   ca_setenv("$ENV::CERT_CN", keyname);
> @@ -226,13 +225,12 @@ ca_request(struct ca *ca, char *keyname,
>  
>   ca_setcnf(ca, keyname);
>  
> + snprintf(key, sizeof(key), "%s/private/%s.key", ca->sslpath, keyname);
>   snprintf(path, sizeof(path), "%s/private/%s.csr", ca->sslpath, keyname);
> - snprintf(cmd, sizeof(cmd), "%s req %s-new"
> - " -key %s/private/%s.key 

Re: Avoid system(3) in ikectl

[Reyk Floeter]

On Wed, May 08, 2019 at 06:44:32PM -0400, Ted Unangst wrote:
> Ted Unangst wrote:
> > Matthew Martin wrote:
> > > I did that originally [1], but Reyk preferred the varargs approach [2],
> > > so I changed the patch to match.
> > 
> > Sorry, only wading into the thread at this point. Seems not everybody has 
> > the
> > same taste in code... Well, we have the original. Let me bring that back.
> 
> OK, here's diff one, but with run() renamed to ca_exec(). I think picking a
> better name was at least one improvement. :)
> 
> And the file: comment added.
> 

His first diff included other things that got cleaned up, please don't
revert to this one (eg. the // comment).

If you want to switch to an array instead of varargs, fine.  But the
definition of char *cmd[] somewhere in the function, especially in an
extra {} block, looks very ugly to me.  That's why I suggested the
varargs in the first place to avoid such a thing.

And we settled on renaming run() to ca_system() instead of
ca_exec()...  but that is just BS.

I'm not ok with this diff, if this has any weight at all.

Reyk

> Index: ikeca.c
> ===
> RCS file: /home/cvs/src/usr.sbin/ikectl/ikeca.c,v
> retrieving revision 1.48
> diff -u -p -r1.48 ikeca.c
> --- ikeca.c   26 Feb 2019 14:21:30 -  1.48
> +++ ikeca.c   8 May 2019 22:41:08 -
> @@ -18,6 +18,7 @@
>  
>  #include 
>  #include 
> +#include 
>  #include 
>  #include 
>  #include 
> @@ -63,12 +64,12 @@
>  
>  struct ca {
>   char sslpath[PATH_MAX];
> - char passfile[PATH_MAX];
> + char passfile[PATH_MAX + 5]; // Includes the "file:" prefix
>   char index[PATH_MAX];
>   char serial[PATH_MAX];
>   char sslcnf[PATH_MAX];
>   char extcnf[PATH_MAX];
> - char batch[PATH_MAX];
> + char*batch;
>   char*caname;
>  };
>  
> @@ -116,6 +117,7 @@ void   ca_setenv(const char *, const cha
>  void  ca_clrenv(void);
>  void  ca_setcnf(struct ca *, const char *);
>  void  ca_create_index(struct ca *);
> +int staticca_exec(char *const []);
>  
>  /* util.c */
>  int   expand_string(char *, size_t, const char *, const char *);
> @@ -130,7 +132,6 @@ int
>  ca_key_create(struct ca *ca, char *keyname)
>  {
>   struct stat  st;
> - char cmd[PATH_MAX * 2];
>   char path[PATH_MAX];
>  
>   snprintf(path, sizeof(path), "%s/private/%s.key", ca->sslpath, keyname);
> @@ -140,10 +141,8 @@ ca_key_create(struct ca *ca, char *keyna
>   return (0);
>   }
>  
> - snprintf(cmd, sizeof(cmd),
> - "%s genrsa -out %s 2048",
> - PATH_OPENSSL, path);
> - system(cmd);
> + char *cmd[] = { PATH_OPENSSL, "genrsa", "-out", path, "2048", NULL };
> + ca_exec(cmd);
>   chmod(path, 0600);
>  
>   return (0);
> @@ -200,9 +199,9 @@ ca_delkey(struct ca *ca, char *keyname)
>  int
>  ca_request(struct ca *ca, char *keyname, int type)
>  {
> - charcmd[PATH_MAX * 2];
>   charhostname[HOST_NAME_MAX+1];
>   charname[128];
> + charkey[PATH_MAX];
>   charpath[PATH_MAX];
>  
>   ca_setenv("$ENV::CERT_CN", keyname);
> @@ -226,13 +225,12 @@ ca_request(struct ca *ca, char *keyname,
>  
>   ca_setcnf(ca, keyname);
>  
> + snprintf(key, sizeof(key), "%s/private/%s.key", ca->sslpath, keyname);
>   snprintf(path, sizeof(path), "%s/private/%s.csr", ca->sslpath, keyname);
> - snprintf(cmd, sizeof(cmd), "%s req %s-new"
> - " -key %s/private/%s.key -out %s -config %s",
> - PATH_OPENSSL, ca->batch, ca->sslpath, keyname,
> - path, ca->sslcnf);
>  
> - system(cmd);
> + char *cmd[] = { PATH_OPENSSL, "req", "-new", "-key", key, "-out", path,
> + "-config", ca->sslcnf, ca->batch, NULL };
> + ca_exec(cmd);
>   chmod(path, 0600);
>  
>   return (0);
> @@ -241,8 +239,11 @@ ca_request(struct ca *ca, char *keyname,
>  int
>  ca_sign(struct ca *ca, char *keyname, int type)
>  {
> - charcmd[PATH_MAX * 2];
> - const char  *extensions = NULL;
> + charcakey[PATH_MAX];
> + charcacrt[PATH_MAX];
> + charout[PATH_MAX];
> + charin[PATH_MAX];
> + char*extensions = NULL;
>  
>   if (type == HOST_IPADDR) {
>   extensions = "x509v3_IPAddr";
> @@ -258,19 +259,16 @@ ca_sign(struct ca *ca, char *keyname, in
>   ca_setenv("$ENV::CASERIAL", ca->serial);
>   ca_setcnf(ca, keyname);
>  
> - snprintf(cmd, sizeof(cmd),
> - "%s ca -config %s -keyfile %s/private/ca.key"
> - " -cert %s/ca.crt"
> - " -extfile %s -extensions %s -out %s/%s.crt"
> - " -in %s/private/%s.csr"
> - " -passin file:%s -out

Re: Avoid system(3) in ikectl

[Reyk Floeter]

On Wed, May 08, 2019 at 01:06:30PM -0500, Matthew Martin wrote:
> ping
> 

The diff looks good now.  I otherwise agree with tedu.

OK reyk@

> On Thu, Apr 25, 2019 at 11:21:00PM -0500, Matthew Martin wrote:
> > On Thu, Apr 25, 2019 at 08:59:56PM -0600, Theo de Raadt wrote:
> > > > +   argv = alloca((n + 1) * sizeof(*argv));
> > > 
> > > Our source tree is exceedingly sparing in the use of alloca().
> > > This will not do.
> > 
> > Was staying as close as possible to exec.c, but avoiding alloca is
> > preferable; replaced with reallocarray. err message is "reallocarray" to
> > match style with the calloc call in the same file.
> > 
> > 
> diff --git usr.sbin/ikectl/ikeca.c usr.sbin/ikectl/ikeca.c
> index bac76ab9c2f..eda957d1b2f 100644
> --- usr.sbin/ikectl/ikeca.c
> +++ usr.sbin/ikectl/ikeca.c
> @@ -18,11 +18,13 @@
>  
>  #include 
>  #include 
> +#include 
>  #include 
>  #include 
>  #include 
>  #include 
>  #include 
> +#include 
>  #include 
>  #include 
>  #include 
> @@ -63,12 +65,12 @@
>  
>  struct ca {
>   char sslpath[PATH_MAX];
> - char passfile[PATH_MAX];
> + char passfile[PATH_MAX + 5]; // Includes the "file:" prefix
>   char index[PATH_MAX];
>   char serial[PATH_MAX];
>   char sslcnf[PATH_MAX];
>   char extcnf[PATH_MAX];
> - char batch[PATH_MAX];
> + char*batch;
>   char*caname;
>  };
>  
> @@ -116,6 +118,7 @@ void   ca_setenv(const char *, const char *);
>  void  ca_clrenv(void);
>  void  ca_setcnf(struct ca *, const char *);
>  void  ca_create_index(struct ca *);
> +static void   ca_system(char *, ...);
>  
>  /* util.c */
>  int   expand_string(char *, size_t, const char *, const char *);
> @@ -130,7 +133,6 @@ int
>  ca_key_create(struct ca *ca, char *keyname)
>  {
>   struct stat  st;
> - char cmd[PATH_MAX * 2];
>   char path[PATH_MAX];
>  
>   snprintf(path, sizeof(path), "%s/private/%s.key", ca->sslpath, keyname);
> @@ -140,10 +142,7 @@ ca_key_create(struct ca *ca, char *keyname)
>   return (0);
>   }
>  
> - snprintf(cmd, sizeof(cmd),
> - "%s genrsa -out %s 2048",
> - PATH_OPENSSL, path);
> - system(cmd);
> + ca_system(PATH_OPENSSL, "genrsa", "-out", path, "2048", NULL);
>   chmod(path, 0600);
>  
>   return (0);
> @@ -200,9 +199,9 @@ ca_delkey(struct ca *ca, char *keyname)
>  int
>  ca_request(struct ca *ca, char *keyname, int type)
>  {
> - charcmd[PATH_MAX * 2];
>   charhostname[HOST_NAME_MAX+1];
>   charname[128];
> + charkey[PATH_MAX];
>   charpath[PATH_MAX];
>  
>   ca_setenv("$ENV::CERT_CN", keyname);
> @@ -226,13 +225,11 @@ ca_request(struct ca *ca, char *keyname, int type)
>  
>   ca_setcnf(ca, keyname);
>  
> + snprintf(key, sizeof(key), "%s/private/%s.key", ca->sslpath, keyname);
>   snprintf(path, sizeof(path), "%s/private/%s.csr", ca->sslpath, keyname);
> - snprintf(cmd, sizeof(cmd), "%s req %s-new"
> - " -key %s/private/%s.key -out %s -config %s",
> - PATH_OPENSSL, ca->batch, ca->sslpath, keyname,
> - path, ca->sslcnf);
>  
> - system(cmd);
> + ca_system(PATH_OPENSSL, "req", "-new", "-key", key, "-out", path,
> + "-config", ca->sslcnf, ca->batch, NULL);
>   chmod(path, 0600);
>  
>   return (0);
> @@ -241,8 +238,11 @@ ca_request(struct ca *ca, char *keyname, int type)
>  int
>  ca_sign(struct ca *ca, char *keyname, int type)
>  {
> - charcmd[PATH_MAX * 2];
> - const char  *extensions = NULL;
> + charcakey[PATH_MAX];
> + charcacrt[PATH_MAX];
> + charout[PATH_MAX];
> + charin[PATH_MAX];
> + char*extensions = NULL;
>  
>   if (type == HOST_IPADDR) {
>   extensions = "x509v3_IPAddr";
> @@ -258,19 +258,15 @@ ca_sign(struct ca *ca, char *keyname, int type)
>   ca_setenv("$ENV::CASERIAL", ca->serial);
>   ca_setcnf(ca, keyname);
>  
> - snprintf(cmd, sizeof(cmd),
> - "%s ca -config %s -keyfile %s/private/ca.key"
> - " -cert %s/ca.crt"
> - " -extfile %s -extensions %s -out %s/%s.crt"
> - " -in %s/private/%s.csr"
> - " -passin file:%s -outdir %s -batch",
> - PATH_OPENSSL, ca->sslcnf, ca->sslpath,
> - ca->sslpath,
> - ca->extcnf, extensions, ca->sslpath, keyname,
> - ca->sslpath, keyname,
> - ca->passfile, ca->sslpath);
> + snprintf(cakey, sizeof(cakey), "%s/private/ca.key", ca->sslpath);
> + snprintf(cacrt, sizeof(cacrt), "%s/ca.crt", ca->sslpath);
> + snprintf(out, sizeof(out), "%s/%s.crt", ca->sslpath, keyname);
> + snprintf(in, sizeof(in), "%s/private/%s.csr", ca->sslpath, keyname);
>  

Re: relayd websocket

[Reyk Floeter]

On Wed, May 08, 2019 at 07:07:43PM +0200, Reyk Floeter wrote:
> On Wed, May 08, 2019 at 06:26:45PM +0200, Reyk Floeter wrote:
> > On Wed, Mar 06, 2019 at 05:36:32PM +0100, Sebastian Benoit wrote:
> > > Rivo Nurges(rivo.nur...@smit.ee) on 2019.03.05 22:42:13 +:
> > > > Hi!
> > > > 
> > > > On 3/5/19 10:36 PM, Claudio Jeker wrote:
> > > > > I guess that this would need strcasestr() instead of strcasecmp(), 
> > > > > since you
> > > > > are looking for the substring "Upgrade" in value. Maybe more is 
> > > > > needed if
> > > > > we want to be sure that 'Connection: Upgrade-maybe' does not match.
> > > > 
> > > > You are correct about strcasestr. "Connection: Upgrade-maybe" would 
> > > > need 
> > > > to have correct "Upgrade: websocket". Anyway, lets be strict.
> > > > 
> > > > Does something like this make sense?
> > > 
> > > i think the seperator list needs to include '\t' 
> > > because https://tools.ietf.org/html/rfc7230#appendix-B includes HTAB.
> > > 
> > > And i dont think you can mix "," with " \t" seperators,
> > > because otherwise "Foo Upgrade, Bar" will match.
> > > 
> > > Something more is needed to parse elements of a header.
> > >  
> > 
> > I would reshuffle the websocket handling a bit as I don't think that
> > we need the http priv struct.  We can lookup the parsed headers later.
> > 
> > The attached diff moves it all into one places and introduces a new
> > function kv_find_value() that is able to to match headers with
> > multiple values (I think we might need it elsewhere.  And even if not,
> > I would prefer to have this in a function intead of stuffing it into
> > relay_read_http).
> > 
> > A minor question is if the lookup should be done before or after
> > applying the filters (relay_test - my diff does it afterwards, the
> > current code does it before).
> > 
> > Tests?  Comments?  Ok?
> > 
> 
> I keep on replying to my own diffs...  the updated one below uses
> strcasecmp instead of strcasestr in kv_find_value().  There is no need
> for substring- or fn- matching in this function.
> 

Next one...

benno@ pointed out that the RFC allows whitespace before the comma.
We also don't have to strip \r\n as it is done by relayd's kv parser.

OK?

Reyk

Index: usr.sbin/relayd/http.h
===
RCS file: /cvs/src/usr.sbin/relayd/http.h,v
retrieving revision 1.10
diff -u -p -u -p -r1.10 http.h
--- usr.sbin/relayd/http.h  4 Mar 2019 21:25:03 -   1.10
+++ usr.sbin/relayd/http.h  8 May 2019 18:34:09 -
@@ -251,10 +251,4 @@ struct http_descriptor {
struct kvtreehttp_headers;
 };
 
-struct relay_http_priv {
-#define HTTP_CONNECTION_UPGRADE0x01
-#define HTTP_UPGRADE_WEBSOCKET 0x02
-   int  http_upgrade_req;
-};
-
 #endif /* HTTP_H */
Index: usr.sbin/relayd/relay.c
===
RCS file: /cvs/src/usr.sbin/relayd/relay.c,v
retrieving revision 1.242
diff -u -p -u -p -r1.242 relay.c
--- usr.sbin/relayd/relay.c 4 Mar 2019 21:25:03 -   1.242
+++ usr.sbin/relayd/relay.c 8 May 2019 18:34:09 -
@@ -1410,13 +1410,7 @@ relay_session(struct rsession *con)
return;
}
 
-   if (rlay->rl_proto->type == RELAY_PROTO_HTTP) {
-   if (relay_http_priv_init(con) == -1) {
-   relay_close(con,
-   "failed to allocate http session data", 1);
-   return;
-   }
-   } else {
+   if (rlay->rl_proto->type != RELAY_PROTO_HTTP) {
if (rlay->rl_conf.fwdmode == FWD_TRANS)
relay_bindanyreq(con, 0, IPPROTO_TCP);
else if (relay_connect(con) == -1) {
Index: usr.sbin/relayd/relay_http.c
===
RCS file: /cvs/src/usr.sbin/relayd/relay_http.c,v
retrieving revision 1.72
diff -u -p -u -p -r1.72 relay_http.c
--- usr.sbin/relayd/relay_http.c4 Mar 2019 21:25:03 -   1.72
+++ usr.sbin/relayd/relay_http.c8 May 2019 18:34:09 -
@@ -110,17 +110,6 @@ relay_http_init(struct relay *rlay)
 }
 
 int
-relay_http_priv_init(struct rsession *con)
-{
-   struct relay_http_priv  *p;
-   if ((p = calloc(1, sizeof(struct relay_http_priv))) == NULL)
-   return (-1);
-
-   con->se_priv = p;
-   return (0);
-}
-
-int
 relay_httpdesc_init(struct ctl_re

Re: vm.conf: boot-device

[Reyk Floeter]

On Wed, May 08, 2019 at 06:47:53PM +0200, Anton Lindqvist wrote:
> Hi,
> A first stab at adding support for option `-B device' to vm.conf(5).
> With the diff below, I'm able to add a dedicated VM to be used with
> autoinstall(5):
> 
>   vm "amd64-install" {
>   disable
>   boot $snapshots "bsd.rd"
>   disk $home "amd64.qcow2"
>   boot-device net
>   local interface
>   }
> 
> The name `boot-device' is of course up for debate. Also, should allow support
> also be considered?
> 
> Comments? OK?
> 

Nice, thanks for adding this.  When I did vm.conf vs vmctl, I always
liked to have vm.conf feature-complete or even to have it as a
superset of vmctl - it is just nicer to add complex options with a
grammar than command line flags (yay getsubopt(3)!).

Could you adjust the manpage based on vmctl(8)?  It is much more
verbose and why should vm.conf(5) lack the details?

For the grammar, I think it should be done without the "-"...
- boot "bsd.foo"
- boot device net

...as the parser should be able to handle this.  We should have made
it more flexible but I guess we don't want to change the grammar now?
- boot image "bsd.foo"
- boot device net

Reyk

> Index: parse.y
> ===
> RCS file: /cvs/src/usr.sbin/vmd/parse.y,v
> retrieving revision 1.50
> diff -u -p -r1.50 parse.y
> --- parse.y   13 Feb 2019 22:57:08 -  1.50
> +++ parse.y   28 Apr 2019 13:54:50 -
> @@ -120,12 +120,13 @@ typedef struct {
>  
>  
>  %token   INCLUDE ERROR
> -%token   ADD ALLOW BOOT CDROM DISABLE DISK DOWN ENABLE FORMAT GROUP INET6
> -%token   INSTANCE INTERFACE LLADDR LOCAL LOCKED MEMORY NIFS OWNER PATH 
> PREFIX
> -%token   RDOMAIN SIZE SOCKET SWITCH UP VM VMID
> +%token   ADD ALLOW BOOT BOOTDEVICE CDROM DISABLE DISK DOWN ENABLE FORMAT 
> GROUP
> +%token   INET6 INSTANCE INTERFACE LLADDR LOCAL LOCKED MEMORY NET NIFS 
> OWNER
> +%token   PATH PREFIX RDOMAIN SIZE SOCKET SWITCH UP VM VMID
>  %token NUMBER
>  %token STRING
>  %type  lladdr
> +%type  bootdevice
>  %type  disable
>  %type  image_format
>  %type  local
> @@ -456,6 +457,9 @@ vm_opts   : disable   {
>   }
>   vmc.vmc_flags |= VMOP_CREATE_KERNEL;
>   }
> + | BOOTDEVICE bootdevice {
> + vmc.vmc_bootdevice = $2;
> + }
>   | CDROM string  {
>   if (vcp->vcp_cdrom[0] != '\0') {
>   yyerror("cdrom specified more than once");
> @@ -703,6 +707,11 @@ disable  : ENABLE{ $$ = 
> 0; }
>   | DISABLE   { $$ = 1; }
>   ;
>  
> +bootdevice   : CDROM { $$ = VMBOOTDEV_CDROM; }
> + | DISK  { $$ = VMBOOTDEV_DISK; }
> + | NET   { $$ = VMBOOTDEV_NET; }
> + ;
> +
>  optcomma : ','
>   |
>   ;
> @@ -755,6 +764,7 @@ lookup(char *s)
>   { "add",ADD },
>   { "allow",  ALLOW },
>   { "boot",   BOOT },
> + { "boot-device",BOOTDEVICE },
>   { "cdrom",  CDROM },
>   { "disable",DISABLE },
>   { "disk",   DISK },
> @@ -772,6 +782,7 @@ lookup(char *s)
>   { "local",  LOCAL },
>   { "locked", LOCKED },
>   { "memory", MEMORY },
> + { "net",NET },
>   { "owner",  OWNER },
>   { "prefix", PREFIX },
>   { "rdomain",RDOMAIN },
> Index: vm.conf.5
> ===
> RCS file: /cvs/src/usr.sbin/vmd/vm.conf.5,v
> retrieving revision 1.42
> diff -u -p -r1.42 vm.conf.5
> --- vm.conf.5 7 Mar 2019 18:54:06 -   1.42
> +++ vm.conf.5 28 Apr 2019 13:54:50 -
> @@ -144,6 +144,13 @@ See
>  Kernel or BIOS image to load when booting the VM.
>  If not specified, the default is to boot using the BIOS image in
>  .Pa /etc/firmware/vmm-bios .
> +.It Cm boot-device Ar device
> +Force VM to boot of
> +.Ar device
> +which may be either
> +.Ar cdrom , disk
> +or
> +.Ar net .
>  .It Cm cdrom Ar path
>  ISO image file.
>  .It Cm enable
> 

Re: relayd websocket

[Reyk Floeter]

On Wed, May 08, 2019 at 06:26:45PM +0200, Reyk Floeter wrote:
> On Wed, Mar 06, 2019 at 05:36:32PM +0100, Sebastian Benoit wrote:
> > Rivo Nurges(rivo.nur...@smit.ee) on 2019.03.05 22:42:13 +:
> > > Hi!
> > > 
> > > On 3/5/19 10:36 PM, Claudio Jeker wrote:
> > > > I guess that this would need strcasestr() instead of strcasecmp(), 
> > > > since you
> > > > are looking for the substring "Upgrade" in value. Maybe more is needed 
> > > > if
> > > > we want to be sure that 'Connection: Upgrade-maybe' does not match.
> > > 
> > > You are correct about strcasestr. "Connection: Upgrade-maybe" would need 
> > > to have correct "Upgrade: websocket". Anyway, lets be strict.
> > > 
> > > Does something like this make sense?
> > 
> > i think the seperator list needs to include '\t' 
> > because https://tools.ietf.org/html/rfc7230#appendix-B includes HTAB.
> > 
> > And i dont think you can mix "," with " \t" seperators,
> > because otherwise "Foo Upgrade, Bar" will match.
> > 
> > Something more is needed to parse elements of a header.
> >  
> 
> I would reshuffle the websocket handling a bit as I don't think that
> we need the http priv struct.  We can lookup the parsed headers later.
> 
> The attached diff moves it all into one places and introduces a new
> function kv_find_value() that is able to to match headers with
> multiple values (I think we might need it elsewhere.  And even if not,
> I would prefer to have this in a function intead of stuffing it into
> relay_read_http).
> 
> A minor question is if the lookup should be done before or after
> applying the filters (relay_test - my diff does it afterwards, the
> current code does it before).
> 
> Tests?  Comments?  Ok?
> 

I keep on replying to my own diffs...  the updated one below uses
strcasecmp instead of strcasestr in kv_find_value().  There is no need
for substring- or fn- matching in this function.

Reyk

Index: usr.sbin/relayd/http.h
===
RCS file: /cvs/src/usr.sbin/relayd/http.h,v
retrieving revision 1.10
diff -u -p -u -p -r1.10 http.h
--- usr.sbin/relayd/http.h  4 Mar 2019 21:25:03 -   1.10
+++ usr.sbin/relayd/http.h  8 May 2019 16:55:59 -
@@ -251,10 +251,4 @@ struct http_descriptor {
struct kvtreehttp_headers;
 };
 
-struct relay_http_priv {
-#define HTTP_CONNECTION_UPGRADE0x01
-#define HTTP_UPGRADE_WEBSOCKET 0x02
-   int  http_upgrade_req;
-};
-
 #endif /* HTTP_H */
Index: usr.sbin/relayd/relay.c
===
RCS file: /cvs/src/usr.sbin/relayd/relay.c,v
retrieving revision 1.242
diff -u -p -u -p -r1.242 relay.c
--- usr.sbin/relayd/relay.c 4 Mar 2019 21:25:03 -   1.242
+++ usr.sbin/relayd/relay.c 8 May 2019 16:56:00 -
@@ -1410,13 +1410,7 @@ relay_session(struct rsession *con)
return;
}
 
-   if (rlay->rl_proto->type == RELAY_PROTO_HTTP) {
-   if (relay_http_priv_init(con) == -1) {
-   relay_close(con,
-   "failed to allocate http session data", 1);
-   return;
-   }
-   } else {
+   if (rlay->rl_proto->type != RELAY_PROTO_HTTP) {
if (rlay->rl_conf.fwdmode == FWD_TRANS)
relay_bindanyreq(con, 0, IPPROTO_TCP);
else if (relay_connect(con) == -1) {
Index: usr.sbin/relayd/relay_http.c
===
RCS file: /cvs/src/usr.sbin/relayd/relay_http.c,v
retrieving revision 1.72
diff -u -p -u -p -r1.72 relay_http.c
--- usr.sbin/relayd/relay_http.c4 Mar 2019 21:25:03 -   1.72
+++ usr.sbin/relayd/relay_http.c8 May 2019 16:56:01 -
@@ -110,17 +110,6 @@ relay_http_init(struct relay *rlay)
 }
 
 int
-relay_http_priv_init(struct rsession *con)
-{
-   struct relay_http_priv  *p;
-   if ((p = calloc(1, sizeof(struct relay_http_priv))) == NULL)
-   return (-1);
-
-   con->se_priv = p;
-   return (0);
-}
-
-int
 relay_httpdesc_init(struct ctl_relay_event *cre)
 {
struct http_descriptor  *desc;
@@ -163,13 +152,13 @@ relay_read_http(struct bufferevent *bev,
struct relay*rlay = con->se_relay;
struct protocol *proto = rlay->rl_proto;
struct evbuffer *src = EVBUFFER_INPUT(bev);
-   struct relay_http_priv  *priv = con->se_priv;
char*line = NULL, *key, *value;
char*urlproto, *host, *path;
   

Re: relayd websocket

[Reyk Floeter]

On Wed, Mar 06, 2019 at 05:36:32PM +0100, Sebastian Benoit wrote:
> Rivo Nurges(rivo.nur...@smit.ee) on 2019.03.05 22:42:13 +:
> > Hi!
> > 
> > On 3/5/19 10:36 PM, Claudio Jeker wrote:
> > > I guess that this would need strcasestr() instead of strcasecmp(), since 
> > > you
> > > are looking for the substring "Upgrade" in value. Maybe more is needed if
> > > we want to be sure that 'Connection: Upgrade-maybe' does not match.
> > 
> > You are correct about strcasestr. "Connection: Upgrade-maybe" would need 
> > to have correct "Upgrade: websocket". Anyway, lets be strict.
> > 
> > Does something like this make sense?
> 
> i think the seperator list needs to include '\t' 
> because https://tools.ietf.org/html/rfc7230#appendix-B includes HTAB.
> 
> And i dont think you can mix "," with " \t" seperators,
> because otherwise "Foo Upgrade, Bar" will match.
> 
> Something more is needed to parse elements of a header.
>  

I would reshuffle the websocket handling a bit as I don't think that
we need the http priv struct.  We can lookup the parsed headers later.

The attached diff moves it all into one places and introduces a new
function kv_find_value() that is able to to match headers with
multiple values (I think we might need it elsewhere.  And even if not,
I would prefer to have this in a function intead of stuffing it into
relay_read_http).

A minor question is if the lookup should be done before or after
applying the filters (relay_test - my diff does it afterwards, the
current code does it before).

Tests?  Comments?  Ok?

Reyk

Index: usr.sbin/relayd/http.h
===
RCS file: /cvs/src/usr.sbin/relayd/http.h,v
retrieving revision 1.10
diff -u -p -u -p -r1.10 http.h
--- usr.sbin/relayd/http.h  4 Mar 2019 21:25:03 -   1.10
+++ usr.sbin/relayd/http.h  8 May 2019 16:17:38 -
@@ -251,10 +251,4 @@ struct http_descriptor {
struct kvtreehttp_headers;
 };
 
-struct relay_http_priv {
-#define HTTP_CONNECTION_UPGRADE0x01
-#define HTTP_UPGRADE_WEBSOCKET 0x02
-   int  http_upgrade_req;
-};
-
 #endif /* HTTP_H */
Index: usr.sbin/relayd/relay.c
===
RCS file: /cvs/src/usr.sbin/relayd/relay.c,v
retrieving revision 1.242
diff -u -p -u -p -r1.242 relay.c
--- usr.sbin/relayd/relay.c 4 Mar 2019 21:25:03 -   1.242
+++ usr.sbin/relayd/relay.c 8 May 2019 16:17:39 -
@@ -1410,13 +1410,7 @@ relay_session(struct rsession *con)
return;
}
 
-   if (rlay->rl_proto->type == RELAY_PROTO_HTTP) {
-   if (relay_http_priv_init(con) == -1) {
-   relay_close(con,
-   "failed to allocate http session data", 1);
-   return;
-   }
-   } else {
+   if (rlay->rl_proto->type != RELAY_PROTO_HTTP) {
if (rlay->rl_conf.fwdmode == FWD_TRANS)
relay_bindanyreq(con, 0, IPPROTO_TCP);
else if (relay_connect(con) == -1) {
Index: usr.sbin/relayd/relay_http.c
===
RCS file: /cvs/src/usr.sbin/relayd/relay_http.c,v
retrieving revision 1.72
diff -u -p -u -p -r1.72 relay_http.c
--- usr.sbin/relayd/relay_http.c4 Mar 2019 21:25:03 -   1.72
+++ usr.sbin/relayd/relay_http.c8 May 2019 16:17:40 -
@@ -110,17 +110,6 @@ relay_http_init(struct relay *rlay)
 }
 
 int
-relay_http_priv_init(struct rsession *con)
-{
-   struct relay_http_priv  *p;
-   if ((p = calloc(1, sizeof(struct relay_http_priv))) == NULL)
-   return (-1);
-
-   con->se_priv = p;
-   return (0);
-}
-
-int
 relay_httpdesc_init(struct ctl_relay_event *cre)
 {
struct http_descriptor  *desc;
@@ -163,13 +152,13 @@ relay_read_http(struct bufferevent *bev,
struct relay*rlay = con->se_relay;
struct protocol *proto = rlay->rl_proto;
struct evbuffer *src = EVBUFFER_INPUT(bev);
-   struct relay_http_priv  *priv = con->se_priv;
char*line = NULL, *key, *value;
char*urlproto, *host, *path;
int  action, unique, ret;
const char  *errstr;
size_t   size, linelen;
struct kv   *hdr = NULL;
+   struct kv   *upgrade = NULL, *upgrade_ws = NULL;
 
getmonotime(&con->se_tv_last);
cre->timedout = 0;
@@ -398,17 +387,6 @@ relay_read_http(struct bufferevent *bev,
unique = 0;
 
if (cre->line != 1) {
-   if (cre->dir == RELAY_DIR_REQUEST) {
-   if (strcasecmp("Connection", key) == 0 &&
-   strcasecmp("Upgrade", value) == 0)
-   priv->h

relayd: fix filter rules with forward to statement

[Reyk Floeter]

Hi,

the attached diff fixes filter rules with "forward to" statement in
persistent (keep-alive) connections.  See the XXX comment below.

```relayd.conf
log connection
table  {
127.0.0.1
}
table  {
127.0.0.1
}
table  {
127.0.0.1
}
http protocol pathfwd {
return error

# XXX The following workaround is not needed anymore:
#match header set "Connection" value "close"

pass path "/a/*" forward to 
pass path "/b/*" forward to 
#match request path log "*"
}
relay pathfwd {
listen on 0.0.0.0 port 80
protocol pathfwd
forward to  port 8082
forward to  port 8080
forward to  port 8081
}
```

OK?

reyk

Index: usr.sbin/relayd/relay.c
===
RCS file: /cvs/src/usr.sbin/relayd/relay.c,v
retrieving revision 1.242
diff -u -p -u -p -r1.242 relay.c
--- usr.sbin/relayd/relay.c 4 Mar 2019 21:25:03 -   1.242
+++ usr.sbin/relayd/relay.c 8 May 2019 14:26:40 -
@@ -76,11 +76,14 @@ int  relay_tls_ctx_create(struct relay 
 voidrelay_tls_transaction(struct rsession *,
struct ctl_relay_event *);
 voidrelay_tls_handshake(int, short, void *);
-voidrelay_connect_retry(int, short, void *);
 voidrelay_tls_connected(struct ctl_relay_event *);
 voidrelay_tls_readcb(int, short, void *);
 voidrelay_tls_writecb(int, short, void *);
 
+voidrelay_connect_retry(int, short, void *);
+voidrelay_connect_state(struct rsession *,
+   struct ctl_relay_event *, enum relay_state);
+
 extern void bufferevent_read_pressure_cb(struct evbuffer *, size_t,
size_t, void *);
 
@@ -654,6 +657,7 @@ relay_socket_listen(struct sockaddr_stor
 void
 relay_connected(int fd, short sig, void *arg)
 {
+   char obuf[128];
struct rsession *con = arg;
struct relay*rlay = con->se_relay;
struct protocol *proto = rlay->rl_proto;
@@ -696,6 +700,22 @@ relay_connected(int fd, short sig, void 
 
DPRINTF("%s: session %d: successful", __func__, con->se_id);
 
+   /* Log destination if it was changed in a keep-alive connection */
+   if ((con->se_table != con->se_table0) &&
+   (env->sc_conf.opts & (RELAYD_OPT_LOGCON|RELAYD_OPT_LOGCONERR))) {
+   con->se_table0 = con->se_table;
+   memset(&obuf, 0, sizeof(obuf));
+   (void)print_host(&con->se_out.ss, obuf, sizeof(obuf));
+   if (asprintf(&msg, " -> %s:%d",
+   obuf, ntohs(con->se_out.port)) == -1) {
+   relay_abort_http(con, 500,
+   "connection changed and asprintf failed", 0);
+   return;
+   }
+   relay_log(con, msg);
+   free(msg);
+   }
+
switch (rlay->rl_proto->type) {
case RELAY_PROTO_HTTP:
if (relay_httpdesc_init(out) == -1) {
@@ -1465,6 +1485,17 @@ relay_bindany(int fd, short event, void 
 }
 
 void
+relay_connect_state(struct rsession *con, struct ctl_relay_event *cre,
+enum relay_state new)
+{
+   DPRINTF("%s: session %d: %s state %s -> %s",
+   __func__, con->se_id,
+   cre->dir == RELAY_DIR_REQUEST ? "accept" : "connect",
+   relay_state(cre->state), relay_state(new));
+   cre->state = new;
+}
+
+void
 relay_connect_retry(int fd, short sig, void *arg)
 {
struct timeval   evtpause = { 1, 0 };
@@ -1533,9 +1564,9 @@ relay_connect_retry(int fd, short sig, v
}
 
if (rlay->rl_conf.flags & F_TLSINSPECT)
-   con->se_out.state = STATE_PRECONNECT;
+   relay_connect_state(con, &con->se_out, STATE_PRECONNECT);
else
-   con->se_out.state = STATE_CONNECTED;
+   relay_connect_state(con, &con->se_out, STATE_CONNECTED);
relay_inflight--;
DPRINTF("%s: inflight decremented, now %d",__func__, relay_inflight);
 
@@ -1560,7 +1591,7 @@ relay_preconnect(struct rsession *con)
con->se_id, privsep_process);
rv = relay_connect(con);
if (con->se_out.state == STATE_CONNECTED)
-   con->se_out.state = STATE_PRECONNECT;
+   relay_connect_state(con, &con->se_out, STATE_PRECONNECT);
return (rv);
 }
 
@@ -1585,7 +1616,7 @@ relay_connect(struct rsession *con)
return (-1);
}
relay_connected(con->se_out.s, EV_WRITE, con);
-   con->se_out.state = STATE_CONNECTED;
+   relay_connect_state(con, &con->se_out, STATE_CONNECTED);
return (0);
}
 
@@ -1642,7 +1673,7 @@ relay_connect(struct rsession *con)
evtimer_add(&rlay->rl_evt, &evtpause);
 
/* this connect is pending */
-

Re: iked curve25519

[Reyk Floeter]

I like the idea of switching it to the proper ID.

Reyk

> Am 30.03.2019 um 20:31 schrieb Stuart Henderson :
> 
> curve25519 had a proper ID (31) assigned in 2016 but we still have
> the draft private-use ID in iked. Any thoughts on whether we can just
> cut across to the proper ID, or whether that will be too painful?
> Are many people using this already?
> 

Re: delete dead code: tc_getfrequency

[Reyk Floeter]

We happen to use this in the vmm pvclock diff.

Reyk

> Am 26.03.2019 um 16:28 schrieb Scott Cheloha :
> 
> Dead since import by my reckoning... so, ~15 years dead?
> 
> ok?
> 
> Index: kern/kern_tc.c
> ===
> RCS file: /cvs/src/sys/kern/kern_tc.c,v
> retrieving revision 1.43
> diff -u -p -r1.43 kern_tc.c
> --- kern/kern_tc.c25 Mar 2019 23:32:00 -1.43
> +++ kern/kern_tc.c26 Mar 2019 15:27:51 -
> @@ -346,14 +346,6 @@ tc_init(struct timecounter *tc)
>timecounter = tc;
> }
> 
> -/* Report the frequency of the current timecounter. */
> -u_int64_t
> -tc_getfrequency(void)
> -{
> -
> -return (timehands->th_counter->tc_frequency);
> -}
> -
> /*
>  * Step our concept of UTC, aka the realtime clock.
>  * This is done by modifying our estimate of when we booted.
> Index: sys/timetc.h
> ===
> RCS file: /cvs/src/sys/sys/timetc.h,v
> retrieving revision 1.8
> diff -u -p -r1.8 timetc.h
> --- sys/timetc.h25 Mar 2019 23:32:00 -1.8
> +++ sys/timetc.h26 Mar 2019 15:27:51 -
> @@ -89,7 +89,6 @@ extern struct rwlock tc_lock;
> 
> extern struct timecounter *timecounter;
> 
> -u_int64_t tc_getfrequency(void);
> voidtc_init(struct timecounter *tc);
> voidtc_setclock(const struct timespec *ts);
> voidtc_setrealtimeclock(const struct timespec *ts);
> 

Re: httpd: New log format to log X-Forwarded-{For|Port} headers

[Reyk Floeter]

Hi,

On Mon, Mar 04, 2019 at 02:06:02PM +0100, Bruno Flueckiger wrote:
> I've completely reworked my patch for httpd(8). The last patch broke the
> log format combined. And the config option was ugly. This time I've
> added another log format called forwarded. It appends two fields to the
> log format combined: The first field contains the value of the header
> X-Forwarded-For and the second one the value of X-Forwarded-Port. If
> either of the headers is empty or missing a dash (-) is written.
> 
> The new log format is compatible with log analyzing tools like Webalizer
> or GoAccess. If you run httpd(8) behind a proxy like relayd(8) the new
> log format finally gives you a way to track the origin of the requests.
> 

Your diff looks clean and makes a lot of sense.

Especially since X-Forwarded-For is a feature in relayd that I first
used and documented around 2006/2007.  Adding the forwarded style to
httpd is a complementary feature in OpenBSD and not something for a
random external web stack.

OK reyk@

Anyone else, any objections?

Reyk

> Cheers,
> Bruno
> 
> Index: usr.sbin/httpd/httpd.conf.5
> ===
> RCS file: /cvs/src/usr.sbin/httpd/httpd.conf.5,v
> retrieving revision 1.103
> diff -u -p -r1.103 httpd.conf.5
> --- usr.sbin/httpd/httpd.conf.5   19 Feb 2019 11:37:26 -  1.103
> +++ usr.sbin/httpd/httpd.conf.5   27 Feb 2019 15:26:48 -
> @@ -450,7 +450,8 @@ The
>  .Ar style
>  can be
>  .Cm common ,
> -.Cm combined
> +.Cm combined ,
> +.Cm forwarded
>  or
>  .Cm connection .
>  The styles
> @@ -459,6 +460,14 @@ and
>  .Cm combined
>  write a log entry after each request similar to the standard Apache
>  and nginx access log formats.
> +The style
> +.Cm forwarded
> +extends the style
> +.Cm combined
> +by appending two fields containing the values of the headers
> +.Ar X-Forwarded-For
> +and
> +.Ar X-Forwarded-Port .
>  The style
>  .Cm connection
>  writes a summarized log entry after each connection,
> Index: usr.sbin/httpd/httpd.h
> ===
> RCS file: /cvs/src/usr.sbin/httpd/httpd.h,v
> retrieving revision 1.143
> diff -u -p -r1.143 httpd.h
> --- usr.sbin/httpd/httpd.h19 Feb 2019 11:37:26 -  1.143
> +++ usr.sbin/httpd/httpd.h27 Feb 2019 15:26:48 -
> @@ -437,7 +437,8 @@ SPLAY_HEAD(client_tree, client);
>  enum log_format {
>   LOG_FORMAT_COMMON,
>   LOG_FORMAT_COMBINED,
> - LOG_FORMAT_CONNECTION
> + LOG_FORMAT_CONNECTION,
> + LOG_FORMAT_FORWARDED
>  };
>  
>  struct log_file {
> Index: usr.sbin/httpd/parse.y
> ===
> RCS file: /cvs/src/usr.sbin/httpd/parse.y,v
> retrieving revision 1.110
> diff -u -p -r1.110 parse.y
> --- usr.sbin/httpd/parse.y19 Feb 2019 11:37:26 -  1.110
> +++ usr.sbin/httpd/parse.y27 Feb 2019 15:26:48 -
> @@ -140,7 +140,7 @@ typedef struct {
>  %token   PROTOCOLS REQUESTS ROOT SACK SERVER SOCKET STRIP STYLE SYSLOG 
> TCP TICKET
>  %token   TIMEOUT TLS TYPE TYPES HSTS MAXAGE SUBDOMAINS DEFAULT PRELOAD 
> REQUEST
>  %token   ERROR INCLUDE AUTHENTICATE WITH BLOCK DROP RETURN PASS REWRITE
> -%token   CA CLIENT CRL OPTIONAL PARAM
> +%token   CA CLIENT CRL OPTIONAL PARAM FORWARDED
>  %token STRING
>  %token NUMBER
>  %typeport
> @@ -1024,6 +1024,11 @@ logstyle   : COMMON{
>   srv_conf->flags |= SRVFLAG_LOG;
>   srv_conf->logformat = LOG_FORMAT_CONNECTION;
>   }
> + | FORWARDED {
> + srv_conf->flags &= ~SRVFLAG_NO_LOG;
> + srv_conf->flags |= SRVFLAG_LOG;
> + srv_conf->logformat = LOG_FORMAT_FORWARDED;
> + }
>   ;
>  
>  filter   : block RETURN NUMBER optstring {
> @@ -1295,6 +1300,7 @@ lookup(char *s)
>   { "ecdhe",  ECDHE },
>   { "error",  ERR },
>   { "fastcgi",FCGI },
> + { "forwarded",  FORWARDED },
>   { "hsts",   HSTS },
>   { "include",INCLUDE },
>   { "index",  INDEX },
> Index: usr.sbin/httpd/server_http.c
> ===
> RCS file: /cvs/src/usr.sbin/httpd/server_http.c,v
> retrieving revision 1.129
> diff -u -p -r1.129 server_http.c
> --- usr.sbin/httpd/server_http.c  10 Feb 2019 13:41:27 -  1.129
> +++ usr.sbin/httpd/server_http.c  27 Feb 2019 15:26:49 -
> @@ -1632,7 +1632,7 @@ server_log_http(struct client *clt, unsi
>   static char  tstamp[64];
>   static char  ip[INET6_ADDRSTRLEN];
>   time_t   t;
> - struct kvkey, *agent, *referrer;
> + struct kvke

Re: Avoid system(3) in ikectl

[Reyk Floeter]

On Wed, Mar 06, 2019 at 10:42:15PM -0600, Matthew Martin wrote:
> I had sent a similar patch a while back. There seemed to me some
> interest, but it was never comitted. Updated to apply to -current.
> 

I vaguely remember that there was a diff that had issues that I didn't
like for different reasons.  The explanation in your other mail (about
using it with doas, escaping of special characters) make some sense,
so I wouldn't be opposed to the idea in general.

But the diff is not OK as it is; a few suggestions:

- I see that "PATH_MAX + 5" is for the "file:" prefix.  This lacks a
comment as it looks confusing at first sight.

- I don't like the way how you run your run() function: declaring a
*cmd[] variable just before calling the function, very often in a
nested {} block, doesn't align to the style C code is written in
OpenBSD.

I think I have also commented the last time that you should use an
execl-interface instead that uses varags terminated by a NULL instead.

int
ca_system(const char *arg, ...);

for example
ca_system(PATH_OPENSSL, "x509", "-h", NULL);

You can re-construct the argv[] in the function internally, looping
over the va_list.

- And why does run() return int if you never check the return value?
The current system() calls are not checked either but that doesn't
mean that a local function would have to define an unused return
value.  Blindly err'ing out in the function wouldn't help either as I
guess that there are a few calls that might fail if something exists
but allow ikectl to proceed without problems.

Reyk

> diff --git ikeca.c ikeca.c
> index bac76ab9c2f..01e600abb2c 100644
> --- ikeca.c
> +++ ikeca.c
> @@ -18,6 +18,7 @@
>  
>  #include 
>  #include 
> +#include 
>  #include 
>  #include 
>  #include 
> @@ -63,12 +64,12 @@
>  
>  struct ca {
>   char sslpath[PATH_MAX];
> - char passfile[PATH_MAX];
> + char passfile[PATH_MAX + 5];
>   char index[PATH_MAX];
>   char serial[PATH_MAX];
>   char sslcnf[PATH_MAX];
>   char extcnf[PATH_MAX];
> - char batch[PATH_MAX];
> + char*batch;
>   char*caname;
>  };
>  
> @@ -116,6 +117,7 @@ void   ca_setenv(const char *, const char *);
>  void  ca_clrenv(void);
>  void  ca_setcnf(struct ca *, const char *);
>  void  ca_create_index(struct ca *);
> +int staticrun(char *const []);
>  
>  /* util.c */
>  int   expand_string(char *, size_t, const char *, const char *);
> @@ -130,7 +132,6 @@ int
>  ca_key_create(struct ca *ca, char *keyname)
>  {
>   struct stat  st;
> - char cmd[PATH_MAX * 2];
>   char path[PATH_MAX];
>  
>   snprintf(path, sizeof(path), "%s/private/%s.key", ca->sslpath, keyname);
> @@ -140,10 +141,8 @@ ca_key_create(struct ca *ca, char *keyname)
>   return (0);
>   }
>  
> - snprintf(cmd, sizeof(cmd),
> - "%s genrsa -out %s 2048",
> - PATH_OPENSSL, path);
> - system(cmd);
> + char *cmd[] = { PATH_OPENSSL, "genrsa", "-out", path, "2048", NULL };
> + run(cmd);
>   chmod(path, 0600);
>  
>   return (0);
> @@ -200,9 +199,9 @@ ca_delkey(struct ca *ca, char *keyname)
>  int
>  ca_request(struct ca *ca, char *keyname, int type)
>  {
> - charcmd[PATH_MAX * 2];
>   charhostname[HOST_NAME_MAX+1];
>   charname[128];
> + charkey[PATH_MAX];
>   charpath[PATH_MAX];
>  
>   ca_setenv("$ENV::CERT_CN", keyname);
> @@ -226,13 +225,12 @@ ca_request(struct ca *ca, char *keyname, int type)
>  
>   ca_setcnf(ca, keyname);
>  
> + snprintf(key, sizeof(key), "%s/private/%s.key", ca->sslpath, keyname);
>   snprintf(path, sizeof(path), "%s/private/%s.csr", ca->sslpath, keyname);
> - snprintf(cmd, sizeof(cmd), "%s req %s-new"
> - " -key %s/private/%s.key -out %s -config %s",
> - PATH_OPENSSL, ca->batch, ca->sslpath, keyname,
> - path, ca->sslcnf);
>  
> - system(cmd);
> + char *cmd[] = { PATH_OPENSSL, "req", "-new", "-key", key, "-out", path,
> + "-config", ca->sslcnf, ca->batch, NULL };
> + run(cmd);
>   chmod(path, 0600);
>  
>   return (0);
> @@ -241,8 +239,11 @@ ca_request(struct ca *ca, char *keyname, int type)
>  int
>  ca_sign(struct ca *ca, char *keyname, int type)
>  {
> - charcmd[PATH_MAX * 2];
> - const char  *extensions = NULL;
> + charcakey[PATH_MAX];
> + charcacrt[PATH_MAX];
> + charout[PATH_MAX];
> + charin[PATH_MAX];
> + char*extensions = NULL;
>  
>   if (type == HOST_IPADDR) {
>   extensions = "x509v3_IPAddr";
> @@ -258,19 +259,16 @@ ca_sign(struct ca *ca, char *keyname, int type)
>   ca_setenv("$ENV::CASERIAL", ca->serial);
>   ca_se

Re: add more bootdevices to vmctl

[Reyk Floeter]

OK reyk@

Please think about the manpage.

> Am 10.12.2018 um 22:35 schrieb Claudio Jeker :
> 
> Now that fw_cfg support is in vmd it makes sense to have -B disk
> and -B cdrom. Also error out if the option is not known.
> 
> This allows to use -B cdrom to force booting from the cdrom disk image
> e.g. to update the VM image.
> -- 
> :wq Claudio
> 
> Index: main.c
> ===
> RCS file: /cvs/src/usr.sbin/vmctl/main.c,v
> retrieving revision 1.50
> diff -u -p -r1.50 main.c
> --- main.c6 Dec 2018 09:23:15 -1.50
> +++ main.c8 Dec 2018 06:59:17 -
> @@ -856,8 +856,14 @@ ctl_start(struct parse_result *res, int 
>case 'B':
>if (res->bootdevice)
>errx(1, "boot device specified multiple times");
> -if (strcmp("net", optarg) == 0)
> +if (strcmp("disk", optarg) == 0)
> +res->bootdevice = VMBOOTDEV_DISK;
> +else if (strcmp("cdrom", optarg) == 0)
> +res->bootdevice = VMBOOTDEV_CDROM;
> +else if (strcmp("net", optarg) == 0)
>res->bootdevice = VMBOOTDEV_NET;
> +else
> +errx(1, "unknown boot device %s", optarg);
>break;
>case 'r':
>if (res->isopath)
> 

Re: carp though bridge with vmd

[Reyk Floeter]

Hi,

as a general note for virtual switches and clouds that don’t support CARP due 
to restrictions on multicast and/or additional MACs: I use carppeer and lladdr 
of the parent interface in such cases.

That doesn’t mean that you should need it with vmd and bridge and we have to 
look into this.

Reyk

> Am 09.12.2018 um 16:56 schrieb Mischa :
> 
> Hi All,
> 
> Is there a way to get carp working through a bridge?
> I am currently testing to see whether I can have 2 vmd VMs on different hosts 
> use carp between them.
> The current state that I am currently at is, both VMs are master.
> 
> Setup on both hosts is the same, bridge1 with em0 as interface.
> 
> # vm.conf
> switch "uplink_bridge1" {
>interface bridge1
> }
> vm "lb1" {
>disable
>disk "/home/mischa/vmm/lb1.img"
>interface tap {
>switch "uplink_bridge1"
>}
> }
> 
> lb1 carp config:
> inet 192.168.0.100 255.255.255.0 NONE vhid 1 pass  carpdev vio0 advbase 
> 10 advskew 100
> 
> lb2 carp config:
> inet 192.168.0.100 255.255.255.0 NONE vhid 1 pass  carpdev vio0 advbase 
> 10 advskew 110
> 
> Is there anything that can be configured on the bridge side?
> 
> Mischa
> 

Re: change reboot behaviour in vmd

[Reyk Floeter]

Sure, the bootdevice trick makes sense. Another trick could probably be to try 
to switch to the hard disk after first boot (install) but this would make the 
logic and config a bit ugly.

OK for the diff below.

The only thing that I’m a bit concerned about is that -B might turn a bit into 
a qemu-like getopt madness, especially if it would gain boot order or something 
like this. Please don’t complicate it further. But it still looks fine.

And -B should have options vm.conf. vm.conf should always have the same or more 
and not less options. The vm.conf grammar, on the other hand, allows to add 
more complex configurations.

Reyk

> Am 07.12.2018 um 20:55 schrieb Carlos Cardenas :
> 
>> On Thu, Dec 06, 2018 at 10:33:24AM +0100, Claudio Jeker wrote:
>> So doing autoinstall with -B net is great but one thing I was missing is
>> changing the reboot behaviour of vmd to exit at a guest reboot.
>> I came up with this minimal diff that does the trick for me. Now maybe it
>> would be better to have a proper flag for this instead of overloading 
>> vmc_bootdevice with it.
>> 
>> What is the preferred way of doing this?
>> -- 
>> :wq Claudio
> 
> I'm ok with this. 
> 
> reyk@, what are your thoughts?
> 
> +--+
> Carlos
> 
>> 
>> Index: vmd.c
>> ===
>> RCS file: /cvs/src/usr.sbin/vmd/vmd.c,v
>> retrieving revision 1.107
>> diff -u -p -r1.107 vmd.c
>> --- vmd.c4 Dec 2018 08:15:09 -1.107
>> +++ vmd.c4 Dec 2018 09:11:51 -
>> @@ -452,7 +452,8 @@ vmd_dispatch_vmm(int fd, struct privsep_
>>__func__, vmr.vmr_id);
>>break;
>>}
>> -if (vmr.vmr_result != EAGAIN) {
>> +if (vmr.vmr_result != EAGAIN ||
>> +vm->vm_params.vmc_bootdevice) {
>>if (vm->vm_from_config)
>>vm_stop(vm, 0, __func__);
>>else
>> 

Re: pvclock(4)

[Reyk Floeter]

On Tue, Dec 04, 2018 at 05:43:48AM -0800, Chris Cappuccio wrote:
> Of course printf instead of panic for testers 
> 

Oh, right, thanks!

@john:  Does this "slightly less simple" diff work for you?

@phessler, Chris: Maybe we should get this fix tested and in, wait for
reports, and I can use the time to think about my other option.  What
do you think?

Reyk

Index: sys/dev/pv/pvclock.c
===
RCS file: /cvs/src/sys/dev/pv/pvclock.c,v
retrieving revision 1.2
diff -u -p -u -p -r1.2 pvclock.c
--- sys/dev/pv/pvclock.c24 Nov 2018 13:12:29 -  1.2
+++ sys/dev/pv/pvclock.c4 Dec 2018 14:03:57 -
@@ -70,6 +70,11 @@ uint  pvclock_get_timecount(struct timec
 voidpvclock_read_time_info(struct pvclock_softc *,
struct pvclock_time_info *);
 
+static inline uint32_t
+pvclock_read_begin(const struct pvclock_time_info *);
+static inline int
+pvclock_read_done(const struct pvclock_time_info *, uint32_t);
+
 struct cfattach pvclock_ca = {
sizeof(struct pvclock_softc),
pvclock_match,
@@ -127,8 +132,11 @@ pvclock_match(struct device *parent, voi
 void
 pvclock_attach(struct device *parent, struct device *self, void *aux)
 {
-   struct pvclock_softc*sc = (struct pvclock_softc *)self;
-   paddr_t  pa;
+   struct pvclock_softc*sc = (struct pvclock_softc *)self;
+   struct pvclock_time_info*ti;
+   paddr_t  pa;
+   uint32_t version;
+   uint8_t  flags;
 
if ((sc->sc_time = km_alloc(PAGE_SIZE,
&kv_any, &kp_zero, &kd_nowait)) == NULL) {
@@ -143,6 +151,19 @@ pvclock_attach(struct device *parent, st
 
wrmsr(KVM_MSR_SYSTEM_TIME, pa | PVCLOCK_SYSTEM_TIME_ENABLE);
sc->sc_paddr = pa;
+
+   ti = sc->sc_time;
+   do {
+   version = pvclock_read_begin(ti);
+   flags = ti->ti_flags;
+   } while (!pvclock_read_done(ti, version));
+
+   if ((flags & PVCLOCK_FLAG_TSC_STABLE) == 0) {
+   wrmsr(KVM_MSR_SYSTEM_TIME, pa & ~PVCLOCK_SYSTEM_TIME_ENABLE);
+   km_free(sc->sc_time, PAGE_SIZE, &kv_any, &kp_zero);
+   printf(": unstable clock\n");
+   return;
+   }
 
sc->sc_tc = &pvclock_timecounter;
sc->sc_tc->tc_name = DEVNAME(sc);

Re: pvclock(4)

[Reyk Floeter]

On Tue, Dec 04, 2018 at 12:46:06PM +0100, Peter Hessler wrote:
> On 2018 Dec 03 (Mon) at 16:56:10 -0800 (-0800), Chris Cappuccio wrote:
> :Reyk Floeter [r...@openbsd.org] wrote:
> :>
> :> Yes, KVM???s stable bit is not a reliable indication as it is seems to 
> depend on the capabilities of the KVM version and not the actual availability 
> of the feature on the particular hardware. How annoying.
> :>
> :> As mentioned before: I???d like to disable pvclock for now and I can do 
> that in the morning CET if nobody beats me to it.
> :>
> :> I have an idea how to deal with old platforms afterwards but this needs 
> some more tests and thoughts.
> :>
> :
> :Perhaps the solution is as "simple" as checking the status of the bit
> :after the presence of the bit is established ?
> :
> 
> This makes sense, OK
> 

See my other mail, this diff is not OK.

Reyk

> 
> 
> 
> :Index: pvclock.c
> :===
> :RCS file: /cvs/src/sys/dev/pv/pvclock.c,v
> :retrieving revision 1.2
> :diff -u -p -u -r1.2 pvclock.c
> :--- pvclock.c24 Nov 2018 13:12:29 -  1.2
> :+++ pvclock.c4 Dec 2018 00:53:56 -
> :@@ -127,8 +127,10 @@ pvclock_match(struct device *parent, voi
> : void
> : pvclock_attach(struct device *parent, struct device *self, void *aux)
> : {
> :-struct pvclock_softc*sc = (struct pvclock_softc *)self;
> :-paddr_t  pa;
> :+struct pvclock_softc*sc = (struct pvclock_softc *)self;
> :+struct pvclock_time_info*ti;
> :+paddr_t  pa;
> :+uint8_t  flags;
> : 
> : if ((sc->sc_time = km_alloc(PAGE_SIZE,
> : &kv_any, &kp_zero, &kd_nowait)) == NULL) {
> :@@ -151,6 +153,13 @@ pvclock_attach(struct device *parent, st
> : 
> : /* Better than HPET but below TSC */
> : sc->sc_tc->tc_quality = 1500;
> :+
> :+ti = sc->sc_time;
> :+flags = ti->ti_flags;
> :+if ((flags & PVCLOCK_FLAG_TSC_STABLE) == 0) {
> :+printf(": unstable timestamp counter\n");
> :+return;
> :+}
> : 
> : tc_init(sc->sc_tc);
> : 
> :
> 
> -- 
> fortune -as

Re: pvclock(4)

[Reyk Floeter]

On Mon, Dec 03, 2018 at 04:56:10PM -0800, Chris Cappuccio wrote:
> Reyk Floeter [r...@openbsd.org] wrote:
> >
> > Yes, KVM???s stable bit is not a reliable indication as it is seems to 
> > depend on the capabilities of the KVM version and not the actual 
> > availability of the feature on the particular hardware. How annoying.
> >
> > As mentioned before: I???d like to disable pvclock for now and I can do 
> > that in the morning CET if nobody beats me to it.
> >
> > I have an idea how to deal with old platforms afterwards but this needs 
> > some more tests and thoughts.
> >
> 
> Perhaps the solution is as "simple" as checking the status of the bit
> after the presence of the bit is established ?
> 

The approach makes sense but it is not the right way to read the
value.  It only works if you don't hit a refresh cycle of the host.
Better diff below.

But I'm not sure if this is enough and if the "stable" flag mit appear
occasionally.  So I'd like to disable pvclock for now until we did
some more testing and tried different options.

Reyk

Index: sys/dev/pv/pvclock.c
===
RCS file: /cvs/src/sys/dev/pv/pvclock.c,v
retrieving revision 1.2
diff -u -p -u -p -r1.2 pvclock.c
--- sys/dev/pv/pvclock.c24 Nov 2018 13:12:29 -  1.2
+++ sys/dev/pv/pvclock.c4 Dec 2018 12:09:06 -
@@ -70,6 +70,11 @@ uint  pvclock_get_timecount(struct timec
 voidpvclock_read_time_info(struct pvclock_softc *,
struct pvclock_time_info *);
 
+static inline uint32_t
+pvclock_read_begin(const struct pvclock_time_info *);
+static inline int
+pvclock_read_done(const struct pvclock_time_info *, uint32_t);
+
 struct cfattach pvclock_ca = {
sizeof(struct pvclock_softc),
pvclock_match,
@@ -127,8 +132,11 @@ pvclock_match(struct device *parent, voi
 void
 pvclock_attach(struct device *parent, struct device *self, void *aux)
 {
-   struct pvclock_softc*sc = (struct pvclock_softc *)self;
-   paddr_t  pa;
+   struct pvclock_softc*sc = (struct pvclock_softc *)self;
+   struct pvclock_time_info*ti;
+   paddr_t  pa;
+   uint32_t version;
+   uint8_t  flags;
 
if ((sc->sc_time = km_alloc(PAGE_SIZE,
&kv_any, &kp_zero, &kd_nowait)) == NULL) {
@@ -143,6 +151,19 @@ pvclock_attach(struct device *parent, st
 
wrmsr(KVM_MSR_SYSTEM_TIME, pa | PVCLOCK_SYSTEM_TIME_ENABLE);
sc->sc_paddr = pa;
+
+   ti = sc->sc_time;
+   do {
+   version = pvclock_read_begin(ti);
+   flags = ti->ti_flags;
+   } while (!pvclock_read_done(ti, version));
+
+   if ((flags & PVCLOCK_FLAG_TSC_STABLE) == 0) {
+   wrmsr(KVM_MSR_SYSTEM_TIME, pa & ~PVCLOCK_SYSTEM_TIME_ENABLE);
+   km_free(sc->sc_time, PAGE_SIZE, &kv_any, &kp_zero);
+   panic(": unstable clock\n");
+   return;
+   }
 
sc->sc_tc = &pvclock_timecounter;
sc->sc_tc->tc_name = DEVNAME(sc);

Re: pvclock(4)

[Reyk Floeter]

> Am 04.12.2018 um 00:52 schrieb Chris Cappuccio :
> 
> johnw [johnw.m...@gmail.com] wrote:
>> 
>> Hi, after disable pvclock, it can boot with new kernel again, thanks.
> ...
>> cpu0: Intel(R) Core(TM)2 Duo CPU E8400 @ 3.00GHz, 105.29 MHz, 06-17-0a
>> cpu0: 
>> FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,MMX,FXSR,SSE,SSE2,SS,SSE3,SSSE3,CX16,SSE4.1,x2APIC,DEADLINE,XSAVE,HV,NXE,LONG,LAHF,PERF,ARAT,MELTDOWN
> ...
> 
> This CPU clearly doesn't have the invariant TSC (it is required according
> to Reyk Floeter's tech@ posting from Nov 19th.) So, pvclock does not handle
> this situation properly, apparently checking for KVM_FEATURE_CLOCKSOURCE2
> and KVM_FEATURE_CLOCKSOURCE_STABLE_BIT is not enough. 

Yes, KVM’s stable bit is not a reliable indication as it is seems to depend on 
the capabilities of the KVM version and not the actual availability of the 
feature on the particular hardware. How annoying.

As mentioned before: I‘d like to disable pvclock for now and I can do that in 
the morning CET if nobody beats me to it.

I have an idea how to deal with old platforms afterwards but this needs some 
more tests and thoughts.

Reyk

Re: pvclock(4)

[Reyk Floeter]

Hi,

> Am 29.11.2018 um 05:27 schrieb johnw :
> 
> 
>> So far I only got positive reports.  Where are the problems? ;)
> 
>> Otherwise: OK?
> 
>> Reyk
> 
> Hi, my kvm/quest/openbsd-amd64 can not boot, after upgrade to today current 
> (28-nov-2018).
> 

thanks for reporting.

Do you have a full dmesg for me?

Reyk

> It work before upgrade (OpenBSD 6.4-current (GENERIC.MP) #447: Sun Nov 18 
> 17:25:58 MST 2018)
> 
> Host: 4.18.0-3-amd64 #1 SMP Debian 4.18.20-2 (2018-11-23) x86_64 GNU/Linux
> qemu: QEMU emulator version 2.12.0 (Debian 1:2.12+dfsg-3+b1)
> panic photo: https://ibb.co/gjWBhL4
> panic photo: https://ibb.co/HDrJ7Q0
> panic photo: https://ibb.co/g96J5s7
> 
> Thanks.
> 
> -- 
> Key fingerprint: CDB3 6C62 254B C088 1E5D DD32 182C 97DB CF2C 80AC

Re: pvclock(4)

[Reyk Floeter]

> Am 25.11.2018 um 05:02 schrieb Greg Steuck :
> 
> I realize this report is practically useless, but better out than in 
> (according to Shrek).
> I found this in the logs of my GCE VM running syzkaller bot. No further 
> details were preserved...
> 
> 2018/11/24 09:53:48 ci-openbsd-main: poll: 
> 94bf4886dbb69e9fbf0f92f975fc23f16fc5c80f
> 2018/11/24 09:53:48 ci-openbsd-main: building kernel...
> 2018/11/24 09:54:03 ci-openbsd-main: testing image...
> 2018/11/24 10:04:07 ci-openbsd-main: VM boot failed with: panic: pvclock0: 
> unstable result on stable clock
> 
> The host is running
> OpenBSD 6.4-current (GENERIC.MP) #456: Tue Nov 20 08:46:59 MST 2018
> dera...@amd64.openbsd.org:/usr/src/sys/arch/amd64/compile/GENERIC.MP
> 
> The VM kernel at the time was built at "zap 10 tab leading whitespace before 
> 'struct evp_pkey_ctx_st {'", so maybe "only attach pvclock(4) inside a KVM 
> guest" would've fixed it?

Yes, correct. Sorry for that glitch.

Reyk

Re: typo in vxlan.4

[Reyk Floeter]

On Thu, Nov 22, 2018 at 11:50:52AM -0500, Josh Grosse wrote:
> The page refers to vmx0 rather than vxlan0 in its pf.conf
> guidance.
> 

... I committed a change to use em0 instead of vmx0.

vmx0 wasn't a good example (I wrote the manpage around the time when
vmx(4) was new and I used it as a parent for vxlan(4)).

Thanks for the report!

Reyk

> Index: vxlan.4
> ===
> RCS file: /systems/cvs/src/share/man/man4/vxlan.4,v
> retrieving revision 1.7
> diff -u -p -r1.7 vxlan.4
> --- vxlan.4   22 Feb 2018 01:35:04 -  1.7
> +++ vxlan.4   22 Nov 2018 16:43:37 -
> @@ -123,9 +123,9 @@ endpoint policies in
>  .Xr pf.conf 5 :
>  .Bd -literal -offset indent
>  table  { 192.168.1.200, 192.168.1.201 }
> -block in on vmx0
> -pass out on vmx0
> -pass in on vmx0 proto udp from  to port vxlan
> +block in on vxlan0
> +pass out on vxlan0
> +pass in on vxlan0 proto udp from  to port vxlan
>  .Ed
>  .Pp
>  The Time-to-Live (TTL) value of the tunnel can be set to 1 or a low
> 

Re: typo in vxlan.4

[Reyk Floeter]

On Thu, Nov 22, 2018 at 11:50:52AM -0500, Josh Grosse wrote:
> The page refers to vmx0 rather than vxlan0 in its pf.conf
> guidance.
> 

Are you sure?  In this example, vmx0 is the parent interface that
receives VXLAN UDP packets - it could also be em0, ix0, or whatever
you like.

Reyk

> Index: vxlan.4
> ===
> RCS file: /systems/cvs/src/share/man/man4/vxlan.4,v
> retrieving revision 1.7
> diff -u -p -r1.7 vxlan.4
> --- vxlan.4   22 Feb 2018 01:35:04 -  1.7
> +++ vxlan.4   22 Nov 2018 16:43:37 -
> @@ -123,9 +123,9 @@ endpoint policies in
>  .Xr pf.conf 5 :
>  .Bd -literal -offset indent
>  table  { 192.168.1.200, 192.168.1.201 }
> -block in on vmx0
> -pass out on vmx0
> -pass in on vmx0 proto udp from  to port vxlan
> +block in on vxlan0
> +pass out on vxlan0
> +pass in on vxlan0 proto udp from  to port vxlan
>  .Ed
>  .Pp
>  The Time-to-Live (TTL) value of the tunnel can be set to 1 or a low
> 

Re: pvclock(4)

[Reyk Floeter]

On Mon, Nov 19, 2018 at 01:12:46PM +0100, Reyk Floeter wrote:
> the attached diff is another attempt at implementing a pvclock(4)
> guest driver.  This improves the clock on KVM and replaces the need
> for using the VM-expensive acpihpet(4).
> 

So far I only got positive reports.  Where are the problems? ;)

Otherwise: OK?

Reyk

> Index: share/man/man4/pvclock.4
> ===
> RCS file: share/man/man4/pvclock.4
> diff -N share/man/man4/pvclock.4
> --- /dev/null 1 Jan 1970 00:00:00 -
> +++ share/man/man4/pvclock.4  19 Nov 2018 11:48:33 -
> @@ -0,0 +1,45 @@
> +.\"  $OpenBSD$
> +.\"
> +.\" Copyright (c) 2018 Reyk Floeter 
> +.\"
> +.\" Permission to use, copy, modify, and distribute this software for any
> +.\" purpose with or without fee is hereby granted, provided that the above
> +.\" copyright notice and this permission notice appear in all copies.
> +.\"
> +.\" THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES
> +.\" WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF
> +.\" MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR
> +.\" ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES
> +.\" WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN
> +.\" ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF
> +.\" OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
> +.\"
> +.Dd $Mdocdate$
> +.Dt PVCLOCK 4
> +.Os
> +.Sh NAME
> +.Nm pvclock
> +.Nd paravirtual clock driver
> +.Sh SYNOPSIS
> +.Cd "pvclock* at pvbus?
> +.Sh DESCRIPTION
> +The
> +.Nm
> +driver supports the paravirtual clock that is available in KVM and
> +other hypervisors.
> +.Nm
> +uses a shared page between the host and the hypervisor to synchronize
> +the TSC clock in an efficient way.
> +.Sh SEE ALSO
> +.Xr pvbus 4
> +.Sh HISTORY
> +The
> +.Nm
> +driver first appeared in
> +.Ox 6.5 .
> +.Sh AUTHORS
> +.An -nosplit
> +The
> +.Nm
> +driver was written by
> +.An Reyk Floeter Aq Mt r...@openbsd.org .
> Index: sys/arch/amd64/conf/GENERIC
> ===
> RCS file: /cvs/src/sys/arch/amd64/conf/GENERIC,v
> retrieving revision 1.464
> diff -u -p -u -p -r1.464 GENERIC
> --- sys/arch/amd64/conf/GENERIC   26 Oct 2018 20:26:19 -  1.464
> +++ sys/arch/amd64/conf/GENERIC   19 Nov 2018 11:48:33 -
> @@ -79,6 +79,8 @@ ipmi0   at mainbus? disable # IPMI
>  
>  vmt0 at pvbus?   # VMware Tools
>  
> +pvclock0 at pvbus?   # KVM pvclock
> +
>  xen0 at pvbus?   # Xen HVM domU
>  xnf* at xen? # Xen Netfront
>  xbf* at xen? # Xen Blkfront
> Index: sys/dev/pv/files.pv
> ===
> RCS file: /cvs/src/sys/dev/pv/files.pv,v
> retrieving revision 1.14
> diff -u -p -u -p -r1.14 files.pv
> --- sys/dev/pv/files.pv   24 Aug 2018 16:07:01 -  1.14
> +++ sys/dev/pv/files.pv   19 Nov 2018 11:48:33 -
> @@ -8,6 +8,11 @@ device   pvbus
>  attach   pvbus at mainbus
>  file dev/pv/pvbus.c  pvbus   needs-flag
>  
> +# KVM clock
> +device   pvclock
> +attach   pvclock at pvbus
> +file dev/pv/pvclock.cpvclock needs-flag
> +
>  # VMware Tools
>  device   vmt
>  attach       vmt at pvbus
> Index: sys/dev/pv/pvclock.c
> ===
> RCS file: sys/dev/pv/pvclock.c
> diff -N sys/dev/pv/pvclock.c
> --- /dev/null 1 Jan 1970 00:00:00 -
> +++ sys/dev/pv/pvclock.c  19 Nov 2018 11:48:33 -
> @@ -0,0 +1,229 @@
> +/*   $OpenBSD$   */
> +
> +/*
> + * Copyright (c) 2018 Reyk Floeter 
> + *
> + * Permission to use, copy, modify, and distribute this software for any
> + * purpose with or without fee is hereby granted, provided that the above
> + * copyright notice and this permission notice appear in all copies.
> + *
> + * THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES
> + * WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF
> + * MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR
> + * ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES
> + * WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN
> + * ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF
> + * OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS

Re: rad: add support for listening on interface groups

[Reyk Floeter]

On Sat, Nov 17, 2018 at 02:11:58PM +0100, Klemens Nanni wrote:
> On Fri, Nov 16, 2018 at 08:56:52PM +0100, Reyk Floeter wrote:
> > > the following diff allows rad(8) to watch interface groups.  This
> > > allows to automatically add/remove interfaces in a given group.
> > > 
> > > For example, I put "interface tap" into rad.conf and it automatically
> > > serves my VM interfaces.  You could also configure a custom group in
> > > vm.conf and rad.conf.  I'm working on IPv6 for vmd that needs it.
> Nice idea/work, I like it.
> 
> > > For reasons that I don't remember, I always missed this feature in
> > > rtadvd(8)[RIP].  It was amazingly simple to add it to rad(8) as it
> > > already reinitializes itself on interface changes.
> > > 
> > > This diff includes the previous ENXIO fix to prevent it from crashing
> > > when a cloner interface such as tap is destroyed.
> > > 
> > 
> > Updated diff after committing the ENXIO fix.
> > 
> > Two explanations:
> > 
> > - merge_ra_interfaces() calls merge_ra_interface() for each configured
> > interface (explicit config) and for each member that is found in an
> > interface group.  It is basically just some code shuffling.
> > 
> > - ra_iface->name is split into ra_iface->name and ra_iface->conf as
> > "name" indicates and actual (found) interface name ("tap0") and "conf"
> > is used to lookup the config where it was derived from ("tap" or
> > "tap0", depending on the configuration).
> This begs the question about priority in rad.conf(5). It's always been
> possible to define duplicate `interface' blocks although it didn't make
> any sense.
> 
> Now it does since you could define a default config for the entire group
> and add further interface specific blocks.
> 
> As it currently is, the first matching block wins, so vether0 would
> always get RDNS ::11 whether a more specific block exists or not:
> 
>   interface vether  { dns { nameserver ::11 } }
>   interface vether0 { dns { nameserver ::22 } }
> 
> Maybe we should clarify behaviour in rad.conf(5) regardless of your
> changes?
> 

Updated diff:
- less indentation in merge_ra_interface()
- manpage with additional clarification about first match

OK?

Reyk

Index: usr.sbin/rad/frontend.c
===
RCS file: /cvs/src/usr.sbin/rad/frontend.c,v
retrieving revision 1.17
diff -u -p -u -p -r1.17 frontend.c
--- usr.sbin/rad/frontend.c 16 Nov 2018 19:45:40 -  1.17
+++ usr.sbin/rad/frontend.c 20 Nov 2018 15:21:16 -
@@ -70,6 +70,7 @@
 #include 
 #include 
 
+#include 
 #include 
 #include 
 #include 
@@ -101,6 +102,7 @@ struct ra_iface {
TAILQ_ENTRY(ra_iface)   entry;
struct ra_prefix_conf_head  prefixes;
charname[IF_NAMESIZE];
+   charconf[IF_NAMESIZE];
uint32_tif_index;
int removed;
int prefix_count;
@@ -116,6 +118,7 @@ void frontend_startup(void);
 voidicmp6_receive(int, short, void *);
 voidjoin_all_routers_mcast_group(struct ra_iface *);
 voidleave_all_routers_mcast_group(struct ra_iface *);
+voidmerge_ra_interface(struct ra_iface_conf *, char *);
 voidmerge_ra_interfaces(void);
 struct ra_iface*find_ra_iface_by_id(uint32_t);
 struct ra_iface*find_ra_iface_by_name(char *);
@@ -688,36 +691,81 @@ find_ra_iface_conf(struct ra_iface_conf_
 }
 
 void
+merge_ra_interface(struct ra_iface_conf *ra_iface_conf, char *name)
+{
+   struct ra_iface *ra_iface;
+   uint32_t if_index;
+
+   if ((ra_iface = find_ra_iface_by_name(name)) != NULL) {
+   log_debug("keeping interface %s", name);
+   ra_iface->removed = 0;
+   return;
+   }
+
+   log_debug("new interface %s", name);
+   if ((if_index = if_nametoindex(name)) == 0)
+   return;
+   log_debug("adding interface %s", name);
+   if ((ra_iface = calloc(1, sizeof(*ra_iface))) == NULL)
+   fatal("%s", __func__);
+
+   strlcpy(ra_iface->name, name, sizeof(ra_iface->name));
+   strlcpy(ra_iface->conf, ra_iface_conf->name,
+   sizeof(ra_iface->conf));
+
+   ra_iface->if_index = if_index;
+   SIMPLEQ_INIT(&ra_iface->prefixes);
+   TAILQ_INSERT_TAIL(&ra_interfaces, ra_iface, entry);
+ 

Re: vmd: add support for local inet6 interfaces

[Reyk Floeter]

Hi

On Fri, Nov 16, 2018 at 05:35:03PM +0100, Reyk Floeter wrote:
> "local interface" (-L) is an amazing feature and I use it every day;
> but it is IPv4-only and now I realized that I need IPv6 too.
> 
> The attached diff implements IPv6 support for local interfaces.
> 

Updated diff with feedback from kn@ and ccardenas@ (thanks, Carlos,
for testing!) with the following changes:

- keep the auto-generated local inet6 prefix across reloads.
- clarify the manpage describing the fd00::/8 prefix
- comments and minor things

OK?

Reyk

Index: usr.sbin/vmd/config.c
===
RCS file: /cvs/src/usr.sbin/vmd/config.c,v
retrieving revision 1.54
diff -u -p -u -p -r1.54 config.c
--- usr.sbin/vmd/config.c   26 Oct 2018 11:24:45 -  1.54
+++ usr.sbin/vmd/config.c   20 Nov 2018 14:22:49 -
@@ -42,17 +42,44 @@
 /* Supported bridge types */
 const char *vmd_descsw[] = { "switch", "bridge", NULL };
 
+static int  config_init_localprefix(struct vmd_config *);
+
+static int
+config_init_localprefix(struct vmd_config *cfg)
+{
+   struct sockaddr_in6 *sin6;
+
+   if (host(VMD_DHCP_PREFIX, &cfg->cfg_localprefix) == -1)
+   return (-1);
+
+   /* IPv6 is disabled by default */
+   cfg->cfg_flags &= ~VMD_CFG_INET6;
+
+   /* Generate random IPv6 prefix only once */
+   if (cfg->cfg_flags & VMD_CFG_AUTOINET6)
+   return (0);
+   if (host(VMD_ULA_PREFIX, &cfg->cfg_localprefix6) == -1)
+   return (-1);
+   /* Randomize the 56 bits "Global ID" and "Subnet ID" */
+   sin6 = ss2sin6(&cfg->cfg_localprefix6.ss);
+   arc4random_buf(&sin6->sin6_addr.s6_addr[1], 7);
+   cfg->cfg_flags |= VMD_CFG_AUTOINET6;
+
+   return (0);
+}
+
 int
 config_init(struct vmd *env)
 {
-   struct privsep  *ps = &env->vmd_ps;
-   unsigned int what;
+   struct privsep  *ps = &env->vmd_ps;
+   unsigned int what;
 
/* Global configuration */
ps->ps_what[PROC_PARENT] = CONFIG_ALL;
ps->ps_what[PROC_VMM] = CONFIG_VMS;
 
-   if (host(VMD_DHCP_PREFIX, &env->vmd_cfg.cfg_localprefix) == -1)
+   /* Local prefix */
+   if (config_init_localprefix(&env->vmd_cfg) == -1)
return (-1);
 
/* Other configuration */
@@ -90,7 +117,7 @@ config_purge(struct vmd *env, unsigned i
__func__, ps->ps_title[privsep_process]);
 
/* Reset global configuration (prefix was verified before) */
-   (void)host(VMD_DHCP_PREFIX, &env->vmd_cfg.cfg_localprefix);
+   config_init_localprefix(&env->vmd_cfg);
 
/* Reset other configuration */
what = ps->ps_what[privsep_process] & reset;
Index: usr.sbin/vmd/dhcp.c
===
RCS file: /cvs/src/usr.sbin/vmd/dhcp.c,v
retrieving revision 1.5
diff -u -p -u -p -r1.5 dhcp.c
--- usr.sbin/vmd/dhcp.c 17 Aug 2018 07:12:28 -  1.5
+++ usr.sbin/vmd/dhcp.c 20 Nov 2018 14:22:49 -
@@ -109,7 +109,7 @@ dhcp_request(struct vionet_dev *dev, cha
resp.xid = req.xid;
 
if ((client_addr.s_addr =
-   vm_priv_addr(&env->vmd_cfg.cfg_localprefix,
+   vm_priv_addr(&env->vmd_cfg,
dev->vm_vmid, dev->idx, 1)) == 0)
return (-1);
memcpy(&resp.yiaddr, &client_addr,
@@ -119,7 +119,7 @@ dhcp_request(struct vionet_dev *dev, cha
ss2sin(&pc.pc_dst)->sin_port = htons(CLIENT_PORT);
 
if ((server_addr.s_addr =
-   vm_priv_addr(&env->vmd_cfg.cfg_localprefix,
+   vm_priv_addr(&env->vmd_cfg,
dev->vm_vmid, dev->idx, 0)) == 0)
return (-1);
memcpy(&resp.siaddr, &server_addr,
Index: usr.sbin/vmd/parse.y
===
RCS file: /cvs/src/usr.sbin/vmd/parse.y,v
retrieving revision 1.48
diff -u -p -u -p -r1.48 parse.y
--- usr.sbin/vmd/parse.y1 Nov 2018 00:18:44 -   1.48
+++ usr.sbin/vmd/parse.y20 Nov 2018 14:22:49 -
@@ -120,9 +120,9 @@ typedef struct {
 
 
 %token INCLUDE ERROR
-%token ADD ALLOW BOOT CDROM DISABLE DISK DOWN ENABLE FORMAT GROUP INSTANCE
-%token INTERFACE LLADDR LOCAL LOCKED MEMORY NIFS OWNER PATH PREFIX RDOMAIN
-%token SIZE SOCKET SWITCH UP VM VMID
+%token ADD ALLOW BOOT CDROM DISABLE DISK DOWN ENABLE FORMAT GROUP INET6
+%token INSTANCE INTERFACE LLADDR LOCAL LOCKED MEMORY NIFS OWNER PATH PREFIX
+%token RDOMAIN SIZE SOCKET SWITCH UP VM VMID
 %token   NUMBER
 %token   STRING
 %typelladdr
@@ -181,10 +181,27 @@ varset: STRING '=' STRING {
}
;
 
-main   : LOCAL PREFIX STRING {
+main   

pvclock(4)

[Reyk Floeter]

Hi,

the attached diff is another attempt at implementing a pvclock(4)
guest driver.  This improves the clock on KVM and replaces the need
for using the VM-expensive acpihpet(4).

While pvclock(4) is available on KVM, Xen, Hyper-V (in a modified
form), it currently only attaches under KVM:

pvbus0 at mainbus0: KVM
pvclock0 at pvbus0

A few notes:

- The "invtsc" workaround to get a real TSC is not always possible, as
it breaks migration and a few other things.  If you're running in a
cloud, it is very unlikely that you'll get it enabled.

- pvclock suffered from the same problem as early TSC implementations:
it wasn't synced across CPUs, so the OS had to do it manually.  In
OpenBSD, we never supported that and we decided to only support the
newer "invariant" TSCs that started to show up around SkyLake.  And I
think that's why the previous pvlock implementations for OpenBSD
didn't take off.

- So my diff does the same for pvclock what we did for TSC: only
support the "stable" pvclock that guarantees to provide a synchronized
clock by either doing the synchronization on the host side or by
providing it based on an invariant TSC on the host itself.  So this
diff might not work on older KVM versions or CPUs; I'd be happy to see
some test results.

- I did not copy+paste from the Linux and/or FreeBSD (they have
suprising similarities but different licenses), and I wasn't convinced
that it is worth copying the asm code that they use instead of the
following multiplication:
ctr = ((delta * mul_frac) >> 32) + system_time;
Let me know if I'm wrong and if this code provides significant benefits:
https://github.com/freebsd/freebsd/blob/1d6e4247415d264485ee94b59fdbc12e0c566fd0/sys/x86/x86/pvclock.c#L90-L124

- This driver only implements the "system time" clock, and not the
static "wall clock".  I didn't see a point in supporting the wall
clock as its timestamp is only ever updated when you cold-start the
VM.  The wall clock struct is still defined in the driver for
reference but could probably be removed.

- I assigned a timecounter priority of 1500 to be in the middle
between acpihpet(1000) and tsc(2000).  If TSC is available, we should
probably use it instead; even if pvclock is theoretically better as it
is TSC-based and optimized for VMs.

- This is only a guest driver, not a driver for vmm(4)/vmd(8).  But it
could solve some issues if somebody would implement it for vmm(4)...

-  I'm running it on a busy build machine on Exoscale.ch for at least
a week now.  This machine runs stuff that utilizes all CPUs.  With
HPET, we've sometimes experienced livelocks but the pvclock(4) driver
made it much snappier.

$ time sleep 2
0m02.00s real 0m00.00s user 0m00.01s system
$ cat /var/db/ntpd.drift  
-4.679
$ sysctl kern.timecounter 
kern.timecounter.tick=1
kern.timecounter.timestepwarnings=1
kern.timecounter.hardware=pvclock0
kern.timecounter.choice=i8254(0) pvclock0(1500) acpihpet0(1000) 
acpitimer0(1000) dummy(-100)
$ sysctl hw.model hw.ncpuonline hw.vendor hw.product hw.version
hw.model=Intel Xeon Processor (Skylake)
hw.ncpuonline=8
hw.vendor=Apache Software Foundation
hw.product=CloudStack KVM Hypervisor
hw.version=pc-i440fx-2.11

Feedback? Tests? OKs?

Reyk

Index: share/man/man4/pvclock.4
===
RCS file: share/man/man4/pvclock.4
diff -N share/man/man4/pvclock.4
--- /dev/null   1 Jan 1970 00:00:00 -
+++ share/man/man4/pvclock.419 Nov 2018 11:48:33 -
@@ -0,0 +1,45 @@
+.\"$OpenBSD$
+.\"
+.\" Copyright (c) 2018 Reyk Floeter 
+.\"
+.\" Permission to use, copy, modify, and distribute this software for any
+.\" purpose with or without fee is hereby granted, provided that the above
+.\" copyright notice and this permission notice appear in all copies.
+.\"
+.\" THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES
+.\" WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF
+.\" MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR
+.\" ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES
+.\" WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN
+.\" ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF
+.\" OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
+.\"
+.Dd $Mdocdate$
+.Dt PVCLOCK 4
+.Os
+.Sh NAME
+.Nm pvclock
+.Nd paravirtual clock driver
+.Sh SYNOPSIS
+.Cd "pvclock* at pvbus?
+.Sh DESCRIPTION
+The
+.Nm
+driver supports the paravirtual clock that is available in KVM and
+other hypervisors.
+.Nm
+uses a shared page between the host and the hypervisor to synchronize
+the TSC clock in an efficient way.
+.Sh 

Re: rad: add support for listening on interface groups

[Reyk Floeter]

On Fri, Nov 16, 2018 at 04:41:12PM +0100, Reyk Floeter wrote:
> Hi,
> 
> the following diff allows rad(8) to watch interface groups.  This
> allows to automatically add/remove interfaces in a given group.
> 
> For example, I put "interface tap" into rad.conf and it automatically
> serves my VM interfaces.  You could also configure a custom group in
> vm.conf and rad.conf.  I'm working on IPv6 for vmd that needs it.
> 
> For reasons that I don't remember, I always missed this feature in
> rtadvd(8)[RIP].  It was amazingly simple to add it to rad(8) as it
> already reinitializes itself on interface changes.
> 
> This diff includes the previous ENXIO fix to prevent it from crashing
> when a cloner interface such as tap is destroyed.
> 

Updated diff after committing the ENXIO fix.

Two explanations:

- merge_ra_interfaces() calls merge_ra_interface() for each configured
interface (explicit config) and for each member that is found in an
interface group.  It is basically just some code shuffling.

- ra_iface->name is split into ra_iface->name and ra_iface->conf as
"name" indicates and actual (found) interface name ("tap0") and "conf"
is used to lookup the config where it was derived from ("tap" or
"tap0", depending on the configuration).

Reyk

Index: usr.sbin/rad/frontend.c
===
RCS file: /cvs/src/usr.sbin/rad/frontend.c,v
retrieving revision 1.17
diff -u -p -u -p -r1.17 frontend.c
--- usr.sbin/rad/frontend.c 16 Nov 2018 19:45:40 -  1.17
+++ usr.sbin/rad/frontend.c 16 Nov 2018 19:46:29 -
@@ -70,6 +70,7 @@
 #include 
 #include 
 
+#include 
 #include 
 #include 
 #include 
@@ -101,6 +102,7 @@ struct ra_iface {
TAILQ_ENTRY(ra_iface)   entry;
struct ra_prefix_conf_head  prefixes;
charname[IF_NAMESIZE];
+   charconf[IF_NAMESIZE];
uint32_tif_index;
int removed;
int prefix_count;
@@ -116,6 +118,7 @@ void frontend_startup(void);
 voidicmp6_receive(int, short, void *);
 voidjoin_all_routers_mcast_group(struct ra_iface *);
 voidleave_all_routers_mcast_group(struct ra_iface *);
+voidmerge_ra_interface(struct ra_iface_conf *, char *);
 voidmerge_ra_interfaces(void);
 struct ra_iface*find_ra_iface_by_id(uint32_t);
 struct ra_iface*find_ra_iface_by_name(char *);
@@ -688,36 +691,81 @@ find_ra_iface_conf(struct ra_iface_conf_
 }
 
 void
+merge_ra_interface(struct ra_iface_conf *ra_iface_conf, char *name)
+{
+   struct ra_iface *ra_iface;
+   uint32_t if_index;
+
+   ra_iface = find_ra_iface_by_name(name);
+   if (ra_iface == NULL) {
+   log_debug("new interface %s", name);
+   if ((if_index = if_nametoindex(name)) == 0)
+   return;
+   log_debug("adding interface %s", name);
+   if ((ra_iface = calloc(1, sizeof(*ra_iface))) == NULL)
+   fatal("%s", __func__);
+
+   strlcpy(ra_iface->name, name, sizeof(ra_iface->name));
+   strlcpy(ra_iface->conf, ra_iface_conf->name,
+   sizeof(ra_iface->conf));
+
+   ra_iface->if_index = if_index;
+   SIMPLEQ_INIT(&ra_iface->prefixes);
+   TAILQ_INSERT_TAIL(&ra_interfaces, ra_iface, entry);
+   join_all_routers_mcast_group(ra_iface);
+   } else {
+   log_debug("keeping interface %s", name);
+   ra_iface->removed = 0;
+   }
+}
+
+void
 merge_ra_interfaces(void)
 {
struct ra_iface_conf*ra_iface_conf;
struct ra_prefix_conf   *ra_prefix_conf;
struct ra_iface *ra_iface;
-   uint32_t if_index;
+   struct ifgroupreqifgr;
+   struct ifg_req  *ifg;
+   char*name;
+   unsigned int len;
 
TAILQ_FOREACH(ra_iface, &ra_interfaces, entry)
ra_iface->removed = 1;
 
SIMPLEQ_FOREACH(ra_iface_conf, &frontend_conf->ra_iface_list, entry) {
-   ra_iface = find_ra_iface_by_name(ra_iface_conf->name);
-   if (ra_iface == NULL) {
-   log_debug("new interface %s", ra_iface_conf->name);
-   if ((if_index = if_nametoindex(ra_iface_conf->name))
-   == 0)
-   continue;
-   log_debug("adding interface 

Re: rad: don't try to leave the multicast group on detached interfaces

[Reyk Floeter]

On Fri, Nov 16, 2018 at 07:33:47PM +0100, Florian Obser wrote:
> On Fri, Nov 16, 2018 at 04:30:28PM +0100, Reyk Floeter wrote:
> > Hi,
> > 
> > the following patch prevents rad(8) from aborting when an interface is
> > detached (or a clone destroyed).  It is no fatal condition for rad as
> > it otherwise handles interface events just fine by reinitializing
> > itself on interface route messages.  One additional style bit included.
> > 
> > OK?
> 
> I think we should just remove the error handling, like this:
> 
> -   if (setsockopt(icmp6sock, IPPROTO_IPV6, IPV6_LEAVE_GROUP,
> -   &all_routers, sizeof(all_routers)) == -1)
> -   fatal("IPV6_LEAVE_GROUP(%s)", ra_iface->name);
> + setsockopt(icmp6sock, IPPROTO_IPV6, IPV6_LEAVE_GROUP, &all_routers,
> +   sizeof(all_routers));
> 
> It's somewhere on my todo list to go through rad(8) and slaacd(8) and
> check each fatal if we realy want to crash. I think here we should
> just plow along.
> 
> Either way, OK florian@
> 

Thanks, I did it like you suggested and committed it.

Reyk

> > 
> > Reyk
> > 
> > Index: frontend.c
> > ===
> > RCS file: /cvs/src/usr.sbin/rad/frontend.c,v
> > retrieving revision 1.16
> > diff -u -p -u -p -r1.16 frontend.c
> > --- frontend.c  15 Aug 2018 16:48:20 -  1.16
> > +++ frontend.c  16 Nov 2018 15:22:35 -
> > @@ -648,12 +648,13 @@ leave_all_routers_mcast_group(struct ra_
> > log_debug("leaving multicast group on %s", ra_iface->name);
> > all_routers.ipv6mr_interface = ra_iface->if_index;
> > if (setsockopt(icmp6sock, IPPROTO_IPV6, IPV6_LEAVE_GROUP,
> > -   &all_routers, sizeof(all_routers)) == -1)
> > +   &all_routers, sizeof(all_routers)) == -1 && errno != ENXIO)
> > fatal("IPV6_LEAVE_GROUP(%s)", ra_iface->name);
> >  }
> >  
> >  struct ra_iface*
> > -find_ra_iface_by_id(uint32_t if_index) {
> > +find_ra_iface_by_id(uint32_t if_index)
> > +{
> > struct ra_iface *ra_iface;
> >  
> > TAILQ_FOREACH(ra_iface, &ra_interfaces, entry) {
> 
> -- 
> I'm not entirely sure you are real.

vmd: add support for local inet6 interfaces

[Reyk Floeter]

Hi,

"local interface" (-L) is an amazing feature and I use it every day;
but it is IPv4-only and now I realized that I need IPv6 too.

The attached diff implements IPv6 support for local interfaces.

A few notes and limitations:

- Unlike the embedded IPv4 DHCP server, it does not implement a
DHCPv6/rtsol responder in vmd.  It relies on a rad(8) change that I've
sent earlier today.  Configuring rad is easy enough and IPv6 users are
used to jumping though extra hoops: use my rad diff and run the daemon
with "interface tap" in /etc/rad.conf.

- It is disabled by default.  You can enable it with a global option
"local inet6" (to get a runtime random fd00::/8 ULA prefix) or "local
inet6 prefix xxx::/64" (to configure your own prefix).  For
simplicity, the prefix is a global and not a per-VM option.

- Once enabled, IPv6 will be enabled and an additional IPv6 address
configured on the host's VM tap(4) interface whenever you create it
with "local interface" / -L.

- The IPv6 address is derived from the configured prefix and the IPv4
address of the local interface on the VM side.  This way it embeds the
VM and interface Id and you can even pf af-to it to IPv4 again!

```
vm_priv_ifconfig: interface tap0 address 100.64.9.2/31
vm_priv_ifconfig: interface tap0 address fdfc:6be5:806:930a:6440:903:0:1/96

100.64.9.3
```

- The resulting address is suitable for rad(8) - just run "ifconfig
vio0 inet6 autoconf" in the guest and you'll get your /96 IPv6
address.

```
vio0: 
flags=208b43 
mtu 1500
lladdr fe:e1:bb:d1:88:4f
index 1 priority 0 llprio 3
groups: egress
media: Ethernet autoselect
status: active
inet 100.64.9.3 netmask 0xfffe
inet6 fe80::7a1f:6128:505d:4ea5%vio0 prefixlen 64 scopeid 0x1
inet6 fdfc:6be5:806:930a:6440:903:b11c:516 prefixlen 96 autoconf 
autoconfprivacy pltime 86063 vltime 604794
inet6 fdfc:6be5:806:930a:6440:903:d457:347a prefixlen 96 autoconf 
pltime 604794 vltime 2591994
```

- The only problem is that the IPv6 address is nondeterministic where
you cannot guess the VM's IPv6 address "from the outside" (32 bits of
entropy for the guest IP).  It tried it with a /127 prefix but
slaacd/rad don't handle this very well as it has a 50% chance of
creating a duplicate with the host's IP.  I didn't attempt to "fix" it
as it would probably be incompatible with other rtsol clients.  So I
eventually decided that this is not important as I would still use the
IPv4 address to log in - the IPv6 address is primarily used for
outbound connections.

OK?

Reyk

Index: usr.sbin/vmd/config.c
===
RCS file: /cvs/src/usr.sbin/vmd/config.c,v
retrieving revision 1.54
diff -u -p -u -p -r1.54 config.c
--- usr.sbin/vmd/config.c   26 Oct 2018 11:24:45 -  1.54
+++ usr.sbin/vmd/config.c   16 Nov 2018 15:53:10 -
@@ -42,17 +42,40 @@
 /* Supported bridge types */
 const char *vmd_descsw[] = { "switch", "bridge", NULL };
 
+int config_init_localprefix(struct vmd_config *);
+
+int
+config_init_localprefix(struct vmd_config *cfg)
+{
+   struct sockaddr_in6 *sin6;
+
+   if (host(VMD_DHCP_PREFIX, &cfg->cfg_localprefix) == -1)
+   return (-1);
+
+   if (host(VMD_ULA_PREFIX, &cfg->cfg_localprefix6) == -1)
+   return (-1);
+   /* Randomize the 56 bits "Global ID" and "Subnet ID" */
+   sin6 = ss2sin6(&cfg->cfg_localprefix6.ss);
+   arc4random_buf(&sin6->sin6_addr.s6_addr[1], 7);
+
+   /* IPv6 is disabled by default */
+   cfg->cfg_flags &= ~VMD_CFG_INET6;
+
+   return (0);
+}
+
 int
 config_init(struct vmd *env)
 {
-   struct privsep  *ps = &env->vmd_ps;
-   unsigned int what;
+   struct privsep  *ps = &env->vmd_ps;
+   unsigned int what;
 
/* Global configuration */
ps->ps_what[PROC_PARENT] = CONFIG_ALL;
ps->ps_what[PROC_VMM] = CONFIG_VMS;
 
-   if (host(VMD_DHCP_PREFIX, &env->vmd_cfg.cfg_localprefix) == -1)
+   /* Local prefix */
+   if (config_init_localprefix(&env->vmd_cfg) == -1)
return (-1);
 
/* Other configuration */
@@ -90,7 +113,7 @@ config_purge(struct vmd *env, unsigned i
__func__, ps->ps_title[privsep_process]);
 
/* Reset global configuration (prefix was verified before) */
-   (void)host(VMD_DHCP_PREFIX, &env->vmd_cfg.cfg_localprefix);
+   config_init_localprefix(&env->vmd_cfg);
 
/* Reset other configuration */
what = ps->ps_what[privsep_process] & reset;
Index: usr.sbin/vmd/dhcp.c
===
RCS file: /cvs/src/usr.sbin/vmd/dhcp.c,v
retrieving revision 1.5
diff -u -p -u -p -r1.5 dhcp.c
--- usr.sbin/vmd/dhcp.c 17 Aug 2018 07:12:28 -  1.5
+++ usr.sbin/vmd/dhcp.c 16 Nov 2

rad: add support for listening on interface groups

[Reyk Floeter]

Hi,

the following diff allows rad(8) to watch interface groups.  This
allows to automatically add/remove interfaces in a given group.

For example, I put "interface tap" into rad.conf and it automatically
serves my VM interfaces.  You could also configure a custom group in
vm.conf and rad.conf.  I'm working on IPv6 for vmd that needs it.

For reasons that I don't remember, I always missed this feature in
rtadvd(8)[RIP].  It was amazingly simple to add it to rad(8) as it
already reinitializes itself on interface changes.

This diff includes the previous ENXIO fix to prevent it from crashing
when a cloner interface such as tap is destroyed.

Reyk

Index: usr.sbin/rad/frontend.c
===
RCS file: /cvs/src/usr.sbin/rad/frontend.c,v
retrieving revision 1.16
diff -u -p -u -p -r1.16 frontend.c
--- usr.sbin/rad/frontend.c 15 Aug 2018 16:48:20 -  1.16
+++ usr.sbin/rad/frontend.c 16 Nov 2018 15:31:24 -
@@ -70,6 +70,7 @@
 #include 
 #include 
 
+#include 
 #include 
 #include 
 #include 
@@ -101,6 +102,7 @@ struct ra_iface {
TAILQ_ENTRY(ra_iface)   entry;
struct ra_prefix_conf_head  prefixes;
charname[IF_NAMESIZE];
+   charconf[IF_NAMESIZE];
uint32_tif_index;
int removed;
int prefix_count;
@@ -116,6 +118,7 @@ void frontend_startup(void);
 voidicmp6_receive(int, short, void *);
 voidjoin_all_routers_mcast_group(struct ra_iface *);
 voidleave_all_routers_mcast_group(struct ra_iface *);
+voidmerge_ra_interface(struct ra_iface_conf *, char *);
 voidmerge_ra_interfaces(void);
 struct ra_iface*find_ra_iface_by_id(uint32_t);
 struct ra_iface*find_ra_iface_by_name(char *);
@@ -648,12 +651,13 @@ leave_all_routers_mcast_group(struct ra_
log_debug("leaving multicast group on %s", ra_iface->name);
all_routers.ipv6mr_interface = ra_iface->if_index;
if (setsockopt(icmp6sock, IPPROTO_IPV6, IPV6_LEAVE_GROUP,
-   &all_routers, sizeof(all_routers)) == -1)
+   &all_routers, sizeof(all_routers)) == -1 && errno != ENXIO)
fatal("IPV6_LEAVE_GROUP(%s)", ra_iface->name);
 }
 
 struct ra_iface*
-find_ra_iface_by_id(uint32_t if_index) {
+find_ra_iface_by_id(uint32_t if_index)
+{
struct ra_iface *ra_iface;
 
TAILQ_FOREACH(ra_iface, &ra_interfaces, entry) {
@@ -688,36 +692,81 @@ find_ra_iface_conf(struct ra_iface_conf_
 }
 
 void
+merge_ra_interface(struct ra_iface_conf *ra_iface_conf, char *name)
+{
+   struct ra_iface *ra_iface;
+   uint32_t if_index;
+
+   ra_iface = find_ra_iface_by_name(name);
+   if (ra_iface == NULL) {
+   log_debug("new interface %s", name);
+   if ((if_index = if_nametoindex(name)) == 0)
+   return;
+   log_debug("adding interface %s", name);
+   if ((ra_iface = calloc(1, sizeof(*ra_iface))) == NULL)
+   fatal("%s", __func__);
+
+   strlcpy(ra_iface->name, name, sizeof(ra_iface->name));
+   strlcpy(ra_iface->conf, ra_iface_conf->name,
+   sizeof(ra_iface->conf));
+
+   ra_iface->if_index = if_index;
+   SIMPLEQ_INIT(&ra_iface->prefixes);
+   TAILQ_INSERT_TAIL(&ra_interfaces, ra_iface, entry);
+   join_all_routers_mcast_group(ra_iface);
+   } else {
+   log_debug("keeping interface %s", name);
+   ra_iface->removed = 0;
+   }
+}
+
+void
 merge_ra_interfaces(void)
 {
struct ra_iface_conf*ra_iface_conf;
struct ra_prefix_conf   *ra_prefix_conf;
struct ra_iface *ra_iface;
-   uint32_t if_index;
+   struct ifgroupreqifgr;
+   struct ifg_req  *ifg;
+   char*name;
+   unsigned int len;
 
TAILQ_FOREACH(ra_iface, &ra_interfaces, entry)
ra_iface->removed = 1;
 
SIMPLEQ_FOREACH(ra_iface_conf, &frontend_conf->ra_iface_list, entry) {
-   ra_iface = find_ra_iface_by_name(ra_iface_conf->name);
-   if (ra_iface == NULL) {
-   log_debug("new interface %s", ra_iface_conf->name);
-   if ((if_index = if_nametoindex(ra_iface_conf->name))
-   == 0)
-   continue;
-   log_debug("adding interface %s", ra_iface_conf->name);
-   if ((ra_iface = calloc(1, sizeof(*ra_iface))) == NULL)
-   fatal("%s", __func__);
-
-   (void) strlcpy(ra_iface->name, ra_iface_c

rad: don't try to leave the multicast group on detached interfaces

[Reyk Floeter]

Hi,

the following patch prevents rad(8) from aborting when an interface is
detached (or a clone destroyed).  It is no fatal condition for rad as
it otherwise handles interface events just fine by reinitializing
itself on interface route messages.  One additional style bit included.

OK?

Reyk

Index: frontend.c
===
RCS file: /cvs/src/usr.sbin/rad/frontend.c,v
retrieving revision 1.16
diff -u -p -u -p -r1.16 frontend.c
--- frontend.c  15 Aug 2018 16:48:20 -  1.16
+++ frontend.c  16 Nov 2018 15:22:35 -
@@ -648,12 +648,13 @@ leave_all_routers_mcast_group(struct ra_
log_debug("leaving multicast group on %s", ra_iface->name);
all_routers.ipv6mr_interface = ra_iface->if_index;
if (setsockopt(icmp6sock, IPPROTO_IPV6, IPV6_LEAVE_GROUP,
-   &all_routers, sizeof(all_routers)) == -1)
+   &all_routers, sizeof(all_routers)) == -1 && errno != ENXIO)
fatal("IPV6_LEAVE_GROUP(%s)", ra_iface->name);
 }
 
 struct ra_iface*
-find_ra_iface_by_id(uint32_t if_index) {
+find_ra_iface_by_id(uint32_t if_index)
+{
struct ra_iface *ra_iface;
 
TAILQ_FOREACH(ra_iface, &ra_interfaces, entry) {

Re: Qcow2: Clean up logging/error handling

[Reyk Floeter]

On Sat, Nov 03, 2018 at 01:53:08PM -0700, Ori Bernstein wrote:
> On Tue, 30 Oct 2018 23:01:50 -0700, Mike Larkin  wrote:
> 
> > On Tue, Oct 30, 2018 at 10:41:21PM -0700, o...@eigenstate.org wrote:
> > > > On Tue, Oct 30, 2018 at 10:32:37PM -0700, Ori Bernstein wrote:
> > > >> On Tue, 30 Oct 2018 22:29:01 -0700, Mike Larkin  
> > > >> wrote:
> > > >> 
> > > >> > > -  if (disk->base->clustersz != disk->clustersz) {
> > > >> > > -  log_warn("%s: all disks must share clustersize",
> > > >> > > +  fatal("%s: could not open %s", basepath, 
> > > >> > > __func__);
> > > >> > 
> > > >> > is this right?
> > > >> > 
> > > >> > > +  if (qc2_open(disk->base, fds + 1, nfd - 1) == -1)
> > > >> > > +  fatalx("%s: could not open %s", basepath, 
> > > >> > > __func__);
> > > >> > 
> > > >> > same
> > > >> > 
> > > >> > > +  if (disk->base->clustersz != disk->clustersz)
> > > >> > > +  fatalx("%s: all disk parts must share 
> > > >> > > clustersize",
> > > >> > >__func__);
> > > >> > > -  goto error;
> > > >> > > -  }
> > > >> > > -  }
> > > >> 
> > > >> Good question. I think from earlier discussion, we wanted to fail if 
> > > >> we could
> > > >> not open a disk image -- especially since these ones are actually 
> > > >> *parts* of
> > > >> a disk image, meaning that we've got a corrupt image.
> > > >> 
> > > >> I can easily change these back to warns + error returns.
> > > >> 
> > > >> -- 
> > > >> Ori Bernstein
> > > > 
> > > > I was referring to the argument order.
> > > 
> > > Oh, that's obviously wrong. Updated.
> > > 
> > 
> > with that fix, go for it. we can hack on it further in-tree.
> > 
> 
> Anyone want to give me a second ok?
> 

2nd OK reyk@

Re: Reuse VM ids.

[Reyk Floeter]

On Sat, Oct 27, 2018 at 02:53:16PM -0700, Ori Bernstein wrote:
> On Fri, 26 Oct 2018 01:57:15 +0200, Reyk Floeter  wrote:
> 
> > On Tue, Oct 23, 2018 at 10:21:08PM -0700, Ori Bernstein wrote:
> > > On Mon, 8 Oct 2018 07:59:15 -0700, Bob Beck  wrote:
> > > 
> > > > works here and I like it.  but probably for after unlock
> > > > 
> > > 
> > > It's after unlock -- pinging for OKs.
> > > 
> > 
> > Not yet.  Please include the VM's uid in the claim, e.g.
> > 
> > claim_vmid(const char *name, uid_t uid)
> > 
> > It is not a strong protection, but it doesn't make sense that other
> > users can run a VM with the same name and get the claimed Id.
> > 
> > Reyk
> > 
> 
> Updated.
> 
> Ok?
> 

Sorry for the delay...

Two minor things:

- I think config_purge() should clear env->vmd_known as it does with
vmd_vms and vmd_switches.

- The use of static functions (static uint32_t vm_claimid) is disputable
but somewhat uncommon in OpenBSD.  But even static functions do need a
prototype ("All functions are prototyped somewhere." - style(9)).

Otherwise OK reyk@

> diff --git usr.sbin/vmd/config.c usr.sbin/vmd/config.c
> index a749e3595b5..31f46bf3c5b 100644
> --- usr.sbin/vmd/config.c
> +++ usr.sbin/vmd/config.c
> @@ -60,7 +60,10 @@ config_init(struct vmd *env)
>   if (what & CONFIG_VMS) {
>   if ((env->vmd_vms = calloc(1, sizeof(*env->vmd_vms))) == NULL)
>   return (-1);
> + if ((env->vmd_known = calloc(1, sizeof(*env->vmd_known))) == 
> NULL)
> + return (-1);
>   TAILQ_INIT(env->vmd_vms);
> + TAILQ_INIT(env->vmd_known);
>   }
>   if (what & CONFIG_SWITCHES) {
>   if ((env->vmd_switches = calloc(1,
> diff --git usr.sbin/vmd/vmd.c usr.sbin/vmd/vmd.c
> index 8053b02620f..0683812f9f0 100644
> --- usr.sbin/vmd/vmd.c
> +++ usr.sbin/vmd/vmd.c
> @@ -1169,6 +1169,28 @@ vm_remove(struct vmd_vm *vm, const char *caller)
>   free(vm);
>  }
>  
> +static uint32_t
> +vm_claimid(const char *name, int uid)
> +{
> + struct name2id *n2i = NULL;
> +
> + TAILQ_FOREACH(n2i, env->vmd_known, entry)
> + if (strcmp(n2i->name, name) == 0 && n2i->uid == uid)
> + return n2i->id;
> +
> + if (++env->vmd_nvm == 0)
> + fatalx("too many vms");
> + if ((n2i = calloc(1, sizeof(struct name2id))) == NULL)
> + fatalx("could not alloc vm name");
> + n2i->id = env->vmd_nvm;
> + n2i->uid = uid;
> + if (strlcpy(n2i->name, name, sizeof(n2i->name)) >= sizeof(n2i->name))
> + fatalx("overlong vm name");
> + TAILQ_INSERT_TAIL(env->vmd_known, n2i, entry);
> +
> + return n2i->id;
> +}
> +
>  int
>  vm_register(struct privsep *ps, struct vmop_create_params *vmc,
>  struct vmd_vm **ret_vm, uint32_t id, uid_t uid)
> @@ -1300,11 +1322,8 @@ vm_register(struct privsep *ps, struct 
> vmop_create_params *vmc,
>   vm->vm_cdrom = -1;
>   vm->vm_iev.ibuf.fd = -1;
>  
> - if (++env->vmd_nvm == 0)
> - fatalx("too many vms");
> -
>   /* Assign a new internal Id if not specified */
> - vm->vm_vmid = id == 0 ? env->vmd_nvm : id;
> + vm->vm_vmid = (id == 0) ? vm_claimid(vcp->vcp_name, uid) : id;
>  
>   log_debug("%s: registering vm %d", __func__, vm->vm_vmid);
>   TAILQ_INSERT_TAIL(env->vmd_vms, vm, vm_entry);
> diff --git usr.sbin/vmd/vmd.h usr.sbin/vmd/vmd.h
> index 7ae4e4bd65e..047b5bb3e00 100644
> --- usr.sbin/vmd/vmd.h
> +++ usr.sbin/vmd/vmd.h
> @@ -284,6 +284,14 @@ struct vmd_user {
>  };
>  TAILQ_HEAD(userlist, vmd_user);
>  
> +struct name2id {
> + charname[VMM_MAX_NAME_LEN];
> + int uid;
> + int32_t id;
> + TAILQ_ENTRY(name2id)entry;
> +};
> +TAILQ_HEAD(name2idlist, name2id);
> +
>  struct address {
>   struct sockaddr_storage  ss;
>   int  prefixlen;
> @@ -308,6 +316,7 @@ struct vmd {
>  
>   uint32_t vmd_nvm;
>   struct vmlist   *vmd_vms;
> + struct name2idlist  *vmd_known;
>   uint32_t vmd_nswitches;
>   struct switchlist   *vmd_switches;
>   struct userlist *vmd_users;
> 
> -- 
> Ori Bernstein

Re: Qcow2: Clean up logging/error handling

[Reyk Floeter]

Most of these are fatal and log_debug. Keep the __func__ there! But we’ll 
remove it from other logging functions where it was never intended to be used 
and potentially reword the warnings nicely.

Reyk

> Am 28.10.2018 um 00:39 schrieb Ori Bernstein :
> 
>> On Sat, 27 Oct 2018 16:15:32 -0600, "Theo de Raadt"  
>> wrote:
>> 
>> I quite dislike when software uses __func__ to tell me what internal
>> function had an error.
>> 
>> Why should a user see an error with the string virtio_qcow2_get_base,
>> qc2_open, qc2_pwrite, etc?
>> 
>> It feels unpolished.
>> 
> 
> Possibly, but it is consistent with the rest of vmd (16 instances are
> from vioqcow2.c)
> 
>$ grep log.*__func__ *.c | wc -l
>162
> 
> But many of them are on lines that wrap. Overcounting a bit:
> 
>$ grep __func__ *.c | wc -l
>474
> 
> If we want to change it, I think it's better as an independent patch that
> converts all of vmd.
> 
> -- 
>Ori Bernstein
> 

Re: Reuse VM ids.

[Reyk Floeter]

On Tue, Oct 23, 2018 at 10:21:08PM -0700, Ori Bernstein wrote:
> On Mon, 8 Oct 2018 07:59:15 -0700, Bob Beck  wrote:
> 
> > works here and I like it.  but probably for after unlock
> > 
> 
> It's after unlock -- pinging for OKs.
> 

Not yet.  Please include the VM's uid in the claim, e.g.

claim_vmid(const char *name, uid_t uid)

It is not a strong protection, but it doesn't make sense that other
users can run a VM with the same name and get the claimed Id.

Reyk

> diff --git usr.sbin/vmd/config.c usr.sbin/vmd/config.c
> index af12b790002..522bae32501 100644
> --- usr.sbin/vmd/config.c
> +++ usr.sbin/vmd/config.c
> @@ -61,7 +61,10 @@ config_init(struct vmd *env)
>   if (what & CONFIG_VMS) {
>   if ((env->vmd_vms = calloc(1, sizeof(*env->vmd_vms))) == NULL)
>   return (-1);
> + if ((env->vmd_known = calloc(1, sizeof(*env->vmd_known))) == 
> NULL)
> + return (-1);
>   TAILQ_INIT(env->vmd_vms);
> + TAILQ_INIT(env->vmd_known);
>   }
>   if (what & CONFIG_SWITCHES) {
>   if ((env->vmd_switches = calloc(1,
> diff --git usr.sbin/vmd/vmd.c usr.sbin/vmd/vmd.c
> index 18a5e0d3d5d..732691b4381 100644
> --- usr.sbin/vmd/vmd.c
> +++ usr.sbin/vmd/vmd.c
> @@ -1169,6 +1169,27 @@ vm_remove(struct vmd_vm *vm, const char *caller)
>   free(vm);
>  }
>  
> +static uint32_t
> +claim_vmid(const char *name)
> +{
> + struct name2id *n2i = NULL;
> +
> + TAILQ_FOREACH(n2i, env->vmd_known, entry)
> + if (strcmp(n2i->name, name) == 0)
> + return n2i->id;
> +
> + if (++env->vmd_nvm == 0)
> + fatalx("too many vms");
> + if ((n2i = calloc(1, sizeof(struct name2id))) == NULL)
> + fatalx("could not alloc vm name");
> + n2i->id = env->vmd_nvm;
> + if (strlcpy(n2i->name, name, sizeof(n2i->name)) >= sizeof(n2i->name))
> + fatalx("overlong vm name");
> + TAILQ_INSERT_TAIL(env->vmd_known, n2i, entry);
> +
> + return n2i->id;
> +}
> +
>  int
>  vm_register(struct privsep *ps, struct vmop_create_params *vmc,
>  struct vmd_vm **ret_vm, uint32_t id, uid_t uid)
> @@ -1300,11 +1321,8 @@ vm_register(struct privsep *ps, struct 
> vmop_create_params *vmc,
>   vm->vm_cdrom = -1;
>   vm->vm_iev.ibuf.fd = -1;
>  
> - if (++env->vmd_nvm == 0)
> - fatalx("too many vms");
> -
>   /* Assign a new internal Id if not specified */
> - vm->vm_vmid = id == 0 ? env->vmd_nvm : id;
> + vm->vm_vmid = (id == 0) ? claim_vmid(vcp->vcp_name) : id;
>  
>   log_debug("%s: registering vm %d", __func__, vm->vm_vmid);
>   TAILQ_INSERT_TAIL(env->vmd_vms, vm, vm_entry);
> diff --git usr.sbin/vmd/vmd.h usr.sbin/vmd/vmd.h
> index b7c012854e8..86fad536e59 100644
> --- usr.sbin/vmd/vmd.h
> +++ usr.sbin/vmd/vmd.h
> @@ -276,6 +276,13 @@ struct vmd_user {
>  };
>  TAILQ_HEAD(userlist, vmd_user);
>  
> +struct name2id {
> + charname[VMM_MAX_NAME_LEN];
> + int32_t id;
> + TAILQ_ENTRY(name2id)entry;
> +};
> +TAILQ_HEAD(name2idlist, name2id);
> +
>  struct address {
>   struct sockaddr_storage  ss;
>   int  prefixlen;
> @@ -300,6 +307,7 @@ struct vmd {
>  
>   uint32_t vmd_nvm;
>   struct vmlist   *vmd_vms;
> + struct name2idlist  *vmd_known;
>   uint32_t vmd_nswitches;
>   struct switchlist   *vmd_switches;
>   struct userlist *vmd_users;
> 

Re: Qcow2 Disk Size Fix.

[Reyk Floeter]

On Mon, Oct 22, 2018 at 11:54:34PM -0700, Ori Bernstein wrote:
> On Mon, 22 Oct 2018 22:57:39 -0700, Ori Bernstein  wrote:
> 
> > While I'm at it, this patch fixes up a few nits around logging, where
> > warn was used instead of warnx, leading to some bogus error strings
> > being printed, and adds a few checks that we should have been doing.
> > This patch also removes a lie in a comment.
> > 
> > OK?
> 
> And, since there was a request to split the patch, here's the fix,
> isolated from the rest of the changes:
> 

Looks good.

OK reyk

> diff --git usr.sbin/vmd/vioqcow2.c usr.sbin/vmd/vioqcow2.c
> index 3a215599d49..d1a00b7acfb 100644
> --- usr.sbin/vmd/vioqcow2.c
> +++ usr.sbin/vmd/vioqcow2.c
> @@ -79,15 +79,15 @@ struct qcdisk {
>   int   fd;
>   uint64_t *l1;
>   off_t end;
> - uint32_t  clustersz;
> + off_t clustersz;
>   off_t disksz; /* In bytes */
> - uint32_t cryptmethod;
> + uint32_t  cryptmethod;
>  
>   uint32_t l1sz;
>   off_tl1off;
>  
>   off_trefoff;
> - uint32_t refsz;
> + off_trefsz;
>  
>   uint32_t nsnap;
>   off_tsnapoff;
> @@ -207,7 +207,7 @@ qc2_open(struct qcdisk *disk, int *fds, size_t nfd)
>   struct qcheader header;
>   uint64_t backingoff;
>   uint32_t backingsz;
> - size_t i;
> + off_t i;
>   int version, fd;
>  
>   pthread_rwlock_init(&disk->lock, NULL);
> 
> -- 
> Ori Bernstein

-- 

vmctl "convert": create new disk images from an input file

[Reyk Floeter]

Hi tech@,

there is one more diff in our qcow2 queue which lets you convert disk
images into anoter format.  You create a new image from an input file:

- The "standard" use case:

vmctl create foo.qcow2 -i foo.raw

- The other way around:

vmctl create foo.raw -i foo.qcow2

- Using the explicit format:

vmctl create foo.raw -i qcow2:foo.qc2

- You can also increase the disk size of an existing image :

vmctl create foo.qcow2 -s 10G
...
vmctl create bar.qcow2 -i foo.qcow2 -s 20G

- Or flatten a qcow2 image that has a base image, converting it back
into a single qcow2 or raw image:

vmctl create foo.qcow2 -b foo-base.qcow2
...
vmctl create bar.qcow2 -i foo.qcow2

- Or convert a flat raw image back to a sparse file (that doesn't use
disk space for trailing zeroes).

vmctl create bar.raw -i foo.raw

This was possible by re-using Ori's qcow2 code from vmd in vmctl.

Reyk

Index: usr.sbin/vmd/config.c
===
RCS file: /cvs/src/usr.sbin/vmd/config.c,v
retrieving revision 1.51
diff -u -p -u -p -r1.51 config.c
--- usr.sbin/vmd/config.c   8 Oct 2018 16:32:01 -   1.51
+++ usr.sbin/vmd/config.c   8 Oct 2018 17:39:36 -
@@ -35,7 +35,6 @@
 #include 
 #include 
 #include 
-#include 
 
 #include "proc.h"
 #include "vmd.h"
@@ -191,7 +190,6 @@ config_setvm(struct privsep *ps, struct 
char ifname[IF_NAMESIZE], *s;
char path[PATH_MAX];
char base[PATH_MAX];
-   char expanded[PATH_MAX];
unsigned int unit;
 
errno = 0;
@@ -323,7 +321,7 @@ config_setvm(struct privsep *ps, struct 
oflags = O_RDONLY|O_NONBLOCK;
aflags = R_OK;
n = virtio_get_base(diskfds[i][j], base, sizeof base,
-   vmc->vmc_disktypes[i]);
+   vmc->vmc_disktypes[i], path);
if (n == 0)
break;
if (n == -1) {
@@ -331,30 +329,6 @@ config_setvm(struct privsep *ps, struct 
"base %s for disk %s", vcp->vcp_name,
base, vcp->vcp_disks[i]);
goto fail;
-   }
-   /*
-* Relative paths should be interpreted relative
-* to the disk image, rather than relative to the
-* directory vmd happens to be running in, since
-* this is the only userful interpretation.
-*/
-   if (base[0] == '/') {
-   if (realpath(base, path) == NULL) {
-   log_warn("unable to resolve %s", base);
-   goto fail;
-   }
-   } else {
-   s = dirname(path);
-   if (snprintf(expanded, sizeof(expanded),
-   "%s/%s", s, base) >= (int)sizeof(expanded)) 
{
-   log_warn("path too long: %s/%s",
-   s, base);
-   goto fail;
-   }
-   if (realpath(expanded, path) == NULL) {
-   log_warn("unable to resolve %s", base);
-   goto fail;
-   }
}
}
}
Index: usr.sbin/vmd/vioqcow2.c
===
RCS file: /cvs/src/usr.sbin/vmd/vioqcow2.c,v
retrieving revision 1.8
diff -u -p -u -p -r1.8 vioqcow2.c
--- usr.sbin/vmd/vioqcow2.c 8 Oct 2018 16:32:01 -   1.8
+++ usr.sbin/vmd/vioqcow2.c 8 Oct 2018 17:39:37 -
@@ -27,6 +27,7 @@
 #include 
 #include 
 #include 
+#include 
 #include 
 
 #include "vmd.h"
@@ -137,11 +138,13 @@ virtio_init_qcow2(struct virtio_backing 
 }
 
 ssize_t
-virtio_qcow2_get_base(int fd, char *path, size_t npath)
+virtio_qcow2_get_base(int fd, char *path, size_t npath, const char *dpath)
 {
+   char expanded[PATH_MAX];
struct qcheader header;
uint64_t backingoff;
uint32_t backingsz;
+   char *s = NULL;
 
if (pread(fd, &header, sizeof(header), 0) != sizeof(header)) {
log_warnx("%s: short read on header", __func__);
@@ -153,19 +156,47 @@ virtio_qcow2_get_base(int fd, char *path
}
backingoff = be64toh(header.backingoff);
backingsz = be32toh(header.backingsz);
-   if (backingsz != 0) {
-   if (backingsz >= npath - 1) {
-   log_war

Re: Reuse VM ids.

[Reyk Floeter]

On Sun, Oct 07, 2018 at 07:31:45PM -0700, Ori Bernstein wrote:
> Keep a list of known vms, and reuse the VM IDs. This means that when using
> '-L', the IP addresses of the VMs are stable.
> 

After you conviced me about its use case, 3 comments:
1. it has to wait until after release
2. please following the naming vm_claimid instead of claim_vmid
3. you should bind it to name + vm_uid, not just name

I was actually wondering if we should use randomized non-linear vm ids
(e.g. with knuth shuffle) in the future, but that's a different topic
for a different time.  This diff wouldn't conflict with it.

Reyk

> diff --git usr.sbin/vmd/config.c usr.sbin/vmd/config.c
> index af12b790002..522bae32501 100644
> --- usr.sbin/vmd/config.c
> +++ usr.sbin/vmd/config.c
> @@ -61,7 +61,10 @@ config_init(struct vmd *env)
>   if (what & CONFIG_VMS) {
>   if ((env->vmd_vms = calloc(1, sizeof(*env->vmd_vms))) == NULL)
>   return (-1);
> + if ((env->vmd_known = calloc(1, sizeof(*env->vmd_known))) == 
> NULL)
> + return (-1);
>   TAILQ_INIT(env->vmd_vms);
> + TAILQ_INIT(env->vmd_known);
>   }
>   if (what & CONFIG_SWITCHES) {
>   if ((env->vmd_switches = calloc(1,
> diff --git usr.sbin/vmd/vmd.c usr.sbin/vmd/vmd.c
> index 18a5e0d3d5d..732691b4381 100644
> --- usr.sbin/vmd/vmd.c
> +++ usr.sbin/vmd/vmd.c
> @@ -1169,6 +1169,27 @@ vm_remove(struct vmd_vm *vm, const char *caller)
>   free(vm);
>  }
>  
> +static uint32_t
> +claim_vmid(const char *name)
> +{
> + struct name2id *n2i = NULL;
> +
> + TAILQ_FOREACH(n2i, env->vmd_known, entry)
> + if (strcmp(n2i->name, name) == 0)
> + return n2i->id;
> +
> + if (++env->vmd_nvm == 0)
> + fatalx("too many vms");
> + if ((n2i = calloc(1, sizeof(struct name2id))) == NULL)
> + fatalx("could not alloc vm name");
> + n2i->id = env->vmd_nvm;
> + if (strlcpy(n2i->name, name, sizeof(n2i->name)) >= sizeof(n2i->name))
> + fatalx("overlong vm name");
> + TAILQ_INSERT_TAIL(env->vmd_known, n2i, entry);
> +
> + return n2i->id;
> +}
> +
>  int
>  vm_register(struct privsep *ps, struct vmop_create_params *vmc,
>  struct vmd_vm **ret_vm, uint32_t id, uid_t uid)
> @@ -1300,11 +1321,8 @@ vm_register(struct privsep *ps, struct 
> vmop_create_params *vmc,
>   vm->vm_cdrom = -1;
>   vm->vm_iev.ibuf.fd = -1;
>  
> - if (++env->vmd_nvm == 0)
> - fatalx("too many vms");
> -
>   /* Assign a new internal Id if not specified */
> - vm->vm_vmid = id == 0 ? env->vmd_nvm : id;
> + vm->vm_vmid = (id == 0) ? claim_vmid(vcp->vcp_name) : id;
>  
>   log_debug("%s: registering vm %d", __func__, vm->vm_vmid);
>   TAILQ_INSERT_TAIL(env->vmd_vms, vm, vm_entry);
> diff --git usr.sbin/vmd/vmd.h usr.sbin/vmd/vmd.h
> index b7c012854e8..86fad536e59 100644
> --- usr.sbin/vmd/vmd.h
> +++ usr.sbin/vmd/vmd.h
> @@ -276,6 +276,13 @@ struct vmd_user {
>  };
>  TAILQ_HEAD(userlist, vmd_user);
>  
> +struct name2id {
> + charname[VMM_MAX_NAME_LEN];
> + int32_t id;
> + TAILQ_ENTRY(name2id)entry;
> +};
> +TAILQ_HEAD(name2idlist, name2id);
> +
>  struct address {
>   struct sockaddr_storage  ss;
>   int  prefixlen;
> @@ -300,6 +307,7 @@ struct vmd {
>  
>   uint32_t vmd_nvm;
>   struct vmlist   *vmd_vms;
> + struct name2idlist  *vmd_known;
>   uint32_t vmd_nswitches;
>   struct switchlist   *vmd_switches;
>   struct userlist *vmd_users;
> 
> -- 
> Ori Bernstein

-- 

Re: vmd: rate-limit to avoid reboot loops

[Reyk Floeter]

> Am 06.10.2018 um 10:55 schrieb Consus :
> 
>> On 23:32 Fri 05 Oct, Reyk Floeter wrote:
>> Hi,
>> 
>> it sometimes happens that a VM is stuck in a reboot loop.  This isn't
>> very pleasent for vmd, so this diff attempts to introduce a hard
>> rate-limit: if the VM rebooted after less than VM_START_RATE_SEC (6)
>> seconds, increment a counter.  If this happens VM_START_RATE_LIMIT (3)
>> times in a row, stop the VM.
>> 
>> The idea is that it might be desirable in some cases to reboot quickly
>> (you're either really fast on the boot prompt, or you use something
>> like grub that can automatically reboot into a previous kernel).  But
>> if this happens too often (more than 3 times), something is wrong and
>> cannot be intended, not even in the worst Linux/grub/unikernel/...
>> situation.
>> 
>> These limits are a guessed default.
>> 
>> Test case: I dd'ed random bytes to a kernel after some initial bytes,
>> keeping the original size of the kernel.  The boot loader loads the
>> header, the complete kernel, tries to boot it and *boom*, reset ;)
>> 
>> Comments?  Concerns?  Better ideas?  OKs?
>> 
>> Reyk
> 
> At least there should be an easy way to enable/disable this at will.
> This can be troublesome when someone is trying to fix early
> not-so-obvious Linux kernel / initrd bugs. Or when testing new kernel
> code that sometimes results in a panic.
> 

Could you explain why?

When you debug a kernel bug, why would you want to reboot loop it within a few 
seconds?

And people who use it for fuzzing are able to change the defines and to 
recompile the code ;-)

Reyk

vmd: rate-limit to avoid reboot loops

[Reyk Floeter]

Hi,

it sometimes happens that a VM is stuck in a reboot loop.  This isn't
very pleasent for vmd, so this diff attempts to introduce a hard
rate-limit: if the VM rebooted after less than VM_START_RATE_SEC (6)
seconds, increment a counter.  If this happens VM_START_RATE_LIMIT (3)
times in a row, stop the VM.

The idea is that it might be desirable in some cases to reboot quickly
(you're either really fast on the boot prompt, or you use something
like grub that can automatically reboot into a previous kernel).  But
if this happens too often (more than 3 times), something is wrong and
cannot be intended, not even in the worst Linux/grub/unikernel/...
situation.

These limits are a guessed default.

Test case: I dd'ed random bytes to a kernel after some initial bytes,
keeping the original size of the kernel.  The boot loader loads the
header, the complete kernel, tries to boot it and *boom*, reset ;)

Comments?  Concerns?  Better ideas?  OKs?

Reyk

Index: usr.sbin/vmd/config.c
===
RCS file: /cvs/src/usr.sbin/vmd/config.c,v
retrieving revision 1.50
diff -u -p -u -p -r1.50 config.c
--- usr.sbin/vmd/config.c   7 Aug 2018 14:49:05 -   1.50
+++ usr.sbin/vmd/config.c   5 Oct 2018 21:15:12 -
@@ -187,6 +187,7 @@ config_setvm(struct privsep *ps, struct 
char ifname[IF_NAMESIZE], *s;
char path[PATH_MAX];
unsigned int unit;
+   struct timeval   tv, rate, since_last;
 
errno = 0;
 
@@ -204,6 +205,39 @@ config_setvm(struct privsep *ps, struct 
goto fail;
}
}
+
+   /*
+* Rate-limit the VM so that it cannot restart in a loop:
+* if the VM restarts after less than VM_START_RATE_SEC seconds,
+* we increment the limit counter.  After VM_START_RATE_LIMIT
+* of suchs fast reboots the VM is stopped.
+*/
+   getmonotime(&tv);
+   if (vm->vm_start_tv.tv_sec) {
+   timersub(&tv, &vm->vm_start_tv, &since_last);
+
+   rate.tv_sec = VM_START_RATE_SEC;
+   rate.tv_usec = 0;
+   if (timercmp(&since_last, &rate, <))
+   vm->vm_start_limit++;
+   else {
+   /* Reset counter */
+   vm->vm_start_limit = 0;
+   }
+
+   log_debug("%s: vm %u restarted after %lld.%ld seconds,"
+   " limit %d/%d", __func__, vcp->vcp_id, since_last.tv_sec,
+   since_last.tv_usec, vm->vm_start_limit,
+   VM_START_RATE_LIMIT);
+
+   if (vm->vm_start_limit >= VM_START_RATE_LIMIT) {
+   log_warnx("%s: vm %u restarted too quickly",
+   __func__, vcp->vcp_id);
+   errno = EPERM;
+   goto fail;
+   }
+   }
+   vm->vm_start_tv = tv;
 
diskfds = reallocarray(NULL, vcp->vcp_ndisks, sizeof(*diskfds));
if (diskfds == NULL) {
Index: usr.sbin/vmd/vmd.c
===
RCS file: /cvs/src/usr.sbin/vmd/vmd.c,v
retrieving revision 1.102
diff -u -p -u -p -r1.102 vmd.c
--- usr.sbin/vmd/vmd.c  29 Sep 2018 22:33:09 -  1.102
+++ usr.sbin/vmd/vmd.c  5 Oct 2018 21:15:12 -
@@ -1918,3 +1918,14 @@ prefixlen2mask(uint8_t prefixlen)
 
return (htonl(0x << (32 - prefixlen)));
 }
+
+void
+getmonotime(struct timeval *tv)
+{
+   struct timespec  ts;
+
+   if (clock_gettime(CLOCK_MONOTONIC, &ts))
+   fatal("clock_gettime");
+
+   TIMESPEC_TO_TIMEVAL(tv, &ts);
+}
Index: usr.sbin/vmd/vmd.h
===
RCS file: /cvs/src/usr.sbin/vmd/vmd.h,v
retrieving revision 1.81
diff -u -p -u -p -r1.81 vmd.h
--- usr.sbin/vmd/vmd.h  1 Oct 2018 09:31:15 -   1.81
+++ usr.sbin/vmd/vmd.h  5 Oct 2018 21:15:13 -
@@ -54,6 +54,10 @@
 #define VMD_SWITCH_TYPE"bridge"
 #define VM_DEFAULT_MEMORY  512
 
+/* Rate-limit fast reboots */
+#define VM_START_RATE_SEC  6   /* min. seconds since last reboot */
+#define VM_START_RATE_LIMIT3   /* max. number of fast reboots */
+
 /* default user instance limits */
 #define VM_DEFAULT_USER_MAXCPU 4
 #define VM_DEFAULT_USER_MAXMEM 2048
@@ -260,6 +264,10 @@ struct vmd_vm {
int  vm_receive_fd;
struct vmd_user *vm_user;
 
+   /* For rate-limiting */
+   struct timeval   vm_
- Message truncated -

Re: Qcow2: External snapshots

[Reyk Floeter]

On Wed, Oct 03, 2018 at 11:41:41PM -0700, Ori Bernstein wrote:
> diff --git usr.sbin/vmd/config.c usr.sbin/vmd/config.c
> index 550b73c1a39..68be738d304 100644
> --- usr.sbin/vmd/config.c
> +++ usr.sbin/vmd/config.c
> @@ -35,6 +35,7 @@
>  #include 
>  #include 
>  #include 
> +#include 
>  
>  #include "proc.h"
>  #include "vmd.h"
> @@ -176,16 +177,21 @@ config_getreset(struct vmd *env, struct imsg *imsg)
>  int
>  config_setvm(struct privsep *ps, struct vmd_vm *vm, uint32_t peerid, uid_t 
> uid)
>  {
> + int diskfds[VMM_MAX_DISKS_PER_VM][VM_MAX_BASE_PER_DISK];
>   struct vmd_if   *vif;
>   struct vmop_create_params *vmc = &vm->vm_params;
>   struct vm_create_params *vcp = &vmc->vmc_params;
> - unsigned int i;
> + unsigned int i, j;
>   int  fd = -1, vmboot = 0;
> - int  kernfd = -1, *diskfds = NULL, *tapfds = NULL;
> + int  kernfd = -1;
> + int *tapfds;

keep tapfds = NULL or you might cause a segfault in the goto fail case...

[snip]
>   if (tapfds != NULL) {
>   for (i = 0; i < vcp->vcp_nnics; i++)
>   close(tapfds[i]);

...here (same function).

Reyk

Re: Qcow2: External snapshots

[Reyk Floeter]

On Wed, Oct 03, 2018 at 11:41:41PM -0700, Ori Bernstein wrote:
> Thanks, another update based on Reyk's feeback and fixes.
> 

You missed one thing: jmc@'s manpage comments.

For everything else:  Looks good!  Tests work fine.  OK reyk@

Reyk

> diff --git regress/usr.sbin/vmd/diskfmt/Makefile 
> regress/usr.sbin/vmd/diskfmt/Makefile
> index c2a5f42d5f6..1f8673e0e26 100644
> --- regress/usr.sbin/vmd/diskfmt/Makefile
> +++ regress/usr.sbin/vmd/diskfmt/Makefile
> @@ -11,7 +11,7 @@
>  VMD_DIR=$(BSDSRCDIR)/usr.sbin/vmd/
>  
>  PROG=vioscribble
> -SRCS=vioscribble.c $(VMD_DIR)/vioqcow2.c $(VMD_DIR)/vioraw.c
> +SRCS=vioscribble.c vioqcow2.c vioraw.c
>  CFLAGS+=-I$(VMD_DIR) -pthread
>  LDFLAGS+=-pthread
>  
> @@ -26,3 +26,6 @@ scribble-images:
>  .PHONY: ${REGRESS_TARGETS} scribble-images
>  
>  .include 
> +
> +vioqcow2.c vioraw.c: $(VMD_DIR)/vioqcow2.c $(VMD_DIR)/vioraw.c
> + cp $(VMD_DIR)/vioqcow2.c $(VMD_DIR)/vioraw.c .
> diff --git regress/usr.sbin/vmd/diskfmt/vioscribble.c 
> regress/usr.sbin/vmd/diskfmt/vioscribble.c
> index 14d720db652..1da8efedac7 100644
> --- regress/usr.sbin/vmd/diskfmt/vioscribble.c
> +++ regress/usr.sbin/vmd/diskfmt/vioscribble.c
> @@ -122,16 +122,18 @@ main(int argc, char **argv)
>   verbose = !!getenv("VERBOSE");
>   qcfd = open("scribble.qc2", O_RDWR);
>   rawfd = open("scribble.raw", O_RDWR);
> - if (qcfd == -1 || virtio_init_qcow2(&qcowfile, &qcsz, qcfd) == -1)
> + if (qcfd == -1)
>   err(1, "unable to open qcow");
> - if (rawfd == -1 || virtio_init_raw(&rawfile, &rawsz, rawfd) == -1)
> + if (virtio_init_qcow2(&qcowfile, &qcsz, &qcfd, 1) == -1)
> + err(1, "unable to init qcow");
> + if (rawfd == -1 || virtio_init_raw(&rawfile, &rawsz, &rawfd, 1) == -1)
>   err(1, "unable to open raw");
>  
>   srandom_deterministic(123);
>  
>   /* scribble to both disks */
>   printf("scribbling...\n");
> - for (i = 0; i < 16; i++) {
> + for (i = 0; i < 1024*16; i++) {
>   off = (random() % DISKSZ);
>   len = random() % sizeof buf + 1;
>   fill(off, buf, sizeof buf);
> diff --git usr.sbin/vmctl/main.c usr.sbin/vmctl/main.c
> index 8748ecfdedc..a3ab4672370 100644
> --- usr.sbin/vmctl/main.c
> +++ usr.sbin/vmctl/main.c
> @@ -67,7 +67,8 @@ int  ctl_receive(struct parse_result *, int, char 
> *[]);
>  
>  struct ctl_command ctl_commands[] = {
>   { "console",CMD_CONSOLE,ctl_console,"id" },
> - { "create", CMD_CREATE, ctl_create, "\"path\" -s size", 1 },
> + { "create", CMD_CREATE, ctl_create,
> + "\"path\" [-s size] [-b base]", 1 },
>   { "load",   CMD_LOAD,   ctl_load,   "\"path\"" },
>   { "log",CMD_LOG,ctl_log,"[verbose|brief]" },
>   { "reload", CMD_RELOAD, ctl_reload, "" },
> @@ -538,47 +539,55 @@ int
>  ctl_create(struct parse_result *res, int argc, char *argv[])
>  {
>   int  ch, ret, type;
> - const char  *paths[2], *disk, *format;
> + const char  *disk, *format, *base;
>  
>   if (argc < 2)
>   ctl_usage(res->ctl);
>  
> + base = NULL;
>   type = parse_disktype(argv[1], &disk);
>  
> - paths[0] = disk;
> - paths[1] = NULL;
> -
> - if (unveil(paths[0], "rwc") == -1)
> + if (pledge("stdio rpath wpath cpath unveil", NULL) == -1)
> + err(1, "pledge");
> + if (unveil(disk, "rwc") == -1)
>   err(1, "unveil");
>  
> - if (pledge("stdio rpath wpath cpath", NULL) == -1)
> - err(1, "pledge");
>   argc--;
>   argv++;
>  
> - while ((ch = getopt(argc, argv, "s:")) != -1) {
> + while ((ch = getopt(argc, argv, "s:b:")) != -1) {
>   switch (ch) {
>   case 's':
>   if (parse_size(res, optarg, 0) != 0)
>   errx(1, "invalid size: %s", optarg);
>   break;
> + case 'b':
> + base = optarg;
> + if (unveil(base, "r") == -1)
> + err(1, "unveil");
> + break;
>   default:
>   ctl_usage(res->ctl);
>   /* NOTREACHED */
>   }
>   }
> + if (unveil(NULL, NULL))
> + err(1, "unveil");
>  
> - if (res->size == 0) {
> - fprintf(stderr, "missing size argument\n");
> + if (base && type != VMDF_QCOW2)
> + errx(1, "base images require qcow2 disk format");
> + if (res->size == 0 && !base) {
> + fprintf(stderr, "could not create %s: missing size argument\n",
> + disk);
>   ctl_usage(res->ctl);
>   }
>  
>   if (type == VMDF_QCOW2) {
>   format = "qcow2";
> - ret = create_qc2_imagefile(paths[0], res->size);
> + ret = create_qc2_imagefile(disk, base, res->size);
>   

Re: Qcow2: External snapshots

[Reyk Floeter]

On Tue, Oct 02, 2018 at 11:13:35PM -0700, Ori Bernstein wrote:
> On Mon, 1 Oct 2018 11:24:01 -0700, Ori Bernstein  wrote:
> 
> > On Mon, 1 Oct 2018 12:55:12 +0200
> > Reyk Floeter  wrote:
> > 
> > > Hi Ori,
> > > 
> > > On Sun, Sep 30, 2018 at 12:27:00PM -0700, Ori Bernstein wrote:
> > > > I've added support to vmd for external snapshots. That is,
> > > > snapshots that are derived from a base image. Data lookups
> > > > start in the derived image, and if the derived image does not
> > > > contain some data, the search proceeds ot the base image.
> > > > Multiple derived images may exist off of a single base image.
> > > > 
> > > 
> > > Nice work!  This will be quite useful, thanks.
> > > 
> > > I think I broke your diff as my last commit to derive the raw/qcow2
> > > format introduced some conflicts.  I had posted it on hackers@ and
> > > forgot that your aren't on the internal list yet - sorry for that.
> 
> Updated version. Changes from the last diff:
> 
> - Merge in syntax changes. 
> - Don't over-read when getting the base images.
> - Fix relative paths in base images.
> - Allow multiple derived images to use a single base image, and allow a user
>   with only read permisssions to base their images on top of it.
> - Probe the base image size, use/validate it when craeting disk images.
> - Fix style a bit (long lines, changing from sizeof foo to sizeof(foo).
> - Move a define out of vmmvar.h

You're still using VMM_MAX_BASE_PER_DISK instead of
VM_MAX_BASE_PER_DISK in the code.  I patched the diff to make it
compile without the vmmvar.h change here.

> - And update the manpage with these changes.
> - Improve error checking around creating/resolving base disk paths.
> 

Light testing works except of an issue with read-only base images; the
required fix is in the comments below.

Other than that, it is really cool to run many VMs from a single base
image.  In my tests, I installed OpenBSD once and started a few VMs
using the installed disk as a base.

More comments below.

Reyk

> 
> diff --git regress/usr.sbin/vmd/diskfmt/Makefile 
> regress/usr.sbin/vmd/diskfmt/Makefile
> index c2a5f42d5f6..1f8673e0e26 100644
> --- regress/usr.sbin/vmd/diskfmt/Makefile
> +++ regress/usr.sbin/vmd/diskfmt/Makefile
> @@ -11,7 +11,7 @@
>  VMD_DIR=$(BSDSRCDIR)/usr.sbin/vmd/
>  
>  PROG=vioscribble
> -SRCS=vioscribble.c $(VMD_DIR)/vioqcow2.c $(VMD_DIR)/vioraw.c
> +SRCS=vioscribble.c vioqcow2.c vioraw.c
>  CFLAGS+=-I$(VMD_DIR) -pthread
>  LDFLAGS+=-pthread
>  
> @@ -26,3 +26,6 @@ scribble-images:
>  .PHONY: ${REGRESS_TARGETS} scribble-images
>  
>  .include 
> +
> +vioqcow2.c vioraw.c: $(VMD_DIR)/vioqcow2.c $(VMD_DIR)/vioraw.c
> + cp $(VMD_DIR)/vioqcow2.c $(VMD_DIR)/vioraw.c .
> diff --git regress/usr.sbin/vmd/diskfmt/vioscribble.c 
> regress/usr.sbin/vmd/diskfmt/vioscribble.c
> index 14d720db652..1da8efedac7 100644
> --- regress/usr.sbin/vmd/diskfmt/vioscribble.c
> +++ regress/usr.sbin/vmd/diskfmt/vioscribble.c
> @@ -122,16 +122,18 @@ main(int argc, char **argv)
>   verbose = !!getenv("VERBOSE");
>   qcfd = open("scribble.qc2", O_RDWR);
>   rawfd = open("scribble.raw", O_RDWR);
> - if (qcfd == -1 || virtio_init_qcow2(&qcowfile, &qcsz, qcfd) == -1)
> + if (qcfd == -1)
>   err(1, "unable to open qcow");
> - if (rawfd == -1 || virtio_init_raw(&rawfile, &rawsz, rawfd) == -1)
> + if (virtio_init_qcow2(&qcowfile, &qcsz, &qcfd, 1) == -1)
> + err(1, "unable to init qcow");
> + if (rawfd == -1 || virtio_init_raw(&rawfile, &rawsz, &rawfd, 1) == -1)
>   err(1, "unable to open raw");
>  
>   srandom_deterministic(123);
>  
>   /* scribble to both disks */
>   printf("scribbling...\n");
> - for (i = 0; i < 16; i++) {
> + for (i = 0; i < 1024*16; i++) {
>   off = (random() % DISKSZ);
>   len = random() % sizeof buf + 1;
>   fill(off, buf, sizeof buf);
> diff --git usr.sbin/vmctl/main.c usr.sbin/vmctl/main.c
> index 8748ecfdedc..4637256452b 100644
> --- usr.sbin/vmctl/main.c
> +++ usr.sbin/vmctl/main.c
> @@ -67,7 +67,8 @@ int  ctl_receive(struct parse_result *, int, char 
> *[]);
>  
>  struct ctl_command ctl_commands[] = {
>   { "console",CMD_CONSOLE,ctl_console,"id" },
> - { "create", CMD_CREATE, ctl_create, "\"path\" -s size", 1 },
> + { "create", CMD_CREATE, ctl_c

Re: vmd losing VMs

[Reyk Floeter]

Ok thanks.  I wonder if there is a tool that can log everything from
the VM's console ... It would be useful to see what happens to the VM
before it goes into zombie mode.

Reyk

On Tue, Oct 02, 2018 at 10:33:11AM -0700, Greg Steuck wrote:
> I believe I was unable to ssh into them or get cu to elicit any characters.
> I'll verify next time it happens.
> 
> On Tue, Oct 2, 2018 at 10:20 AM Reyk Floeter  wrote:
> 
> > On Tue, Oct 02, 2018 at 10:10:41AM -0700, Greg Steuck wrote:
> > > Naturally, bugs don't solve themselves :) Here's a log, it's not very
> > > useful due to the lack of debugging symbols. Notice, that runaway vmds
> > > don't die on their own, they just spin out of control. I'll do VMM_DEBUG
> > > next.
> > >
> >
> > "they just spin out of control" - maybe I've missed the previous
> > details, but do you know what happens to these VMs?  Are they stuck in
> > ddb or in a reboot loop?
> >
> > Reyk
> >
> > > ci-openbsd# sysctl kern.nosuidcoredump=3 && mkdir /var/crash/vmd
> > > kern.nosuidcoredump: 1 -> 3
> > > ci-openbsd#  sysctl kern.nosuidcoredump
> > > kern.nosuidcoredump=3
> > > ci-openbsd# ps ax | grep vmd
> > > 32653 ??  Isp 0:01.28 /usr/sbin/vmd
> > > 89585 ??  Is  0:00.14 vmd: priv (vmd)
> > > 50191 ??  Isp 0:01.88 vmd: control (vmd)
> > > 33160 ??  Isp 0:08.55 vmd: vmm (vmd)
> > > 52853 ??  Rp/1  280:12.56 vmd: ci-openbsd-main-1 (vmd)
> > >  3238 ??  Rp/2   48:56.13 vmd: ci-openbsd-main-0 (vmd)
> > > 44625 ??  Rp/02:54.05 vmd: ci-openbsd-main-2 (vmd)
> > > 42187 p5  R+p/0   0:00.00 grep vmd (ksh)
> > > ci-openbsd# vmctl status
> > >ID   PID VCPUS  MAXMEM  CURMEM TTYOWNER NAME
> > >   239 44625 1512M181M   ttyp2syzkaller ci-openbsd-main-2
> > > 1 - 1512M   -   -syzkaller syzkaller
> > > ci-openbsd# kill 52853
> > > ci-openbsd# ps ax | grep vmd
> > > 32653 ??  Ssp 0:01.28 /usr/sbin/vmd
> > > 89585 ??  Is  0:00.14 vmd: priv (vmd)
> > > 50191 ??  Ssp 0:01.88 vmd: control (vmd)
> > > 33160 ??  Ssp 0:08.55 vmd: vmm (vmd)
> > > 52853 ??  Rp/1  280:30.24 vmd: ci-openbsd-main-1 (vmd)
> > >  3238 ??  Rp/2   49:14.27 vmd: ci-openbsd-main-0 (vmd)
> > > 44625 ??  Rp/03:12.25 vmd: ci-openbsd-main-2 (vmd)
> > > 27783 p5  R+p/1   0:00.00 grep vmd (ksh)
> > > ci-openbsd# ps ax | grep syz
> > > 50771 ??  Is  0:00.01 /sbin/mount_mfs -s 10G /dev/sd0b
> > > /syzkaller/ramdisk
> > > 70067 ??  Is  0:00.16 sshd: syzkaller [priv] (sshd)
> > >   957 ??  S   0:01.09 sshd: syzkaller@ttyp3 (sshd)
> > > 28743 ??  I  18:00.14 syzkaller/current/bin/syz-manager -config
> > > /syzkaller/managers/main/current/manager.cfg
> > >  5869 ??  Ip  0:00.07 ssh -p 22 -i
> > /syzkaller/managers/main/current/key
> > > -F /dev/null -o UserKnownHostsFile=/dev/null -o BatchMode=yes -o Identi
> > > 57895 ??  Is  0:00.08 sshd: syzkaller [priv] (sshd)
> > > 77889 ??  S   0:00.17 sshd: syzkaller@ttyp4 (sshd)
> > > 39218 p5  R+/00:00.00 grep syz
> > > 50644 00- D   4:09.87 ./syz-ci -config ./config-openbsd.ci
> > > 60603 00- Ip  0:00.05 tee syz-ci.log
> > > ci-openbsd# kill 50644 28743
> > > ci-openbsd# ps ax | grep syz
> > > 50771 ??  Is  0:00.01 /sbin/mount_mfs -s 10G /dev/sd0b
> > > /syzkaller/ramdisk
> > > 70067 ??  Is  0:00.16 sshd: syzkaller [priv] (sshd)
> > >   957 ??  I   0:01.09 sshd: syzkaller@ttyp3 (sshd)
> > > 57895 ??  Is  0:00.08 sshd: syzkaller [priv] (sshd)
> > > 77889 ??  S   0:00.18 sshd: syzkaller@ttyp4 (sshd)
> > > 15816 p5  R+/10:00.00 grep syz
> > > ci-openbsd# rcctl stop vmd
> > > vmd(ok)
> > > ci-openbsd# ps ax | grep vmd
> > > 52853 ??  Rp/1  281:59.20 vmd: ci-openbsd-main-1 (vmd)
> > >  3238 ??  Rp/2   50:42.87 vmd: ci-openbsd-main-0 (vmd)
> > > 19166 p5  R+/00:00.00 grep vmd
> > > ci-openbsd# kill -ABRT 52853
> > > ci-openbsd# ps ax | grep vmd
> > >  3238 ??  Rp/2   52:06.55 vmd: ci-openbsd-main-0 (vmd)
> > > 55423 p5  S+p 0:00.01 grep vmd
> > > ci-openbsd# dmesg | tail
> > > pms0 at pckbc0 (aux slot)
> > > wsmouse0 at pms0 mux 0
> > > pcppi0 at isa0 port 0x61
> > > spkr0 at pcppi0
> > > vmm0 at mainbus0: VMX/EPT (using slow L1TF mitigation)
> > > vscsi0 at root
> > > sc

Re: vmd losing VMs

[Reyk Floeter]

On Tue, Oct 02, 2018 at 10:10:41AM -0700, Greg Steuck wrote:
> Naturally, bugs don't solve themselves :) Here's a log, it's not very
> useful due to the lack of debugging symbols. Notice, that runaway vmds
> don't die on their own, they just spin out of control. I'll do VMM_DEBUG
> next.
> 

"they just spin out of control" - maybe I've missed the previous
details, but do you know what happens to these VMs?  Are they stuck in
ddb or in a reboot loop?

Reyk

> ci-openbsd# sysctl kern.nosuidcoredump=3 && mkdir /var/crash/vmd
> kern.nosuidcoredump: 1 -> 3
> ci-openbsd#  sysctl kern.nosuidcoredump
> kern.nosuidcoredump=3
> ci-openbsd# ps ax | grep vmd
> 32653 ??  Isp 0:01.28 /usr/sbin/vmd
> 89585 ??  Is  0:00.14 vmd: priv (vmd)
> 50191 ??  Isp 0:01.88 vmd: control (vmd)
> 33160 ??  Isp 0:08.55 vmd: vmm (vmd)
> 52853 ??  Rp/1  280:12.56 vmd: ci-openbsd-main-1 (vmd)
>  3238 ??  Rp/2   48:56.13 vmd: ci-openbsd-main-0 (vmd)
> 44625 ??  Rp/02:54.05 vmd: ci-openbsd-main-2 (vmd)
> 42187 p5  R+p/0   0:00.00 grep vmd (ksh)
> ci-openbsd# vmctl status
>ID   PID VCPUS  MAXMEM  CURMEM TTYOWNER NAME
>   239 44625 1512M181M   ttyp2syzkaller ci-openbsd-main-2
> 1 - 1512M   -   -syzkaller syzkaller
> ci-openbsd# kill 52853
> ci-openbsd# ps ax | grep vmd
> 32653 ??  Ssp 0:01.28 /usr/sbin/vmd
> 89585 ??  Is  0:00.14 vmd: priv (vmd)
> 50191 ??  Ssp 0:01.88 vmd: control (vmd)
> 33160 ??  Ssp 0:08.55 vmd: vmm (vmd)
> 52853 ??  Rp/1  280:30.24 vmd: ci-openbsd-main-1 (vmd)
>  3238 ??  Rp/2   49:14.27 vmd: ci-openbsd-main-0 (vmd)
> 44625 ??  Rp/03:12.25 vmd: ci-openbsd-main-2 (vmd)
> 27783 p5  R+p/1   0:00.00 grep vmd (ksh)
> ci-openbsd# ps ax | grep syz
> 50771 ??  Is  0:00.01 /sbin/mount_mfs -s 10G /dev/sd0b
> /syzkaller/ramdisk
> 70067 ??  Is  0:00.16 sshd: syzkaller [priv] (sshd)
>   957 ??  S   0:01.09 sshd: syzkaller@ttyp3 (sshd)
> 28743 ??  I  18:00.14 syzkaller/current/bin/syz-manager -config
> /syzkaller/managers/main/current/manager.cfg
>  5869 ??  Ip  0:00.07 ssh -p 22 -i /syzkaller/managers/main/current/key
> -F /dev/null -o UserKnownHostsFile=/dev/null -o BatchMode=yes -o Identi
> 57895 ??  Is  0:00.08 sshd: syzkaller [priv] (sshd)
> 77889 ??  S   0:00.17 sshd: syzkaller@ttyp4 (sshd)
> 39218 p5  R+/00:00.00 grep syz
> 50644 00- D   4:09.87 ./syz-ci -config ./config-openbsd.ci
> 60603 00- Ip  0:00.05 tee syz-ci.log
> ci-openbsd# kill 50644 28743
> ci-openbsd# ps ax | grep syz
> 50771 ??  Is  0:00.01 /sbin/mount_mfs -s 10G /dev/sd0b
> /syzkaller/ramdisk
> 70067 ??  Is  0:00.16 sshd: syzkaller [priv] (sshd)
>   957 ??  I   0:01.09 sshd: syzkaller@ttyp3 (sshd)
> 57895 ??  Is  0:00.08 sshd: syzkaller [priv] (sshd)
> 77889 ??  S   0:00.18 sshd: syzkaller@ttyp4 (sshd)
> 15816 p5  R+/10:00.00 grep syz
> ci-openbsd# rcctl stop vmd
> vmd(ok)
> ci-openbsd# ps ax | grep vmd
> 52853 ??  Rp/1  281:59.20 vmd: ci-openbsd-main-1 (vmd)
>  3238 ??  Rp/2   50:42.87 vmd: ci-openbsd-main-0 (vmd)
> 19166 p5  R+/00:00.00 grep vmd
> ci-openbsd# kill -ABRT 52853
> ci-openbsd# ps ax | grep vmd
>  3238 ??  Rp/2   52:06.55 vmd: ci-openbsd-main-0 (vmd)
> 55423 p5  S+p 0:00.01 grep vmd
> ci-openbsd# dmesg | tail
> pms0 at pckbc0 (aux slot)
> wsmouse0 at pms0 mux 0
> pcppi0 at isa0 port 0x61
> spkr0 at pcppi0
> vmm0 at mainbus0: VMX/EPT (using slow L1TF mitigation)
> vscsi0 at root
> scsibus2 at vscsi0: 256 targets
> softraid0 at root
> scsibus3 at softraid0: 256 targets
> root on sd0a (dd61083aafe9fd0b.a) swap on sd0b dump on sd0b
> ci-openbsd# ps ax | grep vmd
>  3238 ??  Rp/2   52:06.88 vmd: ci-openbsd-main-0 (vmd)
> 19175 p5  R+/00:00.00 grep vmd
> ci-openbsd# kill 3238
> ci-openbsd# ps ax | grep vmd
>  3238 ??  Rp/2   52:18.52 vmd: ci-openbsd-main-0 (vmd)
> 27516 p5  R+/30:00.00 grep vmd
> ci-openbsd# kill -ABRT 3238
> ci-openbsd# ps ax | grep vmd
>  3238 ??  Rp/2   52:27.71 (vmd)
> 93083 p5  R+/30:00.00 grep vmd
> ci-openbsd# ps ax | grep vmd
> 95984 p5  S+p 0:00.01 grep vmd
> ci-openbsd# ls -l /var/crash/vmd
> total 668864
> -rw---  1 root  wheel  200320568 Oct  2 09:47 3238.core
> -rw---  1 root  wheel  141988032 Oct  2 09:46 52853.core
> ci-openbsd# gdb /usr/sb
> ci-openbsd# file /var/crash/vmd/52853.core
> /var/crash/vmd/52853.core: ELF 64-bit LSB core file x86-64, version 1
> ci-openbsd# gdb /usr/sbin/vmd /var/crash/vmd/52853.core
> GNU gdb 6.3
> Copyright 2004 Free Software Foundation, Inc.
> GDB is free software, covered by the GNU General Public License, and you are
> welcome to change it and/or distribute copies of it under certain
> conditions.
> Type "show copying" to see the conditions.
> There is absolutely no warranty for GDB.  Type "show warranty" for details.
> This GDB was configured as "amd64-unknown-openbsd6.4"...(no debugging
> symbols found)
> 
> Core was generated by `vmd'.
> Program terminated with signal 6, Aborted.
> Reading symbol

Re: Qcow2: External snapshots

[Reyk Floeter]

Hi Ori,

On Sun, Sep 30, 2018 at 12:27:00PM -0700, Ori Bernstein wrote:
> I've added support to vmd for external snapshots. That is,
> snapshots that are derived from a base image. Data lookups
> start in the derived image, and if the derived image does not
> contain some data, the search proceeds ot the base image.
> Multiple derived images may exist off of a single base image.
> 

Nice work!  This will be quite useful, thanks.

I think I broke your diff as my last commit to derive the raw/qcow2
format introduced some conflicts.  I had posted it on hackers@ and
forgot that your aren't on the internal list yet - sorry for that.

> A limitation of this format is that modifying the base image
> will corrupt the derived image.
> 
> This change also adds support for creating disk derived disk
> images to vmctl.  To use it:
> 
>   vmctl create derived.img -s 16G -b base.img -f qcow2
> 

I removed -f fmt to be more consistent and the new syntax will be

vmctl create qcow2:derived.img -s 16G -b base.img

or

vmctl create derived.qcow2 -s 16G -b base.img

but we should be able to derive it from the base as well (there's now
base in raw images), so the following should work as well: 

vmctl create derived.img -s 16G -b base.img

> The main implementation change is that we now probe base
> images before sending the disk FDs to the VM, which means that
> we can actually open the images.
> 
> The base image paths may be relative. If they are relative,
> they are interpreted relative to the location of the derived
> image, and not relative to the directory where vmd happens to
> be running.
> 

OK, that needs some care + review.

> For review, a bit of scrutiny could be directed to the
> messaging.  It relies on imsg being in-order, which seems to
> be the case, but isn't documented in the manpage -- If I can't
> rely on that, the protocol needs to be tweaked.
> 

imsgs are guaranteed to be in order as long as you don't mux them with
other messages from the same sender in an async way.

> After this change, we send imsgs to the same disk index
> repeatedly, and each message adds another base to the stack of
> images. So, for example, if I have 2images image that look
> like this:
> 
>   disk0 -> base0 -> base1
>   disk1
> 
> Then we send the following messages:
> 
>   VMDOP_START_VM_DISK (i=0, fd=open(disk0))
>   VMDOP_START_VM_DISK (i=0, fd=open(base0))
>   VMDOP_START_VM_DISK (i=0, fd=open(base1))
> 
>   VMDOP_START_VM_DISK (i=1, fd=open(disk1))
> 

Makes sense.

> This also opens the door to ephemeral snapshots, which vmd can
> implicitly create when it starts a vm, and removes
> automatically on exit.
> 

Please be extremely careful with the design here.  Unlike qemu, a vmd
VM is not able to create new files itself and it should never be able
to do it.  So when we create snapshots, we need to find a way that the
parent prepares the file, sends the fd, and asks the VM process to use
it.

> Testing has been the usual -- OpenBSD installs, a bit of catting,
> and some random 'dd'. Heavier use and testing would be appreciated.
> 

I will test the updated diff that includes the second fix and the merge ;)

Initial comments inline below.

Reyk

> 
> 
> diff --git regress/usr.sbin/vmd/diskfmt/Makefile 
> regress/usr.sbin/vmd/diskfmt/Makefile
> index c2a5f42d5f6..1f8673e0e26 100644
> --- regress/usr.sbin/vmd/diskfmt/Makefile
> +++ regress/usr.sbin/vmd/diskfmt/Makefile
> @@ -11,7 +11,7 @@
>  VMD_DIR=$(BSDSRCDIR)/usr.sbin/vmd/
>  
>  PROG=vioscribble
> -SRCS=vioscribble.c $(VMD_DIR)/vioqcow2.c $(VMD_DIR)/vioraw.c
> +SRCS=vioscribble.c vioqcow2.c vioraw.c
>  CFLAGS+=-I$(VMD_DIR) -pthread
>  LDFLAGS+=-pthread
>  
> @@ -26,3 +26,6 @@ scribble-images:
>  .PHONY: ${REGRESS_TARGETS} scribble-images
>  
>  .include 
> +
> +vioqcow2.c vioraw.c: $(VMD_DIR)/vioqcow2.c $(VMD_DIR)/vioraw.c
> + cp $(VMD_DIR)/vioqcow2.c $(VMD_DIR)/vioraw.c .
> diff --git regress/usr.sbin/vmd/diskfmt/vioscribble.c 
> regress/usr.sbin/vmd/diskfmt/vioscribble.c
> index 14d720db652..1da8efedac7 100644
> --- regress/usr.sbin/vmd/diskfmt/vioscribble.c
> +++ regress/usr.sbin/vmd/diskfmt/vioscribble.c
> @@ -122,16 +122,18 @@ main(int argc, char **argv)
>   verbose = !!getenv("VERBOSE");
>   qcfd = open("scribble.qc2", O_RDWR);
>   rawfd = open("scribble.raw", O_RDWR);
> - if (qcfd == -1 || virtio_init_qcow2(&qcowfile, &qcsz, qcfd) == -1)
> + if (qcfd == -1)
>   err(1, "unable to open qcow");
> - if (rawfd == -1 || virtio_init_raw(&rawfile, &rawsz, rawfd) == -1)
> + if (virtio_init_qcow2(&qcowfile, &qcsz, &qcfd, 1) == -1)
> + err(1, "unable to init qcow");
> + if (rawfd == -1 || virtio_init_raw(&rawfile, &rawsz, &rawfd, 1) == -1)
>   err(1, "unable to open raw");
>  
>   srandom_deterministic(123);
>  
>   /* scribble to both disks */
>   printf("scribbling...\n");
> - for (i = 0; i < 16; i++) {
> + for (i = 0; i < 1024*16;

Re: vmd stdio /dev/null

[Reyk Floeter]

Hi,

I already gave my OK for relayd but asked to adjust all proc.c instances.

OK reyk@

> Am 09.09.2018 um 23:47 schrieb Mike Larkin :
> 
>> On Sun, Sep 09, 2018 at 11:45:07PM +0200, Alexander Bluhm wrote:
>> Hi,
>> 
>> Like the other proc.c daemons, vmd(8) children do not detach from
>> the terminal properly.
>> 
>> ok?
>> 
> 
> Reads ok to me but reyk@ should approve.
> 
>> bluhm
>> 
>> Index: src/usr.sbin/vmd/proc.c
>> ===
>> RCS file: /data/mirror/openbsd/cvs/src/usr.sbin/vmd/proc.c,v
>> retrieving revision 1.17
>> diff -u -p -r1.17 proc.c
>> --- src/usr.sbin/vmd/proc.c5 Aug 2018 08:20:54 -1.17
>> +++ src/usr.sbin/vmd/proc.c9 Sep 2018 21:26:32 -
>> @@ -29,13 +29,14 @@
>> #include 
>> #include 
>> #include 
>> +#include 
>> #include 
>> #include 
>> #include 
>> 
>> #include "proc.h"
>> 
>> -void proc_exec(struct privsep *, struct privsep_proc *, unsigned int,
>> +void proc_exec(struct privsep *, struct privsep_proc *, unsigned int, 
>> int,
>>int, char **);
>> void proc_setup(struct privsep *, struct privsep_proc *, unsigned int);
>> void proc_open(struct privsep *, int, int);
>> @@ -80,7 +81,7 @@ proc_getid(struct privsep_proc *procs, u
>> 
>> void
>> proc_exec(struct privsep *ps, struct privsep_proc *procs, unsigned int nproc,
>> -int argc, char **argv)
>> +int debug, int argc, char **argv)
>> {
>>unsigned int proc, nargc, i, proc_i;
>>char**nargv;
>> @@ -141,6 +142,16 @@ proc_exec(struct privsep *ps, struct pri
>>} else if (fcntl(fd, F_SETFD, 0) == -1)
>>fatal("fcntl");
>> 
>> +/* Daemons detach from terminal. */
>> +if (!debug && (fd =
>> +open(_PATH_DEVNULL, O_RDWR, 0)) != -1) {
>> +(void)dup2(fd, STDIN_FILENO);
>> +(void)dup2(fd, STDOUT_FILENO);
>> +(void)dup2(fd, STDERR_FILENO);
>> +if (fd > 2)
>> +(void)close(fd);
>> +}
>> +
>>execvp(argv[0], nargv);
>>fatal("%s: execvp", __func__);
>>break;
>> @@ -191,7 +202,7 @@ proc_connect(struct privsep *ps)
>> 
>> void
>> proc_init(struct privsep *ps, struct privsep_proc *procs, unsigned int nproc,
>> -int argc, char **argv, enum privsep_procid proc_id)
>> +int debug, int argc, char **argv, enum privsep_procid proc_id)
>> {
>>struct privsep_proc*p = NULL;
>>struct privsep_pipes*pa, *pb;
>> @@ -231,7 +242,7 @@ proc_init(struct privsep *ps, struct pri
>>}
>> 
>>/* Engage! */
>> -proc_exec(ps, procs, nproc, argc, argv);
>> +proc_exec(ps, procs, nproc, debug, argc, argv);
>>return;
>>}
>> 
>> Index: src/usr.sbin/vmd/proc.h
>> ===
>> RCS file: /data/mirror/openbsd/cvs/src/usr.sbin/vmd/proc.h,v
>> retrieving revision 1.15
>> diff -u -p -r1.15 proc.h
>> --- src/usr.sbin/vmd/proc.h5 Aug 2018 08:20:54 -1.15
>> +++ src/usr.sbin/vmd/proc.h9 Sep 2018 21:15:50 -
>> @@ -156,7 +156,7 @@ struct privsep_fd {
>> #define PROC_MAX_INSTANCES32
>> 
>> /* proc.c */
>> -void proc_init(struct privsep *, struct privsep_proc *, unsigned int,
>> +void proc_init(struct privsep *, struct privsep_proc *, unsigned int, 
>> int,
>>int, char **, enum privsep_procid);
>> void proc_kill(struct privsep *);
>> void proc_connect(struct privsep *ps);
>> Index: src/usr.sbin/vmd/vmd.c
>> ===
>> RCS file: /data/mirror/openbsd/cvs/src/usr.sbin/vmd/vmd.c,v
>> retrieving revision 1.98
>> diff -u -p -r1.98 vmd.c
>> --- src/usr.sbin/vmd/vmd.c15 Jul 2018 14:36:54 -1.98
>> +++ src/usr.sbin/vmd/vmd.c9 Sep 2018 21:25:57 -
>> @@ -792,7 +792,8 @@ main(int argc, char **argv)
>>ps->ps_title[proc_id] = title;
>> 
>>/* only the parent returns */
>> -proc_init(ps, procs, nitems(procs), argc0, argv, proc_id);
>> +proc_init(ps, procs, nitems(procs), env->vmd_debug, argc0, argv,
>> +proc_id);
>> 
>>log_procinit("parent");
>>if (!env->vmd_debug && daemon(0, 0) == -1)
>> 
> 

mg: add set-case-replace

[Reyk Floeter]

Hi,

the new case-preserving replace is a very nice feature - 
unless you want to replace a string with the exact capitalisation.

>From florian@'s original commit message:
> Use (mostly) the same capitalisation in the replacement string as in
> 
> the replaced string:
> 
> replacing foo with bar turns
> 
> foo
> Foo
> FOO
> 
> into
> 
> bar
> Bar
> BAR
> 
> OK phessler, benno

Now I had the case that I wanted to replace FOO with bar, not BAR.

Emacs has an option "case-replace" to turn it off - it can be set to
nil to not preserve the case of the replaced string.

This adds set-case-replace.

Ok?

Reyk

Index: usr.bin/mg/def.h
===
RCS file: /cvs/src/usr.bin/mg/def.h,v
retrieving revision 1.155
diff -u -p -u -p -r1.155 def.h
--- usr.bin/mg/def.h14 Apr 2016 17:05:32 -  1.155
+++ usr.bin/mg/def.h28 Aug 2018 11:59:49 -
@@ -391,6 +391,7 @@ int  ldelete(RSIZE, int);
 int ldelnewline(void);
 int lreplace(RSIZE, char *);
 char *  linetostr(const struct line *);
+int setcasereplace(int, int);
 
 /* yank.c X */
 
Index: usr.bin/mg/funmap.c
===
RCS file: /cvs/src/usr.bin/mg/funmap.c,v
retrieving revision 1.53
diff -u -p -u -p -r1.53 funmap.c
--- usr.bin/mg/funmap.c 14 Apr 2016 17:05:32 -  1.53
+++ usr.bin/mg/funmap.c 28 Aug 2018 11:59:49 -
@@ -184,6 +184,7 @@ static struct funmap functnames[] = {
 #ifdef REGEX
{setcasefold, "set-case-fold-search",},
 #endif /* REGEX */
+   {setcasereplace, "set-case-replace",},
{set_default_mode, "set-default-mode",},
{setfillcol, "set-fill-column",},
{setmark, "set-mark-command",},
Index: usr.bin/mg/line.c
===
RCS file: /cvs/src/usr.bin/mg/line.c,v
retrieving revision 1.60
diff -u -p -u -p -r1.60 line.c
--- usr.bin/mg/line.c   12 Jul 2018 12:38:56 -  1.60
+++ usr.bin/mg/line.c   28 Aug 2018 11:59:49 -
@@ -27,6 +27,22 @@
 
 #include "def.h"
 
+intcasereplace = TRUE;
+
+/*
+ * Preserve the case of the replaced string.
+ */
+int
+setcasereplace(int f, int n)
+{
+   if (f & FFARG)
+   casereplace = n > 0;
+   else
+   casereplace = !casereplace;
+   ewprintf("Case-replace is %sabled", casereplace ? "en" : "dis");
+   return (TRUE);
+}
+
 /*
  * Allocate a new line of size `used'.  lrealloc() can be called if the line
  * ever needs to grow beyond that.
@@ -516,7 +532,7 @@ lreplace(RSIZE plen, char *st)
RSIZE n;
int s, doto, is_query_capitalised = 0, is_query_allcaps = 0;
int is_replace_alllower = 0;
-   char *repl;
+   char *repl = NULL;
 
if ((s = checkdirty(curbp)) != TRUE)
return (s);
@@ -531,11 +547,14 @@ lreplace(RSIZE plen, char *st)
ewprintf("out of memory");
return (FALSE);
}
+   rlen = strlen(repl);
 
undo_boundary_enable(FFRAND, 0);
-
(void)backchar(FFARG | FFRAND, (int)plen);
 
+   if (casereplace != TRUE)
+   goto done;
+
lp = curwp->w_dotp;
doto = curwp->w_doto;
n = plen;
@@ -556,9 +575,6 @@ lreplace(RSIZE plen, char *st)
}
}
 
-   (void)ldelete(plen, KNONE);
-
-   rlen = strlen(repl);
for (n = 0, is_replace_alllower = 1; n < rlen && is_replace_alllower;
n++)
is_replace_alllower = !isupper((unsigned char)repl[n]);
@@ -572,6 +588,8 @@ lreplace(RSIZE plen, char *st)
}
}
 
+ done:
+   (void)ldelete(plen, KNONE);
region_put_data(repl, rlen);
lchange(WFFULL);
 
Index: usr.bin/mg/mg.1
===
RCS file: /cvs/src/usr.bin/mg/mg.1,v
retrieving revision 1.106
diff -u -p -u -p -r1.106 mg.1
--- usr.bin/mg/mg.1 11 Dec 2017 07:27:07 -  1.106
+++ usr.bin/mg/mg.1 28 Aug 2018 11:59:49 -
@@ -846,6 +846,9 @@ Currently only affects fill-paragraph.
 Set case-fold searching, causing case not to matter
 in regular expression searches.
 This is the default.
+.It set-case-replace
+Preserve the case of the replaced string.
+This is the default.
 .It set-default-mode
 Append the supplied mode to the list of default modes
 used by subsequent buffer creation.

Re: vmd/vmctl: allow to boot cdrom-only VMs

[Reyk Floeter]

Hi,

it’s not like that, maybe I wasn’t clear. Let me try to explain:

- you can boot from disk (-d)
- you can boot an OpenBSD kernel or use a custom BIOS (-b)
- you can boot from CDROM (-r)

vmd/vmctl checks if at least one option is available.

If you specify neither of these options, you can potentially boot from network 
or via telekinesis. But vmd doesn’t support network boot (only with a PXE BIOS 
payload that conflicts with sgabios) and I cannot judge your telekinesis skills 
to make a VM boot without any kernel.

Reyk

> Am 23.08.2018 um 18:05 schrieb Todd T. Fries :
> 
> This makes me wonder. Does it make sense to support booting a kernel without
> disks?  Some people have heard of the phrase 'diskless' ;-)
> 
> Penned by Reyk Floeter on 20180822 13:35.23, we have:
> | Hi,
> | 
> | vmctl doesn't allow to boot VMs with only a CDROM.  I see no reason
> | for it and vmd already allows CDROM-only.
> | 
> | OK?
> | 
> | Via https://twitter.com/wizardishungry/status/1032327323125727232
> | "Jon Williams @wizardishungry
> | @reykfloeter Could you consider allowing booting ISO-only vms in 6.4?
> | This is helpful for running container hosts (e.g. boot2docker)."
> | 
> | Reyk
> | 
> | Index: usr.sbin/vmctl/vmctl.c
> | ===
> | RCS file: /cvs/src/usr.sbin/vmctl/vmctl.c,v
> | retrieving revision 1.54
> | diff -u -p -u -p -r1.54 vmctl.c
> | --- usr.sbin/vmctl/vmctl.c12 Jul 2018 12:04:49 -1.54
> | +++ usr.sbin/vmctl/vmctl.c22 Aug 2018 18:29:44 -
> | @@ -98,8 +98,8 @@ vm_start(uint32_t start_id, const char *
> |  errx(1, "too many disks");
> |  else if (ndisks == 0)
> |  warnx("starting without disks");
> | -if (kernel == NULL && ndisks == 0)
> | -errx(1, "no kernel or disk specified");
> | +if (kernel == NULL && ndisks == 0 && !iso)
> | +errx(1, "no kernel or disk/cdrom specified");
> |  if (nnics == -1)
> |  nnics = 0;
> |  if (nnics > VMM_MAX_NICS_PER_VM)
> 
> -- 
> Todd T. Fries . http://todd.fries.net/pgp.txt . @unix2mars . github:toddfries

vmd/vmctl: allow to boot cdrom-only VMs

[Reyk Floeter]

Hi,

vmctl doesn't allow to boot VMs with only a CDROM.  I see no reason
for it and vmd already allows CDROM-only.

OK?

Via https://twitter.com/wizardishungry/status/1032327323125727232
"Jon Williams @wizardishungry
@reykfloeter Could you consider allowing booting ISO-only vms in 6.4?
This is helpful for running container hosts (e.g. boot2docker)."

Reyk

Index: usr.sbin/vmctl/vmctl.c
===
RCS file: /cvs/src/usr.sbin/vmctl/vmctl.c,v
retrieving revision 1.54
diff -u -p -u -p -r1.54 vmctl.c
--- usr.sbin/vmctl/vmctl.c  12 Jul 2018 12:04:49 -  1.54
+++ usr.sbin/vmctl/vmctl.c  22 Aug 2018 18:29:44 -
@@ -98,8 +98,8 @@ vm_start(uint32_t start_id, const char *
errx(1, "too many disks");
else if (ndisks == 0)
warnx("starting without disks");
-   if (kernel == NULL && ndisks == 0)
-   errx(1, "no kernel or disk specified");
+   if (kernel == NULL && ndisks == 0 && !iso)
+   errx(1, "no kernel or disk/cdrom specified");
if (nnics == -1)
nnics = 0;
if (nnics > VMM_MAX_NICS_PER_VM)

Re: vmctl: add unveil

[Reyk Floeter]

On Tue, Aug 21, 2018 at 09:51:52PM -0700, Carlos Cardenas wrote:
> Patch to unveil vmctl.
> 
> Comments/OK?
> 

OK reyk

btw. paths[0] is an artifact from the old pledge paths argument.
semarie@ removed it in -r1.13 of main.c but we both overlooked that
paths[2] is not needed anymore and could just be turned into a
non-array path variable.

Reyk

> +--+
> Carlos

> Index: main.c
> ===
> RCS file: /home/los/cvs/src/usr.sbin/vmctl/main.c,v
> retrieving revision 1.39
> diff -u -p -r1.39 main.c
> --- main.c12 Jul 2018 14:53:37 -  1.39
> +++ main.c18 Aug 2018 23:22:39 -
> @@ -160,7 +160,7 @@ parse(int argc, char *argv[])
>  
>   if (!ctl->has_pledge) {
>   /* pledge(2) default if command doesn't have its own pledge */
> - if (pledge("stdio rpath exec unix getpw", NULL) == -1)
> + if (pledge("stdio rpath exec unix getpw unveil", NULL) == -1)
>   err(1, "pledge");
>   }
>   if (ctl->main(&res, argc, argv) != 0)
> @@ -185,6 +185,8 @@ vmmaction(struct parse_result *res)
>   unsigned int flags;
>  
>   if (ctl_sock == -1) {
> + if (unveil(SOCKET_NAME, "r") == -1)
> + err(1, "unveil");
>   if ((ctl_sock = socket(AF_UNIX,
>   SOCK_STREAM|SOCK_CLOEXEC, 0)) == -1)
>   err(1, "socket");
> @@ -477,6 +479,10 @@ ctl_create(struct parse_result *res, int
>  
>   paths[0] = argv[1];
>   paths[1] = NULL;
> +
> + if (unveil(paths[0], "rwc") == -1)
> + err(1, "unveil");
> +
>   if (pledge("stdio rpath wpath cpath", NULL) == -1)
>   err(1, "pledge");
>   argc--;
> @@ -759,6 +765,8 @@ __dead void
>  ctl_openconsole(const char *name)
>  {
>   closefrom(STDERR_FILENO + 1);
> + if (unveil(VMCTL_CU, "x") == -1)
> + err(1, "unveil");
>   execl(VMCTL_CU, VMCTL_CU, "-l", name, "-s", "115200", (char *)NULL);
>   err(1, "failed to open the console");
>  }


-- 

Re: vmd: sync DPADD with LDADD

[Reyk Floeter]

OK reyk@

> Am 24.06.2018 um 07:57 schrieb Gleydson Soares :
> 
> sync DPADD with LDADD adding missing ${LIBPTHREAD} to ensure
> that binary is rebuilt in case of pthread library changes.
> Index: Makefile
> ===
> RCS file: /cvs/src/usr.sbin/vmd/Makefile,v
> retrieving revision 1.17
> diff -u -p -r1.17 Makefile
> --- Makefile3 Jan 2018 05:39:56 -1.17
> +++ Makefile24 Jun 2018 05:31:47 -
> @@ -15,7 +15,7 @@ CFLAGS+=-Wshadow -Wpointer-arith -Wcast
> CFLAGS+=-Wsign-compare
> 
> LDADD+=-lutil -lpthread -levent
> -DPADD+=${LIBUTIL} ${LIBEVENT}
> +DPADD+=${LIBUTIL} ${LIBPTHREAD} ${LIBEVENT}
> 
> YFLAGS=
> 

ypldap: sync aldap from ldap(1)

[Reyk Floeter]

Hi,

the _url code was broken and disabled in ypldap's aldap - I fixed it
for ldap(1).  The other chunk is a DEBUG message fix, not compiled by
default.

OK?

Index: usr.sbin/ypldap/aldap.c
===
RCS file: /cvs/src/usr.sbin/ypldap/aldap.c,v
retrieving revision 1.39
diff -u -p -u -p -r1.39 aldap.c
--- usr.sbin/ypldap/aldap.c 8 Feb 2018 18:02:06 -   1.39
+++ usr.sbin/ypldap/aldap.c 21 Jun 2018 08:29:52 -
@@ -1,5 +1,5 @@
-/* $Id: aldap.c,v 1.39 2018/02/08 18:02:06 jca Exp $ */
-/* $OpenBSD: aldap.c,v 1.39 2018/02/08 18:02:06 jca Exp $ */
+/* $Id: aldap.c,v 1.2 2018/06/21 08:27:35 reyk Exp $ */
+/* $OpenBSD: aldap.c,v 1.2 2018/06/21 08:27:35 reyk Exp $ */
 
 /*
  * Copyright (c) 2008 Alexander Schrijver 
@@ -693,16 +693,14 @@ aldap_free_attr(char **values)
return (1);
 }
 
-#if 0
 void
 aldap_free_url(struct aldap_url *lu)
 {
free(lu->buffer);
-   free(lu->filter);
 }
 
 int
-aldap_parse_url(char *url, struct aldap_url *lu)
+aldap_parse_url(const char *url, struct aldap_url *lu)
 {
char*p, *forward, *forward2;
const char  *errstr = NULL;
@@ -712,10 +710,20 @@ aldap_parse_url(char *url, struct aldap_
return (-1);
 
/* protocol */
-   if (strncasecmp(LDAP_URL, p, strlen(LDAP_URL)) != 0)
-   goto fail;
-   lu->protocol = LDAP;
-   p += strlen(LDAP_URL);
+   if (strncasecmp(LDAP_URL, p, strlen(LDAP_URL)) == 0) {
+   lu->protocol = LDAP;
+   p += strlen(LDAP_URL);
+   } else if (strncasecmp(LDAPS_URL, p, strlen(LDAPS_URL)) == 0) {
+   lu->protocol = LDAPS;
+   p += strlen(LDAPS_URL);
+   } else if (strncasecmp(LDAPTLS_URL, p, strlen(LDAPTLS_URL)) == 0) {
+   lu->protocol = LDAPTLS;
+   p += strlen(LDAPTLS_URL);
+   } else if (strncasecmp(LDAPI_URL, p, strlen(LDAPI_URL)) == 0) {
+   lu->protocol = LDAPI;
+   p += strlen(LDAPI_URL);
+   } else
+   lu->protocol = -1;
 
/* host and optional port */
if ((forward = strchr(p, '/')) != NULL)
@@ -795,7 +803,6 @@ aldap_parse_url(char *url, struct aldap_
if (p)
lu->filter = p;
 done:
-   free(url);
return (1);
 fail:
free(lu->buffer);
@@ -805,7 +812,7 @@ fail:
 
 int
 aldap_search_url(struct aldap *ldap, char *url, int typesonly, int sizelimit,
-int timelimit)
+int timelimit, struct aldap_page_control *page)
 {
struct aldap_url *lu;
 
@@ -816,7 +823,7 @@ aldap_search_url(struct aldap *ldap, cha
goto fail;
 
if (aldap_search(ldap, lu->dn, lu->scope, lu->filter, lu->attributes,
-   typesonly, sizelimit, timelimit) == -1)
+   typesonly, sizelimit, timelimit, page) == -1)
goto fail;
 
aldap_free_url(lu);
@@ -825,7 +832,6 @@ fail:
aldap_free_url(lu);
return (-1);
 }
-#endif /* 0 */
 
 /*
  * internal functions
@@ -1277,7 +1283,7 @@ ldap_debug_elements(struct ber_element *
fprintf(stderr, "\n");
break;
}
-   fprintf(stderr, "string \"%.*s\"\n",  len, buf);
+   fprintf(stderr, "string \"%.*s\"\n",  (int)len, buf);
break;
case BER_TYPE_NULL: /* no payload */
case BER_TYPE_EOC:
Index: usr.sbin/ypldap/aldap.h
===
RCS file: /cvs/src/usr.sbin/ypldap/aldap.h,v
retrieving revision 1.10
diff -u -p -u -p -r1.10 aldap.h
--- usr.sbin/ypldap/aldap.h 30 May 2017 09:33:31 -  1.10
+++ usr.sbin/ypldap/aldap.h 21 Jun 2018 08:29:52 -
@@ -1,5 +1,5 @@
-/* $Id: aldap.h,v 1.10 2017/05/30 09:33:31 jmatthew Exp $ */
-/* $OpenBSD: aldap.h,v 1.10 2017/05/30 09:33:31 jmatthew Exp $ */
+/* $Id: aldap.h,v 1.1.1.1 2018/06/13 15:45:57 reyk Exp $ */
+/* $OpenBSD: aldap.h,v 1.1.1.1 2018/06/13 15:45:57 reyk Exp $ */
 
 /*
  * Copyright (c) 2008 Alexander Schrijver 
@@ -25,6 +25,10 @@
 #include "ber.h"
 
 #define LDAP_URL   "ldap://";
+#define LDAPS_URL  "ldaps://"
+#define LDAPTLS_URL"ldap+tls://"
+#define LDAPI_URL  "ldapi://"
+
 #define LDAP_PORT  389
 #define LDAPS_PORT 636
 #define LDAP_PAGED_OID "1.2.840.113556.1.4.319"
@@ -79,7 +83,9 @@ struct aldap_message {
 
 enum aldap_protocol {
LDAP,
-   LDAPS
+   LDAPS,
+   LDAPTLS,
+   LDAPI
 };
 
 struct aldap_url {
@@ -222,11 +228,10 @@ char  *aldap_get_dn(struct aldap_message 
 char   *aldap_get_diagmsg(struct aldap_message *);
 char   **aldap_get_references(struct aldap_message *);
 voidaldap_free_references(char **values);
-#if 0
-int aldap_parse_url(char *, struct aldap_url *);
+int aldap_parse_url(const char *, struct aldap_url *);
 voidaldap_free_

tech

[Reyk Floeter]

Hi,

when we develop code in OpenBSD, we have the choice of reviewing or
sharing our patches privately between individual developers, on an
internal list, or here on tech@.


>From https://www.openbsd.org/mail.html:

tech
Discussion of technical topics for OpenBSD developers and
advanced users. This is not a "tech support" forum; do not use it as
such. OpenBSD developers will often make patches to implement new
features and other important changes available for public testing
through this list. 


When I share code on tech@ it is intended to be reviewed and tested by
the wider community, not just developers.  I'm writing this mail
because I get a lot of cheerful feedback about some features, like the
recent httpd rewrite support, but almost no feedback on tech@ itself.

In the rewrite case, people were asking me in private emails, on
github, on Twitter: "Where is the feature?" or "Why does it take so
long to review?".  Despite the fact that the latter made me angry,
you're not my customer and this is voluntary work, I have the
following request:


Please read tech@, test diffs, share your thoughts (but keep the noise
on misc@).  Don't be shy: tech is NOT a list that is only for OpenBSD
developers.  If you're reading this list you're probably already an
"advanced user" - so take some time and give us your feedback!


Thanks!
Reyk

Re: httpd 3/3: request rewrite

[Reyk Floeter]

anyone?

On Wed, Jun 13, 2018 at 05:20:55PM +0200, Reyk Floeter wrote:
> On Wed, May 30, 2018 at 12:36:05AM +0200, Reyk Floeter wrote:
> > as mentioned in the big diff before, this implements rewrites.  This
> > diff applies on top of the previous ones.
> > 
> > Implement the "request rewrite" option for internal rewrites.
> > 
> > For example:
> > 
> > location match "/page/(%d+)/.*" {
> > request rewrite "/static/index.php?id=%1&$QUERY_STRING"
> > }
> > 
> > Please note that httpd uses patterns(7) and not regex.
> > 
> 
> Same, diff, re-applied on top of the previous commits.
> 
> OK?
> 
> Reyk
> 
> Index: usr.sbin/httpd/config.c
> ===
> RCS file: /cvs/src/usr.sbin/httpd/config.c,v
> retrieving revision 1.54
> diff -u -p -u -p -r1.54 config.c
> --- usr.sbin/httpd/config.c   19 May 2018 13:56:56 -  1.54
> +++ usr.sbin/httpd/config.c   13 Jun 2018 15:18:24 -
> @@ -476,6 +476,13 @@ config_getserver_config(struct httpd *en
>   &parent->default_type, sizeof(struct media_type));
>   }
>  
> + f = SRVFLAG_PATH_REWRITE|SRVFLAG_NO_PATH_REWRITE;
> + if ((srv_conf->flags & f) == 0) {
> + srv_conf->flags |= parent->flags & f;
> + (void)strlcpy(srv_conf->path, parent->path,
> + sizeof(srv_conf->path));
> + }
> +
>   f = SRVFLAG_SERVER_HSTS;
>   srv_conf->flags |= parent->flags & f;
>   srv_conf->hsts_max_age = parent->hsts_max_age;
> Index: usr.sbin/httpd/httpd.conf.5
> ===
> RCS file: /cvs/src/usr.sbin/httpd/httpd.conf.5,v
> retrieving revision 1.97
> diff -u -p -u -p -r1.97 httpd.conf.5
> --- usr.sbin/httpd/httpd.conf.5   13 Jun 2018 15:08:24 -  1.97
> +++ usr.sbin/httpd/httpd.conf.5   13 Jun 2018 15:18:25 -
> @@ -198,6 +198,8 @@ argument can be used with return codes i
>  .Sq Location:
>  header for redirection to a specified URI.
>  .Pp
> +It is possible to rewrite the request to redirect it to a different
> +external location.
>  The
>  .Ar uri
>  may contain predefined macros that will be expanded at runtime:
> @@ -396,10 +398,10 @@ the
>  using pattern matching instead of shell globbing rules,
>  see
>  .Xr patterns 7 .
> -The pattern may contain captures that can be used in the
> -.Ar uri
> -of an enclosed
> +The pattern may contain captures that can be used in an enclosed
>  .Ic block return
> +or
> +.Ic request rewrite
>  option.
>  .It Oo Ic no Oc Ic log Op Ar option
>  Set the specified logging options.
> @@ -462,6 +464,19 @@ in a location.
>  Configure the options for the request path.
>  Valid options are:
>  .Bl -tag -width Ds
> +.It Oo Ic no Oc Ic rewrite Ar path
> +Enable or disable rewriting of the request.
> +Unlike the redirection with
> +.Ic block return ,
> +this will change the request path internally before
> +.Nm httpd
> +makes a final decision about the matching location.
> +The
> +.Ar path
> +argument may contain predefined macros that will be expanded at runtime.
> +See the
> +.Ic block return
> +option for the list of supported macros.
>  .It Ic strip Ar number
>  Strip
>  .Ar number
> @@ -721,6 +736,17 @@ server "example.com" {
>  
>  server "www.example.com" {
>   listen on 10.0.0.1 port 80
> +}
> +.Ed
> +The request can also be rewritten with the
> +.Ic request rewrite
> +directive:
> +.Bd -literal -offset indent
> +server "example.com" {
> + listen on * port 80
> + location match "/old/(.*)" {
> + request rewrite "/new/%1"
> + }
>  }
>  .Ed
>  .Sh SEE ALSO
> Index: usr.sbin/httpd/httpd.h
> ===
> RCS file: /cvs/src/usr.sbin/httpd/httpd.h,v
> retrieving revision 1.137
> diff -u -p -u -p -r1.137 httpd.h
> --- usr.sbin/httpd/httpd.h19 May 2018 13:56:56 -  1.137
> +++ usr.sbin/httpd/httpd.h13 Jun 2018 15:18:25 -
> @@ -398,13 +398,15 @@ SPLAY_HEAD(client_tree, client);
>  #define SRVFLAG_SERVER_MATCH 0x0020
>  #define SRVFLAG_SERVER_HSTS  0x0040
>  #define SRVFLAG_DEFAULT_TYPE 0x0080
> +#define SRVFLAG_PATH_REWRITE 0x0100
> +#define SRVFLAG_NO_PATH_REWRITE  0x0200
>  
>  #define SRVFLAG_BITS 

ldap(1) -y secretfile

[Reyk Floeter]

Hi,

the following diff adds support for reading the bind secret from a
file; this allows to hide it from ps.  The -y flag is once again
compatible with OpenLDAP's client.

Pointed out by Tim Chase on Twitter.

OK?

Reyk

Index: usr.bin/ldap/ldap.1
===
RCS file: /cvs/src/usr.bin/ldap/ldap.1,v
retrieving revision 1.5
diff -u -p -u -p -r1.5 ldap.1
--- usr.bin/ldap/ldap.1 17 Jun 2018 23:12:48 -  1.5
+++ usr.bin/ldap/ldap.1 20 Jun 2018 14:17:27 -
@@ -31,6 +31,7 @@
 .Op Fl l Ar timelimit
 .Op Fl s Ar scope
 .Op Fl w Ar secret
+.Op Fl y Ar secretfile
 .Op Fl z Ar sizelimit
 .Op Ar arguments ...
 .Sh DESCRIPTION
@@ -152,6 +153,13 @@ Use simple authentication.
 This is the default as
 .Nm
 does not support SASL authentication.
+.It Fl y Ar secretfile
+Read the bind secret from the first line of the specified file or from
+standard input if the
+.Ar secretfile
+argument is
+.Sq - .
+The file must not be world-readable if it is a regular file.
 .It Fl Z
 Enable TLS using the StartTLS operation.
 .It Fl z Ar sizelimit
Index: usr.bin/ldap/ldapclient.c
===
RCS file: /cvs/src/usr.bin/ldap/ldapclient.c,v
retrieving revision 1.1.1.1
diff -u -p -u -p -r1.1.1.1 ldapclient.c
--- usr.bin/ldap/ldapclient.c   13 Jun 2018 15:45:58 -  1.1.1.1
+++ usr.bin/ldap/ldapclient.c   20 Jun 2018 14:17:27 -
@@ -19,6 +19,7 @@
 #include 
 #include 
 #include 
+#include 
 #include 
 #include 
 
@@ -55,6 +56,7 @@
 #define LDAPHOST   "localhost"
 #define LDAPFILTER "(objectClass=*)"
 #define LDIF_LINELENGTH79
+#define LDAPPASSMAX1024
 
 struct ldapc {
struct aldap*ldap_al;
@@ -95,8 +97,8 @@ usage(void)
 
fprintf(stderr,
 "usage: %s search [-LvxZ] [-b basedn] [-c capath] [-D binddn] [-H host]\n"
-"   [-l timelimit] [-s scope] [-w secret|-W] [-z sizelimit]\n"
-"   [filter] [attributes ...]\n",
+"   [-l timelimit] [-s scope] [-w secret|-W] [-y secretfile]\n"
+"   [-z sizelimit] [filter] [attributes ...]\n",
__progname);
 
exit(1);
@@ -105,12 +107,14 @@ usage(void)
 int
 main(int argc, char *argv[])
 {
-   char passbuf[BUFSIZ];
-   const char  *errstr, *url = NULL;
+   char passbuf[LDAPPASSMAX];
+   const char  *errstr, *url = NULL, *secretfile = NULL;
+   struct stat  st;
struct ldapc ldap;
struct ldapc_search  ls;
int  ch;
int  verbose = 1;
+   FILE*fp;
 
if (pledge("stdio inet unix tty rpath dns", NULL) == -1)
err(1, "pledge");
@@ -135,7 +139,7 @@ main(int argc, char *argv[])
argc--;
argv++;
 
-   while ((ch = getopt(argc, argv, "b:c:D:H:Ll:s:vWw:xZz:")) != -1) {
+   while ((ch = getopt(argc, argv, "b:c:D:H:Ll:s:vWw:xy:Zz:")) != -1) {
switch (ch) {
case 'b':
ls.ls_basedn = optarg;
@@ -182,6 +186,10 @@ main(int argc, char *argv[])
case 'x':
/* provided for compatibility */
break;
+   case 'y':
+   secretfile = optarg;
+   ldap.ldap_flags |= F_NEEDAUTH;
+   break;
case 'Z':
ldap.ldap_flags |= F_STARTTLS;
break;
@@ -224,6 +232,27 @@ main(int argc, char *argv[])
if (ldap.ldap_binddn == NULL) {
log_warnx("missing -D binddn");
usage();
+   }
+   if (secretfile != NULL) {
+   if (ldap.ldap_secret != NULL)
+   errx(1, "conflicting -w/-y options");
+
+   /* read password from stdin or file (first line) */
+   if (strcmp(secretfile, "-") == 0)
+   fp = stdin;
+   else if (stat(secretfile, &st) == -1)
+   err(1, "failed to access %s", secretfile);
+   else if (S_ISREG(st.st_mode) && (st.st_mode & S_IROTH))
+   errx(1, "%s is world-readable", secretfile);
+   else if ((fp = fopen(secretfile, "r")) == NULL)
+   err(1, "failed to open %s", secretfile);
+   if (fgets(passbuf, sizeof(passbuf), fp) == NULL)
+   err(1, "failed to read %s", secretfile);
+   if (fp != stdin)
+   fclose(fp);
+
+   passbuf[strcspn(passbuf, "\n")] = '\0';
+   ldap.ldap_secret = passbuf;
}
if (ldap.l

Re: httpd response mimetype bug

[Reyk Floeter]

On Sat, Jan 13, 2018 at 05:23:35PM +0100, Sebastian Benoit wrote:
> Hiltjo Posthuma(hil...@codemadness.org) on 2018.01.13 13:08:38 +0100:
> > On Sat, Jan 13, 2018 at 09:39:44AM +0100, Anton Lindqvist wrote:
> > > On Tue, Jan 09, 2018 at 05:38:57PM +0100, Hidv?gi G?bor wrote:
> > > > >Synopsis: httpd reports wrong mimetype when item is in the browser 
> > > > >cache
> > > > >Category: httpd
> > > > >Environment:
> > > > System  : OpenBSD 6.2
> > > > Details : OpenBSD 6.2 (GENERIC) #91: Wed Oct  4 00:35:21 MDT
> > > > 2017
> > > > 
> > > > dera...@armv7.openbsd.org:/usr/src/sys/arch/armv7/compile/GENERIC
> > > > 
> > > > Architecture: OpenBSD.armv7
> > > > Machine : armv7
> > > > >Description:
> > > > 
> > > > httpd serves static files (eg. images) with Last-Modified http header. 
> > > > When
> > > > a browser next time asks whether this file changed (sends 
> > > > If-Modified-Since
> > > > http header) httpd responds with wrong mimetype, 'text/html' when the
> > > > resource is in the browser cache (304 Not Modified status code).
> > > > 
> > > > >How-To-Repeat:
> > > > 
> > > > This bug is common, not arm only. When for example you open this image:
> > > > https://man.openbsd.org/openbsd.gif
> > > > 
> > > > in a browser with developer tools (F12) open, on the network tab you can
> > > > take a look at the response headers, mimetype is correct (image/gif). 
> > > > After
> > > > opening press refresh (F5) and look at the response headers again, and 
> > > > you
> > > > get the incorrect mimetype, 'text/html'.
> > > > 
> > > > >Fix:
> > > > 
> > > > check httpd source
> > > 
> > > Please try out this diff, it makes sure to set the correct MIME-type and
> > > not respond with a body if the resource has not changed. Sending this to
> > > tech@ as well.
> > > 
> > > Index: server_file.c
> > > ===
> > > RCS file: /cvs/src/usr.sbin/httpd/server_file.c,v
> > > retrieving revision 1.65
> > > diff -u -p -r1.65 server_file.c
> > > --- server_file.c 2 Feb 2017 22:19:59 -   1.65
> > > +++ server_file.c 12 Jan 2018 19:10:20 -
> > > @@ -230,8 +230,15 @@ server_file_request(struct httpd *env, s
> > >   goto abort;
> > >   }
> > >  
> > > - if ((ret = server_file_modified_since(clt->clt_descreq, st)) != -1)
> > > - return (ret);
> > > + if (server_file_modified_since(clt->clt_descreq, st) == 0) {
> > > + media = media_find_config(env, srv_conf, path);
> > > + ret = server_response_http(clt, 304, media, 0,
> > > + st->st_mtim.tv_sec);
> > > + if (ret != -1)
> > > + goto done;
> > > + else
> > > + goto fail;
> > > + }
> > >  
> > >   /* Now open the file, should be readable or we have another problem */
> > >   if ((fd = open(path, O_RDONLY)) == -1)
> > > @@ -663,10 +670,10 @@ server_file_modified_since(struct http_d
> > >   if (strptime(since->kv_value,
> > >   "%a, %d %h %Y %T %Z", &tm) != NULL &&
> > >   timegm(&tm) >= st->st_mtim.tv_sec)
> > > - return (304);
> > > + return (0);
> > >   }
> > >  
> > > - return (-1);
> > > + return (1);
> > >  }
> > >  
> > >  int
> > > 
> > 
> > Hey,
> > 
> > I've tested your patch.
> > 
> > When requesting a non-modified CSS file:
> > 
> > #!/bin/sh
> > host="127.0.0.1"
> > port="6970"
> > printf 'GET /style.css HTTP/1.1\r\nHost: %s:%s\r\nIf-Modified-Since: 
> > Sat, 16 Dec 2017 13:07:53 GMT\r\nConnection: close\r\n\r\n' "$host" "$port" 
> > | \
> > nc "$host" "$port"
> > 
> > Full HTTP response:
> > 
> > HTTP/1.1 304 Not Modified
> > Connection: close
> > Content-Length: 0
> > Content-Type: text/css
> > Date: Sat, 13 Jan 2018 11:54:13 GMT
> > Last-Modified: Sun, 05 Mar 2017 12:22:05 GMT
> > Server: OpenBSD httpd
> > 
> > I wonder if httpd should just omit the response header Content-Length and
> > Content-Type entirely for this statuscode. Some httpd such as Nginx just
> > omit them aswell.
> 
> rfc7230  HTTP/1.1 Message Syntax and Routing page 29f.
> 
>A server MAY send a Content-Length header field in a 304 (Not
>Modified) response to a conditional GET request (Section 4.1 of
>[RFC7232]); a server MUST NOT send Content-Length in such a response
>   *unless its field-value equals the decimal number of octets that would*
>have been sent in the payload body of a 200 (OK) response to the same
>request.
> 

So I think it is better to omit the Content-Length header.  I'm afraid
that a (potentially broken) client will wait for a body even with the
304 response.

The diff below is based on Anton's fix but without changing the code's
style to a boolean 0/1.  It also avoids an mtime in the future (same
as the other server_response_http() further below), and omits the
Content-Length by accepting a -1 size.

Reyk

Index: usr.sbin/httpd/server_fil

Re: httpd 3/3: request rewrite

[Reyk Floeter]

On Wed, May 30, 2018 at 12:36:05AM +0200, Reyk Floeter wrote:
> as mentioned in the big diff before, this implements rewrites.  This
> diff applies on top of the previous ones.
> 
> Implement the "request rewrite" option for internal rewrites.
> 
> For example:
> 
> location match "/page/(%d+)/.*" {
> request rewrite "/static/index.php?id=%1&$QUERY_STRING"
> }
> 
> Please note that httpd uses patterns(7) and not regex.
> 

Same, diff, re-applied on top of the previous commits.

OK?

Reyk

Index: usr.sbin/httpd/config.c
===
RCS file: /cvs/src/usr.sbin/httpd/config.c,v
retrieving revision 1.54
diff -u -p -u -p -r1.54 config.c
--- usr.sbin/httpd/config.c 19 May 2018 13:56:56 -  1.54
+++ usr.sbin/httpd/config.c 13 Jun 2018 15:18:24 -
@@ -476,6 +476,13 @@ config_getserver_config(struct httpd *en
&parent->default_type, sizeof(struct media_type));
}
 
+   f = SRVFLAG_PATH_REWRITE|SRVFLAG_NO_PATH_REWRITE;
+   if ((srv_conf->flags & f) == 0) {
+   srv_conf->flags |= parent->flags & f;
+   (void)strlcpy(srv_conf->path, parent->path,
+   sizeof(srv_conf->path));
+   }
+
f = SRVFLAG_SERVER_HSTS;
srv_conf->flags |= parent->flags & f;
srv_conf->hsts_max_age = parent->hsts_max_age;
Index: usr.sbin/httpd/httpd.conf.5
===
RCS file: /cvs/src/usr.sbin/httpd/httpd.conf.5,v
retrieving revision 1.97
diff -u -p -u -p -r1.97 httpd.conf.5
--- usr.sbin/httpd/httpd.conf.5 13 Jun 2018 15:08:24 -  1.97
+++ usr.sbin/httpd/httpd.conf.5 13 Jun 2018 15:18:25 -
@@ -198,6 +198,8 @@ argument can be used with return codes i
 .Sq Location:
 header for redirection to a specified URI.
 .Pp
+It is possible to rewrite the request to redirect it to a different
+external location.
 The
 .Ar uri
 may contain predefined macros that will be expanded at runtime:
@@ -396,10 +398,10 @@ the
 using pattern matching instead of shell globbing rules,
 see
 .Xr patterns 7 .
-The pattern may contain captures that can be used in the
-.Ar uri
-of an enclosed
+The pattern may contain captures that can be used in an enclosed
 .Ic block return
+or
+.Ic request rewrite
 option.
 .It Oo Ic no Oc Ic log Op Ar option
 Set the specified logging options.
@@ -462,6 +464,19 @@ in a location.
 Configure the options for the request path.
 Valid options are:
 .Bl -tag -width Ds
+.It Oo Ic no Oc Ic rewrite Ar path
+Enable or disable rewriting of the request.
+Unlike the redirection with
+.Ic block return ,
+this will change the request path internally before
+.Nm httpd
+makes a final decision about the matching location.
+The
+.Ar path
+argument may contain predefined macros that will be expanded at runtime.
+See the
+.Ic block return
+option for the list of supported macros.
 .It Ic strip Ar number
 Strip
 .Ar number
@@ -721,6 +736,17 @@ server "example.com" {
 
 server "www.example.com" {
listen on 10.0.0.1 port 80
+}
+.Ed
+The request can also be rewritten with the
+.Ic request rewrite
+directive:
+.Bd -literal -offset indent
+server "example.com" {
+   listen on * port 80
+   location match "/old/(.*)" {
+   request rewrite "/new/%1"
+   }
 }
 .Ed
 .Sh SEE ALSO
Index: usr.sbin/httpd/httpd.h
===
RCS file: /cvs/src/usr.sbin/httpd/httpd.h,v
retrieving revision 1.137
diff -u -p -u -p -r1.137 httpd.h
--- usr.sbin/httpd/httpd.h  19 May 2018 13:56:56 -  1.137
+++ usr.sbin/httpd/httpd.h  13 Jun 2018 15:18:25 -
@@ -398,13 +398,15 @@ SPLAY_HEAD(client_tree, client);
 #define SRVFLAG_SERVER_MATCH   0x0020
 #define SRVFLAG_SERVER_HSTS0x0040
 #define SRVFLAG_DEFAULT_TYPE   0x0080
+#define SRVFLAG_PATH_REWRITE   0x0100
+#define SRVFLAG_NO_PATH_REWRITE0x0200
 
 #define SRVFLAG_BITS   \
"\10\01INDEX\02NO_INDEX\03AUTO_INDEX\04NO_AUTO_INDEX"   \
"\05ROOT\06LOCATION\07FCGI\10NO_FCGI\11LOG\12NO_LOG\13SOCKET"   \
"\14SYSLOG\15NO_SYSLOG\16TLS\17ACCESS_LOG\20ERROR_LOG"  \
"\21AUTH\22NO_AUTH\23BLOCK\24NO_BLOCK\25LOCATION_MATCH" \
-   "\26SERVER_MATCH\27SERVER_HSTS\30DEFAULT_TYPE"
+   "\26SERVER_MATCH\27SERVER_HSTS\30DEFAULT_TYPE\31PATH\32NO_PATH"
 
 #define TCPFLAG_NODELAY0x01
 #define TCPFLAG_NNODELAY   0x02
@@ -470,8 +472,9 @@ struct server_config {
uint32_t   

Re: dhcrelay bpf bugs

[Reyk Floeter]

> Am 05.06.2018 um 16:27 schrieb asaxena9021 :
> 
> Hi David,
> 
> I am facing this problem with lpf.c
> 
> In function receive_packet , My packet is somehow is getting corrupted.
> 
> DHCP packet content is like below:
> 
> DHCP Packet Content:
>opcode : 15
>hardwaretype : 205
>hlen : 8
>hops : 0
>xid : 1157628060
>secs : 51797
>flags : 16384
>addr ciaddr : @▒▒dd
>addr yiaddr : dd
>siaddr  : d
>giaddr :
>chaddr :
>sname  : @▒_dd
>file  : @▒▒dd
>options : Vishal add_relay_agent_options:1132
> 
> 
> I am not able to understand where is my dhcp discover packet is getting
> corrupted. I also post a question about it on stackoverflow.
> 
> https://stackoverflow.com/questions/50655937/isc-dhcrelay-is-not-forwarding-the-discover-packets-to-dhcp-server?noredirect=1#comment88380485_50655937
> 
>   
> 

OpenBSD’s dhcrelay is not ISC dhcrelay.

Reyk

> Please check.
> 
> Please help me in this.
> 
> Thanks,
> Akash Saxena.
> 
> 
> 
> --
> Sent from: 
> http://openbsd-archive.7691.n7.nabble.com/openbsd-dev-tech-f151936.html
> 

Re: [Patch] mg(1): Experimental UTF-8 support

[Reyk Floeter]

> Am 30.05.2018 um 10:10 schrieb Leonid Bobrov :
> 
>> On Wed, May 30, 2018 at 09:05:12AM +0200, Hiltjo Posthuma wrote:
>>> On Tue, May 29, 2018 at 05:22:43PM +0300, Leonid Bobrov wrote:
 On Tue, May 29, 2018 at 03:33:08PM +0200, Henning Brauer wrote:
 Hi,
 
 very welcome!
 
 I have applied the diff and don't notice immediate breakage. Pls poke
>>> 
>>> You didn't notice cursor movement bugs? o_O
>>> Well, I'm giving example: авыавыавы
>>> To move from start to end of that word, you have to press M-f 3 times.
>>> 
>>> Also you might notice you have to press C-f twice to move one character
>>> forward.
>>> 
>>> https://github.com/hboetes/mg/commits/display-wide-characters
>>> 
>> 
>> The viewing looks good, but I can confirm to also notice the "character"
>> movement bug.
>> 
>> locale is all set to "en_US.UTF-8".
>> 
>> A good test file is I think:
>> https://www.cl.cam.ac.uk/~mgk25/ucs/examples/UTF-8-demo.txt
>> 
>> Thanks for the work so far. UTF-8 support would be really nice! :)
>> 
> 
> Thanks goes to S. Gilles , he made it :)
> 
> Recent commits fixed cursor movement bugs, my work is to send fresh
> patches to this list to make sure mg merges UTF-8 support <3
> 

I‘m in favor of this, mg is my number one productivity tool and UTF-8 support 
would be really really great. I will test it as well.

Reyk

>>> There are new commits, I'll test them later and send a new diff,
>>> so I hope mg is ready to support UTF-8, yeah <3
>>> 
 me in a few days to give an ok (and hope same brave soul takes
 committing on his plate) assuming I don't run into trouble.
 
 ciao
 
 Henning
 
>>> 
>> 
>> -- 
>> Kind regards,
>> Hiltjo
>> 
> 

Re: httpd request rewrite

[Reyk Floeter]

> Am 30.05.2018 um 10:12 schrieb Sebastian Benoit :
> 
> Sebastian Benoit(be...@openbsd.org) on 2018.05.30 10:10:51 +0200:
>> Reyk Floeter(r...@openbsd.org) on 2018.05.29 18:48:31 +0200:
>>> Hi,
>>> 
>>> it's about time.
>>> 
>>>server "default" {
>>>listen on * port 80
>>>location match "/de/(.*)" {
>>>request rewrite "/ch/%1"
>>>}
>>>}
>>> 
>>> You can also you the macros as in the "block return" external
>>> redirects.  So maybe something like:
>>> 
>>>server "default" {
>>>listen on * port 80
>>>location match "/(.*)" {
>>>request rewrite "/$HTTP_HOST/%1"
>>>}
>>>}
>>> 
>>> Tests? OK?
>> 
>> nice,
>> 
>> but why use %1 instead of $1 as all other mechanisms of this kind do (that i
>> knwo of)?
> 
> to answer myself: patterns(7)

Right ;-) I’m going to stick with it.

Reyk

Re: httpd 2/3: rename "root strip" to "request strip"

[Reyk Floeter]

On Wed, May 30, 2018 at 12:32:12AM +0200, Reyk Floeter wrote:
> Hi,
> 
> this diff applies on top of the previous one.
> 
> Rename "root strip" to "request strip"
> 
> The root strip option name was semantically incorrect as it does not
> strip the root but the request path.  This is a grammar change and it
> also needs a heads up and a change in other documentation (such as
> acme-client(1)).
> 

This is the related documentation diff for the tree.  Sorry for the
inconvenience, but it is better for the correctness of the grammar.

Reyk

Index: etc/examples/httpd.conf
===
RCS file: /cvs/src/etc/examples/httpd.conf,v
retrieving revision 1.19
diff -u -p -u -p -r1.19 httpd.conf
--- etc/examples/httpd.conf 11 Apr 2018 15:51:50 -  1.19
+++ etc/examples/httpd.conf 29 May 2018 22:44:24 -
@@ -4,7 +4,7 @@ server "example.com" {
listen on * port 80
location "/.well-known/acme-challenge/*" {
root "/acme"
-   root strip 2
+   request strip 2
}
location * {
block return 302 "https://$HTTP_HOST$REQUEST_URI";
@@ -22,6 +22,6 @@ server "example.com" {
}
location "/.well-known/acme-challenge/*" {
root "/acme"
-   root strip 2
+   request strip 2
}
 }
Index: usr.sbin/acme-client/acme-client.1
===
RCS file: /cvs/src/usr.sbin/acme-client/acme-client.1,v
retrieving revision 1.23
diff -u -p -u -p -r1.23 acme-client.1
--- usr.sbin/acme-client/acme-client.1  17 Oct 2017 22:47:58 -  1.23
+++ usr.sbin/acme-client/acme-client.1  29 May 2018 22:44:24 -
@@ -83,7 +83,7 @@ which will properly map response challen
 .Bd -literal -offset indent
 location "/.well-known/acme-challenge/*" {
root "/acme"
-   root strip 2
+   request strip 2
 }
 .Ed
 .Sh FILES
Index: usr.sbin/acme-client/acme-client.conf.5
===
RCS file: /cvs/src/usr.sbin/acme-client/acme-client.conf.5,v
retrieving revision 1.11
diff -u -p -u -p -r1.11 acme-client.conf.5
--- usr.sbin/acme-client/acme-client.conf.5 27 Nov 2017 01:58:52 -  
1.11
+++ usr.sbin/acme-client/acme-client.conf.5 29 May 2018 22:44:24 -
@@ -169,7 +169,7 @@ server "example.com" {
 tls key "/etc/ssl/private/example.com.key"
 location "/.well-known/acme-challenge/*" {
 root "/acme"
-root strip 2
+request strip 2
 }
 root "/htdocs"
 }
Index: regress/usr.sbin/acme-client/httpd.conf
===
RCS file: /cvs/src/regress/usr.sbin/acme-client/httpd.conf,v
retrieving revision 1.1
diff -u -p -u -p -r1.1 httpd.conf
--- regress/usr.sbin/acme-client/httpd.conf 25 Jun 2017 21:33:23 -  
1.1
+++ regress/usr.sbin/acme-client/httpd.conf 29 May 2018 22:44:24 -
@@ -3,6 +3,6 @@ server "default" {
listen on "*" port 80
location "/.well-known/acme-challenge/*" {
root "/acme"
-   root strip 2
+   request strip 2
}
 }

httpd 3/3: request rewrite

[Reyk Floeter]

ON LOG LOGDIR MATCH MAXIMUM NO NODELAY OCSP ON PORT PREFORK
 %token PROTOCOLS REQUESTS ROOT SACK SERVER SOCKET STRIP STYLE SYSLOG TCP TICKET
 %token TIMEOUT TLS TYPE TYPES HSTS MAXAGE SUBDOMAINS DEFAULT PRELOAD REQUEST
-%token ERROR INCLUDE AUTHENTICATE WITH BLOCK DROP RETURN PASS
+%token ERROR INCLUDE AUTHENTICATE WITH BLOCK DROP RETURN PASS REWRITE
 %token CA CLIENT CRL OPTIONAL
 %token   STRING
 %token   NUMBER
@@ -815,7 +815,23 @@ requestflags_l : requestflags optcommanl requestflags_l
| requestflags optnl
;
 
-requestflags   : STRIP NUMBER  {
+requestflags   : REWRITE STRING{
+   if (strlcpy(srv->srv_conf.path, $2,
+   sizeof(srv->srv_conf.path)) >=
+   sizeof(srv->srv_conf.path)) {
+   yyerror("request path too long");
+   free($2);
+   YYERROR;
+   }
+   free($2);
+   srv->srv_conf.flags |= SRVFLAG_PATH_REWRITE;
+   srv->srv_conf.flags &= ~SRVFLAG_NO_PATH_REWRITE;
+   }
+   | NO REWRITE{
+   srv->srv_conf.flags |= SRVFLAG_NO_PATH_REWRITE;
+   srv->srv_conf.flags &= ~SRVFLAG_PATH_REWRITE;
+   }
+   | STRIP NUMBER  {
if ($2 < 0 || $2 > INT_MAX) {
yyerror("invalid strip number");
YYERROR;
@@ -1272,6 +1288,7 @@ lookup(char *s)
{ "request",REQUEST },
{ "requests",   REQUESTS },
{ "return", RETURN },
+   { "rewrite",REWRITE },
{ "root",   ROOT },
{ "sack",   SACK },
{ "server", SERVER },
diff --git usr.sbin/httpd/server_http.c usr.sbin/httpd/server_http.c
index 5a8fc43..c9af556 100644
--- usr.sbin/httpd/server_http.c
+++ usr.sbin/httpd/server_http.c
@@ -1,7 +1,7 @@
 /* $OpenBSD: server_http.c,v 1.119 2018/04/06 13:02:07 florian Exp $   
*/
 
 /*
- * Copyright (c) 2006 - 2017 Reyk Floeter 
+ * Copyright (c) 2006 - 2018 Reyk Floeter 
  *
  * Permission to use, copy, modify, and distribute this software for any
  * purpose with or without fee is hereby granted, provided that the above
@@ -1170,13 +1170,16 @@ server_response(struct httpd *httpd, struct client *clt)
struct kv   *kv, key, *host;
struct str_find  sm;
int  portval = -1, ret;
-   char*hostval;
+   char*hostval, *query;
const char  *errstr = NULL;
 
-   /* Canonicalize the request path */
+   /* Decode the URL */
if (desc->http_path == NULL ||
-   url_decode(desc->http_path) == NULL ||
-   canonicalize_path(desc->http_path, path, sizeof(path)) == NULL)
+   url_decode(desc->http_path) == NULL)
+   goto fail;
+
+   /* Canonicalize the request path */
+   if (canonicalize_path(desc->http_path, path, sizeof(path)) == NULL)
goto fail;
free(desc->http_path);
if ((desc->http_path = strdup(path)) == NULL)
@@ -1280,6 +1283,42 @@ server_response(struct httpd *httpd, struct client *clt)
/* Now search for the location */
srv_conf = server_getlocation(clt, desc->http_path);
 
+   /* Optional rewrite */
+   if (srv_conf->flags & SRVFLAG_PATH_REWRITE) {
+   /* Expand macros */
+   if (server_expand_http(clt, srv_conf->path,
+   path, sizeof(path)) == NULL)
+   goto fail;
+
+   /*
+* Reset and update the query.  The updated query must already
+* be URL encoded - either specified by the user or by using the
+* original $QUERY_STRING.
+*/
+   free(desc->http_query);
+   desc->http_query = NULL;
+   if ((query = strchr(path, '?')) != NULL) {
+   *query++ = '\0';
+   if ((desc->http_query = strdup(query)) == NULL)
+   goto fail;
+   }
+
+   /* Canonicalize the updated request path */
+   if (canonicalize_path(path,
+   path, sizeof(path)) == NULL)
+   goto fail;
+
+   log_debug("%s: rewrote %s -> %s?%s", __func__,
+   desc->http_path, path, desc->http_query);
+
+   free(

httpd 2/3: rename "root strip" to "request strip"

[Reyk Floeter]

Hi,

this diff applies on top of the previous one.

Rename "root strip" to "request strip"

The root strip option name was semantically incorrect as it does not
strip the root but the request path.  This is a grammar change and it
also needs a heads up and a change in other documentation (such as
acme-client(1)).

OK?

Reyk

diff --git usr.sbin/httpd/httpd.conf.5 usr.sbin/httpd/httpd.conf.5
index 9c7efbc..2a71c8d 100644
--- usr.sbin/httpd/httpd.conf.5
+++ usr.sbin/httpd/httpd.conf.5
@@ -458,12 +458,18 @@ instead of the log files.
 Disable any previous
 .Ic block
 in a location.
-.It Ic root Ar option
-Configure the document root and options for the request path.
+.It Ic request Ar option
+Configure the options for the request path.
 Valid options are:
 .Bl -tag -width Ds
-.It Ar directory
-Set the document root of the server.
+.It Ic strip Ar number
+Strip
+.Ar number
+path components from the beginning of the request path before looking
+up the stripped-down path at the document root.
+.El
+.It Ic root Ar directory
+Configure the document root of the server.
 The
 .Ar directory
 is a pathname within the
diff --git usr.sbin/httpd/parse.y usr.sbin/httpd/parse.y
index 2cdfab5..5766e96 100644
--- usr.sbin/httpd/parse.y
+++ usr.sbin/httpd/parse.y
@@ -486,6 +486,7 @@ serveroptsl : LISTEN ON STRING opttls port  {
YYERROR;
}
}
+   | request
| root
| directory
| logformat
@@ -804,7 +805,17 @@ rootflags  : STRING{
free($1);
srv->srv_conf.flags |= SRVFLAG_ROOT;
}
-   | STRIP NUMBER  {
+   ;
+
+request: REQUEST requestflags
+   | REQUEST '{' optnl requestflags_l '}'
+   ;
+
+requestflags_l : requestflags optcommanl requestflags_l
+   | requestflags optnl
+   ;
+
+requestflags   : STRIP NUMBER  {
if ($2 < 0 || $2 > INT_MAX) {
yyerror("invalid strip number");
YYERROR;

httpd 1/3: don't encode the query string twice

[Reyk Floeter]

Hi,

the first diff fixes a bug that I found with the work on rewrites.

The http_query is already url_encoded; don't encode it twice.

This fixes a bug in the macros and log file handler that
double-encoded the query.  This does not change FCGI as it was already
handling the query correctly.

OK?

Reyk

diff --git usr.sbin/httpd/httpd.conf.5 usr.sbin/httpd/httpd.conf.5
index df4ea10..9c7efbc 100644
--- usr.sbin/httpd/httpd.conf.5
+++ usr.sbin/httpd/httpd.conf.5
@@ -206,7 +206,7 @@ may contain predefined macros that will be expanded at 
runtime:
 .It Ic $DOCUMENT_URI
 The request path.
 .It Ic $QUERY_STRING
-The optional query string of the request.
+The URL encoded query string of the request.
 .It Ic $REMOTE_ADDR
 The IP address of the connected client.
 .It Ic $REMOTE_PORT
@@ -218,7 +218,7 @@ The request path and optional query string.
 .It Ic $SERVER_ADDR
 The configured IP address of the server.
 .It Ic $SERVER_PORT
-The configured TCP server port of the server.
+The configured TCP port of the server.
 .It Ic $SERVER_NAME
 The name of the server.
 .It Ic $HTTP_HOST
diff --git usr.sbin/httpd/server_http.c usr.sbin/httpd/server_http.c
index c4c0240..5a8fc43 100644
--- usr.sbin/httpd/server_http.c
+++ usr.sbin/httpd/server_http.c
@@ -1023,7 +1023,7 @@ server_expand_http(struct client *clt, const char *val, 
char *buf,
 {
struct http_descriptor  *desc = clt->clt_descreq;
struct server_config*srv_conf = clt->clt_srv_conf;
-   char ibuf[128], *str, *path, *query;
+   char ibuf[128], *str, *path;
const char  *errstr = NULL, *p;
size_t   size;
int  n, ret;
@@ -1067,10 +1067,8 @@ server_expand_http(struct client *clt, const char *val, 
char *buf,
if (desc->http_query == NULL) {
ret = expand_string(buf, len, "$QUERY_STRING", "");
} else {
-   if ((query = url_encode(desc->http_query)) == NULL)
-   return (NULL);
-   ret = expand_string(buf, len, "$QUERY_STRING", query);
-   free(query);
+   ret = expand_string(buf, len, "$QUERY_STRING",
+   desc->http_query);
}
if (ret != 0)
return (NULL);
@@ -1119,13 +1117,8 @@ server_expand_http(struct client *clt, const char *val, 
char *buf,
if (desc->http_query == NULL) {
str = path;
} else {
-   if ((query = url_encode(desc->http_query)) == NULL) {
-   free(path);
-   return (NULL);
-   }
-   ret = asprintf(&str, "%s?%s", path, query);
+   ret = asprintf(&str, "%s?%s", path, desc->http_query);
free(path);
-   free(query);
if (ret == -1)
return (NULL);
}
@@ -1591,7 +1584,6 @@ server_log_http(struct client *clt, unsigned int code, 
size_t len)
int  ret = -1;
char*user = NULL;
char*path = NULL;
-   char*query = NULL;
char*version = NULL;
char*referrer_v = NULL;
char*agent_v = NULL;
@@ -1635,9 +1627,6 @@ server_log_http(struct client *clt, unsigned int code, 
size_t len)
if (desc->http_path &&
(path = url_encode(desc->http_path)) == NULL)
goto done;
-   if (desc->http_query &&
-   (query = url_encode(desc->http_query)) == NULL)
-   goto done;
 
ret = evbuffer_add_printf(clt->clt_log,
"%s %s - %s [%s] \"%s %s%s%s%s%s\" %03d %zu\n",
@@ -1646,7 +1635,7 @@ server_log_http(struct client *clt, unsigned int code, 
size_t len)
server_httpmethod_byid(desc->http_method),
desc->http_path == NULL ? "" : path,
desc->http_query == NULL ? "" : "?",
-   desc->http_query == NULL ? "" : query,
+   desc->http_query == NULL ? "" : desc->http_query,
desc->http_version == NULL ? "" : " ",
desc->http_version == NULL ? "" : version,
code, len);
@@ -1679,9 +1668,6 @@ server_log_http(struct client *clt, unsigned int code, 
size_t len)
if (desc->http_path &&
(path = url_encode(desc->http_path)) == NULL)
goto done;
-   if (desc->http_query &&
-   (query = url_encode(desc->http_query)) == NULL)
-   goto done;

Re: httpd request rewrite

[Reyk Floeter]

On Tue, May 29, 2018 at 10:00:22PM +0200, Hiltjo Posthuma wrote:
> On Tue, May 29, 2018 at 06:48:31PM +0200, Reyk Floeter wrote:
> > Hi,
> > 
> > it's about time.
> > 
> > server "default" {
> > listen on * port 80
> > location match "/de/(.*)" {
> > request rewrite "/ch/%1"
> > }
> > }
> > 
> > You can also you the macros as in the "block return" external
> > redirects.  So maybe something like:
> > 
> > server "default" {
> > listen on * port 80
> > location match "/(.*)" {
> > request rewrite "/$HTTP_HOST/%1"
> > }
> > }
> > 
> 
> The syntax looks clear in my opinion.
> 
> Would it be a good idea in your opinion to have request path rewriting in
> relayd also? For example to rewrite the request path for the (internal)
> application?
> 
> For example for the url: "http://somesite.org/gopherproxy"; to rewrite the path
> "/gopherproxy" to "/" for the application?
> 
> in Nginx my current rule is like:
> 
>   location /gopherproxy/ {
>   rewrite/gopherproxy/(.*) /$1 break;
> 
>   proxy_pass   http://127.0.0.1:6969/;
>   proxy_set_header Host $host;
>   proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
>   proxy_pass_header Server;
>   }
> 
> I have read the man page and relayd source-code, but have not found a way to
> replace part of the pattern in the path (only a fixed string with "set").
> 

Well, maybe.  relayd is related but a different topic.  The filters
are still on my list.

Reyk

> > Tests? OK?
> > 
> > Please note that this diff intentionally breaks the "root strip"
> > option because it changes the grammar to "request strip".  "root
> > strip" was semantically wrong but we didn't have a better place to put
> > it.  An current.html entry can be made for the required grammar change.
> > 
> > Reyk
> > 
> > Index: usr.sbin/httpd/config.c
> > ===
> > RCS file: /cvs/src/usr.sbin/httpd/config.c,v
> > retrieving revision 1.54
> > diff -u -p -u -p -r1.54 config.c
> > --- usr.sbin/httpd/config.c 19 May 2018 13:56:56 -  1.54
> > +++ usr.sbin/httpd/config.c 29 May 2018 16:35:29 -
> > @@ -476,6 +476,13 @@ config_getserver_config(struct httpd *en
> > &parent->default_type, sizeof(struct media_type));
> > }
> >  
> > +   f = SRVFLAG_PATH_REWRITE|SRVFLAG_NO_PATH_REWRITE;
> > +   if ((srv_conf->flags & f) == 0) {
> > +   srv_conf->flags |= parent->flags & f;
> > +   (void)strlcpy(srv_conf->path, parent->path,
> > +   sizeof(srv_conf->path));
> > +   }
> > +
> > f = SRVFLAG_SERVER_HSTS;
> > srv_conf->flags |= parent->flags & f;
> > srv_conf->hsts_max_age = parent->hsts_max_age;
> > Index: usr.sbin/httpd/httpd.conf.5
> > ===
> > RCS file: /cvs/src/usr.sbin/httpd/httpd.conf.5,v
> > retrieving revision 1.95
> > diff -u -p -u -p -r1.95 httpd.conf.5
> > --- usr.sbin/httpd/httpd.conf.5 23 May 2018 19:02:50 -  1.95
> > +++ usr.sbin/httpd/httpd.conf.5 29 May 2018 16:35:29 -
> > @@ -198,6 +198,8 @@ argument can be used with return codes i
> >  .Sq Location:
> >  header for redirection to a specified URI.
> >  .Pp
> > +It is possible to rewrite the request to redirect it to a different
> > +external location.
> >  The
> >  .Ar uri
> >  may contain predefined macros that will be expanded at runtime:
> > @@ -396,10 +398,10 @@ the
> >  using pattern matching instead of shell globbing rules,
> >  see
> >  .Xr patterns 7 .
> > -The pattern may contain captures that can be used in the
> > -.Ar uri
> > -of an enclosed
> > +The pattern may contain captures that can be used in an enclosed
> >  .Ic block return
> > +or
> > +.Ic request rewrite
> >  option.
> >  .It Oo Ic no Oc Ic log Op Ar option
> >  Set the specified logging options.
> > @@ -458,12 +460,31 @@ instead of the log files.
> >  Disable any previous
> >  .Ic block
> >  in a location.
> 

Re: httpd request rewrite

[Reyk Floeter]

On Tue, May 29, 2018 at 06:48:31PM +0200, Reyk Floeter wrote:
> it's about time.
> 
>   server "default" {
>   listen on * port 80
>   location match "/de/(.*)" {
>   request rewrite "/ch/%1"
>   }
>   }
> 
> Tests? OK?
> 

I didn't handle the query in my previous diff.  For example, a very
typical rewrite to handle a slug:

location match "/page/(%d+)/.*" {
request rewrite "/static/index.php?id=%1"
}

The new diff below became very large, I will split it into 3 and resend them.

Reyk

Index: usr.sbin/httpd/config.c
===
RCS file: /cvs/src/usr.sbin/httpd/config.c,v
retrieving revision 1.54
diff -u -p -u -p -r1.54 config.c
--- usr.sbin/httpd/config.c 19 May 2018 13:56:56 -  1.54
+++ usr.sbin/httpd/config.c 29 May 2018 21:41:55 -
@@ -476,6 +476,13 @@ config_getserver_config(struct httpd *en
&parent->default_type, sizeof(struct media_type));
}
 
+   f = SRVFLAG_PATH_REWRITE|SRVFLAG_NO_PATH_REWRITE;
+   if ((srv_conf->flags & f) == 0) {
+   srv_conf->flags |= parent->flags & f;
+   (void)strlcpy(srv_conf->path, parent->path,
+   sizeof(srv_conf->path));
+   }
+
f = SRVFLAG_SERVER_HSTS;
srv_conf->flags |= parent->flags & f;
srv_conf->hsts_max_age = parent->hsts_max_age;
Index: usr.sbin/httpd/httpd.conf.5
===
RCS file: /cvs/src/usr.sbin/httpd/httpd.conf.5,v
retrieving revision 1.95
diff -u -p -u -p -r1.95 httpd.conf.5
--- usr.sbin/httpd/httpd.conf.5 23 May 2018 19:02:50 -  1.95
+++ usr.sbin/httpd/httpd.conf.5 29 May 2018 21:41:55 -
@@ -198,6 +198,8 @@ argument can be used with return codes i
 .Sq Location:
 header for redirection to a specified URI.
 .Pp
+It is possible to rewrite the request to redirect it to a different
+external location.
 The
 .Ar uri
 may contain predefined macros that will be expanded at runtime:
@@ -206,7 +208,7 @@ may contain predefined macros that will 
 .It Ic $DOCUMENT_URI
 The request path.
 .It Ic $QUERY_STRING
-The optional query string of the request.
+The URL encoded query string of the request.
 .It Ic $REMOTE_ADDR
 The IP address of the connected client.
 .It Ic $REMOTE_PORT
@@ -218,7 +220,7 @@ The request path and optional query stri
 .It Ic $SERVER_ADDR
 The configured IP address of the server.
 .It Ic $SERVER_PORT
-The configured TCP server port of the server.
+The configured TCP port of the server.
 .It Ic $SERVER_NAME
 The name of the server.
 .It Ic $HTTP_HOST
@@ -396,10 +398,10 @@ the
 using pattern matching instead of shell globbing rules,
 see
 .Xr patterns 7 .
-The pattern may contain captures that can be used in the
-.Ar uri
-of an enclosed
+The pattern may contain captures that can be used in an enclosed
 .Ic block return
+or
+.Ic request rewrite
 option.
 .It Oo Ic no Oc Ic log Op Ar option
 Set the specified logging options.
@@ -458,12 +460,31 @@ instead of the log files.
 Disable any previous
 .Ic block
 in a location.
-.It Ic root Ar option
-Configure the document root and options for the request path.
+.It Ic request Ar option
+Configure the options for the request path.
 Valid options are:
 .Bl -tag -width Ds
-.It Ar directory
-Set the document root of the server.
+.It Oo Ic no Oc Ic rewrite Ar path
+Enable or disable rewriting of the request.
+Unlike the redirection with
+.Ic block return ,
+this will change the request path internally before
+.Nm httpd
+makes a final decision about the matching location.
+The
+.Ar path
+argument may contain predefined macros that will be expanded at runtime.
+See the
+.Ic block return
+option for the list of supported macros.
+.It Ic strip Ar number
+Strip
+.Ar number
+path components from the beginning of the request path before looking
+up the stripped-down path at the document root.
+.El
+.It Ic root Ar directory
+Configure the document root of the server.
 The
 .Ar directory
 is a pathname within the
@@ -472,12 +493,6 @@ root directory of
 .Nm httpd .
 If not specified, it defaults to
 .Pa /htdocs .
-.It Ic strip Ar number
-Strip
-.Ar number
-path components from the beginning of the request path before looking
-up the stripped-down path at the document root.
-.El
 .It Ic tcp Ar option
 Enable or disable the specified TCP/IP options; see
 .Xr tcp 4
@@ -715,6 +730,17 @@ server "example.com" {
 
 server "www.example.com" {
listen on 10.0.0.1 port 80
+}
+.Ed
+The request can also be rewritten with the
+.Ic request rewrite
+directive: 
+.Bd -literal -offset indent
+server "example.com"

httpd request rewrite

[Reyk Floeter]

Hi,

it's about time.

server "default" {
listen on * port 80
location match "/de/(.*)" {
request rewrite "/ch/%1"
}
}

You can also you the macros as in the "block return" external
redirects.  So maybe something like:

server "default" {
listen on * port 80
location match "/(.*)" {
request rewrite "/$HTTP_HOST/%1"
}
}

Tests? OK?

Please note that this diff intentionally breaks the "root strip"
option because it changes the grammar to "request strip".  "root
strip" was semantically wrong but we didn't have a better place to put
it.  An current.html entry can be made for the required grammar change.

Reyk

Index: usr.sbin/httpd/config.c
===
RCS file: /cvs/src/usr.sbin/httpd/config.c,v
retrieving revision 1.54
diff -u -p -u -p -r1.54 config.c
--- usr.sbin/httpd/config.c 19 May 2018 13:56:56 -  1.54
+++ usr.sbin/httpd/config.c 29 May 2018 16:35:29 -
@@ -476,6 +476,13 @@ config_getserver_config(struct httpd *en
&parent->default_type, sizeof(struct media_type));
}
 
+   f = SRVFLAG_PATH_REWRITE|SRVFLAG_NO_PATH_REWRITE;
+   if ((srv_conf->flags & f) == 0) {
+   srv_conf->flags |= parent->flags & f;
+   (void)strlcpy(srv_conf->path, parent->path,
+   sizeof(srv_conf->path));
+   }
+
f = SRVFLAG_SERVER_HSTS;
srv_conf->flags |= parent->flags & f;
srv_conf->hsts_max_age = parent->hsts_max_age;
Index: usr.sbin/httpd/httpd.conf.5
===
RCS file: /cvs/src/usr.sbin/httpd/httpd.conf.5,v
retrieving revision 1.95
diff -u -p -u -p -r1.95 httpd.conf.5
--- usr.sbin/httpd/httpd.conf.5 23 May 2018 19:02:50 -  1.95
+++ usr.sbin/httpd/httpd.conf.5 29 May 2018 16:35:29 -
@@ -198,6 +198,8 @@ argument can be used with return codes i
 .Sq Location:
 header for redirection to a specified URI.
 .Pp
+It is possible to rewrite the request to redirect it to a different
+external location.
 The
 .Ar uri
 may contain predefined macros that will be expanded at runtime:
@@ -396,10 +398,10 @@ the
 using pattern matching instead of shell globbing rules,
 see
 .Xr patterns 7 .
-The pattern may contain captures that can be used in the
-.Ar uri
-of an enclosed
+The pattern may contain captures that can be used in an enclosed
 .Ic block return
+or
+.Ic request rewrite
 option.
 .It Oo Ic no Oc Ic log Op Ar option
 Set the specified logging options.
@@ -458,12 +460,31 @@ instead of the log files.
 Disable any previous
 .Ic block
 in a location.
-.It Ic root Ar option
-Configure the document root and options for the request path.
+.It Ic request Ar option
+Configure the options for the request path.
 Valid options are:
 .Bl -tag -width Ds
-.It Ar directory
-Set the document root of the server.
+.It Oo Ic no Oc Ic rewrite Ar path
+Enable or disable rewriting of the request.
+Unlike the redirection with
+.Ic block return ,
+this will change the request path internally before
+.Nm httpd
+makes a final decision about the matching location.
+The
+.Ar path
+argument may contain predefined macros that will be expanded at runtime.
+See the
+.Ic block return
+option for the list of supported macros.
+.It Ic strip Ar number
+Strip
+.Ar number
+path components from the beginning of the request path before looking
+up the stripped-down path at the document root.
+.El
+.It Ic root Ar directory
+Configure the document root of the server.
 The
 .Ar directory
 is a pathname within the
@@ -472,12 +493,6 @@ root directory of
 .Nm httpd .
 If not specified, it defaults to
 .Pa /htdocs .
-.It Ic strip Ar number
-Strip
-.Ar number
-path components from the beginning of the request path before looking
-up the stripped-down path at the document root.
-.El
 .It Ic tcp Ar option
 Enable or disable the specified TCP/IP options; see
 .Xr tcp 4
@@ -715,6 +730,17 @@ server "example.com" {
 
 server "www.example.com" {
listen on 10.0.0.1 port 80
+}
+.Ed
+The request can also be rewritten with the
+.Ic request rewrite
+directive: 
+.Bd -literal -offset indent
+server "example.com" {
+   listen on * port 80
+   location match "/old/(.*)" {
+   request rewrite "/new/%1"
+   }
 }
 .Ed
 .Sh SEE ALSO
Index: usr.sbin/httpd/httpd.h
===
RCS file: /cvs/src/usr.sbin/httpd/httpd.h,v
retrieving revision 1.137
diff -u -p -u -p -r1.137 httpd.h
--- usr.sbin/httpd/httpd.h  19 May 2018 13:56:56 -  1.137
+++ usr.sbin/httpd/httpd.h  29 May 2018 16:35:30 -
@@ -398,13 +398,15 @@ SPLAY_HEAD(client_tree, client);
 #define SRVFLAG_SERVER_MATCH   0x0020
 #define SRVFLAG_

Re: ldapd: add bsd.schema?

[Reyk Floeter]

On Fri, May 18, 2018 at 10:42:43AM +0200, Reyk Floeter wrote:
> So, OK?
> 

As sthen@ pointed out, I had shadowPassword in both MUST and MAY.
(The reason is that I intended to move it from MUST to MAY, but forgot
to remove the MUST).  userPassword in nis.schema is also MAY, so it
must not be MUST but it must be MAY.  A user also may not have a cn so
remove it from MUST as well.

s/MUST/MAY/ I get an OK?

Index: etc/examples/ldapd.conf
===
RCS file: /cvs/src/etc/examples/ldapd.conf,v
retrieving revision 1.1
diff -u -p -u -p -r1.1 ldapd.conf
--- etc/examples/ldapd.conf 11 Jul 2014 21:20:10 -  1.1
+++ etc/examples/ldapd.conf 18 May 2018 10:09:45 -
@@ -3,6 +3,7 @@
 schema "/etc/ldap/core.schema"
 schema "/etc/ldap/inetorgperson.schema"
 schema "/etc/ldap/nis.schema"
+schema "/etc/ldap/bsd.schema"
 
 listen on lo0
 listen on "/var/run/ldapi"
Index: usr.sbin/ldapd/Makefile
===
RCS file: /cvs/src/usr.sbin/ldapd/Makefile,v
retrieving revision 1.15
diff -u -p -u -p -r1.15 Makefile
--- usr.sbin/ldapd/Makefile 20 Jan 2017 11:55:08 -  1.15
+++ usr.sbin/ldapd/Makefile 18 May 2018 10:09:45 -
@@ -17,7 +17,8 @@ CFLAGS+=  -Wshadow -Wpointer-arith -Wcast
 CFLAGS+=   -Wsign-compare
 CLEANFILES+=   y.tab.h parse.c
 
-SCHEMA_FILES=  core.schema \
+SCHEMA_FILES=  bsd.schema \
+   core.schema \
inetorgperson.schema \
nis.schema
 
Index: usr.sbin/ldapd/schema/bsd.schema
===
RCS file: usr.sbin/ldapd/schema/bsd.schema
diff -N usr.sbin/ldapd/schema/bsd.schema
--- /dev/null   1 Jan 1970 00:00:00 -
+++ usr.sbin/ldapd/schema/bsd.schema18 May 2018 10:09:45 -
@@ -0,0 +1,17 @@
+attributetype ( 1.3.6.1.4.1.30155.115.2 NAME 'shadowPassword'
+   DESC 'POSIX hashed password'
+   EQUALITY caseExactIA5Match
+   SYNTAX 1.3.6.1.4.1.1466.115.121.1.26 )
+
+attributetype ( 1.3.6.1.4.1.30155.115.3 NAME 'sshPublicKey'
+   DESC 'SSH public key'
+   EQUALITY caseExactIA5Match
+   SYNTAX 1.3.6.1.4.1.1466.115.121.1.26 )
+
+objectclass ( 1.3.6.1.4.1.30155.115.1 NAME 'bsdAccount'
+   SUP top
+   AUXILIARY
+   DESC 'Abstraction of an account with OpenBSD attributes'
+   MUST ( uid )
+   MAY ( shadowPassword $ shadowExpire $ modifyTimestamp $ userClass $
+   sshPublicKey ))

Re: ldapd: add bsd.schema?

[Reyk Floeter]

So, OK?

On Tue, May 15, 2018 at 02:24:19PM +0200, Reyk Floeter wrote:
> Hi,
> 
> could we add an LDAP schema file that makes it easier to use sshd's
> "AuthorizedKeysCommand"?
> 
> While most howtos out there agree on the attribute name
> "sshPublicKey", there is no common LDAP schema that implements it.
> Some people patch nis.schema (which seems a bad idea), others add
> their own schema files.
> 
> What about adding our own schema (using OpenBSD's allocated
> 1.3.6.1.4.1.30155 PEN) that includes the required "sshPublicKey"
> attribute?  It can be used to extend existing LDAP users with the
> additional bsdAccount objectClass.
> 
> The "shadowPassword" attribute is useful for ypldap(8) + ldapd(8)
> without login_ldap (for example, userPassword: {BSDAUTH}reyk,
> shadowPassword: $2b$10$...).
> 
> Comments?
> 
> Reyk
> 
> Index: etc/examples/ldapd.conf
> ===
> RCS file: /cvs/src/etc/examples/ldapd.conf,v
> retrieving revision 1.1
> diff -u -p -u -p -r1.1 ldapd.conf
> --- etc/examples/ldapd.conf   11 Jul 2014 21:20:10 -  1.1
> +++ etc/examples/ldapd.conf   15 May 2018 12:09:57 -
> @@ -3,6 +3,7 @@
>  schema "/etc/ldap/core.schema"
>  schema "/etc/ldap/inetorgperson.schema"
>  schema "/etc/ldap/nis.schema"
> +schema "/etc/ldap/bsd.schema"
>  
>  listen on lo0
>  listen on "/var/run/ldapi"
> Index: usr.sbin/ldapd/Makefile
> ===
> RCS file: /cvs/src/usr.sbin/ldapd/Makefile,v
> retrieving revision 1.15
> diff -u -p -u -p -r1.15 Makefile
> --- usr.sbin/ldapd/Makefile   20 Jan 2017 11:55:08 -  1.15
> +++ usr.sbin/ldapd/Makefile   15 May 2018 12:09:57 -
> @@ -17,7 +17,8 @@ CFLAGS+=-Wshadow -Wpointer-arith -Wcast
>  CFLAGS+= -Wsign-compare
>  CLEANFILES+= y.tab.h parse.c
>  
> -SCHEMA_FILES=core.schema \
> +SCHEMA_FILES=bsd.schema \
> + core.schema \
>   inetorgperson.schema \
>   nis.schema
>  
> Index: usr.sbin/ldapd/schema/bsd.schema
> ===
> RCS file: usr.sbin/ldapd/schema/bsd.schema
> diff -N usr.sbin/ldapd/schema/bsd.schema
> --- /dev/null 1 Jan 1970 00:00:00 -
> +++ usr.sbin/ldapd/schema/bsd.schema  15 May 2018 12:09:57 -
> @@ -0,0 +1,17 @@
> +attributetype ( 1.3.6.1.4.1.30155.115.2 NAME 'shadowPassword'
> + DESC 'POSIX hashed password'
> + EQUALITY caseExactIA5Match
> + SYNTAX 1.3.6.1.4.1.1466.115.121.1.26 )
> +
> +attributetype ( 1.3.6.1.4.1.30155.115.3 NAME 'sshPublicKey'
> + DESC 'SSH public key'
> + EQUALITY caseExactIA5Match
> + SYNTAX 1.3.6.1.4.1.1466.115.121.1.26 )
> +
> +objectclass ( 1.3.6.1.4.1.30155.115.1 NAME 'bsdAccount'
> + SUP top
> + AUXILIARY
> + DESC 'Abstraction of an account with OpenBSD attributes'
> + MUST ( cn $ uid $ shadowPassword )
> + MAY ( shadowPassword $ shadowExpire $ modifyTimestamp $ userClass $
> + sshPublicKey ))

-- 

Re: [patch] httpd: add tls client certificate authentication

[Reyk Floeter]

It will! The diff is nice and OK.

> Am 16.05.2018 um 22:33 schrieb Jan Klemkow :
> 
> Hi Jack,
> 
>> On Wed, May 16, 2018 at 05:32:56PM +0930, Jack Burton wrote:
>> I figured that if we can agree on this much, so httpd can be used for
>> the authentication-only case (which is all non-fastcgi sites would want)
>> straight away, that's be a good first step -- then we can come back and
>> argue the toss over how much client cert data is necessary/sufficient
>> to pass through for authorisation/accounting purposes.
> 
> I agree with you.  Lets commit this first step.
> 
> I tested your diff on current amd64 and sparc64 with an own PKI [1] and
> the firefox browser.  Every thing works for me.  I also looked down your
> source code which also seams fine to me.
> 
> I tested the optional and non-optional "client ca" configuration, as
> well as the revocation and the fastcgi environment feature on both
> architectures.  Everything works so far.
> 
> Hopefully, it will committed this time! :-)
> 
> Thanks!
> Jan
> 
> [1]: https://github.com/younix/ca
> 
>> There's also a trivial regression test (unchanged from last year), which
>> I'll post again separately next.
>> 
>> 
>> Index: config.c
>> ===
>> RCS file: /cvs/src/usr.sbin/httpd/config.c,v
>> retrieving revision 1.53
>> diff -u -p -r1.53 config.c
>> --- config.c19 Jul 2017 17:36:25 -1.53
>> +++ config.c16 May 2018 07:59:10 -
>> @@ -304,10 +304,18 @@ config_setserver_tls(struct httpd *env, 
>> 
>>log_debug("%s: configuring tls for %s", __func__, srv_conf->name);
>> 
>> +if (config_settls(env, srv, TLS_CFG_CA, "ca", srv_conf->tls_ca,
>> +srv_conf->tls_ca_len) != 0)
>> +return (-1);
>> +
>>if (config_settls(env, srv, TLS_CFG_CERT, "cert", srv_conf->tls_cert,
>>srv_conf->tls_cert_len) != 0)
>>return (-1);
>> 
>> +if (config_settls(env, srv, TLS_CFG_CRL, "crl", srv_conf->tls_crl,
>> +srv_conf->tls_crl_len) != 0)
>> +return (-1);
>> +
>>if (config_settls(env, srv, TLS_CFG_KEY, "key", srv_conf->tls_key,
>>srv_conf->tls_key_len) != 0)
>>return (-1);
>> @@ -431,6 +439,7 @@ config_getserver_config(struct httpd *en
>> 
>>f = SRVFLAG_TLS;
>>srv_conf->flags |= parent->flags & f;
>> +srv_conf->tls_flags = parent->tls_flags;
>> 
>>f = SRVFLAG_ACCESS_LOG;
>>if ((srv_conf->flags & f) == 0) {
>> @@ -655,9 +664,21 @@ config_getserver_tls(struct httpd *env, 
>>}
>> 
>>switch (tls_conf.tls_type) {
>> +case TLS_CFG_CA:
>> +if (config_gettls(env, srv_conf, &tls_conf, "ca", p, len,
>> +&srv_conf->tls_ca, &srv_conf->tls_ca_len) != 0)
>> +goto fail;
>> +break;
>> +
>>case TLS_CFG_CERT:
>>if (config_gettls(env, srv_conf, &tls_conf, "cert", p, len,
>>&srv_conf->tls_cert, &srv_conf->tls_cert_len) != 0)
>> +goto fail;
>> +break;
>> +
>> +case TLS_CFG_CRL:
>> +if (config_gettls(env, srv_conf, &tls_conf, "crl", p, len,
>> +&srv_conf->tls_crl, &srv_conf->tls_crl_len) != 0)
>>goto fail;
>>break;
>> 
>> Index: httpd.conf.5
>> ===
>> RCS file: /cvs/src/usr.sbin/httpd/httpd.conf.5,v
>> retrieving revision 1.90
>> diff -u -p -r1.90 httpd.conf.5
>> --- httpd.conf.511 Apr 2018 15:50:46 -1.90
>> +++ httpd.conf.516 May 2018 07:59:10 -
>> @@ -342,6 +342,10 @@ The revision of the HTTP specification u
>> .It Ic SERVER_SOFTWARE
>> The server software name of
>> .Xr httpd 8 .
>> +.It Ic TLS_PEER_VERIFY
>> +A variable that is set to a comma separated list of TLS client verification
>> +features in use
>> +.Pq omitted when TLS client verification is not in use .
>> .El
>> .It Ic hsts Oo Ar option Oc
>> Enable HTTP Strict Transport Security.
>> @@ -526,6 +530,23 @@ will be used (strong crypto cipher suite
>> See the CIPHERS section of
>> .Xr openssl 1
>> for information about SSL/TLS cipher suites and preference lists.
>> +.It Ic client ca Ar cafile Oo Ic crl Ar crlfile Oc Op Ic optional
>> +Require
>> +.Po
>> +or, if
>> +.Ic optional
>> +is specified, request but do not require
>> +.Pc
>> +TLS client certificates whose authenticity can be verified
>> +against the CA certificate(s) in
>> +.Ar cafile
>> +in order to proceed beyond the TLS handshake.
>> +With
>> +.Ic crl
>> +specified, additionally require that no certificate in the client chain be
>> +listed as revoked in the CRL(s) in
>> +.Ar crlfile .
>> +CA certificates and CRLs should be PEM encoded.
>> .It Ic dhe Ar params
>> Specify the DHE parameters to use for DHE cipher suites.
>> Valid parameter values are none, legacy and auto.
>> Index: httpd.h
>> ===
>> RCS file: /cvs/src/usr.sbin/httpd/httpd.h,v
>> retrieving revision 1.136
>> diff -u -p -r1.136 httpd.h
>> ---

ldapd: add bsd.schema?

[Reyk Floeter]

Hi,

could we add an LDAP schema file that makes it easier to use sshd's
"AuthorizedKeysCommand"?

While most howtos out there agree on the attribute name
"sshPublicKey", there is no common LDAP schema that implements it.
Some people patch nis.schema (which seems a bad idea), others add
their own schema files.

What about adding our own schema (using OpenBSD's allocated
1.3.6.1.4.1.30155 PEN) that includes the required "sshPublicKey"
attribute?  It can be used to extend existing LDAP users with the
additional bsdAccount objectClass.

The "shadowPassword" attribute is useful for ypldap(8) + ldapd(8)
without login_ldap (for example, userPassword: {BSDAUTH}reyk,
shadowPassword: $2b$10$...).

Comments?

Reyk

Index: etc/examples/ldapd.conf
===
RCS file: /cvs/src/etc/examples/ldapd.conf,v
retrieving revision 1.1
diff -u -p -u -p -r1.1 ldapd.conf
--- etc/examples/ldapd.conf 11 Jul 2014 21:20:10 -  1.1
+++ etc/examples/ldapd.conf 15 May 2018 12:09:57 -
@@ -3,6 +3,7 @@
 schema "/etc/ldap/core.schema"
 schema "/etc/ldap/inetorgperson.schema"
 schema "/etc/ldap/nis.schema"
+schema "/etc/ldap/bsd.schema"
 
 listen on lo0
 listen on "/var/run/ldapi"
Index: usr.sbin/ldapd/Makefile
===
RCS file: /cvs/src/usr.sbin/ldapd/Makefile,v
retrieving revision 1.15
diff -u -p -u -p -r1.15 Makefile
--- usr.sbin/ldapd/Makefile 20 Jan 2017 11:55:08 -  1.15
+++ usr.sbin/ldapd/Makefile 15 May 2018 12:09:57 -
@@ -17,7 +17,8 @@ CFLAGS+=  -Wshadow -Wpointer-arith -Wcast
 CFLAGS+=   -Wsign-compare
 CLEANFILES+=   y.tab.h parse.c
 
-SCHEMA_FILES=  core.schema \
+SCHEMA_FILES=  bsd.schema \
+   core.schema \
inetorgperson.schema \
nis.schema
 
Index: usr.sbin/ldapd/schema/bsd.schema
===
RCS file: usr.sbin/ldapd/schema/bsd.schema
diff -N usr.sbin/ldapd/schema/bsd.schema
--- /dev/null   1 Jan 1970 00:00:00 -
+++ usr.sbin/ldapd/schema/bsd.schema15 May 2018 12:09:57 -
@@ -0,0 +1,17 @@
+attributetype ( 1.3.6.1.4.1.30155.115.2 NAME 'shadowPassword'
+   DESC 'POSIX hashed password'
+   EQUALITY caseExactIA5Match
+   SYNTAX 1.3.6.1.4.1.1466.115.121.1.26 )
+
+attributetype ( 1.3.6.1.4.1.30155.115.3 NAME 'sshPublicKey'
+   DESC 'SSH public key'
+   EQUALITY caseExactIA5Match
+   SYNTAX 1.3.6.1.4.1.1466.115.121.1.26 )
+
+objectclass ( 1.3.6.1.4.1.30155.115.1 NAME 'bsdAccount'
+   SUP top
+   AUXILIARY
+   DESC 'Abstraction of an account with OpenBSD attributes'
+   MUST ( cn $ uid $ shadowPassword )
+   MAY ( shadowPassword $ shadowExpire $ modifyTimestamp $ userClass $
+   sshPublicKey ))

Re: ldapd: fix log and format string errors

[Reyk Floeter]

On Mon, May 14, 2018 at 12:45:18PM +0200, Reyk Floeter wrote:
> Hi,
> 
> the following patch updates ldapd to use log.c from vmd/relayd/etc.
> 
> Notes:
> 
> - This log.c uses format attributes that helped to fix some format
> errors and two actual bugs:
>- There was a missing argument in a log_warn in namespace.c
>- the ldape child never inherited the log level correctly (-vv)
> 
> - This change removes the timestamp from log_debug() - we usually don't
> do that and you can still prepend a timestamp by piping it through an
> additional tool if you're using a foreground process supervisor (such
> as daemontool's multilog TAI).
> 
> - The BER dump and hexdump are only printed with loglevel > 2
> (logmsg.c).  Without this patch, this code was never used as the
> parent only passed a single -v to the exec'ed ldape child.
> 
> OK?
> 

While here, fix an old typo in the namespace.c bit: alreay -> already
(pointed out by Aaron Miller).

OK?  Anyone?

Reyk

Index: usr.sbin/ldapd/conn.c
===
RCS file: /cvs/src/usr.sbin/ldapd/conn.c,v
retrieving revision 1.15
diff -u -p -u -p -r1.15 conn.c
--- usr.sbin/ldapd/conn.c   8 Feb 2018 18:02:06 -   1.15
+++ usr.sbin/ldapd/conn.c   15 May 2018 07:34:04 -
@@ -132,7 +132,7 @@ request_dispatch(struct request *req)
}
 
if (requests[i].fn == NULL) {
-   log_warnx("unhandled request %d (not implemented)", req->type);
+   log_warnx("unhandled request %lu (not implemented)", req->type);
ldap_respond(req, LDAP_PROTOCOL_ERROR);
}
 }
@@ -166,7 +166,7 @@ conn_dispatch(struct conn *conn)
request_free(req);
return -1;
}
-   log_debug("consumed %d bytes", conn->ber.br_rptr - rptr);
+   log_debug("consumed %ld bytes", conn->ber.br_rptr - rptr);
 
/* Read message id and request type.
 */
@@ -183,7 +183,7 @@ conn_dispatch(struct conn *conn)
ldap_debug_elements(req->root, req->type,
"received request on fd %d", conn->fd);
 
-   log_debug("got request type %d, id %lld", req->type, req->msgid);
+   log_debug("got request type %lu, id %lld", req->type, req->msgid);
request_dispatch(req);
return 0;
 }
Index: usr.sbin/ldapd/control.c
===
RCS file: /cvs/src/usr.sbin/ldapd/control.c,v
retrieving revision 1.15
diff -u -p -u -p -r1.15 control.c
--- usr.sbin/ldapd/control.c20 Jan 2017 11:55:08 -  1.15
+++ usr.sbin/ldapd/control.c15 May 2018 07:34:04 -
@@ -261,7 +261,7 @@ control_imsgev(struct imsgev *iev, int c
imsgev_compose(iev_ldapd, IMSG_CTL_LOG_VERBOSE, 0, 0, -1,
&verbose, sizeof(verbose));
 
-   log_verbose(verbose);
+   log_setverbose(verbose);
break;
default:
log_warnx("%s: unexpected imsg %d", __func__, imsg->hdr.type);
Index: usr.sbin/ldapd/filter.c
===
RCS file: /cvs/src/usr.sbin/ldapd/filter.c,v
retrieving revision 1.4
diff -u -p -u -p -r1.4 filter.c
--- usr.sbin/ldapd/filter.c 20 Jan 2017 11:55:08 -  1.4
+++ usr.sbin/ldapd/filter.c 15 May 2018 07:34:04 -
@@ -101,7 +101,7 @@ ldap_filt_subs_value(struct ber_element 
return 1; /* no match */
break;
default:
-   log_warnx("invalid subfilter type %d", type);
+   log_warnx("invalid subfilter type %lu", type);
return -1;
}
}
Index: usr.sbin/ldapd/ldapd.c
===
RCS file: /cvs/src/usr.sbin/ldapd/ldapd.c,v
retrieving revision 1.23
diff -u -p -u -p -r1.23 ldapd.c
--- usr.sbin/ldapd/ldapd.c  1 Mar 2017 00:50:12 -   1.23
+++ usr.sbin/ldapd/ldapd.c  15 May 2018 07:34:05 -
@@ -36,6 +36,7 @@
 #include 
 #include 
 #include 
+#include 
 
 #include "ldapd.h"
 #include "log.h"
@@ -123,7 +124,7 @@ main(int argc, char *argv[])
struct event ev_sighup;
struct stat  sb;
 
-   log_init(1);/* log to stderr until daemonized */
+   log_init(1, LOG_DAEMON);/* log to stderr until daemonized */
 
saved_argv0 = argv[0];
if (saved_argv0 == NULL)
@@ -180,7 +181,7 @@ main(int argc, char *argv[])
if (getpwnam(LDAPD_USER) == NULL)
errx(1, "unknown user %s", LDAPD_USER);
 
-   log_verbose(verbose);
+   log_setverbose(verbose);
stats.starte

ldapd: fix log and format string errors

[Reyk Floeter]

Hi,

the following patch updates ldapd to use log.c from vmd/relayd/etc.

Notes:

- This log.c uses format attributes that helped to fix some format
errors and two actual bugs:
   - There was a missing argument in a log_warn in namespace.c
   - the ldape child never inherited the log level correctly (-vv)

- This change removes the timestamp from log_debug() - we usually don't
do that and you can still prepend a timestamp by piping it through an
additional tool if you're using a foreground process supervisor (such
as daemontool's multilog TAI).

- The BER dump and hexdump are only printed with loglevel > 2
(logmsg.c).  Without this patch, this code was never used as the
parent only passed a single -v to the exec'ed ldape child.

OK?

Reyk

Index: usr.sbin/ldapd/conn.c
===
RCS file: /cvs/src/usr.sbin/ldapd/conn.c,v
retrieving revision 1.15
diff -u -p -u -p -r1.15 conn.c
--- usr.sbin/ldapd/conn.c   8 Feb 2018 18:02:06 -   1.15
+++ usr.sbin/ldapd/conn.c   14 May 2018 10:28:15 -
@@ -132,7 +132,7 @@ request_dispatch(struct request *req)
}
 
if (requests[i].fn == NULL) {
-   log_warnx("unhandled request %d (not implemented)", req->type);
+   log_warnx("unhandled request %lu (not implemented)", req->type);
ldap_respond(req, LDAP_PROTOCOL_ERROR);
}
 }
@@ -166,7 +166,7 @@ conn_dispatch(struct conn *conn)
request_free(req);
return -1;
}
-   log_debug("consumed %d bytes", conn->ber.br_rptr - rptr);
+   log_debug("consumed %ld bytes", conn->ber.br_rptr - rptr);
 
/* Read message id and request type.
 */
@@ -183,7 +183,7 @@ conn_dispatch(struct conn *conn)
ldap_debug_elements(req->root, req->type,
"received request on fd %d", conn->fd);
 
-   log_debug("got request type %d, id %lld", req->type, req->msgid);
+   log_debug("got request type %lu, id %lld", req->type, req->msgid);
request_dispatch(req);
return 0;
 }
Index: usr.sbin/ldapd/control.c
===
RCS file: /cvs/src/usr.sbin/ldapd/control.c,v
retrieving revision 1.15
diff -u -p -u -p -r1.15 control.c
--- usr.sbin/ldapd/control.c20 Jan 2017 11:55:08 -  1.15
+++ usr.sbin/ldapd/control.c14 May 2018 10:28:15 -
@@ -261,7 +261,7 @@ control_imsgev(struct imsgev *iev, int c
imsgev_compose(iev_ldapd, IMSG_CTL_LOG_VERBOSE, 0, 0, -1,
&verbose, sizeof(verbose));
 
-   log_verbose(verbose);
+   log_setverbose(verbose);
break;
default:
log_warnx("%s: unexpected imsg %d", __func__, imsg->hdr.type);
Index: usr.sbin/ldapd/filter.c
===
RCS file: /cvs/src/usr.sbin/ldapd/filter.c,v
retrieving revision 1.4
diff -u -p -u -p -r1.4 filter.c
--- usr.sbin/ldapd/filter.c 20 Jan 2017 11:55:08 -  1.4
+++ usr.sbin/ldapd/filter.c 14 May 2018 10:28:15 -
@@ -101,7 +101,7 @@ ldap_filt_subs_value(struct ber_element 
return 1; /* no match */
break;
default:
-   log_warnx("invalid subfilter type %d", type);
+   log_warnx("invalid subfilter type %lu", type);
return -1;
}
}
Index: usr.sbin/ldapd/ldapd.c
===
RCS file: /cvs/src/usr.sbin/ldapd/ldapd.c,v
retrieving revision 1.23
diff -u -p -u -p -r1.23 ldapd.c
--- usr.sbin/ldapd/ldapd.c  1 Mar 2017 00:50:12 -   1.23
+++ usr.sbin/ldapd/ldapd.c  14 May 2018 10:28:15 -
@@ -36,6 +36,7 @@
 #include 
 #include 
 #include 
+#include 
 
 #include "ldapd.h"
 #include "log.h"
@@ -123,7 +124,7 @@ main(int argc, char *argv[])
struct event ev_sighup;
struct stat  sb;
 
-   log_init(1);/* log to stderr until daemonized */
+   log_init(1, LOG_DAEMON);/* log to stderr until daemonized */
 
saved_argv0 = argv[0];
if (saved_argv0 == NULL)
@@ -180,7 +181,7 @@ main(int argc, char *argv[])
if (getpwnam(LDAPD_USER) == NULL)
errx(1, "unknown user %s", LDAPD_USER);
 
-   log_verbose(verbose);
+   log_setverbose(verbose);
stats.started_at = time(0);
tls_init();
 
@@ -192,6 +193,8 @@ main(int argc, char *argv[])
exit(0);
}
 
+   log_init(debug, LOG_DAEMON);
+
if (eflag)
ldape(debug, verbose, csockpath);
 
@@ -205,7 +208,6 @@ main(int argc, char *argv[])
err(1, "failed to daemonize");
}
 
-   log_init(debug);
log_info("startup");
 
if (socketpair(AF_UNIX, SOCK_STREAM | SOCK_CLOEXEC | SOCK_NONBLOCK,
@@ -

Re: ldapd: filter rules on attributes

[Reyk Floeter]

On Sat, May 12, 2018 at 08:56:48PM +1000, Jonathan Matthew wrote:
> > This only supports "write" (modify, add, delete) and not "read"
> > (search) filter rules.  The search mode will be more complicated and I
> > will look at this later.
> > 
> > Thoughts?  OK?
> 
> ok.  Read filters would be good to have, but this is a nice addition as it is.
> 

OK, here is one attempt at implementing attribute read filters:
it simply filters out forbidden attributes from the search result.

1. But what should be the desired logic for read attribute filter rules?
a) filter out each attribute that is not readable and continue
b) fail with error 50 (insuf. access) if a child attribute is not readable

2. Should read access to the parent dn be required?
For example:
deny read access to any by self
allow read access to any attribute helloWorld by self
Should this
a) deny access to the helloWorld attribute or
b) allow access to the helloWorld attribute (but no other attribute)

The diff below implements the easiest option 1a) + 2a).

Thoughts?

Reyk

Index: usr.sbin/ldapd/ldapd.conf.5
===
RCS file: /cvs/src/usr.sbin/ldapd/ldapd.conf.5,v
retrieving revision 1.23
diff -u -p -u -p -r1.23 ldapd.conf.5
--- usr.sbin/ldapd/ldapd.conf.5 14 May 2018 07:53:47 -  1.23
+++ usr.sbin/ldapd/ldapd.conf.5 14 May 2018 08:58:40 -
@@ -252,7 +252,6 @@ The scope scope can be restricted to an 
 .Bl -tag -width Ds
 .It attribute Ar name
 The filter rule applies to the specified attribute.
-Attributes can only be specified for write access rules.
 .El
 .Pp
 Finally, the filter rule can match a bind DN:
Index: usr.sbin/ldapd/parse.y
===
RCS file: /cvs/src/usr.sbin/ldapd/parse.y,v
retrieving revision 1.27
diff -u -p -u -p -r1.27 parse.y
--- usr.sbin/ldapd/parse.y  14 May 2018 07:53:47 -  1.27
+++ usr.sbin/ldapd/parse.y  14 May 2018 08:58:40 -
@@ -1164,12 +1164,6 @@ mk_aci(int type, int rights, enum scope 
aci->scope,
aci->subject ? aci->subject : "any");
 
-   if (aci->attribute && aci->rights != ACI_WRITE) {
-   yyerror("attributes only supported for write access filters");
-   free(aci);
-   return NULL;
-   }
-
return aci;
 }
 
Index: usr.sbin/ldapd/search.c
===
RCS file: /cvs/src/usr.sbin/ldapd/search.c,v
retrieving revision 1.19
diff -u -p -u -p -r1.19 search.c
--- usr.sbin/ldapd/search.c 14 May 2018 07:53:47 -  1.19
+++ usr.sbin/ldapd/search.c 14 May 2018 08:58:40 -
@@ -102,7 +102,7 @@ search_result(const char *dn, size_t dnl
struct ber_element  *root, *elm, *filtered_attrs = NULL, *link, *a;
struct ber_element  *prev, *next;
char*adesc;
-   void*buf;
+   void*buf, *searchdn = NULL;
 
if ((root = ber_add_sequence(NULL)) == NULL)
goto fail;
@@ -111,10 +111,20 @@ search_result(const char *dn, size_t dnl
goto fail;
link = filtered_attrs;
 
+   if ((searchdn = strndup(dn, dnlen)) == NULL)
+   goto fail;
+
for (prev = NULL, a = attrs->be_sub; a; a = next) {
if (ber_get_string(a->be_sub, &adesc) != 0)
goto fail;
-   if (should_include_attribute(adesc, search, 0)) {
+   /*
+* Check if read access to the attribute is allowed and if it
+* should be included in the search result.  The attribute is
+* filtered out in the result if one of these conditions fails.
+*/
+   if (authorized(search->conn, search->ns, ACI_READ,
+   searchdn, adesc, LDAP_SCOPE_BASE) &&
+   should_include_attribute(adesc, search, 0)) {
next = a->be_next;
if (prev != NULL)
prev->be_next = a->be_next; /* unlink a */
@@ -152,11 +162,13 @@ search_result(const char *dn, size_t dnl
return -1;
}
 
+   free(searchdn);
return 0;
 fail:
log_warn("search result");
if (root)
ber_free_elements(root);
+   free(searchdn);
return -1;
 }
 

ldapd: filter rules on attributes

[Reyk Floeter]

Hi!

(resent to tech@)
the following ldapd patch allows filter rules to match on attributes.

This can be used to allow users to change their password (and a few
other things) but not their entire dn.

For example, in ldapd.conf:

allow read access to any by self
allow write access to any attribute shadowPassword by self
allow write access to any attribute sshPublicKey by self
allow write access to any attribute gecos by self
allow write access to any attribute loginShell by self

Alternatively, this would also work:

allow write access to any by self
deny write access to any attribute mail by self

This only supports "write" (modify, add, delete) and not "read"
(search) filter rules.  The search mode will be more complicated and I
will look at this later.

Thoughts?  OK?

Reyk

Index: usr.sbin/ldapd/auth.c
===
RCS file: /cvs/src/usr.sbin/ldapd/auth.c,v
retrieving revision 1.12
diff -u -p -u -p -r1.12 auth.c
--- usr.sbin/ldapd/auth.c   20 Jan 2017 11:55:08 -  1.12
+++ usr.sbin/ldapd/auth.c   11 May 2018 14:09:01 -
@@ -33,7 +33,7 @@
 
 static int
 aci_matches(struct aci *aci, struct conn *conn, struct namespace *ns,
-char *dn, int rights, enum scope scope)
+char *dn, int rights, char *attr, enum scope scope)
 {
struct btval key;
 
@@ -98,6 +98,13 @@ aci_matches(struct aci *aci, struct conn
return 0;
}
 
+   if (aci->attribute != NULL) {
+   if (attr == NULL)
+   return 0;
+   if (strcasecmp(aci->attribute, attr) != 0)
+   return 0;
+   }
+
return 1;
 }
 
@@ -105,7 +112,7 @@ aci_matches(struct aci *aci, struct conn
  */
 int
 authorized(struct conn *conn, struct namespace *ns, int rights, char *dn,
-int scope)
+char *attr, int scope)
 {
struct aci  *aci;
int  type = ACI_ALLOW;
@@ -124,33 +131,41 @@ authorized(struct conn *conn, struct nam
if ((rights & (ACI_WRITE | ACI_CREATE)) != 0)
type = ACI_DENY;
 
-   log_debug("requesting %02X access to %s by %s, in namespace %s",
+   log_debug("requesting %02X access to %s%s%s by %s, in namespace %s",
rights,
dn ? dn : "any",
+   attr ? " attribute " : "",
+   attr ? attr : "",
conn->binddn ? conn->binddn : "any",
ns ? ns->suffix : "global");
 
SIMPLEQ_FOREACH(aci, &conf->acl, entry) {
-   if (aci_matches(aci, conn, ns, dn, rights, scope)) {
+   if (aci_matches(aci, conn, ns, dn, rights,
+   attr, scope)) {
type = aci->type;
-   log_debug("%s by: %s %02X access to %s by %s",
+   log_debug("%s by: %s %02X access to %s%s%s by %s",
type == ACI_ALLOW ? "allowed" : "denied",
aci->type == ACI_ALLOW ? "allow" : "deny",
aci->rights,
aci->target ? aci->target : "any",
+   aci->attribute ? " attribute " : "",
+   aci->attribute ? aci->attribute : "",
aci->subject ? aci->subject : "any");
}
}
 
if (ns != NULL) {
SIMPLEQ_FOREACH(aci, &ns->acl, entry) {
-   if (aci_matches(aci, conn, ns, dn, rights, scope)) {
+   if (aci_matches(aci, conn, ns, dn, rights,
+   attr, scope)) {
type = aci->type;
-   log_debug("%s by: %s %02X access to %s by %s",
+   log_debug("%s by: %s %02X access to %s%s%s by 
%s",
type == ACI_ALLOW ? "allowed" : "denied",
aci->type == ACI_ALLOW ? "allow" : "deny",
aci->rights,
aci->target ? aci->target : "any",
+   aci->attribute ? " attribute " : "",
+   aci->attribute ? aci->attribute : "",
aci->subject ? aci->subject : "any");
}
}
@@ -319,7 +334,7 @@ ldap_auth_simple(struct request *req, ch
return LDAP_INVALID_CREDENTIALS;
} else {
if (!authorized(req->conn, ns, ACI_BIND, binddn,
-   LDAP_SCOPE_BASE))
+   NULL, LDAP_SCOPE_BASE))
return LDAP_INSUFFICIENT_ACCESS;
 
elm = namespace_get(ns, binddn);
Index: usr.sbin/ldapd/ldapd.conf.5
===
RCS file: /cvs/src/usr.sbin/ldapd/ldapd.conf.5,v
retrieving revision 1.22
diff -u -p -u

Re: relayctl friendlier

[Reyk Floeter]

eOn Fri, May 11, 2018 at 01:53:58PM +0300, Kapetanakis Giannis wrote:
> Hi,
> 
> By default we have:
> 
> # relayctl show
> missing argument:
> valid commands/args:
>   summary
>   hosts
>   redirects
>   relays
>   routers
>   sessions
> 
> On the other hand:
> # relayctl host
> usage: relayctl [-s socket] command [argument ...]
> 
> # relayctl host dis
> missing argument:
> valid commands/args:
>   
> 
> I think it's better if it is like:
> 
> # ./relayctl host
> missing argument:
> valid commands/args:
>   disable
>   enable
> 
> same for table, redirect
> 

Thanks, the diff looks fine

> If this is accepted maybe NOTOKEN can be completely removed from code.
> 

It is just a value and it is better to keep it - the parser code is
somewhat generic and used by many tools in OpenBSD.

Reyk

> regards,
> 
> G
> 
> Index: parser.c
> ===
> RCS file: /cvs/src/usr.sbin/relayctl/parser.c,v
> retrieving revision 1.27
> diff -u -p -r1.27 parser.c
> --- parser.c22 Jan 2015 17:42:09 -  1.27
> +++ parser.c11 May 2018 10:52:11 -
> @@ -81,21 +81,18 @@ static const struct token t_show[] = {
>  };
>  
>  static const struct token t_rdr[] = {
> -   {NOTOKEN,   "", NONE,   NULL},
> {KEYWORD,   "disable",  RDR_DISABLE,t_rdr_id},
> {KEYWORD,   "enable",   RDR_ENABLE, t_rdr_id},
> {ENDTOKEN,  "", NONE,   NULL}
>  };
>  
>  static const struct token t_table[] = {
> -   {NOTOKEN,   "", NONE,   NULL},
> {KEYWORD,   "disable",  TABLE_DISABLE,  t_table_id},
> {KEYWORD,   "enable",   TABLE_ENABLE,   t_table_id},
> {ENDTOKEN,  "", NONE,   NULL}
>  };
>  
>  static const struct token t_host[] = {
> -   {NOTOKEN,   "", NONE,   NULL},
> {KEYWORD,   "disable",  HOST_DISABLE,   t_host_id},
> {KEYWORD,   "enable",   HOST_ENABLE,t_host_id},
> {ENDTOKEN,  "", NONE,   NULL}
> 

--