Re: Move ancient ksh bug from README to man page

[Theo Buehler]

On Sun, Dec 06, 2015 at 12:00:36AM -0500, Michael McConville wrote:
> Thoughts? ok?
> 

It makes sense to me to document this in the manual, and not in a README
file that no-one reads, so ok from me, but please wait for jmc@'s input.

Note that this bug used to be documented in the manual, but it was first
commented out by jmc in -r1.91 (2005) and then deleted by you in -r1.167
this year.

Moreover, shouldn't this bug be mentioned in sh.1 as well?

> 
> Index: README
> ===
> RCS file: /cvs/src/bin/ksh/README,v
> retrieving revision 1.15
> diff -u -p -r1.15 README
> --- README5 Dec 2015 19:40:45 -   1.15
> +++ README6 Dec 2015 04:58:27 -
> @@ -15,8 +15,3 @@ Files of interest:
>   NOTES   lists of known bugs in pdksh, at ksh, and posix.
>   PROJECTSlist of things that need to be done in pdksh.
>   LEGAL   A file detailing legal issues concerning pdksh.
> -
> -
> -BTW, THE MOST FREQUENTLY REPORTED BUG IS
> - echo hi | read a; echo $a   # Does not print hi
> -I'm aware of this and there is no need to report it.
> Index: ksh.1
> ===
> RCS file: /cvs/src/bin/ksh/ksh.1,v
> retrieving revision 1.171
> diff -u -p -r1.171 ksh.1
> --- ksh.1 24 Nov 2015 21:07:31 -  1.171
> +++ ksh.1 6 Dec 2015 04:58:29 -
> @@ -5628,3 +5628,9 @@ The
>  .Pa CONTRIBUTORS
>  file in the source distribution contains a more complete list of people and
>  their part in the shell's development.
> +.Sh BUGS
> +Since time immemorial, the below command has not printed
> +.Qq hi ,
> +as would be expected:
> +.Pp
> +.Dl $ echo hi | read a; echo $a
> 

Re: libgtop2: Use getifaddrs(3) instead of KVM

[Jasper Lievisse Adriaanse]

On Sat, Dec 05, 2015 at 05:05:07PM +0100, Martin Pieuchot wrote:
> Here's a rewrite of glibtop_get_netload_p().  I tested it with custom
> code because I could not trigger this code path with our ports.
> 
> This unbreaks devel/libgtop2 after the recent  commit.
> 
> I believe this should go upstream, how should I submit this?
I can merge it directly upstream (with proper git authorship set). Note that
our libgtop could use an update but as of yet I didn't get around to
fixing the build breakage yet. When that's been done I'll merge your patch
upstream, alright?

> ok?
OK with me.
 
> Index: Makefile
> ===
> RCS file: /cvs/ports/devel/libgtop2/Makefile,v
> retrieving revision 1.130
> diff -u -p -r1.130 Makefile
> --- Makefile  22 May 2015 11:31:13 -  1.130
> +++ Makefile  5 Dec 2015 15:58:01 -
> @@ -6,7 +6,7 @@ GNOME_VERSION=2.30.0
>  GNOME_PROJECT=   libgtop
>  PKGNAME= libgtop2-${VERSION}
>  
> -REVISION=3
> +REVISION=4
>  
>  SHARED_LIBS= gtop-2.09.0 # .10.0
>  
> Index: patches/patch-sysdeps_openbsd_netload_c
> ===
> RCS file: patches/patch-sysdeps_openbsd_netload_c
> diff -N patches/patch-sysdeps_openbsd_netload_c
> --- /dev/null 1 Jan 1970 00:00:00 -
> +++ patches/patch-sysdeps_openbsd_netload_c   5 Dec 2015 15:59:31 -
> @@ -0,0 +1,345 @@
> +$OpenBSD$
> +
> +Rewrite of glibtop_get_netload_p() to use getifaddrs(3) instead of KVM.
> +
> +--- sysdeps/openbsd/netload.c.orig   Mon Apr 28 23:09:24 2014
>  sysdeps/openbsd/netload.cSat Dec  5 16:27:56 2015
> +@@ -1,48 +1,39 @@
> +-/* Copyright (C) 1998-99 Martin Baulig
> +-   This file is part of LibGTop 1.0.
> ++/*
> ++ * Copyright (c) 2015 Martin Pieuchot 
> ++ *
> ++ * Permission to use, copy, modify, and distribute this software for any
> ++ * purpose with or without fee is hereby granted, provided that the above
> ++ * copyright notice and this permission notice appear in all copies.
> ++ *
> ++ * THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES
> ++ * WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF
> ++ * MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR
> ++ * ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES
> ++ * WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN
> ++ * ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF
> ++ * OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
> ++ */
> + 
> +-   Contributed by Martin Baulig , October 1998.
> ++#include "config.h"
> + 
> +-   LibGTop is free software; you can redistribute it and/or modify it
> +-   under the terms of the GNU General Public License as published by
> +-   the Free Software Foundation; either version 2 of the License,
> +-   or (at your option) any later version.
> ++#include 
> ++#include 
> ++#include 
> + 
> +-   LibGTop is distributed in the hope that it will be useful, but WITHOUT
> +-   ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
> +-   FITNESS FOR A PARTICULAR PURPOSE.  See the GNU General Public License
> +-   for more details.
> +-
> +-   You should have received a copy of the GNU General Public License
> +-   along with LibGTop; see the file COPYING. If not, write to the
> +-   Free Software Foundation, Inc., 59 Temple Place - Suite 330,
> +-   Boston, MA 02111-1307, USA.
> +-*/
> +-
> +-#include 
> +-#include 
> +-#include 
> +-#include 
> +-
> +-#include 
> +-
> +-#include 
> +-
> + #include 
> + #include 
> +-#include 
> + 
> +-#include 
> +-
> +-#include 
> +-
> + #include 
> +-#define _KERNEL
> + #include 
> +-#undef _KERNEL
> + #include 
> + 
> ++#include 
> ++
> ++#include "glibtop.h"
> ++#include "glibtop/netload.h"
> ++
> + static const unsigned long _glibtop_sysdeps_netload =
> ++(1L << GLIBTOP_NETLOAD_MTU) +
> + (1L << GLIBTOP_NETLOAD_IF_FLAGS) +
> + (1L << GLIBTOP_NETLOAD_PACKETS_IN) +
> + (1L << GLIBTOP_NETLOAD_PACKETS_OUT) +
> +@@ -55,183 +46,115 @@ static const unsigned long _glibtop_sysdeps_netload =
> + (1L << GLIBTOP_NETLOAD_ERRORS_TOTAL) +
> + (1L << GLIBTOP_NETLOAD_COLLISIONS);
> + 
> +-static const unsigned _glibtop_sysdeps_netload_data =
> +-(1L << GLIBTOP_NETLOAD_ADDRESS) +
> +-(1L << GLIBTOP_NETLOAD_SUBNET) +
> +-(1L << GLIBTOP_NETLOAD_MTU);
> +-
> +-/* nlist structure for kernel access */
> +-static struct nlist nlst [] = {
> +-{ "_ifnet" },
> +-{ 0 }
> +-};
> +-
> +-/* Init function. */
> +-
> + void
> + _glibtop_init_netload_p (glibtop *server)
> + {
> + server->sysdeps.netload = _glibtop_sysdeps_netload;
> +-
> +-if (kvm_nlist (server->machine.kd, nlst) < 0)
> +-glibtop_error_io_r (server, "kvm_nlist");
> + }
> + 
> +-/* Provides Network statistics. */
> +-
> + void
> + 

Re: [patch v3] cwm: Preserve stacking order during cycling

[Vadim Vygonets]

Even when cycling in group, all visible windows should be
restacked.  Patch version 3.

Vadik.

Quoth Vadim Vygonets on Sun, Nov 22, 2015:
> I accidentally killed restacking on group_show().  Sorry about
> that.  Here's version 2 of the patch.
> 
> Quoth Vadim Vygonets on Sat, Nov 21, 2015:
> > After cycling through many windows, the original window may be
> > obscured by many others, and if you still want to see its
> > contents you end up doing the Alt-Tab-Tab-Tab-Tab-Tab, Alt-Tab,
> > Alt-Tab dance.
> > 
> > This patch adds restacking of windows during cycling.  Hold Alt,
> > press Tab and a window will be raised.  Press Tab again while
> > still holding Alt and that window will be lowered back before
> > another is raised.  Once you release Alt, the original window
> > will be hidden behind no more than one other (the target),
> > assuming it was raised before.

-- 
Of course [nobody reads the docs that come with the OS] -- that
would be too easy and too quick.  People know that the Unix Way
is difficult and they prefer to keep it that way.
-- Greg Black
Index: calmwm.h
===
RCS file: /cvs/xenocara/app/cwm/calmwm.h,v
retrieving revision 1.311
diff -u -r1.311 calmwm.h
--- calmwm.h	12 Nov 2015 21:28:03 -	1.311
+++ calmwm.h	22 Nov 2015 17:28:04 -
@@ -62,6 +62,9 @@
 #define CWM_CLIENT_RCYCLE	0x0002
 #define CWM_CLIENT_CYCLE_INGRP	0x0004
 
+#define CWM_CLIENT_RESTACK_GROUP	0x0001
+#define CWM_CLIENT_RESTACK_HIDDEN	0x0002
+
 #define CWM_CLIENT_TILE_HORIZ	0x0001
 #define CWM_CLIENT_TILE_VERT	0x0002
 
@@ -385,6 +388,7 @@
 void			 client_applysizehints(struct client_ctx *);
 void			 client_config(struct client_ctx *);
 struct client_ctx	*client_current(void);
+void			 client_restack(struct client_ctx_q *, int);
 void			 client_cycle(struct screen_ctx *, int);
 void			 client_cycle_leave(struct screen_ctx *);
 void			 client_delete(struct client_ctx *);
Index: client.c
===
RCS file: /cvs/xenocara/app/cwm/client.c,v
retrieving revision 1.214
diff -u -r1.214 client.c
--- client.c	12 Nov 2015 18:33:30 -	1.214
+++ client.c	22 Nov 2015 17:28:04 -
@@ -664,6 +664,54 @@
 }
 
 void
+client_restack(struct client_ctx_q *clientq, int flags)
+{
+#define CLIENTQ_FOREACH(var, head, ingrp)\
+	for((var) = TAILQ_FIRST(head);	\
+	(var) != TAILQ_END(head);	\
+	(var) = (ingrp) ?		\
+		TAILQ_NEXT(var, group_entry) : TAILQ_NEXT(var, entry))
+	struct client_ctx	*cc;
+	Window			*winlist;
+	int			 i, lastempty = -1;
+	int			 nwins = 0, highstack = 0;
+
+	CLIENTQ_FOREACH(cc, clientq, flags & CWM_CLIENT_RESTACK_GROUP) {
+		if (!(flags & CWM_CLIENT_RESTACK_HIDDEN) &&
+		(cc->flags & CLIENT_HIDDEN)) {
+			continue;
+		}
+		if (cc->stackingorder > highstack)
+			highstack = cc->stackingorder;
+	}
+	winlist = xreallocarray(NULL, (highstack + 1), sizeof(*winlist));
+
+	/* Invert the stacking order for XRestackWindows(). */
+	CLIENTQ_FOREACH(cc, clientq, flags & CWM_CLIENT_RESTACK_GROUP) {
+		if (!(flags & CWM_CLIENT_RESTACK_HIDDEN) &&
+		(cc->flags & CLIENT_HIDDEN)) {
+			continue;
+		}
+		winlist[highstack - cc->stackingorder] = cc->win;
+		nwins++;
+	}
+
+	/* Un-sparseify */
+	for (i = 0; i <= highstack; i++) {
+		if (!winlist[i] && lastempty == -1)
+			lastempty = i;
+		else if (winlist[i] && lastempty != -1) {
+			winlist[lastempty] = winlist[i];
+			if (++lastempty == i)
+lastempty = -1;
+		}
+	}
+
+	XRestackWindows(X_Dpy, winlist, nwins);
+	free(winlist);
+}
+
+void
 client_cycle(struct screen_ctx *sc, int flags)
 {
 	struct client_ctx	*newcc, *oldcc;
@@ -704,9 +752,14 @@
 		}
 	}
 
-	/* reset when cycling mod is released. XXX I hate this hack */
-	sc->cycling = 1;
 	client_ptrsave(oldcc);
+	if (!sc->cycling) {
+		/* reset when cycling mod is released. XXX I hate this hack */
+		sc->cycling = 1;
+		screen_updatestackingorder(sc);
+	} else {
+		client_restack(>clientq, 0);
+	}
 	client_ptrwarp(newcc);
 }
 
Index: group.c
===
RCS file: /cvs/xenocara/app/cwm/group.c,v
retrieving revision 1.121
diff -u -r1.121 group.c
--- group.c	10 Nov 2015 20:05:33 -	1.121
+++ group.c	22 Nov 2015 17:28:04 -
@@ -34,7 +34,6 @@
 
 static struct group_ctx	*group_next(struct group_ctx *);
 static struct group_ctx	*group_prev(struct group_ctx *);
-static void		 group_restack(struct group_ctx *);
 static void		 group_setactive(struct group_ctx *);
 
 const char *num_to_name[] = {
@@ -82,43 +81,9 @@
 			client_unhide(cc);
 	}
 
-	group_restack(gc);
+	client_restack(>clientq,
+	CWM_CLIENT_RESTACK_GROUP | CWM_CLIENT_RESTACK_HIDDEN);
 	group_setactive(gc);
-}
-
-static void
-group_restack(struct group_ctx *gc)
-{
-	struct client_ctx	*cc;
-	Window			*winlist;
-	int			 i, lastempty = -1;
-	int			 nwins = 0, highstack = 0;
-
-	TAILQ_FOREACH(cc, >clientq, group_entry) {
-		if (cc->stackingorder > 

[patch] cwm: Simplify client_toggle_*()

[Vadim Vygonets]

Use XOR for toggling bits.  Mostly cosmetic.

Vadik.

-- 
When in doubt, be yourself.  And if that fails, su root.
Index: client.c
===
RCS file: /cvs/xenocara/app/cwm/client.c,v
retrieving revision 1.214
diff -u -r1.214 client.c
--- client.c	12 Nov 2015 18:33:30 -	1.214
+++ client.c	6 Dec 2015 14:24:15 -
@@ -256,33 +256,21 @@
 	if (cc->flags & CLIENT_FULLSCREEN)
 		return;
 
-	if (cc->flags & CLIENT_FREEZE)
-		cc->flags &= ~CLIENT_FREEZE;
-	else
-		cc->flags |= CLIENT_FREEZE;
-
+	cc->flags ^= CLIENT_FREEZE;
 	xu_ewmh_set_net_wm_state(cc);
 }
 
 void
 client_toggle_hidden(struct client_ctx *cc)
 {
-	if (cc->flags & CLIENT_HIDDEN)
-		cc->flags &= ~CLIENT_HIDDEN;
-	else
-		cc->flags |= CLIENT_HIDDEN;
-
+	cc->flags ^= CLIENT_HIDDEN;
 	xu_ewmh_set_net_wm_state(cc);
 }
 
 void
 client_toggle_sticky(struct client_ctx *cc)
 {
-	if (cc->flags & CLIENT_STICKY)
-		cc->flags &= ~CLIENT_STICKY;
-	else
-		cc->flags |= CLIENT_STICKY;
-
+	cc->flags ^= CLIENT_STICKY;
 	xu_ewmh_set_net_wm_state(cc);
 }
 

[PATCH] vmm(4) manpage missed the vmctl rename

[trondd]

Update manpage reference from vmmctl(8) to vmctl(8)

Tim.


Index: vmm.4
===
RCS file: /cvs/src/share/man/man4/man4.amd64/vmm.4,v
retrieving revision 1.3
diff -u -p -r1.3 vmm.4
--- vmm.4   13 Nov 2015 07:55:37 -  1.3
+++ vmm.4   6 Dec 2015 16:27:41 -
@@ -72,7 +72,7 @@ For more information, consult the CPU ve
 .Xr intro 4 ,
 .Xr virtio 4 ,
 .Xr vmd 8 ,
-.Xr vmmctl 8
+.Xr vmctl 8
 .Sh HISTORY
 The
 .Nm

Re: ugenctl for attaching USB devices to ugen instead of their specific driver

[Stuart Henderson]

On 2015/12/06 06:02, Mickael Torres wrote:
> Hello,
> 
> This is a kernel patch plus a utility called ugenctl I use to allow
> selected USB devices to attach as ugen(4) instead of their more specific
> driver. My use case is a Microchip "PICkit 2 Microcontroller Programmer"
> that attaches as uhid(4), but the command line utility pk2cmd wants a
> udev(4) device. There maybe are some other use cases.

If a device is generally pointless with uhid, we can just knock it
out completely in the kernel, see the UQ_BAD_HID mechanism. I think this
applies to your device.

The bigger problem is "dual use" devices; e.g. some want UPS to attach
to upd(4), others want ugen(4) for use with NUT/apcupsd. Your code is
partially useful for these, but because it just changes things at attach,
won't survive a reboot - if it could instead force a device to detach
and reattach to ugen it would help a lot more cases.

> It is mostly quick and dirty, but does the job.
> 
> It can be found there: https://github.com/mickaeltorres/openbsd_ugenctl or
> inlined at the end of this mail.
> 
> Cheers,
> Mike.
> 
> Here is the kernel patch:
> 
> Index: usb.c
> ===
> RCS file: /cvs/src/sys/dev/usb/usb.c,v
> retrieving revision 1.107
> diff -u -p -r1.107 usb.c
> --- usb.c   14 Mar 2015 03:38:50 -  1.107
> +++ usb.c   6 Dec 2015 04:29:22 -
> @@ -137,6 +137,8 @@ const struct cfattach usb_ca = {
> usb_activate,
>  };
> 
> +struct usb_device_ugen_force usb_udf;
> +
>  int
>  usb_match(struct device *parent, void *match, void *aux)
>  {
> @@ -156,6 +158,7 @@ usb_attach(struct device *parent, struct
> TAILQ_INIT(_generic_tasks);
> usb_run_tasks = usb_run_abort_tasks = 1;
> kthread_create_deferred(usb_create_task_threads, NULL);
> +   bzero(_udf, sizeof(usb_udf));
> }
> usb_nbuses++;
> 
> @@ -792,6 +795,23 @@ usbioctl(dev_t devt, u_long cmd, caddr_t
> return (error);
> }
> 
> +   case USB_DEVICE_FORCE_UGEN:
> +   {
> +   struct usb_device_ugen_force *udf =
> +   (struct usb_device_ugen_force *)data;
> +
> +   if (udf->udf_set) // replace our list
> +   {
> +   memcpy(_udf, udf, sizeof(usb_udf));
> +   return (0);
> +   }
> +   else  // return the list
> +   {
> +   memcpy(udf, _udf, sizeof(usb_udf));
> +   return (0);
> +   }
> +   }
> +
> default:
> return (EINVAL);
> }
> Index: usb.h
> ===
> RCS file: /cvs/src/sys/dev/usb/usb.h,v
> retrieving revision 1.53
> diff -u -p -r1.53 usb.h
> --- usb.h   9 Jul 2015 05:40:44 -   1.53
> +++ usb.h   6 Dec 2015 04:29:23 -
> @@ -749,6 +749,13 @@ struct usb_device_stats {
> u_long  uds_requests[4];/* indexed by transfer type UE_* */
>  };
> 
> +#define USB_DEVICE_UGEN_FORCE_MAX 16
> +struct usb_device_ugen_force {
> +int udf_set;
> +u_short udf_ven_id[USB_DEVICE_UGEN_FORCE_MAX];
> +u_short udf_dev_id[USB_DEVICE_UGEN_FORCE_MAX];
> +};
> +
>  /* USB controller */
>  #define USB_REQUEST_IOWR('U', 1, struct usb_ctl_request)
>  #define USB_SETDEBUG   _IOW ('U', 2, unsigned int)
> @@ -758,6 +765,7 @@ struct usb_device_stats {
>  #define USB_DEVICE_GET_CDESC   _IOWR('U', 6, struct usb_device_cdesc)
>  #define USB_DEVICE_GET_FDESC   _IOWR('U', 7, struct usb_device_fdesc)
>  #define USB_DEVICE_GET_DDESC   _IOWR('U', 8, struct usb_device_ddesc)
> +#define USB_DEVICE_FORCE_UGEN   _IOWR('U', 9, struct
> usb_device_ugen_force)
> 
>  /* Generic HID device */
>  #define USB_GET_REPORT_DESC_IOR ('U', 21, struct usb_ctl_report_desc)
> Index: usb_subr.c
> ===
> RCS file: /cvs/src/sys/dev/usb/usb_subr.c,v
> retrieving revision 1.117
> diff -u -p -r1.117 usb_subr.c
> --- usb_subr.c  23 Mar 2015 22:26:01 -  1.117
> +++ usb_subr.c  6 Dec 2015 04:29:23 -
> @@ -840,6 +840,7 @@ usbd_probe_and_attach(struct device *par
> struct device *dv;
> struct usbd_interface **ifaces;
> extern struct rwlock usbpalock;
> +   extern struct usb_device_ugen_force usb_udf;
> 
> rw_enter_write();
> 
> @@ -855,6 +856,14 @@ usbd_probe_and_attach(struct device *par
> uaa.product = UGETW(dd->idProduct);
> uaa.release = UGETW(dd->bcdDevice);
> 
> +   /* Check if this device is in the ugen force list */
> +   for (i = 0; i < USB_DEVICE_UGEN_FORCE_MAX; i++)
> +   if ((usb_udf.udf_ven_id[i] == uaa.vendor) &&
> +   (usb_udf.udf_dev_id[i] == uaa.product))
> +   break;
> +   if (i != USB_DEVICE_UGEN_FORCE_MAX)

Re: Move ancient ksh bug from README to man page

[Michael McConville]

Jérémie Courrèges-Anglas wrote:
> Jason McIntyre  writes:
> 
> > On Sun, Dec 06, 2015 at 04:03:16AM -0500, Ted Unangst wrote:
> >> Jason McIntyre wrote:
> >> > the trouble is i think there are some known bugs with ksh. i
> >> > think it probably would be better to keep a note of them in a
> >> > separate file, as is done now. i'm not really sure if we want to
> >> > try and clutter the page with every bug we find.
> >> 
> >> this particular bug seemed likely to hurt people. i think it will
> >> save people time to document it. 
> >
> > yes, fair enough.
> 
> It's not a bug so I'd rather add it to CAVEATS.

Why not? I don't know much shell trivia.

> Also "Since time immemorial" looks superfluous.

Maybe a cute way of phrasing it, but I though it worthwhile to briefly
mention that this has been around for >20 years. It suggests that it's
well-known and not easy to fix (or, as you say, not a bug at all).

Re: taskctx and revisiting if_start serialisation

[Hrvoje Popovski]

On 6.12.2015. 5:00, David Gwynne wrote:
> the current code for serialising if_start calls for mpsafe nics does what it 
> says.
> 
> however, kettenis realised it doesnt help us much when we're trying
> to coordinate between the start and txeof side of a driver when
> setting or clearing oactive. in particular, a start routine can
> figure out there's no more space, and then set oactive. txeof could
> be running on another cpu emptying the ring and clearing it. if
> that clear runs in between the other cpus space check and
> ifq_set_oactive, then the nic will be marked full and the stack
> wont ever call start on it again.
> 
> so it can be argued that start and txeof should be serialised.
> indeed, other platforms do exactly that.
> 
> the least worst mechanism we have for doing that is taskqs. however,
> all my experiments deferring start to a taskq end up significantly
> hurting performance.
> 
> dragonfly appears to have some of the semantics we want. according
> to sephe, start and txeof are serialised, but they can be directly
> called from anywhere. however, if one cpu is trying to run start
> while the other is in txeof, it figures it out and makes the other
> cpu run txeof on the first cpus behalf. the first cpu then simply
> returns cos it knows the other cpu will end up doing the work.
> 
> the implementation is tied very much to that specific situation,
> and its hard for me to grok cos im not familiar with their locking
> infrastructure.
> 
> the dfly code has the (slight) caveat that you cant run txeof and
> start concurrently, it forces them to be serialised.
> 
> while toying with ideas on how to solve kettenis' oactive problem,
> i came up with the following.
> 
> it combines tasks with direct dispatch, and borrows the current
> ifq_serialiser/pool/scsi serialisation algorithm.
> 
> the idea is you have a taskctx, which represents a serialising
> context for tasks. tasks are submitted to the taskctx, and the code
> will try to run the tasks immediately rather than defer them to a
> thread. if there is contention on the context, the contending cpu
> yields after queueing the task because the other cpu is responsible
> for running all pending tasks to completion.
> 
> it also simplifies the barrier operations a lot.
> 
> the diff below implements a generic taskctx framework, and cuts the
> mpsafe if_start() implementation over to it.
> 
> myx is also changed to only clr oactive from within the taskctx
> serialiser, thereby avoiding the race, but keeps the bulk of txeof
> outside the serialiser so it can run concurrently with start.
> 
> other nics are free to serialise start and txeof within the
> ifq_serializer if they want, or not, it is up to them.
> 
> thoughts? tests? opinions on messy .h files?


Hi,

after applying this patches over cvs source from few hours (no
additional patches for ix and em) it seems that something isn't right...

freshly rebooted box, sending 2Mpps over ix (82599) and i'm getting
around 50kpps on receiver... over x540 around 100kpps

Re: taskctx and revisiting if_start serialisation

[Hrvoje Popovski]

On 6.12.2015. 15:56, Hrvoje Popovski wrote:
> On 6.12.2015. 5:00, David Gwynne wrote:
>> the current code for serialising if_start calls for mpsafe nics does what it 
>> says.
>>
>> however, kettenis realised it doesnt help us much when we're trying
>> to coordinate between the start and txeof side of a driver when
>> setting or clearing oactive. in particular, a start routine can
>> figure out there's no more space, and then set oactive. txeof could
>> be running on another cpu emptying the ring and clearing it. if
>> that clear runs in between the other cpus space check and
>> ifq_set_oactive, then the nic will be marked full and the stack
>> wont ever call start on it again.
>>
>> so it can be argued that start and txeof should be serialised.
>> indeed, other platforms do exactly that.
>>
>> the least worst mechanism we have for doing that is taskqs. however,
>> all my experiments deferring start to a taskq end up significantly
>> hurting performance.
>>
>> dragonfly appears to have some of the semantics we want. according
>> to sephe, start and txeof are serialised, but they can be directly
>> called from anywhere. however, if one cpu is trying to run start
>> while the other is in txeof, it figures it out and makes the other
>> cpu run txeof on the first cpus behalf. the first cpu then simply
>> returns cos it knows the other cpu will end up doing the work.
>>
>> the implementation is tied very much to that specific situation,
>> and its hard for me to grok cos im not familiar with their locking
>> infrastructure.
>>
>> the dfly code has the (slight) caveat that you cant run txeof and
>> start concurrently, it forces them to be serialised.
>>
>> while toying with ideas on how to solve kettenis' oactive problem,
>> i came up with the following.
>>
>> it combines tasks with direct dispatch, and borrows the current
>> ifq_serialiser/pool/scsi serialisation algorithm.
>>
>> the idea is you have a taskctx, which represents a serialising
>> context for tasks. tasks are submitted to the taskctx, and the code
>> will try to run the tasks immediately rather than defer them to a
>> thread. if there is contention on the context, the contending cpu
>> yields after queueing the task because the other cpu is responsible
>> for running all pending tasks to completion.
>>
>> it also simplifies the barrier operations a lot.
>>
>> the diff below implements a generic taskctx framework, and cuts the
>> mpsafe if_start() implementation over to it.
>>
>> myx is also changed to only clr oactive from within the taskctx
>> serialiser, thereby avoiding the race, but keeps the bulk of txeof
>> outside the serialiser so it can run concurrently with start.
>>
>> other nics are free to serialise start and txeof within the
>> ifq_serializer if they want, or not, it is up to them.
>>
>> thoughts? tests? opinions on messy .h files?
> 
> 
> Hi,
> 
> after applying this patches over cvs source from few hours (no
> additional patches for ix and em) it seems that something isn't right...
> 
> freshly rebooted box, sending 2Mpps over ix (82599) and i'm getting
> around 50kpps on receiver... over x540 around 100kpps


on other box i have bge (5270) and everything seems fine... sending
1Mpps and getting cca 650kpps

[patch] cwm: Fix moving of clients to group 0

[Vadim Vygonets]

Hi,

I have a shy client (stalonetray) that hides at start-up.

What normally happens with new clients assigned to group 0 is
this:

- client appears
- client is initialized by cwm
- cwm decides it should be in group 0
- group_assign() sends it a _NET_WM_DESKTOP property change
  request

So far so good, but with stalonetray it doesn't stop there:

- cwm receives a _NET_WM_DESKTOP property change notification
  (presumabily from the client)
- cwm moves the client to group 0
- group_movetogroup(), due to its internal logic, decides to hide
  the client
- client_hide() sends it a WM_STATE property change request
- group_assign() sends it another _NET_WM_DESKTOP property change
  request
- no further _NET_WM_DESKTOP property change notifications are
  received, so that's it

This patch fixes the case of moving clients from group 0 to group
0, as done for other groups.

Yours,
Vadik.

-- 
The Consultant's Curse:
When the customer has beaten upon you long enough, give him
what he asks for, instead of what he needs.  This is very strong
medicine, and is normally only required once.
Index: group.c
===
RCS file: /cvs/xenocara/app/cwm/group.c,v
retrieving revision 1.121
diff -u -r1.121 group.c
--- group.c	10 Nov 2015 20:05:33 -	1.121
+++ group.c	25 Nov 2015 16:36:02 -
@@ -152,19 +152,21 @@
 group_movetogroup(struct client_ctx *cc, int idx)
 {
 	struct screen_ctx	*sc = cc->sc;
-	struct group_ctx	*gc;
+	struct group_ctx	*gc = NULL;
 
 	if (idx < 0 || idx >= CALMWM_NGROUPS)
 		errx(1, "group_movetogroup: index out of range (%d)", idx);
 
-	TAILQ_FOREACH(gc, >groupq, entry) {
-		if (gc->num == idx)
-			break;
+	if (idx != 0) {
+		TAILQ_FOREACH(gc, >groupq, entry) {
+			if (gc->num == idx)
+break;
+		}
 	}
 
 	if (cc->gc == gc)
 		return;
-	if (group_holds_only_hidden(gc))
+	if (gc != NULL && group_holds_only_hidden(gc))
 		client_hide(cc);
 	group_assign(gc, cc);
 }

Re: [PATCH] vmm(4) manpage missed the vmctl rename

[Mike Larkin]

On Sun, Dec 06, 2015 at 11:32:24AM -0500, trondd wrote:
> Update manpage reference from vmmctl(8) to vmctl(8)
> 
> Tim.
> 
> 
> Index: vmm.4
> ===
> RCS file: /cvs/src/share/man/man4/man4.amd64/vmm.4,v
> retrieving revision 1.3
> diff -u -p -r1.3 vmm.4
> --- vmm.4   13 Nov 2015 07:55:37 -  1.3
> +++ vmm.4   6 Dec 2015 16:27:41 -
> @@ -72,7 +72,7 @@ For more information, consult the CPU ve
>  .Xr intro 4 ,
>  .Xr virtio 4 ,
>  .Xr vmd 8 ,
> -.Xr vmmctl 8
> +.Xr vmctl 8
>  .Sh HISTORY
>  The
>  .Nm
>

Thanks, I'll commit it. 

Re: OpenBSD errata, Dec 3, 2015

[Bob Beck]

As many people have already noticed and mentioned, s/-13/-31/g in the
CVE numbers assigned as part of  the great CVE game.

No, I can't "change the announcement" as I can't go edit the internet
to change public mailing list archives.. The CVE numbers are correct
in the patches and everywhere else that matters.


On Thu, Dec 3, 2015 at 11:01 PM, Bob Beck  wrote:
>
> Four new OpenSSL CVE's were released today, which OpenSSL deemed to be
> not of sufficient severity to warrant advance disclosure.
>
> OpenBSD/LibreSSL is not vulnerable to two of these CVE's.
>
> CVE-2015-1393: Recently introduced in OpenSSL only. We did not merge
> this because it gave miod@ a bad feeling.
>
> CVE-2015-1394: NULL pointer dereference in client side certificate
> validation. It was reported to OpenSSL on Aug 27, 2015, and kept
> secret from the community until Dec 3, 2015 by OpenSSL and the
> reporter of the bug.
>
> CVE-2015-1395: Memory leak in PKCS7 - not reachable from TLS/SSL
>
> CVE-2015-1396: String handling bug in code we deleted long ago, using
> a function that all uses of which were flensed from LibreSSL shortly
> after it's creation.
>
> Fixes have been commited for both CVE-2015-1394 and CVE-2015-1395.
> CVE-2015-1394 warrants an errata.
>
> The errata for CVE-2015-1394 is available for OpenBSD 5.8 and OpenBSD
> 5.7 from the master site as well as the mirrors:
>
> http://ftp.openbsd.org/pub/OpenBSD/patches/5.8/common/009_clientcert.patch.sig
> http://ftp.openbsd.org/pub/OpenBSD/patches/5.7/common/021_clientcert.patch.sig

Re: vmm update

[Mike Larkin]

On Thu, Nov 26, 2015 at 06:31:34PM +0100, Norman Golisz wrote:
> On Thu Nov 26 2015 15:11, Karel Gardas wrote:
> > Not sure, but on misc you can search for "vmm uvm_fault in vmware
> > player/workstation when Intel VT/AMD-v not enabled" thread from which
> > it looks like vmm requires extended-page tables virtualization
> > feature.
> 
> Looks like this could indeed be related ...
> 
> > Certainly this is not presented on my T500 so I would guess
> > it's neither on your T400. Please verify yourself.
> 
> You're right, it isn't here either:
> 
> > vmm0 at mainbus0: initialized
> > vmm0: 2 VMX capable CPU(s), 0 are EPT capable
> 

Yes, for the moment this is the case. I have diffs for shadow paging
(which will allow running on these cpus) but they rotted and I pulled
them out of the tree. It's on the list of things to do though.

I'm committing a diff now that will block this sort of thing so at
least it doesn't panic.

nc -r: remove modulo bias and better shuffling

[Theo Buehler]

The current implementation of the selection of a random sequence of
ports in nc -r suffers from modulo bias and a biased shuffling
procedure.  Use arc4random_uniform() and the Fisher-Yates shuffle
instead.

Index: usr.bin/nc/netcat.c
===
RCS file: /cvs/src/usr.bin/nc/netcat.c,v
retrieving revision 1.144
diff -u -p -r1.144 netcat.c
--- usr.bin/nc/netcat.c 23 Nov 2015 01:23:56 -  1.144
+++ usr.bin/nc/netcat.c 6 Dec 2015 18:28:39 -
@@ -1303,8 +1303,8 @@ build_ports(char *p)
int y;
char *c;
 
-   for (x = 0; x <= (hi - lo); x++) {
-   y = (arc4random() & 0x) % (hi - lo);
+   for (x = hi - lo; x >= 1; x--) {
+   y = lo + arc4random_uniform(x + 1);
c = portlist[x];
portlist[x] = portlist[y];
portlist[y] = c;

Re: Move ancient ksh bug from README to man page

[Jérémie Courrèges-Anglas]

Michael McConville  writes:

> Jeremie Courreges-Anglas wrote:
>> Jason McIntyre  writes:
>> 
>> > On Sun, Dec 06, 2015 at 04:03:16AM -0500, Ted Unangst wrote:
>> >> Jason McIntyre wrote:
>> >> > the trouble is i think there are some known bugs with ksh. i
>> >> > think it probably would be better to keep a note of them in a
>> >> > separate file, as is done now. i'm not really sure if we want to
>> >> > try and clutter the page with every bug we find.
>> >> 
>> >> this particular bug seemed likely to hurt people. i think it will
>> >> save people time to document it. 
>> >
>> > yes, fair enough.
>> 
>> It's not a bug so I'd rather add it to CAVEATS.
>
> Why not? I don't know much shell trivia.

Huh.  It's an implementation detail, you just can't rely on the result.
In pdksh, the 'read' builtin in the pipeline is ran in a subshell - which
can't affect the value of the "a" variable in its parent.  Other
POSIX-compliant shells behave the same as pdksh, eg. ports/shells/dash.

As I said there is no bug, so if you want to document it, BUGS does not
make sense to me.

>> Also "Since time immemorial" looks superfluous.
>
> Maybe a cute way of phrasing it, but I though it worthwhile to briefly
> mention that this has been around for >20 years. It suggests that it's
> well-known and not easy to fix (or, as you say, not a bug at all).

If the manpage mentions it then it is well-known, right?  Also I'd
rather *say* that it is not a bug, rather than trying to suggest it.

-- 
jca | PGP : 0x1524E7EE / 5135 92C1 AD36 5293 2BDF  DDCC 0DFA 74AE 1524 E7EE

Re: Move ancient ksh bug from README to man page

[Michael McConville]

Ted Unangst wrote:
> Jason McIntyre wrote:
> > the trouble is i think there are some known bugs with ksh. i think
> > it probably would be better to keep a note of them in a separate
> > file, as is done now. i'm not really sure if we want to try and
> > clutter the page with every bug we find.
> 
> this particular bug seemed likely to hurt people. i think it will save
> people time to document it. 

This was my perspective as well. Forgot to include it in my original
mail.

I don't think we should use the man page to document arcane, complex
bugs. Just important stuff like this.

Re: OpenBSD errata, Dec 3, 2015

[Артур Истомин]

On Sun, Dec 06, 2015 at 06:04:18AM -0500, Ted Unangst wrote:
> Bob Beck wrote:
> > Fixes have been commited for both CVE-2015-1394 and CVE-2015-1395.
> > CVE-2015-1394 warrants an errata.
> 
> > The errata for CVE-2015-1394 is available for OpenBSD 5.8 and OpenBSD
> > 5.7 from the master site as well as the mirrors:
> 
> To clear up any confusion, the CVE numbers should be 3194 and 3195.

and ftp, isakmpd and iked need to be rebuilded for 5.7 and 5.8

Re: Move ancient ksh bug from README to man page

[Jason McIntyre]

On Sun, Dec 06, 2015 at 09:17:21AM +0100, Theo Buehler wrote:
> On Sun, Dec 06, 2015 at 12:00:36AM -0500, Michael McConville wrote:
> > Thoughts? ok?
> > 
> 
> It makes sense to me to document this in the manual, and not in a README
> file that no-one reads, so ok from me, but please wait for jmc@'s input.
> 

i already replied off-list.

> Note that this bug used to be documented in the manual, but it was first
> commented out by jmc in -r1.91 (2005) and then deleted by you in -r1.167
> this year.
> 

well, that's embarrassing ;) i think it was because of the redirection
upstream and the fact that the bug was already covered in the readme.
my commit message was poor, but there was a lot of stuff too trivial to
list i guess.

my advice to michael was that i had no real opinion about this but that
his diff was excessively wordy.

the trouble is i think there are some known bugs with ksh. i think it
probably would be better to keep a note of them in a separate file, as
is done now. i'm not really sure if we want to try and clutter the page
with every bug we find.

but i'm not the one doing work on this, so i didn;t want to say either
way.

> Moreover, shouldn't this bug be mentioned in sh.1 as well?
> 

no. sh(1) attempts only to document features safe to use if you are
interested in portability. i am not interested in trying to replicate
info from ksh(1). you really need to read ksh(1) for stuff like that.

jmc

Re: OpenBSD errata, Dec 3, 2015

[Ted Unangst]

Bob Beck wrote:
> Fixes have been commited for both CVE-2015-1394 and CVE-2015-1395.
> CVE-2015-1394 warrants an errata.

> The errata for CVE-2015-1394 is available for OpenBSD 5.8 and OpenBSD
> 5.7 from the master site as well as the mirrors:

To clear up any confusion, the CVE numbers should be 3194 and 3195.

Re: [UPDATE] freetype-2.6.2

[Christian Weisgerber]

David Coppa:

> Here's the update to freetype-2.6.2.
> 
> It shouldn't cause any fallout, but who knows with freetype... So
> probably a ports bulk build can be useful.

I ran a bulk build with this on amd64.  Unfortunately, a few hundred
packages couldn't be built due to unrelated fallout from the network
hackathon, but there were no problems with freetype.

-- 
Christian "naddy" Weisgerber  na...@mips.inka.de

Re: mpsafe re(4)

[Dimitris Papastamos]

On Sat, Dec 05, 2015 at 06:11:51PM +0100, Jonathan Matthew wrote:
> The main interesting bit here is the txeof and start loops, which previously
> operated based on the prod/cons indices and the contents of the tx queue,
> but now just uses the indices as that's the only way to get a consistent view
> of the tx queue state.
> 
> At the moment I don't think the tx ring is big enough to use IFQ_DEQUEUE
> instead of ifq_deq_begin/commit, but maybe I'm wrong about that.
> 
> can someone try this on an APU1?

I've tested this on my router and it seems to work okay.  I've also used
tcpbench with various combinations.

re0 at pci2 dev 0 function 0 "Realtek 8168" rev 0x0c: RTL8168G/8111G
(0x4c00), msi, address 80:ee:73:9f:1d:3e
re1 at pci3 dev 0 function 0 "Realtek 8168" rev 0x0c: RTL8168G/8111G
(0x4c00), msi, address 80:ee:73:9f:1d:3d

Re: taskctx and revisiting if_start serialisation

[Norman Golisz]

Hi,

I can't comment on the code itself, but there's

>  The
>  taskq
>  API provides a mechanism to defer work to a process context.
>  .Pp
> +The
> +taskctx
> +API provides a mechanism to serialise work in a single context.
> +A taskctx guarantees that all work submitted to it will not run
> +concurrently and can therefore provide exclusive access to a resource.
> +It attempts to run the submitted work immediately, unless another
> +another CPU is already running work in the taskctx.
   ^^^
one "another" too many.

ugenctl for attaching USB devices to ugen instead of their specific driver

[Mickael Torres]

Hello,

This is a kernel patch plus a utility called ugenctl I use to allow 
selected USB devices to attach as ugen(4) instead of their more specific 
driver. My use case is a Microchip "PICkit 2 Microcontroller Programmer" 
that attaches as uhid(4), but the command line utility pk2cmd wants a 
udev(4) device. There maybe are some other use cases.


It is mostly quick and dirty, but does the job.

It can be found there: https://github.com/mickaeltorres/openbsd_ugenctl 
or inlined at the end of this mail.


Cheers,
Mike.

Here is the kernel patch:

Index: usb.c
===
RCS file: /cvs/src/sys/dev/usb/usb.c,v
retrieving revision 1.107
diff -u -p -r1.107 usb.c
--- usb.c   14 Mar 2015 03:38:50 -  1.107
+++ usb.c   6 Dec 2015 04:29:22 -
@@ -137,6 +137,8 @@ const struct cfattach usb_ca = {
usb_activate,
 };

+struct usb_device_ugen_force usb_udf;
+
 int
 usb_match(struct device *parent, void *match, void *aux)
 {
@@ -156,6 +158,7 @@ usb_attach(struct device *parent, struct
TAILQ_INIT(_generic_tasks);
usb_run_tasks = usb_run_abort_tasks = 1;
kthread_create_deferred(usb_create_task_threads, NULL);
+   bzero(_udf, sizeof(usb_udf));
}
usb_nbuses++;

@@ -792,6 +795,23 @@ usbioctl(dev_t devt, u_long cmd, caddr_t
return (error);
}

+   case USB_DEVICE_FORCE_UGEN:
+   {
+   struct usb_device_ugen_force *udf =
+   (struct usb_device_ugen_force *)data;
+
+   if (udf->udf_set) // replace our list
+   {
+   memcpy(_udf, udf, sizeof(usb_udf));
+   return (0);
+   }
+   else  // return the list
+   {
+   memcpy(udf, _udf, sizeof(usb_udf));
+   return (0);
+   }
+   }
+
default:
return (EINVAL);
}
Index: usb.h
===
RCS file: /cvs/src/sys/dev/usb/usb.h,v
retrieving revision 1.53
diff -u -p -r1.53 usb.h
--- usb.h   9 Jul 2015 05:40:44 -   1.53
+++ usb.h   6 Dec 2015 04:29:23 -
@@ -749,6 +749,13 @@ struct usb_device_stats {
u_long  uds_requests[4];/* indexed by transfer type UE_* 
*/

 };

+#define USB_DEVICE_UGEN_FORCE_MAX 16
+struct usb_device_ugen_force {
+int udf_set;
+u_short udf_ven_id[USB_DEVICE_UGEN_FORCE_MAX];
+u_short udf_dev_id[USB_DEVICE_UGEN_FORCE_MAX];
+};
+
 /* USB controller */
 #define USB_REQUEST_IOWR('U', 1, struct usb_ctl_request)
 #define USB_SETDEBUG   _IOW ('U', 2, unsigned int)
@@ -758,6 +765,7 @@ struct usb_device_stats {
 #define USB_DEVICE_GET_CDESC   _IOWR('U', 6, struct usb_device_cdesc)
 #define USB_DEVICE_GET_FDESC   _IOWR('U', 7, struct usb_device_fdesc)
 #define USB_DEVICE_GET_DDESC   _IOWR('U', 8, struct usb_device_ddesc)
+#define USB_DEVICE_FORCE_UGEN   _IOWR('U', 9, struct 
usb_device_ugen_force)


 /* Generic HID device */
 #define USB_GET_REPORT_DESC_IOR ('U', 21, struct 
usb_ctl_report_desc)

Index: usb_subr.c
===
RCS file: /cvs/src/sys/dev/usb/usb_subr.c,v
retrieving revision 1.117
diff -u -p -r1.117 usb_subr.c
--- usb_subr.c  23 Mar 2015 22:26:01 -  1.117
+++ usb_subr.c  6 Dec 2015 04:29:23 -
@@ -840,6 +840,7 @@ usbd_probe_and_attach(struct device *par
struct device *dv;
struct usbd_interface **ifaces;
extern struct rwlock usbpalock;
+   extern struct usb_device_ugen_force usb_udf;

rw_enter_write();

@@ -855,6 +856,14 @@ usbd_probe_and_attach(struct device *par
uaa.product = UGETW(dd->idProduct);
uaa.release = UGETW(dd->bcdDevice);

+   /* Check if this device is in the ugen force list */
+   for (i = 0; i < USB_DEVICE_UGEN_FORCE_MAX; i++)
+   if ((usb_udf.udf_ven_id[i] == uaa.vendor) &&
+   (usb_udf.udf_dev_id[i] == uaa.product))
+   break;
+   if (i != USB_DEVICE_UGEN_FORCE_MAX)
+   goto generic;
+
/* First try with device specific drivers. */
DPRINTF(("usbd_probe_and_attach trying device specific 
drivers\n"));

dv = config_found(parent, , usbd_print);


And here is the ugenctl utility:

Makefile:

#   $OpenBSD: Makefile,v 1.1 2000/02/03 21:52:15 jakob Exp $
#   $NetBSD: Makefile,v 1.2 1998/07/12 20:40:45 augustss Exp $

PROG=   ugenctl
MAN=ugenctl.8

.include 

ugenctl.c:

#include 
#include 
#include 
#include 
#include 
#include 
#include 

struct usb_change_list {
u_short vid[USB_DEVICE_UGEN_FORCE_MAX];
u_short pid[USB_DEVICE_UGEN_FORCE_MAX];
int rem[USB_DEVICE_UGEN_FORCE_MAX];
int end;
};

#define USB_PATH "/dev/usb0"

extern char *__progname;

void

Re: [patch] cwm: Preserve stacking order during cycling

[Артур Истомин]

On Wed, Nov 25, 2015 at 05:44:28PM +0100, Vadim Vygonets wrote:
> Quoth Артур Истомин on Tue, Nov 24, 2015:
> > Yes, exactly. Example: https://imgur.com/rUPxpTF There is mplayer behind
> > firefox. In the beginning everything is working properly. Alt+Tab work for
> > all three windows. Some time later mplayer does not appear anymore through
> > Alt+Tab and i need to press Alt+Down, Alt+Down.
> 
> Does it only happen to windows that are fully obscured?  Do you
> have to get the mouse pointer into the window before you can
> cycle to it again?
> 
> If you're willing, we can do the following: you update to current
> plus my patches (I won't roll back), I prepare a patch with debug
> prints, you run it until you encounter the bug and send me the
> output, and we try to understand what's going on.  Interested?

Vadik, sorry for silent. I upgraded cwm to latest from current two
days ago (need to comment out pledge's stuff) and my bug disappeared :)

Re: malloc canaries and validation

[Theo de Raadt]

> It would also be interesting to try out a more aggressive form of
> freeunmap for 64-bit where the allocations are purged with MADV_FREE
> and then the virtual memory is kept out of circulation with a similar
> FIFO queue approach. Could potentially do it by default when malloc
> hints are enabled, so it wouldn't need a new option exposed (but it
> would change the MADV_FREE option into something that enhances security
> at the expense of more VM fragmentation rather than a performance vs.
> memory trade-off so that may not make much sense after all).
> 
> It's the same issue as the junk validation feature where there's a need
> for a reliable delay to get the most out of the feature. Randomization
> does help, but it's not as good as knowing that virtual memory doesn't
> go back into circulation until some configured amount of allocator
> throughput has occurred.

Kept out of circulation?  It sounds like it would be incredibly
expensive data-structure wise for the kernel to even attempt such a
gaurantee..

On a 64-bit VA / 48-bit PA system, the address space should be large
enough that this does not occur commonly due to mmap ASLR; or at
least, when it does occur, the reuse cannot be predicted usably.

So what is the purpose of your proposal?  Is it to opportunistically
find bugs?  Or to resist attack in UAF situations by "gauranteeing"
the memory is unallocated at time of reuse?

#include 
#include 
#include 

main()
{
int max = 10;

while (max--) {
void *p = mmap(NULL, 4096, PROT_READ, MAP_ANON, -1, 0);
munmap(p, 4096);
printf("%p\n", p);
}
}

./test | sort | uniq -c | sort -n | awk '{print $1}' | sort | uniq -c

90880 1  (addresses only used once)
 4314 2
  123 3
3 4
1 111(I wonder what this subtle bias is due to)

These are independent allocations of course, but maybe we can get a trend
out of this.  Except in really large memory programs, I suspect a UAF would
find the memory unallocation, as long as free() has pushed it out via
munmap().  (Curiously, your other proposal is about the part of the cache
which is not munmap()'d...)

Add in the subtle effects of guarding we also do, and it seems we are
in good shape.

Maybe you are on a system with where mmap ASLR isn't as good?  The
OpenBSD one isn't perfect, because we are trying to maintain a small
amount of page-table locality (so that N-level page tables don't have
to be walked fully from root for every access that misses in the TLB).

The result above feels "good enough" to make reliable UAF attack after
free()->munmap() very difficult.  Am I missing something which would
justify what seems like a huge cost?

Re: nc -r: remove modulo bias and better shuffling

[Theo de Raadt]

> Since I don't think your name is Steve, you're going to touch that,
> please get rid of the PORT_MAX_LEN/calloc/snprintf dance and just use
> asprintf() instead so I don't need more therapy reading it.. (look at
> my eye twitching...)

The part I find most incredible in the code is...

ntohs(atoi(portlist[i])),

and then digging deeper... to see how the portname (as a string)
is passed down to socks... and reversed into an integer...

AGhh... I just went blind.

Re: malloc canaries and validation

[Daniel Micay]

It would be great to land this simple initial implementation and move
from there. I have ideas on how to make these features better but I'm
wary of doing a lot of work out-of-tree. If it lands in some form, that
would go a long way to encouraging further work on it. I basically just
don't want to end up maintaining a whole allocator implementation.

For detecting UAF, the ideal would be having a FIFO ring buffer placed
in front of the randomized delayed chunk array. The delay would then
depend on the amount of memory that's set aside for this. For example,
delaying up to 4M of small allocations (which is a lot, but totally
doable with modern hardware) would go a long way to catching most
writes after free via this junk validation feature. Could potentially
be configurable as the cache size is. It also becomes increasingly more
useful to check the currently delayed allocations at exit rather than
just when they get flushed out of the delayed array (and/or queue) if
there's actually a longer form of delay. It isn't all that useful right
now, but it would be a valuable component of this if it was fancier.

Right now it basically relies on luck to detect anything, unless the
allocation is immediately written to after a free and before the next
free. It's a place where a mix of determinism and non-determinism would
be best, i.e. a guaranteed amount of delay, but without losing the
randomization since that is useful for other reasons (randomized heap
layout).

The malloc canaries mostly just don't play well with the current power
of two size classes. They'd be very cheap if the size classes were
finer grained, i.e. accepting more external fragmentation but reducing
the internal fragmentation (wasted padding). There's a nice technique
used in jemalloc based on spacing classes rather than size classes but
it isn't appropriate in OpenBSD malloc because it would lose the nice
property of stuff being isolated based on size (and it would really be
best to *expand* that property more towards what PartitionAlloc does
rather than losing it).

Re: malloc canaries and validation

[Daniel Micay]

On Sun, 2015-12-06 at 17:10 -0700, Theo de Raadt wrote:
> Kept out of circulation?  It sounds like it would be incredibly
> expensive data-structure wise for the kernel to even attempt such a
> gaurantee..

I was just expecting it to be a FIFO ring buffer. It would have a limit
on the number of mappings it can delay along with a maximum upper bound
on the total mapping size (and it would flush the oldest mappings out
to stay under that threshold). It would increase virtual memory
fragmentation so it could be expensive.

I was thinking about it from a userspace perspective which is why I had
mentioned purging the delayed mappings with MADV_FREE. It wouldn't make
sense to do it in userspace though. Simply unmapping right away and
approaching it in the kernel is better.

> So what is the purpose of your proposal?  Is it to opportunistically
> find bugs?  Or to resist attack in UAF situations by "gauranteeing"
> the memory is unallocated at time of reuse?

Guaranteeing that the memory is not reused until a given amount of
allocator throughput happens. And reliable immediate detection of the
access since it will be guaranteed to fault.

> Maybe you are on a system with where mmap ASLR isn't as good?  The
> OpenBSD one isn't perfect, because we are trying to maintain a small
> amount of page-table locality (so that N-level page tables don't have
> to be walked fully from root for every access that misses in the
> TLB).

Yeah, the Linux kernel's mmap ASLR only randomizes the base. So I'm not
getting the same security properties as OpenBSD malloc has on OpenBSD.

I could implement fine-grained randomization in the kernel for 64-bit
(Android has no virtual memory to spare on 32-bit) but I thought a
delay might be useful even with that in place as it fully prevents
reuse for a given period of "time" (allocator throughput). It depends
on how aggressive the mmap randomization is and perhaps on the level of
control that an attacker can exert over memory allocation.

Re: nc -r: remove modulo bias and better shuffling

[Bob Beck]

Since I don't think your name is Steve, you're going to touch that,
please get rid of the PORT_MAX_LEN/calloc/snprintf dance and just use
asprintf() instead so I don't need more therapy reading it.. (look at
my eye twitching...)


On Sun, Dec 6, 2015 at 2:01 PM, Theo Buehler  wrote:
> On Sun, Dec 06, 2015 at 07:37:27PM +0100, Theo Buehler wrote:
>> The current implementation of the selection of a random sequence of
>> ports in nc -r suffers from modulo bias and a biased shuffling
>> procedure.  Use arc4random_uniform() and the Fisher-Yates shuffle
>> instead.
>
> daniel@ pointed out that it might be nicer to combine initialization
> and shuffling, as in ip_id.c, so here's a second version of the patch.
> The patch is bigger, but the resulting code looks tighter.
>
> Index: usr.bin/nc/netcat.c
> ===
> RCS file: /cvs/src/usr.bin/nc/netcat.c,v
> retrieving revision 1.144
> diff -u -p -r1.144 netcat.c
> --- usr.bin/nc/netcat.c 23 Nov 2015 01:23:56 -  1.144
> +++ usr.bin/nc/netcat.c 6 Dec 2015 20:53:20 -
> @@ -1289,6 +1289,23 @@ build_ports(char *p)
> lo = cp;
> }
>
> +   if (rflag) {
> +   /*
> +* Initialize portlist[] with a random permutation.
> +* Based on Knuth, as in ip_randomid() in ip_id.c.
> +*/
> +   for (x = 0; x <= hi - lo; x++) {
> +   cp = arc4random_uniform(x + 1);
> +   portlist[x] = portlist[cp];
> +   portlist[cp] = calloc(1, PORT_MAX_LEN);
> +   if (portlist[cp] == NULL)
> +   err(1, NULL);
> +   snprintf(portlist[cp], PORT_MAX_LEN, "%d",
> +   x + lo);
> +   }
> +   return;
> +   }
> +
> /* Load ports sequentially. */
> for (cp = lo; cp <= hi; cp++) {
> portlist[x] = calloc(1, PORT_MAX_LEN);
> @@ -1296,19 +1313,6 @@ build_ports(char *p)
> err(1, NULL);
> snprintf(portlist[x], PORT_MAX_LEN, "%d", cp);
> x++;
> -   }
> -
> -   /* Randomly swap ports. */
> -   if (rflag) {
> -   int y;
> -   char *c;
> -
> -   for (x = 0; x <= (hi - lo); x++) {
> -   y = (arc4random() & 0x) % (hi - lo);
> -   c = portlist[x];
> -   portlist[x] = portlist[y];
> -   portlist[y] = c;
> -   }
> }
> } else {
> hi = strtonum(p, 1, PORT_MAX, );
>

Re: malloc canaries and validation

[Daniel Micay]

It would also be interesting to try out a more aggressive form of
freeunmap for 64-bit where the allocations are purged with MADV_FREE
and then the virtual memory is kept out of circulation with a similar
FIFO queue approach. Could potentially do it by default when malloc
hints are enabled, so it wouldn't need a new option exposed (but it
would change the MADV_FREE option into something that enhances security
at the expense of more VM fragmentation rather than a performance vs.
memory trade-off so that may not make much sense after all).

It's the same issue as the junk validation feature where there's a need
for a reliable delay to get the most out of the feature. Randomization
does help, but it's not as good as knowing that virtual memory doesn't
go back into circulation until some configured amount of allocator
throughput has occurred.

Re: ugenctl for attaching USB devices to ugen instead of their specific driver

[Mike]

On 2015-12-06 20:10, Ian Darwin wrote:

On 2015-12-06 12:23 PM, Stuart Henderson wrote:

On 2015/12/06 06:02, Mickael Torres wrote:

Hello,

This is a kernel patch plus a utility called ugenctl I use to allow
selected USB devices to attach as ugen(4) instead of their more 
specific
driver. My use case is a Microchip "PICkit 2 Microcontroller 
Programmer"

that attaches as uhid(4), but the command line utility pk2cmd wants a
udev(4) device. There maybe are some other use cases.

If a device is generally pointless with uhid, we can just knock it
out completely in the kernel, see the UQ_BAD_HID mechanism. I think 
this

applies to your device.

The bigger problem is "dual use" devices; e.g. some want UPS to attach
to upd(4), others want ugen(4) for use with NUT/apcupsd. Your code is
partially useful for these, but because it just changes things at 
attach,

won't survive a reboot - if it could instead force a device to detach
and reattach to ugen it would help a lot more cases.

One fairly common (I believe) use is with USB printers that attach as 
ulpt
but CUPS wants as ugen - for these, this is useful as it stands - just 
run it
from rc. Do people want their UPS to sometimes be upd and other times 
be
a ugen? I can see there might be "I want to change this now" cases, but 
I
suspect the majority could be done once at each boot and be quite 
useful.


Hello,

an updated version that detach/reattach if a device is already 
connected, when

adding and when removing from the ugen list.
Seems to work well, but I don't have a lot of hardware to test it.
What do you think ?

https://github.com/mickaeltorres/openbsd_ugenctl and inlined.

Cheers,
Mike.

kernel patch below:

Index: usb.c
===
RCS file: /cvs/src/sys/dev/usb/usb.c,v
retrieving revision 1.107
diff -u -p -r1.107 usb.c
--- usb.c   14 Mar 2015 03:38:50 -  1.107
+++ usb.c   6 Dec 2015 23:19:48 -
@@ -137,6 +137,8 @@ const struct cfattach usb_ca = {
usb_activate,
 };

+struct usb_device_ugen_force usb_udf;
+
 int
 usb_match(struct device *parent, void *match, void *aux)
 {
@@ -156,6 +158,7 @@ usb_attach(struct device *parent, struct
TAILQ_INIT(_generic_tasks);
usb_run_tasks = usb_run_abort_tasks = 1;
kthread_create_deferred(usb_create_task_threads, NULL);
+   bzero(_udf, sizeof(usb_udf));
}
usb_nbuses++;

@@ -792,6 +795,37 @@ usbioctl(dev_t devt, u_long cmd, caddr_t
return (error);
}

+   case USB_DEVICE_FORCE_UGEN:
+   {
+   struct usb_device_ugen_force *udf =
+   (struct usb_device_ugen_force *)data;
+
+   switch (udf->udf_cmd) {
+   case USB_DEVICE_UGEN_FORCE_GET:
+   {
+   memcpy(udf, _udf, sizeof(usb_udf));
+   break;
+   }
+   case USB_DEVICE_UGEN_FORCE_SET:
+   {
+   memcpy(_udf, udf, sizeof(usb_udf));
+   break;
+   }
+   case USB_DEVICE_UGEN_FORCE_RESET:
+   {
+   struct usbd_device *dev;
+
+   if (udf->udf_addr < 1 ||
+   udf->udf_addr >= USB_MAX_DEVICES)
+   return (EINVAL);
+   dev = sc->sc_bus->devices[udf->udf_addr];
+   usbd_reset_port(dev->myhub, 
dev->powersrc->portno);

+   usb_needs_reattach(dev);
+   break;
+   }
+   default:
+   return (EINVAL);
+   }
+   return (0);
+   }
+
default:
return (EINVAL);
}
Index: usb.h
===
RCS file: /cvs/src/sys/dev/usb/usb.h,v
retrieving revision 1.53
diff -u -p -r1.53 usb.h
--- usb.h   9 Jul 2015 05:40:44 -   1.53
+++ usb.h   6 Dec 2015 23:19:48 -
@@ -749,6 +749,17 @@ struct usb_device_stats {
u_long  uds_requests[4];/* indexed by transfer type UE_* 
*/

 };

+#define USB_DEVICE_UGEN_FORCE_MAX 16
+#define USB_DEVICE_UGEN_FORCE_GET 0
+#define USB_DEVICE_UGEN_FORCE_SET 1
+#define USB_DEVICE_UGEN_FORCE_RESET 2
+struct usb_device_ugen_force {
+int udf_cmd;
+u_char udf_addr;
+u_short udf_ven_id[USB_DEVICE_UGEN_FORCE_MAX];
+u_short udf_dev_id[USB_DEVICE_UGEN_FORCE_MAX];
+};
+
 /* USB controller */
 #define USB_REQUEST_IOWR('U', 1, struct usb_ctl_request)
 #define USB_SETDEBUG   _IOW ('U', 2, unsigned int)
@@ -758,6 +769,7 @@ struct usb_device_stats {
 #define USB_DEVICE_GET_CDESC   _IOWR('U', 6, struct usb_device_cdesc)
 #define USB_DEVICE_GET_FDESC   _IOWR('U', 7, struct usb_device_fdesc)
 #define USB_DEVICE_GET_DDESC   _IOWR('U', 8, struct usb_device_ddesc)
+#define 

Re: nc -r: remove modulo bias and better shuffling

[Bob Beck]

> and then digging deeper... to see how the portname (as a string)
> is passed down to socks... and reversed into an integer...
>
> AGhh... I just went blind.

Maybe since we're already linking in libcrypto/asn1 - if he needs to
store integers as strings all the time..

Re: nc -r: remove modulo bias and better shuffling

[Bob Beck]

Someone help.. Both Theo and I can't reach our epi-pens



On Sun, Dec 6, 2015 at 6:54 PM, Theo de Raadt  wrote:
>> Since I don't think your name is Steve, you're going to touch that,
>> please get rid of the PORT_MAX_LEN/calloc/snprintf dance and just use
>> asprintf() instead so I don't need more therapy reading it.. (look at
>> my eye twitching...)
>
> The part I find most incredible in the code is...
>
> ntohs(atoi(portlist[i])),
>
> and then digging deeper... to see how the portname (as a string)
> is passed down to socks... and reversed into an integer...
>
> AGhh... I just went blind.

Re: nc -r: remove modulo bias and better shuffling

[Bob Beck]

Theo (not deraadt) ignore our screams of pain, fix the thing to use
asprintf and then I'll go further with you on it ;)


On Sun, Dec 6, 2015 at 7:11 PM, Bob Beck  wrote:
>> and then digging deeper... to see how the portname (as a string)
>> is passed down to socks... and reversed into an integer...
>>
>> AGhh... I just went blind.
>
> Maybe since we're already linking in libcrypto/asn1 - if he needs to
> store integers as strings all the time..

Re: nc -r: remove modulo bias and better shuffling

[Theo Buehler]

On Sun, Dec 06, 2015 at 07:12:28PM -0700, Bob Beck wrote:
> Theo (not deraadt) ignore our screams of pain, fix the thing to use
> asprintf and then I'll go further with you on it ;)

ok, there you go :)

> On Sun, Dec 6, 2015 at 7:11 PM, Bob Beck  wrote:
> >> and then digging deeper... to see how the portname (as a string)
> >> is passed down to socks... and reversed into an integer...
> >>
> >> AGhh... I just went blind.
> >
> > Maybe since we're already linking in libcrypto/asn1 - if he needs to
> > store integers as strings all the time..
> 

Index: usr.bin/nc/netcat.c
===
RCS file: /cvs/src/usr.bin/nc/netcat.c,v
retrieving revision 1.144
diff -u -p -r1.144 netcat.c
--- usr.bin/nc/netcat.c 23 Nov 2015 01:23:56 -  1.144
+++ usr.bin/nc/netcat.c 7 Dec 2015 02:17:05 -
@@ -58,7 +58,6 @@
 #include "atomicio.h"
 
 #define PORT_MAX   65535
-#define PORT_MAX_LEN   6
 #define UNIX_DG_TMP_SOCKET_SIZE19
 
 #define POLL_STDIN 0
@@ -1289,25 +1288,22 @@ build_ports(char *p)
lo = cp;
}
 
-   /* Load ports sequentially. */
-   for (cp = lo; cp <= hi; cp++) {
-   portlist[x] = calloc(1, PORT_MAX_LEN);
-   if (portlist[x] == NULL)
-   err(1, NULL);
-   snprintf(portlist[x], PORT_MAX_LEN, "%d", cp);
-   x++;
-   }
-
-   /* Randomly swap ports. */
+   /*
+* Initialize portlist with a random permutation.  Based on
+* Knuth, as in ip_randomid() in sys/netinet/ip_id.c.
+*/
if (rflag) {
-   int y;
-   char *c;
-
-   for (x = 0; x <= (hi - lo); x++) {
-   y = (arc4random() & 0x) % (hi - lo);
-   c = portlist[x];
-   portlist[x] = portlist[y];
-   portlist[y] = c;
+   for (x = 0; x <= hi - lo; x++) {
+   cp = arc4random_uniform(x + 1);
+   portlist[x] = portlist[cp];
+   if (asprintf(portlist[cp], "%d", x + lo) < 0)
+   err(1, "asprintf");
+   }
+   } else { /* Load ports sequentially. */
+   for (cp = lo; cp <= hi; cp++) {
+   if (asprintf(portlist[x], "%d", cp) < 0)
+   err(1, "asprintf");
+   x++;
}
}
} else {

Re: nc -r: remove modulo bias and better shuffling

[Theo Buehler]

On Mon, Dec 07, 2015 at 03:20:21AM +0100, Theo Buehler wrote:
> On Sun, Dec 06, 2015 at 07:12:28PM -0700, Bob Beck wrote:
> > Theo (not deraadt) ignore our screams of pain, fix the thing to use
> > asprintf and then I'll go further with you on it ;)
> 
> ok, there you go :)
> 
> > On Sun, Dec 6, 2015 at 7:11 PM, Bob Beck  wrote:
> > >> and then digging deeper... to see how the portname (as a string)
> > >> is passed down to socks... and reversed into an integer...
> > >>
> > >> AGhh... I just went blind.
> > >
> > > Maybe since we're already linking in libcrypto/asn1 - if he needs to
> > > store integers as strings all the time..
> > 

and again I manage to send the wrong diff. meh.


Index: usr.bin/nc/netcat.c
===
RCS file: /cvs/src/usr.bin/nc/netcat.c,v
retrieving revision 1.144
diff -u -p -r1.144 netcat.c
--- usr.bin/nc/netcat.c 23 Nov 2015 01:23:56 -  1.144
+++ usr.bin/nc/netcat.c 7 Dec 2015 02:25:14 -
@@ -58,7 +58,6 @@
 #include "atomicio.h"
 
 #define PORT_MAX   65535
-#define PORT_MAX_LEN   6
 #define UNIX_DG_TMP_SOCKET_SIZE19
 
 #define POLL_STDIN 0
@@ -1289,25 +1288,22 @@ build_ports(char *p)
lo = cp;
}
 
-   /* Load ports sequentially. */
-   for (cp = lo; cp <= hi; cp++) {
-   portlist[x] = calloc(1, PORT_MAX_LEN);
-   if (portlist[x] == NULL)
-   err(1, NULL);
-   snprintf(portlist[x], PORT_MAX_LEN, "%d", cp);
-   x++;
-   }
-
-   /* Randomly swap ports. */
+   /*
+* Initialize portlist with a random permutation.  Based on
+* Knuth, as in ip_randomid() in sys/netinet/ip_id.c.
+*/
if (rflag) {
-   int y;
-   char *c;
-
-   for (x = 0; x <= (hi - lo); x++) {
-   y = (arc4random() & 0x) % (hi - lo);
-   c = portlist[x];
-   portlist[x] = portlist[y];
-   portlist[y] = c;
+   for (x = 0; x <= hi - lo; x++) {
+   cp = arc4random_uniform(x + 1);
+   portlist[x] = portlist[cp];
+   if (asprintf([cp], "%d", x + lo) < 0)
+   err(1, "asprintf");
+   }
+   } else { /* Load ports sequentially. */
+   for (cp = lo; cp <= hi; cp++) {
+   if (asprintf([x], "%d", cp) < 0)
+   err(1, "asprintf");
+   x++;
}
}
} else {

Re: nc -r: remove modulo bias and better shuffling

[Theo Buehler]

On Sun, Dec 06, 2015 at 07:37:27PM +0100, Theo Buehler wrote:
> The current implementation of the selection of a random sequence of
> ports in nc -r suffers from modulo bias and a biased shuffling
> procedure.  Use arc4random_uniform() and the Fisher-Yates shuffle
> instead.

daniel@ pointed out that it might be nicer to combine initialization
and shuffling, as in ip_id.c, so here's a second version of the patch.
The patch is bigger, but the resulting code looks tighter.

Index: usr.bin/nc/netcat.c
===
RCS file: /cvs/src/usr.bin/nc/netcat.c,v
retrieving revision 1.144
diff -u -p -r1.144 netcat.c
--- usr.bin/nc/netcat.c 23 Nov 2015 01:23:56 -  1.144
+++ usr.bin/nc/netcat.c 6 Dec 2015 20:53:20 -
@@ -1289,6 +1289,23 @@ build_ports(char *p)
lo = cp;
}
 
+   if (rflag) {
+   /*
+* Initialize portlist[] with a random permutation.
+* Based on Knuth, as in ip_randomid() in ip_id.c.
+*/
+   for (x = 0; x <= hi - lo; x++) {
+   cp = arc4random_uniform(x + 1);
+   portlist[x] = portlist[cp];
+   portlist[cp] = calloc(1, PORT_MAX_LEN);
+   if (portlist[cp] == NULL)
+   err(1, NULL);
+   snprintf(portlist[cp], PORT_MAX_LEN, "%d",
+   x + lo);
+   }
+   return;
+   }
+
/* Load ports sequentially. */
for (cp = lo; cp <= hi; cp++) {
portlist[x] = calloc(1, PORT_MAX_LEN);
@@ -1296,19 +1313,6 @@ build_ports(char *p)
err(1, NULL);
snprintf(portlist[x], PORT_MAX_LEN, "%d", cp);
x++;
-   }
-
-   /* Randomly swap ports. */
-   if (rflag) {
-   int y;
-   char *c;
-
-   for (x = 0; x <= (hi - lo); x++) {
-   y = (arc4random() & 0x) % (hi - lo);
-   c = portlist[x];
-   portlist[x] = portlist[y];
-   portlist[y] = c;
-   }
}
} else {
hi = strtonum(p, 1, PORT_MAX, );

Re: nc -r: remove modulo bias and better shuffling

[Theo Buehler]

On Sun, Dec 06, 2015 at 07:37:27PM +0100, Theo Buehler wrote:
> The current implementation of the selection of a random sequence of
> ports in nc -r suffers from modulo bias and a biased shuffling
> procedure.  Use arc4random_uniform() and the Fisher-Yates shuffle
> instead.

Sorry, I attached the wrong patch.

Index: usr.bin/nc/netcat.c
===
RCS file: /cvs/src/usr.bin/nc/netcat.c,v
retrieving revision 1.144
diff -u -p -r1.144 netcat.c
--- usr.bin/nc/netcat.c 23 Nov 2015 01:23:56 -  1.144
+++ usr.bin/nc/netcat.c 6 Dec 2015 18:46:42 -
@@ -1303,8 +1303,8 @@ build_ports(char *p)
int y;
char *c;
 
-   for (x = 0; x <= (hi - lo); x++) {
-   y = (arc4random() & 0x) % (hi - lo);
+   for (x = hi - lo; x >= 1; x--) {
+   y = arc4random_uniform(x + 1);
c = portlist[x];
portlist[x] = portlist[y];
portlist[y] = c;

Re: ugenctl for attaching USB devices to ugen instead of their specific driver

[Mickael Torres]

Hello Stuart,

On 2015-12-06 18:23, Stuart Henderson wrote:

On 2015/12/06 06:02, Mickael Torres wrote:

Hello,

This is a kernel patch plus a utility called ugenctl I use to allow
selected USB devices to attach as ugen(4) instead of their more 
specific
driver. My use case is a Microchip "PICkit 2 Microcontroller 
Programmer"

that attaches as uhid(4), but the command line utility pk2cmd wants a
udev(4) device. There maybe are some other use cases.


If a device is generally pointless with uhid, we can just knock it
out completely in the kernel, see the UQ_BAD_HID mechanism. I think 
this

applies to your device.

The bigger problem is "dual use" devices; e.g. some want UPS to attach
to upd(4), others want ugen(4) for use with NUT/apcupsd. Your code is
partially useful for these, but because it just changes things at 
attach,

won't survive a reboot - if it could instead force a device to detach
and reattach to ugen it would help a lot more cases.



Actually another utility (pic32prog, but I can't seem to make it work 
right now) wants a uhid device, so it was easier to have a dynamic way 
to switch. I'll try to modify this to have the device doing a 
detach/reattach automatically, thanks for the idea.


Cheers,
Mike.


It is mostly quick and dirty, but does the job.

It can be found there: 
https://github.com/mickaeltorres/openbsd_ugenctl or

inlined at the end of this mail.

Cheers,
Mike.

Here is the kernel patch:

Index: usb.c
===
RCS file: /cvs/src/sys/dev/usb/usb.c,v
retrieving revision 1.107
diff -u -p -r1.107 usb.c
--- usb.c   14 Mar 2015 03:38:50 -  1.107
+++ usb.c   6 Dec 2015 04:29:22 -
@@ -137,6 +137,8 @@ const struct cfattach usb_ca = {
usb_activate,
 };

+struct usb_device_ugen_force usb_udf;
+
 int
 usb_match(struct device *parent, void *match, void *aux)
 {
@@ -156,6 +158,7 @@ usb_attach(struct device *parent, struct
TAILQ_INIT(_generic_tasks);
usb_run_tasks = usb_run_abort_tasks = 1;
kthread_create_deferred(usb_create_task_threads, 
NULL);

+   bzero(_udf, sizeof(usb_udf));
}
usb_nbuses++;

@@ -792,6 +795,23 @@ usbioctl(dev_t devt, u_long cmd, caddr_t
return (error);
}

+   case USB_DEVICE_FORCE_UGEN:
+   {
+   struct usb_device_ugen_force *udf =
+   (struct usb_device_ugen_force *)data;
+
+   if (udf->udf_set) // replace our list
+   {
+   memcpy(_udf, udf, sizeof(usb_udf));
+   return (0);
+   }
+   else  // return the list
+   {
+   memcpy(udf, _udf, sizeof(usb_udf));
+   return (0);
+   }
+   }
+
default:
return (EINVAL);
}
Index: usb.h
===
RCS file: /cvs/src/sys/dev/usb/usb.h,v
retrieving revision 1.53
diff -u -p -r1.53 usb.h
--- usb.h   9 Jul 2015 05:40:44 -   1.53
+++ usb.h   6 Dec 2015 04:29:23 -
@@ -749,6 +749,13 @@ struct usb_device_stats {
u_long  uds_requests[4];/* indexed by transfer type 
UE_* */

 };

+#define USB_DEVICE_UGEN_FORCE_MAX 16
+struct usb_device_ugen_force {
+int udf_set;
+u_short udf_ven_id[USB_DEVICE_UGEN_FORCE_MAX];
+u_short udf_dev_id[USB_DEVICE_UGEN_FORCE_MAX];
+};
+
 /* USB controller */
 #define USB_REQUEST_IOWR('U', 1, struct usb_ctl_request)
 #define USB_SETDEBUG   _IOW ('U', 2, unsigned int)
@@ -758,6 +765,7 @@ struct usb_device_stats {
 #define USB_DEVICE_GET_CDESC   _IOWR('U', 6, struct usb_device_cdesc)
 #define USB_DEVICE_GET_FDESC   _IOWR('U', 7, struct usb_device_fdesc)
 #define USB_DEVICE_GET_DDESC   _IOWR('U', 8, struct usb_device_ddesc)
+#define USB_DEVICE_FORCE_UGEN   _IOWR('U', 9, struct
usb_device_ugen_force)

 /* Generic HID device */
 #define USB_GET_REPORT_DESC_IOR ('U', 21, struct 
usb_ctl_report_desc)

Index: usb_subr.c
===
RCS file: /cvs/src/sys/dev/usb/usb_subr.c,v
retrieving revision 1.117
diff -u -p -r1.117 usb_subr.c
--- usb_subr.c  23 Mar 2015 22:26:01 -  1.117
+++ usb_subr.c  6 Dec 2015 04:29:23 -
@@ -840,6 +840,7 @@ usbd_probe_and_attach(struct device *par
struct device *dv;
struct usbd_interface **ifaces;
extern struct rwlock usbpalock;
+   extern struct usb_device_ugen_force usb_udf;

rw_enter_write();

@@ -855,6 +856,14 @@ usbd_probe_and_attach(struct device *par
uaa.product = UGETW(dd->idProduct);
uaa.release = UGETW(dd->bcdDevice);

+   /* Check if this device is in the ugen force list */
+   for (i = 0; i < USB_DEVICE_UGEN_FORCE_MAX; i++)
+   if ((usb_udf.udf_ven_id[i] == uaa.vendor) &&
+  

Re: Overflowable int -> size_t in grep

[Michael McConville]

Michael McConville wrote:
> This isn't a grave issue, but I came across it while exploring integer
> overflow and think it's worth sharing.
> 
> grep represents line numbers with an int, which predictably overflows
> for inputs with >= 2^31 newlines. This is easy to demonstrate using the
> -n option and a debugging printf.
> 
> The below diff fixes this, and tunes up a for loop while I'm in there.
> 
> ok?

I may also have found a gcc bug in the process. When I build with:

> env CC=clang CFLAGS=' -fwrapv ' make

it aborts on overflow, as expected. However,

> env CFLAGS=' -fwrapv ' make

overflows withough complaint.

Am I missing something?

[patch] Convert modulus to arc4random_uniform

[Matthew Martin]

Theo's diff inspired me to look for other cases of modulo bias. The
following diff converts most modulus operations on a random number to
use arc4random_uniform or & as appropriate. I excluded

lib/libsqlite3/src/resolve.c
regress/lib/libevent/test-time.c
usr.sbin/nsd/rrl.c
usr.sbin/nsd/util.c
usr.sbin/nsd/xfrd.c

as they seem to have upstreams. The only other case is games/wump/wump.c
which has

if (arc4random() % 2 == 1)

This is safe and seems obvious enough to me.

- Matthew Martin


Index: games/atc/update.c
===
RCS file: /cvs/src/games/atc/update.c,v
retrieving revision 1.16
diff -u -p -r1.16 update.c
--- games/atc/update.c  9 Dec 2014 05:01:14 -   1.16
+++ games/atc/update.c  7 Dec 2015 06:42:17 -
@@ -59,6 +59,15 @@ atcrandom()
return arc4random();
 }
 
+uint32_t
+atcrandom_uniform(uint32_t upper_bound)
+{
+   if (seeded)
+   return random() % upper_bound;
+   else
+   return arc4random_uniform(upper_bound);
+}
+
 void
 update(int dummy)
 {
@@ -212,7 +221,7 @@ update(int dummy)
 * Otherwise, prop jobs show up *on* entrance.  Remember that
 * we don't update props on odd updates.
 */
-   if ((atcrandom() % sp->newplane_time) == 0)
+   if (atcrandom_uniform(sp->newplane_time) == 0)
addplane();
 }
 
@@ -308,10 +317,10 @@ addplane(void)
memset(, 0, sizeof (p));
 
p.status = S_MARKED;
-   p.plane_type = atcrandom() % 2;
+   p.plane_type = atcrandom_uniform(2);
 
num_starts = sp->num_exits + sp->num_airports;
-   rnd = atcrandom() % num_starts;
+   rnd = atcrandom_uniform(num_starts);
 
if (rnd < sp->num_exits) {
p.dest_type = T_EXIT;
@@ -324,7 +333,7 @@ addplane(void)
/* loop until we get a plane not near another */
for (i = 0; i < num_starts; i++) {
/* loop till we get a different start point */
-   while ((rnd2 = atcrandom() % num_starts) == rnd)
+   while ((rnd2 = atcrandom_uniform(num_starts)) == rnd)
;
if (rnd2 < sp->num_exits) {
p.orig_type = T_EXIT;
Index: games/hack/hack.mklev.c
===
RCS file: /cvs/src/games/hack/hack.mklev.c,v
retrieving revision 1.7
diff -u -p -r1.7 hack.mklev.c
--- games/hack/hack.mklev.c 27 Oct 2009 23:59:25 -  1.7
+++ games/hack/hack.mklev.c 7 Dec 2015 06:42:17 -
@@ -66,8 +66,8 @@
 #include 
 #include "hack.h"
 
-#define somex() ((random()%(croom->hx-croom->lx+1))+croom->lx)
-#define somey() ((random()%(croom->hy-croom->ly+1))+croom->ly)
+#define somex() (arc4random_uniform(croom->hx-croom->lx+1)+croom->lx)
+#define somey() (arc4random_uniform(croom->hy-croom->ly+1)+croom->ly)
 
 #defineXLIM4   /* define minimum required space around a room 
*/
 #defineYLIM3
Index: games/hack/rnd.c
===
RCS file: /cvs/src/games/hack/rnd.c,v
retrieving revision 1.5
diff -u -p -r1.5 rnd.c
--- games/hack/rnd.c27 Oct 2009 23:59:25 -  1.5
+++ games/hack/rnd.c7 Dec 2015 06:42:17 -
@@ -61,7 +61,7 @@
  * ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
  */
 
-#define RND(x) ((random()>>3) % x)
+#define RND(x) (arc4random_uniform(x))
 
 #include 
 
Index: lib/libc/stdlib/rand.c
===
RCS file: /cvs/src/lib/libc/stdlib/rand.c,v
retrieving revision 1.15
diff -u -p -r1.15 rand.c
--- lib/libc/stdlib/rand.c  13 Sep 2015 08:31:47 -  1.15
+++ lib/libc/stdlib/rand.c  7 Dec 2015 06:42:17 -
@@ -50,7 +50,7 @@ int
 rand(void)
 {
if (rand_deterministic == 0)
-   return (arc4random() % ((u_int)RAND_MAX + 1));
+   return (arc4random_uniform((u_int)RAND_MAX + 1));
return (rand_r());
 }
 
Index: sbin/dhclient/dhclient.c
===
RCS file: /cvs/src/sbin/dhclient/dhclient.c,v
retrieving revision 1.367
diff -u -p -r1.367 dhclient.c
--- sbin/dhclient/dhclient.c5 Dec 2015 13:09:11 -   1.367
+++ sbin/dhclient/dhclient.c7 Dec 2015 06:42:18 -
@@ -1285,15 +1285,13 @@ send_discover(void)
 */
if (!client->interval)
client->interval = config->initial_interval;
-   else {
-   client->interval += (arc4random() >> 2) %
-   (2 * client->interval);
-   }
+   else
+   client->interval += arc4random_uniform(2 * client->interval);
 
/* Don't backoff past cutoff. */
if (client->interval > config->backoff_cutoff)
-   client->interval = ((config->backoff_cutoff / 2) +
-   ((arc4random() >> 2) % config->backoff_cutoff));
+   client->interval = (config->backoff_cutoff / 

Re: Overflowable int -> size_t in grep

[Otto Moerbeek]

On Mon, Dec 07, 2015 at 01:36:22AM -0500, Michael McConville wrote:

> This isn't a grave issue, but I came across it while exploring integer
> overflow and think it's worth sharing.
> 
> grep represents line numbers with an int, which predictably overflows
> for inputs with >= 2^31 newlines. This is easy to demonstrate using the
> -n option and a debugging printf.
> 
> The below diff fixes this, and tunes up a for loop while I'm in there.

how does this fix work on 32-bit platforms? Better use offset_t. Also,
use %zu to print size_t's. The for loop change, I dunno. 

-Otto

> 
> ok?
> 
> 
> Index: grep.h
> ===
> RCS file: /cvs/src/usr.bin/grep/grep.h,v
> retrieving revision 1.22
> diff -u -p -r1.22 grep.h
> --- grep.h16 Mar 2015 13:26:52 -  1.22
> +++ grep.h7 Dec 2015 06:26:55 -
> @@ -43,7 +43,7 @@
>  
>  typedef struct {
>   size_t   len;
> - int  line_no;
> + size_t   line_no;
>   off_toff;
>   char*file;
>   char*dat;
> Index: util.c
> ===
> RCS file: /cvs/src/usr.bin/grep/util.c,v
> retrieving revision 1.50
> diff -u -p -r1.50 util.c
> --- util.c25 Jun 2015 02:04:08 -  1.50
> +++ util.c7 Dec 2015 06:26:55 -
> @@ -124,7 +124,7 @@ procfile(char *fn)
>  
>   if (Bflag > 0)
>   initqueue();
> - for (c = 0;  c == 0 || !(lflag || qflag); ) {
> + for (c = 0;  c == 0 || !(lflag || qflag); c += t) {
>   ln.off += ln.len + 1;
>   if ((ln.dat = grep_fgetln(f, )) == NULL)
>   break;
> @@ -138,7 +138,6 @@ procfile(char *fn)
>   enqueue();
>   linesqueued++;
>   }
> - c += t;
>   }
>   if (Bflag > 0)
>   clearqueue();
> @@ -623,7 +622,7 @@ printline(str_t *line, int sep, regmatch
>   if (nflag) {
>   if (n)
>   putchar(sep);
> - printf("%d", line->line_no);
> + printf("%llu", (unsigned long long)line->line_no);
>   ++n;
>   }
>   if (bflag) {

Overflowable int -> size_t in grep

[Michael McConville]

This isn't a grave issue, but I came across it while exploring integer
overflow and think it's worth sharing.

grep represents line numbers with an int, which predictably overflows
for inputs with >= 2^31 newlines. This is easy to demonstrate using the
-n option and a debugging printf.

The below diff fixes this, and tunes up a for loop while I'm in there.

ok?


Index: grep.h
===
RCS file: /cvs/src/usr.bin/grep/grep.h,v
retrieving revision 1.22
diff -u -p -r1.22 grep.h
--- grep.h  16 Mar 2015 13:26:52 -  1.22
+++ grep.h  7 Dec 2015 06:26:55 -
@@ -43,7 +43,7 @@
 
 typedef struct {
size_t   len;
-   int  line_no;
+   size_t   line_no;
off_toff;
char*file;
char*dat;
Index: util.c
===
RCS file: /cvs/src/usr.bin/grep/util.c,v
retrieving revision 1.50
diff -u -p -r1.50 util.c
--- util.c  25 Jun 2015 02:04:08 -  1.50
+++ util.c  7 Dec 2015 06:26:55 -
@@ -124,7 +124,7 @@ procfile(char *fn)
 
if (Bflag > 0)
initqueue();
-   for (c = 0;  c == 0 || !(lflag || qflag); ) {
+   for (c = 0;  c == 0 || !(lflag || qflag); c += t) {
ln.off += ln.len + 1;
if ((ln.dat = grep_fgetln(f, )) == NULL)
break;
@@ -138,7 +138,6 @@ procfile(char *fn)
enqueue();
linesqueued++;
}
-   c += t;
}
if (Bflag > 0)
clearqueue();
@@ -623,7 +622,7 @@ printline(str_t *line, int sep, regmatch
if (nflag) {
if (n)
putchar(sep);
-   printf("%d", line->line_no);
+   printf("%llu", (unsigned long long)line->line_no);
++n;
}
if (bflag) {

Re: Overflowable int -> size_t in grep

[Michael McConville]

Otto Moerbeek wrote:
> On Mon, Dec 07, 2015 at 01:36:22AM -0500, Michael McConville wrote:
> > This isn't a grave issue, but I came across it while exploring integer
> > overflow and think it's worth sharing.
> > 
> > grep represents line numbers with an int, which predictably overflows
> > for inputs with >= 2^31 newlines. This is easy to demonstrate using the
> > -n option and a debugging printf.
> > 
> > The below diff fixes this, and tunes up a for loop while I'm in there.
> 
> how does this fix work on 32-bit platforms? Better use offset_t.

Good catch, thanks.

Does this look better? Or is PRId64 preferred for off_t?


Index: grep.h
===
RCS file: /cvs/src/usr.bin/grep/grep.h,v
retrieving revision 1.22
diff -u -p -r1.22 grep.h
--- grep.h  16 Mar 2015 13:26:52 -  1.22
+++ grep.h  7 Dec 2015 07:30:52 -
@@ -43,7 +43,7 @@
 
 typedef struct {
size_t   len;
-   int  line_no;
+   off_tline_no;
off_toff;
char*file;
char*dat;
Index: util.c
===
RCS file: /cvs/src/usr.bin/grep/util.c,v
retrieving revision 1.50
diff -u -p -r1.50 util.c
--- util.c  25 Jun 2015 02:04:08 -  1.50
+++ util.c  7 Dec 2015 07:30:52 -
@@ -124,7 +124,7 @@ procfile(char *fn)
 
if (Bflag > 0)
initqueue();
-   for (c = 0;  c == 0 || !(lflag || qflag); ) {
+   for (c = 0;  c == 0 || !(lflag || qflag); c += t) {
ln.off += ln.len + 1;
if ((ln.dat = grep_fgetln(f, )) == NULL)
break;
@@ -138,7 +138,6 @@ procfile(char *fn)
enqueue();
linesqueued++;
}
-   c += t;
}
if (Bflag > 0)
clearqueue();
@@ -623,7 +622,7 @@ printline(str_t *line, int sep, regmatch
if (nflag) {
if (n)
putchar(sep);
-   printf("%d", line->line_no);
+   printf("%jd", line->line_no);
++n;
}
if (bflag) {

Re: [UPDATE] freetype-2.6.2

[Matthieu Herrb]

On Sun, Dec 06, 2015 at 01:44:46PM +0100, Christian Weisgerber wrote:
> David Coppa:
> 
> > Here's the update to freetype-2.6.2.
> > 
> > It shouldn't cause any fallout, but who knows with freetype... So
> > probably a ports bulk build can be useful.
> 
> I ran a bulk build with this on amd64.  Unfortunately, a few hundred
> packages couldn't be built due to unrelated fallout from the network
> hackathon, but there were no problems with freetype.
> 
Yes, I also rebuilt my usual set of packages (which include all the
low-level gtk+/kde libs) without problems.

David you can commit that update.
-- 
Matthieu Herrb


signature.asc
Description: PGP signature