Author: asomers
Date: Wed May 4 22:34:11 2016
New Revision: 299090
URL: https://svnweb.freebsd.org/changeset/base/299090
Log:
Improve performance and functionality of the bitstring(3) api
Two new functions are provided, bit_ffs_at() and bit_ffc_at(), which allow
for efficient searching of set or cleared bits starting from any bit offset
within the bit string.
Performance is improved by operating on longs instead of bytes and using
ffsl() for searches within a long. ffsl() is a compiler builtin in both
clang and gcc for most architectures, converting what was a brute force
while loop search into a couple of instructions.
All of the bitstring(3) API continues to be contained in the header file.
Some of the functions are large enough that perhaps they should be uninlined
and moved to a library, but that is beyond the scope of this commit.
sys/sys/bitstring.h:
Convert the majority of the existing bit string implementation from
macros to inline functions.
Properly protect the implementation from inadvertant macro expansion
when included in a user's program by prefixing all private
macros/functions and local variables with '_'.
Add bit_ffs_at() and bit_ffc_at(). Implement bit_ffs() and
bit_ffc() in terms of their "at" counterparts.
Provide a kernel implementation of bit_alloc(), making the full API
usable in the kernel.
Improve code documenation.
share/man/man3/bitstring.3:
Add pre-exisiting API bit_ffc() to the synopsis.
Document new APIs.
Document the initialization state of the bit strings
allocated/declared by bit_alloc() and bit_decl().
Correct documentation for bitstr_size(). The original code comments
indicate the size is in bytes, not "elements of bitstr_t". The new
implementation follows this lead. Only hastd assumed "elements"
rather than bytes and it has been corrected.
etc/mtree/BSD.tests.dist:
tests/sys/Makefile:
tests/sys/sys/Makefile:
tests/sys/sys/bitstring.c:
Add tests for all existing and new functionality.
include/bitstring.h
Include all headers needed by sys/bitstring.h
lib/libbluetooth/bluetooth.h:
usr.sbin/bluetooth/hccontrol/le.c:
Include bitstring.h instead of sys/bitstring.h.
sbin/hastd/activemap.c:
Correct usage of bitstr_size().
sys/dev/xen/blkback/blkback.c
Use new bit_alloc.
sys/kern/subr_unit.c:
Remove hard-coded assumption that sizeof(bitstr_t) is 1. Get rid of
unrb.busy, which caches the number of bits set in unrb.map. When
INVARIANTS are disabled, nothing needs to know that information.
callapse_unr can be adapted to use bit_ffs and bit_ffc instead.
Eliminating unrb.busy saves memory, simplifies the code, and
provides a slight speedup when INVARIANTS are disabled.
sys/net/flowtable.c:
Use the new kernel implementation of bit-alloc, instead of hacking
the old libc-dependent macro.
sys/sys/param.h
Update __FreeBSD_version to indicate availability of new API
Submitted by: gibbs, asomers
Reviewed by: gibbs, ngie
MFC after: 4 weeks
Sponsored by: Spectra Logic Corp
Differential Revision: https://reviews.freebsd.org/D6004
Added:
head/tests/sys/sys/
head/tests/sys/sys/Makefile (contents, props changed)
head/tests/sys/sys/bitstring_test.c (contents, props changed)
Modified:
head/etc/mtree/BSD.tests.dist
head/include/bitstring.h
head/lib/libbluetooth/bluetooth.h
head/sbin/hastd/activemap.c
head/share/man/man3/bitstring.3
head/sys/dev/xen/blkback/blkback.c
head/sys/kern/subr_unit.c
head/sys/net/flowtable.c
head/sys/sys/bitstring.h
head/sys/sys/param.h
head/tests/sys/Makefile
head/usr.sbin/bluetooth/hccontrol/le.c
Modified: head/etc/mtree/BSD.tests.dist
==============================================================================
--- head/etc/mtree/BSD.tests.dist Wed May 4 22:27:22 2016
(r299089)
+++ head/etc/mtree/BSD.tests.dist Wed May 4 22:34:11 2016
(r299090)
@@ -460,6 +460,8 @@
..
posixshm
..
+ sys
+ ..
vfs
..
vm
Modified: head/include/bitstring.h
==============================================================================
--- head/include/bitstring.h Wed May 4 22:27:22 2016 (r299089)
+++ head/include/bitstring.h Wed May 4 22:34:11 2016 (r299090)
@@ -29,6 +29,8 @@
#ifndef _BITSTRING_H_
#define _BITSTRING_H_
+#include <stdlib.h>
+#include <strings.h>
#include <sys/bitstring.h>
#endif /* _BITSTRING_H_ */
Modified: head/lib/libbluetooth/bluetooth.h
==============================================================================
--- head/lib/libbluetooth/bluetooth.h Wed May 4 22:27:22 2016
(r299089)
+++ head/lib/libbluetooth/bluetooth.h Wed May 4 22:34:11 2016
(r299090)
@@ -35,14 +35,16 @@
#define _BLUETOOTH_H_
#include <sys/types.h>
-#include <sys/bitstring.h>
#include <sys/endian.h>
#include <sys/ioctl.h>
#include <sys/socket.h>
#include <sys/uio.h>
#include <sys/un.h>
+
#include <errno.h>
#include <netdb.h>
+#include <bitstring.h>
+
#include <netgraph/ng_message.h>
#include <netgraph/bluetooth/include/ng_hci.h>
#include <netgraph/bluetooth/include/ng_l2cap.h>
Modified: head/sbin/hastd/activemap.c
==============================================================================
--- head/sbin/hastd/activemap.c Wed May 4 22:27:22 2016 (r299089)
+++ head/sbin/hastd/activemap.c Wed May 4 22:34:11 2016 (r299090)
@@ -162,7 +162,7 @@ activemap_init(struct activemap **ampp,
amp->am_extentsize = extentsize;
amp->am_extentshift = bitcount32(extentsize - 1);
amp->am_nextents = ((mediasize - 1) / extentsize) + 1;
- amp->am_mapsize = sizeof(bitstr_t) * bitstr_size(amp->am_nextents);
+ amp->am_mapsize = bitstr_size(amp->am_nextents);
amp->am_diskmapsize = roundup2(amp->am_mapsize, sectorsize);
amp->am_ndirty = 0;
amp->am_syncoff = -2;
@@ -552,7 +552,7 @@ activemap_calc_ondisk_size(uint64_t medi
PJDLOG_ASSERT(powerof2(sectorsize));
nextents = ((mediasize - 1) / extentsize) + 1;
- mapsize = sizeof(bitstr_t) * bitstr_size(nextents);
+ mapsize = bitstr_size(nextents);
return (roundup2(mapsize, sectorsize));
}
Modified: head/share/man/man3/bitstring.3
==============================================================================
--- head/share/man/man3/bitstring.3 Wed May 4 22:27:22 2016
(r299089)
+++ head/share/man/man3/bitstring.3 Wed May 4 22:34:11 2016
(r299090)
@@ -27,23 +27,54 @@
.\" OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
.\" SUCH DAMAGE.
.\"
+.\" Copyright (c) 2014 Spectra Logic Corporation
+.\" All rights reserved.
+.\"
+.\" Redistribution and use in source and binary forms, with or without
+.\" modification, are permitted provided that the following conditions
+.\" are met:
+.\" 1. Redistributions of source code must retain the above copyright
+.\" notice, this list of conditions, and the following disclaimer,
+.\" without modification.
+.\" 2. Redistributions in binary form must reproduce at minimum a disclaimer
+.\" substantially similar to the "NO WARRANTY" disclaimer below
+.\" ("Disclaimer") and any redistribution must be conditioned upon
+.\" including a substantially similar Disclaimer requirement for further
+.\" binary redistribution.
+.\"
+.\" NO WARRANTY
+.\" THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+.\" "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+.\" LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTIBILITY AND FITNESS FOR
+.\" A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+.\" HOLDERS OR CONTRIBUTORS BE LIABLE FOR SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
+.\" DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS
+.\" OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION)
+.\" HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT,
+.\" STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING
+.\" IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
+.\" POSSIBILITY OF SUCH DAMAGES.
+.\"
.\" @(#)bitstring.3 8.1 (Berkeley) 7/19/93
.\" $FreeBSD$
.\"
-.Dd October 17, 2015
+.Dd May 4, 2016
.Dt BITSTRING 3
.Os
.Sh NAME
.Nm bit_alloc ,
.Nm bit_clear ,
.Nm bit_decl ,
+.Nm bit_ffc ,
.Nm bit_ffs ,
+.Nm bit_ffc_at ,
+.Nm bit_ffs_at ,
.Nm bit_nclear ,
.Nm bit_nset ,
.Nm bit_set ,
-.Nm bitstr_size ,
-.Nm bit_test
-.Nd bit-string manipulation macros
+.Nm bit_test ,
+.Nm bitstr_size
+.Nd bit-string manipulation functions and macros
.Sh SYNOPSIS
.In bitstring.h
.Ft bitstr_t *
@@ -57,6 +88,10 @@
.Ft void
.Fn bit_ffs "bitstr_t *name" "int nbits" "int *value"
.Ft void
+.Fn bit_ffc_at "bitstr_t *name" "int start" "int nbits" "int *value"
+.Ft void
+.Fn bit_ffs_at "bitstr_t *name" "int start" "int nbits" "int *value"
+.Ft void
.Fn bit_nclear "bitstr_t *name" "int start" "int stop"
.Ft void
.Fn bit_nset "bitstr_t *name" "int start" "int stop"
@@ -69,7 +104,7 @@
.Sh DESCRIPTION
These macros operate on strings of bits.
.Pp
-The macro
+The function
.Fn bit_alloc
returns a pointer of type
.Dq Fa "bitstr_t *"
@@ -78,23 +113,31 @@ to sufficient space to store
bits, or
.Dv NULL
if no space is available.
+If successful, the returned bit string is initialized with all bits cleared.
.Pp
The macro
.Fn bit_decl
-allocates sufficient space to store
+declares a bit string with sufficient space to store
.Fa nbits
-bits on the stack.
+bits.
+.Fn bit_decl
+may be used to include statically sized bit strings in structure
+definitions or to create bit strings on the stack.
+Users of this macro are responsible for initialization of the bit string,
+typically via a global initialization of the containing struct or use of the
+.Fn bit_nset
+or
+.Fn bin_nclear
+functions.
.Pp
The macro
.Fn bitstr_size
-returns the number of elements of type
-.Fa bitstr_t
-necessary to store
+returns the number of bytes necessary to store
.Fa nbits
bits.
This is useful for copying bit strings.
.Pp
-The macros
+The functions
.Fn bit_clear
and
.Fn bit_set
@@ -107,7 +150,7 @@ The
.Fn bit_nset
and
.Fn bit_nclear
-macros
+functions
set or clear the zero-based numbered bits from
.Fa start
through
@@ -117,16 +160,28 @@ in the bit string
.Pp
The
.Fn bit_test
-macro
+function
evaluates to non-zero if the zero-based numbered bit
.Fa bit
of bit string
.Fa name
is set, and zero otherwise.
.Pp
+The function
+.Fn bit_ffc
+stores in the location referenced by
+.Fa value
+the zero-based number of the first bit not set in the array of
+.Fa nbits
+bits referenced by
+.Fa name .
+If all bits are set, the location referenced by
+.Fa value
+is set to \-1.
+.Pp
The
.Fn bit_ffs
-macro
+function
stores in the location referenced by
.Fa value
the zero-based number of the first bit set in the array of
@@ -137,19 +192,40 @@ If no bits are set, the location referen
.Fa value
is set to \-1.
.Pp
-The macro
-.Fn bit_ffc
+The function
+.Fn bit_ffc_at
stores in the location referenced by
.Fa value
the zero-based number of the first bit not set in the array of
.Fa nbits
bits referenced by
-.Fa name .
-If all bits are set, the location referenced by
+.Fa name ,
+at or after the zero-based bit index
+.Fa start .
+If all bits at or after
+.Fa start
+are set, the location referenced by
+.Fa value
+is set to \-1.
+.Pp
+The
+.Fn bit_ffs_at
+function
+stores in the location referenced by
+.Fa value
+the zero-based number of the first bit set in the array of
+.Fa nbits
+bits referenced by
+.Fa name ,
+at or after the zero-based bit index
+.Fa start .
+If no bits are set after
+.Fa start ,
+the location referenced by
.Fa value
is set to \-1.
.Pp
-The arguments to these macros are evaluated only once and may safely
+The arguments in bit string macros are evaluated only once and may safely
have side effects.
.Sh EXAMPLES
.Bd -literal -offset indent
Modified: head/sys/dev/xen/blkback/blkback.c
==============================================================================
--- head/sys/dev/xen/blkback/blkback.c Wed May 4 22:27:22 2016
(r299089)
+++ head/sys/dev/xen/blkback/blkback.c Wed May 4 22:34:11 2016
(r299090)
@@ -977,8 +977,8 @@ xbb_get_gntaddr(struct xbb_xen_reqlist *
static uint8_t *
xbb_get_kva(struct xbb_softc *xbb, int nr_pages)
{
- intptr_t first_clear;
- intptr_t num_clear;
+ int first_clear;
+ int num_clear;
uint8_t *free_kva;
int i;
@@ -1027,7 +1027,7 @@ xbb_get_kva(struct xbb_softc *xbb, int n
first_clear + nr_pages - 1);
free_kva = xbb->kva +
- (uint8_t *)(first_clear * PAGE_SIZE);
+ (uint8_t *)((intptr_t)first_clear * PAGE_SIZE);
KASSERT(free_kva >= (uint8_t *)xbb->kva &&
free_kva + (nr_pages * PAGE_SIZE) <=
@@ -2967,10 +2967,6 @@ xbb_connect_ring(struct xbb_softc *xbb)
return 0;
}
-/* Needed to make bit_alloc() macro work */
-#define calloc(count, size) malloc((count)*(size), M_XENBLOCKBACK,
\
- M_NOWAIT|M_ZERO);
-
/**
* Size KVA and pseudo-physical address allocations based on negotiated
* values for the size and number of I/O requests, and the size of our
@@ -2989,7 +2985,7 @@ xbb_alloc_communication_mem(struct xbb_s
xbb->kva_size = xbb->reqlist_kva_size +
(xbb->ring_config.ring_pages * PAGE_SIZE);
- xbb->kva_free = bit_alloc(xbb->reqlist_kva_pages);
+ xbb->kva_free = bit_alloc(xbb->reqlist_kva_pages, M_XENBLOCKBACK,
M_NOWAIT);
if (xbb->kva_free == NULL)
return (ENOMEM);
Modified: head/sys/kern/subr_unit.c
==============================================================================
--- head/sys/kern/subr_unit.c Wed May 4 22:27:22 2016 (r299089)
+++ head/sys/kern/subr_unit.c Wed May 4 22:34:11 2016 (r299090)
@@ -67,13 +67,13 @@
* N is the number of the highest unit allocated.
*/
+#include <sys/param.h>
#include <sys/types.h>
#include <sys/_unrhdr.h>
#ifdef _KERNEL
#include <sys/bitstring.h>
-#include <sys/param.h>
#include <sys/malloc.h>
#include <sys/kernel.h>
#include <sys/systm.h>
@@ -169,7 +169,7 @@ mtx_assert(struct mtx *mp, int flag)
* element:
* If ptr is NULL, it represents a run of free items.
* If ptr points to the unrhdr it represents a run of allocated items.
- * Otherwise it points to an bitstring of allocated items.
+ * Otherwise it points to a bitstring of allocated items.
*
* For runs the len field is the length of the run.
* For bitmaps the len field represents the number of allocated items.
@@ -183,14 +183,33 @@ struct unr {
};
struct unrb {
- u_char busy;
- bitstr_t map[sizeof(struct unr) - 1];
+ bitstr_t map[sizeof(struct unr) / sizeof(bitstr_t)];
};
-CTASSERT(sizeof(struct unr) == sizeof(struct unrb));
+CTASSERT((sizeof(struct unr) % sizeof(bitstr_t)) == 0);
+
+/* Number of bits we can store in the bitmap */
+#define NBITS (8 * sizeof(((struct unrb*)NULL)->map))
+
+/* Is the unrb empty in at least the first len bits? */
+static inline bool
+ub_empty(struct unrb *ub, int len) {
+ int first_set;
+
+ bit_ffs(ub->map, len, &first_set);
+ return (first_set == -1);
+}
+
+/* Is the unrb full? That is, is the number of set elements equal to len? */
+static inline bool
+ub_full(struct unrb *ub, int len)
+{
+ int first_clear;
+
+ bit_ffc(ub->map, len, &first_clear);
+ return (first_clear == -1);
+}
-/* Number of bits in the bitmap */
-#define NBITS ((int)sizeof(((struct unrb *)NULL)->map) * 8)
#if defined(DIAGNOSTIC) || !defined(_KERNEL)
/*
@@ -214,16 +233,13 @@ check_unrhdr(struct unrhdr *uh, int line
if (up->ptr != uh && up->ptr != NULL) {
ub = up->ptr;
KASSERT (up->len <= NBITS,
- ("UNR inconsistency: len %u max %d (line %d)\n",
+ ("UNR inconsistency: len %u max %zd (line %d)\n",
up->len, NBITS, line));
z++;
w = 0;
for (x = 0; x < up->len; x++)
if (bit_test(ub->map, x))
w++;
- KASSERT (w == ub->busy,
- ("UNR inconsistency: busy %u found %u (line %d)\n",
- ub->busy, w, line));
y += w;
} else if (up->ptr != NULL)
y += up->len;
@@ -239,7 +255,7 @@ check_unrhdr(struct unrhdr *uh, int line
#else
static __inline void
-check_unrhdr(struct unrhdr *uh, int line)
+check_unrhdr(struct unrhdr *uh __unused, int line __unused)
{
}
@@ -417,32 +433,24 @@ optimize_unr(struct unrhdr *uh)
a = us->len;
l = us->ptr == uh ? 1 : 0;
ub = (void *)us;
- ub->busy = 0;
- if (l) {
+ bit_nclear(ub->map, 0, NBITS - 1);
+ if (l)
bit_nset(ub->map, 0, a);
- ub->busy += a;
- } else {
- bit_nclear(ub->map, 0, a);
- }
if (!is_bitmap(uh, uf)) {
- if (uf->ptr == NULL) {
+ if (uf->ptr == NULL)
bit_nclear(ub->map, a, a + uf->len - 1);
- } else {
+ else
bit_nset(ub->map, a, a + uf->len - 1);
- ub->busy += uf->len;
- }
uf->ptr = ub;
uf->len += a;
us = uf;
} else {
ubf = uf->ptr;
for (l = 0; l < uf->len; l++, a++) {
- if (bit_test(ubf->map, l)) {
+ if (bit_test(ubf->map, l))
bit_set(ub->map, a);
- ub->busy++;
- } else {
+ else
bit_clear(ub->map, a);
- }
}
uf->len = a;
delete_unr(uh, uf->ptr);
@@ -464,19 +472,16 @@ optimize_unr(struct unrhdr *uh)
delete_unr(uh, uf);
} else if (uf->ptr == uh) {
bit_nset(ub->map, us->len, us->len + uf->len - 1);
- ub->busy += uf->len;
us->len += uf->len;
TAILQ_REMOVE(&uh->head, uf, list);
delete_unr(uh, uf);
} else {
ubf = uf->ptr;
for (l = 0; l < uf->len; l++, us->len++) {
- if (bit_test(ubf->map, l)) {
+ if (bit_test(ubf->map, l))
bit_set(ub->map, us->len);
- ub->busy++;
- } else {
+ else
bit_clear(ub->map, us->len);
- }
}
TAILQ_REMOVE(&uh->head, uf, list);
delete_unr(uh, ubf);
@@ -499,10 +504,10 @@ collapse_unr(struct unrhdr *uh, struct u
/* If bitmap is all set or clear, change it to runlength */
if (is_bitmap(uh, up)) {
ub = up->ptr;
- if (ub->busy == up->len) {
+ if (ub_full(ub, up->len)) {
delete_unr(uh, up->ptr);
up->ptr = uh;
- } else if (ub->busy == 0) {
+ } else if (ub_empty(ub, up->len)) {
delete_unr(uh, up->ptr);
up->ptr = NULL;
}
@@ -600,11 +605,9 @@ alloc_unrl(struct unrhdr *uh)
up->len--;
} else { /* bitmap */
ub = up->ptr;
- KASSERT(ub->busy < up->len, ("UNR bitmap confusion"));
bit_ffc(ub->map, up->len, &y);
KASSERT(y != -1, ("UNR corruption: No clear bit in bitmap."));
bit_set(ub->map, y);
- ub->busy++;
x += y;
}
uh->busy++;
@@ -688,7 +691,6 @@ alloc_unr_specificl(struct unrhdr *uh, u
ub = up->ptr;
if (bit_test(ub->map, i) == 0) {
bit_set(ub->map, i);
- ub->busy++;
goto done;
} else
return (-1);
@@ -807,7 +809,6 @@ free_unrl(struct unrhdr *uh, u_int item,
("UNR: Freeing free item %d (bitmap)\n", item));
bit_clear(ub->map, item);
uh->busy--;
- ub->busy--;
collapse_unr(uh, up);
return;
}
@@ -905,7 +906,7 @@ print_unr(struct unrhdr *uh, struct unr
printf("alloc\n");
else {
ub = up->ptr;
- printf("bitmap(%d) [", ub->busy);
+ printf("bitmap [");
for (x = 0; x < up->len; x++) {
if (bit_test(ub->map, x))
printf("#");
@@ -1025,7 +1026,7 @@ main(int argc, char **argv)
printf("sizeof(struct unr) %zu\n", sizeof(struct unr));
printf("sizeof(struct unrb) %zu\n", sizeof(struct unrb));
printf("sizeof(struct unrhdr) %zu\n", sizeof(struct unrhdr));
- printf("NBITS %d\n", NBITS);
+ printf("NBITS %lu\n", NBITS);
x = 1;
for (m = 0; m < count * reps; m++) {
j = random();
Modified: head/sys/net/flowtable.c
==============================================================================
--- head/sys/net/flowtable.c Wed May 4 22:27:22 2016 (r299089)
+++ head/sys/net/flowtable.c Wed May 4 22:34:11 2016 (r299090)
@@ -741,10 +741,6 @@ flowtable_lookup_common(struct flowtable
return (flowtable_insert(ft, hash, key, keylen, fibnum));
}
-/*
- * used by the bit_alloc macro
- */
-#define calloc(count, size) malloc((count)*(size), M_FTABLE, M_WAITOK | M_ZERO)
static void
flowtable_alloc(struct flowtable *ft)
{
@@ -759,11 +755,10 @@ flowtable_alloc(struct flowtable *ft)
bitstr_t **b;
b = zpcpu_get_cpu(ft->ft_masks, i);
- *b = bit_alloc(ft->ft_size);
+ *b = bit_alloc(ft->ft_size, M_FTABLE, M_WAITOK);
}
- ft->ft_tmpmask = bit_alloc(ft->ft_size);
+ ft->ft_tmpmask = bit_alloc(ft->ft_size, M_FTABLE, M_WAITOK);
}
-#undef calloc
static void
flowtable_free_stale(struct flowtable *ft, struct rtentry *rt, int maxidle)
Modified: head/sys/sys/bitstring.h
==============================================================================
--- head/sys/sys/bitstring.h Wed May 4 22:27:22 2016 (r299089)
+++ head/sys/sys/bitstring.h Wed May 4 22:34:11 2016 (r299090)
@@ -29,118 +29,231 @@
* OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
* SUCH DAMAGE.
*
+ * Copyright (c) 2014 Spectra Logic Corporation
+ * All rights reserved.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ * 1. Redistributions of source code must retain the above copyright
+ * notice, this list of conditions, and the following disclaimer,
+ * without modification.
+ * 2. Redistributions in binary form must reproduce at minimum a disclaimer
+ * substantially similar to the "NO WARRANTY" disclaimer below
+ * ("Disclaimer") and any redistribution must be conditioned upon
+ * including a substantially similar Disclaimer requirement for further
+ * binary redistribution.
+ *
+ * NO WARRANTY
+ * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+ * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+ * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTIBILITY AND FITNESS FOR
+ * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+ * HOLDERS OR CONTRIBUTORS BE LIABLE FOR SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
+ * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS
+ * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION)
+ * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT,
+ * STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING
+ * IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
+ * POSSIBILITY OF SUCH DAMAGES.
+ *
* $FreeBSD$
*/
-
#ifndef _SYS_BITSTRING_H_
#define _SYS_BITSTRING_H_
-typedef unsigned char bitstr_t;
-
-/* internal macros */
- /* byte of the bitstring bit is in */
-#define _bit_byte(bit) \
- ((bit) >> 3)
-
- /* mask for the bit within its byte */
-#define _bit_mask(bit) \
- (1 << ((bit)&0x7))
-
-/* external macros */
- /* bytes in a bitstring of nbits bits */
-#define bitstr_size(nbits) \
- (((nbits) + 7) >> 3)
-
- /* allocate a bitstring */
-#define bit_alloc(nbits) \
- (bitstr_t *)calloc((size_t)bitstr_size(nbits), sizeof(bitstr_t))
+#ifdef _KERNEL
+#include <sys/libkern.h>
+#include <sys/malloc.h>
+#endif
+
+typedef unsigned long bitstr_t;
+
+/*---------------------- Private Implementation Details
----------------------*/
+#define _BITSTR_MASK (~0UL)
+#define _BITSTR_BITS (sizeof(bitstr_t) * 8)
+
+/* bitstr_t in bit string containing the bit. */
+static inline int
+_bit_idx(int _bit)
+{
+ return (_bit / _BITSTR_BITS);
+}
+
+/* bit number within bitstr_t at _bit_idx(_bit). */
+static inline int
+_bit_offset(int _bit)
+{
+ return (_bit % _BITSTR_BITS);
+}
+
+/* Mask for the bit within its long. */
+static inline bitstr_t
+_bit_mask(int _bit)
+{
+ return (1UL << _bit_offset(_bit));
+}
+
+static inline bitstr_t
+_bit_make_mask(int _start, int _stop)
+{
+ return ((_BITSTR_MASK << _bit_offset(_start)) &
+ (_BITSTR_MASK >> (_BITSTR_BITS - _bit_offset(_stop) - 1)));
+}
+
+/*----------------------------- Public Interface
-----------------------------*/
+/* Number of bytes consumed by a bit string of nbits bits */
+#define bitstr_size(_nbits) \
+ (((_nbits) + _BITSTR_BITS - 1) / 8)
+
+/* Allocate a bit string initialized with no bits set. */
+#ifdef _KERNEL
+static inline bitstr_t *
+bit_alloc(int _nbits, struct malloc_type *type, int flags)
+{
+ return ((bitstr_t *)malloc(bitstr_size(_nbits), type, flags | M_ZERO));
+}
+#else
+static inline bitstr_t *
+bit_alloc(int _nbits)
+{
+ return ((bitstr_t *)calloc(bitstr_size(_nbits), 1));
+}
+#endif
- /* allocate a bitstring on the stack */
+/* Allocate a bit string on the stack with no bits set. */
#define bit_decl(name, nbits) \
- ((name)[bitstr_size(nbits)])
+ ((name)[bitstr_size(nbits) / sizeof(bitstr_t)])
- /* is bit N of bitstring name set? */
-#define bit_test(name, bit) \
- ((name)[_bit_byte(bit)] & _bit_mask(bit))
-
- /* set bit N of bitstring name */
-#define bit_set(name, bit) \
- ((name)[_bit_byte(bit)] |= _bit_mask(bit))
-
- /* clear bit N of bitstring name */
-#define bit_clear(name, bit) \
- ((name)[_bit_byte(bit)] &= ~_bit_mask(bit))
-
- /* clear bits start ... stop in bitstring */
-#define bit_nclear(name, start, stop) do { \
- register bitstr_t *_name = (name); \
- register int _start = (start), _stop = (stop); \
- register int _startbyte = _bit_byte(_start); \
- register int _stopbyte = _bit_byte(_stop); \
- if (_startbyte == _stopbyte) { \
- _name[_startbyte] &= ((0xff >> (8 - (_start&0x7))) | \
- (0xff << ((_stop&0x7) + 1))); \
- } else { \
- _name[_startbyte] &= 0xff >> (8 - (_start&0x7)); \
- while (++_startbyte < _stopbyte) \
- _name[_startbyte] = 0; \
- _name[_stopbyte] &= 0xff << ((_stop&0x7) + 1); \
- } \
-} while (0)
-
- /* set bits start ... stop in bitstring */
-#define bit_nset(name, start, stop) do { \
- register bitstr_t *_name = (name); \
- register int _start = (start), _stop = (stop); \
- register int _startbyte = _bit_byte(_start); \
- register int _stopbyte = _bit_byte(_stop); \
- if (_startbyte == _stopbyte) { \
- _name[_startbyte] |= ((0xff << (_start&0x7)) & \
- (0xff >> (7 - (_stop&0x7)))); \
- } else { \
- _name[_startbyte] |= 0xff << ((_start)&0x7); \
- while (++_startbyte < _stopbyte) \
- _name[_startbyte] = 0xff; \
- _name[_stopbyte] |= 0xff >> (7 - (_stop&0x7)); \
- } \
-} while (0)
-
- /* find first bit clear in name */
-#define bit_ffc(name, nbits, value) do { \
- register bitstr_t *_name = (name); \
- register int _byte, _nbits = (nbits); \
- register int _stopbyte = _bit_byte(_nbits - 1), _value = -1; \
- if (_nbits > 0) \
- for (_byte = 0; _byte <= _stopbyte; ++_byte) \
- if (_name[_byte] != 0xff) { \
- bitstr_t _lb; \
- _value = _byte << 3; \
- for (_lb = _name[_byte]; (_lb&0x1); \
- ++_value, _lb >>= 1); \
- break; \
- } \
- if (_value >= nbits) \
- _value = -1; \
- *(value) = _value; \
-} while (0)
-
- /* find first bit set in name */
-#define bit_ffs(name, nbits, value) do { \
- register bitstr_t *_name = (name); \
- register int _byte, _nbits = (nbits); \
- register int _stopbyte = _bit_byte(_nbits - 1), _value = -1; \
- if (_nbits > 0) \
- for (_byte = 0; _byte <= _stopbyte; ++_byte) \
- if (_name[_byte]) { \
- bitstr_t _lb; \
- _value = _byte << 3; \
- for (_lb = _name[_byte]; !(_lb&0x1); \
- ++_value, _lb >>= 1); \
- break; \
- } \
- if (_value >= nbits) \
- _value = -1; \
- *(value) = _value; \
-} while (0)
+/* Is bit N of bit string set? */
+static inline int
+bit_test(const bitstr_t *_bitstr, int _bit)
+{
+ return ((_bitstr[_bit_idx(_bit)] & _bit_mask(_bit)) != 0);
+}
+
+/* Set bit N of bit string. */
+static inline void
+bit_set(bitstr_t *_bitstr, int _bit)
+{
+ _bitstr[_bit_idx(_bit)] |= _bit_mask(_bit);
+}
+
+/* clear bit N of bit string name */
+static inline void
+bit_clear(bitstr_t *_bitstr, int _bit)
+{
+ _bitstr[_bit_idx(_bit)] &= ~_bit_mask(_bit);
+}
+
+/* Set bits start ... stop inclusive in bit string. */
+static inline void
+bit_nset(bitstr_t *_bitstr, int _start, int _stop)
+{
+ bitstr_t *_stopbitstr;
+
+ _stopbitstr = _bitstr + _bit_idx(_stop);
+ _bitstr += _bit_idx(_start);
+
+ if (_bitstr == _stopbitstr) {
+ *_bitstr |= _bit_make_mask(_start, _stop);
+ } else {
+ *_bitstr |= _bit_make_mask(_start, _BITSTR_BITS - 1);
+ while (++_bitstr < _stopbitstr)
+ *_bitstr = _BITSTR_MASK;
+ *_stopbitstr |= _bit_make_mask(0, _stop);
+ }
+}
+
+/* Clear bits start ... stop inclusive in bit string. */
+static inline void
+bit_nclear(bitstr_t *_bitstr, int _start, int _stop)
+{
+ bitstr_t *_stopbitstr;
+
+ _stopbitstr = _bitstr + _bit_idx(_stop);
+ _bitstr += _bit_idx(_start);
+
+ if (_bitstr == _stopbitstr) {
+ *_bitstr &= ~_bit_make_mask(_start, _stop);
+ } else {
+ *_bitstr &= ~_bit_make_mask(_start, _BITSTR_BITS - 1);
+ while (++_bitstr < _stopbitstr)
+ *_bitstr = 0;
+ *_stopbitstr &= ~_bit_make_mask(0, _stop);
+ }
+}
+
+/* Find the first bit set in bit string at or after bit start. */
+static inline void
+bit_ffs_at(bitstr_t *_bitstr, int _start, int _nbits, int *_result)
+{
+ bitstr_t *_curbitstr;
+ bitstr_t *_stopbitstr;
+ bitstr_t _test;
+ int _value, _offset;
+
+ if (_nbits > 0) {
+ _curbitstr = _bitstr + _bit_idx(_start);
+ _stopbitstr = _bitstr + _bit_idx(_nbits - 1);
+
+ _test = *_curbitstr;
+ if (_bit_offset(_start) != 0)
+ _test &= _bit_make_mask(_start, _BITSTR_BITS - 1);
+ while (_test == 0 && _curbitstr < _stopbitstr)
+ _test = *(++_curbitstr);
+
+ _offset = ffsl(_test);
+ _value = ((_curbitstr - _bitstr) * _BITSTR_BITS) + _offset - 1;
+ if (_offset == 0 || _value >= _nbits)
+ _value = -1;
+ } else {
+ _value = -1;
+ }
+ *_result = _value;
+}
+
+/* Find the first bit clear in bit string at or after bit start. */
+static inline void
+bit_ffc_at(bitstr_t *_bitstr, int _start, int _nbits, int *_result)
+{
+ bitstr_t *_curbitstr;
+ bitstr_t *_stopbitstr;
+ bitstr_t _test;
+ int _value, _offset;
+
+ if (_nbits > 0) {
+ _curbitstr = _bitstr + _bit_idx(_start);
+ _stopbitstr = _bitstr + _bit_idx(_nbits - 1);
+
+ _test = *_curbitstr;
+ if (_bit_offset(_start) != 0)
+ _test |= _bit_make_mask(0, _start - 1);
+ while (_test == _BITSTR_MASK && _curbitstr < _stopbitstr)
+ _test = *(++_curbitstr);
+
+ _offset = ffsl(~_test);
+ _value = ((_curbitstr - _bitstr) * _BITSTR_BITS) + _offset - 1;
+ if (_offset == 0 || _value >= _nbits)
+ _value = -1;
+ } else {
+ _value = -1;
+ }
+ *_result = _value;
+}
+
+/* Find the first bit set in bit string. */
+static inline void
+bit_ffs(bitstr_t *_bitstr, int _nbits, int *_result)
+{
+ bit_ffs_at(_bitstr, /*start*/0, _nbits, _result);
+}
+
+/* Find the first bit clear in bit string. */
+static inline void
+bit_ffc(bitstr_t *_bitstr, int _nbits, int *_result)
+{
+ bit_ffc_at(_bitstr, /*start*/0, _nbits, _result);
+}
-#endif /* !_SYS_BITSTRING_H_ */
+#endif /* _SYS_BITSTRING_H_ */
Modified: head/sys/sys/param.h
==============================================================================
--- head/sys/sys/param.h Wed May 4 22:27:22 2016 (r299089)
+++ head/sys/sys/param.h Wed May 4 22:34:11 2016 (r299090)
@@ -58,7 +58,7 @@
* in the range 5 to 9.
*/
#undef __FreeBSD_version
-#define __FreeBSD_version 1100106 /* Master, propagated to newvers */
+#define __FreeBSD_version 1100107 /* Master, propagated to newvers */
/*
* __FreeBSD_kernel__ indicates that this system uses the kernel of FreeBSD,
Modified: head/tests/sys/Makefile
==============================================================================
--- head/tests/sys/Makefile Wed May 4 22:27:22 2016 (r299089)
+++ head/tests/sys/Makefile Wed May 4 22:34:11 2016 (r299090)
@@ -19,6 +19,7 @@ TESTS_SUBDIRS+= mqueue
TESTS_SUBDIRS+= netinet
TESTS_SUBDIRS+= opencrypto
TESTS_SUBDIRS+= posixshm
+TESTS_SUBDIRS+= sys
TESTS_SUBDIRS+= vfs
TESTS_SUBDIRS+= vm
Added: head/tests/sys/sys/Makefile
==============================================================================
--- /dev/null 00:00:00 1970 (empty, because file is newly added)
+++ head/tests/sys/sys/Makefile Wed May 4 22:34:11 2016 (r299090)
@@ -0,0 +1,13 @@
+# $FreeBSD$
+
+PACKAGE= tests
+FILESGROUPS= TESTS
+TESTSPACKAGE= ${PACKAGE}
+
+TESTSDIR= ${TESTSBASE}/sys/sys
+
+ATF_TESTS_C= bitstring_test
+
+WARNS?= 5
+
+.include <bsd.test.mk>
Added: head/tests/sys/sys/bitstring_test.c
==============================================================================
--- /dev/null 00:00:00 1970 (empty, because file is newly added)
+++ head/tests/sys/sys/bitstring_test.c Wed May 4 22:34:11 2016
(r299090)
@@ -0,0 +1,359 @@
+/*-
+ * Copyright (c) 2014 Spectra Logic Corporation
+ * All rights reserved.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ * 1. Redistributions of source code must retain the above copyright
+ * notice, this list of conditions, and the following disclaimer,
+ * without modification.
+ * 2. Redistributions in binary form must reproduce at minimum a disclaimer
+ * substantially similar to the "NO WARRANTY" disclaimer below
+ * ("Disclaimer") and any redistribution must be conditioned upon
+ * including a substantially similar Disclaimer requirement for further
+ * binary redistribution.
+ *
+ * NO WARRANTY
+ * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+ * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+ * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTIBILITY AND FITNESS FOR
+ * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+ * HOLDERS OR CONTRIBUTORS BE LIABLE FOR SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
+ * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS
+ * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION)
+ * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT,
+ * STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING
+ * IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
+ * POSSIBILITY OF SUCH DAMAGES.
+ *
+ * $FreeBSD$
+ */
+#include <sys/param.h>
+
+#include <bitstring.h>
+#include <stdio.h>
+
+#include <atf-c.h>
+
+typedef void (testfunc_t)(bitstr_t *bstr, int nbits, const char *memloc);
+
+static void
+bitstring_run_stack_test(testfunc_t *test, int nbits)
+{
+ bitstr_t bit_decl(bitstr, nbits);
+
+ test(bitstr, nbits, "stack");
+}
+
+static void
+bitstring_run_heap_test(testfunc_t *test, int nbits)
+{
+ bitstr_t *bitstr = bit_alloc(nbits);
+
+ test(bitstr, nbits, "heap");
+}
+
+static void
+bitstring_test_runner(testfunc_t *test)
+{
+ const int bitstr_sizes[] = {
+ 0,
+ 1,
+ _BITSTR_BITS - 1,
+ _BITSTR_BITS,
+ _BITSTR_BITS + 1,
+ 2 * _BITSTR_BITS - 1,
+ 2 * _BITSTR_BITS,
+ 1023,
+ 1024
+ };
+
+ for (unsigned long i = 0; i < nitems(bitstr_sizes); i++) {
*** DIFF OUTPUT TRUNCATED AT 1000 LINES ***
_______________________________________________
svn-src-head@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"
