Module Name: othersrc Committed By: dyoung Date: Wed Dec 2 23:39:51 UTC 2015
Modified Files: othersrc/external/bsd/arfe/dt: Makefile README core.c core.h dec.c dec.h dt.c hex.c hex.h ipv4.c ipv4.h macaddr.c macaddr.h portability.h testit othersrc/external/bsd/arfe/dt/rr: daily-output-2-daily-output-5 daily-output-5-daily-output-2 daily-output-5-daily-output-6 daily-output-6-daily-output-5 daily-output-6-daily-output-7 daily-output-7-daily-output-6 if-re0-if-wm0 if-wm0-if-re0 ifconfig.0-ifconfig.1 ifconfig.0-logger-ifconfig.0 ifconfig.1-ifconfig.0 logger-ifconfig.0-ifconfig.0 netstat-s.0-netstat-s.1 netstat-s.1-netstat-s.0 quack123quack-quack456quack quack456quack-quack123quack quackabequack-quackdariaquack quackdariaquack-quackabequack wm0-a-wm0-b wm0-b-wm0-a othersrc/external/bsd/arfe/it: Makefile README it.c othersrc/external/bsd/arfe/tt: Makefile README tt.c othersrc/external/bsd/arfe/tt/tests/1: match transform Removed Files: othersrc/external/bsd/arfe/dt/t: abecedarian abracadabra quack123quack quack456quack quackabequack quackdariaquack Log Message: Executive summary: ARFE now understands C-like symbols. I'm using a different algorithm to figure out where to subdivide the longest common subsequence search. I'm actually computing an edit distance instead of the longest common sequence, but the algorithms are duals so there's not much practical difference. I've discarded some tests, and added at least one new one. Qualify %d for ptrdiff_t, %td. Quiets compilation on 64-bit Darwin. Change the type of the dynamic program cells from size_t to cell_t. For now, a cell_t is just a struct containing a size_t score. Add algq(), a routine for finding k such that lcs(A[1:m/2], B[1:k]) | lcs(A[m/2+1:m], B[k+1:n]) = lcs(A, B). algq() is based on the function Half(i, j) defined in Jeff Erickson's (je...@cs.illinois.edu) lecture notes on advanced dynamic programming. See http://jeffe.cs.illinois.edu/teaching/algorithms/. Add to the Makefile (commented out) lines for tracking code coverage. Use gcov <source file> to see the coverage. Make the cleandir target remove gcov(1)-related files. Delete a bunch of dead code and the now unnecessary argument to algc(), expected_lcs. We don't need backwards subslices any more, so get rid of that. When I got rid of the slice_t member `backward' and all of its uses, GCC inlined clocc_ends_at() with a really bad effect on performance (>10s on elmendorf for the t/netstat-s.[01] test, instead of <8s). I marked clocc_ends_at() __noinline for a net performance gain. The gcc version is (NetBSD nb2 20110806) 4.5.3, btw. Disable the dbg_assert()s for more reliable performance comparisons. Rename algq's splitn argument to splitnp since that's my convention for arguments of that kind. In algq(), don't get(A, i) m x n times, just get(A, i) m times. Lightly constify. Provide __noinline on non-NetBSD systems. Sprinkle the $ARFE$ keyword. Rename algc -> findlcs, algq -> findsplitn. Make Subversion fill $ARFE$ in macaddr.h. Extract the tags target from {dt,it,tt}/Makefile, put it in ./Makefile.inc. We only ever call findsplitn(..., true), so get rid of the do_clocc argument. Add an experimental routine, count_records(), that tries to count the records in its second slice_t argument, using the first slice_t argument as record template. Change the class-occurrence (clocc_t) score, clocc_score(), to one plus the minimum length of the class occurrences, from one plus the product of the class occurrences' lengths. This speeds things up a bit. Simplify findsplitn() by pulling common statements out from if-else branches, et cetera. In clocc_starts_in_slice_at(), pass the wlenp argument to clocc_starts_at(). Nothing passed a non-NULL wlenp to clocc_starts_in_slice_at(), so this doesn't make any functional difference. Compute the edit distance instead of the longest common subsequence. The one algorithm is a dual of the other. I may find it easier to add to the edit distance algorithm improvements like affine gap penalties, hence the change. Snapshot of work in progress. These changes make things quite a bit slower! Add affine gap penalties. Bring count_records() in line with findsplitn(), adding affine gap penalties. Update the instrumentation. Count up the number of gaps accumulated. XXX This change makes 'dt netstat-s.0 netstat-s.1' more than twice as XXX slow as it used to be, owing largely (I think) to the increase in XXX size of a cell_t, where three ssize_t's track the number of gaps. Exit with a message and error return code if we run out of slots for class occurrences. The class-occurrence array is still statically allocated---yech. I'm going to fix it one of these days, I promise. Stop detecting occurrences of class "string" (KIND_STRING), which consisted of the names 'abe', 'ada', and 'daria'. Remove the tests related to that. Start detecting occurrences of class "symbol" (KIND_SYMBOL), which resemble C symbol names: they start with a letter of the alphabet or underscore. Following characters are letters, numbers, or underscore. Update tests to match: the netstat and ifconfig tests produce much more sensible results, now. Delete the 'quack<number>quack' tests, since the symbol detector matches the entire string, now, and the tests don't stand for any practical use-case. Add test #5 to tt, which demonstrates how one can use a symbol in the match template to match a symbol in the input for reproduction in the transform template. To generate a diff of this commit: cvs rdiff -u -r1.6 -r1.7 othersrc/external/bsd/arfe/dt/Makefile \ othersrc/external/bsd/arfe/dt/hex.c othersrc/external/bsd/arfe/dt/hex.h \ othersrc/external/bsd/arfe/dt/ipv4.c othersrc/external/bsd/arfe/dt/ipv4.h cvs rdiff -u -r1.10 -r1.11 othersrc/external/bsd/arfe/dt/README cvs rdiff -u -r1.3 -r1.4 othersrc/external/bsd/arfe/dt/core.c \ othersrc/external/bsd/arfe/dt/core.h othersrc/external/bsd/arfe/dt/testit cvs rdiff -u -r1.1 -r1.2 othersrc/external/bsd/arfe/dt/dec.c \ othersrc/external/bsd/arfe/dt/dec.h \ othersrc/external/bsd/arfe/dt/macaddr.h cvs rdiff -u -r1.14 -r1.15 othersrc/external/bsd/arfe/dt/dt.c cvs rdiff -u -r1.4 -r1.5 othersrc/external/bsd/arfe/dt/macaddr.c cvs rdiff -u -r1.2 -r1.3 othersrc/external/bsd/arfe/dt/portability.h cvs rdiff -u -r1.2 -r1.3 \ othersrc/external/bsd/arfe/dt/rr/daily-output-2-daily-output-5 \ othersrc/external/bsd/arfe/dt/rr/daily-output-5-daily-output-2 \ othersrc/external/bsd/arfe/dt/rr/daily-output-5-daily-output-6 \ othersrc/external/bsd/arfe/dt/rr/daily-output-6-daily-output-5 \ othersrc/external/bsd/arfe/dt/rr/daily-output-6-daily-output-7 \ othersrc/external/bsd/arfe/dt/rr/daily-output-7-daily-output-6 cvs rdiff -u -r1.4 -r1.5 othersrc/external/bsd/arfe/dt/rr/if-re0-if-wm0 \ othersrc/external/bsd/arfe/dt/rr/if-wm0-if-re0 cvs rdiff -u -r1.5 -r1.6 \ othersrc/external/bsd/arfe/dt/rr/ifconfig.0-ifconfig.1 \ othersrc/external/bsd/arfe/dt/rr/ifconfig.0-logger-ifconfig.0 \ othersrc/external/bsd/arfe/dt/rr/ifconfig.1-ifconfig.0 \ othersrc/external/bsd/arfe/dt/rr/logger-ifconfig.0-ifconfig.0 \ othersrc/external/bsd/arfe/dt/rr/wm0-a-wm0-b \ othersrc/external/bsd/arfe/dt/rr/wm0-b-wm0-a cvs rdiff -u -r1.1 -r1.2 \ othersrc/external/bsd/arfe/dt/rr/netstat-s.0-netstat-s.1 \ othersrc/external/bsd/arfe/dt/rr/netstat-s.1-netstat-s.0 \ othersrc/external/bsd/arfe/dt/rr/quack123quack-quack456quack \ othersrc/external/bsd/arfe/dt/rr/quack456quack-quack123quack \ othersrc/external/bsd/arfe/dt/rr/quackabequack-quackdariaquack \ othersrc/external/bsd/arfe/dt/rr/quackdariaquack-quackabequack cvs rdiff -u -r1.1 -r0 othersrc/external/bsd/arfe/dt/t/abecedarian \ othersrc/external/bsd/arfe/dt/t/abracadabra \ othersrc/external/bsd/arfe/dt/t/quack123quack \ othersrc/external/bsd/arfe/dt/t/quack456quack \ othersrc/external/bsd/arfe/dt/t/quackabequack \ othersrc/external/bsd/arfe/dt/t/quackdariaquack cvs rdiff -u -r1.5 -r1.6 othersrc/external/bsd/arfe/it/Makefile cvs rdiff -u -r1.8 -r1.9 othersrc/external/bsd/arfe/it/README cvs rdiff -u -r1.1 -r1.2 othersrc/external/bsd/arfe/it/it.c cvs rdiff -u -r1.3 -r1.4 othersrc/external/bsd/arfe/tt/Makefile cvs rdiff -u -r1.5 -r1.6 othersrc/external/bsd/arfe/tt/README cvs rdiff -u -r1.2 -r1.3 othersrc/external/bsd/arfe/tt/tt.c cvs rdiff -u -r1.1 -r1.2 othersrc/external/bsd/arfe/tt/tests/1/match \ othersrc/external/bsd/arfe/tt/tests/1/transform Please note that diffs are not public domain; they are subject to the copyright notices on the relevant files.
Modified files: Index: othersrc/external/bsd/arfe/dt/Makefile diff -u othersrc/external/bsd/arfe/dt/Makefile:1.6 othersrc/external/bsd/arfe/dt/Makefile:1.7 --- othersrc/external/bsd/arfe/dt/Makefile:1.6 Thu Oct 8 22:00:56 2015 +++ othersrc/external/bsd/arfe/dt/Makefile Wed Dec 2 23:39:51 2015 @@ -1,12 +1,12 @@ -# $ARFE: Makefile 262 2015-10-08 21:47:24Z dyoung $ +# $ARFE: Makefile 294 2015-11-27 16:33:21Z dyoung $ NOMAN= .include <bsd.own.mk> -DPINCS+=core.h dec.h hex.h ipv4.h macaddr.h +DPINCS+=core.h dec.h hex.h ipv4.h macaddr.h sym.h PROG=dt -SRCS+=core.c dec.c dt.c hex.c ipv4.c macaddr.c +SRCS+=core.c dec.c dt.c hex.c ipv4.c macaddr.c sym.c #CPPFLAGS+=-DHB_DEBUG -CPPFLAGS+=-DHB_ASSERT +#CPPFLAGS+=-DHB_ASSERT DBG+=-g -O3 #DBG+=-g -O0 #COPTS+=-pg @@ -14,10 +14,13 @@ DBG+=-g -O3 CFLAGS+=-std=c99 WARNS=5 -tags: $(SRCS) $(DPINCS) $(INCS) - @rm -f $(.OBJDIR)/tags - @ctags -atwd $(.ALLSRC) - @sort -o $(.OBJDIR)/tags $(.OBJDIR)/tags +CLEANFILES+=*.gcda *.gcno + +# +# profiling stuff +# +#DBG+=-fprofile-arcs -ftest-coverage +#LDADD+=-lgcov test: @sh $(.CURDIR)/testit Index: othersrc/external/bsd/arfe/dt/hex.c diff -u othersrc/external/bsd/arfe/dt/hex.c:1.6 othersrc/external/bsd/arfe/dt/hex.c:1.7 --- othersrc/external/bsd/arfe/dt/hex.c:1.6 Wed Sep 23 19:32:34 2015 +++ othersrc/external/bsd/arfe/dt/hex.c Wed Dec 2 23:39:51 2015 @@ -1,5 +1,5 @@ -/* $NetBSD: hex.c,v 1.6 2015/09/23 19:32:34 dyoung Exp $ */ -/* $ARFE: hex.c 258 2015-09-23 19:31:17Z dyoung $ */ +/* $NetBSD: hex.c,v 1.7 2015/12/02 23:39:51 dyoung Exp $ */ +/* $ARFE: hex.c 264 2015-10-08 22:28:01Z dyoung $ */ /*- * Copyright (c) 2014,2015 David Young <dyo...@netbsd.org> Index: othersrc/external/bsd/arfe/dt/hex.h diff -u othersrc/external/bsd/arfe/dt/hex.h:1.6 othersrc/external/bsd/arfe/dt/hex.h:1.7 --- othersrc/external/bsd/arfe/dt/hex.h:1.6 Wed Sep 23 19:32:34 2015 +++ othersrc/external/bsd/arfe/dt/hex.h Wed Dec 2 23:39:51 2015 @@ -1,5 +1,5 @@ -/* $NetBSD: hex.h,v 1.6 2015/09/23 19:32:34 dyoung Exp $ */ -/* $ARFE: hex.h 258 2015-09-23 19:31:17Z dyoung $ */ +/* $NetBSD: hex.h,v 1.7 2015/12/02 23:39:51 dyoung Exp $ */ +/* $ARFE: hex.h 264 2015-10-08 22:28:01Z dyoung $ */ /*- * Copyright (c) 2014,2015 David Young <dyo...@netbsd.org> Index: othersrc/external/bsd/arfe/dt/ipv4.c diff -u othersrc/external/bsd/arfe/dt/ipv4.c:1.6 othersrc/external/bsd/arfe/dt/ipv4.c:1.7 --- othersrc/external/bsd/arfe/dt/ipv4.c:1.6 Wed Sep 23 19:32:34 2015 +++ othersrc/external/bsd/arfe/dt/ipv4.c Wed Dec 2 23:39:51 2015 @@ -1,5 +1,5 @@ -/* $NetBSD: ipv4.c,v 1.6 2015/09/23 19:32:34 dyoung Exp $ */ -/* $ARFE: ipv4.c 258 2015-09-23 19:31:17Z dyoung $ */ +/* $NetBSD: ipv4.c,v 1.7 2015/12/02 23:39:51 dyoung Exp $ */ +/* $ARFE: ipv4.c 264 2015-10-08 22:28:01Z dyoung $ */ /*- * Copyright (c) 2014,2015 David Young <dyo...@netbsd.org> Index: othersrc/external/bsd/arfe/dt/ipv4.h diff -u othersrc/external/bsd/arfe/dt/ipv4.h:1.6 othersrc/external/bsd/arfe/dt/ipv4.h:1.7 --- othersrc/external/bsd/arfe/dt/ipv4.h:1.6 Wed Sep 23 19:32:34 2015 +++ othersrc/external/bsd/arfe/dt/ipv4.h Wed Dec 2 23:39:51 2015 @@ -1,5 +1,5 @@ -/* $NetBSD: ipv4.h,v 1.6 2015/09/23 19:32:34 dyoung Exp $ */ -/* $ARFE: ipv4.h 258 2015-09-23 19:31:17Z dyoung $ */ +/* $NetBSD: ipv4.h,v 1.7 2015/12/02 23:39:51 dyoung Exp $ */ +/* $ARFE: ipv4.h 264 2015-10-08 22:28:01Z dyoung $ */ /*- * Copyright (c) 2014,2015 David Young <dyo...@netbsd.org> Index: othersrc/external/bsd/arfe/dt/README diff -u othersrc/external/bsd/arfe/dt/README:1.10 othersrc/external/bsd/arfe/dt/README:1.11 --- othersrc/external/bsd/arfe/dt/README:1.10 Wed Sep 23 19:32:34 2015 +++ othersrc/external/bsd/arfe/dt/README Wed Dec 2 23:39:51 2015 @@ -1,5 +1,5 @@ -$ARFE: README 258 2015-09-23 19:31:17Z dyoung $ -$NetBSD: README,v 1.10 2015/09/23 19:32:34 dyoung Exp $ +$ARFE: README 264 2015-10-08 22:28:01Z dyoung $ +$NetBSD: README,v 1.11 2015/12/02 23:39:51 dyoung Exp $ DT---(d)ifferentiate (t)ext---finds a longest common subsequence (LCS) of two texts where the numbers and IPv4 addresses are "wild": a span Index: othersrc/external/bsd/arfe/dt/core.c diff -u othersrc/external/bsd/arfe/dt/core.c:1.3 othersrc/external/bsd/arfe/dt/core.c:1.4 --- othersrc/external/bsd/arfe/dt/core.c:1.3 Thu Oct 8 22:00:56 2015 +++ othersrc/external/bsd/arfe/dt/core.c Wed Dec 2 23:39:51 2015 @@ -1,5 +1,5 @@ -/* $NetBSD: core.c,v 1.3 2015/10/08 22:00:56 dyoung Exp $ */ -/* $ARFE$ */ +/* $NetBSD: core.c,v 1.4 2015/12/02 23:39:51 dyoung Exp $ */ +/* $ARFE: core.c 297 2015-12-02 23:16:23Z dyoung $ */ /*- * Copyright (c) 2014,2015 David Young <dyo...@netbsd.org> @@ -46,6 +46,7 @@ #include "hex.h" #include "ipv4.h" #include "macaddr.h" +#include "sym.h" union freeslice; typedef union freeslice freeslice_t; @@ -55,18 +56,17 @@ union freeslice { freeslice_t *next; }; -static const char *names[] = { - "abe" - , "ada" - , "daria" -}; - -const size_t no_expected_lcs = SIZE_T_MAX; - static bool freeslices_initialized __aligned(COHERENCY_UNIT); static freeslice_t *first_freeslice __aligned(COHERENCY_UNIT); static freeslice_t freeslices[2048]; +static const int gap_open = 1, gap_extend = 6; +static const subcell_t infinity = { + .score = SSIZE_MAX / 2 + , .gaps = SSIZE_MAX / 2 + , .splitn = 0 +}; + static void init_freeslices(void) { @@ -111,8 +111,6 @@ newslice(char *buf, size_t buflen) if (s == NULL) return NULL; - s->backward = false; - s->first = buf; s->last = buf + buflen - 1; @@ -166,7 +164,7 @@ get(const slice_t *s, int i) { dbg_assert(0 <= i && i < length(s)); - return (s->backward) ? s->last[-i] : s->first[i]; + return s->first[i]; } static inline const char * @@ -174,25 +172,7 @@ getp(const slice_t *s, int i) { dbg_assert(0 <= i && i < length(s)); - return (s->backward) ? &s->last[-i] : &s->first[i]; -} - -static void -extract(const slice_t *s, unsigned int i, unsigned int j, char *o) -{ - unsigned int k, inc; - char *p; - - inc = (i < j) ? 1 : -1; - p = o; - - for (k = i; ; k += inc) { - *p++ = get(s, k); - if (k == j) - break; - } - - *p = '\0'; + return &s->first[i]; } static slice_t * @@ -201,6 +181,7 @@ subslice(const slice_t *s, int i, int j) slice_t *ss; const char *first, *last; + dbg_assert(i <= j); dbg_assert(MAX(i, j) < length(s)); ss = slice_alloc(); @@ -211,15 +192,8 @@ subslice(const slice_t *s, int i, int j) first = getp(s, i); last = getp(s, j); - ss->backward = first > last; - - if (ss->backward) { - ss->first = last; - ss->last = first; - } else { - ss->first = first; - ss->last = last; - } + ss->first = first; + ss->last = last; return ss; } @@ -254,7 +228,7 @@ found: return true; } -static bool +static bool __noinline clocc_ends_at(const cloccs_t *o, int last, ssize_t *wlenp, clocc_kind_t *kindp) { const clocc_t *occ; @@ -286,19 +260,16 @@ found: static int index_versus_basis(const slice_t *basis, const slice_t *s, int i) { - dbg_assert(!basis->backward); - return getp(s, i) - getp(basis, 0); } static bool clocc_starts_in_slice_at(const cloccs_t *o, const slice_t *basis, - const slice_t *s, int i, size_t *wlenp) + const slice_t *s, int i, ssize_t *wlenp) { const int iprime = index_versus_basis(basis, s, i); - return (s->backward) ? clocc_ends_at(o, iprime, NULL, NULL) - : clocc_starts_at(o, iprime, NULL, NULL); + return clocc_starts_at(o, iprime, wlenp, NULL); } static bool @@ -307,8 +278,7 @@ clocc_ends_in_slice_at(const cloccs_t *o { const int iprime = index_versus_basis(basis, s, i); - return (s->backward) ? clocc_starts_at(o, iprime, wlenp, kindp) - : clocc_ends_at(o, iprime, wlenp, kindp); + return clocc_ends_at(o, iprime, wlenp, kindp); } static chain_t * @@ -601,6 +571,20 @@ dec_clocc(const scratch_t *scratch, int } static chain_t * +sym_clocc(const scratch_t *scratch, int idx_a, int idx_b) +{ + const clocc_t * const a = &scratch->Aoccs.occs[idx_a], + * const b = &scratch->Boccs.occs[idx_b]; + const int column = cloccs_to_column(a, b); + + dbg2_printf("%s @ %d ; %s @ %d -> column %d\n", + a->val_u.u_sym, a->column, + b->val_u.u_sym, b->column, column); + + return newchain_with_match_idx(strdup(a->val_u.u_sym), column, idx_b); +} + +static chain_t * clocc(const scratch_t *scratch, int idx_a, int idx_b) { const clocc_t * const a = &scratch->Aoccs.occs[idx_a], @@ -614,6 +598,9 @@ clocc(const scratch_t *scratch, int idx_ (a->qual == b->qual) ? b->qual : QUAL_NONE); } + if (b->kind == KIND_SYM) + return sym_clocc(scratch, idx_a, idx_b); + if (b->kind == KIND_INTMAX_DEC) return dec_clocc(scratch, idx_a, idx_b); @@ -623,9 +610,6 @@ clocc(const scratch_t *scratch, int idx_ if (b->kind == KIND_MACADDR) return mac_clocc(scratch, idx_a, idx_b); - if (b->kind == KIND_STRING) - return newchain(strdup("<word>"), 0); - return newchain(strdup("<clocc>"), 0); } @@ -636,19 +620,21 @@ empty(void) } static inline ssize_t -clocc_score(ssize_t a, ssize_t b) +clocc_score(clocc_kind_t kind, ssize_t a, ssize_t b) { - return a * b + 1; + if (kind == KIND_SYM) + return gap_extend * (MAX(a, b) + MIN(a, b)) / 2; + return gap_extend * (MIN(a, b) - MAX(a, b)) - gap_open; } static void -algb(const slice_t *A, const slice_t *B, scratch_t *scratch, size_t *LL, - bool do_clocc) +findsplitn(const slice_t *A, const slice_t *B, scratch_t *scratch, int splitm, + unsigned int *splitnp) { const ssize_t m = length(A); const ssize_t n = length(B); ssize_t i, j; - size_t *K[2], *Kclocc, *Kcur, *Kprev, *Ktmp; + cell_t *K[2], *Kclocc, *Kcur, *Kprev, *Ktmp; struct { clocc_kind_t A, B; } kind; @@ -663,8 +649,12 @@ algb(const slice_t *A, const slice_t *B, /* A copy of the K[] at which a clocc last began. */ Kclocc = scratch->Kclocc; - for (j = 0; j < n + 1; j++) - K[0][j] = 0; + for (j = 0; j < n + 1; j++) { + K[0][j].match = infinity; + K[0][j].ins.score = gap_open + j * gap_extend; + K[0][j].ins.splitn = 0; + K[0][j].del = infinity; + } /* make -1 a valid index on all arrays */ Kprev = K[0] + 1; @@ -672,12 +662,262 @@ algb(const slice_t *A, const slice_t *B, Kclocc++; for (i = 0; i < m; i++) { - const bool clocc_starts_this_row = do_clocc && + Kprev[-1].match = infinity; + Kcur[-1].match = infinity; + Kprev[-1].del.score = gap_open + i * gap_extend; + Kcur[-1].del.score = gap_open + (i + 1) * gap_extend; + Kprev[-1].ins = infinity; + Kcur[-1].ins = infinity; + const bool clocc_starts_this_row = + clocc_starts_in_slice_at(&scratch->Aoccs, scratch->Abasis, + A, i, NULL); + + if (clocc_starts_this_row) { + dbg_assert(!inside_clocc); + dbg_printf("%s: row %zd, clocc starts\n", __func__, i); + memcpy(&Kclocc[-1], &Kprev[-1], + sizeof(Kclocc[0]) * (n + 1)); + inside_clocc = true; + } + + const bool clocc_ends_this_row = inside_clocc && + clocc_ends_in_slice_at(&scratch->Aoccs, scratch->Abasis, + A, i, &wlen.A, &kind.A); + + const char Ai = get(A, i); + + for (j = 0; j < n; j++) { + + struct { + ssize_t del, ins, least, match; + } cand, candw; + + /* del ^ + * ins < + */ + cand.match = + Kprev[j].match.score + gap_open + gap_extend; + cand.del = Kprev[j].del.score + gap_extend; + cand.ins = Kprev[j].ins.score + gap_open + gap_extend; + cand.least = MIN(cand.match, MIN(cand.del, cand.ins)); + + if (i < splitm) + Kcur[j].del.splitn = UINT_MAX; + else if (i == splitm) + Kcur[j].del.splitn = j + 1; + else if (cand.least == cand.ins) + Kcur[j].del.splitn = Kprev[j].ins.splitn; + else if (cand.least == cand.del) + Kcur[j].del.splitn = Kprev[j].del.splitn; + else /* if (cand.least == cand.match) */ + Kcur[j].del.splitn = Kprev[j].match.splitn; + + Kcur[j].del.score = cand.least; + dbg_printf("%s set K[%zd][%zd].del.score to %zd\n", + __func__, i, j, Kcur[j].del.score); + + cand.match = + Kcur[j - 1].match.score + gap_open + gap_extend; + cand.del = Kcur[j - 1].del.score + gap_open + gap_extend; + cand.ins = Kcur[j - 1].ins.score + gap_extend; + cand.least = MIN(cand.match, MIN(cand.del, cand.ins)); + + if (i < splitm) + Kcur[j].ins.splitn = UINT_MAX; + else if (i == splitm) + Kcur[j].ins.splitn = j + 1; + else if (cand.least == cand.ins) + Kcur[j].ins.splitn = Kcur[j - 1].ins.splitn; + else if (cand.least == cand.del) + Kcur[j].ins.splitn = Kcur[j - 1].del.splitn; + else /* if (cand.least == cand.match) */ + Kcur[j].ins.splitn = Kcur[j - 1].match.splitn; + + Kcur[j].ins.score = cand.least; + dbg_printf("%s set K[%zd][%zd].ins.score to %zd\n", + __func__, i, j, Kcur[j].ins.score); + + cand.match = Kprev[j - 1].match.score; + cand.del = Kprev[j - 1].del.score; + cand.ins = Kprev[j - 1].ins.score; + cand.least = + MIN(cand.match, MIN(cand.del, cand.ins)); + + if (Ai != get(B, j)) { + cand.match += gap_open + gap_extend * 2; + cand.del += gap_open + gap_extend * 2; + cand.ins += gap_open + gap_extend * 2; + cand.least += gap_open + gap_extend * 2; + } + + if (clocc_ends_this_row && + clocc_ends_in_slice_at(&scratch->Boccs, + scratch->Bbasis, B, j, + &wlen.B, &kind.B) && + kind.A == kind.B && /* match kind */ + wlen.B <= j + 1 /* must not begin outside of B */) { + const ssize_t step = + clocc_score(kind.A, wlen.A, wlen.B); + dbg_printf("%s visits clocc boundary at A[%d] " + "and B[%d] step %zu\n", __func__, + index_versus_basis(scratch->Abasis, A, i), + index_versus_basis(scratch->Bbasis, B, j), + step); + candw.match = + Kclocc[j - wlen.B].match.score + step; + candw.del = + Kclocc[j - wlen.B].del.score + step; + candw.ins = + Kclocc[j - wlen.B].ins.score + step; + candw.least = MIN(candw.match, + MIN(candw.del, candw.ins)); + } else { + candw = cand; + } + const ssize_t least = MIN(cand.least, candw.least); + if (i < splitm) + Kcur[j].match.splitn = UINT_MAX; + else if (i == splitm) + Kcur[j].match.splitn = j + 1; + else if (least == cand.ins) { + Kcur[j].match.splitn = Kprev[j - 1].ins.splitn; + } else if (least == cand.del) { + Kcur[j].match.splitn = Kprev[j - 1].del.splitn; + } else if (least == cand.match) { + Kcur[j].match.splitn = + Kprev[j - 1].match.splitn; + } else if (least == candw.ins) { + Kcur[j].match.splitn = + Kclocc[j - wlen.B].ins.splitn; + } else if (least == candw.del) { + Kcur[j].match.splitn = + Kclocc[j - wlen.B].del.splitn; + } else /* if (least == candw.match) */ { + Kcur[j].match.splitn = + Kclocc[j - wlen.B].match.splitn; + } + Kcur[j].match.score = least; + dbg_printf("%s set K[%zd][%zd].match.score to %zu%s\n", + __func__, i, j, Kcur[j].match.score, + inside_clocc ? "*" : ""); + } + if (clocc_ends_this_row) { + dbg_printf("%s: row %zd, clocc ends\n", __func__, i); + inside_clocc = false; + } + + Ktmp = Kprev; + Kprev = Kcur; + Kcur = Ktmp; + } +#if 0 + dbg_assert(Kcur[-1].score == m - 1 && Kprev[-1].score == m); +#endif + const cell_t *c = &Kprev[n - 1]; + const ssize_t least = + MIN(c->match.score, MIN(c->del.score, c->ins.score)); + if (least == c->ins.score) + *splitnp = Kprev[n - 1].ins.splitn; + else if (least == c->del.score) + *splitnp = Kprev[n - 1].del.splitn; + else /* if (least == c->match.score) */ + *splitnp = Kprev[n - 1].match.splitn; +} + +size_t +count_records(const slice_t *A, const slice_t *B, scratch_t *scratch) +{ + const ssize_t m = length(A); + const ssize_t n = length(B); + ssize_t i, i0, j; + cell_t *K[2], *Kclocc, *Kcur, *Kprev, *Ktmp; + struct { + clocc_kind_t A, B; + } kind; + struct { + ssize_t A, B; + } wlen; + bool inside_clocc = false; + + K[0] = scratch->K[0]; + K[1] = scratch->K[1]; + + /* A copy of the K[] at which a clocc last began. */ + Kclocc = scratch->Kclocc; + + for (j = 0; j < n + 1; j++) { + K[0][j].match = infinity; + + K[0][j].ins.score = gap_open + j * gap_extend; + K[0][j].ins.gaps = 1; + K[0][j].ins.splitn = 0; + + K[0][j].del = infinity; + } + + /* make -1 a valid index on all arrays */ + Kprev = K[0] + 1; + Kcur = K[1] + 1; + Kclocc++; + + ssize_t prev_gaps = SSIZE_MAX; + ssize_t prev_score = SSIZE_MAX; + ssize_t prev_diff = SSIZE_MAX; + ssize_t prev_avg = SSIZE_MAX; + ssize_t prev_avg_diff = 0; + size_t nrec = 0; + + for (i0 = i = 0; ; i0++, i++) { + Kprev[-1].match = infinity; + Kcur[-1].match = infinity; + /* TBD: i0, or i? */ + Kprev[-1].del.score = gap_open + i0 * gap_extend; + Kprev[-1].del.gaps = 1; + Kcur[-1].del.score = gap_open + (i0 + 1) * gap_extend; + Kcur[-1].del.gaps = 1; + Kprev[-1].ins = infinity; + Kcur[-1].ins = infinity; + + if (i == m) { + dbg_assert(!inside_clocc); + nrec++; + const ssize_t least = + MIN(MIN(Kprev[n - 1].del.score, + Kprev[n - 1].ins.score), + Kprev[n - 1].match.score); + ssize_t gaps; + int splitn; + if (least == Kprev[n - 1].del.score) { + splitn = Kprev[n - 1].del.splitn; + gaps = Kprev[n - 1].del.gaps; + } else if (least == Kprev[n - 1].ins.score) { + splitn = Kprev[n - 1].ins.splitn; + gaps = Kprev[n - 1].ins.gaps; + } else /* if (least == Kprev[n - 1].match.score) */ { + splitn = Kprev[n - 1].match.splitn; + gaps = Kprev[n - 1].match.gaps; + } +#if 1 + const ssize_t avg = gaps / nrec; + fprintf(stderr, "%s: %zu records, splitn %d, gaps %zu, d gaps %zd, score %zd -> %zd, " + "d %zd, dd %zd, avg g / r %zd\n", + __func__, nrec, splitn, gaps, gaps - prev_gaps, prev_score, least, + least - prev_score, least - prev_score - prev_diff, avg); +#endif + prev_diff = least - prev_score; + prev_score = least; + prev_avg_diff = avg - prev_avg; + prev_avg = avg; + prev_gaps = gaps; + i = 0; + } + + const bool clocc_starts_this_row = clocc_starts_in_slice_at(&scratch->Aoccs, scratch->Abasis, A, i, NULL); if (clocc_starts_this_row) { - assert(!inside_clocc); + dbg_assert(!inside_clocc); dbg_printf("%s: row %zd, clocc starts\n", __func__, i); memcpy(&Kclocc[-1], &Kprev[-1], sizeof(Kclocc[0]) * (n + 1)); @@ -688,12 +928,82 @@ algb(const slice_t *A, const slice_t *B, clocc_ends_in_slice_at(&scratch->Aoccs, scratch->Abasis, A, i, &wlen.A, &kind.A); + const char Ai = get(A, i); + for (j = 0; j < n; j++) { - size_t u = Kprev[j], ul = Kprev[j - 1], l = Kcur[j - 1]; + struct { + ssize_t del, ins, least, match; + } cand, candw; + + /* del ^ + * ins < + */ + cand.match = + Kprev[j].match.score + gap_open + gap_extend; + cand.del = Kprev[j].del.score + gap_extend; +#if 1 + if (i == 0) + cand.del += gap_open; +#endif + cand.ins = Kprev[j].ins.score + gap_open + gap_extend; + cand.least = MIN(cand.match, MIN(cand.del, cand.ins)); - if (get(A, i) == get(B, j)) - ul++; + if (cand.least == cand.ins) { + Kcur[j].del = Kprev[j].ins; + Kcur[j].del.gaps++; + } else if (cand.least == cand.del) { + Kcur[j].del = Kprev[j].del; +#if 1 + if (i == 0) + Kcur[j].del.gaps++; +#endif + } else /* if (cand.least == cand.match) */ { + Kcur[j].del = Kprev[j].match; + Kcur[j].del.gaps++; + } + + Kcur[j].del.score = cand.least; + + if (i == 0) + Kcur[j].del.splitn = j; + + dbg_printf("%s set K[%zd][%zd].del.score to %zd\n", + __func__, i, j, Kcur[j].del.score); + + cand.match = + Kcur[j - 1].match.score + gap_open + gap_extend; + cand.del = Kcur[j - 1].del.score + gap_open + gap_extend; + cand.ins = Kcur[j - 1].ins.score + gap_extend; + cand.least = MIN(cand.match, MIN(cand.del, cand.ins)); + + if (cand.least == cand.ins) { + Kcur[j].ins = Kcur[j - 1].ins; + } else if (cand.least == cand.del) { + Kcur[j].ins = Kcur[j - 1].del; + Kcur[j].ins.gaps++; + } else /* if (cand.least == cand.match) */ { + Kcur[j].ins = Kcur[j - 1].match; + Kcur[j].ins.gaps++; + } + + Kcur[j].ins.score = cand.least; + + dbg_printf("%s set K[%zd][%zd].ins.score to %zd\n", + __func__, i, j, Kcur[j].ins.score); + + cand.match = Kprev[j - 1].match.score; + cand.del = Kprev[j - 1].del.score; + cand.ins = Kprev[j - 1].ins.score; + cand.least = + MIN(cand.match, MIN(cand.del, cand.ins)); + + if (Ai != get(B, j)) { + cand.match += gap_open + gap_extend * 2; + cand.del += gap_open + gap_extend * 2; + cand.ins += gap_open + gap_extend * 2; + cand.least += gap_open + gap_extend * 2; + } if (clocc_ends_this_row && clocc_ends_in_slice_at(&scratch->Boccs, @@ -701,19 +1011,55 @@ algb(const slice_t *A, const slice_t *B, &wlen.B, &kind.B) && kind.A == kind.B && /* match kind */ wlen.B <= j + 1 /* must not begin outside of B */) { - const size_t step = clocc_score(wlen.A, wlen.B), - nscore = Kclocc[j - wlen.B] + step; + const ssize_t step = + clocc_score(kind.A, wlen.A, wlen.B); dbg_printf("%s visits clocc boundary at A[%d] " "and B[%d] step %zu\n", __func__, index_versus_basis(scratch->Abasis, A, i), index_versus_basis(scratch->Bbasis, B, j), step); - Kcur[j] = MAX(MAX(l, u), - MAX(ul, nscore)); - } else - Kcur[j] = MAX(MAX(u, ul), l); - dbg_printf("%s set K[%zd][%zd] to %zu%s\n", - __func__, i, j, Kcur[j], inside_clocc ? "*" : ""); + dbg_assert(i != 0); + candw.match = + Kclocc[j - wlen.B].match.score + step; + candw.del = + Kclocc[j - wlen.B].del.score + step; + candw.ins = + Kclocc[j - wlen.B].ins.score + step; + candw.least = MIN(candw.match, + MIN(candw.del, candw.ins)); + } else { + candw = cand; + } + const ssize_t least = MIN(cand.least, candw.least); + if (least == cand.ins) { + Kcur[j].match = Kprev[j - 1].ins; + if (i == 0) + Kcur[j].match.splitn = j - 1; + } else if (least == cand.del) { + Kcur[j].match = Kprev[j - 1].del; + if (i == 0) + Kcur[j].match.splitn = j - 1; + } else if (least == cand.match) { + Kcur[j].match = Kprev[j - 1].match; + if (i == 0) + Kcur[j].match.splitn = j - 1; + } else if (least == candw.ins) { + Kcur[j].match = Kclocc[j - wlen.B].ins; + if (i == 0) + Kcur[j].match.splitn = j - wlen.B; + } else if (least == candw.del) { + Kcur[j].match = Kclocc[j - wlen.B].del; + if (i == 0) + Kcur[j].match.splitn = j - wlen.B; + } else /* if (least == candw.match) */ { + Kcur[j].match = Kclocc[j - wlen.B].match; + if (i == 0) + Kcur[j].match.splitn = j - wlen.B; + } + Kcur[j].match.score = least; + dbg_printf("%s set K[%zd][%zd].match.score to %zu%s\n", + __func__, i, j, Kcur[j].match.score, + inside_clocc ? "*" : ""); } if (clocc_ends_this_row) { dbg_printf("%s: row %zd, clocc ends\n", __func__, i); @@ -724,10 +1070,6 @@ algb(const slice_t *A, const slice_t *B, Kprev = Kcur; Kcur = Ktmp; } - assert(Kprev[-1] == 0 && Kcur[-1] == 0 && Kclocc[-1] == 0); - Kprev--; - for (j = 0; j < n + 1; j++) - LL[j] = Kprev[j]; } static const clocc_t * @@ -853,7 +1195,7 @@ interval_to_clocc_index(cloccs_t *o, int } static inline chain_t * -algc_return(const size_t m, const size_t n, const origin_t origin, +findlcs_return(const size_t m, const size_t n, const origin_t origin, size_t *lcsp, size_t lcs, chain_t *c) { if (lcsp != NULL) { @@ -897,22 +1239,16 @@ joinchains(chain_t *ac, chain_t *bc) } chain_t * -algc(const slice_t *A, const slice_t *B, scratch_t *scratch, - const origin_t origin, const size_t expected_lcs, size_t *lcsp) +findlcs(const slice_t *A, const slice_t *B, scratch_t *scratch, + const origin_t origin, size_t *lcsp) { chain_t *c1, *c2, *c; - unsigned int j, k; + unsigned int splitn; const size_t m = length(A), n = length(B); struct { int A, B; } ivlidx; - struct { - size_t topl; - size_t botr; - size_t tot; - } lcs = {.tot = 0}; - const slice_t *B1n; - slice_t *A1i, *Amip1, *Bn1; + slice_t *A1i; slice_t *B1k, *Aip1m, *Bkp1n; dbg_printf("%s([%d, %zu], [%d, %zu])\n", __func__, @@ -920,8 +1256,7 @@ algc(const slice_t *A, const slice_t *B, origin.j, origin.j + n - 1); if (n == 0) { - assert(expected_lcs == no_expected_lcs || 0 == expected_lcs); - return algc_return(m, n, origin, lcsp, 0, empty()); + return findlcs_return(m, n, origin, lcsp, 0, empty()); } clocc_kind_t kind; @@ -936,106 +1271,43 @@ algc(const slice_t *A, const slice_t *B, 0, n - 1, &start, &end, kind); if (ivlidx.B != -1) { - dbg_assert(expected_lcs == no_expected_lcs || - (size_t)clocc_score(end - start + 1, m) == expected_lcs); - return algc_return(m, n, origin, lcsp, - clocc_score(end - start + 1, m), + return findlcs_return(m, n, origin, lcsp, + clocc_score(kind, end - start + 1, m), clocc(scratch, ivlidx.A, ivlidx.B)); } } if (m == 1) { - char A0 = get(A, 0); + const char A0 = get(A, 0); if (ispresent(B, A0)) { - dbg_assert(expected_lcs == no_expected_lcs || - 1 == expected_lcs); - return algc_return(m, n, origin, lcsp, 1, + return findlcs_return(m, n, origin, lcsp, -1, newchain(strndup(&A0, 1), 0)); } - assert(expected_lcs == no_expected_lcs || 0 == expected_lcs); - - return algc_return(m, n, origin, lcsp, 0, empty()); + return findlcs_return(m, n, origin, lcsp, 0, empty()); } const unsigned int i = split_interval(&scratch->Aoccs, origin.i, m); - A1i = subslice(A, 0, i - 1); - B1n = B; - Amip1 = subslice(A, m - 1, i); - Bn1 = subslice(B, n - 1, 0); - algb(A1i, B1n, scratch, scratch->L1, true); - dbg_printf("%s called algb once\n", __func__); - algb(Amip1, Bn1, scratch, scratch->L2, true); - dbg_printf("%s called algb twice\n", __func__); - - for (k = j = 0; j < n + 1; j++) { - if (scratch->L1[j] + scratch->L2[n - j] > lcs.tot) { - k = j; - lcs.topl = scratch->L1[j]; - lcs.botr = scratch->L2[n - j]; - lcs.tot = lcs.topl + lcs.botr; - } - } - dbg_printf("%s chose k %u (%zu - %u = %zu) lcs %zu\n", __func__, - k, n, k, n - k, lcs.tot); - - assert(k != 0 || lcs.topl == 0); - - if (expected_lcs != no_expected_lcs && lcs.tot != expected_lcs) { - struct { - char l[12], c, r[12]; - } actx = {.l = "12345.12345", .r = "12345.12345"}, - bctx = {.l = "12345.12345", .r = "12345.12345"}; - - dbg_printf("%s expected lcs %zu, got %zu + %zu = %zu at " - "[%d | %d + %u | %d + %zu, %d | %d + %u | %d + %zu]\n", - __func__, expected_lcs, lcs.topl, lcs.botr, lcs.tot, - origin.i, origin.i, i - 1, origin.i, m - 1, - origin.j, origin.j, k, origin.j, n - 1); - - extract(A, i - MIN(i, 6), i - MIN(i, 2), actx.l); - actx.c = get(A, i - 1); - extract(A, i, MIN(i + 4, m - 1), actx.r); - - dbg_printf("%s A context %s [ %c ] %s", - __func__, actx.l, actx.c, actx.r); - - if (k > 0) { - extract(B, k - MIN(k, 6), k - MIN(k, 2), bctx.l); - bctx.c = get(B, k - 1); - extract(B, MIN(k, n - 1), MIN(k + 4, n - 1), bctx.r); - - dbg_printf(", B context %s [ %c ] %s\n", - bctx.l, bctx.c, bctx.r); - } else - dbg_printf("\n"); - - dbg_printf("bailing\n"); - algb(A, B, scratch, scratch->L1, true); - const slice_t *Am1 = subslice(A, m - 1, 0); - algb(Am1, Bn1, scratch, scratch->L2, true); - - dbg_printf("algb(A, B) -> %zu fwd, -> %zu bwd\n", - scratch->L1[n], scratch->L2[n]); - abort(); - } + findsplitn(A, B, scratch, i - 1, &splitn); size_t lcs1, lcs2; - if (k >= 1) { - B1k = subslice(B, 0, k - 1); - c1 = algc(A1i, B1k, scratch, origin, lcs.topl, &lcs1); + if (splitn >= 1) { + B1k = subslice(B, 0, splitn - 1); + A1i = subslice(A, 0, i - 1); + c1 = findlcs(A1i, B1k, scratch, origin, &lcs1); + slice_free(A1i); slice_free(B1k); } else { lcs1 = 0; c1 = empty(); } - if (k < n) { + if (splitn < n) { Aip1m = subslice(A, i, m - 1); - Bkp1n = subslice(B, k, n - 1); - c2 = algc(Aip1m, Bkp1n, scratch, - (origin_t){origin.i + i, origin.j + k}, lcs.botr, &lcs2); + Bkp1n = subslice(B, splitn, n - 1); + c2 = findlcs(Aip1m, Bkp1n, scratch, + (origin_t){origin.i + i, origin.j + splitn}, &lcs2); slice_free(Bkp1n); slice_free(Aip1m); } else { @@ -1043,11 +1315,8 @@ algc(const slice_t *A, const slice_t *B, c2 = empty(); } - slice_free(A1i); - slice_free(Amip1); - slice_free(Bn1); c = joinchains(c1, c2); - return algc_return(m, n, origin, lcsp, lcs1 + lcs2, c); + return findlcs_return(m, n, origin, lcsp, lcs1 + lcs2, c); } void @@ -1060,9 +1329,9 @@ scratch_init(scratch_t *scratch, size_t scratch->Kclocc = malloc(sizeof(scratch->Kclocc[0]) * (n + 1)); scratch->Abasis = Abasis; scratch->Bbasis = Bbasis; - scratch->expected_lcs = SIZE_T_MAX; } +#if 0 static int slicestr(const slice_t *s, const char *t) { @@ -1081,19 +1350,51 @@ slicestr(const slice_t *s, const char *t } return -1; } +#endif + +static clocc_t * +newocc(cloccs_t *o) +{ + if (o->noccs >= (int)__arraycount(o->occs)) { + errno = ENOMEM; + return NULL; + } + return &o->occs[o->noccs++]; +} + +static void +emit_sym_clocc(sym_detection_t *d, void *arg) +{ + cloccs_t *o = arg; + clocc_t *occ; + + dbg_printf("found sym %s @ [%d, %d]\n", d->d_chars, + d->d_idx.start, d->d_idx.stop); + + if ((occ = newocc(o)) == NULL) + err(EXIT_FAILURE, "%s: newocc", __func__); + + occ->column = d->d_column.start; + occ->first = d->d_idx.start; + occ->last = d->d_idx.stop; + occ->kind = KIND_SYM; + occ->qual = QUAL_NONE; + strlcpy(&occ->val_u.u_sym[0], &d->d_chars[0], sizeof(occ->val_u.u_sym)); +} static void emit_mac_clocc(mac_detection_t *d, void *arg) { cloccs_t *o = arg; + clocc_t *occ; uint8_t addr[6]; int rc; dbg_printf("found mac string %s @ [%d, %d]\n", d->d_string, d->d_idx.start, d->d_idx.stop); - if (o->noccs >= (int)__arraycount(o->occs)) - return; + if ((occ = newocc(o)) == NULL) + err(EXIT_FAILURE, "%s: newocc", __func__); rc = sscanf(d->d_string, "%" SCNx8 ":%" SCNx8 ":%" SCNx8 ":%" SCNx8 ":%" SCNx8 ":%" SCNx8 "", @@ -1107,20 +1408,20 @@ emit_mac_clocc(mac_detection_t *d, void d->d_string, d->d_idx.start, d->d_idx.stop, addr[0], addr[1], addr[2], addr[3], addr[4], addr[5]); - o->occs[o->noccs].column = d->d_column.start; - o->occs[o->noccs].first = d->d_idx.start; - o->occs[o->noccs].last = d->d_idx.stop; - o->occs[o->noccs].kind = KIND_MACADDR; - o->occs[o->noccs].qual = QUAL_NONE; - memcpy(&o->occs[o->noccs].val_u.u_macaddr[0], &addr[0], - sizeof(o->occs[o->noccs].val_u.u_macaddr)); - o->noccs++; + occ->column = d->d_column.start; + occ->first = d->d_idx.start; + occ->last = d->d_idx.stop; + occ->kind = KIND_MACADDR; + occ->qual = QUAL_NONE; + memcpy(&occ->val_u.u_macaddr[0], &addr[0], + sizeof(occ->val_u.u_macaddr)); } static void emit_ipv4_clocc(ipv4_detection_t *d, void *arg) { cloccs_t *o = arg; + clocc_t *occ; struct { uint32_t net; uint32_t host; @@ -1130,8 +1431,8 @@ emit_ipv4_clocc(ipv4_detection_t *d, voi dbg_printf("found ipv4 string %s @ [%d, %d]\n", d->d_string, d->d_idx.start, d->d_idx.stop); - if (o->noccs >= (int)__arraycount(o->occs)) - return; + if ((occ = newocc(o)) == NULL) + err(EXIT_FAILURE, "%s: newocc", __func__); rc = inet_net_pton(AF_INET, d->d_string, &addr.net, sizeof(addr.net)); @@ -1142,19 +1443,19 @@ emit_ipv4_clocc(ipv4_detection_t *d, voi dbg_printf("converted ipv4 string %s @ [%d, %d] -> %08" PRIx32 "\n", d->d_string, d->d_idx.start, d->d_idx.stop, addr.host); - o->occs[o->noccs].column = d->d_column.start; - o->occs[o->noccs].first = d->d_idx.start; - o->occs[o->noccs].last = d->d_idx.stop; - o->occs[o->noccs].kind = KIND_IPv4; - o->occs[o->noccs].qual = QUAL_NONE; - o->occs[o->noccs].val_u.u_ipv4 = addr.host; - o->noccs++; + occ->column = d->d_column.start; + occ->first = d->d_idx.start; + occ->last = d->d_idx.stop; + occ->kind = KIND_IPv4; + occ->qual = QUAL_NONE; + occ->val_u.u_ipv4 = addr.host; } static void emit_dec_clocc(dec_detection_t *d, void *arg) { cloccs_t *o = arg; + clocc_t *occ; intmax_t val, ropval = 0; char *end, *ropend; unsigned int log10denom = 0; @@ -1165,8 +1466,8 @@ emit_dec_clocc(dec_detection_t *d, void dbg_printf("found decimal string %s @ [%d, %d]\n", d->d_digits, d->d_idx.start, d->d_idx.stop); - if (o->noccs >= (int)__arraycount(o->occs)) - return; + if ((occ = newocc(o)) == NULL) + err(EXIT_FAILURE, "%s: newocc", __func__); if (d->d_digits[0] == '-') { next = &d->d_digits[1]; @@ -1192,7 +1493,7 @@ emit_dec_clocc(dec_detection_t *d, void ropval = strtoimax(end, &ropend, 10); if (*ropend != '\0' && errno == ERANGE) { - warnx("%s: over/underflow at %d", __func__, + warnx("%s: over/underflow at %td", __func__, d->d_column.start + end - d->d_digits); } for (; end < ropend; end++) { @@ -1201,20 +1502,20 @@ emit_dec_clocc(dec_detection_t *d, void } } - o->occs[o->noccs].column = d->d_column.start; - o->occs[o->noccs].first = d->d_idx.start; - o->occs[o->noccs].last = d->d_idx.stop; - o->occs[o->noccs].kind = KIND_INTMAX_DEC; - o->occs[o->noccs].qual = QUAL_NONE; - o->occs[o->noccs].val_u.u_dec.num = sign * (val * denom + ropval); - o->occs[o->noccs].val_u.u_dec.log10denom = log10denom; - o->noccs++; + occ->column = d->d_column.start; + occ->first = d->d_idx.start; + occ->last = d->d_idx.stop; + occ->kind = KIND_INTMAX_DEC; + occ->qual = QUAL_NONE; + occ->val_u.u_dec.num = sign * (val * denom + ropval); + occ->val_u.u_dec.log10denom = log10denom; } static void emit_hex_clocc(hex_detection_t *d, void *arg, bool use0x) { cloccs_t *o = arg; + clocc_t *occ; intmax_t val; char *end; bool allcaps; @@ -1227,21 +1528,20 @@ emit_hex_clocc(hex_detection_t *d, void dbg_printf("found hex string %s @ [%d, %d]%s\n", d->d_digits, d->d_idx.start, d->d_idx.stop, allcaps ? " all CAPS" : ""); - if (o->noccs >= (int)__arraycount(o->occs)) - return; + if ((occ = newocc(o)) == NULL) + err(EXIT_FAILURE, "%s: newocc", __func__); val = strtoimax(d->d_digits, &end, 16); if (*end != '\0' && errno == ERANGE) warnx("%s: over/underflow at %d", __func__, d->d_column.start); - o->occs[o->noccs].column = d->d_column.start; - o->occs[o->noccs].first = d->d_idx.start; - o->occs[o->noccs].last = d->d_idx.stop; - o->occs[o->noccs].kind = use0x ? KIND_INTMAX_0xHEX : KIND_INTMAX_HEX; - o->occs[o->noccs].qual = allcaps ? QUAL_ALLCAPS : QUAL_NONE; - o->occs[o->noccs].val_u.u_uintmax = val; - o->noccs++; + occ->column = d->d_column.start; + occ->first = d->d_idx.start; + occ->last = d->d_idx.stop; + occ->kind = use0x ? KIND_INTMAX_0xHEX : KIND_INTMAX_HEX; + occ->qual = allcaps ? QUAL_ALLCAPS : QUAL_NONE; + occ->val_u.u_uintmax = val; } static int @@ -1259,6 +1559,12 @@ clocc_width(const clocc_t *c) return c->last - c->first + 1; } +static int +clocc_prio(const clocc_t *c) +{ + return (c->kind == KIND_SYM) ? 0 : 1; +} + static void cloccs_dedup(cloccs_t *o) { @@ -1274,6 +1580,9 @@ cloccs_dedup(cloccs_t *o) } if (clocc_width(r) > clocc_width(l)) *l = *r; + else if (clocc_width(l) <= clocc_width(r) && + clocc_prio(r) > clocc_prio(l)) + *l = *r; for (j = i + 1; j < o->noccs; j++) o->occs[j - 1] = o->occs[j]; o->noccs--; @@ -1312,28 +1621,33 @@ cloccs_init(cloccs_t *o, const slice_t * hex_parser_t *hex_parser; ipv4_parser_t *ipv4_parser; mac_parser_t *mac_parser; - size_t column, i; + sym_parser_t *sym_parser; + size_t column; size_t n = length(s); unsigned int j; + o->noccs = 0; clocc_htbl_init(&o->htbl); - o->noccs = 0; - for (i = 0; i < __arraycount(names); i++) { - int loc; + sym_parser = sym_parser_alloc(emit_sym_clocc, o); + if (sym_parser == NULL) + goto parse_mac; - loc = slicestr(s, names[i]); - if (loc == -1) - continue; + column = 0; + for (j = 0; j < n; j++) { + char c; - o->occs[o->noccs].first = loc; - o->occs[o->noccs].last = loc + strlen(names[i]) - 1; - o->occs[o->noccs].kind = KIND_STRING; - dbg_printf("found %s at %d - %d\n", - names[i], o->occs[o->noccs].first, o->occs[o->noccs].last); - o->noccs++; + c = get(s, j); + ++column; + sym_parser_drive(sym_parser, j, column, c); + if (c == '\n') + column = 0; } + sym_parser_drive(sym_parser, n, column + 1, -1); + + sym_parser_free(sym_parser); +parse_mac: mac_parser = mac_parser_alloc(emit_mac_clocc, o); if (mac_parser == NULL) goto parse_ipv4; Index: othersrc/external/bsd/arfe/dt/core.h diff -u othersrc/external/bsd/arfe/dt/core.h:1.3 othersrc/external/bsd/arfe/dt/core.h:1.4 --- othersrc/external/bsd/arfe/dt/core.h:1.3 Thu Oct 8 22:00:56 2015 +++ othersrc/external/bsd/arfe/dt/core.h Wed Dec 2 23:39:51 2015 @@ -1,5 +1,5 @@ -/* $NetBSD: core.h,v 1.3 2015/10/08 22:00:56 dyoung Exp $ */ -/* $ARFE: core.h 262 2015-10-08 21:47:24Z dyoung $ */ +/* $NetBSD: core.h,v 1.4 2015/12/02 23:39:51 dyoung Exp $ */ +/* $ARFE: core.h 297 2015-12-02 23:16:23Z dyoung $ */ /*- * Copyright (c) 2014,2015 David Young <dyo...@netbsd.org> @@ -92,7 +92,6 @@ typedef struct origin { typedef struct slice { const char *first; const char *last; - bool backward; } slice_t; typedef enum { @@ -102,7 +101,7 @@ typedef enum { , KIND_INTMAX_0xHEX , KIND_IPv4 , KIND_MACADDR - , KIND_STRING + , KIND_SYM } clocc_kind_t; typedef enum { @@ -131,6 +130,7 @@ typedef struct clocc { uintmax_t u_uintmax; uint32_t u_ipv4; uint8_t u_macaddr[6]; + char u_sym[64]; /* XXX don't limit like this */ } val_u; TAILQ_ENTRY(clocc) bucket; chainelt_t *match; @@ -161,23 +161,32 @@ enum { }; typedef struct clocc_htbl { - clocc_bucket_t bucket[1024]; + clocc_bucket_t bucket[4096]; } clocc_htbl_t; typedef struct cloccs { int noccs; - clocc_t occs[1024]; + clocc_t occs[4096]; clocc_htbl_t htbl; } cloccs_t; +typedef struct subcell { + ssize_t score; + int splitn; + ssize_t gaps; +} subcell_t; + +typedef struct cell { + subcell_t ins, del, match; +} cell_t; + typedef struct scratch { size_t *L1; /* n + 1 */ size_t *L2; /* n + 1 */ - size_t *K[2]; /* 2 x (n + 1) */ - size_t *Kclocc; /* n + 1 */ + cell_t *K[2]; /* 2 x (n + 1) */ + cell_t *Kclocc; /* n + 1 */ cloccs_t Aoccs, Boccs; slice_t *Abasis, *Bbasis; - size_t expected_lcs; char ipv4_op; /* 'p' for prefix (smallest containing subnet), * 'l' for copy left argument */ @@ -192,17 +201,16 @@ typedef struct scratch { */ } scratch_t; -extern const size_t no_expected_lcs; - slice_t *file_to_slice(const char *); void emit_transformed_text(chain_t *, cloccs_t *, cloccs_t *, const slice_t *, const slice_t *); void printchain(chain_t *); void cloccs_init(cloccs_t *, const slice_t *); void cloccs_hash(cloccs_t *, const slice_t *); -chain_t *algc(const slice_t *, const slice_t *, scratch_t *, const origin_t, - const size_t, size_t *); +chain_t *findlcs(const slice_t *, const slice_t *, scratch_t *, const origin_t, + size_t *); void scratch_init(scratch_t *, size_t, slice_t *, slice_t *); ssize_t length(const slice_t *); +size_t count_records(const slice_t *, const slice_t *, scratch_t *); #endif /* _CORE_H_ */ Index: othersrc/external/bsd/arfe/dt/testit diff -u othersrc/external/bsd/arfe/dt/testit:1.3 othersrc/external/bsd/arfe/dt/testit:1.4 --- othersrc/external/bsd/arfe/dt/testit:1.3 Fri Sep 11 01:50:42 2015 +++ othersrc/external/bsd/arfe/dt/testit Wed Dec 2 23:39:51 2015 @@ -1,5 +1,5 @@ #!/bin/sh -# $ARFE: testit 239 2015-09-10 22:49:40Z dyoung $ +# $ARFE: testit 297 2015-12-02 23:16:23Z dyoung $ DT=${OBJDIR:-.}/dt @@ -19,8 +19,6 @@ while read l r ; do done <<EOF netstat-s.0 netstat-s.1 netstat-s.1 netstat-s.0 -abecedarian abracadabra -abracadabra abecedarian daily-output-2 daily-output-5 daily-output-5 daily-output-2 daily-output-5 daily-output-6 @@ -31,10 +29,6 @@ ifconfig.0 ifconfig.1 ifconfig.1 ifconfig.0 logger-ifconfig.0 ifconfig.0 ifconfig.0 logger-ifconfig.0 -quack123quack quack456quack -quack456quack quack123quack -quackabequack quackdariaquack -quackdariaquack quackabequack wm0-a wm0-b wm0-b wm0-a if-re0 if-wm0 Index: othersrc/external/bsd/arfe/dt/dec.c diff -u othersrc/external/bsd/arfe/dt/dec.c:1.1 othersrc/external/bsd/arfe/dt/dec.c:1.2 --- othersrc/external/bsd/arfe/dt/dec.c:1.1 Thu Oct 8 22:00:56 2015 +++ othersrc/external/bsd/arfe/dt/dec.c Wed Dec 2 23:39:51 2015 @@ -1,5 +1,5 @@ -/* $NetBSD: dec.c,v 1.1 2015/10/08 22:00:56 dyoung Exp $ */ -/* $ARFE: dec.c 262 2015-10-08 21:47:24Z dyoung $ */ +/* $NetBSD: dec.c,v 1.2 2015/12/02 23:39:51 dyoung Exp $ */ +/* $ARFE: dec.c 264 2015-10-08 22:28:01Z dyoung $ */ /*- * Copyright (c) 2014,2015 David Young <dyo...@netbsd.org> Index: othersrc/external/bsd/arfe/dt/dec.h diff -u othersrc/external/bsd/arfe/dt/dec.h:1.1 othersrc/external/bsd/arfe/dt/dec.h:1.2 --- othersrc/external/bsd/arfe/dt/dec.h:1.1 Thu Oct 8 22:00:56 2015 +++ othersrc/external/bsd/arfe/dt/dec.h Wed Dec 2 23:39:51 2015 @@ -1,5 +1,5 @@ -/* $NetBSD: dec.h,v 1.1 2015/10/08 22:00:56 dyoung Exp $ */ -/* $ARFE: dec.h 262 2015-10-08 21:47:24Z dyoung $ */ +/* $NetBSD: dec.h,v 1.2 2015/12/02 23:39:51 dyoung Exp $ */ +/* $ARFE: dec.h 264 2015-10-08 22:28:01Z dyoung $ */ /*- * Copyright (c) 2014,2015 David Young <dyo...@netbsd.org> Index: othersrc/external/bsd/arfe/dt/macaddr.h diff -u othersrc/external/bsd/arfe/dt/macaddr.h:1.1 othersrc/external/bsd/arfe/dt/macaddr.h:1.2 --- othersrc/external/bsd/arfe/dt/macaddr.h:1.1 Fri Sep 11 01:50:42 2015 +++ othersrc/external/bsd/arfe/dt/macaddr.h Wed Dec 2 23:39:51 2015 @@ -1,5 +1,5 @@ -/* $NetBSD: macaddr.h,v 1.1 2015/09/11 01:50:42 dyoung Exp $ */ -/* $ARFE$ */ +/* $NetBSD: macaddr.h,v 1.2 2015/12/02 23:39:51 dyoung Exp $ */ +/* $ARFE: macaddr.h 286 2015-11-18 05:09:15Z dyoung $ */ /*- * Copyright (c) 2014,2015 David Young <dyo...@netbsd.org> Index: othersrc/external/bsd/arfe/dt/dt.c diff -u othersrc/external/bsd/arfe/dt/dt.c:1.14 othersrc/external/bsd/arfe/dt/dt.c:1.15 --- othersrc/external/bsd/arfe/dt/dt.c:1.14 Wed Sep 23 19:32:34 2015 +++ othersrc/external/bsd/arfe/dt/dt.c Wed Dec 2 23:39:51 2015 @@ -1,5 +1,5 @@ -/* $NetBSD: dt.c,v 1.14 2015/09/23 19:32:34 dyoung Exp $ */ -/* $ARFE: dt.c 258 2015-09-23 19:31:17Z dyoung $ */ +/* $NetBSD: dt.c,v 1.15 2015/12/02 23:39:51 dyoung Exp $ */ +/* $ARFE: dt.c 289 2015-11-19 23:51:04Z dyoung $ */ /*- * Copyright (c) 2014,2015 David Young <dyo...@netbsd.org> @@ -79,8 +79,11 @@ main(int argc, char **argv) scratch.dec_op = '-'; scratch.hex_op = '&'; - O = algc(left, match, &scratch, (origin_t){0, 0}, no_expected_lcs, - &lcs); +#if 0 + const size_t nrec = count_records(left, match, &scratch); + fprintf(stderr, "%zu records\n", nrec); +#endif + O = findlcs(left, match, &scratch, (origin_t){0, 0}, &lcs); dbg_printf("lcs = %zu\n", lcs); printchain(O); Index: othersrc/external/bsd/arfe/dt/macaddr.c diff -u othersrc/external/bsd/arfe/dt/macaddr.c:1.4 othersrc/external/bsd/arfe/dt/macaddr.c:1.5 --- othersrc/external/bsd/arfe/dt/macaddr.c:1.4 Thu Oct 8 22:00:56 2015 +++ othersrc/external/bsd/arfe/dt/macaddr.c Wed Dec 2 23:39:51 2015 @@ -1,5 +1,5 @@ -/* $NetBSD: macaddr.c,v 1.4 2015/10/08 22:00:56 dyoung Exp $ */ -/* $ARFE: macaddr.c 260 2015-09-23 19:37:44Z dyoung $ */ +/* $NetBSD: macaddr.c,v 1.5 2015/12/02 23:39:51 dyoung Exp $ */ +/* $ARFE: macaddr.c 264 2015-10-08 22:28:01Z dyoung $ */ /*- * Copyright (c) 2014,2015 David Young <dyo...@netbsd.org> Index: othersrc/external/bsd/arfe/dt/portability.h diff -u othersrc/external/bsd/arfe/dt/portability.h:1.2 othersrc/external/bsd/arfe/dt/portability.h:1.3 --- othersrc/external/bsd/arfe/dt/portability.h:1.2 Thu Oct 8 22:00:56 2015 +++ othersrc/external/bsd/arfe/dt/portability.h Wed Dec 2 23:39:51 2015 @@ -1,7 +1,8 @@ -/* $NetBSD: portability.h,v 1.2 2015/10/08 22:00:56 dyoung Exp $ */ -/* $ARFE: portability.h 261 2015-09-23 19:39:10Z dyoung $ */ +/* $NetBSD: portability.h,v 1.3 2015/12/02 23:39:51 dyoung Exp $ */ +/* $ARFE: portability.h 281 2015-11-18 04:49:11Z dyoung $ */ #ifndef __NetBSD__ +#define __noinline /* */ #define __aligned(__x) /* */ #define __arraycount(__x) (sizeof(__x) / sizeof(__x[0])) #endif Index: othersrc/external/bsd/arfe/dt/rr/daily-output-2-daily-output-5 diff -u othersrc/external/bsd/arfe/dt/rr/daily-output-2-daily-output-5:1.2 othersrc/external/bsd/arfe/dt/rr/daily-output-2-daily-output-5:1.3 --- othersrc/external/bsd/arfe/dt/rr/daily-output-2-daily-output-5:1.2 Thu Oct 8 22:00:56 2015 +++ othersrc/external/bsd/arfe/dt/rr/daily-output-2-daily-output-5 Wed Dec 2 23:39:51 2015 @@ -7,7 +7,7 @@ Checking subsystem status: disks: Filesystem Size Used Avail %Cap iUsed iAvail %iCap Mount /dev/wd0a 0G 0G -4M 0% 15 -15 0% / -/dev/wd0f 0G 0G 0G 0% 0 0 0% /home/dyoung/pkgobj +/dev/wd1f 0G 0G 0G 0% 0 0 0% /home/dyoung/pkgobj Index: othersrc/external/bsd/arfe/dt/rr/daily-output-5-daily-output-2 diff -u othersrc/external/bsd/arfe/dt/rr/daily-output-5-daily-output-2:1.2 othersrc/external/bsd/arfe/dt/rr/daily-output-5-daily-output-2:1.3 --- othersrc/external/bsd/arfe/dt/rr/daily-output-5-daily-output-2:1.2 Thu Oct 8 22:00:56 2015 +++ othersrc/external/bsd/arfe/dt/rr/daily-output-5-daily-output-2 Wed Dec 2 23:39:51 2015 @@ -1,13 +1,13 @@ -Uptime: 0: 0AM up -3 day, 0:0, 10 users, load averages: -0.13, -0.04, -0.01 +Uptime: 0: 0AM up -3 days, 0:0, 10 users, load averages: -0.13, -0.04, -0.01 Checking subsystem status: disks: Filesystem Size Used Avail %Cap iUsed iAvail %iCap Mount /dev/wd0a 0G 0G 4M 0% -15 15 0% / -/dev/wd0f 0G 0G 0G 0% 0 0 0% /home/dyoung/pkgobj +/dev/wd1f 0G 0G 0G 0% 0 0 0% /home/dyoung/pkgobj Index: othersrc/external/bsd/arfe/dt/rr/daily-output-5-daily-output-6 diff -u othersrc/external/bsd/arfe/dt/rr/daily-output-5-daily-output-6:1.2 othersrc/external/bsd/arfe/dt/rr/daily-output-5-daily-output-6:1.3 --- othersrc/external/bsd/arfe/dt/rr/daily-output-5-daily-output-6:1.2 Thu Oct 8 22:00:56 2015 +++ othersrc/external/bsd/arfe/dt/rr/daily-output-5-daily-output-6 Wed Dec 2 23:39:51 2015 @@ -7,7 +7,7 @@ Checking subsystem status: disks: Filesystem Size Used Avail %Cap iUsed iAvail %iCap Mount /dev/wd0a 0G 0G 0M 0% 4 -4 0% / -/dev/wd0f 0G 0G 0G 0% 0 0 0% /home/dyoung/pkgobj +/dev/wd1f 0G 0G 0G 0% 0 0 0% /home/dyoung/pkgobj Index: othersrc/external/bsd/arfe/dt/rr/daily-output-6-daily-output-5 diff -u othersrc/external/bsd/arfe/dt/rr/daily-output-6-daily-output-5:1.2 othersrc/external/bsd/arfe/dt/rr/daily-output-6-daily-output-5:1.3 --- othersrc/external/bsd/arfe/dt/rr/daily-output-6-daily-output-5:1.2 Thu Oct 8 22:00:56 2015 +++ othersrc/external/bsd/arfe/dt/rr/daily-output-6-daily-output-5 Wed Dec 2 23:39:51 2015 @@ -7,7 +7,7 @@ Checking subsystem status: disks: Filesystem Size Used Avail %Cap iUsed iAvail %iCap Mount /dev/wd0a 0G 0G 0M 0% -4 4 0% / -/dev/wd0f 0G 0G 0G 0% 0 0 0% /home/dyoung/pkgobj +/dev/wd1f 0G 0G 0G 0% 0 0 0% /home/dyoung/pkgobj Index: othersrc/external/bsd/arfe/dt/rr/daily-output-6-daily-output-7 diff -u othersrc/external/bsd/arfe/dt/rr/daily-output-6-daily-output-7:1.2 othersrc/external/bsd/arfe/dt/rr/daily-output-6-daily-output-7:1.3 --- othersrc/external/bsd/arfe/dt/rr/daily-output-6-daily-output-7:1.2 Thu Oct 8 22:00:56 2015 +++ othersrc/external/bsd/arfe/dt/rr/daily-output-6-daily-output-7 Wed Dec 2 23:39:51 2015 @@ -7,7 +7,7 @@ Checking subsystem status: disks: Filesystem Size Used Avail %Cap iUsed iAvail %iCap Mount /dev/wd0a 0G 0G -1M 0% -1 1 0% / -/dev/wd0f 0G 0G 0G 0% 0 0 0% /home/dyoung/pkgobj +/dev/wd1f 0G 0G 0G 0% 0 0 0% /home/dyoung/pkgobj Index: othersrc/external/bsd/arfe/dt/rr/daily-output-7-daily-output-6 diff -u othersrc/external/bsd/arfe/dt/rr/daily-output-7-daily-output-6:1.2 othersrc/external/bsd/arfe/dt/rr/daily-output-7-daily-output-6:1.3 --- othersrc/external/bsd/arfe/dt/rr/daily-output-7-daily-output-6:1.2 Thu Oct 8 22:00:56 2015 +++ othersrc/external/bsd/arfe/dt/rr/daily-output-7-daily-output-6 Wed Dec 2 23:39:51 2015 @@ -7,7 +7,7 @@ Checking subsystem status: disks: Filesystem Size Used Avail %Cap iUsed iAvail %iCap Mount /dev/wd0a 0G 0G 1M 0% 1 -1 0% / -/dev/wd0f 0G 0G 0G 0% 0 0 0% /home/dyoung/pkgobj +/dev/wd1f 0G 0G 0G 0% 0 0 0% /home/dyoung/pkgobj Index: othersrc/external/bsd/arfe/dt/rr/if-re0-if-wm0 diff -u othersrc/external/bsd/arfe/dt/rr/if-re0-if-wm0:1.4 othersrc/external/bsd/arfe/dt/rr/if-re0-if-wm0:1.5 --- othersrc/external/bsd/arfe/dt/rr/if-re0-if-wm0:1.4 Fri Sep 11 01:50:43 2015 +++ othersrc/external/bsd/arfe/dt/rr/if-re0-if-wm0 Wed Dec 2 23:39:51 2015 @@ -1,8 +1,8 @@ - 0: flags= 0<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> mtu 0 - capabilities=3f80<TSO0,IP0CSUM_Rx,IP0CSUM_Tx,TCP0CSUM_Rx,TCP0CSUM_Tx,UDP0CSUM_Rx,UDP0CSUM_Tx> - enabled=3f80<TSO0,IP0CSUM_Rx,IP0CSUM_Tx,TCP0CSUM_Rx,TCP0CSUM_Tx,UDP0CSUM_Rx,UDP0CSUM_Tx> +re0: flags= 0<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> mtu 0 + capabilities=3f80<TSO4,IP4CSUM_Rx,IP4CSUM_Tx,TCP4CSUM_Rx,TCP4CSUM_Tx,UDP4CSUM_Rx,UDP4CSUM_Tx> + enabled=3f80<TSO4,IP4CSUM_Rx,IP4CSUM_Tx,TCP4CSUM_Rx,TCP4CSUM_Tx,UDP4CSUM_Rx,UDP4CSUM_Tx> address: 00:00:00:00:00:00 - media: Ethernet autoselect (e) - status: aie + media: Ethernet autoselect (none) + status: carrier inet 0.0.0.0 netmask 0xff000000 broadcast 0.0.0.0 - inet0 fe80::-25:cff:fe00:a080%0 prefixlen 0 scopeid 0x0 + inet6 fe80::-25:cff:fe00:a080%re0 prefixlen 0 scopeid 0x0 Index: othersrc/external/bsd/arfe/dt/rr/if-wm0-if-re0 diff -u othersrc/external/bsd/arfe/dt/rr/if-wm0-if-re0:1.4 othersrc/external/bsd/arfe/dt/rr/if-wm0-if-re0:1.5 --- othersrc/external/bsd/arfe/dt/rr/if-wm0-if-re0:1.4 Fri Sep 11 01:50:43 2015 +++ othersrc/external/bsd/arfe/dt/rr/if-wm0-if-re0 Wed Dec 2 23:39:51 2015 @@ -1,8 +1,8 @@ - 0: flags= 0<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> mtu 0 - capabilities=3f80<TSO0,IP0CSUM_Rx,IP0CSUM_Tx,TCP0CSUM_Rx,TCP0CSUM_Tx,UDP0CSUM_Rx,UDP-2CSUM_Tx> - enabled=3f80<TSO0,IP0CSUM_Rx,IP0CSUM_Tx,TCP0CSUM_Rx,TCP0CSUM_Tx,UDP0CSUM_Rx,UDP-2CSUM_Tx> +wm0: flags= 0<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> mtu 0 + capabilities=3f80<TSO4,IP4CSUM_Rx,IP4CSUM_Tx,TCP4CSUM_Rx,TCP4CSUM_Tx,UDP4CSUM_Rx,UDP4CSUM_Tx> + enabled=3f80<TSO4,IP4CSUM_Rx,IP4CSUM_Tx,TCP4CSUM_Rx,TCP4CSUM_Tx,UDP4CSUM_Rx,UDP4CSUM_Tx> address: 00:00:00:00:00:00 - media: Ethernet autoselect (e) - status: cie + media: Ethernet autoselect (duplex) + status: active inet 0.0.0.0 netmask 0xff000000 broadcast 0.0.0.0 - inet0 fe80:: 25:cff:fe00:a080%0 prefixlen 0 scopeid 0x0 + inet6 fe80:: 25:cff:fe00:a080%wm0 prefixlen 0 scopeid 0x0 Index: othersrc/external/bsd/arfe/dt/rr/ifconfig.0-ifconfig.1 diff -u othersrc/external/bsd/arfe/dt/rr/ifconfig.0-ifconfig.1:1.5 othersrc/external/bsd/arfe/dt/rr/ifconfig.0-ifconfig.1:1.6 --- othersrc/external/bsd/arfe/dt/rr/ifconfig.0-ifconfig.1:1.5 Fri Sep 11 01:50:43 2015 +++ othersrc/external/bsd/arfe/dt/rr/ifconfig.0-ifconfig.1 Wed Dec 2 23:39:51 2015 @@ -1,10 +1,10 @@ wm0: flags= 0<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> mtu 0 - capabilities=2bf80<TSO0,IP0CSUM_Rx,IP0CSUM_Tx,TCP0CSUM_Rx,TCP0CSUM_Tx,UDP0CSUM_Rx,UDP0CSUM_Tx,TCP0CSUM_Tx,UDP0CSUM_Tx> - enabled=2bf80<TSO0,IP0CSUM_Rx,IP0CSUM_Tx,TCP0CSUM_Rx,TCP0CSUM_Tx,UDP0CSUM_Rx,UDP0CSUM_Tx,TCP0CSUM_Tx,UDP0CSUM_Tx> + capabilities=2bf80<TSO4,IP4CSUM_Rx,IP4CSUM_Tx,TCP4CSUM_Rx,TCP4CSUM_Tx,UDP4CSUM_Rx,UDP4CSUM_Tx,TCP6CSUM_Tx,UDP6CSUM_Tx> + enabled=2bf80<TSO4,IP4CSUM_Rx,IP4CSUM_Tx,TCP4CSUM_Rx,TCP4CSUM_Tx,UDP4CSUM_Rx,UDP4CSUM_Tx,TCP6CSUM_Tx,UDP6CSUM_Tx> address: 00:0a:0b:cd:01:ef media: Ethernet autoselect ( 0baseT full-duplex) status: active input: 61 packets, 5432 bytes, 3 multicasts output: 30 packets, 3436 bytes, 0 multicasts inet 10.0.1.17 netmask 0xffffff00 broadcast 10.0.1.255 - inet0 fe80::20a:bff:fecd:1ef%wm0 prefixlen 0 scopeid 0x1 + inet6 fe80::20a:bff:fecd:1ef%wm0 prefixlen 0 scopeid 0x1 Index: othersrc/external/bsd/arfe/dt/rr/ifconfig.0-logger-ifconfig.0 diff -u othersrc/external/bsd/arfe/dt/rr/ifconfig.0-logger-ifconfig.0:1.5 othersrc/external/bsd/arfe/dt/rr/ifconfig.0-logger-ifconfig.0:1.6 --- othersrc/external/bsd/arfe/dt/rr/ifconfig.0-logger-ifconfig.0:1.5 Fri Sep 11 01:50:43 2015 +++ othersrc/external/bsd/arfe/dt/rr/ifconfig.0-logger-ifconfig.0 Wed Dec 2 23:39:51 2015 @@ -1,10 +1,10 @@ wm0: flags=0<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> mtu 0 -capabilities=2bf80<TSO0,IP0CSUM_Rx,IP0CSUM_Tx,TCP0CSUM_Rx,TCP0CSUM_Tx,UDP0CSUM_Rx,UDP0CSUM_Tx,TCP0CSUM_Tx,UDP0CSUM_Tx> -enabled=2bf80<TSO0,IP0CSUM_Rx,IP0CSUM_Tx,TCP0CSUM_Rx,TCP0CSUM_Tx,UDP0CSUM_Rx,UDP0CSUM_Tx,TCP0CSUM_Tx,UDP0CSUM_Tx> +capabilities=2bf80<TSO4,IP4CSUM_Rx,IP4CSUM_Tx,TCP4CSUM_Rx,TCP4CSUM_Tx,UDP4CSUM_Rx,UDP4CSUM_Tx,TCP6CSUM_Tx,UDP6CSUM_Tx> +enabled=2bf80<TSO4,IP4CSUM_Rx,IP4CSUM_Tx,TCP4CSUM_Rx,TCP4CSUM_Tx,UDP4CSUM_Rx,UDP4CSUM_Tx,TCP6CSUM_Tx,UDP6CSUM_Tx> address: 00:0a:0b:cd:01:ef media: Ethernet autoselect (0baseT full-duplex) status: active input: 0 packets, 0 bytes, 0 multicasts output: 0 packets, 0 bytes, 0 multicasts inet 10.0.1.17 netmask 0xffffff00 broadcast 10.0.1.255 -inet0 fe80::20a:bff:fecd:1ef%wm0 prefixlen 0 scopeid 0x1 +inet6 fe80::20a:bff:fecd:1ef%wm0 prefixlen 0 scopeid 0x1 Index: othersrc/external/bsd/arfe/dt/rr/ifconfig.1-ifconfig.0 diff -u othersrc/external/bsd/arfe/dt/rr/ifconfig.1-ifconfig.0:1.5 othersrc/external/bsd/arfe/dt/rr/ifconfig.1-ifconfig.0:1.6 --- othersrc/external/bsd/arfe/dt/rr/ifconfig.1-ifconfig.0:1.5 Fri Sep 11 01:50:43 2015 +++ othersrc/external/bsd/arfe/dt/rr/ifconfig.1-ifconfig.0 Wed Dec 2 23:39:51 2015 @@ -1,10 +1,10 @@ wm0: flags= 0<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> mtu 0 - capabilities=2bf80<TSO0,IP0CSUM_Rx,IP0CSUM_Tx,TCP0CSUM_Rx,TCP0CSUM_Tx,UDP0CSUM_Rx,UDP0CSUM_Tx,TCP0CSUM_Tx,UDP0CSUM_Tx> - enabled=2bf80<TSO0,IP0CSUM_Rx,IP0CSUM_Tx,TCP0CSUM_Rx,TCP0CSUM_Tx,UDP0CSUM_Rx,UDP0CSUM_Tx,TCP0CSUM_Tx,UDP0CSUM_Tx> + capabilities=2bf80<TSO4,IP4CSUM_Rx,IP4CSUM_Tx,TCP4CSUM_Rx,TCP4CSUM_Tx,UDP4CSUM_Rx,UDP4CSUM_Tx,TCP6CSUM_Tx,UDP6CSUM_Tx> + enabled=2bf80<TSO4,IP4CSUM_Rx,IP4CSUM_Tx,TCP4CSUM_Rx,TCP4CSUM_Tx,UDP4CSUM_Rx,UDP4CSUM_Tx,TCP6CSUM_Tx,UDP6CSUM_Tx> address: 00:0a:0b:cd:01:ef media: Ethernet autoselect ( 0baseT full-duplex) status: active input: -61 packets, -5432 bytes, -3 multicasts output: -30 packets, -3436 bytes, 0 multicasts inet 10.0.1.17 netmask 0xffffff00 broadcast 10.0.1.255 - inet0 fe80::20a:bff:fecd:1ef%wm0 prefixlen 0 scopeid 0x1 + inet6 fe80::20a:bff:fecd:1ef%wm0 prefixlen 0 scopeid 0x1 Index: othersrc/external/bsd/arfe/dt/rr/logger-ifconfig.0-ifconfig.0 diff -u othersrc/external/bsd/arfe/dt/rr/logger-ifconfig.0-ifconfig.0:1.5 othersrc/external/bsd/arfe/dt/rr/logger-ifconfig.0-ifconfig.0:1.6 --- othersrc/external/bsd/arfe/dt/rr/logger-ifconfig.0-ifconfig.0:1.5 Fri Sep 11 01:50:43 2015 +++ othersrc/external/bsd/arfe/dt/rr/logger-ifconfig.0-ifconfig.0 Wed Dec 2 23:39:51 2015 @@ -1,10 +1,10 @@ wm0: flags=0<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> mtu 0 -capabilities=2bf80<TSO0,IP0CSUM_Rx,IP0CSUM_Tx,TCP0CSUM_Rx,TCP0CSUM_Tx,UDP0CSUM_Rx,UDP0CSUM_Tx,TCP0CSUM_Tx,UDP0CSUM_Tx> -enabled=2bf80<TSO0,IP0CSUM_Rx,IP0CSUM_Tx,TCP0CSUM_Rx,TCP0CSUM_Tx,UDP0CSUM_Rx,UDP0CSUM_Tx,TCP0CSUM_Tx,UDP0CSUM_Tx> +capabilities=2bf80<TSO4,IP4CSUM_Rx,IP4CSUM_Tx,TCP4CSUM_Rx,TCP4CSUM_Tx,UDP4CSUM_Rx,UDP4CSUM_Tx,TCP6CSUM_Tx,UDP6CSUM_Tx> +enabled=2bf80<TSO4,IP4CSUM_Rx,IP4CSUM_Tx,TCP4CSUM_Rx,TCP4CSUM_Tx,UDP4CSUM_Rx,UDP4CSUM_Tx,TCP6CSUM_Tx,UDP6CSUM_Tx> address: 00:0a:0b:cd:01:ef media: Ethernet autoselect (0baseT full-duplex) status: active input: 0 packets, 0 bytes, 0 multicasts output: 0 packets, 0 bytes, 0 multicasts inet 10.0.1.17 netmask 0xffffff00 broadcast 10.0.1.255 -inet0 fe80::20a:bff:fecd:1ef%wm0 prefixlen 0 scopeid 0x1 +inet6 fe80::20a:bff:fecd:1ef%wm0 prefixlen 0 scopeid 0x1 Index: othersrc/external/bsd/arfe/dt/rr/wm0-a-wm0-b diff -u othersrc/external/bsd/arfe/dt/rr/wm0-a-wm0-b:1.5 othersrc/external/bsd/arfe/dt/rr/wm0-a-wm0-b:1.6 --- othersrc/external/bsd/arfe/dt/rr/wm0-a-wm0-b:1.5 Fri Sep 11 01:50:43 2015 +++ othersrc/external/bsd/arfe/dt/rr/wm0-a-wm0-b Wed Dec 2 23:39:51 2015 @@ -1,12 +1,12 @@ wm0: flags= 0<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> mtu 0 - capabilities=2bf80<TSO0,IP0CSUM_Rx,IP0CSUM_Tx,TCP0CSUM_Rx,TCP0CSUM_Tx,UDP0CSUM_Rx,UDP0CSUM_Tx,TCP0CSUM_Tx,UDP0CSUM_Tx> - enabled=2bf80<TSO0,IP0CSUM_Rx,IP0CSUM_Tx,TCP0CSUM_Rx,TCP0CSUM_Tx,UDP0CSUM_Rx,UDP0CSUM_Tx,TCP0CSUM_Tx,UDP0CSUM_Tx> + capabilities=2bf80<TSO4,IP4CSUM_Rx,IP4CSUM_Tx,TCP4CSUM_Rx,TCP4CSUM_Tx,UDP4CSUM_Rx,UDP4CSUM_Tx,TCP6CSUM_Tx,UDP6CSUM_Tx> + enabled=2bf80<TSO4,IP4CSUM_Rx,IP4CSUM_Tx,TCP4CSUM_Rx,TCP4CSUM_Tx,UDP4CSUM_Rx,UDP4CSUM_Tx,TCP6CSUM_Tx,UDP6CSUM_Tx> address: 00:0a:0b:cd:01:ef media: Ethernet autoselect ( 0baseT full-duplex) status: active input: 112 packets, 11380 bytes, 11 multicasts output: 60 packets, 11440 bytes, 0 multicasts inet 10.0.1.17 netmask 0xffffff00 broadcast 10.0.1.255 - inet0 fe80::20a:bff:fecd:1ef%wm0 prefixlen 0 scopeid 0x1 + inet6 fe80::20a:bff:fecd:1ef%wm0 prefixlen 0 scopeid 0x1 Index: othersrc/external/bsd/arfe/dt/rr/wm0-b-wm0-a diff -u othersrc/external/bsd/arfe/dt/rr/wm0-b-wm0-a:1.5 othersrc/external/bsd/arfe/dt/rr/wm0-b-wm0-a:1.6 --- othersrc/external/bsd/arfe/dt/rr/wm0-b-wm0-a:1.5 Fri Sep 11 01:50:43 2015 +++ othersrc/external/bsd/arfe/dt/rr/wm0-b-wm0-a Wed Dec 2 23:39:51 2015 @@ -1,12 +1,12 @@ wm0: flags= 0<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> mtu 0 - capabilities=2bf80<TSO0,IP0CSUM_Rx,IP0CSUM_Tx,TCP0CSUM_Rx,TCP0CSUM_Tx,UDP0CSUM_Rx,UDP0CSUM_Tx,TCP0CSUM_Tx,UDP0CSUM_Tx> - enabled=2bf80<TSO0,IP0CSUM_Rx,IP0CSUM_Tx,TCP0CSUM_Rx,TCP0CSUM_Tx,UDP0CSUM_Rx,UDP0CSUM_Tx,TCP0CSUM_Tx,UDP0CSUM_Tx> + capabilities=2bf80<TSO4,IP4CSUM_Rx,IP4CSUM_Tx,TCP4CSUM_Rx,TCP4CSUM_Tx,UDP4CSUM_Rx,UDP4CSUM_Tx,TCP6CSUM_Tx,UDP6CSUM_Tx> + enabled=2bf80<TSO4,IP4CSUM_Rx,IP4CSUM_Tx,TCP4CSUM_Rx,TCP4CSUM_Tx,UDP4CSUM_Rx,UDP4CSUM_Tx,TCP6CSUM_Tx,UDP6CSUM_Tx> address: 00:0a:0b:cd:01:ef media: Ethernet autoselect ( 0baseT full-duplex) status: active input: -112 packets, -11380 bytes, -11 multicasts output: -60 packets, -11440 bytes, 0 multicasts inet 10.0.1.17 netmask 0xffffff00 broadcast 10.0.1.255 - inet0 fe80::20a:bff:fecd:1ef%wm0 prefixlen 0 scopeid 0x1 + inet6 fe80::20a:bff:fecd:1ef%wm0 prefixlen 0 scopeid 0x1 Index: othersrc/external/bsd/arfe/dt/rr/netstat-s.0-netstat-s.1 diff -u othersrc/external/bsd/arfe/dt/rr/netstat-s.0-netstat-s.1:1.1 othersrc/external/bsd/arfe/dt/rr/netstat-s.0-netstat-s.1:1.2 --- othersrc/external/bsd/arfe/dt/rr/netstat-s.0-netstat-s.1:1.1 Mon Aug 10 21:10:59 2015 +++ othersrc/external/bsd/arfe/dt/rr/netstat-s.0-netstat-s.1 Wed Dec 2 23:39:51 2015 @@ -207,7 +207,7 @@ IPsec ipcomp: IPcomp histogram: 0 ipcomp input bytes 0 ipcomp output bytes -ip0: +ip6: 91 total packets received 0 with size smaller than minimum 0 with data size < data length @@ -237,7 +237,7 @@ ip0: hop by hop: 0 TCP: 89 UDP: 0 - ICMP0: 2 + ICMP6: 2 Mbuf statistics: 0 one mbufs two or more mbuf: @@ -250,9 +250,9 @@ ip0: 0 failures of source address selection 91 forward cache hit 0 forward cache miss -icmp0: - 0 calls to icmp0_error - 0 errors not generated because old message was icmp0 or so +icmp6: + 0 calls to icmp6_error + 0 errors not generated because old message was icmp6 or so 0 errors not generated because of rate limitation Output packet histogram: multicast listener report: 0 @@ -292,7 +292,7 @@ icmp0: 0 router advertisement routes dropped 0 bad redirect messages 0 path MTU changes -tcp0: +tcp6: 52 packets sent 48 data packets ( 6672 bytes) 0 data packets ( 0 bytes) retransmitted @@ -356,7 +356,7 @@ tcp0: 0 successful ECN handshakes 0 packets with ECN CE bit 0 packets ECN ECT(0) bit -udp0: +udp6: 0 datagrams received 0 with incomplete header 0 with bad data length field @@ -450,7 +450,7 @@ IPsec ipcomp: IPcomp histogram: 0 ipcomp input bytes 0 ipcomp output bytes -pim0: +pim6: 0 messages received 0 messages received with too few bytes 0 messages received with bad checksum @@ -458,7 +458,7 @@ pim0: 0 registers received 0 bad registers received 0 registers sent -rip0: +rip6: 0 messages received 0 checksum calculations on inbound 0 messages with bad checksum Index: othersrc/external/bsd/arfe/dt/rr/netstat-s.1-netstat-s.0 diff -u othersrc/external/bsd/arfe/dt/rr/netstat-s.1-netstat-s.0:1.1 othersrc/external/bsd/arfe/dt/rr/netstat-s.1-netstat-s.0:1.2 --- othersrc/external/bsd/arfe/dt/rr/netstat-s.1-netstat-s.0:1.1 Mon Aug 10 21:10:59 2015 +++ othersrc/external/bsd/arfe/dt/rr/netstat-s.1-netstat-s.0 Wed Dec 2 23:39:51 2015 @@ -207,7 +207,7 @@ IPsec ipcomp: IPcomp histogram: 0 ipcomp input bytes 0 ipcomp output bytes -ip0: +ip6: -91 total packets received 0 with size smaller than minimum 0 with data size < data length @@ -237,7 +237,7 @@ ip0: hop by hop: 0 TCP: -89 UDP: 0 - ICMP0: -2 + ICMP6: -2 Mbuf statistics: 0 one mbufs two or more mbuf: @@ -250,9 +250,9 @@ ip0: 0 failures of source address selection -91 forward cache hit 0 forward cache miss -icmp0: - 0 calls to icmp0_error - 0 errors not generated because old message was icmp0 or so +icmp6: + 0 calls to icmp6_error + 0 errors not generated because old message was icmp6 or so 0 errors not generated because of rate limitation Output packet histogram: multicast listener report: 0 @@ -292,7 +292,7 @@ icmp0: 0 router advertisement routes dropped 0 bad redirect messages 0 path MTU changes -tcp0: +tcp6: -52 packets sent -48 data packets ( -6672 bytes) 0 data packets ( 0 bytes) retransmitted @@ -356,7 +356,7 @@ tcp0: 0 successful ECN handshakes 0 packets with ECN CE bit 0 packets ECN ECT(0) bit -udp0: +udp6: 0 datagrams received 0 with incomplete header 0 with bad data length field @@ -450,7 +450,7 @@ IPsec ipcomp: IPcomp histogram: 0 ipcomp input bytes 0 ipcomp output bytes -pim0: +pim6: 0 messages received 0 messages received with too few bytes 0 messages received with bad checksum @@ -458,7 +458,7 @@ pim0: 0 registers received 0 bad registers received 0 registers sent -rip0: +rip6: 0 messages received 0 checksum calculations on inbound 0 messages with bad checksum Index: othersrc/external/bsd/arfe/dt/rr/quack123quack-quack456quack diff -u othersrc/external/bsd/arfe/dt/rr/quack123quack-quack456quack:1.1 othersrc/external/bsd/arfe/dt/rr/quack123quack-quack456quack:1.2 --- othersrc/external/bsd/arfe/dt/rr/quack123quack-quack456quack:1.1 Mon Aug 10 21:10:59 2015 +++ othersrc/external/bsd/arfe/dt/rr/quack123quack-quack456quack Wed Dec 2 23:39:51 2015 @@ -1 +0,0 @@ -quack333quack Index: othersrc/external/bsd/arfe/dt/rr/quack456quack-quack123quack diff -u othersrc/external/bsd/arfe/dt/rr/quack456quack-quack123quack:1.1 othersrc/external/bsd/arfe/dt/rr/quack456quack-quack123quack:1.2 --- othersrc/external/bsd/arfe/dt/rr/quack456quack-quack123quack:1.1 Mon Aug 10 21:10:59 2015 +++ othersrc/external/bsd/arfe/dt/rr/quack456quack-quack123quack Wed Dec 2 23:39:51 2015 @@ -1 +0,0 @@ -quack-333quack Index: othersrc/external/bsd/arfe/dt/rr/quackabequack-quackdariaquack diff -u othersrc/external/bsd/arfe/dt/rr/quackabequack-quackdariaquack:1.1 othersrc/external/bsd/arfe/dt/rr/quackabequack-quackdariaquack:1.2 --- othersrc/external/bsd/arfe/dt/rr/quackabequack-quackdariaquack:1.1 Mon Aug 10 21:10:59 2015 +++ othersrc/external/bsd/arfe/dt/rr/quackabequack-quackdariaquack Wed Dec 2 23:39:51 2015 @@ -1 +0,0 @@ -quack<word>quack Index: othersrc/external/bsd/arfe/dt/rr/quackdariaquack-quackabequack diff -u othersrc/external/bsd/arfe/dt/rr/quackdariaquack-quackabequack:1.1 othersrc/external/bsd/arfe/dt/rr/quackdariaquack-quackabequack:1.2 --- othersrc/external/bsd/arfe/dt/rr/quackdariaquack-quackabequack:1.1 Mon Aug 10 21:10:59 2015 +++ othersrc/external/bsd/arfe/dt/rr/quackdariaquack-quackabequack Wed Dec 2 23:39:51 2015 @@ -1 +0,0 @@ -quack<word>quack Index: othersrc/external/bsd/arfe/it/Makefile diff -u othersrc/external/bsd/arfe/it/Makefile:1.5 othersrc/external/bsd/arfe/it/Makefile:1.6 --- othersrc/external/bsd/arfe/it/Makefile:1.5 Thu Oct 8 22:00:56 2015 +++ othersrc/external/bsd/arfe/it/Makefile Wed Dec 2 23:39:51 2015 @@ -2,7 +2,7 @@ NOMAN= .include <bsd.own.mk> PROG=it -SRCS+=core.c dec.c hex.c ipv4.c it.c macaddr.c +SRCS+=core.c dec.c hex.c ipv4.c it.c macaddr.c sym.c #CPPFLAGS+=-DHB_DEBUG CPPFLAGS+=-I$(.CURDIR)/../dt DBG+=-g -O3 @@ -12,11 +12,6 @@ DBG+=-g -O3 CFLAGS+=-std=c99 WARNS=5 -tags: $(SRCS) $(INCS) - @rm -f $(.OBJDIR)/tags - @ctags -atwd $(.ALLSRC) - @sort -o $(.OBJDIR)/tags $(.OBJDIR)/tags - .PATH: $(.CURDIR)/../dt .include <bsd.prog.mk> Index: othersrc/external/bsd/arfe/it/README diff -u othersrc/external/bsd/arfe/it/README:1.8 othersrc/external/bsd/arfe/it/README:1.9 --- othersrc/external/bsd/arfe/it/README:1.8 Wed Sep 23 19:32:34 2015 +++ othersrc/external/bsd/arfe/it/README Wed Dec 2 23:39:51 2015 @@ -1,5 +1,5 @@ -$ARFE: README 258 2015-09-23 19:31:17Z dyoung $ -$NetBSD: README,v 1.8 2015/09/23 19:32:34 dyoung Exp $ +$ARFE: README 264 2015-10-08 22:28:01Z dyoung $ +$NetBSD: README,v 1.9 2015/12/02 23:39:51 dyoung Exp $ IT---(i)ntegrate (t)ext---is a variation on DT that adds decimal numbers instead of subtracts, and bitwise-ORs hexadecimal numbers instead of Index: othersrc/external/bsd/arfe/it/it.c diff -u othersrc/external/bsd/arfe/it/it.c:1.1 othersrc/external/bsd/arfe/it/it.c:1.2 --- othersrc/external/bsd/arfe/it/it.c:1.1 Tue Sep 22 01:12:09 2015 +++ othersrc/external/bsd/arfe/it/it.c Wed Dec 2 23:39:51 2015 @@ -1,4 +1,4 @@ -/* $NetBSD: it.c,v 1.1 2015/09/22 01:12:09 dyoung Exp $ */ +/* $NetBSD: it.c,v 1.2 2015/12/02 23:39:51 dyoung Exp $ */ /* $ARFE: dt.c 247 2015-09-14 02:52:57Z dyoung $ */ /*- @@ -79,8 +79,7 @@ main(int argc, char **argv) scratch.dec_op = '+'; scratch.hex_op = '|'; - O = algc(left, match, &scratch, (origin_t){0, 0}, no_expected_lcs, - &lcs); + O = findlcs(left, match, &scratch, (origin_t){0, 0}, &lcs); dbg_printf("lcs = %zu\n", lcs); printchain(O); Index: othersrc/external/bsd/arfe/tt/Makefile diff -u othersrc/external/bsd/arfe/tt/Makefile:1.3 othersrc/external/bsd/arfe/tt/Makefile:1.4 --- othersrc/external/bsd/arfe/tt/Makefile:1.3 Thu Oct 8 22:00:56 2015 +++ othersrc/external/bsd/arfe/tt/Makefile Wed Dec 2 23:39:51 2015 @@ -2,7 +2,7 @@ NOMAN= .include <bsd.own.mk> PROG=tt -SRCS+=core.c dec.c hex.c ipv4.c macaddr.c tt.c +SRCS+=core.c dec.c hex.c ipv4.c macaddr.c sym.c tt.c #CPPFLAGS+=-DHB_DEBUG CPPFLAGS+=-DHB_ASSERT CPPFLAGS+=-I$(.CURDIR)/../dt @@ -13,11 +13,6 @@ DBG+=-g -O3 CFLAGS+=-std=c99 WARNS=5 -tags: $(SRCS) $(INCS) - @rm -f $(.OBJDIR)/tags - @ctags -atwd $(.ALLSRC) - @sort -o $(.OBJDIR)/tags $(.OBJDIR)/tags - test: ${.OBJDIR}/${PROG} @OBJDIR=${.OBJDIR} sh testit.sh Index: othersrc/external/bsd/arfe/tt/README diff -u othersrc/external/bsd/arfe/tt/README:1.5 othersrc/external/bsd/arfe/tt/README:1.6 --- othersrc/external/bsd/arfe/tt/README:1.5 Wed Sep 23 19:32:34 2015 +++ othersrc/external/bsd/arfe/tt/README Wed Dec 2 23:39:51 2015 @@ -1,5 +1,5 @@ -$ARFE: README 258 2015-09-23 19:31:17Z dyoung $ -$NetBSD: README,v 1.5 2015/09/23 19:32:34 dyoung Exp $ +$ARFE: README 264 2015-10-08 22:28:01Z dyoung $ +$NetBSD: README,v 1.6 2015/12/02 23:39:51 dyoung Exp $ TT---(t)ransform (t)ext---transforms its input based on a match/transform-template pair that exemplifies the changes that should Index: othersrc/external/bsd/arfe/tt/tt.c diff -u othersrc/external/bsd/arfe/tt/tt.c:1.2 othersrc/external/bsd/arfe/tt/tt.c:1.3 --- othersrc/external/bsd/arfe/tt/tt.c:1.2 Wed Sep 23 19:32:34 2015 +++ othersrc/external/bsd/arfe/tt/tt.c Wed Dec 2 23:39:51 2015 @@ -1,4 +1,4 @@ -/* $NetBSD: tt.c,v 1.2 2015/09/23 19:32:34 dyoung Exp $ */ +/* $NetBSD: tt.c,v 1.3 2015/12/02 23:39:51 dyoung Exp $ */ /* $ARFE$ */ /*- @@ -82,8 +82,7 @@ main(int argc, char **argv) transform = file_to_slice(argv[3]); cloccs_init(&Toccs, transform); - O = algc(left, match, &scratch, (origin_t){0, 0}, no_expected_lcs, - &lcs); + O = findlcs(left, match, &scratch, (origin_t){0, 0}, &lcs); dbg_printf("lcs = %zu\n", lcs); emit_transformed_text(O, &scratch.Boccs, &Toccs, match, transform); Index: othersrc/external/bsd/arfe/tt/tests/1/match diff -u othersrc/external/bsd/arfe/tt/tests/1/match:1.1 othersrc/external/bsd/arfe/tt/tests/1/match:1.2 --- othersrc/external/bsd/arfe/tt/tests/1/match:1.1 Fri Sep 11 01:52:20 2015 +++ othersrc/external/bsd/arfe/tt/tests/1/match Wed Dec 2 23:39:51 2015 @@ -4,6 +4,6 @@ wm0: flags=8843<UP,BROADCAST,RUNNING,SIM address: 00:0a:0b:cd:01:ef media: Ethernet autoselect (1000baseT full-duplex) status: active - input: 745177 packets, 455120099 bytes, 74377 multicasts + input: 745177 packets, 455120099 bytes, 0 multicasts output: 540500 packets, 121290742 bytes, 253 multicasts inet 10.0.1.17 netmask 0xffffff00 broadcast 10.0.1.255 Index: othersrc/external/bsd/arfe/tt/tests/1/transform diff -u othersrc/external/bsd/arfe/tt/tests/1/transform:1.1 othersrc/external/bsd/arfe/tt/tests/1/transform:1.2 --- othersrc/external/bsd/arfe/tt/tests/1/transform:1.1 Fri Sep 11 01:52:20 2015 +++ othersrc/external/bsd/arfe/tt/tests/1/transform Wed Dec 2 23:39:51 2015 @@ -1,3 +1,3 @@ wm0 - 455120099 input bytes in 745177 packets (74377 multicast) + 455120099 input bytes in 745177 packets (0 multicast) 121290742 output bytes in 540500 packets (253 multicast)