libevent: endless loop on excessively large buffers

2019-05-02 Thread Tobias Stoeckmann
It is possible to trigger an endless loop or out of boundary write
on 64 bit systems with evbuffer_readline calls for buffers which
exceed 4 GB (i.e. overflow uint).

for (i = 0; i < len; i++)

Variable i is unsigned int and len size_t. This leads to an endless
loop if len is larger than UINT_MAX.

If the loop ends exactly at UINT_MAX, then a malloc with additional
space for an ending '\0' leads to an overflow, effectively allocating
0 bytes and therefore an out of boundary write occurs.

This is a proof of concept for an endless loop:

---
#include 
#include 
#include 
#include 
#include 

int
main(void)
{
char *buf;
const size_t len = 0x1UL;
struct evbuffer *buffer;
char *line;

if ((buf = malloc(len)) == NULL)
err(1, "malloc");
memset(buf, 'A', len);

if ((buffer = evbuffer_new()) == NULL)
errx(1, "evbuffer_new");
if (evbuffer_expand(buffer, len + 1))
errx(1, "evbuffer_expand");
evbuffer_add(buffer, buf, len);
evbuffer_add(buffer, "", 1);

printf("%p\n", evbuffer_readline(buffer));
return 0;
}
---

Generally this is a rather theoretical case. Normal users are not
allowed to allocate so much memory. But better be safe than sorry,
especially if login.conf values were adjusted (or the process runs
as root).

This patch completely removes "unsigned int" from buffer.c.


Index: buffer.c
===
RCS file: /cvs/src/lib/libevent/buffer.c,v
retrieving revision 1.31
diff -u -p -u -p -r1.31 buffer.c
--- buffer.c18 Mar 2017 01:48:43 -  1.31
+++ buffer.c1 May 2019 11:00:29 -
@@ -188,7 +188,7 @@ evbuffer_readline(struct evbuffer *buffe
u_char *data = EVBUFFER_DATA(buffer);
size_t len = EVBUFFER_LENGTH(buffer);
char *line;
-   unsigned int i;
+   size_t i;
 
for (i = 0; i < len; i++) {
if (data[i] == '\r' || data[i] == '\n')
@@ -232,7 +232,7 @@ evbuffer_readln(struct evbuffer *buffer,
u_char *start_of_eol, *end_of_eol;
size_t len = EVBUFFER_LENGTH(buffer);
char *line;
-   unsigned int i, n_to_copy, n_to_drain;
+   size_t i, n_to_copy, n_to_drain;
 
if (n_read_out)
*n_read_out = 0;



Re: httpd: New log format to log X-Forwarded-{For|Port} headers

2019-05-02 Thread Bruno Flückiger
On 03.05., Theo Buehler wrote:
> On Fri, Mar 08, 2019 at 10:52:28AM +0100, Reyk Floeter wrote:
> > Hi,
> >
> > On Mon, Mar 04, 2019 at 02:06:02PM +0100, Bruno Flueckiger wrote:
> > > I've completely reworked my patch for httpd(8). The last patch broke the
> > > log format combined. And the config option was ugly. This time I've
> > > added another log format called forwarded. It appends two fields to the
> > > log format combined: The first field contains the value of the header
> > > X-Forwarded-For and the second one the value of X-Forwarded-Port. If
> > > either of the headers is empty or missing a dash (-) is written.
> > >
> > > The new log format is compatible with log analyzing tools like Webalizer
> > > or GoAccess. If you run httpd(8) behind a proxy like relayd(8) the new
> > > log format finally gives you a way to track the origin of the requests.
> > >
> >
> > Your diff looks clean and makes a lot of sense.
> >
> > Especially since X-Forwarded-For is a feature in relayd that I first
> > used and documented around 2006/2007.  Adding the forwarded style to
> > httpd is a complementary feature in OpenBSD and not something for a
> > random external web stack.
> >
> > OK reyk@
> >
> > Anyone else, any objections?
>
> That would be really nice to have. Did this slip through the cracks or
> are there concerns with this diff?
>

In an earlier discussion there were some concerns about log analyzers
like Webalizer and GoAccess. For GoAccess the following awk script
converts the access.log to the combined format:

#!/usr/bin/awk

{
if ($0 ~ /newsyslog/) next
ip = $(NF - 1)
sub(/127.0.0.1/, ip)
for (i = 2; i <= NF - 2; i++)
printf("%s ", $i)
printf("\n")
}

Cheers,
Bruno



Re: httpd: New log format to log X-Forwarded-{For|Port} headers

2019-05-02 Thread Mischa Peters



> On 3 May 2019, at 04:59, Theo Buehler  wrote:
> 
>> On Fri, Mar 08, 2019 at 10:52:28AM +0100, Reyk Floeter wrote:
>> Hi,
>> 
>>> On Mon, Mar 04, 2019 at 02:06:02PM +0100, Bruno Flueckiger wrote:
>>> I've completely reworked my patch for httpd(8). The last patch broke the
>>> log format combined. And the config option was ugly. This time I've
>>> added another log format called forwarded. It appends two fields to the
>>> log format combined: The first field contains the value of the header
>>> X-Forwarded-For and the second one the value of X-Forwarded-Port. If
>>> either of the headers is empty or missing a dash (-) is written.
>>> 
>>> The new log format is compatible with log analyzing tools like Webalizer
>>> or GoAccess. If you run httpd(8) behind a proxy like relayd(8) the new
>>> log format finally gives you a way to track the origin of the requests.
>>> 
>> 
>> Your diff looks clean and makes a lot of sense.
>> 
>> Especially since X-Forwarded-For is a feature in relayd that I first
>> used and documented around 2006/2007.  Adding the forwarded style to
>> httpd is a complementary feature in OpenBSD and not something for a
>> random external web stack.
>> 
>> OK reyk@
>> 
>> Anyone else, any objections?
> 
> That would be really nice to have. Did this slip through the cracks or
> are there concerns with this diff?
> 

I believe it fell through the cracks. Would be super useful. 

Mischa

>> 
>> Reyk
>> 
>>> Cheers,
>>> Bruno
>>> 
>>> Index: usr.sbin/httpd/httpd.conf.5
>>> ===
>>> RCS file: /cvs/src/usr.sbin/httpd/httpd.conf.5,v
>>> retrieving revision 1.103
>>> diff -u -p -r1.103 httpd.conf.5
>>> --- usr.sbin/httpd/httpd.conf.519 Feb 2019 11:37:26 -1.103
>>> +++ usr.sbin/httpd/httpd.conf.527 Feb 2019 15:26:48 -
>>> @@ -450,7 +450,8 @@ The
>>> .Ar style
>>> can be
>>> .Cm common ,
>>> -.Cm combined
>>> +.Cm combined ,
>>> +.Cm forwarded
>>> or
>>> .Cm connection .
>>> The styles
>>> @@ -459,6 +460,14 @@ and
>>> .Cm combined
>>> write a log entry after each request similar to the standard Apache
>>> and nginx access log formats.
>>> +The style
>>> +.Cm forwarded
>>> +extends the style
>>> +.Cm combined
>>> +by appending two fields containing the values of the headers
>>> +.Ar X-Forwarded-For
>>> +and
>>> +.Ar X-Forwarded-Port .
>>> The style
>>> .Cm connection
>>> writes a summarized log entry after each connection,
>>> Index: usr.sbin/httpd/httpd.h
>>> ===
>>> RCS file: /cvs/src/usr.sbin/httpd/httpd.h,v
>>> retrieving revision 1.143
>>> diff -u -p -r1.143 httpd.h
>>> --- usr.sbin/httpd/httpd.h19 Feb 2019 11:37:26 -1.143
>>> +++ usr.sbin/httpd/httpd.h27 Feb 2019 15:26:48 -
>>> @@ -437,7 +437,8 @@ SPLAY_HEAD(client_tree, client);
>>> enum log_format {
>>>LOG_FORMAT_COMMON,
>>>LOG_FORMAT_COMBINED,
>>> -LOG_FORMAT_CONNECTION
>>> +LOG_FORMAT_CONNECTION,
>>> +LOG_FORMAT_FORWARDED
>>> };
>>> 
>>> struct log_file {
>>> Index: usr.sbin/httpd/parse.y
>>> ===
>>> RCS file: /cvs/src/usr.sbin/httpd/parse.y,v
>>> retrieving revision 1.110
>>> diff -u -p -r1.110 parse.y
>>> --- usr.sbin/httpd/parse.y19 Feb 2019 11:37:26 -1.110
>>> +++ usr.sbin/httpd/parse.y27 Feb 2019 15:26:48 -
>>> @@ -140,7 +140,7 @@ typedef struct {
>>> %tokenPROTOCOLS REQUESTS ROOT SACK SERVER SOCKET STRIP STYLE SYSLOG TCP 
>>> TICKET
>>> %tokenTIMEOUT TLS TYPE TYPES HSTS MAXAGE SUBDOMAINS DEFAULT PRELOAD 
>>> REQUEST
>>> %tokenERROR INCLUDE AUTHENTICATE WITH BLOCK DROP RETURN PASS REWRITE
>>> -%tokenCA CLIENT CRL OPTIONAL PARAM
>>> +%tokenCA CLIENT CRL OPTIONAL PARAM FORWARDED
>>> %tokenSTRING
>>> %token  NUMBER
>>> %typeport
>>> @@ -1024,6 +1024,11 @@ logstyle: COMMON{
>>>srv_conf->flags |= SRVFLAG_LOG;
>>>srv_conf->logformat = LOG_FORMAT_CONNECTION;
>>>}
>>> +| FORWARDED{
>>> +srv_conf->flags &= ~SRVFLAG_NO_LOG;
>>> +srv_conf->flags |= SRVFLAG_LOG;
>>> +srv_conf->logformat = LOG_FORMAT_FORWARDED;
>>> +}
>>>;
>>> 
>>> filter: block RETURN NUMBER optstring{
>>> @@ -1295,6 +1300,7 @@ lookup(char *s)
>>>{ "ecdhe",ECDHE },
>>>{ "error",ERR },
>>>{ "fastcgi",FCGI },
>>> +{ "forwarded",FORWARDED },
>>>{ "hsts",HSTS },
>>>{ "include",INCLUDE },
>>>{ "index",INDEX },
>>> Index: usr.sbin/httpd/server_http.c
>>> ===
>>> RCS file: /cvs/src/usr.sbin/httpd/server_http.c,v
>>> retrieving revision 1.129
>>> diff -u -p -r1.129 server_http.c
>>> --- usr.sbin/httpd/server_http.c10 Feb 2019 13:41:27 -1.129
>>> +++ usr.sbin/httpd/server_http.c  

Re: perldoc: fix man output & formatting

2019-05-02 Thread Andrew Hewus Fresh
On Fri, May 03, 2019 at 04:27:15AM +0200, Ingo Schwarze wrote:
> Hi Todd & Andrew,
> 
> Andrew Fresh wrote on Thu, May 02, 2019 at 09:53:29AM -0700:
> > On Thu, May 02, 2019 at 10:21:15AM -0600, Todd C. Miller wrote:
> >> On Thu, 02 May 2019 10:53:37 -0500, Andrew Daugherity wrote:
> 
> >>> I reported this to FreeBSD ports a couple months ago [2], and they
> >>> provided a fix [3] which repairs the -oMan output, and makes that the
> >>> default.  Their fix applies cleanly to the OpenBSD tree and works, but
> >>> I have no idea why mandoc requires resetting $?.  (Also, their ToMan
> >>> patch has a previously-included hunk for MANWIDTH=tty, but it doesn't
> >>> seem to do anything for me?)
> 
> >> There problem is a missing waitpid() call, so $? is not actually
> >> set.  Something like the following should be better, though I haven't
> >> tested it yet.
> 
> > This seems to work fine for me, OK afresh1@
> 
> OK schwarze@, too.
> 
> > I submitted this upstream as well, so hopefully shouldn't have to keep
> > the patch for too long.
> > https://github.com/mrallen1/Pod-Perldoc/pull/39
> 
> Thanks.
> 
> > Still looking at defaulting to -oMan, but probably a good idea.
> 
> I agree in principle that -oMan ought to become the default.


There is an upstream patch to do this if the "nroffer" is a new-enough
groff or an older groff with the pager less.  I added a comment
recommending that the should also set the pager to "man" if the
"nroffer" is mandoc, which is the default on the BSD's it recognizes.
I'm not sure if I can add a patch directly to a GitHub PR that I didn't
create, but I may still make one and attach it somehow.


In any case, I'll probably commit this slightly cleaned up version
tomorrow as it's to late right now to make sure I didn't clean it too
far.

 
Index: gnu/usr.bin/perl/cpan/Pod-Perldoc/lib/Pod/Perldoc.pm
===
RCS file: /cvs/src/gnu/usr.bin/perl/cpan/Pod-Perldoc/lib/Pod/Perldoc.pm,v
retrieving revision 1.4
diff -u -p -r1.4 Perldoc.pm
--- gnu/usr.bin/perl/cpan/Pod-Perldoc/lib/Pod/Perldoc.pm13 Feb 2019 
21:15:14 -  1.4
+++ gnu/usr.bin/perl/cpan/Pod-Perldoc/lib/Pod/Perldoc.pm3 May 2019 
04:29:51 -
@@ -486,6 +486,8 @@ sub init_formatter_class_list {
 
   $self->opt_M_with('Pod::Perldoc::ToPod');   # the always-there fallthru
   $self->opt_o_with('text');
+  $self->opt_o_with('man')
+if $ENV{TERM} && $ENV{TERM} !~ /dumb|emacs|none|unknown/i;
 
   return;
 }
Index: gnu/usr.bin/perl/cpan/Pod-Perldoc/lib/Pod/Perldoc/ToMan.pm
===
RCS file: /cvs/src/gnu/usr.bin/perl/cpan/Pod-Perldoc/lib/Pod/Perldoc/ToMan.pm,v
retrieving revision 1.8
diff -u -p -r1.8 ToMan.pm
--- gnu/usr.bin/perl/cpan/Pod-Perldoc/lib/Pod/Perldoc/ToMan.pm  13 Feb 2019 
21:15:14 -  1.8
+++ gnu/usr.bin/perl/cpan/Pod-Perldoc/lib/Pod/Perldoc/ToMan.pm  3 May 2019 
04:29:51 -
@@ -352,6 +352,9 @@ sub _filter_through_nroff {
close $writer;
$self->debug( "Done writing\n" );
 
+   # wait for it to exit
+   waitpid( $pid, 0 );
+
# read any leftovers
$done .= do { local $/; <$reader> };
$self->debug( sprintf "Done reading. Output is %d bytes\n",



Re: kqueue.2: formatting fixes and minor HISTORY expansion

2019-05-02 Thread Ingo Schwarze
Hi Fabio,

Fabio Scotoni wrote on Thu, May 02, 2019 at 03:33:42PM +0200:

> I've taken a stab at improving kqueue.2 formatting.
> Most of the changes are markup fixes.

All your formatting decisions are good.

> I used ".Dv NULL" over plain "null" in accordance with
> lib/libc/stdlib/malloc.3 rev. 1.113.

You are right.
There isn't much wrong with using other manual pages as examples
of good formatting practices (the risk of picking bad ones is
relatively small in OpenBSD).

The authoritative sources are, in this case:

   $ LESS=+t man -O tag=Dv mdoc

   https://mandoc.bsd.lv/mdoc/appendix/markup.html
  (search for "null pointer")

> I also added a note to the HISTORY section that kqueue()/kevent() have
> been available in OpenBSD since 2.9;
> the wording matches growfs(8).

Yes, that is the standard wording.

> I'm not sure how to handle the undocumented EPERM that is returned if a
> pledge(2) does not include "proc" when an attempt is made to attach to a
> process with EVFILT_PROC.
> It does feel somewhat non-obvious, but is noted in neither pledge(2) nor
> kqueue(2).

Seems like a good question to me, but i'll not mix that into a mere
formatting cleanup commit.

Committed with minor tweaks, thanks.
  Ingo



Re: perldoc: fix man output & formatting

2019-05-02 Thread Ingo Schwarze
Hi Andrew,

Andrew Daugherity wrote on Thu, May 02, 2019 at 10:53:37AM -0500:

> Also, their ToMan patch has a previously-included hunk
> for MANWIDTH=tty,

All that does is suppress a warning message "non-numeric MANWIDTH"
when a user has MANWIDTH=tty in their environment.  No idea why any
user would have that.  I see no reason to merge that part of the
patch to OpenBSD.

> but it doesn't seem to do anything for me?)

No, it doesn't do anything on OpenBSD because _get_columns() never
gets called when mandoc(1) is being used.

Then again, there is no real reason why _get_columns() isn't used
with mandoc: mandoc does support setting the output width.
Not via "-rLL=" though, that is a groffism.

If people want perldoc(1) to respect the MANWIDTH environment
variable, we could commit the following patch (and feed it upstream).

I don't really care either way.

Regarding the patch below, note that i'm deliberately not doing any
validation or truncation.  People who trust environment variables
enough to use them but not enough to use them without validation
seem silly to me.  On the one hand, the existing code in ToMan.pm
already does validation on MANWIDTH which does not need to be
repeated.  On the other hand, even if it wouldn't, why not leave
the validation to mandoc(1), which already validates the -O width=
argument?

In mandoc(1)/man(1) itself, i decided against supporting the MANWIDTH
variable.  In general, i hate environment variables.  They stealthily
change the behaviour of programs, making debugging harder and causing
needless headaches about security boundaries when leaking from one
process into its children.  In man(1) and mandoc(1), supporting
MANWIDTH would be pointless because you can already use the -O width=
option if you want to, which is clearer and less sneaky.

Yours,
  Ingo


Index: ToMan.pm
===
RCS file: /cvs/src/gnu/usr.bin/perl/cpan/Pod-Perldoc/lib/Pod/Perldoc/ToMan.pm,v
retrieving revision 1.8
diff -u -p -r1.8 ToMan.pm
--- ToMan.pm13 Feb 2019 21:15:14 -  1.8
+++ ToMan.pm3 May 2019 02:49:03 -
@@ -227,6 +227,10 @@ sub _collect_nroff_switches {
push @render_switches, '-rLL=' . (int $c) . 'n' if $cols > 80;
}
 
+   if( $self->_is_mandoc ) {
+   push @render_switches, '-Owidth=' . $self->_get_columns;
+   }
+
# I hear persistent reports that adding a -c switch to $render
# solves many people's problems.  But I also hear that some mans
# don't have a -c switch, so that unconditionally adding it here



Re: httpd: New log format to log X-Forwarded-{For|Port} headers

2019-05-02 Thread Theo Buehler
On Fri, Mar 08, 2019 at 10:52:28AM +0100, Reyk Floeter wrote:
> Hi,
> 
> On Mon, Mar 04, 2019 at 02:06:02PM +0100, Bruno Flueckiger wrote:
> > I've completely reworked my patch for httpd(8). The last patch broke the
> > log format combined. And the config option was ugly. This time I've
> > added another log format called forwarded. It appends two fields to the
> > log format combined: The first field contains the value of the header
> > X-Forwarded-For and the second one the value of X-Forwarded-Port. If
> > either of the headers is empty or missing a dash (-) is written.
> > 
> > The new log format is compatible with log analyzing tools like Webalizer
> > or GoAccess. If you run httpd(8) behind a proxy like relayd(8) the new
> > log format finally gives you a way to track the origin of the requests.
> > 
> 
> Your diff looks clean and makes a lot of sense.
> 
> Especially since X-Forwarded-For is a feature in relayd that I first
> used and documented around 2006/2007.  Adding the forwarded style to
> httpd is a complementary feature in OpenBSD and not something for a
> random external web stack.
> 
> OK reyk@
> 
> Anyone else, any objections?

That would be really nice to have. Did this slip through the cracks or
are there concerns with this diff?

> 
> Reyk
> 
> > Cheers,
> > Bruno
> > 
> > Index: usr.sbin/httpd/httpd.conf.5
> > ===
> > RCS file: /cvs/src/usr.sbin/httpd/httpd.conf.5,v
> > retrieving revision 1.103
> > diff -u -p -r1.103 httpd.conf.5
> > --- usr.sbin/httpd/httpd.conf.5 19 Feb 2019 11:37:26 -  1.103
> > +++ usr.sbin/httpd/httpd.conf.5 27 Feb 2019 15:26:48 -
> > @@ -450,7 +450,8 @@ The
> >  .Ar style
> >  can be
> >  .Cm common ,
> > -.Cm combined
> > +.Cm combined ,
> > +.Cm forwarded
> >  or
> >  .Cm connection .
> >  The styles
> > @@ -459,6 +460,14 @@ and
> >  .Cm combined
> >  write a log entry after each request similar to the standard Apache
> >  and nginx access log formats.
> > +The style
> > +.Cm forwarded
> > +extends the style
> > +.Cm combined
> > +by appending two fields containing the values of the headers
> > +.Ar X-Forwarded-For
> > +and
> > +.Ar X-Forwarded-Port .
> >  The style
> >  .Cm connection
> >  writes a summarized log entry after each connection,
> > Index: usr.sbin/httpd/httpd.h
> > ===
> > RCS file: /cvs/src/usr.sbin/httpd/httpd.h,v
> > retrieving revision 1.143
> > diff -u -p -r1.143 httpd.h
> > --- usr.sbin/httpd/httpd.h  19 Feb 2019 11:37:26 -  1.143
> > +++ usr.sbin/httpd/httpd.h  27 Feb 2019 15:26:48 -
> > @@ -437,7 +437,8 @@ SPLAY_HEAD(client_tree, client);
> >  enum log_format {
> > LOG_FORMAT_COMMON,
> > LOG_FORMAT_COMBINED,
> > -   LOG_FORMAT_CONNECTION
> > +   LOG_FORMAT_CONNECTION,
> > +   LOG_FORMAT_FORWARDED
> >  };
> >  
> >  struct log_file {
> > Index: usr.sbin/httpd/parse.y
> > ===
> > RCS file: /cvs/src/usr.sbin/httpd/parse.y,v
> > retrieving revision 1.110
> > diff -u -p -r1.110 parse.y
> > --- usr.sbin/httpd/parse.y  19 Feb 2019 11:37:26 -  1.110
> > +++ usr.sbin/httpd/parse.y  27 Feb 2019 15:26:48 -
> > @@ -140,7 +140,7 @@ typedef struct {
> >  %token PROTOCOLS REQUESTS ROOT SACK SERVER SOCKET STRIP STYLE SYSLOG 
> > TCP TICKET
> >  %token TIMEOUT TLS TYPE TYPES HSTS MAXAGE SUBDOMAINS DEFAULT PRELOAD 
> > REQUEST
> >  %token ERROR INCLUDE AUTHENTICATE WITH BLOCK DROP RETURN PASS REWRITE
> > -%token CA CLIENT CRL OPTIONAL PARAM
> > +%token CA CLIENT CRL OPTIONAL PARAM FORWARDED
> >  %token   STRING
> >  %token   NUMBER
> >  %type  port
> > @@ -1024,6 +1024,11 @@ logstyle : COMMON{
> > srv_conf->flags |= SRVFLAG_LOG;
> > srv_conf->logformat = LOG_FORMAT_CONNECTION;
> > }
> > +   | FORWARDED {
> > +   srv_conf->flags &= ~SRVFLAG_NO_LOG;
> > +   srv_conf->flags |= SRVFLAG_LOG;
> > +   srv_conf->logformat = LOG_FORMAT_FORWARDED;
> > +   }
> > ;
> >  
> >  filter : block RETURN NUMBER optstring {
> > @@ -1295,6 +1300,7 @@ lookup(char *s)
> > { "ecdhe",  ECDHE },
> > { "error",  ERR },
> > { "fastcgi",FCGI },
> > +   { "forwarded",  FORWARDED },
> > { "hsts",   HSTS },
> > { "include",INCLUDE },
> > { "index",  INDEX },
> > Index: usr.sbin/httpd/server_http.c
> > ===
> > RCS file: /cvs/src/usr.sbin/httpd/server_http.c,v
> > retrieving revision 1.129
> > diff -u -p -r1.129 server_http.c
> > --- usr.sbin/httpd/server_http.c10 Feb 2019 13:41:27 -  1.129
> > +++ usr.sbin/h

Re: perldoc: fix man output & formatting

2019-05-02 Thread Ingo Schwarze
Hi Todd & Andrew,

Andrew Fresh wrote on Thu, May 02, 2019 at 09:53:29AM -0700:
> On Thu, May 02, 2019 at 10:21:15AM -0600, Todd C. Miller wrote:
>> On Thu, 02 May 2019 10:53:37 -0500, Andrew Daugherity wrote:

>>> I reported this to FreeBSD ports a couple months ago [2], and they
>>> provided a fix [3] which repairs the -oMan output, and makes that the
>>> default.  Their fix applies cleanly to the OpenBSD tree and works, but
>>> I have no idea why mandoc requires resetting $?.  (Also, their ToMan
>>> patch has a previously-included hunk for MANWIDTH=tty, but it doesn't
>>> seem to do anything for me?)

>> There problem is a missing waitpid() call, so $? is not actually
>> set.  Something like the following should be better, though I haven't
>> tested it yet.

> This seems to work fine for me, OK afresh1@

OK schwarze@, too.

> I submitted this upstream as well, so hopefully shouldn't have to keep
> the patch for too long.
> https://github.com/mrallen1/Pod-Perldoc/pull/39

Thanks.

> Still looking at defaulting to -oMan, but probably a good idea.

I agree in principle that -oMan ought to become the default.

Yours,
  Ingo


>> Index: gnu/usr.bin/perl/cpan/Pod-Perldoc/lib/Pod/Perldoc/ToMan.pm
>> ===
>> RCS file: 
>> /cvs/src/gnu/usr.bin/perl/cpan/Pod-Perldoc/lib/Pod/Perldoc/ToMan.pm,v
>> retrieving revision 1.8
>> diff -u -p -u -r1.8 ToMan.pm
>> --- gnu/usr.bin/perl/cpan/Pod-Perldoc/lib/Pod/Perldoc/ToMan.pm   13 Feb 
>> 2019 21:15:14 -  1.8
>> +++ gnu/usr.bin/perl/cpan/Pod-Perldoc/lib/Pod/Perldoc/ToMan.pm   2 May 
>> 2019 16:16:05 -
>> @@ -352,6 +352,9 @@ sub _filter_through_nroff {
>>  close $writer;
>>  $self->debug( "Done writing\n" );
>>  
>> +# wait for it to exit
>> +waitpid( $pid, 0 );
>> +
>>  # read any leftovers
>>  $done .= do { local $/; <$reader> };
>>  $self->debug( sprintf "Done reading. Output is %d bytes\n",
>> 


> I wish life had an UNDO function.

I didn't know you were *that* cruel.
All professional Chess and Go players are going to starve
because no game will ever get finished.



Re: httpd: avoid opening log files on "no log"

2019-05-02 Thread Solene Rapenne
On Thu, May 02, 2019 at 10:36:29PM +0200, Klemens Nanni wrote:
> httpd(8) still creates/opens log files with `no log' in httpd.conf(5): 
> 
>  [no] log [option]
>  Set the specified logging options.  Logging is enabled by default
>  using the standard access and error log files, but can be changed
>  per server or location.  Use the no log directive to disable
>  logging of any requests.  Multiple options may be specified
>  within curly braces.  Valid options are:
> 
> I want to quickly serve files with nothing but a config file:
> 
>   # cat /etc/httpd.conf
>   chroot "/foo"
>   server "default" {
>   listen on "*" port www
>   no log
>   root "/"
>   }
>   # httpd -d
>   doas (k...@eru.my.domain) password:
>   startup
>   failed to open /foo/logs/access.log: No such file or directory
>   parent: failed to open log file
>   logger exiting, pid 1150
>   server exiting, pid 42968
>   server exiting, pid 51707
> 
> Diff below fixes this.
> 
> Feedback? OK?

works for me on amd64
a big YES for this!!

it doesn't even require man page change.



httpd: avoid opening log files on "no log"

2019-05-02 Thread Klemens Nanni
httpd(8) still creates/opens log files with `no log' in httpd.conf(5): 

 [no] log [option]
 Set the specified logging options.  Logging is enabled by default
 using the standard access and error log files, but can be changed
 per server or location.  Use the no log directive to disable
 logging of any requests.  Multiple options may be specified
 within curly braces.  Valid options are:

I want to quickly serve files with nothing but a config file:

# cat /etc/httpd.conf
chroot "/foo"
server "default" {
listen on "*" port www
no log
root "/"
}
# httpd -d
doas (k...@eru.my.domain) password:
startup
failed to open /foo/logs/access.log: No such file or directory
parent: failed to open log file
logger exiting, pid 1150
server exiting, pid 42968
server exiting, pid 51707

Diff below fixes this.

Feedback? OK?

Index: usr.sbin/httpd/logger.c
===
RCS file: /cvs/src/usr.sbin/httpd/logger.c,v
retrieving revision 1.21
diff -u -p -r1.21 logger.c
--- usr.sbin/httpd/logger.c 7 Feb 2018 03:28:05 -   1.21
+++ usr.sbin/httpd/logger.c 2 May 2019 20:13:04 -
@@ -197,7 +197,7 @@ logger_open(struct server *srv, struct s
 {
struct log_file *log, *logfile = NULL, *errfile = NULL;
 
-   if (srv_conf->flags & SRVFLAG_SYSLOG)
+   if (srv_conf->flags & (SRVFLAG_SYSLOG | SRVFLAG_NO_LOG))
return (0);
 
/* disassociate */



Re: libevent: endless loop on excessively large buffers

2019-05-02 Thread Nicholas Marriott


ok nicm



On Thu, May 02, 2019 at 06:59:33PM +0200, Tobias Stöckmann wrote:
> It is possible to trigger an endless loop or out of boundary write
> on 64 bit systems with evbuffer_readline calls for buffers which
> exceed 4 GB (i.e. overflow uint).
> 
>   for (i = 0; i < len; i++)
> 
> Variable i is unsigned int and len size_t. This leads to an endless
> loop if len is larger than UINT_MAX.
> 
> If the loop ends exactly at UINT_MAX, then a malloc with additional
> space for an ending '\0' leads to an overflow, effectively allocating
> 0 bytes and therefore an out of boundary write occurs.
> 
> This is a proof of concept for an endless loop:
> 
> #include 
> #include 
> #include 
> #include 
> #include 
> 
> int
> main(void)
> {
>   char *buf;
>   const size_t len = 0x1UL;
>   struct evbuffer *buffer;
>   char *line;
> 
>   if ((buf = malloc(len)) == NULL)
>   err(1, "malloc");
>   memset(buf, 'A', len);
> 
>   if ((buffer = evbuffer_new()) == NULL)
>   errx(1, "evbuffer_new");
>   if (evbuffer_expand(buffer, len + 1))
>   errx(1, "evbuffer_expand");
>   evbuffer_add(buffer, buf, len);
>   evbuffer_add(buffer, "", 1);
> 
>   printf("%p\n", evbuffer_readline(buffer));
>   return 0;
> }
> 
> Generally this is a rather theoretical case. Normal users are not
> allowed to allocate so much memory. But better be safe than sorry,
> especially if login.conf values were adjusted (or the process runs
> as root).
> 
> This patch completely removes "unsigned int" from buffer.c.
> 
> 
> Index: buffer.c
> ===
> RCS file: /cvs/src/lib/libevent/buffer.c,v
> retrieving revision 1.31
> diff -u -p -u -p -r1.31 buffer.c
> --- buffer.c  18 Mar 2017 01:48:43 -  1.31
> +++ buffer.c  1 May 2019 11:00:29 -
> @@ -188,7 +188,7 @@ evbuffer_readline(struct evbuffer *buffe
>   u_char *data = EVBUFFER_DATA(buffer);
>   size_t len = EVBUFFER_LENGTH(buffer);
>   char *line;
> - unsigned int i;
> + size_t i;
> 
>   for (i = 0; i < len; i++) {
>   if (data[i] == '\r' || data[i] == '\n')
> @@ -232,7 +232,7 @@ evbuffer_readln(struct evbuffer *buffer,
>   u_char *start_of_eol, *end_of_eol;
>   size_t len = EVBUFFER_LENGTH(buffer);
>   char *line;
> - unsigned int i, n_to_copy, n_to_drain;
> + size_t i, n_to_copy, n_to_drain;
> 
>   if (n_read_out)
>   *n_read_out = 0;
> 



Re: libevent: endless loop on excessively large buffers

2019-05-02 Thread Ted Unangst
Tobias Stöckmann wrote:
> Generally this is a rather theoretical case. Normal users are not
> allowed to allocate so much memory. But better be safe than sorry,
> especially if login.conf values were adjusted (or the process runs
> as root).
> 
> This patch completely removes "unsigned int" from buffer.c.

yes please.



libevent: endless loop on excessively large buffers

2019-05-02 Thread Tobias Stöckmann
It is possible to trigger an endless loop or out of boundary write
on 64 bit systems with evbuffer_readline calls for buffers which
exceed 4 GB (i.e. overflow uint).

for (i = 0; i < len; i++)

Variable i is unsigned int and len size_t. This leads to an endless
loop if len is larger than UINT_MAX.

If the loop ends exactly at UINT_MAX, then a malloc with additional
space for an ending '\0' leads to an overflow, effectively allocating
0 bytes and therefore an out of boundary write occurs.

This is a proof of concept for an endless loop:

#include 
#include 
#include 
#include 
#include 

int
main(void)
{
char *buf;
const size_t len = 0x1UL;
struct evbuffer *buffer;
char *line;

if ((buf = malloc(len)) == NULL)
err(1, "malloc");
memset(buf, 'A', len);

if ((buffer = evbuffer_new()) == NULL)
errx(1, "evbuffer_new");
if (evbuffer_expand(buffer, len + 1))
errx(1, "evbuffer_expand");
evbuffer_add(buffer, buf, len);
evbuffer_add(buffer, "", 1);

printf("%p\n", evbuffer_readline(buffer));
return 0;
}

Generally this is a rather theoretical case. Normal users are not
allowed to allocate so much memory. But better be safe than sorry,
especially if login.conf values were adjusted (or the process runs
as root).

This patch completely removes "unsigned int" from buffer.c.


Index: buffer.c
===
RCS file: /cvs/src/lib/libevent/buffer.c,v
retrieving revision 1.31
diff -u -p -u -p -r1.31 buffer.c
--- buffer.c18 Mar 2017 01:48:43 -  1.31
+++ buffer.c1 May 2019 11:00:29 -
@@ -188,7 +188,7 @@ evbuffer_readline(struct evbuffer *buffe
u_char *data = EVBUFFER_DATA(buffer);
size_t len = EVBUFFER_LENGTH(buffer);
char *line;
-   unsigned int i;
+   size_t i;

for (i = 0; i < len; i++) {
if (data[i] == '\r' || data[i] == '\n')
@@ -232,7 +232,7 @@ evbuffer_readln(struct evbuffer *buffer,
u_char *start_of_eol, *end_of_eol;
size_t len = EVBUFFER_LENGTH(buffer);
char *line;
-   unsigned int i, n_to_copy, n_to_drain;
+   size_t i, n_to_copy, n_to_drain;

if (n_read_out)
*n_read_out = 0;



Re: perldoc: fix man output & formatting

2019-05-02 Thread Andrew Hewus Fresh
On Thu, May 02, 2019 at 10:21:15AM -0600, Todd C. Miller wrote:
> On Thu, 02 May 2019 10:53:37 -0500, Andrew Daugherity wrote:
> 
> > I reported this to FreeBSD ports a couple months ago [2], and they
> > provided a fix [3] which repairs the -oMan output, and makes that the
> > default.  Their fix applies cleanly to the OpenBSD tree and works, but
> > I have no idea why mandoc requires resetting $?.  (Also, their ToMan
> > patch has a previously-included hunk for MANWIDTH=tty, but it doesn't
> > seem to do anything for me?)
> 
> There problem is a missing waitpid() call, so $? is not actually
> set.  Something like the following should be better, though I haven't
> tested it yet.

This seems to work fine for me, OK afresh1@

I submitted this upstream as well, so hopefully shouldn't have to keep
the patch for too long.
https://github.com/mrallen1/Pod-Perldoc/pull/39

Still looking at defaulting to -oMan, but probably a good idea.


>  - todd
> 
> Index: gnu/usr.bin/perl/cpan/Pod-Perldoc/lib/Pod/Perldoc/ToMan.pm
> ===
> RCS file: 
> /cvs/src/gnu/usr.bin/perl/cpan/Pod-Perldoc/lib/Pod/Perldoc/ToMan.pm,v
> retrieving revision 1.8
> diff -u -p -u -r1.8 ToMan.pm
> --- gnu/usr.bin/perl/cpan/Pod-Perldoc/lib/Pod/Perldoc/ToMan.pm13 Feb 
> 2019 21:15:14 -  1.8
> +++ gnu/usr.bin/perl/cpan/Pod-Perldoc/lib/Pod/Perldoc/ToMan.pm2 May 
> 2019 16:16:05 -
> @@ -352,6 +352,9 @@ sub _filter_through_nroff {
>   close $writer;
>   $self->debug( "Done writing\n" );
>  
> + # wait for it to exit
> + waitpid( $pid, 0 );
> +
>   # read any leftovers
>   $done .= do { local $/; <$reader> };
>   $self->debug( sprintf "Done reading. Output is %d bytes\n",
> 

-- 
andrew - http://afresh1.com

I wish life had an UNDO function.



Re: perldoc: fix man output & formatting

2019-05-02 Thread Todd C . Miller
On Thu, 02 May 2019 10:53:37 -0500, Andrew Daugherity wrote:

> I reported this to FreeBSD ports a couple months ago [2], and they
> provided a fix [3] which repairs the -oMan output, and makes that the
> default.  Their fix applies cleanly to the OpenBSD tree and works, but
> I have no idea why mandoc requires resetting $?.  (Also, their ToMan
> patch has a previously-included hunk for MANWIDTH=tty, but it doesn't
> seem to do anything for me?)

There problem is a missing waitpid() call, so $? is not actually
set.  Something like the following should be better, though I haven't
tested it yet.

 - todd

Index: gnu/usr.bin/perl/cpan/Pod-Perldoc/lib/Pod/Perldoc/ToMan.pm
===
RCS file: /cvs/src/gnu/usr.bin/perl/cpan/Pod-Perldoc/lib/Pod/Perldoc/ToMan.pm,v
retrieving revision 1.8
diff -u -p -u -r1.8 ToMan.pm
--- gnu/usr.bin/perl/cpan/Pod-Perldoc/lib/Pod/Perldoc/ToMan.pm  13 Feb 2019 
21:15:14 -  1.8
+++ gnu/usr.bin/perl/cpan/Pod-Perldoc/lib/Pod/Perldoc/ToMan.pm  2 May 2019 
16:16:05 -
@@ -352,6 +352,9 @@ sub _filter_through_nroff {
close $writer;
$self->debug( "Done writing\n" );
 
+   # wait for it to exit
+   waitpid( $pid, 0 );
+
# read any leftovers
$done .= do { local $/; <$reader> };
$self->debug( sprintf "Done reading. Output is %d bytes\n",



perldoc: fix man output & formatting

2019-05-02 Thread Andrew Daugherity
After upgrading to 6.5 and thus perl 5.28, the man pages displayed by
perldoc (e.g. 'perldoc Digest') or via Perl scripts making use of
pod2usage lack formatting such as bold and underlining.  In fact, text
which used to be underlined is now wrapped in *asterisks* (bold text
is just displayed as normal text.)

This is apparently due to an upstream change in Perl, which changed
the default output from Term to Text. I found a Debian bug report
about it [1], wherein they propose two possible workarounds:
export PERLDOC=-oTerm PERLDOC_PAGER='less -R' # works like Stretch, Jessie
export PERLDOC=-oMan  # works like Wheezy

-oTerm works, even without setting 'less -R'.
-oMan does not work at all with mandoc (and didn't with Perl 5.24 either):

$ perldoc -oMan Digest
Error from pipe to /usr/bin/mandoc!

 at /usr/libdata/perl5/Pod/Perldoc.pm line 1512.
Falling back to Pod because there was a problem!

 at /usr/libdata/perl5/Pod/Perldoc.pm line 1512.
Error while formatting with Pod::Perldoc::ToMan:
 Can't read-open Pod::Perldoc::ToMan=HASH(0x53c3a9d95c8): No such file
or directory
Aborting
 at /usr/libdata/perl5/Pod/Perldoc/ToMan.pm line 407.


 at /usr/bin/perldoc line 13.
Got a 0-length file from /usr/libdata/perl5/Digest.pm via Pod::Perldoc::ToMan!?

 at /usr/bin/perldoc line 13.


I reported this to FreeBSD ports a couple months ago [2], and they
provided a fix [3] which repairs the -oMan output, and makes that the
default.  Their fix applies cleanly to the OpenBSD tree and works, but
I have no idea why mandoc requires resetting $?.  (Also, their ToMan
patch has a previously-included hunk for MANWIDTH=tty, but it doesn't
seem to do anything for me?)

I also don't understand why they included OS checks ('unless
$self->is_mswin32' etc.) in the -oMan default setting, so here is
their patch with only the TERM check:

diff --git gnu/usr.bin/perl/cpan/Pod-Perldoc/lib/Pod/Perldoc.pm
gnu/usr.bin/perl/cpan/Pod-Perldoc/lib/Pod/Perldoc.pm
index bb6ffc83efb..342efbd200a 100644
--- gnu/usr.bin/perl/cpan/Pod-Perldoc/lib/Pod/Perldoc.pm
+++ gnu/usr.bin/perl/cpan/Pod-Perldoc/lib/Pod/Perldoc.pm
@@ -486,6 +486,8 @@ sub init_formatter_class_list {

   $self->opt_M_with('Pod::Perldoc::ToPod');   # the always-there fallthru
   $self->opt_o_with('text');
+  $self->opt_o_with('man') if ($ENV{TERM} && (($ENV{TERM} || '') !~
+/dumb|emacs|none|unknown/i));

   return;
 }
diff --git gnu/usr.bin/perl/cpan/Pod-Perldoc/lib/Pod/Perldoc/ToMan.pm
gnu/usr.bin/perl/cpan/Pod-Perldoc/lib/Pod/Perldoc/ToMan.pm
index bfcb5c40ee6..34a0287dd71 100644
--- gnu/usr.bin/perl/cpan/Pod-Perldoc/lib/Pod/Perldoc/ToMan.pm
+++ gnu/usr.bin/perl/cpan/Pod-Perldoc/lib/Pod/Perldoc/ToMan.pm
@@ -310,6 +310,9 @@ sub _filter_through_nroff {
  require IPC::Open3;
  require IO::Handle;

+ # Reset this???
+ $?=0;
+
  my $pid = IPC::Open3::open3(
  my $writer,
  my $reader,


-Andrew

[1] https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=917530
[2] https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=236092
[3] https://svnweb.freebsd.org/ports?view=revision&revision=495738



kqueue.2: formatting fixes and minor HISTORY expansion

2019-05-02 Thread Fabio Scotoni
I've taken a stab at improving kqueue.2 formatting.

Most of the changes are markup fixes.
I used ".Dv NULL" over plain "null" in accordance with
lib/libc/stdlib/malloc.3 rev. 1.113.

I also added a note to the HISTORY section that kqueue()/kevent() have
been available in OpenBSD since 2.9;
the wording matches growfs(8).

I'm not sure how to handle the undocumented EPERM that is returned if a
pledge(2) does not include "proc" when an attempt is made to attach to a
process with EVFILT_PROC.
It does feel somewhat non-obvious, but is noted in neither pledge(2) nor
kqueue(2).

Index: lib/libc/sys/kqueue.2
===
RCS file: /cvs/src/lib/libc/sys/kqueue.2,v
retrieving revision 1.37
diff -u -p -u -p -r1.37 kqueue.2
--- lib/libc/sys/kqueue.2   13 Jan 2018 17:13:12 -  1.37
+++ lib/libc/sys/kqueue.2   2 May 2019 13:28:27 -
@@ -67,9 +67,9 @@ is not returned.
 Multiple events which trigger the filter do not result in multiple
 kevents being placed on the kqueue; instead, the filter will aggregate
 the events into a single
-.Li struct kevent .
+.Vt struct kevent .
 Calling
-.Fn close
+.Xr close 2
 on a file descriptor will remove any kevents that reference the descriptor.
 .Pp
 .Fn kqueue
@@ -83,8 +83,8 @@ is used to register events with the queu
 events to the user.
 .Fa changelist
 is a pointer to an array of
-.Va kevent
-structures, as defined in
+.Vt struct kevent ,
+as defined in
 .In sys/event.h .
 All changes contained in the
 .Fa changelist
@@ -107,19 +107,23 @@ specified unlike
 .Xr select 2 .
 If
 .Fa timeout
-is a non-null pointer, it specifies a maximum interval to wait
+is not
+.Dv NULL ,
+it specifies a maximum interval to wait
 for an event, which will be interpreted as a
-.Li struct timespec .
+.Vt struct timespec .
 If
 .Fa timeout
-is a null pointer,
+is
+.Dv NULL ,
 .Fn kevent
 waits indefinitely.
 To effect a poll, the
 .Fa timeout
-argument should be non-null, pointing to a zero-valued
-.Va timespec
-structure.
+argument should not be
+.Dv NULL ,
+pointing to a zero-valued
+.Vt struct timespec .
 The same array may be used for the
 .Fa changelist
 and
@@ -130,7 +134,7 @@ is a macro which is provided for ease of
 kevent structure.
 .Pp
 The
-.Va kevent
+.Vt kevent
 structure is defined as:
 .Bd -literal
 struct kevent {
@@ -144,28 +148,28 @@ struct kevent {
 .Ed
 .Pp
 The fields of
-.Li struct kevent
+.Vt struct kevent
 are:
 .Bl -tag -width XXXfilter
-.It ident
+.It Fa ident
 Value used to identify this event.
 The exact interpretation is determined by the attached filter,
 but often is a file descriptor.
-.It filter
+.It Fa filter
 Identifies the kernel filter used to process this event.
 The pre-defined system filters are described below.
-.It flags
+.It Fa flags
 Actions to perform on the event.
-.It fflags
+.It Fa fflags
 Filter-specific flags.
-.It data
+.It Fa data
 Filter-specific data value.
-.It udata
+.It Fa udata
 Opaque user-defined value passed through the kernel unchanged.
 .El
 .Pp
 The
-.Va flags
+.Fa flags
 field can contain the following values:
 .Bl -tag -width XXXEV_ONESHOT
 .It Dv EV_ADD
@@ -200,7 +204,7 @@ to return with
 .Dv EV_ERROR
 set without draining any pending events after updating events in the kqueue.
 When a filter is successfully added the
-.Va data
+.Fa data
 field will be zero.
 This flag is useful for making bulk changes to a kqueue.
 .It Dv EV_ONESHOT
@@ -222,9 +226,9 @@ below.
 .Pp
 The predefined system filters are listed below.
 Arguments may be passed to and from the filter via the
-.Va fflags
+.Fa fflags
 and
-.Va data
+.Fa data
 fields in the kevent structure.
 .Bl -tag -width EVFILT_SIGNAL
 .It Dv EVFILT_READ
@@ -235,9 +239,9 @@ on the descriptor type.
 .Bl -tag -width 2n
 .It Sockets
 Sockets which have previously been passed to
-.Fn listen
+.Xr listen 2
 return when there is an incoming connection pending.
-.Va data
+.Fa data
 contains the size of the listen backlog.
 .Pp
 Other socket descriptors return when there is data to be read,
@@ -248,47 +252,47 @@ This may be overridden with a per-filter
 time the filter is added by setting the
 .Dv NOTE_LOWAT
 flag in
-.Va fflags ,
+.Fa fflags ,
 and specifying the new low water mark in
-.Va data .
+.Fa data .
 On return,
-.Va data
+.Fa data
 contains the number of bytes in the socket buffer.
 .Pp
 If the read direction of the socket has shutdown, then the filter
 also sets
 .Dv EV_EOF
 in
-.Va flags ,
+.Fa flags ,
 and returns the socket error (if any) in
-.Va fflags .
+.Fa fflags .
 It is possible for EOF to be returned (indicating the connection is gone)
 while there is still data pending in the socket buffer.
 .It Vnodes
 Returns when the file pointer is not at the end of file.
-.Va data
+.Fa data
 contains the offset from current position to end of file,
 and may be negative.
 If
 .Dv NOTE_EOF
 is set in
-.Va fflags ,
+.Fa fflags ,
 .Fn kevent
 will also return when the file pointer is at the end of file.
 The end of file con

Re: patch axen(4) (WIP)

2019-05-02 Thread Nils Frohberg
Any there further interest in these patches? I was running these
since 61 days, doing lots of rsync and TimeMachine (netatalk)
backups. Didn't have any crashes or (noticeable) network dropouts.

$ uptime
11:05AM  up 61 days, 16:54, 1 user, load averages: 0.35, 0.38, 0.38
$ 

On Thu, Feb 28, 2019 at 07:53:39AM +0100, Nils Frohberg wrote:
> On Tue, Feb 26, 2019 at 08:57:57PM +0200, Artturi Alm wrote:
> > On Tue, Feb 26, 2019 at 03:00:15PM +0100, Nils Frohberg wrote:
> > > On Mon, Feb 25, 2019 at 03:50:48PM -0300, Martin Pieuchot wrote:
> > > > On 25/02/19(Mon) 14:52, Nils Frohberg wrote:
> > > > > Hi,
> > > > > 
> > > > > as mentioned previously, I'm looking into axen(4). While searching
> > > > > for the cause of a panic (fixed since, thanks mpi@) I started to
> > > > > rewrite parts of the driver. References were mainly the FreeBSD and
> > > > > Linux drivers.
> > > > 
> > > > Please try to isolate parts of your diff that fixes issues and cosmetic
> > > > changes.  The simpler it gets the easier it is for us to review it.
> > > 
> > > Sure, I'll send separate patches.
> > > 
> > > (I cvs up'ed my src forder in order to test compile the individual
> > > patches and found out that xhci.c,v1.91 breaks my USB devices. I'll
> > > send a mail to bugs@ later.)
> > > 
> > > > > I didn't get around to much testing/debugging lately, therefore I
> > > > > wanted to share the current state (diff below).
> > > > > 
> > > > > The current state works a lot better than previously (for me). I
> > > > > used to have a huge amount of ierrs (aprrox. 1 ierr per ipkt) and
> > > > > often no packets would be transferred at all (or stop being 
> > > > > transferred
> > > > > after some time).
> > > > 
> > > > Do you know why?  What were the problems?
> > > 
> > > I'm not 100% sure, since I did a lot of back and forth. It finally
> > > got better once I disabled EEE and lowered the watermark levels.
> > > 
> > 
> > Have you looked at what NetBSD has done with their axen(4)? there has
> > been 20commits in 2019 so far[0], while some of them are possibly,
> > idk., useless to us(thinking about hw checksum offloading), there was
> > some bug fixes that did look relevant to me, but i succesfully
> > installed&built kernels on nfs over axen(4) a couple of weeks ago,
> > so the bugs it has didn't feel critical enough for me to make
> > a branch for them. that was on arm64/dwctwo(4), tbh. i haven't been
> > happy with axen(4) on amd64/{e,x}hci(4) myself in the past either. :]
> 
> I wrote the diff last December. I looked at NetBSD's code back then, 
> but they didn't have any significant changes.
> 
> At a cursory glance, many changes are similar to mine. But there are 
> a few things that should be worth looking at.
> 
> > I guess i'm trying to say maybe it wouldn't hurt to sync a bit before
> > deviating as much as atleast your whole WIP diff did. I haven't read
> > your separate patches yet, but i'll try to get around to also testing
> > those before weekend:]
> 
> The separate patches are (more or less) the big patch split up via 
> $(git add -p). There are a few things that might be worth looking 
> into, such as the pause water levels, enabling EEE, axen_bulk_size 
> values, buffer sizes, ...
> 
> More testing would be great. Especially since this is the only box 
> I have where I can attach it to xhci.
> 
> > -Artturi
> > 
> > [0] https://github.com/NetBSD/src/commits/trunk/sys/dev/usb/if_axen.c
> > 
>