Re: Problem with re(4) Ethernet driver has resurfaced in 11-STABLE and HEAD

[O. Hartmann]

On Mon, 22 May 2017 10:55:46 +0900
YongHyeon PYUN  wrote:


Just for my curiosity: do you have "options FLOWTABLE" defined in your kernel
config?

> On Sun, May 21, 2017 at 09:29:42AM +, Thomas Mueller wrote:
> > from YongHyeon PYUN:
> >   
> > > [removed stable@ from CC]  
> >   
> > > > I recently updated my 10.1-STABLE to 11.0-STABLE and find I can no
> > > > longer connect with the Ethernet.  
> >   
> > > > dhclient re0 produces  
> >   
> > > > DHCPDISCOVER on re0 to 255.255.255.255 port 67 interval 4
> > > > DHCPDISCOVER on re0 to 255.255.255.255 port 67 interval 11
> > > > DHCPDISCOVER on re0 to 255.255.255.255 port 67 interval 19
> > > > DHCPDISCOVER on re0 to 255.255.255.255 port 67 interval 7
> > > > DHCPDISCOVER on re0 to 255.255.255.255 port 67 interval 14
> > > > DHCPDISCOVER on re0 to 255.255.255.255 port 67 interval 5
> > > > No DHCPOFFERS received.
> > > > No working leases in persistent database - sleeping.  
> > 
> >   
> > > If you assign an static IPv4 address to re(4) are you able to use
> > > the network interface?  
> >   
> > > AFAIK there was no significant re(4) changes for a long time. Could
> > > you show us back trace information?  
> > 
> > Problem with re(4) reappeared in both 11.0-STABLE and HEAD, but OK to trim
> > stable@ since changes/fixes would go to HEAD first.
> > 
> > No connection with static IPv4 address. 
> > 
> > Where do I get back trace information?
> >   
> 
> https://www.freebsd.org/doc/en/books/developers-handbook/kerneldebug.html
> 
> > Problem was more severe with HEAD in that OS immediately crashed into
> > debugger, while in 11.0-STABLE, only the connection failed but may have
> > left memory unstable. 
> 
> I guess you have two issues. No connection with static IPv4 address
> and kernel crash.  Couldn't you obtain a kernel crash dump when you
> encounter kernel panic? If this is no crash dump you probably have
> some missing kernel configuration.
> 
> > I can still connect on that computer with Hiro H50191 USB wireless adapter,
> > driver rsu.
> > 
> > Tom
> >   
> ___
> freebsd-current@freebsd.org mailing list
> https://lists.freebsd.org/mailman/listinfo/freebsd-current
> To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"

___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"

Re: Problem with re(4) Ethernet driver has resurfaced in 11-STABLE and HEAD

[YongHyeon PYUN]

On Sun, May 21, 2017 at 09:29:42AM +, Thomas Mueller wrote:
> from YongHyeon PYUN:
> 
> > [removed stable@ from CC]
> 
> > > I recently updated my 10.1-STABLE to 11.0-STABLE and find I can no longer 
> > > connect with the Ethernet.
> 
> > > dhclient re0 produces
> 
> > > DHCPDISCOVER on re0 to 255.255.255.255 port 67 interval 4
> > > DHCPDISCOVER on re0 to 255.255.255.255 port 67 interval 11
> > > DHCPDISCOVER on re0 to 255.255.255.255 port 67 interval 19
> > > DHCPDISCOVER on re0 to 255.255.255.255 port 67 interval 7
> > > DHCPDISCOVER on re0 to 255.255.255.255 port 67 interval 14
> > > DHCPDISCOVER on re0 to 255.255.255.255 port 67 interval 5
> > > No DHCPOFFERS received.
> > > No working leases in persistent database - sleeping.
> 
> 
> > If you assign an static IPv4 address to re(4) are you able to use
> > the network interface?
> 
> > AFAIK there was no significant re(4) changes for a long time. Could
> > you show us back trace information?
> 
> Problem with re(4) reappeared in both 11.0-STABLE and HEAD, but OK to trim 
> stable@ since changes/fixes would go to HEAD first.
> 
> No connection with static IPv4 address. 
> 
> Where do I get back trace information?
> 

https://www.freebsd.org/doc/en/books/developers-handbook/kerneldebug.html

> Problem was more severe with HEAD in that OS immediately crashed into 
> debugger, while in 11.0-STABLE, only the connection failed but may have left 
> memory unstable.
> 

I guess you have two issues. No connection with static IPv4 address
and kernel crash.  Couldn't you obtain a kernel crash dump when you
encounter kernel panic? If this is no crash dump you probably have
some missing kernel configuration.

> I can still connect on that computer with Hiro H50191 USB wireless adapter, 
> driver rsu.
> 
> Tom
> 
___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"

Re: The future of the roff toolchain

[Mike Karels]

On 21 May 2017, at 15:36, Benjamin Kaduk wrote:


On Sun, May 21, 2017 at 02:57:33PM +0200, Baptiste Daroussin wrote:


What I want to propose now, it to render them as PDF (html?) once and 
push them
somewhere (to be defined) as static document on our documentation 
website.

Please doceng@ provide me a location where to push them.

And then remove bsd.doc.mk from FreeBSD 12.0 along with the removal 
of groff.
I also want to remove most of roff related tools (the one provided by 
toolchains
available in ports) for which we kept a BSD version (not really 
maintained in

base):
namely:
- checknr
- vgrind
- colcrt

Only keeping:
- col (useful in other places than roff)
- soelim (also used for manpages and we have a clean BSD licensed 
version which

  is also now parts of mandoc)


That sounds like a good plan to me.  Thank you for putting in all
the time trying out the alternate roff toolchains!


+1.  I agree that /usr/share/doc is strictly legacy; only the man pages 
matter for base.



___
freebsd-a...@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-arch
To unsubscribe, send any mail to 
"freebsd-arch-unsubscr...@freebsd.org"

___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"

Re: The futur of the roff toolchain

[Eric van Gyzen]

I like all of this.  Thanks for your very thorough research and effort.

Eric


On 05/21/2017 07:57, Baptiste Daroussin wrote:
> Hi all,
>
> I have been working for a while to try to import a modern roff toolchain into
> base.
>
> I didn't like the initial approach that consisted in simply removing all roff
> toolchain in base.
>
> Recap of the situation in base:
> * We have GNU roff version 1.19.2 in base (latest GPLv2 version). Lots of bug
>   fixes has been made upstream in newer version (GPLv3) in particular 
> regarding
>   unicode but not only. (and we cannot update it anymore)
> * GNU roff is now only used to generate the documentation in share/doc and as 
> a
>   fallback for manpages which mandoc does not support.
>
> On the manpages front:
> * No manpages in base are not supported by mandoc except groff manpages
>   themselves
> * man(1) can fallback on ports version of groff if installed (for ports not
>   providing manpages not compatible with mandoc)
>
> Alternatives to GNU roff:
> * Heirloom doctools (which I tried to import) licensed both CDDL/BSD (in C)
> * neartoff http://litcave.rudi.ir/ BSD licensed (in C)
>
> I went the road of using heirloom doctools it is 90% compatible with GNU roff,
> good enough for all our base roff based documents.
>
> After getting down that road for a while, including lots of patches sent
> upstream (thanks them for being so reactive and integrating them quickly as 
> well
> as fixing the issues I wasn't able to fix myself quickly).
>
> The problem is there are lot of corner small corner cases where heirloom is
> different from GNU roff and hard to make it compatible. While this is corner
> cases, it breaks document generation for some large documents people are
> writing. Those users could use (and actually would benefit a lot from it) GNU
> roff from the ports tree, but have to be careful about the path of the tool 
> they
> call to ensure only calling the one from GNU roff and not the one (with the 
> same
> name) from heirloom doctools.
>
> Concerning neatroff it is barely compatible with GNU roff, so not an option
> (last I tested at least).
>
> I would like to change this approach and get back to the initial approach 
> taken
> by others before I jumped in and I would like just entirely remove the roff
> toolchain from base and let people rely on GNU roff from ports.
>
> man(1) is already asking the user to install groff from ports if the manpage
> cannot be read with mandoc.
>
> No the problem left is documentations available in share/doc.
>
> I would like to push them elsewhere. Those documents are mostly useful for
> historical reason (hence we want to keep them) but not really for daily use of
> modern FreeBSD.
> Another issue with those documentation, they are installed as text/ascii 
> version
> in base, which makes most of them not really readable (as the documents has 
> not
> be written for a ascii/text target but more for a PDF/html view - using pic(1)
> for example)
>
> A plan was to push as sources in the svn doc repository and continue to build
> them. This approach also have an issue: over the time roff evolved a bit and
> while working on heirloom doctools import I had to fix a bunch of markup to 
> make
> the rendering of those documents clean (also meaning almost noone should read
> them considering some were not really readable).
>
> What I want to propose now, it to render them as PDF (html?) once and push 
> them
> somewhere (to be defined) as static document on our documentation website.
> Please doceng@ provide me a location where to push them.
>
> And then remove bsd.doc.mk from FreeBSD 12.0 along with the removal of groff.
> I also want to remove most of roff related tools (the one provided by 
> toolchains
> available in ports) for which we kept a BSD version (not really maintained in
> base):
> namely:
> - checknr
> - vgrind
> - colcrt
>
> Only keeping:
> - col (useful in other places than roff)
> - soelim (also used for manpages and we have a clean BSD licensed version 
> which
>   is also now parts of mandoc)
>
> Best regards,
> Bapt

___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"

Re: 64-bit inodes (ino64) Status Update and Call for Testing

[Jilles Tjoelker]

On Sun, May 21, 2017 at 05:25:35PM +0300, Konstantin Belousov wrote:
> On Sun, May 21, 2017 at 04:03:55PM +0200, Jilles Tjoelker wrote:
> > On Sun, May 21, 2017 at 03:31:18PM +0300, Konstantin Belousov wrote:
> > > On Sun, May 21, 2017 at 02:14:56PM +0200, Jilles Tjoelker wrote:
> > > > We have another type in this area which is too small in some situations:
> > > > uint8_t for struct dirent.d_namlen. For filesystems that store filenames
> > > > as upto 255 UTF-16 code units, the name to be stored in d_name may be
> > > > upto 765 bytes long in UTF-8. This was reported in PR 204643. The code
> > > > currently handles this by returning the short (8.3) name, but this name
> > > > may not be present or usable, leaving the file inaccessible.

> > > > Actually allowing longer names seems too complicated to add to the ino64
> > > > change, but changing d_namlen to uint16_t (using d_pad0 space) and
> > > > skipping entries with d_namlen > 255 in libc may be helpful.

> > > > Note that applications using the deprecated readdir_r() will not be able
> > > > to read such long names, since the API does not allow specifying that a
> > > > larger buffer has been provided. (This could be avoided by making struct
> > > > dirent.d_name 766 bytes long instead of 256.)

> > > > Unfortunately, the existence of readdir_r() also prevents changing
> > > > struct dirent.d_name to the more correct flexible array.

> > > Yes, changing the size of d_name at this stage of the project is out of
> > > question. My reading of your proposal is that we should extend the size
> > > of d_namlen to uint16_t, am I right ? Should we go to 32bit directly
> > > then, perhaps ?

> > Yes, my proposal is to change d_namlen to uint16_t.

> > Making it 32 bits is not useful with the 16-bit d_reclen, and increasing
> > d_reclen does not seem useful to me with the current model of
> > getdirentries() where the whole dirent must fit into the caller's
> > buffer.

> Bumping it now might cause less churn later, even if unused, but ok.

> > > I did not committed the change below, nor did I tested or even build it.

> > I'd like to skip overlong names in the native readdir_r() as well, so
> > that long name support can be added to the kernel later without causing
> > buffer overflows with applications using FreeBSD 12.0 libc.

> > The native readdir() does not seem to have such a problem.

> Again, not even compiled.

Looks good to me.

> [patch snipped]

-- 
Jilles Tjoelker
___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"

Re: The future of the roff toolchain

[Benjamin Kaduk]

On Sun, May 21, 2017 at 02:57:33PM +0200, Baptiste Daroussin wrote:
> 
> What I want to propose now, it to render them as PDF (html?) once and push 
> them
> somewhere (to be defined) as static document on our documentation website.
> Please doceng@ provide me a location where to push them.
> 
> And then remove bsd.doc.mk from FreeBSD 12.0 along with the removal of groff.
> I also want to remove most of roff related tools (the one provided by 
> toolchains
> available in ports) for which we kept a BSD version (not really maintained in
> base):
> namely:
> - checknr
> - vgrind
> - colcrt
> 
> Only keeping:
> - col (useful in other places than roff)
> - soelim (also used for manpages and we have a clean BSD licensed version 
> which
>   is also now parts of mandoc)

That sounds like a good plan to me.  Thank you for putting in all
the time trying out the alternate roff toolchains!

-Ben
___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"

Re: The futur of the roff toolchain

[Joel Dahl]

On Sun, May 21, 2017 at 02:57:33PM +0200, Baptiste Daroussin wrote:
> 
> I would like to change this approach and get back to the initial approach 
> taken
> by others before I jumped in and I would like just entirely remove the roff
> toolchain from base and let people rely on GNU roff from ports.
> 
> What I want to propose now, it to render them as PDF (html?) once and push 
> them
> somewhere (to be defined) as static document on our documentation website.
> Please doceng@ provide me a location where to push them.
> 
> And then remove bsd.doc.mk from FreeBSD 12.0 along with the removal of groff.

Yes. I definitely support these ideas. :-)

-- 
Joel
___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"

Re: 64-bit inodes (ino64) Status Update and Call for Testing

[Konstantin Belousov]

On Sun, May 21, 2017 at 04:03:55PM +0200, Jilles Tjoelker wrote:
> On Sun, May 21, 2017 at 03:31:18PM +0300, Konstantin Belousov wrote:
> > On Sun, May 21, 2017 at 02:14:56PM +0200, Jilles Tjoelker wrote:
> > > We have another type in this area which is too small in some situations:
> > > uint8_t for struct dirent.d_namlen. For filesystems that store filenames
> > > as upto 255 UTF-16 code units, the name to be stored in d_name may be
> > > upto 765 bytes long in UTF-8. This was reported in PR 204643. The code
> > > currently handles this by returning the short (8.3) name, but this name
> > > may not be present or usable, leaving the file inaccessible.
> 
> > > Actually allowing longer names seems too complicated to add to the ino64
> > > change, but changing d_namlen to uint16_t (using d_pad0 space) and
> > > skipping entries with d_namlen > 255 in libc may be helpful.
> 
> > > Note that applications using the deprecated readdir_r() will not be able
> > > to read such long names, since the API does not allow specifying that a
> > > larger buffer has been provided. (This could be avoided by making struct
> > > dirent.d_name 766 bytes long instead of 256.)
> 
> > > Unfortunately, the existence of readdir_r() also prevents changing
> > > struct dirent.d_name to the more correct flexible array.
> 
> > Yes, changing the size of d_name at this stage of the project is out of
> > question. My reading of your proposal is that we should extend the size
> > of d_namlen to uint16_t, am I right ? Should we go to 32bit directly
> > then, perhaps ?
> 
> Yes, my proposal is to change d_namlen to uint16_t.
> 
> Making it 32 bits is not useful with the 16-bit d_reclen, and increasing
> d_reclen does not seem useful to me with the current model of
> getdirentries() where the whole dirent must fit into the caller's
> buffer.
Bumping it now might cause less churn later, even if unused, but ok.

> 
> > I did not committed the change below, nor did I tested or even build it.
> 
> I'd like to skip overlong names in the native readdir_r() as well, so
> that long name support can be added to the kernel later without causing
> buffer overflows with applications using FreeBSD 12.0 libc.
> 
> The native readdir() does not seem to have such a problem.

Again, not even compiled.

diff --git a/lib/libc/gen/readdir-compat11.c b/lib/libc/gen/readdir-compat11.c
index 1c52f563c75..a865ab9157e 100644
--- a/lib/libc/gen/readdir-compat11.c
+++ b/lib/libc/gen/readdir-compat11.c
@@ -41,6 +41,7 @@ __FBSDID("$FreeBSD$");
 #define_WANT_FREEBSD11_DIRENT
 #include 
 #include 
+#include 
 #include 
 #include 
 #include 
@@ -53,10 +54,12 @@ __FBSDID("$FreeBSD$");
 
 #include "gen-compat.h"
 
-static void
+static bool
 freebsd11_cvtdirent(struct freebsd11_dirent *dstdp, struct dirent *srcdp)
 {
 
+   if (srcdp->d_namelen >= sizeof(dstdp->d_name))
+   return (false);
dstdp->d_type = srcdp->d_type;
dstdp->d_namlen = srcdp->d_namlen;
dstdp->d_fileno = srcdp->d_fileno;  /* truncate */
@@ -65,6 +68,7 @@ freebsd11_cvtdirent(struct freebsd11_dirent *dstdp, struct 
dirent *srcdp)
bzero(dstdp->d_name + dstdp->d_namlen,
dstdp->d_reclen - offsetof(struct freebsd11_dirent, d_name) -
dstdp->d_namlen);
+   return (true);
 }
 
 struct freebsd11_dirent *
@@ -75,13 +79,15 @@ freebsd11_readdir(DIR *dirp)
 
if (__isthreaded)
_pthread_mutex_lock(>dd_lock);
-   dp = _readdir_unlocked(dirp, 1);
+   dp = _readdir_unlocked(dirp, RDU_SKIP);
if (dp != NULL) {
if (dirp->dd_compat_de == NULL)
dirp->dd_compat_de = malloc(sizeof(struct
freebsd11_dirent));
-   freebsd11_cvtdirent(dirp->dd_compat_de, dp);
-   dstdp = dirp->dd_compat_de;
+   if (freebsd11_cvtdirent(dirp->dd_compat_de, dp))
+   dstdp = dirp->dd_compat_de;
+   else
+   dstdp = NULL;
} else
dstdp = NULL;
if (__isthreaded)
@@ -101,8 +107,10 @@ freebsd11_readdir_r(DIR *dirp, struct freebsd11_dirent 
*entry,
if (error != 0)
return (error);
if (xresult != NULL) {
-   freebsd11_cvtdirent(entry, );
-   *result = entry;
+   if (freebsd11_cvtdirent(entry, ))
+   *result = entry;
+   else /* should not happen due to RDU_SHORT */
+   *result = NULL;
} else
*result = NULL;
return (0);
diff --git a/lib/libc/gen/readdir.c b/lib/libc/gen/readdir.c
index dc7b0df18b2..c30d48273b8 100644
--- a/lib/libc/gen/readdir.c
+++ b/lib/libc/gen/readdir.c
@@ -49,7 +49,7 @@ __FBSDID("$FreeBSD$");
  * get next entry in a directory.
  */
 struct dirent *
-_readdir_unlocked(DIR *dirp, int skip)
+_readdir_unlocked(DIR *dirp, int flags)
 {
struct dirent *dp;
long 

Re: 64-bit inodes (ino64) Status Update and Call for Testing

[Jilles Tjoelker]

On Sun, May 21, 2017 at 03:31:18PM +0300, Konstantin Belousov wrote:
> On Sun, May 21, 2017 at 02:14:56PM +0200, Jilles Tjoelker wrote:
> > We have another type in this area which is too small in some situations:
> > uint8_t for struct dirent.d_namlen. For filesystems that store filenames
> > as upto 255 UTF-16 code units, the name to be stored in d_name may be
> > upto 765 bytes long in UTF-8. This was reported in PR 204643. The code
> > currently handles this by returning the short (8.3) name, but this name
> > may not be present or usable, leaving the file inaccessible.

> > Actually allowing longer names seems too complicated to add to the ino64
> > change, but changing d_namlen to uint16_t (using d_pad0 space) and
> > skipping entries with d_namlen > 255 in libc may be helpful.

> > Note that applications using the deprecated readdir_r() will not be able
> > to read such long names, since the API does not allow specifying that a
> > larger buffer has been provided. (This could be avoided by making struct
> > dirent.d_name 766 bytes long instead of 256.)

> > Unfortunately, the existence of readdir_r() also prevents changing
> > struct dirent.d_name to the more correct flexible array.

> Yes, changing the size of d_name at this stage of the project is out of
> question. My reading of your proposal is that we should extend the size
> of d_namlen to uint16_t, am I right ? Should we go to 32bit directly
> then, perhaps ?

Yes, my proposal is to change d_namlen to uint16_t.

Making it 32 bits is not useful with the 16-bit d_reclen, and increasing
d_reclen does not seem useful to me with the current model of
getdirentries() where the whole dirent must fit into the caller's
buffer.

> I did not committed the change below, nor did I tested or even build it.

I'd like to skip overlong names in the native readdir_r() as well, so
that long name support can be added to the kernel later without causing
buffer overflows with applications using FreeBSD 12.0 libc.

The native readdir() does not seem to have such a problem.

> [patch snipped]

-- 
Jilles Tjoelker
___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"

The futur of the roff toolchain

[Baptiste Daroussin]

Hi all,

I have been working for a while to try to import a modern roff toolchain into
base.

I didn't like the initial approach that consisted in simply removing all roff
toolchain in base.

Recap of the situation in base:
* We have GNU roff version 1.19.2 in base (latest GPLv2 version). Lots of bug
  fixes has been made upstream in newer version (GPLv3) in particular regarding
  unicode but not only. (and we cannot update it anymore)
* GNU roff is now only used to generate the documentation in share/doc and as a
  fallback for manpages which mandoc does not support.

On the manpages front:
* No manpages in base are not supported by mandoc except groff manpages
  themselves
* man(1) can fallback on ports version of groff if installed (for ports not
  providing manpages not compatible with mandoc)

Alternatives to GNU roff:
* Heirloom doctools (which I tried to import) licensed both CDDL/BSD (in C)
* neartoff http://litcave.rudi.ir/ BSD licensed (in C)

I went the road of using heirloom doctools it is 90% compatible with GNU roff,
good enough for all our base roff based documents.

After getting down that road for a while, including lots of patches sent
upstream (thanks them for being so reactive and integrating them quickly as well
as fixing the issues I wasn't able to fix myself quickly).

The problem is there are lot of corner small corner cases where heirloom is
different from GNU roff and hard to make it compatible. While this is corner
cases, it breaks document generation for some large documents people are
writing. Those users could use (and actually would benefit a lot from it) GNU
roff from the ports tree, but have to be careful about the path of the tool they
call to ensure only calling the one from GNU roff and not the one (with the same
name) from heirloom doctools.

Concerning neatroff it is barely compatible with GNU roff, so not an option
(last I tested at least).

I would like to change this approach and get back to the initial approach taken
by others before I jumped in and I would like just entirely remove the roff
toolchain from base and let people rely on GNU roff from ports.

man(1) is already asking the user to install groff from ports if the manpage
cannot be read with mandoc.

No the problem left is documentations available in share/doc.

I would like to push them elsewhere. Those documents are mostly useful for
historical reason (hence we want to keep them) but not really for daily use of
modern FreeBSD.
Another issue with those documentation, they are installed as text/ascii version
in base, which makes most of them not really readable (as the documents has not
be written for a ascii/text target but more for a PDF/html view - using pic(1)
for example)

A plan was to push as sources in the svn doc repository and continue to build
them. This approach also have an issue: over the time roff evolved a bit and
while working on heirloom doctools import I had to fix a bunch of markup to make
the rendering of those documents clean (also meaning almost noone should read
them considering some were not really readable).

What I want to propose now, it to render them as PDF (html?) once and push them
somewhere (to be defined) as static document on our documentation website.
Please doceng@ provide me a location where to push them.

And then remove bsd.doc.mk from FreeBSD 12.0 along with the removal of groff.
I also want to remove most of roff related tools (the one provided by toolchains
available in ports) for which we kept a BSD version (not really maintained in
base):
namely:
- checknr
- vgrind
- colcrt

Only keeping:
- col (useful in other places than roff)
- soelim (also used for manpages and we have a clean BSD licensed version which
  is also now parts of mandoc)

Best regards,
Bapt


signature.asc
Description: PGP signature

Re: 64-bit inodes (ino64) Status Update and Call for Testing

[Konstantin Belousov]

On Sun, May 21, 2017 at 02:14:56PM +0200, Jilles Tjoelker wrote:
> We have another type in this area which is too small in some situations:
> uint8_t for struct dirent.d_namlen. For filesystems that store filenames
> as upto 255 UTF-16 code units, the name to be stored in d_name may be
> upto 765 bytes long in UTF-8. This was reported in PR 204643. The code
> currently handles this by returning the short (8.3) name, but this name
> may not be present or usable, leaving the file inaccessible.
> 
> Actually allowing longer names seems too complicated to add to the ino64
> change, but changing d_namlen to uint16_t (using d_pad0 space) and
> skipping entries with d_namlen > 255 in libc may be helpful.
> 
> Note that applications using the deprecated readdir_r() will not be able
> to read such long names, since the API does not allow specifying that a
> larger buffer has been provided. (This could be avoided by making struct
> dirent.d_name 766 bytes long instead of 256.)
> 
> Unfortunately, the existence of readdir_r() also prevents changing
> struct dirent.d_name to the more correct flexible array.

Yes, changing the size of d_name at this stage of the project is out of
question. My reading of your proposal is that we should extend the size
of d_namlen to uint16_t, am I right ? Should we go to 32bit directly
then, perhaps ?

I did not committed the change below, nor did I tested or even build it.

diff --git a/lib/libc/gen/readdir-compat11.c b/lib/libc/gen/readdir-compat11.c
index 1c52f563c75..18d85adaa63 100644
--- a/lib/libc/gen/readdir-compat11.c
+++ b/lib/libc/gen/readdir-compat11.c
@@ -41,6 +41,7 @@ __FBSDID("$FreeBSD$");
 #define_WANT_FREEBSD11_DIRENT
 #include 
 #include 
+#include 
 #include 
 #include 
 #include 
@@ -53,10 +54,12 @@ __FBSDID("$FreeBSD$");
 
 #include "gen-compat.h"
 
-static void
+static bool
 freebsd11_cvtdirent(struct freebsd11_dirent *dstdp, struct dirent *srcdp)
 {
 
+   if (srcdp->d_namelen >= sizeof(dstdp->d_name))
+   return (false);
dstdp->d_type = srcdp->d_type;
dstdp->d_namlen = srcdp->d_namlen;
dstdp->d_fileno = srcdp->d_fileno;  /* truncate */
@@ -65,6 +68,7 @@ freebsd11_cvtdirent(struct freebsd11_dirent *dstdp, struct 
dirent *srcdp)
bzero(dstdp->d_name + dstdp->d_namlen,
dstdp->d_reclen - offsetof(struct freebsd11_dirent, d_name) -
dstdp->d_namlen);
+   return (true);
 }
 
 struct freebsd11_dirent *
@@ -80,8 +84,10 @@ freebsd11_readdir(DIR *dirp)
if (dirp->dd_compat_de == NULL)
dirp->dd_compat_de = malloc(sizeof(struct
freebsd11_dirent));
-   freebsd11_cvtdirent(dirp->dd_compat_de, dp);
-   dstdp = dirp->dd_compat_de;
+   if (freebsd11_cvtdirent(dirp->dd_compat_de, dp))
+   dstdp = dirp->dd_compat_de;
+   else
+   dstdp = NULL;
} else
dstdp = NULL;
if (__isthreaded)
@@ -101,8 +107,10 @@ freebsd11_readdir_r(DIR *dirp, struct freebsd11_dirent 
*entry,
if (error != 0)
return (error);
if (xresult != NULL) {
-   freebsd11_cvtdirent(entry, );
-   *result = entry;
+   if (freebsd11_cvtdirent(entry, ))
+   *result = entry;
+   else
+   *result = NULL;
} else
*result = NULL;
return (0);
diff --git a/sys/kern/vfs_syscalls.c b/sys/kern/vfs_syscalls.c
index 784af836aee..27b2635030d 100644
--- a/sys/kern/vfs_syscalls.c
+++ b/sys/kern/vfs_syscalls.c
@@ -3733,7 +3733,8 @@ freebsd11_kern_getdirentries(struct thread *td, int fd, 
char *ubuf, u_int count,
if (dp->d_reclen == 0)
break;
MPASS(dp->d_reclen >= _GENERIC_DIRLEN(0));
-   /* dp->d_namlen <= sizeof(dstdp.d_name) - 1 always */
+   if (dp->d_namlen >= sizeof(dstdp.d_name))
+   continue;
dstdp.d_type = dp->d_type;
dstdp.d_namlen = dp->d_namlen;
dstdp.d_fileno = dp->d_fileno;  /* truncate */
diff --git a/sys/sys/dirent.h b/sys/sys/dirent.h
index 341855d0530..691c4e8f90f 100644
--- a/sys/sys/dirent.h
+++ b/sys/sys/dirent.h
@@ -67,8 +67,9 @@ struct dirent {
off_t  d_off;   /* directory offset of entry */
__uint16_t d_reclen;/* length of this record */
__uint8_t  d_type;  /* file type, see below */
-   __uint8_t  d_namlen;/* length of string in d_name */
-   __uint32_t d_pad0;
+   __uint8_t  d_pad0
+   __uint16_t d_namlen;/* length of string in d_name */
+   __uint16_t d_pad1;
 #if __BSD_VISIBLE
 #defineMAXNAMLEN   255
chard_name[MAXNAMLEN + 1];  /* name must be no longer than this */

Re: 64-bit inodes (ino64) Status Update and Call for Testing

[Jilles Tjoelker]

On Thu, Apr 20, 2017 at 10:43:14PM +0300, Konstantin Belousov wrote:
> Inodes are data structures corresponding to objects in a file system,
> such as files and directories. FreeBSD has historically used 32-bit
> values to identify inodes, which limits file systems to somewhat under
> 2^32 objects. Many modern file systems internally use 64-bit identifiers
> and FreeBSD needs to follow suit to properly and fully support these
> file systems.

> The 64-bit inode project, also known as ino64, started life many years
> ago as a project by Gleb Kurtsou (gleb@).  After that time several
> people have had a hand in updating it and addressing regressions, after
> mckusick@ picked up and updated the patch, and acted as a flag-waver.

> Sponsored by the FreeBSD Foundation I have spent a significant effort
> on outstanding issues and integration -- fixing compat32 ABI, NFS and
> ZFS, addressing ABI compat issues and investigating and fixing ports
> failures.  rmacklem@ provided feedback on NFS changes, emaste@ and
> jhb@ provided feedback and review on the ABI transition support. pho@
> performed extensive testing and identified a number of issues that
> have now been fixed.  kris@ performed an initial ports investigation
> followed by an exp-run by antoine@. emaste@ helped with organization
> of the process.

> This note explains how to perform useful testing of the ino64 branch,
> beyond typical smoke tests.

> 1. Overview.

> The ino64 branch extends the basic system types ino_t and dev_t from
> 32-bit to 64-bit, and nlink_t from 16-bit to 64-bit.  The struct dirent
> layout is modified due to the larger size of ino_t, and also gains a
> d_off (directory offset) member. As ino64 implies an ABI change anyway
> the struct statfs f_mntfromname[] and f_mntonname[] array length
> MNAMELEN is increased from 88 to 1024, to allow for longer mount path
> names.

> ABI breakage is mitigated by providing compatibility using versioned
> symbols, ingenious use of the existing padding in structures, and by
> employing other tricks.  Unfortunately, not everything can be fixed,
> especially outside the base system.  For instance, third-party APIs
> which pass struct stat around are broken in backward and forward-
> incompatible way.

We have another type in this area which is too small in some situations:
uint8_t for struct dirent.d_namlen. For filesystems that store filenames
as upto 255 UTF-16 code units, the name to be stored in d_name may be
upto 765 bytes long in UTF-8. This was reported in PR 204643. The code
currently handles this by returning the short (8.3) name, but this name
may not be present or usable, leaving the file inaccessible.

Actually allowing longer names seems too complicated to add to the ino64
change, but changing d_namlen to uint16_t (using d_pad0 space) and
skipping entries with d_namlen > 255 in libc may be helpful.

Note that applications using the deprecated readdir_r() will not be able
to read such long names, since the API does not allow specifying that a
larger buffer has been provided. (This could be avoided by making struct
dirent.d_name 766 bytes long instead of 256.)

Unfortunately, the existence of readdir_r() also prevents changing
struct dirent.d_name to the more correct flexible array.

-- 
Jilles Tjoelker
___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"

Re: Problem with re(4) Ethernet driver has resurfaced in 11-STABLE and HEAD

[Baho Utot]

On 05/21/17 05:29, Thomas Mueller wrote:

from YongHyeon PYUN:


[removed stable@ from CC]



I recently updated my 10.1-STABLE to 11.0-STABLE and find I can no longer 
connect with the Ethernet.



dhclient re0 produces



DHCPDISCOVER on re0 to 255.255.255.255 port 67 interval 4
DHCPDISCOVER on re0 to 255.255.255.255 port 67 interval 11
DHCPDISCOVER on re0 to 255.255.255.255 port 67 interval 19
DHCPDISCOVER on re0 to 255.255.255.255 port 67 interval 7
DHCPDISCOVER on re0 to 255.255.255.255 port 67 interval 14
DHCPDISCOVER on re0 to 255.255.255.255 port 67 interval 5
No DHCPOFFERS received.
No working leases in persistent database - sleeping.




If you assign an static IPv4 address to re(4) are you able to use
the network interface?



AFAIK there was no significant re(4) changes for a long time. Could
you show us back trace information?


Problem with re(4) reappeared in both 11.0-STABLE and HEAD, but OK to trim 
stable@ since changes/fixes would go to HEAD first.

No connection with static IPv4 address.

Where do I get back trace information?

Problem was more severe with HEAD in that OS immediately crashed into debugger, 
while in 11.0-STABLE, only the connection failed but may have left memory 
unstable.

I can still connect on that computer with Hiro H50191 USB wireless adapter, 
driver rsu.

Tom

___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"


I have the same problem with this driver:

 urtwn - Realtek RTL8188CU/RTL8188RU/RTL8188EU/RTL8192CU USB IEEE
 802.11b/g/n wireless network device

On this platform:

FreeBSD desktop.example.com 11.0-RELEASE-p9 FreeBSD 11.0-RELEASE-p9 #0 
r316958: Sat Apr 15 09:25:18 EDT 2017 
r...@desktop.example.com:/usr/obj/usr/src/sys/GENERIC  amd64


Had to quit using it under FreeBSD works fine in Win7.

___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"

Re: Problem with re(4) Ethernet driver has resurfaced in 11-STABLE and HEAD

[Thomas Mueller]

from YongHyeon PYUN:

> [removed stable@ from CC]

> > I recently updated my 10.1-STABLE to 11.0-STABLE and find I can no longer 
> > connect with the Ethernet.

> > dhclient re0 produces

> > DHCPDISCOVER on re0 to 255.255.255.255 port 67 interval 4
> > DHCPDISCOVER on re0 to 255.255.255.255 port 67 interval 11
> > DHCPDISCOVER on re0 to 255.255.255.255 port 67 interval 19
> > DHCPDISCOVER on re0 to 255.255.255.255 port 67 interval 7
> > DHCPDISCOVER on re0 to 255.255.255.255 port 67 interval 14
> > DHCPDISCOVER on re0 to 255.255.255.255 port 67 interval 5
> > No DHCPOFFERS received.
> > No working leases in persistent database - sleeping.


> If you assign an static IPv4 address to re(4) are you able to use
> the network interface?

> AFAIK there was no significant re(4) changes for a long time. Could
> you show us back trace information?

Problem with re(4) reappeared in both 11.0-STABLE and HEAD, but OK to trim 
stable@ since changes/fixes would go to HEAD first.

No connection with static IPv4 address. 

Where do I get back trace information?

Problem was more severe with HEAD in that OS immediately crashed into debugger, 
while in 11.0-STABLE, only the connection failed but may have left memory 
unstable.

I can still connect on that computer with Hiro H50191 USB wireless adapter, 
driver rsu.

Tom

___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"

Bug in make setting wrong MAKESYSPATH

[Thomas Mueller]

I tried building ports, starting with ports-mgmt/synth, on HEAD (12-current) 
and ran into difficulties with syntax error in bsd.compiler.mk .

With PORTSDIR on another partition, mounted as /BETA1, I got these errors, but 
not when I null-mounted /BETA1/usr/ports as /usr/ports.

I shouldn't have to resort to this kludge, didn't have to in the recent past.

This bug shows in both 11.0-STABLE and 12.0-CURRENT.

I looked into "man make" and found that make got the wrong path for 
MAKESYSPATH, setting to /BETA1/usr/share/mk instead of what it should be, 
/usr/share/mk .

Going into /BETA1/usr/ports/archivers/zip (for a short and simple example), 
make all-depends-list   produced

sh: Syntax error: ")" unexpected
make: "/BETA1/usr/share/mk/bsd.compiler.mk" line 52: warning: "echo 4.0.0 
4.0.0) | awk -F. '{print $1 * 1 + $2 * 100 + $3;}'" returned non-zero status
sh: Syntax error: ")" unexpected
make[1]: "/BETA1/usr/share/mk/bsd.compiler.mk" line 52: warning: "echo 4.0.0 
4.0.0) | awk -F. '{print $1 * 1 + $2 * 100 + $3;}'" returned non-zero status
/BETA1/usr/ports/ports-mgmt/pkg

make -m /usr/share/mk all-depends-list produces
/BETA1/usr/ports/ports-mgmt/pkg

This looks like a bug that ought to be fixed, though there is a workaround 
using "make -m /usr/share/mk ..." every time, or presumably, setting 
MAKESYSPATH=/usr/share/mk
in the environment.

Should I file a bug report?

I could get much more verbose outputs when there are more dependencies, such as 
in /BETA1/usr/ports/ports-mgmt/synth, or more so, 
/BETA1/usr/ports/www/seamonkey

I also noticed that in newer versions of FreeBSD, /usr/share/mk/bsd.compiler.mk 
has greatly increased in size (not a bug. except when "make" goes to the wrong 
MAKESYSPATH.

Maybe add in .cshrc and .profile
MAKESYSPATH=/usr/share/mk
?

Tom

___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"

Re: Maintainershjip status regarding re(4) Ethernet driver?

[Kevin Lo]

On Sun, May 21, 2017 at 04:38:46AM +, Thomas Mueller wrote:
> 
> Who is the maintainer, if any, for re(4) Ethernet driver that is again giving 
> me trouble on Intel Ivy Bridge computer with MSI Z77 MPOWER motherboard?
> 
> I remember Kevin Lo, but have checked the web archives for freebsd-current 
> and freebsd-net, and find Kevin Lo's last posts were during October 2016.
> 
> Is Kevin Lo still with FreeBSD?
> 
> I just opened a bugzilla account with this Ethernet re(4) connectivity 
> problem on my mind.
> 
> I posted the details on this list five days ago but haven't had any response.

I have no idea why you cc'd me.  I'm not the maintainer of re(4).

> Tom

Kevin
___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"