Re: svn commit: r322315 - stable/10/sys/netinet

[Andrey Chernov]

On 09.08.2017 16:26, Michael Tuexen wrote:
> Author: tuexen
> Date: Wed Aug  9 13:26:12 2017
> New Revision: 322315
> URL: https://svnweb.freebsd.org/changeset/base/322315
> 
> Log:
>   MFC r317208:
>   
>   Syncoockies can be used in combination with the syncache. If the cache
>   overflows, syncookies are used.

It cause error for i386 stable-10

 -ffreestanding -fstack-protector -gdwarf-2 -Werror
../../../netinet/tcp_syncache.c
../../../netinet/tcp_syncache.c:280:50: error: implicit conversion from
  'long long' to 'time_t' (aka 'int') changes value from
  -9223372036854775808 to 0 [-Werror,-Wconstant-conversion]
V_tcp_syncache.hashbase[i].sch_last_overflow = INT64_MIN;
 ~ ^
./x86/_stdint.h:89:41: note: expanded from macro 'INT64_MIN'
#define INT64_MIN   (-0x7fffLL-1)
 ~^~
1 error generated.
*** Error code 1
___
svn-src-all@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-all
To unsubscribe, send any mail to "svn-src-all-unsubscr...@freebsd.org"

Re: svn commit: r316642 - head/sys/dev/syscons

[Andrey Chernov]

On 09.04.2017 13:32, Bruce Evans wrote:
> How do you initialize actual use of sc->cursor_char?  It is only used
> when SC_NO_FONT_LOADING is not defined.  Configuring SC_PIXEL_MODE
> forces SC_NO_FONT_LOADING to be undefined.  Except to test sc->cursor_char,

I use text console mode with loading national codepage font. I don't
touch sc->cursor_char directly, I only use 'vidcontrol -M' to move range
start to 0x03 since 0xd0-0xd3 and 0xd4 are valid characters in my
codepage. Loading font and changing mousechar start are both through
rc.conf variables:

font8x14="cp866-8x14"
font8x16="cp866b-8x16"
font8x8="cp866-8x8"
mousechar_start="3"

> is almost always defined.  Apparently the char cursor is turned off with
> loadable fonts since it would mess up special fonts more than the default
> font.  But not as much as the mouse cursor.

If you mean just cursor, I have it (even not block but underline
cursor). It is not strictly char (i.e. text cursor), but emulation of it
via loaded font change on the fly. Via rc.conf variable too:

cursor="destructive"

___
svn-src-all@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-all
To unsubscribe, send any mail to "svn-src-all-unsubscr...@freebsd.org"

Re: svn commit: r316642 - head/sys/dev/syscons

[Andrey Chernov]

On 09.04.2017 10:33, Bruce Evans wrote:
> On Sun, 9 Apr 2017, Andrey Chernov wrote:
> 
>> On 09.04.2017 7:54, Andrey Chernov wrote:
>>> On 08.04.2017 13:00, Bruce Evans wrote:
>>>> Log:
>>>>   Quick fix for removal of the mouse cursor in vga direct graphics
>>>> modes
>>>> ...
>>>
>>> Please don't forget that this 5 cells range (0xd0-0xd4, last one becomes
>>> SC_CURSOR_CHAR) can be redefined with "vidcontrol -M", it is needed for
>>> the case they overlap valid characters region for some code tables.
>>
>> Probably we just can change default to remove additional vidcontrol -M
>> setup. I usually have this range started with 0x03.
> 
> That can only work if you also avoid 9-bit text modes.  9-bit modes
> give 1-bit separations between characters, except for those in the
> graphics range (starting at about 0xd0).

Then let is stays as it is. For me small line in the mouse cursor looks
less distractive that junk in all 4 chars around, which happens if fonts
codepage have 0xd0-0xd3 as valid characters.

___
svn-src-all@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-all
To unsubscribe, send any mail to "svn-src-all-unsubscr...@freebsd.org"

Re: svn commit: r316642 - head/sys/dev/syscons

[Andrey Chernov]

On 09.04.2017 9:46, Bruce Evans wrote:
>> Please don't forget that this 5 cells range (0xd0-0xd4, last one becomes
>> SC_CURSOR_CHAR) can be redefined with "vidcontrol -M", it is needed for
>> the case they overlap valid characters region for some code tables.
> 
> I didn't forget it, but these cells are only used in text mode and there
> are only 4 of them AFAIK (-M goes up to 252, which leaves space for only
> 4 cells 252-255).  Text mode is is one case that has no renderer support
> for the mouse cursor.

Here is 5th one (depends of SC_MOUSE_CHAR which can be set in the kernel
config):

#if SC_MOUSE_CHAR <= SC_CURSOR_CHAR && SC_CURSOR_CHAR < (SC_MOUSE_CHAR + 4)
#undef SC_CURSOR_CHAR
#define SC_CURSOR_CHAR  (SC_MOUSE_CHAR + 4)
#endif

> NOTES gives the example of setting it to 0x3.  Then if the configured
> SC_CURSOR_CHAR is inside the 4-char range for the mouse cursor, then

AFAIK SC_CURSOR_CHAR kernel config or vidcontrol is not supported.

> There is no way to reconfigure the cursor char AFAIK.  In fact, it
> doesn't even seem to be configured -- SC_CURSOR_CHAR is never used,

I remember times it was used but not remember how, log history needs to
be browsed.

___
svn-src-all@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-all
To unsubscribe, send any mail to "svn-src-all-unsubscr...@freebsd.org"

Re: svn commit: r316642 - head/sys/dev/syscons

[Andrey Chernov]

On 09.04.2017 7:54, Andrey Chernov wrote:
> On 08.04.2017 13:00, Bruce Evans wrote:
>> Log:
>>   Quick fix for removal of the mouse cursor in vga direct graphics modes
>>   (that is, in all supported 8, 15, 16 and 24-color modes).  Moving the
>>   mouse cursor while holding down a button (giving cut marking) left a
>>   trail of garbage from misremoved mouse cursors (usually colored
>>   rectangles and not cursor shapes).  Cases with a button not held down
>>   worked better and may even have worked.
>>   
>>   No renderer support for removing (software) mouse cursors is needed
>>   (and many renderers don't have any), since sc_remove_mouse_image()
>>   marks for update the region containing the image and usually much
>>   more.  The mouse cursor can be (partially) over as many as 4 character
>>   cells, and removing it in only the 1-4 cells occupied by it would be
>>   best for efficiency and for avoiding flicker.  
> 
> Please don't forget that this 5 cells range (0xd0-0xd4, last one becomes
> SC_CURSOR_CHAR) can be redefined with "vidcontrol -M", it is needed for
> the case they overlap valid characters region for some code tables.

Probably we just can change default to remove additional vidcontrol -M
setup. I usually have this range started with 0x03.
___
svn-src-all@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-all
To unsubscribe, send any mail to "svn-src-all-unsubscr...@freebsd.org"

Re: svn commit: r316642 - head/sys/dev/syscons

[Andrey Chernov]

On 08.04.2017 13:00, Bruce Evans wrote:
> Log:
>   Quick fix for removal of the mouse cursor in vga direct graphics modes
>   (that is, in all supported 8, 15, 16 and 24-color modes).  Moving the
>   mouse cursor while holding down a button (giving cut marking) left a
>   trail of garbage from misremoved mouse cursors (usually colored
>   rectangles and not cursor shapes).  Cases with a button not held down
>   worked better and may even have worked.
>   
>   No renderer support for removing (software) mouse cursors is needed
>   (and many renderers don't have any), since sc_remove_mouse_image()
>   marks for update the region containing the image and usually much
>   more.  The mouse cursor can be (partially) over as many as 4 character
>   cells, and removing it in only the 1-4 cells occupied by it would be
>   best for efficiency and for avoiding flicker.  

Please don't forget that this 5 cells range (0xd0-0xd4, last one becomes
SC_CURSOR_CHAR) can be redefined with "vidcontrol -M", it is needed for
the case they overlap valid characters region for some code tables.
___
svn-src-all@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-all
To unsubscribe, send any mail to "svn-src-all-unsubscr...@freebsd.org"

Re: svn commit: r316477 - head/usr.bin/grep

[Andrey Chernov]

On 04.04.2017 16:41, Kyle Evans wrote:
> Upon further review, newer versions of GNU grep(1) drop the explicit
> '00' and add the \33[K dropped in this commit a few lines above the
> mentioned one. In that case, we might as well revert those two lines and
> consider this an improvement. I guess it depends on the overall outlook
> of the tests, though -- should we introduce more tests that fail by
> default as they go in, or adapt similar (not strictly buggy, but not
> ideal) behavior to start with and then improve after we can grant
> bsdgrep(1) replacement status?

It seems new GNU grep takes right course, so I vote to make our thing
like there and not as in its obsoleted version.

\33[K clears from cursor to the rest of the line (with background color,
in case terminal have "ut" termcap capability). Consider how multi-line
pattern will look with it and without.

___
svn-src-all@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-all
To unsubscribe, send any mail to "svn-src-all-unsubscr...@freebsd.org"

Re: svn commit: r316477 - head/usr.bin/grep

[Andrey Chernov]

On 04.04.2017 16:05, Bruce Evans wrote:
> This might be to work around a bug in a terminal driver.  I needed the
> 0 (but just one) to reset the attributes in my attribute tests, on at
> least 1 of the tested terminals (current sc, current vt, old sc, old
> xterm, old cygwin xterm).  CSI m seems to be working for sc now, and the
> teken code looks OK.

I never meet such one. Even most earlier specs tells about CSI m and it
works in sc from its first day. You can check how often CSI m used by
inspecting /etc/termcap changes, there is no 00 too.

___
svn-src-all@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-all
To unsubscribe, send any mail to "svn-src-all-unsubscr...@freebsd.org"

Re: svn commit: r316477 - head/usr.bin/grep

[Andrey Chernov]

On 04.04.2017 16:09, Kyle Evans wrote:
> 
> First, apologies, I must rewind: what did you  mean initially by "We
> don't need to handle internally [...]"
>  -- the second half I understand, but it occurs to me that we should
> already be internally handling \33[m = \33[00m as defined by ANSI.

Of course we must handle (and already handle) both according to ANSI,
but few CPU cycles here, then in another place, yet another, etc. making
big time in sum when CPU does nothing useful but resource eating.

> On Tue, Apr 4, 2017 at 7:37 AM, Andrey Chernov <a...@freebsd.org
> <mailto:a...@freebsd.org>> wrote:
> 
> IMHO everyday usage by everyone weights much more than occasional
> regression tests run which can be fixed instead of this place. F.e. we
> already do a lot of local fixes in the NetBSD regression tests instead
> of pretending to mimic NetBSD in 100% in the system itself.
> 
> 
> This is where it gets kind of finnicky, and I'll step out and let others
> weigh in -- the test I refer to was specifically written by me
> (https://reviews.freebsd.org/D10112), and it's entirely to make it
> easier for me to check and quantify to others where we're actually
> regressing or improving as I continue fixing things in bsdgrep(1). This
> is easier to do so when we don't have trivial failures due to minor
> formatting differences when they should ultimately be interpreted
> equivalently either way, such as in this case. I expect to be find other
> quirky bugs in gnugrep(1) while reviewing the remaining PRs in Bugzilla,
> and thus expect this to be more important in the months to come.

I see another problem, but it sounds a bit artificially. What if someone
decides to parse grep-colored output and assumes gnu grep only (which is
usual in Linux). I can't determine probability of such situation.
Usually people use generic ANSI parser which understand both cases (f.e.
to convert it to HTML).

___
svn-src-all@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-all
To unsubscribe, send any mail to "svn-src-all-unsubscr...@freebsd.org"

Re: svn commit: r316477 - head/usr.bin/grep

[Andrey Chernov]

On 04.04.2017 15:24, Kyle Evans wrote:
> On Tue, Apr 4, 2017 at 7:07 AM, Andrey Chernov <a...@freebsd.org
> <mailto:a...@freebsd.org>> wrote:
> 
> On 04.04.2017 2:16, Ed Maste wrote:
> >   if (color)
> > - fprintf(stdout, "\33[m\33[K");
> > + fprintf(stdout, "\33[00m\33[K");
> 
> Please back that one out. We don't need to handle internally or print
> remotely excessive 00.
> At least according to
> https://en.wikipedia.org/wiki/ANSI_escape_code
> <https://en.wikipedia.org/wiki/ANSI_escape_code>
> "With no parameters, CSI m is treated as CSI 0 m (reset / normal), which
> is typical of most of the ANSI escape sequences."
> 
> 
> Hi ache@,
> 
> This specific change was made in the name of explicitly matching colored
> output of GNU grep for simplification of regression test purposes,
> rather than for good form. Is it still unacceptable to do so?

IMHO everyday usage by everyone weights much more than occasional
regression tests run which can be fixed instead of this place. F.e. we
already do a lot of local fixes in the NetBSD regression tests instead
of pretending to mimic NetBSD in 100% in the system itself.

___
svn-src-all@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-all
To unsubscribe, send any mail to "svn-src-all-unsubscr...@freebsd.org"

Re: svn commit: r316477 - head/usr.bin/grep

[Andrey Chernov]

On 04.04.2017 2:16, Ed Maste wrote:
>   if (color) 
> - fprintf(stdout, "\33[m\33[K");
> + fprintf(stdout, "\33[00m\33[K");

Please back that one out. We don't need to handle internally or print
remotely excessive 00.
At least according to
https://en.wikipedia.org/wiki/ANSI_escape_code
"With no parameters, CSI m is treated as CSI 0 m (reset / normal), which
is typical of most of the ANSI escape sequences."

___
svn-src-all@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-all
To unsubscribe, send any mail to "svn-src-all-unsubscr...@freebsd.org"

Re: svn commit: r316069 - in stable/10: contrib/ntp contrib/ntp/adjtimed contrib/ntp/clockstuff contrib/ntp/html contrib/ntp/include contrib/ntp/include/isc contrib/ntp/kernel contrib/ntp/kernel/sys c

[Andrey Chernov]

On 28.03.2017 7:48, Xin LI wrote:
> Author: delphij
> Date: Tue Mar 28 04:48:55 2017
> New Revision: 316069
> URL: https://svnweb.freebsd.org/changeset/base/316069
> 
> Log:
>   MFC r315871: MFV r315791: ntp 4.2.8p10.

Should /etc/ntp.conf be updated to?

___
svn-src-all@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-all
To unsubscribe, send any mail to "svn-src-all-unsubscr...@freebsd.org"

Re: svn commit: r310433 - head/lib/libc/stdio

[Andrey Chernov]

On 23.12.2016 1:39, Conrad Meyer wrote:
> This was unjustified and inappropriate.

I don't think so. While being able to back it out or fix it by another
way by yourself, you just doing nothing for several days. Your personal
reasons of inability to act properly so long time weights less than
project consistency at whole, so backing it out from anybody is
justified and appropriate in such situations.

> On Thu, Dec 22, 2016 at 2:30 PM, Ngie Cooper  wrote:
>> Author: ngie
>> Date: Thu Dec 22 22:30:42 2016
>> New Revision: 310433
>> URL: https://svnweb.freebsd.org/changeset/base/310433
>>
>> Log:
>>   Revert r310138
>>
>>   Adding %b support to vfprintf for parity with kernel space requires
>>   more discussion/review.
>>
>>   In particular, many parties were concerned over introducing a
>>   non-standard format qualifier to *printf(3) which didn't already
>>   exist in other OSes, e.g. Linux, thus making code which used %b
>>   harder to port to other operating systems.
>>
>>   Requested by: many
>>
>> Modified:
>>   head/lib/libc/stdio/vfprintf.c
>>
>> Modified: head/lib/libc/stdio/vfprintf.c
>> ==
>> --- head/lib/libc/stdio/vfprintf.c  Thu Dec 22 21:56:41 2016
>> (r310432)
>> +++ head/lib/libc/stdio/vfprintf.c  Thu Dec 22 22:30:42 2016
>> (r310433)
>> @@ -611,37 +611,6 @@ reswitch:  switch (ch) {
>> case 'z':
>> flags |= SIZET;
>> goto rflag;
>> -   case 'b':
>> -   {
>> -   const char *q;
>> -   int anybitset, bit;
>> -
>> -   ulval = (u_int)GETARG(int);
>> -   cp = GETARG(char *);
>> -
>> -   q = __ultoa(ulval, buf + BUF, *cp++, 0, xdigs_lower);
>> -   PRINT(q, buf + BUF - q);
>> -
>> -   if (ulval == 0)
>> -   break;
>> -
>> -   for (anybitset = 0; *cp;) {
>> -   bit = *cp++;
>> -   if (ulval & (1 << (bit - 1))) {
>> -   PRINT(anybitset ? "," : "<", 1);
>> -   q = cp;
>> -   for (; (bit = *cp) > ' '; ++cp)
>> -   continue;
>> -   PRINT(q, cp - q);
>> -   anybitset = 1;
>> -   } else
>> -   for (; *cp > ' '; ++cp)
>> -   continue;
>> -   }
>> -   if (anybitset)
>> -   PRINT(">", 1);
>> -   }
>> -   continue;
>> case 'C':
>> flags |= LONGINT;
>> /*FALLTHROUGH*/
>>
> 

___
svn-src-all@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-all
To unsubscribe, send any mail to "svn-src-all-unsubscr...@freebsd.org"

Re: svn commit: r306294 - stable/10/sys/compat/linux

[Andrey Chernov]

On 24.09.2016 13:34, Dmitry Chagin wrote:
> Author: dchagin
> Date: Sat Sep 24 10:34:35 2016
> New Revision: 306294
> URL: https://svnweb.freebsd.org/changeset/base/306294
> 
> Log:
>   MFC r305896:
>   
>   Implement BLKSSZGET ioctl for the Linuxulator.

Off topic: is there any chance to get linux ALSA working, at least its
PulseAudio plugin? Currently linux PulseAudio itself works nice
(transfering all sounds from remote machine to local), but linux ALSA
PulseAudio plugin exits with thread assert error. The same things
compiled natively (i.e. ALSA + PulseAudio plugin for FreeBSD) works
nice. I mean this ALSA setup:

pcm.!default {
type pulse
hint.description "PulseAudio"
}

ctl.!default {
type pulse
}

___
svn-src-all@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-all
To unsubscribe, send any mail to "svn-src-all-unsubscr...@freebsd.org"

Re: svn commit: r305981 - head/usr.bin/cmp

[Andrey Chernov]

On 19.09.2016 19:13, Conrad E. Meyer wrote:
> + /*
> +  * Cache NLS data, for strerror, for err(3), before entering capability
> +  * mode.
> +  */
> + (void)catopen("libc", NL_CAT_LOCALE);

This code should be common for all programs which use err(), perror()
etc. since they all should call setlocale() with LC_ALL. If they not do
it, they should be fixed in the same time.


___
svn-src-all@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-all
To unsubscribe, send any mail to "svn-src-all-unsubscr...@freebsd.org"

Re: svn commit: r304928 - in head/lib/libc: amd64/sys i386/sys sys

[Andrey Chernov]

On 29.08.2016 22:10, Konstantin Belousov wrote:
> On Mon, Aug 29, 2016 at 09:16:29PM +0300, Andrey Chernov wrote:
>> Previous text is documented everywhere and describing usual good
>> practice, so it should remains in place, in that means r304928 should be
>> reverted, because replace proper way of doing things with obsoleted
>> feature description.
>>
>> What I suggest is not _replace_ old text with new, but _add_ new text as
>> describing current and not recommended way, with the direct mention that
>> this feature is obsoleted and not recommended for relay on it.
>>
> 
> diff --git a/lib/libc/sys/ptrace.2 b/lib/libc/sys/ptrace.2
> index 48802f4..d406efc 100644
> --- a/lib/libc/sys/ptrace.2
> +++ b/lib/libc/sys/ptrace.2
> @@ -2,7 +2,7 @@
>  .\"  $NetBSD: ptrace.2,v 1.2 1995/02/27 12:35:37 cgd Exp $
>  .\"
>  .\" This file is in the public domain.
> -.Dd August 28, 2016
> +.Dd August 29, 2016
>  .Dt PTRACE 2
>  .Os
>  .Sh NAME
> @@ -900,19 +900,29 @@ argument is ignored.
>  .Pp
>  Additionally, other machine-specific requests can exist.
>  .Sh RETURN VALUES
> +Most requests return 0 on success and \-1 on error.
>  Some requests can cause
>  .Fn ptrace
>  to return
>  \-1
> -as a non-error value; to disambiguate,
> +as a non-error value, among them are
> +.Dv PT_READ_I
> +and
> +.Dv PT_READ_D ,
> +which return the value read from the process memory on success.
> +To disambiguate,
>  .Va errno
> -is set to 0 in the libc wrapper for the
> -.Fn ptrace
> -system call and
> +can be set to 0 before the call and checked afterwards.
> +.Pp
> +Current
>  .Fn ptrace
> -callers can reliably check
> +implementation always sets
> +.Va errno
> +to 0 before calling into kernel, both for historic reasons and for
> +consistency with other operating systems.
> +It is recommended to assign zero to
>  .Va errno
> -for non-zero value afterwards.
> +explicitely for future compatibility.
>  .Sh ERRORS
>  The
>  .Fn ptrace
> 

Ok.


___
svn-src-all@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-all
To unsubscribe, send any mail to "svn-src-all-unsubscr...@freebsd.org"

Re: svn commit: r304928 - in head/lib/libc: amd64/sys i386/sys sys

[Andrey Chernov]

On 29.08.2016 21:16, Andrey Chernov wrote:
> On 29.08.2016 21:04, Konstantin Belousov wrote:
>> On Mon, Aug 29, 2016 at 08:46:47PM +0300, Andrey Chernov wrote:
>>> Either we implement this wrapper or left all things as is, we need to
>>> document internal errno clearing additionally, to not make people wonder
>>> why errno becomes 0, probably with the mention that program should not
>>> relay on this obsoleted implementation feature.
>> It was done in r304928.  John want to revert this change, it seems.
>>
> Previous text is documented everywhere and describing usual good
> practice, so it should remains in place, in that means r304928 should be
> reverted, because replace proper way of doing things with obsoleted
> feature description.
> 
> What I suggest is not _replace_ old text with new, but _add_ new text as
> describing current and not recommended way, with the direct mention that
> this feature is obsoleted and not recommended for relay on it.

I.e. add to old text something like this (my English is bad, so it
probably needs correction):
"Currently ptrace(2) always set errno to 0 at the start, but portable
program should not relay on this obsoleted feature".

___
svn-src-all@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-all
To unsubscribe, send any mail to "svn-src-all-unsubscr...@freebsd.org"

Re: svn commit: r304928 - in head/lib/libc: amd64/sys i386/sys sys

[Andrey Chernov]

On 29.08.2016 21:04, Konstantin Belousov wrote:
> On Mon, Aug 29, 2016 at 08:46:47PM +0300, Andrey Chernov wrote:
>> Either we implement this wrapper or left all things as is, we need to
>> document internal errno clearing additionally, to not make people wonder
>> why errno becomes 0, probably with the mention that program should not
>> relay on this obsoleted implementation feature.
> It was done in r304928.  John want to revert this change, it seems.
> 
Previous text is documented everywhere and describing usual good
practice, so it should remains in place, in that means r304928 should be
reverted, because replace proper way of doing things with obsoleted
feature description.

What I suggest is not _replace_ old text with new, but _add_ new text as
describing current and not recommended way, with the direct mention that
this feature is obsoleted and not recommended for relay on it.


___
svn-src-all@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-all
To unsubscribe, send any mail to "svn-src-all-unsubscr...@freebsd.org"

Re: svn commit: r304928 - in head/lib/libc: amd64/sys i386/sys sys

[Andrey Chernov]

On 29.08.2016 19:51, John Baldwin wrote:
>> int
>> ptrace(int request, pid_t pid, caddr_t addr, int data)
>> {
>>
>>  errno = 0;
>>  return (__sys_ptrace(request, pid, addr, data));
>> }
> 
> Certainly I think having a C wrapper like this makes more sense than
> doing it all in assembly N times.  I would probably prefer to keep the
> manpage language the way it is though.

Either we implement this wrapper or left all things as is, we need to
document internal errno clearing additionally, to not make people wonder
why errno becomes 0, probably with the mention that program should not
relay on this obsoleted implementation feature.

IMHO, it will be better to not clear errno at all, considering 'before
call' way documented everywhere and consumers behavior analyzed by kib@

___
svn-src-all@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-all
To unsubscribe, send any mail to "svn-src-all-unsubscr...@freebsd.org"

Re: svn commit: r304928 - in head/lib/libc: amd64/sys i386/sys sys

[Andrey Chernov]

On 29.08.2016 18:34, Konstantin Belousov wrote:
> On Mon, Aug 29, 2016 at 06:05:50PM +0300, Andrey Chernov wrote:
>> No surprise since they all share the same initial implementation. You
>> can add NetBSD and OpenBSD too. Errno clearing as undocumented side
>> effect is not what we need, and every system documents that user must
>> clear errno.
> Of course they do not share initial implementation. Solaris ptrace(2) is
> the recent clean room implementation of the interface on top of their
> procfs(4).  Glibc provides the code specific to linux, and they only
> clear errno for requests which might return -1 legitimately, and only do
> it after.
> 
> I would understand your argument if I referenced Open/NetBSD, but I did not.
> 

Of course I don't mean that they share the same code, it is impossible.
By sharing I mean they look at the same initial implementation (or its
already existent forks), writing their own code, so this errno clearing
piece becomes common as becomes common that really documented another
way is.

But all this is history. Now we have
a) Undocumented errno clearing (as bare minimum).
b) Inconsistency with other libc functions and syscalls related to
forced errno clearing (in wider scope).

___
svn-src-all@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-all
To unsubscribe, send any mail to "svn-src-all-unsubscr...@freebsd.org"

Re: svn commit: r304928 - in head/lib/libc: amd64/sys i386/sys sys

[Andrey Chernov]

On 29.08.2016 10:07, Konstantin Belousov wrote:
> On Mon, Aug 29, 2016 at 09:58:13AM +0300, Konstantin Belousov wrote:
>> On Sun, Aug 28, 2016 at 04:09:51PM -0700, John Baldwin wrote:
>>> OTOH, given that we explicitly documented it as not being true, I suspect
>>> any applications that are using ptrace() are going off the documentation, 
>>> not
>>> the implementation artifact.  Note that Linux's ptrace() documents the same
>>> requirement as before this change (caller is required to clear errno), so I
>>> doubt there is any actual software out there that expects the
>>> FreeBSD-specific behavior.  Given that and the extra maintenance overhead of
>>> having to dink with errno in assembly on X architectures, I'd rather we keep
>>> the old language in the manpage and remove the 'errno' frobbing in the 
>>> system
>>> call wrappers.  To be honest, my first response to this commit was one of
>>> surprise that we modify errno directly as that is inconsistent with other
>>> system calls.  (I haven't looked to see if any other system call wrappers
>>> modify errno for non-error cases.)
>>
>> The problematic calls are PT_PEEK_I and PT_PEEK_D, as far as I understand.
>>
>> I dug into the ptrace(2) consumers, I found a lot of things using
>> it which I would not expect to use, besides usual suspects of gdb
>> lldb libunwind reptyr etc.  Most surprising was that even high-profile
>> consumers including gdb sometimes fail to check errno after PT_PEEK. On
>> the other hand, I did not found a case in gdb where errno is checked
>> after PT_PEEK but not zeroed before the syscall.
>>
>> I almost agreed with you after the reading, but then I decided to look
>> into glibc just in case.  What I found there is really fascinating.
>> From glibc/sysdeps/unix/sysv/linux:
>>   res = INLINE_SYSCALL (ptrace, 4, request, pid, addr, data);
>>   if (res >= 0 && request > 0 && request < 4)
>> {
>>   __set_errno (0);
>>   return ret;
>> }
>> #define PTRACE_PEEKTEXT 1
>> #define PTRACE_PEEKDATA 2
>> #define PTRACE_PEEKUSR  3
>>
>> In the end, I might consider changing the ptrace wrappers into
>> consolidated C source, it would look like that
>>
>> int
>> ptrace(int request, pid_t pid, caddr_t addr, int data)
>> {
>>
>>  errno = 0;
>>  return (__sys_ptrace(request, pid, addr, data));
>> }
> 
> And Solaris libc, where ptrace() is the wrapper around procfs, starts its
> implementation this way:
> usr/src/lib/libc/i386/sys/ptrace.c
>   /*
>* Process the request.
>*/
>   errno = 0;
>   switch (request) {
>   case 1: /* PTRACE_PEEKTEXT */
>   case 2: /* PTRACE_PEEKDATA */
> 

No surprise since they all share the same initial implementation. You
can add NetBSD and OpenBSD too. Errno clearing as undocumented side
effect is not what we need, and every system documents that user must
clear errno.

___
svn-src-all@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-all
To unsubscribe, send any mail to "svn-src-all-unsubscr...@freebsd.org"

Re: svn commit: r304928 - in head/lib/libc: amd64/sys i386/sys sys

[Andrey Chernov]

On 28.08.2016 8:28, Bruce Evans wrote:
>> How hard it will be to bring ptrace() to what C99 expects? Perhaps now
>> time is suited well to change some obsoleted things.
> 
> This should be safe to change, since portable applications like gdb can't
> assume that the implemementation clobbers errno for them.

It looks safe for me too.

> Even FreeBSD's man page doesn't document the FreeBSD behaviour.  It
> documents, with poor wording, that applications must set errno as usual:
> 
> %%%
> RETURN VALUES
>  Some requests can cause ptrace() to return -1 as a non-error value; to
>  disambiguate, errno can be set to 0 before the call and checked
>  afterwards.
> %%%
> 
> The poor wording is just "errno can be set to 0".  It _must_ be set to 0.
> Also, the function gurantees to not clobber errno so that this checking
> is guaranteed to work.

Yes, I already mention this thing in my hour ago (related to your
answer) reply. We even don't have documented that ptrace() itself
overwrites errno, but document usual practice by setting it to 0 by its
user instead.

>> "conforming implementation may have extensions (including additional
>> library functions), provided they do not alter the behavior of any
>> strictly conforming program.3)"
>>
>> ptrace() is extension (additional library function) so can't set errno
>> to 0 (it breaks strictly conforming program).
> 
> Use of ptrace() makes a program very far from stricty conforming.  Only
> quality of implementation requires ptrace() to follow the usual rules.

It was terms mistake from my side, I already correct myself in my hour
ago reply.

___
svn-src-all@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-all
To unsubscribe, send any mail to "svn-src-all-unsubscr...@freebsd.org"

Re: svn commit: r304928 - in head/lib/libc: amd64/sys i386/sys sys

[Andrey Chernov]

On 28.08.2016 5:33, Andrey Chernov wrote:
> "conforming implementation may have extensions (including additional
> library functions), provided they do not alter the behavior of any
> strictly conforming program.3)"
> 
> ptrace() is extension (additional library function) so can't set errno
> to 0 (it breaks strictly conforming program).

Sorry for misguiding in this particular part. I confuse the "strictly
conforming program" with the "conforming program" term used there too.
"Strictly conforming program" can't call ptrace().

In any case our own ptrace(2) manpage suggest to set errno to 0 manually
_before_ ptrace() call and do not relay on ptrace() to do it by itself:

"Some requests can cause ptrace() to return -1 as a non-error value; to
disambiguate, errno can be set to 0 before the call and checked afterwards."

It will be better to stay common policy about errno even for extensions
to not keep exclusions in the mind.

___
svn-src-all@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-all
To unsubscribe, send any mail to "svn-src-all-unsubscr...@freebsd.org"

Re: svn commit: r304928 - in head/lib/libc: amd64/sys i386/sys sys

[Andrey Chernov]

On 28.08.2016 4:52, Konstantin Belousov wrote:
 POSIX: "For each thread of a process, the value of errno shall not be
 affected by function calls or assignments to errno by other threads."
>>> And ?  What should the citation add new to the substance
>>> of the code change ?
>>
>> This is for your comment "On both i386 and amd64, the errno symbol was
>> directly  referenced, which only works correctly in single-threaded
>> process."
> I still do not understand what you want to say there. Errno as the
> symbol existing in the symbol table of libc, gives 'POSIX errno' value
> for the main thread. Preprocessor definition converts C language
> accesses to errno into some indirections which result in accesses to
> per-thread errno location. The bug in x86 asm code was due to direct
> usage of errno.

This particular quote is not describing a problem, it supports your change.

> I know that POSIX requires that POSIX-defined functions did not modified
> errno except on error, 

POSIX don't say it. You may modify errno to any value besides 0 while
returning success from the function excepting only those functions where
POSIX directly states they can't modify errno. I.e. only 0 is disallowed
in all cases.

> I agree that it would be more
> consistent for ptrace(2) to not do that as well, but the behaviour is
> already there for 35 years and I do not view the 'consistency' as a serious
> reason to break ABI and introduce random failures for innocent consumers
> of it.

How hard it will be to bring ptrace() to what C99 expects? Perhaps now
time is suited well to change some obsoleted things.

>> Already done: ptrace == "any library function".
> *Shaking head*
> 
> 'Library functions' references in the context of C99/C11 are implicitely
> limited to the functions defined by the chapter 7 Library of the standard.

No, they are limited to the libraries described in ISO/IEC International
Standards family which have C standard library among them. C standard
library described in ANSI C standard which is:
ISO/IEC (1999). ISO/IEC 9899:1999(E): Programming Languages.
libc is C standard library with extensions, and C99 directly says about
them:

"conforming implementation may have extensions (including additional
library functions), provided they do not alter the behavior of any
strictly conforming program.3)"

ptrace() is extension (additional library function) so can't set errno
to 0 (it breaks strictly conforming program).

> To play this sillyness to the end, please answer whether I am allowed to
> provide static functions definitions in the libraries headers ? E.g. clause
> 7.1.2 6 of C11 says:
> Any declaration of a library function shall have external linkage.

You can, as long as your static function (i.e. extension) do not alter
the behavior of any strictly conforming program (see the quote above).


___
svn-src-all@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-all
To unsubscribe, send any mail to "svn-src-all-unsubscr...@freebsd.org"

Re: svn commit: r304928 - in head/lib/libc: amd64/sys i386/sys sys

[Andrey Chernov]

On 28.08.2016 4:37, Andrey Chernov wrote:
> On 28.08.2016 4:25, Andrey Chernov wrote:
>>> Point me to a single line in C99 which mentions ptrace().
> 
> Already done: ptrace == "any library function".

To elaborate it more, C99 does not have finite list of library
functions, but it says about standard libraries in general (not user
libraries), and ptrace() belongs to standard library according to its
manpage:
Standard C Library (libc, -lc)

___
svn-src-all@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-all
To unsubscribe, send any mail to "svn-src-all-unsubscr...@freebsd.org"

Re: svn commit: r304928 - in head/lib/libc: amd64/sys i386/sys sys

[Andrey Chernov]

On 28.08.2016 4:25, Andrey Chernov wrote:
>> Point me to a single line in C99 which mentions ptrace().

Already done: ptrace == "any library function".


___
svn-src-all@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-all
To unsubscribe, send any mail to "svn-src-all-unsubscr...@freebsd.org"

Re: svn commit: r304928 - in head/lib/libc: amd64/sys i386/sys sys

[Andrey Chernov]

On 28.08.2016 4:15, Konstantin Belousov wrote:
>> POSIX: "No function in this volume of POSIX.1-2008 shall set errno to zero."
> I am quite curious where ptrace(2) is defined by POSIX.

POSIX just repeats C99 statement for its own functions, supporting this
rule too, but C99 rule is more general and related to any library functions.

>> POSIX: "For each thread of a process, the value of errno shall not be
>> affected by function calls or assignments to errno by other threads."
> And ?  What should the citation add new to the substance
> of the code change ?

This is for your comment "On both i386 and amd64, the errno symbol was
directly  referenced, which only works correctly in single-threaded
process."

>> C99 statement sounds stricter:
>> "The value of errno is zero at program startup, but is never set to zero
>> by any library function. 176)"
>> And syscall is not different from library function from C99 point of view.
> Point me to a single line in C99 which mentions ptrace().
> 
> Do you understand what did the commit changed, and what it did not ?
> Setting errno to zero before the syscall was the existing behaviour
> before the change, and I did not modified anything there. But previous
> wrapper set errno to zero in main thread even if called from some other
> thread, which was the bug fixed.

If you may notice, I don't blame you and don't say that you introduce
setting errno to 0. I just want to bring your attention to the problem
while you are in that area and familiar with it.


___
svn-src-all@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-all
To unsubscribe, send any mail to "svn-src-all-unsubscr...@freebsd.org"

Re: svn commit: r304928 - in head/lib/libc: amd64/sys i386/sys sys

[Andrey Chernov]

On 28.08.2016 4:04, Andrey Chernov wrote:
> On 28.08.2016 3:56, Konstantin Belousov wrote:
>> On Sun, Aug 28, 2016 at 03:38:10AM +0300, Andrey Chernov wrote:
>>> On 28.08.2016 2:03, Konstantin Belousov wrote:
>>>>   Since ptrace(2) syscall can return -1 for non-error situations, libc
>>>>   wrappers set errno to 0 before performing the syscall, as the service
>>>>   to the caller.
>>>
>>> Both C99 and POSIX directly prohibits any standard function to set errno
>>> to 0. ptrace() should either choose other errno to indicate non-error
>>> situation or change return -1 to something else.
>>>
>> ptrace(2) is not a standard function.
>>
> 
> C99 statement sounds stricter:
> "The value of errno is zero at program startup, but is never set to zero
> by any library function. 176)"
> And syscall is not different from library function from C99 point of view.
> 

>> And, we cannot break ABI for the syscall.

We can fix already broken (from standards point of view) ABI for the
syscall.

___
svn-src-all@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-all
To unsubscribe, send any mail to "svn-src-all-unsubscr...@freebsd.org"

Re: svn commit: r304928 - in head/lib/libc: amd64/sys i386/sys sys

[Andrey Chernov]

On 28.08.2016 3:56, Konstantin Belousov wrote:
> On Sun, Aug 28, 2016 at 03:38:10AM +0300, Andrey Chernov wrote:
>> On 28.08.2016 2:03, Konstantin Belousov wrote:
>>>   Since ptrace(2) syscall can return -1 for non-error situations, libc
>>>   wrappers set errno to 0 before performing the syscall, as the service
>>>   to the caller.
>>
>> Both C99 and POSIX directly prohibits any standard function to set errno
>> to 0. ptrace() should either choose other errno to indicate non-error
>> situation or change return -1 to something else.
>>
> ptrace(2) is not a standard function.
> And, we cannot break ABI for the syscall.
> 

C99 statement sounds stricter:
"The value of errno is zero at program startup, but is never set to zero
by any library function. 176)"
And syscall is not different from library function from C99 point of view.

___
svn-src-all@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-all
To unsubscribe, send any mail to "svn-src-all-unsubscr...@freebsd.org"

Re: svn commit: r304928 - in head/lib/libc: amd64/sys i386/sys sys

[Andrey Chernov]

On 28.08.2016 3:38, Andrey Chernov wrote:
> On 28.08.2016 2:03, Konstantin Belousov wrote:
>>   Since ptrace(2) syscall can return -1 for non-error situations, libc
>>   wrappers set errno to 0 before performing the syscall, as the service
>>   to the caller.
> 
> Both C99 and POSIX directly prohibits any standard function to set errno
> to 0. ptrace() should either choose other errno to indicate non-error
> situation or change return -1 to something else.
> 
...and don't touch errno.

POSIX: "No function in this volume of POSIX.1-2008 shall set errno to zero."

> On both i386 and amd64, the errno symbol was directly
> referenced, which only works correctly in single-threaded process.

POSIX: "For each thread of a process, the value of errno shall not be
affected by function calls or assignments to errno by other threads."
___
svn-src-all@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-all
To unsubscribe, send any mail to "svn-src-all-unsubscr...@freebsd.org"

Re: svn commit: r304928 - in head/lib/libc: amd64/sys i386/sys sys

[Andrey Chernov]

On 28.08.2016 2:03, Konstantin Belousov wrote:
>   Since ptrace(2) syscall can return -1 for non-error situations, libc
>   wrappers set errno to 0 before performing the syscall, as the service
>   to the caller.

Both C99 and POSIX directly prohibits any standard function to set errno
to 0. ptrace() should either choose other errno to indicate non-error
situation or change return -1 to something else.

___
svn-src-all@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-all
To unsubscribe, send any mail to "svn-src-all-unsubscr...@freebsd.org"

Re: svn commit: r304911 - head/lib/libc/net

[Andrey Chernov]

On 27.08.2016 20:37, Brooks Davis wrote:
> On Sat, Aug 27, 2016 at 02:43:13PM +, Andrey A. Chernov wrote:
>> Author: ache
>> Date: Sat Aug 27 14:43:13 2016
>> New Revision: 304911
>> URL: https://svnweb.freebsd.org/changeset/base/304911
>>
>> Log:
>>   The formal behavior of qsort is unstable with regard to objects that
>>   are equal. Unfortunately, RFC 3484 requires that otherwise equal objects
>>   remain in the order supplied by the DNS server. The present code attempts
>>   to deal with this by returning -1 for objects that are equal (i.e.,
>>   returns that the first parameter is less then the second parameter).
>>   Unfortunately, the qsort API does not state that the first parameter
>>   passed in is in any particular position in the list.
> 
> Why not switch to mergesort which is stable by design?

megresort() is 1) slower 2) requires additional memory allocated.



signature.asc
Description: OpenPGP digital signature

Re: svn commit: r304781 - head/etc/rc.d

[Andrey Chernov]

On 25.08.2016 5:58, Cy Schubert wrote:
> Author: cy
> Date: Thu Aug 25 02:58:41 2016
> New Revision: 304781
> URL: https://svnweb.freebsd.org/changeset/base/304781
> 
> Log:
>   Add logic to replace the working ntp leap-seconds file in /var/db
>   if it contains a $FreeBSD$ header. The header will cause the file
>   to fail checksum of the hash causing ntpd to ignore the file.

You don't need any logic, just remove $FreeBSD$ keyword from it. ntpd
can be called manually with this file too.


___
svn-src-all@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-all
To unsubscribe, send any mail to "svn-src-all-unsubscr...@freebsd.org"

Re: svn commit: r304703 - head/include/xlocale

[Andrey Chernov]

On 23.08.2016 23:53, Ngie Cooper (yaneurabeya) wrote:
> 
>> On Aug 23, 2016, at 13:33, Andrey A. Chernov  wrote:
>>
>> Author: ache
>> Date: Tue Aug 23 20:33:56 2016
>> New Revision: 304703
>> URL: https://svnweb.freebsd.org/changeset/base/304703
>>
>> Log:
>>  LC_*_MASK bit shifting order was partially broken from the initial commit
>>  time at year 2012. Only LC_COLLATE_MASK and LC_CTYPE_MASK are in the
>>  right order.
>>
>>  The order here should match XLC_* from "xlocale_private.h" which, in turn,
>>  match LC_* publicly visible order from  which determines how
>>  locale components are stored in the structure.
>>  LC_*_MASK -> XLC_* translation done as "ffs(mask) - 1" in the querylocale()
>>  and equivalent shift loop in the newlocale(), so mapped to some wrong
>>  components (excluding two mentioned above).
>>
>>  Formally the fix is ABI breakage, but old code using those masks
>>  never works properly in any case.
>>  Only newlocale() and querylocale() are affected.
>>
>>  MFC after:  7 days
>>
>> Modified:
>>  head/include/xlocale/_locale.h
> 
>   Should __FreeBSD_version be bumped?
> Thanks,
> -Ngie
> 

Yes, I'll do it later.




signature.asc
Description: OpenPGP digital signature

Re: svn commit: r303716 - head/crypto/openssh

[Andrey Chernov]

On 08.08.2016 1:48, Xin Li wrote:
> Well, despite the fact that I have to admit that I get locked out from
> my own storage box too, however (even without wearing any hat) I am for
> the change and would blame myself for being lazy in adopting the change
> when the upstream have announced it earlier about a year ago.
> 
> Compatibility with legacy software/hardware, sure, but if we don't stop
> at some point, it would be like SSL 2.0 which people have pointed out
> several flaws in 1995 and take 16 years to get deprecated and still bite
> people in 2014.
> 
> We should do something like what OpenSSH have done by creating a page
> describing the motivation, the impact, the temporary but discouraged
> workaround, etc., and mention it in the release notes to prevent people
> from being bite.

I agree. I am not seeking hardly, stopping after first solution, but at
least one workaround found: using security/putty port. It still supports
all obsoleted stuff. One disadvantage: in terminal mode it requires X11.





signature.asc
Description: OpenPGP digital signature

Re: svn commit: r303716 - head/crypto/openssh

[Andrey Chernov]

On 08.08.2016 2:01, Andrey Chernov wrote:
> On 08.08.2016 1:48, Xin Li wrote:
>> Well, despite the fact that I have to admit that I get locked out from
>> my own storage box too, however (even without wearing any hat) I am for
>> the change and would blame myself for being lazy in adopting the change
>> when the upstream have announced it earlier about a year ago.
>>
>> Compatibility with legacy software/hardware, sure, but if we don't stop
>> at some point, it would be like SSL 2.0 which people have pointed out
>> several flaws in 1995 and take 16 years to get deprecated and still bite
>> people in 2014.
>>
>> We should do something like what OpenSSH have done by creating a page
>> describing the motivation, the impact, the temporary but discouraged
>> workaround, etc., and mention it in the release notes to prevent people
>> from being bite.
> 
> I agree. I am not seeking hardly, stopping after first solution, but at
> least one workaround found: using security/putty port. It still supports
> all obsoleted stuff. One disadvantage: in terminal mode it requires X11.

Forgot to mention: openssh keys must be converted to putty keys format
first using puttygen.




signature.asc
Description: OpenPGP digital signature

Re: svn commit: r303716 - head/crypto/openssh

[Andrey Chernov]

On 08.08.2016 0:28, Andrey Chernov wrote:
> On 08.08.2016 0:20, Warner Losh wrote:
>>
>>> On Aug 7, 2016, at 3:11 PM, Andrey Chernov <a...@freebsd.org> wrote:
>>>
>>>> OTOH, FreeBSD has a documented deprecation process that says things will
>>>> continue working for a major release after being formally deprecated.
>>>
>>> FreeBSD 11 is not released yet (betas are not counted), stable-10 too,
>>> so it is right time to deprecate for them.
>>
>> Nice try, but feature freeze was months ago. Have you got buy in from the
>> security officer and the release engineer?
>>
>> I didn’t think so...
> 
> I must confess I don't track when feature freeze happens, so this is
> upon to re@ team to decide for 11. I am still right about stable-10 at
> least.

BTW, as I just check, commit for stable-11
drop SSH1 support, disable DSA by default
was
Approved by:re (gjb)

You may ask re@ is it intentional or overlook. Minimum DH size increased
to 2048 and turning off DH 1024 diffie-hellman-group1-sha1 happens much
earlier, about year ago.
___
svn-src-all@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-all
To unsubscribe, send any mail to "svn-src-all-unsubscr...@freebsd.org"

Re: svn commit: r303716 - head/crypto/openssh

[Andrey Chernov]

On 08.08.2016 0:20, Warner Losh wrote:
> 
>> On Aug 7, 2016, at 3:11 PM, Andrey Chernov <a...@freebsd.org> wrote:
>>
>>> OTOH, FreeBSD has a documented deprecation process that says things will
>>> continue working for a major release after being formally deprecated.
>>
>> FreeBSD 11 is not released yet (betas are not counted), stable-10 too,
>> so it is right time to deprecate for them.
> 
> Nice try, but feature freeze was months ago. Have you got buy in from the
> security officer and the release engineer?
> 
> I didn’t think so...

I must confess I don't track when feature freeze happens, so this is
upon to re@ team to decide for 11. I am still right about stable-10 at
least.

___
svn-src-all@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-all
To unsubscribe, send any mail to "svn-src-all-unsubscr...@freebsd.org"

Re: svn commit: r303716 - head/crypto/openssh

[Andrey Chernov]

On 07.08.2016 23:40, Peter Jeremy wrote:
> On 2016-Aug-07 15:25:54 +0300, Andrey Chernov <a...@freebsd.org> wrote:
>> You should address your complains to original openssh author instead, it
>> was his decision to get rid of weak algos.
> 
> No.  It's up to the person who imported the code into FreeBSD to understand
> why the change was made and to be able to justify it to the FreeBSD
> community.  Firstly, security is not absolute - it's always a cost-benefit
> tradeoff and different communities may make different tradeoffs.  Secondly,
> the importer needs to be confident that the code is actually an improvement,
> not an attempt by a bad actor to undermine security.

It is pretty clear for everybody who interested in security why this
change is made and why it is actually an improvement. Tuning it (or not)
to different obsoleted environment and how to do it (if yes) is
completely another question which, IMHO will be better resolved
consulting with the author and not by mechanically restoring removed
weak stuff with each new openssh release.

>> In my personal opinion, if
>> your hardware is outdated, just drop it out.
> 
> This is part of the cost-benefit analysis.  Replacing hardware has a real
> cost.  If it's inside a datacentre, where the management LAN is isolated
> from the rest of the world, there may be virtually no benefit to disabling
> "weak" ciphers.

As I already say in this discussion twice, it is just my personal
opinion and I am not insisting on it. Just ignore it if you like.

> OTOH, FreeBSD has a documented deprecation process that says things will
> continue working for a major release after being formally deprecated.

FreeBSD 11 is not released yet (betas are not counted), stable-10 too,
so it is right time to deprecate for them.





signature.asc
Description: OpenPGP digital signature

Re: svn commit: r303716 - head/crypto/openssh

[Andrey Chernov]

On 07.08.2016 22:56, Slawa Olhovchenkov wrote:
> On Sun, Aug 07, 2016 at 10:42:56PM +0300, Andrey Chernov wrote:
> 
>> On 07.08.2016 22:10, Slawa Olhovchenkov wrote:
>>> On Sun, Aug 07, 2016 at 10:02:52PM +0300, Andrey Chernov wrote:
>>>
>>>> On 07.08.2016 21:52, Slawa Olhovchenkov wrote:
>>>>>> Why you still not
>>>>>> send your opinion to the author?
>>>>>>
>>>>>
>>>>> I am not sure about suitable response from autor.
>>>>> May be project [FreeBSD] choise some compromise.
>>>>
>>>> IMHO blindly choosing some compromise without asking author's opinion
>>>> first will be unwise. I will be glad in case someone from secteam@
>>>> discuss that with the author. Moreover, careless attempt to stay
>>>> compatible by any price can weak connections passed out of the server room.
>>>>
>>>
>>> In generaly I am accept this.
>>> For this specific case enforcing strong crypo like Internet filtering
>>> from suicide.
>>>
>>
>> BTW, there is at least one alternative. F.e. security/putty from ports
>> still support weak min DH 1024, diffie-hellman-group1-sha1 and even des
>> you need. I don't check others because I am too lazy.
> 
> Last time I am see (over six years ago) putty just don't work (my
> coworker try on Linux). I am to lazy check now.
> 
Good news: I just try it and it works.
___
svn-src-all@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-all
To unsubscribe, send any mail to "svn-src-all-unsubscr...@freebsd.org"

Re: svn commit: r303716 - head/crypto/openssh

[Andrey Chernov]

On 07.08.2016 22:10, Slawa Olhovchenkov wrote:
> On Sun, Aug 07, 2016 at 10:02:52PM +0300, Andrey Chernov wrote:
> 
>> On 07.08.2016 21:52, Slawa Olhovchenkov wrote:
>>>> Why you still not
>>>> send your opinion to the author?
>>>>
>>>
>>> I am not sure about suitable response from autor.
>>> May be project [FreeBSD] choise some compromise.
>>
>> IMHO blindly choosing some compromise without asking author's opinion
>> first will be unwise. I will be glad in case someone from secteam@
>> discuss that with the author. Moreover, careless attempt to stay
>> compatible by any price can weak connections passed out of the server room.
>>
> 
> In generaly I am accept this.
> For this specific case enforcing strong crypo like Internet filtering
> from suicide.
> 

BTW, there is at least one alternative. F.e. security/putty from ports
still support weak min DH 1024, diffie-hellman-group1-sha1 and even des
you need. I don't check others because I am too lazy.
___
svn-src-all@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-all
To unsubscribe, send any mail to "svn-src-all-unsubscr...@freebsd.org"

Re: svn commit: r303716 - head/crypto/openssh

[Andrey Chernov]

On 07.08.2016 21:52, Slawa Olhovchenkov wrote:
>> Why you still not
>> send your opinion to the author?
>>
> 
> I am not sure about suitable response from autor.
> May be project [FreeBSD] choise some compromise.

IMHO blindly choosing some compromise without asking author's opinion
first will be unwise. I will be glad in case someone from secteam@
discuss that with the author. Moreover, careless attempt to stay
compatible by any price can weak connections passed out of the server room.

___
svn-src-all@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-all
To unsubscribe, send any mail to "svn-src-all-unsubscr...@freebsd.org"

Re: svn commit: r303716 - head/crypto/openssh

[Andrey Chernov]

On 07.08.2016 21:23, Slawa Olhovchenkov wrote:
> On Sun, Aug 07, 2016 at 09:06:37PM +0300, Andrey Chernov wrote:
> 
>> On 07.08.2016 20:43, Andrey Chernov wrote:
>>> On 07.08.2016 20:37, Slawa Olhovchenkov wrote:
>>>> On Sun, Aug 07, 2016 at 08:34:55PM +0300, Andrey Chernov wrote:
>>>>
>>>>> On 07.08.2016 20:31, Andrey Chernov wrote:
>>>>>> On 07.08.2016 19:14, Bruce Simpson wrote:
>>>>>>> On 07/08/16 15:40, Warner Losh wrote:
>>>>>>>> That’s a cop-out answer. We, as a project, need to articulate to our
>>>>>>>> users, whom we care about, why this rather obnoxious hit to usability
>>>>>>>> was taken. The answer must be more complete than “We just disabled
>>>>>>>> it because upstream disabled it for reasons we’re too lazy to explain
>>>>>>>> or document how to work around"
>>>>>>>
>>>>>>> Alcatel-Lucent OmniSwitch 6800 login broken (pfSense 2.3.2 which
>>>>>>> accepted the upstream change, workaround no-go)
>>>>>>>
>>>>>>> [2.3.2-RELEASE][r...@gw.lab]/root: ssh -l admin
>>>>>>> -oKexAlgorithms=+diffie-hellman-group1-sha1 192.168.1.XXX
>>>>>>> Fssh_ssh_dispatch_run_fatal: Connection to 192.168.1.XXX port 22: DH GEX
>>>>>>> group out of range
>>>>>>>
>>>>>>
>>>>>> DH prime size must be at least 2048, openssh now refuse lower values.
>>>>>> Commonly used DH size 1024 can be easily broken. See https://weakdh.org
>>>>>>
>>>>> diffie-hellman-group1-sha1 use DH 1024 and insecure sha1 both.
>>>>
>>>> IMHO, this is wrong choise: totaly lost of control now vs teoretical
>>>> compromise of control in the future.
>>>
>>> Please note that it was not my choice and I can't answer what to do with
>>> non-upgradeable hardware question, address it to the author. I just tell
>>> you _why_ it happens.
>>>
>>
>> BTW, compromise is practical enough. From https://weakdh.org/ "A close
>> reading of published NSA leaks shows that the agency's attacks on VPNs
>> are consistent with having achieved such a break."
> 
> For this compromise need
> 
> 1) NSA interesed to me

This particular condition is not necessary, they can decrypt all traffic
with weak DH primes passed through main channels in USA and perhaps
partially in Europe (depends on mutual agreement), then find interesting
keywords to spy more closely afterwards.

> 2) NSA must be able to access to weak device for traffic
> intercept
> 
> This is imposible at this time.
> 
> Also, if NSA can be able to intercept such traffic weak crypto will be
> last resort of my trouble.

About the rest, I am not the person to argue with. Why you still not
send your opinion to the author?

___
svn-src-all@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-all
To unsubscribe, send any mail to "svn-src-all-unsubscr...@freebsd.org"

Re: svn commit: r303716 - head/crypto/openssh

[Andrey Chernov]

On 07.08.2016 20:43, Andrey Chernov wrote:
> On 07.08.2016 20:37, Slawa Olhovchenkov wrote:
>> On Sun, Aug 07, 2016 at 08:34:55PM +0300, Andrey Chernov wrote:
>>
>>> On 07.08.2016 20:31, Andrey Chernov wrote:
>>>> On 07.08.2016 19:14, Bruce Simpson wrote:
>>>>> On 07/08/16 15:40, Warner Losh wrote:
>>>>>> That’s a cop-out answer. We, as a project, need to articulate to our
>>>>>> users, whom we care about, why this rather obnoxious hit to usability
>>>>>> was taken. The answer must be more complete than “We just disabled
>>>>>> it because upstream disabled it for reasons we’re too lazy to explain
>>>>>> or document how to work around"
>>>>>
>>>>> Alcatel-Lucent OmniSwitch 6800 login broken (pfSense 2.3.2 which
>>>>> accepted the upstream change, workaround no-go)
>>>>>
>>>>> [2.3.2-RELEASE][r...@gw.lab]/root: ssh -l admin
>>>>> -oKexAlgorithms=+diffie-hellman-group1-sha1 192.168.1.XXX
>>>>> Fssh_ssh_dispatch_run_fatal: Connection to 192.168.1.XXX port 22: DH GEX
>>>>> group out of range
>>>>>
>>>>
>>>> DH prime size must be at least 2048, openssh now refuse lower values.
>>>> Commonly used DH size 1024 can be easily broken. See https://weakdh.org
>>>>
>>> diffie-hellman-group1-sha1 use DH 1024 and insecure sha1 both.
>>
>> IMHO, this is wrong choise: totaly lost of control now vs teoretical
>> compromise of control in the future.
> 
> Please note that it was not my choice and I can't answer what to do with
> non-upgradeable hardware question, address it to the author. I just tell
> you _why_ it happens.
> 

BTW, compromise is practical enough. From https://weakdh.org/ "A close
reading of published NSA leaks shows that the agency's attacks on VPNs
are consistent with having achieved such a break."

___
svn-src-all@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-all
To unsubscribe, send any mail to "svn-src-all-unsubscr...@freebsd.org"

Re: svn commit: r303716 - head/crypto/openssh

[Andrey Chernov]

On 07.08.2016 20:37, Slawa Olhovchenkov wrote:
> On Sun, Aug 07, 2016 at 08:34:55PM +0300, Andrey Chernov wrote:
> 
>> On 07.08.2016 20:31, Andrey Chernov wrote:
>>> On 07.08.2016 19:14, Bruce Simpson wrote:
>>>> On 07/08/16 15:40, Warner Losh wrote:
>>>>> That’s a cop-out answer. We, as a project, need to articulate to our
>>>>> users, whom we care about, why this rather obnoxious hit to usability
>>>>> was taken. The answer must be more complete than “We just disabled
>>>>> it because upstream disabled it for reasons we’re too lazy to explain
>>>>> or document how to work around"
>>>>
>>>> Alcatel-Lucent OmniSwitch 6800 login broken (pfSense 2.3.2 which
>>>> accepted the upstream change, workaround no-go)
>>>>
>>>> [2.3.2-RELEASE][r...@gw.lab]/root: ssh -l admin
>>>> -oKexAlgorithms=+diffie-hellman-group1-sha1 192.168.1.XXX
>>>> Fssh_ssh_dispatch_run_fatal: Connection to 192.168.1.XXX port 22: DH GEX
>>>> group out of range
>>>>
>>>
>>> DH prime size must be at least 2048, openssh now refuse lower values.
>>> Commonly used DH size 1024 can be easily broken. See https://weakdh.org
>>>
>> diffie-hellman-group1-sha1 use DH 1024 and insecure sha1 both.
> 
> IMHO, this is wrong choise: totaly lost of control now vs teoretical
> compromise of control in the future.

Please note that it was not my choice and I can't answer what to do with
non-upgradeable hardware question, address it to the author. I just tell
you _why_ it happens.

___
svn-src-all@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-all
To unsubscribe, send any mail to "svn-src-all-unsubscr...@freebsd.org"

Re: svn commit: r303716 - head/crypto/openssh

[Andrey Chernov]

On 07.08.2016 19:14, Bruce Simpson wrote:
> On 07/08/16 15:40, Warner Losh wrote:
>> That’s a cop-out answer. We, as a project, need to articulate to our
>> users, whom we care about, why this rather obnoxious hit to usability
>> was taken. The answer must be more complete than “We just disabled
>> it because upstream disabled it for reasons we’re too lazy to explain
>> or document how to work around"
> 
> Alcatel-Lucent OmniSwitch 6800 login broken (pfSense 2.3.2 which
> accepted the upstream change, workaround no-go)
> 
> [2.3.2-RELEASE][r...@gw.lab]/root: ssh -l admin
> -oKexAlgorithms=+diffie-hellman-group1-sha1 192.168.1.XXX
> Fssh_ssh_dispatch_run_fatal: Connection to 192.168.1.XXX port 22: DH GEX
> group out of range
> 

DH prime size must be at least 2048, openssh now refuse lower values.
Commonly used DH size 1024 can be easily broken. See https://weakdh.org
___
svn-src-all@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-all
To unsubscribe, send any mail to "svn-src-all-unsubscr...@freebsd.org"

Re: svn commit: r303716 - head/crypto/openssh

[Andrey Chernov]

On 07.08.2016 20:31, Andrey Chernov wrote:
> On 07.08.2016 19:14, Bruce Simpson wrote:
>> On 07/08/16 15:40, Warner Losh wrote:
>>> That’s a cop-out answer. We, as a project, need to articulate to our
>>> users, whom we care about, why this rather obnoxious hit to usability
>>> was taken. The answer must be more complete than “We just disabled
>>> it because upstream disabled it for reasons we’re too lazy to explain
>>> or document how to work around"
>>
>> Alcatel-Lucent OmniSwitch 6800 login broken (pfSense 2.3.2 which
>> accepted the upstream change, workaround no-go)
>>
>> [2.3.2-RELEASE][r...@gw.lab]/root: ssh -l admin
>> -oKexAlgorithms=+diffie-hellman-group1-sha1 192.168.1.XXX
>> Fssh_ssh_dispatch_run_fatal: Connection to 192.168.1.XXX port 22: DH GEX
>> group out of range
>>
> 
> DH prime size must be at least 2048, openssh now refuse lower values.
> Commonly used DH size 1024 can be easily broken. See https://weakdh.org
> 
diffie-hellman-group1-sha1 use DH 1024 and insecure sha1 both.
___
svn-src-all@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-all
To unsubscribe, send any mail to "svn-src-all-unsubscr...@freebsd.org"

Re: svn commit: r303716 - head/crypto/openssh

[Andrey Chernov]

On 07.08.2016 20:31, Andrey Chernov wrote:
> On 07.08.2016 19:14, Bruce Simpson wrote:
>> On 07/08/16 15:40, Warner Losh wrote:
>>> That’s a cop-out answer. We, as a project, need to articulate to our
>>> users, whom we care about, why this rather obnoxious hit to usability
>>> was taken. The answer must be more complete than “We just disabled
>>> it because upstream disabled it for reasons we’re too lazy to explain
>>> or document how to work around"
>>
>> Alcatel-Lucent OmniSwitch 6800 login broken (pfSense 2.3.2 which
>> accepted the upstream change, workaround no-go)
>>
>> [2.3.2-RELEASE][r...@gw.lab]/root: ssh -l admin
>> -oKexAlgorithms=+diffie-hellman-group1-sha1 192.168.1.XXX
>> Fssh_ssh_dispatch_run_fatal: Connection to 192.168.1.XXX port 22: DH GEX
>> group out of range
>>
> 
> DH prime size must be at least 2048, openssh now refuse lower values.
> Commonly used DH size 1024 can be easily broken. See https://weakdh.org
> 

___
svn-src-all@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-all
To unsubscribe, send any mail to "svn-src-all-unsubscr...@freebsd.org"

Re: svn commit: r303716 - head/crypto/openssh

[Andrey Chernov]

On 07.08.2016 17:40, Warner Losh wrote:
> 
>> On Aug 7, 2016, at 7:21 AM, Andrey Chernov <a...@freebsd.org> wrote:
>>>
>>>> We can't turn our security
>>>> team into compatibility team, by constantly restoring removed code, such
>>>> code quickly becomes outdated and may add new security holes even being
>>>> inactive.
>>>
>>> What is security hole by present this ciphers in _client_?
>>
>> It is obvious, but it will be better for you to ask openssh author about
>> his decisions, I have no intention to act like explainer of his action.
> 
> That’s a cop-out answer. We, as a project, need to articulate to our
> users, whom we care about, why this rather obnoxious hit to usability
> was taken. The answer must be more complete than “We just disabled
> it because upstream disabled it for reasons we’re too lazy to explain
> or document how to work around"

Maybe I am too lazy, but in this particular case I prefer explanation
from the author rather then my own explanations. In general my guessing
of author intentions related to compatibility case can be not correct
enough, so I don't want anybody relay on it. I.e. I don't want to
mislead anybody.

___
svn-src-all@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-all
To unsubscribe, send any mail to "svn-src-all-unsubscr...@freebsd.org"

Re: svn commit: r303716 - head/crypto/openssh

[Andrey Chernov]

On 07.08.2016 15:52, Slawa Olhovchenkov wrote:
>> You should address your complains to original openssh author instead, it
>> was his decision to get rid of weak algos. In my personal opinion, if
>> your hardware is outdated, just drop it out.
> 
> Hardware outdated by outdated main function, not by outdated ssh
> upstream.

There are too many reason for outdating hardware without losing its main
function in real world. But I don't want to conduct a lecture. As I
already say, it is just my personal opinion.

> 
>> We can't turn our security
>> team into compatibility team, by constantly restoring removed code, such
>> code quickly becomes outdated and may add new security holes even being
>> inactive.
> 
> What is security hole by present this ciphers in _client_?

It is obvious, but it will be better for you to ask openssh author about
his decisions, I have no intention to act like explainer of his action.


___
svn-src-all@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-all
To unsubscribe, send any mail to "svn-src-all-unsubscr...@freebsd.org"

Re: svn commit: r303716 - head/crypto/openssh

[Andrey Chernov]

On 07.08.2016 14:59, Bruce Simpson wrote:
> On 07/08/16 12:43, Oliver Pinter wrote:
>>> I was able to override this (somewhat unilateral, to my mind)
>>> deprecation of the DH key exchange by using this option:
>>> -oKexAlgorithms=+diffie-hellman-group1-sha1
>>
>> You can add this option to /etc/ssh/ssh.conf or ~/.ssh/config too.
> 
> Can this at least be added (commented out, if you really want to enforce
> this policy on users out-of-the-box) to the former file in FreeBSD
> itself? And a note added to UPDATING?
> 
> Otherwise, it's almost as though those behind the change are assuming
> that users will just know exactly what to do in their operational
> situation. That's a good way to cause problems for folk using FreeBSD in
> IT operations.
> 
> (systemd epitomises this kind of foot shooting.)
> 
> I understand already - you want to deprecate a set of key exchanges, and
> believe in setting an example - but the rest of the world might not be
> ready for that just yet.
> 

You should address your complains to original openssh author instead, it
was his decision to get rid of weak algos. In my personal opinion, if
your hardware is outdated, just drop it out. We can't turn our security
team into compatibility team, by constantly restoring removed code, such
code quickly becomes outdated and may add new security holes even being
inactive.
___
svn-src-all@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-all
To unsubscribe, send any mail to "svn-src-all-unsubscr...@freebsd.org"

Re: svn commit: r303264 - head/usr.bin/calendar/calendars/ru_RU.UTF-8

[Andrey Chernov]

On 02.08.2016 12:26, Gleb Smirnoff wrote:
> You got your own commit access now, so please do it yourself if you
> are confident enough about the change. I am not :)

Really this date is in the middle, see
http://bibliopskov.ru/4november.htm
___
svn-src-all@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-all
To unsubscribe, send any mail to "svn-src-all-unsubscr...@freebsd.org"

Re: svn commit: r303530 - head/lib/libc/gen

[Andrey Chernov]

On 30.07.2016 11:52, Bruce Evans wrote:
> On Sat, 30 Jul 2016, Andrey Chernov wrote:
> 
>> On 30.07.2016 7:15, Bruce Evans wrote:
>>> On Sat, 30 Jul 2016, Andrey A. Chernov wrote:
>>>
>>>> Log:
>>>>  Reset errno for readdirfunc() before contunue.
>>>
>>> In C99, library functions are not permitted to set errno to 0.  The
>>> glob()
>>> family shouldn't use a different (worse) convention, and POSIX doesn't
>>> seem to have any special wording to allow different behaviour.

BTW, POSIX says so too: "No function in this volume of POSIX.1-2008
shall set errno to 0."

>> This is historic practice for this function at least since GLOB_LIMIT
>> was introduced (in 2001) and common across NetBSD/OpenBSD. Existent
>> programs may relay on that to check that limit is reached and not
>> allocation error, so with few additional overwriting from my side I add
>> nothing new:
>>
>> Revision 80525
>> Modified Sun Jul 29 00:52:37 2001 UTC (15 years ago) by mikeh
>> ...
>> errno = 0 is documented. See glob(3), GLOB_NOSPACE section too.
> 
> Hmm.  This is not in the BUGS section.  Strangely, the related non-bug
> that glob() may fail and set errno (to a reasonable but nonstandard and
> undocumented value?) is in the BUGS section.  I guess this is a bug
> in conjunction with the promise to set errno to 0 for some failures.
> 
>> The real problem is that glob(3) is very limited in error return codes,
>> so they reuse existent codes with errno hack.
> 
> It should have just used the least-unclosely-related standard errno
> instead of 0.

I was thinking about this way too, f.e. E2BIG or ERANGE are nice
candidatures, but still can't decide to break cross-BSD compatibility
and historic practice.

___
svn-src-all@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-all
To unsubscribe, send any mail to "svn-src-all-unsubscr...@freebsd.org"

Re: svn commit: r303530 - head/lib/libc/gen

[Andrey Chernov]

On 30.07.2016 7:15, Bruce Evans wrote:
> On Sat, 30 Jul 2016, Andrey A. Chernov wrote:
> 
>> Log:
>>  Reset errno for readdirfunc() before contunue.
> 
> In C99, library functions are not permitted to set errno to 0.  The glob()
> family shouldn't use a different (worse) convention, and POSIX doesn't
> seem to have any special wording to allow different behaviour.

This is historic practice for this function at least since GLOB_LIMIT
was introduced (in 2001) and common across NetBSD/OpenBSD. Existent
programs may relay on that to check that limit is reached and not
allocation error, so with few additional overwriting from my side I add
nothing new:

Revision 80525
Modified Sun Jul 29 00:52:37 2001 UTC (15 years ago) by mikeh

Rename the GLOB_MAXPATH flag of glob(3) to GLOB_LIMIT to be compatible
with NetBSD and OpenBSD. glob(3) will now return GLOB_NOSPACE with
errno set to 0 instead of GLOB_LIMIT when we match more than `gl_matchc'
patterns. GLOB_MAXPATH has been left as an alias of GLOB_LIMIT to
maintain backwards compatibility.

errno = 0 is documented. See glob(3), GLOB_NOSPACE section too.

The real problem is that glob(3) is very limited in error return codes,
so they reuse existent codes with errno hack.
___
svn-src-all@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-all
To unsubscribe, send any mail to "svn-src-all-unsubscr...@freebsd.org"

Re: svn commit: r303354 - head/share/timedef

[Andrey Chernov]

On 27.07.2016 3:46, Kevin Lo wrote:
> Author: kevlo
> Date: Wed Jul 27 00:46:48 2016
> New Revision: 303354
> URL: https://svnweb.freebsd.org/changeset/base/303354
> 
> Log:
>   Add weekday to the date format.

This file is generated, so this change will be lost after next
generation. It will be better to fix generation script instead.

> 
> Modified:
>   head/share/timedef/zh_TW.UTF-8.src
> 
> Modified: head/share/timedef/zh_TW.UTF-8.src
> ==
> --- head/share/timedef/zh_TW.UTF-8.srcWed Jul 27 00:08:01 2016
> (r303353)
> +++ head/share/timedef/zh_TW.UTF-8.srcWed Jul 27 00:46:48 2016
> (r303354)
> @@ -56,14 +56,14 @@
>  %Y/%m/%d
>  #
>  # c_fmt
> -%Y年%m月%e日 %p%I:%M:%S [%Z]
> +%Y年%m月%e日%A %p%I:%M:%S [%Z]
>  #
>  # AM/PM
>  上午
>  下午
>  #
>  # date_fmt
> -%Y年%m月%e日 %p%I:%M:%S [%Z]
> +%Y年%m月%e日%A %p%I:%M:%S [%Z]
>  #
>  # Long month names (without case ending)
>  1月
> 

___
svn-src-all@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-all
To unsubscribe, send any mail to "svn-src-all-unsubscr...@freebsd.org"

Re: svn commit: r303218 - head/tools/tools/locale/tools

[Andrey Chernov]

On 23.07.2016 15:32, Baptiste Daroussin wrote:
> On Sat, Jul 23, 2016 at 02:58:31PM +0300, Andrey Chernov wrote:
>> On 23.07.2016 14:52, Baptiste Daroussin wrote:
>>> On Sat, Jul 23, 2016 at 02:41:39PM +0300, Andrey Chernov wrote:
>>>> On 23.07.2016 14:24, Baptiste Daroussin wrote:
>>>>> Author: bapt
>>>>> Date: Sat Jul 23 11:24:05 2016
>>>>> New Revision: 303218
>>>>> URL: https://svnweb.freebsd.org/changeset/base/303218
>>>>>
>>>>> Log:
>>>>>   Add another hack to add weekday to date format to more locales
>>>>>
>>>>>   While here only initialize the iconv converter when needed
>>>>>
>>>>> Modified:
>>>>>   head/tools/tools/locale/tools/cldr2def.pl
>>>>
>>>> BTW, is there any progress in generating all collates with -m ${area} as
>>>> you planned?
>>>>
>>> you mean the @modifier?
>>
>> No, I mean that:
>>>> Perhaps all uk_UA should be generated with -m ${area} too - Ukrainians
>>>> have extended Cyrillics in their alphabet and I am not sure that their
>>>> order match default Unicode order for Cyrillic. The same about be_BY,
>>>> bg_BG and sr_Cyrl_RS. They may match or may not, I don't check such
>>> deeply.
>>>>
>>>> It seems the safest side will be to generate all with -m ${area}
>>>>
>>> Yes that is my plan, except for the 1 or 2 that cannot be generated
>> >from cldr
>>> directly
>>
> 
> Nope I haven't made any progress, and I actually did a bunch of tests of some
> (honnestly not all locales) trying to get a representative set and could not
> find differnce in ordering between locales generated with -m ${area}.
> 
> It would also needs lots of work on the generation tools.
> 
> I first need to write a tool that compares collation rules for various ${area}
> (not done yet) to see how much this is really needed and use it in the
> generation tools.
> 
> I will try to do that for 12 if it is really worth it (based on the result 
> from
> that comparison tools)

IMHO we don't need comparison tools, just blindly adding -m ${area} to
generating of every collate with different area be enough. CLDR may
involve from time to time including national alphabets changes (which
may happen often enough too), so it will be better to not run such tests
every time with new CLDR import, but just the make code which do it
correctly independently (-m ${area}) and forget about it, it will be
much easier to maintain. Moreover, you'll spend more time writing such
testing tool than adding -m ${area} :)





signature.asc
Description: OpenPGP digital signature

Re: svn commit: r303264 - head/usr.bin/calendar/calendars/ru_RU.UTF-8

[Andrey Chernov]

What happens with the lines below?

On 24.07.2016 13:35, Gleb Smirnoff wrote:
> Modified: head/usr.bin/calendar/calendars/ru_RU.UTF-8/calendar.orthodox
> -Пасха-7  Вход Господень в Иерусалим. Вербное Воскресенье
> +Пасха-7  Вход Господень в Иерусалим. Вербное Воскресент.ье
> +ПасхаВоскресент.ие Христово
> +Пасха+39 Вознесент.ие

> Modified: head/usr.bin/calendar/calendars/ru_RU.UTF-8/calendar.pagan
> -14 мар   Новый Год, Овсень малый
> +14 марта Новый Год, Овсент.ь малый
> -20 мар*  Весенние равноденствие
> - 7 апр   День Марены (сдвинутое весеннее равноденствие)
> +20 марта*Весент.ние равноденствие
> + 7 апр.  День Марены (сдвинутое весент.нее равноденствие)
> + 6 мая   День Дажьбога, Овсент.ь большой

___
svn-src-all@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-all
To unsubscribe, send any mail to "svn-src-all-unsubscr...@freebsd.org"

Re: svn commit: r303218 - head/tools/tools/locale/tools

[Andrey Chernov]

On 23.07.2016 14:52, Baptiste Daroussin wrote:
> On Sat, Jul 23, 2016 at 02:41:39PM +0300, Andrey Chernov wrote:
>> On 23.07.2016 14:24, Baptiste Daroussin wrote:
>>> Author: bapt
>>> Date: Sat Jul 23 11:24:05 2016
>>> New Revision: 303218
>>> URL: https://svnweb.freebsd.org/changeset/base/303218
>>>
>>> Log:
>>>   Add another hack to add weekday to date format to more locales
>>>
>>>   While here only initialize the iconv converter when needed
>>>
>>> Modified:
>>>   head/tools/tools/locale/tools/cldr2def.pl
>>
>> BTW, is there any progress in generating all collates with -m ${area} as
>> you planned?
>>
> you mean the @modifier?

No, I mean that:
>> Perhaps all uk_UA should be generated with -m ${area} too - Ukrainians
>> have extended Cyrillics in their alphabet and I am not sure that their
>> order match default Unicode order for Cyrillic. The same about be_BY,
>> bg_BG and sr_Cyrl_RS. They may match or may not, I don't check such
>deeply.
>>
>> It seems the safest side will be to generate all with -m ${area}
>>
>Yes that is my plan, except for the 1 or 2 that cannot be generated
>from cldr
>directly




signature.asc
Description: OpenPGP digital signature

Re: svn commit: r303218 - head/tools/tools/locale/tools

[Andrey Chernov]

On 23.07.2016 14:24, Baptiste Daroussin wrote:
> Author: bapt
> Date: Sat Jul 23 11:24:05 2016
> New Revision: 303218
> URL: https://svnweb.freebsd.org/changeset/base/303218
> 
> Log:
>   Add another hack to add weekday to date format to more locales
>
>   While here only initialize the iconv converter when needed
>
> Modified:
>   head/tools/tools/locale/tools/cldr2def.pl

BTW, is there any progress in generating all collates with -m ${area} as
you planned?

___
svn-src-all@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-all
To unsubscribe, send any mail to "svn-src-all-unsubscr...@freebsd.org"

Re: svn commit: r302601 - in head/sys: arm/include arm64/include

[Andrey Chernov]

On 12.07.2016 11:35, Andrew Turner wrote:
> On Tue, 12 Jul 2016 00:37:48 + (UTC)
> "Andrey A. Chernov"  wrote:
> 
>> Author: ache
>> Date: Tue Jul 12 00:37:48 2016
>> New Revision: 302601
>> URL: https://svnweb.freebsd.org/changeset/base/302601
>>
>> Log:
>>   I don't know why unsigned int is choosed for wchar_t here, but
>> WCHAR_MAX should be <= WINT_MAX. It is bigger, __UINT_MAX > INT32_MAX
> 
> Because the ABI either requires us to use an unsigned int [1], or the
> preferred type is unsigned int [2]. In the latter case the other choice
> is unsigned short, it would seem this is for Windows.

Thanx for explanation. Perhaps we need to use 32bit unsigned int for
other architectures too (instead of 32bit signed int), because no
L'' literals produce negative value and locale enumerates positive
values only.

BTW, this commit is already backed out.

> 
> Andrew
> 
> [1]
> http://infocenter.arm.com/help/topic/com.arm.doc.ihi0055c/IHI0055C_beta_aapcs64.pdf
> [2]
> http://infocenter.arm.com/help/topic/com.arm.doc.ihi0042f/IHI0042F_aapcs.pdf
> 

___
svn-src-all@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-all
To unsubscribe, send any mail to "svn-src-all-unsubscr...@freebsd.org"

Re: svn commit: r302026 - in head: share/monetdef share/msgdef share/numericdef share/timedef tools/tools/locale/tools

[Andrey Chernov]

On 21.06.2016 10:12, Baptiste Daroussin wrote:
> On Mon, Jun 20, 2016 at 10:14:04PM +0300, Andrey Chernov wrote:
>> On 20.06.2016 9:45, Baptiste Daroussin wrote:
>>> Author: bapt
>>> Date: Mon Jun 20 06:45:42 2016
>>> New Revision: 302026
>>> URL: https://svnweb.freebsd.org/changeset/base/302026
>>>
>>> Log:
>>>   Fix generation of locales with multiple variants
>>
>> Thanx.
>> Just want to note, even if we stay with RFC 5646 language tags instead
>> of ISO 639 ones with @modifier (per ISO 15897), current tags are
>> incorrect because have "_" instead of "-" which makes parsing harder,
>> because "_" is territory separator and someone may not expect several
>> "_" exists. Per RFC 5646 we need names like
>> sr-Cyrl_RS.UTF-8.src
>> and not
>> sr_Cyrl_RS.UTF-8.src
>>
> I have a patch that create the @modifier version meaning
> for instance:
> sr_RS.UTF-8@[modifier]
> 
> it also adds an alias sr_RS.UTF-8 which is the cyrillic version (following the
> what has been done on linux for this locale)
> 
> I am seeking for your opinion on a policy to handle the locales with variants.
> I am hesitating between 2 options:
> 1/ Provide all locales that may have modifier:
> 
> - for sr_RS:
> sr_RS.UTF-8@cyrillic
> sr_RS.UTF-8@latin
> 
> and sr_RS.UTF-8 (which is actually the same as sr_RS.UTF-8@cyrillic)
> 
> - for zh_TW
> zh_TW.UTF-8@hant
>  and zh_TW.UTF-8 (which is an alias on zh_TW.UTF-8@hant)
> 
> - for mn_MN
> mn_MN.UTF-8@cyrillic
> mn_MN.UTF-8 (which is an alias on mn_MN.UTF-8@cyrillic)
> 
> 2/ Only provide the @version for the ones for which we have an ambiguity
> 
> - for sr_RS:
> sr_RS.UTF-8@latin
> sr_RS.UTF-8 (would be the cyrillic one)
> 
> - for zh_TW
> zh_TW.UTF-8 (no @modifier version)
> 
> - for mn_MN
> mn_MN.UTF-8 (no @modifier version)
> 
> 
> I do like the first (more explicit and simpler to do with our code while still
> compatible with the second). Linux only does the second.
> 
> But I understand the first can be confusing for languages with (for now) only
> one variant supported like users asking themselves:
> which one should I choose: mn_MN.UTF-8 or mn_MN.UTF-8@cyrillic?
> They might not now they are actually the same
> 
> Any opinion?

Since @modifier exists just to avoid ambiguity, we definitely don't need
to add f.e. @cyrillic to every cyrillic-based locale and @latin to every
latin-based one, and so on.
Difference between no @modifier and some @modifier is enough to avoid
ambiguity too, so I vote for Linux way.
GNU libintl drops any @modifier in any case and use its own default
encoding it tries to convert to user one later.




signature.asc
Description: OpenPGP digital signature

Re: svn commit: r302026 - in head: share/monetdef share/msgdef share/numericdef share/timedef tools/tools/locale/tools

[Andrey Chernov]

On 20.06.2016 9:45, Baptiste Daroussin wrote:
> Author: bapt
> Date: Mon Jun 20 06:45:42 2016
> New Revision: 302026
> URL: https://svnweb.freebsd.org/changeset/base/302026
> 
> Log:
>   Fix generation of locales with multiple variants

Thanx.
Just want to note, even if we stay with RFC 5646 language tags instead
of ISO 639 ones with @modifier (per ISO 15897), current tags are
incorrect because have "_" instead of "-" which makes parsing harder,
because "_" is territory separator and someone may not expect several
"_" exists. Per RFC 5646 we need names like
sr-Cyrl_RS.UTF-8.src
and not
sr_Cyrl_RS.UTF-8.src

___
svn-src-all@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-all
To unsubscribe, send any mail to "svn-src-all-unsubscr...@freebsd.org"

Re: svn commit: r302016 - in head: share/timedef tools/tools/locale/tools

[Andrey Chernov]

On 19.06.2016 22:40, Baptiste Daroussin wrote:
> On Sun, Jun 19, 2016 at 10:02:26PM +0300, Andrey Chernov wrote:
>> On 19.06.2016 20:21, Andrey Chernov wrote:
>>> On 19.06.2016 20:10, Baptiste Daroussin wrote:
>>>> On Sun, Jun 19, 2016 at 08:05:30PM +0300, Andrey Chernov wrote:
>>>>> On 19.06.2016 19:52, Andrey Chernov wrote:
>>>>>> On 19.06.2016 19:47, Andrey Chernov wrote:
>>>>>>> On 19.06.2016 18:49, Baptiste Daroussin wrote:
>>>>>>>> On Sun, Jun 19, 2016 at 04:52:34PM +0200, Baptiste Daroussin wrote:
>>>>>>>>> On Sun, Jun 19, 2016 at 07:57:49AM +0300, Andrey Chernov wrote:
>>>>>>>>>> On 19.06.2016 6:44, Hajimu UMEMOTO wrote:
>>>>>>>>>>> Log:
>>>>>>>>>>>   - Prefer to use %d over %e where the day of the month should be 
>>>>>>>>>>> zero
>>>>>>>>>>> filled.
>>>>>>>>>>>   - Since %e means the day of the month as well, regard %e as same 
>>>>>>>>>>> as %d
>>>>>>>>>>> in md_order.
>>>>>>>>>>
>>>>>>>>>> Nonsense doubled formats in sr_*_RS locales and nonsense md_order 
>>>>>>>>>> there too.
>>>>>>>>>
>>>>>>>>> Crap I'll dig into it, thanks for spotting it.
>>>>>>>>>
>>>>>>>>> Best regards,
>>>>>>>>> Bapt
>>>>>>>>
>>>>>>>> Should be fixed by:
>>>>>>>> https://people.freebsd.org/~bapt/locale-triple-components.diff
>>>>>>>>
>>>>>>>> The perl script generating the locales was badly handling locales with 
>>>>>>>> 3
>>>>>>>> components, fixed now.
>>>>>>>>
>>>>>>>> Sorry about that
>>>>>>>>
>>>>>>>> Best regards,
>>>>>>>> Bapt
>>>>>>>>
>>>>>>>
>>>>>>> sr locales are badly named before anything else, proper format is
>>>>>>> language[_COUNTRY][.encoding][@variant]
>>>>>>> I.e.
>>>>>>> sr_RS.UTF-8@Latn
>>>>>>> sr_RS.ISO8859-2@Latn
>>>>>>> sr_RS.ISO8859-t@Cyrl
>>>>>>>
>>>>>>> I don't know, if out utilities (f.e. locale(1)) can handle @variant 
>>>>>>> suffix.
>>>>>>>
>>>>>>>
>>>>>> =sr_RS.ISO8859-5@Cyrl
>>>>>> =our, and not utilities only but libc too
>>>>>
>>>>> Since right now we have only single @variant per each encoding, and no
>>>>> @variant for other locales, we can just drop it and use right names like:
>>>>> sr_RS.UTF-8
>>>>
>>>> We have 2 for the above the Cyrl version and the Latn version
>>>>
>>>>> sr_RS.ISO8859-2
>>>>> sr_RS.ISO8859-5
>>>>
>>>> We can make aliases easily for that is that is what people want, not the 
>>>> we have
>>>> the same for other locales:
>>>>
>>>> zh_Hans_CN.GB18030
>>>> zh_Hans_CN.GB2312
>>>> zh_Hans_CN.GBK
>>>> zh_Hans_CN.UTF-8
>>>> zh_Hans_CN.eucCN
>>>> zh_Hant_HK.UTF-8
>>>> zh_Hant_TW.Big5
>>>> zh_Hant_TW.UTF-8
>>>>
>>>> With some aliases for some of the previously existing ones:
>>>> zh_TW.Big5
>>>> zh_TW.UTF-8
>>>> zh_CN.GB18030
>>>> zh_CN.GB2312
>>>> zh_CN.GBK
>>>> zh_CN.UTF-8
>>>> zh_CN.eucCN
>>>> zh_HK.UTF-8
>>>>
>>>> Some of the new locales (that didn't exist before) I have only imported 
>>>> the new
>>>> name format:
>>>> kk_Cyrl_KZ.UTF-8
>>>> mn_Cyrl_MN.UTF-8
>>>> sr_Cyrl_RS.ISO8859-5
>>>> sr_Cyrl_RS.UTF-8
>>>> sr_Latn_RS.ISO8859-2
>>>> sr_Latn_RS.UTF-8
>>>>
>>>> I would prefer staying on the new syntax given it is allowed by some RFC 
>>>> and it
>>>> is slowly being adopted everywhere else.
>>>>
>>>> Btw another RFC: https://www.ietf.org/rfc/rfc5646.txt
>>>>
>>>> Best regards,
>>>> Bapt
>>>>
>>>
>>> Old POSIX don't use RFC 5646, there are ISO-639 two-letter codes. When
>>> encodings are different, there is no needs to specify variants, they
>>> have meaning only for the same encoding.
>>>
>>>
>>
>> I found POSIX reference about @-syntax:
>> [language[_territory][.codeset][@modifier]]
>> http://pubs.opengroup.org/onlinepubs/9699919799/basedefs/V1_chap08.html
>> It says nothing about language, territory or codeset standards.
>>
>> They comes from another standard, ISO 15897:
>> "8. Natural language, as specified in ISO 639
>> 9. Territory, as two-letter form of ISO 3166"
>> I can't find there something about codeset/charmap standartization, but
>> perhaps I just overlook some reference.
>>
> 
> Thanks a lot for that reference I wonder how I failed at finding it, I read
> twice the locale definition in opengroup documents.
> 
> I have found yet proof in the code, but from my testings (using nl_langinfo())
> 
> I can see that the @modifier is properly supported.
> 
> I have 2 proposal then:
> 1/ Keep the current version and add aliases with "@" for the one in variants
> 2/ Rename all the variants we have now into the POSIX version with "@"

To minimize confusion among people and better cross-platform
compatibility IMHO we need to see what Linux have here (in parts it not
violates POSIX and ISO standards, of course). We have different charmap
names (ISO-* vs. ISO*), and we have it already too long to change as
many people thinks, but new sr and other locales can be easily set in
line while they are still new.




signature.asc
Description: OpenPGP digital signature

Re: svn commit: r302016 - in head: share/timedef tools/tools/locale/tools

[Andrey Chernov]

On 19.06.2016 20:21, Andrey Chernov wrote:
> On 19.06.2016 20:10, Baptiste Daroussin wrote:
>> On Sun, Jun 19, 2016 at 08:05:30PM +0300, Andrey Chernov wrote:
>>> On 19.06.2016 19:52, Andrey Chernov wrote:
>>>> On 19.06.2016 19:47, Andrey Chernov wrote:
>>>>> On 19.06.2016 18:49, Baptiste Daroussin wrote:
>>>>>> On Sun, Jun 19, 2016 at 04:52:34PM +0200, Baptiste Daroussin wrote:
>>>>>>> On Sun, Jun 19, 2016 at 07:57:49AM +0300, Andrey Chernov wrote:
>>>>>>>> On 19.06.2016 6:44, Hajimu UMEMOTO wrote:
>>>>>>>>> Log:
>>>>>>>>>   - Prefer to use %d over %e where the day of the month should be zero
>>>>>>>>> filled.
>>>>>>>>>   - Since %e means the day of the month as well, regard %e as same as 
>>>>>>>>> %d
>>>>>>>>> in md_order.
>>>>>>>>
>>>>>>>> Nonsense doubled formats in sr_*_RS locales and nonsense md_order 
>>>>>>>> there too.
>>>>>>>
>>>>>>> Crap I'll dig into it, thanks for spotting it.
>>>>>>>
>>>>>>> Best regards,
>>>>>>> Bapt
>>>>>>
>>>>>> Should be fixed by:
>>>>>> https://people.freebsd.org/~bapt/locale-triple-components.diff
>>>>>>
>>>>>> The perl script generating the locales was badly handling locales with 3
>>>>>> components, fixed now.
>>>>>>
>>>>>> Sorry about that
>>>>>>
>>>>>> Best regards,
>>>>>> Bapt
>>>>>>
>>>>>
>>>>> sr locales are badly named before anything else, proper format is
>>>>> language[_COUNTRY][.encoding][@variant]
>>>>> I.e.
>>>>> sr_RS.UTF-8@Latn
>>>>> sr_RS.ISO8859-2@Latn
>>>>> sr_RS.ISO8859-t@Cyrl
>>>>>
>>>>> I don't know, if out utilities (f.e. locale(1)) can handle @variant 
>>>>> suffix.
>>>>>
>>>>>
>>>> =sr_RS.ISO8859-5@Cyrl
>>>> =our, and not utilities only but libc too
>>>
>>> Since right now we have only single @variant per each encoding, and no
>>> @variant for other locales, we can just drop it and use right names like:
>>> sr_RS.UTF-8
>>
>> We have 2 for the above the Cyrl version and the Latn version
>>
>>> sr_RS.ISO8859-2
>>> sr_RS.ISO8859-5
>>
>> We can make aliases easily for that is that is what people want, not the we 
>> have
>> the same for other locales:
>>
>> zh_Hans_CN.GB18030
>> zh_Hans_CN.GB2312
>> zh_Hans_CN.GBK
>> zh_Hans_CN.UTF-8
>> zh_Hans_CN.eucCN
>> zh_Hant_HK.UTF-8
>> zh_Hant_TW.Big5
>> zh_Hant_TW.UTF-8
>>
>> With some aliases for some of the previously existing ones:
>> zh_TW.Big5
>> zh_TW.UTF-8
>> zh_CN.GB18030
>> zh_CN.GB2312
>> zh_CN.GBK
>> zh_CN.UTF-8
>> zh_CN.eucCN
>> zh_HK.UTF-8
>>
>> Some of the new locales (that didn't exist before) I have only imported the 
>> new
>> name format:
>> kk_Cyrl_KZ.UTF-8
>> mn_Cyrl_MN.UTF-8
>> sr_Cyrl_RS.ISO8859-5
>> sr_Cyrl_RS.UTF-8
>> sr_Latn_RS.ISO8859-2
>> sr_Latn_RS.UTF-8
>>
>> I would prefer staying on the new syntax given it is allowed by some RFC and 
>> it
>> is slowly being adopted everywhere else.
>>
>> Btw another RFC: https://www.ietf.org/rfc/rfc5646.txt
>>
>> Best regards,
>> Bapt
>>
> 
> Old POSIX don't use RFC 5646, there are ISO-639 two-letter codes. When
> encodings are different, there is no needs to specify variants, they
> have meaning only for the same encoding.
> 
> 

I found POSIX reference about @-syntax:
[language[_territory][.codeset][@modifier]]
http://pubs.opengroup.org/onlinepubs/9699919799/basedefs/V1_chap08.html
It says nothing about language, territory or codeset standards.

They comes from another standard, ISO 15897:
"8. Natural language, as specified in ISO 639
9. Territory, as two-letter form of ISO 3166"
I can't find there something about codeset/charmap standartization, but
perhaps I just overlook some reference.






signature.asc
Description: OpenPGP digital signature

Re: svn commit: r302016 - in head: share/timedef tools/tools/locale/tools

[Andrey Chernov]

On 19.06.2016 20:10, Baptiste Daroussin wrote:
> On Sun, Jun 19, 2016 at 08:05:30PM +0300, Andrey Chernov wrote:
>> On 19.06.2016 19:52, Andrey Chernov wrote:
>>> On 19.06.2016 19:47, Andrey Chernov wrote:
>>>> On 19.06.2016 18:49, Baptiste Daroussin wrote:
>>>>> On Sun, Jun 19, 2016 at 04:52:34PM +0200, Baptiste Daroussin wrote:
>>>>>> On Sun, Jun 19, 2016 at 07:57:49AM +0300, Andrey Chernov wrote:
>>>>>>> On 19.06.2016 6:44, Hajimu UMEMOTO wrote:
>>>>>>>> Log:
>>>>>>>>   - Prefer to use %d over %e where the day of the month should be zero
>>>>>>>> filled.
>>>>>>>>   - Since %e means the day of the month as well, regard %e as same as 
>>>>>>>> %d
>>>>>>>> in md_order.
>>>>>>>
>>>>>>> Nonsense doubled formats in sr_*_RS locales and nonsense md_order there 
>>>>>>> too.
>>>>>>
>>>>>> Crap I'll dig into it, thanks for spotting it.
>>>>>>
>>>>>> Best regards,
>>>>>> Bapt
>>>>>
>>>>> Should be fixed by:
>>>>> https://people.freebsd.org/~bapt/locale-triple-components.diff
>>>>>
>>>>> The perl script generating the locales was badly handling locales with 3
>>>>> components, fixed now.
>>>>>
>>>>> Sorry about that
>>>>>
>>>>> Best regards,
>>>>> Bapt
>>>>>
>>>>
>>>> sr locales are badly named before anything else, proper format is
>>>> language[_COUNTRY][.encoding][@variant]
>>>> I.e.
>>>> sr_RS.UTF-8@Latn
>>>> sr_RS.ISO8859-2@Latn
>>>> sr_RS.ISO8859-t@Cyrl
>>>>
>>>> I don't know, if out utilities (f.e. locale(1)) can handle @variant suffix.
>>>>
>>>>
>>> =sr_RS.ISO8859-5@Cyrl
>>> =our, and not utilities only but libc too
>>
>> Since right now we have only single @variant per each encoding, and no
>> @variant for other locales, we can just drop it and use right names like:
>> sr_RS.UTF-8
> 
> We have 2 for the above the Cyrl version and the Latn version
> 
>> sr_RS.ISO8859-2
>> sr_RS.ISO8859-5
> 
> We can make aliases easily for that is that is what people want, not the we 
> have
> the same for other locales:
> 
> zh_Hans_CN.GB18030
> zh_Hans_CN.GB2312
> zh_Hans_CN.GBK
> zh_Hans_CN.UTF-8
> zh_Hans_CN.eucCN
> zh_Hant_HK.UTF-8
> zh_Hant_TW.Big5
> zh_Hant_TW.UTF-8
> 
> With some aliases for some of the previously existing ones:
> zh_TW.Big5
> zh_TW.UTF-8
> zh_CN.GB18030
> zh_CN.GB2312
> zh_CN.GBK
> zh_CN.UTF-8
> zh_CN.eucCN
> zh_HK.UTF-8
> 
> Some of the new locales (that didn't exist before) I have only imported the 
> new
> name format:
> kk_Cyrl_KZ.UTF-8
> mn_Cyrl_MN.UTF-8
> sr_Cyrl_RS.ISO8859-5
> sr_Cyrl_RS.UTF-8
> sr_Latn_RS.ISO8859-2
> sr_Latn_RS.UTF-8
> 
> I would prefer staying on the new syntax given it is allowed by some RFC and 
> it
> is slowly being adopted everywhere else.
> 
> Btw another RFC: https://www.ietf.org/rfc/rfc5646.txt
> 
> Best regards,
> Bapt
> 

Old POSIX don't use RFC 5646, there are ISO-639 two-letter codes. When
encodings are different, there is no needs to specify variants, they
have meaning only for the same encoding.




signature.asc
Description: OpenPGP digital signature

Re: svn commit: r302016 - in head: share/timedef tools/tools/locale/tools

[Andrey Chernov]

On 19.06.2016 20:05, Baptiste Daroussin wrote:
> On Sun, Jun 19, 2016 at 07:47:27PM +0300, Andrey Chernov wrote:
>> On 19.06.2016 18:49, Baptiste Daroussin wrote:
>>> On Sun, Jun 19, 2016 at 04:52:34PM +0200, Baptiste Daroussin wrote:
>>>> On Sun, Jun 19, 2016 at 07:57:49AM +0300, Andrey Chernov wrote:
>>>>> On 19.06.2016 6:44, Hajimu UMEMOTO wrote:
>>>>>> Log:
>>>>>>   - Prefer to use %d over %e where the day of the month should be zero
>>>>>> filled.
>>>>>>   - Since %e means the day of the month as well, regard %e as same as %d
>>>>>> in md_order.
>>>>>
>>>>> Nonsense doubled formats in sr_*_RS locales and nonsense md_order there 
>>>>> too.
>>>>
>>>> Crap I'll dig into it, thanks for spotting it.
>>>>
>>>> Best regards,
>>>> Bapt
>>>
>>> Should be fixed by:
>>> https://people.freebsd.org/~bapt/locale-triple-components.diff
>>>
>>> The perl script generating the locales was badly handling locales with 3
>>> components, fixed now.
>>>
>>> Sorry about that
>>>
>>> Best regards,
>>> Bapt
>>>
>>
>> sr locales are badly named before anything else, proper format is
>> language[_COUNTRY][.encoding][@variant]
>> I.e.
>> sr_RS.UTF-8@Latn
>> sr_RS.ISO8859-2@Latn
>> sr_RS.ISO8859-t@Cyrl
> 
> According to what?
>>
>> I don't know, if out utilities (f.e. locale(1)) can handle @variant suffix.
>>
> Our syntax is the syntax for the variants that has been the widly adopted:
> Microsoft, ICU, java, python's babel internationalisation thing,
> cldr (of course), CPAN components (Datime::Locales for example)
> 
> This is respecting RFC 3066 with the little difference that we use '_' instead
> of '-'
> 
> Best regards,
> Bapt
> 

According to POSIX, but I can't find the reference right now, it is
related to XPG3 or XPG4. It was already many years.
First reference from Google
http://www.boost.org/doc/libs/1_60_0/libs/locale/doc/html/locale_gen.html
Next from Gentoo:
https://wiki.gentoo.org/wiki/Localization/Guide
Microsoft, Linux and GNU iconv understand @-syntax





signature.asc
Description: OpenPGP digital signature

Re: svn commit: r302016 - in head: share/timedef tools/tools/locale/tools

[Andrey Chernov]

On 19.06.2016 19:52, Andrey Chernov wrote:
> On 19.06.2016 19:47, Andrey Chernov wrote:
>> On 19.06.2016 18:49, Baptiste Daroussin wrote:
>>> On Sun, Jun 19, 2016 at 04:52:34PM +0200, Baptiste Daroussin wrote:
>>>> On Sun, Jun 19, 2016 at 07:57:49AM +0300, Andrey Chernov wrote:
>>>>> On 19.06.2016 6:44, Hajimu UMEMOTO wrote:
>>>>>> Log:
>>>>>>   - Prefer to use %d over %e where the day of the month should be zero
>>>>>> filled.
>>>>>>   - Since %e means the day of the month as well, regard %e as same as %d
>>>>>> in md_order.
>>>>>
>>>>> Nonsense doubled formats in sr_*_RS locales and nonsense md_order there 
>>>>> too.
>>>>
>>>> Crap I'll dig into it, thanks for spotting it.
>>>>
>>>> Best regards,
>>>> Bapt
>>>
>>> Should be fixed by:
>>> https://people.freebsd.org/~bapt/locale-triple-components.diff
>>>
>>> The perl script generating the locales was badly handling locales with 3
>>> components, fixed now.
>>>
>>> Sorry about that
>>>
>>> Best regards,
>>> Bapt
>>>
>>
>> sr locales are badly named before anything else, proper format is
>> language[_COUNTRY][.encoding][@variant]
>> I.e.
>> sr_RS.UTF-8@Latn
>> sr_RS.ISO8859-2@Latn
>> sr_RS.ISO8859-t@Cyrl
>>
>> I don't know, if out utilities (f.e. locale(1)) can handle @variant suffix.
>>
>>
> =sr_RS.ISO8859-5@Cyrl
> =our, and not utilities only but libc too

Since right now we have only single @variant per each encoding, and no
@variant for other locales, we can just drop it and use right names like:
sr_RS.UTF-8
sr_RS.ISO8859-2
sr_RS.ISO8859-5





signature.asc
Description: OpenPGP digital signature

Re: svn commit: r302016 - in head: share/timedef tools/tools/locale/tools

[Andrey Chernov]

On 19.06.2016 18:49, Baptiste Daroussin wrote:
> On Sun, Jun 19, 2016 at 04:52:34PM +0200, Baptiste Daroussin wrote:
>> On Sun, Jun 19, 2016 at 07:57:49AM +0300, Andrey Chernov wrote:
>>> On 19.06.2016 6:44, Hajimu UMEMOTO wrote:
>>>> Log:
>>>>   - Prefer to use %d over %e where the day of the month should be zero
>>>> filled.
>>>>   - Since %e means the day of the month as well, regard %e as same as %d
>>>> in md_order.
>>>
>>> Nonsense doubled formats in sr_*_RS locales and nonsense md_order there too.
>>
>> Crap I'll dig into it, thanks for spotting it.
>>
>> Best regards,
>> Bapt
> 
> Should be fixed by:
> https://people.freebsd.org/~bapt/locale-triple-components.diff
> 
> The perl script generating the locales was badly handling locales with 3
> components, fixed now.
> 
> Sorry about that
> 
> Best regards,
> Bapt
> 

sr locales are badly named before anything else, proper format is
language[_COUNTRY][.encoding][@variant]
I.e.
sr_RS.UTF-8@Latn
sr_RS.ISO8859-2@Latn
sr_RS.ISO8859-t@Cyrl

I don't know, if out utilities (f.e. locale(1)) can handle @variant suffix.




signature.asc
Description: OpenPGP digital signature

Re: svn commit: r302016 - in head: share/timedef tools/tools/locale/tools

[Andrey Chernov]

On 19.06.2016 19:47, Andrey Chernov wrote:
> On 19.06.2016 18:49, Baptiste Daroussin wrote:
>> On Sun, Jun 19, 2016 at 04:52:34PM +0200, Baptiste Daroussin wrote:
>>> On Sun, Jun 19, 2016 at 07:57:49AM +0300, Andrey Chernov wrote:
>>>> On 19.06.2016 6:44, Hajimu UMEMOTO wrote:
>>>>> Log:
>>>>>   - Prefer to use %d over %e where the day of the month should be zero
>>>>> filled.
>>>>>   - Since %e means the day of the month as well, regard %e as same as %d
>>>>> in md_order.
>>>>
>>>> Nonsense doubled formats in sr_*_RS locales and nonsense md_order there 
>>>> too.
>>>
>>> Crap I'll dig into it, thanks for spotting it.
>>>
>>> Best regards,
>>> Bapt
>>
>> Should be fixed by:
>> https://people.freebsd.org/~bapt/locale-triple-components.diff
>>
>> The perl script generating the locales was badly handling locales with 3
>> components, fixed now.
>>
>> Sorry about that
>>
>> Best regards,
>> Bapt
>>
> 
> sr locales are badly named before anything else, proper format is
> language[_COUNTRY][.encoding][@variant]
> I.e.
> sr_RS.UTF-8@Latn
> sr_RS.ISO8859-2@Latn
> sr_RS.ISO8859-t@Cyrl
> 
> I don't know, if out utilities (f.e. locale(1)) can handle @variant suffix.
> 
> 
=sr_RS.ISO8859-5@Cyrl
=our, and not utilities only but libc too




signature.asc
Description: OpenPGP digital signature

Re: svn commit: r302016 - in head: share/timedef tools/tools/locale/tools

[Andrey Chernov]

On 19.06.2016 8:31, Hajimu UMEMOTO wrote:
> Hi,
> 
>>>>>> On Sun, 19 Jun 2016 07:57:49 +0300
>>>>>> Andrey Chernov <a...@freebsd.org> said:
> 
> ache> Nonsense doubled formats in sr_*_RS locales and nonsense md_order there 
> too.
> 
> It seems there was the problem before my commit.  Only saw a little,
> I'm not sure were it comes.

I understand, just notice that.


___
svn-src-all@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-all
To unsubscribe, send any mail to "svn-src-all-unsubscr...@freebsd.org"

Re: svn commit: r302016 - in head: share/timedef tools/tools/locale/tools

[Andrey Chernov]

On 19.06.2016 6:44, Hajimu UMEMOTO wrote:
> Log:
>   - Prefer to use %d over %e where the day of the month should be zero
> filled.
>   - Since %e means the day of the month as well, regard %e as same as %d
> in md_order.

Nonsense doubled formats in sr_*_RS locales and nonsense md_order there too.

___
svn-src-all@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-all
To unsubscribe, send any mail to "svn-src-all-unsubscr...@freebsd.org"

Re: svn commit: r301777 - head/lib/libc/stdio

[Andrey Chernov]

On 10.06.2016 17:14, Pedro Giffuni wrote:
>> Because now strcoll_l() is used as before and it is broken in wide char
>> converting process. It is broken for _all_, not just for vfscanf()
>> ranges. It should never fail with any args.
>> Real fix should be in strcoll_l().
>>
> 
> Yes, it is broken as before however the objective of r301461 was only
> to address the ABI breakage. If I revert r301777 then I have to backout
> r301461 too, and then we have again the ABI breakage.
> 
> I understand you want to fix all but we are under code freeze and
> I have to go back to a known working (although still broken) state.

I can't make strcoll or vfscanf to drop core in my simple tests.
Too little info in the bug report.
In any case here is vfscanf.c fix attached (by removing collation range).

>> It is dependent. vfscanf() and regcomp() use the same code for range
>> collation. If we remove range collation from everywhere, we save
>> vfscanf(), but strcoll_l() remains broken.
>>
>>
> 
> We have had a broken regex for a very long time, and we new that
> before collation was introduced. I didn't take that decision but
> moving to libtre was a lot more work and we really had to have
> collation now.

No, we have perfectly working single byte regex with collation ranges
for many years until those wchars was invaded the code very recently.

Index: vfscanf.c
===
--- vfscanf.c   (revision 301779)
+++ vfscanf.c   (working copy)
@@ -816,9 +816,7 @@
 static const u_char *
 __sccl(char *tab, const u_char *fmt)
 {
-   int c, n, v, i;
-   struct xlocale_collate *table =
-   (struct 
xlocale_collate*)__get_locale()->components[XLC_COLLATE];
+   int c, n, v;
 
/* first `clear' the whole table */
c = *fmt++; /* first char hat => negated scanset */
@@ -871,29 +869,15 @@
 * we just stored in the table (c).
 */
n = *fmt;
-   if (n == ']'
-   || (table->__collate_load_error ? n < c :
-   __wcollate_range_cmp(table, n, c) < 0
-  )
-  ) {
+   if (n == ']' || n < c) {
c = '-';
break;  /* resume the for(;;) */
}
fmt++;
-   /* fill in the range */
-   if (table->__collate_load_error) {
-   do {
-   tab[++c] = v;
-   } while (c < n);
-   } else {
-   for (i = 0; i < 256; i ++)
-   if (__wcollate_range_cmp(table, c, i) < 
0 &&
-   __wcollate_range_cmp(table, i, n) 
<= 0
-  )
-   tab[i] = v;
-   }
+   do {/* fill in the range */
+   tab[++c] = v;
+   } while (c < n);
 #if 1  /* XXX another disgusting compatibility hack */
-   c = n;
/*
 * Alas, the V7 Unix scanf also treats formats
 * such as [a-c-e] as `the letters a through e'.
___
svn-src-all@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-all
To unsubscribe, send any mail to "svn-src-all-unsubscr...@freebsd.org"

Re: svn commit: r301777 - head/lib/libc/stdio

[Andrey Chernov]

On 10.06.2016 15:03, Pedro Giffuni wrote:
> (Sorry I have been offline/sleep for a while)
> 
> Hello;
> 
> I cannot back this out because of this report:
> 
> http://docs.freebsd.org/cgi/mid.cgi?20160608132224.GA15454
> 
> This change only re-states what has been working before the ABI breakage
> fix in r301461.

Because now strcoll_l() is used as before and it is broken in wide char
converting process. It is broken for _all_, not just for vfscanf()
ranges. It should never fail with any args.
Real fix should be in strcoll_l().

> The fix to regex is completely independent.

It is dependent. vfscanf() and regcomp() use the same code for range
collation. If we remove range collation from everywhere, we save
vfscanf(), but strcoll_l() remains broken.

___
svn-src-all@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-all
To unsubscribe, send any mail to "svn-src-all-unsubscr...@freebsd.org"

Re: svn commit: r301777 - head/lib/libc/stdio

[Andrey Chernov]

Please back it out. __sccl() use plain chars, you can't just call wide
chars __wcollate_range_cmp() from it. Never intermix plain chars and
wide chars without conversion. The problem is somewhere else. The code
used for [a-z] ranges in the format. Moreover, even if conversion added
this code can't work with wchars by definition since uses the same loop
as in regcomp()
for (i = 0; i < 256; i ++)

Counting completely broken regcomp, our locale quickly becomes
intermixing junk of chars and wchars without any conversion. Please
anybody stop hacking here, especially blindly adding wchars everywhere
without analyzing rest of code.
If you see 256 or UCHAR_MAX somewhere, this function can't be converted
to wchars.

On 10.06.2016 8:21, Pedro F. Giffuni wrote:
> Author: pfg
> Date: Fri Jun 10 05:21:52 2016
> New Revision: 301777
> URL: https://svnweb.freebsd.org/changeset/base/301777
> 
> Log:
>   Fix regression from r301461.
>   
>   The fix to the __collate_range_cmp() ABI breakage missed some replacements
>   in libc's vfscanf().  Replace them with __wcollate_range_cmp() which
>   does what is expected.
>   
>   This was breaking applications like xterm and pidgin when using wide
>   characters.
>   
>   Reported by:Vitalij Satanivskij
>   Approved by:re
> 
> Modified:
>   head/lib/libc/stdio/vfscanf.c
> 
> Modified: head/lib/libc/stdio/vfscanf.c
> ==
> --- head/lib/libc/stdio/vfscanf.c Fri Jun 10 04:04:55 2016
> (r301776)
> +++ head/lib/libc/stdio/vfscanf.c Fri Jun 10 05:21:52 2016
> (r301777)
> @@ -873,7 +873,7 @@ doswitch:
>   n = *fmt;
>   if (n == ']'
>   || (table->__collate_load_error ? n < c :
> - __collate_range_cmp (table, n, c) < 0
> + __wcollate_range_cmp(table, n, c) < 0
>  )
>  ) {
>   c = '-';
> @@ -887,8 +887,8 @@ doswitch:
>   } while (c < n);
>   } else {
>   for (i = 0; i < 256; i ++)
> - if (   __collate_range_cmp (table, c, 
> i) < 0
> - && __collate_range_cmp (table, i, 
> n) <= 0
> + if (__wcollate_range_cmp(table, c, i) < 
> 0 &&
> + __wcollate_range_cmp(table, i, n) 
> <= 0
>  )
>   tab[i] = v;
>   }
> 

___
svn-src-all@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-all
To unsubscribe, send any mail to "svn-src-all-unsubscr...@freebsd.org"

Re: svn commit: r301577 - stable/10/etc/periodic/daily

[Andrey Chernov]

The code still is called only if $daily_ntpd_leapfile_enable is YES and
it is NO by default.

On 08.06.2016 6:08, Cy Schubert wrote:
> Author: cy
> Date: Wed Jun  8 03:08:37 2016
> New Revision: 301577
> URL: https://svnweb.freebsd.org/changeset/base/301577
> 
> Log:
>   MFC r301102:
>   
>   Don't rely on $ntpd_enable to periodically fetch the latest leapfile.
>   
>   Suggested by:   cperciva
> 
> Modified:
>   stable/10/etc/periodic/daily/480.leapfile-ntpd
> Directory Properties:
>   stable/10/   (props changed)
> 
> Modified: stable/10/etc/periodic/daily/480.leapfile-ntpd
> ==
> --- stable/10/etc/periodic/daily/480.leapfile-ntpdWed Jun  8 02:39:10 
> 2016(r301576)
> +++ stable/10/etc/periodic/daily/480.leapfile-ntpdWed Jun  8 03:08:37 
> 2016(r301577)
> @@ -16,10 +16,10 @@ case "$daily_ntpd_leapfile_enable" in
>   case "$daily_ntpd_avoid_congestion" in
>   [Yy][Ee][Ss])
>   # Avoid dogpiling
> - (sleep $(jot -r 1 0 86400); service ntpd fetch) &
> + (sleep $(jot -r 1 0 86400); service ntpd onefetch) &
>   ;;
>   *)
> - service ntpd fetch
> + service ntpd onefetch
>   ;;
>   esac
>   ;;
> 

___
svn-src-all@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-all
To unsubscribe, send any mail to "svn-src-all-unsubscr...@freebsd.org"

Re: svn commit: r301226 - in head: etc etc/defaults etc/periodic/security etc/rc.d lib lib/libblacklist libexec libexec/blacklistd-helper share/mk tools/build/mk usr.sbin usr.sbin/blacklistctl usr.sbi

[Andrey Chernov]

On 06.06.2016 20:52, Slawa Olhovchenkov wrote:
>> BTW, it is good idea: to check first, is supported firewall enabled, and
>> only then enable blacklistd by default.
> 
> What purpose? SUDDENLY lockout access to own host after some mistake
> in password?

I agree that default 1 day locking is too restricting. I prefer floating
scale of fail2ban which have several 15min locks (which is not big
lockout for yourself) before one big lock after them, but currently
blacklistd can't do that.

___
svn-src-all@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-all
To unsubscribe, send any mail to "svn-src-all-unsubscr...@freebsd.org"

Re: svn commit: r301226 - in head: etc etc/defaults etc/periodic/security etc/rc.d lib lib/libblacklist libexec libexec/blacklistd-helper share/mk tools/build/mk usr.sbin usr.sbin/blacklistctl usr.sbi

[Andrey Chernov]

On 06.06.2016 20:22, Ian Lepore wrote:
> On Mon, 2016-06-06 at 20:06 +0300, Andrey Chernov wrote:
>> As variant, I keep hope blacklist sh helper will teach about ipfw
>> soon,
>> it looks possible. Then it can be re-enabled by default.
> 
> No, it should still not be enabled by default.  Maybe it should be
> enabled in response to some question in the installer, or maybe even
> better, enabled only if some firewall software that understands it is
> also enabled.  But afaik, all the available firewalls are disabled by
> default in defaults/rc.conf, and this should be too.

BTW, it is good idea: to check first, is supported firewall enabled, and
only then enable blacklistd by default.

___
svn-src-all@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-all
To unsubscribe, send any mail to "svn-src-all-unsubscr...@freebsd.org"

Re: svn commit: r301226 - in head: etc etc/defaults etc/periodic/security etc/rc.d lib lib/libblacklist libexec libexec/blacklistd-helper share/mk tools/build/mk usr.sbin usr.sbin/blacklistctl usr.sbi

[Andrey Chernov]

On 06.06.2016 20:00, Ian Lepore wrote:
> Probably everyone assumed (like I did) that it would be disabled by
> default, and didn't notice that wasn't the case.  Your response
> indicates the problem with "default enabled"... you mention enabling
> packet filtering in pf.conf, my response is:  WTF is pf.conf and why
> are you assuming I do any kind of packet filtering?
> 
> I have literally dozens of systems here running freebsd, only one of
> them runs ipfw, and most of them are systems with small memory and
> wimpy processors, so why would I want extra do-nothing network daemons
> running on them by default?

As variant, I keep hope blacklist sh helper will teach about ipfw soon,
it looks possible. Then it can be re-enabled by default.


___
svn-src-all@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-all
To unsubscribe, send any mail to "svn-src-all-unsubscr...@freebsd.org"

Re: svn commit: r301461 - in head/lib/libc: gen locale regex

[Andrey Chernov]

On 05.06.2016 22:12, Pedro F. Giffuni wrote:
>   When collation support was brought in, the second and third
>   arguments in __collate_range_cmp() were changed from int to
>   wchar_t, breaking the ABI. Change them to a "char" type which
>   makes more sense and keeps the ABI compatible.

Not only that breaks ABI, but changing strcoll_l() to wcscoll_l() in the
__collate_range_cmp() too (now fixed), while this function is visible
outside of libc.

___
svn-src-all@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-all
To unsubscribe, send any mail to "svn-src-all-unsubscr...@freebsd.org"

Re: svn commit: r301461 - in head/lib/libc: gen locale regex

[Andrey Chernov]

On 05.06.2016 22:12, Pedro F. Giffuni wrote:
> --- head/lib/libc/regex/regcomp.c Sun Jun  5 18:16:33 2016
> (r301460)
> +++ head/lib/libc/regex/regcomp.c Sun Jun  5 19:12:52 2016
> (r301461)
> @@ -821,10 +821,10 @@ p_b_term(struct parse *p, cset *cs)
>   (void)REQUIRE((uch)start <= (uch)finish, 
> REG_ERANGE);
>   CHaddrange(p, cs, start, finish);
>   } else {
> - (void)REQUIRE(__collate_range_cmp(table, start, 
> finish) <= 0, REG_ERANGE);
> + (void)REQUIRE(__wcollate_range_cmp(table, 
> start, finish) <= 0, REG_ERANGE);
>   for (i = 0; i <= UCHAR_MAX; i++) {
> - if (   __collate_range_cmp(table, 
> start, i) <= 0
> - && __collate_range_cmp(table, i, 
> finish) <= 0
> + if (   __wcollate_range_cmp(table, 
> start, i) <= 0
> + && __wcollate_range_cmp(table, i, 
> finish) <= 0
>  )
>   CHadd(p, cs, i);
>   }
> 

As I already mention in PR, we have broken regcomp after someone adds
wchar_t support there. Now regcomp ranges works only for the first 256
wchars of the current locale, notice that loop upper limit:
for (i = 0; i <= UCHAR_MAX; i++) {
In general, ranges are either broken in regcomp now or are memory
eating. We have bitmask only for the first 256 wchars, all other added
to the range literally. Imagine what happens if someone specify full
Unicode range in regexp.

Proper fix will be adding bitmask for the whole Unicode range, and even
in that case regcomp attempting to use collation in ranges will be
_very_slow_ since needs to check all Unicode chars in its
for (i = 0; i <= Max_Unicode_wchar; i++) {
loop.

Better stop pretending that we are able to do collation support in the
ranges, since POSIX cares about its own locale only here:
"In the POSIX locale, a range expression represents the set of collating
elements that fall between two elements in the collation sequence,
inclusive. In other locales, a range expression has unspecified
behavior: strictly conforming applications shall not rely on whether the
range expression is valid, or on the set of collating elements matched."

Until whole Unicode range bitmask will be implemented (if ever), better
stop pretending to honor collation order, we just can't do it with
wchars now and do what NetBSD/OpenBSD does (using wchar_t) instead. It
does not prevent memory eating on big ranges (bitmask is needed, see
above), but at least fix the thing that only first 256 wchars are
considered.

___
svn-src-all@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-all
To unsubscribe, send any mail to "svn-src-all-unsubscr...@freebsd.org"

Re: svn commit: r301277 - in head: . lib/clang lib/clang/include/clang/Basic lib/clang/libclangbasic

[Andrey Chernov]

On 03.06.2016 20:44, Bryan Drewery wrote:
> Thanks!
> 
> I think this helps Andrey's recent clang rebuild issues as well since
> the quite-common file was being touched often.

Thanx!

> 
> On 6/3/2016 9:17 AM, Dimitry Andric wrote:
>> Author: dim
>> Date: Fri Jun  3 16:17:36 2016
>> New Revision: 301277
>> URL: https://svnweb.freebsd.org/changeset/base/301277
>>
>> Log:
>>   For clang, move the definition of FREEBSD_CC_VERSION into its own header
>>   file, lib/clang/freebsd_cc_version.h, instead of reusing Version.inc.
>>   The header is only included from one .cpp file in the clang tree.
>>   
>>   This minimizes the number of .cpp files that need to be rebuilt if the
>>   version is bumped.
>>   
>>   Discussed with:bdrewery
>>
>> Added:
>>   head/lib/clang/freebsd_cc_version.h   (contents, props changed)
>> Modified:
>>   head/Makefile.inc1
>>   head/lib/clang/include/clang/Basic/Version.inc
>>   head/lib/clang/libclangbasic/Makefile
>>
>> Modified: head/Makefile.inc1
>> ==
>> --- head/Makefile.inc1   Fri Jun  3 15:33:21 2016(r301276)
>> +++ head/Makefile.inc1   Fri Jun  3 16:17:36 2016(r301277)
>> @@ -99,7 +99,7 @@ _expected_compiler_type=   gcc
>>  .if ${_expected_compiler_type} == "clang"
>>  CROSS_COMPILER_FREEBSD_VERSION!= \
>>  awk '$$2 == "FREEBSD_CC_VERSION" {printf("%d\n", $$3)}' \
>> -${SRCDIR}/lib/clang/include/clang/Basic/Version.inc || echo unknown
>> +${SRCDIR}/lib/clang/freebsd_cc_version.h || echo unknown
>>  CROSS_COMPILER_VERSION!= \
>>  awk '$$2 == "CLANG_VERSION" {split($$3, a, "."); print a[1] * 1 + 
>> a[2] * 100 + a[3]}' \
>>  ${SRCDIR}/lib/clang/include/clang/Basic/Version.inc || echo unknown
>>
>> Added: head/lib/clang/freebsd_cc_version.h
>> ==
>> --- /dev/null00:00:00 1970   (empty, because file is newly added)
>> +++ head/lib/clang/freebsd_cc_version.h  Fri Jun  3 16:17:36 2016
>> (r301277)
>> @@ -0,0 +1,3 @@
>> +/* $FreeBSD$ */
>> +
>> +#define FREEBSD_CC_VERSION  114
>>
>> Modified: head/lib/clang/include/clang/Basic/Version.inc
>> ==
>> --- head/lib/clang/include/clang/Basic/Version.inc   Fri Jun  3 15:33:21 
>> 2016(r301276)
>> +++ head/lib/clang/include/clang/Basic/Version.inc   Fri Jun  3 16:17:36 
>> 2016(r301277)
>> @@ -8,5 +8,3 @@
>>  #define CLANG_VENDOR"FreeBSD "
>>  
>>  #define SVN_REVISION"262564"
>> -
>> -#define FREEBSD_CC_VERSION  114U
>>
>> Modified: head/lib/clang/libclangbasic/Makefile
>> ==
>> --- head/lib/clang/libclangbasic/MakefileFri Jun  3 15:33:21 2016
>> (r301276)
>> +++ head/lib/clang/libclangbasic/MakefileFri Jun  3 16:17:36 2016
>> (r301277)
>> @@ -50,3 +50,6 @@ TGHDRS=AttrHasAttributeImpl \
>>  
>>  # XX: work around GCC bug 67888
>>  CFLAGS.gcc += -fpermissive
>> +
>> +# Ensure FREEBSD_CC_VERSION is defined for Targets.cpp
>> +CFLAGS.Targets.cpp+= -include ../freebsd_cc_version.h
>>
> 
> 




signature.asc
Description: OpenPGP digital signature

Re: svn commit: r300956 - head/lib/libc/stdlib

[Andrey Chernov]

On 31.05.2016 17:17, Bruce Evans wrote:
> Our rand should use just 1, and it is dangerous to change RAND_MAX again,
> but can we even change the sequences?  Something might depend on
> reproducing the old sequences.

This question already arises many times. The sequence must be
reproducible during program runtime only, it is impossible to change
implementation at all otherwise. If someone needs something always
reproducible it needs to implement their own PRNG.

___
svn-src-all@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-all
To unsubscribe, send any mail to "svn-src-all-unsubscr...@freebsd.org"

Re: svn commit: r300956 - head/lib/libc/stdlib

[Andrey Chernov]

This implementation of shifts 32-bit and 64-bit PRNGs looks more
reliable than Mersenne primes and don't use Bignums:
http://wwwmaths.anu.edu.au/~brent/ftp/random/xorgens305.tar.gz
PDF article inside.

On 31.05.2016 14:30, Andrey Chernov wrote:
> On 31.05.2016 12:58, Bruce Evans wrote:
>> On Tue, 31 May 2016, Andrey Chernov wrote:
>>
>>> On 31.05.2016 9:48, Bruce Evans wrote:
>>>>> Perhaps you can find some ideas, answers and PRNG comparison in the
>>>>> original paper:
>>>>> http://www.firstpr.com.au/dsp/rand31/p1192-park.pdf
>>>>
>>>> The ones with Mersenne primes and tweaked Mersenne primes in the
>>>> reference
>>>> (lanl?) given by pfg@ seem OK.
>>>
>>> It should be again correctly implemented to not overflow 64bit (as any
>>> other 64bit generator too).
>>>
>>> Moreover, it have visible patterns in the low order bits. This one from
>>> the same site is better
>>> http://nuclear.llnl.gov/CNP/rng/rngman/node7.html
>>> but still have some pattern too and 61-bit. Next one does not suffer
>>> from the pattern at all
>>> http://nuclear.llnl.gov/CNP/rng/rngman/node8.html
>>> but is 2^64-2^10+1.
>>
>> That's the tweaked Mersenne prime one.
>>
>> Surely there is a pattern in the bits for all of these, and the better
>> ones just hide the pattern better from normal uses and tests?
> 
> Excepting bad first one from pfg@
> http://nuclear.llnl.gov/CNP/rng/rngman/node4.html
> as I understand, they not hide (i.e. drop) the pattern but mix it with
> other part in calculations, it cause whole range reduced.
> 
>>> You can download SPRNG library implementing all of them here:
>>> http://www.sprng.org/RNG/
>>> For me it is overcomplicated.
>>
>> The general case is certainly too complicated.  It would let the user
>> specify all the parameters for the LCG and generate optimal code for
>> the choice on the fly.  Maybe also change the parameters with the seed.
>> But even most implementors don't know how to choose the parameters.
>> That's just for the not so good LCG family of RNGs.
> 
> All mentioned PRNGs with exact parameters (info.h) can be found in the
> subdirs under SRC, f.e. lcg64 or pmlcfg. They are still looks
> complicated and use GMP Bignum library for calculations.

___
svn-src-all@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-all
To unsubscribe, send any mail to "svn-src-all-unsubscr...@freebsd.org"

Re: svn commit: r300956 - head/lib/libc/stdlib

[Andrey Chernov]

On 31.05.2016 12:58, Bruce Evans wrote:
> On Tue, 31 May 2016, Andrey Chernov wrote:
> 
>> On 31.05.2016 9:48, Bruce Evans wrote:
>>>> Perhaps you can find some ideas, answers and PRNG comparison in the
>>>> original paper:
>>>> http://www.firstpr.com.au/dsp/rand31/p1192-park.pdf
>>>
>>> The ones with Mersenne primes and tweaked Mersenne primes in the
>>> reference
>>> (lanl?) given by pfg@ seem OK.
>>
>> It should be again correctly implemented to not overflow 64bit (as any
>> other 64bit generator too).
>>
>> Moreover, it have visible patterns in the low order bits. This one from
>> the same site is better
>> http://nuclear.llnl.gov/CNP/rng/rngman/node7.html
>> but still have some pattern too and 61-bit. Next one does not suffer
>> from the pattern at all
>> http://nuclear.llnl.gov/CNP/rng/rngman/node8.html
>> but is 2^64-2^10+1.
> 
> That's the tweaked Mersenne prime one.
> 
> Surely there is a pattern in the bits for all of these, and the better
> ones just hide the pattern better from normal uses and tests?

Excepting bad first one from pfg@
http://nuclear.llnl.gov/CNP/rng/rngman/node4.html
as I understand, they not hide (i.e. drop) the pattern but mix it with
other part in calculations, it cause whole range reduced.

>> You can download SPRNG library implementing all of them here:
>> http://www.sprng.org/RNG/
>> For me it is overcomplicated.
> 
> The general case is certainly too complicated.  It would let the user
> specify all the parameters for the LCG and generate optimal code for
> the choice on the fly.  Maybe also change the parameters with the seed.
> But even most implementors don't know how to choose the parameters.
> That's just for the not so good LCG family of RNGs.

All mentioned PRNGs with exact parameters (info.h) can be found in the
subdirs under SRC, f.e. lcg64 or pmlcfg. They are still looks
complicated and use GMP Bignum library for calculations.

>> For fixups, see full
>> explanation in the comment of stdlib/div.c
> 
> That was what I was complaining about.  div.c is for C90 (misspelled
> "ANSI").  div.c hasn't caught up with C99.  C99 broke this by changing
> the rules for integer division to disallow correct rounding for and
> require consistently incorrect rounding.  Correct rounding for a
> positive divisor is towards minus infinity so that the remainder is
> not negative.  This is modulo arithmetic and has good algebraic
> properties.  C99 requires rounding minus infinity, at least for positive
> divisors, under the extension "reliable integer division".  In C90,
> the rounding is implementation- defined, so it may be correct, but it
> is "unreliable" since it can be anything.
> 
> In C90, div() is specified as giving "reliable" division, and that is
> what the fixups implement.  This now wastes time to change nothing.
> If the hardware does correct rounding, then the compiler already
> had to do the fixup and id should have produced good MD code for this.
> The C code then adds not so good MI code.

It seems POSIX says nothing about any fixup and negative results:
"These functions shall compute the quotient and remainder of the
division of the numerator numer by the denominator denom. If the
division is inexact, the resulting quotient is the long integer (for the
ldiv( ) function) or long long integer (for the lldiv( ) function) of
lesser magnitude that is the nearest to the algebraic quotient. If the
result cannot be represented, the behavior is undefined; otherwise, quot
* denom+rem shall equal numer."
Does it mean that this fixup should be removed?

___
svn-src-all@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-all
To unsubscribe, send any mail to "svn-src-all-unsubscr...@freebsd.org"

Re: svn commit: r300956 - head/lib/libc/stdlib

[Andrey Chernov]

On 31.05.2016 9:48, Bruce Evans wrote:
>> Perhaps you can find some ideas, answers and PRNG comparison in the
>> original paper:
>> http://www.firstpr.com.au/dsp/rand31/p1192-park.pdf
> 
> The ones with Mersenne primes and tweaked Mersenne primes in the reference
> (lanl?) given by pfg@ seem OK.

It should be again correctly implemented to not overflow 64bit (as any
other 64bit generator too).

Moreover, it have visible patterns in the low order bits. This one from
the same site is better
http://nuclear.llnl.gov/CNP/rng/rngman/node7.html
but still have some pattern too and 61-bit. Next one does not suffer
from the pattern at all
http://nuclear.llnl.gov/CNP/rng/rngman/node8.html
but is 2^64-2^10+1.

You can download SPRNG library implementing all of them here:
http://www.sprng.org/RNG/
For me it is overcomplicated.

>> clang -O uses single "idivl" calculating both quotient and reminder for
>> current code, but for ldiv(3) case call/storage/additional calcs
>> overhead will be added. ldiv(3) does not reduce anything, see
>> stdlib/ldiv.c
> 
> The extern functions give lots of overhead.  Sometimes they get replaced
> automatically by builtins, so it is unclear when the extern functions are
> actually used.  ldiv.c compiles to idivl for the division part but has
> lots of extras for the fixups.  The fixups do nothing except waste time
> on most hardware/cstd combinations.  IIRC, C99 requires direct division
> to have the same poor rounding rules as idiv(), so idiv() is not really
> needed in C99 and the fixups in it are negatively needed.  The builtins
> know what is needed so they don't do the fixups in the usual case that
> the hardware follows the poor rounding rules.

We don't have ldiv() builtin in any cae. For fixups, see full
explanation in the comment of stdlib/div.c

___
svn-src-all@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-all
To unsubscribe, send any mail to "svn-src-all-unsubscr...@freebsd.org"

Re: svn commit: r300965 - head/lib/libc/stdlib

[Andrey Chernov]

On 31.05.2016 8:53, Bruce Evans wrote:
> On Tue, 31 May 2016, Andrey Chernov wrote:
> 
>> On 31.05.2016 6:42, Bruce Evans wrote:
>>>
>>> Er, I already said which types are better -- [u]int_fast32_t here.
>>
>> [u]int_fast32_t have _at_least_ 32 bits. int32_t in the initial PRNG can
>> be changed since does not overflow and involve several calculations, but
>> uint_fast32_t is needed just for two operations:
> 
> I think you mean a native uint32_t is needed for 2 operations.
> 
>> *f += *r;
>> i = (*f >> 1) & 0x7fff;
> 
> This takes 2 operations (add and shift) with native uint32_t.  It takes 4
> logical operations (maybe more physically, or less after optimization)
> with emulated uint32_t (add, mask to 32 bits (maybe move to another
> register to do this), shift, mask to 32 bits).  When you write the final
> mask explicitly, it is to 31 bits and optimizing this away is especially
> easy in both cases.
> 
>> We need to assign values from uint32_t to uint_fast32_t (since array
>> size can't be changed),
> 
> FP code using double_t is similar: data in tables should normally be
> in doubles since double_t might be too much larger; data in function
> parameters is almost always in doubles since APIs are deficient and
> don't even support double_t as an arg; then it is best to assign to
> a double_t variable since if you just use the double then expressions
> using it will promote it to double_t but it is too easy to lose this
> expansion too early.  It takes extra variables and a little more code
> for the assignments, but the extra variables are optimized away in
> cases where there is no expansion.
> 
>> do this single operation fast and store them
>> back into array of uint32_t. I doubt that much gain can comes from it
>> and even pessimization in some cases. Better let compiler do its job
>> here.
> 
> It's never a pessimization if the compiler does its job.
> 
> It is good to practice this on a simple 2-step operation.  Think of a
> multi-step operation where each step requires clipping to 32 bits.
> Using uint32_t for the calculation is just a concise way of writing
> "& 0x" after every step (even ones that don't need it).  It
> is difficult and sometimes impossible for the compiler to optimize
> away these masks across a large number of steps.  Sometimes this is
> easy for the programmer.

The biggest problem so far is that fast types for [u]int32_t are exact
_the_same_ as not fast for i386 and amd64, see /usr/include/x86/_types.h
Without any gain on major platforms I don't think this change is needed.


___
svn-src-all@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-all
To unsubscribe, send any mail to "svn-src-all-unsubscr...@freebsd.org"

Re: svn commit: r300965 - head/lib/libc/stdlib

[Andrey Chernov]

On 31.05.2016 6:42, Bruce Evans wrote:
> 
> Er, I already said which types are better -- [u]int_fast32_t here.

[u]int_fast32_t have _at_least_ 32 bits. int32_t in the initial PRNG can
be changed since does not overflow and involve several calculations, but
uint_fast32_t is needed just for two operations:
*f += *r;
i = (*f >> 1) & 0x7fff;
We need to assign values from uint32_t to uint_fast32_t (since array
size can't be changed), do this single operation fast and store them
back into array of uint32_t. I doubt that much gain can comes from it
and even pessimization in some cases. Better let compiler do its job here.

___
svn-src-all@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-all
To unsubscribe, send any mail to "svn-src-all-unsubscr...@freebsd.org"

Re: svn commit: r300965 - head/lib/libc/stdlib

[Andrey Chernov]

On 30.05.2016 6:09, Bruce Evans wrote:
> On Sun, 29 May 2016, Conrad Meyer wrote:
> 
>> Does clang actually generate different code with this change?
> 
> It should, on exotic arches.
> 
>> On Sun, May 29, 2016 at 9:39 AM, Andrey A. Chernov 
>> wrote:
>>> Log:
>>>   Micro optimize: C standard guarantees that right shift for unsigned
>>> value
>>>   fills left bits with zero, and we have exact 32bit unsigned value
>>>   (uint32_t), so there is no reason to add "& 0x7fff" here.
> 
> Using uint32_t at all is an unportable pessimization.  On exotic arches
> that don't have native uint32_t registers, a theoretically perfect
> implementation wouldn't implement uint32_t.  This would expose the
> brokenness of broken code, so a practical implementation would emulate
> uint32_t so that the broken code would just run slower for arithmetic
> and much slower for locked memory operations.
> 
> uint32_t might be implemented not very slowly using 128-bit integer
> registers, or more slowly using the 53-bit mantissa part of an IEEE
> double precision floating point register.
> 
> If uint32_t is emulated, then the compiler is forced to act as if the
> code uses a longer type and does "& 0x" after every operation.
> Thes extra operations can only be combined sometimes.  More careful
> code can use a minimal number of this or similar "&" operations.  In
> checksum calculations, one "&" at the end is usually enough.
> 
>>> Modified: head/lib/libc/stdlib/random.c
> 
> random.c was mostly written before uint32_t was standard, so it used
> u_long and long.  Perhaps it wasn't careful enough with the "&"s to
> actually work unless u_long is precisely uint32_t and long is normal
> 2's complement with benign overflow.
> 
> Anyway, it was "fixed" (unimproved) using s/u_long/uint32_/ in most
> places where the API/ABI doesn't require longs (there is 1 dubious
> long left in a comment).  The correct fix is s/u_long/uint_fast32_t
> in most places and s/u_long/uint_least32_t/ in some places and then
> fix any missing "&"'s.  The "fast" and "least" types always exist,
> unlike the fixed-width types, and using them asks for time/space
> efficiency instead of emulated fixed-width.
> 
> On non-exotic arches, fast == least == fixed-width, so the correct
> substitution works as a quick fix even with missing "&"s.
> 
> It is not necessary to use the newfangled standard integer types to
> fix this here, since correct use of long types would work (they give
> 32 bits), but long is wasteful if it actually 64 bits or longer.
> 
> Even larger problems are looming with uintmax_t.  Any code that is
> careful enough to use it is likely to break or be bloated if it is
> expanded.  This is just like using u_long in old random().
> 
>>> ==
>>>
>>> --- head/lib/libc/stdlib/random.c   Sun May 29 16:32:56
>>> 2016(r300964)
>>> +++ head/lib/libc/stdlib/random.c   Sun May 29 16:39:28
>>> 2016(r300965)
>>> @@ -430,7 +430,7 @@ random(void)
>>>  */
>>> f = fptr; r = rptr;
>>> *f += *r;
>>> -   i = (*f >> 1) & 0x7fff; /* chucking least
>>> random bit */
>>> +   i = *f >> 1;/* chucking least random bit */
> 
> This gives an "&" to restore in the version with correct substitutions.
> 
> It also breaks the indentation.  (This file mostly indents comments to the
> right of code to column 40, but column 48 was used here and now column 32
> is used.)
> 
>>> if (++f >= end_ptr) {
>>> f = state;
>>> ++r;
> 
> Bruce
> 

I don't introduce uint32_t and int32_t here and don't have a slightest
idea of which types will be better to change them. F.e. *f += *r;
suppose unsigned 32bit overflow which don't naturally happens for large
types. Assigning uint32_t to some large type then clip it to smaller
after calculation - all of that can produce more code than save for
calculation itself.

___
svn-src-all@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-all
To unsubscribe, send any mail to "svn-src-all-unsubscr...@freebsd.org"

Re: svn commit: r300956 - head/lib/libc/stdlib

[Andrey Chernov]

On 30.05.2016 5:17, Bruce Evans wrote:
> On Sun, 29 May 2016, Andrey A. Chernov wrote:
> 
>> Log:
>>  1) Unifdef USE_WEAK_SEEDING since it is too obsolete to support and
>> makes
>>  reading hard.
> 
> Good.
> 
>>  2) Instead of doing range transformation in each and every function
>> here,
>>  do it single time directly in do_rand(). One "mod" operation overhead
>> is not
>>  a big deal, but the code looks nicer and possible future functions
>> additions
>>  or PRNG change do not miss range transformations neither have
>> unneeded ones.
> 
> The whole implementation is silly.  It is manually optimized for 1980's
> compilers.  More below.
> 
>>  3) Use POSIX argument types for visible functions (cosmetic).
> 
> Not sure I like type changes.
> 
>> Modified: head/lib/libc/stdlib/rand.c
>> ==
>>
>> --- head/lib/libc/stdlib/rand.cSun May 29 12:21:54 2016(r300955)
>> +++ head/lib/libc/stdlib/rand.cSun May 29 13:57:06 2016(r300956)
>> @@ -48,14 +48,6 @@ __FBSDID("$FreeBSD$");
>> static int
>> do_rand(unsigned long *ctx)
>> {
>> -#ifdef  USE_WEAK_SEEDING
>> -/*
>> - * Historic implementation compatibility.
>> - * The random sequences do not vary much with the seed,
>> - * even with overflowing.
>> - */
>> -return ((*ctx = *ctx * 1103515245 + 12345) % ((u_long)RAND_MAX +
>> 1));
> 
> This is a good implementation of a not very good LCG, made very bad by
> botching RAND_MAX.  The magic numbers except for RAND_MAX are copied
> from the example in the C90 spec.  I think they are good enough there.
> The comment in at least the C99 spec says "// RAND_MAX assumed to be
> 32767".  This means that these magic numbers were chosen to work with
> this value of RAND_MAX.  (unsigned) longs are used to give a period
> much longer than RAND_MAX and for technical reasons.  Taking the modulo
> to many fewer bits than the minimum of 32 for an unsigned long then
> disguises the linearity.  The BSD version almost completly breaks this
> on arches with 32 bit longs by taking the modulo to 31 bits (mod 32 bits
> would give complete breakage).  Arches with 64-bit longs accidentally
> work a bit better, by the coefficients are poorly chosen -- they should
> be 64 bits and the arithmetic 128 bits.
> 
>> -#else   /* !USE_WEAK_SEEDING */
>> /*
>>  * Compute x = (7^5 * x) mod (2^31 - 1)
>>  * without overflowing 31 bits:
> 
> These coefficients are probably better, but they are still basically
> 32-bit ones and thus not very good for more than a 15-bit RAND_MAX,
> and the details of the calculation are excessively optimized for 1980's
> compilers and 32-bit uintmax_t.
> 
> This can be written as x = (1687 * x) % 2147483647 (with some care about
> type sizes and signedness and overflow.  It then looks like an even worse
> LCG than the botched C90 one, at least with the botch making its internals
> more visible.  E.g., when x = 1, the first couple of iterations don't even
> involve the linear term in 31 bits.
> 
> Even 1980's compiler technology was not far from reducing the division
> to a multiplication.  The LCG expression would then reduce to
> (uintN_t)(A * x + B) where N is either 32 or 64.  Perhaps N needs to
> be 64 even with the small coeefficients, due to the divisor being large
> and not a power of 2.  But if we have 64-bit arithmetic, then we can
> choose much better coefficients than the C90 32-bit ones or the ACM
> barely 16-bit ones, and uses A * x + B directly, giving a 64-bit period,
> and have a chance of our 31-bit RAND_MAX finally working.
> 
>> @@ -66,48 +58,34 @@ do_rand(unsigned long *ctx)
>>  */
>> long hi, lo, x;
>>
>> -/* Must be in [1, 0x7ffe] range at this point. */
>> -hi = *ctx / 127773;
>> -lo = *ctx % 127773;
>> +/* Transform to [1, 0x7ffe] range. */
>> +x = (*ctx % 0x7ffe) + 1;
>> +hi = x / 127773;
>> +lo = x % 127773;
>> x = 16807 * lo - 2836 * hi;
>> if (x < 0)
>> x += 0x7fff;
> 
> This does the division more magically but more slowly than the compiler
> would do.   It uses one division and one remainder, and doesn't use
> the newfangled (late 1980's) ldiv() function to explicitly try to
> reduce these to one hardware divrem operation.  But compilers can
> easily do this reduction.  I think compilers can't easily (or perhaps
> at all) reduce to an A * x + B expression.  It isn't clear if using
> signed long here makes things easier or harder for compilers.  The
> algorithm is special to avoid overflow with signed longs, but it the
> compiler might not understand this.  Then -fwrapv would inhibit it
> from doing much reduction, and -fno-wrapv is just complicated.
> 
>> -*ctx = x;
>> /* Transform to [0, 0x7ffd] range. */
>> -return (x - 1);
>> -#endif  /* !USE_WEAK_SEEDING */
>> +x--;
>> +*ctx = x;
>> +return (x);
>> }
>>
>>
>> int
>> -rand_r(unsigned int *ctx)
>> +rand_r(unsigned *ctx)
> 
> You didn't change the type, but 

Re: svn commit: r300965 - head/lib/libc/stdlib

[Andrey Chernov]

On 29.05.2016 22:39, Conrad Meyer wrote:
> Does clang actually generate different code with this change?

Yes, without -O it generates
andl$2147483647, ...
With -O nothing is changed.
In general, it can be not clang and I see no point to left junk code in
any case.

> 
> On Sun, May 29, 2016 at 9:39 AM, Andrey A. Chernov  wrote:
>> Author: ache
>> Date: Sun May 29 16:39:28 2016
>> New Revision: 300965
>> URL: https://svnweb.freebsd.org/changeset/base/300965
>>
>> Log:
>>   Micro optimize: C standard guarantees that right shift for unsigned value
>>   fills left bits with zero, and we have exact 32bit unsigned value
>>   (uint32_t), so there is no reason to add "& 0x7fff" here.
>>
>>   MFC after:  1 week
>>
>> Modified:
>>   head/lib/libc/stdlib/random.c
>>
>> Modified: head/lib/libc/stdlib/random.c
>> ==
>> --- head/lib/libc/stdlib/random.c   Sun May 29 16:32:56 2016
>> (r300964)
>> +++ head/lib/libc/stdlib/random.c   Sun May 29 16:39:28 2016
>> (r300965)
>> @@ -430,7 +430,7 @@ random(void)
>>  */
>> f = fptr; r = rptr;
>> *f += *r;
>> -   i = (*f >> 1) & 0x7fff; /* chucking least random bit 
>> */
>> +   i = *f >> 1;/* chucking least random bit */
>> if (++f >= end_ptr) {
>> f = state;
>> ++r;
>>
> 

___
svn-src-all@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-all
To unsubscribe, send any mail to "svn-src-all-unsubscr...@freebsd.org"

Re: svn commit: r300377 - head/sys/compat/ndis

[Andrey Chernov]

On 22.05.2016 3:06, Bruce Evans wrote:
> On Sat, 21 May 2016, Pedro Giffuni wrote:
> 
>> On 05/21/16 16:55, Conrad Meyer wrote:
>>> On Sat, May 21, 2016 at 1:40 PM, Pedro Giffuni  wrote:


 On 05/21/16 14:05, Conrad Meyer wrote:
> Won't this still return a negative integer in many cases?
>
> random(9) returns u_long, whereas this rand() routine returns 'int'.
>
> Even on architectures where long is the same size as ordinary
> integers, the range of possible results of the 'random() / 2 + 1'
> expression, before implicit cast to signed, is [1, 2^31] (inclusive).

 According to:
 sys/libkern/random.c

 The result is uniform on [0, 2^31 - 1].
>>>
>>> Ah, I missed that.  Sorry!  In that case, I'm not sure why this is
>>> needed — the result fits in a non-negative 2's complement signed
>>> integer.
>>
>> Actually, I had missed it too. And I also had no idea we were working
>> around the zero singularity.
>>
>> I will revert the change and will do an adjustment for the case where
>> we use 0 as a seed (which in MS should be equivalent to 1).
> 
> The libc version has complications related to this.  The libkern
> version has rotted by not being kept up to date with the libc version.
> 4.4BSD-Lite has sort of the reverse bitrot -- in libc, it only has the
> bad LCG that copied from an example in the 1990 C standard, while
> it has the better LCG copied from 1988 Communications in the ACM in
> libkern.
> FreeBSD still has the ACM version in libkern, and has a fixed copy
> of that in libc, with the bad old version under an ifdef.
> 
> The libc version now adjusts the range from [0, 0x7fff] to
> 0, 0x7ffd] and reduces RAND_MAX by 2 to match.  The claimed uniformity
> for the larger range is very wrong, since the ACM algorithm can only
> produce numbers in the range [1(or is it 0?), 0x7ffe] starting from a
> seed in the range [1, 0x7ffe(or is it 1 higher?)].  There are problems
> at both extremities, and it isn't clear if the new or old adjustments to
> avoid them preserve uniformity.  It is clear that the range was at least
> 1 too high, since the ACM algorithm does a modulo by 0x7fff.

libc version does range adjustment for better uniformity only for
rand(3), not for random(3). There is no RAND_MAX constant in the
random(3) API. POSIX require that random(3) should stay in [0, 2^31-1].

___
svn-src-all@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-all
To unsubscribe, send any mail to "svn-src-all-unsubscr...@freebsd.org"

Re: svn commit: r300384 - head/sys/compat/ndis

[Andrey Chernov]

On 22.05.2016 3:29, Pedro F. Giffuni wrote:
>  static void
>  srand(unsigned int seed)
>  {
>  
> + if (seed == 0)
> + seed = 1;
>   srandom(seed);
>  }

IMHO it is not needed. srandom() already handles seed == 0 case, there
is no point to remap it to another constant.
___
svn-src-all@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-all
To unsubscribe, send any mail to "svn-src-all-unsubscr...@freebsd.org"

Re: svn commit: r298367 - head/lib/libc/locale

[Andrey Chernov]

On 21.04.2016 9:29, Baptiste Daroussin wrote:
 Modified: head/lib/libc/locale/ascii.c
 ==
 --- head/lib/libc/locale/ascii.c   Wed Apr 20 20:43:05 2016
 (r298366)
 +++ head/lib/libc/locale/ascii.c   Wed Apr 20 20:44:30 2016
 (r298367)
 @@ -133,11 +133,14 @@ _ascii_mbsnrtowcs(wchar_t * __restrict d
  
if (dst == NULL) {
s = memchr(*src, '\0', nms);
 +  if (s == NULL)
 +  return (nms);
 +
if (*s & 0x80) {
errno = EILSEQ;
return ((size_t)-1);
}
 -  return (s != NULL ? s - *src : nms);
 +  return (s - *src);
}
  
s = *src;

>>>
>>> The whole code is incorrect, only the very first char is checked, there
>>> must be a loop like in -stable:
>>>
>>> if (dst == NULL) {
>>> for (s = *src; nms > 0 && *s != '\0'; s++, nms--) {
>>> if (*s & 0x80) {
>>> errno = EILSEQ;
>>> return ((size_t)-1);
>>> }
>>> }
>>> return (s - *src);
>>> }
>>>
>>> Since svn history is lost on deleting, I don't know why incorrect
>>> version was committed.
>>>
>>
>> Typo, the very first == the very last, i.e. only NUL char is checked
>> which always pass.
>>
> 
> I have restored the history (I hope correctly)
> 
> Bapt
> 

All the restored history is related to none.c, but ascii.c was made
afterwards a bit differently and transition history between none.c ->
ascii.c is lost in any case somehow.
We still have correct version in -stable.
The diff is attached.
BTW, recent none.c from which ascii.c is made don't have this two
copyrights too:
- * Copyright 2013 Garrett D'Amore 
- * Copyright 2010 Nexenta Systems, Inc.  All rights reserved.
Index: ascii.c
===
--- ascii.c (.../head/lib/libc/locale/ascii.c)  (revision 298395)
+++ ascii.c (.../stable/10/lib/libc/locale/ascii.c) (working copy)
@@ -1,6 +1,4 @@
-/*
- * Copyright 2013 Garrett D'Amore 
- * Copyright 2010 Nexenta Systems, Inc.  All rights reserved.
+/*-
  * Copyright (c) 2002-2004 Tim J. Robbins. All rights reserved.
  * Copyright (c) 1993
  * The Regents of the University of California.  All rights reserved.
@@ -36,8 +34,6 @@
  * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY
  * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
  * SUCH DAMAGE.
- *
- * @(#)none.c  8.1 (Berkeley) 6/4/93
  */
 
 #include 
@@ -65,7 +61,7 @@
size_t, size_t, mbstate_t * __restrict);
 
 int
-_ascii_init(struct xlocale_ctype *l, _RuneLocale *rl)
+_ascii_init(struct xlocale_ctype *l,_RuneLocale *rl)
 {
 
l->__mbrtowc = _ascii_mbrtowc;
@@ -82,6 +78,7 @@
 static int
 _ascii_mbsinit(const mbstate_t *ps __unused)
 {
+
/*
 * Encoding is not state dependent - we are always in the
 * initial state.
@@ -93,6 +90,7 @@
 _ascii_mbrtowc(wchar_t * __restrict pwc, const char * __restrict s, size_t n,
 mbstate_t * __restrict ps __unused)
 {
+
if (s == NULL)
/* Reset to initial shift state (no-op) */
return (0);
@@ -132,13 +130,11 @@
size_t nchr;
 
if (dst == NULL) {
-   s = memchr(*src, '\0', nms);
-   if (s == NULL)
-   return (nms);
-
-   if (*s & 0x80) {
-   errno = EILSEQ;
-   return ((size_t)-1);
+   for (s = *src; nms > 0 && *s != '\0'; s++, nms--) {
+   if (*s & 0x80) {
+   errno = EILSEQ;
+   return ((size_t)-1);
+   }
}
return (s - *src);
}
@@ -193,3 +189,4 @@
*src = s;
return (nchr);
 }
+


signature.asc
Description: OpenPGP digital signature

Re: svn commit: r298367 - head/lib/libc/locale

[Andrey Chernov]

On 21.04.2016 3:57, Andrey Chernov wrote:
> On 20.04.2016 23:44, Baptiste Daroussin wrote:
>> Author: bapt
>> Date: Wed Apr 20 20:44:30 2016
>> New Revision: 298367
>> URL: https://svnweb.freebsd.org/changeset/base/298367
>>
>> Log:
>>   Check the returned value of memchr(3) before using it
>>   
>>   Reported by:   Coverity
>>   CID:   1338530
>>
>> Modified:
>>   head/lib/libc/locale/ascii.c
>>
>> Modified: head/lib/libc/locale/ascii.c
>> ==
>> --- head/lib/libc/locale/ascii.c Wed Apr 20 20:43:05 2016
>> (r298366)
>> +++ head/lib/libc/locale/ascii.c Wed Apr 20 20:44:30 2016
>> (r298367)
>> @@ -133,11 +133,14 @@ _ascii_mbsnrtowcs(wchar_t * __restrict d
>>  
>>  if (dst == NULL) {
>>  s = memchr(*src, '\0', nms);
>> +if (s == NULL)
>> +return (nms);
>> +
>>  if (*s & 0x80) {
>>  errno = EILSEQ;
>>  return ((size_t)-1);
>>  }
>> -return (s != NULL ? s - *src : nms);
>> +return (s - *src);
>>  }
>>  
>>  s = *src;
>>
> 
> The whole code is incorrect, only the very first char is checked, there
> must be a loop like in -stable:
> 
>   if (dst == NULL) {
> for (s = *src; nms > 0 && *s != '\0'; s++, nms--) {
> if (*s & 0x80) {
> errno = EILSEQ;
> return ((size_t)-1);
> }
> }
> return (s - *src);
>   }
> 
> Since svn history is lost on deleting, I don't know why incorrect
> version was committed.
> 

Typo, the very first == the very last, i.e. only NUL char is checked
which always pass.

___
svn-src-all@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-all
To unsubscribe, send any mail to "svn-src-all-unsubscr...@freebsd.org"

Re: svn commit: r298367 - head/lib/libc/locale

[Andrey Chernov]

On 20.04.2016 23:44, Baptiste Daroussin wrote:
> Author: bapt
> Date: Wed Apr 20 20:44:30 2016
> New Revision: 298367
> URL: https://svnweb.freebsd.org/changeset/base/298367
> 
> Log:
>   Check the returned value of memchr(3) before using it
>   
>   Reported by:Coverity
>   CID:1338530
> 
> Modified:
>   head/lib/libc/locale/ascii.c
> 
> Modified: head/lib/libc/locale/ascii.c
> ==
> --- head/lib/libc/locale/ascii.c  Wed Apr 20 20:43:05 2016
> (r298366)
> +++ head/lib/libc/locale/ascii.c  Wed Apr 20 20:44:30 2016
> (r298367)
> @@ -133,11 +133,14 @@ _ascii_mbsnrtowcs(wchar_t * __restrict d
>  
>   if (dst == NULL) {
>   s = memchr(*src, '\0', nms);
> + if (s == NULL)
> + return (nms);
> +
>   if (*s & 0x80) {
>   errno = EILSEQ;
>   return ((size_t)-1);
>   }
> - return (s != NULL ? s - *src : nms);
> + return (s - *src);
>   }
>  
>   s = *src;
> 

The whole code is incorrect, only the very first char is checked, there
must be a loop like in -stable:

if (dst == NULL) {
for (s = *src; nms > 0 && *s != '\0'; s++, nms--) {
if (*s & 0x80) {
errno = EILSEQ;
return ((size_t)-1);
}
}
return (s - *src);
}

Since svn history is lost on deleting, I don't know why incorrect
version was committed.

___
svn-src-all@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-all
To unsubscribe, send any mail to "svn-src-all-unsubscr...@freebsd.org"

Re: svn commit: r297407 - head/lib/libc/stdio

[Andrey Chernov]

On 30.03.2016 4:32, Pedro F. Giffuni wrote:
> --- head/lib/libc/stdio/freopen.c Wed Mar 30 01:08:08 2016
> (r297406)
> +++ head/lib/libc/stdio/freopen.c Wed Mar 30 01:32:08 2016
> (r297407)
> @@ -66,7 +66,8 @@ freopen(const char * __restrict file, co
>   (void) fclose(fp);
>   errno = sverrno;
>   return (NULL);
> - }
> + } else
> + sverrno = 0;

This one is wrong too, see my detailed reply on the next commit.

-- 
http://ache.vniz.net/
___
svn-src-all@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-all
To unsubscribe, send any mail to "svn-src-all-unsubscr...@freebsd.org"

Re: svn commit: r297408 - head/lib/libc/stdio

[Andrey Chernov]

On 30.03.2016 9:13, Pedro F. Giffuni wrote:
>*/
> + sverrno = 0;
>   if (file == NULL) {

It introduces the bug of touching errno. Previous code does it right
because 'goto finish;' can't happen with (f < 0) where errno is restored
later.

The whole errno code is complex here for reading at least.
It microoptimize errno handling by removing unnecessary assignment, but
it does not gain much in reality.

Better change for easy reading will be to set
sverrno = errno;
once right at the function enter and remove it in all other places.

In any case errno touching should be backed out.

-- 
http://ache.vniz.net/
___
svn-src-all@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-all
To unsubscribe, send any mail to "svn-src-all-unsubscr...@freebsd.org"

Re: svn commit: r294981 - in stable/10/sys/boot: common efi efi/boot1 efi/include efi/libefi efi/loader efi/loader/arch/amd64 fdt i386/libi386

[Andrey Chernov]

On 30.01.2016 2:36, Andrey Chernov wrote:
> I am not sure this is exact commit which cause this, but any operation
> in /sys/boot/efi ends like this (by hand, not from buildworld):
> (i386-stable machine, clang compiler)
> 
> ===> efi (obj)
> make[1]: "/usr/src/sys/boot/efi/Makefile" line 4: Malformed conditional
> (${COMPILER_TYPE} != "gcc")
> make[1]: Fatal errors encountered -- cannot continue
> make[1]: stopped in /usr/src/sys/boot/efi
> *** Error code 1
> 
> Stop.
> make: stopped in /usr/src/sys/boot
> 

As Makefile shows, there is no include to pick COMPILER_TYPE and it is
undefined. You need at least to move bsd.subdir.mk up.

# $FreeBSD: stable/10/sys/boot/efi/Makefile 294981 2016-01-28 12:11:42Z
smh $

# In-tree GCC does not support __attribute__((ms_abi)).
.if ${COMPILER_TYPE} != "gcc"

.if ${MACHINE_CPUARCH} == "amd64"
SUBDIR+=libefi loader boot1
.endif

.endif # ${COMPILER_TYPE} != "gcc"

.include 



-- 
http://ache.vniz.net/
___
svn-src-all@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-all
To unsubscribe, send any mail to "svn-src-all-unsubscr...@freebsd.org"

Re: svn commit: r294981 - in stable/10/sys/boot: common efi efi/boot1 efi/include efi/libefi efi/loader efi/loader/arch/amd64 fdt i386/libi386

[Andrey Chernov]

I am not sure this is exact commit which cause this, but any operation
in /sys/boot/efi ends like this (by hand, not from buildworld):
(i386-stable machine, clang compiler)

===> efi (obj)
make[1]: "/usr/src/sys/boot/efi/Makefile" line 4: Malformed conditional
(${COMPILER_TYPE} != "gcc")
make[1]: Fatal errors encountered -- cannot continue
make[1]: stopped in /usr/src/sys/boot/efi
*** Error code 1

Stop.
make: stopped in /usr/src/sys/boot

-- 
http://ache.vniz.net/
___
svn-src-all@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-all
To unsubscribe, send any mail to "svn-src-all-unsubscr...@freebsd.org"

Re: svn commit: r292823 - in stable/10/sys: conf netinet

[Andrey Chernov]

On 28.12.2015 6:15, Patrick Kelsey wrote:
> This is explained in the top comment in sys/netinet/tcp_fastopen.c, but
> I will repeat it here and elaborate a little.  It is disabled by default
> in the kernel build as a conservative measure until it is exercised more
> widely, given the modifications it introduces to the TCP state machine
> code, syncache code, etc.  When you do enable it in the kernel build
> (and after some point in the future when it is enabled by default),
> there is still a sysctl that governs its availability in the system that
> must be enabled before it can be used.
>  
> -Patrick

Thanx, if I understand it correctly, is not ready for real use yet but
just for experiments. See my other comment about TCP_RFC7413_MAX_KEYS

-- 
http://ache.vniz.net/
___
svn-src-all@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-all
To unsubscribe, send any mail to "svn-src-all-unsubscr...@freebsd.org"