Re: Send international text with mail(1) - proposal and patches
Crystal Kolipe wrote in : |On Thu, Oct 12, 2023 at 12:36:48AM +0200, Steffen Nurpmeso wrote: |> Non-7-bit clean headers need RFC 2047 (and/or RFC 2231) encoding. | |The use of MIME encoded words to encode header content is no longer |considered best practice. See, for example RFC 6532. Yes there is SMTPUTF8, which is a special protocol. The /global MIME thing i personally have _never_ seen in practice. I have downloaded the RFC on 2012-07-23. |But as Omar said, let's get the basics of any new functionality |sorted out before jumping ahead. We don't really want to break |mail in some unexpected and non-obvious way. I cannot comment on that. I am pretty sure i have never seen /global yet. Ie an archive search here reveals only three mails where i mention them in the text; the last is from a thread from nmh-work...@nongnu.org from July this year, and let me shamelessly quote Ken Hornstein who said on 2023-07-23 The message/global MIME type (a RFC822 message but with UTF-8 everywhere) has a suggested file extension of ".u8msg", which I have never personally seen "in the wild" anywhere. ¯\_(ツ)_/¯ --steffen | |Der Kragenbaer,The moon bear, |der holt sich munter he cheerfully and one by one |einen nach dem anderen runter wa.ks himself off |(By Robert Gernhardt)
Re: Send international text with mail(1) - proposal and patches
Hello Omar. Omar Polo wrote in <2HJQ4VX5L4J1P.3G4A39B0RA6T7@venera>: ... |>MUAs always set appropriate MIME headers. RFC 2046 section 4.1.2 |>paragraph 8 also "strongly" recommends the explicit inclusion of a |>"charset" parameter even for us-ascii. So that really went me looking again, and i read The default character set, which must be assumed in the absence of a charset parameter, is US-ASCII. I have read the following though. Still, you know... ... |>Consequently, i think using 8bit is indeed better for our mail(1) |>than quoted-printable or base64. I have nothing to say beside that, but want to point out that to the best of my knowledge 8bit content-type only refers to MIME part contents, it does _not_ refer to any email headers. Non-7-bit clean headers need RFC 2047 (and/or RFC 2231) encoding. So letting aside any email addresses which possibly would require IDNA encoding, | if (hp->h_subject != NULL && w & GSUBJECT) |- fprintf(fo, "Subject: %s\n", hp->h_subject), gotcha++; |+ fprintf(fo, "Subject: %s\n", hp->h_subject); that is not (again, to the best of my knowledge, i had to read again all those standards, .. after many years) covered by ... |+ if (multibyte) |+ fprintf(fo, "Content-Transfer-Encoding: 8bit\n" |+ "Content-Type: text/plain; charset=utf-8\n"); That is to say: just in case someone thinks this. Ciao Omar! --steffen | |Der Kragenbaer,The moon bear, |der holt sich munter he cheerfully and one by one |einen nach dem anderen runter wa.ks himself off |(By Robert Gernhardt)
Re: Send international text with mail(1) - proposal and patches
Steffen Nurpmeso wrote in <20230920214009.w5mrf%stef...@sdaoden.eu>: |Ingo Schwarze wrote in | : | ... ||I just checked - even though i'm using the higer-level mutt(1) MUA ||most of the time and even though the shell i'm starting mutt(1) from ||has LC_CTYPE=C.UTF-8 set on that particular machine, the last sixteen ||mails i sent all contained the explicit header || || Content-Type: text/plain; charset=us-ascii || ||and intentionally so. Yes, i do occasionally send UTF-8 mail on | |To be a hundred percent correct: MIME is not needed at all in that That is to say, to be correct myself: like RFC 2045 says, "MIME defines a number of new RFC 822 header fields that are used to describe the content of a MIME entity". Yet if there is no MIME entity but only a plain RFC 822/2822/5322 internet message format, there is nothing to describe. [.]there are still circumstances in which it might be desirable for a mail-processing agent to know whether a message was composed with the new standard in mind. Therefore, this document defines a new header field, "MIME-Version", which is to be used to declare the version of the Internet message body format standard in use. Messages composed in accordance with this document MUST include such a header field, with the following verbatim text: But normally OpenBSD Mail does not, so no "MIME-Version: 1.0", because no The presence of this header field is an assertion that the message has been composed in compliance with this document. |case, unless a transfer-encoding had to be used (you do not show |that header), maybe because of overlong lines to-be-folded, or for |whatever reason. (But it is swallowed by consumers of course.) That would at least be my point of view. --steffen | |Der Kragenbaer,The moon bear, |der holt sich munter he cheerfully and one by one |einen nach dem anderen runter wa.ks himself off |(By Robert Gernhardt)
Re: Send international text with mail(1) - proposal and patches
Ingo Schwarze wrote in : ... |I just checked - even though i'm using the higer-level mutt(1) MUA |most of the time and even though the shell i'm starting mutt(1) from |has LC_CTYPE=C.UTF-8 set on that particular machine, the last sixteen |mails i sent all contained the explicit header | | Content-Type: text/plain; charset=us-ascii | |and intentionally so. Yes, i do occasionally send UTF-8 mail on To be a hundred percent correct: MIME is not needed at all in that case, unless a transfer-encoding had to be used (you do not show that header), maybe because of overlong lines to-be-folded, or for whatever reason. (But it is swallowed by consumers of course.) --steffen | |Der Kragenbaer,The moon bear, |der holt sich munter he cheerfully and one by one |einen nach dem anderen runter wa.ks himself off |(By Robert Gernhardt)
Re: [feature] ssh-agent: new -A option (like -a) that overwrites existing sockets
Andreas Kähäri wrote in : |On Wed, Sep 13, 2023 at 03:08:40PM +0200, Moritz Fain wrote: |> Most of the code is already there; it's basically just adding a new flag. |> |> Happy to hear your feedback! | |My initial reaction is that it's easy to run "rm -f" before starting |the agent with the existing "-a" option. a bit off-topic but i had to go a long way with a thing of mine +/* Unfortunately pre v0.8 versions had an undocumented problem: in case the + * server socket was already existing upon startup (server did have not chance + * to perform cleanup), no server would ever have been started, and missing + * policy server would cause postfix to refuse acting. A "rm -f PG-SOCKET" in + * a pre-postfix-startup-script avoids this, but it was never announced to be + * necessary. v0.8 added a "reassurance" lock file to automatize this */ And heck! what a complicated thing with file-locking (aka "ADDRINUSE with taken write lock means former server was not properly shutdown") and all that. It is -- in my opinion -- a deficit of UNIX-domain sockets that they continue to exist if the server is gone, as you cannot bind() to it again, no?, it is nothing bad a dead and useless corpse without soul. --steffen | |Der Kragenbaer,The moon bear, |der holt sich munter he cheerfully and one by one |einen nach dem anderen runter wa.ks himself off |(By Robert Gernhardt)
Re: [OpenSMTPD] Setting personal mailserver
Sagar Acharya wrote in : |Thanks Peter, your comments were very helpful and I made some progress | |I have currently hosted server at 587. I have also set | |_submission._tcp.humaaraartha.in. SRV | |records which point to 587. However, I think such a thing is not impleme\ |nted by default to be detected by mailservers, perhaps, SMTPD. | |Is such a check on other ports in case 25 connection is not established \ |implemented? Submission is for mail-user-agents, whereas SMTP is for mail-transfer-agents. Having said that, for yourself you can do whatever you want, but you cannot expect that the SMTP servers of the world play nicely with it. That is: if you relay your own mail and that of all computers of your own to one SMTP server, which then sends those emails out to the world, then you can configure all your computers to relay to a specific port, and drive the SMTP server on that. But the world will respond to your :25. --steffen | |Der Kragenbaer,The moon bear, |der holt sich munter he cheerfully and one by one |einen nach dem anderen runter wa.ks himself off |(By Robert Gernhardt)
Re: autopledge
Theo de Raadt wrote in
<93466.1685743...@cvs.openbsd.org>:
|We will wait for the demo.
...
|Leah Rowe wrote:
...
|> Yeah I was kinda thinking, just have it be a tool to *assist* but not
|> to automatically pledge the program itself. It wouldn't replace
|> human-performed auditing or analysis.
|>
|> You'd run it just to get a basic gist of where you're going, for
|> different code paths (which are affected by how you use the program).
After having such problems with the linux way of doing things
i thought i write a strace(1) rule to have it easier "later on".
(Ie, "peek the program" a bit.)
I then asked for a generic "when" addition to the strace(1)
tool[1], because of the excessive things the program does in the
setup phase, before it enters the sandbox. (Without response ;)
[1] https://lists.strace.io/pipermail/strace-devel/2023-April/011305.html
But strace is a bit of a help (should be a shell script, and skip
over most of that maybe):
# test-strace {{{
test-strace: all
if [ "$(VAL_OS_SANDBOX)" -ne 0 ]; then echo >&2 this will not do; exit
1; fi;\
trap "rm -rf .z .b.rc .r.rc .c.xout .c.out .s.strace .c.strace" EXIT;
trap "exit 1" INT HUP QUIT TERM;\
mkdir .z || exit 2;\
{ \
echo action=DEFER_IF_PERMIT 4.2.0;echo;\
echo action=DUNNO;echo;\
echo action=REJECT;echo;\
echo action=DUNNO;echo;\
echo action=DUNNO;echo;\
echo action=REJECT;echo;\
} > .c.xout || exit 3;\
echo test.localdomain > .b.rc || exit 4;\
echo test2.localdomain > .z/a.rc || exit 5;\
pwd=$$(pwd);\
{ \
echo msg-defer DEFER_IF_PERMIT 4.2.0;\
echo store-path $$pwd/.z; echo block-file $$pwd/.b.rc; echo
allow-file $$pwd/.z/a.rc;\
echo verbose; echo verbose; echo count 1; echo delay-min 0;\
} > .r.rc || exit 6;\
\
strace -f -c -U name -o .s.strace ./"$(VAL_NAME)" -R $$pwd/.r.rc
--startup & [ $$? -eq 0 ] || exit 10;\
^ 1. Server (keeps on going).
sleep 2;\
{ \
echo recipient=x1@y; echo sender=y@z; echo client_address=127.1.2.2;
echo client_name=xy; echo;\
echo recipient=x1@y; echo sender=y@z; echo client_address=127.1.2.2;
echo client_name=test2.localdomain; echo;\
echo recipient=x1@y; echo sender=y@z; echo client_address=127.1.2.2;
echo client_name=test.localdomain; echo;\
echo recipient=x1@y; echo sender=y@z; echo client_address=127.1.2.2;
echo client_name=xy; echo;\
} | strace -c -U name -o .c.strace ./"$(VAL_NAME)" -R $$pwd/.r.rc >>
.c.out || exit 11;\
^ 2. Client.
sleep 2;\
\
./"$(VAL_NAME)" -R $$pwd/.r.rc --status || exit 12;\
./"$(VAL_NAME)" -R $$pwd/.r.rc --shutdown || exit 13;\
./"$(VAL_NAME)" -R $$pwd/.r.rc --status && exit 14;\
\
echo once >> .r.rc || exit 20;\
strace -A -f -c -U name -o .s.strace ./"$(VAL_NAME)" -R $$pwd/.r.rc
--startup & [ $$? -eq 0 ] || exit 21;\
^ More and different client code paths.
{ \
echo recipient=x1@y; echo sender=y@z; echo client_address=127.1.2.2;
echo client_name=xy; echo;\
echo this should not create result;echo;\
} | strace -A -c -U name -o .c.strace ./"$(VAL_NAME)" -R $$pwd/.r.rc >>
.c.out || exit 22;\
^ More client code paths.
sleep 2;\
./"$(VAL_NAME)" -R $$pwd/.r.rc --status || exit 23;\
\
echo 'block xy' >> .r.rc || exit 24;\
kill -HUP $$(cat $$pwd/.z/"$(VAL_NAME)".pid) || exit 25;\
sleep 2;\
kill -USR1 $$(cat $$pwd/.z/"$(VAL_NAME)".pid) || exit 26;\
sleep 2;\
kill -USR2 $$(cat $$pwd/.z/"$(VAL_NAME)".pid) || exit 27;\
sleep 2;\
{ \
echo recipient=x1@y; echo sender=y@z; echo client_address=127.1.2.2;
echo client_name=xy; echo;\
} | strace -A -c -U name -o .c.strace ./"$(VAL_NAME)" -R $$pwd/.r.rc >>
.c.out || exit 28;\
^ Even more client paths.
\
./"$(VAL_NAME)" -R $$pwd/.r.rc --status || exit 29;\
./"$(VAL_NAME)" -R $$pwd/.r.rc --shutdown || exit 30;\
\
diff -u .c.xout .c.out; echo diff said $$?;\
\
And now the fun part: generate system call listing from what we
have, distinct for client and server:
< .c.strace awk '\
BEGIN{c=hot=0}\
/^-+$$/{hot=!hot;next}\
{if(!hot) next; for(i=1; i <= c; ++i) if(a[i] == $$1) next;
a[++c] = $$1}\
END{for(i=1;i<=c;++i) print "a_Y(SYS_" a[i] "),"}\
' > .c.txt;\
echo 'VAL_OS_SANDBOX_CLIENT_RULES="'$$(cat .c.txt)'"';\
\
< .s.strace awk '\
BEGIN{c=hot=0}\
/^-+$$/{hot=!hot;next}\
{if(!hot) next; for(i=1; i <= c; ++i) if(a[i] == $$1) next;
a[++c] = $$1}\
END{for(i=1;i<=c;++i) print "a_Y(SYS_" a[i] "),"}\
' > .
Re: usr.bin/mail: cmd3.c:bangexp(): "borked"?, and BSD fails POSIX compat
Steffen Nurpmeso wrote in
<20230502214014._ziz6%stef...@sdaoden.eu>:
|Hallo, and sorry for the cross-post, but so all in one (maybe) go.
|
|This is about a niche "feature" of mail, the shell command "bang"
|(! / ~! in compose mode):
...
|POSIX now says / will say
|
| If the bang variable is set, each unescaped occurrence of '!' in
| command shall be replaced with the command executed by the
| previous ! command or ˜! command escape.
|
|thus
|
| - All commands entered for ! and ~! shall be stored.
|
| - If "bang" is set, an unquoted ! shall be replaced by that
|storage.
...
|It will fail to properly "bang" a second time[.]
Eh, forget this please. Of course not, one simply inserts the old
string without expansion.
Since vim(1) offers the same "bang"ing, and merges \! to ! (source
comment speaks of vi-compat) i keep it. (It actually requires
'\!' or \\! ro avoid its expansion in ":echo BLA".)
Ciao.
static struct str last_bang;
struct n_string xbang, *bang;
char c;
char const *cp_orig;
boole bangit, changed;
NYD_IN;
bangit = ok_blook(bang);
changed = FAL0;
cp_orig = cp;
for(bang = n_string_creat(&xbang); (c = *cp++) != '\0';){
if(bangit && c == '!'){
changed = TRU1;
if(last_bang.l > 0)
bang = n_string_push_buf(bang, last_bang.s,
last_bang.l);
}else{
if(c == '\\' && *cp == '!'){
changed = TRU1;
++cp;
c = '!';
}
bang = n_string_push_c(bang, c);
}
}
if(last_bang.s != NIL)
su_FREE(last_bang.s);
last_bang.s = n_string_cp(bang);
last_bang.l = bang->s_len;
bang = n_string_drop_ownership(bang);
n_string_gut(bang);
if(bangit){
cp = last_bang.s;
if(changed && (n_psonce & n_PSO_INTERACTIVE))
fprintf(n_stdout, "!%s\n", cp);
}else
cp = cp_orig;
NYD_OU;
return cp;
--steffen
|
|Der Kragenbaer,The moon bear,
|der holt sich munter he cheerfully and one by one
|einen nach dem anderen runter wa.ks himself off
|(By Robert Gernhardt)
|~~
|..and in spring, hear David Leonard sing..
|
|The black bear, The black bear,
|blithely holds his own holds himself at leisure
|beating it, up and down tossing over his ups and downs with pleasure
|~~
|Farewell, dear collar bear
usr.bin/mail: cmd3.c:bangexp(): "borked"?, and BSD fails POSIX compat
Hallo, and sorry for the cross-post, but so all in one (maybe) go.
This is about a niche "feature" of mail, the shell command "bang"
(! / ~! in compose mode):
? !echo no!bang
no!bang
!
? set bang
? ! echo no!bang
!echo nobang
nobang
!
? ! !
!echo nobang
nobang
!
? ! echo no!bang
!echo noecho nobangbang
noecho nobangbang
!
?
In short: they all do it differently, and they all do it "wrong".
- Apple mail simply does not act unless "bang" is set: bangexp()
simply returns.
- Free-, Net- and OpenBSD do not look for a "bang" variable at
all, they simply do bang-style expansion whether that is set or
not.
- V10 mailx (and the code i maintain) do some mix (expand the
"last bang" only if "bang" is set, but know about \! escapes and
such.
POSIX now says / will say
If the bang variable is set, each unescaped occurrence of '!' in
command shall be replaced with the command executed by the
previous ! command or ˜! command escape.
thus
- All commands entered for ! and ~! shall be stored.
- If "bang" is set, an unquoted ! shall be replaced by that
storage.
The code everywhere is more or less what Kurt A. Shoens wrote as
commit ae3dba0da2068b0de91ef163ea95fe774196a501
Author: Kurt A. Schoens
AuthorDate: 1980-10-09 02:48:47 -0800
Commit: Kurt A. Schoens
CommitDate: 1980-10-09 02:48:47 -0800
Made shell escapes expand !'s to previous command
SCCS-vsn: usr.bin/mail/cmd3.c 1.2
It will fail to properly "bang" a second time if some escaped
question mark was present in the first round,
while(*cp){
...
if (*cp == '!') {
BANGEXP
...
continue;
}
...
if (*cp == '\\' && cp[1] == '!') {
...
*cp2++ = '!';
cp += 2;
changed++;
[fallthru]
}
...
*cp2++ = *cp++;
and it stores and pushes through the second escape for "\!\!".
I am thinking of dropping this entirely, as it can properly work
only once if some escape took place, and i do not know how to fix
this the right way. (And the line editor has history.)
--steffen
|
|Der Kragenbaer,The moon bear,
|der holt sich munter he cheerfully and one by one
|einen nach dem anderen runter wa.ks himself off
|(By Robert Gernhardt)
|~~
|..and in spring, hear David Leonard sing..
|
|The black bear, The black bear,
|blithely holds his own holds himself at leisure
|beating it, up and down tossing over his ups and downs with pleasure
|~~
|Farewell, dear collar bear
Re: [PATCH] Add -executable, -readable, and -writable options to /usr/bin/find
Jared Harper wrote in : |On Sat, Apr 1, 2023 at 12:06 AM Theo Buehler wrote: ... |I'm not sure these are equivalent. My (limited?) understanding is these |examples are checking whether the file's owner has the specified |permissions. | |The intention for my patch is to return true if the caller of find has |the specified permissions. Ya it rather equals test(1)'s -r, -w, -x in that spirit. "access(2) is dead, do not use it" comes to mind. Btw and off-topic, i just converted a program to use pledge and unveil, and i think the combination is a really, really great thing. (Aand especially in conjunction with that special syslog system call.) For programmers to use, futhermore. (What is missing is drawing the lottery prize, unfortunately.) --steffen | |Der Kragenbaer,The moon bear, |der holt sich munter he cheerfully and one by one |einen nach dem anderen runter wa.ks himself off |(By Robert Gernhardt)
Re: Fix broken UTF-8 decoding
Crystal Kolipe wrote in
:
|Currently it is not possible to use unicode codepoints > 0xFF on the \
|console,
|because our UTF-8 decoding logic is badly broken.
|
|The code in question is in wsemul_subr.c, wsemul_getchar().
|
|The problem is that we calculate the number of bytes in a multi-byte
|sequence by just looking at the high bits in turn:
...
|This is wrong, for several reasons.
Just to note there are also holes, UTF-8 sequences are not
necessarily well-formed (per se -- maybe they are when you control
their generation, of course). It is actually a real mess:
if(LIKELY(x <= 0x7Fu))
c = x;
/* 0xF8, but Unicode guarantees maximum of 0x10u -> F4 8F BF BF.
* Unicode 9.0, 3.9, UTF-8, Table 3-7. Well-Formed UTF-8 Byte Sequences
*/
else if(LIKELY(x > 0xC0u && x <= 0xF4u)){
if(LIKELY(x < 0xE0u)){
if(UNLIKELY(l < 1))
goto jenobuf;
--l;
c = (x &= 0x1Fu);
}else if(LIKELY(x < 0xF0u)){
if(UNLIKELY(l < 2))
goto jenobuf;
l -= 2;
x1 = x;
c = (x &= 0x0Fu);
/* Second byte constraints */
x = S(u8,*cp++);
switch(x1){
case 0xE0u:
if(UNLIKELY(x < 0xA0u || x > 0xBFu))
goto jerr;
break;
case 0xEDu:
if(UNLIKELY(x < 0x80u || x > 0x9Fu))
goto jerr;
break;
default:
if(UNLIKELY((x & 0xC0u) != 0x80u))
goto jerr;
break;
}
c <<= 6;
c |= (x &= 0x3Fu);
}else{
if(UNLIKELY(l < 3))
goto jenobuf;
l -= 3;
x1 = x;
c = (x &= 0x07u);
/* Third byte constraints */
x = S(u8,*cp++);
switch(x1){
case 0xF0u:
if(UNLIKELY(x < 0x90u || x > 0xBFu))
goto jerr;
break;
case 0xF4u:
if(UNLIKELY((x & 0xF0u) != 0x80u)) /* 80..8F */
goto jerr;
break;
default:
if(UNLIKELY((x & 0xC0u) != 0x80u))
goto jerr;
break;
}
c <<= 6;
c |= (x &= 0x3Fu);
x = S(u8,*cp++);
if(UNLIKELY((x & 0xC0u) != 0x80u))
goto jerr;
c <<= 6;
c |= (x &= 0x3Fu);
}
x = S(u8,*cp++);
if(UNLIKELY((x & 0xC0u) != 0x80u))
goto jerr;
c <<= 6;
c |= x & 0x3Fu;
}else
goto jerr;
--steffen
|
|Der Kragenbaer,The moon bear,
|der holt sich munter he cheerfully and one by one
|einen nach dem anderen runter wa.ks himself off
|(By Robert Gernhardt)
Re: Does openssl know about ROP?
Theo de Raadt wrote in
<53203.1675056...@cvs.openbsd.org>:
|I was reading some openssl source code, in particular the x86 assembly
|language files (which accelerate some crypto operations), and I find
|many cases where data tables are intentionally inserted into text (code)
|segments, and those tables include the byte value 0xC3.
|
|By intentional, I mean there's a comment, don't let me judge the tone \
|of it:
|
|&set_label("AES_Td",64);# Yes! I keep it in the code segment!
This could get you going with only TEXT and stack/BSS!
Boost cache hotness and reduce memory wastage!!
It was a real thing -- i am in full sympathy to the neat idea (a
common thing at some time i lifed through).
...
|Far be it from me to suggest that the security experts over there in
|OpenSSL land are unaware of modern exploitation methods! Very far from
|that, very very far.
I would presume the above code to be very old. There were times
when that project was lingering "on empty", and speeding assembler
optimizations was major traffic on their ML.
On most operating systems code segment is readable anyway.
I would presume adjusting execution pointer to such a byte would
result in abortion because often compilations use
stack-protector-strong and such (sigh). (..Yes, yes..)
But congratulations to your idea, surely many interested readers
have an eye on it!
They have just finished their full QUIC implementation, by
sheer coincidence i was posting a very minor issue that day
shortly after, so i saw it in the tree.
--steffen
|
|Der Kragenbaer,The moon bear,
|der holt sich munter he cheerfully and one by one
|einen nach dem anderen runter wa.ks himself off
|(By Robert Gernhardt)
Re: grdc: show timezone when TZ is set
David Goerger wrote in : |Sunday, 20220918 13:38+, Florian Obser wrote: |>I'm happy with that, let's do this then |>- fix the offset calculation |>- output tm->tm_zone in addition to TZ to be able to spot typos. | |I like the overall diff (thanks!), but one minor formatting nit is that |I believe it's more common under ISO 8601[1] to print UTC timezone |offset as "+/-%H%M", e.g. "-0400", not as "-4h00". At least, that's |what I'm familiar with in SMTP headers. | |Updated diff below, where only change from your latest diff is in the |second printw(3) statement. Tested against both TZ='America/New_York' |(-0400) and TZ='Asia/Kolkata' (+0530). | |Again, thanks for polishing this old "game"! Only to mention that nothing prevents timezones from using second offsets also. This is a political thing, and the TZ DB even is about to start supporting millisecond resolution (i think). Anyhow, since you mentioned the SMTP standard, that was surely an all american miss to go for +-HHMM instead of +-HHMMSS. The CLOCK_TAI clock is off by only seconds. (And someone wants to use its own TZ file, surely atomic-whatever-genitals thus!) --steffen | |Der Kragenbaer,The moon bear, |der holt sich munter he cheerfully and one by one |einen nach dem anderen runter wa.ks himself off |(By Robert Gernhardt)
Re: Picky, but much more efficient arc4random_uniform!
Joerg Sonnenberger wrote in : |Am Wed, May 18, 2022 at 12:49:21PM +0200 schrieb Steffen Nurpmeso: |> What surprised me was that the Apple code requires more calls, and |> that today divisions and multiplications still matter. I think it |> was the Cyrix 166+ (or was it Athlon 1600+) where +,-,<<,>> was |> one cycle, * was ten cycles, and %,/ was fourty cycles. But |> i think the Core i5 8th Gen that i have requires one cycle for all |> of them. | |I dare you to find any CPU optimized for speed where division and modulo |are as fast as multiplication. What you have nowadays is a single cycle |for basic operations like additions and shifts, about three cycles for |multiplications (on x86 at least) and an order of magnitude more for |division. That's not really surprising either if you consider that Thanks. I really had not looked since about 2005. (In fact i never really looked, i think a c't magazine CD once came over with x86 interrupt lists, CPU cycle usage, and a (sketchy) RFC collection.) Hm, it even seems the number of cycles increased for division .. i do not know which numbers i had seen by then, did they include cache work at all? Well. So this was that. |integer multiplication has a fast circuit implementation where as |division is often implemented as cascading conditional subtraction. |The difference matters enough turning the reminder operation in the ELF |hash into essentially (x - x * (1/size) * size) give a 2% speed up for a |large application like Firefox ten years ago. Interesting. Well you live in pkgsrc with all the mess that all those huge software suites introduce, whereas i still "jerk" if i produce software which needs more than EXEC (/ RODATA) and BSS, and "shudder" for all the CTOR and DTOR and TLS (but ok: cool) and all the other stuff that is found in ELF, and handled by the linker. (I have ELF v1.2 and TLS-ABI, and even glanced over them a really long time ago though. But you know, it is so far off! And it makes no fun thinking about that :)) |Joerg --End of --steffen | |Der Kragenbaer,The moon bear, |der holt sich munter he cheerfully and one by one |einen nach dem anderen runter wa.ks himself off |(By Robert Gernhardt)
Re: Picky, but much more efficient arc4random_uniform!
Steffen Nurpmeso wrote in <20220518104921.sr0ht%stef...@sdaoden.eu>: ... |The web site that has been linked from the man from the country |that has an even much worse Earth Country Overshoot Day than |Germany and is almost en par with Australia or even USA (April |3rd, pooh; never again a Saab! Cuba: Nov 25th, Jamaica Dec 20th) Finland is March 31st. Man is that dirty. --steffen | |Der Kragenbaer,The moon bear, |der holt sich munter he cheerfully and one by one |einen nach dem anderen runter wa.ks himself off |(By Robert Gernhardt)
Re: Picky, but much more efficient arc4random_uniform!
Philip Guenther wrote in
:
|On Tue, May 17, 2022 at 1:10 PM Steffen Nurpmeso \
|wrote:
|> Joerg Sonnenberger wrote in
|> :
|>|Am Fri, May 13, 2022 at 09:43:26AM -0500 schrieb Luke Small:
|>|> I made a couple new versions of a new kind of arc4random_uniform-like
|> ...
|>|If your main use case is limiting the amount of cryptography when using
|>|small bounds, there is a much simpler approach to be taken here. For
|>|boundaries below 256, use arc4random_buf to extract one byte if bound is
|>|a power of two, otherwise two. This gives most of the performance
...
|> You can use (really implemented) _buf() if you need a 8-bit or
|> 16-bit etc number.
|>
|> I find that _uniform() often makes no difference to a simple
|> modulo because like the comment in _uniform() says "p > 0.5 (worst
|>case, usually far better", and usually RNGs sprinkle bits nicely,
|
|What does that statement mean? You seem to be saying "module is uniform,
|except when it isn't, which could be almost half the time for some cases,
|but when it's uniform it's uniform, so why bother making it actually
|correct and dependable".
|
|I mean, what does that _mean_??? It's as if I said "my text handling
Well it means this thread was too long.
|program handles all characters uniformly, except those with accents, but
|that's less than 10% of the characters I type, so it handles all characters
|uniformly." WTF, NO!
But it also means that
// calculates 2**32 % range
uint32_t t = (-range) % range;
for (;;) {
uint32_t r = rng();
if (r >= t)
where range is a very small number results in a very, very low
probability that r>=t is not true. For 16-bit 0x two zero
bytes had to occur in the upper 16-bit. And worse for 64-bit RNG.
So this is what i said. It depends on the application.
This gets hypothetic and is anyway beyond my mathematical
capabilities. I mean i look at /etc/ssh/moduli, so much on
cramping of random numbers.
The web site that has been linked from the man from the country
that has an even much worse Earth Country Overshoot Day than
Germany and is almost en par with Australia or even USA (April
3rd, pooh; never again a Saab! Cuba: Nov 25th, Jamaica Dec 20th)
said the bias for the number 52 is 0.0121%. And what i posted
had ~0.008 that rand<0x01FF aka 32-bit high byte is 0, for
32-bit from getrandom(2) as well as mine (in "strong" mode,
SipHash on ARC4).
You need quite some samples to be able to find a statistical
frequency of that order. And it depends on the application of
that many samples exist.
And even TCP from RFC 793 / September 1981 has a 32-bit sequence
number.
But sure Philip, of course, yes, of course you are right: simply
call _uniform() and "milk the shit out of the random range" --
just use it and forget about the problem.
What surprised me was that the Apple code requires more calls, and
that today divisions and multiplications still matter. I think it
was the Cyrix 166+ (or was it Athlon 1600+) where +,-,<<,>> was
one cycle, * was ten cycles, and %,/ was fourty cycles. But
i think the Core i5 8th Gen that i have requires one cycle for all
of them. (And somewhere i read that there are CPUs where the
shift operators are now more expensive than multiplication.)
I do not really track any of that since 2005. That is nice: you
buy a laptop and suddenly have a NVME SSD that can 1.5GB/sec.
Wow.
|> 0 bytes "do not occur", so a 32-bit RNG value "is" >=0x01FF in
|> most cases for "my RNG" (of 10 803/759/793 NOT; 776/805/793
|> NOT for Linux getrandom(2)), which is a pretty high cut off.
...
|Where do these ideas come from, that "0 bytes 'do not occur'"?? If your
|rand generator doesn't provide zero bytes at the expected frequency, you
|know, 1 in 256, then you're using a garbage random number generator.
|Please stop making such suggestions here because THEY ARE NOT TRUE ABOUT
|OPENBSD. Do ya'll not bother to test the claims that you make?
...
|for (;;) {
|u = arc4random();
|if ((u & 0xff00) == 0 ||
|(u & 0x00ff) == 0 ||
|(u & 0xff00) == 0 ||
|(u & 0x00ff) == 0)
|break;
That is any-byte-0.
|00b82e5c58
|ab47880036
Seems random to me.
:)
--steffen
|
|Der Kragenbaer,The moon bear,
|der holt sich munter he cheerfully and one by one
|einen nach dem anderen runter wa.ks himself off
|(By Robert Gernhardt)
Re: Picky, but much more efficient arc4random_uniform!
Steffen Nurpmeso wrote in
<20220517220924.xohqc%stef...@sdaoden.eu>:
|Joerg Sonnenberger wrote in
| :
||Am Fri, May 13, 2022 at 09:43:26AM -0500 schrieb Luke Small:
||> I made a couple new versions of a new kind of arc4random_uniform-like
...
|0 bytes "do not occur", so a 32-bit RNG value "is" >=0x01FF in
|most cases for "my RNG" (of 10 803/759/793 NOT; 776/805/793
|NOT for Linux getrandom(2)), which is a pretty high cut off.
...
/*@ lotto.cxx - output six random numbers */
#include
#include
#include
//#define NYDPROF_ENABLE
//#define NYD_ENABLE
//#define NYD2_ENABLE
#include
NSPC_USE(su)
static u32
bounded_rand(u32 range, u32 rv){
for(u32 t = -range % range;;){
if(rv >= t)
return rv % range;
log::write(log::emerg, "NOFIT");
}
}
int
main(void){
u32 uni[50], mod[50], rv;
state::create_core(NIL, (state::debug | log::debug), state::err_nopass);
su_mem_set(uni, 0, sizeof uni);
su_mem_set(mod, 0, sizeof mod);
u32 au=0;
for(u32 i = 0; i < 10; ++i){
random::builtin_generate(&rv, sizeof rv, state::err_nopass);
//if(getrandom(&rv, sizeof rv, GRND_NONBLOCK) == -1)
// log::write(log::emerg, "getrandom(2)");
if(rv < 0x01FF)
log::write(log::info, "AU %u - 0x%X", ++au,rv);
++mod[rv % NELEM(mod)];
++uni[bounded_rand(NELEM(mod), rv)];
}
u32 unilo, modlo, unihi, modhi;
unilo = modlo = max::u32;
unihi = modhi = 0;
for(u32 i = 0; i < NELEM(uni); ++i){
unilo = get_min(unilo, uni[i]);
modlo = get_min(modlo, mod[i]);
unihi = get_max(unihi, uni[i]);
modhi = get_max(modhi, mod[i]);
log::write(log::info, "%u - %u / %u %s",
i, uni[i], mod[i], (uni[i] != mod[i] ? "!!!" : "="));
}
log::write(log::info, "MIN %u / %u, MAX %u / %u - au %u",
unilo, modlo, unihi, modhi,au);
return 0;
}
#include
// s-it-mode
--steffen
|
|Der Kragenbaer,The moon bear,
|der holt sich munter he cheerfully and one by one
|einen nach dem anderen runter wa.ks himself off
|(By Robert Gernhardt)
Re: Picky, but much more efficient arc4random_uniform!
Joerg Sonnenberger wrote in : |Am Fri, May 13, 2022 at 09:43:26AM -0500 schrieb Luke Small: |> I made a couple new versions of a new kind of arc4random_uniform-like ... |If your main use case is limiting the amount of cryptography when using |small bounds, there is a much simpler approach to be taken here. For |boundaries below 256, use arc4random_buf to extract one byte if bound is |a power of two, otherwise two. This gives most of the performance |benefit without complicating the algorithm. Extracting two bytes ensures |that the propability of success is > 99% and the double extracting |doesn't eat up the benefits. You can use (really implemented) _buf() if you need a 8-bit or 16-bit etc number. I find that _uniform() often makes no difference to a simple modulo because like the comment in _uniform() says "p > 0.5 (worst case, usually far better", and usually RNGs sprinkle bits nicely, 0 bytes "do not occur", so a 32-bit RNG value "is" >=0x01FF in most cases for "my RNG" (of 10 803/759/793 NOT; 776/805/793 NOT for Linux getrandom(2)), which is a pretty high cut off. Using _uniform() just because of its name seems strange thus. --steffen | |Der Kragenbaer,The moon bear, |der holt sich munter he cheerfully and one by one |einen nach dem anderen runter wa.ks himself off |(By Robert Gernhardt)
Re: Picky, but much more efficient arc4random_uniform!
Philip Guenther wrote in : |On Sun, 15 May 2022, Steffen Nurpmeso wrote: |> Stuart Henderson wrote in |... |>|what's the perceived problem you're wanting to solve? and does that |>|problem actually exist in the first place? |> |> The problem is that if have a low upper bound then modulo will "remove a |> lot of randomization". For example if you have a program which |> generates Lotto numbers (1..49), then using _uniform() as it is will |> generate many duplicates. | |Wut. The *WHOLE POINT* of arc4random_uniform() is that it has uniform |distribution. Says right so in the manpage. If an implementation of that |API fails to do that, it's a broken implementation. I always admired its source code comments and have been left stunning more than just once. --steffen | |Der Kragenbaer,The moon bear, |der holt sich munter he cheerfully and one by one |einen nach dem anderen runter wa.ks himself off |(By Robert Gernhardt)
Re: Picky, but much more efficient arc4random_uniform!
Stuart Henderson wrote in
:
|On 2022/05/14 06:56, Luke Small wrote:
|> If I use arc4random_uniform() repeatedly to create a random distribution \
|> of
|> say numbers less than 0x1000 or even something weird like 0x1300 will the
|> random distribution be better with arc4random_uniform() or with mine?
|
|there's no point to have a choice of different arc4random_uniform_xyz
|with different characteristics, how is somebody going to pick the
|"right" one?
|
|the bottom of netbsd's version of the arc4random(3) manual says it well:
|
| One may be tempted to create new APIs to accommodate different \
| security
| models and performance constraints without unpleasant surprises \
| on dif-
| ferent operating systems. This should not be done lightly, though,
| because there are already too many different choices, and too \
| many oppor-
| tunities for programmers to reach for one and pick the wrong one.
|
|what's the perceived problem you're wanting to solve? and does that
|problem actually exist in the first place?
The problem is that if have a low upper bound then modulo will
"remove a lot of randomization". For example if you have
a program which generates Lotto numbers (1..49), then using
_uniform() as it is will generate many duplicates.
I used the approach i found in a TeX file over twenty years ago
("RANDOM.TEX, v.1 (Donald Arseneau) from TeTeX,
texmf/tex/generic/misc/random.tex"; and now that i look, he
published a .v2 not too long ago), it was named _clip() as it
produces something in between a minimum and a maximum:
_max -= _min;
++_max;
_min = Maximum::sir - 2; // m - 2 = 2^(32|64) - 3
if (_max != 0)
_min /= _max;
for(;;) {
uir i;
i = _random;
i += M1::uir;
if(_min != 0)
i /= _min;
if(i >= _max && (i != 0 || _max != 0)) {
_random *= 16807; // just MUL
if(_random == 0)
_random = 1;
continue;
}
_random = i;
break;
}
_random += origmin;
_Assert(1 && _random >= origmin && _random <= origmax);
That is the random number is multiplied with a, eh, non-prime, as
long as we do not fit.
However i found this became a terrible mess with 64-bit numbers,
so i removed the function, aka did not "port it over" when
i implemented a random number generator for my own use cases.
Maybe i should have increased the multiplicator.
It is really, really bad. :-)
I only ever used it for a Lotto number program anyhow, i think,
and so i thought it would be better to simply read a one byte
random and skip those that do not fit, instead of making them
always fit with modulo 50. That is, application specific
workaround. Things are less harmful for large numbers, anyway.
In fact i have to say i was astonished to see someone using this
function, not too long ago i saw a FreeBSD commit fly by which
uses it!
I was a bit interested when i saw Luke's message, but i am no
mathematician and, like i said, i in fact never needed the
functionality. And the question i would have is how small
upper_bound has to be for the modulo problem to really kick in.
And even if, whether it matters. For example, if you have
a bitset of 1024 "in-flight things", and the small upper bound
hits a used slot, you could simply search forward until you find
a free one. I mean, with 1024 possible values the number of
possibilities is anyhow restricted.
--steffen
|
|Der Kragenbaer,The moon bear,
|der holt sich munter he cheerfully and one by one
|einen nach dem anderen runter wa.ks himself off
|(By Robert Gernhardt)
Re: clang: compile static analyzer
Joerg Sonnenberger wrote in
:
|Am Fri, Jan 21, 2022 at 12:45:56AM +0100 schrieb Steffen Nurpmeso:
|> Fwiw, i have been astonished by this thread. I found scan-build
|> to generate a lot of false warnings, so much indeed that i stopped
|> using it .. in summer 2017.
|
|I've spend time on the static analyzer output in NetBSD and I wouldn't
|say so much that it creates too many false warnings, but that the pure
Surely i talk only small things.
|text version is not helpful. The HTML output at least explains the
Yes i like that from Coverity you get a text email (or at least
last i used it, development branch in work for two years and that
not yet, and master mostly stable; and the others never, hmm) with
at least the points of interest of all new defects. Most of the
time that is all i need!
|reasoning. From those pre-conditions, it is often easy to deduce why it
|is a false positive from *other* conditions in the program. Properly
|asserting those would certainly improve code.
Like is said somewhen in this thread, you are forced to instrument
code like grazy. And that sucks like grazy; just one or two days
ago, for example, i finally decided to do a FALLTHRU macro and
replace all /* FALLTHRU */ comments with it. One should think
this is a matured thing covered by unit tests on the compiler
front, but i am conservative and just enable it with the compiler
versions i have at hand (working around specific compiler bugs
/ on particular OSs is one of the things), but you stumble
whenever you go! So for gcc 11.2.0 i now have made this
"UNUSED(0);__attribute__ ((fallthrough));" because without the
UNUSED(0); like for clang gcc complains "a label can only be part
of a statement and a declaration is not a statement" in a very
very normal case: fallthrough condition (but only one of the many,
mind you! iirc). Or is "case A:\nFALLTHRU\ncase B:.." forbidden
now too? Honestly i find all these things very much annoying and
not helpful at all. I'd rather have some more keywords like
"unroll" before loop keywords or whatever, so that i have the
control rather than the compiler (i had just read "Who's afraid of
a big bad optimizing compiler?" of LWN some hm week ago).
So this is all theory. Or take this
// xxx clang 5.0.1 BUG: needed this-> to find superclass field
gview(void) {this->csdv_parent = NIL; this->csdv_node = NIL;}
Only there of the many. And whatever. Nono.
|The biggest advantage in coverity is the logic they have for preserving
|the state of analysis across code changes, e.g. once you tag a reported
|issue as analyzed and not a problem, it tries very hard to not show it
|again.
Yes! Even without instrumentation that is.
I mean, hey!, this is _very_ complicated stuff. It is just that
i _never_ had luck with scan-build, really. And the example
i posted last here... Shit always happens, but if you _can_
analyze code execution paths through all units involved in the
run, i would have expected that this condition cannot produce
a false positive, because it cannot. If NIL it jumps away never
to come back. It cannot be NIL in the code that follows, period.
-Weverything is pretty exhaustive, i try to write new code so it
passes this.
--steffen
|
|Der Kragenbaer,The moon bear,
|der holt sich munter he cheerfully and one by one
|einen nach dem anderen runter wa.ks himself off
|(By Robert Gernhardt)
Re: clang: compile static analyzer
Hello. Steffen Nurpmeso wrote in <20220121154612.frou2%stef...@sdaoden.eu>: |Andre Smagin wrote in | <20220120231806.0463e0c792db3e3e5fc07...@smagin.com>: ||On Fri, 21 Jan 2022 00:45:56 +0100 ||Steffen Nurpmeso wrote: ||> I found scan-build to generate a lot of false warnings, so much indeed ||> that i stopped using it .. in summer 2017. || ||You, and most others, (no sarcasm at all here) are much better ||at C than I will ever be. I am not even at "amateur" level - more like ||a part-time hobby. I mostly fiddle with text data compression for fun. || ||For me, clang analyzer is more than helpful. It detects errors that ||most of you, professional programmers, would never make. You probably ||don't even realize the power of it, since you never make such ||embarrassing mistakes while coding as I do. For me, every time there was No no all that not true you know. ||a warning from clang, it was fully warranted. Took me couple days in Yes. Use -Weverything and it can help you quite a lot (if the masses of warnings from the standard libraries do not make you run away). ||some cases to figure out why, but it was always justified and I learned ||a lot from it. But scan-build is just an *annoyance*. ||I install clang from ports solely for the analyzer part. It would be ||nice if it was included in base installation - some of us just want a ||basic idiot-check tool available when trying to program things. But ||ports installation works as well too. So for example, because it really bothered me when i tried it "against better knowledge". See for example this "simple" control flow (p is a union of a void* and a char* pointer, for easy casting): 1499 1500 p.c = NIL; ← Null pointer value stored to 'p.c' → ..nothing with p.c here, only to be able to jump in case of error and have no munmap(2) happen.. 1505 else if((p.v = mmap(NIL, S(uz,st.st_size), PROT_READ, MAP_SHARED, i, 0)) == NIL) ... 1511 if((mbase = p.v) == NIL) ← Assuming the condition is false ... So no error, no jump to error return next. p.[vc] is not NIL. It is not here nor will anything but auto-increment happen to it any more. Now a simple loop over the memory mmap(2)ed into p.[vc]. ... Loop condition is true. Entering loop body ... → 1523if(*p.c != '\n') ← Dereference of null pointer (loaded from field 'c') You know. But how this. How can -Weverything go down a complicated code flow and get things right that you (me! that is.) simply does not see, but scan-build bails on such a simple first-level-cannot-happen flow analysis error? And such things happen all the time. And from my experience Coverity does not have such problems, but especially not with such a frequency. Only false positives with this run. But -Weverything is really a good thing, i would only wish gcc had it too, because i am such an idiot with all the many options. --steffen | |Der Kragenbaer,The moon bear, |der holt sich munter he cheerfully and one by one |einen nach dem anderen runter wa.ks himself off |(By Robert Gernhardt)
Re: [patch] httpd static gzip compression
j...@bitminer.ca wrote in <07cf90c2d8adab83ffd36ef69ebcd...@bitminer.ca>: |On 2022-02-27 19:28, Solène wrote: |> If I remember well, lighttpd will compress server side upon request if |> a compressed version of that file doesn't exist. |> |> This is entirely different than how httpd behave, because it doesn't |> compress files at all. The separate path to avoid ambiguities would still address one concern de Raadt had a couple of months back if i recall correctly. And for example doing a dirname and adding ."CVS" before the basename.gz would only slightly complicate aka enlargen the patch. My main use case for www.openbsd.org (having a quick look at manuals) would not be covered by that though, and ML archives are not hosted either. HTML compresses a lot, i find it desirable. |I would describe the automatic selection of available .gz files as |a hand-crafted compression cache. | |This is not user-friendly to either system admins or webmasters |(assuming they are different people). | |My proposal was to automate the cache-invalidate logic (so-to-speak), |until/unless the file owner (sysadmin or webmaster) updates the gz |file. This reduces a possible error path. | |There are a few use-cases for the gz feature: | |1) large files built exactly once; the gz file is therefore built |once, and hand-crafted compression is selected in config and by |creating the gz file. The file owner(s) know what they are doing. | |2) large files updated a few times; perhaps the owner(s) will |remember to update gz files. This is where the proposal fits in. |Even though they know what they are doing, helping invalidate the |cache is a good thing until the cache is updated. | |3) a set of large files which need consistent treatment: outside |the scope of such a program at httpd or it's users. | |Please consider the above as reasoning for somewhat more logic |than was first suggested. --steffen | |Der Kragenbaer,The moon bear, |der holt sich munter he cheerfully and one by one |einen nach dem anderen runter wa.ks himself off |(By Robert Gernhardt)
Re: [patch] httpd static gzip compression
Steffen Nurpmeso wrote in
<20220227225619.pzahj%stef...@sdaoden.eu>:
|Solène wrote in
| <3311e74f-1ad8-49b3-96d7-3f3c7f2af...@perso.pw>:
||27 févr. 2022 19:37:20 j...@bitminer.ca:
||
||> Would it be too much to ask to defend the (poor) web master against
||> own-goal errors?
||>
||> That is, approximately:
||>
||> if ((access(gzpath, R_OK) == 0) &&
||> (stat(gzpath, &gzst) == 0) &&
||> (gzst->st_mtim.tv_sec >=
||> st->st_mtim.tv_sec)) { /* new test */
||> path = gzpath; st = &gzst; kv_add(&resp->http_headers,
||> "Content-Encoding", "gzip");
||> }
||>
||> (apologies for formatting errors)
||>
||> In english: the gz file must be the same age as or newer than the
||> original.
||>
||> My assumption being that "static" files are not always static.
||> And correctly updating .gz files requires a bit of a delete-update-recre\
||> \
||> ate
||> dance.
|
||I'd prefer not have much logic for this so it's easier to understand \
||for admins. This feels wrong to serve a file or another depending on \
||their timestamp.
|
|Sorry for stepping into this again, but lighttpd compress uses
|a special folder for this, like this ambiguities cannot happen.
(And a simple cron job can remove things which have been unused
for a long time.)
--steffen
|
|Der Kragenbaer,The moon bear,
|der holt sich munter he cheerfully and one by one
|einen nach dem anderen runter wa.ks himself off
|(By Robert Gernhardt)
Re: [patch] httpd static gzip compression
Solène wrote in
<3311e74f-1ad8-49b3-96d7-3f3c7f2af...@perso.pw>:
|27 févr. 2022 19:37:20 j...@bitminer.ca:
|
|> Would it be too much to ask to defend the (poor) web master against
|> own-goal errors?
|>
|> That is, approximately:
|>
|> if ((access(gzpath, R_OK) == 0) &&
|> (stat(gzpath, &gzst) == 0) &&
|> (gzst->st_mtim.tv_sec >=
|> st->st_mtim.tv_sec)) { /* new test */
|> path = gzpath; st = &gzst; kv_add(&resp->http_headers,
|> "Content-Encoding", "gzip");
|> }
|>
|> (apologies for formatting errors)
|>
|> In english: the gz file must be the same age as or newer than the
|> original.
|>
|> My assumption being that "static" files are not always static.
|> And correctly updating .gz files requires a bit of a delete-update-recre\
|> ate
|> dance.
|I'd prefer not have much logic for this so it's easier to understand \
|for admins. This feels wrong to serve a file or another depending on \
|their timestamp.
Sorry for stepping into this again, but lighttpd compress uses
a special folder for this, like this ambiguities cannot happen.
--steffen
|
|Der Kragenbaer,The moon bear,
|der holt sich munter he cheerfully and one by one
|einen nach dem anderen runter wa.ks himself off
|(By Robert Gernhardt)
Re: clang: compile static analyzer
Andre Smagin wrote in <20220120231806.0463e0c792db3e3e5fc07...@smagin.com>: |On Fri, 21 Jan 2022 00:45:56 +0100 |Steffen Nurpmeso wrote: | |> I found scan-build to generate a lot of false warnings, so much indeed |> that i stopped using it .. in summer 2017. | |You, and most others, (no sarcasm at all here) are much better |at C than I will ever be. I am not even at "amateur" level - more like |a part-time hobby. I mostly fiddle with text data compression for fun. | |For me, clang analyzer is more than helpful. It detects errors that |most of you, professional programmers, would never make. You probably |don't even realize the power of it, since you never make such |embarrassing mistakes while coding as I do. For me, every time there was |a warning from clang, it was fully warranted. Took me couple days in |some cases to figure out why, but it was always justified and I learned |a lot from it. | |I install clang from ports solely for the analyzer part. It would be |nice if it was included in base installation - some of us just want a |basic idiot-check tool available when trying to program things. But |ports installation works as well too. I nonetheless think of overlaying to get rid of it again here. This is hobby here .. they prefer that instead of cryptomining, maybe. I am pretty sure noone here uses all the utilities. I personally enable a lot of compiler warnings, 'am not there with clang's -Weverything, but lots of others. Not much remains to do if that works. I find that Coverity that you scratched from my message does a really good job out of the box, maybe scan-build is better if you instrument the code like grazy, you know, all the attributes that can be attached to lots of places to make the code like C# or Rust. Or so. 'Got the ashes to prove it. (P.S.: my try was before the FreeBSD commit series with dead assignments, 2022-01-14 22:18:04.) --steffen | |Der Kragenbaer,The moon bear, |der holt sich munter he cheerfully and one by one |einen nach dem anderen runter wa.ks himself off |(By Robert Gernhardt)
Re: clang: compile static analyzer
Alexander Bluhm wrote in
:
|On Fri, Jan 14, 2022 at 05:59:24PM +0100, Claudio Jeker wrote:
|> On Fri, Jan 14, 2022 at 04:44:49PM +, Stuart Henderson wrote:
|>> On 2022/01/14 16:52, Rafael Sadowski wrote:
|>>> On Fri Jan 14, 2022 at 03:17:21PM +0100, Tobias Heider wrote:
| clang ships with a pretty useful static analyzer to find all kinds \
| of bugs
| in C and C++ code:
|
| https://clang-analyzer.llvm.org/
|
| I use it regularly to check my own diffs and found plenty of bugs \
| I could
| have missed otherwise. While we have the code in base we don't \
| actually
| build it into our libclang currently, so the only ways to use it are
| manually modifying the Makefiles or installing llvm from ports.
|
| I was wondering if anyone else uses this and if there was any \
| interest to
| have this in our base clang?
Fwiw, i have been astonished by this thread. I found scan-build
to generate a lot of false warnings, so much indeed that i stopped
using it .. in summer 2017.
So i tried again and except for a lot of "dead assignments",
which, well, are dead assignments but then again it is my style of
coding since on x86 there is a return register and so i am used to
write it like this, just in case, you know, and then "self" aka
*this is ready to go, there were mostly false warnings.
Coverity.com on the other hand always surprises me over and over
again with what it finds, .. and i do not even get paid for that.
'Only sayin' because clang here is sheer unbelievable almost 800MB
installed, and i wonder what for, even if (!) compared to gcc.
I do not use clang-format (though this might be an error of mine).
But mind you, i do not instrument my code for lint and successors
(except FALLTHRU, and that luckily now that compilers start
screaming for such cases, heck i am not autopilot driving). Now
that i am on github (again, mostly to ease people's burden with my
bug reports) i think i will not even download the newest Coverity
software but start using github's coverity hook.
You know, i struggle here because of
mv clang-tools-extra-$version.src clang-$version.src/tools/extra
cmake -S $name-$version.src -B build -G Ninja \
-D CMAKE_INSTALL_PREFIX=/usr \
-D CMAKE_INSTALL_LIBEXECDIR=lib/clang \
-D CMAKE_BUILD_TYPE=Release \
-D CMAKE_C_FLAGS_RELEASE="$CFLAGS" \
-D CMAKE_CXX_FLAGS_RELEASE="$CXXFLAGS" \
-D LLVM_EXTERNAL_LIT=/usr/bin/lit \
-D LLVM_PARALLEL_COMPILE_JOBS="${JOBS:-1}" \
-Wno-dev
cmake --build build
That is, i have to have it all or put more effort in maintaining
my system. I hate it. :)
--steffen
|
|Der Kragenbaer,The moon bear,
|der holt sich munter he cheerfully and one by one
|einen nach dem anderen runter wa.ks himself off
|(By Robert Gernhardt)
Re: cwm: fix invalid KeyCode in configuration
Jan Stary wrote in
:
|On Nov 18 20:13:03, h...@stare.cz wrote:
|> On Nov 16 21:33:31, hen...@camandro.org wrote:
|>> I've tried to setup a line like:
|>> bind-key XF86MonBrightnessDown ""
|>> in my .cwmrc and the result was that no key event was sent to my \
|>> windows.
|>
|> Please excuse my X ignorance, but shouldn't XF86MonBrightnessDown
|> be catched by the X server (to take the brightness down),
|> as opposed to passing it on to cwm?
|
|Or even sooner? For example, xev(1) does not report anything
|when I press [Fn]+[LightsDown] on my Thinkpad T400 running cwm.
Just tell him that his keyboard does not generate the event, and
that 0 is NoSymbol or how it is called.
I have
bind-key XF86AudioLowerVolume "adjust-xf86-mediahooks.sh XF86AudioLowerVolume"
bind-key XF86AudioRaiseVolume "adjust-xf86-mediahooks.sh XF86AudioRaiseVolume"
bind-key XF86AudioMute "adjust-xf86-mediahooks.sh XF86AudioMute"
bind-key XF86MonBrightnessDown "adjust-xf86-mediahooks.sh
XF86MonBrightnessDown"
bind-key XF86MonBrightnessUp "adjust-xf86-mediahooks.sh XF86MonBrightnessUp"
..but it just ends up via
XF86MonBrightnessDown)
isf=/run/.acpid_brightness msg='Brightness: ';;
XF86MonBrightnessUp)
isf=/run/.acpid_brightness msg='Brightness: ';;
esac
etc as
LC_ALL=C xmessage -center -timeout 1 "${msg}" /dev/null 2>&1 &
--End of
--steffen
|
|Der Kragenbaer,The moon bear,
|der holt sich munter he cheerfully and one by one
|einen nach dem anderen runter wa.ks himself off
|(By Robert Gernhardt)
Re: [patch] httpd static gzip compression
... |You would be asking for | |https://exmaple.com/whatever/ls.1 | |and with Accept-Encoding: gzip in the http header, and the |webserver would then look if it has a file | |/whatever/ls.1.gz | |(instead of without .gz) in its document tree and send you that, with |"Content-Encoding: gzip" http header. As an outsider i find this thread very amusing. As you all know the normal approach is to have a cache directory where some "compress" module performs on-the-fly storage if the client asks for some file, accepts compression, and the compressed version yet does not exist. Funnily i once got not even an answer when i asked for static compression, since on-the-fly compression was already available, "so what", i have forgotten which webserver that has been. Cleanup via cron anyhow. Now something for Theo, from the webserver i use. #if defined HAVE_ZLIB_H && defined HAVE_LIBZ # define USE_ZLIB # include #endif #ifndef Z_DEFAULT_COMPRESSION #define Z_DEFAULT_COMPRESSION -1 #endif #ifndef MAX_WBITS #define MAX_WBITS 15 #endif #if defined HAVE_BZLIB_H && defined HAVE_LIBBZ2 # define USE_BZ2LIB /* we don't need stdio interface */ # define BZ_NO_STDIO # include #endif #if defined HAVE_BROTLI_ENCODE_H && defined HAVE_BROTLI # define USE_BROTLI # include #endif #if defined HAVE_ZSTD_H && defined HAVE_ZSTD # define USE_ZSTD # include #endif #if defined HAVE_SYS_MMAN_H && defined HAVE_MMAP && defined ENABLE_MMAP #define USE_MMAP --steffen | |Der Kragenbaer,The moon bear, |der holt sich munter he cheerfully and one by one |einen nach dem anderen runter wa.ks himself off |(By Robert Gernhardt)
Re: ftp(1): add SOCKS proxy support
Nick Gasson wrote in <877dovsfjk@bertha.nickg.me.uk>: |On 11/27/20 05:59 AM, Steffen Nurpmeso wrote: |> Nick Gasson wrote in |> <87im9srza8@bertha.nickg.me.uk>: |>|Hi, |>| |>|I often need to go through a SOCKS proxy to access certain sites. The |>|diff below adds SOCKS5 support to ftp(1) for HTTP transfers, similar to |>|curl(1). Enabled when http_proxy is set to a socks5:// URL. |>| |>|Also fixes two existing memory leaks: proxyurl (set to NULL on line 646 |>|before freeing) and sslpath (never freed). |>| |>|Tested with ssh -D and a few other SOCKS5 proxies. Also verified the |>|existing HTTP proxy feature still works with squid(8). |> |> By the way, the $SOCKS5_PROXY environment variable becomes used |> for automatic selection of SOCKS5. (Some things on FreeBSD, |> lynx(1), and, hm, the MUA i maintain, s-nail; maybe more.) |> | |(Sorry for the late reply.) For me - no problem. | |Yes I see FreeBSD fetch added SOCKS5_PROXY recently. I've updated the |diff below to support that too. Anyone interested? Fine. That looks good to me as socks code in general, i do not use named constants because RFC 1928 CONNECT request is the reference, and all SOCKS code is in one function. I also output the errors as strings, for the codes you have to read RFC 1928 yourself, which is pain :) Btw. i see lots of problems with SOCKS5 proxy support of ssh on Linux, i proxy also firefox and we go over wireless on a bad link, and due to the massive parallelization this reaches a hundred concurrent sockets very fast when browsing German/Austrian/English newspapers .. and when ssh gets stuck then we hang 'till the muxer dies. Happens almost every day. 100 should not hit any Linux limit here. But that has nothing to do with OpenBSD and fetch. And you. A nice weekend and Ciao from Germany, --steffen | |Der Kragenbaer,The moon bear, |der holt sich munter he cheerfully and one by one |einen nach dem anderen runter wa.ks himself off |(By Robert Gernhardt)
Re: ftp(1): add SOCKS proxy support
Nick Gasson wrote in <87im9srza8@bertha.nickg.me.uk>: |Hi, | |I often need to go through a SOCKS proxy to access certain sites. The |diff below adds SOCKS5 support to ftp(1) for HTTP transfers, similar to |curl(1). Enabled when http_proxy is set to a socks5:// URL. | |Also fixes two existing memory leaks: proxyurl (set to NULL on line 646 |before freeing) and sslpath (never freed). | |Tested with ssh -D and a few other SOCKS5 proxies. Also verified the |existing HTTP proxy feature still works with squid(8). By the way, the $SOCKS5_PROXY environment variable becomes used for automatic selection of SOCKS5. (Some things on FreeBSD, lynx(1), and, hm, the MUA i maintain, s-nail; maybe more.) --steffen | |Der Kragenbaer,The moon bear, |der holt sich munter he cheerfully and one by one |einen nach dem anderen runter wa.ks himself off |(By Robert Gernhardt)
Re: top: Fill last character in process line
Klemens Nanni wrote in <20200603124948.irvdnxrxa5g75pmi@eru>: |On Wed, Jun 03, 2020 at 12:45:35PM +0100, Stuart Henderson wrote: |> It should check terminal capabilities for this, see termcap(5). |> If 'am' (auto-margin) is set then it shouldn't write to the final column. |> If 'xn' is set then it's OK in some circumstances (it's probably \ |> easier to |> skip writing to the final column if this is set too). |Thanks mark and Stuart, I did not know about auto-margin (or autoWrap as |xterm(1) seems to call it). | |What I understand is that writing to the screen's last terminal should |be avoided in terminal without the "am" capability, presumably because |it would cause a line wrap - is that correct? | |Preliminary testing however indicates to me that at least xterm(1) |behaves the same in top's interactive screen with my patch, regardless |of the auto-margin capablility. | |According to termcap(5) I did the following to disable "am", with |tput(1) I verify that it gets indeed disabled: | | $ echo $TERM ; tput am ; echo $? | xterm | 0 | $ TERM=vt100-nam ; tput am ; echo $? | 1 | |But in both cases, starting ./obj/top in the very same terminal/shell |behaves the same, that is to say the last column is written properly and |I see no line wrap or any change of behaviour. I do not think that you en- or disable "am", it indicates whether the terminal has the capability or not. I have n_TERMCAP_QUERY_am, /* am/am, BOOL | auto_right_margin */ n_TERMCAP_QUERY_sam,/* sam/YE, BOOL | semi_auto_right_margin */ n_TERMCAP_QUERY_xenl, /* xenl/xn, BOOL | eat_newline_glitch */ ...^^terminfo/termcap^^ names | terminfo variable eat_newline_glitch ensures that \n written after margin does not cause a line wrap. /* TODO We do not handle !n_TERMCAP_QUERY_sam in this software! */ if(!n_termcap_query(n_TERMCAP_QUERY_am, &tv) || n_termcap_query(n_TERMCAP_QUERY_xenl, &tv)) n_psonce |= n_PSO_TERMCAP_FULLWIDTH; ... /* Then search for right boundary. Dependent upon n_PSO_FULLWIDTH (termcap * am/xn) We leave the rightmost column empty because some terminals * [cw]ould wrap the line if we write into that, or not. * TODO We do not deal with !n_TERMCAP_QUERY_sam */ --End of <20200603124948.irvdnxrxa5g75pmi@eru> --steffen | |Der Kragenbaer,The moon bear, |der holt sich munter he cheerfully and one by one |einen nach dem anderen runter wa.ks himself off |(By Robert Gernhardt)
Re: openssl.1: Tag command names
Ingo Schwarze wrote in <20200218160703.gb27...@athene.usta.de>: |Hi, | |Steffen Nurpmeso wrote on Tue, Feb 18, 2020 at 04:52:48PM +0100: | |> i just want to add that there is still the mdocmx mdoc macro |> extension available, and is working fine for more than half |> a decade. I have not ported that to groff 1.22.4, but it is |> available for groff 1.22.3. It can much more than .Tg, of course, | |Yes, and i said half a decade ago that it was ill-designed |and the .Mx user interface is bloated. It deserves to die. | |Would you please delete me from the AUTHORS section in | | https://www.sdaoden.eu/code-mdocmx.html | |That was mere fooling around with immature ideas, and i clearly |said back then that what we ended up with in 2014 was not yet good |enough to consider implementing it, and i said that the time clearly |wasn't ripe to do a proper design. I totally disagree with you. It works wonderful, you get table of contents, inter- as well as intra-document links, freely defined anchors and links, and you get interactivity inside a TTY manual page with patched grotty. The effort is minimal, and only command options (for non-headers) could improve it. Not immature, not ill-designed, nonsense! But i can remove you from the list if you want to. --steffen | |Der Kragenbaer,The moon bear, |der holt sich munter he cheerfully and one by one |einen nach dem anderen runter wa.ks himself off |(By Robert Gernhardt)
Re: openssl.1: Tag command names
Hello. I have not followed this thread but Ingo Schwarze wrote in <20200218153053.ga27...@athene.usta.de>: |Klemens Nanni wrote on Mon, Feb 17, 2020 at 05:19:27PM +0100: | |> I'd like to commit this soon, it allows me to jump to the command I'm |> looking for, e.g. ":tx509" shows me the synopsis right away. |> |> FWIW, some Linux distributions ship with separate manuals, e.g. x509(1SS\ |> L). ... |> Patch was done with a VIM macro by adding a new line after each `.Sh' |> line with the respective name but lowercased, so no typos in the added |> strings. | |That placement triggers an issue that already existed before .Tg arrived. i just want to add that there is still the mdocmx mdoc macro extension available, and is working fine for more than half a decade. I have not ported that to groff 1.22.4, but it is available for groff 1.22.3. It can much more than .Tg, of course, you get cross references and a lot of other things. | troff: openssl.1:204: warning: macro 'Tg' not defined Of course that is with .Mx, too. --steffen | |Der Kragenbaer,The moon bear, |der holt sich munter he cheerfully and one by one |einen nach dem anderen runter wa.ks himself off |(By Robert Gernhardt)
Re: cdio(1): remove CDDB support
Marc Espie wrote in <20191229101350.ga17...@nausicaa.home>: |On Sat, Dec 28, 2019 at 07:48:45PM -0500, Bryan Steele wrote: |> With FreeDB announcing[0] that the service will be shutting down as of |> March 31st of 2020, and the only other alternative (MusicBrainz) already |> having shutdown their freedb/cddb gateway in favour of their own API |> early this year, it likely makes sense to remove support from cdio(1). |> |> CDDB is used to retrieve music CD metadata over the Internet, e.g: |> Artist and Track names. |> |> I left in support for the "cdid" command as it may be useful for |> archival(?) purposes, if not that can go too. |> |> Cc: espie@ as he wrote this code, & maybe he still has music on CDs. | |How about we wait until the service actually shuts down ? | |Who knows, maybe someone else will pick up ? | |It's fairly sad, as this means you would have (again) to annotate mp3s |you make from your own CDs by hand. Yeah! --End of <20191229101350.ga17...@nausicaa.home> --steffen | |Der Kragenbaer,The moon bear, |der holt sich munter he cheerfully and one by one |einen nach dem anderen runter wa.ks himself off |(By Robert Gernhardt)
Re: syscall call-from verification
Theo de Raadt wrote in <91679.1574892...@cvs.openbsd.org>: |Steffen Nurpmeso wrote: |1> Theo de Raadt wrote in <29275.1574888...@cvs.openbsd.org>: |>|The following change only permits system calls from address-ranges |>|in the process which system calls are expected from. |> ... |>|Unfortunately our current go build model hasn't followed solaris/macos |>|approach yet of calling libc stubs, and uses the inappropriate "embed |>|system calls directly" method, so for now we'll need to authorize \ |>|the main |>|program text as well. A comment in exec_elf.c explains this. |>| |>|If go is adapted to call library-based system call stubs on OpenBSD as |> |> May i ask -- does this really mean that the (theoretic) |> possibility of writing a small assembler program which performs |> direct system calls will no longer be possible? |> Whereas i see "static binary: main program does system calls", |> a future change could very well restrict the allowed address range |> some more even then? | |Hopefully once go (and other environments which do the same) are |converted to use libc stubs, yes. Unless your binary is static (which |is PIC/PIE, which is already an ABI hurdle). | |Program to the API rather than the ABI. When we see benefits, we |change the ABI more often than the API. | |I have altered the ABI. Pray I do not alter it further. Ok. It was that cool feeling once you had learned assembler, only you (, the assembler) and the OS. Thanks, and ciao! --steffen | |Der Kragenbaer,The moon bear, |der holt sich munter he cheerfully and one by one |einen nach dem anderen runter wa.ks himself off |(By Robert Gernhardt)
Re: syscall call-from verification
Theo de Raadt wrote in <29275.1574888...@cvs.openbsd.org>: |The following change only permits system calls from address-ranges |in the process which system calls are expected from. ... |Unfortunately our current go build model hasn't followed solaris/macos |approach yet of calling libc stubs, and uses the inappropriate "embed |system calls directly" method, so for now we'll need to authorize the main |program text as well. A comment in exec_elf.c explains this. | |If go is adapted to call library-based system call stubs on OpenBSD as May i ask -- does this really mean that the (theoretic) possibility of writing a small assembler program which performs direct system calls will no longer be possible? Whereas i see "static binary: main program does system calls", a future change could very well restrict the allowed address range some more even then? --steffen | |Der Kragenbaer,The moon bear, |der holt sich munter he cheerfully and one by one |einen nach dem anderen runter wa.ks himself off |(By Robert Gernhardt)
Re: sysupgrade(8) and http_proxy
trondd wrote in <20191105013833.cy2_d%tro...@kagu-tsuchi.com>: |Steffen Nurpmeso wrote: |> trondd wrote in <49f29107642e86c17283b0582a9f09f4.squir...@mail.kagu-tsu\ |> chi.com>: |>|On Sun, November 3, 2019 12:02 pm, trondd wrote: |>|> On Sun, November 3, 2019 6:27 am, Florian Obser wrote: |>|>> On Sun, Nov 03, 2019 at 12:21:59PM +0100, Antoine Jacoutot wrote: |>|>>> On Sun, Nov 03, 2019 at 12:16:56PM +0100, Florian Obser wrote: |> ... |>|I've tested the diff and it works as expected in my environment. I don't |>|need a username and password for proxy access but it populates the |>|rc.firsttime file fine. |>| |>|The quote() function is actually pretty simple. ... |> I felt a bit undecided from your first mail on, maybe also because |> of your mailer, but wanted to post the all-compatible all-shell |> quote of and from Robert Elz here. ... |> Of course, if in install.sub there is already your quote function, |> that surely is preferred. ... |To clarify, it's not my quote function. It's been in install.sub since |2010 [0] and was used when http_proxy was added there [1]. I figured it |was a good idea so pulled it in for sysupgrade as well. | |[0] http://cvsweb.openbsd.org/src/distrib/miniroot/install.sub?rev=1.628\ |&content-type=text/x-cvsweb-markup |[1] http://cvsweb.openbsd.org/src/distrib/miniroot/install.sub?rev=1.105\ |8&content-type=text/x-cvsweb-markup Ok! --steffen | |Der Kragenbaer,The moon bear, |der holt sich munter he cheerfully and one by one |einen nach dem anderen runter wa.ks himself off |(By Robert Gernhardt)
Re: sysupgrade(8) and http_proxy
trondd wrote in <49f29107642e86c17283b0582a9f09f4.squir...@mail.kagu-tsu\
chi.com>:
|On Sun, November 3, 2019 12:02 pm, trondd wrote:
|> On Sun, November 3, 2019 6:27 am, Florian Obser wrote:
|>> On Sun, Nov 03, 2019 at 12:21:59PM +0100, Antoine Jacoutot wrote:
|>>> On Sun, Nov 03, 2019 at 12:16:56PM +0100, Florian Obser wrote:
...
|I've tested the diff and it works as expected in my environment. I don't
|need a username and password for proxy access but it populates the
|rc.firsttime file fine.
|
|The quote() function is actually pretty simple.
|
|quote() (
|# Since this is a subshell we won't pollute the calling namespace.
|for _a; do
|# alias string to Q, does escaping and quoting
|alias Q=$_a;
|# set variable back to value of alias
|_a=$(alias Q);
|# print variable, chopping off alias definition
|# no newline, don't substitute the escape sequences
|# we made above
|print -rn -- " ${_a#Q=}"
|done | sed '1s/ //'
|echo
|)
I felt a bit undecided from your first mail on, maybe also because
of your mailer, but wanted to post the all-compatible all-shell
quote of and from Robert Elz here.
#@ Round trip quote strings in POSIX shell. E.g.,
#@set -- x 'a \ b' "foo'" "\\'b\\a\\r\\" Aä
#@printf "%s: <%s><%s><%s><%s><%s>\n" "$#" "${1}" "${2}" "${3}" "$4" "$5"
#@saved_parameters=`quote_rndtrip "$@"`
#@eval "set -- $saved_parameters"
#@printf "%s: <%s><%s><%s><%s><%s>\n" "$#" "${1}" "${2}" "${3}" "$4" "$5"
#
# 2017 Robert Elz (kre).
...
# Though slower use a subshell version instead of properly restoring $IFS
# and flags, as elder shells may not be able to properly restore flags via
# "set +o" as later standardized in POSIX, and it seems overkill to handle
# all possible forms of output "set +o" may or may not actually generate.
quote__rndtrip() (
case "$1" in
*\'*) ;;
*) printf "'%s'" "$1"; return 0;;
esac
a="$1" s= e=
while case "$a" in
\'*) a=${a#?}; s="${s}'";;
*\') a=${a%?}; e="${e}'";;
'') printf "${s}${e}"; exit 0;;
*) false;;
esac
do
continue
done
IFS=\'
set -f
set -- $a
r="${1}"
shift
for a
do
r="${r}'\\''${a}"
done
printf "${s}'%s'${e}" "${r}"
exit 0
)
quote_rndtrip() (
j=
for i
do
[ -n "$j" ] && printf ' '
j=' '
quote__rndtrip "$i"
done
)
quote_string() (
j=
for i
do
[ -n "$j" ] && printf '\\ '
j=' '
quote__rndtrip "$i"
done
)
|$ export "test=fancy ' stuff #and not a comment"
|$ ./quote.ksh
|$ cat test.out
|
|export 'http_proxy=fancy '\'' stuff #and not a comment'
|
|$ export "test=even
|> this works #"
|$ ./quote.ksh
|$ cat test.out
|
|export 'http_proxy=even
|this works #'
Of course, if in install.sub there is already your quote function,
that surely is preferred.
Good night,
--steffen
|
|Der Kragenbaer,The moon bear,
|der holt sich munter he cheerfully and one by one
|einen nach dem anderen runter wa.ks himself off
|(By Robert Gernhardt)
Re: Thermal zone support for arm64
Mark Kettenis wrote in <543892945e900...@bloch.sibelius.xs4all.nl>: |Many of the cheap arm64 (and armv7) boards will overheat if you run |the CPU cores at full throttle for a while. Adding a heatsink may |help a little bit, but not enough. Some boards have a microcontroller |that monitors the temperature and throttles the CPUs if necessary. |Other boards don't and will eventually hit a critical temperature |where it will either do an emergency powerdown or will start to become |unreliable. | |In order to prevent this, the OS is supposed to monitor the |temperature and cool the device (either actively or passively) when |the temperature gets too high. There are device tree bindings for |so-called thermal zones that link together temperature sensors and |cooling devices and define trip points that define the temperatures at |which we have to start cooling. Most boards use passive cooling |through reducing the CPU clock speed and voltage. This is very interesting. These dense (x86-64) packages heat up immensely and very fast, even with hyperthreading turned off (though overheating no longer occurs like so). I hooked into a Linux bug report a few months back, my new box (i jumped a decade of hardware development in March/April) just did/does not keep up, ending in massive fan power stepping in. I asked why no adaptive strategy is used instead, and mentioned a fan control shell script i have, as simple as it is, keeping a notion of the last and the current level, which makes for a difference, and the heat up/cool down trend, which makes for a distance. Like that a simple adaption can take place. Well, in the meantime LWN reported work on the scheduler, so that hot CPUs are scheduled less work than others, which is of course an approach way more sophisticated. --steffen | |Der Kragenbaer,The moon bear, |der holt sich munter he cheerfully and one by one |einen nach dem anderen runter wa.ks himself off |(By Robert Gernhardt)
Re: regress/bgpd: allow specifying daemon binary
Theo de Raadt wrote in <87985.1537307...@cvs.openbsd.org>: |I honestly think this is a foolishly complicated. Maybe for OpenBSD only software. But i think it is worth the hazzle whenever affordable for certain types of software. |Just install the program, then run regress. Install an older version |without the broken changes if it doesn't work. | |I tire of these interactions between environment variables, |base build methods, fork+exec paths in privsep programs, and now |getting tied into regress tests. Ok. Yes that i understand. |In a word, YUCK. But, this is a makefile. And also one which is most often run in the magic environment of BSD make system. It could even fail to run as necessary by using prechecks, or restart itself via "env -i" otherwise. (You know all this of course.) |I think this isn't "convenience". Rather it comes off as artifically |complicated, trying to solve a problem which doesn't need to be exist |at all. Perhaps even perceiving there to be a problem which needs |solving via such abstration is the true problem. This is why i step in: i have found it valuable more than once. For example i (this is me you know) was able to interest the busybox maintain for an awk bug, he even downloaded reproducers; like so (i trim this, from a bugzilla page, issue 10596): You can use the mdocmx.1 file from the same URL [1] mdocmx.sh is at, it is the smallest (3010 bytes) i have that can be processed by the script: ?0[steffen@essex]$ AWK=nawk ./mdocmx.sh
Re: regarding OpenSSL License change
"Michael W. Lucas" wrote: |On Fri, Mar 24, 2017 at 02:37:58PM +0100, Sebastian Benoit wrote: |> It's about "You cannot change the licence without consent of the \ |> author" and |> "We just assume that you say yes to this because we dont care about your |> rights", which is morally and legally wrong. | |It's very simple. Four words. | |"Silence is not consent." | |Not in contracts. Not in sex. And not in licensing. You can say this word. This is funny now, .. that you say this. No no, no. I fail to respond to that that is to say. --steffen
Re: regarding OpenSSL License change
Sebastian Benoit wrote: |Steffen Nurpmeso(stef...@sdaoden.eu) on 2017.03.24 14:03:45 +0100: |> Bob Beck wrote: ... |> According to [1] the chosen license is however the "best" academic |> license, and the only one which allows patent protection. Best in |> sofar as all tested items are green. The Mozilla license was |> surely not possible? |> |> [1] http://www.osscc.net/en/licenses.html#compatibility ... |>|thats really not cool |> |> As far as i understand it, using the Apache license gives more |> protection to end users than the current license does, at least if |> patents get involved. |> |> .. |>|> Apparently lawyers are being paid to help them push this through. Is |>|> that being paid for by donations people gave after Heartbleed? Is |>|> this why people donated? |> |> The license is even better for end-users as the current license? | |But it's not about "this licence is better than that licence". Of course it is, even not being personally involved looking at the file headers it would be a wonderful cleanup if this jungle could be replaced with a single copyright header. |The code has a licence and they dont respect that. |It's about "You cannot change the licence without consent of the author" \ Like it is stated in the file header. |and |"We just assume that you say yes to this because we dont care about your |rights", which is morally and legally wrong. That is, the way you say it, absurd. Morally wrong is, with 58 percent loss of life since 1971, to fly 4 kilometres for three days of hacking or a week of holiday including soiling of historic sites and stealing towels and anything else which fits into the suitcase from the hotel. Buying a new car or a new phone so-and-so often, because of the same reason. Or eating meat more than once a week, or at all in fast food restaurants, at least if you live in Germany, like you do?, because this is why the rainforests die, and the animals live under terrible conditions, without sun light, without any space for living, and without that word that cannot be used on an american list, but anyway cows will never feel the ton of a hot steaming bull body but instead the plastic glove of a Volkswagen driver, up to the shoulder. But even if you don't care about the animals, it is still morally wrong because we first world people no longer eat ears, heads, feets, and all that is shipped for a ridiculous amount of money to Africa, were thousand year old traditions die since decades due to that, because Farmers cannot afford this price, and if they do they soil the acres, and if they don't they leave their land and go to the cities, where they need more water than the land can offer, and so you loose-loose and the deserts grow further, and this goes on since decades. And not talking at all about the growing resistance of bacteria for antibiotics, also since decades. Or having never cared for details but going on like a zombie and voting the next demagogue that comes along and promises whatever. Or, worse, even doing this on purpose because the human heart never gets enough. So this and much more is morally wrong, but asking all contributors for a license change, a free license that seems to be the "best license" for freedom, as has been verified, for the massively and growing more massively still material world, where some money-backed lawyers could enforce a shutdown of services if some patent would be violated, for example, the word "morally wrong" should be carefully chosen in my opinion. I also sometimes have the impression that OpenSSL has become a heavy truck that blindly rolls over the little flowers, though. On the other hand i have received even two messages for different addresses for contributions so marginal that it is almost laughable that someone asks me at all. The thing is, if i, with these contributions, would really be allowed to veto the entire switchover, then the world will stand still, because there are, in fact, many little pissers all around us. And this as an European. I for one think like this, but of course other contributions are of much more value than mine, and if there would be a "no" from such a contributor, things may or even will be different. --steffen
Re: regarding OpenSSL License change
Bob Beck wrote: ... Disclaimer: i have read about licenses many years ago (likely over a decade, i stopped reading the german computer magazine c't somewhen in 2005). I like and use the ISC license that your project has chosen and fosters whenever i can. According to [1] the chosen license is however the "best" academic license, and the only one which allows patent protection. Best in sofar as all tested items are green. The Mozilla license was surely not possible? [1] http://www.osscc.net/en/licenses.html#compatibility Interesting to me is that this is the third time this year that this topic comes up, in January i had a private communication with Jörg Schilling (who provided this link, again), i think a month ago there was a thread on the Austrian Linux User list, and now we have this one. ... |thats really not cool As far as i understand it, using the Apache license gives more protection to end users than the current license does, at least if patents get involved. .. |> Apparently lawyers are being paid to help them push this through. Is |> that being paid for by donations people gave after Heartbleed? Is |> this why people donated? The license is even better for end-users as the current license? --steffen
Re: UTF-8 support for colrm(1)
Stephane Chazelas wrote:
|2016-01-11 16:10:28 +0100, Steffen Nurpmeso:
|[...]
|> And your diff includes
|>
|> +.Pp
|> +For compatibility with
|> +.St -p1003.1-2008
|> +.Xr fold 1 ,
|> +if a double-width character is followed by two backspace characters
|> +instead of the usual one, both are regarded as belonging to that
|> +character, and the second one does not decrement the column count.
|>
|> Have you actually ever encountered anything that behaves like this
|> in canonical mode? I have not, except that all tested terminals
|> (a very restricted set, Thomas Dickey and Marc Lehmann, to name
|> a few, would know much better than myself) do so in non-canonical
|> mode. And that is weird enough given that they delete the glyph
|> that makes up the double-width column and, in order to achieve
|> that, place a space before the cursor.
|>
|> But i think POSIX text utilities behave in canonical mode, because
|> beforehand the quote from above we read
|>
|> Although terminal input in canonical processing mode requires
|> the erase character (frequently set to ) to erase the
|> previous character (not byte or column position), terminal
|> output is not buffered and is extremely difficult, if not
|> impossible, to parse correctly; the interpretation depends
|> entirely on the physical device that actually
|> displays/prints/stores the output.
|>
|> So if this is true then i think this is even worth a POSIX issue?
|I'm under the impression you're mixing two things, the \b
|processing on input and on output. I don't think the input
|processing matters as far as colrm is concerned.
It seems to me you are right.
|$ printf '|\uFF21\b\b|\n'
|||
|$ printf '|\uFF21\b|\n'
|||
|
|in both xterm and gnome-terminal. As in, you need two backspace
|characters to delete that character. With only one, the cursor
|moves back one column, and if you write another character, the
|double-width glyph is erased (leaving an empty single-width
|space and your replacement character).
|
|So it's right that colrm should assume that
|\b\b doesn't change the cursor
|position.
Oh i want to make clear that i never had any doubt Ingo looked
into this thoroughly.
|The terminal device line-discipline (so in the kernel), in
|canonical mode, when you *type* backspace/delete after *typing*
|that double-width A, at least on Linux echos back only one \b
|("\b \b", not "\b\b \b\b"), which means it doesn't erase that
|double-width A properly.
|
|I don't think that issue can be solved, as \b is the sequence to
|move the cursor to the left by one column. So one has to issue
|\b\b to move the cursor over a double-width character. Linux (at
|least) has a IUTF8 termios setting (stty iutf8) to tell the line
|discipline that the charset is UTF-8, so for instance when you
|type é, it erases the é instead of its last byte from
|its internal buffer (and echoes "\b \b"), but it doesn't know
|the width of each character as would be displayed by the
|terminal, and that is almost impossible to achieve at the line
|discipline level.
|
|Of relevance:
|
|https://unix.stackexchange.com/questions/245013/get-the-display-width\
|-of-a-string-of-characters
|http://eev.ee/blog/2015/09/12/dark-corners-of-unicode/#combining-char\
|acters-and-character-width
Interesting pointers, but especially the first with the following
discussion, thanks! In the shell, call-through hooks like
"posix.wcwidth STRING", "posix.wcswidth STRING" or similar would
be a good thing to have, and it still seems to me that UNIX/POSIX
has a lot of room improving the situation regarding real native
language support. (e.g., Perl is so much more sophisticated in
this area).
Nonetheless.. Sorry for the noise!
--steffen
Re: UTF-8 support for colrm(1)
Hallo Ingo, i CC: the POSIX list since that is an interesting conclusion of yours! Ingo Schwarze wrote: |Ingo Schwarze wrote on Wed, Dec 23, 2015 at 07:44:05PM +0100: |>>> For example, colrm(1). |> So, remember this rule: |> |> ++ |>| Backspace removes the previous character, no matter its width. | |> ++++++ ... POSIX says (p. 2773, l. 90953 ff.) In all known internationalized implementations, the utilities producing output for mixed column-width output assume that a character backs up one column position and outputs enough characters to return to the start of the character when is used to provide local line motions to support underlining and emboldening operations. And your diff includes +.Pp +For compatibility with +.St -p1003.1-2008 +.Xr fold 1 , +if a double-width character is followed by two backspace characters +instead of the usual one, both are regarded as belonging to that +character, and the second one does not decrement the column count. Have you actually ever encountered anything that behaves like this in canonical mode? I have not, except that all tested terminals (a very restricted set, Thomas Dickey and Marc Lehmann, to name a few, would know much better than myself) do so in non-canonical mode. And that is weird enough given that they delete the glyph that makes up the double-width column and, in order to achieve that, place a space before the cursor. But i think POSIX text utilities behave in canonical mode, because beforehand the quote from above we read Although terminal input in canonical processing mode requires the erase character (frequently set to ) to erase the previous character (not byte or column position), terminal output is not buffered and is extremely difficult, if not impossible, to parse correctly; the interpretation depends entirely on the physical device that actually displays/prints/stores the output. So if this is true then i think this is even worth a POSIX issue? I repeat that i have not yet encountered any utility which behaves the way that POSIX describes and Ingo tries to address with special processing? --steffen
Re: UTF-8 support for colrm(1)
Ingo Schwarze wrote: |Steffen Nurpmeso wrote on Wed, Dec 23, 2015 at 11:45:36AM +0100: |> Ingo Schwarze wrote: |>> For example, colrm(1). |>> |>> 4. The backspace character (U+0008) backs up by one display position |>>rather than by one character. That causes miscounting when |>>backspace follows a zero-width or double-width character. | |> this however is unfortunately common behaviour for terminals, too. |So, remember this rule: | | ++ | | Backspace removes the previous character, no matter its width. | | ++++++ | |||| | |||| __ .d$$b .' TO$;\ / : TP._; / _.; :Tb| / / ;j$j _.-" d .' .. d; / /P' dP. |\ / " .d$$$P' |\^"l .' `T$P^""""" : ._.' _.'; `-.-".-'-' ._. _.-".-" `.-" _ ._ .-" -(.g$$$b. .' ""^^T$$$P^).(: _/ -" /.' /:/; ._.'-'`-' ")/ /;/; `-.-"..--"" " / / ; .-" ..--""-' : ..--""--.-" (\ .-(\ ..--"" `-\(\/;` _. : ;`- :\ ; bug --steffen
Re: UTF-8 support for colrm(1)
Hello Ingo, Ingo Schwarze wrote: |For example, colrm(1). |4. The backspace character (U+0008) backs up by one display position | rather than by one character. That causes miscounting when | backspace follows a zero-width or double-width character. this however is unfortunately common behaviour for terminals, too. I explicitly misused this misfeature in my N(ail)C(ommand)L(ine editor) commented as * We do not handle character widths because the terminal must deal with that * anyway on the one hand, and also wcwidth(3) doesn't support zero-width * characters by definition on the other. We're addicted. yet noone ever cared. I.e., you see the cursor advancing by two spaces when a double-width glyph is inserted (e.g., xterm, nsterm) but backspace and cub1/le will loose that knowledge! Especially mysterious when then deleting at that position since it'll remove the double-width character and place a space before the cursor! For v14.9 of my thing i'm in the process of warping "NCL" to a new "NLE", the commit message of which will say Use termcap.c abstraction for movement and other such terminal capability related operations instead of unrolling anything ourselfs. As a part of this, honour wcwidth(3): i will never understand why terminals support double-width character insertion and move the cursor accordingly even in non-canonical mode, but fail to deal with that with successive cursor movements or \b. But it is like that so NLE has to deal with it. Finally do. I.e., BS==wcwidth(3)*BS/cub1/le for NLE. But what i'm trying to say is that it seems that for plain \b the ship has sailed a long time ago. --steffen
Re: Flag to set from address in mail(1)
"Todd C. Miller" wrote: |On Wed, 07 Jan 2015 12:11:40 +0100, Steffen Nurpmeso wrote: |> I seem to recall that OpenBSD dropped -f in December (i don't know |> why), but clashing a POSIX argument doesn't seem to be a good |> idea. Heirloom mailx and S-nail use the -r option for the purpose |> in question, and i think it would be nice if the BSD Mails would |> agree in this respect. | |Mail still supports -f mboxname. The support for passing arbitrary |sendmail options at the end was dropped for security reasons. My Ok, then i had a false impression back in December. I.e., it was a bit helter-skelter for me and i would have preferred the necessity to reverse the burden of proof for *expandaddr* (a completely braindead name for what happens by the way, aliases get expanded regardless of its setting), so that one could have used set noexpandaddr or set noexpandaddr=fail as a homogenous augmentation of check strictness. (And additionally =restrict and =restrict-fail for S-nail.) Without that reversal it is inhomogenous: not set means it is disallowed (changing a behaviour that existed for over two decades [what i think without really knowing]), but for further restrictions to be applicable one either had to invent yet another variable or add arguments to reverse the meaning of the set variable again: "expandaddr=fail" is odd. That is bad. I didn't implement command line argument suppression, since we require a separating "--" terminator and i still think that who passes through something like this 1:1 is on the completely false road regarding CGI programming. But i'm still learning and i see that it would be nice to be able to disallow such additional arguments on the command line. I think i have to implement an *expandargv* for that -- and again with that "reversed" meaning. Shit. |plan all along has been to use -r like nail when I get a chance. Great, real thanks for that. (S-nail passes the -r argument through the address "parser" and uses the result for sendmail(1)s -f, support for -F will require quite some work but can come from the same -r argument, just as given. But still.) |At this point I'm leaning towards using "sendmail -t -oi" and just |putting the From: line in directly instead of relying on sendmail/smtpctl |to add it for us. We do support message resending and i'm very unsure on how portable sendmail(1)s -t is in respect to resend messages, so i think that route is not possible for us. Thanks and ciao, --steffen
Re: Flag to set from address in mail(1)
Hello, tro...@gmail.com wrote: |>Sorry, my fault. Try the diff below. |> |>Nathanael | |Yup, this works! Now I can filter by sender in smptd to use the right |SMTP server. sorry for jumping in like that (i only get these message via a simple NNTP-to-MBOX fetcher), yet i want to point out that -F is taken by POSIX [1]: -f [UP] [Option Start] Read messages from the file named by the file operand instead of the system mailbox. (See also folder.) If no file operand is specified, read messages from mbox instead of the system mailbox. [Option End] -F [UP] [Option Start] Record the message in a file named after the first recipient. The name is the login-name portion of the address found first on the To: line in the mail header. Overrides the record variable, if set (see Internal Variables in mailx). [Option End] [1] http://pubs.opengroup.org/onlinepubs/9699919799/utilities/mailx.html I seem to recall that OpenBSD dropped -f in December (i don't know why), but clashing a POSIX argument doesn't seem to be a good idea. Heirloom mailx and S-nail use the -r option for the purpose in question, and i think it would be nice if the BSD Mails would agree in this respect. Thanks and Ciao, --steffen
