Re: clarification needed: shell 'exec' + function (builtin, ???)
[I bring back austin-group-l, ok? Thorsten Glaser wrote in : |Steffen Nurpmeso dixit: | |> #include |> #include |> #include |> #include |> int main(void){ |> char inb[16], oub[16], *inbp, *oubp; |> iconv_t id; |> size_t inl, oul; |> |> memcpy(inbp = inb, "a\303\244c", sizeof("a\303\244c")); |> inl = sizeof("a\303\244c") -1; | |Not -1 otherwise oub will not be NUL-terminated and end with junk: | |$ ./a.out |Converting 4 |GOT Sure thing. Just like below. Normally stack pages are cow forked from zero if i understand that right. But maybe i do not. |Without the trailing NUL, stateful conversation may also be |incomplete… | |> oul = sizeof oub; |> oubp = oub; |> |> if((id = iconv_open("ascii", "utf8")) == (iconv_t)-1) |> return 1; | |Throws 1 because you need "utf-8", but with it, see above. Well names and iconv are a thing. Especially regarding Unicode (and nl_langinfo(CODESET), if i remember UnixWare right). |> fprintf(stderr, "Converting %lu <%s>\n",(unsigned long)inl, inbp); |> if(iconv(id, , , , ) == (size_t)-1){ |>fprintf(stderr, "Fail <%s>\n", strerror(errno)); |>return 2; |>} |> fprintf(stderr, "GOT <%s>\n", oub); |> iconv_close(id); |> return 0; |>} |> |>you should get replacement characters out of the box? | |Citrus iconv agrees. Its manpage says: | | If the string pointed to by *src contains a character which is valid | under the source codeset but can not be converted to the destination | codeset, the character is replaced by an "invalid character" which | depends on the destination codeset, e.g., '?', and the conversion \ | is con- | tinued. iconv() returns the number of such "invalid conversions". That was my thinking. Thanks for confirming this. --End of --steffen | |Der Kragenbaer,The moon bear, |der holt sich munter he cheerfully and one by one |einen nach dem anderen runter wa.ks himself off |(By Robert Gernhardt)
Re: mail encoding not-fun (was Re: clarification needed: shell 'exec' + function (builtin, ???))
Thorsten Glaser via austin-group-l at The Open Group wrote in : |Steffen Nurpmeso via austin-group-l at The Open Group dixit: | |>|This is because m4.opengroup.org runs qmail, the arsehole under the MTAs, |>|which auto-converted the mail from quoted-printable to 8bit, sending it |>|as 8bit even to MTAs that don't offer 8BITMIME (I configured my sendmail |>|not to do that as well, so I got the same truncated mail back :( other |>|than qmail, exim is known to break the MIME and SMTP standards like \ |>|that). |> |>Naaah, not true Thorsten. At least this time. | |This one *is* correct, as I got the broken message back as well. |It contains an embedded NUL. | |But apparently, this was not the cause of J�rg’s problem ☻ Evil, you. Hey, i live also on IRC since ~1.5 years for the first time ever, and on this Linux-Distro (just released 3.6 two days ago) here is someone active from Düsseldorf, and i now sometimes listen to La Düsseldorf from La Düsseldorf. (Of course Neu! and Lilo Engel are less populist.) Lots of longing for the 70s here. Tja. All he would need to do would be to upgrade to a newer version, i think i innocently prod him two times on that. |>Related to my MUA. |[…] |>I have been able to save the mail as file and to run iconv(1) on the \ |>content. | |oic | |>Maybe a problem is that the first missing line is a line with a character \ |>that |>is not part of ISO-8859-1 | |Yes, of course, I have been writing in UTF-8 for a while. Not so here, even though i have ~/.kent.xmodmaprc keycode adjustments for some German and French quotation marks, i find it hard to go "random Unicode". I usually use altgr/g-a in vim and the look in UnicodeData.txt to find the codepoint ;) |[00:02] gecko: benutzt du emacs ? |[00:03] nö [00:03] nur n normalen mac Graphical selection may be a winner here, indeed. |[00:04] argl [00:04] ne den editor | -- Vutral und gecko2 in #deutsch (NB: Editor? Betriebssystem.) --End of --steffen | |Der Kragenbaer,The moon bear, |der holt sich munter he cheerfully and one by one |einen nach dem anderen runter wa.ks himself off |(By Robert Gernhardt)
Re: clarification needed: shell 'exec' + function (builtin, ???)
Hallo Jörg, all, Joerg Schilling wrote in <20201210004945.i3n8e%sch...@schily.net>: |Steffen Nurpmeso wrote: |> this is an iconv(3)-related error that was fixed in later version |> of the mailer you use. The very error came up on the ML this |> year[1], basically you use LATIN1 on your box, as could be |> expected, but Thorsten is known to be a Unicode character |> "junkie", so to say. | |You are correct, Yep -- unfortunately. |I have been able to save the mail as file and to run iconv(1) on the \ |content. Yes, we temporarily did not restart for ILSEQ, if your prompt would include "set prompt='\${^ERRNAME}', for example, you would have seen that an error happened. But of course we are tolerant for weird base64, so we should be tolerant for weird iconv, thus i "restored the original behavirour", so to say. That reminds me of iconv weirdness regarding hard-to-test replacement characters, which makes testing really hard. Wasn't there an issue on that going on, being able to specify it explicitly, and whether it stands for an entire character or for by-byte sequences would be a great improvement. While talking about iconv, i got closed glibc bug[1] as "resolved invalid", but wouldn't you all agree that in the following #include #include #include #include int main(void){ char inb[16], oub[16], *inbp, *oubp; iconv_t id; size_t inl, oul; memcpy(inbp = inb, "a\303\244c", sizeof("a\303\244c")); inl = sizeof("a\303\244c") -1; oul = sizeof oub; oubp = oub; if((id = iconv_open("ascii", "utf8")) == (iconv_t)-1) return 1; fprintf(stderr, "Converting %lu <%s>\n",(unsigned long)inl, inbp); if(iconv(id, , , , ) == (size_t)-1){ fprintf(stderr, "Fail <%s>\n", strerror(errno)); return 2; } fprintf(stderr, "GOT <%s>\n", oub); iconv_close(id); return 0; } you should get replacement characters out of the box? I said by then $ /tmp/zt Converting 4 Fail whereas musl gives $ ./zt Converting 4 GOT and i still think musl is totally right (also by giving only one replacement character. [1] https://sourceware.org/bugzilla/show_bug.cgi?id=22908 |Maybe a problem is that the first missing line is a line with a character \ |that |is not part of ISO-8859-1 Yes, transliteration should possibly be possible. On the other hand, if i change the above to if((id = iconv_open("ascii//TRANSLIT", "utf8")) == (iconv_t)-1) i get Converting 4 GOT and with if((id = iconv_open("ascii//TRANSLIT", "utf8")) == (iconv_t)-1) we are back at the error. Ciao, --steffen | |Der Kragenbaer,The moon bear, |der holt sich munter he cheerfully and one by one |einen nach dem anderen runter wa.ks himself off |(By Robert Gernhardt)
Re: clarification needed: shell 'exec' + function (builtin, ???)
Hallo Jörg, Joerg Schilling wrote in <2020120933.yyo5w%sch...@schily.net>: |"shwaresyst via austin-group-l at The Open Group" wrote: ... |> Hi *, | |Hi, | |here is where the original mail ended for me. Interesting that you did get |more content. Is there any idea, why I received only the first line \ |from the |original mail? this is an iconv(3)-related error that was fixed in later version of the mailer you use. The very error came up on the ML this year[1], basically you use LATIN1 on your box, as could be expected, but Thorsten is known to be a Unicode character "junkie", so to say. commit a9ec20d6 Author: Steffen Nurpmeso AuthorDate: 2020-04-23 17:29:57 +0200 Fix: "revert" [ab0cd3b8] from 2017-10-20.. (Claus Assmann).. (FIX iconv for main body part (since EVER!) (Doug McIlroy, Random832)..) was nonsense in sofar as we now generated ILSEQ errors, but without giving any error message, so that users (without nice *prompt*) normally had no indication of what happened, but looked at a partial message. This is inacceptible, so instead simply use replacement until we possibly have a better way out at some later time. This changeset is in v14.9.19. [1] https://www.mail-archive.com/s-mailx@lists.sdaoden.eu/msg01013.html P.S.: i will release v14.9.20 before Christian Christmas, and that should just work neatly on SchilliX, too, including readily prepared catman manual (to be downloaded as an extra). --steffen | |Der Kragenbaer,The moon bear, |der holt sich munter he cheerfully and one by one |einen nach dem anderen runter wa.ks himself off |(By Robert Gernhardt)
Re: clarification needed: shell 'exec' + function (builtin, ???)
austin-grou...@opengroup.org wrote in : |Joerg Schilling via austin-group-l at The Open Group dixit: | |>here is where the original mail ended for me. Interesting that you did get | |This is because m4.opengroup.org runs qmail, the arsehole under the MTAs, |which auto-converted the mail from quoted-printable to 8bit, sending it |as 8bit even to MTAs that don't offer 8BITMIME (I configured my sendmail |not to do that as well, so I got the same truncated mail back :( other |than qmail, exim is known to break the MIME and SMTP standards like that). Naaah, not true Thorsten. At least this time. Related to my MUA. --steffen | |Der Kragenbaer,The moon bear, |der holt sich munter he cheerfully and one by one |einen nach dem anderen runter wa.ks himself off |(By Robert Gernhardt)
Re: An additional quote test for check.t
Thorsten Glaserwrote: |>i think, the \r is expanded, the output is | |a bit unsure what exactly you’re talking about, but you cannot |ever use echo to display strings portably. | |Use 'print -r -- "$foo"' to display the contents of the foo parameter. | |Is there anything you think mksh fails to do correctly? If so, |please indicate _exactly_ where and what and how. Oh yes, cool, ehm, that is right. With printf and arguments all is fine indeed. Great. Enough shame for the weekend.. Sorry. Ciao. --steffen | |Ralph says i must not use signatures which spread the light!
An additional quote test for check.t
Hallo Thorsten, I do not know whether you have followed the thread on the standard list, but Robert Elz has written a shell function which allows round-tripping quoted strings from and to the positional parameter stack, Stephane did so in awk(1), effectively we are talking about stephane_quote() { LC_ALL=C awk -v q="'" -v b='\\' ' function quote(s){ gsub(q, q b q q, s) return q s q } BEGIN{ sep = "" for(i = 1; i < ARGC; i++){ printf "%s", sep quote(ARGV[i]) sep = " " } if(sep) print "" }' "$@" } mksh fails to roundtrip the string set -- x 'a \ b' "foo'" "\\'b\\a\\r\\" A echo "$#: <${1}><${2}><${3}><$4><$5>" saved_parameters=$(robert_quote "$@") eval "set -- $saved_parameters" echo "$#: <${1}><${2}><${3}><$4><$5>" i think, the \r is expanded, the output is \><\'b instead of 5: <\'b\a\r\> Interested in the attached addition to check.t? --steffen | |Ralph says i must not use signatures which spread the light! diff --git a/check.t b/check.t index e0c8d5e..7cc08d6 100644 --- a/check.t +++ b/check.t @@ -6835,6 +6835,61 @@ expected-stdout: 3 10 . 4 -2147483646 . --- +name: quote-roundtrip +description: + Check roundtrip quoting behaviour via Robert Elz quote() +stdin: + robert__quote() { + case "$1" in + *\'*) ;; + *) printf "'%s'" "$1"; return 0;; + esac + __A__="$1" __S__= __E__= + while case "$__A__" in + \'*)__A__=${__A__#?}; __S__="${__S__}'";; + *\')__A__=${__A__%?}; __E__="${__E__}'";; + '') printf "${__S__}${__E__}"; return 0;; + *) false;; + esac + do + continue + done + _save_IFS="${IFS}"; ${IFS+":"} unset _save_IFS + _save_OPTS="$(set +o)" + IFS=\' + set -f + set -- $__A__ + _result_="${1}" + shift + for __A__ + do + _result_="${_result_}'\\''${__A__}" + done + printf "${__S__}'%s'${__E__}" "${_result_}" + # now clean up + IFS="${_save_IFS}"; ${_save_IFS+":"} unset IFS + eval "${_save_OPTS}" + unset _result_ _save_IFS _save_OPTS __A__ __E__ __S__ + return 0; + } + robert_quote() { + j= + for i + do + [ -n "$j" ] && printf ' ' + j=' ' + robert__quote "$i" + done + } + set -- x 'a \ b' "foo'" "\\'b\\a\\r\\" A + echo "$#: <${1}><${2}><${3}><$4><$5>" + saved_parameters=$(robert_quote "$@") + eval "set -- $saved_parameters" + echo "$#: <${1}><${2}><${3}><$4><$5>" +expected-stdout: + 5: <\'b\a\r\> + 5: <\'b\a\r\> +--- name: readonly-0 description: Ensure readonly is honoured for assignments and unset
Re: alias expansion bug with matching { ... }
Hallo! Thorsten Glaserwrote: |Dixi quod… |>Commit ID: 10058E5A1067AFA1BBA ... |Oh well. For the sake of consistency. But fixing this DID uncover |a re-entrancy bug in 'typeset -f' ouput (which IS also used in some |other places internally): ... |When deferring alias interpretation until the end, we do get: ... |This is problematic (in the first two) and inconsistent (in the last) case. ... |To make this work but not get “\./pfn” stupidly, I had to limit the |allowed characters in “alias”, but ksh93 does so too, as does POSIX. Would you mind re-allowing hyphen-minus again? That really has become very common in $PATH, and the new ksh_isalias() seems to omit it. For example, ?0[steffen@wales]$ alias|wc -l 86 ?0[steffen@wales]$ alias|grep -F -- -|wc -l 78 ?0[steffen@wales]$ ls /usr/bin|wc -l 1565 ?0[steffen@wales]$ ls /usr/bin|grep -F -- -|wc -l 372 I seem to remember that i have opened an Austin Group issue to include hyphen-minus in the set of allowed characters, that is [1], … i seem to be too lazy to look into the standard to resolve the aberrations on notational aspects of the desired change that have evolved on the tracker, though. Still, bash etc. support it. In the other window i updated the desired action. [1] http://austingroupbugs.net/view.php?id=1050 Ciao! --steffen
Re: trap recursion until SIGSEGV (non-interactive)
Thorsten Glaser t...@mirbsd.de wrote: |Steffen Nurpmeso dixit: |An interactive shell or '/usr/bin/mksh -c exec ./t.sh' limits |the storm in the same way that bash(1) does |Running 'bash t.sh' shows… | |au |au |au |after zap |au |au |au |au |au [ au] |exit |au | |… weirdly enough, but this is hardly consistent with ..consistency was what i had in mind. |the documentation of the trap builtin. And you cannot |portably catch stack underruns anyway (which is what |happens here, just like if you have a function that |recurses infinitely). I would prefer seeing a stack recursion limit exceeded or similar before SIGSEGV happens because of recursive functions, but it is possibly no good if mksh would be different. --steffen
Re: trap recursion until SIGSEGV (non-interactive)
Thorsten Glaser t...@mirbsd.de wrote: |Steffen Nurpmeso dixit: |I would prefer seeing a stack recursion limit exceeded or |similar before SIGSEGV happens because of recursive functions, | |Yeah, but I have no idea how to do that. If you find one… tell me. Anyway i'm thankful i was worth an answer for that -- current bash (i usually only have a stale 3.2) shows the same behaviour and after searching the web a person that ran into the same problem with bash didn't get one! And i would have said the same as someone else i saw in a different, even older message: use a recursion counter, how wacky that may be. But i guess it is not worth any effort, especially given that No. 1 changed _to_ this behaviour (even completely, exec and even interactive [but via subshell]!) and that i saw it the first time because of a bug. It's just that i am with the one that got no answer in that it didn't feel right. Just forget about it. --steffen
trap recursion until SIGSEGV (non-interactive)
Hello Thorsten, i wrote a buggy shell script similar to the crap below, but it crashed mksh(1): cat t.sh _EOT trap echo au; kill $$ TERM trap echo exit EXIT zap=`kill -TERM $$` echo after zap exit _EOT An interactive shell or '/usr/bin/mksh -c exec ./t.sh' limits the storm in the same way that bash(1) does, but running the script directly will loop until crash. (dash(1)s HEAD behaves the very same btw.) --steffen
Bum behaviour of hash(1)
Hello, just stumbled over a makefile which does RANLIB=(hash ranlib) /dev/null 21 || exit 0; ranlib and had a problem: ?0[]$ for i in ba da k mk z; do echo --${i}sh:; eval ${i}sh -c '(hash nosuch)||echo ok; (hash vim)||echo bad'; done --bash: bash: line 0: hash: nosuch: not found ok --dash: dash: 1: hash: nosuch: not found ok --ksh: --mksh: --zsh: zsh:hash:1: no such command: nosuch ok And also (packed in shell archives, grmpf) ?1[]$ bourne.sh + -c '(hash vim)||echo au' ?0[]$ bourne.sh + -c '(hash vims)||echo au' /var/folders/Da/DaZX3-k5G8a57zw6MSmjJTM/-Tmp-//.bournesh/sh: vims: not found au ?0[]$ schily.sh + -c '(hash vim)||echo au' ?0[]$ schily.sh + -c '(hash vims)||echo au' /var/folders/Da/DaZX3-k5G8a57zw6MSmjJTM/-Tmp-//.schily.sh/sh: vims: not found au Ciao, --steffen
Re: Bum behaviour of hash(1)
Thorsten Glaser t...@mirbsd.de wrote: |Steffen Nurpmeso dixit: | |just stumbled over a makefile which does | |Fix that Makefile. Your advice comes too late. Sorry for that. Ciao, --steffen