Re: Anomaly
On 9/20/16 11:04 PM, PePa wrote: > When sourcing this script (version 1), it will print y after receiving > an interrupt, but not in the 2 different versions (2 and 3). > > # version 1 > echo x > sleep 99 > echo y > > # version 2 > echo x; sleep 99 > echo y > > # version 3 > echo x > sleep 99; echo y > > Is this a bug or expected behaviour?? Well, it's definitely a difference between three simple commands and a command list. I think the right behavior is that exhibited by versions 2 and 3: the interrupt causes the execution of the sourced script to halt. I'll take a look at it. Chet -- ``The lyf so short, the craft so long to lerne.'' - Chaucer ``Ars longa, vita brevis'' - Hippocrates Chet Ramey, UTech, CWRUc...@case.eduhttp://cnswww.cns.cwru.edu/~chet/
Re: "HISTSIZE=999999999" cause bash failure after lastest upgrade
On 9/25/16 3:51 PM, Sean Zha wrote: > Bash Version: 4.4 > Patch Level: 0 > Release Status: release > > Description: > I use a huge value for HISTSIZE (=9) to enable infinite > history items. The actural size of ~/.bash_history is only 4MB now. > Everything worked fine before the lastest upgrade. Now bash refuse > me to login due to memory allocation failure. After choosing > a smaller HISTSIZE, bash still eatup too much unnesssary memory. > > Repeat-By: > open a workable terminal with small HISTSIZE setting, > $ echo HISTSIZE=9 > ~/test.rc > $ bash --rcfile ~/test.rc > bash: xmalloc: cannot allocate 88 bytes (114688 bytes > allocated) Since you've specified the desired history size, bash tries to allocate enough entries to hold all of the entries. The assumption is that this will reduce the number of allocations/reallocations and the number of times the history list has to be copied as it's reallocated. I suppose I should cap the maximum value for which that occurs instead of trusting the supplied max number of entries. Chet -- ``The lyf so short, the craft so long to lerne.'' - Chaucer ``Ars longa, vita brevis'' - Hippocrates Chet Ramey, UTech, CWRUc...@case.eduhttp://cnswww.cns.cwru.edu/~chet/
Re: bash history with mixed timestamps slow and broken
On 9/24/16 2:17 PM, Hubert Schmid wrote: > Configuration Information [Automatically generated, do not change]: > Machine: x86_64 > OS: linux-gnu > Compiler: gcc > Compilation CFLAGS: -DPROGRAM='bash' -DCONF_HOSTTYPE='x86_64' > -DCONF_OSTYPE='linux-gnu' -DCONF_MACHTYPE='x86_64-pc-linux-gnu' > -DCONF_VENDOR='pc' -DLOCALEDIR='/usr/share/locale' -DPACKAGE='bash' -DSHELL > -DHAVE_CONFIG_H -I. -I../. -I.././include -I.././lib -Wdate-time > -D_FORTIFY_SOURCE=2 -g -O2 -fdebug-prefix-map=/build/bash-31ueiH/bash-4.4=. > -fstack-protector-strong -Wformat -Werror=format-security -Wall > -Wno-parentheses -Wno-format-security > uname output: Linux vivo 4.7.0-1-amd64 #1 SMP Debian 4.7.4-2 (2016-09-19) > x86_64 GNU/Linux > Machine Type: x86_64-pc-linux-gnu > > Bash Version: 4.4 > Patch Level: 0 > Release Status: release > > Description: > If the history file (`.bash_history`) starts with a timestamp > (`HIST_TIMESTAMP_START`), and contains lines that have been written > without timestamps, then reading the history file is (a) very slow > because (b) consecutive lines without timestamps are merged into a > single history entry with quadratic complexity. > > Apparently, this problem didn't exist in the previous version (4.3). One of the most frequently-requested features for the bash and readline history implementations is a way to preserve multi-line commands across shell sessions. It's been the subject of numerous previous discussions on bug-bash. There hasn't been a good way to do that, since there isn't good support for it in the traditional readline (flat) history file format. I decided to implement a heuristic: if the history file starts with a timestamp, and the application using the history library -- in this case, bash -- has indicated that it's interested in writing timestamps to the history file (which is off by default), the history file reading code assumes that timestamps appear consistently in the history file and it can use them as markers to delimit commands. The appending behavior isn't really quadratic: the code simply keeps reallocating the line and appending to it. You can imagine how long it takes to append a million commands to a single buffer. You've managed to identify the most degenerate case. -- ``The lyf so short, the craft so long to lerne.'' - Chaucer ``Ars longa, vita brevis'' - Hippocrates Chet Ramey, UTech, CWRUc...@case.eduhttp://cnswww.cns.cwru.edu/~chet/
Re: [BUG] false positive: [ ! -o optname ]
On 9/25/16 5:57 PM, Martijn Dekker wrote: > Op 25-09-16 om 22:40 schreef isabella parakiss: >> On 9/25/16, Martijn Dekker wrote: >>> The '!' operator in the legacy test/[ builtin does not invert the result >>> of the -o operator. Consequently the command >>> >>> [ ! -o noclobber ] >>> >>> amounts to a no-op, always returning exit status 0. > [...] >> [ ! -o noclobber ] means >> is '!' a non empty string? if not, is 'noclobber' a non empty string? > > Wow. Yeah, I suppose that's one possible interpretation. I was going for > the POSIX one: > > | These primaries can be combined with the following operators: > | > | ! expression > | True if expression is false. False if expression is true. > Ref: > http://pubs.opengroup.org/onlinepubs/9699919799/utilities/test.html#tag_20_128 You need to look at the part of the Posix description where the behavior is described as being based on the number of arguments. In particular: 3 arguments: If $2 is a binary primary, perform the binary test of $1 and $3. -a and -o are considered binary primaries for the purposes of a three- argument test. This is spelled out pretty clearly in the bash documentation. Chet -- ``The lyf so short, the craft so long to lerne.'' - Chaucer ``Ars longa, vita brevis'' - Hippocrates Chet Ramey, UTech, CWRUc...@case.eduhttp://cnswww.cns.cwru.edu/~chet/
Re: [BUG] false positive: [ ! -o optname ]
Op 25-09-16 om 22:57 schreef Martijn Dekker: > And indeed your interpretation does not apply to something like > "[ ! -e /tmp ]": > > $ [ -e /tmp ]; echo $? > 0 > $ [ ! -e /tmp ]; echo $? > 1 > > However, the supposed synonym -a acts differently: > > $ [ -a /tmp ]; echo $? > 0 > $ [ ! -a /tmp ]; echo $? > 0 Which also makes sense now that I think about it, since -a means "and" so it tests for the non-emptiness of both strings. Bug report withdrawn. Sorry for the noise, - M.
Re: Bash bind bug: infinite loop with memory leak in Readline
On 9/22/16 11:23 AM, Christian Klomp wrote: > Limiting the nesting level sounds like a straightforward solution. > Although I'm not sure whether exposing a configurable level is > necessary. Do people actually nest their macros that deep that no > reasonably safe upper bound can be set? There's no reason to expose the level and allow it to be configured. I just meant capping execution at some maximum nesting level. Chet -- ``The lyf so short, the craft so long to lerne.'' - Chaucer ``Ars longa, vita brevis'' - Hippocrates Chet Ramey, UTech, CWRUc...@case.eduhttp://cnswww.cns.cwru.edu/~chet/
Re: [BUG] false positive: [ ! -o optname ]
Op 25-09-16 om 22:40 schreef isabella parakiss: > On 9/25/16, Martijn Dekker wrote: >> The '!' operator in the legacy test/[ builtin does not invert the result >> of the -o operator. Consequently the command >> >> [ ! -o noclobber ] >> >> amounts to a no-op, always returning exit status 0. [...] > [ ! -o noclobber ] means > is '!' a non empty string? if not, is 'noclobber' a non empty string? Wow. Yeah, I suppose that's one possible interpretation. I was going for the POSIX one: | These primaries can be combined with the following operators: | | ! expression | True if expression is false. False if expression is true. Ref: http://pubs.opengroup.org/onlinepubs/9699919799/utilities/test.html#tag_20_128 And indeed your interpretation does not apply to something like "[ ! -e /tmp ]": $ [ -e /tmp ]; echo $? 0 $ [ ! -e /tmp ]; echo $? 1 However, the supposed synonym -a acts differently: $ [ -a /tmp ]; echo $? 0 $ [ ! -a /tmp ]; echo $? 0 Well, one more entry in the long list of arguments to avoid test/[... Thanks, - M.
"HISTSIZE=999999999" cause bash failure after lastest upgrade
Configuration Information [Automatically generated, do not change]: Machine: x86_64 OS: linux-gnu Compiler: gcc Compilation CFLAGS: -DPROGRAM='bash' -DCONF_HOSTTYPE='x86_64' -DCONF_OSTYPE='linux-gnu' -DCONF_MACHTYPE='x86_64-pc-linux-gnu' -DCONF_VENDOR='pc' -DLOCALEDIR='/usr/share/locale' -DPACKAGE='bash' -DSHELL -DHAVE_CONFIG_H -I. -I../. -I.././include -I.././lib -Wdate-time -D_FORTIFY_SOURCE=2 -g -O2 -fdebug-prefix-map=/build/bash-31ueiH/bash-4.4=. -fstack-protector-strong -Wformat -Werror=format-security -Wall -Wno-parentheses -Wno-format-security uname output: Linux kali 4.6.0-kali1-amd64 #1 SMP Debian 4.6.4-1kali1 (2016-07-21) x86_64 GNU/Linux Machine Type: x86_64-pc-linux-gnu Bash Version: 4.4 Patch Level: 0 Release Status: release Description: I use a huge value for HISTSIZE (=9) to enable infinite history items. The actural size of ~/.bash_history is only 4MB now. Everything worked fine before the lastest upgrade. Now bash refuse me to login due to memory allocation failure. After choosing a smaller HISTSIZE, bash still eatup too much unnesssary memory. Repeat-By: open a workable terminal with small HISTSIZE setting, $ echo HISTSIZE=9 > ~/test.rc $ bash --rcfile ~/test.rc bash: xmalloc: cannot allocate 88 bytes (114688 bytes allocated)
Re: [BUG] false positive: [ ! -o optname ]
On 9/25/16, Martijn Dekker wrote: > The '!' operator in the legacy test/[ builtin does not invert the result > of the -o operator. Consequently the command > > [ ! -o noclobber ] > > amounts to a no-op, always returning exit status 0. > > Proof: > > $ set -o noclobber && [ -o noclobber ] && [ ! -o noclobber ] && echo bug > bug > > Interestingly, mksh has this bug as well, though not the original pdksh. > > Thanks, > > - M. > > [ ! -o noclobber ] means is '!' a non empty string? if not, is 'noclobber' a non empty string? --- xoxo iza
[BUG] false positive: [ ! -o optname ]
The '!' operator in the legacy test/[ builtin does not invert the result of the -o operator. Consequently the command [ ! -o noclobber ] amounts to a no-op, always returning exit status 0. Proof: $ set -o noclobber && [ -o noclobber ] && [ ! -o noclobber ] && echo bug bug Interestingly, mksh has this bug as well, though not the original pdksh. Thanks, - M.
Re: Magnitude of Order "For Loop" performance deltas based on syntax change
On 9/24/16 8:29 AM, Christian Franke wrote: > - Method-2 could be significantly speed up if the order of the array > accesses is reversed: > > for (( i=0; i if (( -(Pi[i] - Pi[i+1]) < min )); then > min=$((-(Pi[i]-Pi[i+1]))) > fi > done > > Result: ~3 seconds > (using 'let' in the 'min' assignment failed with syntax error - Bug?) I assume you mean that you used something like let min=-(Pi[i]-Pi[i+1]) That's a syntax error because `(' is a shell operator, and it needs to be quoted if you want it to appear in a word. Since `let' is a builtin, not part of the shell syntax, you have to make sure that all arguments to it are valid shell words. Chet -- ``The lyf so short, the craft so long to lerne.'' - Chaucer ``Ars longa, vita brevis'' - Hippocrates Chet Ramey, UTech, CWRUc...@case.eduhttp://cnswww.cns.cwru.edu/~chet/
Re: Magnitude of Order "For Loop" performance deltas based on syntax change
On 9/23/16 7:15 PM, Tom McCurdy wrote: > Bash Version: 4.3 > Patch Level: 11 > Release Status: release > > Description: > Two nearly identical "For Loop" setups have large deltas in > performance. See test program below. Was confirmed in IRC chat room by > multiple users. Input is very noticeable with around 100,000 values. > > The script was originally used to take an input file where the first line > would determine how many int values were to follow. The remaining lines > each had values that were sorted, and then the smallest difference between > any two values were found. > > Repeat-By: > > Using script below comment/uncomment each method to test. > On my machine > Method-1 finishes in around 2 seconds > Method-2 finishes in around 72 seconds > > > Code Below (also attached script and test input file) --- > START=$(date +%s.%N) > > read -r N && mapfile -t Pi < <(sort -n) > #echo "Done" > > min=1000 > > #Find the smallest difference between two numbers in sorted array with max > array element = 1000 > > # For-Loop Method-1: is super fast > for (( i=0; i do > current=${Pi[$i]} > next=${Pi[$i+1]} > diff=$(( next - current)) > if [ $diff -lt $min ] > then > min=$diff > fi > done Yes, bash array access is heavily optimized for (increasing) sequential access, the most common case. Bash arrays don't have a size limit, and they're allowed to be very sparse, so the internal implementation is a doubly-linked list. You can make this very fast by keeping a roving pointer that indicates the last-accessed element. Now, what bash does as a consequence of this roving pointer optimization is bias towards references that increase monotonically. Christian Franke hit on the reason that the first construct is so much faster: the reference to Pi[i+1] in the second for loop construct increments the last-referenced element, and the immediate reference of Pi[i] forces bash into a search for it from the beginning of the array. This potentially happens twice: once for the test, and once for the assignment. There are a couple of other ways to speed this up, but that doesn't help you right now, of course. I'll be looking at additional optimizations before bash-5.0 gets released. You'd be better off sticking with method 1 for the time being. The other details don't really make all that much difference, though I will look at the `let' anomaly Christian noticed. Chet -- ``The lyf so short, the craft so long to lerne.'' - Chaucer ``Ars longa, vita brevis'' - Hippocrates Chet Ramey, UTech, CWRUc...@case.eduhttp://cnswww.cns.cwru.edu/~chet/
Re: [PATCH] fix printf %b without an argument
On 9/23/16 8:10 PM, isabella parakiss wrote: > here's a patch that fixes this problem > > $ printf "<%3s><%3b>" > < ><> Thanks for the report. Chet -- ``The lyf so short, the craft so long to lerne.'' - Chaucer ``Ars longa, vita brevis'' - Hippocrates Chet Ramey, UTech, CWRUc...@case.eduhttp://cnswww.cns.cwru.edu/~chet/