Re: Full-width characters can result in an unwanted space in filenames

2017-03-06 Thread Chet Ramey 
On 3/6/17 2:11 PM, Misaki wrote:

> At the risk of annoying people, I am replying to all.
> 
> If a filename is displayed with 'ls', an extra space won't be copied if you
> highlight and copy it (including middle-click paste/X selection). Is it not
> possible to have user input work the same way?

I will have to see whether I can change the redisplay to either not need
or remove the space.

There is a big difference between `ls', which simply takes characters and
writes them to stdout, and an editor that has to keep track of where it
is on the screen, the effect of writing characters to the display, and
how much screen space those characters consume.

-- 
``The lyf so short, the craft so long to lerne.'' - Chaucer
 ``Ars longa, vita brevis'' - Hippocrates
Chet Ramey, UTech, CWRUc...@case.eduhttp://cnswww.cns.cwru.edu/~chet/

Re: Pipe using subshells, variables not saved

2017-03-06 Thread Chet Ramey 
On 3/6/17 2:44 PM, Misaki wrote:
> 
> On Sat, Mar 4, 2017 at 10:12 AM, Chet Ramey  > wrote:
> 
> On 3/4/17 1:04 AM, Misaki wrote:
> > Variables are not saved when using a pipe because both sides are in a
> > subshell environment.
> 
> If you want this to happen, enable the `lastpipe' shell option.
> 
> 
> Thank you. My standpoint is 'whatever makes things easier for new users', I
> definitely don't have detailed knowledge of design decisions. But I see
> this:
> http://unix.stackexchange.com/questions/143958/in-bash-read-after-a-pipe-is-not-setting-values
> 
> Someone says that not using a subshell for pipelines is POSIX-compliant.

Posix says you need to assume that each element of a pipeline is run in
a subshell, but not depend on it.  That's non-specific enough to make
both behaviors conformant.

> Even if bash does use a subshell, is there a reason why lastpipe also
> requires job execution to be off (with 'set -m')?

Yes.  Consider what happens when you type ^Z. What do you do with all
the processes in the pipeline, especially the final one, and how do you
continue the pipeline?  That doesn't even begin to take into account
what you need to do with process groups to keep them synchronized. There
are some fragile solutions there, like forking a copy of the shell when
it sees ^Z and fixing everything up (pipes, file descriptors, process
groups), but that's more trouble than the implementation is worth.

That said, if anyone would like to take a run at it, I'd be glad to
look at the result.


-- 
``The lyf so short, the craft so long to lerne.'' - Chaucer
 ``Ars longa, vita brevis'' - Hippocrates
Chet Ramey, UTech, CWRUc...@case.eduhttp://cnswww.cns.cwru.edu/~chet/

Re: Pipe using subshells, variables not saved

2017-03-06 Thread Misaki 
On Sat, Mar 4, 2017 at 10:12 AM, Chet Ramey  wrote:

> On 3/4/17 1:04 AM, Misaki wrote:
> > Variables are not saved when using a pipe because both sides are in a
> > subshell environment.
>
> If you want this to happen, enable the `lastpipe' shell option.
>

Thank you. My standpoint is 'whatever makes things easier for new users', I
definitely don't have detailed knowledge of design decisions. But I see
this:
http://unix.stackexchange.com/questions/143958/in-bash-read-after-a-pipe-is-not-setting-values

Someone says that not using a subshell for pipelines is POSIX-compliant.
Even if bash does use a subshell, is there a reason why lastpipe also
requires job execution to be off (with 'set -m')?

Re: Full-width characters can result in an unwanted space in filenames

2017-03-06 Thread Misaki 
On Sun, Mar 5, 2017 at 9:39 AM, L A Walsh  wrote:

> Chet Ramey wrote:
>
>> On 3/4/17 12:53 AM, Misaki wrote
>>
>>> Full-width character, followed by at least one full word with a space
>>> after it.
>>>
>>> Example: あa a
>>> With trailing space.
>>>
>>>
>>
>> This is a display artifact. If there is one physical space left on the
>> screen line, but the character to be printed occupies more than one space,
>> bash needs to wrap before the end of the screen line, and it needs to
>> keep track of that.
>>
> [...]
> But try this:
>
>
> echo 'あa a '|wc -m
> 6
>
> There should only be 5 characters.
>
> If I cut/paste her text (in quotes) directly into 'wc -m' (so it
> doesn't go through bash, but is taken directly from 'wc' on its
> stdin), then I get '7' (2 extra chars for the quotes):
>
>  wc -m
>>
> 'あa a '7
>^^ pressed Ctl-D twice to not end line w/another char (like LF).
>
> Maybe that examples allows you to duplicate the problem?
>
>
>
Am I mistaken in thinking that this is the trailing newline from echo, and
that without it the results are consistent?

Re: Full-width characters can result in an unwanted space in filenames

2017-03-06 Thread Misaki 
On Sat, Mar 4, 2017 at 11:54 AM, Chet Ramey  wrote:

> On 3/4/17 12:53 AM, Misaki wrote:
>
> > 15 Mar 2015
> > bash bug
> > Full-width character, followed by at least one full word with a space
> after it.
> >
> > Example: あa a
> >
> > With trailing space.
> >
> > Typing before the full-width character causes the 'あ' to be pushed to
> next
> > line, with a blank space on previous line though if selected and copied
> it
> > will actually produce a space.
>
> This is a display artifact. If there is one physical space left on the
> screen line, but the character to be printed occupies more than one space,
> bash needs to wrap before the end of the screen line, and it needs to
> keep track of that.  It uses a space on the display, but that space doesn't
> appear in the readline buffer and the readline movement commands skip over
> it.
>
>
At the risk of annoying people, I am replying to all.

If a filename is displayed with 'ls', an extra space won't be copied if you
highlight and copy it (including middle-click paste/X selection). Is it not
possible to have user input work the same way?

(Though 'ls' piped to 'more' results in both incorrect display with a
fullwidth character at the newline missing, and less useful copying with
the displayed space and newline copied into the pasted text, and 'less'
doesn't have either of these problems.)

Re: -eq and strings

2017-03-06 Thread Rob la Lau 
On 06-03-17 16:09, Geir Hauge wrote:
> If you're counting milliseconds, pattern matching should be slightly
> faster than your -eq hack.

Without benchmarks, I'll take your word for this.

But this really convinced me:

> See also http://mywiki.wooledge.org/BashFAQ/054

"this method fails if the shell is ksh"

Thanks!

-- 
-- Rob la Lau
--
-- Sysadmin en webdeveloper in ruste
--
--  web : https://ohreally.nl/
--  eml : r...@ohreally.nl
--

Re: -eq and strings

2017-03-06 Thread Geir Hauge 
On Mon, Mar 06, 2017 at 03:31:31PM +0100, Rob la Lau wrote:
> On 06-03-17 15:16, Reuti wrote:
> > if ! expr "$x" : '[[:digit:]]*$' >/dev/null; then echo no; fi
> > 
> > if [ -n "${x//[0-9]/}" ]; then echo no; fi
> 
> True, but regular expressions are usually considered expensive.
> (Must admit I never benchmarked.)
> 
> And I know 1 regex won't considerably slow down my script, but it's
> become a habit to always try and program as light as possible.

If you're counting milliseconds, pattern matching should be slightly
faster than your -eq hack.

is_digit() { [[ -n $1 && $1 != *[![:digit:]]* ]]; }

or the sh variant:

is_digit() { case $1 in ''|*[![:digit:]]*) return 1;; esac; }


See also http://mywiki.wooledge.org/BashFAQ/054

-- 
Geir Hauge

Re: -eq and strings

2017-03-06 Thread Charles Daffern 
On 6 March 2017 14:09:39 GMT+00:00, Rob la Lau  wrote:
>On 06-03-17 14:18, Greg Wooledge wrote:
>> You're misunderstanding.  In a math context, which you are creating
>here
>> by using -eq, the word 'x' is interpreted as a variable name, which
>may
>> contain another variable name, and so on, until finally an integer is
>> discovered.
>
>Thanks. Now that I know it, I can indeed find it in the docs.
>
>I guess this means that actually test and [ are 'broken':
>
>$ test "x" -eq "x" && echo "yes" || echo "no"
>test: invalid integer 'x'
>no

It is my understanding that [ is a standard command mandated by POSIX which may 
be implemented as an external command. It would be unreasonable to expect an 
external command to be able to introspect shell variables, and similarly I 
would be surprised to find a shell which does this substitution in the [ 
command. For that reason it would certainly break my usual expectations for how 
my scripts should run, and it might possibly become a source of bugs. In the 
case of [[ however, it is entirely understandable because it is expected that 
the shell does magic in that case (new parsing rules etc.).
-- 
Sent from my Android device with K-9 Mail. Please excuse my brevity.

Re: Problem: completion changes user-relative path into absolute path breaking relative-path usages

2017-03-06 Thread L A Walsh 



Chet Ramey wrote:

I can't reproduce this with bash-4.4, even when setting `direxpand'
before performing completion on a pathname beginning with `../'.
It is possible that you or your vendor compiled bash with directory
expansion enabled by default (--enable-direxpand-default), which
turns this on.

---
   Bingo.  I checked the build script and it's set to 'on'.
Not sure how it got that way, but I'm going to have
trouble walking w/both feet in mouth...

   Still not recovered from that root-canal just yet,
I guess.  Oh so fun... Sorry for line noise.

Re: Bug: UTF-8 expansion results in extra characters

2017-03-06 Thread L A Walsh 



Eduardo Bustamante wrote:

On Mon, Mar 6, 2017 at 6:50 AM, L A Walsh  wrote:
[...]
  

echo 'あa a '|wc -m
6

There should only be 5 characters.


use echo -n then.

$ echo -n 'あa a '|wc -m
5
  

---
   Yup.  I'm an idiot at times! 


The thing was that I kept copying it into different windows
and I kept getting different behaviors where the count wasn't
consistent.  So, I got overfocussed on reproducing it
otherwise.

   It's still the case that in Misaki's post, there is a space
at the end of the line, but not in the quoted version that Chet
posted...

   Thanks for the head-butt.

   Now if you can tell me what's wrong with my completion
problem, I can go back to sleep... ;^/  *sigh*

Re: Problem: completion changes user-relative path into absolute path breaking relative-path usages

2017-03-06 Thread Chet Ramey 
On 3/6/17 7:33 AM, L A Walsh wrote:

> Under 4.4:
> 
> Ishtar:../util-linux-2.29.2> mydiff ../conf
> Ishtar:../util-linux-2.29.2> mydiff
> /home/tools/util-linux/year-/month-mmm/day-ddd/newwork/very_long_path/config-2292.sh
> 
> diff:
> /home/tools/util-linux/year-/month-mmm/day-ddd/newwork/very_long_path/archive//home/tools/util-linux/year-/month-mmm/day-ddd/newwork/very_long_path/config-2292.sh:
> No such file or directory
> diff:
> /home/tools/util-linux/year-/month-mmm/day-ddd/newwork/very_long_path//home/tools/util-linux/year-/month-mmm/day-ddd/newwork/very_long_path/config-2292.sh:
> No such file or directory
> 
> 
> It throws away the relative path I need/expect and can't
> find anything, but worse it is very confusing throwing out
> an untyped path from the root.
> 
> Please, can this be fixed so the default is to not expand with
> those who want such unwieldy paths can apply some option?

I can't reproduce this with bash-4.4, even when setting `direxpand'
before performing completion on a pathname beginning with `../'.
It is possible that you or your vendor compiled bash with directory
expansion enabled by default (--enable-direxpand-default), which
turns this on.  If you don't want it, you can try turning off the
`direxpand' option.

-- 
``The lyf so short, the craft so long to lerne.'' - Chaucer
 ``Ars longa, vita brevis'' - Hippocrates
Chet Ramey, UTech, CWRUc...@case.eduhttp://cnswww.cns.cwru.edu/~chet/

Re: -eq and strings

2017-03-06 Thread Rob la Lau 
On 06-03-17 15:16, Reuti wrote:
> if ! expr "$x" : '[[:digit:]]*$' >/dev/null; then echo no; fi
> 
> if [ -n "${x//[0-9]/}" ]; then echo no; fi

True, but regular expressions are usually considered expensive.
(Must admit I never benchmarked.)

And I know 1 regex won't considerably slow down my script, but it's
become a habit to always try and program as light as possible.

-- 
-- Rob la Lau
--
-- Sysadmin en webdeveloper in ruste
--
--  web : https://ohreally.nl/
--  eml : r...@ohreally.nl
--

Re: -eq and strings

2017-03-06 Thread Reuti 

> Am 06.03.2017 um 15:09 schrieb Rob la Lau :
> 
> On 06-03-17 14:18, Greg Wooledge wrote:
>> You're misunderstanding.  In a math context, which you are creating here
>> by using -eq, the word 'x' is interpreted as a variable name, which may
>> contain another variable name, and so on, until finally an integer is
>> discovered.
> 
> Thanks. Now that I know it, I can indeed find it in the docs.
> 
> I guess this means that actually test and [ are 'broken':
> 
> $ test "x" -eq "x" && echo "yes" || echo "no"
> test: invalid integer 'x'
> no
> 
> $ [ "x" -eq "x" ] && echo "yes" || echo "no"
> [: invalid integer 'x'
> no
> 
> I hope this won't be fixed, as I rely quite heavily on this behaviour,
> lacking some other short 'is integer' test.

What about these:

if ! expr "$x" : '[[:digit:]]*$' >/dev/null; then echo no; fi

if [ -n "${x//[0-9]/}" ]; then echo no; fi

-- Reuti


> 
> Cheers,
>  Rob
> 
> --
> -- Rob la Lau
> --
> -- Sysadmin en webdeveloper in ruste
> --
> --  web : https://ohreally.nl/
> --  eml : r...@ohreally.nl
> --
> 



signature.asc
Description: Message signed with OpenPGP using GPGMail

Re: Bug: UTF-8 expansion results in extra characters

2017-03-06 Thread Eduardo Bustamante 
On Mon, Mar 6, 2017 at 6:50 AM, L A Walsh  wrote:
[...]
> echo 'あa a '|wc -m
> 6
>
> There should only be 5 characters.
use echo -n then.

$ echo -n 'あa a '|wc -m
5

Re: -eq and strings

2017-03-06 Thread Rob la Lau 
On 06-03-17 14:18, Greg Wooledge wrote:
> You're misunderstanding.  In a math context, which you are creating here
> by using -eq, the word 'x' is interpreted as a variable name, which may
> contain another variable name, and so on, until finally an integer is
> discovered.

Thanks. Now that I know it, I can indeed find it in the docs.

I guess this means that actually test and [ are 'broken':

$ test "x" -eq "x" && echo "yes" || echo "no"
test: invalid integer 'x'
no

$ [ "x" -eq "x" ] && echo "yes" || echo "no"
[: invalid integer 'x'
no

I hope this won't be fixed, as I rely quite heavily on this behaviour,
lacking some other short 'is integer' test.

Cheers,
  Rob

-- 
-- Rob la Lau
--
-- Sysadmin en webdeveloper in ruste
--
--  web : https://ohreally.nl/
--  eml : r...@ohreally.nl
--

Re: unset array[0] doesn't delete the element

2017-03-06 Thread Greg Wooledge 
On Sat, Mar 04, 2017 at 09:10:00PM -0800, Isaac Good wrote:
> Repeat-By:
>   $ arr=(1 2 3); unset arr[-1]; echo "${arr[@]}"
>   1 2 3

On Sat, Mar 04, 2017 at 09:14:09PM -0800, Isaac Good wrote:
> Bug report being withdrawn. I had nullglob set so unset wasn't being
> invoked properly.

The bug is that you failed to quote the 'arr[-1]' word.  Whether you
have nullglob enabled or not, that word *can* be globbed against a file
in the current directory, so it must always be quoted.

imadev:~$ arr=(1 2 3)
imadev:~$ unset 'arr[-1]'
imadev:~$ declare -p arr
declare -a arr=([0]="1" [1]="2")

Good job on detecting the interaction with nullglob, though.

Re: -eq and strings

2017-03-06 Thread Greg Wooledge 
On Sat, Mar 04, 2017 at 01:15:58PM +0100, Rob la Lau wrote:
> Looking a bit further: it doesn't do any comparison on the given strings:
> 
> $ [[ "x" -eq "y" ]] && echo "yes" || echo "no"
> yes

You're misunderstanding.  In a math context, which you are creating here
by using -eq, the word 'x' is interpreted as a variable name, which may
contain another variable name, and so on, until finally an integer is
discovered.  Same for the word 'y'.

> $ [[ "x" -eq "yz" ]] && echo "yes" || echo "no"
> yes

Same for the word 'yz'.

In your case, I suspect the variables x, y and yz are all undefined, which
means they evaluate to 0.

imadev:~$ unset x yz; if [[ x -eq yz ]]; then echo yes; else echo no; fi
yes
imadev:~$ x=3 yz=foo foo=3; if [[ x -eq yz ]]; then echo yes; else echo no; fi
yes
imadev:~$ x=3 yz=foo foo=2; if [[ x -eq yz ]]; then echo yes; else echo no; fi
no

Re: Pipe using subshells, variables not saved

2017-03-06 Thread Greg Wooledge 
On Fri, Mar 03, 2017 at 10:04:23PM -0800, Misaki wrote:
> > reason for piping:
> > echo wat | for i in $(< /dev/stdin); do declare -g new="$i"; done
> >
> > (using find instead of echo, not sure if better way to loop)

while IFS= read -r file; do blah "$file"; done < <(find . -type f)

That's compatible with any filenames that don't contain newlines.
To be 100% compatible with ALL possible filenames, you need to use
a NUL-delimited stream.  On GNU/BSD systems:

while IFS= read -rd '' file; do blah "$file"; done < <(find . -type f -print0)

On recent-enough POSIX systems:

while IFS= read -rd '' file; do blah "$file"; done \
< <(find . -type f -exec printf '%s\0' {} +)

See also http://mywiki.wooledge.org/BashFAQ/024

See also http://mywiki.wooledge.org/DontReadLinesWithFor

Bug: UTF-8 expansion results in extra characters

2017-03-06 Thread L A Walsh 

I didn't see that this was caught and wasn't sure if
it was already covered in what I previously
posted this problem in a follow-up to a similar problem.

It may be the same bug, but I wasn't sure. 


If I past the text in quotes into bash, bash tosses in
an extra character as evidenced by 'wc':


echo 'あa a '|wc -m
6

There should only be 5 characters.

If I cut/paste her text (in quotes) directly into 'wc -m' (so it
doesn't go through bash, but is taken directly from 'wc' on its
stdin), then I get '7' (2 extra chars for the quotes):


 wc -m

'あa a '7
  ^^ pressed Ctl-D twice to not end line w/another char (like LF).

Maybe that examples allows you to duplicate the problem?



I don't think your development setup allows for cut/paste
from an editor or mail-window that accurately copies the
characters.   Something in your text display+copy+insert
doesn't seem to copy the actual characters, but something
that looks similar.

Ex, copying text like this sometimes misses a space on the end:

あa a 


and

Copying from an editor or line that has 'tabs' in it, doesn't
seem to preserve the tabs, so you can't cut+paste script from
an editor into bash without it being becoming disintegrous.

I think its a side-effect of the development tools you are
using (I don't know what they are, but its also a guess, based
on sparse data where specific characters aren't mapped the same
and don't produce the same output in regards to what was
actually presented when someone tries to describe a problem.

It doesn't make a difference in most cases, but in a few, like
the ones mentioned above ;-(

Anyway, like I was trying to say, it's a, perhaps, inaccurate
deduction based scant evidence...  oh well...

Problem: completion changes user-relative path into absolute path breaking relative-path usages

2017-03-06 Thread L A Walsh 


I have used relative paths to a 'base' and 'dest' to allow
comparisons/upgrades, etc, something along the lines of
(this is a rough-spur of the moment example):

cur=where I'm currently working
base=olddir
...
idea that no matter where I move in new tree, path from
'cur' is same as path from 'base' for old stuff

Under 4.3:

Ishtar:tools/../newwork/very_long_path> cd util-linux-2.29.2
Ishtar:../util-linux-2.29.2> printf "cur=$cur\nbase=$base\n"
cur=/home/tools/util-linux/year-/month-mmm/day-ddd/newwork/very_long_path
base=/home/tools/util-linux/year-/month-mmm/day-ddd/newwork/very_long_path/archive

mydiff() {
cpth=${1#../}
diff $base/$cpth $cur/$cpth && echo "Same."
}

Ishtar:../util-linux-2.29.2> cd util-linux-2.29.2
Ishtar:../util-linux-2.29.2> mydiff ../config-2292.sh
Same.


Under 4.4:

Ishtar:../util-linux-2.29.2> mydiff ../conf
Ishtar:../util-linux-2.29.2> mydiff 
/home/tools/util-linux/year-/month-mmm/day-ddd/newwork/very_long_path/config-2292.sh 

diff: 
/home/tools/util-linux/year-/month-mmm/day-ddd/newwork/very_long_path/archive//home/tools/util-linux/year-/month-mmm/day-ddd/newwork/very_long_path/config-2292.sh: 
No such file or directory
diff: 
/home/tools/util-linux/year-/month-mmm/day-ddd/newwork/very_long_path//home/tools/util-linux/year-/month-mmm/day-ddd/newwork/very_long_path/config-2292.sh: 
No such file or directory



It throws away the relative path I need/expect and can't
find anything, but worse it is very confusing throwing out
an untyped path from the root.

Please, can this be fixed so the default is to not expand with
those who want such unwieldy paths can apply some option?

I just recently tried to up my shell from 4.3 to use
4.4 so I'd be better able to catch problems, but this is
enough to send me back again until the rough edges are
more ironed out...  Why would this change become the default?

Hoping for a soon fix,
Thanks
-linda