Re: Unexpected behavior with TZ

2011-04-08 Thread Marc Herbert
Le 07/04/2011 23:16, gmail a écrit :
 I 'm not fully familiar with POSIX specifications regarding locales, for 
 now these specifications let me think that this behavior is expected and 
 that the
 LANG environment variable is taken into account after the LC_* 
 environment variables

Would this help? http://mywiki.wooledge.org/BashFAQ/098






Re: BASH_SUBSHELL documentation misleading

2011-03-24 Thread Marc Herbert
Le 23/03/2011 20:28, Chet Ramey a écrit :
 On 3/23/11 4:15 PM, Sam Liddicott wrote:
 
 Yes. But a new subshell environment has been spawned. Each time that
 happens BASH_SUBSHELL should increase.

 Of course I know how it does work, but the man page isn't clear. It doesn't
 say that the increase is only visible within the subshell and therefore it
 is a measurement of subshell depth.
 
 OK.  What wording would you like to see?
 

I would also like to see this example in the documentation:

$ ( echo sub-$BASH_SUBSHELL ); echo main-$BASH_SUBSHELL
sub-1
main-0

This example answers all doubts raised in this discussion. It also
hints at the fact that:

  ( echo 'sub-$BASH_SUBSHELL' )

does not work like an inexperienced quoter could wrongly assume.





Re: BASH_SUBSHELL documentation misleading

2011-03-24 Thread Marc Herbert
Le 24/03/2011 14:50, Greg Wooledge a écrit :
 On Thu, Mar 24, 2011 at 02:33:19PM +, Marc Herbert wrote:
 I would also like to see this example in the documentation:

 $ ( echo sub-$BASH_SUBSHELL ); echo main-$BASH_SUBSHELL
 sub-1
 main-0
 
 Overkill.

Tradeoff.

 Most of the other features in the manual do not have such
 examples, and if we added examples of this size for every single feature,
 the manual would swell up to a tremendous size.  It's already quite large.

Not every feature is complicated enough that it requires special
documentation care and that it raises a discussion here.






Re: compat31

2011-02-15 Thread Marc Herbert
Le 14/02/2011 15:03, Dennis Williamson a écrit :
 I'm having a discussion with someone who claims that for them on Bash
 4.0.33, with compat31 *off*, they are getting 3.1 behavior with quoted
 patterns in a =~ match.
 
 They're saying that this:
 
 [[ a =~ . ]]  echo match || echo no
 
 results in match.
 
 I cannot reproduce this without doing
 
 shopt -s compat31
 
 or removing the quotes. Are there any other circumstances that could
 produce this behavior?

I think this qualifies:

http://thread.gmane.org/gmane.comp.shells.bash.bugs/15080/focus=15107





Re: Getting rid of Terminated message thanks to SIGINT

2011-02-01 Thread Marc Herbert
 Is it a problem?  Bash prints messages about signal-terminated processes --

 Most people want to know when their jobs die

 ...except when they explicitly kill them.
 
 Then maybe the answer is to suppress the message when a process dies
 due to SIGTERM, as well as SIGINT.
 

Since TERM is the default it seems to make sense to me. Could you
mention this to the bash maintainer? :-)

Would the same also make sense for a couple other signals? I have
always been confused by the respective semantics of signals (assuming
there are well defined semantics in the first place).




Getting rid of Terminated message thanks to SIGINT

2011-01-31 Thread Marc Herbert
Le 30/01/2011 00:12, Chet Ramey a écrit :
 Is it a problem?  Bash prints messages about signal-terminated processes --

 Most people want to know when their jobs die

...except when they explicitly kill them.

 at least those that don't die due to SIGINT or SIGPIPE -- when the
 shell is not interactive. 

Wow! I wish I had known this very useful SIGINT trick earlier. Is it
in a FAQ somewhere?





Re: for; do; done regression ?

2011-01-11 Thread Marc Herbert
Le 10/01/2011 17:05, Roman Rakus a écrit :
 I will say the same on both lists - why to execute another process if 
 you can do the same in the current one? No gain.

Not every for loop is in the critical path. Especially not in shell script.






Re: for; do; done regression ?

2011-01-10 Thread Marc Herbert
Le 10/01/2011 13:38, Greg Wooledge a écrit :
 Don't use seq, ever, because it's Linux-only
 and is a stupid way to count when bash has builtin integer arithmetic.)

seq is not exactly Linux-only but GNU-only.

GNU-specific versus bash-specific, which is worse? I'd say it
depends... on which mailing-list you post :-)







Re: Making $! and $? searchable in the man page

2011-01-07 Thread Marc Herbert
Le 06/01/2011 09:48, Vidar Holen a écrit :

 Finding the meaning of $? and $! in the man page is quite hard for people
 not familiar with the layout and bash terminology (this frequently comes
 up in Freenode #bash). It would be very helpful for them if you could
 simply search for $! to find the description of the parameter !.
 
 Below is a suggestion patch that just adds a $ in front of the parameters
 under Special Parameters to make this possible.
 

Well done Vidar, such problems are (were?) very annoying indeed.





Re: cd with multiple arguments?

2010-12-20 Thread Marc Herbert
Le 17/12/2010 20:57, Bob Proulx a écrit :
 CGI scripts are not normally setuid but are running as the web
 server process owner

You wish...

 Instead they stem from a script running unverified user provided
 input. [...]  It is a problem, and a big one, but completely different from
 having a local user attack against an setuid script and be able to
 gain the priviledge of the script owner.

I do not think it is completely different. A setuid script has
defend itself against input from the local user.


 Using user provided input as commands is a problem no matter what
 language you use.

Some languages make it easy, others not.





Re: cd with multiple arguments?

2010-12-14 Thread Marc Herbert
Le 13/12/2010 19:48, Stephane CHAZELAS a écrit :
 Yes, they're called commands or programs or software.
 
 People tend to forget that before all a shell is a command line
 interpreter.
 
 If you're finding yourself writing complex shell functions that
 don't make use of external commands, then you're heading the
 wrong direction with your shell, or you are heading in one
 direction with the wrong medium (i.e. you need a programming
 language, not a shell).

Well put. I agree very much with this last paragraph, but the original
question was not about a complex shell function.

I should not have used the word library either. Let me say
convenience module instead.  Something like this for instance:
http://www.ukuug.org/events/linux2003/papers/bash_tips/

Ideally end users could download and enable Simon Myers module in
just a few commands. What we currently have instead is each Linux
distribution doing its little bit of fine-tuning, enabled for every
user by default, and painful to customize and override.





Re: declare overrides errexit

2010-09-24 Thread Marc Herbert
Le 24/09/2010 15:38, Greg Wooledge a écrit :
 If this is turning into a general discussion of errexit (set -e) then
 please also see
 
 http://mywiki.wooledge.org/BashFAQ/105   and
 http://fvue.nl/wiki/Bash:_Error_handling

These pages are great and should be read. But from experience, I
totally disagree with GreyCat's conclusion:

  GreyCat's personal recommendation is simple: don't use set -e. Add
  your own error checking instead.

My life has become incredibly easier since I started to put at the top
of every single of my scripts either a line like this:
if $DEBUG; then set -e; fi
.. or even just a plain set -e.

The only effort this requires is the need to append this from time to
time:
   cat foo || true # missing foo: expected, ignore

It is an order of magnitude more convenient and of course much safer
to append || true from time to time than to append || exit 1 after
almost every line. Tested.

(The only other effort is to refrain to file a bug when set -e
sometimes fails to catch a failed command :-)

Bourne shell is the only programming language I know that ignores
errors by default. set -e does not completely repair this design
flaw but it goes a long and appreciated way.





Re: inconsistent field splitting

2010-08-20 Thread Marc Herbert
 There is a big difference between

 $ a=;

 and

 $ a=;


 And for the VERY ignorant what is the difference

In the second case the unescaped semi-colon is a separator totally
unrelated to the assignment, which has no value at all. The variable
is assigned an empty string.





Re: inconsistent field splitting

2010-08-19 Thread Marc Herbert
Since inconsistency is basically not the same rule
everywhere, it typically shows in (good) documentation. Indeed:

  
http://www.opengroup.org/onlinepubs/009695399/utilities/xcu_chap02.html#tag_02_06
  2.6 Word Expansions
  Not all expansions are performed on every word, as explained in the
  following sections.

The above sentence leaves open the scary possibility of a large number
of combinations and special cases, but of course it is not that badly
inconsistent.

Focusing back on field splitting, both the opengroup and the (similar)
bash manual have the same documentation logic: they apparently never
explicit where field splitting is performed (only before/after which
expansions it is performed when it is).

So to understand where field splitting is performed the reader must
assume that it is performed everywhere, except when it is exceptionally
missing from an exhaustive list of expansions (case, assignment,...).
Unclear and inconvenient.

Of course the best alternative is the one mentioned by Greg: assume
that field splitting is consistently performed everywhere and keep
quoting everything all the time. It is a good habit anyway and
avoids bugs due to refactoring.






Re: return from function doesn't work when used in tail of pipeline

2010-08-19 Thread Marc Herbert
 On Thu, 12 Aug 2010 08:16:21 -0400, Greg Wooledge wool...@eeg.ccf.org
 A return that's run in a subshell doesn't cause the parent shell to
 return.

Granted but that does not mean it should behave like exit, instead it
could simply error like this:

$ return
bash: return: can only `return' from a function or sourced script

It would be so much simpler if the use of exit was restricted to
shells and subshells while the use of return was actually restricted
to functions and sourced scripts. Are there many people actually using
return to exit a subshell? That would not really make code easy to
read.


Le 13/08/2010 00:49, Martin Schwenke a écrit :
 I've been programming in shell for 20 years and this one surprised me.
 I have a very strong Unix background and I clearly understand things
 like not being able to set variables in subshells.  It also surprised
 other people, with similar backgrounds, who I showed it to.  It is much
 less obvious than many other shell subtleties.

+1, some time ago I found myself scanning documentation for ages
before finding a barely clear answer to these exit/return questions,
they deserve at least more documentation.


 There are some obvious exceptions to the strict Unix subprocess model in
 shell programming (e.g. I can set a variable without exporting it and
 see it in a subshell)  and I expected this to be one of them.

But the unexported variable example goes from parent to child, whereas
the ability from a subprocess to exit his parent would be quite a
dramatic exception to the rule where a child can never have any
side-effect on his parent.





Re: return from function doesn't work when used in tail of pipeline

2010-08-19 Thread Marc Herbert
 A return that's run in a subshell doesn't cause the parent shell to
 return.

   **SNIP**

 Are there many people actually using
 return to exit a subshell? That would not really make code easy to
 read.



 You misunderstood, or I was unclear. 

*You* misunderstood, or I was unclear :-)

There was NO relation between the parts before and after the *SNIP*. I
had moved on to a suggestion to forbid return in subshells to prevent
any confusion, and was asking about any backward compatibility issue.





inconsistent field splitting

2010-08-18 Thread Marc Herbert
Compare:

   for a in $(echo 1 2); do echo x${a}x; done
x1 2x
   for a in  $(echo 1 2) ; do echo x${a}x; done
x1x
x2x

   a=$(echo 1 2); echo x${a}x
x1 2x
a=$(echo 1 2);  echo x${a}x
x1 2x



Shell quoting is difficult enough; why is such an inconsistency making
it even more confusing?

Uwe Waldmann might give a clue in his excellent Guide to Unix shell
quoting:

   Note that in these [assignment + others] cases, the shell syntax
   allows only a single word, not a sequence of words, so that blank
   interpretation or expansion of globbing characters might result in
   something syntactically illegal.

In other words:

   In order to save you from some very obvious syntax errors, I'll
   make quoting even more confusing than it already is.

Sorry but I am not grateful at all.

Or is there a better rationale for this design?





Re: inconsistent field splitting

2010-08-18 Thread Marc Herbert

 What should a=* or a=$(echo 1 2) do?
 Assign only the first file or just 1 to a? or result in an error?

Probably result in some error; I do not really care in this initial
stage. In any case it should not perform any *implicit* (and
thus confusing) quoting.


 This hardly seem more consistent  or less surprising to me.

Quoting makes a difference in some places but no difference in other
places (because of some kind of implicit quoting). This is not
consistent from a syntax perspective.




Re: inconsistent field splitting

2010-08-18 Thread Marc Herbert
Le 18/08/2010 12:14, Andreas Schwab a écrit :
 Marc Herbert marc.herb...@gmail.com writes:
 In any case it should not perform any *implicit* (and
 thus confusing) quoting.
 
 There is no quoting, the shell just doesn't split into words,

I know but it looks like quoting, because quoting is how you prevent
word splitting.

Sorry but I am not interested at all in how the current implementation
works. I am only wondering why it was *designed* like this.

Try to remove your expert hat for one second and take the stance of
some developer new to the shell, trying to make sense of quoting, word
splitting and the rest.


 which doesn't make any sense in an assignment.

Everything makes sense is a nice feature in programming language
design but extremely low-priority compared to consistency.






Re: Issues when func name is the same with an alias

2010-08-06 Thread Marc Herbert
This is much more code:

 F(){ find $@; }

 This simple func definition will be 4 lines in my coding style. :) And I
 like adding at least one blank line between functions.

Then just exclude .bashrc from your coding style.




Re: Issues when func name is the same with an alias

2010-08-05 Thread Marc Herbert
Le 04/08/2010 15:29, Clark J. Wang a écrit :
 I do not agree. Aliases are much simpler to use than functions.

Please provide examples.





Re: RFE: allow double quotes to group multi-words be treated as 1 arg, w/DQ's stripped off -- including RH =~

2010-08-04 Thread Marc Herbert
Le 04/08/2010 09:27, Davide Brini a écrit :

 From the Changelog:
 
 This document details the changes between this version, bash-3.2-alpha,
 and the previous version, bash-3.1-release.
 ...
 3.  New Features in Bash
 ...
 f.  Quoting the string argument to the [[ command's  =~ operator now forces
 string matching, as with the other pattern-matching operators.

Yet on 3.2.39 (Fedora 10) the old quoted regex behaviour was still there:

echo $BASH_VERSION 
shopt  compat31
w=/home/law/bin/package: line 5: type: xx: not found
[[ $w =~ .*not found.* ]]  echo match


3.2.39(1)-release
compat31off
match





Re: Issues when func name is the same with an alias

2010-08-04 Thread Marc Herbert
Le 04/08/2010 11:39, Clark J. Wang a écrit :

 Seems like I must explicitly use the `function' keyword to define foo() for
 this scenario. Is that the correct behavior?

The correct behaviour is simply not to use aliases, since they bring nothing
to the table compared to functions. Have a look at this:
http://thread.gmane.org/gmane.comp.shells.bash.bugs/13865/focus=13901


About the function keyword have a look at the discussion two days ago.





Re: RFE? request for an undefined attribute for functions

2010-08-03 Thread Marc Herbert
Le 02/08/2010 20:11, Bernd Eggink a écrit :
 The other is that 'function' is clear and 
 self-explaining, while 'name()' wrongly suggests that function 
 parameters should be surrounded by parentheses.

... but only to people whose experience with Unix shells is close to
zero.  Functions are not really an advanced feature.





Re: A note for read builtin

2010-06-17 Thread Marc Herbert
Le 16/06/2010 19:03, Greg Wooledge a écrit :
 On Wed, Jun 16, 2010 at 07:47:03PM +0200, Krzysztof ??elechowski wrote:
 The description of the read builtin [19] would benefit of the following note:
 Warning: A pipeline of the form { echo 1 2 | read a b; } is not useful.  Use 
 { 
 read1 2 a b; } instead.

 That kind of advice is certainly welcome in tutorials, guides, and so on
 which are teaching shell programming.  I don't really think it needs to
 go into the manual, though.  There are hundreds, if not thousands, of
 things you should _not_ do, that shouldn't have to be stated in the
 reference manual.  This is just one of them.

- I do not think this is just one of them. This question
  pops up VERY regularly here, many times more than other
  things that should not be done.

- Consider the very first sentence in the documentation:
  
http://www.gnu.org/software/bash/manual/html_node/Bash-Builtins.html#index-read-142

 read
  One line is read from the standard input, or from...

  Such a description obviously begs for a pipeline; no surprise everyone
  makes this mistake one day or the other.

- The POSIX standard does allow echo 1 2 | read a b to be useful
  (cf. 2.12 Shell Execution Environment). Some alternatives to
  bash actually make it useful. Standard and portability concerns
  definitely belong to a reference manual.





Re: A note for read builtin

2010-06-17 Thread Marc Herbert
 - The POSIX standard does allow echo 1 2 | read a b to be useful
(cf. 2.12 Shell Execution Environment). 
   
 Maybe the POSIX expect shell to execute the last command of pipeline
 not in subshell.

From section 2.12 and from messages posted here in the past I
understand that POSIX allows either one. This ambiguity reinforces the
need for documentation IMHO.




Re: Error Handling with bash script

2010-05-26 Thread Marc Herbert
Le 24/05/2010 17:05, Lenga, Yair a écrit:

 I would like to propose an improvement to the bash error handling:
 Add flag/option err_return, that will cause a user defined shell
 functions to immediately return, when a simple command will return a
 non-zero status code.
 The behavior is similar to the existing '-e', but is different in
 the following case.  Under current bash, the Abort On Error logic
 is suspended if the command is executed under if, while, || or
 similar. It will make it possible to implement much better error
 handling for scripts that involve user defined functions.

I agree, this looks like it could be an extremely useful feature.

The recurring and intricate discussions on this list clearly
demonstrate that errexit:
- draws a lot of interest,
- has a broken design.

Some examples (randomly picked up among many others)
http://thread.gmane.org/gmane.comp.shells.bash.bugs/6071
http://thread.gmane.org/gmane.comp.shells.bash.bugs/12117
http://thread.gmane.org/gmane.comp.shells.bash.bugs/12281
http://thread.gmane.org/gmane.comp.shells.bash.bugs/13726
http://thread.gmane.org/gmane.comp.shells.bash.bugs/13465


Since errexit is hopelessly caught forever in design by committee
and backward compatibility traps, a brand new errreturn feature
could provide something useful instead.

This is basically about some very minimal, poor-man's exceptions. An
extremely basic feature to stop code running wild when something
unexpected and bad happens. Only prehistoric or low-level, operating
systems languages like C miss such a feature.


 Adding error handling to this script can be done in two ways:
 1 - Modifying each functions/script:
 Function f1 {
   cp /new/data /current/data || return 1
   step1  || return 1
   step2  || return 1
 }

I find this slightly better:
Function f1 {
  cp /new/data /current/data || return $?
step1  || return $?
step2  || return $?
}





Re: foo* doesn't match when 'foobar' does; fname=FooBar on xfs w/naming ci option

2010-05-21 Thread Marc Herbert
Le 21/05/2010 09:32, Linda Walsh a écrit :
 I'm not sure if this is bash specific...I'd wager not, but I'm not sure where 
 to
 put it.

I guess it is not.

 I created an xfs file system with the naming version=ci, (case-ignore) flag.
 
 This causes it to match filenames with case ignored.
 
 So an exact match of a file name matches with the case begin ignored,
 i.e. 'foobar' matches 'FooBar'.
 
 But foo* doesn't match.

I think your design problem is that you are trying to perform half of
matching in the shell, and then the other half in the filesystem.


If bash is in case-sensitive mode (the default), then bash will
expands foo* to... foo*. Because there is no file starting with lower
case foo. Then bash will pass this literal star string to xfs. But xfs
does not seem to implement globbing. So xfs will look for 'foo*',
'Foo*', 'FOO*', 'fOo*' with a literal star.

 The bash option to ignore case in wildcards wouldn't be a correct
 option for this, as that would cause it to ignore case on all file
 systems (if I understand it correctly).

Correct because bash does not care about filesystems. This is too low
level for bash (at least in this filename case).


 So how can I get case ignored in wildcards, but only on this file
 system -- consistent with it's creation options?  (version=ci is an
 option at file system creation time).

You could try to implement (case-insensitive) *globbing* in xfs. Quite
a challenge I guess!


 Does bash use a generic regex library or does it have its own?

Warning: globbing is different from regular expressions.

Cheers,

Marc





Re: How to autocomplete after 'which'?

2010-05-21 Thread Marc Herbert
Le 21/05/2010 16:25, Peng Yu a écrit :
 When I type something after 'which', something means a command.
 However, bash doesn't do command completion for the argument after
 'which'. Is there a way to configure bash behave depending on the
 context (in this case, do autocomplete after 'which')?

By the way it is better to use type instead of which. which is
often misleading, compare for instance:

type pwd
which pwd

which is misleading in many other cases.





Re: How to make a directory name with '/' in it?

2010-05-17 Thread Marc Herbert
Le 16/05/2010 16:51, Chris F.A. Johnson a écrit :
 
There are two characters that cannot be in a file or directory
name: / and NUL.

Interesting essay about this:
http://www.dwheeler.com/essays/fixing-unix-linux-filenames.html





Re: How to overwrite a symbolic link?

2010-05-10 Thread Marc Herbert
Le 07/05/2010 16:02, Peng Yu a écrit :
 I can copy the whole
 directory and then modify one file in the newly copied N files. But
 I'll lose track of which file has been changed later on, which is
 important to me.

You will not lose track of the changed files: just run a recursive
diff comparing the old and new directories and you will see what
was/has changed (this is actually the poor man's version control
system).


 Suppose I have N files in a directory that does some job. Lets say M
 (N) files are primary files and N-M files are derived from these M
 files (say, there are a number of programs, each takes some of the M
 files to generate some of the N-M files).
 [...]
 Instead, I'd rather create symbolic link for all the N-M-1 primary
 files but copy only one remaining file and modify it. Remember, I
 can't tell which are primary files and which are derived files. So I
 can not do so.
 
 One solution is create symbolic link to N-1 files as long as I can
 overload '' and '' and the open file function call.

So you are really trying to re-invent some copy-on-write/versioning
file system by hacking symbolic links and redirections. This looks
like a lot of pain. You'd better start by looking at existing
solutions; even if none is ideal for your case, you would at least
leverage the existing knowledge and experience in this field instead
of starting from zero.


 Let me know if you have any thought on solving this problem.
 
Since you are using the wrong tool for the job I am afraid you are
going to be rather alone on this road.

Cheers,

Marc






Re: How to overwrite a symbolic link?

2010-05-07 Thread Marc Herbert
Le 07/05/2010 15:21, Peng Yu a écrit :

 Would you please elaborate a little more on how to use LD_PRELOAD to
 modify the call. If the library (for example, 'open' from stdlib.h)
 is statically compiled in the binary, is LD_PRELOAD going to replace
 it with a different 'open' function?

Header files usually do not hold definitions (only declarations), so
header files usually do not matter to LD_PRELOAD.

LD_PRELOAD plays games with dynamic libraries, it cannot modify static
linkage. But AFAIK 'open' is defined in the libc, so I would be quite
surprised that its definition ends up statically linked into your binary.



PS: you are not supposed to include stdlib.h to get the declaration of 'open'.






gmane gateway

2010-05-05 Thread Marc Herbert
Le 16/04/2010 09:09, pk a écrit :
 it looks like the gateway between this 
 group and the mailing list hasn't worked for a long time. Is this intended? 
 Was it announced somewhere?

Are you referring to the gmane group? If yes it has been working
perfectly for me over the last few months.






Re: Passing variables by reference conflicts with local

2010-05-05 Thread Marc Herbert
 Except that it forks a subshell and consumes trailing newlines, and
 the whole point of this exercise is to avoid forks and spurious
 corruption of trailing newlines.
 
 I will try to find an alternative to eval $( )

The code below allows the callee to return any kind of values (including 
arrays).
It preserves newlines. This code does not try to hack some half-baked 
namespaces.
The optimized, coproc variant does not fork.


 Anything repetitive, done often enough, is observable even in an
 interactive environment.  The bash-completion library of functions does
 a _lot_ of repetitive work, and we're talking about some of the core
 functions used within that library, possibly even hundreds of times over
 the course of a single tab-completion event.

Then maybe it's time to consider a non-shell implementation. Could
some Bash plugin implement the critical path in C?


blackbox1()
{
local a=some private var
printf $1=%s; foo
printf $2=%s; '( some array )'
printf $3=%s; 'string with embedded
new lines
'
}

blackbox2()
{
local a=some other private var
printf $1=%s; bar
}

# BUG: with bash 4.0.35, repeatedly running this function *too fast*
# might error in Too many open files
forkingMain()
{
# Sourcing is safe because we trust our own blackbox functions
source (blackbox1 a b c)
echo --- blackbox1 returned:; declare -p a b c
source (blackbox2 a)
echo --- blackbox2 returned:; declare -p a
}

# Re-uses the same coproc subprocess to avoid forking
coprocMain()
{
local returnedDefs
echo 'blackbox1 a b c' ${COPROC[1]}
read -r -d $'\0' -u ${COPROC[0]} returnedDefs
eval $returnedDefs
echo --- blackbox1 returned:; declare -p a b c

echo 'blackbox2 a' ${COPROC[1]}
read -r -d $'\0' -u ${COPROC[0]} returnedDefs
eval $returnedDefs
echo --- blackbox2 returned:;  declare -p a
}

forkingMain

coproc { while read; do $REPLY; printf '\x00'; done; }
coprocMain






Re: Problem around brackets, and ||

2010-05-04 Thread Marc Herbert
Le 03/05/2010 21:08, Kunszt Árpád a écrit :

 http://mywiki.wooledge.org/BashPitfalls#cmd1_.26.26_cmd2_.7C.7C_cmd3
  
 I read it, thanks. I understand it now.
 
 I read man bash lots of times, but this behavior had escaped my
 attention. There isn't any word about cmd1  cmd2 || cmd3 only the cmd1
  cmd2 and cmd1 || cmd2 mentioned. I think lots of people, including
 myself, thought that  and || is an equivalent to the if ... then ...
 else statement.

Out of curiosity, what made you think that?

I find the following quite intuitive:

   true  false || this_cmd_is_run


true  false is obviously equal to false.



 So my bugreport is now against the documentation. :-)

This is already documented. Excerpts from the List of Commands
section:

*) AND and OR lists are executed with left associativity

Applied here:
   cmdA  cmdB || cmdC   =   { cmdA  cmdB ; } || cmdC


*) The return status of AND and OR lists is the exit status of the
  last command executed in the list

Applied here:   
Whenever cmdB is run and fails, { cmdA  cmdB ; } is
considered to have failed  (and cmdC is run)






Re: Passing variables by reference conflicts with local

2010-05-04 Thread Marc Herbert
Le 04/05/2010 14:40, Eric Blake a écrit :
 Except that in computing tab completion, side effects are _all_ that
 you want - basically, Freddie's problem is how to populate the
 global completion variables from within helper functions.

Of course you want a side-effect in the caller, and my example
achieves that.  I only moved the side-effect from the callee to the
caller, from where it is spurious to where it is wanted.


 Except that it forks a subshell and consumes trailing newlines, and
 the whole point of this exercise is to avoid forks and spurious
 corruption of trailing newlines.

I fail to see the link between completion and newlines but that's
probably just because I am not interested enough. I will try to find
an alternative to eval $( )

Are you are worried about the *cost* of the subshell? How can
interactive completion be performance-sensitive?

In shell there is really not much left you can do if you forbid
yourself from forking. The whole philosophy is based on small
cooperating processes isn't it?
  http://www.faqs.org/docs/artu/ch01s06.html
I guess this is why it still stands today after decades while kids
play with shared memory and race conditions.






Re: Strange behavior of IFS?

2010-04-15 Thread Marc Herbert
Le 15/04/2010 14:58, Clark J. Wang a écrit :

 I don't understand why the $string was still splitted into words since
 it's double quoted. Anyone can give a reasonable explanation?

set -x is often very good at giving explanations. Try this:

  sh -x foo.sh






Re: How cd to a directory with special characters like environment\\-?

2010-04-09 Thread Marc Herbert
Le 08/04/2010 22:58, Peng Yu a écrit :
 I make the following directory and try to cd to it. But I can't. Could
 you let me know what is the correct way of doing so?
 
 $ mkdir environment\\-
 $ cd environmen\\-
 -bash: cd: environmen-: No such file or directory

In such situations I find completion (TAB) really great since it does
the hard quoting work for you (and does no typo).







Re: manpage error-Arithmetic Evaluation of numbers with explicit base

2010-03-29 Thread Marc Herbert
Le 29/03/2010 14:50, Thomas Bartosik a écrit :

 Please don't get me wrong. I have no problem in understanding the
  man page this way, but I do think it is inconsistent.

It's a pity that square brackets are used both in the language itself
and in its syntactic definitions but this is bound to happen when
there is a fairly limited number of characters available.

So what do you suggest? Removing brackets from POSIX is going to be
quite challenging. Getting rid of them in syntactic definitions is
probably even less realistic considering how they are universally used
to describe optional elements (even in EBNF).

So I do not think you can do any better than trying to make sure the
context makes things obvious enough on a case by case basis. What are
your suggestions to make the Arithmetic Evaluation page more obvious?

By the way I do not think the Bash manual is meant for newbies. It is
a reference manual and there are tons and tons of other, free
resources better suited to newbies.


Cheers,

Marc






What is $[ ] ?

2010-03-23 Thread Marc Herbert
Hi,


 I could not find the dollar square brackets $[ ] expansion documented
in the manual. Is it the same as $(( )) ?

Cheers,

Marc



Le 22/03/2010 13:13, tbart...@gmx-topmail.de a écrit :
 Configuration Information [Automatically generated, do not change]:
 Machine: i686
 OS: linux-gnu
 Compiler: i686-pc-linux-gnu-gcc
 Compilation CFLAGS:  -DPROGRAM='bash' -DCONF_HOSTTYPE='i686' 
 -DCONF_OSTYPE='linux-gnu' -DCONF_MACHTYPE='i686-pc-linux-gnu' 
 -DCONF_VENDOR='pc' -DLOCALEDIR='/usr/share/locale' -DPACKAGE='bash' -DSHELL 
 -DHAVE_CONFIG_H   -I.  -I. -I./include -I./lib  
 -DDEFAULT_PATH_VALUE='/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin'
  -DSTANDARD_UTILS_PATH='/bin:/usr/bin:/sbin:/usr/sbin' 
 -DSYS_BASHRC='/etc/bash/bashrc' -DSYS_BASH_LOGOUT='/etc/bash/bash_logout' 
 -DNON_INTERACTIVE_LOGIN_SHELLS -DSSH_SOURCE_BASHRC -O2 -march=core2 
 -mfpmath=sse -msse4.1 -fomit-frame-pointer -pipe
 uname output: Linux blackknight 2.6.31-gentoo-r6 #1 SMP PREEMPT Mon Nov 30 
 19:03:35 CET 2009 i686 Intel(R) Core(TM)2 Duo CPU T9300 @ 2.50GHz 
 GenuineIntel GNU/Linux
 Machine Type: i686-pc-linux-gnu
 
 Bash Version: 4.0
 Patch Level: 35
 Release Status: release
 
 Description:
 The man page seems to be wrong regarding the handling of numbers with 
 different bases. It states one has to use [base#]n although it seems to be 
 [base#n] that needs to be used...
 (I know I do not have the latest bash, but with 4.1 it is the same in the 
 man page.)
 
 Repeat-By:
 blackknight ~ $ echo $[2#]10
 010
 blackknight ~ $ echo $[2#10]
 2
 
 Fix:
 Correct the man page







Re: Bash manual - interactive shell definition

2010-03-12 Thread Marc Herbert
 Could this sentence:

 An interactive shell is one started without non-option arguments,
 unless -sis specified, without specifying the
 -c option, and whose input and error output are both connected to terminals
 (as determined by isatty(3)), or one started with the -i option. 

 be any more confusing?
 
 Is seems pretty clearly stated to me.

Please enlighten us with the priority of English boolean operators.

I have never seen a natural language sentence with so many boolean operators.





Re: Return status of command substitution with $(...) gets lost

2010-03-11 Thread Marc Herbert
Chet Ramey wrote:

 To Posix, assignment statements never fail -- assignment errors cause
 non- interactive shells to exit, period.  In that case, it's possible
 to reflect the exit status of a command substitution in the exit
 status of a command consisting only of assignment statements,...

It's possible... and required by POSIX, right? 

At least this is my understanding of section 2.9.1 Simple Commands.
More precisely: ... with the exit status of the LAST command
substitution performed

Just to make things less abstract, here is some (simplified) code I
often find useful:

local _res
if _res=$( some_code... ); then
   echo $_res
elif _res=$( some_other_code... );
   echo $_res
else
   return 1
fi


 ... though that has problems of its own and is only reliable on commands
 consisting of a single assignment.

Here you seem to imply this is actually an advanced and thus
non-standard feature (why I have no clue). An unreliable feature would
never have been standardized for sure...






Re: Confused about how bash breaks input into words

2010-02-24 Thread Marc Herbert
Eric Blake a écrit :
 
 Another good reference is POSIX:
 http://www.opengroup.org/onlinepubs/9699919799/utilities/V3_chap02.html#tag_18_03

A less rigorous and easier reading is the Guide to Unix Shell Quoting:

http://www.mpi-inf.mpg.de/~uwe/lehre/unixffb/quoting-guide.html#para:sh-input-interp







Re: Is there a special variable for the directory where the script is in?

2010-02-12 Thread Marc Herbert
pk a écrit :
 Peng Yu wrote:
 
 $0 gives the file name of the script. I could use several shell
 command to get the directory where the script is in. But I'm wondering
 if there is an easy-to-use variable that refers to the directory where
 the script is in?
 
 See this page:
 
 http://mywiki.wooledge.org/BashFAQ/028

This is well informed, very useful and very interesting page is
considering the case where:

- you want your widely distributed and very portable script to be called
  in any way from anywhere (including from a pipe from Mars).
- hard-coding the location of your configuration files and libraries is
  not a problem.

I am sure this is the most common case. But this is definitely not my
case. Actually, my requirements are *the exact opposite*. For instance I
do not want anyone to use my script but me. So I just do this instead:

   source  $(dirname $0)/functions.sh

The fact that it might break whenever someone else uses my script in a
way I did not plan is a feature (in this respect, this code is even too
robust).





Re: process substitution and trailing file descriptors

2010-02-12 Thread Marc Herbert
Ian wrote:
  The manual suggests I could move and close file descriptors with
 
  [n]digit-
 
  but I would need the equivalent of
 
  command1 (...)-
 
  Digit might very well mean (just a) digit but here the process
 substitution, of course, is replaced with /dev/fd/63, say, certainly
 not a digit.
 
What about something like this (highly non-portable):

run_with_tee()
{
_run_with_tee  (tee out)  $@
}

_run_with_tee()
{
local fdpipeout=${1#/proc/self/fd/}; shift
# Redirect+close not to leak the extra pipe FD
$fdpipeout- $@
}

run_with_tee someCommand


Greg Wooledge wrote:
 When the shortcuts are too short, you need to fall back to the original
 tools.  In this case, () is really just a shortcut for create a FIFO,
 and open it.  Doing so by hand should give you the manual control you
 need.  At the very least, you can tell bash which FD number to use when
 you open the FIFO.

Yeah, but then you have the hassle to cleanup/delete the FIFO by yourself.
And leave tons of them behind in case of errors.






Re: + vs. [^/]* - Regular Expression bug?

2010-02-10 Thread Marc Herbert

 * means zero or more characters. It found zero and stopped.

I do not think that using a '*' star alone ever makes sense since it is
always matches.

The star is useful when anchored to something else. Like for instance
in:   .*Khodabocus

'+' is a star anchored to something.


Anyway this has nothing to do with bash.





Re: Error when script uses CRLF line endings w/ if stmt

2010-02-05 Thread Marc Herbert
Jan Schampera a écrit :

 Moreover, POSIX talks about newline here, which is a \n. Though I
 didn't read through all the rationales, I just took a quick look, maybe
 it's not limited to \n.

'\n' can be two characters outside of POSIX, see man fopen.

 -

It would feel good to use CRLF to bash Windows once again. Isn't
ridiculous and painful to have two characters for only one thing?

Unfortunately CRLF pre-dates Windows by decades. You can argue it is
older and more standard than LF. For instance many internet protocols
use it.

Now I wonder how come a third convention emerged. Just to think different?






Re: variable assignment in string returning function

2010-01-27 Thread Marc Herbert
Sharuzzaman Ahmat Raslan a écrit :
 
 Somehow, the backtick for foo() execute the function, echoing the correct
 output, but fails to set the variable $gang to the correct value.

The variable is set, but in a different sub shell. backticks fork a sub shell.

By the way $( ) is preferred to backticks, see FAQs





Re: exec, redirections and variable expansions

2010-01-26 Thread Marc Herbert
mike bakhterev a écrit :
 
 Description:
   Something is wrong with variable substitution in the exec invocation.
   When i try to close file descriptor whose number is in variable X with
   command:
 
   exec $X-

I think the operator is   2   as a whole; you cannot split the two characters.
Operators are parsed before parameter expansion.

In other words, what you are trying to do is a bit like:

I=i F=f 
# Does not work either!
$I$F true; then echo foo; fi


You need eval:

  eval $I$F true; then echo foo; fi # Works
  eval exec $X-   # Works





Re: Selecting out of an array

2010-01-26 Thread Marc Herbert
DennisW a écrit :
  Also, don't use ls like this - it's eyes-only.

Here is a demonstration:

touch filename with spaces
ARRAY=( $(ls) )  # BUG
for f in ${arr...@]}; do echo file: $f; done
ARRAY=( * )  # OK
for f in ${arr...@]}; do echo file: $f; done





Re: Using 'eval'

2010-01-26 Thread Marc Herbert
Gerard a écrit :
 This is probably a dumb question; however, I have a question
 regarding 'eval'.

If you know other programming languages, then it helps to consider
eval as a technique that generates code at run time (and
runs it immediately).





Re: best way to test for empty dir?

2009-12-14 Thread Marc Herbert
Matias A. Fonzo a écrit :
 On Fri, 11 Dec 2009 16:16:13 +
 Marc Herbert marc.herb...@gmail.com wrote:

 In case anyone is interested my winner (so far) is:

 exists()
 {
 [ -e $1 -o -L $1 ]
 }

 

 The -L is redundant.

Not for me. I need -L because I want to consider broken symlinks just
like anything else. A broken symlink would be a bug in my code and I want to
detect it ASAP.


 Because, if the symlink is not broken, the regular file exists ( -e ).

Please forget about correct symlinks. The -L is here for *broken*
symlinks.


 A solution to check the broken symlink is:
 
 [ -e foo -o -L foo -a ! -e foo ]

For which type of foo object does this return a different value than
the above? None.

If common sense is not enough, here is a formal proof that your third
and last test is redundant:

  -e or (-L and ! -e)  == (-e or -L) and (-e or ! -e)  distributivity
  (-e or -L) and 1 complements
   -e or -Lboundedness

http://en.wikipedia.org/wiki/Boolean_logic#Properties





Re: add a way to declare global variables

2009-12-14 Thread Marc Herbert
Bernd Eggink a écrit :
 To avoid misunderstandings, let me add that you are right (only) with 
 respect to variables being used _without_ declaration.

OK, but not having to explicitly declare variables is a feature that
most people expect from dynamic languages, so you can hardly blame them
for doing this all the time in shell.

IMHO Python gets this global/local (implicit) declaration story
right. Of course it is easier when you do not have 30 years of POSIX
legacy to support (but just the experience).

http://docs.python.org/tutorial/controlflow.html#defining-functions





Re: xtrace output on new file descriptor

2009-12-11 Thread Marc Herbert
Brian J. Murrell a écrit :
 
 Is equally difficult?

Do you mean equally difficult as in impossible? Then I would say no,
it looks easier :-)


 Or can I more easily play with FD duplication and redirection to
 achieve that, even if it means adding a word at the end of things
 I want on the real stdout?

I guess the answer is yes. You cannot redirect the output of set -x,
but you can redirect your own outputs.


# Duplicate a descriptor to stdout to escape redirections, like for
# instance in command substitution.
exec 31; PRESERVED_STDOUT=3

foo()
{
{
echo this is a message the user should see on stdout 
} ${PRESERVED_STDOUT}

echo this is the function return value
}

bar=$(foo)
echo bar==$bar





Re: best way to test for empty dir?

2009-12-11 Thread Marc Herbert
 empty_dir()
 { 
 test x$(echo $1/*$2) = x$1'/*'$2
 }

pk wrote:
 This fails if the directory contains a file called *. 

Yes. Unlike the ones below, empty_dir() above considers as empty a
directory that has a SINGLE element named '*'. Since I am not so
interested in files named '*', I think I can live with that!


Chris  others wrote:
 is_file2()
 { 
 for f
 do
 [ -e $f ]  return
 done
 return 1
 }

I think I like this one.

Andreas:
 This will still fail if the first file happens to be a dangling symlink.

Yes, but I think it's valuable to refine fail again. Dangling symlinkS
(not just the first one) will be ignored, just like they were not
here. Some might find this acceptable.


For purists, does this one works even better?

is_file3()
{ 
for f
do
[ -e $f -o -L $f ]  return
done
return 1
}



Thanks to everyone who answered, appreciated.





Re: best way to test for empty dir?

2009-12-11 Thread Marc Herbert
Sven Mascheck a écrit :
 Chris F.A. Johnson wrote:
 
 This has been discussed more than once in c.u.s; check the
 archives.
 
 and that's why we better discuss it here now?

I think Chris' message was more like: let's not discuss it at all and
just read the archives  :-]


In case anyone is interested my winner (so far) is:

exists()
{
[ -e $1 -o -L $1 ]
}

if exists foo/*; then
  for f in foo/*; do
...
  done
fi





Re: output redirection with process substitution asynchronous?

2009-12-08 Thread Marc Herbert
pk a écrit :
 
 I disagree. All the further changes in the requirements because creating a 
 subshell or being asynchronous is not acceptable etc. are not a goal in 
 themselves, but rather the indicators that he's trying to accomplish 
 something else.
 

I think he just want side-effects like in this recent example from Chris:

http://thread.gmane.org/gmane.comp.shells.bash.bugs/13863/focus=13907

Granted: if he was explaining in greater detail which side-effects he wants,
people might be able to suggest better alternatives.






Re: output redirection with process substitution asynchronous?

2009-12-07 Thread Marc Herbert
 pjodrr wrote:
 It would be nice if you explained what it is you're attempting to do, rather 
 than ask for a solution for what you're thinking would do that.

To be honest that is the first thing he (tried to) do:

pjodrr wrote:
 how can I prefix every line of output of some command with a
 timestamp? 


What is wrong with the following:

prefix_with_date () 
{ 
while read; do
printf '%s: %s\n' $(date) $REPLY;
done
}

seq 4 | prefix_with_date
ls | prefix_with_date





Re: output redirection with process substitution asynchronous?

2009-12-07 Thread Marc Herbert
Marc Herbert wrote:
 What is wrong with the following:
 
 prefix_with_date () 
 { 
 while read; do
 printf '%s: %s\n' $(date) $REPLY;
 done
 }
 
 seq 4 | prefix_with_date
 ls | prefix_with_date

Sorry I missed the fact that you want to run your commands in the current shell.


There are no real coroutines in shell. The current shell process does
not know how to schedule two pieces of code. So each time two
pieces of code communicate through a pipe they have to be run in two
*concurrent* processes (typically: subshells).  The producer and
consumer of a pipe must run independently of each other. Whether you
are using process substitution or the more usual and portable pipe |
does not matter here: you need concurrency.

So at least one of: 1. your prefixer code, 2. your unknown command
has to run in a independent process, asynchronous with the current
shell.

Since you absolutely want your unknown commands to run in the current
shell, then it is your prefixing code that has to run in a
concurrent process.

Now I do not really see any other way to avoid the ugliness of
concurrently printing on stdout than to wait for your concurrent
prefixer to complete, more or less like you did.


A variant is to ask socat to handle the cleanup actions for you like
this:

prefix_with_date()
{
local P=/tmp/dataorig.$$
socat -u  PIPE:${P}  SYSTEM:'while read; do echo $(date):\\ $REPLY; done' 
  
socatPID=$!
until [ -e ${P} ]; do sleep 1; done
$@  ${P}
wait $socatPID
}

prefix_with_date  seq 5






Re: bash 3.2.39 -- race condition

2009-11-30 Thread Marc Herbert
Douglas Moyes a écrit :
 I nearly brought down a server today.
 
 # alias d=perl-script 
 

A long time ago I read somewhere that aliases are not recommended. I
think they do not bring anything on the table compared to functions, do
they?

Anyway I have always done without them, but without understanding what I
gained. Now I know a bit better, thanks!





Re: IFS handling and read

2009-11-30 Thread Marc Herbert
Eric Blake a écrit :
 
 This is E4 in the FAQ:
 ftp://ftp.cwru.edu/pub/bash/FAQ
 
 POSIX permits, but does not require, that the final element of a pipeline
 be executed in a subshell.  Bash uses the subshell, ksh does not.
 Variable assignments in a subshell do not affect the parent.

I am regularly bitten by this. This is a major pain; it makes read very
inconvenient to use (whatever IFS is).

Could this be changed in the future?

Cheers,

Marc





Re: IFS handling and read

2009-11-30 Thread Marc Herbert
Lhunath (Maarten B.) a écrit :
 On 30 Nov 2009, at 11:34, Marc Herbert wrote:
 
 Eric Blake a écrit :
 This is E4 in the FAQ:
 ftp://ftp.cwru.edu/pub/bash/FAQ

 Instead of ''commands | read var''
 Use ''read var  (commands)''
 I hardly see a need to change the existing implementation.

As mentioned in the FAQ, ''read var  (commands)'' is not portable.

All alternatives in the FAQ (portable or not) are less readable than a
simple pipe. They are all more verbose and introduce an extra level of
nesting when you have only one command. They all need to be read
backwards with respect to the execution flow. If you want to keep your
code readable, they practically all force you to define a function for
commands as soon as you have more than a few commands.

Every entry in an FAQ is by mere definition a problem that many people
wast... spend time on.

It is admittedly not a question of life or death but some other shells
apparently have it so why not bash? Just asking.





Re: IFS handling and read

2009-11-30 Thread Marc Herbert
Chris F.A. Johnson a écrit :
Why should it be the last element of a pipeline that is executed in
the current shell and not the first?


Because that's POSIX' choice?


Because the last element is the last one in the data stream. So it feels
more natural to get everything from the last element rather than side
effects from the first and stdout from the last.



Suppose that I have a group of commands that sets some variables
and outputs information to the screen, for example (this is much
oversimplified):

Thanks for the example. I find this less common than using read.





3. Basic Shell Features versus POSIX features

2009-11-26 Thread Marc Herbert
Hi,

 I more or less assumed that section 3. Basic Shell Features is all
POSIX, in contrast to section 6. Bash Features which starts like this:
This section 6 describes features unique to Bash.

I think I was wrong. There seems to be at least two non-POSIX features
documented in the Basic section (brace expansion and process
substitution).

Does the manual make any attempt to draw a line between POSIX and
non-POSIX features? Does the unique to bash introduction sentence
really make sense since most of these features are also found in other
shells? Thanks in advance...


Cheers,

Marc





Re: Error handling question

2009-11-10 Thread Marc Herbert
Chris F.A. Johnson a écrit :

 It isn't desirable. However, commands will sometimes fail; they
 are intended to fail. Their failure provides necessary information
 to the script.

Because it lacks proper exceptions, the language is indeed creating a
confusion between exceptional errors and normal return values. But I
found it surprisingly easy to deal with this confusion in practice.


 For example, if you want to know whether user 'john' is in the
 password file:
 
 grep -q ^john: /etc/passwd

 You want to get the return code and execute code depending on the result

I do.

 (which can be 0, 1 or 1; 'if grep ...' is not adequate).

I find if grep ... adequate. Ignoring 1 here is just another small
hole in the set -e safety net (and it will log an error message).

In the few cases where I am really afraid of exceptional errors in a test
I can check the pre-conditions by myself. For instance:

 set -e
  ...
 [ -r /etc/passwd ] || fatal could not read /etc/passwd
 if grep -q ^john: /etc/passwd; then ...


But honestly, how many times in real life have you seen code actually
checking for errors of commands as simple as grep /etc/passwd ?


 Then you are not taking advantage of the power of the shell. Using
 a return code provides more than a two-way branch.

I do not consider exceptions as an actual branch, which makes most
commands two-way branch or less. Whenever I want to return something
more complex than 0 / 1 then I do not use return at all but printf
for instance.  An integer is too poor anyway; so I use exit statuses
only for simple boolean tests xor exceptions. The only minor issue is
that it is not possible for boolean tests to throw exceptions: each
boolean test creates a small hole in the safety net. I can live with
that.

Do you have an other example than grep maybe?


 Ignoring errors should be the exception and not the rule.
 
 Not using set -e does not imply ignoring errors; one *uses*
 errors.

For the clarity of this discussion could you please refrain to confuse
exceptional errors with normal return values of tests? I know that the
design of the language creates this confusion in the first place, but I
think the documentation does not and sticks to the English meaning. See
for instance man grep:

EXIT STATUS
The following exit values are returned: 
0
  One or more lines were selected. 
1
  No lines were selected. 
1
  An ERROR occurred.


Cheers,

Marc





Re: Error handling question

2009-11-10 Thread Marc Herbert
Chet Ramey a écrit :
 The fact that set -e is not the default looks like a big design
 mistake to me.
 
 Wow.  That ship sailed 30 years ago.

but forgot set -e on the quay?


See also this ship:
http://www.reddit.com/r/programming/comments/7l89a/null_references_the_billion_dollar_mistake/





Re: Error handling question

2009-11-10 Thread Marc Herbert
Marc Herbert a écrit :
 Chris F.A. Johnson a écrit :
 For example, if you want to know whether user 'john' is in the
 password file:

 grep -q ^john: /etc/passwd

 You want to get the return code and execute code depending on the result

 (which can be 0, 1 or 1; 'if grep ...' is not adequate).
 
 I find if grep ... adequate. Ignoring 1 here is just another small
 hole in the set -e safety net (and it will log an error message).


Just found this, even better: not even any extra hole in the set -e
safety net.


  set -e
  ...
  if grep -q ^john: /etc/passwd; then
  echo FOUND
  else grepstatus=$?
  # inspect $grepstatus for errors
  fi






Re: [PATCH] silent job monitor when 'set +m'

2009-11-09 Thread Marc Herbert
Chet Ramey a écrit :
 Sure.  Since the status messages are written to stderr, you can save
 file descriptor 2 and temporarily (or permanently, depending on your
 needs) redirect it to /dev/null.

 That means another subshell.
 
 It doesn't require a subshell to save and restore stderr.

At first I thought the same. And then I read this:

http://bash-hackers.org/wiki/doku.php/snipplets/kill_bg_job_without_message

  It's also useless to temporarily redirect stderr when you call the
  kill command, since the successful termination of the job, the
  termination of the kill command and the message from the shell may not
  happen at the same time. And a blind sleep after the kill would be
  just a workaround.


Doesn't some permanently redirected subshell solves this race condition?





Re: Error handling question

2009-11-09 Thread Marc Herbert
Greg Wooledge a écrit :
 
 The problem being how to use set -e in a consistent manner across all
 shells?  You can't.  set -e is unpredictable, unreliable, and should be
 shunned.  As you can see by the last dozen or so message on this mailing
 list, not even bash gurus (other than Chet) can figure out its semantics.
 

The fact that set -e is not the default looks like a big design
mistake to me.  I cannot understand how ignoring errors in a program and
going on like nothing happened is a desirable thing. When I write a
succession of commands in a script, a later command depends on the
success of earlier commands more often than not and there is no point
going on with the rest of script. Ignoring errors should be the
exception and not the rule. The same can probably be said for C.


 Check for errors yourself on the commands that matter.  That way you'll
 know what will cause termination and what won't.  Yes, I know, it means
 your script is longer.  But longer is better than unpredictable.

ALL my commands matter so I practically cannot check the return status
of everyone of them.  Instead I use set -e systematically. It
works. It is unpredictable but I do not to care: a safety net with a few
holes is way better than none at all. The very few times I actually want
to ignore an error I just append  || true to the corresponding
command.






Re: [OT] Re: how to start in overwrite-mode

2009-10-29 Thread Marc Herbert
Chet Ramey a écrit :
 Functions were the last thing added to sh by Steve Bourne before he
 left Bell Labs (he finished in 1983)

From Steve Bourne himself:
http://www.computerworld.com.au/article/279011/-z_programming_languages_bourne_shell_sh?pp=3

(very interesting interview)






Re: slight OT: shell-script programing style -- origins and change?

2009-10-27 Thread Marc Herbert
Stephane CHAZELAS a écrit :
 By convention, _environment_ variables are upper-case, which helps
 in a shell script to distinguish them with ordinary variables. I
 would consider shell scripts with all upper-case variables to be
 poorly written, and I agree with the fact that it looks like
 yelling and is less easy to read.

 It's OK to use upper-case for environment variables as you
 generally want to have the environment as small as possible.


This looks indeed like a larger Unix tradition than just a shell script
one. John mentioned C macros, but Sun's Java conventions also recommend
UPPER_CASE for constants. Hasn't Perl the same convention?

Here I purposely confuse globals and constants since they overlap a lot
in practice (the less mutable are the globals, the safer).

In languages lacking actual namespaces it is a useful convention to have
all globals isolated into the well-known upper case namespace.






Re: 100% cpu with: while true ;do read /dev/null ;done

2009-10-23 Thread Marc Herbert
Jo King a écrit :
 thanks for reply. agree it's not a
 bug with bash.

 i am inclinded to agree with the comments but bash is
 popular and attracts a lot of newbies into scripting - a bit
 of smart 'correction' of their wrong ways would at least
 keep the cpu available for other processes (even a sleep for
 100ms would make a huge difference).

Newbies should just stay clear from developing any advanced shell
script. There is way too much rope too hang themselves and your example
is just one among many other ways. There is plenty other, newer
programming environments that are far easier and safer to play with and
learn.

By the way the same hold for configuring firewalls. It becomes even
worse when it relates to security. (Security by newbies... ahem,
really?)

Cheers,

Marc




Re: cp command will copy to subdirectory without appending /

2009-10-23 Thread Marc Herbert

 The problem is dead-simple. You cannot run this command multiple times:
 
 cp -R  ./dirfoo  ./dirfoo.backup
 

Sorry to add yet another (last) off topic message but I must do justice
to GNU cp.  GNU cp actually does support the above thanks to a
(non-standard) option:

 cp -R -T ./dirfoo  ./dirfoo.backup

For details see:  info coreutils Target Directory





Re: set -e in bash 4

2009-10-21 Thread Marc Herbert
Tobias Poschwatta a écrit :
 In the following test case test1.sh, bash 4 behaves differently from
 bash 3.1. The behaviour of bash 3.1 is what I would expect to be
 correct.  Is this a bug in bash 4 or an intentional change?

Isn't this related to this change?
http://thread.gmane.org/gmane.comp.shells.bash.bugs/13465






Re: cp command will copy to subdirectory without appending /

2009-10-21 Thread Marc Herbert
Bob Proulx a écrit :
 Todd Partridge wrote:
 
 If the target has an appended '/' then the destination must be a directory.

But with cp (GNU coreutils) 6.12, appending '/' to the target does not seem
to make any difference?

 
 I think the proper behavior here for 'cp -R test2 test' would be to
 error and print that 'Folder already exists'.
 
 Of course that would break decades of scripts which expect the
 standard behavior.  I don't understand why would you change this long
 standing useful behavior.  Could you say a few words more about what
 your problem is with the current behavior?

The problem is dead-simple. You cannot run this command multiple times:

cp -R  ./dirfoo  ./dirfoo.backup

That's all.


The usual workaround is to rm -rf ./dirfoo.backup systematically. It
is tedious and easy to forget. And it prevents adding any non-dirfoo
content to dirfoo.backup.

cp is too clever. It should not behave in a completely different way
depending on the external environment. It should ideally depend only on
the user input (that is: trailing slash or not). pedanticThis is
called Referential transparency/pedantic

rsync's opinion on this topic is interesting. The rm -rf workaround is
obviously not acceptable for rsync. But rsync did not want to depart
from existing syntax either. So rsync added its own, somewhat funny
workaround: appending a trailing slash to the source directory.

 
 Appending a / would imply the directory:
 cp -R test2 test/
 This usage will remove the ambiguity of the command between the copy
 function and the rename function.


 Please rephrase your question and send it to bug-coreut...@gnu.org.

This is probably not worth the hassle considering that any change would
break decades of scripts.






Re: process substitution

2009-10-15 Thread Marc Herbert
 On Mon, Oct 12, 2009 at 1:14 PM, Ralf Goertz
 is it possible to have a process substitution with both input and output
 redirection? So far I use the following work-around

 cat parentprocess.sh:
 #!/bin/bash
 mkfifo fifo 2/dev/null
 exec 5 (./subprocess.sh  fifo)
 exec 6 (cat  fifo)

I think exec 6fifo is enough here?


 Is there another way to do that, something like fork, which wouldn't use
 a named pipe explicitely?

socat provides a terse solution where you still have to name the
pipes, but their clean-up comes for free. Moreover it works with any
shell.

socat is very flexible, below is only one of the ways to do it.


#!/bin/sh

socat EXEC:./subprocess.sh  PIPE:tosub.$$,nonblock!!PIPE:fromsub.$$

# The alternative to polling is creating the pipes in
# advance. But this would require a finally clause to clean them.
until test -p fromsub.$$; do sleep 1; done

exec 6fromsub.$$
exec 5tosub.$$

echo input to subprocess 15
echo done sending input
# [etc.]




Re: Infinite loop on clearing PWD

2009-10-13 Thread Marc Herbert
Jan Schampera a écrit :
 Marc Herbert schrieb:
 
 Repeat-By:
 $ unset PWD
 Fix = readonly PWD  ?
 
 This is not a fix, this is a workaround.

Sorry Jan: my suggestion was not addressed to you, it was addressed to the 
maintainer.

The idea is to make PWD read-only *by default*.






Re: Infinite loop on clearing PWD

2009-10-12 Thread Marc Herbert
jeng...@medozas.de a écrit :
 Description:
   bash enters into an infinite loop when the environment variable PWD
   is unset. Probably not a smart thing to do, but since this variable
   is so important, it should probably be ensured that it does not get
   an empty value. (It does handle non-absolute paths in some random
   fashion, too.)
 
 Repeat-By:
   $ unset PWD

Fix = readonly PWD  ?








Re: ignoring comments in a 'cat' call

2009-10-09 Thread Marc Herbert
Bob Proulx a écrit :
  Tamlyn1978 wrote:
  Is there a way to make 'cat' ignore the comments or is there a better
  alternative to cat in this case?
  
  Note that if 'cat' didn't exactly reproduce the contents of input on
  the output I would consider that a grave bug.

Well... GNU cat has a number of options, and almost every single one is
munging the output  :-) 

 
  Instead of using $(cat SOMEFILE) it is better to avoid the extra
  process and use $( SOMEFILE) instead.  It is built into the shell and
  is the right way to do that task.

It is more efficient but I think it is not standard.


Cheers,

Marc





Re: redirection after braced block

2009-10-06 Thread Marc Herbert
Greg Wooledge a écrit :
 On Mon, Oct 05, 2009 at 07:55:33PM +0200, clemens fischer wrote:
   {
   ... a number of commands
   } 21 | ${prog_log} ${logfile}
 

 If you're having trouble debugging your script, and you want to be
 absolutely sure whether the commands in the braces are executing, you
 should use commands that affect the file system (touch, etc.).

I tend to do it like this:

{
   exec 2/tmp/mytrace.$$
   set -x
   ...

} | whatever






Re: ulimit and ssh?

2009-09-14 Thread Marc Herbert
peter360 a écrit :
 
 Thanks for the explanation.   So my understanding of the way ssh works is
 still incorrect.  I am confused about at which point the two parameters,
 -c and ulimit -a were converted into three, -c, ulimit, and -a.  I
 guess I need to read the source of ssh and bash to really understand this...

Enabling tracing options usually helps a lot.


Instead of this:
sshlocalhost bash-c 'ulimit -a'

Try this:
ssh -v localhost bash -x -c 'ulimit -a'


(And as usual, check out the Guide to unix shell quoting)




Re: how to keep newlines in ``

2009-08-27 Thread Marc Herbert
Greg Wooledge a écrit :
 On Wed, Aug 26, 2009 at 03:04:16PM -0400, Sam Steingold wrote:
 foo=`ls`
 echo $foo
 
 echo $foo

What Greg did not explicitly said:

95% of the time you write $foo unquoted, you are wrong and should have
written double-quoted $foo instead.

This not just about newlines. See A Guide to Unix Shell Quoting for
details: http://www.mpi-inf.mpg.de/~uwe/lehre/unixffb/quoting-guide.html

Another better practice is to use printf, since $foo could start with a
minus sign:

  printf '%s' $foo




Re: bug with 'set -e' + 'trap 0' + syntax error

2009-08-21 Thread Marc Herbert
Stefano Lattarini a écrit :

 I thought that when bash detect a syntax errors in a script,
 it would pass a $? != 0 to the code in the exit trap, regardless
 of whether `set -e' is active or not.

 
 I think this can be classified as a bug in bash (in some
 situations, a very nasty one). Please let me know if I
 have misunderstood something, or if there is a simple
 workaround.

I am not familiar with the standard(s), but... isn't having expectations in
the case of syntax errors a bit far-fetched?





Re: Assigning to multiple variables on same line

2009-08-11 Thread Marc Herbert
Yang Zhang a écrit :
 Why does assigning to multiple variables on the same line exhibit 
 sequential consistency normally but not for local variables?

You might be interested in another difference:

g () 
{ 
local x
x=$(exit 3); echo $?
local   y=$(exit 3); echo $?
}

$ g
3
0

I now tend to prefer splitting the local declaration from the variable 
definition...






Re: $var invokes function?

2009-08-10 Thread Marc Herbert
BuraphaLinux Server a écrit :
 Not exactly what you asked for, but works the same:
 
 #! /bin/bash
 today() {
   date
 }
 
 printf today is %s\n $(today)
 exit 0
 
 It is easier to just use $(date) directly though.

More direct, but less flexible.


printf arbitrary date is %s\n $(dateFunc)

dateFunc() { date; }# today
dateFunc() { echo 2009-1-2; }
dateFunc() { whatever; }






Re: Bash does not read up the whole script which it is currently executing

2009-08-06 Thread Marc Herbert
Ivan Zahariev a écrit :

 Same here, as Marc said.
 
 I think that usually we are reading the script from a file and this is 
 the use-case we must focus on. Currently, we have the problem I 
 described when executing a script from a file and I think this must be 
 fixed/changed.

Hey, wait. I am not saying that this must be changed. I can imagine
there are good reasons for the current behaviour (like for instance
performance or conformance).

Just not the reason put by John.





Re: Bash does not read up the whole script which it is currently executing

2009-08-05 Thread Marc Herbert
John Reiser a écrit :
 On 08/04/2009 12:48 AM, fam...@icdsoft.com wrote:
  First I would like to say that I'm not sure if this is a bug or a 
 feature of Bash.
  If it is a feature, please let me know how to turn it off; or better 
 make it disabled by default...
  
  The problem is that Bash does not read up the whole script which it is 
 currently executing.
  As a result of this, if we update the script file with a newer version 
 while it is running, this may lead to unpredicted results.
 
 It is an intended design feature that the shell reads only as much as 
 necessary
 to proceed at the moment.  This is required so that subprocesses can share 
 the same
 stdin as the shell.  Bytes for input to the shell alternate with bytes for 
 input
 to each subprocess that does not have redirection (or closure) specified for 
 its stdin.

I am not sure I get this... first of all, the script itself is usually
not read from stdin (but from fd 255 in bash).

Now considering the seldom cases where the script is actually read from
stdin, are you saying that: it is a wanted feature that subprocesses can
concurrently slurp and steal the code executed by their parent? Wow,
that sounds really weird and not something I would wish, but rather a
design limitation that I should constantly worry about.

Regards,

Marc








read -t 0 not supported? (poll)

2009-07-22 Thread Marc Herbert
Hi,

  It seems polling using read -t 0 is not supported. Apparently because the 
implementation is based on alarm() (as opposed to be based on poll()/select()).

- is this true?
- if yes, doesn't this deserve more explicit documentation?

Would anyone know about an alternative, convenient way to poll() ? Thanks in 
advance!

Cheers,

Marc




3-lines long segfault

2009-07-15 Thread Marc Herbert

The following script crashes bash versions 2.05, 3.2.25, 4.0.16,...


trap echo USR1caught USR1
read -t 1
foo=$( kill -USR1 $$ )
echo 'Yeah! no Segmentation fault!'



Cheers,

Marc