ANNOUNCE: ParBASH 0.1 release - parallel processing in BASH
Hello,
I'd like to announce the release of the 0.1 version of ParBASH. Using
ParBASH, it is possible to write bash scripts that can be
automatically parallelized on SMP, multicore, and distributed systems
using Apache Hadoop.
Here is an example script to find top 10 references for Barack Obama
pages on wikipedia using Amazon EC2:
wiki.sh:
cat hdfs:/wikipedia-out/* | grep Obama | \
perl -ne 'while (//g) { print
"$1\n"; }' |\perl -ne 'if (/http:\/\/([^\/]+)(\/|$)/) { print
"$1\n"; }' |\
perl -ne '
if (/([^\.]\.)+([^\.]+\.[a-zA-Z]{2,3}\.[^\.]+)$/) { print "$2\n";}
else if (/([^\.]+\.[a-zA-Z]{2,3}\.[^\.]+)$/) { print "$1\n";}
else if (/([^\.]\.)*([^\.]+\.[^\.]+)$/) { print "$2\n"; }' |\
sort | uniq -c > hdfs:/out
How and why of wiki.sh and parbash on
http://cloud-dev.blogspot.com/2009/06/introduction-to-parbash.html
Source code and more examples:
http://code.google.com/p/parbash
If someone wants to try compiling the code and play around with it,
please contact me, I can help you get started.
Thanks,
Milenko
Re: Help with script - doesn't work properly from cron
Thanks everyone for your pointers - I ended up with this:
#!/bin/bash
BACKUP_DIR="/media/disk/AUTOMATED_BACKUPS/DB_DAILY"
CURRENT_DIR=$BACKUP_DIR/`date +%d`
DATABASES="$(mysql -uroot -pNewSecretPw -Bse 'show databases')"
echo 'Backing up databases: '$DATABASES
cd /
rm -rf "$CURRENT_DIR"
mkdir -p "$CURRENT_DIR"
cd "$CURRENT_DIR" || exit 1
for DB in $DATABASES
do
mysqldump -uroot -pNewSecretPw "$DB" | bzip2 > "${DB}_$(date
+%Y-%m-%d_%H:%M).sql.bz2"
done
exit 0
It worked on the first manual run - keeping my fingers crossed for tonight!
:-P
Bob Proulx wrote:
>
> Greg Wooledge wrote:
>> Erik Olof Wahlstrom wrote:
>> > /usr/bin/mysqldump -uroot -pHardAsMySql321 "$DB" | bzip2 >
>> > "$DB"_`date +%Y-%m-%d_%k.%M`".sql.bz2"
>>
>> # Long line, probably broken by your mailer. For clarity, I'd
>> # write it on two lines explicitily:
>>
>> mysqldump -uroot -pHardAsMySql321 "$DB" |
>> bzip2 > "${DB}_$(date +%Y-%m-%d_%k.%M).sql.bz2"
>
> You might want to set umask restrictive so that the backup file
> created by the redirection isn't world readable.
>
> umask 077
>
> I have never liked whitespace in filenames and %k may produce spaces
> in filenames. Better IMNHO to use %H here and avoid the space. Why
> use a dot instead of the more common colon in the time?
>
> mysqldump -uroot -pHardAsMySql321 "$DB" |
> bzip2 > "${DB}_$(date +%Y-%m-%d_%H:%M).sql.bz2"
>
>> You could also consider writing it this way:
>>
>> cd /
>> rm -rf "$CURRENT_DIR"
>> mkdir -p "$CURRENT_DIR"
>> cd "$CURRENT_DIR" || exit 1
>>
>> Then you don't even need to check whether it already exists.
>
> Why bother changing directory there at all? :-)
>
> mysqldump -uroot -pHardAsMySql321 "$DB" |
> bzip2 > "$CURRENT_DIR/${DB}_$(date +%Y-%m-%d_%H:%M).sql.bz2"
>
>> Bernd Eggink wrote:
>> > You could replace the whole if-then-else clause by
>> >
>> > mkdir -p $CURRENT_DIR
>> > cd $CURRENT_DIR
>> > rm -f *
>>
>> Another failure to check the results of "cd" before doing "rm *". This
>> can and will lead to disasters.
>
> Doing 'rm *' just makes me extremely nervous even when care is taken
> to ensure that it is going to work okay. I think it is safer to avoid
> it. Let me suggest the following:
>
>> > CURRENT_DIR=$BACKUP_DIR/`date +%d`
>> ...
>> cd "$CURRENT_DIR" || exit 1
>> ...
>> mysqldump -uroot -pHardAsMySql321 "$DB" |
>> bzip2 > "${DB}_$(date +%Y-%m-%d_%k.%M).sql.bz2"
>
> In this case the file name is going to be SOMETHING.sql.bz2. Instead
> of using 'rm *' it would make me much less nervous to qualify that
> file glob with 'rm -f *.sql.bz2' at least so that if this were to escape
> and be invoked elsewhere it would have limited damage potential.
>
> cd "$CURRENT_DIR" || exit 1
> rm -f *.sql.bz2
>
>> > Is there a better way to clear out last months files before making the
>> > current backups?
>
> Personally I have the following line in my /etc/cron.d/local-mysql
> crontab. The 'savelog' command with -c7 will save seven days of
> datestamped backups (sufficient for my case) without further code.
> That could easily be -c30 to save 30 days worth. The 'savelog'
> command pretty much handles all of your issues with datestamping and
> cycling files very easily.
>
> 30 3 * * * root umask 077 ; mysqldump
> --defaults-file=/etc/mysql/debian.cnf --all-databases | gzip >
> /var/backups/mysql.dump ; savelog -q -d -l -C -c7 /var/backups/mysql.dump
>
> Bob
>
>
>
>
--
View this message in context:
http://www.nabble.com/Help-with-script---doesn%27t-work-properly-from-cron-tp24542164p24579084.html
Sent from the Gnu - Bash mailing list archive at Nabble.com.
Re: 3-lines long segfault
trap "echo USR1caught" USR1 read -t 1 foo=$( kill -USR1 $$ ) echo 'Yeah! no Segmentation fault!' >>> Thanks for the report. >> Thanks for bash in the first place. >> >> >>> Fixed in the next version. >> Chet, is there any workaround for existing versions? > > Sure. There's the usual idiom. It's unusual to see a script that uses > command substitution for this. [...] > In general, if you want to do something like set an alarm to go off in > the future, [...] I want not. I think you did not (and could not) guess what I want to perform here. These 3 lines were isolated (with great pain) from a script several hundreds lines long. So let me flesh it a little bit: export MASTER_PID=$$ trap "incrementSomeCounter" USR1 ... read -t 1 # code totally unrelated to the trap ... foo=$( ... kill -USR1 $MASTER_PID # increment parent's counter ... kill -USR1 $MASTER_PID # increment parent's counter again ... echo "returnValue" ) >> I mean, there must be some memory corruption involved in here, so is >> it possible to dodge it without upgrading? For instance, would simply >> avoiding the "-t" option of "read" be enough to guarantee no memory >> corruption? Other? Thanks a lot in advance! > It's a combination of the assignment statement and the trap handling. > Yes, there is memory corruption that occurs, because the trap handler > doesn't unwind-protect a global variable used by the expansion and > execution code. Sorry I do not understand "unwind-protect". In more practical terms, are you saying here that: - removing the "-t" option from "read" dodges the immediate crash, but not the memory corruption? So bash might still crash later? - in other words, triggering a trap from a command expansion will always corrupt memory? (I mean: without your recent fix) Cheers, Marc
Re: 3-lines long segfault
Marc Herbert wrote:
> Chet Ramey a écrit :
>> Marc Herbert wrote:
>>> The following script crashes bash versions 2.05, 3.2.25, 4.0.16,...
>>>
>>>
>>> trap "echo USR1caught" USR1
>>> read -t 1
>>> foo=$( kill -USR1 $$ )
>>> echo 'Yeah! no Segmentation fault!'
>> Thanks for the report.
>
> Thanks for bash in the first place.
>
>
>> Fixed in the next version.
>
> Chet, is there any workaround for existing versions?
Sure. There's the usual idiom. It's unusual to see a script that uses
command substitution for this. Most existing scripts use simple
subshells to do it, whether (command) or asynchronous commands using `&'.
In general, if you want to do something like set an alarm to go off in
the future, say after 10 seconds, you do something like
{ sleep 10; kill -USR1 $$; } &
and perform a potentially blocking operation.
> I mean, there must be some memory corruption involved in here, so is it
> possible to dodge it without upgrading? For instance, would simply avoiding
> the "-t" option of "read" be enough to guarantee no memory corruption? Other?
> Thanks a lot in advance!
It's a combination of the assignment statement and the trap handling.
Yes, there is memory corruption that occurs, because the trap handler
doesn't unwind-protect a global variable used by the expansion and
execution code.
Chet
--
``The lyf so short, the craft so long to lerne.'' - Chaucer
Chet Ramey, ITS, CWRUc...@case.eduhttp://cnswww.cns.cwru.edu/~chet/
Re: Help with script - doesn't work properly from cron
Greg Wooledge wrote:
> Erik Olof Wahlstrom wrote:
> > /usr/bin/mysqldump -uroot -pHardAsMySql321 "$DB" | bzip2 >
> > "$DB"_`date +%Y-%m-%d_%k.%M`".sql.bz2"
>
> # Long line, probably broken by your mailer. For clarity, I'd
># write it on two lines explicitily:
>
> mysqldump -uroot -pHardAsMySql321 "$DB" |
> bzip2 > "${DB}_$(date +%Y-%m-%d_%k.%M).sql.bz2"
You might want to set umask restrictive so that the backup file
created by the redirection isn't world readable.
umask 077
I have never liked whitespace in filenames and %k may produce spaces
in filenames. Better IMNHO to use %H here and avoid the space. Why
use a dot instead of the more common colon in the time?
mysqldump -uroot -pHardAsMySql321 "$DB" |
bzip2 > "${DB}_$(date +%Y-%m-%d_%H:%M).sql.bz2"
> You could also consider writing it this way:
>
> cd /
> rm -rf "$CURRENT_DIR"
> mkdir -p "$CURRENT_DIR"
> cd "$CURRENT_DIR" || exit 1
>
> Then you don't even need to check whether it already exists.
Why bother changing directory there at all? :-)
mysqldump -uroot -pHardAsMySql321 "$DB" |
bzip2 > "$CURRENT_DIR/${DB}_$(date +%Y-%m-%d_%H:%M).sql.bz2"
> Bernd Eggink wrote:
> > You could replace the whole if-then-else clause by
> >
> > mkdir -p $CURRENT_DIR
> > cd $CURRENT_DIR
> > rm -f *
>
> Another failure to check the results of "cd" before doing "rm *". This
> can and will lead to disasters.
Doing 'rm *' just makes me extremely nervous even when care is taken
to ensure that it is going to work okay. I think it is safer to avoid
it. Let me suggest the following:
> > CURRENT_DIR=$BACKUP_DIR/`date +%d`
> ...
> cd "$CURRENT_DIR" || exit 1
> ...
> mysqldump -uroot -pHardAsMySql321 "$DB" |
>bzip2 > "${DB}_$(date +%Y-%m-%d_%k.%M).sql.bz2"
In this case the file name is going to be SOMETHING.sql.bz2. Instead
of using 'rm *' it would make me much less nervous to qualify that
file glob with 'rm -f *.sql.bz2' at least so that if this were to escape
and be invoked elsewhere it would have limited damage potential.
cd "$CURRENT_DIR" || exit 1
rm -f *.sql.bz2
> > Is there a better way to clear out last months files before making the
> > current backups?
Personally I have the following line in my /etc/cron.d/local-mysql
crontab. The 'savelog' command with -c7 will save seven days of
datestamped backups (sufficient for my case) without further code.
That could easily be -c30 to save 30 days worth. The 'savelog'
command pretty much handles all of your issues with datestamping and
cycling files very easily.
30 3 * * * root umask 077 ; mysqldump --defaults-file=/etc/mysql/debian.cnf
--all-databases | gzip > /var/backups/mysql.dump ; savelog -q -d -l -C -c7
/var/backups/mysql.dump
Bob
Re: bash 4.x filters out environmental variables containing a dot in the name
Hi Chet, Chet Ramey wrote: > Posix also says that "variables" are inherited from the environment. That > word has a very specific meaning, as was reiterated during the $@ and set -u > discussion. The same "variables" language is used when Posix talks about > creating the environment for shell execution environments. > > The question is whether "tolerant" just means that the shell doesn't display > a warning message about the assignment, as it does when you use an invalid > variable name in an assignment statement, or exit with a variable assignment > error, or dump core with a seg fault, as in many historical versions of sh. > It may or may not also mean that the shell passes inherited invalid variable > names to child processes. > > It seems, though, that there might be enough use for me to try and make it > work. While I'm not wild about creating yet another class of variable, > there might be a way to do it simply. What's the current status of this issue? I'd like to offer my help for testing the fix since I have a strong interest to get the problem (linux kernel (UML) can't be compiled properly with bash 4.x) resolved. Thank you very much in advance! Best regards, Christian
Re: 3-lines long segfault
Chet Ramey a écrit : > Marc Herbert wrote: >> The following script crashes bash versions 2.05, 3.2.25, 4.0.16,... >> >> >> trap "echo USR1caught" USR1 >> read -t 1 >> foo=$( kill -USR1 $$ ) >> echo 'Yeah! no Segmentation fault!' > > Thanks for the report. Thanks for bash in the first place. > Fixed in the next version. Chet, is there any workaround for existing versions? I mean, there must be some memory corruption involved in here, so is it possible to dodge it without upgrading? For instance, would simply avoiding the "-t" option of "read" be enough to guarantee no memory corruption? Other? Thanks a lot in advance! I know: the answer to this question is probably in the latest revision of the source code, but... I doubt I will fully understand what is happening there. And I am surely not the only one in this case. Cheers, Marc
Patch to make readline parse CSI keycodes properly
Hi,
This is about a small problem with readline that has long been bugging
me: when pressing an unbound key or key combination in bash and other
readline programs, somewhat cryptic characters are often inserted into
the command line. For example, you might get "~" for Insert, "4~" for
F12, or ";5A" for Ctrl+Up. This can be particularly confusing for new
users.
The reason for this issue is that readline does not parse control
sequences as defined in section 5.4 of the ECMA-48 standard correctly.
VT100 terminals and its numerous descendants of course use those for
many keys or key combinations that don't have single-character codes.
They start with the control sequence introducer (CSI), normally
represented by "\e[", followed by parameter and intermediate bytes in
the range from 0x20 to 0x3F, followed by a final byte in the range
from 0x40 to 0x7E. Readline, at least in its default config, stops
parsing these sequences after the first byte following the "\e[",
leaving any remaining bytes to spill onto the command line.
As far as I understand it, there's currently no way to address this in
the readline config, short of
explicitly binding all the CSI keycodes to nothing anyway. Well
actually, a single binding seems to suffice for codes that differ only
in the last character, but that doesn't help with the F-keys and the
likes of Insert. Either way, it would be a rather long list of
bindings, which would slow down startup and lead to an explosion in
the number of keymaps allocated internally by readline.
Hence the small patch below, which avoids such inefficiencies. It
defines a function called 'skip-csi-seq' that skips over the
parameter, intermediate, and final bytes. This is assigned to "\e[".
Since bindings for longer sequences take precedence over shorter ones,
CSI sequences that do have functions bound to them continue to work.
Here's the result in the output of 'bind':
$ bind -p | grep e\\[
"\e[D": backward-char
"\e[H": beginning-of-line
"\e[3~": delete-char
"\e[F": end-of-line
"\e[C": forward-char
"\e[B": next-history
"\e[A": previous-history
"\e[": skip-csi-seq
This is working correctly here, including when adding bindings for
longer keycodes: no more odd characters being inserted when
accidentally hitting the wrong key. Please consider this for inclusion
into readline.
Kind regards,
Andy
--- emacs_keymap.c 2009-01-04 19:32:32.0 +
+++ emacs_keymap.c 2009-07-19 08:23:32.796875000 +0100
@@ -417,7 +417,7 @@
{ ISFUNC, rl_do_lowercase_version }, /* Meta-Z */
/* Some more punctuation. */
- { ISFUNC, (rl_command_func_t *)0x0 },/* Meta-[ *//* was
rl_arrow_keys */
+ { ISFUNC, rl_skip_csi_seq }, /* Meta-[ *//* was
rl_arrow_keys */
{ ISFUNC, rl_delete_horizontal_space }, /* Meta-\ */
{ ISFUNC, (rl_command_func_t *)0x0 },/* Meta-] */
{ ISFUNC, (rl_command_func_t *)0x0 },/* Meta-^ */
--- funmap.c2009-01-04 19:32:33.0 +
+++ funmap.c2009-07-19 08:25:34.90625 +0100
@@ -123,6 +123,7 @@
{ "revert-line", rl_revert_line },
{ "self-insert", rl_insert },
{ "set-mark", rl_set_mark },
+ { "skip-csi-seq", rl_skip_csi_seq },
{ "start-kbd-macro", rl_start_kbd_macro },
{ "tab-insert", rl_tab_insert },
{ "tilde-expand", rl_tilde_expand },
--- readline.h 2009-07-19 05:28:30.0 +0100
+++ readline.h 2009-07-19 08:23:38.546875000 +0100
@@ -197,6 +197,7 @@
/* Miscellaneous bindable commands. */
extern int rl_abort PARAMS((int, int));
extern int rl_tty_status PARAMS((int, int));
+extern int rl_skip_csi_seq PARAMS((int, int));
/* Bindable commands for incremental and non-incremental history
searching. */
extern int rl_history_search_forward PARAMS((int, int));
--- text.c 2009-01-04 19:32:34.0 +
+++ text.c 2009-07-19 08:23:45.890625000 +0100
@@ -571,6 +571,18 @@
}
int
+rl_skip_csi_seq (count, c)
+ int count, c;
+{
+ RL_SETSTATE(RL_STATE_MOREINPUT);
+ do
+c = rl_read_key ();
+ while (c >= 0x20 && c < 0x40);
+ RL_UNSETSTATE(RL_STATE_MOREINPUT);
+ return 0;
+}
+
+int
rl_arrow_keys (count, c)
int count, c;
{
Re: [OT] regarding 'time' builtin
Greg Wooledge wrote: > On Fri, Jul 17, 2009 at 06:42:03PM -0400, Chris Jones wrote: >> Sometimes I wonder if Greg is a "real" person.. not a smart program that >> can generate the correct answers to all the questions you had about bash >> utilization - and may have been too shy to ask. > > Either I'm a real person, or I'm a brain in a vat that's being fed > false sensory input to make me think I'm a real person They do amazing things down the street... -- ``The lyf so short, the craft so long to lerne.'' - Chaucer Chet Ramey, ITS, CWRUc...@case.eduhttp://cnswww.cns.cwru.edu/~chet/
Re: Help with script - doesn't work properly from cron
On Fri, Jul 17, 2009 at 02:53:21PM -0700, Erik Olof Wahlstrom wrote:
> #!/bin/bash
> BACKUP_DIR="/media/disk/AUTOMATED_BACKUPS/DB_DAILY"
>
> CURRENT_DIR=$BACKUP_DIR/`date +%d`
# See how you call date here without an explicit path? That's good.
> DATABASES="$(/usr/bin/mysql -uUsername -pPassword -Bse 'show databases')"
> echo 'Backing up databases: '$DATABASES
echo "Backing up databases: $DATABASES"
> if [ -e "$CURRENT_DIR" ]
if [ -d "$CURRENT_DIR" ]
# Of course, if it's a file, you would have problems below
> then
> cd $CURRENT_DIR
> /bin/rm *
cd "$CURRENT_DIR" || exit 1
rm -f *
# NEVER do "rm *" after a "cd" without checking that the cd
# succeeded!
# You don't need an explicit path on rm. You didn't need one
# on date, so why put one on rm?
# In this particular version of this particular script, you can
# get away with not quoting the $CURRENT_DIR parameter expansion,
# but quoting it will save your butt if someone changes it, e.g.:
# CURRENT_DIR='/media/disk/c/Program Files'
> else
> /bin/mkdir $CURRENT_DIR
mkdir -p "$CURRENT_DIR"
> cd $CURRENT_DIR
cd "$CURRENT_DIR" || exit 1
> fi
>
> for DB in $DATABASES
> do
> /usr/bin/mysqldump -uroot -pHardAsMySql321 "$DB" | bzip2 >
> "$DB"_`date +%Y-%m-%d_%k.%M`".sql.bz2"
# Long line, probably broken by your mailer. For clarity, I'd
# write it on two lines explicitily:
mysqldump -uroot -pHardAsMySql321 "$DB" |
bzip2 > "${DB}_$(date +%Y-%m-%d_%k.%M).sql.bz2"
> done
>
> exit 0
>
> Can any skilled eyes see why this doesn't work when it is run from the roots
> crontab?
None of the changes I suggested should really affect that. Perhaps it's
some sort of mysql-specific thing?
> Additionally, when I run the script as root in the terminal, I get the
> following output:
>
> Backing up databases: information_schema db1 db2 db3
> /bin/rm: cannot remove `*': No such file or directory
The "rm -f" will suppress that error message.
> Is there a better way to clear out last months files before making the
> current backups?
You could also consider writing it this way:
cd /
rm -rf "$CURRENT_DIR"
mkdir -p "$CURRENT_DIR"
cd "$CURRENT_DIR" || exit 1
Then you don't even need to check whether it already exists.
On Sat, Jul 18, 2009 at 11:25:13AM +0200, Bernd Eggink wrote:
> You could replace the whole if-then-else clause by
>
> mkdir -p $CURRENT_DIR
> cd $CURRENT_DIR
> rm -f *
Another failure to check the results of "cd" before doing "rm *". This
can and will lead to disasters.
Re: [OT] regarding 'time' builtin
On Fri, Jul 17, 2009 at 06:42:03PM -0400, Chris Jones wrote: > Sometimes I wonder if Greg is a "real" person.. not a smart program that > can generate the correct answers to all the questions you had about bash > utilization - and may have been too shy to ask. Either I'm a real person, or I'm a brain in a vat that's being fed false sensory input to make me think I'm a real person The wiki is a collaborative group effort, and while I've written more than a small amount of its content, I'm just one of many contributors.
Re: Setting READLINE_POINT isn't always applied (4.0.24)
Chet Ramey wrote:
> Henning Bekel wrote:
>> Hello,
>> If I try to change READLINE_LINE and READLINE_POINT from a
>> function bound via bind -x, then setting READLINE_POINT is not
>> applied every second time I invoke the function. Instead, the
>> cursor is placed at the end of the line.
>>
>> Simple test case:
>>
>> test_rl () {
>> READLINE_LINE="$READLINE_LINE#edited"
>> READLINE_POINT=3
>> }
>>
>> bind -x '"\ew": test_rl'
>>
>> And keep hitting \ew... every first time the cursor is placed
>> at position 3, every second time it ends up at the end of the
>> line.
>>
>> Is this the intended behavior or is it a bug?
>
> It's a bug. A fix will appear as a patch to bash-4.0.
Thanks!
>> Also, is it intended that the modified line is drawn on a new
>> line? For example, if I wrote a function that increments the
>> word at READLINE_POINT if it contains only digits, and i wanted
>> to increase a value by hitting a keyseq five times, I'd end up
>> with five lines being printed in my terminal. I'd love to see
>> this work like readline's own editing functions (e.g.
>> upcase-word) or macros that edit the line without printing it
>> again on a new line. But maybe I just misinterpreted the
>> purpose of this new feature. If so, could you explain it's
>> intended purpose?
>
> It's intended. Since the command executed as a result of `bind
> -x' is an arbitrary one that can do anything it wants to the
> display, it's better to completely redraw the line.
I already assumed this, just thought I'd make sure, thanks.
> I wonder if we could specify whether or not to completely redraw
> the line with a return status.
As far as I see it there hasn't been a reason to return a specific
value from a function bound via -x in the past, has there? Still,
maybe some users have done so out of principle (as in "I just
always return 0 on success for any function"), and thus giving
meaning to the return value might result in unexpected behaviour
for existing code... not sure about this.
Alternatively, couldn't another option like '-X' be introduced for
bind to explicitly request not to redraw the line completely? Or
would you consider this feature creep?
Best Regards,
Henning
Re: bash 4: source does not search current dir
On Mon, Jul 20, 2009 at 3:20 AM, Chet Ramey wrote: > Denys Vlasenko wrote: >> I assume . command does not search current directory anymore >> in bash 4 for a reason. Perhaps it is specified by relevant >> standard, although I did not find it. >> >> This incompatible change affects me a lot. I imagine >> I am not alone with this. > > This happens only when in Posix mode, and is what Posix specifies. > I fixed a bug in versions predating bash-4.0 that caused it to not > work correctly; that's probably what you're seeing. Yep. I assumed from the start that this change wasn't done on a whim. I am not complaining about it. My mail was mostly about this part: "To forestall questions from users like me, consider adding information about it to this file: http://tiswww.case.edu/php/chet/bash/COMPAT"; since this is clearly a compat issue, and it affects many users. -- vda
Re: echo "-e" produces no outputs
2009-07-19 21:07:20 -0400, Chet Ramey: [...] > > Note that it's a known non POSIX-conformance of bash. > > > > POSIX is explicit that > > > > echo -e > > > > Should output "-e\n". > > That's why bash has the `xpg_echo' option. You can build bash in such > a way that it's always enabled. [...] Well, that doesn't help when you intend to write a portable script. If you write a script that has "echo -e", POSIX guarantees that it outputs "-e\n" as long as you're on a POSIX system in a POSIX environment. It's hardly documented anywhere that for instance, on Linux (where sh is generally bash based), to get into a POSIX environment, you would need to have "SHELLOPTS=xpg_echo" in your environment. Anyway, note that xpg_echo doesn't turn on POSIX conformance: $ ./bash --norc bash-4.0$ shopt -s xpg_echo bash-4.0$ echo -e bash-4.0$ echo $BASH_VERSION 4.0.0(1)-release bash-4.0$ (same with 3.2). "xpg_echo" turns on Unix conformance only for arguments to echo that contain "\"s (and the behavior for those is unspecified by POSIX, that's part of the XSI extension). What POSIX says is that: echo -n And echo 'xxx\yyy' are unspecified. The rest is clearly specified. Unix specifies echo fully. echo -n should output "-n" echo '\t\c' should output "" Note that bash conforms to the LSB on this which allows any option to "echo". I wouldn't be surprised if future versions of POSIX did the same as in effect, many shell implementations beside bash do support other options beside "-n". regards, Stephane
