Re: [openstack-dev] [tripleo] /bin/bash vs. /bin/sh

[Ben Nemec]

On 04/15/2014 02:44 PM, Clint Byrum wrote:

Excerpts from Ben Nemec's message of 2014-04-14 09:26:17 -0700:

tldr: I propose we use bash explicitly for all diskimage-builder scripts
(at least for the short-term - see details below).

This is something that was raised on my linting changes to enable set -o
pipefail.  That is a bash-ism, so it could break in the
diskimage-builder scripts that are run using /bin/sh.  Two possible
fixes for that: switch to /bin/bash, or don't use -o pipefail



What about this:

if ! [ "$SHEBANG" = "#!/bin/bash" ] ; then
   report_warning Non bash shebang, skipping script lint
fi


I was thinking along the same lines, although at least for the moment I 
would like to leave the +x check enabled for all shebangs since it still 
doesn't make sense to have a shebang without +x.





But I think this raises a bigger question - does diskimage-builder
require bash?  If so, I think we should just add a rule to enforce that
/bin/bash is the shell used for everything.  I know we have a bunch of
bash-isms in the code already, so at least in the short-term I think
this is probably the way to go, so we can get the benefits of things
like -o pipefail and lose the ambiguity we have right now.  For
reference, a quick grep of the diskimage-builder source shows we have
150 scripts using bash explicitly and only 24 that are plain sh, so
making the code truly shell-agnostic is likely to be a significant
amount of work.


Yes, diskimage-builder is bash, not posix shell. We're not masochists.
;)



In the long run it might be nice to have cross-shell compatibility, but
if we're going to do that I think we need a couple of things: 1) Someone
to do the work (I don't have a particular need to run dib in not-bash,
so I'm not signing up for that :-) 2) Testing in other shells -
obviously just changing /bin/bash to /bin/sh doesn't mean we actually
support anything but bash.  We really need to be gating on other shells
if we're going to make a significant effort to support them.  It's not
good to ask reviewers to try to catch every bash-ism proposed in a
change.  This also relates to some of the unit testing work that is
going on right now too - if we had better unit test coverage of the
scripts we would be able to do this more easily.



I suggest that diskimage-builder's included elements should be /bin/bash
only. When we have an element linting tool, non bash shebangs should be
warnings we should enforce "no warnings". For t-i-e, we can strive for
no warnings, but that would be a stretch goal and may involve refining
the warnings.


This doesn't seem to be a problem in dib - almost everything was 
explicitly bash already, and the scripts that weren't are pretty trivial.


The ramdisk init script remains a thorn in my side for this though - 
based on our conversation last Friday we don't want that to have the 
same set -eu behavior as the other scripts (since init failing causes a 
kernel panic), and based on Chris's comment in 
http://lists.openstack.org/pipermail/openstack-dev/2014-April/032753.html we 
don't even want that to be explicitly a bash script, except that it is 
today.  So I think we need an exception mechanism like the pep8 #noqa 
tag (or something similar) to note that init should basically be ignored 
by the lint checks since it needs to violate most of the current ones.


For the moment, I'm thinking I'll silently ignore /bin/sh scripts, 
convert init to be one of those, and work on the warning/exception 
mechanism in the future.


-Ben

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [tripleo] /bin/bash vs. /bin/sh

[Clint Byrum]

Excerpts from Ghe Rivero's message of 2014-04-15 04:31:19 -0700:
> +1 to use bash as the default shell. So far, all major distros use bash
> as the default one (except Debian which uses dash).
> An about rewriting the code in Python, I agree that shell is complicated
> for large programs, but writing anything command oriented in other than
> shell is a nightmare. But there are some parts that can benefit from that.
> 

Side note, Ubuntu uses dash as /bin/sh.

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [tripleo] /bin/bash vs. /bin/sh

[Clint Byrum]

Excerpts from Ben Nemec's message of 2014-04-14 09:26:17 -0700:
> tldr: I propose we use bash explicitly for all diskimage-builder scripts 
> (at least for the short-term - see details below).
> 
> This is something that was raised on my linting changes to enable set -o 
> pipefail.  That is a bash-ism, so it could break in the 
> diskimage-builder scripts that are run using /bin/sh.  Two possible 
> fixes for that: switch to /bin/bash, or don't use -o pipefail
>

What about this:

if ! [ "$SHEBANG" = "#!/bin/bash" ] ; then
  report_warning Non bash shebang, skipping script lint
fi

> But I think this raises a bigger question - does diskimage-builder 
> require bash?  If so, I think we should just add a rule to enforce that 
> /bin/bash is the shell used for everything.  I know we have a bunch of 
> bash-isms in the code already, so at least in the short-term I think 
> this is probably the way to go, so we can get the benefits of things 
> like -o pipefail and lose the ambiguity we have right now.  For 
> reference, a quick grep of the diskimage-builder source shows we have 
> 150 scripts using bash explicitly and only 24 that are plain sh, so 
> making the code truly shell-agnostic is likely to be a significant 
> amount of work.

Yes, diskimage-builder is bash, not posix shell. We're not masochists.
;)

> 
> In the long run it might be nice to have cross-shell compatibility, but 
> if we're going to do that I think we need a couple of things: 1) Someone 
> to do the work (I don't have a particular need to run dib in not-bash, 
> so I'm not signing up for that :-) 2) Testing in other shells - 
> obviously just changing /bin/bash to /bin/sh doesn't mean we actually 
> support anything but bash.  We really need to be gating on other shells 
> if we're going to make a significant effort to support them.  It's not 
> good to ask reviewers to try to catch every bash-ism proposed in a 
> change.  This also relates to some of the unit testing work that is 
> going on right now too - if we had better unit test coverage of the 
> scripts we would be able to do this more easily.
>

I suggest that diskimage-builder's included elements should be /bin/bash
only. When we have an element linting tool, non bash shebangs should be
warnings we should enforce "no warnings". For t-i-e, we can strive for
no warnings, but that would be a stretch goal and may involve refining
the warnings.

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [tripleo] /bin/bash vs. /bin/sh

[Chmouel Boudjnah]

FWIW: we are using bash in devstack if we were going to try to make it
POSIX bourne shell (or whatever /bin/sh is) it would have been a huge pain.


On Tue, Apr 15, 2014 at 1:25 PM, Dougal Matthews  wrote:

> Another +1 for using bash. Sounds like an easy win.
>
>
> On 15/04/14 12:31, Ghe Rivero wrote:
>
>> +1 to use bash as the default shell. So far, all major distros use bash
>> as the default one (except Debian which uses dash).
>> An about rewriting the code in Python, I agree that shell is complicated
>> for large programs, but writing anything command oriented in other than
>> shell is a nightmare. But there are some parts that can benefit from that.
>>
>> Ghe Rivero
>>
>> On 04/15/2014 11:05 AM, Chris Jones wrote:
>>
>>> Hi
>>>
>>> On 15 April 2014 09:14, Daniel P. Berrange >> > wrote:
>>>
>>> I supose that rewriting the code to be in Python is out of the
>>> question ?  IMHO shell is just a terrible language for doing any
>>> program that is remotely complicated (ie longer than 10 lines of
>>>
>>>
>>> I don't think it's out of the question - where something makes sense
>>> to switch to Python, that would seem like a worthwhile thing to be
>>> doing. I do think it's a different question though - we can quickly
>>> flip things from /bin/sh to /bin/bash without affecting their
>>> suitability for replacement with python.
>>>
>>> --
>>> Cheers,
>>>
>>> Chris
>>>
>>>
>>> ___
>>> OpenStack-dev mailing list
>>> OpenStack-dev@lists.openstack.org
>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>>
>>
>>
>>
>> ___
>> OpenStack-dev mailing list
>> OpenStack-dev@lists.openstack.org
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
>>
>
> ___
> OpenStack-dev mailing list
> OpenStack-dev@lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [tripleo] /bin/bash vs. /bin/sh

[Dougal Matthews]

Another +1 for using bash. Sounds like an easy win.

On 15/04/14 12:31, Ghe Rivero wrote:

+1 to use bash as the default shell. So far, all major distros use bash
as the default one (except Debian which uses dash).
An about rewriting the code in Python, I agree that shell is complicated
for large programs, but writing anything command oriented in other than
shell is a nightmare. But there are some parts that can benefit from that.

Ghe Rivero

On 04/15/2014 11:05 AM, Chris Jones wrote:

Hi

On 15 April 2014 09:14, Daniel P. Berrange mailto:berra...@redhat.com>> wrote:

I supose that rewriting the code to be in Python is out of the
question ?  IMHO shell is just a terrible language for doing any
program that is remotely complicated (ie longer than 10 lines of


I don't think it's out of the question - where something makes sense
to switch to Python, that would seem like a worthwhile thing to be
doing. I do think it's a different question though - we can quickly
flip things from /bin/sh to /bin/bash without affecting their
suitability for replacement with python.

--
Cheers,

Chris


___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev




___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev




___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [tripleo] /bin/bash vs. /bin/sh

[Jay Dobies]

+1 to using bash, the argument about not keeping POSIX compliance for 
the sake of it makes sense to me.


On 04/15/2014 07:31 AM, Ghe Rivero wrote:

+1 to use bash as the default shell. So far, all major distros use bash
as the default one (except Debian which uses dash).
An about rewriting the code in Python, I agree that shell is complicated
for large programs, but writing anything command oriented in other than
shell is a nightmare. But there are some parts that can benefit from that.

Ghe Rivero

On 04/15/2014 11:05 AM, Chris Jones wrote:

Hi

On 15 April 2014 09:14, Daniel P. Berrange mailto:berra...@redhat.com>> wrote:

I supose that rewriting the code to be in Python is out of the
question ?  IMHO shell is just a terrible language for doing any
program that is remotely complicated (ie longer than 10 lines of


I don't think it's out of the question - where something makes sense
to switch to Python, that would seem like a worthwhile thing to be
doing. I do think it's a different question though - we can quickly
flip things from /bin/sh to /bin/bash without affecting their
suitability for replacement with python.

--
Cheers,

Chris


___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev




___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [tripleo] /bin/bash vs. /bin/sh

[Ryan Brady]

- Original Message -
> From: "Chris Jones" 
> To: openst...@nemebean.com, "OpenStack Development Mailing List (not for 
> usage questions)"
> 
> Sent: Monday, April 14, 2014 2:24:57 PM
> Subject: Re: [openstack-dev] [tripleo] /bin/bash vs. /bin/sh
> 
> Hi
> 
> Apart from special cases like the ramdisk's /init, which is a script that
> needs to run in busybox's shell, everything should be using bash. There's no
> point us tying ourselves in knots trying to achieve POSIX compliance for the
> sake of it, when bashisms are super useful.

+1 for the pragmatic approach.

> 
> Cheers,
> 
> Chris
> 
> 
> On 14 April 2014 17:26, Ben Nemec < openst...@nemebean.com > wrote:
> 
> 
> tldr: I propose we use bash explicitly for all diskimage-builder scripts (at
> least for the short-term - see details below).
> 
> This is something that was raised on my linting changes to enable set -o
> pipefail. That is a bash-ism, so it could break in the diskimage-builder
> scripts that are run using /bin/sh. Two possible fixes for that: switch to
> /bin/bash, or don't use -o pipefail
> 
> But I think this raises a bigger question - does diskimage-builder require
> bash? If so, I think we should just add a rule to enforce that /bin/bash is
> the shell used for everything. I know we have a bunch of bash-isms in the
> code already, so at least in the short-term I think this is probably the way
> to go, so we can get the benefits of things like -o pipefail and lose the
> ambiguity we have right now. For reference, a quick grep of the
> diskimage-builder source shows we have 150 scripts using bash explicitly and
> only 24 that are plain sh, so making the code truly shell-agnostic is likely
> to be a significant amount of work.
> 
> In the long run it might be nice to have cross-shell compatibility, but if
> we're going to do that I think we need a couple of things: 1) Someone to do
> the work (I don't have a particular need to run dib in not-bash, so I'm not
> signing up for that :-) 2) Testing in other shells - obviously just changing
> /bin/bash to /bin/sh doesn't mean we actually support anything but bash. We
> really need to be gating on other shells if we're going to make a
> significant effort to support them. It's not good to ask reviewers to try to
> catch every bash-ism proposed in a change. This also relates to some of the
> unit testing work that is going on right now too - if we had better unit
> test coverage of the scripts we would be able to do this more easily.
> 
> Thoughts?
> 
> Thanks.
> 
> -Ben
> 
> __ _
> OpenStack-dev mailing list
> OpenStack-dev@lists.openstack. org
> http://lists.openstack.org/ cgi-bin/mailman/listinfo/ openstack-dev
> 
> 
> 
> --
> Cheers,
> 
> Chris
> 
> ___
> OpenStack-dev mailing list
> OpenStack-dev@lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> 

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [tripleo] /bin/bash vs. /bin/sh

[Petr Blaho]

On Mon, Apr 14, 2014 at 07:24:57PM +0100, Chris Jones wrote:
> Hi
> 
> Apart from special cases like the ramdisk's /init, which is a script that 
> needs
> to run in busybox's shell, everything should be using bash. There's no point 
> us
> tying ourselves in knots trying to achieve POSIX compliance for the sake of 
> it,
> when bashisms are super useful.

+1, especially for "tying ourselves in knots trying to achieve POSIX compliance 
for
the sake of it"
> 
> Cheers,
> 
> Chris
> 
> 
> On 14 April 2014 17:26, Ben Nemec  wrote:
> 
> tldr: I propose we use bash explicitly for all diskimage-builder scripts
> (at least for the short-term - see details below).
> 
> This is something that was raised on my linting changes to enable set -o
> pipefail.  That is a bash-ism, so it could break in the diskimage-builder
> scripts that are run using /bin/sh.  Two possible fixes for that: switch 
> to
> /bin/bash, or don't use -o pipefail
> 
> But I think this raises a bigger question - does diskimage-builder require
> bash?  If so, I think we should just add a rule to enforce that /bin/bash
> is the shell used for everything.  I know we have a bunch of bash-isms in
> the code already, so at least in the short-term I think this is probably
> the way to go, so we can get the benefits of things like -o pipefail and
> lose the ambiguity we have right now.  For reference, a quick grep of the
> diskimage-builder source shows we have 150 scripts using bash explicitly
> and only 24 that are plain sh, so making the code truly shell-agnostic is
> likely to be a significant amount of work.
> 
> In the long run it might be nice to have cross-shell compatibility, but if
> we're going to do that I think we need a couple of things: 1) Someone to 
> do
> the work (I don't have a particular need to run dib in not-bash, so I'm 
> not
> signing up for that :-) 2) Testing in other shells - obviously just
> changing /bin/bash to /bin/sh doesn't mean we actually support anything 
> but
> bash.  We really need to be gating on other shells if we're going to make 
> a
> significant effort to support them.  It's not good to ask reviewers to try
> to catch every bash-ism proposed in a change.  This also relates to some 
> of
> the unit testing work that is going on right now too - if we had better
> unit test coverage of the scripts we would be able to do this more easily.
> 
> Thoughts?
> 
> Thanks.
> 
> -Ben
> 
> ___
> OpenStack-dev mailing list
> OpenStack-dev@lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> 
> 
> 
> 
> --
> Cheers,
> 
> Chris

> ___
> OpenStack-dev mailing list
> OpenStack-dev@lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


-- 
Petr Blaho, pbl...@redhat.com
Software Engineer

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [tripleo] /bin/bash vs. /bin/sh

[Ghe Rivero]

+1 to use bash as the default shell. So far, all major distros use bash
as the default one (except Debian which uses dash).
An about rewriting the code in Python, I agree that shell is complicated
for large programs, but writing anything command oriented in other than
shell is a nightmare. But there are some parts that can benefit from that.

Ghe Rivero

On 04/15/2014 11:05 AM, Chris Jones wrote:
> Hi
>
> On 15 April 2014 09:14, Daniel P. Berrange  > wrote:
>
> I supose that rewriting the code to be in Python is out of the
> question ?  IMHO shell is just a terrible language for doing any
> program that is remotely complicated (ie longer than 10 lines of
>
>
> I don't think it's out of the question - where something makes sense
> to switch to Python, that would seem like a worthwhile thing to be
> doing. I do think it's a different question though - we can quickly
> flip things from /bin/sh to /bin/bash without affecting their
> suitability for replacement with python.
>
> -- 
> Cheers,
>
> Chris
>
>
> ___
> OpenStack-dev mailing list
> OpenStack-dev@lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [tripleo] /bin/bash vs. /bin/sh

[Chris Jones]

Hi

On 15 April 2014 09:14, Daniel P. Berrange  wrote:
>
> I supose that rewriting the code to be in Python is out of the
> question ?  IMHO shell is just a terrible language for doing any
> program that is remotely complicated (ie longer than 10 lines of


I don't think it's out of the question - where something makes sense to
switch to Python, that would seem like a worthwhile thing to be doing. I do
think it's a different question though - we can quickly flip things from
/bin/sh to /bin/bash without affecting their suitability for replacement
with python.

-- 
Cheers,

Chris
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [tripleo] /bin/bash vs. /bin/sh

[Daniel P. Berrange]

On Mon, Apr 14, 2014 at 11:26:17AM -0500, Ben Nemec wrote:
> tldr: I propose we use bash explicitly for all diskimage-builder
> scripts (at least for the short-term - see details below).
> 
> This is something that was raised on my linting changes to enable
> set -o pipefail.  That is a bash-ism, so it could break in the
> diskimage-builder scripts that are run using /bin/sh.  Two possible
> fixes for that: switch to /bin/bash, or don't use -o pipefail
> 
> But I think this raises a bigger question - does diskimage-builder
> require bash?  If so, I think we should just add a rule to enforce
> that /bin/bash is the shell used for everything.  I know we have a
> bunch of bash-isms in the code already, so at least in the
> short-term I think this is probably the way to go, so we can get the
> benefits of things like -o pipefail and lose the ambiguity we have
> right now.  For reference, a quick grep of the diskimage-builder
> source shows we have 150 scripts using bash explicitly and only 24
> that are plain sh, so making the code truly shell-agnostic is likely
> to be a significant amount of work.
> 
> In the long run it might be nice to have cross-shell compatibility,
> but if we're going to do that I think we need a couple of things: 1)
> Someone to do the work (I don't have a particular need to run dib in
> not-bash, so I'm not signing up for that :-) 2) Testing in other
> shells - obviously just changing /bin/bash to /bin/sh doesn't mean
> we actually support anything but bash.  We really need to be gating
> on other shells if we're going to make a significant effort to
> support them.  It's not good to ask reviewers to try to catch every
> bash-ism proposed in a change.  This also relates to some of the
> unit testing work that is going on right now too - if we had better
> unit test coverage of the scripts we would be able to do this more
> easily.
> 
> Thoughts?

I supose that rewriting the code to be in Python is out of the
question ?  IMHO shell is just a terrible language for doing any
program that is remotely complicated (ie longer than 10 lines of
shell), for the reasons you are unfortunately illustrating here,
among many others.

Regards,
Daniel
-- 
|: http://berrange.com  -o-http://www.flickr.com/photos/dberrange/ :|
|: http://libvirt.org  -o- http://virt-manager.org :|
|: http://autobuild.org   -o- http://search.cpan.org/~danberr/ :|
|: http://entangle-photo.org   -o-   http://live.gnome.org/gtk-vnc :|

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [tripleo] /bin/bash vs. /bin/sh

[Chris Jones]

Hi

Apart from special cases like the ramdisk's /init, which is a script that
needs to run in busybox's shell, everything should be using bash. There's
no point us tying ourselves in knots trying to achieve POSIX compliance for
the sake of it, when bashisms are super useful.

Cheers,

Chris


On 14 April 2014 17:26, Ben Nemec  wrote:

> tldr: I propose we use bash explicitly for all diskimage-builder scripts
> (at least for the short-term - see details below).
>
> This is something that was raised on my linting changes to enable set -o
> pipefail.  That is a bash-ism, so it could break in the diskimage-builder
> scripts that are run using /bin/sh.  Two possible fixes for that: switch to
> /bin/bash, or don't use -o pipefail
>
> But I think this raises a bigger question - does diskimage-builder require
> bash?  If so, I think we should just add a rule to enforce that /bin/bash
> is the shell used for everything.  I know we have a bunch of bash-isms in
> the code already, so at least in the short-term I think this is probably
> the way to go, so we can get the benefits of things like -o pipefail and
> lose the ambiguity we have right now.  For reference, a quick grep of the
> diskimage-builder source shows we have 150 scripts using bash explicitly
> and only 24 that are plain sh, so making the code truly shell-agnostic is
> likely to be a significant amount of work.
>
> In the long run it might be nice to have cross-shell compatibility, but if
> we're going to do that I think we need a couple of things: 1) Someone to do
> the work (I don't have a particular need to run dib in not-bash, so I'm not
> signing up for that :-) 2) Testing in other shells - obviously just
> changing /bin/bash to /bin/sh doesn't mean we actually support anything but
> bash.  We really need to be gating on other shells if we're going to make a
> significant effort to support them.  It's not good to ask reviewers to try
> to catch every bash-ism proposed in a change.  This also relates to some of
> the unit testing work that is going on right now too - if we had better
> unit test coverage of the scripts we would be able to do this more easily.
>
> Thoughts?
>
> Thanks.
>
> -Ben
>
> ___
> OpenStack-dev mailing list
> OpenStack-dev@lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>



-- 
Cheers,

Chris
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [tripleo] /bin/bash vs. /bin/sh

[Dan Prince]

- Original Message -
> From: "Ben Nemec" 
> To: "OpenStack Development Mailing List (not for usage questions)" 
> 
> Sent: Monday, April 14, 2014 12:26:17 PM
> Subject: [openstack-dev] [tripleo] /bin/bash vs. /bin/sh
> 
> tldr: I propose we use bash explicitly for all diskimage-builder scripts
> (at least for the short-term - see details below).
> 
> This is something that was raised on my linting changes to enable set -o
> pipefail.  That is a bash-ism, so it could break in the
> diskimage-builder scripts that are run using /bin/sh.  Two possible
> fixes for that: switch to /bin/bash, or don't use -o pipefail
> 
> But I think this raises a bigger question - does diskimage-builder
> require bash?  If so, I think we should just add a rule to enforce that
> /bin/bash is the shell used for everything. 

This sounds like the easiest (and possibly best) solution here to me.

> I know we have a bunch of
> bash-isms in the code already, so at least in the short-term I think
> this is probably the way to go, so we can get the benefits of things
> like -o pipefail and lose the ambiguity we have right now.  For
> reference, a quick grep of the diskimage-builder source shows we have
> 150 scripts using bash explicitly and only 24 that are plain sh, so
> making the code truly shell-agnostic is likely to be a significant
> amount of work.
> 
> In the long run it might be nice to have cross-shell compatibility, but
> if we're going to do that I think we need a couple of things: 1) Someone
> to do the work (I don't have a particular need to run dib in not-bash,
> so I'm not signing up for that :-) 2) Testing in other shells -
> obviously just changing /bin/bash to /bin/sh doesn't mean we actually
> support anything but bash.  We really need to be gating on other shells
> if we're going to make a significant effort to support them.  It's not
> good to ask reviewers to try to catch every bash-ism proposed in a
> change.  This also relates to some of the unit testing work that is
> going on right now too - if we had better unit test coverage of the
> scripts we would be able to do this more easily.
> 
> Thoughts?

I see the fact that DIB uses /bin/bash as mostly an implementation detail. So 
long as we keep this internal to DIB it shouldn't matter what shell we use 
right? And on that note why not just make things simpler and require and use 
/bin/bash?

The only exception would be for scripts that might need to be sourced by end 
users in their own environment shells. And for those we can certainly try to 
make things as generic as possible.

> 
> Thanks.
> 
> -Ben
> 
> ___
> OpenStack-dev mailing list
> OpenStack-dev@lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> 

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [tripleo] /bin/bash vs. /bin/sh

[Ben Nemec]

On 04/14/2014 11:53 AM, Doug Hellmann wrote:

As someone who maintains a shell-script project across several
"sh-like" shells, I can say the edge cases cause enough work and
testing hassles that you shouldn't undertake it lightly. Is there any
particular need to run these tools under a non-bash shell?


I don't have one, but I've seen a few changes proposed to make the 
scripts work with other shells (although in at least one case I believe 
it was in a script explicitly using bash, so I'm not sure what the 
purpose was...).  And those edge cases are why I would like to stop 
trying to support not-bash unless someone else wants to step up and own 
that effort.


Thanks for the input.

-Ben



Doug

On Mon, Apr 14, 2014 at 12:26 PM, Ben Nemec  wrote:

tldr: I propose we use bash explicitly for all diskimage-builder scripts (at
least for the short-term - see details below).

This is something that was raised on my linting changes to enable set -o
pipefail.  That is a bash-ism, so it could break in the diskimage-builder
scripts that are run using /bin/sh.  Two possible fixes for that: switch to
/bin/bash, or don't use -o pipefail

But I think this raises a bigger question - does diskimage-builder require
bash?  If so, I think we should just add a rule to enforce that /bin/bash is
the shell used for everything.  I know we have a bunch of bash-isms in the
code already, so at least in the short-term I think this is probably the way
to go, so we can get the benefits of things like -o pipefail and lose the
ambiguity we have right now.  For reference, a quick grep of the
diskimage-builder source shows we have 150 scripts using bash explicitly and
only 24 that are plain sh, so making the code truly shell-agnostic is likely
to be a significant amount of work.

In the long run it might be nice to have cross-shell compatibility, but if
we're going to do that I think we need a couple of things: 1) Someone to do
the work (I don't have a particular need to run dib in not-bash, so I'm not
signing up for that :-) 2) Testing in other shells - obviously just changing
/bin/bash to /bin/sh doesn't mean we actually support anything but bash.  We
really need to be gating on other shells if we're going to make a
significant effort to support them.  It's not good to ask reviewers to try
to catch every bash-ism proposed in a change.  This also relates to some of
the unit testing work that is going on right now too - if we had better unit
test coverage of the scripts we would be able to do this more easily.

Thoughts?

Thanks.

-Ben

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [tripleo] /bin/bash vs. /bin/sh

[Doug Hellmann]

As someone who maintains a shell-script project across several
"sh-like" shells, I can say the edge cases cause enough work and
testing hassles that you shouldn't undertake it lightly. Is there any
particular need to run these tools under a non-bash shell?

Doug

On Mon, Apr 14, 2014 at 12:26 PM, Ben Nemec  wrote:
> tldr: I propose we use bash explicitly for all diskimage-builder scripts (at
> least for the short-term - see details below).
>
> This is something that was raised on my linting changes to enable set -o
> pipefail.  That is a bash-ism, so it could break in the diskimage-builder
> scripts that are run using /bin/sh.  Two possible fixes for that: switch to
> /bin/bash, or don't use -o pipefail
>
> But I think this raises a bigger question - does diskimage-builder require
> bash?  If so, I think we should just add a rule to enforce that /bin/bash is
> the shell used for everything.  I know we have a bunch of bash-isms in the
> code already, so at least in the short-term I think this is probably the way
> to go, so we can get the benefits of things like -o pipefail and lose the
> ambiguity we have right now.  For reference, a quick grep of the
> diskimage-builder source shows we have 150 scripts using bash explicitly and
> only 24 that are plain sh, so making the code truly shell-agnostic is likely
> to be a significant amount of work.
>
> In the long run it might be nice to have cross-shell compatibility, but if
> we're going to do that I think we need a couple of things: 1) Someone to do
> the work (I don't have a particular need to run dib in not-bash, so I'm not
> signing up for that :-) 2) Testing in other shells - obviously just changing
> /bin/bash to /bin/sh doesn't mean we actually support anything but bash.  We
> really need to be gating on other shells if we're going to make a
> significant effort to support them.  It's not good to ask reviewers to try
> to catch every bash-ism proposed in a change.  This also relates to some of
> the unit testing work that is going on right now too - if we had better unit
> test coverage of the scripts we would be able to do this more easily.
>
> Thoughts?
>
> Thanks.
>
> -Ben
>
> ___
> OpenStack-dev mailing list
> OpenStack-dev@lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [tripleo] /bin/bash vs. /bin/sh

[Ben Nemec]

tldr: I propose we use bash explicitly for all diskimage-builder scripts 
(at least for the short-term - see details below).


This is something that was raised on my linting changes to enable set -o 
pipefail.  That is a bash-ism, so it could break in the 
diskimage-builder scripts that are run using /bin/sh.  Two possible 
fixes for that: switch to /bin/bash, or don't use -o pipefail


But I think this raises a bigger question - does diskimage-builder 
require bash?  If so, I think we should just add a rule to enforce that 
/bin/bash is the shell used for everything.  I know we have a bunch of 
bash-isms in the code already, so at least in the short-term I think 
this is probably the way to go, so we can get the benefits of things 
like -o pipefail and lose the ambiguity we have right now.  For 
reference, a quick grep of the diskimage-builder source shows we have 
150 scripts using bash explicitly and only 24 that are plain sh, so 
making the code truly shell-agnostic is likely to be a significant 
amount of work.


In the long run it might be nice to have cross-shell compatibility, but 
if we're going to do that I think we need a couple of things: 1) Someone 
to do the work (I don't have a particular need to run dib in not-bash, 
so I'm not signing up for that :-) 2) Testing in other shells - 
obviously just changing /bin/bash to /bin/sh doesn't mean we actually 
support anything but bash.  We really need to be gating on other shells 
if we're going to make a significant effort to support them.  It's not 
good to ask reviewers to try to catch every bash-ism proposed in a 
change.  This also relates to some of the unit testing work that is 
going on right now too - if we had better unit test coverage of the 
scripts we would be able to do this more easily.


Thoughts?

Thanks.

-Ben

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev