Re: [gentoo-dev] escaping variables in sed expressions

[Ciaran McCreesh]

On Wed, 16 Apr 2008 19:17:51 +0200
Frank Gruellich <[EMAIL PROTECTED]> wrote:
> I was not able to create a filename or path containing it.  (Anyone
> else?)

Unix file names can't contain / or null.

> Unfortunately that stupid sed does not work with \x00 as
> delimiter...

Nor do most Unix apps, since they tend to be written in C using all
those C library functions that work on null terminated strings.

Null introduces far more problems than it solves, character-wise...

-- 
Ciaran McCreesh


signature.asc
Description: PGP signature

Re: [gentoo-dev] escaping variables in sed expressions

[Frank Gruellich]

* Marius Mauch <[EMAIL PROTECTED]> 15. Apr 08:
> On Tue, 15 Apr 2008 16:17:54 +0200
> Frank Gruellich <[EMAIL PROTECTED]> wrote:
> > * Santiago M. Mola <[EMAIL PROTECTED]> 15. Apr 08:
> > > On Tue, Apr 15, 2008 at 1:14 PM, Marijn Schouten (hkBst)
> > > Currently is use ':' as sed delimiter when paths are involved. I'd
> > > also like to hear from you about proper delimiters if you think ':'
> > > is not safe enough.
> > Even though it's probably stupid to use it, but ':' is a valid
> > character within a path.  I've no solution for this problem, however.
> Valid maybe (but then pretty much every character is valid),

I've been a bigmouth so I couldn't sleep last night thinking about that
problem (which in fact happened to me sometimes).  The very last
character I'd expect in a path would be the NUL char (\x00).  I was not
able to create a filename or path containing it.  (Anyone else?)
Unfortunately that stupid sed does not work with \x00 as delimiter...

But because a path will not contain a \x00 you can replace all
"$delimiter_of_your_dreams" with a \x00 and later change it back to the
original without adding any new.

Looks like:

(0) [EMAIL PROTECTED] [~] % echo '/foo/bar/foo:baz/baz bar/laber_rabarber/' |tr 
'/' '\0' |sed 's/a/o/g' |tr '\0' '/'
/foo/bor/foo:boz/boz bor/lober_roborber/
(0) [EMAIL PROTECTED] [~] %

Moving that to OP's problem he could maybe use something like:

tr '/' '\0' Makefile.new

This obviously introduces the problem that a Makefile contains more than
just paths and there could be a \x00 somewhere... but... well...

> but colon is used as path delimiter in many other contexts (e.g.
> $PATH) so it's rather unlikely to be used.

I'm *very* paranoid.  ;-)

Kind regards,
 Frank.
-- 
Sigmentation fault
-- 
gentoo-dev@lists.gentoo.org mailing list

Re: [gentoo-dev] escaping variables in sed expressions

[Mike Frysinger]

On Tuesday 15 April 2008, Marijn Schouten (hkBst) wrote:
> Hi list,
>
> it seems I have been using some fragile sed expression and I'd like to tap
> the collective wisdom for avoiding doing that in the future.
>
> dev-scheme/slib-3.1.5-r1 currently does
>
> sed "s_prefix = /usr/local/_prefix = ${D}/usr/_" -i Makefile

ignoring the escape character (which is currently standardized in the tree as 
a colon), expressions like this are inherently fragile due to makefile 
changes.  i tend to do:
sed -i '/^prefix[[:space:]]*=/s:=.*:=/usr' Makefile
-mike


signature.asc
Description: This is a digitally signed message part.

Re: [gentoo-dev] escaping variables in sed expressions

[Marius Mauch]

On Tue, 15 Apr 2008 16:17:54 +0200
Frank Gruellich <[EMAIL PROTECTED]> wrote:

> * Santiago M. Mola <[EMAIL PROTECTED]> 15. Apr 08:
> > On Tue, Apr 15, 2008 at 1:14 PM, Marijn Schouten (hkBst)
> > Currently is use ':' as sed delimiter when paths are involved. I'd
> > also like to hear from you about proper delimiters if you think ':'
> > is not safe enough.
> > 
> > AFAIK, the only corner case which would make this fail would be
> > Windows paths (C:/gentoo-prefix).
> 
> Even though it's probably stupid to use it, but ':' is a valid
> character within a path.  I've no solution for this problem, however.

Valid maybe (but then pretty much every character is valid), but colon
is used as path delimiter in many other contexts (e.g. $PATH) so it's
rather unlikely to be used.

Marius
-- 
gentoo-dev@lists.gentoo.org mailing list

Re: [gentoo-dev] escaping variables in sed expressions

[Marijn Schouten (hkBst)]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

Petteri Räty wrote:
| You should just fix the Makefile to respect DESTDIR and send the patch
| upstream.

You're right of course, but that is only the right way for this specific 
instance and not
a general way of handling tricky sed expressions.

Marijn

- --
Marijn Schouten (hkBst), Gentoo Lisp project, Gentoo ML
, #gentoo-{lisp,ml} on FreeNode
-BEGIN PGP SIGNATURE-
Version: GnuPG v2.0.9 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iEYEARECAAYFAkgEyckACgkQp/VmCx0OL2z7+QCdHx8n7Py3CQJy1tJMmYkyCmJW
yUgAoIVlu60bCfPJqC1iNIYc8AegKrCm
=1ItB
-END PGP SIGNATURE-
--
gentoo-dev@lists.gentoo.org mailing list

Re: [gentoo-dev] escaping variables in sed expressions

[Petteri Räty]

Marijn Schouten (hkBst) kirjoitti:

Hi list,

it seems I have been using some fragile sed expression and I'd like to 
tap the collective

wisdom for avoiding doing that in the future.

dev-scheme/slib-3.1.5-r1 currently does

sed "s_prefix = /usr/local/_prefix = ${D}/usr/_" -i Makefile

to make it not violate the sandbox. However a user had set
PORTAGE_TMPDIR=/home/gentoo_overflow/tmp causing the sed expression to 
contain too may

underscores and failing.[1]

There are several option to handle this. I could use a less common 
delimiter or I could
escape it: ${D//_/\_} instead of ${D}. I could use a sed expression that 
doesn't suffer

from this problem (thanks to dleverton):

sed -ne '\_^prefix = /usr/local_!{p;d}' -e "iprefix = ${D}" -i Makefile

Comments?

Marijn

[1]: http://bugs.gentoo.org/show_bug.cgi?id=217735



You should just fix the Makefile to respect DESTDIR and send the patch 
upstream.


Regards,
Petteri



signature.asc
Description: OpenPGP digital signature

Re: [gentoo-dev] escaping variables in sed expressions

[Frank Gruellich]

* Santiago M. Mola <[EMAIL PROTECTED]> 15. Apr 08:
> On Tue, Apr 15, 2008 at 1:14 PM, Marijn Schouten (hkBst)
> Currently is use ':' as sed delimiter when paths are involved. I'd
> also like to hear from you about proper delimiters if you think ':' is
> not safe enough.
> 
> AFAIK, the only corner case which would make this fail would be
> Windows paths (C:/gentoo-prefix).

Even though it's probably stupid to use it, but ':' is a valid character
within a path.  I've no solution for this problem, however.

Kind regards,
 Frank.
-- 
Sigmentation fault
-- 
gentoo-dev@lists.gentoo.org mailing list

Re: [gentoo-dev] escaping variables in sed expressions

[Ulrich Mueller]

> On Tue, 15 Apr 2008, Marijn Schouten (hkBst) wrote:

> There are several option to handle this. I could use a less common
> delimiter or I could escape it: ${D//_/\_} instead of ${D}. I could
> use a sed expression that doesn't suffer from this problem (thanks
> to dleverton):

> sed -ne '\_^prefix = /usr/local_!{p;d}' -e "iprefix = ${D}" -i Makefile

Hi Marijn,

both approaches won't work since you would have to escape not only the
pattern delimiter, but also sed special characters (like "\" and "&")
_and_ make special characters (e.g. the dollar sign).

Probably better to pass it to make via a command argument or
environment variable.

Ulrich
-- 
gentoo-dev@lists.gentoo.org mailing list

Re: [gentoo-dev] escaping variables in sed expressions

[David Leverton]

On Tuesday 15 April 2008 12:14:57 Marijn Schouten (hkBst) wrote:
> There are several option to handle this. I could use a less common
> delimiter or I could escape it: ${D//_/\_} instead of ${D}. I could use a
> sed expression that doesn't suffer from this problem (thanks to dleverton):
>
> sed -ne '\_^prefix = /usr/local_!{p;d}' -e "iprefix = ${D}" -i Makefile

Just to clarify, I didn't think of escaping when I suggested this.  Escaping 
is probably cleaner, and certainly easier to understand.
-- 
gentoo-dev@lists.gentoo.org mailing list

Re: [gentoo-dev] escaping variables in sed expressions

[Fabian Groffen]

On 15-04-2008 13:05:26 +0200, Santiago M. Mola wrote:
> On Tue, Apr 15, 2008 at 1:14 PM, Marijn Schouten (hkBst)
> <[EMAIL PROTECTED]> wrote:
> >
> >  Hi list,
> >
> >  it seems I have been using some fragile sed expression and I'd like to tap
> > the collective
> >  wisdom for avoiding doing that in the future.
> >
> >  dev-scheme/slib-3.1.5-r1 currently does
> >
> >  sed "s_prefix = /usr/local/_prefix = ${D}/usr/_" -i Makefile
> >
> >  to make it not violate the sandbox. However a user had set
> >  PORTAGE_TMPDIR=/home/gentoo_overflow/tmp causing the sed expression to
> > contain too may
> >  underscores and failing.[1]
> >
> >  There are several option to handle this. I could use a less common
> > delimiter or I could
> >  escape it: ${D//_/\_} instead of ${D}. I could use a sed expression that
> > doesn't suffer
> >  from this problem (thanks to dleverton):
> >
> >  sed -ne '\_^prefix = /usr/local_!{p;d}' -e "iprefix = ${D}" -i Makefile
> >
> >  Comments?
> >
> 
> Currently is use ':' as sed delimiter when paths are involved. I'd
> also like to hear from you about proper delimiters if you think ':' is
> not safe enough.

I met one case where : was indeed a problem, but that was in
CFLAGS/LDFLAGS replacements.  Some linkers accept (and do require)
arguments that are like "-mg:2512s".

> AFAIK, the only corner case which would make this fail would be
> Windows paths (C:/gentoo-prefix).

C:\ iirc, but Cygwin seems to map this as /cygdrive/C, Interix as
/dev/fs/C, command prompt I have no clue how portage could ever normally
work there.  SpikeSource's SpikeWAMP uses Cygwin underneath, so
Portage/ebuilds will see the mapped paths only, never heard of any
problems from them regarding this either.


-- 
Fabian Groffen
Gentoo on a different level
-- 
gentoo-dev@lists.gentoo.org mailing list

Re: [gentoo-dev] escaping variables in sed expressions

[Santiago M. Mola]

On Tue, Apr 15, 2008 at 1:14 PM, Marijn Schouten (hkBst)
<[EMAIL PROTECTED]> wrote:
>
>  Hi list,
>
>  it seems I have been using some fragile sed expression and I'd like to tap
> the collective
>  wisdom for avoiding doing that in the future.
>
>  dev-scheme/slib-3.1.5-r1 currently does
>
>  sed "s_prefix = /usr/local/_prefix = ${D}/usr/_" -i Makefile
>
>  to make it not violate the sandbox. However a user had set
>  PORTAGE_TMPDIR=/home/gentoo_overflow/tmp causing the sed expression to
> contain too may
>  underscores and failing.[1]
>
>  There are several option to handle this. I could use a less common
> delimiter or I could
>  escape it: ${D//_/\_} instead of ${D}. I could use a sed expression that
> doesn't suffer
>  from this problem (thanks to dleverton):
>
>  sed -ne '\_^prefix = /usr/local_!{p;d}' -e "iprefix = ${D}" -i Makefile
>
>  Comments?
>

Currently is use ':' as sed delimiter when paths are involved. I'd
also like to hear from you about proper delimiters if you think ':' is
not safe enough.

AFAIK, the only corner case which would make this fail would be
Windows paths (C:/gentoo-prefix).

Regards,

-- 
Santiago M. Mola
Jabber ID: [EMAIL PROTECTED]
-- 
gentoo-dev@lists.gentoo.org mailing list

[gentoo-dev] escaping variables in sed expressions

[Marijn Schouten (hkBst)]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

Hi list,

it seems I have been using some fragile sed expression and I'd like to tap the 
collective
wisdom for avoiding doing that in the future.

dev-scheme/slib-3.1.5-r1 currently does

sed "s_prefix = /usr/local/_prefix = ${D}/usr/_" -i Makefile

to make it not violate the sandbox. However a user had set
PORTAGE_TMPDIR=/home/gentoo_overflow/tmp causing the sed expression to contain 
too may
underscores and failing.[1]

There are several option to handle this. I could use a less common delimiter or 
I could
escape it: ${D//_/\_} instead of ${D}. I could use a sed expression that 
doesn't suffer
from this problem (thanks to dleverton):

sed -ne '\_^prefix = /usr/local_!{p;d}' -e "iprefix = ${D}" -i Makefile

Comments?

Marijn

[1]: http://bugs.gentoo.org/show_bug.cgi?id=217735

- --
Marijn Schouten (hkBst), Gentoo Lisp project, Gentoo ML
, #gentoo-{lisp,ml} on FreeNode
-BEGIN PGP SIGNATURE-
Version: GnuPG v2.0.9 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iEYEARECAAYFAkgEjjEACgkQp/VmCx0OL2zGDQCcCcgx1/g/UXpB38HIjKjNhmL6
S4MAoK1aXJS6SW9FaZT4i2iaeo6AlD2u
=Id31
-END PGP SIGNATURE-
--
gentoo-dev@lists.gentoo.org mailing list