Re: [OE-core] [PATCH v2 2/3] oe.path: preserve xattr in copytree() and copyhardlinktree()

2016-08-31 Thread Richard Purdie 
On Wed, 2016-08-31 at 08:41 +0800, Mark Hatle wrote:
> On 8/30/16 9:05 PM, Joshua Lock wrote:
> > Pass appropriate options to tar invocations in copytree() and
> > copyhardlinktree() to ensure that any extended attributes on the
> > files
> > are preserved during the copy.
> > 
> > We have to drop the use cpio in "Copy-pass" mode in
> > copyhardlinktree()
> > because cpio doesn't support extended attributes on files. Instead
> > we
> > revert back to using cp with different patterns depending on
> > whether
> > or not the directory contains dot files.
> > 
> > Signed-off-by: Joshua Lock 
> > ---
> >  meta/lib/oe/path.py | 11 ---
> >  1 file changed, 8 insertions(+), 3 deletions(-)
> > 
> > diff --git a/meta/lib/oe/path.py b/meta/lib/oe/path.py
> > index 3c07df3..631c3b4 100644
> > --- a/meta/lib/oe/path.py
> > +++ b/meta/lib/oe/path.py
> > @@ -65,7 +65,7 @@ def copytree(src, dst):
> >  # This way we also preserve hardlinks between files in the
> > tree.
> >  
> >  bb.utils.mkdirhier(dst)
> > -cmd = 'tar -cf - -C %s -p . | tar -xf - -C %s' % (src, dst)
> > +cmd = "tar --xattrs --xattrs-include='*' -cf - -C %s -p . |
> > tar --xattrs --xattrs-include='*' -xf - -C %s" % (src, dst)
> >  subprocess.check_output(cmd, shell=True,
> > stderr=subprocess.STDOUT)
> >  
> >  def copyhardlinktree(src, dst):
> > @@ -77,9 +77,14 @@ def copyhardlinktree(src, dst):
> >  if (os.stat(src).st_dev ==  os.stat(dst).st_dev):
> >  # Need to copy directories only with tar first since cp
> > will error if two 
> >  # writers try and create a directory at the same time
> > -cmd = 'cd %s; find . -type d -print | tar -cf - -C %s -p -
> > -no-recursion --files-from - | tar -xf - -C %s' % (src, src, dst)
> > +cmd = "cd %s; find . -type d -print | tar --xattrs -
> > -xattrs-include='*' -cf - -C %s -p --no-recursion --files-from - |
> > tar --xattrs --xattrs-include='*' -xf - -C %s" % (src, src, dst)
> >  subprocess.check_output(cmd, shell=True,
> > stderr=subprocess.STDOUT)
> > -cmd = 'cd %s; find . -print0 | cpio --null -pdlu %s' %
> > (src, dst)
> > +if os.path.isdir(src):
> > +import glob
> > +if len(glob.glob('%s/.??*' % src)) > 0:
> > +src = src + '/.??* '
> > +src = src + '/*'
> > +cmd = 'cp -afl --preserve=xattr %s %s' % (src, dst)
> 
> 'cp -a' is a gnu-ism.  I'm not sure if this will cause any problems,
> but it has
> in the past when people have used non coreutils 'cp'.
> 
> (I'm not sure if we're using 'cp -a' anywhere, but I know most of the
> time we
> try to use the discrete set of arguments instead as they're more
> POSIX.)

We have a number of cp -a calls around. I'm not against fixing that but
for now I'm going to take the patch as I have bigger problems to deal
with.

Cheers,

Richard


-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [OE-core] [PATCH v2 2/3] oe.path: preserve xattr in copytree() and copyhardlinktree()

2016-08-30 Thread Mark Hatle 
On 8/30/16 9:05 PM, Joshua Lock wrote:
> Pass appropriate options to tar invocations in copytree() and
> copyhardlinktree() to ensure that any extended attributes on the files
> are preserved during the copy.
> 
> We have to drop the use cpio in "Copy-pass" mode in copyhardlinktree()
> because cpio doesn't support extended attributes on files. Instead we
> revert back to using cp with different patterns depending on whether
> or not the directory contains dot files.
> 
> Signed-off-by: Joshua Lock 
> ---
>  meta/lib/oe/path.py | 11 ---
>  1 file changed, 8 insertions(+), 3 deletions(-)
> 
> diff --git a/meta/lib/oe/path.py b/meta/lib/oe/path.py
> index 3c07df3..631c3b4 100644
> --- a/meta/lib/oe/path.py
> +++ b/meta/lib/oe/path.py
> @@ -65,7 +65,7 @@ def copytree(src, dst):
>  # This way we also preserve hardlinks between files in the tree.
>  
>  bb.utils.mkdirhier(dst)
> -cmd = 'tar -cf - -C %s -p . | tar -xf - -C %s' % (src, dst)
> +cmd = "tar --xattrs --xattrs-include='*' -cf - -C %s -p . | tar --xattrs 
> --xattrs-include='*' -xf - -C %s" % (src, dst)
>  subprocess.check_output(cmd, shell=True, stderr=subprocess.STDOUT)
>  
>  def copyhardlinktree(src, dst):
> @@ -77,9 +77,14 @@ def copyhardlinktree(src, dst):
>  if (os.stat(src).st_dev ==  os.stat(dst).st_dev):
>  # Need to copy directories only with tar first since cp will error 
> if two 
>  # writers try and create a directory at the same time
> -cmd = 'cd %s; find . -type d -print | tar -cf - -C %s -p 
> --no-recursion --files-from - | tar -xf - -C %s' % (src, src, dst)
> +cmd = "cd %s; find . -type d -print | tar --xattrs 
> --xattrs-include='*' -cf - -C %s -p --no-recursion --files-from - | tar 
> --xattrs --xattrs-include='*' -xf - -C %s" % (src, src, dst)
>  subprocess.check_output(cmd, shell=True, stderr=subprocess.STDOUT)
> -cmd = 'cd %s; find . -print0 | cpio --null -pdlu %s' % (src, dst)
> +if os.path.isdir(src):
> +import glob
> +if len(glob.glob('%s/.??*' % src)) > 0:
> +src = src + '/.??* '
> +src = src + '/*'
> +cmd = 'cp -afl --preserve=xattr %s %s' % (src, dst)

'cp -a' is a gnu-ism.  I'm not sure if this will cause any problems, but it has
in the past when people have used non coreutils 'cp'.

(I'm not sure if we're using 'cp -a' anywhere, but I know most of the time we
try to use the discrete set of arguments instead as they're more POSIX.)

(Otherwise the change looks good to me.)

>  subprocess.check_output(cmd, shell=True, stderr=subprocess.STDOUT)
>  else:
>  copytree(src, dst)
> 

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH v2 2/3] oe.path: preserve xattr in copytree() and copyhardlinktree()

2016-08-30 Thread Joshua Lock 
Pass appropriate options to tar invocations in copytree() and
copyhardlinktree() to ensure that any extended attributes on the files
are preserved during the copy.

We have to drop the use cpio in "Copy-pass" mode in copyhardlinktree()
because cpio doesn't support extended attributes on files. Instead we
revert back to using cp with different patterns depending on whether
or not the directory contains dot files.

Signed-off-by: Joshua Lock 
---
 meta/lib/oe/path.py | 11 ---
 1 file changed, 8 insertions(+), 3 deletions(-)

diff --git a/meta/lib/oe/path.py b/meta/lib/oe/path.py
index 3c07df3..631c3b4 100644
--- a/meta/lib/oe/path.py
+++ b/meta/lib/oe/path.py
@@ -65,7 +65,7 @@ def copytree(src, dst):
 # This way we also preserve hardlinks between files in the tree.
 
 bb.utils.mkdirhier(dst)
-cmd = 'tar -cf - -C %s -p . | tar -xf - -C %s' % (src, dst)
+cmd = "tar --xattrs --xattrs-include='*' -cf - -C %s -p . | tar --xattrs 
--xattrs-include='*' -xf - -C %s" % (src, dst)
 subprocess.check_output(cmd, shell=True, stderr=subprocess.STDOUT)
 
 def copyhardlinktree(src, dst):
@@ -77,9 +77,14 @@ def copyhardlinktree(src, dst):
 if (os.stat(src).st_dev ==  os.stat(dst).st_dev):
 # Need to copy directories only with tar first since cp will error if 
two 
 # writers try and create a directory at the same time
-cmd = 'cd %s; find . -type d -print | tar -cf - -C %s -p 
--no-recursion --files-from - | tar -xf - -C %s' % (src, src, dst)
+cmd = "cd %s; find . -type d -print | tar --xattrs 
--xattrs-include='*' -cf - -C %s -p --no-recursion --files-from - | tar 
--xattrs --xattrs-include='*' -xf - -C %s" % (src, src, dst)
 subprocess.check_output(cmd, shell=True, stderr=subprocess.STDOUT)
-cmd = 'cd %s; find . -print0 | cpio --null -pdlu %s' % (src, dst)
+if os.path.isdir(src):
+import glob
+if len(glob.glob('%s/.??*' % src)) > 0:
+src = src + '/.??* '
+src = src + '/*'
+cmd = 'cp -afl --preserve=xattr %s %s' % (src, dst)
 subprocess.check_output(cmd, shell=True, stderr=subprocess.STDOUT)
 else:
 copytree(src, dst)
-- 
2.7.4

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core