Re: os.walk and recursive deletion
27 Oct 2007 17:38:10 GMT, Marc 'BlackJack' Rintsch [EMAIL PROTECTED]: On Sat, 27 Oct 2007 18:07:44 +0200, Martin Marcher wrote: I'm playing around with os.walk and I made up del_tree(path) which I think is correct (in terms of the algorithm, but not as python wants it :)). It's not correct in terms of the algorithm if you take the algorithm of `os.walk()` into the equation. `os.walk()` is itself diving recursivly into the subdirectories… True but isn't the problem that I need either backtracking to remember which directories are empty and thus can be deleted or that I need to do another recursion (like in my function) which returns as soon as the new root is deleted or at least empty? I mean having a directory structure with 3 subdiretories, each of them has files in it else: for subdir in subdirs: subdir = os.path.join(cwd, subdir) print We need to recurse into: %s % (subdir, ) del_tree(subdir) …and here you are calling the your function recursively which then calls again `os.walk()` on that subdirectory. That's a little bit too much. I call it recursively here because I * either need to backtrack to remember which directories are empty (and leafs) * or I just recurse my function which yields an OSError because a directory the original os.walk will visit doesn't exist anymore because some recursion already deleted it. (correct me if I'm wrong - but I'd like to stick with os.walk for this as an exercise) Or `shutil.rmtree()`. :-) Nice function but not part of my exercise :) - No it isn't homework just doing it to play around. thanks martin PS: as I'm writing this i just discovered that I missed the topdown=False argument for os.walk. Playing a bit with it the docs seem to say that is more what I want than topdown=True -- http://noneisyours.marcher.name http://feeds.feedburner.com/NoneIsYours -- http://mail.python.org/mailman/listinfo/python-list
os.walk and recursive deletion
Hello,
I'm playing around with os.walk and I made up del_tree(path) which I
think is correct (in terms of the algorithm, but not as python wants
it :)).
As soon as some directory is deleted the iterator of os.walk chokes.
OK that is somehow clear to me as it can't be valid anymore since it
can't go to the just deleted directory but it is in the iterator.
I generated the test data with (bash, don't know of other shell
expansion features)
# root=test
# mkdir $root
# mkdir -p
#
$root/{a,b,c,d,e,f,g,h,i,j,k,l,m,n,o,p,q,r,s,t,u,v,w,x,y,z}/{a,b,c,d,e,f,g,h,i,j,k,l,m,n,o,p,q,r,s,t,u,v,w,x,y,z}
# touch
$root/{a,b,c,d,e,f,g,h,i,j,k,l,m,n,o,p,q,r,s,t,u,v,w,x,y,z}/{0,1,2,3,4,5,6,7,8,9}
# touch
$root/{a,b,c,d,e,f,g,h,i,j,k,l,m,n,o,p,q,r,s,t,u,v,w,x,y,z}/{a,b,c,d,e,f,g,h,i,j,k,l,m,n,o,p,q,r,s,t,u,v,w,x,y,z}/{0,1,2,3,4,5,6,7,8,9}
Here goes the script, as far as I tested it it runs fine in test mode.
Now that is just use the print statements like attached below. as soon
as os.rmdir and os.unlink are involved the iterator chokes. How do I
get around that, or what should I use to be able to recursively delete
everything under a certain subdir?
#--snip
import sys
import os
import unittest
def del_tree(path):
walker = os.walk(path)
for entry in walker:
cwd = entry[0]
subdirs = entry[1]
files = entry[2]
print Checking directory: %s %(cwd, )
if files:
for file in files:
file = os.path.join(cwd, file)
print The file is now: %s % (file, )
#os.unlink(file)
print OK directory %s has no more files, checking for
subdirs... % (cwd, )
else:
print OK directory %s NEVER HAD FILES, checking for
subdirs... % (cwd, )
if not subdirs:
print We can delete: %s % (cwd, )
#os.rmdir(cwd)
else:
for subdir in subdirs:
subdir = os.path.join(cwd, subdir)
print We need to recurse into: %s % (subdir, )
del_tree(subdir)
#os.rmdir(path)
print Removing: %s % (path, )
#--snap
--
http://noneisyours.marcher.name
http://feeds.feedburner.com/NoneIsYours
--
http://mail.python.org/mailman/listinfo/python-list
Re: os.walk and recursive deletion
Martin Marcher wrote:
Hello,
I'm playing around with os.walk and I made up del_tree(path) which I
think is correct (in terms of the algorithm, but not as python wants
it :)).
As soon as some directory is deleted the iterator of os.walk chokes.
OK that is somehow clear to me as it can't be valid anymore since it
can't go to the just deleted directory but it is in the iterator.
I generated the test data with (bash, don't know of other shell
expansion features)
# root=test
# mkdir $root
# mkdir -p
#
$root/{a,b,c,d,e,f,g,h,i,j,k,l,m,n,o,p,q,r,s,t,u,v,w,x,y,z}/{a,b,c,d,e,f,g,h,i,j,k,l,m,n,o,p,q,r,s,t,u,v,w,x,y,z}
# touch
$root/{a,b,c,d,e,f,g,h,i,j,k,l,m,n,o,p,q,r,s,t,u,v,w,x,y,z}/{0,1,2,3,4,5,6,7,8,9}
# touch
$root/{a,b,c,d,e,f,g,h,i,j,k,l,m,n,o,p,q,r,s,t,u,v,w,x,y,z}/{a,b,c,d,e,f,g,h,i,j,k,l,m,n,o,p,q,r,s,t,u,v,w,x,y,z}/{0,1,2,3,4,5,6,7,8,9}
Here goes the script, as far as I tested it it runs fine in test mode.
Now that is just use the print statements like attached below. as soon
as os.rmdir and os.unlink are involved the iterator chokes. How do I
get around that, or what should I use to be able to recursively delete
everything under a certain subdir?
#--snip
import sys
import os
import unittest
def del_tree(path):
walker = os.walk(path)
for entry in walker:
cwd = entry[0]
subdirs = entry[1]
files = entry[2]
print Checking directory: %s %(cwd, )
if files:
for file in files:
file = os.path.join(cwd, file)
print The file is now: %s % (file, )
#os.unlink(file)
print OK directory %s has no more files, checking for
subdirs... % (cwd, )
else:
print OK directory %s NEVER HAD FILES, checking for
subdirs... % (cwd, )
if not subdirs:
print We can delete: %s % (cwd, )
#os.rmdir(cwd)
else:
for subdir in subdirs:
subdir = os.path.join(cwd, subdir)
print We need to recurse into: %s % (subdir, )
del_tree(subdir)
#os.rmdir(path)
print Removing: %s % (path, )
#--snap
While it is possible to mix recursion and os.walk()...
def del_tree(path):
for cwd, subdirs, files in os.walk(path):
print Checking directory: %s %(cwd, )
if files:
for file in files:
file = os.path.join(cwd, file)
print The file is now: %s % (file, )
os.unlink(file)
print OK directory %s has no more files, checking for subdirs...
% (cwd, )
else:
print OK directory %s NEVER HAD FILES, checking for subdirs... %
(cwd, )
for subdir in subdirs:
subdir = os.path.join(cwd, subdir)
print We need to recurse into: %s % (subdir, )
del_tree(subdir)
break # ugly but necessary
print Removing: %s % (path, )
os.rmdir(path)
...the idea behind it is that you use it instead of recursion:
def del_tree(root):
for path, dirs, files in os.walk(root, False):
for fn in files:
os.unlink(os.path.join(path, fn))
for dn in dirs:
os.rmdir(os.path.join(path, dn))
os.rmdir(root)
The second parameter to os.walk() ensures that the tree is walked
depth-first, i. e. the contents of a directory are seen before the
directory itself. Personally, I would prefer a recursive implementation
based on os.listdir(),
def del_tree(root):
for name in os.listdir(root):
path = os.path.join(root, name)
if os.path.isdir(path):
del_tree(path)
else:
os.unlink(path)
os.rmdir(root)
an approach that is also taken by shutils.rmtree().
Peter
--
http://mail.python.org/mailman/listinfo/python-list
Re: os.walk and recursive deletion
On Sat, 27 Oct 2007 18:07:44 +0200, Martin Marcher wrote: Hello, I'm playing around with os.walk and I made up del_tree(path) which I think is correct (in terms of the algorithm, but not as python wants it :)). It's not correct in terms of the algorithm if you take the algorithm of `os.walk()` into the equation. Here goes the script, as far as I tested it it runs fine in test mode. Now that is just use the print statements like attached below. as soon as os.rmdir and os.unlink are involved the iterator chokes. How do I get around that, or what should I use to be able to recursively delete everything under a certain subdir? #--snip import sys import os import unittest def del_tree(path): walker = os.walk(path) `os.walk()` is itself diving recursivly into the subdirectories… for entry in walker: cwd = entry[0] subdirs = entry[1] files = entry[2] print Checking directory: %s %(cwd, ) if files: for file in files: file = os.path.join(cwd, file) print The file is now: %s % (file, ) #os.unlink(file) print OK directory %s has no more files, checking for subdirs... % (cwd, ) else: print OK directory %s NEVER HAD FILES, checking for subdirs... % (cwd, ) if not subdirs: print We can delete: %s % (cwd, ) #os.rmdir(cwd) else: for subdir in subdirs: subdir = os.path.join(cwd, subdir) print We need to recurse into: %s % (subdir, ) del_tree(subdir) …and here you are calling the your function recursively which then calls again `os.walk()` on that subdirectory. That's a little bit too much. Just use `os.listdir()` (and `os.path.isdir()`) in your recursive function. #os.rmdir(path) print Removing: %s % (path, ) #--snap Or `shutil.rmtree()`. :-) Ciao, Marc 'BlackJack' Rintsch -- http://mail.python.org/mailman/listinfo/python-list
