Re: is file open in system ? - other than lsof
bvidinli schrieb: > is there a way to find out if file open in system ? - > please write if you know a way other than lsof. because lsof if slow for me. > i need a faster way. > i deal with thousands of files... so, i need a faster / python way for this. > thanks. I think you can do this with inotify. It's an event based notification mechanism for linux kernel 2.6.13 and up. It has python bindings available (google for pyinotify). You will receive events like IN_OPEN,IN_CLOSE,etc. and keep track of opened files this way. hth Paul -- http://mail.python.org/mailman/listinfo/python-list
Re: Fwd: is file open in system ? - other than lsof
On 2008-04-17, bvidinli <[EMAIL PROTECTED]> wrote: > is there another way, any python command sequence that i can check if > a file is open at the time of "before i process file" > > i am not interested in " the file may be written after i access it.." > the important point is " the time at i first access it." > > my routine is something like: > for i in listoffiles: > checkfileopen(i) > processfile() This code does not give you what you are after; in between 'checkfileopen()' and 'processfile()' somebody may open the file and mess with it. I quite doubt that you can get what you want, at OS level. Even if you get exclusive open(), you are not safe imho. After you opened the file for access (and before you perform the read() call), another process may also open it, and alter the data while you are reading. The same may happen between 2 read() calls. Note that a read() at OS level is not the same as a read() at Python (or C fread()) level. Python pretends to read an entire file in one call, but the OS is using disk-blocks of the underlying file system as unit of read/write. In other words, even if nobody messes with the file at the moment you open it, you don't necessarily get an uncorrupted file afaik. Sincerely, Albert -- http://mail.python.org/mailman/listinfo/python-list
Fwd: is file open in system ? - other than lsof
the idea of eg does not work for me because,
what i do:
get list of files,
check 1st file open ?
process 1st file, (this may take several seconds... )
check 2nd file open? i have to check it here, because 2nd file may be
opened while i process file 1
process 2nd file, (this may take several seconds... )
and so on..
in this case, having list of open files at begining of processes is useless...
anyway thanks.
-- Forwarded message --
From: Nick Craig-Wood <[EMAIL PROTECTED]>
Date: 17.Nis.2008 19:30
Subject: Re: is file open in system ? - other than lsof
To: python-list@python.org
Thomas Guettler <[EMAIL PROTECTED]> wrote:
> bvidinli schrieb:
> > is there a way to find out if file open in system ? -
> > please write if you know a way other than lsof. because lsof if
slow for me.
> > i need a faster way.
> > i deal with thousands of files... so, i need a faster / python
way for this.
> > thanks.
>
> On Linux there are symlinks from /proc/PID/fd to the open
> files. You could use this:
>
> #!/usr/bin/env python
> import os
> pids=os.listdir('/proc')
> for pid in sorted(pids):
> try:
> int(pid)
> except ValueError:
> continue
> fd_dir=os.path.join('/proc', pid, 'fd')
> for file in os.listdir(fd_dir):
> try:
> link=os.readlink(os.path.join(fd_dir, file))
> except OSError:
> continue
> print pid, link
Unfortunately I think that is pretty much exactly what lsof does so is
unlikely to be any faster!
However if you have 1000s of files to check you only need to do the
above scan once and build a dict with all open files in whereas lsof
will do it once per call so that may be a useful speedup.
Eg
import os
pids=os.listdir('/proc')
open_files = {}
for pid in sorted(pids):
try:
int(pid)
except ValueError:
continue
fd_dir=os.path.join('/proc', pid, 'fd')
try:
fds = os.listdir(fd_dir)
except OSError:
continue
for file in fds:
try:
link=os.readlink(os.path.join(fd_dir, file))
except OSError:
continue
if not os.path.exists(link):
continue
open_files.setdefault(link, []).append(pid)
for link in sorted(open_files.keys()):
print "%s : %s" % (link, ", ".join(map(str, open_files[link])))
You might want to consider http://pyinotify.sourceforge.net/ depending
on exactly what you are doing...
--
Nick Craig-Wood <[EMAIL PROTECTED]> -- http://www.craig-wood.com/nick
--
http://mail.python.org/mailman/listinfo/python-list
--
İ.Bahattin Vidinli
Elk-Elektronik Müh.
---
iletisim bilgileri (Tercih sirasina gore):
skype: bvidinli (sesli gorusme icin, www.skype.com)
msn: [EMAIL PROTECTED]
yahoo: bvidinli
+90.532.7990607
+90.505.5667711
--
http://mail.python.org/mailman/listinfo/python-list
Re: is file open in system ? - other than lsof
Thomas Guettler <[EMAIL PROTECTED]> wrote:
> bvidinli schrieb:
> > is there a way to find out if file open in system ? -
> > please write if you know a way other than lsof. because lsof if slow for
> > me.
> > i need a faster way.
> > i deal with thousands of files... so, i need a faster / python way for this.
> > thanks.
>
> On Linux there are symlinks from /proc/PID/fd to the open
> files. You could use this:
>
> #!/usr/bin/env python
> import os
> pids=os.listdir('/proc')
> for pid in sorted(pids):
> try:
> int(pid)
> except ValueError:
> continue
> fd_dir=os.path.join('/proc', pid, 'fd')
> for file in os.listdir(fd_dir):
> try:
> link=os.readlink(os.path.join(fd_dir, file))
> except OSError:
> continue
> print pid, link
Unfortunately I think that is pretty much exactly what lsof does so is
unlikely to be any faster!
However if you have 1000s of files to check you only need to do the
above scan once and build a dict with all open files in whereas lsof
will do it once per call so that may be a useful speedup.
Eg
import os
pids=os.listdir('/proc')
open_files = {}
for pid in sorted(pids):
try:
int(pid)
except ValueError:
continue
fd_dir=os.path.join('/proc', pid, 'fd')
try:
fds = os.listdir(fd_dir)
except OSError:
continue
for file in fds:
try:
link=os.readlink(os.path.join(fd_dir, file))
except OSError:
continue
if not os.path.exists(link):
continue
open_files.setdefault(link, []).append(pid)
for link in sorted(open_files.keys()):
print "%s : %s" % (link, ", ".join(map(str, open_files[link])))
You might want to consider http://pyinotify.sourceforge.net/ depending
on exactly what you are doing...
--
Nick Craig-Wood <[EMAIL PROTECTED]> -- http://www.craig-wood.com/nick
--
http://mail.python.org/mailman/listinfo/python-list
Re: is file open in system ? - other than lsof
bvidinli schrieb:
> is there a way to find out if file open in system ? -
> please write if you know a way other than lsof. because lsof if slow for me.
> i need a faster way.
> i deal with thousands of files... so, i need a faster / python way for this.
> thanks.
>
>
On Linux there are symlinks from /proc/PID/fd to the open
files. You could use this:
#!/usr/bin/env python
import os
pids=os.listdir('/proc')
for pid in sorted(pids):
try:
int(pid)
except ValueError:
continue
fd_dir=os.path.join('/proc', pid, 'fd')
for file in os.listdir(fd_dir):
try:
link=os.readlink(os.path.join(fd_dir, file))
except OSError:
continue
print pid, link
--
Thomas Guettler, http://www.thomas-guettler.de/
E-Mail: guettli (*) thomas-guettler + de
--
http://mail.python.org/mailman/listinfo/python-list
Fwd: is file open in system ? - other than lsof
Here is (hope) complete info for my question:
i use linux/unix
i use python 2.3
my subject is a single file at a time. but i have to check thousands
of files in a loop.
so, since there is thousands of files to process, i dont want to use
like os.system('lsof filename')
because lsof is slow regarding, when you called it thousands of times.
is there another way, any python command sequence that i can check if
a file is open at the time of "before i process file"
i am not interested in " the file may be written after i access it.."
the important point is " the time at i first access it."
my routine is something like:
for i in listoffiles:
checkfileopen(i)
processfile()
thats it...
i hope this will clear my question..
thank you a lot.
-- Forwarded message --
From: Chris McAloney <[EMAIL PROTECTED]>
Date: 16.Nis.2008 17:00
Subject: Re: is file open in system ? - other than lsof
To: python-list@python.org
On 16-Apr-08, at 9:20 AM, A.T.Hofkamp wrote:
> On 2008-04-16, bvidinli <[EMAIL PROTECTED]> wrote:
>> is there a way to find out if file open in system ? -
>> please write if you know a way other than lsof. because lsof if
>> slow for me.
>> i need a faster way.
>> i deal with thousands of files... so, i need a faster / python way
>> for this.
>> thanks.
>
> This is not a Python question but an OS question.
> (Python is not going to deliver what the OS doesn't provide).
>
> Please first find an alternative way at OS level (ie ask this
> question at an
> appropiate OS news group). Once you have found that, you can think
> about Python
> support for that alternative.
I agree with Albert that this is very operating-system specific.
Since you mentioned 'lsof', I'll assume that you are at least using a
Unix variant, meaning that the fcntl module will be available to you,
so you can check if the file is already locked.
Beyond that, I think more information on your application would be
necessary before we could give you a solid answer. Do you only need
to know if the file is open, or do you want only the files that are
open for writing? If you only care about the files that are open for
writing, then checking for a write-lock with fcntl will probably do
the trick. Are you planning to check all of the "thousands of files"
individually to determine if they're open? If so, I think it's
unlikely that doing this from Python will actually be faster than a
single 'lsof' call.
If you're on Linux, you might also want to have a look at the /proc
directory tree ("man proc"), as this is where lsof gets its
information from on Linux machines.
Chris
--
http://mail.python.org/mailman/listinfo/python-list
--
İ.Bahattin Vidinli
Elk-Elektronik Müh.
---
iletisim bilgileri (Tercih sirasina gore):
skype: bvidinli (sesli gorusme icin, www.skype.com)
msn: [EMAIL PROTECTED]
yahoo: bvidinli
+90.532.7990607
+90.505.5667711
--
http://mail.python.org/mailman/listinfo/python-list
Re: is file open in system ? - other than lsof
On Wed, Apr 16, 2008 at 10:00 AM, Chris McAloney <[EMAIL PROTECTED]> wrote:
> On 16-Apr-08, at 9:20 AM, A.T.Hofkamp wrote:
> > On 2008-04-16, bvidinli <[EMAIL PROTECTED]> wrote:
> >> is there a way to find out if file open in system ? -
> >> please write if you know a way other than lsof. because lsof if
> >> slow for me.
> >> i need a faster way.
> >> i deal with thousands of files... so, i need a faster / python way
> >> for this.
> >> thanks.
> >
> > This is not a Python question but an OS question.
> > (Python is not going to deliver what the OS doesn't provide).
> >
> > Please first find an alternative way at OS level (ie ask this
> > question at an
> > appropiate OS news group). Once you have found that, you can think
> > about Python
> > support for that alternative.
>
> I agree with Albert that this is very operating-system specific.
> Since you mentioned 'lsof', I'll assume that you are at least using a
> Unix variant, meaning that the fcntl module will be available to you,
> so you can check if the file is already locked.
>
> Beyond that, I think more information on your application would be
> necessary before we could give you a solid answer. Do you only need
> to know if the file is open, or do you want only the files that are
> open for writing? If you only care about the files that are open for
> writing, then checking for a write-lock with fcntl will probably do
> the trick. Are you planning to check all of the "thousands of files"
> individually to determine if they're open? If so, I think it's
> unlikely that doing this from Python will actually be faster than a
> single 'lsof' call.
>
> If you're on Linux, you might also want to have a look at the /proc
> directory tree ("man proc"), as this is where lsof gets its
> information from on Linux machines.
>
> Chris
> --
>
I know this is a python list, but if speed is such an issue you might want
to consider writing in C/C++. Both would be considerably faster than
python.
> http://mail.python.org/mailman/listinfo/python-list
>
--
http://mail.python.org/mailman/listinfo/python-list
Re: is file open in system ? - other than lsof
On 16-Apr-08, at 9:20 AM, A.T.Hofkamp wrote:
> On 2008-04-16, bvidinli <[EMAIL PROTECTED]> wrote:
>> is there a way to find out if file open in system ? -
>> please write if you know a way other than lsof. because lsof if
>> slow for me.
>> i need a faster way.
>> i deal with thousands of files... so, i need a faster / python way
>> for this.
>> thanks.
>
> This is not a Python question but an OS question.
> (Python is not going to deliver what the OS doesn't provide).
>
> Please first find an alternative way at OS level (ie ask this
> question at an
> appropiate OS news group). Once you have found that, you can think
> about Python
> support for that alternative.
I agree with Albert that this is very operating-system specific.
Since you mentioned 'lsof', I'll assume that you are at least using a
Unix variant, meaning that the fcntl module will be available to you,
so you can check if the file is already locked.
Beyond that, I think more information on your application would be
necessary before we could give you a solid answer. Do you only need
to know if the file is open, or do you want only the files that are
open for writing? If you only care about the files that are open for
writing, then checking for a write-lock with fcntl will probably do
the trick. Are you planning to check all of the "thousands of files"
individually to determine if they're open? If so, I think it's
unlikely that doing this from Python will actually be faster than a
single 'lsof' call.
If you're on Linux, you might also want to have a look at the /proc
directory tree ("man proc"), as this is where lsof gets its
information from on Linux machines.
Chris
--
http://mail.python.org/mailman/listinfo/python-list
Re: is file open in system ? - other than lsof
On 2008-04-16, bvidinli <[EMAIL PROTECTED]> wrote: > is there a way to find out if file open in system ? - > please write if you know a way other than lsof. because lsof if slow for me. > i need a faster way. > i deal with thousands of files... so, i need a faster / python way for this. > thanks. This is not a Python question but an OS question. (Python is not going to deliver what the OS doesn't provide). Please first find an alternative way at OS level (ie ask this question at an appropiate OS news group). Once you have found that, you can think about Python support for that alternative. Sincerely, Albert -- http://mail.python.org/mailman/listinfo/python-list
is file open in system ? - other than lsof
is there a way to find out if file open in system ? - please write if you know a way other than lsof. because lsof if slow for me. i need a faster way. i deal with thousands of files... so, i need a faster / python way for this. thanks. -- İ.Bahattin Vidinli Elk-Elektronik Müh. --- iletisim bilgileri (Tercih sirasina gore): skype: bvidinli (sesli gorusme icin, www.skype.com) msn: [EMAIL PROTECTED] yahoo: bvidinli +90.532.7990607 +90.505.5667711 -- http://mail.python.org/mailman/listinfo/python-list
