binutils "strings" like functionality?
Hey all: I am working on a little script that needs to pull the strings out of a binary file, and then manipulate them with python. The command line utility "strings" (part of binutils) has exactly the functionality I need, but I was thinking about trying to implement this in pure python. I did some reading on opening and reading binary files, etc., and was just wondering if people think this is possible, or worth my time (as a learning exercise), or if something like this already exists. -cjl -- http://mail.python.org/mailman/listinfo/python-list
Re: binutils "strings" like functionality?
I don't know anything about binutils strings, other than that a quick google reveals that you can search binary files for printable strings with it. In python the following code will read a binary file into a single string : long_string = open(filename, 'rb').read() You can then slice long_string, iterate over it... or do whatever you like with it. Now you *ought* to be creating a proepr file handle and closing it. You probably don't want to read the whole file into memory at once, etc But implementing these thigns are trivial. What particular aspect of 'strings' did you want to implement ? Regards, Fuzzy http://www.voidspace.org.uk/python/index.shtml -- http://mail.python.org/mailman/listinfo/python-list
Re: binutils "strings" like functionality?
Take a look at python's struct module in the standard library. Many people use it to manipulate binary objects at the byte level with great success. Larry Bates cjl wrote: > Hey all: > > I am working on a little script that needs to pull the strings out of a > binary file, and then manipulate them with python. > > The command line utility "strings" (part of binutils) has exactly the > functionality I need, but I was thinking about trying to implement this > in pure python. > > I did some reading on opening and reading binary files, etc., and was > just wondering if people think this is possible, or worth my time (as a > learning exercise), or if something like this already exists. > > -cjl > -- http://mail.python.org/mailman/listinfo/python-list
Re: binutils "strings" like functionality?
cjl wrote: Hey all: I am working on a little script that needs to pull the strings out of a binary file, and then manipulate them with python. The command line utility "strings" (part of binutils) has exactly the functionality I need, but I was thinking about trying to implement this in pure python. I did some reading on opening and reading binary files, etc., and was just wondering if people think this is possible, or worth my time (as a learning exercise), or if something like this already exists. I think a bare-bones version is pretty simple: - read the file - use re.split() to split on non-printable characters (string.printable could be handy here) - print anything in the resulting list that is at least 4 characters Since you mention this as a learning exercise I'll refrain from posting code but it's only a handful of lines...of course implementing all of 'strings' is a bit more work. Kent -- http://mail.python.org/mailman/listinfo/python-list
Re: binutils "strings" like functionality?
Kent Johnson wrote: cjl wrote: Hey all: I am working on a little script that needs to pull the strings out of a binary file, and then manipulate them with python. The command line utility "strings" (part of binutils) has exactly the functionality I need, but I was thinking about trying to implement this in pure python. I did some reading on opening and reading binary files, etc., and was just wondering if people think this is possible, or worth my time (as a learning exercise), or if something like this already exists. I think a bare-bones version is pretty simple: - read the file - use re.split() to split on non-printable characters (string.printable could be handy here) - print anything in the resulting list that is at least 4 characters The binutils version may well use a zero terminator character to differentiate between random chunks of printable characters and those explicitly genreated as strings from C. Or it may not ... Since you mention this as a learning exercise I'll refrain from posting code but it's only a handful of lines...of course implementing all of 'strings' is a bit more work. regards Steve -- Meet the Python developers and your c.l.py favorites March 23-25 Come to PyCon DC 2005 http://www.pycon.org/ Steve Holden http://www.holdenweb.com/ -- http://mail.python.org/mailman/listinfo/python-list
Re: binutils "strings" like functionality?
"cjl" wrote:
> I am working on a little script that needs to pull the strings out of a
> binary file, and then manipulate them with python.
>
> The command line utility "strings" (part of binutils) has exactly the
> functionality I need, but I was thinking about trying to implement this
> in pure python.
something like this could work:
import re
text = open(file, "rb").read()
for m in re.finditer("([\x20-\x7f]{4,})[\n\0]", text):
print m.start(), repr(m.group(1))
you may wish to modify the "[\x20-\x7f]" part to match your definition of
"printable characters". "[-,.!?\w ]" is a reasonable choice in many cases...
if the files can be huge, use the mmap module to map the file into memory,
and run the RE on the mapped view.
--
http://mail.python.org/mailman/listinfo/python-list
Re: binutils "strings" like functionality?
Fredrik Lundh wrote:
> something like this could work:
>
> import re
>
> text = open(file, "rb").read()
>
> for m in re.finditer("([\x20-\x7f]{4,})[\n\0]", text):
> print m.start(), repr(m.group(1))
Hey...that worked. I actually modified:
for m in re.finditer("([\x20-\x7f]{4,})[\n\0]", text):
to
for m in re.finditer("([\x20-\x7f]{4,})", text):
and now the output is nearly identical to 'strings'. One problem
exists, in that if the binary file contains a string
"monkey/chicken/dog/cat" it is printed as "mokey//chicken//dog//cat",
and I don't know enough to figure out where the extra "/" is coming
from.
Help?
-CJL
--
http://mail.python.org/mailman/listinfo/python-list
Re: binutils "strings" like functionality?
"cjl" <[EMAIL PROTECTED]> writes:
> Fredrik Lundh wrote:
>
>> something like this could work:
>>
>> import re
>>
>> text = open(file, "rb").read()
>>
>> for m in re.finditer("([\x20-\x7f]{4,})[\n\0]", text):
>> print m.start(), repr(m.group(1))
>
> Hey...that worked. I actually modified:
>
> for m in re.finditer("([\x20-\x7f]{4,})[\n\0]", text):
>
> to
>
> for m in re.finditer("([\x20-\x7f]{4,})", text):
>
> and now the output is nearly identical to 'strings'. One problem
> exists, in that if the binary file contains a string
> "monkey/chicken/dog/cat" it is printed as "mokey//chicken//dog//cat",
> and I don't know enough to figure out where the extra "/" is coming
> from.
Are you sure it's monkey/chicken/dog/cat, and not
monkey\chicken\dog\cat? The later one will print monkey\\chicken...
because of the repr() call.
Also, you probably want it as [\x20-\x7e] (the DEL character \x7f
isn't printable). You're also missing tabs (\t).
The GNU binutils string utility looks for \t or [\x20-\x7e].
--
|>|\/|<
/--\
|David M. Cooke
|cookedm(at)physics(dot)mcmaster(dot)ca
--
http://mail.python.org/mailman/listinfo/python-list
Re: binutils "strings" like functionality?
David M. Cooke wrote: > Are you sure it's monkey/chicken/dog/cat, and not > monkey\chicken\dog\cat? The later one will print monkey\\chicken... > because of the repr() call. > > Also, you probably want it as [\x20-\x7e] (the DEL character \x7f > isn't printable). You're also missing tabs (\t). > > The GNU binutils string utility looks for \t or [\x20-\x7e]. Yeah, it is "monkey\chicken\dog\cat", thank you for pointing that out. I started snooping throught the sourcecode for 'strings' in binutils to see what it matches against, but I don't really know how to program, so you just saved me even more time. Thanks again! -CJL -- http://mail.python.org/mailman/listinfo/python-list
Re: binutils "strings" like functionality?
In <[EMAIL PROTECTED]>, cjl wrote: > I am working on a little script that needs to pull the strings out of a > binary file, and then manipulate them with python. > > The command line utility "strings" (part of binutils) has exactly the > functionality I need, but I was thinking about trying to implement this > in pure python. > > I did some reading on opening and reading binary files, etc., and was > just wondering if people think this is possible, or worth my time (as a > learning exercise), or if something like this already exists. If you find it interesting and challenging go ahead and try it. It is possible and with the help of the 're' and 'mmap' modules it should be quite easy to filter strings out of files without the need to load them into memory at once. Ciao, Marc 'BlackJack' Rintsch -- http://mail.python.org/mailman/listinfo/python-list
