Re: [Tutor] how to convert '\xf0' to 0xf0?
What exactly do you want to achieve? I suspect its not what you asked!
To convert the character whose hex value of F0 to an integer value of
F0
you can use ord() (or just reinterpret the character as a string
using the struct module).
ord('\xf0')
240
To display the hex value of a character as a hex string you can use
string formatting or the hex() function:
c = '\xF0'
0x%x % ord(c)
'0xf0'
0x%X % ord(c)
'0xF0'
hex(ord(c))
'0xf0'
Remember that the representation of a number is always a string.
The value of a number is always binary and can be represented in
any base with the default in Python being decimal.
HTH,
--
Alan Gauld
Author of the Learn to Program web site
http://www.freenetpages.co.uk/hp/alan.gauld
___
Tutor maillist - Tutor@python.org
http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] reading output from a c executable.
On Fri, 12 Dec 2008 03:13:16 -0800, Ravi Kondamuru wrote: Denis, These are 32bit, 64bit counters (essentially numbers). Bob, There are well over 10K counters in the log file that are updated every 5 secs. If a counter1's graph was requested, log will have to be parsed once to get the data points. If a user asked for a counter2, now it needs to be retrieved from the log file. Which essentially means having to go through the logfile again. This is because i want to try to avoid using a database to store values after parsing the file. thanks, Ravi. Instead of using a file, why not use pipes? Using pipes is surely much faster than reading from a file, and using pipes allow you to send log entries in messages which means you don't need to reparse the whole logfile on every update, just the parts that have changed. ___ Tutor maillist - Tutor@python.org http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] reading output from a c executable.
Denis, These are 32bit, 64bit counters (essentially numbers). Bob, There are well over 10K counters in the log file that are updated every 5 secs. If a counter1's graph was requested, log will have to be parsed once to get the data points. If a user asked for a counter2, now it needs to be retrieved from the log file. Which essentially means having to go through the logfile again. This is because i want to try to avoid using a database to store values after parsing the file. thanks, Ravi. On Thu, Dec 11, 2008 at 7:54 PM, bob gailer bgai...@gmail.com wrote: Ravi Kondamuru wrote: Reasons for C: 1. The log files I am working are 60-100MB files. I *assumed* using C will reduce the parse time. 2. The records themselves are variable length and hence was concerned about the complexity for implementation in python. 3. Since I am not using a database, each request to refresh the graph (changing time duration for display) will have to re-parse the file all over again. And hence speed in parsing the log is important. There should never be a need to reparse the same file. Please present the case more specifically. -- Bob Gailer Chapel Hill NC 919-636-4239 ___ Tutor maillist - Tutor@python.org http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] advice on regex matching for dates?
On Thu, 11 Dec 2008 23:38:52 +0100, spir wrote:
Serdar Tumgoren a écrit :
Hey everyone,
I was wondering if there is a way to use the datetime module to check
for variations on a month name when performing a regex match?
In the script below, I created a regex pattern that checks for dates in
the following pattern: August 31, 2007. If there's a match, I can
then print the capture date and the line from which it was extracted.
While it works in this isolated case, it struck me as not very
flexible. What happens when I inevitably get data that has dates
formatted in a different way? Do I have to create some type of library
that contains variations on each month name (i.e. - January, Jan., 01,
1...) and use that to parse each line?
Or is there some way to use datetime to check for date patterns when
using regex? Is there a best practice in this area that I'm unaware
of in this area?
Apologies if this question has been answered elsewhere. I wasn't sure
how to research this topic (beyond standard datetime docs), but I'd be
happy to RTM if someone can point me to some resources.
Any suggestions are welcome (including optimizations of the code
below).
Regards,
Serdar
#!/usr/bin/env python
import re, sys
sourcefile = open(sys.argv[1],'r')
pattern =
re.compile(r'(?PmonthJanuary|February|March|April|May|June|July|
August|September|October|November|December)\s(?Pday\d{1,2}),\s(?Pyear
\d{4})')
pattern2 = re.compile(r'Return to List')
counter = 0
for line in sourcefile:
x = pattern.search(line)
break_point = pattern2.match(line)
if x:
counter +=1
print %s %d, %d == %s % ( x.group('month'),
int(x.group('day')),
int(x.group('year')), line ),
elif break_point:
break
print counter
sourcefile.close()
I just found a simple, but nice, trick to make regexes less unlegible.
Using substrings to represent sub-patterns. E.g. instead of:
p =
re.compile(r'(?PmonthJanuary|February|March|April|May|June|July|
August|September|October|November|December)\s(?Pday\d{1,2}),\s(?Pyear
\d{4})')
write first:
month =
r'(?PmonthJanuary|February|March|April|May|June|July|August|September|
October|November|December)'
day = r'(?Pday\d{1,2})'
year = r'(?Pyear\d{4})'
then:
p = re.compile( r%s\s%s,\s%s % (month,day,year) ) or even:
p = re.compile( r%(month)s\s%(day)s,\s%(year)s %
{'month':month,'day':day,'year':year} )
You might realize that that wouldn't help much and might even hide bugs
(or make it harder to see). regex IS hairy in the first place.
I'd vote for namespace in regex to allow a pattern inside a special pair
tags to be completely ignorant of anything outside it. And also I'd vote
for a compiled-pattern to be used as a subpattern. But then, with all
those limbs, I might as well use a full-blown syntax parser instead.
___
Tutor maillist - Tutor@python.org
http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] reading output from a c executable.
Ravi Kondamuru wrote: Denis, These are 32bit, 64bit counters (essentially numbers). Bob, There are well over 10K counters in the log file that are updated every 5 secs. If a counter1's graph was requested, log will have to be parsed once to get the data points. If a user asked for a counter2, now it needs to be retrieved from the log file. Which essentially means having to go through the logfile again. This is because i want to try to avoid using a database to store values after parsing the file. Thank you. When you said same file I did not realize that it meant same name modified data! If the file is a mix of 32 and 64 bit numbers, how do you know the length of each counter? -- Bob Gailer Chapel Hill NC 919-636-4239 ___ Tutor maillist - Tutor@python.org http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] reading output from a c executable.
Ravi Kondamuru wrote:
Denis, These are 32bit, 64bit counters (essentially numbers).
Bob, There are well over 10K counters in the log file that are updated
every 5 secs. If a counter1's graph was requested, log will have to be
parsed once to get the data points. If a user asked for a counter2,
now it needs to be retrieved from the log file. Which essentially
means having to go through the logfile again. This is because i want
to try to avoid using a database to store values after parsing the file.
Here is a little test program I wrote to check timing on a file of
*100,000* 64 bit counters.
import time, struct
ctrs = 10
s = time.time()
# create a file of 800,000 characters (100,000 64 bit numbers)
f = open('ravi.dat', 'wb')
for i in range(ctrs):
f.write('abcdefgh') # it does not matter what we write as long as it
is 8 bytes.
print time.time() - s
s = time.time()
l = []
# read the file
f = open('ravi.dat', 'rb').read()
# unpack each 8 characters into an integer and collect in a list
for b in range(0, ctrs, 8):
n = struct.unpack('q', f[b:b+8])
l.append(n)
print time.time() - s
Writing the file took 0.14 seconds on my computer
Reading and unpacking 0.04 seconds.
I think you can set performance issues aside.
There is a principle in programming that proposes:
1 - get a program running. Preferably in Python as development time is
much faster.
2 - check its performance.
3 - refine as needed.
--
Bob Gailer
Chapel Hill NC
919-636-4239
___
Tutor maillist - Tutor@python.org
http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] regex sub/super patterns ( was:advice on regex matching for dates?)
Lie Ryan a écrit :
I just found a simple, but nice, trick to make regexes less unlegible.
Using substrings to represent sub-patterns. E.g. instead of:
p =
re.compile(r'(?PmonthJanuary|February|March|April|May|June|July|
August|September|October|November|December)\s(?Pday\d{1,2}),\s(?Pyear
\d{4})')
write first:
month =
r'(?PmonthJanuary|February|March|April|May|June|July|August|September|
October|November|December)'
day = r'(?Pday\d{1,2})'
year = r'(?Pyear\d{4})'
then:
p = re.compile( r%s\s%s,\s%s % (month,day,year) ) or even:
p = re.compile( r%(month)s\s%(day)s,\s%(year)s %
{'month':month,'day':day,'year':year} )
You might realize that that wouldn't help much and might even hide bugs
(or make it harder to see). regex IS hairy in the first place.
I'd vote for namespace in regex to allow a pattern inside a special pair
tags to be completely ignorant of anything outside it. And also I'd vote
for a compiled-pattern to be used as a subpattern. But then, with all
those limbs, I might as well use a full-blown syntax parser instead.
Just tried to implement something like that for fun. The trick I chose is to
reuse the {} in pattern strings, as (as far as I know) the curly braces are
used only for repetition; hence there should be not conflict. A format would
then look like that:
format = r...{subpatname}...{subpatname}...
For instance:
format = rrecord: {num}{n} -- name:{id}
(Look at the code below.)
Found 2 issues:
-1- Regex patterns haven't any 'string' attribute to recover the string they
have beeen built from. As they aren't python objects, we cannot set anything on
them. So that we need to play with strings directly. (Which means that, once
the sub-pattern tested, we must copy its string.)
-2- There must be a scope to look for subpattern strings per name. the only
practicle solution I could think at is to make super-patterns ,of full
grammars, instances of a class that copes with the details. Subpattern strings
have to be attrs of this object.
___
from re import compile as pattern
class Grammar(object):
identifier = [a-zA-Z_][a-zA-Z_0-9]*
sub_pattern = pattern(\{%s\} % identifier)
def subString(self,result):
name = result.group()[1:-1]
try:
return self.__dict__[name]
except KeyError:
raise AttributeError(Cannot find sub-pattern string
'%s'. % name)
def makePattern(self, format=None):
format = self.format if format is None else format
self.pattern_string =
Grammar.sub_pattern.sub(self.subString,format)
print %s --\n%s % (self.format,self.pattern_string)
self.pattern = pattern(self.pattern_string)
return self.pattern
if __name__ == __main__:
record = Grammar()
record.num = rn[°\.oO]\ ?
record.n= r[0-9]+
record.id = r[a-zA-Z_][a-zA-Z_0-9]*
record.format = rrecord: {num}{n} -- name:{id}
record.makePattern()
text =
record: no 123 -- name:blah
record: n.456 -- name:foo
record: n°789 -- name:foo_bar
result = record.pattern.findall(text)
print result,'\n'
# with format attr and invalid format
bloo_format = rrecord: {num}{n} -- name:{id}{bloo_bar}
record.makePattern(bloo_format)
result = record.pattern.findall(text)
print result
___
Tutor maillist - Tutor@python.org
http://mail.python.org/mailman/listinfo/tutor
[Tutor] Ask a class for it's methods
I have a list containing strings like : func1[] func2[1,2] func3[blah] I want to turn them into method calls (with numeric or string arguments) on a supplied object. I'm trying to figure out the best way to do this. Since these lists could be very big, and the methods could be rather complex (mainly graphics manipulation) I would like to start by getting a list of the object's methods and make sure that all the strings are valid. Is there a way to ask an object for a list of it's methods (with argument requirements if possible)? The next question is once I've validated the list, what's the easiest way to turn the list element into a method call? I'll be parsing the string to separate out the method name and arguments. If I store the method name (say func1) in a variable, say var, could I do object.var() and have if call the func1 method in object? Thanks for your help and I'll surely have more questions as I get along, Thanks Basu -- The ByteBaker : http://www.bytebaker.com ___ Tutor maillist - Tutor@python.org http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Ask a class for it's methods
Shrutarshi Basu technorapt...@gmail.com wrote I have a list containing strings like : func1[] func2[1,2] func3[blah] I want to turn them into method calls (with numeric or string arguments) on a supplied object. The easiest way is to call getattr() which will return a reference to the method if it exists. be rather complex (mainly graphics manipulation) I would like to start by getting a list of the object's methods and make sure that all the strings are valid. Why not just handle it using try/except? Thats exactly what exceptions were designed for. Is there a way to ask an object for a list of it's methods (with argument requirements if possible)? You can try dir() but that won't give you the parameters. But again try/except can catch an invalid call and you can detect whether the number of parameters is wrong in the except clause. method name (say func1) in a variable, say var, could I do object.var() and have if call the func1 method in object? no, but you can do this: class C: ... def f(self, x): print x ... c = C() dir(c) ['__doc__', '__module__', 'f'] m = getattr(c,'f') m(42) 42 try: m(43,'foo') ... except TypeError: ... # parse the error message here ... print 'Method of', c, 'takes ??? arguments' ... Method of __main__.C instance at 0x01BF8350 takes ??? arguments HTH, -- Alan Gauld Author of the Learn to Program web site http://www.freenetpages.co.uk/hp/alan.gauld ___ Tutor maillist - Tutor@python.org http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Ask a class for it's methods
I normally would use exceptions, because I think exceptions are a great idea. But since the functions may be time-consuming graphics functions and the lists could easily be hundreds of such calls, I don't want the user to sit around for something that might fail. Of course, I'm just starting so my assumptions about time might not turn out to be valid, so I could just use exceptions in the end. This is an option I'm exploring. Getattr and dir seem to be the way to go for now, so I'll be trying to apply them over the weekend and see how it turns out. Any more ideas welcome. Thanks, Basu -- The ByteBaker : http://www.bytebaker.com ___ Tutor maillist - Tutor@python.org http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Ask a class for it's methods
On Fri, 12 Dec 2008 20:05:23 -0500, Shrutarshi Basu wrote: I normally would use exceptions, because I think exceptions are a great idea. But since the functions may be time-consuming graphics functions and the lists could easily be hundreds of such calls, I don't want the user to sit around for something that might fail. Of course, I'm just starting so my assumptions about time might not turn out to be valid, so I could just use exceptions in the end. This is an option I'm exploring. The general rule of thumb is usually that exception is slow on the exceptional cases arise, while being faster if the try block doesn't fail. On the other hand, using if-block is usually slow all over, but not as slow as exception's exceptional cases. ___ Tutor maillist - Tutor@python.org http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Ask a class for it's methods
On Fri, Dec 12, 2008 at 06:06:35PM -0500, Shrutarshi Basu wrote: I have a list containing strings like : func1[] func2[1,2] func3[blah] I want to turn them into method calls (with numeric or string arguments) on a supplied object. I'm trying to figure out the best way to do this. Since these lists could be very big, and the methods could be rather complex (mainly graphics manipulation) I would like to start by getting a list of the object's methods and make sure that all the strings are valid. Is there a way to ask an object for a list of it's methods (with argument requirements if possible)? Well, there are ways, but they are not reliable by design. Objects can return dynamically methods. So use something like this: if callable(getattr(obj, func1)): # func1 exists. Guess nowaday with Python3 released, you should not use callable, but instead test on __call__ if hasattr(getattr(obj, func1), __call__): The next question is once I've validated the list, what's the easiest way to turn the list element into a method call? I'll be parsing the string to separate out the method name and arguments. If I store the method name (say func1) in a variable, say var, could I do object.var() and have if call the func1 method in object? getattr(object, var)() # calls the method named in var: class A: def x(self, a, b): return a + b instance = A() name = x args_positional = (1, 2) print getattr(instance, name)(*args_positional) # prints 3 args_by_name = dict(a=1, b=2) print getattr(instance, name)(**args_by_name) # prints 3 too. Andreas ___ Tutor maillist - Tutor@python.org http://mail.python.org/mailman/listinfo/tutor
