Re: binary file compare...
On Apr 17, 4:54 am, Nigel Rantor wig...@wiggly.org wrote: Adam Olsen wrote: On Apr 16, 11:15 am, SpreadTooThin bjobrie...@gmail.com wrote: And yes he is right CRCs hashing all have a probability of saying that the files are identical when in fact they are not. Here's the bottom line. It is either: A) Several hundred years of mathematics and cryptography are wrong. The birthday problem as described is incorrect, so a collision is far more likely than 42 trillion trillion to 1. You are simply the first person to have noticed it. B) Your software was buggy, or possibly the input was maliciously produced. Or, a really tiny chance that your particular files contained a pattern that provoked bad behaviour from MD5. Finding a specific limitation of the algorithm is one thing. Claiming that the math is fundamentally wrong is quite another. You are confusing yourself about probabilities young man. Just becasue something is extremely unlikely does not mean it can't happen on the first attempt. This is true *no matter how big the numbers are*. If you persist in making these ridiculous claims that people *cannot* have found collisions then as I said, that's up to you, but I'm not going to employ you to do anything except make tea. Thanks, Nigel You know this is just insane. I'd be satisfied with a CRC16 or something in the situation i'm in. I have two large files, one local and one remote. Transferring every byte across the internet to be sure that the two files are identical is just not feasible. If two servers one on one side and the other on the other side both calculate the CRCs and transmit the CRCs for comparison I'm happy. -- http://mail.python.org/mailman/listinfo/python-list
Re: binary file compare...
On Apr 16, 3:16 am, Nigel Rantor wig...@wiggly.org wrote: Adam Olsen wrote: On Apr 15, 12:56 pm, Nigel Rantor wig...@wiggly.org wrote: Adam Olsen wrote: The chance of *accidentally* producing a collision, although technically possible, is so extraordinarily rare that it's completely overshadowed by the risk of a hardware or software failure producing an incorrect result. Not when you're using them to compare lots of files. Trust me. Been there, done that, got the t-shirt. Using hash functions to tell whether or not files are identical is an error waiting to happen. But please, do so if it makes you feel happy, you'll just eventually get an incorrect result and not know it. Please tell us what hash you used and provide the two files that collided. MD5 If your hash is 256 bits, then you need around 2**128 files to produce a collision. This is known as a Birthday Attack. I seriously doubt you had that many files, which suggests something else went wrong. Okay, before I tell you about the empirical, real-world evidence I have could you please accept that hashes collide and that no matter how many samples you use the probability of finding two files that do collide is small but not zero. Which is the only thing I've been saying. Yes, it's unlikely. Yes, it's possible. Yes, it happens in practice. If you are of the opinion though that a hash function can be used to tell you whether or not two files are identical then you are wrong. It really is that simple. I'm not sitting here discussing this for my health, I'm just trying to give the OP the benefit of my experience, I have worked with other people who insisted on this route and had to find out the hard way that it was a Bad Idea (tm). They just wouldn't be told. Regards, Nige And yes he is right CRCs hashing all have a probability of saying that the files are identical when in fact they are not. -- http://mail.python.org/mailman/listinfo/python-list
Re: binary file compare...
On Apr 15, 8:04 am, Grant Edwards inva...@invalid wrote: On 2009-04-15, Martin mar...@marcher.name wrote: Hi, On Mon, Apr 13, 2009 at 10:03 PM, Grant Edwards inva...@invalid wrote: On 2009-04-13, SpreadTooThin bjobrie...@gmail.com wrote: I want to compare two binary files and see if they are the same. I see the filecmp.cmp function but I don't get a warm fuzzy feeling that it is doing a byte by byte comparison of two files to see if they are they same. Perhaps I'm being dim, but how else are you going to decide if two files are the same unless you compare the bytes in the files? I'd say checksums, just about every download relies on checksums to verify you do have indeed the same file. That's slower than a byte-by-byte compare. You could hash them and compare the hashes, but that's a lot more work than just comparing the two byte streams. hashing is not exactly much mork in it's simplest form it's 2 lines per file. I meant a lot more CPU time/cycles. -- Grant Edwards grante Yow! Was my SOY LOAF left at out in th'RAIN? It tastes visi.com REAL GOOD!! I'd like to add my 2 cents here.. (Thats 1.8 cents US) All I was trying to get was a clarification of the documentation of the cmp method. It isn't clear. byte by byte comparison is good enough for me as long as there are no cache issues. a check sum is not good because it doesn't guarantee that 1 + 2 + 3 == 3 + 2 + 1 a crc of any sort is more work than a byte by byte comparison and doesn't give you any more information. -- http://mail.python.org/mailman/listinfo/python-list
binary file compare...
I want to compare two binary files and see if they are the same. I see the filecmp.cmp function but I don't get a warm fuzzy feeling that it is doing a byte by byte comparison of two files to see if they are they same. What should I be using if not filecmp.cmp? -- http://mail.python.org/mailman/listinfo/python-list
Re: binary file compare...
On Apr 13, 2:00 pm, Przemyslaw Kaminski cge...@gmail.com wrote: SpreadTooThin wrote: I want to compare two binary files and see if they are the same. I see the filecmp.cmp function but I don't get a warm fuzzy feeling that it is doing a byte by byte comparison of two files to see if they are they same. What should I be using if not filecmp.cmp? Well, here's somethinghttp://www.daniweb.com/forums/thread115959.html but it seems from the post on the bottom that filecmp does comparison of binary files. I just want to be clear, the comparison is not just based on file size and creation date but by a byte by byte comparison of the data in each file. -- http://mail.python.org/mailman/listinfo/python-list
Re: binary file compare...
On Apr 13, 2:03 pm, Grant Edwards inva...@invalid wrote: On 2009-04-13, SpreadTooThin bjobrie...@gmail.com wrote: I want to compare two binary files and see if they are the same. I see the filecmp.cmp function but I don't get a warm fuzzy feeling that it is doing a byte by byte comparison of two files to see if they are they same. Perhaps I'm being dim, but how else are you going to decide if two files are the same unless you compare the bytes in the files? You could hash them and compare the hashes, but that's a lot more work than just comparing the two byte streams. What should I be using if not filecmp.cmp? I don't understand what you've got against comparing the files when you stated that what you wanted to do was compare the files. I think its just the way the documentation was worded http://www.python.org/doc/2.5.2/lib/module-filecmp.html Unless shallow is given and is false, files with identical os.stat() signatures are taken to be equal. Files that were compared using this function will not be compared again unless their os.stat() signature changes. So to do a comparison: filecmp.cmp(filea, fileb, False) ? -- Grant Edwards grante Yow! Look DEEP into the at OPENINGS!! Do you see any visi.com ELVES or EDSELS ... or a HIGHBALL?? ... -- http://mail.python.org/mailman/listinfo/python-list
Re: binary file compare...
On Apr 13, 2:37 pm, Grant Edwards inva...@invalid wrote: On 2009-04-13, Grant Edwards inva...@invalid wrote: On 2009-04-13, SpreadTooThin bjobrie...@gmail.com wrote: I want to compare two binary files and see if they are the same. I see the filecmp.cmp function but I don't get a warm fuzzy feeling that it is doing a byte by byte comparison of two files to see if they are they same. Perhaps I'm being dim, but how else are you going to decide if two files are the same unless you compare the bytes in the files? You could hash them and compare the hashes, but that's a lot more work than just comparing the two byte streams. What should I be using if not filecmp.cmp? I don't understand what you've got against comparing the files when you stated that what you wanted to do was compare the files. Doh! I misread your post and thought were weren't getting a warm fuzzying feeling _because_ it was doing a byte-byte compare. Now I'm a bit confused. Are you under the impression it's _not_ doing a byte-byte compare? Here's the code: def _do_cmp(f1, f2): bufsize = BUFSIZE fp1 = open(f1, 'rb') fp2 = open(f2, 'rb') while True: b1 = fp1.read(bufsize) b2 = fp2.read(bufsize) if b1 != b2: return False if not b1: return True It looks like a byte-by-byte comparison to me. Note that when this function is called the file lengths have already been compared and found to be equal. -- Grant Edwards grante Yow! Alright, you!! at Imitate a WOUNDED SEAL visi.com pleading for a PARKING SPACE!! I am indeed under the impression that it is not always doing a byte by byte comparison... as well the documentation states: Compare the files named f1 and f2, returning True if they seem equal, False otherwise. That word... Sem... makes me wonder. Thanks for the code! :) -- http://mail.python.org/mailman/listinfo/python-list
Convert to C/C++?
I am wondering if someone who knows the implemention of python's time
could help converting this to c/c++
nanoseconds = int(time.time() * 1e9)
# 0x01b21dd213814000 is the number of 100-ns intervals between
the
# UUID epoch 1582-10-15 00:00:00 and the Unix epoch 1970-01-01
00:00:00.
self.timestamp = int(nanoseconds/100) + 0x01b21dd213814000L
self.clock_seq = random.randrange(114L) # instead of stable
storage
self.time_low = self.timestamp 0xL
self.time_mid = (self.timestamp 32L) 0xL
self.time_hi_version = (self.timestamp 48L) 0x0fffL
self.clock_seq_low = self.clock_seq 0xffL
self.clock_seq_hi_variant = (self.clock_seq 8L) 0x3fL
#print 'timestamp ', self.timestamp, self.time_low,
self.time_mid,
self.time_hi_version
#print 'clock_seq ', self.clock_seq, self.clock_seq_low,
self.clock_seq_hi_variant
vs unix gettimeofday
int gettimeofday(struct timeval *tp, struct timezone *tzp);
struct timeval {
long tv_sec; /* seconds since Jan. 1, 1970 */
long tv_usec;/* and microseconds */
};
struct timezone {
int tz_minuteswest; /* of Greenwich */
int tz_dsttime; /* type of dst correction to apply */
};
--
http://mail.python.org/mailman/listinfo/python-list
Re: scared about refrences...
Bruno Desthuilliers wrote: SpreadTooThin a écrit : Bruno Desthuilliers wrote: Nick Vatamaniuc a écrit : (snip) In Python all the primitives are copied and all other entities are references. Plain wrong. There's no primitives (ie : primitive data types) in Python, only objects. And they all get passed the same way. so.. def fn(x): x = x + 1 print x a = 2 fn(a) fn(2) Wouldn't you say that this is being passed by value rather than by refrence? It's not passed by value. when in fn(), the *local* name 'x' is bound to (IOW:references) the exact same object you passed to fn(). Then you rebind this (local) name to *another* object. def fn((x): print id(x) x = x + 1 print id(x) n = 1256 print id(n) fn(n) def fn2(alist): print id(alist) alist.append(42) mylist = [] print id(mylist) fn2(mylist) print mylist print id(mylist) There's nothing like pass by value or pass by reference in Python (and you'll notice I didn't claimed anything about this - just that the 'argument passing scheme' was the same for all objects). What we call variables in Python are name=object bindings. When passing a variable to a function, the reference to the objet is bound to the name of the argument in the function's namespace. So the *name* is local to the function (hence rebinding the name to another objet doesn't impact the name=object binding in the caller's namespace), but this name really refers to the same object (Python doesn't copy anything unless explicitely told to do so). HTH I realize I may be beating a dead horse here... but... a = 2 fn(a) 3 print a 2 So in some cases the it is safe to assume that your variables to function will not change in other cases it is not.. but they are all the same... -- http://mail.python.org/mailman/listinfo/python-list
Re: scared about refrences...
Bruno Desthuilliers wrote: Nick Vatamaniuc a écrit : (snip) In Python all the primitives are copied and all other entities are references. Plain wrong. There's no primitives (ie : primitive data types) in Python, only objects. And they all get passed the same way. so.. def fn(x): x = x + 1 print x a = 2 fn(a) fn(2) Wouldn't you say that this is being passed by value rather than by refrence? -- http://mail.python.org/mailman/listinfo/python-list
Re: scared about refrences...
J. Clifford Dyer wrote:
SpreadTooThin wrote:
J. Clifford Dyer wrote:
SpreadTooThin wrote:
Steven D'Aprano wrote:
On Mon, 30 Oct 2006 13:10:47 -0800, SpreadTooThin wrote:
I seems that some of the objects in the list don't get along well with
deep copy..
See my second example post that used deepcopy... When run blows up...
When it blows up, is there a lot of shrapnel, or just smoke and fire?
Is the shrapnel mostly metal, or is it plastic and glass?
In short, if we don't know what's happening, we can't help.
* Did the program spit out a bunch of text you didn't understand?
If so, show us the text. That text may be incomprehensible at first,
but it contains crucial clues.
* Did it close your python window without a word?
Tell us.
* Did your computer freeze up?
Tell us.
If you don't tell us what went wrong *exactly*, you won't get a
satisfactory answer.
I would assume that looking at the code you should be able to tell..
Silly me.. Here.. is the log.. If I were helping.. I would have cut
and pasted the code myself and ran it.. instead of trying to interpret
this...
I know it seems unnecessary to post the traceback when I could get the
same thing by running your code on my machine, but it's actually useful,
for a couple reasons: First, when I run the code, I might not get an
error, or if I do, it might not be the same error you were getting, and
then we'd be on a wild goose chase. This could be because your python
installation is goofy, or because you copied in your code incorrectly.
Shit happens, and I'd rather not even start down one of those blind
alleys. Third, it provides a useful frame for how to look at your
code. While a traceback might look like a big mess at first, it's
actually pretty easy to skim through once you get used to it, and it
tells me where to focus my attention in your code.
array('H', [1, 2, 3]) ['a', 'b', 'c']
Traceback (most recent call last):
File
/Volumes/Data/Users/bjobrien/Applications/Komodo.app/Contents/SharedSupport/dbgp/pythonlib/dbgp/client.py,
line 1806, in runMain
self.dbg.runfile(debug_args[0], debug_args)
File
/Volumes/Data/Users/bjobrien/Applications/Komodo.app/Contents/SharedSupport/dbgp/pythonlib/dbgp/client.py,
line 1529, in runfile
h_execfile(file, args, module=main, tracer=self)
File
/Volumes/Data/Users/bjobrien/Applications/Komodo.app/Contents/SharedSupport/dbgp/pythonlib/dbgp/client.py,
line 590, in __init__
execfile(file, globals, locals)
File /Volumes/Data/Users/bjobrien/Desktop/pythonDICOM/Text-1.py,
line 20, in __main__
test(t)
File /Volumes/Data/Users/bjobrien/Desktop/pythonDICOM/Text-1.py,
line 16, in test
t = copy.deepcopy(x)
File
/Library/Frameworks/Python.framework/Versions/2.4/lib/python2.4/copy.py,
line 174, in deepcopy
y = copier(x, memo)
File
/Library/Frameworks/Python.framework/Versions/2.4/lib/python2.4/copy.py,
line 305, in _deepcopy_inst
state = deepcopy(state, memo)
File
/Library/Frameworks/Python.framework/Versions/2.4/lib/python2.4/copy.py,
line 174, in deepcopy
y = copier(x, memo)
File
/Library/Frameworks/Python.framework/Versions/2.4/lib/python2.4/copy.py,
line 268, in _deepcopy_dict
y[deepcopy(key, memo)] = deepcopy(value, memo)
File
/Library/Frameworks/Python.framework/Versions/2.4/lib/python2.4/copy.py,
line 185, in deepcopy
y = copier(x, memo)
TypeError: __deepcopy__() takes no arguments (1 given)
Cheers,
Cliff
Thanks, that's very helpful. Playing with your code a bit, I narrowed
the problem down to the array.array() structure. Looking at
help(array), there's a method defined called __deepcopy__, which, it
seems, takes no arguments, while deepcopy is passing it one argument.
Looks like a bug in the array module to me. Maybe others with more
experience using array will have some deeper insight.
I don't understand why python would insist that everything must be a
refrence...
It is of course helpful sometime but other times its not... and now
I'm sorta out
of luck...
I don't know how to make this structure immutable... Pickle it? Seems
very
inefficient to me...
Every time I pass a variable now I will worry that it will be changed
by the function...
I haven't worried about things like this since the very early days of
BASIC
I don't know.. maybe I have more to learn.
Cheers,
Cliff
--
http://mail.python.org/mailman/listinfo/python-list
scared about refrences...
I'm really worried that python may is doing some things I wasn't expecting... but lets see... if I pass a list to a function def fn(myList): and in that function I modify an element in the list, then does the callers list get modied as well. def fn(list): list[1] = 0 myList = [1, 2, 3] print myList fn(myList) print myList [1,2,3] [1,0,3] How can I avoid this? In this case this is a really simplified example but the effects are the same... How do I specify or create deep copies of objects that may contain other objects that may contain other object that may contain other objects -- http://mail.python.org/mailman/listinfo/python-list
Re: scared about refrences...
Marc 'BlackJack' Rintsch wrote: In [EMAIL PROTECTED], SpreadTooThin wrote: I'm really worried that python may is doing some things I wasn't expecting... but lets see... Expect that Python never copies something if don't ask explicitly for a copy. if I pass a list to a function def fn(myList): and in that function I modify an element in the list, then does the callers list get modied as well. def fn(list): list[1] = 0 myList = [1, 2, 3] print myList fn(myList) print myList [1,2,3] [1,0,3] How can I avoid this? In this case this is a really simplified example but the effects are the same... In this case: def fn(lst): lst = list(lst) lst[1] = 0 How do I specify or create deep copies of objects that may contain other objects that may contain other object that may contain other objects See the `copy` module especially `copy.deepcopy()`. I'm aware of __deepcopy__ but does that mean that every object in the object needs to have its own deep copy? And how do I ensure that? Ciao, Marc 'BlackJack' Rintsch -- http://mail.python.org/mailman/listinfo/python-list
Re: scared about refrences...
Marc 'BlackJack' Rintsch wrote:
In [EMAIL PROTECTED], SpreadTooThin
wrote:
I'm really worried that python may is doing some things I wasn't
expecting... but lets see...
Expect that Python never copies something if don't ask explicitly for a
copy.
if I pass a list to a function def fn(myList):
and in that function I modify an element in the list, then does the
callers list get modied as well.
def fn(list):
list[1] = 0
myList = [1, 2, 3]
print myList
fn(myList)
print myList
[1,2,3]
[1,0,3]
How can I avoid this? In this case this is a really simplified example
but the effects are the same...
In this case:
def fn(lst):
lst = list(lst)
lst[1] = 0
How do I specify or create deep copies of objects that may contain
other objects that may contain other object that may contain other
objects
See the `copy` module especially `copy.deepcopy()`.
This appears to be the right thing to do to me.. (but what do I know?)
I tried this which more closely resembles my project but this doesn't
work:
import array
import copy
class test:
def __init__(self):
self.a = array.array('H', [1, 2, 3])
self.b = ['a', 'b', 'c']
def dump(self):
print self.a, self.b
t = test()
t.dump()
def testit(x):
t = copy.deepcopy(x)
t.a[1] = 0
t.b[1] = 0
testit(t)
t.dump()
--
http://mail.python.org/mailman/listinfo/python-list
Re: scared about refrences...
Steven D'Aprano wrote: On Mon, 30 Oct 2006 13:10:47 -0800, SpreadTooThin wrote: How do I specify or create deep copies of objects that may contain other objects that may contain other object that may contain other objects See the `copy` module especially `copy.deepcopy()`. This appears to be the right thing to do to me.. (but what do I know?) Yes, copy.deepcopy() is the thing you want. But remember Fredrik's advice that well-designed Python code should not need to copy data structures often. I don't think I've ever needed to use deepcopy, and rarely copy.copy(). In general, functions should not modify their caller's data. So this is bad practice: def print_list(alist): Print a sorted list alist.sort() # modifies the caller's data -- bad! for index, value in enumerate: print Value %s at index %d % (index, value) This is better: def print_list(alist): Print a sorted list alist = alist[:] # makes a local shallow copy of the list alist.sort() # safe to modify now for index, value in enumerate: print Value %s at index %d % (index, value) But notice that you only need a shallow copy, not a deep copy, because you aren't modifying the objects within the list, only the list itself. I tried this which more closely resembles my project but this doesn't work: Unfortunately my crystal ball is back at the shop being repaired, so you'll have to explain what doesn't work means in this case. Does it raise an exception? If so, please post the exception. Does it do something different from what you expected? Then what did you expect, and what did it do? I seems that some of the objects in the list don't get along well with deep copy.. See my second example post that used deepcopy... When run blows up... -- Steven. -- http://mail.python.org/mailman/listinfo/python-list
Re: scared about refrences...
J. Clifford Dyer wrote:
SpreadTooThin wrote:
Steven D'Aprano wrote:
On Mon, 30 Oct 2006 13:10:47 -0800, SpreadTooThin wrote:
How do I specify or create deep copies of objects that may contain
other objects that may contain other object that may contain other
objects
See the `copy` module especially `copy.deepcopy()`.
This appears to be the right thing to do to me.. (but what do I know?)
Yes, copy.deepcopy() is the thing you want.
But remember Fredrik's advice that well-designed Python code should not
need to copy data structures often. I don't think I've ever needed to use
deepcopy, and rarely copy.copy().
In general, functions should not modify their caller's data. So this is
bad practice:
def print_list(alist):
Print a sorted list
alist.sort() # modifies the caller's data -- bad!
for index, value in enumerate:
print Value %s at index %d % (index, value)
This is better:
def print_list(alist):
Print a sorted list
alist = alist[:] # makes a local shallow copy of the list
alist.sort() # safe to modify now
for index, value in enumerate:
print Value %s at index %d % (index, value)
But notice that you only need a shallow copy, not a deep copy, because you
aren't modifying the objects within the list, only the list itself.
I tried this which more closely resembles my project but this doesn't
work:
Unfortunately my crystal ball is back at the shop being repaired, so
you'll have to explain what doesn't work means in this case. Does it
raise an exception? If so, please post the exception. Does it do something
different from what you expected? Then what did you expect, and what did
it do?
I seems that some of the objects in the list don't get along well with
deep copy..
See my second example post that used deepcopy... When run blows up...
When it blows up, is there a lot of shrapnel, or just smoke and fire?
Is the shrapnel mostly metal, or is it plastic and glass?
In short, if we don't know what's happening, we can't help.
* Did the program spit out a bunch of text you didn't understand?
If so, show us the text. That text may be incomprehensible at first,
but it contains crucial clues.
* Did it close your python window without a word?
Tell us.
* Did your computer freeze up?
Tell us.
If you don't tell us what went wrong *exactly*, you won't get a
satisfactory answer.
I would assume that looking at the code you should be able to tell..
Silly me.. Here.. is the log.. If I were helping.. I would have cut
and pasted the code myself and ran it.. instead of trying to interpret
this...
array('H', [1, 2, 3]) ['a', 'b', 'c']
Traceback (most recent call last):
File
/Volumes/Data/Users/bjobrien/Applications/Komodo.app/Contents/SharedSupport/dbgp/pythonlib/dbgp/client.py,
line 1806, in runMain
self.dbg.runfile(debug_args[0], debug_args)
File
/Volumes/Data/Users/bjobrien/Applications/Komodo.app/Contents/SharedSupport/dbgp/pythonlib/dbgp/client.py,
line 1529, in runfile
h_execfile(file, args, module=main, tracer=self)
File
/Volumes/Data/Users/bjobrien/Applications/Komodo.app/Contents/SharedSupport/dbgp/pythonlib/dbgp/client.py,
line 590, in __init__
execfile(file, globals, locals)
File /Volumes/Data/Users/bjobrien/Desktop/pythonDICOM/Text-1.py,
line 20, in __main__
test(t)
File /Volumes/Data/Users/bjobrien/Desktop/pythonDICOM/Text-1.py,
line 16, in test
t = copy.deepcopy(x)
File
/Library/Frameworks/Python.framework/Versions/2.4/lib/python2.4/copy.py,
line 174, in deepcopy
y = copier(x, memo)
File
/Library/Frameworks/Python.framework/Versions/2.4/lib/python2.4/copy.py,
line 305, in _deepcopy_inst
state = deepcopy(state, memo)
File
/Library/Frameworks/Python.framework/Versions/2.4/lib/python2.4/copy.py,
line 174, in deepcopy
y = copier(x, memo)
File
/Library/Frameworks/Python.framework/Versions/2.4/lib/python2.4/copy.py,
line 268, in _deepcopy_dict
y[deepcopy(key, memo)] = deepcopy(value, memo)
File
/Library/Frameworks/Python.framework/Versions/2.4/lib/python2.4/copy.py,
line 185, in deepcopy
y = copier(x, memo)
TypeError: __deepcopy__() takes no arguments (1 given)
Cheers,
Cliff
--
http://mail.python.org/mailman/listinfo/python-list
Re: mean ans std dev of an array?
Paul Rubin wrote: SpreadTooThin [EMAIL PROTECTED] writes: print a.mean() print a.std_dev() Is there a way to calculate the mean and standard deviation on array data? Well, you could use numpy or whatever. If you want to calculate directly, you could do something like (untested): n = len(a) mean = sum(a) / n sd = sqrt(sum((x-mean)**2 for x in a) / n) This last line looks like it will be very slow... Escpecially if the array is large... So speed is the real issue here... If there is a faster way... like transferring the array to a different container class... but what? -- http://mail.python.org/mailman/listinfo/python-list
mean ans std dev of an array?
import array
a = array.array('f', [1,2,3])
print a.mean()
print a.std_dev()
Is there a way to calculate the mean and standard deviation on array
data?
Do I need to import it into a Numeric Array to do this?
--
http://mail.python.org/mailman/listinfo/python-list
Komodo
Why is it that (On MAC OS X) in Komodo 3.5 Professional, if I try to find something in my script, I am unable to change the text it is searching for? -- http://mail.python.org/mailman/listinfo/python-list
making a valid file name...
Hi I'm writing a python script that creates directories from user input. Sometimes the user inputs characters that aren't valid characters for a file or directory name. Here are the characters that I consider to be valid characters... valid = ':./,^0123456789abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ ' if I have a string called fname I want to go through each character in the filename and if it is not a valid character, then I want to replace it with a space. This is what I have: def fixfilename(fname): valid = ':.\,^0123456789abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ ' for i in range(len(fname)): if valid.find(fname[i]) 0: fname[i] = ' ' return fname Anyone think of a simpler solution? -- http://mail.python.org/mailman/listinfo/python-list
loop iterators?
Hi I need to get a for loop to give me an element from a list and an index number... for example: i = 0 l = ['a','b','c'] for e in l: print l, i i = i + 1 Is there a way to get rid of the i processing in this script? Sorry I'm still trying to learn python tricks here... -- http://mail.python.org/mailman/listinfo/python-list
Need a strange sort method...
I have a list and I need to do a custom sort on it... for example: a = [1,2,3,4,5,6,7,8,9,10] #Although not necessarily in order def cmp(i,j): #to be defined in this thread. a.sort(cmp) print a [1,4,7,10, 2,5,8, 3,6,9] So withouth making this into an IQ test. Its more like 1 4 7 10 2 5 8 3 6 9 Read that top to bottom from column 1 to column 4. When you get to the bottom of a column move to the next column. Get it? -- http://mail.python.org/mailman/listinfo/python-list
Re: Need a strange sort method...
Fredrik Lundh wrote: SpreadTooThin wrote: I have a list and I need to do a custom sort on it... Its more like 1 4 7 10 2 5 8 3 6 9 that's trivial to do with slicing, of course. what makes you think you need to do this by calling the sort method ? /F You are of course correct.. There might be a way to do this with slicing and i % 3 -- http://mail.python.org/mailman/listinfo/python-list
Re: Need a strange sort method...
Simon Brunning wrote: On 10/16/06, Simon Brunning [EMAIL PROTECTED] wrote: a = [1,2,3,4,5,6,7,8,9,10] a.sort(key=lambda item: (((item-1) %3), item)) a [1, 4, 7, 10, 2, 5, 8, 3, 6, 9] Re-reading the OP's post, perhaps sorting isn't what's required: a[::3] + a[1::3] + a[2::3] [1, 4, 7, 10, 2, 5, 8, 3, 6, 9] -- Cheers, Simon B [EMAIL PROTECTED] http://www.brunningonline.net/simon/blog/ Ok so this is what I got.. but there is an odd side effect: def reslice(series): series.sort() i = 1 newseries = series[::3] while(1): c = series[i::3] if len(c) = 3: newseries = newseries + c else: break i = i + 1 return newseries a = [2,1,4,3,6,5,8,7,10,9] b = reslice(a) print b [1, 4, 7, 10, 2, 5, 8, 3, 6, 9, 4, 7, 10] I have one extra 10 that I shouldn't... -- http://mail.python.org/mailman/listinfo/python-list
Re: Need a strange sort method...
SpreadTooThin wrote: Simon Brunning wrote: On 10/16/06, Simon Brunning [EMAIL PROTECTED] wrote: a = [1,2,3,4,5,6,7,8,9,10] a.sort(key=lambda item: (((item-1) %3), item)) a [1, 4, 7, 10, 2, 5, 8, 3, 6, 9] Re-reading the OP's post, perhaps sorting isn't what's required: a[::3] + a[1::3] + a[2::3] [1, 4, 7, 10, 2, 5, 8, 3, 6, 9] -- Cheers, Simon B [EMAIL PROTECTED] http://www.brunningonline.net/simon/blog/ Ok so this is what I got.. but there is an odd side effect: def reslice(series): series.sort() i = 1 newseries = series[::3] while(1): c = series[i::3] if len(c) = 3: newseries = newseries + c else: break i = i + 1 return newseries a = [2,1,4,3,6,5,8,7,10,9] b = reslice(a) print b [1, 4, 7, 10, 2, 5, 8, 3, 6, 9, 4, 7, 10] I have one extra 10 that I shouldn't... I think my loop termination is incorrect... maybe I should just stop when my new series size is the same size as the original? -- http://mail.python.org/mailman/listinfo/python-list
Re: Need a strange sort method...
Tim Chase wrote:
for example:
a = [1,2,3,4,5,6,7,8,9,10] #Although not necessarily in order
def cmp(i,j): #to be defined in this thread.
Well, if you're willing to give up doing it in a cmp() method,
you can do it as such:
a.sort()
chunk_size = 3
[a[i::chunk_size] for i in range(chunk_size)]
[[1, 4, 7, 10], [2, 5, 8], [3, 6, 9]]
If you need it in a flat list, rather than as a list of
chunk_size lists (which are handy for iterating over in many
cases), there are ways of obtaining it, such as the hackish
sum([a[i::chunk_size] for i in range(chunk_size)], [])
[1, 4, 7, 10, 2, 5, 8, 3, 6, 9]
There are likely good recipes for flattening a list. I just
happen not to have any at my fingertips.
I'm not sure it's possible to do in a cmp() method, given that it
requires apriori knowledge of the dataset (are the numbers
contiguous?). Unless, of course, you have such a list...
However, as a benefit, this method should work no matter what the
list contains, as long as they're comparable to each other for an
initial sorting:
a = [chr(ord('a') + i) for i in range(10)]
# a.sort() if it were needed
a
['a', 'b', 'c', 'd', 'e', 'f', 'g', 'h', 'i', 'j']
sum([a[i::chunk_size] for i in range(chunk_size)], [])
['a', 'd', 'g', 'j', 'b', 'e', 'h', 'c', 'f', 'i']
-tkc
Yes this is excellent...
Doh... How easy was that! :)
[a[i::chunk_size] for i in range(chunk_size)]
--
http://mail.python.org/mailman/listinfo/python-list
Re: operator overloading + - / * = etc...
Daniel Nogradi wrote: Can these operators be overloaded? If so. How? http://www.python.org/doc/ref/numeric-types.html HTH, Daniel Thanks everyone. -- http://mail.python.org/mailman/listinfo/python-list
class print?
Hi... Many python object can be printed simpy by saying: print obj what method(s) need to be implemented in my own classes such that print dumps the object? -- http://mail.python.org/mailman/listinfo/python-list
operator overloading + - / * = etc...
Can these operators be overloaded? If so. How? -- http://mail.python.org/mailman/listinfo/python-list
Re: socket client server... simple example... not working...
Jean-Paul Calderone wrote:
On 4 Oct 2006 19:31:38 -0700, SpreadTooThin [EMAIL PROTECTED] wrote:
client:
import socket
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.connect((192.168.1.101, 8080))
print 'Connected'
s.send('ABCD')
Here you didn't check the return value of send to determine if all of the
string was copied to the kernel buffer to be sent, so you may have only
succeeded in sending part of 'ABCD'.
buffer = s.recv(4)
in the above call, 4 is the maximum number of bytes recv will return. It
looks as though you are expecting it to return exactly 4 bytes, but in order
to get that, you will need to check the length of the return value and call
recv again with a lower limit until the combination of the return values of
each call gives a total length of 4.
print buffer
s.send('exit')
Again, you didn't check the return value of send.
server:
serversocket = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
serversocket.bind((192.168.1.101, 8080))
serversocket.listen(5)
print 'Listen'
(clientsocket, address) = serversocket.accept()
print 'Accepted'
flag = True
while flag:
chunk = serversocket.recv(4)
You're calling recv on serversocket instead of on clientsocket. You're also
relying on recv to return exactly 4 bytes, which it may not do.
if chunk == '':
raise RuntimeError, socket connection broken
elif chunk == 'exit':
flag = False
else:
serversocket.send(chunk)
Another missing check of the return value of send.
print 'Done'
Server says!
Listen
Accepted
Traceback (most recent call last):
File server.py, line 11, in ?
chunk = serversocket.recv(4)
socket.error: (57, 'Socket is not connected')
Client says:
Connected
What have I done wrong now!
I recommend switching to Twisted. The Twisted equivalent (I guess - the
protocol defined above is strange and complex (probably unintentionally, due
to the things you left out, like any form of delimiter) and I doubt I really
understand the end goal you are working towards), minus bugs (untested):
# client.py
from twisted.internet import reactor, protocol
class Client(protocol.Protocol):
buf = ''
def connectionMade(self):
self.transport.write('ABCD')
def dataReceived(self, data):
self.buf += data
if len(self.buf) = 4:
reactor.stop()
protocol.ClientCreator(reactor, Client).connectTCP('192.168.1.101', 8080)
reactor.run()
# server.py
from twisted.internet import reactor, protocol
class Server(protocol.Protocol):
buf = ''
def dataReceived(self, bytes):
self.buf += bytes
exit = self.buf.find('exit')
if exit != -1:
self.transport.write(self.buf[:exit])
self.buf = self.buf[exit + 4:]
reactor.stop()
else:
self.transport.write(self.buf)
self.buf = ''
f = protocol.ServerFactory()
f.protocol = Server
reactor.listenTCP('192.168.1.101', 8080, f)
reactor.run()
Hope this helps,
Jean-Paul
Jean-Paul many thanks for this and your effort.
but why is it every time I try to do something with 'stock' python I
need another package?
By the time I've finished my project there are like 5 3rd party add-ons
to be installed.
I know I'm a python newbie... but I'm far from a developer newbie and
that can be a recipe for
disaster. The stock socket should work and I think I've missed an
obvious bug in the code other
than checking the return status.
--
http://mail.python.org/mailman/listinfo/python-list
Re: socket client server... simple example... not working...
Jean-Paul Calderone wrote: On 5 Oct 2006 07:01:50 -0700, SpreadTooThin [EMAIL PROTECTED] wrote: [snip] Jean-Paul many thanks for this and your effort. but why is it every time I try to do something with 'stock' python I need another package? Maybe you are trying to do things that are too complex :) No quite the contrary.. Which is why I want to keep it simple... By the time I've finished my project there are like 5 3rd party add-ons to be installed. I don't generally find this to be problematic. I have because it usually means makeing on many platforms... Most of the time this is the nightmare. I know I'm a python newbie... but I'm far from a developer newbie and that can be a recipe for disaster. Not every library can be part of the standard library, neither can the standard library satisfy every possible use-case. Relying on 3rd party modules isn't a bad thing. No but the less number of lines of code I have to support the better. The stock socket should work and I think I've missed an obvious bug in the code other than checking the return status. It was indeed as you said I was trying to read/write on the server socket not the client socket. (of the server module) Well, I did mention one bug other than failure to check return values. Maybe you missed it, since it was in the middle. Go back and re-read my response. Thanks again. B. Jean-Paul -- http://mail.python.org/mailman/listinfo/python-list
Re: Need help with syntax on inheritance.
Peter Otten wrote: SpreadTooThin wrote: If you are deriving a new class from another class, that you must (I assume) know the initializer of the other class. So in myClass import array class myClass(arrary.array): def __init__(self, now here I need to put array's constructor parameters..., then mine): array.array.__init__(self, typecode[, initializer]) self.mine = mine So I'm confused... array has a typecode parameter and an optional initiializer... So could you help me with the class construction here please? Normally you would do # won't work class Array(array.array): def __init__(self, typecode, initalizer=(), mine=None): array.array.__init__(self, typecode, initializer) self.mine = mine However, array.array is a bit harder to subclass: # should work class Array(array.array): def __new__(cls, typecode, initializer=(), mine=None): return array.array.__new__(cls, typecode, initializer) def __init__(self, typecode, initializer=(), mine=None): array.array.__init__(self, typecode, initializer) self.mine = mine See if you can get away by making the array an attribute of your class instead. Thanks. the =() syntax indicates what? Just slightly off topic here but if Array had a bunch of initializers of its own, must all the 'optional' parameters be on the right.. ie the last parameters? Peter -- http://mail.python.org/mailman/listinfo/python-list
socket client server... simple example... not working...
client:
import socket
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.connect((192.168.1.101, 8080))
print 'Connected'
s.send('ABCD')
buffer = s.recv(4)
print buffer
s.send('exit')
server:
serversocket = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
serversocket.bind((192.168.1.101, 8080))
serversocket.listen(5)
print 'Listen'
(clientsocket, address) = serversocket.accept()
print 'Accepted'
flag = True
while flag:
chunk = serversocket.recv(4)
if chunk == '':
raise RuntimeError, socket connection broken
elif chunk == 'exit':
flag = False
else:
serversocket.send(chunk)
print 'Done'
Server says!
Listen
Accepted
Traceback (most recent call last):
File server.py, line 11, in ?
chunk = serversocket.recv(4)
socket.error: (57, 'Socket is not connected')
Client says:
Connected
What have I done wrong now!
--
http://mail.python.org/mailman/listinfo/python-list
array tofile fromfile tosocket? fromsocket?
Hi I am using arrays in an application. They have been working very well for me as I have been using them for reading and writing data to and from files whose endianisms can be big or little endian. However now I need handle data that can be transferred over a socket. I would rather not have to go in and change every occurance of tofile and fromfile to a function but realize that is what I may have to do. I was under the impression that a file and socket were interchangable (like a file descriptor). Does anyone have an idea that might help me minimize the impact on my code? -- http://mail.python.org/mailman/listinfo/python-list
Re: array tofile fromfile tosocket? fromsocket?
Thomas Jollans wrote: On Tue, 03 Oct 2006 10:29:00 -0700, SpreadTooThin [EMAIL PROTECTED] let this slip: I was under the impression that a file and socket were interchangable (like a file descriptor). Does anyone have an idea that might help me minimize the impact on my code? are you looking for sockets' makefile method ? Yes.. but also I'm thinking about a new python class based on array... like (psuedo code) class myArray(array): def todevice(self, device): if device is a file: self.tofile else writesocket(data) def fromdevice(self, device): Is this advisable? -- Thomas Jollans alias free-zombie -- http://mail.python.org/mailman/listinfo/python-list
Need help with syntax on inheritance.
If you are deriving a new class from another class, that you must (I assume) know the initializer of the other class. So in myClass import array class myClass(arrary.array): def __init__(self, now here I need to put array's constructor parameters..., then mine): array.array.__init__(self, typecode[, initializer]) self.mine = mine So I'm confused... array has a typecode parameter and an optional initiializer... So could you help me with the class construction here please? -- http://mail.python.org/mailman/listinfo/python-list
Need help with an array problem.
Basically I think the problem is in converting from a 32 bit integer to
a 16 bit integer.
I have two arrays:
import array
a = array.array('L', [65537])
b = array.array('H', [0])
b[0] = a[0]
Which gives an overflow message
So can't I truncate the long by discaring the upper bits ..
Like b[0] = 0x a[0]
How does one normally cast an object from long to short?
--
http://mail.python.org/mailman/listinfo/python-list
Re: Need help with an array problem.
Robert Kern wrote: [EMAIL PROTECTED] wrote: To your question on casting long to short. This is how: a=1234L # long b=int(a) # int (short) No, a Python int is a C long. A Python long is an arbitrary-precision number and does not correspond to any C type. -- So there is no short(number) casting? Robert Kern I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth. -- Umberto Eco -- http://mail.python.org/mailman/listinfo/python-list
how do you know if open failed?
f = open('myfile.bin', 'rb')
How do I know if there was an error opening my file?
--
http://mail.python.org/mailman/listinfo/python-list
get process id...
How does one get the process id? Is there a method for windows and unix (mac os x etc...) -- http://mail.python.org/mailman/listinfo/python-list
How do I define a global constant...
How do I define a constant that I can use in my script... For example lets say I have a file called constants.py and in there I have PI = 3.14 in my test script I do: from constants import * How do I access PI later on? Pardon my newbie questions... -- http://mail.python.org/mailman/listinfo/python-list
how do you convert and array of doubles into floats?
I have some code...
import array
a = array.array('d')
f = open('file.raw')
a.fromfile(f, 10)
now I need to convert them into floats (32 bit...) what do i do?
--
http://mail.python.org/mailman/listinfo/python-list
Re: how do you convert and array of doubles into floats?
Tim Peters wrote:
[Marc 'BlackJack' Rintsch]
What about:
b = array.array('f', a)
[Diez B. Roggisch]
AFAIK d and f are synonym for arrays, as python doesn't distinguish
between these two on a type-level. And double it is in the end.
While Python has no type of its own corresponding to the native C
`float`, the `array` and `struct` modules do understand the native C
`float` . A Python float (same as a C `double`) gets converted to a C
`float` when stored into one of those, and a C `float` is converted to
a Python float (C `double`) when a value is extracted.
from array import array
x = 1.01
x
1.01
array('d', [x])[0] # full precision is preserved
1.01
array('d', [x])[0] == x
True
array('f', [x])[0] # trailing bits are lost
1.0
array('f', [x])[0] == x
False
The point being that when I say
b.tofile(f) I expect the write to write 32 bit floats.
--
http://mail.python.org/mailman/listinfo/python-list
