Re: Scanning a file
David Rasmussen wrote: snip If you must know, the above one-liner actually counts the number of frames in an MPEG2 file. I want to know this number for a number of files for various reasons. I don't want it to take forever. snip Don't you risk getting more frames than the file actually have? What if the encoded data happens to have the magic byte values for something else? -- Lasse Vågsæther Karlsen http://usinglvkblog.blogspot.com/ mailto:[EMAIL PROTECTED] PGP KeyID: 0x2A42A1C2 -- http://mail.python.org/mailman/listinfo/python-list
Re: dreaming in Python
The Eternal Squire wrote: All, I have to tell all of you this, at least for some laughs. Honestly, I had the silliest dream involving the language last night. I dreamt that it was a decade into the future and that Grand Central Station in NYC was installing a cement and steel computer core for directing its trains and station elevators... and for some reason I was supervising a technician on the project. I asked the technician, what version of Python was being used, and he said 2.1... I said better install 2.4! So he loaded a chip into the 15 ton cylindrical core, then the core was lowered by winch into a manhole cover. The lights in the station lit, and I shouted: One Python to Rule Them All And In The Darkness Bind Them! Then I took an elevator downstairs to log into the terminal. Who did I meet on a neighboring terminal other than the BDFL? He was doing something esoteric and strange on the terminal but I was just having trouble logging in. End of dream. Anyone ever have a wierd engineering dream sometime in thier career? The Eternal Squire Just sit still while the men in white come to pick you up :) -- Lasse Vågsæther Karlsen http://usinglvkblog.blogspot.com/ mailto:[EMAIL PROTECTED] PGP KeyID: 0x2A42A1C2 -- http://mail.python.org/mailman/listinfo/python-list
Re: Need Python Pro for Help!! Plzz
[EMAIL PROTECTED] wrote: Need python Pro at [EMAIL PROTECTED] , if u wanna help, plz reply to that address. We are python beginners. need a real good Python Programmer for Help. TIA!! If you need help, post questions to the newsgroups. That way you might get help from several people instead of just one. -- Lasse Vågsæther Karlsen http://usinglvkblog.blogspot.com/ mailto:[EMAIL PROTECTED] PGP KeyID: 0x2A42A1C2 -- http://mail.python.org/mailman/listinfo/python-list
Re: how to associate files with application
Dennis Lee Bieber wrote: On Fri, 28 Oct 2005 09:48:27 -0400, Colin J. Williams [EMAIL PROTECTED] declaimed the following in comp.lang.python: I'm no Windows expert but I think that, using Windows Explorer, one can, with a right mouse click, select Open With. snip There are several ways to do this using Windows Explorer. I was under the assumption the OP wanted to know how he could automate it since that is what you typically want to do with applications you write. -- Lasse Vågsæther Karlsen http://usinglvkblog.blogspot.com/ mailto:[EMAIL PROTECTED] PGP KeyID: 0x2A42A1C2 -- http://mail.python.org/mailman/listinfo/python-list
Re: Microsoft Hatred FAQ
David Schwartz wrote: Lasse Vågsæther Karlsen wrote: David Schwartz wrote: Burger King won't let you sell Whoppers or buy their burger patties wholesale no matter what you want to call your store unless you take the whole franchise deal. It's an all-or-nothing package. With very few limits, companies do get to choose how their products are branded, marketed, and sold. Yes, and that's not what Microsoft has ever done. There have always been lots of shops selling Microsoft merchandise without being a Microsoft franchise in the sense Burger King shops are. Right, Microsoft imposed a lesser restriction. They allowed you to sell competing products, but charged you a fee. That's why I still say your comparison is a bad one. It shows that Microsoft's purportedly draconian restrictions are much less than restrictions that people don't even bat an eye at. DS Ok, let me just make my opinion very clear on this and then I'll just leave this thread altogether. I think you are comparing apples and oranges so whatever conclusion you manage to draw from that is in my eyes invalid. It doesn't matter, in my opinion, if you managed to conclude that Microsoft was the saints themselves because, in my opinion, your reasoning is not valid. I'm not saying one way or the other, I'm just picking at your reasoning. To me it sounds like concluding that the prices of RAM will drop because the swallows are flying high this fall. But enough, I'll just leave it. -- Lasse Vågsæther Karlsen http://usinglvkblog.blogspot.com/ mailto:[EMAIL PROTECTED] PGP KeyID: 0x2A42A1C2 -- http://mail.python.org/mailman/listinfo/python-list
Re: Suggestion for (re)try statement
Sori Schwimmer wrote: Hi, I think that would be useful to have an improved version of the try statement, as follows: try(retrys=0,timeout=0): snip sleep(timeout) snip At the very least, timeout is the wrong wording, delay would be more appropriate. A timeout is usually associated with starting a task and waiting for it to complete, and continuing if it fails to complete in a given timeframe, typically also aborting the task at the same time (ie. executing a database query, connecting to a server, waiting for an event/lock, etc.). -- Lasse Vågsæther Karlsen http://usinglvkblog.blogspot.com/ mailto:[EMAIL PROTECTED] PGP KeyID: 0x2A42A1C2 -- http://mail.python.org/mailman/listinfo/python-list
Re: how to associate files with application
Ashok wrote: hi, i want to know how to make a specific type of file open in an application i developed in python when the user clicks on the file.(in windows) for eg. a .txt file when clicked opens in notepad, a .doc file when clicked opens in MS-word. In the same way i want to make a .xyz file open in the application i developed when clicked. thanks in advance for any advice. You need to add several registry keys to do this, here's a short version of what you need to do: Example assumes you want to: 1. associate .ext with C:\Program Files\MyProgram\prog.exe 2. pass on any extra arguments to prog.exe (ie. test.ext 1 2 3 would send 1 2 3 as well to prog.exe) 3. associate the icon of prog.exe to any file with a .ext extension Ok, here's what you need to do: 1. Under HKEY_CLASSES_ROOT, add a key (folder) with the name .ext 2. Open that key, and set the (Default) value to MyProgramExtendedFile (this name is something you choose yourself and should be a identifier that identifies the file type. If your program supports several types of files, make up unique identifiers for each.) 3. Under HKEY_CLASSES_ROOT, add another key, this time with the same name you made up in 2. above, ie MyProgramExtendedFile 4. Open that key, and set the (Default) value to a textual description of the type of file. This is what will show up in explorer in the file type column. If you leave this empty, the description will be .EXT File 5. Inside MyProgramExtendedFile, add another key with the name shell (lower-case is typical, can probably be Shell or whatever) 6. Inside shell, create another key with the name open 7. Inside open, create another key with the name command 8. Inside command, Set the (Default) value to: C:\Program Files\MyProgram\prog.exe %1 %* Note that you need the quotes as specified above, exactly like written 9. Go back to MyProgramExtendedFile and create another key with the name DefaultIcon 10. Inside DefaultIcon, set (Default) value to: C:\Program Files\MyProgram\prog.exe, 0 This will pick the first icon in prog.exe resource to show for the files. Use 1 for second, etc. There are also other commands you can add. If you want to be able to right-click on the file and select a menu item to process the file in a specific way, for instance by passing along specific parameters to prog.exe, you can add more keys than open on the level open is created. The (Default) value inside the key is then the text of the menu item. To find examples, just find a file extension in Windows that behaves the way you want your own to behave and look through HKEY_CLASSES_ROOT\.ext to find the details you want. -- Lasse Vågsæther Karlsen http://usinglvkblog.blogspot.com/ mailto:[EMAIL PROTECTED] PGP KeyID: 0x2A42A1C2 -- http://mail.python.org/mailman/listinfo/python-list
Re: Spambayes modifications with web services
[EMAIL PROTECTED] wrote: In the last few months many personal website owners (such as myself) have found that spammers have been using their domain names to masquerade as valid users to send spam, normally in the form of: snip So, as a web programmer and someone who specialises in getting good results on Google, I realised that I could simply post every spammer website on a Google optimized page, which if searched for on Google would return something like: WARNING: DO NOT BUY FROM THIS WEBSITE. THE SPAMMER IS A RUSSIAN MAFIA CROOK WHO WILL STEAL YOUR MONEY. snip So basically a DoS attack could now be simply performed by crafting a spam message and adding the url to your target and then sending it out to as many users you can think of ? (DoS not in the typical form, but the effect would be just as real, deny them of legitimate customers) Nice plan sherlock. -- Lasse Vågsæther Karlsen http://usinglvkblog.blogspot.com/ mailto:[EMAIL PROTECTED] PGP KeyID: 0x2A42A1C2 -- http://mail.python.org/mailman/listinfo/python-list
Sorting with only a partial order definition
I have a list of items and a rule for ordering them. Unfortunately, the rule is not complete so it won't define the correct order for any two items in that list. In other words, if I pick two random items from the list I may or may not have a rule that dictates the order of those two items. The rule could be implicit in that I got rules for other items, for instance: [a, b, c, d] a b b c If I now pick a and c out of the list I would not know wether a or c comes first, unless I grab the rules for ab and bc and imply that ac from those two. As such, there could be potentially many correct results from ordering such a list. (d is unspecified above so any position for d is actually legal) Is there anything in Python that will help me sort a list with something like this ? For instance, given the following list of items: items = [a, b, c, d] and the following two rules: a comes before d d comes before b then the following is a list of correct results (could be more though): [a, d, b, c] [a, c, d, b] [a, d, c, b] ... Note that this is an arbitrary example. The real list is a list of lists containing database rows, ie. something like this: [[10, Column 2, Column 3], [15, Column 2, Column 3], ...] If there isn't anything built in, does anyone have an idea about how to go about creating such an ordering function ? Tweaking a cmp-like function to return 0 for undefined didn't seem to work so there must be a slightly more intelligent solution than that. Perhaps the rules would have to be checked in a specific order. Doing a combinatorial solution and checking the rules aren't really an option as on last count there was close to 20K rows. There's also a lot of rules (also coming from a database). I was thinking I could do a sort based on one of the rules and use a stable sort for the following rules but doing that sometimes rearranged the list so that it was now incorrect going by the previous rules. Here's my initial test for doing a cmp-like solution: order = set([(a, d), (d, b)]) def my_cmp(a, b): if (a, b) in order: print a, , b return -1 elif (b, a) in order: print a, , b return +1 else: print a, ?, b return 0 items = [c, b, a, d] items.sort(cmp=my_cmp) print items This prints out: b ? c a ? b d a ['c', 'b', 'a', 'd'] Which is obviously incorrect. Any help or pointers would be most welcome. -- Lasse Vågsæther Karlsen http://usinglvkblog.blogspot.com/ mailto:[EMAIL PROTECTED] PGP KeyID: 0x2A42A1C2 -- http://mail.python.org/mailman/listinfo/python-list
Re: Sorting with only a partial order definition
Paul Rubin wrote: Lasse Vågsæther Karlsen [EMAIL PROTECTED] writes: I have a list of items and a rule for ordering them. Unfortunately, the rule is not complete so it won't define the correct order for any two items in that list. In other words, if I pick two random items from the list I may or may not have a rule that dictates the order of those two items. The rule could be implicit in that I got rules for other items, for instance: That's called topological sorting and any reference on graph algorithms will describe how to do it. I don't know of Python code offhand but it's easy to write. http://en.wikipedia.org/wiki/Topological_sorting gives a straightforward linear-time algorithm. Thank you both. -- Lasse Vågsæther Karlsen http://usinglvkblog.blogspot.com/ mailto:[EMAIL PROTECTED] PGP KeyID: 0x2A42A1C2 -- http://mail.python.org/mailman/listinfo/python-list
Re: Sorting with only a partial order definition
Lasse Vågsæther Karlsen wrote: Paul Rubin wrote: Lasse Vågsæther Karlsen [EMAIL PROTECTED] writes: I have a list of items and a rule for ordering them. snip Ok, managed to implement the algorithm. Might not be the optimal solution (memory and speed-wise) but it worked and doesn't take too long to run either so I'm going to stick with it. I have a different question though, along the same lines, but one that doesn't need a solution, just want to know if something like it exists. A while back I had an application that had a lot of items that it needed to order. The problem, however, was that the rules was not defined at all. Instead it was opted for a visual solution where the user would be presented with images and had to rearrange them in the order that was necessary. The application was one I helped build for a friend which combined images from several cameras and allowed him to sort them according to the contents so that he could get a timeline formed. The date/time stamps on the cameras was not directly usable for various reasons so the visual ordering was what we ended up on. For instance, two of the cameras was not digital ones so they had no timestamp except for the one provided by the scanning software. In that application we talked about presenting the user with two and two images and he just had to click on the image that came first. The problem with this was to try to present the right images to the user so that he had to minimize the number of clicks. In other words, try to pick nodes in the graph and ask the user to provide the direction of the edge, and the picking algorithm would work in such a way that the number of edges would be minimized. Not sure if I'm explaining it correctly. The solution we ended up with was to present the user with all the images in one big timeline and just let him drag them around. This worked. What I was wondering about is if there is an algorithm that would do what I want? Ie. help me pick the nodes so as to minimize the number of edges. Obviously the answer to the first pair of nodes will influence which nodes will be subsequently picked, so each answer would stear the algorithm in a way, not just go into the final problem. If anyone got the name of such an algorithm or something I would like to look at it at least. Application is built and deployed so I'm not looking for a solution to implement. -- Lasse Vågsæther Karlsen http://usinglvkblog.blogspot.com/ mailto:[EMAIL PROTECTED] PGP KeyID: 0x2A42A1C2 -- http://mail.python.org/mailman/listinfo/python-list
Re: Microsoft Hatred FAQ
David Schwartz wrote: Roedy Green wrote: snip competing products. (Just as Burger King corporate will not you sell Big Macs in the same store in which you sell Whoppers.) Rather odd comparison don't you think ? A better comparison would be if Burger King purchases the fries from a factory that says that Burger King has to give out a pack of fries with all meals, regardless of the type of meal, or they are going to raise the price. In other words, you'll be forced to take a pack of fries with your ice cream, salad or what not. Considering that McDonalds have been selling meals with potato-boats (don't know the correct english term for it, carved potato pieces fried), they'd have to give you a pack of fries with your meal regardless, even if you want to replace the fries with potato-boats. Also, in this case Burger King won't sell you is not the same as can't sell you, which seems to be the case with this whole Microsoft discussion. I'm pretty sure you wouldn't be able to easily buy a computer from Microsoft with OS/2 installed or vice versa either and I'm not sure they would be obliged to do so either. However, controlling what an independant outlet is doing, that's different. -- Lasse Vågsæther Karlsen http://usinglvkblog.blogspot.com/ mailto:[EMAIL PROTECTED] PGP KeyID: 0x2A42A1C2 -- http://mail.python.org/mailman/listinfo/python-list
Re: Microsoft Hatred FAQ
David Schwartz wrote: Iain King wrote: Don't you see how your metaphor doesn't work? No. It would only be fitting if Microsoft OWNED the outlet. Huh? I would think that if I set up a shop and wanted to have the word Microsoft as part of the shop name, there would be some rules dictating what products I could and could not sell, yes. Wether those rules are set forth in a law somewhere or Microsoft set them forth themselves, I would find it hard to believe that the law would prohibit them from doing so. Otherwise I could set up a shop, call it Microsoft Porsgrunn and sell machines with only Linux installed. I think Microsoft would be allowed to say No, you can't do that. -- Lasse Vågsæther Karlsen http://usinglvkblog.blogspot.com/ mailto:[EMAIL PROTECTED] PGP KeyID: 0x2A42A1C2 -- http://mail.python.org/mailman/listinfo/python-list
Re: Microsoft Hatred FAQ
Roedy Green wrote: On Thu, 27 Oct 2005 04:06:16 -0700, David Schwartz [EMAIL PROTECTED] wrote, quoted or indirectly quoted someone who said : Right I understand that. You could have complied simply by only selling computers with Windows preinstalled. In other words, you could have treated this the same as a demand for franchise or exclusivity if you had wanted to. It is obvious to everyone WHY MS did this, to maintain monopoly. But ignore motive for a while and see what they actually did and exactly how they intended to carry out he threat of destroying my business. What they did is clearly criminal. The hard part is proving it. Like any smart criminal who makes a threat, MS left no paper trail.. snip 2. it was a threat to force me to commit a criminal act -- namely extract money from people and hand it to Microsoft and give those What, specifically, is the criminal act of which you speak? -- Lasse Vågsæther Karlsen http://usinglvkblog.blogspot.com/ mailto:[EMAIL PROTECTED] PGP KeyID: 0x2A42A1C2 -- http://mail.python.org/mailman/listinfo/python-list
Re: Microsoft Hatred FAQ
David Schwartz wrote: Lasse Vågsæther Karlsen wrote: I would think that if I set up a shop and wanted to have the word Microsoft as part of the shop name, there would be some rules dictating what products I could and could not sell, yes. Wether those rules are set forth in a law somewhere or Microsoft set them forth themselves, I would find it hard to believe that the law would prohibit them from doing so. Otherwise I could set up a shop, call it Microsoft Porsgrunn and sell machines with only Linux installed. I think Microsoft would be allowed to say No, you can't do that. Burger King won't let you sell Whoppers or buy their burger patties wholesale no matter what you want to call your store unless you take the whole franchise deal. It's an all-or-nothing package. With very few limits, companies do get to choose how their products are branded, marketed, and sold. Yes, and that's not what Microsoft has ever done. There have always been lots of shops selling Microsoft merchandise without being a Microsoft franchise in the sense Burger King shops are. That's why I still say your comparison is a bad one. -- Lasse Vågsæther Karlsen http://usinglvkblog.blogspot.com/ mailto:[EMAIL PROTECTED] PGP KeyID: 0x2A42A1C2 -- http://mail.python.org/mailman/listinfo/python-list
Re: sort problem
How about: list.sort(key=lambda x: x[3]) Does that work? -- http://mail.python.org/mailman/listinfo/python-list
Re: How to get a raised exception from other thread
Steve Holden wrote: snip Why should the coder of this software have to go through this deliberate set up attrition, to get at this functionality, just because it wasn't intented to be used in such a way by the developers? Because otherwise people who know no better will use the feature for purposes where it's not the best way to achieve the required functionality, leading to yet more endless discussions about why it doesn't work. Asynchronous signalling between threads is an accident waiting to happen in the hands of an inexperienced programmer. I can't agree with that opinion. There's tons of features of existing programming languages and systems, Python included, that is pretty much guaranteed to be misused, and badly at that. We can't remove those just on the off chance someone might start a thread on a newsgroup about why this is or isn't the best way to do things. If we do that then I guess list comprehension is out the window because there's tons of posts (from me included) that get this wrong. The point of a programming language is not to dumb down the environment to a point where it is impossible to write bad code. Instead it's about empowering the programmers to be able to accomplish their tasks. In any case, this exception-in-another-thread problem has inherent issues which must be solved, among others: - C extension code can't be interrupted - an exception thrown at the wrong time in a finally/except block might cause more problems than it intends to solve So until a good implementation exists, there shouldn't be any point in actually discussing the motives of the programmers who wishes to use the feature. snip -- Lasse Vågsæther Karlsen http://usinglvkblog.blogspot.com/ mailto:[EMAIL PROTECTED] PGP KeyID: 0x2A42A1C2 -- http://mail.python.org/mailman/listinfo/python-list
Re: Function to execute only once
Bengt Richter wrote: snip tmp = 0 def execute(): ... global tmp, execute ... tmp = cellvar = tmp + 1 ... def execute(): ... return cellvar ... return tmp snip On man did this put my head into a spin :P -- Lasse Vågsæther Karlsen http://usinglvkblog.blogspot.com/ mailto:[EMAIL PROTECTED] PGP KeyID: 0x2A42A1C2 -- http://mail.python.org/mailman/listinfo/python-list
Re: How to get a raised exception from other thread
Peter Hansen wrote: [EMAIL PROTECTED] wrote: Nevermind. I found a better solution. I used shared memory to create a keep-alive flag. I then use the select function with a specified timeout, and recheck the keep-alive flag after each timeout. As Dennis points out, your original attempt was destined to fail because you were calling the method from the main thread, not the one you wanted snip A similar problem exists in .NET but they solved it by creating an Abort method that raises an exception in the thread itself, basically doing what you wanted to do. However, this leads to all sorts of problem, particular when that exception is raised in a finally block in that thread, which could mean that the finally block did not fully execute, leading to resource leaks or similar problems. They pondered a while about constructing non-interruptible blocks of code for .NET 2.0 but I'm not entirely sure how that worked out. In any case, the best way is to use some kind of signal that the thread reacts to, either by having it specifically wait for a signallable object (like an event) or just by checking a boolean variable or similar for a magic value that means now is a good time for you to terminate. -- Lasse Vågsæther Karlsen http://usinglvkblog.blogspot.com/ mailto:[EMAIL PROTECTED] PGP KeyID: 0x2A42A1C2 -- http://mail.python.org/mailman/listinfo/python-list
Re: Perl-Python-a-Day: Sorting
Diez B. Roggisch wrote: snip Diez [1] http://xahlee.org/PageTwo_dir/Personal_dir/mi_pixra.html Oh man... Talk about ... bummer. Seriously, who do we call to get someone with a straightjacket to show up at his home? -- Lasse Vågsæther Karlsen http://usinglvkblog.blogspot.com/ mailto:[EMAIL PROTECTED] PGP KeyID: 0x2A42A1C2 -- http://mail.python.org/mailman/listinfo/python-list
Re: bizarro world (was Re: Python Doc Problem Example: sort() (reprise))
Bryan wrote: mr. xah... would you be willing to give a lecture at pycon 2006? i'm sure you would draw a huge crowd and a lot of people would like to meet you in person... thanks. I think that would be a highly un-pythonesque crowd. Python isn't much in the sense of limitations, but that crowd probably needs to be limited in one way or another, like only 2 rotten fruits per person or similar. -- Lasse Vågsæther Karlsen http://usinglvkblog.blogspot.com/ mailto:[EMAIL PROTECTED] PGP KeyID: 0x2A42A1C2 -- http://mail.python.org/mailman/listinfo/python-list
Re: crosswords helper program
gg wrote: I plan to write a program in Python in order to help me doing crosswords, I was wondering if such a program already existed. Basically it will get the number of letters of the word (5, 10, 12...) then the letters known (B in second letter, E in 5th letter...) and then search in a dictionary the words matching this criteria Regards Gerard Pretty straightforward with a regular expression: import re words = [BULLETIN, BALLPARK, BUSINESS] r = re.compile(^BU..N..S$, re.IGNORECASE) for word in words: if r.match(word): print word You would probably want to allow the user to input both some basic format for your program and convert it to a regular expression as well as input a full pattern, because then the user can have words that have a U or a E in the second letter, simply because he doesn't know which one is right yet. -- Lasse Vågsæther Karlsen http://usinglvkblog.blogspot.com/ mailto:[EMAIL PROTECTED] PGP KeyID: 0x2A42A1C2 -- http://mail.python.org/mailman/listinfo/python-list
Re: Perl-Python-a-Day: Sorting
Xah Lee wrote:
Sorting in Perl
In Perl, to sort a list, do like this:
@li=(1,9,2,3);
@li2 = sort {$a = $b} @li;
print join(' ', @li2);
In Perl, sort is a function, not some Object Oriented thing. It returns
the sorted result as another list. This is very simple and nice.
Like the sorted function in Python ?
li2 = sorted(li)
you can also specify a key and a cmp function if you need to.
snip
In this way, the cost of the internals of the ordering function is
avoided. (it runs on each list element once) However, your huge list is
copied 1 extra time. So, there are pros and cons. Because this work
around is very complex in both its semantics and syntax, it has
acquired a coolness factor among Perl coders, and is given the name
Schwartzian Transform.
That's about the worst explanation I've seen of Scwartzian Transform
anywhere. Why deviate from what *every other source* does? Give an
example, like the typical one of sorting filenames based on the size of
the file they refer to. If you get the size in the comparison function,
you can potentially get the size (which is a rather costly operation)
many times for each file. Instead, you prefetch it once for each
filename and store that in the list, and use the size as the key for
sorting.
So while your explanation is technically correct, you lost some of the
explaining factors. In other words, using Google lets anyone wanting to
know about Schwartzian Transform find *much* better explanations than
your mumbo jumbo.
But that's no surprise, you're Xah Lee anyway, loosing out is your
middle name.
BTW, this way of doing the sort is nothing special for Perl, that
construct can be seen many places simply because it does not fight any
of Perls problems with sorting, instead it overcomes a common problem
with sorting in any language.
It is interesting to note what compiler flaws can do to imperative
languages and its people. In Python, the language syntax is tainted. In
tainted how?
Perl, a complex construct is invented. In both camps, the basic
invented how?
mathematics of sorting and its implementation aspects are completely
belied.
belied how?
It's interesting to note that these fact posts of yours are nothing
bud badly worded opinions.
--
Lasse Vågsæther Karlsen
http://usinglvkblog.blogspot.com/
mailto:[EMAIL PROTECTED]
PGP KeyID: 0x2A42A1C2
--
http://mail.python.org/mailman/listinfo/python-list
Re: Looking for info on Python's memory allocation
Sybren Stuvel wrote: Steven D'Aprano enlightened us with: he says that every time you try to append to a list that is already full, Python doubles the size of the list. This wastes no more than snip If, on the other hand, you double the memory every time you run out, you have to copy much less data, and in the end it turns out you need roughly N steps to add N items to the list. That's a lot better, isn't it? This begs a different question along the same lines. If I have a generator or other iterable producing a vast number of items, and use it like this: s = [k for k in iterable] if I know beforehand how many items iterable would possibly yield, would a construct like this be faster and use less memory? s = [0] * len(iterable) for i in xrange(len(iterable)): s[i] = iterable.next() -- Lasse Vågsæther Karlsen http://usinglvkblog.blogspot.com/ mailto:[EMAIL PROTECTED] PGP KeyID: 0x2A42A1C2 -- http://mail.python.org/mailman/listinfo/python-list
Re: Merging sorted lists/iterators/generators into one stream of values...
snip Another idea for this method would be that in some cases I noticed that it was useful to know which source each element would come from as well, as well as removing duplicates from the results. For instance s1 = [1, 3, 5, 7] s2 = [2, 3, 4] for k, s in merge_by_sort(s1, s2): print k, from source, s this would print: 1 from source 0 2 from source 1 3 from source 1 3 from source 0 4 from source 1 5 from source 0 7 from source 0 and the above list has 3 twice, so possibly: 1 from sources [0] 2 from sources [1] 3 from sources [0, 1] 4 from sources [1] 5 from sources [0] 7 from sources [0] This latter one would be a slightly more heavy method as it would have to compare the N first elements of the list or heap to figure out what indices to yield as well. However, the previous solution could be: def merge_by_sort(*sources, **options): if cmp in options: comparison = options[cmp] else: comparison = cmp iterables = [] for index, source in enumerate(sources): try: source = iter(source) iterables.append([source.next(), index, source]) except StopIteration: pass iterables.sort(cmp=comparison, key=lambda x: x[0], reverse=True) while iterables: yield iterables[-1][0], iterables[-1][1] try: iterables[-1][0] = iterables[-1][2].next() if len(iterables) 1 and comparison(iterables[-1][0], iterables[-2][0]) 0: iterables.sort(comparison, key=lambda x: x[0], reverse=True) except StopIteration: iterables.pop(-1) -- Lasse Vågsæther Karlsen http://usinglvkblog.blogspot.com/ mailto:[EMAIL PROTECTED] PGP KeyID: 0x2A42A1C2 -- http://mail.python.org/mailman/listinfo/python-list
How to do *args, **kwargs properly
I must be missing something but what is the proper way to do a function using such arguments ? Specifically I'm looking for: - ability to take an unspecified number of positional arguments - ability to take optional named arguments that follows the first arguments - raise appropriate errors if I use the wrong named arguments for instance: fn(1, 2, 3) fn(1, 2, 3, cmp=lambda x, y: y-x) fn(1, 2, 3, cpm=lambda x, y: y-x) # TypeError on this Is the following the way to go ? def fn(*values, **options): comparison = cmp for key, value in options.iteritems(): if key == cmp: comparison = value else: raise TypeError, '%s' is an invalid keyword argument for this function % key # rest of function here or is the proper python way simply this: def fn(*values, **options): if cmp in options: comparison = options[cmp] else: comparison = cmp # rest of function here and thus ignoring the wrong parameter names? -- Lasse Vågsæther Karlsen http://usinglvkblog.blogspot.com/ mailto:[EMAIL PROTECTED] PGP KeyID: 0x2A42A1C2 -- http://mail.python.org/mailman/listinfo/python-list
Re: How to do *args, **kwargs properly
Max M wrote: Lasse Vågsæther Karlsen wrote: I must be missing something but what is the proper way to do a function using such arguments ? - ability to take an unspecified number of positional arguments You should probably pass a sequence to the method instead. You can do it the other way, but it's poor coding style in most cases. So what you're saying is that instead of: def fn(*values, **options): I should use: def fn(values, cmp=cmp): in this specific case? and then instead of: fn(1, 2, 3, cmp=...) this: fn([1, 2, 3], cmp=...) snip I think I'll re-write to use a list instead. -- Lasse Vågsæther Karlsen http://usinglvkblog.blogspot.com/ mailto:[EMAIL PROTECTED] PGP KeyID: 0x2A42A1C2 -- http://mail.python.org/mailman/listinfo/python-list
Re: Merging sorted lists/iterators/generators into one stream of values...
Lasse Vågsæther Karlsen wrote: snip Another idea for this method would be that in some cases I noticed that it was useful to know which source each element would come from as well, as well as removing duplicates from the results. snip The removing duplicates problem would probably be best as a separate function and it occurs to me that perhaps Python has such a function already. Again, this function would need the following criteria: 1. Be able to iterate through something other than a list 2. Yield the values, not return a list 3. Take an arbitrary cmp function to determine what is a duplicate As sugar, perhaps also the following criteria: - Ability to combine the duplicates through a special function A simple first-version function I hacked together does this: def unique(source, cmp=cmp, key=None, combine=None): it = iter(source) first = True value = it.next() values = [value] while True: try: value = it.next() if key is not None: cmp_result = cmp(values[0][key], value[key]) else: cmp_result = cmp(values[0], value) if cmp_result == 0: values.append(value) else: if combine is not None: yield combine(values) else: yield values[0] values = [value] except StopIteration: if combine is not None: yield combine(values) else: yield values[0] break raise StopIteration Note that this function does not do any sorting so if the source that it gets the values from is not sorted, the result will be very wrong. This is again due to my criteria of being able to handle cursors retrieving data from a database and thus avoid loading everything into memory. The combine function is passed a list of duplicate values and must return a value that will be yielded out of unique. Example of usage: def sum_counts(values): value = values[0][0] sum = 0 for row in values: sum += row[1] return value, sum fruits = [[Apple, 10], [Apple, 15], [Banana, 23], [Orange, 17], [Orange, 17]] for fruit, total_sum in unique(fruits, key=0, combine=sum_counts): print fruit, has a sum of, total_sum This will produce: Apple has a sum of 25 Banana has a sum of 23 Orange has a sum of 34 Function name is perhaps not the best one. It occurs to me that this is the GROUP BY function in SQL so perhaps a different name is better, but then again this might be moot if such a function already exists somewhere :) -- Lasse Vågsæther Karlsen http://usinglvkblog.blogspot.com/ mailto:[EMAIL PROTECTED] PGP KeyID: 0x2A42A1C2 -- http://mail.python.org/mailman/listinfo/python-list
Re: How to do *args, **kwargs properly
George Sakkis wrote: Lasse Vågsæther Karlsen [EMAIL PROTECTED] wrote: snip I think I'll re-write to use a list instead Actually in most cases you don't need to assume it's a list; any iterable is usually good enough. You can always turn it into a list (or a tuple or a set or..) in the function if you really have to. So when you do have to and when you don't ? You don't have to if all you do is iterate over the elements. This is true even if you want to iterate more than once; just use the itertools.tee() function to create N independent iterators. To sum up, a better signature for your function is likely to be def fn(iterable, cmp=cmp). Yep, that sounds like a plan :) -- Lasse Vågsæther Karlsen http://usinglvkblog.blogspot.com/ mailto:[EMAIL PROTECTED] PGP KeyID: 0x2A42A1C2 -- http://mail.python.org/mailman/listinfo/python-list
Re: Merging sorted lists/iterators/generators into one stream of values...
George Sakkis wrote: Function name is perhaps not the best one. It occurs to me that this is the GROUP BY in SQL so perhaps a different name is better, but then again this might be moot if such a already exists somewhere :) Amazing, you keep reinventing things, even with the exact same name :) from itertools import imap,groupby from operator import itemgetter for fruit,group in groupby(fruits, itemgetter(0)): print fruit, has a sum of, sum(imap(itemgetter(1),group)) For this to work as intended, fruits has to be already sorted by the same key given to grouby; otherwise just replace fruits with sorted(fruits, itemgetter(0)). By the way, read all the functions in the itertools module (http://docs.python.org/lib/itertools-functions.html), it will save you a lot of time. George Itertools, meh, there's that wheel again :) Didn't know about this one so thank you :) -- Lasse Vågsæther Karlsen http://usinglvkblog.blogspot.com/ mailto:[EMAIL PROTECTED] PGP KeyID: 0x2A42A1C2 -- http://mail.python.org/mailman/listinfo/python-list
Re: new forum -- homework help/chit chat/easy communication
Fredrik Lundh wrote: snip Unlike mainstream component programming, scripts usually do not introduce new components but simply wire existing ones. Scripts can be seen as introducing behavior but no new state. /.../ Of course, there is nothing to stop a scripting language from introducing persistent state -- it then simply turns into a normal programming language. -- Clemens Szyperski, in Component Software: snip That description seems to describe whatever is written more than whatever it is written in, or in other words, it describes the difference between a script and a program, not between a scripting language and a programming language. I think that at one time, scripting languages was something that lived within other programs, like Office, and couldn't be used by themselves without running it inside that program, and as thus was a way to add minor functions and things to that program. Nowadays a lot of the scripting languages have turned programming languages so I think the difference is small. -- Lasse Vågsæther Karlsen http://usinglvkblog.blogspot.com/ mailto:[EMAIL PROTECTED] PGP KeyID: 0x2A42A1C2 -- http://mail.python.org/mailman/listinfo/python-list
Re: Jargons of Info Tech industry
Tim Tyler wrote: snip Only if your photos are so obscure and confusing that they need captions. Here's Johnny with the dog. Here is Johnny with the dog again. This one is Johnny on his own. Here is the dog. Oh look, it is Johnny with the dog again -- that's the dog on the left, in case it isn't clear. Just for a change, this is Johnny wearing a hat. It is blue with a feather in it, in case you couldn't tell from, oh I don't know, looking at the actual picture. What have you got against captions? Giving photos captions is a *very* common practice. Perhaps he has a search engine that can find blue hats in an image and recognize people? -- Lasse Vågsæther Karlsen http://usinglvkblog.blogspot.com/ mailto:[EMAIL PROTECTED] PGP KeyID: 0x2A42A1C2 -- http://mail.python.org/mailman/listinfo/python-list
Function decorator that caches function results
After working through a fair number of the challenges at
www.mathschallenge.net, I noticed that some long-running functions can
be helped *a lot* by caching their function results and retrieving from
cache instead of calculating again. This means that often I can use a
natural recursive implementation instead of unwinding the recursive
calls to avoid big exponential running-times.
Ok, so I thought, how about creating a decorator that caches the
function results and retrieves them from cache if possible, otherwise it
calls the function and store the value in the cache for the next invokation.
This is what I came up with so far:
def cache_function(fn):
cache = {}
def cached_result(*args, **kwargs):
if args in cache:
return cache[args]
result = fn(*args, **kwargs)
cache[args] = result
return result
return cached_result
Example of usage:
@cache_function
def fibonacci(idx):
if idx in (1, 2):
return 1
else:
return fibonacci(idx-1) + fibonacci(idx-2)
for index in range(1, 101):
print index, fibonacci(index)
this example goes from taking exponential time to run to being near
instantaneous.
However, this:
for index in range(1, 101):
print index, fibonacci(idx = index)
this one uses the kwargs list of arguments, and I'm not sure how to
change my function to take that into account.
Does anyone have any clues as to how to do that?
The solution would have to consider fibonacci(50) and fibonacci(idx =
50) as the same call and thus retrieve the second one from the cache.
--
Lasse Vågsæther Karlsen
http://usinglvkblog.blogspot.com/
mailto:[EMAIL PROTECTED]
PGP KeyID: 0x2A42A1C2
--
http://mail.python.org/mailman/listinfo/python-list
Re: Function decorator that caches function results
[EMAIL PROTECTED] wrote:
On Sat, Oct 08, 2005 at 01:53:28PM +, Duncan Booth wrote:
Unless the results stored in the cache are very large data structures, I
would suggest that you simply store (args,kwargs) as the cache key and
accept the hit that sometime you'll cache the same call multiple times.
... except that dicts cannot be dict keys
Another 'memoize' decorator uses this to get the key:
kw = kwargs.items()
kw.sort()
key = (args, tuple(kw))
http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/325905
Jeff
Yeah, but as far as I can see it, this one too fails to recognize
situations where the function is called twice with essentially the same
values, except that in one call it uses named arguments:
k1 = fibonacci(100)
k2 = fibonacci(idx = 100)
this is essentially the same call, except the second one uses a named
argument, which means the function will be invoked and a second cache
entry will be stored.
Granted, not a big problem in most such cases, but here's my augmented
function. Bare in mind that I'm 2 weeks into Python so there's bound to
be room for improvement :)
def cache(fn):
cache = {}
arg_names = inspect.getargspec(fn)[0]
def cached_result(*args, **kwargs):
# If function is called without parameters, call it without
using the cache
if len(args) == 0 and len(kwargs) == 0:
return fn()
# Work out all parameter names and values
values = {}
for i in range(len(args)):
values[arg_names[i]] = args[i]
for key in kwargs:
values[key] = kwargs[key]
key = tuple([(key, value) for (key, value) in
sorted(values.iteritems())])
# Check cache and return cached value if possible
if key in cache:
return cache[key]
# Work out new value, cache it and return it
result = fn(*args, **kwargs)
cache[key] = result
return result
# Return wrapper function
return cached_result
--
Lasse Vågsæther Karlsen
http://usinglvkblog.blogspot.com/
mailto:[EMAIL PROTECTED]
PGP KeyID: 0x2A42A1C2
--
http://mail.python.org/mailman/listinfo/python-list
Re: Function decorator that caches function results
Sam Pointon wrote:
What about not storing args at all? Something like this:
def cache_function(func, args_list):
cache = {}
def cached_result(*args, **kwargs):
kwargs.update(dict(zip(args_list, args)))
if kwargs in cache:
return cache[kwargs]
result = func(**kwargs)
cache[kwargs] = result
return result
return cached_result
args_list is a list of all the argument names, so that they can be
converted into keyword arguments.
I'll take a look at the zip function, didn't know about that one, but
your example also has the problem that dictionaries can't be used as
dictionary keys, but that can be easily solved. I think I can simplify
my solution with some of yours.
The parameter names can be gotten by doing a inspect.getargspec(fn)[0]
so that can be done by the decorator function.
--
Lasse Vågsæther Karlsen
http://usinglvkblog.blogspot.com/
mailto:[EMAIL PROTECTED]
PGP KeyID: 0x2A42A1C2
--
http://mail.python.org/mailman/listinfo/python-list
Re: Function decorator that caches function results
Lasse Vågsæther Karlsen wrote:
Sam Pointon wrote:
What about not storing args at all? Something like this:
snip
Ok, here's my updated version:
class cache(object):
def __init__(self, timeout=0):
self.timeout = timeout
self.cache = {}
def __call__(self, fn):
arg_names = inspect.getargspec(fn)[0]
def cached_result(*args, **kwargs):
# Update named arguments with positional argument values
kwargs.update(dict(zip(arg_names, args)))
# Work out key as a tuple of ('argname', value) pairs
key = tuple(sorted(kwargs.items()))
# Check cache and return cached value if possible
if key in self.cache:
(value, last_time) = self.cache[key]
if self.timeout = 0 or time.time() - last_time =
self.timeout:
return value
# Work out new value, cache it and return it
result = fn(**kwargs)
self.cache[key] = (result, time.time())
return result
# Return wrapper function
return cached_result
Changed from previous versions:
- converted to class, must use () on decorator now
- added timeout, results will be recalculated when it expires
- timeout=0 means no timeout, results will always be reused
- now handles both positional and keyword arguments
--
Lasse Vågsæther Karlsen
http://usinglvkblog.blogspot.com/
mailto:[EMAIL PROTECTED]
PGP KeyID: 0x2A42A1C2
--
http://mail.python.org/mailman/listinfo/python-list
Re: Function decorator that caches function results
Fredrik Lundh wrote: snip k1 = fibonacci(100) k2 = fibonacci(idx = 100) snip whoever writes code like that deserves to be punished. I'd say this thread points to a misunderstanding of what keyword arguments are, and how they should be used. the basic rule is that you shouldn't mix and match; use positional arguments for things that are documented to be positional snip I agree completely. However, it turns out I have to communicate code with and from people that have a coding standard that dictates using keyword arguments for all interfaces. Those functions will also benefit from the cache system as many of them involves database lookups. In any case, your response gave me another thing that my solution won't handle so I'm going to just leave it as it is and look at it if I ever need it for positional arguments, which I honestly don't believe I will. -- Lasse Vågsæther Karlsen http://usinglvkblog.blogspot.com/ mailto:[EMAIL PROTECTED] PGP KeyID: 0x2A42A1C2 -- http://mail.python.org/mailman/listinfo/python-list
Re: Merging sorted lists/iterators/generators into one stream of values...
Alex Martelli wrote: George Sakkis [EMAIL PROTECTED] wrote: snip Yes, it's a little inconvenient that the builtin heap doesn't take a comparison operation but you can easily roll your own heap by transforming each item to a (key,item) tuple. Now that I'm thinking about it, it might be a good addition to the cookbook. I believe Python 2.5 adds a key= argument to heapq's functions... snip I will revisit the heapq solution when 2.5 is released then. Thanks for the heads up. For the moment I will stay with the list solution that Mike came up with slightly changed to accomodate tips and pointers from others in this thread. -- Lasse Vågsæther Karlsen http://usinglvkblog.blogspot.com/ mailto:[EMAIL PROTECTED] PGP KeyID: 0x2A42A1C2 -- http://mail.python.org/mailman/listinfo/python-list
Re: Merging sorted lists/iterators/generators into one stream of values...
George Sakkis wrote: snip Just added a recipe at http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/440673. You can try both and see if there's any significant performance difference for your data. snip Thanks, will take a look at it later. The sort solution seems to work nicely. Might be a big difference if I have a lot of sources though as I bet the overhead in doing a sort of N items gets higher than doing a manipulation of a heap to place an item in the right spot, but with 4-5 or a few more sources might not make an impact at all. -- Lasse Vågsæther Karlsen http://usinglvkblog.blogspot.com/ mailto:[EMAIL PROTECTED] PGP KeyID: 0x2A42A1C2 -- http://mail.python.org/mailman/listinfo/python-list
Re: Function decorator that caches function results
George Sakkis wrote: snip Cool, you re-invented the memoization pattern: http://en.wikipedia.org/wiki/Memoization http://aspn.activestate.com/ASPN/search?query=memoizex=0y=0section=PYTHONCKBKtype=Subsection Yes, it's kinda discouraging that most interesting ideas have already been conceived, implemented and used by others...wink snip I know, I've been scouring over the ASPN recipes and digging through code like there's no tomorrow. But, I don't view it as pointless even though there are existing implementations and solutions out there. First of all, I don't like to stuff a lot of what is obviously library type of code into a project, unless I can reference a library that got that function or class or whatnot. Creates a rather big maintenance nightmare :) So, I would have to stuff that into a library, which is what I did with my own function (thank you for helping btw). The recipe on ASPN is probably better than what I got, but... I understand how my solution came about and what makes it tick. Secondly, I've been programming Python for, what, about 11 days now or so, so I want to re-implement as much as possibly right now, even to the point where I create a worse solution than an existing one, as long as it works for me, just to be able to learn the nuances of Python, because Python is ... different than what I'm used to. For instance, in C# and .NET you got attributes, but they don't actually do anything on their own, in other words you can't tag a method and have the operation of that method deviate from a similar method without the attribute, unless you pick one of the attributes the compiler knows about, so it's just meta-data that sits silent until some other method goes around to look for it. In Python I've now learned that a function is just an object like everything else and I can wrap a new object around it to modify its behaviour, and I can use the decorator pattern to do it. I'm probably going to be back here in a few days or even hours with another task where you can probably cough up dozens of existing source code solutions that I could use. For instance, there's this thing I've heard of called the wheel. :) -- Lasse Vågsæther Karlsen http://usinglvkblog.blogspot.com/ mailto:[EMAIL PROTECTED] PGP KeyID: 0x2A42A1C2 -- http://mail.python.org/mailman/listinfo/python-list
Re: new forum -- homework help/chit chat/easy communication
[EMAIL PROTECTED] wrote: I've launched a new forum not too long ago, and I invite you all to go there: www.wizardsolutionsusa.com (click on the forum link). We offer all kinds of help, and for those of you who just like to talk, there's a chit chat section just for you...Just remember that forum communication is much easier, safer, and faster. Easier than what? Having to look into each forum to see if something is new? That's easier? Safer than what? Using a web browser? That's safe? Faster? That page loads 10 posts in the same speed I get 700 posts with usenet. Don't think so matey. Nice try though. -- Lasse Vågsæther Karlsen http://usinglvkblog.blogspot.com/ mailto:[EMAIL PROTECTED] PGP KeyID: 0x2A42A1C2 -- http://mail.python.org/mailman/listinfo/python-list
Re: new forum -- homework help/chit chat/easy communication
Fredrik Lundh wrote: Lasse Vågsæther Karlsen wrote: Don't think so matey. oh, come on. a site run by some random guy in North Carolina has to be safer, faster and more reliable than a distributed communication system that has been around since that guy was born... Yes, of course, my mistake, it's rather obvious now that you point it out. -- Lasse Vågsæther Karlsen http://usinglvkblog.blogspot.com/ mailto:[EMAIL PROTECTED] PGP KeyID: 0x2A42A1C2 -- http://mail.python.org/mailman/listinfo/python-list
Re: Daisy Daisy, give me your answer do
Leave Xah Lee alone, he's a troll, he got no interested in doing anything but to provoke people on usenet. -- Lasse Vågsæther Karlsen http://usinglvkblog.blogspot.com/ mailto:[EMAIL PROTECTED] PGP KeyID: 0x2A42A1C2 -- http://mail.python.org/mailman/listinfo/python-list
Re: Jargons of Info Tech industry
Mike Meyer wrote: If you've got a browser with a better solution, what's the browser, and what's the solution? There is no single solution. On one side you got control freaks who condemn everyone who dares send an email with something other than what you've got your own email set up to use. You dare specify the font sized when I finally figured out that 10 is just right? Infidel! On the other side you got people who would like to have slightly more control over their email formatting than the ability to hit enter to denote a line break. Whaddya mean I can't specify bold text to emphasize a point? As long as you got something called a feature, there will always be people who will be able to abuse it. Now, if you want to get into a big huff because someone you knows use a font that is slightly oversized because of Windows, then I think you missed the point of the email altogether, which was probably to convey a message. Just the same as people that sends bright red text on green background seems to miss the whole issue of appropriate. It doesn't matter what is used, decided, controlled, allowed, removed, whatever. Some people will always like it, some will hate it, some will abuse it, but most of all there will always be people that will discuss it. Hopefully most people will use it for what it is. In any case, html email is here to stay. Or perhaps I should remove html and say richly formatted, whatever that might mean in the future. But trying to keep your email world into a pure text-based no-formatting-whatsoever world, that's a fantasy bubble that is bound to burst, sooner rather than later. Deal with it. -- Lasse Vågsæther Karlsen http://usinglvkblog.blogspot.com/ mailto:[EMAIL PROTECTED] PGP KeyID: 0x2A42A1C2 -- http://mail.python.org/mailman/listinfo/python-list
Re: new forum -- homework help/chit chat/easy communication
Brandon K wrote: Hrm...i find it demeaning to relegate Python to a scripting language while Visual Basic is in the software development section. Python so outdoes VB in every way shape and form. snip In that respect I would very much like to see a definition of scripting language as well :) In other words, what is the difference between a scripting language and a programming language. -- Lasse Vågsæther Karlsen http://usinglvkblog.blogspot.com/ mailto:[EMAIL PROTECTED] PGP KeyID: 0x2A42A1C2 -- http://mail.python.org/mailman/listinfo/python-list
Re: Merging sorted lists/iterators/generators into one stream of values...
Ok, that one looks more sleak than what I came up with. Couple of things I learn from your solution, please correct me if I misunderstood something: 1. list containing other lists will sort itself based on first element on lists inside ? 2. sort(), pop() is not costly operations Other than that you seem to not use the cmp operator but that's easily fixed. This one looks much better than mine, here's what I came up with: def merge_sorted(iterables, comparer=None): Generator that will take a list of iterables/generators that is individually sorted, and then produce values in a sorted order by taking the lowest value from each source each call. @param iterables: List of iterables/generators to retrieve values from @type iterables: List of iterables/generators @param comparer: Optional fn(v1, v2) function that can compare two values, should return 0 if v1v2, 0 if v1v2 and ==0 if v1==v2. @type comparer: function-object, example: lambda x, y: x-y @note: The list of iterables can actually be anything that produces a list of iterables, so you can use a function that yields lists for instance. # First convert whatever we're given into a list of sources iterables = [iterable for iterable in iterables] # This series of if-statements will determine how many sources we have and work out sub-problems # that are manageable. if len(iterables) != 2: if len(iterables) == 0: # List, but no sources pass elif len(iterables) == 1: # Only 1 source, just return its contents for value in iterables[0]: yield value elif len(iterables) == 3: # 3 sources, sub-divide into 0 -- (1, 2) left_iterable = iterables[0] right_iterable = merge_sorted([iterables[1], iterables[2]], comparer) for value in merge_sorted([left_iterable, right_iterable], comparer): yield value elif len(iterables) == 4: # 4 sources, sub-divide into (0, 1) -- (2, 3) left_iterable = merge_sorted([iterables[0], iterables[1]], comparer) right_iterable = merge_sorted([iterables[2], iterables[3]], comparer) for value in merge_sorted((left_iterable, right_iterable), comparer): yield value elif len(iterables) 4: # 4 sources, sub-divide into (0, 1) -- (2, ...) left_iterable = merge_sorted([iterables[0], iterables[1]], comparer) right_iterable = merge_sorted(iterables[2:], comparer) for value in merge_sorted((left_iterable, right_iterable), comparer): yield value raise StopIteration # The method will only get here if there is only two sources, which is an easy case to handle i1 = iter(iterables[0]) i2 = iter(iterables[1]) # Grab the first two values from the two sources, if possible try: v1 = i1.next() has_v1 = True except StopIteration: has_v1 = False try: v2 = i2.next() has_v2 = True except StopIteration: has_v2 = False # As long as we got values from both generates/iterators/whatever, compare and yield the lowest of the # two, and then get the next value from the corresponding source while has_v1 and has_v2: # Work out which of v1 and v2 comes first if comparer is not None: comp = comparer(v1, v2) if comp = 0: yield_v1 = True else: yield_v1 = False else: if v1 = v2: yield_v1 = True else: yield_v1 = False # Yield the next value, then grab a new value from the corresponding source if yield_v1: yield v1 try: v1 = i1.next() except StopIteration: has_v1 = False else: yield v2 try: v2 = i2.next() except StopIteration: has_v2 = False # When we get here, we got 3 possibilities: # 1. has_v1 == True, has_v2 == False -- yield rest of v1/i1 and just exit on StopIteration exception # 2. has_v1 == False, has_v1 == True -- yield rest of v2/i2 and just exit on StopIteration exception # 3. has_v1 == has_v2 == False -- while-loops will skip, function falls off the end while has_v1: yield v1 v1 = i1.next() while has_v2: yield v2 v2 = i2.next() -- http://mail.python.org/mailman/listinfo/python-list
Re: Merging sorted lists/iterators/generators into one stream of values...
Thanks, that looks like Mike's solution except that it uses the built-in heapq module. While this one contains less code than Mike's solution it seems to lack the ability to control the comparison operation, which means it won't work in my case. I need to both be able to sort on an arbitrary field name (could be done using a list where I place the field first), and also to be able to sort in a different order than smallest-first. Perhaps by adjusting the data that is returned through each source would do that. I'll look into it. -- http://mail.python.org/mailman/listinfo/python-list
Re: Merging sorted lists/iterators/generators into one stream of values...
Lasse Vågsæther Karlsen wrote: I need to merge several sources of values into one stream of values. All of the sources are sorted already and I need to retrieve the values from snip Ok, after working through the various sources and solutions, here's what I finally ended up with: def merge_sorted(comparison, *sources): iterables = [] for source in sources: try: source = iter(source) iterables.append([source.next(), source]) except StopIteration: pass iterables.sort(cmp=comparison, key=lambda x: x[0]) while iterables: yield iterables[0][0] try: iterables[0][0] = iterables[0][1].next() if len(iterables) 1 and comparison(iterables[0][0], iterables[1][0]) 0: iterables.sort(comparison, key=lambda x: x[0]) except StopIteration: iterables.pop(0) Thanks to Mike and George for the solutions and pointers. -- Lasse Vågsæther Karlsen http://usinglvkblog.blogspot.com/ mailto:[EMAIL PROTECTED] PGP KeyID: 0x2A42A1C2 -- http://mail.python.org/mailman/listinfo/python-list
Re: Contest snub?
[EMAIL PROTECTED] wrote: Notice anything strange here? The Python entry seems to have edged the PHP entries, but is not declared the victor. Source is missing as well (the archive is empty.) http://www.apress.com/promo/fractal/seesource.html H... an irrational fear of snakes perhaps? Clearly Python is superior because it doesn't need any code at all to do fractals. Surely you can see that? :) I question the timings though, 2+ minutes to execute nothing doesn't seem very pythonesque. -- Lasse Vågsæther Karlsen http://usinglvkblog.blogspot.com/ mailto:[EMAIL PROTECTED] PGP KeyID: 0x2A42A1C2 -- http://mail.python.org/mailman/listinfo/python-list
Merging sorted lists/iterators/generators into one stream of values...
I need to merge several sources of values into one stream of values. All of the sources are sorted already and I need to retrieve the values from them all in sorted order. In other words: s1 = [10, 20, 30, 40, 50] s2 = [15, 25] s3 = [17, 27, 37] for value in ???(s1, s2, s3): print value will print out 10, 15, 17, 20, 25, 27, 30, 37, 40, 50 in that order. The sources are cursors retrieving data from several databases, not from the same server, and there's a potential for a large number of rows from several of the sources. As such, any method that would load it all into memory and sort it is no good as it would too much memory. Is there a function or whatnot in Python that will do what I want? I have come up with my own method but since it uses generators I'm not sure how much overhead there is. Additionally, since the values retrieved from the cursors will be dictionaries of fieldname:value pairs, the method would either need to handle that construct (and be told what fieldname to sort on), or be able to take a function object to use for the comparison operation. Basically, I'll use my own function for this unless someone can point me to something that is already available. Couldn't seem to find anything in the builtin modules but I didn't find glob.glob when I was looking for how to find filenames either so who knows what it's called :) Since I need to deal with a variable list of sources (that is, 2 in one application, 3 in another and 4-5 in a third), a simple 2-source method isn't enough but if it's better than what I do for 2 sources then I can make a wrapper for it, since that's what I do anyway. -- Lasse Vågsæther Karlsen http://usinglvkblog.blogspot.com/ mailto:[EMAIL PROTECTED] PGP KeyID: 0x2A42A1C2 -- http://mail.python.org/mailman/listinfo/python-list
Re: IDLE won't run on WinXP
striker wrote: I just downloaded and installed Python 2.4.2 on a Windows XP machine. Everything I have tried so far has worked except for IDLE. Is there some setting that I need to check? Any help would be greatly appreciated. Kevin It runs perfectly for me, on several Windows XP machines running a variety of service packs, so it's not a general Windows XP problem. Do you get any error messages? any indication as to what might be wrong that you can share with use? Does it crash ? Does anything happen when you click the start menu item? Just a thought though, during the installation of Python on one machine it mentioned that a dll was missing from my machine, try uninstalling it and reinstalling it and see if the text you get at some point tells you that this dll is missing. I find it various places on the web, and I remember the text said something about Idle or Pythonwin, so it might be what you need. I think the name was something like MSVCRT71.DLL but don't think I got it 100% right just there... :P -- Lasse Vågsæther Karlsen http://usinglvkblog.blogspot.com/ mailto:[EMAIL PROTECTED] PGP KeyID: 0x2A42A1C2 -- http://mail.python.org/mailman/listinfo/python-list
Re: While and If messing up my program?
The specific error in your code, is that when cnto == len(ttllst), then doing ttllst[cnto] will give you that error. The list is indexed from 0 to len-1, which means that doing list[len(list)] will give that error. -- http://mail.python.org/mailman/listinfo/python-list
Re: New project coming up...stay with Python, or go with a dot net language??? Your thoughts please!
While Microsoft and other big software vendors might have a roadmap that ties you very tightly in with their budget, and also changes that roadmap from time to time which breaks your current software, a lot of open source projects have no roadmap at all. This means that a .x.y.2 upgrade might very well kill your software in the same way a 6-7 upgrade with VB might. The biggest reason for this, as far as I can tell, is that open source projects are very much built to integrate with other projects, simply because the source is available and it's thus far easier to integrate them, but that also means that unless you upgrade two integrated projects at the same time, you risk an upgrade in one breaking the other. As an example, Komodo from ActiveState has serious problems (on my 4 computers) debugging functions with Python 2.4.2. Right now I don't know if Python 2.4.2 fixed a bug Komodo depended on or if 2.4.2 introduced a bug breaking Komodo, or if my computer is some freaky thing that just refuses to behave. I'm reinstalling my office computer these days so I'll see what happens then, but until that's done, I've had to go back to 2.4.1 in order to debug properly. With a 6.0 to 7.0 upgrade for a project, I expect problems. Even Microsoft came out and said this new release will definitely break your software projects. The whole point of an upgrade is that something that did X before, does Y now, something different. Preferrably Y = X+1, but sometimes it's radically different. I think a main point in a lot of projects is that unless you have to upgrade, don't. That way you can avoid a lot of problems by default. Now, as for OP, the only way to easily get full support for all the new Microsoft technology when it comes out will probably be to upgrade to the latest and the greatest of the Microsoft tools. I bet the Python community will react quicker than you can turn your head, but the tools will still not be the same as Microsoft. Does it mean they won't be as good ? No, it means they will be different. It means that you probably can't follow Microsoft help when dealing with Python modules. And that, is exactly as it should be. -- http://mail.python.org/mailman/listinfo/python-list
Re: New project coming up...stay with Python, or go with a dot net language??? Your thoughts please!
Ok, when re-reading my post it seems that I'm saying that Python has no roadmap. That was not my intent. I meant projects other than Python, even though the problems I got with 2.4.2 is real, I suspect there's something in Komodo that is the problem since I can run all my python programs with 2.4.2 without problems, just not in Komodo. I'll go wash my mouth now :P -- http://mail.python.org/mailman/listinfo/python-list
Question about stdin and PATHEXT on Windows XP
I got a loop like this: while True: line = sys.stdin.readline() if not line: break line = line.rstrip() log_message(options.channel_name, line) print line this loop will read text if I execute it from a command prompt like this: dir | python.exe log.py however, since I've set PATHEXT to contain .PY, I tried this: dir | log.py which ended up with the loop just skipping. Does anyone know how to fix this problem ? Being able to use PATHEXT execution (just execute python file directly) and still being able to grab stdin? I assume that since there is some magic involved in invoking python.exe here, stdin gets munged on the way. -- Lasse Vågsæther Karlsen http://usinglvkblog.blogspot.com/ mailto:[EMAIL PROTECTED] PGP KeyID: 0x2A42A1C2 -- http://mail.python.org/mailman/listinfo/python-list
Re: Question about stdin and PATHEXT on Windows XP
Lasse Vågsæther Karlsen wrote: I got a loop like this: snip Ok, I've found some information. I don't like the answer but it doesn't seem to be any good way to do this without hacking it one way or the other. If someone has a brilliant answer that isn't in google yet, please let me know. -- Lasse Vågsæther Karlsen http://usinglvkblog.blogspot.com/ mailto:[EMAIL PROTECTED] PGP KeyID: 0x2A42A1C2 -- http://mail.python.org/mailman/listinfo/python-list
Documenting members
I have a Connection class that exposes members for the hostname, etc. ie. class Connection: def __init__(...): self.server = server is there any way to document this server member? The only way I know of right now is to expose it as a property and add the doc string to that definition. I've tried finding an example in the lib code installed with Python but can't seem to track down anything that is documented like that. -- Lasse Vågsæther Karlsen http://usinglvkblog.blogspot.com/ mailto:[EMAIL PROTECTED] PGP KeyID: 0x2A42A1C2 -- http://mail.python.org/mailman/listinfo/python-list
Module organization
I am slowly learning Python and I'm already starting to write some minor modules for myself. Undoubtedly there are better modules available either built-in or 3rd party that do the same as mine and much more but I need to learn it one way or another anyway. What I'm wondering about is module organization. I created my own directory for storing my modules and added the full path to this to PYTHONPATH (Windows XP platform). This means that import modulename works for my modules now. However, the name of my module is db or database, a module name that will likely conflict with other modules I might encounter later. As such, I was thinking of doing the same that the distutils set of modules have done, by creating a subdirectory and storing them there, so I created a lvk directory in that directory of mine and moved the module in there, but now import lvk.modulename doesn't find the module. Is there a trick to this? Do I have to store my own modules beneath C:\Python24\Lib? or can I use the organization I've tried just with some minor fixes to make python locate my modules? -- Lasse Vågsæther Karlsen http://usinglvkblog.blogspot.com/ mailto:[EMAIL PROTECTED] PGP KeyID: 0x2A42A1C2 -- http://mail.python.org/mailman/listinfo/python-list
Documenting properties
I notice that if I use this syntax: def classname: ... ## # closes the database connection and releases the resources. def close(self): ## # Returns a list of fields fields = property() then doing: help (classname) then the text is listed for the property and the method, whereas if I do this: classname.close.__doc__ then nothing is listed, and to get that I have to use the .. syntax to document: def close(self): closes the datab... then classname.close.__doc__ shows the text. So, my question is, is there a way to get __doc__ support for properties, in effect, use the xxx syntax for documenting properties. Is the preferred way to use xxx or # to document ? Whatever is preferred, what's the upside/downsides of the two beyond what I just explained? -- Lasse Vågsæther Karlsen http://usinglvkblog.blogspot.com/ mailto:[EMAIL PROTECTED] PGP KeyID: 0x2A42A1C2 -- http://mail.python.org/mailman/listinfo/python-list
