Re: [Tutor] ftplib: retrbinary all files in directory
Israel Carr wrote: > I'm using ftplib to connect to a server, and I want to grab every file > from a specific folder. The filenames change frequently. What is the > best way to use retrbinary with wildcards(or some better option) to > transfer all files? > > Thanks, > Israel > > ___ > Tutor maillist - Tutor@python.org > http://mail.python.org/mailman/listinfo/tutor > You can try ftpmirror.py, which is included with python. From http://docs.python.org/lib/module-ftplib.html: """ The file Tools/scripts/ftpmirror.py in the Python source distribution is a script that can mirror FTP sites, or portions thereof, using the ftplib module. It can be used as an extended example that applies this module. """ Good luck. Rich ___ Tutor maillist - Tutor@python.org http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Class and Scope Question
Tim Johnson wrote: > The following test script is kind of got me baffled: > #!/usr/local/bin/python > class Eval: > def __getitem__(self,key): > return eval(key) > ##def test(): > ## i = 100 > ## b = ["My", "name", "is", "Tim"] > ## test = "this is number %(str(i))s for a test %(' '.join(b))s" > ## s = test % Eval() > ## print s > ##test() > i = 100 > b = ["My", "name", "is", "Tim"] > test = "this is number %(str(i))s for a test %(' '.join(b))s" > print test % Eval() > ## > Running this gives me the following : > [EMAIL PROTECTED] baker]$ python test.py > this is number 100 for a test My name is Tim > ## cool! > Now if I comment out the last four lines and uncomment > the previous 7, I get the following error message: > [EMAIL PROTECTED] baker]$ python test.py > Traceback (most recent call last): > File "test.py", line 11, in ? > test() > File "test.py", line 9, in test > s = test % Eval() > File "test.py", line 4, in __getitem__ > return eval(key) > File "", line 0, in ? > NameError: name 'i' is not defined > ## It's been a llnngg day. What am I missing here? > Explanation and solution is no doubt going to further edify > me about python. > > TIA > tim > Tim, http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/66018 has a more general solution in the comments at the bottom. Basically, you need to put an __init__ in the Eval class, pass locals() and globals() to it, and use those passed-in values in your call to the eval() built-in. I believe the result you see is due to python's lexical scoping rules. Scope is determined by a function- or class-definition's position in the text of the program. def f(): x = 20 def g(): print x g() def h(): x = 50 i() def i(): print x if __name__ == '__main__': f() # works h() # doesn't work Here's another example: g = 'module level' class A: def print_locals_and_globals(self): for k,v in locals().items()+globals().items(): print k,v def func(): # a couple names that won't appear # in a's locals or globals i = 10 s = 'a string' a = A() a.print_locals_and_globals() if __name__ == '__main__': func() So it seems the locals and globals visible to an instance of a class is determined by where the class is defined, not by where the instance is created. HTH, Rich ___ Tutor maillist - Tutor@python.org http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] design questions: pythonic approach to ostriches
Brian van den Broek wrote: Hi all, I am wondering about the Pythonic way to handle the problem of ostriches, emus, and penguins. (I cannot recall from where I got the example.) Here's what I mean: class Bird(object): def fly(self): # flying logic here def lay(self): # egg-laying logic here # more bird methods class Ostrich(Bird): # ostriches can't fly, so what to do? I've explored a number of solutions; here they are with what I see as to cons and problems: The simplest thing is to emulate the ostrich and pretend the problem doesn't exist. But, putting one's head in the sand looks likely to cause troubles in that this route break encapsulation, requiring callers to know enough not to call the fly method of an ostrich. So, that's no good. class Ostrich(Bird): def fly(self): pass seems only marginally better, in that it gives the external appearance of flight, whereas what is needed is a "Hey, I don't fly" signal. The next thought was to over-ride Ostrich.fly as def fly(self): raise NotImplementedError That seems better, but also a bit confusing; the way I understand it, NotImplementedError is, in the first instance, for abstract classes or for marking work in progress. But Ostrich.fly doesn't fit either case. That makes me think to define a custom exception, say class OstrichError(NotImplementedError): '''A custom exception for cases of the "Ostrich problem". Intended to be raised by methods in a subclass over-riding methods of the parent which don't make sense for the subclass to actually implement.''' def __init__(self): NotImplementedError.__init__(self) But, since the problem isn't one that my penetrating genius discovered, I am inclined to think that were this the ideal solution, there'd be a (better named) exception class builtin to Python already. A complicated class hierarchy like class Bird(object): # bird logic class FlyingBird(Bird): def fly(self): # flying logic here class FlightlessBird(Bird): # any particularly flightless logic here class Ostrich(FlightlessBird): # ostrich logic seems an invitation to difficulty. My analogy will soon break, but some birds build nests and sing, others, no so much, etc. Flat is soon to give way to deeply nested. I also tried to delete the inherited Bird.fly method within Ostrich.__init__, but class Ostrich(Bird): def __init__(self): del self.__dict__['fly'] raises a KeyError, whereas def __init__(self): del Ostrich.__dict__['fly'] raises: TypeError: object does not support item deletion Do I have the syntax of the last approach wrong? Or is there no way to remove a method from a class? If the latter, what to do about flightless fowl? Thanks and best, Brian vdB Brian, I've seen the strategy pattern used in this kind of scenario. Your strategies can be encapulated classes, and methods within the Bird classes delegate to the strategy classes - so you achieve desired behavior though composition, rather than inheritance. Here's a quick example (which doesn't really address any of your questions and probably has its own problems): class CantFly(object): def fly(self): print "I can't fly" class CanFly(object): def fly(self): print "I can fly" class Bird(object): def __init__(self,flightBehavior): self.flightBehavior = flightBehavior def fly(self): self.flightBehavior.fly() b1 = Bird(CantFly()) b2 = Bird(CanFly()) for b in (b1,b2): b.fly() Good luck. Rich ___ Tutor maillist - Tutor@python.org http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] iterator question for a toy class
Kent Johnson wrote: Rich Krauter wrote: 2) Or, if you really want your __eq__ method to use the iterator returned by __iter__(), def __eq__(self, other): for i, j in map(None,self, other): if i != j: return False return True That's not right either, it will compare Foo([None], []) == Foo([], []) for example. Kent, Yikes, I should have seen that. Thanks for pointing out the error. Rich ___ Tutor maillist - Tutor@python.org http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] iterator question for a toy class
Marcus Goldfish wrote: I'm trying to understand custom iterators in Python, and created the following toy class for sequence comparison, which seems to work: class Foo(object): """A toy class to experiment with __eq__ and __iter__""" def __init__(self, listA, listB): self.head, self.tail = listA, listB def __iter__(self): return iter(self.head[:]+self.tail[:]) def __eq__(self, other): """Foo instances are equal if their respective subsequences, head, tail, are in the same order""" diff = [i for i, j in zip(self, other) if i != j] return len(diff) == 0 f1 = Foo( [1,2], ['a','b','c'] ) f2 = Foo( [1,2], ['a','b','c'] ) f3 = Foo( [1,2], ['a','b','d'] ) f1 == f2, f1 == f3 (True, False) I'm not really sure if I'm implementing iter() correctly, for instance: should I make copies of the sublists? Should I try to implement my own next() method? Advice, comments, and links to good tutorial web pages on iterators are appreciated! Hi Marcus, Here are some points I noticed - 1) I would probably change your __eq__ method to something like this: def __eq__(self,other): return (self.head,self.tail) == (other.head,other.tail) 2) Or, if you really want your __eq__ method to use the iterator returned by __iter__(), def __eq__(self, other): for i, j in map(None,self, other): if i != j: return False return True One reason this might be a little better than your version is that using zip() as you have, f1 = Foo( [1,2], ['a','b','c']) and f2 = Foo( [1,2], ['a','b','c','q'] ) would compare equal. Another reason is that this version returns False as soon as it figures out the lists are not equal. 3) It's not a big deal, but instead of what you have in __iter__(), I might import itertools and then write something like def __iter__(self): return itertools.chain(self.head,self.tail) 4) In this specific case, if you want to use the iterator returned by __iter__ in your __eq__ method, I think you should avoid using a next(). Here's a silly example why: class Foo2(object): def __init__(self, listA,listB): self.head,self.tail = listA,listB self._current = -1 self._cache = None def _fill_cache(self): """save some state""" if not self._cache: self._cache = self.head + self.tail def __iter__(self): """the iterator is the iterable object itself""" return self def next(self): """make the object an iterator""" self._fill_cache() self._current += 1 if self._current >= len(self._cache): raise StopIteration return self._cache[self._current] def __eq__(self, other): for i, j in map(None,self, other): if i != j: return False return True if __name__ == '__main__': f1 = Foo2( [1,2], ['a','b','c']) f2 = Foo2( [1,2], ['a','b','c']) f3 = Foo2( [1,2], ['a','b','d']) print f1 == f2, f1 == f3, f1 == f3 # notice the output here Good luck. Rich ___ Tutor maillist - Tutor@python.org http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] odd behavior within __init__
Orri Ganel wrote: Hello all, As part of a project i'm doing (mostly for the fun of it), I have a class which creates a sort of wrapper around any object to make it suitable for use in a custom container. However, if the class receives an already wrapped object, I want it to just return the object (same id and everything as the original). Now, the following seems to work in the __init__ method (due to output), but then it disappears as soon as the __init__ method is left: class Node: ... def __init__(self, cargo=None, prev=None, next=None, nod=False): """x.__init__(...) initializes x; see x.__class__.__doc__ for signature""" if not isinstance(cargo, Node) or nod: self.cargo = cargo self.prev = prev self.next = next else: self = cargo print id(self), id(cargo) print self.cargo a = Node(1) b = Node(a) 12932600 12932600 1 id(b) 12960632 Any ideas on why this happens, or suggestions as to how to implement the behavior I'm looking for (in which b and a would refer to the same object, have the same id, etc.), would be greatly appreciated. Thanks in advance, Orri Orri, Maybe you could use a factory. It would allow you to simplify your Node class, and encapsulate the instantiation behavior you want outside the class. class Node(object): def __init__(self,cargo=None,prev=None,next=None): self.cargo = cargo self.prev = prev self.next = next def factory(cargo=None,prev=None,next=None): if isinstance(cargo,Node): return cargo else: return Node(cargo,prev,next) n1 = factory(1) print id(n1) n2 = factory(n1) print id(n2) Good luck, Rich ___ Tutor maillist - Tutor@python.org http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] problems with doctest: apparent interferance between tests (LONG)
Brian van den Broek wrote: [text and code snipped] FWIW, I ran your code using python 2.3.4 on linux with the following modifications, and it worked fine (no failing tests out of 9): 1- added 'from sets import Set as set' since set() isn't built in in 2.3 2- changed True back to False in the following test: '''Creates a new _Check_point instance; appends it to .data. >>> # print "Testing check_point (intentional fail)." >>> print Wall_clock.is_instanced True 3- omitted the optionflags=doctest.ELLIPSIS argument from the following code, since it's not available in python 2.3's doctest: def _test(): import doctest, sys doctest.testmod(sys.modules[__name__], report = True, optionflags = doctest.ELLIPSIS) Running the same code with 2.4 gave 3 failures out of 9 tests. Rich ___ Tutor maillist - Tutor@python.org http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] dictionary dispatch for object instance attributes question
Brian van den Broek wrote: [snip text] class A: def __init__(self): self.something = None self.something_else = None self.still_another_thing = None def update(self, data): for key in metadata_dict: if data.startswith(key): exec('''self.%s = """%s"""''' %(metadata_dict[key], data[len(key):])) # triple quotes as there may be quotes in metadata # values break [snip usage example and additional text] Brian, You could use setattr(self,metadata_dict[key],data[len(key):]). Rich ___ Tutor maillist - Tutor@python.org http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] regex problem
Michael Powe wrote: Hello, I'm having erratic results with a regex. I'm hoping someone can pinpoint the problem. This function removes HTML formatting codes from a text email that is poorly exported -- it is supposed to be a text version of an HTML mailing, but it's basically just a text version of the HTML page. I'm not after anything elaborate, but it has gotten to be a bit of an itch. ;-) def parseFile(inFile) : import re bSpace = re.compile("^ ") multiSpace = re.compile(r"\s\s+") nbsp = re.compile(r" ") HTMLRegEx = re.compile(r"(<|<)/?((!--.*--)|(STYLE.*STYLE)|(P|BR|b|STRONG))/?(>|>) ",re.I) f = open(inFile,"r") lines = f.readlines() newLines = [] for line in lines : line = HTMLRegEx.sub(' ',line) line = bSpace.sub('',line) line = nbsp.sub(' ',line) line = multiSpace.sub(' ',line) newLines.append(line) f.close() return newLines Now, the main issue I'm looking at is with the multiSpace regex. When applied, this removes some blank lines but not others. I don't want it to remove any blank lines, just contiguous multiple spaces in a line. Hi Michael, If you use '\s\s+', and a line has ' \n' (space then newline) at the end, the space and the newline will match and be substituted. If the line ends in 'some chars\n' or the line is just '\n', the newline will stay. An alternate approach might be to first get rid of any leading or trailing whitespace (including \r|\n), then get rid of 'internal' repeated space, with string methods. Parsing html using regexes is likely to break easily; HTMLParser is a better solution, but may it may seem more complicated at first. It may be worthwhile for you to look into that module; someone here would be able to help if necessary. Short of the HTMLParser approach, I would try to reduce the dependence on regexes, using string methods where you can. I would try something like this (untested) in your for loop to start: for line in lines: line = line.strip() line = line.replace(' ',' ') line = HTMLRegEx.sub(' ',line) line = ' '.join(line.split()) newLines.append(line) The only pattern left is HTMLRegEx. Using HTMLParser you could probably remove regexes completely. When I used perl for most things, I don't think I wrote a single script that didn't use regexes. Now, I use python for most things, and I don't think I have a single python module that imports re. Weird. BTB, this also illustrates a difference between python and perl -- in perl, i can change "line" and it automatically changes the entry in the array; this doesn't work in python. A bit annoying, actually. ;-) I once had the same trouble with python - some features annoyed me because they weren't like perl. Now I have the reverse problem, only this time my annoyance is justified. :) Thanks for any help. If there's a better way to do this, I'm open to suggestions on that regard, too. Good luck. Rich ___ Tutor maillist - Tutor@python.org http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] CGI help
David Holland wrote: Rich, That worked although I had to set the address to :- http://127.0.0.1:8000/friends1.html Thanks. Good catch, sorry about that. I noticed you had spaces in the "how many" field in your form. Did your cgi still work? If not, I guess you figured that out by now too. Interestingly, I had trouble getting the server/cgi to work with python 2.4/Windows XP. If I started the server with python 2.4 and put 2.4 in the cgi's #! line, the cgi didn't work. If I used 2.3 to start the server, and put 2.3 or 2.4 in the cgi's #! line, the cgi worked. I tried the same combinations on linux, and they all worked fine. Anyone else seen this problem with python 2.4 on windows xp? I looked quickly at diffs of the relevant codes between 2.3 and 2.4, and I did a quick bug search on sourceforge, but didn't turn up anything yet. Of course it may not be a bug; it may be a problem with my setup. The files I used are posted below, in case anyone wants to try to reproduce the result I saw. Thanks. Rich <> import CGIHTTPServer import BaseHTTPServer def run(server_class=BaseHTTPServer.HTTPServer, handler_class=CGIHTTPServer.CGIHTTPRequestHandler): server_address = ('', 8000) httpd = server_class(server_address, handler_class) httpd.serve_forever() run() <> <> #!C:/Python24/python.exe print "Content-Type: text/html" print import cgi,cgitb cgitb.enable() reshtml = ''' CGI Demo Friends list for:%s Your name is: %s You have %s friends. ''' form = cgi.FieldStorage() who = form.getfirst('person') howmany = form.getfirst('howmany') print reshtml % (who, who, howmany) <> <> CGI Demo Friends list for New User Enter your name How many friends do you have? 0 10 25 50 100> <> ___ Tutor maillist - Tutor@python.org http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] CGI help
David Holland wrote: I have written my first cgi script :- The .html code is :- Friends CGI Demo (Static Screen) Friends list for:New User Enter your name> How many friends do you have ? 0 10 25 50 100 The python code is :- #!/usr/bin/env python import cgi reshtml = '''Content-Type: text/html\n Friends CGI Demo (dynamic screen) Friends list for: %s Your name is: %s You have %s friends. ''' form = cgi.FieldStorage() who = form['person'].value howmany = form['howmany'].value print reshtml % (who, who, howmany) Now when I open the page using konqueror and click on submit. Instead of the code running, it justs opens the .py program in a different window. Any ideas ? Hi David, You can use a small python script to provide a cgi web server for testing: import BaseHTTPServer import CGIHTTPServer def run(server_class=BaseHTTPServer.HTTPServer, handler_class=CGIHTTPServer.CGIHTTPRequestHandler): server_address = ('', 8000) httpd = server_class(server_address, handler_class) httpd.serve_forever() run() Put this in a file called, say, server.py. In the same directory as server.py, put a directory called cgi-bin. Put your python cgi scripts in cgi-bin. Modify the action tag in your html form to point to /cgi-bin/friends1.py. Click on the server.py icon, or run it from the command line. Finally, point your browser to http://127.0.0.1:8000/cgi-bin/friends1.py. You'll proably want to use apache or some other server if you want to do more than just test or demo cgi scripts locally, but this can be a useful alternative for testing simple cgi programs without installing additional software on your machine. Good luck. Rich ___ Tutor maillist - Tutor@python.org http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Basic question -> How to use a class from a file?
Bernard Lebel wrote: Okay here comes the next question. In that class, I have 3 functions. Actually, one of them calls the two others. However when I call these functions (wich are placed before the caller in the file), I get an error saying that the global name x (the function name) is not defined. What am I doing wrong? Hi Bernard, This is just a guess since you didn't post your code, but did you call the 'bare' methods in your class definition, rather than calling the methods through an instance (which is usually called 'self' inside class definitions)? I think the following may be similar to what you wanted to do: class A(object): def __init__(self,data=None): self.data = data def methodA(self): self.methodB() self.methodC() def methodB(self): print "in methodB" print self.data def methodC(self): print "in methodC" print self.data if __name__ == '__main__': a1 = A(5) a1.methodA() a2 = A([1,2,3]) a2.methodA() Hope this helps. Rich ___ Tutor maillist - Tutor@python.org http://mail.python.org/mailman/listinfo/tutor
RE: [Tutor] How to substitute an element of a list as a pattern for re.compile()
kumar s wrote: I have Question: How can I substitute an object as a pattern in making a pattern. x = 30 pattern = re.compile(x) Kumar, You can use string interpolation to insert x into a string, which can then be compiled into a pattern: x = 30 pat = re.compile('%s'%x) I really doubt regular expressions will speed up your current searching algorithm. You probably need to reconsider the data structures you are using to represent your data. I have a list of numbers that I have to match in another list and write them to a new file: List 1: range_cors range_cors[1:5] ['161:378', '334:3', '334:4', '65:436'] List 2: seq seq[0:2] ['>probe:HG-U133A_2:1007_s_at:416:177; Interrogation_Position=3330; Antisense;', 'CACCCAGCTGGTCCTGTGGATGGGA'] Can you re-process your second list? One option might be to store that list instead as a dict, where the keys are what you want to search by (maybe a string like '12:34' or a tuple like (12,34)). Maybe something like the following: >>> range_cors = ['12:34','34:56'] >>> seq = {'12:34': ['some 12:34 data'], ...'34:56': ['some 34:56'data','more 34:56 data']} >>> for item in range_cors: ... print seq[item] ... ['some 12:34 data'] ['some 34:56 data','more 34:56 data'] Why is this better? If you have m lines of data and n patterns to search for, then using either of your methods you perform n searches per line, totalling approx. m*n operations. You have to complete approx. m*n operations whether you use the string searching version, or re searching version. If you pre-process the data so that it can be stored in and retrieved from a dict, pre-processing to get your data into that dict costs you roughly m operations, but your n pattern lookups into that dict cost you only n operations, so you only have to complete approx. m+n operations. A slow method: sequences = [] for elem1 in range_cors: for index,elem2 in enumerate(seq): if elem1 in elem2: sequences.append(elem2) sequences.append(seq[index+1]) A faster method (probably): for i in range(len(range_cors)): for index,m in enumerate(seq): pat = re.compile(i) if re.search(pat,seq[m]): p.append(seq[m]) p.append(seq[index+1]) I am getting errors, because I am trying to create an element as a pattern in re.compile(). pat = re.compile('%s'%i) would probably get rid of the error message, but that's probably still not what you want. Questions: 1. Is it possible to do this. If so, how can I do this. You can try, but I doubt regular expressions will help; that approach will probably be even slower. Can any one help correcting my piece of code and suggesting where I went wrong. I would scrap what you have and try using a better data structure. I don't know enough about your data to make more specific processing recommendations; but you can probably avoid those nested loops with some careful data pre-processing. You'll likely get better suggestions if you post a more representative sample of your data, and explain exactly what you want as output. Good luck. Rich ___ Tutor maillist - Tutor@python.org http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] sorting and editing large data files
Scott Melnyk wrote: Hello! I recently suffered a loss of programming files (and I had been putting off my backups...) [snip] #regular expression to pull out gene, transcript and exon ids info=re.compile('^(ENSG\d+\.\d).+(ENST\d+\.\d).+(ENSE\d+\.\d)+') #above is match gene, transcript, then one or more exons #TFILE = open(sys.argv[1], 'r' )#read the various transcripts from WFILE=open(sys.argv[1], 'w')# file to write 2 careful with 'w' will overwrite old info in file W2FILE=open(sys.argv[2], 'w') #this file will have the names of redundant exons import sets def getintersections(fname='Z:\datasets\h35GroupedDec15b.txt'): exonSets = {} f = open(fname) for line in f: if line.startswith('ENS'): parts = line.split() gene = parts[0] transcript = parts[1] exons = parts[2:] exonSets.setdefault(gene, sets.Set(exons)).intersection(sets.Set(exons)) ^^ return exonSets Hi Scott, There may be other problems, but here's one thing I noticed: exonSets.setdefault(gene, sets.Set(exons)).intersection(sets.Set(exons)) should be exonSets.setdefault(gene, sets.Set(exons)).intersection_update(sets.Set(exons)) Hope that helps. Rich ___ Tutor maillist - [EMAIL PROTECTED] http://mail.python.org/mailman/listinfo/tutor