Re: creating pipelines in python

2009-11-25 Thread per
there other candidates? thank you. On Nov 23, 4:02 am, Paul Rudin wrote: > per writes: > > hi all, > > > i am looking for a python package to make it easier to create a > > "pipeline" of scripts (all in python). what i do right now is have a > > set of

creating pipelines in python

2009-11-22 Thread per
hi all, i am looking for a python package to make it easier to create a "pipeline" of scripts (all in python). what i do right now is have a set of scripts that produce certain files as output, and i simply have a "master" script that checks at each stage whether the output of the previous script

Re: efficiently splitting up strings based on substrings

2009-09-05 Thread per
On Sep 5, 7:07 pm, "Rhodri James" wrote: > On Sat, 05 Sep 2009 23:54:08 +0100, per wrote: > > On Sep 5, 6:42 pm, "Rhodri James" wrote: > >> On Sat, 05 Sep 2009 22:54:41 +0100, per wrote: > >> > I'm trying to efficiently "split&

Re: efficiently splitting up strings based on substrings

2009-09-05 Thread per
On Sep 5, 6:42 pm, "Rhodri James" wrote: > On Sat, 05 Sep 2009 22:54:41 +0100, per wrote: > > I'm trying to efficiently "split" strings based on what substrings > > they are made up of. > > i have a set of strings that are comprised of know

efficiently splitting up strings based on substrings

2009-09-05 Thread per
I'm trying to efficiently "split" strings based on what substrings they are made up of. i have a set of strings that are comprised of known substrings. For example, a, b, and c are substrings that are not identical to each other, e.g.: a = "0" * 5 b = "1" * 5 c = "2" * 5 Then my_string might be:

allowing output of code that is unittested?

2009-07-15 Thread per
hi all, i am using the standard unittest module to unit test my code. my code contains several print statements which i noticed are repressed when i call my unit tests using: if __name__ == '__main__': suite = unittest.TestLoader().loadTestsFromTestCase(TestMyCode) unittest.TextTestRunner

Re: fastest native python database?

2009-06-17 Thread per
est option. thanks for the suggestion, will look into gadfly in the meantime. On Jun 17, 11:38 pm, Emile van Sebille wrote: > On 6/17/2009 8:28 PM per said... > > > hi all, > > > i'm looking for a native python package to run a very simple data > > base. i was orig

fastest native python database?

2009-06-17 Thread per
hi all, i'm looking for a native python package to run a very simple data base. i was originally using cpickle with dictionaries for my problem, but i was making dictionaries out of very large text files (around 1000MB in size) and pickling was simply too slow. i am not looking for fancy SQL oper

Re: generating random tuples in python

2009-04-20 Thread per
On Apr 20, 11:08 pm, Steven D'Aprano wrote: > On Mon, 20 Apr 2009 11:39:35 -0700, per wrote: > > hi all, > > > i am generating a list of random tuples of numbers between 0 and 1 using > > the rand() function, as follows: > > > for i in range(0, n): &g

generating random tuples in python

2009-04-20 Thread per
hi all, i am generating a list of random tuples of numbers between 0 and 1 using the rand() function, as follows: for i in range(0, n): rand_tuple = (rand(), rand(), rand()) mylist.append(rand_tuple) when i generate this list, some of the random tuples might be very close to each other, nume

Re: splitting a large dictionary into smaller ones

2009-03-22 Thread per
On Mar 22, 10:51 pm, Paul Rubin <http://phr...@nospam.invalid> wrote: > per writes: > > i would like to split the dictionary into smaller ones, containing > > only hundreds of thousands of keys, and then try to pickle them. > > That already sounds like the wrong app

splitting a large dictionary into smaller ones

2009-03-22 Thread per
hi all, i have a very large dictionary object that is built from a text file that is about 800 MB -- it contains several million keys. ideally i would like to pickle this object so that i wouldnt have to parse this large file to compute the dictionary every time i run my program. however currentl

loading program's global variables in ipython

2009-03-22 Thread per
hi all, i have a file that declares some global variables, e.g. myglobal1 = 'string' myglobal2 = 5 and then some functions. i run it using ipython as follows: [1] %run myfile.py i notice then that myglobal1 and myglobal2 are not imported into python's interactive namespace. i'd like them too -

parsing tab separated data efficiently into numpy/pylab arrays

2009-03-13 Thread per
hi all, what's the most efficient / preferred python way of parsing tab separated data into arrays? for example if i have a file containing two columns one corresponding to names the other numbers: col1\t col 2 joe\t 12.3 jane \t 155.0 i'd like to parse into an array() such that i

speeding up reading files (possibly with cython)

2009-03-07 Thread per
hi all, i have a program that essentially loops through a textfile file thats about 800 MB in size containing tab separated data... my program parses this file and stores its fields in a dictionary of lists. for line in file: split_values = line.strip().split('\t') # do stuff with split_value

Re: setting PYTHONPATH to override system wide site-packages

2009-02-28 Thread per
On Feb 28, 11:53 pm, per wrote: > On Feb 28, 11:24 pm, Carl Banks wrote: > > > > > On Feb 28, 7:30 pm, per wrote: > > > > hi all, > > > > i recently installed a new version of a package using python setup.py > > > install --prefix=/my/homed

Re: setting PYTHONPATH to override system wide site-packages

2009-02-28 Thread per
On Feb 28, 11:24 pm, Carl Banks wrote: > On Feb 28, 7:30 pm, per wrote: > > > hi all, > > > i recently installed a new version of a package using python setup.py > > install --prefix=/my/homedir on a system where i don't have root > > access. the old packa

setting PYTHONPATH to override system wide site-packages

2009-02-28 Thread per
hi all, i recently installed a new version of a package using python setup.py install --prefix=/my/homedir on a system where i don't have root access. the old package still resides in /usr/lib/python2.5/site- packages/ and i cannot erase it. i set my python path as follows in ~/.cshrc setenv PYT

Re: optimizing large dictionaries

2009-01-15 Thread Per Freem
thanks to everyone for the excellent suggestions. a few follow up q's: 1] is Try-Except really slower? my dict actually has two layers, so my_dict[aKey][bKeys]. the aKeys are very small (less than 100) where as the bKeys are the ones that are in the millions. so in that case, doing a Try-Except o

optimizing large dictionaries

2009-01-15 Thread Per Freem
hello i have an optimization questions about python. i am iterating through a file and counting the number of repeated elements. the file has on the order of tens of millions elements... i create a dictionary that maps elements of the file that i want to count to their number of occurs. so i iter

Re: efficient interval containment lookup

2009-01-12 Thread Per Freem
cy so much... thanks. On Jan 13, 12:24 am, brent wrote: > On Jan 12, 8:55 pm, Per Freem wrote: > > > > > On Jan 12, 10:58 pm, Steven D'Aprano > > > wrote: > > > On Mon, 12 Jan 2009 14:49:43 -0800, Per Freem wrote: > > > > thanks for your repli

Re: efficient interval containment lookup

2009-01-12 Thread Per Freem
i forgot to add, my naive_find is: def naive_find(intervals, start, stop): results = [] for interval in intervals: if interval.start >= start and interval.stop <= stop: results.append(interval) return results On Jan 12, 11:55 pm, Per Freem wrote: > On Jan 12, 10:58 p

Re: efficient interval containment lookup

2009-01-12 Thread Per Freem
On Jan 12, 10:58 pm, Steven D'Aprano wrote: > On Mon, 12 Jan 2009 14:49:43 -0800, Per Freem wrote: > > thanks for your replies -- a few clarifications and questions. the > > is_within operation is containment, i.e. (a,b) is within (c,d) iff a > >>= c and b <= d

Re: efficient interval containment lookup

2009-01-12 Thread Per Freem
thanks for your replies -- a few clarifications and questions. the is_within operation is containment, i.e. (a,b) is within (c,d) iff a >= c and b <= d. Note that I am not looking for intervals that overlap... this is why interval trees seem to me to not be relevant, as the overlapping interval pro

efficient interval containment lookup

2009-01-12 Thread Per Freem
hello, suppose I have two lists of intervals, one significantly larger than the other. For example listA = [(10, 30), (5, 25), (100, 200), ...] might contain thousands of elements while listB (of the same form) might contain hundreds of thousands or millions of elements. I want to count how many i

RE: listdir reports [Error 1006] The volume for a file has been externally altered so that the opened file is no longer valid

2009-01-08 Thread Per Olav Kroka
FYI: the '/*.*' is part of the error message returned. -Original Message- From: ch...@rebertia.com [mailto:ch...@rebertia.com] On Behalf Of Chris Rebert Sent: Wednesday, January 07, 2009 6:40 PM To: Per Olav Kroka Cc: python-list@python.org Subject: Re: listdir reports [Error

Re: Program eating memory, but only on one machine? (Solved, sort of)

2007-01-22 Thread Per B.Sederberg
Per B.Sederberg princeton.edu> writes: > I'll see if I can make a really small example program that eats up memory on > our cluster. That way we'll have something easy to work with. Now this is weird. I figured out the bug and it turned out that every time you call numpy

Re: Program eating memory, but only on one machine?

2007-01-22 Thread Per B.Sederberg
eally small example program that eats up memory on our cluster. That way we'll have something easy to work with. Thanks, Per -- http://mail.python.org/mailman/listinfo/python-list

Program eating memory, but only on one machine?

2007-01-22 Thread Per B. Sederberg
ifference I can see is that our cluster is stuck on a really old version of gcc with the RedHat Enterprise that's on there, but I found no suggestions of memory issues online. So, does anyone have any suggestions for how I can debug this problem? If my program ate up memory on all machines, then I would know where to start and would blame some horrible programming on my end. This just seems like a less straightforward problem. Thanks for any help, Per -- http://mail.python.org/mailman/listinfo/python-list

Re: (question) How to use python get access to google search without query quota limit

2006-05-05 Thread Per
Yeah, Thanks Am, I can be considered as an advanced google user, presumably.. But I am not a advanced programmer yet. If everyone can generate unlimited number of queries, soon the user-query-data, which I believe is google's most advantage, will be in chaos. Can they simply ignore some queries f

(question) How to use python get access to google search without query quota limit

2006-05-05 Thread Per
I am doing a Natural Language processing project for academic use, I think google's rich retrieval information and query-segment might be of help, I downloaded google api, but there is query limit(1000/day), How can I write python code to simulate the browser-like-activity to submit more than 10k

Re: Is there such an idiom?

2006-03-19 Thread Per
Thanks Ron, surely set is the simplest way to understand the question, to see whether there is a non-empty intersection. But I did the following thing in a silly way, still not sure whether it is going to be linear time. def foo(): l = [...] s = [...] dic = {} for i in l: d

Is there such an idiom?

2006-03-19 Thread Per
http://jaynes.colorado.edu/PythonIdioms.html """Use dictionaries for searching, not lists. To find items in common between two lists, make the first into a dictionary and then look for items in the second in it. Searching a list for an item is linear-time, while searching a dict for an item is con

serial port server cnhd38

2005-02-22 Thread per . bergstrom
ort port 20: this port goes to cnhd47/console port port 21: this port goes to cnhd46/console port To connect to a port, just enter the following command: telnet msp-t01 ... an extra should give you the prompt. is always 20 is the port number... example, connect to cnhd47/console port: telne

test

2004-12-23 Thread Per Erik Stendahl
sdfdsafasd -- http://mail.python.org/mailman/listinfo/python-list