[Tutor] Speed it up...
Hi,
I have wrote the following lines that work fine, but are quite slow, are
there any obvious things I can consider to speed things up?
Thanks
import MySQLdb
import scipy
import csv
dbtest=MySQLdb.connect(host=***,user=***,passwd=***)
cursor=dbtest.cursor()
cursor.execute(SELECT tfs_name FROM tfs_sites GROUP by tfs_name)
result=cursor.fetchall()
dbtest.close()
TFname=[]
for row in result:
TFname.append(row[0])
del result
T={}
i=0
for TF in TFname:
while i1:
dbtest=MySQLdb.connect(host=***,user=***,passwd=***,db=***)
cursor=dbtest.cursor()
cursor.execute(SELECT tfs_chr,tfs_pos,tfs_val FROM tfs_sites
WHERE tfs_name='%s'%(TF))
result=cursor.fetchall()
TFchr=[]
TFpos=[]
TFval=[]
i+=1
for row in result:
TFchr.append(row[0])
TFpos.append(row[1])
TFval.append(row[2])
TFc=[[],[],[],[],[],[],[],[],[],[],[],[],[],[],[],[],[],[]]
counter=0
for TFsite in TFchr:
TFc[(int(TFsite)-1)].append((int(TFpos[counter]),int(TFval[counter])))
T[TF]=TFc
counter+=1
--
View this message in context:
http://old.nabble.com/Speed-it-up...-tp28691677p28691677.html
Sent from the Python - tutor mailing list archive at Nabble.com.
___
Tutor maillist - Tutor@python.org
To unsubscribe or change subscription options:
http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] import data (txt/csv) into list/array and manipulation
Cool, Does anyone else have any other thoughts on this problem? -- View this message in context: http://www.nabble.com/import-data-%28txt-csv%29-into-list-array-and-manipulation-tp20424075p20623480.html Sent from the Python - tutor mailing list archive at Nabble.com. ___ Tutor maillist - Tutor@python.org http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] import data (txt/csv) into list/array and manipulation
Hi, so for this part of the problem it goes a bit like this: I have a CSV file (file1) that contains three columns, column one contains a unique ID type str, columns two and three contain start and stop coordinates type int. the other file (file2) contains two columns, column one contains a single coordinate type int and the second column contains a value type float. What I would like to do is for example be able to grab the values from file2 that lies within range defind by the start,stop coordinates associated with an ID from file1. But most importantly I would like to be able to grab say the values from file1 that are from range((start-300),start) for every single ID in file1, I guess plot them in an array and then calculate the sum/ of these values and plot them, ie for ob1 in file get values from range((1025-300),1025), for ob2((1090-300),1090) for ob3((2200-300),2200) and then plot/calculate the sum assuming the have the same start coordinate, so x axis would be (step) values from 0-300 and y axis would be the sum of values from ob1,2,3 for every single step value from 0-300. does this make sense/ cheers http://www.nabble.com/file/p20599488/file1.csv file1.csv http://www.nabble.com/file/p20599488/file2.csv file2.csv Kent Johnson wrote: On Thu, Nov 13, 2008 at 9:50 AM, trias [EMAIL PROTECTED] wrote: PS I could maybe upload a couple of small example flies or a schematic to see what I mean A small example would be very helpful. Also please subscribe to the list. Kent ___ Tutor maillist - Tutor@python.org http://mail.python.org/mailman/listinfo/tutor -- View this message in context: http://www.nabble.com/import-data-%28txt-csv%29-into-list-array-and-manipulation-tp20424075p20599488.html Sent from the Python - tutor mailing list archive at Nabble.com. ___ Tutor maillist - Tutor@python.org http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] import data (txt/csv) into list/array and manipulation
Hi again, I got a bit better in python the last few days, but looking at some codes it almost looks impossible to catch up. but definitely want to fight it, looks well worth the effort, plus it probably works exponentially :) I read a little bit about the interval/segment trees, and it looks that their efficiency lies in the efficiency of the algorithms associated with the lookup/indexing modules. Now although I am too newbie to be able to implement the code from the bx-python guys (quicksect.py) understand some basics, disecting a list of objects str,int(start),int(end) on a median basis and store information on the nodes etc Assuming I get this to work some time, and I get back a list of intervals of interest. I would like to use these intervals (str,int,int) to search in a file that contains a fixed step range, where its int in that range is associated with an int(value) (probably best format this file as a dictionary=signaldict) to call all keys within range(interval) and plot values. I think it would be better to print these values in another array, so that I can then say sum the values from all the intervals for each step in the range (assuming I have exported a fixed length of keys from the signaldict) and plot in a graph Well don't mean to have the problem solved for me, but if you fancy to contribute with any kind of help you are welcome cheers PS I could maybe upload a couple of small example flies or a schematic to see what I mean -- View this message in context: http://www.nabble.com/import-data-%28txt-csv%29-into-list-array-and-manipulation-tp20424075p20481629.html Sent from the Python - tutor mailing list archive at Nabble.com. ___ Tutor maillist - Tutor@python.org http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] import data (txt/csv) into list/array and manipulation
Hi all, Thanks so much for the help, I will have a look at the suggestions as well as the other thread,links this week and should post here when I have tried them/need more help. Thanks -- View this message in context: http://www.nabble.com/import-data-%28txt-csv%29-into-list-array-and-manipulation-tp20424075p20435477.html Sent from the Python - tutor mailing list archive at Nabble.com. ___ Tutor maillist - Tutor@python.org http://mail.python.org/mailman/listinfo/tutor
[Tutor] import data (txt/csv) into list/array and manipulation
Hi, I have started learning python (any online help content suggestions are welcome) and want to write a couple of scripts to do simple numeric calculations on array data. filetype(1) I have reference files (ie file.csv) that contain three columns with variable rows, first column is type str contains a unique identifier name, and the other two columns are int type contain two reference values (start,stop(genomic location reference values). **maybe I should import this as dictionary list** filetype(2) The other file contains signal data in three columns, column one is a unique identifier type int, and the other two columns contain two type int values (genomic location reference values) ** import this as array/list I want to map the location of filetype(2) with respect to filetype(1) and be able to do averaging of signal if I align all filetype one objects. Thanks -- View this message in context: http://www.nabble.com/import-data-%28txt-csv%29-into-list-array-and-manipulation-tp20424075p20424075.html Sent from the Python - tutor mailing list archive at Nabble.com. ___ Tutor maillist - Tutor@python.org http://mail.python.org/mailman/listinfo/tutor
