Re: [Tutor] Downloading data from web
On 10/4/2010 7:01 PM Crusier said... I am trying to extract web data and put it into a database for analysis.I am just wondering what is the best way to do it and I will try to dig up information from there. Please help me how can I kick start this project. Check out beautiful soup. http://www.crummy.com/software/BeautifulSoup/ Emile ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Matching relational data
On Mon, Oct 4, 2010 at 6:31 PM, Steven D'Aprano wrote: > On Mon, 4 Oct 2010 10:52:02 am Alan Gauld wrote: > >> One of the best programming stats tools is R >> >> http://en.wikipedia.org/wiki/R_%28programming_language%29 >> >> There is a python bionding for R too. >> >> Although R may be overkill for what you want, but at least >> you'll know the theory and math are correct! > > Or you could use numpy and scipy, which are rapidly becoming the choice > for numeric and scientific applications over R. > > > -- > Steven D'Aprano > ___ > Tutor maillist - tu...@python.org > To unsubscribe or change subscription options: > http://mail.python.org/mailman/listinfo/tutor > I'm amazed that you didn't catch the fact that it doesn't report that above fifty is a type of correlation and below is a match of correlation as well. Divergence is a pattern as well. Golly why didn't you pick up on that buddy, pal? Although I like the improved you gave, would you agree that readability would be better inclined to additions due to readability at an elementary level of python, but something a statistician could add on too, with limited python experience, but still contribute to the code? Seriously OT though, I ask in the same respect that i want the 'purer' aspect to review, but deny it in the aspect that it does hinder the open source mentality(from what I can see): but here's a more refined correlation pattern to review, in the thought process tha I can make million dollar oftware in the privacy of my home, and more importantly by 'myself'(with a little help from my friends) 50 % < begins to match mergence 50% > begins to match divergence 0 = matches perfect divergence 100 = matches perfect mergence ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: http://mail.python.org/mailman/listinfo/tutor
[Tutor] Downloading data from web
I am trying to extract web data and put it into a database for analysis.I am just wondering what is the best way to do it and I will try to dig up information from there. Please help me how can I kick start this project. Cheers, Hank ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] subprocess.call warning
On Mon, 4 Oct 2010 05:42:16 pm Norman Khine wrote: > >> the first calls the 'sox' library which joins all the .wav files > >> into one file and then i use the 'wav2swf' library to create a SWF > >> output of the file. > >> > >> can the code be improved? > > > > Code can always be improved :-) What do you consider an > > improvement? Easy to maintain in the future? Faster? Smaller? > > to make it faster. Faster??? Given that you are manipulating multiple wav files and joining them together into a swf file, I can't believe that you seriously believe that the Python code is likely to be the bottleneck. If it takes ten seconds to process the wav files into a swf file, who cares if you speed the Python code up from 0.02 seconds to 0.01 seconds? But I could be wrong... I look forward to seeing the profiling results from your code. -- Steven D'Aprano ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Matching relational data
On Mon, 4 Oct 2010 10:52:02 am Alan Gauld wrote: > One of the best programming stats tools is R > > http://en.wikipedia.org/wiki/R_%28programming_language%29 > > There is a python bionding for R too. > > Although R may be overkill for what you want, but at least > you'll know the theory and math are correct! Or you could use numpy and scipy, which are rapidly becoming the choice for numeric and scientific applications over R. -- Steven D'Aprano ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] EXECUTING PYTHON AND SQL STAMENTS
On Mon, Oct 4, 2010 at 9:04 AM, Susana Iraiis Delgado Rodriguez < susana.delgad...@utzmg.edu.mx> wrote: > I'm developing a module to execute an external command. The module executes > the command, but in order to make my code useful I have to enter some sql > staments. This is my code: > Question, what database system are you trying to access? Python, via its standard library, and also by easily available 3rd party libraries. has ample facilities to allow for database access without having to do external OS calls. You may have a perfectly valid reason for designing your code the way you have, but I thought this was worth asking. --Bill > > ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Matching relational data
On Tue, 5 Oct 2010 02:04:09 am David Hutto wrote:
> a = ['+','-','+','-','+','-','+','-','+','-','+']
> b = ['-','+','-','+','-','-','-','+','-','-','-']
>
> count = 0
> lena = len(a)
> lenb = len(b)
> if lena == lenb:
> for num in range(0,lena):
> if a[num] == b[num]:
> print 'match'
> count += 1
> else:
> print 'Lists are not same length for comparison'
> per = (100/lena)
> print count * per, '% match'
a = '+-+-+-+-+-+'
b = '-+-+---+---'
if len(a) != len(b):
raise ValueError('lists are not the same length')
count = sum(ca == cb for (ca, cb) in zip(a, b))
--
Steven D'Aprano
___
Tutor maillist - Tutor@python.org
To unsubscribe or change subscription options:
http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Matching relational data
Thanks for your replies, and reminding me of statistics(probably a quick crash course). The data I plan to use would be equally charted by a constant timestep in it's movement(type of movement graphed being dictated by the object being measured) So I thought that maybe the easiest way would be something like the following: a = ['+','-','+','-','+','-','+','-','+','-','+'] b = ['-','+','-','+','-','-','-','+','-','-','-'] count = 0 lena = len(a) lenb = len(b) if lena == lenb: for num in range(0,lena): if a[num] == b[num]: print 'match' count += 1 else: print 'Lists are not same length for comparison' per = (100/lena) print count * per, '% match' The plus and minus signs would represent the movement over an equal amount of time(say one minute/hour) Although I'm going to the statistics for the rest of this, anyone who has an improved solution let me know. Thanks, David ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] EXECUTING PYTHON AND SQL STAMENTS
On 04-Oct-10 07:32, Norman Khine wrote:
On Mon, Oct 4, 2010 at 4:04 PM, Susana Iraiis Delgado Rodriguez
wrote:
os.system('"C:/Archivos de programa/FWTools2.4.7/bin/ogr2ogr.exe"'+" "
+"arg1" +" "+ "-where" +" "+ "arg2" +" " +"arg3")
You're including the text "arg1" in the command, not the value of the
arg1 variable. (Likewise for arg2 and arg3). There are some
unnecessary extra strings in there too. Your os.system() call, slightly
simplified, is:
os.system('"C:/Archivos de programa/FWTools2.4.7/bin/ogr2ogr.exe
"+"arg1" +" -where "+"arg2" +" " +"arg3")
but what I think you meant was
os.system('"C:/Archivos de programa/FWTools2.4.7/bin/ogr2ogr.exe "+arg1
+" -where "+arg2 +" " +arg3)
There are even better ways to accomplish this task too, but I'm focusing
on what looks like you ran into with this first.
___
Tutor maillist - Tutor@python.org
To unsubscribe or change subscription options:
http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] EXECUTING PYTHON AND SQL STAMENTS
On Mon, Oct 4, 2010 at 4:04 PM, Susana Iraiis Delgado Rodriguez
wrote:
> I'm developing a module to execute an external command. The module executes
> the command, but in order to make my code useful I have to enter some sql
> staments. This is my code:
> from dbf import *
> from osgeo import ogr
> import os
> import sys
> def call():
> print "Ingresa el nombre para el nuevo mapa"
> arg1 = "R1G-GEODESIA2.shp"
> print arg1
> print "Ingresa la condicion"
> arg2 = "LAYER = 'R1G-GEODESIA'"
> print arg2
> print "Ingresa el nombre del mapa original"
> arg3 = 'C:/Python26/tapalpa_05_plani_point.shp'
> print arg3
> os.system('"C:/Archivos de programa/FWTools2.4.7/setfw"')
> os.system('"C:/Archivos de programa/FWTools2.4.7/bin/ogr2ogr.exe"'+" "
> +"arg1" +" "+ "-where" +" "+ "arg2" +" " +"arg3")
> call()
> The problem is that when I run the module it throws the error:
> Unable to open datasource`arg3' with the following drivers.
maybe the path to arg3 is incorrect? did arg1 shapefile load without a problem?
> ESRI Shapefile
> MapInfo File
> UK .NTFSDTS
> TIGER
> S57
> DGN
> VRT
> REC
> Memory
> BNA
> CSV
> NAS
> GML
> GPX
> KML
> GeoJSON
> Interlis 1
> Interlis 2
> GMT
> SQLite
> ODBC
> PGeo
> OGDI
> PostgreSQL
> MySQL
> XPlane
> AVCBin
> AVCE00
> DXF
> Geoconcept
> GeoRSS
> GPSTrackMaker
> VFK
> Can you help me please?
> ___
> Tutor maillist - tu...@python.org
> To unsubscribe or change subscription options:
> http://mail.python.org/mailman/listinfo/tutor
>
>
--
˙uʍop ǝpısdn p,uɹnʇ pןɹoʍ ǝɥʇ ǝǝs noʎ 'ʇuǝɯɐן sǝɯıʇ ǝɥʇ puɐ 'ʇuǝʇuoɔ
ǝq s,ʇǝן ʇǝʎ
%>>> "".join( [ {'*':'@','^':'.'}.get(c,None) or
chr(97+(ord(c)-83)%26) for c in ",adym,*)&uzq^zqf" ] )
___
Tutor maillist - Tutor@python.org
To unsubscribe or change subscription options:
http://mail.python.org/mailman/listinfo/tutor
[Tutor] EXECUTING PYTHON AND SQL STAMENTS
I'm developing a module to execute an external command. The module executes
the command, but in order to make my code useful I have to enter some sql
staments. This is my code:
from dbf import *
from osgeo import ogr
import os
import sys
def call():
print "Ingresa el nombre para el nuevo mapa"
arg1 = "R1G-GEODESIA2.shp"
print arg1
print "Ingresa la condicion"
arg2 = "LAYER = 'R1G-GEODESIA'"
print arg2
print "Ingresa el nombre del mapa original"
arg3 = 'C:/Python26/tapalpa_05_plani_point.shp'
print arg3
os.system('"C:/Archivos de programa/FWTools2.4.7/setfw"')
os.system('"C:/Archivos de programa/FWTools2.4.7/bin/ogr2ogr.exe"'+" "
+"arg1" +" "+ "-where" +" "+ "arg2" +" " +"arg3")
call()
The problem is that when I run the module it throws the error:
Unable to open datasource`arg3' with the following drivers.
ESRI Shapefile
MapInfo File
UK .NTFSDTS
TIGER
S57
DGN
VRT
REC
Memory
BNA
CSV
NAS
GML
GPX
KML
GeoJSON
Interlis 1
Interlis 2
GMT
SQLite
ODBC
PGeo
OGDI
PostgreSQL
MySQL
XPlane
AVCBin
AVCE00
DXF
Geoconcept
GeoRSS
GPSTrackMaker
VFK
Can you help me please?
___
Tutor maillist - Tutor@python.org
To unsubscribe or change subscription options:
http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] (de)serialization questions
Hi Alan, Wow, I'm impressed! You're right, it's near Glencoe! Coming from Edingburgh it's a couple of miles just before Glencoe. Marvellous place! Cheers!! Albert-Jan ~~ All right, but apart from the sanitation, the medicine, education, wine, public order, irrigation, roads, a fresh water system, and public health, what have the Romans ever done for us? ~~ --- On Mon, 10/4/10, Alan Gauld wrote: From: Alan Gauld Subject: Re: [Tutor] (de)serialization questions To: tutor@python.org Date: Monday, October 4, 2010, 1:46 AM "Albert-Jan Roskam" wrote > It makes much more sense to maintain one database table instead of 3 csv files > for the three data typists' output. To be pedantic you will probably want several tables but they will all be in one file... :-) > Btw, your private website is nice too. Nice pictures! Well done, not many people notice that section :-) I keep meaning to add another dozen or so pages, but finding time > Do you recognize where this was taken: > http://yfrog.com/n0scotland046j . Could be any of a dozen places but if pushed I'll guess the Rannoch Moor. Maybe the top end of Glencoe? But the layer of low cloud wipes out too much to be sure. > You're lucky to live in a beautiful place like Scotland I think so :-) -- Alan Gauld Author of the Learn to Program web site http://www.alan-g.me.uk/ ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: http://mail.python.org/mailman/listinfo/tutor ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] subprocess.call warning
thank you for the reply
On Mon, Oct 4, 2010 at 12:47 AM, Steven D'Aprano wrote:
> On Mon, 4 Oct 2010 06:44:54 am Norman Khine wrote:
>> hello, from the docs http://docs.python.org/library/subprocess.html i
>> see there is a WARNING about deadlock when using the subprocess.call.
>> in my code i have this
>>
>> http://pastie.org/1197024
>
>
> The warning says:
>
> Like Popen.wait(), this will deadlock when using stdout=PIPE
> and/or stderr=PIPE and the child process generates enough
> output to a pipe such that it blocks waiting for the OS pipe
> buffer to accept more data.
>
> Since you don't set either stdout or stderr to PIPE, you shouldn't have
> to worry about a deadlock.
ah ok
>
>
>> the first calls the 'sox' library which joins all the .wav files into
>> one file and then i use the 'wav2swf' library to create a SWF output
>> of the file.
>>
>> can the code be improved?
>
> Code can always be improved :-) What do you consider an improvement?
> Easy to maintain in the future? Faster? Smaller?
to make it faster.
>
> What does get_abspath do? It sounds like it *may* be your own version of
> os.path.abspath.
i am using the iTools python library
http://docs.hforge.org/itools/web.html#public-api
http://git.hforge.org/?p=itools.git;a=blob;f=core/utils.py
>
> I'm not sure why you have a list called "imgtext" that doesn't contain
> text. It seems to be a list of partial file names rather than "image
> text".
the code is for my own implementation of a captcha to see how python
interacts with system libraries such as SOX and WAV2SWF and of course
to learn.
http://pastie.org/1197919
but yes, it is 'text' so i will change it in the next version.
>
> Your code, as given, can't work, because you call isupper() on integers
> 1 2 and 3. That will fail.
>
> I'd probably prefer to generate the filenames differently:
>
> def fix_filename(fname, base=sound_path):
> fname = str(fname)
> if fname.isupper():
> fname = "upper_%s" % fname.lower()
> return os.path.join(base, fname + '.wav')
>
> and then call it:
>
> filenames = ['A', 'b', 'c', 'D', 1, 2, 3]
> for name in filenames:
> name = fix_filename(name)
> sox_filenames.append(name)
>
>
> --
> Steven D'Aprano
> ___
> Tutor maillist - tu...@python.org
> To unsubscribe or change subscription options:
> http://mail.python.org/mailman/listinfo/tutor
>
--
˙uʍop ǝpısdn p,uɹnʇ pןɹoʍ ǝɥʇ ǝǝs noʎ 'ʇuǝɯɐן sǝɯıʇ ǝɥʇ puɐ 'ʇuǝʇuoɔ
ǝq s,ʇǝן ʇǝʎ
%>>> "".join( [ {'*':'@','^':'.'}.get(c,None) or
chr(97+(ord(c)-83)%26) for c in ",adym,*)&uzq^zqf" ] )
___
Tutor maillist - Tutor@python.org
To unsubscribe or change subscription options:
http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Pythonic nested lists
"col speed" wrote
HI again, I realise that I should have included more information in
the
above e-mail. Here goes:
a/ I have a 9*9 nested list called "rows", which contains the given
numbers
in a sudoku puzzle - with "0"s where the blanks go.
b/ I create roworder (roworder = [[str(j)+str(i) for i in range(9)]
for j in
range(9)]) - [["00" --> "08"], ["10" --> "18"] --> ["80" -->
"88"]]
c/ I populate a dictionary ("dic") with keys from "roworder" and
values from
"rows"
d/ I loop through the dict, changing any value"0" to be set(range(1,
10))- a
list of "possible numbers".
e/ Then I do:
for i in roworder:
notPoss = set([dic[j] for j in i if type(dic[j]) == int])
for k in j:
if type(dic[k]) == set:
dic[k].difference_update(notPoss)
thus removing the numbers that exist in the row from the "possible
numbers"
I need to do this for the columns and squares aswell, which is why I
do:
That approach should work.
Personally I'd probably create a Square class and bae my data on 9
squares.
Each square can return a row(of 3 items) given an index and a
column(of 3 items)
given an index.
class Square:
def __init__(self, size=3): self.cells = [0 for n in
range(size*size)] ...
def __str__(self):...
def getRow(self,n): ...
def getCol(self,n):...
def setCell(self,x,y,value)
def check(self):...
Thus I'd build my rows and columns dynamically from the squares rather
than the other way round. It just feels more natural to me.
But your approach should work too.
However I confess I haven't studied you code in detail.
Is there any specific problem or question you have or are you looking
for general feedback?
HTH,
--
Alan Gauld
Author of the Learn to Program web site
http://www.alan-g.me.uk/
___
Tutor maillist - Tutor@python.org
To unsubscribe or change subscription options:
http://mail.python.org/mailman/listinfo/tutor
[Tutor] Pythonic nested lists
Message: 6
Date: Tue, 28 Sep 2010 13:15:26 +0700
From: col speed
To: tutor@python.org
Subject: [Tutor] Pythonic nested lists
Message-ID:
Content-Type: text/plain; charset="iso-8859-1"
Hi all,
I've been trying to write a programme that solves sudoku problems for a
while now. I'm getting close, but would like to ask a few questions about
the most pythonic way of doing some things.
I've decided to create nested lists (row order, column order and square
order) which correspond with dictionary keys - the values of which are the
numbers given in the puzzle - so I can loop through the dict in different
orders.
I think the first two are OK, but the square order list seems extremely
messy and I would love some pointers. What I have is as follows:
roworder = [[str(j)+str(i) for i in range(9)] for j in range(9)]
colorder = zip(*roworder)
#here comes the problem!
start, index = 0,0
sqorder = []
for i in range(9):
sqorder.append([])
for j in range(start, start+3):
sqorder[i].extend(roworder[j][index:index+3])
if index == 6:
index = 0
else:
index += 3
if i == 2 or i == 5:
start += 3
Any comments at all would be gratefully received.
Thanks in advance
Colin
___
HI again, I realise that I should have included more information in the
above e-mail. Here goes:
a/ I have a 9*9 nested list called "rows", which contains the given numbers
in a sudoku puzzle - with "0"s where the blanks go.
b/ I create roworder (roworder = [[str(j)+str(i) for i in range(9)] for j in
range(9)]) - [["00" --> "08"], ["10" --> "18"] --> ["80" --> "88"]]
c/ I populate a dictionary ("dic") with keys from "roworder" and values from
"rows"
d/ I loop through the dict, changing any value"0" to be set(range(1, 10))- a
list of "possible numbers".
e/ Then I do:
for i in roworder:
notPoss = set([dic[j] for j in i if type(dic[j]) == int])
for k in j:
if type(dic[k]) == set:
dic[k].difference_update(notPoss)
thus removing the numbers that exist in the row from the "possible numbers"
I need to do this for the columns and squares aswell, which is why I do:
colorder = zip(*roworder) - (create a nested list "column wise" rather than
"row wise")
and want to create a "square order" list - which is what the above mess
does.
I hope this clarifies things.
Thanks again
Colin
___
Tutor maillist - Tutor@python.org
To unsubscribe or change subscription options:
http://mail.python.org/mailman/listinfo/tutor
