[Tutor] Organizing 15500 records, how?
With more than 15000 records you would be better off using a relational database. Although it will create more work to start with (you'll have to learn it), it will save you a lot of work in the medium and long term. Almost any relational database can be accessed from python.As it is just for your own use SQLite might be the most appropiate (it has a very small footprint) but MySQL is excellent and so are many others. To use a relational database you might think about learning SQL. It is very easy (especially if you you know any Boolean algebra) and is a language that has been used almost unchanged for decades and shows every sign of staying here for a long time. In computing it is one of the most useful things you can learn. There is a good introductory, interactive tutorial athttp://sqlcourse.com/ If you feel you need another abstraction layer on top of this you could look at SQLObject http://www.sqlobject.org/. Personally I would recommend that you start with MySQLhttp://www.mysql.com. It is open source, easy to install and use, stable and fast. But with SQL motors you have lots of good choices. Peter Jessop On 12/13/06, Thomas [EMAIL PROTECTED] wrote: I'm writing a program to analyse the profiles of the 15500 users of my forum. I have the profiles as html files stored locally and I'm using ClientForm to extract the various details from the html form in each file. My goal is to identify lurking spammers but also to learn how to better spot spammers by calculating statistical correlations in the data against known spammers. I need advise with how to organise my data. There are 50 fields in each profile, some fields will be much more use than others so I though about creating say 10 files to start off with that contained dictionaries of userid to field value. That way I'm dealing with 10 to 50 files instead of 15500. Also, I am inexperienced with using classes but eager to learn and wonder if they would be any help in this case. Any advise much appreciated and thanks in advance, Thomas ___ Tutor maillist - Tutor@python.org http://mail.python.org/mailman/listinfo/tutor ___ Tutor maillist - Tutor@python.org http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] [python-win32] Python as scripting glue, WAS Python for sysadmin
I am a windows system administrator and started learning python (as an alternative to VBScript) for the following reasons. 1) VBScript has serious limitations as a language 2) Needed access to TCP/IP protocols 3) Ability to to write scripts with GUI. 4) Portability Windows system administrators need to know VBScript because that is what most other Windows admins use and thus most examples are written in this language. However point 1-4 above are serious limitations of VBScript Now Powershell has arrived windows has its own usable shell. Of course the popular tools (and shells) for Unix have been available for a long time on Windows but the problem is not just the shell. If it is more appropiat to use awk or bash or sed on Linux, why not use awk or bash or sed on Windows.? Python does not compete with the shell but complements it. Indeed for a system administrator learning Python is a great investment in the future precisely because what you learn can then be used with other operating systems. Powershell can use .NET objects but, as I understand it, cannot use .NET libraries to produce interactive gui scripts. However I think it addresses point 2) above as .NET does come with TCP/IP libraries. The only drawback to using Python for Windows System administration is the lack of uptodate books dedicated to it. A good book dedicated to this topic will sell well. Peter Jessop MSCE ___ Tutor maillist - Tutor@python.org http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] How to make the loop work?
It's basically correct but you need to convert the raw_input to integer. c=0 d=int(raw_input(input number limit: )) while 1: c = c + 1 if c == d: break print c ___ Tutor maillist - Tutor@python.org http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Question about network server in python
I think the problem here is the 'break' statement. Does it not put you outside the while loop whereas in order to keep the server socket open you need it to loop forever. I also think that the s.accept should be inside the while loop. ___ Tutor maillist - Tutor@python.org http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Question about network server in python
import socket host = '' port = 57000 s=socket.socket(socket.AF_INET, socket.SOCK_STREAM) s.setsockopt(socket.SOL_SOCKET,socket.SO_REUSEADDR,1) s.bind((host,port)) s.listen(5) while 1: client,addr=s.accept() client.send(Connected to the server\n) #if someCondition: # cliente.close() ___ Tutor maillist - Tutor@python.org http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Mod-python
I have about 17 lines of text (about system information) that I need to pass
to
a remote web server AND I've been asked to send this data via Apache. I have
to
write a python script that will fetch a URL to pass this text. I understand
that
if I want to use the POST method, I would need to have a page with a submit
button, and this is not the case. I also know that with GET, the length of the
url is limited. What is the best way to send a long string?
When you say you that you want to send this data via Apache do you
mean that the web server you are sending to is running Apache or that
you are communicating from one server to another?
If you simply want to emulate a a web page with a submit button that
sends a Post you do it with code a bit like the following
import urllib, urllib2
url = http://www.somesite.com/somefolder/exampl1.cgi;
dict = {}
dict[field1] = value1
dict[field2] = value2
...
dict[fieldn] = valuen
urldata = urllib.urlencode(dict)
req = urllib2.Request(url)
req.add_header('User-agent','Mozilla/4.0 (compatible; MSIE 5.5; Windows NT 5.0')
fd = urllib2.urlopen(req,urldata)
Regards
Peter Jessop
Hope anyone can point me to the right direction..
Thanks,
Patricia
___
Tutor maillist - Tutor@python.org
http://mail.python.org/mailman/listinfo/tutor
___
Tutor maillist - Tutor@python.org
http://mail.python.org/mailman/listinfo/tutor
[Tutor] (OT) Monitorising WEB POSTs
I am building a python application to automate information capture from a web site. I want to use a python application to request and process the information instead of using the site's WEB page. However I am having problems finding out exactly what data the server expects. I have a firefox add-in called Web Developer that gives me good information about the form (fields names etc..) But what I want to do is find out exactly what POST data my browser is sending. I am aware of two ways of doing this. a) A sniffer to read the udp packet sent from my computer b) Send the page to my own WEB server Is there a program or add in that will give me this information without resorting to the solutions above. Peter Jessop ___ Tutor maillist - Tutor@python.org http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] python application on a web page
Tkinter is a multiplatform toolkit for producing programs with graphical interface. But multiplatform refers to BSD, Linux, Mackintosh, Windows etc. If you want to produce the output on the WEB you need to produce the output in HTML or in some way that most users can see it in their Browsers. (Java Script, Flash, etc...) This forum caters well for beginners (and more advanced) and the quality and helpfulness are very high. Here are a some links that may be useful. http://wiki.python.org/moin/BeginnersGuide/NonProgrammers http://www.python.org/doc/Intros.html http://diveintopython.org/ ___ Tutor maillist - Tutor@python.org http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Truly generic database API
It depends on your system constraints. If you are only developing on Windows then you can use ODBC. ODBC supports writing to text files. It is an old fashioned technology and is not very fast. However it is well supported and mature. ___ Tutor maillist - Tutor@python.org http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] (OT) Monitorising WEB POSTs
Matthew Fantastic! Exactly what I needed ___ Tutor maillist - Tutor@python.org http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Reference a variable from a string whose value is the name of the variable
Kent Thanks for your reply. The structure is for sending form variables and values to a web server with POST. I am using urllib.urlencode which accepts a list or dictionary as argument. The idea is to look for airline tickets. The airline I buy from only lets me search by data but I want to automate the process by POSTing the form for a range of dates so that effectively I can search by price not by date. I will then parse the returned HTML using HTMLParser or BeautifulSoup. The form has a large number of variables and I just want to find a tidy way to organise the information. Peter ___ Tutor maillist - Tutor@python.org http://mail.python.org/mailman/listinfo/tutor
[Tutor] Reference a variable from a string whose value is the name of the variable
The best way to explain my problem is with an example f0_n = field0 f0_v =value0 f1_n=field1 f1_v=value1 ... f100_n = field100 f100_v = value100 I now want to define a list of 2ples of the form [(f0_n,f0_v),(f1_n,f1_v),...,(f100_n,f100_v)] I wish to define the list using a for loop, i.e. data = [ ] for i in xrange(1,101): data = data.append((f %i _n, f %i_v)) I have put the % sign above. Obviously it is not like that but how does one do it? The aim is to reference the s variable whose name is the string that I create concatenating f + str(i)+_n and f+str(i)+_v Thanks Peter Jessop ___ Tutor maillist - Tutor@python.org http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Reference a variable from a string whose value is the name of the variable
Thanks Andre I realise now I did not make it too clear. Basically the variable names are predictable but the values aren't. Maybe I should have expressed it like this f0_n = arbitrayValue f0_v =another valule f1_n=something else f1_v=etc.. ... f100_n = another value f100_v = nexvalue I then wish to pass the list of 2ples as an argument to a function and thus I need the variable themselves and not the string values. ___ Tutor maillist - Tutor@python.org http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Reference a variable from a string whose value is the name of the variable
André Thanks a lot for clearing this up for me. Regards Peter Jessop ___ Tutor maillist - Tutor@python.org http://mail.python.org/mailman/listinfo/tutor
[Tutor] Problem installing MySQLdb under windows
I am running Python 2.4.3, MySQL 5.0.On trying to install MySQLdb 1.2.1 I ran into the following problem.D:\Python24\Lib\site-packages\MySQLdbsetup.py buildrunning buildrunning build_pycopying MySQLdb\release.py - build\lib.win32- 2.4\MySQLdbrunning build_exterror: The .NET Framework SDK needs to be installed before building extensions for Python.Is it really necessary to install The .NET Framework SDK (354Mb) or is there a simpler way? Thanks for any helpPeter Jessop ___ Tutor maillist - Tutor@python.org http://mail.python.org/mailman/listinfo/tutor
[Tutor] Problem installing MySQLdb under windows
Thanks Liam and Kent.Problem is now sorted.Which brings me on to my next question... ___ Tutor maillist - Tutor@python.org http://mail.python.org/mailman/listinfo/tutor
[Tutor] Generating static WEB sites
I am looking at generating static web pages.What I wish to do is periodically regenerate a WEB site form HTML, text and MySQL data.I have been experimenting with HTMLgen and this seems to be adequate for what I need. However HTMLgen does not seem to have been updated for a while and I was wondering if I should be looking at any other packages.Any suggestions?ThanksPeter Jessop ___ Tutor maillist - Tutor@python.org http://mail.python.org/mailman/listinfo/tutor
[Tutor] Generalising system processes
Good day. Is there a library in Python that generalises the display of processes in the system. I am looking for a consistent way of seeing processess, (a python equivalent of ps(unix) or tasklist (windows). I have googled but the only suggestion I have found was to install ps on Windows. Thanks Peter Jessop ___ Tutor maillist - Tutor@python.org http://mail.python.org/mailman/listinfo/tutor
