msiexec and python-2.6.1.msi

[cjl]

With previous versions of the Python Windows msi installer, for
example 2.5.4,
I could run the following command from the windows cmd prompt:

msiexec /a C:\python-2.5.4.msi /qn TARGETDIR=C:\python

This would result in a 'full' python installation in the C:\python
directory, inside of which I would find the (necessary) file
msvcr70.dll. Perhaps it was actually msvcr71.dll most recently, I
can't remember. Of
course, this would not register extensions, but I did not want it to.

When I try the same command with python-2.6.1.msi, the file
msvcr90.dll and the manifest file are not created in the python
directory, nor anywhere else on the system.

If, instead, I double click the python-2.6.1.msi installer, the
msvcr90.dll file is created. I have tried msiexec with '/a' and '/i'
command line switches, and every possible combination of options, all
to no avail.

I can't figure out if this is a regression, but it used to work.

I am basing my attempts on the following page:
http://www.python.org/download/releases/2.5/msi/

I could not find a similar page for the 2.6 release.

Can anyone point me in the right direction to figure out what I'm
doing wrong?

Thanks,
-cjlesh
--
http://mail.python.org/mailman/listinfo/python-list

Windows PIL installer question

[cjl]

Is there any way to run the PIL installer from the command line on
Windows in 'silent' mode, without displaying the install screens or
requiring user interaction?
--
http://mail.python.org/mailman/listinfo/python-list

Windows Python install vs. MSI extract question

[cjl]

OK -- this might be a strange question.

If I do a 'full' install of Python on Windows XP, the result is a
directory 'C:\Python25'. Depending on whether I install for all users
or just me, the 'python25.dll' might end up in 'C:\Python25', or in
the Windows system directory. If I copy python25.dll to the 'C:
\Python25' directory, I now should have everything that Python
installs all in on spot, correct?

Now, if I download the Python installer, create an empty directory 'C:
\python' and type the following at the command prompt:

msiexec /a python-2.5.4.msi /qb TARGETDIR=C:\python

Is there any difference between the folder that resulted from running
the installer and the folder that resulted from extracting the MSI?

I put together a portable version of Django (www.instantdjango.com),
and want to experiment with multiple versions of Python on Windows
without running the installer.

Besides the changes to the registry and environment variable, am I
missing anything when I extract the MSI?

Thanks in advance,
cjlesh
--
http://mail.python.org/mailman/listinfo/python-list

newbie question: parsing street name from address

[cjl]

P:

I am working on a project that requires geocoding, and have written a
very simple geocoder that uses the Google service.

I would like to be able to extract the name of the street from the
addresses in my data, however they vary significantly. Here a some
examples:

25 Main St
2500 14th St
12 Bennet Pkwy
Pearl St
Bennet Rd and Main st
19th St

As you can see, sometimes I have the house number, and sometimes I do
not. Sometimes the street name is a number. Sometimes I simply have
the names of intersecting streets.

I would like to be able to parse the above into the following:

Main St
14th St
Bennet Pkwy
Pearl St
Bennet Rd
Main St
19th St

How might I approach this complex parsing problem?

-CJL

-- 
http://mail.python.org/mailman/listinfo/python-list

csv.reader length?

[cjl]

P:

Stupid question:

reader = csv.reader(open('somefile.csv'))
for row in reader:
do something

Any way to determine the "length" of the reader (the number of rows)
before iterating through the rows?

-CJL

-- 
http://mail.python.org/mailman/listinfo/python-list

Newbie Question: python mysqldb performance question

[cjl]

Group:

I'm new to python and new to mysql.

I have a csv file that is about 200,000 rows that I want to add to a
mysql database.  Yes, I know that I can do this directly from the
mysql command line, but I am doing it through a python script so that
I can munge the data before adding it.

I have the following example code:

conn = MySQLdb.connect(db="database", host="localhost", user="root",
passwd="password")
c = conn.cursor()

reader = csv.reader(open(sys.argv[1]))
for row in reader:
data1, data2, data3, data4 = row
data = (data1,data2,data3,data4)
c.execute("""insert into datatable values (%s, %s, %s, %s)""",
data)
conn.commit()

This takes a really long time to execute, on the order of minutes.
Directly importing the csv file into mysql using 'load infile' takes
seconds.

What am I doing wrong? What can I do to speed up the operation?

Thanks in advance,
cjl

-- 
http://mail.python.org/mailman/listinfo/python-list

Simple sqlite3 question

[cjl]

P:

I am using python 2.5.1 on windows. I have the following code:

conn = sqlite3.connect('.\optiondata')
c = conn.cursor()
try:
c.execute('''create table options (ssymbol text, strike real,
osymbol text, bid real, mpp real, upp real)''')
except sqlite3.OperationalError:
pass

I am hoping the above code creates a new database file named
'optiondata' with a single table named 'options', or connects to it if
already created. Am I off the mark here?

I also have a function that does the following:

c.execute("""insert into options values (?,?,?,?,?,?)""",data)

When I run the script and there is no file named optiondata, one is
created and the correct data is added to it.  If I run the script
again then the data from the first run seems to be replaced with the
data from the second run. I expected that the data from the second run
would be appended to the database file, not replace it.

Can anyone point me in the right direction to get the expected
behavior?

-cjl

-- 
http://mail.python.org/mailman/listinfo/python-list

Beautiful Soup iterator question....

[cjl]

P:

I am screen-scraping a table. The table has an unknown number of rows,
but each row has exactly 8 cells.  I would like to extract the data
from the cells, but the first three cells in each row have their data
nested inside other tags.

So I have the following code:

for row in table.findAll("tr"):
for cell in row.findAll("td"):
print cell.contents[0]

This code prints out all the data, but of course the first three cells
still contain their unwanted tags.

I would like to do something like this:

for cell1, cell2, cell3, cell4, cell5, cell6, cell7, cell8 in
row.findAll("td"):

Then treat each cell differently.

I can't figure this out. Can anyone point me in the right direction?

-CJL

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: parsing tables with beautiful soup?

[cjl]

DB:

Thank you, that worked perfectly.

-CJL

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: parsing tables with beautiful soup?

[cjl]

This works:

for row in soup.find("table",{"class": "class_name"}):
for cell in row:
print cell.contents[0]

Is there a better way to do this?

-cjl

-- 
http://mail.python.org/mailman/listinfo/python-list

parsing tables with beautiful soup?

[cjl]

I am learning python and beautiful soup, and I'm stuck.

A web page has a table that contains data I would like to scrape. The
table has a unique class, so I can use:

soup.find("table", {"class": "class_name"})

This isolates the table. So far, so good. Next, this table has a
certain number of rows (I won't know ahead of time how many), and each
row has a set number of cells (which will be constant).

I couldn't find example code on how to loop through the contents of
the rows and cells of a table using beautiful soup. I'm guessing I
need an outer loop for the rows and an inner loop for the cells, but I
don't know how to iterate over the tags that I want.  The beautiful
soup documentation is a little beyond me at this point.

Can anyone point me in the right direction?

thanks again,
cjl

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: difference between urllib2.urlopen and firefox view 'page source'?

[cjl]

Group:

Thank you for all the informative replies, they have helped me figure
things out. Next up is learning beautiful soup.

Thank you for the code example, but I am trying to learn how to
'screen scrape', because Yahoo does make historical stock data
available using the CSV format, but they do not do this for stock
options, which is what I am ultimately attempting to scrap.

Here is what I have so far, I know how broken and ugly it is:

import urllib2, sys
from BeautifulSoup import BeautifulSoup

page = urllib2.urlopen("http://finance.yahoo.com/q/op?s="; +
sys.argv[1])
soup = BeautifulSoup(page)
print soup.find("table",{"id" :"yfncsubtit"}).big.b.contents[0]

This actually works, and will print out the current stock price for
whatever ticker symbol you supply as the command line argument when
you launch this script. Later I will add error checking, etc.

Any advice on how I am using beautiful soup in the above code?

thanks again,
cjl

-- 
http://mail.python.org/mailman/listinfo/python-list

difference between urllib2.urlopen and firefox view 'page source'?

[cjl]

Hi.

I am trying to screen scrape some stock data from yahoo, so I am
trying to use urllib2 to retrieve the html and beautiful soup for the
parsing.

Maybe (most likely) I am doing something wrong, but when I use
urllib2.urlopen to fetch a page, and when I view 'page source' of the
exact same URL in firefox, I am seeing slight differences in the raw
html.

Do I need to set a browser agent so yahoo thinks urllib2 is firefox?
Is yahoo detecting that urllib2 doesn't process javascript, and
passing different data?

-cjl

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Running python from a usb drive

[cjl]

Thorsten:

Thank you for your reply.

> Setting environment variables has only effect on the process itself
> and the subprocesses. This has nothing to do with Windows, it's the
> same with Linux.

True, and the changes to path and pythonpath are gone after I close the
console window, but the results of the assoc and ftype commands are
changes to the registry that linger. If I run my setup on a computer
that has python installed, I will overwrite the pre-existing registry
settings. I can restore them with a script, but I can't guarantee that
my restore script will run.

I'm still looking for a way to modify these temporarily, but it looks
like I might be "up the creek" on this one. Oh well.

Thanks again,
CJL

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Running python from a usb drive

[cjl]

John:

> Congratulations. Now: how much time have you spent, and how much per
> hour are you worth? hours * dollars_per_hour < GBP_to_USD(4.99) ???

Since you mention it, I am currently earning about $200 an hour (when
I'm working), and I spent about 3 hours on this, so this cost me about
$600. I think 4.99 GBP (the price of movable python) translates to
about 9 or 10 dollars. So all told this cost me about $590.

Well worth it at twice the price, because I did it myself, I learned
from doing it, others my learn from it, and it is "open", unlike
movable python.

Any ideas about how to set file type associations without writing to
the registry?

Thanks again,
CJL

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Running python from a usb drive

[cjl]

Tim:

> That would be because it's PATHEXT not PATHTEXT (it's sort
> of like a PATH but for EXTensions).

Doh.

Me fail English? That's unpossible.

Thanks, I think everything is working with my poor man's movable
python.
Last on my todo list is to restore the assoc and ftype settings when
I'm done, and I found:

http://portableapps.com/node/121

Which seems to show how to do that.

CJL

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Running python from a usb drive

[cjl]

Jason:

Thanks! That worked...in fact, almost everything is now working as
expected (so far).

Here is my batch file:

echo "Making changes to path and file associations..."
path =
%PATH%;%CD%Python24;%CD%Python24\libs;%CD%Python24\Scripts;%CD%Python24\Lib\site-packages;%CD%Python24\DLLs
set PYTHONPATH=%CD%Python24
ASSOC .py=Python.File
ASSOC .pyc=Python.CompiledFile
ASSOC .pyo=Python.CompiledFile
ASSOC .pyw=Python.NoConFile
FTYPE Python.File=%CD%Python24\python.exe "%%1" %%*
FTYPE Python.CompiledFile=%CD%Python24\python.exe "%%1" %%*
FTYPE Python.NoConFile=%CD%Python24\pythonw.exe "%%1" %%*
set PATHTEXT=.py;%PATHTEXT%
CMD

I'm still having a problem with setting PATHTEXT...I should be able to
now type django-admin at the cmd prompt and have it work, but I need to
still type django-admin.py  ... I'm not sure what's wrong with my
setting of pathtext.

Any ideas?

Thanks again,
CJL

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Running python from a usb drive

[cjl]

Hey all:

It seems no matter what I do the %1 gets replaced by paramaters sent to
the batch file...there must be some way of "escaping" this, but I can't
find the answer (yet) with google. Anyone?

-CJL

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Running python from a usb drive

[cjl]

Hey all:

I'm getting closer. My startpython.bat file is now:

path=%PATH%;%CD%Python24;%CD%Python24\libs;%CD%Python24\Scripts;%CD%Python24\Lib\site-packages;%CD%Python24\DLLs
set PYTHONPATH=%CD%Python24
ASSOC .py=Python.File
ASSOC .pyc=Python.CompiledFile
ASSOC .pyo=Python.CompiledFile
ASSOC .pyw=Python.NoConFile
FTYPE Python.File=%CD%Python24\python.exe "%1" %*
FTYPE Python.CompiledFile=%CD%Python24\python.exe "%1" %*
FTYPE Python.NoConFile=%CD%Python24\pythonw.exe "%1" %*
set PATHTEXT=%PATHTEXT%;.py;.pyc;.pyo;.pyw
cmd

I am having a problem with the ftype commands as written above. If I
type them from the command line exactly like above they work. But for
some reason they do not work from my batch file. Anyone familiar with
bath file syntax that can help me? I am very close here...

Thanks again,
CJL

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Running python from a usb drive

[cjl]

Uwe:

Thank you for your reply.

> is pythonpath really case insensitive on windows ?

I think so. After running my batch file, I can load the python
interpreter by typing 'python', and can then type 'import django'
without error. This lives in the site-packages directory, so it is
finding it.

However, there is a python script that lives in python24\Scripts called
'django-admin.py', which I am adding to my path. When I type
'django-admin.py --help' I get an error that it couldn't load the
django module. When I type 'python E:\python24\django-admin.py --help'
I get the correct output, and no errors, so I know it is loading the
module.

I guess I am trying to figure out why, and what the top "shebang" line
should be for the script django-admin.py, because the removable drive
can have different drive letters, so I can't hard code it.

thanks again,
CJL

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Running python from a usb drive

[cjl]

Jordan:

Thank you for your reply.

> If making a usb version of python was that easy, movable python would
> be open source.

I knew about movable python, but I'm not using it because it's not open
source. I guess those guys but some work into it, and feel like a small
fee is appropriate, but I guess I would rather use something open, even
if it means I have to make it myself.

> Copying the Python24 directory is a good start, but doesn't include the
> enormous number of registry keys that are included in the install and
> are probably needed for the complete capabilites of python.

Is the source of the windows python installer available? I guess I
could see what registry keys they are setting...

> Environmental Variables? Try setting the user/system variable "pythonpath"

I do set pythonpath, see above.

Any other ideas?

Thanks again,
CJL

-- 
http://mail.python.org/mailman/listinfo/python-list

Running python from a usb drive

[cjl]

Hey:

I am trying to run python from a usb drive under windows xp. I
installed python "for this user" on to a machine, then copied the
entire Python24 directory to the usb drive.

I have the following in a batch file at the root of the drive:

@path=%PATH%;%CD%Python24;%CD%Python24\libs;%CD%Python24\Scripts;%CD%Python24\Lib\site-packages;%CD%Python24\DLLs
@set pythonpath = %CD%Python24
@cmd

When I double click the file and type 'python' at the prompt I am in a
working python environment, and I am able to import modules in the
site-packages directory (in this case, django).

However, when I run a script directly from the cmd prompt (in this case
'django-admin.py' the script runs, but fails to import modules from the
site-packages directory.

Is there a command line option to python.exe or an environment variable
I can set to remedy this?

Any other thoughts?

Thanks in advance,
CJL

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Which is easier? Translating from C++ or from Java...

[cjl]

Hey all:

Thanks for the responses...

I've found a third open source implementation in pascal (delphi), and
was wondering how well that would translate to python?

-cjl

-- 
http://mail.python.org/mailman/listinfo/python-list

Which is easier? Translating from C++ or from Java...

[cjl]

Hey all:

I'm working on a 'pure' python port of some existing software.

Implementations of what I'm trying to accomplish are available (open
source) in C++ and in Java.

Which would be easier for me to use as a reference?

I'm not looking for automated tools, just trying to gather opinions on
which language is easier to understand / rewrite as python.

-cjl

-- 
http://mail.python.org/mailman/listinfo/python-list

left padding zeroes on a string...

[cjl]

Hey all:

I want to convert strings (ex. '3', '32') to strings with left padded
zeroes (ex. '003', '032'), so I tried this:

string1 = '32'
string2 = "%03s" % (string1)
print string2

>32

This doesn't work. If I cast string1 as an int it works:

string1 = '32'
int2 = "%03d" % (int(string1))
print int2

>032

Of course, I need to cast the result back to a string to use it. Why
doesn't the first example work?

-cjl

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: binutils "strings" like functionality?

[cjl]

David M. Cooke wrote:

> Are you sure it's monkey/chicken/dog/cat, and not
> monkey\chicken\dog\cat? The later one will print monkey\\chicken...
> because of the repr() call.
>
> Also, you probably want it as [\x20-\x7e] (the DEL character \x7f
> isn't printable). You're also missing tabs (\t).
>
> The GNU binutils string utility looks for \t or [\x20-\x7e].

Yeah, it is "monkey\chicken\dog\cat", thank you for pointing that out.

I started snooping throught the sourcecode for 'strings' in binutils to
see what it matches against, but I don't really know how to program, so
you just saved me even more time. Thanks again!

-CJL

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: binutils "strings" like functionality?

[cjl]

Fredrik Lundh wrote:

> something like this could work:
>
> import re
>
> text = open(file, "rb").read()
>
> for m in re.finditer("([\x20-\x7f]{4,})[\n\0]", text):
> print m.start(), repr(m.group(1))


Hey...that worked. I actually modified:

for m in re.finditer("([\x20-\x7f]{4,})[\n\0]", text):

to

for m in re.finditer("([\x20-\x7f]{4,})", text):

and now the output is nearly identical to 'strings'. One problem
exists, in that if the binary file contains a string
"monkey/chicken/dog/cat" it is printed as "mokey//chicken//dog//cat",
and I don't know enough to figure out where the extra "/" is coming
from.

Help?

-CJL

-- 
http://mail.python.org/mailman/listinfo/python-list

binutils "strings" like functionality?

[cjl]

Hey all:

I am working on a little script that needs to pull the strings out of a
binary file, and then manipulate them with python.

The command line utility "strings" (part of binutils) has exactly the
functionality I need, but I was thinking about trying to implement this
in pure python.

I did some reading on opening and reading binary files, etc., and was
just wondering if people think this is possible, or worth my time (as a
learning exercise), or if something like this already exists.

-cjl

-- 
http://mail.python.org/mailman/listinfo/python-list