Waiting for receiving data

[Anjanesh Lekshminarayanan]

fp = urllib.urlopen(url)
data = fp.read()

Retrieving XML data via an XML service API.
Very often network gets stuck in between. No errors / exceptions.

CTRL+C

  File "get-xml.py", line 32, in 
fp = urllib.urlopen(url)
  File "/usr/lib/python2.6/urllib.py", line 87, in urlopen
return opener.open(url)
  File "/usr/lib/python2.6/urllib.py", line 206, in open
return getattr(self, name)(url)
  File "/usr/lib/python2.6/urllib.py", line 348, in open_http
errcode, errmsg, headers = h.getreply()
  File "/usr/lib/python2.6/httplib.py", line 1048, in getreply
response = self._conn.getresponse()
  File "/usr/lib/python2.6/httplib.py", line 974, in getresponse
response.begin()
  File "/usr/lib/python2.6/httplib.py", line 391, in begin
version, status, reason = self._read_status()
  File "/usr/lib/python2.6/httplib.py", line 349, in _read_status
line = self.fp.readline()
  File "/usr/lib/python2.6/socket.py", line 397, in readline
data = recv(1)
KeyboardInterrupt

Is there I can do to try something else if its taking too long to
retrieve from the network ? Like kill previous attempt and retry ?

Thanks
Anjanesh Lekshmnarayanan
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: print function in python3.1

[Anjanesh Lekshminarayanan]

> Maybe it would help if you explained what you are actually trying to
> accomplish.

import csv
f = csv.reader(open('data.txt'), delimiter='\t') # 2GB text file
sql = "INSERT INTO `data` VALUES (NULL,%s,%s,%s,%s,%s);"
for row in f:
print (sql, (row[0],row[1],row[2],row[3],row[4]))

$ python3 parse.py3 > data.sql

But because of print() being a function in Py3,
print (sql, (row[0],row[1],row[2],row[3],row[4]))
prints
INSERT INTO `data` VALUES (NULL, '%s', '%s', '%s', '%s', '%s');
('142', 'abcde', '2006-03-01 05:17:14', '', '')
instead of
INSERT INTO `data` VALUES (NULL, '142', 'abcde', '2006-03-01 05:17:14', '', '');

Anjanesh
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: print function in python3.1

[Anjanesh Lekshminarayanan]

As of now, there is no mysql adaptor for Python3. Hence cant use escape_string()

> I don't have the slightest clue what you want to say with that.

-- 
Anjanesh Lekshmnarayanan
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: print function in python3.1

[Anjanesh Lekshminarayanan]

> Depending on your DB-adapter, you are out of luck here. Either connect to a 
> db even if you don't need it, or try & see if you can locate the 
> implementation in the module somehow.

ImportError: No module named MySQLdb
MySQLdb only available in Python2.

-- 
Anjanesh Lekshmnarayanan
-- 
http://mail.python.org/mailman/listinfo/python-list

print function in python3.1

[Anjanesh Lekshminarayanan]

Python 3.1.1

sql = "INSERT INTO `tbl` VALUES (NULL, '%s', '%s', '%s', '%s', '%s');"
for row in fp:
print (sql, (row[0],row[1],row[2],row[3],row[4]))
.
INSERT INTO `tbl` VALUES (NULL, '%s', '%s', '%s', '%s', '%s'); ('142',
'abc', '2006-04-09 02:19:24', '', '')
.
Why is it showing %s in the output ?

1. I dont want to sql % () because that doesnt escape the strings
2. I cant use conn.escape_string(r) because Im not connected to a
database. Output script to file and import to database loated
elsewhere.
-- 
Anjanesh Lekshmnarayanan
-- 
http://mail.python.org/mailman/listinfo/python-list

Integer Division

[Anjanesh Lekshminarayanan]

>>> a = 1
>>> b = 25
>>> a / b
0
>>> float(a) / b
0.040001
>>>

>>> from __future__ import division
>>> a = 1
>>> b = 25
>>> a / b
0.040001
>>>

In what simple way can I get just 0.04 ?

-- 
Anjanesh Lekshmnarayanan
-- 
http://mail.python.org/mailman/listinfo/python-list

array next pointer

[Anjanesh Lekshminarayanan]

>>> a = ['cat','dog','elephant']
>>> a.next()
Traceback (most recent call last):
  File "", line 1, in 
AttributeError: 'list' object has no attribute 'next'
>>>
Is there something that imtates PHP's next() ? (http://php.net/next)
--
http://mail.python.org/mailman/listinfo/python-list

Re: Multiple conditional expression

[Anjanesh Lekshminarayanan]

> How do we know that from the what the OP posted?
Its CGI alright.
spaces = form.has_key('spaces') and form.getvalue('spaces') == '1'

But I just dont see how
spaces = (form.has_key('spaces') ? form.getvalue('spaces') == 1 ?
True: False : False)
is complicated in anyway. Its not that hard to read at all.

Thanks.
--
http://mail.python.org/mailman/listinfo/python-list

Re: UnicodeDecodeError: 'charmap' codec can't decode byte 0x9d in position 10442: character maps to

[Anjanesh Lekshminarayanan]

> (1) what is produced on Anjanesh's machine
>>> sys.getdefaultencoding()
'utf-8'

> (2) it looks like a small snippet from a Python source file!
Its a file containing just JSON data - but has some unicode characters
as well as it has data from the web.

> Anjanesh, Is it a .py file
Its a .json file. I have a bunch of these json files which Im parsing.
using json library.

> Instead of "something like", please report exactly what is there:
>
> print(ascii(open('the_file', 'rb').read()[10442-20:10442+21]))
>>> print(ascii(open('the_file', 'rb').read()[10442-20:10442+21]))
b'":42,"query":"0 1\xc2\xbb\xc3\x9d \\u2021 0\\u201a0 \\u2'

> Trouble with cases like this is as soon as they become interesting, the OP 
> often
snatches somebody's one-liner that "works" (i.e. doesn't raise an exception),
makes a quick break for the county line, and they're not seen again :-)

Actually, I moved the files to my Ubuntu PC which has Python 2.5.2 and
didnt give the encoding issue. I just couldnt spend that much time on
why a couple of these files had encoding issues in Py3 since I had to
parse a whole lot of files.
--
http://mail.python.org/mailman/listinfo/python-list

Multiple conditional expression

[Anjanesh Lekshminarayanan]

How do I achieve something like this using python ?
spaces = (form.has_key('spaces') ? form.getvalue('spaces') == 1 ? True
: False : False)

spaces = True if form.getvalue('spaces') == 1 if
form.has_key('spaces') else False else False
-- 
Anjanesh Lekshmnarayanan
--
http://mail.python.org/mailman/listinfo/python-list

Re: UnicodeDecodeError: 'charmap' codec can't decode byte 0x9d in position 10442: character maps to

[Anjanesh Lekshminarayanan]

> It does auto-detect it as cp1252- look at the files in the traceback and
> you'll see lib\encodings\cp1252.py. Since cp1252 seems to be the wrong
> encoding, try opening it as utf-8 or latin1 and see if that fixes it.

Thanks a lot ! utf-8 and latin1 were accepted !
--
http://mail.python.org/mailman/listinfo/python-list

UnicodeDecodeError: 'charmap' codec can't decode byte 0x9d in position 10442: character maps to

[Anjanesh Lekshminarayanan]

Im reading a file. But there seems to be some encoding error.

>>> f = open(filename)
>>> data = f.read()
Traceback (most recent call last):
  File "", line 1, in 
data = f.read()
  File "C:\Python30\lib\io.py", line 1724, in read
decoder.decode(self.buffer.read(), final=True))
  File "C:\Python30\lib\io.py", line 1295, in decode
output = self.decoder.decode(input, final=final)
  File "C:\Python30\lib\encodings\cp1252.py", line 23, in decode
return codecs.charmap_decode(input,self.errors,decoding_table)[0]
UnicodeDecodeError: 'charmap' codec can't decode byte 0x9d in position
10442: character maps to 

The string at position 10442 is something like this :
"query":"0 1»Ý \u2021 0\u201a0 \u2021»Ý","

So what encoding value am I supposed to give ? I tried f =
open(filename, encoding="cp1252") but still same error. I guess
Python3 auto-detects it as cp1252
-- 
Anjanesh Lekshmnarayanan
--
http://mail.python.org/mailman/listinfo/python-list

Re: Method returning an Iterable Object

[Anjanesh Lekshminarayanan]

But how come a raise StopIteration in the next() method doesnt need to
be caught ? It works without breaking.

class twoTimes:
max = 10**10

def __init__(self, n):
self.__n = n

def next(self):
if self.__n > self.max:
raise StopIteration
self.__n *= 2
return self.__n

def __iter__(self):
return self


t = twoTimes(5)
c = 0

print (t.next())
print (t.next())

for n in t:
print n


Anjanesh
--
http://mail.python.org/mailman/listinfo/python-list

Re: Method returning an Iterable Object

[Anjanesh Lekshminarayanan]

> You can also replace the whole class with a function thusly:
>
>def two_times(n):
>for k in itertools.count(1):
>yield n * (2**k)
>
> This function is then called a generator (because it generates an
> iterator). You can now say
>
>infinitely_doubling_numbers = two_times(2)
>for number in in infinitely_doubling_numbers:
>print number

Oh..this is new. Will checkup itertools. Thanks.

>> t = twoTimes(5)
>> while (n in t.getNext()): # while (n in t):
>> print (n)
>>
> You are aware that this is an infinite loop, as is my example above BTW?
> (Probably just an example, but I ask just in case.)

I was aware this was an infinite loop - just didnt want to put more
code for an example.
--
http://mail.python.org/mailman/listinfo/python-list

Method returning an Iterable Object

[Anjanesh Lekshminarayanan]

Is there a way to return an iterable object ?

class twoTimes:
def __init__(self, n):
self.__n = n

def getNext():
self.__n *= 2
return self.__n


t = twoTimes(5)
while (n in t.getNext()): # while (n in t):
print (n)

-- 
Anjanesh Lekshmnarayanan
--
http://mail.python.org/mailman/listinfo/python-list

Re: Py3 - converting bytes to ascii

[Anjanesh Lekshminarayanan]

The problem seems be solved with urllib.request.urlretrieve()
I think the binary information read() was giving had headers like 
content-size - but not HTTP headers.
The first couple of bytes indicate how much content to read and after 
reading that content, the next set of bytes indicate the next chunk.

--
http://mail.python.org/mailman/listinfo/python-list

Py3 - converting bytes to ascii

[Anjanesh Lekshminarayanan]

Using Python 3.0

res = urllib.request.urlopen(url)
f = open('file.txt', 'wb') # Since res.read() returns bytes
f.write(res.read())

But newline and return feeds are stored as b14, 58a as text in the text file.

So how do I to convert res.read() to ascii on opening the file in
ascii mode f = open('file.txt', 'w')?

Thanks
--
http://mail.python.org/mailman/listinfo/python-list

Re: mod_python resources

[Anjanesh Lekshminarayanan]

Same requirement here.
But isnt there any mod_python for Python 3.0 ?
Or do we need to build it from source ourselves ?

I was hoping there would be mod_wsgi binaries for Python 3.0.
--
http://mail.python.org/mailman/listinfo/python-list

Re: PHP's str_replace ?

[Anjanesh Lekshminarayanan]

Matthias Huening wrote:

Matthias Huening (10.09.2008 16:07):

Anjanesh Lekshminarayanan (10.09.2008 15:50):

In PHP, if I do
str_replace(array('a', 'e', 'i', 'o', 'u'), '-', $str)
it'll replace all vowels with a hyphen in string $str.

Is there some equivalent in Python ?


What about something like this:

import re
new_str = re.sub('([aeiou])-', r'\1', str)



Sorry - I misinterpreted your question.
Try this instead:

import re
new_str = re.sub('[aeiou]', '-', str)

Wow - this is neat. Thanks

--
http://mail.python.org/mailman/listinfo/python-list

PHP's str_replace ?

[Anjanesh Lekshminarayanan]

In PHP, if I do
str_replace(array('a', 'e', 'i', 'o', 'u'), '-', $str)
it'll replace all vowels with a hyphen in string $str.

Is there some equivalent in Python ?

Thanks
--
http://mail.python.org/mailman/listinfo/python-list

Re: EOF

[Anjanesh Lekshminarayanan]

Thanks for the shutil.copyfileobj.
Oddly, the EOFError didnt work though.

Gabriel Genellina wrote:

En Fri, 22 Aug 2008 16:53:58 -0300, Wojtek Walczak <[EMAIL PROTECTED]> escribió:

  

On Fri, 22 Aug 2008 22:18:37 +0530, Anjanesh Lekshminarayanan wrote:



Im trying to download a file from a server. But how do I detect EOF ?
  

Whenever read() method returns empty string/list.




while f1: # When to stop ?
  

 retval = f1.read()
 if not retval:
break
 f2.write(retval)



Those read() should be read(size) - read() tries to get the whole contents in 
memory.
Anyway, shutil.copyfileobj already implements that logic.

  


--
http://mail.python.org/mailman/listinfo/python-list

EOF

[Anjanesh Lekshminarayanan]

|Hi

Im trying to download a file from a server. But how do I detect EOF ?
||


import urllib2

f1 = urllib2.urlopen('ftp://username:[EMAIL PROTECTED]/data.zip')
f2 = file("data.zip", "wb")

while f1: # When to stop ?
 f2.write(f1.read(1024))

f1.close()
f2.close()

||

I can get the size & use it in a for loop :

length = float(f1.info().getheader("Content-Length"))
block = 1024

for i in range(0, int(length/block + 1)):
   s = f1.read(block)
   f2.write(s)|||


But this will not work for a file whose size is not known. So I need to 
get this done via EOF method.


Thanks
--
http://mail.python.org/mailman/listinfo/python-list