[Tutor] BadPickleGet error
Traceback (most recent call last): File /usr/share/rss2email/rss2email.py, line 748, in module else: run() File /usr/share/rss2email/rss2email.py, line 488, in run feeds, feedfileObject = load() File /usr/share/rss2email/rss2email.py, line 439, in load feeds = pickle.load(feedfileObject) cPickle.BadPickleGet: 2574'8 Could someone please explain what the number(s) after 'BadPickleGet' mean? I assume the data file is corrupt but the question is can I fix it with an editor using that info? -- We pay the debts of the last generation by issuing bonds payable by the next generation. -- Lawerence J.Peter Rick Pasottor...@niof.nethttp://www.niof.net ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: http://mail.python.org/mailman/listinfo/tutor
[Tutor] string encoding
I'm using BeautifulSoup to process a webpage. One of the fields has a unicode character in it. (It's the 'registered trademark' symbol.) When I try to write this string to another file I get this error: UnicodeEncodeError: 'ascii' codec can't encode characters in position 31-32: ordinal not in range(128) In the interpreter the offending string portion shows as: 'Realtors\xc2\xae'. How can I deal with this single string? The rest of the document works fine. -- Freedom can't be kept for nothing. If you set a high value on liberty, you must set a low value on everything else. -- Lucius Annaeus Seneca, 65 A.D. Rick Pasottor...@niof.nethttp://www.niof.net ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] string encoding
On Fri, Jun 18, 2010 at 12:24:25PM +1000, Lie Ryan wrote:
On 06/18/10 06:41, Rick Pasotto wrote:
I'm using BeautifulSoup to process a webpage. One of the fields has a
unicode character in it. (It's the 'registered trademark' symbol.) When
I try to write this string to another file I get this error:
UnicodeEncodeError: 'ascii' codec can't encode characters in position
31-32: ordinal not in range(128)
In the interpreter the offending string portion shows as:
'Realtors\xc2\xae'.
How can I deal with this single string? The rest of the document works
fine.
You need to tell BeautifulSoup the encoding of the HTML document. You
can encode this information in either the:
- (preferred) Encoding is specified externally from HTTP Header
ContentType declaration, e.g.:
Content-Type: text/html; charset=utf-8
- HTML ContentType declaration: e.g.
meta http-equiv=Content-Type content=text/html; charset=utf-8
The document has:
meta http-equiv=Content-Type content=text/html; charset=iso-8859-1
When I look at the document in vim and when I 'print' in python I see
the two characters of an acented capital A and the circled 'r'.
latin1word = 'Sacr\xe9 bleu!'
unicodeword = unicode(latin1word, 'latin-1')
print unicodeword
TypeError: decoding Unicode is not supported
If this works but Beautiful Soup doesn't, there's probably a bug in
Beautiful Soup. However, if this doesn't work, the problem's with your
Python setup. Python is playing it safe and not sending non-ASCII
characters to your terminal. There are two ways to override this behavior.
1. The easy way is to remap standard output to a converter that's not
afraid to send ISO-Latin-1 or UTF-8 characters to the terminal.
import codecs
import sys
streamWriter = codecs.lookup('utf-8')[-1]
sys.stdout = streamWriter(sys.stdout)
codecs.lookup returns a number of bound methods and other objects
related to a codec. The last one is a StreamWriter object capable of
wrapping an output stream.
Those four lines executed but I still get
TypeError: decoding Unicode is not supported
Remember, even if your terminal display is restricted to ASCII, you can
still use Beautiful Soup to parse, process, and write documents in UTF-8
and other encodings. You just can't print certain strings with print.
I can print the string fine. It's f.write(string_with_unicode) that fails with:
UnicodeEncodeError: 'ascii' codec can't encode characters in position 31-32:
ordinal not in range(128)
Shouldn't I be able to f.write() *any* 8bit byte(s)?
repr() gives: uRealtors\\xc2\\xae
BTW, I'm running python 2.5.5 on debian linux.
--
Making fun of born-again christians is like hunting dairy cows with a
high powered rifle and scope. -- P.J. O'Rourke
Rick Pasottor...@niof.nethttp://www.niof.net
___
Tutor maillist - Tutor@python.org
To unsubscribe or change subscription options:
http://mail.python.org/mailman/listinfo/tutor
[Tutor] working with email module
I subscribe to an email list that distributes image files. I'd like to automate the process of saving the images to individual files. I've got it mostly figured out with two exceptions. 1) Sometimes msg.filename returns 'None' even though msg.get_content_type returns 'image/jpeg' and the actual message (looking at it with a file viewer) *does* have a filename parameter. When I use the 'v' command in 'mutt' the filename shows so why does python fail to find it? 2) Is it possible to 'detach' a single subpart of a message? The docs show msg.attach() to create an email but I can't find anything that would allow deleting an attachment. I know how to delete an entire message from a mailbox. -- There is no medicine like hope, no incentive so great, and no tonic so powerful as expectation of something tomorrow. -- Orison Marden Rick Pasottor...@niof.nethttp://www.niof.net ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] MySQLdb field type
On Mon, Aug 10, 2009 at 07:22:03PM -0400, Kent Johnson wrote: On Mon, Aug 10, 2009 at 3:52 PM, Rick Pasottor...@niof.net wrote: Not exactly what I'm looking for. The type_code in the tuple is a number. For one of my tables I see the following numbers: 1, 2, 10, 252, 253, 254. What I wanted was a listing telling me which number was which type_code. Ah, right. I guess this will vary for different db modules. Which one are you using? I didn't realize there were different MySQLdb modules. -- Wit has truth in it; wise cracking is simply calisthenics with words. -- Dorothy Parker Rick Pasottor...@niof.nethttp://www.niof.net ___ Tutor maillist - Tutor@python.org http://mail.python.org/mailman/listinfo/tutor
[Tutor] MySQLdb field type
After executing a query that returns values, cursor.description contains a list of tuples, one tuple for each field. Each tuple has seven values consisting of (name,type_code,display_size,internal_size,precision,scale,null_ok). Where can I find a listing of what the various type_codes are? Do I need to experiment and make my own list? -- As empty vessels make the loudest sound, so they that have the least wit are the greatest blabbers. -- Plato Rick Pasottor...@niof.nethttp://www.niof.net ___ Tutor maillist - Tutor@python.org http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] MySQLdb field type
On Mon, Aug 10, 2009 at 02:50:01PM -0400, Kent Johnson wrote: On Mon, Aug 10, 2009 at 1:08 PM, Rick Pasottor...@niof.net wrote: After executing a query that returns values, cursor.description contains a list of tuples, one tuple for each field. Each tuple has seven values consisting of (name,type_code,display_size,internal_size,precision,scale,null_ok). Where can I find a listing of what the various type_codes are? Do I need to experiment and make my own list? See the section Type Objects and Constructors in the DB-API spec: http://dinsdale.python.org/dev/peps/pep-0249/ Not exactly what I'm looking for. The type_code in the tuple is a number. For one of my tables I see the following numbers: 1, 2, 10, 252, 253, 254. What I wanted was a listing telling me which number was which type_code. -- The dichotomy between personal liberties and property rights is a false one. Property does not have rights. People have rights. ... In fact, a fundamental interdependence exists between the personal right to liberty and the personal right to property. -- Potter Stewart, U.S. Supreme Court Justice, Lynch v. Household Finance Corp., 1972 Rick Pasottor...@niof.nethttp://www.niof.net ___ Tutor maillist - Tutor@python.org http://mail.python.org/mailman/listinfo/tutor
[Tutor] segmentation fault
I've got a script that I wrote several years ago and have been happily using daily. Suddenly when I try to run it I get a segmentation fault. I'm running debian testing so probably some recent update caused the breakage. How can I find out what's gone wrong? -- Happiness is having a large, loving, caring, close-knit family in another city. -- George Burns Rick Pasottor...@niof.nethttp://www.niof.net ___ Tutor maillist - Tutor@python.org http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] segmentation fault
On Sun, Jul 12, 2009 at 09:54:26AM -0700, Emile van Sebille wrote: On 7/12/2009 9:09 AM Rick Pasotto said... I've got a script that I wrote several years ago and have been happily using daily. Suddenly when I try to run it I get a segmentation fault. I'm running debian testing so probably some recent update caused the breakage. How can I find out what's gone wrong? If you're certain it's not otherwise environmental (out of space, bad memory or disk, packet flooding on network, etc), I'd try rolling back recent changes if that's convenient. Sometimes an strace on the process leads me in the right direction. If neither get you started, I'd try littering print statements through the code the track progress and try to narrow down where in the code it's happening. I've sometimes stumbled over a specific line causing the problem, and can eliminate the problem by refactoring. Be sure to report what you find upstream. There are probably other ways to isolate the problem depending on your focus. Maybe someone will point you in those directions. Of course, this is not entirely unexpected when you're running testing. It could resolve itself on its own. I downgraded 2.5.2-1.1 to 2.5.2-1 and all is well. I've sent a reportbug that includes the last several lines of an strace. Thanks for pointing me in the right direction. -- Upon common theatres, indeed, the applause of the audience is of more importance to the actors than their own approbation. But upon the stage of life, while conscience claps, let the world hiss! On the contrary if conscience disapproves, the loudest applauses of the world are of little value. -- John Adams Rick Pasottor...@niof.nethttp://www.niof.net ___ Tutor maillist - Tutor@python.org http://mail.python.org/mailman/listinfo/tutor
[Tutor] help with sorted()
I have a dictionary that looks like: d = {k:[v1,[v2,v3,v4]]}
v1,v2,v3,v4 are integers.
I want to print the dictionary sorted by v1, high to low.
sorted(d,operator.itemgetter(0),reverse=True)
gives me the keys in some order different than if I just looped through
the dictionary and not in key order but also not in any order that I can
see. It really appears random.
sorted(d) does give me a sorted list of keys.
How do I sort on v1? How would I sort on v3?
--
Many people think that if they were only in some other place, or had
some other job, they would be happy. Well, that is doubtful. So get as
much happiness out of what you are doing as you can and don't put off
being happy until some future date. -- Dale Carnegie
Rick Pasotto[EMAIL PROTECTED]http://www.niof.net
___
Tutor maillist - Tutor@python.org
http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] where to report a bug?
On Fri, Jul 25, 2008 at 03:11:57PM -0700, Dave Kuhlman wrote: On Fri, Jul 25, 2008 at 10:04:35AM -0400, Rick Pasotto wrote: I have a script that works fine on my linux machine but bombs out when run under windows using the exact same data files. The script downloads a file then unzips it and then scans the resulting file for certain records. Under Windows it gets a memory error in the unzip routine. Where should I send the error report? You might want to post your code here. Someone might be able to suggest a fix (to your code) or at least a work-around. One question -- Are you downloading the file and unzipping it with the zipfile module? Or, are you downloading the file and then writing the file to disk. If you are writing the file to disk, make sure that you open the file in binary mode, since a zipped file is binary data. If you fail to do so, your code will work on Linux/UNIX, but not on Windows. I've written code with that bug myself. That was a good thought but when I double checked the code I found that I was already opening the write file in 'wb' mode. Here's where things stand now. I searched comp.lang.python for 'zipfile' and found several messages that talked about this error. It seems that Windows doesn't handle memory allocation very well on large files (surprise, surprise). The module was written to deal with small files and so that was not a consideration in writing the module. One of the messages included a workaround that replaced the 'write' method with an 'explode' funtion that opperated on 8k chunks instead of the whole file. This had the added advantage of my being able to tap into the function to display a progress indicator. Again, it worked fine on my linux machine but failed on Windows, but this time with a different error. This error really surprises me. The new explode function uses the zipfile.ZipInfo.file_offset attribute. Windows reports that there is no such attribute. Turns out that while it exists in 2.4, which is what I am still running, it does *not* exist in 2.5, which is what the python.org Windows installer uses. I have 2.5 on my linux machine and when I run python2.5 there, ZipInfo.file_offset is not in the dir(zipfile.ZipInfo) list. All the other attributes match. Where did it go? And why? Was it intentionally removed? If so, why and how do I get its functionality? I haven't spotted any reference to the change in the changelists I've seen but maybe I haven't looked in the right places. What should be my next step? -- You are the only one who can use your ability. It is an awesome responsibility. -- Zig Zigler Rick Pasotto[EMAIL PROTECTED]http://www.niof.net ___ Tutor maillist - Tutor@python.org http://mail.python.org/mailman/listinfo/tutor
[Tutor] where to report a bug?
I have a script that works fine on my linux machine but bombs out when run under windows using the exact same data files. The script downloads a file then unzips it and then scans the resulting file for certain records. Under Windows it gets a memory error in the unzip routine. Where should I send the error report? -- With the first link, the chain is forged. The first speech censured, the first thought forbidden, the first freedom denied chains us all irrevocably. -- Judge Aaron Satie (As quoted by J.L. Picard) Rick Pasotto[EMAIL PROTECTED]http://www.niof.net ___ Tutor maillist - Tutor@python.org http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Updating MySQL Database
On Sun, Oct 07, 2007 at 06:07:45PM -0700, wormwood_3 wrote:
Hello all,
I have a script which takes data from a file or MySQL DB, looks up
some stuff, then can print results to console or file. I would also
like it to be able to update a MySQL database with the results. Does
anyone have any ideas on how to do this? I can update records just
fine, but what is the best way to do LOTS of updates aside from
running an update statement per record? Using that method, for
example, assuming I have a list of results, each line of the form
ip,fqdn:
for line in inputlist:
updatequery = update resultstable set fqdn = line.split(,)[1]
where ip = line.split(,)[0];
connection = MySQLdb.connect(db=self.todatabase, host=self.host,
user=self.user, passwd=self.passwd, port=int(self.port))
cursor = connection.cursor()
cursor.execute(updatequery)
queryresults = cursor.fetchall()
cursor.close()
connection.close()
But this means making a connection and query for every line of
results, which is a lot. Any ideas on optimization?
Good grief! Open the connection and create the cursor ONCE, use it as
many times as you need, and then close it (them). Closing the connection
automatically closes the cursor.
CAUTION: the following is not fully tested but I believe it to be correct.
connection = MySQLdb.connect(db=self.todatabase,host-self.host,
user=self.user, passwd=self.passwd, port=int(self.port))
cursor = connection.cursor()
for line in inputlist:
(ip,fqdn) = line.split(',')
updatequery = update resultstable set %s where ip = %s % (fqdn,ip)
cursor.execute(updatequery)
connection.close()
Alternatively you could do:
connection = MySQLdb.connect(db=self.todatabase,host-self.host,
user=self.user, passwd=self.passwd, port=int(self.port))
cursor = connection.cursor()
updatequery = update resultstable set %s where ip = %s
for line in inputlist:
vals = list(line.split(','))
vals.reverse()
cursor.execute(updatequery,vals)
connection.close()
I think the second version is more efficient.
or you could do:
connection = MySQLdb.connect(db=self.todatabase,host-self.host,
user=self.user, passwd=self.passwd, port=int(self.port))
cursor = connection.cursor()
updatequery = update resultstable set %s where ip = %s
for line in inputlist:
(ip,fqdn) = line.split(',')
cursor.execute(updatequery,(fqdn,ip))
connection.close()
Note that for these last two there is a comma between the arguments to
cursor.execute() rather than a percent.
--
I am only one. But I am one. I cannot do everything but I can do
something. And I will not let what I cannot do interfere with what
I can do. -- Edward E. Hule
Rick Pasotto[EMAIL PROTECTED]http://www.niof.net
___
Tutor maillist - Tutor@python.org
http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] shebang problem
On Sat, Nov 04, 2006 at 06:11:03PM -0600, Brian van den Broek wrote: Hi all, I'm still getting comfortable with Linux and this might be an OS rather than a python problem. I am trying to make a script directly executable. I've reviewed the 2nd ed of the Nutshell, and I cannot work out what I'm doing wrong. I'm running ubunutu 6.10 (edgy eft). Here's a copy past of my command line: [EMAIL PROTECTED]:~/test$ which python /usr/bin/python [EMAIL PROTECTED]:~/test$ ls -la shebangtest.py -rwxr-xr-- 1 brian brian 68 2006-11-04 02:29 shebangtest.py [EMAIL PROTECTED]:~/test$ cat shebangtest.py #!/usr/bin/python if __name__ == '__main__': print It works [EMAIL PROTECTED]:~/test$ shebangtest bash: shebangtest: command not found [EMAIL PROTECTED]:~/test$ shebangtest.py bash: shebangtest.py: command not found Most likely the current directory is not in your path. Type in 'echo $PATH' and look for a single period between colons. It is correct *not* to have the current directory in your path for security reasons. Now type in: './shebangtest' and your program should run. Putting the './' in front tells bash to use the command in the current directory. -- Rousseau was convinced that God, nature and man were wrong. I know that this opinion still sways many minds, but mine is not one of them. -- Frédéric Bastiat (1801-1850) Rick Pasotto[EMAIL PROTECTED]http://www.niof.net ___ Tutor maillist - Tutor@python.org http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] New GUI module
On Sun, Sep 04, 2005 at 03:28:11PM +0100, Alan G wrote: Tix has actually been included since Python 2.2! But there are still no *python* docs. The Tcl docs are very confusing. -- Happiness is the interval between periods of unhappiness. -- Don Marquis Rick Pasotto[EMAIL PROTECTED]http://www.niof.net ___ Tutor maillist - Tutor@python.org http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Split a string into characters
On Tue, Aug 23, 2005 at 01:18:43PM +1200, Hans Dushanthakumar wrote:
Hi,
A quick one...
How do I split a string like Hans into a list of characters
['H','a','n','s']?
Note that there are no spaces in the original string.
Str.split() without any arguments looks for whitespace as splitting
character. So, this doesn't serve the purpose.
And str.split() appears to be illegal.
list('Hans')
--
History will be kind to me, for I intend to write it.
-- Winston Churchill
Rick Pasotto[EMAIL PROTECTED]http://www.niof.net
___
Tutor maillist - Tutor@python.org
http://mail.python.org/mailman/listinfo/tutor
