Re: changing URLs in webpages, python solutions?
On Jan 18, 4:07 am, Simon Forman wrote: > Hey folks, > > I'm having trouble finding this through google so I figured I'd ask > here. > > I want to take a webpage, find all URLs (links, img src, etc.) and > rewrite them in-place, and I'd like to do it in python (pure python > preferred.) > > I know I could probably roll my own halfway decent solution in a day > or so, like with Beautiful Soup or something, but I'd much rather re- > use someone else's work if I can. I'm pretty sure this falls into the > category of "someone else has had, and solved, this problem before." > > Any recommendations? > > Thanks in advance, > ~Simon You can use code search engines. I founded there but now i don't have. -- http://mail.python.org/mailman/listinfo/python-list
Mod python set cookie for all subdomains
Hi..
I set cookie from www.domain.com but i'cant read this cookie from
subdomain.domain.com
How can i set cookie for all subdomains ?
My set cookie code is:
cookie = Cookie.Cookie('Login-Data', data)
cookie.expires = time.time() + 992
Cookie.add_cookie(req, cookie)
--
http://mail.python.org/mailman/listinfo/python-list
How can i find a file size on the web ?
Hi.. Can i find a file size witdhout download? For example: www.roche.com/rochea_z_sp.pdf How can i find its size with python ? I'm sorry for my bad english. -- http://mail.python.org/mailman/listinfo/python-list
Re: return image in mod python
On Nov 19, 5:32 pm, "Diez B. Roggisch" <[EMAIL PROTECTED]> wrote:
> Abandoned wrote:
> > On Nov 19, 12:36 pm, "Diez B. Roggisch" <[EMAIL PROTECTED]> wrote:
> >> Abandoned wrote:
> >> > Hi i have a problem.
>
> >> > def showimage(req):
> >> > from PIL import Image
> >> > im=Image.open("c:\image-2.jpg")
> >> > im.thumbnail((800,600), Image.ANTIALIAS)
> >> > req.sendfile(im)
>
> >> > give me some error.
>
> >> Really? I don't see any error. So there can't be one.
>
> >> Diez
>
> > It gives me this error:
> > req.sendfile(im)
>
> > TypeError: argument 1 must be string, not instance
>
> > Sendfile uses as:
> > req.sendfile("C:/img.jpg")
> > But i don't know how can i return when i use PIL before return ?
>
> Then don't use sendfile. I don't know mod_python, but I'm pretty sure you
> can output what you want somehow to the output stream. So, do that.
>
> Or go down the easy road, create a temp-file to store the image and send
> that.
>
> Diez- Hide quoted text -
>
> - Show quoted text -
Thank you very much.
Yes if i save this i show with sendfile but i must show this witdhout
save.
--
http://mail.python.org/mailman/listinfo/python-list
Re: return image in mod python
On Nov 19, 12:36 pm, "Diez B. Roggisch" <[EMAIL PROTECTED]> wrote:
> Abandoned wrote:
> > Hi i have a problem.
>
> > def showimage(req):
> > from PIL import Image
> > im=Image.open("c:\image-2.jpg")
> > im.thumbnail((800,600), Image.ANTIALIAS)
> > req.sendfile(im)
>
> > give me some error.
>
> Really? I don't see any error. So there can't be one.
>
> Diez
It gives me this error:
req.sendfile(im)
TypeError: argument 1 must be string, not instance
Sendfile uses as:
req.sendfile("C:/img.jpg")
But i don't know how can i return when i use PIL before return ?
--
http://mail.python.org/mailman/listinfo/python-list
return image in mod python
Hi i have a problem.
def showimage(req):
from PIL import Image
im=Image.open("c:\image-2.jpg")
im.thumbnail((800,600), Image.ANTIALIAS)
req.sendfile(im)
give me some error.
How can i return this image witdhout save ?
--
http://mail.python.org/mailman/listinfo/python-list
Re: how can i return a image in mod python ?
On Nov 18, 10:27 am, Arnaud Delobelle <[EMAIL PROTECTED]> wrote:
> On Nov 18, 6:46 am, Abandoned <[EMAIL PROTECTED]> wrote:
>
> > Hi..
> > I want to show the pictures with mod python directly.
>
> > def showimage(req):
> > some process...
> > open /var/www/a.jpg and print
>
> > for example if i open:
> > domain.com/a.py/showimage
> > It must show meimagedirectly (no redirect or html)
>
> > How can i do it ?
> > I'm sorry for my bad english.
> > Kind Regards
>
> How about:
>
> def showimage(req):
> req.content_type="image/jpeg" # Change to youimagetype
> req.sendfile("/path/to/image.jpg")
>returnapache.OK
>
> HTH
>
> BTW mod_python has its own list :)
>
> --
> Arnaud
Thank you but i have a another problem.
def showimage(req):
from PIL import Image
im=Image.open("c:\image-2.jpg")
im.thumbnail((800,600), Image.ANTIALIAS)
req.sendfile(im)
give me some error.
How can i return this image witdhout save ?
--
http://mail.python.org/mailman/listinfo/python-list
how can i return a image in mod python ?
Hi.. I want to show the pictures with mod python directly. def showimage(req): some process... open /var/www/a.jpg and print for example if i open: domain.com/a.py/showimage It must show me image directly (no redirect or html) How can i do it ? I'm sorry for my bad english. Kind Regards -- http://mail.python.org/mailman/listinfo/python-list
how can i return a image in mod python ?
Hi.. I want to show the pictures with mod python directly. ( it support cache and image headers) def showimage(req): some process... open /var/www/a.jpg and print How can i do it ? I'm sorry for my bad english. Kind Regards -- http://mail.python.org/mailman/listinfo/python-list
Re: Copy database with python..
On Nov 2, 4:19 pm, Paul McNett <[EMAIL PROTECTED]> wrote:
> Abandoned wrote:
> > Hi.
> > I want to copy my database but python give me error when i use this
> > command.
> > cursor.execute("pg_dump mydata > old.dump")
> > What is the problem ? And how can i copy the database with python ?
>
> You are just going to have to give us more to go on. Please post the
> entire traceback of the error that you see (copy/paste it from your
> terminal).
>
> You can't issue system commands using the cursor's execute() method and
> expect that to work. execute() is for executing SQL, DML, or DDL, not
> for doing shell stuff.
>
> Try:
>
> import os
> os.system("pg_dump mydata > /tmp/old.dump")
>
> but I'm not versed in postgressql, so this probably won't work exactly
> as written. You'd need to run that code from the server hosting the
> postgresql database.
>
> > Note: The database's size is 200 GB
>
> Well, then you may want to make sure you have enough room on the target
> volume before trying to dump the file! Note that the dump file could
> conceivably be much larger (or much smaller) than the database itself.
>
> --
> pkm ~http://paulmcnett.com
Are there any way to copy database without dump or any temp files ?
(If there is a temp my harddisk not enough for this operation :( )
--
http://mail.python.org/mailman/listinfo/python-list
Re: Copy database with python..
On Nov 2, 4:11 pm, Martin Sand Christensen <[EMAIL PROTECTED]> wrote:
> >>>>> "Abandoned" == Abandoned <[EMAIL PROTECTED]> writes:
>
> Abandoned> I want to copy my database but python give me error when i
> Abandoned> use this command. cursor.execute("pg_dump mydata > old.dump")
> Abandoned> What is the problem ?
>
> cursor.execute() is for executing SQL commands, and this is not an SQL
> command, but rather a shell command.
>
> Abandoned> And how can i copy the database with python ? Note: The
> Abandoned> database's size is 200 GB
>
> If you want to do this from Python, run it as a separate process.
>
> Martin
Yes i understand thank you.
Now i find that maybe help the other users.
import os
os.system("su postgres")
...
..
--
http://mail.python.org/mailman/listinfo/python-list
Copy database with python..
Hi.
I want to copy my database but python give me error when i use this
command.
cursor.execute("pg_dump mydata > old.dump")
What is the problem ? And how can i copy the database with python ?
Note: The database's size is 200 GB
--
http://mail.python.org/mailman/listinfo/python-list
Re: 3 number and dot..
On Oct 31, 10:50 pm, Roberto Bonvallet <[EMAIL PROTECTED]> wrote:
> On 31 oct, 16:58, Abandoned <[EMAIL PROTECTED]> wrote:
>
>
>
> > Hi..
> > I want to do this:
> > for examle:
> > 12332321 ==> 12.332.321
>
> > How can i do?
> >>> x = 12332321
> >>> '.'.join(''.join(i for n, i in g) for k, g in
> >>> groupby(enumerate(reversed(str(x))), lambda (n, i): n//3))[::-1]
> '12.332.321'
>
> --
> Roberto Bonvallet
I'm sorry but it give me error "no module named groupby"
My python version is 2.51
--
http://mail.python.org/mailman/listinfo/python-list
Re: 3 number and dot..
On Oct 31, 10:50 pm, Roberto Bonvallet <[EMAIL PROTECTED]> wrote:
> On 31 oct, 16:58, Abandoned <[EMAIL PROTECTED]> wrote:
>
>
>
> > Hi..
> > I want to do this:
> > for examle:
> > 12332321 ==> 12.332.321
>
> > How can i do?
> >>> x = 12332321
> >>> '.'.join(''.join(i for n, i in g) for k, g in
> >>> groupby(enumerate(reversed(str(x))), lambda (n, i): n//3))[::-1]
> '12.332.321'
>
> --
> Roberto Bonvallet
Thank you but it give me this error:
NameError: name 'groupby' is not defined
--
http://mail.python.org/mailman/listinfo/python-list
Re: 3 number and dot..
On Oct 31, 10:38 pm, Paul McGuire <[EMAIL PROTECTED]> wrote: > On Oct 31, 2:58 pm, Abandoned <[EMAIL PROTECTED]> wrote: > > > Hi.. > > I want to do this: > > for examle: > > 12332321 ==> 12.332.321 > > > How can i do? > >>> x = (12332321,) > >>> while (x[0]>0): x=divmod(x[0],1000)+x[1:] > ... > >>> x > (0, 12, 332, 321) > >>> ".".join(map(str,x[1:])) > > '12.332.321' > > -- Paul Hmm. When the number as 1023 the result is 1.23 :( How can i fix it ? -- http://mail.python.org/mailman/listinfo/python-list
Re: 3 number and dot..
On Oct 31, 10:38 pm, Paul McGuire <[EMAIL PROTECTED]> wrote: > On Oct 31, 2:58 pm, Abandoned <[EMAIL PROTECTED]> wrote: > > > Hi.. > > I want to do this: > > for examle: > > 12332321 ==> 12.332.321 > > > How can i do? > >>> x = (12332321,) > >>> while (x[0]>0): x=divmod(x[0],1000)+x[1:] > ... > >>> x > (0, 12, 332, 321) > >>> ".".join(map(str,x[1:])) > > '12.332.321' > > -- Paul It's very good thank you paul! -- http://mail.python.org/mailman/listinfo/python-list
Re: 3 number and dot..
On Oct 31, 10:18 pm, Paul McNett <[EMAIL PROTECTED]> wrote:
> Abandoned wrote:
> > Hi..
> > I want to do this:
> > for examle:
> > 12332321 ==> 12.332.321
>
> > How can i do?
>
> Assuming that the dots are always in the 3rd and 7th position in the string:
>
> def conv(s, sep="."):
>l = [s[0:3], s[3:6], s[6:]]
>return sep.join(l)
>
> print conv("12332321")
>
> --
> pkm ~http://paulmcnett.com
But it's starts from the end..
print conv("12332321")
123.323.21
This is wrong it would be 12.332.321
--
http://mail.python.org/mailman/listinfo/python-list
3 number and dot..
Hi.. I want to do this: for examle: 12332321 ==> 12.332.321 How can i do? -- http://mail.python.org/mailman/listinfo/python-list
Re: interesting threading result..
On Oct 29, 11:29 pm, Alan Franzoni
<[EMAIL PROTECTED]> wrote:
> Il Mon, 29 Oct 2007 09:08:01 -0700, Abandoned ha scritto:
>
> > I'm sorry my bad english.
> > King regards..
>
> You must have messed up with something. You didn't post a working snippet
> in first place, so you should do it now. I suppose you're working with some
> editor or IDE which mantains the very same instance of a python interpreter
> in memory; you probably have done something like
>
> end.append(current)
>
> at a certain point in your tests.
>
> I converted your pseudocode to python like that:
>
> import threading
>
> class GetData(threading.Thread):
>def __init__(self, name):
> threading.Thread.__init__(self)
> self.name = name
>
>def run(self):
> self.data={2:3, 3:4}
>
> nlist = []
> current = GetData("a")
> nlist.append(current)
> current.start()
> end=[]
> dd=nlist[0]
> dd.join()
> result=dd.data
> print result
> end.append(result)
> print end
>
> and I just get the result I would expect:
>
> {2: 3, 3: 4}
> [{2: 3, 3: 4}]
>
> --
> Alan Franzoni <[EMAIL PROTECTED]>
> -
> Togli .xyz dalla mia email per contattarmi.
> Remove .xyz from my address in order to contact me.
> -
> GPG Key Fingerprint (Key ID = FE068F3E):
> 5C77 9DC3 BD5B 3A28 E7BC 921A 0255 42AA FE06 8F3E
Thanks you..
Now this is my real code and the problem continue.
What is the mistake?
import threading
class GetData(threading.Thread):
def __init__(self, name):
threading.Thread.__init__(self)
self.name = name
def run(self):
self.data={2:3, 3:4}
i=0
datas={"n": [['msn']]}
nglist=nlist={0: list()}
for x in query:
tip=x
for a in datas[x]:
for o in a:
if len(o)>1:
current = GetData(o)
nglist[i].append(current)
current.start()
i=i+1
for x in range(0,len(nglist)):
for jk in range(0,len(nglist[x])):
jkl=nglist[x][jk]
jkl.join()
sonuc=jkl.data
nlist[x].append(sonuc)
print nlist
{0: [, {2: 3, 3: 4}]}
--
http://mail.python.org/mailman/listinfo/python-list
interesting threading result..
Hi..
I have a interesting threading result..
class GetData(threading.Thread):
def __init__(self, name):
threading.Thread.__init__(self)
self.name = name
def run(self):
self.data={2:3, 3:4...}
current = GetData(a)
nlist.append(current)
current.start()
end=[]
dd=nlist[0]
dd.join()
result=dd.data
print result
(and it gives me : {2:3, 3:4...} )
end.append(result)
print end
(and it gives me: [, {2:3, 3:4...}] )
What happen ??
What is the , ?
In the first print there isn't it but second print it is in there.
I don't want ..
How can i escape from it ?
I'm sorry my bad english.
King regards..
--
http://mail.python.org/mailman/listinfo/python-list
Which index can i use ?
Hi.. I want to do index in postgresql & python. My table: id(int) | id2(int) | w(int) | d(int) My query: select id, w where id=x and id2=y (sometimes and d=z) I have too many insert and select operation on this table. And which index type can i use ? Btree, Rtree, Gist or Hash ? Also I want to unique (id, id2).. Now this is my index. is it give me good performance ? CREATE UNIQUE INDEX ind1 ON test USING btree (id, id2) CREATE INDEX ind2 ON test USING btree (id) CREATE INDEX ind3 ON test USING btree (id2) CREATE INDEX ind4 ON test USING btree (w) CREATE INDEX ind5 ON test USING btree (d) I'm too sorry my bad english. King regards.. -- http://mail.python.org/mailman/listinfo/python-list
Re: insert string problems..
On Oct 28, 9:45 am, Steven D'Aprano <[EMAIL PROTECTED]
cybersource.com.au> wrote:
> On Sun, 28 Oct 2007 00:24:34 -0700, Abandoned wrote:
> > Hi..
> > I want to insert some data to postgresql.. My insert code:
> > yer="019"
> > cursor.execute("INSERT INTO ids_%s (id) VALUES (%s)", (yer, id)) I don't
> > want to use % when the insert operation.
>
> > in this code give me this error:
> > psycopg2.ProgrammingError: syntax error at or near "'019'" LINE 1:
> > SELECT link_id from linkkeywords_'019'
>
> > if i do int(yer) , 019 change to 19 .. How can i do int yer string with
> > 0 ?
>
> Integers with a leading 0 are interpreted as base 8 (octal). You can't
> write 019, because there is no digit "9" in octal.
>
> Why do you need a leading zero?
>
> --
> Steven.
Thank you steven.
I must use 019 beacause my system algoritm in this way..
And what about second question ?
--
http://mail.python.org/mailman/listinfo/python-list
Re: insert string problems..
Also..
a="123,245,1235,663"
cursor.execute("SELECT id, name FROM data WHERE id in (%s)", (a,))
In this query must be:
SELECT id, name FROM data WHERE id in (123,245,1235,663)
but it looks:
SELECT id, name FROM data WHERE id in ("123,245,1235,663")
How can i delete " ?
--
http://mail.python.org/mailman/listinfo/python-list
insert string problems..
Hi..
I want to insert some data to postgresql..
My insert code:
yer="019"
cursor.execute("INSERT INTO ids_%s (id) VALUES (%s)", (yer, id))
I don't want to use % when the insert operation.
in this code give me this error:
psycopg2.ProgrammingError: syntax error at or near "'019'"
LINE 1: SELECT link_id from linkkeywords_'019'
if i do int(yer) , 019 change to 19 ..
How can i do int yer string with 0 ?
--
http://mail.python.org/mailman/listinfo/python-list
Re: How can i do proxy in urllib.urlopen?
I read the urllib reference and set http_proxy="my proxy". But it didn't work. I can't even get authenticated. Is there anyway that we can set the proxy? -- http://mail.python.org/mailman/listinfo/python-list
How can i do proxy in urllib.urlopen?
Hi i want to do proxy in urllib.urlopen how can i do this ? And which proxy type have i can prefer ? -- http://mail.python.org/mailman/listinfo/python-list
threading problem..
Hi.. I want to threading but i have a interesting error.. == class SelectAll(threading.Thread): def __init__(self, name): threading.Thread.__init__(self) self.name = name #kelime def run(self): self.result=... nglist=[] current = SelectAll(name) nglist.append(current) current.start() for aaa in nglist: aaa.join() = and it gives me this error: aaa.join() AttributeError: 'dict' object has no attribute 'join' What is the problem i can't understand this error :( -- http://mail.python.org/mailman/listinfo/python-list
Delete all not allowed characters..
Hi.. I want to delete all now allowed characters in my text. I use this function: def clear(s1=""): if s1: allowed = [u'+',u'0',u'1',u'2',u'3',u'4',u'5',u'6',u'7',u'8',u'9',u' ', u'Ş', u'ş', u'Ö', u'ö', u'Ü', u'ü', u'Ç', u'ç', u'İ', u'ı', u'Ğ', u'ğ', 'A', 'C', 'B', 'E', 'D', 'G', 'F', 'I', 'H', 'K', 'J', 'M', 'L', 'O', 'N', 'Q', 'P', 'S', 'R', 'U', 'T', 'W', 'V', 'Y', 'X', 'Z', 'a', 'c', 'b', 'e', 'd', 'g', 'f', 'i', 'h', 'k', 'j', 'm', 'l', 'o', 'n', 'q', 'p', 's', 'r', 'u', 't', 'w', 'v', 'y', 'x', 'z'] s1 = "".join(ch for ch in s1 if ch in allowed) return s1 And my problem this function replace the character to "" but i want to " " for example: input: Exam%^^ple output: Exam ple I want to this output but in my code output "Example" How can i do quickly because the text is very long.. -- http://mail.python.org/mailman/listinfo/python-list
Parallel insert to postgresql with thread
Hi..
I use the threading module for the fast operation. But i have some
problems..
This is my code sample:
=
conn =
psycopg2.connect(user='postgres',password='postgres',database='postgres')
cursor = conn.cursor()
class paralel(Thread):
def __init__ (self, veriler, sayii):
Thread.__init__(self)
def run(self):
save(a, b, c)
def save(a,b,c):
cursor.execute("INSERT INTO keywords (keyword) VALUES
('%s')" % a)
conn.commit()
cursor.execute("SELECT
CURRVAL('keywords_keyword_id_seq')")
idd=cursor.fetchall()
return idd[0][0]
def start(hiz):
datas=[]
for a in datas:
current = paralel(a, sayii)
current.start()
==
And it gives me different errors to try parallel insert. My querys
work in normal operation but in paralel don't work.
How can i insert data to postgresql the same moment ?
errors:
no results to fetch
cursor already closed
--
http://mail.python.org/mailman/listinfo/python-list
Re: How can i protect text format ?
On Oct 23, 4:41 pm, "J. Clifford Dyer" <[EMAIL PROTECTED]> wrote: > On Tue, Oct 23, 2007 at 06:30:18AM -0700, Abandoned wrote regarding How can i > protect text format ?: > > > > > Hi.. > > I want to do a forum with python but i have a problem.. > > > I have a and i write: > > line 1 hi > > line 2 how r u > > > And then i save to this database ( colomn data type is text) > > And than it looks "line 1 hi line 2 how r u".. > > How can i protect \n characters ? > > > -- > >http://mail.python.org/mailman/listinfo/python-list > > Are you sure they're getting lost on the way into the DB? Could they be > getting lost on the way out? Say... into an HTML page? > > You need to pinpoint your problem more clearly to get a helpful response. > > Cheers, > Cliff Yes when i selected the data it have one row (there isn't \n) -- http://mail.python.org/mailman/listinfo/python-list
How can i protect text format ?
Hi.. I want to do a forum with python but i have a problem.. I have a and i write: line 1 hi line 2 how r u And then i save to this database ( colomn data type is text) And than it looks "line 1 hi line 2 how r u".. How can i protect \n characters ? -- http://mail.python.org/mailman/listinfo/python-list
Re: Selection error...
> cursor.execute("SELECT id from templinks where url=%s", (URL,) )
Yes i already try this before ask.
URL.decode('latin-1').encode('utf-8') to transcode it into utf-8.
i know this but how do you know the html encoding's a latin-1 ? Html
encoding is to be iso-8859-9 or ascii or cp1254 ... For this reason
URL.decode('latin-1').encode('utf-8') must give me error. I'm trying
now and whet it finished i write the result here.
Thank you very much for your helps.
--
http://mail.python.org/mailman/listinfo/python-list
Selection error...
Hi..
I want to select datas but some datas give me this error:
psycopg2.ProgrammingError: invalid byte sequence for encoding "UTF8":
0xc720
HINT: This error can also happen if the byte sequence does not match
the encoding expected by the server, which is controlled by
"client_encoding".
My select command:
cursor.execute("SELECT id from templinks where url='%s'"% URL)
This problem occured only select data when i try insert the same data
it doesn't give me error..
Note: My database is UTF-8 and the URLS come from different web sites'
html code
How can i fix this problem ?
I'm sorry my bad english..
--
http://mail.python.org/mailman/listinfo/python-list
Re: How can i store pickle dumps data ?
On Oct 20, 5:32 pm, "Diez B. Roggisch" <[EMAIL PROTECTED]> wrote:
> Abandoned schrieb:
>
>
>
>
>
> > Hi..
> > I have some dictionary as {2:3, 4:6, 5:7.}
> > I want to pickle to these dictionary and save to database but i
> > couldn't..
>
> > My code for adding..
> > a=eval(dict)
> > a=pickle.dumps(a, -1)
> > cursor2.execute("INSERT INTO cache VALUES ('%s', %s)", (x[0],a))
> > conn2.commit()
>
> > i try in UTF-8 postresql database (data type BYTEA) and it gives me
> > this error:
> >> psycopg2.ProgrammingError: invalid byte sequence for encoding "UTF8":
> >> 0x80
> >> HINT: This error can also happen if the byte sequence does not match
> >> the encoding expected by the server, which is controlled by
> >> "client_encoding".
>
> > i try in LATIN5 postgresql database (data type BYTEA) and it add first
>
> http://mail.python.org/pipermail/python-list/2004-September/280867.html
>
> Diez- Hide quoted text -
>
> - Show quoted text -
Yes this is wonderfull.
I am very happy thank you very much Diez.
--
http://mail.python.org/mailman/listinfo/python-list
How can i store pickle dumps data ?
Hi..
I have some dictionary as {2:3, 4:6, 5:7.}
I want to pickle to these dictionary and save to database but i
couldn't..
My code for adding..
a=eval(dict)
a=pickle.dumps(a, -1)
cursor2.execute("INSERT INTO cache VALUES ('%s', %s)", (x[0],a))
conn2.commit()
i try in UTF-8 postresql database (data type BYTEA) and it gives me
this error:
> psycopg2.ProgrammingError: invalid byte sequence for encoding "UTF8":
> 0x80
> HINT: This error can also happen if the byte sequence does not match
> the encoding expected by the server, which is controlled by
> "client_encoding".
i try in LATIN5 postgresql database (data type BYTEA) and it add first
10 element and than gives me this error:
cursor2.execute("INSERT INTO cache VALUES ('%s', %s)", (x[0],a))
psycopg2.ProgrammingError: invalid input syntax for type bytea
i try in LATIN5 (data type TEXT) and it add all element but when i
want to load it give me this error:
pickle.loads(SELECTED_DATA);
EOFError
Now how can i store the pickle data ?
Which encoding and which data type ?
Please help me i'm sorry my bad english.
King Regards..
--
http://mail.python.org/mailman/listinfo/python-list
Re: Convert string to command..
On Oct 18, 8:53 pm, Hrvoje Niksic <[EMAIL PROTECTED]> wrote:
> Abandoned <[EMAIL PROTECTED]> writes:
> > > When you load it, convert the string to dict with cPickle.loads
> > > instead of with eval.
>
> > Yes i understand and this very very good ;)
>
> Good! :-)
>
> > psycopg2.ProgrammingError: invalid byte sequence for encoding "UTF8":
> > 0x80
> > HINT: This error can also happen if the byte sequence does not match
> > the encoding expected by the server, which is controlled by
> > "client_encoding".
>
> Use a different column type for cache2's column, one more appropriate
> for storing binary characters (perhaps BYTEA for Postgres). Don't
> forget to also use a bind variable, something like:
>
> cursor.execute("INSERT INTO cache2 VALUES (?)", a)
>
> Using "INSERT ... ('%s')" % (a) won't work, since the huge binary
> string in a can contain arbitrary characters, including the single
> quote.
I tryed:
cursor.execute("INSERT INTO cache2 VALUES (?)", a)
and
cursor.execute("INSERT INTO cache2 VALUES (%s)", (a,) )
but the result is same..
> psycopg2.ProgrammingError: invalid byte sequence for encoding "UTF8":
> 0x80
> HINT: This error can also happen if the byte sequence does not match
> the encoding expected by the server, which is controlled by
> "client_encoding".
--
http://mail.python.org/mailman/listinfo/python-list
Re: Convert string to command..
On Oct 18, 7:40 pm, Hrvoje Niksic <[EMAIL PROTECTED]> wrote:
> Abandoned <[EMAIL PROTECTED]> writes:
> > Sorry i can't understand :(
> > Yes my database already has data in the "{..}" format and i select
> > this and i want to use it for dictionary..
>
> But, do you use Python to create that data? If so, simply convert it
> to pickle binary format instead of "{...}". As explained in my other
> post:
>
> > id-1 | all
> > 56{68:66, 98:32455, 62:655}
>
> If you use Python to create this cache table, then simply don't dump
> it as a dictionary, but as a pickle:
>
> id-1 | all
> 56
>
> When you load it, convert the string to dict with cPickle.loads
> instead of with eval.
Yes i understand and this very very good ;)
But i have a problem..
a=eval(a)
a=pickle.dumps(a, -1)
cursor.execute("INSERT INTO cache2 VALUES ('%s')" % (a))
conn.commit()
and give me error:
psycopg2.ProgrammingError: invalid byte sequence for encoding "UTF8":
0x80
HINT: This error can also happen if the byte sequence does not match
the encoding expected by the server, which is controlled by
"client_encoding".
--
http://mail.python.org/mailman/listinfo/python-list
Re: Convert string to command..
On Oct 18, 7:02 pm, Hrvoje Niksic <[EMAIL PROTECTED]> wrote:
> Abandoned <[EMAIL PROTECTED]> writes:
> > import cPickle as pickle
> > a="{2:3,4:6,2:7}"
> > s=pickle.dumps(a, -1)
> > g=pickle.loads(s);
> > print g
> > '{2:3,4:6,2:7}'
>
> > Thank you very much for your answer but result is a string ??
>
> Because you gave it a string. If you give it a dict, you'll get a
> dict:
>
> >>> import cPickle as pickle
> >>> a = {1:2, 3:4}
> >>> s = pickle.dumps(a, -1)
> >>> g = pickle.loads(s)
> >>> g
>
> {1: 2, 3: 4}
>
> If your existing database already has data in the "{...}" format, then
> eval it only the first time. Then you'll get the dict which you can
> cache thruogh the use of dumps/loads.
Sorry i can't understand :(
Yes my database already has data in the "{..}" format and i select
this and i want to use it for dictionary..
in your example:
first data is a string
finally data is already string
I want to command like eval. (eval is not good because it is slow for
my project)
--
http://mail.python.org/mailman/listinfo/python-list
Re: Convert string to command..
On Oct 18, 6:57 pm, "Diez B. Roggisch" <[EMAIL PROTECTED]> wrote:
> Abandoned wrote:
> > On Oct 18, 6:35 pm, "Diez B. Roggisch" <[EMAIL PROTECTED]> wrote:
> >> Abandoned wrote:
> >> > On Oct 18, 6:14 pm, "Diez B. Roggisch" <[EMAIL PROTECTED]> wrote:
> >> >> Abandoned wrote:
> >> >> > Thanks you all answer..
> >> >> > But "eval" is very slow at very big dictionary {2:3,4:5,6:19}
> >> >> > (100.000 elements)
> >> >> > Is there any easy alternative ?
>
> >> >> How big? How slow? For me, a 1-element list takes 0.04 seconds to
> >> >> be parsed. Which I find fast.
>
> >> >> Diez
>
> >> > 173.000 dict elements and it tooks 2.2 seconds this very big time for
> >> > my project
>
> >> Where does the data come from?
>
> >> Diez
>
> > Data come from database..
> > I want to cache to speed up my system and i save the dictionary to
> > database for speed up but eval is very slow for do this.
> > Not: 2.2 second only eval operation.
>
> Does the dictionary change often?
>
> And you should store a pickle to the database then. Besides, making a
> database-query of that size (after all, we're talking a few megs here) will
> take a while as well - so are you SURE the 2.2 seconds are a problem? Or is
> it just that you think they are?
>
> Diez- Hide quoted text -
>
> - Show quoted text -
I'm very confused :(
I try to explain main problem...
I have a table like this:
id-1 | id-2 | value
23 24 34
56 68 66
56 98 32455
55 62 655
56 28 123
( 3 millions elements)
I select where id=56 and 100.000 rows are selecting but this took 2
second. (very big for my project)
I try cache to speed up this select operation..
And create a cache table:
id-1 | all
56{68:66, 98:32455, 62:655}
When i select where id 56 i select 1 row and its took 0.09 second but
i must convert text to dictionary..
Have you got any idea what can i do this conver operation ?
or
Have you got any idea what can i do cache for this table ?
--
http://mail.python.org/mailman/listinfo/python-list
Re: Convert string to command..
On Oct 18, 6:51 pm, Marc 'BlackJack' Rintsch <[EMAIL PROTECTED]> wrote:
> On Thu, 18 Oct 2007 08:41:30 -0700, Abandoned wrote:
> > import cPickle as pickle
> > a="{2:3,4:6,2:7}"
> > s=pickle.dumps(a, -1)
> > g=pickle.loads(s);
> > print g
> > '{2:3,4:6,2:7}'
>
> > Thank you very much for your answer but result is a string ??
>
> In Python terms yes, strings in Python can contain any byte value. If you
> want to put this into a database you need a BLOB column or encode it as
> base64 or something similar more ASCII safe.
>
> Ciao,
> Marc 'BlackJack' Rintsch
'{2:3,4:6,2:7}' already in database, i select this and convert to real
dictionary..
--
http://mail.python.org/mailman/listinfo/python-list
Re: Convert string to command..
On Oct 18, 6:35 pm, "Diez B. Roggisch" <[EMAIL PROTECTED]> wrote:
> Abandoned wrote:
> > On Oct 18, 6:14 pm, "Diez B. Roggisch" <[EMAIL PROTECTED]> wrote:
> >> Abandoned wrote:
> >> > Thanks you all answer..
> >> > But "eval" is very slow at very big dictionary {2:3,4:5,6:19}
> >> > (100.000 elements)
> >> > Is there any easy alternative ?
>
> >> How big? How slow? For me, a 1-element list takes 0.04 seconds to be
> >> parsed. Which I find fast.
>
> >> Diez
>
> > 173.000 dict elements and it tooks 2.2 seconds this very big time for
> > my project
>
> Where does the data come from?
>
> Diez
Data come from database..
I want to cache to speed up my system and i save the dictionary to
database for speed up but eval is very slow for do this.
Not: 2.2 second only eval operation.
--
http://mail.python.org/mailman/listinfo/python-list
Re: Convert string to command..
On Oct 18, 6:26 pm, Hrvoje Niksic <[EMAIL PROTECTED]> wrote:
> Abandoned <[EMAIL PROTECTED]> writes:
> > 173.000 dict elements and it tooks 2.2 seconds this very big time
> > for my project
>
> If you're generating the string from Python, use cPickle instead.
> Much faster:
>
> >>> import time
> >>> d = dict((i, i+1) for i in xrange(17))
> >>> len(d)
> 17
> >>> s=repr(d)
> >>> t0 = time.time(); d2 = eval(s); t1 = time.time()
> >>> t1-t0
> 1.5457899570465088
> >>> import cPickle as pickle
> >>> s = pickle.dumps(d, -1)
> >>> len(s)
> 1437693
> >>> t0 = time.time(); d2 = pickle.loads(s); t1 = time.time()
> >>> t1-t0
>
> 0.060307979583740234>>> len(d2)
>
> 17
>
> That is 25x speedup. Note that cPickle's format is binary. Using the
> textual format makes for more readable pickles, but reduces the
> speedup to "only" 9.5x on my machine.
>
> P.S.
> Before someone says that using pickle is unsafe, remember that he is
> considering *eval* as the alternative. :-)
import cPickle as pickle
a="{2:3,4:6,2:7}"
s=pickle.dumps(a, -1)
g=pickle.loads(s);
print g
'{2:3,4:6,2:7}'
Thank you very much for your answer but result is a string ??
--
http://mail.python.org/mailman/listinfo/python-list
Re: Convert string to command..
On Oct 18, 6:14 pm, "Diez B. Roggisch" <[EMAIL PROTECTED]> wrote:
> Abandoned wrote:
> > Thanks you all answer..
> > But "eval" is very slow at very big dictionary {2:3,4:5,6:19}
> > (100.000 elements)
> > Is there any easy alternative ?
>
> How big? How slow? For me, a 1-element list takes 0.04 seconds to be
> parsed. Which I find fast.
>
> Diez
173.000 dict elements and it tooks 2.2 seconds this very big time for
my project
--
http://mail.python.org/mailman/listinfo/python-list
Re: Convert string to command..
Thanks you all answer..
But "eval" is very slow at very big dictionary {2:3,4:5,6:19}
(100.000 elements)
Is there any easy alternative ?
--
http://mail.python.org/mailman/listinfo/python-list
Convert string to command..
I want to convert a string to command.. For example i have a string: a="['1']" I want to do this list.. How can i do ? -- http://mail.python.org/mailman/listinfo/python-list
Re: Order by value in dictionary
I tried these: def largest_sort(d, n): return sorted(d, key=d.__getitem__, reverse=True)[:n] def largest_heap(d, n): return heapq.nlargest(n, d, d.__getitem__) def sortt(d): sorted_items = sorted((item[1], item[0]) for item in d.iteritems(), reverse=True) return map(operator.itemgetter(1), sorted_items) . and the fastest method is second ;) -- http://mail.python.org/mailman/listinfo/python-list
Re: Order by value in dictionary
Very very thanks everbody.. These are some method.. Now the fastest method is second.. 1 === def sortt(d): items=d.items() backitems=[ [v[1],v[0]] for v in items] backitems.sort() #boyut=len(backitems) #backitems=backitems[boyut-500:] a=[ backitems[i][1] for i in range(0,len(backitems))] a.reverse() return a 2 = import operator def sortt(d): backitems=d.items() boyut=len(backitems) backitems=backitems[boyut-500:] backitems=sorted(backitems, key=operator.itemgetter(1)) a=[ backitems[i][0] for i in range(0,len(backitems))] a.reverse() return a 3 = def sortt(d): backitems=d.items() backitems.sort(lambda x,y:cmp(x[1],y[1])) backitems=sorted(backitems, key=operator.itemgetter(1)) a=[ backitems[i][0] for i in range(0,len(backitems))] a.reverse() return a == 4 === import heapq def sortt(d): backitems=d.items() backitems=heapq.nlargest(1000, backitems, operator.itemgetter(1)) a=[ backitems[i][0] for i in range(0,len(backitems))] a.reverse() return a -- http://mail.python.org/mailman/listinfo/python-list
Order by value in dictionary
Hi..
I have a dictionary like these:
a={'a': '1000', 'b': '18000', 'c':'40', 'd': '600'} .. 100.000
element
I want to sort this by value and i want to first 100 element..
Result must be:
[b, a, d, c .] ( first 100 element)
I done this using FOR and ITERATOR but it tooks 1 second and this is
very big time to my project.
I want to learn the fastest method..
I'm sorry my bad english.
Please help me.
King regards..
--
http://mail.python.org/mailman/listinfo/python-list
int to str in list elements..
Hi.. I have a list as a=[1, 2, 3 ] (4 million elements) and b=",".join(a) than TypeError: sequence item 0: expected string, int found I want to change list to a=['1','2','3'] but i don't want to use FOR because my list very very big. I'm sorry my bad english. King regards -- http://mail.python.org/mailman/listinfo/python-list
Re: Pil image module, "mode" bug..
On Oct 7, 4:47 pm, Michal Bozon <[EMAIL PROTECTED]> wrote:
> On Sun, 07 Oct 2007 06:03:06 -0700, Abandoned wrote:
> > Hi..
> > I find the picture color with:
> > im=Image.open("/%s" %name)
> > color=im.mode #p=black & beyaz rgb=color L=grey
>
> > This usually work true but in these pictures:
> >http://malatya.meb.gov.tr/images/alt/ilsis_logo.gif
> >http://malatya.meb.gov.tr/images/meb.gif
>
> > Say me P (black&white) but these pictures are color..
>
> > What is the reason of this ?
>
> > I'm sorry my bad english
>
> P does mean palette, black&white is a special case of palette, with two
> colors.
How can i understand the picture color ? (black &white or color or
grey)
--
http://mail.python.org/mailman/listinfo/python-list
Pil image module, "mode" bug..
Hi..
I find the picture color with:
im=Image.open("/%s" %name)
color=im.mode #p=black & beyaz rgb=color L=grey
This usually work true but in these pictures:
http://malatya.meb.gov.tr/images/alt/ilsis_logo.gif
http://malatya.meb.gov.tr/images/meb.gif
Say me P (black&white) but these pictures are color..
What is the reason of this ?
I'm sorry my bad english
--
http://mail.python.org/mailman/listinfo/python-list
configure urllib.urlretrieve timeout
Hi.. I want to set 30 second urllib.urlretrieve timeout.. Because if urllib.urlretrieve can't connect to page wait 1-2 hour... I download the images to my server with urlretrieve if you know the better way please help me. I'm sorry my bad english.. -- http://mail.python.org/mailman/listinfo/python-list
Segmentation fault..
Hi.. I run a my script and 3-4 minutes later give me an error "segmentation fault". What is the reason of this error ? I use in my script: Threading Psycopg2 open & write to txt urlopen My platform is ubuntu linux. I'm sorry my bad english. King regards.. -- http://mail.python.org/mailman/listinfo/python-list
remove list elements..
Hi.. I have a problem.. list1=[11, 223, 334, 4223...] 1 million element list2=[22,223,4223,2355...] 500.000 element I want to difference list1 to list2 but order very importent.. My result must be: list3=[11,334,...] I do this use FOR easly but the speed very imported for me. I want to the fastest method please help me. I'm sorry my bad english. King regards.. -- http://mail.python.org/mailman/listinfo/python-list
Re: Boolean parser..
http://pyparsing.wikispaces.com/space/showimage/searchparser.py this is searchparser but i can't understant to use this. Can anybody explain this how to use ? -- http://mail.python.org/mailman/listinfo/python-list
Re: Boolean parser..
On 4 Ekim, 22:29, Abandoned <[EMAIL PROTECTED]> wrote: > Hi.. > I try a boolean parser in python since 1 weak.. But i can't do this > because this is very complicated :( > Do you know any blooean parser script in python or how do i write a > boolean parser ? > example query: ((google or yahoo) or (live msn)) not web > I'm sorry my bad english. > King Regards.. This is a example written in perl " http://www.planet-source-code.com/vb/scripts/ShowCode.asp?txtCodeId=350&lngWId=6"; but i don't find any python example -- http://mail.python.org/mailman/listinfo/python-list
Boolean parser..
Hi.. I try a boolean parser in python since 1 weak.. But i can't do this because this is very complicated :( Do you know any blooean parser script in python or how do i write a boolean parser ? example query: ((google or yahoo) or (live msn)) not web I'm sorry my bad english. King Regards.. -- http://mail.python.org/mailman/listinfo/python-list
Duplicate content filter..
Hi.. I'm working a search engine project now. And i have a problem. My problem is Duplicate Contents.. I can find the percentage of similarity between two pages but i have a 5 millions index and i search 5 million page contents to find one duplicate :( I want to a idea for how can i find duplicate pages quickly and fast ? Please help me, i'm sorry my bad english. King regards.. -- http://mail.python.org/mailman/listinfo/python-list
Re: Combine two dictionary...
On 1 Ekim, 20:41, Carsten Haese <[EMAIL PROTECTED]> wrote:
> On Mon, 2007-10-01 at 10:24 -0700, Abandoned wrote:
> > Hi..
> > dict1={1: 4, 3: 5}... and 2 millions element
> > dict2={3: 3, 8: 6}... and 3 millions element
>
> > I want to combine dict1 and dict2 and i don't want to use FOR because
> > i need to performance.
>
> You'll have to be a bit more precise here about how and where you want
> to "combine". Should the result be a new dictionary or should it replace
> one of dict1 or dict2? You're also not saying how you want conflicts
> between dict1 and dict2 should be resolved. For example, both dict1 and
> dict2 say something about the key 3. One says 5, the other says 3.
> Should the combined dict say 3 or 5 or (3,5) or maybe even 4?
>
> Also, based on your earlier posts from today, I'm wondering if those
> dicts come from executing SQL queries. If so, you might want to consider
> combining the results in SQL (using a UNION query) instead of combining
> them in Python.
>
> --
> Carsten Haesehttp://informixdb.sourceforge.net
I want to total score..
For example
> > dict1={1: 4, 3: 5}... and 2 millions element
> > dict2={3: 3, 8: 6}... and 3 millions element
result should be dict3={1:4, 3:8, 8:6}
--
http://mail.python.org/mailman/listinfo/python-list
Combine two dictionary...
Hi..
dict1={1: 4, 3: 5}... and 2 millions element
dict2={3: 3, 8: 6}... and 3 millions element
I want to combine dict1 and dict2 and i don't want to use FOR because
i need to performance.
I'm sorry my bed english.
King regards..
--
http://mail.python.org/mailman/listinfo/python-list
Re: Select as dictionary...
Also if i need a list id what can i do ?
aia.execute("SELECT id, w from list")
links=aia.fetchall()
I want to..
idlist=[1, 2, 3] ( I don't want to use FOR and APPEND because the
query have 2 million result and i want to speed)
--
http://mail.python.org/mailman/listinfo/python-list
Re: Select as dictionary...
On 1 Ekim, 18:13, Bruno Desthuilliers wrote:
> J. Clifford Dyer a écrit :
>
> > On Mon, Oct 01, 2007 at 03:50:59PM +0200, Bruno Desthuilliers wrote
> > regarding Re: Select as dictionary...:
> >> IIRC, postgres' db-api connector (well, at least one of them - I don't
> >> know which one you're using) has a DictCursor. You should find all you
> >> want to know in the relevant doc.
>
> > Correct me if I'm wrong, but I think the DictCursor would output the
> > following:
>
> > [{'id': 1, 'value': 5}, {'id': 2, 'value': 5}, ...]
>
> > But the OP wanted one dict with IDs as keys:
>
> > {1: 5, 2: 5, ...}
>
> Hmmm... I did answer too fast, indeed. Thanks for pointing that out.
Very very very thanks to everybody.
I'm a new user in this group and you are wonderfull persons :)
--
http://mail.python.org/mailman/listinfo/python-list
